mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
8ce931ab763541736bd08d0507b4777543532762
34
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Owen LinandGitHub
|
703793c22e |
feat(core, mcp): cache codex_apps tools in memory (#29003)
## Description This makes Codex Apps tool reads use a shared in-memory snapshot instead of rereading the disk cache every time `list_all_tools()` runs. Disk still seeds the cache on startup and gets updated after successful fetches, but it is no longer the live read path. The core change is that `McpManager` now owns a process-scoped `CodexAppsToolsCache`. Codex threads in the same app-server process now share this Codex Apps in-memory tools snapshot. The snapshot is keyed by the Codex home plus the Codex Apps identity: the active Codex auth user/workspace and the effective Codex Apps MCP source config. There's already code to hard-refresh the cache, so we respect it in this PR. ## Local benchmark I ran a local steady-state microbenchmark of the exact repeated Codex Apps cached-tools read this PR removes, using the same real local cache payload in both trees: `3,678,138` bytes and `381` tools. The cache file was already warm in the OS page cache, so this measures same-process reread/deserialization work rather than cold-disk latency or full turn latency. Each run is 25 iterations (mimicking a turn that makes 25 inference calls). | Version | Run 1 | Run 2 | Avg | |---|---:|---:|---:| | `origin/main` disk read + JSON deserialize + `filter_tools` | `50.755 ms` | `52.894 ms` | `51.825 ms` | | This branch in-memory `current_tools` + `filter_tools` | `0.740 ms` | `0.778 ms` | `0.759 ms` | That removes about `51 ms` from each repeated Codex Apps cached-tools read on this machine, roughly `68x` faster for that subpath. It is useful evidence for the hot path this PR changes, but not a claim that every production turn gets `51 ms` faster; end-to-end impact also depends on the rest of `list_all_tools()` and tool-payload construction. This is on my M2 Max macbook, so with a slower disk this would be much worse (and indeed we did see this really blew up turn runtime with a slow disk). |
||
 jifandGitHub
|
b215961a56 |
Support OAuth for HTTP MCP servers from selected executor plugins (#28529)
## Why #28522 routes selected-plugin HTTP MCP traffic through the owning executor, but OAuth bootstrap and refresh still used host-local clients. Executor-only servers therefore cannot complete discovery or login through the same network boundary as the MCP connection. ## What changed - adapt `codex_exec_server::HttpClient` to RMCP 1.8's `OAuthHttpClient` contract - let RMCP own discovery, dynamic registration, PKCE, token exchange, and refresh - route auth status, persisted-token startup, and app-server login through the server runtime while preserving the existing local discovery path - add optional `threadId` to `mcpServer/oauth/login` and echo it in the completion notification - implement RMCP's redirect policy and 1 MiB OAuth response limit over executor HTTP - cover selected-thread OAuth discovery and login through an executor-only route Depends on #28522. |
||
|
jifandGitHub
|
3e39e92f03 |
Keep executor plugin MCP paths URI-native (#29628)
## Why
Executor-owned plugin roots are `PathUri`, but MCP config normalization
still converts them into a native `Path` using the app-server host's
rules. Relative `cwd` values can therefore resolve against the wrong
filesystem when host and executor path conventions differ.
This PR keeps executor MCP paths URI-native until the selected
environment launches the server, while retaining the existing host
parser behavior.
## What changed
- Keep one shared MCP normalization path with narrow host-`Path` and
executor-`PathUri` entrypoints.
- Preserve native host resolution for locally installed plugin MCP
configs.
- For executor configs, default `cwd` to the plugin root and resolve
relative working directories with the root URI's path convention.
- Accept explicit executor `file:` URIs only when they remain within the
selected plugin root.
- Preserve the selected environment id and existing remote
environment-variable ownership rules.
- Route the executor plugin provider through the URI-native entrypoint
without converting the root on the host.
- Ensure `codex doctor` does not probe executor-owned stdio commands or
foreign working directories on the host.
- Cover foreign Windows roots, relative and absolute executor working
directories, traversal rejection, runtime resolution, and doctor
behavior.
```text
plugin root: file:///C:/plugins/demo
configured cwd: scripts
|
v
resolved cwd: file:///C:/plugins/demo/scripts
|
v
launch through the selected executor
```
No new provider or filesystem abstraction is introduced.
## Stack
1. #29614 — add lexical `PathUri` containment.
2. #29620 — share URI-native manifest path resolution.
3. #28918 — keep selected plugin roots and resources URI-native.
4. #29626 — load executor skills without host path conversion.
5. **This PR** — resolve executor MCP working directories without host
path conversion.
|
||
|
jifandGitHub
|
0afe559318 |
skills: cache orchestrator resources per thread (#28336)
## Why Hosted orchestrator skills are read through the remote MCP resource server. Within one thread, the same catalog or skill resource can be requested multiple times by prompt injection and the `skills.list` / `skills.read` tools. Re-fetching adds latency and can make those surfaces observe different remote contents during the same thread. This is a follow-up to #28333: orchestrator skills remain limited to threads without a local executor, and those threads now get a stable per-thread view of the remote skill data they use. ## What changed - Reuse the existing per-thread orchestrator catalog snapshot for `skills.list` and `skills.read` availability checks. - Cache successful orchestrator resource reads by authority, package, and resource so prompt injection and tool calls share the same contents. - Keep the cache memory-only and bounded to 100 resources and 8 MiB per thread. - Leave host and executor skill reads unchanged, and do not cache failed remote reads. ## Verification - Extended the app-server MCP resource integration test to read the same hosted skill resource twice and verify that the remote server receives one read. - The same test verifies that catalog discovery and the selected skill's main prompt are each fetched only once per thread. |
||
|
jifandGitHub
|
c3a479620f |
Add selected-plugin precedence and attribution to the MCP catalog (#27884)
## Why **In short:** this PR resolves already-discovered MCP registrations. It does not read selected plugins or discover their MCP servers. The resolved MCP catalog currently builds config and auto-discovered plugin registrations before runtime contributors are applied. A thread-selected plugin needs a distinct precedence tier in that same initial resolution pass: otherwise a disabled lower-precedence winner can leave stale name-level state behind, and the winning MCP tools cannot be attributed to the selected package reliably. This PR adds that catalog boundary before executor discovery is connected. ## What changed - Added an explicit selected-plugin registration tier between auto-discovered plugins and explicit config. - Collected selected-plugin contributions before the initial catalog build, while leaving compatibility and generic extension overlays in their existing runtime phase. - Retained the winning plugin ID and display name directly on plugin-owned catalog registrations. - Derived MCP tool provenance from the winning catalog entry instead of joining against local-only plugin summaries. - Retained the winning selected server's tool approval policy in the running connection manager, so a selected registration cannot inherit approval behavior from a losing local plugin. - Kept remembered approval session-scoped for selected plugins until there is an authority-aware persistence contract; Codex will not write approval back to an unrelated local plugin. - Preserved existing name-level disabled vetoes for discovered plugins and config, while keeping a selected package's own disabled registration scoped to that registration. - Preserved deterministic selection order and existing config, compatibility, and extension precedence. The resulting order is: ```text auto-discovered plugin < selected plugin < explicit config < compatibility registration < extension overlay ``` ## Behavior and scope This is a catalog and provenance change only. No production host contributes selected-plugin MCP registrations yet, so existing local MCP behavior remains unchanged. The stacked follow-up, #27870, installs the executor plugin provider that produces these registrations. App-server activation remains a separate final step. ## Verification Focused tests cover precedence, deterministic selected-plugin conflicts, disabled-veto behavior across catalog phases, managed requirements before selected-plugin resolution, winning-server approval policy, and attribution when local and selected packages share an ID or server name. CI owns execution of the test suite. |
||
|
jifandGitHub
|
17b9f4843e |
Extract shared plugin MCP config parsing (#27863)
## Why
We want a thread-selected plugin to eventually expose stdio MCP servers
that run on the executor owning that plugin.
The existing plugin MCP parser lived inside `core-plugins` and was
coupled to the host filesystem loader. Reusing it from an executor
provider would either duplicate MCP normalization or make the plugin
package layer own MCP runtime semantics. This PR creates the shared
MCP-owned boundary first.
In simple terms:
```text
plugin .mcp.json
|
v
shared parser in codex-mcp
|
+-- Declared placement: preserve current local-plugin behavior
|
+-- Environment placement: produce config bound to one executor
```
This builds on the authority-bound plugin descriptors from #27692. It
intentionally does not discover, register, or launch executor MCP
servers yet.
## What changed
- Moved plugin MCP file parsing and normalization from `core-plugins`
into `codex-mcp`.
- Kept support for both existing file shapes: a top-level server map and
an object containing `mcpServers`.
- Kept per-server failure isolation: one invalid server does not discard
valid siblings, while malformed top-level JSON still fails the whole
file.
- Updated the existing local plugin loader to use `Declared` placement,
preserving its current transport, OAuth, relative `cwd`, and error
behavior.
- Added `Environment` placement for the next stacked PR:
- the selected environment ID overrides anything declared by the plugin;
- missing stdio `cwd` defaults to the plugin root;
- relative `cwd` is resolved beneath the plugin root and cannot traverse
outside it;
- bare or source-less environment-variable references resolve on a
non-local executor;
- explicit orchestrator environment-variable forwarding is rejected for
executor-owned plugins.
## User impact
None in this PR. Existing local plugin MCP loading follows the same
behavior through the shared parser. The executor placement mode is not
connected to thread startup until the follow-up registration PR.
## Assumptions
- A selected capability root's environment is authoritative. A plugin
cannot redirect its stdio process to the orchestrator or another
executor.
- Relative working directories belong under the plugin package root.
Explicit absolute working directories remain valid within the owning
environment.
- For a non-local executor, unqualified environment-variable names refer
to that executor. Reading an orchestrator variable requires an explicit
contract and is rejected for now.
- Parsing only produces normalized `McpServerConfig` values. Process
startup remains owned by the existing MCP runtime and connection
manager.
## Follow-ups
1. Add the executor MCP provider and catalog registration: read the
selected plugin's MCP config through the same executor filesystem,
support stdio only, freeze the result per active thread, apply managed
policy, and resolve name collisions as discovered plugin < selected
plugin < explicit config.
2. Install that provider in app-server and add an end-to-end test
proving `thread/start.selectedCapabilityRoots` launches and calls the
MCP tool on the selected executor, preserves the frozen registration
across refresh, and does not expose it to an unselected thread.
3. After the initial executor-stdio vertical, define
resume/fork/environment-replacement semantics, executor HTTP placement,
warning delivery, common MCP tool-context bounds, and move remaining MCP
source composition above core.
## Verification
- `cargo check -p codex-mcp -p codex-core-plugins --tests`
- `just bazel-lock-check`
- Added focused parser coverage for legacy local normalization, executor
authority, working-directory handling, and environment-variable
sourcing.
|
||
|
jifandGitHub
|
4a5a676499 |
Resolve MCP server registrations through a catalog (#27634)
## Why MCP servers currently come from user config, local plugins, compatibility Apps synthesis, and host extensions. Those sources were composed by mutating a shared map, leaving registration identity, precedence, removal, and provenance implicit in assembly order. Before adding executor-owned MCPs, Codex needs one durable resolution boundary above `McpConnectionManager`. This PR introduces that boundary while preserving current server configuration, policy, and runtime behavior. Executor-scoped registrations and explicit policy layers remain follow-ups. ## What changed - Add typed `McpServerRegistration` inputs and an immutable `ResolvedMcpCatalog` in `codex-mcp`. - Retain each registration's complete `McpServerConfig`, including its environment binding, while recording its source and provenance. - Preserve the existing structural precedence between plugin, config, compatibility, and ordered extension sources. - Resolve equal-precedence actions by contribution order; provenance IDs are used only for diagnostics and cannot affect the winner. - Preserve extension removals and the existing name-scoped `enabled = false` veto. - Report same-tier conflicts with every contender and the final catalog outcome, including whether the winning action registers or removes the server. - Require MCP contributors to provide a stable diagnostic identity. - Derive materialized server maps and plugin ownership from the resolved catalog. `McpConnectionManager`, transport startup, tool calls, and resource routing continue to consume the same effective `McpServerConfig` values. ## Scope This PR does not add new MCP capabilities or change user-visible behavior. It does not add executor plugin discovery, thread-scoped registrations, dynamic refresh generations, or new user/managed policy semantics. ## Verification - Added focused catalog coverage for source precedence, complete configuration preservation, disabled vetoes, plugin ownership, contribution-order tie breaking, removal outcomes, and conflict diagnostics. - Extended hosted Apps coverage for ordered extension removal and Apps-disabled hosts with and without the hosted extension installed. - `cargo check -p codex-mcp --tests -p codex-extension-api -p codex-core` |
||
|
jifandGitHub
|
a287c5dffd |
skills: make backend plugin skills invocable without an executor (#27387)
## Why #27198 made the extension-owned `codex_apps` MCP connection the hosted plugin runtime, but its `mcp/skill` resources still bypassed the skills extension. App-server could list and read those resources through generic MCP APIs, but a thread with no selected environment did not expose them in the model's skills catalog or load their `SKILL.md` through `$skill`. Hosted skills should stay remote while using the same typed catalog, source authority, deduplication, bounded contextual catalog, and selected-skill prompt injection as host and executor skills. They should not be downloaded or exposed as ambient filesystem paths. ## What changed - Add a session-scoped `McpResourceClient` over the replaceable MCP connection manager so resource list/read calls follow startup and refresh replacements. - Add a `BackendSkillProvider` that pages `codex_apps` resources, accepts bounded and validated `mcp/skill` entries, and reads a selected skill's `SKILL.md` through the same MCP connection. - Register the remote provider in app-server and include it in the skills catalog even when a thread has no selected capability roots or executor. - Contribute hosted skill metadata through the bounded `AvailableSkillsInstructions` developer-context path, exclude remote entries from per-turn catalog injection, and classify `<skills>` messages as contextual developer content so rollback can trim and rebuild them correctly. ## Testing - Extend the app-server MCP resource integration test with `environments: []` to exercise two-page discovery, filter a non-`mcp/skill` resource, verify the escaped developer catalog entry and user-role `<skill>` fragment containing the fetched `SKILL.md`, and preserve generic MCP resource reads. - Add core event-mapping coverage that classifies `<skills>` developer messages as contextual history. |
||
|
jifandGitHub
|
9cd11e9e62 |
Use plugin-service MCP as the hosted plugin runtime (#27198)
## Stack - Base: #27191 - This PR is the third vertical and should be reviewed against `jif/external-plugins-2`, not `main`. ## Why #27191 moves the host-owned Apps MCP registration behind an extension contributor, but deliberately preserves the existing endpoint-selection feature while that contribution contract lands. App-server can therefore resolve the server through extensions, yet the hosted plugin endpoint is still selected through temporary `apps_mcp_path_override` plumbing. That is not the long-term plugin model. A plugin can bundle skills, connectors, MCP servers, and hooks, and those components do not all need the same source or execution environment. In particular, an authenticated HTTP MCP server can expose plugin capabilities directly from a backend without an executor or an orchestrator filesystem. This PR completes that hosted vertical. App-server's MCP extension now owns the aggregate hosted plugin runtime at `/ps/mcp`. Connector actions continue to arrive as MCP tools, while backend-provided skills arrive as MCP resources and use Codex's existing resource list/read paths. No second backend client, skill filesystem, or generic plugin activation framework is introduced. The backend route remains the hosted implementation. This change replaces Codex's temporary endpoint-selection mechanism, not the service behind the endpoint. ## What changed ### Hosted plugin runtime The MCP extension now contributes `codex_apps` as the hosted plugin runtime rather than as a configurable Apps endpoint: - `https://chatgpt.com` resolves to `https://chatgpt.com/backend-api/ps/mcp`; - a bare custom ChatGPT base resolves to `/api/codex/ps/mcp`; - the existing product-SKU header and ChatGPT authentication behavior are preserved; - executor availability is never consulted for this streamable HTTP transport. The same MCP connection carries both component shapes supported by the hosted endpoint: - connector actions are discovered and invoked as MCP tools; - hosted skills are enumerated and read as MCP resources through the existing `list_mcp_resources` and `read_mcp_resource` paths. This keeps component access in the subsystem that already owns the protocol instead of downloading backend skills into an orchestrator filesystem or inventing a parallel hosted-skill client. ### Explicit runtime ordering `McpManager` now resolves the reserved `codex_apps` entry in three ordered phases: 1. install the legacy Apps fallback for compatibility; 2. apply ordered extension `Set` or `Remove` overlays; 3. apply the final ChatGPT-auth gate without synthesizing the server again. This ordering is important: - an ordinary configured or plugin MCP server cannot claim the auth-bearing `codex_apps` name; - an extension-contributed hosted runtime wins over the fallback; - an extension `Remove` remains authoritative; - a host without the MCP extension retains the legacy Apps endpoint and current local-only behavior. The temporary `legacy_apps_mcp_loader_enabled` coordination flag is no longer needed. ### Remove the path override The `apps_mcp_path_override` feature and its runtime plumbing are removed, including: - the feature registry entry and structured feature config; - `Config` and `McpConfig` fields; - config schema output; - config-lock materialization; - URL override handling in `codex-mcp`. Existing boolean and structured forms still deserialize as ignored compatibility input. They are omitted from new serialized config, and config-lock comparison normalizes the removed input so older locks remain replayable. ### App-server coverage App-server MCP fixtures now serve the hosted route at `/api/codex/ps/mcp`. Existing resource-read and tool/elicitation flows therefore exercise the extension-owned endpoint rather than succeeding through the legacy fallback. The stack also adds the missing `codex_chatgpt::connectors` re-export for the manager-backed connector helper introduced in #27191. ## Compatibility - App-server installs the extension and uses `/ps/mcp` for the hosted runtime. - CLI and other hosts that do not install the extension retain the legacy Apps endpoint. - Apps disabled or non-ChatGPT authentication removes `codex_apps` from the effective runtime view. - Existing local plugins, local skills, executor-selected skills, configured MCP servers, and MCP OAuth behavior are otherwise unchanged. - Backend plugin enablement remains account/workspace state owned by the hosted endpoint; this PR does not add thread-local backend plugin selection. ## Architectural fit The stack now proves two independent runtime shapes: 1. #27184 resolves filesystem-backed skills through the executor that owns a selected root. 2. #27191 and this PR resolve a backend-hosted HTTP MCP through an extension with no executor. Together they preserve the intended separation: - selection identifies a plugin/root when explicit selection is needed; - each component's owning extension resolves its concrete access mechanism; - execution stays with the runtime required by that component; - existing skills, MCP, connector, and hook subsystems remain the downstream consumers. ## Planned follow-ups 1. **Executor stdio MCP:** selecting an executor plugin registers a manifest-declared stdio MCP server and executes it in the environment that owns the plugin. 2. **Optional backend selection:** only if CCA needs thread-local selection distinct from backend account/workspace enablement, add a concrete backend-owned capability location and surface those selected skills through the skills catalog. 3. **Connector metadata and hooks:** activate those plugin components through their existing owning subsystems, with executor hooks remaining environment-bound. 4. **Propagation and persistence:** define explicit resume, fork, subagent, refresh, and environment-removal semantics once selected roots have multiple real consumers. 5. **Local convergence:** migrate legacy local skill, MCP, connector, and hook paths behind their owning extensions one vertical at a time, then remove duplicate core managers and compatibility plumbing after parity. ## Verification Coverage in this change exercises: - extension-owned `/backend-api/ps/mcp` registration without an executor; - preservation of the legacy endpoint in hosts without the extension; - extension `Set` and `Remove` precedence over the legacy fallback; - ChatGPT-auth gating for the reserved server; - hosted MCP resource reads with and without an active thread; - connector tool invocation and MCP elicitation through the hosted route; - ignored boolean and structured forms of the removed path override; - config-lock replay compatibility for the removed feature. `cargo check -p codex-features -p codex-mcp-extension -p codex-app-server` passes. Tests and Clippy were not run locally under the current development instruction; CI provides the full validation pass. |
||
|
jifandGitHub
|
4ec3b8eeea |
Route hosted Apps MCP through extensions (#27191)
## Stack - Base: #27184 - This PR is the second vertical and should be reviewed against `jif/external-plugins-1`, not `main`. ## Why CCA is moving toward a split runtime where the orchestrator may have no filesystem or executor, but it still needs to activate remotely hosted plugin components. HTTP MCP servers are the simplest complete example: they need configuration and host authentication, but they do not need an executor process. The Apps MCP endpoint is currently synthesized by a special-purpose loader inside the MCP runtime. That works locally, but it leaves hosted MCP activation outside the extension model being established in #27184. It also makes the Apps path a poor foundation for plugins whose skills, MCP servers, connectors, and hooks may come from different sources or execute in different places. This PR moves that one behavior behind an extension-owned contribution while preserving the existing local fallback. It deliberately does not introduce a generic plugin activation framework. ## What changed ### MCP extension contribution `codex-extension-api` gains an ordered `McpServerContributor` contract. A contributor returns typed `Set` or `Remove` overlays for MCP server configuration; later contributors win for the names they own. The contract stays at the existing MCP configuration boundary. Extensions do not create a second connection manager or transport abstraction. ### Hosted Apps MCP extension A new `codex-mcp-extension` contributes the reserved `codex_apps` server from the existing Apps feature, ChatGPT base URL, path override, and product SKU configuration. When `apps_mcp_path_override` is enabled for `https://chatgpt.com`, the resulting streamable HTTP endpoint is `https://chatgpt.com/backend-api/ps/mcp`. The existing ChatGPT-auth gate remains authoritative, so this server can run in an orchestrator-only process without being exposed for API-key sessions. ### One resolved runtime view `McpManager` now distinguishes three views: - **configured:** config- and plugin-backed servers before extension overlays; - **runtime:** configured servers plus host-installed extension contributions; - **effective:** runtime servers after auth gating and compatibility built-ins. App-server installs the hosted MCP extension and uses the runtime view for thread startup, refresh, status, threadless resource reads, connector discovery, and MCP OAuth lookup. This keeps `mcpServer/oauth/login` consistent with the servers exposed by the other MCP APIs. The hosted Apps server itself continues to use existing ChatGPT host authentication rather than MCP OAuth. ## Compatibility Hosts that do not install the MCP extension retain the existing Apps MCP synthesis path. This preserves current local-only, CLI, and standalone-host behavior while app-server exercises the extension path. Disabling Apps removes the reserved `codex_apps` entry, and losing ChatGPT auth removes it from the effective runtime view. Executor availability is not consulted for this HTTP transport. ## Follow-ups The next vertical will resolve a manifest-declared stdio MCP server from an executor-selected plugin root and execute it in the environment that owns that root. Later verticals can add backend-owned skills, connector metadata, hooks, durable selection semantics, and incremental local convergence without changing the component-specific runtime boundaries introduced here. ## Verification Focused coverage was added for: - contributing the hosted Apps MCP at `/backend-api/ps/mcp` without an executor; - requiring ChatGPT auth in the effective runtime view; - removing a reserved configured Apps server when the Apps feature is disabled. `cargo check -p codex-app-server -p codex-mcp-extension -p codex-extension-api -p codex-mcp` passed. Tests and Clippy were not run locally under the current development instruction; CI provides the full validation pass. |
||
 Gabriel PealandGitHub
|
577ec03bf8 |
[codex] Support ui visibility meta for tools (#24700)
## Summary Adds support for the same ui.visibility metadata as resources [spec](https://github.com/modelcontextprotocol/ext-apps/blob/main/specification/draft/apps.mdx#resource-discovery) |
||
 starr-openaiandGitHub
|
298e5cfce1 |
Route MCP servers through explicit environments (#23583)
## Summary - route each configured MCP server through an explicit per-server `environment_id` instead of a manager-wide remote toggle - default omitted `environment_id` to `local`, resolve named ids through `EnvironmentManager`, and fail only the affected MCP server when an explicit id is unknown - keep local stdio on the existing local launcher path for now, while named-environment stdio uses the selected environment backend and requires an absolute `cwd` - allow local HTTP MCP servers to keep using the ambient HTTP client when no local `Environment` is configured; named-environment HTTP MCPs use that environment's HTTP client ## Validation - devbox Bazel build: `bazel build --bes_backend= --bes_results_url= //codex-rs/cli:codex //codex-rs/rmcp-client:test_stdio_server //codex-rs/rmcp-client:test_streamable_http_server` - devbox app-server config matrix with real `config.toml` / `environments.toml` files covering omitted local, explicit local, omitted local under remote default, explicit remote stdio, local HTTP without local env, explicit remote HTTP, local stdio without local env, unknown explicit env, and remote stdio without `cwd` |
||
|
jif-oaiandGitHub
|
32b1ae7099 |
chore: drop built-in MCPs (#22173)
Drop something that was never used |
||
 pakrym-oaiandGitHub
|
857e731478 |
[codex] Remove string-keyed MCP tool maps (#21454)
## Summary This PR removes the synthetic `HashMap<String, ToolInfo>` keys from MCP tool discovery. `McpConnectionManager::list_all_tools()` now returns normalized `Vec<ToolInfo>`, and downstream code derives identity from `ToolInfo::canonical_tool_name()`. The motivation is to keep model-visible tool identity on `ToolName`/`ToolInfo` instead of parallel string map keys, so future namespace changes do not have to preserve otherwise-unused lookup keys. ## Changes - Rename the MCP normalization path from `qualify_tools` to `normalize_tools_for_model` and return tool values directly. - Flow MCP tool lists through connectors, plugin injection, router/spec building, code mode, and tool search as vectors/slices. - Keep direct/deferred subtraction local to `mcp_tool_exposure`, using `ToolName` values. - Update tests to compare `ToolName` instances where MCP identity matters. ## Validation - `cargo test -p codex-mcp test_normalize_tools` - `cargo test -p codex-core mcp_tool_exposure` - `cargo test -p codex-core direct_mcp_tools_register_namespaced_handlers` - `cargo test -p codex-core search_tool_registers_namespaced_mcp_tool_aliases` - `just fix -p codex-mcp` - `just fix -p codex-core` |
||
|
b2268999fe |
feat: make built-in MCPs first-class runtime servers (#21356)
## DISCLAIMER This is experimental and no production service must rely on this ## Why Built-in MCPs are product-owned runtime capabilities, but they were previously flattened into the same config-backed stdio path as user-configured servers. That made them depend on a hidden `codex builtin-mcp` re-exec path, exposed them through config-oriented CLI flows, and erased distinctions the runtime needs to preserve—most notably whether an MCP call should count as external context for memory-mode pollution. ## What changed - Model product-owned built-ins separately from config-backed MCP servers via `BuiltinMcpServer` and `EffectiveMcpServer`. - Launch built-ins in process through a reusable async transport instead of the hidden `builtin-mcp` stdio subcommand. - Keep config-oriented CLI operations such as `codex mcp list/get/login/logout` scoped to configured servers, while merging built-ins only into the effective runtime server set. - Retain server metadata after launch so parallel-tool support and context classification come from the live server set; built-in `memories` is now classified as local Codex state rather than external context. ## Test plan - `cargo test -p codex-mcp` - `cargo test -p codex-core --test suite builtin_memories_mcp_call_does_not_mark_thread_memory_mode_polluted_when_configured` --------- Co-authored-by: Codex <noreply@openai.com> |
||
 Clark DuVallandGitHub
|
346070a424 |
Route opted-in MCP elicitations through Guardian (#19431)
# Motivation Browser Use origin-access prompts are MCP elicitations, not direct tool-call approval prompts, so they were bypassing the Guardian approval path. We need a generic opt-in that lets eligible MCP elicitations use Guardian when the current turn already routes approvals there. # Description Add a generic elicitation reviewer hook in codex-mcp and wire codex-core to pass a Guardian reviewer callback when creating the MCP connection manager. The reviewer validates explicit mcp_tool_call opt-in metadata, builds a Guardian MCP tool-call review request from server/tool/connector metadata and tool params, and maps Guardian approval, denial, timeout, and cancellation decisions back to MCP elicitation responses. The new option to trigger this in the `_meta` object is: ``` "codex_request_type": "approval_request", ``` # Testing - RUST_MIN_STACK=8388608 NEXTEST_STATUS_LEVEL=leak cargo nextest run --no-fail-fast --cargo-profile ci-test --test-threads 2 - cargo clippy --tests -- -D warnings - cargo fmt -- --config imports_granularity=Item --check - cargo shear - pnpm run format - python3 .github/scripts/verify_cargo_workspace_manifests.py - python3 .github/scripts/verify_tui_core_boundary.py - python3 .github/scripts/verify_bazel_clippy_lints.py - git diff --check |
||
|
pakrym-oaiandGitHub
|
712305be47 |
Remove core MCP list tools op (#21281)
## Why The core `Op::ListMcpTools` request path is no longer needed. Keeping it around left a dead request/response surface alongside the app-server MCP inventory APIs that own current server status listing. ## What Changed - Removed `Op::ListMcpTools`, `EventMsg::McpListToolsResponse`, and the core handler that built the MCP snapshot response. - Removed the now-unused `codex-mcp` snapshot wrapper/export and passive event handling arms in rollout and MCP-server consumers. - Updated tests that used the old op as a synchronization hook to wait on existing startup/skills events, and deleted the plugin test that only exercised the removed listing op. ## Validation - `cargo test -p codex-protocol` - `cargo test -p codex-mcp` - `cargo test -p codex-rollout -p codex-rollout-trace -p codex-mcp-server` - `cargo test -p codex-core --test all pending_input::queued_inter_agent_mail` - `cargo test -p codex-core --test all rmcp_client::stdio_mcp_tool_call_includes_sandbox_state_meta` - `cargo test -p codex-core --test all rmcp_client::stdio_image_responses` - `just fix -p codex-core -p codex-protocol -p codex-mcp -p codex-rollout -p codex-rollout-trace -p codex-mcp-server` |
||
|
|
ca257b6ce5 |
chore: spawn MCP for memories (#21214)
Co-authored-by: Codex <noreply@openai.com> |
||
 Matthew ZengandGitHub
|
f9a907aebe |
Support Codex Apps auth elicitations (#19193)
## Summary - request URL-mode MCP elicitations when Codex Apps tool calls fail with connector auth metadata - route Codex Apps auth URL elicitations into the TUI app-link flow ## Test plan - `just fmt` - `cargo test -p codex-core mcp_tool_call::tests` - `cargo test -p codex-mcp` - `cargo test -p codex-tui bottom_pane::app_link_view::tests` - `just fix -p codex-core` - `just fix -p codex-mcp` - `just fix -p codex-tui` Also attempted broader local runs: - `cargo test -p codex-core` fails in unrelated config/request-permission/proxy-sensitive tests under the current Codex Desktop environment. - `cargo test -p codex-tui` fails in unrelated status snapshots/trust-default tests because the ambient environment renders workspace-write/network permission defaults. |
||
 maja-openaiandGitHub
|
a5ebedef67 |
Bypass review for always-allow MCP tools in auto-review (#20069)
## Why When an MCP or app tool is configured with approval mode `approve` (always allow), users expect that decision to be authoritative. In guardian auto-review mode, ARC could still return `ask-user`, which then routed the approval question into guardian with the ARC reason as context. That meant a tool explicitly configured as always allowed still went through both safety monitors before running. This change keeps the existing ARC behavior for non-auto-review sessions, but avoids the ARC-to-guardian sequence when `approvals_reviewer = auto_review` and the tool approval mode is `approve`. ## What changed - Short-circuit MCP tool approval handling when `approval_mode == approve` and `approvals_reviewer == auto_review`. - Updated the MCP approval regression test so the auto-review case asserts neither ARC nor guardian is called. - Preserved existing tests that verify ARC can still block always-allow MCP tools outside guardian auto-review mode. ## Verification - `cargo test -p codex-core --lib mcp_tool_call` |
||
|
0bda8161a2 |
Split MCP connection modules (#19725)
## Why The MCP connection manager module had grown to mix orchestration, RMCP client startup, elicitation handling, Codex Apps cache and naming behavior, tool qualification and filtering, and runtime data. The previous stacked PRs split these responsibilities incrementally; this PR collapses that work into one self-contained refactor on latest main. ## What changed - Move McpConnectionManager into connection_manager.rs. - Move RMCP client lifecycle, startup, and uncached tool listing into rmcp_client.rs. - Move elicitation request tracking and policy handling into elicitation.rs. - Move Codex Apps cache, key, filtering, and naming helpers into codex_apps.rs. - Rename the tool-name helper module to tools.rs and move ToolInfo, tool filtering, schema masking, and qualification there. - Move runtime and sandbox shared types into runtime.rs. - Preserve latest main PermissionProfile-based MCP elicitation auto-approval behavior. ## Verification - just fmt - cargo check -p codex-mcp - cargo check -p codex-mcp --tests - cargo check -p codex-core --------- Co-authored-by: Codex <noreply@openai.com> |
||
|
Ahmed IbrahimandGitHub
|
022f81df1f |
[codex] Order codex-mcp items by visibility (#19526)
## Why The visibility cleanup in the base PR reduced what `codex-mcp` exposes, but several files still made reviewers read private support machinery before the public or crate-facing entry points. This ordering pass makes each file easier to scan: exported API first, crate-visible MCP internals next, then private helpers in breadth-first order from the higher-level MCP flows to leaf utilities. ## What Changed - Reordered `codex-mcp` exports so the runtime, configuration, snapshot, auth, and helper surfaces are grouped by visibility and reader importance. - Moved public and crate-visible MCP items ahead of private helpers in the auth, MCP planning/snapshot, connection manager, and tool-name modules. - Kept the change mechanical, with no behavior changes intended. ## Verification - `cargo check -p codex-mcp` |
||
|
Ahmed IbrahimandGitHub
|
706490ab1b |
[codex] Prune unused codex-mcp API and duplicate helpers (#19524)
## Why `codex-mcp` currently exposes more API than the rest of the workspace uses. Some of that surface is simply visibility that can be tightened, and some of it is public helper code that remains compiler-valid because it is exported even though no workspace caller uses it. That distinction matters: Rust does not warn on exported API just because the current workspace does not call it. This PR intentionally treats those exported-but-workspace-unreferenced paths as stale `codex-mcp` surface. The main example is MCP skill dependency collection, where the active implementation now lives in `codex-rs/core/src/mcp_skill_dependencies.rs`; keeping the older `codex-mcp` copy makes it unclear which implementation owns skill MCP installation. ## What Changed - Pruned unused `codex-mcp` re-exports from `codex-mcp/src/lib.rs`. - Removed non-runtime helper methods from `McpConnectionManager` so it stays focused on live MCP clients. - Made `ToolPluginProvenance` lookup methods crate-private. - Removed workspace-unreferenced snapshot wrapper APIs and qualified-tool grouping helpers. - Deleted the duplicate `codex-mcp` skill dependency module and tests now that skill MCP dependency handling is owned by `core`. ## Verification - `cargo check -p codex-mcp` |
||
 efrazer-oaiandGitHub
|
be75785504 |
fix: fully revert agent identity runtime wiring (#18757)
## Summary This PR fully reverts the previously merged Agent Identity runtime integration from the old stack: https://github.com/openai/codex/pull/17387/changes It removes the Codex-side task lifecycle wiring, rollout/session persistence, feature flag plumbing, lazy `auth.json` mutation, background task auth paths, and request callsite changes introduced by that stack. This leaves the repo in a clean pre-AgentIdentity integration state so the follow-up PRs can reintroduce the pieces in smaller reviewable layers. ## Stack 1. This PR: full revert 2. https://github.com/openai/codex/pull/18871: move Agent Identity business logic into a crate 3. https://github.com/openai/codex/pull/18785: add explicit AgentIdentity auth mode and startup task allocation 4. https://github.com/openai/codex/pull/18811: migrate auth callsites through AuthProvider ## Testing Tests: targeted Rust checks, cargo-shear, Bazel lock check, and CI. |
||
|
Matthew ZengandGitHub
|
1132ef887c |
Make MCP resource read threadless (#18292)
## Summary Making thread id optional so that we can better cache resources for MCPs for connectors since their resource templates is universal and not particular to projects. - Make `mcpServer/resource/read` accept an optional `threadId` - Read resources from the current MCP config when no thread is supplied - Keep the existing thread-scoped path when `threadId` is present - Update the generated schemas, README, and integration coverage ## Testing - `just write-app-server-schema` - `just fmt` - `cargo test -p codex-app-server-protocol` - `cargo test -p codex-mcp` - `cargo test -p codex-app-server --test all mcp_resource` - `just fix -p codex-mcp` - `just fix -p codex-app-server-protocol` - `just fix -p codex-app-server` |
||
 AdrianandGitHub
|
904c751a40 |
[codex] Use background agent task auth for backend calls (#18094)
## Summary Introduces a single background/control-plane agent task for ChatGPT backend requests that do not have a thread-scoped task, with `AuthManager` owning the default ChatGPT backend authorization decision. Callers now ask `AuthManager` for the default ChatGPT backend authorization header. `AuthManager` decides whether that is bearer or background AgentAssertion based on config/internal state, while low-level bootstrap paths can explicitly request bearer-only auth. This PR is stacked on PR4 and focuses on the shared background task auth plumbing plus the first tranche of backend/control-plane consumers. The remaining callsite wiring is split into PR4.2 to keep review size down. ## Stack - PR1: https://github.com/openai/codex/pull/17385 - add `features.use_agent_identity` - PR2: https://github.com/openai/codex/pull/17386 - register agent identities when enabled - PR3: https://github.com/openai/codex/pull/17387 - register agent tasks when enabled - PR3.1: https://github.com/openai/codex/pull/17978 - persist and prewarm registered tasks per thread - PR4: https://github.com/openai/codex/pull/17980 - use task-scoped `AgentAssertion` for downstream calls - PR4.1: this PR - introduce AuthManager-owned background/control-plane `AgentAssertion` auth - PR4.2: https://github.com/openai/codex/pull/18260 - use background task auth for additional backend/control-plane calls ## What Changed - add background task registration and assertion minting inside `codex-login` - persist `agent_identity.background_task_id` separately from per-session task state - make `BackgroundAgentTaskManager` private to `codex-login`; call sites do not instantiate or pass it around - teach `AuthManager` the ChatGPT backend base URL and feature-derived background auth mode from resolved config - expose bearer-only helpers for bootstrap/registration/refresh-style paths that must not use AgentAssertion - wire `AuthManager` default ChatGPT authorization through app listing, connector directory listing, remote plugins, MCP status/listing, analytics, and core-skills remote calls - preserve bearer fallback when the feature is disabled, the backend host is unsupported, or background task registration is not available ## Validation - `just fmt` - `cargo check -p codex-core -p codex-login -p codex-analytics -p codex-app-server -p codex-cloud-requirements -p codex-cloud-tasks -p codex-models-manager -p codex-chatgpt -p codex-model-provider -p codex-mcp -p codex-core-skills` - `cargo test -p codex-login agent_identity` - `cargo test -p codex-model-provider bearer_auth_provider` - `cargo test -p codex-core agent_assertion` - `cargo test -p codex-app-server remote_control` - `cargo test -p codex-cloud-requirements fetch_cloud_requirements` - `cargo test -p codex-models-manager manager::tests` - `cargo test -p codex-chatgpt` - `cargo test -p codex-cloud-tasks` - `just fix -p codex-core -p codex-login -p codex-analytics -p codex-app-server -p codex-cloud-requirements -p codex-cloud-tasks -p codex-models-manager -p codex-chatgpt -p codex-model-provider -p codex-mcp -p codex-core-skills` - `just fix -p codex-app-server` - `git diff --check` |
||
|
|
996aa23e4c |
[5/6] Wire executor-backed MCP stdio (#18212)
## Summary - Add the executor-backed RMCP stdio transport. - Wire MCP stdio placement through the executor environment config. - Cover local and executor-backed stdio paths with the existing MCP test helpers. ## Stack ```text o #18027 [6/6] Fail exec client operations after disconnect │ @ #18212 [5/6] Wire executor-backed MCP stdio │ o #18087 [4/6] Abstract MCP stdio server launching │ o #18020 [3/6] Add pushed exec process events │ o #18086 [2/6] Support piped stdin in exec process API │ o #18085 [1/6] Add MCP server environment config │ o main ``` --------- Co-authored-by: Codex <noreply@openai.com> |
||
 Michael BolinandGitHub
|
66533ddc61 |
mcp: remove codex/sandbox-state custom request support (#17957)
## Why #17763 moved sandbox-state delivery for MCP tool calls to request `_meta` via the `codex/sandbox-state-meta` experimental capability. Keeping the older `codex/sandbox-state` capability meant Codex still maintained a second transport that pushed updates with the custom `codex/sandbox-state/update` request at server startup and when the session sandbox policy changed. That duplicate MCP path is redundant with the per-tool-call metadata path and makes the sandbox-state contract larger than needed. The existing managed network proxy refresh on sandbox-policy changes is still needed, so this keeps that behavior separate from the removed MCP notification. ## What Changed - Removed the exported `MCP_SANDBOX_STATE_CAPABILITY` and `MCP_SANDBOX_STATE_METHOD` constants. - Removed detection of `codex/sandbox-state` during MCP initialization and stopped sending `codex/sandbox-state/update` at server startup. - Removed the `McpConnectionManager::notify_sandbox_state_change` plumbing while preserving the managed network proxy refresh when a user turn changes sandbox policy. - Slimmed `McpConnectionManager::new` so startup paths pass only the initial `SandboxPolicy` needed for MCP elicitation state. - Kept `codex/sandbox-state-meta` support intact; servers that opt in still receive the current `SandboxState` on tool-call request `_meta` ([remaining call path](https://github.com/openai/codex/blob/ff2d3c1e72ff08ce13743b99605d19d338edd51c/codex-rs/core/src/mcp_tool_call.rs#L487-L526)). - Added regression coverage for refreshing the live managed network proxy on a per-turn sandbox-policy change. ## Verification - `cargo test -p codex-core new_turn_refreshes_managed_network_proxy_for_sandbox_change` - `cargo test -p codex-mcp` |
||
 aaronl-openaiandGitHub
|
42528a905d |
Send sandbox state through MCP tool metadata (#17763)
## Changes Allows MCPs to opt in to receiving sandbox config info through `_meta` on model-initiated tool calls. This lets MCPs adhere to the thread's sandbox if they choose to. ## Details - Adds the `codex/sandbox-state-meta` experimental MCP capability. - Tracks whether each MCP server advertises that capability. - When a server opts in, `codex-core` injects the current `SandboxState` into model-initiated MCP tool-call request `_meta`. ## Verification - added an integration test for the capability |
||
|
Matthew ZengandGitHub
|
d7f99b0fa6 |
[mcp] Expand tool search to custom MCPs. (#16944)
- [x] Expand tool search to custom MCPs.
- [x] Rename several variables/fields to be more generic.
Updated tool & server name lifecycles:
**Raw Identity**
ToolInfo.server_name is raw MCP server name.
ToolInfo.tool.name is raw MCP tool name.
MCP calls route back to raw via parse_tool_name() returning
(tool.server_name, tool.tool.name).
mcpServerStatus/list now groups by raw server and keys tools by
Tool.name: mod.rs:599
App-server just forwards that grouped raw snapshot:
codex_message_processor.rs:5245
**Callable Names**
On list-tools, we create provisional callable_namespace / callable_name:
mcp_connection_manager.rs:1556
For non-app MCP, provisional callable name starts as raw tool name.
For codex-apps, provisional callable name is sanitized and strips
connector name/id prefix; namespace includes connector name.
Then qualify_tools() sanitizes callable namespace + name to ASCII alnum
/ _ only: mcp_tool_names.rs:128
Note: this is stricter than Responses API. Hyphen is currently replaced
with _ for code-mode compatibility.
**Collision Handling**
We do initially collapse example-server and example_server to the same
base.
Then qualify_tools() detects distinct raw namespace identities behind
the same sanitized namespace and appends a hash to the callable
namespace: mcp_tool_names.rs:137
Same idea for tool-name collisions: hash suffix goes on callable tool
name.
Final list_all_tools() map key is callable_namespace + callable_name:
mcp_connection_manager.rs:769
**Direct Model Tools**
Direct MCP tool declarations use the full qualified sanitized key as the
Responses function name.
The raw rmcp Tool is converted but renamed for model exposure.
**Tool Search / Deferred**
Tool search result namespace = final ToolInfo.callable_namespace:
tool_search.rs:85
Tool search result nested name = final ToolInfo.callable_name:
tool_search.rs:86
Deferred tool handler is registered as "{namespace}:{name}":
tool_registry_plan.rs:248
When a function call comes back, core recombines namespace + name, looks
up the full qualified key, and gets the raw server/tool for MCP
execution: codex.rs:4353
**Separate Legacy Snapshot**
collect_mcp_snapshot_from_manager_with_detail() still returns a map
keyed by qualified callable name.
mcpServerStatus/list no longer uses that; it uses
McpServerStatusSnapshot, which is raw-inventory shaped.
|
||
|
244b15c95d |
feat: add Codex Apps sediment file remapping (#15197)
## Summary - bridge Codex Apps tools that declare `_meta["openai/fileParams"]` through the OpenAI file upload flow - mask those file params in model-visible tool schemas so the model provides absolute local file paths instead of raw file payload objects - rewrite those local file path arguments client-side into `ProvidedFilePayload`-shaped objects before the normal MCP tool call ## Details - applies to scalar and array file params declared in `openai/fileParams` - Codex uploads local files directly to the backend and uses the uploaded file metadata to build the MCP tool arguments locally - this PR is input-only ## Verification - `just fmt` - `cargo test -p codex-core mcp_tool_call -- --nocapture` --------- Co-authored-by: Codex <noreply@openai.com> |
||
|
01537f0bd2 |
Auto-approve MCP server elicitations in Full Access mode (#17164)
Currently, when a MCP server sends an elicitation to Codex running in Full Access (`sandbox_policy: DangerFullAccess` + `approval_policy: Never`), the elicitations are auto-cancelled. This PR updates the automatic handling of MCP elicitations to be consistent with other approvals in full-access, where they are auto-approved. Because MCP elicitations may actually require user input, this mechanism is limited to empty form elicitations. ## Changeset - Add policy helper shared with existing MCP tool call approval auto-approve - Update `ElicitationRequestManager` to auto-approve elicitations in full access when `can_auto_accept_elicitation` is true. - Add tests Co-authored-by: Codex <noreply@openai.com> |
||
|
pakrym-oaiandGitHub
|
413c1e1fdf |
[codex] reduce module visibility (#16978)
## Summary - reduce public module visibility across Rust crates, preferring private or crate-private modules with explicit crate-root public exports - update external call sites and tests to use the intended public crate APIs instead of reaching through module trees - add the module visibility guideline to AGENTS.md ## Validation - `cargo check --workspace --all-targets --message-format=short` passed before the final fix/format pass - `just fix` completed successfully - `just fmt` completed successfully - `git diff --check` passed |
||
|
Ahmed IbrahimandGitHub
|
59b68f5519 |
Extract MCP into codex-mcp crate (#15919)
- Split MCP runtime/server code out of `codex-core` into the new `codex-mcp` crate. New/moved public structs/types include `McpConfig`, `McpConnectionManager`, `ToolInfo`, `ToolPluginProvenance`, `CodexAppsToolsCacheKey`, and the `McpManager` API (`codex_mcp::mcp::McpManager` plus the `codex_core::mcp::McpManager` wrapper/shim). New/moved functions include `with_codex_apps_mcp`, `configured_mcp_servers`, `effective_mcp_servers`, `collect_mcp_snapshot`, `collect_mcp_snapshot_from_manager`, `qualified_mcp_tool_name_prefix`, and the MCP auth/skill-dependency helpers. Why: this creates a focused MCP crate boundary and shrinks `codex-core` without forcing every consumer to migrate in the same PR. - Move MCP server config schema and persistence into `codex-config`. New/moved structs/enums include `AppToolApproval`, `McpServerToolConfig`, `McpServerConfig`, `RawMcpServerConfig`, `McpServerTransportConfig`, `McpServerDisabledReason`, and `codex_config::ConfigEditsBuilder`. New/moved functions include `load_global_mcp_servers` and `ConfigEditsBuilder::replace_mcp_servers`/`apply`. Why: MCP TOML parsing/editing is config ownership, and this keeps config validation/round-tripping (including per-tool approval overrides and inline bearer-token rejection) in the config crate instead of `codex-core`. - Rewire `codex-core`, app-server, and plugin call sites onto the new crates. Updated `Config::to_mcp_config(&self, plugins_manager)`, `codex-rs/core/src/mcp.rs`, `codex-rs/core/src/connectors.rs`, `codex-rs/core/src/codex.rs`, `CodexMessageProcessor::list_mcp_server_status_task`, and `utils/plugins/src/mcp_connector.rs` to build/pass the new MCP config/runtime types. Why: plugin-provided MCP servers still merge with user-configured servers, and runtime auth (`CodexAuth`) is threaded into `with_codex_apps_mcp` / `collect_mcp_snapshot` explicitly so `McpConfig` stays config-only. |