Commit Graph

17 Commits

  • Process-group cleanup for stdio MCP servers to prevent orphan process storms (#10710)
    This PR changes stdio MCP child processes to run in their own process
    group
    * Add guarded teardown in codex-rmcp-client: send SIGTERM to the group
    first, then SIGKILL after a short grace period.
    * Add terminate_process_group helper in process_group.rs.
    * Add Unix regression test in process_group_cleanup.rs to verify wrapper
    + grandchild are reaped on client drop.
    
    Addresses reported MCP process/thread storm: #10581
  • feat: replace custom mcp-types crate with equivalents from rmcp (#10349)
    We started working with MCP in Codex before
    https://crates.io/crates/rmcp was mature, so we had our own crate for
    MCP types that was generated from the MCP schema:
    
    
    https://github.com/openai/codex/blob/8b95d3e082376f4cb23e92641705a22afb28a9da/codex-rs/mcp-types/README.md
    
    Now that `rmcp` is more mature, it makes more sense to use their MCP
    types in Rust, as they handle details (like the `_meta` field) that our
    custom version ignored. Though one advantage that our custom types had
    is that our generated types implemented `JsonSchema` and `ts_rs::TS`,
    whereas the types in `rmcp` do not. As such, part of the work of this PR
    is leveraging the adapters between `rmcp` types and the serializable
    types that are API for us (app server and MCP) introduced in #10356.
    
    Note this PR results in a number of changes to
    `codex-rs/app-server-protocol/schema`, which merit special attention
    during review. We must ensure that these changes are still
    backwards-compatible, which is possible because we have:
    
    ```diff
    - export type CallToolResult = { content: Array<ContentBlock>, isError?: boolean, structuredContent?: JsonValue, };
    + export type CallToolResult = { content: Array<JsonValue>, structuredContent?: JsonValue, isError?: boolean, _meta?: JsonValue, };
    ```
    
    so `ContentBlock` has been replaced with the more general `JsonValue`.
    Note that `ContentBlock` was defined as:
    
    ```typescript
    export type ContentBlock = TextContent | ImageContent | AudioContent | ResourceLink | EmbeddedResource;
    ```
    
    so the deletion of those individual variants should not be a cause of
    great concern.
    
    Similarly, we have the following change in
    `codex-rs/app-server-protocol/schema/typescript/Tool.ts`:
    
    ```
    - export type Tool = { annotations?: ToolAnnotations, description?: string, inputSchema: ToolInputSchema, name: string, outputSchema?: ToolOutputSchema, title?: string, };
    + export type Tool = { name: string, title?: string, description?: string, inputSchema: JsonValue, outputSchema?: JsonValue, annotations?: JsonValue, icons?: Array<JsonValue>, _meta?: JsonValue, };
    ```
    
    so:
    
    - `annotations?: ToolAnnotations` ➡️ `JsonValue`
    - `inputSchema: ToolInputSchema` ➡️ `JsonValue`
    - `outputSchema?: ToolOutputSchema` ➡️ `JsonValue`
    
    and two new fields: `icons?: Array<JsonValue>, _meta?: JsonValue`
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/10349).
    * #10357
    * __->__ #10349
    * #10356
  • Validate CODEX_HOME before resolving (#10249)
    Summary
    - require `CODEX_HOME` to point to an existing directory before
    canonicalizing and surface clear errors otherwise
    - share the same helper logic in both `core` and `rmcp-client` and add
    unit tests that cover missing, non-directory, valid, and default paths
    
    This addresses #9222
  • feat: support proxy for ws connection (#9409)
    unfortunately tokio-tungstenite doesn't support proxy configuration
    outbox, while https://github.com/snapview/tokio-tungstenite/pull/370 is
    in review, we can depend on source code for now.
  • add generated jsonschema for config.toml (#8956)
    ### What
    Add JSON Schema generation for `config.toml`, with checked‑in
    `docs/config.schema.json`. We can move the schema elsewhere if preferred
    (and host it if there's demand).
    
    Add fixture test to prevent drift and `just write-config-schema` to
    regenerate on schema changes.
    
    Generate MCP config schema from `RawMcpServerConfig` instead of
    `McpServerConfig` because that is the runtime type used for
    deserialization.
    
    Populate feature flag values into generated schema so they can be
    autocompleted.
    
    ### Tests
    Added tests + regenerate script to prevent drift. Tested autocompletions
    using generated jsonschema locally with Even Better TOML.
    
    
    
    https://github.com/user-attachments/assets/5aa7cd39-520c-4a63-96fb-63798183d0bc
  • feat: introduce codex-utils-cargo-bin as an alternative to assert_cmd::Command (#8496)
    This PR introduces a `codex-utils-cargo-bin` utility crate that
    wraps/replaces our use of `assert_cmd::Command` and
    `escargot::CargoBuild`.
    
    As you can infer from the introduction of `buck_project_root()` in this
    PR, I am attempting to make it possible to build Codex under
    [Buck2](https://buck2.build) as well as `cargo`. With Buck2, I hope to
    achieve faster incremental local builds (largely due to Buck2's
    [dice](https://buck2.build/docs/insights_and_knowledge/modern_dice/)
    build strategy, as well as benefits from its local build daemon) as well
    as faster CI builds if we invest in remote execution and caching.
    
    See
    https://buck2.build/docs/getting_started/what_is_buck2/#why-use-buck2-key-advantages
    for more details about the performance advantages of Buck2.
    
    Buck2 enforces stronger requirements in terms of build and test
    isolation. It discourages assumptions about absolute paths (which is key
    to enabling remote execution). Because the `CARGO_BIN_EXE_*` environment
    variables that Cargo provides are absolute paths (which
    `assert_cmd::Command` reads), this is a problem for Buck2, which is why
    we need this `codex-utils-cargo-bin` utility.
    
    My WIP-Buck2 setup sets the `CARGO_BIN_EXE_*` environment variables
    passed to a `rust_test()` build rule as relative paths.
    `codex-utils-cargo-bin` will resolve these values to absolute paths,
    when necessary.
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/8496).
    * #8498
    * __->__ #8496
  • chore: add cargo-deny configuration (#7119)
    - add GitHub workflow running cargo-deny on push/PR
    - document cargo-deny allowlist with workspace-dep notes and advisory
    ignores
    - align workspace crates to inherit version/edition/license for
    consistent checks
  • Fix FreeBSD/OpenBSD builds: target-specific keyring features and BSD hardening (#6680)
    ## Summary
    Builds on FreeBSD and OpenBSD were failing due to globally enabled
    Linux-specific keyring features and hardening code paths not gated by
    OS. This PR scopes keyring native backends to the
    appropriate targets, disables default features at the workspace root,
    and adds a BSD-specific hardening function. Linux/macOS/Windows behavior
    remains unchanged, while FreeBSD/OpenBSD
      now build and run with a supported backend.
    
    ## Key Changes
    
      - Keyring features:
    - Disable keyring default features at the workspace root to avoid
    pulling Linux backends on non-Linux.
    - Move native backend features into target-specific sections in the
    affected crates:
              - Linux: linux-native-async-persistent
              - macOS: apple-native
              - Windows: windows-native
              - FreeBSD/OpenBSD: sync-secret-service
      - Process hardening:
          - Add pre_main_hardening_bsd() for FreeBSD/OpenBSD, applying:
              - Set RLIMIT_CORE to 0
              - Clear LD_* environment variables
    - Simplify process-hardening Cargo deps to unconditional libc (avoid
    conflicting OS fragments).
      - No changes to CODEX_SANDBOX_* behavior.
    
    ## Rationale
    
    - Previously, enabling keyring native backends globally pulled
    Linux-only features on BSD, causing build errors.
    - Hardening logic was tailored for Linux/macOS; BSD builds lacked a
    gated path with equivalent safeguards.
    - Target-scoped features and BSD hardening make the crates portable
    across these OSes without affecting existing behavior elsewhere.
    
    ## Impact by Platform
    
      - Linux: No functional change; backends now selected via target cfg.
      - macOS: No functional change; explicit apple-native mapping.
      - Windows: No functional change; explicit windows-native mapping.
    - FreeBSD/OpenBSD: Builds succeed using sync-secret-service; BSD
    hardening applied during startup.
    
    ## Testing
    
    - Verified compilation across affected crates with target-specific
    features.
    - Smoke-checked that Linux/macOS/Windows feature sets remain identical
    functionally after scoping.
    - On BSD, confirmed keyring resolves to sync-secret-service and
    hardening compiles.
    
    ## Risks / Compatibility
    
      - Minimal risk: only feature scoping and OS-gated additions.
    - No public API changes in the crates; runtime behavior on non-BSD
    platforms is preserved.
    - On BSD, the new hardening clears LD_*; this is consistent with
    security posture on other Unix platforms.
    
    ## Reviewer Notes
    
    - Pay attention to target-specific sections for keyring in the affected
    Cargo.toml files.
    - Confirm pre_main_hardening_bsd() mirrors the safe subset of
    Linux/macOS hardening without introducing Linux-only calls.
    - Confirm no references to CODEX_SANDBOX_ENV_VAR or
    CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR were added/modified.
    
    ## Checklist
    
      - Disable keyring default features at workspace root.
    - Target-specific keyring features mapped per OS
    (Linux/macOS/Windows/BSD).
      - Add BSD hardening (RLIMIT_CORE=0, clear LD_*).
      - Simplify process-hardening dependencies to unconditional libc.
      - No changes to sandbox env var code.
      - Formatting and linting: just fmt + just fix -p for changed crates.
      - Project tests pass for changed crates; broader suite unchanged.
    
    ---------
    
    Co-authored-by: celia-oai <celia@openai.com>
  • fix: resolve Windows MCP server execution for script-based tools (#3828)
    ## What?
    
    Fixes MCP server initialization failures on Windows when using
    script-based tools like `npx`, `pnpm`, and `yarn` that rely on
    `.cmd`/`.bat` files rather than `.exe` binaries.
    
    Fixes #2945
    
    ## Why?
    
    Windows users encounter "program not found" errors when configuring MCP
    servers with commands like `npx` in their `~/.codex/config.toml`. This
    happens because:
    
    - Tools like `npx` are batch scripts (`npx.cmd`) on Windows, not
    executable binaries
    - Rust's `std::process::Command` bypasses the shell and cannot execute
    these scripts directly
    - The Windows shell normally handles this by checking `PATHEXT` for
    executable extensions
    
    Without this fix, Windows users must specify full paths or add `.cmd`
    extensions manually, which breaks cross-platform compatibility.
    
    ## How?
    
    Added platform-specific program resolution using the `which` crate to
    find the correct executable path:
    
    - **Windows**: Resolves programs through PATH/PATHEXT to find
    `.cmd`/`.bat` scripts
    - **Unix**: Returns the program unchanged (no-op, as Unix handles
    scripts natively)
    
    ### Changes
    
    - Added `which = "6"` dependency to `mcp-client/Cargo.toml`
    - Implemented `program_resolver` module in `mcp_client.rs` with
    platform-specific resolution
    - Added comprehensive tests for both Windows and Unix behavior
    
    ### Testing
    
    Added platform-specific tests to verify:
    - Unix systems execute scripts without extensions
    - Windows fails without proper extensions
    - Windows succeeds with explicit extensions
    - Cross-platform resolution enables successful execution
    
    **Tested on:**
    - Windows 11 (NT 10.0.26100.0 x64)
    - PowerShell 5.1 & 7+, CMD, Git Bash
    - MCP servers: playwright, context7, supabase
    - WSL (verified no regression)
    
    **Local checks passed:**
    ```bash
    cargo test && cargo clippy --tests && cargo fmt -- --config imports_granularity=Item
    ```
    
    ### Results
    
    **Before:**
    ```
    🖐 MCP client for `playwright` failed to start: program not found
    ```
    
    **After:**
    ```
    🖐 MCP client for `playwright` failed to start: request timed out
    ```
    
    Windows users can now use simple commands like `npx` in their config
    without specifying full paths or extensions. The timeout issue is a
    separate concern that will be addressed in a follow-up PR.
    
    ---------
    
    Co-authored-by: Eric Traut <etraut@openai.com>
  • [Auth] Add keyring support for Codex CLI (#5591)
    Follow-up PR to #5569. Add Keyring Support for Auth Storage in Codex CLI
    as well as a hybrid mode (default to persisting in keychain but fall
    back to file when unavailable.)
    
    It also refactors out the keyringstore implementation from rmcp-client
    [here](https://github.com/openai/codex/blob/main/codex-rs/rmcp-client/src/oauth.rs)
    to a new keyring-store crate.
    
    There will be a follow-up that picks the right credential mode depending
    on the config, instead of hardcoding `AuthCredentialsStoreMode::File`.
  • [MCP] Add support for resources (#5239)
    This PR adds support for [MCP
    resources](https://modelcontextprotocol.io/specification/2025-06-18/server/resources)
    by adding three new tools for the model:
    1. `list_resources`
    2. `list_resource_templates`
    3. `read_resource`
    
    These 3 tools correspond to the [three primary MCP resource protocol
    messages](https://modelcontextprotocol.io/specification/2025-06-18/server/resources#protocol-messages).
    
    Example of listing and reading a GitHub resource tempalte
    <img width="2984" height="804" alt="CleanShot 2025-10-15 at 17 31 10"
    src="https://github.com/user-attachments/assets/89b7f215-2e2a-41c5-90dd-b932ac84a585"
    />
    
    `/mcp` with Figma configured
    <img width="2984" height="442" alt="CleanShot 2025-10-15 at 18 29 35"
    src="https://github.com/user-attachments/assets/a7578080-2ed2-4c59-b9b4-d8461f90d8ee"
    />
    
    Fixes #4956
  • [MCP] Allow specifying cwd and additional env vars (#5246)
    This makes stdio mcp servers more flexible by allowing users to specify
    the cwd to run the server command from and adding additional environment
    variables to be passed through to the server.
    
    Example config using the test server in this repo:
    ```toml
    [mcp_servers.test_stdio]
    cwd = "/Users/<user>/code/codex/codex-rs"
    command = "cargo"
    args = ["run", "--bin", "test_stdio_server"]
    env_vars = ["MCP_TEST_VALUE"]
    ```
    
    @bolinfest I know you hate these env var tests but let's roll with this
    for now. I may take a stab at the env guard + serial macro at some
    point.
  • [MCP] Add auth status to MCP servers (#4918)
    This adds a queryable auth status for MCP servers which is useful:
    1. To determine whether a streamable HTTP server supports auth or not
    based on whether or not it supports RFC 8414-3.2
    2. Allow us to build a better user experience on top of MCP status
  • [MCP] Add support for MCP Oauth credentials (#4517)
    This PR adds oauth login support to streamable http servers when
    `experimental_use_rmcp_client` is enabled.
    
    This PR is large but represents the minimal amount of work required for
    this to work. To keep this PR smaller, login can only be done with
    `codex mcp login` and `codex mcp logout` but it doesn't appear in `/mcp`
    or `codex mcp list` yet. Fingers crossed that this is the last large MCP
    PR and that subsequent PRs can be smaller.
    
    Under the hood, credentials are stored using platform credential
    managers using the [keyring crate](https://crates.io/crates/keyring).
    When the keyring isn't available, it falls back to storing credentials
    in `CODEX_HOME/.credentials.json` which is consistent with how other
    coding agents handle authentication.
    
    I tested this on macOS, Windows, WSL (ubuntu), and Linux. I wasn't able
    to test the dbus store on linux but did verify that the fallback works.
    
    One quirk is that if you have credentials, during development, every
    build will have its own ad-hoc binary so the keyring won't recognize the
    reader as being the same as the write so it may ask for the user's
    password. I may add an override to disable this or allow
    users/enterprises to opt-out of the keyring storage if it causes issues.
    
    <img width="5064" height="686" alt="CleanShot 2025-09-30 at 19 31 40"
    src="https://github.com/user-attachments/assets/9573f9b4-07f1-4160-83b8-2920db287e2d"
    />
    <img width="745" height="486" alt="image"
    src="https://github.com/user-attachments/assets/9562649b-ea5f-4f22-ace2-d0cb438b143e"
    />
  • [MCP] Add experimental support for streamable HTTP MCP servers (#4317)
    This PR adds support for streamable HTTP MCP servers when the
    `experimental_use_rmcp_client` is enabled.
    
    To set one up, simply add a new mcp server config with the url:
    ```
    [mcp_servers.figma]
    url = "http://127.0.0.1:3845/mcp"
    ```
    
    It also supports an optional `bearer_token` which will be provided in an
    authorization header. The full oauth flow is not supported yet.
    
    The config parsing will throw if it detects that the user mixed and
    matched config fields (like command + bearer token or url + env).
    
    The best way to review it is to review `core/src` and then
    `rmcp-client/src/rmcp_client.rs` first. The rest is tests and
    propagating the `Transport` struct around the codebase.
    
    Example with the Figma MCP:
    <img width="5084" height="1614" alt="CleanShot 2025-09-26 at 13 35 40"
    src="https://github.com/user-attachments/assets/eaf2771e-df3e-4300-816b-184d7dec5a28"
    />
  • [MCP] Introduce an experimental official rust sdk based mcp client (#4252)
    The [official Rust
    SDK](https://github.com/modelcontextprotocol/rust-sdk/tree/57fc428c578a1a3fe851ee0838bf068bda120eb3)
    has come a long way since we first started our mcp client implementation
    5 months ago and, today, it is much more complete than our own
    stdio-only implementation.
    
    This PR introduces a new config flag `experimental_use_rmcp_client`
    which will use a new mcp client powered by the sdk instead of our own.
    
    To keep this PR simple, I've only implemented the same stdio MCP
    functionality that we had but will expand on it with future PRs.
    
    ---------
    
    Co-authored-by: pakrym-oai <pakrym@openai.com>