Commit Graph

28 Commits

  • chore: cleanup Config instantiation codepaths (#8226)
    This PR does various types of cleanup before I can proceed with more
    ambitious changes to config loading.
    
    First, I noticed duplicated code across these two methods:
    
    
    https://github.com/openai/codex/blob/774bd9e432fa2e0f4e059e97648cf92216912e19/codex-rs/core/src/config/mod.rs#L314-L324
    
    
    https://github.com/openai/codex/blob/774bd9e432fa2e0f4e059e97648cf92216912e19/codex-rs/core/src/config/mod.rs#L334-L344
    
    This has now been consolidated in
    `load_config_as_toml_with_cli_overrides()`.
    
    Further, I noticed that `Config::load_with_cli_overrides()` took two
    similar arguments:
    
    
    https://github.com/openai/codex/blob/774bd9e432fa2e0f4e059e97648cf92216912e19/codex-rs/core/src/config/mod.rs#L308-L311
    
    The difference between `cli_overrides` and `overrides` was not
    immediately obvious to me. At first glance, it appears that one should
    be able to be expressed in terms of the other, but it turns out that
    some fields of `ConfigOverrides` (such as `cwd` and
    `codex_linux_sandbox_exe`) are, by design, not configurable via a
    `.toml` file or a command-line `--config` flag.
    
    That said, I discovered that many callers of
    `Config::load_with_cli_overrides()` were passing
    `ConfigOverrides::default()` for `overrides`, so I created two separate
    methods:
    
    - `Config::load_with_cli_overrides(cli_overrides: Vec<(String,
    TomlValue)>)`
    - `Config::load_with_cli_overrides_and_harness_overrides(cli_overrides:
    Vec<(String, TomlValue)>, harness_overrides: ConfigOverrides)`
    
    The latter has a long name, as it is _not_ what should be used in the
    common case, so the extra typing is designed to draw attention to this
    fact. I tried to update the existing callsites to use the shorter name,
    where possible.
    
    Further, in the cases where `ConfigOverrides` is used, usually only a
    limited subset of fields are actually set, so I updated the declarations
    to leverage `..Default::default()` where possible.
  • Update RMCP client config guidance (#7895)
    ## Summary
    - update CLI OAuth guidance to reference `features.rmcp_client` instead
    of the deprecated experimental flag
    - keep login/logout help text consistent with the new feature flag
    
    ## Testing
    - `cargo test -p codex-cli`
    
    
    ------
    [Codex
    Task](https://chatgpt.com/codex/tasks/task_i_693b3e0bf27c832cb66d585847a552ab)
  • fix(cli): correct mcp add usage order (#6827)
    ## Summary
    - add an explicit `override_usage` string to `AddArgs` so clap prints
    `<NAME>` before the command/url choice, matching the actual parser and
    docs
    
    ### Before
    
    Usage: codex mcp add [OPTIONS] <COMMAND|--url <URL>> <NAME>
    
    
    ### After
    
    Usage: codex mcp add [OPTIONS] <NAME> [--url <URL> | -- <COMMAND>...]
    
    ---------
    
    Signed-off-by: kyuheon-kr <kyuheon.kr@gmail.com>
  • fix: Update the deprecation message to link to the docs (#6211)
    The deprecation message is currently a bit confusing. Users may not
    understand what is `[features].x`. I updated the docs and the
    deprecation message for more guidance.
    
    ---------
    
    Co-authored-by: Gabriel Peal <gpeal@users.noreply.github.com>
  • Fix typo in error message for OAuth login (#6159)
    Error message for attempting to OAuth with a remote RCP is incorrect and
    misleading. The correct config is
    
    ```
    [features]
    rmcp_client = true
    ```
    
    Co-authored-by: Eric Traut <etraut@openai.com>
  • Windows Sandbox - Alpha version (#4905)
    - Added the new codex-windows-sandbox crate that builds both a library
    entry point (run_windows_sandbox_capture) and a CLI executable to launch
    commands inside a Windows restricted-token sandbox, including ACL
    management, capability SID provisioning, network lockdown, and output
    capture
    (windows-sandbox-rs/src/lib.rs:167, windows-sandbox-rs/src/main.rs:54).
    - Introduced the experimental WindowsSandbox feature flag and wiring so
    Windows builds can opt into the sandbox:
    SandboxType::WindowsRestrictedToken, the in-process execution path, and
    platform sandbox selection now honor the flag (core/src/features.rs:47,
    core/src/config.rs:1224, core/src/safety.rs:19,
    core/src/sandboxing/mod.rs:69, core/src/exec.rs:79,
    core/src/exec.rs:172).
    - Updated workspace metadata to include the new crate and its
    Windows-specific dependencies so the core crate can link against it
    (codex-rs/
        Cargo.toml:91, core/Cargo.toml:86).
    - Added a PowerShell bootstrap script that installs the Windows
    toolchain, required CLI utilities, and builds the workspace to ease
    development
        on the platform (scripts/setup-windows.ps1:1).
    - Landed a Python smoke-test suite that exercises
    read-only/workspace-write policies, ACL behavior, and network denial for
    the Windows sandbox
        binary (windows-sandbox-rs/sandbox_smoketests.py:1).
  • chore: config editor (#5878)
    The goal is to have a single place where we actually write files
    
    In a follow-up PR, will move everything config related in a dedicated
    module and move the helpers in a dedicated file
  • [MCP] Properly gate login after mcp add with experimental_use_rmcp_client (#5653)
    There was supposed to be a check here like in other places.
  • [MCP] Add support for specifying scopes for MCP oauth (#5487)
    ```
    codex mcp login server_name --scopes=scope1,scope2,scope3
    ```
    
    Fixes #5480
  • [MCP] Dedicated error message for GitHub MCPs missing a personal access token (#5393)
    Because the GitHub MCP is one of the most popular MCPs and it
    confusingly doesn't support OAuth, we should make it more clear how to
    make it work so people don't think Codex is broken.
  • [MCP] Add configuration options to enable or disable specific tools (#5367)
    Some MCP servers expose a lot of tools. In those cases, it is reasonable
    to allow/denylist tools for Codex to use so it doesn't get overwhelmed
    with too many tools.
    
    The new configuration options available in the `mcp_server` toml table
    are:
    * `enabled_tools`
    * `disabled_tools`
    
    Fixes #4796
  • [MCP] Allow specifying cwd and additional env vars (#5246)
    This makes stdio mcp servers more flexible by allowing users to specify
    the cwd to run the server command from and adding additional environment
    variables to be passed through to the server.
    
    Example config using the test server in this repo:
    ```toml
    [mcp_servers.test_stdio]
    cwd = "/Users/<user>/code/codex/codex-rs"
    command = "cargo"
    args = ["run", "--bin", "test_stdio_server"]
    env_vars = ["MCP_TEST_VALUE"]
    ```
    
    @bolinfest I know you hate these env var tests but let's roll with this
    for now. I may take a stab at the env guard + serial macro at some
    point.
  • [MCP] Allow specifying custom headers with streamable http servers (#5241)
    This adds two new config fields to streamable http mcp servers:
    `http_headers`: a map of key to value
    `env_http_headers` a map of key to env var which will be resolved at
    request time
    
    All headers will be passed to all MCP requests to that server just like
    authorization headers.
    
    There is a test ensuring that headers are not passed to other servers.
    
    Fixes #5180
  • [MCP] Prompt mcp login when adding a streamable HTTP server that supports oauth (#5193)
    1. If Codex detects that a `codex mcp add -url …` server supports oauth,
    it will auto-initiate the login flow.
    2. If the TUI starts and a MCP server supports oauth but isn't logged
    in, it will give the user an explicit warning telling them to log in.
  • feat: feature flag (#4948)
    Add proper feature flag instead of having custom flags for everything.
    This is just for experimental/wip part of the code
    It can be used through CLI:
    ```bash
    codex --enable unified_exec --disable view_image_tool
    ```
    
    Or in the `config.toml`
    ```toml
    # Global toggles applied to every profile unless overridden.
    [features]
    apply_patch_freeform = true
    view_image_tool = false
    ```
    
    Follow-up:
    In a following PR, the goal is to have a default have `bundles` of
    features that we can associate to a model
  • [MCP] Add auth status to MCP servers (#4918)
    This adds a queryable auth status for MCP servers which is useful:
    1. To determine whether a streamable HTTP server supports auth or not
    based on whether or not it supports RFC 8414-3.2
    2. Allow us to build a better user experience on top of MCP status
  • [MCP] Add an enabled config field (#4917)
    This lets users more easily toggle MCP servers.
  • [MCP] Add support for streamable http servers with codex mcp add and replace bearer token handling (#4904)
    1. You can now add streamable http servers via the CLI
    2. As part of this, I'm also changing the existing bearer_token plain
    text config field with ane env var
    
    ```
    mcp add github --url https://api.githubcopilot.com/mcp/ --bearer-token-env-var=GITHUB_PAT
    ```
  • [MCP] Add the ability to explicitly specify a credentials store (#4857)
    This lets users/companies explicitly choose whether to force/disallow
    the keyring/fallback file storage for mcp credentials.
    
    People who develop with Codex will want to use this until we sign
    binaries or else each ad-hoc debug builds will require keychain access
    on every build. I don't love this and am open to other ideas for how to
    handle that.
    
    
    ```toml
    mcp_oauth_credentials_store = "auto"
    mcp_oauth_credentials_store = "file"
    mcp_oauth_credentials_store = "keyrung"
    ```
    Defaults to `auto`
  • add(core): managed config (#3868)
    ## Summary
    
    - Factor `load_config_as_toml` into `core::config_loader` so config
    loading is reusable across callers.
    - Layer `~/.codex/config.toml`, optional `~/.codex/managed_config.toml`,
    and macOS managed preferences (base64) with recursive table merging and
    scoped threads per source.
    
    ## Config Flow
    
    ```
    Managed prefs (macOS profile: com.openai.codex/config_toml_base64)
                                   ▲
                                   │
    ~/.codex/managed_config.toml   │  (optional file-based override)
                                   ▲
                                   │
                    ~/.codex/config.toml (user-defined settings)
    ```
    
    - The loader searches under the resolved `CODEX_HOME` directory
    (defaults to `~/.codex`).
    - Managed configs let administrators ship fleet-wide overrides via
    device profiles which is useful for enforcing certain settings like
    sandbox or approval defaults.
    - For nested hash tables: overlays merge recursively. Child tables are
    merged key-by-key, while scalar or array values replace the prior layer
    entirely. This lets admins add or tweak individual fields without
    clobbering unrelated user settings.
  • [MCP] Add support for MCP Oauth credentials (#4517)
    This PR adds oauth login support to streamable http servers when
    `experimental_use_rmcp_client` is enabled.
    
    This PR is large but represents the minimal amount of work required for
    this to work. To keep this PR smaller, login can only be done with
    `codex mcp login` and `codex mcp logout` but it doesn't appear in `/mcp`
    or `codex mcp list` yet. Fingers crossed that this is the last large MCP
    PR and that subsequent PRs can be smaller.
    
    Under the hood, credentials are stored using platform credential
    managers using the [keyring crate](https://crates.io/crates/keyring).
    When the keyring isn't available, it falls back to storing credentials
    in `CODEX_HOME/.credentials.json` which is consistent with how other
    coding agents handle authentication.
    
    I tested this on macOS, Windows, WSL (ubuntu), and Linux. I wasn't able
    to test the dbus store on linux but did verify that the fallback works.
    
    One quirk is that if you have credentials, during development, every
    build will have its own ad-hoc binary so the keyring won't recognize the
    reader as being the same as the write so it may ask for the user's
    password. I may add an override to disable this or allow
    users/enterprises to opt-out of the keyring storage if it causes issues.
    
    <img width="5064" height="686" alt="CleanShot 2025-09-30 at 19 31 40"
    src="https://github.com/user-attachments/assets/9573f9b4-07f1-4160-83b8-2920db287e2d"
    />
    <img width="745" height="486" alt="image"
    src="https://github.com/user-attachments/assets/9562649b-ea5f-4f22-ace2-d0cb438b143e"
    />
  • fix: separate codex mcp into codex mcp-server and codex app-server (#4471)
    This is a very large PR with some non-backwards-compatible changes.
    
    Historically, `codex mcp` (or `codex mcp serve`) started a JSON-RPC-ish
    server that had two overlapping responsibilities:
    
    - Running an MCP server, providing some basic tool calls.
    - Running the app server used to power experiences such as the VS Code
    extension.
    
    This PR aims to separate these into distinct concepts:
    
    - `codex mcp-server` for the MCP server
    - `codex app-server` for the "application server"
    
    Note `codex mcp` still exists because it already has its own subcommands
    for MCP management (`list`, `add`, etc.)
    
    The MCP logic continues to live in `codex-rs/mcp-server` whereas the
    refactored app server logic is in the new `codex-rs/app-server` folder.
    Note that most of the existing integration tests in
    `codex-rs/mcp-server/tests/suite` were actually for the app server, so
    all the tests have been moved with the exception of
    `codex-rs/mcp-server/tests/suite/mod.rs`.
    
    Because this is already a large diff, I tried not to change more than I
    had to, so `codex-rs/app-server/tests/common/mcp_process.rs` still uses
    the name `McpProcess` for now, but I will do some mechanical renamings
    to things like `AppServer` in subsequent PRs.
    
    While `mcp-server` and `app-server` share some overlapping functionality
    (like reading streams of JSONL and dispatching based on message types)
    and some differences (completely different message types), I ended up
    doing a bit of copypasta between the two crates, as both have somewhat
    similar `message_processor.rs` and `outgoing_message.rs` files for now,
    though I expect them to diverge more in the near future.
    
    One material change is that of the initialize handshake for `codex
    app-server`, as we no longer use the MCP types for that handshake.
    Instead, we update `codex-rs/protocol/src/mcp_protocol.rs` to add an
    `Initialize` variant to `ClientRequest`, which takes the `ClientInfo`
    object we need to update the `USER_AGENT_SUFFIX` in
    `codex-rs/app-server/src/message_processor.rs`.
    
    One other material change is in
    `codex-rs/app-server/src/codex_message_processor.rs` where I eliminated
    a use of the `send_event_as_notification()` method I am generally trying
    to deprecate (because it blindly maps an `EventMsg` into a
    `JSONNotification`) in favor of `send_server_notification()`, which
    takes a `ServerNotification`, as that is intended to be a custom enum of
    all notification types supported by the app server. So to make this
    update, I had to introduce a new variant of `ServerNotification`,
    `SessionConfigured`, which is a non-backwards compatible change with the
    old `codex mcp`, and clients will have to be updated after the next
    release that contains this PR. Note that
    `codex-rs/app-server/tests/suite/list_resume.rs` also had to be update
    to reflect this change.
    
    I introduced `codex-rs/utils/json-to-toml/src/lib.rs` as a small utility
    crate to avoid some of the copying between `mcp-server` and
    `app-server`.
  • [MCP] Add experimental support for streamable HTTP MCP servers (#4317)
    This PR adds support for streamable HTTP MCP servers when the
    `experimental_use_rmcp_client` is enabled.
    
    To set one up, simply add a new mcp server config with the url:
    ```
    [mcp_servers.figma]
    url = "http://127.0.0.1:3845/mcp"
    ```
    
    It also supports an optional `bearer_token` which will be provided in an
    authorization header. The full oauth flow is not supported yet.
    
    The config parsing will throw if it detects that the user mixed and
    matched config fields (like command + bearer token or url + env).
    
    The best way to review it is to review `core/src` and then
    `rmcp-client/src/rmcp_client.rs` first. The rest is tests and
    propagating the `Transport` struct around the codebase.
    
    Example with the Figma MCP:
    <img width="5084" height="1614" alt="CleanShot 2025-09-26 at 13 35 40"
    src="https://github.com/user-attachments/assets/eaf2771e-df3e-4300-816b-184d7dec5a28"
    />
  • timeouts for mcp tool calls (#3959)
    defaults to 60sec, overridable with MCP_TOOL_TIMEOUT or on a per-server
    basis in the config.
  • initial mcp add interface (#3543)
    Adds `codex mcp add`, `codex mcp list`, `codex mcp remove`. Currently writes to global config.