Commit Graph

472 Commits

  • Add text element metadata to protocol, app server, and core (#9331)
    The second part of breaking up PR
    https://github.com/openai/codex/pull/9116
    
    Summary:
    
    - Add `TextElement` / `ByteRange` to protocol user inputs and user
    message events with defaults.
    - Thread `text_elements` through app-server v1/v2 request handling and
    history rebuild.
    - Preserve UI metadata only in user input/events (not `ContentItem`)
    while keeping local image attachments in user events for rehydration.
    
    Details:
    
    - Protocol: `UserInput::Text` carries `text_elements`;
    `UserMessageEvent` carries `text_elements` + `local_images`.
    Serialization includes empty vectors for backward compatibility.
    - app-server-protocol: v1 defines `V1TextElement` / `V1ByteRange` in
    camelCase with conversions; v2 uses its own camelCase wrapper.
    - app-server: v1/v2 input mapping includes `text_elements`; thread
    history rebuilds include them.
    - Core: user event emission preserves UI metadata while model history
    stays clean; history replay round-trips the metadata.
  • [search] allow explicitly disabling web search (#9249)
    moving `web_search` rollout serverside, so need a way to explicitly
    disable search + signal eligibility from the client.
    
    - Add `x‑oai‑web‑search‑eligible` header that signifies whether the
    request can have web search.
    - Only attach the `web_search` tool when the resolved `WebSearchMode` is
    `Live` or `Cached`.
  • Propagate MCP disabled reason (#9207)
    Indicate why MCP servers are disabled when they are disabled by
    requirements:
    
    ```
    ➜  codex git:(main) ✗ just codex mcp list
    cargo run --bin codex -- "$@"
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.27s
         Running `target/debug/codex mcp list`
    Name         Command          Args  Env  Cwd  Status                                                                  Auth
    docs         docs-mcp         -     -    -    disabled: requirements (MDM com.openai.codex:requirements_toml_base64)  Unsupported
    hello_world  hello-world-mcp  -     -    -    disabled: requirements (MDM com.openai.codex:requirements_toml_base64)  Unsupported
    
    ➜  codex git:(main) ✗ just c
    cargo run --bin codex -- "$@"
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.90s
         Running `target/debug/codex`
    ╭─────────────────────────────────────────────╮
    │ >_ OpenAI Codex (v0.0.0)                    │
    │                                             │
    │ model:     gpt-5.2 xhigh   /model to change │
    │ directory: ~/code/codex/codex-rs            │
    ╰─────────────────────────────────────────────╯
    
    /mcp
    
    🔌  MCP Tools
    
      • No MCP tools available.
    
      • docs (disabled)
        • Reason: requirements (MDM com.openai.codex:requirements_toml_base64)
    
      • hello_world (disabled)
        • Reason: requirements (MDM com.openai.codex:requirements_toml_base64)
    ```
  • Add text element metadata to types (#9235)
    Initial type tweaking PR to make the diff of
    https://github.com/openai/codex/pull/9116 smaller
    
    This should not change any behavior, just adds some fields to types
  • add WebSearchMode enum (#9216)
    ### What
    Add `WebSearchMode` enum (disabled, cached live, defaults to cached) to
    config + V2 protocol. This enum takes precedence over legacy flags:
    `web_search_cached`, `web_search_request`, and `tools.web_search`.
    
    Keep `--search` as live.
    
    ### Tests
    Added tests
  • Use current model for review (#9179)
    Instead of having a hard-coded default review model, use the current
    model for running `/review` unless one is specified in the config.
    
    Also inherit current reasoning effort
  • chore: clamp min yield time for empty write_stdin (#9156)
    After evals, 0 impact on performance
  • Renew cache ttl on etag match (#9174)
    so we don't do unnecessary fetches
  • clean models manager (#9168)
    Have only the following Methods:
    - `list_models`: getting current available models
    - `try_list_models`: sync version no refresh for tui use
    - `get_default_model`: get the default model (should be tightened to
    core and received on session configuration)
    - `get_model_info`: get `ModelInfo` for a specific model (should be
    tightened to core but used in tests)
    - `refresh_if_new_etag`: trigger refresh on different etags
    
    Also move the cache to its own struct
  • Fix flakiness in WebSocket tests (#9169)
    The connection was being added to the list after the WebSocket response
    was sent.
    
    So the test can sometimes race and observe connections before the list
    was updated.
    
    After this change, connection and request is added to the list before
    the response is sent.
  • Restrict MCP servers from requirements.toml (#9101)
    Enterprises want to restrict the MCP servers their users can use.
    
    Admins can now specify an allowlist of MCPs in `requirements.toml`. The
    MCP servers are matched on both Name and Transport (local path or HTTP
    URL) -- both must match to allow the MCP server. This prevents
    circumventing the allowlist by renaming MCP servers in user config. (It
    is still possible to replace the local path e.g. rewrite say
    `/usr/local/github-mcp` with a nefarious MCP. We could allow hash
    pinning in the future, but that would break updates. I also think this
    represents a broader, out-of-scope problem.)
    
    We introduce a new field to Constrained: "normalizer". In general, it is
    a fn(T) -> T and applies when `Constrained<T>.set()` is called. In this
    particular case, it disables MCP servers which do not match the
    allowlist. An alternative solution would remove this and instead throw a
    ConstraintError. That would stop Codex launching if any MCP server was
    configured which didn't match. I think this is bad.
    
    We currently reuse the enabled flag on MCP servers to disable them, but
    don't propagate any information about why they are disabled. I'd like to
    add that in a follow up PR, possibly by switching out enabled with an
    enum.
    
    In action:
    
    ```
    # MCP server config has two MCPs. We are going to allowlist one of them.
    ➜  codex git:(gt/restrict-mcps) ✗ cat ~/.codex/config.toml | grep mcp_servers -A1
    [mcp_servers.hello_world]
    command = "hello-world-mcp"
    --
    [mcp_servers.docs]
    command = "docs-mcp"
    
    # Restrict the MCPs to the hello_world MCP.
    ➜  codex git:(gt/restrict-mcps) ✗ defaults read com.openai.codex requirements_toml_base64 | base64 -d
    [mcp_server_allowlist.hello_world]
    command = "hello-world-mcp"
    
    # List the MCPs, observe hello_world is enabled and docs is disabled.
    ➜  codex git:(gt/restrict-mcps) ✗ just codex mcp list
    cargo run --bin codex -- "$@"
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.25s
         Running `target/debug/codex mcp list`
    Name         Command          Args  Env  Cwd  Status    Auth
    docs         docs-mcp         -     -    -    disabled  Unsupported
    hello_world  hello-world-mcp  -     -    -    enabled   Unsupported
    
    # Remove the restrictions.
    ➜  codex git:(gt/restrict-mcps) ✗ defaults delete com.openai.codex requirements_toml_base64
    
    # Observe both MCPs are enabled.
    ➜  codex git:(gt/restrict-mcps) ✗ just codex mcp list
    cargo run --bin codex -- "$@"
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.25s
         Running `target/debug/codex mcp list`
    Name         Command          Args  Env  Cwd  Status   Auth
    docs         docs-mcp         -     -    -    enabled  Unsupported
    hello_world  hello-world-mcp  -     -    -    enabled  Unsupported
    
    # A new requirements that updates the command to one that does not match.
    ➜  codex git:(gt/restrict-mcps) ✗ cat ~/requirements.toml
    [mcp_server_allowlist.hello_world]
    command = "hello-world-mcp-v2"
    
    # Use those requirements.
    ➜  codex git:(gt/restrict-mcps) ✗ defaults write com.openai.codex requirements_toml_base64 "$(base64 -i /Users/gt/requirements.toml)"
    
    # Observe both MCPs are disabled.
    ➜  codex git:(gt/restrict-mcps) ✗ just codex mcp list
    cargo run --bin codex -- "$@"
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.75s
         Running `target/debug/codex mcp list`
    Name         Command          Args  Env  Cwd  Status    Auth
    docs         docs-mcp         -     -    -    disabled  Unsupported
    hello_world  hello-world-mcp  -     -    -    disabled  Unsupported
    ```
  • Support response.done and add integration tests (#9129)
    The agent loop using a persistent incremental web socket connection.
  • Use markdown for migration screen (#8952)
    Next steps will be routing this to model info
  • Websocket append support (#9128)
    Support an incremental append request in websocket transport.
  • Reuse websocket connection (#9127)
    Reuses the connection but still sends full requests.
  • Add model client sessions (#9102)
    Maintain a long-running session.
  • Assemble sandbox/approval/network prompts dynamically (#8961)
    - Add a single builder for developer permissions messaging that accepts
    SandboxPolicy and approval policy. This builder now drives the developer
    “permissions” message that’s injected at session start and any time
    sandbox/approval settings change.
    - Trim EnvironmentContext to only include cwd, writable roots, and
    shell; removed sandbox/approval/network duplication and adjusted XML
    serialization and tests accordingly.
    
    Follow-up: adding a config value to replace the developer permissions
    message for custom sandboxes.
  • Remove unused conversation_id header (#9107)
    It's an exact copy of session_id
  • Label attached images so agent can understand in-message labels (#8950)
    Agent wouldn't "see" attached images and would instead try to use the
    view_file tool:
    <img width="1516" height="504" alt="image"
    src="https://github.com/user-attachments/assets/68a705bb-f962-4fc1-9087-e932a6859b12"
    />
    
    In this PR, we wrap image content items in XML tags with the name of
    each image (now just a numbered name like `[Image #1]`), so that the
    model can understand inline image references (based on name). We also
    put the image content items above the user message which the model seems
    to prefer (maybe it's more used to definitions being before references).
    
    We also tweak the view_file tool description which seemed to help a bit
    
    Results on a simple eval set of images:
    
    Before
    <img width="980" height="310" alt="image"
    src="https://github.com/user-attachments/assets/ba838651-2565-4684-a12e-81a36641bf86"
    />
    
    After
    <img width="918" height="322" alt="image"
    src="https://github.com/user-attachments/assets/10a81951-7ee6-415e-a27e-e7a3fd0aee6f"
    />
    
    ```json
    [
      {
        "id": "single_describe",
        "prompt": "Describe the attached image in one sentence.",
        "images": ["image_a.png"]
      },
      {
        "id": "single_color",
        "prompt": "What is the dominant color in the image? Answer with a single color word.",
        "images": ["image_b.png"]
      },
      {
        "id": "orientation_check",
        "prompt": "Is the image portrait or landscape? Answer in one sentence.",
        "images": ["image_c.png"]
      },
      {
        "id": "detail_request",
        "prompt": "Look closely at the image and call out any small details you notice.",
        "images": ["image_d.png"]
      },
      {
        "id": "two_images_compare",
        "prompt": "I attached two images. Are they the same or different? Briefly explain.",
        "images": ["image_a.png", "image_b.png"]
      },
      {
        "id": "two_images_captions",
        "prompt": "Provide a short caption for each image (Image 1, Image 2).",
        "images": ["image_c.png", "image_d.png"]
      },
      {
        "id": "multi_image_rank",
        "prompt": "Rank the attached images from most colorful to least colorful.",
        "images": ["image_a.png", "image_b.png", "image_c.png"]
      },
      {
        "id": "multi_image_choice",
        "prompt": "Which image looks more vibrant? Answer with 'Image 1' or 'Image 2'.",
        "images": ["image_b.png", "image_d.png"]
      }
    ]
    ```
  • Refactor remote models tests to use TestCodex builder (#8940)
    - add `with_model_provider` to the test codex builder
    - replace the bespoke remote models harness with `TestCodex` in
    `remote_models` tests
  • feat: add support for building with Bazel (#8875)
    This PR configures Codex CLI so it can be built with
    [Bazel](https://bazel.build) in addition to Cargo. The `.bazelrc`
    includes configuration so that remote builds can be done using
    [BuildBuddy](https://www.buildbuddy.io).
    
    If you are familiar with Bazel, things should work as you expect, e.g.,
    run `bazel test //... --keep-going` to run all the tests in the repo,
    but we have also added some new aliases in the `justfile` for
    convenience:
    
    - `just bazel-test` to run tests locally
    - `just bazel-remote-test` to run tests remotely (currently, the remote
    build is for x86_64 Linux regardless of your host platform). Note we are
    currently seeing the following test failures in the remote build, so we
    still need to figure out what is happening here:
    
    ```
    failures:
        suite::compact::manual_compact_twice_preserves_latest_user_messages
        suite::compact_resume_fork::compact_resume_after_second_compaction_preserves_history
        suite::compact_resume_fork::compact_resume_and_fork_preserve_model_history_view
    ```
    
    - `just build-for-release` to build release binaries for all
    platforms/architectures remotely
    
    To setup remote execution:
    - [Create a buildbuddy account](https://app.buildbuddy.io/) (OpenAI
    employees should also request org access at
    https://openai.buildbuddy.io/join/ with their `@openai.com` email
    address.)
    - [Copy your API key](https://app.buildbuddy.io/docs/setup/) to
    `~/.bazelrc` (add the line `build
    --remote_header=x-buildbuddy-api-key=YOUR_KEY`)
    - Use `--config=remote` in your `bazel` invocations (or add `common
    --config=remote` to your `~/.bazelrc`, or use the `just` commands)
    
    ## CI
    
    In terms of CI, this PR introduces `.github/workflows/bazel.yml`, which
    uses Bazel to run the tests _locally_ on Mac and Linux GitHub runners
    (we are working on supporting Windows, but that is not ready yet). Note
    that the failures we are seeing in `just bazel-remote-test` do not occur
    on these GitHub CI jobs, so everything in `.github/workflows/bazel.yml`
    is green right now.
    
    The `bazel.yml` uses extra config in `.github/workflows/ci.bazelrc` so
    that macOS CI jobs build _remotely_ on Linux hosts (using the
    `docker://docker.io/mbolin491/codex-bazel` Docker image declared in the
    root `BUILD.bazel`) using cross-compilation to build the macOS
    artifacts. Then these artifacts are downloaded locally to GitHub's macOS
    runner so the tests can be executed natively. This is the relevant
    config that enables this:
    
    ```
    common:macos --config=remote
    common:macos --strategy=remote
    common:macos --strategy=TestRunner=darwin-sandbox,local
    ```
    
    Because of the remote caching benefits we get from BuildBuddy, these new
    CI jobs can be extremely fast! For example, consider these two jobs that
    ran all the tests on Linux x86_64:
    
    - Bazel 1m37s
    https://github.com/openai/codex/actions/runs/20861063212/job/59940545209?pr=8875
    - Cargo 9m20s
    https://github.com/openai/codex/actions/runs/20861063192/job/59940559592?pr=8875
    
    For now, we will continue to run both the Bazel and Cargo jobs for PRs,
    but once we add support for Windows and running Clippy, we should be
    able to cutover to using Bazel exclusively for PRs, which should still
    speed things up considerably. We will probably continue to run the Cargo
    jobs post-merge for commits that land on `main` as a sanity check.
    
    Release builds will also continue to be done by Cargo for now.
    
    Earlier attempt at this PR: https://github.com/openai/codex/pull/8832
    Earlier attempt to add support for Buck2, now abandoned:
    https://github.com/openai/codex/pull/8504
    
    ---------
    
    Co-authored-by: David Zbarsky <dzbarsky@gmail.com>
    Co-authored-by: Michael Bolin <mbolin@openai.com>
  • Add 5s timeout to models list call + integration test (#8942)
    - Enforce a 5s timeout around the remote models refresh to avoid hanging
    /models calls.
  • fix: increase timeout for wait_for_event() for Bazel (#8946)
    This seems to be necessary to get the Bazel builds on ARM Linux to go
    green on https://github.com/openai/codex/pull/8875.
    
    I don't feel great about timeout-whack-a-mole, but we're still learning
    here...
  • Attempt to reload auth as a step in 401 recovery (#8880)
    When authentication fails, first attempt to reload the auth from file
    and then attempt to refresh it.
  • fix: increase timeout for tests that have been flaking with timeout issues (#8932)
    I have seen this test flake out sometimes when running the macOS build
    using Bazel in CI: https://github.com/openai/codex/pull/8875. Perhaps
    Bazel runs with greater parallelism, inducing a heavier load, causing an
    issue?
  • Immutable CodexAuth (#8857)
    Historically we started with a CodexAuth that knew how to refresh it's
    own tokens and then added AuthManager that did a different kind of
    refresh (re-reading from disk).
    
    I don't think it makes sense for both `CodexAuth` and `AuthManager` to
    be mutable and contain behaviors.
    
    Move all refresh logic into `AuthManager` and keep `CodexAuth` as a data
    object.
  • fix: leverage codex_utils_cargo_bin() in codex-rs/core/tests/suite (#8887)
    This eliminates our dependency on the `escargot` crate and better
    prepares us for Bazel builds: https://github.com/openai/codex/pull/8875.
  • feat: metrics capabilities (#8318)
    Add metrics capabilities to Codex. The `README.md` is up to date.
    
    This will not be merged with the metrics before this PR of course:
    https://github.com/openai/codex/pull/8350
  • feat: introduce find_resource! macro that works with Cargo or Bazel (#8879)
    To support Bazelification in https://github.com/openai/codex/pull/8875,
    this PR introduces a new `find_resource!` macro that we use in place of
    our existing logic in tests that looks for resources relative to the
    compile-time `CARGO_MANIFEST_DIR` env var.
    
    To make this work, we plan to add the following to all `rust_library()`
    and `rust_test()` Bazel rules in the project:
    
    ```
    rustc_env = {
        "BAZEL_PACKAGE": native.package_name(),
    },
    ```
    
    Our new `find_resource!` macro reads this value via
    `option_env!("BAZEL_PACKAGE")` so that the Bazel package _of the code
    using `find_resource!`_ is injected into the code expanded from the
    macro. (If `find_resource()` were a function, then
    `option_env!("BAZEL_PACKAGE")` would always be
    `codex-rs/utils/cargo-bin`, which is not what we want.)
    
    Note we only consider the `BAZEL_PACKAGE` value when the `RUNFILES_DIR`
    environment variable is set at runtime, indicating that the test is
    being run by Bazel. In this case, we have to concatenate the runtime
    `RUNFILES_DIR` with the compile-time `BAZEL_PACKAGE` value to build the
    path to the resource.
    
    In testing this change, I discovered one funky edge case in
    `codex-rs/exec-server/tests/common/lib.rs` where we have to _normalize_
    (but not canonicalize!) the result from `find_resource!` because the
    path contains a `common/..` component that does not exist on disk when
    the test is run under Bazel, so it must be semantically normalized using
    the [`path-absolutize`](https://crates.io/crates/path-absolutize) crate
    before it is passed to `dotslash fetch`.
    
    Because this new behavior may be non-obvious, this PR also updates
    `AGENTS.md` to make humans/Codex aware that this API is preferred.
  • Add feature for optional request compression (#8767)
    Adds a new feature
    `enable_request_compression` that will compress using zstd requests to
    the codex-backend. Currently only enabled for codex-backend so only enabled for openai providers when using chatgpt::auth even when the feature is enabled
    
    Added a new info log line too for evaluating the compression ratio and
    overhead off compressing before requesting. You can enable with
    `RUST_LOG=$RUST_LOG,codex_client::transport=info`
    
    ```
    2026-01-06T00:09:48.272113Z  INFO codex_client::transport: Compressed request body with zstd pre_compression_bytes=28914 post_compression_bytes=11485 compression_duration_ms=0
    ```
  • Override truncation policy at model info level (#8856)
    We used to override truncation policy by comparing model info vs config
    value in context manager. A better way to do it is to construct model
    info using the config value
  • fix: implement 'Allow this session' for apply_patch approvals (#8451)
    **Summary**
    This PR makes “ApprovalDecision::AcceptForSession / don’t ask again this
    session” actually work for `apply_patch` approvals by caching approvals
    based on absolute file paths in codex-core, properly wiring it through
    app-server v2, and exposing the choice in both TUI and TUI2.
    - This brings `apply_patch` calls to be at feature-parity with general
    shell commands, which also have a "Yes, and don't ask again" option.
    - This also fixes VSCE's "Allow this session" button to actually work.
    
    While we're at it, also split the app-server v2 protocol's
    `ApprovalDecision` enum so execpolicy amendments are only available for
    command execution approvals.
    
    **Key changes**
    - Core: per-session patch approval allowlist keyed by absolute file
    paths
    - Handles multi-file patches and renames/moves by recording both source
    and destination paths for `Update { move_path: Some(...) }`.
    - Extend the `Approvable` trait and `ApplyPatchRuntime` to work with
    multiple keys, because an `apply_patch` tool call can modify multiple
    files. For a request to be auto-approved, we will need to check that all
    file paths have been approved previously.
    - App-server v2: honor AcceptForSession for file changes
    - File-change approval responses now map AcceptForSession to
    ReviewDecision::ApprovedForSession (no longer downgraded to plain
    Approved).
    - Replace `ApprovalDecision` with two enums:
    `CommandExecutionApprovalDecision` and `FileChangeApprovalDecision`
    - TUI / TUI2: expose “don’t ask again for these files this session”
    - Patch approval overlays now include a third option (“Yes, and don’t
    ask again for these files this session (s)”).
        - Snapshot updates for the approval modal.
    
    **Tests added/updated**
    - Core:
    - Integration test that proves ApprovedForSession on a patch skips the
    next patch prompt for the same file
    - App-server:
    - v2 integration test verifying
    FileChangeApprovalDecision::AcceptForSession works properly
    
    **User-visible behavior**
    - When the user approves a patch “for session”, future patches touching
    only those previously approved file(s) will no longer prompt gain during
    that session (both via app-server v2 and TUI/TUI2).
    
    **Manual testing**
    Tested both TUI and TUI2 - see screenshots below.
    
    TUI:
    <img width="1082" height="355" alt="image"
    src="https://github.com/user-attachments/assets/adcf45ad-d428-498d-92fc-1a0a420878d9"
    />
    
    
    TUI2:
    <img width="1089" height="438" alt="image"
    src="https://github.com/user-attachments/assets/dd768b1a-2f5f-4bd6-98fd-e52c1d3abd9e"
    />
  • Merge Modelfamily into modelinfo (#8763)
    - Merge ModelFamily into ModelInfo
    - Remove logic for adding instructions to apply patch
    - Add compaction limit and visible context window to `ModelInfo`
  • chore: unify conversation with thread name (#8830)
    Done and verified by Codex + refactor feature of RustRover
  • chore: stabilize core tool parallelism test (#8805)
    Set login=false for the shell tool in the timing-based parallelism test
    so it does not depend on slow user login shells, making the test
    deterministic without user-facing changes. This prevents occasional
    flakes when running locally.
  • Enable model upgrade popup even when selected model is no longer in picker (#8802)
    With `config.toml`:
    ```
    model = "gpt-5.1-codex"
    ```
    (where `gpt-5.1-codex` has `show_in_picker: false` in
    [`model_presets.rs`](https://github.com/openai/codex/blob/main/codex-rs/core/src/models_manager/model_presets.rs);
    this happens if the user hasn't used codex in a while so they didn't see
    the popup before their model was changed to `show_in_picker: false`)
    
    The upgrade picker used to not show (because `gpt-5.1-codex` was
    filtered out of the model list in code). Now, the filtering is done
    downstream in tui and app-server, so the model upgrade popup shows:
    
    <img width="1503" height="227" alt="Screenshot 2026-01-06 at 5 04 37 PM"
    src="https://github.com/user-attachments/assets/26144cc2-0b3f-4674-ac17-e476781ec548"
    />
  • add web_search_cached flag (#8795)
    Add `web_search_cached` feature to config. Enables `web_search` tool
    with access only to cached/indexed results (see
    [docs](https://platform.openai.com/docs/guides/tools-web-search#live-internet-access)).
    
    This takes precedence over the existing `web_search_request`, which
    continues to enable `web_search` over live results as it did before.
    
    `web_search_cached` is disabled for review mode, as `web_search_request`
    is.