Commit Graph

267 Commits

  • fix: Scope ExecSessionManager to Session instead of using global singleton (#2664)
    The `SessionManager` in `exec_command` owns a number of
    `ExecCommandSession` objects where `ExecCommandSession` has a
    non-trivial implementation of `Drop`, so we want to be able to drop an
    individual `SessionManager` to help ensure things get cleaned up in a
    timely fashion. To that end, we should have one `SessionManager` per
    session rather than one global one for the lifetime of the CLI process.
  • fix: build is broken on main; introduce ToolsConfigParams to help fix (#2663)
    `ToolsConfig::new()` taking a large number of boolean params was hard to
    manage and it finally bit us (see
    https://github.com/openai/codex/pull/2660). This changes
    `ToolsConfig::new()` so that it takes a struct (and also reduces the
    visibility of some members, where possible).
  • Fix cache hit rate by making MCP tools order deterministic (#2611)
    Fixes https://github.com/openai/codex/issues/2610
    
    This PR sorts the tools in `get_openai_tools` by name to ensure a
    consistent MCP tool order.
    
    Currently, MCP servers are stored in a HashMap, which does not guarantee
    ordering. As a result, the tool order changes across turns, effectively
    breaking prompt caching in multi-turn sessions.
    
    An alternative solution would be to replace the HashMap with an ordered
    structure, but that would require a much larger code change. Given that
    it is unrealistic to have so many MCP tools that sorting would cause
    performance issues, this lightweight fix is chosen instead.
    
    By ensuring deterministic tool order, this change should significantly
    improve cache hit rates and prevent users from hitting usage limits too
    quickly. (For reference, my own sessions last week reached the limit
    unusually fast, with cache hit rates falling below 1%.)
    
    ## Result
    
    After this fix, sessions with MCP servers now show caching behavior
    almost identical to sessions without MCP servers.
    Without MCP             |  With MCP
    :-------------------------:|:-------------------------:
    <img width="1368" height="1634" alt="image"
    src="https://github.com/user-attachments/assets/26edab45-7be8-4d6a-b471-558016615fc8"
    /> | <img width="1356" height="1632" alt="image"
    src="https://github.com/user-attachments/assets/5f3634e0-3888-420b-9aaf-deefd9397b40"
    />
  • [apply_patch] disable default freeform tool (#2643)
    ## Summary
    We're seeing some issues in the freeform tool - let's disable by default
    until it stabilizes.
    
    ## Testing
    - [x] Ran locally, confirmed codex-cli could make edits
  • Add web search tool (#2371)
    Adds web_search tool, enabling the model to use Responses API web_search
    tool.
    - Disabled by default, enabled by --search flag
    - When --search is passed, exposes web_search_request function tool to
    the model, which triggers user approval. When approved, the model can
    use the web_search tool for the remainder of the turn
    <img width="1033" height="294" alt="image"
    src="https://github.com/user-attachments/assets/62ac6563-b946-465c-ba5d-9325af28b28f"
    />
    
    ---------
    
    Co-authored-by: easong-openai <easong@openai.com>
  • send-aggregated output (#2364)
    We want to send an aggregated output of stderr and stdout so we don't
    have to aggregate it stderr+stdout as we lose order sometimes.
    
    ---------
    
    Co-authored-by: Gabriel Peal <gpeal@users.noreply.github.com>
  • fork conversation from a previous message (#2575)
    This can be the underlying logic in order to start a conversation from a
    previous message. will need some love in the UI.
    
    Base for building this: #2588
  • Move models.rs to protocol (#2595)
    Moving models.rs to protocol so we can use them in `Codex` operations
  • fix: prefer sending MCP structuredContent as the function call response, if available (#2594)
    Prior to this change, when we got a `CallToolResult` from an MCP server,
    we JSON-serialized its `content` field as the `content` to send back to
    the model as part of the function call output that we send back to the
    model. This meant that we were dropping the `structuredContent` on the
    floor.
    
    Though reading
    https://modelcontextprotocol.io/specification/2025-06-18/schema#tool, it
    appears that if `outputSchema` is specified, then `structuredContent`
    should be set, which seems to be a "higher-fidelity" response to the
    function call. This PR updates our handling of `CallToolResult` to
    prefer using the JSON-serialization of `structuredContent`, if present,
    using `content` as a fallback.
    
    Also, it appears that the sense of `success` was inverted prior to this
    PR!
  • test: simplify tests in config.rs (#2586)
    this is much easier to read, thanks @bolinfest for the suggestion.
  • [config] Detect git worktrees for project trust (#2585)
    ## Summary
    When resolving our current directory as a project, we want to be a
    little bit more clever:
    1. If we're in a sub-directory of a git repo, resolve our project
    against the root of the git repo
    2. If we're in a git worktree, resolve the project against the root of
    the git repo
    
    ## Testing
    - [x] Added unit tests
    - [x] Confirmed locally with a git worktree (the one i was using for
    this feature)
  • [apply_patch] freeform apply_patch tool (#2576)
    ## Summary
    GPT-5 introduced the concept of [custom
    tools](https://platform.openai.com/docs/guides/function-calling#custom-tools),
    which allow the model to send a raw string result back, simplifying
    json-escape issues. We are migrating gpt-5 to use this by default.
    
    However, gpt-oss models do not support custom tools, only normal
    functions. So we keep both tool definitions, and provide whichever one
    the model family supports.
    
    ## Testing
    - [x] Tested locally with various models
    - [x] Unit tests pass
  • Add AuthManager and enhance GetAuthStatus command (#2577)
    This PR adds a central `AuthManager` struct that manages the auth
    information used across conversations and the MCP server. Prior to this,
    each conversation and the MCP server got their own private snapshots of
    the auth information, and changes to one (such as a logout or token
    refresh) were not seen by others.
    
    This is especially problematic when multiple instances of the CLI are
    run. For example, consider the case where you start CLI 1 and log in to
    ChatGPT account X and then start CLI 2 and log out and then log in to
    ChatGPT account Y. The conversation in CLI 1 is still using account X,
    but if you create a new conversation, it will suddenly (and
    unexpectedly) switch to account Y.
    
    With the `AuthManager`, auth information is read from disk at the time
    the `ConversationManager` is constructed, and it is cached in memory.
    All new conversations use this same auth information, as do any token
    refreshes.
    
    The `AuthManager` is also used by the MCP server's GetAuthStatus
    command, which now returns the auth method currently used by the MCP
    server.
    
    This PR also includes an enhancement to the GetAuthStatus command. It
    now accepts two new (optional) input parameters: `include_token` and
    `refresh_token`. Callers can use this to request the in-use auth token
    and can optionally request to refresh the token.
    
    The PR also adds tests for the login and auth APIs that I recently added
    to the MCP server.
  • feat(gpt5): add model_verbosity for GPT‑5 via Responses API (#2108)
    **Summary**
    - Adds `model_verbosity` config (values: low, medium, high).
    - Sends `text.verbosity` only for GPT‑5 family models via the Responses
    API.
    - Updates docs and adds serialization tests.
    
    **Motivation**
    - GPT‑5 introduces a verbosity control to steer output length/detail
    without pro
    mpt surgery.
    - Exposing it as a config knob keeps prompts stable and makes behavior
    explicit
    and repeatable.
    
    **Changes**
    - Config:
      - Added `Verbosity` enum (low|medium|high).
    - Added optional `model_verbosity` to `ConfigToml`, `Config`, and
    `ConfigProfi
    le`.
    - Request wiring:
      - Extended `ResponsesApiRequest` with optional `text` object.
    - Populates `text.verbosity` only when model family is `gpt-5`; omitted
    otherw
    ise.
    - Tests:
    - Verifies `text.verbosity` serializes when set and is omitted when not
    set.
    - Docs:
      - Added “GPT‑5 Verbosity” section in `codex-rs/README.md`.
      - Added `model_verbosity` section to `codex-rs/config.md`.
    
    **Usage**
    - In `~/.codex/config.toml`:
      - `model = "gpt-5"`
      - `model_verbosity = "low"` (or `"medium"` default, `"high"`)
    - CLI override example:
      - `codex -c model="gpt-5" -c model_verbosity="high"`
    
    **API Impact**
    - Requests to GPT‑5 via Responses API include: `text: { verbosity:
    "low|medium|h
    igh" }` when configured.
    - For legacy models or Chat Completions providers, `text` is omitted.
    
    **Backward Compatibility**
    - Default behavior unchanged when `model_verbosity` is not set (server
    default “
    medium”).
    
    **Testing**
    - Added unit tests for serialization/omission of `text.verbosity`.
    - Ran `cargo fmt` and `cargo test --all-features` (all green).
    
    **Docs**
    - `README.md`: new “GPT‑5 Verbosity” note under Config with example.
    - `config.md`: new `model_verbosity` section.
    
    **Out of Scope**
    - No changes to temperature/top_p or other GPT‑5 parameters.
    - No changes to Chat Completions wiring.
    
    **Risks / Notes**
    - If OpenAI changes the wire shape for verbosity, we may need to update
    `Respons
    esApiRequest`.
    - Behavior gated to `gpt-5` model family to avoid unexpected effects
    elsewhere.
    
    **Checklist**
    - [x] Code gated to GPT‑5 family only
    - [x] Docs updated (`README.md`, `config.md`)
    - [x] Tests added and passing
    - [x] Formatting applied
    
    Release note: Add `model_verbosity` config to control GPT‑5 output verbosity via the Responses API (low|medium|high).
  • [apply-patch] Clean up apply-patch tool definitions (#2539)
    ## Summary
    We've experienced a bit of drift in system prompting for `apply_patch`:
    - As pointed out in #2030 , our prettier formatting started altering
    prompt.md in a few ways
    - We introduced a separate markdown file for apply_patch instructions in
    #993, but currently duplicate them in the prompt.md file
    - We added a first-class apply_patch tool in #2303, which has yet
    another definition
    
    This PR starts to consolidate our logic in a few ways:
    - We now only use
    `apply_patch_tool_instructions.md](https://github.com/openai/codex/compare/dh--apply-patch-tool-definition?expand=1#diff-d4fffee5f85cb1975d3f66143a379e6c329de40c83ed5bf03ffd3829df985bea)
    for system instructions
    - We no longer include apply_patch system instructions if the tool is
    specified
    
    I'm leaving the definition in openai_tools.rs as duplicated text for now
    because we're going to be iterated on the first-class tool soon.
    
    ## Testing
    - [x] Added integration tests to verify prompt stability
    - [x] Tested locally with several different models (gpt-5, gpt-oss,
    o4-mini)
  • [shell_tool] Small updates to ensure shell consistency (#2571)
    ## Summary
    Small update to hopefully improve some shell edge cases, and make the
    function clearer to the model what is going on. Keeping `timeout` as an
    alias means that calls with the previous name will still work.
    
    ## Test Plan
    - [x] Tested locally, model still works
  • core: write explicit [projects] tables for trusted projects (#2523)
    all of my trust_level settings in my ~/.codex/config.toml were on one
    line.
  • read all AGENTS.md up to git root (#2532)
    This updates our logic for AGENTS.md to match documented behavior, which
    is to read all AGENTS.md files from cwd up to git root.
  • [prompt] xml-format EnvironmentContext (#2272)
    ## Summary
    Before we land #2243, let's start printing environment_context in our
    preferred format. This struct will evolve over time with new
    information, xml gives us a balance of human readable without too much
    parsing, llm readable, and extensible.
    
    Also moves us over to an Option-based struct, so we can easily provide
    diffs to the model.
    
    ## Testing
    - [x] Updated tests to reflect new format
  • Bridge command generation to powershell when on Windows (#2319)
    ## What? Why? How?
    - When running on Windows, codex often tries to invoke bash commands,
    which commonly fail (unless WSL is installed)
    - Fix: Detect if powershell is available and, if so, route commands to
    it
    - Also add a shell_name property to environmental context for codex to
    default to powershell commands when running in that environment
    
    ## Testing
    - Tested within WSL and powershell (e.g. get top 5 largest files within
    a folder and validated that commands generated were powershell commands)
    - Tested within Zsh
    - Updated unit tests
    
    ---------
    
    Co-authored-by: Eddy Escardo <eddy@openai.com>
  • Fix login for internal employees (#2528)
    This PR:
    - fixes for internal employee because we currently want to prefer SIWC
    for them.
    - fixes retrying forever on unauthorized access. we need to break
    eventually on max retries.
  • detect terminal and include in request headers (#2437)
    This adds the terminal version to the UA header.
  • Refresh ChatGPT auth token (#2484)
    ChatGPT token's live for only 1 hour. If the session is longer we don't
    refresh the token. We should get the expiry timestamp and attempt to
    refresh before it.
  • chore: Rust 1.89 promoted file locking to the standard library, so prefer stdlib to fs2 (#2467)
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2467).
    * __->__ #2467
    * #2465
  • chore: upgrade to Rust 1.89 (#2465)
    Codex created this PR from the following prompt:
    
    > upgrade this entire repo to Rust 1.89. Note that this requires
    updating codex-rs/rust-toolchain.toml as well as the workflows in
    .github/. Make sure that things are "clippy clean" as this change will
    likely uncover new Clippy errors. `just fmt` and `cargo clippy --tests`
    are sufficient to check for correctness
    
    Note this modifies a lot of lines because it folds nested `if`
    statements using `&&`.
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2465).
    * #2467
    * __->__ #2465
  • [tui] Support /mcp command (#2430)
    ## Summary
    Adds a `/mcp` command to list active tools. We can extend this command
    to allow configuration of MCP tools, but for now a simple list command
    will help debug if your config.toml and your tools are working as
    expected.
  • Show login options when not signed in with ChatGPT (#2440)
    Motivation: we have users who uses their API key although they want to
    use ChatGPT account. We want to give them the chance to always login
    with their account.
    
    This PR displays login options when the user is not signed in with
    ChatGPT. Even if you have set an OpenAI API key as an environment
    variable, you will still be prompted to log in with ChatGPT.
    
    We’ve also added a new flag, `always_use_api_key_signing` false by
    default, which ensures you are never asked to log in with ChatGPT and
    always defaults to using your API key.
    
    
    
    https://github.com/user-attachments/assets/b61ebfa9-3c5e-4ab7-bf94-395c23a0e0af
    
    After ChatGPT sign in:
    
    
    https://github.com/user-attachments/assets/d58b366b-c46a-428f-a22f-2ac230f991c0
  • Add an operation to override current task context (#2431)
    - Added an operation to override current task context
    - Added a test to check that cache stays the same
  • consolidate reasoning enums into one (#2428)
    We have three enums for each of reasoning summaries and reasoning effort
    with same values. They can be consolidated into one.
  • chore: move mcp-server/src/wire_format.rs to protocol/src/mcp_protocol.rs (#2423)
    The existing `wire_format.rs` should share more types with the
    `codex-protocol` crate (like `AskForApproval` instead of maintaining a
    parallel `CodexToolCallApprovalPolicy` enum), so this PR moves
    `wire_format.rs` into `codex-protocol`, renaming it as
    `mcp-protocol.rs`. We also de-dupe types, where appropriate.
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2423).
    * #2424
    * __->__ #2423
  • fix: introduce EventMsg::TurnAborted (#2365)
    Introduces `EventMsg::TurnAborted` that should be sent in response to
    `Op::Interrupt`.
    
    In the MCP server, updates the handling of a
    `ClientRequest::InterruptConversation` request such that it sends the
    `Op::Interrupt` but does not respond to the request until it sees an
    `EventMsg::TurnAborted`.
  • Fix #2296 Add "minimal" reasoning effort for GPT 5 models (#2326)
    This pull request resolves #2296; I've confirmed if it works by:
    
    1. Add settings to ~/.codex/config.toml:
    ```toml
    model_reasoning_effort = "minimal"
    ```
    
    2. Run the CLI:
    ```
    cd codex-rs
    cargo build && RUST_LOG=trace cargo run --bin codex
    /status
    tail -f ~/.codex/log/codex-tui.log
    ```
    
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • feat: introduce Op:UserTurn (#2329)
    This introduces `Op::UserTurn`, which makes it possible to override many
    of the fields that were set when the `Session` was originally created
    when creating a new conversation turn. This is one way we could support
    changing things like `model` or `cwd` in the middle of the conversation,
    though we may want to consider making each field optional, or
    alternatively having a separate `Op` that mutates the `TurnContext`
    associated with a `submission_loop()`.
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2329).
    * #2345
    * __->__ #2329
    * #2343
    * #2340
    * #2338
  • feat: introduce TurnContext (#2343)
    This PR introduces `TurnContext`, which is designed to hold a set of
    fields that should be constant for a turn of a conversation. Note that
    the fields of `TurnContext` were previously governed by `Session`.
    
    Ultimately, we want to enable users to change these values between turns
    (changing model, approval policy, etc.), though in the current
    implementation, the `TurnContext` is constant for the entire
    conversation.
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2345).
    * #2345
    * #2329
    * __->__ #2343
    * #2340
    * #2338
  • fix: introduce MutexExt::lock_unchecked() so we stop ignoring unwrap() throughout codex.rs (#2340)
    This way we are sure a dangerous `unwrap()` does not sneak in!
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2340).
    * #2345
    * #2329
    * #2343
    * __->__ #2340
    * #2338
  • fix: tighten up checks against writable folders for SandboxPolicy (#2338)
    I was looking at the implementation of `Session::get_writable_roots()`,
    which did not seem right, as it was a copy of writable roots, which is
    not guaranteed to be in sync with the `sandbox_policy` field.
    
    I looked at who was calling `get_writable_roots()` and its only call
    site was `apply_patch()` in `codex-rs/core/src/apply_patch.rs`, which
    took the roots and forwarded them to `assess_patch_safety()` in
    `safety.rs`. I updated `assess_patch_safety()` to take `sandbox_policy:
    &SandboxPolicy` instead of `writable_roots: &[PathBuf]` (and replaced
    `Session::get_writable_roots()` with `Session::get_sandbox_policy()`).
    
    Within `safety.rs`, it was fairly easy to update
    `is_write_patch_constrained_to_writable_paths()` to work with
    `SandboxPolicy`, and in particular, it is far more accurate because, for
    better or worse, `SandboxPolicy::get_writable_roots_with_cwd()` _returns
    an empty vec_ for `SandboxPolicy::DangerFullAccess`, suggesting that
    _nothing_ is writable when in reality _everything_ is writable. With
    this PR, `is_write_patch_constrained_to_writable_paths()` now does the
    right thing for each variant of `SandboxPolicy`.
    
    I thought this would be the end of the story, but it turned out that
    `test_writable_roots_constraint()` in `safety.rs` needed to be updated,
    as well. In particular, the test was writing to
    `std::env::current_dir()` instead of a `TempDir`, which I suspect was a
    holdover from earlier when `SandboxPolicy::WorkspaceWrite` would always
    make `TMPDIR` writable on macOS, which made it hard to write tests to
    verify `SandboxPolicy` in `TMPDIR`. Fortunately, we now have
    `exclude_tmpdir_env_var` as an option on
    `SandboxPolicy::WorkspaceWrite`, so I was able to update the test to
    preserve the existing behavior, but to no longer write to
    `std::env::current_dir()`.
    
    
    
    
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/2338).
    * #2345
    * #2329
    * #2343
    * #2340
    * __->__ #2338
  • [tools] Add apply_patch tool (#2303)
    ## Summary
    We've been seeing a number of issues and reports with our synthetic
    `apply_patch` tool, e.g. #802. Let's make this a real tool - in my
    anecdotal testing, it's critical for GPT-OSS models, but I'd like to
    make it the standard across GPT-5 and codex models as well.
    
    ## Testing
    - [x] Tested locally
    - [x] Integration test
  • Added allow-expect-in-tests / allow-unwrap-in-tests (#2328)
    This PR:
    * Added the clippy.toml to configure allowable expect / unwrap usage in
    tests
    * Removed as many expect/allow lines as possible from tests
    * moved a bunch of allows to expects where possible
    
    Note: in integration tests, non `#[test]` helper functions are not
    covered by this so we had to leave a few lingering `expect(expect_used`
    checks around
  • fix: run python_multiprocessing_lock_works integration test on Mac and Linux (#2318)
    The high-order bit on this PR is that it makes it so `sandbox.rs` tests
    both Mac and Linux, as we introduce a general
    `spawn_command_under_sandbox()` function with platform-specific
    implementations for testing.
    
    An important, and interesting, discovery in porting the test to Linux is
    that (for reasons cited in the code comments), `/dev/shm` has to be
    added to `writable_roots` on Linux in order for `multiprocessing.Lock`
    to work there. Granting write access to `/dev/shm` comes with some
    degree of risk, so we do not make this the default for Codex CLI.
    
    Piggybacking on top of #2317, this moves the
    `python_multiprocessing_lock_works` test yet again, moving
    `codex-rs/core/tests/sandbox.rs` to `codex-rs/exec/tests/sandbox.rs`
    because in `codex-rs/exec/tests` we can use `cargo_bin()` like so:
    
    ```
    let codex_linux_sandbox_exe = assert_cmd::cargo::cargo_bin("codex-exec");
    ```
    
    which is necessary so we can use `codex_linux_sandbox_exe` and therefore
    `spawn_command_under_linux_sandbox` in an integration test.
    
    This also moves `spawn_command_under_linux_sandbox()` out of `exec.rs`
    and into `landlock.rs`, which makes things more consistent with
    `seatbelt.rs` in `codex-core`.
    
    For reference, https://github.com/openai/codex/pull/1808 is the PR that
    made the change to Seatbelt to get this test to pass on Mac.
  • fix: move general sandbox tests to codex-rs/core/tests/sandbox.rs (#2317)
    Previous to this PR, `codex-rs/core/tests/sandbox.rs` contained
    integration tests that were specific to Seatbelt. This PR moves those
    tests to `codex-rs/core/src/seatbelt.rs` and designates
    `codex-rs/core/tests/sandbox.rs` to be used as the home for
    cross-platform (well, Mac and Linux...) sandbox tests.
    
    To start, this migrates
    `python_multiprocessing_lock_works_under_seatbelt()` from #1823 to the
    new `sandbox.rs` because this is the type of thing that should work on
    both Mac _and_ Linux, though I still need to do some work to clean up
    the test so it works on both platforms.
  • test(core): add seatbelt sem lock tests (#1823)
    ## Summary
    - add a unit test to ensure the macOS seatbelt policy allows POSIX
    semaphores
    - add a macOS-only test that runs a Python multiprocessing Lock under
    Seatbelt
    
    ## Testing
    - `cargo test -p codex_core seatbelt_base_policy_allows_ipc_posix_sem
    --no-fail-fast` (failed: failed to download from
    `https://static.crates.io/crates/tokio-stream/0.1.17/download`)
    - `cargo test -p codex_core seatbelt_base_policy_allows_ipc_posix_sem
    --no-fail-fast --offline` (failed: attempting to make an HTTP request,
    but --offline was specified)
    - `cargo test --all-features --no-fail-fast --offline` (failed:
    attempting to make an HTTP request, but --offline was specified)
    - `just fmt` (failed: command not found: just)
    - `just fix` (failed: command not found: just)
    
    Ran tests locally to confirm it passes on master and failed before my
    previous change
    
    ------
    https://chatgpt.com/codex/tasks/task_i_6890f221e0a4833381cfb53e11499bcc