Commit Graph

96 Commits

  • feat: panic if Constrained<WebSearchMode> does not support Disabled (#11470)
    If this happens, this is a logical error on our part and we should fix
    it.
  • chore: persist turn_id in rollout session and make turn_id uuid based (#11246)
    Problem:
    1. turn id is constructed in-memory;
    2. on resuming threads, turn_id might not be unique;
    3. client cannot no the boundary of a turn from rollout files easily.
    
    This PR does three things:
    1. persist `task_started` and `task_complete` events;
    1. persist `turn_id` in rollout turn events;
    5. generate turn_id as unique uuids instead of incrementing it in
    memory.
    
    This helps us resolve the issue of clients wanting to have unique turn
    ids for resuming a thread, and knowing the boundry of each turn in
    rollout files.
    
    example debug logs
    ```
    2026-02-11T00:32:10.746876Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=8 turn=Turn { id: "019c4a07-d809-74c3-bc4b-fd9618487b4b", items: [UserMessage { id: "item-24", content: [Text { text: "hi", text_elements: [] }] }, AgentMessage { id: "item-25", text: "Hi. I’m in the workspace with your current changes loaded and ready. Send the next task and I’ll execute it end-to-end." }], status: Completed, error: None }
    2026-02-11T00:32:10.746888Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=9 turn=Turn { id: "019c4a18-1004-76c0-a0fb-a77610f6a9b8", items: [UserMessage { id: "item-26", content: [Text { text: "hello", text_elements: [] }] }, AgentMessage { id: "item-27", text: "Hello. Ready for the next change in `codex-rs`; I can continue from the current in-progress diff or start a new task." }], status: Completed, error: None }
    2026-02-11T00:32:10.746899Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=10 turn=Turn { id: "019c4a19-41f0-7db0-ad78-74f1503baeb8", items: [UserMessage { id: "item-28", content: [Text { text: "hello", text_elements: [] }] }, AgentMessage { id: "item-29", text: "Hello. Send the specific change you want in `codex-rs`, and I’ll implement it and run the required checks." }], status: Completed, error: None }
    ```
    
    backward compatibility:
    if you try to resume an old session without task_started and
    task_complete event populated, the following happens:
    - If you resume and do nothing: those reconstructed historical IDs can
    differ next time you resume.
    - If you resume and send a new turn: the new turn gets a fresh UUID from
    live submission flow and is persisted, so that new turn’s ID is stable
    on later resumes.
    I think this behavior is fine, because we only care about deterministic
    turn id once a turn is triggered.
  • Prefer websocket transport when model opts in (#11386)
    Summary
    - add a `prefer_websockets` field to `ModelInfo`, defaulting to `false`
    in all fixtures and constructors
    - wire the new flag into websocket selection so models that opt in
    always use websocket transport even when the feature gate is off
    
    Testing
    - Not run (not requested)
  • feat: retain NetworkProxy, when appropriate (#11207)
    As of this PR, `SessionServices` retains a
    `Option<StartedNetworkProxy>`, if appropriate.
    
    Now the `network` field on `Config` is `Option<NetworkProxySpec>`
    instead of `Option<NetworkProxy>`.
    
    Over in `Session::new()`, we invoke `NetworkProxySpec::start_proxy()` to
    create the `StartedNetworkProxy`, which is a new struct that retains the
    `NetworkProxy` as well as the `NetworkProxyHandle`. (Note that `Drop` is
    implemented for `NetworkProxyHandle` to ensure the proxies are shutdown
    when it is dropped.)
    
    The `NetworkProxy` from the `StartedNetworkProxy` is threaded through to
    the appropriate places.
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/11207).
    * #11285
    * __->__ #11207
  • feat(sandbox): enforce proxy-aware network routing in sandbox (#11113)
    ## Summary
    - expand proxy env injection to cover common tool env vars
    (`HTTP_PROXY`/`HTTPS_PROXY`/`ALL_PROXY`/`NO_PROXY` families +
    tool-specific variants)
    - harden macOS Seatbelt network policy generation to route through
    inferred loopback proxy endpoints and fail closed when proxy env is
    malformed
    - thread proxy-aware Linux sandbox flags and add minimal bwrap netns
    isolation hook for restricted non-proxy runs
    - add/refresh tests for proxy env wiring, Seatbelt policy generation,
    and Linux sandbox argument wiring
  • feat: tie shell snapshot to cwd (#11231)
    Fix for this: https://github.com/openai/codex/issues/11223
    
    Basically we tie the shell snapshot to a `cwd` to handle `cwd`-based env
    setups
  • feat: search_tool (#10657)
    **Why We Did This**
    - The goal is to reduce MCP tool context pollution by not exposing the
    full MCP tool list up front
    - It forces an explicit discovery step (`search_tool_bm25`) so the model
    narrows tool scope before making MCP calls, which helps relevance and
    lowers prompt/tool clutter.
    
    **What It Changed**
    - Added a new experimental feature flag `search_tool` in
    `core/src/features.rs:90` and `core/src/features.rs:430`.
    - Added config/schema support for that flag in
    `core/config.schema.json:214` and `core/config.schema.json:1235`.
    - Added BM25 dependency (`bm25`) in `Cargo.toml:129` and
    `core/Cargo.toml:23`.
    - Added new tool handler `search_tool_bm25` in
    `core/src/tools/handlers/search_tool_bm25.rs:18`.
    - Registered the handler and tool spec in
    `core/src/tools/handlers/mod.rs:11` and `core/src/tools/spec.rs:780` and
    `core/src/tools/spec.rs:1344`.
    - Extended `ToolsConfig` to carry `search_tool` enablement in
    `core/src/tools/spec.rs:32` and `core/src/tools/spec.rs:56`.
    - Injected dedicated developer instructions for tool-discovery workflow
    in `core/src/codex.rs:483` and `core/src/codex.rs:1976`, using
    `core/templates/search_tool/developer_instructions.md:1`.
    - Added session state to store one-shot selected MCP tools in
    `core/src/state/session.rs:27` and `core/src/state/session.rs:131`.
    - Added filtering so when feature is enabled, only selected MCP tools
    are exposed on the next request (then consumed) in
    `core/src/codex.rs:3800` and `core/src/codex.rs:3843`.
    - Added E2E suite coverage for
    enablement/instructions/hide-until-search/one-turn-selection in
    `core/tests/suite/search_tool.rs:72`,
    `core/tests/suite/search_tool.rs:109`,
    `core/tests/suite/search_tool.rs:147`, and
    `core/tests/suite/search_tool.rs:218`.
    - Refactored test helper utilities to support config-driven tool
    collection in `core/tests/suite/tools.rs:281`.
    
    **Net Behavioral Effect**
    - With `search_tool` **off**: existing MCP behavior (tools exposed
    normally).
    - With `search_tool` **on**: MCP tools start hidden, model must call
    `search_tool_bm25`, and only returned `selected_tools` are available for
    the next model call.
  • Move warmup to the task level (#11216)
    Instead of storing a special connection on the client level make the
    regular task responsible for establishing a normal client session and
    open a connection on it.
    
    Then when the turn is started we pass in a pre-established session.
  • feat: do not close unified exec processes across turns (#10799)
    With this PR we do not close the unified exec processes (i.e. background
    terminals) at the end of a turn unless:
    * The user interrupt the turn
    * The user decide to clean the processes through `app-server` or
    `/clean`
    
    I made sure that `codex exec` correctly kill all the processes
  • feat: include NetworkConfig through ExecParams (#11105)
    This PR adds the following field to `Config`:
    
    ```rust
    pub network: Option<NetworkProxy>,
    ```
    
    Though for the moment, it will always be initialized as `None` (this
    will be addressed in a subsequent PR).
    
    This PR does the work to thread `network` through to `execute_exec_env()`, `process_exec_tool_call()`, and `UnifiedExecRuntime.run()` to ensure it is available whenever we span a process.
  • Defer persistence of rollout file (#11028)
    - Defer rollout persistence for fresh threads (`InitialHistory::New`):
    keep rollout events in memory and only materialize rollout file + state
    DB row on first `EventMsg::UserMessage`.
    - Keep precomputed rollout path available before materialization.
    - Change `thread/start` to build thread response from live config
    snapshot and optional precomputed path.
    - Improve pre-materialization behavior in app-server/TUI: clearer
    invalid-request errors for file-backed ops and a friendlier `/fork` “not
    ready yet” UX.
    - Update tests to match deferred semantics across
    start/read/archive/unarchive/fork/resume/review flows.
    - Improved resilience of user_shell test, which should be unrelated to
    this change but must be affected by timing changes
    
    For Reviewers:
    * The primary change is in recorder.rs
    * Most of the other changes were to fix up broken assumptions in
    existing tests
    
    Testing:
    * Manually tested CLI
    * Exercised app server paths by manually running IDE Extension with
    rebuilt CLI binary
    * Only user-visible change is that `/fork` in TUI generates visible
    error if used prior to first turn
  • feat: add support for allowed_web_search_modes in requirements.toml (#10964)
    This PR makes it possible to disable live web search via an enterprise
    config even if the user is running in `--yolo` mode (though cached web
    search will still be available). To do this, create
    `/etc/codex/requirements.toml` as follows:
    
    ```toml
    # "live" is not allowed; "disabled" is allowed even though not listed explicitly.
    allowed_web_search_modes = ["cached"]
    ```
    
    Or set `requirements_toml_base64` MDM as explained on
    https://developers.openai.com/codex/security/#locations.
    
    ### Why
    - Enforce admin/MDM/`requirements.toml` constraints on web-search
    behavior, independent of user config and per-turn sandbox defaults.
    - Ensure per-turn config resolution and review-mode overrides never
    crash when constraints are present.
    
    ### What
    - Add `allowed_web_search_modes` to requirements parsing and surface it
    in app-server v2 `ConfigRequirements` (`allowedWebSearchModes`), with
    fixtures updated.
    - Define a requirements allowlist type (`WebSearchModeRequirement`) and
    normalize semantics:
      - `disabled` is always implicitly allowed (even if not listed).
      - An empty list is treated as `["disabled"]`.
    - Make `Config.web_search_mode` a `Constrained<WebSearchMode>` and apply
    requirements via `ConstrainedWithSource<WebSearchMode>`.
    - Update per-turn resolution (`resolve_web_search_mode_for_turn`) to:
    - Prefer `Live → Cached → Disabled` when
    `SandboxPolicy::DangerFullAccess` is active (subject to requirements),
    unless the user preference is explicitly `Disabled`.
    - Otherwise, honor the user’s preferred mode, falling back to an allowed
    mode when necessary.
    - Update TUI `/debug-config` and app-server mapping to display
    normalized `allowed_web_search_modes` (including implicit `disabled`).
    - Fix web-search integration tests to assert cached behavior under
    `SandboxPolicy::ReadOnly` (since `DangerFullAccess` legitimately prefers
    `live` when allowed).
  • Treat compaction failure as failure state (#10927)
    - Return compaction errors from local and remote compaction flows.\n-
    Stop turns/tasks when auto-compaction fails instead of continuing
    execution.
  • Allow user shell commands to run alongside active turns (#10513)
    Summary
    - refactor user shell command execution into a shared helper and add
    modes for standalone vs active-turn execution
    - run user shell commands asynchronously when a turn is already active
    so they don’t replace or abort the current turn
    - extend the tests to cover the new behavior and add the generated Codex
    environment manifest
    
    Testing
    - Not run (not requested)
  • Stop client from being state carrier (#10595)
    I'd like to make client session wide. This requires shedding all random
    state it has to carry.
  • Inject CODEX_THREAD_ID into the terminal environment (#10096)
    Inject CODEX_THREAD_ID (when applicable) into the terminal environment
    so that the agent (and skills) can refer to the current thread / session
    ID.
    
    Discussion:
    https://openai.slack.com/archives/C095U48JNL9/p1769542492067109
  • chore: add phase to message responseitem (#10455)
    ### What
    
    add wiring for `phase` field on `ResponseItem::Message` to lay
    groundwork for differentiating model preambles and final messages.
    currently optional.
    
    follows pattern in #9698.
    
    updated schemas with `just write-app-server-schema` so we can see type
    changes.
    
    ### Tests
    Updated existing tests for SSE parsing and hydrating from history
  • Plan mode: stream proposed plans, emit plan items, and render in TUI (#9786)
    ## Summary
    - Stream proposed plans in Plan Mode using `<proposed_plan>` tags parsed
    in core, emitting plan deltas plus a plan `ThreadItem`, while stripping
    tags from normal assistant output.
    - Persist plan items and rebuild them on resume so proposed plans show
    in thread history.
    - Wire plan items/deltas through app-server protocol v2 and render a
    dedicated proposed-plan view in the TUI, including the “Implement this
    plan?” prompt only when a plan item is present.
    
    ## Changes
    
    ### Core (`codex-rs/core`)
    - Added a generic, line-based tag parser that buffers each line until it
    can disprove a tag prefix; implements auto-close on `finish()` for
    unterminated tags. `codex-rs/core/src/tagged_block_parser.rs`
    - Refactored proposed plan parsing to wrap the generic parser.
    `codex-rs/core/src/proposed_plan_parser.rs`
    - In plan mode, stream assistant deltas as:
      - **Normal text** → `AgentMessageContentDelta`
      - **Plan text** → `PlanDelta` + `TurnItem::Plan` start/completion  
      (`codex-rs/core/src/codex.rs`)
    - Final plan item content is derived from the completed assistant
    message (authoritative), not necessarily the concatenated deltas.
    - Strips `<proposed_plan>` blocks from assistant text in plan mode so
    tags don’t appear in normal messages.
    (`codex-rs/core/src/stream_events_utils.rs`)
    - Persist `ItemCompleted` events only for plan items for rollout replay.
    (`codex-rs/core/src/rollout/policy.rs`)
    - Guard `update_plan` tool in Plan Mode with a clear error message.
    (`codex-rs/core/src/tools/handlers/plan.rs`)
    - Updated Plan Mode prompt to:  
      - keep `<proposed_plan>` out of non-final reasoning/preambles  
      - require exact tag formatting  
      - allow only one `<proposed_plan>` block per turn  
      (`codex-rs/core/templates/collaboration_mode/plan.md`)
    
    ### Protocol / App-server protocol
    - Added `TurnItem::Plan` and `PlanDeltaEvent` to core protocol items.
    (`codex-rs/protocol/src/items.rs`, `codex-rs/protocol/src/protocol.rs`)
    - Added v2 `ThreadItem::Plan` and `PlanDeltaNotification` with
    EXPERIMENTAL markers and note that deltas may not match the final plan
    item. (`codex-rs/app-server-protocol/src/protocol/v2.rs`)
    - Added plan delta route in app-server protocol common mapping.
    (`codex-rs/app-server-protocol/src/protocol/common.rs`)
    - Rebuild plan items from persisted `ItemCompleted` events on resume.
    (`codex-rs/app-server-protocol/src/protocol/thread_history.rs`)
    
    ### App-server
    - Forward plan deltas to v2 clients and map core plan items to v2 plan
    items. (`codex-rs/app-server/src/bespoke_event_handling.rs`,
    `codex-rs/app-server/src/codex_message_processor.rs`)
    - Added v2 plan item tests.
    (`codex-rs/app-server/tests/suite/v2/plan_item.rs`)
    
    ### TUI
    - Added a dedicated proposed plan history cell with special background
    and padding, and moved “• Proposed Plan” outside the highlighted block.
    (`codex-rs/tui/src/history_cell.rs`, `codex-rs/tui/src/style.rs`)
    - Only show “Implement this plan?” when a plan item exists.
    (`codex-rs/tui/src/chatwidget.rs`,
    `codex-rs/tui/src/chatwidget/tests.rs`)
    
    <img width="831" height="847" alt="Screenshot 2026-01-29 at 7 06 24 PM"
    src="https://github.com/user-attachments/assets/69794c8c-f96b-4d36-92ef-c1f5c3a8f286"
    />
    
    ### Docs / Misc
    - Updated protocol docs to mention plan deltas.
    (`codex-rs/docs/protocol_v1.md`)
    - Minor plumbing updates in exec/debug clients to tolerate plan deltas.
    (`codex-rs/debug-client/src/reader.rs`, `codex-rs/exec/...`)
    
    ## Tests
    - Added core integration tests:
      - Plan mode strips plan from agent messages.
      - Missing `</proposed_plan>` closes at end-of-message.  
      (`codex-rs/core/tests/suite/items.rs`)
    - Added unit tests for generic tag parser (prefix buffering, non-tag
    lines, auto-close). (`codex-rs/core/src/tagged_block_parser.rs`)
    - Existing app-server plan item tests in v2.
    (`codex-rs/app-server/tests/suite/v2/plan_item.rs`)
    
    ## Notes / Behavior
    - Plan output no longer appears in standard assistant text in Plan Mode;
    it streams via `PlanDelta` and completes as a `TurnItem::Plan`.
    - The final plan item content is authoritative and may diverge from
    streamed deltas (documented as experimental).
    - Reasoning summaries are not filtered; prompt instructs the model not
    to include `<proposed_plan>` outside the final plan message.
    
    ## Codex Author
    `codex fork 019bec2d-b09d-7450-b292-d7bcdddcdbfb`
  • feat: reduce span exposition (#10171)
    This only avoids the creation of duplicates spans
  • fix: enable per-turn updates to web search mode (#10040)
    web_search can now be updated per-turn, for things like changes to
    sandbox policy.
    
    `SandboxPolicy::DangerFullAccess` now sets web_search to `live`, and the
    default is still `cached`.
    
    Added integration tests.
  • remove sandbox globals. (#9797)
    Threads sandbox updates through OverrideTurnContext for active turn
    Passes computed sandbox type into safety/exec
  • fix(app-server, core): defer initial context write to rollout file until first turn (#9950)
    ### Overview
    Currently calling `thread/resume` will always bump the thread's
    `updated_at` timestamp. This PR makes it the `updated_at` timestamp
    changes only if a turn is triggered.
    
    ### Additonal context
    What we typically do on resuming a thread is **always** writing “initial
    context” to the rollout file immediately. This initial context includes:
    - Developer instructions derived from sandbox/approval policy + cwd
    - Optional developer instructions (if provided)
    - Optional collaboration-mode instructions
    - Optional user instructions (if provided)
    - Environment context (cwd, shell, etc.)
    
    This PR defers writing the “initial context” to the rollout file until
    the first `turn/start`, so we don't inadvertently bump the thread's
    `updated_at` timestamp until a turn is actually triggered.
    
    This works even though both `thread/resume` and `turn/start` accept
    overrides (such as `model`, `cwd`, etc.) because the initial context is
    seeded from the effective `TurnContext` in memory, computed at
    `turn/start` time, after both sets of overrides have been applied.
    
    **NOTE**: This is a very short-lived solution until we introduce sqlite.
    Then we can remove this.
  • make cached web_search client-side default (#9974)
    [Experiment](https://console.statsig.com/50aWbk2p4R76rNX9lN5VUw/experiments/codex_web_search_rollout/summary)
    for default cached `web_search` completed; cached chosen as default.
    
    Update client to reflect that.
  • Support end_turn flag (#9698)
    Experimental flag that signals the end of the turn.
  • fix: prevent repeating interrupted turns (#9043)
    ## What
    Record a model-visible `<turn_aborted>` marker in history when a turn is
    interrupted, and treat it as a session prefix.
    
    ## Why
    When a turn is interrupted, Codex emits `TurnAborted` but previously did
    not persist anything model-visible in the conversation history. On the
    next user turn, the model can’t tell the previous work was aborted and
    may resume/repeat earlier actions (including duplicated side effects
    like re-opening PRs).
    
    Fixes: https://github.com/openai/codex/issues/9042
    
    ## How
    On `TurnAbortReason::Interrupted`, append a hidden user message
    containing a `<turn_aborted>…</turn_aborted>` marker and flush.
    Treat `<turn_aborted>` like `<environment_context>` for session-prefix
    filtering.
    Add a regression test to ensure follow-up turns don’t repeat side
    effects from an aborted turn.
    
    ## Testing
    `just fmt`
    `just fix -p codex-core`
    `cargo test -p codex-core -- --test-threads=1`
    `cargo test --all-features -- --test-threads=1`
    
    ---------
    
    Co-authored-by: Skylar Graika <sgraika127@gmail.com>
    Co-authored-by: jif-oai <jif@openai.com>
    Co-authored-by: Eric Traut <etraut@openai.com>
  • Act on reasoning-included per turn (#9402)
    - Reset reasoning-included flag each turn and update compaction test
  • Fix invalid input error on Azure endpoint (#9387)
    Users of Azure endpoints are reporting that when they use `/review`,
    they sometimes see an error "Invalid 'input[3].id". I suspect this is
    specific to the Azure implementation of the `responses` API. The Azure
    team generally copies the OpenAI code for this endpoint, but they do
    have minor differences and sometimes lag in rolling out bug fixes or
    updates.
    
    The error appears to be triggered because the `/review` implementation
    is using a user ID with a colon in it.
    
    Addresses #9360
  • feat: run user commands under user snapshot (#9357)
    The initial goal is for user snapshots to have access to aliases etc
  • [search] allow explicitly disabling web search (#9249)
    moving `web_search` rollout serverside, so need a way to explicitly
    disable search + signal eligibility from the client.
    
    - Add `x‑oai‑web‑search‑eligible` header that signifies whether the
    request can have web search.
    - Only attach the `web_search` tool when the resolved `WebSearchMode` is
    `Live` or `Cached`.
  • add WebSearchMode enum (#9216)
    ### What
    Add `WebSearchMode` enum (disabled, cached live, defaults to cached) to
    config + V2 protocol. This enum takes precedence over legacy flags:
    `web_search_cached`, `web_search_request`, and `tools.web_search`.
    
    Keep `--search` as live.
    
    ### Tests
    Added tests
  • Use current model for review (#9179)
    Instead of having a hard-coded default review model, use the current
    model for running `/review` unless one is specified in the config.
    
    Also inherit current reasoning effort
  • fix: include project instructions in /review subagent (#8899)
    Include project-level AGENTS.md and skills in /review sessions so the
    review sub-agent uses the same instruction pipeline as standard runs,
    keeping reviewer context aligned with normal sessions.
  • feat: metrics capabilities (#8318)
    Add metrics capabilities to Codex. The `README.md` is up to date.
    
    This will not be merged with the metrics before this PR of course:
    https://github.com/openai/codex/pull/8350
  • chore: unify conversation with thread name (#8830)
    Done and verified by Codex + refactor feature of RustRover
  • Rename OpenAI models to models manager (#8346)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.