Commit Graph

269 Commits

  • [codex] Remove redundant SQLite dynamic tool storage (#24819)
    ## Why
    
    Dynamic tools are defined at thread start and already stored in rollout
    `SessionMeta`, which restores resumed and forked sessions. Persisting
    the same tools through SQLite creates a second runtime persistence path
    that is unnecessary prework for the explicit namespace refactor.
    
    ## What changed
    
    - Restore missing thread-start dynamic tools directly from rollout
    history, including when SQLite is enabled.
    - Remove SQLite dynamic-tool reads, writes, backfill, and thread
    metadata patch plumbing.
    - Add SQLite-enabled resume integration coverage that verifies a
    rollout-defined dynamic tool is still sent after resume.
    
    ## Compatibility
    
    The existing `thread_dynamic_tools` table is intentionally not dropped
    even though it's now unused. Older Codex binaries are allowed to open
    databases migrated by newer binaries and still reference this table;
    dropping it would break that mixed-version path. See
    [here](https://github.com/openai/codex/blob/main/codex-rs/state/src/migrations.rs#L10-L11).
    
    ## Verification
    
    - `just test -p codex-state -p codex-rollout -p codex-thread-store`
    - `just test -p codex-core --test all
    resume_restores_dynamic_tools_from_rollout_with_sqlite_enabled`
  • fix(linux-sandbox): preserve shell cleanup on interruption (#22729)
    ## Why
    Interrupted `shell_command` calls can race with the outer tool-dispatch
    cancellation path. When that happens, the runtime future may be dropped
    before the spawned process gets a chance to run `SIGTERM` cleanup. For
    bwrapd-backed Linux sandbox commands, that can leave synthetic
    protected-path mount bookkeeping such as `.git/.codex` registrations
    under `/tmp` behind after a TUI interruption.
    
    The relevant cancellation points are the outer dispatch race in
    [`core/src/tools/parallel.rs`](https://github.com/openai/codex/blob/bd184ba84703cc924921ed883f0cf17d3dba60ff/codex-rs/core/src/tools/parallel.rs#L91-L132)
    and the process shutdown logic in
    [`core/src/exec.rs`](https://github.com/openai/codex/blob/bd184ba84703cc924921ed883f0cf17d3dba60ff/codex-rs/core/src/exec.rs#L1367-L1393).
    
    ## What changed
    - Keep `shell_command` dispatch alive long enough for the runtime to
    finish cancellation cleanup instead of immediately returning the
    synthetic aborted response.
    - Fold shell-turn cancellation into the existing `ExecExpiration` path
    in
    [`core/src/tools/runtimes/shell.rs`](https://github.com/openai/codex/blob/bd184ba84703cc924921ed883f0cf17d3dba60ff/codex-rs/core/src/tools/runtimes/shell.rs#L267-L274),
    so cancellation and timeout behavior stay centralized.
    - On cancellation, send `SIGTERM` first, wait briefly for cleanup to
    run, then hard-kill any remaining descendants in the original process
    group.
    - Treat `ESRCH` as an already-gone process-group cleanup case in
    `codex-utils-pty`, which keeps best-effort teardown from surfacing a
    stale-process race as an error.
    
    ## Verification
    - `cargo test -p codex-core cancellation`
    - Added regression coverage for:
      - `shell_tool_cancellation_waits_for_runtime_cleanup`
      - `process_exec_tool_call_cancellation_allows_sigterm_cleanup`
  • [codex] add compaction metadata to turn headers (#24368)
    ## Summary
    - Add `request_kind` values for foreground turn, startup prewarm,
    compaction, and detached memory model requests.
    - Attach compaction dispatch metadata to local Responses, legacy
    `/v1/responses/compact`, and remote v2 compact requests.
    - Add the existing logical context-window identifier as `window_id` on
    turn-owned model request metadata.
    - Keep identity fields optional for detached memory requests, while
    still emitting `request_kind="memory"` in non-git/no-sandbox workspaces.
    
    ## Root Cause
    `x-codex-turn-metadata` has more than one producer. Foreground turns and
    compaction requests own a real turn and should carry that turn identity.
    Detached memory stage-one requests do not own a foreground turn, so
    absent identity fields are valid rather than missing data. Startup
    websocket prewarm is also a model request, but it has `generate=false`
    and must not be counted as a foreground turn.
    
    `thread_source` or session source identifies where a thread came from
    (for example review, guardian, or another subagent). `request_kind`
    identifies what the current outbound model request is doing (`turn`,
    `prewarm`, `compaction`, or `memory`). A review or guardian thread can
    issue either a normal turn request or a compaction request, so source
    cannot replace request kind.
    
    ## Behavior / Impact
    - Ordinary foreground requests send `request_kind="turn"`, their real
    identity fields, and `window_id="<thread_id>:<window_generation>"`.
    - Startup websocket warmup requests send `request_kind="prewarm"` so
    they are not counted as foreground turns.
    - Compaction requests send `request_kind="compaction"`, their real
    owning turn identity, the existing `window_id`, and
    `compaction.{trigger,reason,implementation,phase,strategy}`.
    - Detached memory stage-one requests send `request_kind="memory"`
    without `session_id`, `thread_id`, `turn_id`, or `window_id`; when no
    workspace metadata exists, the kind-only header is still emitted.
    - `session_id`, `thread_id`, `turn_id`, and `window_id` remain optional
    in the header schema because detached memory requests do not own a
    foreground turn or context window.
    - `window_id` is not a new ID system: it is copied from the already-sent
    `x-codex-window-id` / WS client metadata value at model-request dispatch
    time.
    - Existing `x-codex-window-id` HTTP/WS emission, value format,
    generation advancement, resume behavior, and fork reset behavior are
    unchanged.
    - `request_kind`, `window_id`, and upstream turn-owned identity fields
    remain schema-owned; input `responsesapi_client_metadata` cannot replace
    their canonical values.
    - No table, DAG, export, app-server API, or MCP `_meta` schema changes
    are included.
    
    A compaction attempt stopped by a pre-compact hook issues no model
    request and therefore has no request header; its outcome remains in
    analytics events. Status, error, duration, and token deltas also remain
    analytics fields rather than request-header fields.
    
    Future detached-memory attribution using a real initiating turn ID as
    `trigger_turn_id` is intentionally not part of this PR.
    
    ## Sync With Main
    - Final pushed head `716342e79` is rebased onto `origin/main@0d37db4b2`.
    - The metadata conflict came from upstream `#24160`, which added
    `forked_from_thread_id` on the same `turn_metadata` surface. Resolution
    preserves that field and its protection from client metadata override
    alongside this PR's request-kind, compaction, and window-id fields.
    - While resolving the overlapping commits, I removed an accidental
    recursive model-request overlay and a duplicate detached-memory header
    builder before completing the rebase.
    
    ## Latency / User Experience Boundary
    - Foreground turns perform no new filesystem, git, or network work. New
    fields are inserted into metadata already serialized for outgoing
    requests.
    - Compaction issues the same model/HTTP requests with the same prompt,
    model, service tier, and sampling settings; only metadata bytes change.
    - Startup prewarm already sent metadata; it is now correctly classified
    as `prewarm`.
    - Non-git detached memory now sends a small kind-only metadata header
    rather than no header.
    - This client diff adds no user-visible latency mechanism beyond
    negligible serialization and header bytes on already-existing requests.
    
    ## Validation
    On conflict-resolved head `1d35c2cfb` based on `origin/main@487521733`:
    - `just fmt` (passed)
    - `just fix -p codex-core` (passed)
    - `git diff --check origin/main...HEAD` (passed)
    - `just test -p codex-core -E 'test(turn_metadata) |
    test(websocket_first_turn_uses_startup_prewarm_and_create) |
    test(responses_stream_includes_turn_metadata_header_for_git_workspace_e2e)
    |
    test(responses_websocket_forwards_turn_metadata_on_initial_and_incremental_create)
    | test(remote_compact_v2_retries_failures_with_stream_retry_budget) |
    test(window_id_advances_after_compact_persists_on_resume_and_resets_on_fork)'`
    (`23 passed`; `bench-smoke` passed)
    - `just test -p codex-app-server -E
    'test(turn_start_forwards_client_metadata_to_responses_request_v2) |
    test(turn_start_forwards_client_metadata_to_responses_websocket_request_body_v2)
    | test(auto_compaction_remote_emits_started_and_completed_items)'` (`3
    passed`; `bench-smoke` passed)
    - `just test -p codex-memories-write` (`29 passed`; `bench-smoke`
    passed)
  • fix(core): instrument stalled tool-listing handoff (#24667)
    ## Why
    
    When a turn needs a follow-up request after tool output is recorded,
    Codex can still appear stuck in `Thinking` before the next `/responses`
    request is opened. The existing local trace showed the last completed
    response and the absence of a new backend request, but it did not show
    whether the stall was in tool-router preparation or later request setup.
    
    Issue: N/A (internal incident investigation)
    
    ## What Changed
    
    Added trace spans around the pre-stream tool-router handoff in
    `core/src/session/turn.rs`, including the `built_tools` phase and the
    MCP manager read lock.
    
    Added per-server MCP tool-listing spans and trace breadcrumbs in
    `codex-mcp/src/connection_manager.rs` with startup snapshot /
    startup-complete state so a pending MCP client is visible in feedback
    logs instead of looking like a silent hang.
    
    ## Verification
    
    - `just fmt`
    - `just test -p codex-mcp`
    - `just test -p codex-core` (prior full rerun fails in this workspace on
    unrelated integration tests: code-mode output length expectations, one
    shell timeout formatting assertion, and shell snapshot timeouts; latest
    review-fix rerun compiled and passed 1160 tests before I stopped the
    abnormally slow unrelated suite)
  • [codex-analytics] add grouped session id to runtime events (#24655)
    ## Why
    - Runtime analytics events report `thread_id`, which identifies the
    individual thread emitting an event
    - They don't report `session_id`, which identifies the shared session
    for a root thread and its subagent threads
    - Emitting both identifiers allows analytics to group related activity
    
    ## What Changed
    - Adds `session_id` to relevant analytics events (thread_initalized,
    turn, turn_steer, compaction, guardian_review)
    - Tracks each thread's session ID in the analytics reducer so subsequent
    thread scoped events emit the same value
    - Carries the shared session ID through subagent initialization
    
    ## Verification
    - `just test -p codex-analytics` validates event payloads and subagent
    session grouping.
    - Focused `codex-app-server` tests validate session IDs for thread,
    turn, and steer events.
    - Focused `codex-core` tests validate root and subagent session ID
    propagation.
  • Add forked_from_thread_id turn metadata (#24160)
    ## Why
    
    When Codex calls responsesapi, we currently send `session_id`,
    `thread_id`, and `turn_id` among other things as
    `client_metadata["x-codex-turn-metadata"]`. This PR adds
    `forked_from_thread_id` which helps explain the "lineage" of a forked
    thread.
    
    ## What's changed
    
    - Track the immediate history source copied into a forked thread through
    thread/session creation, including subagent and review turn metadata
    paths.
    - Include `forked_from_thread_id` in Codex turn metadata while
    preventing turn-scoped Responses API client metadata from overwriting
    Codex-owned lineage fields.
    - Add coverage for fork lineage in turn metadata and the app-server
    Responses API request path.
  • Add experimental turn additional context (#24154)
    ## Summary
    
    Adds experimental `additionalContext` support to `turn/start` and
    `turn/steer` so clients can provide ephemeral external context, such as
    browser or automation state, without turning that plumbing into a
    visible user prompt or triggering user-prompt lifecycle behavior.
    
    ## API Shape
    
    The parameter shape is:
    
    ```ts
    additionalContext?: Record<string, {
      value: string
      kind: "untrusted" | "application"
    }> | null
    ```
    
    Example:
    
    ```json
    {
      "additionalContext": {
        "browser_info": {
          "value": "Active tab is CI failures.",
          "kind": "untrusted"
        },
        "automation_info": {
          "value": "CI rerun is in progress.",
          "kind": "application"
        }
      }
    }
    ```
    
    The keys are opaque and caller-defined.
    
    ## Context Injection
    
    When provided, accepted entries are inserted into model context as
    hidden contextual message items, not as visible thread user-message
    items.
    
    `kind: "untrusted"` entries are inserted with role `user`:
    
    ```text
    <external_${key}>${value}</external_${key}>
    ```
    
    `kind: "application"` entries are inserted with role `developer`:
    
    ```text
    <${key}>${value}</${key}>
    ```
    
    Values are not escaped. Each value is truncated to 1k approximate tokens
    before wrapping.
    
    For `turn/start`, accepted additional context is inserted before normal
    user input. For `turn/steer`, additional context is merged only when the
    steer includes non-empty user input; context-only steers still reject as
    empty input.
    
    ## Dedupe Strategy
    
    `AdditionalContextStore` lives on session state and stores the latest
    complete additional-context map.
    
    Each `turn/start` or non-empty `turn/steer` treats its
    `additionalContext` as the current complete set of values. Entries are
    injected only when the key is new or the exact entry for that key
    changed, including `value` or `kind`. After merging, the store is
    replaced with the provided map, so omitted keys are removed from the
    retained set and can be injected again later if reintroduced.
    
    Omitting `additionalContext`, passing `null`, or passing an empty object
    resets the store to empty and injects nothing.
    
    ## What Changed
    
    - Threads experimental v2 `additionalContext` through app-server into
    core turn start and steer handling.
    - Adds separate contextual fragment types for untrusted user-role
    context and application developer-role context.
    - Uses pending response input items so additional context can be
    combined with normal user input without treating it as prompt text.
    - Adds integration coverage for start/steer flow, role routing,
    dedupe/reset behavior, deletion/re-add behavior, hook-blocked input
    behavior, empty context-only steer rejection, external-fragment marker
    matching, and truncation.
  • feat: add manual and remote_v2 tags to compaction metric (#24608)
    ## Why
    `codex.task.compact` only distinguished `local` vs `remote`, which made
    it hard to answer simple counter questions in Statsig. Manual `/compact`
    and automatic compaction were collapsed together, and the legacy remote
    path was also collapsed with `remote_compaction_v2`.
    
    ## What Changed
    - route `codex.task.compact` through a shared helper in
    `core/src/tasks/mod.rs`
    - add a `manual=true|false` tag so manual and automatic compaction can
    be counted separately
    - split the remote tag into `remote` and `remote_v2`
    - emit the metric from the inline auto-compaction path in
    `core/src/session/turn.rs` as well as the manual `CompactTask` path in
    `core/src/tasks/compact.rs`
    - add focused unit coverage for the new tag shapes in
    `core/src/tasks/mod_tests.rs`
    
    ## Verification
    - added unit coverage in `core/src/tasks/mod_tests.rs` covering manual
    `remote_v2` tags and automatic `local` tags
  • Move MCP tool naming mode into manager (#21576)
    ## Why
    
    The `non_prefixed_mcp_tool_names` feature should be applied where MCP
    tools become model-visible, not by remapping names later in core.
    Keeping the decision in `McpConnectionManager` construction makes
    `ToolInfo` the single shaped view that spec building, deferred tool
    search, routing, and unavailable-tool placeholders can consume directly.
    
    This also preserves the existing external behavior while the feature is
    off, and keeps the feature-on behavior for code mode and hooks explicit
    at the manager boundary.
    
    ## What Changed
    
    - Add `McpToolNameMode` to `codex-mcp` and flow it through `McpConfig`
    into `McpConnectionManager::new`.
    - Normalize MCP `ToolInfo` names in the manager using either
    legacy-prefixed namespaces or non-prefixed namespaces; the legacy path
    adds `mcp__` without restoring the old trailing namespace suffix.
    - Remove the core-side MCP name remapping path so specs, tool search,
    session resolution, and unavailable-tool placeholder construction use
    the manager-provided `ToolName` values directly.
    - Keep code mode flattening on the `__` namespace separator.
    - Preserve hook compatibility by giving non-prefixed MCP hook names
    legacy `mcp__...` matcher aliases.
    - Add/adjust integration and unit coverage for non-prefixed code-mode
    behavior, hook matching with the feature on and off, and manager-level
    legacy prefixing.
    
    ## Testing
    
    - `cargo test -p codex-mcp --lib`
    - `cargo test -p codex-core --lib tools::spec::tests -- --nocapture`
    - `cargo test -p codex-core --lib mcp_tools -- --nocapture`
    - `cargo test -p codex-core --lib mcp_tool_exposure -- --nocapture`
    - `cargo test -p codex-core --test all mcp_tool -- --nocapture`
    - `cargo test -p codex-core --test all search_tool -- --nocapture`
    - `cargo test -p codex-core --test all hooks_mcp -- --nocapture`
    - `cargo test -p codex-core --test all
    code_mode_uses_non_prefixed_mcp_tool_names_when_feature_enabled --
    --nocapture`
    - `cargo test -p codex-tools`
    - `cargo test -p codex-features`
  • [codex] Make active turn task singular (#24105)
    ## Why
    
    `ActiveTurn` already runs at most one task: starting a task requires
    that no task is present, and replacement aborts existing work first.
    Representing that state as an `IndexMap` leaves a multi-task shape for a
    single-task invariant and makes each lifecycle lookup operate like a
    collection lookup.
    
    The slot remains optional because goal continuation uses an empty active
    turn as a reservation while deciding whether to start continuation work.
    
    ## What changed
    
    - Replace `ActiveTurn.tasks` with `task: Option<RunningTask>`.
    - Update task abort/completion, session lookup and steering, input-queue
    matching, goal reservation, and network-approval lookup to operate on
    the singular slot.
    - Mutate the singular task slot directly instead of retaining
    collection-era add/remove/take helpers.
    - Record token usage on the completing active task span without a
    regular-task-only opt-in flag.
    
    ## Validation
    
    - `cargo test -p codex-core --lib session::tests::steer_input`
    - `cargo test -p codex-core --lib
    session::tests::abort_empty_active_turn_preserves_pending_input`
    - `cargo test -p codex-core --lib
    session::tests::queued_response_items_for_next_turn_move_into_next_active_turn`
    - `cargo test -p codex-core --lib
    session::tests::active_goal_continuation_runs_again_after_no_tool_turn`
    - `cargo test -p codex-core --lib
    session::tests::abort_regular_task_emits_turn_aborted_only`
    - `cargo test -p codex-core --lib session::input_queue::tests`
  • centralize Responses retry policy (#24131)
    ## Why
    
    #23951 added remote compaction v2 retries, but it left the retry and WS
    -> HTTPS fallback behavior duplicated between normal Responses turns and
    compaction. This follow-up centralizes the common retry handling so
    future changes to fallback, retry delay, retry notifications, and retry
    sleep do not have to be kept in sync across both callsites.
    
    ## What changed
    
    - Added `core/src/responses_retry.rs` with a shared handler for
    retryable Responses stream errors.
    - Reused that handler from normal turn sampling and remote compaction
    v2.
    - Kept each callsite responsible for its retry budget: normal turns
    still use `stream_max_retries`, while compaction v2 still uses
    `min(stream_max_retries, 2)`.
    - Preserved caller-specific behavior around non-retryable errors,
    context-window errors, usage-limit errors, and compact-specific final
    failure logging.
    
    The shared handler now owns:
    
    - WS -> HTTPS fallback warning emission
    - retry delay selection, including server-requested stream retry delay
    - retry logging
    - first-WebSocket-retry notification suppression
    - `Reconnecting... n/max` stream-error notification
    - sleeping before the next retry attempt
    
    ## Verification
    
    - `cargo test -p codex-core remote_compact_v2`
    - `cargo test -p codex-core websocket_fallback`
    - `just fix -p codex-core`
    
    Did not run the full workspace test suite.
    
    ---------
    
    Co-authored-by: jif-oai <jif@openai.com>
  • package: include zsh fork in Codex package (#23756)
    ## Why
    
    The package layout gives Codex a stable place for runtime helpers that
    should travel with the entrypoint. `shell_zsh_fork` still required users
    to configure `zsh_path` manually, even though we already publish
    prebuilt zsh fork artifacts.
    
    This PR builds on #24129 and uses the shared DotSlash artifact fetcher
    to include the zsh fork in Codex packages when a matching target
    artifact exists. Packaged Codex builds can then discover the bundled
    fork automatically; the user/profile `zsh_path` override is removed so
    the feature uses the package-managed artifact instead of a legacy path
    knob.
    
    ## What Changed
    
    - Added `scripts/codex_package/codex-zsh`, a checked-in DotSlash
    manifest for the current macOS arm64 and Linux zsh fork artifacts.
    - Taught `scripts/build_codex_package.py` to fetch the matching zsh fork
    artifact and install it at `codex-resources/zsh/bin/zsh` when available
    for the selected target.
    - Added package layout validation for the optional bundled zsh resource.
    - Added `InstallContext::bundled_zsh_path()` and
    `InstallContext::bundled_zsh_bin_dir()` for package-layout resource
    discovery.
    - Threaded the packaged zsh path through config loading as the runtime
    `zsh_path` for packaged installs, and removed the config/profile/CLI
    override path.
    - Kept the packaged default zsh override typed as `AbsolutePathBuf`
    until the existing runtime `Config::zsh_path` boundary.
    - Updated app-server zsh-fork integration tests to spawn
    `codex-app-server` from a temporary package layout with
    `codex-resources/zsh/bin/zsh`, matching the new packaged discovery path
    instead of setting `zsh_path` in config.
    - Switched package executable copying from metadata-preserving `copy2()`
    to `copyfile()` plus explicit executable bits, which avoids macOS
    file-flag failures when local smoke tests use system binaries as inputs.
    
    ## Testing
    
    To verify that the `zsh` executable from the Codex package is picked up
    correctly, first I ran:
    
    ```shell
    ./scripts/build_codex_package.py
    ```
    
    which created:
    
    ```
    /private/var/folders/vw/x2knqmks50sfhfpy27nftl900000gp/T/codex-package-pms94kdp/
    ```
    
    so then I ran:
    
    ```
    /private/var/folders/vw/x2knqmks50sfhfpy27nftl900000gp/T/codex-package-pms94kdp/bin/codex exec --enable shell_zsh_fork 'run `echo $0`'
    ```
    
    which reported the following, as expected:
    
    ```
    /private/var/folders/vw/x2knqmks50sfhfpy27nftl900000gp/T/codex-package-pms94kdp/codex-resources/zsh/bin/zsh
    ```
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23756).
    * #23768
    * __->__ #23756
  • [codex] Remove external client session reset plumbing (#24157)
    ## Why
    
    The turn loop no longer needs to decide when a `ModelClientSession`
    should reset its websocket state after compaction. That reset behavior
    belongs inside the model client, where the websocket cache and retry
    state are owned. The repo guidance now calls this out explicitly so
    future changes let the incremental request logic decide whether the
    previous request can be reused.
    
    ## What Changed
    
    - Removed the `reset_client_session` return value from pre-sampling and
    auto-compact helpers in `core/src/session/turn.rs`.
    - Changed compaction helpers to return `CodexResult<()>` so callers only
    handle success or failure.
    - Made `ModelClientSession::reset_websocket_session` private to
    `core/src/client.rs`, leaving it callable only from model-client
    internals.
    - Added `AGENTS.md` guidance not to call `reset_client_session`
    unnecessarily.
    
    ## Validation
    
    - `just test -p codex-core session::turn`
  • [codex] Use TurnInput for session task input (#24151)
    ## Why
    
    The idea here is to erase the difference between initial and followup
    inputs to a turn. Followup inputs are already represented as TurnInput.
    
    Eventual goal is not to have explicit on task input at all and pull
    everything from input Q.
    
    ## What Changed
    
    - Changes `SessionTask::run` and the erased `AnySessionTask::run` path
    to accept `Vec<TurnInput>`.
    - Wraps user-submitted spawn input as `TurnInput::UserInput` at the
    session task start boundary.
    - Updates `run_turn` to record initial `TurnInput` using the same hook
    and recording path used for pending input.
    - Keeps review-specific conversion local to `ReviewTask`, where the
    sub-Codex one-shot API still expects `Vec<UserInput>`.
    - Moves the synthetic compact prompt into `CompactTask` and starts
    compact tasks with empty task input.
    
    ## Validation
    
    - `cargo check -p codex-core`
    - `just test -p codex-core -E
    'test(task_finish_emits_turn_item_lifecycle_for_leftover_pending_user_input)
    | test(queued_response_items_for_next_turn_move_into_next_active_turn) |
    test(steered_input_reopens_mailbox_delivery_for_current_turn)'`
  • Add trace_id to TurnStartedEvent (#23980)
    ## Why
    [Recent PR](https://github.com/openai/codex/pull/22709) removed
    `trace_id` from `TurnContextItem`.
    
    ## What changed
    - Add to `TurnStartedEvent` so rollout consumers can correlate turns
    with telemetry traces.
    - Note that the branch name is out of date because I originally re-added
    to `TurnContextItem`, but we decided to move it to `TurnStartedEvent`.
    
    ## Verification
    - `cargo test -p codex-protocol`
    - `cargo test -p codex-core --lib
    regular_turn_emits_turn_started_without_waiting_for_startup_prewarm`
    - `cargo test -p codex-core --test all
    emits_warning_when_resumed_model_differs`
    - `cargo test -p codex-rollout`
    - `cargo test -p codex-state`
  • otel: drop legacy profile usage telemetry (#24061)
    ## Summary
    - drop the dead legacy profile usage metric and active-profile
    conversation-start fields
    - update role comments so they describe provider and service-tier
    preservation without legacy config-profile wording
    - pair the code cleanup with the file-backed profile docs update in
    openai/developers-website#1476
    
    ## Testing
    - `just fmt`
    - `cargo test -p codex-otel`
    - `cargo test -p codex-core` *(fails: existing stack overflow in
    `mcp_tool_call::tests::guardian_mode_mcp_denial_returns_rationale_message`)*
    - `cargo test -p codex-core --lib
    mcp_tool_call::tests::guardian_mode_mcp_denial_returns_rationale_message`
    *(fails with the same stack overflow)*
  • config: remove legacy profile write paths (#24055)
    ## Why
    
    [#23883](https://github.com/openai/codex/pull/23883) moved the
    user-facing `--profile` flag onto profile v2 and
    [#23886](https://github.com/openai/codex/pull/23886) removed CLI
    forwarding for the legacy profile-v1 path. Core and TUI config
    persistence still carried `active_profile` and
    `ConfigEditsBuilder::with_profile`, which let later writes continue
    targeting legacy `[profiles.<name>]` tables after profile selection
    moved to profile-v2 config files.
    
    ## What
    
    - Remove legacy profile routing from
    [`ConfigEditsBuilder`](https://github.com/openai/codex/blob/4b38e9c22e762261d7f7eef49d8a21792e241a06/codex-rs/core/src/config/edit.rs#L1064-L1294),
    so core config edits no longer carry `with_profile` or infer
    `[profiles.*]` write targets from a `profile` key.
    - Drop `active_profile` plumbing from runtime `Config`, TUI
    startup/state, app-server config override forwarding, and Windows
    sandbox setup persistence.
    - Make app-server-backed TUI config edits use unscoped model,
    service-tier, feature, Auto-review, plan-mode, and Windows sandbox paths
    through
    [`tui/src/config_update.rs`](https://github.com/openai/codex/blob/4b38e9c22e762261d7f7eef49d8a21792e241a06/codex-rs/tui/src/config_update.rs#L43-L112).
    - Update config edit coverage so legacy `profile` state stays untouched
    by direct model writes, and remove tests whose only contract was the
    deleted profile-scoped persistence path.
    
    ## Testing
    
    - Not run locally.
  • Remove plugin hooks feature flag (#22552)
    # Why
    
    This is a follow-up stacked on top of the `plugin_hooks` default-on
    change. Once we are comfortable making plugin hooks part of the normal
    plugin behavior, the separate feature flag stops buying us much and
    leaves extra branching/cache state behind.
    
    # What
    
    - remove the `PluginHooks` feature and generated config-schema entries
    - make plugin hook loading/listing follow plugin enablement directly
    - drop plugin-manager cache/state that only existed to distinguish
    hook-flag toggles
    - remove tests and fixtures that modeled `plugin_hooks = true/false`
  • Route MCP servers through explicit environments (#23583)
    ## Summary
    - route each configured MCP server through an explicit per-server
    `environment_id` instead of a manager-wide remote toggle
    - default omitted `environment_id` to `local`, resolve named ids through
    `EnvironmentManager`, and fail only the affected MCP server when an
    explicit id is unknown
    - keep local stdio on the existing local launcher path for now, while
    named-environment stdio uses the selected environment backend and
    requires an absolute `cwd`
    - allow local HTTP MCP servers to keep using the ambient HTTP client
    when no local `Environment` is configured; named-environment HTTP MCPs
    use that environment's HTTP client
    
    ## Validation
    - devbox Bazel build: `bazel build --bes_backend= --bes_results_url=
    //codex-rs/cli:codex //codex-rs/rmcp-client:test_stdio_server
    //codex-rs/rmcp-client:test_streamable_http_server`
    - devbox app-server config matrix with real `config.toml` /
    `environments.toml` files covering omitted local, explicit local,
    omitted local under remote default, explicit remote stdio, local HTTP
    without local env, explicit remote HTTP, local stdio without local env,
    unknown explicit env, and remote stdio without `cwd`
  • Honor client-resolved service tier defaults (#23537)
    ## Why
    
    Model catalog responses can now advertise a nullable
    `default_service_tier` for each model. Codex needs to preserve three
    distinct states all the way from config/app-server inputs to inference:
    
    - no explicit service tier, so the client may apply the current model
    catalog default when FastMode is enabled
    - explicit `default`, meaning the user intentionally wants standard
    routing
    - explicit catalog tier ids such as `priority`, `flex`, or future tiers
    
    Keeping those states distinct prevents the UI from showing one tier
    while core sends another, especially after model switches or app-server
    `thread/start` / `turn/start` updates.
    
    ## What Changed
    
    - Plumbed `default_service_tier` through model catalog protocol types,
    app-server model responses, generated schemas, model cache fixtures, and
    provider/model-manager conversions.
    - Added the request-only `default` service tier sentinel and normalized
    legacy config spelling so `fast` in `config.toml` still materializes as
    the runtime/request id `priority`.
    - Moved catalog default resolution to the TUI/client side, including
    recomputing the effective service tier when model/FastMode-dependent
    surfaces change.
    - Updated app-server thread lifecycle config construction so
    `serviceTier: null` preserves explicit standard-routing intent by
    mapping to `default` instead of internal `None`.
    - Kept core responsible for validating explicit tiers against the
    current model and stripping `default` before `/v1/responses`, without
    applying catalog defaults itself.
    
    ## Validation
    
    - `CARGO_INCREMENTAL=0 cargo build -p codex-cli`
    - `CARGO_INCREMENTAL=0 cargo test -p codex-app-server model_list`
    - `cargo test -p codex-tui service_tier`
    - `cargo test -p codex-protocol service_tier_for_request`
    - `cargo test -p codex-core get_service_tier`
    - `RUST_MIN_STACK=8388608 CARGO_INCREMENTAL=0 cargo test -p codex-core
    service_tier`
  • Add SubagentStop hook (#22873)
    # What
    
    <img width="1792" height="1024" alt="image"
    src="https://github.com/user-attachments/assets/8f81d232-5813-4994-a61d-e42a05a93a3e"
    />
    
    `SubagentStop` runs when a thread-spawned subagent turn is about to
    finish. Thread-spawned subagents use `SubagentStop` instead of the
    normal root-agent `Stop` hook.
    
    Configured handlers match on `agent_type`. Hook input includes the
    normal stop fields plus:
    
    - `agent_id`: the child thread id.
    - `agent_type`: the resolved subagent type.
    - `agent_transcript_path`: the child subagent transcript path.
    - `transcript_path`: the parent thread transcript path.
    - `last_assistant_message`: the final assistant message from the child
    turn, when available.
    - `stop_hook_active`: `true` when the child is already continuing
    because an earlier stop-like hook blocked completion.
    
    `SubagentStop` shares the same completion-control semantics as `Stop`,
    scoped to the child turn:
    
    - No decision allows the child turn to finish.
    - `decision: "block"` with a non-empty `reason` records that reason as
    hook feedback and continues the child with that prompt.
    - `continue: false` stops the child turn. If `stopReason` is present,
    Codex surfaces it as the stop reason.
    
    # Lifecycle Scope
    
    Only thread-spawned subagents run `SubagentStop`.
    
    Internal/system subagents such as Review, Compact, MemoryConsolidation,
    and Other do not run normal `Stop` hooks and do not run `SubagentStop`.
    This avoids exposing synthetic matcher labels for internal
    implementation paths.
    
    # Stack
    
    1. #22782: add `SubagentStart`.
    2. This PR: add `SubagentStop`.
    3. #22882: add subagent identity to normal hook inputs.
  • core: refresh active permission profiles at runtime (#22931)
    ## Why
    
    Once a named permission profile is selected, runtime state has to keep
    that profile identity intact instead of collapsing back to anonymous
    effective permissions. The session refresh path also needs to rebuild
    profile-derived network proxy state so active profile switches take
    effect consistently.
    
    ## What changed
    
    - Preserve the active permission profile through session updates.
    - Rebuild profile-derived runtime/network configuration when the active
    profile changes.
    - Keep the runtime path aligned with the current session configuration
    APIs.
    - Tighten the affected tests, including the Windows delete-pending
    memory-file case that was intermittently tripping CI.
    
    ## Stack
    
    1. **This PR**: runtime/session/network propagation for active
    permission profiles.
    2. [#23708](https://github.com/openai/codex/pull/23708): TUI selection
    plumbing and guardrail flow.
    3. [#21559](https://github.com/openai/codex/pull/21559): profile-aware
    `/permissions` menu and custom profile display.
    
    <img width="1296" height="906" alt="image"
    src="https://github.com/user-attachments/assets/077fa3a7-80cb-4925-80b1-d2395018d90a"
    />
  • Support compact SessionStart hooks (#21272)
    # Why
    
    Compaction replaces the live conversation history, so hooks that use
    `SessionStart` to re-inject durable model context need a way to run
    again after that rewrite.
    
    Related - #19905 adds dedicated compact lifecycle hooks
    
    # What
    
    - add `compact` as a supported `SessionStart` source and matcher value
    - change pending `SessionStart` state from a single slot to a small FIFO
    queue so `resume` / `startup` / `clear` can be preserved alongside a
    later `compact`
    - drain all queued `SessionStart` sources before the next model request,
    preserving their original order
    
    # Testing
    
    The new integration coverage verifies both the basic `compact` matcher
    path and the stacked `resume` -> `compact` case where both hooks
    contribute `additionalContext` to the next model turn.
  • feat: account active goal progress in the goal extension (#23696)
    ## Why
    
    The goal extension can create and surface goals, but the live
    turn-accounting path still stopped short of persisting active-goal
    progress. That leaves token and wall-clock usage, plus
    `ThreadGoalUpdated` events, out of sync with the extension boundary once
    work actually advances or a goal transitions out of active state.
    
    ## What changed
    
    - Teach `GoalAccountingState` to track the current turn, active goal,
    token deltas, and wall-clock progress snapshots against the persisted
    goal id.
    - Flush active-goal accounting from tool-finish, turn-stop, and
    turn-abort lifecycle hooks, and emit `ThreadGoalUpdated` events when
    persisted progress changes.
    - Route `create_goal` and `update_goal` through the same accounting
    state so new goals start from the right baseline, final progress is
    flushed before status changes, and `update_goal` can mark a goal
    `blocked` as well as `complete`.
    - Keep budget-limited goals accruing through the end of the turn while
    clearing local active-goal state once a turn or explicit update is
    finished.
    - Expand backend and lifecycle coverage around store ids, baseline
    reset, tool-finish accounting, budget-limited carry-through, and
    blocked-goal updates.
    
    ## Testing
    
    - Added focused backend coverage in
    `codex-rs/ext/goal/tests/goal_extension_backend.rs` for baseline reset,
    tool-finish accounting, budget-limited turns, and blocked-goal updates.
    - Extended `codex-rs/core/src/session/tests.rs` to assert that lifecycle
    inputs expose the expected session, thread, and turn store ids.
  • feat: expose turn-start metadata to extensions (#23688)
    ## Why
    
    The goal extension needs more context when a turn starts than
    `turn_store` alone provides.
    
    In particular, goal accounting needs the stable turn id, the effective
    collaboration mode, and the cumulative token-usage baseline captured at
    turn start so it can:
    
    - suppress goal accounting for plan-mode turns
    - compute exact per-turn deltas from cumulative `total_token_usage`
    snapshots instead of relying on the most recent usage event alone
    - keep the extension-owned accounting path aligned with the host turn
    lifecycle
    
    ## What
    
    - extend `codex_extension_api::TurnStartInput` to expose `turn_id`,
    `collaboration_mode`, and `token_usage_at_turn_start`
    - pass the full `TurnContext` plus the captured token-usage baseline
    through the turn-start lifecycle emission path
    - initialize goal turn accounting from the turn-start baseline and
    collaboration mode
    - switch goal token accounting to compute deltas from cumulative
    `total_token_usage` snapshots
    - add coverage for the new turn-start lifecycle fields and for
    goal-accounting baseline behavior
    
    ## Testing
    
    - added `turn_start_lifecycle_exposes_turn_metadata_and_token_baseline`
    in `codex-rs/core/src/session/tests.rs`
    - added `ext/goal/tests/accounting.rs` coverage for baseline-aware goal
    accounting and plan-mode suppression
  • feat: rename 3 (#23669)
    Just a mechanical renaming
  • Warn on invalid UTF-8 in AGENTS.md files (#23232)
    Fixes #23223.
    
    ## Why
    
    Malformed AGENTS instructions should not fail silently. The reported
    issue had invalid UTF-8 in a global `AGENTS.md`; before this change,
    Codex treated that decode failure like a missing file, so the personal
    instructions disappeared without a user-visible explanation and the
    rollout had no `# AGENTS.md instructions` block.
    
    Project-level AGENTS files already used lossy decoding, so their
    instructions still appeared, but invalid bytes were replaced without
    telling the user. Global and project AGENTS files should behave
    consistently: keep usable instruction text when possible, and surface a
    diagnostic when bytes had to be replaced.
    
    ## What changed
    
    Global `AGENTS.override.md` and `AGENTS.md` loading now reads bytes and
    decodes with replacement characters on invalid UTF-8, matching
    project-level AGENTS behavior. Both global and project AGENTS loading
    now emit a startup warning when invalid UTF-8 is found, and both keep
    the instruction text with invalid byte sequences replaced.
    
    Missing files, non-file candidates, empty files, and the existing
    `AGENTS.override.md` before `AGENTS.md` precedence keep their current
    behavior.
    
    ## How users see it
    
    The warnings flow through the existing startup warning surface.
    App-server clients receive config-time startup warnings as
    `configWarning` notifications during initialization, and thread startup
    emits startup warnings as thread-scoped `warning` notifications.
    
    Global AGENTS invalid UTF-8 warnings can appear on both surfaces.
    Project-level AGENTS invalid UTF-8 warnings are discovered while
    building thread instructions, so they appear as thread-scoped `warning`
    notifications. Clients that render warning notifications in the
    conversation surface show the message as a visible diagnostic instead of
    silently hiding or altering instructions.
  • Move plugin and skill warmup into session startup (#23535)
    ## Why
    
    Plugin and skill loading is useful as warmup and early validation, but
    session startup does not need to wait for that work before it can
    continue building the session. Keeping it on the serial startup path
    adds avoidable latency to every fresh thread start.
    
    We still want invalid skill configurations to show up quickly, and we
    want the warmup to exercise the same plugin and skill manager caches
    that the normal turn path uses.
    
    ## What changed
    
    - moved plugin and skill warmup into the session startup async path
    instead of eagerly awaiting it on the serial setup path
    - kept the warmup using the session's resolved filesystem/environment
    context so skill loading still sees the right roots
    - preserved early skill-load error logging so broken skill
    configurations still surface during startup
    - left the per-turn plugin and skill loading path unchanged, so turns
    still use the normal cached managers
    
    ## Testing
    
    - Not run locally; relying on CI for validation.
  • Split plugin install discovery into list and request tools (#23372)
    ## Summary
    - Add `list_available_plugins_to_install` as the inventory step for
    plugin and connector install suggestions.
    - Slim `request_plugin_install` so it only handles the actual
    elicitation, instead of carrying the full discoverable list in its
    prompt.
    - Emit send-time telemetry when an install elicitation is dispatched,
    including requested tool identity in the event payload.
    - Emit install-result telemetry through `SessionTelemetry`, including
    tool type, user response action, and completion status.
    - Update registration and tests to cover the new two-step flow while
    keeping the existing `tool_suggest` feature gate unchanged.
    
    ## Testing
    - `just fmt`
    - `cargo test -p codex-tools`
    - `cargo test -p codex-core request_plugin_install`
    - `cargo test -p codex-core list_available_plugins_to_install`
    - `cargo test -p codex-core
    install_suggestion_tools_can_be_registered_without_search_tool`
    - `cargo test -p codex-otel
    manager_records_plugin_install_suggestion_metric`
    - `cargo test -p codex-otel
    manager_records_plugin_install_elicitation_sent_metric`
    - `just fix -p codex-core`
    - `just fix -p codex-tools`
    - `just fix -p codex-otel`
    - `cargo check -p codex-core`
  • Make local environment optional in EnvironmentManager (#23369)
    ## Summary
    - make `EnvironmentManager` local environment/runtime paths optional
    - simplify constructor surface around snapshot materialization
    - rename local env accessors to `require_local_environment` /
    `try_local_environment`
    
    ## Validation
    - devbox Bazel build for touched crate surfaces
    - `//codex-rs/exec-server:exec-server-unit-tests`
    - `//codex-rs/app-server-client:app-server-client-unit-tests`
    - filtered touched `//codex-rs/core:core-unit-tests` cases
  • Add SubagentStart hook (#22782)
    # What
    
    `SubagentStart` runs once when Codex creates a thread-spawned subagent,
    before that child sends its first model request. Thread-spawned
    subagents use `SubagentStart` instead of the normal root-agent
    `SessionStart` hook.
    
    Configured handlers match on the subagent `agent_type`, using the same
    value passed to `spawn_agent`. When no agent type is specified, Codex
    uses the default agent type.
    
    Hook input includes the normal session-start fields plus:
    
    - `agent_id`: the child thread id.
    - `agent_type`: the resolved subagent type.
    
    `SubagentStart` may return `hookSpecificOutput.additionalContext`. That
    context is added to the child conversation before the first model
    request.
    
    # Lifecycle Scope
    
    Only thread-spawned subagents run `SubagentStart`.
    
    Internal/system subagents such as Review, Compact, MemoryConsolidation,
    and Other do not run normal `SessionStart` hooks and do not run
    `SubagentStart`. This avoids exposing synthetic matcher labels for
    internal implementation paths.
    
    Also the `SessionStart` hook no longer fires for subagents, this matches
    behavior with other coding agents' implementation
    
    # Stack
    
    1. This PR: add `SubagentStart`.
    2. #22873: add `SubagentStop`.
    3. #22882: add subagent identity to normal hook inputs.
  • Make deny canonical for filesystem permission entries (#23493)
    ## Why
    Filesystem permission profiles used `none` for deny-read entries, which
    is less direct than the action the entry actually represents. This
    change makes `deny` the canonical filesystem permission spelling while
    preserving compatibility for older configs that still send `none`.
    
    ## What changed
    - rename `FileSystemAccessMode::None` to `Deny`
    - serialize and generate schemas with `deny` as the canonical value
    - retain `none` only as a legacy input alias for temporary config
    compatibility
    - update filesystem glob diagnostics and regression coverage to use the
    canonical spelling
    - refresh config and app-server schema fixtures to match the new wire
    shape
    
    ## Validation
    - `cargo test -p codex-protocol`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-core config_toml_deserializes_permission_profiles
    --lib`
    - `cargo test -p codex-core
    read_write_glob_patterns_still_reject_non_subpath_globs --lib`
    
    Earlier in the session, a broad `cargo test -p codex-core` run reached
    unrelated pre-existing failures in timing/snapshot/git-info tests under
    this environment; the targeted surfaces touched by this PR passed
    cleanly.
  • [codex] Preserve steer input as user input (#23405)
    ## Why
    
    Steered input was queued as a `ResponseInputItem`, then parsed back into
    a user message before recording. That path loses information that only
    exists on `UserInput`, such as UI text elements.
    
    This change keeps turn-local pending input typed as either original
    `UserInput` or existing response items, so steered user input reaches
    user-message recording without being reconstructed from a response item.
    
    ## What changed
    
    - Add `TurnInput` for active-turn pending input.
    - Queue `Session::steer_input` as `TurnInput::UserInput`.
    - Run pending-input hook inspection only for `TurnInput::UserInput`.
    - Process drained pending input item by item: accepted items are
    recorded, blocked items append hook context and are skipped.
    - Remove the pending-input prepend/requeue path.
    
    ## Validation
    
    - `just fmt`
    - `just fix -p codex-core`
    - `RUST_MIN_STACK=16777216 cargo test -p codex-core --lib
    session::tests::task_finish_emits_turn_item_lifecycle_for_leftover_pending_user_input
    -- --nocapture`
    - `RUST_MIN_STACK=16777216 cargo test -p codex-core --lib steer_input`
    - `RUST_MIN_STACK=16777216 cargo test -p codex-core --lib pending_input`
    - `RUST_MIN_STACK=16777216 cargo test -p codex-core --test all
    pending_input`
    - `RUST_MIN_STACK=16777216 cargo test -p codex-core` (unit tests passed:
    1835 passed, 0 failed, 4 ignored; integration `all` target failed due
    missing helper binaries such as `codex`/`test_stdio_server` plus
    unrelated MCP/search/code-mode expectations)
  • [codex] Move hook request plumbing into hook runtime (#23388)
    ## Why
    
    `run_turn` was still hand-building hook payloads and lifecycle events
    for a couple of hook paths. Most hook call sites already delegate
    request construction and event emission to `hook_runtime`, which keeps
    turn orchestration focused on model-flow decisions rather than hook
    plumbing.
    
    This also keeps the legacy `after_agent` message extraction next to the
    legacy hook dispatch instead of leaving response-item walking in
    `run_turn`.
    
    ## What changed
    
    - Added `run_stop_hooks` in `hook_runtime` to build `StopRequest`, emit
    preview start events, run the hook, and emit completion events.
    - Added `run_legacy_after_agent_hook` in `hook_runtime` to build and
    dispatch the legacy `AfterAgent` hook payload, including extracting
    input messages from response items.
    - Updated `run_turn` to call the hook runtime helpers and keep only the
    resulting continuation/block/stop decisions inline.
    - Removed the repeated pending session-start hook check from the run
    loop.
    
    ## Validation
    
    - `cargo test -p codex-core hook_runtime`
  • [codex] Allow empty turn/start requests (#23409)
    ## Why
    
    `turn/start` already accepts an input array on the wire, including an
    empty array, but core treated empty input as a no-op before the turn
    could reach the model. App-server clients need to be able to start a
    real turn even when there is no new user message, for example to let the
    model proceed from existing thread context.
    
    ## What changed
    
    - Removed the `run_turn` early return that skipped empty-input turns
    when there was no pending input.
    - Kept empty active-turn steering rejected by moving the `steer_input`
    empty-input check until after core has determined whether there is an
    active regular turn.
    - Empty regular turns now refresh `previous_turn_settings` like other
    regular turns, so follow-up context injection state advances
    consistently.
    - Added an app-server v2 integration test proving `turn/start` with
    `input: []` emits started/completed notifications, sends one Responses
    request, and does not synthesize an empty user message.
    
    ## Validation
    
    - `cargo test -p codex-app-server --test all
    turn_start_with_empty_input_runs_model_request`
  • Defer v1 multi-agent tools behind tool search (#23144)
    Summary: defer v1 multi-agent tools when tool_search and namespace tools
    are available; keep concise searchable descriptions and move the v1
    usage guidance into developer instructions; add targeted coverage.
    Testing: not run per request; ran just fmt.
  • Add body_after_prefix auto-compact token limit scope (#22870)
    ## Why
    
    `model_auto_compact_token_limit` has only been able to budget the full
    active context. That makes it hard to set a small "growth since
    compaction" budget for sessions that preserve a large carried window
    prefix: the preserved prefix can consume the whole budget and force
    immediate repeated compaction.
    
    This PR adds an opt-in `body_after_prefix` scope so callers can apply
    `model_auto_compact_token_limit` to sampled output and later growth
    after the current carried prefix, while still forcing compaction before
    the full model context window is exhausted.
    
    ## What changed
    
    - Adds `AutoCompactTokenLimitScope` with the existing `total` behavior
    as the default and a new `body_after_prefix` mode:
    [`config_types.rs`](https://github.com/openai/codex/blob/973806b1cb35792555bead994cb3ed94656eb171/codex-rs/protocol/src/config_types.rs#L24-L37).
    - Threads `model_auto_compact_token_limit_scope` through config loading,
    `Config`, `core-api`, and app-server v2 schema/TypeScript generation.
    - Records the first observed input-token count for a `body_after_prefix`
    compaction window and uses it as the baseline when deciding whether the
    scoped auto-compaction budget is exhausted:
    [`turn.rs`](https://github.com/openai/codex/blob/973806b1cb35792555bead994cb3ed94656eb171/codex-rs/core/src/session/turn.rs#L743-L781).
    - Keeps a hard context-window cap in `body_after_prefix`, so scoped
    budgeting cannot let the active context overrun the usable window.
    
    ## Verification
    
    Added compact-suite coverage for the two key behaviors:
    `body_after_prefix` does not re-compact just because the carried prefix
    is larger than the scoped budget, and it still compacts when the total
    active context reaches the configured context window:
    [`compact.rs`](https://github.com/openai/codex/blob/973806b1cb35792555bead994cb3ed94656eb171/codex-rs/core/tests/suite/compact.rs#L3003-L3128).
  • Remove ToolsConfig from tool planning (#22835)
    ## Why
    
    `codex-tools` is meant to hold reusable tool primitives, but
    `ToolsConfig` had become a second copy of core runtime decisions instead
    of a small shared contract. It carried provider capabilities, auth/model
    gates, permission and environment state, web/search/image feature gates,
    multi-agent settings, and goal availability from core into `codex-tools`
    ([definition](https://github.com/openai/codex/blob/22dd9ad3929253ed24d7ee4f10f238e95ab25f37/codex-rs/tools/src/tool_config.rs#L97),
    [stored on each
    `TurnContext`](https://github.com/openai/codex/blob/22dd9ad3929253ed24d7ee4f10f238e95ab25f37/codex-rs/core/src/session/turn_context.rs#L87)).
    Every session/context variant then had to build and mutate that snapshot
    before assembling tools.
    
    This PR removes that master object instead of renaming it. Tool planning
    now reads the live `TurnContext`, where `codex-core` already owns those
    decisions, while `codex-tools` keeps only reusable primitives and a
    generic `ToolSetBuilder`/`ToolSet` accumulator.
    
    ## What Changed
    
    - Removed `ToolsConfig` / `ToolsConfigParams` from `codex-tools`; the
    crate keeps the shared helpers that still belong there, including
    request-user-input mode selection, shell backend/type resolution,
    `UnifiedExecShellMode`, and `ToolEnvironmentMode`.
    - Replaced config-snapshot planning with `ToolRouter::from_turn_context`
    and a `spec_plan` pipeline over `CoreToolPlanContext`, deriving provider
    capabilities, auth gates, model support, feature gates, environment
    count, goal support, multi-agent options, web search, and image
    generation from the authoritative turn state.
    - Added generic `codex_tools::ToolSetBuilder` / `ToolSet`, plus the
    small core adapter needed to accumulate `CoreToolRuntime` values and
    hosted model specs.
    - Added the `tool_family::shell` registration module and moved
    shell/unified-exec/memory accounting call sites to read the narrow
    per-turn fields directly.
    - Narrowed `TurnContext` to the remaining explicit per-turn fields
    needed by planning: `available_models`, `unified_exec_shell_mode`, and
    `goal_tools_supported`.
    - Reworked MCP exposure and tool-search setup so deferred/direct MCP
    behavior is driven by the current turn rather than a precomputed config
    snapshot.
    - Replaced the large expected-spec fixture tests with focused
    behavior-level coverage for shell tools, environments, goal and
    agent-job gates, MCP direct/deferred exposure, tool search,
    request-plugin-install, code mode, multi-agent mode, hosted tools, and
    extension executor dispatch.
    
    ## Verification
    
    - `cargo check -p codex-tools`
    - `cargo check -p codex-core --lib`
    - `cargo test -p codex-tools`
    - `cargo test -p codex-core spec_plan --lib`
    - `cargo test -p codex-core router --lib`
  • feat: dedicated goal DB (#23300)
    ## Why
    
    Thread goals are moving toward extension-owned runtime behavior, but
    their persisted state was still stored in the shared state database.
    This makes the goal store harder to isolate and keeps future storage
    splits tied to ad hoc runtime plumbing.
    
    This PR gives goals their own SQLite database while keeping the existing
    `StateRuntime` entry point. The goal is to make this the pattern for
    adding more dedicated runtime databases later.
    
    This also reduce load on existing DB and reduce contention
    
    ## Limitation
    Thread preview from goal is not supported anymore. I'm looking into this
    [EDIT]: solved
    
    ## What changed
    
    - Added a dedicated `goals_1.sqlite` database with its own
    `goals_migrations` directory.
    - Moved `thread_goals` creation into the goals DB migration set.
    - Dropped the old `thread_goals` table from the main state DB with a
    normal state migration. There is intentionally no backfill for existing
    goal rows.
    - Changed `GoalStore` to be backed only by the goals DB pool.
    - Removed the old goal-write side effect that filled empty
    `threads.preview` values from the goal objective.
    - Added shared runtime DB path metadata so startup, telemetry, `codex
    doctor`, and repair handling can include future DBs without bespoke path
    lists.
    - Updated Bazel compile data so the new goals migration directory is
    available to `sqlx::migrate!`.
    
    ## Verification
    
    - `cargo check --tests -p codex-state -p codex-cli -p codex-core -p
    codex-app-server`
    - `just fix -p codex-state`
    - `just fix -p codex-cli`
    - `just fix -p codex-app-server`
  • Preserve context baselines for full-history agent forks (#23352)
    ## Why
    
    Full-history agent forks should continue from the same prompt prefix as
    the parent. Dropping the stored `TurnContext` baseline forced the child
    to rebuild startup context on its first turn, which can duplicate
    developer instructions and also loses the cache continuity that a
    full-history fork is supposed to preserve.
    
    Truncated forks are different: once we keep only the last N turns, the
    original prompt prefix is no longer intact, so the child must establish
    a fresh context baseline.
    
    ## What changed
    
    - Preserve `RolloutItem::TurnContext` when forking with
    `SpawnAgentForkMode::FullHistory`, and keep dropping it for truncated
    forks:
    https://github.com/openai/codex/blob/4090717d94c1fc7f33c9bd122be133a0c5752052/codex-rs/core/src/agent/control.rs#L98-L126
    and
    https://github.com/openai/codex/blob/4090717d94c1fc7f33c9bd122be133a0c5752052/codex-rs/core/src/agent/control.rs#L399-L401
    - Remove the special-case MultiAgentV2 usage-hint filtering path.
    Full-history fork now preserves the cached developer prefix instead of
    trying to reconstruct part of it.
    - Extend the fork coverage to assert both sides of the contract:
    full-history forks keep the parent reference baseline, while last-N
    forks rebuild context after truncation:
    https://github.com/openai/codex/blob/4090717d94c1fc7f33c9bd122be133a0c5752052/codex-rs/core/src/agent/control_tests.rs#L603-L759
    and
    https://github.com/openai/codex/blob/4090717d94c1fc7f33c9bd122be133a0c5752052/codex-rs/core/src/agent/control_tests.rs#L854-L977
    
    ## Verification
    
    - `cargo test -p codex-core
    spawn_agent_can_fork_parent_thread_history_with_sanitized_items --
    --nocapture`
    - `RUST_MIN_STACK=16777216 cargo test -p codex-core
    spawn_agent_fork_last_n_turns_keeps_only_recent_turns -- --nocapture`
  • Remove explicit connector tool undeferral (#23390)
    ## Summary
    - remove the explicit-connector carveout that kept mentioned app tools
    directly exposed instead of deferred
    - keep the surviving explicit-mention reconstruction only for analytics,
    preserving `codex_app_mentioned` and `codex_app_used.invoke_type`
    - trim the now-unused prompt/tool-exposure plumbing and refresh coverage
    around always-defer behavior
    
    ## Verification
    - `just fmt`
    - `cargo test -p codex-analytics`
    - `cargo test -p codex-core` *(one transient timeout in
    `shell_snapshot::tests::macos_zsh_snapshot_includes_sections`; isolated
    rerun passed)*
    - `cargo test -p codex-core --lib
    shell_snapshot::tests::macos_zsh_snapshot_includes_sections`
    - `cargo test -p codex-core --test all
    explicit_app_mentions_respect_always_defer`
    - `cargo test -p codex-core --lib
    mcp_tool_exposure::tests::always_defer_feature_defers_apps_too`
    - `just fix -p codex-analytics`
    - `just fix -p codex-core`
  • [5 of 7] Replace OverrideTurnContext with ThreadSettings (#22508)
    **Stack position:** [5 of 7]
    
    ## Summary
    
    This PR adds `Op::ThreadSettings`, a queued settings-only update
    mechanism for changing stored thread settings without starting a new
    turn. It also removes the legacy `Op::OverrideTurnContext` in the same
    layer, so reviewers can see the replacement and deletion together.
    
    ## Changes
    
    - Add `Op::ThreadSettings` for settings-only queued updates.
    - Emit `ThreadSettingsApplied` with the effective thread settings
    snapshot after core applies an update.
    - Route settings-only updates through the same submission queue as user
    input.
    - Migrate remaining `OverrideTurnContext` tests and callers to the
    queued `Op::ThreadSettings` path.
    - Delete `Op::OverrideTurnContext` from the core protocol and submission
    loop.
    
    This stack addresses #20656 and #22090.
    
    ## Stack
    
    1. [1 of 7] [Add thread settings to
    UserInput](https://github.com/openai/codex/pull/23080)
    2. [2 of 7] [Remove
    UserInputWithTurnContext](https://github.com/openai/codex/pull/23081)
    3. [3 of 7] [Remove
    UserTurn](https://github.com/openai/codex/pull/23075)
    4. [4 of 7] [Placeholder for OverrideTurnContext
    cleanup](https://github.com/openai/codex/pull/23087)
    5. [5 of 7] [Replace OverrideTurnContext with
    ThreadSettings](https://github.com/openai/codex/pull/22508) (this PR)
    6. [6 of 7] [Add app-server thread settings
    API](https://github.com/openai/codex/pull/22509)
    7. [7 of 7] [Sync TUI thread
    settings](https://github.com/openai/codex/pull/22510)
  • [codex] Extract turn skill and plugin injections (#23396)
    ## Why
    
    `run_turn` had accumulated the turn-scoped skill, plugin, app, MCP,
    connector-selection, and analytics setup inline. That made the
    orchestration path harder to scan even though the actual turn item
    injection still needs to stay in `run_turn` so ordering is explicit.
    
    ## What changed
    
    This extracts that setup into `build_skills_and_plugins`, which returns
    the combined injection `ResponseItem`s and the explicitly enabled
    connector IDs. `run_turn` now keeps the required orchestration pieces:
    context update recording, user input handling, connector selection
    merge, and the explicit per-item `record_conversation_items` calls for
    injection items.
    
    The refactor keeps the change LOC-neutral in `core/src/session/turn.rs`
    and preserves the existing response-item based injection path.
    
    ## Validation
    
    - `cargo test -p codex-core collect_explicit_app_ids_from_skill_items`
    - `just fix -p codex-core`
  • [3 of 7] Remove UserTurn (#23075)
    **Stack position:** [3 of 7]
    
    ## Summary
    
    This PR finishes the input-op consolidation by moving the remaining
    `Op::UserTurn` callers onto `Op::UserInput` and deleting `Op::UserTurn`.
    This touches a lot of files, but it is a low-risk mechanical migration.
    
    ## Stack
    
    1. [1 of 7] [Add thread settings to
    UserInput](https://github.com/openai/codex/pull/23080)
    2. [2 of 7] [Remove
    UserInputWithTurnContext](https://github.com/openai/codex/pull/23081)
    3. [3 of 7] [Remove
    UserTurn](https://github.com/openai/codex/pull/23075) (this PR)
    4. [4 of 7] [Placeholder for OverrideTurnContext
    cleanup](https://github.com/openai/codex/pull/23087)
    5. [5 of 7] [Replace OverrideTurnContext with
    ThreadSettings](https://github.com/openai/codex/pull/22508)
    6. [6 of 7] [Add app-server thread settings
    API](https://github.com/openai/codex/pull/22509)
    7. [7 of 7] [Sync TUI thread
    settings](https://github.com/openai/codex/pull/22510)
  • [2 of 7] Remove UserInputWithTurnContext (#23081)
    **Stack position:** [2 of 7]
    
    ## Summary
    
    This PR removes the overlapping `Op::UserInputWithTurnContext` variant
    now that `Op::UserInput` can carry thread settings overrides directly.
    
    ## Stack
    
    1. [1 of 7] [Add thread settings to
    UserInput](https://github.com/openai/codex/pull/23080)
    2. [2 of 7] [Remove
    UserInputWithTurnContext](https://github.com/openai/codex/pull/23081)
    (this PR)
    3. [3 of 7] [Remove
    UserTurn](https://github.com/openai/codex/pull/23075)
    4. [4 of 7] [Placeholder for OverrideTurnContext
    cleanup](https://github.com/openai/codex/pull/23087)
    5. [5 of 7] [Replace OverrideTurnContext with
    ThreadSettings](https://github.com/openai/codex/pull/22508)
    6. [6 of 7] [Add app-server thread settings
    API](https://github.com/openai/codex/pull/22509)
    7. [7 of 7] [Sync TUI thread
    settings](https://github.com/openai/codex/pull/22510)
  • [1 of 7] Add thread settings to UserInput (#23080)
    **Stack position:** [1 of 7]
    
    ## Summary
    
    The first three PRs in this stack are a cleanup pass before the actual
    thread settings API work.
    
    Today, core has several overlapping "user input" ops: `UserInput`,
    `UserInputWithTurnContext`, and `UserTurn`. They differ mostly in how
    much next-turn state they carry, which makes the later queued thread
    settings update harder to reason about and review.
    
    This PR starts that cleanup by adding the shared
    `ThreadSettingsOverrides` payload and allowing `Op::UserInput` to carry
    it. Existing variants remain in place here, so this layer is mostly a
    behavior-preserving API shape change plus mechanical constructor
    updates.
    
    ## End State After PR3
    
    By the end of PR3, `Op::UserInput` is the only "user input" core op. It
    can carry optional thread settings overrides for callers that need to
    update stored defaults with a turn, while callers without updates use
    empty settings. `Op::UserInputWithTurnContext` and `Op::UserTurn` are
    deleted.
    
    ## End State After PR5
    
    By the end of PR5, core will have only two ops for this area:
    
    - `Op::UserInput` for user-input-bearing submissions.
    - `Op::ThreadSettings` for settings-only updates.
    
    ## Stack
    
    1. [1 of 7] [Add thread settings to
    UserInput](https://github.com/openai/codex/pull/23080) (this PR)
    2. [2 of 7] [Remove
    UserInputWithTurnContext](https://github.com/openai/codex/pull/23081)
    3. [3 of 7] [Remove
    UserTurn](https://github.com/openai/codex/pull/23075)
    4. [4 of 7] [Placeholder for OverrideTurnContext
    cleanup](https://github.com/openai/codex/pull/23087)
    5. [5 of 7] [Replace OverrideTurnContext with
    ThreadSettings](https://github.com/openai/codex/pull/22508)
    6. [6 of 7] [Add app-server thread settings
    API](https://github.com/openai/codex/pull/22509)
    7. [7 of 7] [Sync TUI thread
    settings](https://github.com/openai/codex/pull/22510)
  • cleanup: Remove skill env var dependency prompting (#22721)
    Deletes the skill env var dependency prompt feature and its runtime
    path. env_var entries in skill dependency metadata are now silently
    ignored during skill loading.
  • [codex] Remove external websocket session resets (#23384)
    ## Why
    
    Compaction now installs replacement history inside the session, but the
    turn and compaction callers were still reaching into
    `ModelClientSession` to reset websocket transport state after that
    install. That made a transport-level reset part of the compaction API
    even though websocket incremental request selection already checks
    whether the next request is a strict extension of the previous one and
    falls back to a full `response.create` when it is not.
    
    ## What changed
    
    - Removed the compaction-side calls to `reset_websocket_session` from
    `compact.rs` and `session/turn.rs`.
    - Simplified pre-sampling and mid-turn compaction helpers so they return
    `CodexResult<()>` instead of carrying a reset flag.
    - Made `ModelClientSession::reset_websocket_session` private to
    `client.rs`, leaving only the websocket timeout recovery path inside the
    client as a caller.
    
    ## Validation
    
    - `cargo test -p codex-core --test all
    responses_websocket_creates_on_non_prefix`
    - `cargo test -p codex-core --test all
    steered_user_input_waits_for_model_continuation_after_mid_turn_compact`
    - `cargo test -p codex-core --test all
    pre_sampling_compact_runs_on_switch_to_smaller_context_model`
  • [codex] Move pending input into input queue (#22728)
    ## Why
    
    Pending model input was split across `Session`, `TurnState`, and the
    agent mailbox. That made it easy for new paths to manage queued user
    input or mailbox delivery outside the intended ownership boundary.
    
    This PR consolidates the model-facing input lifecycle behind the session
    input queue so turn-local pending input, next-turn queued items, and
    mailbox delivery coordination are owned in one place.
    
    ## What Changed
    
    - Added `session/input_queue.rs` to own pending input queues and mailbox
    delivery coordination.
    - Removed the standalone `agent/mailbox.rs` channel wrapper and store
    mailbox items directly in the input queue.
    - Moved pending-input mutations off `TurnState`; `TurnState` now exposes
    the queue-owned storage directly for now.
    - Routed abort cleanup, mailbox delivery phase changes, next-turn queued
    items, and active-turn pending input through `InputQueue`.
    - Boxed stack-heavy agent resume/fork startup futures that the refactor
    pushed over the default test stack.
    - Updated session, task, goal, stream-event, and multi-agent call sites
    and tests to use the new queue ownership.
    
    ## Verification
    
    - `cargo test -p codex-core --lib agent::control::tests`
    - `cargo test -p codex-core --lib
    agent::control::tests::resume_closed_child_reopens_open_descendants --
    --exact`
    - `cargo test -p codex-core --lib
    agent::control::tests::spawn_agent_fork_last_n_turns_keeps_only_recent_turns
    -- --exact`
    - `cargo test -p codex-core --lib
    agent::control::tests::resume_thread_subagent_restores_stored_nickname_and_role
    -- --exact`
    - `cargo test -p codex-core` was also run; it completed with 1814
    passed, 4 ignored, and one timeout in
    `agent::control::tests::resume_thread_subagent_restores_stored_nickname_and_role`,
    which passed when rerun in isolation.