Commit Graph

815 Commits

  • [5/6] Wire executor-backed MCP stdio (#18212)
    ## Summary
    - Add the executor-backed RMCP stdio transport.
    - Wire MCP stdio placement through the executor environment config.
    - Cover local and executor-backed stdio paths with the existing MCP test
    helpers.
    
    ## Stack
    ```text
    o  #18027 [6/6] Fail exec client operations after disconnect
    │
    @  #18212 [5/6] Wire executor-backed MCP stdio
    │
    o  #18087 [4/6] Abstract MCP stdio server launching
    │
    o  #18020 [3/6] Add pushed exec process events
    │
    o  #18086 [2/6] Support piped stdin in exec process API
    │
    o  #18085 [1/6] Add MCP server environment config
    │
    o  main
    ```
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Update image outputs to default to high detail (#18386)
    Do not assume the default `detail`.
  • Add max context window model metadata (#18382)
    Adds max_context_window to model metadata and routes core context-window
    reads through resolved model info. Config model_context_window overrides
    are clamped to max_context_window when present; without an override, the
    model context_window is used.
  • [codex] Add owner nudge app-server API (#18220)
    ## Summary
    
    Second PR in the split from #17956. Stacked on #18227.
    
    - adds app-server v2 protocol/schema support for
    `account/sendAddCreditsNudgeEmail`
    - adds the backend-client `send_add_credits_nudge_email` request and
    request body mapping
    - handles the app-server request with auth checks, backend call, and
    cooldown mapping
    - adds the disabled `workspace_owner_usage_nudge` feature flag and
    focused app-server/backend tests
    
    ## Validation
    
    - `cargo test -p codex-backend-client`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-app-server rate_limits`
    - `cargo test -p codex-tui workspace_`
    - `cargo test -p codex-tui status_`
    - `just fmt`
    - `just fix -p codex-backend-client`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-app-server`
    - `just fix -p codex-tui`
  • [codex] Describe uninstalled cross-repo plugin reads (#18449)
    ## Summary
    - Populate `PluginDetail.description` in core for uninstalled cross-repo
    plugins when detailed fields are unavailable until install.
    - Include the source Git URL plus optional path/ref/sha details in that
    fallback description.
    - Keep `details_unavailable_reason` as the structured signal while
    app-server forwards the description normally.
    - Add plugin-read coverage proving the response does not clone the
    remote source just to show the message.
    
    ## Why
    Uninstalled cross-repo plugins intentionally return sparse detail data
    so listing/reading does not clone the plugin source. Without a
    description, Desktop and TUI detail pages look like an ordinary empty
    plugin. This gives users a concrete explanation and source pointer while
    keeping the existing structured reason available for callers.
    
    ## Validation
    - `just fmt`
    - `cargo test -p codex-core
    read_plugin_for_config_uninstalled_git_source_requires_install_without_cloning`
    - `cargo test -p codex-app-server plugin_read --test all`
    - `just fix -p codex-core`
    - `just fix -p codex-app-server`
    
    Note: `cargo test -p codex-app-server` was also attempted before the
    latest refactor and failed broadly in unrelated v2
    thread/realtime/review/skills suites; the new plugin-read test passed in
    that run as well.
  • feat: Budget skill metadata and surface trimming as a warning (#18298)
    Cap the model-visible skills section to a small share of the context
    window, with a fallback character budget, and keep only as many implicit
    skills as fit within that budget.
    
    Emit a non-fatal warning when enabled skills are omitted, and add a new
    app-server warning notification
    
    Record thread-start skill metrics for total enabled skills, kept skills,
    and whether truncation happened
    
    ---------
    
    Co-authored-by: Matthew Zeng <mzeng@openai.com>
    Co-authored-by: Codex <noreply@openai.com>
  • fix: trust-gate project hooks and exec policies (#14718)
    ## Summary
    - trust-gate project `.codex` layers consistently, including repos that
    have `.codex/hooks.json` or `.codex/execpolicy/*.rules` but no
    `.codex/config.toml`
    - keep disabled project layers in the config stack so nested trusted
    project layers still resolve correctly, while preventing hooks and exec
    policies from loading until the project is trusted
    - update app-server/TUI onboarding copy to make the trust boundary
    explicit and add regressions for loader, hooks, exec-policy, and
    onboarding coverage
    
    ## Security
    Before this change, an untrusted repo could auto-load project hooks or
    exec policies from `.codex/` as long as `config.toml` was absent. This
    makes trust the single gate for project-local config, hooks, and exec
    policies.
    
    ## Stack
    - Parent of #15936
    
    ## Test
    - cargo test -p codex-core without_config_toml
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • feat: Add remote plugin fields to plugin API (#17277)
    ## Summary
    Update the plugin API for the new remote plugin model.
    
    The mental model is no longer “keep local plugin state in sync with
    remote.” Instead, local and remote plugins are becoming separate
    sources. Remote catalog entries can be shown directly from the remote
    API before installation; after installation they are still downloaded
    into the local cache for execution, but remote installed state will come
    from the API and be held in memory rather than being read from config.
    
    • ## API changes
    - Remove `forceRemoteSync` from `plugin/list`, `plugin/install`, and
    `plugin/uninstall`.
      - Remove `remoteSyncError` from `plugin/list`.
      - Add remote-capable metadata to `plugin/list` / `plugin/read`:
        - nullable `marketplaces[].path`
        - `source: { type: "remote", downloadUrl }`
        - URL asset fields alongside local path fields:
      `composerIconUrl`, `logoUrl`, `screenshotUrls`
      - Make `plugin/read` and `plugin/install` source-compatible:
        - `marketplacePath?: AbsolutePathBuf | null`
        - `remoteMarketplaceName?: string | null`
        - exactly one source is required at runtime
  • bazel: use native rust test sharding (#18082)
    ## Why
    
    The large Rust test suites are slow and include some of our flakiest
    tests, so we want to run them with Bazel native sharding while keeping
    shard membership stable between runs.
    
    This is the simpler follow-up to the explicit-label experiment in
    #17998. Since #18397 upgraded Codex to `rules_rs` `0.0.58`, which
    includes the stable test-name hashing support from
    hermeticbuild/rules_rust#14, this PR only needs to wire Codex's Bazel
    macros into that support.
    
    Using native sharding preserves BuildBuddy's sharded-test UI and Bazel's
    per-shard test action caching. Using stable name hashing avoids
    reshuffling every test when one test is added or removed.
    
    ## What Changed
    
    `codex_rust_crate` now accepts `test_shard_counts` and applies the right
    Bazel/rules_rust attributes to generated unit and integration test
    rules. Matched tests are also marked `flaky = True`, giving them Bazel's
    default three attempts.
    
    This PR shards these labels 8 ways:
    
    ```text
    //codex-rs/core:core-all-test
    //codex-rs/core:core-unit-tests
    //codex-rs/app-server:app-server-all-test
    //codex-rs/app-server:app-server-unit-tests
    //codex-rs/tui:tui-unit-tests
    ```
    
    ## Verification
    
    `bazel query --output=build` over the selected public labels and their
    inner unit-test binaries confirmed the expected `shard_count = 8`,
    `flaky = True`, and `experimental_enable_sharding = True` attributes.
    
    Also verified that we see the shards as expected in BuildBuddy so they
    can be analyzed independently.
    
    Co-authored-by: Codex <noreply@openai.com>
  • [codex] Add cross-repo plugin sources to marketplace manifests (#18017)
    ## Summary
    - add first-class marketplace support for git-backed plugin sources
    - keep the newer marketplace parsing behavior from `main`, including
    alternate manifest locations and string local sources
    - materialize remote plugin sources during install, detail reads, and
    non-curated cache refresh
    - expose git plugin source metadata through the app-server protocol
    
    ## Details
    This teaches the marketplace parser to accept all of the following:
    - local string sources such as `"source": "./plugins/foo"`
    - local object sources such as
    `{"source":"local","path":"./plugins/foo"}`
    - remote repo-root sources such as
    `{"source":"url","url":"https://github.com/org/repo.git"}`
    - remote subdir sources such as
    `{"source":"git-subdir","url":"owner/repo","path":"plugins/foo","ref":"main","sha":"..."}`
    
    It also preserves the newer tolerant behavior from `main`: invalid or
    unsupported plugin entries are skipped instead of breaking the whole
    marketplace.
    
    ## Validation
    - `cargo test -p codex-core plugins::marketplace::tests`
    - `just fix -p codex-core`
    - `just fmt`
    
    ## Notes
    - A full `cargo test -p codex-core` run still hit unrelated existing
    failures in agent and multi-agent tests during this session; the
    marketplace-focused suite passed after the rebase resolution.
  • refactor: narrow async lock guard lifetimes (#18211)
    Follow-up to https://github.com/openai/codex/pull/18178, where we called
    out enabling the await-holding lint as a follow-up.
    
    The long-term goal is to enable Clippy coverage for async guards held
    across awaits. This PR is intentionally only the first, low-risk cleanup
    pass: it narrows obvious lock guard lifetimes and leaves
    `codex-rs/Cargo.toml` unchanged so the lint is not enabled until the
    remaining cases are fixed or explicitly justified. It intentionally
    leaves the active-turn/turn-state locking pattern alone because those
    checks and mutations need to stay atomic.
    
    ## Common fixes used here
    
    These are the main patterns reviewers should expect in this PR, and they
    are also the patterns to reach for when fixing future `await_holding_*`
    findings:
    
    - **Scope the guard to the synchronous work.** If the code only needs
    data from a locked value, move the lock into a small block, clone or
    compute the needed values, and do the later `.await` after the block.
    - **Use direct one-line mutations when there is no later await.** Cases
    like `map.lock().await.remove(&id)` are acceptable when the guard is
    only needed for that single mutation and the statement ends before any
    async work.
    - **Drain or clone work out of the lock before notifying or awaiting.**
    For example, the JS REPL drains pending exec senders into a local vector
    and the websocket writer clones buffered envelopes before it serializes
    or sends them.
    - **Use a `Semaphore` only when serialization is intentional across
    async work.** The test serialization guards intentionally span awaited
    setup or execution, so using a semaphore communicates "one at a time"
    without holding a mutex guard.
    - **Remove the mutex when there is only one owner.** The PTY stdin
    writer task owns `stdin` directly; the old `Arc<Mutex<_>>` did not
    protect shared access because nothing else had access to the writer.
    - **Do not split locks that protect an atomic invariant.** This PR
    deliberately leaves active-turn/turn-state paths alone because those
    checks and mutations need to stay atomic. Those cases should be fixed
    separately with a design change or documented with `#[expect]`.
    
    ## What changed
    
    - Narrow scoped async mutex guards in app-server, JS REPL, network
    approval, remote-control websocket, and the RMCP test server.
    - Replace test-only async mutex serialization guards with semaphores
    where the guard intentionally lives across async work.
    - Let the PTY pipe writer task own stdin directly instead of wrapping it
    in an async mutex.
    
    ## Verification
    
    - `just fix -p codex-core -p codex-app-server -p codex-rmcp-client -p
    codex-shell-escalation -p codex-utils-pty -p codex-utils-readiness`
    - `just clippy -p codex-core`
    - `cargo test -p codex-core -p codex-app-server -p codex-rmcp-client -p
    codex-shell-escalation -p codex-utils-pty -p codex-utils-readiness` was
    run; the app-server suite passed, and `codex-core` failed in the local
    sandbox on six otel approval tests plus
    `suite::user_shell_cmd::user_shell_command_does_not_set_network_sandbox_env_var`,
    which appear to depend on local command approval/default rules and
    `CODEX_SANDBOX_NETWORK_DISABLED=1` in this environment.
  • [codex] Propagate rate limit reached type (#18227)
    ## Summary
    
    First PR in the split from #17956.
    
    - adds the core/app-server `RateLimitReachedType` shape
    - maps backend `rate_limit_reached_type` into Codex rate-limit snapshots
    - carries the field through app-server notifications/responses and
    generated schemas
    - updates existing constructors/tests for the new optional field
    
    ## Validation
    
    - `cargo test -p codex-backend-client`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-app-server rate_limits`
    - `cargo test -p codex-tui workspace_`
    - `cargo test -p codex-tui status_`
    - `just fmt`
    - `just fix -p codex-backend-client`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-app-server`
    - `just fix -p codex-tui`
  • Add sorting/backwardsCursor to thread/list and new thread/turns/list api (#17305)
    To improve performance of UI loads from the app, add two main
    improvements:
    1. The `thread/list` api now gets a `sortDirection` request field and a
    `backwardsCursor` to the response, which lets you paginate forwards and
    backwards from a window. This lets you fetch the first few items to
    display immediately while you paginate to fill in history, then can
    paginate "backwards" on future loads to catch up with any changes since
    the last UI load without a full reload of the entire data set.
    2. Added a new `thread/turns/list` api which also has sortDirection and
    backwardsCursor for the same behavior as `thread/list`, allowing you the
    same small-fetch for immediate display followed by background fill-in
    and resync catchup.
  • enable tool search over dynamic tools (#18263)
    ## Summary
    
    - Normalize deferred MCP and dynamic tools into `ToolSearchEntry` values
    before constructing `ToolSearchHandler`.
    - Move the tool-search entry adapter out of `tools/handlers` and into
    `tools/tool_search_entry.rs` so the handlers directory stays focused on
    handlers.
    - Keep `ToolSearchHandler` operating over one generic entry list for
    BM25 search, namespace grouping, and per-bucket default limits.
    
    ## Why
    
    Follow-up cleanup for #17849. The dynamic tool-search support made the
    handler juggle source-specific MCP and dynamic tool lists, index
    arithmetic, output conversion, and namespace emission. This keeps source
    adaptation outside the handler so the search loop itself is smaller and
    source-agnostic.
    
    ## Validation
    
    - `just fmt`
    - `cargo test -p codex-core tools::handlers::tool_search::tests`
    - `git diff --check`
    - `cargo test -p codex-core` currently fails in unrelated
    `plugins::manager::tests::list_marketplaces_ignores_installed_roots_missing_from_config`;
    rerunning that single test fails the same way at
    `core/src/plugins/manager_tests.rs:1692`.
    
    ---------
    
    Co-authored-by: pash <pash@openai.com>
  • codex: route thread/read persistence through thread store (#18352)
    Summary
    - replace the thread/read persisted-load helper with
    ThreadStore::read_thread
    - move SQLite/rollout summary, name, fork metadata, and history loading
    for persisted reads into LocalThreadStore
    - leave getConversationSummary unchanged for a later PR
    
    Context
    - Replaces closed stacked PR #18232 after PR #18231 merged and its base
    branch was deleted.
  • Guardian -> Auto-Review (#18021)
    This PR is a user-facing change for our rebranding of guardian to
    auto-review.
  • feat(config): support managed deny-read requirements (#17740)
    ## Summary
    - adds managed requirements support for deny-read filesystem entries
    - constrains config layers so managed deny-read requirements cannot be
    widened by user-controlled config
    - surfaces managed deny-read requirements through debug/config plumbing
    
    This PR lets managed requirements inject deny-read filesystem
    constraints into the effective filesystem sandbox policy.
    User-controlled config can still choose the surrounding permission
    profile, but it cannot remove or weaken the managed deny-read entries.
    
    ## Managed deny-read shape
    A managed requirements file can declare exact paths and glob patterns
    under `[permissions.filesystem]`:
    
    ```toml
    # /etc/codex/requirements.toml
    [permissions.filesystem]
    deny_read = [
      "/Users/alice/.gitconfig",
      "/Users/alice/.ssh",
      "./managed-private/**/*.env",
    ]
    ```
    
    Those entries are compiled into the effective filesystem policy as
    `access = none` rules, equivalent in shape to filesystem permission
    entries like:
    
    ```toml
    [permissions.workspace.filesystem]
    "/Users/alice/.gitconfig" = "none"
    "/Users/alice/.ssh" = "none"
    "/absolute/path/to/managed-private/**/*.env" = "none"
    ```
    
    The important difference is that the managed entries come from
    requirements, so lower-precedence user config cannot remove them or make
    those paths readable again.
    
    Relative managed `deny_read` entries are resolved relative to the
    directory containing the managed requirements file. Glob entries keep
    their glob suffix after the non-glob prefix is normalized.
    
    ## Runtime behavior
    - Managed `deny_read` entries are appended to the effective
    `FileSystemSandboxPolicy` after the selected permission profile is
    resolved.
    - Exact paths become `FileSystemPath::Path { access: None }`; glob
    patterns become `FileSystemPath::GlobPattern { access: None }`.
    - When managed deny-read entries are present, `sandbox_mode` is
    constrained to `read-only` or `workspace-write`; `danger-full-access`
    and `external-sandbox` cannot silently bypass the managed read-deny
    policy.
    - On Windows, the managed deny-read policy is enforced for direct file
    tools, but shell subprocess reads are not sandboxed yet, so startup
    emits a warning for that platform.
    - `/debug-config` shows the effective managed requirement as
    `permissions.filesystem.deny_read` with its source.
    
    ## Stack
    1. #15979 - glob deny-read policy/config/direct-tool support
    2. #18096 - macOS and Linux sandbox enforcement
    3. This PR - managed deny-read requirements
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Sync local plugin imports, async remote imports, refresh caches after… (#18246)
    … import
    
    ## Why
    
    `externalAgentConfig/import` used to spawn plugin imports in the
    background and return immediately. That meant local marketplace imports
    could still be in flight when the caller refreshed plugin state, so
    newly imported plugins would not show up right away.
    
    This change makes local marketplace imports complete before the RPC
    returns, while keeping remote marketplace imports asynchronous so we do
    not block on remote fetches.
    
    ## What changed
    
    - split plugin migration details into local and remote marketplace
    imports based on the external config source
    - import local marketplaces synchronously during
    `externalAgentConfig/import`
    - return pending remote plugin imports to the app-server so it can
    finish them in the background
    - clear the plugin and skills caches before responding to plugin
    imports, and again after background remote imports complete, so the next
    `plugin/list` reloads fresh state
    - keep marketplace source parsing encapsulated behind
    `is_local_marketplace_source(...)` instead of re-exporting the internal
    enum
    - add core and app-server coverage for the synchronous local import path
    and the pending remote import path
    
    ## Verification
    
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-core` (currently fails an existing unrelated
    test:
    `config_loader::tests::cli_override_can_update_project_local_mcp_server_when_project_is_trusted`)
    - `cargo test` (currently fails existing `codex-app-server` integration
    tests in MCP/skills/thread-start areas, plus the unrelated `codex-core`
    failure above)
  • [codex] Revoke ChatGPT tokens on logout (#17825)
    ## Summary
    
    This changes Codex logout so managed ChatGPT auth is revoked against
    AuthAPI before local auth state is removed. CLI logout, TUI `/logout`,
    and the app-server account logout path now use the token-revoking logout
    flow instead of only deleting `auth.json` / credential store state.
    
    ## Root Cause
    
    Logout previously cleared only local auth storage. That removed Codex's
    local credentials but did not ask the backend to invalidate the
    refresh/access token state associated with a managed ChatGPT login.
    
    ## Behavior
    
    For managed ChatGPT auth, logout sends the stored refresh token to
    `https://auth.openai.com/oauth/revoke` with `token_type_hint:
    refresh_token` and the Codex OAuth client id, then deletes all local
    auth stores after revocation succeeds. If only an access token is
    available, it falls back to revoking that access token. API key auth and
    externally supplied `chatgptAuthTokens` are still only cleared locally
    because Codex does not own a refresh token for those modes.
    
    Revocation failures are fail-closed: if Codex cannot load stored auth or
    the backend revoke call fails, logout returns an error and leaves local
    auth in place so the user can retry instead of silently clearing local
    state while backend tokens remain valid.
    
    ## Validation
    ran local version of `codex-cli` with staging overrides/harness for auth
    
    ran `codex login` then `codex logout`:
    
    saw auth.json clear and  backend revocation endpoints were called
    
    ```
    POST /oauth/revoke
    status: 200
    
    revoking access token
    should clear auth session
    clearing auth session due to token revocation
    successfully revoked session and access token
    CANONICAL-API-LINE Response: status='200' method='POST' path='/oauth/revoke
    ```
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • codex: split thread/read view loading (#18231)
    Summary
    - refactor thread/read into explicit persisted-load, live-load, and
    merge steps
    - preserve existing SQLite/filesystem/live-thread behavior exactly
    - keep ThreadStore migration out of this PR so the next PR is easier to
    review
    
    Validation
    - this one's a pure reorganization that relies on existing test coverage
  • feat: Handle alternate plugin manifest paths (#18182)
    Load plugin manifests through a shared discoverable-path helper so
    manifest reads, installs, and skill names all see the same alternate
    manifest location.
  • Refactor config loading to use filesystem abstraction (#18209)
    Initial pass propagating FileSystem through config loading.
  • Use in-process app-server for unknown-thread MCP read test (#18196)
    ## Summary
    - Switch the unknown-thread MCP resource read test from the stdio
    subprocess to the in-process app-server path.
    - Keep the assertion focused on the returned error message while
    avoiding child-process teardown timing issues in nextest.
    
    ## Testing
    - Not run (not requested)
  • [codex] Update realtime V2 VAD silence delay and 1.5 prompt (#18092)
    ## Summary
    
    - set the realtime v2 server VAD silence delay to 500ms
    - update the default realtime 1.5 backend prompt to the v4 text
    - keep the session payload and prompt rendering tests aligned with those
    changes
    
    ## Why
    
    - the VAD change gives the voice path a longer pause before ending the
    user's turn
    - the prompt change makes the default bundled realtime prompt match the
    current v4 content
    
    ## Validation
    
    - `cargo +1.93.0 test -p codex-core realtime_prompt --manifest-path
    /tmp/codex-realtime-v2-vad-prompt-v4/codex-rs/Cargo.toml`
    - `CARGO_TARGET_DIR=/tmp/codex-pr-v4-target cargo +1.93.0 test -p
    codex-api
    realtime_v2_session_update_includes_background_agent_tool_and_handoff_output_item
    --manifest-path
    /tmp/codex-realtime-v2-vad-prompt-v4/codex-rs/Cargo.toml`
    - `CARGO_TARGET_DIR=/tmp/codex-pr-v4-target cargo +1.93.0 test -p
    codex-app-server --test all
    'suite::v2::realtime_conversation::realtime_webrtc_start_emits_sdp_notification'
    --manifest-path /tmp/codex-realtime-v2-vad-prompt-v4/codex-rs/Cargo.toml
    -- --exact`
  • Register agent tasks behind use_agent_identity (#17387)
    ## Summary
    
    Stack PR3 for feature-gated agent identity support.
    
    This PR adds per-thread agent task registration behind
    `features.use_agent_identity`. Tasks are minted on the first real user
    turn and cached in thread runtime state for later turns.
    
    ## Stack
    
    - PR1: https://github.com/openai/codex/pull/17385 - add
    `features.use_agent_identity`
    - PR2: https://github.com/openai/codex/pull/17386 - register agent
    identities when enabled
    - PR3: https://github.com/openai/codex/pull/17387 - this PR, original
    task registration slice
    - PR3.1: https://github.com/openai/codex/pull/17978 - persist and
    prewarm registered tasks per thread
    - PR4: https://github.com/openai/codex/pull/17980 - use `AgentAssertion`
    downstream when enabled
    
    ## Validation
    
    Covered as part of the local stack validation pass:
    
    - `just fmt`
    - `cargo test -p codex-core --lib agent_identity`
    - `cargo test -p codex-core --lib agent_assertion`
    - `cargo test -p codex-core --lib websocket_agent_task`
    - `cargo test -p codex-api api_bridge`
    - `cargo build -p codex-cli --bin codex`
    
    ## Notes
    
    The full local app-server E2E path is still being debugged after PR
    creation. The current branch stack is directionally ready for review
    while that follow-up continues.
  • fix(app-server): replay token usage after resume and fork (#18023)
    ## Problem
    
    When a user resumed or forked a session, the TUI could render the
    restored thread history immediately, but it did not receive token usage
    until a later model turn emitted a fresh usage event. That left the
    context/status UI blank or stale during the exact window where the user
    expects resumed state to look complete. Core already reconstructed token
    usage from the rollout; the missing behavior was app-server lifecycle
    replay to the client that just attached.
    
    ## Mental model
    
    Token usage has two representations. The rollout is the durable source
    of historical `TokenCount` events, and the core session cache is the
    in-memory snapshot reconstructed from that rollout on resume or fork.
    App-server v2 clients do not read core state directly; they learn about
    usage through `thread/tokenUsage/updated`. The fix keeps those roles
    separate: core exposes the restored `TokenUsageInfo`, and app-server
    sends one targeted notification after a successful `thread/resume` or
    `thread/fork` response when that restored snapshot exists.
    
    This notification is not a new model event. It is a replay of
    already-persisted state for the client that just attached. That
    distinction matters because using the normal core event path here would
    risk duplicating `TokenCount` entries in the rollout and making future
    resumes count historical usage twice.
    
    ## Non-goals
    
    This change does not add a new protocol method or payload shape. It
    reuses the existing v2 `thread/tokenUsage/updated` notification and the
    TUI’s existing handler for that notification.
    
    This change does not alter how token usage is computed, accumulated,
    compacted, or written during turns. It only exposes the token usage that
    resume and fork reconstruction already restored.
    
    This change does not broadcast historical usage replay to every
    subscribed client. The replay is intentionally scoped to the connection
    that requested resume or fork so already-attached clients are not
    surprised by an old usage update while they may be rendering live
    activity.
    
    ## Tradeoffs
    
    Sending the usage notification after the JSON-RPC response preserves a
    clear lifecycle order: the client first receives the thread object, then
    receives restored usage for that thread. The tradeoff is that usage is
    still a notification rather than part of the `thread/resume` or
    `thread/fork` response. That keeps the protocol shape stable and avoids
    duplicating usage fields across response types, but clients must
    continue listening for notifications after receiving the response.
    
    The helper selects the latest non-in-progress turn id for the replayed
    usage notification. This is conservative because restored usage belongs
    to completed persisted accounting, not to newly attached in-flight work.
    The fallback to the last turn preserves a stable wire payload for
    unusual histories, but histories with no meaningful completed turn still
    have a weak attribution story.
    
    ## Architecture
    
    Core already seeds `Session` token state from the last persisted rollout
    `TokenCount` during `InitialHistory::Resumed` and
    `InitialHistory::Forked`. The new core accessor exposes the complete
    `TokenUsageInfo` through `CodexThread` without giving app-server direct
    session mutation authority.
    
    App-server calls that accessor from three lifecycle paths: cold
    `thread/resume`, running-thread resume/rejoin, and `thread/fork`. In
    each path, the server sends the normal response first, then calls a
    shared helper that converts core usage into
    `ThreadTokenUsageUpdatedNotification` and sends it only to the
    requesting connection.
    
    The tests build fake rollouts with a user turn plus a persisted token
    usage event. They then exercise `thread/resume` and `thread/fork`
    without starting another model turn, proving that restored usage arrives
    before any next-turn token event could be produced.
    
    ## Observability
    
    The primary debug path is the app-server JSON-RPC stream. After
    `thread/resume` or `thread/fork`, a client should see the response
    followed by `thread/tokenUsage/updated` when the source rollout includes
    token usage. If the notification is absent, check whether the rollout
    contains an `event_msg` payload of type `token_count`, whether core
    reconstruction seeded `Session::token_usage_info`, and whether the
    connection stayed attached long enough to receive the targeted
    notification.
    
    The notification is sent through the existing
    `OutgoingMessageSender::send_server_notification_to_connections` path,
    so existing app-server tracing around server notifications still
    applies. Because this is a replay, not a model turn event, debugging
    should start at the resume/fork handlers rather than the turn event
    translation in `bespoke_event_handling`.
    
    ## Tests
    
    The focused regression coverage is `cargo test -p codex-app-server
    emits_restored_token_usage`, which covers both resume and fork. The core
    reconstruction guard is `cargo test -p codex-core
    record_initial_history_seeds_token_info_from_rollout`.
    
    Formatting and lint/fix passes were run with `just fmt`, `just fix -p
    codex-core`, and `just fix -p codex-app-server`. Full crate test runs
    surfaced pre-existing unrelated failures in command execution and plugin
    marketplace tests; the new token usage tests passed in focused runs and
    within the app-server suite before the unrelated command execution
    failure.
  • Make thread unsubscribe test deterministic (#18000)
    ## Summary
    - replace the unsubscribe-during-turn test's sleep/polling flow with a
    gated streaming SSE response
    - add request-count notification support to the streaming SSE test
    server so the test can wait for the in-flight Responses request
    deterministically
    
    ## Scope
    - codex-rs/app-server/tests/suite/v2/thread_unsubscribe.rs
    - codex-rs/core/tests/common/streaming_sse.rs
    
    ## Validation
    - Not run locally; this is a narrow extraction from the prior CI-green
    branch.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Refactor AGENTS.md discovery into AgentsMdManager (#18035)
    Encapsulate Agents MD processing a bit and drop user_instructions_path
    from config.
  • Auto-upgrade configured marketplaces (#17425)
    ## Summary
    - Add best-effort auto-upgrade for user-configured Git marketplaces
    recorded in `config.toml`.
    - Track the last activated Git revision with `last_revision` so
    unchanged marketplace sources skip clone work.
    - Trigger the upgrade from plugin startup and `plugin/list`, while
    preserving existing fail-open plugin behavior with warning logs rather
    than new user-visible errors.
    
    ## Details
    - Remote configured marketplaces use `git ls-remote` to compare the
    source/ref against the recorded revision.
    - Upgrades clone into a staging directory, validate that
    `.agents/plugins/marketplace.json` exists and that the manifest name
    matches the configured marketplace key, then atomically activate the new
    root.
    - Local `.agents/plugins/marketplace.json` marketplaces remain live
    filesystem state and are not auto-pulled.
    - Existing non-curated plugin cache refresh is kicked after successful
    marketplace root upgrades.
    
    ## Validation
    - `just write-config-schema`
    - `cargo test -p codex-core marketplace_upgrade`
    - `cargo check -p codex-cli -p codex-app-server`
    - `just fix -p codex-core`
    
    Did not run the complete `cargo test` suite because the repo
    instructions require asking before a full core workspace run.
  • fix: auth preflight (#18117)
    Fix app-server startup when `remote_control = true` is enabled without
    ChatGPT auth.
    
    Remote control now starts in a degraded/retrying state instead of
    failing app-server initialization, so Desktop is not stranded before the
    initial initialize handshake.
  • chore: unify memory drop endpoints (#18134)
    Unify all the memories drop behind a single implementation that drops
    both the main memories and the extensions
  • debug: windows flake (#18135)
    Make sure Bazel logs shows every errors so that we can debug flakes +
    fix a small flake on Windows by updating the sleep command to a
    `Start-Sleep` instead of a PowerShell nested command (otherwise we had
    double nesting which is absurdely slow)
  • Fix MCP startup cancellation through app server (#18078)
    Addresses https://github.com/openai/codex/issues/17143
    
    Problem: TUI interrupts without an active turn stopped cancelling slow
    MCP startup after routing through the app-server APIs.
    
    Solution: Route no-active-turn interrupts through app-server as startup
    cancels, acknowledge them immediately, and emit cancelled MCP startup
    updates.
    
    Testing: I manually confirmed that MCP cancellation didn't work prior to
    this PR and works after the fix was in place.
  • Extract plugin loading and marketplace logic into codex-core-plugins (#18070)
    Split plugin loading, marketplace, and related infrastructure out of
    core into codex-core-plugins, while keeping the core-facing
    configuration and orchestration flow in codex-core.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [codex][mcp] Add resource uri meta to tool call item. (#17831)
    - [x] Add resource uri meta to tool call item so that the app-server
    client can start prefetching resources immediately without loading mcp
    server status.
  • Async config loading (#18022)
    Parts of config will come from executor. Prepare for that by making
    config loading methods async.
  • [codex] Make command exec delta tests chunk tolerant (#17999)
    ## Summary
    - Make command/exec output-delta tests accumulate streamed chunks
    instead of assuming complete logical output in a single notification.
    - Collect stdout and stderr independently so stream interleaving does
    not fail the pipe streaming test.
    
    ## Why
    The command/exec protocol exposes output as deltas, so tests should not
    rely on chunk boundaries being stable. A line like `out-start\n` may
    arrive split across multiple notifications, and stdout/stderr
    notifications may interleave.
    
    ## Validation
    - `just fmt`
    - `git diff --check`
    - `cargo test -p codex-app-server suite::v2::command_exec`
  • Clarify realtime v2 context and handoff messages (#17896)
    ## Summary
    - wrap realtime startup context in
    `<startup_context>...</startup_context>` tags
    - prefix V2 mirrored user text and relayed backend text with `[USER]` /
    `[BACKEND]`
    - remove the V2 progress suffix and replace the final V2 handoff output
    with a short completion acknowledgement while preserving the existing V1
    wrapper
    
    ## Testing
    - cargo test -p codex-api
    realtime_v2_session_update_includes_background_agent_tool_and_handoff_output_item
    -- --exact
    - cargo test -p codex-app-server webrtc_v2_background_agent_
    - cargo test -p codex-app-server webrtc_v2_text_input_is_
    - cargo test -p codex-core conversation_user_text_turn_is_
  • [docs] Revert extra changes from PR 17848 (#18003)
    ## Summary
    
    1. Revert https://github.com/openai/codex/pull/17848 so the Bazel and
    `BUILD` file changes leave `main`.
    2. Prepare for a narrower follow up that restores only `SECURITY.md`.
    
    ## Validation
    
    1. Reviewed the revert diff against `main`.
    2. Ran a clean diff check before push.
  • Dismiss stale app-server requests after remote resolution (#15134)
    Dismiss stale TUI app-server approvals after remote resolution
    
    When an approval, user-input prompt, or elicitation request is resolved
    by another client, the TUI now dismisses the matching local UI instead
    of leaving stale prompts behind and emitting a misleading local
    cancellation.
    
    This change teaches pending app-server request tracking to map
    `serverRequest/resolved` notifications back to the concrete request type
    and stable request key, then propagates that resolved request into TUI
    prompt state. Approval, request-user-input, and MCP elicitation overlays
    now drop the resolved current or queued request quietly, advance to the
    next queued request when present, and avoid emitting abort/cancel events
    for stale UI.
    
    The latest update also retires matching prompts while they are still
    deferred behind active streaming and suppresses buffered active-thread
    requests whose app-server request id has already been resolved before
    drain. `ChatWidget` removes a resolved request from both the deferred
    interrupt queue and the materialized bottom-pane stack, while
    active-thread request handling verifies the app-server request is still
    pending before showing a prompt. Lifecycle events such as exec begin/end
    remain queued so approved work can still render normally.
    
    Tests cover resolved-request mapping, overlay dismissal behavior,
    deferred prompt pruning for same-turn user input, exec approval IDs,
    lifecycle-event retention, and the buffered active-thread ordering
    regression.
    
    Validation:
    - `just fmt`
    - `git diff --check`
    - `cargo test -p codex-tui
    resolved_buffered_approval_does_not_become_actionable_after_drain`
    - `cargo test -p codex-tui
    enqueue_primary_thread_session_replays_buffered_approval_after_attach`
    - `cargo test -p codex-tui chatwidget::interrupts`
    - `just fix -p codex-tui`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Migrate archive/unarchive to local ThreadStore (#17892)
    # Summary
    - implement local ThreadStore archive/unarchive operations
    - implement local ThreadStore read_thread operation
    - break up the various ThreadStore local method implementations into
    separate files
    - migrate app-server archive/unarchive and core archive fixture to use
    ThreadStore (but not all read operations yet!)
    - use the ThreadStore's read operation as a proxy check for thread
    persistence/existence in the app server code
    - move all other filesystem operations related to archive (path
    validation etc) into the local thread store.
    
    # Tests
    - add dedicated local store archive/unarchive tests
  • [docs] Add security boundaries reference in SECURITY.md (#17848)
    ## Summary
    1. Add a Security Boundaries section to `SECURITY.md`.
    2. Point readers to the Codex Agent approvals and security documentation
    for sandboxing, approvals, and network controls.
    
    ## Validation
    1. Reviewed the `SECURITY.md` diff in a clean worktree.
    2. No tests run. Docs only change.
  • app-server: track remote-control seq IDs per stream (#17902)
    ## Summary
    
    - Track outbound remote-control sequence IDs independently for each
    client stream.
    - Retain unacked outbound messages per stream using FIFO buffers.
    - Require stream-scoped acks and update tests for contiguous per-stream
    sequencing.
    
    ## Why
    
    The remote-control peer uses outbound sequence gaps to detect lost
    messages and re-initialize. A single global outbound sequence counter
    can create apparent gaps on an individual stream when another stream
    receives an interleaved message.
    
    ## Validation
    
    - `just fmt`
    - `cargo test -p codex-app-server remote_control`
    - `just fix -p codex-app-server`
    - `git diff --check`
  • [codex] Add local thread store listing (#17824)
    Builds on top of #17659 
    
    Move the filesystem + sqlite thread listing-related operations inside of
    a local ThreadStore implementation and call ThreadStore from the places
    that used to perform these filesystem/sqlite operations.
    
    This is the first of a series of PRs that will implement the rest of the
    local ThreadStore.
    
    Testing:
    - added unit tests for the thread store implementation
    - adjusted some unit tests in the realtime + personality packages whose
    callsites changed. Specifically I'm trying to hide ThreadMetadata inside
    of the local implementation and make ThreadMetadata a sqlite
    implementation detail concern rather than a public interface, preferring
    the more generate StoredThread interface instead
    - added a corner case test for the personality migration package that
    wasn't covered by the existing test suite
    - adjust the behavior of searched thread listing to run the existing
    local rollout repair/backfill pass _before_ querying SQLite results, so
    callers using ThreadStore::list_threads do not miss matches after a
    partial metadata warm-up
  • Register agent identities behind use_agent_identity (#17386)
    ## Summary
    
    Stack PR 2 of 4 for feature-gated agent identity support.
    
    This PR adds agent identity registration behind
    `features.use_agent_identity`. It keeps the app-server protocol
    unchanged and starts registration after ChatGPT auth exists rather than
    requiring a client restart.
    
    ## Stack
    
    - PR1: https://github.com/openai/codex/pull/17385 - add
    `features.use_agent_identity`
    - PR2: https://github.com/openai/codex/pull/17386 - this PR
    - PR3: https://github.com/openai/codex/pull/17387 - register agent tasks
    when enabled
    - PR4: https://github.com/openai/codex/pull/17388 - use `AgentAssertion`
    downstream when enabled
    
    ## Validation
    
    Covered as part of the local stack validation pass:
    
    - `just fmt`
    - `cargo test -p codex-core --lib agent_identity`
    - `cargo test -p codex-core --lib agent_assertion`
    - `cargo test -p codex-core --lib websocket_agent_task`
    - `cargo test -p codex-api api_bridge`
    - `cargo build -p codex-cli --bin codex`
    
    ## Notes
    
    The full local app-server E2E path is still being debugged after PR
    creation. The current branch stack is directionally ready for review
    while that follow-up continues.
  • register all mcp tools with namespace (#17404)
    stacked on #17402.
    
    MCP tools returned by `tool_search` (deferred tools) get registered in
    our `ToolRegistry` with a different format than directly available
    tools. this leads to two different ways of accessing MCP tools from our
    tool catalog, only one of which works for each. fix this by registering
    all MCP tools with the namespace format, since this info is already
    available.
    
    also, direct MCP tools are registered to responsesapi without a
    namespace, while deferred MCP tools have a namespace. this means we can
    receive MCP `FunctionCall`s in both formats from namespaces. fix this by
    always registering MCP tools with namespace, regardless of deferral
    status.
    
    make code mode track `ToolName` provenance of tools so it can map the
    literal JS function name string to the correct `ToolName` for
    invocation, rather than supporting both in core.
    
    this lets us unify to a single canonical `ToolName` representation for
    each MCP tool and force everywhere to use that one, without supporting
    fallbacks.