Commit Graph

269 Commits

  • feat(auto-review) Handle request_permissions calls (#18393)
    ## Summary
    When auto-review is enabled, it should handle request_permissions tool.
    We'll need to clean up the UX but I'm planning to do that in a separate
    pass
    
    ## Testing
    - [x] Ran locally
    <img width="893" height="396" alt="Screenshot 2026-04-17 at 1 16 13 PM"
    src="https://github.com/user-attachments/assets/4c045c5f-1138-4c6c-ac6e-2cb6be4514d8"
    />
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add session config loader interface (#18208)
    ## Why
    
    Cloud-hosted sessions need a way for the service that starts or manages
    a thread to provide session-owned config without treating all config as
    if it came from the same user/project/workspace TOML stack.
    
    The important boundary is ownership: some values should be controlled by
    the session/orchestrator, some by the authenticated user, and later some
    may come from the executor. The earlier broad config-store shape made
    that boundary too fuzzy and overlapped heavily with the existing
    filesystem-backed config loader. This PR starts with the smaller piece
    we need now: a typed session config loader that can feed the existing
    config layer stack while preserving the normal precedence and merge
    behavior.
    
    ## What Changed
    
    - Added `ThreadConfigLoader` and related typed payloads in
    `codex-config`.
    - `SessionThreadConfig` currently supports `model_provider`,
    `model_providers`, and feature flags.
    - `UserThreadConfig` is present as an ownership boundary, but does not
    yet add TOML-backed fields.
    - `NoopThreadConfigLoader` preserves existing behavior when no external
    loader is configured.
      - `StaticThreadConfigLoader` supports tests and simple callers.
    
    - Taught thread config sources to produce ordinary `ConfigLayerEntry`
    values so the existing `ConfigLayerStack` remains the place where
    precedence and merging happen.
    
    - Wired the loader through `ConfigBuilder`, the config loader, and
    app-server startup paths so app-server can provide session-owned config
    before deriving a thread config.
    
    - Added coverage for:
      - translating typed thread config into config layers,
    - inserting thread config layers into the stack at the right precedence,
    - applying session-provided model provider and feature settings when
    app-server derives config from thread params.
    
    ## Follow-Ups
    
    This intentionally stops short of adding the remote/service transport.
    The next pieces are expected to be:
    
    1. Define the proto/API shape for this interface.
    2. Add a client implementation that can source session config from the
    service side.
    
    ## Verification
    
    - Added unit coverage in `codex-config` for the loader and layer
    conversion.
    - Added `codex-core` config loader coverage for thread config layer
    precedence.
    - Added app-server coverage that verifies session thread config wins
    over request-provided config for model provider and feature settings.
  • Wire the PatchUpdated events through app_server (#18289)
    Wires patch_updated events through app_server. These events are parsed
    and streamed while apply_patch is being written by the model. Also adds 500ms of buffering to the patch_updated events in the diff_consumer.
    
    The eventual goal is to use this to display better progress indicators in
    the codex app.
  • Update models.json (#18586)
    - Replace the active models-manager catalog with the deleted core
    catalog contents.
    - Replace stale hardcoded test model slugs with current bundled model
    slugs.
    - Keep this as a stacked change on top of the cleanup PR.
  • refactor: use semaphores for async serialization gates (#18403)
    This is the second cleanup in the await-holding lint stack. The
    higher-level goal, following https://github.com/openai/codex/pull/18178
    and https://github.com/openai/codex/pull/18398, is to enable Clippy
    coverage for guards held across `.await` points without carrying broad
    suppressions.
    
    The stack is working toward enabling Clippy's
    [`await_holding_lock`](https://rust-lang.github.io/rust-clippy/master/index.html#await_holding_lock)
    lint and the configurable
    [`await_holding_invalid_type`](https://rust-lang.github.io/rust-clippy/master/index.html#await_holding_invalid_type)
    lint for Tokio guard types.
    
    Several existing fields used `tokio::sync::Mutex<()>` only as
    one-at-a-time async gates. Those guards intentionally lived across
    `.await` while an operation was serialized. A mutex over `()` suggests
    protected data and trips the await-holding lint shape; a single-permit
    `tokio::sync::Semaphore` expresses the intended serialization directly.
    
    ## What changed
    
    - Replace `Mutex<()>` serialization gates with `Semaphore::new(1)` for
    agent identity ensure, exec policy updates, guardian review session
    reuse, plugin remote sync, managed network proxy refresh, auth token
    refresh, and RMCP session recovery.
    - Update call sites from `lock().await` / `try_lock()` to
    `acquire().await` / `try_acquire()`.
    - Map closed-semaphore errors into the existing local error types, even
    though these semaphores are owned for the lifetime of their managers.
    - Update session test builders for the new
    `managed_network_proxy_refresh_lock` type.
    
    ## Verification
    
    - The split stack was verified at the final lint-enabling head with
    `just clippy`.
    
    
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/18403).
    * #18698
    * #18423
    * #18418
    * __->__ #18403
  • [codex] Use background agent task auth for backend calls (#18094)
    ## Summary
    
    Introduces a single background/control-plane agent task for ChatGPT
    backend requests that do not have a thread-scoped task, with
    `AuthManager` owning the default ChatGPT backend authorization decision.
    
    Callers now ask `AuthManager` for the default ChatGPT backend
    authorization header. `AuthManager` decides whether that is bearer or
    background AgentAssertion based on config/internal state, while
    low-level bootstrap paths can explicitly request bearer-only auth.
    
    This PR is stacked on PR4 and focuses on the shared background task auth
    plumbing plus the first tranche of backend/control-plane consumers. The
    remaining callsite wiring is split into PR4.2 to keep review size down.
    
    ## Stack
    
    - PR1: https://github.com/openai/codex/pull/17385 - add
    `features.use_agent_identity`
    - PR2: https://github.com/openai/codex/pull/17386 - register agent
    identities when enabled
    - PR3: https://github.com/openai/codex/pull/17387 - register agent tasks
    when enabled
    - PR3.1: https://github.com/openai/codex/pull/17978 - persist and
    prewarm registered tasks per thread
    - PR4: https://github.com/openai/codex/pull/17980 - use task-scoped
    `AgentAssertion` for downstream calls
    - PR4.1: this PR - introduce AuthManager-owned background/control-plane
    `AgentAssertion` auth
    - PR4.2: https://github.com/openai/codex/pull/18260 - use background
    task auth for additional backend/control-plane calls
    
    ## What Changed
    
    - add background task registration and assertion minting inside
    `codex-login`
    - persist `agent_identity.background_task_id` separately from
    per-session task state
    - make `BackgroundAgentTaskManager` private to `codex-login`; call sites
    do not instantiate or pass it around
    - teach `AuthManager` the ChatGPT backend base URL and feature-derived
    background auth mode from resolved config
    - expose bearer-only helpers for bootstrap/registration/refresh-style
    paths that must not use AgentAssertion
    - wire `AuthManager` default ChatGPT authorization through app listing,
    connector directory listing, remote plugins, MCP status/listing,
    analytics, and core-skills remote calls
    - preserve bearer fallback when the feature is disabled, the backend
    host is unsupported, or background task registration is not available
    
    ## Validation
    
    - `just fmt`
    - `cargo check -p codex-core -p codex-login -p codex-analytics -p
    codex-app-server -p codex-cloud-requirements -p codex-cloud-tasks -p
    codex-models-manager -p codex-chatgpt -p codex-model-provider -p
    codex-mcp -p codex-core-skills`
    - `cargo test -p codex-login agent_identity`
    - `cargo test -p codex-model-provider bearer_auth_provider`
    - `cargo test -p codex-core agent_assertion`
    - `cargo test -p codex-app-server remote_control`
    - `cargo test -p codex-cloud-requirements fetch_cloud_requirements`
    - `cargo test -p codex-models-manager manager::tests`
    - `cargo test -p codex-chatgpt`
    - `cargo test -p codex-cloud-tasks`
    - `just fix -p codex-core -p codex-login -p codex-analytics -p
    codex-app-server -p codex-cloud-requirements -p codex-cloud-tasks -p
    codex-models-manager -p codex-chatgpt -p codex-model-provider -p
    codex-mcp -p codex-core-skills`
    - `just fix -p codex-app-server`
    - `git diff --check`
  • feat: add metric to track the number of turns with memory usage (#18662)
    Add a metric `codex.turn.memory` to know if a turn used memories or not.
    This is not part of the other turn metrics as a label to limit
    cardinality
  • [codex] Use AgentAssertion downstream behind use_agent_identity (#17980)
    ## Summary
    
    This is the AgentAssertion downstream slice for feature-gated agent
    identity support, replacing the oversized AgentAssertion slice from PR
    #17807.
    
    It isolates task-scoped downstream AgentAssertion wiring on top of the
    merged PR3.1 work without re-carrying the earlier agent registration,
    task registration, or task-state history.
    
    This PR includes the task-scoped bug-fix call sites from the review:
    generic file upload auth, MCP OpenAI file upload auth, and ARC monitor
    auth. Broader user/control-plane calls move to PR4.1 and PR4.2.
    
    ## Stack
    
    - PR1: https://github.com/openai/codex/pull/17385 - add
    `features.use_agent_identity`
    - PR2: https://github.com/openai/codex/pull/17386 - register agent
    identities when enabled
    - PR3: https://github.com/openai/codex/pull/17387 - register agent tasks
    when enabled
    - PR3.1: https://github.com/openai/codex/pull/17978 - persist and
    prewarm registered tasks per thread
    - PR4: this PR - use task-scoped `AgentAssertion` downstream when
    enabled
    - PR4.1: https://github.com/openai/codex/pull/18094 - introduce
    AuthManager-owned background/control-plane `AgentAssertion` auth
    - PR4.2: https://github.com/openai/codex/pull/18260 - use background
    task auth for additional backend/control-plane calls
    
    ## What Changed
    
    - add AgentAssertion envelope generation in `codex-core`
    - route downstream HTTP and websocket auth through AgentAssertion when
    an agent task is present
    - extend the model-provider auth provider so non-bearer authorization
    schemes can be passed through cleanly
    - make generic file uploads attach the full authorization header value
    - make MCP OpenAI file uploads use the cached thread agent task
    assertion when present
    - make ARC monitor calls use the cached thread agent task assertion when
    present
    
    ## Why
    
    The original PR had drifted ancestry and showed a much larger diff than
    the semantic change actually required. Restacking it onto PR3.1 keeps
    the reviewable surface down to the downstream assertion slice.
    
    ## Validation
    
    - `just fmt`
    - `cargo check -p codex-core -p codex-login -p codex-analytics -p
    codex-app-server -p codex-cloud-requirements -p codex-cloud-tasks -p
    codex-models-manager -p codex-chatgpt -p codex-model-provider -p
    codex-mcp -p codex-core-skills`
    - `cargo test -p codex-model-provider bearer_auth_provider`
    - `cargo test -p codex-core agent_assertion`
    - `cargo test -p codex-app-server remote_control`
    - `cargo test -p codex-cloud-requirements fetch_cloud_requirements`
    - `cargo test -p codex-models-manager manager::tests`
    - `cargo test -p codex-chatgpt`
    - `cargo test -p codex-cloud-tasks`
    - `cargo test -p codex-login agent_identity`
    - `just fix -p codex-core -p codex-login -p codex-analytics -p
    codex-app-server -p codex-cloud-requirements -p codex-cloud-tasks -p
    codex-models-manager -p codex-chatgpt -p codex-model-provider -p
    codex-mcp -p codex-core-skills`
    - `just fix -p codex-app-server`
    - `git diff --check`
  • chore(multiagent) skills instructions toggle (#18596)
    ## Summary
    Support toggling the skills message off.
    
    ## Test Plan
    - [x] Updated unit tests
  • Persist and prewarm agent tasks per thread (#17978)
    ## Summary
    - persist registered agent tasks in the session state update stream so
    the thread can reuse them
    - prewarm task registration once identity registration succeeds, while
    keeping startup failures best-effort
    - isolate the session-side task lifecycle into a dedicated module so
    AgentIdentityManager and RegisteredAgentTask do not leak across as many
    core layers
    
    ## Testing
    - cargo test -p codex-core startup_agent_task_prewarm
    - cargo test -p codex-core
    cached_agent_task_for_current_identity_clears_stale_task
    - cargo test -p codex-core record_initial_history_
  • [5/6] Wire executor-backed MCP stdio (#18212)
    ## Summary
    - Add the executor-backed RMCP stdio transport.
    - Wire MCP stdio placement through the executor environment config.
    - Cover local and executor-backed stdio paths with the existing MCP test
    helpers.
    
    ## Stack
    ```text
    o  #18027 [6/6] Fail exec client operations after disconnect
    │
    @  #18212 [5/6] Wire executor-backed MCP stdio
    │
    o  #18087 [4/6] Abstract MCP stdio server launching
    │
    o  #18020 [3/6] Add pushed exec process events
    │
    o  #18086 [2/6] Support piped stdin in exec process API
    │
    o  #18085 [1/6] Add MCP server environment config
    │
    o  main
    ```
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Revert "[codex] drain mailbox only at request boundaries" (#18325)
    ## Summary
    - Reverts PR #17749 so queued inter-agent mail can again preempt after
    reasoning/commentary output item boundaries.
    - Applies the revert to the current `codex/turn.rs` module layout and
    restores the prior pending-input test expectations/snapshots.
    
    ## Testing
    - `just fmt`
    - `cargo test -p codex-core --test all pending_input`
    - `cargo test -p codex-core` failed in unrelated
    `tools::js_repl::tests::js_repl_imported_local_files_can_access_repl_globals`:
    dotslash download hit `mktemp: mkdtemp failed ... Operation not
    permitted` in the sandbox temp dir.
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add max context window model metadata (#18382)
    Adds max_context_window to model metadata and routes core context-window
    reads through resolved model info. Config model_context_window overrides
    are clamped to max_context_window when present; without an override, the
    model context_window is used.
  • feat: Budget skill metadata and surface trimming as a warning (#18298)
    Cap the model-visible skills section to a small share of the context
    window, with a fallback character budget, and keep only as many implicit
    skills as fit within that budget.
    
    Emit a non-fatal warning when enabled skills are omitted, and add a new
    app-server warning notification
    
    Record thread-start skill metrics for total enabled skills, kept skills,
    and whether truncation happened
    
    ---------
    
    Co-authored-by: Matthew Zeng <mzeng@openai.com>
    Co-authored-by: Codex <noreply@openai.com>
  • fix: trust-gate project hooks and exec policies (#14718)
    ## Summary
    - trust-gate project `.codex` layers consistently, including repos that
    have `.codex/hooks.json` or `.codex/execpolicy/*.rules` but no
    `.codex/config.toml`
    - keep disabled project layers in the config stack so nested trusted
    project layers still resolve correctly, while preventing hooks and exec
    policies from loading until the project is trusted
    - update app-server/TUI onboarding copy to make the trust boundary
    explicit and add regressions for loader, hooks, exec-policy, and
    onboarding coverage
    
    ## Security
    Before this change, an untrusted repo could auto-load project hooks or
    exec policies from `.codex/` as long as `config.toml` was absent. This
    makes trust the single gate for project-local config, hooks, and exec
    policies.
    
    ## Stack
    - Parent of #15936
    
    ## Test
    - cargo test -p codex-core without_config_toml
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • fix: fix fs sandbox helper for apply_patch (#18296)
    ## Summary
    
    - pass split filesystem sandbox policy/cwd through apply_patch contexts,
    while omitting legacy-equivalent policies to keep payloads small
    - keep the fs helper compatible with legacy Landlock by avoiding helper
    read-root permission expansion in that mode and disabling helper network
    access
    
    ## Root Cause
    
    `d626dc38950fb40a1a5ad0a8ffab2485e3348c53` routed exec-server filesystem
    operations through a sandboxed helper. That path forwarded legacy
    Landlock into a helper policy shape that could require direct
    split-policy enforcement. Sandboxed `apply_patch` hit that edge through
    the filesystem abstraction.
    
    The same 0.121 edit-regression path is consistent with #18354: normal
    writes route through the `apply_patch` filesystem helper, fail under
    sandbox, and then surface the generic retry-without-sandbox prompt.
    
    Fixes #18069
    Fixes #18354
    
    ## Validation
    
    - `cd codex-rs && just fmt`
    - earlier branch validation before merging current `origin/main` and
    dropping the now-separate PATH fix:
      - `cd codex-rs && cargo test -p codex-exec-server`
    - `cd codex-rs && cargo test -p codex-core file_system_sandbox_context`
      - `cd codex-rs && just fix -p codex-exec-server`
      - `cd codex-rs && just fix -p codex-core`
      - `git diff --check`
      - `cd codex-rs && cargo clean`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • refactor: narrow async lock guard lifetimes (#18211)
    Follow-up to https://github.com/openai/codex/pull/18178, where we called
    out enabling the await-holding lint as a follow-up.
    
    The long-term goal is to enable Clippy coverage for async guards held
    across awaits. This PR is intentionally only the first, low-risk cleanup
    pass: it narrows obvious lock guard lifetimes and leaves
    `codex-rs/Cargo.toml` unchanged so the lint is not enabled until the
    remaining cases are fixed or explicitly justified. It intentionally
    leaves the active-turn/turn-state locking pattern alone because those
    checks and mutations need to stay atomic.
    
    ## Common fixes used here
    
    These are the main patterns reviewers should expect in this PR, and they
    are also the patterns to reach for when fixing future `await_holding_*`
    findings:
    
    - **Scope the guard to the synchronous work.** If the code only needs
    data from a locked value, move the lock into a small block, clone or
    compute the needed values, and do the later `.await` after the block.
    - **Use direct one-line mutations when there is no later await.** Cases
    like `map.lock().await.remove(&id)` are acceptable when the guard is
    only needed for that single mutation and the statement ends before any
    async work.
    - **Drain or clone work out of the lock before notifying or awaiting.**
    For example, the JS REPL drains pending exec senders into a local vector
    and the websocket writer clones buffered envelopes before it serializes
    or sends them.
    - **Use a `Semaphore` only when serialization is intentional across
    async work.** The test serialization guards intentionally span awaited
    setup or execution, so using a semaphore communicates "one at a time"
    without holding a mutex guard.
    - **Remove the mutex when there is only one owner.** The PTY stdin
    writer task owns `stdin` directly; the old `Arc<Mutex<_>>` did not
    protect shared access because nothing else had access to the writer.
    - **Do not split locks that protect an atomic invariant.** This PR
    deliberately leaves active-turn/turn-state paths alone because those
    checks and mutations need to stay atomic. Those cases should be fixed
    separately with a design change or documented with `#[expect]`.
    
    ## What changed
    
    - Narrow scoped async mutex guards in app-server, JS REPL, network
    approval, remote-control websocket, and the RMCP test server.
    - Replace test-only async mutex serialization guards with semaphores
    where the guard intentionally lives across async work.
    - Let the PTY pipe writer task own stdin directly instead of wrapping it
    in an async mutex.
    
    ## Verification
    
    - `just fix -p codex-core -p codex-app-server -p codex-rmcp-client -p
    codex-shell-escalation -p codex-utils-pty -p codex-utils-readiness`
    - `just clippy -p codex-core`
    - `cargo test -p codex-core -p codex-app-server -p codex-rmcp-client -p
    codex-shell-escalation -p codex-utils-pty -p codex-utils-readiness` was
    run; the app-server suite passed, and `codex-core` failed in the local
    sandbox on six otel approval tests plus
    `suite::user_shell_cmd::user_shell_command_does_not_set_network_sandbox_env_var`,
    which appear to depend on local command approval/default rules and
    `CODEX_SANDBOX_NETWORK_DISABLED=1` in this environment.
  • [codex] Propagate rate limit reached type (#18227)
    ## Summary
    
    First PR in the split from #17956.
    
    - adds the core/app-server `RateLimitReachedType` shape
    - maps backend `rate_limit_reached_type` into Codex rate-limit snapshots
    - carries the field through app-server notifications/responses and
    generated schemas
    - updates existing constructors/tests for the new optional field
    
    ## Validation
    
    - `cargo test -p codex-backend-client`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-app-server rate_limits`
    - `cargo test -p codex-tui workspace_`
    - `cargo test -p codex-tui status_`
    - `just fmt`
    - `just fix -p codex-backend-client`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-app-server`
    - `just fix -p codex-tui`
  • Move codex module under session (#18249)
    ## Summary
    - rename the core codex module root to session/mod.rs without using
    #[path]
    - move the codex module directory and tests under core/src/session
    - remove session/mod.rs reexports so call sites use explicit child
    module paths
    
    ## Testing
    - cargo test -p codex-core --lib
    - cargo check -p codex-core --tests
    - just fmt
    - just fix -p codex-core
    - git diff --check