Commit Graph

15 Commits

  • Simplify MCP tool handler plumbing (#21595)
    ## Why
    The MCP tool path had accumulated a few core-owned special cases: a
    dedicated payload variant, resolver plumbing, a legacy `AfterToolUse`
    translation path, and a side channel for parallel-call metadata. That
    made `ToolRegistry` and the spec builder know more about MCP than they
    needed to.
    
    This change moves MCP-specific execution details back onto `ToolInfo`
    and `McpHandler` so `codex-core` can treat MCP calls like normal
    function calls while still preserving MCP-specific dispatch and
    telemetry behavior where it belongs.
    
    ## What changed
    - removed `resolve_mcp_tool_info`, `ToolPayload::Mcp`, `ToolKind`, and
    the remaining registry-side MCP resolver path
    - stored MCP routing metadata directly on `McpHandler` and `ToolInfo`,
    including `supports_parallel_tool_calls`
    - deleted the legacy `AfterToolUse` consumer in `core`, which removes
    the need for handler-specific `after_tool_use_payload` implementations
    - switched tool-result telemetry to handler-provided tags and kept
    MCP-specific dispatch payload construction inside the handler
    - simplified tool spec planning/building by passing `ToolInfo` directly
    and dropping the direct/deferred MCP wrapper structs and the
    parallel-server side table
    
    ## Testing
    - `cargo check -p codex-core -p codex-mcp -p codex-otel`
    - `cargo test -p codex-core
    mcp_parallel_support_uses_exact_payload_server`
    - `cargo test -p codex-core
    direct_mcp_tools_register_namespaced_handlers`
    - `cargo test -p codex-core
    search_tool_description_lists_each_mcp_source_once`
    - `cargo test -p codex-mcp
    list_all_tools_uses_startup_snapshot_while_client_is_pending`
    - `just fix -p codex-core -p codex-mcp -p codex-otel`
  • fix(core): truncate large mcp tool outputs in rollouts (#20260)
    ## Why
    Large MCP tool call outputs can make rollout JSONL files enormous. In
    the session that motivated this change, the biggest JSONL records were:
    - `event_msg/mcp_tool_call_end`
    - `response_item/function_call_output`
    
    both containing the same unbounded MCP payloads - just 3 MCP tool calls
    that each were multi-hundred MBs 馃槺
    
    This PR truncates both of those JSONL records.
    
    ## How
    
    #### For `response_item/function_call_output`
    Unified exec already bounds tool output before it is injected into
    model-facing history, which also keeps the corresponding rollout
    `response_item/function_call_output` records small.
    
    MCP should follow the same pattern: truncate the model-facing tool
    output at the tool-output boundary, while leaving code-mode/raw hook
    consumers alone.
    
    #### For `event_msg/mcp_tool_call_end`
    `McpToolCallEnd` also needs its own bounded event copy because it is the
    app-server/replay/UI event shape that backs `ThreadItem::McpToolCall`.
    Unfortunately this is _not_ downstream of the `ToolOutput` trait.
    
    ## Model behavior 
    Model behavior is actually unchanged as a result of this PR. 
    
    Before this PR, MCP output was:
    1. Converted to `FunctionCallOutput`.
    2. Recorded into in-memory history.
    3. Truncated by `ContextManager::record_items()` before later model
    turns saw it.
    
    After this branch, MCP output is truncated earlier, in
    `McpToolOutput::response_payload()`, using the same helper. Then
    `ContextManager::record_items()` sees an already-truncated output and
    effectively has little/no additional work to do.
    
    So the model should still see the same kind of truncated function-call
    output. The practical difference is where truncation happens: earlier,
    before rollout persistence/app-server emission can see the giant
    payload.
    
    ## Verification
    
    - `cargo test -p codex-core mcp_tool_output`
    - `cargo test -p codex-core
    mcp_tool_call::tests::truncate_mcp_tool_result_for_event`
    - `cargo test -p codex-core
    mcp_post_tool_use_payload_uses_model_tool_name_args_and_result`
    - `just fmt`
    - `just fix -p codex-core`
    - `git diff --check`
  • Support MCP tools in hooks (#18385)
    ## Summary
    
    Lifecycle hooks currently treat `PreToolUse`, `PostToolUse`, and
    `PermissionRequest` as Bash-only flows
    - hook schema constrains `tool_name` to `Bash`
    - hook input assumes a command-shaped `tool_input`
    - core hook dispatch path passes only shell command strings
    
    That means hooks cannot target MCP tools even though MCP tool names are
    model-visible and stable
    
    This change generalizes those hook paths so they can match and receive
    payloads for MCP tools while preserving the existing Bash behavior.
    
    ## Reviewer Notes
    
    I think these are the key files
    - `codex-rs/core/src/tools/handlers/mcp.rs`
    - `codex-rs/core/src/mcp_tool_call.rs`
    
    Otherwise the changes across apply_patch, shell, and unified_exec are
    mainly to rewire everything to be `tool_input` based instead of just
    `command` so that it'll make sense for MCP tools.
    
    ## Changes
    
    - Allow `PreToolUse`, `PostToolUse`, and `PermissionRequest` hook inputs
    to carry arbitrary `tool_name` and `tool_input` values instead of
    hard-coding `Bash` and command-only payloads.
    - Add MCP hook payload support through `McpHandler`, using the
    model-visible tool name from `ToolInvocation` and the raw MCP arguments
    as `tool_input`.
    - Include MCP tool responses in `PostToolUse` by serializing
    `McpToolOutput` into the hook response payload.
    - Run `PermissionRequest` hooks for MCP approval requests after
    remembered approval checks and before falling back to user-facing MCP
    elicitation.
    - Preserve exact matching for literal hook matchers like `Bash` and
    `mcp__memory__create_entities`, while keeping regex matcher support for
    patterns like `mcp__memory__.*` and `mcp__.*__write.*`.
    
    ---------
    
    Co-authored-by: Andrei Eternal <eternal@openai.com>
    Co-authored-by: Codex <noreply@openai.com>
  • hooks: emit Bash PostToolUse when exec_command completes via write_stdin (#18888)
    Fixes #16246.
    
    ## Why
    
    `exec_command` already emits `PreToolUse`, but long-running unified exec
    commands that finish on a later `write_stdin` poll could miss the
    matching `PostToolUse`. That left the Bash hook lifecycle inconsistent,
    broke expectations around `tool_use_id` and `tool_input.command`, and
    meant `PostToolUse` block/replacement feedback could fail to replace the
    final session output before it reached model context.
    
    This keeps the fix scoped to the `exec_command` / `write_stdin`
    lifecycle. Broader non-Bash hook expansion is still out of scope here
    and remains tracked separately in #16732.
    
    ## What changed
    
    - Compute and store `PostToolUsePayload` while handlers still have
    access to their concrete output type, and carry `tool_use_id` through
    that payload.
    - Preserve the original hook-facing `exec_command` string through
    unified exec state (`ExecCommandRequest`, `ProcessEntry`,
    `PreparedProcessHandles`, and `ExecCommandToolOutput`) via
    `hook_command`, and remove the now-unused `session_command` output
    metadata.
    - Emit exactly one Bash `PostToolUse` for long-running `exec_command`
    sessions when a later `write_stdin` poll observes final completion,
    using the original `exec_command` call id and hook-facing command.
    - Keep one-shot `exec_command` behavior aligned with the same payload
    construction, including interactive completions that return a final
    result directly.
    - Apply `PostToolUse` block/replacement feedback before the final
    `write_stdin` completion output is sent back to the model.
    - Keep `write_stdin` itself out of `PreToolUse` matching so it continues
    to act as transport/polling for the original Bash tool call.
    - Restore plain matcher behavior for tool-name matchers such as `Bash`
    and `Edit|Write`, while still treating patterns with regex characters
    (for example `mcp__.*`) as regexes.
    - Add unit coverage for unified exec payload construction and parallel
    session separation, plus a core integration regression that verifies a
    blocked `PostToolUse` replaces the final `write_stdin` output in model
    context.
    
    ## Testing
    
    - `cargo test -p codex-hooks`
    - `cargo test -p codex-core post_tool_use_payload`
    - `cargo test -p codex-core
    post_tool_use_blocks_when_exec_session_completes_via_write_stdin`
  • [tool search] support namespaced deferred dynamic tools (#18413)
    Deferred dynamic tools need to round-trip a namespace so a tool returned
    by `tool_search` can be called through the same registry key that core
    uses for dispatch.
    
    This change adds namespace support for dynamic tool specs/calls,
    persists it through app-server thread state, and routes dynamic tool
    calls by full `ToolName` while still sending the app the leaf tool name.
    Deferred dynamic tools must provide a namespace; non-deferred dynamic
    tools may remain top-level.
    
    It also introduces `LoadableToolSpec` as the shared
    function-or-namespace Responses shape used by both `tool_search` output
    and dynamic tool registration, so dynamic tools use the same wrapping
    logic in both paths.
    
    Validation:
    - `cargo test -p codex-tools`
    - `cargo test -p codex-core tool_search`
    
    ---------
    
    Co-authored-by: Sayan Sisodiya <sayan@openai.com>
  • Update image outputs to default to high detail (#18386)
    Do not assume the default `detail`.
  • Support original-detail metadata on MCP image outputs (#17714)
    ## Summary
    - honor `_meta["codex/imageDetail"] == "original"` on MCP image content
    and map it to `detail: "original"` where supported
    - strip that detail back out when the active model does not support
    original-detail image inputs
    - update code-mode `image(...)` to accept individual MCP image blocks
    - teach `js_repl` / `codex.emitImage(...)` to preserve the same hint
    from raw MCP image outputs
    - document the new `_meta` contract and add generic RMCP-backed coverage
    across protocol, core, code-mode, and js_repl paths
  • Add MCP tool wall time to model output (#17406)
    Include MCP wall time in the output so the model is aware of how long
    it's calls are taking.
  • Support anyOf and enum in JsonSchema (#16875)
    This brings us into better alignment with the JSON schema subset that is
    supported in
    <https://developers.openai.com/api/docs/guides/structured-outputs#supported-schemas>,
    and also allows us to render richer function signatures in code mode
    (e.g., anyOf{null, OtherObjectType})
  • fix(core) revert Command line in unified exec output (#17031)
    ## Summary
    https://github.com/openai/codex/pull/13860 changed the serialized output
    format of Unified Exec. This PR reverts those changes and some related
    test changes
    
    ## Testing
    - [x] Update tests
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Remove client_common tool re-exports (#16482)
    ## Why
    
    `codex-rs/core/src/client_common.rs` still had a `tools` re-export
    module that forwarded `codex_tools` types back into `codex-core`. After
    the earlier extraction work in #16379, #16471, #16477, and #16481, that
    extra layer no longer adds value.
    
    Removing it keeps dependencies explicit: the `codex-core` modules that
    actually use `ToolSpec` and related types now depend on `codex_tools`
    directly instead of reaching through `client_common`.
    
    ## What Changed
    
    - removed the `client_common::tools` re-export module from
    `core/src/client_common.rs`
    - updated the remaining `codex-core` consumers to import `codex_tools`
    directly
    - adjusted the affected test code to reference
    `codex_tools::ResponsesApiTool` directly as well
    
    This is a mechanical cleanup only. It does not change tool behavior or
    runtime logic.
    
    ## Testing
    
    - `cargo test -p codex-core client_common::tests`
    - `cargo test -p codex-core tools::router::tests`
    - `cargo test -p codex-core tools::context::tests`
    - `cargo test -p codex-core tools::spec::tests`
  • Extract update_plan tool spec into codex-tools (#16481)
    ## Why
    
    `codex-rs/core/src/tools/handlers/plan.rs` still owned both the
    `update_plan` runtime handler and the static tool definition. The tool
    definition is pure metadata, so keeping it in `codex-core` works against
    the ongoing effort to move tool-spec code into `codex-tools` and keep
    `codex-core` focused on orchestration and execution paths.
    
    This continues the extraction work from #16379, #16471, and #16477.
    
    ## What Changed
    
    - added `codex-rs/tools/src/plan_tool.rs` with
    `create_update_plan_tool()`
    - re-exported that constructor from `codex-rs/tools/src/lib.rs`
    - updated `codex-rs/core/src/tools/spec.rs` and
    `codex-rs/core/src/tools/spec_tests.rs` to use the `codex-tools` export
    instead of a core-local static
    - removed the old `PLAN_TOOL` definition from
    `codex-rs/core/src/tools/handlers/plan.rs`; the `PlanHandler` runtime
    logic still stays in `codex-core`
    - tightened two `codex-core` aliases to `#[cfg(test)]` now that
    production code no longer needs them
    
    ## Testing
    
    - `cargo test -p codex-tools`
    - `cargo test -p codex-core tools::spec::tests`
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16481).
    * #16482
    * __->__ #16481
  • Add notify to code-mode (#14842)
    Allows model to send an out-of-band notification.
    
    The notification is injected as another tool call output for the same
    call_id.
  • Add Smart Approvals guardian review across core, app-server, and TUI (#13860)
    ## Summary
    - add `approvals_reviewer = "user" | "guardian_subagent"` as the runtime
    control for who reviews approval requests
    - route Smart Approvals guardian review through core for command
    execution, file changes, managed-network approvals, MCP approvals, and
    delegated/subagent approval flows
    - expose guardian review in app-server with temporary unstable
    `item/autoApprovalReview/{started,completed}` notifications carrying
    `targetItemId`, `review`, and `action`
    - update the TUI so Smart Approvals can be enabled from `/experimental`,
    aligned with the matching `/approvals` mode, and surfaced clearly while
    reviews are pending or resolved
    
    ## Runtime model
    This PR does not introduce a new `approval_policy`.
    
    Instead:
    - `approval_policy` still controls when approval is needed
    - `approvals_reviewer` controls who reviewable approval requests are
    routed to:
      - `user`
      - `guardian_subagent`
    
    `guardian_subagent` is a carefully prompted reviewer subagent that
    gathers relevant context and applies a risk-based decision framework
    before approving or denying the request.
    
    The `smart_approvals` feature flag is a rollout/UI gate. Core runtime
    behavior keys off `approvals_reviewer`.
    
    When Smart Approvals is enabled from the TUI, it also switches the
    current `/approvals` settings to the matching Smart Approvals mode so
    users immediately see guardian review in the active thread:
    - `approval_policy = on-request`
    - `approvals_reviewer = guardian_subagent`
    - `sandbox_mode = workspace-write`
    
    Users can still change `/approvals` afterward.
    
    Config-load behavior stays intentionally narrow:
    - plain `smart_approvals = true` in `config.toml` remains just the
    rollout/UI gate and does not auto-set `approvals_reviewer`
    - the deprecated `guardian_approval = true` alias migration does
    backfill `approvals_reviewer = "guardian_subagent"` in the same scope
    when that reviewer is not already configured there, so old configs
    preserve their original guardian-enabled behavior
    
    ARC remains a separate safety check. For MCP tool approvals, ARC
    escalations now flow into the configured reviewer instead of always
    bypassing guardian and forcing manual review.
    
    ## Config stability
    The runtime reviewer override is stable, but the config-backed
    app-server protocol shape is still settling.
    
    - `thread/start`, `thread/resume`, and `turn/start` keep stable
    `approvalsReviewer` overrides
    - the config-backed `approvals_reviewer` exposure returned via
    `config/read` (including profile-level config) is now marked
    `[UNSTABLE]` / experimental in the app-server protocol until we are more
    confident in that config surface
    
    ## App-server surface
    This PR intentionally keeps the guardian app-server shape narrow and
    temporary.
    
    It adds generic unstable lifecycle notifications:
    - `item/autoApprovalReview/started`
    - `item/autoApprovalReview/completed`
    
    with payloads of the form:
    - `{ threadId, turnId, targetItemId, review, action? }`
    
    `review` is currently:
    - `{ status, riskScore?, riskLevel?, rationale? }`
    - where `status` is one of `inProgress`, `approved`, `denied`, or
    `aborted`
    
    `action` carries the guardian action summary payload from core when
    available. This lets clients render temporary standalone pending-review
    UI, including parallel reviews, even when the underlying tool item has
    not been emitted yet.
    
    These notifications are explicitly documented as `[UNSTABLE]` and
    expected to change soon.
    
    This PR does **not** persist guardian review state onto `thread/read`
    tool items. The intended follow-up is to attach guardian review state to
    the reviewed tool item lifecycle instead, which would improve
    consistency with manual approvals and allow thread history / reconnect
    flows to replay guardian review state directly.
    
    ## TUI behavior
    - `/experimental` exposes the rollout gate as `Smart Approvals`
    - enabling it in the TUI enables the feature and switches the current
    session to the matching Smart Approvals `/approvals` mode
    - disabling it in the TUI clears the persisted `approvals_reviewer`
    override when appropriate and returns the session to default manual
    review when the effective reviewer changes
    - `/approvals` still exposes the reviewer choice directly
    - the TUI renders:
    - pending guardian review state in the live status footer, including
    parallel review aggregation
      - resolved approval/denial state in history
    
    ## Scope notes
    This PR includes the supporting core/runtime work needed to make Smart
    Approvals usable end-to-end:
    - shell / unified-exec / apply_patch / managed-network / MCP guardian
    review
    - delegated/subagent approval routing into guardian review
    - guardian review risk metadata and action summaries for app-server/TUI
    - config/profile/TUI handling for `smart_approvals`, `guardian_approval`
    alias migration, and `approvals_reviewer`
    - a small internal cleanup of delegated approval forwarding to dedupe
    fallback paths and simplify guardian-vs-parent approval waiting (no
    intended behavior change)
    
    Out of scope for this PR:
    - redesigning the existing manual approval protocol shapes
    - persisting guardian review state onto app-server `ThreadItem`s
    - delegated MCP elicitation auto-review (the current delegated MCP
    guardian shim only covers the legacy `RequestUserInput` path)
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • fix: move inline codex-rs/core unit tests into sibling files (#14444)
    ## Why
    PR #13783 moved the `codex.rs` unit tests into `codex_tests.rs`. This
    applies the same extraction pattern across the rest of `codex-rs/core`
    so the production modules stay focused on runtime code instead of large
    inline test blocks.
    
    Keeping the tests in sibling files also makes follow-up edits easier to
    review because product changes no longer have to share a file with
    hundreds or thousands of lines of test scaffolding.
    
    ## What changed
    - replaced each inline `mod tests { ... }` in `codex-rs/core/src/**`
    with a path-based module declaration
    - moved each extracted unit test module into a sibling `*_tests.rs`
    file, using `mod_tests.rs` for `mod.rs` modules
    - preserved the existing `cfg(...)` guards and module-local structure so
    the refactor remains structural rather than behavioral
    
    ## Testing
    - `cargo test -p codex-core --lib` (`1653 passed; 0 failed; 5 ignored`)
    - `just fix -p codex-core`
    - `cargo fmt --check`
    - `cargo shear`