15 Commits

  • feat: use run agent task auth for inference (#19051)
    ## Stack
    
    This is PR 3 of the simplified HAI single-run-task stack:
    
    - [#19047](https://github.com/openai/codex/pull/19047) Agent Identity
    assertion and task-registration primitives, including the shared
    run-task helper used by existing Agent Identity JWT auth.
    - [#19049](https://github.com/openai/codex/pull/19049)
    Disabled-by-default ChatGPT auth opt-in that provisions/reuses persisted
    Agent Identity runtime auth and its single run task.
    - [#19051](https://github.com/openai/codex/pull/19051) Run-scoped
    provider auth that uses one backend-owned task id for first-party
    inference and compaction requests.
    
    [#19054](https://github.com/openai/codex/pull/19054) collapsed out of
    the active stack because the simplified design no longer needs a
    separate background/control-plane task helper.
    
    ## Summary
    
    This PR moves Agent Identity usage into provider auth resolution. That
    keeps `AgentAssertion` auth tied to first-party OpenAI provider requests
    instead of applying a late session-wide override that could affect
    local, custom, Bedrock, API-key, or external-bearer providers.
    
    What changed:
    
    - adds a small `ProviderAuthScope` struct carrying the run auth policy
    and session source needed by provider-scoped auth resolution
    - lets `Session` opt the existing `ModelClient` into `ChatGptAuth`
    policy when `use_agent_identity` is enabled, without adding a second
    model-client constructor
    - resolves Agent Identity only for first-party OpenAI provider auth
    paths
    - uses the persisted run task id from the `AgentIdentityAuth` record to
    build `AgentAssertion` auth for Responses requests
    - routes shared request setup through scoped provider auth so unary
    compact requests use the same run-task assertion path as inference turns
    - keeps local/custom/Bedrock/env-key/external-bearer provider auth
    unchanged
    - lets missing run-task state surface through the existing model-request
    error path instead of silently falling back to bearer auth
    
    This PR intentionally does not create thread-scoped, target-scoped, or
    background-scoped task identities. The run task is the only task Codex
    registers in this POC shape.
    
    ## Testing
    
    - `just test -p codex-model-provider`
    - `just test -p codex-core client::tests::provider_auth_scope_uses`
    - `just test -p codex-core remote_compact_uses_agent_identity_assertion`
  • Simplify MCP tool handler plumbing (#21595)
    ## Why
    The MCP tool path had accumulated a few core-owned special cases: a
    dedicated payload variant, resolver plumbing, a legacy `AfterToolUse`
    translation path, and a side channel for parallel-call metadata. That
    made `ToolRegistry` and the spec builder know more about MCP than they
    needed to.
    
    This change moves MCP-specific execution details back onto `ToolInfo`
    and `McpHandler` so `codex-core` can treat MCP calls like normal
    function calls while still preserving MCP-specific dispatch and
    telemetry behavior where it belongs.
    
    ## What changed
    - removed `resolve_mcp_tool_info`, `ToolPayload::Mcp`, `ToolKind`, and
    the remaining registry-side MCP resolver path
    - stored MCP routing metadata directly on `McpHandler` and `ToolInfo`,
    including `supports_parallel_tool_calls`
    - deleted the legacy `AfterToolUse` consumer in `core`, which removes
    the need for handler-specific `after_tool_use_payload` implementations
    - switched tool-result telemetry to handler-provided tags and kept
    MCP-specific dispatch payload construction inside the handler
    - simplified tool spec planning/building by passing `ToolInfo` directly
    and dropping the direct/deferred MCP wrapper structs and the
    parallel-server side table
    
    ## Testing
    - `cargo check -p codex-core -p codex-mcp -p codex-otel`
    - `cargo test -p codex-core
    mcp_parallel_support_uses_exact_payload_server`
    - `cargo test -p codex-core
    direct_mcp_tools_register_namespaced_handlers`
    - `cargo test -p codex-core
    search_tool_description_lists_each_mcp_source_once`
    - `cargo test -p codex-mcp
    list_all_tools_uses_startup_snapshot_while_client_is_pending`
    - `just fix -p codex-core -p codex-mcp -p codex-otel`
  • [codex] reduce module visibility (#16978)
    ## Summary
    - reduce public module visibility across Rust crates, preferring private
    or crate-private modules with explicit crate-root public exports
    - update external call sites and tests to use the intended public crate
    APIs instead of reaching through module trees
    - add the module visibility guideline to AGENTS.md
    
    ## Validation
    - `cargo check --workspace --all-targets --message-format=short` passed
    before the final fix/format pass
    - `just fix` completed successfully
    - `just fmt` completed successfully
    - `git diff --check` passed
  • chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)
    ## Why
    
    `argument-comment-lint` was green in CI even though the repo still had
    many uncommented literal arguments. The main gap was target coverage:
    the repo wrapper did not force Cargo to inspect test-only call sites, so
    examples like the `latest_session_lookup_params(true, ...)` tests in
    `codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.
    
    This change cleans up the existing backlog, makes the default repo lint
    path cover all Cargo targets, and starts rolling that stricter CI
    enforcement out on the platform where it is currently validated.
    
    ## What changed
    
    - mechanically fixed existing `argument-comment-lint` violations across
    the `codex-rs` workspace, including tests, examples, and benches
    - updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
    `tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
    `--all-targets` unless the caller explicitly narrows the target set
    - fixed both wrappers so forwarded cargo arguments after `--` are
    preserved with a single separator
    - documented the new default behavior in
    `tools/argument-comment-lint/README.md`
    - updated `rust-ci` so the macOS lint lane keeps the plain wrapper
    invocation and therefore enforces `--all-targets`, while Linux and
    Windows temporarily pass `-- --lib --bins`
    
    That temporary CI split keeps the stricter all-targets check where it is
    already cleaned up, while leaving room to finish the remaining Linux-
    and Windows-specific target-gated cleanup before enabling
    `--all-targets` on those runners. The Linux and Windows failures on the
    intermediate revision were caused by the wrapper forwarding bug, not by
    additional lint findings in those lanes.
    
    ## Validation
    
    - `bash -n tools/argument-comment-lint/run.sh`
    - `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
    - shell-level wrapper forwarding check for `-- --lib --bins`
    - shell-level wrapper forwarding check for `-- --tests`
    - `just argument-comment-lint`
    - `cargo test` in `tools/argument-comment-lint`
    - `cargo test -p codex-terminal-detection`
    
    ## Follow-up
    
    - Clean up remaining Linux-only target-gated callsites, then switch the
    Linux lint lane back to the plain wrapper invocation.
    - Clean up remaining Windows-only target-gated callsites, then switch
    the Windows lint lane back to the plain wrapper invocation.
  • Add auth 401 observability to client bug reports (#14611)
    CXC-392
    
      [With
      401](https://openai.sentry.io/issues/7333870443/?project=4510195390611458&query=019ce8f8-560c-7f10-a00a-c59553740674&referrer=issue-stream)
      <img width="1909" height="555" alt="401 auth tags in Sentry"
      src="https://github.com/user-attachments/assets/412ea950-61c4-4780-9697-15c270971ee3"
      />
    
    
      - auth_401_*: preserved facts from the latest unauthorized response snapshot
      - auth_*: latest auth-related facts from the latest request attempt
      - auth_recovery_*: unauthorized recovery state and follow-up result
    
    
      Without 401
      <img width="1917" height="522" alt="happy-path auth tags in Sentry"
      src="https://github.com/user-attachments/assets/3381ed28-8022-43b0-b6c0-623a630e679f"
      />
    
      ###### Summary
      - Add client-visible 401 diagnostics for auth attachment, upstream auth classification, and 401 request id / cf-ray correlation.
      - Record unauthorized recovery mode, phase, outcome, and retry/follow-up status without changing auth behavior.
      - Surface the highest-signal auth and recovery fields on uploaded client bug reports so they are usable in Sentry.
      - Preserve original unauthorized evidence under `auth_401_*` while keeping follow-up result tags separate.
    
      ###### Rationale (from spec findings)
      - The dominant bucket needed proof of whether the client attached auth before send or upstream still classified the request as missing auth.
      - Client uploads needed to show whether unauthorized recovery ran and what the client tried next.
      - Request id and cf-ray needed to be preserved on the unauthorized response so server-side correlation is immediate.
      - The bug-report path needed the same auth evidence as the request telemetry path, otherwise the observability would not be operationally useful.
    
      ###### Scope
      - Add auth 401 and unauthorized-recovery observability in `codex-rs/core`, `codex-rs/codex-api`, and `codex-rs/otel`, including feedback-tag surfacing.
      - Keep auth semantics, refresh behavior, retry behavior, endpoint classification, and geo-denial follow-up work out of this PR.
    
      ###### Trade-offs
      - This exports only safe auth evidence: header presence/name, upstream auth classification, request ids, and recovery state. It does not export token values or raw upstream bodies.
      - This keeps websocket connection reuse as a transport clue because it can help distinguish stale reused sessions from fresh reconnects.
      - Misroute/base-url classification and geo-denial are intentionally deferred to a separate follow-up PR so this review stays focused on the dominant auth 401 bucket.
    
      ###### Client follow-up
      - PR 2 will add misroute/provider and geo-denial observability plus the matching feedback-tag surfacing.
      - A separate host/app-server PR should log auth-decision inputs so pre-send host auth state can be correlated with client request evidence.
      - `device_id` remains intentionally separate until there is a safe existing source on the feedback upload path.
    
      ###### Testing
      - `cargo test -p codex-core refresh_available_models_sorts_by_priority`
      - `cargo test -p codex-core emit_feedback_request_tags_`
      - `cargo test -p codex-core emit_feedback_auth_recovery_tags_`
      - `cargo test -p codex-core auth_request_telemetry_context_tracks_attached_auth_and_retry_phase`
      - `cargo test -p codex-core extract_response_debug_context_decodes_identity_headers`
      - `cargo test -p codex-core identity_auth_details`
      - `cargo test -p codex-core telemetry_error_messages_preserve_non_http_details`
      - `cargo test -p codex-core --all-features --no-run`
      - `cargo test -p codex-otel otel_export_routing_policy_routes_api_request_auth_observability`
      - `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_connect_auth_observability`
      - `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_request_transport_observability`
  • chore(otel): rename OtelManager to SessionTelemetry (#13808)
    ## Summary
    This is a purely mechanical refactor of `OtelManager` ->
    `SessionTelemetry` to better convey what the struct is doing. No
    behavior change.
    
    ## Why
    
    `OtelManager` ended up sounding much broader than what this type
    actually does. It doesn't manage OTEL globally; it's the session-scoped
    telemetry surface for emitting log/trace events and recording metrics
    with consistent session metadata (`app_version`, `model`, `slug`,
    `originator`, etc.).
    
    `SessionTelemetry` is a more accurate name, and updating the call sites
    makes that boundary a lot easier to follow.
    
    ## Validation
    
    - `just fmt`
    - `cargo test -p codex-otel`
    - `cargo test -p codex-core`
  • feat(otel, core): record turn TTFT and TTFM metrics in codex-core (#13630)
    ### Summary
    This adds turn-level latency metrics for the first model output and the
    first completed agent message.
    - `codex.turn.ttft.duration_ms` starts at turn start and records on the
    first output signal we see from the model. That includes normal
    assistant text, reasoning deltas, and non-text outputs like tool-call
    items.
    - `codex.turn.ttfm.duration_ms` also starts at turn start, but it
    records when the first agent message finishes streaming rather than when
    its first delta arrives.
    
    ### Implementation notes
    The timing is tracked in codex-core, not app-server, so the definition
    stays consistent across CLI, TUI, and app-server clients.
    
    I reused the existing turn lifecycle boundary that already drives
    `codex.turn.e2e_duration_ms`, stored the turn start timestamp in turn
    state, and record each metric once per turn.
    
    I also wired the new metric names into the OTEL runtime metrics summary
    so they show up in the same in-memory/debug snapshot path as the
    existing timing metrics.
  • Add MCP server context to otel tool_result logs (#12267)
    Summary
    - capture the origin for each configured MCP server and expose it via
    the connection manager
    - plumb MCP server name/origin into tool logging and emit
    codex.tool_result events with those fields
    - add unit coverage for origin parsing and extend OTEL tests to assert
    empty MCP fields for non-MCP tools
    - currently not logging full urls or url paths to prevent logging
    potentially sensitive data
    
    Testing
    - Not run (not requested)
  • add sandbox policy and sandbox name to codex.tool.call metrics (#10711)
    This will give visibility into the comparative success rate of the
    Windows sandbox implementations compared to other platforms.
  • fix(auth): isolate chatgptAuthTokens concept to auth manager and app-server (#10423)
    So that the rest of the codebase (like TUI) don't need to be concerned
    whether ChatGPT auth was handled by Codex itself or passed in via
    app-server's external auth mode.
  • Add websocket telemetry metrics and labels (#10316)
    Summary
    - expose websocket telemetry hooks through the responses client so
    request durations and event processing can be reported
    - record websocket request/event metrics and emit runtime telemetry
    events that the history UI now surfaces
    - improve tests to cover websocket telemetry reporting and guard runtime
    summary updates
    
    
    <img width="824" height="79" alt="Screenshot 2026-01-31 at 5 28 12 PM"
    src="https://github.com/user-attachments/assets/ea9a7965-d8b4-4e3c-a984-ef4fdc44c81d"
    />
  • feat: show runtime metrics in console (#10278)
    Summary of changes:
    
    - Adds a new feature flag: runtime_metrics
      - Declared in core/src/features.rs
      - Added to core/config.schema.json
      - Wired into OTEL init in core/src/otel_init.rs
    
    - Enables on-demand runtime metric snapshots in OTEL
      - Adds runtime_metrics: bool to otel/src/config.rs
      - Enables experimental custom reader features in otel/Cargo.toml
      - Adds snapshot/reset/summary APIs in:
        - otel/src/lib.rs
        - otel/src/metrics/client.rs
        - otel/src/metrics/config.rs
        - otel/src/metrics/error.rs
    
    - Defines metric names and a runtime summary builder
      - New files:
        - otel/src/metrics/names.rs
        - otel/src/metrics/runtime_metrics.rs
      - Summarizes totals for:
        - Tool calls
        - API requests
        - SSE/streaming events
    
    - Instruments metrics collection in OTEL manager
      - otel/src/traces/otel_manager.rs now records:
        - API call counts + durations
        - SSE event counts + durations (success/failure)
        - Tool call metrics now use shared constants
    
    - Surfaces runtime metrics in the TUI
      - Resets runtime metrics at turn start in tui/src/chatwidget.rs
    - Displays metrics in the final separator line in
    tui/src/history_cell.rs
    
    - Adds tests
      - New OTEL tests:
        - otel/tests/suite/snapshot.rs
        - otel/tests/suite/runtime_summary.rs
      - New TUI test:
    - final_message_separator_includes_runtime_metrics in
    tui/src/history_cell.rs
    
    Scope:
    - 19 files changed
    - ~652 insertions, 38 deletions
    
    
    <img width="922" height="169" alt="Screenshot 2026-01-30 at 4 11 34 PM"
    src="https://github.com/user-attachments/assets/1efd754d-a16d-4564-83a5-f4442fd2f998"
    />