108 Commits

• feat: add Bedrock API key as a managed auth mode (#27443)

  ## Why
  
  Codex needs to manage Amazon Bedrock API key credentials through the
  existing auth lifecycle instead of introducing a separate auth manager
  or provider-specific credential file. Treating Bedrock API key login as
  a primary auth mode gives it the same persistence, keyring, reload, and
  logout behavior as the existing OpenAI API key and ChatGPT modes.
  
  The credential is valid only for the `amazon-bedrock` model provider.
  OpenAI-compatible providers must reject this auth mode rather than
  treating the Bedrock key as an OpenAI bearer token.
  
  ## What changed
  
  - Added `bedrockApiKey` as an app-server `AuthMode` and
  `CodexAuth::BedrockApiKey` as a primary `AuthManager` mode.
  - Added `BedrockApiKeyAuth`, containing the API key and AWS region, to
  the existing `AuthDotJson` payload stored in `$CODEX_HOME/auth.json` or
  the configured keyring backend.
  - Added `login_with_bedrock_api_key(...)`, parallel to
  `login_with_api_key(...)`, which replaces the current stored login with
  Bedrock credentials.
  - Reused generic auth reload and logout behavior instead of adding a
  Bedrock-specific auth manager or logout path.
  - Updated login restrictions, status reporting, diagnostics, telemetry
  classification, generated app-server schemas, and auth fixtures for the
  new mode.
  - Added explicit errors when Bedrock API key auth is selected with an
  OpenAI-compatible model provider.
  
  This PR establishes managed storage and auth-mode behavior. Routing the
  managed key and region into Amazon Bedrock requests will be in follow-up
  PRs.

  Celia Chen · 2026-06-10 20:42:38 -07:00
  06afd63f4a
• [codex] Add reusable OTEL gauge instruments (#27057)

  ## Why
  
  Exec-server observability needs current-value measurements in addition
  to counters. The reusable OTEL client should expose that primitive
  without coupling it to exec-server runtime behavior.
  
  ## What changed
  
  - Adds integer gauge instruments, with optional descriptions.
  - Caches gauges by name and description so instrument metadata remains
  part of the declaration identity.
  - Covers gauge values, descriptions, merged attributes, and OTLP HTTP
  export.
  
  This PR only adds the gauge primitive. It does not add second-based
  duration histograms or exec-server adoption.
  
  ## Stack
  
  1. #26091: counter descriptions
  2. **#27057: gauge instruments**
  3. #27058: second-based duration histograms
  
  Related independent coverage: #27059 tests OTLP HTTP log and trace event
  export.
  
  ## Validation
  
  - `just test -p codex-otel`
  - `just fix -p codex-otel`
  - `just fmt`

  richardopenai · 2026-06-10 21:36:38 +00:00
  7e5e41daea
• [codex] Add OTEL counter descriptions (#26091)

  ## Why
  
  Metric descriptions should be declared with reusable OTEL instruments
  instead of being coupled to individual consumers. Counter descriptions
  are the smallest API primitive needed by the exec-server observability
  work.
  
  ## What changed
  
  - Adds `counter_with_description` while preserving the existing counter
  API.
  - Caches counters by name and description so instrument metadata remains
  part of the declaration identity.
  - Covers the exported description together with the existing value and
  attribute contract.
  
  This PR only adds counter descriptions. It does not add gauges,
  second-based durations, or exec-server adoption.
  
  ## Stack
  
  1. **#26091: counter descriptions**
  2. #27057: gauge instruments
  3. #27058: second-based duration histograms
  
  Related independent coverage: #27059 tests OTLP HTTP log and trace event
  export.
  
  The `codex-exec-server` bounded service tag now stays with the
  exec-server adoption change instead of this reusable infrastructure
  stack.
  
  ## Validation
  
  - `just test -p codex-otel`
  - `just fix -p codex-otel`
  - `just fmt`

  richardopenai · 2026-06-08 22:29:51 +00:00
  feca160da4
• [codex-rs] support v2 personal access tokens (#25731)

  ## Summary
  
  - add v2 personal access token support for `codex login
  --with-access-token` and `CODEX_ACCESS_TOKEN`
  - classify opaque `at-` tokens separately from legacy Agent Identity
  JWTs
  - hydrate required ChatGPT account metadata through AuthAPI
  `/v1/user-auth-credential/whoami`
  - use PATs directly as bearer tokens while preserving existing ChatGPT
  account surfaces
  - expose PAT-backed auth as the explicit `personalAccessToken`
  app-server auth mode
  
  ## Implementation
  
  PAT auth is intentionally small and stateless. Loading a PAT performs
  one AuthAPI metadata request, stores the hydrated metadata in the
  in-memory auth object, and redacts the secret from debug output. Legacy
  Agent Identity JWT handling remains unchanged. The shared access-token
  classifier lives in a private neutral module because it dispatches
  between both credential types.
  
  PAT hydration fails closed when AuthAPI omits any required metadata,
  including email. Hydrated metadata is intentionally not persisted:
  startup performs a live `whoami` preflight so revoked tokens or changed
  account metadata are not accepted from a stale cache.
  
  ## Workspace restriction scope
  
  This change intentionally does **not** apply
  `forced_chatgpt_workspace_id` to PAT authentication. The setting is a
  client-side config guardrail, not an authorization boundary, and PAT
  does not currently require workspace-ID parity. The PAT login and
  `CODEX_ACCESS_TOKEN` paths therefore validate through AuthAPI without
  threading workspace-restriction state through access-token loading.
  Existing workspace checks for non-PAT auth remain on their established
  paths.
  
  ## App-server compatibility
  
  The public app-server `AuthMode` is shared across v1 and v2, and
  PAT-backed auth reports `personalAccessToken` through both APIs.
  Following human review, this intentionally removes the temporary v1
  compatibility mapping that reported PATs as `chatgpt`; the deprecated v1
  API is kept in parity with v2 rather than maintaining a separate closed
  enum. Clients with exhaustive auth-mode handling in either API version
  must add the new case and should generally treat it as ChatGPT-backed
  unless they need PAT-specific behavior.
  
  The v1 auth-status response still omits the raw PAT when `includeToken`
  is requested because that response cannot carry the account metadata
  needed to reuse the credential safely. Persisted PAT auth also omits the
  new enum value so older Codex builds can deserialize `auth.json` and
  infer PAT auth from the credential field after a rollback.
  
  ## Validation
  
  Latest review-fix validation:
  
  - `CARGO_INCREMENTAL=0 just test -p codex-login` (126 passed)
  - `CARGO_INCREMENTAL=0 just test -p codex-cli` (263 passed)
  - `CARGO_INCREMENTAL=0 just test -p codex-cli
  stored_auth_validation_handles_personal_access_token`
  - `CARGO_INCREMENTAL=0 just test -p codex-app-server-protocol` (226
  passed)
  - `CARGO_INCREMENTAL=0 just test -p codex-models-manager
  refresh_available_models_uses_remote_only_catalog_for_chatgpt_auth`
  - `CARGO_INCREMENTAL=0 just test -p codex-tui
  existing_non_oauth_chatgpt_login_counts_as_signed_in`
  - `CARGO_INCREMENTAL=0 just fix -p codex-login -p
  codex-app-server-protocol -p codex-models-manager -p codex-tui -p
  codex-cli`
  - `just fmt`
  - `git diff --check`
  
  The broader `codex-tui` suite previously compiled and ran 2,834 tests.
  Three unrelated environment-sensitive guardian/IDE-socket tests failed
  after retries; the PAT-relevant TUI coverage passed.

  cooper-oai · 2026-06-05 17:36:18 -07:00
  df7818c7d1
• [codex] Forward turn moderation metadata through app-server (#25710)

  ## Why
  First-party backends can supply turn-scoped moderation metadata that
  app-server clients need for client-side presentation. Exposing this as
  an experimental typed notification lets opted-in clients consume it
  without interpreting raw Responses API events.
  
  ## What changed
  - forward `response.metadata.openai_chatgpt_moderation_metadata` from
  Responses API SSE and WebSocket streams as turn-scoped moderation
  metadata
  - emit the experimental app-server v2 `turn/moderationMetadata`
  notification with `{ threadId, turnId, metadata }`
  - add app-server integration coverage for the typed moderation metadata
  notification
  
  ## Testing
  - `just test -p codex-core
  build_ws_client_metadata_includes_window_lineage_and_turn_metadata`
  - `just test -p codex-core` (fails locally: 46 failures and 1 timeout,
  primarily missing `test_stdio_server` and shell snapshot timeouts)
  - `just test -p codex-app-server-protocol`
  - `just test -p codex-app-server
  turn_moderation_metadata_emits_typed_notification_v2`
  - `just test -p codex-app-server` (fails locally: 792 passed, 10 failed,
  and 5 timed out; failures are in existing environment-sensitive tests,
  primarily because nested macOS `sandbox-exec` is not permitted)
  - `just write-app-server-schema --experimental --schema-root
  /tmp/codex-app-server-schema-experimental`

  carlc-oai · 2026-06-05 02:41:06 -07:00
  55aa071b17
• Encrypt multi-agent v2 message payloads (#26210)

  ## Why
  
  Multi-agent v2 currently routes agent instructions through normal tool
  arguments and inter-agent context. That means the parent model can emit
  plaintext task text, Codex can persist it in history/rollouts, and the
  recipient can receive it as ordinary assistant-message JSON.
  
  This changes the v2 path so agent instructions stay encrypted between
  model calls: Responses encrypts the `message` argument returned by the
  model, Codex forwards only that ciphertext, and Responses decrypts it
  internally for the recipient model.
  
  ## What changed
  
  - Mark the v2 `message` parameter as encrypted for `spawn_agent`,
  `send_message`, and `followup_task`.
  - Treat multi-agent v2 tool `message` values as ciphertext
  unconditionally.
  - Store v2 inter-agent task text in
  `InterAgentCommunication.encrypted_content` with empty plaintext
  `content`.
  - Convert encrypted inter-agent communications into the Responses
  `agent_message` input item before sending the child request.
  - Preserve `agent_message` items across history, rollout, compaction,
  telemetry, and app-server schema paths.
  - Leave multi-agent v1 unchanged.
  
  ## Message shape
  
  The model still calls the v2 tools with a `message` argument, but that
  value is now ciphertext:
  
  ```json
  {
    "name": "spawn_agent",
    "arguments": {
      "task_name": "worker",
      "message": "<ciphertext>"
    }
  }
  ```
  
  Codex stores the task as encrypted inter-agent communication:
  
  ```json
  {
    "author": "/root",
    "recipient": "/root/worker",
    "content": "",
    "encrypted_content": "<ciphertext>",
    "trigger_turn": true
  }
  ```
  
  When Codex builds the recipient request, it forwards the ciphertext
  using the new Responses input item:
  
  ```json
  {
    "type": "agent_message",
    "author": "/root",
    "recipient": "/root/worker",
    "content": [
      {
        "type": "encrypted_content",
        "encrypted_content": "<ciphertext>"
      }
    ]
  }
  ```
  
  Responses decrypts that item internally for the recipient model.
  
  ## Context impact
  
  - Parent context no longer carries plaintext v2 agent task instructions
  from these tool arguments.
  - Codex rollout/history stores ciphertext for v2 agent instructions.
  - Recipient requests receive an `agent_message` item instead of
  assistant commentary JSON for encrypted task delivery.
  - Plaintext completion/status notifications are still plaintext because
  they are Codex-generated status messages, not encrypted model tool
  arguments.
  
  ## Validation
  
  - `just test -p codex-tools`
  - `just test -p codex-protocol`
  - `just test -p codex-rollout`
  - `just test -p codex-rollout-trace`
  - `just test -p codex-otel`
  - `just write-app-server-schema`

  jif · 2026-06-05 10:25:57 +02:00
  5f4d06ef18
• [codex] Emit sandbox outcome telemetry event (#25955)

  ## Summary
  
  Adds a dedicated `codex.sandbox_outcome` telemetry event so we can query
  sandbox edge outcomes without threading sandbox metadata through
  tool-result output types.
  
  This is meant to make sandbox failures and approved escalation retries
  visible in OTEL while keeping the existing `codex.tool_result` event
  shape focused on tool completion data.
  
  ## What changed
  
  - Adds `SessionTelemetry::sandbox_outcome(...)`, which emits
  `codex.sandbox_outcome` as both a log and trace event.
  - Records the tool name, call id, sandbox outcome, initial attempt
  duration, and escalated attempt duration when a retry runs.
  - Emits `denied` when the sandbox blocks execution and no retry is run.
  - Emits `timed_out` and `signal` when those sandbox errors surface from
  tool execution.
  - Emits `escalated` when the initial sandboxed attempt fails and the
  approved unsandboxed retry succeeds.
  - Adds OTEL coverage for the new event payload, including timing fields.
  
  ## Validation
  
  - `RUST_MIN_STACK=8388608 just test -p codex-core
  sandbox_outcome_event_records_outcome
  handle_sandbox_error_user_approves_retry_records_tool_decision`
  - `just test -p codex-otel
  otel_export_routing_policy_routes_tool_result_log_and_trace_events
  runtime_metrics_summary_collects_tool_api_and_streaming_metrics`
  - `just fix -p codex-core`
  - `just fix -p codex-otel`

  rreichel3-oai · 2026-06-04 20:58:14 -04:00
  ecae412740
• [codex] Support model-defined reasoning efforts (#26444)

  ## Summary
  - accept non-empty model-defined reasoning effort values while
  preserving built-in effort behavior
  - propagate the non-Copy effort type through core, app-server, TUI,
  telemetry, and persistence call sites
  - preserve string wire encoding and expose an open-string schema for
  clients
  - update model selection and shortcut behavior for model-advertised
  effort values
  
  ## Root cause
  `ReasoningEffort` gained a string-backed custom variant, so it could no
  longer implement `Copy` or rely on derived closed-enum serialization.
  Existing consumers still moved effort values from shared references and
  assumed a fixed built-in value set.
  
  ## Validation
  - `just fmt`
  - Local tests and compilation were not run per request; relying on CI.

  Ahmed Ibrahim · 2026-06-04 13:36:24 -07:00
  8ac304c299
• Add Guardian review metrics (#24897)

  ## Why
  
  Guardian reviews already emit analytics events, but we do not expose
  aggregate OpenTelemetry metrics for review volume, latency, token usage,
  or terminal outcomes. That makes it harder to monitor Guardian behavior
  during rollouts and to compare review outcomes by source, action type,
  session kind, model, and failure mode.
  
  ## What Changed
  
  - Added Guardian review metric names for count, total duration, time to
  first token, and token usage in `codex-rs/otel`.
  - Added `core/src/guardian/metrics.rs` to convert
  `GuardianReviewAnalyticsResult` into sanitized metric tags covering
  decision, terminal status, failure reason, approval request source,
  reviewed action, session kind, risk/outcome, model, reasoning effort,
  and context/truncation state.
  - Emitted the new metrics from `track_guardian_review` for each terminal
  Guardian review result.
  
  ## Testing
  
  - Added
  `guardian_review_metrics_record_counts_durations_and_token_usage`, which
  verifies the emitted count, duration, TTFT, token usage histograms, and
  tag set through the in-memory metrics exporter.

  jif-oai · 2026-05-28 14:07:25 +02:00
  3abf96739b
• otel: drop legacy profile usage telemetry (#24061)

  ## Summary
  - drop the dead legacy profile usage metric and active-profile
  conversation-start fields
  - update role comments so they describe provider and service-tier
  preservation without legacy config-profile wording
  - pair the code cleanup with the file-backed profile docs update in
  openai/developers-website#1476
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-otel`
  - `cargo test -p codex-core` *(fails: existing stack overflow in
  `mcp_tool_call::tests::guardian_mode_mcp_denial_returns_rationale_message`)*
  - `cargo test -p codex-core --lib
  mcp_tool_call::tests::guardian_mode_mcp_denial_returns_rationale_message`
  *(fails with the same stack overflow)*

  jif-oai · 2026-05-22 13:14:44 +02:00
  47476e8a8a
• Split plugin install discovery into list and request tools (#23372)

  ## Summary
  - Add `list_available_plugins_to_install` as the inventory step for
  plugin and connector install suggestions.
  - Slim `request_plugin_install` so it only handles the actual
  elicitation, instead of carrying the full discoverable list in its
  prompt.
  - Emit send-time telemetry when an install elicitation is dispatched,
  including requested tool identity in the event payload.
  - Emit install-result telemetry through `SessionTelemetry`, including
  tool type, user response action, and completion status.
  - Update registration and tests to cover the new two-step flow while
  keeping the existing `tool_suggest` feature gate unchanged.
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-tools`
  - `cargo test -p codex-core request_plugin_install`
  - `cargo test -p codex-core list_available_plugins_to_install`
  - `cargo test -p codex-core
  install_suggestion_tools_can_be_registered_without_search_tool`
  - `cargo test -p codex-otel
  manager_records_plugin_install_suggestion_metric`
  - `cargo test -p codex-otel
  manager_records_plugin_install_elicitation_sent_metric`
  - `just fix -p codex-core`
  - `just fix -p codex-tools`
  - `just fix -p codex-otel`
  - `cargo check -p codex-core`

  Matthew Zeng · 2026-05-19 14:45:37 -07:00
  8335b56c33
• goal: pause continuation loops on usage limits and blockers (#23094)

  Addresses #22833, #22245, #23067
  
  ## Why
  `/goal` can keep synthesizing turns even when the next turn cannot make
  meaningful progress. Hard usage exhaustion can replay failing turns, and
  repeated permission or external-resource blockers can keep burning
  tokens while waiting for user or system intervention.
  
  ## What changed
  - Add resumable `blocked` and `usageLimited` goal states. As with
  `paused`, goal continuation stops with these states.
  - Move to `usageLimited` after usage-limit failures.
  - Allow the built-in `update_goal` tool to set `blocked` only under
  explicit repeated-impasse guidance. Updated goal continuation prompt to
  specify that agent should use `blocked` only when it has made at least
  three attempts to get past an impasse.
  
  Most of the files touched by this PR are because of the small app server
  protocol update.
  
  ## Validation
  
  I manually reproduced a number of situations where an agent can run into
  a true impasse and verified that it properly enters `blocked` state. I
  then resumed and verified that it once again entered `blocked` state
  several turns later if the impasse still exists.
  
  I also manually reproduced the usage-limit condition by creating a
  simulated responses API endpoint that returns 429 errors with the
  appropriate error message. Verified that the goal runtime properly moves
  the goal into `usageLimited` state and TUI UI updates appropriately.
  Verified that `/goal resume` resumes (and immediately goes back into
  `ussageLImited` state if appropriate).
  
  
  ## Follow-up PRs
  
  Small changes will be needed to the GUI clients to properly handle the
  two new states.

  Eric Traut · 2026-05-18 11:28:53 -07:00
  0d344aca9b
• chore: goal resumed metrics (#23301)

  Add metrics for goal resume

  jif-oai · 2026-05-18 15:19:23 +02:00
  500ef67ed1
• [codex] Use compaction_trigger item for remote compaction v2 (#22809)

  ## Why
  
  Remote compaction v2 was still using `context_compaction` as both the
  request trigger and the compacted output shape. The Responses API now
  has the landed contract for this flow: Codex sends a dedicated `{
  "type": "compaction_trigger" }` input item, and the backend returns the
  standard `compaction` output item with encrypted content.
  
  This aligns the v2 path with that wire contract while preserving the
  existing local compacted-history post-processing behavior.
  
  ## What changed
  
  - Add `ResponseItem::CompactionTrigger` and regenerate the app-server
  protocol schema fixtures.
  - Send `compaction_trigger` from `remote_compaction_v2` instead of a
  payload-less `context_compaction`.
  - Collect exactly one backend `compaction` output item, then reuse the
  existing compacted-history rebuilding path.
  - Treat the trigger item as a transient request marker rather than model
  output or persisted rollout/memory content.
  
  ## Verification
  
  - `cargo test -p codex-protocol compaction_trigger`
  - `cargo test -p codex-core remote_compact_v2`
  - `cargo test -p codex-core compact_remote_v2`
  - `cargo test -p codex-core
  responses_websocket_sends_response_processed_after_remote_compaction_v2`
  - `just write-app-server-schema`
  - `cargo test -p codex-app-server-protocol schema_fixtures`

  jif-oai · 2026-05-15 11:40:35 +02:00
  0322ac3df8
• Simplify MCP tool handler plumbing (#21595)

  ## Why
  The MCP tool path had accumulated a few core-owned special cases: a
  dedicated payload variant, resolver plumbing, a legacy `AfterToolUse`
  translation path, and a side channel for parallel-call metadata. That
  made `ToolRegistry` and the spec builder know more about MCP than they
  needed to.
  
  This change moves MCP-specific execution details back onto `ToolInfo`
  and `McpHandler` so `codex-core` can treat MCP calls like normal
  function calls while still preserving MCP-specific dispatch and
  telemetry behavior where it belongs.
  
  ## What changed
  - removed `resolve_mcp_tool_info`, `ToolPayload::Mcp`, `ToolKind`, and
  the remaining registry-side MCP resolver path
  - stored MCP routing metadata directly on `McpHandler` and `ToolInfo`,
  including `supports_parallel_tool_calls`
  - deleted the legacy `AfterToolUse` consumer in `core`, which removes
  the need for handler-specific `after_tool_use_payload` implementations
  - switched tool-result telemetry to handler-provided tags and kept
  MCP-specific dispatch payload construction inside the handler
  - simplified tool spec planning/building by passing `ToolInfo` directly
  and dropping the direct/deferred MCP wrapper structs and the
  parallel-server side table
  
  ## Testing
  - `cargo check -p codex-core -p codex-mcp -p codex-otel`
  - `cargo test -p codex-core
  mcp_parallel_support_uses_exact_payload_server`
  - `cargo test -p codex-core
  direct_mcp_tools_register_namespaced_handlers`
  - `cargo test -p codex-core
  search_tool_description_lists_each_mcp_source_once`
  - `cargo test -p codex-mcp
  list_all_tools_uses_startup_snapshot_while_client_is_pending`
  - `just fix -p codex-core -p codex-mcp -p codex-otel`

  pakrym-oai · 2026-05-12 00:11:31 +00:00
  ed5944ba1d
• Add production startup and TTFT telemetry (#22198)

  ## Why
  
  While investigating `codex exec hi` startup latency, the useful
  questions were not "is startup slow?" but "which durable bucket is slow
  in production?"
  
  The path we observed has a few distinct stages:
  
  1. `thread/start` creates the session
  2. startup prewarm builds the turn context, tools, and prompt
  3. startup prewarm warms the websocket
  4. the first real turn resolves the prewarm
  5. the model produces the first token
  
  Before this PR, production telemetry had some of the raw measurements
  already:
  
  - aggregate startup-prewarm duration / age-at-first-turn metrics
  - TTFT as a metric
  - websocket request telemetry
  
  But there was no coherent production event stream for the startup
  breakdown itself, and TTFT was metric-only. That made it hard to answer
  the same latency questions from OpenTelemetry-backed logs without adding
  one-off local instrumentation.
  
  ## What changed
  
  Add durable production telemetry on the existing `SessionTelemetry`
  path:
  
  - new `codex.startup_phase` OTel log/trace events plus
  `codex.startup.phase.duration_ms`
  - new `codex.turn_ttft` OTel log/trace events while preserving the
  existing TTFT metric
  
  The startup phase event is emitted for the coarse buckets we actually
  observed while running `exec hi`:
  
  - `thread_start_create_thread`
  - `startup_prewarm_total`
  - `startup_prewarm_create_turn_context`
  - `startup_prewarm_build_tools`
  - `startup_prewarm_build_prompt`
  - `startup_prewarm_websocket_warmup`
  - `startup_prewarm_resolve`
  
  These phases are intentionally low-cardinality so they remain safe as
  production telemetry tags.
  
  ## Why this shape
  
  This keeps the instrumentation on the same production path as the rest
  of the session telemetry instead of adding a local debug-only trace
  mode. It also avoids changing startup behavior:
  
  - prewarm still runs
  - no control flow changes
  - no extra remote calls
  - no user-visible behavior changes
  
  One boundary is intentional: very early process bootstrap that happens
  before a session exists is not included here, because this PR uses
  session-scoped production telemetry. The expensive buckets we were
  trying to understand after `thread/start` are now covered durably.
  
  ## Verification
  
  - `cargo test -p codex-otel`
  - `cargo test -p codex-core turn_timing`
  - `cargo test -p codex-core
  regular_turn_emits_turn_started_without_waiting_for_startup_prewarm`
  - `cargo test -p codex-core
  interrupting_regular_turn_waiting_on_startup_prewarm_emits_turn_aborted`
  - `cargo test -p codex-app-server thread_start`
  - `just fix -p codex-otel -p codex-core -p codex-app-server`
  
  I also ran `cargo test -p codex-core`; it built successfully and then
  hit an existing unrelated stack overflow in
  `tools::handlers::multi_agents::tests::tool_handlers_cascade_close_and_resume_and_keep_explicitly_closed_subtrees_closed`.

  Matthew Zeng · 2026-05-11 23:58:36 +00:00
  e15ecc9c35
• Add process-scoped SQLite telemetry (#22154)

  ## Summary
  - add SQLite init, backfill-gate, and fallback telemetry without
  introducing a cross-cutting state-db access wrapper
  - install one process-scoped telemetry sink after OTEL startup and let
  low-level state/rollout paths emit through it directly
  - add process-start metrics for the process owners that initialize
  SQLite
  
  ---------
  
  Co-authored-by: Owen Lin <owen@openai.com>

  jif-oai · 2026-05-11 11:32:40 -07:00
  b401666ca5
• codex-otel: validate provider span attributes consistently (#21749)

  Provider initialization installs process-global OTEL state, so invalid
  trace metadata needs to fail before setup begins.
  
  Use the same span attribute validator as config loading when traces are
  exported so provider startup enforces the config contract without
  duplicating validation logic.

  bbrown-oai · 2026-05-08 08:20:49 -07:00
  607b0dd1f0
• codex-otel: add configurable trace metadata (#21556)

  Add Codex config for static trace span attributes and structured W3C
  tracestate field upserts. The config flows through OtelSettings so
  callers can attach trace metadata without touching every span call site.
  
  Apply span attributes with an SDK span processor so every exported
  trace span carries the configured metadata. Model tracestate as nested
  member fields so configured keys can be upserted while unrelated
  propagated state in the same member is preserved.
  
  Validate configured tracestate before installing provider-global state,
  including header-unsafe values the SDK does not reject by itself. This
  keeps Codex from propagating malformed trace context from config.
  
  Update the config schema, public docs, and OTLP loopback coverage for
  config parsing, span export, propagation, and invalid-header rejection.

  bbrown-oai · 2026-05-07 16:06:57 -07:00
  31b233c7c6
• revert legacy notify deprecation (#21152)

  # Why
  
  Revert #20524 for now because the computer use plugin has not migrated
  off legacy `notify` yet. Keeping the deprecation in place today would
  show users a warning before the plugin path is ready to move, so this
  rolls the change back until that migration is complete.
  
  # What
  
  - revert the legacy `notify` deprecation change from #20524
  - restore the prior `notify` behavior and remove the temporary
  deprecation metrics/docs from that change
  
  Once the computer use plugin has migrated, we can land the same
  deprecation again.

  Abhinav · 2026-05-05 10:34:44 -07:00
  13be504063
• Add goal lifecycle metrics (#20799)

  ## Why
  
  Adding goal metrics makes it possible to track how often goals are
  created, completed, and stopped by budget limits, plus the final token
  and wall-clock usage for terminal outcomes.
  
  ## What Changed
  
  - Added OpenTelemetry metric constants for goal lifecycle tracking:
  - `codex.goal.created`: increments each time a new persisted goal is
  created or an existing goal is replaced with a new objective.
  - `codex.goal.completed`: increments when a goal transitions to
  `complete`.
  - `codex.goal.budget_limited`: increments when a goal transitions to
  `budget_limited` because its token budget has been reached.
  - `codex.goal.token_count`: records the final persisted token count when
  a goal transitions to `complete` or `budget_limited`.
  - `codex.goal.duration_s`: records the final persisted elapsed
  wall-clock time, in seconds, when a goal transitions to `complete` or
  `budget_limited`.
  - Emitted creation metrics when a goal is created or replaced.
  - Emitted terminal outcome counters and final usage histograms when a
  goal transitions to `complete` or `budget_limited`, avoiding
  double-counting later in-flight accounting for already budget-limited
  goals.
  - Added focused `codex-core` tests for create/complete metrics and
  one-time budget-limit metrics.

  Eric Traut · 2026-05-05 09:21:54 -07:00
  91b7350187
• feat: add remote compaction v2 Responses client path (#20773)

  ## Why
  
  This adds the `remote_compaction_v2` client path so remote compaction
  can run through the normal Responses stream and install a
  `context_compaction` item that trigger a compaction.
  
  The goal is to migrate some of the compaction logic on the client side
  
  We keeps the v2 transport behind a feature flag while letting follow-up
  requests reuse the compacted context instead of falling back to the
  legacy compaction item shape.
  
  ## What changed
  
  - add `ResponseItem::ContextCompaction` and refresh the generated
  app-server / schema / TypeScript fixtures that expose response items on
  the wire
  - add `core/src/compact_remote_v2.rs` to send compaction through the
  standard streamed Responses client, require exactly one
  `context_compaction` output item, and install that item into compacted
  history
  - route manual compact and auto-compaction through the v2 path when
  `remote_compaction_v2` is enabled, while keeping the existing remote
  compaction path as the fallback
  - preserve the new item type across history retention, follow-up request
  construction, telemetry, rollout persistence, and rollout-trace
  normalization
  - add targeted coverage for the feature flag, `context_compaction`
  serialization, rollout-trace normalization, and remote-compaction
  follow-up behavior
  
  ## Verification
  
  - added protocol tests for `context_compaction`
  serialization/deserialization in `protocol/src/models.rs`
  - added rollout-trace coverage for `context_compaction` normalization in
  `rollout-trace/src/reducer/conversation_tests.rs`
  - added remote compaction integration coverage for v2 follow-up reuse
  and mixed compaction output streams in
  `core/tests/suite/compact_remote.rs`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-05-04 14:15:01 +02:00
  d927f61208
• deprecate legacy notify (#20524)

  # Why
  
  `notify` is the remaining compatibility surface from the legacy hook
  implementation. The newer lifecycle hook engine now owns the active hook
  system, so we should start steering users away from adding new `notify`
  configs before removing the old path entirely. This also adds a
  lightweight watchpoint for the deprecation so we can see how much legacy
  usage remains before the clean drop.
  
  # What
  
  - emit a startup deprecation notice when a non-empty `notify` command is
  configured
  - emit `codex.notify.configured` when a session starts with legacy
  `notify` configured
  - emit `codex.notify.run` when the legacy notify path fires after a
  completed turn
  - mark `notify` as deprecated in the config schema and repo docs
  - remove the orphaned `codex-rs/hooks/src/user_notification.rs` file
  that is no longer compiled
  - add regression coverage for the new deprecation notice
  
  # Next steps
  
  A follow-up PR can remove the legacy notify path entirely once we are
  ready for the clean drop. Before then, we can watch
  `codex.notify.configured` and `codex.notify.run` to understand the
  deprecation impact and remaining active usage. The cleanup PR should
  then delete the `notify` config field, the `legacy_notify`
  implementation, the old compatibility dispatch types and callsites that
  only exist for the legacy path, and the remaining compatibility
  docs/tests.
  
  # Testing
  
  - `cargo test -p codex-hooks`
  - `cargo test -p codex-config`
  - `cargo test -p codex-core emits_deprecation_notice_for_notify`

  Abhinav · 2026-05-01 17:35:21 +00:00
  78baa20780
• install WFP filters for Windows sandbox setup (#20101)

  ## Summary
  
  This PR installs a first wave of WFP (Windows Filtering Platform)
  filters that reduce the surface area of network egress vulnerabilities
  for the Windows Sandbox.
  
  - Add persistent Windows Filtering Platform provider, sublayer, and
  filters for the Windows sandbox offline account.
  - Install WFP filters during elevated full setup, log failures
  non-fatally, and emit setup metrics when analytics are enabled.
  - Bump the Windows sandbox setup version so existing users rerun full
  setup and receive the new filters.
  
  ## What WFP is
  Windows Filtering Platform (WFP) is the low-level Windows networking
  policy engine underneath things like Windows Firewall. It lets
  privileged code install persistent filtering rules at specific network
  stack layers, with conditions like "only traffic from this Windows
  account" or "only this remote port," and an action like block.
  
  In this change, we create a Codex-owned persistent WFP provider and
  sublayer, then install block filters scoped to the Windows sandbox's
  offline user account via `ALE_USER_ID`. That means the filters are
  targeted at sandboxed processes running as that account, rather than
  globally affecting the host.
  
  ## Initial filter set
  We are starting with 12 concrete WFP filters across a few high-value
  bypass surfaces. The table below describes the filter families rather
  than one filter per row:
  
  | Area | Concrete filters | Purpose |
  | --- | --- | --- |
  | ICMP | 4 filters: ICMP v4/v6 on `ALE_AUTH_CONNECT` and
  `ALE_RESOURCE_ASSIGNMENT` | Block direct ping-style network reachability
  checks from the offline account. |
  | DNS | 2 filters: remote port `53` on `ALE_AUTH_CONNECT_V4/V6` | Block
  direct DNS queries that bypass our intended proxy/offline path. |
  | DNS-over-TLS | 2 filters: remote port `853` on
  `ALE_AUTH_CONNECT_V4/V6` | Block encrypted DNS attempts that could
  bypass ordinary DNS interception. |
  | SMB / NetBIOS | 4 filters: remote ports `445` and `139` on
  `ALE_AUTH_CONNECT_V4/V6` | Block Windows file-sharing/network share
  traffic from sandboxed processes. |
  
  For IPv4/IPv6 coverage, the port-based filters are installed on both
  `ALE_AUTH_CONNECT_V4` and `ALE_AUTH_CONNECT_V6`. ICMP also gets both
  connect-layer and resource-assignment-layer coverage because ICMP
  traffic is shaped differently from ordinary TCP/UDP port traffic.
  
  ## Validation
  - `cargo fmt -p codex-windows-sandbox` (completed with existing
  stable-rustfmt warnings about `imports_granularity = Item`)
  - `cargo test -p codex-windows-sandbox wfp::tests`
  - `cargo test -p codex-windows-sandbox` (fails in existing legacy
  PowerShell sandbox tests because `Microsoft.PowerShell.Utility` could
  not be loaded; WFP tests passed before that failure)

  iceweasel-oai · 2026-04-30 12:39:01 -07:00
  8121710ffe
• [codex] Add token usage to turn tracing spans (#19432)

  ## Why
  
  Slow Codex turns are easier to debug when token usage is visible in the
  trace itself, without joining against separate analytics. This adds
  token usage to existing turn-handling spans for regular user turns only.
  
  [Example
  turn](https://openai.datadoghq.com/apm/trace/9d353efa2cb5de1f4c5b93dc33c3df04?colorBy=service&graphType=flamegraph&shouldShowLegend=true&sort=time&spanID=3555541504891512675&spanViewType=metadata&traceQuery=)
  <img width="1447" height="967" alt="Screenshot 2026-04-24 at 3 03 07 PM"
  src="https://github.com/user-attachments/assets/ab7bb187-e7fc-41f0-a366-6c44610b2b2c"
  />
  
  ## What Changed
  
  Added response-level token fields on completed handle_responses spans:
  
  gen_ai.usage.input_tokens
  gen_ai.usage.cache_read.input_tokens
  gen_ai.usage.output_tokens
  codex.usage.reasoning_output_tokens
  codex.usage.total_tokens
  Added aggregate token fields on regular turn spans:
  
  codex.turn.token_usage.*
  Added an explicit regular-turn opt-in via
  SessionTask::records_turn_token_usage_on_span() so this is not coupled
  to span-name strings.
  
  ## Testing
  
  - `cargo test -p codex-otel`
  - `cargo test -p codex-core
  turn_and_completed_response_spans_record_token_usage`
  - `just fmt`
  - `just fix -p codex-core`
  - `just fix -p codex-otel`
  - Manual local Electron/app-server smoke test: regular user turn emits
  the new span fields
  
  Known status: `cargo test -p codex-core` was attempted and failed in
  unrelated existing areas: config approvals, request-permissions,
  git-info ordering, and subagent metadata persistence.

  charley-openai · 2026-04-28 11:41:32 -07:00
  de2ccf9473
• Remove ghost snapshots (#19481)

  ## Summary
  - Remove `ghost_snapshot` / `GhostCommit` from the Responses API surface
  and generated SDK/schema artifacts.
  - Keep legacy config loading compatible, but make undo a no-op that
  reports the feature is unavailable.
  - Clean up core history, compaction, telemetry, rollout, and tests to
  stop carrying ghost snapshot items.
  
  ## Testing
  - Unit tests passed for `codex-protocol`, `codex-core` targeted undo and
  compaction flows, `codex-rollout`, and `codex-app-server-protocol`.
  - Regenerated config and app-server schemas plus Python SDK artifacts
  and verified they match the checked-in outputs.

  pakrym-oai · 2026-04-27 18:48:57 -07:00
  4e05f3053c
• Add safety check notification and error handling (#19055)

  Adds a new app-server notification that fires when a user account has
  been flagged for potential safety reasons.

  Eric Traut · 2026-04-22 22:24:12 -07:00
  bbff4ee61a
• feat: Fairly trim skill descriptions within context budget (#18925)

  Preserve skill name/path entries whenever possible and trim descriptions
  first, using round-robin character allocation so short descriptions do
  not waste budget.

  xl-openai · 2026-04-22 12:33:29 -07:00
  b77791c228
• feat: add explicit AgentIdentity auth mode (#18785)

  ## Summary
  
  This PR adds `CodexAuth::AgentIdentity` as an explicit auth mode.
  
  An AgentIdentity auth record is a standalone `auth.json` mode. When
  `AuthManager::auth().await` loads that mode, it registers one
  process-scoped task and stores it in runtime-only state on the auth
  value. Header creation stays synchronous after that because the task is
  initialized before callers receive the auth object.
  
  This PR also removes the old feature flag path. AgentIdentity is
  selected by explicit auth mode, not by a hidden flag or lazy mutation of
  ChatGPT auth records.
  
  Reference old stack: https://github.com/openai/codex/pull/17387/changes
  
  ## Design Decisions
  
  - AgentIdentity is a real auth enum variant because it can be the only
  credential in `auth.json`.
  - The process task is ephemeral runtime state. It is not serialized and
  is not stored in rollout/session data.
  - Account/user metadata needed by existing Codex backend checks lives on
  the AgentIdentity record for now.
  - `is_chatgpt_auth()` remains token-specific.
  - `uses_codex_backend()` is the broader predicate for ChatGPT-token auth
  and AgentIdentity auth.
  
  ## Stack
  
  1. https://github.com/openai/codex/pull/18757: full revert
  2. https://github.com/openai/codex/pull/18871: isolated Agent Identity
  crate
  3. This PR: explicit AgentIdentity auth mode and startup task allocation
  4. https://github.com/openai/codex/pull/18811: migrate Codex backend
  auth callsites through AuthProvider
  5. https://github.com/openai/codex/pull/18904: accept AgentIdentity JWTs
  and load `CODEX_AGENT_IDENTITY`
  
  ## Testing
  
  Tests: targeted Rust checks, cargo-shear, Bazel lock check, and CI.

  efrazer-oai · 2026-04-21 22:33:24 -07:00
  69c8913e24
• feat: add metric to track the number of turns with memory usage (#18662)

  Add a metric `codex.turn.memory` to know if a turn used memories or not.
  This is not part of the other turn metrics as a label to limit
  cardinality

  jif-oai · 2026-04-20 14:31:22 +01:00
  2c59806fe0
• feat: Budget skill metadata and surface trimming as a warning (#18298)

  Cap the model-visible skills section to a small share of the context
  window, with a fallback character budget, and keep only as many implicit
  skills as fit within that budget.
  
  Emit a non-fatal warning when enabled skills are omitted, and add a new
  app-server warning notification
  
  Record thread-start skill metrics for total enabled skills, kept skills,
  and whether truncation happened
  
  ---------
  
  Co-authored-by: Matthew Zeng <mzeng@openai.com>
  Co-authored-by: Codex <noreply@openai.com>

  xl-openai · 2026-04-17 18:11:47 -07:00
  3f7222ec76
• Stream apply_patch changes (#17862)

  Adds new events for streaming apply_patch changes from responses api.
  This is to enable clients to show progress during file writes.
  
  Caveat: This does not work with apply_patch in function call mode, since
  that required adding streaming json parsing.

  Akshay Nathan · 2026-04-16 18:12:19 -07:00
  7995c66032
• Add OTEL metrics for hook runs (#18026)

  # Why
  We already emit analytics for completed hook runs, but we don't have
  matching OTEL metrics to track hook volume and latency.
  
  # What
  - add `codex.hooks.run` and `codex.hooks.run.duration_ms`
  - tag both metrics with `hook_name`, `source`, and `status`
  - emit the metrics from the completed hook path
  
  Verified locally against a dummy OTLP collector
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Abhinav · 2026-04-16 21:30:38 +00:00
  d9c71d41a9
• [codex] reduce module visibility (#16978)

  ## Summary
  - reduce public module visibility across Rust crates, preferring private
  or crate-private modules with explicit crate-root public exports
  - update external call sites and tests to use the intended public crate
  APIs instead of reaching through module trees
  - add the module visibility guideline to AGENTS.md
  
  ## Validation
  - `cargo check --workspace --all-targets --message-format=short` passed
  before the final fix/format pass
  - `just fix` completed successfully
  - `just fmt` completed successfully
  - `git diff --check` passed

  pakrym-oai · 2026-04-07 08:03:35 -07:00
  413c1e1fdf
• otel: remove the last workspace crate feature (#16469)

  ## Why
  
  `codex-otel` still carried `disable-default-metrics-exporter`, which was
  the last remaining workspace crate feature.
  
  We are removing workspace crate features because they do not fit our
  current build model well:
  
  - our Bazel setup does not honor crate features today, which can let
  feature-gated issues go unnoticed
  - they create extra crate build permutations that we want to avoid
  
  For this case, the feature was only being used to keep the built-in
  Statsig metrics exporter off in test and debug-oriented contexts. This
  repo already treats `debug_assertions` as the practical proxy for that
  class of behavior, so OTEL should follow the same convention instead of
  keeping a dedicated crate feature alive.
  
  ## What changed
  
  - removed `disable-default-metrics-exporter` from
  `codex-rs/otel/Cargo.toml`
  - removed the `codex-otel` dev-dependency feature activation from
  `codex-rs/core/Cargo.toml`
  - changed `codex-rs/otel/src/config.rs` so the built-in
  `OtelExporter::Statsig` default resolves to `None` when
  `debug_assertions` is enabled, with a focused unit test covering that
  behavior
  - removed the final feature exceptions from
  `.github/scripts/verify_cargo_workspace_manifests.py`, so workspace
  crate features are now hard-banned instead of temporarily allowlisted
  - expanded the verifier error message to explain the Bazel mismatch and
  build-permutation cost behind that policy
  
  ## How tested
  
  - `python3 .github/scripts/verify_cargo_workspace_manifests.py`
  - `cargo test -p codex-otel`
  - `cargo test -p codex-core
  metrics_exporter_defaults_to_statsig_when_missing`
  - `cargo test -p codex-app-server app_server_default_analytics_`
  - `just bazel-lock-check`

  Michael Bolin · 2026-04-01 13:45:23 -07:00
  323aa968c3
• chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)

  ## Why
  
  `argument-comment-lint` was green in CI even though the repo still had
  many uncommented literal arguments. The main gap was target coverage:
  the repo wrapper did not force Cargo to inspect test-only call sites, so
  examples like the `latest_session_lookup_params(true, ...)` tests in
  `codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.
  
  This change cleans up the existing backlog, makes the default repo lint
  path cover all Cargo targets, and starts rolling that stricter CI
  enforcement out on the platform where it is currently validated.
  
  ## What changed
  
  - mechanically fixed existing `argument-comment-lint` violations across
  the `codex-rs` workspace, including tests, examples, and benches
  - updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
  `tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
  `--all-targets` unless the caller explicitly narrows the target set
  - fixed both wrappers so forwarded cargo arguments after `--` are
  preserved with a single separator
  - documented the new default behavior in
  `tools/argument-comment-lint/README.md`
  - updated `rust-ci` so the macOS lint lane keeps the plain wrapper
  invocation and therefore enforces `--all-targets`, while Linux and
  Windows temporarily pass `-- --lib --bins`
  
  That temporary CI split keeps the stricter all-targets check where it is
  already cleaned up, while leaving room to finish the remaining Linux-
  and Windows-specific target-gated cleanup before enabling
  `--all-targets` on those runners. The Linux and Windows failures on the
  intermediate revision were caused by the wrapper forwarding bug, not by
  additional lint findings in those lanes.
  
  ## Validation
  
  - `bash -n tools/argument-comment-lint/run.sh`
  - `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
  - shell-level wrapper forwarding check for `-- --lib --bins`
  - shell-level wrapper forwarding check for `-- --tests`
  - `just argument-comment-lint`
  - `cargo test` in `tools/argument-comment-lint`
  - `cargo test -p codex-terminal-detection`
  
  ## Follow-up
  
  - Clean up remaining Linux-only target-gated callsites, then switch the
  Linux lint lane back to the plain wrapper invocation.
  - Clean up remaining Windows-only target-gated callsites, then switch
  the Windows lint lane back to the plain wrapper invocation.

  Michael Bolin · 2026-03-27 19:00:44 -07:00
  61dfe0b86c
• plugins: Clean up stale curated plugin sync temp dirs and add sync metrics (#16035)

  1. Keep curated plugin staging directories under TempDir ownership until
  activation succeeds, so failed git/HTTP sync attempts do not leak
  plugins-clone-*.
  2. Best-effort clean up stale plugins-clone-* directories before
  creating a new staged repo, using a conservative age threshold.
  3. Emit OTEL counters for curated plugin startup sync transport attempts
  and final outcome across git and HTTP paths.

  xl-openai · 2026-03-27 14:21:18 -07:00
  81abb44f68
• chore: tty metric (#15766)
  jif-oai · 2026-03-25 13:34:43 +00:00
  047ea642d2
• Move auth code into login crate (#15150)

  - Move the auth implementation and token data into codex-login.
  - Keep codex-core re-exporting that surface from codex-login for
  existing callers.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-03-19 18:58:17 -07:00
  2aa4873802
• Log automated reviewer approval sources distinctly (#15201)

  ## Summary
  
  - log guardian-reviewed tool approvals as `source=automated_reviewer` in
  `codex.tool_decision`
  - keep direct user approvals as `source=user` and config-driven
  approvals as `source=config`
  
  ## Testing
  
  -
  `/Users/gabec/.codex/skills/codex-oss-fastdev/scripts/codex-rs-fmt-quiet.sh`
  -
  `/Users/gabec/.codex/skills/codex-oss-fastdev/scripts/codex-rs-test-quiet.sh
  -p codex-otel` (fails in sandboxed loopback bind tests under
  `otel/tests/suite/otlp_http_loopback.rs`)
  - `cargo test -p codex-core guardian -- --nocapture` (original-tree run
  reached Guardian tests and only hit sandbox-related listener/proxy
  failures)
  
  Co-authored-by: Codex <noreply@openai.com>

  gabec-openai · 2026-03-19 12:10:41 -07:00
  fe287ac467
• chore: add metrics for profile (#15180)
  jif-oai · 2026-03-19 15:48:02 +00:00
  2cf4d5ef35
• Revert "fix: harden plugin feature gating" (#15102)

  Reverts openai/codex#15020
  
  I messed up the commit in my PR and accidentally merged changes that
  were still under review.

  xl-openai · 2026-03-18 15:19:29 -07:00
  86982ca1f9
• fix: harden plugin feature gating (#15020)

  1. Use requirement-resolved config.features as the plugin gate.
  2. Guard plugin/list, plugin/read, and related flows behind that gate.
  3. Skip bad marketplace.json files instead of failing the whole list.
  4. Simplify plugin state and caching.

  xl-openai · 2026-03-18 10:11:43 -07:00
  580f32ad2a
• Add auth env observability (#14905)

  CXC-410 Emit Env Var Status with `/feedback` report
  
  Add more observability on top of #14611 
  
  [Unset](https://openai.sentry.io/issues/7340419168/?project=4510195390611458&query=019cfa8d-c1ba-7002-96fa-e35fc340551d&referrer=issue-stream)
  
  [Set](https://openai.sentry.io/issues/7340426331/?project=4510195390611458&query=019cfa91-aba1-7823-ab7e-762edfbc0ed4&referrer=issue-stream)
  <img width="1063" height="610" alt="image"
  src="https://github.com/user-attachments/assets/937ab026-1c2d-4757-81d5-5f31b853113e"
  />
  
  
  ###### Summary
  - Adds auth-env telemetry that records whether key auth-related env
  overrides were present on session start and request paths.
  - Threads those auth-env fields through `/responses`, websocket, and
  `/models` telemetry and feedback metadata.
  - Buckets custom provider `env_key` configuration to a safe
  `"configured"` value instead of emitting raw config text.
  - Keeps the slice observability-only: no raw token values or raw URLs
  are emitted.
  
  ###### Rationale (from spec findings)
  - 401 and auth-path debugging needs a way to distinguish env-driven auth
  paths from sessions with no auth env override.
  - Startup and model-refresh failures need the same auth-env diagnostics
  as normal request failures.
  - Feedback and Sentry tags need the same auth-env signal as OTel events
  so reports can be triaged consistently.
  - Custom provider config is user-controlled text, so the telemetry
  contract must stay presence-only / bucketed.
  
  ###### Scope
  - Adds a small `AuthEnvTelemetry` bundle for env presence collection and
  threads it through the main request/session telemetry paths.
  - Does not add endpoint/base-url/provider-header/geo routing attribution
  or broader telemetry API redesign.
  
  ###### Trade-offs
  - `provider_env_key_name` is bucketed to `"configured"` instead of
  preserving the literal configured env var name.
  - `/models` is included because startup/model-refresh auth failures need
  the same diagnostics, but broader parity work remains out of scope.
  - This slice keeps the existing telemetry APIs and layers auth-env
  fields onto them rather than redesigning the metadata model.
  
  ###### Client follow-up
  - Add the separate endpoint/base-url attribution slice if routing-source
  diagnosis is still needed.
  - Add provider-header or residency attribution only if auth-env presence
  proves insufficient in real reports.
  - Revisit whether any additional auth-related env inputs need safe
  bucketing after more 401 triage data.
  
  ###### Testing
  - `cargo test -p codex-core emit_feedback_request_tags -- --nocapture`
  - `cargo test -p codex-core
  collect_auth_env_telemetry_buckets_provider_env_key_name -- --nocapture`
  - `cargo test -p codex-core
  models_request_telemetry_emits_auth_env_feedback_tags_on_failure --
  --nocapture`
  - `cargo test -p codex-otel
  otel_export_routing_policy_routes_api_request_auth_observability --
  --nocapture`
  - `cargo test -p codex-otel
  otel_export_routing_policy_routes_websocket_connect_auth_observability
  -- --nocapture`
  - `cargo test -p codex-otel
  otel_export_routing_policy_routes_websocket_request_transport_observability
  -- --nocapture`
  - `cargo test -p codex-core --no-run --message-format short`
  - `cargo test -p codex-otel --no-run --message-format short`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Colin Young · 2026-03-17 14:26:27 -07:00
  0d2ff40a58
• fix(core): prevent hanging turn/start due to websocket warming issues (#14838)

  ## Description
  
  This PR fixes a bad first-turn failure mode in app-server when the
  startup websocket prewarm hangs. Before this change, `initialize ->
  thread/start -> turn/start` could sit behind the prewarm for up to five
  minutes, so the client would not see `turn/started`, and even
  `turn/interrupt` would block because the turn had not actually started
  yet.
  
  Now, we:
  - set a (configurable) timeout of 15s for websocket startup time,
  exposed as `websocket_startup_timeout_ms` in config.toml
  - `turn/started` is sent immediately on `turn/start` even if the
  websocket is still connecting
  - `turn/interrupt` can be used to cancel a turn that is still waiting on
  the websocket warmup
  - the turn task will wait for the full 15s websocket warming timeout
  before falling back
  
  ## Why
  
  The old behavior made app-server feel stuck at exactly the moment the
  client expects turn lifecycle events to start flowing. That was
  especially painful for external clients, because from their point of
  view the server had accepted the request but then went silent for
  minutes.
  
  ## Configuring the websocket startup timeout
  Can set it in config.toml like this:
  ```
  [model_providers.openai]
  supports_websockets = true
  websocket_connect_timeout_ms = 15000
  ```

  Owen Lin · 2026-03-17 10:07:46 -07:00
  6ea041032b
• Apply argument comment lint across codex-rs (#14652)

  ## Why
  
  Once the repo-local lint exists, `codex-rs` needs to follow the
  checked-in convention and CI needs to keep it from drifting. This commit
  applies the fallback `/*param*/` style consistently across existing
  positional literal call sites without changing those APIs.
  
  The longer-term preference is still to avoid APIs that require comments
  by choosing clearer parameter types and call shapes. This PR is
  intentionally the mechanical follow-through for the places where the
  existing signatures stay in place.
  
  After rebasing onto newer `main`, the rollout also had to cover newly
  introduced `tui_app_server` call sites. That made it clear the first cut
  of the CI job was too expensive for the common path: it was spending
  almost as much time installing `cargo-dylint` and re-testing the lint
  crate as a representative test job spends running product tests. The CI
  update keeps the full workspace enforcement but trims that extra
  overhead from ordinary `codex-rs` PRs.
  
  ## What changed
  
  - keep a dedicated `argument_comment_lint` job in `rust-ci`
  - mechanically annotate remaining opaque positional literals across
  `codex-rs` with exact `/*param*/` comments, including the rebased
  `tui_app_server` call sites that now fall under the lint
  - keep the checked-in style aligned with the lint policy by using
  `/*param*/` and leaving string and char literals uncommented
  - cache `cargo-dylint`, `dylint-link`, and the relevant Cargo
  registry/git metadata in the lint job
  - split changed-path detection so the lint crate's own `cargo test` step
  runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes
  - continue to run the repo wrapper over the `codex-rs` workspace, so
  product-code enforcement is unchanged
  
  Most of the code changes in this commit are intentionally mechanical
  comment rewrites or insertions driven by the lint itself.
  
  ## Verification
  
  - `./tools/argument-comment-lint/run.sh --workspace`
  - `cargo test -p codex-tui-app-server -p codex-tui`
  - parsed `.github/workflows/rust-ci.yml` locally with PyYAML
  
  ---
  
  * -> #14652
  * #14651

  Michael Bolin · 2026-03-16 16:48:15 -07:00
  b77fe8fefe
• Add auth 401 observability to client bug reports (#14611)

  CXC-392
  
    [With
    401](https://openai.sentry.io/issues/7333870443/?project=4510195390611458&query=019ce8f8-560c-7f10-a00a-c59553740674&referrer=issue-stream)
    <img width="1909" height="555" alt="401 auth tags in Sentry"
    src="https://github.com/user-attachments/assets/412ea950-61c4-4780-9697-15c270971ee3"
    />
  
  
    - auth_401_*: preserved facts from the latest unauthorized response snapshot
    - auth_*: latest auth-related facts from the latest request attempt
    - auth_recovery_*: unauthorized recovery state and follow-up result
  
  
    Without 401
    <img width="1917" height="522" alt="happy-path auth tags in Sentry"
    src="https://github.com/user-attachments/assets/3381ed28-8022-43b0-b6c0-623a630e679f"
    />
  
    ###### Summary
    - Add client-visible 401 diagnostics for auth attachment, upstream auth classification, and 401 request id / cf-ray correlation.
    - Record unauthorized recovery mode, phase, outcome, and retry/follow-up status without changing auth behavior.
    - Surface the highest-signal auth and recovery fields on uploaded client bug reports so they are usable in Sentry.
    - Preserve original unauthorized evidence under `auth_401_*` while keeping follow-up result tags separate.
  
    ###### Rationale (from spec findings)
    - The dominant bucket needed proof of whether the client attached auth before send or upstream still classified the request as missing auth.
    - Client uploads needed to show whether unauthorized recovery ran and what the client tried next.
    - Request id and cf-ray needed to be preserved on the unauthorized response so server-side correlation is immediate.
    - The bug-report path needed the same auth evidence as the request telemetry path, otherwise the observability would not be operationally useful.
  
    ###### Scope
    - Add auth 401 and unauthorized-recovery observability in `codex-rs/core`, `codex-rs/codex-api`, and `codex-rs/otel`, including feedback-tag surfacing.
    - Keep auth semantics, refresh behavior, retry behavior, endpoint classification, and geo-denial follow-up work out of this PR.
  
    ###### Trade-offs
    - This exports only safe auth evidence: header presence/name, upstream auth classification, request ids, and recovery state. It does not export token values or raw upstream bodies.
    - This keeps websocket connection reuse as a transport clue because it can help distinguish stale reused sessions from fresh reconnects.
    - Misroute/base-url classification and geo-denial are intentionally deferred to a separate follow-up PR so this review stays focused on the dominant auth 401 bucket.
  
    ###### Client follow-up
    - PR 2 will add misroute/provider and geo-denial observability plus the matching feedback-tag surfacing.
    - A separate host/app-server PR should log auth-decision inputs so pre-send host auth state can be correlated with client request evidence.
    - `device_id` remains intentionally separate until there is a safe existing source on the feedback upload path.
  
    ###### Testing
    - `cargo test -p codex-core refresh_available_models_sorts_by_priority`
    - `cargo test -p codex-core emit_feedback_request_tags_`
    - `cargo test -p codex-core emit_feedback_auth_recovery_tags_`
    - `cargo test -p codex-core auth_request_telemetry_context_tracks_attached_auth_and_retry_phase`
    - `cargo test -p codex-core extract_response_debug_context_decodes_identity_headers`
    - `cargo test -p codex-core identity_auth_details`
    - `cargo test -p codex-core telemetry_error_messages_preserve_non_http_details`
    - `cargo test -p codex-core --all-features --no-run`
    - `cargo test -p codex-otel otel_export_routing_policy_routes_api_request_auth_observability`
    - `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_connect_auth_observability`
    - `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_request_transport_observability`

  Colin Young · 2026-03-14 15:38:51 -07:00
  d692b74007
• feat: search_tool migrate to bring you own tool of Responses API (#14274)

  ## Why
  
  to support a new bring your own search tool in Responses
  API(https://developers.openai.com/api/docs/guides/tools-tool-search#client-executed-tool-search)
  we migrating our bm25 search tool to use official way to execute search
  on client and communicate additional tools to the model.
  
  ## What
  - replace the legacy `search_tool_bm25` flow with client-executed
  `tool_search`
  - add protocol, SSE, history, and normalization support for
  `tool_search_call` and `tool_search_output`
  - return namespaced Codex Apps search results and wire namespaced
  follow-up tool calls back into MCP dispatch

  Anton Panasenko · 2026-03-11 17:51:51 -07:00
  77b0c75267
• feat(core): emit turn metric for network proxy state (#14250)

  ## Summary
  - add a per-turn `codex.turn.network_proxy` metric constant
  - emit the metric from turn completion using the live managed proxy
  enabled state
  - add focused tests for active and inactive tag emission

  viyatb-oai · 2026-03-11 12:33:10 -07:00
  52a3bde6cc
• fix(otel): make HTTP trace export survive app-server runtimes (#14300)

  ## Summary
  
  This PR fixes OTLP HTTP trace export in runtimes where the previous
  exporter setup was unreliable, especially around app-server usage. It
  also removes the old `codex_otel::otel_provider` compatibility shim and
  switches remaining call sites over to the crate-root
  `codex_otel::OtelProvider` export.
  
  ## What changed
  
  - Use a runtime-safe OTLP HTTP trace exporter path for Tokio runtimes.
  - Add an async HTTP client path for trace export when we are already
  inside a multi-thread Tokio runtime.
  - Make provider shutdown flush traces before tearing down the tracer
  provider.
  - Add loopback coverage that verifies traces are actually sent to
  `/v1/traces`:
    - outside Tokio
    - inside a multi-thread Tokio runtime
    - inside a current-thread Tokio runtime
  - Remove the `codex_otel::otel_provider` shim and update remaining
  imports.
  
  ## Why
  
  I hit cases where spans were being created correctly but never made it
  to the collector. The issue turned out to be in exporter/runtime
  behavior rather than the span plumbing itself. This PR narrows that gap
  and gives us regression coverage for the actual export path.

  Owen Lin · 2026-03-11 12:33:10 -07:00
  fa1242c83b