86 Commits

• auth: move domain mode below app wire types (#29721)

  ## Why
  
  Authentication mode is a domain concept used by login, model selection,
  telemetry, and transports. Keeping the canonical type in app-server
  protocol forces those lower-level crates to depend on an unrelated wire
  API.
  
  ## What changed
  
  - Added canonical `codex_protocol::auth::AuthMode` domain values.
  - Kept the app-server wire DTO unchanged and added an explicit app-side
  conversion.
  - Removed production app-server-protocol dependencies from login,
  model-provider-info, models-manager, and otel call paths.
  
  ## Stack
  
  This is PR 2 of 6, stacked on [PR
  #29714](https://github.com/openai/codex/pull/29714). Review only the
  delta from `codex/split-json-rpc-protocols`. Next: [PR
  #29722](https://github.com/openai/codex/pull/29722).
  
  ## Validation
  
  - Auth and login coverage passed in the focused protocol/domain test
  run.
  - App-server account and auth conversion coverage passed.

  Adam Perry @ OpenAI · 2026-06-24 03:10:20 +00:00
  31372078d1
• core: use current step environments for tools (#29547)

  ## Why
  
  With deferred executors, an environment can become ready between two
  sampling requests in the same turn. The model-visible environment
  update, advertised tools, and eventual tool execution must all describe
  the same request-time view.
  
  Otherwise, a request built while only environment B is ready can
  advertise a tool without an `environment_id`; if higher-priority
  environment A becomes ready before execution, that call could silently
  run in A instead.
  
  This PR is stacked on #29527.
  
  ## Design
  
  `run_turn` captures one `Arc<StepContext>` at each sampling-request
  boundary. That step owns the request's `TurnContext` and environment
  snapshot.
  
  - World-state environment updates and tool planning borrow that same
  step.
  - `ToolCallRuntime` retains the `Arc` while asynchronous tool calls
  execute.
  - `ToolInvocation` carries the step to handlers; its temporary `turn`
  compatibility field is derived from the same object.
  - `ToolRouter` does not retain `StepContext`; it only uses it while
  constructing the request's tool set.
  - With `DeferredExecutor` disabled, step capture keeps using the
  environments frozen at turn start.
  
  Simply: every sampling request gets one consistent picture of its
  environments, from what the model sees through where its tool calls run.
  
  ## What changed
  
  - Build environment-dependent tool specs from the current request's
  `StepContext`.
  - Use that same step for unified exec, legacy shell, `apply_patch`,
  `view_image`, and `request_permissions` execution.
  - Hide environment-backed tools, including `request_permissions`, while
  no environment is attached.
  - Resolve legacy shell paths and metadata from the selected step
  environment instead of the stale turn-start environment.
  - Capture explicit steps at non-turn-loop boundaries such as compaction,
  prompt debug, and startup prewarm.
  - Reconcile prompt-debug history from the same step used to build its
  tools.
  
  ## Follow-up
  
  - Bind yielded code-mode cells to the tool runtime that created them, so
  nested calls made after yielding continue to use the originating
  request's `StepContext`.
  
  ## Test plan
  
  - `just test -p codex-core
  deferred_executor_updates_context_and_tools_after_startup`
  - `just test -p codex-core
  environment_count_controls_environment_backed_tools`
  - `just test -p codex-core
  build_prompt_input_includes_context_and_user_message`

  sayan-oai · 2026-06-23 20:21:13 +00:00
  4cc6a4bab5
• core: use turn-owned world state for inline compaction (#29527)

  ## Why
  
  Follow-up to #29249 and its [compaction review
  thread](https://github.com/openai/codex/pull/29249#discussion_r3455055101).
  
  During a turn, environment readiness can change between sampling
  requests. Inline compaction must render the same model-visible
  `WorldState` used by the request it follows. Rebuilding that state
  during compaction can observe a newer environment, make replacement
  history disagree with what the model saw, and suppress the next
  environment update.
  
  ## What changed
  
  - Make `run_turn` own the current `Arc<WorldState>` and replace it only
  between sampling requests.
  - Build each state from an explicitly chosen environment snapshot, diff
  deferred-executor steps against the turn-owned state, and retain the
  latest state in `ContextManager` only for cross-turn and resume
  tracking.
  - Pass the exact turn-owned state into inline compaction and explicit
  new-context-window replacement.
  - Carry that state with
  `InitialContextInjection::BeforeLastUserMessage`, so replacement context
  and its stored baseline cannot come from different snapshots.
  - Remove obsolete state-recapture helpers and ambiguous TurnContext-only
  WorldState builders.
  - Add an integration test that moves an environment from starting to
  ready during a paused turn, triggers compaction, and verifies the next
  request receives the readiness update exactly once.
  
  ## Test plan
  
  - `just test -p codex-core
  deferred_executor_compaction_preserves_then_updates_environment_once`
  - `just test -p codex-core process_compacted_history`
  - `just test -p codex-core mid_turn_continuation_compaction`
  - `just test -p codex-core build_initial_context`
  - `just test -p codex-core
  ignores_session_prefix_messages_when_truncating`

  sayan-oai · 2026-06-23 10:33:19 -07:00
  d1d11cac05
• [codex] Use input items for Responses Lite tools (#27946)

  When using Responses Lite, we should all use `additional_tools` and a
  developer item instead of the top level tools array & instructions
  field. This keeps things 1-to-1.
  
  Forced namespacing for _all_ tools will land in a following PR after
  some coordination & fixes in Responses API (around collisions & return
  items).
  
  The goal is to eventually expand the scope of this to _all_ requests
  from codex, but that will require larger coordination across providers &
  slower rollout.

  rka-oai · 2026-06-22 23:56:16 -07:00
  33cc928d33
• core: rename metadata -> internal_chat_message_metadata_passthrough (#28968)

  ## Description
  This PR cuts Codex over from generic `ResponseItem.metadata` (introduced
  here: https://github.com/openai/codex/pull/28355) to
  `ResponseItem.internal_chat_message_metadata_passthrough`, which is the
  blessed path and has strongly-typed keys.
  
  For now we have to drop this MAv2 usage of `metadata`:
  https://github.com/openai/codex/pull/28561 until we figure out where
  that should live.

  Owen Lin · 2026-06-22 11:11:25 -07:00
  5b95745eae
• core: add context window lineage IDs (#29256)

  ## Why
  
  The rendered `<token_budget>` fragment identifies the thread and current
  context window, but it does not expose enough lineage to identify the
  first window in the thread or the immediately preceding window. Those
  IDs also need to remain stable across compaction, resume, and rollback.
  
  ## What changed
  
  - Track first, previous, and current UUIDv7 context-window IDs in
  auto-compaction state.
  - Render `thread_id`, `first_window_id`, `previous_window_id`, and the
  current window ID in the full `<token_budget>` fragment.
  - Persist the first and previous window IDs in compacted rollout
  checkpoints and restore them during rollout reconstruction.
  - Preserve compatibility with older compacted records that do not
  contain the new optional fields.
  - Update focused state, rendering, reconstruction, rollback, and
  serialization coverage.
  
  ## Validation
  
  - `just test -p codex-core token_budget`
  - `just test -p codex-protocol compacted_item::tests`
  - `just test -p codex-core tracks_prefill_and_window_boundaries`
  - `just test -p codex-core
  reconstruct_history_uses_replacement_history_verbatim`
  - `just test -p codex-core
  thread_rollback_restores_cleared_reference_context_item_after_compaction`

  pakrym-oai · 2026-06-20 13:15:49 -07:00
  d1209bddfc
• core: assign item IDs to compacted replacement history (#29012)

  ## Why
  
  Remote v2 compaction can return replacement-history items without IDs.
  Because replacement history is installed directly, those items bypass
  normal history preparation and remain ID-less in later Responses
  requests even when the `item_ids` feature is enabled.
  
  ## What changed
  
  - Pass the active `TurnContext` into `replace_compacted_history`.
  - When `item_ids` is enabled, assign missing IDs before installing and
  persisting replacement history.
  - Rebuild `CompactedItem` from the prepared history so live and
  persisted replacement histories match.
  - Add integration coverage requiring IDs on every ID-capable input item
  in the initial, remote v2 compaction, and post-compaction requests.
  
  ## Test plan
  
  - `just test -p codex-core response_item_ids`
  - `just test -p codex-core websocket_v2_test_codex_shell_chain`
  - `just test -p codex-core remote_compaction_parity_pre_turn_auto`
  - `just test -p codex-app-server
  thread_inject_items_adds_raw_response_items_to_thread_history`

  pakrym-oai · 2026-06-18 18:49:57 -07:00
  70a6aa2634
• core: add UUIDv7 context window IDs (#28953)

  ## Why
  
  The token-budget context currently identifies a context window by its
  thread-local sequence number. A UUIDv7 gives the model a stable opaque
  identity that remains fixed for a window and rotates when compaction or
  `new_context` starts the next one.
  
  ## What changed
  
  - Preserve the existing monotonic value as `window_number` and add a
  UUIDv7 `window_id` to `CompactedItem`.
  - Generate and rotate the UUID with auto-compaction window state,
  persist it alongside the number, and reconstruct it on resume and
  rollback.
  - Accept legacy compacted rollout records where the numeric `window_id`
  represented the window number.
  - Use the UUID only in token-budget context; existing request headers
  and metadata continue using `thread_id:window_number`.
  
  ## Testing
  
  - `just test -p codex-protocol compacted_item::tests`
  - `just test -p codex-core token_budget`

  pakrym-oai · 2026-06-18 17:00:49 -07:00
  5c12034e42
• [codex] Add optional IDs to response items (#28812)

  ## Why
  
  `ResponseItem` variants do not have a consistent internal ID shape: some
  variants carry required IDs, some carry optional IDs, and some cannot
  represent an ID at all. The existing fields also use inconsistent serde,
  TypeScript, and JSON-schema annotations. A single enum-level access path
  is needed before history recording can assign and retain IDs.
  
  This PR establishes that internal model only. It intentionally does not
  generate or serialize IDs; allocation and wire persistence are isolated
  in the stacked follow-up.
  
  ## What changed
  
  - Give every concrete `ResponseItem` variant an `Option<String>` ID
  field.
  - Apply the same internal-only annotations to every ID field:
  `#[serde(default, skip_serializing)]`, `#[ts(skip)]`, and
  `#[schemars(skip)]`.
  - Add `ResponseItem::id()` and `ResponseItem::set_id()` as the shared
  accessors.
  - Preserve IDs when history items are rewritten for truncation.
  - Adapt consumers that previously assumed reasoning and image-generation
  IDs were required.
  - Regenerate app-server schemas so the hidden fields are represented
  consistently.
  
  The serde catch-all `ResponseItem::Other` remains ID-less because it
  must remain a unit variant.
  
  ## Test plan
  
  - `cargo check --tests -p codex-core -p codex-api -p codex-rollout-trace
  -p codex-image-generation-extension`
  - `just test -p codex-protocol`
  - `just test -p codex-app-server-protocol`
  - `just test -p codex-api -p codex-rollout-trace -p
  codex-image-generation-extension`
  - `just test -p codex-core event_mapping`

  pakrym-oai · 2026-06-17 18:27:43 -07:00
  dbd2857f4b
• core: remove redundant TurnContext and Prompt fields (#28638)

  ## Why
  
  `TurnContext` had accumulated dead fields and cached projections of
  values already owned by its per-turn `Config` or `ModelInfo`. Keeping
  both copies made ownership unclear and allowed artificial split-brain
  states, such as a compatibility hash differing from the model metadata
  it came from.
  
  `Prompt` similarly carried a write-only personality after personality
  selection had already been materialized into its base instructions.
  
  This makes the canonical owner explicit: configuration-backed values
  come from `config`, model-derived values come from `model_info`, and
  prompts contain only data consumed by request construction.
  
  ## What changed
  
  - Remove the unused `ghost_snapshot`, `codex_self_exe`, and
  `thread_source` fields.
  - Remove duplicate `comp_hash`, `truncation_policy`, `features`,
  `shell_environment_policy`, `codex_linux_sandbox_exe`, `compact_prompt`,
  and `tool_mode` fields.
  - Read those values directly from `TurnContext::config` or
  `TurnContext::model_info` at their consumers.
  - Remove the write-only `Prompt::personality` field and its constructor
  assignments.
  - Preserve review-turn inheritance of the parent turn's shell policy,
  Linux sandbox executable, and compact prompt through the review config.
  
  ## Testing
  
  - `cargo check -p codex-core --tests`

  pakrym-oai · 2026-06-16 22:17:24 -07:00
  172b2218a5
• feat(core): add metadata field to ResponseItem (#28355)

  ## Description
  
  This PR adds an optional `metadata` field to `ResponseItem` for
  Responses API calls. Only mechanical plumbing, no actual values
  populated and sent yet. Turns out just adding a new field to
  `ResponseItem` has quite a large blast radius already.
  
  This change is backwards compatible because `metadata` is optional and
  omitted when absent, so existing response items and rollout history
  without it still deserialize and requests that do not set it keep the
  same wire shape. For provider compatibility, we strip out `metadata`
  before non-OpenAI Responses requests so Azure and AWS Bedrock never see
  this field.
  
  My followup PR here will actually make use of it to start storing and
  passing along `turn_id`: https://github.com/openai/codex/pull/28360
  
  ## What changed
  
  - Added `ResponseItemMetadata` with optional `turn_id`, plus optional
  `metadata` on Responses API item variants and inter-agent communication.
  - Preserved item metadata through response-item rewrites such as
  truncation, missing tool-output synthesis, compaction history
  rebuilding, visible-history conversion, rollout/resume, and generated
  app-server schemas/types.
  - Strip item metadata from non-OpenAI Responses requests while
  preserving it for OpenAI-shaped requests.
  - Updated the mechanical fixture/test construction churn required by the
  new optional field.

  Owen Lin · 2026-06-15 15:05:28 -07:00
  040dafa32d
• [codex] Send turn state through compact requests (#28002)

  ## Context
  
  Inline compaction is part of the active logical turn. Compact requests
  and the sampling requests around them should use the same turn state,
  including when compaction is the first request to establish it.
  
  ## Change
  
  Pass the turn-scoped `OnceLock` directly to inline v1 compaction so
  `/responses/compact` includes an established value in the existing HTTP
  header. Capture `x-codex-turn-state` from the compact response into that
  same lock, allowing pre-turn compact to establish the value that
  subsequent sampling reuses.
  
  V2 compact already uses the normal Responses HTTP/WebSocket path and
  continues to share the same `OnceLock` without separate plumbing. The
  first returned value wins for the logical turn.
  
  ## Test plan
  
  Integration coverage verifies that:
  
  - pre-turn v1 compact can establish state for the first sampling request
  - inline v1 compact receives established state over HTTP
  - inline v2 compact reuses established state over HTTP
  - inline v2 compact reuses established state over WebSocket
  
  CI validates the full change.

  Ahmed Ibrahim · 2026-06-13 01:27:58 -07:00
  f297b9f07d
• core: Consolidate Responses API Codex metadata (#27122)

  ## What
  Introduce a `CodexResponsesMetadata` struct that defines all the core
  metadata we send to Responses API. Example fields are `thread_id`,
  `turn_id`, `window_id`, etc.
  
  Going forward, `client_metadata["x-codex-turn-metadata"]` will be the
  canonical way Codex sends metadata to Responses API across both HTTP and
  websocket transports.
  
  For now, we continue to emit the existing top-level HTTP headers and
  top-level `client_metadata` fields from the same
  `CodexResponsesMetadata` struct for compatibility reasons.
  
  Also, app-server clients who specify additional
  `responsesapi_client_metadata` via `turn/start` and `turn/steer` will
  have those fields merged into
  `client_metadata["x-codex-turn-metadata"]`, but cannot override the
  reserved fields that core uses (i.e. the fields in
  `CodexResponsesMetadata`).
  
  ## Why
  
  Responses API request instrumentation is the source of truth for
  downstream Codex analytics that join requests by Codex IDs such as
  session, thread, turn, and context window. Before this change, those
  values were assembled through several request-specific paths: HTTP
  request bodies, websocket handshake headers, websocket `response.create`
  payloads, compaction requests, and the rich `x-codex-turn-metadata`
  envelope all had their own wiring.
  
  That made metadata propagation easy to drift across API-key/direct
  Responses API requests, ChatGPT-auth/proxied requests, websocket
  requests, and compaction requests. It also made additions like
  `window_id` error-prone because a field could be added to one transport
  projection but missed in another.
  
  ## What changed
  
  - Added `CodexResponsesMetadata` as the core-owned snapshot for Codex
  metadata sent to ResponsesAPI.
  - Render `client_metadata["x-codex-turn-metadata"]`, flat
  `client_metadata` projections, and direct compatibility headers from
  that same snapshot.
  - Include the known Codex-owned fields in the turn metadata blob,
  including installation/session/thread/turn/window IDs, request kind,
  lineage, sandbox/workspace metadata, timing, and compaction details.
  - Treat app-server `responsesapi_client_metadata` as enrichment for the
  Codex turn metadata blob while preventing those extras from overriding
  Codex-owned fields.
  - Use the same metadata path for normal turns, websocket prewarm, local
  compaction, remote v1 compaction, and remote v2 compaction.
  - Keep websocket connection-only preconnect metadata separate so
  handshakes carry compatibility identity headers without inventing a fake
  turn metadata blob.
  
  ## Verification
  
  - `cargo check -p codex-core`
  - `just fix -p codex-core`

  Owen Lin · 2026-06-11 13:42:09 -07:00
  14df0e8833
• [codex-analytics] Emit structured compaction codex errors (#27082)

  ## Summary
  - replace raw compaction `error` analytics with `codex_error_kind` and
  `codex_error_http_status_code`
  - derive compaction error telemetry from `CodexErr` using the same
  `CodexErrKind` mapping and HTTP status helper used by turn events
  - remove the pre-compact hook stop reason from the internal compaction
  outcome now that it is no longer emitted as raw analytics text
  
  ## Why
  Compaction `error` was a raw `CodexErr::to_string()` value, which can
  carry free-form provider or user-derived text. Structured Codex error
  fields preserve useful low-cardinality telemetry without sending the raw
  string.
  
  ## Validation
  - `just fmt`
  - `just test -p codex-analytics`
  - `just test -p codex-core
  compact::tests::build_token_limited_compacted_history_appends_summary_message`
  
  Attempted `just test -p codex-core`; the changed crate compiled, but the
  full target failed in unrelated environment-dependent tests such as
  missing helper binaries and shell snapshot timeouts.

  rhan-oai · 2026-06-11 06:07:06 +00:00
  8d9f33c87c
• [codex] Add token budget context feature (#27438)

  ## Why
  
  The model should be able to see bounded context-window budget metadata
  when the `token_budget` feature is enabled. The full-window message is
  only injected with full context, while normal turns get a smaller
  follow-up only when reported usage first crosses a budget threshold.
  
  ## What changed
  
  - Added the `TokenBudget` feature flag.
  - Added `<token_budget>` developer fragments for full context-window
  metadata and current-window remaining tokens.
  - Inserted the threshold message during normal turn handling by
  comparing token usage before and after sampling, avoiding persistent
  threshold bookkeeping.
  - Added core integration coverage for full-context-only metadata and
  25/50/75 percent threshold messages.
  
  ## Verification
  
  - `just test -p codex-core token_budget`
  - `git diff --check`

  pakrym-oai · 2026-06-10 20:07:06 -07:00
  658af936fd
• [codex] Store compact window id in rollout (#27264)

  ## Why
  
  Compaction window identity is part of session history, not model-client
  transport state. Persisting it with the compacted rollout item lets
  resumed threads continue from the reconstructed window without keeping
  mutable window state on `ModelClient`.
  
  ## What changed
  
  - Added `window_id` to `CompactedItem` and stamp it when
  `replace_compacted_history` installs compacted history.
  - Moved auto-compact window id ownership into `AutoCompactWindow` /
  `SessionState`; `ModelClient` now receives the request window id from
  callers instead of storing it.
  - Returned `window_id` from rollout reconstruction for resume.
  Reconstruction uses the newest surviving compacted item's stored
  `window_id` when present, and falls back to the legacy compacted-item
  count when it is absent.
  - Kept fork startup at the fresh default window id and updated direct
  model-client tests to pass explicit test window ids.
  
  ## Validation
  
  - `cargo check -p codex-core --tests`

  pakrym-oai · 2026-06-10 08:47:16 -07:00
  30ddb3325e
• [codex] Remove remote compaction failure log (#27106)

  ## Why
  
  `log_remote_compact_failure` was the only consumer of the
  compact-request logging payload and most of the token-usage breakdown
  fields. Once that failure log is removed, keeping the surrounding
  carrier types leaves dead plumbing in the compaction path and context
  manager.
  
  ## What changed
  
  - Remove `log_remote_compact_failure`, `CompactRequestLogData`, and the
  v2 wrapper that only fed that log.
  - Let both remote compaction implementations return the original
  compaction error directly.
  - Replace `TotalTokenUsageBreakdown` with a narrow helper that returns
  only the remaining value needed by compaction analytics.
  - Keep `estimate_response_item_model_visible_bytes` private to the
  context manager implementation.
  
  ## Validation
  
  - `cargo check -p codex-core`

  pakrym-oai · 2026-06-08 19:23:35 -07:00
  08cb633c06
• [codex-analytics] report compaction analytics details (#26680)

  ## Why
  
  Compaction analytics adds retained image count and compaction summary
  output tokens for v1.5 specifically.
  
  ## What changed
  
  - Add nullable `retained_image_count` and `compaction_summary_tokens`
  fields to `codex_compaction_event`.
  - Populate them only for `responses_compaction_v2`: retained images come
  from the retained v2 compacted history, and summary tokens come from
  `response.completed.token_usage.output_tokens`.
  - Leave local and legacy remote compaction events as `null` for these
  detail fields.
  
  ## Verification
  
  - `just fmt`
  - `just fix -p codex-core`
  - `just test -p codex-core
  build_v2_compacted_history_counts_retained_input_images`
  - `git diff --check`

  rhan-oai · 2026-06-08 10:52:31 -07:00
  f1c18df9ae
• Encrypt multi-agent v2 message payloads (#26210)

  ## Why
  
  Multi-agent v2 currently routes agent instructions through normal tool
  arguments and inter-agent context. That means the parent model can emit
  plaintext task text, Codex can persist it in history/rollouts, and the
  recipient can receive it as ordinary assistant-message JSON.
  
  This changes the v2 path so agent instructions stay encrypted between
  model calls: Responses encrypts the `message` argument returned by the
  model, Codex forwards only that ciphertext, and Responses decrypts it
  internally for the recipient model.
  
  ## What changed
  
  - Mark the v2 `message` parameter as encrypted for `spawn_agent`,
  `send_message`, and `followup_task`.
  - Treat multi-agent v2 tool `message` values as ciphertext
  unconditionally.
  - Store v2 inter-agent task text in
  `InterAgentCommunication.encrypted_content` with empty plaintext
  `content`.
  - Convert encrypted inter-agent communications into the Responses
  `agent_message` input item before sending the child request.
  - Preserve `agent_message` items across history, rollout, compaction,
  telemetry, and app-server schema paths.
  - Leave multi-agent v1 unchanged.
  
  ## Message shape
  
  The model still calls the v2 tools with a `message` argument, but that
  value is now ciphertext:
  
  ```json
  {
    "name": "spawn_agent",
    "arguments": {
      "task_name": "worker",
      "message": "<ciphertext>"
    }
  }
  ```
  
  Codex stores the task as encrypted inter-agent communication:
  
  ```json
  {
    "author": "/root",
    "recipient": "/root/worker",
    "content": "",
    "encrypted_content": "<ciphertext>",
    "trigger_turn": true
  }
  ```
  
  When Codex builds the recipient request, it forwards the ciphertext
  using the new Responses input item:
  
  ```json
  {
    "type": "agent_message",
    "author": "/root",
    "recipient": "/root/worker",
    "content": [
      {
        "type": "encrypted_content",
        "encrypted_content": "<ciphertext>"
      }
    ]
  }
  ```
  
  Responses decrypts that item internally for the recipient model.
  
  ## Context impact
  
  - Parent context no longer carries plaintext v2 agent task instructions
  from these tool arguments.
  - Codex rollout/history stores ciphertext for v2 agent instructions.
  - Recipient requests receive an `agent_message` item instead of
  assistant commentary JSON for encrypted task delivery.
  - Plaintext completion/status notifications are still plaintext because
  they are Codex-generated status messages, not encrypted model tool
  arguments.
  
  ## Validation
  
  - `just test -p codex-tools`
  - `just test -p codex-protocol`
  - `just test -p codex-rollout`
  - `just test -p codex-rollout-trace`
  - `just test -p codex-otel`
  - `just write-app-server-schema`

  jif · 2026-06-05 10:25:57 +02:00
  5f4d06ef18
• [codex] Support model-defined reasoning efforts (#26444)

  ## Summary
  - accept non-empty model-defined reasoning effort values while
  preserving built-in effort behavior
  - propagate the non-Copy effort type through core, app-server, TUI,
  telemetry, and persistence call sites
  - preserve string wire encoding and expose an open-string schema for
  clients
  - update model selection and shortcut behavior for model-advertised
  effort values
  
  ## Root cause
  `ReasoningEffort` gained a string-backed custom variant, so it could no
  longer implement `Copy` or rely on derived closed-enum serialization.
  Existing consumers still moved effort values from shared references and
  assumed a fixed built-in value set.
  
  ## Validation
  - `just fmt`
  - Local tests and compilation were not run per request; relying on CI.

  Ahmed Ibrahim · 2026-06-04 13:36:24 -07:00
  8ac304c299
• [codex-analytics] report compaction request token counts (#25946)

  ## Why
  
  Compaction analytics need token counts that better represent the request
  being compacted. The existing session snapshot can diverge from the
  actual remote compaction request after output rewriting, and remote v2
  can use server-side Responses usage when available.
  
  ## What changed
  
  - Add an optional `active_context_tokens_before` override to
  `CompactionAnalyticsAttempt::track(...)` for remote compaction when it
  has a better before-token value than the begin-time session snapshot.
  The local `/compact` path passes no override.
  - For remote v1 `responses_compact`, subtract the estimated token delta
  from pre-compaction output rewriting from the session snapshot, capped
  by locally-added tokens since the last successful API response.
  - For remote v2 `responses_compaction_v2`, use the same bounded
  output-rewrite fallback as remote v1, then overwrite
  `active_context_tokens_before` with server `token_usage.input_tokens`
  from the `response.completed` event when present.
  - Keep the existing v2 compaction-output validation while carrying the
  completed response token usage through `collect_compaction_output`.
  
  ## Verification
  
  - `just fmt`
  - `just test -p codex-core
  collect_compaction_output_accepts_additional_output_items`
  - `git diff --check`

  rhan-oai · 2026-06-04 03:00:44 +00:00
  c143a86de8
• Rewrite oversized tool outputs during remote compaction (#26251)

  ## Why
  
  When trying to fit history under compaction limit rewrite output items
  instead of removing them entirely. Otherwise we're breaking
  incrementality in relation to the previous response.

  pakrym-oai · 2026-06-03 15:25:50 -07:00
  4231472c03
• [codex-analytics] Track CodexErr details in turn analytics (#25707)

  ## Summary
  - add analytics-only `CodexErr` telemetry to `codex_turn_event` while
  leaving existing `turn_error` unchanged
  - record terminal `CodexErr` facts from core immediately before the
  existing turn error event is sent
  - emit source-truth `codex_error_*` fields for downstream analytics,
  including the raw `CodexErr::InvalidRequest(String)` message as
  `codex_error_subreason`
  
  ## Validation
  - `just test -p codex-analytics`
  - attempted `just test -p codex-core`, but the local run timed out
  across unrelated integration suites in this environment and is not being
  used as validation

  rhan-oai · 2026-06-02 11:40:35 -07:00
  8e4b92d294
• [codex] add compaction metadata to turn headers (#24368)

  ## Summary
  - Add `request_kind` values for foreground turn, startup prewarm,
  compaction, and detached memory model requests.
  - Attach compaction dispatch metadata to local Responses, legacy
  `/v1/responses/compact`, and remote v2 compact requests.
  - Add the existing logical context-window identifier as `window_id` on
  turn-owned model request metadata.
  - Keep identity fields optional for detached memory requests, while
  still emitting `request_kind="memory"` in non-git/no-sandbox workspaces.
  
  ## Root Cause
  `x-codex-turn-metadata` has more than one producer. Foreground turns and
  compaction requests own a real turn and should carry that turn identity.
  Detached memory stage-one requests do not own a foreground turn, so
  absent identity fields are valid rather than missing data. Startup
  websocket prewarm is also a model request, but it has `generate=false`
  and must not be counted as a foreground turn.
  
  `thread_source` or session source identifies where a thread came from
  (for example review, guardian, or another subagent). `request_kind`
  identifies what the current outbound model request is doing (`turn`,
  `prewarm`, `compaction`, or `memory`). A review or guardian thread can
  issue either a normal turn request or a compaction request, so source
  cannot replace request kind.
  
  ## Behavior / Impact
  - Ordinary foreground requests send `request_kind="turn"`, their real
  identity fields, and `window_id="<thread_id>:<window_generation>"`.
  - Startup websocket warmup requests send `request_kind="prewarm"` so
  they are not counted as foreground turns.
  - Compaction requests send `request_kind="compaction"`, their real
  owning turn identity, the existing `window_id`, and
  `compaction.{trigger,reason,implementation,phase,strategy}`.
  - Detached memory stage-one requests send `request_kind="memory"`
  without `session_id`, `thread_id`, `turn_id`, or `window_id`; when no
  workspace metadata exists, the kind-only header is still emitted.
  - `session_id`, `thread_id`, `turn_id`, and `window_id` remain optional
  in the header schema because detached memory requests do not own a
  foreground turn or context window.
  - `window_id` is not a new ID system: it is copied from the already-sent
  `x-codex-window-id` / WS client metadata value at model-request dispatch
  time.
  - Existing `x-codex-window-id` HTTP/WS emission, value format,
  generation advancement, resume behavior, and fork reset behavior are
  unchanged.
  - `request_kind`, `window_id`, and upstream turn-owned identity fields
  remain schema-owned; input `responsesapi_client_metadata` cannot replace
  their canonical values.
  - No table, DAG, export, app-server API, or MCP `_meta` schema changes
  are included.
  
  A compaction attempt stopped by a pre-compact hook issues no model
  request and therefore has no request header; its outcome remains in
  analytics events. Status, error, duration, and token deltas also remain
  analytics fields rather than request-header fields.
  
  Future detached-memory attribution using a real initiating turn ID as
  `trigger_turn_id` is intentionally not part of this PR.
  
  ## Sync With Main
  - Final pushed head `716342e79` is rebased onto `origin/main@0d37db4b2`.
  - The metadata conflict came from upstream `#24160`, which added
  `forked_from_thread_id` on the same `turn_metadata` surface. Resolution
  preserves that field and its protection from client metadata override
  alongside this PR's request-kind, compaction, and window-id fields.
  - While resolving the overlapping commits, I removed an accidental
  recursive model-request overlay and a duplicate detached-memory header
  builder before completing the rebase.
  
  ## Latency / User Experience Boundary
  - Foreground turns perform no new filesystem, git, or network work. New
  fields are inserted into metadata already serialized for outgoing
  requests.
  - Compaction issues the same model/HTTP requests with the same prompt,
  model, service tier, and sampling settings; only metadata bytes change.
  - Startup prewarm already sent metadata; it is now correctly classified
  as `prewarm`.
  - Non-git detached memory now sends a small kind-only metadata header
  rather than no header.
  - This client diff adds no user-visible latency mechanism beyond
  negligible serialization and header bytes on already-existing requests.
  
  ## Validation
  On conflict-resolved head `1d35c2cfb` based on `origin/main@487521733`:
  - `just fmt` (passed)
  - `just fix -p codex-core` (passed)
  - `git diff --check origin/main...HEAD` (passed)
  - `just test -p codex-core -E 'test(turn_metadata) |
  test(websocket_first_turn_uses_startup_prewarm_and_create) |
  test(responses_stream_includes_turn_metadata_header_for_git_workspace_e2e)
  |
  test(responses_websocket_forwards_turn_metadata_on_initial_and_incremental_create)
  | test(remote_compact_v2_retries_failures_with_stream_retry_budget) |
  test(window_id_advances_after_compact_persists_on_resume_and_resets_on_fork)'`
  (`23 passed`; `bench-smoke` passed)
  - `just test -p codex-app-server -E
  'test(turn_start_forwards_client_metadata_to_responses_request_v2) |
  test(turn_start_forwards_client_metadata_to_responses_websocket_request_body_v2)
  | test(auto_compaction_remote_emits_started_and_completed_items)'` (`3
  passed`; `bench-smoke` passed)
  - `just test -p codex-memories-write` (`29 passed`; `bench-smoke`
  passed)

  ningyi-oai · 2026-05-27 11:09:33 -07:00
  bee78806a9
• Add trace_id to TurnStartedEvent (#23980)

  ## Why
  [Recent PR](https://github.com/openai/codex/pull/22709) removed
  `trace_id` from `TurnContextItem`.
  
  ## What changed
  - Add to `TurnStartedEvent` so rollout consumers can correlate turns
  with telemetry traces.
  - Note that the branch name is out of date because I originally re-added
  to `TurnContextItem`, but we decided to move it to `TurnStartedEvent`.
  
  ## Verification
  - `cargo test -p codex-protocol`
  - `cargo test -p codex-core --lib
  regular_turn_emits_turn_started_without_waiting_for_startup_prewarm`
  - `cargo test -p codex-core --test all
  emits_warning_when_resumed_model_differs`
  - `cargo test -p codex-rollout`
  - `cargo test -p codex-state`

  mchen-oai · 2026-05-22 13:10:56 -07:00
  3c83e57bfa
• feat: retain remote compaction truncation parity in v2 (#23728)

  ## Why
  
  Remote compaction now has two implementations: the existing
  server-rebuilt v1 path and the newer client-rebuilt v2 path behind
  `remote_compaction_v2`. The v1 path bounds retained
  user/developer/system history before installing the compaction item,
  while v2 was previously carrying the full retained history forward. That
  made the two paths diverge for large pre-compaction transcripts even
  though they are meant to preserve the same compaction contract.
  
  This aligns v2 with the retained-history budget expected from v1 so
  switching the feature flag does not materially change which
  pre-compaction messages survive into the rebuilt history.
  
  ## What changed
  
  - Apply a retained-message character budget while rebuilding v2
  compacted history in `core/src/compact_remote_v2.rs`.
  - Keep newest retained messages first, truncate the boundary message
  with the shared `truncate_text(...)` helper, and drop older retained
  messages once the budget is exhausted.
  - Preserve non-text retained message content such as images while
  truncating text content.
  - Use the current `64_000` token retained-message default translated to
  the existing `4x` character budget.
  
  ## Testing
  
  - `cargo test -p codex-core compact_remote_v2::tests::`
  - Added focused coverage for newest-first retention and truncating
  multipart retained messages without dropping images.

  jif-oai · 2026-05-21 15:07:03 +02:00
  94442b7f95
• Remove explicit connector tool undeferral (#23390)

  ## Summary
  - remove the explicit-connector carveout that kept mentioned app tools
  directly exposed instead of deferred
  - keep the surviving explicit-mention reconstruction only for analytics,
  preserving `codex_app_mentioned` and `codex_app_used.invoke_type`
  - trim the now-unused prompt/tool-exposure plumbing and refresh coverage
  around always-defer behavior
  
  ## Verification
  - `just fmt`
  - `cargo test -p codex-analytics`
  - `cargo test -p codex-core` *(one transient timeout in
  `shell_snapshot::tests::macos_zsh_snapshot_includes_sections`; isolated
  rerun passed)*
  - `cargo test -p codex-core --lib
  shell_snapshot::tests::macos_zsh_snapshot_includes_sections`
  - `cargo test -p codex-core --test all
  explicit_app_mentions_respect_always_defer`
  - `cargo test -p codex-core --lib
  mcp_tool_exposure::tests::always_defer_feature_defers_apps_too`
  - `just fix -p codex-analytics`
  - `just fix -p codex-core`

  sayan-oai · 2026-05-18 21:33:46 -07:00
  1dd9bf9a74
• [codex] Use compaction_trigger item for remote compaction v2 (#22809)

  ## Why
  
  Remote compaction v2 was still using `context_compaction` as both the
  request trigger and the compacted output shape. The Responses API now
  has the landed contract for this flow: Codex sends a dedicated `{
  "type": "compaction_trigger" }` input item, and the backend returns the
  standard `compaction` output item with encrypted content.
  
  This aligns the v2 path with that wire contract while preserving the
  existing local compacted-history post-processing behavior.
  
  ## What changed
  
  - Add `ResponseItem::CompactionTrigger` and regenerate the app-server
  protocol schema fixtures.
  - Send `compaction_trigger` from `remote_compaction_v2` instead of a
  payload-less `context_compaction`.
  - Collect exactly one backend `compaction` output item, then reuse the
  existing compacted-history rebuilding path.
  - Treat the trigger item as a transient request marker rather than model
  output or persisted rollout/memory content.
  
  ## Verification
  
  - `cargo test -p codex-protocol compaction_trigger`
  - `cargo test -p codex-core remote_compact_v2`
  - `cargo test -p codex-core compact_remote_v2`
  - `cargo test -p codex-core
  responses_websocket_sends_response_processed_after_remote_compaction_v2`
  - `just write-app-server-schema`
  - `cargo test -p codex-app-server-protocol schema_fixtures`

  jif-oai · 2026-05-15 11:40:35 +02:00
  0322ac3df8
• [codex] Filter legacy warning messages during compaction (#22243)

  ## Why
  
  Older sessions can contain model-warning records persisted as `user`
  messages, including the unified exec process-limit warning, the
  `apply_patch`-via-`exec_command` warning, and the model-mismatch
  high-risk cyber fallback warning. Those warnings are no longer produced
  as conversation history items, but when old sessions compact they should
  still be recognized as injected context rather than preserved as real
  user turns.
  
  ## What changed
  
  - Removed `record_model_warning` and the production paths that emitted
  these warning messages into conversation history.
  - Added `LegacyUnifiedExecProcessLimitWarning`,
  `LegacyApplyPatchExecCommandWarning`, and `LegacyModelMismatchWarning`
  contextual fragments that are used only for matching old persisted
  messages.
  - Registered the legacy fragments with contextual user message detection
  so compaction filters them through the existing fragment path.
  - Added focused compaction coverage for old warning messages being
  dropped during compacted-history processing.
  
  ## Testing
  
  - `cargo test -p codex-core warning`
  - `just fix -p codex-core`

  pakrym-oai · 2026-05-11 19:51:51 -07:00
  79c65f816c
• Omit service_tier from remote /responses/compact requests under API auth (#21676)

  ## Summary
  
  API-key-auth remote compaction requests should not inherit
  `service_tier` from normal `/responses` turns. This path needs to match
  API auth expectations, while ChatGPT-auth remote compaction should keep
  reusing the shared request fields that still apply there.
  
  This change keeps the decision inline in
  `codex-rs/core/src/compact_remote.rs` only. Under API key auth, the
  classic remote `/responses/compact` path now omits `service_tier`; under
  ChatGPT auth, it keeps reusing the configured tier.
  `codex-rs/core/src/compact_remote_v2.rs` is unchanged. The remote
  compaction parity coverage and snapshots were updated to assert the
  API-key omission and preserve the ChatGPT-auth behavior.
  
  ## Testing
  
  - Updated remote compaction parity coverage in
  `codex-rs/core/tests/suite/compact_remote.rs` and the corresponding
  snapshots.

  Ahmed Ibrahim · 2026-05-08 11:15:14 +03:00
  71d80f9a14
• Add compact lifecycle hooks (started by vincentkoc - external contrib) (#19905)

  Based on work from Vincent K -
  https://github.com/openai/codex/pull/19060
  
  <img width="1836" height="642" alt="CleanShot 2026-04-29 at 20 47 40@2x"
  src="https://github.com/user-attachments/assets/b647bb89-65fe-40c8-80b0-7a6b7c984634"
  />
  
  ## Why
  
  Compaction rewrites the conversation context that future model turns
  receive, but hooks currently have no deterministic lifecycle point
  around that rewrite. This adds compact lifecycle hooks so users can
  audit manual and automatic compaction, surface hook messages in the UI,
  and run post-compaction follow-up without overloading tool or prompt
  hooks.
  
  ## What Changed
  
  - Added `PreCompact` and `PostCompact` hook events across hook config,
  discovery, dispatch, generated schemas, app-server notifications,
  analytics, and TUI hook rendering.
  - Added trigger matching for compact hooks with the documented `manual`
  and `auto` matcher values.
  - Wired `PreCompact` before both local and remote compaction, and
  `PostCompact` after successful local or remote compaction.
  - Kept compact hook command input to lifecycle metadata: session id,
  Codex turn id, transcript path, cwd, hook event name, model, and
  trigger.
  - Made compact stdout handling consistent with other hooks: plain stdout
  is ignored as debug output, while malformed JSON-looking stdout is
  reported as failed hook output.
  - Added integration coverage for compact hook dispatch, trigger
  matching, post-compact execution, and the audited behavior that
  `decision:"block"` does not block compaction.
  
  ## Out of Scope
  
  - Hook-specific compaction blocking is not implemented;
  `decision:"block"` and exit-code-2 blocking semantics are intentionally
  unsupported for `PreCompact`.
  - Custom compaction instructions are not exposed to compact hooks in
  this PR.
  - Compact summaries, summary character counts, and summary previews are
  not exposed to compact hooks in this PR.
  
  ## Verification
  
  - `cargo test -p codex-hooks`
  - `cargo test -p codex-core
  manual_pre_compact_block_decision_does_not_block_compaction`
  - `cargo test -p codex-app-server hooks_list`
  - `cargo test -p codex-core config_schema_matches_fixture`
  - `cargo test -p codex-tui hooks_browser`
  
  ## Docs
  
  The developer documentation for Codex hooks should be updated alongside
  this feature to document `PreCompact` and `PostCompact`, the
  `manual`/`auto` matcher values, and the compact hook payload fields.
  
  ---------
  
  Co-authored-by: Vincent Koc <vincentkoc@ieee.org>

  Andrei Eternal · 2026-05-06 18:08:31 -07:00
  527d52df03
• 2- Use string service tiers in session protocol (#20971)

  ## Summary
  - break service tier session/op/app-server protocol fields from the
  closed enum to string tier ids
  - send the service tier string directly through model requests, prewarm,
  compaction, memories, and TUI/app-server turn starts
  - regenerate app-server protocol JSON/TypeScript schemas, removing the
  standalone ServiceTier TS enum
  
  ## Verification
  - just fmt
  - cargo check -p codex-core -p codex-app-server -p codex-tui
  - just write-app-server-schema
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-05-06 18:00:21 +03:00
  be1d3cff93
• Propagate cache key and service tiers in compact (#21249)

  ## Why
  
  `/responses/compact` should preserve the request-affinity fields that
  apply to the active auth mode. ChatGPT-auth compact requests need the
  effective `service_tier`, and compact requests for every auth mode need
  the stable `prompt_cache_key`, so compaction does not quietly lose
  routing or cache behavior that normal sampling already has.
  
  This follows the request-parity direction from #20719, but keeps the net
  change focused on the compact payload fields needed here.
  
  ## What changed
  
  - Add `service_tier` and `prompt_cache_key` to the compact endpoint
  input payload.
  - Build the remote compact payload from the existing responses request
  builder output so `Fast` still maps to `priority` when compact sends a
  service tier.
  - Pass the turn service tier into remote compaction, but only include it
  in compact payloads for ChatGPT-backed auth.
  - Keep `prompt_cache_key` on compact payloads for all auth modes.
  - Add request-body diff snapshot coverage in
  `core/tests/suite/compact_remote.rs` for:
  - API-key auth reusing `prompt_cache_key` while omitting `service_tier`
  even when `Fast` is configured.
    - ChatGPT auth reusing both `service_tier` and `prompt_cache_key`.
  - Drive the snapshot coverage through five varied turns: plain text,
  multi-part text, tool-call continuation, image+text input, local-shell
  continuation, and final-turn reasoning output.
  
  ## Verification
  
  - Added insta snapshots for compact request-body parity against the last
  normal `/responses` request after five varied turns.
  - Not run locally per repo guidance; relying on GitHub CI for test
  execution.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-05-06 13:38:43 +03:00
  5d6f23a27b
• feat: add remote compaction v2 Responses client path (#20773)

  ## Why
  
  This adds the `remote_compaction_v2` client path so remote compaction
  can run through the normal Responses stream and install a
  `context_compaction` item that trigger a compaction.
  
  The goal is to migrate some of the compaction logic on the client side
  
  We keeps the v2 transport behind a feature flag while letting follow-up
  requests reuse the compacted context instead of falling back to the
  legacy compaction item shape.
  
  ## What changed
  
  - add `ResponseItem::ContextCompaction` and refresh the generated
  app-server / schema / TypeScript fixtures that expose response items on
  the wire
  - add `core/src/compact_remote_v2.rs` to send compaction through the
  standard streamed Responses client, require exactly one
  `context_compaction` output item, and install that item into compacted
  history
  - route manual compact and auto-compaction through the v2 path when
  `remote_compaction_v2` is enabled, while keeping the existing remote
  compaction path as the fallback
  - preserve the new item type across history retention, follow-up request
  construction, telemetry, rollout persistence, and rollout-trace
  normalization
  - add targeted coverage for the feature flag, `context_compaction`
  serialization, rollout-trace normalization, and remote-compaction
  follow-up behavior
  
  ## Verification
  
  - added protocol tests for `context_compaction`
  serialization/deserialization in `protocol/src/models.rs`
  - added rollout-trace coverage for `context_compaction` normalization in
  `rollout-trace/src/reducer/conversation_tests.rs`
  - added remote compaction integration coverage for v2 follow-up reuse
  and mixed compaction output streams in
  `core/tests/suite/compact_remote.rs`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-05-04 14:15:01 +02:00
  d927f61208
• Remove ghost snapshots (#19481)

  ## Summary
  - Remove `ghost_snapshot` / `GhostCommit` from the Responses API surface
  and generated SDK/schema artifacts.
  - Keep legacy config loading compatible, but make undo a no-op that
  reports the feature is unavailable.
  - Clean up core history, compaction, telemetry, rollout, and tests to
  stop carrying ghost snapshot items.
  
  ## Testing
  - Unit tests passed for `codex-protocol`, `codex-core` targeted undo and
  compaction flows, `codex-rollout`, and `codex-app-server-protocol`.
  - Regenerated config and app-server schemas plus Python SDK artifacts
  and verified they match the checked-in outputs.

  pakrym-oai · 2026-04-27 18:48:57 -07:00
  4e05f3053c
• [rollout_trace] Add debug trace reduction command (#18880)

  ## Summary
  
  Adds the debug CLI entry point for reducing recorded rollout traces.
  This gives developers a direct way to inspect whether the emitted trace
  stream reduces into the expected conversation/runtime model.
  
  ## Stack
  
  This is PR 5/5 in the rollout trace stack.
  
  - [#18876](https://github.com/openai/codex/pull/18876): Add rollout
  trace crate
  - [#18877](https://github.com/openai/codex/pull/18877): Record core
  session rollout traces
  - [#18878](https://github.com/openai/codex/pull/18878): Trace tool and
  code-mode boundaries
  - [#18879](https://github.com/openai/codex/pull/18879): Trace sessions
  and multi-agent edges
  - [#18880](https://github.com/openai/codex/pull/18880): Add debug trace
  reduction command
  
  ## Review Notes
  
  This PR is intentionally last: it depends on the trace crate, core
  recorder, runtime/tool events, and session/agent edge data all existing.
  The command should remain a debug/developer tool and avoid adding new
  runtime behavior.
  
  The useful review question is whether the CLI exposes the reducer in the
  smallest practical way for local inspection without turning the debug
  command into a supported user-facing workflow.

  cassirer-openai · 2026-04-24 01:56:48 +00:00
  e3c8720a99
• [rollout_trace] Record core session rollout traces (#18877)

  ## Summary
  
  Wires rollout trace recording into `codex-core` session and turn
  execution. This records the core model request/response, compaction, and
  session lifecycle boundaries needed for replay without yet tracing every
  nested runtime/tool boundary.
  
  ## Stack
  
  This is PR 2/5 in the rollout trace stack.
  
  - [#18876](https://github.com/openai/codex/pull/18876): Add rollout
  trace crate
  - [#18877](https://github.com/openai/codex/pull/18877): Record core
  session rollout traces
  - [#18878](https://github.com/openai/codex/pull/18878): Trace tool and
  code-mode boundaries
  - [#18879](https://github.com/openai/codex/pull/18879): Trace sessions
  and multi-agent edges
  - [#18880](https://github.com/openai/codex/pull/18880): Add debug trace
  reduction command
  
  ## Review Notes
  
  This layer is the first live integration point. The important review
  question is whether trace recording is isolated from normal session
  behavior: trace failures should not become user-visible execution
  failures, and recording should preserve the existing turn/session
  lifecycle semantics.
  
  The PR depends on the reducer/data model from the first stack entry and
  only introduces the core recorder surface that later PRs use for richer
  runtime and relationship events.

  cassirer-openai · 2026-04-22 17:00:48 +00:00
  f67383bcba
• Allow guardian bare allow output (#18797)

  ## Summary
  
  Allow guardian to skip other fields and output only
  `{"outcome":"allow"}` when the command is low risk.
  This change lets guardian reviews use a non-strict text format while
  keeping the JSON schema itself as plain user-visible schema data, so
  transport strictness is carried out-of-band instead of through a schema
  marker key.
  
  ## What changed
  
  - Add an explicit `output_schema_strict` flag to model prompts and pass
  it into `codex-api` text formatting.
  - Set guardian reviewer prompts to non-strict schema validation while
  preserving strict-by-default behavior for normal callers.
  - Update the guardian output contract so definitely-low-risk decisions
  may return only `{"outcome":"allow"}`.
  - Treat bare allow responses as low-risk approvals in the guardian
  parser.
  - Add tests and snapshots covering the non-strict guardian request and
  optional guardian output fields.
  
  ## Verification
  
  - `cargo test -p codex-core guardian::tests::guardian`
  - `cargo test -p codex-core guardian::tests::`
  - `cargo test -p codex-core client_common::tests::`
  - `cargo test -p codex-protocol
  user_input_serialization_includes_final_output_json_schema`
  - `cargo test -p codex-api`
  - `git diff --check`
  
  Note: `cargo test -p codex-core` was also attempted, but this desktop
  environment injects ambient config/proxy state that causes unrelated
  config/session tests expecting pristine defaults to fail.
  
  ---------
  
  Co-authored-by: Dylan Hurd <dylan.hurd@openai.com>
  Co-authored-by: Codex <noreply@openai.com>

  maja-openai · 2026-04-21 15:37:12 -07:00
  ef00014a46
• Move codex module under session (#18249)

  ## Summary
  - rename the core codex module root to session/mod.rs without using
  #[path]
  - move the codex module directory and tests under core/src/session
  - remove session/mod.rs reexports so call sites use explicit child
  module paths
  
  ## Testing
  - cargo test -p codex-core --lib
  - cargo check -p codex-core --tests
  - just fmt
  - just fix -p codex-core
  - git diff --check

  pakrym-oai · 2026-04-17 16:18:53 +00:00
  71e4c6fa17
• [codex-analytics] add compaction analytics event (#17155)

  - event for compaction analytics
  - introduces thread-connection and thread metadata caches for data
  denormalization, expected to be useful for denormalization onto core
  emitted events in general
  - threads analytics event client into core (mirrors approved
  implementation in #16640)
  - denormalizes key thread metadata: thread_source, subagent_source,
  parent_thread_id, as well as app-server client and runtime metadata)
  - compaction strategy defaults to memento, forward compatible with
  expected prefill_compaction strategy
  
  1. Manual standalone compact, local
  `INFO | 2026-04-09 17:35:50 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d0-5cfb-70c0-bef9-165c3bf9b2df', 'turn_id':
  '019d74d0-d7f6-7c81-acc6-aae2030243d6', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'manual', 'reason':
  'user_requested', 'implementation': 'responses', 'phase':
  'standalone_turn', 'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 20170, 'active_context_tokens_after':
  4830, 'started_at': 1775781337, 'completed_at': 1775781350,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 13524} | `
  
  2. Auto pre-turn compact, local
  `INFO | 2026-04-09 17:37:30 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d2-45ef-71d1-9c93-23cc0c13d988', 'turn_id':
  '019d74d2-7b42-7372-9f0e-c0da3f352328', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'auto', 'reason':
  'context_limit', 'implementation': 'responses', 'phase': 'pre_turn',
  'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 20063, 'active_context_tokens_after':
  4822, 'started_at': 1775781444, 'completed_at': 1775781449,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 5497} | `
  
  3. Auto mid-turn compact, local
  `INFO | 2026-04-09 17:38:28 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d3-212f-7a20-8c0a-4816a978675e', 'turn_id':
  '019d74d3-3ee1-7462-89f6-2ffbeefcd5e3', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'auto', 'reason':
  'context_limit', 'implementation': 'responses', 'phase': 'mid_turn',
  'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 20325, 'active_context_tokens_after':
  14641, 'started_at': 1775781500, 'completed_at': 1775781508,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 7507} | `
  
  4. Remote /responses/compact, manual standalone
  `INFO | 2026-04-09 17:40:20 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d4-7a11-78a1-89f7-0535a1149416', 'turn_id':
  '019d74d4-e087-7183-9c20-b1e40b7578c0', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'manual', 'reason':
  'user_requested', 'implementation': 'responses_compact', 'phase':
  'standalone_turn', 'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 23461, 'active_context_tokens_after':
  6171, 'started_at': 1775781601, 'completed_at': 1775781620,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 18971} | `

  rhan-oai · 2026-04-10 13:03:54 -07:00
  5779be314a
• fix(app-server) revert null instructions changes (#17047)
  Dylan Hurd · 2026-04-07 15:18:34 -07:00
  6c36e7d688
• Honor null thread instructions (#16964)

  - Treat explicit null thread instructions as a blank-slate override
  while preserving omitted-field fallback behavior.
  - Preserve null through rollout resume/fork and keep explicit empty
  strings distinct.
  - Add app-server v2 start/fork coverage for the tri-state instruction
  params.

  Ahmed Ibrahim · 2026-04-07 04:10:19 +00:00
  24c598e8a9
• [codex-analytics] add protocol-native turn timestamps (#16638)

  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16638).
  * #16870
  * #16706
  * #16659
  * #16641
  * #16640
  * __->__ #16638

  rhan-oai · 2026-04-06 16:22:59 -07:00
  756c45ec61
• remove temporary ownership re-exports (#16626)

  Stacked on #16508.
  
  This removes the temporary `codex-core` / `codex-login` re-export shims
  from the ownership split and rewrites callsites to import directly from
  `codex-model-provider-info`, `codex-models-manager`, `codex-api`,
  `codex-protocol`, `codex-feedback`, and `codex-response-debug-context`.
  
  No behavior change intended; this is the mechanical import cleanup layer
  split out from the ownership move.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-03 00:33:34 -07:00
  af8a9d2d2b
• core: remove cross-crate re-exports from lib.rs (#16512)

  ## Why
  
  `codex-core` was re-exporting APIs owned by sibling `codex-*` crates,
  which made downstream crates depend on `codex-core` as a proxy module
  instead of the actual owner crate.
  
  Removing those forwards makes crate boundaries explicit and lets leaf
  crates drop unnecessary `codex-core` dependencies. In this PR, this
  reduces the dependency on `codex-core` to `codex-login` in the following
  files:
  
  ```
  codex-rs/backend-client/Cargo.toml
  codex-rs/mcp-server/tests/common/Cargo.toml
  ```
  
  ## What
  
  - Remove `codex-rs/core/src/lib.rs` re-exports for symbols owned by
  `codex-login`, `codex-mcp`, `codex-rollout`, `codex-analytics`,
  `codex-protocol`, `codex-shell-command`, `codex-sandboxing`,
  `codex-tools`, and `codex-utils-path`.
  - Delete the `default_client` forwarding shim in `codex-rs/core`.
  - Update in-crate and downstream callsites to import directly from the
  owning `codex-*` crate.
  - Add direct Cargo dependencies where callsites now target the owner
  crate, and remove `codex-core` from `codex-rs/backend-client`.

  Michael Bolin · 2026-04-01 23:06:24 -07:00
  aa2403e2eb
• [hooks] use a user message > developer message for prompt continuation (#14867)

  ## Summary
  
  Persist Stop-hook continuation prompts as `user` messages instead of
  hidden `developer` messages + some requested integration tests
  
  This is a followup to @pakrym 's comment in
  https://github.com/openai/codex/pull/14532 to make sure stop-block
  continuation prompts match training for turn loops
  
  - Stop continuation now writes `<hook_prompt hook_run_id="...">stop
  hook's user prompt<hook_prompt>`
  - Introduces quick-xml dependency, though we already indirectly depended
  on it anyway via syntect
  - This PR only has about 500 lines of actual logic changes, the rest is
  tests/schema
  
  ## Testing
  
  Example run (with a sessionstart hook and 3 stop hooks) - this shows
  context added by session start, then two stop hooks sending their own
  additional prompts in a new turn. The model responds with a single
  message addressing both. Then when that turn ends, the hooks detect that
  they just ran using `stop_hook_active` and decide not to infinite loop
  
  test files for this (unzip, move codex -> .codex):
  [codex.zip](https://github.com/user-attachments/files/26075806/codex.zip)
  
  ```
  › cats
  
  
  • Running SessionStart hook: lighting the observatory
  
  SessionStart hook (completed)
    warning: Hi, I'm a session start hook for wizard-tower (startup).
    hook context: A wimboltine stonpet is an exotic cuisine from hyperspace
  
  • Cats are tiny zen wizards, my friend: equal parts nap, mystery, and chaos. If you want, we can talk
    cat facts, cat breeds, cat names, or build something cat-themed in this repo.
  
  • Running Stop hook: checking the tower wards
  
  • Running Stop hook: sacking the guards
  
  • Running Stop hook: hiring the guards
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook reviewed the completed reply (177 chars).
  
  Stop hook (blocked)
    warning: Wizard Tower Stop hook continuing conversation
    feedback: cook the stonpet
  
  Stop hook (blocked)
    warning: Wizard Tower Stop hook continuing conversation
    feedback: eat the cooked stonpet
  
  • Stonpet’s cooked, aloha style: flash-seared over a blue quasiflame, glazed with nebula salt, and
    rested until the hyperspace juices settle.
  
    Now we eat with gratitude, my friend. One mindful bite in, and the flavor is pure cosmic surf:
    smoky, bright, and totally out of this dimension.
  
  • Running Stop hook: checking the tower wards
  
  • Running Stop hook: sacking the guards
  
  • Running Stop hook: hiring the guards
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook reviewed the completed reply (285 chars).
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook saw a second pass and stayed calm to avoid a loop.
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook saw a second pass and stayed calm to avoid a loop.
  ```

  Andrei Eternal · 2026-03-19 10:53:08 -07:00
  267499bed8
• Apply argument comment lint across codex-rs (#14652)

  ## Why
  
  Once the repo-local lint exists, `codex-rs` needs to follow the
  checked-in convention and CI needs to keep it from drifting. This commit
  applies the fallback `/*param*/` style consistently across existing
  positional literal call sites without changing those APIs.
  
  The longer-term preference is still to avoid APIs that require comments
  by choosing clearer parameter types and call shapes. This PR is
  intentionally the mechanical follow-through for the places where the
  existing signatures stay in place.
  
  After rebasing onto newer `main`, the rollout also had to cover newly
  introduced `tui_app_server` call sites. That made it clear the first cut
  of the CI job was too expensive for the common path: it was spending
  almost as much time installing `cargo-dylint` and re-testing the lint
  crate as a representative test job spends running product tests. The CI
  update keeps the full workspace enforcement but trims that extra
  overhead from ordinary `codex-rs` PRs.
  
  ## What changed
  
  - keep a dedicated `argument_comment_lint` job in `rust-ci`
  - mechanically annotate remaining opaque positional literals across
  `codex-rs` with exact `/*param*/` comments, including the rebased
  `tui_app_server` call sites that now fall under the lint
  - keep the checked-in style aligned with the lint policy by using
  `/*param*/` and leaving string and char literals uncommented
  - cache `cargo-dylint`, `dylint-link`, and the relevant Cargo
  registry/git metadata in the lint job
  - split changed-path detection so the lint crate's own `cargo test` step
  runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes
  - continue to run the repo wrapper over the `codex-rs` workspace, so
  product-code enforcement is unchanged
  
  Most of the code changes in this commit are intentionally mechanical
  comment rewrites or insertions driven by the lint itself.
  
  ## Verification
  
  - `./tools/argument-comment-lint/run.sh --workspace`
  - `cargo test -p codex-tui-app-server -p codex-tui`
  - parsed `.github/workflows/rust-ci.yml` locally with PyYAML
  
  ---
  
  * -> #14652
  * #14651

  Michael Bolin · 2026-03-16 16:48:15 -07:00
  b77fe8fefe
• Add code_mode_only feature (#14617)

  Summary
  - add the code_mode_only feature flag/config schema and wire its
  dependency on code_mode
  - update code mode tool descriptions to list nested tools with detailed
  headers
  - restrict available tools for prompt and exec descriptions when
  code_mode_only is enabled and test the behavior
  
  Testing
  - Not run (not requested)

  pakrym-oai · 2026-03-13 13:30:19 -07:00
  477a2dd345
• Reapply "Pass more params to compaction" (#14298) (#14521)

  This reverts commit 8af97ce4b08fdedadc6037851b5e20cc653e9536.
  
  Confirmed that this runs locally without the previous issues with tool
  use

  Rasmus Rygaard · 2026-03-12 23:27:21 +00:00
  53d5972226
• feat: search_tool migrate to bring you own tool of Responses API (#14274)

  ## Why
  
  to support a new bring your own search tool in Responses
  API(https://developers.openai.com/api/docs/guides/tools-tool-search#client-executed-tool-search)
  we migrating our bm25 search tool to use official way to execute search
  on client and communicate additional tools to the model.
  
  ## What
  - replace the legacy `search_tool_bm25` flow with client-executed
  `tool_search`
  - add protocol, SSE, history, and normalization support for
  `tool_search_call` and `tool_search_output`
  - return namespaced Codex Apps search results and wire namespaced
  follow-up tool calls back into MCP dispatch

  Anton Panasenko · 2026-03-11 17:51:51 -07:00
  77b0c75267