37 Commits

• core: wrap token budget window context (#29494)

  Token-budget initial context carries thread and context-window lineage
  that the model should treat as one structured context-window block.
  Wrapping it in `<context_window>` makes that boundary explicit while
  preserving the existing window id content.
  
  Before this change, the window identifiers were injected as an untagged
  developer text fragment:
  
  ```text
  Thread id <THREAD_ID>.
  First context window id: <FIRST_WINDOW_ID>
  Current context window id: <WINDOW_ID>
  Previous context window id: <PREVIOUS_WINDOW_ID>
  ```
  
  After this change, the same payload is wrapped as a context-window
  block:
  
  ```text
  <context_window>
  Thread id: <THREAD_ID>
  First context window id: <FIRST_WINDOW_ID>
  Current context window id: <WINDOW_ID>
  Previous context window id: <PREVIOUS_WINDOW_ID>
  </context_window>
  ```
  
  This adds shared `CONTEXT_WINDOW_*_TAG` protocol constants, updates
  `TokenBudgetContext` to render with those markers, treats the new
  wrapper as contextual developer content when mapping history, and
  refreshes the token-budget request-shape assertions and snapshot.
  
  Verification:
  - `just test -p codex-core token_budget`
  - `just test -p codex-core
  recognizes_context_window_as_contextual_developer_content`

  Michael Bolin · 2026-06-22 23:37:49 +00:00
  f1945de3b7
• [codex] simplify token budget context (#29295)

  ## Why
  
  The token-budget feature currently adds remaining-token messages
  whenever usage crosses the 25%, 50%, and 75% thresholds. Those periodic
  inserts create prompt churn without requiring action, while the
  near-compaction reminder and explicit `get_context_remaining` tool
  already cover actionable and on-demand budget information.
  
  The context-window lineage block is also easier to scan as plain labeled
  text than as a `<token_budget>`-wrapped fragment.
  
  ## What changed
  
  - Stop recording automatic remaining-token messages at percentage
  thresholds.
  - Render context-window lineage in `First`, `Current`, `Previous` order
  with colon-separated labels.
  - Omit the `Previous` line for the first context window.
  - Remove `<token_budget>` wrappers from newly rendered lineage,
  near-compaction reminders, and `get_context_remaining` output.
  - Keep recognizing legacy wrapped fragments so existing rollouts remain
  compatible.
  - Remove the post-sampling token snapshot that was only needed by the
  periodic threshold path.
  
  ## Testing
  
  - `just test -p codex-core token_budget` (11 tests passed)

  pakrym-oai · 2026-06-20 21:50:09 -07:00
  aaf737fa59
• [codex] prototype mcp_history thread hint injection (#29259)

  ## Why
  
  Prototype whether the harness can invoke the `mcp_history` MCP while
  constructing full initial context and expose its thread hint to the
  model without requiring a model-issued tool call.
  
  The prototype builds on the context-window lineage added by #29256 and
  is now based directly on `main`.
  
  ## What changed
  
  - Call `mcp_history/thread_hint` with no arguments while building the
  full `<token_budget>` context.
  - Pass the current `threadId` through MCP request metadata, matching the
  normal MCP tool-call path.
  - Serialize only the unstructured `content` result and append it inside
  `<token_budget>` when the call succeeds.
  - Omit the additional context when the MCP call or content serialization
  fails.
  
  ## Prototype limitations
  
  - The direct call bypasses the normal model-initiated MCP approval,
  lifecycle-event, telemetry, and result-sanitization path.
  - The call has no prototype-specific timeout, result-size cap, or
  per-window cache.
  - MCP latency is added to full-context construction, including
  applicable compaction paths.
  
  ## Validation
  
  - `just test -p codex-core token_budget`

  pakrym-oai · 2026-06-20 18:02:02 -07:00
  b6d6be2a84
• core: add context window lineage IDs (#29256)

  ## Why
  
  The rendered `<token_budget>` fragment identifies the thread and current
  context window, but it does not expose enough lineage to identify the
  first window in the thread or the immediately preceding window. Those
  IDs also need to remain stable across compaction, resume, and rollback.
  
  ## What changed
  
  - Track first, previous, and current UUIDv7 context-window IDs in
  auto-compaction state.
  - Render `thread_id`, `first_window_id`, `previous_window_id`, and the
  current window ID in the full `<token_budget>` fragment.
  - Persist the first and previous window IDs in compacted rollout
  checkpoints and restore them during rollout reconstruction.
  - Preserve compatibility with older compacted records that do not
  contain the new optional fields.
  - Update focused state, rendering, reconstruction, rollback, and
  serialization coverage.
  
  ## Validation
  
  - `just test -p codex-core token_budget`
  - `just test -p codex-protocol compacted_item::tests`
  - `just test -p codex-core tracks_prefill_and_window_boundaries`
  - `just test -p codex-core
  reconstruct_history_uses_replacement_history_verbatim`
  - `just test -p codex-core
  thread_rollback_restores_cleared_reference_context_item_after_compaction`

  pakrym-oai · 2026-06-20 13:15:49 -07:00
  d1209bddfc
• core: add UUIDv7 context window IDs (#28953)

  ## Why
  
  The token-budget context currently identifies a context window by its
  thread-local sequence number. A UUIDv7 gives the model a stable opaque
  identity that remains fixed for a window and rotates when compaction or
  `new_context` starts the next one.
  
  ## What changed
  
  - Preserve the existing monotonic value as `window_number` and add a
  UUIDv7 `window_id` to `CompactedItem`.
  - Generate and rotate the UUID with auto-compaction window state,
  persist it alongside the number, and reconstruct it on resume and
  rollback.
  - Accept legacy compacted rollout records where the numeric `window_id`
  represented the window number.
  - Use the UUID only in token-budget context; existing request headers
  and metadata continue using `thread_id:window_number`.
  
  ## Testing
  
  - `just test -p codex-protocol compacted_item::tests`
  - `just test -p codex-core token_budget`

  pakrym-oai · 2026-06-18 17:00:49 -07:00
  5c12034e42
• Support plaintext agent messages (#27830)

  ## Why
  
  Multi-agent v2 `send_message` deliveries already reach the receiving
  model as typed `agent_message` items with encrypted content.
  Child-completion notifications are generated by Codex itself, so their
  content is plaintext and previously fell back to a serialized JSON
  envelope inside an assistant message.
  
  With plaintext `input_text` supported for `agent_message`, both delivery
  paths can use the same model-visible type while preserving explicit
  author and recipient metadata.
  
  ## What changed
  
  - add plaintext `input_text` support to `AgentMessageInputContent` and
  regenerate the affected app-server schemas
  - preserve `InterAgentCommunication` as structured mailbox input instead
  of converting it to assistant text
  - record delivered communications as typed `agent_message` history items
  - persist a dedicated rollout item so local delivery metadata such as
  `trigger_turn` remains available without leaking into the Responses
  request
  - reconstruct typed agent messages on resume and preserve fork-turn
  truncation behavior
  - remove request-time assistant-content parsing
  - preserve plaintext and encrypted inter-agent deliveries in stage-one
  memory inputs
  - normalize and link plaintext and encrypted agent messages in rollout
  traces without treating inbound messages as child results
  - cover the real MultiAgent V2 child-completion path end to end with
  deterministic mailbox synchronization
  
  ## Verification
  
  - `just test -p codex-core
  plaintext_multi_agent_v2_completion_sends_agent_message`
  - `just test -p codex-core input_queue_drains_mailbox_in_delivery_order
  record_initial_history_reconstructs_typed_inter_agent_message
  fork_turn_positions_use_inter_agent_delivery_metadata`
  - `just test -p codex-memories-write
  serializes_inter_agent_communications_for_memory`
  - `just test -p codex-rollout-trace
  agent_messages_preserve_routing_and_content
  sub_agent_started_activity_creates_spawn_edge`
  - `just test -p codex-rollout-trace
  agent_result_edge_falls_back_to_child_thread_without_result_message`
  - `just test -p codex-protocol -p codex-rollout -p
  codex-app-server-protocol`

  jif · 2026-06-12 13:50:04 -07:00
  8f2d6416ce
• Include thread id in token budget context (#27663)

  ## Why
  
  The token budget full-context fragment identifies the current context
  window, but not the thread that owns that window. Including the thread
  id makes the initial context-window metadata self-contained, and
  `get_context_remaining` also needs to be usable from Code Mode without
  forcing callers to parse the model-facing fragment string.
  
  ## What changed
  
  - Include the session thread id in the initial `<token_budget>` context
  fragment.
  - Expose `get_context_remaining` as a Code Mode nested tool while
  keeping `new_context` direct-model-only.
  - Keep direct model-facing `get_context_remaining` output as the
  existing `<token_budget>` text fragment.
  - Return only `tokens_left` from the Code Mode structured result for
  `get_context_remaining`.
  - Update token-budget integration tests and add Code Mode coverage for
  the structured result.
  
  ## Verification
  
  - `just test -p codex-core token_budget`
  - `just test -p codex-core
  code_mode_get_context_remaining_returns_structured_result`
  - `just test -p core_test_support redacted_text_mode_normalizes_uuids`

  pakrym-oai · 2026-06-11 15:10:29 -07:00
  7516eb5c70
• core: Consolidate Responses API Codex metadata (#27122)

  ## What
  Introduce a `CodexResponsesMetadata` struct that defines all the core
  metadata we send to Responses API. Example fields are `thread_id`,
  `turn_id`, `window_id`, etc.
  
  Going forward, `client_metadata["x-codex-turn-metadata"]` will be the
  canonical way Codex sends metadata to Responses API across both HTTP and
  websocket transports.
  
  For now, we continue to emit the existing top-level HTTP headers and
  top-level `client_metadata` fields from the same
  `CodexResponsesMetadata` struct for compatibility reasons.
  
  Also, app-server clients who specify additional
  `responsesapi_client_metadata` via `turn/start` and `turn/steer` will
  have those fields merged into
  `client_metadata["x-codex-turn-metadata"]`, but cannot override the
  reserved fields that core uses (i.e. the fields in
  `CodexResponsesMetadata`).
  
  ## Why
  
  Responses API request instrumentation is the source of truth for
  downstream Codex analytics that join requests by Codex IDs such as
  session, thread, turn, and context window. Before this change, those
  values were assembled through several request-specific paths: HTTP
  request bodies, websocket handshake headers, websocket `response.create`
  payloads, compaction requests, and the rich `x-codex-turn-metadata`
  envelope all had their own wiring.
  
  That made metadata propagation easy to drift across API-key/direct
  Responses API requests, ChatGPT-auth/proxied requests, websocket
  requests, and compaction requests. It also made additions like
  `window_id` error-prone because a field could be added to one transport
  projection but missed in another.
  
  ## What changed
  
  - Added `CodexResponsesMetadata` as the core-owned snapshot for Codex
  metadata sent to ResponsesAPI.
  - Render `client_metadata["x-codex-turn-metadata"]`, flat
  `client_metadata` projections, and direct compatibility headers from
  that same snapshot.
  - Include the known Codex-owned fields in the turn metadata blob,
  including installation/session/thread/turn/window IDs, request kind,
  lineage, sandbox/workspace metadata, timing, and compaction details.
  - Treat app-server `responsesapi_client_metadata` as enrichment for the
  Codex turn metadata blob while preventing those extras from overriding
  Codex-owned fields.
  - Use the same metadata path for normal turns, websocket prewarm, local
  compaction, remote v1 compaction, and remote v2 compaction.
  - Keep websocket connection-only preconnect metadata separate so
  handshakes carry compatibility identity headers without inventing a fake
  turn metadata blob.
  
  ## Verification
  
  - `cargo check -p codex-core`
  - `just fix -p codex-core`

  Owen Lin · 2026-06-11 13:42:09 -07:00
  14df0e8833
• [codex] Add new context window tool (#27488)

  ## Why
  
  The token budget feature tells the model how much room remains in the
  current context window. When the model decides the current window is no
  longer useful, it needs a way to ask Codex to start over with a fresh
  context window without spending tokens on a compaction summary.
  
  This PR adds that model-requestable escape hatch on top of #27438.
  
  ## What changed
  
  - Added a direct-model-only `new_context` tool behind
  `Feature::TokenBudget`.
  - Stores the tool request on `AutoCompactWindow` and consumes it after
  sampling so the next follow-up request in the same turn starts in the
  new window.
  - Starts the new window as a no-summary compaction checkpoint that
  contains only fresh initial context, not preserved conversation history.
  - Keeps the new window aligned with token-budget startup context,
  including the `Current context window Z` message.
  - Added integration coverage and a snapshot showing the same-turn
  `new_context` flow into a fresh full-context follow-up request.
  
  ## Validation
  
  - `just test -p codex-core token_budget`

  pakrym-oai · 2026-06-11 03:39:07 +00:00
  87ab01834a
• Stop mirroring Codex user input into realtime (#27116)

  ## Why
  
  The realtime frontend model and the backing Codex thread should present
  one coherent assistant. Raw typed messages, steers, and worker reports
  belong to the orchestrator; the frontend model should receive the
  orchestrator's user-facing result rather than a second copy of those
  inputs.
  
  Today normal `turn/start` input is automatically inserted into the
  realtime conversation, while `turn/steer` is not. Besides creating
  inconsistent context, this can make the frontend model react
  independently before Codex has produced the response it should speak.
  
  ## What changed
  
  - Remove automatic accepted-user-input mirroring into realtime
  - Remove the mirror-only echo-suppression flag and dead V2 prefix helper
  - Preserve explicit app-to-realtime text injection and FEM-to-Codex
  delegation
  - Replace the positive mirror tests and obsolete snapshots with a
  negative routing regression test
  
  ## Test plan
  
  - `cargo test -p codex-core
  conversation_user_text_turn_is_not_sent_to_realtime`
  - `cargo test -p codex-core
  conversation_startup_context_is_truncated_and_sent_once_per_start`
  - `cargo test -p codex-core inbound_handoff_request_starts_turn`

  guinness-oai · 2026-06-09 15:20:01 -07:00
  cc8325f181
• Add HTTP window ID to Responses client metadata (#26923)

  ## Summary
  
  - Keep the existing `x-codex-window-id` HTTP header unchanged.
  - Also send the same window ID in Responses `client_metadata`, allowing
  supported backend paths to surface it as
  `x-client-meta-x-codex-window-id`.
  - Cover normal HTTP Responses and remote compaction v2 requests without
  changing window generation or compaction behavior.
  
  ## Why
  
  In the `2026-06-06T23` production hour, all 28,729 HTTP compaction
  requests had `window_id` in `x-codex-turn-metadata`, but only 73
  retained the direct `x-codex-window-id` header. The request-body
  `client_metadata` path is already used for installation ID and is
  preserved through supported Responses API paths.
  
  This is additive metadata only. It does not change the direct header,
  request count, model input, compaction routing, window generation, or
  user response behavior.
  
  Legacy `/v1/responses/compact` is intentionally unchanged. Its current
  server-side `CompressBody` schema does not accept `client_metadata` and
  rejects unknown fields, so supporting that path requires a backend
  schema change before the Codex client can safely send this field.
  
  ## Validation
  
  - Current head: `219baef3c`, rebased onto `origin/main` at `26d932983`.
  - The post-rebase diff remains limited to the original five files (`22`
  insertions, `6` deletions); the legacy experiment remains fully
  reverted.
  - `just test -p codex-core
  responses_stream_includes_subagent_header_on_review`: passed; validates
  normal HTTP Responses metadata.
  - `just test -p codex-core
  remote_compact_v2_reuses_compaction_trigger_for_followups`: passed;
  validates remote compaction v2.
  - `just test -p codex-core
  remote_manual_compact_chatgpt_auth_reuses_service_tier_and_prompt_cache_key`:
  passed; validates that legacy compact keeps its accepted payload shape.
  - `just test -p codex-core
  remote_manual_compact_api_auth_omits_service_tier_and_reuses_prompt_cache_key`:
  passed; validates the legacy API-key payload as well.
  - `just fmt`: passed; an unrelated root `justfile` rewrite produced by
  the formatter was discarded.
  - `git diff --check origin/main...HEAD`: passed.
  
  The focused server pytest could not start in the local monorepo
  environment because test setup is missing the `dotenv` module. Server
  source and tests explicitly show that `CompressBody` omits
  `client_metadata` and `/v1/responses/compact` returns HTTP 400 for
  unknown body fields.

  ningyi-oai · 2026-06-08 10:49:59 -07:00
  a815311466
• [codex] remove plain image wrapper spans (#24652)

  ## Why
  
  Remote image submissions currently wrap native `input_image` spans in
  literal `<image>` and `</image>` text spans. Those extra prompt tokens
  add structure without providing label or routing information.
  
  ## What Changed
  
  - Serialize `UserInput::Image` directly as an `input_image` content
  span.
  - Preserve named local-image framing and legacy wrapper parsing for
  labeled attachments and existing histories.
  - Update existing request-shape expectations for drag-and-drop images,
  model switching, and compaction.
  
  ## Validation
  
  - `just test -p codex-protocol`
  - Focused `codex-core` run covering
  `drag_drop_image_persists_rollout_request_shape`,
  `model_change_from_image_to_text_strips_prior_image_content`, and
  `snapshot_request_shape_pre_turn_compaction_including_incoming_user_message`
  
  ## Notes
  
  - A broader `just test -p codex-core` run was attempted; the affected
  tests passed, while the overall run failed in unrelated CLI, MCP, and
  tooling tests plus a `thread_manager` timeout.

  pakrym-oai · 2026-05-26 15:49:37 -07:00
  46391f7efa
• Add experimental turn additional context (#24154)

  ## Summary
  
  Adds experimental `additionalContext` support to `turn/start` and
  `turn/steer` so clients can provide ephemeral external context, such as
  browser or automation state, without turning that plumbing into a
  visible user prompt or triggering user-prompt lifecycle behavior.
  
  ## API Shape
  
  The parameter shape is:
  
  ```ts
  additionalContext?: Record<string, {
    value: string
    kind: "untrusted" | "application"
  }> | null
  ```
  
  Example:
  
  ```json
  {
    "additionalContext": {
      "browser_info": {
        "value": "Active tab is CI failures.",
        "kind": "untrusted"
      },
      "automation_info": {
        "value": "CI rerun is in progress.",
        "kind": "application"
      }
    }
  }
  ```
  
  The keys are opaque and caller-defined.
  
  ## Context Injection
  
  When provided, accepted entries are inserted into model context as
  hidden contextual message items, not as visible thread user-message
  items.
  
  `kind: "untrusted"` entries are inserted with role `user`:
  
  ```text
  <external_${key}>${value}</external_${key}>
  ```
  
  `kind: "application"` entries are inserted with role `developer`:
  
  ```text
  <${key}>${value}</${key}>
  ```
  
  Values are not escaped. Each value is truncated to 1k approximate tokens
  before wrapping.
  
  For `turn/start`, accepted additional context is inserted before normal
  user input. For `turn/steer`, additional context is merged only when the
  steer includes non-empty user input; context-only steers still reject as
  empty input.
  
  ## Dedupe Strategy
  
  `AdditionalContextStore` lives on session state and stores the latest
  complete additional-context map.
  
  Each `turn/start` or non-empty `turn/steer` treats its
  `additionalContext` as the current complete set of values. Entries are
  injected only when the key is new or the exact entry for that key
  changed, including `value` or `kind`. After merging, the store is
  replaced with the provided map, so omitted keys are removed from the
  retained set and can be injected again later if reintroduced.
  
  Omitting `additionalContext`, passing `null`, or passing an empty object
  resets the store to empty and injects nothing.
  
  ## What Changed
  
  - Threads experimental v2 `additionalContext` through app-server into
  core turn start and steer handling.
  - Adds separate contextual fragment types for untrusted user-role
  context and application developer-role context.
  - Uses pending response input items so additional context can be
  combined with normal user input without treating it as prompt text.
  - Adds integration coverage for start/steer flow, role routing,
  dedupe/reset behavior, deletion/re-add behavior, hook-blocked input
  behavior, empty context-only steer rejection, external-fragment marker
  matching, and truncation.

  pakrym-oai · 2026-05-26 13:02:34 -07:00
  768848ab6f
• [codex] Remove unused legacy shell tools (#22246)

  ## Why
  
  Recent session history showed no active use of the raw `shell`,
  `local_shell`, or `container.exec` execution surfaces. Keeping those
  handlers/specs wired into core leaves duplicate shell execution paths
  alongside the supported `shell_command` and unified exec tools.
  
  ## What changed
  
  - Removed the raw `shell` handler/spec and its `ShellToolCallParams`
  protocol helper.
  - Removed the legacy `local_shell` and `container.exec` handler/spec
  plumbing while preserving persisted-history compatibility for old
  response items.
  - Normalized model/config `default` and `local` shell selections to
  `shell_command`.
  - Pruned tests that exercised removed raw-shell/local-shell/apply-patch
  variants and kept coverage on `shell_command`, unified exec, and
  freeform `apply_patch`.
  
  ## Verification
  
  - `git diff --check`
  - `cargo test -p codex-protocol`
  - `cargo test -p codex-tools`
  - `cargo test -p codex-core tools::handlers::shell`
  - `cargo test -p codex-core tools::spec`
  - `cargo test -p codex-core tools::router`
  - `cargo test -p codex-core
  active_call_preserves_triggering_command_context`
  - `cargo test -p codex-core guardian_tests`
  - `cargo test -p codex-core --test all shell_serialization`
  - `cargo test -p codex-core --test all apply_patch_cli`
  - `cargo test -p codex-core --test all shell_command_`
  - `cargo test -p codex-core --test all local_shell`
  - `cargo test -p codex-core --test all otel::`
  - `cargo test -p codex-core --test all hooks::`
  - `just fix -p codex-core`
  - `just fix -p codex-tools`

  pakrym-oai · 2026-05-13 16:43:25 +00:00
  83decfa300
• Omit service_tier from remote /responses/compact requests under API auth (#21676)

  ## Summary
  
  API-key-auth remote compaction requests should not inherit
  `service_tier` from normal `/responses` turns. This path needs to match
  API auth expectations, while ChatGPT-auth remote compaction should keep
  reusing the shared request fields that still apply there.
  
  This change keeps the decision inline in
  `codex-rs/core/src/compact_remote.rs` only. Under API key auth, the
  classic remote `/responses/compact` path now omits `service_tier`; under
  ChatGPT auth, it keeps reusing the configured tier.
  `codex-rs/core/src/compact_remote_v2.rs` is unchanged. The remote
  compaction parity coverage and snapshots were updated to assert the
  API-key omission and preserve the ChatGPT-auth behavior.
  
  ## Testing
  
  - Updated remote compaction parity coverage in
  `codex-rs/core/tests/suite/compact_remote.rs` and the corresponding
  snapshots.

  Ahmed Ibrahim · 2026-05-08 11:15:14 +03:00
  71d80f9a14
• Propagate cache key and service tiers in compact (#21249)

  ## Why
  
  `/responses/compact` should preserve the request-affinity fields that
  apply to the active auth mode. ChatGPT-auth compact requests need the
  effective `service_tier`, and compact requests for every auth mode need
  the stable `prompt_cache_key`, so compaction does not quietly lose
  routing or cache behavior that normal sampling already has.
  
  This follows the request-parity direction from #20719, but keeps the net
  change focused on the compact payload fields needed here.
  
  ## What changed
  
  - Add `service_tier` and `prompt_cache_key` to the compact endpoint
  input payload.
  - Build the remote compact payload from the existing responses request
  builder output so `Fast` still maps to `priority` when compact sends a
  service tier.
  - Pass the turn service tier into remote compaction, but only include it
  in compact payloads for ChatGPT-backed auth.
  - Keep `prompt_cache_key` on compact payloads for all auth modes.
  - Add request-body diff snapshot coverage in
  `core/tests/suite/compact_remote.rs` for:
  - API-key auth reusing `prompt_cache_key` while omitting `service_tier`
  even when `Fast` is configured.
    - ChatGPT auth reusing both `service_tier` and `prompt_cache_key`.
  - Drive the snapshot coverage through five varied turns: plain text,
  multi-part text, tool-call continuation, image+text input, local-shell
  continuation, and final-turn reasoning output.
  
  ## Verification
  
  - Added insta snapshots for compact request-body parity against the last
  normal `/responses` request after five varied turns.
  - Not run locally per repo guidance; relying on GitHub CI for test
  execution.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-05-06 13:38:43 +03:00
  5d6f23a27b
• Update image outputs to default to high detail (#18386)

  Do not assume the default `detail`.

  pakrym-oai · 2026-04-18 11:01:12 -07:00
  53b1570367
• Revert "[codex] drain mailbox only at request boundaries" (#18325)

  ## Summary
  - Reverts PR #17749 so queued inter-agent mail can again preempt after
  reasoning/commentary output item boundaries.
  - Applies the revert to the current `codex/turn.rs` module layout and
  restores the prior pending-input test expectations/snapshots.
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-core --test all pending_input`
  - `cargo test -p codex-core` failed in unrelated
  `tools::js_repl::tests::js_repl_imported_local_files_can_access_repl_globals`:
  dotslash download hit `mktemp: mkdtemp failed ... Operation not
  permitted` in the sandbox temp dir.
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-18 09:53:48 -07:00
  e3c2acb9cd
• Clarify realtime v2 context and handoff messages (#17896)

  ## Summary
  - wrap realtime startup context in
  `<startup_context>...</startup_context>` tags
  - prefix V2 mirrored user text and relayed backend text with `[USER]` /
  `[BACKEND]`
  - remove the V2 progress suffix and replace the final V2 handoff output
  with a short completion acknowledgement while preserving the existing V1
  wrapper
  
  ## Testing
  - cargo test -p codex-api
  realtime_v2_session_update_includes_background_agent_tool_and_handoff_output_item
  -- --exact
  - cargo test -p codex-app-server webrtc_v2_background_agent_
  - cargo test -p codex-app-server webrtc_v2_text_input_is_
  - cargo test -p codex-core conversation_user_text_turn_is_

  bxie-openai · 2026-04-15 16:26:20 -07:00
  c2bdb7812c
• [codex] drain mailbox only at request boundaries (#17749)

  This changes multi-agent v2 mailbox handling so incoming inter-agent
  messages no longer preempt an in-flight sampling stream at reasoning or
  commentary output-item boundaries.

  Thibault Sottiaux · 2026-04-13 22:09:51 -07:00
  05c5829923
• Cap realtime mirrored user turns (#17685)

  Cap mirrored user text sent to realtime with the existing 300-token turn
  budget while preserving the full model turn.
  
  Adds integration coverage for capped realtime mirror payloads.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-13 14:31:18 -07:00
  ec0133f5f8
• Mirror user text into realtime (#17520)

  - Let typed user messages submit while realtime is active and mirror
  accepted text into the realtime text stream.
  - Add integration coverage and snapshot for outbound realtime text.

  Ahmed Ibrahim · 2026-04-12 15:03:14 -07:00
  d840b247d7
• Budget realtime current thread context (#17519)

  Select Current Thread startup context by budget from newest turns, cap
  each rendered turn at 300 approximate tokens, and add formatter plus
  integration snapshot coverage.

  Ahmed Ibrahim · 2026-04-12 11:59:09 -07:00
  4db60d5d8b
• Preempt mailbox mail after reasoning/commentary items (#16725)

  Send pending mailbox mail after completed reasoning or commentary items
  so follow-up requests can pick it up mid-turn.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-03 18:29:05 -07:00
  e4f1b3a65e
• Trim pre-turn context updates during rollback (#15577)

  ## Summary
  - trim contiguous developer/contextual-user pre-turn updates when
  rollback cuts back to a user turn
  - add a focused history regression test for the trim behavior
  - update the rollback request-boundary snapshots to show the fixed
  non-duplicating context shape
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Charley Cunningham · 2026-03-24 12:43:53 -07:00
  2d61357c76
• [codex] Add rollback context duplication snapshot (#15562)

  ## What changed
  - adds a targeted snapshot test for rollback with contextual diffs in
  `codex_tests.rs`
  - snapshots the exact model-visible request input before the rolled-back
  turn and on the follow-up request after rollback
  - shows the duplicate developer and environment context pair appearing
  again before the follow-up user message
  
  ## Why
  Rollback currently rewinds the reference context baseline without
  rewinding the live session overrides. On the next turn, the same
  contextual diff is emitted again and duplicated in the request sent to
  the model.
  
  ## Impact
  - makes the regression visible in a canonical snapshot test
  - keeps the snapshot on the shared `context_snapshot` path without
  adding new formatting helpers
  - gives a direct repro for future fixes to rollback/context
  reconstruction
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Charley Cunningham · 2026-03-23 15:36:23 -07:00
  0f34b14b41
• move plugin/skill instructions into dev msg and reorder (#14609)

  Move the general `Apps`, `Skills` and `Plugins` instructions blocks out
  of `user_instructions` and into the developer message, with new `Apps ->
  Skills -> Plugins` order for better clarity.
  
  Also wrap those sections in stable XML-style instruction tags (like
  other sections) and update prompt-layout tests/snapshots. This makes the
  tests less brittle in snapshot output (we can parse the sections), and
  it consolidates the capability instructions in one place.
  
  #### Tests
  Updated snapshots, added tests.
  
  `<AGENTS_MD>` disappearing in snapshots is expected: before this change,
  the wrapped user-instructions message was kept alive by `Skills`
  content. Now that `Skills` and `Plugins` are in the developer message,
  that wrapper only appears when there is real
  project-doc/user-instructions content.
  
  ---------
  
  Co-authored-by: Charley Cunningham <ccunningham@openai.com>

  sayan-oai · 2026-03-13 20:51:01 -07:00
  d272f45058
• Defer initial context insertion until the first turn (#14313)

  ## Summary
  - defer fresh-session `build_initial_context()` until the first real
  turn instead of seeding model-visible context during startup
  - rely on the existing `reference_context_item == None` turn-start path
  to inject full initial context on that first real turn (and again after
  baseline resets such as compaction)
  - add a regression test for `InitialHistory::New` and update affected
  deterministic tests / snapshots around developer-message layout,
  collaboration instructions, personality updates, and compact request
  shapes
  
  ## Notes
  - this PR does not add any special empty-thread `/compact` behavior
  - most of the snapshot churn is the direct result of moving the initial
  model-visible context from startup to the first real turn, so first-turn
  request layouts no longer contain a pre-user startup copy of permissions
  / environment / other developer-visible context
  - remote manual `/compact` with no prior user still skips the remote
  compact request; local first-turn `/compact` still issues a compact
  request, but that request now reflects the lack of startup-seeded
  context
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Charley Cunningham · 2026-03-11 12:33:10 -07:00
  f5bb338fdb
• Replay thread rollback from rollout history (#13615)

  - Replay thread rollback from the persisted rollout history instead of
  truncating in-memory state.\n- Add rollback coverage, including
  rollback-behind-compaction snapshot coverage.

  Ahmed Ibrahim · 2026-03-05 16:40:09 -08:00
  629cb15bc6
• Revert "Revert "realtime prompt changes"" (#13398)

  Reverts openai/codex#13385

  Ahmed Ibrahim · 2026-03-03 14:41:26 -08:00
  041c896509
• Record realtime close marker on replacement (#13058)

  ## Summary
  - record a realtime close developer message when a new realtime session
  replaces an active one
  - assert the replacement marker through the mocked responses request
  path
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>
  Co-authored-by: Charles Cunningham <ccunningham@openai.com>

  Ahmed Ibrahim · 2026-03-01 13:54:12 -08:00
  0aeb55bf08
• feat: add post-compaction sub-agent infos (#12774)

  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-02-26 18:55:34 +00:00
  3404ecff15
• core: bundle settings diff updates into one dev/user envelope (#12417)

  ## Summary
  - bundle contextual prompt injection into at most one developer message
  plus one contextual user message in both:
    - per-turn settings updates
    - initial context insertion
  - preserve `<model_switch>` across compaction by rebuilding it through
  canonical initial-context injection, instead of relying on
  strip/reattach hacks
  - centralize contextual user fragment detection in one shared definition
  table and reuse it for parsing/compaction logic
  - keep `AGENTS.md` in its natural serialized format:
    - `# AGENTS.md instructions for {dirname}`
    - `<INSTRUCTIONS>...</INSTRUCTIONS>`
  - simplify related tests/helpers and accept the expected snapshot/layout
  updates from bundled multi-part messages
  
  ## Why
  The goal is to converge toward a simpler, more intentional prompt shape
  where contextual updates are consistently represented as one developer
  envelope plus one contextual user envelope, while keeping parsing and
  compaction behavior aligned with that representation.
  
  ## Notable details
  - the temporary `SettingsUpdateEnvelope` wrapper was removed; these
  paths now return `Vec<ResponseItem>` directly
  - local/remote compaction no longer rely on model-switch strip/restore
  helpers
  - contextual user detection is now driven by shared fragment definitions
  instead of ad hoc matcher assembly
  - AGENTS/user instructions are still the same logical context; only the
  synthetic `<user_instructions>` wrapper was replaced by the natural
  AGENTS text format
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-app-server
  codex_message_processor::tests::extract_conversation_summary_prefers_plain_user_messages
  -- --exact`
  - `cargo test -p codex-core
  compact::tests::collect_user_messages_filters_session_prefix_entries
  --lib -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact::snapshot_request_shape_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact_remote::snapshot_request_shape_remote_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_apps_guidance_as_developer_message_when_enabled'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_developer_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_user_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::resume_includes_initial_messages_and_sends_prior_items'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::review::review_input_isolated_from_parent_history' -- --exact`
  - `cargo test -p codex-exec --test all
  'suite::resume::exec_resume_last_respects_cwd_filter_and_all_flag' --
  --exact`
  - `cargo test -p core_test_support
  context_snapshot::tests::full_text_mode_preserves_unredacted_text --
  --exact`
  
  ## Notes
  - I also ran several targeted `compact`, `compact_remote`,
  `prompt_caching`, `model_visible_layout`, and `event_mapping` tests
  while iterating on prompt-shape changes.
  - I have not claimed a clean full-workspace `cargo test` from this
  environment because local sandbox/resource conditions have previously
  produced unrelated failures in large workspace runs.

  Charley Cunningham · 2026-02-26 00:12:08 -08:00
  07aefffb1f
• Fix compaction context reinjection and model baselines (#12252)

  ## Summary
  - move regular-turn context diff/full-context persistence into
  `run_turn` so pre-turn compaction runs before incoming context updates
  are recorded
  - after successful pre-turn compaction, rely on a cleared
  `reference_context_item` to trigger full context reinjection on the
  follow-up regular turn (manual `/compact` keeps replacement history
  summary-only and also clears the baseline)
  - preserve `<model_switch>` when full context is reinjected, and inject
  it *before* the rest of the full-context items
  - scope `reference_context_item` and `previous_model` to regular user
  turns only so standalone tasks (`/compact`, shell, review, undo) cannot
  suppress future reinjection or `<model_switch>` behavior
  - make context-diff persistence + `reference_context_item` updates
  explicit in the regular-turn path, with clearer docs/comments around the
  invariant
  - stop persisting local `/compact` `RolloutItem::TurnContext` snapshots
  (only regular turns persist `TurnContextItem` now)
  - simplify resume/fork previous-model/reference-baseline hydration by
  looking up the last surviving turn context from rollout lifecycle
  events, including rollback and compaction-crossing handling
  - remove the legacy fallback that guessed from bare `TurnContext`
  rollouts without lifecycle events
  - update compaction/remote-compaction/model-visible snapshots and
  compact test assertions (including remote compaction mock response
  shape)
  
  ## Why
  We were persisting incoming context items before spawning the regular
  turn task, which let pre-turn compaction requests accidentally include
  incoming context diffs without the new user message. Fixing that exposed
  follow-on baseline issues around `/compact`, resume/fork, and standalone
  tasks that could cause duplicate context injection or suppress
  `<model_switch>` instructions.
  
  This PR re-centers the invariants around regular turns:
  - regular turns persist model-visible context diffs/full reinjection and
  update the `reference_context_item`
  - standalone tasks do not advance those regular-turn baselines
  - compaction clears the baseline when replacement history may have
  stripped the referenced context diffs
  
  ## Follow-ups (TODOs left in code)
  - `TODO(ccunningham)`: fix rollback/backtracking baseline handling more
  comprehensively
  - `TODO(ccunningham)`: include pending incoming context items in
  pre-turn compaction threshold estimation
  - `TODO(ccunningham)`: inject updated personality spec alongside
  `<model_switch>` so some model-switch paths can avoid forced full
  reinjection
  - `TODO(ccunningham)`: review task turn lifecycle
  (`TurnStarted`/`TurnComplete`) behavior and emit task-start context
  diffs for task types that should have them (excluding `/compact`)
  
  ## Validation
  - `just fmt`
  - CI should cover the updated compaction/resume/model-visible snapshot
  expectations and rollout-hydration behavior
  - I did **not** rerun the full local test suite after the latest
  resume-lookup / rollout-persistence simplifications

  Charley Cunningham · 2026-02-20 23:13:08 -08:00
  bb0ac5be70
• Add model-visible context layout snapshot tests (#12073)

  ## Summary
  - add a dedicated `core/tests/suite/model_visible_layout.rs` snapshot
  suite to materialize model-visible request layout in high-value
  scenarios
  - add three reviewer-focused snapshot scenarios:
    - turn-level context updates (cwd / permissions / personality)
    - first post-resume turn with model hydration + personality change
  - first post-resume turn where pre-turn model override matches rollout
  model
  - wire the new suite into `core/tests/suite/mod.rs`
  - commit generated `insta` snapshots under `core/tests/suite/snapshots/`
  
  ## Why
  This creates a stable, reviewable baseline of model-visible context
  layout against `main` before follow-on context-management refactors. It
  lets subsequent PRs show focused snapshot diffs for behavior changes
  instead of introducing the test surface and behavior changes at once.
  
  ## Testing
  - `just fmt`
  - `INSTA_UPDATE=always cargo test -p codex-core model_visible_layout`

  Charley Cunningham · 2026-02-17 22:30:29 -08:00
  c16f9daaaf
• Unify remote compaction snapshot mocks around default endpoint behavior (#12050)

  ## Summary
  - standardize remote compaction test mocking around one default behavior
  in shared helpers
  - make default remote compact mocks mirror production shape: keep
  `message/user` + `message/developer`, drop assistant/tool artifacts,
  then append a summary user message
  - switch non-special `compact_remote` tests to the shared default mock
  instead of ad-hoc JSON payloads
  
  ## Special-case tests that still use explicit mocks
  - remote compaction error payload / HTTP failure behavior
  - summary-only compact output behavior
  - manual `/compact` with no prior user messages
  - stale developer-instruction injection coverage
  
  ## Why
  This removes inconsistent manual remote compaction fixtures and gives us
  one source of truth for normal remote compact behavior, while preserving
  explicit mocks only where tests intentionally cover non-default
  behavior.

  Charley Cunningham · 2026-02-17 18:18:47 -08:00
  eb68767f2f
• core: snapshot tests for compaction requests, post-compaction layout, some additional compaction tests (#11487)

  This PR keeps compaction context-layout test coverage separate from
  runtime compaction behavior changes, so runtime logic review can stay
  focused.
  
  ## Included
  - Adds reusable context snapshot helpers in
  `core/tests/common/context_snapshot.rs` for rendering model-visible
  request/history shapes.
  - Standardizes helper naming for readability:
    - `format_request_input_snapshot`
    - `format_response_items_snapshot`
    - `format_labeled_requests_snapshot`
    - `format_labeled_items_snapshot`
  - Expands snapshot coverage for both local and remote compaction flows:
    - pre-turn auto-compaction
    - pre-turn failure/context-window-exceeded paths
    - mid-turn continuation compaction
    - manual `/compact` with and without prior user turns
  - Captures both sides where relevant:
    - compaction request shape
    - post-compaction history layout shape
  - Adds/uses shared request-inspection helpers so assertions target
  structured request content instead of ad-hoc JSON string parsing.
  - Aligns snapshots/assertions to current behavior and leaves explicit
  `TODO(ccunningham)` notes where behavior is known and intentionally
  deferred.
  
  ## Not Included
  - No runtime compaction logic changes.
  - No model-visible context/state behavior changes.

  Charley Cunningham · 2026-02-14 19:57:10 -08:00
  85034b189e