94 Commits

• core: remove redundant TurnContext and Prompt fields (#28638)

  ## Why
  
  `TurnContext` had accumulated dead fields and cached projections of
  values already owned by its per-turn `Config` or `ModelInfo`. Keeping
  both copies made ownership unclear and allowed artificial split-brain
  states, such as a compatibility hash differing from the model metadata
  it came from.
  
  `Prompt` similarly carried a write-only personality after personality
  selection had already been materialized into its base instructions.
  
  This makes the canonical owner explicit: configuration-backed values
  come from `config`, model-derived values come from `model_info`, and
  prompts contain only data consumed by request construction.
  
  ## What changed
  
  - Remove the unused `ghost_snapshot`, `codex_self_exe`, and
  `thread_source` fields.
  - Remove duplicate `comp_hash`, `truncation_policy`, `features`,
  `shell_environment_policy`, `codex_linux_sandbox_exe`, `compact_prompt`,
  and `tool_mode` fields.
  - Read those values directly from `TurnContext::config` or
  `TurnContext::model_info` at their consumers.
  - Remove the write-only `Prompt::personality` field and its constructor
  assignments.
  - Preserve review-turn inheritance of the parent turn's shell policy,
  Linux sandbox executable, and compact prompt through the review config.
  
  ## Testing
  
  - `cargo check -p codex-core --tests`

  pakrym-oai · 2026-06-16 22:17:24 -07:00
  172b2218a5
• feat(core): add metadata field to ResponseItem (#28355)

  ## Description
  
  This PR adds an optional `metadata` field to `ResponseItem` for
  Responses API calls. Only mechanical plumbing, no actual values
  populated and sent yet. Turns out just adding a new field to
  `ResponseItem` has quite a large blast radius already.
  
  This change is backwards compatible because `metadata` is optional and
  omitted when absent, so existing response items and rollout history
  without it still deserialize and requests that do not set it keep the
  same wire shape. For provider compatibility, we strip out `metadata`
  before non-OpenAI Responses requests so Azure and AWS Bedrock never see
  this field.
  
  My followup PR here will actually make use of it to start storing and
  passing along `turn_id`: https://github.com/openai/codex/pull/28360
  
  ## What changed
  
  - Added `ResponseItemMetadata` with optional `turn_id`, plus optional
  `metadata` on Responses API item variants and inter-agent communication.
  - Preserved item metadata through response-item rewrites such as
  truncation, missing tool-output synthesis, compaction history
  rebuilding, visible-history conversion, rollout/resume, and generated
  app-server schemas/types.
  - Strip item metadata from non-OpenAI Responses requests while
  preserving it for OpenAI-shaped requests.
  - Updated the mechanical fixture/test construction churn required by the
  new optional field.

  Owen Lin · 2026-06-15 15:05:28 -07:00
  040dafa32d
• core: Consolidate Responses API Codex metadata (#27122)

  ## What
  Introduce a `CodexResponsesMetadata` struct that defines all the core
  metadata we send to Responses API. Example fields are `thread_id`,
  `turn_id`, `window_id`, etc.
  
  Going forward, `client_metadata["x-codex-turn-metadata"]` will be the
  canonical way Codex sends metadata to Responses API across both HTTP and
  websocket transports.
  
  For now, we continue to emit the existing top-level HTTP headers and
  top-level `client_metadata` fields from the same
  `CodexResponsesMetadata` struct for compatibility reasons.
  
  Also, app-server clients who specify additional
  `responsesapi_client_metadata` via `turn/start` and `turn/steer` will
  have those fields merged into
  `client_metadata["x-codex-turn-metadata"]`, but cannot override the
  reserved fields that core uses (i.e. the fields in
  `CodexResponsesMetadata`).
  
  ## Why
  
  Responses API request instrumentation is the source of truth for
  downstream Codex analytics that join requests by Codex IDs such as
  session, thread, turn, and context window. Before this change, those
  values were assembled through several request-specific paths: HTTP
  request bodies, websocket handshake headers, websocket `response.create`
  payloads, compaction requests, and the rich `x-codex-turn-metadata`
  envelope all had their own wiring.
  
  That made metadata propagation easy to drift across API-key/direct
  Responses API requests, ChatGPT-auth/proxied requests, websocket
  requests, and compaction requests. It also made additions like
  `window_id` error-prone because a field could be added to one transport
  projection but missed in another.
  
  ## What changed
  
  - Added `CodexResponsesMetadata` as the core-owned snapshot for Codex
  metadata sent to ResponsesAPI.
  - Render `client_metadata["x-codex-turn-metadata"]`, flat
  `client_metadata` projections, and direct compatibility headers from
  that same snapshot.
  - Include the known Codex-owned fields in the turn metadata blob,
  including installation/session/thread/turn/window IDs, request kind,
  lineage, sandbox/workspace metadata, timing, and compaction details.
  - Treat app-server `responsesapi_client_metadata` as enrichment for the
  Codex turn metadata blob while preventing those extras from overriding
  Codex-owned fields.
  - Use the same metadata path for normal turns, websocket prewarm, local
  compaction, remote v1 compaction, and remote v2 compaction.
  - Keep websocket connection-only preconnect metadata separate so
  handshakes carry compatibility identity headers without inventing a fake
  turn metadata blob.
  
  ## Verification
  
  - `cargo check -p codex-core`
  - `just fix -p codex-core`

  Owen Lin · 2026-06-11 13:42:09 -07:00
  14df0e8833
• [codex-analytics] Emit structured compaction codex errors (#27082)

  ## Summary
  - replace raw compaction `error` analytics with `codex_error_kind` and
  `codex_error_http_status_code`
  - derive compaction error telemetry from `CodexErr` using the same
  `CodexErrKind` mapping and HTTP status helper used by turn events
  - remove the pre-compact hook stop reason from the internal compaction
  outcome now that it is no longer emitted as raw analytics text
  
  ## Why
  Compaction `error` was a raw `CodexErr::to_string()` value, which can
  carry free-form provider or user-derived text. Structured Codex error
  fields preserve useful low-cardinality telemetry without sending the raw
  string.
  
  ## Validation
  - `just fmt`
  - `just test -p codex-analytics`
  - `just test -p codex-core
  compact::tests::build_token_limited_compacted_history_appends_summary_message`
  
  Attempted `just test -p codex-core`; the changed crate compiled, but the
  full target failed in unrelated environment-dependent tests such as
  missing helper binaries and shell snapshot timeouts.

  rhan-oai · 2026-06-11 06:07:06 +00:00
  8d9f33c87c
• [codex-analytics] report cached input tokens for v2 compaction (#27103)

  ## Summary
  
  - add nullable `cached_input_tokens` to the compaction analytics event
  - populate it from response usage for compaction v2
  - leave it `null` for other compaction implementations
  
  This adds visibility into prompt-cache usage for v2 compaction without
  changing compaction behavior.
  
  ## Testing
  
  - `just test -p codex-analytics`
  - `just test -p codex-core
  collect_compaction_output_accepts_additional_output_items`

  rhan-oai · 2026-06-10 22:47:22 -07:00
  383708e74e
• [codex] Add token budget context feature (#27438)

  ## Why
  
  The model should be able to see bounded context-window budget metadata
  when the `token_budget` feature is enabled. The full-window message is
  only injected with full context, while normal turns get a smaller
  follow-up only when reported usage first crosses a budget threshold.
  
  ## What changed
  
  - Added the `TokenBudget` feature flag.
  - Added `<token_budget>` developer fragments for full context-window
  metadata and current-window remaining tokens.
  - Inserted the threshold message during normal turn handling by
  comparing token usage before and after sampling, avoiding persistent
  threshold bookkeeping.
  - Added core integration coverage for full-context-only metadata and
  25/50/75 percent threshold messages.
  
  ## Verification
  
  - `just test -p codex-core token_budget`
  - `git diff --check`

  pakrym-oai · 2026-06-10 20:07:06 -07:00
  658af936fd
• [codex] Store compact window id in rollout (#27264)

  ## Why
  
  Compaction window identity is part of session history, not model-client
  transport state. Persisting it with the compacted rollout item lets
  resumed threads continue from the reconstructed window without keeping
  mutable window state on `ModelClient`.
  
  ## What changed
  
  - Added `window_id` to `CompactedItem` and stamp it when
  `replace_compacted_history` installs compacted history.
  - Moved auto-compact window id ownership into `AutoCompactWindow` /
  `SessionState`; `ModelClient` now receives the request window id from
  callers instead of storing it.
  - Returned `window_id` from rollout reconstruction for resume.
  Reconstruction uses the newest surviving compacted item's stored
  `window_id` when present, and falls back to the legacy compacted-item
  count when it is absent.
  - Kept fork startup at the fresh default window id and updated direct
  model-client tests to pass explicit test window ids.
  
  ## Validation
  
  - `cargo check -p codex-core --tests`

  pakrym-oai · 2026-06-10 08:47:16 -07:00
  30ddb3325e
• [codex-analytics] report compaction analytics details (#26680)

  ## Why
  
  Compaction analytics adds retained image count and compaction summary
  output tokens for v1.5 specifically.
  
  ## What changed
  
  - Add nullable `retained_image_count` and `compaction_summary_tokens`
  fields to `codex_compaction_event`.
  - Populate them only for `responses_compaction_v2`: retained images come
  from the retained v2 compacted history, and summary tokens come from
  `response.completed.token_usage.output_tokens`.
  - Leave local and legacy remote compaction events as `null` for these
  detail fields.
  
  ## Verification
  
  - `just fmt`
  - `just fix -p codex-core`
  - `just test -p codex-core
  build_v2_compacted_history_counts_retained_input_images`
  - `git diff --check`

  rhan-oai · 2026-06-08 10:52:31 -07:00
  f1c18df9ae
• [codex] Support model-defined reasoning efforts (#26444)

  ## Summary
  - accept non-empty model-defined reasoning effort values while
  preserving built-in effort behavior
  - propagate the non-Copy effort type through core, app-server, TUI,
  telemetry, and persistence call sites
  - preserve string wire encoding and expose an open-string schema for
  clients
  - update model selection and shortcut behavior for model-advertised
  effort values
  
  ## Root cause
  `ReasoningEffort` gained a string-backed custom variant, so it could no
  longer implement `Copy` or rely on derived closed-enum serialization.
  Existing consumers still moved effort values from shared references and
  assumed a fixed built-in value set.
  
  ## Validation
  - `just fmt`
  - Local tests and compilation were not run per request; relying on CI.

  Ahmed Ibrahim · 2026-06-04 13:36:24 -07:00
  8ac304c299
• [codex-analytics] report compaction request token counts (#25946)

  ## Why
  
  Compaction analytics need token counts that better represent the request
  being compacted. The existing session snapshot can diverge from the
  actual remote compaction request after output rewriting, and remote v2
  can use server-side Responses usage when available.
  
  ## What changed
  
  - Add an optional `active_context_tokens_before` override to
  `CompactionAnalyticsAttempt::track(...)` for remote compaction when it
  has a better before-token value than the begin-time session snapshot.
  The local `/compact` path passes no override.
  - For remote v1 `responses_compact`, subtract the estimated token delta
  from pre-compaction output rewriting from the session snapshot, capped
  by locally-added tokens since the last successful API response.
  - For remote v2 `responses_compaction_v2`, use the same bounded
  output-rewrite fallback as remote v1, then overwrite
  `active_context_tokens_before` with server `token_usage.input_tokens`
  from the `response.completed` event when present.
  - Keep the existing v2 compaction-output validation while carrying the
  completed response token usage through `collect_compaction_output`.
  
  ## Verification
  
  - `just fmt`
  - `just test -p codex-core
  collect_compaction_output_accepts_additional_output_items`
  - `git diff --check`

  rhan-oai · 2026-06-04 03:00:44 +00:00
  c143a86de8
• chore: mechanical rename (#26156)

  Rename `Session::conversation_id` to `Session::thread_id` with an auto
  refactor in RustRover

  jif · 2026-06-03 15:38:30 +02:00
  c9ae0f48a1
• [codex-analytics] Track CodexErr details in turn analytics (#25707)

  ## Summary
  - add analytics-only `CodexErr` telemetry to `codex_turn_event` while
  leaving existing `turn_error` unchanged
  - record terminal `CodexErr` facts from core immediately before the
  existing turn error event is sent
  - emit source-truth `codex_error_*` fields for downstream analytics,
  including the raw `CodexErr::InvalidRequest(String)` message as
  `codex_error_subreason`
  
  ## Validation
  - `just test -p codex-analytics`
  - attempted `just test -p codex-core`, but the local run timed out
  across unrelated integration suites in this environment and is not being
  used as validation

  rhan-oai · 2026-06-02 11:40:35 -07:00
  8e4b92d294
• [codex] Consolidate shared prompts in codex-prompts (#25151)

  ## Why
  
  `codex_core` is consistently a bottleneck for incremental builds during
  iteration. The simplest fix is to make the crate smaller.
  
  ## Summary
  
  `codex-core` owns several reusable prompt renderers and static prompt
  assets, which makes the crate harder to split apart.
  
  Rename `codex-review-prompts` to `codex-prompts` and move shared review,
  goal, permissions, compaction, realtime, hierarchical AGENTS.md, and
  `apply_patch` prompts into it. Move prompt-only tests and update
  consumers and `CODEOWNERS`.
  
  ## Validation
  
  - `just test -p codex-prompts -p codex-apply-patch`
  - `just test -p codex-core prompt_caching`
  - Bazel builds for the affected crates

  Adam Perry @ OpenAI · 2026-06-01 18:45:07 +00:00
  ba2b67f9cd
• [codex] Store pending response items directly (#24865)
  pakrym-oai · 2026-05-28 07:23:08 -07:00
  1c7832ffa3
• [codex] add compaction metadata to turn headers (#24368)

  ## Summary
  - Add `request_kind` values for foreground turn, startup prewarm,
  compaction, and detached memory model requests.
  - Attach compaction dispatch metadata to local Responses, legacy
  `/v1/responses/compact`, and remote v2 compact requests.
  - Add the existing logical context-window identifier as `window_id` on
  turn-owned model request metadata.
  - Keep identity fields optional for detached memory requests, while
  still emitting `request_kind="memory"` in non-git/no-sandbox workspaces.
  
  ## Root Cause
  `x-codex-turn-metadata` has more than one producer. Foreground turns and
  compaction requests own a real turn and should carry that turn identity.
  Detached memory stage-one requests do not own a foreground turn, so
  absent identity fields are valid rather than missing data. Startup
  websocket prewarm is also a model request, but it has `generate=false`
  and must not be counted as a foreground turn.
  
  `thread_source` or session source identifies where a thread came from
  (for example review, guardian, or another subagent). `request_kind`
  identifies what the current outbound model request is doing (`turn`,
  `prewarm`, `compaction`, or `memory`). A review or guardian thread can
  issue either a normal turn request or a compaction request, so source
  cannot replace request kind.
  
  ## Behavior / Impact
  - Ordinary foreground requests send `request_kind="turn"`, their real
  identity fields, and `window_id="<thread_id>:<window_generation>"`.
  - Startup websocket warmup requests send `request_kind="prewarm"` so
  they are not counted as foreground turns.
  - Compaction requests send `request_kind="compaction"`, their real
  owning turn identity, the existing `window_id`, and
  `compaction.{trigger,reason,implementation,phase,strategy}`.
  - Detached memory stage-one requests send `request_kind="memory"`
  without `session_id`, `thread_id`, `turn_id`, or `window_id`; when no
  workspace metadata exists, the kind-only header is still emitted.
  - `session_id`, `thread_id`, `turn_id`, and `window_id` remain optional
  in the header schema because detached memory requests do not own a
  foreground turn or context window.
  - `window_id` is not a new ID system: it is copied from the already-sent
  `x-codex-window-id` / WS client metadata value at model-request dispatch
  time.
  - Existing `x-codex-window-id` HTTP/WS emission, value format,
  generation advancement, resume behavior, and fork reset behavior are
  unchanged.
  - `request_kind`, `window_id`, and upstream turn-owned identity fields
  remain schema-owned; input `responsesapi_client_metadata` cannot replace
  their canonical values.
  - No table, DAG, export, app-server API, or MCP `_meta` schema changes
  are included.
  
  A compaction attempt stopped by a pre-compact hook issues no model
  request and therefore has no request header; its outcome remains in
  analytics events. Status, error, duration, and token deltas also remain
  analytics fields rather than request-header fields.
  
  Future detached-memory attribution using a real initiating turn ID as
  `trigger_turn_id` is intentionally not part of this PR.
  
  ## Sync With Main
  - Final pushed head `716342e79` is rebased onto `origin/main@0d37db4b2`.
  - The metadata conflict came from upstream `#24160`, which added
  `forked_from_thread_id` on the same `turn_metadata` surface. Resolution
  preserves that field and its protection from client metadata override
  alongside this PR's request-kind, compaction, and window-id fields.
  - While resolving the overlapping commits, I removed an accidental
  recursive model-request overlay and a duplicate detached-memory header
  builder before completing the rebase.
  
  ## Latency / User Experience Boundary
  - Foreground turns perform no new filesystem, git, or network work. New
  fields are inserted into metadata already serialized for outgoing
  requests.
  - Compaction issues the same model/HTTP requests with the same prompt,
  model, service tier, and sampling settings; only metadata bytes change.
  - Startup prewarm already sent metadata; it is now correctly classified
  as `prewarm`.
  - Non-git detached memory now sends a small kind-only metadata header
  rather than no header.
  - This client diff adds no user-visible latency mechanism beyond
  negligible serialization and header bytes on already-existing requests.
  
  ## Validation
  On conflict-resolved head `1d35c2cfb` based on `origin/main@487521733`:
  - `just fmt` (passed)
  - `just fix -p codex-core` (passed)
  - `git diff --check origin/main...HEAD` (passed)
  - `just test -p codex-core -E 'test(turn_metadata) |
  test(websocket_first_turn_uses_startup_prewarm_and_create) |
  test(responses_stream_includes_turn_metadata_header_for_git_workspace_e2e)
  |
  test(responses_websocket_forwards_turn_metadata_on_initial_and_incremental_create)
  | test(remote_compact_v2_retries_failures_with_stream_retry_budget) |
  test(window_id_advances_after_compact_persists_on_resume_and_resets_on_fork)'`
  (`23 passed`; `bench-smoke` passed)
  - `just test -p codex-app-server -E
  'test(turn_start_forwards_client_metadata_to_responses_request_v2) |
  test(turn_start_forwards_client_metadata_to_responses_websocket_request_body_v2)
  | test(auto_compaction_remote_emits_started_and_completed_items)'` (`3
  passed`; `bench-smoke` passed)
  - `just test -p codex-memories-write` (`29 passed`; `bench-smoke`
  passed)

  ningyi-oai · 2026-05-27 11:09:33 -07:00
  bee78806a9
• Add trace_id to TurnStartedEvent (#23980)

  ## Why
  [Recent PR](https://github.com/openai/codex/pull/22709) removed
  `trace_id` from `TurnContextItem`.
  
  ## What changed
  - Add to `TurnStartedEvent` so rollout consumers can correlate turns
  with telemetry traces.
  - Note that the branch name is out of date because I originally re-added
  to `TurnContextItem`, but we decided to move it to `TurnStartedEvent`.
  
  ## Verification
  - `cargo test -p codex-protocol`
  - `cargo test -p codex-core --lib
  regular_turn_emits_turn_started_without_waiting_for_startup_prewarm`
  - `cargo test -p codex-core --test all
  emits_warning_when_resumed_model_differs`
  - `cargo test -p codex-rollout`
  - `cargo test -p codex-state`

  mchen-oai · 2026-05-22 13:10:56 -07:00
  3c83e57bfa
• [codex] Remove external websocket session resets (#23384)

  ## Why
  
  Compaction now installs replacement history inside the session, but the
  turn and compaction callers were still reaching into
  `ModelClientSession` to reset websocket transport state after that
  install. That made a transport-level reset part of the compaction API
  even though websocket incremental request selection already checks
  whether the next request is a strict extension of the previous one and
  falls back to a full `response.create` when it is not.
  
  ## What changed
  
  - Removed the compaction-side calls to `reset_websocket_session` from
  `compact.rs` and `session/turn.rs`.
  - Simplified pre-sampling and mid-turn compaction helpers so they return
  `CodexResult<()>` instead of carrying a reset flag.
  - Made `ModelClientSession::reset_websocket_session` private to
  `client.rs`, leaving only the websocket timeout recovery path inside the
  client as a caller.
  
  ## Validation
  
  - `cargo test -p codex-core --test all
  responses_websocket_creates_on_non_prefix`
  - `cargo test -p codex-core --test all
  steered_user_input_waits_for_model_continuation_after_mid_turn_compact`
  - `cargo test -p codex-core --test all
  pre_sampling_compact_runs_on_switch_to_smaller_context_model`

  pakrym-oai · 2026-05-19 01:13:38 +00:00
  17d552fb4d
• Revert "[codex] Harden overflow auto-compaction recovery" (#22170)

  Reverts openai/codex#22141

  Ahmed Ibrahim · 2026-05-11 19:33:15 +03:00
  69f3183a8e
• [codex] Harden overflow auto-compaction recovery (#22141)

  ## Why
  Dogfooder feedback exposed two correctness gaps in normal-loop overflow
  recovery:
  
  1. a sampling request that hit `ContextWindowExceeded` could keep
  re-entering auto-compaction indefinitely if the compacted retry still
  did not fit, and
  2. local compact-history rebuilds flattened user messages down to text,
  so an overflowing `[image, "what is this?"]` turn could be retried
  without the image after compaction.
  
  That means recovery could either fail to terminate cleanly or proceed
  with a materially weakened version of the user request.
  
  ## What changed
  - Move normal-loop `ContextWindowExceeded` handling into the sampling
  retry loop, so successful rescue compaction consumes the provider retry
  budget instead of creating an unbounded outer-turn loop.
  - Keep compacted user-history rebuilds structured:
  `collect_user_messages` now carries user `UserInput` content rather than
  flattened strings, and `build_compacted_history` reconstructs full user
  messages from that structured representation.
  - Preserve image inputs while retaining the existing text-budget
  truncation behavior for compacted user history.
  - Preserve existing compaction-task failure handling and client-session
  reset behavior while bounding repeated overflow retries.
  - Add focused regression coverage for:
    - recovery after a normal-loop overflow,
    - retry-budget exhaustion after repeated overflow,
    - local recovery preserving image + text input,
    - remote recovery preserving image + text input,
    - remote compaction v2 preserving image + text input, and
    - compaction failure still terminating cleanly.
  
  The main behavior changes are in `codex-rs/core/src/session/turn.rs` and
  `codex-rs/core/src/compact.rs`.
  
  ## Verification
  - Not run locally; relying on PR CI for this update.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-05-11 16:16:49 +00:00
  15e79f3c26
• Add compact lifecycle hooks (started by vincentkoc - external contrib) (#19905)

  Based on work from Vincent K -
  https://github.com/openai/codex/pull/19060
  
  <img width="1836" height="642" alt="CleanShot 2026-04-29 at 20 47 40@2x"
  src="https://github.com/user-attachments/assets/b647bb89-65fe-40c8-80b0-7a6b7c984634"
  />
  
  ## Why
  
  Compaction rewrites the conversation context that future model turns
  receive, but hooks currently have no deterministic lifecycle point
  around that rewrite. This adds compact lifecycle hooks so users can
  audit manual and automatic compaction, surface hook messages in the UI,
  and run post-compaction follow-up without overloading tool or prompt
  hooks.
  
  ## What Changed
  
  - Added `PreCompact` and `PostCompact` hook events across hook config,
  discovery, dispatch, generated schemas, app-server notifications,
  analytics, and TUI hook rendering.
  - Added trigger matching for compact hooks with the documented `manual`
  and `auto` matcher values.
  - Wired `PreCompact` before both local and remote compaction, and
  `PostCompact` after successful local or remote compaction.
  - Kept compact hook command input to lifecycle metadata: session id,
  Codex turn id, transcript path, cwd, hook event name, model, and
  trigger.
  - Made compact stdout handling consistent with other hooks: plain stdout
  is ignored as debug output, while malformed JSON-looking stdout is
  reported as failed hook output.
  - Added integration coverage for compact hook dispatch, trigger
  matching, post-compact execution, and the audited behavior that
  `decision:"block"` does not block compaction.
  
  ## Out of Scope
  
  - Hook-specific compaction blocking is not implemented;
  `decision:"block"` and exit-code-2 blocking semantics are intentionally
  unsupported for `PreCompact`.
  - Custom compaction instructions are not exposed to compact hooks in
  this PR.
  - Compact summaries, summary character counts, and summary previews are
  not exposed to compact hooks in this PR.
  
  ## Verification
  
  - `cargo test -p codex-hooks`
  - `cargo test -p codex-core
  manual_pre_compact_block_decision_does_not_block_compaction`
  - `cargo test -p codex-app-server hooks_list`
  - `cargo test -p codex-core config_schema_matches_fixture`
  - `cargo test -p codex-tui hooks_browser`
  
  ## Docs
  
  The developer documentation for Codex hooks should be updated alongside
  this feature to document `PreCompact` and `PostCompact`, the
  `manual`/`auto` matcher values, and the compact hook payload fields.
  
  ---------
  
  Co-authored-by: Vincent Koc <vincentkoc@ieee.org>

  Andrei Eternal · 2026-05-06 18:08:31 -07:00
  527d52df03
• 2- Use string service tiers in session protocol (#20971)

  ## Summary
  - break service tier session/op/app-server protocol fields from the
  closed enum to string tier ids
  - send the service tier string directly through model requests, prewarm,
  compaction, memories, and TUI/app-server turn starts
  - regenerate app-server protocol JSON/TypeScript schemas, removing the
  standalone ServiceTier TS enum
  
  ## Verification
  - just fmt
  - cargo check -p codex-core -p codex-app-server -p codex-tui
  - just write-app-server-schema
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-05-06 18:00:21 +03:00
  be1d3cff93
• feat: add remote compaction v2 Responses client path (#20773)

  ## Why
  
  This adds the `remote_compaction_v2` client path so remote compaction
  can run through the normal Responses stream and install a
  `context_compaction` item that trigger a compaction.
  
  The goal is to migrate some of the compaction logic on the client side
  
  We keeps the v2 transport behind a feature flag while letting follow-up
  requests reuse the compacted context instead of falling back to the
  legacy compaction item shape.
  
  ## What changed
  
  - add `ResponseItem::ContextCompaction` and refresh the generated
  app-server / schema / TypeScript fixtures that expose response items on
  the wire
  - add `core/src/compact_remote_v2.rs` to send compaction through the
  standard streamed Responses client, require exactly one
  `context_compaction` output item, and install that item into compacted
  history
  - route manual compact and auto-compaction through the v2 path when
  `remote_compaction_v2` is enabled, while keeping the existing remote
  compaction path as the fallback
  - preserve the new item type across history retention, follow-up request
  construction, telemetry, rollout persistence, and rollout-trace
  normalization
  - add targeted coverage for the feature flag, `context_compaction`
  serialization, rollout-trace normalization, and remote-compaction
  follow-up behavior
  
  ## Verification
  
  - added protocol tests for `context_compaction`
  serialization/deserialization in `protocol/src/models.rs`
  - added rollout-trace coverage for `context_compaction` normalization in
  `rollout-trace/src/reducer/conversation_tests.rs`
  - added remote compaction integration coverage for v2 follow-up reuse
  and mixed compaction output streams in
  `core/tests/suite/compact_remote.rs`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-05-04 14:15:01 +02:00
  d927f61208
• [codex] Remove unused event messages (#20511)

  ## Why
  
  Several legacy `EventMsg` variants were still emitted or mapped even
  though clients either ignored them or had moved to item/lifecycle
  events. `Op::Undo` had also degraded to an unavailable shim, so this
  removes that dead task path instead of preserving a command that cannot
  do useful work.
  
  `McpStartupComplete`, `WebSearchBegin`, and `ImageGenerationBegin` are
  intentionally kept because useful consumers still depend on them: MCP
  startup completion drives readiness behavior, and the begin events let
  app-server/core consumers surface in-progress web-search and
  image-generation items before the final payload arrives.
  
  ## What Changed
  
  - Removed weak legacy event variants and payloads from `codex-protocol`,
  including legacy agent deltas, background events, and undo lifecycle
  events.
  - Kept/restored `EventMsg::McpStartupComplete`,
  `EventMsg::WebSearchBegin`, and `EventMsg::ImageGenerationBegin` with
  serializer and emission coverage.
  - Updated core, rollout, MCP server, app-server thread history,
  review/delegate filtering, and tests to rely on the useful replacement
  events that remain.
  - Removed `Op::Undo`, `UndoTask`, the undo test module, and stale TUI
  slash-command comments.
  - Stopped agent job/background progress and compaction retry notices
  from emitting `BackgroundEvent` payloads.
  
  ## Verification
  
  - `cargo check -p codex-protocol -p codex-app-server-protocol -p
  codex-core -p codex-rollout -p codex-rollout-trace -p codex-mcp-server`
  - `cargo test -p codex-protocol -p codex-app-server-protocol -p
  codex-rollout -p codex-rollout-trace -p codex-mcp-server`
  - `cargo test -p codex-core --test all suite::items`
  - `just fix -p codex-protocol -p codex-app-server-protocol -p codex-core
  -p codex-rollout -p codex-rollout-trace -p codex-mcp-server`
  - Earlier coverage on this PR also included `codex-mcp`, `codex-tui`,
  core library tests, MCP/plugin/delegate/review/agent job tests, and MCP
  startup TUI tests.

  pakrym-oai · 2026-04-30 20:03:26 -07:00
  f50c02d7bc
• Remove ghost snapshots (#19481)

  ## Summary
  - Remove `ghost_snapshot` / `GhostCommit` from the Responses API surface
  and generated SDK/schema artifacts.
  - Keep legacy config loading compatible, but make undo a no-op that
  reports the feature is unavailable.
  - Clean up core history, compaction, telemetry, rollout, and tests to
  stop carrying ghost snapshot items.
  
  ## Testing
  - Unit tests passed for `codex-protocol`, `codex-core` targeted undo and
  compaction flows, `codex-rollout`, and `codex-app-server-protocol`.
  - Regenerated config and app-server schemas plus Python SDK artifacts
  and verified they match the checked-in outputs.

  pakrym-oai · 2026-04-27 18:48:57 -07:00
  4e05f3053c
• [codex-analytics] remove ga flag (#19863)
  rhan-oai · 2026-04-27 19:29:19 +00:00
  215d5a8f7c
• Delete unused ResponseItem::Message.end_turn (#19605)

  This field is unused. Delete it.

  Andrey Mishchenko · 2026-04-26 17:18:09 -07:00
  35bc6e3d01
• [rollout_trace] Record core session rollout traces (#18877)

  ## Summary
  
  Wires rollout trace recording into `codex-core` session and turn
  execution. This records the core model request/response, compaction, and
  session lifecycle boundaries needed for replay without yet tracing every
  nested runtime/tool boundary.
  
  ## Stack
  
  This is PR 2/5 in the rollout trace stack.
  
  - [#18876](https://github.com/openai/codex/pull/18876): Add rollout
  trace crate
  - [#18877](https://github.com/openai/codex/pull/18877): Record core
  session rollout traces
  - [#18878](https://github.com/openai/codex/pull/18878): Trace tool and
  code-mode boundaries
  - [#18879](https://github.com/openai/codex/pull/18879): Trace sessions
  and multi-agent edges
  - [#18880](https://github.com/openai/codex/pull/18880): Add debug trace
  reduction command
  
  ## Review Notes
  
  This layer is the first live integration point. The important review
  question is whether trace recording is isolated from normal session
  behavior: trace failures should not become user-visible execution
  failures, and recording should preserve the existing turn/session
  lifecycle semantics.
  
  The PR depends on the reducer/data model from the first stack entry and
  only introduces the core recorder surface that later PRs use for richer
  runtime and relationship events.

  cassirer-openai · 2026-04-22 17:00:48 +00:00
  f67383bcba
• Move codex module under session (#18249)

  ## Summary
  - rename the core codex module root to session/mod.rs without using
  #[path]
  - move the codex module directory and tests under core/src/session
  - remove session/mod.rs reexports so call sites use explicit child
  module paths
  
  ## Testing
  - cargo test -p codex-core --lib
  - cargo check -p codex-core --tests
  - just fmt
  - just fix -p codex-core
  - git diff --check

  pakrym-oai · 2026-04-17 16:18:53 +00:00
  71e4c6fa17
• feat: add opt-in provider runtime abstraction (#17713)

  ## Summary
  
  - Add `codex-model-provider` as the runtime home for model-provider
  behavior that does not belong in `codex-core`, `codex-login`, or
  `codex-api`.
  - The new crate wraps configured `ModelProviderInfo` in a
  `ModelProvider` trait object that can resolve the API provider config,
  provider-scoped auth manager, and request auth provider for each call.
  - This centralizes provider auth behavior in one place today, and gives
  us an extension point for future provider-specific auth, model listing,
  request setup, and related runtime behavior.
  
  ## Tests
  Ran tests manually to make sure that provider auth under different
  configs still work as expected.
  
  ---------
  
  Co-authored-by: pakrym-oai <pakrym@openai.com>

  Celia Chen · 2026-04-17 02:27:45 +00:00
  a803790a10
• Support remote compaction for Azure responses providers (#17958)

  Azure Responses providers were still falling back to local compaction
  because the compaction gate only checked
  `ModelProviderInfo::is_openai()`.
  
  Move the capability check onto `ModelProviderInfo` with
  `supports_remote_compaction()`, backed by the existing Azure Responses
  endpoint detection used in `codex-api`, and have `core::compact`
  delegate to that helper.
  
  Add regression coverage for:
  - OpenAI providers using remote compaction
  - Azure providers using remote compaction
  - non-OpenAI/non-Azure providers staying on the local path
  
  resolves #17773
  
  ---------
  
  Co-authored-by: Michael Bolin <mbolin@openai.com>

  Ivan Murashko · 2026-04-15 13:05:11 -07:00
  f2a4925f63
• [codex-analytics] feature plumbing and emittance (#16640)

  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16640).
  * #16870
  * #16706
  * #16641
  * __->__ #16640

  rhan-oai · 2026-04-13 23:11:49 -07:00
  b704df85b8
• [codex-analytics] add compaction analytics event (#17155)

  - event for compaction analytics
  - introduces thread-connection and thread metadata caches for data
  denormalization, expected to be useful for denormalization onto core
  emitted events in general
  - threads analytics event client into core (mirrors approved
  implementation in #16640)
  - denormalizes key thread metadata: thread_source, subagent_source,
  parent_thread_id, as well as app-server client and runtime metadata)
  - compaction strategy defaults to memento, forward compatible with
  expected prefill_compaction strategy
  
  1. Manual standalone compact, local
  `INFO | 2026-04-09 17:35:50 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d0-5cfb-70c0-bef9-165c3bf9b2df', 'turn_id':
  '019d74d0-d7f6-7c81-acc6-aae2030243d6', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'manual', 'reason':
  'user_requested', 'implementation': 'responses', 'phase':
  'standalone_turn', 'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 20170, 'active_context_tokens_after':
  4830, 'started_at': 1775781337, 'completed_at': 1775781350,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 13524} | `
  
  2. Auto pre-turn compact, local
  `INFO | 2026-04-09 17:37:30 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d2-45ef-71d1-9c93-23cc0c13d988', 'turn_id':
  '019d74d2-7b42-7372-9f0e-c0da3f352328', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'auto', 'reason':
  'context_limit', 'implementation': 'responses', 'phase': 'pre_turn',
  'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 20063, 'active_context_tokens_after':
  4822, 'started_at': 1775781444, 'completed_at': 1775781449,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 5497} | `
  
  3. Auto mid-turn compact, local
  `INFO | 2026-04-09 17:38:28 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d3-212f-7a20-8c0a-4816a978675e', 'turn_id':
  '019d74d3-3ee1-7462-89f6-2ffbeefcd5e3', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'auto', 'reason':
  'context_limit', 'implementation': 'responses', 'phase': 'mid_turn',
  'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 20325, 'active_context_tokens_after':
  14641, 'started_at': 1775781500, 'completed_at': 1775781508,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 7507} | `
  
  4. Remote /responses/compact, manual standalone
  `INFO | 2026-04-09 17:40:20 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d4-7a11-78a1-89f7-0535a1149416', 'turn_id':
  '019d74d4-e087-7183-9c20-b1e40b7578c0', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'manual', 'reason':
  'user_requested', 'implementation': 'responses_compact', 'phase':
  'standalone_turn', 'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 23461, 'active_context_tokens_after':
  6171, 'started_at': 1775781601, 'completed_at': 1775781620,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 18971} | `

  rhan-oai · 2026-04-10 13:03:54 -07:00
  5779be314a
• [codex-analytics] add protocol-native turn timestamps (#16638)

  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16638).
  * #16870
  * #16706
  * #16659
  * #16641
  * #16640
  * __->__ #16638

  rhan-oai · 2026-04-06 16:22:59 -07:00
  756c45ec61
• [codex] add context-window lineage headers (#16758)

  This change adds client-owned context-window and parent thread id
  headers to all requests to responses api.

  Thibault Sottiaux · 2026-04-04 05:54:31 +00:00
  9e19004bc2
• remove temporary ownership re-exports (#16626)

  Stacked on #16508.
  
  This removes the temporary `codex-core` / `codex-login` re-export shims
  from the ownership split and rewrites callsites to import directly from
  `codex-model-provider-info`, `codex-models-manager`, `codex-api`,
  `codex-protocol`, `codex-feedback`, and `codex-response-debug-context`.
  
  No behavior change intended; this is the mechanical import cleanup layer
  split out from the ownership move.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-03 00:33:34 -07:00
  af8a9d2d2b
• core: remove cross-crate re-exports from lib.rs (#16512)

  ## Why
  
  `codex-core` was re-exporting APIs owned by sibling `codex-*` crates,
  which made downstream crates depend on `codex-core` as a proxy module
  instead of the actual owner crate.
  
  Removing those forwards makes crate boundaries explicit and lets leaf
  crates drop unnecessary `codex-core` dependencies. In this PR, this
  reduces the dependency on `codex-core` to `codex-login` in the following
  files:
  
  ```
  codex-rs/backend-client/Cargo.toml
  codex-rs/mcp-server/tests/common/Cargo.toml
  ```
  
  ## What
  
  - Remove `codex-rs/core/src/lib.rs` re-exports for symbols owned by
  `codex-login`, `codex-mcp`, `codex-rollout`, `codex-analytics`,
  `codex-protocol`, `codex-shell-command`, `codex-sandboxing`,
  `codex-tools`, and `codex-utils-path`.
  - Delete the `default_client` forwarding shim in `codex-rs/core`.
  - Update in-crate and downstream callsites to import directly from the
  owning `codex-*` crate.
  - Add direct Cargo dependencies where callsites now target the owner
  crate, and remove `codex-core` from `codex-rs/backend-client`.

  Michael Bolin · 2026-04-01 23:06:24 -07:00
  aa2403e2eb
• Move string truncation helpers into codex-utils-string (#15572)

  - move the shared byte-based middle truncation logic from `core` into
  `codex-utils-string`
  - keep token-specific truncation in `codex-core` so rollout can reuse
  the shared helper in the next stacked PR
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-03-24 15:45:40 -07:00
  062fa7a2bb
• Apply argument comment lint across codex-rs (#14652)

  ## Why
  
  Once the repo-local lint exists, `codex-rs` needs to follow the
  checked-in convention and CI needs to keep it from drifting. This commit
  applies the fallback `/*param*/` style consistently across existing
  positional literal call sites without changing those APIs.
  
  The longer-term preference is still to avoid APIs that require comments
  by choosing clearer parameter types and call shapes. This PR is
  intentionally the mechanical follow-through for the places where the
  existing signatures stay in place.
  
  After rebasing onto newer `main`, the rollout also had to cover newly
  introduced `tui_app_server` call sites. That made it clear the first cut
  of the CI job was too expensive for the common path: it was spending
  almost as much time installing `cargo-dylint` and re-testing the lint
  crate as a representative test job spends running product tests. The CI
  update keeps the full workspace enforcement but trims that extra
  overhead from ordinary `codex-rs` PRs.
  
  ## What changed
  
  - keep a dedicated `argument_comment_lint` job in `rust-ci`
  - mechanically annotate remaining opaque positional literals across
  `codex-rs` with exact `/*param*/` comments, including the rebased
  `tui_app_server` call sites that now fall under the lint
  - keep the checked-in style aligned with the lint policy by using
  `/*param*/` and leaving string and char literals uncommented
  - cache `cargo-dylint`, `dylint-link`, and the relevant Cargo
  registry/git metadata in the lint job
  - split changed-path detection so the lint crate's own `cargo test` step
  runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes
  - continue to run the repo wrapper over the `codex-rs` workspace, so
  product-code enforcement is unchanged
  
  Most of the code changes in this commit are intentionally mechanical
  comment rewrites or insertions driven by the lint itself.
  
  ## Verification
  
  - `./tools/argument-comment-lint/run.sh --workspace`
  - `cargo test -p codex-tui-app-server -p codex-tui`
  - parsed `.github/workflows/rust-ci.yml` locally with PyYAML
  
  ---
  
  * -> #14652
  * #14651

  Michael Bolin · 2026-03-16 16:48:15 -07:00
  b77fe8fefe
• fix: move inline codex-rs/core unit tests into sibling files (#14444)

  ## Why
  PR #13783 moved the `codex.rs` unit tests into `codex_tests.rs`. This
  applies the same extraction pattern across the rest of `codex-rs/core`
  so the production modules stay focused on runtime code instead of large
  inline test blocks.
  
  Keeping the tests in sibling files also makes follow-up edits easier to
  review because product changes no longer have to share a file with
  hundreds or thousands of lines of test scaffolding.
  
  ## What changed
  - replaced each inline `mod tests { ... }` in `codex-rs/core/src/**`
  with a path-based module declaration
  - moved each extracted unit test module into a sibling `*_tests.rs`
  file, using `mod_tests.rs` for `mod.rs` modules
  - preserved the existing `cfg(...)` guards and module-local structure so
  the refactor remains structural rather than behavioral
  
  ## Testing
  - `cargo test -p codex-core --lib` (`1653 passed; 0 failed; 5 ignored`)
  - `just fix -p codex-core`
  - `cargo fmt --check`
  - `cargo shear`

  Michael Bolin · 2026-03-12 08:16:36 -07:00
  0c8a36676a
• chore(otel): rename OtelManager to SessionTelemetry (#13808)

  ## Summary
  This is a purely mechanical refactor of `OtelManager` ->
  `SessionTelemetry` to better convey what the struct is doing. No
  behavior change.
  
  ## Why
  
  `OtelManager` ended up sounding much broader than what this type
  actually does. It doesn't manage OTEL globally; it's the session-scoped
  telemetry surface for emitting log/trace events and recording metrics
  with consistent session metadata (`app_version`, `model`, `slug`,
  `originator`, etc.).
  
  `SessionTelemetry` is a more accurate name, and updating the call sites
  makes that boundary a lot easier to follow.
  
  ## Validation
  
  - `just fmt`
  - `cargo test -p codex-otel`
  - `cargo test -p codex-core`

  Owen Lin · 2026-03-06 16:23:30 -08:00
  289ed549cf
• Remove Responses V1 websocket implementation (#13364)

  V2 is the way to go!

  pakrym-oai · 2026-03-03 11:32:53 -07:00
  69df12efb3
• add fast mode toggle (#13212)

  - add a local Fast mode setting in codex-core (similar to how model id
  is currently stored on disk locally)
  - send `service_tier=priority` on requests when Fast is enabled
  - add `/fast` in the TUI and persist it locally
  - feature flag

  pash-openai · 2026-03-02 20:29:33 -08:00
  2f5b01abd6
• Record realtime close marker on replacement (#13058)

  ## Summary
  - record a realtime close developer message when a new realtime session
  replaces an active one
  - assert the replacement marker through the mocked responses request
  path
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>
  Co-authored-by: Charles Cunningham <ccunningham@openai.com>

  Ahmed Ibrahim · 2026-03-01 13:54:12 -08:00
  0aeb55bf08
• Unify rollout reconstruction with resume/fork TurnContext hydration (#12612)

  ## Summary
  
  This PR unifies rollout history reconstruction and resume/fork metadata
  hydration under a single `Session::reconstruct_history_from_rollout`
  implementation.
  
  The key change from main is that replay metadata now comes from the same
  reconstruction pass that rebuilds model-visible history, instead of
  doing a second bespoke rollout scan to recover `previous_model` /
  `reference_context_item`.
  
  ## What Changed
  
  ### Unified reconstruction output
  
  `reconstruct_history_from_rollout` now returns a single
  `RolloutReconstruction` bundle containing:
  
  - rebuilt `history`
  - `previous_model`
  - `reference_context_item`
  
  Resume and fork both consume that shared output directly.
  
  ### Reverse replay core
  
  The reconstruction logic moved into
  `codex-rs/core/src/codex/rollout_reconstruction.rs` and now scans
  rollout items newest-to-oldest.
  
  That reverse pass:
  
  - derives `previous_model`
  - derives whether `reference_context_item` is preserved or cleared
  - stops early once it has both resume metadata and a surviving
  `replacement_history` checkpoint
  
  History materialization is still bridged eagerly for now by replaying
  only the surviving suffix forward, which keeps the history result stable
  while moving the control flow toward the future lazy reverse loader
  design.
  
  ### Removed bespoke context lookup
  
  This deletes `last_rollout_regular_turn_context_lookup` and its separate
  compaction-aware scan.
  
  The previous model / baseline metadata is now computed from the same
  replay state that rebuilds history, so resume/fork cannot drift from the
  reconstructed transcript view.
  
  ### `TurnContextItem` persistence contract
  
  `TurnContextItem` is now treated as the replay source of truth for
  durable model-visible baselines.
  
  This PR keeps the following contract explicit:
  
  - persist `TurnContextItem` for the first real user turn so resume can
  recover `previous_model`
  - persist it for later turns that emit model-visible context updates
  - if mid-turn compaction reinjects full initial context into replacement
  history, persist a fresh `TurnContextItem` after `Compacted` so
  resume/fork can re-establish the baseline from the rewritten history
  - do not treat manual compaction or pre-sampling compaction as creating
  a new durable baseline on their own
  
  ## Behavior Preserved
  
  - rollback replay stays aligned with `drop_last_n_user_turns`
  - rollback skips only user turns
  - incomplete active user turns are dropped before older finalized turns
  when rollback applies
  - unmatched aborts do not consume the current active turn
  - missing abort IDs still conservatively clear stale compaction state
  - compaction clears `reference_context_item` until a later
  `TurnContextItem` re-establishes it
  - `previous_model` still comes from the newest surviving user turn that
  established one
  
  ## Tests
  
  Targeted validation run for the current branch shape:
  
  - `cd codex-rs && cargo test -p codex-core --lib
  codex::rollout_reconstruction_tests -- --nocapture`
  - `cd codex-rs && just fmt`
  
  The branch also extracts the rollout reconstruction tests into
  `codex-rs/core/src/codex/rollout_reconstruction_tests.rs` so this logic
  has a dedicated home instead of living inline in `codex.rs`.

  Charley Cunningham · 2026-02-27 13:50:45 -08:00
  695957a348
• core: bundle settings diff updates into one dev/user envelope (#12417)

  ## Summary
  - bundle contextual prompt injection into at most one developer message
  plus one contextual user message in both:
    - per-turn settings updates
    - initial context insertion
  - preserve `<model_switch>` across compaction by rebuilding it through
  canonical initial-context injection, instead of relying on
  strip/reattach hacks
  - centralize contextual user fragment detection in one shared definition
  table and reuse it for parsing/compaction logic
  - keep `AGENTS.md` in its natural serialized format:
    - `# AGENTS.md instructions for {dirname}`
    - `<INSTRUCTIONS>...</INSTRUCTIONS>`
  - simplify related tests/helpers and accept the expected snapshot/layout
  updates from bundled multi-part messages
  
  ## Why
  The goal is to converge toward a simpler, more intentional prompt shape
  where contextual updates are consistently represented as one developer
  envelope plus one contextual user envelope, while keeping parsing and
  compaction behavior aligned with that representation.
  
  ## Notable details
  - the temporary `SettingsUpdateEnvelope` wrapper was removed; these
  paths now return `Vec<ResponseItem>` directly
  - local/remote compaction no longer rely on model-switch strip/restore
  helpers
  - contextual user detection is now driven by shared fragment definitions
  instead of ad hoc matcher assembly
  - AGENTS/user instructions are still the same logical context; only the
  synthetic `<user_instructions>` wrapper was replaced by the natural
  AGENTS text format
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-app-server
  codex_message_processor::tests::extract_conversation_summary_prefers_plain_user_messages
  -- --exact`
  - `cargo test -p codex-core
  compact::tests::collect_user_messages_filters_session_prefix_entries
  --lib -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact::snapshot_request_shape_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact_remote::snapshot_request_shape_remote_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_apps_guidance_as_developer_message_when_enabled'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_developer_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_user_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::resume_includes_initial_messages_and_sends_prior_items'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::review::review_input_isolated_from_parent_history' -- --exact`
  - `cargo test -p codex-exec --test all
  'suite::resume::exec_resume_last_respects_cwd_filter_and_all_flag' --
  --exact`
  - `cargo test -p core_test_support
  context_snapshot::tests::full_text_mode_preserves_unredacted_text --
  --exact`
  
  ## Notes
  - I also ran several targeted `compact`, `compact_remote`,
  `prompt_caching`, `model_visible_layout`, and `event_mapping` tests
  while iterating on prompt-shape changes.
  - I have not claimed a clean full-workspace `cargo test` from this
  environment because local sandbox/resource conditions have previously
  produced unrelated failures in large workspace runs.

  Charley Cunningham · 2026-02-26 00:12:08 -08:00
  07aefffb1f
• Fix compaction context reinjection and model baselines (#12252)

  ## Summary
  - move regular-turn context diff/full-context persistence into
  `run_turn` so pre-turn compaction runs before incoming context updates
  are recorded
  - after successful pre-turn compaction, rely on a cleared
  `reference_context_item` to trigger full context reinjection on the
  follow-up regular turn (manual `/compact` keeps replacement history
  summary-only and also clears the baseline)
  - preserve `<model_switch>` when full context is reinjected, and inject
  it *before* the rest of the full-context items
  - scope `reference_context_item` and `previous_model` to regular user
  turns only so standalone tasks (`/compact`, shell, review, undo) cannot
  suppress future reinjection or `<model_switch>` behavior
  - make context-diff persistence + `reference_context_item` updates
  explicit in the regular-turn path, with clearer docs/comments around the
  invariant
  - stop persisting local `/compact` `RolloutItem::TurnContext` snapshots
  (only regular turns persist `TurnContextItem` now)
  - simplify resume/fork previous-model/reference-baseline hydration by
  looking up the last surviving turn context from rollout lifecycle
  events, including rollback and compaction-crossing handling
  - remove the legacy fallback that guessed from bare `TurnContext`
  rollouts without lifecycle events
  - update compaction/remote-compaction/model-visible snapshots and
  compact test assertions (including remote compaction mock response
  shape)
  
  ## Why
  We were persisting incoming context items before spawning the regular
  turn task, which let pre-turn compaction requests accidentally include
  incoming context diffs without the new user message. Fixing that exposed
  follow-on baseline issues around `/compact`, resume/fork, and standalone
  tasks that could cause duplicate context injection or suppress
  `<model_switch>` instructions.
  
  This PR re-centers the invariants around regular turns:
  - regular turns persist model-visible context diffs/full reinjection and
  update the `reference_context_item`
  - standalone tasks do not advance those regular-turn baselines
  - compaction clears the baseline when replacement history may have
  stripped the referenced context diffs
  
  ## Follow-ups (TODOs left in code)
  - `TODO(ccunningham)`: fix rollback/backtracking baseline handling more
  comprehensively
  - `TODO(ccunningham)`: include pending incoming context items in
  pre-turn compaction threshold estimation
  - `TODO(ccunningham)`: inject updated personality spec alongside
  `<model_switch>` so some model-switch paths can avoid forced full
  reinjection
  - `TODO(ccunningham)`: review task turn lifecycle
  (`TurnStarted`/`TurnComplete`) behavior and emit task-start context
  diffs for task types that should have them (excluding `/compact`)
  
  ## Validation
  - `just fmt`
  - CI should cover the updated compaction/resume/model-visible snapshot
  expectations and rollout-hydration behavior
  - I did **not** rerun the full local test suite after the latest
  resume-lookup / rollout-persistence simplifications

  Charley Cunningham · 2026-02-20 23:13:08 -08:00
  bb0ac5be70
• Move previous turn context tracking into ContextManager history (#12179)

  ## Summary
  - add `previous_context_item: Option<TurnContextItem>` to
  `ContextManager`
  - expose session/state accessors for reading and updating the stored
  previous context item
  - switch settings diffing to use `TurnContextItem` instead of
  `TurnContext`
  - remove submission-loop local `previous_context` and persist the
  previous context item in history
  
  ## Testing
  - `just fmt`
  - `just fix -p codex-core`
  - `cargo test -p codex-core --test all model_switching::`
  - `cargo test -p codex-core --test all collaboration_instructions::`
  - `cargo test -p codex-core --test all personality::`
  - `cargo test -p codex-core --test all
  permissions_messages::permissions_message_not_added_when_no_change`

  Charley Cunningham · 2026-02-19 09:56:20 -08:00
  f2d5842ed1
• Handle model-switch base instructions after compaction (#11659)

  Strip trailing <model_switch> during model-switch compaction request,
  and append <model_switch> after model switch compaction

  Charley Cunningham · 2026-02-13 19:02:53 -08:00
  67e577da53
• turn metadata followups (#11782)

  some trivial simplifications from #11677

  pash-openai · 2026-02-13 14:59:16 -08:00
  a5e8e69d18
• turn metadata: per-turn non-blocking (#11677)
  pash-openai · 2026-02-13 12:48:29 -08:00
  6c0a924203