64 Commits

• [codex-analytics] Track CodexErr details in turn analytics (#25707)

  ## Summary
  - add analytics-only `CodexErr` telemetry to `codex_turn_event` while
  leaving existing `turn_error` unchanged
  - record terminal `CodexErr` facts from core immediately before the
  existing turn error event is sent
  - emit source-truth `codex_error_*` fields for downstream analytics,
  including the raw `CodexErr::InvalidRequest(String)` message as
  `codex_error_subreason`
  
  ## Validation
  - `just test -p codex-analytics`
  - attempted `just test -p codex-core`, but the local run timed out
  across unrelated integration suites in this environment and is not being
  used as validation

  rhan-oai · 2026-06-02 11:40:35 -07:00
  8e4b92d294
• [codex] add compaction metadata to turn headers (#24368)

  ## Summary
  - Add `request_kind` values for foreground turn, startup prewarm,
  compaction, and detached memory model requests.
  - Attach compaction dispatch metadata to local Responses, legacy
  `/v1/responses/compact`, and remote v2 compact requests.
  - Add the existing logical context-window identifier as `window_id` on
  turn-owned model request metadata.
  - Keep identity fields optional for detached memory requests, while
  still emitting `request_kind="memory"` in non-git/no-sandbox workspaces.
  
  ## Root Cause
  `x-codex-turn-metadata` has more than one producer. Foreground turns and
  compaction requests own a real turn and should carry that turn identity.
  Detached memory stage-one requests do not own a foreground turn, so
  absent identity fields are valid rather than missing data. Startup
  websocket prewarm is also a model request, but it has `generate=false`
  and must not be counted as a foreground turn.
  
  `thread_source` or session source identifies where a thread came from
  (for example review, guardian, or another subagent). `request_kind`
  identifies what the current outbound model request is doing (`turn`,
  `prewarm`, `compaction`, or `memory`). A review or guardian thread can
  issue either a normal turn request or a compaction request, so source
  cannot replace request kind.
  
  ## Behavior / Impact
  - Ordinary foreground requests send `request_kind="turn"`, their real
  identity fields, and `window_id="<thread_id>:<window_generation>"`.
  - Startup websocket warmup requests send `request_kind="prewarm"` so
  they are not counted as foreground turns.
  - Compaction requests send `request_kind="compaction"`, their real
  owning turn identity, the existing `window_id`, and
  `compaction.{trigger,reason,implementation,phase,strategy}`.
  - Detached memory stage-one requests send `request_kind="memory"`
  without `session_id`, `thread_id`, `turn_id`, or `window_id`; when no
  workspace metadata exists, the kind-only header is still emitted.
  - `session_id`, `thread_id`, `turn_id`, and `window_id` remain optional
  in the header schema because detached memory requests do not own a
  foreground turn or context window.
  - `window_id` is not a new ID system: it is copied from the already-sent
  `x-codex-window-id` / WS client metadata value at model-request dispatch
  time.
  - Existing `x-codex-window-id` HTTP/WS emission, value format,
  generation advancement, resume behavior, and fork reset behavior are
  unchanged.
  - `request_kind`, `window_id`, and upstream turn-owned identity fields
  remain schema-owned; input `responsesapi_client_metadata` cannot replace
  their canonical values.
  - No table, DAG, export, app-server API, or MCP `_meta` schema changes
  are included.
  
  A compaction attempt stopped by a pre-compact hook issues no model
  request and therefore has no request header; its outcome remains in
  analytics events. Status, error, duration, and token deltas also remain
  analytics fields rather than request-header fields.
  
  Future detached-memory attribution using a real initiating turn ID as
  `trigger_turn_id` is intentionally not part of this PR.
  
  ## Sync With Main
  - Final pushed head `716342e79` is rebased onto `origin/main@0d37db4b2`.
  - The metadata conflict came from upstream `#24160`, which added
  `forked_from_thread_id` on the same `turn_metadata` surface. Resolution
  preserves that field and its protection from client metadata override
  alongside this PR's request-kind, compaction, and window-id fields.
  - While resolving the overlapping commits, I removed an accidental
  recursive model-request overlay and a duplicate detached-memory header
  builder before completing the rebase.
  
  ## Latency / User Experience Boundary
  - Foreground turns perform no new filesystem, git, or network work. New
  fields are inserted into metadata already serialized for outgoing
  requests.
  - Compaction issues the same model/HTTP requests with the same prompt,
  model, service tier, and sampling settings; only metadata bytes change.
  - Startup prewarm already sent metadata; it is now correctly classified
  as `prewarm`.
  - Non-git detached memory now sends a small kind-only metadata header
  rather than no header.
  - This client diff adds no user-visible latency mechanism beyond
  negligible serialization and header bytes on already-existing requests.
  
  ## Validation
  On conflict-resolved head `1d35c2cfb` based on `origin/main@487521733`:
  - `just fmt` (passed)
  - `just fix -p codex-core` (passed)
  - `git diff --check origin/main...HEAD` (passed)
  - `just test -p codex-core -E 'test(turn_metadata) |
  test(websocket_first_turn_uses_startup_prewarm_and_create) |
  test(responses_stream_includes_turn_metadata_header_for_git_workspace_e2e)
  |
  test(responses_websocket_forwards_turn_metadata_on_initial_and_incremental_create)
  | test(remote_compact_v2_retries_failures_with_stream_retry_budget) |
  test(window_id_advances_after_compact_persists_on_resume_and_resets_on_fork)'`
  (`23 passed`; `bench-smoke` passed)
  - `just test -p codex-app-server -E
  'test(turn_start_forwards_client_metadata_to_responses_request_v2) |
  test(turn_start_forwards_client_metadata_to_responses_websocket_request_body_v2)
  | test(auto_compaction_remote_emits_started_and_completed_items)'` (`3
  passed`; `bench-smoke` passed)
  - `just test -p codex-memories-write` (`29 passed`; `bench-smoke`
  passed)

  ningyi-oai · 2026-05-27 11:09:33 -07:00
  bee78806a9
• Add trace_id to TurnStartedEvent (#23980)

  ## Why
  [Recent PR](https://github.com/openai/codex/pull/22709) removed
  `trace_id` from `TurnContextItem`.
  
  ## What changed
  - Add to `TurnStartedEvent` so rollout consumers can correlate turns
  with telemetry traces.
  - Note that the branch name is out of date because I originally re-added
  to `TurnContextItem`, but we decided to move it to `TurnStartedEvent`.
  
  ## Verification
  - `cargo test -p codex-protocol`
  - `cargo test -p codex-core --lib
  regular_turn_emits_turn_started_without_waiting_for_startup_prewarm`
  - `cargo test -p codex-core --test all
  emits_warning_when_resumed_model_differs`
  - `cargo test -p codex-rollout`
  - `cargo test -p codex-state`

  mchen-oai · 2026-05-22 13:10:56 -07:00
  3c83e57bfa
• feat: retain remote compaction truncation parity in v2 (#23728)

  ## Why
  
  Remote compaction now has two implementations: the existing
  server-rebuilt v1 path and the newer client-rebuilt v2 path behind
  `remote_compaction_v2`. The v1 path bounds retained
  user/developer/system history before installing the compaction item,
  while v2 was previously carrying the full retained history forward. That
  made the two paths diverge for large pre-compaction transcripts even
  though they are meant to preserve the same compaction contract.
  
  This aligns v2 with the retained-history budget expected from v1 so
  switching the feature flag does not materially change which
  pre-compaction messages survive into the rebuilt history.
  
  ## What changed
  
  - Apply a retained-message character budget while rebuilding v2
  compacted history in `core/src/compact_remote_v2.rs`.
  - Keep newest retained messages first, truncate the boundary message
  with the shared `truncate_text(...)` helper, and drop older retained
  messages once the budget is exhausted.
  - Preserve non-text retained message content such as images while
  truncating text content.
  - Use the current `64_000` token retained-message default translated to
  the existing `4x` character budget.
  
  ## Testing
  
  - `cargo test -p codex-core compact_remote_v2::tests::`
  - Added focused coverage for newest-first retention and truncating
  multipart retained messages without dropping images.

  jif-oai · 2026-05-21 15:07:03 +02:00
  94442b7f95
• Remove explicit connector tool undeferral (#23390)

  ## Summary
  - remove the explicit-connector carveout that kept mentioned app tools
  directly exposed instead of deferred
  - keep the surviving explicit-mention reconstruction only for analytics,
  preserving `codex_app_mentioned` and `codex_app_used.invoke_type`
  - trim the now-unused prompt/tool-exposure plumbing and refresh coverage
  around always-defer behavior
  
  ## Verification
  - `just fmt`
  - `cargo test -p codex-analytics`
  - `cargo test -p codex-core` *(one transient timeout in
  `shell_snapshot::tests::macos_zsh_snapshot_includes_sections`; isolated
  rerun passed)*
  - `cargo test -p codex-core --lib
  shell_snapshot::tests::macos_zsh_snapshot_includes_sections`
  - `cargo test -p codex-core --test all
  explicit_app_mentions_respect_always_defer`
  - `cargo test -p codex-core --lib
  mcp_tool_exposure::tests::always_defer_feature_defers_apps_too`
  - `just fix -p codex-analytics`
  - `just fix -p codex-core`

  sayan-oai · 2026-05-18 21:33:46 -07:00
  1dd9bf9a74
• [codex] Use compaction_trigger item for remote compaction v2 (#22809)

  ## Why
  
  Remote compaction v2 was still using `context_compaction` as both the
  request trigger and the compacted output shape. The Responses API now
  has the landed contract for this flow: Codex sends a dedicated `{
  "type": "compaction_trigger" }` input item, and the backend returns the
  standard `compaction` output item with encrypted content.
  
  This aligns the v2 path with that wire contract while preserving the
  existing local compacted-history post-processing behavior.
  
  ## What changed
  
  - Add `ResponseItem::CompactionTrigger` and regenerate the app-server
  protocol schema fixtures.
  - Send `compaction_trigger` from `remote_compaction_v2` instead of a
  payload-less `context_compaction`.
  - Collect exactly one backend `compaction` output item, then reuse the
  existing compacted-history rebuilding path.
  - Treat the trigger item as a transient request marker rather than model
  output or persisted rollout/memory content.
  
  ## Verification
  
  - `cargo test -p codex-protocol compaction_trigger`
  - `cargo test -p codex-core remote_compact_v2`
  - `cargo test -p codex-core compact_remote_v2`
  - `cargo test -p codex-core
  responses_websocket_sends_response_processed_after_remote_compaction_v2`
  - `just write-app-server-schema`
  - `cargo test -p codex-app-server-protocol schema_fixtures`

  jif-oai · 2026-05-15 11:40:35 +02:00
  0322ac3df8
• [codex] Filter legacy warning messages during compaction (#22243)

  ## Why
  
  Older sessions can contain model-warning records persisted as `user`
  messages, including the unified exec process-limit warning, the
  `apply_patch`-via-`exec_command` warning, and the model-mismatch
  high-risk cyber fallback warning. Those warnings are no longer produced
  as conversation history items, but when old sessions compact they should
  still be recognized as injected context rather than preserved as real
  user turns.
  
  ## What changed
  
  - Removed `record_model_warning` and the production paths that emitted
  these warning messages into conversation history.
  - Added `LegacyUnifiedExecProcessLimitWarning`,
  `LegacyApplyPatchExecCommandWarning`, and `LegacyModelMismatchWarning`
  contextual fragments that are used only for matching old persisted
  messages.
  - Registered the legacy fragments with contextual user message detection
  so compaction filters them through the existing fragment path.
  - Added focused compaction coverage for old warning messages being
  dropped during compacted-history processing.
  
  ## Testing
  
  - `cargo test -p codex-core warning`
  - `just fix -p codex-core`

  pakrym-oai · 2026-05-11 19:51:51 -07:00
  79c65f816c
• Omit service_tier from remote /responses/compact requests under API auth (#21676)

  ## Summary
  
  API-key-auth remote compaction requests should not inherit
  `service_tier` from normal `/responses` turns. This path needs to match
  API auth expectations, while ChatGPT-auth remote compaction should keep
  reusing the shared request fields that still apply there.
  
  This change keeps the decision inline in
  `codex-rs/core/src/compact_remote.rs` only. Under API key auth, the
  classic remote `/responses/compact` path now omits `service_tier`; under
  ChatGPT auth, it keeps reusing the configured tier.
  `codex-rs/core/src/compact_remote_v2.rs` is unchanged. The remote
  compaction parity coverage and snapshots were updated to assert the
  API-key omission and preserve the ChatGPT-auth behavior.
  
  ## Testing
  
  - Updated remote compaction parity coverage in
  `codex-rs/core/tests/suite/compact_remote.rs` and the corresponding
  snapshots.

  Ahmed Ibrahim · 2026-05-08 11:15:14 +03:00
  71d80f9a14
• Add compact lifecycle hooks (started by vincentkoc - external contrib) (#19905)

  Based on work from Vincent K -
  https://github.com/openai/codex/pull/19060
  
  <img width="1836" height="642" alt="CleanShot 2026-04-29 at 20 47 40@2x"
  src="https://github.com/user-attachments/assets/b647bb89-65fe-40c8-80b0-7a6b7c984634"
  />
  
  ## Why
  
  Compaction rewrites the conversation context that future model turns
  receive, but hooks currently have no deterministic lifecycle point
  around that rewrite. This adds compact lifecycle hooks so users can
  audit manual and automatic compaction, surface hook messages in the UI,
  and run post-compaction follow-up without overloading tool or prompt
  hooks.
  
  ## What Changed
  
  - Added `PreCompact` and `PostCompact` hook events across hook config,
  discovery, dispatch, generated schemas, app-server notifications,
  analytics, and TUI hook rendering.
  - Added trigger matching for compact hooks with the documented `manual`
  and `auto` matcher values.
  - Wired `PreCompact` before both local and remote compaction, and
  `PostCompact` after successful local or remote compaction.
  - Kept compact hook command input to lifecycle metadata: session id,
  Codex turn id, transcript path, cwd, hook event name, model, and
  trigger.
  - Made compact stdout handling consistent with other hooks: plain stdout
  is ignored as debug output, while malformed JSON-looking stdout is
  reported as failed hook output.
  - Added integration coverage for compact hook dispatch, trigger
  matching, post-compact execution, and the audited behavior that
  `decision:"block"` does not block compaction.
  
  ## Out of Scope
  
  - Hook-specific compaction blocking is not implemented;
  `decision:"block"` and exit-code-2 blocking semantics are intentionally
  unsupported for `PreCompact`.
  - Custom compaction instructions are not exposed to compact hooks in
  this PR.
  - Compact summaries, summary character counts, and summary previews are
  not exposed to compact hooks in this PR.
  
  ## Verification
  
  - `cargo test -p codex-hooks`
  - `cargo test -p codex-core
  manual_pre_compact_block_decision_does_not_block_compaction`
  - `cargo test -p codex-app-server hooks_list`
  - `cargo test -p codex-core config_schema_matches_fixture`
  - `cargo test -p codex-tui hooks_browser`
  
  ## Docs
  
  The developer documentation for Codex hooks should be updated alongside
  this feature to document `PreCompact` and `PostCompact`, the
  `manual`/`auto` matcher values, and the compact hook payload fields.
  
  ---------
  
  Co-authored-by: Vincent Koc <vincentkoc@ieee.org>

  Andrei Eternal · 2026-05-06 18:08:31 -07:00
  527d52df03
• 2- Use string service tiers in session protocol (#20971)

  ## Summary
  - break service tier session/op/app-server protocol fields from the
  closed enum to string tier ids
  - send the service tier string directly through model requests, prewarm,
  compaction, memories, and TUI/app-server turn starts
  - regenerate app-server protocol JSON/TypeScript schemas, removing the
  standalone ServiceTier TS enum
  
  ## Verification
  - just fmt
  - cargo check -p codex-core -p codex-app-server -p codex-tui
  - just write-app-server-schema
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-05-06 18:00:21 +03:00
  be1d3cff93
• Propagate cache key and service tiers in compact (#21249)

  ## Why
  
  `/responses/compact` should preserve the request-affinity fields that
  apply to the active auth mode. ChatGPT-auth compact requests need the
  effective `service_tier`, and compact requests for every auth mode need
  the stable `prompt_cache_key`, so compaction does not quietly lose
  routing or cache behavior that normal sampling already has.
  
  This follows the request-parity direction from #20719, but keeps the net
  change focused on the compact payload fields needed here.
  
  ## What changed
  
  - Add `service_tier` and `prompt_cache_key` to the compact endpoint
  input payload.
  - Build the remote compact payload from the existing responses request
  builder output so `Fast` still maps to `priority` when compact sends a
  service tier.
  - Pass the turn service tier into remote compaction, but only include it
  in compact payloads for ChatGPT-backed auth.
  - Keep `prompt_cache_key` on compact payloads for all auth modes.
  - Add request-body diff snapshot coverage in
  `core/tests/suite/compact_remote.rs` for:
  - API-key auth reusing `prompt_cache_key` while omitting `service_tier`
  even when `Fast` is configured.
    - ChatGPT auth reusing both `service_tier` and `prompt_cache_key`.
  - Drive the snapshot coverage through five varied turns: plain text,
  multi-part text, tool-call continuation, image+text input, local-shell
  continuation, and final-turn reasoning output.
  
  ## Verification
  
  - Added insta snapshots for compact request-body parity against the last
  normal `/responses` request after five varied turns.
  - Not run locally per repo guidance; relying on GitHub CI for test
  execution.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-05-06 13:38:43 +03:00
  5d6f23a27b
• feat: add remote compaction v2 Responses client path (#20773)

  ## Why
  
  This adds the `remote_compaction_v2` client path so remote compaction
  can run through the normal Responses stream and install a
  `context_compaction` item that trigger a compaction.
  
  The goal is to migrate some of the compaction logic on the client side
  
  We keeps the v2 transport behind a feature flag while letting follow-up
  requests reuse the compacted context instead of falling back to the
  legacy compaction item shape.
  
  ## What changed
  
  - add `ResponseItem::ContextCompaction` and refresh the generated
  app-server / schema / TypeScript fixtures that expose response items on
  the wire
  - add `core/src/compact_remote_v2.rs` to send compaction through the
  standard streamed Responses client, require exactly one
  `context_compaction` output item, and install that item into compacted
  history
  - route manual compact and auto-compaction through the v2 path when
  `remote_compaction_v2` is enabled, while keeping the existing remote
  compaction path as the fallback
  - preserve the new item type across history retention, follow-up request
  construction, telemetry, rollout persistence, and rollout-trace
  normalization
  - add targeted coverage for the feature flag, `context_compaction`
  serialization, rollout-trace normalization, and remote-compaction
  follow-up behavior
  
  ## Verification
  
  - added protocol tests for `context_compaction`
  serialization/deserialization in `protocol/src/models.rs`
  - added rollout-trace coverage for `context_compaction` normalization in
  `rollout-trace/src/reducer/conversation_tests.rs`
  - added remote compaction integration coverage for v2 follow-up reuse
  and mixed compaction output streams in
  `core/tests/suite/compact_remote.rs`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-05-04 14:15:01 +02:00
  d927f61208
• Remove ghost snapshots (#19481)

  ## Summary
  - Remove `ghost_snapshot` / `GhostCommit` from the Responses API surface
  and generated SDK/schema artifacts.
  - Keep legacy config loading compatible, but make undo a no-op that
  reports the feature is unavailable.
  - Clean up core history, compaction, telemetry, rollout, and tests to
  stop carrying ghost snapshot items.
  
  ## Testing
  - Unit tests passed for `codex-protocol`, `codex-core` targeted undo and
  compaction flows, `codex-rollout`, and `codex-app-server-protocol`.
  - Regenerated config and app-server schemas plus Python SDK artifacts
  and verified they match the checked-in outputs.

  pakrym-oai · 2026-04-27 18:48:57 -07:00
  4e05f3053c
• [rollout_trace] Add debug trace reduction command (#18880)

  ## Summary
  
  Adds the debug CLI entry point for reducing recorded rollout traces.
  This gives developers a direct way to inspect whether the emitted trace
  stream reduces into the expected conversation/runtime model.
  
  ## Stack
  
  This is PR 5/5 in the rollout trace stack.
  
  - [#18876](https://github.com/openai/codex/pull/18876): Add rollout
  trace crate
  - [#18877](https://github.com/openai/codex/pull/18877): Record core
  session rollout traces
  - [#18878](https://github.com/openai/codex/pull/18878): Trace tool and
  code-mode boundaries
  - [#18879](https://github.com/openai/codex/pull/18879): Trace sessions
  and multi-agent edges
  - [#18880](https://github.com/openai/codex/pull/18880): Add debug trace
  reduction command
  
  ## Review Notes
  
  This PR is intentionally last: it depends on the trace crate, core
  recorder, runtime/tool events, and session/agent edge data all existing.
  The command should remain a debug/developer tool and avoid adding new
  runtime behavior.
  
  The useful review question is whether the CLI exposes the reducer in the
  smallest practical way for local inspection without turning the debug
  command into a supported user-facing workflow.

  cassirer-openai · 2026-04-24 01:56:48 +00:00
  e3c8720a99
• [rollout_trace] Record core session rollout traces (#18877)

  ## Summary
  
  Wires rollout trace recording into `codex-core` session and turn
  execution. This records the core model request/response, compaction, and
  session lifecycle boundaries needed for replay without yet tracing every
  nested runtime/tool boundary.
  
  ## Stack
  
  This is PR 2/5 in the rollout trace stack.
  
  - [#18876](https://github.com/openai/codex/pull/18876): Add rollout
  trace crate
  - [#18877](https://github.com/openai/codex/pull/18877): Record core
  session rollout traces
  - [#18878](https://github.com/openai/codex/pull/18878): Trace tool and
  code-mode boundaries
  - [#18879](https://github.com/openai/codex/pull/18879): Trace sessions
  and multi-agent edges
  - [#18880](https://github.com/openai/codex/pull/18880): Add debug trace
  reduction command
  
  ## Review Notes
  
  This layer is the first live integration point. The important review
  question is whether trace recording is isolated from normal session
  behavior: trace failures should not become user-visible execution
  failures, and recording should preserve the existing turn/session
  lifecycle semantics.
  
  The PR depends on the reducer/data model from the first stack entry and
  only introduces the core recorder surface that later PRs use for richer
  runtime and relationship events.

  cassirer-openai · 2026-04-22 17:00:48 +00:00
  f67383bcba
• Allow guardian bare allow output (#18797)

  ## Summary
  
  Allow guardian to skip other fields and output only
  `{"outcome":"allow"}` when the command is low risk.
  This change lets guardian reviews use a non-strict text format while
  keeping the JSON schema itself as plain user-visible schema data, so
  transport strictness is carried out-of-band instead of through a schema
  marker key.
  
  ## What changed
  
  - Add an explicit `output_schema_strict` flag to model prompts and pass
  it into `codex-api` text formatting.
  - Set guardian reviewer prompts to non-strict schema validation while
  preserving strict-by-default behavior for normal callers.
  - Update the guardian output contract so definitely-low-risk decisions
  may return only `{"outcome":"allow"}`.
  - Treat bare allow responses as low-risk approvals in the guardian
  parser.
  - Add tests and snapshots covering the non-strict guardian request and
  optional guardian output fields.
  
  ## Verification
  
  - `cargo test -p codex-core guardian::tests::guardian`
  - `cargo test -p codex-core guardian::tests::`
  - `cargo test -p codex-core client_common::tests::`
  - `cargo test -p codex-protocol
  user_input_serialization_includes_final_output_json_schema`
  - `cargo test -p codex-api`
  - `git diff --check`
  
  Note: `cargo test -p codex-core` was also attempted, but this desktop
  environment injects ambient config/proxy state that causes unrelated
  config/session tests expecting pristine defaults to fail.
  
  ---------
  
  Co-authored-by: Dylan Hurd <dylan.hurd@openai.com>
  Co-authored-by: Codex <noreply@openai.com>

  maja-openai · 2026-04-21 15:37:12 -07:00
  ef00014a46
• Move codex module under session (#18249)

  ## Summary
  - rename the core codex module root to session/mod.rs without using
  #[path]
  - move the codex module directory and tests under core/src/session
  - remove session/mod.rs reexports so call sites use explicit child
  module paths
  
  ## Testing
  - cargo test -p codex-core --lib
  - cargo check -p codex-core --tests
  - just fmt
  - just fix -p codex-core
  - git diff --check

  pakrym-oai · 2026-04-17 16:18:53 +00:00
  71e4c6fa17
• [codex-analytics] add compaction analytics event (#17155)

  - event for compaction analytics
  - introduces thread-connection and thread metadata caches for data
  denormalization, expected to be useful for denormalization onto core
  emitted events in general
  - threads analytics event client into core (mirrors approved
  implementation in #16640)
  - denormalizes key thread metadata: thread_source, subagent_source,
  parent_thread_id, as well as app-server client and runtime metadata)
  - compaction strategy defaults to memento, forward compatible with
  expected prefill_compaction strategy
  
  1. Manual standalone compact, local
  `INFO | 2026-04-09 17:35:50 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d0-5cfb-70c0-bef9-165c3bf9b2df', 'turn_id':
  '019d74d0-d7f6-7c81-acc6-aae2030243d6', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'manual', 'reason':
  'user_requested', 'implementation': 'responses', 'phase':
  'standalone_turn', 'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 20170, 'active_context_tokens_after':
  4830, 'started_at': 1775781337, 'completed_at': 1775781350,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 13524} | `
  
  2. Auto pre-turn compact, local
  `INFO | 2026-04-09 17:37:30 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d2-45ef-71d1-9c93-23cc0c13d988', 'turn_id':
  '019d74d2-7b42-7372-9f0e-c0da3f352328', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'auto', 'reason':
  'context_limit', 'implementation': 'responses', 'phase': 'pre_turn',
  'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 20063, 'active_context_tokens_after':
  4822, 'started_at': 1775781444, 'completed_at': 1775781449,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 5497} | `
  
  3. Auto mid-turn compact, local
  `INFO | 2026-04-09 17:38:28 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d3-212f-7a20-8c0a-4816a978675e', 'turn_id':
  '019d74d3-3ee1-7462-89f6-2ffbeefcd5e3', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'auto', 'reason':
  'context_limit', 'implementation': 'responses', 'phase': 'mid_turn',
  'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 20325, 'active_context_tokens_after':
  14641, 'started_at': 1775781500, 'completed_at': 1775781508,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 7507} | `
  
  4. Remote /responses/compact, manual standalone
  `INFO | 2026-04-09 17:40:20 | codex_backend.routers.analytics_events |
  analytics_events.track_analytics_events:526 | Tracked
  codex_compaction_event event params={'thread_id':
  '019d74d4-7a11-78a1-89f7-0535a1149416', 'turn_id':
  '019d74d4-e087-7183-9c20-b1e40b7578c0', 'product_surface': 'codex',
  'app_server_client': {'product_client_id': 'CODEX_CLI', 'client_name':
  'codex-tui', 'client_version': '0.0.0', 'rpc_transport': 'in_process',
  'experimental_api_enabled': True}, 'runtime': {'codex_rs_version':
  '0.0.0', 'runtime_os': 'macos', 'runtime_os_version': '26.4.0',
  'runtime_arch': 'aarch64'}, 'trigger': 'manual', 'reason':
  'user_requested', 'implementation': 'responses_compact', 'phase':
  'standalone_turn', 'strategy': 'memento', 'status': 'completed',
  'active_context_tokens_before': 23461, 'active_context_tokens_after':
  6171, 'started_at': 1775781601, 'completed_at': 1775781620,
  'thread_source': 'user', 'subagent_source': None, 'parent_thread_id':
  None, 'error': None, 'duration_ms': 18971} | `

  rhan-oai · 2026-04-10 13:03:54 -07:00
  5779be314a
• fix(app-server) revert null instructions changes (#17047)
  Dylan Hurd · 2026-04-07 15:18:34 -07:00
  6c36e7d688
• Honor null thread instructions (#16964)

  - Treat explicit null thread instructions as a blank-slate override
  while preserving omitted-field fallback behavior.
  - Preserve null through rollout resume/fork and keep explicit empty
  strings distinct.
  - Add app-server v2 start/fork coverage for the tri-state instruction
  params.

  Ahmed Ibrahim · 2026-04-07 04:10:19 +00:00
  24c598e8a9
• [codex-analytics] add protocol-native turn timestamps (#16638)

  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16638).
  * #16870
  * #16706
  * #16659
  * #16641
  * #16640
  * __->__ #16638

  rhan-oai · 2026-04-06 16:22:59 -07:00
  756c45ec61
• remove temporary ownership re-exports (#16626)

  Stacked on #16508.
  
  This removes the temporary `codex-core` / `codex-login` re-export shims
  from the ownership split and rewrites callsites to import directly from
  `codex-model-provider-info`, `codex-models-manager`, `codex-api`,
  `codex-protocol`, `codex-feedback`, and `codex-response-debug-context`.
  
  No behavior change intended; this is the mechanical import cleanup layer
  split out from the ownership move.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-03 00:33:34 -07:00
  af8a9d2d2b
• core: remove cross-crate re-exports from lib.rs (#16512)

  ## Why
  
  `codex-core` was re-exporting APIs owned by sibling `codex-*` crates,
  which made downstream crates depend on `codex-core` as a proxy module
  instead of the actual owner crate.
  
  Removing those forwards makes crate boundaries explicit and lets leaf
  crates drop unnecessary `codex-core` dependencies. In this PR, this
  reduces the dependency on `codex-core` to `codex-login` in the following
  files:
  
  ```
  codex-rs/backend-client/Cargo.toml
  codex-rs/mcp-server/tests/common/Cargo.toml
  ```
  
  ## What
  
  - Remove `codex-rs/core/src/lib.rs` re-exports for symbols owned by
  `codex-login`, `codex-mcp`, `codex-rollout`, `codex-analytics`,
  `codex-protocol`, `codex-shell-command`, `codex-sandboxing`,
  `codex-tools`, and `codex-utils-path`.
  - Delete the `default_client` forwarding shim in `codex-rs/core`.
  - Update in-crate and downstream callsites to import directly from the
  owning `codex-*` crate.
  - Add direct Cargo dependencies where callsites now target the owner
  crate, and remove `codex-core` from `codex-rs/backend-client`.

  Michael Bolin · 2026-04-01 23:06:24 -07:00
  aa2403e2eb
• [hooks] use a user message > developer message for prompt continuation (#14867)

  ## Summary
  
  Persist Stop-hook continuation prompts as `user` messages instead of
  hidden `developer` messages + some requested integration tests
  
  This is a followup to @pakrym 's comment in
  https://github.com/openai/codex/pull/14532 to make sure stop-block
  continuation prompts match training for turn loops
  
  - Stop continuation now writes `<hook_prompt hook_run_id="...">stop
  hook's user prompt<hook_prompt>`
  - Introduces quick-xml dependency, though we already indirectly depended
  on it anyway via syntect
  - This PR only has about 500 lines of actual logic changes, the rest is
  tests/schema
  
  ## Testing
  
  Example run (with a sessionstart hook and 3 stop hooks) - this shows
  context added by session start, then two stop hooks sending their own
  additional prompts in a new turn. The model responds with a single
  message addressing both. Then when that turn ends, the hooks detect that
  they just ran using `stop_hook_active` and decide not to infinite loop
  
  test files for this (unzip, move codex -> .codex):
  [codex.zip](https://github.com/user-attachments/files/26075806/codex.zip)
  
  ```
  › cats
  
  
  • Running SessionStart hook: lighting the observatory
  
  SessionStart hook (completed)
    warning: Hi, I'm a session start hook for wizard-tower (startup).
    hook context: A wimboltine stonpet is an exotic cuisine from hyperspace
  
  • Cats are tiny zen wizards, my friend: equal parts nap, mystery, and chaos. If you want, we can talk
    cat facts, cat breeds, cat names, or build something cat-themed in this repo.
  
  • Running Stop hook: checking the tower wards
  
  • Running Stop hook: sacking the guards
  
  • Running Stop hook: hiring the guards
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook reviewed the completed reply (177 chars).
  
  Stop hook (blocked)
    warning: Wizard Tower Stop hook continuing conversation
    feedback: cook the stonpet
  
  Stop hook (blocked)
    warning: Wizard Tower Stop hook continuing conversation
    feedback: eat the cooked stonpet
  
  • Stonpet’s cooked, aloha style: flash-seared over a blue quasiflame, glazed with nebula salt, and
    rested until the hyperspace juices settle.
  
    Now we eat with gratitude, my friend. One mindful bite in, and the flavor is pure cosmic surf:
    smoky, bright, and totally out of this dimension.
  
  • Running Stop hook: checking the tower wards
  
  • Running Stop hook: sacking the guards
  
  • Running Stop hook: hiring the guards
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook reviewed the completed reply (285 chars).
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook saw a second pass and stayed calm to avoid a loop.
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook saw a second pass and stayed calm to avoid a loop.
  ```

  Andrei Eternal · 2026-03-19 10:53:08 -07:00
  267499bed8
• Apply argument comment lint across codex-rs (#14652)

  ## Why
  
  Once the repo-local lint exists, `codex-rs` needs to follow the
  checked-in convention and CI needs to keep it from drifting. This commit
  applies the fallback `/*param*/` style consistently across existing
  positional literal call sites without changing those APIs.
  
  The longer-term preference is still to avoid APIs that require comments
  by choosing clearer parameter types and call shapes. This PR is
  intentionally the mechanical follow-through for the places where the
  existing signatures stay in place.
  
  After rebasing onto newer `main`, the rollout also had to cover newly
  introduced `tui_app_server` call sites. That made it clear the first cut
  of the CI job was too expensive for the common path: it was spending
  almost as much time installing `cargo-dylint` and re-testing the lint
  crate as a representative test job spends running product tests. The CI
  update keeps the full workspace enforcement but trims that extra
  overhead from ordinary `codex-rs` PRs.
  
  ## What changed
  
  - keep a dedicated `argument_comment_lint` job in `rust-ci`
  - mechanically annotate remaining opaque positional literals across
  `codex-rs` with exact `/*param*/` comments, including the rebased
  `tui_app_server` call sites that now fall under the lint
  - keep the checked-in style aligned with the lint policy by using
  `/*param*/` and leaving string and char literals uncommented
  - cache `cargo-dylint`, `dylint-link`, and the relevant Cargo
  registry/git metadata in the lint job
  - split changed-path detection so the lint crate's own `cargo test` step
  runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes
  - continue to run the repo wrapper over the `codex-rs` workspace, so
  product-code enforcement is unchanged
  
  Most of the code changes in this commit are intentionally mechanical
  comment rewrites or insertions driven by the lint itself.
  
  ## Verification
  
  - `./tools/argument-comment-lint/run.sh --workspace`
  - `cargo test -p codex-tui-app-server -p codex-tui`
  - parsed `.github/workflows/rust-ci.yml` locally with PyYAML
  
  ---
  
  * -> #14652
  * #14651

  Michael Bolin · 2026-03-16 16:48:15 -07:00
  b77fe8fefe
• Add code_mode_only feature (#14617)

  Summary
  - add the code_mode_only feature flag/config schema and wire its
  dependency on code_mode
  - update code mode tool descriptions to list nested tools with detailed
  headers
  - restrict available tools for prompt and exec descriptions when
  code_mode_only is enabled and test the behavior
  
  Testing
  - Not run (not requested)

  pakrym-oai · 2026-03-13 13:30:19 -07:00
  477a2dd345
• Reapply "Pass more params to compaction" (#14298) (#14521)

  This reverts commit 8af97ce4b08fdedadc6037851b5e20cc653e9536.
  
  Confirmed that this runs locally without the previous issues with tool
  use

  Rasmus Rygaard · 2026-03-12 23:27:21 +00:00
  53d5972226
• feat: search_tool migrate to bring you own tool of Responses API (#14274)

  ## Why
  
  to support a new bring your own search tool in Responses
  API(https://developers.openai.com/api/docs/guides/tools-tool-search#client-executed-tool-search)
  we migrating our bm25 search tool to use official way to execute search
  on client and communicate additional tools to the model.
  
  ## What
  - replace the legacy `search_tool_bm25` flow with client-executed
  `tool_search`
  - add protocol, SSE, history, and normalization support for
  `tool_search_call` and `tool_search_output`
  - return namespaced Codex Apps search results and wire namespaced
  follow-up tool calls back into MCP dispatch

  Anton Panasenko · 2026-03-11 17:51:51 -07:00
  77b0c75267
• Revert "Pass more params to compaction" (#14298)
  Rasmus Rygaard · 2026-03-11 12:33:10 -07:00
  7f22329389
• Pass more params to compaction (#14247)

  Pass more params to /compact. This should give us parity with the
  /responses endpoint to improve caching.
  
  I'm torn about the MCP await. Blocking will give us parity but it seems
  like we explicitly don't block on MCPs. Happy either way

  Rasmus Rygaard · 2026-03-11 12:33:09 -07:00
  2621ba17e3
• chore(otel): rename OtelManager to SessionTelemetry (#13808)

  ## Summary
  This is a purely mechanical refactor of `OtelManager` ->
  `SessionTelemetry` to better convey what the struct is doing. No
  behavior change.
  
  ## Why
  
  `OtelManager` ended up sounding much broader than what this type
  actually does. It doesn't manage OTEL globally; it's the session-scoped
  telemetry surface for emitting log/trace events and recording metrics
  with consistent session metadata (`app_version`, `model`, `slug`,
  `originator`, etc.).
  
  `SessionTelemetry` is a more accurate name, and updating the call sites
  makes that boundary a lot easier to follow.
  
  ## Validation
  
  - `just fmt`
  - `cargo test -p codex-otel`
  - `cargo test -p codex-core`

  Owen Lin · 2026-03-06 16:23:30 -08:00
  289ed549cf
• image-gen-core (#13290)

  Core tool-calling for image-gen, handles requesting and receiving logic
  for images using response API

  Won Park · 2026-03-03 23:11:28 -08:00
  fa2306b303
• Record realtime close marker on replacement (#13058)

  ## Summary
  - record a realtime close developer message when a new realtime session
  replaces an active one
  - assert the replacement marker through the mocked responses request
  path
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>
  Co-authored-by: Charles Cunningham <ccunningham@openai.com>

  Ahmed Ibrahim · 2026-03-01 13:54:12 -08:00
  0aeb55bf08
• Unify rollout reconstruction with resume/fork TurnContext hydration (#12612)

  ## Summary
  
  This PR unifies rollout history reconstruction and resume/fork metadata
  hydration under a single `Session::reconstruct_history_from_rollout`
  implementation.
  
  The key change from main is that replay metadata now comes from the same
  reconstruction pass that rebuilds model-visible history, instead of
  doing a second bespoke rollout scan to recover `previous_model` /
  `reference_context_item`.
  
  ## What Changed
  
  ### Unified reconstruction output
  
  `reconstruct_history_from_rollout` now returns a single
  `RolloutReconstruction` bundle containing:
  
  - rebuilt `history`
  - `previous_model`
  - `reference_context_item`
  
  Resume and fork both consume that shared output directly.
  
  ### Reverse replay core
  
  The reconstruction logic moved into
  `codex-rs/core/src/codex/rollout_reconstruction.rs` and now scans
  rollout items newest-to-oldest.
  
  That reverse pass:
  
  - derives `previous_model`
  - derives whether `reference_context_item` is preserved or cleared
  - stops early once it has both resume metadata and a surviving
  `replacement_history` checkpoint
  
  History materialization is still bridged eagerly for now by replaying
  only the surviving suffix forward, which keeps the history result stable
  while moving the control flow toward the future lazy reverse loader
  design.
  
  ### Removed bespoke context lookup
  
  This deletes `last_rollout_regular_turn_context_lookup` and its separate
  compaction-aware scan.
  
  The previous model / baseline metadata is now computed from the same
  replay state that rebuilds history, so resume/fork cannot drift from the
  reconstructed transcript view.
  
  ### `TurnContextItem` persistence contract
  
  `TurnContextItem` is now treated as the replay source of truth for
  durable model-visible baselines.
  
  This PR keeps the following contract explicit:
  
  - persist `TurnContextItem` for the first real user turn so resume can
  recover `previous_model`
  - persist it for later turns that emit model-visible context updates
  - if mid-turn compaction reinjects full initial context into replacement
  history, persist a fresh `TurnContextItem` after `Compacted` so
  resume/fork can re-establish the baseline from the rewritten history
  - do not treat manual compaction or pre-sampling compaction as creating
  a new durable baseline on their own
  
  ## Behavior Preserved
  
  - rollback replay stays aligned with `drop_last_n_user_turns`
  - rollback skips only user turns
  - incomplete active user turns are dropped before older finalized turns
  when rollback applies
  - unmatched aborts do not consume the current active turn
  - missing abort IDs still conservatively clear stale compaction state
  - compaction clears `reference_context_item` until a later
  `TurnContextItem` re-establishes it
  - `previous_model` still comes from the newest surviving user turn that
  established one
  
  ## Tests
  
  Targeted validation run for the current branch shape:
  
  - `cd codex-rs && cargo test -p codex-core --lib
  codex::rollout_reconstruction_tests -- --nocapture`
  - `cd codex-rs && just fmt`
  
  The branch also extracts the rollout reconstruction tests into
  `codex-rs/core/src/codex/rollout_reconstruction_tests.rs` so this logic
  has a dedicated home instead of living inline in `codex.rs`.

  Charley Cunningham · 2026-02-27 13:50:45 -08:00
  695957a348
• feat: add post-compaction sub-agent infos (#12774)

  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-02-26 18:55:34 +00:00
  3404ecff15
• core: bundle settings diff updates into one dev/user envelope (#12417)

  ## Summary
  - bundle contextual prompt injection into at most one developer message
  plus one contextual user message in both:
    - per-turn settings updates
    - initial context insertion
  - preserve `<model_switch>` across compaction by rebuilding it through
  canonical initial-context injection, instead of relying on
  strip/reattach hacks
  - centralize contextual user fragment detection in one shared definition
  table and reuse it for parsing/compaction logic
  - keep `AGENTS.md` in its natural serialized format:
    - `# AGENTS.md instructions for {dirname}`
    - `<INSTRUCTIONS>...</INSTRUCTIONS>`
  - simplify related tests/helpers and accept the expected snapshot/layout
  updates from bundled multi-part messages
  
  ## Why
  The goal is to converge toward a simpler, more intentional prompt shape
  where contextual updates are consistently represented as one developer
  envelope plus one contextual user envelope, while keeping parsing and
  compaction behavior aligned with that representation.
  
  ## Notable details
  - the temporary `SettingsUpdateEnvelope` wrapper was removed; these
  paths now return `Vec<ResponseItem>` directly
  - local/remote compaction no longer rely on model-switch strip/restore
  helpers
  - contextual user detection is now driven by shared fragment definitions
  instead of ad hoc matcher assembly
  - AGENTS/user instructions are still the same logical context; only the
  synthetic `<user_instructions>` wrapper was replaced by the natural
  AGENTS text format
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-app-server
  codex_message_processor::tests::extract_conversation_summary_prefers_plain_user_messages
  -- --exact`
  - `cargo test -p codex-core
  compact::tests::collect_user_messages_filters_session_prefix_entries
  --lib -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact::snapshot_request_shape_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact_remote::snapshot_request_shape_remote_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_apps_guidance_as_developer_message_when_enabled'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_developer_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_user_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::resume_includes_initial_messages_and_sends_prior_items'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::review::review_input_isolated_from_parent_history' -- --exact`
  - `cargo test -p codex-exec --test all
  'suite::resume::exec_resume_last_respects_cwd_filter_and_all_flag' --
  --exact`
  - `cargo test -p core_test_support
  context_snapshot::tests::full_text_mode_preserves_unredacted_text --
  --exact`
  
  ## Notes
  - I also ran several targeted `compact`, `compact_remote`,
  `prompt_caching`, `model_visible_layout`, and `event_mapping` tests
  while iterating on prompt-shape changes.
  - I have not claimed a clean full-workspace `cargo test` from this
  environment because local sandbox/resource conditions have previously
  produced unrelated failures in large workspace runs.

  Charley Cunningham · 2026-02-26 00:12:08 -08:00
  07aefffb1f
• Fix compaction context reinjection and model baselines (#12252)

  ## Summary
  - move regular-turn context diff/full-context persistence into
  `run_turn` so pre-turn compaction runs before incoming context updates
  are recorded
  - after successful pre-turn compaction, rely on a cleared
  `reference_context_item` to trigger full context reinjection on the
  follow-up regular turn (manual `/compact` keeps replacement history
  summary-only and also clears the baseline)
  - preserve `<model_switch>` when full context is reinjected, and inject
  it *before* the rest of the full-context items
  - scope `reference_context_item` and `previous_model` to regular user
  turns only so standalone tasks (`/compact`, shell, review, undo) cannot
  suppress future reinjection or `<model_switch>` behavior
  - make context-diff persistence + `reference_context_item` updates
  explicit in the regular-turn path, with clearer docs/comments around the
  invariant
  - stop persisting local `/compact` `RolloutItem::TurnContext` snapshots
  (only regular turns persist `TurnContextItem` now)
  - simplify resume/fork previous-model/reference-baseline hydration by
  looking up the last surviving turn context from rollout lifecycle
  events, including rollback and compaction-crossing handling
  - remove the legacy fallback that guessed from bare `TurnContext`
  rollouts without lifecycle events
  - update compaction/remote-compaction/model-visible snapshots and
  compact test assertions (including remote compaction mock response
  shape)
  
  ## Why
  We were persisting incoming context items before spawning the regular
  turn task, which let pre-turn compaction requests accidentally include
  incoming context diffs without the new user message. Fixing that exposed
  follow-on baseline issues around `/compact`, resume/fork, and standalone
  tasks that could cause duplicate context injection or suppress
  `<model_switch>` instructions.
  
  This PR re-centers the invariants around regular turns:
  - regular turns persist model-visible context diffs/full reinjection and
  update the `reference_context_item`
  - standalone tasks do not advance those regular-turn baselines
  - compaction clears the baseline when replacement history may have
  stripped the referenced context diffs
  
  ## Follow-ups (TODOs left in code)
  - `TODO(ccunningham)`: fix rollback/backtracking baseline handling more
  comprehensively
  - `TODO(ccunningham)`: include pending incoming context items in
  pre-turn compaction threshold estimation
  - `TODO(ccunningham)`: inject updated personality spec alongside
  `<model_switch>` so some model-switch paths can avoid forced full
  reinjection
  - `TODO(ccunningham)`: review task turn lifecycle
  (`TurnStarted`/`TurnComplete`) behavior and emit task-start context
  diffs for task types that should have them (excluding `/compact`)
  
  ## Validation
  - `just fmt`
  - CI should cover the updated compaction/resume/model-visible snapshot
  expectations and rollout-hydration behavior
  - I did **not** rerun the full local test suite after the latest
  resume-lookup / rollout-persistence simplifications

  Charley Cunningham · 2026-02-20 23:13:08 -08:00
  bb0ac5be70
• Handle model-switch base instructions after compaction (#11659)

  Strip trailing <model_switch> during model-switch compaction request,
  and append <model_switch> after model switch compaction

  Charley Cunningham · 2026-02-13 19:02:53 -08:00
  67e577da53
• chore: persist turn_id in rollout session and make turn_id uuid based (#11246)

  Problem:
  1. turn id is constructed in-memory;
  2. on resuming threads, turn_id might not be unique;
  3. client cannot no the boundary of a turn from rollout files easily.
  
  This PR does three things:
  1. persist `task_started` and `task_complete` events;
  1. persist `turn_id` in rollout turn events;
  5. generate turn_id as unique uuids instead of incrementing it in
  memory.
  
  This helps us resolve the issue of clients wanting to have unique turn
  ids for resuming a thread, and knowing the boundry of each turn in
  rollout files.
  
  example debug logs
  ```
  2026-02-11T00:32:10.746876Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=8 turn=Turn { id: "019c4a07-d809-74c3-bc4b-fd9618487b4b", items: [UserMessage { id: "item-24", content: [Text { text: "hi", text_elements: [] }] }, AgentMessage { id: "item-25", text: "Hi. I’m in the workspace with your current changes loaded and ready. Send the next task and I’ll execute it end-to-end." }], status: Completed, error: None }
  2026-02-11T00:32:10.746888Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=9 turn=Turn { id: "019c4a18-1004-76c0-a0fb-a77610f6a9b8", items: [UserMessage { id: "item-26", content: [Text { text: "hello", text_elements: [] }] }, AgentMessage { id: "item-27", text: "Hello. Ready for the next change in `codex-rs`; I can continue from the current in-progress diff or start a new task." }], status: Completed, error: None }
  2026-02-11T00:32:10.746899Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=10 turn=Turn { id: "019c4a19-41f0-7db0-ad78-74f1503baeb8", items: [UserMessage { id: "item-28", content: [Text { text: "hello", text_elements: [] }] }, AgentMessage { id: "item-29", text: "Hello. Send the specific change you want in `codex-rs`, and I’ll implement it and run the required checks." }], status: Completed, error: None }
  ```
  
  backward compatibility:
  if you try to resume an old session without task_started and
  task_complete event populated, the following happens:
  - If you resume and do nothing: those reconstructed historical IDs can
  differ next time you resume.
  - If you resume and send a new turn: the new turn gets a fresh UUID from
  live submission flow and is persisted, so that new turn’s ID is stable
  on later resumes.
  I think this behavior is fine, because we only care about deterministic
  turn id once a turn is triggered.

  Celia Chen · 2026-02-11 03:56:01 +00:00
  641d5268fa
• Strip unsupported images from prompt history to guard against model switch (#11349)

  - Make `ContextManager::for_prompt` modality-aware and strip input_image
  content when the active model is text-only.
  - Added a test for multi-model -> text-only model switch

  Ahmed Ibrahim · 2026-02-10 11:58:00 -08:00
  5e01450963
• core: add focused diagnostics for remote compaction overflows (#11133)

  ## Summary
  - add targeted remote-compaction failure diagnostics in compact_remote
  logging
  - log the specific values needed to explain overflow timing:
    - last_api_response_total_tokens
    - estimated_tokens_of_items_added_since_last_successful_api_response
    - estimated_bytes_of_items_added_since_last_successful_api_response
    - failing_compaction_request_body_bytes
  - simplify breakdown naming and remove
  last_api_response_total_bytes_estimate (it was an approximation and not
  useful for debugging)
  
  ## Why
  When compaction fails with context_length_exceeded, we need concrete,
  low-ambiguity numbers that map directly to:
  1) what the API most recently reported, and
  2) what local history added since then.
  
  This keeps the failure logs actionable without adding broad, noisy
  metrics.
  
  ## Testing
  - just fmt
  - cargo test -p codex-core

  Charley Cunningham · 2026-02-09 12:42:20 -08:00
  9450cd9ce5
• Treat compaction failure as failure state (#10927)

  - Return compaction errors from local and remote compaction flows.\n-
  Stop turns/tasks when auto-compaction fails instead of continuing
  execution.

  Ahmed Ibrahim · 2026-02-06 13:51:46 -08:00
  ba8b5d9018
• core: refresh developer instructions after compaction replacement history (#10574)

  ## Summary
  
  When replaying compacted history (especially `replacement_history` from
  remote compaction), we should not keep stale developer messages from
  older session state. This PR trims developer-
  role messages from compacted replacement history and reinjects fresh
  developer instructions derived from current turn/session state.
  
  This aligns compaction replay behavior with the intended "fresh
  instructions after summary" model.
  
  ## Problem
  
  Compaction replay had two paths:
  
  - `Compacted { replacement_history: None }`: rebuilt with fresh initial
  context
  - `Compacted { replacement_history: Some(...) }`: previously used raw
  replacement history as-is
  
  The second path could carry stale developer instructions
  (permissions/personality/collab-mode guidance) across session changes.
  
  ## What Changed
  
  ### 1) Added helper to refresh compacted developer instructions
  
  - **File:** `codex-rs/core/src/compact.rs`
  - **Function:** `refresh_compacted_developer_instructions(...)`
  
  Behavior:
  - remove all `ResponseItem::Message { role: "developer", .. }` from
  compacted history
  - append fresh developer messages from current
  `build_initial_context(...)`
  
  ### 2) Applied helper in remote compaction flow
  
  - **File:** `codex-rs/core/src/compact_remote.rs`
  - After receiving compact endpoint output, refresh developer
  instructions before replacing history and persisting
  `replacement_history`.
  
  ### 3) Applied helper while reconstructing history from rollout
  
  - **File:** `codex-rs/core/src/codex.rs`
  - In `reconstruct_history_from_rollout(...)`, when processing
  `Compacted` entries with `replacement_history`, refresh developer
  instructions instead of directly replacing with raw history.
  
  ## Non-Goals / Follow-up
  
  This PR does **not** address the existing first-turn-after-resume
  double-injection behavior.
  A follow-up PR will handle resume-time dedup/idempotence separately.
  
  If you want, I can also give you a shorter “squash-merge friendly”
  version of the description.
  
  ## Codex author
  `codex fork 019c25e6-706e-75d1-9198-688ec00a8256`

  Charley Cunningham · 2026-02-06 12:25:08 -08:00
  143daadb31
• Fix remote compaction estimator/payload instruction small mismatch (#10692)

  ## Summary
  This PR fixes a deterministic mismatch in remote compaction where
  pre-trim estimation and the `/v1/responses/compact` payload could use
  different base instructions.
  
  Before this change:
  - pre-trim estimation used model-derived instructions
  (`model_info.get_model_instructions(...)`)
  - compact payload used session base instructions
  (`sess.get_base_instructions()`)
  
  After this change:
  - remote pre-trim estimation and compact payload both use the same
  `BaseInstructions` instance from session state.
  
  ## Changes
  - Added a shared estimator entry point in `ContextManager`:
  - `estimate_token_count_with_base_instructions(&self, base_instructions:
  &BaseInstructions) -> Option<i64>`
  - Kept `estimate_token_count(&TurnContext)` as a thin wrapper that
  resolves model/personality instructions and delegates to the new helper.
  - Updated remote compaction flow to fetch base instructions once and
  reuse it for both:
    - trim preflight estimation
    - compact request payload construction
  - Added regression coverage for parity and behavior:
    - unit test verifying explicit-base estimator behavior
  - integration test proving remote compaction uses session override
  instructions and trims accordingly
  
  ## Why this matters
  This removes a deterministic divergence source where pre-trim could
  think the request fits while the actual compact request exceeded context
  because its instructions were longer/different.
  
  ## Scope
  In scope:
  - estimator/payload base-instructions parity in remote compaction
  
  Out of scope:
  - retry-on-`context_length_exceeded`
  - compaction threshold/headroom policy changes
  - broader trimming policy changes
  
  ## Codex author:
  `codex fork 019c2b24-c2df-7b31-a482-fb8cf7a28559`

  Charley Cunningham · 2026-02-04 23:24:06 -08:00
  dc7007beaa
• Session-level model client (#10664)

  Make ModelClient a session-scoped object.
  Move state that is session level onto the client, and make state that is
  per-turn explicit on corresponding methods.
  Stop taking a huge Config object, instead only pass in values that are
  actually needed.
  
  ---------
  
  Co-authored-by: Josh McKinney <joshka@openai.com>

  pakrym-oai · 2026-02-04 16:58:48 -08:00
  0e8d359da9
• Stop client from being state carrier (#10595)

  I'd like to make client session wide. This requires shedding all random
  state it has to carry.

  pakrym-oai · 2026-02-04 09:05:37 -08:00
  7f20357611
• Trim compaction input (#10374)

  Two fixes:
  
  1. Include trailing tool output in the total context size calculation.
  Otherwise when checking whether compaction should run we ignore newly
  added outputs.
  2. Trim trailing tool output/tool calls until we can fit the request
  into the model context size. Otherwise the compaction endpoint will fail
  to compact. We only trim items that can be reproduced again by the model
  (tool calls, tool call outputs).

  pakrym-oai · 2026-02-02 19:03:11 -08:00
  cbfd2a37cc
• Do not append items on override turn context (#10354)
  pakrym-oai · 2026-02-01 18:51:26 -08:00
  03fcd12e77
• Plan mode: stream proposed plans, emit plan items, and render in TUI (#9786)

  ## Summary
  - Stream proposed plans in Plan Mode using `<proposed_plan>` tags parsed
  in core, emitting plan deltas plus a plan `ThreadItem`, while stripping
  tags from normal assistant output.
  - Persist plan items and rebuild them on resume so proposed plans show
  in thread history.
  - Wire plan items/deltas through app-server protocol v2 and render a
  dedicated proposed-plan view in the TUI, including the “Implement this
  plan?” prompt only when a plan item is present.
  
  ## Changes
  
  ### Core (`codex-rs/core`)
  - Added a generic, line-based tag parser that buffers each line until it
  can disprove a tag prefix; implements auto-close on `finish()` for
  unterminated tags. `codex-rs/core/src/tagged_block_parser.rs`
  - Refactored proposed plan parsing to wrap the generic parser.
  `codex-rs/core/src/proposed_plan_parser.rs`
  - In plan mode, stream assistant deltas as:
    - **Normal text** → `AgentMessageContentDelta`
    - **Plan text** → `PlanDelta` + `TurnItem::Plan` start/completion  
    (`codex-rs/core/src/codex.rs`)
  - Final plan item content is derived from the completed assistant
  message (authoritative), not necessarily the concatenated deltas.
  - Strips `<proposed_plan>` blocks from assistant text in plan mode so
  tags don’t appear in normal messages.
  (`codex-rs/core/src/stream_events_utils.rs`)
  - Persist `ItemCompleted` events only for plan items for rollout replay.
  (`codex-rs/core/src/rollout/policy.rs`)
  - Guard `update_plan` tool in Plan Mode with a clear error message.
  (`codex-rs/core/src/tools/handlers/plan.rs`)
  - Updated Plan Mode prompt to:  
    - keep `<proposed_plan>` out of non-final reasoning/preambles  
    - require exact tag formatting  
    - allow only one `<proposed_plan>` block per turn  
    (`codex-rs/core/templates/collaboration_mode/plan.md`)
  
  ### Protocol / App-server protocol
  - Added `TurnItem::Plan` and `PlanDeltaEvent` to core protocol items.
  (`codex-rs/protocol/src/items.rs`, `codex-rs/protocol/src/protocol.rs`)
  - Added v2 `ThreadItem::Plan` and `PlanDeltaNotification` with
  EXPERIMENTAL markers and note that deltas may not match the final plan
  item. (`codex-rs/app-server-protocol/src/protocol/v2.rs`)
  - Added plan delta route in app-server protocol common mapping.
  (`codex-rs/app-server-protocol/src/protocol/common.rs`)
  - Rebuild plan items from persisted `ItemCompleted` events on resume.
  (`codex-rs/app-server-protocol/src/protocol/thread_history.rs`)
  
  ### App-server
  - Forward plan deltas to v2 clients and map core plan items to v2 plan
  items. (`codex-rs/app-server/src/bespoke_event_handling.rs`,
  `codex-rs/app-server/src/codex_message_processor.rs`)
  - Added v2 plan item tests.
  (`codex-rs/app-server/tests/suite/v2/plan_item.rs`)
  
  ### TUI
  - Added a dedicated proposed plan history cell with special background
  and padding, and moved “• Proposed Plan” outside the highlighted block.
  (`codex-rs/tui/src/history_cell.rs`, `codex-rs/tui/src/style.rs`)
  - Only show “Implement this plan?” when a plan item exists.
  (`codex-rs/tui/src/chatwidget.rs`,
  `codex-rs/tui/src/chatwidget/tests.rs`)
  
  <img width="831" height="847" alt="Screenshot 2026-01-29 at 7 06 24 PM"
  src="https://github.com/user-attachments/assets/69794c8c-f96b-4d36-92ef-c1f5c3a8f286"
  />
  
  ### Docs / Misc
  - Updated protocol docs to mention plan deltas.
  (`codex-rs/docs/protocol_v1.md`)
  - Minor plumbing updates in exec/debug clients to tolerate plan deltas.
  (`codex-rs/debug-client/src/reader.rs`, `codex-rs/exec/...`)
  
  ## Tests
  - Added core integration tests:
    - Plan mode strips plan from agent messages.
    - Missing `</proposed_plan>` closes at end-of-message.  
    (`codex-rs/core/tests/suite/items.rs`)
  - Added unit tests for generic tag parser (prefix buffering, non-tag
  lines, auto-close). (`codex-rs/core/src/tagged_block_parser.rs`)
  - Existing app-server plan item tests in v2.
  (`codex-rs/app-server/tests/suite/v2/plan_item.rs`)
  
  ## Notes / Behavior
  - Plan output no longer appears in standard assistant text in Plan Mode;
  it streams via `PlanDelta` and completes as a `TurnItem::Plan`.
  - The final plan item content is authoritative and may diverge from
  streamed deltas (documented as experimental).
  - Reasoning summaries are not filtered; prompt instructs the model not
  to include `<proposed_plan>` outside the final plan message.
  
  ## Codex Author
  `codex fork 019bec2d-b09d-7450-b292-d7bcdddcdbfb`

  Charley Cunningham · 2026-01-30 18:59:30 +00:00
  ec4a2d07e4
• compaction (#10034)

  # External (non-OpenAI) Pull Request Requirements
  
  Before opening this Pull Request, please read the dedicated
  "Contributing" markdown file or your PR may be closed:
  https://github.com/openai/codex/blob/main/docs/contributing.md
  
  If your PR conforms to our contribution guidelines, replace this text
  with a detailed and high quality description of your changes.
  
  Include a link to a bug report or enhancement request.

  Ahmed Ibrahim · 2026-01-28 11:36:11 -08:00
  b7edeee8ca