78 Commits

• ensure thread.history_mode is immutable (#30261)

  ## Description
  
  This PR makes `thread.history_mode` immutable after the thread's
  canonical first `SessionMeta` has been written. Later same-thread
  `SessionMeta` lines are compatibility metadata writes, not a new thread
  definition.
  
  Without this, an older binary could append a `SessionMeta` that omits
  `history_mode`; when a newer binary replays it, serde defaults that
  missing field to `legacy` and SQLite could downgrade a paginated thread.
  
  ## Why
  
  `history_mode` is the persisted thread storage contract.
  Paginated-thread fail-closed behavior and SQLite memory filtering depend
  on it staying aligned with canonical rollout metadata, especially when
  multiple Codex binary versions can touch the same local rollout.
  
  ## What changed
  
  - Stop generic rollout metadata replay from overwriting `history_mode`
  from later `SessionMeta` items.
  - Remove `history_mode` from `ThreadMetadataPatch`, so mutable metadata
  sync and app-server metadata updates cannot rewrite it.
  - When local metadata sync has to recreate a missing SQLite row, recover
  `history_mode` from the rollout's canonical first `SessionMeta` instead
  of from a mutable patch.
  - Keep the in-memory thread store using the created thread's canonical
  `history_mode` instead of metadata patches.
  - Fill the one remaining core test `CreateThreadParams` initializer with
  the new `history_mode` field; Bazel CI caught this after the parent
  history-mode PR landed.
  
  ## Validation
  
  - `just fmt`
  - `just test -p codex-thread-store`
  - `just test -p codex-state
  session_meta_does_not_set_model_or_reasoning_effort`

  Owen Lin · 2026-06-26 12:32:31 -07:00
  812cd2bb57
• feat(app-server): add history_mode to thread (#29927)

  ## Description
  
  This PR adds a new `historyMode = "legacy" | "paginated"` to `Thread`.
  This will be stored in `SessionMeta` in the JSONL rollout file and as a
  new column in the SQLite thread_metadata table, and exposed on
  `thread/start` and on the `Thread` object in app-server.
  
  ## What changed
  
  - Added canonical `ThreadHistoryMode` with `legacy` and `paginated`,
  defaulting old and new SessionMeta to `legacy`.
  - Carried `history_mode` through core session config, ThreadStore stored
  metadata, local/in-memory stores, rollout metadata extraction, and the
  existing SQLite `threads` table.
  - Added experimental `historyMode` to app-server v2 `Thread` and
  `thread/start`.
  - Made paginated stored threads metadata-discoverable but unsupported
  for legacy full-history reads, `load_history`, live resume, and create
  paths.
  - Regenerated app-server schema fixtures and added
  protocol/state/thread-store/app-server coverage for persistence and
  fail-closed behavior.
  
  ## Compatibility floor
  Because users may be running various versions of Codex binaries on the
  same machine (TUI, Codex App, etc.), we will need to establish a
  compatibility floor for upcoming paginated threads, which will change
  how thread storage reads and writes work.
  
  The overall plan here:
  ```
  Release N:
  - Add historyMode to SessionMeta / Thread / SQLite metadata.
  - Teach binaries to understand paginated threads.
  - If a binary sees `historyMode="paginated"` but does not support the paginated contract, it refuses to resume/mutate the thread.
  - Default remains `"legacy"`.
  
  Release N+1:
  - First-party clients start opting into paginated threads where appropriate.
  - Internal dogfood / staged rollout.
  - Measure old-client usage and paginated-thread unsupported errors.
  
  Release N+2:
  - Only after Release N+ is overwhelmingly deployed, make paginated the default.
  - Accept that a small tail of N-1-or-older binaries may not understand paginated threads.
  ```
  
  The important behavior change is fail-closed handling for a binary that
  encounters a persisted `paginated` thread before it knows how to fully
  support paginated history. In app-server, if a thread is `paginated`, we
  will:
  
  - allow metadata-only discovery paths like `thread/list` and
  `thread/read(includeTurns=false)`, so clients can still see the thread
  and inspect its `historyMode`
  - reject legacy full-history/live-thread paths like
  `thread/read(includeTurns=true)` and `thread/resume` with an unsupported
  JSON-RPC error
  - avoid silently treating an unknown or future `historyMode` as `legacy`
  
  Under the hood, the ThreadStore layer also rejects legacy operations
  that would need to load or replay the full thread history for a
  paginated thread. That gives us the behavior we want for Release N:
  future paginated threads are visible, but this binary fails closed
  instead of trying to operate on them as if they were legacy threads.

  Owen Lin · 2026-06-26 09:12:42 -07:00
  5267e805fb
• Persist selected capability roots and resolve availability per model step (#29856)

  ## Why
  
  `selectedCapabilityRoots` is durable thread intent: “use this capability
  root from environment `worker`.”
  
  The important product assumption is:
  
  > One environment ID always names the same logical executor and stable
  contents.
  
  `worker` does not silently change from executor A to an unrelated
  executor B. The process-local connection handle for `worker` can still
  be replaced while Codex is running, though, for example when
  `environment/add` registers a fresh handle for the same logical
  environment.
  
  The thread should persist only the stable selection. Each model step
  should pair that selection with the exact ready handle captured for that
  step.
  
  ## The boundary
  
  ```text
  persisted thread intent
    plugin@1 -> environment "worker"
                  |
                  | capture the current step
                  v
  model-step view
    unavailable, or
    plugin@1 + worker's exact captured ready handle
  ```
  
  The environment ID is the stable identity and cache key. The
  `Arc<Environment>` is only a process-local handle retained so consumers
  of one model step use the same captured environment. It is never
  persisted and it does not imply different environment contents.
  
  ## What changes
  
  ### Persist the stable selection
  
  Selected roots are written into `SessionMeta` and restored with the
  thread. Forked subagents inherit the same selections, including
  bounded-history forks.
  
  Only stable data is persisted: root ID, environment ID, and root path.
  
  ### Capture readiness together with the exact handle
  
  The environment snapshot records:
  
  ```rust
  environment_id -> Some(Arc<Environment>) // ready in this step
  environment_id -> None                   // still starting in this step
  ```
  
  This prevents readiness and execution from coming from different
  registry snapshots.
  
  For example:
  
  ```text
  step snapshot: worker -> handle A, ready
  environment/add: worker -> fresh handle B for the same logical environment
  current step: plugin@1 still uses captured handle A
  ```
  
  Without carrying handle A in the snapshot, the resolver could combine “A
  was ready” with handle B and treat B as ready before it had finished
  starting.
  
  This does not change cache invalidation. Stable capability metadata
  remains identified by environment ID and capability root. Replacing a
  process-local handle under the same stable environment ID does not
  invalidate or rediscover that metadata.
  
  ### Resolve availability per model step
  
  - A ready captured environment produces resolved roots using its
  captured handle.
  - A starting, missing, or failed environment is omitted from that step.
  - A selected lazy environment that is outside the turn's captured
  environment set is asked to start, and a later step can observe it as
  ready.
  - No capability files are scanned here.
  
  Transient transport disconnects remain the remote client's reconnect
  concern. This PR models initial attachment/readiness; it does not add
  live socket-connectivity state.
  
  ## Example
  
  ```text
  thread selection: plugin@1 -> environment "worker"
  
  step 1: worker is starting -> plugin@1 unavailable
  step 2: worker is ready    -> plugin@1 resolves through worker's captured handle
  step 3: fresh local handle -> current step remains pinned; a later step captures its own view
  ```
  
  Temporary unavailability does not discard the durable selection. Later
  PRs can retain stable metadata caches while projecting only currently
  available capabilities into model-visible World State.
  
  ## Compatibility
  
  The app-server request shape does not change. Older rollouts without
  `selected_capability_roots` deserialize to an empty list.
  
  ## Stack
  
  1. **This PR:** persist stable selected roots and resolve them through
  an exact model-step handle.
  2. #29960: cache stable skill metadata and project available skills into
  World State.
  3. #29946: cache stable plugin declarations and manage the separate live
  MCP runtime.

  jif · 2026-06-25 17:49:43 +00:00
  8f02973d25
• [2/3] core: persist world state in rollouts (#29835)

  ## Why
  
  `WorldState` currently remembers its model-visible diff baseline only in
  memory. That leaves no durable source for restoring the exact baseline
  after resume, fork, rollback, or compaction.
  
  This is the second PR in the WorldState persistence stack, built on
  #29833 and following #29249. It records durable state transitions; the
  next PR will replay them during rollout reconstruction.
  
  ## What
  
  - Add a `world_state` rollout item containing either a full snapshot or
  an RFC 7386 JSON Merge Patch.
  - Persist a full snapshot after initial context and after compaction
  establishes a new context window.
  - Persist non-empty patches when later sampling steps or turns advance
  the WorldState baseline.
  - Write model-visible history before its matching WorldState record, so
  an interrupted write can only cause a safe repeated update on replay.
  - Preserve WorldState records for full-history forks while excluding
  them from thread previews, metadata, and app-server history
  materialization.
  
  Older binaries read rollout lines independently, so they skip the
  unknown `world_state` records while retaining the rest of the thread.
  
  ## Testing
  
  - `just test -p codex-core
  snapshot_merge_patch_changes_and_removes_nested_values`
  - `just test -p codex-core
  world_state_baseline_deduplicates_until_history_is_replaced`
  - `just test -p codex-core
  deferred_executor_compaction_preserves_then_updates_environment_once`
  - `just test -p codex-protocol`
  - `just test -p codex-rollout`
  - `just test -p codex-state`
  - `just test -p codex-thread-store`
  - `just test -p codex-app-server-protocol`

  sayan-oai · 2026-06-24 20:13:49 -07:00
  fa036d39aa
• feat(app-server): list descendant threads by ancestor (#29591)

  ## Why
  
  `thread/list` can filter direct children with `parentThreadId`, but
  clients cannot request an entire spawned subtree. Discovering every
  descendant requires repeated client-side requests and gives up the
  database's existing filtering and pagination path.
  
  ## What changed
  
  Experimental clients can use `ancestorThreadId` to return strict
  descendants at any depth while `parentThreadId` retains its direct-child
  meaning. The filters are mutually exclusive, the ancestor is excluded,
  and every result preserves its immediate `parentThreadId` so callers can
  reconstruct the tree.
  
  ## How it works
  
  - **Explicit relationship:** Internal list parameters distinguish direct
  children from transitive descendants without changing the meaning of
  `parentThreadId`.
  - **Existing graph:** Persisted parent-child spawn edges remain the
  source of truth, so descendant lookup needs no schema migration or
  ancestry cache.
  - **Indexed traversal:** A recursive SQLite query starts from the
  parent-edge index, walks each generation, and applies thread filters,
  sorting, and cursor pagination in the same database request.
  - **Reconstructable results:** The response stays flat and normally
  ordered while carrying each descendant's immediate parent.
  
  ## Verification
  
  Ran 550 tests across the protocol, state, rollout, and thread-store
  crates, then reran the four focused state, store, and app-server
  descendant-listing tests after the final diff reduction. Scoped Clippy
  and formatting checks passed. Stable and experimental schema generation
  was checked; the stable fixtures remain unchanged while the experimental
  schema includes the new field.

  Brent Traut · 2026-06-24 13:08:14 -07:00
  8057603d0c
• Persist agent messages as response items (#29829)

  ## Why
  
  Inter-agent messages are recorded in live history as
  `ResponseItem::AgentMessage`, but rollouts stored
  `InterAgentCommunication` and rebuilt the response item during resume.
  This made the rollout differ from the actual Responses history.
  
  ## What changed
  
  - store the prepared `agent_message` response item directly
  - keep `trigger_turn` in a small local metadata record for fork
  truncation
  - keep reading older `inter_agent_communication` rollout items

  jif · 2026-06-24 15:43:10 +01:00
  b4f0f3eff1
• Support thread-level originator overrides (#29477)

  ## Why
  
  Work(TPP) threads can be launched from the Desktop app, but if they all
  keep the Desktop app's default originator then downstream attribution
  cannot distinguish local Work launches from cloud-backed Work launches.
  `thread/start.serviceName` already carries that launch signal, while
  `SessionMeta.originator` is the durable thread-level value that survives
  resume and fork.
  
  This change converts the Desktop Work service names into an effective
  originator at thread creation time, persists that originator with the
  thread, and keeps using it for later model requests and memory writes.
  
  ## What changed
  
  - Map `CODEX_WORK_LOCAL` and `CODEX_WORK_CLOUD` service names to
  per-thread originators, while preserving
  `CODEX_INTERNAL_ORIGINATOR_OVERRIDE` as the highest-precedence override.
  - Persist the effective originator in `SessionMeta.originator`, read it
  back on resume/fork, and inherit the parent originator for subagent
  spawns when there is no persisted session metadata.
  - Handle truncated `SpawnAgentForkMode::LastNTurns` forks by falling
  back to the live parent originator when the forked history no longer
  includes `SessionMeta`.
  - Thread the per-thread originator through Responses headers,
  websocket/compaction request paths, thread-store creation, rollout
  metadata, and memory stage-one telemetry.
  
  ## Verification
  
  - `just test -p codex-core
  agent::control::tests::spawn_thread_subagent_inherits_parent_originator_without_fork
  agent::control::tests::spawn_thread_subagent_fork_last_n_turns_inherits_parent_originator_without_session_meta
  thread_manager::tests::originator_override_precedes_service_name_remapping`
  - `just test -p codex-core
  agent::control::tests::resume_thread_subagent_restores_stored_metadata_and_effective_multi_agent_mode`
  - `just test -p codex-memories-write`
  - `just fix -p codex-core -p codex-memories-write`
  - `git diff --check`

  alexsong-oai · 2026-06-23 17:23:38 -07:00
  1acb722e8a
• core: persist initial context window metadata (#29519)

  ## Why
  
  PR #29494 made context-window IDs visible to the model by wrapping the
  token-budget window payload in `<context_window>`, but rollout JSONL
  consumers still could not see the initial window identity by tailing the
  session file. Compacted rollout items carry window IDs only after
  compaction has happened, so a session with no compaction had no durable
  JSONL record for window 0.
  
  This change gives tailing consumers a stable initial-window record at
  session creation time.
  
  ## What Changed
  
  - Added `session_meta.context_window.window_id` for the initial
  context-window identity.
  - `CreateThreadParams` now requires `initial_window_id: String`, so
  thread-store callers cannot accidentally create new threads without
  window-0 metadata.
  - Live thread creation derives the persisted initial window ID from the
  same `AutoCompactWindowIds` used to initialize `SessionState`, keeping
  runtime state and JSONL metadata aligned.
  - Rollout reconstruction uses `session_meta.context_window.window_id` as
  the initial-window fallback and derives `window_number = 0`,
  `first_window_id = window_id`, and `previous_window_id = None`
  internally.
  - Fork reconstruction intentionally uses the same rollout reconstruction
  path; consumers that need to distinguish copied initial-window metadata
  can use the rollout `thread_id`.
  - Legacy compactions without `window_number` still use compaction-count
  fallback accounting instead of being reset to window 0 by the
  initial-window fallback.
  - Compacted rollout metadata still takes precedence once compaction
  records exist, preserving the richer chain fields there.
  
  ## JSONL Shape
  
  Real rollout JSONL is one object per line. This example is expanded for
  readability, but shows the new initial `session_meta.context_window`
  record followed by the existing compacted rollout item shape that also
  carries window IDs:
  
  ```jsonl
  {
    "timestamp": "2026-06-22T12:00:00.000Z",
    "type": "session_meta",
    "payload": {
      "session_id": "<THREAD_ID>",
      "id": "<THREAD_ID>",
      "timestamp": "2026-06-22T12:00:00.000Z",
      "cwd": "/repo",
      "originator": "codex",
      "cli_version": "0.0.0",
      "source": "cli",
      "model_provider": "<MODEL_PROVIDER>",
      "context_window": {
        "window_id": "<INITIAL_WINDOW_ID>"
      }
    }
  }
  ...
  {
    "timestamp": "2026-06-22T12:34:56.000Z",
    "type": "compacted",
    "payload": {
      "message": "<COMPACTION_SUMMARY>",
      "replacement_history": [
        "..."
      ],
      "window_number": 1,
      "first_window_id": "<INITIAL_WINDOW_ID>",
      "previous_window_id": "<INITIAL_WINDOW_ID>",
      "window_id": "<NEXT_WINDOW_ID>"
    }
  }
  ```
  
  The nested `context_window` object is intentional: it gives rollout
  consumers a stable namespace for context-window metadata while only
  writing the non-derivable initial `window_id`. For the initial window,
  `window_number`, `first_window_id`, and `previous_window_id` are derived
  internally instead of being written to the rollout.
  
  ## Verification
  
  - `just test -p codex-protocol`
  - `just test -p codex-rollout
  recorder_materializes_on_flush_with_pending_items`
  - `just test -p codex-core reconstruct_history`
  - `just test -p codex-core
  record_initial_history_reconstructs_forked_transcript`
  - `just test -p codex-thread-store`
  - `just test -p codex-state`
  - `just test -p codex-app-server
  thread_read_returns_summary_without_turns`
  - `just test -p codex-rollout persistence_metrics`

  Michael Bolin · 2026-06-23 21:50:50 +00:00
  01f89c8c59
• [codex] Instrument rollout persistence bytes (#29498)

  - Add 1%-sampled rollout persistence metrics that report per-item and
  per-thread JSON byte totals before and after filtering when metrics
  export is enabled.
  - Tag each item with its exact response or event variant, including
  nested turn-item kinds for conditionally persisted completion events, so
  aggregate cloud-storage impact can be estimated by policy choice.

  Tom · 2026-06-23 09:26:30 -07:00
  02c326f646
• Share resumed rollout history (#28426)

  ## Summary
  
  Resuming a persisted thread currently deep-clones its complete rollout
  history several times. `InitialHistory` is retained for the app-server
  response, copied into thread persistence, and copied again by read-only
  accessors. These copies scale with the complete rollout rather than the
  bounded model context and add measurable latency for large sessions.
  
  This change stores resumed rollout history in `Arc<Vec<RolloutItem>>`.
  Rollout loading wraps the parsed vector once, while app-server response
  construction, session initialization, and thread persistence share it
  through inexpensive `Arc` clones. Read-only history access now returns a
  borrowed slice, and fork paths use `Arc::unwrap_or_clone` where they
  genuinely need mutable ownership. Rollout reconstruction also consumes
  its temporary context instead of cloning the reconstructed model
  history.
  
  The serialized representation remains unchanged. In an artificial 123 MB
  rollout benchmark, sharing resumed history reduced cold resume latency
  by roughly 9–10%. The affected crates compile with their test targets,
  all 80 thread-store tests pass, and the Bazel dependency lock remains
  valid.

  Charlie Marsh · 2026-06-23 10:23:25 -04:00
  330ae6a516
• core: rename metadata -> internal_chat_message_metadata_passthrough (#28968)

  ## Description
  This PR cuts Codex over from generic `ResponseItem.metadata` (introduced
  here: https://github.com/openai/codex/pull/28355) to
  `ResponseItem.internal_chat_message_metadata_passthrough`, which is the
  blessed path and has strongly-typed keys.
  
  For now we have to drop this MAv2 usage of `metadata`:
  https://github.com/openai/codex/pull/28561 until we figure out where
  that should live.

  Owen Lin · 2026-06-22 11:11:25 -07:00
  5b95745eae
• Persist session IDs across thread resume (#29327)

  ## Summary
  
  A cold-resumed subagent kept its durable thread ID but could receive a
  new session ID, splitting one agent tree across multiple sessions after
  a restart.
  
  Persist the root session ID in every rollout `SessionMeta`, carry it
  through thread creation, and restore it before initializing the resumed
  `Session` and `AgentControl`.
  
  ## Behavior
  
  For a nested agent tree:
  
  ```text
  root session R
    parent thread P
      child thread C
  ```
  
  The child rollout stores:
  
  ```text
  session_id:       R
  parent_thread_id: P
  id:               C
  ```
  
  After a cold resume, the child still belongs to root session `R` while
  its immediate parent remains `P`. The integration coverage uses distinct
  values for all three IDs so it catches restoring the session from
  `parent_thread_id`.
  
  ## Legacy rollouts
  
  Previous rollouts have `id` but no `session_id`. `SessionMetaLine`
  deserialization treats a missing `session_id` as `id`, keeping those
  files readable, listable, and resumable. When a legacy subagent is
  resumed through its root, that synthesized child ID no longer overrides
  the inherited root-scoped `AgentControl`. New rollouts always persist
  the explicit root session ID.

  jif · 2026-06-22 09:36:08 +02:00
  6d15bb3d17
• core: add context window lineage IDs (#29256)

  ## Why
  
  The rendered `<token_budget>` fragment identifies the thread and current
  context window, but it does not expose enough lineage to identify the
  first window in the thread or the immediately preceding window. Those
  IDs also need to remain stable across compaction, resume, and rollback.
  
  ## What changed
  
  - Track first, previous, and current UUIDv7 context-window IDs in
  auto-compaction state.
  - Render `thread_id`, `first_window_id`, `previous_window_id`, and the
  current window ID in the full `<token_budget>` fragment.
  - Persist the first and previous window IDs in compacted rollout
  checkpoints and restore them during rollout reconstruction.
  - Preserve compatibility with older compacted records that do not
  contain the new optional fields.
  - Update focused state, rendering, reconstruction, rollback, and
  serialization coverage.
  
  ## Validation
  
  - `just test -p codex-core token_budget`
  - `just test -p codex-protocol compacted_item::tests`
  - `just test -p codex-core tracks_prefill_and_window_boundaries`
  - `just test -p codex-core
  reconstruct_history_uses_replacement_history_verbatim`
  - `just test -p codex-core
  thread_rollback_restores_cleared_reference_context_item_after_compaction`

  pakrym-oai · 2026-06-20 13:15:49 -07:00
  d1209bddfc
• [codex] trace pre-sampling skill and persistence latency (#29042)
  rphilizaire-openai · 2026-06-19 10:13:27 -07:00
  a57b268d61
• core: add UUIDv7 context window IDs (#28953)

  ## Why
  
  The token-budget context currently identifies a context window by its
  thread-local sequence number. A UUIDv7 gives the model a stable opaque
  identity that remains fixed for a window and rotates when compaction or
  `new_context` starts the next one.
  
  ## What changed
  
  - Preserve the existing monotonic value as `window_number` and add a
  UUIDv7 `window_id` to `CompactedItem`.
  - Generate and rotate the UUID with auto-compaction window state,
  persist it alongside the number, and reconstruct it on resume and
  rollback.
  - Accept legacy compacted rollout records where the numeric `window_id`
  represented the window number.
  - Use the UUID only in token-budget context; existing request headers
  and metadata continue using `thread_id:window_number`.
  
  ## Testing
  
  - `just test -p codex-protocol compacted_item::tests`
  - `just test -p codex-core token_budget`

  pakrym-oai · 2026-06-18 17:00:49 -07:00
  5c12034e42
• [codex] Make thread store turn filter optional (#28949)

  Make `ListItemsParams::turn_id` optional so callers can list persisted
  items across an entire thread or narrow the result to one turn. This
  aligns the thread-store API and documentation with thread-wide item
  listing while preserving the optional turn-filter behavior for
  implementations.

  Tom · 2026-06-18 12:13:31 -07:00
  01a2df2947
• [codex] Add optional IDs to response items (#28812)

  ## Why
  
  `ResponseItem` variants do not have a consistent internal ID shape: some
  variants carry required IDs, some carry optional IDs, and some cannot
  represent an ID at all. The existing fields also use inconsistent serde,
  TypeScript, and JSON-schema annotations. A single enum-level access path
  is needed before history recording can assign and retain IDs.
  
  This PR establishes that internal model only. It intentionally does not
  generate or serialize IDs; allocation and wire persistence are isolated
  in the stacked follow-up.
  
  ## What changed
  
  - Give every concrete `ResponseItem` variant an `Option<String>` ID
  field.
  - Apply the same internal-only annotations to every ID field:
  `#[serde(default, skip_serializing)]`, `#[ts(skip)]`, and
  `#[schemars(skip)]`.
  - Add `ResponseItem::id()` and `ResponseItem::set_id()` as the shared
  accessors.
  - Preserve IDs when history items are rewritten for truncation.
  - Adapt consumers that previously assumed reasoning and image-generation
  IDs were required.
  - Regenerate app-server schemas so the hidden fields are represented
  consistently.
  
  The serde catch-all `ResponseItem::Other` remains ID-less because it
  must remain a unit variant.
  
  ## Test plan
  
  - `cargo check --tests -p codex-core -p codex-api -p codex-rollout-trace
  -p codex-image-generation-extension`
  - `just test -p codex-protocol`
  - `just test -p codex-app-server-protocol`
  - `just test -p codex-api -p codex-rollout-trace -p
  codex-image-generation-extension`
  - `just test -p codex-core event_mapping`

  pakrym-oai · 2026-06-17 18:27:43 -07:00
  dbd2857f4b
• [codex] Restore thread recency with compatible migration history (#28671)

  ## Summary
  
  - Revert #28655, restoring the thread `recencyAt` behavior introduced by
  #27910.
  - Move `threads_recency_at` to migration 0039 so it no longer collides
  with `external_agent_config_imports` at version 0038.
  - Repair databases that already applied the recency migration as version
  38 by moving the matching migration-history row to version 39 before
  SQLx validation. The current version-38 migration can then apply
  normally.
  
  ## Validation
  
  - `just test -p codex-state
  migrations::tests::repairs_recency_migration_that_was_applied_as_version_38`
  - `just test -p codex-state -p codex-rollout -p codex-thread-store -p
  codex-app-server-protocol -p codex-tui`: 3,439 passed; six TUI tests
  could not open the machine's existing read-only incident database at
  `~/.codex/sqlite/state_5.sqlite`.
  - `just fix -p codex-state`
  - `just fmt`
  - Verified that state migration versions are unique.

  Jeremy Rose · 2026-06-17 18:52:18 +00:00
  7dc7096ae1
• Revert thread recencyAt for sidebar ordering (#28655)

  ## Why
  
  Revert #27910 to remove the newly introduced thread `recencyAt`
  persistence and API behavior from `main`.
  
  ## What changed
  
  This reverts commit `fac3158c2a783095768076489815f361fa9b0db4`,
  including the state migration, thread-store propagation, app-server API
  surface, generated schemas, and related tests.
  
  ## Validation
  
  Not run before opening; relying on CI for the initial fast signal.

  pakrym-oai · 2026-06-16 21:39:30 -07:00
  cb15c64760
• thread-store: fix response fixture compilation (#28642)

  ## Why
  
  A `codex-thread-store` test fixture still constructs
  `ResponseItem::FunctionCallOutput` without its required `metadata`
  field, preventing the crate's test targets from compiling on `main`.
  
  ## What changed
  
  - Set the fixture's response-item metadata to `None`.
  
  ## Testing
  
  - `cargo check -p codex-thread-store --tests`

  pakrym-oai · 2026-06-16 19:16:16 -07:00
  6f77491e95
• [codex] core: restore absolute turn context cwd (#28629)

  ## Why
  
  #28152 jumped the gun on moving the rollout format to store URIs, and
  would likely break compat with some features that don't go through the
  same types as the core logic.
  
  ## What
  
  Make `TurnContextItem.cwd` an `AbsolutePathBuf` again, remove test added
  for `PathUri` serialization in rollouts. Also drops a bunch of error
  paths that are no longer needed.

  Adam Perry @ OpenAI · 2026-06-16 19:05:26 -07:00
  3d02c443bc
• Add thread recencyAt for sidebar ordering (#27910)

  ## Summary
  
  Add a server-owned `recencyAt` timestamp and `recency_at` thread-list
  sort key for product recency ordering while preserving the existing
  meaning of `updatedAt` as the latest persisted thread mutation.
  
  This is the server-side alternative to #27697. Rather than narrowing
  `updatedAt`, clients can sort the sidebar by `recency_at` and continue
  treating `updatedAt` as mutation time.
  
  Paired Codex Apps PR:
  [openai/openai#1024599](https://github.com/openai/openai/pull/1024599)
  
  ## Contract
  
  - `recencyAt` initializes when a thread is created.
  - A turn start advances `recencyAt` monotonically.
  - Commentary, agent output, tool results, token/accounting updates, turn
  completion, archive, unarchive, resume, and generic metadata writes do
  not advance it.
  - `updatedAt` retains its existing behavior and continues to advance for
  persisted thread mutations.
  - Current servers populate `recencyAt`; the response field is optional
  in generated TypeScript so clients connected to older servers can fall
  back to `updatedAt`.
  - Filesystem-only fallback uses existing updated/mtime ordering when
  SQLite is unavailable.
  
  ## Persistence and compatibility
  
  Migration 0038 adds second- and millisecond-precision recency columns,
  backfills them from the existing updated timestamp, creates list
  indexes, and includes an insert trigger so older binaries writing to a
  migrated database seed recency without causing later mutations to
  advance it.
  
  Generic metadata upserts preserve existing recency values. Turn-start
  updates use a dedicated monotonic touch, and process-local allocation
  keeps millisecond cursor values unique. State DB list, search, read,
  filtered-list repair, rollout fallback propagation, and app-server
  conversions all carry the new field.
  
  ## API
  
  `Thread` responses include:
  
  ```ts
  recencyAt?: number
  ```
  
  `thread/list` and `thread/search` accept:
  
  ```json
  { "sortKey": "recency_at" }
  ```
  
  Generated TypeScript and JSON schemas are included.
  
  ## Validation
  
  - `just test -p codex-state` — 146 passed
  - `just test -p codex-rollout` — 69 passed
  - `just test -p codex-thread-store` — 81 passed
  - `just test -p codex-app-server-protocol` — 231 passed
  - Focused app-server list ordering, response mapping, archive/unarchive,
  and resume lifecycle tests passed
  - Scoped `just fix` for state, rollout, thread-store,
  app-server-protocol, and app-server
  - `just fmt`
  - `git diff --check`
  - Independent correctness, simplicity, elegance, security, and
  test-quality reviews; actionable ordering, lifecycle, query-projection,
  and timestamp-uniqueness findings were addressed

  Jeremy Rose · 2026-06-16 17:06:22 -07:00
  fac3158c2a
• core: render remote environment cwd natively (#28152)

  ## Why
  
  Model-visible `<environment_context>` should match the environment of
  the executor, not of the app server.
  
  Stacked on #28146.
  
  ## What
  
  - Keep selected environment cwd values as `PathUri` while building
  environment context.
  - Render cwd text using the path convention represented by the URI, with
  the canonical URI as a fallback.
  - Preserve compatibility with legacy `TurnContextItem.cwd` values when
  reconstructing and diffing context.
  - Extend the Wine-backed remote Windows test to assert that the model
  sees `powershell` and `C:\windows`.

  Adam Perry @ OpenAI · 2026-06-16 16:17:47 -07:00
  4c79527e31
• Use PathUri in filesystem permission paths for exec-server (#28165)

  ## Why
  
  Progress towards letting app-server and exec-server run on different
  platforms, specifically for sandbox configuration.
  
  ## What
  
  - Make the filesystem path containment hierarchy generic, defaulting to
  `AbsolutePathBuf` for now.
  - Have clients specify `AbsolutePathBuf` or `PathUri` directly where
  needed.
  - Use `PathUri` throughout exec-server filesystem protocol and trait
  boundaries.
  - Implement `From` for conversion to path URIs and `TryFrom` for
  fallible conversion to absolute paths through the generic type
  hierarchy.

  Adam Perry @ OpenAI · 2026-06-15 23:55:23 +00:00
  46f17930b6
• feat(app-server): filter threads by parent (#26662)

  ## Why
  
  Clients that display or coordinate spawned subagents need an
  authoritative snapshot of a thread's immediate spawned children when
  they connect to app-server or recover after missing live events.
  `thread/list` cannot query by parent, so clients must otherwise scan
  unrelated threads or reconstruct relationships from rollout history and
  transient events.
  
  The direct spawn relationship already exists in persisted
  `thread_spawn_edges` state. Review and Guardian threads do not
  participate in that lifecycle and are intentionally outside this
  filter's scope.
  
  ## What changed
  
  This adds an experimental `parentThreadId` filter to `thread/list`.
  Parent-filtered requests return direct spawned children from persisted
  state while preserving the existing response shape, explicit filters,
  sorting, and timestamp-only cursor behavior. The lookup does not read
  rollout transcripts or recursively return descendants.
  
  Supersedes #25112 with the narrower `thread/list` filter approach.
  
  ## How it works
  
  1. An experimental client passes a valid thread ID as `parentThreadId`.
  2. App-server routes the list through the existing thread-store and
  state-database boundaries.
  3. SQLite selects threads whose IDs have a direct persisted spawn edge
  from that parent.
  4. Omitted provider and source filters include all values; explicit
  filters keep ordinary `thread/list` semantics.
  5. Grandchildren, Review threads, and Guardian threads are excluded.
  
  ## Verification
  
  State (144 tests), rollout (69 tests), and focused app-server
  thread-list (31 tests) suites passed. Scoped Clippy checks and
  repository formatting also passed. Coverage includes direct spawned
  children, omitted grandchildren, pagination, malformed IDs, mixed source
  kinds, explicit filters, and operation without rollout files.

  Brent Traut · 2026-06-14 00:14:26 -07:00
  dfd03ea01b
• Support plaintext agent messages (#27830)

  ## Why
  
  Multi-agent v2 `send_message` deliveries already reach the receiving
  model as typed `agent_message` items with encrypted content.
  Child-completion notifications are generated by Codex itself, so their
  content is plaintext and previously fell back to a serialized JSON
  envelope inside an assistant message.
  
  With plaintext `input_text` supported for `agent_message`, both delivery
  paths can use the same model-visible type while preserving explicit
  author and recipient metadata.
  
  ## What changed
  
  - add plaintext `input_text` support to `AgentMessageInputContent` and
  regenerate the affected app-server schemas
  - preserve `InterAgentCommunication` as structured mailbox input instead
  of converting it to assistant text
  - record delivered communications as typed `agent_message` history items
  - persist a dedicated rollout item so local delivery metadata such as
  `trigger_turn` remains available without leaking into the Responses
  request
  - reconstruct typed agent messages on resume and preserve fork-turn
  truncation behavior
  - remove request-time assistant-content parsing
  - preserve plaintext and encrypted inter-agent deliveries in stage-one
  memory inputs
  - normalize and link plaintext and encrypted agent messages in rollout
  traces without treating inbound messages as child results
  - cover the real MultiAgent V2 child-completion path end to end with
  deterministic mailbox synchronization
  
  ## Verification
  
  - `just test -p codex-core
  plaintext_multi_agent_v2_completion_sends_agent_message`
  - `just test -p codex-core input_queue_drains_mailbox_in_delivery_order
  record_initial_history_reconstructs_typed_inter_agent_message
  fork_turn_positions_use_inter_agent_delivery_metadata`
  - `just test -p codex-memories-write
  serializes_inter_agent_communications_for_memory`
  - `just test -p codex-rollout-trace
  agent_messages_preserve_routing_and_content
  sub_agent_started_activity_creates_spawn_edge`
  - `just test -p codex-rollout-trace
  agent_result_edge_falls_back_to_child_thread_without_result_message`
  - `just test -p codex-protocol -p codex-rollout -p
  codex-app-server-protocol`

  jif · 2026-06-12 13:50:04 -07:00
  8f2d6416ce
• [codex] Remove async_trait from first-party code (#27475)

  ## Why
  
  First-party async traits should expose their `Send` contracts explicitly
  without requiring `async_trait`. This completes the migration pattern
  established in #27303 and #27304.
  
  ## What changed
  
  - Replaced the remaining first-party `async_trait` traits with native
  return-position `impl Future + Send` where statically dispatched and
  explicit boxed `Send` futures where object safety is required.
  - Kept implementations behavior-preserving, outlining existing async
  bodies into inherent methods where that keeps the diff reviewable.
  - Removed all direct first-party `async-trait` dependencies and the
  workspace dependency declaration.
  - Added a cargo-deny policy that permits `async-trait` only through the
  remaining transitive wrapper crates.
  - Updated `rand` from 0.8.5 to 0.8.6 to resolve RUSTSEC-2026-0097 and
  keep the full cargo-deny check passing.
  
  ## Validation
  
  - `just test -p codex-exec-server`: 216 passed, 2 skipped.
  - `just test -p codex-model-provider`: 39 passed.
  - `just test -p codex-core` and `just test`: changed tests passed;
  remaining failures are environment-sensitive suites unrelated to this
  migration.
  - `cargo deny check`
  - `just fix`
  - `just fmt`
  - `cargo shear`
  - `just bazel-lock-check`

  Adam Perry @ OpenAI · 2026-06-11 18:16:39 -07:00
  5a56caf18c
• [codex] Move persistence policy application into ThreadStore (#27318)

  Move the application of the persistence policy into the thread store, so
  thread stores can get raw append items rather than canonical append
  items. This will enable store-specific projections over the raw input
  items.

  Tom · 2026-06-11 16:24:12 -07:00
  b2cc01fc7c
• Add app-server thread/delete API (#25018)

  ## Why
  
  Clients can archive and unarchive threads today, but there is no
  app-server API for permanently removing a thread. Deletion also needs to
  cover the full session tree: deleting a main thread should remove
  spawned subagent threads and the related local metadata instead of
  leaving orphaned rollout files, goals, or subagent state behind.
  
  ## What
  
  - Adds the v2 `thread/delete` request and `thread/deleted` notification,
  with the response shape kept consistent with `thread/archive`.
  - Implements local hard delete for active and archived rollout files.
  - Deletes the requested thread's state DB row as the commit point, then
  best-effort cleans associated state including spawned descendants,
  goals, spawn edges, logs, dynamic tools, and agent job assignments.
  - Updates app-server API docs and generated protocol schema/TypeScript
  fixtures.

  Eric Traut · 2026-06-10 11:22:12 -07:00
  a19d43a40a
• Fix compressed rollout search path matching (#27407)

  ## Why
  
  `thread/search` found content inside compressed rollouts but could drop
  the result when joining it with SQLite-backed thread metadata. Search
  returned the physical `.jsonl.zst` path while SQLite retained the
  logical `.jsonl` path, so exact path matching failed.
  
  ## What changed
  
  - Key rollout search matches by their canonical logical `.jsonl` path,
  independent of the on-disk representation.
  - Canonicalize thread-list paths before joining them with content-search
  matches.
  - Update compressed-rollout coverage to assert the logical-path
  contract.
  
  ## Validation
  
  - Ran `just fmt`.
  - Ran `git diff --check`.
  - Tests and Clippy were intentionally left to CI.

  jif · 2026-06-10 19:23:42 +02:00
  d3abd8774e
• [codex] Store compact window id in rollout (#27264)

  ## Why
  
  Compaction window identity is part of session history, not model-client
  transport state. Persisting it with the compacted rollout item lets
  resumed threads continue from the reconstructed window without keeping
  mutable window state on `ModelClient`.
  
  ## What changed
  
  - Added `window_id` to `CompactedItem` and stamp it when
  `replace_compacted_history` installs compacted history.
  - Moved auto-compact window id ownership into `AutoCompactWindow` /
  `SessionState`; `ModelClient` now receives the request window id from
  callers instead of storing it.
  - Returned `window_id` from rollout reconstruction for resume.
  Reconstruction uses the newest surviving compacted item's stored
  `window_id` when present, and falls back to the legacy compacted-item
  count when it is absent.
  - Kept fork startup at the fresh default window id and updated direct
  model-client tests to pass explicit test window ids.
  
  ## Validation
  
  - `cargo check -p codex-core --tests`

  pakrym-oai · 2026-06-10 08:47:16 -07:00
  30ddb3325e
• [codex-analytics] add extensible feature thread sources (#27063)

  ## Why
  - `ThreadSource` currently defines a closed set of core-owned values
  - Product features also create threads for background or scheduled work
  - Adding every product-specific value to the core enum would require
  repeated `codex-rs` protocol changes
  - Feature-backed values let product callers provide precise attribution
  while preserving the existing core classifications
  
  ## What Changed
  - Adds `ThreadSource::Feature(String)` for app-owned thread source
  values
  - Represents all app-server v2 thread sources as scalar strings, so a
  feature source is supplied as `"automation"`
  - Persists and emits the feature's plain string label, so `"automation"`
  produces `thread_source="automation"` in analytics
  - Keeps `user`, `subagent`, and `memory_consolidation` as explicit
  core-owned values and regenerates the app-server schemas and TypeScript
  bindings
  
  ## Verification
  - `just write-app-server-schema`
  - `cargo check --workspace`
  - `just test -p codex-protocol
  feature_thread_source_serializes_as_its_app_owned_label`
  - `just test -p codex-app-server-protocol
  thread_sources_round_trip_as_scalar_labels`
  - `cargo test -p codex-analytics
  thread_initialized_event_serializes_expected_shape`
  - `just fmt`

  marksteinbrick-oai · 2026-06-09 12:27:10 -07:00
  a71e040df5
• Avoid rereading rollout history during cold resume (#27031)

  ## Summary
  
  - reuse the history-bearing `StoredThread` loaded while probing for a
  running thread
  - avoid rereading and reparsing the rollout when that probe finds no
  active process
  - reload after shutting down a loaded thread because shutdown may flush
  newer rollout items
  - add a regression test that verifies cold resume performs one
  history-bearing store read
  
  ## Problem
  
  `thread/resume` first reads the persisted thread with history while
  checking whether the thread is
  already running. When no running process exists, cold resume currently
  falls through to
  `resume_thread_from_rollout`, which reads and parses the same history
  again.
  
  That duplicate work grows with rollout size and remains on the
  synchronous resume path even when
  the caller requests `excludeTurns`.
  
  ## Background
  
  The duplicate read was introduced by #24528, which fixed resume
  overrides for idle cached
  threads. To support resumes specified by rollout path,
  `resume_running_thread` began loading the
  stored thread with history so it could resolve the canonical thread ID
  and determine whether a
  cached `CodexThread` was already loaded.
  
  That history is needed when the loaded-thread path handles the request.
  On a cold miss, however,
  the function's boolean result could only report that no loaded thread
  handled the request. It
  discarded the history-bearing `StoredThread`, and the normal cold-resume
  path immediately loaded
  and parsed the same rollout again.
  
  This change preserves the idle cached-thread behavior from #24528 while
  allowing the cold-resume
  path to reuse the probe result.
  
  ## Performance
  
  I benchmarked real retained rollouts using isolated `CODEX_HOME`
  directories, explicit rollout
  paths, debug builds of the commit and its exact parent, and alternating
  parent/patch order. The
  table below uses `thread/resume` with `excludeTurns: true`; response
  payload sizes were identical.
  
  | Rollout size | Records | Parent median | Patch median | Median paired
  saving |
  | ---: | ---: | ---: | ---: | ---: |
  | 6 MB | 3,574 | 541 ms | 441 ms | 132 ms |
  | 30 MB | 15,220 | 1.505 s | 1.041 s | 701 ms |
  | 60 MB | 31,453 | 2.644 s | 1.742 s | 970 ms |
  | 149 MB | 100,874 | 10.506 s | 7.156 s | 3.350 s |
  | 559 MB | 259,734 | 27.759 s | 16.725 s | 9.836 s |
  
  The absolute saving increases with thread size, as expected when
  removing one complete JSONL
  history read and parse. Total resume time is also content-dependent, so
  the relationship is not
  perfectly linear.
  
  I also tested full-history resume with `excludeTurns: false`. The
  response payload was
  byte-identical between variants, and the same size-dependent improvement
  remained visible:
  
  | Rollout size | Parent median | Patch median | Median paired saving |
  | ---: | ---: | ---: | ---: |
  | 6 MB | 1.052 s | 904 ms | 270 ms |
  | 30 MB | 2.667 s | 1.762 s | 924 ms |
  | 60 MB | 8.464 s | 6.272 s | 3.680 s |
  | 149 MB | 26.719 s | 12.118 s | 14.601 s |
  | 559 MB | 40.359 s | 25.475 s | 16.590 s |
  
  ## Validation
  
  - `just test -p codex-app-server
  cold_thread_resume_reuses_non_local_history_probe`
  - `just fix -p codex-app-server -p codex-thread-store`
  - `just fmt`

  Zanie Blue · 2026-06-09 11:16:27 -05:00
  6a9a49b334
• Add extra config to StoredThread, leave empty for now (#27092)
  Boyang Niu · 2026-06-09 00:38:35 +00:00
  e127a0cf99
• [codex] Support model-defined reasoning efforts (#26444)

  ## Summary
  - accept non-empty model-defined reasoning effort values while
  preserving built-in effort behavior
  - propagate the non-Copy effort type through core, app-server, TUI,
  telemetry, and persistence call sites
  - preserve string wire encoding and expose an open-string schema for
  clients
  - update model selection and shortcut behavior for model-advertised
  effort values
  
  ## Root cause
  `ReasoningEffort` gained a string-backed custom variant, so it could no
  longer implement `Copy` or rely on derived closed-enum serialization.
  Existing consumers still moved effort values from shared references and
  assumed a fixed built-in value set.
  
  ## Validation
  - `just fmt`
  - Local tests and compilation were not run per request; relying on CI.

  Ahmed Ibrahim · 2026-06-04 13:36:24 -07:00
  8ac304c299
• Persist multi-agent runtime metadata (#25721)

  Stack split from #25708. Original PR intentionally left open. This
  second PR persists multi-agent runtime metadata through thread creation,
  rollout recording, and thread storage.

  jif-oai · 2026-06-02 13:05:20 +02:00
  0c5ccd18ab
• feat: reuse compressed rollout search snippets (#25814)

  ## Summary
  - teach rollout search to return precomputed snippets for compressed
  rollouts
  - reuse those snippets in local thread search instead of reopening
  matching compressed files
  - keep the no-`rg` fallback single-pass and add regression coverage for
  the compressed path
  
  ## Why
  `thread/search` currently decodes matching compressed rollouts twice:
  once to discover the matching path and again to extract the snippet
  shown in results. That defeats a meaningful part of the compressed-read
  optimization work.
  
  ## Impact
  Compressed rollout hits now pay one decode pass on the search path while
  plain `.jsonl` hits keep the existing ripgrep-driven flow.
  
  ## Validation
  - `just test -p codex-rollout`
  - `just test -p codex-thread-store`
  - `just fix -p codex-rollout`
  - `just fix -p codex-thread-store`
  - `just fmt`

  jif-oai · 2026-06-02 11:32:36 +02:00
  45912a6dc6
• app-server: remove experimental persist_extended_history bool flag (#25712)

  ## Summary
  
  Remove the dead experimental `persistExtendedHistory` app-server flag
  and collapse rollout persistence to the single policy app-server already
  used.
  
  ## What Changed
  
  - Removed `persistExtendedHistory` from v2 thread start/resume/fork
  params and deleted its deprecation notice path.
  - Removed the persistence-mode enums and plumbing through core, rollout,
  and thread-store.
  - Made rollout filtering mode-free, keeping the existing limited
  persisted-history behavior.
  
  ## Test Plan
  
  - `just write-app-server-schema`
  - `cargo nextest run --no-fail-fast -p codex-app-server-protocol
  schema_fixtures`
  - `cargo nextest run --no-fail-fast -p codex-app-server
  thread_shell_command_history_responses_exclude_persisted_command_executions`
  - `cargo nextest run --no-fail-fast -p codex-rollout -p
  codex-thread-store`
  - final `rg` for removed flag/type names

  Owen Lin · 2026-06-01 23:33:42 +00:00
  11e0f3d3ae
• Reject directory rollout paths for pathless side chats (#25661)

  ## Why
  
  Fixes openai/codex#20944.
  
  Desktop side chats are intentionally ephemeral and pathless. They can
  still accept live turns while loaded, but after a reload there is no
  persisted rollout to resume. In the reported failure mode, Desktop could
  send `$CODEX_HOME` as the resume/fork path for one of these pathless
  side chats.
  
  `thread/resume` and `thread/fork` prefer an explicit `path` over
  `threadId`, and rollout path lookup only checked that a candidate
  existed. That let `$CODEX_HOME` pass as a rollout path, so the later
  rollout reader tried to open a directory and surfaced the low-level `Is
  a directory` error.
  
  ## What Changed
  
  - Reject explicit rollout paths that resolve to a directory or other
  non-file before attempting to read rollout history.
  - Make `codex_rollout::existing_rollout_path` return only plain or
  compressed rollout candidates that are actual files.
  - Add an app-server regression test that creates an ephemeral fork, runs
  a turn while the side thread is loaded, simulates reload, then verifies
  both `thread/resume` and `thread/fork` reject `$CODEX_HOME` with `path
  is a directory` instead of the OS-level directory-read error.
  - Rebase over the `TestAppServer` rename and update the remaining stale
  test harness call sites to use `TestAppServer` with `app_server` local
  variables.
  
  Relevant code:
  
  - `thread-store/src/local/read_thread.rs` validates explicit rollout
  paths before rollout reading:
  https://github.com/openai/codex/blob/25b47c8f425d351aaba4baa955a8092064a1707b/codex-rs/thread-store/src/local/read_thread.rs#L146-L165
  - `rollout/src/compression.rs` now requires file metadata for plain and
  compressed rollout candidates:
  https://github.com/openai/codex/blob/25b47c8f425d351aaba4baa955a8092064a1707b/codex-rs/rollout/src/compression.rs#L940-L950
  - The repro test covers the pathless ephemeral side-chat reload case:
  https://github.com/openai/codex/blob/25b47c8f425d351aaba4baa955a8092064a1707b/codex-rs/app-server/tests/suite/v2/thread_fork.rs#L774-L886
  
  ## Verification
  
  - `just test -p codex-app-server
  pathless_ephemeral_thread_rejects_codex_home_path_after_reload`

  Michael Bolin · 2026-06-01 16:02:06 -07:00
  b89bf1ef47
• Read compressed rollouts and materialize before append (#25087)

  ## Why
  
  Local rollout compression needs a cold `.jsonl.zst` representation
  without letting compressed physical paths leak into append-mode writers.
  The unsafe case is resume or metadata update code successfully reading a
  compressed rollout and then appending raw JSONL bytes to the zstd file.
  
  This PR folds the former #25088 materialization slice into the
  read-support PR so the reader changes and append-safety invariant land
  together.
  
  ## What Changed
  
  - Teach rollout readers, discovery, listing, search, and ID lookup to
  understand compressed `.jsonl.zst` rollouts.
  - Keep `.jsonl` as the logical/stored rollout path while allowing read
  paths to open either plain or compressed storage.
  - Materialize compressed rollouts back to plain `.jsonl` before
  append-mode writes, including resume and direct metadata append paths.
  - Preserve compressed-file permissions when materializing back to plain
  JSONL.
  - Refresh thread-store resolved rollout paths after compatibility
  metadata writes so reconciliation follows the materialized file.
  - Avoid treating transient compression temp files as real rollout lookup
  results.
  
  ## Remaining Stack
  
  #25089 remains the separate worker PR. It is based directly on this PR
  and stays behind the disabled `local_thread_store_compression` feature
  flag.
  
  The worker still has a broader coordination question: a resume or
  metadata update can race with background compression while a plain file
  is being replaced by `.jsonl.zst`. This PR handles the read and
  materialize-before-append primitives; it does not make the worker
  production-ready.
  
  ## Validation
  
  - `just test -p codex-rollout`
  - `just test -p codex-thread-store`
  - `just fix -p codex-rollout`
  - `just fix -p codex-thread-store`
  - `just bazel-lock-check`

  jif-oai · 2026-06-01 15:14:19 +02:00
  a8a6071279
• store and expose parent_thread_id on Threads (#25113)

  ## Why
  
  This PR
  https://github.com/openai/codex/pull/24161#discussion_r3325692763
  revealed a subagent data modeling issue, where we overloaded
  `forked_from_id` to also mean `parent_thread_id`. That's incorrect since
  guardian and review subagents can be a subagent and NOT fork the main
  thread's history.
  
  The solution here is to explicitly store a new `parent_thread_id` on
  `SessionMeta`, alongside `forked_from_id` which already exists. While
  we're at it, also expose it in the app-server protocol on the `Thread`
  object.
  
  A thread->subagent relationship and a fork of thread history are
  orthogonal concepts.
  
  ## What Changed
  
  - Added top-level `parent_thread_id` persistence on `SessionMeta` and
  runtime/session plumbing through `SessionConfiguredEvent`,
  `CodexSpawnArgs`, `SessionConfiguration`, `ThreadConfigSnapshot`,
  `TurnContext`, and `ModelClient`.
  - Made turn metadata, request headers, analytics, and subagent-start
  events read the separate runtime/top-level parent field instead of
  deriving general parent lineage from `SessionSource` or
  `forked_from_thread_id`.
  - Passed parent lineage separately at delegated subagent, review,
  guardian, agent-job, and multi-agent spawn construction sites;
  copied-history fork lineage remains derived only from `InitialHistory`.
  - Persisted and exposed parent lineage through rollout/thread-store
  projections and app-server v2 `Thread.parentThreadId`.
  - Updated app-server README text and regenerated app-server schema
  fixtures for the additive `parentThreadId` response field.

  Owen Lin · 2026-06-01 04:33:20 +00:00
  cf0911076f
• thread-store: store permission profiles (#23165)

  ## Why
  
  `SandboxPolicy` is the legacy compatibility shape, but
  `codex-thread-store` still exposed it through `StoredThread`,
  `ThreadMetadataPatch`, and live metadata sync. That kept thread-store
  consumers tied to the legacy representation and meant richer permission
  profile data could not round-trip through thread metadata or cold
  rollout reconciliation.
  
  ## What Changed
  
  - Replaced thread-store `sandbox_policy` API fields with canonical
  `PermissionProfile` fields.
  - Persist new permission-profile metadata as canonical JSON in the
  existing SQLite metadata slot while continuing to read older legacy
  sandbox policy values.
  - Updated local, in-memory, live metadata sync, and rollout extraction
  paths to propagate `TurnContextItem::permission_profile()`.
  - Re-materialize legacy permission metadata against the final rollout
  cwd when rollout-derived metadata replaces stale SQLite summaries.
  - Updated affected app-server and core test constructors to build
  `PermissionProfile` values directly.
  
  ## Test Plan
  
  - `cargo test -p codex-state`
  - `cargo test -p codex-thread-store`
  - `cargo test -p codex-app-server
  summary_from_stored_thread_preserves_millisecond_precision --lib`
  - `cargo test -p codex-core realtime_context --lib`

  Michael Bolin · 2026-05-29 11:55:31 -07:00
  a1ecf0cf1c
• [codex] Add user input client ids (#24653)

  ## Summary
  
  Adds an optional `clientId` field to app-server v2 `UserInput` and
  carries it through the core `UserInput` model so clients can correlate
  echoed user input items without relying on payload equality.
  
  ## Details
  
  - Adds `client_id: Option<String>` to core `UserInput` variants.
  - Exposes the v2 app-server field as `clientId` on the wire and in
  generated TypeScript.
  - Preserves the id when converting between app-server v2 and core
  protocol types.
  - Regenerates app-server schema fixtures.
  
  ## Validation
  
  - `just fmt`
  - `just write-app-server-schema`
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-protocol`
  - `just fix -p codex-app-server-protocol`
  - `just fix -p codex-protocol`
  - `git diff --check`

  Alexi Christakis · 2026-05-28 14:54:39 -07:00
  e92c952b2e
• [codex] Remove redundant SQLite dynamic tool storage (#24819)

  ## Why
  
  Dynamic tools are defined at thread start and already stored in rollout
  `SessionMeta`, which restores resumed and forked sessions. Persisting
  the same tools through SQLite creates a second runtime persistence path
  that is unnecessary prework for the explicit namespace refactor.
  
  ## What changed
  
  - Restore missing thread-start dynamic tools directly from rollout
  history, including when SQLite is enabled.
  - Remove SQLite dynamic-tool reads, writes, backfill, and thread
  metadata patch plumbing.
  - Add SQLite-enabled resume integration coverage that verifies a
  rollout-defined dynamic tool is still sent after resume.
  
  ## Compatibility
  
  The existing `thread_dynamic_tools` table is intentionally not dropped
  even though it's now unused. Older Codex binaries are allowed to open
  databases migrated by newer binaries and still reference this table;
  dropping it would break that mixed-version path. See
  [here](https://github.com/openai/codex/blob/main/codex-rs/state/src/migrations.rs#L10-L11).
  
  ## Verification
  
  - `just test -p codex-state -p codex-rollout -p codex-thread-store`
  - `just test -p codex-core --test all
  resume_restores_dynamic_tools_from_rollout_with_sqlite_enabled`

  sayan-oai · 2026-05-27 17:57:32 -07:00
  304d15cab0
• [codex] Add rollout-backed thread content search (#23519)

  ## Summary
  - add experimental `thread/search` for local rollout-backed thread
  search using `rg` over JSONL rollouts
  - return search-specific result rows with optional previews instead of
  storing preview data on `StoredThread` or ordinary `Thread` responses
  - keep `thread/list` separate from full-content search and document the
  new app-server surface
  
  ## Testing
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-app-server
  thread_search_returns_content_and_title_matches -- --nocapture`

  Francis Chalissery · 2026-05-21 11:52:24 -07:00
  ac0bff27e7
• Preserve image detail in app-server inputs (#20693)

  ## Summary
  
  - Add optional image detail to user image inputs across core, app-server
  v2, thread history/event mapping, and the generated app-server
  schemas/types.
  - Preserve requested detail when serializing Responses image inputs:
  omitted detail stays on the existing `high` default, while explicit
  `original` keeps local images on the original-resolution path.
  - Support `high`/`original` consistently for tool image outputs,
  including MCP `codex/imageDetail`, code-mode image helpers, and
  `view_image`.

  Curtis 'Fjord' Hawthorne · 2026-05-15 15:04:04 -07:00
  8543e39885
• [codex] Soften SQLite metadata sync failures (#22899)

  ## Summary
  - keep transcript-derived local thread metadata SQLite failures
  best-effort
  - preserve hard failures for explicit git-only metadata updates that
  still require SQLite state
  - add regression coverage for the soft-vs-hard metadata update policy
  
  ## Root cause
  The live thread metadata sync introduced after v0.131.0-alpha.8 moved
  append-derived metadata writes above the rollout writer. Those SQLite
  writes now propagated through the live thread flush path, so a corrupted
  optional state DB could surface as a transcript persistence warning even
  when JSONL writes still succeeded.
  
  The hard failures were introduced in #22236

  Tom · 2026-05-15 21:37:27 +00:00
  249d50aafc
• Unify thread metadata updates above store (#22236)

  - make ThreadStore::update_thread_metadata accept a broad range of
  metadata patches
  - keep ThreadStore::append_items as raw canonical history append (no
  metadata side effects)
  - in the local store, write these metadata updates to a combination of
  sqlite and rollout jsonl files for backwards-compat. It special cases
  which fields need to go into jsonl vs sqlite vs whatever, confining the
  awkwardness to just this implementation
  - in remote stores we can simply persist the metadata directly to a
  database, no special casing required.
  - move the "implicit metadata updates triggered by appending rollout
  items" from the RolloutRecorder (which is local-threadstore-specific) to
  the LiveThread layer above the ThreadStore, inside of a private helper
  utility called ThreadMetadataSync. LiveThread calls ThreadStore
  append_items and update_metadata separately.
  - Add a generic update metadata method to ThreadManager that works on
  both live threads and "cold" threads
  - Call that ThreadManager method from app server code, so app server
  doesn't need to worry about whether the thread is live or not

  Tom · 2026-05-13 00:28:15 +00:00
  c51c65ad09
• Use goal preview metadata for goal-first threads (#21981)

  Fixes #20792
  
  ## Why
  
  `/goal`-first threads are valid resumable threads, but they can be
  missing from `codex resume` and app recents because discovery depends on
  metadata derived from a normal first user message.
  
  PR #21489 attempted to fix this by using the goal objective as
  `first_user_message`. Review feedback pointed out that
  `first_user_message` does more than provide visible text today: it gates
  listing, supplies preview text, and participates in deciding whether a
  later title should surface as a distinct thread name. Reusing it for the
  goal objective could leave a `/goal`-first thread with
  `first_user_message=<goal>` and `title=<later prompt>`, even though the
  goal should only provide the initial visible preview.
  
  This PR follows that feedback by and keeps the `first_user_message` as
  is but introduces a new `preview` field to separate concerns. The
  `preview` field is populated from the first user message or the goal
  objective. We can extend it in the future to include other sources.
  
  ## What Changed
  
  - Added internal thread `preview` metadata in `codex-state`, including a
  SQLite migration that backfills from `first_user_message` and from
  existing `thread_goals` objectives when needed.
  - Treated `ThreadGoalUpdated` as preview-bearing metadata so goal-first
  threads can be listed and searched without mutating
  `first_user_message`.
  - Updated rollout listing, state queries, thread-store conversion, and
  app-server mapping to use preview metadata while continuing to expose
  the existing public `preview` field.
  - Preserved title/name distinctness behavior around literal
  `first_user_message`, so a later normal prompt after `/goal` does not
  surface as a separate name just because the goal supplied the initial
  preview.
  - Preserved compatibility for older/internal metadata writes by deriving
  preview from `first_user_message` when explicit preview metadata is
  absent.
  
  ## Verification
  
  - Manually verified that a thread that starts with a `/goal <objective>`
  shows up in the resume picker.

  Eric Traut · 2026-05-11 10:12:46 -07:00
  f10ddc3f13
• [codex] Remove remote thread store implementation (#21596)

  Remove the remote thread-store backend and checked-in protobuf
  artifacts. We've moved these into another crate that link against this
  one.
  
  Also remove the config settings for thread store backend selection,
  since we'll instead pass an instantiated thread store into the core-api
  crate's main entrypoint.

  Tom · 2026-05-08 00:02:46 +00:00
  79ad209ce6