93 Commits

• Support plaintext agent messages (#27830)

  ## Why
  
  Multi-agent v2 `send_message` deliveries already reach the receiving
  model as typed `agent_message` items with encrypted content.
  Child-completion notifications are generated by Codex itself, so their
  content is plaintext and previously fell back to a serialized JSON
  envelope inside an assistant message.
  
  With plaintext `input_text` supported for `agent_message`, both delivery
  paths can use the same model-visible type while preserving explicit
  author and recipient metadata.
  
  ## What changed
  
  - add plaintext `input_text` support to `AgentMessageInputContent` and
  regenerate the affected app-server schemas
  - preserve `InterAgentCommunication` as structured mailbox input instead
  of converting it to assistant text
  - record delivered communications as typed `agent_message` history items
  - persist a dedicated rollout item so local delivery metadata such as
  `trigger_turn` remains available without leaking into the Responses
  request
  - reconstruct typed agent messages on resume and preserve fork-turn
  truncation behavior
  - remove request-time assistant-content parsing
  - preserve plaintext and encrypted inter-agent deliveries in stage-one
  memory inputs
  - normalize and link plaintext and encrypted agent messages in rollout
  traces without treating inbound messages as child results
  - cover the real MultiAgent V2 child-completion path end to end with
  deterministic mailbox synchronization
  
  ## Verification
  
  - `just test -p codex-core
  plaintext_multi_agent_v2_completion_sends_agent_message`
  - `just test -p codex-core input_queue_drains_mailbox_in_delivery_order
  record_initial_history_reconstructs_typed_inter_agent_message
  fork_turn_positions_use_inter_agent_delivery_metadata`
  - `just test -p codex-memories-write
  serializes_inter_agent_communications_for_memory`
  - `just test -p codex-rollout-trace
  agent_messages_preserve_routing_and_content
  sub_agent_started_activity_creates_spawn_edge`
  - `just test -p codex-rollout-trace
  agent_result_edge_falls_back_to_child_thread_without_result_message`
  - `just test -p codex-protocol -p codex-rollout -p
  codex-app-server-protocol`

  jif · 2026-06-12 13:50:04 -07:00
  8f2d6416ce
• [codex] Remove async_trait from first-party code (#27475)

  ## Why
  
  First-party async traits should expose their `Send` contracts explicitly
  without requiring `async_trait`. This completes the migration pattern
  established in #27303 and #27304.
  
  ## What changed
  
  - Replaced the remaining first-party `async_trait` traits with native
  return-position `impl Future + Send` where statically dispatched and
  explicit boxed `Send` futures where object safety is required.
  - Kept implementations behavior-preserving, outlining existing async
  bodies into inherent methods where that keeps the diff reviewable.
  - Removed all direct first-party `async-trait` dependencies and the
  workspace dependency declaration.
  - Added a cargo-deny policy that permits `async-trait` only through the
  remaining transitive wrapper crates.
  - Updated `rand` from 0.8.5 to 0.8.6 to resolve RUSTSEC-2026-0097 and
  keep the full cargo-deny check passing.
  
  ## Validation
  
  - `just test -p codex-exec-server`: 216 passed, 2 skipped.
  - `just test -p codex-model-provider`: 39 passed.
  - `just test -p codex-core` and `just test`: changed tests passed;
  remaining failures are environment-sensitive suites unrelated to this
  migration.
  - `cargo deny check`
  - `just fix`
  - `just fmt`
  - `cargo shear`
  - `just bazel-lock-check`

  Adam Perry @ OpenAI · 2026-06-11 18:16:39 -07:00
  5a56caf18c
• [codex] Compact when comp_hash changes (#27520)

  ## Summary
  - snapshot `comp_hash` into `TurnContext` when the turn is created and
  use that snapshot as the downstream source of truth
  - persist the turn hash in rollout context and recover it into
  previous-turn settings during resume and fork replay
  - compact existing history with the previous model only when both
  adjacent turns provide hashes and the values differ
  - record `comp_hash_changed` as the compaction reason
  - cover ordinary transitions, resume, and missing-hash compatibility
  with end-to-end tests
  
  ## Why
  History produced under one compaction-compatible model configuration may
  not be safe to carry directly into another. Compacting at the turn
  boundary converts that history before context updates and the new user
  message are added. Persisting the turn snapshot in `TurnContextItem`
  makes the same protection work after resuming a rollout.
  
  A missing hash is not treated as evidence of incompatibility. `None →
  Some`, `Some → None`, and `None → None` do not trigger compaction; only
  `Some(previous) → Some(current)` with unequal values does.
  
  ## Stack
  - depends on #27532
  - #27532 is based directly on `main`
  
  ## Testing
  - `just test -p codex-core pre_sampling_compact_` — 6 passed
  - `just test -p codex-core
  turn_context_item_uses_turn_context_comp_hash_snapshot` — passed
  - `just fix -p codex-core -p codex-protocol -p codex-analytics -p
  codex-models-manager`

  Ahmed Ibrahim · 2026-06-11 04:11:26 +00:00
  ba4925b3c2
• fix: Auto-recover from corrupted sqlite databases (#26859)

  Further investigation of the sqlite incidents showed that the problems
  are due to corruption from the older version of SQLite that we recently
  upgraded, and that the data is truly corrupted in the root database --
  recovery of all data is not possible. Given that the data is
  reconstructable from the rollouts on disk, we should just auto-backup
  the database and let codex rebuild the rollout info from the disk
  rollouts.
  
  The new behavior is that appserver auto-backs-up and rebuilds (with logs
  reflecting that behavior). The CLI now pops a message letting you know
  this happened and the paths of the backed-up corrupt db and the new
  database. There is also context added so that the desktop app can read
  the rebuild info from it and inform the user with it.

  David de Regt · 2026-06-10 11:24:29 -07:00
  3691fe5b76
• Add app-server thread/delete API (#25018)

  ## Why
  
  Clients can archive and unarchive threads today, but there is no
  app-server API for permanently removing a thread. Deletion also needs to
  cover the full session tree: deleting a main thread should remove
  spawned subagent threads and the related local metadata instead of
  leaving orphaned rollout files, goals, or subagent state behind.
  
  ## What
  
  - Adds the v2 `thread/delete` request and `thread/deleted` notification,
  with the response shape kept consistent with `thread/archive`.
  - Implements local hard delete for active and archived rollout files.
  - Deletes the requested thread's state DB row as the commit point, then
  best-effort cleans associated state including spawned descendants,
  goals, spawn edges, logs, dynamic tools, and agent job assignments.
  - Updates app-server API docs and generated protocol schema/TypeScript
  fixtures.

  Eric Traut · 2026-06-10 11:22:12 -07:00
  a19d43a40a
• Fix compressed rollout search path matching (#27407)

  ## Why
  
  `thread/search` found content inside compressed rollouts but could drop
  the result when joining it with SQLite-backed thread metadata. Search
  returned the physical `.jsonl.zst` path while SQLite retained the
  logical `.jsonl` path, so exact path matching failed.
  
  ## What changed
  
  - Key rollout search matches by their canonical logical `.jsonl` path,
  independent of the on-disk representation.
  - Canonicalize thread-list paths before joining them with content-search
  matches.
  - Update compressed-rollout coverage to assert the logical-path
  contract.
  
  ## Validation
  
  - Ran `just fmt`.
  - Ran `git diff --check`.
  - Tests and Clippy were intentionally left to CI.

  jif · 2026-06-10 19:23:42 +02:00
  d3abd8774e
• Reduce archive rollout lookup CPU (#27276)

  ## Why
  
  Archiving a thread can spike app-server CPU when the state DB does not
  have a usable rollout path. The archive path falls back to locating the
  rollout by thread id; because rollout filenames already contain the
  UUID, the cheap fallback should find the file directly before invoking
  broader file search.
  
  ## What Changed
  
  - In `codex-rs/rollout/src/list.rs`, try the exact rollout filename
  lookup before `codex-file-search`.
  - Keep fuzzy search as the final legacy fallback when no filename match
  is found.
  - Preserve the legacy fallback when the filename scan hits a traversal
  error, so an inaccessible stale subtree does not block lookup elsewhere.
  
  ## Verification
  
  - `just test -p codex-rollout`
  - `just test -p codex-thread-store`
  - `just test -p codex-app-server thread_archive`

  Eric Traut · 2026-06-10 09:28:12 -07:00
  c365b8a4ab
• [codex] Store compact window id in rollout (#27264)

  ## Why
  
  Compaction window identity is part of session history, not model-client
  transport state. Persisting it with the compacted rollout item lets
  resumed threads continue from the reconstructed window without keeping
  mutable window state on `ModelClient`.
  
  ## What changed
  
  - Added `window_id` to `CompactedItem` and stamp it when
  `replace_compacted_history` installs compacted history.
  - Moved auto-compact window id ownership into `AutoCompactWindow` /
  `SessionState`; `ModelClient` now receives the request window id from
  callers instead of storing it.
  - Returned `window_id` from rollout reconstruction for resume.
  Reconstruction uses the newest surviving compacted item's stored
  `window_id` when present, and falls back to the legacy compacted-item
  count when it is absent.
  - Kept fork startup at the fresh default window id and updated direct
  model-client tests to pass explicit test window ids.
  
  ## Validation
  
  - `cargo check -p codex-core --tests`

  pakrym-oai · 2026-06-10 08:47:16 -07:00
  30ddb3325e
• multi-agent: add path-based v2 activity tracking (#27007)

  ## Why
  
  Multi-agent v2 identifies agents by canonical paths, but its tool
  handlers still emitted the larger legacy collaboration begin/end events
  built around nickname and role metadata. App-server, rollout-trace,
  analytics, and TUI consumers therefore lacked one compact path-based
  completion signal that behaved consistently across live events and
  replay.
  
  The TUI also needs a bounded `/agent` status surface for v2 agents. It
  should use recent local activity for previews, refresh liveness without
  loading full histories, and keep the legacy picker available when no
  path-backed v2 agent is known.
  
  ## What changed
  
  - Replace the v2 `spawn_agent`, `send_message`, `followup_task`, and
  `interrupt_agent` legacy lifecycle emissions with a success-only
  `SubAgentActivity` event. The event records the tool call ID, occurrence
  time, affected thread, canonical agent path, and `started`,
  `interacted`, or `interrupted` kind.
  - Expose the activity as a completion-only app-server v2
  `subAgentActivity` thread item in live notifications and reconstructed
  history, regenerate the protocol schemas, and count it in sub-agent tool
  analytics.
  - Track canonical paths from live activity and loaded-thread metadata in
  the TUI, and render the activity in live and replayed transcripts.
  - Make `/agent` list running path-backed agents with summaries from
  bounded local event buffers. Each summary is capped at 240 graphemes,
  the scan is capped at six recent items, only the last three wrapped
  lines are shown, and command output is omitted. Liveness falls back to
  metadata-only `thread/read` when local turn state is unavailable.
  - Persist the activity as a terminal rollout-trace runtime payload and
  reduce it to the corresponding spawn, send, follow-up, or close
  interaction edge. `interrupt_agent` is classified as a close-edge
  operation.
  - Preserve the legacy picker when no path-backed v2 agent is known.
  
  ## Compatibility
  
  App-server v2 clients that consumed `collabAgentToolCall` begin/end
  pairs for these tools must handle the new completion-only
  `subAgentActivity` item. Legacy v1 collaboration behavior is unchanged.
  
  ## Screenshot
  
  <img width="684" height="288" alt="Screenshot 2026-06-08 at 15 40 47"
  src="https://github.com/user-attachments/assets/194b3cd0-619d-45fb-b587-cf3e2b1b8a1d"
  />
  
  ## Testing
  
  - `just test -p codex-app-server-protocol`
  - `just test -p codex-rollout-trace`
  - Added focused coverage for activity analytics, terminal trace
  serialization, spawn-edge reduction, `interrupt_agent` classification,
  TUI status rendering without aggregated command output, and clearing
  stale running state after a completed turn.

  jif · 2026-06-09 12:14:48 +02:00
  fae2709320
• [codex] Forward turn moderation metadata through app-server (#25710)

  ## Why
  First-party backends can supply turn-scoped moderation metadata that
  app-server clients need for client-side presentation. Exposing this as
  an experimental typed notification lets opted-in clients consume it
  without interpreting raw Responses API events.
  
  ## What changed
  - forward `response.metadata.openai_chatgpt_moderation_metadata` from
  Responses API SSE and WebSocket streams as turn-scoped moderation
  metadata
  - emit the experimental app-server v2 `turn/moderationMetadata`
  notification with `{ threadId, turnId, metadata }`
  - add app-server integration coverage for the typed moderation metadata
  notification
  
  ## Testing
  - `just test -p codex-core
  build_ws_client_metadata_includes_window_lineage_and_turn_metadata`
  - `just test -p codex-core` (fails locally: 46 failures and 1 timeout,
  primarily missing `test_stdio_server` and shell snapshot timeouts)
  - `just test -p codex-app-server-protocol`
  - `just test -p codex-app-server
  turn_moderation_metadata_emits_typed_notification_v2`
  - `just test -p codex-app-server` (fails locally: 792 passed, 10 failed,
  and 5 timed out; failures are in existing environment-sensitive tests,
  primarily because nested macOS `sandbox-exec` is not permitted)
  - `just write-app-server-schema --experimental --schema-root
  /tmp/codex-app-server-schema-experimental`

  carlc-oai · 2026-06-05 02:41:06 -07:00
  55aa071b17
• Encrypt multi-agent v2 message payloads (#26210)

  ## Why
  
  Multi-agent v2 currently routes agent instructions through normal tool
  arguments and inter-agent context. That means the parent model can emit
  plaintext task text, Codex can persist it in history/rollouts, and the
  recipient can receive it as ordinary assistant-message JSON.
  
  This changes the v2 path so agent instructions stay encrypted between
  model calls: Responses encrypts the `message` argument returned by the
  model, Codex forwards only that ciphertext, and Responses decrypts it
  internally for the recipient model.
  
  ## What changed
  
  - Mark the v2 `message` parameter as encrypted for `spawn_agent`,
  `send_message`, and `followup_task`.
  - Treat multi-agent v2 tool `message` values as ciphertext
  unconditionally.
  - Store v2 inter-agent task text in
  `InterAgentCommunication.encrypted_content` with empty plaintext
  `content`.
  - Convert encrypted inter-agent communications into the Responses
  `agent_message` input item before sending the child request.
  - Preserve `agent_message` items across history, rollout, compaction,
  telemetry, and app-server schema paths.
  - Leave multi-agent v1 unchanged.
  
  ## Message shape
  
  The model still calls the v2 tools with a `message` argument, but that
  value is now ciphertext:
  
  ```json
  {
    "name": "spawn_agent",
    "arguments": {
      "task_name": "worker",
      "message": "<ciphertext>"
    }
  }
  ```
  
  Codex stores the task as encrypted inter-agent communication:
  
  ```json
  {
    "author": "/root",
    "recipient": "/root/worker",
    "content": "",
    "encrypted_content": "<ciphertext>",
    "trigger_turn": true
  }
  ```
  
  When Codex builds the recipient request, it forwards the ciphertext
  using the new Responses input item:
  
  ```json
  {
    "type": "agent_message",
    "author": "/root",
    "recipient": "/root/worker",
    "content": [
      {
        "type": "encrypted_content",
        "encrypted_content": "<ciphertext>"
      }
    ]
  }
  ```
  
  Responses decrypts that item internally for the recipient model.
  
  ## Context impact
  
  - Parent context no longer carries plaintext v2 agent task instructions
  from these tool arguments.
  - Codex rollout/history stores ciphertext for v2 agent instructions.
  - Recipient requests receive an `agent_message` item instead of
  assistant commentary JSON for encrypted task delivery.
  - Plaintext completion/status notifications are still plaintext because
  they are Codex-generated status messages, not encrypted model tool
  arguments.
  
  ## Validation
  
  - `just test -p codex-tools`
  - `just test -p codex-protocol`
  - `just test -p codex-rollout`
  - `just test -p codex-rollout-trace`
  - `just test -p codex-otel`
  - `just write-app-server-schema`

  jif · 2026-06-05 10:25:57 +02:00
  5f4d06ef18
• Persist multi-agent runtime metadata (#25721)

  Stack split from #25708. Original PR intentionally left open. This
  second PR persists multi-agent runtime metadata through thread creation,
  rollout recording, and thread storage.

  jif-oai · 2026-06-02 13:05:20 +02:00
  0c5ccd18ab
• Add multi-agent runtime metadata types (#25720)

  Stack split from #25708. Original PR intentionally left open. This first
  PR adds the multi-agent runtime metadata types and catalog plumbing used
  by the rest of the stack.

  jif-oai · 2026-06-02 12:10:14 +02:00
  3f1fb7ed8b
• feat: reuse compressed rollout search snippets (#25814)

  ## Summary
  - teach rollout search to return precomputed snippets for compressed
  rollouts
  - reuse those snippets in local thread search instead of reopening
  matching compressed files
  - keep the no-`rg` fallback single-pass and add regression coverage for
  the compressed path
  
  ## Why
  `thread/search` currently decodes matching compressed rollouts twice:
  once to discover the matching path and again to extract the snippet
  shown in results. That defeats a meaningful part of the compressed-read
  optimization work.
  
  ## Impact
  Compressed rollout hits now pay one decode pass on the search path while
  plain `.jsonl` hits keep the existing ripgrep-driven flow.
  
  ## Validation
  - `just test -p codex-rollout`
  - `just test -p codex-thread-store`
  - `just fix -p codex-rollout`
  - `just fix -p codex-thread-store`
  - `just fmt`

  jif-oai · 2026-06-02 11:32:36 +02:00
  45912a6dc6
• app-server: remove experimental persist_extended_history bool flag (#25712)

  ## Summary
  
  Remove the dead experimental `persistExtendedHistory` app-server flag
  and collapse rollout persistence to the single policy app-server already
  used.
  
  ## What Changed
  
  - Removed `persistExtendedHistory` from v2 thread start/resume/fork
  params and deleted its deprecation notice path.
  - Removed the persistence-mode enums and plumbing through core, rollout,
  and thread-store.
  - Made rollout filtering mode-free, keeping the existing limited
  persisted-history behavior.
  
  ## Test Plan
  
  - `just write-app-server-schema`
  - `cargo nextest run --no-fail-fast -p codex-app-server-protocol
  schema_fixtures`
  - `cargo nextest run --no-fail-fast -p codex-app-server
  thread_shell_command_history_responses_exclude_persisted_command_executions`
  - `cargo nextest run --no-fail-fast -p codex-rollout -p
  codex-thread-store`
  - final `rg` for removed flag/type names

  Owen Lin · 2026-06-01 23:33:42 +00:00
  11e0f3d3ae
• Reject directory rollout paths for pathless side chats (#25661)

  ## Why
  
  Fixes openai/codex#20944.
  
  Desktop side chats are intentionally ephemeral and pathless. They can
  still accept live turns while loaded, but after a reload there is no
  persisted rollout to resume. In the reported failure mode, Desktop could
  send `$CODEX_HOME` as the resume/fork path for one of these pathless
  side chats.
  
  `thread/resume` and `thread/fork` prefer an explicit `path` over
  `threadId`, and rollout path lookup only checked that a candidate
  existed. That let `$CODEX_HOME` pass as a rollout path, so the later
  rollout reader tried to open a directory and surfaced the low-level `Is
  a directory` error.
  
  ## What Changed
  
  - Reject explicit rollout paths that resolve to a directory or other
  non-file before attempting to read rollout history.
  - Make `codex_rollout::existing_rollout_path` return only plain or
  compressed rollout candidates that are actual files.
  - Add an app-server regression test that creates an ephemeral fork, runs
  a turn while the side thread is loaded, simulates reload, then verifies
  both `thread/resume` and `thread/fork` reject `$CODEX_HOME` with `path
  is a directory` instead of the OS-level directory-read error.
  - Rebase over the `TestAppServer` rename and update the remaining stale
  test harness call sites to use `TestAppServer` with `app_server` local
  variables.
  
  Relevant code:
  
  - `thread-store/src/local/read_thread.rs` validates explicit rollout
  paths before rollout reading:
  https://github.com/openai/codex/blob/25b47c8f425d351aaba4baa955a8092064a1707b/codex-rs/thread-store/src/local/read_thread.rs#L146-L165
  - `rollout/src/compression.rs` now requires file metadata for plain and
  compressed rollout candidates:
  https://github.com/openai/codex/blob/25b47c8f425d351aaba4baa955a8092064a1707b/codex-rs/rollout/src/compression.rs#L940-L950
  - The repro test covers the pathless ephemeral side-chat reload case:
  https://github.com/openai/codex/blob/25b47c8f425d351aaba4baa955a8092064a1707b/codex-rs/app-server/tests/suite/v2/thread_fork.rs#L774-L886
  
  ## Verification
  
  - `just test -p codex-app-server
  pathless_ephemeral_thread_rejects_codex_home_path_after_reload`

  Michael Bolin · 2026-06-01 16:02:06 -07:00
  b89bf1ef47
• Add rollout compression histograms (#25680)

  ## Summary
  
  Stacked on #25679. Add histogram telemetry for rollout compression
  runtime, per-file compression time, byte sizes, and compression ratio.
  
  ## Changes
  
  - Emit `codex.rollout_compression.run.duration_ms` tagged by final run
  status.
  - Emit `codex.rollout_compression.file.duration_ms` tagged by file
  outcome.
  - Emit source and compressed byte histograms for compression
  candidates/results.
  - Emit `codex.rollout_compression.file.compression_ratio` for successful
  compressions, recorded as integer basis points.
  
  ## Validation
  
  - `just fmt`
  - `just test -p codex-rollout`
  - `just fix -p codex-rollout`

  jif-oai · 2026-06-01 22:54:25 +02:00
  7c285f9e9c
• Add rollout compression counters (#25679)

  ## Summary
  
  Add counter telemetry for the local rollout compression worker so we can
  see when it runs, why it skips, and how individual file/materialization
  paths resolve.
  
  ## Changes
  
  - Emit `codex.rollout_compression.run` with statuses for start,
  completion, failure, duplicate-run skip, and missing runtime skip.
  - Emit `codex.rollout_compression.file` outcomes for scanned,
  compressed, skipped, and failed compression candidates.
  - Emit `codex.rollout_compression.temp_cleanup` and
  `codex.rollout_compression.materialize` counters for cleanup and
  decompression paths.
  
  ## Validation
  
  - `just fmt`
  - `just test -p codex-rollout`
  - `just fix -p codex-rollout`

  jif-oai · 2026-06-01 22:26:32 +02:00
  9f4fac8ec4
• Throttle repeated rollout compression runs (#25659)

  ## Why
  
  [#25089](https://github.com/openai/codex/pull/25089) introduced the
  background worker that compresses cold archived rollouts, and
  [#25654](https://github.com/openai/codex/pull/25654) made that pass
  faster once it starts. But the worker still deleted
  `rollout-compression.lock` on successful exit, so the existing six-hour
  staleness window only helped with overlapping or crashed workers. Each
  new local thread-store initialization could immediately rescan archived
  rollouts even if a full pass had just finished.
  
  This change keeps the existing marker around long enough to throttle
  redundant reruns. The worker is still best-effort, but it no longer does
  repeated startup scans when nothing new is eligible for compression.
  
  ## What Changed
  
  - Replace the drop-scoped `CompressionLock` with a
  `CompressionRunMarker` that claims the existing
  `.tmp/rollout-compression.lock` path and leaves it in place after
  success.
  - Reuse the existing six-hour staleness window to block both overlapping
  starts and immediate reruns, while still letting a stale marker be
  reclaimed.
  - Update the worker docs and debug logging to describe the new "already
  running or recently ran" behavior.
  - Extend the rollout compression tests to assert that a successful run
  leaves the marker behind and that a fresh marker suppresses a new run.
  
  ## Validation
  
  - `just test -p codex-rollout`

  jif-oai · 2026-06-01 20:46:54 +02:00
  c3cdf3c007
• Parallelize cold rollout compression (#25654)

  ## Why
  
  [#25089](https://github.com/openai/codex/pull/25089) added the
  background worker for compressing cold archived rollouts, but the worker
  still processed files effectively one at a time: each compression job
  was sent to `spawn_blocking` and then awaited before the next file
  started. On machines with a backlog of archived rollouts, that makes
  catch-up slower than it needs to be even though the actual compression
  work already runs off the async runtime.
  
  ## What Changed
  
  - Queue rollout compression work in a `JoinSet` while directory
  traversal continues.
  - Cap the worker at two in-flight compression jobs so it can overlap
  compression without turning the background task into unbounded blocking
  work.
  - Drain pending jobs before returning, including the
  `read_dir.next_entry()` error path, so every launched job still
  contributes to the final `compressed`, `skipped`, and `failed` stats.
  - Treat task join failures the same way as compression failures in the
  worker's warning and failure accounting.

  jif-oai · 2026-06-01 19:54:52 +02:00
  917a9a41a3
• Compress cold local rollouts (#25089)

  ## Rollout compression stack
  
  This stack splits #24941 into reviewable steps for local rollout
  compression. The design is intentionally staged:
  
  1. Teach readers, listing, search, and lookup to understand compressed
  rollouts.
  2. Make append and resume paths materialize compressed rollouts back to
  plain JSONL before writing.
  3. Add a disabled-by-default worker that can compress cold archived
  rollouts behind `local_thread_store_compression`.
  
  The key invariant is that writers append to plain `.jsonl`. A
  `.jsonl.zst` file is a cold/read representation; if a write is needed,
  the compressed file is materialized back to plain JSONL first. Readers
  prefer plain `.jsonl` when both forms exist and can fall back to the
  compressed sibling during transitions.
  
  The worker is deliberately the last PR and remains behind an
  under-development feature flag. It currently scans only
  `archived_sessions`, not active `sessions`, because active sessions have
  the highest resume/append race risk. That means this stack does not yet
  compress most unarchived local history.
  
  ## Known race / follow-up
  
  The remaining unresolved design question is writer/compressor
  coordination. Even for archived rollouts, a resume or metadata update
  can append while the worker is replacing the plain file with
  `.jsonl.zst`; the current double-stat checks narrow but do not fully
  eliminate the window where a writer has opened the plain file before
  unlink. Do not treat the worker PR as production-ready until we either:
  
  - prevent append/resume paths from racing archived compression, or
  - introduce a shared representation/append lock or equivalent
  coordination.
  
  The first two PRs are useful independently: they make compressed
  rollouts readable and make append paths safely recover back to plain
  JSONL. The third PR isolates the worker behavior so that coordination
  issue is reviewable separately.
  
  ## Validation
  
  Focused local validation for the stack includes:
  
  - `just test -p codex-rollout`
  - `just test -p codex-thread-store` where thread-store paths were
  touched
  - `just test -p codex-features` for the feature flag slice
  - `just bazel-lock-check` after dependency graph changes
  - scoped `just fix -p ...` passes for changed crates
  
  CI is still the source of truth for the full platform matrix.
  
  ## This PR in the stack
  
  This is PR 3/3, based on #25088. It adds the under-development feature
  flag and starts the best-effort background worker when enabled. The
  worker currently compresses only cold archived rollouts, skips active
  sessions, verifies compressed output, preserves mtime and permissions,
  keeps a store-level lock heartbeat, and cleans stale temp files.
  
  Stack order:
  
  1. #25087: read compressed local rollouts.
  2. #25088: materialize compressed rollouts before append.
  3. This PR: add the disabled local compression worker.

  jif-oai · 2026-06-01 18:35:58 +02:00
  01cb97851b
• Preserve renamed thread titles during reconciliation (#25624)

  ## Summary
  - preserve existing explicit SQLite thread titles during rollout
  reconciliation/backfill when the incoming rollout title is only
  first-message-derived
  - keep stale inferred-title repair behavior while avoiding session-index
  scans during startup backfill
  - add a regression test for renamed titles surviving reconcile
  
  ## Testing
  - just fmt
  - just test -p codex-rollout
  - just test -p codex-state

  jif-oai · 2026-06-01 18:33:05 +02:00
  3cdce52865
• [codex] fix compressed rollout fixture SessionMeta initialization (#25628)

  ## Summary
  - initialize `parent_thread_id` in the compressed rollout test fixture's
  `SessionMeta`
  - restore rollout test compilation across Bazel test, clippy,
  release-build, and argument-comment-lint jobs
  
  ## Root cause
  PR #25087 (`Read compressed rollouts and materialize before append`)
  added `codex-rs/rollout/src/compression_tests.rs` in merge commit
  `a8a6071279b6f3112fcc5fc3fee69c48473d7149`. Its `write_rollout` fixture
  constructs `SessionMeta` without the required `parent_thread_id` field,
  causing `error[E0063]` when Bazel compiles `rollout-unit-tests-bin` on
  `main` and downstream PRs.
  
  ## Validation
  - `UV_CACHE_DIR=/private/tmp/codex-uv-cache just fmt`
  - `just test -p codex-rollout` (`59` tests passed; bench smoke passed)
  - `git diff --check`
  - manually audited the touched Rust diff for positional literal argument
  comments; the change adds no positional callsite
  
  ## Local lint blocker
  - `just argument-comment-lint` could not reach source inspection locally
  because Bazel's LLVM dependency fails analysis:
  `compiler-rt/BUILD.bazel` glob `include/sanitizer/*.h` matched no files.

  Felipe Coury · 2026-06-01 16:43:21 +02:00
  4eded02f52
• Read compressed rollouts and materialize before append (#25087)

  ## Why
  
  Local rollout compression needs a cold `.jsonl.zst` representation
  without letting compressed physical paths leak into append-mode writers.
  The unsafe case is resume or metadata update code successfully reading a
  compressed rollout and then appending raw JSONL bytes to the zstd file.
  
  This PR folds the former #25088 materialization slice into the
  read-support PR so the reader changes and append-safety invariant land
  together.
  
  ## What Changed
  
  - Teach rollout readers, discovery, listing, search, and ID lookup to
  understand compressed `.jsonl.zst` rollouts.
  - Keep `.jsonl` as the logical/stored rollout path while allowing read
  paths to open either plain or compressed storage.
  - Materialize compressed rollouts back to plain `.jsonl` before
  append-mode writes, including resume and direct metadata append paths.
  - Preserve compressed-file permissions when materializing back to plain
  JSONL.
  - Refresh thread-store resolved rollout paths after compatibility
  metadata writes so reconciliation follows the materialized file.
  - Avoid treating transient compression temp files as real rollout lookup
  results.
  
  ## Remaining Stack
  
  #25089 remains the separate worker PR. It is based directly on this PR
  and stays behind the disabled `local_thread_store_compression` feature
  flag.
  
  The worker still has a broader coordination question: a resume or
  metadata update can race with background compression while a plain file
  is being replaced by `.jsonl.zst`. This PR handles the read and
  materialize-before-append primitives; it does not make the worker
  production-ready.
  
  ## Validation
  
  - `just test -p codex-rollout`
  - `just test -p codex-thread-store`
  - `just fix -p codex-rollout`
  - `just fix -p codex-thread-store`
  - `just bazel-lock-check`

  jif-oai · 2026-06-01 15:14:19 +02:00
  a8a6071279
• store and expose parent_thread_id on Threads (#25113)

  ## Why
  
  This PR
  https://github.com/openai/codex/pull/24161#discussion_r3325692763
  revealed a subagent data modeling issue, where we overloaded
  `forked_from_id` to also mean `parent_thread_id`. That's incorrect since
  guardian and review subagents can be a subagent and NOT fork the main
  thread's history.
  
  The solution here is to explicitly store a new `parent_thread_id` on
  `SessionMeta`, alongside `forked_from_id` which already exists. While
  we're at it, also expose it in the app-server protocol on the `Thread`
  object.
  
  A thread->subagent relationship and a fork of thread history are
  orthogonal concepts.
  
  ## What Changed
  
  - Added top-level `parent_thread_id` persistence on `SessionMeta` and
  runtime/session plumbing through `SessionConfiguredEvent`,
  `CodexSpawnArgs`, `SessionConfiguration`, `ThreadConfigSnapshot`,
  `TurnContext`, and `ModelClient`.
  - Made turn metadata, request headers, analytics, and subagent-start
  events read the separate runtime/top-level parent field instead of
  deriving general parent lineage from `SessionSource` or
  `forked_from_thread_id`.
  - Passed parent lineage separately at delegated subagent, review,
  guardian, agent-job, and multi-agent spawn construction sites;
  copied-history fork lineage remains derived only from `InitialHistory`.
  - Persisted and exposed parent lineage through rollout/thread-store
  projections and app-server v2 `Thread.parentThreadId`.
  - Updated app-server README text and regenerated app-server schema
  fixtures for the additive `parentThreadId` response field.

  Owen Lin · 2026-06-01 04:33:20 +00:00
  cf0911076f
• Remove stale rollout TODO tests (#25106)

  ## Summary
  
  Remove a stale `TODO(jif)` block of commented-out rollout listing tests
  that still referenced an older listing API.
  
  The current rollout listing behavior is covered by the active state DB
  and filesystem fallback tests, so keeping the dead commented tests just
  adds noise.
  
  ## Validation
  
  - `just fmt`
  - `just test -p codex-rollout`

  jif-oai · 2026-05-29 17:09:00 +02:00
  b40ad0d84d
• Surface filesystem permission profiles in prompt context (#23924)

  ## Summary
  Some permission profiles can encode filesystem reads that should remain
  unavailable to the agent. Before this change, the model-visible context
  and automatic approval review prompt summarized the effective
  permissions as a legacy sandbox mode, which can omit permission-profile
  filesystem entries from escalation decisions.
  
  For example, a profile can grant workspace access while denying a
  private subtree across every workspace root:
  
  ```toml
  default_permissions = "restricted-workspace"
  
  [permissions.restricted-workspace.workspace_roots]
  "/Users/alice/project" = true
  "/Users/alice/other-project" = true
  
  [permissions.restricted-workspace.filesystem]
  ":minimal" = "read"
  
  [permissions.restricted-workspace.filesystem.":workspace_roots"]
  "." = "write"
  "private" = "deny"
  "private/**" = "deny"
  ```
  
  The context window now describes the workspace roots and effective
  filesystem side of the `PermissionProfile` directly, with deny entries
  marked as non-escalatable:
  
  ```xml
  <environment_context>
    <cwd>/Users/alice/project</cwd>
    <shell>zsh</shell>
    <filesystem><workspace_roots><root>/Users/alice/project</root><root>/Users/alice/other-project</root></workspace_roots><permission_profile type="managed"><file_system type="restricted"><entry access="read"><special>:minimal</special></entry><entry access="write"><path>/Users/alice/project</path></entry><entry access="write"><path>/Users/alice/other-project</path></entry><entry access="deny" escalatable="false"><path>/Users/alice/project/private</path></entry><entry access="deny" escalatable="false"><path>/Users/alice/other-project/private</path></entry><entry access="deny" escalatable="false"><glob>/Users/alice/project/private/**</glob></entry><entry access="deny" escalatable="false"><glob>/Users/alice/other-project/private/**</glob></entry></file_system></permission_profile></filesystem>
  </environment_context>
  ```
  
  Managed requirements can impose the same kind of deny-read restriction:
  
  ```toml
  [permissions.filesystem]
  deny_read = [
    "/Users/alice/project/private",
    "/Users/alice/project/private/**",
  ]
  ```
  
  The automatic approval review prompt also receives the parent turn's
  denied-read context, so review decisions can account for the active
  permission profile.
  
  ## What Changed
  - Render the effective filesystem profile in `<environment_context>`,
  including profile type, filesystem entries, workspace roots, and
  non-escalatable deny entries.
  - Persist effective `workspace_roots` in `TurnContextItem` so
  resumed/replayed context does not have to bind `:workspace_roots`
  through legacy `cwd` fallback.
  - Add explicit permission instructions that denied reads are policy
  restrictions, not escalation targets.
  - Pass the parent turn's denied-read context into automatic approval
  reviews.
  - Add targeted coverage for prompt rendering, workspace-root
  materialization, replay context, and review prompt context.
  - Keep the prompt-context test expectations platform-aware so the same
  filesystem rendering assertions pass on Unix and Windows paths.
  
  ## Testing
  - `just test -p codex-core
  context::environment_context::tests::serialize_environment_context_with_full_filesystem_profile`
  - `just test -p codex-core
  context::environment_context::tests::turn_context_item_filesystem_uses_workspace_roots_instead_of_cwd`
  - `just test -p codex-core
  context::permissions_instructions::permissions_instructions_tests::builds_permissions_from_profile_with_denied_reads`
  - `just fix -p codex-core`
  
  I also attempted `just test -p codex-core`; the changed prompt-context
  tests passed, but the full local run did not complete cleanly in this
  sandboxed macOS environment due unrelated user-shell `CODEX_SANDBOX*`
  expectations and integration-test timeouts.

  Michael Bolin · 2026-05-28 14:56:53 -07:00
  e7dda8070e
• [codex] Add user input client ids (#24653)

  ## Summary
  
  Adds an optional `clientId` field to app-server v2 `UserInput` and
  carries it through the core `UserInput` model so clients can correlate
  echoed user input items without relying on payload equality.
  
  ## Details
  
  - Adds `client_id: Option<String>` to core `UserInput` variants.
  - Exposes the v2 app-server field as `clientId` on the wire and in
  generated TypeScript.
  - Preserves the id when converting between app-server v2 and core
  protocol types.
  - Regenerates app-server schema fixtures.
  
  ## Validation
  
  - `just fmt`
  - `just write-app-server-schema`
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-protocol`
  - `just fix -p codex-app-server-protocol`
  - `just fix -p codex-protocol`
  - `git diff --check`

  Alexi Christakis · 2026-05-28 14:54:39 -07:00
  e92c952b2e
• [codex] Remove redundant SQLite dynamic tool storage (#24819)

  ## Why
  
  Dynamic tools are defined at thread start and already stored in rollout
  `SessionMeta`, which restores resumed and forked sessions. Persisting
  the same tools through SQLite creates a second runtime persistence path
  that is unnecessary prework for the explicit namespace refactor.
  
  ## What changed
  
  - Restore missing thread-start dynamic tools directly from rollout
  history, including when SQLite is enabled.
  - Remove SQLite dynamic-tool reads, writes, backfill, and thread
  metadata patch plumbing.
  - Add SQLite-enabled resume integration coverage that verifies a
  rollout-defined dynamic tool is still sent after resume.
  
  ## Compatibility
  
  The existing `thread_dynamic_tools` table is intentionally not dropped
  even though it's now unused. Older Codex binaries are allowed to open
  databases migrated by newer binaries and still reference this table;
  dropping it would break that mixed-version path. See
  [here](https://github.com/openai/codex/blob/main/codex-rs/state/src/migrations.rs#L10-L11).
  
  ## Verification
  
  - `just test -p codex-state -p codex-rollout -p codex-thread-store`
  - `just test -p codex-core --test all
  resume_restores_dynamic_tools_from_rollout_with_sqlite_enabled`

  sayan-oai · 2026-05-27 17:57:32 -07:00
  304d15cab0
• Move memory state to a dedicated SQLite DB (#24591)

  ## Summary
  
  Generated memory rows and their stage-one/stage-two job state currently
  live in `state_5.sqlite` alongside thread metadata. That makes memory
  cleanup and regeneration share the main state schema even though those
  rows are memory-pipeline data and can be rebuilt independently from the
  durable thread records.
  
  This PR moves the memory-owned tables into a dedicated
  `memories_1.sqlite` runtime database while keeping thread metadata in
  `state_5.sqlite`.
  
  ## Changes
  
  - Adds a separate memories DB runtime, migrator, path helpers, telemetry
  kind, and Bazel compile data for `state/memory_migrations`.
  - Introduces `MemoryStore` behind `StateRuntime::memories()` and moves
  memory table/job operations onto that store.
  - Drops the old memory tables from the state DB and recreates their
  schema in `state/memory_migrations/0001_memories.sql`.
  - Updates memory startup, citation usage tracking, rollout pollution
  handling, `debug clear-memories`, and app-server `memory/reset` to
  operate through the memories DB.
  - Preserves cross-DB behavior by hydrating thread metadata from the
  state DB when selecting visible memory outputs and checking stage-one
  staleness.
  
  ## Verification
  
  - Added/updated `codex-state` tests for deleted-thread memory visibility
  and already-polluted phase-two enqueue behavior.
  - Updated `debug clear-memories`, app-server `memory/reset`, and
  memories startup tests to seed and assert memory rows through
  `memories_1.sqlite`.

  jif-oai · 2026-05-26 20:07:25 +02:00
  aad59a0916
• Log rollout writer OS errors (#24474)

  ## Why
  
  Refs #24425.
  
  We have seen rollout JSONL corruption that appears consistent with a
  rollout write failing after partially appending a line, followed by a
  retry that appends the same item again. The available user logs did not
  include the underlying OS error, so it is hard to tell whether the
  trigger was `ENOSPC`, quota exhaustion, a filesystem error, or something
  else.
  
  This PR adds the missing diagnostics for future reports.
  
  ## What changed
  
  - Include `ErrorKind` and `raw_os_error()` in rollout writer failure
  logs.
  - Preserve the existing append-only rollout write path; this PR is
  diagnostic-only.
  
  ## Verification
  
  - `just test -p codex-rollout`

  Eric Traut · 2026-05-26 10:33:22 +02:00
  e8651516f4
• [codex] Make thread search case-insensitive (#23921)

  ## Summary
  - make rollout content search prefilter rollout files case-insensitively
  - keep the no-ripgrep fallback scan and visible snippet matcher aligned
  with that behavior
  - cover a lowercase `thread/search` query matching mixed-case
  conversation content
  
  ## Why
  The rollout-backed `thread/search` path used exact string matching in
  both its `rg` prefilter and semantic snippet generation. A content
  result could be missed solely because the query casing did not match the
  stored conversation text.
  
  ## Validation
  - `just fmt`
  - `cargo test -p codex-app-server thread_search_returns_content_matches`
  - `cargo test -p codex-rollout`
  - `just bazel-lock-update`
  - `just bazel-lock-check`
  - `cargo build -p codex-cli`
  - launched a local Electron dev instance with the rebuilt CLI binary

  Francis Chalissery · 2026-05-21 14:14:01 -07:00
  05cf2fc4ce
• [codex] Add rollout-backed thread content search (#23519)

  ## Summary
  - add experimental `thread/search` for local rollout-backed thread
  search using `rg` over JSONL rollouts
  - return search-specific result rows with optional previews instead of
  storing preview data on `StoredThread` or ordinary `Thread` responses
  - keep `thread/list` separate from full-content search and document the
  new app-server surface
  
  ## Testing
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-app-server
  thread_search_returns_content_and_title_matches -- --nocapture`

  Francis Chalissery · 2026-05-21 11:52:24 -07:00
  ac0bff27e7
• [5 of 7] Replace OverrideTurnContext with ThreadSettings (#22508)

  **Stack position:** [5 of 7]
  
  ## Summary
  
  This PR adds `Op::ThreadSettings`, a queued settings-only update
  mechanism for changing stored thread settings without starting a new
  turn. It also removes the legacy `Op::OverrideTurnContext` in the same
  layer, so reviewers can see the replacement and deletion together.
  
  ## Changes
  
  - Add `Op::ThreadSettings` for settings-only queued updates.
  - Emit `ThreadSettingsApplied` with the effective thread settings
  snapshot after core applies an update.
  - Route settings-only updates through the same submission queue as user
  input.
  - Migrate remaining `OverrideTurnContext` tests and callers to the
  queued `Op::ThreadSettings` path.
  - Delete `Op::OverrideTurnContext` from the core protocol and submission
  loop.
  
  This stack addresses #20656 and #22090.
  
  ## Stack
  
  1. [1 of 7] [Add thread settings to
  UserInput](https://github.com/openai/codex/pull/23080)
  2. [2 of 7] [Remove
  UserInputWithTurnContext](https://github.com/openai/codex/pull/23081)
  3. [3 of 7] [Remove
  UserTurn](https://github.com/openai/codex/pull/23075)
  4. [4 of 7] [Placeholder for OverrideTurnContext
  cleanup](https://github.com/openai/codex/pull/23087)
  5. [5 of 7] [Replace OverrideTurnContext with
  ThreadSettings](https://github.com/openai/codex/pull/22508) (this PR)
  6. [6 of 7] [Add app-server thread settings
  API](https://github.com/openai/codex/pull/22509)
  7. [7 of 7] [Sync TUI thread
  settings](https://github.com/openai/codex/pull/22510)

  Eric Traut · 2026-05-18 21:03:51 -07:00
  a668379abf
• [codex] Trim unused TurnContextItem fields (#22709)

  ## Why
  
  `TurnContextItem` is the durable baseline used to reconstruct context
  diffs across resume/fork. Most of the old persisted-only fields on it
  are no longer read, so keeping them in rollout snapshots adds schema
  surface and state that can drift without affecting reconstruction.
  
  `summary` is the exception: older Codex versions require it to
  deserialize `turn_context` records, so keep writing a default
  compatibility value until that schema surface can be removed safely.
  
  ## What changed
  
  - Removed the unused persisted fields from `TurnContextItem`: trace ids,
  user/developer instructions, output schema, and truncation policy.
  - Kept `summary` with a compatibility comment and made
  `TurnContext::to_turn_context_item` write `ReasoningSummary::Auto`
  instead of live turn state.
  - Updated rollout/context reconstruction fixtures for the retained
  summary field.
  
  ## Verification
  
  - `cargo test -p codex-protocol --lib turn_context_item`
  - `cargo test -p codex-rollout
  resume_candidate_matches_cwd_reads_latest_turn_context`
  - `cargo test -p codex-state turn_context`
  - `cargo test -p codex-core --lib
  new_default_turn_captures_current_span_trace_id`
  - `cargo test -p codex-core --lib
  record_initial_history_resumed_turn_context_after_compaction_reestablishes_reference_context_item`
  - `cargo test -p codex-core --test all
  emits_warning_when_resumed_model_differs`
  - `git diff --check`

  pakrym-oai · 2026-05-18 21:54:36 +00:00
  f2368b7de6
• Preserve image detail in app-server inputs (#20693)

  ## Summary
  
  - Add optional image detail to user image inputs across core, app-server
  v2, thread history/event mapping, and the generated app-server
  schemas/types.
  - Preserve requested detail when serializing Responses image inputs:
  omitted detail stays on the existing `high` default, while explicit
  `original` keeps local images on the original-resolution path.
  - Support `high`/`original` consistently for tool image outputs,
  including MCP `codex/imageDetail`, code-mode image helpers, and
  `view_image`.

  Curtis 'Fjord' Hawthorne · 2026-05-15 15:04:04 -07:00
  8543e39885
• [codex] Use compaction_trigger item for remote compaction v2 (#22809)

  ## Why
  
  Remote compaction v2 was still using `context_compaction` as both the
  request trigger and the compacted output shape. The Responses API now
  has the landed contract for this flow: Codex sends a dedicated `{
  "type": "compaction_trigger" }` input item, and the backend returns the
  standard `compaction` output item with encrypted content.
  
  This aligns the v2 path with that wire contract while preserving the
  existing local compacted-history post-processing behavior.
  
  ## What changed
  
  - Add `ResponseItem::CompactionTrigger` and regenerate the app-server
  protocol schema fixtures.
  - Send `compaction_trigger` from `remote_compaction_v2` instead of a
  payload-less `context_compaction`.
  - Collect exactly one backend `compaction` output item, then reuse the
  existing compacted-history rebuilding path.
  - Treat the trigger item as a transient request marker rather than model
  output or persisted rollout/memory content.
  
  ## Verification
  
  - `cargo test -p codex-protocol compaction_trigger`
  - `cargo test -p codex-core remote_compact_v2`
  - `cargo test -p codex-core compact_remote_v2`
  - `cargo test -p codex-core
  responses_websocket_sends_response_processed_after_remote_compaction_v2`
  - `just write-app-server-schema`
  - `cargo test -p codex-app-server-protocol schema_fixtures`

  jif-oai · 2026-05-15 11:40:35 +02:00
  0322ac3df8
• Unify thread metadata updates above store (#22236)

  - make ThreadStore::update_thread_metadata accept a broad range of
  metadata patches
  - keep ThreadStore::append_items as raw canonical history append (no
  metadata side effects)
  - in the local store, write these metadata updates to a combination of
  sqlite and rollout jsonl files for backwards-compat. It special cases
  which fields need to go into jsonl vs sqlite vs whatever, confining the
  awkwardness to just this implementation
  - in remote stores we can simply persist the metadata directly to a
  database, no special casing required.
  - move the "implicit metadata updates triggered by appending rollout
  items" from the RolloutRecorder (which is local-threadstore-specific) to
  the LiveThread layer above the ThreadStore, inside of a private helper
  utility called ThreadMetadataSync. LiveThread calls ThreadStore
  append_items and update_metadata separately.
  - Add a generic update metadata method to ThreadManager that works on
  both live threads and "cold" threads
  - Call that ThreadManager method from app server code, so app server
  doesn't need to worry about whether the thread is live or not

  Tom · 2026-05-13 00:28:15 +00:00
  c51c65ad09
• Add process-scoped SQLite telemetry (#22154)

  ## Summary
  - add SQLite init, backfill-gate, and fallback telemetry without
  introducing a cross-cutting state-db access wrapper
  - install one process-scoped telemetry sink after OTEL startup and let
  low-level state/rollout paths emit through it directly
  - add process-start metrics for the process owners that initialize
  SQLite
  
  ---------
  
  Co-authored-by: Owen Lin <owen@openai.com>

  jif-oai · 2026-05-11 11:32:40 -07:00
  b401666ca5
• Use goal preview metadata for goal-first threads (#21981)

  Fixes #20792
  
  ## Why
  
  `/goal`-first threads are valid resumable threads, but they can be
  missing from `codex resume` and app recents because discovery depends on
  metadata derived from a normal first user message.
  
  PR #21489 attempted to fix this by using the goal objective as
  `first_user_message`. Review feedback pointed out that
  `first_user_message` does more than provide visible text today: it gates
  listing, supplies preview text, and participates in deciding whether a
  later title should surface as a distinct thread name. Reusing it for the
  goal objective could leave a `/goal`-first thread with
  `first_user_message=<goal>` and `title=<later prompt>`, even though the
  goal should only provide the initial visible preview.
  
  This PR follows that feedback by and keeps the `first_user_message` as
  is but introduces a new `preview` field to separate concerns. The
  `preview` field is populated from the first user message or the goal
  objective. We can extend it in the future to include other sources.
  
  ## What Changed
  
  - Added internal thread `preview` metadata in `codex-state`, including a
  SQLite migration that backfills from `first_user_message` and from
  existing `thread_goals` objectives when needed.
  - Treated `ThreadGoalUpdated` as preview-bearing metadata so goal-first
  threads can be listed and searched without mutating
  `first_user_message`.
  - Updated rollout listing, state queries, thread-store conversion, and
  app-server mapping to use preview metadata while continuing to expose
  the existing public `preview` field.
  - Preserved title/name distinctness behavior around literal
  `first_user_message`, so a later normal prompt after `/goal` does not
  surface as a separate name just because the goal supplied the initial
  preview.
  - Preserved compatibility for older/internal metadata writes by deriving
  preview from `first_user_message` when explicit preview metadata is
  absent.
  
  ## Verification
  
  - Manually verified that a thread that starts with a `/goal <objective>`
  shows up in the resume picker.

  Eric Traut · 2026-05-11 10:12:46 -07:00
  f10ddc3f13
• Reapply "Move skills watcher to app-server" (#21652)

  ## Why
  
  PR #21460 reverted the earlier move of skills change watching from
  `codex-core` into app-server. This reapplies that boundary change so
  app-server owns client-facing `skills/changed` notifications and core no
  longer carries the watcher.
  
  ## What
  
  - Restore the app-server `SkillsWatcher` and register it from thread
  listener setup.
  - Remove the core-owned skills watcher and its core live-reload
  integration surface.
  - Restore app-server coverage for `skills/changed` notifications after a
  watched skill file changes.
  
  ## Validation
  
  - `cargo test -p codex-app-server --test all
  suite::v2::skills_list::skills_changed_notification_is_emitted_after_skill_change
  -- --exact --nocapture`
  - `cargo test -p codex-core --lib --no-run`

  pakrym-oai · 2026-05-08 17:41:15 -07:00
  408e6218ab
• [codex] make shutdown pending-touch test deterministic (#21550)

  ## What changed
  - rewrote `shutdown_flushes_pending_metadata_irrelevant_updated_at` to
  seed an existing pending `updated_at` touch directly in
  `RolloutWriterState`
  - kept the shutdown test focused on draining a pending touch, leaving
  the separate coalescing test to cover timing-based deferral
  
  ## Why
  The old test had to complete several async operations inside the 50 ms
  test-only coalescing window. When that sequence took longer, the second
  flush updated `threads.updated_at` immediately and the pre-shutdown
  equality assertion failed, even though shutdown behavior was correct.
  
  ## Validation
  - `cargo test -p codex-rollout
  shutdown_flushes_pending_metadata_irrelevant_updated_at`
  - `cargo test -p codex-rollout`
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-05-08 10:48:45 +02:00
  d9feaffffb
• [codex-analytics] plumb protocol-native review timing (#21434)

  ## Why
  
  We want terminal tool review analytics, but the reducer should not stamp
  review timing from its own wall clock.
  
  This PR plumbs review timing through the real protocol and app-server
  seams so downstream analytics can consume the emitter's timestamps
  directly. Guardian reviews keep their enriched `started_at` /
  `completed_at` analytics fields by deriving those legacy second-based
  values from the same protocol-native millisecond lifecycle timestamps,
  rather than sampling a separate analytics clock.
  
  ## What changed
  
  - add `started_at_ms` to user approval request payloads
  - add `started_at_ms` / `completed_at_ms` to guardian review
  notifications
  - preserve Guardian review `started_at` / `completed_at` enrichment from
  the protocol-native timing source
  - stamp typed `ServerResponse` analytics facts with app-server-observed
  `completed_at_ms`
  - thread the new timing fields through core, protocol, app-server, TUI,
  and analytics fixtures
  
  ## Verification
  
  - `cargo test -p codex-app-server outgoing_message --manifest-path
  codex-rs/Cargo.toml`
  - `cargo test -p codex-app-server-protocol guardian --manifest-path
  codex-rs/Cargo.toml`
  - `cargo test -p codex-tui guardian --manifest-path codex-rs/Cargo.toml`
  - `cargo test -p codex-analytics analytics_client_tests --manifest-path
  codex-rs/Cargo.toml`
  
  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/21434).
  * #18748
  * __->__ #21434
  * #18747
  * #17090
  * #17089
  * #20514

  rhan-oai · 2026-05-07 20:31:41 -07:00
  99016ec732
• Revert "Move skills watcher to app-server" (#21460)

  Reverts openai/codex#21287

  pakrym-oai · 2026-05-07 02:24:20 +00:00
  103dc2b6ae
• Move skills watcher to app-server (#21287)

  ## Why
  
  Skills update notifications are app-server API behavior, but the watcher
  lived in `codex-core` and surfaced through
  `EventMsg::SkillsUpdateAvailable`. Moving the watcher out keeps core
  focused on thread execution and lets app-server own both cache
  invalidation and the `skills/changed` notification.
  
  ## What changed
  
  - Added an app-server-owned skills watcher that watches local skill
  roots, clears the shared skills cache, and emits `skills/changed`
  directly.
  - Registers skill watches from the common app-server thread listener
  attach path, including direct starts, resumes, and app-server-observed
  child or forked threads.
  - Stores the `WatchRegistration` on `ThreadState`, so listener
  replacement, thread teardown, idle unload, and app-server shutdown
  deregister by dropping the RAII guard.
  - Removed `EventMsg::SkillsUpdateAvailable`, the core watcher, and the
  old core live-reload test.
  - Extended the app-server skills change test to verify a cached skills
  list is refreshed after a filesystem change without forcing reload.
  
  ## Validation
  
  - `cargo check -p codex-core -p codex-app-server -p codex-mcp-server -p
  codex-rollout -p codex-rollout-trace`
  - `cargo test -p codex-app-server
  skills_changed_notification_is_emitted_after_skill_change`

  pakrym-oai · 2026-05-06 15:38:11 -07:00
  d5eea229cc
• Remove core MCP list tools op (#21281)

  ## Why
  
  The core `Op::ListMcpTools` request path is no longer needed. Keeping it
  around left a dead request/response surface alongside the app-server MCP
  inventory APIs that own current server status listing.
  
  ## What Changed
  
  - Removed `Op::ListMcpTools`, `EventMsg::McpListToolsResponse`, and the
  core handler that built the MCP snapshot response.
  - Removed the now-unused `codex-mcp` snapshot wrapper/export and passive
  event handling arms in rollout and MCP-server consumers.
  - Updated tests that used the old op as a synchronization hook to wait
  on existing startup/skills events, and deleted the plugin test that only
  exercised the removed listing op.
  
  ## Validation
  
  - `cargo test -p codex-protocol`
  - `cargo test -p codex-mcp`
  - `cargo test -p codex-rollout -p codex-rollout-trace -p
  codex-mcp-server`
  - `cargo test -p codex-core --test all
  pending_input::queued_inter_agent_mail`
  - `cargo test -p codex-core --test all
  rmcp_client::stdio_mcp_tool_call_includes_sandbox_state_meta`
  - `cargo test -p codex-core --test all
  rmcp_client::stdio_image_responses`
  - `just fix -p codex-core -p codex-protocol -p codex-mcp -p
  codex-rollout -p codex-rollout-trace -p codex-mcp-server`

  pakrym-oai · 2026-05-06 11:20:34 -07:00
  712305be47
• rollout: coalesce thread updated_at touches (#21367)

  ## Why
  
  Metadata-irrelevant rollout events currently refresh
  `threads.updated_at` on every flush. That keeps thread recency accurate,
  but it also turns high-frequency agent output into unnecessary SQLite
  writes. Recency only needs to advance periodically during an active
  session, while the final suppressed touch still needs to be persisted
  before shutdown.
  
  ## What changed
  
  - coalesce touch-only `updated_at` writes in the rollout writer, with a
  short production interval between persisted touches
  - retain the latest suppressed touch and flush it during shutdown so the
  thread is not left stale
  - extend rollout recorder coverage for coalesced touches, delayed
  refresh, shutdown flushing, and the existing missing-thread fallback
  path
  
  ## Verification
  
  - Added regression coverage in `rollout/src/recorder_tests.rs` for
  coalescing and shutdown flushing behavior.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-05-06 19:32:24 +02:00
  0e821b380a
• Move message history out of core (#21278)

  ## Why
  
  Message history was implemented inside `codex-core` and surfaced through
  core protocol ops and `SessionConfiguredEvent` fields even though the
  current consumer is TUI-local prompt recall. That made core own UI
  history persistence and exposed `history_log_id` / `history_entry_count`
  through surfaces that app-server and other clients do not need.
  
  This change moves message history persistence out of core and keeps the
  recall plumbing local to the TUI.
  
  ## What changed
  
  - Added a new `codex-message-history` crate for appending, looking up,
  trimming, and reading metadata from `history.jsonl`.
  - Removed core protocol history ops/events: `AddToHistory`,
  `GetHistoryEntryRequest`, and `GetHistoryEntryResponse`.
  - Removed `history_log_id` and `history_entry_count` from
  `SessionConfiguredEvent` and updated exec/MCP/test fixtures accordingly.
  - Updated the TUI to dispatch local app events for message-history
  append/lookup and keep its persistent-history metadata in TUI session
  state.
  
  ## Validation
  
  - `cargo test -p codex-message-history -p codex-protocol`
  - `cargo test -p codex-exec event_processor_with_json_output`
  - `cargo test -p codex-mcp-server outgoing_message`
  - `cargo test -p codex-tui`
  - `just fix -p codex-message-history -p codex-protocol -p codex-core -p
  codex-tui -p codex-exec -p codex-mcp-server`

  pakrym-oai · 2026-05-06 08:35:42 -07:00
  2004173cd7
• [codex-analytics] rework thread_source for thread analytics (#20949)

  ## Summary
  - make `thread_source` an explicit optional thread-level field on
  `thread/start`, `thread/fork`, and returned thread payloads
  - persist `thread_source` in rollout/session metadata so resumed live
  threads retain the original value
  - replace the old best-effort `session_source` -> `thread_source`
  mapping with an explicit caller-supplied analytics classification
  
  ## Why
  Before this change, analytics `thread_source` was populated by a
  best-effort mapping from `session_source`. `session_source` describes
  the runtime/client surface, not the actual thread-level origin, so that
  projection was not accurate enough to distinguish cases such as `user`,
  `subagent`, `memory_consolidation`, and future thread origins reliably.
  
  Making `thread_source` explicit keeps one thread-level analytics field
  while letting callers provide the real classification directly instead
  of recovering it indirectly from `session_source`.
  
  ## Impact
  For new analytics events, `thread_source` now reflects the explicit
  thread-level classification supplied by the caller rather than an
  inferred value derived from `session_source`. Existing protocol fields
  remain optional; callers that omit `threadSource` now produce `null`
  instead of a best-effort inferred value.
  
  ## Validation
  - `just write-app-server-schema`
  - `cargo test -p codex-analytics -p codex-core -p
  codex-app-server-protocol --no-run`
  - `cargo test -p codex-app-server-protocol
  generated_ts_optional_nullable_fields_only_in_params`
  - `cargo test -p codex-analytics
  thread_initialized_event_serializes_expected_shape`
  - `cargo test -p codex-core
  resume_stopped_thread_from_rollout_preserves_thread_source`

  rhan-oai · 2026-05-06 02:12:31 +00:00
  b3d4f1a9f0
• [codex] Remove legacy ListSkills op (#21282)

  ## Why
  
  `skills/list` is already exposed through app-server v2 and covered by
  the app-server test suite. Keeping the separate core `Op::ListSkills`
  path leaves a duplicate legacy protocol surface that no longer needs to
  be maintained.
  
  ## What Changed
  
  - Removed `Op::ListSkills` and `EventMsg::ListSkillsResponse` from the
  core protocol.
  - Deleted the corresponding core session handler and stale core
  integration tests.
  - Removed rollout/MCP ignore branches and protocol v1 docs references
  for the deleted event/op.
  - Left app-server `skills/list` and its existing coverage intact.
  
  ## Validation
  
  - `cargo test -p codex-protocol`
  - `cargo test -p codex-core --test all suite::skills`
  - `cargo check -p codex-mcp-server -p codex-rollout -p
  codex-rollout-trace`
  - `just fix -p codex-core`

  pakrym-oai · 2026-05-05 18:58:18 -07:00
  136e442e95