17 Commits

• [codex] Route live thread writes through ThreadStore (#18882)

  Begin migrating the thread write codepaths to ThreadStore.
  
  This starts using ThreadStore inside of core session code, not only in
  the app server code.
  
  Rework the interfaces around thread recording/persistence. We're left
  with the following:
  
  * `ThreadManager`: owns the process-level registry of loaded threads and
  handles cross-thread orchestration: start, resume, fork, lookup, remove,
  and route ops to running CodexThreads.
  * `CodexThread`: represents one loaded/running thread from the outside.
  It is the handle app-server and callers use to submit ops, inspect
  session metadata, and shut the thread down.
  * `LiveThread`: session-owned persistence lifecycle handle for one
  active thread. Core session code uses it to append rollout items,
  materialize lazy persistence, flush, shutdown, discard init-failed
  writers, and load that thread’s persisted history.
  * `ThreadStore`: storage backend abstraction. It answers “how are
  threads persisted, read, listed, updated, archived?” Local and remote
  implementations live behind this trait.
  * `LocalThreadStore`: local ThreadStore implementation. It owns the
  file/sqlite-specific details and keeps RolloutRecorder as a local
  implementation detail.
  
  This is a few too many Thread abstractions for my liking, but they do
  all represent different concepts / needs / layers.
  
  Migration note: in places where the core code explicitly requires a
  path, rather than a thread ID, throw an error if we're running with a
  remote store.
  
  Cover the new local live-writer lifecycle with focused tests and
  preserve app-server thread-start behavior, including ephemeral pathless
  sessions.

  Tom · 2026-04-23 10:17:09 -07:00
  f1923a38b1
• Support multiple cwd filters for thread list (#18502)

  ## Summary
  
  - Teach app-server `thread/list` to accept either a single `cwd` or an
  array of cwd filters, returning threads whose recorded session cwd
  matches any requested path
  - Add `useStateDbOnly` as an explicit opt-in fast path for callers that
  want to answer `thread/list` from SQLite without scanning JSONL rollout
  files
  - Preserve backwards compatibility: by default, `thread/list` still
  scans JSONL rollouts and repairs SQLite state
  - Wire the new cwd array and SQLite-only options through app-server,
  local/remote thread-store, rollout listing, generated TypeScript/schema
  fixtures, proto output, and docs
  
  ## Test Plan
  
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-rollout`
  - `cargo test -p codex-thread-store`
  - `cargo test -p codex-app-server thread_list`
  - `just fmt`
  - `just fix -p codex-app-server-protocol -p codex-rollout -p
  codex-thread-store -p codex-app-server`
  - `cargo build -p codex-cli --bin codex`

  acrognale-oai · 2026-04-22 06:10:09 -04:00
  4f8c58f737
• Add sorting/backwardsCursor to thread/list and new thread/turns/list api (#17305)

  To improve performance of UI loads from the app, add two main
  improvements:
  1. The `thread/list` api now gets a `sortDirection` request field and a
  `backwardsCursor` to the response, which lets you paginate forwards and
  backwards from a window. This lets you fetch the first few items to
  display immediately while you paginate to fill in history, then can
  paginate "backwards" on future loads to catch up with any changes since
  the last UI load without a full reload of the entire data set.
  2. Added a new `thread/turns/list` api which also has sortDirection and
  backwardsCursor for the same behavior as `thread/list`, allowing you the
  same small-fetch for immediate display followed by background fill-in
  and resync catchup.

  David de Regt · 2026-04-17 11:49:02 -07:00
  eaf78e43f2
• Move codex module under session (#18249)

  ## Summary
  - rename the core codex module root to session/mod.rs without using
  #[path]
  - move the codex module directory and tests under core/src/session
  - remove session/mod.rs reexports so call sites use explicit child
  module paths
  
  ## Testing
  - cargo test -p codex-core --lib
  - cargo check -p codex-core --tests
  - just fmt
  - just fix -p codex-core
  - git diff --check

  pakrym-oai · 2026-04-17 16:18:53 +00:00
  71e4c6fa17
• Refactor config loading to use filesystem abstraction (#18209)

  Initial pass propagating FileSystem through config loading.

  pakrym-oai · 2026-04-17 00:51:21 +00:00
  9effa0509f
• [codex] Make realtime startup context truncation deterministic (#18172)

  ## Summary
  
  - remove the final whole-blob truncation pass from realtime
  startup-context assembly
  - enforce fixed per-section budgets, including each section heading
  - keep the existing per-section caps and raise the overall realtime
  startup-context budget to `5300`, matching the sum of those section
  budgets
  - add focused tests for the new wrapping and section-budget behavior
  
  ## Why
  
  The previous flow truncated each section and then middle-truncated the
  final combined startup-context blob again. Small input changes could
  shift that combined cut point, which made retained context unstable and
  caused nondeterministic tests.
  
  ## Impact
  
  Startup context now preserves section boundaries and ordering
  deterministically. Each section is still budgeted independently, but the
  final assembled blob is no longer truncated again as a single opaque
  string. To match that design, the overall startup-context token budget
  is updated to the sum of the existing section budgets rather than
  lowering the section caps.
  
  ## Validation
  
  - `cargo +1.93.0 test -p codex-core realtime_context`
  - `cargo +1.93.0 test -p codex-core --test all
  suite::realtime_conversation::conversation_start_injects_startup_context_from_thread_history
  -- --exact`
  - `cargo +1.93.0 test -p codex-core --test all
  suite::realtime_conversation::conversation_startup_context_current_thread_selects_many_turns_by_budget
  -- --exact`
  - `cargo +1.93.0 test -p codex-core --test all
  suite::realtime_conversation::conversation_startup_context_falls_back_to_workspace_map
  -- --exact`
  - `cargo +1.93.0 test -p codex-core --test all
  suite::realtime_conversation::conversation_startup_context_is_truncated_and_sent_once_per_start
  -- --exact`

  bxie-openai · 2026-04-16 13:51:43 -07:00
  37bf42d5d5
• Async config loading (#18022)

  Parts of config will come from executor. Prepare for that by making
  config loading methods async.

  pakrym-oai · 2026-04-15 19:18:38 -07:00
  bd61737e8a
• Clarify realtime v2 context and handoff messages (#17896)

  ## Summary
  - wrap realtime startup context in
  `<startup_context>...</startup_context>` tags
  - prefix V2 mirrored user text and relayed backend text with `[USER]` /
  `[BACKEND]`
  - remove the V2 progress suffix and replace the final V2 handoff output
  with a short completion acknowledgement while preserving the existing V1
  wrapper
  
  ## Testing
  - cargo test -p codex-api
  realtime_v2_session_update_includes_background_agent_tool_and_handoff_output_item
  -- --exact
  - cargo test -p codex-app-server webrtc_v2_background_agent_
  - cargo test -p codex-app-server webrtc_v2_text_input_is_
  - cargo test -p codex-core conversation_user_text_turn_is_

  bxie-openai · 2026-04-15 16:26:20 -07:00
  c2bdb7812c
• [codex] Add local thread store listing (#17824)

  Builds on top of #17659 
  
  Move the filesystem + sqlite thread listing-related operations inside of
  a local ThreadStore implementation and call ThreadStore from the places
  that used to perform these filesystem/sqlite operations.
  
  This is the first of a series of PRs that will implement the rest of the
  local ThreadStore.
  
  Testing:
  - added unit tests for the thread store implementation
  - adjusted some unit tests in the realtime + personality packages whose
  callsites changed. Specifically I'm trying to hide ThreadMetadata inside
  of the local implementation and make ThreadMetadata a sqlite
  implementation detail concern rather than a public interface, preferring
  the more generate StoredThread interface instead
  - added a corner case test for the personality migration package that
  wasn't covered by the existing test suite
  - adjust the behavior of searched thread listing to run the existing
  local rollout repair/backfill pass _before_ querying SQLite results, so
  callers using ThreadStore::list_threads do not miss matches after a
  partial metadata warm-up

  Tom · 2026-04-15 11:34:27 -07:00
  cdfcd2ca92
• Cap realtime mirrored user turns (#17685)

  Cap mirrored user text sent to realtime with the existing 300-token turn
  budget while preserving the full model turn.
  
  Adds integration coverage for capped realtime mirror payloads.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-13 14:31:18 -07:00
  ec0133f5f8
• Budget realtime current thread context (#17519)

  Select Current Thread startup context by budget from newest turns, cap
  each rendered turn at 300 approximate tokens, and add formatter plus
  integration snapshot coverage.

  Ahmed Ibrahim · 2026-04-12 11:59:09 -07:00
  4db60d5d8b
• Move string truncation helpers into codex-utils-string (#15572)

  - move the shared byte-based middle truncation logic from `core` into
  `codex-utils-string`
  - keep token-specific truncation in `codex-core` so rollout can reuse
  the shared helper in the next stacked PR
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-03-24 15:45:40 -07:00
  062fa7a2bb
• Move git utilities into a dedicated crate (#15564)

  - create `codex-git-utils` and move the shared git helpers into it with
  file moves preserved for diff readability
  - move the `GitInfo` helpers out of `core` so stacked rollout work can
  depend on the shared crate without carrying its own git info module
  
  ---------
  
  Co-authored-by: Ahmed Ibrahim <219906144+aibrahim-oai@users.noreply.github.com>
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-03-24 13:26:23 -07:00
  0f957a93cd
• [stack 3/4] Add current thread context to realtime startup (#14829)

  ## Stack Position
  3/4. Top-of-stack sibling built on #14830.
  
  ## Base
  - #14830
  
  ## Sibling
  - #14827
  
  ## Scope
  - Extend the realtime startup context with a bounded summary of the
  latest thread turns for continuity.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-03-17 05:11:05 +00:00
  79f476e47d
• Apply argument comment lint across codex-rs (#14652)

  ## Why
  
  Once the repo-local lint exists, `codex-rs` needs to follow the
  checked-in convention and CI needs to keep it from drifting. This commit
  applies the fallback `/*param*/` style consistently across existing
  positional literal call sites without changing those APIs.
  
  The longer-term preference is still to avoid APIs that require comments
  by choosing clearer parameter types and call shapes. This PR is
  intentionally the mechanical follow-through for the places where the
  existing signatures stay in place.
  
  After rebasing onto newer `main`, the rollout also had to cover newly
  introduced `tui_app_server` call sites. That made it clear the first cut
  of the CI job was too expensive for the common path: it was spending
  almost as much time installing `cargo-dylint` and re-testing the lint
  crate as a representative test job spends running product tests. The CI
  update keeps the full workspace enforcement but trims that extra
  overhead from ordinary `codex-rs` PRs.
  
  ## What changed
  
  - keep a dedicated `argument_comment_lint` job in `rust-ci`
  - mechanically annotate remaining opaque positional literals across
  `codex-rs` with exact `/*param*/` comments, including the rebased
  `tui_app_server` call sites that now fall under the lint
  - keep the checked-in style aligned with the lint policy by using
  `/*param*/` and leaving string and char literals uncommented
  - cache `cargo-dylint`, `dylint-link`, and the relevant Cargo
  registry/git metadata in the lint job
  - split changed-path detection so the lint crate's own `cargo test` step
  runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes
  - continue to run the repo wrapper over the `codex-rs` workspace, so
  product-code enforcement is unchanged
  
  Most of the code changes in this commit are intentionally mechanical
  comment rewrites or insertions driven by the lint itself.
  
  ## Verification
  
  - `./tools/argument-comment-lint/run.sh --workspace`
  - `cargo test -p codex-tui-app-server -p codex-tui`
  - parsed `.github/workflows/rust-ci.yml` locally with PyYAML
  
  ---
  
  * -> #14652
  * #14651

  Michael Bolin · 2026-03-16 16:48:15 -07:00
  b77fe8fefe
• fix: move inline codex-rs/core unit tests into sibling files (#14444)

  ## Why
  PR #13783 moved the `codex.rs` unit tests into `codex_tests.rs`. This
  applies the same extraction pattern across the rest of `codex-rs/core`
  so the production modules stay focused on runtime code instead of large
  inline test blocks.
  
  Keeping the tests in sibling files also makes follow-up edits easier to
  review because product changes no longer have to share a file with
  hundreds or thousands of lines of test scaffolding.
  
  ## What changed
  - replaced each inline `mod tests { ... }` in `codex-rs/core/src/**`
  with a path-based module declaration
  - moved each extracted unit test module into a sibling `*_tests.rs`
  file, using `mod_tests.rs` for `mod.rs` modules
  - preserved the existing `cfg(...)` guards and module-local structure so
  the refactor remains structural rather than behavioral
  
  ## Testing
  - `cargo test -p codex-core --lib` (`1653 passed; 0 failed; 5 ignored`)
  - `just fix -p codex-core`
  - `cargo fmt --check`
  - `cargo shear`

  Michael Bolin · 2026-03-12 08:16:36 -07:00
  0c8a36676a
• Refine realtime startup context formatting (#13560)

  ## Summary
  - group recent work by git repo when available, otherwise by directory
  - render recent work as bounded user asks with per-thread cwd context
  - exclude hidden files and directories from workspace trees

  Ahmed Ibrahim · 2026-03-05 16:31:20 -08:00
  6cf0ed4e79