25 Commits

• Add thread metadata update endpoint to app server (#13280)

  ## Summary
  - add the v2 `thread/metadata/update` API, including
  protocol/schema/TypeScript exports and app-server docs
  - patch stored thread `gitInfo` in sqlite without resuming the thread,
  with validation plus support for explicit `null` clears
  - repair missing sqlite thread rows from rollout data before patching,
  and make those repairs safe by inserting only when absent and updating
  only git columns so newer metadata is not clobbered
  - keep sqlite authoritative for mutable thread git metadata by
  preserving existing sqlite git fields during reconcile/backfill and only
  using rollout `SessionMeta` git fields to fill gaps
  - add regression coverage for the endpoint, repair paths, concurrent
  sqlite writes, clearing git fields, and rollout/backfill reconciliation
  - fix the login server shutdown race so cancelling before the waiter
  starts still terminates `block_until_done()` correctly
  
  ## Testing
  - `cargo test -p codex-state
  apply_rollout_items_preserves_existing_git_branch_and_fills_missing_git_fields`
  - `cargo test -p codex-state
  update_thread_git_info_preserves_newer_non_git_metadata`
  - `cargo test -p codex-core
  backfill_sessions_preserves_existing_git_branch_and_fills_missing_git_fields`
  - `cargo test -p codex-app-server thread_metadata_update`
  - `cargo test`
  - currently fails in existing `codex-core` grep-files tests with
  `unsupported call: grep_files`:
      - `suite::grep_files::grep_files_tool_collects_matches`
      - `suite::grep_files::grep_files_tool_reports_empty_results`

  joeytrasatti-openai · 2026-03-03 15:56:11 -08:00
  935754baa3
• feat: polluted memories (#13008)

  Add a feature flag to disable memory creation for "polluted"

  jif-oai · 2026-03-02 11:57:32 +00:00
  b649953845
• feat: add git info to memories (#12940)
  jif-oai · 2026-02-26 20:14:13 +00:00
  a6065d30f4
• feat: memories forgetting (#12900)

  Add diff based memory forgetting

  jif-oai · 2026-02-26 13:19:57 +00:00
  382fa338b3
• Add rollout path to memory files and search for them during read (#12684)

  Co-authored-by: jif-oai <jif@openai.com>

  Wendy Jiao · 2026-02-26 10:57:01 +00:00
  52aa49db1b
• Agent jobs (spawn_agents_on_csv) + progress UI (#10935)

  ## Summary
  - Add agent job support: spawn a batch of sub-agents from CSV, auto-run,
  auto-export, and store results in SQLite.
  - Simplify workflow: remove run/resume/get-status/export tools; spawn is
  deterministic and completes in one call.
  - Improve exec UX: stable, single-line progress bar with ETA; suppress
  sub-agent chatter in exec.
  
  ## Why
  Enables map-reduce style workflows over arbitrarily large repos using
  the existing Codex orchestrator. This addresses review feedback about
  overly complex job controls and non-deterministic monitoring.
  
  ## Demo (progress bar)
  ```
  ./codex-rs/target/debug/codex exec \
    --enable collab \
    --enable sqlite \
    --full-auto \
    --progress-cursor \
    -c agents.max_threads=16 \
    -C /Users/daveaitel/code/codex \
    - <<'PROMPT'
  Create /tmp/agent_job_progress_demo.csv with columns: path,area and 30 rows:
  path = item-01..item-30, area = test.
  
  Then call spawn_agents_on_csv with:
  - csv_path: /tmp/agent_job_progress_demo.csv
  - instruction: "Run `python - <<'PY'` to sleep a random 0.3–1.2s, then output JSON with keys: path, score (int). Set score = 1."
  - output_csv_path: /tmp/agent_job_progress_demo_out.csv
  PROMPT
  ```
  
  ## Review feedback addressed
  - Auto-start jobs on spawn; removed run/resume/status/export tools.
  - Auto-export on success.
  - More descriptive tool spec + clearer prompts.
  - Avoid deadlocks on spawn failure; pending/running handled safely.
  - Progress bar no longer scrolls; stable single-line redraw.
  
  ## Tests
  - `cd codex-rs && cargo test -p codex-exec`
  - `cd codex-rs && cargo build -p codex-cli`

  daveaitel-openai · 2026-02-24 21:00:19 +00:00
  dcab40123f
• feat: add nick name to sub-agents (#12320)

  Adding random nick name to sub-agents. Used for UX
  
  At the same time, also storing and wiring the role of the sub-agent

  jif-oai · 2026-02-20 14:39:49 +00:00
  0f9eed3a6f
• feat: add --search to just log (#11995)

  Summary
  - extend the log client to accept an optional `--search` substring
  filter when querying codex-state logs
  - propagate the filter through `LogQuery` and apply it in
  `push_log_filters` via `INSTR(message, ...)`
  - add an integration test that exercises the new search filtering
  behavior
  
  Testing
  - Not run (not requested)

  jif-oai · 2026-02-17 14:19:52 +00:00
  31d4bfdde0
• Add process_uuid to sqlite logs (#11534)

  ## Summary
  This PR is the first slice of the per-session `/feedback` logging work:
  it adds a process-unique identifier to SQLite log rows.
  
  It does **not** change `/feedback` sourcing behavior yet.
  
  ## Changes
  - Add migration `0009_logs_process_id.sql` to extend `logs` with:
    - `process_uuid TEXT`
    - `idx_logs_process_uuid` index
  - Extend state log models:
    - `LogEntry.process_uuid: Option<String>`
    - `LogRow.process_uuid: Option<String>`
  - Stamp each log row with a stable per-process UUID in the sqlite log
  layer:
    - generated once per process as `pid:<pid>:<uuid>`
  - Update sqlite log insert/query paths to persist and read
  `process_uuid`:
    - `INSERT INTO logs (..., process_uuid, ...)`
    - `SELECT ..., process_uuid, ... FROM logs`
  
  ## Why
  App-server runs many sessions in one process. This change provides a
  process-scoping primitive we need for follow-up `/feedback` work, so
  threadless/process-level logs can be associated with the emitting
  process without mixing across processes.
  
  ## Non-goals in this PR
  - No `/feedback` transport/source changes
  - No attachment size changes
  - No sqlite retention/trim policy changes
  
  ## Testing
  - `just fmt`
  - CI will run the full checks

  Charley Cunningham · 2026-02-14 17:27:22 -08:00
  fce4ad9cf4
• feat: add slug in name (#11739)
  jif-oai · 2026-02-13 15:24:03 +00:00
  db66d827be
• Add cwd to memory files (#11591)

  Add cwd to memory files so that model can deal with multi cwd memory
  better.
  
  ---------
  
  Co-authored-by: jif-oai <jif@openai.com>

  Wendy Jiao · 2026-02-12 17:46:49 +00:00
  88c5ca2573
• feat: make sandbox read access configurable with ReadOnlyAccess (#11387)

  `SandboxPolicy::ReadOnly` previously implied broad read access and could
  not express a narrower read surface.
  This change introduces an explicit read-access model so we can support
  user-configurable read restrictions in follow-up work, while preserving
  current behavior today.
  
  It also ensures unsupported backends fail closed for restricted-read
  policies instead of silently granting broader access than intended.
  
  ## What
  
  - Added `ReadOnlyAccess` in protocol with:
    - `Restricted { include_platform_defaults, readable_roots }`
    - `FullAccess`
  - Updated `SandboxPolicy` to carry read-access configuration:
    - `ReadOnly { access: ReadOnlyAccess }`
    - `WorkspaceWrite { ..., read_only_access: ReadOnlyAccess }`
  - Preserved existing behavior by defaulting current construction paths
  to `ReadOnlyAccess::FullAccess`.
  - Threaded the new fields through sandbox policy consumers and call
  sites across `core`, `tui`, `linux-sandbox`, `windows-sandbox`, and
  related tests.
  - Updated Seatbelt policy generation to honor restricted read roots by
  emitting scoped read rules when full read access is not granted.
  - Added fail-closed behavior on Linux and Windows backends when
  restricted read access is requested but not yet implemented there
  (`UnsupportedOperation`).
  - Regenerated app-server protocol schema and TypeScript artifacts,
  including `ReadOnlyAccess`.
  
  ## Compatibility / rollout
  
  - Runtime behavior remains unchanged by default (`FullAccess`).
  - API/schema changes are in place so future config wiring can enable
  restricted read access without another policy-shape migration.

  Michael Bolin · 2026-02-11 18:31:14 -08:00
  abbd74e2be
• feat: mem v2 - PR5 (#11372)
  jif-oai · 2026-02-10 23:22:55 +00:00
  2c9be54c9a
• chore: unify memory job flow (#11334)
  jif-oai · 2026-02-10 20:26:39 +00:00
  a6e9469fa4
• feat: align memory phase 1 and make it stronger (#11300)

  ## Align with the new phase-1 design
  
  Basically we know run phase 1 in parallel by considering:
  * Max 64 rollouts
  * Max 1 month old
  * Consider the most recent first
  
  This PR also adds stronger parallelization capabilities by detecting
  stale jobs, retry policies, ownership of computation to prevent double
  computations etc etc

  jif-oai · 2026-02-10 13:42:09 +00:00
  1d5eba0090
• memories: add extraction and prompt module foundation (#11200)

  ## Summary
  - add the new `core/src/memories` module (phase-one parsing, rollout
  filtering, storage, selection, prompts)
  - add Askama-backed memory templates for stage-one input/system and
  consolidation prompts
  - add module tests for parsing, filtering, path bucketing, and summary
  maintenance
  
  ## Testing
  - just fmt
  - cargo test -p codex-core --lib memories::

  jif-oai · 2026-02-10 10:10:24 +00:00
  6049ff02a0
• Leverage state DB metadata for thread summaries (#10621)

  Summary:
  - read conversation summaries and cwd info from the state DB when
  possible so we no longer rely on rollout files for metadata and avoid
  extra I/O
  - persist CLI version in thread metadata, surface it through summary
  builders, and add the necessary DB migration hooks
  - simplify thread listing by using enriched state DB data directly
  rather than reading rollout heads
  
  Testing:
  - Not run (not requested)

  jif-oai · 2026-02-05 16:39:11 +00:00
  9ee746afd6
• feat: resumable backfill (#10745)

  ## Summary
  
  This PR makes SQLite rollout backfill resumable and repeatable instead
  of one-shot-on-db-create.
  
  ## What changed
  
  - Added a persisted backfill state table:
    - state/migrations/0008_backfill_state.sql
  - Tracks status (pending|running|complete), last_watermark, and
  last_success_at.
  - Added backfill state model/types in codex-state:
    - BackfillState, BackfillStatus (state/src/model/backfill_state.rs)
  - Added runtime APIs to manage backfill lifecycle/progress:
    - get_backfill_state
    - mark_backfill_running
    - checkpoint_backfill
    - mark_backfill_complete
  - Updated core startup behavior:
  - Backfill now runs whenever state is not Complete (not only when DB
  file is newly created).
  - Reworked backfill execution:
  - Collect rollout files, derive deterministic watermark per path, sort,
  resume from last_watermark.
  - Process in batches (BACKFILL_BATCH_SIZE = 200), checkpoint after each
  batch.
    - Mark complete with last_success_at at the end.
  
  ## Why
  
  Previous behavior could leave users permanently partially backfilled if
  the process exited during initial async backfill. This change allows
  safe continuation across restarts and avoids restarting from scratch.

  jif-oai · 2026-02-05 14:34:34 +00:00
  4033f905c6
• feat: add phase 1 mem db (#10634)

  - Schema: thread_id (PK, FK to threads.id with cascade delete),
  trace_summary, memory_summary, updated_at.
  - Migration: creates the table and an index on (updated_at DESC,
  thread_id DESC) for efficient recent-first reads.
    - Runtime API (DB-only):
        - `get_thread_memory(thread_id)`: fetch one memory row.
  - `upsert_thread_memory(thread_id, trace_summary, memory_summary)`:
  insert/update by thread id and always advance updated_at.
  - `get_last_n_thread_memories_for_cwd(cwd, n)`: join thread_memory with
  threads and return newest n rows for an exact cwd match.
  - Model layer: introduced ThreadMemory and row conversion types to keep
  query decoding typed and consistent with existing state models.

  jif-oai · 2026-02-04 21:38:39 +00:00
  4922b3e571
• feat: improve logs client (#10229)
  jif-oai · 2026-01-30 18:23:18 +01:00
  eff11f792b
• feat: reduce span exposition (#10171)

  This only avoids the creation of duplicates spans

  jif-oai · 2026-01-29 18:15:22 +00:00
  798c4b3260
• chore: unify log queries (#10152)

  Unify log queries to only have SQLX code in the runtime and use it for
  both the log client and for tests

  jif-oai · 2026-01-29 16:28:15 +00:00
  e6c4f548ab
• feat: adding thread ID to logs + filter in the client (#10150)
  jif-oai · 2026-01-29 16:53:30 +01:00
  89c5f3c4d4
• chore: improve client (#10149)

  <img width="883" height="84" alt="Screenshot 2026-01-29 at 11 13 12"
  src="https://github.com/user-attachments/assets/090a2fec-94ed-4c0f-aee5-1653ed8b1439"
  />

  jif-oai · 2026-01-29 11:25:22 +01:00
  4ba911d48c
• feat: add log db (#10086)

  Add a log DB. The goal is just to store our logs in a `.sqlite` DB to
  make it easier to crawl them and drop the oldest ones.

  jif-oai · 2026-01-29 10:23:03 +01:00
  780482da84