45 Commits

• [codex-analytics] emit goal lifecycle analytics (#27078)

  ## Why
  - Currently, there is no analytics event for `/goal` behavior
  - Existing events cannot identify goal execution or its resulting
  outcome
  - The original update in
  [#26182](https://github.com/openai/codex/pull/26182) was implemented
  before `/goal` moved into `codex-goal-extension`.
  
  ## What Changed
  - Adds `codex_goal_event` serialization and enrichment to
  `codex-analytics`
  - Emits goal events from the canonical `codex-goal-extension` mutation
  and accounting paths:
    - `created` when a new logical goal is persisted
    - `usage_accounted` when cumulative goal usage is persisted
    - `status_changed` when the stored goal status changes
    - `cleared` when the goal is deleted
  - Preserves causal `turn_id` for turn driven events and uses null
  attribution for external or idle lifecycle events
  - Changes goal deletion to return the deleted row so `cleared` retains
  the stable goal ID
  
  ## Event Details
  
  Includes standard analytics metadata along with goal specific fields:
  - `goal_id`: Stable ID stored in the local SQLite goal row and shared
  across the goal's events
  - `event_kind`: Observed operation (see the 4 lifecycle events cited in
  the above bullet)
  - `goal_status`: Resulting or last stored status: `active`, `paused`,
  `blocked`, `usage_limited`, etc.
    - `has_token_budget`: Indicates whether a token budget is configured
    - `turn_id`: Causal turn ID, or null when no causal turn exists
  - `cumulative_tokens_accounted`: Cumulative tokens on `usage_accounted`
  events; null otherwise
  - `cumulative_time_accounted_seconds`: Cumulative active time on
  `usage_accounted` events; null otherwise
  
  ## Validation
  - `just test -p codex-analytics -p codex-state -p codex-goal-extension`
  - `just test -p codex-core -E 'test(/goal/)'`
  - `just test -p codex-app-server`
  - `cargo build -p codex-analytics -p codex-core -p codex-state -p
  codex-app-server`

  marksteinbrick-oai · 2026-06-09 18:45:54 -07:00
  608b8b1cc6
• [codex-analytics] add extensible feature thread sources (#27063)

  ## Why
  - `ThreadSource` currently defines a closed set of core-owned values
  - Product features also create threads for background or scheduled work
  - Adding every product-specific value to the core enum would require
  repeated `codex-rs` protocol changes
  - Feature-backed values let product callers provide precise attribution
  while preserving the existing core classifications
  
  ## What Changed
  - Adds `ThreadSource::Feature(String)` for app-owned thread source
  values
  - Represents all app-server v2 thread sources as scalar strings, so a
  feature source is supplied as `"automation"`
  - Persists and emits the feature's plain string label, so `"automation"`
  produces `thread_source="automation"` in analytics
  - Keeps `user`, `subagent`, and `memory_consolidation` as explicit
  core-owned values and regenerates the app-server schemas and TypeScript
  bindings
  
  ## Verification
  - `just write-app-server-schema`
  - `cargo check --workspace`
  - `just test -p codex-protocol
  feature_thread_source_serializes_as_its_app_owned_label`
  - `just test -p codex-app-server-protocol
  thread_sources_round_trip_as_scalar_labels`
  - `cargo test -p codex-analytics
  thread_initialized_event_serializes_expected_shape`
  - `just fmt`

  marksteinbrick-oai · 2026-06-09 12:27:10 -07:00
  a71e040df5
• [codex] Support model-defined reasoning efforts (#26444)

  ## Summary
  - accept non-empty model-defined reasoning effort values while
  preserving built-in effort behavior
  - propagate the non-Copy effort type through core, app-server, TUI,
  telemetry, and persistence call sites
  - preserve string wire encoding and expose an open-string schema for
  clients
  - update model selection and shortcut behavior for model-advertised
  effort values
  
  ## Root cause
  `ReasoningEffort` gained a string-backed custom variant, so it could no
  longer implement `Copy` or rely on derived closed-enum serialization.
  Existing consumers still moved effort values from shared references and
  assumed a fixed built-in value set.
  
  ## Validation
  - `just fmt`
  - Local tests and compilation were not run per request; relying on CI.

  Ahmed Ibrahim · 2026-06-04 13:36:24 -07:00
  8ac304c299
• Preserve renamed thread titles during reconciliation (#25624)

  ## Summary
  - preserve existing explicit SQLite thread titles during rollout
  reconciliation/backfill when the incoming rollout title is only
  first-message-derived
  - keep stale inferred-title repair behavior while avoiding session-index
  scans during startup backfill
  - add a regression test for renamed titles surviving reconcile
  
  ## Testing
  - just fmt
  - just test -p codex-rollout
  - just test -p codex-state

  jif-oai · 2026-06-01 18:33:05 +02:00
  3cdce52865
• Move memory state to a dedicated SQLite DB (#24591)

  ## Summary
  
  Generated memory rows and their stage-one/stage-two job state currently
  live in `state_5.sqlite` alongside thread metadata. That makes memory
  cleanup and regeneration share the main state schema even though those
  rows are memory-pipeline data and can be rebuilt independently from the
  durable thread records.
  
  This PR moves the memory-owned tables into a dedicated
  `memories_1.sqlite` runtime database while keeping thread metadata in
  `state_5.sqlite`.
  
  ## Changes
  
  - Adds a separate memories DB runtime, migrator, path helpers, telemetry
  kind, and Bazel compile data for `state/memory_migrations`.
  - Introduces `MemoryStore` behind `StateRuntime::memories()` and moves
  memory table/job operations onto that store.
  - Drops the old memory tables from the state DB and recreates their
  schema in `state/memory_migrations/0001_memories.sql`.
  - Updates memory startup, citation usage tracking, rollout pollution
  handling, `debug clear-memories`, and app-server `memory/reset` to
  operate through the memories DB.
  - Preserves cross-DB behavior by hydrating thread metadata from the
  state DB when selecting visible memory outputs and checking stage-one
  staleness.
  
  ## Verification
  
  - Added/updated `codex-state` tests for deleted-thread memory visibility
  and already-polluted phase-two enqueue behavior.
  - Updated `debug clear-memories`, app-server `memory/reset`, and
  memories startup tests to seed and assert memory rows through
  `memories_1.sqlite`.

  jif-oai · 2026-05-26 20:07:25 +02:00
  aad59a0916
• goal: pause continuation loops on usage limits and blockers (#23094)

  Addresses #22833, #22245, #23067
  
  ## Why
  `/goal` can keep synthesizing turns even when the next turn cannot make
  meaningful progress. Hard usage exhaustion can replay failing turns, and
  repeated permission or external-resource blockers can keep burning
  tokens while waiting for user or system intervention.
  
  ## What changed
  - Add resumable `blocked` and `usageLimited` goal states. As with
  `paused`, goal continuation stops with these states.
  - Move to `usageLimited` after usage-limit failures.
  - Allow the built-in `update_goal` tool to set `blocked` only under
  explicit repeated-impasse guidance. Updated goal continuation prompt to
  specify that agent should use `blocked` only when it has made at least
  three attempts to get past an impasse.
  
  Most of the files touched by this PR are because of the small app server
  protocol update.
  
  ## Validation
  
  I manually reproduced a number of situations where an agent can run into
  a true impasse and verified that it properly enters `blocked` state. I
  then resumed and verified that it once again entered `blocked` state
  several turns later if the impasse still exists.
  
  I also manually reproduced the usage-limit condition by creating a
  simulated responses API endpoint that returns 429 errors with the
  appropriate error message. Verified that the goal runtime properly moves
  the goal into `usageLimited` state and TUI UI updates appropriately.
  Verified that `/goal resume` resumes (and immediately goes back into
  `ussageLImited` state if appropriate).
  
  
  ## Follow-up PRs
  
  Small changes will be needed to the GUI clients to properly handle the
  two new states.

  Eric Traut · 2026-05-18 11:28:53 -07:00
  0d344aca9b
• Use goal preview metadata for goal-first threads (#21981)

  Fixes #20792
  
  ## Why
  
  `/goal`-first threads are valid resumable threads, but they can be
  missing from `codex resume` and app recents because discovery depends on
  metadata derived from a normal first user message.
  
  PR #21489 attempted to fix this by using the goal objective as
  `first_user_message`. Review feedback pointed out that
  `first_user_message` does more than provide visible text today: it gates
  listing, supplies preview text, and participates in deciding whether a
  later title should surface as a distinct thread name. Reusing it for the
  goal objective could leave a `/goal`-first thread with
  `first_user_message=<goal>` and `title=<later prompt>`, even though the
  goal should only provide the initial visible preview.
  
  This PR follows that feedback by and keeps the `first_user_message` as
  is but introduces a new `preview` field to separate concerns. The
  `preview` field is populated from the first user message or the goal
  objective. We can extend it in the future to include other sources.
  
  ## What Changed
  
  - Added internal thread `preview` metadata in `codex-state`, including a
  SQLite migration that backfills from `first_user_message` and from
  existing `thread_goals` objectives when needed.
  - Treated `ThreadGoalUpdated` as preview-bearing metadata so goal-first
  threads can be listed and searched without mutating
  `first_user_message`.
  - Updated rollout listing, state queries, thread-store conversion, and
  app-server mapping to use preview metadata while continuing to expose
  the existing public `preview` field.
  - Preserved title/name distinctness behavior around literal
  `first_user_message`, so a later normal prompt after `/goal` does not
  surface as a separate name just because the goal supplied the initial
  preview.
  - Preserved compatibility for older/internal metadata writes by deriving
  preview from `first_user_message` when explicit preview metadata is
  absent.
  
  ## Verification
  
  - Manually verified that a thread that starts with a `/goal <objective>`
  shows up in the resume picker.

  Eric Traut · 2026-05-11 10:12:46 -07:00
  f10ddc3f13
• [codex-analytics] rework thread_source for thread analytics (#20949)

  ## Summary
  - make `thread_source` an explicit optional thread-level field on
  `thread/start`, `thread/fork`, and returned thread payloads
  - persist `thread_source` in rollout/session metadata so resumed live
  threads retain the original value
  - replace the old best-effort `session_source` -> `thread_source`
  mapping with an explicit caller-supplied analytics classification
  
  ## Why
  Before this change, analytics `thread_source` was populated by a
  best-effort mapping from `session_source`. `session_source` describes
  the runtime/client surface, not the actual thread-level origin, so that
  projection was not accurate enough to distinguish cases such as `user`,
  `subagent`, `memory_consolidation`, and future thread origins reliably.
  
  Making `thread_source` explicit keeps one thread-level analytics field
  while letting callers provide the real classification directly instead
  of recovering it indirectly from `session_source`.
  
  ## Impact
  For new analytics events, `thread_source` now reflects the explicit
  thread-level classification supplied by the caller rather than an
  inferred value derived from `session_source`. Existing protocol fields
  remain optional; callers that omit `threadSource` now produce `null`
  instead of a best-effort inferred value.
  
  ## Validation
  - `just write-app-server-schema`
  - `cargo test -p codex-analytics -p codex-core -p
  codex-app-server-protocol --no-run`
  - `cargo test -p codex-app-server-protocol
  generated_ts_optional_nullable_fields_only_in_params`
  - `cargo test -p codex-analytics
  thread_initialized_event_serializes_expected_shape`
  - `cargo test -p codex-core
  resume_stopped_thread_from_rollout_preserves_thread_source`

  rhan-oai · 2026-05-06 02:12:31 +00:00
  b3d4f1a9f0
• feat: trigger memories from user turns with cooldown (#19970)

  ## Why
  
  Memory startup was tied to thread lifecycle events such as create, load,
  and fork. That can run memory work before a thread receives real user
  input, and it makes startup cost scale with thread management instead of
  actual turns. Moving the trigger to `thread/sendInput` keeps memory
  startup aligned with the first real user turn and lets it use the
  current thread config at turn time.
  
  The idea is to prevent ghost cost due to pre-warm triggered by the app
  
  Turn-based startup can also make global phase-2 consolidation easier to
  request repeatedly, so this adds a success cooldown and tightens the
  default startup scan window.
  
  ## What Changed
  
  - Start `codex_memories_write::start_memories_startup_task` after a
  non-empty `thread/sendInput` turn is submitted, instead of from thread
  create/load/fork paths:
  https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/app-server/src/codex_message_processor.rs#L6477-L6487
  - Expose `CodexThread::config()` so app-server can pass the live config
  into memory startup at turn time.
  - Add a six-hour successful-run cooldown for global phase-2
  consolidation via `SkippedCooldown`:
  https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/state/src/runtime/memories.rs#L963-L966
  - Reduce memory startup defaults to at most 2 rollouts over 10 days:
  https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/config/src/types.rs#L31-L34
  
  ## Verification
  
  Updated the memory runtime coverage around phase-2 reclaim behavior,
  including `phase2_global_lock_respects_success_cooldown`.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-28 16:23:13 +02:00
  a9e5c34083
• Allow Phase 2 memory claims after retry exhaustion (#19809)

  ## Why
  
  The Phase 2 memories job row is only the global lock for the git-backed
  memory workspace. Manual memory edits do not enqueue new Stage 1 work,
  so a Phase 2 row with `retry_remaining = 0` could be skipped before the
  worker ever claimed the lock and generated `phase2_workspace_diff.md`.
  
  That left workspace-only changes unconsolidated after repeated failures,
  even when retry backoff had elapsed and the filesystem had real diffable
  work.
  
  ## What Changed
  
  - Allow `try_claim_global_phase2_job` to claim the Phase 2 lock after
  the retry budget is exhausted, while still respecting active `retry_at`
  backoff and fresh running leases.
  - Treat `SkippedRetryUnavailable` for Phase 2 as backoff-only, and
  update the outcome docs to match.
  - Clamp Phase 2 retry bookkeeping at zero when failed attempts are
  recorded.
  
  ## Verification
  
  - Added
  `phase2_global_lock_can_be_claimed_after_retry_budget_is_exhausted` to
  cover the exhausted-budget lock claim path.
  - Ran `cargo test -p codex-state`.

  jif-oai · 2026-04-27 14:58:11 +02:00
  5d314f324c
• feat: use git-backed workspace diffs for memory consolidation (#18982)

  ## Why
  
  This PR make the `morpheus` agent (memory phase 2) use a git diff to
  start it's consolidation. The workflow is the following:
  1. The agent acquire a lock
  2. If `.codex/memories` does not exist or is not a git root, initialize
  everything (and make a first empty commit)
  3. Update `raw_memories.md` and `rollout_summaries/` as before.
  Basically we select max N phase 1 memories based on a given policy
  4. We use git (`gix`) to get a diff between the current state of
  `.codex/memories` and the last commit.
  5. Dump the diff in `phase2_workspace_diff.md`
  6. Spawn `morpheus` and point it to `phase2_workspace_diff.md`
  7. Wait for `morpheus` to be done
  8. Re-create a new `.git` and make one single commit on it. We do this
  because we don't want to preserve history through `.git` and this is
  cheap anyway
  9. We release the lock
  On top of this, we keep the retry policies etc etc
  
  The goals of this new workflow are:
  * Better support of any memory extensions such as `chronicle`
  * Allow the user to manually edit memories and this will be considered
  by the phase 2 agent
   
  As a follow-up we will need to add support for user's edition while
  `morpheus` is running
  
  ## What Changed
  
  - Added memory workspace helpers that prepare the git baseline, compute
  the diff, write `phase2_workspace_diff.md`, and reset the baseline after
  successful consolidation.
  - Updated Phase 2 to sync current inputs into `raw_memories.md` and
  `rollout_summaries/`, prune old extension resources, skip clean
  workspaces, and run the consolidation subagent only when the workspace
  has changes.
  - Tightened Phase 2 job ownership around long-running consolidation with
  heartbeats and an ownership check before resetting the baseline.
  - Simplified the prompt and state APIs so DB watermarks are bookkeeping,
  while workspace dirtiness decides whether consolidation work exists.
  - Updated the memory pipeline README and tests for workspace diffs,
  extension-resource cleanup, pollution-driven forgetting, selection
  ranking, and baseline persistence.
  
  ## Verification
  
  - Added/updated coverage in `core/src/memories/tests.rs`,
  `core/src/memories/workspace_tests.rs`, `state/src/runtime/memories.rs`,
  and `core/tests/suite/memories.rs`.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-27 14:32:44 +02:00
  01ab25dbb5
• Add goal persistence foundation (1 / 5) (#18073)

  Adds the persisted goal foundation for the rest of the stack. This PR is
  intentionally limited to feature flag and state-layer behavior;
  app-server APIs, model tools, runtime continuation, and TUI UX are
  layered in later PRs.
  
  ## Why
  
  Goal mode needs durable thread-level state before clients or model tools
  can safely build on it. The state layer needs to know whether a goal
  exists, what objective it tracks, whether it is active, paused,
  budget-limited, or complete, and how much time/token usage has already
  been accounted.
  
  ## What changed
  
  - Added the `goals` feature flag and generated config schema entry.
  - Added the `thread_goals` state table and Rust model for persisted
  thread goals.
  - Added state runtime APIs for creating, replacing, updating, deleting,
  and accounting goal usage.
  - Added `goal_id`-based stale update protection so an old goal update
  cannot overwrite a replacement.
  - Kept this PR scoped to persistence and state runtime behavior, with no
  app-server, model-facing, continuation, or TUI behavior yet.
  
  ## Verification
  
  - Added state runtime coverage for goal creation, replacement, stale
  update protection, status transitions, token-budget behavior, and usage
  accounting.

  Eric Traut · 2026-04-24 20:51:38 -07:00
  0ee737cea6
• feat: log client use min log level (#18661)

  In the log client, use the log level filter as a minimum severity
  instead of exact match
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-20 14:40:39 +01:00
  e1c289e11b
• Add sorting/backwardsCursor to thread/list and new thread/turns/list api (#17305)

  To improve performance of UI loads from the app, add two main
  improvements:
  1. The `thread/list` api now gets a `sortDirection` request field and a
  `backwardsCursor` to the response, which lets you paginate forwards and
  backwards from a window. This lets you fetch the first few items to
  display immediately while you paginate to fill in history, then can
  paginate "backwards" on future loads to catch up with any changes since
  the last UI load without a full reload of the entire data set.
  2. Added a new `thread/turns/list` api which also has sortDirection and
  backwardsCursor for the same behavior as `thread/list`, allowing you the
  same small-fetch for immediate display followed by background fill-in
  and resync catchup.

  David de Regt · 2026-04-17 11:49:02 -07:00
  eaf78e43f2
• Moving updated-at timestamps to unique millisecond times (#17489)

  To allow the ability to have guaranteed-unique cursors, we make two
  important updates:
  * Add new updated_at_ms and created_at_ms columns that are in
  millisecond precision
  * Guarantee uniqueness -- if multiple items are inserted at the same
  millisecond, bump the new one by one millisecond until it becomes unique
  
  This lets us use single-number cursors for forwards and backwards paging
  through resultsets and guarantee that the cursor is a fixed point to do
  (timestamp > cursor) and get new items only.
  
  This updated implementation is backwards-compatible since multiple
  appservers can be running and won't handle the previous method well.

  David de Regt · 2026-04-14 11:55:34 -04:00
  4f2fc3e3fa
• chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)

  ## Why
  
  `argument-comment-lint` was green in CI even though the repo still had
  many uncommented literal arguments. The main gap was target coverage:
  the repo wrapper did not force Cargo to inspect test-only call sites, so
  examples like the `latest_session_lookup_params(true, ...)` tests in
  `codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.
  
  This change cleans up the existing backlog, makes the default repo lint
  path cover all Cargo targets, and starts rolling that stricter CI
  enforcement out on the platform where it is currently validated.
  
  ## What changed
  
  - mechanically fixed existing `argument-comment-lint` violations across
  the `codex-rs` workspace, including tests, examples, and benches
  - updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
  `tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
  `--all-targets` unless the caller explicitly narrows the target set
  - fixed both wrappers so forwarded cargo arguments after `--` are
  preserved with a single separator
  - documented the new default behavior in
  `tools/argument-comment-lint/README.md`
  - updated `rust-ci` so the macOS lint lane keeps the plain wrapper
  invocation and therefore enforces `--all-targets`, while Linux and
  Windows temporarily pass `-- --lib --bins`
  
  That temporary CI split keeps the stricter all-targets check where it is
  already cleaned up, while leaving room to finish the remaining Linux-
  and Windows-specific target-gated cleanup before enabling
  `--all-targets` on those runners. The Linux and Windows failures on the
  intermediate revision were caused by the wrapper forwarding bug, not by
  additional lint findings in those lanes.
  
  ## Validation
  
  - `bash -n tools/argument-comment-lint/run.sh`
  - `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
  - shell-level wrapper forwarding check for `-- --lib --bins`
  - shell-level wrapper forwarding check for `-- --tests`
  - `just argument-comment-lint`
  - `cargo test` in `tools/argument-comment-lint`
  - `cargo test -p codex-terminal-detection`
  
  ## Follow-up
  
  - Clean up remaining Linux-only target-gated callsites, then switch the
  Linux lint lane back to the plain wrapper invocation.
  - Clean up remaining Windows-only target-gated callsites, then switch
  the Windows lint lane back to the plain wrapper invocation.

  Michael Bolin · 2026-03-27 19:00:44 -07:00
  61dfe0b86c
• feat: change multi-agent to use path-like system instead of uuids (#15313)

  This PR add an URI-based system to reference agents within a tree. This
  comes from a sync between research and engineering.
  
  The main agent (the one manually spawned by a user) is always called
  `/root`. Any sub-agent spawned by it will be `/root/agent_1` for example
  where `agent_1` is chosen by the model.
  
  Any agent can contact any agents using the path.
  
  Paths can be used either in absolute or relative to the calling agents
  
  Resume is not supported for now on this new path

  jif-oai · 2026-03-20 18:23:48 +00:00
  79ad7b247b
• feat: add graph representation of agent network (#15056)

  Add a representation of the agent graph. This is now used for:
  * Cascade close agents (when I close a parent, it close the kids)
  * Cascade resume (oposite)
  
  Later, this will also be used for post-compaction stuffing of the
  context
  
  Direct fix for: https://github.com/openai/codex/issues/14458

  jif-oai · 2026-03-19 10:21:25 +00:00
  70cdb17703
• Align SQLite feedback logs with feedback formatter (#13494)

  ## Summary
  - store a pre-rendered `feedback_log_body` in SQLite so `/feedback`
  exports keep span prefixes and structured event fields
  - render SQLite feedback exports with timestamps and level prefixes to
  match the old in-memory feedback formatter, while preserving existing
  trailing newlines
  - count `feedback_log_body` in the SQLite retention budget so structured
  or span-prefixed rows still prune correctly
  - bound `/feedback` row loading in SQL with the retention estimate, then
  apply exact whole-line truncation in Rust so uploads stay capped without
  splitting lines
  
  ## Details
  - add a `feedback_log_body` column to `logs` and backfill it from
  `message` for existing rows
  - capture span names plus formatted span and event fields at write time,
  since SQLite does not retain enough structure to reconstruct the old
  formatter later
  - keep SQLite feedback queries scoped to the requested thread plus
  same-process threadless rows
  - restore a SQL-side cumulative `estimated_bytes` cap for feedback
  export queries so over-retained partitions do not load every matching
  row before truncation
  - add focused formatting coverage for exported feedback lines and parity
  coverage against `tracing_subscriber`
  
  ## Testing
  - cargo test -p codex-state
  - just fix -p codex-state
  - just fmt
  
  codex author: `codex resume 019ca1b0-0ecc-78b1-85eb-6befdd7e4f1f`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Charley Cunningham · 2026-03-18 22:44:31 +00:00
  ebbbc52ce4
• Feat: CXA-1831 Persist latest model and reasoning effort in sqlite (#14859)

  ### Summary
  The goal is for us to get the latest turn model and reasoning effort on
  thread/resume is no override is provided on the thread/resume func call.
  This is the part 1 which we write the model and reasoning effort for a
  thread to the sqlite db and there will be a followup PR to consume the
  two new fields on thread/resume.
  
  [part 2 PR is currently WIP](https://github.com/openai/codex/pull/14888)
  and this one can be merged independently.

  Shijie Rao · 2026-03-17 10:14:34 -07:00
  8e258eb3f5
• Add thread metadata update endpoint to app server (#13280)

  ## Summary
  - add the v2 `thread/metadata/update` API, including
  protocol/schema/TypeScript exports and app-server docs
  - patch stored thread `gitInfo` in sqlite without resuming the thread,
  with validation plus support for explicit `null` clears
  - repair missing sqlite thread rows from rollout data before patching,
  and make those repairs safe by inserting only when absent and updating
  only git columns so newer metadata is not clobbered
  - keep sqlite authoritative for mutable thread git metadata by
  preserving existing sqlite git fields during reconcile/backfill and only
  using rollout `SessionMeta` git fields to fill gaps
  - add regression coverage for the endpoint, repair paths, concurrent
  sqlite writes, clearing git fields, and rollout/backfill reconciliation
  - fix the login server shutdown race so cancelling before the waiter
  starts still terminates `block_until_done()` correctly
  
  ## Testing
  - `cargo test -p codex-state
  apply_rollout_items_preserves_existing_git_branch_and_fills_missing_git_fields`
  - `cargo test -p codex-state
  update_thread_git_info_preserves_newer_non_git_metadata`
  - `cargo test -p codex-core
  backfill_sessions_preserves_existing_git_branch_and_fills_missing_git_fields`
  - `cargo test -p codex-app-server thread_metadata_update`
  - `cargo test`
  - currently fails in existing `codex-core` grep-files tests with
  `unsupported call: grep_files`:
      - `suite::grep_files::grep_files_tool_collects_matches`
      - `suite::grep_files::grep_files_tool_reports_empty_results`

  joeytrasatti-openai · 2026-03-03 15:56:11 -08:00
  935754baa3
• feat: polluted memories (#13008)

  Add a feature flag to disable memory creation for "polluted"

  jif-oai · 2026-03-02 11:57:32 +00:00
  b649953845
• feat: add git info to memories (#12940)
  jif-oai · 2026-02-26 20:14:13 +00:00
  a6065d30f4
• feat: memories forgetting (#12900)

  Add diff based memory forgetting

  jif-oai · 2026-02-26 13:19:57 +00:00
  382fa338b3
• Add rollout path to memory files and search for them during read (#12684)

  Co-authored-by: jif-oai <jif@openai.com>

  Wendy Jiao · 2026-02-26 10:57:01 +00:00
  52aa49db1b
• Agent jobs (spawn_agents_on_csv) + progress UI (#10935)

  ## Summary
  - Add agent job support: spawn a batch of sub-agents from CSV, auto-run,
  auto-export, and store results in SQLite.
  - Simplify workflow: remove run/resume/get-status/export tools; spawn is
  deterministic and completes in one call.
  - Improve exec UX: stable, single-line progress bar with ETA; suppress
  sub-agent chatter in exec.
  
  ## Why
  Enables map-reduce style workflows over arbitrarily large repos using
  the existing Codex orchestrator. This addresses review feedback about
  overly complex job controls and non-deterministic monitoring.
  
  ## Demo (progress bar)
  ```
  ./codex-rs/target/debug/codex exec \
    --enable collab \
    --enable sqlite \
    --full-auto \
    --progress-cursor \
    -c agents.max_threads=16 \
    -C /Users/daveaitel/code/codex \
    - <<'PROMPT'
  Create /tmp/agent_job_progress_demo.csv with columns: path,area and 30 rows:
  path = item-01..item-30, area = test.
  
  Then call spawn_agents_on_csv with:
  - csv_path: /tmp/agent_job_progress_demo.csv
  - instruction: "Run `python - <<'PY'` to sleep a random 0.3–1.2s, then output JSON with keys: path, score (int). Set score = 1."
  - output_csv_path: /tmp/agent_job_progress_demo_out.csv
  PROMPT
  ```
  
  ## Review feedback addressed
  - Auto-start jobs on spawn; removed run/resume/status/export tools.
  - Auto-export on success.
  - More descriptive tool spec + clearer prompts.
  - Avoid deadlocks on spawn failure; pending/running handled safely.
  - Progress bar no longer scrolls; stable single-line redraw.
  
  ## Tests
  - `cd codex-rs && cargo test -p codex-exec`
  - `cd codex-rs && cargo build -p codex-cli`

  daveaitel-openai · 2026-02-24 21:00:19 +00:00
  dcab40123f
• feat: add nick name to sub-agents (#12320)

  Adding random nick name to sub-agents. Used for UX
  
  At the same time, also storing and wiring the role of the sub-agent

  jif-oai · 2026-02-20 14:39:49 +00:00
  0f9eed3a6f
• feat: add --search to just log (#11995)

  Summary
  - extend the log client to accept an optional `--search` substring
  filter when querying codex-state logs
  - propagate the filter through `LogQuery` and apply it in
  `push_log_filters` via `INSTR(message, ...)`
  - add an integration test that exercises the new search filtering
  behavior
  
  Testing
  - Not run (not requested)

  jif-oai · 2026-02-17 14:19:52 +00:00
  31d4bfdde0
• Add process_uuid to sqlite logs (#11534)

  ## Summary
  This PR is the first slice of the per-session `/feedback` logging work:
  it adds a process-unique identifier to SQLite log rows.
  
  It does **not** change `/feedback` sourcing behavior yet.
  
  ## Changes
  - Add migration `0009_logs_process_id.sql` to extend `logs` with:
    - `process_uuid TEXT`
    - `idx_logs_process_uuid` index
  - Extend state log models:
    - `LogEntry.process_uuid: Option<String>`
    - `LogRow.process_uuid: Option<String>`
  - Stamp each log row with a stable per-process UUID in the sqlite log
  layer:
    - generated once per process as `pid:<pid>:<uuid>`
  - Update sqlite log insert/query paths to persist and read
  `process_uuid`:
    - `INSERT INTO logs (..., process_uuid, ...)`
    - `SELECT ..., process_uuid, ... FROM logs`
  
  ## Why
  App-server runs many sessions in one process. This change provides a
  process-scoping primitive we need for follow-up `/feedback` work, so
  threadless/process-level logs can be associated with the emitting
  process without mixing across processes.
  
  ## Non-goals in this PR
  - No `/feedback` transport/source changes
  - No attachment size changes
  - No sqlite retention/trim policy changes
  
  ## Testing
  - `just fmt`
  - CI will run the full checks

  Charley Cunningham · 2026-02-14 17:27:22 -08:00
  fce4ad9cf4
• feat: add slug in name (#11739)
  jif-oai · 2026-02-13 15:24:03 +00:00
  db66d827be
• Add cwd to memory files (#11591)

  Add cwd to memory files so that model can deal with multi cwd memory
  better.
  
  ---------
  
  Co-authored-by: jif-oai <jif@openai.com>

  Wendy Jiao · 2026-02-12 17:46:49 +00:00
  88c5ca2573
• feat: make sandbox read access configurable with ReadOnlyAccess (#11387)

  `SandboxPolicy::ReadOnly` previously implied broad read access and could
  not express a narrower read surface.
  This change introduces an explicit read-access model so we can support
  user-configurable read restrictions in follow-up work, while preserving
  current behavior today.
  
  It also ensures unsupported backends fail closed for restricted-read
  policies instead of silently granting broader access than intended.
  
  ## What
  
  - Added `ReadOnlyAccess` in protocol with:
    - `Restricted { include_platform_defaults, readable_roots }`
    - `FullAccess`
  - Updated `SandboxPolicy` to carry read-access configuration:
    - `ReadOnly { access: ReadOnlyAccess }`
    - `WorkspaceWrite { ..., read_only_access: ReadOnlyAccess }`
  - Preserved existing behavior by defaulting current construction paths
  to `ReadOnlyAccess::FullAccess`.
  - Threaded the new fields through sandbox policy consumers and call
  sites across `core`, `tui`, `linux-sandbox`, `windows-sandbox`, and
  related tests.
  - Updated Seatbelt policy generation to honor restricted read roots by
  emitting scoped read rules when full read access is not granted.
  - Added fail-closed behavior on Linux and Windows backends when
  restricted read access is requested but not yet implemented there
  (`UnsupportedOperation`).
  - Regenerated app-server protocol schema and TypeScript artifacts,
  including `ReadOnlyAccess`.
  
  ## Compatibility / rollout
  
  - Runtime behavior remains unchanged by default (`FullAccess`).
  - API/schema changes are in place so future config wiring can enable
  restricted read access without another policy-shape migration.

  Michael Bolin · 2026-02-11 18:31:14 -08:00
  abbd74e2be
• feat: mem v2 - PR5 (#11372)
  jif-oai · 2026-02-10 23:22:55 +00:00
  2c9be54c9a
• chore: unify memory job flow (#11334)
  jif-oai · 2026-02-10 20:26:39 +00:00
  a6e9469fa4
• feat: align memory phase 1 and make it stronger (#11300)

  ## Align with the new phase-1 design
  
  Basically we know run phase 1 in parallel by considering:
  * Max 64 rollouts
  * Max 1 month old
  * Consider the most recent first
  
  This PR also adds stronger parallelization capabilities by detecting
  stale jobs, retry policies, ownership of computation to prevent double
  computations etc etc

  jif-oai · 2026-02-10 13:42:09 +00:00
  1d5eba0090
• memories: add extraction and prompt module foundation (#11200)

  ## Summary
  - add the new `core/src/memories` module (phase-one parsing, rollout
  filtering, storage, selection, prompts)
  - add Askama-backed memory templates for stage-one input/system and
  consolidation prompts
  - add module tests for parsing, filtering, path bucketing, and summary
  maintenance
  
  ## Testing
  - just fmt
  - cargo test -p codex-core --lib memories::

  jif-oai · 2026-02-10 10:10:24 +00:00
  6049ff02a0
• Leverage state DB metadata for thread summaries (#10621)

  Summary:
  - read conversation summaries and cwd info from the state DB when
  possible so we no longer rely on rollout files for metadata and avoid
  extra I/O
  - persist CLI version in thread metadata, surface it through summary
  builders, and add the necessary DB migration hooks
  - simplify thread listing by using enriched state DB data directly
  rather than reading rollout heads
  
  Testing:
  - Not run (not requested)

  jif-oai · 2026-02-05 16:39:11 +00:00
  9ee746afd6
• feat: resumable backfill (#10745)

  ## Summary
  
  This PR makes SQLite rollout backfill resumable and repeatable instead
  of one-shot-on-db-create.
  
  ## What changed
  
  - Added a persisted backfill state table:
    - state/migrations/0008_backfill_state.sql
  - Tracks status (pending|running|complete), last_watermark, and
  last_success_at.
  - Added backfill state model/types in codex-state:
    - BackfillState, BackfillStatus (state/src/model/backfill_state.rs)
  - Added runtime APIs to manage backfill lifecycle/progress:
    - get_backfill_state
    - mark_backfill_running
    - checkpoint_backfill
    - mark_backfill_complete
  - Updated core startup behavior:
  - Backfill now runs whenever state is not Complete (not only when DB
  file is newly created).
  - Reworked backfill execution:
  - Collect rollout files, derive deterministic watermark per path, sort,
  resume from last_watermark.
  - Process in batches (BACKFILL_BATCH_SIZE = 200), checkpoint after each
  batch.
    - Mark complete with last_success_at at the end.
  
  ## Why
  
  Previous behavior could leave users permanently partially backfilled if
  the process exited during initial async backfill. This change allows
  safe continuation across restarts and avoids restarting from scratch.

  jif-oai · 2026-02-05 14:34:34 +00:00
  4033f905c6
• feat: add phase 1 mem db (#10634)

  - Schema: thread_id (PK, FK to threads.id with cascade delete),
  trace_summary, memory_summary, updated_at.
  - Migration: creates the table and an index on (updated_at DESC,
  thread_id DESC) for efficient recent-first reads.
    - Runtime API (DB-only):
        - `get_thread_memory(thread_id)`: fetch one memory row.
  - `upsert_thread_memory(thread_id, trace_summary, memory_summary)`:
  insert/update by thread id and always advance updated_at.
  - `get_last_n_thread_memories_for_cwd(cwd, n)`: join thread_memory with
  threads and return newest n rows for an exact cwd match.
  - Model layer: introduced ThreadMemory and row conversion types to keep
  query decoding typed and consistent with existing state models.

  jif-oai · 2026-02-04 21:38:39 +00:00
  4922b3e571
• feat: improve logs client (#10229)
  jif-oai · 2026-01-30 18:23:18 +01:00
  eff11f792b
• feat: reduce span exposition (#10171)

  This only avoids the creation of duplicates spans

  jif-oai · 2026-01-29 18:15:22 +00:00
  798c4b3260
• chore: unify log queries (#10152)

  Unify log queries to only have SQLX code in the runtime and use it for
  both the log client and for tests

  jif-oai · 2026-01-29 16:28:15 +00:00
  e6c4f548ab
• feat: adding thread ID to logs + filter in the client (#10150)
  jif-oai · 2026-01-29 16:53:30 +01:00
  89c5f3c4d4
• chore: improve client (#10149)

  <img width="883" height="84" alt="Screenshot 2026-01-29 at 11 13 12"
  src="https://github.com/user-attachments/assets/090a2fec-94ed-4c0f-aee5-1653ed8b1439"
  />

  jif-oai · 2026-01-29 11:25:22 +01:00
  4ba911d48c
• feat: add log db (#10086)

  Add a log DB. The goal is just to store our logs in a `.sqlite` DB to
  make it easier to crawl them and drop the oldest ones.

  jif-oai · 2026-01-29 10:23:03 +01:00
  780482da84