109 Commits

• Persist session IDs across thread resume (#29327)

  ## Summary
  
  A cold-resumed subagent kept its durable thread ID but could receive a
  new session ID, splitting one agent tree across multiple sessions after
  a restart.
  
  Persist the root session ID in every rollout `SessionMeta`, carry it
  through thread creation, and restore it before initializing the resumed
  `Session` and `AgentControl`.
  
  ## Behavior
  
  For a nested agent tree:
  
  ```text
  root session R
    parent thread P
      child thread C
  ```
  
  The child rollout stores:
  
  ```text
  session_id:       R
  parent_thread_id: P
  id:               C
  ```
  
  After a cold resume, the child still belongs to root session `R` while
  its immediate parent remains `P`. The integration coverage uses distinct
  values for all three IDs so it catches restoring the session from
  `parent_thread_id`.
  
  ## Legacy rollouts
  
  Previous rollouts have `id` but no `session_id`. `SessionMetaLine`
  deserialization treats a missing `session_id` as `id`, keeping those
  files readable, listable, and resumable. When a legacy subagent is
  resumed through its root, that synthesized child ID no longer overrides
  the inherited root-scoped `AgentControl`. New rollouts always persist
  the explicit root session ID.

  jif · 2026-06-22 09:36:08 +02:00
  6d15bb3d17
• [codex] Restore thread recency with compatible migration history (#28671)

  ## Summary
  
  - Revert #28655, restoring the thread `recencyAt` behavior introduced by
  #27910.
  - Move `threads_recency_at` to migration 0039 so it no longer collides
  with `external_agent_config_imports` at version 0038.
  - Repair databases that already applied the recency migration as version
  38 by moving the matching migration-history row to version 39 before
  SQLx validation. The current version-38 migration can then apply
  normally.
  
  ## Validation
  
  - `just test -p codex-state
  migrations::tests::repairs_recency_migration_that_was_applied_as_version_38`
  - `just test -p codex-state -p codex-rollout -p codex-thread-store -p
  codex-app-server-protocol -p codex-tui`: 3,439 passed; six TUI tests
  could not open the machine's existing read-only incident database at
  `~/.codex/sqlite/state_5.sqlite`.
  - `just fix -p codex-state`
  - `just fmt`
  - Verified that state migration versions are unique.

  Jeremy Rose · 2026-06-17 18:52:18 +00:00
  7dc7096ae1
• Revert thread recencyAt for sidebar ordering (#28655)

  ## Why
  
  Revert #27910 to remove the newly introduced thread `recencyAt`
  persistence and API behavior from `main`.
  
  ## What changed
  
  This reverts commit `fac3158c2a783095768076489815f361fa9b0db4`,
  including the state migration, thread-store propagation, app-server API
  surface, generated schemas, and related tests.
  
  ## Validation
  
  Not run before opening; relying on CI for the initial fast signal.

  pakrym-oai · 2026-06-16 21:39:30 -07:00
  cb15c64760
• [codex] core: restore absolute turn context cwd (#28629)

  ## Why
  
  #28152 jumped the gun on moving the rollout format to store URIs, and
  would likely break compat with some features that don't go through the
  same types as the core logic.
  
  ## What
  
  Make `TurnContextItem.cwd` an `AbsolutePathBuf` again, remove test added
  for `PathUri` serialization in rollouts. Also drops a bunch of error
  paths that are no longer needed.

  Adam Perry @ OpenAI · 2026-06-16 19:05:26 -07:00
  3d02c443bc
• Add thread recencyAt for sidebar ordering (#27910)

  ## Summary
  
  Add a server-owned `recencyAt` timestamp and `recency_at` thread-list
  sort key for product recency ordering while preserving the existing
  meaning of `updatedAt` as the latest persisted thread mutation.
  
  This is the server-side alternative to #27697. Rather than narrowing
  `updatedAt`, clients can sort the sidebar by `recency_at` and continue
  treating `updatedAt` as mutation time.
  
  Paired Codex Apps PR:
  [openai/openai#1024599](https://github.com/openai/openai/pull/1024599)
  
  ## Contract
  
  - `recencyAt` initializes when a thread is created.
  - A turn start advances `recencyAt` monotonically.
  - Commentary, agent output, tool results, token/accounting updates, turn
  completion, archive, unarchive, resume, and generic metadata writes do
  not advance it.
  - `updatedAt` retains its existing behavior and continues to advance for
  persisted thread mutations.
  - Current servers populate `recencyAt`; the response field is optional
  in generated TypeScript so clients connected to older servers can fall
  back to `updatedAt`.
  - Filesystem-only fallback uses existing updated/mtime ordering when
  SQLite is unavailable.
  
  ## Persistence and compatibility
  
  Migration 0038 adds second- and millisecond-precision recency columns,
  backfills them from the existing updated timestamp, creates list
  indexes, and includes an insert trigger so older binaries writing to a
  migrated database seed recency without causing later mutations to
  advance it.
  
  Generic metadata upserts preserve existing recency values. Turn-start
  updates use a dedicated monotonic touch, and process-local allocation
  keeps millisecond cursor values unique. State DB list, search, read,
  filtered-list repair, rollout fallback propagation, and app-server
  conversions all carry the new field.
  
  ## API
  
  `Thread` responses include:
  
  ```ts
  recencyAt?: number
  ```
  
  `thread/list` and `thread/search` accept:
  
  ```json
  { "sortKey": "recency_at" }
  ```
  
  Generated TypeScript and JSON schemas are included.
  
  ## Validation
  
  - `just test -p codex-state` — 146 passed
  - `just test -p codex-rollout` — 69 passed
  - `just test -p codex-thread-store` — 81 passed
  - `just test -p codex-app-server-protocol` — 231 passed
  - Focused app-server list ordering, response mapping, archive/unarchive,
  and resume lifecycle tests passed
  - Scoped `just fix` for state, rollout, thread-store,
  app-server-protocol, and app-server
  - `just fmt`
  - `git diff --check`
  - Independent correctness, simplicity, elegance, security, and
  test-quality reviews; actionable ordering, lifecycle, query-projection,
  and timestamp-uniqueness findings were addressed

  Jeremy Rose · 2026-06-16 17:06:22 -07:00
  fac3158c2a
• core: render remote environment cwd natively (#28152)

  ## Why
  
  Model-visible `<environment_context>` should match the environment of
  the executor, not of the app server.
  
  Stacked on #28146.
  
  ## What
  
  - Keep selected environment cwd values as `PathUri` while building
  environment context.
  - Render cwd text using the path convention represented by the URI, with
  the canonical URI as a fallback.
  - Preserve compatibility with legacy `TurnContextItem.cwd` values when
  reconstructing and diffing context.
  - Extend the Wine-backed remote Windows test to assert that the model
  sees `powershell` and `C:\windows`.

  Adam Perry @ OpenAI · 2026-06-16 16:17:47 -07:00
  4c79527e31
• [codex] Record external agent import results (#28396)

  ## Summary
  - restore `externalAgentConfig/import/progress` notifications while
  keeping `externalAgentConfig/import/completed` as the must-deliver event
  - persist completed external-agent config imports in state DB by
  `importId`, including concrete success/failure details for config,
  AGENTS.md, skills, plugins, MCP servers, subagents, hooks, commands, and
  sessions
  - add `externalAgentConfig/import/readHistories` so clients can recover
  persisted import results after missing the live completion notification
  - include `errorType` on import failures in protocol
  responses/notifications and persisted DB JSON so future code can
  classify failures without another wire/storage shape change
  
  ## Validation
  - `git diff --check`
  - `just test -p codex-state external_agent_config_imports`
  - `just test -p codex-app-server-protocol`
  - `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-sqlite-read-details
  just test -p codex-app-server
  external_agent_config_import_sends_completion_notification_for_sync_only_import`
  
  Also ran earlier broader checks before publishing:
  - `just test -p codex-state`
  -
  `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-external-agent-test-sqlite
  just test -p codex-app-server external_agent_config`
  - `just test -p codex-external-agent-migration`

  charlesgong-openai · 2026-06-15 23:17:24 -07:00
  314fa3d25b
• feat(app-server): filter threads by parent (#26662)

  ## Why
  
  Clients that display or coordinate spawned subagents need an
  authoritative snapshot of a thread's immediate spawned children when
  they connect to app-server or recover after missing live events.
  `thread/list` cannot query by parent, so clients must otherwise scan
  unrelated threads or reconstruct relationships from rollout history and
  transient events.
  
  The direct spawn relationship already exists in persisted
  `thread_spawn_edges` state. Review and Guardian threads do not
  participate in that lifecycle and are intentionally outside this
  filter's scope.
  
  ## What changed
  
  This adds an experimental `parentThreadId` filter to `thread/list`.
  Parent-filtered requests return direct spawned children from persisted
  state while preserving the existing response shape, explicit filters,
  sorting, and timestamp-only cursor behavior. The lookup does not read
  rollout transcripts or recursively return descendants.
  
  Supersedes #25112 with the narrower `thread/list` filter approach.
  
  ## How it works
  
  1. An experimental client passes a valid thread ID as `parentThreadId`.
  2. App-server routes the list through the existing thread-store and
  state-database boundaries.
  3. SQLite selects threads whose IDs have a direct persisted spawn edge
  from that parent.
  4. Omitted provider and source filters include all values; explicit
  filters keep ordinary `thread/list` semantics.
  5. Grandchildren, Review threads, and Guardian threads are excluded.
  
  ## Verification
  
  State (144 tests), rollout (69 tests), and focused app-server
  thread-list (31 tests) suites passed. Scoped Clippy checks and
  repository formatting also passed. Coverage includes direct spawned
  children, omitted grandchildren, pagination, malformed IDs, mixed source
  kinds, explicit filters, and operation without rollout files.

  Brent Traut · 2026-06-14 00:14:26 -07:00
  dfd03ea01b
• Support plaintext agent messages (#27830)

  ## Why
  
  Multi-agent v2 `send_message` deliveries already reach the receiving
  model as typed `agent_message` items with encrypted content.
  Child-completion notifications are generated by Codex itself, so their
  content is plaintext and previously fell back to a serialized JSON
  envelope inside an assistant message.
  
  With plaintext `input_text` supported for `agent_message`, both delivery
  paths can use the same model-visible type while preserving explicit
  author and recipient metadata.
  
  ## What changed
  
  - add plaintext `input_text` support to `AgentMessageInputContent` and
  regenerate the affected app-server schemas
  - preserve `InterAgentCommunication` as structured mailbox input instead
  of converting it to assistant text
  - record delivered communications as typed `agent_message` history items
  - persist a dedicated rollout item so local delivery metadata such as
  `trigger_turn` remains available without leaking into the Responses
  request
  - reconstruct typed agent messages on resume and preserve fork-turn
  truncation behavior
  - remove request-time assistant-content parsing
  - preserve plaintext and encrypted inter-agent deliveries in stage-one
  memory inputs
  - normalize and link plaintext and encrypted agent messages in rollout
  traces without treating inbound messages as child results
  - cover the real MultiAgent V2 child-completion path end to end with
  deterministic mailbox synchronization
  
  ## Verification
  
  - `just test -p codex-core
  plaintext_multi_agent_v2_completion_sends_agent_message`
  - `just test -p codex-core input_queue_drains_mailbox_in_delivery_order
  record_initial_history_reconstructs_typed_inter_agent_message
  fork_turn_positions_use_inter_agent_delivery_metadata`
  - `just test -p codex-memories-write
  serializes_inter_agent_communications_for_memory`
  - `just test -p codex-rollout-trace
  agent_messages_preserve_routing_and_content
  sub_agent_started_activity_creates_spawn_edge`
  - `just test -p codex-rollout-trace
  agent_result_edge_falls_back_to_child_thread_without_result_message`
  - `just test -p codex-protocol -p codex-rollout -p
  codex-app-server-protocol`

  jif · 2026-06-12 13:50:04 -07:00
  8f2d6416ce
• feat(app-server): persist remote-control desired state (#27445)

  ## Why
  
  Remote-control runtime enablement and persisted enrollment preference
  were represented by separate flags. That made startup rehydration, RPC
  persistence, and new-enrollment seeding race with one another, and it
  did not cleanly distinguish runtime-only CLI or daemon starts from
  durable app-server RPC changes.
  
  ## What Changed
  
  - Replace the parallel enablement, seed, and rehydration flags with one
  transport-owned `RemoteControlDesiredState`.
  - Add nullable enrollment-scoped persistence and preserve existing
  preferences during enrollment upserts.
  - Rehydrate plain startup only after auth and client scope resolve,
  without overwriting a concurrent RPC transition.
  - Make ordinary `remoteControl/enable` and `remoteControl/disable`
  durable while retaining `ephemeral: true` for runtime-only callers.
  - Have the daemon explicitly request ephemeral enablement and regenerate
  the app-server schemas.
  
  ## Verification
  
  - Covered migration and `NULL`/`0`/`1` persistence round trips.
  - Covered plain-start rehydration and runtime-only versus durable
  enrollment seeding.
  - Covered durable enable, durable disable, and ephemeral enable through
  app-server RPC.
  - Covered the daemon's exact `{ "ephemeral": true }` request payload.
  
  Related issue: N/A (internal remote-control persistence architecture
  change).

  Anton Panasenko · 2026-06-11 21:28:52 -07:00
  d61dfeb23a
• fix: Auto-recover from corrupted sqlite databases (#26859)

  Further investigation of the sqlite incidents showed that the problems
  are due to corruption from the older version of SQLite that we recently
  upgraded, and that the data is truly corrupted in the root database --
  recovery of all data is not possible. Given that the data is
  reconstructable from the rollouts on disk, we should just auto-backup
  the database and let codex rebuild the rollout info from the disk
  rollouts.
  
  The new behavior is that appserver auto-backs-up and rebuilds (with logs
  reflecting that behavior). The CLI now pops a message letting you know
  this happened and the paths of the backed-up corrupt db and the new
  database. There is also context added so that the desktop app can read
  the rebuild info from it and inform the user with it.

  David de Regt · 2026-06-10 11:24:29 -07:00
  3691fe5b76
• Add app-server thread/delete API (#25018)

  ## Why
  
  Clients can archive and unarchive threads today, but there is no
  app-server API for permanently removing a thread. Deletion also needs to
  cover the full session tree: deleting a main thread should remove
  spawned subagent threads and the related local metadata instead of
  leaving orphaned rollout files, goals, or subagent state behind.
  
  ## What
  
  - Adds the v2 `thread/delete` request and `thread/deleted` notification,
  with the response shape kept consistent with `thread/archive`.
  - Implements local hard delete for active and archived rollout files.
  - Deletes the requested thread's state DB row as the commit point, then
  best-effort cleans associated state including spawned descendants,
  goals, spawn edges, logs, dynamic tools, and agent job assignments.
  - Updates app-server API docs and generated protocol schema/TypeScript
  fixtures.

  Eric Traut · 2026-06-10 11:22:12 -07:00
  a19d43a40a
• Index visible thread list ordering (#27391)

  ## Summary
  
  - add partial SQLite indexes for visible thread lists ordered by
  creation or update time
  - match the `archived` and non-empty `preview` filters used by
  `thread/list`
  - add query-plan coverage for both supported sort orders
  
  ## Query performance
  
  Benchmarked the production query shape on a snapshot of my database with
  ~10k threads before and after applying these indexes. The query selected
  the full thread projection with `archived = 0`, `preview <> ''`, the
  `openai` provider filter, and a page size of 201. Results are the mean
  of 30 runs after 5 warmups:
  
  | Query | Before | After | Speedup |
  | --- | ---: | ---: | ---: |
  | First page, `created_at_ms DESC` | 132.3 ms | 15.1 ms | 8.78x |
  | First page, `updated_at_ms DESC` | 123.6 ms | 15.5 ms | 7.99x |
  | Cursor page near row 4,000, `created_at_ms DESC` | 51.8 ms | 16.8 ms |
  3.07x |
  | Cursor page near row 4,000, `updated_at_ms DESC` | 52.4 ms | 17.1 ms |
  3.06x |
  
  Before this change, SQLite used `idx_threads_archived`, filtered the
  candidate rows, and built a temporary B-tree for the requested ordering.
  With the partial indexes, SQLite reads matching visible rows directly in
  timestamp order and stops at the page limit. `EXPLAIN QUERY PLAN` no
  longer reports `USE TEMP B-TREE FOR ORDER BY`.
  
  The result rows were identical before and after. The two partial indexes
  occupy approximately 168 KiB combined on this snapshot.
  
  ## Performance under contention
  
  I noticed this issue on a database with high-contention and tried to use
  simulated contention to validate the performance in that context.
  
  A synthetic SQLite benchmark ran five concurrent readers, matching the
  state database pool size, and fetched 101 rows per query. Results are
  the median of three runs on fresh copies of the same database snapshot:
  
  | Query | Before | After |
  | --- | ---: | ---: |
  | `created_at_ms` mean latency under saturation | 328 ms | 12 ms |
  | `created_at_ms` throughput | 16 queries/s | 412 queries/s |
  | `updated_at_ms` mean latency under saturation | 336 ms | 14 ms |
  | `updated_at_ms` throughput | 15 queries/s | 357 queries/s |
  
  For a burst of 100 queries queued through five connections, p95
  completion time fell from 6.90 seconds to 226 ms for `created_at_ms`,
  and from 6.31 seconds to 473 ms for `updated_at_ms`.
  
  ## Validation
  
  - `just test -p codex-state` (135 tests passed)
  - query-plan regression covers created-at and updated-at ordering,
  requires the corresponding index, and rejects `TEMP B-TREE`
  - `just fmt`

  Zanie Blue · 2026-06-10 11:52:17 -05:00
  2ef007dc1a
• [codex-analytics] emit goal lifecycle analytics (#27078)

  ## Why
  - Currently, there is no analytics event for `/goal` behavior
  - Existing events cannot identify goal execution or its resulting
  outcome
  - The original update in
  [#26182](https://github.com/openai/codex/pull/26182) was implemented
  before `/goal` moved into `codex-goal-extension`.
  
  ## What Changed
  - Adds `codex_goal_event` serialization and enrichment to
  `codex-analytics`
  - Emits goal events from the canonical `codex-goal-extension` mutation
  and accounting paths:
    - `created` when a new logical goal is persisted
    - `usage_accounted` when cumulative goal usage is persisted
    - `status_changed` when the stored goal status changes
    - `cleared` when the goal is deleted
  - Preserves causal `turn_id` for turn driven events and uses null
  attribution for external or idle lifecycle events
  - Changes goal deletion to return the deleted row so `cleared` retains
  the stable goal ID
  
  ## Event Details
  
  Includes standard analytics metadata along with goal specific fields:
  - `goal_id`: Stable ID stored in the local SQLite goal row and shared
  across the goal's events
  - `event_kind`: Observed operation (see the 4 lifecycle events cited in
  the above bullet)
  - `goal_status`: Resulting or last stored status: `active`, `paused`,
  `blocked`, `usage_limited`, etc.
    - `has_token_budget`: Indicates whether a token budget is configured
    - `turn_id`: Causal turn ID, or null when no causal turn exists
  - `cumulative_tokens_accounted`: Cumulative tokens on `usage_accounted`
  events; null otherwise
  - `cumulative_time_accounted_seconds`: Cumulative active time on
  `usage_accounted` events; null otherwise
  
  ## Validation
  - `just test -p codex-analytics -p codex-state -p codex-goal-extension`
  - `just test -p codex-core -E 'test(/goal/)'`
  - `just test -p codex-app-server`
  - `cargo build -p codex-analytics -p codex-core -p codex-state -p
  codex-app-server`

  marksteinbrick-oai · 2026-06-09 18:45:54 -07:00
  608b8b1cc6
• Allow creating a new goal after completion (#26681)

  ## Why
  
  Users have indicated that they want an agent to be able to create a new
  goal for itself after completing the previous goal. Currently, that's
  not possible because agents cannot overwrite an existing goal even if
  it's complete. This PR removes this limitation and allows `create_goal`
  to overwrite an existing goal if it is in the `complete` state.
  
  ## What changed
  
  `create_goal` now replaces the existing goal only when its status is
  `complete`. The replacement is performed atomically in the goal store,
  creates a fresh active goal with reset usage, and continues to reject
  creation while any unfinished goal exists. App server clients see a
  single `thread/goal/updated` event when the previous goal is replaced
  with the new one.
  
  The tool description and error message now reflect these semantics.
  
  ## What didn't change
  
  Agents are not allowed to create a new goal (overwrite their existing
  goal) if an existing goal is still active, blocked, paused, or in any
  other state other than "completed".

  Eric Traut · 2026-06-09 13:23:31 -07:00
  36bf63a5cf
• [codex-analytics] add extensible feature thread sources (#27063)

  ## Why
  - `ThreadSource` currently defines a closed set of core-owned values
  - Product features also create threads for background or scheduled work
  - Adding every product-specific value to the core enum would require
  repeated `codex-rs` protocol changes
  - Feature-backed values let product callers provide precise attribution
  while preserving the existing core classifications
  
  ## What Changed
  - Adds `ThreadSource::Feature(String)` for app-owned thread source
  values
  - Represents all app-server v2 thread sources as scalar strings, so a
  feature source is supplied as `"automation"`
  - Persists and emits the feature's plain string label, so `"automation"`
  produces `thread_source="automation"` in analytics
  - Keeps `user`, `subagent`, and `memory_consolidation` as explicit
  core-owned values and regenerates the app-server schemas and TypeScript
  bindings
  
  ## Verification
  - `just write-app-server-schema`
  - `cargo check --workspace`
  - `just test -p codex-protocol
  feature_thread_source_serializes_as_its_app_owned_label`
  - `just test -p codex-app-server-protocol
  thread_sources_round_trip_as_scalar_labels`
  - `cargo test -p codex-analytics
  thread_initialized_event_serializes_expected_shape`
  - `just fmt`

  marksteinbrick-oai · 2026-06-09 12:27:10 -07:00
  a71e040df5
• Avoid no-op backfill state writes (#26420)

  ## Summary
  
  - avoid acquiring SQLite's writer slot when the singleton backfill row
  already exists
  - preserve race-safe repair when the row is missing
  - add regressions for writer contention and missing-row repair
  
  ## Why
  
  State runtime initialization and backfill-state reads previously
  executed
  `INSERT ... ON CONFLICT DO NOTHING` even in the steady state. SQLite
  still
  enters the writer path for that statement, so TUI and app-server startup
  could
  wait behind another writer for up to the configured five-second busy
  timeout.
  
  ## Validation
  
  - `just test -p codex-state` (134 tests passed)
  - `just fix -p codex-state`
  - `just fmt`

  Zanie Blue · 2026-06-09 10:50:13 -05:00
  18ce671fed
• Add multi-agent runtime metadata types (#25720)

  Stack split from #25708. Original PR intentionally left open. This first
  PR adds the multi-agent runtime metadata types and catalog plumbing used
  by the rest of the stack.

  jif-oai · 2026-06-02 12:10:14 +02:00
  3f1fb7ed8b
• store and expose parent_thread_id on Threads (#25113)

  ## Why
  
  This PR
  https://github.com/openai/codex/pull/24161#discussion_r3325692763
  revealed a subagent data modeling issue, where we overloaded
  `forked_from_id` to also mean `parent_thread_id`. That's incorrect since
  guardian and review subagents can be a subagent and NOT fork the main
  thread's history.
  
  The solution here is to explicitly store a new `parent_thread_id` on
  `SessionMeta`, alongside `forked_from_id` which already exists. While
  we're at it, also expose it in the app-server protocol on the `Thread`
  object.
  
  A thread->subagent relationship and a fork of thread history are
  orthogonal concepts.
  
  ## What Changed
  
  - Added top-level `parent_thread_id` persistence on `SessionMeta` and
  runtime/session plumbing through `SessionConfiguredEvent`,
  `CodexSpawnArgs`, `SessionConfiguration`, `ThreadConfigSnapshot`,
  `TurnContext`, and `ModelClient`.
  - Made turn metadata, request headers, analytics, and subagent-start
  events read the separate runtime/top-level parent field instead of
  deriving general parent lineage from `SessionSource` or
  `forked_from_thread_id`.
  - Passed parent lineage separately at delegated subagent, review,
  guardian, agent-job, and multi-agent spawn construction sites;
  copied-history fork lineage remains derived only from `InitialHistory`.
  - Persisted and exposed parent lineage through rollout/thread-store
  projections and app-server v2 `Thread.parentThreadId`.
  - Updated app-server README text and regenerated app-server schema
  fixtures for the additive `parentThreadId` response field.

  Owen Lin · 2026-06-01 04:33:20 +00:00
  cf0911076f
• Add subagent lineage metadata for responsesapi (#24161)

  ## Why
  
  We recently added `forked_from_thread_id` which lets us trace where a
  thread's _context_ comes from, but we also want to understand subagent
  lineage (e.g. which parent thread spawned this subagent? what kind of
  subagent is it?) which is orthogonal.
  
  This PR adds `parent_thread_id` and `subagent_kind` to the
  `x-codex-turn-metadata` header sent to ResponsesAPI.
  
  ## What changed
  
  - Adds `parent_thread_id` and `subagent_kind` to core-owned
  `x-codex-turn-metadata`.
  - Restores persisted `SessionSource` and `ThreadSource` from resumed
  session metadata so cold-resumed subagent threads keep their lineage on
  later Responses API requests.
  - Centralizes parent-thread extraction on `SessionSource` /
  `SubAgentSource` and reuses it in the Responses client, analytics, agent
  control, and state parsing paths.
  - Extends reserved-key, git-enrichment, thread-spawn, and app-server v2
  metadata coverage for the new lineage fields.
  
  ## Verification
  
  - Not run locally per request.
  - Added focused coverage in `core/src/turn_metadata_tests.rs` and
  `app-server/tests/suite/v2/client_metadata.rs`.

  Owen Lin · 2026-05-29 11:28:12 -07:00
  fc9cf62efb
• [codex] Remove redundant SQLite dynamic tool storage (#24819)

  ## Why
  
  Dynamic tools are defined at thread start and already stored in rollout
  `SessionMeta`, which restores resumed and forked sessions. Persisting
  the same tools through SQLite creates a second runtime persistence path
  that is unnecessary prework for the explicit namespace refactor.
  
  ## What changed
  
  - Restore missing thread-start dynamic tools directly from rollout
  history, including when SQLite is enabled.
  - Remove SQLite dynamic-tool reads, writes, backfill, and thread
  metadata patch plumbing.
  - Add SQLite-enabled resume integration coverage that verifies a
  rollout-defined dynamic tool is still sent after resume.
  
  ## Compatibility
  
  The existing `thread_dynamic_tools` table is intentionally not dropped
  even though it's now unused. Older Codex binaries are allowed to open
  databases migrated by newer binaries and still reference this table;
  dropping it would break that mixed-version path. See
  [here](https://github.com/openai/codex/blob/main/codex-rs/state/src/migrations.rs#L10-L11).
  
  ## Verification
  
  - `just test -p codex-state -p codex-rollout -p codex-thread-store`
  - `just test -p codex-core --test all
  resume_restores_dynamic_tools_from_rollout_with_sqlite_enabled`

  sayan-oai · 2026-05-27 17:57:32 -07:00
  304d15cab0
• Bump SQLx to pick up newer bundled SQLite (#24728)

  ## Why
  
  Codex stores thread, log, goal, and memory state in bundled SQLite
  databases through SQLx. We have a suspected SQLite WAL-reset corruption
  issue under heavy concurrent writer load, especially when multiple
  subagents are active. The existing `sqlx 0.8.6` dependency kept us on an
  older `libsqlite3-sys` / bundled SQLite, so this PR moves the SQLx stack
  far enough forward to pick up the newer bundled SQLite library.
  
  ## What changed
  
  - Bump the workspace `sqlx` dependency to `0.9.0`.
  - Use the SQLx 0.9 feature names explicitly: `runtime-tokio`,
  `tls-rustls`, and `sqlite-bundled`.
  - Update `Cargo.lock` so `sqlx-sqlite` resolves through `libsqlite3-sys
  0.37.0`.
  - Refresh `MODULE.bazel.lock` for the dependency changes.
  - Adapt `codex-state` to SQLx 0.9:
  - build dynamic state queries with `QueryBuilder<Sqlite>` instead of
  passing dynamic `String`s to `sqlx::query`;
  - remove the old `QueryBuilder` lifetime parameter from helper
  signatures;
  - preserve SQLx's new `Migrator` fields when constructing runtime
  migrators.
  
  ## Verification
  
  - `just test -p codex-state`
  - `just bazel-lock-check`
  - `cargo check -p codex-state --tests`

  jif-oai · 2026-05-27 18:44:07 +02:00
  379511dcea
• Uprev Rust toolchain pins to 1.95.0 (#24684)

  ## Summary
  - Bump the workspace Rust toolchain from `1.93.0` to `1.95.0` across
  Cargo, Bazel, CI, release workflows, devcontainers, and the Codex
  environment config.
  - Refresh `MODULE.bazel.lock` so the Bazel Rust toolchain artifacts
  match the new version.
  - Leave purpose-specific toolchains unchanged, including the
  `argument-comment-lint` nightly and the upstream `rusty_v8` `1.91.0`
  build pin.
  - Includes fixes for new lints from `just fix` and a few codex-authored
  fixes for lints without a suggestion.

  Adam Perry @ OpenAI · 2026-05-26 20:59:47 -07:00
  cca1e0ba1d
• Move memory state to a dedicated SQLite DB (#24591)

  ## Summary
  
  Generated memory rows and their stage-one/stage-two job state currently
  live in `state_5.sqlite` alongside thread metadata. That makes memory
  cleanup and regeneration share the main state schema even though those
  rows are memory-pipeline data and can be rebuilt independently from the
  durable thread records.
  
  This PR moves the memory-owned tables into a dedicated
  `memories_1.sqlite` runtime database while keeping thread metadata in
  `state_5.sqlite`.
  
  ## Changes
  
  - Adds a separate memories DB runtime, migrator, path helpers, telemetry
  kind, and Bazel compile data for `state/memory_migrations`.
  - Introduces `MemoryStore` behind `StateRuntime::memories()` and moves
  memory table/job operations onto that store.
  - Drops the old memory tables from the state DB and recreates their
  schema in `state/memory_migrations/0001_memories.sql`.
  - Updates memory startup, citation usage tracking, rollout pollution
  handling, `debug clear-memories`, and app-server `memory/reset` to
  operate through the memories DB.
  - Preserves cross-DB behavior by hydrating thread metadata from the
  state DB when selecting visible memory outputs and checking stage-one
  staleness.
  
  ## Verification
  
  - Added/updated `codex-state` tests for deleted-thread memory visibility
  and already-polluted phase-two enqueue behavior.
  - Updated `debug clear-memories`, app-server `memory/reset`, and
  memories startup tests to seed and assert memory rows through
  `memories_1.sqlite`.

  jif-oai · 2026-05-26 20:07:25 +02:00
  aad59a0916
• feat: rename 2 (#23668)

  Just a mechanical renaming

  jif-oai · 2026-05-20 12:11:44 +02:00
  9483b09ea4
• feat: rename 3 (#23669)

  Just a mechanical renaming

  jif-oai · 2026-05-20 12:07:06 +02:00
  66d5edf825
• feat: rename 1 (#23667)

  Just a mechanical renaming

  jif-oai · 2026-05-20 12:05:58 +02:00
  93456320ef
• feat: dedicated goal DB (#23300)

  ## Why
  
  Thread goals are moving toward extension-owned runtime behavior, but
  their persisted state was still stored in the shared state database.
  This makes the goal store harder to isolate and keeps future storage
  splits tied to ad hoc runtime plumbing.
  
  This PR gives goals their own SQLite database while keeping the existing
  `StateRuntime` entry point. The goal is to make this the pattern for
  adding more dedicated runtime databases later.
  
  This also reduce load on existing DB and reduce contention
  
  ## Limitation
  Thread preview from goal is not supported anymore. I'm looking into this
  [EDIT]: solved
  
  ## What changed
  
  - Added a dedicated `goals_1.sqlite` database with its own
  `goals_migrations` directory.
  - Moved `thread_goals` creation into the goals DB migration set.
  - Dropped the old `thread_goals` table from the main state DB with a
  normal state migration. There is intentionally no backfill for existing
  goal rows.
  - Changed `GoalStore` to be backed only by the goals DB pool.
  - Removed the old goal-write side effect that filled empty
  `threads.preview` values from the goal objective.
  - Added shared runtime DB path metadata so startup, telemetry, `codex
  doctor`, and repair handling can include future DBs without bespoke path
  lists.
  - Updated Bazel compile data so the new goals migration directory is
  available to `sqlx::migrate!`.
  
  ## Verification
  
  - `cargo check --tests -p codex-state -p codex-cli -p codex-core -p
  codex-app-server`
  - `just fix -p codex-state`
  - `just fix -p codex-cli`
  - `just fix -p codex-app-server`

  jif-oai · 2026-05-19 11:11:41 +02:00
  ba57aab13a
• goal: pause continuation loops on usage limits and blockers (#23094)

  Addresses #22833, #22245, #23067
  
  ## Why
  `/goal` can keep synthesizing turns even when the next turn cannot make
  meaningful progress. Hard usage exhaustion can replay failing turns, and
  repeated permission or external-resource blockers can keep burning
  tokens while waiting for user or system intervention.
  
  ## What changed
  - Add resumable `blocked` and `usageLimited` goal states. As with
  `paused`, goal continuation stops with these states.
  - Move to `usageLimited` after usage-limit failures.
  - Allow the built-in `update_goal` tool to set `blocked` only under
  explicit repeated-impasse guidance. Updated goal continuation prompt to
  specify that agent should use `blocked` only when it has made at least
  three attempts to get past an impasse.
  
  Most of the files touched by this PR are because of the small app server
  protocol update.
  
  ## Validation
  
  I manually reproduced a number of situations where an agent can run into
  a true impasse and verified that it properly enters `blocked` state. I
  then resumed and verified that it once again entered `blocked` state
  several turns later if the impasse still exists.
  
  I also manually reproduced the usage-limit condition by creating a
  simulated responses API endpoint that returns 429 errors with the
  appropriate error message. Verified that the goal runtime properly moves
  the goal into `usageLimited` state and TUI UI updates appropriately.
  Verified that `/goal resume` resumes (and immediately goes back into
  `ussageLImited` state if appropriate).
  
  
  ## Follow-up PRs
  
  Small changes will be needed to the GUI clients to properly handle the
  two new states.

  Eric Traut · 2026-05-18 11:28:53 -07:00
  0d344aca9b
• chore: isolate thread goal storage behind GoalStore (#23295)

  ## Why
  
  Thread goal persistence is being prepared for a dedicated storage
  boundary. Before that split, goal-specific reads, writes, accounting,
  and cleanup were exposed directly on `StateRuntime`, so core and
  app-server callsites stayed coupled to the full runtime instead of a
  goal-specific store.
  
  This PR introduces that boundary without changing the goal wire API or
  current persistence behavior. Callers now go through
  `StateRuntime::thread_goals()` and the new `GoalStore`, while
  `GoalStore` still uses the existing state DB pool underneath.
  
  ## What changed
  
  - Added `GoalStore` in `state/src/runtime/goals.rs` and exposed it from
  `StateRuntime` via `thread_goals()`.
  - Moved thread-goal reads, writes, status updates, pause, delete, and
  usage accounting onto `GoalStore`.
  - Updated core session goal handling, app-server goal RPCs, resume
  snapshots, and goal tests to use the store boundary.
  - Kept thread deletion responsible for cascading goal cleanup by
  deleting the goal through the store only after a thread row is removed.
  
  ## Testing
  
  - Existing goal persistence, resume, and accounting tests were updated
  to exercise the new `GoalStore` access path.

  jif-oai · 2026-05-18 14:47:05 +02:00
  7ee7fe239f
• Add process-scoped SQLite telemetry (#22154)

  ## Summary
  - add SQLite init, backfill-gate, and fallback telemetry without
  introducing a cross-cutting state-db access wrapper
  - install one process-scoped telemetry sink after OTEL startup and let
  low-level state/rollout paths emit through it directly
  - add process-start metrics for the process owners that initialize
  SQLite
  
  ---------
  
  Co-authored-by: Owen Lin <owen@openai.com>

  jif-oai · 2026-05-11 11:32:40 -07:00
  b401666ca5
• Fix goal update and add /goal edit command in TUI (#21954)

  ## Why
  
  Users have requested the ability to edit a goal's objective after a goal
  has been created. This PR exposes a new `/goal edit` command in the TUI
  to address this request.
  
  In the process of implementing this, I also noticed an existing bug in
  the goal runtime. When a goal's objective is updated through the
  `thread/goal/set` app server API, the goal runtime didn't emit a new
  steering prompt to tell the agent about the new objective. This PR also
  fixes this hole.
  
  ## What Changed
  
  - Adds `/goal edit` in the TUI, opening an edit box prefilled with the
  current goal objective.
  - Keeps active and paused goals in their current state, resets completed
  goals to active, keeps budget-limited goals budget-limited, and
  preserves the existing token budget.
  - Changes the existing `thread/goal/set` behavior so editing an
  objective preserves goal accounting instead of resetting it. The older
  reset-on-new-objective behavior was left over from before
  `thread/goal/clear`; clients that need to reset accounting can now clear
  the existing goal and create a new one.
  - Reuses the existing goal set API path; this does not add or change
  app-server protocol surface area.
  - Adds a dedicated goal runtime steering prompt when an externally
  persisted goal mutation changes the objective, so active turns receive
  the updated objective.
  
  ## Validation
  
  - Make sure `/goal edit` returns an error if no goal currently exists
  - Make sure `/goal edit` displays an edit box that can be optionally
  canceled with no side effects
  - Make sure that an edited goal results in a steer so the agent starts
  pursuing the new objective
  - Make sure the new objective is reflected in the goal if you use
  `/goal` to display the goal summary
  - Make sure that `/goal edit` doesn't reset the token budget, time/token
  accounting on the updated goal

  Eric Traut · 2026-05-11 10:49:19 -07:00
  1e65b3e0af
• Use goal preview metadata for goal-first threads (#21981)

  Fixes #20792
  
  ## Why
  
  `/goal`-first threads are valid resumable threads, but they can be
  missing from `codex resume` and app recents because discovery depends on
  metadata derived from a normal first user message.
  
  PR #21489 attempted to fix this by using the goal objective as
  `first_user_message`. Review feedback pointed out that
  `first_user_message` does more than provide visible text today: it gates
  listing, supplies preview text, and participates in deciding whether a
  later title should surface as a distinct thread name. Reusing it for the
  goal objective could leave a `/goal`-first thread with
  `first_user_message=<goal>` and `title=<later prompt>`, even though the
  goal should only provide the initial visible preview.
  
  This PR follows that feedback by and keeps the `first_user_message` as
  is but introduces a new `preview` field to separate concerns. The
  `preview` field is populated from the first user message or the goal
  objective. We can extend it in the future to include other sources.
  
  ## What Changed
  
  - Added internal thread `preview` metadata in `codex-state`, including a
  SQLite migration that backfills from `first_user_message` and from
  existing `thread_goals` objectives when needed.
  - Treated `ThreadGoalUpdated` as preview-bearing metadata so goal-first
  threads can be listed and searched without mutating
  `first_user_message`.
  - Updated rollout listing, state queries, thread-store conversion, and
  app-server mapping to use preview metadata while continuing to expose
  the existing public `preview` field.
  - Preserved title/name distinctness behavior around literal
  `first_user_message`, so a later normal prompt after `/goal` does not
  surface as a separate name just because the goal supplied the initial
  preview.
  - Preserved compatibility for older/internal metadata writes by deriving
  preview from `first_user_message` when explicit preview metadata is
  absent.
  
  ## Verification
  
  - Manually verified that a thread that starts with a `/goal <objective>`
  shows up in the resume picker.

  Eric Traut · 2026-05-11 10:12:46 -07:00
  f10ddc3f13
• sqlite: no more destructive version bumps (#21847)

  ## Why
  
  We'd like SQLite state to become required and load-bearing. As a first
  step, let's remove the mechanism that allows us to blow away the SQLite
  DB on a version bump, and instead rely on graceful migrations.
  
  The original motivation
  ([PR](https://github.com/openai/codex/pull/10623)) behind this mechanism
  was to care less about backwards compatibility while SQLite was being
  landed, but I'd say it's quite important now to keep the data in it.
  
  ## What changed
  
  - Make `STATE_DB_FILENAME` and `LOGS_DB_FILENAME` the full canonical
  filenames: `state_5.sqlite` and `logs_2.sqlite`.
  - Remove `STATE_DB_VERSION` / `LOGS_DB_VERSION` and the helper that
  constructed filenames from versions.
  - Stop `StateRuntime::init` from scanning for or deleting older SQLite
  DB filenames at startup.
  - Delete the tests that encoded legacy state/logs DB deletion behavior.
  
  ## Verification
  
  - `cargo test -p codex-state`

  Owen Lin · 2026-05-08 17:29:44 -07:00
  95ca276373
• device-key: clean up unused crate (#21487)
  Ruslan Nigmatullin · 2026-05-07 09:01:44 -07:00
  e64a8979b0
• feat: move auto vaccum (#21378)

  The initial vaccum is not needed anymore. We can consider all the DBs
  have been reclaimed by now

  jif-oai · 2026-05-06 19:32:28 +02:00
  ab43db44a2
• [codex-analytics] rework thread_source for thread analytics (#20949)

  ## Summary
  - make `thread_source` an explicit optional thread-level field on
  `thread/start`, `thread/fork`, and returned thread payloads
  - persist `thread_source` in rollout/session metadata so resumed live
  threads retain the original value
  - replace the old best-effort `session_source` -> `thread_source`
  mapping with an explicit caller-supplied analytics classification
  
  ## Why
  Before this change, analytics `thread_source` was populated by a
  best-effort mapping from `session_source`. `session_source` describes
  the runtime/client surface, not the actual thread-level origin, so that
  projection was not accurate enough to distinguish cases such as `user`,
  `subagent`, `memory_consolidation`, and future thread origins reliably.
  
  Making `thread_source` explicit keeps one thread-level analytics field
  while letting callers provide the real classification directly instead
  of recovering it indirectly from `session_source`.
  
  ## Impact
  For new analytics events, `thread_source` now reflects the explicit
  thread-level classification supplied by the caller rather than an
  inferred value derived from `session_source`. Existing protocol fields
  remain optional; callers that omit `threadSource` now produce `null`
  instead of a best-effort inferred value.
  
  ## Validation
  - `just write-app-server-schema`
  - `cargo test -p codex-analytics -p codex-core -p
  codex-app-server-protocol --no-run`
  - `cargo test -p codex-app-server-protocol
  generated_ts_optional_nullable_fields_only_in_params`
  - `cargo test -p codex-analytics
  thread_initialized_event_serializes_expected_shape`
  - `cargo test -p codex-core
  resume_stopped_thread_from_rollout_preserves_thread_source`

  rhan-oai · 2026-05-06 02:12:31 +00:00
  b3d4f1a9f0
• codex: route metadata updates through ThreadStore (#20576)

  - Route `thread/metadata/update` through
  `ThreadStore::update_thread_metadata`.
  - Add `LocalThreadStore` git metadata patch support for set, partial
  update, and clear semantics.
  - Add some unit tests for the new thread store code
  - Remove a lot of dead code/tests!

  Tom · 2026-05-04 20:09:41 -07:00
  707e51bd8b
• Add agent graph store interface (#19229)

  ## Summary
  
  Persisted subagent parent/child topology currently leaks through
  `StateRuntime`'s SQLite-specific thread-spawn helpers. This PR
  introduces a narrow `AgentGraphStore` boundary so follow-up work can
  route graph operations through a local or remote store without coupling
  orchestration code directly to the state DB graph API.
  
  ## Changes
  
  - Adds the new `codex-agent-graph-store` crate.
  - Defines a flat `AgentGraphStore` trait for the v1 graph surface:
  upsert edge, set edge status, list direct children, and list
  descendants.
  - Adds public graph types for `ThreadSpawnEdgeStatus`,
  `AgentGraphStoreError`, and `AgentGraphStoreResult`.
  - Implements `LocalAgentGraphStore` on top of an existing
  `codex_state::StateRuntime`, preserving today's SQLite-backed
  `thread_spawn_edges` behavior.
  - Registers the crate in Cargo/Bazel metadata.
  
  This PR only adds the local contract and implementation; call-site
  migration and the remote gRPC store are left to the follow-up PRs in the
  stack.
  
  ## Testing
  
  - `cargo test -p codex-agent-graph-store`
  
  The new unit tests cover local parity with the existing `StateRuntime`
  graph methods, `Open`/`Closed` filtering, status updates, and stable
  breadth-first descendant ordering.

  Rasmus Rygaard · 2026-04-29 22:48:26 +00:00
  782191547c
• feat: trigger memories from user turns with cooldown (#19970)

  ## Why
  
  Memory startup was tied to thread lifecycle events such as create, load,
  and fork. That can run memory work before a thread receives real user
  input, and it makes startup cost scale with thread management instead of
  actual turns. Moving the trigger to `thread/sendInput` keeps memory
  startup aligned with the first real user turn and lets it use the
  current thread config at turn time.
  
  The idea is to prevent ghost cost due to pre-warm triggered by the app
  
  Turn-based startup can also make global phase-2 consolidation easier to
  request repeatedly, so this adds a success cooldown and tightens the
  default startup scan window.
  
  ## What Changed
  
  - Start `codex_memories_write::start_memories_startup_task` after a
  non-empty `thread/sendInput` turn is submitted, instead of from thread
  create/load/fork paths:
  https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/app-server/src/codex_message_processor.rs#L6477-L6487
  - Expose `CodexThread::config()` so app-server can pass the live config
  into memory startup at turn time.
  - Add a six-hour successful-run cooldown for global phase-2
  consolidation via `SkippedCooldown`:
  https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/state/src/runtime/memories.rs#L963-L966
  - Reduce memory startup defaults to at most 2 rollouts over 10 days:
  https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/config/src/types.rs#L31-L34
  
  ## Verification
  
  Updated the memory runtime coverage around phase-2 reclaim behavior,
  including `phase2_global_lock_respects_success_cooldown`.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-28 16:23:13 +02:00
  a9e5c34083
• Stabilize memory Phase 2 input ordering (#19967)

  ## Why
  
  Phase 2 still needs to choose the most relevant stage-1 memory outputs
  by usage and recency, but exposing that ranking as the rendered
  `raw_memories.md` order creates unnecessary large diff. Usage-count or
  timestamp changes can reshuffle otherwise unchanged memories, making the
  workspace diff noisy and giving the consolidation prompt a misleading
  recency signal from file position.
  This fix will reduce token consumption
  
  ## What Changed
  
  - Keep the existing top-N Phase 2 selection ranking by `usage_count`,
  `last_usage`, `source_updated_at`, and `thread_id`.
  - Return the selected rows in stable ascending `thread_id` order before
  syncing Phase 2 filesystem inputs.
  - Update the memory README, raw memories header, and consolidation
  prompt so they describe the stable order and tell the prompt to use
  metadata and workspace diffs instead of file order as the recency
  signal.
  - Adjust the memory runtime tests to use deterministic thread IDs and
  assert the stable return order separately from the ranked selection
  semantics.
  
  ## Test Coverage
  
  - Existing memory runtime tests in
  `codex-rs/state/src/runtime/memories.rs` now cover the stable returned
  ordering for Phase 2 inputs.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-28 13:32:05 +02:00
  fa127be25f
• nit: one more fix (#19813)

  Fix this:
  https://github.com/openai/codex/pull/19812#discussion_r3147529230

  jif-oai · 2026-04-27 15:32:31 +02:00
  f431ec12c9
• Avoid rewriting Phase 2 selection on clean workspace (#19812)

  ## Why
  
  Phase 2 can now claim the global consolidation lock on startup even when
  the git-backed memory workspace is already clean. The clean-workspace
  path still finalized through the normal Phase 2 success path, which
  clears and re-marks `selected_for_phase2` rows. That made no-op startups
  perform avoidable writes to `stage1_outputs`, creating unnecessary DB
  I/O and contention when no memory files changed.
  
  ## What Changed
  
  - Added a preserving-selection Phase 2 finalizer in `codex-state` that
  only marks the global job row as succeeded.
  - Kept the existing `mark_global_phase2_job_succeeded` behavior for real
  consolidation runs, where the selected Phase 2 snapshot must be
  rewritten.
  - Switched the `succeeded_no_workspace_changes` branch in
  `core/src/memories/phase2.rs` to use the preserving-selection finalizer.
  - Added a regression test that installs a SQLite trigger on
  `stage1_outputs` and verifies the clean finalizer performs zero updates
  there.
  
  ## Testing
  
  - `cargo test -p codex-state`
  - `cargo test -p codex-core memories::tests::phase2`

  jif-oai · 2026-04-27 15:14:16 +02:00
  79b4f691a6
• Allow Phase 2 memory claims after retry exhaustion (#19809)

  ## Why
  
  The Phase 2 memories job row is only the global lock for the git-backed
  memory workspace. Manual memory edits do not enqueue new Stage 1 work,
  so a Phase 2 row with `retry_remaining = 0` could be skipped before the
  worker ever claimed the lock and generated `phase2_workspace_diff.md`.
  
  That left workspace-only changes unconsolidated after repeated failures,
  even when retry backoff had elapsed and the filesystem had real diffable
  work.
  
  ## What Changed
  
  - Allow `try_claim_global_phase2_job` to claim the Phase 2 lock after
  the retry budget is exhausted, while still respecting active `retry_at`
  backoff and fresh running leases.
  - Treat `SkippedRetryUnavailable` for Phase 2 as backoff-only, and
  update the outcome docs to match.
  - Clamp Phase 2 retry bookkeeping at zero when failed attempts are
  recorded.
  
  ## Verification
  
  - Added
  `phase2_global_lock_can_be_claimed_after_retry_budget_is_exhausted` to
  cover the exhausted-budget lock claim path.
  - Ran `cargo test -p codex-state`.

  jif-oai · 2026-04-27 14:58:11 +02:00
  5d314f324c
• feat: use git-backed workspace diffs for memory consolidation (#18982)

  ## Why
  
  This PR make the `morpheus` agent (memory phase 2) use a git diff to
  start it's consolidation. The workflow is the following:
  1. The agent acquire a lock
  2. If `.codex/memories` does not exist or is not a git root, initialize
  everything (and make a first empty commit)
  3. Update `raw_memories.md` and `rollout_summaries/` as before.
  Basically we select max N phase 1 memories based on a given policy
  4. We use git (`gix`) to get a diff between the current state of
  `.codex/memories` and the last commit.
  5. Dump the diff in `phase2_workspace_diff.md`
  6. Spawn `morpheus` and point it to `phase2_workspace_diff.md`
  7. Wait for `morpheus` to be done
  8. Re-create a new `.git` and make one single commit on it. We do this
  because we don't want to preserve history through `.git` and this is
  cheap anyway
  9. We release the lock
  On top of this, we keep the retry policies etc etc
  
  The goals of this new workflow are:
  * Better support of any memory extensions such as `chronicle`
  * Allow the user to manually edit memories and this will be considered
  by the phase 2 agent
   
  As a follow-up we will need to add support for user's edition while
  `morpheus` is running
  
  ## What Changed
  
  - Added memory workspace helpers that prepare the git baseline, compute
  the diff, write `phase2_workspace_diff.md`, and reset the baseline after
  successful consolidation.
  - Updated Phase 2 to sync current inputs into `raw_memories.md` and
  `rollout_summaries/`, prune old extension resources, skip clean
  workspaces, and run the consolidation subagent only when the workspace
  has changes.
  - Tightened Phase 2 job ownership around long-running consolidation with
  heartbeats and an ownership check before resetting the baseline.
  - Simplified the prompt and state APIs so DB watermarks are bookkeeping,
  while workspace dirtiness decides whether consolidation work exists.
  - Updated the memory pipeline README and tests for workspace diffs,
  extension-resource cleanup, pollution-driven forgetting, selection
  ranking, and baseline persistence.
  
  ## Verification
  
  - Added/updated coverage in `core/src/memories/tests.rs`,
  `core/src/memories/workspace_tests.rs`, `state/src/runtime/memories.rs`,
  and `core/tests/suite/memories.rs`.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-27 14:32:44 +02:00
  01ab25dbb5
• Add goal persistence foundation (1 / 5) (#18073)

  Adds the persisted goal foundation for the rest of the stack. This PR is
  intentionally limited to feature flag and state-layer behavior;
  app-server APIs, model tools, runtime continuation, and TUI UX are
  layered in later PRs.
  
  ## Why
  
  Goal mode needs durable thread-level state before clients or model tools
  can safely build on it. The state layer needs to know whether a goal
  exists, what objective it tracks, whether it is active, paused,
  budget-limited, or complete, and how much time/token usage has already
  been accounted.
  
  ## What changed
  
  - Added the `goals` feature flag and generated config schema entry.
  - Added the `thread_goals` state table and Rust model for persisted
  thread goals.
  - Added state runtime APIs for creating, replacing, updating, deleting,
  and accounting goal usage.
  - Added `goal_id`-based stale update protection so an old goal update
  cannot overwrite a replacement.
  - Kept this PR scoped to persistence and state runtime behavior, with no
  app-server, model-facing, continuation, or TUI behavior yet.
  
  ## Verification
  
  - Added state runtime coverage for goal creation, replacement, stale
  update protection, status transitions, token-budget behavior, and usage
  accounting.

  Eric Traut · 2026-04-24 20:51:38 -07:00
  0ee737cea6
• app-server: persist device key bindings in sqlite (#19206)

  ## Why
  
  Device-key providers should only own platform key material. The
  account/client binding used to authorize a signing payload is app-server
  state, and keeping that state in provider-specific metadata makes the
  same check harder to audit and harder to share across platform
  implementations.
  
  Persisting the binding in the shared state database gives the device-key
  crate a platform-neutral source of truth before it asks a provider to
  sign. It also lets app-server move potentially blocking key operations
  off the main message processor path, which matters once providers may
  wait for OS authentication prompts.
  
  ## What changed
  
  - Add a `device_key_bindings` state migration plus `StateRuntime`
  helpers keyed by `key_id`.
  - Add an async `DeviceKeyBindingStore` abstraction to `codex-device-key`
  and use it from `DeviceKeyStore::create` and `DeviceKeyStore::sign`.
  - Keep provider calls behind async store methods and run the synchronous
  provider work through `spawn_blocking`.
  - Wire app-server device-key RPC handling to the SQLite-backed binding
  store and spawn response/error delivery tasks for device-key requests.
  - Run the turn-start tracing test on the existing larger current-thread
  test harness after the larger async surface made the default test stack
  too small locally.
  
  ## Validation
  
  - `cargo test -p codex-device-key`
  - `cargo test -p codex-state device_key`
  - `cargo test -p codex-state`
  - `cargo test -p codex-app-server device_key`
  - `cargo test -p codex-app-server
  message_processor::tracing_tests::turn_start_jsonrpc_span_parents_core_turn_spans`
  - `cargo test -p codex-app-server`
  - `just fix -p codex-device-key`
  - `just fix -p codex-state`
  - `just fix -p codex-app-server`
  - `just bazel-lock-update`
  - `just bazel-lock-check`
  - `git diff --check`

  Ruslan Nigmatullin · 2026-04-23 21:55:56 -07:00
  19badb0be2
• Support multiple cwd filters for thread list (#18502)

  ## Summary
  
  - Teach app-server `thread/list` to accept either a single `cwd` or an
  array of cwd filters, returning threads whose recorded session cwd
  matches any requested path
  - Add `useStateDbOnly` as an explicit opt-in fast path for callers that
  want to answer `thread/list` from SQLite without scanning JSONL rollout
  files
  - Preserve backwards compatibility: by default, `thread/list` still
  scans JSONL rollouts and repairs SQLite state
  - Wire the new cwd array and SQLite-only options through app-server,
  local/remote thread-store, rollout listing, generated TypeScript/schema
  fixtures, proto output, and docs
  
  ## Test Plan
  
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-rollout`
  - `cargo test -p codex-thread-store`
  - `cargo test -p codex-app-server thread_list`
  - `just fmt`
  - `just fix -p codex-app-server-protocol -p codex-rollout -p
  codex-thread-store -p codex-app-server`
  - `cargo build -p codex-cli --bin codex`

  acrognale-oai · 2026-04-22 06:10:09 -04:00
  4f8c58f737
• fix: fully revert agent identity runtime wiring (#18757)

  ## Summary
  
  This PR fully reverts the previously merged Agent Identity runtime
  integration from the old stack:
  https://github.com/openai/codex/pull/17387/changes
  
  It removes the Codex-side task lifecycle wiring, rollout/session
  persistence, feature flag plumbing, lazy `auth.json` mutation,
  background task auth paths, and request callsite changes introduced by
  that stack.
  
  This leaves the repo in a clean pre-AgentIdentity integration state so
  the follow-up PRs can reintroduce the pieces in smaller reviewable
  layers.
  
  ## Stack
  
  1. This PR: full revert
  2. https://github.com/openai/codex/pull/18871: move Agent Identity
  business logic into a crate
  3. https://github.com/openai/codex/pull/18785: add explicit
  AgentIdentity auth mode and startup task allocation
  4. https://github.com/openai/codex/pull/18811: migrate auth callsites
  through AuthProvider
  
  ## Testing
  
  Tests: targeted Rust checks, cargo-shear, Bazel lock check, and CI.

  efrazer-oai · 2026-04-21 14:30:55 -07:00
  be75785504
• [tool search] support namespaced deferred dynamic tools (#18413)

  Deferred dynamic tools need to round-trip a namespace so a tool returned
  by `tool_search` can be called through the same registry key that core
  uses for dispatch.
  
  This change adds namespace support for dynamic tool specs/calls,
  persists it through app-server thread state, and routes dynamic tool
  calls by full `ToolName` while still sending the app the leaf tool name.
  Deferred dynamic tools must provide a namespace; non-deferred dynamic
  tools may remain top-level.
  
  It also introduces `LoadableToolSpec` as the shared
  function-or-namespace Responses shape used by both `tool_search` output
  and dynamic tool registration, so dynamic tools use the same wrapping
  logic in both paths.
  
  Validation:
  - `cargo test -p codex-tools`
  - `cargo test -p codex-core tool_search`
  
  ---------
  
  Co-authored-by: Sayan Sisodiya <sayan@openai.com>

  pash-openai · 2026-04-21 14:13:08 +08:00
  dc1a8f2190