Commit Graph

157 Commits

  • [codex] Support model-defined reasoning efforts (#26444)
    ## Summary
    - accept non-empty model-defined reasoning effort values while
    preserving built-in effort behavior
    - propagate the non-Copy effort type through core, app-server, TUI,
    telemetry, and persistence call sites
    - preserve string wire encoding and expose an open-string schema for
    clients
    - update model selection and shortcut behavior for model-advertised
    effort values
    
    ## Root cause
    `ReasoningEffort` gained a string-backed custom variant, so it could no
    longer implement `Copy` or rely on derived closed-enum serialization.
    Existing consumers still moved effort values from shared references and
    assumed a fixed built-in value set.
    
    ## Validation
    - `just fmt`
    - Local tests and compilation were not run per request; relying on CI.
  • Reduce SQLite contention from OpenTelemetry SDK debug logs (#26396)
    ## Summary
    
    - skip `opentelemetry_sdk` DEBUG and TRACE events before formatting or
    queueing them for the SQLite log sink
    - preserve INFO, WARN, and ERROR events from the SDK, along with TRACE
    events from application targets
    - add a persistence-level regression test for the target and level
    policy
    
    ## Why
    
    OpenTelemetry's batch log processor emits internal
    `BatchLogProcessor.ExportingDueToTimer` meta-events every second per
    Codex process. In measured high-fanout `logs_2.sqlite` databases,
    low-level `opentelemetry_sdk` events accounted for over 30% of retained
    rows (30-60% on the machines of people I asked to check).
    
    Persisting this SDK bookkeeping across many processes adds substantial
    write volume and contention without representing application activity.
    
    ## Validation
    
    - `just test -p codex-state` (132/132 tests passed, plus bench smoke)
    - `just fix -p codex-state`
    - `just fmt`
  • Add multi-agent runtime metadata types (#25720)
    Stack split from #25708. Original PR intentionally left open. This first
    PR adds the multi-agent runtime metadata types and catalog plumbing used
    by the rest of the stack.
  • Preserve renamed thread titles during reconciliation (#25624)
    ## Summary
    - preserve existing explicit SQLite thread titles during rollout
    reconciliation/backfill when the incoming rollout title is only
    first-message-derived
    - keep stale inferred-title repair behavior while avoiding session-index
    scans during startup backfill
    - add a regression test for renamed titles surviving reconcile
    
    ## Testing
    - just fmt
    - just test -p codex-rollout
    - just test -p codex-state
  • store and expose parent_thread_id on Threads (#25113)
    ## Why
    
    This PR
    https://github.com/openai/codex/pull/24161#discussion_r3325692763
    revealed a subagent data modeling issue, where we overloaded
    `forked_from_id` to also mean `parent_thread_id`. That's incorrect since
    guardian and review subagents can be a subagent and NOT fork the main
    thread's history.
    
    The solution here is to explicitly store a new `parent_thread_id` on
    `SessionMeta`, alongside `forked_from_id` which already exists. While
    we're at it, also expose it in the app-server protocol on the `Thread`
    object.
    
    A thread->subagent relationship and a fork of thread history are
    orthogonal concepts.
    
    ## What Changed
    
    - Added top-level `parent_thread_id` persistence on `SessionMeta` and
    runtime/session plumbing through `SessionConfiguredEvent`,
    `CodexSpawnArgs`, `SessionConfiguration`, `ThreadConfigSnapshot`,
    `TurnContext`, and `ModelClient`.
    - Made turn metadata, request headers, analytics, and subagent-start
    events read the separate runtime/top-level parent field instead of
    deriving general parent lineage from `SessionSource` or
    `forked_from_thread_id`.
    - Passed parent lineage separately at delegated subagent, review,
    guardian, agent-job, and multi-agent spawn construction sites;
    copied-history fork lineage remains derived only from `InitialHistory`.
    - Persisted and exposed parent lineage through rollout/thread-store
    projections and app-server v2 `Thread.parentThreadId`.
    - Updated app-server README text and regenerated app-server schema
    fixtures for the additive `parentThreadId` response field.
  • thread-store: store permission profiles (#23165)
    ## Why
    
    `SandboxPolicy` is the legacy compatibility shape, but
    `codex-thread-store` still exposed it through `StoredThread`,
    `ThreadMetadataPatch`, and live metadata sync. That kept thread-store
    consumers tied to the legacy representation and meant richer permission
    profile data could not round-trip through thread metadata or cold
    rollout reconciliation.
    
    ## What Changed
    
    - Replaced thread-store `sandbox_policy` API fields with canonical
    `PermissionProfile` fields.
    - Persist new permission-profile metadata as canonical JSON in the
    existing SQLite metadata slot while continuing to read older legacy
    sandbox policy values.
    - Updated local, in-memory, live metadata sync, and rollout extraction
    paths to propagate `TurnContextItem::permission_profile()`.
    - Re-materialize legacy permission metadata against the final rollout
    cwd when rollout-derived metadata replaces stale SQLite summaries.
    - Updated affected app-server and core test constructors to build
    `PermissionProfile` values directly.
    
    ## Test Plan
    
    - `cargo test -p codex-state`
    - `cargo test -p codex-thread-store`
    - `cargo test -p codex-app-server
    summary_from_stored_thread_preserves_millisecond_precision --lib`
    - `cargo test -p codex-core realtime_context --lib`
  • Add subagent lineage metadata for responsesapi (#24161)
    ## Why
    
    We recently added `forked_from_thread_id` which lets us trace where a
    thread's _context_ comes from, but we also want to understand subagent
    lineage (e.g. which parent thread spawned this subagent? what kind of
    subagent is it?) which is orthogonal.
    
    This PR adds `parent_thread_id` and `subagent_kind` to the
    `x-codex-turn-metadata` header sent to ResponsesAPI.
    
    ## What changed
    
    - Adds `parent_thread_id` and `subagent_kind` to core-owned
    `x-codex-turn-metadata`.
    - Restores persisted `SessionSource` and `ThreadSource` from resumed
    session metadata so cold-resumed subagent threads keep their lineage on
    later Responses API requests.
    - Centralizes parent-thread extraction on `SessionSource` /
    `SubAgentSource` and reuses it in the Responses client, analytics, agent
    control, and state parsing paths.
    - Extends reserved-key, git-enrichment, thread-spawn, and app-server v2
    metadata coverage for the new lineage fields.
    
    ## Verification
    
    - Not run locally per request.
    - Added focused coverage in `core/src/turn_metadata_tests.rs` and
    `app-server/tests/suite/v2/client_metadata.rs`.
  • Surface filesystem permission profiles in prompt context (#23924)
    ## Summary
    Some permission profiles can encode filesystem reads that should remain
    unavailable to the agent. Before this change, the model-visible context
    and automatic approval review prompt summarized the effective
    permissions as a legacy sandbox mode, which can omit permission-profile
    filesystem entries from escalation decisions.
    
    For example, a profile can grant workspace access while denying a
    private subtree across every workspace root:
    
    ```toml
    default_permissions = "restricted-workspace"
    
    [permissions.restricted-workspace.workspace_roots]
    "/Users/alice/project" = true
    "/Users/alice/other-project" = true
    
    [permissions.restricted-workspace.filesystem]
    ":minimal" = "read"
    
    [permissions.restricted-workspace.filesystem.":workspace_roots"]
    "." = "write"
    "private" = "deny"
    "private/**" = "deny"
    ```
    
    The context window now describes the workspace roots and effective
    filesystem side of the `PermissionProfile` directly, with deny entries
    marked as non-escalatable:
    
    ```xml
    <environment_context>
      <cwd>/Users/alice/project</cwd>
      <shell>zsh</shell>
      <filesystem><workspace_roots><root>/Users/alice/project</root><root>/Users/alice/other-project</root></workspace_roots><permission_profile type="managed"><file_system type="restricted"><entry access="read"><special>:minimal</special></entry><entry access="write"><path>/Users/alice/project</path></entry><entry access="write"><path>/Users/alice/other-project</path></entry><entry access="deny" escalatable="false"><path>/Users/alice/project/private</path></entry><entry access="deny" escalatable="false"><path>/Users/alice/other-project/private</path></entry><entry access="deny" escalatable="false"><glob>/Users/alice/project/private/**</glob></entry><entry access="deny" escalatable="false"><glob>/Users/alice/other-project/private/**</glob></entry></file_system></permission_profile></filesystem>
    </environment_context>
    ```
    
    Managed requirements can impose the same kind of deny-read restriction:
    
    ```toml
    [permissions.filesystem]
    deny_read = [
      "/Users/alice/project/private",
      "/Users/alice/project/private/**",
    ]
    ```
    
    The automatic approval review prompt also receives the parent turn's
    denied-read context, so review decisions can account for the active
    permission profile.
    
    ## What Changed
    - Render the effective filesystem profile in `<environment_context>`,
    including profile type, filesystem entries, workspace roots, and
    non-escalatable deny entries.
    - Persist effective `workspace_roots` in `TurnContextItem` so
    resumed/replayed context does not have to bind `:workspace_roots`
    through legacy `cwd` fallback.
    - Add explicit permission instructions that denied reads are policy
    restrictions, not escalation targets.
    - Pass the parent turn's denied-read context into automatic approval
    reviews.
    - Add targeted coverage for prompt rendering, workspace-root
    materialization, replay context, and review prompt context.
    - Keep the prompt-context test expectations platform-aware so the same
    filesystem rendering assertions pass on Unix and Windows paths.
    
    ## Testing
    - `just test -p codex-core
    context::environment_context::tests::serialize_environment_context_with_full_filesystem_profile`
    - `just test -p codex-core
    context::environment_context::tests::turn_context_item_filesystem_uses_workspace_roots_instead_of_cwd`
    - `just test -p codex-core
    context::permissions_instructions::permissions_instructions_tests::builds_permissions_from_profile_with_denied_reads`
    - `just fix -p codex-core`
    
    I also attempted `just test -p codex-core`; the changed prompt-context
    tests passed, but the full local run did not complete cleanly in this
    sandboxed macOS environment due unrelated user-shell `CODEX_SANDBOX*`
    expectations and integration-test timeouts.
  • [codex] Add user input client ids (#24653)
    ## Summary
    
    Adds an optional `clientId` field to app-server v2 `UserInput` and
    carries it through the core `UserInput` model so clients can correlate
    echoed user input items without relying on payload equality.
    
    ## Details
    
    - Adds `client_id: Option<String>` to core `UserInput` variants.
    - Exposes the v2 app-server field as `clientId` on the wire and in
    generated TypeScript.
    - Preserves the id when converting between app-server v2 and core
    protocol types.
    - Regenerates app-server schema fixtures.
    
    ## Validation
    
    - `just fmt`
    - `just write-app-server-schema`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-protocol`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-protocol`
    - `git diff --check`
  • [codex] Remove redundant SQLite dynamic tool storage (#24819)
    ## Why
    
    Dynamic tools are defined at thread start and already stored in rollout
    `SessionMeta`, which restores resumed and forked sessions. Persisting
    the same tools through SQLite creates a second runtime persistence path
    that is unnecessary prework for the explicit namespace refactor.
    
    ## What changed
    
    - Restore missing thread-start dynamic tools directly from rollout
    history, including when SQLite is enabled.
    - Remove SQLite dynamic-tool reads, writes, backfill, and thread
    metadata patch plumbing.
    - Add SQLite-enabled resume integration coverage that verifies a
    rollout-defined dynamic tool is still sent after resume.
    
    ## Compatibility
    
    The existing `thread_dynamic_tools` table is intentionally not dropped
    even though it's now unused. Older Codex binaries are allowed to open
    databases migrated by newer binaries and still reference this table;
    dropping it would break that mixed-version path. See
    [here](https://github.com/openai/codex/blob/main/codex-rs/state/src/migrations.rs#L10-L11).
    
    ## Verification
    
    - `just test -p codex-state -p codex-rollout -p codex-thread-store`
    - `just test -p codex-core --test all
    resume_restores_dynamic_tools_from_rollout_with_sqlite_enabled`
  • Bump SQLx to pick up newer bundled SQLite (#24728)
    ## Why
    
    Codex stores thread, log, goal, and memory state in bundled SQLite
    databases through SQLx. We have a suspected SQLite WAL-reset corruption
    issue under heavy concurrent writer load, especially when multiple
    subagents are active. The existing `sqlx 0.8.6` dependency kept us on an
    older `libsqlite3-sys` / bundled SQLite, so this PR moves the SQLx stack
    far enough forward to pick up the newer bundled SQLite library.
    
    ## What changed
    
    - Bump the workspace `sqlx` dependency to `0.9.0`.
    - Use the SQLx 0.9 feature names explicitly: `runtime-tokio`,
    `tls-rustls`, and `sqlite-bundled`.
    - Update `Cargo.lock` so `sqlx-sqlite` resolves through `libsqlite3-sys
    0.37.0`.
    - Refresh `MODULE.bazel.lock` for the dependency changes.
    - Adapt `codex-state` to SQLx 0.9:
    - build dynamic state queries with `QueryBuilder<Sqlite>` instead of
    passing dynamic `String`s to `sqlx::query`;
    - remove the old `QueryBuilder` lifetime parameter from helper
    signatures;
    - preserve SQLx's new `Migrator` fields when constructing runtime
    migrators.
    
    ## Verification
    
    - `just test -p codex-state`
    - `just bazel-lock-check`
    - `cargo check -p codex-state --tests`
  • Uprev Rust toolchain pins to 1.95.0 (#24684)
    ## Summary
    - Bump the workspace Rust toolchain from `1.93.0` to `1.95.0` across
    Cargo, Bazel, CI, release workflows, devcontainers, and the Codex
    environment config.
    - Refresh `MODULE.bazel.lock` so the Bazel Rust toolchain artifacts
    match the new version.
    - Leave purpose-specific toolchains unchanged, including the
    `argument-comment-lint` nightly and the upstream `rusty_v8` `1.91.0`
    build pin.
    - Includes fixes for new lints from `just fix` and a few codex-authored
    fixes for lints without a suggestion.
  • Move memory state to a dedicated SQLite DB (#24591)
    ## Summary
    
    Generated memory rows and their stage-one/stage-two job state currently
    live in `state_5.sqlite` alongside thread metadata. That makes memory
    cleanup and regeneration share the main state schema even though those
    rows are memory-pipeline data and can be rebuilt independently from the
    durable thread records.
    
    This PR moves the memory-owned tables into a dedicated
    `memories_1.sqlite` runtime database while keeping thread metadata in
    `state_5.sqlite`.
    
    ## Changes
    
    - Adds a separate memories DB runtime, migrator, path helpers, telemetry
    kind, and Bazel compile data for `state/memory_migrations`.
    - Introduces `MemoryStore` behind `StateRuntime::memories()` and moves
    memory table/job operations onto that store.
    - Drops the old memory tables from the state DB and recreates their
    schema in `state/memory_migrations/0001_memories.sql`.
    - Updates memory startup, citation usage tracking, rollout pollution
    handling, `debug clear-memories`, and app-server `memory/reset` to
    operate through the memories DB.
    - Preserves cross-DB behavior by hydrating thread metadata from the
    state DB when selecting visible memory outputs and checking stage-one
    staleness.
    
    ## Verification
    
    - Added/updated `codex-state` tests for deleted-thread memory visibility
    and already-polluted phase-two enqueue behavior.
    - Updated `debug clear-memories`, app-server `memory/reset`, and
    memories startup tests to seed and assert memory rows through
    `memories_1.sqlite`.
  • Add doctor thread inventory audit (#24305)
    ## Why
    
    Users have been reporting missing sessions in the app. The app server
    thread listing is backed by the SQLite state DB, but the durable source
    of truth for a thread still exists on disk as rollout JSONL. When the
    state DB is incomplete, doctor should be able to show the mismatch
    directly instead of leaving users with a generic state health result.
    
    ## What changed
    
    This adds a `threads` doctor check that compares active and archived
    rollout files under `CODEX_HOME` with rows in the SQLite `threads`
    table. The check reports missing rollout rows, stale DB rows, archive
    flag mismatches, duplicate rollout thread IDs, duplicate DB paths,
    source/provider summaries, and bounded samples of affected rollout
    paths.
    
    It also adds a read-only state audit helper in `codex-rs/state` so
    doctor can inspect thread rows without creating, migrating, or repairing
    the database.
    
    ## Sample output
    
    ```text
      ⚠ threads      rollout files are missing from the state DB
          default model provider   openai
          rollout DB active files  3910
          rollout DB archived files 2037
          rollout DB scan errors   0
          rollout DB malformed file names 0
          rollout DB scan cap reached false
          rollout DB rows          5499
          rollout DB active rows   3462
          rollout DB archived rows 2037
          rollout DB missing active rows 448
          rollout DB missing archived rows 0
          rollout DB stale rows    0
          rollout DB archive mismatches 0
          rollout DB duplicate rollout thread ids 0
          rollout DB duplicate DB paths 0
          rollout DB model providers openai=5359, lmstudio=35, mock_provider=33, lite_llm=26, proxy=26, ollama=15, lms=4, local-usage-limit=1
          rollout DB sources       vscode=2587, cli=1494, subagent:thread_spawn=577, subagent:other=502, exec=281, subagent:memory_consolidation=46, subagent:review=9, unknown=3
          rollout DB missing active sample ~/.codex/sessions/2026/0…857e-a923c712e066.jsonl
          rollout DB missing active sample ~/.codex/sessions/2025/0…877a-766dff25c68d.jsonl
          rollout DB missing active sample ~/.codex/sessions/2025/0…a8b1-7bbadc836f6e.jsonl
          rollout DB missing active sample ~/.codex/sessions/2025/0…a218-e6197f3f62f8.jsonl
          rollout DB missing active sample ~/.codex/sessions/2025/0…9011-7e30784f9932.jsonl
    ```
  • fix: main (#23675)
    Fix main due to conflicting merges
    This is only fixing some imports and mechanics
  • feat: rename 2 (#23668)
    Just a mechanical renaming
  • feat: rename 3 (#23669)
    Just a mechanical renaming
  • feat: rename 1 (#23667)
    Just a mechanical renaming
  • feat: dedicated goal DB (#23300)
    ## Why
    
    Thread goals are moving toward extension-owned runtime behavior, but
    their persisted state was still stored in the shared state database.
    This makes the goal store harder to isolate and keeps future storage
    splits tied to ad hoc runtime plumbing.
    
    This PR gives goals their own SQLite database while keeping the existing
    `StateRuntime` entry point. The goal is to make this the pattern for
    adding more dedicated runtime databases later.
    
    This also reduce load on existing DB and reduce contention
    
    ## Limitation
    Thread preview from goal is not supported anymore. I'm looking into this
    [EDIT]: solved
    
    ## What changed
    
    - Added a dedicated `goals_1.sqlite` database with its own
    `goals_migrations` directory.
    - Moved `thread_goals` creation into the goals DB migration set.
    - Dropped the old `thread_goals` table from the main state DB with a
    normal state migration. There is intentionally no backfill for existing
    goal rows.
    - Changed `GoalStore` to be backed only by the goals DB pool.
    - Removed the old goal-write side effect that filled empty
    `threads.preview` values from the goal objective.
    - Added shared runtime DB path metadata so startup, telemetry, `codex
    doctor`, and repair handling can include future DBs without bespoke path
    lists.
    - Updated Bazel compile data so the new goals migration directory is
    available to `sqlx::migrate!`.
    
    ## Verification
    
    - `cargo check --tests -p codex-state -p codex-cli -p codex-core -p
    codex-app-server`
    - `just fix -p codex-state`
    - `just fix -p codex-cli`
    - `just fix -p codex-app-server`
  • [codex] Trim unused TurnContextItem fields (#22709)
    ## Why
    
    `TurnContextItem` is the durable baseline used to reconstruct context
    diffs across resume/fork. Most of the old persisted-only fields on it
    are no longer read, so keeping them in rollout snapshots adds schema
    surface and state that can drift without affecting reconstruction.
    
    `summary` is the exception: older Codex versions require it to
    deserialize `turn_context` records, so keep writing a default
    compatibility value until that schema surface can be removed safely.
    
    ## What changed
    
    - Removed the unused persisted fields from `TurnContextItem`: trace ids,
    user/developer instructions, output schema, and truncation policy.
    - Kept `summary` with a compatibility comment and made
    `TurnContext::to_turn_context_item` write `ReasoningSummary::Auto`
    instead of live turn state.
    - Updated rollout/context reconstruction fixtures for the retained
    summary field.
    
    ## Verification
    
    - `cargo test -p codex-protocol --lib turn_context_item`
    - `cargo test -p codex-rollout
    resume_candidate_matches_cwd_reads_latest_turn_context`
    - `cargo test -p codex-state turn_context`
    - `cargo test -p codex-core --lib
    new_default_turn_captures_current_span_trace_id`
    - `cargo test -p codex-core --lib
    record_initial_history_resumed_turn_context_after_compaction_reestablishes_reference_context_item`
    - `cargo test -p codex-core --test all
    emits_warning_when_resumed_model_differs`
    - `git diff --check`
  • goal: pause continuation loops on usage limits and blockers (#23094)
    Addresses #22833, #22245, #23067
    
    ## Why
    `/goal` can keep synthesizing turns even when the next turn cannot make
    meaningful progress. Hard usage exhaustion can replay failing turns, and
    repeated permission or external-resource blockers can keep burning
    tokens while waiting for user or system intervention.
    
    ## What changed
    - Add resumable `blocked` and `usageLimited` goal states. As with
    `paused`, goal continuation stops with these states.
    - Move to `usageLimited` after usage-limit failures.
    - Allow the built-in `update_goal` tool to set `blocked` only under
    explicit repeated-impasse guidance. Updated goal continuation prompt to
    specify that agent should use `blocked` only when it has made at least
    three attempts to get past an impasse.
    
    Most of the files touched by this PR are because of the small app server
    protocol update.
    
    ## Validation
    
    I manually reproduced a number of situations where an agent can run into
    a true impasse and verified that it properly enters `blocked` state. I
    then resumed and verified that it once again entered `blocked` state
    several turns later if the impasse still exists.
    
    I also manually reproduced the usage-limit condition by creating a
    simulated responses API endpoint that returns 429 errors with the
    appropriate error message. Verified that the goal runtime properly moves
    the goal into `usageLimited` state and TUI UI updates appropriately.
    Verified that `/goal resume` resumes (and immediately goes back into
    `ussageLImited` state if appropriate).
    
    
    ## Follow-up PRs
    
    Small changes will be needed to the GUI clients to properly handle the
    two new states.
  • chore: isolate thread goal storage behind GoalStore (#23295)
    ## Why
    
    Thread goal persistence is being prepared for a dedicated storage
    boundary. Before that split, goal-specific reads, writes, accounting,
    and cleanup were exposed directly on `StateRuntime`, so core and
    app-server callsites stayed coupled to the full runtime instead of a
    goal-specific store.
    
    This PR introduces that boundary without changing the goal wire API or
    current persistence behavior. Callers now go through
    `StateRuntime::thread_goals()` and the new `GoalStore`, while
    `GoalStore` still uses the existing state DB pool underneath.
    
    ## What changed
    
    - Added `GoalStore` in `state/src/runtime/goals.rs` and exposed it from
    `StateRuntime` via `thread_goals()`.
    - Moved thread-goal reads, writes, status updates, pause, delete, and
    usage accounting onto `GoalStore`.
    - Updated core session goal handling, app-server goal RPCs, resume
    snapshots, and goal tests to use the store boundary.
    - Kept thread deletion responsible for cascading goal cleanup by
    deleting the goal through the store only after a thread row is removed.
    
    ## Testing
    
    - Existing goal persistence, resume, and accounting tests were updated
    to exercise the new `GoalStore` access path.
  • Preserve image detail in app-server inputs (#20693)
    ## Summary
    
    - Add optional image detail to user image inputs across core, app-server
    v2, thread history/event mapping, and the generated app-server
    schemas/types.
    - Preserve requested detail when serializing Responses image inputs:
    omitted detail stays on the existing `high` default, while explicit
    `original` keeps local images on the original-resolution path.
    - Support `high`/`original` consistently for tool image outputs,
    including MCP `codex/imageDetail`, code-mode image helpers, and
    `view_image`.
  • feat(cli): add codex doctor diagnostics (#22336)
    ## Why
    
    Users and support need a single command that captures the local Codex
    runtime, configuration, auth, terminal, network, and state shape without
    asking the user to know which diagnostic depth to choose first. `codex
    doctor` now runs the useful checks by default and makes the detailed
    human output the default because the command is usually run when someone
    already needs context.
    
    The command also targets concrete support failure modes we have seen
    while iterating on the design:
    
    - update-target mismatches like #21956, where the installed package
    manager target can differ from the running executable
    - terminal and multiplexer issues that depend on `TERM`, tmux/zellij
    state, color handling, and TTY metadata
    - provider-specific HTTP/WebSocket connectivity, including ChatGPT
    WebSocket handshakes and API-key/provider endpoint reachability
    - local state/log SQLite integrity problems and large rollout
    directories
    - feedback reports that need an attached, redacted diagnostic snapshot
    without asking the user to run a second command
    
    ## What Changed
    
    - Adds `codex doctor` as a grouped CLI diagnostic report with default
    detailed output and `--summary` for the compact view.
    - Adds stable report sections for Environment, Configuration, Updates,
    Connectivity, and Background Server, plus a top Notes block that
    promotes anomalies such as available updates, large rollout directories,
    optional MCP issues, and mixed auth signals.
    - Adds runtime provenance, install consistency, bundled/system search
    readiness, terminal/multiplexer metadata, `config.toml` parse status,
    auth mode details, sandbox details, feature flag summaries, update
    cache/latest-version state, app-server daemon state, SQLite integrity
    checks, rollout statistics, and provider-aware network diagnostics.
    - Adds ChatGPT WebSocket diagnostics that report the negotiated HTTP
    upgrade as `HTTP 101 Switching Protocols` and include timeout, DNS,
    auth, and provider context in detailed output.
    - Makes reachability provider-aware: API-key OpenAI setups check the API
    endpoint, ChatGPT auth checks the ChatGPT path, and custom/AWS/local
    providers check configured HTTP endpoints when available.
    - Adds structured, redacted JSON output where `checks` is keyed by check
    id and `details` is a key/value object for support tooling.
    - Integrates doctor with feedback uploads by attaching a best-effort
    `codex-doctor-report.json` report and adding derived Sentry tags for
    overall status and failing/warning checks.
    - Updates the TUI feedback consent copy so users can see that the doctor
    report is included when logs/diagnostics are uploaded.
    - Updates the CLI bug issue template to ask reporters for `codex doctor
    --json` and render pasted reports as JSON.
    
    ## Example Output
    
    The examples below are sanitized from local smoke runs with `--no-color`
    so the structure is reviewable in plain text.
    
    ### `codex doctor`
    
    ```text
    Codex Doctor v0.0.0 · macos-aarch64
    
    Notes
       ↑ updates      0.130.0 available (current 0.0.0, dismissed 0.128.0)
       ⚠ rollouts     1,526 active files · 2.53 GB on disk
       ⚠ mcp          MCP configuration has optional issues
       ⚠ auth         mixed auth signals: ChatGPT login plus API key env var; HTTP reachability uses API-key mode
    ─────────────────────────────────────────────────────────────
    
    Environment
      ✓ runtime      local debug build
          version                  0.0.0
          install method           other
          commit                   unknown
          executable               ~/code/codex.fcoury-doct…x-rs/target/debug/codex
      ✓ install      consistent
          context                  other
          managed by               npm: no · bun: no · package root —
          PATH entries (2)         ~/.local/share/mise/installs/node/24/bin/codex
                                   ~/.local/share/mise/shims/codex
      ✓ search       ripgrep 15.1.0 (system, `rg`)
      ✓ terminal     Ghostty 1.3.2-main-+b0f827665 · tmux 3.6a · TERM=xterm-256color
          terminal                 Ghostty
          TERM_PROGRAM             ghostty
          terminal version         1.3.2-main-+b0f827665
          TERM                     xterm-256color
          multiplexer              tmux 3.6a
          tmux extended-keys       on
          tmux allow-passthrough   on
          tmux set-clipboard       on
      ✓ state        databases healthy
          CODEX_HOME               ~/.codex (dir)
          state DB                 ~/.codex/state_5.sqlite (file) · integrity ok
          log DB                   ~/.codex/logs_2.sqlite (file) · integrity ok
          active rollouts          1,526 files · 2.53 GB (avg 1.70 MB)
          archived rollouts        8 files · 3.84 MB (avg 491.11 KB)
    
    Configuration
      ✓ config       loaded
          model                    gpt-5.5 · openai
          cwd                      ~/code/codex.fcoury-doctor/codex-rs
          config.toml              ~/.codex/config.toml
          config.toml parse        ok
          MCP servers              1
          feature flags            36 enabled · 7 overridden (full list with --all)
          overrides                code_mode, code_mode_only, memories, chronicle, goals, remote_control, prevent_idle_sleep
      ✓ auth         auth is configured
          auth storage mode        File
          auth file                ~/.codex/auth.json
          auth env vars present    OPENAI_API_KEY
          stored auth mode         chatgpt
          stored API key           false
          stored ChatGPT tokens    true
          stored agent identity    false
      ⚠ mcp          MCP configuration has optional issues — Set the missing MCP env vars or disable the affected server.
          configured servers       1
          disabled servers         0
          streamable_http servers  1
          optional reachability    openaiDeveloperDocs: https://developers.openai.com/mcp (HEAD connect failed; GET connect failed)
      ✓ sandbox      restricted fs + restricted network · approval OnRequest
          approval policy          OnRequest
          filesystem sandbox       restricted
          network sandbox          restricted
    
    Connectivity
      ✓ network      network-related environment looks readable
      ✓ websocket    connected (HTTP 101 Switching Protocols) · 15s timeout
          model provider           openai
          provider name            OpenAI
          wire API                 responses
          supports websockets      true
          connect timeout          15000 ms
          auth mode                chatgpt
          endpoint                 wss://chatgpt.com/backend-api/<redacted>
          DNS                      2 IPv4, 2 IPv6, first IPv6
          handshake result         HTTP 101 Switching Protocols
      ✗ reachability one or more required provider endpoints are unreachable over HTTP — Check proxy, VPN, firewall, DNS, and custom CA configuration.
          reachability mode        API key auth
          openai API               https://api.openai.com/v1 connect failed (required)
    
    Background Server
      ○ app-server   not running (ephemeral mode)
    
    ─────────────────────────────────────────────────────────────
    11 ok · 1 idle · 4 notes · 1 warn · 1 fail failed
    
    --summary compact output           --all expand truncated lists
    --json redacted report
    ```
    
    ### `codex doctor --summary`
    
    ```text
    Codex Doctor v0.0.0 · macos-aarch64
    
    Notes
       ↑ updates      0.130.0 available (current 0.0.0, dismissed 0.128.0)
       ⚠ rollouts     1,526 active files · 2.53 GB on disk
       ⚠ mcp          MCP configuration has optional issues
       ⚠ auth         mixed auth signals: ChatGPT login plus API key env var; HTTP reachability uses API-key mode
    ─────────────────────────────────────────────────────────────
    
    Environment
      ✓ runtime      local debug build
      ✓ install      consistent
      ✓ search       ripgrep 15.1.0 (system, `rg`)
      ✓ terminal     Ghostty 1.3.2-main-+b0f827665 · tmux 3.6a · TERM=xterm-256color
      ✓ state        databases healthy
    
    Configuration
      ✓ config       loaded
      ✓ auth         auth is configured
      ⚠ mcp          MCP configuration has optional issues — Set the missing MCP env vars or disable the affected server.
      ✓ sandbox      restricted fs + restricted network · approval OnRequest
    
    Updates
      ✓ updates      update configuration is locally consistent
    
    Connectivity
      ✓ network      network-related environment looks readable
      ✓ websocket    connected (HTTP 101 Switching Protocols) · 15s timeout
      ✗ reachability one or more required provider endpoints are unreachable over HTTP — Check proxy, VPN, firewall, DNS, and custom CA configuration.
    
    Background Server
      ○ app-server   not running (ephemeral mode)
    
    ─────────────────────────────────────────────────────────────
    11 ok · 1 idle · 4 notes · 1 warn · 1 fail failed
    
    Run codex doctor without --summary for detailed diagnostics.
    --all expand truncated lists       --json redacted report
    ```
    
    ### `codex doctor --json` shape
    
    ```json
    {
      "schema_version": 1,
      "overall_status": "fail",
      "checks": {
        "runtime.provenance": {
          "id": "runtime.provenance",
          "category": "Environment",
          "status": "ok",
          "summary": "local debug build",
          "details": {
            "version": "0.0.0",
            "install method": "other",
            "commit": "unknown"
          }
        },
        "sandbox.helpers": {
          "id": "sandbox.helpers",
          "category": "Configuration",
          "status": "ok",
          "summary": "restricted fs + restricted network · approval OnRequest",
          "details": {
            "approval policy": "OnRequest",
            "filesystem sandbox": "restricted",
            "network sandbox": "restricted"
          }
        }
      }
    }
    ```
    
    ### `/feedback` new sentry attachment
    
    <img width="938" height="798" alt="CleanShot 2026-05-13 at 15 36 14"
    src="https://github.com/user-attachments/assets/715e62e0-d7b4-4fea-a35a-fd5d5d33c4c0"
    />
    
    ### New section in CLI issue template
    
    <img width="1164" height="435" alt="CleanShot 2026-05-13 at 15 47 24"
    src="https://github.com/user-attachments/assets/9081dc25-a28c-4afa-8ba1-e299c2b4031d"
    />
    
    ## How to Test
    
    1. Run `cargo run --bin codex -- doctor --no-color`.
    2. Confirm the detailed report is the default and includes promoted
    Notes, grouped sections, terminal details, state DB integrity, rollout
    stats, provider reachability, WebSocket diagnostics, and app-server
    status.
    3. Run `cargo run --bin codex -- doctor --summary --no-color`.
    4. Confirm the compact view keeps the same sections and summary counts
    but omits detailed key/value rows.
    5. Run `cargo run --bin codex -- doctor --json`.
    6. Confirm the output is redacted JSON, `checks` is an object keyed by
    check id, and each check's `details` is a key/value object.
    7. Preview the CLI bug issue template and confirm the `Codex doctor
    report` field appears after the terminal field, asks for `codex doctor
    --json`, and renders pasted output as JSON.
    8. Start a feedback flow that includes logs.
    9. Confirm the upload consent copy lists `codex-doctor-report.json`
    alongside the log attachments.
    
    Targeted tests:
    
    - `cargo test -p codex-cli doctor`
    - `cargo test -p codex-app-server
    doctor_report_tags_summarize_status_counts`
    - `cargo test -p codex-feedback`
    - `cargo test -p codex-tui feedback_view`
    - `just argument-comment-lint`
    - `git diff --check`
  • Add process-scoped SQLite telemetry (#22154)
    ## Summary
    - add SQLite init, backfill-gate, and fallback telemetry without
    introducing a cross-cutting state-db access wrapper
    - install one process-scoped telemetry sink after OTEL startup and let
    low-level state/rollout paths emit through it directly
    - add process-start metrics for the process owners that initialize
    SQLite
    
    ---------
    
    Co-authored-by: Owen Lin <owen@openai.com>
  • Fix goal update and add /goal edit command in TUI (#21954)
    ## Why
    
    Users have requested the ability to edit a goal's objective after a goal
    has been created. This PR exposes a new `/goal edit` command in the TUI
    to address this request.
    
    In the process of implementing this, I also noticed an existing bug in
    the goal runtime. When a goal's objective is updated through the
    `thread/goal/set` app server API, the goal runtime didn't emit a new
    steering prompt to tell the agent about the new objective. This PR also
    fixes this hole.
    
    ## What Changed
    
    - Adds `/goal edit` in the TUI, opening an edit box prefilled with the
    current goal objective.
    - Keeps active and paused goals in their current state, resets completed
    goals to active, keeps budget-limited goals budget-limited, and
    preserves the existing token budget.
    - Changes the existing `thread/goal/set` behavior so editing an
    objective preserves goal accounting instead of resetting it. The older
    reset-on-new-objective behavior was left over from before
    `thread/goal/clear`; clients that need to reset accounting can now clear
    the existing goal and create a new one.
    - Reuses the existing goal set API path; this does not add or change
    app-server protocol surface area.
    - Adds a dedicated goal runtime steering prompt when an externally
    persisted goal mutation changes the objective, so active turns receive
    the updated objective.
    
    ## Validation
    
    - Make sure `/goal edit` returns an error if no goal currently exists
    - Make sure `/goal edit` displays an edit box that can be optionally
    canceled with no side effects
    - Make sure that an edited goal results in a steer so the agent starts
    pursuing the new objective
    - Make sure the new objective is reflected in the goal if you use
    `/goal` to display the goal summary
    - Make sure that `/goal edit` doesn't reset the token budget, time/token
    accounting on the updated goal
  • Use goal preview metadata for goal-first threads (#21981)
    Fixes #20792
    
    ## Why
    
    `/goal`-first threads are valid resumable threads, but they can be
    missing from `codex resume` and app recents because discovery depends on
    metadata derived from a normal first user message.
    
    PR #21489 attempted to fix this by using the goal objective as
    `first_user_message`. Review feedback pointed out that
    `first_user_message` does more than provide visible text today: it gates
    listing, supplies preview text, and participates in deciding whether a
    later title should surface as a distinct thread name. Reusing it for the
    goal objective could leave a `/goal`-first thread with
    `first_user_message=<goal>` and `title=<later prompt>`, even though the
    goal should only provide the initial visible preview.
    
    This PR follows that feedback by and keeps the `first_user_message` as
    is but introduces a new `preview` field to separate concerns. The
    `preview` field is populated from the first user message or the goal
    objective. We can extend it in the future to include other sources.
    
    ## What Changed
    
    - Added internal thread `preview` metadata in `codex-state`, including a
    SQLite migration that backfills from `first_user_message` and from
    existing `thread_goals` objectives when needed.
    - Treated `ThreadGoalUpdated` as preview-bearing metadata so goal-first
    threads can be listed and searched without mutating
    `first_user_message`.
    - Updated rollout listing, state queries, thread-store conversion, and
    app-server mapping to use preview metadata while continuing to expose
    the existing public `preview` field.
    - Preserved title/name distinctness behavior around literal
    `first_user_message`, so a later normal prompt after `/goal` does not
    surface as a separate name just because the goal supplied the initial
    preview.
    - Preserved compatibility for older/internal metadata writes by deriving
    preview from `first_user_message` when explicit preview metadata is
    absent.
    
    ## Verification
    
    - Manually verified that a thread that starts with a `/goal <objective>`
    shows up in the resume picker.
  • sqlite: no more destructive version bumps (#21847)
    ## Why
    
    We'd like SQLite state to become required and load-bearing. As a first
    step, let's remove the mechanism that allows us to blow away the SQLite
    DB on a version bump, and instead rely on graceful migrations.
    
    The original motivation
    ([PR](https://github.com/openai/codex/pull/10623)) behind this mechanism
    was to care less about backwards compatibility while SQLite was being
    landed, but I'd say it's quite important now to keep the data in it.
    
    ## What changed
    
    - Make `STATE_DB_FILENAME` and `LOGS_DB_FILENAME` the full canonical
    filenames: `state_5.sqlite` and `logs_2.sqlite`.
    - Remove `STATE_DB_VERSION` / `LOGS_DB_VERSION` and the helper that
    constructed filenames from versions.
    - Stop `StateRuntime::init` from scanning for or deleting older SQLite
    DB filenames at startup.
    - Delete the tests that encoded legacy state/logs DB deletion behavior.
    
    ## Verification
    
    - `cargo test -p codex-state`
  • feat: move auto vaccum (#21378)
    The initial vaccum is not needed anymore. We can consider all the DBs
    have been reclaimed by now
  • [codex-analytics] rework thread_source for thread analytics (#20949)
    ## Summary
    - make `thread_source` an explicit optional thread-level field on
    `thread/start`, `thread/fork`, and returned thread payloads
    - persist `thread_source` in rollout/session metadata so resumed live
    threads retain the original value
    - replace the old best-effort `session_source` -> `thread_source`
    mapping with an explicit caller-supplied analytics classification
    
    ## Why
    Before this change, analytics `thread_source` was populated by a
    best-effort mapping from `session_source`. `session_source` describes
    the runtime/client surface, not the actual thread-level origin, so that
    projection was not accurate enough to distinguish cases such as `user`,
    `subagent`, `memory_consolidation`, and future thread origins reliably.
    
    Making `thread_source` explicit keeps one thread-level analytics field
    while letting callers provide the real classification directly instead
    of recovering it indirectly from `session_source`.
    
    ## Impact
    For new analytics events, `thread_source` now reflects the explicit
    thread-level classification supplied by the caller rather than an
    inferred value derived from `session_source`. Existing protocol fields
    remain optional; callers that omit `threadSource` now produce `null`
    instead of a best-effort inferred value.
    
    ## Validation
    - `just write-app-server-schema`
    - `cargo test -p codex-analytics -p codex-core -p
    codex-app-server-protocol --no-run`
    - `cargo test -p codex-app-server-protocol
    generated_ts_optional_nullable_fields_only_in_params`
    - `cargo test -p codex-analytics
    thread_initialized_event_serializes_expected_shape`
    - `cargo test -p codex-core
    resume_stopped_thread_from_rollout_preserves_thread_source`
  • [codex] Move thread naming to app server (#21260)
    ## Why
    
    Thread names are app-server metadata now, backed by the thread store and
    sqlite state database. Keeping a core `SetThreadName` op plus a rollout
    `thread_name_updated` event made rename persistence live in the wrong
    layer and required historical replay support for an event that new
    app-server flows should not write.
    
    ## What changed
    
    - Removed `Op::SetThreadName` and `EventMsg::ThreadNameUpdated` from the
    core protocol and deleted the core handler path that appended rename
    events to rollouts.
    - Updated app-server `thread/name/set` so both loaded and unloaded
    threads write through thread-store metadata and app-server emits
    `thread/name/updated` notifications.
    - Updated local thread-store name metadata updates to write sqlite title
    metadata and the legacy thread-name index without appending rollout
    events.
    - Removed state extraction and rollout handling for the deleted
    thread-name event.
    
    ## Validation
    
    - `cargo test -p codex-app-server thread_name_updated_broadcasts`
    - `cargo test -p codex-app-server
    thread_name_set_is_reflected_in_read_list_and_resume`
    - `cargo test -p codex-thread-store
    update_thread_metadata_sets_name_on_active_rollout_and_indexes_name`
    - `cargo test -p codex-state`
    - `cargo check -p codex-mcp-server -p codex-rollout-trace`
    - `just fix -p codex-app-server -p codex-thread-store -p codex-state -p
    codex-mcp-server -p codex-rollout-trace`
    
    ## Docs
    
    No external documentation update is expected for this internal ownership
    change.
  • codex: route metadata updates through ThreadStore (#20576)
    - Route `thread/metadata/update` through
    `ThreadStore::update_thread_metadata`.
    - Add `LocalThreadStore` git metadata patch support for set, partial
    update, and clear semantics.
    - Add some unit tests for the new thread store code
    - Remove a lot of dead code/tests!
  • Add agent graph store interface (#19229)
    ## Summary
    
    Persisted subagent parent/child topology currently leaks through
    `StateRuntime`'s SQLite-specific thread-spawn helpers. This PR
    introduces a narrow `AgentGraphStore` boundary so follow-up work can
    route graph operations through a local or remote store without coupling
    orchestration code directly to the state DB graph API.
    
    ## Changes
    
    - Adds the new `codex-agent-graph-store` crate.
    - Defines a flat `AgentGraphStore` trait for the v1 graph surface:
    upsert edge, set edge status, list direct children, and list
    descendants.
    - Adds public graph types for `ThreadSpawnEdgeStatus`,
    `AgentGraphStoreError`, and `AgentGraphStoreResult`.
    - Implements `LocalAgentGraphStore` on top of an existing
    `codex_state::StateRuntime`, preserving today's SQLite-backed
    `thread_spawn_edges` behavior.
    - Registers the crate in Cargo/Bazel metadata.
    
    This PR only adds the local contract and implementation; call-site
    migration and the remote gRPC store are left to the follow-up PRs in the
    stack.
    
    ## Testing
    
    - `cargo test -p codex-agent-graph-store`
    
    The new unit tests cover local parity with the existing `StateRuntime`
    graph methods, `Open`/`Closed` filtering, status updates, and stable
    breadth-first descendant ordering.
  • Fix log db batch flush flake (#19959)
    ## Why
    
    The log DB writer batches tracing events before inserting them into
    SQLite, but `tokio::time::interval` produces an immediate first tick.
    That meant the inserter could flush the first accepted log entry before
    `batch_size` was reached, making
    `configured_batch_size_flushes_without_explicit_flush` timing-sensitive
    in CI.
    
    ## What Changed
    
    - Consume the interval's startup tick before entering the inserter loop,
    so interval flushing starts after the configured delay.
    - Remove the test's startup sleep, which was masking the race instead of
    proving the batch-size behavior.
    
    ## Validation
    
    - `cargo test -p codex-state`
    - `cargo test -p codex-state
    configured_batch_size_flushes_without_explicit_flush` passed 3
    consecutive focused runs
    - PR checks passed across `rust-ci`, Bazel, `ci`, `sdk`, `cargo-deny`,
    Codespell, blob-size policy, and CLA
  • feat: trigger memories from user turns with cooldown (#19970)
    ## Why
    
    Memory startup was tied to thread lifecycle events such as create, load,
    and fork. That can run memory work before a thread receives real user
    input, and it makes startup cost scale with thread management instead of
    actual turns. Moving the trigger to `thread/sendInput` keeps memory
    startup aligned with the first real user turn and lets it use the
    current thread config at turn time.
    
    The idea is to prevent ghost cost due to pre-warm triggered by the app
    
    Turn-based startup can also make global phase-2 consolidation easier to
    request repeatedly, so this adds a success cooldown and tightens the
    default startup scan window.
    
    ## What Changed
    
    - Start `codex_memories_write::start_memories_startup_task` after a
    non-empty `thread/sendInput` turn is submitted, instead of from thread
    create/load/fork paths:
    https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/app-server/src/codex_message_processor.rs#L6477-L6487
    - Expose `CodexThread::config()` so app-server can pass the live config
    into memory startup at turn time.
    - Add a six-hour successful-run cooldown for global phase-2
    consolidation via `SkippedCooldown`:
    https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/state/src/runtime/memories.rs#L963-L966
    - Reduce memory startup defaults to at most 2 rollouts over 10 days:
    https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/config/src/types.rs#L31-L34
    
    ## Verification
    
    Updated the memory runtime coverage around phase-2 reclaim behavior,
    including `phase2_global_lock_respects_success_cooldown`.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Stabilize memory Phase 2 input ordering (#19967)
    ## Why
    
    Phase 2 still needs to choose the most relevant stage-1 memory outputs
    by usage and recency, but exposing that ranking as the rendered
    `raw_memories.md` order creates unnecessary large diff. Usage-count or
    timestamp changes can reshuffle otherwise unchanged memories, making the
    workspace diff noisy and giving the consolidation prompt a misleading
    recency signal from file position.
    This fix will reduce token consumption
    
    ## What Changed
    
    - Keep the existing top-N Phase 2 selection ranking by `usage_count`,
    `last_usage`, `source_updated_at`, and `thread_id`.
    - Return the selected rows in stable ascending `thread_id` order before
    syncing Phase 2 filesystem inputs.
    - Update the memory README, raw memories header, and consolidation
    prompt so they describe the stable order and tell the prompt to use
    metadata and workspace diffs instead of file order as the recency
    signal.
    - Adjust the memory runtime tests to use deterministic thread IDs and
    assert the stable return order separately from the ranked selection
    semantics.
    
    ## Test Coverage
    
    - Existing memory runtime tests in
    `codex-rs/state/src/runtime/memories.rs` now cover the stable returned
    ordering for Phase 2 inputs.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Avoid rewriting Phase 2 selection on clean workspace (#19812)
    ## Why
    
    Phase 2 can now claim the global consolidation lock on startup even when
    the git-backed memory workspace is already clean. The clean-workspace
    path still finalized through the normal Phase 2 success path, which
    clears and re-marks `selected_for_phase2` rows. That made no-op startups
    perform avoidable writes to `stage1_outputs`, creating unnecessary DB
    I/O and contention when no memory files changed.
    
    ## What Changed
    
    - Added a preserving-selection Phase 2 finalizer in `codex-state` that
    only marks the global job row as succeeded.
    - Kept the existing `mark_global_phase2_job_succeeded` behavior for real
    consolidation runs, where the selected Phase 2 snapshot must be
    rewritten.
    - Switched the `succeeded_no_workspace_changes` branch in
    `core/src/memories/phase2.rs` to use the preserving-selection finalizer.
    - Added a regression test that installs a SQLite trigger on
    `stage1_outputs` and verifies the clean finalizer performs zero updates
    there.
    
    ## Testing
    
    - `cargo test -p codex-state`
    - `cargo test -p codex-core memories::tests::phase2`
  • Allow Phase 2 memory claims after retry exhaustion (#19809)
    ## Why
    
    The Phase 2 memories job row is only the global lock for the git-backed
    memory workspace. Manual memory edits do not enqueue new Stage 1 work,
    so a Phase 2 row with `retry_remaining = 0` could be skipped before the
    worker ever claimed the lock and generated `phase2_workspace_diff.md`.
    
    That left workspace-only changes unconsolidated after repeated failures,
    even when retry backoff had elapsed and the filesystem had real diffable
    work.
    
    ## What Changed
    
    - Allow `try_claim_global_phase2_job` to claim the Phase 2 lock after
    the retry budget is exhausted, while still respecting active `retry_at`
    backoff and fresh running leases.
    - Treat `SkippedRetryUnavailable` for Phase 2 as backoff-only, and
    update the outcome docs to match.
    - Clamp Phase 2 retry bookkeeping at zero when failed attempts are
    recorded.
    
    ## Verification
    
    - Added
    `phase2_global_lock_can_be_claimed_after_retry_budget_is_exhausted` to
    cover the exhausted-budget lock claim path.
    - Ran `cargo test -p codex-state`.
  • feat: use git-backed workspace diffs for memory consolidation (#18982)
    ## Why
    
    This PR make the `morpheus` agent (memory phase 2) use a git diff to
    start it's consolidation. The workflow is the following:
    1. The agent acquire a lock
    2. If `.codex/memories` does not exist or is not a git root, initialize
    everything (and make a first empty commit)
    3. Update `raw_memories.md` and `rollout_summaries/` as before.
    Basically we select max N phase 1 memories based on a given policy
    4. We use git (`gix`) to get a diff between the current state of
    `.codex/memories` and the last commit.
    5. Dump the diff in `phase2_workspace_diff.md`
    6. Spawn `morpheus` and point it to `phase2_workspace_diff.md`
    7. Wait for `morpheus` to be done
    8. Re-create a new `.git` and make one single commit on it. We do this
    because we don't want to preserve history through `.git` and this is
    cheap anyway
    9. We release the lock
    On top of this, we keep the retry policies etc etc
    
    The goals of this new workflow are:
    * Better support of any memory extensions such as `chronicle`
    * Allow the user to manually edit memories and this will be considered
    by the phase 2 agent
     
    As a follow-up we will need to add support for user's edition while
    `morpheus` is running
    
    ## What Changed
    
    - Added memory workspace helpers that prepare the git baseline, compute
    the diff, write `phase2_workspace_diff.md`, and reset the baseline after
    successful consolidation.
    - Updated Phase 2 to sync current inputs into `raw_memories.md` and
    `rollout_summaries/`, prune old extension resources, skip clean
    workspaces, and run the consolidation subagent only when the workspace
    has changes.
    - Tightened Phase 2 job ownership around long-running consolidation with
    heartbeats and an ownership check before resetting the baseline.
    - Simplified the prompt and state APIs so DB watermarks are bookkeeping,
    while workspace dirtiness decides whether consolidation work exists.
    - Updated the memory pipeline README and tests for workspace diffs,
    extension-resource cleanup, pollution-driven forgetting, selection
    ranking, and baseline persistence.
    
    ## Verification
    
    - Added/updated coverage in `core/src/memories/tests.rs`,
    `core/src/memories/workspace_tests.rs`, `state/src/runtime/memories.rs`,
    and `core/tests/suite/memories.rs`.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add goal persistence foundation (1 / 5) (#18073)
    Adds the persisted goal foundation for the rest of the stack. This PR is
    intentionally limited to feature flag and state-layer behavior;
    app-server APIs, model tools, runtime continuation, and TUI UX are
    layered in later PRs.
    
    ## Why
    
    Goal mode needs durable thread-level state before clients or model tools
    can safely build on it. The state layer needs to know whether a goal
    exists, what objective it tracks, whether it is active, paused,
    budget-limited, or complete, and how much time/token usage has already
    been accounted.
    
    ## What changed
    
    - Added the `goals` feature flag and generated config schema entry.
    - Added the `thread_goals` state table and Rust model for persisted
    thread goals.
    - Added state runtime APIs for creating, replacing, updating, deleting,
    and accounting goal usage.
    - Added `goal_id`-based stale update protection so an old goal update
    cannot overwrite a replacement.
    - Kept this PR scoped to persistence and state runtime behavior, with no
    app-server, model-facing, continuation, or TUI behavior yet.
    
    ## Verification
    
    - Added state runtime coverage for goal creation, replacement, stale
    update protection, status transitions, token-budget behavior, and usage
    accounting.
  • Refactor log DB into LogWriter interface (#19234)
    ## Why
    
    This prepares feedback log capture for a future remote app-server hook
    sink without changing the current local SQLite upload path. The
    important boundary is now intentionally small: a log sink is a tracing
    `Layer` that can also flush entries it has accepted.
    
    That keeps the existing SQLite implementation simple while giving the
    upcoming gRPC sink a place to fit beside it. SQLite and gRPC have
    different worker/write semantics, so this PR avoids introducing a shared
    buffered-sink abstraction and instead lets each `LogWriter` own the
    buffering mechanics it needs.
    
    ## What Changed
    
    - Added `LogSinkQueueConfig` with the existing local defaults: queue
    capacity `512`, batch size `128`, and flush interval `2s`.
    - Added `LogDbLayer::start_with_config(...)` while preserving
    `LogDbLayer::start(...)` and `log_db::start(...)` defaults.
    - Introduced the `LogWriter` trait as the minimal shared interface:
    `tracing_subscriber::Layer` plus `flush()`.
    - Made `LogDbLayer` implement `LogWriter`.
    - Kept tracing event formatting inside `LogDbLayer`; it still creates
    one `LogEntry` per tracing event before queueing it for SQLite.
    - Kept normal event capture best-effort and non-blocking via bounded
    `try_send`.
    
    ## Behavior Notes
    
    This does not change the SQLite schema, retention behavior,
    `/feedback/upload`, or Sentry upload behavior. Normal log events still
    drop when the queue is full; explicit `flush()` still waits for queue
    capacity and receiver processing before returning.
    
    ## Verification
    
    - `cargo test -p codex-state log_db`
    - `cargo test -p codex-state`
    - `just fix -p codex-state`
    
    The added tests cover configured batch-size flushing, configured
    interval flushing, queue-full drops, and the flush barrier semantics.
  • app-server: persist device key bindings in sqlite (#19206)
    ## Why
    
    Device-key providers should only own platform key material. The
    account/client binding used to authorize a signing payload is app-server
    state, and keeping that state in provider-specific metadata makes the
    same check harder to audit and harder to share across platform
    implementations.
    
    Persisting the binding in the shared state database gives the device-key
    crate a platform-neutral source of truth before it asks a provider to
    sign. It also lets app-server move potentially blocking key operations
    off the main message processor path, which matters once providers may
    wait for OS authentication prompts.
    
    ## What changed
    
    - Add a `device_key_bindings` state migration plus `StateRuntime`
    helpers keyed by `key_id`.
    - Add an async `DeviceKeyBindingStore` abstraction to `codex-device-key`
    and use it from `DeviceKeyStore::create` and `DeviceKeyStore::sign`.
    - Keep provider calls behind async store methods and run the synchronous
    provider work through `spawn_blocking`.
    - Wire app-server device-key RPC handling to the SQLite-backed binding
    store and spawn response/error delivery tasks for device-key requests.
    - Run the turn-start tracing test on the existing larger current-thread
    test harness after the larger async surface made the default test stack
    too small locally.
    
    ## Validation
    
    - `cargo test -p codex-device-key`
    - `cargo test -p codex-state device_key`
    - `cargo test -p codex-state`
    - `cargo test -p codex-app-server device_key`
    - `cargo test -p codex-app-server
    message_processor::tracing_tests::turn_start_jsonrpc_span_parents_core_turn_spans`
    - `cargo test -p codex-app-server`
    - `just fix -p codex-device-key`
    - `just fix -p codex-state`
    - `just fix -p codex-app-server`
    - `just bazel-lock-update`
    - `just bazel-lock-check`
    - `git diff --check`
  • rollout: persist turn permission profiles (#18281)
    ## Why
    
    Resume and reconstruction need to preserve the permissions that were
    active for each user turn. If rollouts only keep legacy sandbox fields,
    replay cannot faithfully represent profile-shaped overrides introduced
    earlier in the stack.
    
    ## What changed
    
    This records `permission_profile` on user-turn rollout events,
    reconstructs it through history/state extraction, and updates rollout
    reconstruction and related fixtures to keep the field explicit.
    
    ## Verification
    
    - `cargo test -p codex-core --test all permissions_messages --
    --nocapture`
    - `cargo test -p codex-core --test all request_permissions --
    --nocapture`
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/18281).
    * #18288
    * #18287
    * #18286
    * #18285
    * #18284
    * #18283
    * #18282
    * __->__ #18281
  • Support multiple cwd filters for thread list (#18502)
    ## Summary
    
    - Teach app-server `thread/list` to accept either a single `cwd` or an
    array of cwd filters, returning threads whose recorded session cwd
    matches any requested path
    - Add `useStateDbOnly` as an explicit opt-in fast path for callers that
    want to answer `thread/list` from SQLite without scanning JSONL rollout
    files
    - Preserve backwards compatibility: by default, `thread/list` still
    scans JSONL rollouts and repairs SQLite state
    - Wire the new cwd array and SQLite-only options through app-server,
    local/remote thread-store, rollout listing, generated TypeScript/schema
    fixtures, proto output, and docs
    
    ## Test Plan
    
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-rollout`
    - `cargo test -p codex-thread-store`
    - `cargo test -p codex-app-server thread_list`
    - `just fmt`
    - `just fix -p codex-app-server-protocol -p codex-rollout -p
    codex-thread-store -p codex-app-server`
    - `cargo build -p codex-cli --bin codex`
  • fix: fully revert agent identity runtime wiring (#18757)
    ## Summary
    
    This PR fully reverts the previously merged Agent Identity runtime
    integration from the old stack:
    https://github.com/openai/codex/pull/17387/changes
    
    It removes the Codex-side task lifecycle wiring, rollout/session
    persistence, feature flag plumbing, lazy `auth.json` mutation,
    background task auth paths, and request callsite changes introduced by
    that stack.
    
    This leaves the repo in a clean pre-AgentIdentity integration state so
    the follow-up PRs can reintroduce the pieces in smaller reviewable
    layers.
    
    ## Stack
    
    1. This PR: full revert
    2. https://github.com/openai/codex/pull/18871: move Agent Identity
    business logic into a crate
    3. https://github.com/openai/codex/pull/18785: add explicit
    AgentIdentity auth mode and startup task allocation
    4. https://github.com/openai/codex/pull/18811: migrate auth callsites
    through AuthProvider
    
    ## Testing
    
    Tests: targeted Rust checks, cargo-shear, Bazel lock check, and CI.
  • [tool search] support namespaced deferred dynamic tools (#18413)
    Deferred dynamic tools need to round-trip a namespace so a tool returned
    by `tool_search` can be called through the same registry key that core
    uses for dispatch.
    
    This change adds namespace support for dynamic tool specs/calls,
    persists it through app-server thread state, and routes dynamic tool
    calls by full `ToolName` while still sending the app the leaf tool name.
    Deferred dynamic tools must provide a namespace; non-deferred dynamic
    tools may remain top-level.
    
    It also introduces `LoadableToolSpec` as the shared
    function-or-namespace Responses shape used by both `tool_search` output
    and dynamic tool registration, so dynamic tools use the same wrapping
    logic in both paths.
    
    Validation:
    - `cargo test -p codex-tools`
    - `cargo test -p codex-core tool_search`
    
    ---------
    
    Co-authored-by: Sayan Sisodiya <sayan@openai.com>
  • feat: cascade thread archive (#18112)
    Cascade the thread archive endpoint to all the sub-agents in the agent
    tree
    
    Fix: https://github.com/openai/codex/issues/17867
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>