17 Commits

• feat(app-server): list descendant threads by ancestor (#29591)

  ## Why
  
  `thread/list` can filter direct children with `parentThreadId`, but
  clients cannot request an entire spawned subtree. Discovering every
  descendant requires repeated client-side requests and gives up the
  database's existing filtering and pagination path.
  
  ## What changed
  
  Experimental clients can use `ancestorThreadId` to return strict
  descendants at any depth while `parentThreadId` retains its direct-child
  meaning. The filters are mutually exclusive, the ancestor is excluded,
  and every result preserves its immediate `parentThreadId` so callers can
  reconstruct the tree.
  
  ## How it works
  
  - **Explicit relationship:** Internal list parameters distinguish direct
  children from transitive descendants without changing the meaning of
  `parentThreadId`.
  - **Existing graph:** Persisted parent-child spawn edges remain the
  source of truth, so descendant lookup needs no schema migration or
  ancestry cache.
  - **Indexed traversal:** A recursive SQLite query starts from the
  parent-edge index, walks each generation, and applies thread filters,
  sorting, and cursor pagination in the same database request.
  - **Reconstructable results:** The response stays flat and normally
  ordered while carrying each descendant's immediate parent.
  
  ## Verification
  
  Ran 550 tests across the protocol, state, rollout, and thread-store
  crates, then reran the four focused state, store, and app-server
  descendant-listing tests after the final diff reduction. Scoped Clippy
  and formatting checks passed. Stable and experimental schema generation
  was checked; the stable fixtures remain unchanged while the experimental
  schema includes the new field.

  Brent Traut · 2026-06-24 13:08:14 -07:00
  8057603d0c
• [codex] Track plugin install and import telemetry failures (#28731)

  ## Summary
  - Track plugin install failures through the unified
  `codex_plugin_install_failed` event for local installs, remote install
  preflight failures, bundle failures, and remote catalog/backend
  failures.
  - Send classified `error_type` values in plugin install failure
  analytics instead of raw error strings.
  - Stop sending raw external-agent import errors in analytics while
  preserving raw failure details in app-facing import
  notifications/history.
  - Keep raw plugin/migration diagnostics in `tracing::warn!` logs.
  - Keep remote failure plugin names as the existing local placeholder
  (`unknown`) and remove the extra telemetry plugin-name override.
  - Change `ExternalAgentConfigImportParams.source` from a generated enum
  to `string | null`, with legacy `claudeCode` / `claudeCowork` inputs
  normalized to existing analytics values.
  
  ## Testing

  charlesgong-openai · 2026-06-17 13:16:34 -07:00
  3959ab0ffc
• [codex] exec-server: stream files in chunks (#28354)

  ## Why
  
  `fs/readFile` buffers the entire file in one response, which makes large
  remote reads expensive and prevents callers from applying backpressure.
  We need an opt-in streaming path with bounded block sizes while
  preserving the existing single-call API for small and sandboxed reads.
  
  ## What changed
  
  - Add `ExecServerClient::stream`, returning a named `FileReadStream`
  that implements `futures::Stream` and yields immutable 1 MiB byte
  blocks.
  - Add internal `fs/open`, `fs/readBlock`, and `fs/close` RPCs.
  `fs/readBlock` accepts an explicit offset and length.
  - Keep unsandboxed files open between block reads, cap open handles per
  connection, and clean them up on EOF, error, stream drop, explicit
  close, or connection shutdown.
  - Reject platform-sandboxed streaming opens instead of turning the
  one-shot sandbox helper into a persistent server. Existing `fs/readFile`
  behavior is unchanged.
  
  ## Testing
  
  - `just test -p codex-exec-server`
  - Integration coverage for 1 MiB chunking, exact block-boundary EOF,
  sandbox rejection, and continued reads from the opened file after path
  replacement.
  - Handle-manager coverage for non-sequential offsets, variable block
  lengths, the 128-handle limit, and capacity release after close.

  pakrym-oai · 2026-06-16 09:50:55 -07:00
  a4711b88dd
• [codex] Record external agent import results (#28396)

  ## Summary
  - restore `externalAgentConfig/import/progress` notifications while
  keeping `externalAgentConfig/import/completed` as the must-deliver event
  - persist completed external-agent config imports in state DB by
  `importId`, including concrete success/failure details for config,
  AGENTS.md, skills, plugins, MCP servers, subagents, hooks, commands, and
  sessions
  - add `externalAgentConfig/import/readHistories` so clients can recover
  persisted import results after missing the live completion notification
  - include `errorType` on import failures in protocol
  responses/notifications and persisted DB JSON so future code can
  classify failures without another wire/storage shape change
  
  ## Validation
  - `git diff --check`
  - `just test -p codex-state external_agent_config_imports`
  - `just test -p codex-app-server-protocol`
  - `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-sqlite-read-details
  just test -p codex-app-server
  external_agent_config_import_sends_completion_notification_for_sync_only_import`
  
  Also ran earlier broader checks before publishing:
  - `just test -p codex-state`
  -
  `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-external-agent-test-sqlite
  just test -p codex-app-server external_agent_config`
  - `just test -p codex-external-agent-migration`

  charlesgong-openai · 2026-06-15 23:17:24 -07:00
  314fa3d25b
• [codex] Add external agent import result accounting (#28008)

  ## Why
  
  External-agent imports can complete synchronously or continue in the
  background for plugins/sessions. Clients need a stable import id to
  correlate the immediate response with the eventual completion
  notification, and the completion payload needs enough accounting to show
  which artifact types succeeded or failed without hiding partial
  failures.
  
  ## What Changed
  
  - `externalAgentConfig/import` now returns an `importId`;
  `externalAgentConfig/import/completed` includes the same `importId` plus
  type-level `itemResults`.
  - Completed `itemResults` report `successCount`, `errorCount`,
  `successes`, and `rawErrors` for each migrated item type.
  - Added protocol/schema/TypeScript types for import successes, raw
  errors, and type-level results. No progress notification is included in
  the final PR.
  - `ExternalAgentConfigService::import` now returns an outcome object
  with synchronous item results and pending plugin imports.
  - Plugin import outcomes track succeeded/failed marketplaces, plugin
  ids, and raw errors. Plugin failures can be reported in completed
  accounting while later migration items continue.
  - Non-plugin synchronous import failures still fail the request, so
  invalid config/skills-style failures are not reported as a successful
  import response.
  - Session imports now return item results. Successful imports include
  the source session path and imported thread id; prepare, persist,
  ledger, and source-validation failures become raw errors in completion
  accounting where the import can continue.
  - The request processor generates the `importId`, aggregates synchronous
  results with background plugin/session results, and sends a single
  completed notification when all selected work is done.
  - App-server docs and generated schema fixtures were updated for the new
  response/completed payload shapes.
  
  ## Validation
  
  - `just test -p codex-app-server-protocol`
  - `just test -p codex-app-server-client event_requires_delivery`
  - `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-review-sync-error
  just test -p codex-app-server
  external_agent_config_import_returns_error_for_failed_sync_import`
  - `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-review-external-agent
  just test -p codex-app-server external_agent_config`
  
  Note: local sandbox validation used `CODEX_SQLITE_HOME` because the
  default sqlite state path is read-only in this environment.

  charlesgong-openai · 2026-06-15 13:25:42 -07:00
  fc1fb682a7
• feat(app-server): filter threads by parent (#26662)

  ## Why
  
  Clients that display or coordinate spawned subagents need an
  authoritative snapshot of a thread's immediate spawned children when
  they connect to app-server or recover after missing live events.
  `thread/list` cannot query by parent, so clients must otherwise scan
  unrelated threads or reconstruct relationships from rollout history and
  transient events.
  
  The direct spawn relationship already exists in persisted
  `thread_spawn_edges` state. Review and Guardian threads do not
  participate in that lifecycle and are intentionally outside this
  filter's scope.
  
  ## What changed
  
  This adds an experimental `parentThreadId` filter to `thread/list`.
  Parent-filtered requests return direct spawned children from persisted
  state while preserving the existing response shape, explicit filters,
  sorting, and timestamp-only cursor behavior. The lookup does not read
  rollout transcripts or recursively return descendants.
  
  Supersedes #25112 with the narrower `thread/list` filter approach.
  
  ## How it works
  
  1. An experimental client passes a valid thread ID as `parentThreadId`.
  2. App-server routes the list through the existing thread-store and
  state-database boundaries.
  3. SQLite selects threads whose IDs have a direct persisted spawn edge
  from that parent.
  4. Omitted provider and source filters include all values; explicit
  filters keep ordinary `thread/list` semantics.
  5. Grandchildren, Review threads, and Guardian threads are excluded.
  
  ## Verification
  
  State (144 tests), rollout (69 tests), and focused app-server
  thread-list (31 tests) suites passed. Scoped Clippy checks and
  repository formatting also passed. Coverage includes direct spawned
  children, omitted grandchildren, pagination, malformed IDs, mixed source
  kinds, explicit filters, and operation without rollout files.

  Brent Traut · 2026-06-14 00:14:26 -07:00
  dfd03ea01b
• [codex] Speed up external agent session imports (#26637)

  ## Why
  
  Importing large external-agent session histories currently starts a full
  live Codex thread for every imported session. This initializes unrelated
  runtime systems and repeats expensive transcript, metadata, hashing, and
  ledger work.
  
  On a 50-session, 238 MiB fixture, the existing path took roughly 70
  seconds to complete the import and 77 seconds end to end.
  
  ## What changed
  
  - Persist imported sessions directly through `ThreadStore` instead of
  starting full live threads.
  - Process imports through a bounded five-session pipeline.
  - Parse, extract, and hash each source file in one pass.
  - Move blocking source preparation onto the blocking thread pool.
  - Reuse prepared content hashes and update the import ledger once per
  batch.
  - Avoid metadata readback for newly written rollouts.
  - Preserve imported conversation history and visible thread metadata.
  - Keep the implementation out of `codex-core` and avoid changes to the
  public `ThreadStore` trait.
  
  ## Performance
  
  For the same 50-session, 238 MiB fixture:
  
  | Path | Import completion | End to end |
  | --- | ---: | ---: |
  | Existing import | 69.61s | 76.62s |
  | This change | 5.95s | 6.58s |
  
  All 50 sessions imported successfully with no warnings or contention
  signals.
  
  ## Validation
  
  - `just test -p codex-external-agent-sessions`
  - `just test -p codex-app-server external_agent_config_import`
  - Verified imports do not initialize unrelated required MCP servers.
  - Verified previously imported source versions are skipped and changed
  sources can be imported again.
  - Verified imported rollouts remain readable through thread listing and
  history APIs.

  stefanstokic-oai · 2026-06-08 14:16:32 -04:00
  6d8e12ac42
• fix: rename McpServer to TestAppServer (#25701)

  This PR brought to you via VS Code rather than Codex...
  
  - opened `codex-rs/app-server/tests/common/mcp_process.rs`
  - put the cursor on `McpServer`
  - hit `F2` and renamed the symbol to `TestAppServer`
  - went to the file tree
  - hit enter and renamed `mcp_process.rs` to `test_app_server.rs`
  - ran **Save All Files** from the Command Palette
  - ran `just fmt`
  
  The End
  
  (Admittedly, most of the local variables for `TestAppServer` are still
  named `mcp`, though.)

  Michael Bolin · 2026-06-01 21:49:38 +00:00
  6536841d89
• [codex] Add user input client ids (#24653)

  ## Summary
  
  Adds an optional `clientId` field to app-server v2 `UserInput` and
  carries it through the core `UserInput` model so clients can correlate
  echoed user input items without relying on payload equality.
  
  ## Details
  
  - Adds `client_id: Option<String>` to core `UserInput` variants.
  - Exposes the v2 app-server field as `clientId` on the wire and in
  generated TypeScript.
  - Preserves the id when converting between app-server v2 and core
  protocol types.
  - Regenerates app-server schema fixtures.
  
  ## Validation
  
  - `just fmt`
  - `just write-app-server-schema`
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-protocol`
  - `just fix -p codex-app-server-protocol`
  - `just fix -p codex-protocol`
  - `git diff --check`

  Alexi Christakis · 2026-05-28 14:54:39 -07:00
  e92c952b2e
• feat: Add marketplace source filtering and plugin share context (#21419)

  Adds marketplaceKinds to plugin/list for local, workspace-directory, and
  shared-with-me; omitted params keep default local plus gated global
  behavior, while explicit kinds are exact.
  
  Exposes shareContext on plugin summaries from local share mappings and
  remote workspace/shared responses, including remotePluginId and nullable
  creator metadata.
  
  Adds shared-with-me listing through /ps/plugins/workspace/shared,
  renames the workspace remote namespace to workspace-directory, and keeps
  direct remote read/share/install/update/delete paths gated by plugins
  rather than remote_plugin.

  xl-openai · 2026-05-06 16:12:23 -07:00
  11106016ff
• Send external import completion for sync imports (#20379)
  alexsong-oai · 2026-04-30 13:03:21 -07:00
  9121132c8f
• Import external agent sessions in background (#20284)

  Summary:
  - Return from external agent import before session history import
  finishes
  - Run session import work in the background and emit the existing
  completion notification when it is done
  - Serialize session imports so duplicate requests do not create
  duplicate imported threads
  
  Verification:
  - cargo test -p codex-app-server external_agent_config_
  - cargo test -p codex-external-agent-sessions
  - just fix -p codex-app-server
  - just fix -p codex-external-agent-sessions
  - git diff --check

  stefanstokic-oai · 2026-04-30 00:00:41 +00:00
  c8abcbf925
• Consume ai-title from external sessions and add end marker (#20261)

  ## Summary
  - Support Claude Code `ai-title` / `aiTitle` records when detecting and
  importing external agent sessions.
  - Preserve existing `custom-title` / `customTitle` precedence; only fall
  back to `aiTitle` when no custom title is present.
  - Add coverage for both detection and import title selection, including
  the custom-title-over-ai-title case.
  
  ## Testing
  - `cargo test -p codex-external-agent-sessions`
  - `just fix -p codex-external-agent-sessions`

  alexsong-oai · 2026-04-30 00:00:13 +00:00
  7bcd4626c4
• External agent session support (#19895)

  ## Summary
  
  This extends external agent detection/import beyond config artifacts so
  Codex can detect recent sessions files from the external agent home and
  import them into Codex rollout history.
  
  ## What changed
  
  - Added a focused `external_agent_sessions` module for:
    - session discovery
    - source-record parsing
    - rollout construction
    - import ledger tracking
  - Wired session detection/import into the app-server external agent
  config API.
  - Added compaction handling so large imported sessions can be resumed
  safely before the first follow-up turn.
  
  ## Testing
  
  Added coverage for:
  - recent-session detection
  - custom-title handling
  - recency filtering
  - dedupe and re-detect-after-source-change behavior
  - visible imported turn construction
  - backward-compatible import payload deserialization
  - end-to-end RPC import flow
  - rejection of undetected session paths
  - repeat-import behavior
  - large-session compaction before first follow-up
  
  Ran:
  - `cargo test -p codex-app-server external_agent_config_import_ --test
  all`

  stefanstokic-oai · 2026-04-28 17:42:36 +00:00
  4c68bd728f
• test: harden app-server integration tests (#19683)

  ## Why
  
  Windows Bazel runs in the permissions stack exposed that app-server
  integration tests were launching normal plugin startup warmups in every
  subprocess. Those warmups can call
  `https://chatgpt.com/backend-api/plugins/featured` when a test is not
  specifically exercising plugin startup, which adds slow background work,
  noisy stderr, and dependence on external network state. The relevant
  startup/featured-plugin behavior was introduced across #15042 and
  #15264.
  
  A few app-server tests also had long optional waits or unbounded cleanup
  paths, making failures expensive to diagnose and contributing to slow
  Windows shards. One external-agent config test from #18246 used a
  GitHub-style marketplace source, which was enough to exercise the
  pending remote-import path but also meant the background completion task
  could attempt a real clone.
  
  ## What Changed
  
  - Adds explicit `AppServerRuntimeOptions` / `PluginStartupTasks`
  plumbing and a hidden debug-only
  `--disable-plugin-startup-tasks-for-tests` app-server flag, so
  integration tests can suppress startup plugin warmups without adding a
  production env-var gate.
  - Has the app-server test harness pass that hidden flag by default,
  while opting plugin-startup coverage back in for tests that
  intentionally exercise startup sync and featured-plugin warmup behavior.
  - Lowers normal app-server subprocess logging from `info`/`debug` to
  `warn` to avoid multi-megabyte stderr output in Bazel logs.
  - Prevents the external-agent config test from attempting a real
  marketplace clone by using an invalid non-local source while still
  exercising the pending-import completion path.
  - Bounds optional filesystem/realtime waits and fake WebSocket
  test-server shutdown so failures produce targeted timeouts instead of
  hanging a shard.
  - Fixes the Unix script-resolution test in `rmcp-client` to exercise
  PATH resolution directly and include the actual spawn error in failures.
  
  ## Verification
  
  - `cargo check -p codex-app-server`
  - `cargo clippy -p codex-app-server --tests -- -D warnings`
  - `cargo test -p codex-rmcp-client
  program_resolver::tests::test_unix_executes_script_without_extension`
  - `cargo test -p codex-app-server --test all
  external_agent_config_import_sends_completion_notification_after_pending_plugins_finish
  -- --nocapture`
  - `cargo test -p codex-app-server --test all
  plugin_list_uses_warmed_featured_plugin_ids_cache_on_first_request --
  --nocapture`
  - Windows Local Bazel passed with this test-hardening bundle before it
  was extracted from #19606.
  
  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/19683).
  * #19395
  * #19394
  * #19393
  * #19392
  * #19606
  * __->__ #19683

  Michael Bolin · 2026-04-26 12:43:16 -07:00
  ac2bffa443
• feat: Add remote plugin fields to plugin API (#17277)

  ## Summary
  Update the plugin API for the new remote plugin model.
  
  The mental model is no longer “keep local plugin state in sync with
  remote.” Instead, local and remote plugins are becoming separate
  sources. Remote catalog entries can be shown directly from the remote
  API before installation; after installation they are still downloaded
  into the local cache for execution, but remote installed state will come
  from the API and be held in memory rather than being read from config.
  
  • ## API changes
  - Remove `forceRemoteSync` from `plugin/list`, `plugin/install`, and
  `plugin/uninstall`.
    - Remove `remoteSyncError` from `plugin/list`.
    - Add remote-capable metadata to `plugin/list` / `plugin/read`:
      - nullable `marketplaces[].path`
      - `source: { type: "remote", downloadUrl }`
      - URL asset fields alongside local path fields:
    `composerIconUrl`, `logoUrl`, `screenshotUrls`
    - Make `plugin/read` and `plugin/install` source-compatible:
      - `marketplacePath?: AbsolutePathBuf | null`
      - `remoteMarketplaceName?: string | null`
      - exactly one source is required at runtime

  xl-openai · 2026-04-17 16:47:58 -07:00
  26d9894a27
• Sync local plugin imports, async remote imports, refresh caches after… (#18246)

  … import
  
  ## Why
  
  `externalAgentConfig/import` used to spawn plugin imports in the
  background and return immediately. That meant local marketplace imports
  could still be in flight when the caller refreshed plugin state, so
  newly imported plugins would not show up right away.
  
  This change makes local marketplace imports complete before the RPC
  returns, while keeping remote marketplace imports asynchronous so we do
  not block on remote fetches.
  
  ## What changed
  
  - split plugin migration details into local and remote marketplace
  imports based on the external config source
  - import local marketplaces synchronously during
  `externalAgentConfig/import`
  - return pending remote plugin imports to the app-server so it can
  finish them in the background
  - clear the plugin and skills caches before responding to plugin
  imports, and again after background remote imports complete, so the next
  `plugin/list` reloads fresh state
  - keep marketplace source parsing encapsulated behind
  `is_local_marketplace_source(...)` instead of re-exporting the internal
  enum
  - add core and app-server coverage for the synchronous local import path
  and the pending remote import path
  
  ## Verification
  
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-core` (currently fails an existing unrelated
  test:
  `config_loader::tests::cli_override_can_update_project_local_mcp_server_when_project_is_trusted`)
  - `cargo test` (currently fails existing `codex-app-server` integration
  tests in MCP/skills/thread-start areas, plus the unrelated `codex-core`
  failure above)

  alexsong-oai · 2026-04-17 09:34:55 +00:00
  20b4b80426