mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
ee9e0f6387b91cd14008ea59a8e4315b2a2fdacb
175 Commits
-
Pin MCP runtimes to model steps (#30101)
## Why An MCP refresh can replace the session's current manager while a model step is still running. The step must execute calls through the same manager whose tools it advertised. ## Boundary ```text current session MCP runtime | | capture once for this model step v StepContext.mcp - exact MCP config - exact connection manager - exact runtime environment context ``` ```rust pub struct McpRuntimeSnapshot { config: Arc<McpConfig>, manager: Arc<McpConnectionManager>, runtime_context: McpRuntimeContext, } ``` ## Example ```text step A captures runtime A and advertises A's tools refresh publishes runtime B step A tool call -> runtime A next step -> runtime B ``` Capturing the snapshot is only an `Arc` clone. It does not restart MCPs or make an RPC. ## What changes - Captures one MCP runtime in `StepContext`. - Uses it for tool planning, tool calls, resources, approvals, connector attribution, and elicitation. - Publishes replacement runtimes atomically. - Lets an old runtime live only while an in-flight step or request still holds its `Arc`. Most of this diff is mechanical routing from the session-global manager to `step_context.mcp`; it does not introduce selected-plugin discovery yet. ## What does not change - No plugin or extension migration. - No new MCP cache policy. - No environment file watching. - No client sharing between separate managers. ## Stack 1. Extension-owned World State sections. 2. Project executor skills through World State. 3. **This PR:** pin one MCP runtime to each model step. 4. Project selected MCP/app/connector metadata by environment availability. 5. One end-to-end integration scenario.jif ·
2026-06-26 00:53:07 +01:00 -
Persist selected capability roots and resolve availability per model step (#29856)
## Why `selectedCapabilityRoots` is durable thread intent: “use this capability root from environment `worker`.” The important product assumption is: > One environment ID always names the same logical executor and stable contents. `worker` does not silently change from executor A to an unrelated executor B. The process-local connection handle for `worker` can still be replaced while Codex is running, though, for example when `environment/add` registers a fresh handle for the same logical environment. The thread should persist only the stable selection. Each model step should pair that selection with the exact ready handle captured for that step. ## The boundary ```text persisted thread intent plugin@1 -> environment "worker" | | capture the current step v model-step view unavailable, or plugin@1 + worker's exact captured ready handle ``` The environment ID is the stable identity and cache key. The `Arc<Environment>` is only a process-local handle retained so consumers of one model step use the same captured environment. It is never persisted and it does not imply different environment contents. ## What changes ### Persist the stable selection Selected roots are written into `SessionMeta` and restored with the thread. Forked subagents inherit the same selections, including bounded-history forks. Only stable data is persisted: root ID, environment ID, and root path. ### Capture readiness together with the exact handle The environment snapshot records: ```rust environment_id -> Some(Arc<Environment>) // ready in this step environment_id -> None // still starting in this step ``` This prevents readiness and execution from coming from different registry snapshots. For example: ```text step snapshot: worker -> handle A, ready environment/add: worker -> fresh handle B for the same logical environment current step: plugin@1 still uses captured handle A ``` Without carrying handle A in the snapshot, the resolver could combine “A was ready” with handle B and treat B as ready before it had finished starting. This does not change cache invalidation. Stable capability metadata remains identified by environment ID and capability root. Replacing a process-local handle under the same stable environment ID does not invalidate or rediscover that metadata. ### Resolve availability per model step - A ready captured environment produces resolved roots using its captured handle. - A starting, missing, or failed environment is omitted from that step. - A selected lazy environment that is outside the turn's captured environment set is asked to start, and a later step can observe it as ready. - No capability files are scanned here. Transient transport disconnects remain the remote client's reconnect concern. This PR models initial attachment/readiness; it does not add live socket-connectivity state. ## Example ```text thread selection: plugin@1 -> environment "worker" step 1: worker is starting -> plugin@1 unavailable step 2: worker is ready -> plugin@1 resolves through worker's captured handle step 3: fresh local handle -> current step remains pinned; a later step captures its own view ``` Temporary unavailability does not discard the durable selection. Later PRs can retain stable metadata caches while projecting only currently available capabilities into model-visible World State. ## Compatibility The app-server request shape does not change. Older rollouts without `selected_capability_roots` deserialize to an empty list. ## Stack 1. **This PR:** persist stable selected roots and resolve them through an exact model-step handle. 2. #29960: cache stable skill metadata and project available skills into World State. 3. #29946: cache stable plugin declarations and manage the separate live MCP runtime.jif ·
2026-06-25 17:49:43 +00:00 -
core: make AGENTS.md react to environment changes (#29810)
## Why With deferred executors, a turn can begin before a remote environment attaches. AGENTS.md discovery previously ran only during session setup, so instructions from a later environment never reached the model or the session instruction sources. WorldState persistence has now landed, so this uses the durable model-visible baseline directly instead of carrying a temporary resume/fork compatibility path. ## What - Add an `AgentsMdManager` in `SessionServices` to own host instructions, loaded state, and refresh caching. - When `DeferredExecutor` is enabled, refresh AGENTS.md when attached environment selections change and freeze the result in the corresponding `StepContext`. - Represent AGENTS.md as a persisted WorldState section for every session, with bounded initial, replacement, and removal updates. - Remove duplicate AGENTS.md state and rendering from `SessionConfiguration` and `TurnContext`. - Build initial context, per-request updates, and compaction context from the same step-scoped value. - On resume and fork, compare current instructions with the restored WorldState baseline and inject a replacement exactly once when they differ. Builds on #29833, #29835, and #29837. ## Tests - Covers a remote environment becoming ready mid-turn, with AGENTS.md appearing on the next request exactly once and updating canonical instruction sources. - Covers full, unchanged, replaced, and removed AGENTS.md WorldState rendering. - Covers changed instructions across cold resume and fork without duplicate reinjection. - Covers remote-v2 compaction retaining creation-time instructions in the live session and cold resume appending one replacement when the source changed. - Ran focused `codex-core` AGENTS.md, WorldState, and context-update test suites.
sayan-oai ·
2026-06-24 22:57:42 -07:00 -
core: reset context for token budget compaction (#29743)
## Why When `Feature::TokenBudget` is enabled, compaction should behave like `new_context`: start a fresh context window with the standard injected context, without asking the server to summarize old history and without carrying prior user or assistant messages into the next model request. This is still a compaction operation from the client lifecycle perspective. Manual `/compact` and auto-compaction should keep the same observable side effects that clients and hooks expect, including compact hooks and `TurnItem::ContextCompaction`. ## What changed - Added `compact_token_budget` to run token-budget manual and inline auto-compaction through a shared compaction lifecycle. - Split pending `new_context` requests from forced context-window startup: `take_new_context_window_request()` consumes pending requests, and `start_new_context_window()` installs a fresh context window. - Routed token-budget manual `/compact` and inline auto-compaction to install a fresh context window locally instead of calling server/local summarization. - Preserved compact lifecycle side effects for token-budget compaction by running pre/post compact hooks and emitting `ContextCompaction` item start/completion events. - Updated token-budget tests to assert fresh window IDs, absence of server-side compaction calls, dropped prior transcript messages/tool output after reset, and compact hook/item lifecycle behavior. ## Testing - `just test -p codex-core token_budget_context_uses_new_window_after_compaction` - `just test -p codex-core token_budget_compaction_runs_compact_hooks` - `just test -p codex-core token_budget_mid_turn_auto_compaction_resets_before_active_follow_up` --------- Co-authored-by: pakrym-oai <pakrym@openai.com>
Michael Bolin ·
2026-06-23 16:59:04 -07:00 -
core: persist initial context window metadata (#29519)
## Why PR #29494 made context-window IDs visible to the model by wrapping the token-budget window payload in `<context_window>`, but rollout JSONL consumers still could not see the initial window identity by tailing the session file. Compacted rollout items carry window IDs only after compaction has happened, so a session with no compaction had no durable JSONL record for window 0. This change gives tailing consumers a stable initial-window record at session creation time. ## What Changed - Added `session_meta.context_window.window_id` for the initial context-window identity. - `CreateThreadParams` now requires `initial_window_id: String`, so thread-store callers cannot accidentally create new threads without window-0 metadata. - Live thread creation derives the persisted initial window ID from the same `AutoCompactWindowIds` used to initialize `SessionState`, keeping runtime state and JSONL metadata aligned. - Rollout reconstruction uses `session_meta.context_window.window_id` as the initial-window fallback and derives `window_number = 0`, `first_window_id = window_id`, and `previous_window_id = None` internally. - Fork reconstruction intentionally uses the same rollout reconstruction path; consumers that need to distinguish copied initial-window metadata can use the rollout `thread_id`. - Legacy compactions without `window_number` still use compaction-count fallback accounting instead of being reset to window 0 by the initial-window fallback. - Compacted rollout metadata still takes precedence once compaction records exist, preserving the richer chain fields there. ## JSONL Shape Real rollout JSONL is one object per line. This example is expanded for readability, but shows the new initial `session_meta.context_window` record followed by the existing compacted rollout item shape that also carries window IDs: ```jsonl { "timestamp": "2026-06-22T12:00:00.000Z", "type": "session_meta", "payload": { "session_id": "<THREAD_ID>", "id": "<THREAD_ID>", "timestamp": "2026-06-22T12:00:00.000Z", "cwd": "/repo", "originator": "codex", "cli_version": "0.0.0", "source": "cli", "model_provider": "<MODEL_PROVIDER>", "context_window": { "window_id": "<INITIAL_WINDOW_ID>" } } } ... { "timestamp": "2026-06-22T12:34:56.000Z", "type": "compacted", "payload": { "message": "<COMPACTION_SUMMARY>", "replacement_history": [ "..." ], "window_number": 1, "first_window_id": "<INITIAL_WINDOW_ID>", "previous_window_id": "<INITIAL_WINDOW_ID>", "window_id": "<NEXT_WINDOW_ID>" } } ``` The nested `context_window` object is intentional: it gives rollout consumers a stable namespace for context-window metadata while only writing the non-derivable initial `window_id`. For the initial window, `window_number`, `first_window_id`, and `previous_window_id` are derived internally instead of being written to the rollout. ## Verification - `just test -p codex-protocol` - `just test -p codex-rollout recorder_materializes_on_flush_with_pending_items` - `just test -p codex-core reconstruct_history` - `just test -p codex-core record_initial_history_reconstructs_forked_transcript` - `just test -p codex-thread-store` - `just test -p codex-state` - `just test -p codex-app-server thread_read_returns_summary_without_turns` - `just test -p codex-rollout persistence_metrics`
Michael Bolin ·
2026-06-23 21:50:50 +00:00 -
[codex] add configurable token budget compaction reminder (#29255)
## Why The token-budget feature reports coarse remaining-context milestones, but it does not give the model a configurable wrap-up prompt before automatic compaction. A strict threshold-crossing check can also miss resumed or reconfigured windows that are already inside the threshold. ## What changed - Add structured `[features.token_budget]` configuration for an absolute `reminder_threshold_tokens` and bounded `reminder_message_template`; `{n_remaining}` is expanded when the reminder is delivered. - Compute remaining tokens against the next effective auto-compaction boundary, including scoped `body_after_prefix` accounting and the full context-window limit. - Make reminder delivery level-triggered before and after sampling, with one-shot state owned by `AutoCompactWindow` and re-armed on compaction, `new_context`, restore, or history replacement. - Leave the existing initial full-window token-budget context, 25/50/75% notices, and token-budget tools unchanged. - Persist the resolved feature configuration in the session config lock and regenerate the config schema. ## Validation - `just test -p codex-core token_budget` - `just test -p codex-core token_budget_reminder_emits_after_crossing_compaction_threshold` - `just test -p codex-core auto_compact_window` - `just test -p codex-core lock_contains_prompts_and_materializes_features` - `just test -p codex-features` - `just test -p codex-config`pakrym-oai ·
2026-06-20 19:13:42 -07:00 -
core: add context window lineage IDs (#29256)
## Why The rendered `<token_budget>` fragment identifies the thread and current context window, but it does not expose enough lineage to identify the first window in the thread or the immediately preceding window. Those IDs also need to remain stable across compaction, resume, and rollback. ## What changed - Track first, previous, and current UUIDv7 context-window IDs in auto-compaction state. - Render `thread_id`, `first_window_id`, `previous_window_id`, and the current window ID in the full `<token_budget>` fragment. - Persist the first and previous window IDs in compacted rollout checkpoints and restore them during rollout reconstruction. - Preserve compatibility with older compacted records that do not contain the new optional fields. - Update focused state, rendering, reconstruction, rollback, and serialization coverage. ## Validation - `just test -p codex-core token_budget` - `just test -p codex-protocol compacted_item::tests` - `just test -p codex-core tracks_prefill_and_window_boundaries` - `just test -p codex-core reconstruct_history_uses_replacement_history_verbatim` - `just test -p codex-core thread_rollback_restores_cleared_reference_context_item_after_compaction`
pakrym-oai ·
2026-06-20 13:15:49 -07:00 -
core: add UUIDv7 context window IDs (#28953)
## Why The token-budget context currently identifies a context window by its thread-local sequence number. A UUIDv7 gives the model a stable opaque identity that remains fixed for a window and rotates when compaction or `new_context` starts the next one. ## What changed - Preserve the existing monotonic value as `window_number` and add a UUIDv7 `window_id` to `CompactedItem`. - Generate and rotate the UUID with auto-compaction window state, persist it alongside the number, and reconstruct it on resume and rollback. - Accept legacy compacted rollout records where the numeric `window_id` represented the window number. - Use the UUID only in token-budget context; existing request headers and metadata continue using `thread_id:window_number`. ## Testing - `just test -p codex-protocol compacted_item::tests` - `just test -p codex-core token_budget`
pakrym-oai ·
2026-06-18 17:00:49 -07:00 -
current time reminders impl for system clock (varlatency 2/n) (#28824)
Stacked on #28822. ## Summary - add a host-injectable current-time provider with a built-in system implementation - record UTC developer reminders in history immediately before due model requests - keep cadence state per session and force a refresh after compaction This does NOT include the app server client <-> server clock logic. This PR is only for the reminder message & system clock that will be used in prod. ## Testing - `just test -p codex-core varlatency_` - `just clippy -p codex-core -p codex-app-server -p codex-mcp-server -p codex-thread-manager-sample` - `just fmt`
rka-oai ·
2026-06-18 19:18:42 +00:00 -
Support
openai/formextended form elicitations (#27500)# Summary Allow App Server clients to opt into `openai/form` MCP elicitations.
Gabriel Peal ·
2026-06-18 11:54:49 -07:00 -
Replace SkillsManager with SkillsService (#28705)
## Why Host skill discovery was still exposed as a manager even though it is a process-owned service shared by sessions, the app-server catalog, and file-watcher invalidation. The skills extension also consumed an ad hoc loaded-skills wrapper instead of a named immutable snapshot. ## What changed - replace `SkillsManager` with concrete `SkillsService` - make the service cache and return immutable `HostSkillsSnapshot` values - migrate the skills extension host provider to the snapshot boundary - migrate app-server catalog, watcher, and invalidation paths to the service This keeps the service limited to host discovery, caching, roots, and invalidation. Catalog rendering and invocation remain extension responsibilities for the next stacked change.
jif ·
2026-06-17 17:01:06 +02:00 -
[codex] Bind shell snapshots to retained thread environments (#28421)
## Why Shell snapshots are currently session-scoped even though shell and cwd are properties of a selected turn environment. That makes snapshot refresh depend on separate session-cwd plumbing, prevents retained environments from retaining their snapshot work, and can make snapshot construction use a different shell than command execution. This follows #27955 by making the retained thread-environment service own environment snapshot lifecycles. Session configuration remains the requested selection state, while `ThreadEnvironments` remains the source of successfully resolved environments. ## What changed - Configure the shell-snapshot builder before initial environment resolution. - Start each local environment snapshot task when its `TurnEnvironment` is built and retain that shared task while environment ID and cwd still match. - Inherit retained environment snapshots into spawned child threads. - Carry the selected `TurnEnvironment` through shell runtimes so snapshot construction and command execution use the same environment-specific shell and cwd. - Load project instructions and warm plugins/skills after initial environment resolution. - Continue decoding invalid UTF-8 instruction files lossily without emitting a startup warning. - Keep requested selections in `SessionConfiguration`; failed or duplicate resolutions only affect the resolved environment snapshot. ## Validation - `cargo check -p codex-core --tests` - `just test -p codex-home instructions` (6 passed) - Focused environment, instruction, shell-snapshot, and user-shell tests (84 passed) - Focused shell-snapshot, user-shell, and unified-exec tests (126 passed; two event-timing tests passed on retry)
pakrym-oai ·
2026-06-15 20:10:53 -07:00 -
[codex] retain resolved environments across turns (#27955)
## Why Selected execution environments are thread-scoped resources, but startup and turn construction repeatedly resolved their IDs and working directories. That discarded existing environment handles and shell metadata even when a selection had not changed. Session configuration updates also need to affect future turns without changing the resolved environment set already captured by a running turn. ## What changed - Create a `ThreadEnvironments` service inside `Codex` from the spawned `EnvironmentManager` and raw environment selections, then store it on `SessionServices`. - Split service construction from `update_selections`, allowing session configuration updates to mutate the resolved set in place. - Retain an existing `TurnEnvironment` when its environment ID and working directory match; resolve only added or changed selections and remove selections that are no longer present. - Normalize duplicate IDs by keeping the first selection and skip individual selections that fail to resolve instead of rejecting the entire update. - Give each `TurnContext` a cloned `TurnEnvironmentSnapshot`, so later session configuration updates affect future turns without rewriting an active turn. - Reuse the service-owned environment manager and resolved snapshot for startup work, MCP initialization, and child-thread spawning instead of flowing resolved environments through spawn arguments. ## Test plan - `cargo check -p codex-core --tests` - `just test -p codex-core environment_selection` - `just test -p codex-core turn_environments` - `just test -p codex-core session_update_settings_does_not_rewrite_sticky_environment_cwds` - `just test -p codex-core default_turn_does_not_overlay_legacy_fallback_cwd_onto_stored_thread_environments`
pakrym-oai ·
2026-06-15 16:15:07 -07:00 -
core: cache the tool search handler per session (#27258)
## Why Tool router construction rebuilds the deferred-tool BM25 index during session initialization and before each sampling continuation, even when the searchable tool metadata is unchanged. Local profiling measured `append_tool_search_executor` at roughly 113 ms per continuation, making repeated index construction the largest measured router-building cost. ## What changed - Add a session-scoped `ToolSearchHandlerCache` so continuations and user turns can reuse the existing handler. - Key reuse on the complete ordered `Vec<ToolSearchInfo>`, rebuilding when searchable text, loadable tool specs, source metadata, or ordering changes. - Build handlers outside the cache lock and recheck before publishing them, avoiding holding the mutex during index construction. ## Verification - `cache_reuses_identical_search_infos_and_rebuilds_changed_inputs` covers exact cache reuse and invalidation when the ordered search metadata changes. - Local rollout profiling showed the initial router build populating the cache and unchanged later continuations reusing it: - uncached: 118 ms median across 14 spans from 3 rollouts - cached: 4 ms median across 12 spans from 3 rollouts
mchen-oai ·
2026-06-15 14:48:30 -07:00 -
[codex] simplify shell snapshot ownership (#27756)
## Why Shell snapshot lifecycle state was split between `Shell` and `SessionServices`: `Shell` carried the receiver while session code exposed and forwarded the raw sender. That coupled shell identity to mutable snapshot state and made refresh, inheritance, and file lifetime harder to reason about. ## What changed - make each `Arc<ShellSnapshot>` represent one cwd-specific snapshot generation - store the active generation in `SessionServices` with `ArcSwapOption` - have construction start the background build and expose only a cwd-validated snapshot path - use `ShellSnapshotFile` ownership to delete snapshot files automatically - pass snapshot paths explicitly to shell runtimes instead of storing snapshot state on `Shell` - preserve inherited and in-flight generations by pinning their `Arc` while they are in use ## Test plan - `cargo check -p codex-core --lib` - `just test -p codex-core 'shell_snapshot::tests'` - `just test -p codex-core shell_command_snapshot_still_intercepts_apply_patch` - `just test -p codex-core shell_snapshot_deleted_after_shutdown_with_skills`
pakrym-oai ·
2026-06-15 08:18:13 -07:00 -
Make MCP server contributions thread-scoped (#27670)
## Why `selectedCapabilityRoots` belongs to one thread, but MCP contributors previously received only the global Codex config. That left no clean way for a selected executor capability to contribute MCP servers to its own thread. ## What this PR does - Gives MCP contributors a small context containing the config and, for a running thread, its frozen host-seeded inputs. - Uses the same thread inputs during startup, status queries, refreshes, and skill dependency checks. - Keeps threadless MCP operations and the existing hosted Apps behavior unchanged. - Adds coverage showing that two threads resolve independent registrations and that later lifecycle mutations do not change the frozen MCP inputs. This PR does not discover plugin manifests, add MCP servers, or launch anything new. It only establishes the thread-scoped registration boundary. ## Follow-ups - Resolve selected executor plugin roots through their owning environment filesystem. - Convert their stdio MCP declarations into environment-bound registrations and add an executor MCP end-to-end test. ## Verification - `just fmt` - `cargo check --tests -p codex-protocol -p codex-extension-api -p codex-mcp-extension -p codex-core -p codex-app-server` Tests and Clippy were not run.
jif ·
2026-06-12 11:20:34 +02:00 -
skills: make backend plugin skills invocable without an executor (#27387)
## Why #27198 made the extension-owned `codex_apps` MCP connection the hosted plugin runtime, but its `mcp/skill` resources still bypassed the skills extension. App-server could list and read those resources through generic MCP APIs, but a thread with no selected environment did not expose them in the model's skills catalog or load their `SKILL.md` through `$skill`. Hosted skills should stay remote while using the same typed catalog, source authority, deduplication, bounded contextual catalog, and selected-skill prompt injection as host and executor skills. They should not be downloaded or exposed as ambient filesystem paths. ## What changed - Add a session-scoped `McpResourceClient` over the replaceable MCP connection manager so resource list/read calls follow startup and refresh replacements. - Add a `BackendSkillProvider` that pages `codex_apps` resources, accepts bounded and validated `mcp/skill` entries, and reads a selected skill's `SKILL.md` through the same MCP connection. - Register the remote provider in app-server and include it in the skills catalog even when a thread has no selected capability roots or executor. - Contribute hosted skill metadata through the bounded `AvailableSkillsInstructions` developer-context path, exclude remote entries from per-turn catalog injection, and classify `<skills>` messages as contextual developer content so rollback can trim and rebuild them correctly. ## Testing - Extend the app-server MCP resource integration test with `environments: []` to exercise two-page discovery, filter a non-`mcp/skill` resource, verify the escaped developer catalog entry and user-role `<skill>` fragment containing the fetched `SKILL.md`, and preserve generic MCP resource reads. - Add core event-mapping coverage that classifies `<skills>` developer messages as contextual history.
jif ·
2026-06-11 11:28:16 +02:00 -
[codex] Add new context window tool (#27488)
## Why The token budget feature tells the model how much room remains in the current context window. When the model decides the current window is no longer useful, it needs a way to ask Codex to start over with a fresh context window without spending tokens on a compaction summary. This PR adds that model-requestable escape hatch on top of #27438. ## What changed - Added a direct-model-only `new_context` tool behind `Feature::TokenBudget`. - Stores the tool request on `AutoCompactWindow` and consumes it after sampling so the next follow-up request in the same turn starts in the new window. - Starts the new window as a no-summary compaction checkpoint that contains only fresh initial context, not preserved conversation history. - Keeps the new window aligned with token-budget startup context, including the `Current context window Z` message. - Added integration coverage and a snapshot showing the same-turn `new_context` flow into a fresh full-context follow-up request. ## Validation - `just test -p codex-core token_budget`
pakrym-oai ·
2026-06-11 03:39:07 +00:00 -
[codex] Store compact window id in rollout (#27264)
## Why Compaction window identity is part of session history, not model-client transport state. Persisting it with the compacted rollout item lets resumed threads continue from the reconstructed window without keeping mutable window state on `ModelClient`. ## What changed - Added `window_id` to `CompactedItem` and stamp it when `replace_compacted_history` installs compacted history. - Moved auto-compact window id ownership into `AutoCompactWindow` / `SessionState`; `ModelClient` now receives the request window id from callers instead of storing it. - Returned `window_id` from rollout reconstruction for resume. Reconstruction uses the newest surviving compacted item's stored `window_id` when present, and falls back to the legacy compacted-item count when it is absent. - Kept fork startup at the fresh default window id and updated direct model-client tests to pass explicit test window ids. ## Validation - `cargo check -p codex-core --tests`
pakrym-oai ·
2026-06-10 08:47:16 -07:00 -
Use latest-wins MCP manager replacement (#27259)
## Summary We originally addressed startup prewarming holding the read side of `RwLock<McpConnectionManager>` by snapshotting tool-list state. Review feedback identified the broader ownership problem: the outer synchronization should only publish or retrieve the current manager, while MCP operations rely on the manager's internal synchronization. A follow-up preserved operation retirement with a separate gate, but further review questioned whether that synchronization was actually required and whether we could support latest-wins replacement instead. This PR now stores the current MCP manager in `ArcSwap`. Each operation uses `load_full()` to obtain an owned `Arc<McpConnectionManager>`, then performs MCP I/O without retaining the publication mechanism. Refresh cancels obsolete startup work, constructs a replacement, and atomically publishes it. New operations see the latest manager, while operations that already loaded the previous manager retain a valid handle. Refresh happens at a turn boundary, so there should be no active user tool calls to drain. Git history supports dropping the outer `RwLock`. It was introduced in `03ffe4d595` on November 17, 2025 for non-blocking MCP startup: the session published an empty manager, startup initialized that same object while holding the write lock, and readers waited for initialization. `7cd2e84026` on February 19, 2026 removed that two-phase initialization in favor of constructing a fresh manager and swapping it in, explicitly noting that `Option` or `OnceCell` could replace the placeholder design. Hot reload later reused the existing lock to publish a replacement, but I found no indication that the lock was introduced to guarantee in-flight tool calls finish before refresh or shutdown. Terminal shutdown remains separate from refresh: it aborts startup prewarming and active tasks before shutting down the current manager, so tool calls may be interrupted and no model WebSocket work continues after shutdown. Focused regression coverage exercises pending tool-list cancellation, deferred refresh, and startup-prewarm shutdown.
Charlie Marsh ·
2026-06-10 08:33:21 -07:00 -
[codex] Make MCP connection startup fallible (#27261)
## Why Required MCP server startup was enforced in `Session::new` after `McpConnectionManager` had already created the clients. That split let other manager construction paths bypass the same requirement and exposed manager internals solely so the session could validate them. Keeping required-server readiness in the constructor gives every caller one consistent startup contract. ## What changed - make `McpConnectionManager::new` return `anyhow::Result<Self>` and fail when an enabled, required server cannot initialize - pass the startup cancellation token into the constructor so required-server waits remain cancellable - propagate constructor failures through resource reads, connector discovery, and MCP status collection - preserve the active manager and cancellation token when a refreshed replacement fails - keep required-startup failure collection private and cover the constructor error contract directly ## Validation - updated the focused connection-manager test to assert the complete required-server startup error - local tests not run; relying on CI
Ahmed Ibrahim ·
2026-06-10 00:17:58 -07:00 -
feat: count V2 concurrency by active execution (#26969)
## Why Multi-Agent V2 concurrency should count active non-root turns, not resident or durable agent threads. The limit is intentionally best effort: admission checks are synchronous, but concurrent successful checks may overshoot slightly. ## What changed - Keep one root-derived execution limit on the shared `AgentControl`. - Count active V2 subagent turns with an RAII guard owned by `RunningTask`. - Check capacity before spawning or starting an idle agent, including direct app-server `turn/start` submissions. - Preserve queued delivery for agents that are already running. - Exempt automatic idle continuations so `/goal` work is not dropped when capacity is temporarily full. - Keep root and V1 turns outside this limiter. ## Test coverage - `execution_guards_count_active_v2_subagent_turns` - `execution_guards_ignore_root_and_v1_turns` - `v2_nested_spawn_checks_shared_active_execution_capacity`
jif ·
2026-06-08 14:21:28 +02:00 -
Key request-permission grants by environment (#25850)
## Stack 1. This PR (#25850) - Key request-permission grants by environment: stores and applies sticky permission grants per environment id. 2. #25858 - Add `environmentId` to `request_permissions`: lets the model target a selected environment and resolves relative permission paths against it. 3. #25862 - Propagate permission approval environment id: carries the selected environment id through approval events, app-server requests, TUI prompts, and delegate forwarding. 4. #25867 - Add remote request permissions integration coverage: verifies the selected remote environment across request, approval, grant reuse, and exec. #25858, #25862, and #25867 are stacked on this PR and should be reviewed after it. ## Why Multi-environment CCA turns can attach both local and remote executors, but request-permission grants were still effectively cwd-only. Pending permission requests tracked a cwd, while stored turn/session grants had no environment identity, so sticky grants could be reused through the wrong executor context. This makes the first permission-grant step environment-aware without changing the external `request_permissions` payload shape: omitted environment targeting remains bound to the primary turn environment. ## What Changed - Store turn- and session-scoped request-permission grants by `environment_id`. - Keep the selected `TurnEnvironmentSelection` with pending `request_permissions` calls so approval responses normalize and record grants against the same environment. - Resolve relative `request_permissions` file paths against the primary turn environment cwd instead of deprecated `turn.cwd`. - Apply sticky grants in `shell`, `exec_command`, and `apply_patch` by selected environment id while still using the actual tool cwd for cwd-relative permission materialization. - Update Guardian and request-permissions coverage for the environment-keyed grant behavior. ## Testing Not run locally. Added or updated focused coverage for: - `request_permission_grants_are_environment_keyed` - `request_permissions_tool_resolves_relative_paths_against_primary_environment` - related Guardian/request-permissions sticky grant tests
jif ·
2026-06-02 20:16:57 +02:00 -
feat: show enterprise monthly credit limits in status (#24812)
## Summary Enterprise users can have an effective monthly credit limit, but Codex `/status` currently drops that metadata from the account-usage response. This change adds the optional `spend_control.individual_limit` projection to the existing rate-limit snapshot flow. The backend client reads the monthly limit, app-server exposes it as `individualLimit`, and the TUI renders a `Monthly credit limit` row through the existing progress-bar renderer. When the backend does not return an effective monthly limit, existing rate-limit behavior is unchanged. ## Existing backend state The account-usage backend already returns the effective monthly limit and current usage together: ```json { "spend_control": { "reached": false, "individual_limit": { "limit": "25000", "used": "8000", "remaining": "17000", "used_percent": 32, "remaining_percent": 68, "reset_after_seconds": 86400, "reset_at": 1778137680 } } } ``` Before this change, Codex projected rolling `primary` and `secondary` windows plus `credits`. It ignored `spend_control.individual_limit`, so app-server clients and `/status` could not render the monthly cap. The updated flow is: ```text account usage backend -> backend-client reads spend_control.individual_limit -> existing rate-limit snapshot carries optional individual_limit -> app-server exposes optional individualLimit -> TUI renders Monthly credit limit ``` ## App-server contract `account/rateLimits/read` and sparse `account/rateLimits/updated` notifications now include an additive nullable `rateLimits.individualLimit` field: ```json { "individualLimit": { "limit": "25000", "used": "8000", "remainingPercent": 68, "resetsAt": 1778137680 } } ``` In an `account/rateLimits/read` response, `null` means no monthly limit is available. `account/rateLimits/updated` remains a sparse rolling notification: clients merge available values into their most recent `account/rateLimits/read` snapshot or refetch. Nullable account metadata in a rolling notification does not clear a previously observed value. ## Design decisions - Extend the existing rate-limit snapshot instead of introducing a separate request or wire-level update protocol. - Keep the Codex projection narrow: `/status` needs the effective limit, current usage, remaining percentage, and reset timestamp. - Render the monthly row through the existing progress-bar renderer, with one optional detail line for `8,000 of 25,000 credits used`. - Keep the backend response optional so existing accounts and older usage states preserve their current behavior. - Preserve cached monthly metadata when sparse rolling notifications omit it. Live account-usage reads remain authoritative and can clear a removed limit. ## Visual evidence ```text Monthly credit limit: [██████████████░░░░░░] 68% left (resets 07:08 on 7 May) 8,000 of 25,000 credits used ``` Snapshot: `codex-rs/tui/src/status/snapshots/codex_tui__status__tests__status_snapshot_includes_enterprise_monthly_credit_limit.snap` ## Testing Tests: generated app-server schema verification, protocol tests, backend-client tests, app-server integration coverage, TUI snapshot coverage, formatting, and workspace lint cleanup.efrazer-oai ·
2026-06-01 21:25:42 -07:00 -
Add experimental turn additional context (#24154)
## Summary Adds experimental `additionalContext` support to `turn/start` and `turn/steer` so clients can provide ephemeral external context, such as browser or automation state, without turning that plumbing into a visible user prompt or triggering user-prompt lifecycle behavior. ## API Shape The parameter shape is: ```ts additionalContext?: Record<string, { value: string kind: "untrusted" | "application" }> | null ``` Example: ```json { "additionalContext": { "browser_info": { "value": "Active tab is CI failures.", "kind": "untrusted" }, "automation_info": { "value": "CI rerun is in progress.", "kind": "application" } } } ``` The keys are opaque and caller-defined. ## Context Injection When provided, accepted entries are inserted into model context as hidden contextual message items, not as visible thread user-message items. `kind: "untrusted"` entries are inserted with role `user`: ```text <external_${key}>${value}</external_${key}> ``` `kind: "application"` entries are inserted with role `developer`: ```text <${key}>${value}</${key}> ``` Values are not escaped. Each value is truncated to 1k approximate tokens before wrapping. For `turn/start`, accepted additional context is inserted before normal user input. For `turn/steer`, additional context is merged only when the steer includes non-empty user input; context-only steers still reject as empty input. ## Dedupe Strategy `AdditionalContextStore` lives on session state and stores the latest complete additional-context map. Each `turn/start` or non-empty `turn/steer` treats its `additionalContext` as the current complete set of values. Entries are injected only when the key is new or the exact entry for that key changed, including `value` or `kind`. After merging, the store is replaced with the provided map, so omitted keys are removed from the retained set and can be injected again later if reintroduced. Omitting `additionalContext`, passing `null`, or passing an empty object resets the store to empty and injects nothing. ## What Changed - Threads experimental v2 `additionalContext` through app-server into core turn start and steer handling. - Adds separate contextual fragment types for untrusted user-role context and application developer-role context. - Uses pending response input items so additional context can be combined with normal user input without treating it as prompt text. - Adds integration coverage for start/steer flow, role routing, dedupe/reset behavior, deletion/re-add behavior, hook-blocked input behavior, empty context-only steer rejection, external-fragment marker matching, and truncation.pakrym-oai ·
2026-05-26 13:02:34 -07:00 -
[codex] Make active turn task singular (#24105)
## Why `ActiveTurn` already runs at most one task: starting a task requires that no task is present, and replacement aborts existing work first. Representing that state as an `IndexMap` leaves a multi-task shape for a single-task invariant and makes each lifecycle lookup operate like a collection lookup. The slot remains optional because goal continuation uses an empty active turn as a reservation while deciding whether to start continuation work. ## What changed - Replace `ActiveTurn.tasks` with `task: Option<RunningTask>`. - Update task abort/completion, session lookup and steering, input-queue matching, goal reservation, and network-approval lookup to operate on the singular slot. - Mutate the singular task slot directly instead of retaining collection-era add/remove/take helpers. - Record token usage on the completing active task span without a regular-task-only opt-in flag. ## Validation - `cargo test -p codex-core --lib session::tests::steer_input` - `cargo test -p codex-core --lib session::tests::abort_empty_active_turn_preserves_pending_input` - `cargo test -p codex-core --lib session::tests::queued_response_items_for_next_turn_move_into_next_active_turn` - `cargo test -p codex-core --lib session::tests::active_goal_continuation_runs_again_after_no_tool_turn` - `cargo test -p codex-core --lib session::tests::abort_regular_task_emits_turn_aborted_only` - `cargo test -p codex-core --lib session::input_queue::tests`
pakrym-oai ·
2026-05-26 08:20:58 -07:00 -
core: refresh active permission profiles at runtime (#22931)
## Why Once a named permission profile is selected, runtime state has to keep that profile identity intact instead of collapsing back to anonymous effective permissions. The session refresh path also needs to rebuild profile-derived network proxy state so active profile switches take effect consistently. ## What changed - Preserve the active permission profile through session updates. - Rebuild profile-derived runtime/network configuration when the active profile changes. - Keep the runtime path aligned with the current session configuration APIs. - Tighten the affected tests, including the Windows delete-pending memory-file case that was intermittently tripping CI. ## Stack 1. **This PR**: runtime/session/network propagation for active permission profiles. 2. [#23708](https://github.com/openai/codex/pull/23708): TUI selection plumbing and guardrail flow. 3. [#21559](https://github.com/openai/codex/pull/21559): profile-aware `/permissions` menu and custom profile display. <img width="1296" height="906" alt="image" src="https://github.com/user-attachments/assets/077fa3a7-80cb-4925-80b1-d2395018d90a" />
viyatb-oai ·
2026-05-20 21:55:21 +00:00 -
Support compact SessionStart hooks (#21272)
# Why Compaction replaces the live conversation history, so hooks that use `SessionStart` to re-inject durable model context need a way to run again after that rewrite. Related - #19905 adds dedicated compact lifecycle hooks # What - add `compact` as a supported `SessionStart` source and matcher value - change pending `SessionStart` state from a single slot to a small FIFO queue so `resume` / `startup` / `clear` can be preserved alongside a later `compact` - drain all queued `SessionStart` sources before the next model request, preserving their original order # Testing The new integration coverage verifies both the basic `compact` matcher path and the stacked `resume` -> `compact` case where both hooks contribute `additionalContext` to the next model turn.
Abhinav ·
2026-05-20 20:46:19 +00:00 -
Add
body_after_prefixauto-compact token limit scope (#22870)## Why `model_auto_compact_token_limit` has only been able to budget the full active context. That makes it hard to set a small "growth since compaction" budget for sessions that preserve a large carried window prefix: the preserved prefix can consume the whole budget and force immediate repeated compaction. This PR adds an opt-in `body_after_prefix` scope so callers can apply `model_auto_compact_token_limit` to sampled output and later growth after the current carried prefix, while still forcing compaction before the full model context window is exhausted. ## What changed - Adds `AutoCompactTokenLimitScope` with the existing `total` behavior as the default and a new `body_after_prefix` mode: [`config_types.rs`](https://github.com/openai/codex/blob/973806b1cb35792555bead994cb3ed94656eb171/codex-rs/protocol/src/config_types.rs#L24-L37). - Threads `model_auto_compact_token_limit_scope` through config loading, `Config`, `core-api`, and app-server v2 schema/TypeScript generation. - Records the first observed input-token count for a `body_after_prefix` compaction window and uses it as the baseline when deciding whether the scoped auto-compaction budget is exhausted: [`turn.rs`](https://github.com/openai/codex/blob/973806b1cb35792555bead994cb3ed94656eb171/codex-rs/core/src/session/turn.rs#L743-L781). - Keeps a hard context-window cap in `body_after_prefix`, so scoped budgeting cannot let the active context overrun the usable window. ## Verification Added compact-suite coverage for the two key behaviors: `body_after_prefix` does not re-compact just because the carried prefix is larger than the scoped budget, and it still compacts when the total active context reaches the configured context window: [`compact.rs`](https://github.com/openai/codex/blob/973806b1cb35792555bead994cb3ed94656eb171/codex-rs/core/tests/suite/compact.rs#L3003-L3128).
jif-oai ·
2026-05-19 10:19:46 +00:00 -
cleanup: Remove skill env var dependency prompting (#22721)
Deletes the skill env var dependency prompt feature and its runtime path. env_var entries in skill dependency metadata are now silently ignored during skill loading.
xl-openai ·
2026-05-19 01:24:19 +00:00 -
[codex] Move pending input into input queue (#22728)
## Why Pending model input was split across `Session`, `TurnState`, and the agent mailbox. That made it easy for new paths to manage queued user input or mailbox delivery outside the intended ownership boundary. This PR consolidates the model-facing input lifecycle behind the session input queue so turn-local pending input, next-turn queued items, and mailbox delivery coordination are owned in one place. ## What Changed - Added `session/input_queue.rs` to own pending input queues and mailbox delivery coordination. - Removed the standalone `agent/mailbox.rs` channel wrapper and store mailbox items directly in the input queue. - Moved pending-input mutations off `TurnState`; `TurnState` now exposes the queue-owned storage directly for now. - Routed abort cleanup, mailbox delivery phase changes, next-turn queued items, and active-turn pending input through `InputQueue`. - Boxed stack-heavy agent resume/fork startup futures that the refactor pushed over the default test stack. - Updated session, task, goal, stream-event, and multi-agent call sites and tests to use the new queue ownership. ## Verification - `cargo test -p codex-core --lib agent::control::tests` - `cargo test -p codex-core --lib agent::control::tests::resume_closed_child_reopens_open_descendants -- --exact` - `cargo test -p codex-core --lib agent::control::tests::spawn_agent_fork_last_n_turns_keeps_only_recent_turns -- --exact` - `cargo test -p codex-core --lib agent::control::tests::resume_thread_subagent_restores_stored_nickname_and_role -- --exact` - `cargo test -p codex-core` was also run; it completed with 1814 passed, 4 ignored, and one timeout in `agent::control::tests::resume_thread_subagent_restores_stored_nickname_and_role`, which passed when rerun in isolation.
pakrym-oai ·
2026-05-18 15:43:01 -07:00 -
Fix turn extension data task plumbing (#22646)
## Summary - carry the per-turn extension data through RunningTask so abort handling can rebuild SessionTaskContext - update stale test ExtensionData::new() callsites to pass the turn id ## Testing - Not run after PR branch creation; CI will cover.
jif-oai ·
2026-05-14 16:00:06 +02:00 -
feat: move extension scope ids into ExtensionData (#22490)
## Summary - add a scoped level_id to ExtensionData and expose it through level_id() - remove thread_id/turn_id parameters from extension contributor inputs where the scoped ExtensionData already carries that identity - move turn-scoped extension data onto TurnContext so token usage and lifecycle contributors can share the same turn store ## Testing - cargo check -p codex-extension-api -p codex-core --tests - cargo test -p codex-extension-api - cargo test -p codex-guardian - cargo test -p codex-core --lib record_token_usage_info_notifies_extension_contributors - cargo test -p codex-core --lib submission_loop_channel_close_emits_thread_stop_lifecycle - cargo test -p codex-core --lib submission_loop_channel_close_aborts_active_turn_before_thread_stop_lifecycle - just fix -p codex-extension-api - just fix -p codex-guardian - just fix -p codex-core - just fmt ## Note - Attempted cargo test -p codex-core; it aborted in agent::control::tests::spawn_agent_fork_last_n_turns_keeps_only_recent_turns with the existing stack overflow before the full suite completed.
jif-oai ·
2026-05-13 16:13:16 +02:00 -
feat: add turn lifecycle contributors (#22480)
## Why Extensions can already contribute prompt, tool, turn-item, and thread-lifecycle behavior, but there was no explicit host-owned hook for per-turn setup and cleanup. That makes extension-private turn state awkward: an extension either has to stash it outside the turn lifecycle or depend on core runtime objects. This adds a small turn lifecycle boundary. Extensions receive stable identifiers plus the existing session, thread, and turn `ExtensionData` stores, while core keeps owning task scheduling, cancellation, and turn teardown. ## What Changed - Added `TurnLifecycleContributor` with `on_turn_start`, `on_turn_stop`, and `on_turn_abort` callbacks in `codex-rs/ext/extension-api`. - Added typed `TurnStartInput`, `TurnStopInput`, and `TurnAbortInput` payloads that expose `thread_id`, `turn_id`, `session_store`, `thread_store`, and `turn_store`. - Registered and re-exported turn lifecycle contributors through `ExtensionRegistry` and `ExtensionRegistryBuilder`. - Wired `Session` to emit turn start, stop, and abort callbacks from the existing turn/task lifecycle paths. - Carried the turn-scoped `ExtensionData` through `RunningTask` and `RemovedTask` so stop/abort callbacks receive the same turn store created at turn start. ## Verification - Not run locally.
jif-oai ·
2026-05-13 13:47:27 +02:00 -
extension: wire extension registries into sessions (#21737)
## Why [#21736](https://github.com/openai/codex/pull/21736) introduces the typed extension API, but the runtime does not yet carry a registry through thread/session startup or give contributors host-owned stores to read from. This PR wires that host-side path so later feature migrations can move product-specific behavior behind typed contributions without adding another bespoke seam directly to `codex-core`. ## What changed - Thread `ExtensionRegistry<Config>` through `ThreadManager`, `CodexSpawnArgs`, `Session`, and sub-agent spawn paths. - Wire `ThreadStartContributor` and `ContextContributor` - Expose the small supporting surface needed by non-core callers that construct threads directly, including `empty_extension_registry()` through `codex-core-api`. This PR lands the host plumbing only: the app-server registry is still empty, and concrete feature migrations are intended to follow separately.
jif-oai ·
2026-05-11 11:38:18 +02:00 -
Reapply "Move skills watcher to app-server" (#21652)
## Why PR #21460 reverted the earlier move of skills change watching from `codex-core` into app-server. This reapplies that boundary change so app-server owns client-facing `skills/changed` notifications and core no longer carries the watcher. ## What - Restore the app-server `SkillsWatcher` and register it from thread listener setup. - Remove the core-owned skills watcher and its core live-reload integration surface. - Restore app-server coverage for `skills/changed` notifications after a watched skill file changes. ## Validation - `cargo test -p codex-app-server --test all suite::v2::skills_list::skills_changed_notification_is_emitted_after_skill_change -- --exact --nocapture` - `cargo test -p codex-core --lib --no-run`
pakrym-oai ·
2026-05-08 17:41:15 -07:00 -
[codex] request desktop attestation from app (#20619)
## Summary TL;DR: teaches `codex-rs` / app-server to request a desktop-provided attestation token and attach it as `x-oai-attestation` on the scoped ChatGPT Codex request paths.  ## Details This PR teaches the Codex app-server runtime how to request and attach an attestation token. It does not generate DeviceCheck tokens directly; instead, it relies on the connected desktop app to advertise that it can generate attestation and then asks that app for a fresh header value when needed. The flow is: 1. The Codex desktop app connects to app-server. 2. During `initialize`, the app can advertise that it supports `requestAttestation`. 3. Before app-server calls selected ChatGPT Codex endpoints, it sends the internal server request `attestation/generate` to the app. 4. app-server receives a pre-encoded header value back. 5. app-server forwards that value as `x-oai-attestation` on the scoped outbound requests. The code in this repo is mostly protocol and runtime plumbing: it adds the app-server request/response shape, introduces an attestation provider in core, wires that provider into Responses / compaction / realtime setup paths, and covers the intended scoping with tests. The signed macOS DeviceCheck generation remains owned by the desktop app PR. ## Related PR - Codex desktop app implementation: https://github.com/openai/openai/pull/878649 ## Validation <details> <summary>Tests run</summary> ```sh cargo test -p codex-app-server-protocol cargo test -p codex-core attestation --lib cargo test -p codex-app-server --lib attestation ``` Also ran: ```sh just fix -p codex-core just fix -p codex-app-server just fix -p codex-app-server-protocol just fmt just write-app-server-schema ``` </details> <details> <summary>E2E DeviceCheck validation</summary> First validated the signed desktop app boundary directly: launched a packaged signed `Codex.app`, sent `attestation/generate`, decoded the returned `v1.` attestation header, and validated the extracted DeviceCheck token with `personal/jm/verify_devicecheck_token.py` using bundle ID `com.openai.codex`. Apple returned `status_code: 200` and `is_ok: true`. Then ran the fuller app + app-server flow. The packaged `Codex.app` launched a current-branch app-server via `CODEX_CLI_PATH`, and a local MITM proxy intercepted outbound `chatgpt.com` traffic. The app-server requested `attestation/generate` from the real Electron app process, and the intercepted `/backend-api/codex/responses` traffic included `x-oai-attestation` on both routes: ```text GET /backend-api/codex/responses Upgrade: websocket x-oai-attestation: present POST /backend-api/codex/responses Upgrade: none x-oai-attestation: present ``` The captured header decoded to a DeviceCheck token that also validated with Apple for `com.openai.codex` (`status_code: 200`, `is_ok: true`, team `2DC432GLL2`). </details> --------- Co-authored-by: Codex <noreply@openai.com>
Jiaming Zhang ·
2026-05-08 12:36:02 -07:00 -
pakrym-oai ·
2026-05-07 02:24:20 +00:00 -
Move skills watcher to app-server (#21287)
## Why Skills update notifications are app-server API behavior, but the watcher lived in `codex-core` and surfaced through `EventMsg::SkillsUpdateAvailable`. Moving the watcher out keeps core focused on thread execution and lets app-server own both cache invalidation and the `skills/changed` notification. ## What changed - Added an app-server-owned skills watcher that watches local skill roots, clears the shared skills cache, and emits `skills/changed` directly. - Registers skill watches from the common app-server thread listener attach path, including direct starts, resumes, and app-server-observed child or forked threads. - Stores the `WatchRegistration` on `ThreadState`, so listener replacement, thread teardown, idle unload, and app-server shutdown deregister by dropping the RAII guard. - Removed `EventMsg::SkillsUpdateAvailable`, the core watcher, and the old core live-reload test. - Extended the app-server skills change test to verify a cached skills list is refreshed after a filesystem change without forcing reload. ## Validation - `cargo check -p codex-core -p codex-app-server -p codex-mcp-server -p codex-rollout -p codex-rollout-trace` - `cargo test -p codex-app-server skills_changed_notification_is_emitted_after_skill_change`
pakrym-oai ·
2026-05-06 15:38:11 -07:00 -
xl-openai ·
2026-04-30 14:26:14 -07:00 -
Add persisted hook enablement state (#19840)
## Why After `hooks/list` exposes the hook inventory, clients need a way to persist user hook preferences, make those changes effective in already-open sessions, and distinguish user-controllable hooks from managed requirements without adding another bespoke app-server write API. ## What - Extends `hooks/list` entries with effective `enabled` state. - Persists user-level hook state under `hooks.state.<hook-id>` so the model can grow beyond a single boolean over time. - Uses the existing `config/batchWrite` path for hook state updates instead of introducing a dedicated hook write RPC. - Refreshes live session hook engines after config writes so already-open threads observe updated enablement without a restart. ## Stack 1. openai/codex#19705 2. openai/codex#19778 3. This PR - openai/codex#19840 4. openai/codex#19882 ## Reviewer Notes The generated schema files account for much of the raw diff. The core behavior is in: - `hooks/src/config_rules.rs`, which resolves per-hook user state from the config layer stack. - `hooks/src/engine/discovery.rs`, which projects effective enablement into `hooks/list` from source-derived managedness. - `config/src/hook_config.rs`, which defines the new `hooks.state` representation. - `core/src/session/mod.rs`, which rebuilds live hook state after user config reloads. --------- Co-authored-by: Codex <noreply@openai.com>
Abhinav ·
2026-04-30 04:46:32 +00:00 -
[codex] Add token usage to turn tracing spans (#19432)
## Why Slow Codex turns are easier to debug when token usage is visible in the trace itself, without joining against separate analytics. This adds token usage to existing turn-handling spans for regular user turns only. [Example turn](https://openai.datadoghq.com/apm/trace/9d353efa2cb5de1f4c5b93dc33c3df04?colorBy=service&graphType=flamegraph&shouldShowLegend=true&sort=time&spanID=3555541504891512675&spanViewType=metadata&traceQuery=) <img width="1447" height="967" alt="Screenshot 2026-04-24 at 3 03 07 PM" src="https://github.com/user-attachments/assets/ab7bb187-e7fc-41f0-a366-6c44610b2b2c" /> ## What Changed Added response-level token fields on completed handle_responses spans: gen_ai.usage.input_tokens gen_ai.usage.cache_read.input_tokens gen_ai.usage.output_tokens codex.usage.reasoning_output_tokens codex.usage.total_tokens Added aggregate token fields on regular turn spans: codex.turn.token_usage.* Added an explicit regular-turn opt-in via SessionTask::records_turn_token_usage_on_span() so this is not coupled to span-name strings. ## Testing - `cargo test -p codex-otel` - `cargo test -p codex-core turn_and_completed_response_spans_record_token_usage` - `just fmt` - `just fix -p codex-core` - `just fix -p codex-otel` - Manual local Electron/app-server smoke test: regular user turn emits the new span fields Known status: `cargo test -p codex-core` was attempted and failed in unrelated existing areas: config approvals, request-permissions, git-info ordering, and subagent metadata persistence.
charley-openai ·
2026-04-28 11:41:32 -07:00 -
Add goal core runtime (4 / 5) (#18076)
Adds the core runtime behavior for active goals on top of the model tools from PR 3. ## Why A long-running goal should be a core runtime concern, not something every client has to implement. Core owns the turn lifecycle, tool completion boundaries, interruptions, resume behavior, and token usage, so it is the right place to account progress, enforce budgets, and decide when to continue work. ## What changed - Centralized goal lifecycle side effects behind `Session::goal_runtime_apply(GoalRuntimeEvent::...)`. - Starts goal continuation turns only when the session is idle; pending user input and mailbox work take priority. - Accounts token and wall-clock usage at turn, tool, mutation, interrupt, and resume boundaries; `get_thread_goal` remains read-only. - Preserves sub-second wall-clock remainder across accounting boundaries so long-running goals do not drift downward over time. - Treats token budget exhaustion as a soft stop by marking the goal `budget_limited` and injecting wrap-up steering instead of aborting the active turn. - Suppresses budget steering when `update_goal` marks a goal complete. - Pauses active goals on interrupt and auto-reactivates paused goals when a thread resumes outside plan mode. - Suppresses repeated automatic continuation when a continuation turn makes no tool calls. - Added continuation and budget-limit prompt templates. ## Verification - Added focused core coverage for continuation scheduling, accounting boundaries, budget-limit steering, completion accounting, interrupt pause behavior, resume auto-activation, and wall-clock remainder accounting.
Eric Traut ·
2026-04-24 21:16:00 -07:00 -
permissions: make profiles represent enforcement (#19231)
## Why `PermissionProfile` is becoming the canonical permissions abstraction, but the old shape only carried optional filesystem and network fields. It could describe allowed access, but not who is responsible for enforcing it. That made `DangerFullAccess` and `ExternalSandbox` lossy when profiles were exported, cached, or round-tripped through app-server APIs. The important model change is that active permissions are now a disjoint union over the enforcement mode. Conceptually: ```rust pub enum PermissionProfile { Managed { file_system: FileSystemSandboxPolicy, network: NetworkSandboxPolicy, }, Disabled, External { network: NetworkSandboxPolicy, }, } ``` This distinction matters because `Disabled` means Codex should apply no outer sandbox at all, while `External` means filesystem isolation is owned by an outside caller. Those are not equivalent to a broad managed sandbox. For example, macOS cannot nest Seatbelt inside Seatbelt, so an inner sandbox may require the outer Codex layer to use no sandbox rather than a permissive one. ## How Existing Modeling Maps Legacy `SandboxPolicy` remains a boundary projection, but it now maps into the higher-fidelity profile model: - `ReadOnly` and `WorkspaceWrite` map to `PermissionProfile::Managed` with restricted filesystem entries plus the corresponding network policy. - `DangerFullAccess` maps to `PermissionProfile::Disabled`, preserving the “no outer sandbox” intent instead of treating it as a lax managed sandbox. - `ExternalSandbox { network_access }` maps to `PermissionProfile::External { network }`, preserving external filesystem enforcement while still carrying the active network policy. - Split runtime policies that legacy `SandboxPolicy` cannot faithfully express, such as managed unrestricted filesystem plus restricted network, stay `Managed` instead of being collapsed into `ExternalSandbox`. - Per-command/session/turn grants remain partial overlays via `AdditionalPermissionProfile`; full `PermissionProfile` is reserved for complete active runtime permissions. ## What Changed - Change active `PermissionProfile` into a tagged union: `managed`, `disabled`, and `external`. - Keep partial permission grants separate with `AdditionalPermissionProfile` for command/session/turn overlays. - Represent managed filesystem permissions as either `restricted` entries or `unrestricted`; `glob_scan_max_depth` is non-zero when present. - Preserve old rollout compatibility by accepting the pre-tagged `{ network, file_system }` profile shape during deserialization. - Preserve fidelity for important edge cases: `DangerFullAccess` round-trips as `disabled`, `ExternalSandbox` round-trips as `external`, and managed unrestricted filesystem + restricted network stays managed instead of being mistaken for external enforcement. - Preserve configured deny-read entries and bounded glob scan depth when full profiles are projected back into runtime policies, including unrestricted replacements that now become `:root = write` plus deny entries. - Regenerate the experimental app-server v2 JSON/TypeScript schema and update the `command/exec` README example for the tagged `permissionProfile` shape. ## Compatibility Legacy `SandboxPolicy` remains available at config/API boundaries as the compatibility projection. Existing rollout lines with the old `PermissionProfile` shape continue to load. The app-server `permissionProfile` field is experimental, so its v2 wire shape is intentionally updated to match the higher-fidelity model. ## Verification - `just write-app-server-schema` - `cargo check --tests` - `cargo test -p codex-protocol permission_profile` - `cargo test -p codex-protocol preserving_deny_entries_keeps_unrestricted_policy_enforceable` - `cargo test -p codex-app-server-protocol permission_profile_file_system_permissions` - `cargo test -p codex-app-server-protocol serialize_client_response` - `cargo test -p codex-core session_configured_reports_permission_profile_for_external_sandbox` - `just fix` - `just fix -p codex-protocol` - `just fix -p codex-app-server-protocol` - `just fix -p codex-core` - `just fix -p codex-app-server`Michael Bolin ·
2026-04-23 23:02:18 -07:00 -
feat: let model providers own model discovery (#18950)
## Why `codex-models-manager` had grown to own provider-specific concerns: constructing OpenAI-compatible `/models` requests, resolving provider auth, emitting request telemetry, and deciding how provider catalogs should be sourced. That made the manager harder to reuse for providers whose model catalog is not fetched from the OpenAI `/models` endpoint, such as Amazon Bedrock. This change moves provider-specific model discovery behind provider-owned implementations, so the models manager can focus on refresh policy, cache behavior, picker ordering, and model metadata merging. ## What Changed - Introduced a `ModelsManager` trait with separate `OpenAiModelsManager` and `StaticModelsManager` implementations. - Added `ModelsEndpointClient` so OpenAI-compatible HTTP fetching lives outside `codex-models-manager`. - Moved `/models` request construction, provider auth resolution, timeout handling, and request telemetry into `codex-model-provider` via `OpenAiModelsEndpoint`. - Added provider-owned `models_manager(...)` construction so configured OpenAI-compatible providers use `OpenAiModelsManager`, while static/catalog-backed providers can return `StaticModelsManager`. - Added an Amazon Bedrock static model catalog for the GPT OSS Bedrock model IDs. - Updated core/session/thread manager code and tests to depend on `Arc<dyn ModelsManager>`. - Moved offline model test helpers into `codex_models_manager::test_support`. ## Metadata References The Bedrock catalog metadata is based on the official Amazon Bedrock OpenAI model documentation: - [Amazon Bedrock OpenAI models](https://docs.aws.amazon.com/bedrock/latest/userguide/model-parameters-openai.html) lists the Bedrock model IDs, text input/output modalities, and `128,000` token context window for `gpt-oss-20b` and `gpt-oss-120b`. - [Amazon Bedrock `gpt-oss-120b` model card](https://docs.aws.amazon.com/bedrock/latest/userguide/model-card-openai-gpt-oss-120b.html) lists the `bedrock-runtime` model ID `openai.gpt-oss-120b-1:0`, the `bedrock-mantle` model ID `openai.gpt-oss-120b`, text-only modalities, and `128K` context window. - [OpenAI `gpt-oss-120b` model docs](https://developers.openai.com/api/docs/models/gpt-oss-120b) document configurable reasoning effort with `low`, `medium`, and `high`, plus text input/output modality. The display names, default reasoning effort, and priority ordering are Codex-local catalog choices. ## Test Plan - Manually verified app-server model listing with an AWS profile: ```shell CODEX_HOME="$(mktemp -d)" cargo run -p codex-app-server-test-client -- \ --codex-bin ./target/debug/codex \ -c 'model_provider="amazon-bedrock"' \ -c 'model_providers.amazon-bedrock.aws.profile="codex-bedrock"' \ -c 'model_providers.amazon-bedrock.aws.region="us-west-2"' \ model-list ``` The response returned the Bedrock catalog with `openai.gpt-oss-120b-1:0` as the default model and `openai.gpt-oss-20b-1:0` as the second listed model, both text-only and supporting low/medium/high reasoning effort.
Celia Chen ·
2026-04-24 04:28:25 +00:00 -
[rollout_trace] Add debug trace reduction command (#18880)
## Summary Adds the debug CLI entry point for reducing recorded rollout traces. This gives developers a direct way to inspect whether the emitted trace stream reduces into the expected conversation/runtime model. ## Stack This is PR 5/5 in the rollout trace stack. - [#18876](https://github.com/openai/codex/pull/18876): Add rollout trace crate - [#18877](https://github.com/openai/codex/pull/18877): Record core session rollout traces - [#18878](https://github.com/openai/codex/pull/18878): Trace tool and code-mode boundaries - [#18879](https://github.com/openai/codex/pull/18879): Trace sessions and multi-agent edges - [#18880](https://github.com/openai/codex/pull/18880): Add debug trace reduction command ## Review Notes This PR is intentionally last: it depends on the trace crate, core recorder, runtime/tool events, and session/agent edge data all existing. The command should remain a debug/developer tool and avoid adding new runtime behavior. The useful review question is whether the CLI exposes the reducer in the smallest practical way for local inspection without turning the debug command into a supported user-facing workflow.
cassirer-openai ·
2026-04-24 01:56:48 +00:00 -
[codex] Route live thread writes through ThreadStore (#18882)
Begin migrating the thread write codepaths to ThreadStore. This starts using ThreadStore inside of core session code, not only in the app server code. Rework the interfaces around thread recording/persistence. We're left with the following: * `ThreadManager`: owns the process-level registry of loaded threads and handles cross-thread orchestration: start, resume, fork, lookup, remove, and route ops to running CodexThreads. * `CodexThread`: represents one loaded/running thread from the outside. It is the handle app-server and callers use to submit ops, inspect session metadata, and shut the thread down. * `LiveThread`: session-owned persistence lifecycle handle for one active thread. Core session code uses it to append rollout items, materialize lazy persistence, flush, shutdown, discard init-failed writers, and load that thread’s persisted history. * `ThreadStore`: storage backend abstraction. It answers “how are threads persisted, read, listed, updated, archived?” Local and remote implementations live behind this trait. * `LocalThreadStore`: local ThreadStore implementation. It owns the file/sqlite-specific details and keeps RolloutRecorder as a local implementation detail. This is a few too many Thread abstractions for my liking, but they do all represent different concepts / needs / layers. Migration note: in places where the core code explicitly requires a path, rather than a thread ID, throw an error if we're running with a remote store. Cover the new local live-writer lifecycle with focused tests and preserve app-server thread-start behavior, including ephemeral pathless sessions.
Tom ·
2026-04-23 10:17:09 -07:00 -
feat(request-permissions) approve with strict review (#19050)
## Summary Allow the user to approve a request_permissions_tool request with the condition that all commands in the rest of the turn are reviewed by guardian, regardless of sandbox status. ## Testing - [x] Added unit tests - [x] Ran locally
Dylan Hurd ·
2026-04-23 01:56:32 +00:00 -
feat(auto-review) short-circuit (#18890)
## Summary Short circuit the convo if auto-review hits too many denials ## Testing - [x] Added unit tests --------- Co-authored-by: Codex <noreply@openai.com>
Dylan Hurd ·
2026-04-22 20:34:15 +00:00 -
[rollout_trace] Record core session rollout traces (#18877)
## Summary Wires rollout trace recording into `codex-core` session and turn execution. This records the core model request/response, compaction, and session lifecycle boundaries needed for replay without yet tracing every nested runtime/tool boundary. ## Stack This is PR 2/5 in the rollout trace stack. - [#18876](https://github.com/openai/codex/pull/18876): Add rollout trace crate - [#18877](https://github.com/openai/codex/pull/18877): Record core session rollout traces - [#18878](https://github.com/openai/codex/pull/18878): Trace tool and code-mode boundaries - [#18879](https://github.com/openai/codex/pull/18879): Trace sessions and multi-agent edges - [#18880](https://github.com/openai/codex/pull/18880): Add debug trace reduction command ## Review Notes This layer is the first live integration point. The important review question is whether trace recording is isolated from normal session behavior: trace failures should not become user-visible execution failures, and recording should preserve the existing turn/session lifecycle semantics. The PR depends on the reducer/data model from the first stack entry and only introduces the core recorder surface that later PRs use for richer runtime and relationship events.
cassirer-openai ·
2026-04-22 17:00:48 +00:00