mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
4a05d3b282a73dc7baa67f5baae0eaa511ff8d78
1286 Commits
-
feat: disable orchestrator skills for now (#27646)
Temp disable orchestrator-only skills while waiting for the endpoint to be fixed
jif ·
2026-06-11 20:20:26 +02:00 -
[codex] Propagate plugin app categories (#27420)
## What - Parse optional `.app.json` `category` overrides for plugin apps. - Add nullable `category` to `AppSummary` and `AppTemplateSummary` in the app-server protocol. - Fall back from `branding.category` to the first non-empty `app_metadata.categories` value when building app/template summaries. - Regenerate schema/type fixtures and update plugin read/install tests. ## Why The plugin details UI needs a normalized per-app category. Some apps only provide their default category in metadata, while others need a local `.app.json` override.
charlesgong-openai ·
2026-06-11 10:34:41 -07:00 -
skills: decouple the skills extension from core (#27413)
## Why `ext/skills` currently depends on `codex-core` for two host concerns: reading the concrete `Config` type and borrowing core-owned model-context fragment types. That coupling prevents the extension from being assembled independently above core and leaves context that belongs to the skills feature owned by core. This stacked PR introduces the host boundary needed for the broader extension migration while intentionally preserving existing skills behavior. It is stacked on #27404. ## What changed - Adds a small public `SkillsExtensionConfig` view and makes skills installation generic over the host config type. - Requires the host to map its config into that view; app-server supplies the current `Config` values. - Moves the available-skills and selected-skill context fragment implementations into `ext/skills`, preserving their roles, markers, and rendered bytes. - Removes the direct `codex-core` dependency from `codex-skills-extension`. - Keeps local discovery, invocation, side effects, and the `codex-core-skills` compatibility types unchanged for later staged PRs. ## Behavior This adds no capability and is intended to have no user-visible or model-visible behavior change. The install API and ownership boundary change internally; emitted skills context remains byte-for-byte compatible. ## Validation - Updates the skills extension integration coverage to use a host-owned test config. - Asserts the complete rendered catalog and selected-skill fragments, including their roles and markers. - `just bazel-lock-check` - Rust tests and Clippy were not run locally per request; CI will run them.
jif ·
2026-06-11 14:03:53 +02:00 -
skills: render catalog locators by authority (#27591)
## Why Hosted skills introduced by #27388 use opaque `skill://` resource identifiers, but the skills catalog rendered every locator as a `file` and told the model that every skill body lived on disk. That can send the model toward filesystem tools for a resource that must instead be read through its owning authority. The catalog should describe how each source is accessed without changing the underlying discovery or invocation behavior. ## What changed - Render host skills as `file`, executor-owned skills as `environment resource`, orchestrator-owned skills as `orchestrator resource`, and custom-provider skills as `custom resource`. - Update the shared no-alias guidance to describe source locators rather than assuming every skill is stored on the host filesystem. - Direct orchestrator resources through `skills.list` and `skills.read`, and explicitly tell the model not to treat `skill://` identifiers as filesystem paths. - Preserve the existing filesystem and alias behavior for local skills. ## Scope This PR changes only model-visible catalog rendering and guidance. It does not change skill discovery, selection, prompt injection, provider routing, catalog caching or refresh behavior, resource validation, or the `skills.*` tool contract. ## Verification - Extended skills-extension coverage for host-file and executor-resource labels. - Extended the no-executor app-server flow to assert orchestrator-resource wording and non-filesystem guidance.
jif ·
2026-06-11 13:51:04 +02:00 -
test: cover referenced backend skill reads without an executor (#27404)
## Why PR #27388 lets models read child resources referenced by backend plugin skills without an executor. The integration fixture should prove that real flow: the injected `SKILL.md` advertises a child `skill://` resource, and `skills.read` resolves that exact resource through the backend provider. This is stacked on #27388. ## What changed - Adds a child-resource link to the backend skill fixture and asserts that it reaches model context. - Tightens the end-to-end skills test around `skills.list` followed by `skills.read` for the referenced resource. - Splits the existing app-server `mcpResource/read` coverage into a focused test so the generic RPC path remains covered independently. ## Validation - Adds app-server integration coverage for both the referenced backend skill resource and the generic MCP resource read path.
jif ·
2026-06-11 12:55:58 +02:00 -
skills: expose remote skill resource tools (#27388)
## Why PR #27387 makes backend plugin skills discoverable and invocable without an executor, but resources referenced by those skills still sit behind the generic MCP resource surface. The model needs a skills-owned API that preserves the provider authority and package boundary instead of treating remote resources like local files. This is stacked on #27387. ## What - Adds one `skills` namespace with bounded `list` and `read` tools for remote skill providers. - Revalidates `authority + package` against the live remote catalog on every read, then routes the opaque resource ID back through that provider. - Allows the backend provider to read canonical child `skill://` resources while rejecting cross-package, non-canonical, query, fragment, and traversal-shaped URIs. - Caps each serialized tool result at 8 KB. Lists are paginated; reads return an opaque continuation cursor. - Marks the JSON output as external context so memory generation can apply its normal suppression policy. - Deliberately does not add `skills.search`; that waits for a bounded plugin-service search contract. ## Tool contract Pseudo-Python matching the wire shape: ```python from typing import Literal, NotRequired, TypedDict class RemoteSkillAuthority(TypedDict): kind: Literal["remote"] id: str # e.g. "codex_apps" class RemoteSkill(TypedDict): authority: RemoteSkillAuthority package: str # opaque provider-owned package ID name: str description: str main_resource: str # opaque provider-owned SKILL.md ID class SkillsListParams(TypedDict): cursor: NotRequired[str] class SkillsListResult(TypedDict): skills: list[RemoteSkill] next_cursor: str | None warnings: list[str] truncated: bool class SkillsReadParams(TypedDict): authority: RemoteSkillAuthority # copied from skills.list package: str # copied from skills.list resource: str # provider-owned child resource ID cursor: NotRequired[str] # copy next_cursor to continue class SkillsReadResult(TypedDict): resource: str contents: str next_cursor: str | None truncated: bool class Skills: def list(self, params: SkillsListParams) -> SkillsListResult: ... def read(self, params: SkillsReadParams) -> SkillsReadResult: ... ``` There is one namespace for all remote skills, not one tool or MCP server per skill. No resource ID is converted into a filesystem path. ## Backend dependency `/ps/mcp` must support direct reads of child resources such as `skill://plugin_demo/deploy/references/deploy.md`. This PR implements and tests the Codex side of that contract; production child reads remain dependent on the corresponding plugin-service support. Search remains out of scope until that service exposes a bounded search/resource API. ## Validation - Added an app-server integration test covering `skills.list` followed by `skills.read` with no executor. - Ran `just fmt`. - Ran `just bazel-lock-update` and `just bazel-lock-check`. - Did not run Rust tests or Clippy locally, per request; CI will run them.
jif ·
2026-06-11 12:38:04 +02:00 -
core: enable remote compaction v2 by default (#27573)
## Why Remote compaction v2 is ready to become the default for providers that already support remote compaction. Leaving it behind an under-development opt-in keeps eligible sessions on the legacy remote-compaction path. This does not broaden provider eligibility: OpenAI and Azure move to v2, while Bedrock and OSS providers retain their existing local-compaction behavior. ## What changed - Mark `remote_compaction_v2` stable and enable it by default. - Make tests that intentionally cover legacy remote compaction explicitly disable v2. - Update parity coverage so v2 exercises the production default and only legacy mode opts out. ## Verification - `just test -p codex-core auto_compact_runs_after_resume_when_token_usage_is_over_limit auto_compact_counts_encrypted_reasoning_before_last_user auto_compact_runs_when_reasoning_header_clears_between_turns responses_lite_compact_request_uses_lite_transport_contract`
jif ·
2026-06-11 10:07:19 +00:00 -
skills: make backend plugin skills invocable without an executor (#27387)
## Why #27198 made the extension-owned `codex_apps` MCP connection the hosted plugin runtime, but its `mcp/skill` resources still bypassed the skills extension. App-server could list and read those resources through generic MCP APIs, but a thread with no selected environment did not expose them in the model's skills catalog or load their `SKILL.md` through `$skill`. Hosted skills should stay remote while using the same typed catalog, source authority, deduplication, bounded contextual catalog, and selected-skill prompt injection as host and executor skills. They should not be downloaded or exposed as ambient filesystem paths. ## What changed - Add a session-scoped `McpResourceClient` over the replaceable MCP connection manager so resource list/read calls follow startup and refresh replacements. - Add a `BackendSkillProvider` that pages `codex_apps` resources, accepts bounded and validated `mcp/skill` entries, and reads a selected skill's `SKILL.md` through the same MCP connection. - Register the remote provider in app-server and include it in the skills catalog even when a thread has no selected capability roots or executor. - Contribute hosted skill metadata through the bounded `AvailableSkillsInstructions` developer-context path, exclude remote entries from per-turn catalog injection, and classify `<skills>` messages as contextual developer content so rollback can trim and rebuild them correctly. ## Testing - Extend the app-server MCP resource integration test with `environments: []` to exercise two-page discovery, filter a non-`mcp/skill` resource, verify the escaped developer catalog entry and user-role `<skill>` fragment containing the fetched `SKILL.md`, and preserve generic MCP resource reads. - Add core event-mapping coverage that classifies `<skills>` developer messages as contextual history.
jif ·
2026-06-11 11:28:16 +02:00 -
[codex] Pass auth mode to plugin manager (#27517)
## Summary - Add auth mode state to `PluginsManager`. - Sync the plugin manager auth mode when `ThreadManager` is created and when account auth changes. - Route plugin load outcomes through an auth-aware projection hook so follow-up plugin filtering can stay inside `core-plugins`. ## Motivation This prepares plugin capability loading to be configured by auth mode, such as hiding or exposing app/MCP-backed plugin surfaces based on whether the user is using ChatGPT auth or API-key auth, without leaking those details outside the plugin manager. ## Tests - `just fmt` - `just test -p codex-core-plugins` - `env -u CODEX_SANDBOX_NETWORK_DISABLED -u CODEX_SANDBOX just test -p codex-core thread_manager::tests` - `env -u CODEX_SANDBOX_NETWORK_DISABLED -u CODEX_SANDBOX just test -p codex-app-server`
xl-openai ·
2026-06-10 20:57:35 -07:00 -
[codex] Add comp_hash to model metadata (#27532)
## Summary - add optional `comp_hash` metadata to `ModelInfo` - update `ModelInfo` fixtures for the shared schema change - keep older model responses compatible by defaulting the field to `None` ## Why The models endpoint needs an opaque identifier for compaction-compatible model configurations. This PR only exposes that value in model metadata; it does not add it to turn context or change runtime behavior. Follow-up #27520 carries the value through turn context and rollouts, then uses it to trigger compaction. ## Stack - based directly on `main` - replaces #27519, which was accidentally merged into the wrong base branch - functionality follow-up: #27520 ## Testing - `just test -p codex-protocol model_info_defaults_availability_nux_to_none_when_omitted` - `just fix -p codex-core -p codex-protocol -p codex-analytics -p codex-models-manager`
Ahmed Ibrahim ·
2026-06-10 20:42:55 -07:00 -
feat: add Bedrock API key as a managed auth mode (#27443)
## Why Codex needs to manage Amazon Bedrock API key credentials through the existing auth lifecycle instead of introducing a separate auth manager or provider-specific credential file. Treating Bedrock API key login as a primary auth mode gives it the same persistence, keyring, reload, and logout behavior as the existing OpenAI API key and ChatGPT modes. The credential is valid only for the `amazon-bedrock` model provider. OpenAI-compatible providers must reject this auth mode rather than treating the Bedrock key as an OpenAI bearer token. ## What changed - Added `bedrockApiKey` as an app-server `AuthMode` and `CodexAuth::BedrockApiKey` as a primary `AuthManager` mode. - Added `BedrockApiKeyAuth`, containing the API key and AWS region, to the existing `AuthDotJson` payload stored in `$CODEX_HOME/auth.json` or the configured keyring backend. - Added `login_with_bedrock_api_key(...)`, parallel to `login_with_api_key(...)`, which replaces the current stored login with Bedrock credentials. - Reused generic auth reload and logout behavior instead of adding a Bedrock-specific auth manager or logout path. - Updated login restrictions, status reporting, diagnostics, telemetry classification, generated app-server schemas, and auth fixtures for the new mode. - Added explicit errors when Bedrock API key auth is selected with an OpenAI-compatible model provider. This PR establishes managed storage and auth-mode behavior. Routing the managed key and region into Amazon Bedrock requests will be in follow-up PRs.
Celia Chen ·
2026-06-10 20:42:38 -07:00 -
[codex] Remove redundant plugin app auth state (#27465)
## Summary - remove the redundant `needsAuth` field from `AppSummary` and generated app-server schemas - stop `plugin/read` from querying Apps MCP solely to hydrate unused connector auth state - preserve `plugin/install.appsNeedingAuth` membership and `app/list.isAccessible` as the authentication signals ## Why Codex App and TUI do not consume `plugin/read.plugin.apps[].needsAuth`. Hydrating it could establish an Apps MCP connection and discover tools on a cold `plugin/read` request, adding avoidable latency. The plugin APIs are still marked under development, so removing this wire field is preferable to retaining a misleading default. ## Verification - `just write-app-server-schema` - `just fmt` - `just test -p codex-app-server-protocol` - `just test -p codex-app-server plugin_install_uses_remote_apps_needing_auth_response` - `just test -p codex-app-server plugin_install_returns_apps_needing_auth` - `just test -p codex-app-server plugin_read_returns_plugin_details_with_bundle_contents` - `just test -p codex-tui plugin_detail_popup_snapshot_shows_install_actions_and_capability_summaries` - `$xin-build` simplify and debug reviews
xl-openai ·
2026-06-10 17:33:56 -07:00 -
Forward standalone assistant output to realtime (#27319)
## Why When a realtime session is open without an active frontend-model handoff, completed Codex assistant messages are currently dropped. That prevents the frontend model from hearing orchestrator preambles and final responses produced by typed turns or other non-handoff work, which makes the two models present as disconnected personas. Active handoffs already forward each completed assistant message, including preambles. This change leaves those V1 and V2 paths intact and fills only the no-active-handoff gap. ## What changed - Send standalone V1 assistant messages through `conversation.handoff.append` with a stable synthetic handoff ID - Send standalone V2 assistant messages as normal `[BACKEND]` `conversation.item.create` message items, then enqueue `response.create` so the frontend model responds - Preserve the existing active V1 and V2 transport and completion behavior - Continue excluding user messages from realtime mirroring - Skip empty output and cap each complete context injection, including its V2 prefix, at 1,000 tokens - Add end-to-end coverage for both wire formats, V2 response creation, preambles, final responses, and truncation ## Test plan - CI
guinness-oai ·
2026-06-10 21:32:29 +00:00 -
[codex] Skip local curated discovery for remote plugins (#27311)
## Summary - skip the local `openai-curated` marketplace before marketplace loading when tool-suggest discovery uses remote plugins - preserve existing marketplace listing behavior for all other callers and when remote plugins are disabled - add regression coverage proving the curated marketplace is excluded before its malformed manifest can be read ## Why Tool-suggest discovery previously loaded every local `openai-curated` plugin manifest and only discarded that marketplace afterward when remote plugins were enabled. The remote catalog is used in that mode, so the local scan consumed CPU without contributing discoverable plugins. ## Impact Remote-plugin tool suggestion discovery no longer reads the local curated marketplace and its plugin manifests. `openai-bundled`, configured marketplaces, normal `plugin/list` behavior, and local curated discovery when remote plugins are disabled are unchanged. ## Validation - `just test -p codex-core-plugins list_marketplaces_can_skip_openai_curated_before_loading` - `just test -p codex-core list_tool_suggest_discoverable_plugins_omits_openai_curated_when_remote_enabled` - `just fmt` - `git diff --check`
xl-openai ·
2026-06-10 13:11:09 -07:00 -
[codex] add /import for external agents (#27071)
## Why External-agent import should be discoverable and deliberate without blocking startup or claiming the public `codex [PROMPT]` CLI namespace. The slash command keeps the flow local to the interactive TUI and reuses the existing app-server import API. ## What changed - add the user-facing `/import` slash command - detect external-agent importable items only when the command is invoked - run imports through the embedded local app-server - show start and completion messages, refresh configuration, and block duplicate imports while one is pending - reject the flow for unsupported remote and local-daemon sessions ## Validation - `just test -p codex-tui external_agent_config_migration` (10 passed) - manually exercised an isolated TUI fixture with existing external-agent setup and session data using a fresh `CODEX_HOME` - verified picker customization, plugin and session detection, import completion, repeated invocation, and imported-session resume context - the broader `just test -p codex-tui` run passed 2,805 tests, with 2 unrelated guardian feature-flag failures and 4 skipped tests ## Draft follow-ups - review whether completion messaging should remain attached to the initiating chat if the user switches chats during an import - review shutdown semantics for an in-progress background import ## Stack 1. [#27064](https://github.com/openai/codex/pull/27064): remove the startup migration flow 2. [#27065](https://github.com/openai/codex/pull/27065): extract the picker renderer 3. [#27070](https://github.com/openai/codex/pull/27070): add the external-agent import picker UX 4. [#27071](https://github.com/openai/codex/pull/27071): expose the flow through `/import` **This PR is stack item 4.** Draft while the lower stack dependencies are reviewed.
stefanstokic-oai ·
2026-06-10 15:53:15 -04:00 -
feat: make ThreadStore available on ThreadExtensionDependencies (#27439)
Generally useful for extensions.
Michael Bolin ·
2026-06-10 15:17:15 -04:00 -
[codex] Raise app-server recursion limit (#27421)
## Summary Unblock Rust release builds after tracing instrumentation increased the async future query depth beyond rustc's default limit. Set the `codex-app-server` crate recursion limit to 256. This changes compilation only; runtime behavior is unchanged. ## Validation - `just test -p codex-app-server` - `cargo build --release --bin codex-app-server`
Adam Perry @ OpenAI ·
2026-06-10 11:37:14 -07:00 -
fix: Auto-recover from corrupted sqlite databases (#26859)
Further investigation of the sqlite incidents showed that the problems are due to corruption from the older version of SQLite that we recently upgraded, and that the data is truly corrupted in the root database -- recovery of all data is not possible. Given that the data is reconstructable from the rollouts on disk, we should just auto-backup the database and let codex rebuild the rollout info from the disk rollouts. The new behavior is that appserver auto-backs-up and rebuilds (with logs reflecting that behavior). The CLI now pops a message letting you know this happened and the paths of the backed-up corrupt db and the new database. There is also context added so that the desktop app can read the rebuild info from it and inform the user with it.
David de Regt ·
2026-06-10 11:24:29 -07:00 -
Add app-server
thread/deleteAPI (#25018)## Why Clients can archive and unarchive threads today, but there is no app-server API for permanently removing a thread. Deletion also needs to cover the full session tree: deleting a main thread should remove spawned subagent threads and the related local metadata instead of leaving orphaned rollout files, goals, or subagent state behind. ## What - Adds the v2 `thread/delete` request and `thread/deleted` notification, with the response shape kept consistent with `thread/archive`. - Implements local hard delete for active and archived rollout files. - Deletes the requested thread's state DB row as the commit point, then best-effort cleans associated state including spawned descendants, goals, spawn edges, logs, dynamic tools, and agent job assignments. - Updates app-server API docs and generated protocol schema/TypeScript fixtures.
Eric Traut ·
2026-06-10 11:22:12 -07:00 -
Add app-server background terminal process APIs (#26041)
## Summary Codex Apps needs app-server as the source of truth for chat-started background terminals instead of guessing from local process trees. This PR adds experimental v2 APIs to list and terminate background terminals for a loaded thread using app-server process ids, so clients can manage background terminals without local PID discovery. ## Changes - `thread/backgroundTerminals/list` returns paginated background terminal records with `itemId`, app-server `processId`, `command`, `cwd`, nullable `osPid`, nullable `cpuPercent`, and nullable `rssKb`. - `thread/backgroundTerminals/terminate` terminates one running background terminal by app-server `processId` and returns whether a process was terminated. - Background terminal list and terminate operations use unified-exec process manager state as their source of truth.
Eric Traut ·
2026-06-10 11:18:09 -07:00 -
Use plugin-service MCP as the hosted plugin runtime (#27198)
## Stack - Base: #27191 - This PR is the third vertical and should be reviewed against `jif/external-plugins-2`, not `main`. ## Why #27191 moves the host-owned Apps MCP registration behind an extension contributor, but deliberately preserves the existing endpoint-selection feature while that contribution contract lands. App-server can therefore resolve the server through extensions, yet the hosted plugin endpoint is still selected through temporary `apps_mcp_path_override` plumbing. That is not the long-term plugin model. A plugin can bundle skills, connectors, MCP servers, and hooks, and those components do not all need the same source or execution environment. In particular, an authenticated HTTP MCP server can expose plugin capabilities directly from a backend without an executor or an orchestrator filesystem. This PR completes that hosted vertical. App-server's MCP extension now owns the aggregate hosted plugin runtime at `/ps/mcp`. Connector actions continue to arrive as MCP tools, while backend-provided skills arrive as MCP resources and use Codex's existing resource list/read paths. No second backend client, skill filesystem, or generic plugin activation framework is introduced. The backend route remains the hosted implementation. This change replaces Codex's temporary endpoint-selection mechanism, not the service behind the endpoint. ## What changed ### Hosted plugin runtime The MCP extension now contributes `codex_apps` as the hosted plugin runtime rather than as a configurable Apps endpoint: - `https://chatgpt.com` resolves to `https://chatgpt.com/backend-api/ps/mcp`; - a bare custom ChatGPT base resolves to `/api/codex/ps/mcp`; - the existing product-SKU header and ChatGPT authentication behavior are preserved; - executor availability is never consulted for this streamable HTTP transport. The same MCP connection carries both component shapes supported by the hosted endpoint: - connector actions are discovered and invoked as MCP tools; - hosted skills are enumerated and read as MCP resources through the existing `list_mcp_resources` and `read_mcp_resource` paths. This keeps component access in the subsystem that already owns the protocol instead of downloading backend skills into an orchestrator filesystem or inventing a parallel hosted-skill client. ### Explicit runtime ordering `McpManager` now resolves the reserved `codex_apps` entry in three ordered phases: 1. install the legacy Apps fallback for compatibility; 2. apply ordered extension `Set` or `Remove` overlays; 3. apply the final ChatGPT-auth gate without synthesizing the server again. This ordering is important: - an ordinary configured or plugin MCP server cannot claim the auth-bearing `codex_apps` name; - an extension-contributed hosted runtime wins over the fallback; - an extension `Remove` remains authoritative; - a host without the MCP extension retains the legacy Apps endpoint and current local-only behavior. The temporary `legacy_apps_mcp_loader_enabled` coordination flag is no longer needed. ### Remove the path override The `apps_mcp_path_override` feature and its runtime plumbing are removed, including: - the feature registry entry and structured feature config; - `Config` and `McpConfig` fields; - config schema output; - config-lock materialization; - URL override handling in `codex-mcp`. Existing boolean and structured forms still deserialize as ignored compatibility input. They are omitted from new serialized config, and config-lock comparison normalizes the removed input so older locks remain replayable. ### App-server coverage App-server MCP fixtures now serve the hosted route at `/api/codex/ps/mcp`. Existing resource-read and tool/elicitation flows therefore exercise the extension-owned endpoint rather than succeeding through the legacy fallback. The stack also adds the missing `codex_chatgpt::connectors` re-export for the manager-backed connector helper introduced in #27191. ## Compatibility - App-server installs the extension and uses `/ps/mcp` for the hosted runtime. - CLI and other hosts that do not install the extension retain the legacy Apps endpoint. - Apps disabled or non-ChatGPT authentication removes `codex_apps` from the effective runtime view. - Existing local plugins, local skills, executor-selected skills, configured MCP servers, and MCP OAuth behavior are otherwise unchanged. - Backend plugin enablement remains account/workspace state owned by the hosted endpoint; this PR does not add thread-local backend plugin selection. ## Architectural fit The stack now proves two independent runtime shapes: 1. #27184 resolves filesystem-backed skills through the executor that owns a selected root. 2. #27191 and this PR resolve a backend-hosted HTTP MCP through an extension with no executor. Together they preserve the intended separation: - selection identifies a plugin/root when explicit selection is needed; - each component's owning extension resolves its concrete access mechanism; - execution stays with the runtime required by that component; - existing skills, MCP, connector, and hook subsystems remain the downstream consumers. ## Planned follow-ups 1. **Executor stdio MCP:** selecting an executor plugin registers a manifest-declared stdio MCP server and executes it in the environment that owns the plugin. 2. **Optional backend selection:** only if CCA needs thread-local selection distinct from backend account/workspace enablement, add a concrete backend-owned capability location and surface those selected skills through the skills catalog. 3. **Connector metadata and hooks:** activate those plugin components through their existing owning subsystems, with executor hooks remaining environment-bound. 4. **Propagation and persistence:** define explicit resume, fork, subagent, refresh, and environment-removal semantics once selected roots have multiple real consumers. 5. **Local convergence:** migrate legacy local skill, MCP, connector, and hook paths behind their owning extensions one vertical at a time, then remove duplicate core managers and compatibility plumbing after parity. ## Verification Coverage in this change exercises: - extension-owned `/backend-api/ps/mcp` registration without an executor; - preservation of the legacy endpoint in hosts without the extension; - extension `Set` and `Remove` precedence over the legacy fallback; - ChatGPT-auth gating for the reserved server; - hosted MCP resource reads with and without an active thread; - connector tool invocation and MCP elicitation through the hosted route; - ignored boolean and structured forms of the removed path override; - config-lock replay compatibility for the removed feature. `cargo check -p codex-features -p codex-mcp-extension -p codex-app-server` passes. Tests and Clippy were not run locally under the current development instruction; CI provides the full validation pass.
jif ·
2026-06-10 12:54:21 +02:00 -
[codex] Fix post-merge analytics integration failures (#27285)
## Why Recent merges left `main` with analytics integration build failures. Local Cargo runs also made the trimmed-skills test depend on developer-installed skills, while Bazel used an isolated home. ## What changed - Clone `thread_metadata.thread_source` when constructing goal analytics event parameters. - Group app-server thread extension inputs into `ThreadExtensionDependencies`. - Isolate the trimmed-skills test home so its exact fixture count is stable across Cargo and Bazel. ## Validation - `cargo check -p codex-analytics` - `just test -p codex-analytics` (71 tests) - `just test -p codex-app-server` (837 tests; one unrelated zsh-fork timeout passed on retry)
Adam Perry @ OpenAI ·
2026-06-09 20:52:09 -07:00 -
[codex-analytics] emit goal lifecycle analytics (#27078)
## Why - Currently, there is no analytics event for `/goal` behavior - Existing events cannot identify goal execution or its resulting outcome - The original update in [#26182](https://github.com/openai/codex/pull/26182) was implemented before `/goal` moved into `codex-goal-extension`. ## What Changed - Adds `codex_goal_event` serialization and enrichment to `codex-analytics` - Emits goal events from the canonical `codex-goal-extension` mutation and accounting paths: - `created` when a new logical goal is persisted - `usage_accounted` when cumulative goal usage is persisted - `status_changed` when the stored goal status changes - `cleared` when the goal is deleted - Preserves causal `turn_id` for turn driven events and uses null attribution for external or idle lifecycle events - Changes goal deletion to return the deleted row so `cleared` retains the stable goal ID ## Event Details Includes standard analytics metadata along with goal specific fields: - `goal_id`: Stable ID stored in the local SQLite goal row and shared across the goal's events - `event_kind`: Observed operation (see the 4 lifecycle events cited in the above bullet) - `goal_status`: Resulting or last stored status: `active`, `paused`, `blocked`, `usage_limited`, etc. - `has_token_budget`: Indicates whether a token budget is configured - `turn_id`: Causal turn ID, or null when no causal turn exists - `cumulative_tokens_accounted`: Cumulative tokens on `usage_accounted` events; null otherwise - `cumulative_time_accounted_seconds`: Cumulative active time on `usage_accounted` events; null otherwise ## Validation - `just test -p codex-analytics -p codex-state -p codex-goal-extension` - `just test -p codex-core -E 'test(/goal/)'` - `just test -p codex-app-server` - `cargo build -p codex-analytics -p codex-core -p codex-state -p codex-app-server`
marksteinbrick-oai ·
2026-06-09 18:45:54 -07:00 -
Add per-session realtime model and version overrides (#24999)
## Why Clients need to select a realtime session configuration for an individual start without rewriting persisted configuration or restarting the app-server process. ## What Changed - Add optional `model` and `version` fields to `thread/realtime/start` - Forward those optional values through the realtime start operation and apply them only for that session - Preserve existing configured/default behavior when the new fields are omitted - Update generated protocol schema and app-server documentation ## Validation - Added/updated protocol serialization coverage for the new optional request fields - Added focused core coverage for a session override taking precedence over configured realtime selection - Added focused app-server coverage that a request override reaches the realtime WebSocket handshake
guinness-oai ·
2026-06-09 17:54:32 -07:00 -
Route hosted Apps MCP through extensions (#27191)
## Stack - Base: #27184 - This PR is the second vertical and should be reviewed against `jif/external-plugins-1`, not `main`. ## Why CCA is moving toward a split runtime where the orchestrator may have no filesystem or executor, but it still needs to activate remotely hosted plugin components. HTTP MCP servers are the simplest complete example: they need configuration and host authentication, but they do not need an executor process. The Apps MCP endpoint is currently synthesized by a special-purpose loader inside the MCP runtime. That works locally, but it leaves hosted MCP activation outside the extension model being established in #27184. It also makes the Apps path a poor foundation for plugins whose skills, MCP servers, connectors, and hooks may come from different sources or execute in different places. This PR moves that one behavior behind an extension-owned contribution while preserving the existing local fallback. It deliberately does not introduce a generic plugin activation framework. ## What changed ### MCP extension contribution `codex-extension-api` gains an ordered `McpServerContributor` contract. A contributor returns typed `Set` or `Remove` overlays for MCP server configuration; later contributors win for the names they own. The contract stays at the existing MCP configuration boundary. Extensions do not create a second connection manager or transport abstraction. ### Hosted Apps MCP extension A new `codex-mcp-extension` contributes the reserved `codex_apps` server from the existing Apps feature, ChatGPT base URL, path override, and product SKU configuration. When `apps_mcp_path_override` is enabled for `https://chatgpt.com`, the resulting streamable HTTP endpoint is `https://chatgpt.com/backend-api/ps/mcp`. The existing ChatGPT-auth gate remains authoritative, so this server can run in an orchestrator-only process without being exposed for API-key sessions. ### One resolved runtime view `McpManager` now distinguishes three views: - **configured:** config- and plugin-backed servers before extension overlays; - **runtime:** configured servers plus host-installed extension contributions; - **effective:** runtime servers after auth gating and compatibility built-ins. App-server installs the hosted MCP extension and uses the runtime view for thread startup, refresh, status, threadless resource reads, connector discovery, and MCP OAuth lookup. This keeps `mcpServer/oauth/login` consistent with the servers exposed by the other MCP APIs. The hosted Apps server itself continues to use existing ChatGPT host authentication rather than MCP OAuth. ## Compatibility Hosts that do not install the MCP extension retain the existing Apps MCP synthesis path. This preserves current local-only, CLI, and standalone-host behavior while app-server exercises the extension path. Disabling Apps removes the reserved `codex_apps` entry, and losing ChatGPT auth removes it from the effective runtime view. Executor availability is not consulted for this HTTP transport. ## Follow-ups The next vertical will resolve a manifest-declared stdio MCP server from an executor-selected plugin root and execute it in the environment that owns that root. Later verticals can add backend-owned skills, connector metadata, hooks, durable selection semantics, and incremental local convergence without changing the component-specific runtime boundaries introduced here. ## Verification Focused coverage was added for: - contributing the hosted Apps MCP at `/backend-api/ps/mcp` without an executor; - requiring ChatGPT auth in the effective runtime view; - removing a reserved configured Apps server when the Apps feature is disabled. `cargo check -p codex-app-server -p codex-mcp-extension -p codex-extension-api -p codex-mcp` passed. Tests and Clippy were not run locally under the current development instruction; CI provides the full validation pass.
jif ·
2026-06-09 22:44:16 +02:00 -
Eric Ning ·
2026-06-09 12:52:05 -07:00 -
[codex-analytics] add extensible feature thread sources (#27063)
## Why - `ThreadSource` currently defines a closed set of core-owned values - Product features also create threads for background or scheduled work - Adding every product-specific value to the core enum would require repeated `codex-rs` protocol changes - Feature-backed values let product callers provide precise attribution while preserving the existing core classifications ## What Changed - Adds `ThreadSource::Feature(String)` for app-owned thread source values - Represents all app-server v2 thread sources as scalar strings, so a feature source is supplied as `"automation"` - Persists and emits the feature's plain string label, so `"automation"` produces `thread_source="automation"` in analytics - Keeps `user`, `subagent`, and `memory_consolidation` as explicit core-owned values and regenerates the app-server schemas and TypeScript bindings ## Verification - `just write-app-server-schema` - `cargo check --workspace` - `just test -p codex-protocol feature_thread_source_serializes_as_its_app_owned_label` - `just test -p codex-app-server-protocol thread_sources_round_trip_as_scalar_labels` - `cargo test -p codex-analytics thread_initialized_event_serializes_expected_shape` - `just fmt`
marksteinbrick-oai ·
2026-06-09 12:27:10 -07:00 -
Load selected executor skills through extensions (#27184)
## Why CCA is moving toward a split runtime where the orchestrator may not have a filesystem, while executors can expose preinstalled plugins and skills. A thread therefore needs to select capabilities without asking app-server or core to interpret executor-owned paths through the orchestrator's filesystem. The longer-term model is broader than executor skills: - A plugin is a bundle of skills, MCP servers, connectors/apps, and hooks. - A plugin root can be local, executor-owned, or hosted by a backend. - Components inside one plugin can use different access and execution mechanisms. A skill may be read from a filesystem or through backend tools; an HTTP MCP server can run without an executor; a stdio MCP server or hook needs an execution environment. - Core should carry generic extension initialization data. The extension that owns a component should discover it, expose it to the model, and invoke it through the appropriate runtime. This PR establishes that architecture through one complete vertical: selecting a root on an executor, discovering the skills beneath it, exposing those skills to the model, and reading an explicitly invoked `SKILL.md` through the same executor. ## Contract `thread/start` gains an experimental `selectedCapabilityRoots` field: ```json { "selectedCapabilityRoots": [ { "id": "deploy-plugin@1", "location": { "type": "environment", "environmentId": "workspace", "path": "/opt/codex/plugins/deploy" } } ] } ``` The root is intentionally not classified as a "plugin" or "skill" in the API. It can point at a standalone skill, a directory containing several skills, or a plugin containing skills and other components. This PR only teaches the skills extension how to consume it; later extensions can resolve MCP, connector, and hook components from the same selection. The platform-supplied `id` is stable selection identity. The location says which runtime owns the root and gives that runtime an opaque path. App-server does not inspect or canonicalize the path. ## What changed ### Generic thread extension initialization App-server converts selected roots into `ExtensionDataInit`. Core carries that generic initialization value until the final thread ID is known, then creates thread-scoped `ExtensionData` before lifecycle contributors run. This keeps `Session` and core independent of the capability-selection contract. The initialization value is consumed during construction; it is not retained as another long-lived `Session` field. ### Executor-backed skills The skills extension now owns an `ExecutorSkillProvider` that: - resolves the selected environment through `EnvironmentManager` - discovers, canonicalizes, and reads skills through that environment's `ExecutorFileSystem` - contributes the bounded selected-skill catalog as stable developer context - reads an explicitly invoked skill body through the authority that listed it - warns when an environment or root is unavailable - never falls back to the orchestrator filesystem for an executor-owned root Skill catalog and instruction fragments have hard byte bounds, which also bound them below the 10K-token per-item context limit. If a selected executor skill has the same name as a legacy local skill, the executor selection owns that invocation and the local body is not injected a second time. Existing local and bundled skill loading remains in place. Omitting `selectedCapabilityRoots` therefore preserves current local-only behavior. ## Current semantics - Only environment-owned locations are represented in this first contract. - Roots are resolved by the destination extension, not by app-server or core. - An unavailable executor or invalid root produces a warning and no capabilities from that root; it does not trigger a local-filesystem fallback. - Selection applies to a newly started active thread. - MCP servers, connectors, and hooks beneath a selected plugin root are not activated yet. - Selection is not yet persisted or inherited across resume, fork, or subagent creation. Existing local capabilities continue to behave as they do today in those flows. ## Planned vertical follow-ups 1. **Hosted HTTP MCP:** add an extension-backed HTTP MCP source that works without an executor, then replace the special-purpose MCP plugins loader with that implementation. 2. **Executor MCP:** register and execute stdio MCP servers through the environment that owns the selected plugin root. 3. **Backend skills:** add a hosted skill source whose catalog and bodies are accessed through extension tools rather than a filesystem. 4. **Connectors and hooks:** activate those components through their owning extensions, using the same selected-root boundary and component-specific runtime. 5. **Durable selection:** define the desired-selection lifecycle, persist it, and make resume, fork, and subagent inheritance explicit rather than accidental. 6. **Local convergence:** incrementally route existing local plugin, skill, and MCP loading through the same extension model while preserving current local behavior. Each follow-up remains reviewable as an end-to-end capability. The platform selects roots, generic thread extension data carries the selection, and the owning extension resolves and operates its component. ## Verification Coverage added for: - app-server end-to-end discovery and explicit invocation of a skill inside an executor-selected plugin root - exclusive invocation when a selected executor skill collides with a local skill name - executor filesystem authority for discovery, canonicalization, and reads - thread extension initialization before lifecycle contributors run - stable executor catalog context, explicit invocation, context rebuilding, hidden skills, and preserved host/remote catalog behavior Targeted protocol, core-skills, skills-extension, core lifecycle, and app-server executor-skill tests were run during development.jif ·
2026-06-09 19:51:54 +02:00 -
app-server: reject direct input to multi-agent v2 sub-agents (#27173)
## Why Multi-agent v2 sub-agents are owned and coordinated by their parent agent. Allowing an app-server client to start or steer turns on a spawned child bypasses the multi-agent messaging path and creates a second, conflicting source of work for that sub-agent. ## What changed - Reject direct `turn/start` and `turn/steer` requests targeting multi-agent v2 thread-spawn sub-agents. - Identify these targets using both the thread's resolved multi-agent version and its `SubAgentSource::ThreadSpawn` session source, leaving root threads, v1 agents, and other sub-agent types unchanged. - Return a consistent invalid-request error before validating or applying the submitted input. ## Testing - Added an app-server integration test that spawns a real multi-agent v2 child and verifies that direct `turn/start` and `turn/steer` requests are rejected.
jif ·
2026-06-09 19:40:40 +02:00 -
Avoid rereading rollout history during cold resume (#27031)
## Summary - reuse the history-bearing `StoredThread` loaded while probing for a running thread - avoid rereading and reparsing the rollout when that probe finds no active process - reload after shutting down a loaded thread because shutdown may flush newer rollout items - add a regression test that verifies cold resume performs one history-bearing store read ## Problem `thread/resume` first reads the persisted thread with history while checking whether the thread is already running. When no running process exists, cold resume currently falls through to `resume_thread_from_rollout`, which reads and parses the same history again. That duplicate work grows with rollout size and remains on the synchronous resume path even when the caller requests `excludeTurns`. ## Background The duplicate read was introduced by #24528, which fixed resume overrides for idle cached threads. To support resumes specified by rollout path, `resume_running_thread` began loading the stored thread with history so it could resolve the canonical thread ID and determine whether a cached `CodexThread` was already loaded. That history is needed when the loaded-thread path handles the request. On a cold miss, however, the function's boolean result could only report that no loaded thread handled the request. It discarded the history-bearing `StoredThread`, and the normal cold-resume path immediately loaded and parsed the same rollout again. This change preserves the idle cached-thread behavior from #24528 while allowing the cold-resume path to reuse the probe result. ## Performance I benchmarked real retained rollouts using isolated `CODEX_HOME` directories, explicit rollout paths, debug builds of the commit and its exact parent, and alternating parent/patch order. The table below uses `thread/resume` with `excludeTurns: true`; response payload sizes were identical. | Rollout size | Records | Parent median | Patch median | Median paired saving | | ---: | ---: | ---: | ---: | ---: | | 6 MB | 3,574 | 541 ms | 441 ms | 132 ms | | 30 MB | 15,220 | 1.505 s | 1.041 s | 701 ms | | 60 MB | 31,453 | 2.644 s | 1.742 s | 970 ms | | 149 MB | 100,874 | 10.506 s | 7.156 s | 3.350 s | | 559 MB | 259,734 | 27.759 s | 16.725 s | 9.836 s | The absolute saving increases with thread size, as expected when removing one complete JSONL history read and parse. Total resume time is also content-dependent, so the relationship is not perfectly linear. I also tested full-history resume with `excludeTurns: false`. The response payload was byte-identical between variants, and the same size-dependent improvement remained visible: | Rollout size | Parent median | Patch median | Median paired saving | | ---: | ---: | ---: | ---: | | 6 MB | 1.052 s | 904 ms | 270 ms | | 30 MB | 2.667 s | 1.762 s | 924 ms | | 60 MB | 8.464 s | 6.272 s | 3.680 s | | 149 MB | 26.719 s | 12.118 s | 14.601 s | | 559 MB | 40.359 s | 25.475 s | 16.590 s | ## Validation - `just test -p codex-app-server cold_thread_resume_reuses_non_local_history_probe` - `just fix -p codex-app-server -p codex-thread-store` - `just fmt`
Zanie Blue ·
2026-06-09 11:16:27 -05:00 -
app-server: clear stale thread watches after v2 agent interruption (#27166)
## Why PR #27007 moved MultiAgentV2 interruption reporting from the legacy collaboration close event to `SubAgentActivity::Interrupted`. App-server's missing-thread cleanup still ran only for the legacy event, so an interrupted child that had already been unloaded could remain marked as loaded and running in `ThreadWatchManager`. That leaves thread status and running-turn accounting stale, including the count used during graceful shutdown. ## What changed - Handle `SubAgentActivity::Interrupted` separately in app-server event processing. - Remove the child's thread watch when `ThreadManager` no longer has that thread. - Continue forwarding the same completed sub-agent activity notification to clients. ## Testing - Added a regression test that starts with a running watch for an unloaded child, applies the interrupted activity event, and verifies the watch is removed, the running count returns to zero, and the client notification is still emitted.
jif ·
2026-06-09 13:27:53 +02:00 -
multi-agent: add path-based v2 activity tracking (#27007)
## Why Multi-agent v2 identifies agents by canonical paths, but its tool handlers still emitted the larger legacy collaboration begin/end events built around nickname and role metadata. App-server, rollout-trace, analytics, and TUI consumers therefore lacked one compact path-based completion signal that behaved consistently across live events and replay. The TUI also needs a bounded `/agent` status surface for v2 agents. It should use recent local activity for previews, refresh liveness without loading full histories, and keep the legacy picker available when no path-backed v2 agent is known. ## What changed - Replace the v2 `spawn_agent`, `send_message`, `followup_task`, and `interrupt_agent` legacy lifecycle emissions with a success-only `SubAgentActivity` event. The event records the tool call ID, occurrence time, affected thread, canonical agent path, and `started`, `interacted`, or `interrupted` kind. - Expose the activity as a completion-only app-server v2 `subAgentActivity` thread item in live notifications and reconstructed history, regenerate the protocol schemas, and count it in sub-agent tool analytics. - Track canonical paths from live activity and loaded-thread metadata in the TUI, and render the activity in live and replayed transcripts. - Make `/agent` list running path-backed agents with summaries from bounded local event buffers. Each summary is capped at 240 graphemes, the scan is capped at six recent items, only the last three wrapped lines are shown, and command output is omitted. Liveness falls back to metadata-only `thread/read` when local turn state is unavailable. - Persist the activity as a terminal rollout-trace runtime payload and reduce it to the corresponding spawn, send, follow-up, or close interaction edge. `interrupt_agent` is classified as a close-edge operation. - Preserve the legacy picker when no path-backed v2 agent is known. ## Compatibility App-server v2 clients that consumed `collabAgentToolCall` begin/end pairs for these tools must handle the new completion-only `subAgentActivity` item. Legacy v1 collaboration behavior is unchanged. ## Screenshot <img width="684" height="288" alt="Screenshot 2026-06-08 at 15 40 47" src="https://github.com/user-attachments/assets/194b3cd0-619d-45fb-b587-cf3e2b1b8a1d" /> ## Testing - `just test -p codex-app-server-protocol` - `just test -p codex-rollout-trace` - Added focused coverage for activity analytics, terminal trace serialization, spawn-edge reduction, `interrupt_agent` classification, TUI status rendering without aggregated command output, and clearing stale running state after a completed turn.
jif ·
2026-06-09 12:14:48 +02:00 -
[codex] Return workspace directory installed plugins (#27098)
## Summary - return installed `workspace-directory` remote plugins by default in `plugin/installed` - keep shared-with-me installed plugins gated behind `plugin_sharing` - filter remote installed plugin marketplaces by canonical marketplace name instead of coarse workspace scope ## Validation - `just fmt` - `just test -p codex-core-plugins` - `just test -p codex-app-server` - `just fix -p codex-core-plugins` - `just fix -p codex-app-server` - `$xin-build` targeted verification: - `just test -p codex-core-plugins build_remote_installed_plugin_marketplaces_from_cache_filters_by_marketplace_name` - `just test -p codex-app-server plugin_installed_includes_workspace_directory_without_plugin_sharing` - `just test -p codex-app-server plugin_installed_includes_remote_shared_with_me_plugins` - `just test -p codex-app-server plugin_list_omits_shared_with_me_kind_when_plugin_sharing_disabled`
xl-openai ·
2026-06-09 01:23:16 -07:00 -
Use server app auth requirements for remote plugin install (#27085)
## Summary - request `includeAppsNeedingAuth=true` when installing remote plugins - return backend-provided `app_ids_needing_auth` from the remote install client - use those app IDs to populate `appsNeedingAuth` without refetching accessible apps, with fallback for older responses ## Testing - `just fmt` - `just test -p codex-app-server` - `just test -p codex-core-plugins` - real app-server install/uninstall check with Notion remote plugin - subagent review found no blocking issues
xl-openai ·
2026-06-08 21:39:35 -07:00 -
Boyang Niu ·
2026-06-09 00:38:35 +00:00 -
Use cached remote plugin catalog for plugin list (#26932)
## Summary This changes the default remote plugin marketplace listing to use the cached global remote catalog when it is already present on disk. The foreground `plugin/list` response can then return from the local catalog cache instead of waiting on `/ps/plugins/list`. When a cached global catalog was present at the start of the request, `plugin/list` still schedules a background refresh through the existing plugin-list background task path so the disk cache is updated for future requests. Cache misses keep the existing synchronous remote fetch path and write the cache, and they do not schedule an extra duplicate background `/ps/plugins/list` refresh. Installed/enabled state continues to come from the existing remote installed overlay path. This change only affects the global remote catalog directory data used by `plugin/list`. ## Testing - `just fmt` - `just test -p codex-app-server plugin_list_uses_cached_global_remote_catalog_and_refreshes_it` - `just test -p codex-core-plugins` - `git diff --check`
xl-openai ·
2026-06-08 14:47:09 -07:00 -
Route image edits through referenced file paths (#26486)
## Why Image edits should use the exact images selected by the model instead of inferring edit inputs from conversation history. ## What changed - Replaced the image tool's `action` argument with optional `referenced_image_paths`. - Treats omitted or empty references as generation and populated references as editing. - Reads referenced absolute image paths and packages them as image data URLs for the edit request. - Removed the previous history-selection and image-count heuristics. - Updated direct and code-mode tool instructions and calls. - Added an app-server integration test covering an attached image routed to the image edit endpoint. ## Validation - Tested end-to-end on local `just codex` with copy pasted image, attached image, etc. - `just test -p codex-image-generation-extension` - `just test -p codex-app-server standalone_image_edit_uses_attached_model_visible_image` - `just fix -p codex-image-generation-extension` - `just bazel-lock-check`
Won Park ·
2026-06-08 14:23:55 -07:00 -
Pair thread environment settings (#26687)
## Why Thread cwd and environment selections are a single logical setting in core: updating one without the other can silently desynchronize the next-turn execution context. This change makes that relationship explicit in the internal thread settings flow while preserving the existing app-server public API shape. ## What changed - Moved the cwd/environment pair through internal `ThreadSettingsOverrides.environment_settings` instead of a top-level internal `cwd` field. - Kept `thread/settings/update` public params unchanged, with app-server translating top-level `cwd` into the paired internal settings shape. - Moved `Op::UserInput` environment overrides into thread settings so user turns and settings updates use the same core path. - Updated core, app-server, MCP, memories, sample, and test callsites to construct the paired settings shape. ## Verification - `git diff --check` - Local test run starting after PR creation.
pakrym-oai ·
2026-06-08 13:55:15 -07:00 -
[codex] Speed up external agent session imports (#26637)
## Why Importing large external-agent session histories currently starts a full live Codex thread for every imported session. This initializes unrelated runtime systems and repeats expensive transcript, metadata, hashing, and ledger work. On a 50-session, 238 MiB fixture, the existing path took roughly 70 seconds to complete the import and 77 seconds end to end. ## What changed - Persist imported sessions directly through `ThreadStore` instead of starting full live threads. - Process imports through a bounded five-session pipeline. - Parse, extract, and hash each source file in one pass. - Move blocking source preparation onto the blocking thread pool. - Reuse prepared content hashes and update the import ledger once per batch. - Avoid metadata readback for newly written rollouts. - Preserve imported conversation history and visible thread metadata. - Keep the implementation out of `codex-core` and avoid changes to the public `ThreadStore` trait. ## Performance For the same 50-session, 238 MiB fixture: | Path | Import completion | End to end | | --- | ---: | ---: | | Existing import | 69.61s | 76.62s | | This change | 5.95s | 6.58s | All 50 sessions imported successfully with no warnings or contention signals. ## Validation - `just test -p codex-external-agent-sessions` - `just test -p codex-app-server external_agent_config_import` - Verified imports do not initialize unrelated required MCP servers. - Verified previously imported source versions are skipped and changed sources can be imported again. - Verified imported rollouts remain readable through thread listing and history APIs.
stefanstokic-oai ·
2026-06-08 14:16:32 -04:00 -
fix(app-server): avoid blocking connection cleanup (#26852)
## Why Remote-control app-server sessions can reconnect every 5-7 seconds when the shared transport-event queue fills. The queue's consumer handled `ConnectionClosed` by awaiting all in-flight RPCs for the disconnected connection. A stuck RPC therefore blocked processing of replacement connection and initialize events until remote-control forwarding hit its five-second timeout and reconnected again. Related issue: N/A (internal remote-control incident investigation). ## What Changed - Split fast RPC admission closure from draining: `ConnectionRpcGate::close()` rejects queued and future RPCs, while `shutdown()` continues waiting for RPCs that already started. - Close a disconnected connection's RPC gate before spawning the existing RPC drain and resource cleanup in a tracked background task, so the transport-event consumer remains available without waiting for active RPCs. - Reap completed cleanup tasks during normal operation, drain them during graceful shutdown, and abort them during forced shutdown. - Add regression coverage for closing with an active RPC, rejecting post-close requests without polling them, and preserving the existing shutdown wait behavior. ## Verification `just test -p codex-app-server --lib connection_rpc_gate` passes all 6 tests, including the new close-versus-drain regression coverage.
Anton Panasenko ·
2026-06-08 10:20:54 -07:00 -
[codex] Enable standalone web search in code mode (#26719)
## What - Consume plaintext `output` from standalone search while retaining optional `encrypted_output` parsing. - Expose `web.run` to code mode and return search output to nested JavaScript calls. - Cover direct and code-mode standalone search paths with integration tests. ## Why `/v1/alpha/search` now returns plaintext output, which code mode needs to consume standalone search results. ## Test plan - `just test -p codex-api` - `just test -p codex-web-search-extension` - `just test -p codex-core code_mode_can_call_standalone_web_search` - `just test -p codex-app-server standalone_web_search_round_trips_output`
rka-oai ·
2026-06-07 23:18:23 -07:00 -
fix(tui): scope MCP startup status by thread (#26639)
## Why MCP startup failures from spawned subagents were rendered as global notifications, so a child thread's failure could pollute the visible parent transcript. Routing the notification to the child exposed two related replay problems: session refresh could discard the buffered event, and a newly created child `ChatWidget` did not know the expected MCP server set, which could leave its startup spinner running after every server had settled. MCP startup diagnostics should remain visible in the thread that owns the startup without affecting other transcripts. The protocol also needs to support a future app-scoped MCP lifecycle where startup is not owned by any thread. ## Reported Behavior The [originating Slack report](https://openai.slack.com/archives/C08JZTV654K/p1780604538859939) called out that using subagents could turn MCP startup failures into a wall of yellow CLI warnings because repeated failures were not deduplicated. The intended behavior is for those diagnostics to remain visible once in the thread that owns the startup, without polluting the parent transcript. ## What Changed - add nullable `threadId` ownership to `mcpServer/startupStatus/updated` - populate it from the app-server conversation ID for the current thread-scoped lifecycle and regenerate the protocol schema and TypeScript artifacts - treat a missing or null `threadId` as app-scoped without injecting it into the active chat transcript - route and buffer thread-owned MCP startup notifications by thread in the TUI - preserve buffered MCP startup events across child session refresh - seed expected MCP servers before replaying a thread snapshot so startup reaches its terminal state - suppress an identical repeated failure warning for the same server within one startup round The owning thread still renders the detailed failure and final `MCP startup incomplete (...)` summary. ## How to Test 1. Configure an optional MCP server named `smoke` that exits during initialization. 2. Launch the TUI with multi-agent support enabled. 3. Confirm the main thread's own startup failure renders one detailed `smoke` warning and one incomplete-startup summary. 4. Spawn exactly one subagent. 5. Confirm the parent transcript does not receive the subagent's MCP startup failure. 6. Switch to the subagent thread and confirm it contains exactly one detailed `smoke` failure and one incomplete-startup summary. 7. Confirm the subagent's MCP startup spinner disappears and the thread remains usable. 8. Switch between the parent and subagent and confirm the warnings neither move nor duplicate. Targeted tests: - `just test -p codex-app-server-protocol` - `just test -p codex-app-server thread_start_emits_mcp_server_status_updated_notifications` - `just test -p codex-tui mcp_startup` The parent/child behavior and spinner completion were also exercised manually in tmux. `just argument-comment-lint` was attempted but blocked by an unrelated local Bazel LLVM empty-glob failure; touched Rust callsites were inspected manually.
Felipe Coury ·
2026-06-07 20:12:05 -07:00 -
Eric Ning ·
2026-06-06 14:03:00 -07:00 -
permissions: enforce managed permission profile allowlists (#24852)
## Why Permission profile allowlists are an enterprise security boundary, but they also need to compose across the managed requirements layers added in #24620. A map representation lets each requirements layer add, allow, or revoke individual profiles without replacing an entire array. ## Managed Contract Administrators configure the mergeable allow map with `allowed_permission_profiles`. A recommended enterprise configuration explicitly lists every built-in and custom profile users should be able to select: ```toml default_permissions = "review_only" [allowed_permission_profiles] ":read-only" = true ":workspace" = true review_only = true # ":danger-full-access" is intentionally omitted, so it is denied. [permissions.review_only] extends = ":read-only" ``` - Profiles whose effective merged value is `true` are allowed. - Missing profiles and profiles set to `false` are denied. - This is a closed allowlist: built-in profiles and profiles introduced in future versions are denied unless explicitly allowed. - Explicitly list each built-in profile the enterprise wants to make available. Omit built-ins such as `:danger-full-access` when they should remain unavailable. - Set `default_permissions` explicitly to the allowed profile users should receive when they have no local selection. - Higher-precedence layers override only the profile keys they define. - `false` is only needed when a higher-precedence layer must revoke a `true` inherited from a lower layer. - Explicit keys must refer to known built-in or managed profiles. A custom or narrowed allowlist requires an allowed `default_permissions`. For compatibility, if both `:workspace` and `:read-only` are explicitly allowed, an omitted default resolves to `:workspace`; customer configurations should still set the intended default explicitly. When `allowed_permission_profiles` is absent, existing implicit permission and legacy `sandbox_mode` behavior is unchanged. ## What Changed - Add `allowed_permission_profiles` as a `BTreeMap<String, bool>` that merges per profile across requirements layers. - Enforce managed defaults, strict denial of omitted profiles, and the explicitly allowed standard-pair fallback. - Expose `allowedPermissionProfiles` through `configRequirements/read` and regenerate its schemas. - Add regression coverage for map composition and revocation, managed defaults, strict denial of omitted built-ins, and API output. ## Verification - Focused `codex-config` coverage for layered map composition and revocation - Focused `codex-core` coverage for managed defaults, invalid defaults, strict denial of omitted built-ins, and the standard built-in pair - Focused `codex-app-server` coverage for requirements API output - Scoped Clippy for `codex-config`, `codex-core`, `codex-app-server-protocol`, and `codex-app-server` ## Documentation The managed `requirements.toml` documentation should introduce `allowed_permission_profiles` as a closed permission-profile allowlist before this setting is published on developers.openai.com. --------- Co-authored-by: Codex <noreply@openai.com>
viyatb-oai ·
2026-06-05 18:06:29 -07:00 -
[codex-rs] support v2 personal access tokens (#25731)
## Summary - add v2 personal access token support for `codex login --with-access-token` and `CODEX_ACCESS_TOKEN` - classify opaque `at-` tokens separately from legacy Agent Identity JWTs - hydrate required ChatGPT account metadata through AuthAPI `/v1/user-auth-credential/whoami` - use PATs directly as bearer tokens while preserving existing ChatGPT account surfaces - expose PAT-backed auth as the explicit `personalAccessToken` app-server auth mode ## Implementation PAT auth is intentionally small and stateless. Loading a PAT performs one AuthAPI metadata request, stores the hydrated metadata in the in-memory auth object, and redacts the secret from debug output. Legacy Agent Identity JWT handling remains unchanged. The shared access-token classifier lives in a private neutral module because it dispatches between both credential types. PAT hydration fails closed when AuthAPI omits any required metadata, including email. Hydrated metadata is intentionally not persisted: startup performs a live `whoami` preflight so revoked tokens or changed account metadata are not accepted from a stale cache. ## Workspace restriction scope This change intentionally does **not** apply `forced_chatgpt_workspace_id` to PAT authentication. The setting is a client-side config guardrail, not an authorization boundary, and PAT does not currently require workspace-ID parity. The PAT login and `CODEX_ACCESS_TOKEN` paths therefore validate through AuthAPI without threading workspace-restriction state through access-token loading. Existing workspace checks for non-PAT auth remain on their established paths. ## App-server compatibility The public app-server `AuthMode` is shared across v1 and v2, and PAT-backed auth reports `personalAccessToken` through both APIs. Following human review, this intentionally removes the temporary v1 compatibility mapping that reported PATs as `chatgpt`; the deprecated v1 API is kept in parity with v2 rather than maintaining a separate closed enum. Clients with exhaustive auth-mode handling in either API version must add the new case and should generally treat it as ChatGPT-backed unless they need PAT-specific behavior. The v1 auth-status response still omits the raw PAT when `includeToken` is requested because that response cannot carry the account metadata needed to reuse the credential safely. Persisted PAT auth also omits the new enum value so older Codex builds can deserialize `auth.json` and infer PAT auth from the credential field after a rollback. ## Validation Latest review-fix validation: - `CARGO_INCREMENTAL=0 just test -p codex-login` (126 passed) - `CARGO_INCREMENTAL=0 just test -p codex-cli` (263 passed) - `CARGO_INCREMENTAL=0 just test -p codex-cli stored_auth_validation_handles_personal_access_token` - `CARGO_INCREMENTAL=0 just test -p codex-app-server-protocol` (226 passed) - `CARGO_INCREMENTAL=0 just test -p codex-models-manager refresh_available_models_uses_remote_only_catalog_for_chatgpt_auth` - `CARGO_INCREMENTAL=0 just test -p codex-tui existing_non_oauth_chatgpt_login_counts_as_signed_in` - `CARGO_INCREMENTAL=0 just fix -p codex-login -p codex-app-server-protocol -p codex-models-manager -p codex-tui -p codex-cli` - `just fmt` - `git diff --check` The broader `codex-tui` suite previously compiled and ran 2,834 tests. Three unrelated environment-sensitive guardian/IDE-socket tests failed after retries; the PAT-relevant TUI coverage passed.
cooper-oai ·
2026-06-05 17:36:18 -07:00 -
[codex] Remove legacy remote plugin startup sync (#25936)
## Summary - Remove the legacy startup remote plugin sync path that called `/plugins/list` and reconciled curated plugin cache/config. - Remove the `sync_plugins_from_remote` API, its result/error types, startup marker task, and tests that expected the legacy request. - Keep the current remote installed bundle sync and remote catalog flows (`/ps/plugins/installed` and `/ps/plugins/list`) intact. ## Validation - `just fmt` - `git diff --check` - `env HOME=/private/tmp/codex-xin-build-home USERPROFILE=/private/tmp/codex-xin-build-home just test -p codex-core-plugins` - Searched for legacy `/plugins/list` sync references; remaining matches are `/ps/plugins/list` catalog tests/code. ## Notes - `just test -p codex-app-server plugin_list` is currently blocked before running filtered tests by an unrelated compile error in `app-server/tests/suite/v2/image_generation.rs`: `app_test_support::McpProcess` is not exported.
xl-openai ·
2026-06-05 16:33:01 -07:00 -
[2 of 2] Finish moving goal runtime to extension (#26548)
## Stack 1. [#26547](https://github.com/openai/codex/pull/26547) - [1 of 2] Align goal extension with core behavior 2. [#26548](https://github.com/openai/codex/pull/26548) - [2 of 2] Move goal runtime to extension ## Why This PR completes the switch of the goal behavior to the extension-backed runtime and removes the old core goal implementation. ## What Changed - Installs the goal extension for app-server `ThreadManager` sessions. - Routes app-server thread goal `get`, `set`, and `clear` through `GoalService`. - Uses thread-idle lifecycle emission after goal resume and snapshot ordering so the extension can decide whether to continue the goal. - Forwards extension goal updates through a FIFO async app-server notification path so backpressure does not drop them or reorder updates. - Keeps review turns from enabling goal runtime behavior. - Plans extension tools before dynamic tools so built-in goal tool names keep their old precedence when goals are enabled. - Removes the old core goal runtime, core goal tool handlers, and core goal tool specs. - Updates tests that were coupled to the core-owned goal runtime while leaving the legacy `<goal_context>` compatibility path in core for old threads. - Removes the stale cargo-shear ignore now that `codex-goal-extension` is used by the workspace. - Keeps realtime event matching exhaustive after removing the old goal-specific realtime text path. ## Validation - Ran manual `/goal` runs in TUI. Validated time accounting matched wall-clock time and goal lifecycle state transitions.
Eric Traut ·
2026-06-05 14:17:30 -07:00 -
Speed up TUI startup by reusing plugin discovery (#26469)
## Summary TUI startup loads related plugin data from `hooks/list`, session MCP initialization, and plugin skill warmup. These paths repeated filesystem discovery and emitted the same plugin warnings, while `hooks/list` and account/model bootstrap ran serially. This change: - Reuses one immutable plugin load outcome across startup consumers. - Keys the cache only on plugin-relevant configuration. - Single-flights concurrent plugin loads and prevents invalidated loads from repopulating the cache. - Runs hook discovery and account/model bootstrap concurrently. - Preserves configuration-migration ordering, hook review behavior, and accurate startup telemetry. In 10 alternating release-build launches in the Ruff repository with the existing `~/.codex` configuration, median time to the first editable composer decreased from 833ms to 504ms. The branch was faster in 9 of 10 pairs, with a paired median improvement of 312ms.
Charlie Marsh ·
2026-06-05 15:32:43 -04:00 -
Make runtime workspace roots absolute in app-server API (#26552)
Stacked on #26532. ## Why #26532 moves cwd normalization to the app-server/core boundary. `runtimeWorkspaceRoots` still accepted raw paths in v2 requests and in `ConfigOverrides`, which left core responsible for interpreting those roots later. This makes runtime workspace roots follow the same absolute-path boundary as cwd. ## What - Change v2 `runtimeWorkspaceRoots` request fields for `thread/start`, `thread/resume`, `thread/fork`, and `turn/start` to `AbsolutePathBuf`. - Deduplicate already-absolute runtime roots in app-server handlers and pass them through `ConfigOverrides.workspace_roots` as `AbsolutePathBuf`. - Update TUI and exec client request builders to pass absolute runtime roots directly. - Update app-server docs, schema fixtures, and focused tests for absolute runtime roots. ## Testing - `just test -p codex-app-server-protocol` - `just test -p codex-app-server runtime_workspace_roots` - `just test -p codex-core session_permission_profile_rebinds_runtime_workspace_roots` - `just test -p codex-tui app_server_session` - `just test -p codex-exec`
pakrym-oai ·
2026-06-05 11:36:53 -07:00 -
[codex] Add turn profiling analytics (#26484)
## Summary Add flat profiling fields to `codex_turn_event` so analytics can explain where turn wall-clock time is spent without changing tool execution behavior. The profile reports: - time before the first sampling request - sampling time across all attempts and follow-ups - overhead between sampling requests - time blocked in the post-sampling tool drain - time after the final sampling request - sampling request and retry counts ## Implementation - Extend the existing turn timing state with constant-memory phase accounting and one RAII phase guard. - Observe sampling and the existing post-sampling drain only at turn orchestration boundaries. - Keep tool runtime, tool futures, response item handling, and turn lifecycle values unchanged. - Add the profiling fields directly to the existing analytics turn event without changing app-server protocol or rollout persistence. - Use the existing turn `status` to distinguish completed, failed, and interrupted profiles. Exact sampling/tool overlap is intentionally omitted because measuring tool completion accurately would require hooks in the tool execution path. ## Validation - Add app-server end-to-end coverage for a single-sampling turn with no blocking tool work. - Add app-server end-to-end coverage for `request_user_input` blocking followed by a second sampling request. - CI is running on the PR; tests were not executed locally per repository guidance.
Ahmed Ibrahim ·
2026-06-05 11:27:10 -07:00