mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
e2398d0b1667f0ae71ecb4f86ceb9fcc88b315e4
9 Commits
-
Cache plugin namespace during executor skill discovery (#29831)
## Why Executor skill discovery runs before the remote skills catalog is available. For a remote environment, each `ExecutorFileSystem` operation becomes an exec-server RPC. Previously, every discovered `SKILL.md` independently resolved its plugin namespace by walking its ancestors and probing both supported manifest locations. In the common `plugin/skills/<skill>/SKILL.md` layout, that repeats 8 RPCs per skill even though every skill under the plugin root uses the same namespace. These lookups happen while skills are parsed, so their cost grows linearly with the skill count and adds directly to first-turn latency. A selected capability root can also contain standalone skills, multiple sibling plugins, nested plugins, or symlinked directories. The optimization therefore needs to retain the nearest-ancestor namespace for each skill rather than assuming the selected root represents exactly one plugin. ## What changed - record plugin-root candidates from directory entries already returned during skill discovery - prune candidates that are not ancestors of any discovered `SKILL.md` before reading manifests - resolve each relevant plugin root once, with one fallback lookup per canonical traversal root for symlinked directories - select the nearest cached plugin namespace for each discovered skill - avoid namespace lookup entirely when the root contains no skills No additional directory traversal is required. Namespace work now scales with the number of plugin roots that contain discovered skills, rather than the total number of skills or unrelated sibling plugins. Standalone and nested-plugin names keep their previous behavior. ## Benchmarks I used a temporary counting `ExecutorFileSystem` around the real local filesystem. Each filesystem operation was counted as one remote RPC and given 1 ms of injected latency. Each variant ran three times; times below are medians. ### One plugin with 100 skills | Operation | Before | After | Delta | | --- | ---: | ---: | ---: | | `get_metadata` | 1,002 | 303 | -699 | | `read_file` | 200 | 101 | -99 | | `read_directory` | 102 | 102 | 0 | | **Total filesystem RPCs** | **1,304** | **506** | **-798 (-61.2%)** | | **Median load time** | **2.890 s** | **0.997 s** | **2.90× faster** | The namespace-specific work drops from 800 RPCs to 2 in this layout. ### Multiple plugins under one selected root These runs compare the correct pre-optimization implementation with the final nearest-plugin-root cache. The total plugin skill count stays at 100 while the number of plugin roots changes. | Layout | Before RPCs | After RPCs | Reduction | Before | After | Speedup | | --- | ---: | ---: | ---: | ---: | ---: | ---: | | 2 plugins × 50 skills | 1,312 | 530 | 59.6% | 1,819 ms | 711 ms | 2.56× | | 10 plugins × 10 skills | 1,344 | 578 | 57.0% | 1,850 ms | 778 ms | 2.38× | | 50 plugins × 2 skills | 1,504 | 818 | 45.6% | 2,094 ms | 1,086 ms | 1.93× | | 10 plugins × 10 skills + 10 standalone skills | 1,596 | 630 | 60.5% | 2,209 ms | 860 ms | 2.57× | The remaining cost grows with the number of relevant plugin manifests. Each relevant manifest is read once instead of once per skill, while sibling plugins with no discovered skills are not read. Absolute latency savings depend on the executor's real RPC latency. ## Tests - `just test -p codex-core-skills` (109 passed across the library and integration-test binaries) - one integration test covers standalone, outer-plugin, nested-plugin, and unused sibling-plugin layouts, and asserts the exact set of manifests read
jif ·
2026-06-24 17:14:34 +01:00 -
Load executor skills without host path conversion (#29626)
## Why After #28918, selected skill roots are `PathUri`, but the executor skill provider still converts them to the app-server host's `AbsolutePathBuf`. A foreign Windows root therefore cannot be discovered by a Unix host, and the inverse has the same problem. This PR keeps executor skill discovery and reads on the filesystem that owns the selected root while reusing the existing skill rules. ## What changed - Generalize the existing skill traversal to operate on `PathUri` through `ExecutorFileSystem`, preserving its depth, directory, symlink, and sibling-metadata concurrency behavior. - Add a small environment skill loader that reuses the shared discovery, frontmatter validation, dependency parsing, product policy, and prompt-visibility rules. - Keep the environment id and entrypoint `PathUri` in the skill catalog, then route `skills.read` back through the same environment filesystem. - Preserve the executor's path convention when deriving catalog handles, including literal backslashes in POSIX filenames. - Resolve plugin namespaces from nearby manifests through URI-native filesystem reads. - Cover foreign Windows roots, executor-owned reads, namespaces, metadata, policy, and path identity. ```text selected root (PathUri) | v shared discovery over ExecutorFileSystem | v environment-bound catalog entry --skills.read--> same ExecutorFileSystem ``` No second filesystem abstraction or duplicate traversal implementation is introduced. ## Stack 1. #29614 — add lexical `PathUri` containment. 2. #29620 — share URI-native manifest path resolution. 3. #28918 — keep selected plugin roots and resources URI-native. 4. **This PR** — load executor skills without host path conversion. 5. #29628 — resolve executor MCP working directories without host path conversion.
jif ·
2026-06-23 23:26:06 +01:00 -
[codex] Pass plugin namespace into skill loading (#28608)
## What changed - retain the parsed plugin manifest namespace on loaded plugins - carry that namespace through `PluginSkillRoot` and `SkillRoot` - use the provided namespace when qualifying plugin skill names - include the namespace in the skills cache key ## Why Plugin loading has already parsed `plugin.json`, but skill parsing currently walks every `SKILL.md` ancestor and probes/reads the manifest again to reconstruct the same namespace. Passing the parsed namespace removes those repeated filesystem calls, which are particularly costly on remote filesystems. Context: https://openai.slack.com/archives/C0ARA9GF5D4/p1781639496496439?thread_ts=1781202444.891669&cid=C0ARA9GF5D4 ## Impact Plugin skill names remain unchanged. A regression test uses a deliberately different on-disk manifest name to verify that plugin roots use the provided parsed namespace. ## Validation - `just test -p codex-core-skills -p codex-core-plugins -p codex-plugin -p codex-utils-plugins` (352 passed) - `just fix -p codex-core-skills -p codex-core-plugins -p codex-plugin -p codex-utils-plugins` - `just fmt`
Matthew Zeng ·
2026-06-18 00:16:46 -07:00 -
Add executor-owned plugin resolution (#27692)
## Why CCA can select a capability root that lives in an executor environment, but Codex only had a host-filesystem plugin loader. Before selected executor plugins can contribute MCP servers, we need a small package boundary that can answer: > Does this selected root contain a plugin, and if so, what does its manifest > declare? The answer must come from the selected environment's filesystem. A failed executor lookup must never fall back to the orchestrator filesystem. ## What this changes This PR introduces: ```rust PluginProvider::resolve(root) -> Result<Option<ResolvedPlugin>, Error> ``` `ExecutorPluginProvider` resolves one `SelectedCapabilityRoot` through its exact `environment_id`. It checks the recognized manifest locations, reads the manifest through that environment's `ExecutorFileSystem`, and returns an inert `ResolvedPlugin` containing: - the opaque selected-root ID; - the environment-bound plugin root; - the authority-bound manifest resource; - parsed metadata and authority-bound component locators. Descriptor construction rejects manifest or component paths outside the selected package root, so consumers cannot accidentally lose the package boundary when they receive a resolved plugin. If the root has no plugin manifest, resolution returns `None`, allowing the caller to treat it as a standalone capability such as a skill. ```text selected root: repo -> env-1:/workspace/repo | | env-1 filesystem only v .codex-plugin/plugin.json | v ResolvedPlugin { authority, root, manifest } ``` The existing host loader and the new executor provider now share the same manifest parser. Existing `codex-core-plugins::manifest` type paths remain available through re-exports, so host behavior and callers are unchanged. ## Scope This is intentionally a non-user-visible package-resolution PR. It does not: - parse or register plugin MCP server configurations; - activate skills, connectors, hooks, or MCP servers; - change app-server wiring; - introduce host fallback, caching, or lifecycle behavior. #27670 has merged, and this PR is now based directly on `main`. Together with the resolved MCP catalog from #27634, it establishes the inputs needed for the executor stdio MCP vertical without changing the existing MCP runtime. ## Follow-up The next PR will consume `ResolvedPlugin`, read its declared/default MCP config through the same executor filesystem, bind supported stdio servers to that environment, and feed those registrations into the resolved MCP catalog. An app-server E2E will prove that selecting an executor plugin exposes and invokes its tool on the owning executor. Resume/fork semantics, dynamic environment replacement, and non-stdio placement remain separate lifecycle decisions. ## Validation - `just fmt` - `cargo check --tests -p codex-plugin -p codex-core-plugins` - `just bazel-lock-check` - `git diff --check` Test targets were compiled but not executed locally; CI will run the test and Clippy suites.jif ·
2026-06-12 13:37:33 +02:00 -
fix: Allow plugin skills to share plugin-level icon assets (#23776)
Thread the plugin root through plugin skill loading so skill interface icons can reference shared plugin assets, such as ../../assets/logo.svg.
xl-openai ·
2026-05-21 16:11:59 -07:00 -
Add plugin ID to skill analytics (#20923)
## Summary - thread plugin skill roots through the skills loader with their plugin ID - store plugin ID on loaded skill metadata for plugin-provided skills - include plugin ID on skill invocation analytics events ## Test plan - cargo check -p codex-core-skills - cargo check -p codex-core -p codex-core-plugins -p codex-analytics - cargo check -p codex-tui - cargo check -p codex-plugin -p codex-core -p codex-core-plugins -p codex-analytics - cargo check -p codex-app-server - cargo test -p codex-analytics - HOME=/private/tmp/codex-empty-home cargo test -p codex-core-skills - just fix -p codex-core-skills - just fix -p codex-analytics - just fix -p codex-core-plugins - just fix -p codex-core - just fmt - git diff --check
alexsong-oai ·
2026-05-04 20:36:29 -07:00 -
feat: Handle alternate plugin manifest paths (#18182)
Load plugin manifests through a shared discoverable-path helper so manifest reads, installs, and skill names all see the same alternate manifest location.
xl-openai ·
2026-04-16 19:43:19 -07:00 -
Extract MCP into codex-mcp crate (#15919)
- Split MCP runtime/server code out of `codex-core` into the new `codex-mcp` crate. New/moved public structs/types include `McpConfig`, `McpConnectionManager`, `ToolInfo`, `ToolPluginProvenance`, `CodexAppsToolsCacheKey`, and the `McpManager` API (`codex_mcp::mcp::McpManager` plus the `codex_core::mcp::McpManager` wrapper/shim). New/moved functions include `with_codex_apps_mcp`, `configured_mcp_servers`, `effective_mcp_servers`, `collect_mcp_snapshot`, `collect_mcp_snapshot_from_manager`, `qualified_mcp_tool_name_prefix`, and the MCP auth/skill-dependency helpers. Why: this creates a focused MCP crate boundary and shrinks `codex-core` without forcing every consumer to migrate in the same PR. - Move MCP server config schema and persistence into `codex-config`. New/moved structs/enums include `AppToolApproval`, `McpServerToolConfig`, `McpServerConfig`, `RawMcpServerConfig`, `McpServerTransportConfig`, `McpServerDisabledReason`, and `codex_config::ConfigEditsBuilder`. New/moved functions include `load_global_mcp_servers` and `ConfigEditsBuilder::replace_mcp_servers`/`apply`. Why: MCP TOML parsing/editing is config ownership, and this keeps config validation/round-tripping (including per-tool approval overrides and inline bearer-token rejection) in the config crate instead of `codex-core`. - Rewire `codex-core`, app-server, and plugin call sites onto the new crates. Updated `Config::to_mcp_config(&self, plugins_manager)`, `codex-rs/core/src/mcp.rs`, `codex-rs/core/src/connectors.rs`, `codex-rs/core/src/codex.rs`, `CodexMessageProcessor::list_mcp_server_status_task`, and `utils/plugins/src/mcp_connector.rs` to build/pass the new MCP config/runtime types. Why: plugin-provided MCP servers still merge with user-configured servers, and runtime auth (`CodexAuth`) is threaded into `with_codex_apps_mcp` / `collect_mcp_snapshot` explicitly so `McpConfig` stays config-only.
Ahmed Ibrahim ·
2026-04-01 19:03:26 -07:00 -
Extract codex-utils-plugins crate (#15746)
## Summary - extract shared plugin path and manifest helpers into codex-utils-plugins - update codex-core to consume the utility crate ## Testing - CI --------- Co-authored-by: Codex <noreply@openai.com>
Ahmed Ibrahim ·
2026-03-25 11:05:35 -07:00