Load selected executor skills through extensions (#27184)

## Why

CCA is moving toward a split runtime where the orchestrator may not have
a filesystem, while executors can expose preinstalled plugins and
skills. A thread therefore needs to select capabilities without asking
app-server or core to interpret executor-owned paths through the
orchestrator's filesystem.

The longer-term model is broader than executor skills:

- A plugin is a bundle of skills, MCP servers, connectors/apps, and
hooks.
- A plugin root can be local, executor-owned, or hosted by a backend.
- Components inside one plugin can use different access and execution
mechanisms. A skill may be read from a filesystem or through backend
tools; an HTTP MCP server can run without an executor; a stdio MCP
server or hook needs an execution environment.
- Core should carry generic extension initialization data. The extension
that owns a component should discover it, expose it to the model, and
invoke it through the appropriate runtime.

This PR establishes that architecture through one complete vertical:
selecting a root on an executor, discovering the skills beneath it,
exposing those skills to the model, and reading an explicitly invoked
`SKILL.md` through the same executor.

## Contract

`thread/start` gains an experimental `selectedCapabilityRoots` field:

```json
{
  "selectedCapabilityRoots": [
    {
      "id": "deploy-plugin@1",
      "location": {
        "type": "environment",
        "environmentId": "workspace",
        "path": "/opt/codex/plugins/deploy"
      }
    }
  ]
}
```

The root is intentionally not classified as a "plugin" or "skill" in the
API. It can point at a standalone skill, a directory containing several
skills, or a plugin containing skills and other components. This PR only
teaches the skills extension how to consume it; later extensions can
resolve MCP, connector, and hook components from the same selection.

The platform-supplied `id` is stable selection identity. The location
says which runtime owns the root and gives that runtime an opaque path.
App-server does not inspect or canonicalize the path.

## What changed

### Generic thread extension initialization

App-server converts selected roots into `ExtensionDataInit`. Core
carries that generic initialization value until the final thread ID is
known, then creates thread-scoped `ExtensionData` before lifecycle
contributors run.

This keeps `Session` and core independent of the capability-selection
contract. The initialization value is consumed during construction; it
is not retained as another long-lived `Session` field.

### Executor-backed skills

The skills extension now owns an `ExecutorSkillProvider` that:

- resolves the selected environment through `EnvironmentManager`
- discovers, canonicalizes, and reads skills through that environment's
`ExecutorFileSystem`
- contributes the bounded selected-skill catalog as stable developer
context
- reads an explicitly invoked skill body through the authority that
listed it
- warns when an environment or root is unavailable
- never falls back to the orchestrator filesystem for an executor-owned
root

Skill catalog and instruction fragments have hard byte bounds, which
also bound them below the 10K-token per-item context limit. If a
selected executor skill has the same name as a legacy local skill, the
executor selection owns that invocation and the local body is not
injected a second time.

Existing local and bundled skill loading remains in place. Omitting
`selectedCapabilityRoots` therefore preserves current local-only
behavior.

## Current semantics

- Only environment-owned locations are represented in this first
contract.
- Roots are resolved by the destination extension, not by app-server or
core.
- An unavailable executor or invalid root produces a warning and no
capabilities from that root; it does not trigger a local-filesystem
fallback.
- Selection applies to a newly started active thread.
- MCP servers, connectors, and hooks beneath a selected plugin root are
not activated yet.
- Selection is not yet persisted or inherited across resume, fork, or
subagent creation. Existing local capabilities continue to behave as
they do today in those flows.

## Planned vertical follow-ups

1. **Hosted HTTP MCP:** add an extension-backed HTTP MCP source that
works without an executor, then replace the special-purpose MCP plugins
loader with that implementation.
2. **Executor MCP:** register and execute stdio MCP servers through the
environment that owns the selected plugin root.
3. **Backend skills:** add a hosted skill source whose catalog and
bodies are accessed through extension tools rather than a filesystem.
4. **Connectors and hooks:** activate those components through their
owning extensions, using the same selected-root boundary and
component-specific runtime.
5. **Durable selection:** define the desired-selection lifecycle,
persist it, and make resume, fork, and subagent inheritance explicit
rather than accidental.
6. **Local convergence:** incrementally route existing local plugin,
skill, and MCP loading through the same extension model while preserving
current local behavior.

Each follow-up remains reviewable as an end-to-end capability. The
platform selects roots, generic thread extension data carries the
selection, and the owning extension resolves and operates its component.

## Verification

Coverage added for:

- app-server end-to-end discovery and explicit invocation of a skill
inside an executor-selected plugin root
- exclusive invocation when a selected executor skill collides with a
local skill name
- executor filesystem authority for discovery, canonicalization, and
reads
- thread extension initialization before lifecycle contributors run
- stable executor catalog context, explicit invocation, context
rebuilding, hidden skills, and preserved host/remote catalog behavior

Targeted protocol, core-skills, skills-extension, core lifecycle, and
app-server executor-skill tests were run during development.
This commit is contained in:
jif
2026-06-09 19:51:54 +02:00
committed by GitHub
Unverified
parent 1026e9de1b
commit 89ac3ec27c
46 changed files with 1460 additions and 127 deletions
+4
View File
@@ -1994,6 +1994,7 @@ dependencies = [
"codex-rollout",
"codex-sandboxing",
"codex-shell-command",
"codex-skills-extension",
"codex-state",
"codex-thread-store",
"codex-tools",
@@ -3796,8 +3797,11 @@ dependencies = [
"async-trait",
"codex-core",
"codex-core-skills",
"codex-exec-server",
"codex-extension-api",
"codex-protocol",
"codex-utils-absolute-path",
"codex-utils-string",
"pretty_assertions",
"tokio",
]
+1
View File
@@ -212,6 +212,7 @@ codex-sandboxing = { path = "sandboxing" }
codex-secrets = { path = "secrets" }
codex-shell-command = { path = "shell-command" }
codex-shell-escalation = { path = "shell-escalation" }
codex-skills-extension = { path = "ext/skills" }
codex-skills = { path = "skills" }
codex-state = { path = "state" }
codex-stdio-to-uds = { path = "stdio-to-uds" }
@@ -180,6 +180,36 @@
],
"type": "object"
},
"CapabilityRootLocation": {
"description": "Location used to resolve a selected capability root.",
"oneOf": [
{
"description": "A path owned by an execution environment.",
"properties": {
"environmentId": {
"type": "string"
},
"path": {
"type": "string"
},
"type": {
"enum": [
"environment"
],
"title": "EnvironmentCapabilityRootLocationType",
"type": "string"
}
},
"required": [
"environmentId",
"path",
"type"
],
"title": "EnvironmentCapabilityRootLocation",
"type": "object"
}
]
},
"ClientInfo": {
"properties": {
"name": {
@@ -2947,6 +2977,28 @@
}
]
},
"SelectedCapabilityRoot": {
"description": "A user-selected root that can expose one or more runtime capabilities.",
"properties": {
"id": {
"description": "Stable identifier supplied by the capability selection platform.",
"type": "string"
},
"location": {
"allOf": [
{
"$ref": "#/definitions/CapabilityRootLocation"
}
],
"description": "Where the selected root can be resolved."
}
},
"required": [
"id",
"location"
],
"type": "object"
},
"SendAddCreditsNudgeEmailParams": {
"properties": {
"creditType": {
@@ -6765,6 +6765,36 @@
],
"type": "string"
},
"CapabilityRootLocation": {
"description": "Location used to resolve a selected capability root.",
"oneOf": [
{
"description": "A path owned by an execution environment.",
"properties": {
"environmentId": {
"type": "string"
},
"path": {
"type": "string"
},
"type": {
"enum": [
"environment"
],
"title": "EnvironmentCapabilityRootLocationType",
"type": "string"
}
},
"required": [
"environmentId",
"path",
"type"
],
"title": "EnvironmentCapabilityRootLocation",
"type": "object"
}
]
},
"CodexErrorInfo": {
"description": "This translation layer make sure that we expose codex error code in camel case.\n\nWhen an upstream HTTP status is available (for example, from the Responses API or a provider), it is forwarded in `httpStatusCode` on the relevant `codexErrorInfo` variant.",
"oneOf": [
@@ -15061,6 +15091,28 @@
},
"type": "object"
},
"SelectedCapabilityRoot": {
"description": "A user-selected root that can expose one or more runtime capabilities.",
"properties": {
"id": {
"description": "Stable identifier supplied by the capability selection platform.",
"type": "string"
},
"location": {
"allOf": [
{
"$ref": "#/definitions/v2/CapabilityRootLocation"
}
],
"description": "Where the selected root can be resolved."
}
},
"required": [
"id",
"location"
],
"type": "object"
},
"SendAddCreditsNudgeEmailParams": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
@@ -1087,6 +1087,36 @@
],
"type": "string"
},
"CapabilityRootLocation": {
"description": "Location used to resolve a selected capability root.",
"oneOf": [
{
"description": "A path owned by an execution environment.",
"properties": {
"environmentId": {
"type": "string"
},
"path": {
"type": "string"
},
"type": {
"enum": [
"environment"
],
"title": "EnvironmentCapabilityRootLocationType",
"type": "string"
}
},
"required": [
"environmentId",
"path",
"type"
],
"title": "EnvironmentCapabilityRootLocation",
"type": "object"
}
]
},
"ClientInfo": {
"properties": {
"name": {
@@ -11563,6 +11593,28 @@
},
"type": "object"
},
"SelectedCapabilityRoot": {
"description": "A user-selected root that can expose one or more runtime capabilities.",
"properties": {
"id": {
"description": "Stable identifier supplied by the capability selection platform.",
"type": "string"
},
"location": {
"allOf": [
{
"$ref": "#/definitions/CapabilityRootLocation"
}
],
"description": "Where the selected root can be resolved."
}
},
"required": [
"id",
"location"
],
"type": "object"
},
"SendAddCreditsNudgeEmailParams": {
"$schema": "http://json-schema.org/draft-07/schema#",
"properties": {
@@ -64,6 +64,36 @@
}
]
},
"CapabilityRootLocation": {
"description": "Location used to resolve a selected capability root.",
"oneOf": [
{
"description": "A path owned by an execution environment.",
"properties": {
"environmentId": {
"type": "string"
},
"path": {
"type": "string"
},
"type": {
"enum": [
"environment"
],
"title": "EnvironmentCapabilityRootLocationType",
"type": "string"
}
},
"required": [
"environmentId",
"path",
"type"
],
"title": "EnvironmentCapabilityRootLocation",
"type": "object"
}
]
},
"DynamicToolSpec": {
"properties": {
"deferLoading": {
@@ -106,6 +136,28 @@
],
"type": "string"
},
"SelectedCapabilityRoot": {
"description": "A user-selected root that can expose one or more runtime capabilities.",
"properties": {
"id": {
"description": "Stable identifier supplied by the capability selection platform.",
"type": "string"
},
"location": {
"allOf": [
{
"$ref": "#/definitions/CapabilityRootLocation"
}
],
"description": "Where the selected root can be resolved."
}
},
"required": [
"id",
"location"
],
"type": "object"
},
"ThreadSource": {
"enum": [
"user",
@@ -0,0 +1,8 @@
// GENERATED CODE! DO NOT MODIFY BY HAND!
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
/**
* Location used to resolve a selected capability root.
*/
export type CapabilityRootLocation = { "type": "environment", environmentId: string, path: string, };
@@ -0,0 +1,17 @@
// GENERATED CODE! DO NOT MODIFY BY HAND!
// This file was generated by [ts-rs](https://github.com/Aleph-Alpha/ts-rs). Do not edit this file manually.
import type { CapabilityRootLocation } from "./CapabilityRootLocation";
/**
* A user-selected root that can expose one or more runtime capabilities.
*/
export type SelectedCapabilityRoot = {
/**
* Stable identifier supplied by the capability selection platform.
*/
id: string,
/**
* Where the selected root can be resolved.
*/
location: CapabilityRootLocation, };
@@ -40,6 +40,7 @@ export type { ByteRange } from "./ByteRange";
export type { CancelLoginAccountParams } from "./CancelLoginAccountParams";
export type { CancelLoginAccountResponse } from "./CancelLoginAccountResponse";
export type { CancelLoginAccountStatus } from "./CancelLoginAccountStatus";
export type { CapabilityRootLocation } from "./CapabilityRootLocation";
export type { ChatgptAuthTokensRefreshParams } from "./ChatgptAuthTokensRefreshParams";
export type { ChatgptAuthTokensRefreshReason } from "./ChatgptAuthTokensRefreshReason";
export type { ChatgptAuthTokensRefreshResponse } from "./ChatgptAuthTokensRefreshResponse";
@@ -334,6 +335,7 @@ export type { ReviewTarget } from "./ReviewTarget";
export type { SandboxMode } from "./SandboxMode";
export type { SandboxPolicy } from "./SandboxPolicy";
export type { SandboxWorkspaceWrite } from "./SandboxWorkspaceWrite";
export type { SelectedCapabilityRoot } from "./SelectedCapabilityRoot";
export type { SendAddCreditsNudgeEmailParams } from "./SendAddCreditsNudgeEmailParams";
export type { SendAddCreditsNudgeEmailResponse } from "./SendAddCreditsNudgeEmailResponse";
export type { ServerRequestResolvedNotification } from "./ServerRequestResolvedNotification";
@@ -11,6 +11,8 @@ use super::TurnEnvironmentParams;
use super::TurnItemsView;
use super::shared::v2_enum_from_core;
use codex_experimental_api_macros::ExperimentalApi;
pub use codex_protocol::capabilities::CapabilityRootLocation;
pub use codex_protocol::capabilities::SelectedCapabilityRoot;
use codex_protocol::config_types::CollaborationMode;
use codex_protocol::config_types::Personality;
use codex_protocol::config_types::ReasoningSummary;
@@ -153,6 +155,10 @@ pub struct ThreadStartParams {
#[experimental("thread/start.dynamicTools")]
#[ts(optional = nullable)]
pub dynamic_tools: Option<Vec<DynamicToolSpec>>,
/// Capability roots selected for this thread by the hosting platform.
#[experimental("thread/start.selectedCapabilityRoots")]
#[ts(optional = nullable)]
pub selected_capability_roots: Option<Vec<SelectedCapabilityRoot>>,
/// Test-only experimental field used to validate experimental gating and
/// schema filtering behavior in a stable way.
#[experimental("thread/start.mockExperimentalField")]
+1
View File
@@ -49,6 +49,7 @@ codex-hooks = { workspace = true }
codex-otel = { workspace = true }
codex-plugin = { workspace = true }
codex-shell-command = { workspace = true }
codex-skills-extension = { workspace = true }
codex-utils-cli = { workspace = true }
codex-utils-pty = { workspace = true }
codex-backend-client = { workspace = true }
+13 -1
View File
@@ -130,7 +130,7 @@ Example with notification opt-out:
## API Overview
- `thread/start` — create a new thread; emits `thread/started` (including the current `thread.status`) and auto-subscribes you to turn/item events for that thread. When the request includes a `cwd` and the resolved sandbox is `workspace-write` or full access, app-server also marks that project as trusted in the user `config.toml`. Pass `sessionStartSource: "clear"` when starting a replacement thread after clearing the current session so `SessionStart` hooks receive `source: "clear"` instead of the default `"startup"`. Experimental `runtimeWorkspaceRoots` replaces the thread-scoped runtime workspace roots used to materialize `:workspace_roots`; paths must be absolute. For permissions, prefer experimental `permissions` profile selection by id; the legacy `sandbox` shorthand is still accepted but cannot be combined with `permissions`. Experimental `environments` selects the sticky execution environments for turns on the thread; omit it to use the server default, pass `[]` to disable environments, or pass explicit environment ids with per-environment `cwd`.
- `thread/start` — create a new thread; emits `thread/started` (including the current `thread.status`) and auto-subscribes you to turn/item events for that thread. When the request includes a `cwd` and the resolved sandbox is `workspace-write` or full access, app-server also marks that project as trusted in the user `config.toml`. Pass `sessionStartSource: "clear"` when starting a replacement thread after clearing the current session so `SessionStart` hooks receive `source: "clear"` instead of the default `"startup"`. Experimental `runtimeWorkspaceRoots` replaces the thread-scoped runtime workspace roots used to materialize `:workspace_roots`; paths must be absolute. For permissions, prefer experimental `permissions` profile selection by id; the legacy `sandbox` shorthand is still accepted but cannot be combined with `permissions`. Experimental `environments` selects the sticky execution environments for turns on the thread; omit it to use the server default, pass `[]` to disable environments, or pass explicit environment ids with per-environment `cwd`. Experimental `selectedCapabilityRoots` selects environment-owned plugin or standalone-skill roots. Skills found below those roots are listed and read through the owning environment; other plugin capabilities are not activated yet.
- `thread/resume` — reopen an existing thread by id so subsequent `turn/start` calls append to it. Accepts the same permission override rules as `thread/start`.
- `thread/fork` — fork an existing thread into a new thread id by copying the stored history; if the source thread is currently mid-turn, the fork records the same interruption marker as `turn/interrupt` instead of inheriting an unmarked partial turn suffix. The returned `thread.forkedFromId` points at the source thread when known. Accepts `ephemeral: true` for an in-memory temporary fork, emits `thread/started` (including the current `thread.status`), and auto-subscribes you to turn/item events for the new thread. Experimental clients can pass `excludeTurns: true` when they plan to page fork history via `thread/turns/list` instead of receiving the full turn array immediately. Accepts the same permission override rules as `thread/start`.
- `thread/start`, `thread/resume`, and `thread/fork` responses include the legacy `sandbox` compatibility projection. Experimental clients can read `runtimeWorkspaceRoots` for the thread-scoped runtime roots and `activePermissionProfile` for the named or implicit built-in profile identity/provenance when known.
@@ -249,6 +249,18 @@ Start a fresh thread when you need a new Codex conversation.
// "permissions": ":workspace"
// Experimental runtime roots for :workspace_roots materialization:
// "runtimeWorkspaceRoots": ["/Users/me/project", "/Users/me/openai"],
// Experimental capability roots selected by the hosting platform:
"selectedCapabilityRoots": [
{
"id": "github@openai",
"location": {
"type": "environment",
"environmentId": "workspace",
// Opaque to app-server; interpreted in the selected environment.
"path": "/opt/cca/plugins/github"
}
}
],
// Do not send both "sandbox" and "permissions".
"personality": "friendly",
"serviceName": "my_app_server_client", // optional metrics tag (`service_name`)
+6
View File
@@ -32,6 +32,7 @@ pub(crate) fn thread_extensions<S>(
state_db: Option<StateDbHandle>,
thread_manager: Weak<ThreadManager>,
goal_service: Arc<GoalService>,
executor_skill_provider: Arc<dyn codex_skills_extension::SkillProvider>,
) -> Arc<ExtensionRegistry<Config>>
where
S: AgentSpawner<StartThreadOptions, Spawned = NewThread, Error = CodexErr> + 'static,
@@ -51,6 +52,11 @@ where
codex_memories_extension::install(&mut builder, codex_otel::global());
codex_web_search_extension::install(&mut builder, auth_manager.clone());
codex_image_generation_extension::install(&mut builder, auth_manager);
codex_skills_extension::install_with_providers(
&mut builder,
codex_skills_extension::SkillProviders::new()
.with_executor_provider(executor_skill_provider),
);
Arc::new(builder.build())
}
+9 -1
View File
@@ -181,12 +181,19 @@ mod tests {
.await
.expect("refresh tests require state db");
let thread_store = thread_store_from_config(&good_config, Some(state_db.clone()));
let environment_manager = Arc::new(EnvironmentManager::default_for_tests());
let executor_skill_provider: Arc<dyn codex_skills_extension::SkillProvider> = Arc::new(
codex_skills_extension::ExecutorSkillProvider::new_with_restriction_product(
Arc::clone(&environment_manager),
SessionSource::Exec.restriction_product(),
),
);
let thread_manager = Arc::new_cyclic(|thread_manager| {
ThreadManager::new(
&good_config,
auth_manager.clone(),
SessionSource::Exec,
Arc::new(EnvironmentManager::default_for_tests()),
Arc::clone(&environment_manager),
thread_extensions(
guardian_agent_spawner(thread_manager.clone()),
Arc::new(NoopExtensionEventSink),
@@ -194,6 +201,7 @@ mod tests {
Some(state_db.clone()),
thread_manager.clone(),
Arc::new(codex_goal_extension::GoalService::new()),
Arc::clone(&executor_skill_provider),
),
/*analytics_events_client*/ None,
Arc::clone(&thread_store),
@@ -307,6 +307,14 @@ impl MessageProcessor {
// resumed, or forked threads to a different persistence backend/root.
let thread_store = codex_core::thread_store_from_config(config.as_ref(), state_db.clone());
let environment_manager_for_requests = Arc::clone(&environment_manager);
let environment_manager_for_extensions = Arc::clone(&environment_manager);
let restriction_product = session_source.restriction_product();
let executor_skill_provider: Arc<dyn codex_skills_extension::SkillProvider> = Arc::new(
codex_skills_extension::ExecutorSkillProvider::new_with_restriction_product(
environment_manager_for_extensions,
restriction_product,
),
);
let goal_service = Arc::new(GoalService::new());
let thread_manager = Arc::new_cyclic(|thread_manager| {
ThreadManager::new(
@@ -321,6 +329,7 @@ impl MessageProcessor {
state_db.clone(),
thread_manager.clone(),
Arc::clone(&goal_service),
Arc::clone(&executor_skill_provider),
),
Some(analytics_events_client.clone()),
Arc::clone(&thread_store),
@@ -1,5 +1,7 @@
use super::*;
use crate::error_code::method_not_found;
use codex_app_server_protocol::SelectedCapabilityRoot;
use codex_extension_api::ExtensionDataInit;
use codex_protocol::models::BUILT_IN_PERMISSION_PROFILE_DANGER_FULL_ACCESS;
use codex_protocol::models::BUILT_IN_PERMISSION_PROFILE_WORKSPACE;
@@ -833,6 +835,7 @@ impl ThreadRequestProcessor {
base_instructions,
developer_instructions,
dynamic_tools,
selected_capability_roots,
mock_experimental_field: _mock_experimental_field,
experimental_raw_events,
personality,
@@ -888,6 +891,7 @@ impl ThreadRequestProcessor {
config,
typesafe_overrides,
dynamic_tools,
selected_capability_roots.unwrap_or_default(),
session_start_source,
thread_source.map(Into::into),
environment_selections,
@@ -960,6 +964,7 @@ impl ThreadRequestProcessor {
config_overrides: Option<HashMap<String, serde_json::Value>>,
typesafe_overrides: ConfigOverrides,
dynamic_tools: Option<Vec<ApiDynamicToolSpec>>,
selected_capability_roots: Vec<SelectedCapabilityRoot>,
session_start_source: Option<codex_app_server_protocol::ThreadStartSource>,
thread_source: Option<codex_protocol::protocol::ThreadSource>,
environments: Option<Vec<TurnEnvironmentSelection>>,
@@ -1061,6 +1066,10 @@ impl ThreadRequestProcessor {
.collect()
};
let core_dynamic_tool_count = core_dynamic_tools.len();
let mut thread_extension_init = ExtensionDataInit::new();
if !selected_capability_roots.is_empty() {
thread_extension_init.insert(selected_capability_roots);
}
let create_thread_started_at = std::time::Instant::now();
let NewThread {
thread_id,
@@ -1083,6 +1092,7 @@ impl ThreadRequestProcessor {
metrics_service_name: service_name,
parent_trace: request_trace,
environments,
thread_extension_init,
})
.instrument(tracing::info_span!(
"app_server.thread_start.create_thread",
@@ -0,0 +1,148 @@
use std::time::Duration;
use anyhow::Result;
use app_test_support::TestAppServer;
use app_test_support::to_response;
use codex_app_server_protocol::CapabilityRootLocation;
use codex_app_server_protocol::JSONRPCResponse;
use codex_app_server_protocol::RequestId;
use codex_app_server_protocol::SelectedCapabilityRoot;
use codex_app_server_protocol::ThreadStartParams;
use codex_app_server_protocol::ThreadStartResponse;
use codex_app_server_protocol::TurnStartParams;
use codex_app_server_protocol::UserInput;
use core_test_support::responses;
use tempfile::TempDir;
use tokio::time::timeout;
const READ_TIMEOUT: Duration = Duration::from_secs(10);
const SKILL_NAME: &str = "demo-plugin:deploy";
const SKILL_MARKER: &str = "EXECUTOR_SKILL_BODY_MARKER";
const LOCAL_SKILL_MARKER: &str = "LOCAL_SKILL_BODY_MARKER";
#[tokio::test]
async fn selected_executor_root_exposes_plugin_skill() -> Result<()> {
let server = responses::start_mock_server().await;
let response_mock = responses::mount_sse_once(
&server,
responses::sse(vec![
responses::ev_response_created("resp-selected"),
responses::ev_assistant_message("msg-selected", "Done"),
responses::ev_completed("resp-selected"),
]),
)
.await;
let codex_home = TempDir::new()?;
std::fs::write(
codex_home.path().join("config.toml"),
format!(
r#"
model = "mock-model"
approval_policy = "never"
sandbox_mode = "read-only"
model_provider = "mock_provider"
[skills]
include_instructions = true
[model_providers.mock_provider]
name = "Mock provider for test"
base_url = "{}/v1"
wire_api = "responses"
request_max_retries = 0
stream_max_retries = 0
"#,
server.uri()
),
)?;
let local_skill_dir = codex_home.path().join("skills/local-deploy");
std::fs::create_dir_all(&local_skill_dir)?;
std::fs::write(
local_skill_dir.join("SKILL.md"),
format!(
"---\nname: {SKILL_NAME}\ndescription: Colliding local skill.\n---\n\n# Local deploy\n\n{LOCAL_SKILL_MARKER}\n"
),
)?;
let plugin_dir = TempDir::new()?;
let manifest_dir = plugin_dir.path().join(".codex-plugin");
let skill_dir = plugin_dir.path().join("skills/deploy");
std::fs::create_dir_all(&manifest_dir)?;
std::fs::create_dir_all(&skill_dir)?;
std::fs::write(
manifest_dir.join("plugin.json"),
r#"{"name":"demo-plugin"}"#,
)?;
std::fs::write(
skill_dir.join("SKILL.md"),
format!(
"---\nname: deploy\ndescription: Deploy through the executor.\n---\n\n# Deploy\n\n{SKILL_MARKER}\n"
),
)?;
let mut app_server = TestAppServer::new(codex_home.path()).await?;
timeout(READ_TIMEOUT, app_server.initialize()).await??;
let request_id = app_server
.send_thread_start_request(ThreadStartParams {
model: Some("mock-model".to_string()),
selected_capability_roots: Some(vec![SelectedCapabilityRoot {
id: "demo-plugin@1".to_string(),
location: CapabilityRootLocation::Environment {
environment_id: "local".to_string(),
path: plugin_dir.path().to_string_lossy().into_owned(),
},
}]),
..Default::default()
})
.await?;
let response: JSONRPCResponse = timeout(
READ_TIMEOUT,
app_server.read_stream_until_response_message(RequestId::Integer(request_id)),
)
.await??;
let ThreadStartResponse { thread, .. } = to_response(response)?;
let request_id = app_server
.send_turn_start_request(TurnStartParams {
thread_id: thread.id,
input: vec![UserInput::Text {
text: format!("Use ${SKILL_NAME}"),
text_elements: Vec::new(),
}],
..Default::default()
})
.await?;
timeout(
READ_TIMEOUT,
app_server.read_stream_until_response_message(RequestId::Integer(request_id)),
)
.await??;
timeout(
READ_TIMEOUT,
app_server.read_stream_until_notification_message("turn/completed"),
)
.await??;
let request = response_mock.single_request();
assert!(
request
.message_input_texts("developer")
.iter()
.any(|text| text.contains(SKILL_NAME))
);
let skill_fragments = request
.message_input_texts("user")
.into_iter()
.filter(|text| text.starts_with("<skill>"))
.collect::<Vec<_>>();
assert_eq!(1, skill_fragments.len());
let skill_fragment = skill_fragments
.first()
.expect("executor skill instructions should be model-visible");
assert!(skill_fragment.contains(&format!("<name>{SKILL_NAME}</name>")));
assert!(skill_fragment.contains(SKILL_MARKER));
assert!(!skill_fragment.contains(LOCAL_SKILL_MARKER));
Ok(())
}
@@ -12,6 +12,7 @@ mod connection_handling_websocket;
#[cfg(unix)]
mod connection_handling_websocket_unix;
mod dynamic_tools;
mod executor_skills;
mod experimental_api;
mod experimental_feature_list;
mod external_agent_config;
@@ -793,6 +793,7 @@ async fn skills_changed_notification_is_emitted_after_skill_change() -> Result<(
thread_source: None,
dynamic_tools: None,
environments: None,
selected_capability_roots: None,
mock_experimental_field: None,
experimental_raw_events: false,
})
+16 -8
View File
@@ -169,8 +169,8 @@ where
let mut file_systems_by_skill_path: HashMap<AbsolutePathBuf, Arc<dyn ExecutorFileSystem>> =
HashMap::new();
for root in roots {
let root_path = canonicalize_for_skill_identity(&root.path);
let fs = root.file_system;
let root_path = canonicalize_for_skill_identity(fs.as_ref(), &root.path).await;
let skills_before_root = outcome.skills.len();
discover_skills_under_root(
fs.as_ref(),
@@ -471,8 +471,13 @@ fn dedupe_skill_roots_by_path(roots: &mut Vec<SkillRoot>) {
roots.retain(|root| seen.insert(root.path.clone()));
}
fn canonicalize_for_skill_identity(path: &AbsolutePathBuf) -> AbsolutePathBuf {
path.canonicalize().unwrap_or_else(|_| path.clone())
async fn canonicalize_for_skill_identity(
fs: &dyn ExecutorFileSystem,
path: &AbsolutePathBuf,
) -> AbsolutePathBuf {
fs.canonicalize(path, /*sandbox*/ None)
.await
.unwrap_or_else(|_| path.clone())
}
async fn discover_skills_under_root(
@@ -483,8 +488,11 @@ async fn discover_skills_under_root(
plugin_root: Option<&AbsolutePathBuf>,
outcome: &mut SkillLoadOutcome,
) {
let root = canonicalize_for_skill_identity(root);
let plugin_root = plugin_root.map(canonicalize_for_skill_identity);
let root = root.clone();
let plugin_root = match plugin_root {
Some(plugin_root) => Some(canonicalize_for_skill_identity(fs, plugin_root).await),
None => None,
};
match fs.get_metadata(&root, /*sandbox*/ None).await {
Ok(metadata) if metadata.is_directory => {}
@@ -557,7 +565,7 @@ async fn discover_skills_under_root(
}
match fs.read_directory(&path, /*sandbox*/ None).await {
Ok(_) => {
let resolved_dir = canonicalize_for_skill_identity(&path);
let resolved_dir = canonicalize_for_skill_identity(fs, &path).await;
enqueue_dir(
&mut queue,
&mut visited_dirs,
@@ -582,7 +590,7 @@ async fn discover_skills_under_root(
}
if metadata.is_directory {
let resolved_dir = canonicalize_for_skill_identity(&path);
let resolved_dir = canonicalize_for_skill_identity(fs, &path).await;
enqueue_dir(
&mut queue,
&mut visited_dirs,
@@ -672,7 +680,7 @@ async fn parse_skill_file(
)?;
}
let resolved_path = canonicalize_for_skill_identity(path);
let resolved_path = canonicalize_for_skill_identity(fs, path).await;
Ok(SkillMetadata {
name,
+1
View File
@@ -103,6 +103,7 @@ pub(crate) async fn run_codex_thread_interactive(
parent_rollout_thread_trace: codex_rollout_trace::ThreadTraceContext::disabled(),
parent_trace: None,
environment_selections: parent_ctx.environments.clone(),
thread_extension_init: codex_extension_api::ExtensionDataInit::default(),
analytics_events_client: Some(parent_session.services.analytics_events_client.clone()),
thread_store: Arc::clone(&parent_session.services.thread_store),
attestation_provider: parent_session.services.attestation_provider.clone(),
+4
View File
@@ -51,6 +51,7 @@ use codex_config::types::OAuthCredentialsStoreMode;
use codex_exec_server::Environment;
use codex_exec_server::EnvironmentManager;
use codex_exec_server::FileSystemSandboxContext;
use codex_extension_api::ExtensionDataInit;
use codex_extension_api::PromptSlot;
use codex_features::FEATURES;
use codex_features::Feature;
@@ -417,6 +418,7 @@ pub(crate) struct CodexSpawnArgs {
pub(crate) user_shell_override: Option<shell::Shell>,
pub(crate) parent_trace: Option<W3cTraceContext>,
pub(crate) environment_selections: ResolvedTurnEnvironments,
pub(crate) thread_extension_init: ExtensionDataInit,
pub(crate) analytics_events_client: Option<AnalyticsEventsClient>,
pub(crate) thread_store: Arc<dyn ThreadStore>,
pub(crate) attestation_provider: Option<Arc<dyn AttestationProvider>>,
@@ -497,6 +499,7 @@ impl Codex {
parent_rollout_thread_trace,
parent_trace: _,
environment_selections,
thread_extension_init,
analytics_events_client,
thread_store,
attestation_provider,
@@ -641,6 +644,7 @@ impl Codex {
plugins_manager,
mcp_manager.clone(),
extensions,
thread_extension_init,
agent_control,
environment_manager,
analytics_events_client,
+6 -2
View File
@@ -4,6 +4,7 @@ use crate::agents_md::LoadedAgentsMd;
use crate::config::ConstraintError;
use crate::skills::SkillError;
use crate::state::ActiveTurn;
use codex_extension_api::ExtensionDataInit;
use codex_protocol::SessionId;
use codex_protocol::config_types::SERVICE_TIER_DEFAULT_REQUEST_VALUE;
use codex_protocol::config_types::ServiceTier;
@@ -487,6 +488,7 @@ impl Session {
plugins_manager: Arc<PluginsManager>,
mcp_manager: Arc<McpManager>,
extensions: Arc<codex_extension_api::ExtensionRegistry<crate::config::Config>>,
thread_extension_init: ExtensionDataInit,
agent_control: AgentControl,
environment_manager: Arc<EnvironmentManager>,
analytics_events_client: Option<AnalyticsEventsClient>,
@@ -961,8 +963,10 @@ impl Session {
);
let session_extension_data =
codex_extension_api::ExtensionData::new(session_id.to_string());
let thread_extension_data =
codex_extension_api::ExtensionData::new(thread_id.to_string());
let thread_extension_data = codex_extension_api::ExtensionData::new_with_init(
thread_id.to_string(),
thread_extension_init,
);
for contributor in extensions.thread_lifecycle_contributors() {
contributor.on_thread_start(codex_extension_api::ThreadStartInput {
config: config.as_ref(),
+3
View File
@@ -4690,6 +4690,7 @@ async fn session_new_fails_when_zsh_fork_enabled_without_packaged_zsh() {
plugins_manager,
mcp_manager,
Arc::new(codex_extension_api::ExtensionRegistryBuilder::new().build()),
codex_extension_api::ExtensionDataInit::default(),
AgentControl::default(),
Arc::new(codex_exec_server::EnvironmentManager::default_for_tests()),
/*analytics_events_client*/ None,
@@ -5030,6 +5031,7 @@ async fn make_session_with_config_and_rx(
plugins_manager,
mcp_manager,
Arc::new(codex_extension_api::ExtensionRegistryBuilder::new().build()),
codex_extension_api::ExtensionDataInit::default(),
AgentControl::default(),
Arc::new(codex_exec_server::EnvironmentManager::default_for_tests()),
/*analytics_events_client*/ None,
@@ -5131,6 +5133,7 @@ async fn make_session_with_history_source_and_agent_control_and_rx(
plugins_manager,
mcp_manager,
Arc::new(codex_extension_api::ExtensionRegistryBuilder::new().build()),
codex_extension_api::ExtensionDataInit::default(),
agent_control,
Arc::new(codex_exec_server::EnvironmentManager::default_for_tests()),
/*analytics_events_client*/ None,
@@ -731,6 +731,7 @@ async fn guardian_subagent_does_not_inherit_parent_exec_policy_rules() {
environment_selections: ResolvedTurnEnvironments {
turn_environments: Vec::new(),
},
thread_extension_init: codex_extension_api::ExtensionDataInit::default(),
analytics_events_client: None,
thread_store,
attestation_provider: None,
+15
View File
@@ -21,6 +21,7 @@ use codex_app_server_protocol::ThreadHistoryBuilder;
use codex_app_server_protocol::TurnStatus;
use codex_core_plugins::PluginsManager;
use codex_exec_server::EnvironmentManager;
use codex_extension_api::ExtensionDataInit;
use codex_extension_api::ExtensionRegistry;
use codex_extension_api::empty_extension_registry;
use codex_features::Feature;
@@ -182,6 +183,7 @@ pub struct StartThreadOptions {
pub metrics_service_name: Option<String>,
pub parent_trace: Option<W3cTraceContext>,
pub environments: Vec<TurnEnvironmentSelection>,
pub thread_extension_init: ExtensionDataInit,
}
pub(crate) struct ResumeThreadWithHistoryOptions {
@@ -576,6 +578,7 @@ impl ThreadManager {
metrics_service_name: None,
parent_trace: None,
environments,
thread_extension_init: ExtensionDataInit::default(),
}))
.await
}
@@ -614,6 +617,7 @@ impl ThreadManager {
/*inherited_exec_policy*/ None,
options.parent_trace,
options.environments,
options.thread_extension_init,
/*user_shell_override*/ None,
))
.await
@@ -702,6 +706,7 @@ impl ThreadManager {
/*inherited_exec_policy*/ None,
parent_trace,
environments,
/*thread_extension_init*/ ExtensionDataInit::default(),
/*user_shell_override*/ None,
))
.await
@@ -728,6 +733,7 @@ impl ThreadManager {
/*metrics_service_name*/ None,
/*parent_trace*/ None,
environments,
/*thread_extension_init*/ ExtensionDataInit::default(),
/*user_shell_override*/ Some(user_shell_override),
))
.await
@@ -763,6 +769,7 @@ impl ThreadManager {
/*inherited_exec_policy*/ None,
/*parent_trace*/ None,
environments,
/*thread_extension_init*/ ExtensionDataInit::default(),
/*user_shell_override*/ Some(user_shell_override),
))
.await
@@ -931,6 +938,7 @@ impl ThreadManager {
/*metrics_service_name*/ None,
parent_trace,
environments,
/*thread_extension_init*/ ExtensionDataInit::default(),
/*user_shell_override*/ None,
))
.await
@@ -1136,6 +1144,7 @@ impl ThreadManagerState {
inherited_exec_policy,
/*parent_trace*/ None,
environments,
/*thread_extension_init*/ ExtensionDataInit::default(),
/*user_shell_override*/ None,
))
.await
@@ -1172,6 +1181,7 @@ impl ThreadManagerState {
inherited_exec_policy,
/*parent_trace*/ None,
environments,
/*thread_extension_init*/ ExtensionDataInit::default(),
/*user_shell_override*/ None,
))
.await
@@ -1209,6 +1219,7 @@ impl ThreadManagerState {
inherited_exec_policy,
/*parent_trace*/ None,
environments,
/*thread_extension_init*/ ExtensionDataInit::default(),
/*user_shell_override*/ None,
))
.await
@@ -1229,6 +1240,7 @@ impl ThreadManagerState {
metrics_service_name: Option<String>,
parent_trace: Option<W3cTraceContext>,
environments: Vec<TurnEnvironmentSelection>,
thread_extension_init: ExtensionDataInit,
user_shell_override: Option<crate::shell::Shell>,
) -> CodexResult<NewThread> {
Box::pin(self.spawn_thread_with_source(
@@ -1246,6 +1258,7 @@ impl ThreadManagerState {
/*inherited_exec_policy*/ None,
parent_trace,
environments,
thread_extension_init,
user_shell_override,
))
.await
@@ -1268,6 +1281,7 @@ impl ThreadManagerState {
inherited_exec_policy: Option<Arc<crate::exec_policy::ExecPolicyManager>>,
parent_trace: Option<W3cTraceContext>,
environments: Vec<TurnEnvironmentSelection>,
thread_extension_init: ExtensionDataInit,
user_shell_override: Option<crate::shell::Shell>,
) -> CodexResult<NewThread> {
let is_resumed_thread = matches!(&initial_history, InitialHistory::Resumed(_));
@@ -1332,6 +1346,7 @@ impl ThreadManagerState {
user_shell_override,
parent_trace,
environment_selections,
thread_extension_init,
analytics_events_client: self.analytics_events_client.clone(),
thread_store: Arc::clone(&self.thread_store),
attestation_provider: self.attestation_provider.clone(),
+79
View File
@@ -331,6 +331,7 @@ async fn start_thread_rejects_explicit_local_environment_when_default_provider_i
environment_id: "local".to_string(),
cwd: config.cwd.clone(),
}],
thread_extension_init: Default::default(),
})
.await;
let err = match result {
@@ -368,6 +369,7 @@ async fn start_thread_keeps_internal_threads_hidden_from_normal_lookups() {
metrics_service_name: None,
parent_trace: None,
environments: Vec::new(),
thread_extension_init: Default::default(),
})
.await
.expect("internal thread should start");
@@ -384,6 +386,81 @@ async fn start_thread_keeps_internal_threads_hidden_from_normal_lookups() {
assert!(manager.list_thread_ids().await.is_empty());
}
#[tokio::test]
async fn start_thread_seeds_extension_data_before_lifecycle_contributors_run() {
struct InitialMarker(&'static str);
struct InitialDataRecorder {
observed: Arc<std::sync::Mutex<Option<(String, String)>>>,
}
#[async_trait::async_trait]
impl codex_extension_api::ThreadLifecycleContributor<Config> for InitialDataRecorder {
async fn on_thread_start(&self, input: codex_extension_api::ThreadStartInput<'_, Config>) {
let marker = input
.thread_store
.get::<InitialMarker>()
.expect("initial extension data should be available");
*self
.observed
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner) = Some((
input.thread_store.level_id().to_string(),
marker.0.to_string(),
));
}
}
let temp_dir = tempdir().expect("tempdir");
let mut config = test_config().await;
config.codex_home = temp_dir.path().join("codex-home").abs();
config.cwd = config.codex_home.abs();
std::fs::create_dir_all(&config.codex_home).expect("create codex home");
let observed = Arc::new(std::sync::Mutex::new(None));
let mut extensions = codex_extension_api::ExtensionRegistryBuilder::new();
extensions.thread_lifecycle_contributor(Arc::new(InitialDataRecorder {
observed: Arc::clone(&observed),
}));
let manager = ThreadManager::new(
&config,
AuthManager::from_auth_for_testing(CodexAuth::create_dummy_chatgpt_auth_for_testing()),
SessionSource::Exec,
Arc::new(codex_exec_server::EnvironmentManager::default_for_tests()),
Arc::new(extensions.build()),
/*analytics_events_client*/ None,
thread_store_from_config(&config, /*state_db*/ None),
/*state_db*/ None,
TEST_INSTALLATION_ID.to_string(),
/*attestation_provider*/ None,
);
let mut thread_extension_init = codex_extension_api::ExtensionDataInit::new();
thread_extension_init.insert(InitialMarker("seeded"));
let thread = manager
.start_thread_with_options(StartThreadOptions {
config,
initial_history: InitialHistory::New,
session_source: None,
thread_source: None,
dynamic_tools: Vec::new(),
metrics_service_name: None,
parent_trace: None,
environments: Vec::new(),
thread_extension_init,
})
.await
.expect("start thread");
assert_eq!(
observed
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.clone(),
Some((thread.thread_id.to_string(), "seeded".to_string()))
);
}
#[tokio::test]
async fn resume_and_fork_do_not_restore_thread_environments_from_rollout() {
let temp_dir = tempdir().expect("tempdir");
@@ -423,6 +500,7 @@ async fn resume_and_fork_do_not_restore_thread_environments_from_rollout() {
metrics_service_name: None,
parent_trace: None,
environments: environments.clone(),
thread_extension_init: Default::default(),
})
.await
.expect("start source thread");
@@ -693,6 +771,7 @@ async fn resume_stopped_thread_from_rollout_preserves_thread_source() {
metrics_service_name: None,
parent_trace: None,
environments: Vec::new(),
thread_extension_init: Default::default(),
})
.await
.expect("start source thread");
@@ -757,6 +757,7 @@ async fn subagent_stop_replaces_stop_and_skips_internal_subagents() -> Result<()
metrics_service_name: None,
parent_trace: None,
environments: Vec::new(),
thread_extension_init: Default::default(),
})
.await?;
+1
View File
@@ -59,3 +59,4 @@ pub use registry::ExtensionRegistry;
pub use registry::ExtensionRegistryBuilder;
pub use registry::empty_extension_registry;
pub use state::ExtensionData;
pub use state::ExtensionDataInit;
+32 -1
View File
@@ -7,6 +7,32 @@ use std::sync::PoisonError;
type ErasedData = Arc<dyn Any + Send + Sync>;
/// Typed values supplied before an [`ExtensionData`] scope is created.
///
/// Hosts consume this value once to seed a scope before lifecycle contributors
/// run. It does not install extensions or provide persistence.
#[derive(Debug, Default)]
pub struct ExtensionDataInit {
entries: HashMap<TypeId, ErasedData>,
}
impl ExtensionDataInit {
/// Creates an empty extension data initializer.
pub fn new() -> Self {
Self::default()
}
/// Stores `value` as the initial attachment of type `T`.
pub fn insert<T>(&mut self, value: T) -> Option<Arc<T>>
where
T: Any + Send + Sync,
{
self.entries
.insert(TypeId::of::<T>(), Arc::new(value))
.map(downcast_data)
}
}
/// Typed extension-owned data attached to one host object.
#[derive(Debug)]
pub struct ExtensionData {
@@ -17,9 +43,14 @@ pub struct ExtensionData {
impl ExtensionData {
/// Creates an empty attachment map for one host-owned scope.
pub fn new(level_id: impl Into<String>) -> Self {
Self::new_with_init(level_id, ExtensionDataInit::default())
}
/// Creates an attachment map seeded with host-supplied initial data.
pub fn new_with_init(level_id: impl Into<String>, init: ExtensionDataInit) -> Self {
Self {
level_id: level_id.into(),
entries: Mutex::new(HashMap::new()),
entries: Mutex::new(init.entries),
}
}
+3
View File
@@ -17,8 +17,11 @@ workspace = true
async-trait = { workspace = true }
codex-core = { workspace = true }
codex-core-skills = { workspace = true }
codex-exec-server = { workspace = true }
codex-extension-api = { workspace = true }
codex-protocol = { workspace = true }
codex-utils-absolute-path = { workspace = true }
codex-utils-string = { workspace = true }
[dev-dependencies]
pretty_assertions = { workspace = true }
+32 -3
View File
@@ -1,4 +1,5 @@
use codex_core_skills::model::SkillDependencies;
use codex_exec_server::EnvironmentPathRef;
/// Source authority that owns a skill package and must be used to read it.
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
@@ -56,9 +57,37 @@ impl SkillAuthority {
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
pub struct SkillPackageId(pub String);
/// Opaque resource id inside a skill package.
/// Opaque resource id inside a skill package, optionally bound to the
/// environment path that owns its contents.
#[derive(Clone, Debug, PartialEq, Eq, Hash)]
pub struct SkillResourceId(pub String);
pub struct SkillResourceId {
id: String,
environment_path: Option<EnvironmentPathRef>,
}
impl SkillResourceId {
pub fn new(id: impl Into<String>) -> Self {
Self {
id: id.into(),
environment_path: None,
}
}
pub fn environment(id: impl Into<String>, path: EnvironmentPathRef) -> Self {
Self {
id: id.into(),
environment_path: Some(path),
}
}
pub fn as_str(&self) -> &str {
&self.id
}
pub(crate) fn environment_path(&self) -> Option<&EnvironmentPathRef> {
self.environment_path.as_ref()
}
}
/// Metadata shown in the always-visible skills catalog.
#[derive(Clone, Debug, PartialEq, Eq)]
@@ -125,7 +154,7 @@ impl SkillCatalogEntry {
pub(crate) fn rendered_path(&self) -> &str {
self.display_path
.as_deref()
.unwrap_or(self.main_prompt.0.as_str())
.unwrap_or_else(|| self.main_prompt.as_str())
}
}
+81 -23
View File
@@ -6,27 +6,32 @@ use codex_core_skills::SkillInstructions;
use codex_core_skills::injection::InjectedHostSkillPrompts;
use codex_core_skills::injection::SkillInjection;
use codex_extension_api::ConfigContributor;
use codex_extension_api::ContextContributor;
use codex_extension_api::ContextualUserFragment;
use codex_extension_api::ExtensionData;
use codex_extension_api::ExtensionEventSink;
use codex_extension_api::ExtensionRegistryBuilder;
use codex_extension_api::PromptFragment;
use codex_extension_api::ThreadLifecycleContributor;
use codex_extension_api::ThreadStartInput;
use codex_extension_api::TurnInputContext;
use codex_extension_api::TurnInputContributor;
use codex_protocol::capabilities::SelectedCapabilityRoot;
use codex_protocol::protocol::Event;
use codex_protocol::protocol::EventMsg;
use codex_protocol::protocol::WarningEvent;
use crate::catalog::SkillAuthority;
use crate::catalog::SkillCatalogEntry;
use crate::catalog::SkillReadResult;
use crate::catalog::SkillSourceKind;
use crate::provider::HostSkillProvider;
use crate::provider::SkillListQuery;
use crate::provider::SkillReadRequest;
use crate::render::MAX_SKILL_NAME_BYTES;
use crate::render::MAX_SKILL_PATH_BYTES;
use crate::render::available_skills_fragment;
use crate::render::truncate_main_prompt_contents;
use crate::render::truncate_utf8_to_bytes;
use crate::selection::collect_explicit_skill_mentions;
use crate::sources::SkillProviders;
use crate::state::SkillsExtensionConfig;
@@ -42,11 +47,15 @@ struct SkillsExtension {
#[async_trait::async_trait]
impl ThreadLifecycleContributor<Config> for SkillsExtension {
async fn on_thread_start(&self, input: ThreadStartInput<'_, Config>) {
input
let selected_roots = input
.thread_store
.insert(SkillsThreadState::new(SkillsExtensionConfig::from_config(
input.config,
)));
.get::<Vec<SelectedCapabilityRoot>>()
.map(|selected_roots| selected_roots.as_ref().clone())
.unwrap_or_default();
input.thread_store.insert(SkillsThreadState::new(
SkillsExtensionConfig::from_config(input.config),
selected_roots,
));
}
}
@@ -62,11 +71,47 @@ impl ConfigContributor<Config> for SkillsExtension {
if let Some(state) = thread_store.get::<SkillsThreadState>() {
state.set_config(next_config);
} else {
thread_store.insert(SkillsThreadState::new(next_config));
thread_store.insert(SkillsThreadState::new(next_config, Vec::new()));
}
}
}
impl ContextContributor for SkillsExtension {
fn contribute<'a>(
&'a self,
_session_store: &'a ExtensionData,
thread_store: &'a ExtensionData,
) -> std::pin::Pin<Box<dyn std::future::Future<Output = Vec<PromptFragment>> + Send + 'a>> {
Box::pin(async move {
let Some(thread_state) = thread_store.get::<SkillsThreadState>() else {
return Vec::new();
};
let config = thread_state.config();
if !config.include_instructions || thread_state.selected_roots().is_empty() {
return Vec::new();
}
let catalog = self
.providers
.list_for_turn(SkillListQuery {
turn_id: thread_store.level_id().to_string(),
executor_roots: thread_state.selected_roots().to_vec(),
host: None,
include_host_skills: false,
include_bundled_skills: config.bundled_skills_enabled,
include_remote_skills: false,
})
.await;
for warning in &catalog.warnings {
self.emit_warning(thread_store.level_id(), warning.clone());
}
available_skills_fragment(&catalog)
.map(|fragment| PromptFragment::developer_capability(fragment.render()))
.into_iter()
.collect()
})
}
}
#[async_trait::async_trait]
impl TurnInputContributor for SkillsExtension {
async fn contribute(
@@ -84,16 +129,7 @@ impl TurnInputContributor for SkillsExtension {
let host_loaded_skills = turn_store.get::<HostLoadedSkills>();
let query = SkillListQuery {
turn_id: input.turn_id.clone(),
executor_authorities: input
.environments
.iter()
.map(|environment| {
SkillAuthority::new(
SkillSourceKind::Executor,
environment.environment_id.clone(),
)
})
.collect(),
executor_roots: thread_state.selected_roots().to_vec(),
host: host_loaded_skills.clone(),
include_host_skills: true,
include_bundled_skills: config.bundled_skills_enabled,
@@ -106,10 +142,14 @@ impl TurnInputContributor for SkillsExtension {
let selected_entries = collect_explicit_skill_mentions(&input.user_input, &catalog);
let mut fragments: Vec<Box<dyn ContextualUserFragment + Send>> = Vec::new();
if config.include_instructions
&& let Some(fragment) = available_skills_fragment(&catalog)
{
fragments.push(Box::new(fragment));
if config.include_instructions {
let mut turn_catalog = catalog.clone();
turn_catalog
.entries
.retain(|entry| entry.authority.kind != SkillSourceKind::Executor);
if let Some(fragment) = available_skills_fragment(&turn_catalog) {
fragments.push(Box::new(fragment));
}
}
let mut warnings = catalog.warnings.clone();
@@ -132,14 +172,14 @@ impl TurnInputContributor for SkillsExtension {
warnings.push(warning);
}
let injection = SkillInjection {
name: entry.name.clone(),
path: entry.rendered_path().to_string(),
name: truncate_utf8_to_bytes(&entry.name, MAX_SKILL_NAME_BYTES).0,
path: truncate_utf8_to_bytes(entry.rendered_path(), MAX_SKILL_PATH_BYTES).0,
contents,
};
fragments.push(Box::new(SkillInstructions::from(&injection)));
main_prompts_injected = true;
if entry.authority.kind == SkillSourceKind::Host {
injected_host_skill_prompts.insert_path(entry.main_prompt.0.clone());
injected_host_skill_prompts.insert_path(entry.main_prompt.as_str());
}
}
Err(message) => {
@@ -150,6 +190,23 @@ impl TurnInputContributor for SkillsExtension {
}
}
if let Some(host_loaded_skills) = &host_loaded_skills {
for entry in selected_entries
.iter()
.filter(|entry| entry.authority.kind != SkillSourceKind::Host)
{
for host_skill in host_loaded_skills
.outcome()
.skills
.iter()
.filter(|host_skill| host_skill.name == entry.name)
{
injected_host_skill_prompts
.insert_path(host_skill.path_to_skills_md.to_string_lossy());
}
}
}
turn_store.insert(SkillsTurnState {
catalog,
selected_entries,
@@ -206,5 +263,6 @@ pub fn install_with_providers(
});
registry.thread_lifecycle_contributor(extension.clone());
registry.config_contributor(extension.clone());
registry.prompt_contributor(extension.clone());
registry.turn_input_contributor(extension);
}
+2
View File
@@ -8,6 +8,8 @@ mod state;
pub use extension::install;
pub use extension::install_with_providers;
pub use provider::ExecutorSkillProvider;
pub use provider::HostSkillProvider;
pub use provider::SkillProvider;
pub use sources::SkillProviderSource;
pub use sources::SkillProviders;
+4 -1
View File
@@ -2,9 +2,11 @@ use std::future::Future;
use std::pin::Pin;
use std::sync::Arc;
mod executor;
mod host;
use codex_core_skills::HostLoadedSkills;
use codex_protocol::capabilities::SelectedCapabilityRoot;
use crate::catalog::SkillAuthority;
use crate::catalog::SkillCatalog;
@@ -14,12 +16,13 @@ use crate::catalog::SkillReadResult;
use crate::catalog::SkillResourceId;
use crate::catalog::SkillSearchResult;
pub use executor::ExecutorSkillProvider;
pub use host::HostSkillProvider;
#[derive(Clone, Debug)]
pub struct SkillListQuery {
pub turn_id: String,
pub executor_authorities: Vec<SkillAuthority>,
pub executor_roots: Vec<SelectedCapabilityRoot>,
pub host: Option<Arc<HostLoadedSkills>>,
pub include_host_skills: bool,
pub include_bundled_skills: bool,
@@ -0,0 +1,197 @@
use std::path::PathBuf;
use std::sync::Arc;
use codex_core_skills::SkillMetadata;
use codex_core_skills::filter_skill_load_outcome_for_product;
use codex_core_skills::loader::SkillRoot;
use codex_core_skills::loader::load_skills_from_roots;
use codex_exec_server::EnvironmentManager;
use codex_exec_server::EnvironmentPathRef;
use codex_protocol::capabilities::CapabilityRootLocation;
use codex_protocol::protocol::Product;
use codex_protocol::protocol::SkillScope;
use codex_utils_absolute_path::AbsolutePathBuf;
use crate::catalog::SkillAuthority;
use crate::catalog::SkillCatalog;
use crate::catalog::SkillCatalogEntry;
use crate::catalog::SkillPackageId;
use crate::catalog::SkillProviderError;
use crate::catalog::SkillReadResult;
use crate::catalog::SkillResourceId;
use crate::catalog::SkillSearchResult;
use crate::catalog::SkillSourceKind;
use crate::provider::SkillListQuery;
use crate::provider::SkillProvider;
use crate::provider::SkillProviderFuture;
use crate::provider::SkillReadRequest;
use crate::provider::SkillSearchRequest;
/// Discovers and reads skills through the filesystem owned by an execution environment.
#[derive(Clone, Debug)]
pub struct ExecutorSkillProvider {
environment_manager: Arc<EnvironmentManager>,
restriction_product: Option<Product>,
}
impl ExecutorSkillProvider {
pub fn new_with_restriction_product(
environment_manager: Arc<EnvironmentManager>,
restriction_product: Option<Product>,
) -> Self {
Self {
environment_manager,
restriction_product,
}
}
}
impl SkillProvider for ExecutorSkillProvider {
fn list(&self, query: SkillListQuery) -> SkillProviderFuture<'_, SkillCatalog> {
Box::pin(async move {
let mut catalog = SkillCatalog::default();
for selected_root in query.executor_roots {
let selected_root_id = selected_root.id;
let CapabilityRootLocation::Environment {
environment_id,
path,
} = selected_root.location;
let authority =
SkillAuthority::new(SkillSourceKind::Executor, selected_root_id.clone());
let Some(environment) = self.environment_manager.get_environment(&environment_id)
else {
catalog.warnings.push(format!(
"Selected capability root `{selected_root_id}` references unavailable environment `{environment_id}`."
));
continue;
};
let root_path = match executor_absolute_path(&path) {
Ok(root_path) => root_path,
Err(err) => {
catalog.warnings.push(format!(
"Selected capability root `{selected_root_id}` has invalid path `{path}`: {err}"
));
continue;
}
};
let file_system = environment.get_filesystem();
let outcome = filter_skill_load_outcome_for_product(
load_skills_from_roots([SkillRoot {
path: root_path.clone(),
scope: SkillScope::User,
file_system: Arc::clone(&file_system),
plugin_id: None,
plugin_root: None,
}])
.await,
self.restriction_product,
);
catalog.warnings.extend(outcome.errors.iter().map(|err| {
format!(
"Failed to load executor skill at {}: {}",
err.path.display(),
err.message
)
}));
for (skill, enabled) in outcome.skills_with_enabled() {
catalog.push_entry(catalog_entry_from_skill(
skill,
enabled,
authority.clone(),
&selected_root_id,
Arc::clone(&file_system),
));
}
}
Ok(catalog)
})
}
fn read(&self, request: SkillReadRequest) -> SkillProviderFuture<'_, SkillReadResult> {
Box::pin(async move {
if request.authority.kind != SkillSourceKind::Executor {
return Err(SkillProviderError::new(format!(
"executor skill provider cannot read {} resources",
request.authority.kind
)));
}
if request.package.0 != request.resource.as_str() {
return Err(SkillProviderError::new(
"executor skill resource does not match its package",
));
}
let Some(resource_path) = request.resource.environment_path() else {
return Err(SkillProviderError::new(
"executor skill resource is not bound to an environment",
));
};
let contents = resource_path
.read_to_string(/*sandbox*/ None)
.await
.map_err(|err| {
SkillProviderError::new(format!(
"failed to read executor skill resource {}: {err}",
request.resource.as_str()
))
})?;
Ok(SkillReadResult {
resource: request.resource,
contents,
})
})
}
fn search(&self, _request: SkillSearchRequest) -> SkillProviderFuture<'_, SkillSearchResult> {
Box::pin(async { Ok(SkillSearchResult::default()) })
}
}
fn catalog_entry_from_skill(
skill: &SkillMetadata,
enabled: bool,
authority: SkillAuthority,
selected_root_id: &str,
file_system: Arc<dyn codex_exec_server::ExecutorFileSystem>,
) -> SkillCatalogEntry {
let skill_path = skill.path_to_skills_md.to_string_lossy().into_owned();
let normalized_path = skill_path.replace('\\', "/");
let display_path = format!(
"skill://{selected_root_id}/{}",
normalized_path.trim_start_matches('/')
);
let mut entry = SkillCatalogEntry::new(
SkillPackageId(display_path.clone()),
authority,
skill.name.clone(),
skill.description.clone(),
SkillResourceId::environment(
display_path.clone(),
EnvironmentPathRef::new(file_system, skill.path_to_skills_md.clone()),
),
)
.with_short_description(skill.short_description.clone())
.with_display_path(display_path)
.with_dependencies(skill.dependencies.clone());
if !enabled {
entry = entry.disabled();
}
if !skill.allows_implicit_invocation() {
entry = entry.hidden_from_prompt();
}
entry
}
fn executor_absolute_path(path: &str) -> std::io::Result<AbsolutePathBuf> {
let path = PathBuf::from(path);
if !path.is_absolute() {
return Err(std::io::Error::new(
std::io::ErrorKind::InvalidInput,
"executor path must be absolute",
));
}
AbsolutePathBuf::from_absolute_path_checked(path)
}
+5 -5
View File
@@ -55,12 +55,12 @@ impl SkillProvider for HostSkillProvider {
};
let Some(skill) = host_loaded_skills.outcome().skills.iter().find(|skill| {
let skill_path = skill.path_to_skills_md.to_string_lossy();
skill_path == request.resource.0.as_str()
|| skill_path.replace('\\', "/") == request.resource.0
skill_path == request.resource.as_str()
|| skill_path.replace('\\', "/") == request.resource.as_str()
}) else {
return Err(SkillProviderError::new(format!(
"host skill resource is not loaded: {}",
request.resource.0
request.resource.as_str()
)));
};
@@ -70,7 +70,7 @@ impl SkillProvider for HostSkillProvider {
.map_err(|err| {
SkillProviderError::new(format!(
"failed to read host skill resource {}: {err}",
request.resource.0
request.resource.as_str()
))
})?;
@@ -117,7 +117,7 @@ fn catalog_entry_from_skill(skill: &SkillMetadata, enabled: bool) -> SkillCatalo
SkillAuthority::new(SkillSourceKind::Host, HOST_AUTHORITY_ID),
skill.name.clone(),
skill.description.clone(),
SkillResourceId(skill_path),
SkillResourceId::new(skill_path),
)
.with_short_description(skill.short_description.clone())
.with_display_path(display_path)
+15 -14
View File
@@ -2,11 +2,14 @@ use codex_core_skills::render_available_skills_body;
use codex_extension_api::ContextualUserFragment;
use codex_protocol::protocol::SKILLS_INSTRUCTIONS_CLOSE_TAG;
use codex_protocol::protocol::SKILLS_INSTRUCTIONS_OPEN_TAG;
use codex_utils_string::take_bytes_at_char_boundary;
use crate::catalog::SkillCatalog;
const MAX_AVAILABLE_SKILLS_CHARS: usize = 8_000;
const MAX_MAIN_PROMPT_CHARS: usize = 40_000;
const MAX_AVAILABLE_SKILLS_BYTES: usize = 8_000;
const MAX_MAIN_PROMPT_BYTES: usize = 8_000;
pub(crate) const MAX_SKILL_NAME_BYTES: usize = 256;
pub(crate) const MAX_SKILL_PATH_BYTES: usize = 1_024;
#[derive(Debug, Clone, PartialEq, Eq)]
pub(crate) struct AvailableSkillsFragment {
@@ -32,7 +35,7 @@ impl ContextualUserFragment for AvailableSkillsFragment {
}
pub(crate) fn available_skills_fragment(catalog: &SkillCatalog) -> Option<AvailableSkillsFragment> {
let mut total_chars = 0usize;
let mut total_bytes = 0usize;
let mut omitted = 0usize;
let mut skill_lines = Vec::new();
@@ -46,12 +49,12 @@ pub(crate) fn available_skills_fragment(catalog: &SkillCatalog) -> Option<Availa
.as_deref()
.unwrap_or(entry.description.as_str());
let line = render_skill_line(entry.name.as_str(), description, entry.rendered_path());
let next_chars = total_chars.saturating_add(line.chars().count());
if next_chars > MAX_AVAILABLE_SKILLS_CHARS {
let next_bytes = total_bytes.saturating_add(line.len());
if next_bytes > MAX_AVAILABLE_SKILLS_BYTES {
omitted = omitted.saturating_add(1);
continue;
}
total_chars = next_chars;
total_bytes = next_bytes;
skill_lines.push(line);
}
@@ -79,12 +82,10 @@ fn render_skill_line(name: &str, description: &str, path: &str) -> String {
}
pub(crate) fn truncate_main_prompt_contents(contents: &str) -> (String, bool) {
let mut chars = 0usize;
for (index, _) in contents.char_indices() {
if chars == MAX_MAIN_PROMPT_CHARS {
return (contents[..index].to_string(), true);
}
chars = chars.saturating_add(1);
}
(contents.to_string(), false)
truncate_utf8_to_bytes(contents, MAX_MAIN_PROMPT_BYTES)
}
pub(crate) fn truncate_utf8_to_bytes(contents: &str, max_bytes: usize) -> (String, bool) {
let truncated = take_bytes_at_char_boundary(contents, max_bytes);
(truncated.to_string(), truncated.len() < contents.len())
}
+2 -2
View File
@@ -93,12 +93,12 @@ fn push_selected(
}
fn entry_matches_path(entry: &SkillCatalogEntry, path: &str) -> bool {
entry.main_prompt.0 == path
entry.main_prompt.as_str() == path
|| entry.id.0 == path
|| entry
.display_path
.as_deref()
.is_some_and(|display_path| display_path == path)
.is_some_and(|display_path| normalize_skill_path(display_path) == path)
}
fn path_is_skill(path: &str) -> bool {
+1 -1
View File
@@ -46,7 +46,7 @@ impl SkillProviderSource {
fn should_list(&self, query: &SkillListQuery) -> bool {
match &self.kind {
SkillSourceKind::Host => query.include_host_skills,
SkillSourceKind::Executor => !query.executor_authorities.is_empty(),
SkillSourceKind::Executor => !query.executor_roots.is_empty(),
SkillSourceKind::Remote => query.include_remote_skills,
SkillSourceKind::Custom(_) => true,
}
+11 -1
View File
@@ -1,4 +1,5 @@
use codex_core::config::Config;
use codex_protocol::capabilities::SelectedCapabilityRoot;
use std::sync::Mutex;
use crate::catalog::SkillCatalog;
@@ -22,12 +23,17 @@ impl SkillsExtensionConfig {
#[derive(Debug)]
pub(crate) struct SkillsThreadState {
config: Mutex<SkillsExtensionConfig>,
selected_roots: Vec<SelectedCapabilityRoot>,
}
impl SkillsThreadState {
pub(crate) fn new(config: SkillsExtensionConfig) -> Self {
pub(crate) fn new(
config: SkillsExtensionConfig,
selected_roots: Vec<SelectedCapabilityRoot>,
) -> Self {
Self {
config: Mutex::new(config),
selected_roots,
}
}
@@ -44,6 +50,10 @@ impl SkillsThreadState {
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner) = config;
}
pub(crate) fn selected_roots(&self) -> &[SelectedCapabilityRoot] {
&self.selected_roots
}
}
#[derive(Clone, Debug, Default, PartialEq, Eq)]
@@ -0,0 +1,337 @@
use std::io;
use std::path::Path;
use std::sync::Arc;
use std::sync::atomic::AtomicUsize;
use std::sync::atomic::Ordering;
use async_trait::async_trait;
use codex_core_skills::HostLoadedSkills;
use codex_core_skills::loader::SkillRoot;
use codex_core_skills::loader::load_skills_from_roots;
use codex_exec_server::CopyOptions;
use codex_exec_server::CreateDirectoryOptions;
use codex_exec_server::EnvironmentManager;
use codex_exec_server::ExecutorFileSystem;
use codex_exec_server::FileMetadata;
use codex_exec_server::FileSystemResult;
use codex_exec_server::FileSystemSandboxContext;
use codex_exec_server::ReadDirectoryEntry;
use codex_exec_server::RemoveOptions;
use codex_protocol::capabilities::CapabilityRootLocation;
use codex_protocol::capabilities::SelectedCapabilityRoot;
use codex_protocol::protocol::SkillScope;
use codex_skills_extension::ExecutorSkillProvider;
use codex_skills_extension::catalog::SkillReadResult;
use codex_skills_extension::provider::SkillListQuery;
use codex_skills_extension::provider::SkillProvider;
use codex_skills_extension::provider::SkillReadRequest;
use codex_utils_absolute_path::AbsolutePathBuf;
use pretty_assertions::assert_eq;
const SKILL_CONTENTS: &str =
"---\nname: synthetic\ndescription: Synthetic executor skill.\n---\n\nEXECUTOR_ONLY_BODY\n";
static NEXT_TEST_ROOT_ID: AtomicUsize = AtomicUsize::new(0);
struct SyntheticFileSystem {
alias_root: AbsolutePathBuf,
canonical_root: AbsolutePathBuf,
}
impl SyntheticFileSystem {
fn metadata(&self, path: &AbsolutePathBuf) -> io::Result<FileMetadata> {
let skill_dir = self.canonical_root.join("skill");
let skill_path = skill_dir.join("SKILL.md");
let (is_directory, is_file) = if path == &self.canonical_root || path == &skill_dir {
(true, false)
} else if path == &skill_path {
(false, true)
} else {
return Err(io::Error::new(io::ErrorKind::NotFound, "not found"));
};
Ok(FileMetadata {
is_directory,
is_file,
is_symlink: false,
created_at_ms: 0,
modified_at_ms: 0,
})
}
}
#[async_trait]
impl ExecutorFileSystem for SyntheticFileSystem {
async fn canonicalize(
&self,
path: &AbsolutePathBuf,
_sandbox: Option<&FileSystemSandboxContext>,
) -> FileSystemResult<AbsolutePathBuf> {
if path == &self.alias_root {
return Ok(self.canonical_root.clone());
}
self.metadata(path)?;
Ok(path.clone())
}
async fn join(
&self,
base_path: &AbsolutePathBuf,
path: &Path,
) -> FileSystemResult<AbsolutePathBuf> {
Ok(base_path.join(path))
}
async fn parent(&self, path: &AbsolutePathBuf) -> FileSystemResult<Option<AbsolutePathBuf>> {
Ok(path.parent())
}
async fn read_file(
&self,
path: &AbsolutePathBuf,
_sandbox: Option<&FileSystemSandboxContext>,
) -> FileSystemResult<Vec<u8>> {
if path == &self.canonical_root.join("skill/SKILL.md") {
Ok(SKILL_CONTENTS.as_bytes().to_vec())
} else {
Err(io::Error::new(io::ErrorKind::NotFound, "not found"))
}
}
async fn write_file(
&self,
_path: &AbsolutePathBuf,
_contents: Vec<u8>,
_sandbox: Option<&FileSystemSandboxContext>,
) -> FileSystemResult<()> {
Err(io::Error::new(io::ErrorKind::Unsupported, "read only"))
}
async fn create_directory(
&self,
_path: &AbsolutePathBuf,
_options: CreateDirectoryOptions,
_sandbox: Option<&FileSystemSandboxContext>,
) -> FileSystemResult<()> {
Err(io::Error::new(io::ErrorKind::Unsupported, "read only"))
}
async fn get_metadata(
&self,
path: &AbsolutePathBuf,
_sandbox: Option<&FileSystemSandboxContext>,
) -> FileSystemResult<FileMetadata> {
self.metadata(path)
}
async fn read_directory(
&self,
path: &AbsolutePathBuf,
_sandbox: Option<&FileSystemSandboxContext>,
) -> FileSystemResult<Vec<ReadDirectoryEntry>> {
if path == &self.canonical_root {
Ok(vec![ReadDirectoryEntry {
file_name: "skill".to_string(),
is_directory: true,
is_file: false,
}])
} else if path == &self.canonical_root.join("skill") {
Ok(vec![ReadDirectoryEntry {
file_name: "SKILL.md".to_string(),
is_directory: false,
is_file: true,
}])
} else {
Err(io::Error::new(io::ErrorKind::NotFound, "not found"))
}
}
async fn remove(
&self,
_path: &AbsolutePathBuf,
_options: RemoveOptions,
_sandbox: Option<&FileSystemSandboxContext>,
) -> FileSystemResult<()> {
Err(io::Error::new(io::ErrorKind::Unsupported, "read only"))
}
async fn copy(
&self,
_source_path: &AbsolutePathBuf,
_destination_path: &AbsolutePathBuf,
_options: CopyOptions,
_sandbox: Option<&FileSystemSandboxContext>,
) -> FileSystemResult<()> {
Err(io::Error::new(io::ErrorKind::Unsupported, "read only"))
}
}
#[tokio::test]
async fn skill_loading_and_reads_use_the_supplied_executor_file_system() {
let test_root =
std::env::temp_dir().join(format!("codex-executor-skill-fs-{}", std::process::id()));
let alias_root = AbsolutePathBuf::from_absolute_path_checked(test_root.join("alias"))
.expect("absolute path");
let canonical_root = AbsolutePathBuf::from_absolute_path_checked(test_root.join("canonical"))
.expect("absolute path");
assert!(!alias_root.as_path().exists());
assert!(!canonical_root.as_path().exists());
let outcome = load_skills_from_roots([SkillRoot {
path: alias_root.clone(),
scope: SkillScope::User,
file_system: Arc::new(SyntheticFileSystem {
alias_root,
canonical_root: canonical_root.clone(),
}),
plugin_id: None,
plugin_root: None,
}])
.await;
assert_eq!(outcome.errors, Vec::new());
assert_eq!(outcome.skills.len(), 1);
let skill = outcome.skills[0].clone();
assert_eq!(skill.name, "synthetic");
assert_eq!(
skill.path_to_skills_md,
canonical_root.join("skill/SKILL.md")
);
let loaded = HostLoadedSkills::new(Arc::new(outcome));
assert_eq!(
loaded.read_skill_text(&skill).await.expect("skill body"),
SKILL_CONTENTS
);
}
#[tokio::test]
async fn executor_provider_reads_from_the_environment_instance_used_for_listing() {
let test_root = create_local_skill_root("bound-instance").expect("create local skill root");
let root_path = test_root.to_string_lossy().into_owned();
let environment_manager = Arc::new(EnvironmentManager::default_for_tests());
let provider = ExecutorSkillProvider::new_with_restriction_product(
Arc::clone(&environment_manager),
/*restriction_product*/ None,
);
let catalog = provider
.list(SkillListQuery {
turn_id: "turn-1".to_string(),
executor_roots: vec![SelectedCapabilityRoot {
id: "root-a".to_string(),
location: CapabilityRootLocation::Environment {
environment_id: "local".to_string(),
path: root_path,
},
}],
host: None,
include_host_skills: false,
include_bundled_skills: true,
include_remote_skills: false,
})
.await
.expect("list executor skills");
let entry = catalog
.entries
.into_iter()
.next()
.expect("listed executor skill");
let resource = entry.main_prompt.clone();
environment_manager
.upsert_environment("local".to_string(), "http://127.0.0.1:1".to_string())
.expect("replace environment");
assert_eq!(
provider
.read(SkillReadRequest {
authority: entry.authority,
package: entry.id,
resource: resource.clone(),
host: None,
})
.await
.expect("read bound executor skill"),
SkillReadResult {
resource,
contents: SKILL_CONTENTS.to_string(),
}
);
std::fs::remove_dir_all(test_root).expect("remove skill directory");
}
#[tokio::test]
async fn selected_root_id_distinguishes_identical_executor_paths() {
let test_root = create_local_skill_root("root-identity").expect("create local skill root");
let root_path = test_root.to_string_lossy().into_owned();
let canonical_root = AbsolutePathBuf::from_absolute_path_checked(&test_root)
.expect("absolute skill root")
.canonicalize()
.expect("canonicalize skill root")
.to_string_lossy()
.replace('\\', "/");
let provider = ExecutorSkillProvider::new_with_restriction_product(
Arc::new(EnvironmentManager::default_for_tests()),
/*restriction_product*/ None,
);
let catalog = provider
.list(SkillListQuery {
turn_id: "turn-1".to_string(),
executor_roots: ["root-a", "root-b"]
.into_iter()
.map(|id| SelectedCapabilityRoot {
id: id.to_string(),
location: CapabilityRootLocation::Environment {
environment_id: "local".to_string(),
path: root_path.clone(),
},
})
.collect(),
host: None,
include_host_skills: false,
include_bundled_skills: true,
include_remote_skills: false,
})
.await
.expect("list executor skills");
assert_eq!(
catalog
.entries
.iter()
.map(|entry| (
entry.authority.id.clone(),
entry.display_path.clone().expect("display path"),
))
.collect::<Vec<_>>(),
vec![
(
"root-a".to_string(),
format!(
"skill://root-a/{}/skill/SKILL.md",
canonical_root.trim_start_matches('/')
),
),
(
"root-b".to_string(),
format!(
"skill://root-b/{}/skill/SKILL.md",
canonical_root.trim_start_matches('/')
),
),
]
);
std::fs::remove_dir_all(test_root).expect("remove skill directory");
}
fn create_local_skill_root(label: &str) -> io::Result<std::path::PathBuf> {
let id = NEXT_TEST_ROOT_ID.fetch_add(1, Ordering::Relaxed);
let test_root = std::env::temp_dir().join(format!(
"codex-executor-skill-{label}-{}-{id}",
std::process::id()
));
let skill_dir = test_root.join("skill");
std::fs::create_dir_all(&skill_dir)?;
std::fs::write(skill_dir.join("SKILL.md"), SKILL_CONTENTS)?;
Ok(test_root)
}
+135 -64
View File
@@ -14,7 +14,8 @@ use codex_extension_api::ExtensionData;
use codex_extension_api::ExtensionRegistryBuilder;
use codex_extension_api::ThreadStartInput;
use codex_extension_api::TurnInputContext;
use codex_extension_api::TurnInputEnvironment;
use codex_protocol::capabilities::CapabilityRootLocation;
use codex_protocol::capabilities::SelectedCapabilityRoot;
use codex_protocol::protocol::SKILLS_INSTRUCTIONS_OPEN_TAG;
use codex_protocol::protocol::SessionSource;
use codex_protocol::user_input::UserInput;
@@ -112,6 +113,7 @@ async fn installed_extension_loads_host_skills_from_legacy_roots() -> TestResult
.await;
assert_eq!(2, fragments.len());
assert_eq!("developer", fragments[0].role());
assert!(fragments[0].render().contains("demo"));
assert!(fragments[0].render().contains(&skill_prompt_path));
assert_eq!("user", fragments[1].role());
@@ -128,42 +130,35 @@ async fn installed_extension_loads_host_skills_from_legacy_roots() -> TestResult
}
#[tokio::test]
async fn installed_extension_injects_available_catalog_and_selected_entrypoint() -> TestResult {
let host_read_requests = Arc::new(Mutex::new(Vec::new()));
let remote_read_requests = Arc::new(Mutex::new(Vec::new()));
let host_provider = Arc::new(StaticSkillProvider {
async fn selected_executor_catalog_is_context_and_selected_entrypoint_is_turn_input() -> TestResult
{
let read_requests = Arc::new(Mutex::new(Vec::new()));
let executor_provider = Arc::new(StaticSkillProvider {
catalog: SkillCatalog {
entries: vec![test_entry(
SkillSourceKind::Host,
"host",
"host/lint-fix",
SkillSourceKind::Executor,
"env-1",
"executor/lint-fix",
"lint-fix/SKILL.md",
)],
warnings: Vec::new(),
},
read_requests: Arc::clone(&host_read_requests),
read_requests: Arc::clone(&read_requests),
});
let remote_provider = Arc::new(StaticSkillProvider {
catalog: SkillCatalog {
entries: vec![test_entry(
SkillSourceKind::Remote,
"remote",
"remote/lint-fix",
"lint-fix/SKILL.md",
)],
warnings: Vec::new(),
},
read_requests: Arc::clone(&remote_read_requests),
});
let providers = SkillProviders::new()
.with_host_provider(host_provider)
.with_remote_provider(remote_provider);
let providers = SkillProviders::new().with_executor_provider(executor_provider);
let mut builder = ExtensionRegistryBuilder::new();
install_with_providers(&mut builder, providers);
let registry = builder.build();
let session_store = ExtensionData::new("session");
let thread_store = ExtensionData::new("thread");
thread_store.insert(vec![SelectedCapabilityRoot {
id: "lint-fix".to_string(),
location: CapabilityRootLocation::Environment {
environment_id: "env-1".to_string(),
path: "/skills/lint-fix".to_string(),
},
}]);
let session_source = SessionSource::Cli;
let config = default_config().await?;
registry.thread_lifecycle_contributors()[0]
@@ -176,6 +171,17 @@ async fn installed_extension_injects_available_catalog_and_selected_entrypoint()
})
.await;
let prompt_fragments = registry.context_contributors()[0]
.contribute(&session_store, &thread_store)
.await;
assert_eq!(1, prompt_fragments.len());
assert!(
prompt_fragments[0]
.text()
.starts_with(SKILLS_INSTRUCTIONS_OPEN_TAG)
);
assert!(prompt_fragments[0].text().contains("lint-fix"));
let turn_store = ExtensionData::new("turn-1");
let fragments = registry.turn_input_contributors()[0]
.contribute(
@@ -185,11 +191,7 @@ async fn installed_extension_injects_available_catalog_and_selected_entrypoint()
text: "$lint-fix please".to_string(),
text_elements: Vec::new(),
}],
environments: vec![TurnInputEnvironment {
environment_id: "env-1".to_string(),
cwd: std::env::temp_dir(),
is_primary: true,
}],
environments: Vec::new(),
},
&session_store,
&thread_store,
@@ -197,31 +199,23 @@ async fn installed_extension_injects_available_catalog_and_selected_entrypoint()
)
.await;
assert_eq!(2, fragments.len());
assert_eq!("developer", fragments[0].role());
assert!(
fragments[0]
.render()
.starts_with(SKILLS_INSTRUCTIONS_OPEN_TAG)
);
assert!(fragments[0].render().contains("lint-fix"));
assert_eq!("user", fragments[1].role());
assert!(fragments[1].render().contains("<name>lint-fix</name>"));
assert!(fragments[1].render().contains("# Lint Fix"));
assert_eq!(1, fragments.len());
assert_eq!("user", fragments[0].role());
assert!(fragments[0].render().contains("<name>lint-fix</name>"));
assert!(fragments[0].render().contains("# Lint Fix"));
assert_eq!(
vec![(
SkillAuthority::new(SkillSourceKind::Host, "host"),
SkillPackageId("host/lint-fix".to_string()),
SkillResourceId("lint-fix/SKILL.md".to_string()),
SkillAuthority::new(SkillSourceKind::Executor, "env-1"),
SkillPackageId("executor/lint-fix".to_string()),
SkillResourceId::new("lint-fix/SKILL.md"),
)],
read_request_keys(&host_read_requests)
);
assert!(
remote_read_requests
.lock()
.unwrap_or_else(std::sync::PoisonError::into_inner)
.is_empty()
read_request_keys(&read_requests)
);
let rebuilt_prompt_fragments = registry.context_contributors()[0]
.contribute(&session_store, &thread_store)
.await;
assert_eq!(1, rebuilt_prompt_fragments.len());
assert!(rebuilt_prompt_fragments[0].text().contains("lint-fix"));
let next_turn_store = ExtensionData::new("turn-2");
let next_fragments = registry.turn_input_contributors()[0]
@@ -240,9 +234,91 @@ async fn installed_extension_injects_available_catalog_and_selected_entrypoint()
)
.await;
assert_eq!(1, next_fragments.len());
assert_eq!("developer", next_fragments[0].role());
assert!(next_fragments[0].render().contains("lint-fix"));
assert!(next_fragments.is_empty());
Ok(())
}
#[tokio::test]
async fn root_qualified_locator_selects_only_the_matching_executor_skill() -> TestResult {
let read_requests = Arc::new(Mutex::new(Vec::new()));
let root_a_locator = "skill://root-a/shared/lint-fix/SKILL.md";
let root_b_locator = "skill://root-b/shared/lint-fix/SKILL.md";
let executor_provider = Arc::new(StaticSkillProvider {
catalog: SkillCatalog {
entries: [("root-a", root_a_locator), ("root-b", root_b_locator)]
.into_iter()
.map(|(root_id, locator)| {
SkillCatalogEntry::new(
SkillPackageId(locator.to_string()),
SkillAuthority::new(SkillSourceKind::Executor, root_id),
"lint-fix",
"Fix lint errors.",
SkillResourceId::new(locator),
)
.with_display_path(locator)
})
.collect(),
warnings: Vec::new(),
},
read_requests: Arc::clone(&read_requests),
});
let providers = SkillProviders::new().with_executor_provider(executor_provider);
let mut builder = ExtensionRegistryBuilder::new();
install_with_providers(&mut builder, providers);
let registry = builder.build();
let session_store = ExtensionData::new("session");
let thread_store = ExtensionData::new("thread");
thread_store.insert(
[("root-a", "/skills/root-a"), ("root-b", "/skills/root-b")]
.into_iter()
.map(|(id, path)| SelectedCapabilityRoot {
id: id.to_string(),
location: CapabilityRootLocation::Environment {
environment_id: "env-1".to_string(),
path: path.to_string(),
},
})
.collect::<Vec<_>>(),
);
let session_source = SessionSource::Cli;
let config = default_config().await?;
registry.thread_lifecycle_contributors()[0]
.on_thread_start(ThreadStartInput {
config: &config,
session_source: &session_source,
persistent_thread_state_available: true,
session_store: &session_store,
thread_store: &thread_store,
})
.await;
let fragments = registry.turn_input_contributors()[0]
.contribute(
TurnInputContext {
turn_id: "turn-1".to_string(),
user_input: vec![UserInput::Mention {
name: "lint-fix".to_string(),
path: root_b_locator.to_string(),
}],
environments: Vec::new(),
},
&session_store,
&thread_store,
&ExtensionData::new("turn-1"),
)
.await;
assert_eq!(1, fragments.len());
assert!(fragments[0].render().contains(root_b_locator));
assert_eq!(
vec![(
SkillAuthority::new(SkillSourceKind::Executor, "root-b"),
SkillPackageId(root_b_locator.to_string()),
SkillResourceId::new(root_b_locator),
)],
read_request_keys(&read_requests)
);
Ok(())
}
@@ -306,15 +382,14 @@ async fn prompt_hidden_skill_can_still_be_invoked() -> TestResult {
.await;
assert_eq!(2, fragments.len());
let catalog_fragment = fragments[0].render();
assert!(catalog_fragment.contains("visible-skill"));
assert!(!catalog_fragment.contains("hidden-skill"));
assert!(fragments[0].render().contains("visible-skill"));
assert!(!fragments[0].render().contains("hidden-skill"));
assert!(fragments[1].render().contains("<name>hidden-skill</name>"));
assert_eq!(
vec![(
SkillAuthority::new(SkillSourceKind::Host, "host"),
SkillPackageId("host/hidden-skill".to_string()),
SkillResourceId("hidden-skill/SKILL.md".to_string()),
SkillResourceId::new("hidden-skill/SKILL.md"),
)],
read_request_keys(&read_requests)
);
@@ -329,13 +404,9 @@ struct StaticSkillProvider {
}
impl SkillProvider for StaticSkillProvider {
fn list(&self, query: SkillListQuery) -> SkillProviderFuture<'_, SkillCatalog> {
fn list(&self, _query: SkillListQuery) -> SkillProviderFuture<'_, SkillCatalog> {
let catalog = self.catalog.clone();
Box::pin(async move {
assert!(query.include_host_skills);
assert!(query.include_bundled_skills);
Ok(catalog)
})
Box::pin(async move { Ok(catalog) })
}
fn read(&self, request: SkillReadRequest) -> SkillProviderFuture<'_, SkillReadResult> {
@@ -369,7 +440,7 @@ fn test_entry(
SkillAuthority::new(kind, authority_id),
name,
"Fix lint errors.",
SkillResourceId(main_prompt.to_string()),
SkillResourceId::new(main_prompt),
)
.with_display_path(format!("skill://{package_id}/SKILL.md"))
}
+1
View File
@@ -251,6 +251,7 @@ impl MemoryStartupContext {
metrics_service_name: None,
parent_trace: None,
environments,
thread_extension_init: Default::default(),
})
.await?;
+30
View File
@@ -0,0 +1,30 @@
use schemars::JsonSchema;
use serde::Deserialize;
use serde::Serialize;
use ts_rs::TS;
/// A user-selected root that can expose one or more runtime capabilities.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema, TS)]
#[serde(rename_all = "camelCase")]
#[ts(export_to = "v2/")]
pub struct SelectedCapabilityRoot {
/// Stable identifier supplied by the capability selection platform.
pub id: String,
/// Where the selected root can be resolved.
pub location: CapabilityRootLocation,
}
/// Location used to resolve a selected capability root.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize, JsonSchema, TS)]
#[serde(tag = "type", rename_all = "camelCase")]
#[ts(tag = "type")]
#[ts(export_to = "v2/")]
pub enum CapabilityRootLocation {
/// A path owned by an execution environment.
Environment {
#[serde(rename = "environmentId")]
#[ts(rename = "environmentId")]
environment_id: String,
path: String,
},
}
+1
View File
@@ -9,6 +9,7 @@ pub use session_id::SessionId;
pub use thread_id::ThreadId;
pub use tool_name::ToolName;
pub mod approvals;
pub mod capabilities;
pub mod config_types;
pub mod dynamic_tools;
pub mod error;