mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
Project selected plugin runtime by environment availability (#30093)
## Why
Selected plugin metadata is stable, but MCP processes are live runtime
state. They need different lifetimes:
- the MCP extension caches manifest, MCP, and connector declarations for
each stable selected root;
- each model step projects that cached metadata through the roots that
resolved as ready for that exact step;
- the MCP manager is rebuilt only when that availability projection
changes.
This matches executor skills: both features consume the same resolved
step roots instead of inferring readiness from the turn's selected
environments.
## Behavior
```text
E1 not ready for this step
-> no E1 MCP servers or connectors
-> cached plugin metadata stays in ext/mcp
E1 becomes ready
-> reuse cached metadata
-> publish one MCP runtime containing E1 capabilities
same ready roots on the next step
-> reuse the exact runtime; no rediscovery and no MCP restart
resume
-> create new extension thread state and a new MCP runtime
```
All model-facing consumers use the same step snapshot:
```text
resolved selected roots
|
v
extension MCP/connector projection
|
v
{ MCP config, connector snapshot, MCP manager }
|
+-> advertise model tools
+-> build app/connector tools
+-> execute MCP calls
```
## Cache contract
The existing MCP extension owns a cache keyed by the full
`SelectedCapabilityRoot`:
```rust
let state = thread_store.get_or_init(SelectedExecutorPluginMcpState::default);
```
The cache lives with extension thread state. Environment availability
filters projection but does not invalidate metadata. Resume creates new
thread state. There is no file watcher or executor generation because
contents behind a stable environment/root are assumed stable.
## What changes
- Keeps executor plugin discovery and cached metadata in `ext/mcp`.
- Caches MCP and connector declarations together per selected root.
- Uses the step's already-resolved capability roots, including lazy
environments that are not turn environments.
- Reuses the current MCP runtime when the ready-root projection is
unchanged.
- Uses the same step MCP manager and connector snapshot for
model-visible tools and execution.
- Resolves direct thread-scoped MCP requests from the current
selected-root projection.
## Deliberately out of scope
- `app/list` remains based on the latest global host-plugin state; this
PR does not make its response or notifications thread-specific.
- `required = true` startup semantics do not apply to delayed executor
MCP activation.
- No filesystem/content invalidation.
- No transport-disconnect watcher.
- No executor generations or environment replacement semantics.
- No client sharing across complete manager replacements.
## Stack
1. Extension-owned World State sections.
2. Project executor skills through World State.
3. Pin one MCP runtime to each model step.
4. **This PR:** project selected MCP and connector state from
extension-owned metadata.
5. Integration coverage for selected capability availability and resume.
## Verification
-
`selected_plugin_servers_use_managed_requirements_for_the_selected_root_id`
- The stacked integration PR covers unavailable to ready activation,
unchanged-runtime reuse, skills, MCP tools, connector attribution, and
cold resume.
This commit is contained in:
jif
@@ -1,17 +1,22 @@
|
||||
use codex_config::McpServerConfig;
|
||||
|
||||
use crate::ExtensionData;
|
||||
use crate::ExtensionDataInit;
|
||||
|
||||
/// Input supplied while resolving MCP server contributions.
|
||||
///
|
||||
/// Thread-scoped implementations can read the immutable host-seeded inputs
|
||||
/// through [`Self::thread_init`]. Implementations should not retain borrowed
|
||||
/// context after contribution completes.
|
||||
/// Thread-scoped implementations can read stable host inputs through [`Self::thread_init`] and
|
||||
/// keep their cache in [`Self::thread_store`]. Implementations should not retain borrowed context
|
||||
/// after contribution completes.
|
||||
pub struct McpServerContributionContext<'a, C> {
|
||||
/// Host configuration visible during MCP resolution.
|
||||
config: &'a C,
|
||||
/// Initial inputs for the active thread, when resolution is thread-scoped.
|
||||
/// Extension-owned data for the active thread, when resolution is thread-scoped.
|
||||
thread_store: Option<&'a ExtensionData>,
|
||||
/// Stable host inputs for the active thread, when resolution is thread-scoped.
|
||||
thread_init: Option<&'a ExtensionDataInit>,
|
||||
/// Environment IDs whose selected roots may contribute to this exact step.
|
||||
available_environment_ids: Option<&'a [String]>,
|
||||
}
|
||||
|
||||
impl<C> Clone for McpServerContributionContext<'_, C> {
|
||||
@@ -27,15 +32,24 @@ impl<'a, C> McpServerContributionContext<'a, C> {
|
||||
pub fn global(config: &'a C) -> Self {
|
||||
Self {
|
||||
config,
|
||||
thread_store: None,
|
||||
thread_init: None,
|
||||
available_environment_ids: None,
|
||||
}
|
||||
}
|
||||
|
||||
/// Creates context for one active thread runtime.
|
||||
pub fn for_thread(config: &'a C, thread_init: &'a ExtensionDataInit) -> Self {
|
||||
/// Creates context for one model step using only currently available environments.
|
||||
pub fn for_step(
|
||||
config: &'a C,
|
||||
thread_init: &'a ExtensionDataInit,
|
||||
thread_store: &'a ExtensionData,
|
||||
available_environment_ids: &'a [String],
|
||||
) -> Self {
|
||||
Self {
|
||||
config,
|
||||
thread_store: Some(thread_store),
|
||||
thread_init: Some(thread_init),
|
||||
available_environment_ids: Some(available_environment_ids),
|
||||
}
|
||||
}
|
||||
|
||||
@@ -44,10 +58,23 @@ impl<'a, C> McpServerContributionContext<'a, C> {
|
||||
self.config
|
||||
}
|
||||
|
||||
/// Returns the frozen initial inputs when resolving for a running thread.
|
||||
/// Returns extension-owned state when resolving for a running thread.
|
||||
pub fn thread_store(&self) -> Option<&'a ExtensionData> {
|
||||
self.thread_store
|
||||
}
|
||||
|
||||
/// Returns stable host inputs when resolving for a running thread.
|
||||
pub fn thread_init(&self) -> Option<&'a ExtensionDataInit> {
|
||||
self.thread_init
|
||||
}
|
||||
|
||||
/// Returns the exact environment availability projection for a model step.
|
||||
///
|
||||
/// `Some` means contributors must omit selected roots whose environment ID is absent from the
|
||||
/// slice. Global resolution returns `None` because it has no thread environments.
|
||||
pub fn available_environment_ids(&self) -> Option<&'a [String]> {
|
||||
self.available_environment_ids
|
||||
}
|
||||
}
|
||||
|
||||
/// One extension-owned overlay for the runtime MCP server configuration.
|
||||
@@ -66,6 +93,12 @@ pub enum McpServerContribution {
|
||||
selection_order: usize,
|
||||
config: Box<McpServerConfig>,
|
||||
},
|
||||
/// Adds connector IDs declared by a plugin selected for this thread.
|
||||
SelectedPluginConnectors {
|
||||
plugin_id: String,
|
||||
plugin_display_name: String,
|
||||
connector_ids: Vec<String>,
|
||||
},
|
||||
/// Removes a named MCP server.
|
||||
Remove { name: String },
|
||||
}
|
||||
|
||||
@@ -16,6 +16,7 @@ workspace = true
|
||||
codex-core = { workspace = true }
|
||||
codex-core-plugins = { workspace = true }
|
||||
codex-config = { workspace = true }
|
||||
codex-connectors-extension = { workspace = true }
|
||||
codex-exec-server = { workspace = true }
|
||||
codex-extension-api = { workspace = true }
|
||||
codex-features = { workspace = true }
|
||||
|
||||
@@ -1,44 +1,47 @@
|
||||
use codex_connectors_extension::ExecutorPluginConnectorProvider;
|
||||
use codex_core::config::Config;
|
||||
use codex_core_plugins::ExecutorPluginProvider;
|
||||
use codex_exec_server::EnvironmentManager;
|
||||
use codex_extension_api::ExtensionDataInit;
|
||||
use codex_extension_api::ExtensionFuture;
|
||||
use codex_extension_api::McpServerContribution;
|
||||
use codex_extension_api::McpServerContributionContext;
|
||||
use codex_extension_api::McpServerContributor;
|
||||
use codex_protocol::capabilities::CapabilityRootLocation;
|
||||
use codex_protocol::capabilities::SelectedCapabilityRoot;
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
use tokio::sync::OnceCell;
|
||||
use std::sync::Mutex;
|
||||
|
||||
use self::provider::ExecutorPluginMcpProvider;
|
||||
|
||||
mod provider;
|
||||
|
||||
/// Frozen MCP declarations for one selected package.
|
||||
/// Frozen MCP and connector declarations for one selected package.
|
||||
///
|
||||
/// Each server config retains the stable logical environment ID. Reconnection may replace the
|
||||
/// concrete environment instance without changing that authority.
|
||||
#[derive(Clone)]
|
||||
struct SelectedPluginMcpServers {
|
||||
struct SelectedPluginMetadata {
|
||||
plugin_id: String,
|
||||
plugin_display_name: String,
|
||||
selection_order: usize,
|
||||
servers: Vec<(String, codex_config::McpServerConfig)>,
|
||||
connector_ids: Vec<String>,
|
||||
}
|
||||
|
||||
#[derive(Default)]
|
||||
pub(crate) struct SelectedExecutorPluginMcpState {
|
||||
snapshot: OnceCell<Vec<SelectedPluginMcpServers>>,
|
||||
cache: Mutex<Vec<CachedSelectedRoot>>,
|
||||
}
|
||||
|
||||
pub(crate) fn seed_thread_state(thread_init: &mut ExtensionDataInit) {
|
||||
thread_init.insert(SelectedExecutorPluginMcpState::default());
|
||||
struct CachedSelectedRoot {
|
||||
root: SelectedCapabilityRoot,
|
||||
metadata: Option<SelectedPluginMetadata>,
|
||||
}
|
||||
|
||||
pub(crate) struct SelectedExecutorPluginMcpContributor {
|
||||
plugin_provider: ExecutorPluginProvider,
|
||||
mcp_provider: ExecutorPluginMcpProvider,
|
||||
connector_provider: ExecutorPluginConnectorProvider,
|
||||
}
|
||||
|
||||
impl SelectedExecutorPluginMcpContributor {
|
||||
@@ -46,46 +49,87 @@ impl SelectedExecutorPluginMcpContributor {
|
||||
Self {
|
||||
plugin_provider: ExecutorPluginProvider::new(Arc::clone(&environment_manager)),
|
||||
mcp_provider: ExecutorPluginMcpProvider,
|
||||
connector_provider: ExecutorPluginConnectorProvider,
|
||||
}
|
||||
}
|
||||
|
||||
async fn resolve_snapshot(
|
||||
/// Returns metadata for one stable selected root.
|
||||
///
|
||||
/// Successful resolution, including a root that is not a plugin or declares no capabilities,
|
||||
/// is cached until the thread state is dropped. Environment availability never invalidates
|
||||
/// this cache; it only controls whether the cached metadata is projected into a model step.
|
||||
async fn metadata_for_root(
|
||||
&self,
|
||||
selected_roots: &[SelectedCapabilityRoot],
|
||||
) -> Vec<SelectedPluginMcpServers> {
|
||||
let mut snapshot = Vec::new();
|
||||
state: &SelectedExecutorPluginMcpState,
|
||||
selected_root: &SelectedCapabilityRoot,
|
||||
) -> Option<SelectedPluginMetadata> {
|
||||
if let Some(cached) = state
|
||||
.cache
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner)
|
||||
.iter()
|
||||
.find(|cached| cached.root == *selected_root)
|
||||
{
|
||||
return cached.metadata.clone();
|
||||
}
|
||||
|
||||
for (selection_order, selected_root) in selected_roots.iter().enumerate() {
|
||||
let plugin = match self.plugin_provider.resolve_bound(selected_root).await {
|
||||
Ok(Some(plugin)) => plugin,
|
||||
Ok(None) => continue,
|
||||
Err(err) => {
|
||||
tracing::warn!(
|
||||
selected_root = selected_root.id,
|
||||
error = %err,
|
||||
"failed to resolve selected executor plugin for MCP discovery"
|
||||
);
|
||||
continue;
|
||||
}
|
||||
};
|
||||
match self.mcp_provider.load(&plugin).await {
|
||||
Ok(servers) => snapshot.push(SelectedPluginMcpServers {
|
||||
plugin_id: plugin.plugin().selected_root_id().to_string(),
|
||||
plugin_display_name: plugin.plugin().manifest().display_name().to_string(),
|
||||
selection_order,
|
||||
servers,
|
||||
}),
|
||||
Err(err) => {
|
||||
let plugin = match self.plugin_provider.resolve_bound(selected_root).await {
|
||||
Ok(plugin) => plugin,
|
||||
Err(err) => {
|
||||
tracing::warn!(
|
||||
selected_root = selected_root.id,
|
||||
error = %err,
|
||||
"failed to resolve selected executor plugin"
|
||||
);
|
||||
return None;
|
||||
}
|
||||
};
|
||||
let metadata = match plugin {
|
||||
Some(plugin) => {
|
||||
let servers = self.mcp_provider.load(&plugin).await.unwrap_or_else(|err| {
|
||||
tracing::warn!(
|
||||
selected_root = selected_root.id,
|
||||
error = %err,
|
||||
"failed to load selected executor plugin MCP servers"
|
||||
);
|
||||
}
|
||||
Vec::new()
|
||||
});
|
||||
let connector_ids = self
|
||||
.connector_provider
|
||||
.load(&plugin)
|
||||
.await
|
||||
.unwrap_or_else(|err| {
|
||||
tracing::warn!(
|
||||
selected_root = selected_root.id,
|
||||
error = %err,
|
||||
"failed to load selected executor plugin connectors"
|
||||
);
|
||||
Vec::new()
|
||||
})
|
||||
.into_iter()
|
||||
.map(|declaration| declaration.connector_id.0)
|
||||
.collect();
|
||||
Some(SelectedPluginMetadata {
|
||||
plugin_id: plugin.plugin().selected_root_id().to_string(),
|
||||
plugin_display_name: plugin.plugin().manifest().display_name().to_string(),
|
||||
servers,
|
||||
connector_ids,
|
||||
})
|
||||
}
|
||||
None => None,
|
||||
};
|
||||
let mut cache = state
|
||||
.cache
|
||||
.lock()
|
||||
.unwrap_or_else(std::sync::PoisonError::into_inner);
|
||||
if let Some(cached) = cache.iter().find(|cached| cached.root == *selected_root) {
|
||||
return cached.metadata.clone();
|
||||
}
|
||||
|
||||
snapshot
|
||||
cache.push(CachedSelectedRoot {
|
||||
root: selected_root.clone(),
|
||||
metadata: metadata.clone(),
|
||||
});
|
||||
metadata
|
||||
}
|
||||
}
|
||||
|
||||
@@ -102,20 +146,31 @@ impl McpServerContributor<Config> for SelectedExecutorPluginMcpContributor {
|
||||
let Some(thread_init) = context.thread_init() else {
|
||||
return Vec::new();
|
||||
};
|
||||
let Some(thread_store) = context.thread_store() else {
|
||||
return Vec::new();
|
||||
};
|
||||
let Some(selected_roots) = thread_init.get::<Vec<SelectedCapabilityRoot>>() else {
|
||||
return Vec::new();
|
||||
};
|
||||
let Some(state) = thread_init.get::<SelectedExecutorPluginMcpState>() else {
|
||||
tracing::warn!("selected executor plugin MCP state was not initialized");
|
||||
return Vec::new();
|
||||
};
|
||||
let snapshot = state
|
||||
.snapshot
|
||||
.get_or_init(|| self.resolve_snapshot(selected_roots.as_ref()))
|
||||
.await;
|
||||
let state = thread_store.get_or_init(SelectedExecutorPluginMcpState::default);
|
||||
let mut contributions = Vec::new();
|
||||
|
||||
for plugin in snapshot {
|
||||
for (selection_order, selected_root) in selected_roots.iter().enumerate() {
|
||||
let CapabilityRootLocation::Environment { environment_id, .. } =
|
||||
&selected_root.location;
|
||||
if context
|
||||
.available_environment_ids()
|
||||
.is_some_and(|available| {
|
||||
!available
|
||||
.iter()
|
||||
.any(|available| available == environment_id)
|
||||
})
|
||||
{
|
||||
continue;
|
||||
}
|
||||
let Some(plugin) = self.metadata_for_root(&state, selected_root).await else {
|
||||
continue;
|
||||
};
|
||||
let mut servers = plugin.servers.iter().cloned().collect::<HashMap<_, _>>();
|
||||
context
|
||||
.config()
|
||||
@@ -127,10 +182,17 @@ impl McpServerContributor<Config> for SelectedExecutorPluginMcpContributor {
|
||||
name,
|
||||
plugin_id: plugin.plugin_id.clone(),
|
||||
plugin_display_name: plugin.plugin_display_name.clone(),
|
||||
selection_order: plugin.selection_order,
|
||||
selection_order,
|
||||
config: Box::new(config),
|
||||
}
|
||||
}));
|
||||
if !plugin.connector_ids.is_empty() {
|
||||
contributions.push(McpServerContribution::SelectedPluginConnectors {
|
||||
plugin_id: plugin.plugin_id,
|
||||
plugin_display_name: plugin.plugin_display_name,
|
||||
connector_ids: plugin.connector_ids,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
contributions
|
||||
|
||||
@@ -51,10 +51,3 @@ pub fn install_executor_plugins(
|
||||
executor_plugin::SelectedExecutorPluginMcpContributor::new(environment_manager),
|
||||
));
|
||||
}
|
||||
|
||||
/// Seeds the per-thread snapshot used by selected executor plugin MCP discovery.
|
||||
pub fn initialize_executor_plugin_thread_data(
|
||||
thread_init: &mut codex_extension_api::ExtensionDataInit,
|
||||
) {
|
||||
executor_plugin::seed_thread_state(thread_init);
|
||||
}
|
||||
|
||||
@@ -3,6 +3,7 @@ use codex_core::config::Config;
|
||||
use codex_core::config::ConfigBuilder;
|
||||
use codex_exec_server::EnvironmentManager;
|
||||
use codex_exec_server::LOCAL_ENVIRONMENT_ID;
|
||||
use codex_extension_api::ExtensionData;
|
||||
use codex_extension_api::ExtensionDataInit;
|
||||
use codex_extension_api::ExtensionRegistryBuilder;
|
||||
use codex_extension_api::McpServerContribution;
|
||||
@@ -109,12 +110,15 @@ async fn selected_plugin_contributions(
|
||||
path: PathUri::from_host_native_path(plugin_root)?,
|
||||
},
|
||||
}]);
|
||||
codex_mcp_extension::initialize_executor_plugin_thread_data(&mut thread_init);
|
||||
let thread_store = ExtensionData::new_with_init("test-thread", thread_init.clone());
|
||||
let available_environment_ids = vec![LOCAL_ENVIRONMENT_ID.to_string()];
|
||||
|
||||
Ok(registry.mcp_server_contributors()[0]
|
||||
.contribute(McpServerContributionContext::for_thread(
|
||||
.contribute(McpServerContributionContext::for_step(
|
||||
config,
|
||||
&thread_init,
|
||||
&thread_store,
|
||||
&available_environment_ids,
|
||||
))
|
||||
.await
|
||||
.into_iter()
|
||||
@@ -132,7 +136,9 @@ async fn selected_plugin_contributions(
|
||||
selection_order,
|
||||
enabled: config.enabled,
|
||||
},
|
||||
McpServerContribution::Set { .. } | McpServerContribution::Remove { .. } => {
|
||||
McpServerContribution::Set { .. }
|
||||
| McpServerContribution::SelectedPluginConnectors { .. }
|
||||
| McpServerContribution::Remove { .. } => {
|
||||
panic!("expected selected plugin contribution")
|
||||
}
|
||||
})
|
||||
|
||||
Reference in New Issue
Block a user