mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
8f02973d25
## Why
`selectedCapabilityRoots` is durable thread intent: “use this capability
root from environment `worker`.”
The important product assumption is:
> One environment ID always names the same logical executor and stable
contents.
`worker` does not silently change from executor A to an unrelated
executor B. The process-local connection handle for `worker` can still
be replaced while Codex is running, though, for example when
`environment/add` registers a fresh handle for the same logical
environment.
The thread should persist only the stable selection. Each model step
should pair that selection with the exact ready handle captured for that
step.
## The boundary
```text
persisted thread intent
plugin@1 -> environment "worker"
|
| capture the current step
v
model-step view
unavailable, or
plugin@1 + worker's exact captured ready handle
```
The environment ID is the stable identity and cache key. The
`Arc<Environment>` is only a process-local handle retained so consumers
of one model step use the same captured environment. It is never
persisted and it does not imply different environment contents.
## What changes
### Persist the stable selection
Selected roots are written into `SessionMeta` and restored with the
thread. Forked subagents inherit the same selections, including
bounded-history forks.
Only stable data is persisted: root ID, environment ID, and root path.
### Capture readiness together with the exact handle
The environment snapshot records:
```rust
environment_id -> Some(Arc<Environment>) // ready in this step
environment_id -> None // still starting in this step
```
This prevents readiness and execution from coming from different
registry snapshots.
For example:
```text
step snapshot: worker -> handle A, ready
environment/add: worker -> fresh handle B for the same logical environment
current step: plugin@1 still uses captured handle A
```
Without carrying handle A in the snapshot, the resolver could combine “A
was ready” with handle B and treat B as ready before it had finished
starting.
This does not change cache invalidation. Stable capability metadata
remains identified by environment ID and capability root. Replacing a
process-local handle under the same stable environment ID does not
invalidate or rediscover that metadata.
### Resolve availability per model step
- A ready captured environment produces resolved roots using its
captured handle.
- A starting, missing, or failed environment is omitted from that step.
- A selected lazy environment that is outside the turn's captured
environment set is asked to start, and a later step can observe it as
ready.
- No capability files are scanned here.
Transient transport disconnects remain the remote client's reconnect
concern. This PR models initial attachment/readiness; it does not add
live socket-connectivity state.
## Example
```text
thread selection: plugin@1 -> environment "worker"
step 1: worker is starting -> plugin@1 unavailable
step 2: worker is ready -> plugin@1 resolves through worker's captured handle
step 3: fresh local handle -> current step remains pinned; a later step captures its own view
```
Temporary unavailability does not discard the durable selection. Later
PRs can retain stable metadata caches while projecting only currently
available capabilities into model-visible World State.
## Compatibility
The app-server request shape does not change. Older rollouts without
`selected_capability_roots` deserialize to an empty list.
## Stack
1. **This PR:** persist stable selected roots and resolve them through
an exact model-step handle.
2. #29960: cache stable skill metadata and project available skills into
World State.
3. #29946: cache stable plugin declarations and manage the separate live
MCP runtime.
335 lines
9.8 KiB
Rust
335 lines
9.8 KiB
Rust
use anyhow::Result;
|
|
use codex_protocol::SessionId;
|
|
use codex_protocol::ThreadId;
|
|
use codex_protocol::protocol::EventMsg;
|
|
use codex_protocol::protocol::GitInfo;
|
|
use codex_protocol::protocol::SessionMeta;
|
|
use codex_protocol::protocol::SessionMetaLine;
|
|
use codex_protocol::protocol::SessionSource;
|
|
use codex_protocol::protocol::TokenCountEvent;
|
|
use codex_protocol::protocol::TokenUsage;
|
|
use codex_protocol::protocol::TokenUsageInfo;
|
|
use serde_json::json;
|
|
use std::fs;
|
|
use std::fs::FileTimes;
|
|
use std::path::Path;
|
|
use std::path::PathBuf;
|
|
use uuid::Uuid;
|
|
|
|
pub fn rollout_path(codex_home: &Path, filename_ts: &str, thread_id: &str) -> PathBuf {
|
|
let year = &filename_ts[0..4];
|
|
let month = &filename_ts[5..7];
|
|
let day = &filename_ts[8..10];
|
|
codex_home
|
|
.join("sessions")
|
|
.join(year)
|
|
.join(month)
|
|
.join(day)
|
|
.join(format!("rollout-{filename_ts}-{thread_id}.jsonl"))
|
|
}
|
|
|
|
/// Create a minimal rollout file under `CODEX_HOME/sessions/YYYY/MM/DD/`.
|
|
///
|
|
/// - `filename_ts` is the filename timestamp component in `YYYY-MM-DDThh-mm-ss` format.
|
|
/// - `meta_rfc3339` is the envelope timestamp used in JSON lines.
|
|
/// - `preview` is the user message preview text.
|
|
/// - `model_provider` optionally sets the provider in the session meta payload.
|
|
///
|
|
/// Returns the generated conversation/session UUID as a string.
|
|
pub fn create_fake_rollout(
|
|
codex_home: &Path,
|
|
filename_ts: &str,
|
|
meta_rfc3339: &str,
|
|
preview: &str,
|
|
model_provider: Option<&str>,
|
|
git_info: Option<GitInfo>,
|
|
) -> Result<String> {
|
|
create_fake_rollout_with_source(
|
|
codex_home,
|
|
filename_ts,
|
|
meta_rfc3339,
|
|
preview,
|
|
model_provider,
|
|
git_info,
|
|
SessionSource::Cli,
|
|
)
|
|
}
|
|
|
|
/// Creates a minimal rollout whose history includes a persisted token usage event.
|
|
///
|
|
/// Resume and fork tests use this fixture to verify lifecycle replay of restored
|
|
/// usage without starting a model turn. The exact token values are intentionally
|
|
/// non-zero and asymmetric so assertions catch swapped total/last fields and
|
|
/// dropped cached or reasoning counters.
|
|
pub fn create_fake_rollout_with_token_usage(
|
|
codex_home: &Path,
|
|
filename_ts: &str,
|
|
meta_rfc3339: &str,
|
|
preview: &str,
|
|
model_provider: Option<&str>,
|
|
) -> Result<String> {
|
|
let thread_id = create_fake_rollout(
|
|
codex_home,
|
|
filename_ts,
|
|
meta_rfc3339,
|
|
preview,
|
|
model_provider,
|
|
/*git_info*/ None,
|
|
)?;
|
|
let payload = serde_json::to_value(EventMsg::TokenCount(TokenCountEvent {
|
|
info: Some(TokenUsageInfo {
|
|
total_token_usage: TokenUsage {
|
|
input_tokens: 120,
|
|
cached_input_tokens: 20,
|
|
output_tokens: 30,
|
|
reasoning_output_tokens: 10,
|
|
total_tokens: 150,
|
|
},
|
|
last_token_usage: TokenUsage {
|
|
input_tokens: 70,
|
|
cached_input_tokens: 10,
|
|
output_tokens: 20,
|
|
reasoning_output_tokens: 5,
|
|
total_tokens: 90,
|
|
},
|
|
model_context_window: Some(200_000),
|
|
}),
|
|
rate_limits: None,
|
|
}))?;
|
|
let file_path = rollout_path(codex_home, filename_ts, &thread_id);
|
|
let line = json!({
|
|
"timestamp": meta_rfc3339,
|
|
"type": "event_msg",
|
|
"payload": payload
|
|
})
|
|
.to_string();
|
|
fs::write(
|
|
&file_path,
|
|
format!("{}{}\n", fs::read_to_string(&file_path)?, line),
|
|
)?;
|
|
Ok(thread_id)
|
|
}
|
|
|
|
/// Create a minimal rollout file with an explicit session source.
|
|
pub fn create_fake_rollout_with_source(
|
|
codex_home: &Path,
|
|
filename_ts: &str,
|
|
meta_rfc3339: &str,
|
|
preview: &str,
|
|
model_provider: Option<&str>,
|
|
git_info: Option<GitInfo>,
|
|
source: SessionSource,
|
|
) -> Result<String> {
|
|
create_fake_rollout_with_source_and_parent_thread_id(
|
|
codex_home,
|
|
filename_ts,
|
|
meta_rfc3339,
|
|
preview,
|
|
model_provider,
|
|
git_info,
|
|
source,
|
|
/*session_id*/ None,
|
|
/*parent_thread_id*/ None,
|
|
)
|
|
}
|
|
|
|
/// Create a minimal rollout file with an explicit root session and control parent.
|
|
#[allow(clippy::too_many_arguments)]
|
|
pub fn create_fake_parented_rollout_with_source(
|
|
codex_home: &Path,
|
|
filename_ts: &str,
|
|
meta_rfc3339: &str,
|
|
preview: &str,
|
|
model_provider: Option<&str>,
|
|
git_info: Option<GitInfo>,
|
|
source: SessionSource,
|
|
session_id: SessionId,
|
|
parent_thread_id: ThreadId,
|
|
) -> Result<String> {
|
|
create_fake_rollout_with_source_and_parent_thread_id(
|
|
codex_home,
|
|
filename_ts,
|
|
meta_rfc3339,
|
|
preview,
|
|
model_provider,
|
|
git_info,
|
|
source,
|
|
Some(session_id),
|
|
Some(parent_thread_id),
|
|
)
|
|
}
|
|
|
|
#[allow(clippy::too_many_arguments)]
|
|
fn create_fake_rollout_with_source_and_parent_thread_id(
|
|
codex_home: &Path,
|
|
filename_ts: &str,
|
|
meta_rfc3339: &str,
|
|
preview: &str,
|
|
model_provider: Option<&str>,
|
|
git_info: Option<GitInfo>,
|
|
source: SessionSource,
|
|
session_id: Option<SessionId>,
|
|
parent_thread_id: Option<ThreadId>,
|
|
) -> Result<String> {
|
|
let uuid = Uuid::new_v4();
|
|
let uuid_str = uuid.to_string();
|
|
let conversation_id = ThreadId::from_string(&uuid_str)?;
|
|
let session_id = session_id.unwrap_or_else(|| conversation_id.into());
|
|
|
|
let file_path = rollout_path(codex_home, filename_ts, &uuid_str);
|
|
let dir = file_path
|
|
.parent()
|
|
.ok_or_else(|| anyhow::anyhow!("missing rollout parent directory"))?;
|
|
fs::create_dir_all(dir)?;
|
|
|
|
// Build JSONL lines
|
|
let meta = SessionMeta {
|
|
session_id,
|
|
id: conversation_id,
|
|
forked_from_id: None,
|
|
parent_thread_id,
|
|
timestamp: meta_rfc3339.to_string(),
|
|
cwd: PathBuf::from("/"),
|
|
originator: "codex".to_string(),
|
|
cli_version: "0.0.0".to_string(),
|
|
source,
|
|
thread_source: None,
|
|
agent_path: None,
|
|
agent_nickname: None,
|
|
agent_role: None,
|
|
model_provider: model_provider.map(str::to_string),
|
|
base_instructions: None,
|
|
dynamic_tools: None,
|
|
selected_capability_roots: Vec::new(),
|
|
memory_mode: None,
|
|
multi_agent_version: None,
|
|
context_window: None,
|
|
};
|
|
let payload = serde_json::to_value(SessionMetaLine {
|
|
meta,
|
|
git: git_info,
|
|
})?;
|
|
|
|
let lines = [
|
|
json!({
|
|
"timestamp": meta_rfc3339,
|
|
"type": "session_meta",
|
|
"payload": payload
|
|
})
|
|
.to_string(),
|
|
json!({
|
|
"timestamp": meta_rfc3339,
|
|
"type":"response_item",
|
|
"payload": {
|
|
"type":"message",
|
|
"role":"user",
|
|
"content":[{"type":"input_text","text": preview}]
|
|
}
|
|
})
|
|
.to_string(),
|
|
json!({
|
|
"timestamp": meta_rfc3339,
|
|
"type":"event_msg",
|
|
"payload": {
|
|
"type":"user_message",
|
|
"message": preview,
|
|
"kind": "plain"
|
|
}
|
|
})
|
|
.to_string(),
|
|
];
|
|
|
|
fs::write(&file_path, lines.join("\n") + "\n")?;
|
|
let parsed = chrono::DateTime::parse_from_rfc3339(meta_rfc3339)?.with_timezone(&chrono::Utc);
|
|
let times = FileTimes::new().set_modified(parsed.into());
|
|
std::fs::OpenOptions::new()
|
|
.append(true)
|
|
.open(&file_path)?
|
|
.set_times(times)?;
|
|
Ok(uuid_str)
|
|
}
|
|
|
|
pub fn create_fake_rollout_with_text_elements(
|
|
codex_home: &Path,
|
|
filename_ts: &str,
|
|
meta_rfc3339: &str,
|
|
preview: &str,
|
|
text_elements: Vec<serde_json::Value>,
|
|
model_provider: Option<&str>,
|
|
git_info: Option<GitInfo>,
|
|
) -> Result<String> {
|
|
let uuid = Uuid::new_v4();
|
|
let uuid_str = uuid.to_string();
|
|
let conversation_id = ThreadId::from_string(&uuid_str)?;
|
|
|
|
// sessions/YYYY/MM/DD derived from filename_ts (YYYY-MM-DDThh-mm-ss)
|
|
let year = &filename_ts[0..4];
|
|
let month = &filename_ts[5..7];
|
|
let day = &filename_ts[8..10];
|
|
let dir = codex_home.join("sessions").join(year).join(month).join(day);
|
|
fs::create_dir_all(&dir)?;
|
|
|
|
let file_path = dir.join(format!("rollout-{filename_ts}-{uuid}.jsonl"));
|
|
|
|
// Build JSONL lines
|
|
let meta = SessionMeta {
|
|
session_id: conversation_id.into(),
|
|
id: conversation_id,
|
|
forked_from_id: None,
|
|
parent_thread_id: None,
|
|
timestamp: meta_rfc3339.to_string(),
|
|
cwd: PathBuf::from("/"),
|
|
originator: "codex".to_string(),
|
|
cli_version: "0.0.0".to_string(),
|
|
source: SessionSource::Cli,
|
|
thread_source: None,
|
|
agent_path: None,
|
|
agent_nickname: None,
|
|
agent_role: None,
|
|
model_provider: model_provider.map(str::to_string),
|
|
base_instructions: None,
|
|
dynamic_tools: None,
|
|
selected_capability_roots: Vec::new(),
|
|
memory_mode: None,
|
|
multi_agent_version: None,
|
|
context_window: None,
|
|
};
|
|
let payload = serde_json::to_value(SessionMetaLine {
|
|
meta,
|
|
git: git_info,
|
|
})?;
|
|
|
|
let lines = [
|
|
json!( {
|
|
"timestamp": meta_rfc3339,
|
|
"type": "session_meta",
|
|
"payload": payload
|
|
})
|
|
.to_string(),
|
|
json!( {
|
|
"timestamp": meta_rfc3339,
|
|
"type":"response_item",
|
|
"payload": {
|
|
"type":"message",
|
|
"role":"user",
|
|
"content":[{"type":"input_text","text": preview}]
|
|
}
|
|
})
|
|
.to_string(),
|
|
json!( {
|
|
"timestamp": meta_rfc3339,
|
|
"type":"event_msg",
|
|
"payload": {
|
|
"type":"user_message",
|
|
"message": preview,
|
|
"text_elements": text_elements,
|
|
"local_images": []
|
|
}
|
|
})
|
|
.to_string(),
|
|
];
|
|
|
|
fs::write(file_path, lines.join("\n") + "\n")?;
|
|
Ok(uuid_str)
|
|
}
|