mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
a92a5085bd
## Summary
App-server v2 already receives turn-scoped `clientMetadata`, but the
Rust app-server was dropping it before the outbound Responses request.
This change keeps the fix lightweight by threading that metadata through
the existing turn-metadata path rather than inventing a new transport.
## What we're trying to do and why
We want turn-scoped metadata from the app-server protocol layer,
especially fields like Hermes/GAAS run IDs, to survive all the way to
the actual Responses API request so it is visible in downstream
websocket request logging and analytics.
The specific bug was:
- app-server protocol uses camelCase `clientMetadata`
- Responses transport already has an existing turn metadata carrier:
`x-codex-turn-metadata`
- websocket transport already rewrites that header into
`request.request_body.client_metadata["x-codex-turn-metadata"]`
- but the Rust app-server never parsed or stored `clientMetadata`, so
nothing from the app-server request was making it into that existing
path
This PR fixes that without adding a new header or a second metadata
channel.
## How we did it
### Protocol surface
- Add optional `clientMetadata` to v2 `TurnStartParams` and
`TurnSteerParams`
- Regenerate the JSON schema / TypeScript fixtures
- Update app-server docs to describe the field and its behavior
### Runtime plumbing
- Add a dedicated core op for app-server user input carrying turn-scoped
metadata: `Op::UserInputWithClientMetadata`
- Wire `turn/start` and `turn/steer` through that op / signature path
instead of dropping the metadata at the message-processor boundary
- Store the metadata in `TurnMetadataState`
### Transport behavior
- Reuse the existing serialized `x-codex-turn-metadata` payload
- Merge the new app-server `clientMetadata` into that JSON additively
- Do **not** replace built-in reserved fields already present in the
turn metadata payload
- Keep websocket behavior unchanged at the outer shape level: it still
sends only `client_metadata["x-codex-turn-metadata"]`, but that JSON
string now contains the merged fields
- Keep HTTP fallback behavior unchanged except that the existing
`x-codex-turn-metadata` header now includes the merged fields too
### Request shape before / after
Before, a websocket `response.create` looked like:
```json
{
"type": "response.create",
"client_metadata": {
"x-codex-turn-metadata": "{\"session_id\":\"...\",\"turn_id\":\"...\"}"
}
}
```
Even if the app-server caller supplied `clientMetadata`, it was not
represented there.
After, the same request shape is preserved, but the serialized payload
now includes the new turn-scoped fields:
```json
{
"type": "response.create",
"client_metadata": {
"x-codex-turn-metadata": "{\"session_id\":\"...\",\"turn_id\":\"...\",\"fiber_run_id\":\"fiber-start-123\",\"origin\":\"gaas\"}"
}
}
```
## Validation
### Targeted tests added / updated
- protocol round-trip coverage for `clientMetadata` on `turn/start` and
`turn/steer`
- protocol round-trip coverage for `Op::UserInputWithClientMetadata`
- `TurnMetadataState` merge test proving client metadata is added
without overwriting reserved built-in fields
- websocket request-shape test proving outbound `response.create`
contains merged metadata inside
`client_metadata["x-codex-turn-metadata"]`
- app-server integration tests proving:
- `turn/start` forwards `clientMetadata` into the outbound Responses
request path
- websocket warmup + real turn request both behave correctly
- `turn/steer` updates the follow-up request metadata
### Commands run
- `just write-app-server-schema`
- `cargo test -p codex-app-server-protocol`
- `cargo test -p codex-protocol`
- `cargo test -p codex-core
turn_metadata_state_merges_client_metadata_without_replacing_reserved_fields
--lib`
- `cargo test -p codex-core --test all
responses_websocket_preserves_custom_turn_metadata_fields`
- `cargo test -p codex-app-server --test all client_metadata`
- `cargo test -p codex-app-server --test all
turn_start_forwards_client_metadata_to_responses_websocket_request_body_v2
-- --nocapture`
- `just fmt`
- `just fix -p codex-core -p codex-protocol -p codex-app-server-protocol
-p codex-app-server`
- `just fix -p codex-exec -p codex-tui-app-server`
- `just argument-comment-lint`
### Full suite note
`cargo test` in `codex-rs` still fails in:
-
`suite::v2::turn_interrupt::turn_interrupt_resolves_pending_command_approval_request`
I verified that same failure on a clean detached `HEAD` worktree with an
isolated `CARGO_TARGET_DIR`, so it is not caused by this patch.
278 lines
8.7 KiB
Rust
278 lines
8.7 KiB
Rust
use crate::agent::AgentStatus;
|
|
use crate::codex::Codex;
|
|
use crate::codex::SteerInputError;
|
|
use crate::config::ConstraintResult;
|
|
use crate::file_watcher::WatchRegistration;
|
|
use codex_features::Feature;
|
|
use codex_protocol::config_types::ApprovalsReviewer;
|
|
use codex_protocol::config_types::Personality;
|
|
use codex_protocol::config_types::ServiceTier;
|
|
use codex_protocol::error::CodexErr;
|
|
use codex_protocol::error::Result as CodexResult;
|
|
use codex_protocol::models::ContentItem;
|
|
use codex_protocol::models::ResponseInputItem;
|
|
use codex_protocol::models::ResponseItem;
|
|
use codex_protocol::openai_models::ReasoningEffort;
|
|
use codex_protocol::protocol::AskForApproval;
|
|
use codex_protocol::protocol::Event;
|
|
use codex_protocol::protocol::Op;
|
|
use codex_protocol::protocol::SandboxPolicy;
|
|
use codex_protocol::protocol::SessionSource;
|
|
use codex_protocol::protocol::Submission;
|
|
use codex_protocol::protocol::TokenUsage;
|
|
use codex_protocol::protocol::W3cTraceContext;
|
|
use codex_protocol::user_input::UserInput;
|
|
use rmcp::model::ReadResourceRequestParams;
|
|
use std::collections::HashMap;
|
|
use std::path::PathBuf;
|
|
use tokio::sync::Mutex;
|
|
use tokio::sync::watch;
|
|
|
|
use codex_rollout::state_db::StateDbHandle;
|
|
|
|
#[derive(Clone, Debug)]
|
|
pub struct ThreadConfigSnapshot {
|
|
pub model: String,
|
|
pub model_provider_id: String,
|
|
pub service_tier: Option<ServiceTier>,
|
|
pub approval_policy: AskForApproval,
|
|
pub approvals_reviewer: ApprovalsReviewer,
|
|
pub sandbox_policy: SandboxPolicy,
|
|
pub cwd: PathBuf,
|
|
pub ephemeral: bool,
|
|
pub reasoning_effort: Option<ReasoningEffort>,
|
|
pub personality: Option<Personality>,
|
|
pub session_source: SessionSource,
|
|
}
|
|
|
|
pub struct CodexThread {
|
|
pub(crate) codex: Codex,
|
|
rollout_path: Option<PathBuf>,
|
|
out_of_band_elicitation_count: Mutex<u64>,
|
|
_watch_registration: WatchRegistration,
|
|
}
|
|
|
|
/// Conduit for the bidirectional stream of messages that compose a thread
|
|
/// (formerly called a conversation) in Codex.
|
|
impl CodexThread {
|
|
pub(crate) fn new(
|
|
codex: Codex,
|
|
rollout_path: Option<PathBuf>,
|
|
watch_registration: WatchRegistration,
|
|
) -> Self {
|
|
Self {
|
|
codex,
|
|
rollout_path,
|
|
out_of_band_elicitation_count: Mutex::new(0),
|
|
_watch_registration: watch_registration,
|
|
}
|
|
}
|
|
|
|
pub async fn submit(&self, op: Op) -> CodexResult<String> {
|
|
self.codex.submit(op).await
|
|
}
|
|
|
|
pub async fn shutdown_and_wait(&self) -> CodexResult<()> {
|
|
self.codex.shutdown_and_wait().await
|
|
}
|
|
|
|
#[doc(hidden)]
|
|
pub async fn ensure_rollout_materialized(&self) {
|
|
self.codex.session.ensure_rollout_materialized().await;
|
|
}
|
|
|
|
#[doc(hidden)]
|
|
pub async fn flush_rollout(&self) {
|
|
self.codex.session.flush_rollout().await;
|
|
}
|
|
|
|
pub async fn submit_with_trace(
|
|
&self,
|
|
op: Op,
|
|
trace: Option<W3cTraceContext>,
|
|
) -> CodexResult<String> {
|
|
self.codex.submit_with_trace(op, trace).await
|
|
}
|
|
|
|
pub async fn steer_input(
|
|
&self,
|
|
input: Vec<UserInput>,
|
|
expected_turn_id: Option<&str>,
|
|
responsesapi_client_metadata: Option<HashMap<String, String>>,
|
|
) -> Result<String, SteerInputError> {
|
|
self.codex
|
|
.steer_input(input, expected_turn_id, responsesapi_client_metadata)
|
|
.await
|
|
}
|
|
|
|
pub async fn set_app_server_client_info(
|
|
&self,
|
|
app_server_client_name: Option<String>,
|
|
app_server_client_version: Option<String>,
|
|
) -> ConstraintResult<()> {
|
|
self.codex
|
|
.set_app_server_client_info(app_server_client_name, app_server_client_version)
|
|
.await
|
|
}
|
|
|
|
/// Use sparingly: this is intended to be removed soon.
|
|
pub async fn submit_with_id(&self, sub: Submission) -> CodexResult<()> {
|
|
self.codex.submit_with_id(sub).await
|
|
}
|
|
|
|
pub async fn next_event(&self) -> CodexResult<Event> {
|
|
self.codex.next_event().await
|
|
}
|
|
|
|
pub async fn agent_status(&self) -> AgentStatus {
|
|
self.codex.agent_status().await
|
|
}
|
|
|
|
pub(crate) fn subscribe_status(&self) -> watch::Receiver<AgentStatus> {
|
|
self.codex.agent_status.clone()
|
|
}
|
|
|
|
pub(crate) async fn total_token_usage(&self) -> Option<TokenUsage> {
|
|
self.codex.session.total_token_usage().await
|
|
}
|
|
|
|
/// Records a user-role session-prefix message without creating a new user turn boundary.
|
|
pub(crate) async fn inject_user_message_without_turn(&self, message: String) {
|
|
let message = ResponseItem::Message {
|
|
id: None,
|
|
role: "user".to_string(),
|
|
content: vec![ContentItem::InputText { text: message }],
|
|
end_turn: None,
|
|
phase: None,
|
|
};
|
|
let pending_item = match pending_message_input_item(&message) {
|
|
Ok(pending_item) => pending_item,
|
|
Err(err) => {
|
|
debug_assert!(false, "session-prefix message append should succeed: {err}");
|
|
return;
|
|
}
|
|
};
|
|
if self
|
|
.codex
|
|
.session
|
|
.inject_response_items(vec![pending_item])
|
|
.await
|
|
.is_err()
|
|
{
|
|
let turn_context = self.codex.session.new_default_turn().await;
|
|
self.codex
|
|
.session
|
|
.record_conversation_items(turn_context.as_ref(), &[message])
|
|
.await;
|
|
}
|
|
}
|
|
|
|
/// Append a prebuilt message to the thread history without treating it as a user turn.
|
|
///
|
|
/// If the thread already has an active turn, the message is queued as pending input for that
|
|
/// turn. Otherwise it is queued at session scope and a regular turn is started so the agent
|
|
/// can consume that pending input through the normal turn pipeline.
|
|
#[cfg(test)]
|
|
pub(crate) async fn append_message(&self, message: ResponseItem) -> CodexResult<String> {
|
|
let submission_id = uuid::Uuid::new_v4().to_string();
|
|
let pending_item = pending_message_input_item(&message)?;
|
|
if let Err(items) = self
|
|
.codex
|
|
.session
|
|
.inject_response_items(vec![pending_item])
|
|
.await
|
|
{
|
|
self.codex
|
|
.session
|
|
.queue_response_items_for_next_turn(items)
|
|
.await;
|
|
self.codex.session.maybe_start_turn_for_pending_work().await;
|
|
}
|
|
|
|
Ok(submission_id)
|
|
}
|
|
|
|
pub fn rollout_path(&self) -> Option<PathBuf> {
|
|
self.rollout_path.clone()
|
|
}
|
|
|
|
pub fn state_db(&self) -> Option<StateDbHandle> {
|
|
self.codex.state_db()
|
|
}
|
|
|
|
pub async fn config_snapshot(&self) -> ThreadConfigSnapshot {
|
|
self.codex.thread_config_snapshot().await
|
|
}
|
|
|
|
pub async fn read_mcp_resource(
|
|
&self,
|
|
server: &str,
|
|
uri: &str,
|
|
) -> anyhow::Result<serde_json::Value> {
|
|
let result = self
|
|
.codex
|
|
.session
|
|
.read_resource(
|
|
server,
|
|
ReadResourceRequestParams {
|
|
meta: None,
|
|
uri: uri.to_string(),
|
|
},
|
|
)
|
|
.await?;
|
|
|
|
Ok(serde_json::to_value(result)?)
|
|
}
|
|
|
|
pub fn enabled(&self, feature: Feature) -> bool {
|
|
self.codex.enabled(feature)
|
|
}
|
|
|
|
pub async fn increment_out_of_band_elicitation_count(&self) -> CodexResult<u64> {
|
|
let mut guard = self.out_of_band_elicitation_count.lock().await;
|
|
let was_zero = *guard == 0;
|
|
*guard = guard.checked_add(1).ok_or_else(|| {
|
|
CodexErr::Fatal("out-of-band elicitation count overflowed".to_string())
|
|
})?;
|
|
|
|
if was_zero {
|
|
self.codex
|
|
.session
|
|
.set_out_of_band_elicitation_pause_state(/*paused*/ true);
|
|
}
|
|
|
|
Ok(*guard)
|
|
}
|
|
|
|
pub async fn decrement_out_of_band_elicitation_count(&self) -> CodexResult<u64> {
|
|
let mut guard = self.out_of_band_elicitation_count.lock().await;
|
|
if *guard == 0 {
|
|
return Err(CodexErr::InvalidRequest(
|
|
"out-of-band elicitation count is already zero".to_string(),
|
|
));
|
|
}
|
|
|
|
*guard -= 1;
|
|
let now_zero = *guard == 0;
|
|
if now_zero {
|
|
self.codex
|
|
.session
|
|
.set_out_of_band_elicitation_pause_state(/*paused*/ false);
|
|
}
|
|
|
|
Ok(*guard)
|
|
}
|
|
}
|
|
|
|
fn pending_message_input_item(message: &ResponseItem) -> CodexResult<ResponseInputItem> {
|
|
match message {
|
|
ResponseItem::Message { role, content, .. } => Ok(ResponseInputItem::Message {
|
|
role: role.clone(),
|
|
content: content.clone(),
|
|
}),
|
|
_ => Err(CodexErr::InvalidRequest(
|
|
"append_message only supports ResponseItem::Message".to_string(),
|
|
)),
|
|
}
|
|
}
|