mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
Add realtime speech append control (#27917)
## Why Realtime voice harness tuning needs app-side control over what backend Codex text is spoken. Backend orchestrator text is written for a reading UI, so automatically speaking every preamble, progress update, or final assistant message can make the realtime voice model too chatty. For experimentation, clients need two simple controls: keep app/client text-item injection on the existing item-create path, and add an explicit speakable path that app code can call only when it wants realtime to speak. Automatic Codex output also needs an opt-in way to switch from the protocol's default speakable path to regular realtime items, with a caller-provided prefix so prompt wording can be tuned outside core. The default remains unchanged: if a client omits the new start fields and never calls `appendSpeech`, automatic backend output continues down the existing speakable path for the selected realtime protocol. ## What Changed - Adds experimental `thread/realtime/appendSpeech` for app-provided speakable text. - Keeps existing `thread/realtime/appendText` as the item-create API for app-provided realtime text items. - Adds `codexResponsesAsItems` / `codex_responses_as_items` on `thread/realtime/start` to send automatic Codex responses with `conversation.item.create` instead of the protocol's default speakable output path. - Adds `codexResponseItemPrefix` / `codex_response_item_prefix` so clients can prepend experiment instructions to those automatic Codex response items. - Keeps literal `conversation.handoff.append` routing scoped to the v1 speakable path; v2 default speech uses its item/function-output plus `response.create` behavior. - Removes the earlier public silent-context API and hardcoded silent-context prefix. - Updates realtime tests to cover default automatic speakable behavior, opt-in automatic item-create behavior, and explicit `appendSpeech` behavior. ## Validation - `cargo check -p codex-core -p codex-app-server -p codex-api` - `just test -p codex-app-server realtime_conversation` - `just test -p codex-core realtime_conversation` (50/51 passed in the filtered parallel run; the lone failure passed when rerun in isolation) - `just test -p codex-core conversation_mirrors_assistant_message_text_to_realtime_handoff` - `just test -p codex-api e2e_connect_and_exchange_events_against_mock_ws_server` - `just fix -p codex-core` - `just fix -p codex-app-server` - `cargo build -p codex-cli`
This commit is contained in:
committed by
GitHub
Unverified
parent
9728992fab
commit
1d8ff89aa3
@@ -182,6 +182,10 @@ pub struct McpServerRefreshConfig {
|
||||
pub struct ConversationStartParams {
|
||||
/// Overrides the configured realtime architecture for this session only.
|
||||
pub architecture: Option<RealtimeConversationArchitecture>,
|
||||
/// Sends automatic Codex responses as realtime conversation items instead of handoff appends.
|
||||
pub codex_responses_as_items: bool,
|
||||
/// Optional prefix added to automatic Codex response items when `codex_responses_as_items` is set.
|
||||
pub codex_response_item_prefix: Option<String>,
|
||||
/// Overrides the configured realtime model for this session only.
|
||||
pub model: Option<String>,
|
||||
/// Selects whether the realtime session should produce text or audio output.
|
||||
@@ -407,6 +411,11 @@ pub enum ConversationTextRole {
|
||||
Developer,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, PartialEq)]
|
||||
pub struct ConversationSpeechParams {
|
||||
pub text: String,
|
||||
}
|
||||
|
||||
/// Persistent thread-settings overrides that can be applied before user input or
|
||||
/// on their own.
|
||||
#[derive(Debug, Clone, Default, PartialEq)]
|
||||
@@ -503,6 +512,9 @@ pub enum Op {
|
||||
/// Send text input to the running realtime conversation stream.
|
||||
RealtimeConversationText(ConversationTextParams),
|
||||
|
||||
/// Append speakable text to the running realtime conversation stream.
|
||||
RealtimeConversationSpeech(ConversationSpeechParams),
|
||||
|
||||
/// Close the running realtime conversation stream.
|
||||
RealtimeConversationClose,
|
||||
|
||||
@@ -762,6 +774,7 @@ impl Op {
|
||||
Self::RealtimeConversationStart(_) => "realtime_conversation_start",
|
||||
Self::RealtimeConversationAudio(_) => "realtime_conversation_audio",
|
||||
Self::RealtimeConversationText(_) => "realtime_conversation_text",
|
||||
Self::RealtimeConversationSpeech(_) => "realtime_conversation_speech",
|
||||
Self::RealtimeConversationClose => "realtime_conversation_close",
|
||||
Self::RealtimeConversationListVoices => "realtime_conversation_list_voices",
|
||||
Self::UserInput { .. } => "user_input",
|
||||
|
||||
Reference in New Issue
Block a user