22 Commits

• Always use AVAS for realtime WebRTC calls (#28856)

  ## Summary
  
  - Remove the realtime `architecture` selector from core protocol,
  app-server protocol, config parsing, generated schemas, and callers.
  - Always create WebRTC realtime calls with the AVAS query params:
  `intent=quicksilver&architecture=avas`.
  - Keep direct websocket realtime behavior on the existing config/default
  path, while WebRTC starts without an explicit version now default to
  realtime v1 because AVAS requires v1.
  
  ## Notes
  
  - WebRTC realtime now means AVAS. If a caller explicitly asks to start
  WebRTC with realtime v2, Codex rejects that request because the AVAS
  WebRTC path only supports realtime v1. Websocket realtime is separate
  and can still use realtime v2.
  - The old `[realtime] architecture = "realtimeapi" | "avas"` config knob
  is removed. Local configs that still set it will need to delete that
  line.
  - Some app-server tests that were only trying to exercise realtime v2
  protocol behavior now use websocket transport, because WebRTC is
  intentionally locked to AVAS/v1. Separate WebRTC tests cover the AVAS
  query params, v1 startup, SDP flow, and sideband join.
  
  ## Validation
  
  - Merged fresh `origin/main` at `83e6a786a2`.
  - `just fmt`
  - `just write-config-schema`
  - `just write-app-server-schema`
  - `git diff --check`
  - `just test -p codex-api -p codex-core -p codex-app-server-protocol -p
  codex-app-server realtime` (176 passed)
  - `just test -p codex-protocol -p codex-config` (413 passed)

  Peter Bakkum · 2026-06-18 19:11:21 -05:00
  8e7c213f8f
• Support openai/form extended form elicitations (#27500)

  # Summary
  Allow App Server clients to opt into `openai/form` MCP elicitations.

  Gabriel Peal · 2026-06-18 11:54:49 -07:00
  21a599fa56
• [codex] control automatic realtime handoff delivery (#27986)

  ## What
  
  Built on the realtime speech-control plumbing merged in #27917.
  
  - Add optional `codexResponseHandoffPrefix` to `thread/realtime/start`.
  - Apply that prefix only to automatic V1 commentary sent through
  `conversation.handoff.append`; final answers remain unprefixed.
  - Add opt-in `clientManagedHandoffs`. When true, core suppresses
  automatic response handoffs and completion output so delivery is
  controlled by explicit client append APIs.
  - Preserve existing automatic behavior by default.
  `codexResponsesAsItems: true` continues to select item routing when
  client-managed mode is disabled.
  
  ## Why
  
  Voice clients need two delivery policies: automatic background context
  with silent commentary instructions and fully client-owned handoffs.
  Phase-aware prefixing keeps routine commentary silent without
  suppressing the final answer, while client-managed mode lets an app
  decide exactly which updates to append.
  
  ## Validation
  
  - `just fmt`
  - `cargo test -p codex-app-server-protocol
  serialize_thread_realtime_start`
  - `RUST_MIN_STACK=16777216 cargo test -p codex-core --test all
  conversation_handoff_persists_across_item_done_until_turn_complete`
  - `RUST_MIN_STACK=16777216 cargo test -p codex-app-server --test all
  webrtc_v1_client_managed_handoffs_disable_automatic_output`
  - `RUST_MIN_STACK=16777216 cargo test -p codex-app-server --test all
  webrtc_v1_final_automatic_handoff_omits_silent_prefix`
  - `cargo build -p codex-cli --bin codex`
  - Local Codex Apps compatibility check: 43 focused webview tests passed,
  and a live voice session routed through the source-built app-server.
  
  The explicit `RUST_MIN_STACK` avoids a macOS Tokio test-worker stack
  overflow seen with the default test environment.

  jiayuhuang-openai · 2026-06-18 02:22:29 +00:00
  683bd170dc
• Add a toggle for realtime startup context (#28405)

  ## Summary
  - Add `includeStartupContext` to realtime start requests so callers can
  explicitly skip Codex startup context while keeping the backend prompt
  - Thread the new flag through protocol types, request processing, and
  realtime session config
  - Update app-server docs and coverage for the new default and opt-out
  behavior
  
  ## Testing
  - Added protocol serialization coverage for `includeStartupContext`
  - Added realtime integration coverage for starting a session with
  startup context disabled

  guinness-oai · 2026-06-15 17:14:22 -07:00
  d5b4b98370
• Add realtime speech append control (#27917)

  ## Why
  
  Realtime voice harness tuning needs app-side control over what backend
  Codex text is spoken. Backend orchestrator text is written for a reading
  UI, so automatically speaking every preamble, progress update, or final
  assistant message can make the realtime voice model too chatty.
  
  For experimentation, clients need two simple controls: keep app/client
  text-item injection on the existing item-create path, and add an
  explicit speakable path that app code can call only when it wants
  realtime to speak. Automatic Codex output also needs an opt-in way to
  switch from the protocol's default speakable path to regular realtime
  items, with a caller-provided prefix so prompt wording can be tuned
  outside core.
  
  The default remains unchanged: if a client omits the new start fields
  and never calls `appendSpeech`, automatic backend output continues down
  the existing speakable path for the selected realtime protocol.
  
  ## What Changed
  
  - Adds experimental `thread/realtime/appendSpeech` for app-provided
  speakable text.
  - Keeps existing `thread/realtime/appendText` as the item-create API for
  app-provided realtime text items.
  - Adds `codexResponsesAsItems` / `codex_responses_as_items` on
  `thread/realtime/start` to send automatic Codex responses with
  `conversation.item.create` instead of the protocol's default speakable
  output path.
  - Adds `codexResponseItemPrefix` / `codex_response_item_prefix` so
  clients can prepend experiment instructions to those automatic Codex
  response items.
  - Keeps literal `conversation.handoff.append` routing scoped to the v1
  speakable path; v2 default speech uses its item/function-output plus
  `response.create` behavior.
  - Removes the earlier public silent-context API and hardcoded
  silent-context prefix.
  - Updates realtime tests to cover default automatic speakable behavior,
  opt-in automatic item-create behavior, and explicit `appendSpeech`
  behavior.
  
  ## Validation
  
  - `cargo check -p codex-core -p codex-app-server -p codex-api`
  - `just test -p codex-app-server realtime_conversation`
  - `just test -p codex-core realtime_conversation` (50/51 passed in the
  filtered parallel run; the lone failure passed when rerun in isolation)
  - `just test -p codex-core
  conversation_mirrors_assistant_message_text_to_realtime_handoff`
  - `just test -p codex-api
  e2e_connect_and_exchange_events_against_mock_ws_server`
  - `just fix -p codex-core`
  - `just fix -p codex-app-server`
  - `cargo build -p codex-cli`

  guinness-oai · 2026-06-15 16:15:58 -07:00
  1d8ff89aa3
• realtime: add AVAS architecture override (#27720)

  ## Summary
  
  Adds a `RealtimeConversationArchitecture` option for realtime
  conversation startup, with `realtimeapi` as the default and `avas` as an
  opt-in architecture.
  
  The AVAS path is limited to realtime v1 conversational WebRTC starts,
  and WebRTC call creation appends `intent=quicksilver&architecture=avas`
  to `/v1/realtime/calls`. The existing sideband websocket still joins by
  `call_id`.
  
  This also exposes the per-session architecture override through
  app-server v2 `thread/realtime/start` params and updates the config
  schema for `[realtime].architecture`.
  
  ## Validation
  
  - `just fmt`
  - `just write-config-schema`
  - `just test -p codex-api sends_avas_session_call_query_params`
  - `just test -p codex-core -E
  'test(~conversation_webrtc_start_uses_avas_architecture_query)'`
  - `just test -p codex-core -E 'test(realtime_loads_from_config_toml)'`
  - `just test -p codex-app-server-protocol -E
  'test(~serialize_thread_realtime_start) |
  test(generated_ts_optional_nullable_fields_only_in_params)'`
  - `just test -p codex-app-server -E
  'test(realtime_webrtc_start_emits_sdp_notification)'`

  Peter Bakkum · 2026-06-12 18:11:13 +00:00
  6652e82dd0
• Add per-session realtime model and version overrides (#24999)

  ## Why
  
  Clients need to select a realtime session configuration for an
  individual start without rewriting persisted configuration or restarting
  the app-server process.
  
  ## What Changed
  
  - Add optional `model` and `version` fields to `thread/realtime/start`
  - Forward those optional values through the realtime start operation and
  apply them only for that session
  - Preserve existing configured/default behavior when the new fields are
  omitted
  - Update generated protocol schema and app-server documentation
  
  ## Validation
  
  - Added/updated protocol serialization coverage for the new optional
  request fields
  - Added focused core coverage for a session override taking precedence
  over configured realtime selection
  - Added focused app-server coverage that a request override reaches the
  realtime WebSocket handshake

  guinness-oai · 2026-06-09 17:54:32 -07:00
  4a3eac2144
• fix: rename McpServer to TestAppServer (#25701)

  This PR brought to you via VS Code rather than Codex...
  
  - opened `codex-rs/app-server/tests/common/mcp_process.rs`
  - put the cursor on `McpServer`
  - hit `F2` and renamed the symbol to `TestAppServer`
  - went to the file tree
  - hit enter and renamed `mcp_process.rs` to `test_app_server.rs`
  - ran **Save All Files** from the Command Palette
  - ran `just fmt`
  
  The End
  
  (Admittedly, most of the local variables for `TestAppServer` are still
  named `mcp`, though.)

  Michael Bolin · 2026-06-01 21:49:38 +00:00
  6536841d89
• Add thread/settings/update app-server API (#23502)

  ## Why
  
  App-server clients need a way to update a thread's next-turn settings
  without starting a turn, adding transcript content, or waiting for turn
  lifecycle events. This gives settings UI a direct path for durable
  thread settings while clients observe the eventual effective state
  through a notification.
  
  This is a simplified rework of PR
  https://github.com/openai/codex/pull/22509. In particular, it changes
  the `thread/settings/update` api to return immediately rather than
  waiting and returning the effective (updated) thread settings. This
  makes the new api consistent with `turn/start` and greatly reduces the
  complexity of the implementation relative to the earlier attempt.
  
  ## What Changed
  
  - Adds experimental `thread/settings/update` with partial-update request
  fields and an empty acknowledgment response.
  - Adds experimental `thread/settings/updated`, carrying full effective
  `ThreadSettings` and scoped by `threadId` to subscribed clients for the
  affected thread.
  - Shares durable settings validation with `turn/start`, including
  `sandboxPolicy` plus `permissions` rejection and `serviceTier: null`
  clearing.
  - Emits the same settings notification when `turn/start` overrides
  change the stored effective thread settings.
  - Regenerates app-server protocol schema fixtures and updates
  `app-server/README.md`.

  Eric Traut · 2026-05-20 11:03:20 -07:00
  771a4e74ac
• [codex] request desktop attestation from app (#20619)

  ## Summary
  
  TL;DR: teaches `codex-rs` / app-server to request a desktop-provided
  attestation token and attach it as `x-oai-attestation` on the scoped
  ChatGPT Codex request paths.
  
  ![DeviceCheck attestation
  interface](https://raw.githubusercontent.com/openai/codex/dev/jm/devicecheck-diagram-assets/pr-assets/devicecheck-attestation-interface.png)
  
  ## Details
  
  This PR teaches the Codex app-server runtime how to request and attach
  an attestation token. It does not generate DeviceCheck tokens directly;
  instead, it relies on the connected desktop app to advertise that it can
  generate attestation and then asks that app for a fresh header value
  when needed.
  
  The flow is:
  
  1. The Codex desktop app connects to app-server.
  2. During `initialize`, the app can advertise that it supports
  `requestAttestation`.
  3. Before app-server calls selected ChatGPT Codex endpoints, it sends
  the internal server request `attestation/generate` to the app.
  4. app-server receives a pre-encoded header value back.
  5. app-server forwards that value as `x-oai-attestation` on the scoped
  outbound requests.
  
  The code in this repo is mostly protocol and runtime plumbing: it adds
  the app-server request/response shape, introduces an attestation
  provider in core, wires that provider into Responses / compaction /
  realtime setup paths, and covers the intended scoping with tests. The
  signed macOS DeviceCheck generation remains owned by the desktop app PR.
  
  ## Related PR
  
  - Codex desktop app implementation:
  https://github.com/openai/openai/pull/878649
  
  ## Validation
  
  <details>
  <summary>Tests run</summary>
  
  ```sh
  cargo test -p codex-app-server-protocol
  cargo test -p codex-core attestation --lib
  cargo test -p codex-app-server --lib attestation
  ```
  
  Also ran:
  
  ```sh
  just fix -p codex-core
  just fix -p codex-app-server
  just fix -p codex-app-server-protocol
  just fmt
  just write-app-server-schema
  ```
  
  </details>
  
  <details>
  <summary>E2E DeviceCheck validation</summary>
  
  First validated the signed desktop app boundary directly: launched a
  packaged signed `Codex.app`, sent `attestation/generate`, decoded the
  returned `v1.` attestation header, and validated the extracted
  DeviceCheck token with `personal/jm/verify_devicecheck_token.py` using
  bundle ID `com.openai.codex`. Apple returned `status_code: 200` and
  `is_ok: true`.
  
  Then ran the fuller app + app-server flow. The packaged `Codex.app`
  launched a current-branch app-server via `CODEX_CLI_PATH`, and a local
  MITM proxy intercepted outbound `chatgpt.com` traffic. The app-server
  requested `attestation/generate` from the real Electron app process, and
  the intercepted `/backend-api/codex/responses` traffic included
  `x-oai-attestation` on both routes:
  
  ```text
  GET  /backend-api/codex/responses  Upgrade: websocket  x-oai-attestation: present
  POST /backend-api/codex/responses  Upgrade: none       x-oai-attestation: present
  ```
  
  The captured header decoded to a DeviceCheck token that also validated
  with Apple for `com.openai.codex` (`status_code: 200`, `is_ok: true`,
  team `2DC432GLL2`).
  
  </details>
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Jiaming Zhang · 2026-05-08 12:36:02 -07:00
  5f4d0ec343
• realtime: rename provider session ids (#20361)

  ## Summary
  
  Codex is repurposing `session` to mean a thread group, so the realtime
  provider session id should no longer use `session_id` / `sessionId` in
  Codex-facing protocol payloads. This PR renames that provider-specific
  field to `realtime_session_id` / `realtimeSessionId` and intentionally
  breaks clients that still send the old field names.
  
  ## What Changed
  
  - Renamed realtime provider session fields in `ConversationStartParams`,
  `RealtimeConversationStartedEvent`, and `RealtimeEvent::SessionUpdated`.
  - Renamed app-server v2 realtime request and notification fields to
  `realtimeSessionId`.
  - Removed legacy serde aliases for `session_id` / `sessionId`; clients
  must send the new names.
  - Propagated the rename through core realtime startup, app-server
  adapters, codex-api websocket handling, and TUI realtime state.
  - Regenerated app-server protocol schema/TypeScript outputs and updated
  app-server README examples.
  - Kept upstream Realtime API concepts unchanged: provider `session.id`
  parsing and `x-session-id` headers still use the upstream wire names.
  
  ## Testing
  
  - CI is running on the latest pushed commit.
  - Earlier local verification on this PR:
    - `cargo test -p codex-protocol`
  - `CODEX_SKIP_VENDORED_BWRAP=1 cargo test -p codex-core
  realtime_conversation`
    - `cargo test -p codex-app-server-protocol`
  - `CODEX_SKIP_VENDORED_BWRAP=1 cargo test -p codex-app-server
  realtime_conversation`
  - attempted `CODEX_SKIP_VENDORED_BWRAP=1 cargo test -p codex-tui` (local
  linker bus error while linking the test binary)
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-30 13:39:48 +03:00
  8a97f3cf03
• Add realtime output modality and transcript events (#17701)

  - Add outputModality to thread/realtime/start and wire text/audio output
  selection through app-server, core, API, and TUI.\n- Rename the realtime
  transcript delta notification and add a separate transcript done
  notification that forwards final text from item done without correlating
  it with deltas.

  Ahmed Ibrahim · 2026-04-14 00:13:13 -07:00
  2f6fc7c137
• feat: disable memory endpoint (#17626)
  jif-oai · 2026-04-13 18:29:49 +01:00
  46a266cd6a
• Add realtime voice selection (#17176)

  - Add realtime voice selection for realtime/start.
  - Expose the supported v1/v2 voice lists and cover explicit, configured,
  default, and invalid voice paths.

  Ahmed Ibrahim · 2026-04-08 20:19:15 -07:00
  2f9090be62
• Move default realtime prompt into core (#17165)

  - Adds a core-owned realtime backend prompt template and preparation
  path.
  - Makes omitted realtime start prompts use the core default, while null
  or empty prompts intentionally send empty instructions.
  - Covers the core realtime path and app-server v2 path with integration
  coverage.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-08 19:34:40 -07:00
  4c2a1ae31b
• Add WebRTC transport to realtime start (#16960)

  Adds WebRTC startup to the experimental app-server
  `thread/realtime/start` method with an optional transport enum. The
  websocket path remains the default; WebRTC offers create the realtime
  session through the shared start flow and emit the answer SDP via
  `thread/realtime/sdp`.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-07 15:43:38 -07:00
  fb3dcfde1d
• Rename reject approval policy to granular (#14516)
  Jack Mousseau · 2026-03-12 16:38:04 -07:00
  b7dba72dbd
• chore: add a separate reject-policy flag for skill approvals (#14271)

  ## Summary
  - add `skill_approval` to `RejectConfig` and the app-server v2
  `AskForApproval::Reject` payload so skill-script prompts can be
  configured independently from sandbox and rule-based prompts
  - update Unix shell escalation to reject prompts based on the actual
  decision source, keeping prefix rules tied to `rules`, unmatched command
  fallbacks tied to `sandbox_approval`, and skill scripts tied to
  `skill_approval`
  - regenerate the affected protocol/config schemas and expand
  unit/integration coverage for the new flag and skill approval behavior

  Celia Chen · 2026-03-11 12:33:09 -07:00
  c1a424691f
• app-server: propagate nested experimental gating for AskForApproval::Reject (#14191)

  ## Summary
  This change makes `AskForApproval::Reject` gate correctly anywhere it
  appears inside otherwise-stable app-server protocol types.
  
  Previously, experimental gating for `approval_policy: Reject` was
  handled with request-specific logic in `ClientRequest` detection. That
  covered a few request params types, but it did not generalize to other
  nested uses such as `ProfileV2`, `Config`, `ConfigReadResponse`, or
  `ConfigRequirements`.
  
  This PR replaces that ad hoc handling with a generic nested experimental
  propagation mechanism.
  
  ## Testing
  
  seeing this when run app-server-test-client without experimental api
  enabled:
  ```
   initialize response: InitializeResponse { user_agent: "codex-toy-app-server/0.0.0 (Mac OS 26.3.1; arm64) vscode/2.4.36 (codex-toy-app-server; 0.0.0)" }
  > {
  >   "id": "50244f6a-270a-425d-ace0-e9e98205bde7",
  >   "method": "thread/start",
  >   "params": {
  >     "approvalPolicy": {
  >       "reject": {
  >         "mcp_elicitations": false,
  >         "request_permissions": true,
  >         "rules": false,
  >         "sandbox_approval": true
  >       }
  >     },
  >     "baseInstructions": null,
  >     "config": null,
  >     "cwd": null,
  >     "developerInstructions": null,
  >     "dynamicTools": null,
  >     "ephemeral": null,
  >     "experimentalRawEvents": false,
  >     "mockExperimentalField": null,
  >     "model": null,
  >     "modelProvider": null,
  >     "persistExtendedHistory": false,
  >     "personality": null,
  >     "sandbox": null,
  >     "serviceName": null
  >   }
  > }
  < {
  <   "error": {
  <     "code": -32600,
  <     "message": "askForApproval.reject requires experimentalApi capability"
  <   },
  <   "id": "50244f6a-270a-425d-ace0-e9e98205bde7"
  < }
  [verified] thread/start rejected approvalPolicy=Reject without experimentalApi
  ```
  
  ---------
  
  Co-authored-by: celia-oai <celia@openai.com>

  Dylan Hurd · 2026-03-11 12:33:08 -07:00
  d5694529ca
• Add app-server v2 thread realtime API (#12715)

  Add experimental `thread/realtime/*` v2 requests and notifications, then
  route app-server realtime events through that thread-scoped surface with
  integration coverage.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-02-25 09:59:10 -08:00
  947092283a
• feat: opt-out of events in the app-server (#11319)

  Add `optOutNotificationMethods` in the app-server to opt-out events
  based on exact method matching

  jif-oai · 2026-02-10 18:04:52 +00:00
  a364dd8b56
• feat: experimental flags (#10231)

  ## Problem being solved
  - We need a single, reliable way to mark app-server API surface as
  experimental so that:
    1. the runtime can reject experimental usage unless the client opts in
  2. generated TS/JSON schemas can exclude experimental methods/fields for
  stable clients.
  
  Right now that’s easy to drift or miss when done ad-hoc.
  
  ## How to declare experimental methods and fields
  - **Experimental method**: add `#[experimental("method/name")]` to the
  `ClientRequest` variant in `client_request_definitions!`.
  - **Experimental field**: on the params struct, derive `ExperimentalApi`
  and annotate the field with `#[experimental("method/name.field")]` + set
  `inspect_params: true` for the method variant so
  `ClientRequest::experimental_reason()` inspects params for experimental
  fields.
  
  ## How the macro solves it
  - The new derive macro lives in
  `codex-rs/codex-experimental-api-macros/src/lib.rs` and is used via
  `#[derive(ExperimentalApi)]` plus `#[experimental("reason")]`
  attributes.
  - **Structs**:
  - Generates `ExperimentalApi::experimental_reason(&self)` that checks
  only annotated fields.
    - The “presence” check is type-aware:
      - `Option<T>`: `is_some_and(...)` recursively checks inner.
      - `Vec`/`HashMap`/`BTreeMap`: must be non-empty.
      - `bool`: must be `true`.
      - Other types: considered present (returns `true`).
  - Registers each experimental field in an `inventory` with `(type_name,
  serialized field name, reason)` and exposes `EXPERIMENTAL_FIELDS` for
  that type. Field names are converted from `snake_case` to `camelCase`
  for schema/TS filtering.
  - **Enums**:
  - Generates an exhaustive `match` returning `Some(reason)` for annotated
  variants and `None` otherwise (no wildcard arm).
  - **Wiring**:
  - Runtime gating uses `ExperimentalApi::experimental_reason()` in
  `codex-rs/app-server/src/message_processor.rs` to reject requests unless
  `InitializeParams.capabilities.experimental_api == true`.
  - Schema/TS export filters use the inventory list and
  `EXPERIMENTAL_CLIENT_METHODS` from `client_request_definitions!` to
  strip experimental methods/fields when `experimental_api` is false.

  jif-oai · 2026-02-02 11:06:50 +00:00
  3cc9122ee2