22 Commits

• Support openai/form extended form elicitations (#27500)

  # Summary
  Allow App Server clients to opt into `openai/form` MCP elicitations.

  Gabriel Peal · 2026-06-18 11:54:49 -07:00
  21a599fa56
• Uprev Rust toolchain pins to 1.95.0 (#24684)

  ## Summary
  - Bump the workspace Rust toolchain from `1.93.0` to `1.95.0` across
  Cargo, Bazel, CI, release workflows, devcontainers, and the Codex
  environment config.
  - Refresh `MODULE.bazel.lock` so the Bazel Rust toolchain artifacts
  match the new version.
  - Leave purpose-specific toolchains unchanged, including the
  `argument-comment-lint` nightly and the upstream `rusty_v8` `1.91.0`
  build pin.
  - Includes fixes for new lints from `just fix` and a few codex-authored
  fixes for lints without a suggestion.

  Adam Perry @ OpenAI · 2026-05-26 20:59:47 -07:00
  cca1e0ba1d
• Restore app-server websocket listener with auth guard (#22404)

  ## Why
  PR #21843 removed the TCP websocket app-server listener, but that also
  removed functionality that still needs to exist. Restoring it as-is
  would reopen the old remote exposure problem, so this keeps the restored
  listener while making remote and non-loopback usage require explicit
  auth.
  
  ## What Changed
  - Mostly reverts #21843 and reapplies the small merge-conflict
  resolutions needed on top of current main.
  - Restores ws://IP:PORT parsing, the app-server TCP websocket acceptor,
  websocket auth CLI flags, and the associated tests.
  - The only intentional behavior change from the restored code is that
  non-loopback websocket listeners now fail startup unless --ws-auth
  capability-token or --ws-auth signed-bearer-token is configured.
  Loopback listeners remain available for local and SSH-forwarding
  workflows.
  
  ## Reviewer Focus
  Please focus review on the small auth-enforcement delta layered on top
  of the revert:
  
  - codex-rs/app-server-transport/src/transport/websocket.rs:
  start_websocket_acceptor now rejects unauthenticated non-loopback
  websocket binds before accepting connections.
  - codex-rs/app-server-transport/src/transport/auth.rs: helper logic
  classifies unauthenticated non-loopback listeners.
  - codex-rs/app-server/tests/suite/v2/connection_handling_websocket.rs:
  tests cover unauthenticated ws://0.0.0.0 startup rejection and
  authenticated non-loopback capability-token startup.
  
  Everything else is intended to be revert/merge-conflict restoration
  rather than new product behavior.
  
  ## Verification
  
  - Manually verified that TUI remoting is restored and that auth is
  enforced for non-localhost urls.

  Eric Traut · 2026-05-12 18:40:53 -07:00
  51bfb5f3b1
• app-server: remove TCP websocket listener (#21843)

  ## Why
  
  The app-server no longer needs to expose a TCP websocket listener.
  Keeping that transport also kept around a separate listener/auth surface
  that is unnecessary now that local clients can use stdio or the
  Unix-domain control socket, while remote connectivity is handled by
  `remote_control`.
  
  ## What Changed
  
  - Removed `ws://IP:PORT` parsing and the `AppServerTransport::WebSocket`
  startup path.
  - Deleted the app-server websocket listener auth module and removed
  related CLI flags/dependencies.
  - Kept websocket framing only where it is still needed: over the
  Unix-domain control socket and in the outbound `remote_control`
  connection.
  - Updated app-server CLI/help text and `app-server/README.md` to
  document only `stdio://`, `unix://`, `unix://PATH`, and `off` for local
  transports.
  - Converted affected app-server integration coverage from TCP websocket
  listeners to UDS-backed websocket connections, and added a parse test
  that rejects `ws://` listen URLs.
  - Removed the now-unused workspace `constant_time_eq` dependency and
  refreshed `Cargo.lock` after `cargo shear` caught the drift.
  - Moved test app-server UDS socket paths to short Unix temp paths so
  macOS Bazel test sandboxes do not exceed Unix socket path limits.
  
  ## Verification
  
  - Added/updated tests around UDS websocket transport behavior and
  `ws://` listen URL rejection.
  - `cargo shear`
  - `cargo metadata --no-deps --format-version 1`
  - `cargo test -p codex-app-server unix_socket_transport`
  - `cargo test -p codex-app-server unix_socket_disconnect`
  - `just fix -p codex-app-server`
  - `git diff --check`
  
  Local full Rust test execution was blocked before compilation by an
  external fetch failure for the pinned `nornagon/crossterm` git
  dependency. `just bazel-lock-update` and `just bazel-lock-check` were
  retried after the manifest cleanup but remain blocked by external
  BuildBuddy/V8 fetch timeouts.

  Ruslan Nigmatullin · 2026-05-11 10:17:26 -07:00
  a124ddb854
• test: harden app-server integration tests (#19683)

  ## Why
  
  Windows Bazel runs in the permissions stack exposed that app-server
  integration tests were launching normal plugin startup warmups in every
  subprocess. Those warmups can call
  `https://chatgpt.com/backend-api/plugins/featured` when a test is not
  specifically exercising plugin startup, which adds slow background work,
  noisy stderr, and dependence on external network state. The relevant
  startup/featured-plugin behavior was introduced across #15042 and
  #15264.
  
  A few app-server tests also had long optional waits or unbounded cleanup
  paths, making failures expensive to diagnose and contributing to slow
  Windows shards. One external-agent config test from #18246 used a
  GitHub-style marketplace source, which was enough to exercise the
  pending remote-import path but also meant the background completion task
  could attempt a real clone.
  
  ## What Changed
  
  - Adds explicit `AppServerRuntimeOptions` / `PluginStartupTasks`
  plumbing and a hidden debug-only
  `--disable-plugin-startup-tasks-for-tests` app-server flag, so
  integration tests can suppress startup plugin warmups without adding a
  production env-var gate.
  - Has the app-server test harness pass that hidden flag by default,
  while opting plugin-startup coverage back in for tests that
  intentionally exercise startup sync and featured-plugin warmup behavior.
  - Lowers normal app-server subprocess logging from `info`/`debug` to
  `warn` to avoid multi-megabyte stderr output in Bazel logs.
  - Prevents the external-agent config test from attempting a real
  marketplace clone by using an invalid non-local source while still
  exercising the pending-import completion path.
  - Bounds optional filesystem/realtime waits and fake WebSocket
  test-server shutdown so failures produce targeted timeouts instead of
  hanging a shard.
  - Fixes the Unix script-resolution test in `rmcp-client` to exercise
  PATH resolution directly and include the actual spawn error in failures.
  
  ## Verification
  
  - `cargo check -p codex-app-server`
  - `cargo clippy -p codex-app-server --tests -- -D warnings`
  - `cargo test -p codex-rmcp-client
  program_resolver::tests::test_unix_executes_script_without_extension`
  - `cargo test -p codex-app-server --test all
  external_agent_config_import_sends_completion_notification_after_pending_plugins_finish
  -- --nocapture`
  - `cargo test -p codex-app-server --test all
  plugin_list_uses_warmed_featured_plugin_ids_cache_on_first_request --
  --nocapture`
  - Windows Local Bazel passed with this test-hardening bundle before it
  was extracted from #19606.
  
  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/19683).
  * #19395
  * #19394
  * #19393
  * #19392
  * #19606
  * __->__ #19683

  Michael Bolin · 2026-04-26 12:43:16 -07:00
  ac2bffa443
• app-server: implement device key v2 methods (#18430)

  ## Why
  
  The device-key protocol needs an app-server implementation that keeps
  local key operations behind the same request-processing boundary as
  other v2 APIs.
  
  app-server owns request dispatch, transport policy, documentation, and
  JSON-RPC error shaping. `codex-device-key` owns key binding, validation,
  platform provider selection, and signing mechanics. Keeping the adapter
  thin makes the boundary easier to review and avoids moving local
  key-management details into thread orchestration code.
  
  ## What changed
  
  - Added `DeviceKeyApi` as the app-server adapter around
  `DeviceKeyStore`.
  - Converted protocol protection policies, payload variants, algorithms,
  and protection classes to and from the device-key crate types.
  - Encoded SPKI public keys and DER signatures as base64 protocol fields.
  - Routed `device/key/create`, `device/key/public`, and `device/key/sign`
  through `MessageProcessor`.
  - Rejected remote transports before provider access while allowing local
  `stdio` and in-process callers to reach the device-key API.
  - Added stdio, in-process, and websocket tests for device-key validation
  and transport policy.
  - Documented the device-key methods in the app-server v2 method list.
  
  ## Test coverage
  
  - `device_key_create_rejects_empty_account_user_id`
  - `in_process_allows_device_key_requests_to_reach_device_key_api`
  - `device_key_methods_are_rejected_over_websocket`
  
  ## Stack
  
  This is PR 3 of 4 in the device-key app-server stack. It is stacked on
  #18429.
  
  ## Validation
  
  - `cargo test -p codex-app-server device_key`
  - `just fix -p codex-app-server`

  Ruslan Nigmatullin · 2026-04-21 14:07:08 -07:00
  69c3d12274
• Stabilize Bazel tests (timeout tweaks and flake fixes) (#17791)
  David de Regt · 2026-04-16 07:57:51 -07:00
  6adba99f4d
• app-server: Only unload threads which were unused for some time (#17398)

  Currently app-server may unload actively running threads once the last
  connection disconnects, which is not expected.
  Instead track when was the last active turn & when there were any
  subscribers the last time, also add 30 minute idleness/no subscribers
  timer to reduce the churn.

  Ruslan Nigmatullin · 2026-04-13 12:25:26 -07:00
  a5507b59c4
• fix: MCP leaks in app-server (#17223)

  The disconnect path now reuses the same teardown flow as explicit
  unsubscribe, and the thread-state bookkeeping consistently reports only
  threads that lost their last subscriber
  
  https://github.com/openai/codex/issues/16895

  jif-oai · 2026-04-10 15:31:26 +01:00
  8d58899297
• chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)

  ## Why
  
  `argument-comment-lint` was green in CI even though the repo still had
  many uncommented literal arguments. The main gap was target coverage:
  the repo wrapper did not force Cargo to inspect test-only call sites, so
  examples like the `latest_session_lookup_params(true, ...)` tests in
  `codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.
  
  This change cleans up the existing backlog, makes the default repo lint
  path cover all Cargo targets, and starts rolling that stricter CI
  enforcement out on the platform where it is currently validated.
  
  ## What changed
  
  - mechanically fixed existing `argument-comment-lint` violations across
  the `codex-rs` workspace, including tests, examples, and benches
  - updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
  `tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
  `--all-targets` unless the caller explicitly narrows the target set
  - fixed both wrappers so forwarded cargo arguments after `--` are
  preserved with a single separator
  - documented the new default behavior in
  `tools/argument-comment-lint/README.md`
  - updated `rust-ci` so the macOS lint lane keeps the plain wrapper
  invocation and therefore enforces `--all-targets`, while Linux and
  Windows temporarily pass `-- --lib --bins`
  
  That temporary CI split keeps the stricter all-targets check where it is
  already cleaned up, while leaving room to finish the remaining Linux-
  and Windows-specific target-gated cleanup before enabling
  `--all-targets` on those runners. The Linux and Windows failures on the
  intermediate revision were caused by the wrapper forwarding bug, not by
  additional lint findings in those lanes.
  
  ## Validation
  
  - `bash -n tools/argument-comment-lint/run.sh`
  - `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
  - shell-level wrapper forwarding check for `-- --lib --bins`
  - shell-level wrapper forwarding check for `-- --tests`
  - `just argument-comment-lint`
  - `cargo test` in `tools/argument-comment-lint`
  - `cargo test -p codex-terminal-detection`
  
  ## Follow-up
  
  - Clean up remaining Linux-only target-gated callsites, then switch the
  Linux lint lane back to the plain wrapper invocation.
  - Clean up remaining Windows-only target-gated callsites, then switch
  the Windows lint lane back to the plain wrapper invocation.

  Michael Bolin · 2026-03-27 19:00:44 -07:00
  61dfe0b86c
• feat: add websocket auth for app-server (#14847)

  ## Summary
  This change adds websocket authentication at the app-server transport
  boundary and enforces it before JSON-RPC `initialize`, so authenticated
  deployments reject unauthenticated clients during the websocket
  handshake rather than after a connection has already been admitted.
  
  During rollout, websocket auth is opt-in for non-loopback listeners so
  we do not break existing remote clients. If `--ws-auth ...` is
  configured, the server enforces auth during websocket upgrade. If auth
  is not configured, non-loopback listeners still start, but app-server
  logs a warning and the startup banner calls out that auth should be
  configured before real remote use.
  
  The server supports two auth modes: a file-backed capability token, and
  a standard HMAC-signed JWT/JWS bearer token verified with the
  `jsonwebtoken` crate, with optional issuer, audience, and clock-skew
  validation. Capability tokens are normalized, hashed, and compared in
  constant time. Short shared secrets for signed bearer tokens are
  rejected at startup. Requests carrying an `Origin` header are rejected
  with `403` by transport middleware, and authenticated clients present
  credentials as `Authorization: Bearer <token>` during websocket upgrade.
  
  ## Validation
  - `cargo test -p codex-app-server transport::auth`
  - `cargo test -p codex-cli app_server_`
  - `cargo clippy -p codex-app-server --all-targets -- -D warnings`
  - `just bazel-lock-check`
  
  Note: in the broad `cargo test -p codex-app-server
  connection_handling_websocket` run, the touched websocket auth cases
  passed, but unrelated Unix shutdown tests failed with a timeout in this
  environment.
  
  ---------
  
  Co-authored-by: Eric Traut <etraut@openai.com>

  viyatb-oai · 2026-03-25 12:35:57 -07:00
  6124564297
• Stabilize macOS CI test timeouts (#15581)

  ## Summary
  - raise the shell snapshot apply_patch helper timeout to avoid macOS CI
  startup races
  - increase the shared MCP app-server test read timeout so slow
  initialize handshakes do not fail command_exec tests spuriously
  
  ## Testing
  - cargo test -p codex-core
  shell_command_snapshot_still_intercepts_apply_patch
  - cargo test -p codex-app-server
  command_exec_tty_implies_streaming_and_reports_pty_output
  
  Co-authored-by: Codex <noreply@openai.com>

  Dylan Hurd · 2026-03-24 09:33:20 +00:00
  79577355c1
• app-server: reject websocket requests with Origin headers (#14995)

  Reject websocket requests that carry an `Origin` header

  Max Johnson · 2026-03-18 00:24:53 +00:00
  19b887128e
• Stabilize websocket test server binding (#14002)

  ## Summary
  - stop reserving a localhost port in the websocket tests before spawning
  the server
  - let the app-server bind `127.0.0.1:0` itself and read back the actual
  bound websocket address from stderr
  - update the websocket test helpers and callers to use the discovered
  address
  
  ## Why this fixes the flake
  The previous harness reserved a port in the test process, dropped it,
  and then asked the server process to bind that same address. On busy
  runners there is a race between releasing the reservation and the child
  process rebinding it, which can produce sporadic startup failures.
  Binding to port `0` inside the server removes that race entirely, and
  waiting for the server to report the real bound address makes the tests
  connect only after the listener is actually ready.

  Ahmed Ibrahim · 2026-03-09 23:39:56 -07:00
  b39ae9501f
• codex-rs/app-server: add health endpoints for --listen websocket server (#13782)

  Healthcheck endpoints for the websocket server
  
  - serve `GET /readyz` and `GET /healthz` from the same listener used for
  `--listen ws://...`
  - switch the websocket listener over to `axum` upgrade handling instead
  of manual socket parsing
  - add websocket transport coverage for the health endpoints and document
  the new behavior
  
  Testing
  - integration tests
  - built and tested e2e
  
  ```
  > curl -i http://127.0.0.1:9234/readyz
  HTTP/1.1 200 OK
  content-length: 0
  date: Fri, 06 Mar 2026 19:20:23 GMT
  
  >  curl -i http://127.0.0.1:9234/healthz
  HTTP/1.1 200 OK
  content-length: 0
  date: Fri, 06 Mar 2026 19:20:24 GMT
  ```

  Max Johnson · 2026-03-09 22:11:30 +00:00
  66e71cce11
• app-server: Add streaming and tty/pty capabilities to command/exec (#13640)

  * Add an ability to stream stdin, stdout, and stderr
  * Streaming of stdout and stderr has a configurable cap for total amount
  of transmitted bytes (with an ability to disable it)
  * Add support for overriding environment variables
  * Add an ability to terminate running applications (using
  `command/exec/terminate`)
  * Add TTY/PTY support, with an ability to resize the terminal (using
  `command/exec/resize`)

  Ruslan Nigmatullin · 2026-03-06 17:30:17 -08:00
  e9bd8b20a1
• app-server: Emit thread/name/updated event globally (#13674)
  Ruslan Nigmatullin · 2026-03-06 10:25:18 -08:00
  51fcdc760d
• feat(app-server): add tracing to all app-server APIs (#13285)

  ### Overview
  This PR adds the first piece of tracing for app-server JSON-RPC
  requests.
  
  There are two main changes:
  - JSON-RPC requests can now take an optional W3C trace context at the
  top level via a `trace` field (`traceparent` / `tracestate`).
  - app-server now creates a dedicated request span for every inbound
  JSON-RPC request in `MessageProcessor`, and uses the request-level trace
  context as the parent when present.
  
  For compatibility with existing flows, app-server still falls back to
  the TRACEPARENT env var when there is no request-level traceparent.
  
  This PR is intentionally scoped to the app-server boundary. In a
  followup, we'll actually propagate trace context through the async
  handoff into core execution spans like run_turn, which will make
  app-server traces much more useful.
  
  ### Spans
  A few details on the app-server span shape:
  - each inbound request gets its own server span
  - span/resource names are based on the JSON-RPC method (`initialize`,
  `thread/start`, `turn/start`, etc.)
  - spans record transport (stdio vs websocket), request id, connection
  id, and client name/version when available
  - `initialize` stores client metadata in session state so later requests
  on the same connection can reuse it

  Owen Lin · 2026-03-02 16:01:41 -08:00
  d473e8d56d
• codex-rs/app-server: graceful websocket restart on Ctrl-C (#12517)

  ## Summary
  - add graceful websocket app-server restart on Ctrl-C by draining until
  no assistant turns are running
  - stop the websocket acceptor and disconnect existing connections once
  the drain condition is met
  - add a websocket integration test that verifies Ctrl-C waits for an
  in-flight turn before exit
  
  ## Verification
  - `cargo check -p codex-app-server --quiet`
  - `cargo test -p codex-app-server --test all
  suite::v2::connection_handling_websocket`
  - I (maxj) tested remote and local Codex.app
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Max Johnson · 2026-02-24 16:27:59 -08:00
  5163850025
• Reapply "Add app-server transport layer with websocket support" (#11370)

  Reapply "Add app-server transport layer with websocket support" with
  additional fixes from https://github.com/openai/codex/pull/11313/changes
  to avoid deadlocking.
  
  This reverts commit 47356ff83c.
  
  ## Summary
  
  To avoid deadlocking when queues are full, we maintain separate tokio
  tasks dedicated to incoming vs outgoing event handling
  - split the app-server main loop into two tasks in
  `run_main_with_transport`
     - inbound handling (`transport_event_rx`)
     - outbound handling (`outgoing_rx` + `thread_created_rx`)
  - separate incoming and outgoing websocket tasks
  
  ## Validation
  
  Integration tests, testing thoroughly e2e in codex app w/ >10 concurrent
  requests
  
  <img width="1365" height="979" alt="Screenshot 2026-02-10 at 2 54 22 PM"
  src="https://github.com/user-attachments/assets/47ca2c13-f322-4e5c-bedd-25859cbdc45f"
  />
  
  ---------
  
  Co-authored-by: jif-oai <jif@openai.com>

  Max Johnson · 2026-02-11 18:13:39 +00:00
  7053aa5457
• Revert "Add app-server transport layer with websocket support (#10693)" (#11323)

  Suspected cause of deadlocking bug

  Max Johnson · 2026-02-10 17:37:49 +00:00
  47356ff83c
• Add app-server transport layer with websocket support (#10693)

  - Adds --listen <URL> to codex app-server with two listen modes:
        - stdio:// (default, existing behavior)
        - ws://IP:PORT (new websocket transport)
    - Refactors message routing to be connection-aware:
  - Tracks per-connection session state (initialize/experimental
  capability)
        - Routes responses/errors to the originating connection
  - Broadcasts server notifications/requests to initialized connections
  - Updates initialization semantics to be per connection (not
  process-global), and updates app-server docs accordingly.
  - Adds websocket accept/read/write handling (JSON-RPC per text frame,
  ping/pong handling, connection lifecycle events).
  
  Testing
  
  - Unit tests for transport URL parsing and targeted response/error
  routing.
    - New websocket integration test validating:
        - per-connection initialization requirements
        - no cross-connection response leakage
        - same request IDs on different connections route independently.

  Max Johnson · 2026-02-05 20:56:34 +00:00
  8473096efb