21 Commits

• [codex] Add environment shell info (#26480)

  ## Why
  
  Shell detection needs to be available through the `Environment`
  abstraction so callers can ask the selected local or remote environment
  for shell metadata without adding a separate HTTP endpoint or parallel
  info-source path. This keeps shell metadata shaped like the existing
  environment-owned filesystem capability and lets remote environments
  answer through exec-server JSON-RPC.
  
  ## What changed
  
  - Added `environment/info` to the exec-server protocol/client/server and
  exposed `Environment::info()`.
  - Added local and remote environment info providers on `Environment`,
  following the existing capability-provider pattern used for filesystem
  access.
  - Moved the shared shell detection logic into `codex-shell-command` and
  kept core shell APIs as wrappers around that implementation.
  - Returned shell metadata as `EnvironmentInfo { shell: ShellInfo }`
  using the existing shell detection path.
  - Added a remote environment test that calls `Environment::info()`
  through an exec-server-backed environment.
  
  ## Validation
  
  - `git diff --check`
  - `just test -p codex-shell-command`
  - `just test -p codex-core -E 'test(/shell::tests::/)'`\n- `just test -p
  codex-exec-server environment`

  pakrym-oai · 2026-06-04 22:36:25 -07:00
  6a6a5f925e
• exec-server: canonicalize bound filesystem paths (#25149)

  ## Summary
  - add executor filesystem canonicalization as a bound-path operation
  - route remote canonicalization through the exec-server filesystem RPC
  surface
  - keep path normalization attached to the filesystem that owns the path
  
  ## Stack
  - 2/5 in the skills path authority stack extracted from
  https://github.com/openai/codex/pull/25098
  - follows merged https://github.com/openai/codex/pull/25121
  
  ## Validation
  - `cd
  /Users/starr/code/codex-worktrees/pr-25098-restack-review-pr1b/codex-rs
  && just fmt`
  - Not run: tests/checks (not requested)
  - GitHub CI pending on rewritten head

  starr-openai · 2026-06-01 11:53:31 -07:00
  53ac02356e
• Uprev Rust toolchain pins to 1.95.0 (#24684)

  ## Summary
  - Bump the workspace Rust toolchain from `1.93.0` to `1.95.0` across
  Cargo, Bazel, CI, release workflows, devcontainers, and the Codex
  environment config.
  - Refresh `MODULE.bazel.lock` so the Bazel Rust toolchain artifacts
  match the new version.
  - Leave purpose-specific toolchains unchanged, including the
  `argument-comment-lint` nightly and the upstream `rusty_v8` `1.91.0`
  build pin.
  - Includes fixes for new lints from `just fix` and a few codex-authored
  fixes for lints without a suggestion.

  Adam Perry @ OpenAI · 2026-05-26 20:59:47 -07:00
  cca1e0ba1d
• Reconnect disconnected exec-server websocket clients with fresh sessions (#23867)

  ## Summary
  - replace the one-shot lazy remote exec-server cache with a
  lock-protected current client
  - when the cached websocket client is already disconnected, create one
  fresh websocket client/session on the next `get()`
  - keep existing disconnect failure behavior for old process sessions and
  HTTP body streams; do not add session resume or request retry
  
  ## Why
  The prior PR direction was trying to grow into session restore: resume
  the old `session_id`, preserve existing process handles, and add
  reconnect retry policy. That is more machinery than we want for this
  slice.
  
  For now, the useful minimum is simpler: later fresh remote operations
  should not be stuck behind a dead cached websocket client, but anything
  already attached to the dead connection should fail loudly through the
  existing disconnect path. The server already has detached-session
  cleanup via its existing TTL, so this PR does not need to add
  client-side session preservation.
  
  ## What Changed
  - `LazyRemoteExecServerClient::get()` now keeps the current concrete
  client in a small mutex-protected cache plus one async connect lock.
  - If that cached client is still connected, `get()` returns it.
  - If that cached websocket client has observed the transport close,
  `get()` creates a brand-new websocket client with a brand-new
  exec-server session and replaces the cache.
  - If that cached client is stdio-backed, behavior stays one-shot: the
  dead client is returned and later work surfaces the existing disconnect
  error.
  - No `resume_session_id`, backoff, request replay, or existing
  `RemoteExecProcess` rebinding is added here.
  - Added focused websocket coverage that proves two concurrent `get()`
  calls after disconnect share one fresh replacement client/session.

  starr-openai · 2026-05-21 18:43:45 +02:00
  de80fa6e31
• Migrate exec-server remote registration to environments (#23633)

  ## Summary
  - migrate exec-server remote registration naming from executor to
  environment
  - align CLI, public Rust exports, registry error messages, and relay
  test fixtures with the environment registry contract
  - keep the live registration path and response model consistent with
  `/cloud/environment/{environment_id}/register`
  
  ## Verification
  - `cargo test -p codex-exec-server
  remote::tests::register_environment_posts_with_auth_provider_headers
  --manifest-path /Users/richardlee/code/codex/codex-rs/Cargo.toml`
  - `cargo test -p codex-exec-server --test relay
  multiplexed_remote_environment_routes_independent_virtual_streams
  --manifest-path /Users/richardlee/code/codex/codex-rs/Cargo.toml`
  - `cargo check -p codex-cli --manifest-path
  /Users/richardlee/code/codex/codex-rs/Cargo.toml` (still running when PR
  opened; will update after completion if needed)

  richardopenai · 2026-05-20 00:25:04 -07:00
  000bf5ce6d
• Increase exec-server environment transport timeouts (#21825)

  ## Why
  
  The environment-backed exec-server transport currently hardcodes 5
  second connect and initialize timeouts in `client_transport.rs`. That is
  short for SSH-backed stdio environments and remote websocket
  environments, and there is currently no way to raise those values from
  `CODEX_HOME/environments.toml`.
  
  This stacked follow-up raises the default environment transport timeouts
  and lets each configured environment override them in
  `environments.toml`.
  
  ## What Changed
  
  - raise the default environment transport connect and initialize
  timeouts from 5s to 10s
  - store concrete timeout values on `ExecServerTransportParams` instead
  of hardcoding them in `connect_for_transport(...)`
  - add `connect_timeout_sec` and `initialize_timeout_sec` to
  `[[environments]]` entries in `environments.toml`
  - apply parse-time defaults so runtime transport code receives fully
  resolved timeout values
  - reject `connect_timeout_sec` on stdio environments because it only
  applies to websocket transports
  - extend parser tests to cover the new fields and defaults
  
  ## Stack
  
  - base: https://github.com/openai/codex/pull/21794
  - this PR: configurable environment transport timeouts
  
  ## Validation
  
  - `cd
  /Users/starr/code/codex-worktrees/exec-env-timeouts-config-20260508/codex-rs
  && just fmt`
  - not run: tests
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-05-08 16:33:29 -07:00
  faa5d4a5e2
• Add stdio exec-server client transport (#20664)

  ## Why
  
  Configured environments need to connect to exec-server instances that
  are not necessarily already listening on a websocket URL. A
  command-backed stdio transport lets Codex start an exec-server process,
  speak JSON-RPC over its stdio streams, and clean up that child process
  with the client lifetime.
  
  **Stack position:** this is PR 2 of 5. It builds on the server-side
  stdio listener from PR 1 and provides the client transport used by later
  environment/config PRs.
  
  ## What Changed
  
  - Add `ExecServerTransport` variants for websocket URLs and stdio shell
  commands.
  - Add stdio command connection support for `ExecServerClient`.
  - Move websocket/stdio transport setup into `client_transport.rs` so
  `client.rs` stays focused on shared JSON-RPC client, session, HTTP, and
  notification behavior.
  - Tie stdio child process cleanup to the JSON-RPC connection lifetime
  with a RAII lifetime guard.
  - Keep existing websocket environment behavior by adapting URL-backed
  remotes to `ExecServerTransport::WebSocketUrl`.
  
  ## Stack
  
  - 1. https://github.com/openai/codex/pull/20663 - Add stdio exec-server
  listener
  - **2. This PR:** https://github.com/openai/codex/pull/20664 - Add stdio
  exec-server client transport
  - 3. https://github.com/openai/codex/pull/20665 - Make environment
  providers own default selection
  - 4. https://github.com/openai/codex/pull/20666 - Add CODEX_HOME
  environments TOML provider
  - 5. https://github.com/openai/codex/pull/20667 - Load configured
  environments from CODEX_HOME
  
  Split from original draft: https://github.com/openai/codex/pull/20508
  
  ## Validation
  
  Not run locally; this was split out of the original draft stack and then
  refactored to separate transport setup from the base client.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-05-07 23:48:50 +00:00
  a3de5bde6e
• Add cloud executor registration to exec-server (#19575)

  ## Summary
  This PR adds the first `codex-rs` milestone for remote-exec e2e: a local
  `codex exec-server` can now register itself with
  `codex-cloud-environments` and attach to the returned rendezvous
  websocket.
  
  At a high level, `codex exec-server --cloud ...` now:
  - loads ChatGPT auth from normal Codex config
  - registers an executor with `codex-cloud-environments`
  - receives a signed rendezvous websocket URL
  - serves the existing exec-server JSON-RPC protocol over that websocket
  
  ## What Changed
  - Added `--cloud`, `--cloud-base-url`, `--cloud-environment-id`, and
  `--cloud-name` to `codex exec-server`
  - Added a new `exec-server/src/cloud.rs` module that handles:
    - registration requests
    - auth/header setup
    - bounded auth retry on `401/403`
    - reconnect/backoff after websocket disconnects
  - Reused the existing `ConnectionProcessor` / `ExecServerHandler` path
  so cloud mode serves the same exec/filesystem RPC surface as local
  websocket mode
  - Added cloud-specific error variants and minimal docs for the new mode
  
  ## Testing
  Manual e2e test that fully goes through exec server flow with our codex
  cloud agent as orchestrator

  Michael Zeng · 2026-05-05 22:01:48 +00:00
  d0f9d5eba2
• [3/4] Add executor-backed RMCP HTTP client (#18583)

  ### Why
  The RMCP layer needs a Streamable HTTP client that can talk either
  directly over `reqwest` or through the executor HTTP runner without
  duplicating MCP session logic higher in the stack. This PR adds that
  client-side transport boundary so remote Streamable HTTP MCP can reuse
  the same RMCP flow as the local path.
  
  ### What
  - Add a shared `rmcp-client/src/streamable_http/` module with:
    - `transport_client.rs` for the local-or-remote transport enum
    - `local_client.rs` for the direct `reqwest` implementation
    - `remote_client.rs` for the executor-backed implementation
    - `common.rs` for the small shared Streamable HTTP helpers
  - Teach `RmcpClient` to build Streamable HTTP transports in either local
  or remote mode while keeping the existing OAuth ownership in RMCP.
  - Translate remote POST, GET, and DELETE session operations into
  executor `http/request` calls.
  - Preserve RMCP session expiry handling and reconnect behavior for the
  remote transport.
  - Add remote transport coverage in
  `rmcp-client/tests/streamable_http_remote.rs` and keep the shared test
  support in `rmcp-client/tests/streamable_http_test_support.rs`.
  
  ### Verification
  - `cargo check -p codex-rmcp-client`
  - online CI
  
  ### Stack
  1. #18581 protocol
  2. #18582 runner
  3. #18583 RMCP client
  4. #18584 manager wiring and local/remote coverage
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-22 17:38:04 -07:00
  0e78ce80ee
• Support multiple managed environments (#18401)

  ## Summary
  - refactor EnvironmentManager to own keyed environments with
  default/local lookup helpers
  - keep remote exec-server client creation lazy until exec/fs use
  - preserve disabled agent environment access separately from internal
  local environment access
  
  ## Validation
  - not run (per Codex worktree instruction to avoid tests/builds unless
  requested)
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-04-21 15:29:35 -07:00
  ddbe2536be
• [1/4] Add executor HTTP request protocol (#18581)

  ### Why
  Remote streamable HTTP MCP needs a transport-shaped executor primitive
  before the MCP client can move network I/O to the executor. This layer
  keeps the executor unaware of MCP and gives later PRs an ordered
  streaming surface for response bodies.
  
  ### What
  - Add typed `http/request` and `http/request/bodyDelta` protocol
  payloads.
  - Add executor client helpers for buffered and streamed HTTP responses.
  - Route body-delta notifications to request-scoped streams with sequence
  validation and cleanup when a stream finishes or is dropped.
  - Document the new protocol constants, transport structs, public client
  methods, body-stream lifecycle, and request-scoped routing helpers.
  - Add in-memory JSON-RPC client coverage for streamed HTTP response-body
  notifications, with comments spelling out what the test proves and each
  setup/exercise/assert phase.
  
  ### Stack
  1. #18581 protocol
  2. #18582 runner
  3. #18583 RMCP client
  4. #18584 manager wiring and local/remote coverage
  
  ### Verification
  - `just fmt`
  - `cargo check -p codex-exec-server -p codex-rmcp-client --tests`
  - `cargo check -p codex-core --test all` compile-only
  - `git diff --check`
  - Online full CI is running from the `full-ci` branch, including the
  remote Rust test job.
  
  Co-authored-by: Codex <noreply@openai.com>
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-21 02:21:08 +00:00
  d6af7a6c03
• [6/6] Fail exec client operations after disconnect (#18027)

  ## Summary
  - Reject new exec-server client operations once the transport has
  disconnected.
  - Convert pending RPC calls into closed errors instead of synthetic
  server errors.
  - Cover pending read and later write behavior after remote executor
  disconnect.
  
  ## Verification
  - `just fmt`
  - `cargo check -p codex-exec-server`
  
  ## Stack
  ```text
  @  #18027 [6/6] Fail exec client operations after disconnect
  │
  o  #18212 [5/6] Wire executor-backed MCP stdio
  │
  o  #18087 [4/6] Abstract MCP stdio server launching
  │
  o  #18020 [3/6] Add pushed exec process events
  │
  o  #18086 [2/6] Support piped stdin in exec process API
  │
  o  #18085 [1/6] Add MCP server environment config
  │
  o  main
  ```
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-20 23:24:06 +00:00
  9ef1cab6f7
• [3/6] Add pushed exec process events (#18020)

  ## Summary
  - Add a pushed `ExecProcessEvent` stream alongside retained
  `process/read` output.
  - Publish local and remote output, exit, close, and failure events.
  - Cover the event stream with shared local/remote exec process tests.
  
  ## Testing
  - `cargo check -p codex-exec-server`
  - `cargo check -p codex-rmcp-client`
  - Not run: `cargo test` per repo instruction; CI will cover.
  
  ## Stack
  ```text
  o  #18027 [6/6] Fail exec client operations after disconnect
  │
  o  #18212 [5/6] Wire executor-backed MCP stdio
  │
  o  #18087 [4/6] Abstract MCP stdio server launching
  │
  @  #18020 [3/6] Add pushed exec process events
  │
  o  #18086 [2/6] Support piped stdin in exec process API
  │
  o  #18085 [1/6] Add MCP server environment config
  │
  o  main
  ```
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-17 19:07:43 +00:00
  9d3a5cf05e
• feat: move exec-server ownership (#16344)

  This introduces session-scoped ownership for exec-server so ws
  disconnects no longer immediately kill running remote exec processes,
  and it prepares the protocol for reconnect-based resume.
  - add session_id / resume_session_id to the exec-server initialize
  handshake
    - move process ownership under a shared session registry
  - detach sessions on websocket disconnect and expire them after a TTL
  instead of killing processes immediately (we will resume based on this)
  - allow a new connection to resume an existing session and take over
  notifications/ownership
  - I use UUID to make them not predictable as we don't have auth for now
  - make detached-session expiry authoritative at resume time so teardown
  wins at the TTL boundary
  - reject long-poll process/read calls that get resumed out from under an
  older attachment
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-10 14:11:47 +01:00
  085ffb4456
• Add sandbox support to filesystem APIs (#16751)

  ## Summary
  - add optional `sandboxPolicy` support to the app-server filesystem
  request surface
  - thread sandbox-aware filesystem options through app-server and
  exec-server adapters
  - enforce sandboxed read/write access in the filesystem abstraction with
  focused local and remote coverage
  
  ## Validation
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-exec-server file_system`
  - `cargo test -p codex-app-server suite::v2::fs`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-04-08 12:10:48 -07:00
  f383cc980d
• feat: use ProcessId in exec-server (#15866)

  Use a full struct for the ProcessId to increase readability and make it
  easier in the future to make it evolve if needed

  jif-oai · 2026-03-26 16:45:36 +01:00
  6d2f4aaafc
• feat: exec-server prep for unified exec (#15691)

  This PR partially rebase `unified_exec` on the `exec-server` and adapt
  the `exec-server` accordingly.
  
  ## What changed in `exec-server`
  
  1. Replaced the old "broadcast-driven; process-global" event model with
  process-scoped session events. The goal is to be able to have dedicated
  handler for each process.
  2. Add to protocol contract to support explicit lifecycle status and
  stream ordering:
  - `WriteResponse` now returns `WriteStatus` (Accepted, UnknownProcess,
  StdinClosed, Starting) instead of a bool.
    - Added seq fields to output/exited notifications.
    - Added terminal process/closed notification.
  3. Demultiplexed remote notifications into per-process channels. Same as
  for the event sys
  4. Local and remote backends now both implement ExecBackend.
  5. Local backend wraps internal process ID/operations into per-process
  ExecProcess objects.
  6. Remote backend registers a session channel before launch and
  unregisters on failed launch.
  
  ## What changed in `unified_exec`
  
  1. Added unified process-state model and backend-neutral process
  wrapper. This will probably disappear in the future, but it makes it
  easier to keep the work flowing on both side.
  - `UnifiedExecProcess` now handles both local PTY sessions and remote
  exec-server processes through a shared `ProcessHandle`.
  - Added `ProcessState` to track has_exited, exit_code, and terminal
  failure message consistently across backends.
  2. Routed write and lifecycle handling through process-level methods.
  
  ## Some rationals
  
  1. The change centralizes execution transport in exec-server while
  preserving policy and orchestration ownership in core, avoiding
  duplicated launch approval logic. This comes from internal discussion.
  2. Session-scoped events remove coupling/cross-talk between processes
  and make stream ordering and terminal state explicit (seq, closed,
  failed).
  3. The failure-path surfacing (remote launch failures, write failures,
  transport disconnects) makes command tool output and cleanup behavior
  deterministic
  
  ## Follow-ups:
  * Unify the concept of thread ID behind an obfuscated struct
  * FD handling
  * Full zsh-fork compatibility
  * Full network sandboxing compatibility
  * Handle ws disconnection

  jif-oai · 2026-03-26 15:22:34 +01:00
  7dac332c93
• Split exec process into local and remote implementations (#15233)

  ## Summary
  - match the exec-process structure to filesystem PR #15232
  - expose `ExecProcess` on `Environment`
  - make `LocalProcess` the real implementation and `RemoteProcess` a thin
  network proxy over `ExecServerClient`
  - make `ProcessHandler` a thin RPC adapter delegating to `LocalProcess`
  - add a shared local/remote process test
  
  ## Validation
  - `just fmt`
  - `CARGO_TARGET_DIR=~/.cache/cargo-target/codex cargo test -p
  codex-exec-server`
  - `just fix -p codex-exec-server`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-03-20 03:13:08 +00:00
  96a86710c3
• Add exec-server exec RPC implementation (#15090)

  Stacked PR 2/3, based on the stub PR.
  
  Adds the exec RPC implementation and process/event flow in exec-server
  only.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-03-19 19:00:36 +00:00
  1d210f639e
• Remove stdio transport from exec server (#15119)

  Summary
  - delete the deprecated stdio transport plumbing from the exec server
  stack
  - add a basic `exec_server()` harness plus test utilities to start a
  server, send requests, and await events
  - refresh exec-server dependencies, configs, and documentation to
  reflect the new flow
  
  Testing
  - Not run (not requested)
  
  ---------
  
  Co-authored-by: starr-openai <starr@openai.com>
  Co-authored-by: Codex <noreply@openai.com>

  pakrym-oai · 2026-03-19 01:00:35 +00:00
  903660edba
• Add exec-server stub server and protocol docs (#15089)

  Stacked PR 1/3.
  
  This is the initialize-only exec-server stub slice: binary/client
  scaffolding and protocol docs, without exec/filesystem implementation.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-03-19 00:30:05 +00:00
  81996fcde6