24 Commits

• Recover exec process stdin writes (#28895)

  ## Summary
  
  Remote stdio MCP servers send tool calls by writing JSON-RPC bytes
  through `process/write`.
  
  When the exec-server websocket drops at the wrong time, the remote
  process can survive session recovery, but the stdin write can still fail
  back to RMCP as a transport send error. RMCP then closes the stdio MCP
  transport, so tools like `node_repl` are lost even though the
  process/session recovery path is working.
  
  This changes `process/write` to be safe to retry across exec-server
  recovery:
  
  - adds a required `writeId` to `process/write`
  - retries remote `Session::write` with the same `writeId` after
  reconnect
  - remembers accepted write ids per process so duplicate retries return
  `Accepted` without writing the same bytes to child stdin again
  - covers both the client retry path and server-side write id dedupe with
  tests
  
  In simple terms:
  
  ```text
  before:
  write to MCP stdin -> websocket closes -> write errors -> RMCP closes node_repl
  
  after:
  write to MCP stdin -> websocket closes -> reconnect -> retry same writeId
  server either writes once or recognizes it already did
  ```

  jif · 2026-06-18 19:04:26 +02:00
  83e6a786a2
• [codex] exec-server: stream files in chunks (#28354)

  ## Why
  
  `fs/readFile` buffers the entire file in one response, which makes large
  remote reads expensive and prevents callers from applying backpressure.
  We need an opt-in streaming path with bounded block sizes while
  preserving the existing single-call API for small and sandboxed reads.
  
  ## What changed
  
  - Add `ExecServerClient::stream`, returning a named `FileReadStream`
  that implements `futures::Stream` and yields immutable 1 MiB byte
  blocks.
  - Add internal `fs/open`, `fs/readBlock`, and `fs/close` RPCs.
  `fs/readBlock` accepts an explicit offset and length.
  - Keep unsandboxed files open between block reads, cap open handles per
  connection, and clean them up on EOF, error, stream drop, explicit
  close, or connection shutdown.
  - Reject platform-sandboxed streaming opens instead of turning the
  one-shot sandbox helper into a persistent server. Existing `fs/readFile`
  behavior is unchanged.
  
  ## Testing
  
  - `just test -p codex-exec-server`
  - Integration coverage for 1 MiB chunking, exact block-boundary EOF,
  sandbox rejection, and continued reads from the opened file after path
  replacement.
  - Handle-manager coverage for non-sequential offsets, variable block
  lengths, the 128-handle limit, and capacity release after close.

  pakrym-oai · 2026-06-16 09:50:55 -07:00
  a4711b88dd
• Use PathUri in filesystem permission paths for exec-server (#28165)

  ## Why
  
  Progress towards letting app-server and exec-server run on different
  platforms, specifically for sandbox configuration.
  
  ## What
  
  - Make the filesystem path containment hierarchy generic, defaulting to
  `AbsolutePathBuf` for now.
  - Have clients specify `AbsolutePathBuf` or `PathUri` directly where
  needed.
  - Use `PathUri` throughout exec-server filesystem protocol and trait
  boundaries.
  - Implement `From` for conversion to path URIs and `TryFrom` for
  fallible conversion to absolute paths through the generic type
  hierarchy.

  Adam Perry @ OpenAI · 2026-06-15 23:55:23 +00:00
  46f17930b6
• [codex] Carry exec-server cwd as PathUri (#28032)

  ## Why
  
  This is the second-to-last place in the exec-server protocol that needs
  to migrate to URIs to support cross-OS operation.
  
  ## What
  
  - Change `ExecParams.cwd` to `PathUri`.
  - Keep the cwd URI-shaped through core and rmcp producers, converting it
  to `AbsolutePathBuf` only in `LocalProcess::start_process`.
  - Reject non-native cwd URIs before launch and update the affected
  protocol documentation and call sites.

  Adam Perry @ OpenAI · 2026-06-13 20:56:42 +00:00
  0fed4497f5
• [codex] Add size to internal filesystem metadata (#27927)

  ## Why
  
  `ExecutorFileSystem::get_metadata` reports file kind and timestamps but
  not size. Internal callers that need to enforce a size limit therefore
  have to read the complete file first, which is especially wasteful for
  remote filesystems.
  
  This adds the missing internal metadata so consumers can reject
  oversized files before transferring or buffering them. The field is
  named `size`, matching VS Code's `FileStat.size` filesystem convention.
  
  ## What changed
  
  - add `size: u64` to internal `FileMetadata`
  - populate it from the underlying filesystem metadata
  - carry it through sandbox-helper and remote exec-server responses
  - cover files, directories, symlink targets, and sandboxed reads across
  local and remote filesystem implementations
  
  The new field is intentionally not exposed through the app-server API.
  
  ## Testing
  
  - `just test -p codex-exec-server get_metadata`
  - `just test -p codex-exec-server
  file_system_sandboxed_metadata_and_read_allow_readable_root`
  - `just test -p codex-core-plugins`
  - `just test -p codex-skills-extension`

  pakrym-oai · 2026-06-12 12:12:08 -07:00
  76d8f20241
• sandboxing: migrate cwd inputs to PathUri (#27816)

  ## Why
  
  Sandbox cwd values can cross app-server and exec-server host boundaries.
  They should retain URI semantics until the receiving host validates them
  instead of being interpreted early as native paths.
  
  ## What
  
  - Carry `PathUri` through filesystem sandbox contexts, sandbox commands,
  and transform inputs.
  - Convert command and policy cwd once in `SandboxManager::transform`,
  then keep launch requests native.
  - Preserve sandbox cwd over remote filesystem transport and reject
  non-native URIs without fallback.
  - Cache paired native/URI turn-environment cwd values during migration,
  with immutable access to keep them synchronized.
  - Extend existing protocol, forwarding, transform, and core runtime
  tests.

  Adam Perry @ OpenAI · 2026-06-12 11:38:01 -07:00
  52a50aec70
• Remove fs/join and fs/parent from exec-server protocol (#27700)

  ## Summary
  
  Path composition is already handled by `PathUri`, leaving `fs/join` and
  `fs/parent` as redundant exec-server protocol surface. Because
  app-server and exec-server are deployed atomically, these obsolete
  methods can be removed without a compatibility shim.
  
  This removes the protocol constants and payloads, public client APIs,
  server registrations and handlers, and endpoint-only tests. Existing
  in-process `PathUri` join/parent coverage remains.
  
  ## Validation
  
  - `just test -p codex-exec-server` (215 passed, 2 skipped)

  Adam Perry @ OpenAI · 2026-06-11 15:48:53 -07:00
  e069153f2a
• [codex] migrate exec-server filesystem protocol to PathUri (#27653)

  Exec-server filesystem calls should preserve cross-platform `file:` URIs
  across the remote boundary instead of converting them through paths
  native to the client host.
  
  This changes the exec-server filesystem protocol DTOs to use `PathUri`,
  carries those values directly through remote and sandbox-helper
  transports, and keeps legacy native absolute-path request strings
  readable for compatibility. It also updates protocol documentation and
  coverage for URI serialization and non-native URI forwarding.

  Adam Perry @ OpenAI · 2026-06-11 15:09:12 -07:00
  d23bb22f25
• [codex] Handle Ctrl-C for non-TTY unified exec (#26734)

  ## Why
  
  A long-running unified exec process started with `tty: false` could not
  be interrupted via `write_stdin`: ordinary non-TTY stdin writes are
  rejected once stdin is closed, but an exact U+0003 payload should still
  map to a process interrupt. The interrupt should flow through the same
  process lifecycle path as a real signal so Codex preserves
  process-reported output and exit metadata instead of fabricating a
  Ctrl-C exit code or tearing down the session early.
  
  ## What Changed
  
  - Add `process/signal` to exec-server with `ProcessSignal::Interrupt`
  and an empty response.
  - Add a non-consuming `ProcessHandle::signal` path for spawned
  processes; on Unix it sends SIGINT to the process group and leaves
  terminate/hard-kill unchanged.
  - Route non-TTY U+0003 `write_stdin` through `process.signal(...)`
  instead of `terminate`, then let the normal post-write collection path
  drain output and observe exit.
  - Add exec-server coverage where a shell `trap INT` handler prints the
  signal and exits with its own code.
  - Add unified exec coverage where a `tty: false` process traps SIGINT,
  emits output, and exits with its own code.
  
  ## Validation
  
  - `just test -p codex-exec-server
  exec_process_signal_interrupts_process`
  - `just test -p codex-exec-server`
  - `just test -p codex-core
  write_stdin_ctrl_c_interrupts_non_tty_session`

  pakrym-oai · 2026-06-09 15:10:17 -07:00
  f2969f36e8
• [codex] Add environment shell info (#26480)

  ## Why
  
  Shell detection needs to be available through the `Environment`
  abstraction so callers can ask the selected local or remote environment
  for shell metadata without adding a separate HTTP endpoint or parallel
  info-source path. This keeps shell metadata shaped like the existing
  environment-owned filesystem capability and lets remote environments
  answer through exec-server JSON-RPC.
  
  ## What changed
  
  - Added `environment/info` to the exec-server protocol/client/server and
  exposed `Environment::info()`.
  - Added local and remote environment info providers on `Environment`,
  following the existing capability-provider pattern used for filesystem
  access.
  - Moved the shared shell detection logic into `codex-shell-command` and
  kept core shell APIs as wrappers around that implementation.
  - Returned shell metadata as `EnvironmentInfo { shell: ShellInfo }`
  using the existing shell detection path.
  - Added a remote environment test that calls `Environment::info()`
  through an exec-server-backed environment.
  
  ## Validation
  
  - `git diff --check`
  - `just test -p codex-shell-command`
  - `just test -p codex-core -E 'test(/shell::tests::/)'`\n- `just test -p
  codex-exec-server environment`

  pakrym-oai · 2026-06-04 22:36:25 -07:00
  6a6a5f925e
• exec-server: canonicalize bound filesystem paths (#25149)

  ## Summary
  - add executor filesystem canonicalization as a bound-path operation
  - route remote canonicalization through the exec-server filesystem RPC
  surface
  - keep path normalization attached to the filesystem that owns the path
  
  ## Stack
  - 2/5 in the skills path authority stack extracted from
  https://github.com/openai/codex/pull/25098
  - follows merged https://github.com/openai/codex/pull/25121
  
  ## Validation
  - `cd
  /Users/starr/code/codex-worktrees/pr-25098-restack-review-pr1b/codex-rs
  && just fmt`
  - Not run: tests/checks (not requested)
  - GitHub CI pending on rewritten head

  starr-openai · 2026-06-01 11:53:31 -07:00
  53ac02356e
• [codex] Move config loading into codex-config (#19487)

  ## Why
  
  Config loading had become split across crates: `codex-config` owned the
  config types and merge logic, while `codex-core` still owned the loader
  that assembled the layer stack. This change consolidates that
  responsibility in `codex-config`, so the crate that defines config
  behavior also owns how configs are discovered and loaded.
  
  To make that move possible without reintroducing the old dependency
  cycle, the shell-environment policy types and helpers that
  `codex-exec-server` needs now live in `codex-protocol` instead of
  flowing through `codex-config`.
  
  This also makes the migrated loader tests more deterministic on machines
  that already have managed or system Codex config installed by letting
  tests override the system config and requirements paths instead of
  reading the host's `/etc/codex`.
  
  ## What Changed
  
  - moved the config loader implementation from `codex-core` into
  `codex-config::loader` and deleted the old `core::config_loader` module
  instead of leaving a compatibility shim
  - moved shell-environment policy types and helpers into
  `codex-protocol`, then updated `codex-exec-server` and other downstream
  crates to import them from their new home
  - updated downstream callers to use loader/config APIs from
  `codex-config`
  - added test-only loader overrides for system config and requirements
  paths so loader-focused tests do not depend on host-managed config state
  - cleaned up now-unused dependency entries and platform-specific cfgs
  that were surfaced by post-push CI
  
  ## Testing
  
  - `cargo test -p codex-config`
  - `cargo test -p codex-core config_loader_tests::`
  - `cargo test -p codex-protocol -p codex-exec-server -p
  codex-cloud-requirements -p codex-rmcp-client --lib`
  - `cargo test --lib -p codex-app-server-client -p codex-exec`
  - `cargo test --no-run --lib -p codex-app-server`
  - `cargo test -p codex-linux-sandbox --lib`
  - `cargo shear`
  - `just bazel-lock-check`
  
  ## Notes
  
  - I did not chase unrelated full-suite failures outside the migrated
  loader surface.
  - `cargo test -p codex-core --lib` still hits unrelated proxy-sensitive
  failures on this machine, and Windows CI still shows unrelated
  long-running/timeouting test noise outside the loader migration itself.

  pakrym-oai · 2026-04-26 15:10:53 -07:00
  9c3abcd46c
• [2/4] Implement executor HTTP request runner (#18582)

  ### Why
  Remote streamable HTTP MCP needs the executor to perform ordinary HTTP
  requests on the executor side. This keeps network placement aligned with
  `experimental_environment = "remote"` without adding MCP-specific
  executor APIs.
  
  ### What
  - Add an executor-side `http/request` runner backed by `reqwest`.
  - Validate request method and URL scheme, preserving the transport
  boundary at plain HTTP.
  - Return buffered responses for ordinary calls and emit ordered
  `http/request/bodyDelta` notifications for streaming responses.
  - Register the request handler in the exec-server router.
  - Document the runner entrypoint, conversion helpers, body-stream
  bridge, notification sender, timeout behavior, and new integration-test
  helpers.
  - Add exec-server integration tests with the existing websocket harness
  and a local TCP HTTP peer for buffered and streamed responses, with
  comments spelling out what each test proves and its
  setup/exercise/assert phases.
  
  ### Stack
  1. #18581 protocol
  2. #18582 runner
  3. #18583 RMCP client
  4. #18584 manager wiring and local/remote coverage
  
  ### Verification
  - `just fmt`
  - `cargo check -p codex-exec-server -p codex-rmcp-client --tests`
  - `cargo check -p codex-core --test all` compile-only
  - `git diff --check`
  - Online full CI is running from the `full-ci` branch, including the
  remote Rust test job.
  
  Co-authored-by: Codex <noreply@openai.com>
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-22 20:36:34 +00:00
  9360f267f3
• [1/4] Add executor HTTP request protocol (#18581)

  ### Why
  Remote streamable HTTP MCP needs a transport-shaped executor primitive
  before the MCP client can move network I/O to the executor. This layer
  keeps the executor unaware of MCP and gives later PRs an ordered
  streaming surface for response bodies.
  
  ### What
  - Add typed `http/request` and `http/request/bodyDelta` protocol
  payloads.
  - Add executor client helpers for buffered and streamed HTTP responses.
  - Route body-delta notifications to request-scoped streams with sequence
  validation and cleanup when a stream finishes or is dropped.
  - Document the new protocol constants, transport structs, public client
  methods, body-stream lifecycle, and request-scoped routing helpers.
  - Add in-memory JSON-RPC client coverage for streamed HTTP response-body
  notifications, with comments spelling out what the test proves and each
  setup/exercise/assert phase.
  
  ### Stack
  1. #18581 protocol
  2. #18582 runner
  3. #18583 RMCP client
  4. #18584 manager wiring and local/remote coverage
  
  ### Verification
  - `just fmt`
  - `cargo check -p codex-exec-server -p codex-rmcp-client --tests`
  - `cargo check -p codex-core --test all` compile-only
  - `git diff --check`
  - Online full CI is running from the `full-ci` branch, including the
  remote Rust test job.
  
  Co-authored-by: Codex <noreply@openai.com>
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-21 02:21:08 +00:00
  d6af7a6c03
• [2/8] Support piped stdin in exec process API (#18086)

  ## Summary
  - Add an explicit stdin mode to process/start.
  - Keep normal non-interactive exec stdin closed while allowing
  pipe-backed processes.
  
  ## Stack
  ```text
  o  #18027 [8/8] Fail exec client operations after disconnect
  │
  o  #18025 [7/8] Cover MCP stdio tests with executor placement
  │
  o  #18089 [6/8] Wire remote MCP stdio through executor
  │
  o  #18088 [5/8] Add executor process transport for MCP stdio
  │
  o  #18087 [4/8] Abstract MCP stdio server launching
  │
  o  #18020 [3/8] Add pushed exec process events
  │
  @  #18086 [2/8] Support piped stdin in exec process API
  │
  o  #18085 [1/8] Add MCP server environment config
  │
  o  main
  ```
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-16 10:30:10 -07:00
  2ca270d08d
• [codex] Add symlink flag to fs metadata (#17719)

  Add `is_symlink` to FsMetadata struct.

  pakrym-oai · 2026-04-13 17:46:56 -07:00
  f3cbe3d385
• Build remote exec env from exec-server policy (#17216)

  ## Summary
  - add an exec-server `envPolicy` field; when present, the server starts
  from its own process env and applies the shell environment policy there
  - keep `env` as the exact environment for local/embedded starts, but
  make it an overlay for remote unified-exec starts
  - move the shell-environment-policy builder into `codex-config` so Core
  and exec-server share the inherit/filter/set/include behavior
  - overlay only runtime/sandbox/network deltas from Core onto the
  exec-server-derived env
  
  ## Why
  Remote unified exec was materializing the shell env inside Core and
  forwarding the whole map to exec-server, so remote processes could
  inherit the orchestrator machine's `HOME`, `PATH`, etc. This keeps the
  base env on the executor while preserving Core-owned runtime additions
  like `CODEX_THREAD_ID`, unified-exec defaults, network proxy env, and
  sandbox marker env.
  
  ## Validation
  - `just fmt`
  - `git diff --check`
  - `cargo test -p codex-exec-server --lib`
  - `cargo test -p codex-core --lib unified_exec::process_manager::tests`
  - `cargo test -p codex-core --lib exec_env::tests`
  - `cargo test -p codex-core --lib exec_env_tests` (compile-only; filter
  matched 0 tests)
  - `cargo test -p codex-config --lib shell_environment` (compile-only;
  filter matched 0 tests)
  - `just bazel-lock-update`
  
  ## Known local validation issue
  - `just bazel-lock-check` is not runnable in this checkout: it invokes
  `./scripts/check-module-bazel-lock.sh`, which is missing.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>
  Co-authored-by: pakrym-oai <pakrym@openai.com>

  jif-oai · 2026-04-13 09:59:08 +01:00
  bacb92b1d7
• Run exec-server fs operations through sandbox helper (#17294)

  ## Summary
  - run exec-server filesystem RPCs requiring sandboxing through a
  `codex-fs` arg0 helper over stdin/stdout
  - keep direct local filesystem execution for `DangerFullAccess` and
  external sandbox policies
  - remove the standalone exec-server binary path in favor of top-level
  arg0 dispatch/runtime paths
  - add sandbox escape regression coverage for local and remote filesystem
  paths
  
  ## Validation
  - `just fmt`
  - `git diff --check`
  - remote devbox: `cd codex-rs && bazel test --bes_backend=
  --bes_results_url= //codex-rs/exec-server:all` (6/6 passed)
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-04-12 18:36:03 -07:00
  d626dc3895
• feat: move exec-server ownership (#16344)

  This introduces session-scoped ownership for exec-server so ws
  disconnects no longer immediately kill running remote exec processes,
  and it prepares the protocol for reconnect-based resume.
  - add session_id / resume_session_id to the exec-server initialize
  handshake
    - move process ownership under a shared session registry
  - detach sessions on websocket disconnect and expire them after a TTL
  instead of killing processes immediately (we will resume based on this)
  - allow a new connection to resume an existing session and take over
  notifications/ownership
  - I use UUID to make them not predictable as we don't have auth for now
  - make detached-session expiry authoritative at resume time so teardown
  wins at the TTL boundary
  - reject long-poll process/read calls that get resumed out from under an
  older attachment
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-10 14:11:47 +01:00
  085ffb4456
• Add sandbox support to filesystem APIs (#16751)

  ## Summary
  - add optional `sandboxPolicy` support to the app-server filesystem
  request surface
  - thread sandbox-aware filesystem options through app-server and
  exec-server adapters
  - enforce sandboxed read/write access in the filesystem abstraction with
  focused local and remote coverage
  
  ## Validation
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-exec-server file_system`
  - `cargo test -p codex-app-server suite::v2::fs`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-04-08 12:10:48 -07:00
  f383cc980d
• feat: use ProcessId in exec-server (#15866)

  Use a full struct for the ProcessId to increase readability and make it
  easier in the future to make it evolve if needed

  jif-oai · 2026-03-26 16:45:36 +01:00
  6d2f4aaafc
• feat: exec-server prep for unified exec (#15691)

  This PR partially rebase `unified_exec` on the `exec-server` and adapt
  the `exec-server` accordingly.
  
  ## What changed in `exec-server`
  
  1. Replaced the old "broadcast-driven; process-global" event model with
  process-scoped session events. The goal is to be able to have dedicated
  handler for each process.
  2. Add to protocol contract to support explicit lifecycle status and
  stream ordering:
  - `WriteResponse` now returns `WriteStatus` (Accepted, UnknownProcess,
  StdinClosed, Starting) instead of a bool.
    - Added seq fields to output/exited notifications.
    - Added terminal process/closed notification.
  3. Demultiplexed remote notifications into per-process channels. Same as
  for the event sys
  4. Local and remote backends now both implement ExecBackend.
  5. Local backend wraps internal process ID/operations into per-process
  ExecProcess objects.
  6. Remote backend registers a session channel before launch and
  unregisters on failed launch.
  
  ## What changed in `unified_exec`
  
  1. Added unified process-state model and backend-neutral process
  wrapper. This will probably disappear in the future, but it makes it
  easier to keep the work flowing on both side.
  - `UnifiedExecProcess` now handles both local PTY sessions and remote
  exec-server processes through a shared `ProcessHandle`.
  - Added `ProcessState` to track has_exited, exit_code, and terminal
  failure message consistently across backends.
  2. Routed write and lifecycle handling through process-level methods.
  
  ## Some rationals
  
  1. The change centralizes execution transport in exec-server while
  preserving policy and orchestration ownership in core, avoiding
  duplicated launch approval logic. This comes from internal discussion.
  2. Session-scoped events remove coupling/cross-talk between processes
  and make stream ordering and terminal state explicit (seq, closed,
  failed).
  3. The failure-path surfacing (remote launch failures, write failures,
  transport disconnects) makes command tool output and cleanup behavior
  deterministic
  
  ## Follow-ups:
  * Unify the concept of thread ID behind an obfuscated struct
  * FD handling
  * Full zsh-fork compatibility
  * Full network sandboxing compatibility
  * Handle ws disconnection

  jif-oai · 2026-03-26 15:22:34 +01:00
  7dac332c93
• Add exec-server exec RPC implementation (#15090)

  Stacked PR 2/3, based on the stub PR.
  
  Adds the exec RPC implementation and process/event flow in exec-server
  only.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-03-19 19:00:36 +00:00
  1d210f639e
• Add exec-server stub server and protocol docs (#15089)

  Stacked PR 1/3.
  
  This is the initialize-only exec-server stub slice: binary/client
  scaffolding and protocol docs, without exec/filesystem implementation.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-03-19 00:30:05 +00:00
  81996fcde6