44 Commits

• Persist Cloudflare affinity cookies for MCP HTTP (#29516)

  [Codex Thread
  019ef1f9-36e2-7e91-9337-504f097b9dc1](https://codex-thread-link.openai.chatgpt-team.site/thread/019ef1f9-36e2-7e91-9337-504f097b9dc1)
  
  ## Why
  
  Hosted plugin-service Streamable HTTP MCP traffic uses
  `https://chatgpt.com/backend-api/ps/mcp` and depends on Cloudflare's
  `__cflb` cookie for load-balancer affinity. The local and exec-server
  `http/request` path built a fresh reqwest client for each request
  without installing Codex's existing shared ChatGPT Cloudflare cookie
  store, so affinity could be lost between calls.
  
  This is an affinity-hardening change motivated by an incident
  investigation. It does not establish the broader connector-cache
  incident RCA or claim to fix that incident in full.
  
  ## What changed
  
  - Install the existing process-local, strictly allowlisted ChatGPT
  Cloudflare cookie store on the reqwest client used by
  `ReqwestHttpClient`.
  - Fresh clients now share allowed Cloudflare infrastructure cookies
  within the process that originates the local or exec-server network
  request.
  - Keep the existing HTTPS ChatGPT-host and Cloudflare-cookie-name
  restrictions. This does not introduce a general cookie jar or send
  ChatGPT Cloudflare cookies to unrelated hosts.
  
  ## Test coverage
  
  - `codex-client` unit coverage verifies that the existing strict store
  accepts and returns `__cflb` for HTTPS ChatGPT URLs.
  - The exec-server HTTPS integration test sends four independent
  `http/request` calls through a local TLS-intercepting proxy and verifies
  that:
  - `Set-Cookie: __cflb=west` is sent on the next plugin-service request;
    - a later `Set-Cookie: __cflb=central` replaces the stored value;
    - non-Cloudflare session cookies are discarded;
    - no stored ChatGPT Cloudflare cookie is sent to a non-ChatGPT host.
  - `just test -p codex-client` — 38 passed.
  - `just test -p codex-exec-server --test chatgpt_cloudflare_affinity` —
  1 passed.
  - `just bazel-lock-check` — passed.
  
  ## Non-goals
  
  - No persistence of ChatGPT auth, account, session, residency, or
  arbitrary cookies.
  - No cookie persistence for third-party MCP servers.
  - No special composition of caller-provided `Cookie` headers.
  - No plugin-service, connector-cache, Habitat/habicache, routing,
  redirect, or API-contract changes.
  - No broader incident RCA conclusions.

  stevenlee-oai · 2026-06-26 02:23:24 -04:00
  b5866eebd6
• [codex] Trace exec-server JSON-RPC requests (#27466)

  ## Why
  
  Exec-server JSON-RPC calls can cross local and remote transports, but
  trace context stopped at the RPC boundary. That made client and server
  work difficult to correlate when diagnosing latency or failures.
  
  ## What changed
  
  - Propagate the current W3C trace context on outbound JSON-RPC requests.
  - Parent inbound request spans from received trace context.
  - Record the received JSON-RPC method on server spans and keep each span
  open through response enqueue.
  - Add only the OTEL dependencies required by the exec-server crate.
  
  ## Stack
  
  Review and land this stack in order:
  
  1. #27466 — trace exec-server JSON-RPC requests **(this PR)**
  2. #27467 — record bounded connection, request, and process lifecycle
  metrics
  3. #27470 — observe remote registration and Noise rendezvous lifecycle
  
  ## Validation
  
  - `just test -p codex-exec-server --lib` (153 passed)
  - `just bazel-lock-check`
  - `just fix -p codex-exec-server`

  richardopenai · 2026-06-24 12:50:18 -07:00
  74dcce594d
• protocol: separate app and exec RPC ownership (#29714)

  ## Why
  
  The app-server and exec-server expose separate JSON-RPC APIs, but
  exec-server currently sources its serialized protocol and envelope types
  through app-server-oriented code. Giving each API an explicit owner
  makes the crate boundary legible without introducing shared generic
  envelopes.
  
  ## What changed
  
  - Added `codex-exec-server-protocol` to own exec DTOs, process IDs, and
  JSON-RPC envelopes.
  - Updated exec-server clients, transports, handlers, and tests to use
  the new crate.
  - Exposed app-server's existing JSON-RPC types through a public `rpc`
  module while retaining root re-exports.
  - Preserved existing wire shapes, including exec `PathUri` behavior.
  
  ## Stack
  
  This is PR 1 of 6. Next: [PR
  #29721](https://github.com/openai/codex/pull/29721), which moves auth
  mode below the app wire boundary.
  
  ## Validation
  
  - Exec-server protocol and server coverage passed in the focused
  protocol test runs.
  - App-server protocol schema fixtures passed.

  Adam Perry @ OpenAI · 2026-06-23 22:37:31 +00:00
  829f5b6b59
• Prepare managed network sandbox context (#29456)

  ## Why
  
  Managed network configures commands to use local HTTP and SOCKS proxies.
  For commands delegated to the exec server, the proxy environment and the
  sandbox policy were prepared separately. On macOS, that meant a command
  could receive `HTTPS_PROXY=http://127.0.0.1:43123` while Seatbelt still
  denied access to port `43123`.
  
  ## What changed
  
  `NetworkProxy` now prepares the command environment and sandbox context
  together from the same runtime snapshot:
  
  ```text
  Prepared managed network
  ├── command environment: HTTPS_PROXY=http://127.0.0.1:43123
  └── sandbox context: allow outbound to 127.0.0.1:43123
  ```
  
  That context travels with remote exec requests. The exec server
  preserves the managed proxy and CA environment, and macOS Seatbelt
  allows only the prepared loopback proxy ports without enabling broad
  network access or local binding.
  
  The protocol field is optional and the existing enforcement flag remains
  in place, preserving compatibility with callers that do not send the new
  context.

  jif · 2026-06-23 20:07:09 +01:00
  e476fc16ce
• feat(exec-server): add Noise rendezvous environment (#28774)

  ## Why
  
  Codex can run a remote exec server through the Noise relay, but the
  normal
  environment-manager path could not establish an
  environment-registry-backed
  harness connection. Signed rendezvous URLs and harness authorizations
  are
  short-lived, so reconnects must fetch a fresh bundle instead of
  retaining
  stale connection credentials. A stalled registry request must also fail
  within
  the regular remote connection deadline, without exposing these
  credentials in
  debug logs.
  
  Issue: N/A (internal environment-service integration).
  
  ## What Changed
  
  - Add environment-manager configuration for a registry-backed Noise
  rendezvous
    environment.
  - Request a fresh bundle from
  `/cloud/environment/{environment_id}/connect` for every physical harness
    connection, using the existing 10-second remote connection timeout.
  - Share the Environment Registry register, connect, and validate wire
  payloads
    through `codex-exec-server` and `codex-core-api`.
  - Redact the signed rendezvous URL and harness authorization from the
  public
    connect response's `Debug` output.
  - Add focused coverage for registry bundle retrieval, stalled requests,
  and
    credential redaction.

  Anton Panasenko · 2026-06-17 17:20:53 -07:00
  c274a83f8b
• [codex] exec-server: stream files in chunks (#28354)

  ## Why
  
  `fs/readFile` buffers the entire file in one response, which makes large
  remote reads expensive and prevents callers from applying backpressure.
  We need an opt-in streaming path with bounded block sizes while
  preserving the existing single-call API for small and sandboxed reads.
  
  ## What changed
  
  - Add `ExecServerClient::stream`, returning a named `FileReadStream`
  that implements `futures::Stream` and yields immutable 1 MiB byte
  blocks.
  - Add internal `fs/open`, `fs/readBlock`, and `fs/close` RPCs.
  `fs/readBlock` accepts an explicit offset and length.
  - Keep unsandboxed files open between block reads, cap open handles per
  connection, and clean them up on EOF, error, stream drop, explicit
  close, or connection shutdown.
  - Reject platform-sandboxed streaming opens instead of turning the
  one-shot sandbox helper into a persistent server. Existing `fs/readFile`
  behavior is unchanged.
  
  ## Testing
  
  - `just test -p codex-exec-server`
  - Integration coverage for 1 MiB chunking, exact block-boundary EOF,
  sandbox rejection, and continued reads from the opened file after path
  replacement.
  - Handle-manager coverage for non-sequential offsets, variable block
  lengths, the 128-handle limit, and capacity release after close.

  pakrym-oai · 2026-06-16 09:50:55 -07:00
  a4711b88dd
• exec-server: add Noise relay transport (#26242)

  ## Why
  
  Rendezvous forwards traffic between the orchestrator and exec-server.
  The endpoints need to authenticate each other and encrypt that traffic
  without trusting Rendezvous with plaintext or endpoint keys.
  
  ## Changes
  
  - Adds a hybrid Noise IK channel through Clatter using X25519,
  ML-KEM-768, AES-256-GCM, and SHA-256.
  - Binds each handshake to `environment_id`, `executor_registration_id`,
  and `stream_id`.
  - Pins the registry-provided executor key and carries the harness
  authorization inside the encrypted handshake.
  - Orders relay frames before consuming Noise nonces and fragments large
  JSON-RPC messages into bounded records.
  - Bounds handshake payloads, frames, streams, and message reassembly.
  
  Runtime activation is in
  [openai/codex#26245](https://github.com/openai/codex/pull/26245).
  
  ## Stack
  
  1. **[openai/codex#26242](https://github.com/openai/codex/pull/26242)**:
  Noise channel and relay transport
  2. [openai/codex#26245](https://github.com/openai/codex/pull/26245):
  remote registration and runtime activation
  
  ## Verification
  
  - `just test -p codex-exec-server`
  - Oversized initiator payload regression coverage
  - `just fix -p codex-exec-server`
  - `just bazel-lock-check`
  - `cargo shear`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  viyatb-oai · 2026-06-15 16:39:41 -07:00
  428cd44154
• [codex] Remove async_trait from first-party code (#27475)

  ## Why
  
  First-party async traits should expose their `Send` contracts explicitly
  without requiring `async_trait`. This completes the migration pattern
  established in #27303 and #27304.
  
  ## What changed
  
  - Replaced the remaining first-party `async_trait` traits with native
  return-position `impl Future + Send` where statically dispatched and
  explicit boxed `Send` futures where object safety is required.
  - Kept implementations behavior-preserving, outlining existing async
  bodies into inherent methods where that keeps the diff reviewable.
  - Removed all direct first-party `async-trait` dependencies and the
  workspace dependency declaration.
  - Added a cargo-deny policy that permits `async-trait` only through the
  remaining transitive wrapper crates.
  - Updated `rand` from 0.8.5 to 0.8.6 to resolve RUSTSEC-2026-0097 and
  keep the full cargo-deny check passing.
  
  ## Validation
  
  - `just test -p codex-exec-server`: 216 passed, 2 skipped.
  - `just test -p codex-model-provider`: 39 passed.
  - `just test -p codex-core` and `just test`: changed tests passed;
  remaining failures are environment-sensitive suites unrelated to this
  migration.
  - `cargo deny check`
  - `just fix`
  - `just fmt`
  - `cargo shear`
  - `just bazel-lock-check`

  Adam Perry @ OpenAI · 2026-06-11 18:16:39 -07:00
  5a56caf18c
• [codex] migrate ExecutorFileSystem paths to PathUri (#27424)

  ## Why
  
  We're moving exec-server to use PathUri for its internal path
  representations.
  
  ## What
  
  Move `ExecutorFileSystem` APIs to use `PathUri` instead of
  `AbsolutePathBuf`. Future changes will convert higher-level parts of
  exec-server.

  Adam Perry @ OpenAI · 2026-06-11 18:44:18 +00:00
  b2a4e3be27
• [codex] Add environment shell info (#26480)

  ## Why
  
  Shell detection needs to be available through the `Environment`
  abstraction so callers can ask the selected local or remote environment
  for shell metadata without adding a separate HTTP endpoint or parallel
  info-source path. This keeps shell metadata shaped like the existing
  environment-owned filesystem capability and lets remote environments
  answer through exec-server JSON-RPC.
  
  ## What changed
  
  - Added `environment/info` to the exec-server protocol/client/server and
  exposed `Environment::info()`.
  - Added local and remote environment info providers on `Environment`,
  following the existing capability-provider pattern used for filesystem
  access.
  - Moved the shared shell detection logic into `codex-shell-command` and
  kept core shell APIs as wrappers around that implementation.
  - Returned shell metadata as `EnvironmentInfo { shell: ShellInfo }`
  using the existing shell detection path.
  - Added a remote environment test that calls `Environment::info()`
  through an exec-server-backed environment.
  
  ## Validation
  
  - `git diff --check`
  - `just test -p codex-shell-command`
  - `just test -p codex-core -E 'test(/shell::tests::/)'`\n- `just test -p
  codex-exec-server environment`

  pakrym-oai · 2026-06-04 22:36:25 -07:00
  6a6a5f925e
• exec-server: support auth-backed remote executor registration (#22769)

  This updates remote `exec-server` registration to use normal Codex auth
  instead of a registry-issued credential. The registry request is built
  from the existing auth-provider path, which preserves the biscuit-only
  registry contract introduced in
  [openai/openai#924101](https://github.com/openai/openai/pull/924101)
  while removing the old remote registry bearer env var and its direct
  transport assumptions.
  
  The default remote flow uses persisted ChatGPT auth from the normal
  Codex config/storage path. This PR also includes the containerized Agent
  Identity path needed by
  [openai/openai#924260](https://github.com/openai/openai/pull/924260):
  remote `exec-server` accepts `--allow-agent-identity-auth`, permits
  Agent Identity auth loaded from `CODEX_ACCESS_TOKEN` only when that flag
  is present, and reuses the existing Agent task registration plus derived
  `AgentAssertion` header generation. API-key auth remains unsupported,
  and Agent Identity stays opt-in.
  
  Validation performed beyond normal presubmit coverage:
  - `cargo fmt --all --check`
  - `cargo check -p codex-cli`
  - `cargo test -p codex-exec-server`
  - `cargo test -p codex-cli exec_server_agent_identity_auth_flag_`
  - `cargo test -p codex-cli remote_exec_server_auth_mode_`
  
  I also attempted `cargo test -p codex-cli`. The new CLI tests passed
  inside that run, but the suite ended on an unrelated local
  marketplace-state failure in
  `plugin_list_excludes_unconfigured_repo_local_marketplaces`.

  Michael Zeng · 2026-05-16 12:48:28 -07:00
  b200dd1b6f
• feat(exec-server): use protobuf relay frames (#22343)

  ## Why
  
  Remote exec-server now needs one executor websocket to serve multiple
  harness JSON-RPC sessions. Rendezvous routes by `stream_id`, and the
  exec-server side needs to use the same stable relay frame contract
  instead of a hand-rolled JSON shape.
  
  The relay protocol also needs to make ownership boundaries clear:
  harness and executor endpoints own sequencing, acks, retries, duplicate
  suppression, segmentation, and reassembly; rendezvous only routes
  frames.
  
  ## What Changed
  
  - Add the checked-in `codex.exec_server.relay.v1.RelayMessageFrame`
  proto plus generated prost bindings for `codex-exec-server`.
  - Encode remote harness/executor relay traffic as binary protobuf
  websocket frames while keeping local websocket JSON-RPC unchanged.
  - Demux executor-side relay streams into independent
  `ConnectionProcessor` sessions keyed by `stream_id`.
  - Add a programmatic `RemoteExecutorConfig::with_bearer_token(...)`
  constructor for non-CLI callers and integration tests.
  - Add an integration test that starts the remote executor against a fake
  registry/rendezvous websocket and verifies two virtual streams share one
  executor websocket without cross-talk, including per-stream reset
  behavior.
  - Document the remote relay envelope, sequence ranges, `ack`/`ack_bits`,
  and endpoint responsibilities in `exec-server/README.md`.
  
  ## Verification
  
  - `cargo test -p codex-exec-server --test relay
  multiplexed_remote_executor_routes_independent_virtual_streams --
  --exact`
  - `cargo test -p codex-exec-server --test relay`
  - `cargo test -p codex-exec-server` passed outside the sandbox. The
  sandboxed run hit macOS `sandbox-exec: sandbox_apply: Operation not
  permitted` in filesystem sandbox tests.

  Anton Panasenko · 2026-05-12 16:50:45 -07:00
  ac466c0dbd
• [exec-server] serve websocket listener via HTTP upgrade (#21963)

  ## Why
  
  `codex exec-server` should keep the existing public `ws://IP:PORT` URL
  shape while serving that websocket connection through an HTTP upgrade
  path internally. That keeps the client-facing configuration simple and
  allows the listener to work through intermediate HTTP-aware
  infrastructure.
  
  ## What changed
  
  - keep the emitted and configured exec-server URL as `ws://IP:PORT`
  - serve that websocket endpoint through Axum HTTP upgrade handling on
  `/`
  - expose `GET /readyz` from the same listener for readiness checks
  - route upgraded Axum websocket streams through the shared JSON-RPC
  connection machinery
  - initialize the rustls crypto provider before websocket client
  connections
  - preserve inbound binary websocket JSON-RPC parsing for compatibility
  with the prior transport behavior
  
  ## Verification
  
  - `cargo test -p codex-exec-server --test health --test process --test
  websocket --test initialize --test exec_process`

  Ruslan Nigmatullin · 2026-05-11 17:04:21 -07:00
  95d8669ab2
• [codex] support executor registry remote environments (#21323)

  ## Summary
  
  Support registry-backed remote executors end to end so downstream
  services can resolve an executor id into an exec-server URL and make
  that environment available to Codex without relying on the legacy cloud
  environments flow.
  
  ## What changed
  
  - switch remote executor registration to the executor registry bootstrap
  contract
  - allow named remote environments to be inserted into
  `EnvironmentManager` at runtime
  - add the experimental app-server RPC `environment/add` so initialized
  experimental clients can register those remote environments for later
  `thread/start` and `turn/start` selection
  
  ## Validation
  
  Ran focused validation locally:
  - `cargo test -p codex-exec-server environment_manager_`
  - `cargo test -p codex-exec-server
  register_executor_posts_with_bearer_token_header`
  - `cargo test -p codex-app-server-protocol`

  Michael Zeng · 2026-05-08 16:30:07 -07:00
  8f4020846e
• Add CODEX_HOME environments TOML provider (#20666)

  ## Why
  
  After stdio transports and provider-owned defaults exist, Codex needs a
  config-backed provider that can describe more than the single legacy
  `CODEX_EXEC_SERVER_URL` remote. This PR adds that provider without
  activating it in product entrypoints yet, keeping parser/validation
  review separate from runtime wiring.
  
  **Stack position:** this is PR 4 of 5. It builds on PR 3's
  provider/default model and adds the `environments.toml` provider used by
  PR 5.
  
  ## What Changed
  
  - Add `environment_toml.rs` as the TOML-specific home for parsing,
  validation, and provider construction.
  - Keep the TOML schema/provider structs private; the public constructor
  added here is `EnvironmentManager::from_codex_home(...)`.
  - Add `TomlEnvironmentProvider`, including validation for:
    - reserved ids such as `local` and `none`
    - duplicate ids
    - unknown explicit defaults
    - empty programs or URLs
    - exactly one of `url` or `program` per configured environment
  - Support websocket environments with `url = "ws://..."` / `wss://...`.
  - Support stdio-command environments with `program = "..."`.
  - Add helpers to load `environments.toml` from `CODEX_HOME`, but do not
  wire entrypoints to call them yet.
  - Add the `toml` dependency for parsing.
  
  ## Stack
  
  - 1. https://github.com/openai/codex/pull/20663 - Add stdio exec-server
  listener
  - 2. https://github.com/openai/codex/pull/20664 - Add stdio exec-server
  client transport
  - 3. https://github.com/openai/codex/pull/20665 - Make environment
  providers own default selection
  - **4. This PR:** https://github.com/openai/codex/pull/20666 - Add
  CODEX_HOME environments TOML provider
  - 5. https://github.com/openai/codex/pull/20667 - Load configured
  environments from CODEX_HOME
  
  Split from original draft: https://github.com/openai/codex/pull/20508
  
  ## Validation
  
  Not run locally; this was split out of the original draft stack.
  
  ## Documentation
  
  This introduces the config shape for `environments.toml`; user-facing
  documentation should be added before this stack is treated as a
  documented public workflow.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-05-08 01:37:47 +00:00
  07b695190f
• Add cloud executor registration to exec-server (#19575)

  ## Summary
  This PR adds the first `codex-rs` milestone for remote-exec e2e: a local
  `codex exec-server` can now register itself with
  `codex-cloud-environments` and attach to the returned rendezvous
  websocket.
  
  At a high level, `codex exec-server --cloud ...` now:
  - loads ChatGPT auth from normal Codex config
  - registers an executor with `codex-cloud-environments`
  - receives a signed rendezvous websocket URL
  - serves the existing exec-server JSON-RPC protocol over that websocket
  
  ## What Changed
  - Added `--cloud`, `--cloud-base-url`, `--cloud-environment-id`, and
  `--cloud-name` to `codex exec-server`
  - Added a new `exec-server/src/cloud.rs` module that handles:
    - registration requests
    - auth/header setup
    - bounded auth retry on `401/403`
    - reconnect/backoff after websocket disconnects
  - Reused the existing `ConnectionProcessor` / `ExecServerHandler` path
  so cloud mode serves the same exec/filesystem RPC surface as local
  websocket mode
  - Added cloud-specific error variants and minimal docs for the new mode
  
  ## Testing
  Manual e2e test that fully goes through exec server flow with our codex
  cloud agent as orchestrator

  Michael Zeng · 2026-05-05 22:01:48 +00:00
  d0f9d5eba2
• Refactor exec-server filesystem API into codex-file-system (#19892)

  ## Summary
  - Extracted the shared filesystem types and `ExecutorFileSystem` trait
  into a new `codex-file-system` crate
  - Switched `codex-config` and `codex-git-utils` to depend on that crate
  instead of `codex-exec-server`
  - Kept `codex-exec-server` re-exporting the same API for existing
  callers
  
  ## Testing
  - Ran `cargo test -p codex-file-system`
  - Ran `cargo test -p codex-git-utils`
  - Ran `cargo test -p codex-config`
  - Ran `cargo test -p codex-exec-server`
  - Ran `just fix -p codex-file-system`, `just fix -p codex-git-utils`,
  `just fix -p codex-config`, `just fix -p codex-exec-server`
  - Ran `just fmt`
  - Updated and verified the Bazel module lockfile

  Michael Zeng · 2026-04-27 17:43:15 -07:00
  a3350de855
• [codex] Move config loading into codex-config (#19487)

  ## Why
  
  Config loading had become split across crates: `codex-config` owned the
  config types and merge logic, while `codex-core` still owned the loader
  that assembled the layer stack. This change consolidates that
  responsibility in `codex-config`, so the crate that defines config
  behavior also owns how configs are discovered and loaded.
  
  To make that move possible without reintroducing the old dependency
  cycle, the shell-environment policy types and helpers that
  `codex-exec-server` needs now live in `codex-protocol` instead of
  flowing through `codex-config`.
  
  This also makes the migrated loader tests more deterministic on machines
  that already have managed or system Codex config installed by letting
  tests override the system config and requirements paths instead of
  reading the host's `/etc/codex`.
  
  ## What Changed
  
  - moved the config loader implementation from `codex-core` into
  `codex-config::loader` and deleted the old `core::config_loader` module
  instead of leaving a compatibility shim
  - moved shell-environment policy types and helpers into
  `codex-protocol`, then updated `codex-exec-server` and other downstream
  crates to import them from their new home
  - updated downstream callers to use loader/config APIs from
  `codex-config`
  - added test-only loader overrides for system config and requirements
  paths so loader-focused tests do not depend on host-managed config state
  - cleaned up now-unused dependency entries and platform-specific cfgs
  that were surfaced by post-push CI
  
  ## Testing
  
  - `cargo test -p codex-config`
  - `cargo test -p codex-core config_loader_tests::`
  - `cargo test -p codex-protocol -p codex-exec-server -p
  codex-cloud-requirements -p codex-rmcp-client --lib`
  - `cargo test --lib -p codex-app-server-client -p codex-exec`
  - `cargo test --no-run --lib -p codex-app-server`
  - `cargo test -p codex-linux-sandbox --lib`
  - `cargo shear`
  - `just bazel-lock-check`
  
  ## Notes
  
  - I did not chase unrelated full-suite failures outside the migrated
  loader surface.
  - `cargo test -p codex-core --lib` still hits unrelated proxy-sensitive
  failures on this machine, and Windows CI still shows unrelated
  long-running/timeouting test noise outside the loader migration itself.

  pakrym-oai · 2026-04-26 15:10:53 -07:00
  9c3abcd46c
• [3/4] Add executor-backed RMCP HTTP client (#18583)

  ### Why
  The RMCP layer needs a Streamable HTTP client that can talk either
  directly over `reqwest` or through the executor HTTP runner without
  duplicating MCP session logic higher in the stack. This PR adds that
  client-side transport boundary so remote Streamable HTTP MCP can reuse
  the same RMCP flow as the local path.
  
  ### What
  - Add a shared `rmcp-client/src/streamable_http/` module with:
    - `transport_client.rs` for the local-or-remote transport enum
    - `local_client.rs` for the direct `reqwest` implementation
    - `remote_client.rs` for the executor-backed implementation
    - `common.rs` for the small shared Streamable HTTP helpers
  - Teach `RmcpClient` to build Streamable HTTP transports in either local
  or remote mode while keeping the existing OAuth ownership in RMCP.
  - Translate remote POST, GET, and DELETE session operations into
  executor `http/request` calls.
  - Preserve RMCP session expiry handling and reconnect behavior for the
  remote transport.
  - Add remote transport coverage in
  `rmcp-client/tests/streamable_http_remote.rs` and keep the shared test
  support in `rmcp-client/tests/streamable_http_test_support.rs`.
  
  ### Verification
  - `cargo check -p codex-rmcp-client`
  - online CI
  
  ### Stack
  1. #18581 protocol
  2. #18582 runner
  3. #18583 RMCP client
  4. #18584 manager wiring and local/remote coverage
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-22 17:38:04 -07:00
  0e78ce80ee
• [2/4] Implement executor HTTP request runner (#18582)

  ### Why
  Remote streamable HTTP MCP needs the executor to perform ordinary HTTP
  requests on the executor side. This keeps network placement aligned with
  `experimental_environment = "remote"` without adding MCP-specific
  executor APIs.
  
  ### What
  - Add an executor-side `http/request` runner backed by `reqwest`.
  - Validate request method and URL scheme, preserving the transport
  boundary at plain HTTP.
  - Return buffered responses for ordinary calls and emit ordered
  `http/request/bodyDelta` notifications for streaming responses.
  - Register the request handler in the exec-server router.
  - Document the runner entrypoint, conversion helpers, body-stream
  bridge, notification sender, timeout behavior, and new integration-test
  helpers.
  - Add exec-server integration tests with the existing websocket harness
  and a local TCP HTTP peer for buffered and streamed responses, with
  comments spelling out what each test proves and its
  setup/exercise/assert phases.
  
  ### Stack
  1. #18581 protocol
  2. #18582 runner
  3. #18583 RMCP client
  4. #18584 manager wiring and local/remote coverage
  
  ### Verification
  - `just fmt`
  - `cargo check -p codex-exec-server -p codex-rmcp-client --tests`
  - `cargo check -p codex-core --test all` compile-only
  - `git diff --check`
  - Online full CI is running from the `full-ci` branch, including the
  remote Rust test job.
  
  Co-authored-by: Codex <noreply@openai.com>
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-22 20:36:34 +00:00
  9360f267f3
• Stabilize exec-server filesystem tests in CI (#17671)

  ## Summary\n- add an exec-server package-local test helper binary that
  can run exec-server and fs-helper flows\n- route exec-server filesystem
  tests through that helper instead of cross-crate codex helper
  binaries\n- stop relying on Bazel-only extra binary wiring for these
  tests\n\n## Testing\n- not run (per repo guidance for codex changes)
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-04-13 16:53:42 -07:00
  280a4a6d42
• Build remote exec env from exec-server policy (#17216)

  ## Summary
  - add an exec-server `envPolicy` field; when present, the server starts
  from its own process env and applies the shell environment policy there
  - keep `env` as the exact environment for local/embedded starts, but
  make it an overlay for remote unified-exec starts
  - move the shell-environment-policy builder into `codex-config` so Core
  and exec-server share the inherit/filter/set/include behavior
  - overlay only runtime/sandbox/network deltas from Core onto the
  exec-server-derived env
  
  ## Why
  Remote unified exec was materializing the shell env inside Core and
  forwarding the whole map to exec-server, so remote processes could
  inherit the orchestrator machine's `HOME`, `PATH`, etc. This keeps the
  base env on the executor while preserving Core-owned runtime additions
  like `CODEX_THREAD_ID`, unified-exec defaults, network proxy env, and
  sandbox marker env.
  
  ## Validation
  - `just fmt`
  - `git diff --check`
  - `cargo test -p codex-exec-server --lib`
  - `cargo test -p codex-core --lib unified_exec::process_manager::tests`
  - `cargo test -p codex-core --lib exec_env::tests`
  - `cargo test -p codex-core --lib exec_env_tests` (compile-only; filter
  matched 0 tests)
  - `cargo test -p codex-config --lib shell_environment` (compile-only;
  filter matched 0 tests)
  - `just bazel-lock-update`
  
  ## Known local validation issue
  - `just bazel-lock-check` is not runnable in this checkout: it invokes
  `./scripts/check-module-bazel-lock.sh`, which is missing.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>
  Co-authored-by: pakrym-oai <pakrym@openai.com>

  jif-oai · 2026-04-13 09:59:08 +01:00
  bacb92b1d7
• Stabilize exec-server process tests (#17605)

  Problem: After #17294 switched exec-server tests to launch the top-level
  `codex exec-server` command, parallel remote exec-process cases can
  flake while waiting for the child server's listen URL or transport
  shutdown.
  
  Solution: Serialize remote exec-server-backed process tests and harden
  the harness so spawned servers are killed on drop and shutdown waits for
  the child process to exit.

  Eric Traut · 2026-04-13 00:31:13 -07:00
  6550007cca
• Run exec-server fs operations through sandbox helper (#17294)

  ## Summary
  - run exec-server filesystem RPCs requiring sandboxing through a
  `codex-fs` arg0 helper over stdin/stdout
  - keep direct local filesystem execution for `DangerFullAccess` and
  external sandbox policies
  - remove the standalone exec-server binary path in favor of top-level
  arg0 dispatch/runtime paths
  - add sandbox escape regression coverage for local and remote filesystem
  paths
  
  ## Validation
  - `just fmt`
  - `git diff --check`
  - remote devbox: `cd codex-rs && bazel test --bes_backend=
  --bes_results_url= //codex-rs/exec-server:all` (6/6 passed)
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-04-12 18:36:03 -07:00
  d626dc3895
• feat: move exec-server ownership (#16344)

  This introduces session-scoped ownership for exec-server so ws
  disconnects no longer immediately kill running remote exec processes,
  and it prepares the protocol for reconnect-based resume.
  - add session_id / resume_session_id to the exec-server initialize
  handshake
    - move process ownership under a shared session registry
  - detach sessions on websocket disconnect and expire them after a TTL
  instead of killing processes immediately (we will resume based on this)
  - allow a new connection to resume an existing session and take over
  notifications/ownership
  - I use UUID to make them not predictable as we don't have auth for now
  - make detached-session expiry authoritative at resume time so teardown
  wins at the TTL boundary
  - reject long-poll process/read calls that get resumed out from under an
  older attachment
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-10 14:11:47 +01:00
  085ffb4456
• Add sandbox support to filesystem APIs (#16751)

  ## Summary
  - add optional `sandboxPolicy` support to the app-server filesystem
  request surface
  - thread sandbox-aware filesystem options through app-server and
  exec-server adapters
  - enforce sandboxed read/write access in the filesystem abstraction with
  focused local and remote coverage
  
  ## Validation
  - `cargo test -p codex-app-server-protocol`
  - `cargo test -p codex-exec-server file_system`
  - `cargo test -p codex-app-server suite::v2::fs`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-04-08 12:10:48 -07:00
  f383cc980d
• feat: exec-server prep for unified exec (#15691)

  This PR partially rebase `unified_exec` on the `exec-server` and adapt
  the `exec-server` accordingly.
  
  ## What changed in `exec-server`
  
  1. Replaced the old "broadcast-driven; process-global" event model with
  process-scoped session events. The goal is to be able to have dedicated
  handler for each process.
  2. Add to protocol contract to support explicit lifecycle status and
  stream ordering:
  - `WriteResponse` now returns `WriteStatus` (Accepted, UnknownProcess,
  StdinClosed, Starting) instead of a bool.
    - Added seq fields to output/exited notifications.
    - Added terminal process/closed notification.
  3. Demultiplexed remote notifications into per-process channels. Same as
  for the event sys
  4. Local and remote backends now both implement ExecBackend.
  5. Local backend wraps internal process ID/operations into per-process
  ExecProcess objects.
  6. Remote backend registers a session channel before launch and
  unregisters on failed launch.
  
  ## What changed in `unified_exec`
  
  1. Added unified process-state model and backend-neutral process
  wrapper. This will probably disappear in the future, but it makes it
  easier to keep the work flowing on both side.
  - `UnifiedExecProcess` now handles both local PTY sessions and remote
  exec-server processes through a shared `ProcessHandle`.
  - Added `ProcessState` to track has_exited, exit_code, and terminal
  failure message consistently across backends.
  2. Routed write and lifecycle handling through process-level methods.
  
  ## Some rationals
  
  1. The change centralizes execution transport in exec-server while
  preserving policy and orchestration ownership in core, avoiding
  duplicated launch approval logic. This comes from internal discussion.
  2. Session-scoped events remove coupling/cross-talk between processes
  and make stream ordering and terminal state explicit (seq, closed,
  failed).
  3. The failure-path surfacing (remote launch failures, write failures,
  transport disconnects) makes command tool output and cleanup behavior
  deterministic
  
  ## Follow-ups:
  * Unify the concept of thread ID behind an obfuscated struct
  * FD handling
  * Full zsh-fork compatibility
  * Full network sandboxing compatibility
  * Handle ws disconnection

  jif-oai · 2026-03-26 15:22:34 +01:00
  7dac332c93
• Refactor ExecServer filesystem split between local and remote (#15232)

  For each feature we have:
  1. Trait exposed on environment
  2. **Local Implementation** of the trait
  3. Remote implementation that uses the client to proxy via network
  4. Handler implementation that handles PRC requests and calls into
  **Local Implementation**

  pakrym-oai · 2026-03-19 17:08:04 -07:00
  403b397e4e
• Add exec-server exec RPC implementation (#15090)

  Stacked PR 2/3, based on the stub PR.
  
  Adds the exec RPC implementation and process/event flow in exec-server
  only.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-03-19 19:00:36 +00:00
  1d210f639e
• Move environment abstraction into exec server (#15125)

  The idea is that codex-exec exposes an Environment struct with services
  on it. Each of those is a trait.
  
  Depending on construction parameters passed to Environment they are
  either backed by local or remote server but core doesn't see these
  differences.

  pakrym-oai · 2026-03-19 08:31:14 -07:00
  dee03da508
• Add exec-server stub server and protocol docs (#15089)

  Stacked PR 1/3.
  
  This is the initialize-only exec-server stub slice: binary/client
  scaffolding and protocol docs, without exec/filesystem implementation.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  starr-openai · 2026-03-19 00:30:05 +00:00
  81996fcde6
• refactor: delete exec-server and move execve wrapper into shell-escalation (#12632)

  ## Why
  
  We already plan to remove the shell-tool MCP path, and doing that
  cleanup first makes the follow-on `shell-escalation` work much simpler.
  
  This change removes the last remaining reason to keep
  `codex-rs/exec-server` around by moving the `codex-execve-wrapper`
  binary and shared shell test fixtures to the crates/tests that now own
  that functionality.
  
  ## What Changed
  
  ### Delete `codex-rs/exec-server`
  
  - Remove the `exec-server` crate, including the MCP server binary,
  MCP-specific modules, and its test support/test suite
  - Remove `exec-server` from the `codex-rs` workspace and update
  `Cargo.lock`
  
  ### Move `codex-execve-wrapper` into `codex-rs/shell-escalation`
  
  - Move the wrapper implementation into `shell-escalation`
  (`src/unix/execve_wrapper.rs`)
  - Add the `codex-execve-wrapper` binary entrypoint under
  `shell-escalation/src/bin/`
  - Update `shell-escalation` exports/module layout so the wrapper
  entrypoint is hosted there
  - Move the wrapper README content from `exec-server` to
  `shell-escalation/README.md`
  
  ### Move shared shell test fixtures to `app-server`
  
  - Move the DotSlash `bash`/`zsh` test fixtures from
  `exec-server/tests/suite/` to `app-server/tests/suite/`
  - Update `app-server` zsh-fork tests to reference the new fixture paths
  
  ### Keep `shell-tool-mcp` as a shell-assets package
  
  - Update `.github/workflows/shell-tool-mcp.yml` packaging so the npm
  artifact contains only patched Bash/Zsh payloads (no Rust binaries)
  - Update `shell-tool-mcp/package.json`, `shell-tool-mcp/src/index.ts`,
  and docs to reflect the shell-assets-only package shape
  - `shell-tool-mcp-ci.yml` does not need changes because it is already
  JS-only
  
  ## Verification
  
  - `cargo shear`
  - `cargo clippy -p codex-shell-escalation --tests`
  - `just clippy`

  Michael Bolin · 2026-02-23 20:10:22 -08:00
  38f84b6b29
• refactor: normalize unix module layout for exec-server and shell-escalation (#12556)

  ## Why
  Shell execution refactoring in `exec-server` had become split between
  duplicated code paths, which blocked a clean introduction of the new
  reusable shell escalation flow. This commit creates a dedicated
  foundation crate so later shell tooling changes can share one
  implementation.
  
  ## What changed
  - Added the `codex-shell-escalation` crate and moved the core escalation
  pieces (`mcp` protocol/socket/session flow, policy glue) that were
  previously in `exec-server` into it.
  - Normalized `exec-server` Unix structure under a dedicated `unix`
  module layout and kept non-Unix builds narrow.
  - Wired crate/build metadata so `shell-escalation` is a first-class
  workspace dependency for follow-on integration work.
  
  ## Verification
  - Built and linted the stack at this commit point with `just clippy`.
  
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/12556).
  * #12584
  * #12583
  * __->__ #12556

  Michael Bolin · 2026-02-23 09:28:17 -08:00
  5221575f23
• test: vendor zsh fork via DotSlash and stabilize zsh-fork tests (#12518)

  ## Why
  
  The zsh integration tests were still brittle in two ways:
  
  - they relied on `CODEX_TEST_ZSH_PATH` / environment-specific setup, so
  they often did not exercise the patched zsh fork that `shell-tool-mcp`
  ships
  - once the tests consistently used the vendored zsh fork, they exposed
  real Linux-specific zsh-fork issues in CI
  
  In particular, the Linux failures were not just test noise:
  
  - the zsh-fork launch path was dropping `ExecRequest.arg0`, so Linux
  `codex-linux-sandbox` arg0 dispatch did not run and zsh wrapper-mode
  could receive malformed arguments
  - the
  `turn_start_shell_zsh_fork_subcommand_decline_marks_parent_declined_v2`
  test uses the zsh exec bridge (which talks to the parent over a Unix
  socket), but Linux restricted sandbox seccomp denies `connect(2)`,
  causing timeouts on `ubuntu-24.04` x86/arm
  
  This PR makes the zsh tests consistently run against the intended
  vendored zsh fork and fixes/hardens the zsh-fork path so the Linux CI
  signal is meaningful.
  
  ## What Changed
  
  - Added a single shared test-only DotSlash file for the patched zsh fork
  at `codex-rs/exec-server/tests/suite/zsh` (analogous to the existing
  `bash` test resource).
  - Updated both app-server and exec-server zsh tests to use that shared
  DotSlash zsh (no duplicate zsh DotSlash file, no `CODEX_TEST_ZSH_PATH`
  dependency).
  - Updated the app-server zsh-fork test helper to resolve the shared
  DotSlash zsh and avoid silently falling back to host zsh.
  - Kept the app-server zsh-fork tests configured via `config.toml`, using
  a test wrapper path where needed to force `zsh -df` (and rewrite `-lc`
  to `-c`) for the subcommand-decline test.
  - Hardened the app-server subcommand-decline zsh-fork test for CI
  variability:
    - tolerate an extra `/responses` POST with a no-op mock response
  - tolerate non-target approval ordering while remaining strict on the
  two `/usr/bin/true` approvals and decline behavior
  - use `DangerFullAccess` on Linux for this one test because it validates
  zsh approval flow, not Linux sandbox socket restrictions
  - Fixed zsh-fork process launching on Linux by preserving `req.arg0` in
  `ZshExecBridge::execute_shell_request(...)` so `codex-linux-sandbox`
  arg0 dispatch continues to work.
  - Moved `maybe_run_zsh_exec_wrapper_mode()` under
  `arg0_dispatch_or_else(...)` in `app-server` and `cli` so wrapper-mode
  handling coexists correctly with arg0-dispatched helper modes.
  - Consolidated duplicated `dotslash -- fetch` resolution logic into
  shared test support (`core/tests/common/lib.rs`).
  - Updated `codex-rs/exec-server/tests/suite/accept_elicitation.rs` to
  use DotSlash zsh and hardened the zsh elicitation test for Bazel/zsh
  differences by:
    - resolving an absolute `git` path
    - running `git init --quiet .`
  - asserting success / `.git` creation instead of relying on banner text
  
  ## Verification
  
  - `cargo test -p codex-app-server turn_start_zsh_fork -- --nocapture`
  - `cargo test -p codex-exec-server accept_elicitation -- --nocapture`
  - `bazel test //codex-rs/exec-server:exec-server-all-test
  --test_output=streamed --test_arg=--nocapture
  --test_arg=accept_elicitation_for_prompt_rule_with_zsh`
  - CI (`rust-ci`) on the final cleaned commit: `Tests — ubuntu-24.04 -
  x86_64-unknown-linux-gnu` and `Tests — ubuntu-24.04-arm -
  aarch64-unknown-linux-gnu` passed in [run
  22291424358](https://github.com/openai/codex/actions/runs/22291424358)

  Michael Bolin · 2026-02-22 19:39:56 -08:00
  e8949f4507
• chore: remove codex-core public protocol/shell re-exports (#12432)

  ## Why
  
  `codex-rs/core/src/lib.rs` re-exported a broad set of types and modules
  from `codex-protocol` and `codex-shell-command`. That made it easy for
  workspace crates to import those APIs through `codex-core`, which in
  turn hides dependency edges and makes it harder to reduce compile-time
  coupling over time.
  
  This change removes those public re-exports so call sites must import
  from the source crates directly. Even when a crate still depends on
  `codex-core` today, this makes dependency boundaries explicit and
  unblocks future work to drop `codex-core` dependencies where possible.
  
  ## What Changed
  
  - Removed public re-exports from `codex-rs/core/src/lib.rs` for:
  - `codex_protocol::protocol` and related protocol/model types (including
  `InitialHistory`)
    - `codex_protocol::config_types` (`protocol_config_types`)
  - `codex_shell_command::{bash, is_dangerous_command, is_safe_command,
  parse_command, powershell}`
  - Migrated workspace Rust call sites to import directly from:
    - `codex_protocol::protocol`
    - `codex_protocol::config_types`
    - `codex_protocol::models`
    - `codex_shell_command`
  - Added explicit `Cargo.toml` dependencies (`codex-protocol` /
  `codex-shell-command`) in crates that now import those crates directly.
  - Kept `codex-core` internal modules compiling by using `pub(crate)`
  aliases in `core/src/lib.rs` (internal-only, not part of the public
  API).
  - Updated the two utility crates that can already drop a `codex-core`
  dependency edge entirely:
    - `codex-utils-approval-presets`
    - `codex-utils-cli`
  
  ## Verification
  
  - `cargo test -p codex-utils-approval-presets`
  - `cargo test -p codex-utils-cli`
  - `cargo check --workspace --all-targets`
  - `just clippy`

  Michael Bolin · 2026-02-20 23:45:35 -08:00
  1af2a37ada
• feat: introduce codex-utils-cargo-bin as an alternative to assert_cmd::Command (#8496)

  This PR introduces a `codex-utils-cargo-bin` utility crate that
  wraps/replaces our use of `assert_cmd::Command` and
  `escargot::CargoBuild`.
  
  As you can infer from the introduction of `buck_project_root()` in this
  PR, I am attempting to make it possible to build Codex under
  [Buck2](https://buck2.build) as well as `cargo`. With Buck2, I hope to
  achieve faster incremental local builds (largely due to Buck2's
  [dice](https://buck2.build/docs/insights_and_knowledge/modern_dice/)
  build strategy, as well as benefits from its local build daemon) as well
  as faster CI builds if we invest in remote execution and caching.
  
  See
  https://buck2.build/docs/getting_started/what_is_buck2/#why-use-buck2-key-advantages
  for more details about the performance advantages of Buck2.
  
  Buck2 enforces stronger requirements in terms of build and test
  isolation. It discourages assumptions about absolute paths (which is key
  to enabling remote execution). Because the `CARGO_BIN_EXE_*` environment
  variables that Cargo provides are absolute paths (which
  `assert_cmd::Command` reads), this is a problem for Buck2, which is why
  we need this `codex-utils-cargo-bin` utility.
  
  My WIP-Buck2 setup sets the `CARGO_BIN_EXE_*` environment variables
  passed to a `rust_test()` build rule as relative paths.
  `codex-utils-cargo-bin` will resolve these values to absolute paths,
  when necessary.
  
  
  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/8496).
  * #8498
  * __->__ #8496

  Michael Bolin · 2025-12-23 19:29:32 -08:00
  e61bae12e3
• Fixes mcp elicitation test that fails for me when run locally (#8020)
  Eric Traut · 2025-12-15 16:23:04 -08:00
  d9554c8191
• fix: add integration tests for codex-exec-mcp-server with execpolicy (#7617)

  This PR introduces integration tests that run
  [codex-shell-tool-mcp](https://www.npmjs.com/package/@openai/codex-shell-tool-mcp)
  as a user would. Note that this requires running our fork of Bash, so we
  introduce a [DotSlash](https://dotslash-cli.com/) file for `bash` so
  that we can run the integration tests on multiple platforms without
  having to check the binaries into the repository. (As noted in the
  DotSlash file, it is slightly more heavyweight than necessary, which may
  be worth addressing as disk space in CI is limited:
  https://github.com/openai/codex/pull/7678.)
  
  To start, this PR adds two tests:
  
  - `list_tools()` makes the `list_tools` request to the MCP server and
  verifies we get the expected response
  - `accept_elicitation_for_prompt_rule()` defines a `prefix_rule()` with
  `decision="prompt"` and verifies the elicitation flow works as expected
  
  Though the `accept_elicitation_for_prompt_rule()` test **only works on
  Linux**, as this PR reveals that there are currently issues when running
  the Bash fork in a read-only sandbox on Linux. This will have to be
  fixed in a follow-up PR.
  
  Incidentally, getting this test run to correctly on macOS also requires
  a recent fix we made to `brew` that hasn't hit a mainline release yet,
  so getting CI green in this PR required
  https://github.com/openai/codex/pull/7680.

  Michael Bolin · 2025-12-07 06:39:38 +00:00
  3c3d3d1adc
• feat: exec policy integration in shell mcp (#7609)

  adding execpolicy support into the `posix` mcp
  
  Co-authored-by: Michael Bolin <mbolin@openai.com>

  zhao-oai · 2025-12-04 21:55:54 -08:00
  b1c918d8f7
• chore: add cargo-deny configuration (#7119)

  - add GitHub workflow running cargo-deny on push/PR
  - document cargo-deny allowlist with workspace-dep notes and advisory
  ignores
  - align workspace crates to inherit version/edition/license for
  consistent checks

  Josh McKinney · 2025-11-24 12:22:18 -08:00
  ec49b56874
• feat: waiting for an elicitation should not count against a shell tool timeout (#6973)

  Previously, we were running into an issue where we would run the `shell`
  tool call with a timeout of 10s, but it fired an elicitation asking for
  user approval, the time the user took to respond to the elicitation was
  counted agains the 10s timeout, so the `shell` tool call would fail with
  a timeout error unless the user is very fast!
  
  This PR addresses this issue by introducing a "stopwatch" abstraction
  that is used to manage the timeout. The idea is:
  
  - `Stopwatch::new()` is called with the _real_ timeout of the `shell`
  tool call.
  - `process_exec_tool_call()` is called with the `Cancellation` variant
  of `ExecExpiration` because it should not manage its own timeout in this
  case
  - the `Stopwatch` expiration is wired up to the `cancel_rx` passed to
  `process_exec_tool_call()`
  - when an elicitation for the `shell` tool call is received, the
  `Stopwatch` pauses
  - because it is possible for multiple elicitations to arrive
  concurrently, it keeps track of the number of "active pauses" and does
  not resume until that counter goes down to zero
  
  I verified that I can test the MCP server using
  `@modelcontextprotocol/inspector` and specify `git status` as the
  `command` with a timeout of 500ms and that the elicitation pops up and I
  have all the time in the world to respond whereas previous to this PR,
  that would not have been possible.
  
  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/6973).
  * #7005
  * __->__ #6973
  * #6972

  Michael Bolin · 2025-11-20 16:45:38 -08:00
  8e5f38c0f0
• chore: refactor exec-server to prepare it for standalone MCP use (#6944)

  This PR reorganizes things slightly so that:
  
  - Instead of a single multitool executable, `codex-exec-server`, we now
  have two executables:
    - `codex-exec-mcp-server` to launch the MCP server
  - `codex-execve-wrapper` is the `execve(2)` wrapper to use with the
  `BASH_EXEC_WRAPPER` environment variable
  - `BASH_EXEC_WRAPPER` must be a single executable: it cannot be a
  command string composed of an executable with args (i.e., it no longer
  adds the `escalate` subcommand, as before)
  - `codex-exec-mcp-server` takes `--bash` and `--execve` as options.
  Though if `--execve` is not specified, the MCP server will check the
  directory containing `std::env::current_exe()` and attempt to use the
  file named `codex-execve-wrapper` within it. In development, this works
  out since these executables are side-by-side in the `target/debug`
  folder.
  
  With respect to testing, this also fixes an important bug in
  `dummy_exec_policy()`, as I was using `ends_with()` as if it applied to
  a `String`, but in this case, it is used with a `&Path`, so the
  semantics are slightly different.
  
  Putting this all together, I was able to test this by running the
  following:
  
  ```
  ~/code/codex/codex-rs$ npx @modelcontextprotocol/inspector \
      ./target/debug/codex-exec-mcp-server --bash ~/code/bash/bash
  ```
  
  If I try to run `git status` in `/Users/mbolin/code/codex` via the
  `shell` tool from the MCP server:
  
  <img width="1589" height="1335" alt="image"
  src="https://github.com/user-attachments/assets/9db6aea8-7fbc-4675-8b1f-ec446685d6c4"
  />
  
  then I get prompted with the following elicitation, as expected:
  
  <img width="1589" height="1335" alt="image"
  src="https://github.com/user-attachments/assets/21b68fe0-494d-4562-9bad-0ddc55fc846d"
  />
  
  Though a current limitation is that the `shell` tool defaults to a
  timeout of 10s, which means I only have 10s to respond to the
  elicitation. Ideally, the time spent waiting for a response from a human
  should not count against the timeout for the command execution. I will
  address this in a subsequent PR.
  
  ---
  
  Note `~/code/bash/bash` was created by doing:
  
  ```
  cd ~/code
  git clone https://github.com/bminor/bash
  cd bash
  git checkout a8a1c2fac029404d3f42cd39f5a20f24b6e4fe4b
  <apply the patch below>
  ./configure
  make
  ```
  
  The patch:
  
  ```
  diff --git a/execute_cmd.c b/execute_cmd.c
  index 070f5119..d20ad2b9 100644
  --- a/execute_cmd.c
  +++ b/execute_cmd.c
  @@ -6129,6 +6129,19 @@ shell_execve (char *command, char **args, char **env)
     char sample[HASH_BANG_BUFSIZ];
     size_t larray;
  
  +  char* exec_wrapper = getenv("BASH_EXEC_WRAPPER");
  +  if (exec_wrapper && *exec_wrapper && !whitespace (*exec_wrapper))
  +    {
  +      char *orig_command = command;
  +
  +      larray = strvec_len (args);
  +
  +      memmove (args + 2, args, (++larray) * sizeof (char *));
  +      args[0] = exec_wrapper;
  +      args[1] = orig_command;
  +      command = exec_wrapper;
  +    }
  +
  ```

  Michael Bolin · 2025-11-19 16:38:14 -08:00
  13d378f2ce
• fix: prepare ExecPolicy in exec-server for execpolicy2 cutover (#6888)

  This PR introduces an extra layer of abstraction to prepare us for the
  migration to execpolicy2:
  
  - introduces a new trait, `EscalationPolicy`, whose `determine_action()`
  method is responsible for producing the `EscalateAction`
  - the existing `ExecPolicy` typedef is changed to return an intermediate
  `ExecPolicyOutcome` instead of `EscalateAction`
  - the default implementation of `EscalationPolicy`,
  `McpEscalationPolicy`, composes `ExecPolicy`
  - the `ExecPolicyOutcome` includes `codex_execpolicy2::Decision`, which
  has a `Prompt` variant
  - when `McpEscalationPolicy` gets `Decision::Prompt` back from
  `ExecPolicy`, it prompts the user via an MCP elicitation and maps the
  result into an `ElicitationAction`
  - now that the end user can reply to an elicitation with `Decline` or
  `Cancel`, we introduce a new variant, `EscalateAction::Deny`, which the
  client handles by returning exit code `1` without running anything
  
  Note the way the elicitation is created is still not quite right, but I
  will fix that once we have things running end-to-end for real in a
  follow-up PR.

  Michael Bolin · 2025-11-19 13:55:29 -08:00
  056c8f8279
• exec-server (#6630)
  Jeremy Rose · 2025-11-19 00:20:19 +00:00
  c1391b9f94