mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
2dec46e30ae81aa4844cc7a5474ff5c95c6998c0
49 Commits
-
[codex] Record exec-server lifecycle metrics (#27467)
## Summary - Record bounded connection, request, and process lifecycle metrics. - Report active gauges from callbacks on every collection, including delta exports. - Serialize active-count updates so concurrent starts and finishes cannot publish stale values. - Serialize process exit, explicit termination, and shutdown through the process registry so exactly one completion result wins. - Keep the implementation small with single-owner RAII guards and one real OTLP/HTTP integration test using the existing `wiremock` dependency. ## Root cause Process exit and session shutdown previously used cloned completion state. That avoided duplicate emission, but it duplicated lifecycle ownership and made the ordering harder to reason about. The process registry mutex already defines the lifecycle ordering, so the final implementation stores the metric guard and termination flag directly on the process entry. Whichever path claims the entry first owns the completion result. Production metric export uses delta temporality. Event-only synchronous gauge recordings disappear after the next collection when no count changes, so active counts now use observable callbacks that report current state on every collection. The cleanup also removes the constant `result="accepted"` connection tag, redundant route and response assertions, a custom HTTP collector, and fallback initialization machinery that did not add behavior. ## Stack Review and land this stack in order: 1. #27466 — trace exec-server JSON-RPC requests 2. #27467 — record bounded connection, request, and process lifecycle metrics **(this PR)** 3. #27470 — observe remote registration and Noise rendezvous lifecycle ## Validation - `just test -p codex-exec-server --lib` (158 passed) - `just test -p codex-cli --test exec_server` (3 passed) - `just test -p codex-otel observable_gauge_is_collected_on_every_delta_snapshot` (1 passed) - `CARGO_BUILD_JOBS=1 just fix -p codex-otel -p codex-exec-server` - `just fmt` - `git diff --check`
richardopenai ·
2026-06-25 11:02:11 -07:00 -
Support OAuth for HTTP MCP servers from selected executor plugins (#28529)
## Why #28522 routes selected-plugin HTTP MCP traffic through the owning executor, but OAuth bootstrap and refresh still used host-local clients. Executor-only servers therefore cannot complete discovery or login through the same network boundary as the MCP connection. ## What changed - adapt `codex_exec_server::HttpClient` to RMCP 1.8's `OAuthHttpClient` contract - let RMCP own discovery, dynamic registration, PKCE, token exchange, and refresh - route auth status, persisted-token startup, and app-server login through the server runtime while preserving the existing local discovery path - add optional `threadId` to `mcpServer/oauth/login` and echo it in the completion notification - implement RMCP's redirect policy and 1 MiB OAuth response limit over executor HTTP - cover selected-thread OAuth discovery and login through an executor-only route Depends on #28522.
jif ·
2026-06-25 10:31:17 +01:00 -
[codex] Trace exec-server JSON-RPC requests (#27466)
## Why Exec-server JSON-RPC calls can cross local and remote transports, but trace context stopped at the RPC boundary. That made client and server work difficult to correlate when diagnosing latency or failures. ## What changed - Propagate the current W3C trace context on outbound JSON-RPC requests. - Parent inbound request spans from received trace context. - Record the received JSON-RPC method on server spans and keep each span open through response enqueue. - Add only the OTEL dependencies required by the exec-server crate. ## Stack Review and land this stack in order: 1. #27466 — trace exec-server JSON-RPC requests **(this PR)** 2. #27467 — record bounded connection, request, and process lifecycle metrics 3. #27470 — observe remote registration and Noise rendezvous lifecycle ## Validation - `just test -p codex-exec-server --lib` (153 passed) - `just bazel-lock-check` - `just fix -p codex-exec-server`
richardopenai ·
2026-06-24 12:50:18 -07:00 -
Add a bounded filesystem walk RPC (#29841)
Stack 1 of 3. Follow-ups: #29842 and #29844. ## What changes Adds a general bounded `fs/walk` operation to the exec server. The operation returns file and directory entries plus recoverable per-path errors. It skips symlinks, preserves the existing filesystem sandbox routing, and enforces depth, directory, entry, and response-size limits. This PR only defines and wires the filesystem operation. It does not change any callers yet.
jif ·
2026-06-24 16:05:43 +01:00 -
protocol: separate app and exec RPC ownership (#29714)
## Why The app-server and exec-server expose separate JSON-RPC APIs, but exec-server currently sources its serialized protocol and envelope types through app-server-oriented code. Giving each API an explicit owner makes the crate boundary legible without introducing shared generic envelopes. ## What changed - Added `codex-exec-server-protocol` to own exec DTOs, process IDs, and JSON-RPC envelopes. - Updated exec-server clients, transports, handlers, and tests to use the new crate. - Exposed app-server's existing JSON-RPC types through a public `rpc` module while retaining root re-exports. - Preserved existing wire shapes, including exec `PathUri` behavior. ## Stack This is PR 1 of 6. Next: [PR #29721](https://github.com/openai/codex/pull/29721), which moves auth mode below the app wire boundary. ## Validation - Exec-server protocol and server coverage passed in the focused protocol test runs. - App-server protocol schema fixtures passed.
Adam Perry @ OpenAI ·
2026-06-23 22:37:31 +00:00 -
Prepare managed network sandbox context (#29456)
## Why Managed network configures commands to use local HTTP and SOCKS proxies. For commands delegated to the exec server, the proxy environment and the sandbox policy were prepared separately. On macOS, that meant a command could receive `HTTPS_PROXY=http://127.0.0.1:43123` while Seatbelt still denied access to port `43123`. ## What changed `NetworkProxy` now prepares the command environment and sandbox context together from the same runtime snapshot: ```text Prepared managed network ├── command environment: HTTPS_PROXY=http://127.0.0.1:43123 └── sandbox context: allow outbound to 127.0.0.1:43123 ``` That context travels with remote exec requests. The exec server preserves the managed proxy and CA environment, and macOS Seatbelt allows only the prepared loopback proxy ports without enabling broad network access or local binding. The protocol field is optional and the existing enforcement flag remains in place, preserving compatibility with callers that do not send the new context.
jif ·
2026-06-23 20:07:09 +01:00 -
path-uri: clarify host-native path conversion (#29501)
## Why Downstream refactors are producing confusing code with this functionality having a very generic name. Encoding the specific conversion approach in the method name makes it clearer. ## What Rename `PathUri::from_path` to `PathUri::from_host_native_path` and update its Rust call sites.
Adam Perry @ OpenAI ·
2026-06-23 00:02:33 +00:00 -
Apply sandbox intent inside remote exec servers (#29113)
## Why PR #29108 lets the orchestrator send sandbox intent with `process/start` without wrapping the command for its own operating system. This PR completes that boundary by making the executor interpret and enforce the intent using its own filesystem paths and sandbox implementation. For example, a macOS TUI targeting a Linux devbox sends `/bin/bash -lc pwd`. The Linux executor turns that into its own `codex-linux-sandbox ... /bin/bash -lc pwd` launch. ## What changes - Keep `process/start` unchanged when no sandbox intent is present. - Convert sandbox `PathUri` values into native paths on the executor. - Bind symbolic `:workspace_roots` permissions to the executor's native sandbox cwd. - Select the sandbox implementation on the executor and wrap the original command immediately before spawning it. - Reject sandbox-required execution before spawning when the executor cannot enforce the intent. - Pass exec-server runtime paths into process creation so Linux can locate `codex-linux-sandbox`. The boundary is therefore: ```text orchestrator executor original argv + sandbox intent -> select and enforce local sandbox ``` This PR intentionally treats a denied remote command as an ordinary command failure. Draft follow-up #29424 carries a semantic `sandboxDenied` result back to unified exec for the existing approval and retry flow. ## Platform scope Linux and macOS use their existing direct-spawn sandbox transforms. Windows sandboxed remote process launch is intentionally unsupported in this PR. The current Windows direct-spawn wrapper does not correctly preserve arbitrary argv, TTY behavior, or pass the full child environment out of band. The executor rejects the request instead of running it incorrectly or unsandboxed. ## Known follow-ups - The transported permission profile can still contain orchestrator-materialized helper or explicit paths. A `TODO(jif)` marks where the executor boundary should receive pre-host-materialization permission intent. - The sandbox wrapper currently replaces a requested custom inner `arg0`. A `TODO(jif)` marks where this must be preserved or rejected explicitly. - Draft PR #29424 contains the deferred sandbox-denial classification and approval/retry behavior. ## Rollout assumption This executor-sandbox stack is unreleased and its client and executor are expected to move together. This PR does not add mixed-version negotiation with older exec servers.
jif ·
2026-06-22 12:45:37 +02:00 -
Test pipelined scalar exec-server requests (#29325)
## Summary This adds focused coverage for the simpler same-connection scalar request path. The exec-server connection already supports multiple in-flight JSON-RPC scalar requests on one connection. This test locks in that behavior by sending two normal requests before reading either response, without adding a batch frame or any new API surface. ## What changed - Added a processor-level test that initializes an exec-server connection. - Sends two scalar `environment/info` requests back-to-back on the same connection. - Verifies both responses come back on the same connection by request id. Checked locally with: - `just test -p codex-exec-server connection_accepts_pipelined_scalar_requests`
jif ·
2026-06-21 13:40:51 +02:00 -
Carry sandbox intent to remote exec servers (#29108)
## What changed PR #29099 stopped sending the orchestrator's concrete sandbox wrapper to a remote exec-server. Remote commands now arrive as plain native argv. This PR adds the next piece: Codex also sends portable sandbox intent next to that plain argv. For a remote unified-exec command, the request can now include: - the canonical permission profile before local workspace-root materialization - the sandbox cwd and workspace roots as `PathUri` values - Windows sandbox settings - the legacy Landlock setting - whether managed networking must be enforced The important part is that symbolic entries such as `:workspace_roots` stay symbolic while crossing the boundary. The executor can then bind them to its own workspace-root paths instead of receiving orchestrator-local absolute paths. The data travels through `ExecRequest` into `ExecParams`. Older exec-servers can still deserialize requests because the new fields have defaults. ## Why The orchestrator should not decide how another machine implements sandboxing. For example: - a local macOS Codex would normally build a Seatbelt command - a remote Linux executor needs a Linux sandbox command instead The orchestrator now sends the plain command plus the policy it intended to enforce. A later PR can let the exec-server choose and build the correct sandbox for its own operating system. ## Important detail This keeps the portable intent separate from the local `SandboxType`. `SandboxType::None` is ambiguous: - it can mean the command was explicitly approved to run without a sandbox - it can also mean the orchestrator host has no concrete sandbox implementation available Those cases are different for remote execution. This PR adds `sandbox_requested` so an executor can still receive sandbox intent when the orchestrator cannot build a local wrapper. Explicit unsandboxed retries still send no sandbox context. ## Behavior today This PR only transports the intent. The exec-server accepts the new fields but does not apply them yet. Remote commands therefore remain unsandboxed after this PR, just as they are after PR #29099. ## Follow-up The next PR will make exec-server read this portable intent, bind symbolic workspace permissions to executor-native roots, choose the sandbox for its own operating system, build the wrapper locally, and then spawn the command.
jif ·
2026-06-21 12:33:21 +02:00 -
Resume exec-server sessions after disconnect (#28512)
Supersedes #28288 (closed). ## Why A short WebSocket interruption currently ends every client-side process handle, even though exec-server keeps the server session and its processes alive for a short time. This is especially visible for executor-backed stdio MCP servers: a temporary connection loss becomes a permanent `Transport closed` error. The server already has the information needed to resume the session, but the client opens a fresh session instead of using it. This change reconnects below the process and MCP layers. Existing process handles stay valid, missed output is recovered, and the same server-side processes continue running. ## State machine One logical `ExecServerClient` stays alive while its underlying RPC connection changes generations. ```text transport closes +------------------------------------------------+ | v +-------------+ +-------------+ | Connected | | Recovering | +-------------+ +-------------+ ^ | | session resumed, processes caught up | retryable error +------------------------------------------------+ loops until deadline | | deadline or permanent error v +-------------+ | Failed | +-------------+ ``` ### `Connected` - New RPC calls use the current connection. - Process notifications are published in sequence order. - A disconnect only starts recovery if it came from the current connection generation. Late events from older generations cannot replace the active connection. ### `Recovering` - New calls wait instead of choosing a half-connected RPC client. - Existing process handles, wake subscriptions, and event subscriptions stay open. - Streaming HTTP response bodies fail immediately because their byte streams cannot be resumed safely. - Recovery first waits for process starts that were already in flight. A start whose result became ambiguous is cleaned up after reconnection instead of being silently adopted. - The client reconnects with the learned `session_id`. The server may briefly report that the old connection is still attached, so that error is retried until the detach finishes. - The notification consumer starts before the resume handshake completes. This prevents a busy process from filling the notification queue and blocking the initialize response. - Before installing the new connection, the client catches up every recoverable process with `process/read`. ### `Failed` - Recovery stops after 25 seconds or after a permanent error. - Waiting calls are released with one stable disconnect error. - Existing process sessions receive a terminal failure instead of waiting forever. ## Recovering process events Output, exit, and close events share one sequence. During normal operation, the client buffers early events until every lower sequence has been published. After reconnection, the client reads each process starting after its last published sequence: 1. Retained output chunks are inserted by sequence number. 2. Exit and close state are reconstructed in their sequence positions. 3. Events already received as live notifications are ignored as duplicates. 4. Newly contiguous events are published in order. 5. If the server no longer retains enough output to fill a sequence gap, only that process is terminated and failed. The recovered connection remains usable for other processes. The server reports its full next event sequence for unbounded reads, including exit and close events. Closed processes remain readable for the same 30-second window used to retain detached sessions. ## Other details - Detached server sessions are retained for 30 seconds, leaving margin around the client's 25-second recovery deadline. - Session attach and detach update the active notification sender under the same attachment lock, so an old connection cannot clear a newly attached sender. - A dedicated error code distinguishes the temporary "session is still attached" race from permanent initialization errors. - Process starts are identity-checked on both client and server. Cleanup from an older start cannot remove a newer process that reused the same ID. - Mutating requests that were already in flight when the transport closed are not replayed, because the client cannot know whether the server applied them. Requests started after recovery is known wait for the replacement connection. - We assume the server/client version stays in sync (on the before/after this PR) ## User impact Long-running commands and stdio MCP servers can survive a temporary exec-server WebSocket interruption without changing process IDs or losing output produced during the outage.
jif ·
2026-06-17 10:20:39 +02:00 -
[codex] exec-server: stream files in chunks (#28354)
## Why `fs/readFile` buffers the entire file in one response, which makes large remote reads expensive and prevents callers from applying backpressure. We need an opt-in streaming path with bounded block sizes while preserving the existing single-call API for small and sandboxed reads. ## What changed - Add `ExecServerClient::stream`, returning a named `FileReadStream` that implements `futures::Stream` and yields immutable 1 MiB byte blocks. - Add internal `fs/open`, `fs/readBlock`, and `fs/close` RPCs. `fs/readBlock` accepts an explicit offset and length. - Keep unsandboxed files open between block reads, cap open handles per connection, and clean them up on EOF, error, stream drop, explicit close, or connection shutdown. - Reject platform-sandboxed streaming opens instead of turning the one-shot sandbox helper into a persistent server. Existing `fs/readFile` behavior is unchanged. ## Testing - `just test -p codex-exec-server` - Integration coverage for 1 MiB chunking, exact block-boundary EOF, sandbox rejection, and continued reads from the opened file after path replacement. - Handle-manager coverage for non-sequential offsets, variable block lengths, the 128-handle limit, and capacity release after close.
pakrym-oai ·
2026-06-16 09:50:55 -07:00 -
[codex] Carry exec-server cwd as PathUri (#28032)
## Why This is the second-to-last place in the exec-server protocol that needs to migrate to URIs to support cross-OS operation. ## What - Change `ExecParams.cwd` to `PathUri`. - Keep the cwd URI-shaped through core and rmcp producers, converting it to `AbsolutePathBuf` only in `LocalProcess::start_process`. - Reject non-native cwd URIs before launch and update the affected protocol documentation and call sites.
Adam Perry @ OpenAI ·
2026-06-13 20:56:42 +00:00 -
[codex] Add size to internal filesystem metadata (#27927)
## Why `ExecutorFileSystem::get_metadata` reports file kind and timestamps but not size. Internal callers that need to enforce a size limit therefore have to read the complete file first, which is especially wasteful for remote filesystems. This adds the missing internal metadata so consumers can reject oversized files before transferring or buffering them. The field is named `size`, matching VS Code's `FileStat.size` filesystem convention. ## What changed - add `size: u64` to internal `FileMetadata` - populate it from the underlying filesystem metadata - carry it through sandbox-helper and remote exec-server responses - cover files, directories, symlink targets, and sandboxed reads across local and remote filesystem implementations The new field is intentionally not exposed through the app-server API. ## Testing - `just test -p codex-exec-server get_metadata` - `just test -p codex-exec-server file_system_sandboxed_metadata_and_read_allow_readable_root` - `just test -p codex-core-plugins` - `just test -p codex-skills-extension`
pakrym-oai ·
2026-06-12 12:12:08 -07:00 -
sandboxing: migrate cwd inputs to PathUri (#27816)
## Why Sandbox cwd values can cross app-server and exec-server host boundaries. They should retain URI semantics until the receiving host validates them instead of being interpreted early as native paths. ## What - Carry `PathUri` through filesystem sandbox contexts, sandbox commands, and transform inputs. - Convert command and policy cwd once in `SandboxManager::transform`, then keep launch requests native. - Preserve sandbox cwd over remote filesystem transport and reject non-native URIs without fallback. - Cache paired native/URI turn-environment cwd values during migration, with immutable access to keep them synchronized. - Extend existing protocol, forwarding, transform, and core runtime tests.
Adam Perry @ OpenAI ·
2026-06-12 11:38:01 -07:00 -
Remove fs/join and fs/parent from exec-server protocol (#27700)
## Summary Path composition is already handled by `PathUri`, leaving `fs/join` and `fs/parent` as redundant exec-server protocol surface. Because app-server and exec-server are deployed atomically, these obsolete methods can be removed without a compatibility shim. This removes the protocol constants and payloads, public client APIs, server registrations and handlers, and endpoint-only tests. Existing in-process `PathUri` join/parent coverage remains. ## Validation - `just test -p codex-exec-server` (215 passed, 2 skipped)
Adam Perry @ OpenAI ·
2026-06-11 15:48:53 -07:00 -
[codex] migrate exec-server filesystem protocol to PathUri (#27653)
Exec-server filesystem calls should preserve cross-platform `file:` URIs across the remote boundary instead of converting them through paths native to the client host. This changes the exec-server filesystem protocol DTOs to use `PathUri`, carries those values directly through remote and sandbox-helper transports, and keeps legacy native absolute-path request strings readable for compatibility. It also updates protocol documentation and coverage for URI serialization and non-native URI forwarding.
Adam Perry @ OpenAI ·
2026-06-11 15:09:12 -07:00 -
[codex] migrate ExecutorFileSystem paths to PathUri (#27424)
## Why We're moving exec-server to use PathUri for its internal path representations. ## What Move `ExecutorFileSystem` APIs to use `PathUri` instead of `AbsolutePathBuf`. Future changes will convert higher-level parts of exec-server.
Adam Perry @ OpenAI ·
2026-06-11 18:44:18 +00:00 -
[codex] Handle Ctrl-C for non-TTY unified exec (#26734)
## Why A long-running unified exec process started with `tty: false` could not be interrupted via `write_stdin`: ordinary non-TTY stdin writes are rejected once stdin is closed, but an exact U+0003 payload should still map to a process interrupt. The interrupt should flow through the same process lifecycle path as a real signal so Codex preserves process-reported output and exit metadata instead of fabricating a Ctrl-C exit code or tearing down the session early. ## What Changed - Add `process/signal` to exec-server with `ProcessSignal::Interrupt` and an empty response. - Add a non-consuming `ProcessHandle::signal` path for spawned processes; on Unix it sends SIGINT to the process group and leaves terminate/hard-kill unchanged. - Route non-TTY U+0003 `write_stdin` through `process.signal(...)` instead of `terminate`, then let the normal post-write collection path drain output and observe exit. - Add exec-server coverage where a shell `trap INT` handler prints the signal and exits with its own code. - Add unified exec coverage where a `tty: false` process traps SIGINT, emits output, and exits with its own code. ## Validation - `just test -p codex-exec-server exec_process_signal_interrupts_process` - `just test -p codex-exec-server` - `just test -p codex-core write_stdin_ctrl_c_interrupts_non_tty_session`
pakrym-oai ·
2026-06-09 15:10:17 -07:00 -
[codex] Add environment shell info (#26480)
## Why Shell detection needs to be available through the `Environment` abstraction so callers can ask the selected local or remote environment for shell metadata without adding a separate HTTP endpoint or parallel info-source path. This keeps shell metadata shaped like the existing environment-owned filesystem capability and lets remote environments answer through exec-server JSON-RPC. ## What changed - Added `environment/info` to the exec-server protocol/client/server and exposed `Environment::info()`. - Added local and remote environment info providers on `Environment`, following the existing capability-provider pattern used for filesystem access. - Moved the shared shell detection logic into `codex-shell-command` and kept core shell APIs as wrappers around that implementation. - Returned shell metadata as `EnvironmentInfo { shell: ShellInfo }` using the existing shell detection path. - Added a remote environment test that calls `Environment::info()` through an exec-server-backed environment. ## Validation - `git diff --check` - `just test -p codex-shell-command` - `just test -p codex-core -E 'test(/shell::tests::/)'`\n- `just test -p codex-exec-server environment`pakrym-oai ·
2026-06-04 22:36:25 -07:00 -
exec-server: canonicalize bound filesystem paths (#25149)
## Summary - add executor filesystem canonicalization as a bound-path operation - route remote canonicalization through the exec-server filesystem RPC surface - keep path normalization attached to the filesystem that owns the path ## Stack - 2/5 in the skills path authority stack extracted from https://github.com/openai/codex/pull/25098 - follows merged https://github.com/openai/codex/pull/25121 ## Validation - `cd /Users/starr/code/codex-worktrees/pr-25098-restack-review-pr1b/codex-rs && just fmt` - Not run: tests/checks (not requested) - GitHub CI pending on rewritten head
starr-openai ·
2026-06-01 11:53:31 -07:00 -
fix(exec-server): reject websocket requests with Origin headers (#24947)
## Why `codex exec-server` has a local WebSocket listener, but it did not apply the same browser-origin request handling as the `app-server` WebSocket transport. Requests that carry an `Origin` header should not be upgraded by this local transport, keeping both local WebSocket servers consistent and avoiding unexpected browser-initiated connections. ## What changed - Added an Axum middleware guard in `codex-rs/exec-server/src/server/transport.rs` that returns `403 Forbidden` for requests carrying an `Origin` header. - Added an integration test in `codex-rs/exec-server/tests/websocket.rs` that covers rejection of an `Origin`-bearing WebSocket handshake. - Kept ordinary WebSocket clients unchanged: existing no-`Origin` initialization and process behavior remains covered by the crate tests. ## Validation - `just test -p codex-exec-server` test phase (`186 passed`; run outside the parent macOS sandbox so nested sandbox tests can execute) - `just clippy -p codex-exec-server`
viyatb-oai ·
2026-05-28 14:44:14 -07:00 -
[exec-server] serve websocket listener via HTTP upgrade (#21963)
## Why `codex exec-server` should keep the existing public `ws://IP:PORT` URL shape while serving that websocket connection through an HTTP upgrade path internally. That keeps the client-facing configuration simple and allows the listener to work through intermediate HTTP-aware infrastructure. ## What changed - keep the emitted and configured exec-server URL as `ws://IP:PORT` - serve that websocket endpoint through Axum HTTP upgrade handling on `/` - expose `GET /readyz` from the same listener for readiness checks - route upgraded Axum websocket streams through the shared JSON-RPC connection machinery - initialize the rustls crypto provider before websocket client connections - preserve inbound binary websocket JSON-RPC parsing for compatibility with the prior transport behavior ## Verification - `cargo test -p codex-exec-server --test health --test process --test websocket --test initialize --test exec_process`
Ruslan Nigmatullin ·
2026-05-11 17:04:21 -07:00 -
Enable
--deny-warningsforcargo shear(#21616)## Summary In https://github.com/openai/codex/pull/21584, we disabled doctests for crates that lack any doctests. We can enforce that property via `cargo shear --deny-warnings`: crates that lack doctests will be flagged if doctests are enabled, and crates with doctests will be flagged if doctests are disabled. A few additional notes: - By adding `--deny-warnings`, `cargo shear` also flagged a number of modules that were not reachable at all. Some of those have been removed. - This PR removes a usage of `windows_modules!` (since `cargo shear` and `rustfmt` couldn't see through it) in favor of simple `#[cfg(target_os = "windows")]` macros. As a consequence, many of these files exhibit churn in this PR, since they weren't being formatted by `rustfmt` at all on main. - Again, to make the code more analyzable, this PR also removes some usages of `#[path = "cwd_junction.rs"]` in favor of a more standard module structure. The bin sidecar structure is still retained, but, e.g., `windows-sandbox-rs/src/bin/command_runner.rs` was moved to `windows-sandbox-rs/src/bin/command_runner/main.rs`, and so on. --------- Co-authored-by: Codex <noreply@openai.com>
Charlie Marsh ·
2026-05-08 20:29:00 +00:00 -
Add stdio exec-server client transport (#20664)
## Why Configured environments need to connect to exec-server instances that are not necessarily already listening on a websocket URL. A command-backed stdio transport lets Codex start an exec-server process, speak JSON-RPC over its stdio streams, and clean up that child process with the client lifetime. **Stack position:** this is PR 2 of 5. It builds on the server-side stdio listener from PR 1 and provides the client transport used by later environment/config PRs. ## What Changed - Add `ExecServerTransport` variants for websocket URLs and stdio shell commands. - Add stdio command connection support for `ExecServerClient`. - Move websocket/stdio transport setup into `client_transport.rs` so `client.rs` stays focused on shared JSON-RPC client, session, HTTP, and notification behavior. - Tie stdio child process cleanup to the JSON-RPC connection lifetime with a RAII lifetime guard. - Keep existing websocket environment behavior by adapting URL-backed remotes to `ExecServerTransport::WebSocketUrl`. ## Stack - 1. https://github.com/openai/codex/pull/20663 - Add stdio exec-server listener - **2. This PR:** https://github.com/openai/codex/pull/20664 - Add stdio exec-server client transport - 3. https://github.com/openai/codex/pull/20665 - Make environment providers own default selection - 4. https://github.com/openai/codex/pull/20666 - Add CODEX_HOME environments TOML provider - 5. https://github.com/openai/codex/pull/20667 - Load configured environments from CODEX_HOME Split from original draft: https://github.com/openai/codex/pull/20508 ## Validation Not run locally; this was split out of the original draft stack and then refactored to separate transport setup from the base client. --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-07 23:48:50 +00:00 -
Add stdio exec-server listener (#20663)
## Why This stack adds configured exec-server environments, including environments reached over stdio. Before client-side stdio transports or config can use that path, the exec-server binary itself needs a first-class stdio listen mode so it can speak the same JSON-RPC protocol over stdin/stdout that it already speaks over websockets. **Stack position:** this is PR 1 of 5. It is the server-side transport foundation for the stack. ## What Changed - Accept `stdio` and `stdio://` for `codex exec-server --listen`. - Promote the existing stdio `JsonRpcConnection` helper from test-only code into normal exec-server transport code. - Add parse coverage for stdio listen URLs while preserving the existing websocket default. ## Stack - **1. This PR:** https://github.com/openai/codex/pull/20663 - Add stdio exec-server listener - 2. https://github.com/openai/codex/pull/20664 - Add stdio exec-server client transport - 3. https://github.com/openai/codex/pull/20665 - Make environment providers own default selection - 4. https://github.com/openai/codex/pull/20666 - Add CODEX_HOME environments TOML provider - 5. https://github.com/openai/codex/pull/20667 - Load configured environments from CODEX_HOME Split from original draft: https://github.com/openai/codex/pull/20508 ## Validation Not run locally; this was split out of the original draft stack. --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-05-04 11:40:03 -07:00 -
fix(exec-server): retain output until streams close (#18946)
## Why A Mac Bazel run hit a flake in `server::handler::tests::output_and_exit_are_retained_after_notification_receiver_closes` where the read path observed process exit but lost the expected buffered stdout (`first\nsecond\n`). See the [GitHub Actions job](https://github.com/openai/codex/actions/runs/24758468552/job/72436716505) and [BuildBuddy invocation](https://app.buildbuddy.io/invocation/37475a12-4ef2-45fb-ab8a-e49a2aba1d59). The underlying race is that process exit is not the same thing as stdout/stderr closure. If a child or grandchild inherits the pipe write end, or a process duplicates it with `dup2`, the watched process can exit while the stream is still open and more output can still arrive. The exec-server was starting exited-process retention cleanup from the exit event, so the process entry could be removed before the output streams had actually closed. While stress-testing the exec-server unit suite, `server::handler::tests::long_poll_read_fails_after_session_resume` exposed a separate test race: it started a short-lived command that could exit and wake the pending long-poll read before the session-resume assertion observed the resumed-session error. That test is intended to cover resume eviction, not process-exit delivery, so this change keeps the process alive and quiet while the second connection resumes the session. ## What changed - Keep exec-server process entries retained until stdout/stderr streams close, then start the post-exit retention timer from the closed event. - Wake long-poll readers when the closed event is emitted. - Add focused `local_process` unit coverage that proves late output is still retained after the short test retention interval has elapsed, and that closed process entries are eventually evicted. - Add a local and remote regression test where a parent exits while a child keeps inherited stdout open. The child waits on an explicit release file, so the test deterministically observes exit first, releases the child, then requires a nonzero-wait read from the exit sequence to receive the late output. - In `codex-rs/exec-server/src/server/handler/tests.rs`, make `long_poll_read_fails_after_session_resume` run a long-lived silent command instead of a short command that prints and exits. This isolates the test to session-resume behavior and prevents a normal process exit from satisfying the pending long-poll read first. ## Testing - `cargo test -p codex-exec-server exec_process_retains_output_after_exit_until_streams_close` - `cargo test -p codex-exec-server local_process::tests` - `cargo test -p codex-exec-server` - `just fix -p codex-exec-server` - `bazel test //codex-rs/exec-server:exec-server-unit-tests //codex-rs/exec-server:exec-server-exec_process-test //codex-rs/exec-server:exec-server-file_system-test //codex-rs/exec-server:exec-server-http_client-test //codex-rs/exec-server:exec-server-initialize-test //codex-rs/exec-server:exec-server-process-test //codex-rs/exec-server:exec-server-websocket-test` - `bazel test --runs_per_test=25 //codex-rs/exec-server:exec-server-unit-tests` ## Documentation No docs update needed; this is an internal exec-server correctness fix.
Michael Bolin ·
2026-04-23 19:49:58 +00:00 -
[3/4] Add executor-backed RMCP HTTP client (#18583)
### Why The RMCP layer needs a Streamable HTTP client that can talk either directly over `reqwest` or through the executor HTTP runner without duplicating MCP session logic higher in the stack. This PR adds that client-side transport boundary so remote Streamable HTTP MCP can reuse the same RMCP flow as the local path. ### What - Add a shared `rmcp-client/src/streamable_http/` module with: - `transport_client.rs` for the local-or-remote transport enum - `local_client.rs` for the direct `reqwest` implementation - `remote_client.rs` for the executor-backed implementation - `common.rs` for the small shared Streamable HTTP helpers - Teach `RmcpClient` to build Streamable HTTP transports in either local or remote mode while keeping the existing OAuth ownership in RMCP. - Translate remote POST, GET, and DELETE session operations into executor `http/request` calls. - Preserve RMCP session expiry handling and reconnect behavior for the remote transport. - Add remote transport coverage in `rmcp-client/tests/streamable_http_remote.rs` and keep the shared test support in `rmcp-client/tests/streamable_http_test_support.rs`. ### Verification - `cargo check -p codex-rmcp-client` - online CI ### Stack 1. #18581 protocol 2. #18582 runner 3. #18583 RMCP client 4. #18584 manager wiring and local/remote coverage --------- Co-authored-by: Codex <noreply@openai.com>
Ahmed Ibrahim ·
2026-04-22 17:38:04 -07:00 -
exec-server: require explicit filesystem sandbox cwd (#19046)
## Why This is a cleanup PR for the `PermissionProfile` migration stack. #19016 fixed remote exec-server sandbox contexts so Docker-backed filesystem requests use a request/container `cwd` instead of leaking the local test runner `cwd`. That exposed the broader API problem: `FileSystemSandboxContext::new(SandboxPolicy)` could still reconstruct filesystem permissions by reading the exec-server process cwd with `AbsolutePathBuf::current_dir()`. That made `cwd`-dependent legacy entries, such as `:cwd`, `:project_roots`, and relative deny globs, depend on ambient process state instead of the request sandbox `cwd`. As later PRs make `PermissionProfile` the primary permissions abstraction, sandbox contexts should be explicit about whether they carry a request `cwd` or are profile-only. Removing the implicit constructor prevents new call sites from accidentally rebuilding permissions against the wrong `cwd`. ## What changed - Removed `FileSystemSandboxContext::new(SandboxPolicy)`. - Kept production callers on explicit constructors: `from_legacy_sandbox_policy(..., cwd)`, `from_permission_profile(...)`, and `from_permission_profile_with_cwd(...)`. - Updated exec-server test helpers to construct `PermissionProfile` values directly instead of routing through legacy `SandboxPolicy` projections. - Updated the environment regression test to use an explicit restricted profile with no synthetic `cwd`. ## Verification - `cargo test -p codex-exec-server` - `just fix -p codex-exec-server` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/19046). * #18288 * #18287 * #18286 * #18285 * #18284 * #18283 * #18282 * #18281 * #18280 * __->__ #19046
Michael Bolin ·
2026-04-22 23:05:12 +00:00 -
exec-server: expose arg0 alias root to fs sandbox (#19016)
## Why The post-merge `rust-ci-full` run for #18999 still failed the Ubuntu remote `suite::remote_env` sandboxed filesystem tests. That run checked out merge commit `ddde50c611e4800cb805f243ed3c50bbafe7d011`, so the arg0 guard lifetime fix was present. The Docker-backed failure had two remaining pieces: - The sandboxed filesystem helper needs to execute Codex through the `codex-linux-sandbox` arg0 alias path. The helper sandbox was only granting read access to the real Codex executable parent, so the alias parent also has to be visible inside the helper sandbox. - The remote-env tests were building sandbox contexts with `FileSystemSandboxContext::new()`, which captures the local test runner cwd. In the Docker remote exec-server, that host checkout path does not exist, so spawning the filesystem helper failed with `No such file or directory` before the helper could process the request. ## What Changed - Track all helper runtime read roots instead of a single root. - Add both the real Codex executable parent and the `codex-linux-sandbox` alias parent to sandbox readable roots. - Avoid sending an unused local cwd in remote filesystem sandbox contexts when the permission profile has no cwd-dependent entries. - Build the Docker remote-env test sandbox contexts with a cwd path that exists inside the container. - Add unit coverage for the alias-parent root and remote sandbox cwd handling. ## Verification - `cargo test -p codex-exec-server` - `cargo test -p codex-core remote_test_env_sandboxed_read_allows_readable_root` - `just fix -p codex-exec-server` - `just fix -p codex-core`
Michael Bolin ·
2026-04-22 21:34:22 +00:00 -
[2/4] Implement executor HTTP request runner (#18582)
### Why Remote streamable HTTP MCP needs the executor to perform ordinary HTTP requests on the executor side. This keeps network placement aligned with `experimental_environment = "remote"` without adding MCP-specific executor APIs. ### What - Add an executor-side `http/request` runner backed by `reqwest`. - Validate request method and URL scheme, preserving the transport boundary at plain HTTP. - Return buffered responses for ordinary calls and emit ordered `http/request/bodyDelta` notifications for streaming responses. - Register the request handler in the exec-server router. - Document the runner entrypoint, conversion helpers, body-stream bridge, notification sender, timeout behavior, and new integration-test helpers. - Add exec-server integration tests with the existing websocket harness and a local TCP HTTP peer for buffered and streamed responses, with comments spelling out what each test proves and its setup/exercise/assert phases. ### Stack 1. #18581 protocol 2. #18582 runner 3. #18583 RMCP client 4. #18584 manager wiring and local/remote coverage ### Verification - `just fmt` - `cargo check -p codex-exec-server -p codex-rmcp-client --tests` - `cargo check -p codex-core --test all` compile-only - `git diff --check` - Online full CI is running from the `full-ci` branch, including the remote Rust test job. Co-authored-by: Codex <noreply@openai.com> --------- Co-authored-by: Codex <noreply@openai.com>
Ahmed Ibrahim ·
2026-04-22 20:36:34 +00:00 -
fix: fully revert agent identity runtime wiring (#18757)
## Summary This PR fully reverts the previously merged Agent Identity runtime integration from the old stack: https://github.com/openai/codex/pull/17387/changes It removes the Codex-side task lifecycle wiring, rollout/session persistence, feature flag plumbing, lazy `auth.json` mutation, background task auth paths, and request callsite changes introduced by that stack. This leaves the repo in a clean pre-AgentIdentity integration state so the follow-up PRs can reintroduce the pieces in smaller reviewable layers. ## Stack 1. This PR: full revert 2. https://github.com/openai/codex/pull/18871: move Agent Identity business logic into a crate 3. https://github.com/openai/codex/pull/18785: add explicit AgentIdentity auth mode and startup task allocation 4. https://github.com/openai/codex/pull/18811: migrate auth callsites through AuthProvider ## Testing Tests: targeted Rust checks, cargo-shear, Bazel lock check, and CI.
efrazer-oai ·
2026-04-21 14:30:55 -07:00 -
[2/8] Support piped stdin in exec process API (#18086)
## Summary - Add an explicit stdin mode to process/start. - Keep normal non-interactive exec stdin closed while allowing pipe-backed processes. ## Stack ```text o #18027 [8/8] Fail exec client operations after disconnect │ o #18025 [7/8] Cover MCP stdio tests with executor placement │ o #18089 [6/8] Wire remote MCP stdio through executor │ o #18088 [5/8] Add executor process transport for MCP stdio │ o #18087 [4/8] Abstract MCP stdio server launching │ o #18020 [3/8] Add pushed exec process events │ @ #18086 [2/8] Support piped stdin in exec process API │ o #18085 [1/8] Add MCP server environment config │ o main ``` Co-authored-by: Codex <noreply@openai.com>
Ahmed Ibrahim ·
2026-04-16 10:30:10 -07:00 -
[codex] Add symlink flag to fs metadata (#17719)
Add `is_symlink` to FsMetadata struct.
pakrym-oai ·
2026-04-13 17:46:56 -07:00 -
Build remote exec env from exec-server policy (#17216)
## Summary - add an exec-server `envPolicy` field; when present, the server starts from its own process env and applies the shell environment policy there - keep `env` as the exact environment for local/embedded starts, but make it an overlay for remote unified-exec starts - move the shell-environment-policy builder into `codex-config` so Core and exec-server share the inherit/filter/set/include behavior - overlay only runtime/sandbox/network deltas from Core onto the exec-server-derived env ## Why Remote unified exec was materializing the shell env inside Core and forwarding the whole map to exec-server, so remote processes could inherit the orchestrator machine's `HOME`, `PATH`, etc. This keeps the base env on the executor while preserving Core-owned runtime additions like `CODEX_THREAD_ID`, unified-exec defaults, network proxy env, and sandbox marker env. ## Validation - `just fmt` - `git diff --check` - `cargo test -p codex-exec-server --lib` - `cargo test -p codex-core --lib unified_exec::process_manager::tests` - `cargo test -p codex-core --lib exec_env::tests` - `cargo test -p codex-core --lib exec_env_tests` (compile-only; filter matched 0 tests) - `cargo test -p codex-config --lib shell_environment` (compile-only; filter matched 0 tests) - `just bazel-lock-update` ## Known local validation issue - `just bazel-lock-check` is not runnable in this checkout: it invokes `./scripts/check-module-bazel-lock.sh`, which is missing. --------- Co-authored-by: Codex <noreply@openai.com> Co-authored-by: pakrym-oai <pakrym@openai.com>
jif-oai ·
2026-04-13 09:59:08 +01:00 -
Run exec-server fs operations through sandbox helper (#17294)
## Summary - run exec-server filesystem RPCs requiring sandboxing through a `codex-fs` arg0 helper over stdin/stdout - keep direct local filesystem execution for `DangerFullAccess` and external sandbox policies - remove the standalone exec-server binary path in favor of top-level arg0 dispatch/runtime paths - add sandbox escape regression coverage for local and remote filesystem paths ## Validation - `just fmt` - `git diff --check` - remote devbox: `cd codex-rs && bazel test --bes_backend= --bes_results_url= //codex-rs/exec-server:all` (6/6 passed) --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-04-12 18:36:03 -07:00 -
Adrian ·
2026-04-11 09:52:06 -07:00 -
Fix Windows exec-server output test flake (#17409)
Problem: The Windows exec-server test command could let separator whitespace become part of `echo` output, making the exact retained-output assertion flaky. Solution: Tighten the Windows `cmd.exe` command by placing command separators directly after the echoed tokens so stdout remains deterministic while preserving the exact assertion.
Eric Traut ·
2026-04-10 19:24:40 -07:00 -
feat: move exec-server ownership (#16344)
This introduces session-scoped ownership for exec-server so ws disconnects no longer immediately kill running remote exec processes, and it prepares the protocol for reconnect-based resume. - add session_id / resume_session_id to the exec-server initialize handshake - move process ownership under a shared session registry - detach sessions on websocket disconnect and expire them after a TTL instead of killing processes immediately (we will resume based on this) - allow a new connection to resume an existing session and take over notifications/ownership - I use UUID to make them not predictable as we don't have auth for now - make detached-session expiry authoritative at resume time so teardown wins at the TTL boundary - reject long-poll process/read calls that get resumed out from under an older attachment --------- Co-authored-by: Codex <noreply@openai.com>
jif-oai ·
2026-04-10 14:11:47 +01:00 -
Add sandbox support to filesystem APIs (#16751)
## Summary - add optional `sandboxPolicy` support to the app-server filesystem request surface - thread sandbox-aware filesystem options through app-server and exec-server adapters - enforce sandboxed read/write access in the filesystem abstraction with focused local and remote coverage ## Validation - `cargo test -p codex-app-server-protocol` - `cargo test -p codex-exec-server file_system` - `cargo test -p codex-app-server suite::v2::fs` --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-04-08 12:10:48 -07:00 -
Make AGENTS.md discovery FS-aware (#15826)
## Summary - make AGENTS.md discovery and loading fully FS-aware and remove the non-FS discover helper - migrate remote-aware codex-core tests to use TestEnv workspace setup instead of syncing a local workspace copy - add AGENTS.md corner-case coverage, including directory fallbacks and remote-aware integration coverage ## Testing - cargo test -p codex-core project_doc -- --nocapture - cargo test -p codex-core hierarchical_agents -- --nocapture - cargo test -p codex-core agents_md -- --nocapture - cargo test -p codex-tui status -- --nocapture - cargo test -p codex-tui-app-server status -- --nocapture - just fix - just fmt - just bazel-lock-update - just bazel-lock-check - just argument-comment-lint - remote Linux executor tests in progress via scripts/test-remote-env.sh
pakrym-oai ·
2026-04-06 20:26:21 -07:00 -
extract models manager and related ownership from core (#16508)
## Summary - split `models-manager` out of `core` and add `ModelsManagerConfig` plus `Config::to_models_manager_config()` so model metadata paths stop depending on `core::Config` - move login-owned/auth-owned code out of `core` into `codex-login`, move model provider config into `codex-model-provider-info`, move API bridge mapping into `codex-api`, move protocol-owned types/impls into `codex-protocol`, and move response debug helpers into a dedicated `response-debug-context` crate - move feedback tag emission into `codex-feedback`, relocate tests to the crates that now own the code, and keep broad temporary re-exports so this PR avoids a giant import-only rewrite ## Major moves and decisions - created `codex-models-manager` as the owner for model cache/catalog/config/model info logic, including the new `ModelsManagerConfig` struct - created `codex-model-provider-info` as the owner for provider config parsing/defaults and kept temporary `codex-login`/`codex-core` re-exports for old import paths - moved `api_bridge` error mapping + `CoreAuthProvider` into `codex-api`, while `codex-login::api_bridge` temporarily re-exports those symbols and keeps the `auth_provider_from_auth` wrapper - moved `auth_env_telemetry` and `provider_auth` ownership to `codex-login` - moved `CodexErr` ownership to `codex-protocol::error`, plus `StreamOutput`, `bytes_to_string_smart`, and network policy helpers to protocol-owned modules - created `codex-response-debug-context` for `extract_response_debug_context`, `telemetry_transport_error_message`, and related response-debug plumbing instead of leaving that behavior in `core` - moved `FeedbackRequestTags`, `emit_feedback_request_tags`, and `emit_feedback_request_tags_with_auth_env` to `codex-feedback` - deferred removal of temporary re-exports and the mechanical import rewrites to a stacked follow-up PR so this PR stays reviewable ## Test moves - moved auth refresh coverage from `core/tests/suite/auth_refresh.rs` to `login/tests/suite/auth_refresh.rs` - moved text encoding coverage from `core/tests/suite/text_encoding_fix.rs` to `protocol/src/exec_output_tests.rs` - moved model info override coverage from `core/tests/suite/model_info_overrides.rs` to `models-manager/src/model_info_overrides_tests.rs` --------- Co-authored-by: Codex <noreply@openai.com>
Ahmed Ibrahim ·
2026-04-02 23:00:02 -07:00 -
feat: use
ProcessIdinexec-server(#15866)Use a full struct for the ProcessId to increase readability and make it easier in the future to make it evolve if needed
jif-oai ·
2026-03-26 16:45:36 +01:00 -
Split exec process into local and remote implementations (#15233)
## Summary - match the exec-process structure to filesystem PR #15232 - expose `ExecProcess` on `Environment` - make `LocalProcess` the real implementation and `RemoteProcess` a thin network proxy over `ExecServerClient` - make `ProcessHandler` a thin RPC adapter delegating to `LocalProcess` - add a shared local/remote process test ## Validation - `just fmt` - `CARGO_TARGET_DIR=~/.cache/cargo-target/codex cargo test -p codex-exec-server` - `just fix -p codex-exec-server` --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-03-20 03:13:08 +00:00 -
Refactor ExecServer filesystem split between local and remote (#15232)
For each feature we have: 1. Trait exposed on environment 2. **Local Implementation** of the trait 3. Remote implementation that uses the client to proxy via network 4. Handler implementation that handles PRC requests and calls into **Local Implementation**
pakrym-oai ·
2026-03-19 17:08:04 -07:00 -
Move terminal module to terminal-detection crate (#15216)
- Move core/src/terminal.rs and its tests into a standalone terminal-detection workspace crate. - Update direct consumers to depend on codex-terminal-detection and import terminal APIs directly. --------- Co-authored-by: Codex <noreply@openai.com>
Ahmed Ibrahim ·
2026-03-19 14:08:04 -07:00 -
Add exec-server exec RPC implementation (#15090)
Stacked PR 2/3, based on the stub PR. Adds the exec RPC implementation and process/event flow in exec-server only. --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-03-19 19:00:36 +00:00 -
Remove stdio transport from exec server (#15119)
Summary - delete the deprecated stdio transport plumbing from the exec server stack - add a basic `exec_server()` harness plus test utilities to start a server, send requests, and await events - refresh exec-server dependencies, configs, and documentation to reflect the new flow Testing - Not run (not requested) --------- Co-authored-by: starr-openai <starr@openai.com> Co-authored-by: Codex <noreply@openai.com>
pakrym-oai ·
2026-03-19 01:00:35 +00:00 -
Add exec-server stub server and protocol docs (#15089)
Stacked PR 1/3. This is the initialize-only exec-server stub slice: binary/client scaffolding and protocol docs, without exec/filesystem implementation. --------- Co-authored-by: Codex <noreply@openai.com>
starr-openai ·
2026-03-19 00:30:05 +00:00