Commit Graph

39 Commits

  • Resume exec-server sessions after disconnect (#28512)
    Supersedes #28288 (closed).
    
    ## Why
    
    A short WebSocket interruption currently ends every client-side process
    handle, even though exec-server keeps the server session and its
    processes alive for a short time.
    
    This is especially visible for executor-backed stdio MCP servers: a
    temporary connection loss becomes a permanent `Transport closed` error.
    The server already has the information needed to resume the session, but
    the client opens a fresh session instead of using it.
    
    This change reconnects below the process and MCP layers. Existing
    process handles stay valid, missed output is recovered, and the same
    server-side processes continue running.
    
    ## State machine
    
    One logical `ExecServerClient` stays alive while its underlying RPC
    connection changes generations.
    
    ```text
                             transport closes
           +------------------------------------------------+
           |                                                v
    +-------------+                                  +-------------+
    |  Connected  |                                  | Recovering  |
    +-------------+                                  +-------------+
           ^                                                |
           | session resumed, processes caught up           | retryable error
           +------------------------------------------------+ loops until deadline
                                                            |
                                                            | deadline or permanent error
                                                            v
                                                      +-------------+
                                                      |   Failed    |
                                                      +-------------+
    ```
    
    ### `Connected`
    
    - New RPC calls use the current connection.
    - Process notifications are published in sequence order.
    - A disconnect only starts recovery if it came from the current
    connection generation. Late events from older generations cannot replace
    the active connection.
    
    ### `Recovering`
    
    - New calls wait instead of choosing a half-connected RPC client.
    - Existing process handles, wake subscriptions, and event subscriptions
    stay open.
    - Streaming HTTP response bodies fail immediately because their byte
    streams cannot be resumed safely.
    - Recovery first waits for process starts that were already in flight. A
    start whose result became ambiguous is cleaned up after reconnection
    instead of being silently adopted.
    - The client reconnects with the learned `session_id`. The server may
    briefly report that the old connection is still attached, so that error
    is retried until the detach finishes.
    - The notification consumer starts before the resume handshake
    completes. This prevents a busy process from filling the notification
    queue and blocking the initialize response.
    - Before installing the new connection, the client catches up every
    recoverable process with `process/read`.
    
    ### `Failed`
    
    - Recovery stops after 25 seconds or after a permanent error.
    - Waiting calls are released with one stable disconnect error.
    - Existing process sessions receive a terminal failure instead of
    waiting forever.
    
    ## Recovering process events
    
    Output, exit, and close events share one sequence. During normal
    operation, the client buffers early events until every lower sequence
    has been published.
    
    After reconnection, the client reads each process starting after its
    last published sequence:
    
    1. Retained output chunks are inserted by sequence number.
    2. Exit and close state are reconstructed in their sequence positions.
    3. Events already received as live notifications are ignored as
    duplicates.
    4. Newly contiguous events are published in order.
    5. If the server no longer retains enough output to fill a sequence gap,
    only that process is terminated and failed. The recovered connection
    remains usable for other processes.
    
    The server reports its full next event sequence for unbounded reads,
    including exit and close events. Closed processes remain readable for
    the same 30-second window used to retain detached sessions.
    
    ## Other details
    
    - Detached server sessions are retained for 30 seconds, leaving margin
    around the client's 25-second recovery deadline.
    - Session attach and detach update the active notification sender under
    the same attachment lock, so an old connection cannot clear a newly
    attached sender.
    - A dedicated error code distinguishes the temporary "session is still
    attached" race from permanent initialization errors.
    - Process starts are identity-checked on both client and server. Cleanup
    from an older start cannot remove a newer process that reused the same
    ID.
    - Mutating requests that were already in flight when the transport
    closed are not replayed, because the client cannot know whether the
    server applied them. Requests started after recovery is known wait for
    the replacement connection.
    - We assume the server/client version stays in sync (on the before/after
    this PR)
    
    ## User impact
    
    Long-running commands and stdio MCP servers can survive a temporary
    exec-server WebSocket interruption without changing process IDs or
    losing output produced during the outage.
  • [codex] exec-server: stream files in chunks (#28354)
    ## Why
    
    `fs/readFile` buffers the entire file in one response, which makes large
    remote reads expensive and prevents callers from applying backpressure.
    We need an opt-in streaming path with bounded block sizes while
    preserving the existing single-call API for small and sandboxed reads.
    
    ## What changed
    
    - Add `ExecServerClient::stream`, returning a named `FileReadStream`
    that implements `futures::Stream` and yields immutable 1 MiB byte
    blocks.
    - Add internal `fs/open`, `fs/readBlock`, and `fs/close` RPCs.
    `fs/readBlock` accepts an explicit offset and length.
    - Keep unsandboxed files open between block reads, cap open handles per
    connection, and clean them up on EOF, error, stream drop, explicit
    close, or connection shutdown.
    - Reject platform-sandboxed streaming opens instead of turning the
    one-shot sandbox helper into a persistent server. Existing `fs/readFile`
    behavior is unchanged.
    
    ## Testing
    
    - `just test -p codex-exec-server`
    - Integration coverage for 1 MiB chunking, exact block-boundary EOF,
    sandbox rejection, and continued reads from the opened file after path
    replacement.
    - Handle-manager coverage for non-sequential offsets, variable block
    lengths, the 128-handle limit, and capacity release after close.
  • [codex] Carry exec-server cwd as PathUri (#28032)
    ## Why
    
    This is the second-to-last place in the exec-server protocol that needs
    to migrate to URIs to support cross-OS operation.
    
    ## What
    
    - Change `ExecParams.cwd` to `PathUri`.
    - Keep the cwd URI-shaped through core and rmcp producers, converting it
    to `AbsolutePathBuf` only in `LocalProcess::start_process`.
    - Reject non-native cwd URIs before launch and update the affected
    protocol documentation and call sites.
  • [codex] Add size to internal filesystem metadata (#27927)
    ## Why
    
    `ExecutorFileSystem::get_metadata` reports file kind and timestamps but
    not size. Internal callers that need to enforce a size limit therefore
    have to read the complete file first, which is especially wasteful for
    remote filesystems.
    
    This adds the missing internal metadata so consumers can reject
    oversized files before transferring or buffering them. The field is
    named `size`, matching VS Code's `FileStat.size` filesystem convention.
    
    ## What changed
    
    - add `size: u64` to internal `FileMetadata`
    - populate it from the underlying filesystem metadata
    - carry it through sandbox-helper and remote exec-server responses
    - cover files, directories, symlink targets, and sandboxed reads across
    local and remote filesystem implementations
    
    The new field is intentionally not exposed through the app-server API.
    
    ## Testing
    
    - `just test -p codex-exec-server get_metadata`
    - `just test -p codex-exec-server
    file_system_sandboxed_metadata_and_read_allow_readable_root`
    - `just test -p codex-core-plugins`
    - `just test -p codex-skills-extension`
  • sandboxing: migrate cwd inputs to PathUri (#27816)
    ## Why
    
    Sandbox cwd values can cross app-server and exec-server host boundaries.
    They should retain URI semantics until the receiving host validates them
    instead of being interpreted early as native paths.
    
    ## What
    
    - Carry `PathUri` through filesystem sandbox contexts, sandbox commands,
    and transform inputs.
    - Convert command and policy cwd once in `SandboxManager::transform`,
    then keep launch requests native.
    - Preserve sandbox cwd over remote filesystem transport and reject
    non-native URIs without fallback.
    - Cache paired native/URI turn-environment cwd values during migration,
    with immutable access to keep them synchronized.
    - Extend existing protocol, forwarding, transform, and core runtime
    tests.
  • Remove fs/join and fs/parent from exec-server protocol (#27700)
    ## Summary
    
    Path composition is already handled by `PathUri`, leaving `fs/join` and
    `fs/parent` as redundant exec-server protocol surface. Because
    app-server and exec-server are deployed atomically, these obsolete
    methods can be removed without a compatibility shim.
    
    This removes the protocol constants and payloads, public client APIs,
    server registrations and handlers, and endpoint-only tests. Existing
    in-process `PathUri` join/parent coverage remains.
    
    ## Validation
    
    - `just test -p codex-exec-server` (215 passed, 2 skipped)
  • [codex] migrate exec-server filesystem protocol to PathUri (#27653)
    Exec-server filesystem calls should preserve cross-platform `file:` URIs
    across the remote boundary instead of converting them through paths
    native to the client host.
    
    This changes the exec-server filesystem protocol DTOs to use `PathUri`,
    carries those values directly through remote and sandbox-helper
    transports, and keeps legacy native absolute-path request strings
    readable for compatibility. It also updates protocol documentation and
    coverage for URI serialization and non-native URI forwarding.
  • [codex] migrate ExecutorFileSystem paths to PathUri (#27424)
    ## Why
    
    We're moving exec-server to use PathUri for its internal path
    representations.
    
    ## What
    
    Move `ExecutorFileSystem` APIs to use `PathUri` instead of
    `AbsolutePathBuf`. Future changes will convert higher-level parts of
    exec-server.
  • [codex] Handle Ctrl-C for non-TTY unified exec (#26734)
    ## Why
    
    A long-running unified exec process started with `tty: false` could not
    be interrupted via `write_stdin`: ordinary non-TTY stdin writes are
    rejected once stdin is closed, but an exact U+0003 payload should still
    map to a process interrupt. The interrupt should flow through the same
    process lifecycle path as a real signal so Codex preserves
    process-reported output and exit metadata instead of fabricating a
    Ctrl-C exit code or tearing down the session early.
    
    ## What Changed
    
    - Add `process/signal` to exec-server with `ProcessSignal::Interrupt`
    and an empty response.
    - Add a non-consuming `ProcessHandle::signal` path for spawned
    processes; on Unix it sends SIGINT to the process group and leaves
    terminate/hard-kill unchanged.
    - Route non-TTY U+0003 `write_stdin` through `process.signal(...)`
    instead of `terminate`, then let the normal post-write collection path
    drain output and observe exit.
    - Add exec-server coverage where a shell `trap INT` handler prints the
    signal and exits with its own code.
    - Add unified exec coverage where a `tty: false` process traps SIGINT,
    emits output, and exits with its own code.
    
    ## Validation
    
    - `just test -p codex-exec-server
    exec_process_signal_interrupts_process`
    - `just test -p codex-exec-server`
    - `just test -p codex-core
    write_stdin_ctrl_c_interrupts_non_tty_session`
  • [codex] Add environment shell info (#26480)
    ## Why
    
    Shell detection needs to be available through the `Environment`
    abstraction so callers can ask the selected local or remote environment
    for shell metadata without adding a separate HTTP endpoint or parallel
    info-source path. This keeps shell metadata shaped like the existing
    environment-owned filesystem capability and lets remote environments
    answer through exec-server JSON-RPC.
    
    ## What changed
    
    - Added `environment/info` to the exec-server protocol/client/server and
    exposed `Environment::info()`.
    - Added local and remote environment info providers on `Environment`,
    following the existing capability-provider pattern used for filesystem
    access.
    - Moved the shared shell detection logic into `codex-shell-command` and
    kept core shell APIs as wrappers around that implementation.
    - Returned shell metadata as `EnvironmentInfo { shell: ShellInfo }`
    using the existing shell detection path.
    - Added a remote environment test that calls `Environment::info()`
    through an exec-server-backed environment.
    
    ## Validation
    
    - `git diff --check`
    - `just test -p codex-shell-command`
    - `just test -p codex-core -E 'test(/shell::tests::/)'`\n- `just test -p
    codex-exec-server environment`
  • exec-server: canonicalize bound filesystem paths (#25149)
    ## Summary
    - add executor filesystem canonicalization as a bound-path operation
    - route remote canonicalization through the exec-server filesystem RPC
    surface
    - keep path normalization attached to the filesystem that owns the path
    
    ## Stack
    - 2/5 in the skills path authority stack extracted from
    https://github.com/openai/codex/pull/25098
    - follows merged https://github.com/openai/codex/pull/25121
    
    ## Validation
    - `cd
    /Users/starr/code/codex-worktrees/pr-25098-restack-review-pr1b/codex-rs
    && just fmt`
    - Not run: tests/checks (not requested)
    - GitHub CI pending on rewritten head
  • fix(exec-server): reject websocket requests with Origin headers (#24947)
    ## Why
    
    `codex exec-server` has a local WebSocket listener, but it did not apply
    the same browser-origin request handling as the `app-server` WebSocket
    transport. Requests that carry an `Origin` header should not be upgraded
    by this local transport, keeping both local WebSocket servers consistent
    and avoiding unexpected browser-initiated connections.
    
    ## What changed
    
    - Added an Axum middleware guard in
    `codex-rs/exec-server/src/server/transport.rs` that returns `403
    Forbidden` for requests carrying an `Origin` header.
    - Added an integration test in `codex-rs/exec-server/tests/websocket.rs`
    that covers rejection of an `Origin`-bearing WebSocket handshake.
    - Kept ordinary WebSocket clients unchanged: existing no-`Origin`
    initialization and process behavior remains covered by the crate tests.
    
    ## Validation
    
    - `just test -p codex-exec-server` test phase (`186 passed`; run outside
    the parent macOS sandbox so nested sandbox tests can execute)
    - `just clippy -p codex-exec-server`
  • [exec-server] serve websocket listener via HTTP upgrade (#21963)
    ## Why
    
    `codex exec-server` should keep the existing public `ws://IP:PORT` URL
    shape while serving that websocket connection through an HTTP upgrade
    path internally. That keeps the client-facing configuration simple and
    allows the listener to work through intermediate HTTP-aware
    infrastructure.
    
    ## What changed
    
    - keep the emitted and configured exec-server URL as `ws://IP:PORT`
    - serve that websocket endpoint through Axum HTTP upgrade handling on
    `/`
    - expose `GET /readyz` from the same listener for readiness checks
    - route upgraded Axum websocket streams through the shared JSON-RPC
    connection machinery
    - initialize the rustls crypto provider before websocket client
    connections
    - preserve inbound binary websocket JSON-RPC parsing for compatibility
    with the prior transport behavior
    
    ## Verification
    
    - `cargo test -p codex-exec-server --test health --test process --test
    websocket --test initialize --test exec_process`
  • Enable --deny-warnings for cargo shear (#21616)
    ## Summary
    
    In https://github.com/openai/codex/pull/21584, we disabled doctests for
    crates that lack any doctests. We can enforce that property via `cargo
    shear --deny-warnings`: crates that lack doctests will be flagged if
    doctests are enabled, and crates with doctests will be flagged if
    doctests are disabled.
    
    A few additional notes:
    
    - By adding `--deny-warnings`, `cargo shear` also flagged a number of
    modules that were not reachable at all. Some of those have been removed.
    - This PR removes a usage of `windows_modules!` (since `cargo shear` and
    `rustfmt` couldn't see through it) in favor of simple `#[cfg(target_os =
    "windows")]` macros. As a consequence, many of these files exhibit churn
    in this PR, since they weren't being formatted by `rustfmt` at all on
    main.
    - Again, to make the code more analyzable, this PR also removes some
    usages of `#[path = "cwd_junction.rs"]` in favor of a more standard
    module structure. The bin sidecar structure is still retained, but,
    e.g., `windows-sandbox-rs/src/bin/command_runner.rs‎` was moved to
    `windows-sandbox-rs/src/bin/command_runner/main.rs`, and so on.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add stdio exec-server client transport (#20664)
    ## Why
    
    Configured environments need to connect to exec-server instances that
    are not necessarily already listening on a websocket URL. A
    command-backed stdio transport lets Codex start an exec-server process,
    speak JSON-RPC over its stdio streams, and clean up that child process
    with the client lifetime.
    
    **Stack position:** this is PR 2 of 5. It builds on the server-side
    stdio listener from PR 1 and provides the client transport used by later
    environment/config PRs.
    
    ## What Changed
    
    - Add `ExecServerTransport` variants for websocket URLs and stdio shell
    commands.
    - Add stdio command connection support for `ExecServerClient`.
    - Move websocket/stdio transport setup into `client_transport.rs` so
    `client.rs` stays focused on shared JSON-RPC client, session, HTTP, and
    notification behavior.
    - Tie stdio child process cleanup to the JSON-RPC connection lifetime
    with a RAII lifetime guard.
    - Keep existing websocket environment behavior by adapting URL-backed
    remotes to `ExecServerTransport::WebSocketUrl`.
    
    ## Stack
    
    - 1. https://github.com/openai/codex/pull/20663 - Add stdio exec-server
    listener
    - **2. This PR:** https://github.com/openai/codex/pull/20664 - Add stdio
    exec-server client transport
    - 3. https://github.com/openai/codex/pull/20665 - Make environment
    providers own default selection
    - 4. https://github.com/openai/codex/pull/20666 - Add CODEX_HOME
    environments TOML provider
    - 5. https://github.com/openai/codex/pull/20667 - Load configured
    environments from CODEX_HOME
    
    Split from original draft: https://github.com/openai/codex/pull/20508
    
    ## Validation
    
    Not run locally; this was split out of the original draft stack and then
    refactored to separate transport setup from the base client.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add stdio exec-server listener (#20663)
    ## Why
    
    This stack adds configured exec-server environments, including
    environments reached over stdio. Before client-side stdio transports or
    config can use that path, the exec-server binary itself needs a
    first-class stdio listen mode so it can speak the same JSON-RPC protocol
    over stdin/stdout that it already speaks over websockets.
    
    **Stack position:** this is PR 1 of 5. It is the server-side transport
    foundation for the stack.
    
    ## What Changed
    
    - Accept `stdio` and `stdio://` for `codex exec-server --listen`.
    - Promote the existing stdio `JsonRpcConnection` helper from test-only
    code into normal exec-server transport code.
    - Add parse coverage for stdio listen URLs while preserving the existing
    websocket default.
    
    ## Stack
    
    - **1. This PR:** https://github.com/openai/codex/pull/20663 - Add stdio
    exec-server listener
    - 2. https://github.com/openai/codex/pull/20664 - Add stdio exec-server
    client transport
    - 3. https://github.com/openai/codex/pull/20665 - Make environment
    providers own default selection
    - 4. https://github.com/openai/codex/pull/20666 - Add CODEX_HOME
    environments TOML provider
    - 5. https://github.com/openai/codex/pull/20667 - Load configured
    environments from CODEX_HOME
    
    Split from original draft: https://github.com/openai/codex/pull/20508
    
    ## Validation
    
    Not run locally; this was split out of the original draft stack.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • fix(exec-server): retain output until streams close (#18946)
    ## Why
    
    A Mac Bazel run hit a flake in
    `server::handler::tests::output_and_exit_are_retained_after_notification_receiver_closes`
    where the read path observed process exit but lost the expected buffered
    stdout (`first\nsecond\n`). See the [GitHub Actions
    job](https://github.com/openai/codex/actions/runs/24758468552/job/72436716505)
    and [BuildBuddy
    invocation](https://app.buildbuddy.io/invocation/37475a12-4ef2-45fb-ab8a-e49a2aba1d59).
    
    The underlying race is that process exit is not the same thing as
    stdout/stderr closure. If a child or grandchild inherits the pipe write
    end, or a process duplicates it with `dup2`, the watched process can
    exit while the stream is still open and more output can still arrive.
    The exec-server was starting exited-process retention cleanup from the
    exit event, so the process entry could be removed before the output
    streams had actually closed.
    
    While stress-testing the exec-server unit suite,
    `server::handler::tests::long_poll_read_fails_after_session_resume`
    exposed a separate test race: it started a short-lived command that
    could exit and wake the pending long-poll read before the session-resume
    assertion observed the resumed-session error. That test is intended to
    cover resume eviction, not process-exit delivery, so this change keeps
    the process alive and quiet while the second connection resumes the
    session.
    
    ## What changed
    
    - Keep exec-server process entries retained until stdout/stderr streams
    close, then start the post-exit retention timer from the closed event.
    - Wake long-poll readers when the closed event is emitted.
    - Add focused `local_process` unit coverage that proves late output is
    still retained after the short test retention interval has elapsed, and
    that closed process entries are eventually evicted.
    - Add a local and remote regression test where a parent exits while a
    child keeps inherited stdout open. The child waits on an explicit
    release file, so the test deterministically observes exit first,
    releases the child, then requires a nonzero-wait read from the exit
    sequence to receive the late output.
    - In `codex-rs/exec-server/src/server/handler/tests.rs`, make
    `long_poll_read_fails_after_session_resume` run a long-lived silent
    command instead of a short command that prints and exits. This isolates
    the test to session-resume behavior and prevents a normal process exit
    from satisfying the pending long-poll read first.
    
    ## Testing
    
    - `cargo test -p codex-exec-server
    exec_process_retains_output_after_exit_until_streams_close`
    - `cargo test -p codex-exec-server local_process::tests`
    - `cargo test -p codex-exec-server`
    - `just fix -p codex-exec-server`
    - `bazel test //codex-rs/exec-server:exec-server-unit-tests
    //codex-rs/exec-server:exec-server-exec_process-test
    //codex-rs/exec-server:exec-server-file_system-test
    //codex-rs/exec-server:exec-server-http_client-test
    //codex-rs/exec-server:exec-server-initialize-test
    //codex-rs/exec-server:exec-server-process-test
    //codex-rs/exec-server:exec-server-websocket-test`
    - `bazel test --runs_per_test=25
    //codex-rs/exec-server:exec-server-unit-tests`
    
    ## Documentation
    
    No docs update needed; this is an internal exec-server correctness fix.
  • [3/4] Add executor-backed RMCP HTTP client (#18583)
    ### Why
    The RMCP layer needs a Streamable HTTP client that can talk either
    directly over `reqwest` or through the executor HTTP runner without
    duplicating MCP session logic higher in the stack. This PR adds that
    client-side transport boundary so remote Streamable HTTP MCP can reuse
    the same RMCP flow as the local path.
    
    ### What
    - Add a shared `rmcp-client/src/streamable_http/` module with:
      - `transport_client.rs` for the local-or-remote transport enum
      - `local_client.rs` for the direct `reqwest` implementation
      - `remote_client.rs` for the executor-backed implementation
      - `common.rs` for the small shared Streamable HTTP helpers
    - Teach `RmcpClient` to build Streamable HTTP transports in either local
    or remote mode while keeping the existing OAuth ownership in RMCP.
    - Translate remote POST, GET, and DELETE session operations into
    executor `http/request` calls.
    - Preserve RMCP session expiry handling and reconnect behavior for the
    remote transport.
    - Add remote transport coverage in
    `rmcp-client/tests/streamable_http_remote.rs` and keep the shared test
    support in `rmcp-client/tests/streamable_http_test_support.rs`.
    
    ### Verification
    - `cargo check -p codex-rmcp-client`
    - online CI
    
    ### Stack
    1. #18581 protocol
    2. #18582 runner
    3. #18583 RMCP client
    4. #18584 manager wiring and local/remote coverage
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • exec-server: require explicit filesystem sandbox cwd (#19046)
    ## Why
    
    This is a cleanup PR for the `PermissionProfile` migration stack. #19016
    fixed remote exec-server sandbox contexts so Docker-backed filesystem
    requests use a request/container `cwd` instead of leaking the local test
    runner `cwd`. That exposed the broader API problem:
    `FileSystemSandboxContext::new(SandboxPolicy)` could still reconstruct
    filesystem permissions by reading the exec-server process cwd with
    `AbsolutePathBuf::current_dir()`.
    
    That made `cwd`-dependent legacy entries, such as `:cwd`,
    `:project_roots`, and relative deny globs, depend on ambient process
    state instead of the request sandbox `cwd`. As later PRs make
    `PermissionProfile` the primary permissions abstraction, sandbox
    contexts should be explicit about whether they carry a request `cwd` or
    are profile-only. Removing the implicit constructor prevents new call
    sites from accidentally rebuilding permissions against the wrong `cwd`.
    
    ## What changed
    
    - Removed `FileSystemSandboxContext::new(SandboxPolicy)`.
    - Kept production callers on explicit constructors:
    `from_legacy_sandbox_policy(..., cwd)`, `from_permission_profile(...)`,
    and `from_permission_profile_with_cwd(...)`.
    - Updated exec-server test helpers to construct `PermissionProfile`
    values directly instead of routing through legacy `SandboxPolicy`
    projections.
    - Updated the environment regression test to use an explicit restricted
    profile with no synthetic `cwd`.
    
    ## Verification
    
    - `cargo test -p codex-exec-server`
    - `just fix -p codex-exec-server`
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/19046).
    * #18288
    * #18287
    * #18286
    * #18285
    * #18284
    * #18283
    * #18282
    * #18281
    * #18280
    * __->__ #19046
  • exec-server: expose arg0 alias root to fs sandbox (#19016)
    ## Why
    
    The post-merge `rust-ci-full` run for #18999 still failed the Ubuntu
    remote `suite::remote_env` sandboxed filesystem tests. That run checked
    out merge commit `ddde50c611e4800cb805f243ed3c50bbafe7d011`, so the arg0
    guard lifetime fix was present.
    
    The Docker-backed failure had two remaining pieces:
    
    - The sandboxed filesystem helper needs to execute Codex through the
    `codex-linux-sandbox` arg0 alias path. The helper sandbox was only
    granting read access to the real Codex executable parent, so the alias
    parent also has to be visible inside the helper sandbox.
    - The remote-env tests were building sandbox contexts with
    `FileSystemSandboxContext::new()`, which captures the local test runner
    cwd. In the Docker remote exec-server, that host checkout path does not
    exist, so spawning the filesystem helper failed with `No such file or
    directory` before the helper could process the request.
    
    ## What Changed
    
    - Track all helper runtime read roots instead of a single root.
    - Add both the real Codex executable parent and the
    `codex-linux-sandbox` alias parent to sandbox readable roots.
    - Avoid sending an unused local cwd in remote filesystem sandbox
    contexts when the permission profile has no cwd-dependent entries.
    - Build the Docker remote-env test sandbox contexts with a cwd path that
    exists inside the container.
    - Add unit coverage for the alias-parent root and remote sandbox cwd
    handling.
    
    ## Verification
    
    - `cargo test -p codex-exec-server`
    - `cargo test -p codex-core
    remote_test_env_sandboxed_read_allows_readable_root`
    - `just fix -p codex-exec-server`
    - `just fix -p codex-core`
  • [2/4] Implement executor HTTP request runner (#18582)
    ### Why
    Remote streamable HTTP MCP needs the executor to perform ordinary HTTP
    requests on the executor side. This keeps network placement aligned with
    `experimental_environment = "remote"` without adding MCP-specific
    executor APIs.
    
    ### What
    - Add an executor-side `http/request` runner backed by `reqwest`.
    - Validate request method and URL scheme, preserving the transport
    boundary at plain HTTP.
    - Return buffered responses for ordinary calls and emit ordered
    `http/request/bodyDelta` notifications for streaming responses.
    - Register the request handler in the exec-server router.
    - Document the runner entrypoint, conversion helpers, body-stream
    bridge, notification sender, timeout behavior, and new integration-test
    helpers.
    - Add exec-server integration tests with the existing websocket harness
    and a local TCP HTTP peer for buffered and streamed responses, with
    comments spelling out what each test proves and its
    setup/exercise/assert phases.
    
    ### Stack
    1. #18581 protocol
    2. #18582 runner
    3. #18583 RMCP client
    4. #18584 manager wiring and local/remote coverage
    
    ### Verification
    - `just fmt`
    - `cargo check -p codex-exec-server -p codex-rmcp-client --tests`
    - `cargo check -p codex-core --test all` compile-only
    - `git diff --check`
    - Online full CI is running from the `full-ci` branch, including the
    remote Rust test job.
    
    Co-authored-by: Codex <noreply@openai.com>
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • fix: fully revert agent identity runtime wiring (#18757)
    ## Summary
    
    This PR fully reverts the previously merged Agent Identity runtime
    integration from the old stack:
    https://github.com/openai/codex/pull/17387/changes
    
    It removes the Codex-side task lifecycle wiring, rollout/session
    persistence, feature flag plumbing, lazy `auth.json` mutation,
    background task auth paths, and request callsite changes introduced by
    that stack.
    
    This leaves the repo in a clean pre-AgentIdentity integration state so
    the follow-up PRs can reintroduce the pieces in smaller reviewable
    layers.
    
    ## Stack
    
    1. This PR: full revert
    2. https://github.com/openai/codex/pull/18871: move Agent Identity
    business logic into a crate
    3. https://github.com/openai/codex/pull/18785: add explicit
    AgentIdentity auth mode and startup task allocation
    4. https://github.com/openai/codex/pull/18811: migrate auth callsites
    through AuthProvider
    
    ## Testing
    
    Tests: targeted Rust checks, cargo-shear, Bazel lock check, and CI.
  • [2/8] Support piped stdin in exec process API (#18086)
    ## Summary
    - Add an explicit stdin mode to process/start.
    - Keep normal non-interactive exec stdin closed while allowing
    pipe-backed processes.
    
    ## Stack
    ```text
    o  #18027 [8/8] Fail exec client operations after disconnect
    │
    o  #18025 [7/8] Cover MCP stdio tests with executor placement
    │
    o  #18089 [6/8] Wire remote MCP stdio through executor
    │
    o  #18088 [5/8] Add executor process transport for MCP stdio
    │
    o  #18087 [4/8] Abstract MCP stdio server launching
    │
    o  #18020 [3/8] Add pushed exec process events
    │
    @  #18086 [2/8] Support piped stdin in exec process API
    │
    o  #18085 [1/8] Add MCP server environment config
    │
    o  main
    ```
    
    Co-authored-by: Codex <noreply@openai.com>
  • [codex] Add symlink flag to fs metadata (#17719)
    Add `is_symlink` to FsMetadata struct.
  • Build remote exec env from exec-server policy (#17216)
    ## Summary
    - add an exec-server `envPolicy` field; when present, the server starts
    from its own process env and applies the shell environment policy there
    - keep `env` as the exact environment for local/embedded starts, but
    make it an overlay for remote unified-exec starts
    - move the shell-environment-policy builder into `codex-config` so Core
    and exec-server share the inherit/filter/set/include behavior
    - overlay only runtime/sandbox/network deltas from Core onto the
    exec-server-derived env
    
    ## Why
    Remote unified exec was materializing the shell env inside Core and
    forwarding the whole map to exec-server, so remote processes could
    inherit the orchestrator machine's `HOME`, `PATH`, etc. This keeps the
    base env on the executor while preserving Core-owned runtime additions
    like `CODEX_THREAD_ID`, unified-exec defaults, network proxy env, and
    sandbox marker env.
    
    ## Validation
    - `just fmt`
    - `git diff --check`
    - `cargo test -p codex-exec-server --lib`
    - `cargo test -p codex-core --lib unified_exec::process_manager::tests`
    - `cargo test -p codex-core --lib exec_env::tests`
    - `cargo test -p codex-core --lib exec_env_tests` (compile-only; filter
    matched 0 tests)
    - `cargo test -p codex-config --lib shell_environment` (compile-only;
    filter matched 0 tests)
    - `just bazel-lock-update`
    
    ## Known local validation issue
    - `just bazel-lock-check` is not runnable in this checkout: it invokes
    `./scripts/check-module-bazel-lock.sh`, which is missing.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • Run exec-server fs operations through sandbox helper (#17294)
    ## Summary
    - run exec-server filesystem RPCs requiring sandboxing through a
    `codex-fs` arg0 helper over stdin/stdout
    - keep direct local filesystem execution for `DangerFullAccess` and
    external sandbox policies
    - remove the standalone exec-server binary path in favor of top-level
    arg0 dispatch/runtime paths
    - add sandbox escape regression coverage for local and remote filesystem
    paths
    
    ## Validation
    - `just fmt`
    - `git diff --check`
    - remote devbox: `cd codex-rs && bazel test --bes_backend=
    --bes_results_url= //codex-rs/exec-server:all` (6/6 passed)
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Fix Windows exec-server output test flake (#17409)
    Problem: The Windows exec-server test command could let separator
    whitespace become part of `echo` output, making the exact
    retained-output assertion flaky.
    
    Solution: Tighten the Windows `cmd.exe` command by placing command
    separators directly after the echoed tokens so stdout remains
    deterministic while preserving the exact assertion.
  • feat: move exec-server ownership (#16344)
    This introduces session-scoped ownership for exec-server so ws
    disconnects no longer immediately kill running remote exec processes,
    and it prepares the protocol for reconnect-based resume.
    - add session_id / resume_session_id to the exec-server initialize
    handshake
      - move process ownership under a shared session registry
    - detach sessions on websocket disconnect and expire them after a TTL
    instead of killing processes immediately (we will resume based on this)
    - allow a new connection to resume an existing session and take over
    notifications/ownership
    - I use UUID to make them not predictable as we don't have auth for now
    - make detached-session expiry authoritative at resume time so teardown
    wins at the TTL boundary
    - reject long-poll process/read calls that get resumed out from under an
    older attachment
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add sandbox support to filesystem APIs (#16751)
    ## Summary
    - add optional `sandboxPolicy` support to the app-server filesystem
    request surface
    - thread sandbox-aware filesystem options through app-server and
    exec-server adapters
    - enforce sandboxed read/write access in the filesystem abstraction with
    focused local and remote coverage
    
    ## Validation
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-exec-server file_system`
    - `cargo test -p codex-app-server suite::v2::fs`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Make AGENTS.md discovery FS-aware (#15826)
    ## Summary
    - make AGENTS.md discovery and loading fully FS-aware and remove the
    non-FS discover helper
    - migrate remote-aware codex-core tests to use TestEnv workspace setup
    instead of syncing a local workspace copy
    - add AGENTS.md corner-case coverage, including directory fallbacks and
    remote-aware integration coverage
    
    ## Testing
    - cargo test -p codex-core project_doc -- --nocapture
    - cargo test -p codex-core hierarchical_agents -- --nocapture
    - cargo test -p codex-core agents_md -- --nocapture
    - cargo test -p codex-tui status -- --nocapture
    - cargo test -p codex-tui-app-server status -- --nocapture
    - just fix
    - just fmt
    - just bazel-lock-update
    - just bazel-lock-check
    - just argument-comment-lint
    - remote Linux executor tests in progress via scripts/test-remote-env.sh
  • extract models manager and related ownership from core (#16508)
    ## Summary
    - split `models-manager` out of `core` and add `ModelsManagerConfig`
    plus `Config::to_models_manager_config()` so model metadata paths stop
    depending on `core::Config`
    - move login-owned/auth-owned code out of `core` into `codex-login`,
    move model provider config into `codex-model-provider-info`, move API
    bridge mapping into `codex-api`, move protocol-owned types/impls into
    `codex-protocol`, and move response debug helpers into a dedicated
    `response-debug-context` crate
    - move feedback tag emission into `codex-feedback`, relocate tests to
    the crates that now own the code, and keep broad temporary re-exports so
    this PR avoids a giant import-only rewrite
    
    ## Major moves and decisions
    - created `codex-models-manager` as the owner for model
    cache/catalog/config/model info logic, including the new
    `ModelsManagerConfig` struct
    - created `codex-model-provider-info` as the owner for provider config
    parsing/defaults and kept temporary `codex-login`/`codex-core`
    re-exports for old import paths
    - moved `api_bridge` error mapping + `CoreAuthProvider` into
    `codex-api`, while `codex-login::api_bridge` temporarily re-exports
    those symbols and keeps the `auth_provider_from_auth` wrapper
    - moved `auth_env_telemetry` and `provider_auth` ownership to
    `codex-login`
    - moved `CodexErr` ownership to `codex-protocol::error`, plus
    `StreamOutput`, `bytes_to_string_smart`, and network policy helpers to
    protocol-owned modules
    - created `codex-response-debug-context` for
    `extract_response_debug_context`, `telemetry_transport_error_message`,
    and related response-debug plumbing instead of leaving that behavior in
    `core`
    - moved `FeedbackRequestTags`, `emit_feedback_request_tags`, and
    `emit_feedback_request_tags_with_auth_env` to `codex-feedback`
    - deferred removal of temporary re-exports and the mechanical import
    rewrites to a stacked follow-up PR so this PR stays reviewable
    
    ## Test moves
    - moved auth refresh coverage from `core/tests/suite/auth_refresh.rs` to
    `login/tests/suite/auth_refresh.rs`
    - moved text encoding coverage from
    `core/tests/suite/text_encoding_fix.rs` to
    `protocol/src/exec_output_tests.rs`
    - moved model info override coverage from
    `core/tests/suite/model_info_overrides.rs` to
    `models-manager/src/model_info_overrides_tests.rs`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • feat: use ProcessId in exec-server (#15866)
    Use a full struct for the ProcessId to increase readability and make it
    easier in the future to make it evolve if needed
  • Split exec process into local and remote implementations (#15233)
    ## Summary
    - match the exec-process structure to filesystem PR #15232
    - expose `ExecProcess` on `Environment`
    - make `LocalProcess` the real implementation and `RemoteProcess` a thin
    network proxy over `ExecServerClient`
    - make `ProcessHandler` a thin RPC adapter delegating to `LocalProcess`
    - add a shared local/remote process test
    
    ## Validation
    - `just fmt`
    - `CARGO_TARGET_DIR=~/.cache/cargo-target/codex cargo test -p
    codex-exec-server`
    - `just fix -p codex-exec-server`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Refactor ExecServer filesystem split between local and remote (#15232)
    For each feature we have:
    1. Trait exposed on environment
    2. **Local Implementation** of the trait
    3. Remote implementation that uses the client to proxy via network
    4. Handler implementation that handles PRC requests and calls into
    **Local Implementation**
  • Move terminal module to terminal-detection crate (#15216)
    - Move core/src/terminal.rs and its tests into a standalone
    terminal-detection workspace crate.
    - Update direct consumers to depend on codex-terminal-detection and
    import terminal APIs directly.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add exec-server exec RPC implementation (#15090)
    Stacked PR 2/3, based on the stub PR.
    
    Adds the exec RPC implementation and process/event flow in exec-server
    only.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Remove stdio transport from exec server (#15119)
    Summary
    - delete the deprecated stdio transport plumbing from the exec server
    stack
    - add a basic `exec_server()` harness plus test utilities to start a
    server, send requests, and await events
    - refresh exec-server dependencies, configs, and documentation to
    reflect the new flow
    
    Testing
    - Not run (not requested)
    
    ---------
    
    Co-authored-by: starr-openai <starr@openai.com>
    Co-authored-by: Codex <noreply@openai.com>
  • Add exec-server stub server and protocol docs (#15089)
    Stacked PR 1/3.
    
    This is the initialize-only exec-server stub slice: binary/client
    scaffolding and protocol docs, without exec/filesystem implementation.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>