Commit Graph

9 Commits

  • Reconnect disconnected exec-server websocket clients with fresh sessions (#23867)
    ## Summary
    - replace the one-shot lazy remote exec-server cache with a
    lock-protected current client
    - when the cached websocket client is already disconnected, create one
    fresh websocket client/session on the next `get()`
    - keep existing disconnect failure behavior for old process sessions and
    HTTP body streams; do not add session resume or request retry
    
    ## Why
    The prior PR direction was trying to grow into session restore: resume
    the old `session_id`, preserve existing process handles, and add
    reconnect retry policy. That is more machinery than we want for this
    slice.
    
    For now, the useful minimum is simpler: later fresh remote operations
    should not be stuck behind a dead cached websocket client, but anything
    already attached to the dead connection should fail loudly through the
    existing disconnect path. The server already has detached-session
    cleanup via its existing TTL, so this PR does not need to add
    client-side session preservation.
    
    ## What Changed
    - `LazyRemoteExecServerClient::get()` now keeps the current concrete
    client in a small mutex-protected cache plus one async connect lock.
    - If that cached client is still connected, `get()` returns it.
    - If that cached websocket client has observed the transport close,
    `get()` creates a brand-new websocket client with a brand-new
    exec-server session and replaces the cache.
    - If that cached client is stdio-backed, behavior stays one-shot: the
    dead client is returned and later work surfaces the existing disconnect
    error.
    - No `resume_session_id`, backoff, request replay, or existing
    `RemoteExecProcess` rebinding is added here.
    - Added focused websocket coverage that proves two concurrent `get()`
    calls after disconnect share one fresh replacement client/session.
  • Add stdio exec-server client transport (#20664)
    ## Why
    
    Configured environments need to connect to exec-server instances that
    are not necessarily already listening on a websocket URL. A
    command-backed stdio transport lets Codex start an exec-server process,
    speak JSON-RPC over its stdio streams, and clean up that child process
    with the client lifetime.
    
    **Stack position:** this is PR 2 of 5. It builds on the server-side
    stdio listener from PR 1 and provides the client transport used by later
    environment/config PRs.
    
    ## What Changed
    
    - Add `ExecServerTransport` variants for websocket URLs and stdio shell
    commands.
    - Add stdio command connection support for `ExecServerClient`.
    - Move websocket/stdio transport setup into `client_transport.rs` so
    `client.rs` stays focused on shared JSON-RPC client, session, HTTP, and
    notification behavior.
    - Tie stdio child process cleanup to the JSON-RPC connection lifetime
    with a RAII lifetime guard.
    - Keep existing websocket environment behavior by adapting URL-backed
    remotes to `ExecServerTransport::WebSocketUrl`.
    
    ## Stack
    
    - 1. https://github.com/openai/codex/pull/20663 - Add stdio exec-server
    listener
    - **2. This PR:** https://github.com/openai/codex/pull/20664 - Add stdio
    exec-server client transport
    - 3. https://github.com/openai/codex/pull/20665 - Make environment
    providers own default selection
    - 4. https://github.com/openai/codex/pull/20666 - Add CODEX_HOME
    environments TOML provider
    - 5. https://github.com/openai/codex/pull/20667 - Load configured
    environments from CODEX_HOME
    
    Split from original draft: https://github.com/openai/codex/pull/20508
    
    ## Validation
    
    Not run locally; this was split out of the original draft stack and then
    refactored to separate transport setup from the base client.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [2/4] Implement executor HTTP request runner (#18582)
    ### Why
    Remote streamable HTTP MCP needs the executor to perform ordinary HTTP
    requests on the executor side. This keeps network placement aligned with
    `experimental_environment = "remote"` without adding MCP-specific
    executor APIs.
    
    ### What
    - Add an executor-side `http/request` runner backed by `reqwest`.
    - Validate request method and URL scheme, preserving the transport
    boundary at plain HTTP.
    - Return buffered responses for ordinary calls and emit ordered
    `http/request/bodyDelta` notifications for streaming responses.
    - Register the request handler in the exec-server router.
    - Document the runner entrypoint, conversion helpers, body-stream
    bridge, notification sender, timeout behavior, and new integration-test
    helpers.
    - Add exec-server integration tests with the existing websocket harness
    and a local TCP HTTP peer for buffered and streamed responses, with
    comments spelling out what each test proves and its
    setup/exercise/assert phases.
    
    ### Stack
    1. #18581 protocol
    2. #18582 runner
    3. #18583 RMCP client
    4. #18584 manager wiring and local/remote coverage
    
    ### Verification
    - `just fmt`
    - `cargo check -p codex-exec-server -p codex-rmcp-client --tests`
    - `cargo check -p codex-core --test all` compile-only
    - `git diff --check`
    - Online full CI is running from the `full-ci` branch, including the
    remote Rust test job.
    
    Co-authored-by: Codex <noreply@openai.com>
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [6/6] Fail exec client operations after disconnect (#18027)
    ## Summary
    - Reject new exec-server client operations once the transport has
    disconnected.
    - Convert pending RPC calls into closed errors instead of synthetic
    server errors.
    - Cover pending read and later write behavior after remote executor
    disconnect.
    
    ## Verification
    - `just fmt`
    - `cargo check -p codex-exec-server`
    
    ## Stack
    ```text
    @  #18027 [6/6] Fail exec client operations after disconnect
    │
    o  #18212 [5/6] Wire executor-backed MCP stdio
    │
    o  #18087 [4/6] Abstract MCP stdio server launching
    │
    o  #18020 [3/6] Add pushed exec process events
    │
    o  #18086 [2/6] Support piped stdin in exec process API
    │
    o  #18085 [1/6] Add MCP server environment config
    │
    o  main
    ```
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • feat: move exec-server ownership (#16344)
    This introduces session-scoped ownership for exec-server so ws
    disconnects no longer immediately kill running remote exec processes,
    and it prepares the protocol for reconnect-based resume.
    - add session_id / resume_session_id to the exec-server initialize
    handshake
      - move process ownership under a shared session registry
    - detach sessions on websocket disconnect and expire them after a TTL
    instead of killing processes immediately (we will resume based on this)
    - allow a new connection to resume an existing session and take over
    notifications/ownership
    - I use UUID to make them not predictable as we don't have auth for now
    - make detached-session expiry authoritative at resume time so teardown
    wins at the TTL boundary
    - reject long-poll process/read calls that get resumed out from under an
    older attachment
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Make AGENTS.md discovery FS-aware (#15826)
    ## Summary
    - make AGENTS.md discovery and loading fully FS-aware and remove the
    non-FS discover helper
    - migrate remote-aware codex-core tests to use TestEnv workspace setup
    instead of syncing a local workspace copy
    - add AGENTS.md corner-case coverage, including directory fallbacks and
    remote-aware integration coverage
    
    ## Testing
    - cargo test -p codex-core project_doc -- --nocapture
    - cargo test -p codex-core hierarchical_agents -- --nocapture
    - cargo test -p codex-core agents_md -- --nocapture
    - cargo test -p codex-tui status -- --nocapture
    - cargo test -p codex-tui-app-server status -- --nocapture
    - just fix
    - just fmt
    - just bazel-lock-update
    - just bazel-lock-check
    - just argument-comment-lint
    - remote Linux executor tests in progress via scripts/test-remote-env.sh
  • feat: exec-server prep for unified exec (#15691)
    This PR partially rebase `unified_exec` on the `exec-server` and adapt
    the `exec-server` accordingly.
    
    ## What changed in `exec-server`
    
    1. Replaced the old "broadcast-driven; process-global" event model with
    process-scoped session events. The goal is to be able to have dedicated
    handler for each process.
    2. Add to protocol contract to support explicit lifecycle status and
    stream ordering:
    - `WriteResponse` now returns `WriteStatus` (Accepted, UnknownProcess,
    StdinClosed, Starting) instead of a bool.
      - Added seq fields to output/exited notifications.
      - Added terminal process/closed notification.
    3. Demultiplexed remote notifications into per-process channels. Same as
    for the event sys
    4. Local and remote backends now both implement ExecBackend.
    5. Local backend wraps internal process ID/operations into per-process
    ExecProcess objects.
    6. Remote backend registers a session channel before launch and
    unregisters on failed launch.
    
    ## What changed in `unified_exec`
    
    1. Added unified process-state model and backend-neutral process
    wrapper. This will probably disappear in the future, but it makes it
    easier to keep the work flowing on both side.
    - `UnifiedExecProcess` now handles both local PTY sessions and remote
    exec-server processes through a shared `ProcessHandle`.
    - Added `ProcessState` to track has_exited, exit_code, and terminal
    failure message consistently across backends.
    2. Routed write and lifecycle handling through process-level methods.
    
    ## Some rationals
    
    1. The change centralizes execution transport in exec-server while
    preserving policy and orchestration ownership in core, avoiding
    duplicated launch approval logic. This comes from internal discussion.
    2. Session-scoped events remove coupling/cross-talk between processes
    and make stream ordering and terminal state explicit (seq, closed,
    failed).
    3. The failure-path surfacing (remote launch failures, write failures,
    transport disconnects) makes command tool output and cleanup behavior
    deterministic
    
    ## Follow-ups:
    * Unify the concept of thread ID behind an obfuscated struct
    * FD handling
    * Full zsh-fork compatibility
    * Full network sandboxing compatibility
    * Handle ws disconnection
  • Add exec-server exec RPC implementation (#15090)
    Stacked PR 2/3, based on the stub PR.
    
    Adds the exec RPC implementation and process/event flow in exec-server
    only.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add exec-server stub server and protocol docs (#15089)
    Stacked PR 1/3.
    
    This is the initialize-only exec-server stub slice: binary/client
    scaffolding and protocol docs, without exec/filesystem implementation.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>