Commit Graph

13 Commits

  • [codex] migrate exec-server filesystem protocol to PathUri (#27653)
    Exec-server filesystem calls should preserve cross-platform `file:` URIs
    across the remote boundary instead of converting them through paths
    native to the client host.
    
    This changes the exec-server filesystem protocol DTOs to use `PathUri`,
    carries those values directly through remote and sandbox-helper
    transports, and keeps legacy native absolute-path request strings
    readable for compatibility. It also updates protocol documentation and
    coverage for URI serialization and non-native URI forwarding.
  • Allow API-key auth for remote exec-server registration (#24666)
    ## Overview
    Allow remote `codex exec-server` registration to use existing API-key
    auth while restricting where those credentials can be sent.
    
    - Accept `CodexAuth::ApiKey` for the normal `--remote` registration
    path.
    - Restrict API-key remote registration to HTTPS `openai.com` and
    `openai.org` hosts and subdomains, with explicit HTTP loopback support
    for local development.
    - Disable registry registration redirects so credentials cannot be
    forwarded to an unvalidated destination.
    - Retain `--use-agent-identity-auth` as the explicit Agent Identity
    path.
    - Document remote registration using `CODEX_API_KEY`.
    
    ## Big picture
    Callers can now provide an API key directly to `exec-server`
    registration without first establishing ChatGPT login state:
    
    ```sh
    CODEX_API_KEY="$OPENAI_API_KEY" \
    codex exec-server \
      --remote "https://<host>.openai.org/api" \
      --environment-id "$ENVIRONMENT_ID"
    ```
    
    ## Validation
    - `cargo fmt --all` (`just fmt` is not installed on this host)
    - `cargo test -p codex-cli -p codex-exec-server`
  • Migrate exec-server remote registration to environments (#23633)
    ## Summary
    - migrate exec-server remote registration naming from executor to
    environment
    - align CLI, public Rust exports, registry error messages, and relay
    test fixtures with the environment registry contract
    - keep the live registration path and response model consistent with
    `/cloud/environment/{environment_id}/register`
    
    ## Verification
    - `cargo test -p codex-exec-server
    remote::tests::register_environment_posts_with_auth_provider_headers
    --manifest-path /Users/richardlee/code/codex/codex-rs/Cargo.toml`
    - `cargo test -p codex-exec-server --test relay
    multiplexed_remote_environment_routes_independent_virtual_streams
    --manifest-path /Users/richardlee/code/codex/codex-rs/Cargo.toml`
    - `cargo check -p codex-cli --manifest-path
    /Users/richardlee/code/codex/codex-rs/Cargo.toml` (still running when PR
    opened; will update after completion if needed)
  • exec-server: support auth-backed remote executor registration (#22769)
    This updates remote `exec-server` registration to use normal Codex auth
    instead of a registry-issued credential. The registry request is built
    from the existing auth-provider path, which preserves the biscuit-only
    registry contract introduced in
    [openai/openai#924101](https://github.com/openai/openai/pull/924101)
    while removing the old remote registry bearer env var and its direct
    transport assumptions.
    
    The default remote flow uses persisted ChatGPT auth from the normal
    Codex config/storage path. This PR also includes the containerized Agent
    Identity path needed by
    [openai/openai#924260](https://github.com/openai/openai/pull/924260):
    remote `exec-server` accepts `--allow-agent-identity-auth`, permits
    Agent Identity auth loaded from `CODEX_ACCESS_TOKEN` only when that flag
    is present, and reuses the existing Agent task registration plus derived
    `AgentAssertion` header generation. API-key auth remains unsupported,
    and Agent Identity stays opt-in.
    
    Validation performed beyond normal presubmit coverage:
    - `cargo fmt --all --check`
    - `cargo check -p codex-cli`
    - `cargo test -p codex-exec-server`
    - `cargo test -p codex-cli exec_server_agent_identity_auth_flag_`
    - `cargo test -p codex-cli remote_exec_server_auth_mode_`
    
    I also attempted `cargo test -p codex-cli`. The new CLI tests passed
    inside that run, but the suite ended on an unrelated local
    marketplace-state failure in
    `plugin_list_excludes_unconfigured_repo_local_marketplaces`.
  • feat(exec-server): use protobuf relay frames (#22343)
    ## Why
    
    Remote exec-server now needs one executor websocket to serve multiple
    harness JSON-RPC sessions. Rendezvous routes by `stream_id`, and the
    exec-server side needs to use the same stable relay frame contract
    instead of a hand-rolled JSON shape.
    
    The relay protocol also needs to make ownership boundaries clear:
    harness and executor endpoints own sequencing, acks, retries, duplicate
    suppression, segmentation, and reassembly; rendezvous only routes
    frames.
    
    ## What Changed
    
    - Add the checked-in `codex.exec_server.relay.v1.RelayMessageFrame`
    proto plus generated prost bindings for `codex-exec-server`.
    - Encode remote harness/executor relay traffic as binary protobuf
    websocket frames while keeping local websocket JSON-RPC unchanged.
    - Demux executor-side relay streams into independent
    `ConnectionProcessor` sessions keyed by `stream_id`.
    - Add a programmatic `RemoteExecutorConfig::with_bearer_token(...)`
    constructor for non-CLI callers and integration tests.
    - Add an integration test that starts the remote executor against a fake
    registry/rendezvous websocket and verifies two virtual streams share one
    executor websocket without cross-talk, including per-stream reset
    behavior.
    - Document the remote relay envelope, sequence ranges, `ack`/`ack_bits`,
    and endpoint responsibilities in `exec-server/README.md`.
    
    ## Verification
    
    - `cargo test -p codex-exec-server --test relay
    multiplexed_remote_executor_routes_independent_virtual_streams --
    --exact`
    - `cargo test -p codex-exec-server --test relay`
    - `cargo test -p codex-exec-server` passed outside the sandbox. The
    sandboxed run hit macOS `sandbox-exec: sandbox_apply: Operation not
    permitted` in filesystem sandbox tests.
  • Add cloud executor registration to exec-server (#19575)
    ## Summary
    This PR adds the first `codex-rs` milestone for remote-exec e2e: a local
    `codex exec-server` can now register itself with
    `codex-cloud-environments` and attach to the returned rendezvous
    websocket.
    
    At a high level, `codex exec-server --cloud ...` now:
    - loads ChatGPT auth from normal Codex config
    - registers an executor with `codex-cloud-environments`
    - receives a signed rendezvous websocket URL
    - serves the existing exec-server JSON-RPC protocol over that websocket
    
    ## What Changed
    - Added `--cloud`, `--cloud-base-url`, `--cloud-environment-id`, and
    `--cloud-name` to `codex exec-server`
    - Added a new `exec-server/src/cloud.rs` module that handles:
      - registration requests
      - auth/header setup
      - bounded auth retry on `401/403`
      - reconnect/backoff after websocket disconnects
    - Reused the existing `ConnectionProcessor` / `ExecServerHandler` path
    so cloud mode serves the same exec/filesystem RPC surface as local
    websocket mode
    - Added cloud-specific error variants and minimal docs for the new mode
    
    ## Testing
    Manual e2e test that fully goes through exec server flow with our codex
    cloud agent as orchestrator
  • [2/8] Support piped stdin in exec process API (#18086)
    ## Summary
    - Add an explicit stdin mode to process/start.
    - Keep normal non-interactive exec stdin closed while allowing
    pipe-backed processes.
    
    ## Stack
    ```text
    o  #18027 [8/8] Fail exec client operations after disconnect
    │
    o  #18025 [7/8] Cover MCP stdio tests with executor placement
    │
    o  #18089 [6/8] Wire remote MCP stdio through executor
    │
    o  #18088 [5/8] Add executor process transport for MCP stdio
    │
    o  #18087 [4/8] Abstract MCP stdio server launching
    │
    o  #18020 [3/8] Add pushed exec process events
    │
    @  #18086 [2/8] Support piped stdin in exec process API
    │
    o  #18085 [1/8] Add MCP server environment config
    │
    o  main
    ```
    
    Co-authored-by: Codex <noreply@openai.com>
  • Run exec-server fs operations through sandbox helper (#17294)
    ## Summary
    - run exec-server filesystem RPCs requiring sandboxing through a
    `codex-fs` arg0 helper over stdin/stdout
    - keep direct local filesystem execution for `DangerFullAccess` and
    external sandbox policies
    - remove the standalone exec-server binary path in favor of top-level
    arg0 dispatch/runtime paths
    - add sandbox escape regression coverage for local and remote filesystem
    paths
    
    ## Validation
    - `just fmt`
    - `git diff --check`
    - remote devbox: `cd codex-rs && bazel test --bes_backend=
    --bes_results_url= //codex-rs/exec-server:all` (6/6 passed)
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Remove stdio transport from exec server (#15119)
    Summary
    - delete the deprecated stdio transport plumbing from the exec server
    stack
    - add a basic `exec_server()` harness plus test utilities to start a
    server, send requests, and await events
    - refresh exec-server dependencies, configs, and documentation to
    reflect the new flow
    
    Testing
    - Not run (not requested)
    
    ---------
    
    Co-authored-by: starr-openai <starr@openai.com>
    Co-authored-by: Codex <noreply@openai.com>
  • Add exec-server stub server and protocol docs (#15089)
    Stacked PR 1/3.
    
    This is the initialize-only exec-server stub slice: binary/client
    scaffolding and protocol docs, without exec/filesystem implementation.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • refactor: delete exec-server and move execve wrapper into shell-escalation (#12632)
    ## Why
    
    We already plan to remove the shell-tool MCP path, and doing that
    cleanup first makes the follow-on `shell-escalation` work much simpler.
    
    This change removes the last remaining reason to keep
    `codex-rs/exec-server` around by moving the `codex-execve-wrapper`
    binary and shared shell test fixtures to the crates/tests that now own
    that functionality.
    
    ## What Changed
    
    ### Delete `codex-rs/exec-server`
    
    - Remove the `exec-server` crate, including the MCP server binary,
    MCP-specific modules, and its test support/test suite
    - Remove `exec-server` from the `codex-rs` workspace and update
    `Cargo.lock`
    
    ### Move `codex-execve-wrapper` into `codex-rs/shell-escalation`
    
    - Move the wrapper implementation into `shell-escalation`
    (`src/unix/execve_wrapper.rs`)
    - Add the `codex-execve-wrapper` binary entrypoint under
    `shell-escalation/src/bin/`
    - Update `shell-escalation` exports/module layout so the wrapper
    entrypoint is hosted there
    - Move the wrapper README content from `exec-server` to
    `shell-escalation/README.md`
    
    ### Move shared shell test fixtures to `app-server`
    
    - Move the DotSlash `bash`/`zsh` test fixtures from
    `exec-server/tests/suite/` to `app-server/tests/suite/`
    - Update `app-server` zsh-fork tests to reference the new fixture paths
    
    ### Keep `shell-tool-mcp` as a shell-assets package
    
    - Update `.github/workflows/shell-tool-mcp.yml` packaging so the npm
    artifact contains only patched Bash/Zsh payloads (no Rust binaries)
    - Update `shell-tool-mcp/package.json`, `shell-tool-mcp/src/index.ts`,
    and docs to reflect the shell-assets-only package shape
    - `shell-tool-mcp-ci.yml` does not need changes because it is already
    JS-only
    
    ## Verification
    
    - `cargo shear`
    - `cargo clippy -p codex-shell-escalation --tests`
    - `just clippy`
  • feat(shell-tool-mcp): add patched zsh build pipeline (#11668)
    ## Summary
    - add `shell-tool-mcp/patches/zsh-exec-wrapper.patch` against upstream
    zsh `77045ef899e53b9598bebc5a41db93a548a40ca6`
    - add `zsh-linux` and `zsh-darwin` jobs to
    `.github/workflows/shell-tool-mcp.yml`
    - stage zsh binaries under `artifacts/vendor/<target>/zsh/<variant>/zsh`
    - include zsh artifact jobs in `package.needs`
    - mark staged zsh binaries executable during packaging
    
    ## Notes
    - zsh source is cloned from `https://git.code.sf.net/p/zsh/code`
    - workflow pins zsh commit `77045ef899e53b9598bebc5a41db93a548a40ca6`
    - zsh build runs `./Util/preconfig` before `./configure`
    
    ## Validation
    - parsed workflow YAML locally (`yaml-ok`)
    - validated zsh patch applies cleanly with `git apply --check` on a
    fresh zsh clone
  • docs: update the docs for @openai/codex-shell-tool-mcp (#7962)
    The existing version of `shell-tool-mcp/README.md` was not written in a
    way that was meant to be consumed by end-users. This is now fixed.
    
    Added `codex-rs/exec-server/README.md` for the more technical bits.