Commit Graph

11 Commits

  • [codex] Initialize exec-server OpenTelemetry at startup (#25019)
    ## Summary
    
    - Initialize stderr tracing and the configured OpenTelemetry provider
    for local and remote `codex exec-server` startup.
    - Instrument the local and remote server entrypoints with a root runtime
    span.
    - Keep raw Noise environment, registration, and stream identifiers out
    of exported spans while preserving them in local debug events.
    - Keep telemetry setup in a focused CLI module instead of growing the
    top-level command entrypoint.
    
    ## Stack
    
    - Previous: none (`#27058` has merged)
    - Next: #27466
    
    ## Validation
    
    - `just test -p codex-exec-server --lib` (139 passed)
    - `just test -p codex-cli --test exec_server` (3 passed)
    - `just bazel-lock-check`
    - `just fix -p codex-exec-server -p codex-cli`
    - `just fmt`
    
    ---------
    
    Co-authored-by: Richard Lee <richardlee@openai.com>
  • feat(exec-server): add Noise rendezvous environment (#28774)
    ## Why
    
    Codex can run a remote exec server through the Noise relay, but the
    normal
    environment-manager path could not establish an
    environment-registry-backed
    harness connection. Signed rendezvous URLs and harness authorizations
    are
    short-lived, so reconnects must fetch a fresh bundle instead of
    retaining
    stale connection credentials. A stalled registry request must also fail
    within
    the regular remote connection deadline, without exposing these
    credentials in
    debug logs.
    
    Issue: N/A (internal environment-service integration).
    
    ## What Changed
    
    - Add environment-manager configuration for a registry-backed Noise
    rendezvous
      environment.
    - Request a fresh bundle from
    `/cloud/environment/{environment_id}/connect` for every physical harness
      connection, using the existing 10-second remote connection timeout.
    - Share the Environment Registry register, connect, and validate wire
    payloads
      through `codex-exec-server` and `codex-core-api`.
    - Redact the signed rendezvous URL and harness authorization from the
    public
      connect response's `Debug` output.
    - Add focused coverage for registry bundle retrieval, stalled requests,
    and
      credential redaction.
  • exec-server: expose environment registry payloads (#28651)
    ## Why
    
    Services that proxy the exec-server environment registry endpoints need
    to deserialize and forward the same Noise registration and harness-key
    validation payloads. Those wire models currently live as private,
    serialize-only structs in `exec-server`, which forces consumers to
    duplicate the contract.
    
    ## What changed
    
    - Add owned serde models for registration and harness-key validation
    requests and responses.
    - Use those models in the existing exec-server registry client.
    - Re-export the models from `codex-exec-server` and `codex-core-api`.
    - Keep the harness authorization request free of a derived `Debug`
    implementation so it is not accidentally logged.
    
    ## Testing
    
    - Focused exec-server registration and harness-key validation tests: 2
    passed.
    - `cargo check -p codex-core-api`
    
    The full `codex-exec-server` suite compiled and ran 254 tests: 222
    passed, while 32 existing filesystem sandbox tests could not run under
    the nested macOS sandbox (`sandbox_apply: Operation not permitted`).
    
    Co-authored-by: Codex <noreply@openai.com>
  • exec-server: default remote transport to Noise (#26245)
    ## Why
    
    The transport in
    [openai/codex#26242](https://github.com/openai/codex/pull/26242) needs
    to be used by every remote orchestrator-to-executor connection before
    JSON-RPC traffic starts.
    
    ## Changes
    
    - Generates one executor Noise identity when remote exec-server starts
    and registers its public key.
    - Creates a harness identity for each physical remote environment
    connection.
    - Fetches a fresh registry bundle before connecting and validates the
    authenticated harness key before completing the executor handshake.
    - Multiplexes encrypted logical streams over the existing executor
    WebSocket.
    - Adds bounded stream, handshake-failure, and reassembly state.
    - Adds safe lifecycle diagnostics without logging keys, authorizations,
    plaintext, or ciphertext.
    - Covers reconnects, replay rejection, validation failure, framing
    limits, and encrypted JSON-RPC tool traffic.
    
    ## Stack
    
    1. [openai/codex#26242](https://github.com/openai/codex/pull/26242):
    Noise channel and relay transport
    2. **[openai/codex#26245](https://github.com/openai/codex/pull/26245)**:
    remote registration and runtime activation
    
    ## Verification
    
    - `just test -p codex-exec-server`
    - `just fix -p codex-exec-server`
    - `just bazel-lock-check`
    - `cargo shear`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Allow API-key auth for remote exec-server registration (#24666)
    ## Overview
    Allow remote `codex exec-server` registration to use existing API-key
    auth while restricting where those credentials can be sent.
    
    - Accept `CodexAuth::ApiKey` for the normal `--remote` registration
    path.
    - Restrict API-key remote registration to HTTPS `openai.com` and
    `openai.org` hosts and subdomains, with explicit HTTP loopback support
    for local development.
    - Disable registry registration redirects so credentials cannot be
    forwarded to an unvalidated destination.
    - Retain `--use-agent-identity-auth` as the explicit Agent Identity
    path.
    - Document remote registration using `CODEX_API_KEY`.
    
    ## Big picture
    Callers can now provide an API key directly to `exec-server`
    registration without first establishing ChatGPT login state:
    
    ```sh
    CODEX_API_KEY="$OPENAI_API_KEY" \
    codex exec-server \
      --remote "https://<host>.openai.org/api" \
      --environment-id "$ENVIRONMENT_ID"
    ```
    
    ## Validation
    - `cargo fmt --all` (`just fmt` is not installed on this host)
    - `cargo test -p codex-cli -p codex-exec-server`
  • Migrate exec-server remote registration to environments (#23633)
    ## Summary
    - migrate exec-server remote registration naming from executor to
    environment
    - align CLI, public Rust exports, registry error messages, and relay
    test fixtures with the environment registry contract
    - keep the live registration path and response model consistent with
    `/cloud/environment/{environment_id}/register`
    
    ## Verification
    - `cargo test -p codex-exec-server
    remote::tests::register_environment_posts_with_auth_provider_headers
    --manifest-path /Users/richardlee/code/codex/codex-rs/Cargo.toml`
    - `cargo test -p codex-exec-server --test relay
    multiplexed_remote_environment_routes_independent_virtual_streams
    --manifest-path /Users/richardlee/code/codex/codex-rs/Cargo.toml`
    - `cargo check -p codex-cli --manifest-path
    /Users/richardlee/code/codex/codex-rs/Cargo.toml` (still running when PR
    opened; will update after completion if needed)
  • exec-server: support auth-backed remote executor registration (#22769)
    This updates remote `exec-server` registration to use normal Codex auth
    instead of a registry-issued credential. The registry request is built
    from the existing auth-provider path, which preserves the biscuit-only
    registry contract introduced in
    [openai/openai#924101](https://github.com/openai/openai/pull/924101)
    while removing the old remote registry bearer env var and its direct
    transport assumptions.
    
    The default remote flow uses persisted ChatGPT auth from the normal
    Codex config/storage path. This PR also includes the containerized Agent
    Identity path needed by
    [openai/openai#924260](https://github.com/openai/openai/pull/924260):
    remote `exec-server` accepts `--allow-agent-identity-auth`, permits
    Agent Identity auth loaded from `CODEX_ACCESS_TOKEN` only when that flag
    is present, and reuses the existing Agent task registration plus derived
    `AgentAssertion` header generation. API-key auth remains unsupported,
    and Agent Identity stays opt-in.
    
    Validation performed beyond normal presubmit coverage:
    - `cargo fmt --all --check`
    - `cargo check -p codex-cli`
    - `cargo test -p codex-exec-server`
    - `cargo test -p codex-cli exec_server_agent_identity_auth_flag_`
    - `cargo test -p codex-cli remote_exec_server_auth_mode_`
    
    I also attempted `cargo test -p codex-cli`. The new CLI tests passed
    inside that run, but the suite ended on an unrelated local
    marketplace-state failure in
    `plugin_list_excludes_unconfigured_repo_local_marketplaces`.
  • feat(exec-server): use protobuf relay frames (#22343)
    ## Why
    
    Remote exec-server now needs one executor websocket to serve multiple
    harness JSON-RPC sessions. Rendezvous routes by `stream_id`, and the
    exec-server side needs to use the same stable relay frame contract
    instead of a hand-rolled JSON shape.
    
    The relay protocol also needs to make ownership boundaries clear:
    harness and executor endpoints own sequencing, acks, retries, duplicate
    suppression, segmentation, and reassembly; rendezvous only routes
    frames.
    
    ## What Changed
    
    - Add the checked-in `codex.exec_server.relay.v1.RelayMessageFrame`
    proto plus generated prost bindings for `codex-exec-server`.
    - Encode remote harness/executor relay traffic as binary protobuf
    websocket frames while keeping local websocket JSON-RPC unchanged.
    - Demux executor-side relay streams into independent
    `ConnectionProcessor` sessions keyed by `stream_id`.
    - Add a programmatic `RemoteExecutorConfig::with_bearer_token(...)`
    constructor for non-CLI callers and integration tests.
    - Add an integration test that starts the remote executor against a fake
    registry/rendezvous websocket and verifies two virtual streams share one
    executor websocket without cross-talk, including per-stream reset
    behavior.
    - Document the remote relay envelope, sequence ranges, `ack`/`ack_bits`,
    and endpoint responsibilities in `exec-server/README.md`.
    
    ## Verification
    
    - `cargo test -p codex-exec-server --test relay
    multiplexed_remote_executor_routes_independent_virtual_streams --
    --exact`
    - `cargo test -p codex-exec-server --test relay`
    - `cargo test -p codex-exec-server` passed outside the sandbox. The
    sandboxed run hit macOS `sandbox-exec: sandbox_apply: Operation not
    permitted` in filesystem sandbox tests.
  • [exec-server] serve websocket listener via HTTP upgrade (#21963)
    ## Why
    
    `codex exec-server` should keep the existing public `ws://IP:PORT` URL
    shape while serving that websocket connection through an HTTP upgrade
    path internally. That keeps the client-facing configuration simple and
    allows the listener to work through intermediate HTTP-aware
    infrastructure.
    
    ## What changed
    
    - keep the emitted and configured exec-server URL as `ws://IP:PORT`
    - serve that websocket endpoint through Axum HTTP upgrade handling on
    `/`
    - expose `GET /readyz` from the same listener for readiness checks
    - route upgraded Axum websocket streams through the shared JSON-RPC
    connection machinery
    - initialize the rustls crypto provider before websocket client
    connections
    - preserve inbound binary websocket JSON-RPC parsing for compatibility
    with the prior transport behavior
    
    ## Verification
    
    - `cargo test -p codex-exec-server --test health --test process --test
    websocket --test initialize --test exec_process`
  • [codex] support executor registry remote environments (#21323)
    ## Summary
    
    Support registry-backed remote executors end to end so downstream
    services can resolve an executor id into an exec-server URL and make
    that environment available to Codex without relying on the legacy cloud
    environments flow.
    
    ## What changed
    
    - switch remote executor registration to the executor registry bootstrap
    contract
    - allow named remote environments to be inserted into
    `EnvironmentManager` at runtime
    - add the experimental app-server RPC `environment/add` so initialized
    experimental clients can register those remote environments for later
    `thread/start` and `turn/start` selection
    
    ## Validation
    
    Ran focused validation locally:
    - `cargo test -p codex-exec-server environment_manager_`
    - `cargo test -p codex-exec-server
    register_executor_posts_with_bearer_token_header`
    - `cargo test -p codex-app-server-protocol`
  • Add cloud executor registration to exec-server (#19575)
    ## Summary
    This PR adds the first `codex-rs` milestone for remote-exec e2e: a local
    `codex exec-server` can now register itself with
    `codex-cloud-environments` and attach to the returned rendezvous
    websocket.
    
    At a high level, `codex exec-server --cloud ...` now:
    - loads ChatGPT auth from normal Codex config
    - registers an executor with `codex-cloud-environments`
    - receives a signed rendezvous websocket URL
    - serves the existing exec-server JSON-RPC protocol over that websocket
    
    ## What Changed
    - Added `--cloud`, `--cloud-base-url`, `--cloud-environment-id`, and
    `--cloud-name` to `codex exec-server`
    - Added a new `exec-server/src/cloud.rs` module that handles:
      - registration requests
      - auth/header setup
      - bounded auth retry on `401/403`
      - reconnect/backoff after websocket disconnects
    - Reused the existing `ConnectionProcessor` / `ExecServerHandler` path
    so cloud mode serves the same exec/filesystem RPC surface as local
    websocket mode
    - Added cloud-specific error variants and minimal docs for the new mode
    
    ## Testing
    Manual e2e test that fully goes through exec server flow with our codex
    cloud agent as orchestrator