Commit Graph

77 Commits

  • fix(exec-server): retain output until streams close (#18946)
    ## Why
    
    A Mac Bazel run hit a flake in
    `server::handler::tests::output_and_exit_are_retained_after_notification_receiver_closes`
    where the read path observed process exit but lost the expected buffered
    stdout (`first\nsecond\n`). See the [GitHub Actions
    job](https://github.com/openai/codex/actions/runs/24758468552/job/72436716505)
    and [BuildBuddy
    invocation](https://app.buildbuddy.io/invocation/37475a12-4ef2-45fb-ab8a-e49a2aba1d59).
    
    The underlying race is that process exit is not the same thing as
    stdout/stderr closure. If a child or grandchild inherits the pipe write
    end, or a process duplicates it with `dup2`, the watched process can
    exit while the stream is still open and more output can still arrive.
    The exec-server was starting exited-process retention cleanup from the
    exit event, so the process entry could be removed before the output
    streams had actually closed.
    
    While stress-testing the exec-server unit suite,
    `server::handler::tests::long_poll_read_fails_after_session_resume`
    exposed a separate test race: it started a short-lived command that
    could exit and wake the pending long-poll read before the session-resume
    assertion observed the resumed-session error. That test is intended to
    cover resume eviction, not process-exit delivery, so this change keeps
    the process alive and quiet while the second connection resumes the
    session.
    
    ## What changed
    
    - Keep exec-server process entries retained until stdout/stderr streams
    close, then start the post-exit retention timer from the closed event.
    - Wake long-poll readers when the closed event is emitted.
    - Add focused `local_process` unit coverage that proves late output is
    still retained after the short test retention interval has elapsed, and
    that closed process entries are eventually evicted.
    - Add a local and remote regression test where a parent exits while a
    child keeps inherited stdout open. The child waits on an explicit
    release file, so the test deterministically observes exit first,
    releases the child, then requires a nonzero-wait read from the exit
    sequence to receive the late output.
    - In `codex-rs/exec-server/src/server/handler/tests.rs`, make
    `long_poll_read_fails_after_session_resume` run a long-lived silent
    command instead of a short command that prints and exits. This isolates
    the test to session-resume behavior and prevents a normal process exit
    from satisfying the pending long-poll read first.
    
    ## Testing
    
    - `cargo test -p codex-exec-server
    exec_process_retains_output_after_exit_until_streams_close`
    - `cargo test -p codex-exec-server local_process::tests`
    - `cargo test -p codex-exec-server`
    - `just fix -p codex-exec-server`
    - `bazel test //codex-rs/exec-server:exec-server-unit-tests
    //codex-rs/exec-server:exec-server-exec_process-test
    //codex-rs/exec-server:exec-server-file_system-test
    //codex-rs/exec-server:exec-server-http_client-test
    //codex-rs/exec-server:exec-server-initialize-test
    //codex-rs/exec-server:exec-server-process-test
    //codex-rs/exec-server:exec-server-websocket-test`
    - `bazel test --runs_per_test=25
    //codex-rs/exec-server:exec-server-unit-tests`
    
    ## Documentation
    
    No docs update needed; this is an internal exec-server correctness fix.
  • exec-server: wait for close after observed exit (#19130)
    ## Why
    
    Windows CI can flake in
    `server::handler::tests::output_and_exit_are_retained_after_notification_receiver_closes`
    after a process has exited but before both output streams have closed.
    `exec/read` returned immediately whenever `exited` was true, so callers
    that had already observed the exit event could spin instead of
    long-polling for the later `closed` state.
    
    ## What Changed
    
    - Keep returning immediately when a terminal exit event is newly
    observable.
    - Allow later reads, after the caller has advanced past that event, to
    wait for `closed` or new output until `wait_ms` expires.
    
    ## Verification
    
    - CI pending.
  • [3/4] Add executor-backed RMCP HTTP client (#18583)
    ### Why
    The RMCP layer needs a Streamable HTTP client that can talk either
    directly over `reqwest` or through the executor HTTP runner without
    duplicating MCP session logic higher in the stack. This PR adds that
    client-side transport boundary so remote Streamable HTTP MCP can reuse
    the same RMCP flow as the local path.
    
    ### What
    - Add a shared `rmcp-client/src/streamable_http/` module with:
      - `transport_client.rs` for the local-or-remote transport enum
      - `local_client.rs` for the direct `reqwest` implementation
      - `remote_client.rs` for the executor-backed implementation
      - `common.rs` for the small shared Streamable HTTP helpers
    - Teach `RmcpClient` to build Streamable HTTP transports in either local
    or remote mode while keeping the existing OAuth ownership in RMCP.
    - Translate remote POST, GET, and DELETE session operations into
    executor `http/request` calls.
    - Preserve RMCP session expiry handling and reconnect behavior for the
    remote transport.
    - Add remote transport coverage in
    `rmcp-client/tests/streamable_http_remote.rs` and keep the shared test
    support in `rmcp-client/tests/streamable_http_test_support.rs`.
    
    ### Verification
    - `cargo check -p codex-rmcp-client`
    - online CI
    
    ### Stack
    1. #18581 protocol
    2. #18582 runner
    3. #18583 RMCP client
    4. #18584 manager wiring and local/remote coverage
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • exec-server: require explicit filesystem sandbox cwd (#19046)
    ## Why
    
    This is a cleanup PR for the `PermissionProfile` migration stack. #19016
    fixed remote exec-server sandbox contexts so Docker-backed filesystem
    requests use a request/container `cwd` instead of leaking the local test
    runner `cwd`. That exposed the broader API problem:
    `FileSystemSandboxContext::new(SandboxPolicy)` could still reconstruct
    filesystem permissions by reading the exec-server process cwd with
    `AbsolutePathBuf::current_dir()`.
    
    That made `cwd`-dependent legacy entries, such as `:cwd`,
    `:project_roots`, and relative deny globs, depend on ambient process
    state instead of the request sandbox `cwd`. As later PRs make
    `PermissionProfile` the primary permissions abstraction, sandbox
    contexts should be explicit about whether they carry a request `cwd` or
    are profile-only. Removing the implicit constructor prevents new call
    sites from accidentally rebuilding permissions against the wrong `cwd`.
    
    ## What changed
    
    - Removed `FileSystemSandboxContext::new(SandboxPolicy)`.
    - Kept production callers on explicit constructors:
    `from_legacy_sandbox_policy(..., cwd)`, `from_permission_profile(...)`,
    and `from_permission_profile_with_cwd(...)`.
    - Updated exec-server test helpers to construct `PermissionProfile`
    values directly instead of routing through legacy `SandboxPolicy`
    projections.
    - Updated the environment regression test to use an explicit restricted
    profile with no synthetic `cwd`.
    
    ## Verification
    
    - `cargo test -p codex-exec-server`
    - `just fix -p codex-exec-server`
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/19046).
    * #18288
    * #18287
    * #18286
    * #18285
    * #18284
    * #18283
    * #18282
    * #18281
    * #18280
    * __->__ #19046
  • exec-server: expose arg0 alias root to fs sandbox (#19016)
    ## Why
    
    The post-merge `rust-ci-full` run for #18999 still failed the Ubuntu
    remote `suite::remote_env` sandboxed filesystem tests. That run checked
    out merge commit `ddde50c611e4800cb805f243ed3c50bbafe7d011`, so the arg0
    guard lifetime fix was present.
    
    The Docker-backed failure had two remaining pieces:
    
    - The sandboxed filesystem helper needs to execute Codex through the
    `codex-linux-sandbox` arg0 alias path. The helper sandbox was only
    granting read access to the real Codex executable parent, so the alias
    parent also has to be visible inside the helper sandbox.
    - The remote-env tests were building sandbox contexts with
    `FileSystemSandboxContext::new()`, which captures the local test runner
    cwd. In the Docker remote exec-server, that host checkout path does not
    exist, so spawning the filesystem helper failed with `No such file or
    directory` before the helper could process the request.
    
    ## What Changed
    
    - Track all helper runtime read roots instead of a single root.
    - Add both the real Codex executable parent and the
    `codex-linux-sandbox` alias parent to sandbox readable roots.
    - Avoid sending an unused local cwd in remote filesystem sandbox
    contexts when the permission profile has no cwd-dependent entries.
    - Build the Docker remote-env test sandbox contexts with a cwd path that
    exists inside the container.
    - Add unit coverage for the alias-parent root and remote sandbox cwd
    handling.
    
    ## Verification
    
    - `cargo test -p codex-exec-server`
    - `cargo test -p codex-core
    remote_test_env_sandboxed_read_allows_readable_root`
    - `just fix -p codex-exec-server`
    - `just fix -p codex-core`
  • [2/4] Implement executor HTTP request runner (#18582)
    ### Why
    Remote streamable HTTP MCP needs the executor to perform ordinary HTTP
    requests on the executor side. This keeps network placement aligned with
    `experimental_environment = "remote"` without adding MCP-specific
    executor APIs.
    
    ### What
    - Add an executor-side `http/request` runner backed by `reqwest`.
    - Validate request method and URL scheme, preserving the transport
    boundary at plain HTTP.
    - Return buffered responses for ordinary calls and emit ordered
    `http/request/bodyDelta` notifications for streaming responses.
    - Register the request handler in the exec-server router.
    - Document the runner entrypoint, conversion helpers, body-stream
    bridge, notification sender, timeout behavior, and new integration-test
    helpers.
    - Add exec-server integration tests with the existing websocket harness
    and a local TCP HTTP peer for buffered and streamed responses, with
    comments spelling out what each test proves and its
    setup/exercise/assert phases.
    
    ### Stack
    1. #18581 protocol
    2. #18582 runner
    3. #18583 RMCP client
    4. #18584 manager wiring and local/remote coverage
    
    ### Verification
    - `just fmt`
    - `cargo check -p codex-exec-server -p codex-rmcp-client --tests`
    - `cargo check -p codex-core --test all` compile-only
    - `git diff --check`
    - Online full CI is running from the `full-ci` branch, including the
    remote Rust test job.
    
    Co-authored-by: Codex <noreply@openai.com>
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • exec-server: carry filesystem sandbox profiles (#18276)
    ## Why
    
    The exec-server still needs platform sandbox inputs, but the migration
    should preserve the `PermissionProfile` that produced them. Keeping only
    the derived legacy sandbox map would keep `SandboxPolicy` as the
    effective abstraction and would make full-disk vs. restricted profiles
    harder to preserve as the permissions stack starts round-tripping
    profiles.
    
    `PermissionProfile` entries can also be cwd-sensitive (`:cwd`,
    `:project_roots`, relative globs), so the exec-server must carry the
    request sandbox cwd instead of resolving those entries against the
    long-lived exec-server process cwd.
    
    ## What changed
    
    `FileSystemSandboxContext` now carries `permissions: PermissionProfile`
    plus an optional `cwd`:
    
    - removed `sandboxPolicy`, `sandboxPolicyCwd`,
    `fileSystemSandboxPolicy`, and `additionalPermissions`
    - added `permissions` and `cwd`
    - kept the platform knobs `windowsSandboxLevel`,
    `windowsSandboxPrivateDesktop`, and `useLegacyLandlock`
    
    Core turn and apply-patch paths populate the context from the active
    runtime permissions and request cwd. Exec-server derives platform
    `SandboxPolicy`/`FileSystemSandboxPolicy` at the filesystem boundary,
    adds helper runtime reads there, and rejects cwd-dependent profiles that
    arrive without a cwd.
    
    The legacy `FileSystemSandboxContext::new(SandboxPolicy)` constructor
    now preserves the old workspace-write conversion semantics for
    compatibility tests/callers.
    
    ## Verification
    
    - `cargo test -p codex-exec-server`
    - `cargo test -p codex-exec-server sandbox_cwd -- --nocapture`
    - `cargo test -p codex-exec-server
    sandbox_context_new_preserves_legacy_workspace_write_read_only_subpaths
    -- --nocapture`
    - `cargo test -p codex-core --lib
    file_system_sandbox_context_uses_active_attempt -- --nocapture`
  • Support multiple managed environments (#18401)
    ## Summary
    - refactor EnvironmentManager to own keyed environments with
    default/local lookup helpers
    - keep remote exec-server client creation lazy until exec/fs use
    - preserve disabled agent environment access separately from internal
    local environment access
    
    ## Validation
    - not run (per Codex worktree instruction to avoid tests/builds unless
    requested)
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • fix: fully revert agent identity runtime wiring (#18757)
    ## Summary
    
    This PR fully reverts the previously merged Agent Identity runtime
    integration from the old stack:
    https://github.com/openai/codex/pull/17387/changes
    
    It removes the Codex-side task lifecycle wiring, rollout/session
    persistence, feature flag plumbing, lazy `auth.json` mutation,
    background task auth paths, and request callsite changes introduced by
    that stack.
    
    This leaves the repo in a clean pre-AgentIdentity integration state so
    the follow-up PRs can reintroduce the pieces in smaller reviewable
    layers.
    
    ## Stack
    
    1. This PR: full revert
    2. https://github.com/openai/codex/pull/18871: move Agent Identity
    business logic into a crate
    3. https://github.com/openai/codex/pull/18785: add explicit
    AgentIdentity auth mode and startup task allocation
    4. https://github.com/openai/codex/pull/18811: migrate auth callsites
    through AuthProvider
    
    ## Testing
    
    Tests: targeted Rust checks, cargo-shear, Bazel lock check, and CI.
  • [1/4] Add executor HTTP request protocol (#18581)
    ### Why
    Remote streamable HTTP MCP needs a transport-shaped executor primitive
    before the MCP client can move network I/O to the executor. This layer
    keeps the executor unaware of MCP and gives later PRs an ordered
    streaming surface for response bodies.
    
    ### What
    - Add typed `http/request` and `http/request/bodyDelta` protocol
    payloads.
    - Add executor client helpers for buffered and streamed HTTP responses.
    - Route body-delta notifications to request-scoped streams with sequence
    validation and cleanup when a stream finishes or is dropped.
    - Document the new protocol constants, transport structs, public client
    methods, body-stream lifecycle, and request-scoped routing helpers.
    - Add in-memory JSON-RPC client coverage for streamed HTTP response-body
    notifications, with comments spelling out what the test proves and each
    setup/exercise/assert phase.
    
    ### Stack
    1. #18581 protocol
    2. #18582 runner
    3. #18583 RMCP client
    4. #18584 manager wiring and local/remote coverage
    
    ### Verification
    - `just fmt`
    - `cargo check -p codex-exec-server -p codex-rmcp-client --tests`
    - `cargo check -p codex-core --test all` compile-only
    - `git diff --check`
    - Online full CI is running from the `full-ci` branch, including the
    remote Rust test job.
    
    Co-authored-by: Codex <noreply@openai.com>
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [6/6] Fail exec client operations after disconnect (#18027)
    ## Summary
    - Reject new exec-server client operations once the transport has
    disconnected.
    - Convert pending RPC calls into closed errors instead of synthetic
    server errors.
    - Cover pending read and later write behavior after remote executor
    disconnect.
    
    ## Verification
    - `just fmt`
    - `cargo check -p codex-exec-server`
    
    ## Stack
    ```text
    @  #18027 [6/6] Fail exec client operations after disconnect
    │
    o  #18212 [5/6] Wire executor-backed MCP stdio
    │
    o  #18087 [4/6] Abstract MCP stdio server launching
    │
    o  #18020 [3/6] Add pushed exec process events
    │
    o  #18086 [2/6] Support piped stdin in exec process API
    │
    o  #18085 [1/6] Add MCP server environment config
    │
    o  main
    ```
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • protocol: canonicalize file system permissions (#18274)
    ## Why
    
    `PermissionProfile` needs stable, canonical file-system semantics before
    it can become the primary runtime permissions abstraction. Without a
    canonical form, callers have to keep re-deriving legacy sandbox maps and
    profile comparisons remain lossy or order-dependent.
    
    ## What changed
    
    This adds canonicalization helpers for `FileSystemPermissions` and
    `PermissionProfile`, expands special paths into explicit sandbox
    entries, and updates permission request/conversion paths to consume
    those canonical entries. It also tightens the legacy bridge so root-wide
    write profiles with narrower carveouts are not silently projected as
    full-disk legacy access.
    
    ## Verification
    
    - `cargo test -p codex-protocol
    root_write_with_read_only_child_is_not_full_disk_write -- --nocapture`
    - `cargo test -p codex-sandboxing permission -- --nocapture`
    - `cargo test -p codex-tui permissions -- --nocapture`
  • fix: fix fs sandbox helper for apply_patch (#18296)
    ## Summary
    
    - pass split filesystem sandbox policy/cwd through apply_patch contexts,
    while omitting legacy-equivalent policies to keep payloads small
    - keep the fs helper compatible with legacy Landlock by avoiding helper
    read-root permission expansion in that mode and disabling helper network
    access
    
    ## Root Cause
    
    `d626dc38950fb40a1a5ad0a8ffab2485e3348c53` routed exec-server filesystem
    operations through a sandboxed helper. That path forwarded legacy
    Landlock into a helper policy shape that could require direct
    split-policy enforcement. Sandboxed `apply_patch` hit that edge through
    the filesystem abstraction.
    
    The same 0.121 edit-regression path is consistent with #18354: normal
    writes route through the `apply_patch` filesystem helper, fail under
    sandbox, and then surface the generic retry-without-sandbox prompt.
    
    Fixes #18069
    Fixes #18354
    
    ## Validation
    
    - `cd codex-rs && just fmt`
    - earlier branch validation before merging current `origin/main` and
    dropping the now-separate PATH fix:
      - `cd codex-rs && cargo test -p codex-exec-server`
    - `cd codex-rs && cargo test -p codex-core file_system_sandbox_context`
      - `cd codex-rs && just fix -p codex-exec-server`
      - `cd codex-rs && just fix -p codex-core`
      - `git diff --check`
      - `cd codex-rs && cargo clean`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • exec-server: preserve fs helper runtime env (#18380)
    ## Summary
    - preserve a small fs-helper runtime env allowlist (`PATH`, temp vars)
    instead of launching the sandboxed helper with an empty env
    - add unit coverage for the allowlist and transformed sandbox request
    env
    - add a Linux smoke test that starts the test exec-server with a fake
    `bwrap` on `PATH`, runs a sandboxed fs write through the remote fs
    helper path, and asserts that bwrap path was exercised
    
    ## Validation
    - `cd /tmp/codex-worktrees/fs-helper-env-defaults/codex-rs && export
    PATH=$HOME/code/openai/project/dotslash-gen/bin:$HOME/.local/bin:$PATH
    && bazel test --bes_backend= --bes_results_url=
    //codex-rs/exec-server:exec-server-file_system-test
    --test_filter=sandboxed_file_system_helper_finds_bwrap_on_preserved_path`
    - `cd /tmp/codex-worktrees/fs-helper-env-defaults/codex-rs && export
    PATH=$HOME/code/openai/project/dotslash-gen/bin:$HOME/.local/bin:$PATH
    && bazel test --bes_backend= --bes_results_url=
    //codex-rs/exec-server:exec-server-unit-tests
    --test_filter="helper_env|sandbox_exec_request_carries_helper_env"`
    - earlier on this branch before the smoke-test harness adjustment: `cd
    /tmp/codex-worktrees/fs-helper-env-defaults/codex-rs && export
    PATH=$HOME/code/openai/project/dotslash-gen/bin:$HOME/.local/bin:$PATH
    && bazel test --bes_backend= --bes_results_url=
    //codex-rs/exec-server:all`
    
    Co-authored-by: Codex <noreply@openai.com>
  • [3/6] Add pushed exec process events (#18020)
    ## Summary
    - Add a pushed `ExecProcessEvent` stream alongside retained
    `process/read` output.
    - Publish local and remote output, exit, close, and failure events.
    - Cover the event stream with shared local/remote exec process tests.
    
    ## Testing
    - `cargo check -p codex-exec-server`
    - `cargo check -p codex-rmcp-client`
    - Not run: `cargo test` per repo instruction; CI will cover.
    
    ## Stack
    ```text
    o  #18027 [6/6] Fail exec client operations after disconnect
    │
    o  #18212 [5/6] Wire executor-backed MCP stdio
    │
    o  #18087 [4/6] Abstract MCP stdio server launching
    │
    @  #18020 [3/6] Add pushed exec process events
    │
    o  #18086 [2/6] Support piped stdin in exec process API
    │
    o  #18085 [1/6] Add MCP server environment config
    │
    o  main
    ```
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [2/8] Support piped stdin in exec process API (#18086)
    ## Summary
    - Add an explicit stdin mode to process/start.
    - Keep normal non-interactive exec stdin closed while allowing
    pipe-backed processes.
    
    ## Stack
    ```text
    o  #18027 [8/8] Fail exec client operations after disconnect
    │
    o  #18025 [7/8] Cover MCP stdio tests with executor placement
    │
    o  #18089 [6/8] Wire remote MCP stdio through executor
    │
    o  #18088 [5/8] Add executor process transport for MCP stdio
    │
    o  #18087 [4/8] Abstract MCP stdio server launching
    │
    o  #18020 [3/8] Add pushed exec process events
    │
    @  #18086 [2/8] Support piped stdin in exec process API
    │
    o  #18085 [1/8] Add MCP server environment config
    │
    o  main
    ```
    
    Co-authored-by: Codex <noreply@openai.com>
  • Fix fs/readDirectory to skip broken symlinks (#17907)
    ## Summary
    - Skip directory entries whose metadata lookup fails during
    `fs/readDirectory`
    - Add an exec-server regression test covering a broken symlink beside
    valid entries
    
    ## Testing
    - `just fmt`
    - `cargo test -p codex-exec-server` (started, but dependency/network
    updates stalled before completion in this environment)
  • Remove exec-server fs sandbox request preflight (#17883)
    ## Summary
    - Remove the exec-server-side manual filesystem request path preflight
    before invoking the sandbox helper.
    - Keep sandbox helper policy construction and platform sandbox
    enforcement as the access boundary.
    - Add a portable local+remote regression for writing through an
    explicitly configured alias root.
    - Remove the metadata symlink-escape assertion that depended on the
    deleted manual preflight; no replacement metadata-specific access probe
    is added.
    
    ## Tests
    - `cargo test -p codex-exec-server --lib`
    - `cargo test -p codex-exec-server --test file_system`
    - `git diff --check`
  • [codex] Add symlink flag to fs metadata (#17719)
    Add `is_symlink` to FsMetadata struct.
  • Stabilize exec-server filesystem tests in CI (#17671)
    ## Summary\n- add an exec-server package-local test helper binary that
    can run exec-server and fs-helper flows\n- route exec-server filesystem
    tests through that helper instead of cross-crate codex helper
    binaries\n- stop relying on Bazel-only extra binary wiring for these
    tests\n\n## Testing\n- not run (per repo guidance for codex changes)
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Build remote exec env from exec-server policy (#17216)
    ## Summary
    - add an exec-server `envPolicy` field; when present, the server starts
    from its own process env and applies the shell environment policy there
    - keep `env` as the exact environment for local/embedded starts, but
    make it an overlay for remote unified-exec starts
    - move the shell-environment-policy builder into `codex-config` so Core
    and exec-server share the inherit/filter/set/include behavior
    - overlay only runtime/sandbox/network deltas from Core onto the
    exec-server-derived env
    
    ## Why
    Remote unified exec was materializing the shell env inside Core and
    forwarding the whole map to exec-server, so remote processes could
    inherit the orchestrator machine's `HOME`, `PATH`, etc. This keeps the
    base env on the executor while preserving Core-owned runtime additions
    like `CODEX_THREAD_ID`, unified-exec defaults, network proxy env, and
    sandbox marker env.
    
    ## Validation
    - `just fmt`
    - `git diff --check`
    - `cargo test -p codex-exec-server --lib`
    - `cargo test -p codex-core --lib unified_exec::process_manager::tests`
    - `cargo test -p codex-core --lib exec_env::tests`
    - `cargo test -p codex-core --lib exec_env_tests` (compile-only; filter
    matched 0 tests)
    - `cargo test -p codex-config --lib shell_environment` (compile-only;
    filter matched 0 tests)
    - `just bazel-lock-update`
    
    ## Known local validation issue
    - `just bazel-lock-check` is not runnable in this checkout: it invokes
    `./scripts/check-module-bazel-lock.sh`, which is missing.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • Run exec-server fs operations through sandbox helper (#17294)
    ## Summary
    - run exec-server filesystem RPCs requiring sandboxing through a
    `codex-fs` arg0 helper over stdin/stdout
    - keep direct local filesystem execution for `DangerFullAccess` and
    external sandbox policies
    - remove the standalone exec-server binary path in favor of top-level
    arg0 dispatch/runtime paths
    - add sandbox escape regression coverage for local and remote filesystem
    paths
    
    ## Validation
    - `just fmt`
    - `git diff --check`
    - remote devbox: `cd codex-rs && bazel test --bes_backend=
    --bes_results_url= //codex-rs/exec-server:all` (6/6 passed)
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Fix Windows exec-server output test flake (#17409)
    Problem: The Windows exec-server test command could let separator
    whitespace become part of `echo` output, making the exact
    retained-output assertion flaky.
    
    Solution: Tighten the Windows `cmd.exe` command by placing command
    separators directly after the echoed tokens so stdout remains
    deterministic while preserving the exact assertion.
  • feat: move exec-server ownership (#16344)
    This introduces session-scoped ownership for exec-server so ws
    disconnects no longer immediately kill running remote exec processes,
    and it prepares the protocol for reconnect-based resume.
    - add session_id / resume_session_id to the exec-server initialize
    handshake
      - move process ownership under a shared session registry
    - detach sessions on websocket disconnect and expire them after a TTL
    instead of killing processes immediately (we will resume based on this)
    - allow a new connection to resume an existing session and take over
    notifications/ownership
    - I use UUID to make them not predictable as we don't have auth for now
    - make detached-session expiry authoritative at resume time so teardown
    wins at the TTL boundary
    - reject long-poll process/read calls that get resumed out from under an
    older attachment
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [codex] Support remote exec cwd in TUI startup (#17142)
    When running with remote executor the cwd is the remote path. Today we
    check for existence of a local directory on startup and attempt to load
    config from it.
    
    For remote executors don't do that.
  • Add sandbox support to filesystem APIs (#16751)
    ## Summary
    - add optional `sandboxPolicy` support to the app-server filesystem
    request surface
    - thread sandbox-aware filesystem options through app-server and
    exec-server adapters
    - enforce sandboxed read/write access in the filesystem abstraction with
    focused local and remote coverage
    
    ## Validation
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-exec-server file_system`
    - `cargo test -p codex-app-server suite::v2::fs`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [codex] Migrate apply_patch to executor filesystem (#17027)
    - Migrate apply-patch verification and application internals to use the
    async `ExecutorFileSystem` abstraction from `exec-server`.
    - Convert apply-patch `cwd` handling to `AbsolutePathBuf` through the
    verifier/parser/handler boundary.
    
    Doesn't change how the tool itself works.
  • Make AGENTS.md discovery FS-aware (#15826)
    ## Summary
    - make AGENTS.md discovery and loading fully FS-aware and remove the
    non-FS discover helper
    - migrate remote-aware codex-core tests to use TestEnv workspace setup
    instead of syncing a local workspace copy
    - add AGENTS.md corner-case coverage, including directory fallbacks and
    remote-aware integration coverage
    
    ## Testing
    - cargo test -p codex-core project_doc -- --nocapture
    - cargo test -p codex-core hierarchical_agents -- --nocapture
    - cargo test -p codex-core agents_md -- --nocapture
    - cargo test -p codex-tui status -- --nocapture
    - cargo test -p codex-tui-app-server status -- --nocapture
    - just fix
    - just fmt
    - just bazel-lock-update
    - just bazel-lock-check
    - just argument-comment-lint
    - remote Linux executor tests in progress via scripts/test-remote-env.sh
  • Disable env-bound tools when exec server is none (#16349)
    ## Summary
    - make `CODEX_EXEC_SERVER_URL=none` map to an explicit disabled
    environment mode instead of inferring from a missing URL
    - expose environment capabilities (`exec_enabled`, `filesystem_enabled`)
    so tool building can gate behavior explicitly and future
    multi-environment work has a clearer seam
    - suppress env-backed tools when the relevant capability is unavailable,
    including exec tools, `js_repl`, `apply_patch`, `list_dir`, and
    `view_image`
    - keep handler/runtime backstops so disabled environments still reject
    execution if a tool path somehow bypasses registration
    
    ## Testing
    - `just fmt`
    - `cargo test -p codex-exec-server`
    - `cargo test -p codex-tools
    disabled_environment_omits_environment_backed_tools`
    - `cargo test -p codex-tools
    environment_capabilities_gate_exec_and_filesystem_tools_independently`
    - remote devbox Bazel build via `codex-applied-devbox`:
    `//codex-rs/cli:cli`
  • extract models manager and related ownership from core (#16508)
    ## Summary
    - split `models-manager` out of `core` and add `ModelsManagerConfig`
    plus `Config::to_models_manager_config()` so model metadata paths stop
    depending on `core::Config`
    - move login-owned/auth-owned code out of `core` into `codex-login`,
    move model provider config into `codex-model-provider-info`, move API
    bridge mapping into `codex-api`, move protocol-owned types/impls into
    `codex-protocol`, and move response debug helpers into a dedicated
    `response-debug-context` crate
    - move feedback tag emission into `codex-feedback`, relocate tests to
    the crates that now own the code, and keep broad temporary re-exports so
    this PR avoids a giant import-only rewrite
    
    ## Major moves and decisions
    - created `codex-models-manager` as the owner for model
    cache/catalog/config/model info logic, including the new
    `ModelsManagerConfig` struct
    - created `codex-model-provider-info` as the owner for provider config
    parsing/defaults and kept temporary `codex-login`/`codex-core`
    re-exports for old import paths
    - moved `api_bridge` error mapping + `CoreAuthProvider` into
    `codex-api`, while `codex-login::api_bridge` temporarily re-exports
    those symbols and keeps the `auth_provider_from_auth` wrapper
    - moved `auth_env_telemetry` and `provider_auth` ownership to
    `codex-login`
    - moved `CodexErr` ownership to `codex-protocol::error`, plus
    `StreamOutput`, `bytes_to_string_smart`, and network policy helpers to
    protocol-owned modules
    - created `codex-response-debug-context` for
    `extract_response_debug_context`, `telemetry_transport_error_message`,
    and related response-debug plumbing instead of leaving that behavior in
    `core`
    - moved `FeedbackRequestTags`, `emit_feedback_request_tags`, and
    `emit_feedback_request_tags_with_auth_env` to `codex-feedback`
    - deferred removal of temporary re-exports and the mechanical import
    rewrites to a stacked follow-up PR so this PR stays reviewable
    
    ## Test moves
    - moved auth refresh coverage from `core/tests/suite/auth_refresh.rs` to
    `login/tests/suite/auth_refresh.rs`
    - moved text encoding coverage from
    `core/tests/suite/text_encoding_fix.rs` to
    `protocol/src/exec_output_tests.rs`
    - moved model info override coverage from
    `core/tests/suite/model_info_overrides.rs` to
    `models-manager/src/model_info_overrides_tests.rs`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • feat: use ProcessId in exec-server (#15866)
    Use a full struct for the ProcessId to increase readability and make it
    easier in the future to make it evolve if needed
  • feat: exec-server prep for unified exec (#15691)
    This PR partially rebase `unified_exec` on the `exec-server` and adapt
    the `exec-server` accordingly.
    
    ## What changed in `exec-server`
    
    1. Replaced the old "broadcast-driven; process-global" event model with
    process-scoped session events. The goal is to be able to have dedicated
    handler for each process.
    2. Add to protocol contract to support explicit lifecycle status and
    stream ordering:
    - `WriteResponse` now returns `WriteStatus` (Accepted, UnknownProcess,
    StdinClosed, Starting) instead of a bool.
      - Added seq fields to output/exited notifications.
      - Added terminal process/closed notification.
    3. Demultiplexed remote notifications into per-process channels. Same as
    for the event sys
    4. Local and remote backends now both implement ExecBackend.
    5. Local backend wraps internal process ID/operations into per-process
    ExecProcess objects.
    6. Remote backend registers a session channel before launch and
    unregisters on failed launch.
    
    ## What changed in `unified_exec`
    
    1. Added unified process-state model and backend-neutral process
    wrapper. This will probably disappear in the future, but it makes it
    easier to keep the work flowing on both side.
    - `UnifiedExecProcess` now handles both local PTY sessions and remote
    exec-server processes through a shared `ProcessHandle`.
    - Added `ProcessState` to track has_exited, exit_code, and terminal
    failure message consistently across backends.
    2. Routed write and lifecycle handling through process-level methods.
    
    ## Some rationals
    
    1. The change centralizes execution transport in exec-server while
    preserving policy and orchestration ownership in core, avoiding
    duplicated launch approval logic. This comes from internal discussion.
    2. Session-scoped events remove coupling/cross-talk between processes
    and make stream ordering and terminal state explicit (seq, closed,
    failed).
    3. The failure-path surfacing (remote launch failures, write failures,
    transport disconnects) makes command tool output and cleanup behavior
    deterministic
    
    ## Follow-ups:
    * Unify the concept of thread ID behind an obfuscated struct
    * FD handling
    * Full zsh-fork compatibility
    * Full network sandboxing compatibility
    * Handle ws disconnection
  • Add cached environment manager for exec server URL (#15785)
    Add environment manager that is a singleton and is created early in
    app-server (before skill manager, before config loading).
    
    Use an environment variable to point to a running exec server.
  • Split exec process into local and remote implementations (#15233)
    ## Summary
    - match the exec-process structure to filesystem PR #15232
    - expose `ExecProcess` on `Environment`
    - make `LocalProcess` the real implementation and `RemoteProcess` a thin
    network proxy over `ExecServerClient`
    - make `ProcessHandler` a thin RPC adapter delegating to `LocalProcess`
    - add a shared local/remote process test
    
    ## Validation
    - `just fmt`
    - `CARGO_TARGET_DIR=~/.cache/cargo-target/codex cargo test -p
    codex-exec-server`
    - `just fix -p codex-exec-server`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Refactor ExecServer filesystem split between local and remote (#15232)
    For each feature we have:
    1. Trait exposed on environment
    2. **Local Implementation** of the trait
    3. Remote implementation that uses the client to proxy via network
    4. Handler implementation that handles PRC requests and calls into
    **Local Implementation**
  • Move terminal module to terminal-detection crate (#15216)
    - Move core/src/terminal.rs and its tests into a standalone
    terminal-detection workspace crate.
    - Update direct consumers to depend on codex-terminal-detection and
    import terminal APIs directly.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add exec-server exec RPC implementation (#15090)
    Stacked PR 2/3, based on the stub PR.
    
    Adds the exec RPC implementation and process/event flow in exec-server
    only.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add experimental exec server URL handling (#15196)
    Add a config and attempt to start the server.
  • Move environment abstraction into exec server (#15125)
    The idea is that codex-exec exposes an Environment struct with services
    on it. Each of those is a trait.
    
    Depending on construction parameters passed to Environment they are
    either backed by local or remote server but core doesn't see these
    differences.
  • Remove stdio transport from exec server (#15119)
    Summary
    - delete the deprecated stdio transport plumbing from the exec server
    stack
    - add a basic `exec_server()` harness plus test utilities to start a
    server, send requests, and await events
    - refresh exec-server dependencies, configs, and documentation to
    reflect the new flow
    
    Testing
    - Not run (not requested)
    
    ---------
    
    Co-authored-by: starr-openai <starr@openai.com>
    Co-authored-by: Codex <noreply@openai.com>
  • Add exec-server stub server and protocol docs (#15089)
    Stacked PR 1/3.
    
    This is the initialize-only exec-server stub slice: binary/client
    scaffolding and protocol docs, without exec/filesystem implementation.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • refactor: delete exec-server and move execve wrapper into shell-escalation (#12632)
    ## Why
    
    We already plan to remove the shell-tool MCP path, and doing that
    cleanup first makes the follow-on `shell-escalation` work much simpler.
    
    This change removes the last remaining reason to keep
    `codex-rs/exec-server` around by moving the `codex-execve-wrapper`
    binary and shared shell test fixtures to the crates/tests that now own
    that functionality.
    
    ## What Changed
    
    ### Delete `codex-rs/exec-server`
    
    - Remove the `exec-server` crate, including the MCP server binary,
    MCP-specific modules, and its test support/test suite
    - Remove `exec-server` from the `codex-rs` workspace and update
    `Cargo.lock`
    
    ### Move `codex-execve-wrapper` into `codex-rs/shell-escalation`
    
    - Move the wrapper implementation into `shell-escalation`
    (`src/unix/execve_wrapper.rs`)
    - Add the `codex-execve-wrapper` binary entrypoint under
    `shell-escalation/src/bin/`
    - Update `shell-escalation` exports/module layout so the wrapper
    entrypoint is hosted there
    - Move the wrapper README content from `exec-server` to
    `shell-escalation/README.md`
    
    ### Move shared shell test fixtures to `app-server`
    
    - Move the DotSlash `bash`/`zsh` test fixtures from
    `exec-server/tests/suite/` to `app-server/tests/suite/`
    - Update `app-server` zsh-fork tests to reference the new fixture paths
    
    ### Keep `shell-tool-mcp` as a shell-assets package
    
    - Update `.github/workflows/shell-tool-mcp.yml` packaging so the npm
    artifact contains only patched Bash/Zsh payloads (no Rust binaries)
    - Update `shell-tool-mcp/package.json`, `shell-tool-mcp/src/index.ts`,
    and docs to reflect the shell-assets-only package shape
    - `shell-tool-mcp-ci.yml` does not need changes because it is already
    JS-only
    
    ## Verification
    
    - `cargo shear`
    - `cargo clippy -p codex-shell-escalation --tests`
    - `just clippy`
  • refactor: normalize unix module layout for exec-server and shell-escalation (#12556)
    ## Why
    Shell execution refactoring in `exec-server` had become split between
    duplicated code paths, which blocked a clean introduction of the new
    reusable shell escalation flow. This commit creates a dedicated
    foundation crate so later shell tooling changes can share one
    implementation.
    
    ## What changed
    - Added the `codex-shell-escalation` crate and moved the core escalation
    pieces (`mcp` protocol/socket/session flow, policy glue) that were
    previously in `exec-server` into it.
    - Normalized `exec-server` Unix structure under a dedicated `unix`
    module layout and kept non-Unix builds narrow.
    - Wired crate/build metadata so `shell-escalation` is a first-class
    workspace dependency for follow-on integration work.
    
    ## Verification
    - Built and linted the stack at this commit point with `just clippy`.
    
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/12556).
    * #12584
    * #12583
    * __->__ #12556
  • refactor: decouple MCP policy construction from escalate server (#12555)
    ## Why
    The current escalate path in `codex-rs/exec-server` still had policy
    creation coupled to MCP details, which makes it hard to reuse the shell
    execution flow outside the MCP server. This change is part of a broader
    goal to split MCP-specific behavior from shared escalation execution so
    other handlers (for example a future `ShellCommandHandler`) can reuse it
    without depending on MCP request context types.
    
    ## What changed
    - Added a new `EscalationPolicyFactory` abstraction in `mcp.rs`:
      - `crate`-relative path: `codex-rs/exec-server/src/posix/mcp.rs`
    -
    https://github.com/openai/codex/blob/main/codex-rs/exec-server/src/posix/mcp.rs#L87-L107
    - Made `run_escalate_server` in `mcp.rs` accept a policy factory instead
    of constructing `McpEscalationPolicy` directly.
    -
    https://github.com/openai/codex/blob/main/codex-rs/exec-server/src/posix/mcp.rs#L178-L201
    - Introduced `McpEscalationPolicyFactory` that stores MCP-only state
    (`RequestContext`, `preserve_program_paths`) and implements the new
    trait.
    -
    https://github.com/openai/codex/blob/main/codex-rs/exec-server/src/posix/mcp.rs#L100-L117
    - Updated `shell()` to pass a `McpEscalationPolicyFactory` instance into
    `run_escalate_server`, so the server remains the MCP-specific wiring
    layer.
    -
    https://github.com/openai/codex/blob/main/codex-rs/exec-server/src/posix/mcp.rs#L163-L170
    
    ## Verification
    - Build and test execution was not re-run in this pass; changes are
    limited to `mcp.rs` and preserve the existing escalation flow semantics
    by only extracting policy construction behind a factory.
    
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/12555).
    * #12556
    * __->__ #12555
  • chore: remove codex-core public protocol/shell re-exports (#12432)
    ## Why
    
    `codex-rs/core/src/lib.rs` re-exported a broad set of types and modules
    from `codex-protocol` and `codex-shell-command`. That made it easy for
    workspace crates to import those APIs through `codex-core`, which in
    turn hides dependency edges and makes it harder to reduce compile-time
    coupling over time.
    
    This change removes those public re-exports so call sites must import
    from the source crates directly. Even when a crate still depends on
    `codex-core` today, this makes dependency boundaries explicit and
    unblocks future work to drop `codex-core` dependencies where possible.
    
    ## What Changed
    
    - Removed public re-exports from `codex-rs/core/src/lib.rs` for:
    - `codex_protocol::protocol` and related protocol/model types (including
    `InitialHistory`)
      - `codex_protocol::config_types` (`protocol_config_types`)
    - `codex_shell_command::{bash, is_dangerous_command, is_safe_command,
    parse_command, powershell}`
    - Migrated workspace Rust call sites to import directly from:
      - `codex_protocol::protocol`
      - `codex_protocol::config_types`
      - `codex_protocol::models`
      - `codex_shell_command`
    - Added explicit `Cargo.toml` dependencies (`codex-protocol` /
    `codex-shell-command`) in crates that now import those crates directly.
    - Kept `codex-core` internal modules compiling by using `pub(crate)`
    aliases in `core/src/lib.rs` (internal-only, not part of the public
    API).
    - Updated the two utility crates that can already drop a `codex-core`
    dependency edge entirely:
      - `codex-utils-approval-presets`
      - `codex-utils-cli`
    
    ## Verification
    
    - `cargo test -p codex-utils-approval-presets`
    - `cargo test -p codex-utils-cli`
    - `cargo check --workspace --all-targets`
    - `just clippy`
  • Refactor network approvals to host/protocol/port scope (#12140)
    ## Summary
    Simplify network approvals by removing per-attempt proxy correlation and
    moving to session-level approval dedupe keyed by (host, protocol, port).
    Instead of encoding attempt IDs into proxy credentials/URLs, we now
    treat approvals as a destination policy decision.
    
    - Concurrent calls to the same destination share one approval prompt.
    - Different destinations (or same host on different ports) get separate
    prompts.
    - Allow once approves the current queued request group only.
    - Allow for session caches that (host, protocol, port) and auto-allows
    future matching requests.
    - Never policy continues to deny without prompting.
    
    Example:
    - 3 calls: 
      - a.com (line 443)
      - b.com (line 443)
      - a.com (line 443)
    => 2 prompts total (a, b), second a waits on the first decision.
    - a.com:80 is treated separately from a.com line 443
    
    ## Testing
    - `just fmt` (in `codex-rs`)
    - `cargo test -p codex-core tools::network_approval::tests`
    - `cargo test -p codex-core` (unit tests pass; existing
    integration-suite failures remain in this environment)
  • feat(core): add structured network approval plumbing and policy decision model (#11672)
    ### Description
    #### Summary
    Introduces the core plumbing required for structured network approvals
    
    #### What changed
    - Added structured network policy decision modeling in core.
    - Added approval payload/context types needed for network approval
    semantics.
    - Wired shell/unified-exec runtime plumbing to consume structured
    decisions.
    - Updated related core error/event surfaces for structured handling.
    - Updated protocol plumbing used by core approval flow.
    - Included small CLI debug sandbox compatibility updates needed by this
    layer.
    
    #### Why
    establishes the minimal backend foundation for network approvals without
    yet changing high-level orchestration or TUI behavior.
    
    #### Notes
    - Behavior remains constrained by existing requirements/config gating.
    - Follow-up PRs in the stack handle orchestration, UX, and app-server
    integration.
    
    ---------
    
    Co-authored-by: Codex <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
  • feat(shell-tool-mcp): add patched zsh build pipeline (#11668)
    ## Summary
    - add `shell-tool-mcp/patches/zsh-exec-wrapper.patch` against upstream
    zsh `77045ef899e53b9598bebc5a41db93a548a40ca6`
    - add `zsh-linux` and `zsh-darwin` jobs to
    `.github/workflows/shell-tool-mcp.yml`
    - stage zsh binaries under `artifacts/vendor/<target>/zsh/<variant>/zsh`
    - include zsh artifact jobs in `package.needs`
    - mark staged zsh binaries executable during packaging
    
    ## Notes
    - zsh source is cloned from `https://git.code.sf.net/p/zsh/code`
    - workflow pins zsh commit `77045ef899e53b9598bebc5a41db93a548a40ca6`
    - zsh build runs `./Util/preconfig` before `./configure`
    
    ## Validation
    - parsed workflow YAML locally (`yaml-ok`)
    - validated zsh patch applies cleanly with `git apply --check` on a
    fresh zsh clone