Commit Graph

7 Commits

  • linux-sandbox: use standalone bundled bwrap (#21255)
    **Summary**
    - Add `codex-bwrap`, a standalone `bwrap` binary built from the existing
    vendored bubblewrap sources.
    - Remove the linked vendored bwrap path from `codex-linux-sandbox`;
    runtime now prefers system `bwrap` and falls back to bundled
    `codex-resources/bwrap`.
    - Add bundled SHA-256 verification with missing/all-zero digest as the
    dev-mode skip value, then exec the verified file through
    `/proc/self/fd`.
    - Keep `launcher.rs` focused on choosing and dispatching the preferred
    launcher. Bundled lookup, digest verification, and bundled exec now live
    in `linux-sandbox/src/bundled_bwrap.rs`; Bazel runfiles lookup lives in
    `linux-sandbox/src/bazel_bwrap.rs`; shared argv/fd exec helpers live in
    `linux-sandbox/src/exec_util.rs`.
    - Teach Bazel tests to surface the Bazel-built `//codex-rs/bwrap:bwrap`
    through `CARGO_BIN_EXE_bwrap`; `codex-linux-sandbox` only honors that
    fallback in debug Bazel runfiles environments so release/user runtime
    lookup stays tied to `codex-resources/bwrap`.
    - Allow `codex-exec-server` filesystem helpers to preserve just the
    Bazel bwrap/runfiles variables they need in debug Bazel builds, since
    those helpers intentionally rebuild a small environment before spawning
    `codex-linux-sandbox`.
    - Verify the Bazel bwrap target in Linux release CI with a build-only
    check. Running `bwrap --version` is too strong for GitHub runners
    because bubblewrap still attempts namespace setup there.
    
    **Verification**
    - Latest update: `cargo test -p codex-linux-sandbox`
    - Latest update: `just fix -p codex-linux-sandbox`
    - `cargo check --target x86_64-unknown-linux-gnu -p codex-linux-sandbox`
    could not run locally because this macOS machine does not have
    `x86_64-linux-gnu-gcc`; GitHub Linux Bazel CI is expected to cover the
    Linux-only modules.
    - Earlier in this PR: `cargo test -p codex-bwrap`
    - Earlier in this PR: `cargo test -p codex-exec-server`
    - Earlier in this PR: `cargo check --release -p codex-exec-server`
    - Earlier in this PR: `just fix -p codex-linux-sandbox -p
    codex-exec-server`
    - Earlier in this PR: `bazel test --nobuild
    //codex-rs/linux-sandbox:linux-sandbox-all-test
    //codex-rs/core:core-all-test
    //codex-rs/exec-server:exec-server-file_system-test
    //codex-rs/app-server:app-server-all-test` (analysis completed; Bazel
    then refuses to run tests under `--nobuild`)
    - Earlier in this PR: `bazel build --nobuild //codex-rs/bwrap:bwrap`
    - Prior to this update: `just bazel-lock-update`, `just
    bazel-lock-check`, and YAML parse check for
    `.github/workflows/bazel.yml`
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/21255).
    * #21257
    * #21256
    * __->__ #21255
  • Stabilize exec-server filesystem tests in CI (#17671)
    ## Summary\n- add an exec-server package-local test helper binary that
    can run exec-server and fs-helper flows\n- route exec-server filesystem
    tests through that helper instead of cross-crate codex helper
    binaries\n- stop relying on Bazel-only extra binary wiring for these
    tests\n\n## Testing\n- not run (per repo guidance for codex changes)
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Run exec-server fs operations through sandbox helper (#17294)
    ## Summary
    - run exec-server filesystem RPCs requiring sandboxing through a
    `codex-fs` arg0 helper over stdin/stdout
    - keep direct local filesystem execution for `DangerFullAccess` and
    external sandbox policies
    - remove the standalone exec-server binary path in favor of top-level
    arg0 dispatch/runtime paths
    - add sandbox escape regression coverage for local and remote filesystem
    paths
    
    ## Validation
    - `just fmt`
    - `git diff --check`
    - remote devbox: `cd codex-rs && bazel test --bes_backend=
    --bes_results_url= //codex-rs/exec-server:all` (6/6 passed)
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add exec-server stub server and protocol docs (#15089)
    Stacked PR 1/3.
    
    This is the initialize-only exec-server stub slice: binary/client
    scaffolding and protocol docs, without exec/filesystem implementation.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • refactor: delete exec-server and move execve wrapper into shell-escalation (#12632)
    ## Why
    
    We already plan to remove the shell-tool MCP path, and doing that
    cleanup first makes the follow-on `shell-escalation` work much simpler.
    
    This change removes the last remaining reason to keep
    `codex-rs/exec-server` around by moving the `codex-execve-wrapper`
    binary and shared shell test fixtures to the crates/tests that now own
    that functionality.
    
    ## What Changed
    
    ### Delete `codex-rs/exec-server`
    
    - Remove the `exec-server` crate, including the MCP server binary,
    MCP-specific modules, and its test support/test suite
    - Remove `exec-server` from the `codex-rs` workspace and update
    `Cargo.lock`
    
    ### Move `codex-execve-wrapper` into `codex-rs/shell-escalation`
    
    - Move the wrapper implementation into `shell-escalation`
    (`src/unix/execve_wrapper.rs`)
    - Add the `codex-execve-wrapper` binary entrypoint under
    `shell-escalation/src/bin/`
    - Update `shell-escalation` exports/module layout so the wrapper
    entrypoint is hosted there
    - Move the wrapper README content from `exec-server` to
    `shell-escalation/README.md`
    
    ### Move shared shell test fixtures to `app-server`
    
    - Move the DotSlash `bash`/`zsh` test fixtures from
    `exec-server/tests/suite/` to `app-server/tests/suite/`
    - Update `app-server` zsh-fork tests to reference the new fixture paths
    
    ### Keep `shell-tool-mcp` as a shell-assets package
    
    - Update `.github/workflows/shell-tool-mcp.yml` packaging so the npm
    artifact contains only patched Bash/Zsh payloads (no Rust binaries)
    - Update `shell-tool-mcp/package.json`, `shell-tool-mcp/src/index.ts`,
    and docs to reflect the shell-assets-only package shape
    - `shell-tool-mcp-ci.yml` does not need changes because it is already
    JS-only
    
    ## Verification
    
    - `cargo shear`
    - `cargo clippy -p codex-shell-escalation --tests`
    - `just clippy`
  • feat: add support for building with Bazel (#8875)
    This PR configures Codex CLI so it can be built with
    [Bazel](https://bazel.build) in addition to Cargo. The `.bazelrc`
    includes configuration so that remote builds can be done using
    [BuildBuddy](https://www.buildbuddy.io).
    
    If you are familiar with Bazel, things should work as you expect, e.g.,
    run `bazel test //... --keep-going` to run all the tests in the repo,
    but we have also added some new aliases in the `justfile` for
    convenience:
    
    - `just bazel-test` to run tests locally
    - `just bazel-remote-test` to run tests remotely (currently, the remote
    build is for x86_64 Linux regardless of your host platform). Note we are
    currently seeing the following test failures in the remote build, so we
    still need to figure out what is happening here:
    
    ```
    failures:
        suite::compact::manual_compact_twice_preserves_latest_user_messages
        suite::compact_resume_fork::compact_resume_after_second_compaction_preserves_history
        suite::compact_resume_fork::compact_resume_and_fork_preserve_model_history_view
    ```
    
    - `just build-for-release` to build release binaries for all
    platforms/architectures remotely
    
    To setup remote execution:
    - [Create a buildbuddy account](https://app.buildbuddy.io/) (OpenAI
    employees should also request org access at
    https://openai.buildbuddy.io/join/ with their `@openai.com` email
    address.)
    - [Copy your API key](https://app.buildbuddy.io/docs/setup/) to
    `~/.bazelrc` (add the line `build
    --remote_header=x-buildbuddy-api-key=YOUR_KEY`)
    - Use `--config=remote` in your `bazel` invocations (or add `common
    --config=remote` to your `~/.bazelrc`, or use the `just` commands)
    
    ## CI
    
    In terms of CI, this PR introduces `.github/workflows/bazel.yml`, which
    uses Bazel to run the tests _locally_ on Mac and Linux GitHub runners
    (we are working on supporting Windows, but that is not ready yet). Note
    that the failures we are seeing in `just bazel-remote-test` do not occur
    on these GitHub CI jobs, so everything in `.github/workflows/bazel.yml`
    is green right now.
    
    The `bazel.yml` uses extra config in `.github/workflows/ci.bazelrc` so
    that macOS CI jobs build _remotely_ on Linux hosts (using the
    `docker://docker.io/mbolin491/codex-bazel` Docker image declared in the
    root `BUILD.bazel`) using cross-compilation to build the macOS
    artifacts. Then these artifacts are downloaded locally to GitHub's macOS
    runner so the tests can be executed natively. This is the relevant
    config that enables this:
    
    ```
    common:macos --config=remote
    common:macos --strategy=remote
    common:macos --strategy=TestRunner=darwin-sandbox,local
    ```
    
    Because of the remote caching benefits we get from BuildBuddy, these new
    CI jobs can be extremely fast! For example, consider these two jobs that
    ran all the tests on Linux x86_64:
    
    - Bazel 1m37s
    https://github.com/openai/codex/actions/runs/20861063212/job/59940545209?pr=8875
    - Cargo 9m20s
    https://github.com/openai/codex/actions/runs/20861063192/job/59940559592?pr=8875
    
    For now, we will continue to run both the Bazel and Cargo jobs for PRs,
    but once we add support for Windows and running Clippy, we should be
    able to cutover to using Bazel exclusively for PRs, which should still
    speed things up considerably. We will probably continue to run the Cargo
    jobs post-merge for commits that land on `main` as a sanity check.
    
    Release builds will also continue to be done by Cargo for now.
    
    Earlier attempt at this PR: https://github.com/openai/codex/pull/8832
    Earlier attempt to add support for Buck2, now abandoned:
    https://github.com/openai/codex/pull/8504
    
    ---------
    
    Co-authored-by: David Zbarsky <dzbarsky@gmail.com>
    Co-authored-by: Michael Bolin <mbolin@openai.com>