Commit Graph

11 Commits

  • ci: run SDK tests with a Bazel-built codex (#16046)
    ## Why
    
    Before this change, the SDK CI job built `codex` with Cargo before
    running the TypeScript package tests. That step has been getting more
    expensive as the Rust workspace grows, while the repo already has a
    Bazel-backed build path for the CLI.
    
    The SDK tests also need a normal executable path they can spawn
    repeatedly. Moving the job to Bazel exposed an extra CI detail: a plain
    `bazel-bin/...` lookup is not reliable under the Linux config because
    top-level outputs may stay remote and the wrapper emits status lines
    around `cquery` output.
    
    ## What Changed
    
    - taught `sdk/typescript/tests/testCodex.ts` to honor `CODEX_EXEC_PATH`
    before falling back to the local Cargo-style `target/debug/codex` path
    - added `--remote-download-toplevel` to
    `.github/scripts/run-bazel-ci.sh` so workflows can force Bazel to
    materialize top-level outputs on disk after a build
    - switched `.github/workflows/sdk.yml` from `cargo build --bin codex` to
    the shared Bazel CI setup and `//codex-rs/cli:codex` build target
    - changed the SDK workflow to resolve the built CLI with wrapper-backed
    `cquery --output=files`, stage the binary into
    `${GITHUB_WORKSPACE}/.tmp/sdk-ci/codex`, and point the SDK tests at that
    path via `CODEX_EXEC_PATH`
    - kept the warm-up step before Jest and the Bazel repository-cache save
    step
    
    ## Verification
    
    - `bash -n .github/scripts/run-bazel-ci.sh`
    - `./.github/scripts/run-bazel-ci.sh -- cquery --output=files --
    //codex-rs/cli:codex | grep -E '^(/|bazel-out/)' | tail -n 1`
    - `./.github/scripts/run-bazel-ci.sh --remote-download-toplevel -- build
    --build_metadata=TAG_job=sdk -- //codex-rs/cli:codex`
    - `CODEX_EXEC_PATH="$PWD/.tmp/sdk-ci/codex" pnpm --dir sdk/typescript
    test --runInBand`
    - `pnpm --dir sdk/typescript lint`
  • ci: add Bazel clippy workflow for codex-rs (#15955)
    ## Why
    `bazel.yml` already builds and tests the Bazel graph, but `rust-ci.yml`
    still runs `cargo clippy` separately. This PR starts the transition to a
    Bazel-backed lint lane for `codex-rs` so we can eventually replace the
    duplicate Rust build, test, and lint work with Bazel while explicitly
    keeping the V8 Bazel path out of scope for now.
    
    To make that lane practical, the workflow also needs to look like the
    Bazel job we already trust. That means sharing the common Bazel setup
    and invocation logic instead of hand-copying it, and covering the arm64
    macOS path in addition to Linux.
    
    Landing the workflow green also required fixing the first lint findings
    that Bazel surfaced and adding the matching local entrypoint.
    
    ## What changed
    - add a reusable `build:clippy` config to `.bazelrc` and export
    `codex-rs/clippy.toml` from `codex-rs/BUILD.bazel` so Bazel can run the
    repository's existing Clippy policy
    - add `just bazel-clippy` so the local developer entrypoint matches the
    new CI lane
    - extend `.github/workflows/bazel.yml` with a dedicated Bazel clippy job
    for `codex-rs`, scoped to `//codex-rs/... -//codex-rs/v8-poc:all`
    - run that clippy job on Linux x64 and arm64 macOS
    - factor the shared Bazel workflow setup into
    `.github/actions/setup-bazel-ci/action.yml` and the shared Bazel
    invocation logic into `.github/scripts/run-bazel-ci.sh` so the clippy
    and build/test jobs stay aligned
    - fix the first Bazel-clippy findings needed to keep the lane green,
    including the cross-target `cmsghdr::cmsg_len` normalization in
    `codex-rs/shell-escalation/src/unix/socket.rs` and the no-`voice-input`
    dead-code warnings in `codex-rs/tui` and `codex-rs/tui_app_server`
    
    ## Verification
    - `just bazel-clippy`
    - `RUNNER_OS=macOS ./.github/scripts/run-bazel-ci.sh -- build
    --config=clippy --build_metadata=COMMIT_SHA=local-check
    --build_metadata=TAG_job=clippy -- //codex-rs/...
    -//codex-rs/v8-poc:all`
    - `bazel build --config=clippy
    //codex-rs/shell-escalation:shell-escalation`
    - `CARGO_TARGET_DIR=/tmp/codex4-shell-escalation-test cargo test -p
    codex-shell-escalation`
    - `ruby -e 'require "yaml";
    YAML.load_file(".github/workflows/bazel.yml");
    YAML.load_file(".github/actions/setup-bazel-ci/action.yml")'`
    
    ## Notes
    - `CARGO_TARGET_DIR=/tmp/codex4-tui-app-server-test cargo test -p
    codex-tui-app-server` still hits existing guardian-approvals test and
    snapshot failures unrelated to this PR's Bazel-clippy changes.
    
    Related: #15954
  • fix: keep zsh-fork release assets after removing shell-tool-mcp (#15644)
    ## Why
    
    `shell-tool-mcp` and the Bash fork are no longer needed, but the patched
    zsh fork is still relevant for shell escalation and for the
    DotSlash-backed zsh-fork integration tests.
    
    Deleting the old `shell-tool-mcp` workflow also deleted the only
    pipeline that rebuilt those patched zsh binaries. This keeps the package
    removal, while preserving a small release path that can be reused
    whenever `codex-rs/shell-escalation/patches/zsh-exec-wrapper.patch`
    changes.
    
    ## What changed
    
    - removed the `shell-tool-mcp` workspace package, its npm
    packaging/release jobs, the Bash test fixture, and the remaining
    Bash-specific compatibility wiring
    - deleted the old `.github/workflows/shell-tool-mcp.yml` and
    `.github/workflows/shell-tool-mcp-ci.yml` workflows now that their
    responsibilities have been replaced or removed
    - kept the zsh patch under
    `codex-rs/shell-escalation/patches/zsh-exec-wrapper.patch` and updated
    the `codex-rs/shell-escalation` docs/code to describe the zsh-based flow
    directly
    - added `.github/workflows/rust-release-zsh.yml` to build only the three
    zsh binaries that `codex-rs/app-server/tests/suite/zsh` needs today:
      - `aarch64-apple-darwin` on `macos-15`
      - `x86_64-unknown-linux-musl` on `ubuntu-24.04`
      - `aarch64-unknown-linux-musl` on `ubuntu-24.04`
    - extracted the shared zsh build/smoke-test/stage logic into
    `.github/scripts/build-zsh-release-artifact.sh`, made that helper
    directly executable, and now invoke it directly from the workflow so the
    Linux and macOS jobs only keep the OS-specific setup in YAML
    - wired those standalone `codex-zsh-*.tar.gz` assets into
    `rust-release.yml` and added `.github/dotslash-zsh-config.json` so
    releases also publish a `codex-zsh` DotSlash file
    - updated the checked-in `codex-rs/app-server/tests/suite/zsh` fixture
    comments to explain that new releases come from the standalone zsh
    assets, while the checked-in fixture remains pinned to the latest
    historical release until a newer zsh artifact is published
    - tightened a couple of follow-on cleanups in
    `codex-rs/shell-escalation`: the `ExecParams::command` comment now
    describes the shell `-c`/`-lc` string more clearly, and the README now
    points at the same `git.code.sf.net` zsh source URL that the workflow
    uses
    
    ## Testing
    
    - `cargo test -p codex-shell-escalation`
    - `just argument-comment-lint`
    - `bash -n .github/scripts/build-zsh-release-artifact.sh`
    - attempted `cargo test -p codex-core`; unrelated existing failures
    remain, but the touched `tools::runtimes::shell::unix_escalation::*`
    coverage passed during that run
  • V8 Bazel Build (#15021)
    Alternative approach, we use rusty_v8 for all platforms that its
    predefined, but lets build from source a musl v8 version with bazel for
    x86 and aarch64 only. We would need to release this on github and then
    use the release.
  • Fix linux-musl release link failures caused by glibc-only libcap artifacts (#11556)
    Problem:
    The `aarch64-unknown-linux-musl` release build was failing at link time
    with
    `/usr/bin/ld: cannot find -lcap` while building binaries that
    transitively pull
    in `codex-linux-sandbox`.
    
    Why this is the right fix:
    `codex-linux-sandbox` compiles vendored bubblewrap and links `libcap`.
    In the
    musl jobs, we were installing distro `libcap-dev`, which provides
    host/glibc
    artifacts. That is not a valid source of target-compatible static libcap
    for
    musl cross-linking, so the fix is to produce a target-compatible libcap
    inside
    the musl tool bootstrap and point pkg-config at it.
    
    This also closes the CI coverage gap that allowed this to slip through:
    the
    `rust-ci.yml` matrix did not exercise `aarch64-unknown-linux-musl` in
    `release`
    mode. Adding that target/profile combination to CI is the right
    regression
    barrier for this class of failure.
    
    What changed:
    - Updated `.github/scripts/install-musl-build-tools.sh` to install
    tooling
      needed to fetch/build libcap sources (`curl`, `xz-utils`, certs).
    - Added deterministic libcap bootstrap in the musl tool root:
      - download `libcap-2.75` from kernel.org
      - verify SHA256
      - build with the target musl compiler (`*-linux-musl-gcc`)
      - stage `libcap.a` and headers under the target tool root
      - generate a target-scoped `libcap.pc`
    - Exported target `PKG_CONFIG_PATH` so builds resolve the staged musl
    libcap
      instead of host pkg-config/lib paths.
    - Updated `.github/workflows/rust-ci.yml` to add a `release` matrix
    entry for
      `aarch64-unknown-linux-musl` on the ARM runner.
    - Updated `.github/workflows/rust-ci.yml` to set
    `CARGO_PROFILE_RELEASE_LTO=thin` for `release` matrix entries (and keep
    `fat`
    for non-release entries), matching the release-build tradeoff already
    used in
      `rust-release.yml` while reducing CI runtime.
    
    Verification:
    - Reproduced the original failure in CI-like containers:
      - `aarch64-unknown-linux-musl` failed with `cannot find -lcap`.
    - Verified the underlying mismatch by forcing host libcap into the link:
      - link then failed with glibc-specific unresolved symbols
        (`__isoc23_*`, `__*_chk`), confirming host libcap was unsuitable.
    - Verified the fix in CI-like containers after this change:
    - `cargo build -p codex-linux-sandbox --target
    aarch64-unknown-linux-musl --release` -> pass
    - `cargo build -p codex-linux-sandbox --target x86_64-unknown-linux-musl
    --release` -> pass
    - Triggered `rust-ci` on this branch and confirmed the new job appears:
    - `Lint/Build — ubuntu-24.04-arm - aarch64-unknown-linux-musl (release)`
  • build(linux-sandbox): always compile vendored bubblewrap on Linux; remove CODEX_BWRAP_ENABLE_FFI (#11498)
    ## Summary
    This PR removes the temporary `CODEX_BWRAP_ENABLE_FFI` flag and makes
    Linux builds always compile vendored bubblewrap support for
    `codex-linux-sandbox`.
    
    ## Changes
    - Removed `CODEX_BWRAP_ENABLE_FFI` gating from
    `codex-rs/linux-sandbox/build.rs`.
    - Linux builds now fail fast if vendored bubblewrap compilation fails
    (instead of warning and continuing).
    - Updated fallback/help text in
    `codex-rs/linux-sandbox/src/vendored_bwrap.rs` to remove references to
    `CODEX_BWRAP_ENABLE_FFI`.
    - Removed `CODEX_BWRAP_ENABLE_FFI` env wiring from:
      - `.github/workflows/rust-ci.yml`
      - `.github/workflows/bazel.yml`
      - `.github/workflows/rust-release.yml`
    
    ---------
    
    Co-authored-by: David Zbarsky <zbarsky@openai.com>
  • feat: enable premessage-deflate for websockets (#10966)
    note:
    unfortunately, tokio-tungstenite / tungstenite upgrade triggers some
    problems with linker of rama-tls-boring with openssl:
    ```
    error: linking with `/Users/apanasenko/Library/Caches/cargo-zigbuild/0.20.1/zigcc-x86_64-unknown-linux-musl-ff6a.sh` failed: exit status: 1
      |
      = note:  "/Users/apanasenko/Library/Caches/cargo-zigbuild/0.20.1/zigcc-x86_64-unknown-linux-musl-ff6a.sh" "-m64" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/rcrt1.o" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crti.o" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtbeginS.o" "<1 object files omitted>" "-Wl,--as-needed" "-Wl,-Bstatic" "/var/folders/kt/52y_g75x3ng8ktvk3rfwm6400000gp/T/rustcyGQdYm/{liblzma_sys-662a82316f96ec30,libbzip2_sys-bf78a2d58d5cbce6,liblibsqlite3_sys-6c004987fd67a36a,libtree_sitter_bash-220b99a97d331ab7,libtree_sitter-858f0a1dbfea58bd,libzstd_sys-6eb237deec748c5b,libring-2a87376483bf916f,libopenssl_sys-7c189e68b37fe2bb,liblibz_sys-4344eef4345520b1,librama_boring_sys-0414e98115015ee0}.rlib" "-lc++" "-lc++abi" "-lunwind" "-lc" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/libcompiler_builtins-*.rlib" "-L" "/var/folders/kt/52y_g75x3ng8ktvk3rfwm6400000gp/T/rustcyGQdYm/raw-dylibs" "-Wl,-Bdynamic" "-Wl,--eh-frame-hdr" "-Wl,-z,noexecstack" "-nostartfiles" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/libz-sys-ff5ea50d88c28ffb/out/lib" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/ring-bdec3dddc19f5a5e/out" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/openssl-sys-96e0870de3ca22bc/out/openssl-build/install/lib" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/zstd-sys-0cc37a5da1481740/out" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/tree-sitter-72d2418073317c0f/out" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/tree-sitter-bash-bfd293a9f333ce6a/out" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/libsqlite3-sys-b78b2cfb81a330fc/out" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/bzip2-sys-69a145cc859ef275/out/lib" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/lzma-sys-07e92d0b6baa6fd4/out" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/rama-boring-sys-0bc2dfbf669addc4/out/build/crypto/" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/rama-boring-sys-0bc2dfbf669addc4/out/build/ssl/" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/rama-boring-sys-0bc2dfbf669addc4/out/build/" "-L" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/rama-boring-sys-0bc2dfbf669addc4/out/build" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained" "-L" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib" "-o" "/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/deps/codex_network_proxy-d08268b863517761" "-Wl,--gc-sections" "-static-pie" "-Wl,-z,relro,-z,now" "-Wl,-O1" "-Wl,--strip-all" "-nodefaultlibs" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtendS.o" "<sysroot>/lib/rustlib/x86_64-unknown-linux-musl/lib/self-contained/crtn.o"
      = note: some arguments are omitted. use `--verbose` to show all linker arguments
      = note: warning: ignoring deprecated linker optimization setting '1'
              warning: unable to open library directory '/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/rama-boring-sys-0bc2dfbf669addc4/out/build/crypto/': FileNotFound
              ld.lld: error: duplicate symbol: SSL_export_keying_material
              >>> defined at ssl_lib.c:3816 (ssl/ssl_lib.c:3816)
              >>>            libssl-lib-ssl_lib.o:(SSL_export_keying_material) in archive /var/folders/kt/52y_g75x3ng8ktvk3rfwm6400000gp/T/rustcyGQdYm/libopenssl_sys-7c189e68b37fe2bb.rlib
              >>> defined at t1_enc.cc:205 (/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/rama-boring-sys-0bc2dfbf669addc4/out/boringssl/ssl/t1_enc.cc:205)
              >>>            t1_enc.cc.o:(.text.SSL_export_keying_material+0x0) in archive /var/folders/kt/52y_g75x3ng8ktvk3rfwm6400000gp/T/rustcyGQdYm/librama_boring_sys-0414e98115015ee0.rlib
    
              ld.lld: error: duplicate symbol: d2i_ASN1_TIME
              >>> defined at a_time.c:27 (crypto/asn1/a_time.c:27)
              >>>            libcrypto-lib-a_time.o:(d2i_ASN1_TIME) in archive /var/folders/kt/52y_g75x3ng8ktvk3rfwm6400000gp/T/rustcyGQdYm/libopenssl_sys-7c189e68b37fe2bb.rlib
              >>> defined at a_time.cc:34 (/Users/apanasenko/code/codex/codex-rs/target/x86_64-unknown-linux-musl/release/build/rama-boring-sys-0bc2dfbf669addc4/out/boringssl/crypto/asn1/a_time.cc:34)
              >>>            a_time.cc.o:(.text.d2i_ASN1_TIME+0x0) in archive /var/folders/kt/52y_g75x3ng8ktvk3rfwm6400000gp/T/rustcyGQdYm/librama_boring_sys-0414e98115015ee0.rlib
    ``` 
    
    that force me to migrate away from rama-tls-boring to rama-tls-rustls
    and pin `ring` for rustls.
  • feat: introducing a network sandbox proxy (#8442)
    This add a new crate, `codex-network-proxy`, a local network proxy
    service used by Codex to enforce fine-grained network policy (domain
    allow/deny) and to surface blocked network events for interactive
    approvals.
    
    - New crate: `codex-rs/network-proxy/` (`codex-network-proxy` binary +
    library)
    - Core capabilities:
      - HTTP proxy support (including CONNECT tunneling)
      - SOCKS5 proxy support (in the later PR)
    - policy evaluation (allowed/denied domain lists; denylist wins;
    wildcard support)
      - small admin API for polling/reload/mode changes
    - optional MITM support for HTTPS CONNECT to enforce “limited mode”
    method restrictions (later PR)
    
    Will follow up integration with codex in subsequent PRs.
    
    ## Testing
    
    - `cd codex-rs && cargo build -p codex-network-proxy`
    - `cd codex-rs && cargo run -p codex-network-proxy -- proxy`