## Why
It's hard to change the set of required jobs when they're managed in the
GitHub UI, and when each workflow is responsible for choosing it's own
scheduling it's easy to end up with skew between what we enforce on PRs
vs. on main.
## What
- add a `blocking-ci` caller workflow, triggered by pull requests and
pushes to `main`, for Bazel, blob size, cargo-deny, Codespell,
`repo-checks`, rust CI, and SDK CI
- add an `always()` terminal job named `CI required` that fails unless
every called workflow succeeds
- add a `postmerge-ci` caller workflow for `rust-ci-full` and
`v8-canary`, with a terminal `Postmerge CI results` job
- centralize V8 relevance detection in `v8_canary_changes.py`; unrelated
PR and postmerge runs execute metadata only and skip the expensive build
matrices
- leave `v8-canary` outside the blocking gate and leave the external
`cla` check independent
## Rollout
A repository admin must replace the existing required GitHub Actions
contexts with `CI required` in the main-branch ruleset. Retain `cla` as
a separate required check. Until that change is coordinated, this PR
cannot satisfy the old standalone check names. In-flight PRs will need
to be rebased after this lands.
## Why
CI jobs should not silently leave tracked changes or untracked files in
the repository worktree.
## What
- Add a shared final worktree-cleanliness action to 19 checkout-bearing
PR and main CI jobs.
- Ignore the intentional SDK scratch directory and nested V8 checkout.
- Pin Bazelisk in shared CI setup so `.bazelversion` remains
authoritative, avoiding `MODULE.bazel.lock` deltas on Windows runners.
- Leave `rust-ci-full` and release-only workflows unchanged.
- Update `AGENTS.md` to discourage review bots from asking for
`MODULE.bazel.lock` changes.
## Why
These workflows currently hard-code the `codex` runner group and custom
runner labels. That makes the same workflow definitions less portable
across repository copies or renamed repos, even though the runner fleet
follows the repository name scheme. Template the runner identities from
the repository name so `openai/codex` still resolves to the existing
`codex-*` runners while other repos can use their own `<repo>-*` runner
names.
## What Changed
- Replaced custom runner `group` values such as `codex-runners` with
`${{ github.event.repository.name }}-runners`.
- Replaced custom runner labels such as `codex-linux-x64` and
`codex-windows-arm64` with `${{ github.event.repository.name }}-...`.
- Covered direct `runs-on` objects, matrix `runs_on` entries, reusable
workflow runner inputs, and release runner labels.
## Verification
- Parsed all `.github/workflows/*.yml` files as YAML with Ruby.
- Searched `.github/workflows` to confirm no hardcoded runner-field
`codex-runners` or `codex-*` labels remain.
## Why
`BUILDBUDDY_API_KEY` now lives in the `bazel` GitHub Actions environment
as an environment secret. Jobs that need BuildBuddy credentials must opt
into that environment so `${{ secrets.BUILDBUDDY_API_KEY }}` resolves
from the protected environment secret instead of relying on an unscoped
repository/organization secret.
This follows the same environment-secret migration pattern as #26466.
## What Changed
- Attach each workflow job that reads `BUILDBUDDY_API_KEY` to the
`bazel` environment.
- Set `deployment: false` on those job-level environment blocks.
`deployment: false` lets the job enter the `bazel` environment to access
its environment secrets without creating GitHub deployment records for
these CI jobs. That keeps the environment as a secret/access-control
boundary without making ordinary Bazel CI runs look like deploys.
## Validation
- Parsed the modified workflow YAML files with Ruby's YAML parser.
- Checked the modified workflow files for trailing whitespace.
## Why
`just bazel-clippy` ran target discovery with
`--noexperimental_remote_repo_contents_cache`, then ran the build with
the workspace default `--experimental_remote_repo_contents_cache`. Bazel
therefore killed and restarted its server on each transition, slowing
repeated commands and discarding the in-memory analysis cache. An audit
found the same class of startup-option variation in several CI command
sequences.
## What changed
- Keep local lint target-discovery queries on the workspace-default
Bazel server, while making CI target discovery explicitly use the CI
startup options.
- Normalize GitHub Actions launches through the BuildBuddy wrapper to
share `BAZEL_OUTPUT_USER_ROOT` and
`--noexperimental_remote_repo_contents_cache`.
- Route the CI lockfile check and Windows test-shard query through the
same startup configuration.
- Document the startup-option invariant and add wrapper regression
coverage.
## Validation
- Confirmed consecutive local clippy target-discovery runs retained the
same Bazel server PID.
We cross build when using bazel for windows. This causes a couple
hiccups in that v8 does a mksnapshot step that is expecting to snapshot
on the host arch which wasn't matching when we were doing the
crossbuild. This was causing segfault failiures when starting up
codemode from a cross built artifact.
This changes things such that we cross build the library and then run
and link a snapshot on the host machine/arch which is windows. This
gives us a functional snapshot and library that can start code-mode on
windows.
This fixes the build and then fixes two test regressions we had.
## Why
PR #25905 intentionally adds a failing `codex-core` unit test, but its
[Bazel test on Windows
check](https://github.com/openai/codex/actions/runs/26837526950/job/79135369259)
passed. That shows the Bazel configuration introduced by #25156 is not
behaving as expected, so revert it while the configuration can be
investigated separately.
## What changed
Revert #25156 in full, restoring the previous Bazel remote
configuration, CI scripts, workflows, `rusty_v8` handling, and
documentation. This removes the shared BuildBuddy wrapper and its tests.
## Validation
Not run locally; this exact revert was prioritized for a fast rollback.
## Why
Bazel remote configuration was selected in several CI scripts and
workflow steps. That made the BuildBuddy tenant policy easy to duplicate
and harder to audit, especially for fork pull requests that must not use
the OpenAI tenant.
This builds on
[sluongng/buildbuddy-ci-host-routing](https://github.com/openai/codex/compare/main...sluongng:codex:sluongng/buildbuddy-ci-host-routing)
and consolidates the policy in one place.
## What to do if this breaks you
See `codex-rs/docs/bazel.md` for details. TLDR:
1. make a BuildBuddy API key and put it in `~/.bazelrc`
2. if you're an OpenAI employee, add `common
--config=buildbuddy-openai-rbe` to `user.bazelrc` in the repo root
Run `just bazel-test` to ensure it works.
Note that `just bazel-remote-test` no longer exists, you need to select
a remote configuration as documented to use RBE.
## What changed
- Add `.github/scripts/run_bazel_with_buildbuddy.py` as the shared Bazel
wrapper and Python library. It selects the OpenAI host only for trusted
upstream GitHub Actions runs, routes keyed fork runs to the generic
host, and falls back to local Bazel execution when no key is available.
- Move endpoint selection into explicit `.bazelrc` configurations and
update Bazel CI, query helpers, and `rusty_v8` staging to use the shared
policy. Loading-phase target-discovery queries remain local.
- Add wrapper and `rusty_v8` unit coverage, plus `just test-scripts` for
the `.github/scripts` Python tests.
- Document local Bazel usage, `user.bazelrc` setup, BuildBuddy
configurations, and CI behavior in `codex-rs/docs/bazel.md`.
## Validation
- `just test-scripts`
- `bash -n .github/scripts/run-bazel-ci.sh
.github/scripts/run-bazel-query-ci.sh
.github/scripts/run-argument-comment-lint-bazel.sh
scripts/list-bazel-clippy-targets.sh`
- `python3 -m py_compile .github/scripts/run_bazel_with_buildbuddy.py
.github/scripts/test_run_bazel_with_buildbuddy.py
.github/scripts/test_rusty_v8_bazel.py
.github/scripts/rusty_v8_bazel.py`
- `ruff check .github/scripts/run_bazel_with_buildbuddy.py
.github/scripts/test_run_bazel_with_buildbuddy.py
.github/scripts/test_rusty_v8_bazel.py
.github/scripts/rusty_v8_bazel.py`
The codex-windows runner group should be much faster than the default
GHA runners. Since bazel jobs on windows are frequently the long pole
for PRs checks, this will hopefully get people landing a bit faster.
## Summary
- split the single PR-blocking Bazel Windows test leg into four Windows
shard jobs
- preserve the existing required Windows Bazel check name with a
lightweight aggregate gate
- keep Linux/macOS Bazel test jobs and the separate Windows
clippy/release jobs unchanged
## Why
The ordinary PR Windows Bazel test leg was one GitHub Actions job, so
Bazel only had in-job parallelism. This gives that lane real job-level
fanout across separate Windows hosts while keeping the target set
disjoint via stable label hashing.
## Evidence
- final pre-rebase green run: `25774733562`
- Windows shard target counts: `61/212`, `48/212`, `52/212`, `51/212`
- Windows test fanout completed in about 7m29s versus a recent
monolithic median around 22m26s
## Notes
- this is scoped to the Bazel Windows test leg only
- each shard keeps the existing Windows cross-compile/RBE path and
restores the former monolithic Windows test cache
- shard jobs do not upload duplicate repository caches after test work,
keeping cache cleanup off the PR-blocking shard path
- no local validation run; relying on GitHub Actions for the
workflow-shaped check
Co-authored-by: Codex <noreply@openai.com>
## Why
PR CI should test the exact commit that was pushed to the PR branch. By
default, GitHub's `pull_request` event checks out a synthetic merge
commit from `refs/pull/<number>/merge`, so the tested tree can include
an implicit merge with the current base branch instead of matching the
pushed head SHA.
Using the PR head SHA makes each check result correspond to a concrete
commit the author submitted. This also behaves better for stacked PR
workflows, including Sapling stacks and other Git stack tooling: a
middle PR's head commit already contains the lower stack changes in its
tree, without pulling in commits above it or GitHub's temporary merge
ref.
## What Changed
- Set every `actions/checkout` in `pull_request` workflows under
`.github/workflows` to use `github.event.pull_request.head.sha` on PR
events and `github.sha` otherwise.
- Updated `blob-size-policy` to compare
`github.event.pull_request.base.sha` and
`github.event.pull_request.head.sha`, since it no longer checks out
GitHub's merge commit where `HEAD^1`/`HEAD^2` represented the PR range.
## Verification
- Parsed the edited workflow YAML files with Ruby.
- Checked that every checkout block in the `pull_request` workflows has
the PR-head `ref`.
This does two things:
- We use `persist-credentials: false` everywhere now. This is
unfortunately not the default in GitHub Actions, but it prevents
`actions/checkout` from dropping `secrets.GITHUB_TOKEN` onto disk.
- We interpose (some) template expansions through environment variables.
I've limited this to contexts that have non-fixed values; contexts that
are fixed (like `*.result`) are not dangerous to expand directly inline
(but maybe we should clean those up in the future for consistency
anyways).
This is a medium-risk change in terms of CI breakage: I did a scan for
usage of `git push` and other commands that implicitly use the persisted
credential, but couldn't find any. Even still, some implicit usages of
the persisted credentials may be lurking. Please ping ww@ if any issues
arise.
This builds on top of https://github.com/openai/codex/pull/15828 by
ensuring that hash-pinned actions with version comments are fully
qualified, rather than referencing floating/mutable comments like "v7".
This makes actions management tools behave more consistently.
This shouldn't break anything, since it's comment only. But if it does,
ping ww@ 🙂
**Summary**
- Add `codex-bwrap`, a standalone `bwrap` binary built from the existing
vendored bubblewrap sources.
- Remove the linked vendored bwrap path from `codex-linux-sandbox`;
runtime now prefers system `bwrap` and falls back to bundled
`codex-resources/bwrap`.
- Add bundled SHA-256 verification with missing/all-zero digest as the
dev-mode skip value, then exec the verified file through
`/proc/self/fd`.
- Keep `launcher.rs` focused on choosing and dispatching the preferred
launcher. Bundled lookup, digest verification, and bundled exec now live
in `linux-sandbox/src/bundled_bwrap.rs`; Bazel runfiles lookup lives in
`linux-sandbox/src/bazel_bwrap.rs`; shared argv/fd exec helpers live in
`linux-sandbox/src/exec_util.rs`.
- Teach Bazel tests to surface the Bazel-built `//codex-rs/bwrap:bwrap`
through `CARGO_BIN_EXE_bwrap`; `codex-linux-sandbox` only honors that
fallback in debug Bazel runfiles environments so release/user runtime
lookup stays tied to `codex-resources/bwrap`.
- Allow `codex-exec-server` filesystem helpers to preserve just the
Bazel bwrap/runfiles variables they need in debug Bazel builds, since
those helpers intentionally rebuild a small environment before spawning
`codex-linux-sandbox`.
- Verify the Bazel bwrap target in Linux release CI with a build-only
check. Running `bwrap --version` is too strong for GitHub runners
because bubblewrap still attempts namespace setup there.
**Verification**
- Latest update: `cargo test -p codex-linux-sandbox`
- Latest update: `just fix -p codex-linux-sandbox`
- `cargo check --target x86_64-unknown-linux-gnu -p codex-linux-sandbox`
could not run locally because this macOS machine does not have
`x86_64-linux-gnu-gcc`; GitHub Linux Bazel CI is expected to cover the
Linux-only modules.
- Earlier in this PR: `cargo test -p codex-bwrap`
- Earlier in this PR: `cargo test -p codex-exec-server`
- Earlier in this PR: `cargo check --release -p codex-exec-server`
- Earlier in this PR: `just fix -p codex-linux-sandbox -p
codex-exec-server`
- Earlier in this PR: `bazel test --nobuild
//codex-rs/linux-sandbox:linux-sandbox-all-test
//codex-rs/core:core-all-test
//codex-rs/exec-server:exec-server-file_system-test
//codex-rs/app-server:app-server-all-test` (analysis completed; Bazel
then refuses to run tests under `--nobuild`)
- Earlier in this PR: `bazel build --nobuild //codex-rs/bwrap:bwrap`
- Prior to this update: `just bazel-lock-update`, `just
bazel-lock-check`, and YAML parse check for
`.github/workflows/bazel.yml`
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/21255).
* #21257
* #21256
* __->__ #21255
## Why
Bazel CI was not actually exercising some sharded Rust integration-test
targets on macOS. The `rules_rust` sharding wrapper expects a symlink
runfiles tree, but this repo runs Bazel with `--noenable_runfiles`. In
that configuration the wrapper could fail to find the generated test
binary, produce an empty test list, and exit successfully. That made
targets such as `//codex-rs/core:core-all-test` look green even when
Cargo CI could still catch failures in the same Rust tests.
The coverage gap appears to have been introduced by
[#18082](https://github.com/openai/codex/pull/18082), which enabled
rules_rust native sharding on `//codex-rs/core:core-all-test` and the
other large Rust test labels. The manifest-runfiles setup itself
predates that change in
[#10098](https://github.com/openai/codex/pull/10098), but #18082 is
where the affected integration tests started running through the
incompatible rules_rust sharding wrapper.
[#18913](https://github.com/openai/codex/pull/18913) fixed the same
class of issue for wrapped unit-test shards, but integration-test shards
were still going through the rules_rust wrapper until this PR.
We still do not have the V8/code-mode pieces stable under the Bazel CI
cross-compile setup, so this keeps those tests out of Bazel while
restoring coverage for the rest of the sharded Rust integration suites.
Cargo CI remains responsible for V8/code-mode coverage for now.
This change did uncover a real failing core test on `main`:
`approved_folder_write_request_permissions_unblocks_later_apply_patch`.
That fix is split into
[#21060](https://github.com/openai/codex/pull/21060), which enables the
`apply_patch` tool in the test, teaches the aggregate core test binary
to dispatch the sandboxed filesystem helper, canonicalizes the macOS
temp patch target, and isolates the core test harness from managed
local/enterprise config. Keeping that fix separate lets this PR stay
focused on restoring Bazel coverage while documenting the first failure
it exposed.
## What changed
- Build sharded Rust integration tests as manual `*-bin` binaries and
run them through the existing manifest-aware `workspace_root_test`
launcher.
- Keep Bazel sharding on the launcher target so Rust test cases are
still distributed by stable test-name hashing.
- Configure Bazel CI to skip Rust tests whose names contain
`suite::code_mode::`.
- Exclude the standalone `codex-rs/code-mode` and `codex-rs/v8-poc`
unit-test targets from `bazel.yml`.
## Verification
- `bazel query --output=build //codex-rs/core:core-all-test` now shows
`workspace_root_test` wrapping `//codex-rs/core:core-all-test-bin`.
- `bazel test --test_output=all --nocache_test_results
--test_sharding_strategy=disabled //codex-rs/core:core-all-test
--test_filter=suite::request_permissions_tool::approved_folder_write_request_permissions_unblocks_later_apply_patch`
runs the actual Rust test body and passes.
- `bazel test --test_output=errors --nocache_test_results
--test_env=CODEX_BAZEL_TEST_SKIP_FILTERS=suite::code_mode::
//codex-rs/core:core-all-test` runs the sharded target with code-mode
skipped and passes overall locally, with one flaky attempt retried by
the existing `flaky = True` setting.
## Why
#20585 moved the Windows Bazel test job to the cross-compile path, but
the Windows Bazel clippy and verify-release-build jobs were still using
the native Windows/MSVC-host fallback. Those two jobs became the slowest
Windows PR legs, even though both are build-only signal and do not need
to execute the resulting binaries.
## What Changed
- Switches the Windows Bazel clippy job from
`--windows-msvc-host-platform` to `--windows-cross-compile`, so clippy
build actions use Linux RBE while still targeting
`x86_64-pc-windows-gnullvm`.
- Switches the Windows Bazel verify-release-build job to
`--windows-cross-compile` as well. This job only compiles
`cfg(not(debug_assertions))` Rust code under `fastbuild`, so it does not
need a native Windows build host.
- Keeps the old `--skip_incompatible_explicit_targets` behavior only for
fork/community PRs without `BUILDBUDDY_API_KEY`, where `run-bazel-ci.sh`
falls back to the local Windows MSVC-host shape.
- Adds `--windows-cross-compile` support to
`.github/scripts/run-bazel-query-ci.sh`, so target-discovery queries
select the same `ci-windows-cross` config as the subsequent build.
- Threads that option through `scripts/list-bazel-clippy-targets.sh` so
the Windows clippy job discovers targets under the same platform shape
as the subsequent clippy build.
## Verification
Local checks:
```shell
bash -n .github/scripts/run-bazel-query-ci.sh
bash -n scripts/list-bazel-clippy-targets.sh
ruby -e 'require "yaml"; YAML.load_file(".github/workflows/bazel.yml"); puts "ok"'
RUNNER_OS=Linux ./scripts/list-bazel-clippy-targets.sh | grep -c -- '-windows-cross-bin$'
RUNNER_OS=Windows ./scripts/list-bazel-clippy-targets.sh --windows-cross-compile | grep -c -- '-windows-cross-bin$'
```
The Linux target-list check reported `0` Windows-cross internal test
binaries, while the Windows cross target-list check reported `47`,
preserving the test-code clippy coverage shape from the existing Windows
job.
## Status
This is the Bazel PR-CI cross-compilation follow-up to #20485. It is
intentionally split from the Cargo/cargo-xwin release-build PoC so
#20485 can stay as the historical release-build exploration. The
unrelated async-utils test cleanup has been moved to #20686, so this PR
is focused on the Windows Bazel CI path.
The intended tradeoff is now explicit in `.github/workflows/bazel.yml`:
pull requests get the fast Windows cross-compiled Bazel test leg, while
post-merge pushes to `main` run both that fast cross leg and a fully
native Windows Bazel test leg. The native main-only job keeps full
V8/code-mode coverage and gets a 40-minute timeout because it is less
latency-sensitive than PR CI. All other Bazel jobs remain at 30 minutes.
## Why
Windows Bazel PR CI currently does the expensive part of the build on
Windows. A native Windows Bazel test job on `main` completed in about
28m12s, leaving very little headroom under the 30-minute job timeout and
making Windows the slowest PR signal.
#20485 showed that Windows cross-compilation can be materially faster
for Cargo release builds, but PR CI needs Bazel because Bazel owns our
test sharding, flaky-test retries, and integration-test layout. This PR
applies the same high-level shape we already use for macOS Bazel CI:
compile with remote Linux execution, then run platform-specific tests on
the platform runner.
The compromise is deliberately signal-aware: code-mode/V8 changes are
rare enough that PR CI can accept losing the direct V8/code-mode
smoke-test signal temporarily, while `main` still runs the native
Windows job post-merge to catch that class of regression. A follow-up PR
should investigate making the cross-built Windows gnullvm V8 archive
pass the direct V8/code-mode tests so this tradeoff can eventually go
away.
## What Changed
- Adds a `ci-windows-cross` Bazel config that targets
`x86_64-pc-windows-gnullvm`, uses Linux RBE for build actions, and keeps
`TestRunner` actions local on the Windows runner.
- Adds explicit Windows platform definitions for
`windows_x86_64_gnullvm`, `windows_x86_64_msvc`, and a bridge toolchain
that lets gnullvm test targets execute under the Windows MSVC host
platform.
- Updates the Windows Bazel PR test leg to opt into the cross-compile
path via `--windows-cross-compile` and `--remote-download-toplevel`.
- Adds a `test-windows-native-main` job that runs only for `push` events
on `refs/heads/main`, uses the native Windows Bazel path, includes
V8/code-mode smoke tests, and has `timeout-minutes: 40`.
- Keeps fork/community PRs without `BUILDBUDDY_API_KEY` on the previous
local Windows MSVC-host fallback, including
`--host_platform=//:local_windows_msvc` and `--jobs=8`.
- Preserves the existing integration-test shape on non-gnullvm
platforms, while generating Windows-cross wrapper targets only for
`windows_gnullvm`.
- Resolves `CARGO_BIN_EXE_*` values from runfiles at test runtime,
avoiding hard-coded Cargo paths and duplicate test runfiles.
- Extends the V8 Bazel patches enough for the
`x86_64-pc-windows-gnullvm` target and Linux remote execution path.
- Makes the Windows sandbox test cwd derive from `INSTA_WORKSPACE_ROOT`
at runtime when Bazel provides it, because cross-compiled binaries may
contain Linux compile-time paths.
- Keeps the direct V8/code-mode unit smoke tests out of the Windows
cross PR path for now while native Windows CI continues to cover them
post-merge.
## Command Shape
The fast Windows PR test leg invokes the normal Bazel CI wrapper like
this:
```shell
./.github/scripts/run-bazel-ci.sh \
--print-failed-action-summary \
--print-failed-test-logs \
--windows-cross-compile \
--remote-download-toplevel \
-- \
test \
--test_tag_filters=-argument-comment-lint \
--test_verbose_timeout_warnings \
--build_metadata=COMMIT_SHA=${GITHUB_SHA} \
-- \
//... \
-//third_party/v8:all \
-//codex-rs/code-mode:code-mode-unit-tests \
-//codex-rs/v8-poc:v8-poc-unit-tests
```
With the BuildBuddy secret available on Windows, the wrapper selects
`--config=ci-windows-cross` and appends the important Windows-cross
overrides after rc expansion:
```shell
--host_platform=//:rbe
--shell_executable=/bin/bash
--action_env=PATH=/usr/bin:/bin
--host_action_env=PATH=/usr/bin:/bin
--test_env=PATH=${CODEX_BAZEL_WINDOWS_PATH}
```
The native post-merge Windows job intentionally omits
`--windows-cross-compile` and does not exclude the V8/code-mode unit
targets:
```shell
./.github/scripts/run-bazel-ci.sh \
--print-failed-action-summary \
--print-failed-test-logs \
-- \
test \
--test_tag_filters=-argument-comment-lint \
--test_verbose_timeout_warnings \
--build_metadata=COMMIT_SHA=${GITHUB_SHA} \
--build_metadata=TAG_windows_native_main=true \
-- \
//... \
-//third_party/v8:all
```
## Research Notes
The existing macOS Bazel CI config already uses the model we want here:
build actions run remotely with `--strategy=remote`, but `TestRunner`
actions execute on the macOS runner. This PR mirrors that pattern for
Windows with `--strategy=TestRunner=local`.
The important Bazel detail is that `rules_rs` is already targeting
`x86_64-pc-windows-gnullvm` for Windows Bazel PR tests. This PR changes
where the build actions execute; it does not switch the Bazel PR test
target to Cargo, `cargo-nextest`, or the MSVC release target.
Cargo release builds differ from this Bazel path for V8: the normal
Windows Cargo release target is MSVC, and `rusty_v8` publishes prebuilt
Windows MSVC `.lib.gz` archives. The Bazel PR path targets
`windows-gnullvm`; `rusty_v8` does not publish a prebuilt Windows
GNU/gnullvm archive, so this PR builds that archive in-tree. That
Linux-RBE-built gnullvm archive currently crashes in direct V8/code-mode
smoke tests, which is why the workflow keeps native Windows coverage on
`main`.
The less obvious Bazel detail is test wrapper selection. Bazel chooses
the Windows test wrapper (`tw.exe`) from the test action execution
platform, not merely from the Rust target triple. The outer
`workspace_root_test` therefore declares the default test toolchain and
uses the bridge toolchain above so the test action executes on Windows
while its inner Rust binary is built for gnullvm.
The V8 investigation exposed a Windows-client gotcha: even when an
action execution platform is Linux RBE, Bazel can still derive the
genrule shell path from the Windows client. That produced remote
commands trying to run `C:\Program Files\Git\usr\bin\bash.exe` on Linux
workers. The wrapper now passes `--shell_executable=/bin/bash` with
`--host_platform=//:rbe` for the Windows cross path.
The same Windows-client/Linux-RBE boundary also affected
`third_party/v8:binding_cc`: a multiline genrule command can carry CRLF
line endings into Linux remote bash, which failed as `$'\r'`. That
genrule now keeps the `sed` command on one physical shell line while
using an explicit Starlark join so the shell arguments stay readable.
## Verification
Local checks included:
```shell
bash -n .github/scripts/run-bazel-ci.sh
bash -n workspace_root_test_launcher.sh.tpl
ruby -e "require %q{yaml}; YAML.load_file(%q{.github/workflows/bazel.yml}); puts %q{ok}"
RUNNER_OS=Linux ./scripts/list-bazel-clippy-targets.sh
RUNNER_OS=Windows ./scripts/list-bazel-clippy-targets.sh
RUNNER_OS=Linux ./tools/argument-comment-lint/list-bazel-targets.sh
RUNNER_OS=Windows ./tools/argument-comment-lint/list-bazel-targets.sh
```
The Linux clippy and argument-comment target lists contain zero
`*-windows-cross-bin` labels, while the Windows lists still include 47
Windows-cross internal test binaries.
CI evidence:
- Baseline native Windows Bazel test on `main`: success in about 28m12s,
https://github.com/openai/codex/actions/runs/25206257208/job/73907325959
- Green Windows-cross Bazel run on the split PR before adding the
main-only native leg: Windows test 9m16s, Windows release verify 5m10s,
Windows clippy 4m43s,
https://github.com/openai/codex/actions/runs/25231890068
- The latest SHA adds the explicit PR-vs-main tradeoff in `bazel.yml`;
CI is rerunning on that focused diff.
## Follow-Up
A subsequent PR should investigate making a cross-built Windows binary
work with V8/code-mode enabled. Likely options are either making the
Linux-RBE-built `windows-gnullvm` V8 archive correct at runtime, or
evaluating whether a Bazel MSVC target/toolchain can reuse the same
prebuilt MSVC `rusty_v8` archive shape that Cargo release builds already
use.
I think raising it to 45 minutes in
https://github.com/openai/codex/pull/19578 was a mistake for the reasons
explained in the comments in the code. Instead, we attempt to defend
against timeouts by increasing the number of shards in
`app-server-all-test` so that a "true failure" that gets run 3x should
not take as much wall clock time.
Unfortunately, if most of the build graph is invalidated such that there
are few cache hits, the Windows Bazel build for all the tests often
takes more than `30` minutes, so this PR increases the timeout to `45`
minutes until we set up distributed builds.
## Why
The Bazel workflow has multiple jobs that run concurrently for the same
target triple. In particular, the Windows `test`, `clippy`, and
`verify-release-build` jobs could all miss and then attempt to save the
same Bazel repository cache key:
```text
bazel-cache-${target}-${lockhash}
```
Because `actions/cache` entries are immutable, only one job can reserve
that key. The others can report failures such as:
```text
Failed to save: Unable to reserve cache with key bazel-cache-x86_64-pc-windows-gnullvm-..., another job may be creating this cache.
```
Adding only the workflow name would not separate these jobs because they
all run inside the same `Bazel` workflow. The key needs a job-level
namespace as well.
## What Changed
- Added a required `cache-scope` input to
`.github/actions/prepare-bazel-ci/action.yml`.
- Moved Bazel repository cache key construction into the shared action
and exposed the computed key as `repository-cache-key`.
- Exposed the exact restore result as `repository-cache-hit` so save
steps can skip exact cache hits.
- Updated `.github/workflows/bazel.yml` to pass `cache-scope: bazel-${{
github.job }}` for the `test`, `clippy`, and `verify-release-build`
jobs.
- The scoped restore key is now the only fallback. This avoids carrying
a temporary restore path for the old unscoped cache namespace.
## Verification
- Parsed `.github/actions/prepare-bazel-ci/action.yml` and
`.github/workflows/bazel.yml` with Ruby's YAML parser.
- `actionlint` is not installed in this workspace, so I could not run a
GitHub Actions semantic lint locally.
## Why
Unused imports in `core/tests/suite/unified_exec.rs` in the Windows
build were not caught by Bazel CI on
https://github.com/openai/codex/pull/18096. I spot-checked
https://github.com/openai/codex/actions/workflows/rust-ci-full.yml?query=branch%3Amain
and noticed that builds were consistently red. This revealed that our
Cargo builds _were_ properly catching these issues, identifying a
Windows-specific coverage hole in the Bazel clippy job.
The Windows Bazel clippy job uses `--skip_incompatible_explicit_targets`
so it can lint a broad target set without failing immediately on targets
that are genuinely incompatible with Windows. However, with the default
Windows host platform, `rust_test` targets such as
`//codex-rs/core:core-all-test` could be skipped before the clippy
aspect reached their integration-test modules. As a result, the imports
in `core/tests/suite/unified_exec.rs` were not being linted by the
Windows Bazel clippy job at all.
The clippy diagnostic that Windows Bazel should have surfaced was:
```text
error: unused import: `codex_config::Constrained`
--> core\tests\suite\unified_exec.rs:8:5
|
8 | use codex_config::Constrained;
| ^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: `-D unused-imports` implied by `-D warnings`
= help: to override `-D warnings` add `#[allow(unused_imports)]`
error: unused import: `codex_protocol::permissions::FileSystemAccessMode`
--> core\tests\suite\unified_exec.rs:11:5
|
11 | use codex_protocol::permissions::FileSystemAccessMode;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: unused import: `codex_protocol::permissions::FileSystemPath`
--> core\tests\suite\unified_exec.rs:12:5
|
12 | use codex_protocol::permissions::FileSystemPath;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: unused import: `codex_protocol::permissions::FileSystemSandboxEntry`
--> core\tests\suite\unified_exec.rs:13:5
|
13 | use codex_protocol::permissions::FileSystemSandboxEntry;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
error: unused import: `codex_protocol::permissions::FileSystemSandboxPolicy`
--> core\tests\suite\unified_exec.rs:14:5
|
14 | use codex_protocol::permissions::FileSystemSandboxPolicy;
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
```
## What changed
- Run the Windows Bazel clippy job with the MSVC host platform via
`--windows-msvc-host-platform`, matching the Windows Bazel test job.
This keeps `--skip_incompatible_explicit_targets` while ensuring Windows
`rust_test` targets such as `//codex-rs/core:core-all-test` are still
linted.
- Remove the unused imports from `core/tests/suite/unified_exec.rs`.
- Add `--print-failed-action-summary` to
`.github/scripts/run-bazel-ci.sh` so Bazel action failures can be
summarized after the build exits.
## Failure reporting
Once the coverage issue was fixed, an intentionally reintroduced unused
import made the Windows Bazel clippy job fail as expected. That exposed
a separate usability problem: because the job keeps `--keep_going`, the
top-level Bazel output could still end with:
```text
ERROR: Build did NOT complete successfully
FAILED:
```
without the underlying rustc/clippy diagnostic being visible in the
obvious part of the GitHub Actions log.
To keep `--keep_going` while making failures actionable, the wrapper now
scans the captured Bazel console output for failed actions and prints
the matching rustc/clippy diagnostic block. When a diagnostic block is
found, it is emitted both as a GitHub `::error` annotation and as plain
expanded log output, rather than being hidden in a collapsed group.
## Verification
To validate the CI path, I intentionally introduced an unused import in
`core/tests/suite/unified_exec.rs`. The Windows Bazel clippy job failed
as expected, confirming that the integration-test module is now covered
by Bazel clippy. The same failure also verified that the wrapper
surfaces the matching clippy diagnostics directly in the Actions output.
## Summary
- cap the Windows Bazel test lane at `--jobs=8` to reduce local runner
pressure
- keep Linux and macOS Bazel test concurrency unchanged
- make failed-test log tailing resolve `bazel-testlogs` with the same CI
config and Windows host-platform context as the failed invocation
- prefer Bazel-reported `test.log` paths and normalize Windows path
separators before tailing
## Context
The Windows Bazel workflow currently uses `ci-windows`, which does not
inherit the remote executor config. This means the lane runs the `//...`
test suite locally and otherwise falls back to the repo-wide `common
--jobs=30`. The new Windows-only override is intended to reduce local
executor pressure without changing coverage.
## Validation
Not run locally; this is a CI workflow change and the draft PR is
intended to exercise the GitHub Actions lane directly.
---------
Co-authored-by: Codex <noreply@openai.com>
## Summary
1. Revert https://github.com/openai/codex/pull/17848 so the Bazel and
`BUILD` file changes leave `main`.
2. Prepare for a narrower follow up that restores only `SECURITY.md`.
## Validation
1. Reviewed the revert diff against `main`.
2. Ran a clean diff check before push.
## Summary
1. Add a Security Boundaries section to `SECURITY.md`.
2. Point readers to the Codex Agent approvals and security documentation
for sandboxing, approvals, and network controls.
## Validation
1. Reviewed the `SECURITY.md` diff in a clean worktree.
2. No tests run. Docs only change.
## Why
`main` recently needed
[#17691](https://github.com/openai/codex/pull/17691) because code behind
`cfg(not(debug_assertions))` was not being compiled by the Bazel PR
workflow. Our existing CI only built the fast/debug configuration, so
PRs could stay green while release-only Rust code still failed to
compile. This PR adds a release-style compile check that is cheap enough
to run on every PR.
## What Changed
- Added a `verify-release-build` job to `.github/workflows/bazel.yml`.
- Represented each supported OS once in that job's matrix: x64 Linux,
arm64 macOS, and x64 Windows.
- Kept the build close to fastbuild cost by using
`--compilation_mode=fastbuild` while forcing Rust to compile with
`-Cdebug-assertions=no`, which makes `cfg(not(debug_assertions))` true
without also turning on release optimizations or debug-info generation.
- Added comments in `.github/workflows/bazel.yml` and
`scripts/list-bazel-release-targets.sh` to make the job's intent and
target scope explicit.
- Restored the Bazel repository cache save behavior to run after every
non-cancelled job, matching
[#16926](https://github.com/openai/codex/pull/16926), and removed the
now-unused `repository-cache-hit` output from `prepare-bazel-ci`.
- Reused the shared `prepare-bazel-ci` action from the parent PR so the
new job does not duplicate Bazel setup boilerplate.
## Verification
- Used `bazel aquery` on `//codex-rs/tui:codex-tui` to confirm the Rust
compile still uses `opt-level=0` and `debuginfo=0` while passing
`-Cdebug-assertions=no`.
- Parsed `.github/workflows/bazel.yml` as YAML locally.
- Ran `bash -n scripts/list-bazel-release-targets.sh`.
## Why
This stack adds a new Bazel CI lane that verifies Rust code behind
`cfg(not(debug_assertions))`, but adding that job directly to
`.github/workflows/bazel.yml` would duplicate the same setup in multiple
places. Extracting the shared setup first keeps the follow-up change
easier to review and reduces the chance that future Bazel workflow edits
drift apart.
## What Changed
- Added `.github/actions/prepare-bazel-ci/action.yml` as a composite
action for the Bazel job bootstrap shared by multiple workflow jobs.
- Moved the existing Bazel setup, repository-cache restore, and
execution-log setup behind that action.
- Updated the `test` and `clippy` jobs in `.github/workflows/bazel.yml`
to call `prepare-bazel-ci`.
- Exposed `repository-cache-hit` and `repository-cache-path` outputs so
callers can keep the existing cache-save behavior without duplicating
the restore step.
## Verification
- Parsed `.github/workflows/bazel.yml` as YAML locally after rebasing
the stack.
- CI will exercise the refactored jobs end to end.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/17704).
* #17705
* __->__ #17704
## Summary
- Pin Rust git patch dependencies to immutable revisions and make
cargo-deny reject unknown git and registry sources unless explicitly
allowlisted.
- Add checked-in SHA-256 coverage for the current rusty_v8 release
assets, wire those hashes into Bazel, and verify CI override downloads
before use.
- Add rusty_v8 MODULE.bazel update/check tooling plus a Bazel CI guard
so future V8 bumps cannot drift from the checked-in checksum manifest.
- Pin release/lint cargo installs and all external GitHub Actions refs
to immutable inputs.
## Future V8 bump flow
Run these after updating the resolved `v8` crate version and checksum
manifest:
```bash
python3 .github/scripts/rusty_v8_bazel.py update-module-bazel
python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
```
The update command rewrites the matching `rusty_v8_<crate_version>`
`http_file` SHA-256 values in `MODULE.bazel` from
`third_party/v8/rusty_v8_<crate_version>.sha256`. The check command is
also wired into Bazel CI to block drift.
## Notes
- This intentionally excludes RustSec dependency upgrades and
bubblewrap-related changes per request.
- The branch was rebased onto the latest origin/main before opening the
PR.
## Validation
- cargo fetch --locked
- cargo deny check advisories
- cargo deny check
- cargo deny check sources
- python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
- python3 .github/scripts/rusty_v8_bazel.py update-module-bazel
- python3 -m unittest discover -s .github/scripts -p
'test_rusty_v8_bazel.py'
- python3 -m py_compile .github/scripts/rusty_v8_bazel.py
.github/scripts/rusty_v8_module_bazel.py
.github/scripts/test_rusty_v8_bazel.py
- repo-wide GitHub Actions `uses:` audit: all external action refs are
pinned to 40-character SHAs
- yq eval on touched workflows and local actions
- git diff --check
- just bazel-lock-check
## Hash verification
- Confirmed `MODULE.bazel` hashes match
`third_party/v8/rusty_v8_146_4_0.sha256`.
- Confirmed GitHub release asset digests for denoland/rusty_v8
`v146.4.0` and openai/codex `rusty-v8-v146.4.0` match the checked-in
hashes.
- Streamed and SHA-256 hashed all 10 `MODULE.bazel` rusty_v8 asset URLs
locally; every downloaded byte stream matched both `MODULE.bazel` and
the checked-in manifest.
## Pin verification
- Confirmed signing-action pins match the peeled commits for their tag
comments: `sigstore/cosign-installer@v3.7.0`, `azure/login@v2`, and
`azure/trusted-signing-action@v0`.
- Pinned the remaining tag-based action refs in Bazel CI/setup:
`actions/setup-node@v6`, `facebook/install-dotslash@v2`,
`bazelbuild/setup-bazelisk@v3`, and `actions/cache/restore@v5`.
- Normalized all `bazelbuild/setup-bazelisk@v3` refs to the peeled
commit behind the annotated tag.
- Audited Cargo git dependencies: every manifest git dependency uses
`rev` only, every `Cargo.lock` git source has `?rev=<sha>#<same-sha>`,
and `cargo deny check sources` passes with `required-git-spec = "rev"`.
- Shallow-fetched each distinct git dependency repo at its pinned SHA
and verified Git reports each object as a commit.
## Why
Bazel CI had two independent Windows issues:
- The workflow saved/restored `~/.cache/bazel-repo-cache`, but
`.bazelrc` configured `common:ci-windows
--repository_cache=D:/a/.cache/bazel-repo-cache`, so `actions/cache` and
Bazel could point at different directories.
- The Windows `Bazel clippy` job passed the full explicit target list
from `//codex-rs/...`, but some of those explicit targets are
intentionally incompatible with `//:local_windows`.
`run-argument-comment-lint-bazel.sh` already handles that with
`--skip_incompatible_explicit_targets`; the clippy workflow path did
not.
I also tried switching the workflow cache path to
`D:\a\.cache\bazel-repo-cache`, but the Windows clippy job repeatedly
failed with `Failed to restore: Cache service responded with 400`, so
the final change standardizes on `$HOME/.cache/bazel-repo-cache` and
makes cache restore non-fatal.
## What Changed
- Expose one repository-cache path from
`.github/actions/setup-bazel-ci/action.yml` and export that path as
`BAZEL_REPOSITORY_CACHE` so `run-bazel-ci.sh` passes it to Bazel after
`--config=ci-*`.
- Move `actions/cache/restore` out of the composite action into
`.github/workflows/bazel.yml`, and make restore failures non-fatal
there.
- Save exactly the exported cache path in `.github/workflows/bazel.yml`.
- Remove `common:ci-windows
--repository_cache=D:/a/.cache/bazel-repo-cache` from `.bazelrc` so the
Windows CI config no longer disagrees with the workflow cache path.
- Pass `--skip_incompatible_explicit_targets` in the Windows `Bazel
clippy` job so incompatible explicit targets do not fail analysis while
the lint aspect still traverses compatible Rust dependencies.
## Verification
- Parsed `.github/actions/setup-bazel-ci/action.yml` and
`.github/workflows/bazel.yml` with Ruby's YAML loader.
- Resubmitted PR `#16740`; CI is rerunning on the amended commit.
## Why
`cargo clippy --tests` was catching warnings in inline `#[cfg(test)]`
code that the Bazel PR Clippy lane missed. The existing Bazel invocation
linted `//codex-rs/...`, but that did not apply Clippy to the generated
manual `rust_test` binaries, so warnings in targets such as
`//codex-rs/state:state-unit-tests-bin` only surfaced as plain compile
warnings instead of failing the lint job.
## What Changed
- added `scripts/list-bazel-clippy-targets.sh` to expand the Bazel
Clippy target set with the generated manual `rust_test` rules while
still excluding `//codex-rs/v8-poc:all`
- updated `.github/workflows/bazel.yml` to use that expanded target list
in the Bazel Clippy PR job
- updated `just bazel-clippy` to use the same target expansion locally
- updated `.github/workflows/README.md` to document that the Bazel PR
lint lane now covers inline `#[cfg(test)]` code
## Verification
- `./scripts/list-bazel-clippy-targets.sh` includes
`//codex-rs/state:state-unit-tests-bin`
- `bazel build --config=clippy -- //codex-rs/state:state-unit-tests-bin`
now fails with the same unused import in `state/src/runtime/logs.rs`
that `cargo clippy --tests` reports
# Why this PR exists
This PR is trying to fix a coverage gap in the Windows Bazel Rust test
lane.
Before this change, the Windows `bazel test //...` job was nominally
part of PR CI, but a non-trivial set of `//codex-rs/...` Rust test
targets did not actually contribute test signal on Windows. In
particular, targets such as `//codex-rs/core:core-unit-tests`,
`//codex-rs/core:core-all-test`, and `//codex-rs/login:login-unit-tests`
were incompatible during Bazel analysis on the Windows gnullvm platform,
so they never reached test execution there. That is why the
Cargo-powered Windows CI job could surface Windows-only failures that
the Bazel-powered job did not report: Cargo was executing those tests,
while Bazel was silently dropping them from the runnable target set.
The main goal of this PR is to make the Windows Bazel test lane execute
those Rust test targets instead of skipping them during analysis, while
still preserving `windows-gnullvm` as the target configuration for the
code under test. In other words: use an MSVC host/exec toolchain where
Bazel helper binaries and build scripts need it, but continue compiling
the actual crate targets with the Windows gnullvm cfgs that our current
Bazel matrix is supposed to exercise.
# Important scope note
This branch intentionally removes the non-resource-loading `.rs` test
and production-code changes from the earlier
`codex/windows-bazel-rust-test-coverage` branch. The only Rust source
changes kept here are runfiles/resource-loading fixes in TUI tests:
- `codex-rs/tui/src/chatwidget/tests.rs`
- `codex-rs/tui/tests/manager_dependency_regression.rs`
That is deliberate. Since the corresponding tests already pass under
Cargo, this PR is meant to test whether Bazel infrastructure/toolchain
fixes alone are enough to get a healthy Windows Bazel test signal,
without changing test behavior for Windows timing, shell output, or
SQLite file-locking.
# How this PR changes the Windows Bazel setup
## 1. Split Windows host/exec and target concerns in the Bazel test lane
The core change is that the Windows Bazel test job now opts into an MSVC
host platform for Bazel execution-time tools, but only for `bazel test`,
not for the Bazel clippy build.
Files:
- `.github/workflows/bazel.yml`
- `.github/scripts/run-bazel-ci.sh`
- `MODULE.bazel`
What changed:
- `run-bazel-ci.sh` now accepts `--windows-msvc-host-platform`.
- When that flag is present on Windows, the wrapper appends
`--host_platform=//:local_windows_msvc` unless the caller already
provided an explicit `--host_platform`.
- `bazel.yml` passes that wrapper flag only for the Windows `bazel test
//...` job.
- The Bazel clippy job intentionally does **not** pass that flag, so
clippy stays on the default Windows gnullvm host/exec path and continues
linting against the target cfgs we care about.
- `run-bazel-ci.sh` also now forwards `CODEX_JS_REPL_NODE_PATH` on
Windows and normalizes the `node` executable path with `cygpath -w`, so
tests that need Node resolve the runner's Node installation correctly
under the Windows Bazel test environment.
Why this helps:
- The original incompatibility chain was mostly on the **exec/tool**
side of the graph, not in the Rust test code itself. Moving host tools
to MSVC lets Bazel resolve helper binaries and generators that were not
viable on the gnullvm exec platform.
- Keeping the target platform on gnullvm preserves cfg coverage for the
crates under test, which is important because some Windows behavior
differs between `msvc` and `gnullvm`.
## 2. Teach the repo's Bazel Rust macro about Windows link flags and
integration-test knobs
Files:
- `defs.bzl`
- `codex-rs/core/BUILD.bazel`
- `codex-rs/otel/BUILD.bazel`
- `codex-rs/tui/BUILD.bazel`
What changed:
- Replaced the old gnullvm-only linker flag block with
`WINDOWS_RUSTC_LINK_FLAGS`, which now handles both Windows ABIs:
- gnullvm gets `-C link-arg=-Wl,--stack,8388608`
- MSVC gets `-C link-arg=/STACK:8388608`, `-C
link-arg=/NODEFAULTLIB:libucrt.lib`, and `-C link-arg=ucrt.lib`
- Threaded those Windows link flags into generated `rust_binary`,
unit-test binaries, and integration-test binaries.
- Extended `codex_rust_crate(...)` with:
- `integration_test_args`
- `integration_test_timeout`
- Used those new knobs to:
- mark `//codex-rs/core:core-all-test` as a long-running integration
test
- serialize `//codex-rs/otel:otel-all-test` with `--test-threads=1`
- Added `src/**/*.rs` to `codex-rs/tui` test runfiles, because one
regression test scans source files at runtime and Bazel does not expose
source-tree directories unless they are declared as data.
Why this helps:
- Once host-side MSVC tools are available, we still need the generated
Rust test binaries to link correctly on Windows. The MSVC-side
stack/UCRT flags make those binaries behave more like their Cargo-built
equivalents.
- The integration-test macro knobs avoid hardcoding one-off test
behavior in ad hoc BUILD rules and make the generated test targets more
expressive where Bazel and Cargo have different runtime defaults.
## 3. Patch `rules_rs` / `rules_rust` so Windows MSVC exec-side Rust and
build scripts are actually usable
Files:
- `MODULE.bazel`
- `patches/rules_rs_windows_exec_linker.patch`
- `patches/rules_rust_windows_bootstrap_process_wrapper_linker.patch`
- `patches/rules_rust_windows_build_script_runner_paths.patch`
- `patches/rules_rust_windows_exec_msvc_build_script_env.patch`
- `patches/rules_rust_windows_msvc_direct_link_args.patch`
- `patches/rules_rust_windows_process_wrapper_skip_temp_outputs.patch`
- `patches/BUILD.bazel`
What these patches do:
- `rules_rs_windows_exec_linker.patch`
- Adds a `rust-lld` filegroup for Windows Rust toolchain repos,
symlinked to `lld-link.exe` from `PATH`.
- Marks Windows toolchains as using a direct linker driver.
- Supplies Windows stdlib link flags for both gnullvm and MSVC.
- `rules_rust_windows_bootstrap_process_wrapper_linker.patch`
- For Windows MSVC Rust targets, prefers the Rust toolchain linker over
an inherited C++ linker path like `clang++`.
- This specifically avoids the broken mixed-mode command line where
rustc emits MSVC-style `/NOLOGO` / `/LIBPATH:` / `/OUT:` arguments but
Bazel still invokes `clang++.exe`.
- `rules_rust_windows_build_script_runner_paths.patch`
- Normalizes forward-slash execroot-relative paths into Windows path
separators before joining them on Windows.
- Uses short Windows paths for `RUSTC`, `OUT_DIR`, and the build-script
working directory to avoid path-length and quoting issues in third-party
build scripts.
- Exposes `RULES_RUST_BAZEL_BUILD_SCRIPT_RUNNER=1` to build scripts so
crate-local patches can detect "this is running under Bazel's
build-script runner".
- Fixes the Windows runfiles cleanup filter so generated files with
retained suffixes are actually retained.
- `rules_rust_windows_exec_msvc_build_script_env.patch`
- For exec-side Windows MSVC build scripts, stops force-injecting
Bazel's `CC`, `CXX`, `LD`, `CFLAGS`, and `CXXFLAGS` when that would send
GNU-flavored tool paths/flags into MSVC-oriented Cargo build scripts.
- Rewrites or strips GNU-only `--sysroot`, MinGW include/library paths,
stack-protector, and `_FORTIFY_SOURCE` flags on the MSVC exec path.
- The practical effect is that build scripts can fall back to the Visual
Studio toolchain environment already exported by CI instead of crashing
inside Bazel's hermetic `clang.exe` setup.
- `rules_rust_windows_msvc_direct_link_args.patch`
- When using a direct linker on Windows, stops forwarding GNU driver
flags such as `-L...` and `--sysroot=...` that `lld-link.exe` does not
understand.
- Passes non-`.lib` native artifacts as explicit `-Clink-arg=<path>`
entries when needed.
- Filters C++ runtime libraries to `.lib` artifacts on the Windows
direct-driver path.
- `rules_rust_windows_process_wrapper_skip_temp_outputs.patch`
- Excludes transient `*.tmp*` and `*.rcgu.o` files from process-wrapper
dependency search-path consolidation, so unstable compiler outputs do
not get treated as real link search-path inputs.
Why this helps:
- The host-platform split alone was not enough. Once Bazel started
analyzing/running previously incompatible Rust tests on Windows, the
next failures were in toolchain plumbing:
- MSVC-targeted Rust tests were being linked through `clang++` with
MSVC-style arguments.
- Cargo build scripts running under Bazel's Windows MSVC exec platform
were handed Unix/GNU-flavored path and flag shapes.
- Some generated paths were too long or had path-separator forms that
third-party Windows build scripts did not tolerate.
- These patches make that mixed Bazel/Cargo/Rust/MSVC path workable
enough for the test lane to actually build and run the affected crates.
## 4. Patch third-party crate build scripts that were not robust under
Bazel's Windows MSVC build-script path
Files:
- `MODULE.bazel`
- `patches/aws-lc-sys_windows_msvc_prebuilt_nasm.patch`
- `patches/ring_windows_msvc_include_dirs.patch`
- `patches/zstd-sys_windows_msvc_include_dirs.patch`
What changed:
- `aws-lc-sys`
- Detects Bazel's Windows MSVC build-script runner via
`RULES_RUST_BAZEL_BUILD_SCRIPT_RUNNER` or a `bazel-out` manifest-dir
path.
- Uses `clang-cl` for Bazel Windows MSVC builds when no explicit
`CC`/`CXX` is set.
- Allows prebuilt NASM on the Bazel Windows MSVC path even when `nasm`
is not available directly in the runner environment.
- Avoids canonicalizing `CARGO_MANIFEST_DIR` in the Bazel Windows MSVC
case, because that path may point into Bazel output/runfiles state where
preserving the given path is more reliable than forcing a local
filesystem canonicalization.
- `ring`
- Under the Bazel Windows MSVC build-script runner, copies the
pregenerated source tree into `OUT_DIR` and uses that as the
generated-source root.
- Adds include paths needed by MSVC compilation for
Fiat/curve25519/P-256 generated headers.
- Rewrites a few relative includes in C sources so the added include
directories are sufficient.
- `zstd-sys`
- Adds MSVC-only include directories for `compress`, `decompress`, and
feature-gated dictionary/legacy/seekable sources.
- Skips `-fvisibility=hidden` on MSVC targets, where that
GCC/Clang-style flag is not the right mechanism.
Why this helps:
- After the `rules_rust` plumbing started running build scripts on the
Windows MSVC exec path, some third-party crates still failed for
crate-local reasons: wrong compiler choice, missing include directories,
build-script assumptions about manifest paths, or Unix-only C compiler
flags.
- These crate patches address those crate-local assumptions so the
larger toolchain change can actually reach first-party Rust test
execution.
## 5. Keep the only `.rs` test changes to Bazel/Cargo runfiles parity
Files:
- `codex-rs/tui/src/chatwidget/tests.rs`
- `codex-rs/tui/tests/manager_dependency_regression.rs`
What changed:
- Instead of asking `find_resource!` for a directory runfile like
`src/chatwidget/snapshots` or `src`, these tests now resolve one known
file runfile first and then walk to its parent directory.
Why this helps:
- Bazel runfiles are more reliable for explicitly declared files than
for source-tree directories that happen to exist in a Cargo checkout.
- This keeps the tests working under both Cargo and Bazel without
changing their actual assertions.
# What we tried before landing on this shape, and why those attempts did
not work
## Attempt 1: Force `--host_platform=//:local_windows_msvc` for all
Windows Bazel jobs
This did make the previously incompatible test targets show up during
analysis, but it also pushed the Bazel clippy job and some unrelated
build actions onto the MSVC exec path.
Why that was bad:
- Windows clippy started running third-party Cargo build scripts with
Bazel's MSVC exec settings and crashed in crates such as `tree-sitter`
and `libsqlite3-sys`.
- That was a regression in a job that was previously giving useful
gnullvm-targeted lint signal.
What this PR does instead:
- The wrapper flag is opt-in, and `bazel.yml` uses it only for the
Windows `bazel test` lane.
- The clippy lane stays on the default Windows gnullvm host/exec
configuration.
## Attempt 2: Broaden the `rules_rust` linker override to all Windows
Rust actions
This fixed the MSVC test-lane failure where normal `rust_test` targets
were linked through `clang++` with MSVC-style arguments, but it broke
the default gnullvm path.
Why that was bad:
-
`@@rules_rs++rules_rust+rules_rust//util/process_wrapper:process_wrapper`
on the gnullvm exec platform started linking with `lld-link.exe` and
then failed to resolve MinGW-style libraries such as `-lkernel32`,
`-luser32`, and `-lmingw32`.
What this PR does instead:
- The linker override is restricted to Windows MSVC targets only.
- The gnullvm path keeps its original linker behavior, while MSVC uses
the direct Windows linker.
## Attempt 3: Keep everything on pure Windows gnullvm and patch the V8 /
Python incompatibility chain instead
This would have preserved a single Windows ABI everywhere, but it is a
much larger project than this PR.
Why that was not the practical first step:
- The original incompatibility chain ran through exec-side generators
and helper tools, not only through crate code.
- `third_party/v8` is already special-cased on Windows gnullvm because
`rusty_v8` only publishes Windows prebuilts under MSVC names.
- Fixing that path likely means deeper changes in
V8/rules_python/rules_rust toolchain resolution and generator execution,
not just one local CI flag.
What this PR does instead:
- Keep gnullvm for the target cfgs we want to exercise.
- Move only the Windows test lane's host/exec platform to MSVC, then
patch the build-script/linker boundary enough for that split
configuration to work.
## Attempt 4: Validate compatibility with `bazel test --nobuild ...`
This turned out to be a misleading local validation command.
Why:
- `bazel test --nobuild ...` can successfully analyze targets and then
still exit 1 with "Couldn't start the build. Unable to run tests"
because there are no runnable test actions after `--nobuild`.
Better local check:
```powershell
bazel build --nobuild --keep_going --host_platform=//:local_windows_msvc //codex-rs/login:login-unit-tests //codex-rs/core:core-unit-tests //codex-rs/core:core-all-test
```
# Which patches probably deserve upstream follow-up
My rough take is that the `rules_rs` / `rules_rust` patches are the
highest-value upstream candidates, because they are fixing generic
Windows host/exec + MSVC direct-linker behavior rather than
Codex-specific test logic.
Strong upstream candidates:
- `patches/rules_rs_windows_exec_linker.patch`
- `patches/rules_rust_windows_bootstrap_process_wrapper_linker.patch`
- `patches/rules_rust_windows_build_script_runner_paths.patch`
- `patches/rules_rust_windows_exec_msvc_build_script_env.patch`
- `patches/rules_rust_windows_msvc_direct_link_args.patch`
- `patches/rules_rust_windows_process_wrapper_skip_temp_outputs.patch`
Why these seem upstreamable:
- They address general-purpose problems in the Windows MSVC exec path:
- missing direct-linker exposure for Rust toolchains
- wrong linker selection when rustc emits MSVC-style args
- Windows path normalization/short-path issues in the build-script
runner
- forwarding GNU-flavored CC/link flags into MSVC Cargo build scripts
- unstable temp outputs polluting process-wrapper search-path state
Potentially upstreamable crate patches, but likely with more care:
- `patches/zstd-sys_windows_msvc_include_dirs.patch`
- `patches/ring_windows_msvc_include_dirs.patch`
- `patches/aws-lc-sys_windows_msvc_prebuilt_nasm.patch`
Notes on those:
- The `zstd-sys` and `ring` include-path fixes look fairly generic for
MSVC/Bazel build-script environments and may be straightforward to
propose upstream after we confirm CI stability.
- The `aws-lc-sys` patch is useful, but it includes a Bazel-specific
environment probe and CI-specific compiler fallback behavior. That
probably needs a cleaner upstream-facing shape before sending it out, so
upstream maintainers are not forced to adopt Codex's exact CI
assumptions.
Probably not worth upstreaming as-is:
- The repo-local Starlark/test target changes in `defs.bzl`,
`codex-rs/*/BUILD.bazel`, and `.github/scripts/run-bazel-ci.sh` are
mostly Codex-specific policy and CI wiring, not generic rules changes.
# Validation notes for reviewers
On this branch, I ran the following local checks after dropping the
non-resource-loading Rust edits:
```powershell
cargo test -p codex-tui
just --shell 'C:\Program Files\Git\bin\bash.exe' --shell-arg -lc -- fix -p codex-tui
python .\tools\argument-comment-lint\run-prebuilt-linter.py -p codex-tui
just --shell 'C:\Program Files\Git\bin\bash.exe' --shell-arg -lc fmt
```
One local caveat:
- `just argument-comment-lint` still fails on this Windows machine for
an unrelated Bazel toolchain-resolution issue in
`//codex-rs/exec:exec-all-test`, so I used the direct prebuilt linter
for `codex-tui` as the local fallback.
# Expected reviewer takeaway
If this PR goes green, the important conclusion is that the Windows
Bazel test coverage gap was primarily a Bazel host/exec toolchain
problem, not a need to make the Rust tests themselves Windows-specific.
That would be a strong signal that the deleted non-resource-loading Rust
test edits from the earlier branch should stay out, and that future work
should focus on upstreaming the generic `rules_rs` / `rules_rust`
Windows fixes and reducing the crate-local patch surface.
## Why
The main Bazel CI lanes need compact execution logs to investigate cache
misses and unexpected rebuilds, but local users of the shared wrapper
should not pay that log-generation cost by default.
## What Changed
-
[`.github/scripts/run-bazel-ci.sh`](https://github.com/openai/codex/blob/a6ec239a24fd21507bb75a28f17277653f52e3a2/.github/scripts/run-bazel-ci.sh#L149-L153)
now appends `--execution_log_compact_file=...` only when
`CODEX_BAZEL_EXECUTION_LOG_COMPACT_DIR` is set; the caller owns creating
that directory.
-
[`.github/workflows/bazel.yml`](https://github.com/openai/codex/blob/a6ec239a24fd21507bb75a28f17277653f52e3a2/.github/workflows/bazel.yml#L66-L174)
enables that env var only for the main `test` and `clippy` jobs, creates
the temp log directory in each job, and uploads the resulting `*.zst`
files from `runner.temp`.
## Verification
- `bash -n .github/scripts/run-bazel-ci.sh`
- Parsed `.github/workflows/bazel.yml` as YAML.
- Ran a local opt-in wrapper smoke test and confirmed it writes
`execution-log-cquery-local-*.zst` when the caller pre-creates
`CODEX_BAZEL_EXECUTION_LOG_COMPACT_DIR`.
## Why
`argument-comment-lint` had become a PR bottleneck because the repo-wide
lane was still effectively running a `cargo dylint`-style flow across
the workspace instead of reusing Bazel's Rust dependency graph. That
kept the lint enforced, but it threw away the main benefit of moving
this job under Bazel in the first place: metadata reuse and cacheable
per-target analysis in the same shape as Clippy.
This change moves the repo-wide lint onto a native Bazel Rust aspect so
Linux and macOS can lint `codex-rs` without rebuilding the world
crate-by-crate through the wrapper path.
## What Changed
- add a nightly Rust toolchain with `rustc-dev` for Bazel and a
dedicated crate-universe repo for `tools/argument-comment-lint`
- add `tools/argument-comment-lint/driver.rs` and
`tools/argument-comment-lint/lint_aspect.bzl` so Bazel can run the lint
as a custom `rustc_driver`
- switch repo-wide `just argument-comment-lint` and the Linux/macOS
`rust-ci` lanes to `bazel build --config=argument-comment-lint
//codex-rs/...`
- keep the Python/DotSlash wrappers as the package-scoped fallback path
and as the current Windows CI path
- gate the Dylint entrypoint behind a `bazel_native` feature so the
Bazel-native library avoids the `dylint_*` packaging stack
- update the aspect runtime environment so the driver can locate
`rustc_driver` correctly under remote execution
- keep the dedicated `tools/argument-comment-lint` package tests and
wrapper unit tests in CI so the source and packaged entrypoints remain
covered
## Verification
- `python3 -m unittest discover -s tools/argument-comment-lint -p
'test_*.py'`
- `cargo test` in `tools/argument-comment-lint`
- `bazel build
//tools/argument-comment-lint:argument-comment-lint-driver
--@rules_rust//rust/toolchain/channel=nightly`
- `bazel build --config=argument-comment-lint
//codex-rs/utils/path-utils:all`
- `bazel build --config=argument-comment-lint
//codex-rs/rollout:rollout`
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16106).
* #16120
* __->__ #16106
## Why
We want more of the pre-merge Rust signal to come from `bazel.yml`,
especially on Windows. The Bazel test workflow already exercises
`x86_64-pc-windows-gnullvm`, but the Bazel clippy job still only ran on
Linux x64 and macOS arm64. That left a gap where Windows-only Bazel lint
breakages could slip through until the Cargo-based workflow ran.
This change keeps the fix narrow. Rather than expanding the Bazel clippy
target set or changing the shared setup logic, it extends the existing
clippy matrix to the same Windows GNU toolchain that the Bazel test job
already uses.
## What Changed
- add `windows-latest` / `x86_64-pc-windows-gnullvm` to the `clippy` job
matrix in `.github/workflows/bazel.yml`
- update the nearby workflow comment to explain that the goal is to get
Bazel-native Windows lint coverage on the same toolchain as the Bazel
test lane
- leave the Bazel clippy scope unchanged at `//codex-rs/...
-//codex-rs/v8-poc:all`
## Verification
- parsed `.github/workflows/bazel.yml` successfully with Ruby
`YAML.load_file`
## Why
This PR is the current, consolidated follow-up to the earlier Windows
Bazel attempt in #11229. The goal is no longer just to get a tiny
Windows smoke job limping along: it is to make the ordinary Bazel CI
path usable on `windows-latest` for `x86_64-pc-windows-gnullvm`, with
the same broad `//...` test shape that macOS and Linux already use.
The earlier smoke-list version of this work was useful as a foothold,
but it was not a good long-term landing point. Windows Bazel kept
surfacing real issues outside that allowlist:
- GitHub's Windows runner exposed runfiles-manifest bugs such as
`FINDSTR: Cannot open D:MANIFEST`, which broke Bazel test launchers even
when the manifest file existed.
- `rules_rs`, `rules_rust`, LLVM extraction, and Abseil still needed
`windows-gnullvm`-specific fixes for our hermetic toolchain.
- the V8 path needed more work than just turning the Windows matrix
entry back on: `rusty_v8` does not ship Windows GNU artifacts in the
same shape we need, and Bazel's in-tree V8 build needed a set of Windows
GNU portability fixes.
Windows performance pressure also pushed this toward a full solution
instead of a permanent smoke suite. During this investigation we hit
targets such as `//codex-rs/shell-command:shell-command-unit-tests` that
were much more expensive on Windows because they repeatedly spawn real
PowerShell parsers (see #16057 for one concrete example of that
pressure). That made it much more valuable to get the real Windows Bazel
path working than to keep iterating on a narrowly curated subset.
The net result is that this PR now aims for the same CI contract on
Windows that we already expect elsewhere: keep standalone
`//third_party/v8:all` out of the ordinary Bazel lane, but allow V8
consumers under `//codex-rs/...` to build and test transitively through
`//...`.
## What Changed
### CI and workflow wiring
- re-enable the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel
matrix entry in `.github/workflows/bazel.yml`
- move the Windows Bazel output root to `D:\b` and enable `git config
--global core.longpaths true` in
`.github/actions/setup-bazel-ci/action.yml`
- keep the ordinary Bazel target set on Windows aligned with macOS and
Linux by running `//...` while excluding only standalone
`//third_party/v8:all` targets from the normal lane
### Toolchain and module support for `windows-gnullvm`
- patch `rules_rs` so `windows-gnullvm` is modeled as a distinct Windows
exec/toolchain platform instead of collapsing into the generic Windows
shape
- patch `rules_rust` build-script environment handling so llvm-mingw
build-script probes do not inherit unsupported `-fstack-protector*`
flags
- patch the LLVM module archive so it extracts cleanly on Windows and
provides the MinGW libraries this toolchain needs
- patch Abseil so its thread-local identity path matches the hermetic
`windows-gnullvm` toolchain instead of taking an incompatible MinGW
pthread path
- keep both MSVC and GNU Windows targets in the generated Cargo metadata
because the current V8 release-asset story still uses MSVC-shaped names
in some places while the Bazel build targets the GNU ABI
### Windows test-launch and binary-behavior fixes
- update `workspace_root_test_launcher.bat.tpl` to read the runfiles
manifest directly instead of shelling out to `findstr`, which was the
source of the `D:MANIFEST` failures on the GitHub Windows runner
- thread a larger Windows GNU stack reserve through `defs.bzl` so
Bazel-built binaries that pull in V8 behave correctly both under normal
builds and under `bazel test`
- remove the no-longer-needed Windows bootstrap sh-toolchain override
from `.bazelrc`
### V8 / `rusty_v8` Windows GNU support
- export and apply the new Windows GNU patch set from
`patches/BUILD.bazel` / `MODULE.bazel`
- patch the V8 module/rules/source layers so the in-tree V8 build can
produce Windows GNU archives under Bazel
- teach `third_party/v8/BUILD.bazel` to build Windows GNU static
archives in-tree instead of aliasing them to the MSVC prebuilts
- reuse the Linux release binding for the experimental Windows GNU path
where `rusty_v8` does not currently publish a Windows GNU binding
artifact
## Testing
- the primary end-to-end validation for this work is the `Bazel`
workflow plus `v8-canary`, since the hard parts are Windows-specific and
depend on real GitHub runner behavior
- before consolidation back onto this PR, the same net change passed the
full Bazel matrix in [run
23675590471](https://github.com/openai/codex/actions/runs/23675590471)
and passed `v8-canary` in [run
23675590453](https://github.com/openai/codex/actions/runs/23675590453)
- those successful runs included the `windows-latest` /
`x86_64-pc-windows-gnullvm` Bazel job with the ordinary `//...` path,
not the earlier Windows smoke allowlist
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/15952).
* #16067
* __->__ #15952
## Why
`bazel.yml` already builds and tests the Bazel graph, but `rust-ci.yml`
still runs `cargo clippy` separately. This PR starts the transition to a
Bazel-backed lint lane for `codex-rs` so we can eventually replace the
duplicate Rust build, test, and lint work with Bazel while explicitly
keeping the V8 Bazel path out of scope for now.
To make that lane practical, the workflow also needs to look like the
Bazel job we already trust. That means sharing the common Bazel setup
and invocation logic instead of hand-copying it, and covering the arm64
macOS path in addition to Linux.
Landing the workflow green also required fixing the first lint findings
that Bazel surfaced and adding the matching local entrypoint.
## What changed
- add a reusable `build:clippy` config to `.bazelrc` and export
`codex-rs/clippy.toml` from `codex-rs/BUILD.bazel` so Bazel can run the
repository's existing Clippy policy
- add `just bazel-clippy` so the local developer entrypoint matches the
new CI lane
- extend `.github/workflows/bazel.yml` with a dedicated Bazel clippy job
for `codex-rs`, scoped to `//codex-rs/... -//codex-rs/v8-poc:all`
- run that clippy job on Linux x64 and arm64 macOS
- factor the shared Bazel workflow setup into
`.github/actions/setup-bazel-ci/action.yml` and the shared Bazel
invocation logic into `.github/scripts/run-bazel-ci.sh` so the clippy
and build/test jobs stay aligned
- fix the first Bazel-clippy findings needed to keep the lane green,
including the cross-target `cmsghdr::cmsg_len` normalization in
`codex-rs/shell-escalation/src/unix/socket.rs` and the no-`voice-input`
dead-code warnings in `codex-rs/tui` and `codex-rs/tui_app_server`
## Verification
- `just bazel-clippy`
- `RUNNER_OS=macOS ./.github/scripts/run-bazel-ci.sh -- build
--config=clippy --build_metadata=COMMIT_SHA=local-check
--build_metadata=TAG_job=clippy -- //codex-rs/...
-//codex-rs/v8-poc:all`
- `bazel build --config=clippy
//codex-rs/shell-escalation:shell-escalation`
- `CARGO_TARGET_DIR=/tmp/codex4-shell-escalation-test cargo test -p
codex-shell-escalation`
- `ruby -e 'require "yaml";
YAML.load_file(".github/workflows/bazel.yml");
YAML.load_file(".github/actions/setup-bazel-ci/action.yml")'`
## Notes
- `CARGO_TARGET_DIR=/tmp/codex4-tui-app-server-test cargo test -p
codex-tui-app-server` still hits existing guardian-approvals test and
snapshot failures unrelated to this PR's Bazel-clippy changes.
Related: #15954
Replaced ci.bazelrc and v8-ci.bazelrc by custom configs inside the main
.bazelrc file. As a result, github workflows setup is simplified down to
a single '--config=<foo>' flag usage.
Moved the build metadata flags to config=ci.
Added custom tags metadata to help differentiate invocations based on
workflow (bazel vs v8) and os (linux/macos/windows).
Enabled users to override the default values in .bazelrc by using a
user.bazelrc file locally.
Added user.bazelrc to gitignore.
Highlights:
- Trimmed down to just the repository cache for faster upload / download
- Made the cache key only include files that affect external
dependencies (since that's what the repository cache caches) -
MODULE.bazel, codex-rs/Cargo.lock, codex-rs/Cargo.toml
- Split the caching action in to explicit restore / save steps (similar
to your rust CI) which allows us to skip uploads on cache hit, and not
fail the build if upload fails
This should get rid of 842 network fetches that are happening on every
Bazel CI run, while also reducing the Github flakiness @bolinfest
reported. Uploading should be faster (since we're not caching many small
files), and will only happen when MODULE.bazel or Cargo.lock /
Cargo.toml files change.
In my testing, it [took 3s to save the repository
cache](https://github.com/siggisim/codex/actions/runs/23014186143/job/66832859781).
Alternative approach, we use rusty_v8 for all platforms that its
predefined, but lets build from source a musl v8 version with bazel for
x86 and aarch64 only. We would need to release this on github and then
use the release.
The other three Bazel builds have experienced low flakiness in my
experience whereas I find myself re-running the `ubuntu-24.04-arm` jobs
often to shake out the flakes. Disabling for now.
## Summary
Stabilize `js_repl` runtime test setup in CI and move tool-facing
`js_repl` behavior coverage into integration tests.
This is a test/CI change only. No production `js_repl` behavior change
is intended.
## Why
- Bazel test sandboxes (especially on macOS) could resolve a different
`node` than the one installed by `actions/setup-node`, which caused
`js_repl` runtime/version failures.
- `js_repl` runtime tests depend on platform-specific
sandbox/test-harness behavior, so they need explicit gating in a
base-stability commit.
- Several tests in the `js_repl` unit test module were actually
black-box/tool-level behavior tests and fit better in the integration
suite.
## Changes
- Add `actions/setup-node` to the Bazel and Rust `Tests` workflows,
using the exact version pinned in the repo’s Node version file.
- In Bazel (non-Windows), pass `CODEX_JS_REPL_NODE_PATH=$(which node)`
into test env so `js_repl` uses the `actions/setup-node` runtime inside
Bazel tests.
- Add a new integration test suite for `js_repl` tool behavior and
register it in the core integration test suite module.
- Move black-box `js_repl` behavior tests into the integration suite
(persistence/TLA, builtin tool invocation, recursive self-call
rejection, `process` isolation, blocked builtin imports).
- Keep white-box manager/kernel tests in the `js_repl` unit test module.
- Gate `js_repl` runtime tests to run only on macOS and only when a
usable Node runtime is available (skip on other platforms / missing Node
in this commit).
## Impact
- Reduces `js_repl` CI failures caused by Node resolution drift in
Bazel.
- Improves test organization by separating tool-facing behavior tests
from white-box manager/kernel tests.
- Keeps the base commit stable while expanding `js_repl` runtime
coverage.
#### [git stack](https://github.com/magus/git-stack-cli)
- ✅ `1` https://github.com/openai/codex/pull/12372
- 👉 `2` https://github.com/openai/codex/pull/12407
- ⏳ `3` https://github.com/openai/codex/pull/12185
- ⏳ `4` https://github.com/openai/codex/pull/10673
## Summary
Improve Bazel CI failure diagnostics by printing the tail of each failed
target’s test.log directly in the GitHub Actions output.
Today, when a large Bazel test target fails (for example tests of
`codex-core`), the workflow often only shows a target-level Exit 101
plus a path to Bazel’s test.log. That makes it hard to see the actual
failing Rust test and panic without digging into artifacts or
reproducing locally.
This change makes the workflow automatically surface that information
inline.
## What Changed
In .github/workflows/bazel.yml:
- Capture Bazel console output via tee
- Preserve the Bazel exit code when piping (PIPESTATUS[0])
- On failure:
- Parse failed Bazel test targets from FAIL: //... lines
- Resolve Bazel test log directory via bazel info bazel-testlogs
- Print tail -n 200 for each failed target’s test.log
- Group each target’s output in GitHub Actions logs (::group::)
## Bonus
Disable `experimental_remote_repo_contents_cache` to prevent "Permission
Denied"
## Why this change
When Cargo dependencies change, it is easy to end up with an unexpected
local diff in
`MODULE.bazel.lock` after running Bazel. That creates noisy working
copies and pushes lockfile fixes
later in the cycle. This change addresses that pain point directly.
## What this change enforces
The expected invariant is: after dependency updates, `MODULE.bazel.lock`
is already in sync with
Cargo resolution. In practice, running `bazel mod deps` should not
mutate the lockfile in a clean
state. If it does, the dependency update is incomplete.
## How this is enforced
This change adds a single lockfile check script that snapshots
`MODULE.bazel.lock`, runs
`bazel mod deps`, and fails if the file changes. The same check is wired
into local workflow
commands (`just bazel-lock-update` and `just bazel-lock-check`) and into
Bazel CI (Linux x86_64 job)
so drift is caught early and consistently. The developer documentation
is updated in
`codex-rs/docs/bazel.md` and `AGENTS.md` to make the expected flow
explicit.
`MODULE.bazel.lock` is also refreshed in this PR to match the current
Cargo dependency resolution.
## Expected developer workflow
After changing `Cargo.toml` or `Cargo.lock`, run `just
bazel-lock-update`, then run
`just bazel-lock-check`, and include any resulting `MODULE.bazel.lock`
update in the same change.
## Testing
Ran `just bazel-lock-check` locally.
## Summary
This PR removes the temporary `CODEX_BWRAP_ENABLE_FFI` flag and makes
Linux builds always compile vendored bubblewrap support for
`codex-linux-sandbox`.
## Changes
- Removed `CODEX_BWRAP_ENABLE_FFI` gating from
`codex-rs/linux-sandbox/build.rs`.
- Linux builds now fail fast if vendored bubblewrap compilation fails
(instead of warning and continuing).
- Updated fallback/help text in
`codex-rs/linux-sandbox/src/vendored_bwrap.rs` to remove references to
`CODEX_BWRAP_ENABLE_FFI`.
- Removed `CODEX_BWRAP_ENABLE_FFI` env wiring from:
- `.github/workflows/rust-ci.yml`
- `.github/workflows/bazel.yml`
- `.github/workflows/rust-release.yml`
---------
Co-authored-by: David Zbarsky <zbarsky@openai.com>
Adam Perry @ OpenAI ·
Channing Conger ·
Michael Bolin ·
starr-openai ·
William Woodruff ·
Curtis 'Fjord' Hawthorne ·
evawong-oai ·
viyatb-oai ·
Ruslan Nigmatullin ·
Drew Hintz ·
Son Luong Ngoc ·
Siggi Simonarson ·
jif-oai ·
Josh McKinney ·