## Why
`argument-comment-lint` had become a PR bottleneck because the repo-wide
lane was still effectively running a `cargo dylint`-style flow across
the workspace instead of reusing Bazel's Rust dependency graph. That
kept the lint enforced, but it threw away the main benefit of moving
this job under Bazel in the first place: metadata reuse and cacheable
per-target analysis in the same shape as Clippy.
This change moves the repo-wide lint onto a native Bazel Rust aspect so
Linux and macOS can lint `codex-rs` without rebuilding the world
crate-by-crate through the wrapper path.
## What Changed
- add a nightly Rust toolchain with `rustc-dev` for Bazel and a
dedicated crate-universe repo for `tools/argument-comment-lint`
- add `tools/argument-comment-lint/driver.rs` and
`tools/argument-comment-lint/lint_aspect.bzl` so Bazel can run the lint
as a custom `rustc_driver`
- switch repo-wide `just argument-comment-lint` and the Linux/macOS
`rust-ci` lanes to `bazel build --config=argument-comment-lint
//codex-rs/...`
- keep the Python/DotSlash wrappers as the package-scoped fallback path
and as the current Windows CI path
- gate the Dylint entrypoint behind a `bazel_native` feature so the
Bazel-native library avoids the `dylint_*` packaging stack
- update the aspect runtime environment so the driver can locate
`rustc_driver` correctly under remote execution
- keep the dedicated `tools/argument-comment-lint` package tests and
wrapper unit tests in CI so the source and packaged entrypoints remain
covered
## Verification
- `python3 -m unittest discover -s tools/argument-comment-lint -p
'test_*.py'`
- `cargo test` in `tools/argument-comment-lint`
- `bazel build
//tools/argument-comment-lint:argument-comment-lint-driver
--@rules_rust//rust/toolchain/channel=nightly`
- `bazel build --config=argument-comment-lint
//codex-rs/utils/path-utils:all`
- `bazel build --config=argument-comment-lint
//codex-rs/rollout:rollout`
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16106).
* #16120
* __->__ #16106
## Why
We want more of the pre-merge Rust signal to come from `bazel.yml`,
especially on Windows. The Bazel test workflow already exercises
`x86_64-pc-windows-gnullvm`, but the Bazel clippy job still only ran on
Linux x64 and macOS arm64. That left a gap where Windows-only Bazel lint
breakages could slip through until the Cargo-based workflow ran.
This change keeps the fix narrow. Rather than expanding the Bazel clippy
target set or changing the shared setup logic, it extends the existing
clippy matrix to the same Windows GNU toolchain that the Bazel test job
already uses.
## What Changed
- add `windows-latest` / `x86_64-pc-windows-gnullvm` to the `clippy` job
matrix in `.github/workflows/bazel.yml`
- update the nearby workflow comment to explain that the goal is to get
Bazel-native Windows lint coverage on the same toolchain as the Bazel
test lane
- leave the Bazel clippy scope unchanged at `//codex-rs/...
-//codex-rs/v8-poc:all`
## Verification
- parsed `.github/workflows/bazel.yml` successfully with Ruby
`YAML.load_file`
## Why
This PR is the current, consolidated follow-up to the earlier Windows
Bazel attempt in #11229. The goal is no longer just to get a tiny
Windows smoke job limping along: it is to make the ordinary Bazel CI
path usable on `windows-latest` for `x86_64-pc-windows-gnullvm`, with
the same broad `//...` test shape that macOS and Linux already use.
The earlier smoke-list version of this work was useful as a foothold,
but it was not a good long-term landing point. Windows Bazel kept
surfacing real issues outside that allowlist:
- GitHub's Windows runner exposed runfiles-manifest bugs such as
`FINDSTR: Cannot open D:MANIFEST`, which broke Bazel test launchers even
when the manifest file existed.
- `rules_rs`, `rules_rust`, LLVM extraction, and Abseil still needed
`windows-gnullvm`-specific fixes for our hermetic toolchain.
- the V8 path needed more work than just turning the Windows matrix
entry back on: `rusty_v8` does not ship Windows GNU artifacts in the
same shape we need, and Bazel's in-tree V8 build needed a set of Windows
GNU portability fixes.
Windows performance pressure also pushed this toward a full solution
instead of a permanent smoke suite. During this investigation we hit
targets such as `//codex-rs/shell-command:shell-command-unit-tests` that
were much more expensive on Windows because they repeatedly spawn real
PowerShell parsers (see #16057 for one concrete example of that
pressure). That made it much more valuable to get the real Windows Bazel
path working than to keep iterating on a narrowly curated subset.
The net result is that this PR now aims for the same CI contract on
Windows that we already expect elsewhere: keep standalone
`//third_party/v8:all` out of the ordinary Bazel lane, but allow V8
consumers under `//codex-rs/...` to build and test transitively through
`//...`.
## What Changed
### CI and workflow wiring
- re-enable the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel
matrix entry in `.github/workflows/bazel.yml`
- move the Windows Bazel output root to `D:\b` and enable `git config
--global core.longpaths true` in
`.github/actions/setup-bazel-ci/action.yml`
- keep the ordinary Bazel target set on Windows aligned with macOS and
Linux by running `//...` while excluding only standalone
`//third_party/v8:all` targets from the normal lane
### Toolchain and module support for `windows-gnullvm`
- patch `rules_rs` so `windows-gnullvm` is modeled as a distinct Windows
exec/toolchain platform instead of collapsing into the generic Windows
shape
- patch `rules_rust` build-script environment handling so llvm-mingw
build-script probes do not inherit unsupported `-fstack-protector*`
flags
- patch the LLVM module archive so it extracts cleanly on Windows and
provides the MinGW libraries this toolchain needs
- patch Abseil so its thread-local identity path matches the hermetic
`windows-gnullvm` toolchain instead of taking an incompatible MinGW
pthread path
- keep both MSVC and GNU Windows targets in the generated Cargo metadata
because the current V8 release-asset story still uses MSVC-shaped names
in some places while the Bazel build targets the GNU ABI
### Windows test-launch and binary-behavior fixes
- update `workspace_root_test_launcher.bat.tpl` to read the runfiles
manifest directly instead of shelling out to `findstr`, which was the
source of the `D:MANIFEST` failures on the GitHub Windows runner
- thread a larger Windows GNU stack reserve through `defs.bzl` so
Bazel-built binaries that pull in V8 behave correctly both under normal
builds and under `bazel test`
- remove the no-longer-needed Windows bootstrap sh-toolchain override
from `.bazelrc`
### V8 / `rusty_v8` Windows GNU support
- export and apply the new Windows GNU patch set from
`patches/BUILD.bazel` / `MODULE.bazel`
- patch the V8 module/rules/source layers so the in-tree V8 build can
produce Windows GNU archives under Bazel
- teach `third_party/v8/BUILD.bazel` to build Windows GNU static
archives in-tree instead of aliasing them to the MSVC prebuilts
- reuse the Linux release binding for the experimental Windows GNU path
where `rusty_v8` does not currently publish a Windows GNU binding
artifact
## Testing
- the primary end-to-end validation for this work is the `Bazel`
workflow plus `v8-canary`, since the hard parts are Windows-specific and
depend on real GitHub runner behavior
- before consolidation back onto this PR, the same net change passed the
full Bazel matrix in [run
23675590471](https://github.com/openai/codex/actions/runs/23675590471)
and passed `v8-canary` in [run
23675590453](https://github.com/openai/codex/actions/runs/23675590453)
- those successful runs included the `windows-latest` /
`x86_64-pc-windows-gnullvm` Bazel job with the ordinary `//...` path,
not the earlier Windows smoke allowlist
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/15952).
* #16067
* __->__ #15952
## Why
`bazel.yml` already builds and tests the Bazel graph, but `rust-ci.yml`
still runs `cargo clippy` separately. This PR starts the transition to a
Bazel-backed lint lane for `codex-rs` so we can eventually replace the
duplicate Rust build, test, and lint work with Bazel while explicitly
keeping the V8 Bazel path out of scope for now.
To make that lane practical, the workflow also needs to look like the
Bazel job we already trust. That means sharing the common Bazel setup
and invocation logic instead of hand-copying it, and covering the arm64
macOS path in addition to Linux.
Landing the workflow green also required fixing the first lint findings
that Bazel surfaced and adding the matching local entrypoint.
## What changed
- add a reusable `build:clippy` config to `.bazelrc` and export
`codex-rs/clippy.toml` from `codex-rs/BUILD.bazel` so Bazel can run the
repository's existing Clippy policy
- add `just bazel-clippy` so the local developer entrypoint matches the
new CI lane
- extend `.github/workflows/bazel.yml` with a dedicated Bazel clippy job
for `codex-rs`, scoped to `//codex-rs/... -//codex-rs/v8-poc:all`
- run that clippy job on Linux x64 and arm64 macOS
- factor the shared Bazel workflow setup into
`.github/actions/setup-bazel-ci/action.yml` and the shared Bazel
invocation logic into `.github/scripts/run-bazel-ci.sh` so the clippy
and build/test jobs stay aligned
- fix the first Bazel-clippy findings needed to keep the lane green,
including the cross-target `cmsghdr::cmsg_len` normalization in
`codex-rs/shell-escalation/src/unix/socket.rs` and the no-`voice-input`
dead-code warnings in `codex-rs/tui` and `codex-rs/tui_app_server`
## Verification
- `just bazel-clippy`
- `RUNNER_OS=macOS ./.github/scripts/run-bazel-ci.sh -- build
--config=clippy --build_metadata=COMMIT_SHA=local-check
--build_metadata=TAG_job=clippy -- //codex-rs/...
-//codex-rs/v8-poc:all`
- `bazel build --config=clippy
//codex-rs/shell-escalation:shell-escalation`
- `CARGO_TARGET_DIR=/tmp/codex4-shell-escalation-test cargo test -p
codex-shell-escalation`
- `ruby -e 'require "yaml";
YAML.load_file(".github/workflows/bazel.yml");
YAML.load_file(".github/actions/setup-bazel-ci/action.yml")'`
## Notes
- `CARGO_TARGET_DIR=/tmp/codex4-tui-app-server-test cargo test -p
codex-tui-app-server` still hits existing guardian-approvals test and
snapshot failures unrelated to this PR's Bazel-clippy changes.
Related: #15954
Replaced ci.bazelrc and v8-ci.bazelrc by custom configs inside the main
.bazelrc file. As a result, github workflows setup is simplified down to
a single '--config=<foo>' flag usage.
Moved the build metadata flags to config=ci.
Added custom tags metadata to help differentiate invocations based on
workflow (bazel vs v8) and os (linux/macos/windows).
Enabled users to override the default values in .bazelrc by using a
user.bazelrc file locally.
Added user.bazelrc to gitignore.
Highlights:
- Trimmed down to just the repository cache for faster upload / download
- Made the cache key only include files that affect external
dependencies (since that's what the repository cache caches) -
MODULE.bazel, codex-rs/Cargo.lock, codex-rs/Cargo.toml
- Split the caching action in to explicit restore / save steps (similar
to your rust CI) which allows us to skip uploads on cache hit, and not
fail the build if upload fails
This should get rid of 842 network fetches that are happening on every
Bazel CI run, while also reducing the Github flakiness @bolinfest
reported. Uploading should be faster (since we're not caching many small
files), and will only happen when MODULE.bazel or Cargo.lock /
Cargo.toml files change.
In my testing, it [took 3s to save the repository
cache](https://github.com/siggisim/codex/actions/runs/23014186143/job/66832859781).
Alternative approach, we use rusty_v8 for all platforms that its
predefined, but lets build from source a musl v8 version with bazel for
x86 and aarch64 only. We would need to release this on github and then
use the release.
The other three Bazel builds have experienced low flakiness in my
experience whereas I find myself re-running the `ubuntu-24.04-arm` jobs
often to shake out the flakes. Disabling for now.
## Summary
Stabilize `js_repl` runtime test setup in CI and move tool-facing
`js_repl` behavior coverage into integration tests.
This is a test/CI change only. No production `js_repl` behavior change
is intended.
## Why
- Bazel test sandboxes (especially on macOS) could resolve a different
`node` than the one installed by `actions/setup-node`, which caused
`js_repl` runtime/version failures.
- `js_repl` runtime tests depend on platform-specific
sandbox/test-harness behavior, so they need explicit gating in a
base-stability commit.
- Several tests in the `js_repl` unit test module were actually
black-box/tool-level behavior tests and fit better in the integration
suite.
## Changes
- Add `actions/setup-node` to the Bazel and Rust `Tests` workflows,
using the exact version pinned in the repo’s Node version file.
- In Bazel (non-Windows), pass `CODEX_JS_REPL_NODE_PATH=$(which node)`
into test env so `js_repl` uses the `actions/setup-node` runtime inside
Bazel tests.
- Add a new integration test suite for `js_repl` tool behavior and
register it in the core integration test suite module.
- Move black-box `js_repl` behavior tests into the integration suite
(persistence/TLA, builtin tool invocation, recursive self-call
rejection, `process` isolation, blocked builtin imports).
- Keep white-box manager/kernel tests in the `js_repl` unit test module.
- Gate `js_repl` runtime tests to run only on macOS and only when a
usable Node runtime is available (skip on other platforms / missing Node
in this commit).
## Impact
- Reduces `js_repl` CI failures caused by Node resolution drift in
Bazel.
- Improves test organization by separating tool-facing behavior tests
from white-box manager/kernel tests.
- Keeps the base commit stable while expanding `js_repl` runtime
coverage.
#### [git stack](https://github.com/magus/git-stack-cli)
- ✅ `1` https://github.com/openai/codex/pull/12372
- 👉 `2` https://github.com/openai/codex/pull/12407
- ⏳ `3` https://github.com/openai/codex/pull/12185
- ⏳ `4` https://github.com/openai/codex/pull/10673
## Summary
Improve Bazel CI failure diagnostics by printing the tail of each failed
target’s test.log directly in the GitHub Actions output.
Today, when a large Bazel test target fails (for example tests of
`codex-core`), the workflow often only shows a target-level Exit 101
plus a path to Bazel’s test.log. That makes it hard to see the actual
failing Rust test and panic without digging into artifacts or
reproducing locally.
This change makes the workflow automatically surface that information
inline.
## What Changed
In .github/workflows/bazel.yml:
- Capture Bazel console output via tee
- Preserve the Bazel exit code when piping (PIPESTATUS[0])
- On failure:
- Parse failed Bazel test targets from FAIL: //... lines
- Resolve Bazel test log directory via bazel info bazel-testlogs
- Print tail -n 200 for each failed target’s test.log
- Group each target’s output in GitHub Actions logs (::group::)
## Bonus
Disable `experimental_remote_repo_contents_cache` to prevent "Permission
Denied"
## Why this change
When Cargo dependencies change, it is easy to end up with an unexpected
local diff in
`MODULE.bazel.lock` after running Bazel. That creates noisy working
copies and pushes lockfile fixes
later in the cycle. This change addresses that pain point directly.
## What this change enforces
The expected invariant is: after dependency updates, `MODULE.bazel.lock`
is already in sync with
Cargo resolution. In practice, running `bazel mod deps` should not
mutate the lockfile in a clean
state. If it does, the dependency update is incomplete.
## How this is enforced
This change adds a single lockfile check script that snapshots
`MODULE.bazel.lock`, runs
`bazel mod deps`, and fails if the file changes. The same check is wired
into local workflow
commands (`just bazel-lock-update` and `just bazel-lock-check`) and into
Bazel CI (Linux x86_64 job)
so drift is caught early and consistently. The developer documentation
is updated in
`codex-rs/docs/bazel.md` and `AGENTS.md` to make the expected flow
explicit.
`MODULE.bazel.lock` is also refreshed in this PR to match the current
Cargo dependency resolution.
## Expected developer workflow
After changing `Cargo.toml` or `Cargo.lock`, run `just
bazel-lock-update`, then run
`just bazel-lock-check`, and include any resulting `MODULE.bazel.lock`
update in the same change.
## Testing
Ran `just bazel-lock-check` locally.
## Summary
This PR removes the temporary `CODEX_BWRAP_ENABLE_FFI` flag and makes
Linux builds always compile vendored bubblewrap support for
`codex-linux-sandbox`.
## Changes
- Removed `CODEX_BWRAP_ENABLE_FFI` gating from
`codex-rs/linux-sandbox/build.rs`.
- Linux builds now fail fast if vendored bubblewrap compilation fails
(instead of warning and continuing).
- Updated fallback/help text in
`codex-rs/linux-sandbox/src/vendored_bwrap.rs` to remove references to
`CODEX_BWRAP_ENABLE_FFI`.
- Removed `CODEX_BWRAP_ENABLE_FFI` env wiring from:
- `.github/workflows/rust-ci.yml`
- `.github/workflows/bazel.yml`
- `.github/workflows/rust-release.yml`
---------
Co-authored-by: David Zbarsky <zbarsky@openai.com>
This is in response to seeing this on BuildBuddy:
> There were tests whose specified size is too big. Use the
--test_verbose_timeout_warnings command line option to see which ones
these are.
## Summary
- detect whether BUILDBUDDY_API_KEY is present in Bazel CI
- keep existing remote BuildBuddy path when key is available
- add a local fallback path for fork PRs without secrets by clearing
remote cache/executor/BES endpoints
- document each fallback flag inline with links to Bazel docs
## Testing
- ruby -e 'require "yaml";
YAML.load_file(".github/workflows/bazel.yml"); puts "ok"'
- verified Bazel docs/flag references used in workflow comments
## Summary
This PR introduces a gated Bubblewrap (bwrap) Linux sandbox path. The
curent Linux sandbox path relies on in-process restrictions (including
Landlock). Bubblewrap gives us a more uniform filesystem isolation
model, especially explicit writable roots with the option to make some
directories read-only and granular network controls.
This is behind a feature flag so we can validate behavior safely before
making it the default.
- Added temporary rollout flag:
- `features.use_linux_sandbox_bwrap`
- Preserved existing default path when the flag is off.
- In Bubblewrap mode:
- Added internal retry without /proc when /proc mount is not permitted
by the host/container.
## Summary
Upgrade GitHub Actions to their latest versions to ensure compatibility
with Node 24, as Node 20 will reach end-of-life in April 2026.
## Changes
| Action | Old Version(s) | New Version | Release | Files |
|--------|---------------|-------------|---------|-------|
| `actions/cache` |
[`v4`](https://github.com/actions/cache/releases/tag/v4) |
[`v5`](https://github.com/actions/cache/releases/tag/v5) |
[Release](https://github.com/actions/cache/releases/tag/v5) | bazel.yml
|
## Context
Per [GitHub's
announcement](https://github.blog/changelog/2025-09-19-deprecation-of-node-20-on-github-actions-runners/),
Node 20 is being deprecated and runners will begin using Node 24 by
default starting March 4th, 2026.
### Why this matters
- **Node 20 EOL**: April 2026
- **Node 24 default**: March 4th, 2026
- **Action**: Update to latest action versions that support Node 24
### Security Note
Actions that were previously pinned to commit SHAs remain pinned to SHAs
(updated to the latest release SHA) to maintain the security benefits of
immutable references.
### Testing
These changes only affect CI/CD workflow configurations and should not
impact application functionality. The workflows should be tested by
running them on a branch before merging.
Signed-off-by: Salman Muin Kayser Chishti <13schishti@gmail.com>
As explained in `codex-rs/core/BUILD.bazel`, including the repo's own
`AGENTS.md` is a hack to get some tests passing. We should fix this
properly, but I wanted to put stake in the ground ASAP to get `just
bazel-remote-test` working and then add a job to `bazel.yml` to ensure
it keeps working.
This PR configures Codex CLI so it can be built with
[Bazel](https://bazel.build) in addition to Cargo. The `.bazelrc`
includes configuration so that remote builds can be done using
[BuildBuddy](https://www.buildbuddy.io).
If you are familiar with Bazel, things should work as you expect, e.g.,
run `bazel test //... --keep-going` to run all the tests in the repo,
but we have also added some new aliases in the `justfile` for
convenience:
- `just bazel-test` to run tests locally
- `just bazel-remote-test` to run tests remotely (currently, the remote
build is for x86_64 Linux regardless of your host platform). Note we are
currently seeing the following test failures in the remote build, so we
still need to figure out what is happening here:
```
failures:
suite::compact::manual_compact_twice_preserves_latest_user_messages
suite::compact_resume_fork::compact_resume_after_second_compaction_preserves_history
suite::compact_resume_fork::compact_resume_and_fork_preserve_model_history_view
```
- `just build-for-release` to build release binaries for all
platforms/architectures remotely
To setup remote execution:
- [Create a buildbuddy account](https://app.buildbuddy.io/) (OpenAI
employees should also request org access at
https://openai.buildbuddy.io/join/ with their `@openai.com` email
address.)
- [Copy your API key](https://app.buildbuddy.io/docs/setup/) to
`~/.bazelrc` (add the line `build
--remote_header=x-buildbuddy-api-key=YOUR_KEY`)
- Use `--config=remote` in your `bazel` invocations (or add `common
--config=remote` to your `~/.bazelrc`, or use the `just` commands)
## CI
In terms of CI, this PR introduces `.github/workflows/bazel.yml`, which
uses Bazel to run the tests _locally_ on Mac and Linux GitHub runners
(we are working on supporting Windows, but that is not ready yet). Note
that the failures we are seeing in `just bazel-remote-test` do not occur
on these GitHub CI jobs, so everything in `.github/workflows/bazel.yml`
is green right now.
The `bazel.yml` uses extra config in `.github/workflows/ci.bazelrc` so
that macOS CI jobs build _remotely_ on Linux hosts (using the
`docker://docker.io/mbolin491/codex-bazel` Docker image declared in the
root `BUILD.bazel`) using cross-compilation to build the macOS
artifacts. Then these artifacts are downloaded locally to GitHub's macOS
runner so the tests can be executed natively. This is the relevant
config that enables this:
```
common:macos --config=remote
common:macos --strategy=remote
common:macos --strategy=TestRunner=darwin-sandbox,local
```
Because of the remote caching benefits we get from BuildBuddy, these new
CI jobs can be extremely fast! For example, consider these two jobs that
ran all the tests on Linux x86_64:
- Bazel 1m37s
https://github.com/openai/codex/actions/runs/20861063212/job/59940545209?pr=8875
- Cargo 9m20s
https://github.com/openai/codex/actions/runs/20861063192/job/59940559592?pr=8875
For now, we will continue to run both the Bazel and Cargo jobs for PRs,
but once we add support for Windows and running Clippy, we should be
able to cutover to using Bazel exclusively for PRs, which should still
speed things up considerably. We will probably continue to run the Cargo
jobs post-merge for commits that land on `main` as a sanity check.
Release builds will also continue to be done by Cargo for now.
Earlier attempt at this PR: https://github.com/openai/codex/pull/8832
Earlier attempt to add support for Buck2, now abandoned:
https://github.com/openai/codex/pull/8504
---------
Co-authored-by: David Zbarsky <dzbarsky@gmail.com>
Co-authored-by: Michael Bolin <mbolin@openai.com>
Michael Bolin ·
Drew Hintz ·
Son Luong Ngoc ·
Siggi Simonarson ·
Channing Conger ·
Curtis 'Fjord' Hawthorne ·
jif-oai ·
Josh McKinney ·
viyatb-oai ·
Salman Chishti ·
zbarsky-openai ·