Commit Graph

6 Commits

  • fix: pin inputs (#17471)
    ## Summary
    - Pin Rust git patch dependencies to immutable revisions and make
    cargo-deny reject unknown git and registry sources unless explicitly
    allowlisted.
    - Add checked-in SHA-256 coverage for the current rusty_v8 release
    assets, wire those hashes into Bazel, and verify CI override downloads
    before use.
    - Add rusty_v8 MODULE.bazel update/check tooling plus a Bazel CI guard
    so future V8 bumps cannot drift from the checked-in checksum manifest.
    - Pin release/lint cargo installs and all external GitHub Actions refs
    to immutable inputs.
    
    ## Future V8 bump flow
    Run these after updating the resolved `v8` crate version and checksum
    manifest:
    
    ```bash
    python3 .github/scripts/rusty_v8_bazel.py update-module-bazel
    python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
    ```
    
    The update command rewrites the matching `rusty_v8_<crate_version>`
    `http_file` SHA-256 values in `MODULE.bazel` from
    `third_party/v8/rusty_v8_<crate_version>.sha256`. The check command is
    also wired into Bazel CI to block drift.
    
    ## Notes
    - This intentionally excludes RustSec dependency upgrades and
    bubblewrap-related changes per request.
    - The branch was rebased onto the latest origin/main before opening the
    PR.
    
    ## Validation
    - cargo fetch --locked
    - cargo deny check advisories
    - cargo deny check
    - cargo deny check sources
    - python3 .github/scripts/rusty_v8_bazel.py check-module-bazel
    - python3 .github/scripts/rusty_v8_bazel.py update-module-bazel
    - python3 -m unittest discover -s .github/scripts -p
    'test_rusty_v8_bazel.py'
    - python3 -m py_compile .github/scripts/rusty_v8_bazel.py
    .github/scripts/rusty_v8_module_bazel.py
    .github/scripts/test_rusty_v8_bazel.py
    - repo-wide GitHub Actions `uses:` audit: all external action refs are
    pinned to 40-character SHAs
    - yq eval on touched workflows and local actions
    - git diff --check
    - just bazel-lock-check
    
    ## Hash verification
    - Confirmed `MODULE.bazel` hashes match
    `third_party/v8/rusty_v8_146_4_0.sha256`.
    - Confirmed GitHub release asset digests for denoland/rusty_v8
    `v146.4.0` and openai/codex `rusty-v8-v146.4.0` match the checked-in
    hashes.
    - Streamed and SHA-256 hashed all 10 `MODULE.bazel` rusty_v8 asset URLs
    locally; every downloaded byte stream matched both `MODULE.bazel` and
    the checked-in manifest.
    
    ## Pin verification
    - Confirmed signing-action pins match the peeled commits for their tag
    comments: `sigstore/cosign-installer@v3.7.0`, `azure/login@v2`, and
    `azure/trusted-signing-action@v0`.
    - Pinned the remaining tag-based action refs in Bazel CI/setup:
    `actions/setup-node@v6`, `facebook/install-dotslash@v2`,
    `bazelbuild/setup-bazelisk@v3`, and `actions/cache/restore@v5`.
    - Normalized all `bazelbuild/setup-bazelisk@v3` refs to the peeled
    commit behind the annotated tag.
    - Audited Cargo git dependencies: every manifest git dependency uses
    `rev` only, every `Cargo.lock` git source has `?rev=<sha>#<same-sha>`,
    and `cargo deny check sources` passes with `required-git-spec = "rev"`.
    - Shallow-fetched each distinct git dependency repo at its pinned SHA
    and verified Git reports each object as a commit.
  • bazel: enable the full Windows gnullvm CI path (#15952)
    ## Why
    
    This PR is the current, consolidated follow-up to the earlier Windows
    Bazel attempt in #11229. The goal is no longer just to get a tiny
    Windows smoke job limping along: it is to make the ordinary Bazel CI
    path usable on `windows-latest` for `x86_64-pc-windows-gnullvm`, with
    the same broad `//...` test shape that macOS and Linux already use.
    
    The earlier smoke-list version of this work was useful as a foothold,
    but it was not a good long-term landing point. Windows Bazel kept
    surfacing real issues outside that allowlist:
    
    - GitHub's Windows runner exposed runfiles-manifest bugs such as
    `FINDSTR: Cannot open D:MANIFEST`, which broke Bazel test launchers even
    when the manifest file existed.
    - `rules_rs`, `rules_rust`, LLVM extraction, and Abseil still needed
    `windows-gnullvm`-specific fixes for our hermetic toolchain.
    - the V8 path needed more work than just turning the Windows matrix
    entry back on: `rusty_v8` does not ship Windows GNU artifacts in the
    same shape we need, and Bazel's in-tree V8 build needed a set of Windows
    GNU portability fixes.
    
    Windows performance pressure also pushed this toward a full solution
    instead of a permanent smoke suite. During this investigation we hit
    targets such as `//codex-rs/shell-command:shell-command-unit-tests` that
    were much more expensive on Windows because they repeatedly spawn real
    PowerShell parsers (see #16057 for one concrete example of that
    pressure). That made it much more valuable to get the real Windows Bazel
    path working than to keep iterating on a narrowly curated subset.
    
    The net result is that this PR now aims for the same CI contract on
    Windows that we already expect elsewhere: keep standalone
    `//third_party/v8:all` out of the ordinary Bazel lane, but allow V8
    consumers under `//codex-rs/...` to build and test transitively through
    `//...`.
    
    ## What Changed
    
    ### CI and workflow wiring
    
    - re-enable the `windows-latest` / `x86_64-pc-windows-gnullvm` Bazel
    matrix entry in `.github/workflows/bazel.yml`
    - move the Windows Bazel output root to `D:\b` and enable `git config
    --global core.longpaths true` in
    `.github/actions/setup-bazel-ci/action.yml`
    - keep the ordinary Bazel target set on Windows aligned with macOS and
    Linux by running `//...` while excluding only standalone
    `//third_party/v8:all` targets from the normal lane
    
    ### Toolchain and module support for `windows-gnullvm`
    
    - patch `rules_rs` so `windows-gnullvm` is modeled as a distinct Windows
    exec/toolchain platform instead of collapsing into the generic Windows
    shape
    - patch `rules_rust` build-script environment handling so llvm-mingw
    build-script probes do not inherit unsupported `-fstack-protector*`
    flags
    - patch the LLVM module archive so it extracts cleanly on Windows and
    provides the MinGW libraries this toolchain needs
    - patch Abseil so its thread-local identity path matches the hermetic
    `windows-gnullvm` toolchain instead of taking an incompatible MinGW
    pthread path
    - keep both MSVC and GNU Windows targets in the generated Cargo metadata
    because the current V8 release-asset story still uses MSVC-shaped names
    in some places while the Bazel build targets the GNU ABI
    
    ### Windows test-launch and binary-behavior fixes
    
    - update `workspace_root_test_launcher.bat.tpl` to read the runfiles
    manifest directly instead of shelling out to `findstr`, which was the
    source of the `D:MANIFEST` failures on the GitHub Windows runner
    - thread a larger Windows GNU stack reserve through `defs.bzl` so
    Bazel-built binaries that pull in V8 behave correctly both under normal
    builds and under `bazel test`
    - remove the no-longer-needed Windows bootstrap sh-toolchain override
    from `.bazelrc`
    
    ### V8 / `rusty_v8` Windows GNU support
    
    - export and apply the new Windows GNU patch set from
    `patches/BUILD.bazel` / `MODULE.bazel`
    - patch the V8 module/rules/source layers so the in-tree V8 build can
    produce Windows GNU archives under Bazel
    - teach `third_party/v8/BUILD.bazel` to build Windows GNU static
    archives in-tree instead of aliasing them to the MSVC prebuilts
    - reuse the Linux release binding for the experimental Windows GNU path
    where `rusty_v8` does not currently publish a Windows GNU binding
    artifact
    
    ## Testing
    
    - the primary end-to-end validation for this work is the `Bazel`
    workflow plus `v8-canary`, since the hard parts are Windows-specific and
    depend on real GitHub runner behavior
    - before consolidation back onto this PR, the same net change passed the
    full Bazel matrix in [run
    23675590471](https://github.com/openai/codex/actions/runs/23675590471)
    and passed `v8-canary` in [run
    23675590453](https://github.com/openai/codex/actions/runs/23675590453)
    - those successful runs included the `windows-latest` /
    `x86_64-pc-windows-gnullvm` Bazel job with the ordinary `//...` path,
    not the earlier Windows smoke allowlist
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/15952).
    * #16067
    * __->__ #15952
  • Add v8-poc consumer of our new built v8 (#15203)
    This adds a dummy v8-poc project that in Cargo links against our
    prebuilt binaries and the ones provided by rusty_v8 for non musl
    platforms. This demonstrates that we can successfully link and use v8 on
    all platforms that we want to target.
    
    In bazel things are slightly more complicated. Since the libraries as
    published have libc++ linked in already we end up with a lot of double
    linked symbols if we try to use them in bazel land. Instead we fall back
    to building rusty_v8 and v8 from source (cached of course) on the
    platforms we ship to.
    
    There is likely some compatibility drift in the windows bazel builder
    that we'll need to reconcile before we can re-enable them. I'm happy to
    be on the hook to unwind that.
  • V8 Bazel Build (#15021)
    Alternative approach, we use rusty_v8 for all platforms that its
    predefined, but lets build from source a musl v8 version with bazel for
    x86 and aarch64 only. We would need to release this on github and then
    use the release.
  • Add feature-gated freeform js_repl core runtime (#10674)
    ## Summary
    
    This PR adds an **experimental, feature-gated `js_repl` core runtime**
    so models can execute JavaScript in a persistent REPL context across
    tool calls.
    
    The implementation integrates with existing feature gating, tool
    registration, prompt composition, config/schema docs, and tests.
    
    ## What changed
    
    - Added new experimental feature flag: `features.js_repl`.
    - Added freeform `js_repl` tool and companion `js_repl_reset` tool.
    - Gated tool availability behind `Feature::JsRepl`.
    - Added conditional prompt-section injection for JS REPL instructions
    via marker-based prompt processing.
    - Implemented JS REPL handlers, including freeform parsing and pragma
    support (timeout/reset controls).
    - Added runtime resolution order for Node:
      1. `CODEX_JS_REPL_NODE_PATH`
      2. `js_repl_node_path` in config
      3. `PATH`
    - Added JS runtime assets/version files and updated docs/schema.
    
    ## Why
    
    This enables richer agent workflows that require incremental JavaScript
    execution with preserved state, while keeping rollout safe behind an
    explicit feature flag.
    
    ## Testing
    
    Coverage includes:
    
    - Feature-flag gating behavior for tool exposure.
    - Freeform parser/pragma handling edge cases.
    - Runtime behavior (state persistence across calls and top-level `await`
    support).
    
    ## Usage
    
    ```toml
    [features]
    js_repl = true
    ```
    
    Optional runtime override:
    
    - `CODEX_JS_REPL_NODE_PATH`, or
    - `js_repl_node_path` in config.
    
    #### [git stack](https://github.com/magus/git-stack-cli)
    - 👉 `1` https://github.com/openai/codex/pull/10674
    -  `2` https://github.com/openai/codex/pull/10672
    -  `3` https://github.com/openai/codex/pull/10671
    -  `4` https://github.com/openai/codex/pull/10673
    -  `5` https://github.com/openai/codex/pull/10670
  • Vendor ConPtySystem (#7656)
    The repo we were depending on is very large and we need very small part
    of it.
    
    ---------
    
    Co-authored-by: Pavel <pavel@krymets.com>