## Why
`rust-release` now publishes `codex-package-<target>.tar.gz` as the
canonical native package payload. npm staging should consume those
archives directly instead of keeping legacy synthesis code that fetched
`rg`, copied standalone binaries, and rebuilt an approximate package
layout.
That also means the package builder should not know the internal shape
of `codex-package`. It should extract and copy the target payload
wholesale so future layout changes stay localized to the archive
producer.
The release job stages `codex`, `codex-responses-api-proxy`, and
`codex-sdk` together, so native artifact download should be filtered,
observable, and shared across component installs. Since that native
hydration is now only used by release staging, keeping a separate
`install_native_deps.py` CLI adds an extra wrapper without a real
caller.
## What Changed
- Removed legacy `codex-package` synthesis and related compatibility
flags from npm staging.
- Folded the remaining native artifact hydration code into
`scripts/stage_npm_packages.py` and deleted
`codex-cli/scripts/install_native_deps.py`.
- Made platform package staging copy the full extracted target directory
instead of enumerating package entries.
- Kept non-`codex-package` native components under their component
directory name instead of using a legacy destination map.
- Split native staging by component set while sharing one
workflow-artifact cache across the invocation.
- Changed workflow artifact download to select target artifacts by name,
print sizes/progress, and reuse cached artifacts.
- Removed the implicit `CI=true` default from `build_npm_package.py`;
local CI-shaped runs should set that environment explicitly.
- Kept `npm pack` cache/log output in its temporary directory so packing
does not write to the user npm cache.
## Verification
- `python3 -m py_compile scripts/stage_npm_packages.py
codex-cli/scripts/build_npm_package.py`
- `python3 -m unittest discover -s scripts/codex_package -p "test_*.py"`
- `scripts/stage_npm_packages.py --help`
- `codex-cli/scripts/build_npm_package.py --help`
- Ran the release-shaped staging command from `rust-release.yml` against
workflow run https://github.com/openai/codex/actions/runs/26240748758
with `CI=true` set locally to match GitHub Actions:
```sh
CI=true python3 ./scripts/stage_npm_packages.py \
--release-version 0.133.0 \
--workflow-url https://github.com/openai/codex/actions/runs/26240748758 \
--package codex \
--package codex-responses-api-proxy \
--package codex-sdk
```
That completed successfully, downloaded only the six target artifacts
once, reused the cache for `codex-responses-api-proxy`, and produced all
nine npm tarballs. Generated tarballs and staging/artifact temp dirs
were cleaned afterward.
## Why
Installing `@openai/codex` currently places a Dotslash `rg` manifest at
`node_modules/@openai/codex/bin/rg`, even though the native optional
dependency already ships the actual helper under
`vendor/<target>/codex-path/rg`. The launcher prepends that `codex-path`
directory, so the top-level `bin/rg` file is redundant in the npm
install.
The remaining direct consumers of the manifest are package-building
paths: `scripts/codex_package/ripgrep.py` and
`codex-cli/scripts/install_native_deps.py`. Keeping the manifest under
`codex-cli/bin` makes it look like a shipped npm binary, so this moves
it next to the package-builder code that owns it. The checked-in
`@openai/codex` package metadata should likewise describe only the meta
package payload; generated platform packages continue to publish
`vendor`.
## What Changed
- Moved the Dotslash ripgrep manifest from `codex-cli/bin/rg` to
`scripts/codex_package/rg`.
- Updated the package builder, npm native-artifact hydrator, README, and
CLI help text to reference the new manifest location.
- Stopped `codex-cli/scripts/build_npm_package.py` from copying `rg`
into the `@openai/codex` meta package.
- Narrowed the checked-in meta package `files` whitelist to
`bin/codex.js`.
## Verification
- `python3 -m unittest discover -s scripts/codex_package -p "test_*.py"`
- `python3 -m unittest discover -s codex-cli/scripts -p "test_*.py"`
- `python3 -m py_compile codex-cli/scripts/build_npm_package.py
codex-cli/scripts/install_native_deps.py
scripts/codex_package/ripgrep.py scripts/codex_package/cli.py
scripts/stage_npm_packages.py`
- `codex-cli/scripts/build_npm_package.py --package codex --version
0.0.0-test --pack-output <tmp>/codex-meta-no-vendor.tgz`
- `tar -tf <tmp>/codex-meta-no-vendor.tgz` showed only
`package/bin/codex.js`, `package/package.json`, and `package/README.md`.
- Direct staging check showed `codex` uses `files: ["bin/codex.js"]`
while `codex-darwin-arm64` still uses `files: ["vendor"]`.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23833).
* #23836
* __->__ #23833
## Summary
The npm platform packages should stop carrying a bespoke native layout
now that the release workflow builds canonical Codex package archives.
Keeping npm on the same `bin/`, `codex-resources/`, and `codex-path/`
structure lets the Rust package-layout detection behave consistently
across standalone, npm, and future DotSlash installs.
This changes platform npm packages to stage the `codex-package` artifact
for each target under `vendor/<target>`. The Node launcher now resolves
`bin/codex` and prepends `codex-path`, while retaining legacy
`vendor/<target>/codex` and `vendor/<target>/path` fallback support for
local development and migration. The npm staging helper downloads
`codex-package` archives instead of rebuilding the CLI payload from
individual `codex`, `rg`, `bwrap`, and sandbox helper artifacts.
CI still needs to stage npm packages from historical rust-release
workflow artifacts that predate package archives, so the staging scripts
expose an explicit `--allow-legacy-codex-package` fallback. That
fallback synthesizes the canonical package layout from legacy per-binary
artifacts and is wired only into the CI smoke path; release staging
remains strict and continues to require real package archives.
For direct local use, `install_native_deps.py` now points its built-in
default workflow at the same recent artifact run used by CI and
automatically enables legacy package synthesis only when
`--workflow-url` is omitted. Explicit workflow URLs remain strict unless
callers opt in with `--allow-legacy-codex-package`.
## Test plan
- `python3 -m py_compile codex-cli/scripts/build_npm_package.py
codex-cli/scripts/install_native_deps.py scripts/stage_npm_packages.py
scripts/codex_package/cli.py`
- `node --check codex-cli/bin/codex.js`
- `ruby -e 'require "yaml";
YAML.load_file(".github/workflows/rust-release.yml");
YAML.load_file(".github/workflows/ci.yml"); puts "ok"'`
- Staged a synthetic `codex-linux-x64` platform package from a canonical
vendor tree and verified it copied only `bin/`, `codex-path/`,
`codex-resources/`, and `codex-package.json`.
- Imported `install_native_deps.py` and extracted a synthetic
`codex-package-x86_64-unknown-linux-musl.tar.gz` into `vendor/<target>`.
- Ran legacy-layout conversion smokes for Linux, Windows, and unsigned
macOS artifact naming.
- Ran a synthetic `install_native_deps.py` default-workflow smoke that
verifies legacy package synthesis is automatic only when
`--workflow-url` is omitted.
- `NPM_CONFIG_CACHE="$tmp_dir/npm-cache" python3
./scripts/stage_npm_packages.py --release-version 0.125.0 --workflow-url
https://github.com/openai/codex/actions/runs/26131514935 --package codex
--allow-legacy-codex-package --output-dir "$tmp_dir"`
- `node codex-cli/bin/codex.js --version`
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23637).
* #23638
* __->__ #23637
## Summary
Starts the process of getting rid of `--full-auto`, with some
concessions:
1. Fully removes the command from the tui, since it just resolves to the
default permissions there, and encourages users to use the one-time
trust flow if they're not in a trusted repo.
2. Marks the command as deprecated in `codex exec`, in case users are
actively relying on this. We'll remove in an upcoming n+X release.
3. Cleans up some of the `codex sandbox` cli logic, to keep supporting
legacy sandbox policies for now.
This isn't the cleanest setup, but I think it is worthwhile to warn
users for one release before hard-removing it.
## Testing
- [x] Updated unit tests
## Summary
This PR hardens package-manager usage across the repo to reduce
dependency supply-chain risk. It also removes the stale `codex-cli`
Docker path, which was already broken on `main`, instead of keeping a
bitrotted container workflow alive.
## What changed
- Updated pnpm package manager pins and workspace install settings.
- Removed stale `codex-cli` Docker assets instead of trying to keep a
broken local container path alive.
- Added uv settings and lockfiles for the Python SDK packages.
- Updated Python SDK setup docs to use `uv sync`.
## Why
This is primarily a security hardening change. It reduces
package-install and supply-chain risk by ensuring dependency installs go
through pinned package managers, committed lockfiles, release-age
settings, and reviewed build-script controls.
For `codex-cli`, the right follow-up was to remove the local Docker path
rather than keep patching it:
- `codex-cli/Dockerfile` installed `codex.tgz` with `npm install -g`,
which bypassed the repo lockfile and age-gated pnpm settings.
- The local `codex-cli/scripts/build_container.sh` helper was already
broken on `main`: it called `pnpm run build`, but
`codex-cli/package.json` does not define a `build` script.
- The container path itself had bitrotted enough that keeping it would
require extra packaging-specific behavior that was not otherwise needed
by the repo.
## Gaps addressed
- Global npm installs bypassed the repo lockfile in Docker and CLI
reinstall paths, including `codex-cli/Dockerfile` and
`codex-cli/bin/codex.js`.
- CI and Docker pnpm installs used `--frozen-lockfile`, but the repo was
missing stricter pnpm workspace settings for dependency build scripts.
- Python SDK projects had `pyproject.toml` metadata but no committed
`uv.lock` coverage or uv age/index settings in `sdk/python` and
`sdk/python-runtime`.
- The secure devcontainer install path used npm/global install behavior
without a local locked package-manager boundary.
- The local `codex-cli` Docker helper was already broken on `main`, so
this PR removes that stale Docker path instead of preserving a broken
surface.
- pnpm was already pinned, but not to the current repo-wide pnpm version
target.
## Verification
- `pnpm install --frozen-lockfile`
- `.devcontainer/codex-install`: `pnpm install --prod --frozen-lockfile`
- `.devcontainer/codex-install`: `./node_modules/.bin/codex --version`
- `sdk/python`: `uv lock --check`, `uv sync --locked --all-extras
--dry-run`, `uv build`
- `sdk/python-runtime`: `uv lock --check`, `uv sync --locked --dry-run`,
`uv build --wheel`
- `pnpm -r --filter ./sdk/typescript run build`
- `pnpm -r --filter ./sdk/typescript run lint`
- `pnpm -r --filter ./sdk/typescript run test`
- `node --check codex-cli/bin/codex.js`
- `docker build -f .devcontainer/Dockerfile.secure -t codex-secure-test
.`
- `cargo build -p codex-cli`
- repo-wide package-manager audit
https://github.com/openai/codex/pull/11318 introduced logic to publish
platform artifacts as separate npm packages (for example,
`@openai/codex-darwin-arm64`, `@openai/codex-linux-x64`, etc.). That
requires provisioning and maintaining multiple package entries in npm,
which we want to avoid.
We still need to keep the package-size mitigation (platform-specific
payloads), but we want that layout to live under a single npm package
namespace (`@openai/codex`) using dist-tags.
We also need to preserve pre-release workflows where users install
`@openai/codex@alpha` and get platform-appropriate binaries.
Additionally, we want GitHub Release assets to group Codex npm tarballs
together, so platform tarballs should follow the same `codex-npm-*`
filename prefix as the main Codex tarball.
## Release Strategy (New Scheme)
We publish **one npm package name for Codex binaries** (`@openai/codex`)
and use **dist-tags** to select platform-specific payloads. This avoids
creating separate platform package names while keeping the package size
split by platform.
### What gets published
#### Mainline release (`x.y.z`)
- `@openai/codex@latest` (meta package)
- `@openai/codex@darwin-arm64`
- `@openai/codex@darwin-x64`
- `@openai/codex@linux-arm64`
- `@openai/codex@linux-x64`
- `@openai/codex@win32-arm64`
- `@openai/codex@win32-x64`
- `@openai/codex-responses-api-proxy@latest`
- `@openai/codex-sdk@latest`
#### Alpha release (`x.y.z-alpha.N`)
- `@openai/codex@alpha` (meta package)
- `@openai/codex@alpha-darwin-arm64`
- `@openai/codex@alpha-darwin-x64`
- `@openai/codex@alpha-linux-arm64`
- `@openai/codex@alpha-linux-x64`
- `@openai/codex@alpha-win32-arm64`
- `@openai/codex@alpha-win32-x64`
- `@openai/codex-responses-api-proxy@alpha`
- `@openai/codex-sdk@alpha`
As an example, the `package.json` for `@openai/codex@alpha` (using
`0.99.0-alpha.17` as the `version`) would be:
```
{
"name": "@openai/codex",
"version": "0.99.0-alpha.17",
"license": "Apache-2.0",
"bin": {
"codex": "bin/codex.js"
},
"type": "module",
"engines": {
"node": ">=16"
},
"files": [
"bin"
],
"repository": {
"type": "git",
"url": "git+https://github.com/openai/codex.git",
"directory": "codex-cli"
},
"packageManager": "pnpm@10.28.2+sha512.41872f037ad22f7348e3b1debbaf7e867cfd448f2726d9cf74c08f19507c31d2c8e7a11525b983febc2df640b5438dee6023ebb1f84ed43cc2d654d2bc326264",
"optionalDependencies": {
"@openai/codex-linux-x64": "npm:@openai/codex@0.99.0-alpha.17-linux-x64",
"@openai/codex-linux-arm64": "npm:@openai/codex@0.99.0-alpha.17-linux-arm64",
"@openai/codex-darwin-x64": "npm:@openai/codex@0.99.0-alpha.17-darwin-x64",
"@openai/codex-darwin-arm64": "npm:@openai/codex@0.99.0-alpha.17-darwin-arm64",
"@openai/codex-win32-x64": "npm:@openai/codex@0.99.0-alpha.17-win32-x64",
"@openai/codex-win32-arm64": "npm:@openai/codex@0.99.0-alpha.17-win32-arm64"
}
}
```
Note that the keys in `optionalDependencies` have "clean" names, but the
values have the tag embedded.
### Important note
**Note:** Because we never created the new platform package names on npm
(for example,
`@openai/codex-darwin-arm64`) since #11318 landed, there are no extra
npm packages to clean up.
## What changed
### 1. Stage platform tarballs as `@openai/codex` with platform-specific
versions
File: `codex-cli/scripts/build_npm_package.py`
- Added `CODEX_NPM_NAME = "@openai/codex"` and platform metadata
`npm_tag` values:
- `darwin-arm64`, `darwin-x64`, `linux-arm64`, `linux-x64`,
`win32-arm64`, `win32-x64`
- For platform package staging (`codex-<platform>` inputs), switched
generated `package.json` from:
- `name = @openai/codex-<platform>`
to:
- `name = @openai/codex`
- Added `compute_platform_package_version(version, platform_tag)` so
platform tarballs have unique
versions (`<release-version>-<platform-tag>`), which is required because
npm forbids re-publishing
the same `name@version`.
### 2. Point meta package optional dependencies at dist-tags on
`@openai/codex`
File: `codex-cli/scripts/build_npm_package.py`
- Updated `optionalDependencies` generation for the main `codex` package
to use npm alias syntax:
- key remains alias package name (for example,
`@openai/codex-darwin-arm64`) so runtime lookup behavior is unchanged
- value now resolves to `@openai/codex` by dist-tag
- Stable releases emit tags like `npm:@openai/codex@darwin-arm64`.
- Alpha releases (`x.y.z-alpha.N`) emit tags like
`npm:@openai/codex@alpha-darwin-arm64`.
### 3. Publish with per-tarball dist-tags in release CI
File: `.github/workflows/rust-release.yml`
- Reworked npm publish logic to derive the publish tag per tarball
filename:
- platform tarballs publish with `<platform>` tags for stable releases
- platform tarballs publish with `alpha-<platform>` tags for alpha
releases
- top-level tarballs (`codex`, `codex-responses-api-proxy`, `codex-sdk`)
continue using
the existing channel tag policy (`latest` implicit for stable, `alpha`
for alpha)
- Added fail-fast behavior for unexpected tarball names to avoid silent
mispublishes.
### 4. Normalize Codex platform tarball filenames for GitHub Release
grouping
Files: `scripts/stage_npm_packages.py`,
`.github/workflows/rust-release.yml`
- Renamed staged platform tarball filenames from:
- `codex-linux-<arch>-npm-<version>.tgz`
- `codex-darwin-<arch>-npm-<version>.tgz`
- `codex-win32-<arch>-npm-<version>.tgz`
- To:
- `codex-npm-linux-<arch>-<version>.tgz`
- `codex-npm-darwin-<arch>-<version>.tgz`
- `codex-npm-win32-<arch>-<version>.tgz`
This keeps all Codex npm artifacts grouped under a common `codex-npm-`
prefix in GitHub Releases.
### 5. Documentation update
File: `codex-cli/scripts/README.md`
- Updated staging docs to clarify that platform-native variants are
published as dist-tagged
`@openai/codex` artifacts rather than separate npm package names.
## Resulting behavior
- Mainline release:
- `@openai/codex@latest` resolves the meta package
- meta package optional dependencies resolve
`@openai/codex@<platform-tag>`
- Alpha release:
- users can continue installing `@openai/codex@alpha`
- alpha meta package optional dependencies resolve
`@openai/codex@alpha-<platform-tag>`
- Release assets:
- Codex npm tarballs share `codex-npm-` prefix for cleaner grouping in
GitHub Releases
This preserves platform-specific payload distribution while avoiding
separate npm package names and
improves release-asset discoverability.
## Validation notes
- Verified staged `package.json` output for stable and alpha meta
packages includes expected alias targets.
- Verified staged platform package manifests are `name=@openai/codex`
with unique platform-suffixed versions.
- Verified publish tag derivation maps renamed platform tarballs to
expected stable and alpha dist-tags.
When rg download fails during npm package staging, log the
target/platform/url and preserve the original exception as the cause.
Emit GitHub Actions log groups and error annotations so the failure is
easier to spot.
Document why a urlopen timeout is set (the default can hang
indefinitely).
This is to make failures in the specific build step easier to understand
/ work out what's failing rather than having a big wall of text (or at
least having an obvious part of it that helps narrow that wall)
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
The Windows Elevated Sandbox uses two new binaries:
codex-windows-sandbox-setup.exe
codex-command-runner.exe
This PR includes them when installing native deps and packaging for npm
There was a bit of copypasta I put up with when were publishing two
packages to npm, but now that it's three, I created some more scripts to
consolidate things.
With this change, I ran:
```shell
./scripts/stage_npm_packages.py --release-version 0.43.0-alpha.8 --package codex --package codex-responses-api-proxy --package codex-sdk
```
Indeed when it finished, I ended up with:
```shell
$ tree dist
dist
└── npm
├── codex-npm-0.43.0-alpha.8.tgz
├── codex-responses-api-proxy-npm-0.43.0-alpha.8.tgz
└── codex-sdk-npm-0.43.0-alpha.8.tgz
$ tar tzvf dist/npm/codex-sdk-npm-0.43.0-alpha.8.tgz
-rwxr-xr-x 0 0 0 25476720 Oct 26 1985 package/vendor/aarch64-apple-darwin/codex/codex
-rwxr-xr-x 0 0 0 29871400 Oct 26 1985 package/vendor/aarch64-unknown-linux-musl/codex/codex
-rwxr-xr-x 0 0 0 28368096 Oct 26 1985 package/vendor/x86_64-apple-darwin/codex/codex
-rwxr-xr-x 0 0 0 36029472 Oct 26 1985 package/vendor/x86_64-unknown-linux-musl/codex/codex
-rw-r--r-- 0 0 0 10926 Oct 26 1985 package/LICENSE
-rw-r--r-- 0 0 0 30187520 Oct 26 1985 package/vendor/aarch64-pc-windows-msvc/codex/codex.exe
-rw-r--r-- 0 0 0 35277824 Oct 26 1985 package/vendor/x86_64-pc-windows-msvc/codex/codex.exe
-rw-r--r-- 0 0 0 4842 Oct 26 1985 package/dist/index.js
-rw-r--r-- 0 0 0 1347 Oct 26 1985 package/package.json
-rw-r--r-- 0 0 0 9867 Oct 26 1985 package/dist/index.js.map
-rw-r--r-- 0 0 0 12 Oct 26 1985 package/README.md
-rw-r--r-- 0 0 0 4287 Oct 26 1985 package/dist/index.d.ts
```
This PR expands `.github/workflows/rust-release.yml` so that it also
builds and publishes the `npm` module for
`@openai/codex-responses-api-proxy` in addition to `@openai/codex`. Note
both `npm` modules are similar, in that they each contain a single `.js`
file that is a thin launcher around the appropriate native executable.
(Since we have a minimal dependency on Node.js, I also lowered the
minimum version from 20 to 16 and verified that works on my machine.)
As part of this change, we tighten up some of the docs around
`codex-responses-api-proxy` and ensure the details regarding protecting
the `OPENAI_API_KEY` in memory match the implementation.
To test the `npm` build process, I ran:
```
./codex-cli/scripts/build_npm_package.py --package codex-responses-api-proxy --version 0.43.0-alpha.3
```
which stages the `npm` module for `@openai/codex-responses-api-proxy` in
a temp directory, using the binary artifacts from
https://github.com/openai/codex/releases/tag/rust-v0.43.0-alpha.3.
This should make the `codex-responses-api-proxy` binaries available for
all platforms in a GitHub Release as well as a corresponding DotSlash
file.
Making `codex-responses-api-proxy` available as an `npm` module will be
done in a follow-up PR.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/4404).
* __->__ #4406
* #4404
* #4403
I am not sure what is going on, as
https://github.com/openai/codex/pull/3660 introduced this new logic and
I swear that CI was green before I merged that PR, but I am seeing
failures in this CI job this morning. This feels like a
non-backwards-compatible change in `gh`, but that feels unlikely...
Nevertheless, this is what I currently see on my laptop:
```
$ gh --version
gh version 2.76.2 (2025-07-30)
https://github.com/cli/cli/releases/tag/v2.76.2
$ gh run list --workflow .github/workflows/rust-release.yml --branch rust-v0.40.0 --json workflowName,url,headSha --jq 'first(.[])'
{
"headSha": "5268705a69713752adcbd8416ef9e84a683f7aa3",
"url": "https://github.com/openai/codex/actions/runs/17952349351",
"workflowName": ".github/workflows/rust-release.yml"
}
```
Looking at sample output from an old GitHub issue
(https://github.com/cli/cli/issues/6678), it appears that, at least at
one point in time, the `workflowName` was _not_ the path to the
workflow.
We try to ensure ripgrep (`rg`) is provided with Codex.
- For `brew`, we declare it as a dependency of our formula:
https://github.com/Homebrew/homebrew-core/blob/08d82d8b006a19efbe234477bc8b18d35b5fef50/Formula/c/codex.rb#L24
- For `npm`, we declare `@vscode/ripgrep` as a dependency, which
installs the platform-specific binary as part of a `postinstall` script:
https://github.com/openai/codex/blob/fdb8dadcae9f8eec91bc3eb5a17b3f9b19e28505/codex-cli/package.json#L22
- Users who download the CLI directly from GitHub Releases are on their
own.
In practice, I have seen `@vscode/ripgrep` fail on occasion. Here is a
trace from a GitHub workflow:
```
npm error code 1
npm error path /Users/runner/hostedtoolcache/node/20.19.5/arm64/lib/node_modules/@openai/codex/node_modules/@vscode/ripgrep
npm error command failed
npm error command sh -c node ./lib/postinstall.js
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 1 failed, retrying in 2 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 2 failed, retrying in 4 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 3 failed, retrying in 8 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Download attempt 4 failed, retrying in 16 seconds...
npm error Finding release for v13.0.0-13
npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13
npm error Deleting invalid download cache
npm error Error: Request failed: 403
```
To eliminate this error, this PR changes things so that we vendor the
`rg` binary into https://www.npmjs.com/package/@openai/codex so it is
guaranteed to be included when a user runs `npm i -g @openai/codex`.
The downside of this approach is the increase in package size: we
include the `rg` executable for six architectures (in addition to the
six copies of `codex` we already include). In a follow-up, I plan to add
support for "slices" of our npm module, so that soon users will be able
to do:
```
npm install -g @openai/codex@aarch64-apple-darwin
```
Admittedly, this is a sizable change and I tried to clean some things up
in the process:
- `install_native_deps.sh` has been replaced by `install_native_deps.py`
- `stage_release.sh` and `stage_rust_release.py` has been replaced by
`build_npm_package.py`
We now vendor in a DotSlash file for ripgrep (as a modest attempt to
facilitate local testing) and then build up the extension by:
- creating a temp directory and copying `package.json` over to it with
the target value for `"version"`
- finding the GitHub workflow that corresponds to the
`--release-version` and copying the various `codex` artifacts to
respective `vendor/TARGET_TRIPLE/codex` folder
- downloading the `rg` artifacts specified in the DotSlash file and
copying them over to the respective `vendor/TARGET_TRIPLE/path` folder
- if `--pack-output` is specified, runs `npm pack` on the temp directory
To test, I downloaded the artifact produced by this CI job:
https://github.com/openai/codex/actions/runs/17961595388/job/51085840022?pr=3660
and verified that `node ./bin/codex.js 'which -a rg'` worked as
intended.
This deletes the bulk of the `codex-cli` folder and eliminates the logic
that builds the TypeScript code and bundles it into the release.
Since this PR modifies `.github/workflows/rust-release.yml`, to test
changes to the release process, I locally commented out all of the "is
this commit on upstream `main`" checks in
`scripts/create_github_release.sh` and ran:
```
./codex-rs/scripts/create_github_release.sh 0.20.0-alpha.4
```
Which kicked off:
https://github.com/openai/codex/actions/runs/16842085113
And the release artifacts appear legit!
https://github.com/openai/codex/releases/tag/rust-v0.20.0-alpha.4
Historically, the release process for the npm module has been:
- I run `codex-rs/scripts/create_github_release.sh` to kick off a
release for the native artifacts.
- I wait until it is done.
- I run `codex-cli/scripts/stage_rust_release.py` to build the npm
release locally
- I run `npm publish` from my laptop
It has been a longstanding issue to move the npm build to CI. I may
still have to do the `npm publish` manually because it requires 2fac
with `npm`, though I assume we can work that out later.
Note I asked Codex to make these updates, and while they look pretty
good to me, I'm not 100% certain, but let's just merge this and I'll
kick off another alpha build and we'll see what happens?
To date, the build scripts in `codex-cli` still supported building the
old TypeScript version of the Codex CLI to give Windows users something
they can run, but we are just going to have them use the Rust version
like everyone else, so:
- updates `codex-cli/bin/codex.js` so that we run the native binary or
throw if the target platform/arch is not supported (no more conditional
usage based on `CODEX_RUST`, `use-native` file, etc.)
- drops the `--native` flag from `codex-cli/scripts/stage_release.sh`
and updates all the code paths to behave as if `--native` were passed
(i.e., it is the only way to run it now)
Tested this by running:
```
./codex-cli/scripts/stage_rust_release.py --release-version 0.20.0-alpha.2
```
It appears that `0.5.0` was built with `stage_release.sh` instead of
`stage_rust_release.py`, so add docs to clarify this and recommend
running `--version` on the release candidate to verify the right thing
was built.
This is a stopgap solution before migrating the build for the npm
release to GitHub Actions (which is ultimately what should be done to
ensure hermetic builds).
The idea is that instead of continuing to create PRs like
https://github.com/openai/codex/pull/1472 where I have to check in a
change to the `WORKFLOW_URL`, this script uses `gh run list` to get the
`WORKFLOW_URL` dynamically and then threads the value through to
`install_native_deps.sh`.
To create the 0.3.0 release on npm, I ran:
```shell
./codex-cli/scripts/stage_rust_release.py --release-version 0.3.0
```
and then did `npm publish --dry-run` followed by `npm publish` in the
temp directory created by `stage_rust_release.py`.
This introduces two changes to make a quick fix so we can deploy the
Rust CLI for `0.2.0` of `@openai/codex` on npm:
- Updates `WORKFLOW_URL` to point to
https://github.com/openai/codex/actions/runs/15981617627, which is the
GitHub workflow run used to create the binaries for the `0.2.0` release
we published to Homebrew.
- Adds a `--version` option to `stage_release.sh` to specify what the
`version` field in the `package.json` will be.
Locally, I ran the following:
```
./codex-cli/scripts/stage_release.sh --native --version 0.2.0
```
Previously, we only used the `--native` flag to publish to the `native`
tag of `@openai/codex` (e.g., `npm publish --tag native`), but we should
just publish this as the default tag for `0.2.0` to be consistent with
what is in Homebrew.
We can still publish one "final" version of the TypeScript CLI as 0.1.x
later.
Under the hood, this release will still contain `dist/cli.js`,
`bin/codex-linux-sandbox-x64`, and `bin/codex-x86_64-apple-darwin`,
which are not strictly necessary, but we'll fix that in `0.3.0`.
Now that we have published a GitHub Release that contains arm64 musl
artifacts for Linux, update the following scripts to take advantage of
them:
- `dotslash-config.json` now uses musl artifacts for the `linux-aarch64`
target
- `install_native_deps.sh` for the TypeScript CLI now includes
`codex-linux-sandbox-aarch64-unknown-linux-musl` instead of
`codex-linux-sandbox-aarch64-unknown-linux-gnu` for sandboxing
- `codex-cli/bin/codex.js` now checks for `aarch64-unknown-linux-musl`
artifacts instead of `aarch64-unknown-linux-gnu` ones
Added logic so that when we run `./scripts/stage_release.sh --native`
(for the `@native` version of the Node module), we drop a `use-native`
file next to `codex.js`. If present, `codex.js` will now run the Rust
CLI.
Ran `./scripts/stage_release.sh --native` and verified that when the
running `codex.js` in the staged folder:
```
$ /var/folders/wm/f209bc1n2bd_r0jncn9s6j_00000gp/T/tmp.efvEvBlSN6/bin/codex.js --version
codex-cli 0.0.2505220956
```
it ran the expected Rust version of the CLI, as desired.
While here, I also updated the Rust version to one that I cut today,
which includes the new shell environment policy config option:
https://github.com/openai/codex/pull/1061. Note this may "break" some
users if the processes spawned by Codex need extra environment
variables. (We are still working to determine what the right defaults
should be for this option.)
This PR introduces an optional build flag, `--native`, that will build a
version of the Codex npm module that:
- Includes both the Node.js and native Rust versions (for Mac and Linux)
- Will run the native version if `CODEX_RUST=1` is set
- Runs the TypeScript version otherwise
Note this PR also updates the workflow URL to
https://github.com/openai/codex/actions/runs/14872557396, as that is a
build from today that includes everything up through
https://github.com/openai/codex/pull/843.
Test Plan:
In `~/code/codex/codex-cli`, I ran:
```
pnpm stage-release --native
```
The end of the output was:
```
Staged version 0.1.2505121317 for release in /var/folders/wm/f209bc1n2bd_r0jncn9s6j_00000gp/T/tmp.xd2p5ETYGN
Test Node:
node /var/folders/wm/f209bc1n2bd_r0jncn9s6j_00000gp/T/tmp.xd2p5ETYGN/bin/codex.js --help
Test Rust:
CODEX_RUST=1 node /var/folders/wm/f209bc1n2bd_r0jncn9s6j_00000gp/T/tmp.xd2p5ETYGN/bin/codex.js --help
Next: cd "/var/folders/wm/f209bc1n2bd_r0jncn9s6j_00000gp/T/tmp.xd2p5ETYGN" && npm publish --tag native
```
I verified that running each of these commands ran the expected version
of Codex.
While here, I also added `bin` to the `files` list in `package.json`,
which should have been done as part of
https://github.com/openai/codex/pull/757, as that added new entries to
`bin` that were matched by `.gitignore` but should have been included in
a release.
This introduces `./codex-cli/scripts/stage_release.sh`, which is a shell
script that stages a release for the Node.js module in a temp directory.
It updates the release to include these native binaries:
```
bin/codex-linux-sandbox-arm64
bin/codex-linux-sandbox-x64
```
though this PR does not update Codex CLI to use them yet.
When doing local development, run
`./codex-cli/scripts/install_native_deps.sh` to install these in your
own `bin/` folder.
This PR also updates `README.md` to document the new workflow.
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/757).
* #763
* __->__ #757
I suspect this is why some contributors kept accidentally including a
new `codex-cli/package-lock.json` in their PRs.
Note the `Dockerfile` still uses `npm` instead of `pnpm`, but that
appears to be fine. (Probably nicer to globally install as few things as
possible in the image.)
This exploration came out of my review of
https://github.com/openai/codex/pull/414.
`run_in_container.sh` runs Codex in a Docker container like so:
https://github.com/openai/codex/blob/bd1c3deed9f4f103e755baa3f3a45e7a1c1a134b/codex-cli/scripts/run_in_container.sh#L51-L58
But then runs `init_firewall.sh` to set up the firewall to restrict
network access.
Previously, we did this by adding `/usr/local/bin/init_firewall.sh` to
the container and adding a special rule in `/etc/sudoers.d` so the
unprivileged user (`node`) could run the privileged `init_firewall.sh`
script to open up the firewall for `api.openai.com`:
https://github.com/openai/codex/blob/31d0d7a305305ad557035a2edcab60b6be5018d8/codex-cli/Dockerfile#L51-L56
Though I believe this is unnecessary, as we can use `docker exec --user
root` from _outside_ the container to run
`/usr/local/bin/init_firewall.sh` as `root` without adding a special
case in `/etc/sudoers.d`.
This appears to work as expected, as I tested it by doing the following:
```
./codex-cli/scripts/build_container.sh
./codex-cli/scripts/run_in_container.sh 'what is the output of `curl https://www.openai.com`'
```
This was a bit funny because in some of my runs, Codex wasn't convinced
it had network access, so I had to convince it to try the `curl`
request:
As you can see, when it ran `curl -s https\://www.openai.com`, it a
connection failure, so the network policy appears to be working as
intended.
Note this PR also removes `sudo` from the `apt-get install` list in the
`Dockerfile`.
- Docker container name based on work directory
- Centralize container removal logic
- Improve quoting for command arguments
- Ensure workdir is always set and normalized
Resolves: #371
Signed-off-by: BadPirate <badpirate@gmail.com>
Signed-off-by: BadPirate <badpirate@gmail.com>
Michael Bolin ·
Dylan Hurd ·
mcgrew-oai ·
pakrym-oai ·
jif-oai ·
Josh McKinney ·
iceweasel-oai ·
Matan Yemini ·
BadPirate ·
Eric Burke ·
Ilan Bigio ·