mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
dev
7 Commits
-
npm: remove legacy package artifact synthesis (#23836)
## Why `rust-release` now publishes `codex-package-<target>.tar.gz` as the canonical native package payload. npm staging should consume those archives directly instead of keeping legacy synthesis code that fetched `rg`, copied standalone binaries, and rebuilt an approximate package layout. That also means the package builder should not know the internal shape of `codex-package`. It should extract and copy the target payload wholesale so future layout changes stay localized to the archive producer. The release job stages `codex`, `codex-responses-api-proxy`, and `codex-sdk` together, so native artifact download should be filtered, observable, and shared across component installs. Since that native hydration is now only used by release staging, keeping a separate `install_native_deps.py` CLI adds an extra wrapper without a real caller. ## What Changed - Removed legacy `codex-package` synthesis and related compatibility flags from npm staging. - Folded the remaining native artifact hydration code into `scripts/stage_npm_packages.py` and deleted `codex-cli/scripts/install_native_deps.py`. - Made platform package staging copy the full extracted target directory instead of enumerating package entries. - Kept non-`codex-package` native components under their component directory name instead of using a legacy destination map. - Split native staging by component set while sharing one workflow-artifact cache across the invocation. - Changed workflow artifact download to select target artifacts by name, print sizes/progress, and reuse cached artifacts. - Removed the implicit `CI=true` default from `build_npm_package.py`; local CI-shaped runs should set that environment explicitly. - Kept `npm pack` cache/log output in its temporary directory so packing does not write to the user npm cache. ## Verification - `python3 -m py_compile scripts/stage_npm_packages.py codex-cli/scripts/build_npm_package.py` - `python3 -m unittest discover -s scripts/codex_package -p "test_*.py"` - `scripts/stage_npm_packages.py --help` - `codex-cli/scripts/build_npm_package.py --help` - Ran the release-shaped staging command from `rust-release.yml` against workflow run https://github.com/openai/codex/actions/runs/26240748758 with `CI=true` set locally to match GitHub Actions: ```sh CI=true python3 ./scripts/stage_npm_packages.py \ --release-version 0.133.0 \ --workflow-url https://github.com/openai/codex/actions/runs/26240748758 \ --package codex \ --package codex-responses-api-proxy \ --package codex-sdk ``` That completed successfully, downloaded only the six target artifacts once, reused the cache for `codex-responses-api-proxy`, and produced all nine npm tarballs. Generated tarballs and staging/artifact temp dirs were cleaned afterward.
Michael Bolin ·
2026-05-21 20:43:48 +00:00 -
npm: ship platform packages in Codex package layout (#23637)
## Summary The npm platform packages should stop carrying a bespoke native layout now that the release workflow builds canonical Codex package archives. Keeping npm on the same `bin/`, `codex-resources/`, and `codex-path/` structure lets the Rust package-layout detection behave consistently across standalone, npm, and future DotSlash installs. This changes platform npm packages to stage the `codex-package` artifact for each target under `vendor/<target>`. The Node launcher now resolves `bin/codex` and prepends `codex-path`, while retaining legacy `vendor/<target>/codex` and `vendor/<target>/path` fallback support for local development and migration. The npm staging helper downloads `codex-package` archives instead of rebuilding the CLI payload from individual `codex`, `rg`, `bwrap`, and sandbox helper artifacts. CI still needs to stage npm packages from historical rust-release workflow artifacts that predate package archives, so the staging scripts expose an explicit `--allow-legacy-codex-package` fallback. That fallback synthesizes the canonical package layout from legacy per-binary artifacts and is wired only into the CI smoke path; release staging remains strict and continues to require real package archives. For direct local use, `install_native_deps.py` now points its built-in default workflow at the same recent artifact run used by CI and automatically enables legacy package synthesis only when `--workflow-url` is omitted. Explicit workflow URLs remain strict unless callers opt in with `--allow-legacy-codex-package`. ## Test plan - `python3 -m py_compile codex-cli/scripts/build_npm_package.py codex-cli/scripts/install_native_deps.py scripts/stage_npm_packages.py scripts/codex_package/cli.py` - `node --check codex-cli/bin/codex.js` - `ruby -e 'require "yaml"; YAML.load_file(".github/workflows/rust-release.yml"); YAML.load_file(".github/workflows/ci.yml"); puts "ok"'` - Staged a synthetic `codex-linux-x64` platform package from a canonical vendor tree and verified it copied only `bin/`, `codex-path/`, `codex-resources/`, and `codex-package.json`. - Imported `install_native_deps.py` and extracted a synthetic `codex-package-x86_64-unknown-linux-musl.tar.gz` into `vendor/<target>`. - Ran legacy-layout conversion smokes for Linux, Windows, and unsigned macOS artifact naming. - Ran a synthetic `install_native_deps.py` default-workflow smoke that verifies legacy package synthesis is automatic only when `--workflow-url` is omitted. - `NPM_CONFIG_CACHE="$tmp_dir/npm-cache" python3 ./scripts/stage_npm_packages.py --release-version 0.125.0 --workflow-url https://github.com/openai/codex/actions/runs/26131514935 --package codex --allow-legacy-codex-package --output-dir "$tmp_dir"` - `node codex-cli/bin/codex.js --version` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/23637). * #23638 * __->__ #236372026-05-20 12:02:32 -07:00 -
# Use
@openai/codexdist-tags for platform binaries instead of separate package names (#11339)https://github.com/openai/codex/pull/11318 introduced logic to publish platform artifacts as separate npm packages (for example, `@openai/codex-darwin-arm64`, `@openai/codex-linux-x64`, etc.). That requires provisioning and maintaining multiple package entries in npm, which we want to avoid. We still need to keep the package-size mitigation (platform-specific payloads), but we want that layout to live under a single npm package namespace (`@openai/codex`) using dist-tags. We also need to preserve pre-release workflows where users install `@openai/codex@alpha` and get platform-appropriate binaries. Additionally, we want GitHub Release assets to group Codex npm tarballs together, so platform tarballs should follow the same `codex-npm-*` filename prefix as the main Codex tarball. ## Release Strategy (New Scheme) We publish **one npm package name for Codex binaries** (`@openai/codex`) and use **dist-tags** to select platform-specific payloads. This avoids creating separate platform package names while keeping the package size split by platform. ### What gets published #### Mainline release (`x.y.z`) - `@openai/codex@latest` (meta package) - `@openai/codex@darwin-arm64` - `@openai/codex@darwin-x64` - `@openai/codex@linux-arm64` - `@openai/codex@linux-x64` - `@openai/codex@win32-arm64` - `@openai/codex@win32-x64` - `@openai/codex-responses-api-proxy@latest` - `@openai/codex-sdk@latest` #### Alpha release (`x.y.z-alpha.N`) - `@openai/codex@alpha` (meta package) - `@openai/codex@alpha-darwin-arm64` - `@openai/codex@alpha-darwin-x64` - `@openai/codex@alpha-linux-arm64` - `@openai/codex@alpha-linux-x64` - `@openai/codex@alpha-win32-arm64` - `@openai/codex@alpha-win32-x64` - `@openai/codex-responses-api-proxy@alpha` - `@openai/codex-sdk@alpha` As an example, the `package.json` for `@openai/codex@alpha` (using `0.99.0-alpha.17` as the `version`) would be: ``` { "name": "@openai/codex", "version": "0.99.0-alpha.17", "license": "Apache-2.0", "bin": { "codex": "bin/codex.js" }, "type": "module", "engines": { "node": ">=16" }, "files": [ "bin" ], "repository": { "type": "git", "url": "git+https://github.com/openai/codex.git", "directory": "codex-cli" }, "packageManager": "pnpm@10.28.2+sha512.41872f037ad22f7348e3b1debbaf7e867cfd448f2726d9cf74c08f19507c31d2c8e7a11525b983febc2df640b5438dee6023ebb1f84ed43cc2d654d2bc326264", "optionalDependencies": { "@openai/codex-linux-x64": "npm:@openai/codex@0.99.0-alpha.17-linux-x64", "@openai/codex-linux-arm64": "npm:@openai/codex@0.99.0-alpha.17-linux-arm64", "@openai/codex-darwin-x64": "npm:@openai/codex@0.99.0-alpha.17-darwin-x64", "@openai/codex-darwin-arm64": "npm:@openai/codex@0.99.0-alpha.17-darwin-arm64", "@openai/codex-win32-x64": "npm:@openai/codex@0.99.0-alpha.17-win32-x64", "@openai/codex-win32-arm64": "npm:@openai/codex@0.99.0-alpha.17-win32-arm64" } } ``` Note that the keys in `optionalDependencies` have "clean" names, but the values have the tag embedded. ### Important note **Note:** Because we never created the new platform package names on npm (for example, `@openai/codex-darwin-arm64`) since #11318 landed, there are no extra npm packages to clean up. ## What changed ### 1. Stage platform tarballs as `@openai/codex` with platform-specific versions File: `codex-cli/scripts/build_npm_package.py` - Added `CODEX_NPM_NAME = "@openai/codex"` and platform metadata `npm_tag` values: - `darwin-arm64`, `darwin-x64`, `linux-arm64`, `linux-x64`, `win32-arm64`, `win32-x64` - For platform package staging (`codex-<platform>` inputs), switched generated `package.json` from: - `name = @openai/codex-<platform>` to: - `name = @openai/codex` - Added `compute_platform_package_version(version, platform_tag)` so platform tarballs have unique versions (`<release-version>-<platform-tag>`), which is required because npm forbids re-publishing the same `name@version`. ### 2. Point meta package optional dependencies at dist-tags on `@openai/codex` File: `codex-cli/scripts/build_npm_package.py` - Updated `optionalDependencies` generation for the main `codex` package to use npm alias syntax: - key remains alias package name (for example, `@openai/codex-darwin-arm64`) so runtime lookup behavior is unchanged - value now resolves to `@openai/codex` by dist-tag - Stable releases emit tags like `npm:@openai/codex@darwin-arm64`. - Alpha releases (`x.y.z-alpha.N`) emit tags like `npm:@openai/codex@alpha-darwin-arm64`. ### 3. Publish with per-tarball dist-tags in release CI File: `.github/workflows/rust-release.yml` - Reworked npm publish logic to derive the publish tag per tarball filename: - platform tarballs publish with `<platform>` tags for stable releases - platform tarballs publish with `alpha-<platform>` tags for alpha releases - top-level tarballs (`codex`, `codex-responses-api-proxy`, `codex-sdk`) continue using the existing channel tag policy (`latest` implicit for stable, `alpha` for alpha) - Added fail-fast behavior for unexpected tarball names to avoid silent mispublishes. ### 4. Normalize Codex platform tarball filenames for GitHub Release grouping Files: `scripts/stage_npm_packages.py`, `.github/workflows/rust-release.yml` - Renamed staged platform tarball filenames from: - `codex-linux-<arch>-npm-<version>.tgz` - `codex-darwin-<arch>-npm-<version>.tgz` - `codex-win32-<arch>-npm-<version>.tgz` - To: - `codex-npm-linux-<arch>-<version>.tgz` - `codex-npm-darwin-<arch>-<version>.tgz` - `codex-npm-win32-<arch>-<version>.tgz` This keeps all Codex npm artifacts grouped under a common `codex-npm-` prefix in GitHub Releases. ### 5. Documentation update File: `codex-cli/scripts/README.md` - Updated staging docs to clarify that platform-native variants are published as dist-tagged `@openai/codex` artifacts rather than separate npm package names. ## Resulting behavior - Mainline release: - `@openai/codex@latest` resolves the meta package - meta package optional dependencies resolve `@openai/codex@<platform-tag>` - Alpha release: - users can continue installing `@openai/codex@alpha` - alpha meta package optional dependencies resolve `@openai/codex@alpha-<platform-tag>` - Release assets: - Codex npm tarballs share `codex-npm-` prefix for cleaner grouping in GitHub Releases This preserves platform-specific payload distribution while avoiding separate npm package names and improves release-asset discoverability. ## Validation notes - Verified staged `package.json` output for stable and alpha meta packages includes expected alias targets. - Verified staged platform package manifests are `name=@openai/codex` with unique platform-suffixed versions. - Verified publish tag derivation maps renamed platform tarballs to expected stable and alpha dist-tags.
2026-02-10 10:33:47 -08:00 -
jif-oai ·
2026-02-10 14:49:53 +00:00 -
chore: introduce publishing logic for @openai/codex-sdk (#4543)
There was a bit of copypasta I put up with when were publishing two packages to npm, but now that it's three, I created some more scripts to consolidate things. With this change, I ran: ```shell ./scripts/stage_npm_packages.py --release-version 0.43.0-alpha.8 --package codex --package codex-responses-api-proxy --package codex-sdk ``` Indeed when it finished, I ended up with: ```shell $ tree dist dist └── npm ├── codex-npm-0.43.0-alpha.8.tgz ├── codex-responses-api-proxy-npm-0.43.0-alpha.8.tgz └── codex-sdk-npm-0.43.0-alpha.8.tgz $ tar tzvf dist/npm/codex-sdk-npm-0.43.0-alpha.8.tgz -rwxr-xr-x 0 0 0 25476720 Oct 26 1985 package/vendor/aarch64-apple-darwin/codex/codex -rwxr-xr-x 0 0 0 29871400 Oct 26 1985 package/vendor/aarch64-unknown-linux-musl/codex/codex -rwxr-xr-x 0 0 0 28368096 Oct 26 1985 package/vendor/x86_64-apple-darwin/codex/codex -rwxr-xr-x 0 0 0 36029472 Oct 26 1985 package/vendor/x86_64-unknown-linux-musl/codex/codex -rw-r--r-- 0 0 0 10926 Oct 26 1985 package/LICENSE -rw-r--r-- 0 0 0 30187520 Oct 26 1985 package/vendor/aarch64-pc-windows-msvc/codex/codex.exe -rw-r--r-- 0 0 0 35277824 Oct 26 1985 package/vendor/x86_64-pc-windows-msvc/codex/codex.exe -rw-r--r-- 0 0 0 4842 Oct 26 1985 package/dist/index.js -rw-r--r-- 0 0 0 1347 Oct 26 1985 package/package.json -rw-r--r-- 0 0 0 9867 Oct 26 1985 package/dist/index.js.map -rw-r--r-- 0 0 0 12 Oct 26 1985 package/README.md -rw-r--r-- 0 0 0 4287 Oct 26 1985 package/dist/index.d.ts ```2025-10-01 08:29:59 -07:00 -
fix: vendor ripgrep in the npm module (#3660)
We try to ensure ripgrep (`rg`) is provided with Codex. - For `brew`, we declare it as a dependency of our formula: https://github.com/Homebrew/homebrew-core/blob/08d82d8b006a19efbe234477bc8b18d35b5fef50/Formula/c/codex.rb#L24 - For `npm`, we declare `@vscode/ripgrep` as a dependency, which installs the platform-specific binary as part of a `postinstall` script: https://github.com/openai/codex/blob/fdb8dadcae9f8eec91bc3eb5a17b3f9b19e28505/codex-cli/package.json#L22 - Users who download the CLI directly from GitHub Releases are on their own. In practice, I have seen `@vscode/ripgrep` fail on occasion. Here is a trace from a GitHub workflow: ``` npm error code 1 npm error path /Users/runner/hostedtoolcache/node/20.19.5/arm64/lib/node_modules/@openai/codex/node_modules/@vscode/ripgrep npm error command failed npm error command sh -c node ./lib/postinstall.js npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Download attempt 1 failed, retrying in 2 seconds... npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Download attempt 2 failed, retrying in 4 seconds... npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Download attempt 3 failed, retrying in 8 seconds... npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Download attempt 4 failed, retrying in 16 seconds... npm error Finding release for v13.0.0-13 npm error GET https://api.github.com/repos/microsoft/ripgrep-prebuilt/releases/tags/v13.0.0-13 npm error Deleting invalid download cache npm error Error: Request failed: 403 ``` To eliminate this error, this PR changes things so that we vendor the `rg` binary into https://www.npmjs.com/package/@openai/codex so it is guaranteed to be included when a user runs `npm i -g @openai/codex`. The downside of this approach is the increase in package size: we include the `rg` executable for six architectures (in addition to the six copies of `codex` we already include). In a follow-up, I plan to add support for "slices" of our npm module, so that soon users will be able to do: ``` npm install -g @openai/codex@aarch64-apple-darwin ``` Admittedly, this is a sizable change and I tried to clean some things up in the process: - `install_native_deps.sh` has been replaced by `install_native_deps.py` - `stage_release.sh` and `stage_rust_release.py` has been replaced by `build_npm_package.py` We now vendor in a DotSlash file for ripgrep (as a modest attempt to facilitate local testing) and then build up the extension by: - creating a temp directory and copying `package.json` over to it with the target value for `"version"` - finding the GitHub workflow that corresponds to the `--release-version` and copying the various `codex` artifacts to respective `vendor/TARGET_TRIPLE/codex` folder - downloading the `rg` artifacts specified in the DotSlash file and copying them over to the respective `vendor/TARGET_TRIPLE/path` folder - if `--pack-output` is specified, runs `npm pack` on the temp directory To test, I downloaded the artifact produced by this CI job: https://github.com/openai/codex/actions/runs/17961595388/job/51085840022?pr=3660 and verified that `node ./bin/codex.js 'which -a rg'` worked as intended.
2025-09-23 23:00:33 -07:00 -
docs: clarify the build process for the npm release (#1568)
It appears that `0.5.0` was built with `stage_release.sh` instead of `stage_rust_release.py`, so add docs to clarify this and recommend running `--version` on the release candidate to verify the right thing was built.
2025-07-14 09:41:11 -07:00