Commit Graph

8 Commits

  • feat(shell-tool-mcp): add patched zsh build pipeline (#11668)
    ## Summary
    - add `shell-tool-mcp/patches/zsh-exec-wrapper.patch` against upstream
    zsh `77045ef899e53b9598bebc5a41db93a548a40ca6`
    - add `zsh-linux` and `zsh-darwin` jobs to
    `.github/workflows/shell-tool-mcp.yml`
    - stage zsh binaries under `artifacts/vendor/<target>/zsh/<variant>/zsh`
    - include zsh artifact jobs in `package.needs`
    - mark staged zsh binaries executable during packaging
    
    ## Notes
    - zsh source is cloned from `https://git.code.sf.net/p/zsh/code`
    - workflow pins zsh commit `77045ef899e53b9598bebc5a41db93a548a40ca6`
    - zsh build runs `./Util/preconfig` before `./configure`
    
    ## Validation
    - parsed workflow YAML locally (`yaml-ok`)
    - validated zsh patch applies cleanly with `git apply --check` on a
    fresh zsh clone
  • fix: remove references to corepack (#10138)
    Currently, our `npm publish` logic is failing.
    
    There were a number of things that were merged recently that seemed to
    contribute to this situation, though I think we have fixed most of them,
    but this one stands out:
    
    https://github.com/openai/codex/pull/10115
    
    As best I can tell, we tried to fix the pnpm version to a specific hash,
    but we did not do it consistently (though `shell-tool-mcp/package.json`
    had it specified twice...), so for this PR, I ran:
    
    ```
    $ git ls-files | grep package.json
    codex-cli/package.json
    codex-rs/responses-api-proxy/npm/package.json
    package.json
    sdk/typescript/package.json
    shell-tool-mcp/package.json
    ```
    
    and ensured that all of them now have this line:
    
    ```json
      "packageManager": "pnpm@10.28.2+sha512.41872f037ad22f7348e3b1debbaf7e867cfd448f2726d9cf74c08f19507c31d2c8e7a11525b983febc2df640b5438dee6023ebb1f84ed43cc2d654d2bc326264"
    ```
    
    I also went and deleted all of the `corepack` stuff that was added by
    https://github.com/openai/codex/pull/10115.
    
    If someone can explain why we need it and verify it does not break `npm
    publish`, then we can bring it back.
  • update the ci pnpm workflow for shell-tool-mcp to use corepack for pnpm versioning (#10115)
    This updates the CI workflows for shell-tool-mcp to use the pnpm version
    from package.json and print it in the build for verification.
    
    I have read the CLA Document and I hereby sign the CLA
  • fix(ci) more shell-tool-mcp issues (#10111)
    ## Summary
    More pnpm upgrade issues.
  • fix: change codex/sandbox-state/update from a notification to a request (#8142)
    Historically, `accept_elicitation_for_prompt_rule()` was flaky because
    we were using a notification to update the sandbox followed by a `shell`
    tool request that we expected to be subject to the new sandbox config,
    but because [rmcp](https://crates.io/crates/rmcp) MCP servers delegate
    each incoming message to a new Tokio task, messages are not guaranteed
    to be processed in order, so sometimes the `shell` tool call would run
    before the notification was processed.
    
    Prior to this PR, we relied on a generous `sleep()` between the
    notification and the request to reduce the change of the test flaking
    out.
    
    This PR implements a proper fix, which is to use a _request_ instead of
    a notification for the sandbox update so that we can wait for the
    response to the sandbox request before sending the request to the
    `shell` tool call. Previously, `rmcp` did not support custom requests,
    but I fixed that in
    https://github.com/modelcontextprotocol/rust-sdk/pull/590, which made it
    into the `0.12.0` release (see #8288).
    
    This PR updates `shell-tool-mcp` to expect
    `"codex/sandbox-state/update"` as a _request_ instead of a notification
    and sends the appropriate ack. Note this behavior is tied to our custom
    `codex/sandbox-state` capability, which Codex honors as an MCP client,
    which is why `core/src/mcp_connection_manager.rs` had to be updated as
    part of this PR, as well.
    
    This PR also updates the docs at `shell-tool-mcp/README.md`.
  • docs: update the docs for @openai/codex-shell-tool-mcp (#7962)
    The existing version of `shell-tool-mcp/README.md` was not written in a
    way that was meant to be consumed by end-users. This is now fixed.
    
    Added `codex-rs/exec-server/README.md` for the more technical bits.
  • fix: path resolution bug in npx (#7134)
    When running `npx @openai/codex-shell-tool-mcp`, the old code derived
    `__dirname` from `process.argv[1]`, which points to npx’s transient
    wrapper script in
    `~/.npm/_npx/134d0fb7e1a27652/node_modules/.bin/codex-shell-tool-mcp`.
    That made `vendorRoot` resolve to `<npx cache>/vendor`, so the startup
    checks failed with "Required binary missing" because it looked for
    `codex-execve-wrapper` in the wrong place.
    
    By relying on the real module `__dirname` and `path.resolve(__dirname,
    "..", "vendor")`, the package now anchors to its installed location
    under `node_modules/@openai/codex-shell-tool-mcp/`, so the bundled
    binaries are found and npx launches correctly.
  • feat: codex-shell-tool-mcp (#7005)
    This adds a GitHub workflow for building a new npm module we are
    experimenting with that contains an MCP server for running Bash
    commands. The new workflow, `shell-tool-mcp`, is a dependency of the
    general `release` workflow so that we continue to use one version number
    for all artifacts across the project in one GitHub release.
    
    `.github/workflows/shell-tool-mcp.yml` is the primary workflow
    introduced by this PR, which does the following:
    
    - builds the `codex-exec-mcp-server` and `codex-execve-wrapper`
    executables for both arm64 and x64 versions of Mac and Linux (preferring
    the MUSL version for Linux)
    - builds Bash (dynamically linked) for a [comically] large number of
    platforms (both x64 and arm64 for most) with a small patch specified by
    `shell-tool-mcp/patches/bash-exec-wrapper.patch`:
      - `debian-11`
      - `debian-12`
      - `ubuntu-20.04`
      - `ubuntu-22.04`
      - `ubuntu-24.04`
      - `centos-9`
      - `macos-13` (x64 only)
      - `macos-14` (arm64 only)
      - `macos-15` (arm64 only)
    - builds the TypeScript for the [new] Node module declared in the
    `shell-tool-mcp/` folder, which creates `bin/mcp-server.js`
    - adds all of the native binaries to `shell-tool-mcp/vendor/` folder;
    `bin/mcp-server.js` does a runtime check to determine which ones to
    execute
    - uses `npm pack` to create the `.tgz` for the module
    - if `publish: true` is set, invokes the `npm publish` call with the
    `.tgz`
    
    The justification for building Bash for so many different operating
    systems is because, since it is dynamically linked, we want to increase
    our confidence that the version we build is compatible with the glibc
    whatever OS we end up running on. (Note this is less of a concern with
    `codex-exec-mcp-server` and `codex-execve-wrapper` on Linux, as they are
    statically linked.)
    
    This PR also introduces the code for the npm module in `shell-tool-mcp/`
    (the proposed module name is `@openai/codex-shell-tool-mcp`). Initially,
    I intended the module to be a single file of vanilla JavaScript (like
    [`codex-cli/bin/codex.js`](https://github.com/openai/codex/blob/ab5972d447da78d3e4dd8461cf7d43a22e5d2acb/codex-cli/bin/codex.js)),
    but some of the logic seemed a bit tricky, so I decided to port it to
    TypeScript and add unit tests.
    
    `shell-tool-mcp/src/index.ts` defines the `main()` function for the
    module, which performs runtime checks to determine the clang triple to
    find the path to the Rust executables within the `vendor/` folder
    (`resolveTargetTriple()`). It uses a combination of `readOsRelease()`
    and `resolveBashPath()` to determine the correct Bash executable to run
    in the environment. Ultimately, it spawns a command like the following:
    
    ```
    codex-exec-mcp-server \
        --execve codex-execve-wrapper \
        --bash custom-bash "$@"
    ```
    
    Note `.github/workflows/shell-tool-mcp-ci.yml` defines a fairly standard
    CI job for the module (`format`/`build`/`test`).
    
    To test this PR, I pushed this branch to my personal fork of Codex and
    ran the CI job there:
    
    https://github.com/bolinfest/codex/actions/runs/19564311320
    
    Admittedly, the graph looks a bit wild now:
    
    <img width="5115" height="2969" alt="Screenshot 2025-11-20 at 11 44
    58 PM"
    src="https://github.com/user-attachments/assets/cc5ef306-efc1-4ed7-a137-5347e394f393"
    />
    
    But when it finished, I was able to download `codex-shell-tool-mcp-npm`
    from the **Artifacts** for the workflow in an empty temp directory,
    unzip the `.zip` and then the `.tgz` inside it, followed by `xattr -rc
    .` to remove the quarantine bits. Then I ran:
    
    ```shell
    npx @modelcontextprotocol/inspector node /private/tmp/foobar4/package/bin/mcp-server.js
    ```
    
    which launched the MCP Inspector and I was able to use it as expected!
    This bodes well that this should work once the package is published to
    npm:
    
    ```shell
    npx @modelcontextprotocol/inspector npx @openai/codex-shell-tool-mcp
    ```
    
    Also, to verify the package contains what I expect:
    
    ```shell
    /tmp/foobar4/package$ tree
    .
    ├── bin
    │   └── mcp-server.js
    ├── package.json
    ├── README.md
    └── vendor
        ├── aarch64-apple-darwin
        │   ├── bash
        │   │   ├── macos-14
        │   │   │   └── bash
        │   │   └── macos-15
        │   │       └── bash
        │   ├── codex-exec-mcp-server
        │   └── codex-execve-wrapper
        ├── aarch64-unknown-linux-musl
        │   ├── bash
        │   │   ├── centos-9
        │   │   │   └── bash
        │   │   ├── debian-11
        │   │   │   └── bash
        │   │   ├── debian-12
        │   │   │   └── bash
        │   │   ├── ubuntu-20.04
        │   │   │   └── bash
        │   │   ├── ubuntu-22.04
        │   │   │   └── bash
        │   │   └── ubuntu-24.04
        │   │       └── bash
        │   ├── codex-exec-mcp-server
        │   └── codex-execve-wrapper
        ├── x86_64-apple-darwin
        │   ├── bash
        │   │   └── macos-13
        │   │       └── bash
        │   ├── codex-exec-mcp-server
        │   └── codex-execve-wrapper
        └── x86_64-unknown-linux-musl
            ├── bash
            │   ├── centos-9
            │   │   └── bash
            │   ├── debian-11
            │   │   └── bash
            │   ├── debian-12
            │   │   └── bash
            │   ├── ubuntu-20.04
            │   │   └── bash
            │   ├── ubuntu-22.04
            │   │   └── bash
            │   └── ubuntu-24.04
            │       └── bash
            ├── codex-exec-mcp-server
            └── codex-execve-wrapper
    
    26 directories, 26 files
    ```