Commit Graph

3134 Commits

  • consolidate world-writable-directories scanning. (#7234)
    clean up the code for scanning for world writable directories
    
    One path (selecting a sandbox mode from /approvals) was using an
    incorrect method that did not use the new method of creating deny aces
    to prevent writing to those directories. Now all paths are the same.
  • Account for encrypted reasoning for auto compaction (#7113)
    - The total token used returned from the api doesn't account for the
    reasoning items before the assistant message
    - Account for those for auto compaction
    - Add the encrypted reasoning effort in the common tests utils
    - Add a test to make sure it works as expected
  • feat: declare server capability in shell-tool-mcp (#7112)
    This introduces a new feature to Codex when it operates as an MCP
    _client_ where if an MCP _server_ replies that it has an entry named
    `"codex/sandbox-state"` in its _server capabilities_, then Codex will
    send it an MCP notification with the following structure:
    
    ```json
    {
      "method": "codex/sandbox-state/update",
      "params": {
        "sandboxPolicy": {
          "type": "workspace-write",
          "network-access": false,
          "exclude-tmpdir-env-var": false
          "exclude-slash-tmp": false
        },
        "codexLinuxSandboxExe": null,
        "sandboxCwd": "/Users/mbolin/code/codex2"
      }
    }
    ```
    
    or with whatever values are appropriate for the initial `sandboxPolicy`.
    
    **NOTE:** Codex _should_ continue to send the MCP server notifications
    of the same format if these things change over the lifetime of the
    thread, but that isn't wired up yet.
    
    The result is that `shell-tool-mcp` can consume these values so that
    when it calls `codex_core::exec::process_exec_tool_call()` in
    `codex-rs/exec-server/src/posix/escalate_server.rs`, it is now sure to
    call it with the correct values (whereas previously we relied on
    hardcoded values).
    
    While I would argue this is a supported use case within the MCP
    protocol, the `rmcp` crate that we are using today does not support
    custom notifications. As such, I had to patch it and I submitted it for
    review, so hopefully it will be accepted in some form:
    
    https://github.com/modelcontextprotocol/rust-sdk/pull/556
    
    To test out this change from end-to-end:
    
    - I ran `cargo build` in `~/code/codex2/codex-rs/exec-server`
    - I built the fork of Bash in `~/code/bash/bash`
    - I added the following to my `~/.codex/config.toml`:
    
    ```toml
    # Use with `codex --disable shell_tool`.
    [mcp_servers.execshell]
    args = ["--bash", "/Users/mbolin/code/bash/bash"]
    command = "/Users/mbolin/code/codex2/codex-rs/target/debug/codex-exec-mcp-server"
    ```
    
    - From `~/code/codex2/codex-rs`, I ran `just codex --disable shell_tool`
    - When the TUI started up, I verified that the sandbox mode is
    `workspace-write`
    - I ran `/mcp` to verify that the shell tool from the MCP is there:
    
    <img width="1387" height="1400" alt="image"
    src="https://github.com/user-attachments/assets/1a8addcc-5005-4e16-b59f-95cfd06fd4ab"
    />
    
    - Then I asked it:
    
    > what is the output of `gh issue list`
    
    because this should be auto-approved with our existing dummy policy:
    
    
    https://github.com/openai/codex/blob/af63e6eccc35783f1bf4dca3c61adb090efb6b8a/codex-rs/exec-server/src/posix.rs#L157-L164
    
    And it worked:
    
    <img width="1387" height="1400" alt="image"
    src="https://github.com/user-attachments/assets/7568d2f7-80da-4d68-86d0-c265a6f5e6c1"
    />
  • fix: start publishing @openai/codex-shell-tool-mcp to npm (#7123)
    Start publishing as part of the normal release process.
  • bypass sandbox for policy approved commands (#7110)
    allowing cmds greenlit by execpolicy to bypass sandbox + minor refactor
    for a world where we have execpolicy rules with specific sandbox
    requirements
  • refactor: inline sandbox type lookup in process_exec_tool_call (#7122)
    `process_exec_tool_call()` was taking `SandboxType` as a param, but in
    practice, the only place it was constructed was in
    `codex_message_processor.rs` where it was derived from the other
    `sandbox_policy` param, so this PR inlines the logic that decides the
    `SandboxType` into `process_exec_tool_call()`.
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/7122).
    * #7112
    * __->__ #7122
  • support MCP elicitations (#6947)
    No support for request schema yet, but we'll at least show the message
    and allow accept/decline.
    
    <img width="823" height="551" alt="Screenshot 2025-11-21 at 2 44 05 PM"
    src="https://github.com/user-attachments/assets/6fbb892d-ca12-4765-921e-9ac4b217534d"
    />
  • fix(tui): Fail when stdin is not a terminal (#6382)
    Piping to codex fails to do anything useful and locks up the process.
    We currently check for stdout, but not stdin
    
    ```
    ❯ echo foo|just c
    cargo run --bin codex -- "$@"
        Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.21s
         Running `target/debug/codex`
    Error: stdin is not a terminal
    error: Recipe `codex` failed on line 10 with exit code 1
    ```
  • feat: support login as an option on shell-tool-mcp (#7120)
    The unified exec tool has a `login` option that defaults to `true`:
    
    
    https://github.com/openai/codex/blob/3bdcbc72927acd3e0071da3df3247e1da7ec578c/codex-rs/core/src/tools/handlers/unified_exec.rs#L35-L36
    
    This updates the `ExecParams` for `shell-tool-mcp` to support the same
    parameter. Note it is declared as `Option<bool>` to ensure it is marked
    optional in the generated JSON schema.
  • Windows: flag some invocations that launch browsers/URLs as dangerous (#7111)
    Prevent certain Powershell/cmd invocations from reaching the sandbox
    when they are trying to launch a browser, or run a command with a URL,
    etc.
  • [app-server] doc: approvals (#7105)
    Add documentation for shell and apply_patch approvals
  • [app-server] feat: expose gitInfo/cwd/etc. on Thread (#7060)
    Port the new additions from https://github.com/openai/codex/pull/6337 on
    the legacy API to v2. Mainly need `gitInfo` and `cwd` for VSCE.
  • fix(scripts) next_minor_version should reset patch number (#7050)
    ## Summary
    When incrementing the minor version, we should reset patch to 0, rather
    than keeping it.
    
    ## Testing
    - [x] tested locally with dry_run and `get_latest_release_version`
    mocked out
    
    ---------
    
    Co-authored-by: Michael Bolin <mbolin@openai.com>
  • fix: clear out duplicate entries for bash in the GitHub release (#7103)
    https://github.com/openai/codex/pull/7005 introduced a new part of the
    release process that added multiple files named `bash` in the `dist/`
    folder used as the basis of the GitHub Release. I believe that all file
    names in a GitHub Release have to be unique, which is why the recent
    release build failed:
    
    https://github.com/openai/codex/actions/runs/19577669780/job/56070183504
    
    Based on the output of the **List** step, I believe these are the
    appropriate artifacts to delete as a quick fix.
  • [app-server] feat: add Declined status for command exec (#7101)
    Add a `Declined` status for when we request an approval from the user
    and the user declines. This allows us to distinguish from commands that
    actually ran, but failed.
    
    This behaves similarly to apply_patch / FileChange, which does the same
    thing.
  • feat: codex-shell-tool-mcp (#7005)
    This adds a GitHub workflow for building a new npm module we are
    experimenting with that contains an MCP server for running Bash
    commands. The new workflow, `shell-tool-mcp`, is a dependency of the
    general `release` workflow so that we continue to use one version number
    for all artifacts across the project in one GitHub release.
    
    `.github/workflows/shell-tool-mcp.yml` is the primary workflow
    introduced by this PR, which does the following:
    
    - builds the `codex-exec-mcp-server` and `codex-execve-wrapper`
    executables for both arm64 and x64 versions of Mac and Linux (preferring
    the MUSL version for Linux)
    - builds Bash (dynamically linked) for a [comically] large number of
    platforms (both x64 and arm64 for most) with a small patch specified by
    `shell-tool-mcp/patches/bash-exec-wrapper.patch`:
      - `debian-11`
      - `debian-12`
      - `ubuntu-20.04`
      - `ubuntu-22.04`
      - `ubuntu-24.04`
      - `centos-9`
      - `macos-13` (x64 only)
      - `macos-14` (arm64 only)
      - `macos-15` (arm64 only)
    - builds the TypeScript for the [new] Node module declared in the
    `shell-tool-mcp/` folder, which creates `bin/mcp-server.js`
    - adds all of the native binaries to `shell-tool-mcp/vendor/` folder;
    `bin/mcp-server.js` does a runtime check to determine which ones to
    execute
    - uses `npm pack` to create the `.tgz` for the module
    - if `publish: true` is set, invokes the `npm publish` call with the
    `.tgz`
    
    The justification for building Bash for so many different operating
    systems is because, since it is dynamically linked, we want to increase
    our confidence that the version we build is compatible with the glibc
    whatever OS we end up running on. (Note this is less of a concern with
    `codex-exec-mcp-server` and `codex-execve-wrapper` on Linux, as they are
    statically linked.)
    
    This PR also introduces the code for the npm module in `shell-tool-mcp/`
    (the proposed module name is `@openai/codex-shell-tool-mcp`). Initially,
    I intended the module to be a single file of vanilla JavaScript (like
    [`codex-cli/bin/codex.js`](https://github.com/openai/codex/blob/ab5972d447da78d3e4dd8461cf7d43a22e5d2acb/codex-cli/bin/codex.js)),
    but some of the logic seemed a bit tricky, so I decided to port it to
    TypeScript and add unit tests.
    
    `shell-tool-mcp/src/index.ts` defines the `main()` function for the
    module, which performs runtime checks to determine the clang triple to
    find the path to the Rust executables within the `vendor/` folder
    (`resolveTargetTriple()`). It uses a combination of `readOsRelease()`
    and `resolveBashPath()` to determine the correct Bash executable to run
    in the environment. Ultimately, it spawns a command like the following:
    
    ```
    codex-exec-mcp-server \
        --execve codex-execve-wrapper \
        --bash custom-bash "$@"
    ```
    
    Note `.github/workflows/shell-tool-mcp-ci.yml` defines a fairly standard
    CI job for the module (`format`/`build`/`test`).
    
    To test this PR, I pushed this branch to my personal fork of Codex and
    ran the CI job there:
    
    https://github.com/bolinfest/codex/actions/runs/19564311320
    
    Admittedly, the graph looks a bit wild now:
    
    <img width="5115" height="2969" alt="Screenshot 2025-11-20 at 11 44
    58 PM"
    src="https://github.com/user-attachments/assets/cc5ef306-efc1-4ed7-a137-5347e394f393"
    />
    
    But when it finished, I was able to download `codex-shell-tool-mcp-npm`
    from the **Artifacts** for the workflow in an empty temp directory,
    unzip the `.zip` and then the `.tgz` inside it, followed by `xattr -rc
    .` to remove the quarantine bits. Then I ran:
    
    ```shell
    npx @modelcontextprotocol/inspector node /private/tmp/foobar4/package/bin/mcp-server.js
    ```
    
    which launched the MCP Inspector and I was able to use it as expected!
    This bodes well that this should work once the package is published to
    npm:
    
    ```shell
    npx @modelcontextprotocol/inspector npx @openai/codex-shell-tool-mcp
    ```
    
    Also, to verify the package contains what I expect:
    
    ```shell
    /tmp/foobar4/package$ tree
    .
    ├── bin
    │   └── mcp-server.js
    ├── package.json
    ├── README.md
    └── vendor
        ├── aarch64-apple-darwin
        │   ├── bash
        │   │   ├── macos-14
        │   │   │   └── bash
        │   │   └── macos-15
        │   │       └── bash
        │   ├── codex-exec-mcp-server
        │   └── codex-execve-wrapper
        ├── aarch64-unknown-linux-musl
        │   ├── bash
        │   │   ├── centos-9
        │   │   │   └── bash
        │   │   ├── debian-11
        │   │   │   └── bash
        │   │   ├── debian-12
        │   │   │   └── bash
        │   │   ├── ubuntu-20.04
        │   │   │   └── bash
        │   │   ├── ubuntu-22.04
        │   │   │   └── bash
        │   │   └── ubuntu-24.04
        │   │       └── bash
        │   ├── codex-exec-mcp-server
        │   └── codex-execve-wrapper
        ├── x86_64-apple-darwin
        │   ├── bash
        │   │   └── macos-13
        │   │       └── bash
        │   ├── codex-exec-mcp-server
        │   └── codex-execve-wrapper
        └── x86_64-unknown-linux-musl
            ├── bash
            │   ├── centos-9
            │   │   └── bash
            │   ├── debian-11
            │   │   └── bash
            │   ├── debian-12
            │   │   └── bash
            │   ├── ubuntu-20.04
            │   │   └── bash
            │   ├── ubuntu-22.04
            │   │   └── bash
            │   └── ubuntu-24.04
            │       └── bash
            ├── codex-exec-mcp-server
            └── codex-execve-wrapper
    
    26 directories, 26 files
    ```
  • Fixed the deduplicator github action (#7070)
    It stopped working (found zero duplicates) starting three days ago when
    the model was switched from `gpt-5` to `gpt-5.1`. I'm not sure why it
    stopped working. This is an attempt to get it working again by using the
    default model for the codex action (which is presumably
    `gpt-5.1-codex-max`).
  • fix: read max_output_tokens param from config (#4139)
    Request param `max_output_tokens` is documented in
    `https://github.com/openai/codex/blob/main/docs/config.md`,
    but nowhere uses the item in config, this commit read it from config for
    GPT responses API.
    
    see https://github.com/openai/codex/issues/4138 for issue report.
    
    Signed-off-by: Yorling <shallowcloud@yeah.net>
  • Support all types of search actions (#7061)
    Fixes the 
    
    ```
    {
      "error": {
        "message": "Invalid value: 'other'. Supported values are: 'search', 'open_page', and 'find_in_page'.",
        "type": "invalid_request_error",
        "param": "input[150].action.type",
        "code": "invalid_value"
      }
    ```
    error.
    
    
    The actual-actual fix here is supporting absent `query` parameter.
  • Support full powershell paths in is_safe_command (#7055)
    New shell implementation always uses full paths.
  • fix(app-server) remove www warning (#7046)
    ### Summary
    After #7022, we no longer need this warning. We should also clean up the
    schema for the notification, but this is a quick fix to just stop the
    behavior in the VSCE
    
    ## Testing
    - [x] Ran locally
  • feat(tui): default reasoning selection to medium (#7040)
    ## Summary
    - allow selection popups to request an initial highlighted row
    - begin the /models reasoning selector focused on the default effort
    
    ## Testing
    - just fmt
    - just fix -p codex-tui
    - cargo test -p codex-tui
    
    
    
    https://github.com/user-attachments/assets/b322aeb1-e8f3-4578-92f7-5c2fa5ee4c98
    
    
    
    ------
    [Codex
    Task](https://chatgpt.com/codex/tasks/task_i_691f75e8fc188322a910fbe2138666ef)
  • [app-server] update doc with codex error info (#6941)
    Document new codex error info. Also fixed the name from
    `codex_error_code` to `codex_error_info`.
  • feat: waiting for an elicitation should not count against a shell tool timeout (#6973)
    Previously, we were running into an issue where we would run the `shell`
    tool call with a timeout of 10s, but it fired an elicitation asking for
    user approval, the time the user took to respond to the elicitation was
    counted agains the 10s timeout, so the `shell` tool call would fail with
    a timeout error unless the user is very fast!
    
    This PR addresses this issue by introducing a "stopwatch" abstraction
    that is used to manage the timeout. The idea is:
    
    - `Stopwatch::new()` is called with the _real_ timeout of the `shell`
    tool call.
    - `process_exec_tool_call()` is called with the `Cancellation` variant
    of `ExecExpiration` because it should not manage its own timeout in this
    case
    - the `Stopwatch` expiration is wired up to the `cancel_rx` passed to
    `process_exec_tool_call()`
    - when an elicitation for the `shell` tool call is received, the
    `Stopwatch` pauses
    - because it is possible for multiple elicitations to arrive
    concurrently, it keeps track of the number of "active pauses" and does
    not resume until that counter goes down to zero
    
    I verified that I can test the MCP server using
    `@modelcontextprotocol/inspector` and specify `git status` as the
    `command` with a timeout of 500ms and that the elicitation pops up and I
    have all the time in the world to respond whereas previous to this PR,
    that would not have been possible.
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/6973).
    * #7005
    * __->__ #6973
    * #6972
  • fix flaky tool_call_output_exceeds_limit_truncated_chars_limit (#7043)
    I am suspecting this is flaky because of the wall time can become 0,
    0.1, or 1.
  • feat: update process_exec_tool_call() to take a cancellation token (#6972)
    This updates `ExecParams` so that instead of taking `timeout_ms:
    Option<u64>`, it now takes a more general cancellation mechanism,
    `ExecExpiration`, which is an enum that includes a
    `Cancellation(tokio_util::sync::CancellationToken)` variant.
    
    If the cancellation token is fired, then `process_exec_tool_call()`
    returns in the same way as if a timeout was exceeded.
    
    This is necessary so that in #6973, we can manage the timeout logic
    external to the `process_exec_tool_call()` because we want to "suspend"
    the timeout when an elicitation from a human user is pending.
    
    
    
    
    
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/6972).
    * #7005
    * #6973
    * __->__ #6972
  • core: make shell behavior portable on FreeBSD (#7039)
    - Use /bin/sh instead of /bin/bash on FreeBSD/OpenBSD in the process
    group timeout test to avoid command-not-found failures.
    
    - Accept /usr/local/bin/bash as a valid SHELL path to match common
    FreeBSD installations.
    
    - Switch the shell serialization duration test to /bin/sh for improved
    portability across Unix platforms.
    
    With this change, `cargo test -p codex-core --lib` runs and passes on
    FreeBSD.
  • [app-server & core] introduce new codex error code and v2 app-server error events (#6938)
    This PR does two things:
    1. populate a new `codex_error_code` protocol in error events sent from
    core to client;
    2. old v1 core events `codex/event/stream_error` and `codex/event/error`
    will now both become `error`. We also show codex error code for
    turncompleted -> error status.
    
    new events in app server test:
    ```
    < {
    <   "method": "codex/event/stream_error",
    <   "params": {
    <     "conversationId": "019aa34c-0c14-70e0-9706-98520a760d67",
    <     "id": "0",
    <     "msg": {
    <       "codex_error_code": {
    <         "response_stream_disconnected": {
    <           "http_status_code": 401
    <         }
    <       },
    <       "message": "Reconnecting... 2/5",
    <       "type": "stream_error"
    <     }
    <   }
    < }
    
     {
    <   "method": "error",
    <   "params": {
    <     "error": {
    <       "codexErrorCode": {
    <         "responseStreamDisconnected": {
    <           "httpStatusCode": 401
    <         }
    <       },
    <       "message": "Reconnecting... 2/5"
    <     }
    <   }
    < }
    
    < {
    <   "method": "turn/completed",
    <   "params": {
    <     "turn": {
    <       "error": {
    <         "codexErrorCode": {
    <           "responseTooManyFailedAttempts": {
    <             "httpStatusCode": 401
    <           }
    <         },
    <         "message": "exceeded retry limit, last status: 401 Unauthorized, request id: 9a1b495a1a97ed3e-SJC"
    <       },
    <       "id": "0",
    <       "items": [],
    <       "status": "failed"
    <     }
    <   }
    < }
    ```
  • add deny ACEs for world writable dirs (#7022)
    Our Restricted Token contains 3 SIDs (Logon, Everyone, {WorkspaceWrite
    Capability || ReadOnly Capability})
    
    because it must include Everyone, that left us vulnerable to directories
    that allow writes to Everyone. Even though those directories do not have
    ACEs that enable our capability SIDs to write to them, they could still
    be written to even in ReadOnly mode, or even in WorkspaceWrite mode if
    they are outside of a writable root.
    
    A solution to this is to explicitly add *Deny* ACEs to these
    directories, always for the ReadOnly Capability SID, and for the
    WorkspaceWrite SID if the directory is outside of a workspace root.
    
    Under a restricted token, Windows always checks Deny ACEs before Allow
    ACEs so even though our restricted token would allow a write to these
    directories due to the Everyone SID, it fails first because of the Deny
    ACE on the capability SID
  • Attempt to fix unified_exec_formats_large_output_summary flakiness (#7029)
    second attempt to fix this test after
    https://github.com/openai/codex/pull/6884. I think this flakiness is
    happening because yield_time is too small for a 10,000 step loop in
    python.
  • fix flaky test: approval_matrix_covers_all_modes (#7028)
    looks like it sometimes flake around 30. let's give it more time.
  • execpolicycheck command in codex cli (#7012)
    adding execpolicycheck tool onto codex cli
    
    this is useful for validating policies (can be multiple) against
    commands.
    
    it will also surface errors in policy syntax:
    <img width="1150" height="281" alt="Screenshot 2025-11-19 at 12 46
    21 PM"
    src="https://github.com/user-attachments/assets/8f99b403-564c-4172-acc9-6574a8d13dc3"
    />
    
    this PR also changes output format when there's no match in the CLI.
    instead of returning the raw string `noMatch`, we return
    `{"noMatch":{}}`
    
    this PR is a rewrite of: https://github.com/openai/codex/pull/6932 (due
    to the numerous merge conflicts present in the original PR)
    
    ---------
    
    Co-authored-by: Michael Bolin <mbolin@openai.com>
  • increasing user shell timeout to 1 hour (#7025)
    setting user shell timeout to an unreasonably high value since there
    isn't an easy way to have a command run without timeouts
    
    currently, user shell commands timeout is 10 seconds
  • fix: route feedback issue links by category (#6840)
    ## Summary
    - TUI feedback note now only links to the bug-report template when the
    category is bug/bad result.
    - Good result/other feedback shows a thank-you+thread ID instead of
    funneling people to file a bug.
    - Added a helper + unit test so future changes keep the behavior
    consistent.
    
    ## Testing
      - just fmt
      - just fix -p codex-tui
      - cargo test -p codex-tui
    
      Fixes #6839
  • codex-exec: allow resume --last to read prompt #6717 (#6719)
    ### Description
    
    - codex exec --json resume --last "<prompt>" bailed out because clap
    treated the prompt as SESSION_ID. I removed the conflicts_with flag and
    reinterpret that positional as a prompt when
    --last is set, so the flow now keeps working in JSON mode.
    (codex-rs/exec/src/cli.rs:84-104, codex-rs/exec/src/lib.rs:75-130)
    - Added a regression test that exercises resume --last in JSON mode to
    ensure the prompt is accepted and the rollout file is updated.
    (codex-rs/exec/tests/suite/resume.rs:126-178)
    
    ### Testing
    
      - just fmt
      - cargo test -p codex-exec
      - just fix -p codex-exec
      - cargo test -p codex-exec
    
    #6717
    
    Signed-off-by: Dmitri Khokhlov <dkhokhlov@cribl.io>