Commit Graph

92 Commits

  • Removed experimental "command risk assessment" feature (#7799)
    This experimental feature received lukewarm reception during internal
    testing. Removing from the code base.
  • refactoring with_escalated_permissions to use SandboxPermissions instead (#7750)
    helpful in the future if we want more granularity for requesting
    escalated permissions:
    e.g when running in readonly sandbox, model can request to escalate to a
    sandbox that allows writes
  • proposing execpolicy amendment when prompting due to sandbox denial (#7653)
    Currently, we only show the “don’t ask again for commands that start
    with…” option when a command is immediately flagged as needing approval.
    However, there is another case where we ask for approval: When a command
    is initially auto-approved to run within sandbox, but it fails to run
    inside sandbox, we would like to attempt to retry running outside of
    sandbox. This will require a prompt to the user.
    
    This PR addresses this latter case
  • feat(core) Add login to shell_command tool (#6846)
    ## Summary
    Adds the `login` parameter to the `shell_command` tool - optional,
    defaults to true.
    
    ## Testing
    - [x] Tested locally
  • Call models endpoint in models manager (#7616)
    - Introduce `with_remote_overrides` and update
    `refresh_available_models`
    - Put `auth_manager` instead of `auth_mode` on `models_manager`
    - Remove `ShellType` and `ReasoningLevel` to use already existing
    structs
  • Inline response recording and remove process_items indirection (#7310)
    - Inline response recording during streaming: `run_turn` now records
    items as they arrive instead of building a `ProcessedResponseItem` list
    and post‑processing via `process_items`.
    - Simplify turn handling: `handle_output_item_done` returns the
    follow‑up signal + optional tool future; `needs_follow_up` is set only
    there, and in‑flight tool futures are drained once at the end (errors
    logged, no extra state writes).
    - Flattened stream loop: removed `process_items` indirection and the
    extra output queue
    - - Tests: relaxed `tool_parallelism::tool_results_grouped` to allow any
    completion order while still requiring matching call/output IDs.
  • Refactor execpolicy fallback evaluation (#7544)
    ## Refactor of the `execpolicy` crate
    
    To illustrate why we need this refactor, consider an agent attempting to
    run `apple | rm -rf ./`. Suppose `apple` is allowed by `execpolicy`.
    Before this PR, `execpolicy` would consider `apple` and `pear` and only
    render one rule match: `Allow`. We would skip any heuristics checks on
    `rm -rf ./` and immediately approve `apple | rm -rf ./` to run.
    
    To fix this, we now thread a `fallback` evaluation function into
    `execpolicy` that runs when no `execpolicy` rules match a given command.
    In our example, we would run `fallback` on `rm -rf ./` and prevent
    `apple | rm -rf ./` from being run without approval.
  • whitelist command prefix integration in core and tui (#7033)
    this PR enables TUI to approve commands and add their prefixes to an
    allowlist:
    <img width="708" height="605" alt="Screenshot 2025-11-21 at 4 18 07 PM"
    src="https://github.com/user-attachments/assets/56a19893-4553-4770-a881-becf79eeda32"
    />
    
    note: we only show the option to whitelist the command when 
    1) command is not multi-part (e.g `git add -A && git commit -m 'hello
    world'`)
    2) command is not already matched by an existing rule
  • Migrate model family to models manager (#7565)
    This PR moves `ModelsFamily` to `openai_models`. It also propagates
    `ModelsManager` to session services and use it to drive model family. We
    also make `derive_default_model_family` private because it's a step
    towards what we want: one place that gives model configuration.
    
    This is a second step at having one source of truth for models
    information and config: `ModelsManager`.
    
    Next steps would be to remove `ModelsFamily` from config. That's massive
    because it's being used in 41 occasions mostly pre launching `codex`.
    Also, we need to make `find_family_for_model` private. It's also big
    because it's being used in 21 occasions ~ all tests.
  • fix(unified_exec): use platform default shell when unified_exec shell… (#7486)
    # Unified Exec Shell Selection on Windows
    
    ## Problem
    
    reference issue #7466
    
    The `unified_exec` handler currently deserializes model-provided tool
    calls into the `ExecCommandArgs` struct:
    
    ```rust
    #[derive(Debug, Deserialize)]
    struct ExecCommandArgs {
        cmd: String,
        #[serde(default)]
        workdir: Option<String>,
        #[serde(default = "default_shell")]
        shell: String,
        #[serde(default = "default_login")]
        login: bool,
        #[serde(default = "default_exec_yield_time_ms")]
        yield_time_ms: u64,
        #[serde(default)]
        max_output_tokens: Option<usize>,
        #[serde(default)]
        with_escalated_permissions: Option<bool>,
        #[serde(default)]
        justification: Option<String>,
    }
    ```
    
    The `shell` field uses a hard-coded default:
    
    ```rust
    fn default_shell() -> String {
        "/bin/bash".to_string()
    }
    ```
    
    When the model returns a tool call JSON that only contains `cmd` (which
    is the common case), Serde fills in `shell` with this default value.
    Later, `get_command` uses that value as if it were a model-provided
    shell path:
    
    ```rust
    fn get_command(args: &ExecCommandArgs) -> Vec<String> {
        let shell = get_shell_by_model_provided_path(&PathBuf::from(args.shell.clone()));
        shell.derive_exec_args(&args.cmd, args.login)
    }
    ```
    
    On Unix, this usually resolves to `/bin/bash` and works as expected.
    However, on Windows this behavior is problematic:
    
    - The hard-coded `"/bin/bash"` is not a valid Windows path.
    - `get_shell_by_model_provided_path` treats this as a model-specified
    shell, and tries to resolve it (e.g. via `which::which("bash")`), which
    may or may not exist and may not behave as intended.
    - In practice, this leads to commands being executed under a non-default
    or non-existent shell on Windows (for example, WSL bash), instead of the
    expected Windows PowerShell or `cmd.exe`.
    
    The core of the issue is that **"model did not specify `shell`" is
    currently interpreted as "the model explicitly requested `/bin/bash`"**,
    which is both Unix-specific and wrong on Windows.
    
    ## Proposed Solution
    
    Instead of hard-coding `"/bin/bash"` into `ExecCommandArgs`, we should
    distinguish between:
    
    1. **The model explicitly specifying a shell**, e.g.:
    
       ```json
       {
         "cmd": "echo hello",
         "shell": "pwsh"
       }
       ```
    
    In this case, we *do* want to respect the model’s choice and use
    `get_shell_by_model_provided_path`.
    
    2. **The model omitting the `shell` field entirely**, e.g.:
    
       ```json
       {
         "cmd": "echo hello"
       }
       ```
    
    In this case, we should *not* assume `/bin/bash`. Instead, we should use
    `default_user_shell()` and let the platform decide.
    
    To express this distinction, we can:
    
    1. Change `shell` to be optional in `ExecCommandArgs`:
    
       ```rust
       #[derive(Debug, Deserialize)]
       struct ExecCommandArgs {
           cmd: String,
           #[serde(default)]
           workdir: Option<String>,
           #[serde(default)]
           shell: Option<String>,
           #[serde(default = "default_login")]
           login: bool,
           #[serde(default = "default_exec_yield_time_ms")]
           yield_time_ms: u64,
           #[serde(default)]
           max_output_tokens: Option<usize>,
           #[serde(default)]
           with_escalated_permissions: Option<bool>,
           #[serde(default)]
           justification: Option<String>,
       }
       ```
    
    Here, the absence of `shell` in the JSON is represented as `shell:
    None`, rather than a hard-coded string value.
  • chore: make create_approval_requirement_for_command an async fn (#7501)
    I think this might help with https://github.com/openai/codex/pull/7033
    because `create_approval_requirement_for_command()` will soon need
    access to `Session.state`, which is a `tokio::sync::Mutex` that needs to
    be accessed via `async`.
  • bypass sandbox for policy approved commands (#7110)
    allowing cmds greenlit by execpolicy to bypass sandbox + minor refactor
    for a world where we have execpolicy rules with specific sandbox
    requirements
  • feat: update process_exec_tool_call() to take a cancellation token (#6972)
    This updates `ExecParams` so that instead of taking `timeout_ms:
    Option<u64>`, it now takes a more general cancellation mechanism,
    `ExecExpiration`, which is an enum that includes a
    `Cancellation(tokio_util::sync::CancellationToken)` variant.
    
    If the cancellation token is fired, then `process_exec_tool_call()`
    returns in the same way as if a timeout was exceeded.
    
    This is necessary so that in #6973, we can manage the timeout logic
    external to the `process_exec_tool_call()` because we want to "suspend"
    the timeout when an elicitation from a human user is pending.
    
    
    
    
    
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/6972).
    * #7005
    * #6973
    * __->__ #6972
  • Always fallback to real shell (#6953)
    Either cmd.exe or `/bin/sh`.
  • [app-server] feat: v2 apply_patch approval flow (#6760)
    This PR adds the API V2 version of the apply_patch approval flow, which
    centers around `ThreadItem::FileChange`.
    
    This PR wires the new RPC (`item/fileChange/requestApproval`, V2 only)
    and related events (`item/started`, `item/completed` for
    `ThreadItem::FileChange`, which are emitted in both V1 and V2) through
    the app-server
    protocol. The new approval RPC is only sent when the user initiates a
    turn with the new `turn/start` API so we don't break backwards
    compatibility with VSCE.
    
    Similar to https://github.com/openai/codex/pull/6758, the approach I
    took was to make as few changes to the Codex core as possible,
    leveraging existing `EventMsg` core events, and translating those in
    app-server. I did have to add a few additional fields to
    `EventMsg::PatchApplyBegin` and `EventMsg::PatchApplyEnd`, but those
    were fairly lightweight.
    
    However, the `EventMsg`s emitted by core are the following:
    ```
    1) Auto-approved (no request for approval)

    - EventMsg::PatchApplyBegin
    - EventMsg::PatchApplyEnd
    
    2) Approved by user
    - EventMsg::ApplyPatchApprovalRequest
    - EventMsg::PatchApplyBegin
    - EventMsg::PatchApplyEnd
    
    3) Declined by user
    - EventMsg::ApplyPatchApprovalRequest
    - EventMsg::PatchApplyBegin
    - EventMsg::PatchApplyEnd
    ```
    
    For a request triggering an approval, this would result in:
    ```
    item/fileChange/requestApproval
    item/started
    item/completed
    ```
    
    which is different from the `ThreadItem::CommandExecution` flow
    introduced in https://github.com/openai/codex/pull/6758, which does the
    below and is preferable:
    ```
    item/started
    item/commandExecution/requestApproval
    item/completed
    ```
    
    To fix this, we leverage `TurnSummaryStore` on codex_message_processor
    to store a little bit of state, allowing us to fire `item/started` and
    `item/fileChange/requestApproval` whenever we receive the underlying
    `EventMsg::ApplyPatchApprovalRequest`, and no-oping when we receive the
    `EventMsg::PatchApplyBegin` later.
    
    This is much less invasive than modifying the order of EventMsg within
    core (I tried).
    
    The resulting payloads:
    ```
    {
      "method": "item/started",
      "params": {
        "item": {
          "changes": [
            {
              "diff": "Hello from Codex!\n",
              "kind": "add",
              "path": "/Users/owen/repos/codex/codex-rs/APPROVAL_DEMO.txt"
            }
          ],
          "id": "call_Nxnwj7B3YXigfV6Mwh03d686",
          "status": "inProgress",
          "type": "fileChange"
        }
      }
    }
    ```
    
    ```
    {
      "id": 0,
      "method": "item/fileChange/requestApproval",
      "params": {
        "grantRoot": null,
        "itemId": "call_Nxnwj7B3YXigfV6Mwh03d686",
        "reason": null,
        "threadId": "019a9e11-8295-7883-a283-779e06502c6f",
        "turnId": "1"
      }
    }
    ```
    
    ```
    {
      "id": 0,
      "result": {
        "decision": "accept"
      }
    }
    ```
    
    ```
    {
      "method": "item/completed",
      "params": {
        "item": {
          "changes": [
            {
              "diff": "Hello from Codex!\n",
              "kind": "add",
              "path": "/Users/owen/repos/codex/codex-rs/APPROVAL_DEMO.txt"
            }
          ],
          "id": "call_Nxnwj7B3YXigfV6Mwh03d686",
          "status": "completed",
          "type": "fileChange"
        }
      }
    }
    ```
  • execpolicy2 core integration (#6641)
    This PR threads execpolicy2 into codex-core.
    
    activated via feature flag: exec_policy (on by default)
    
    reads and parses all .codexpolicy files in `codex_home/codex`
    
    refactored tool runtime API to integrate execpolicy logic
    
    ---------
    
    Co-authored-by: Michael Bolin <mbolin@openai.com>
  • Move shell to use truncate_text (#6842)
    Move shell to use the configurable `truncate_text`
    
    ---------
    
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • shell_command returns freeform output (#6860)
    Instead of returning structured out and then re-formatting it into
    freeform, return the freeform output from shell_command tool.
    
    Keep `shell` as the default tool for GPT-5.
  • chore(config) enable shell_command (#6843)
    ## Summary
    Enables shell_command as default for `gpt-5*` and `codex-*` models.
    
    ## Testing
    - [x] Updated unit tests
  • Add the utility to truncate by tokens (#6746)
    - This PR is to make it on path for truncating by tokens. This path will
    be initially used by unified exec and context manager (responsible for
    MCP calls mainly).
    - We are exposing new config `calls_output_max_tokens`
    - Use `tokens` as the main budget unit but truncate based on the model
    family by Introducing `TruncationPolicy`.
    - Introduce `truncate_text` as a router for truncation based on the
    mode.
    
    In next PRs:
    - remove truncate_with_line_bytes_budget
    - Add the ability to the model to override the token budget.
  • fixing localshell tool calls (#6823)
    - Local-shell tool responses were always tagged as
    `ExecCommandSource::UserShell` because handler would call
    `run_exec_like` with `is_user_shell_cmd` set to true.
    - Treat `ToolPayload::LocalShell` the same as other model generated
    shell tool calls by deleting `is_user_shell_cmd` from `run_exec_like`
    (since actual user shell commands follow a separate code path)
  • fix(windows) shell_command on windows, minor parsing (#6811)
    ## Summary
    Enables shell_command for windows users, and starts adding some basic
    command parsing here, to at least remove powershell prefixes. We'll
    follow this up with command parsing but I wanted to land this change
    separately with some basic UX.
    
    **NOTE**: This implementation parses bash and powershell on both
    platforms. In theory this is possible, since you can use git bash on
    windows or powershell on linux. In practice, this may not be worth the
    complexity of supporting, so I don't feel strongly about the current
    approach vs. platform-specific branching.
    
    ## Testing
    - [x] Added a bunch of tests 
    - [x] Ran on both windows and os x
  • [app-server] feat: add v2 command execution approval flow (#6758)
    This PR adds the API V2 version of the command‑execution approval flow
    for the shell tool.
    
    This PR wires the new RPC (`item/commandExecution/requestApproval`, V2
    only) and related events (`item/started`, `item/completed`, and
    `item/commandExecution/delta`, which are emitted in both V1 and V2)
    through the app-server
    protocol. The new approval RPC is only sent when the user initiates a
    turn with the new `turn/start` API so we don't break backwards
    compatibility with VSCE.
    
    The approach I took was to make as few changes to the Codex core as
    possible, leveraging existing `EventMsg` core events, and translating
    those in app-server. I did have to add additional fields to
    `EventMsg::ExecCommandEndEvent` to capture the command's input so that
    app-server can statelessly transform these events to a
    `ThreadItem::CommandExecution` item for the `item/completed` event.
    
    Once we stabilize the API and it's complete enough for our partners, we
    can work on migrating the core to be aware of command execution items as
    a first-class concept.
    
    **Note**: We'll need followup work to make sure these APIs work for the
    unified exec tool, but will wait til that's stable and landed before
    doing a pass on app-server.
    
    Example payloads below:
    ```
    {
      "method": "item/started",
      "params": {
        "item": {
          "aggregatedOutput": null,
          "command": "/bin/zsh -lc 'touch /tmp/should-trigger-approval'",
          "cwd": "/Users/owen/repos/codex/codex-rs",
          "durationMs": null,
          "exitCode": null,
          "id": "call_lNWWsbXl1e47qNaYjFRs0dyU",
          "parsedCmd": [
            {
              "cmd": "touch /tmp/should-trigger-approval",
              "type": "unknown"
            }
          ],
          "status": "inProgress",
          "type": "commandExecution"
        }
      }
    }
    ```
    
    ```
    {
      "id": 0,
      "method": "item/commandExecution/requestApproval",
      "params": {
        "itemId": "call_lNWWsbXl1e47qNaYjFRs0dyU",
        "parsedCmd": [
          {
            "cmd": "touch /tmp/should-trigger-approval",
            "type": "unknown"
          }
        ],
        "reason": "Need to create file in /tmp which is outside workspace sandbox",
        "risk": null,
        "threadId": "019a93e8-0a52-7fe3-9808-b6bc40c0989a",
        "turnId": "1"
      }
    }
    ```
    
    ```
    {
      "id": 0,
      "result": {
        "acceptSettings": {
          "forSession": false
        },
        "decision": "accept"
      }
    }
    ```
    
    ```
    {
      "params": {
        "item": {
          "aggregatedOutput": null,
          "command": "/bin/zsh -lc 'touch /tmp/should-trigger-approval'",
          "cwd": "/Users/owen/repos/codex/codex-rs",
          "durationMs": 224,
          "exitCode": 0,
          "id": "call_lNWWsbXl1e47qNaYjFRs0dyU",
          "parsedCmd": [
            {
              "cmd": "touch /tmp/should-trigger-approval",
              "type": "unknown"
            }
          ],
          "status": "completed",
          "type": "commandExecution"
        }
      }
    }
    ```
  • core: add a feature to disable the shell tool (#6481)
    `--disable shell_tool` disables the built-in shell tool. This is useful
    for MCP-only operation.
    
    ---------
    
    Co-authored-by: Michael Bolin <mbolin@openai.com>
  • chore(core) Update shell instructions (#6679)
    ## Summary
    Consolidates `shell` and `shell_command` tool instructions.
    ## Testing 
    - [x] Updated tests, tested locally
  • core/tui: non-blocking MCP startup (#6334)
    This makes MCP startup not block TUI startup. Messages sent while MCPs
    are booting will be queued.
    
    
    https://github.com/user-attachments/assets/96e1d234-5d8f-4932-a935-a675d35c05e0
    
    
    Fixes #6317
    
    ---------
    
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • fix(core) serialize shell_command (#6744)
    ## Summary
    Ensures we're serializing calls to `shell_command`
    
    ## Testing
    - [x] Added unit test
  • feat: better UI for unified_exec (#6515)
    <img width="376" height="132" alt="Screenshot 2025-11-12 at 17 36 22"
    src="https://github.com/user-attachments/assets/ce693f0d-5ca0-462e-b170-c20811dcc8d5"
    />
  • Avoid double truncation (#6631)
    1. Avoid double truncation by giving 10% above the tool default constant
    2. Add tests that fails when const = 1
  • Update default yield time (#6610)
    10s for exec and 250ms for write_stdin
  • Overhaul shell detection and centralize command generation for unified exec (#6577)
    This fixes command display for unified exec. All `cd`s and `ls`es are
    now parsed.
    
    <img width="452" height="237" alt="image"
    src="https://github.com/user-attachments/assets/ce92d81f-f74c-485a-9b34-1eaa29290ec6"
    />
    
    Deletes a ton of tests that were doing nothing from shell.rs.
    
    ---------
    
    Co-authored-by: Pavel Krymets <pavel@krymets.com>
  • Change model picker to include gpt5.1 (#6569)
    - Change the presets
    - Change the tests that make sure we keep the list of tools updated
    - Filter out deprecated models
  • feat: shell_command tool (#6510)
    This adds support for a new variant of the shell tool behind a flag. To
    test, run `codex` with `--enable shell_command_tool`, which will
    register the tool with Codex under the name `shell_command` that accepts
    the following shape:
    
    ```python
    {
      command: str
      workdir: str | None,
      timeout_ms: int | None,
      with_escalated_permissions: bool | None,
      justification: str | None,
    }
    ```
    
    This is comparable to the existing tool registered under
    `shell`/`container.exec`. The primary difference is that it accepts
    `command` as a `str` instead of a `str[]`. The `shell_command` tool
    executes by running `execvp(["bash", "-lc", command])`, though the exact
    arguments to `execvp(3)` depend on the user's default shell.
    
    The hypothesis is that this will simplify things for the model. For
    example, on Windows, instead of generating:
    
    ```json
    {"command": ["pwsh.exe", "-NoLogo", "-Command", "ls -Name"]}
    ```
    
    The model could simply generate:
    
    ```json
    {"command": "ls -Name"}
    ```
    
    As part of this change, I extracted some logic out of `user_shell.rs` as
    `Shell::derive_exec_args()` so that it can be reused in
    `codex-rs/core/src/tools/handlers/shell.rs`. Note the original code
    generated exec arg lists like:
    
    ```javascript
    ["bash", "-lc", command]
    ["zsh", "-lc", command]
    ["pwsh.exe", "-NoProfile", "-Command", command]
    ```
    
    Using `-l` for Bash and Zsh, but then specifying `-NoProfile` for
    PowerShell seemed inconsistent to me, so I changed this in the new
    implementation while also adding a `use_login_shell: bool` option to
    make this explicit. If we decide to add a `login: bool` to
    `ShellCommandToolCallParams` like we have for unified exec:
    
    
    https://github.com/openai/codex/blob/807e2c27f0a9f2e85c50e7e6df5533f0d9b853c7/codex-rs/core/src/tools/handlers/unified_exec.rs#L33-L34
    
    Then this should make it straightforward to support.
  • Add unified exec escalation handling and tests (#6492)
    Similar implementation to the shell tool