Commit Graph

12 Commits

  • feat: add support for OpenAI tool type, local_shell (#961)
    The new `codex-mini-latest` model expects a new tool with `{"type":
    "local_shell"}`. Its contract is similar to the existing `function` tool
    with `"name": "shell"`, so this takes the `local_shell` tool call into
    `ExecParams` and sends it through the existing
    `handle_container_exec_with_params()` code path.
    
    This also adds the following logic when adding the default set of tools
    to a request:
    
    ```rust
    let default_tools = if self.model.starts_with("codex") {
        &DEFAULT_CODEX_MODEL_TOOLS
    } else {
        &DEFAULT_TOOLS
    };
    ```
    
    That is, if the model name starts with `"codex"`, we add `{"type":
    "local_shell"}` to the list of tools; otherwise, we add the
    aforementioned `shell` tool.
    
    To test this, I ran the TUI with `-m codex-mini-latest` and verified
    that it used the `local_shell` tool. Though I also had some entries in
    `[mcp_servers]` in my personal `config.toml`. The `codex-mini-latest`
    model seemed eager to try the tools from the MCP servers first, so I
    have personally commented them out for now, so keep an eye out if you're
    testing `codex-mini-latest`!
    
    Perhaps we should include more details with `{"type": "local_shell"}` or
    update the following:
    
    
    https://github.com/openai/codex/blob/fd0b1b020818dfe8aaf7eb68425f09e86ab1b819/codex-rs/core/prompt.md
    
    For reference, the corresponding change in the TypeScript CLI is
    https://github.com/openai/codex/pull/951.
  • fix: agent instructions were not being included when ~/.codex/instructions.md was empty (#908)
    I had seen issues where `codex-rs` would not always write files without
    me pressuring it to do so, and between that and the report of
    https://github.com/openai/codex/issues/900, I decided to look into this
    further. I found two serious issues with agent instructions:
    
    (1) We were only sending agent instructions on the first turn, but
    looking at the TypeScript code, we should be sending them on every turn.
    
    (2) There was a serious issue where the agent instructions were
    frequently lost:
    
    * The TypeScript CLI appears to keep writing `~/.codex/instructions.md`:
    https://github.com/openai/codex/blob/55142e3e6caddd1e613b71bcb89385ce5cc708bf/codex-cli/src/utils/config.ts#L586
    * If `instructions.md` is present, the Rust CLI uses the contents of it
    INSTEAD OF the default prompt, even if `instructions.md` is empty:
    https://github.com/openai/codex/blob/55142e3e6caddd1e613b71bcb89385ce5cc708bf/codex-rs/core/src/config.rs#L202-L203
    
    The combination of these two things means that I have been using
    `codex-rs` without these key instructions:
    https://github.com/openai/codex/blob/main/codex-rs/core/prompt.md
    
    Looking at the TypeScript code, it appears we should be concatenating
    these three items every time (if they exist):
    
    * `prompt.md`
    * `~/.codex/instructions.md`
    * nearest `AGENTS.md`
    
    This PR fixes things so that:
    
    * `Config.instructions` is `None` if `instructions.md` is empty
    * `Payload.instructions` is now `&'a str` instead of `Option<&'a
    String>` because we should always have _something_ to send
    * `Prompt` now has a `get_full_instructions()` helper that returns a
    `Cow<str>` that will always include the agent instructions first.
  • Disallow expect via lints (#865)
    Adds `expect()` as a denied lint. Same deal applies with `unwrap()`
    where we now need to put `#[expect(...` on ones that we legit want. Took
    care to enable `expect()` in test contexts.
    
    # Tests
    
    ```
    cargo fmt
    cargo clippy --all-features --all-targets --no-deps -- -D warnings
    cargo test
    ```
  • feat: include "reasoning" messages in Rust TUI (#892)
    As shown in the screenshot, we now include reasoning messages from the
    model in the TUI under the heading "codex reasoning":
    
    
    ![image](https://github.com/user-attachments/assets/d8eb3dc3-2f9f-4e95-847e-d24b421249a8)
    
    To ensure these are visible by default when using `o4-mini`, this also
    changes the default value for `summary` (formerly `generate_summary`,
    which is deprecated in favor of `summary` according to the docs) from
    unset to `"auto"`.
  • fix: get responses API working again in Rust (#872)
    I inadvertently regressed support for the Responses API when adding
    support for the chat completions API in
    https://github.com/openai/codex/pull/862. This should get both APIs
    working again, but the chat completions codepath seems more complex than
    necessary. I'll try to clean that up shortly, but I want to get things
    working again ASAP.
  • feat: support the chat completions API in the Rust CLI (#862)
    This is a substantial PR to add support for the chat completions API,
    which in turn makes it possible to use non-OpenAI model providers (just
    like in the TypeScript CLI):
    
    * It moves a number of structs from `client.rs` to `client_common.rs` so
    they can be shared.
    * It introduces support for the chat completions API in
    `chat_completions.rs`.
    * It updates `ModelProviderInfo` so that `env_key` is `Option<String>`
    instead of `String` (for e.g., ollama) and adds a `wire_api` field
    * It updates `client.rs` to choose between `stream_responses()` and
    `stream_chat_completions()` based on the `wire_api` for the
    `ModelProviderInfo`
    * It updates the `exec` and TUI CLIs to no longer fail if the
    `OPENAI_API_KEY` environment variable is not set
    * It updates the TUI so that `EventMsg::Error` is displayed more
    prominently when it occurs, particularly now that it is important to
    alert users to the `CodexErr::EnvVar` variant.
    * `CodexErr::EnvVar` was updated to include an optional `instructions`
    field so we can preserve the behavior where we direct users to
    https://platform.openai.com if `OPENAI_API_KEY` is not set.
    * Cleaned up the "welcome message" in the TUI to ensure the model
    provider is displayed.
    * Updated the docs in `codex-rs/README.md`.
    
    To exercise the chat completions API from OpenAI models, I added the
    following to my `config.toml`:
    
    ```toml
    model = "gpt-4o"
    model_provider = "openai-chat-completions"
    
    [model_providers.openai-chat-completions]
    name = "OpenAI using Chat Completions"
    base_url = "https://api.openai.com/v1"
    env_key = "OPENAI_API_KEY"
    wire_api = "chat"
    ```
    
    Though to test a non-OpenAI provider, I installed ollama with mistral
    locally on my Mac because ChatGPT said that would be a good match for my
    hardware:
    
    ```shell
    brew install ollama
    ollama serve
    ollama pull mistral
    ```
    
    Then I added the following to my `~/.codex/config.toml`:
    
    ```toml
    model = "mistral"
    model_provider = "ollama"
    ```
    
    Note this code could certainly use more test coverage, but I want to get
    this in so folks can start playing with it.
    
    For reference, I believe https://github.com/openai/codex/pull/247 was
    roughly the comparable PR on the TypeScript side.
  • feat: read model_provider and model_providers from config.toml (#853)
    This is the first step in supporting other model providers in the Rust
    CLI. Specifically, this PR adds support for the new entries in `Config`
    and `ConfigOverrides` to specify a `ModelProviderInfo`, which is the
    basic config needed for an LLM provider. This PR does not get us all the
    way there yet because `client.rs` still categorically appends
    `/responses` to the URL and expects the endpoint to support the OpenAI
    Responses API. Will fix that next!
  • Update cargo to 2024 edition (#842)
    Some effects of this change:
    - New formatting changes across many files. No functionality changes
    should occur from that.
    - Calls to `set_env` are considered unsafe, since this only happens in
    tests we wrap them in `unsafe` blocks
  • feat: support mcp_servers in config.toml (#829)
    This adds initial support for MCP servers in the style of Claude Desktop
    and Cursor. Note this PR is the bare minimum to get things working end
    to end: all configured MCP servers are launched every time Codex is run,
    there is no recovery for MCP servers that crash, etc.
    
    (Also, I took some shortcuts to change some fields of `Session` to be
    `pub(crate)`, which also means there are circular deps between
    `codex.rs` and `mcp_tool_call.rs`, but I will clean that up in a
    subsequent PR.)
    
    `codex-rs/README.md` is updated as part of this PR to explain how to use
    this feature. There is a bit of plumbing to route the new settings from
    `Config` to the business logic in `codex.rs`. The most significant
    chunks for new code are in `mcp_connection_manager.rs` (which defines
    the `McpConnectionManager` struct) and `mcp_tool_call.rs`, which is
    responsible for tool calls.
    
    This PR also introduces new `McpToolCallBegin` and `McpToolCallEnd`
    event types to the protocol, but does not add any handlers for them.
    (See https://github.com/openai/codex/pull/836 for initial usage.)
    
    To test, I added the following to my `~/.codex/config.toml`:
    
    ```toml
    # Local build of https://github.com/hideya/mcp-server-weather-js
    [mcp_servers.weather]
    command = "/Users/mbolin/code/mcp-server-weather-js/dist/index.js"
    args = []
    ```
    
    And then I ran the following:
    
    ```
    codex-rs$ cargo run --bin codex exec 'what is the weather in san francisco'
    [2025-05-06T22:40:05] Task started: 1
    [2025-05-06T22:40:18] Agent message: Here’s the latest National Weather Service forecast for San Francisco (downtown, near 37.77° N, 122.42° W):
    
    This Afternoon (Tue):
    • Sunny, high near 69 °F
    • West-southwest wind around 12 mph
    
    Tonight:
    • Partly cloudy, low around 52 °F
    • SW wind 7–10 mph
    ...
    ```
    
    Note that Codex itself is not able to make network calls, so it would
    not normally be able to get live weather information like this. However,
    the weather MCP is [currently] not run under the Codex sandbox, so it is
    able to hit `api.weather.gov` and fetch current weather information.
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/829).
    * #836
    * __->__ #829
  • feat: add ZDR support to Rust implementation (#642)
    This adds support for the `--disable-response-storage` flag across our
    multiple Rust CLIs to support customers who have opted into Zero-Data
    Retention (ZDR). The analogous changes to the TypeScript CLI were:
    
    * https://github.com/openai/codex/pull/481
    * https://github.com/openai/codex/pull/543
    
    For a client using ZDR, `previous_response_id` will never be available,
    so the `input` field of an API request must include the full transcript
    of the conversation thus far. As such, this PR changes the type of
    `Prompt.input` from `Vec<ResponseInputItem>` to `Vec<ResponseItem>`.
    
    Practically speaking, `ResponseItem` was effectively a "superset" of
    `ResponseInputItem` already. The main difference for us is that
    `ResponseItem` includes the `FunctionCall` variant that we have to
    include as part of the conversation history in the ZDR case.
    
    Another key change in this PR is modifying `try_run_turn()` so that it
    returns the `Vec<ResponseItem>` for the turn in addition to the
    `Vec<ResponseInputItem>` produced by `try_run_turn()`. This is because
    the caller of `run_turn()` needs to record the `Vec<ResponseItem>` when
    ZDR is enabled.
    
    To that end, this PR introduces `ZdrTranscript` (and adds
    `zdr_transcript: Option<ZdrTranscript>` to `struct State` in `codex.rs`)
    to take responsibility for maintaining the conversation transcript in
    the ZDR case.
  • [codex-rs] Reliability pass on networking (#658)
    We currently see a behavior that looks like this:
    ```
    2025-04-25T16:52:24.552789Z  WARN codex_core::codex: stream disconnected - retrying turn (1/10 in 232ms)...
    codex> event: BackgroundEvent { message: "stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 1/10 in 232ms…" }
    2025-04-25T16:52:54.789885Z  WARN codex_core::codex: stream disconnected - retrying turn (2/10 in 418ms)...
    codex> event: BackgroundEvent { message: "stream error: stream disconnected before completion: Transport error: error decoding response body; retrying 2/10 in 418ms…" }
    ```
    
    This PR contains a few different fixes that attempt to resolve/improve
    this:
    1. **Remove overall client timeout.** I think
    [this](https://github.com/openai/codex/pull/658/files#diff-c39945d3c42f29b506ff54b7fa2be0795b06d7ad97f1bf33956f60e3c6f19c19L173)
    is perhaps the big fix -- it looks to me like this was actually timing
    out even if events were still coming through, and that was causing a
    disconnect right in the middle of a healthy stream.
    2. **Cap response sizes.** We were frequently sending MUCH larger
    responses than the upstream typescript `codex`, and that was definitely
    not helping. [Fix
    here](https://github.com/openai/codex/pull/658/files#diff-d792bef59aa3ee8cb0cbad8b176dbfefe451c227ac89919da7c3e536a9d6cdc0R21-R26)
    for that one.
    3. **Much higher idle timeout.** Our idle timeout value was much lower
    than typescript.
    4. **Sub-linear backoff.** We were much too aggressively backing off,
    [this](https://github.com/openai/codex/pull/658/files#diff-5d5959b95c6239e6188516da5c6b7eb78154cd9cfedfb9f753d30a7b6d6b8b06R30-R33)
    makes it sub-exponential but maintains the jitter and such.
    
    I was seeing that `stream error: stream disconnected` behavior
    constantly, and anecdotally I can no longer reproduce. It feels much
    snappier.
  • feat: initial import of Rust implementation of Codex CLI in codex-rs/ (#629)
    As stated in `codex-rs/README.md`:
    
    Today, Codex CLI is written in TypeScript and requires Node.js 22+ to
    run it. For a number of users, this runtime requirement inhibits
    adoption: they would be better served by a standalone executable. As
    maintainers, we want Codex to run efficiently in a wide range of
    environments with minimal overhead. We also want to take advantage of
    operating system-specific APIs to provide better sandboxing, where
    possible.
    
    To that end, we are moving forward with a Rust implementation of Codex
    CLI contained in this folder, which has the following benefits:
    
    - The CLI compiles to small, standalone, platform-specific binaries.
    - Can make direct, native calls to
    [seccomp](https://man7.org/linux/man-pages/man2/seccomp.2.html) and
    [landlock](https://man7.org/linux/man-pages/man7/landlock.7.html) in
    order to support sandboxing on Linux.
    - No runtime garbage collection, resulting in lower memory consumption
    and better, more predictable performance.
    
    Currently, the Rust implementation is materially behind the TypeScript
    implementation in functionality, so continue to use the TypeScript
    implmentation for the time being. We will publish native executables via
    GitHub Releases as soon as we feel the Rust version is usable.