Commit Graph

29 Commits

  • feat: Ctrl+J for newline in Rust TUI, default to one line of height (#926)
    While the `TextArea` used in the Rust TUI is "multiline," it is not like
    an HTML `<textarea>` in that it does not wrap, so there was not much
    benefit to setting `MIN_TEXTAREA_ROWS` to `3`, so this PR changes it to
    `1`. Though there are now three ways to "increase" the height due to
    actual linebreaks:
    
    * paste in multiline content (this worked before this PR)
    * pressing `Ctrl+J` will insert a newline
    * if you have your terminal emulator set such that it is possible to
    press something that `crossterm` interprets as "Enter plus some
    modifier," then now that will also work
    
    Now things look a bit more compact on startup:
    
    <img width="745" alt="image"
    src="https://github.com/user-attachments/assets/86e2857f-f31c-46f5-a80b-1ab2120b266e"
    />
  • fix: change EventMsg enum so every variant takes a single struct (#925)
    https://github.com/openai/codex/pull/922 did this for the
    `SessionConfigured` enum variant, and I think it is generally helpful to
    be able to work with the values as each enum variant as their own type,
    so this converts the remaining variants and updates all of the
    callsites.
    
    Added a simple unit test to verify that the JSON-serialized version of
    `Event` does not have any unexpected nesting.
  • fix: tighten up some logic around session timestamps and ids (#922)
    * update `SessionConfigured` event to include the UUID for the session
    * show the UUID in the Rust TUI
    * use local timestamps in log files instead of UTC
    * include timestamps in log file names for easier discovery
  • feat: introduce --profile for Rust CLI (#921)
    This introduces a much-needed "profile" concept where users can specify
    a collection of options under one name and then pass that via
    `--profile` to the CLI.
    
    This PR introduces the `ConfigProfile` struct and makes it a field of
    `CargoToml`. It further updates
    `Config::load_from_base_config_with_overrides()` to respect
    `ConfigProfile`, overriding default values where appropriate. A detailed
    unit test is added at the end of `config.rs` to verify this behavior.
    
    Details on how to use this feature have also been added to
    `codex-rs/README.md`.
  • fix: fix border style for BottomPane (#893)
    This PR fixes things so that:
    
    * when the `BottomPane` is in the `StatusIndicator` state, the border
    should be dim
    * when the `BottomPane` does not have input focus, the border should be
    dim
    
    To make it easier to enforce this invariant, this PR introduces
    `BottomPane::set_state()` that will:
    
    * update `self.state`
    * call `update_border_for_input_focus()`
    * request a repaint
    
    This should make it easier to enforce other updates for state changes
    going forward.
  • feat: include "reasoning" messages in Rust TUI (#892)
    As shown in the screenshot, we now include reasoning messages from the
    model in the TUI under the heading "codex reasoning":
    
    
    ![image](https://github.com/user-attachments/assets/d8eb3dc3-2f9f-4e95-847e-d24b421249a8)
    
    To ensure these are visible by default when using `o4-mini`, this also
    changes the default value for `summary` (formerly `generate_summary`,
    which is deprecated in favor of `summary` according to the docs) from
    unset to `"auto"`.
  • feat: Allow pasting newlines (#866)
    Noticed that when pasting multi-line blocks, each newline was treated
    like a new submission.
    Update tui to handle Paste directly and map newlines to shift+enter.
    
    # Test
    
    Copied this into clipboard:
    ```
    Do nothing.
    Explain this repo to me.
    ```
    
    Pasted in and saw multi-line input. Hitting Enter then submitted the
    full block.
  • feat: support the chat completions API in the Rust CLI (#862)
    This is a substantial PR to add support for the chat completions API,
    which in turn makes it possible to use non-OpenAI model providers (just
    like in the TypeScript CLI):
    
    * It moves a number of structs from `client.rs` to `client_common.rs` so
    they can be shared.
    * It introduces support for the chat completions API in
    `chat_completions.rs`.
    * It updates `ModelProviderInfo` so that `env_key` is `Option<String>`
    instead of `String` (for e.g., ollama) and adds a `wire_api` field
    * It updates `client.rs` to choose between `stream_responses()` and
    `stream_chat_completions()` based on the `wire_api` for the
    `ModelProviderInfo`
    * It updates the `exec` and TUI CLIs to no longer fail if the
    `OPENAI_API_KEY` environment variable is not set
    * It updates the TUI so that `EventMsg::Error` is displayed more
    prominently when it occurs, particularly now that it is important to
    alert users to the `CodexErr::EnvVar` variant.
    * `CodexErr::EnvVar` was updated to include an optional `instructions`
    field so we can preserve the behavior where we direct users to
    https://platform.openai.com if `OPENAI_API_KEY` is not set.
    * Cleaned up the "welcome message" in the TUI to ensure the model
    provider is displayed.
    * Updated the docs in `codex-rs/README.md`.
    
    To exercise the chat completions API from OpenAI models, I added the
    following to my `config.toml`:
    
    ```toml
    model = "gpt-4o"
    model_provider = "openai-chat-completions"
    
    [model_providers.openai-chat-completions]
    name = "OpenAI using Chat Completions"
    base_url = "https://api.openai.com/v1"
    env_key = "OPENAI_API_KEY"
    wire_api = "chat"
    ```
    
    Though to test a non-OpenAI provider, I installed ollama with mistral
    locally on my Mac because ChatGPT said that would be a good match for my
    hardware:
    
    ```shell
    brew install ollama
    ollama serve
    ollama pull mistral
    ```
    
    Then I added the following to my `~/.codex/config.toml`:
    
    ```toml
    model = "mistral"
    model_provider = "ollama"
    ```
    
    Note this code could certainly use more test coverage, but I want to get
    this in so folks can start playing with it.
    
    For reference, I believe https://github.com/openai/codex/pull/247 was
    roughly the comparable PR on the TypeScript side.
  • fix: remove wrapping in Rust TUI that was incompatible with scrolling math (#868)
    I noticed that sometimes I would enter a new message, but it would not
    show up in the conversation history. Even if I focused the conversation
    history and tried to scroll it to the bottom, I could not bring it into
    view. At first, I was concerned that messages were not making it to the
    UI layer, but I added debug statements and verified that was not the
    issue.
    
    It turned out that, previous to this PR, lines that are wider than the
    viewport take up multiple lines of vertical space because `wrap()` was
    set on the `Paragraph` inside the scroll pane. Unfortunately, that broke
    our "scrollbar math" that assumed each `Line` contributes one line of
    height in the UI.
    
    This PR removes the `wrap()`, but introduces a new issue, which is that
    now you cannot see long lines without resizing your terminal window. For
    now, I filed an issue here:
    
    https://github.com/openai/codex/issues/869
    
    I think the long-term fix is to fix our math so it calculates the height
    of a `Line` after it is wrapped given the current width of the viewport.
  • Workspace lints and disallow unwrap (#855)
    Sets submodules to use workspace lints. Added denying unwrap as a
    workspace level lint, which found a couple of cases where we could have
    propagated errors. Also manually labeled ones that were fine by my eye.
  • feat: read model_provider and model_providers from config.toml (#853)
    This is the first step in supporting other model providers in the Rust
    CLI. Specifically, this PR adds support for the new entries in `Config`
    and `ConfigOverrides` to specify a `ModelProviderInfo`, which is the
    basic config needed for an LLM provider. This PR does not get us all the
    way there yet because `client.rs` still categorically appends
    `/responses` to the URL and expects the endpoint to support the OpenAI
    Responses API. Will fix that next!
  • feat: introduce the use of tui-markdown (#851)
    This introduces the use of the `tui-markdown` crate to parse an
    assistant message as Markdown and style it using ANSI for a better user
    experience. As shown in the screenshot below, it has support for syntax
    highlighting for _tagged_ fenced code blocks:
    
    <img width="907" alt="image"
    src="https://github.com/user-attachments/assets/900dc229-80bb-46e8-b1bb-efee4c70ba3c"
    />
    
    That said, `tui-markdown` is not as configurable (or stylish!) as
    https://www.npmjs.com/package/marked-terminal, which is what we use in
    the TypeScript CLI. In particular:
    
    * The styles are hardcoded and `tui_markdown::from_str()` does not take
    any options whatsoever. It uses "bold white" for inline code style which
    does not stand out as much as the yellow used by `marked-terminal`:
    
    
    https://github.com/joshka/tui-markdown/blob/65402cbda70325f34e7ddf6fe1ec629bcd9459cf/tui-markdown/src/lib.rs#L464
    
    I asked Codex to take a first pass at this and it came up with:
    
    https://github.com/joshka/tui-markdown/pull/80
    
    * If a fenced code block is not tagged, then it does not get
    highlighted. I would rather add some logic here:
    
    
    https://github.com/joshka/tui-markdown/blob/65402cbda70325f34e7ddf6fe1ec629bcd9459cf/tui-markdown/src/lib.rs#L262
    
    that uses something like https://pypi.org/project/guesslang/ to examine
    the value of `text` and try to use the appropriate syntax highlighter.
    
    * When we have a fenced code block, we do not want to show the opening
    and closing triple backticks in the output.
    
    To unblock ourselves, we might want to bundle our own fork of
    `tui-markdown` temporarily until we figure out what the shape of the API
    should be and then try to upstream it.
  • Update cargo to 2024 edition (#842)
    Some effects of this change:
    - New formatting changes across many files. No functionality changes
    should occur from that.
    - Calls to `set_env` are considered unsafe, since this only happens in
    tests we wrap them in `unsafe` blocks
  • chore: introduce codex-common crate (#843)
    I started this PR because I wanted to share the `format_duration()`
    utility function in `codex-rs/exec/src/event_processor.rs` with the TUI.
    The question was: where to put it?
    
    `core` should have as few dependencies as possible, so moving it there
    would introduce a dependency on `chrono`, which seemed undesirable.
    `core` already had this `cli` feature to deal with a similar situation
    around sharing common utility functions, so I decided to:
    
    * make `core` feature-free
    * introduce `common`
    * `common` can have as many "special interest" features as it needs,
    each of which can declare their own deps
    * the first two features of common are `cli` and `elapsed`
    
    In practice, this meant updating a number of `Cargo.toml` files,
    replacing this line:
    
    ```toml
    codex-core = { path = "../core", features = ["cli"] }
    ```
    
    with these:
    
    ```toml
    codex-core = { path = "../core" }
    codex-common = { path = "../common", features = ["cli"] }
    ```
    
    Moving `format_duration()` into its own file gave it some "breathing
    room" to add a unit test, so I had Codex generate some tests and new
    support for durations over 1 minute.
  • feat: drop support for q in the Rust TUI since we already support ctrl+d (#799)
    Out of the box, we will make `/` the only official "escape sequence" for
    commands in the Rust TUI. We will look to support `q` (or any string you
    want to use as a "macro") via a plugin, but not make it part of the
    default experience.
    
    Existing `q` users will have to get by with `ctrl+d` for now.
  • feat: show MCP tool calls in TUI (#836)
    Adds logic for the `McpToolCallBegin` and `McpToolCallEnd` events in
    `codex-rs/tui/src/chatwidget.rs` so they get entries in the conversation
    history in the TUI.
    
    Building on top of https://github.com/openai/codex/pull/829, here is the
    result of running:
    
    ```
    cargo run --bin codex -- 'what is the weather in san francisco tomorrow'
    ```
    
    
    ![image](https://github.com/user-attachments/assets/db4a79bb-4988-46cb-acb2-446d5ba9e058)
  • fix: is_inside_git_repo should take the directory as a param (#809)
    https://github.com/openai/codex/pull/800 made `cwd` a property of
    `Config` and made it so the `cwd` is not necessarily
    `std::env::current_dir()`. As such, `is_inside_git_repo()` should check
    `Config.cwd` rather than `std::env::current_dir()`.
    
    This PR updates `is_inside_git_repo()` to take `Config` instead of an
    arbitrary `PathBuf` to force the check to operate on a `Config` where
    `cwd` has been resolved to what the user specified.
  • fix: TUI should use cwd from Config (#808)
    https://github.com/openai/codex/pull/800 made `cwd` a property of
    `Config`, so the TUI should use this instead of running
    `std::env::current_dir()`.
  • feat: make cwd a required field of Config so we stop assuming std::env::current_dir() in a session (#800)
    In order to expose Codex via an MCP server, I realized that we should be
    taking `cwd` as a parameter rather than assuming
    `std::env::current_dir()` as the `cwd`. Specifically, the user may want
    to start a session in a directory other than the one where the MCP
    server has been started.
    
    This PR makes `cwd: PathBuf` a required field of `Session` and threads
    it all the way through, though I think there is still an issue with not
    honoring `workdir` for `apply_patch`, which is something we also had to
    fix in the TypeScript version: https://github.com/openai/codex/pull/556.
    
    This also adds `-C`/`--cd` to change the cwd via the command line.
    
    To test, I ran:
    
    ```
    cargo run --bin codex -- exec -C /tmp 'show the output of ls'
    ```
    
    and verified it showed the contents of my `/tmp` folder instead of
    `$PWD`.
  • fix: overhaul SandboxPolicy and config loading in Rust (#732)
    Previous to this PR, `SandboxPolicy` was a bit difficult to work with:
    
    
    https://github.com/openai/codex/blob/237f8a11e11fdcc793a09e787e48215676d9b95b/codex-rs/core/src/protocol.rs#L98-L108
    
    Specifically:
    
    * It was an `enum` and therefore options were mutually exclusive as
    opposed to additive.
    * It defined things in terms of what the agent _could not_ do as opposed
    to what they _could_ do. This made things hard to support because we
    would prefer to build up a sandbox config by starting with something
    extremely restrictive and only granting permissions for things the user
    as explicitly allowed.
    
    This PR changes things substantially by redefining the policy in terms
    of two concepts:
    
    * A `SandboxPermission` enum that defines permissions that can be
    granted to the agent/sandbox.
    * A `SandboxPolicy` that internally stores a `Vec<SandboxPermission>`,
    but externally exposes a simpler API that can be used to configure
    Seatbelt/Landlock.
    
    Previous to this PR, we supported a `--sandbox` flag that effectively
    mapped to an enum value in `SandboxPolicy`. Though now that
    `SandboxPolicy` is a wrapper around `Vec<SandboxPermission>`, the single
    `--sandbox` flag no longer makes sense. While I could have turned it
    into a flag that the user can specify multiple times, I think the
    current values to use with such a flag are long and potentially messy,
    so for the moment, I have dropped support for `--sandbox` altogether and
    we can bring it back once we have figured out the naming thing.
    
    Since `--sandbox` is gone, users now have to specify `--full-auto` to
    get a sandbox that allows writes in `cwd`. Admittedly, there is no clean
    way to specify the equivalent of `--full-auto` in your `config.toml`
    right now, so we will have to revisit that, as well.
    
    Because `Config` presents a `SandboxPolicy` field and `SandboxPolicy`
    changed considerably, I had to overhaul how config loading works, as
    well. There are now two distinct concepts, `ConfigToml` and `Config`:
    
    * `ConfigToml` is the deserialization of `~/.codex/config.toml`. As one
    might expect, every field is `Optional` and it is `#[derive(Deserialize,
    Default)]`. Consistent use of `Optional` makes it clear what the user
    has specified explicitly.
    * `Config` is the "normalized config" and is produced by merging
    `ConfigToml` with `ConfigOverrides`. Where `ConfigToml` contains a raw
    `Option<Vec<SandboxPermission>>`, `Config` presents only the final
    `SandboxPolicy`.
    
    The changes to `core/src/exec.rs` and `core/src/linux.rs` merit extra
    special attention to ensure we are faithfully mapping the
    `SandboxPolicy` to the Seatbelt and Landlock configs, respectively.
    
    Also, take note that `core/src/seatbelt_readonly_policy.sbpl` has been
    renamed to `codex-rs/core/src/seatbelt_base_policy.sbpl` and that
    `(allow file-read*)` has been removed from the `.sbpl` file as now this
    is added to the policy in `core/src/exec.rs` when
    `sandbox_policy.has_full_disk_read_access()` is `true`.
  • feat: make it possible to set disable_response_storage = true in config.toml (#714)
    https://github.com/openai/codex/pull/642 introduced support for the
    `--disable-response-storage` flag, but if you are a ZDR customer, it is
    tedious to set this every time, so this PR makes it possible to set this
    once in `config.toml` and be done with it.
    
    Incidentally, this tidies things up such that now `init_codex()` takes
    only one parameter: `Config`.
  • fix: drop d as keyboard shortcut for scrolling in the TUI (#704)
    The existing `b` and `space` are sufficient and `d` and `u` default to
    half-page scrolling in `less`, so the way we supported `d` and `u`
    wasn't faithful to that, anyway:
    
    https://man7.org/linux/man-pages/man1/less.1.html
    
    If we decide to bring `d` and `u` back, they should probably match
    `less`?
  • feat: load defaults into Config and introduce ConfigOverrides (#677)
    This changes how instantiating `Config` works and also adds
    `approval_policy` and `sandbox_policy` as fields. The idea is:
    
    * All fields of `Config` have appropriate default values.
    * `Config` is initially loaded from `~/.codex/config.toml`, so values in
    `config.toml` will override those defaults.
    * Clients must instantiate `Config` via
    `Config::load_with_overrides(ConfigOverrides)` where `ConfigOverrides`
    has optional overrides that are expected to be settable based on CLI
    flags.
    
    The `Config` should be defined early in the program and then passed
    down. Now functions like `init_codex()` take fewer individual parameters
    because they can just take a `Config`.
    
    Also, `Config::load()` used to fail silently if `~/.codex/config.toml`
    had a parse error and fell back to the default config. This seemed
    really bad because it wasn't clear why the values in my `config.toml`
    weren't getting picked up. I changed things so that
    `load_with_overrides()` returns `Result<Config>` and verified that the
    various CLIs print a reasonable error if `config.toml` is malformed.
    
    Finally, I also updated the TUI to show which **sandbox** value is being
    used, as we do for other key values like **model** and **approval**.
    This was also a reminder that the various values of `--sandbox` are
    honored on Linux but not macOS today, so I added some TODOs about fixing
    that.
  • fix: write logs to ~/.codex/log instead of /tmp (#669)
    Previously, the Rust TUI was writing log files to `/tmp`, which is
    world-readable and not available on Windows, so that isn't great.
    
    This PR tries to clean things up by adding a function that provides the
    path to the "Codex config dir," e.g., `~/.codex` (though I suppose we
    could support `$CODEX_HOME` to override this?) and then defines other
    paths in terms of the result of `codex_dir()`.
    
    For example, `log_dir()` returns the folder where log files should be
    written which is defined in terms of `codex_dir()`. I updated the TUI to
    use this function. On UNIX, we even go so far as to `chmod 600` the log
    file by default, though as noted in a comment, it's a bit tedious to do
    the equivalent on Windows, so we just let that go for now.
    
    This also changes the default logging level to `info` for `codex_core`
    and `codex_tui` when `RUST_LOG` is not specified. I'm not really sure if
    we should use a more verbose default (it may be helpful when debugging
    user issues), though if so, we should probably also set up log rotation?
  • feat: add ZDR support to Rust implementation (#642)
    This adds support for the `--disable-response-storage` flag across our
    multiple Rust CLIs to support customers who have opted into Zero-Data
    Retention (ZDR). The analogous changes to the TypeScript CLI were:
    
    * https://github.com/openai/codex/pull/481
    * https://github.com/openai/codex/pull/543
    
    For a client using ZDR, `previous_response_id` will never be available,
    so the `input` field of an API request must include the full transcript
    of the conversation thus far. As such, this PR changes the type of
    `Prompt.input` from `Vec<ResponseInputItem>` to `Vec<ResponseItem>`.
    
    Practically speaking, `ResponseItem` was effectively a "superset" of
    `ResponseInputItem` already. The main difference for us is that
    `ResponseItem` includes the `FunctionCall` variant that we have to
    include as part of the conversation history in the ZDR case.
    
    Another key change in this PR is modifying `try_run_turn()` so that it
    returns the `Vec<ResponseItem>` for the turn in addition to the
    `Vec<ResponseInputItem>` produced by `try_run_turn()`. This is because
    the caller of `run_turn()` needs to record the `Vec<ResponseItem>` when
    ZDR is enabled.
    
    To that end, this PR introduces `ZdrTranscript` (and adds
    `zdr_transcript: Option<ZdrTranscript>` to `struct State` in `codex.rs`)
    to take responsibility for maintaining the conversation transcript in
    the ZDR case.
  • feat(tui-rs): add support for mousewheel scrolling (#641)
    It is intuitive to try to scroll the conversation history using the
    mouse in the TUI, but prior to this change, we only supported scrolling
    via keyboard events.
    
    This PR enables mouse capture upon initialization (and disables it on
    exit) such that we get `ScrollUp` and `ScrollDown` events in
    `codex-rs/tui/src/app.rs`. I initially mapped each event to scrolling by
    one line, but that felt sluggish. I decided to introduce
    `ScrollEventHelper` so we could debounce scroll events and measure the
    number of scroll events in a 100ms window to determine the "magnitude"
    of the scroll event. I put in a basic heuristic to start, but perhaps
    someone more motivated can play with it over time.
    
    `ScrollEventHelper` takes care of handling the atomic fields and thread
    management to ensure an `AppEvent::Scroll` event is pumped back through
    the event loop at the appropriate time with the accumulated delta.
  • feat: initial import of Rust implementation of Codex CLI in codex-rs/ (#629)
    As stated in `codex-rs/README.md`:
    
    Today, Codex CLI is written in TypeScript and requires Node.js 22+ to
    run it. For a number of users, this runtime requirement inhibits
    adoption: they would be better served by a standalone executable. As
    maintainers, we want Codex to run efficiently in a wide range of
    environments with minimal overhead. We also want to take advantage of
    operating system-specific APIs to provide better sandboxing, where
    possible.
    
    To that end, we are moving forward with a Rust implementation of Codex
    CLI contained in this folder, which has the following benefits:
    
    - The CLI compiles to small, standalone, platform-specific binaries.
    - Can make direct, native calls to
    [seccomp](https://man7.org/linux/man-pages/man2/seccomp.2.html) and
    [landlock](https://man7.org/linux/man-pages/man7/landlock.7.html) in
    order to support sandboxing on Linux.
    - No runtime garbage collection, resulting in lower memory consumption
    and better, more predictable performance.
    
    Currently, the Rust implementation is materially behind the TypeScript
    implementation in functionality, so continue to use the TypeScript
    implmentation for the time being. We will publish native executables via
    GitHub Releases as soon as we feel the Rust version is usable.