100 Commits

• chore: refactor handle_function_call() into smaller functions (#965)

  Overall, `codex.rs` is still far too large, but at least there's less
  indenting now that things have been moved into smaller functions.
  
  This will also make it easier to introduce the `local_shell` tool in
  https://github.com/openai/codex/pull/961.
  
  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/965).
  * #961
  * __->__ #965

  Michael Bolin · 2025-05-16 14:17:10 -07:00
  dfd54e1433
• feat: add support for file_opener option in Rust, similiar to #911 (#957)

  This ports the enhancement introduced in
  https://github.com/openai/codex/pull/911 (and the fixes in
  https://github.com/openai/codex/pull/919) for the TypeScript CLI to the
  Rust one.

  Michael Bolin · 2025-05-16 11:33:08 -07:00
  1e39189393
• fix: introduce ExtractHeredocError that implements PartialEq (#958)
  Michael Bolin · 2025-05-16 09:42:27 -07:00
  3d9f4fcd8a
• fix: apply patch issue when using different cwd (#942)

  If you run a codex instance outside of the current working directory
  from where you launched the codex binary it won't be able to apply
  patches correctly, even if the sandbox policy allows it. This manifests
  weird behaviours, such as
  
  * Reading the same filename in the binary working directory, and
  overwriting it in the session working directory. e.g. if you have a
  `readme` in both folders it will overwrite the readme in the session
  working directory with the readme in the binary working directory
  *applied with the suggested patch*.
  * The LLM ends up in weird loops trying to verify and debug why the
  apply_patch won't work, and it can result in it applying patches by
  manually writing python or javascript if it figures out that either is
  supported by the system instead.
  
  I added a test-case to ensure that the patch contents are based on the
  cwd.
  
  ## Issue: mixing relative & absolute paths in apply_patch
  
  1. The apply_patch tool use relative paths based on the session working
  directory.
  2. `unified_diff_from_chunks` eventually ends up [reading the source
  file](https://github.com/reflectionai/codex/blob/main/codex-rs/apply-patch/src/lib.rs#L410)
  to figure out what the diff is, by using the relative path.
  3. The changes are targeted using an absolute path derived from the
  current working directory.
  
  The end-result in case session working directory differs from the binary
  working directory: we get the diff for a file relative to the binary
  working directory, and apply it on a file in the session working
  directory.

  Sebastian Lund · 2025-05-16 09:12:16 -07:00
  84e01f4b62
• chore: update exec crate to use std::time instead of chrono (#952)

  When I originally wrote `elapsed.rs`, I realized we were using both
  `std::time` and `chrono` with no real benefit of having both. We should
  try to keep the `exec` subcommand trim (as it also buildable as a
  standalone executable), so this helps tighten things up.

  Michael Bolin · 2025-05-16 08:14:50 -07:00
  30cbfdfa87
• feat: record messages from user in ~/.codex/history.jsonl (#939)

  This is a large change to support a "history" feature like you would
  expect in a shell like Bash.
  
  History events are recorded in `$CODEX_HOME/history.jsonl`. Because it
  is a JSONL file, it is straightforward to append new entries (as opposed
  to the TypeScript file that uses `$CODEX_HOME/history.json`, so to be
  valid JSON, each new entry entails rewriting the entire file). Because
  it is possible for there to be multiple instances of Codex CLI writing
  to `history.jsonl` at once, we use advisory file locking when working
  with `history.jsonl` in `codex-rs/core/src/message_history.rs`.
  
  Because we believe history is a sufficiently useful feature, we enable
  it by default. Though to provide some safety, we set the file
  permissions of `history.jsonl` to be `o600` so that other users on the
  system cannot read the user's history. We do not yet support a default
  list of `SENSITIVE_PATTERNS` as the TypeScript CLI does:
  
  
  https://github.com/openai/codex/blob/3fdf9df1335ac9501e3fb0e61715359145711e8b/codex-cli/src/utils/storage/command-history.ts#L10-L17
  
  We are going to take a more conservative approach to this list in the
  Rust CLI. For example, while `/\b[A-Za-z0-9-_]{20,}\b/` might exclude
  sensitive information like API tokens, it would also exclude valuable
  information such as references to Git commits.
  
  As noted in the updated documentation, users can opt-out of history by
  adding the following to `config.toml`:
  
  ```toml
  [history]
  persistence = "none" 
  ```
  
  Because `history.jsonl` could, in theory, be quite large, we take a[n
  arguably overly pedantic] approach in reading history entries into
  memory. Specifically, we start by telling the client the current number
  of entries in the history file (`history_entry_count`) as well as the
  inode (`history_log_id`) of `history.jsonl` (see the new fields on
  `SessionConfiguredEvent`).
  
  The client is responsible for keeping new entries in memory to create a
  "local history," but if the user hits up enough times to go "past" the
  end of local history, then the client should use the new
  `GetHistoryEntryRequest` in the protocol to fetch older entries.
  Specifically, it should pass the `history_log_id` it was given
  originally and work backwards from `history_entry_count`. (It should
  really fetch history in batches rather than one-at-a-time, but that is
  something we can improve upon in subsequent PRs.)
  
  The motivation behind this crazy scheme is that it is designed to defend
  against:
  
  * The `history.jsonl` being truncated during the session such that the
  index into the history is no longer consistent with what had been read
  up to that point. We do not yet have logic to enforce a `max_bytes` for
  `history.jsonl`, but once we do, we will aspire to implement it in a way
  that should result in a new inode for the file on most systems.
  * New items from concurrent Codex CLI sessions amending to the history.
  Because, in absence of truncation, `history.jsonl` is an append-only
  log, so long as the client reads backwards from `history_entry_count`,
  it should always get a consistent view of history. (That said, it will
  not be able to read _new_ commands from concurrent sessions, but perhaps
  we will introduce a `/` command to reload latest history or something
  down the road.)
  
  Admittedly, my testing of this feature thus far has been fairly light. I
  expect we will find bugs and introduce enhancements/fixes going forward.

  Michael Bolin · 2025-05-15 16:26:23 -07:00
  ce2ecbe72f
• chore: introduce AppEventSender to help fix clippy warnings and update to Rust 1.87 (#948)

  Moving to Rust 1.87 introduced a clippy warning that
  `SendError<AppEvent>` was too large.
  
  In practice, the only thing we ever did when we got this error was log
  it (if the mspc channel is closed, then the app is likely shutting down
  or something, so there's not much to do...), so this finally motivated
  me to introduce `AppEventSender`, which wraps
  `std::sync::mpsc::Sender<AppEvent>` with a `send()` method that invokes
  `send()` on the underlying `Sender` and logs an `Err` if it gets one.
  
  This greatly simplifies the code, as many functions that previously
  returned `Result<(), SendError<AppEvent>>` now return `()`, so we don't
  have to propagate an `Err` all over the place that we don't really
  handle, anyway.
  
  This also makes it so we can upgrade to Rust 1.87 in CI.

  Michael Bolin · 2025-05-15 14:50:30 -07:00
  3fdf9df133
• chore: pin Rust version to 1.86 and use io::Error::other to prepare for 1.87 (#947)

  Previously, our GitHub actions specified the Rust toolchain as
  `dtolnay/rust-toolchain@stable`, which meant the version could change
  out from under us. In this case, the move from 1.86 to 1.87 introduced
  new clippy warnings, causing build failures.
  
  Because it will take a little time to fix all the new clippy warnings,
  this PR pins things to 1.86 for now to unbreak the build.
  
  It also replaces `io::Error::new(io::ErrorKind::Other)` with
  `io::Error::other()` in preparation for 1.87.

  Michael Bolin · 2025-05-15 14:07:16 -07:00
  ec5e82b77c
• chore: expose codex_home via Config (#941)
  Michael Bolin · 2025-05-15 00:30:13 -07:00
  5fc9fc3e3e
• fix: properly wrap lines in the Rust TUI (#937)

  As discussed on
  https://github.com/openai/codex/commit/699ec5a87f09796d17c0202cd92a1dd4d8b4f3f5#commitcomment-156776835,
  to properly support scrolling long content in Ratatui for a sequence of
  cells, we need to:
  
  * take the `Vec<Line>` for each cell
  * using the wrapping logic we want to use at render time, compute the
  _effective line count_ using `Paragraph::line_count()` (see
  `wrapped_line_count_for_cell()` in this PR)
  * sum up the effective line count to compute the height of the area
  being scrolled
  * given a `scroll_position: usize`, index into the list of "effective
  lines" and accumulate the appropriate `Vec<Line>` for the cells that
  should be displayed
  * take that `Vec<Line>` to create a `Paragraph` and use the same
  line-wrapping policy that was used in `wrapped_line_count_for_cell()`
  * display the resulting `Paragraph` and use the accounting to display a
  scrollbar with the appropriate thumb size and offset without having to
  render the `Vec<Line>` for the full history
  
  With this change, lines wrap as I expect and everything appears to
  redraw correctly as I resize my terminal!

  Michael Bolin · 2025-05-14 16:51:41 -07:00
  0b9ef93da5
• chore: handle all cases for EventMsg (#936)

  For now, this removes the `#[non_exhaustive]` directive on `EventMsg` so
  that we are forced to handle all `EventMsg` by default. (We may revisit
  this if/when we publish `core/` as a `lib` crate.) For now, it is
  helpful to have this as a forcing function because we have effectively
  two UIs (`tui` and `exec`) and usually when we add a new variant to
  `EventMsg`, we want to be sure that we update both.

  Michael Bolin · 2025-05-14 13:36:43 -07:00
  34aa1991f1
• feat: add mcp subcommand to CLI to run Codex as an MCP server (#934)

  Previously, running Codex as an MCP server required a standalone binary
  in our Cargo workspace, but this PR makes it available as a subcommand
  (`mcp`) of the main CLI.
  
  Ran this with:
  
  ```
  RUST_LOG=debug npx @modelcontextprotocol/inspector cargo run --bin codex -- mcp
  ```
  
  and verified it worked as expected in the inspector at
  `http://127.0.0.1:6274/`.

  Michael Bolin · 2025-05-14 13:15:41 -07:00
  497c5396c0
• feat: add support for commands in the Rust TUI (#935)

  Introduces support for slash commands like in the TypeScript CLI. We do
  not support the full set of commands yet, but the core abstraction is
  there now.
  
  In particular, we have a `SlashCommand` enum and due to thoughtful use
  of the [strum](https://crates.io/crates/strum) crate, it requires
  minimal boilerplate to add a new command to the list.
  
  The key new piece of UI is `CommandPopup`, though the keyboard events
  are still handled by `ChatComposer`. The behavior is roughly as follows:
  
  * if the first character in the composer is `/`, the command popup is
  displayed (if you really want to send a message to Codex that starts
  with a `/`, simply put a space before the `/`)
  * while the popup is displayed, up/down can be used to change the
  selection of the popup
  * if there is a selection, hitting tab completes the command, but does
  not send it
  * if there is a selection, hitting enter sends the command
  * if the prefix of the composer matches a command, the command will be
  visible in the popup so the user can see the description (commands could
  take arguments, so additional text may appear after the command name
  itself)
  
  
  https://github.com/user-attachments/assets/39c3e6ee-eeb7-4ef7-a911-466d8184975f
  
  Incidentally, Codex wrote almost all the code for this PR!

  Michael Bolin · 2025-05-14 12:55:49 -07:00
  a12e4b0b31
• chore: move each view used in BottomPane into its own file (#928)

  `BottomPane` was getting a bit unwieldy because it maintained a
  `PaneState` enum with three variants and many of its methods had `match`
  statements to handle each variant. To replace the enum, this PR:
  
  * Introduces a `trait BottomPaneView` that has two implementations:
  `StatusIndicatorView` and `ApprovalModalView`.
  * Migrates `PaneState::TextInput` into its own struct, `ChatComposer`,
  that does **not** implement `BottomPaneView`.
  * Updates `BottomPane` so it has `composer: ChatComposer` and
  `active_view: Option<Box<dyn BottomPaneView<'a> + 'a>>`. The idea is
  that `active_view` takes priority and is displayed when it is `Some`;
  otherwise, `ChatComposer` is displayed.
  * While methods of `BottomPane` often have to check whether
  `active_view` is present to decide which component to delegate to, the
  code is more straightforward than before and introducing new
  implementations of `BottomPaneView` should be less painful.
  
  Because we want to retain the `TextArea` owned by `ChatComposer` even
  when another view is displayed, to keep the ownership logic simple, it
  seemed best to keep `ChatComposer` distinct from `BottomPaneView`.

  Michael Bolin · 2025-05-14 10:13:29 -07:00
  0402aef126
• fix: increase timeout for test_dev_null_write (#933)

  After updating this test in https://github.com/openai/codex/pull/923, I
  have been getting some timeouts with this test in CI, so increasing the
  timeout to match that of `test_writable_root`:
  
  
  https://github.com/openai/codex/blob/327cf41f0ff7f0816a141a260704270ed38c9fa4/codex-rs/core/src/landlock.rs#L211-L213

  Michael Bolin · 2025-05-14 10:06:14 -07:00
  399e819c9b
• Add codespell support (config, workflow to detect/not fix) and make it fix some typos (#903)

  More about codespell: https://github.com/codespell-project/codespell .
  
  I personally introduced it to dozens if not hundreds of projects already
  and so far only positive feedback.
  
  CI workflow has 'permissions' set only to 'read' so also should be safe.
  
  Let me know if just want to take typo fixes in and get rid of the CI
  
  ---------
  
  Signed-off-by: Yaroslav O. Halchenko <debian@onerussian.com>

  Yaroslav Halchenko · 2025-05-14 09:39:49 -07:00
  327cf41f0f
• feat: Ctrl+J for newline in Rust TUI, default to one line of height (#926)

  While the `TextArea` used in the Rust TUI is "multiline," it is not like
  an HTML `<textarea>` in that it does not wrap, so there was not much
  benefit to setting `MIN_TEXTAREA_ROWS` to `3`, so this PR changes it to
  `1`. Though there are now three ways to "increase" the height due to
  actual linebreaks:
  
  * paste in multiline content (this worked before this PR)
  * pressing `Ctrl+J` will insert a newline
  * if you have your terminal emulator set such that it is possible to
  press something that `crossterm` interprets as "Enter plus some
  modifier," then now that will also work
  
  Now things look a bit more compact on startup:
  
  <img width="745" alt="image"
  src="https://github.com/user-attachments/assets/86e2857f-f31c-46f5-a80b-1ab2120b266e"
  />

  Michael Bolin · 2025-05-13 21:42:14 -07:00
  1bf00a3a95
• fix: test_dev_null_write() was not using echo as intended (#923)

  I believe this test meant to verify that echoing content to `/dev/null`
  succeeded, but instead, I believe it was testing the equivalent to `echo
  'blah > /dev/null'`.

  Michael Bolin · 2025-05-13 21:40:26 -07:00
  5bf9445351
• fix: change EventMsg enum so every variant takes a single struct (#925)

  https://github.com/openai/codex/pull/922 did this for the
  `SessionConfigured` enum variant, and I think it is generally helpful to
  be able to work with the values as each enum variant as their own type,
  so this converts the remaining variants and updates all of the
  callsites.
  
  Added a simple unit test to verify that the JSON-serialized version of
  `Event` does not have any unexpected nesting.

  Michael Bolin · 2025-05-13 20:44:42 -07:00
  a5f3a34827
• fix: tighten up some logic around session timestamps and ids (#922)

  * update `SessionConfigured` event to include the UUID for the session
  * show the UUID in the Rust TUI
  * use local timestamps in log files instead of UTC
  * include timestamps in log file names for easier discovery

  Michael Bolin · 2025-05-13 19:22:16 -07:00
  e6c206d19d
• feat: introduce --profile for Rust CLI (#921)

  This introduces a much-needed "profile" concept where users can specify
  a collection of options under one name and then pass that via
  `--profile` to the CLI.
  
  This PR introduces the `ConfigProfile` struct and makes it a field of
  `CargoToml`. It further updates
  `Config::load_from_base_config_with_overrides()` to respect
  `ConfigProfile`, overriding default values where appropriate. A detailed
  unit test is added at the end of `config.rs` to verify this behavior.
  
  Details on how to use this feature have also been added to
  `codex-rs/README.md`.

  Michael Bolin · 2025-05-13 16:52:52 -07:00
  3c03c25e56
• restructure flake for codex-rs (#888)

  Right now since the repo is having two different implementations of
  codex, flake was updated to work with both typescript implementation and
  rust implementation

  Adeeb · 2025-05-13 13:08:42 -07:00
  ae809f3721
• fix: agent instructions were not being included when ~/.codex/instructions.md was empty (#908)

  I had seen issues where `codex-rs` would not always write files without
  me pressuring it to do so, and between that and the report of
  https://github.com/openai/codex/issues/900, I decided to look into this
  further. I found two serious issues with agent instructions:
  
  (1) We were only sending agent instructions on the first turn, but
  looking at the TypeScript code, we should be sending them on every turn.
  
  (2) There was a serious issue where the agent instructions were
  frequently lost:
  
  * The TypeScript CLI appears to keep writing `~/.codex/instructions.md`:
  https://github.com/openai/codex/blob/55142e3e6caddd1e613b71bcb89385ce5cc708bf/codex-cli/src/utils/config.ts#L586
  * If `instructions.md` is present, the Rust CLI uses the contents of it
  INSTEAD OF the default prompt, even if `instructions.md` is empty:
  https://github.com/openai/codex/blob/55142e3e6caddd1e613b71bcb89385ce5cc708bf/codex-rs/core/src/config.rs#L202-L203
  
  The combination of these two things means that I have been using
  `codex-rs` without these key instructions:
  https://github.com/openai/codex/blob/main/codex-rs/core/prompt.md
  
  Looking at the TypeScript code, it appears we should be concatenating
  these three items every time (if they exist):
  
  * `prompt.md`
  * `~/.codex/instructions.md`
  * nearest `AGENTS.md`
  
  This PR fixes things so that:
  
  * `Config.instructions` is `None` if `instructions.md` is empty
  * `Payload.instructions` is now `&'a str` instead of `Option<&'a
  String>` because we should always have _something_ to send
  * `Prompt` now has a `get_full_instructions()` helper that returns a
  `Cow<str>` that will always include the agent instructions first.

  Michael Bolin · 2025-05-12 17:24:44 -07:00
  61b881d4e5
• fix: use "thinking" instead of "codex reasoning" as the label for reasoning events in the TUI (#905)
  Michael Bolin · 2025-05-12 15:19:45 -07:00
  55142e3e6c
• fix: navigate initialization phase before tools/list request in MCP client (#904)

  Apparently the MCP server implemented in JavaScript did not require the
  `initialize` handshake before responding to tool list/call, so I missed
  this.

  Michael Bolin · 2025-05-12 15:15:26 -07:00
  115fb0b95d
• Disallow expect via lints (#865)

  Adds `expect()` as a denied lint. Same deal applies with `unwrap()`
  where we now need to put `#[expect(...` on ones that we legit want. Took
  care to enable `expect()` in test contexts.
  
  # Tests
  
  ```
  cargo fmt
  cargo clippy --all-features --all-targets --no-deps -- -D warnings
  cargo test
  ```

  jcoens-openai · 2025-05-12 08:45:46 -07:00
  f3bd143867
• fix: fix border style for BottomPane (#893)

  This PR fixes things so that:
  
  * when the `BottomPane` is in the `StatusIndicator` state, the border
  should be dim
  * when the `BottomPane` does not have input focus, the border should be
  dim
  
  To make it easier to enforce this invariant, this PR introduces
  `BottomPane::set_state()` that will:
  
  * update `self.state`
  * call `update_border_for_input_focus()`
  * request a repaint
  
  This should make it easier to enforce other updates for state changes
  going forward.

  Michael Bolin · 2025-05-10 23:34:13 -07:00
  a1f51bf91b
• feat: include "reasoning" messages in Rust TUI (#892)

  As shown in the screenshot, we now include reasoning messages from the
  model in the TUI under the heading "codex reasoning":
  
  
  ![image](https://github.com/user-attachments/assets/d8eb3dc3-2f9f-4e95-847e-d24b421249a8)
  
  To ensure these are visible by default when using `o4-mini`, this also
  changes the default value for `summary` (formerly `generate_summary`,
  which is deprecated in favor of `summary` according to the docs) from
  unset to `"auto"`.

  Michael Bolin · 2025-05-10 21:43:27 -07:00
  b4785b5f88
• feat: add support for AGENTS.md in Rust CLI (#885)

  The TypeScript CLI already has support for including the contents of
  `AGENTS.md` in the instructions sent with the first turn of a
  conversation. This PR brings this functionality to the Rust CLI.
  
  To be considered, `AGENTS.md` must be in the `cwd` of the session, or in
  one of the parent folders up to a Git/filesystem root (whichever is
  encountered first).
  
  By default, a maximum of 32 KiB of `AGENTS.md` will be included, though
  this is configurable using the new-in-this-PR `project_doc_max_bytes`
  option in `config.toml`.

  Michael Bolin · 2025-05-10 17:52:59 -07:00
  2b122da087
• feat: experimental env var: CODEX_SANDBOX_NETWORK_DISABLED (#879)

  When using Codex to develop Codex itself, I noticed that sometimes it
  would try to add `#[ignore]` to the following tests:
  
  ```
  keeps_previous_response_id_between_tasks()
  retries_on_early_close()
  ```
  
  Both of these tests start a `MockServer` that launches an HTTP server on
  an ephemeral port and requires network access to hit it, which the
  Seatbelt policy associated with `--full-auto` correctly denies. If I
  wasn't paying attention to the code that Codex was generating, one of
  these `#[ignore]` annotations could have slipped into the codebase,
  effectively disabling the test for everyone.
  
  To that end, this PR enables an experimental environment variable named
  `CODEX_SANDBOX_NETWORK_DISABLED` that is set to `1` if the
  `SandboxPolicy` used to spawn the process does not have full network
  access. I say it is "experimental" because I'm not convinced this API is
  quite right, but we need to start somewhere. (It might be more
  appropriate to have an env var like `CODEX_SANDBOX=full-auto`, but the
  challenge is that our newer `SandboxPolicy` abstraction does not map to
  a simple set of enums like in the TypeScript CLI.)
  
  We leverage this new functionality by adding the following code to the
  aforementioned tests as a way to "dynamically disable" them:
  
  ```rust
  if std::env::var(CODEX_SANDBOX_NETWORK_DISABLED_ENV_VAR).is_ok() {
      println!(
          "Skipping test because it cannot execute when network is disabled in a Codex sandbox."
      );
      return;
  }
  ```
  
  We can use the `debug seatbelt --full-auto` command to verify that
  `cargo test` fails when run under Seatbelt prior to this change:
  
  ```
  $ cargo run --bin codex -- debug seatbelt --full-auto -- cargo test
  ---- keeps_previous_response_id_between_tasks stdout ----
  
  thread 'keeps_previous_response_id_between_tasks' panicked at /Users/mbolin/.cargo/registry/src/index.crates.io-1949cf8c6b5b557f/wiremock-0.6.3/src/mock_server/builder.rs:107:46:
  Failed to bind an OS port for a mock server.: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
  
  
  failures:
      keeps_previous_response_id_between_tasks
  
  test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s
  
  error: test failed, to rerun pass `-p codex-core --test previous_response_id`
  ```
  
  Though after this change, the above command succeeds! This means that,
  going forward, when Codex operates on Codex itself, when it runs `cargo
  test`, only "real failures" should cause the command to fail.
  
  As part of this change, I decided to tighten up the codepaths for
  running `exec()` for shell tool calls. In particular, we do it in `core`
  for the main Codex business logic itself, but we also expose this logic
  via `debug` subcommands in the CLI in the `cli` crate. The logic for the
  `debug` subcommands was not quite as faithful to the true business logic
  as I liked, so I:
  
  * refactored a bit of the Linux code, splitting `linux.rs` into
  `linux_exec.rs` and `landlock.rs` in the `core` crate.
  * gating less code behind `#[cfg(target_os = "linux")]` because such
  code does not get built by default when I develop on Mac, which means I
  either have to build the code in Docker or wait for CI signal
  * introduced `macro_rules! configure_command` in `exec.rs` so we can
  have both sync and async versions of this code. The synchronous version
  seems more appropriate for straight threads or potentially fork/exec.

  Michael Bolin · 2025-05-09 18:29:34 -07:00
  fde48aaa0d
• feat: Allow pasting newlines (#866)

  Noticed that when pasting multi-line blocks, each newline was treated
  like a new submission.
  Update tui to handle Paste directly and map newlines to shift+enter.
  
  # Test
  
  Copied this into clipboard:
  ```
  Do nothing.
  Explain this repo to me.
  ```
  
  Pasted in and saw multi-line input. Hitting Enter then submitted the
  full block.

  jcoens-openai · 2025-05-09 11:33:46 -07:00
  78843c3940
• chore: refactor exec() into spawn_child() and consume_truncated_output() (#878)

  This PR is a straight refactor so that creating the `Child` process for
  an `shell` tool call and consuming its output can be separate concerns.
  For the actual tool call, we will always apply
  `consume_truncated_output()`, but for the top-level debug commands in
  the CLI (e.g., `debug seatbelt` and `debug landlock`), we only want to
  use the `spawn_child()` part of `exec()`.
  
  We want the subcommands to match the `shell` tool call usage as
  faithfully as possible. This becomes more important when we introduce a
  new parameter to `spawn_child()` in
  https://github.com/openai/codex/pull/879.
  
  ---
  [//]: # (BEGIN SAPLING FOOTER)
  Stack created with [Sapling](https://sapling-scm.com). Best reviewed
  with [ReviewStack](https://reviewstack.dev/openai/codex/pull/878).
  * #879
  * __->__ #878

  Michael Bolin · 2025-05-09 11:03:58 -07:00
  93817643ee
• fix: make McpConnectionManager tolerant of MCPs that fail to start (#854)

  I added a typo in my `config.toml` such that the `command` for one of my
  `mcp_servers` did not exist and I verified that the error was surfaced
  in the TUI (and that I was still able to use Codex).
  
  
  ![image](https://github.com/user-attachments/assets/f13cc08c-f4c6-40ec-9ab4-a9d75e03152f)

  Michael Bolin · 2025-05-08 23:45:54 -07:00
  27198bfe11
• fix: get responses API working again in Rust (#872)

  I inadvertently regressed support for the Responses API when adding
  support for the chat completions API in
  https://github.com/openai/codex/pull/862. This should get both APIs
  working again, but the chat completions codepath seems more complex than
  necessary. I'll try to clean that up shortly, but I want to get things
  working again ASAP.

  Michael Bolin · 2025-05-08 22:49:15 -07:00
  b940adae8e
• feat: support the chat completions API in the Rust CLI (#862)

  This is a substantial PR to add support for the chat completions API,
  which in turn makes it possible to use non-OpenAI model providers (just
  like in the TypeScript CLI):
  
  * It moves a number of structs from `client.rs` to `client_common.rs` so
  they can be shared.
  * It introduces support for the chat completions API in
  `chat_completions.rs`.
  * It updates `ModelProviderInfo` so that `env_key` is `Option<String>`
  instead of `String` (for e.g., ollama) and adds a `wire_api` field
  * It updates `client.rs` to choose between `stream_responses()` and
  `stream_chat_completions()` based on the `wire_api` for the
  `ModelProviderInfo`
  * It updates the `exec` and TUI CLIs to no longer fail if the
  `OPENAI_API_KEY` environment variable is not set
  * It updates the TUI so that `EventMsg::Error` is displayed more
  prominently when it occurs, particularly now that it is important to
  alert users to the `CodexErr::EnvVar` variant.
  * `CodexErr::EnvVar` was updated to include an optional `instructions`
  field so we can preserve the behavior where we direct users to
  https://platform.openai.com if `OPENAI_API_KEY` is not set.
  * Cleaned up the "welcome message" in the TUI to ensure the model
  provider is displayed.
  * Updated the docs in `codex-rs/README.md`.
  
  To exercise the chat completions API from OpenAI models, I added the
  following to my `config.toml`:
  
  ```toml
  model = "gpt-4o"
  model_provider = "openai-chat-completions"
  
  [model_providers.openai-chat-completions]
  name = "OpenAI using Chat Completions"
  base_url = "https://api.openai.com/v1"
  env_key = "OPENAI_API_KEY"
  wire_api = "chat"
  ```
  
  Though to test a non-OpenAI provider, I installed ollama with mistral
  locally on my Mac because ChatGPT said that would be a good match for my
  hardware:
  
  ```shell
  brew install ollama
  ollama serve
  ollama pull mistral
  ```
  
  Then I added the following to my `~/.codex/config.toml`:
  
  ```toml
  model = "mistral"
  model_provider = "ollama"
  ```
  
  Note this code could certainly use more test coverage, but I want to get
  this in so folks can start playing with it.
  
  For reference, I believe https://github.com/openai/codex/pull/247 was
  roughly the comparable PR on the TypeScript side.

  Michael Bolin · 2025-05-08 21:46:06 -07:00
  e924070cee
• fix: enable clippy on tests (#870)

  https://github.com/openai/codex/pull/855 added the clippy warning to
  disallow `unwrap()`, but apparently we were not verifying that tests
  were "clippy clean" in CI, so I ended up with a lot of local errors in
  VS Code.
  
  This turns on the check in CI and fixes the offenders.

  Michael Bolin · 2025-05-08 16:02:56 -07:00
  a9adb4175c
• fix: remove wrapping in Rust TUI that was incompatible with scrolling math (#868)

  I noticed that sometimes I would enter a new message, but it would not
  show up in the conversation history. Even if I focused the conversation
  history and tried to scroll it to the bottom, I could not bring it into
  view. At first, I was concerned that messages were not making it to the
  UI layer, but I added debug statements and verified that was not the
  issue.
  
  It turned out that, previous to this PR, lines that are wider than the
  viewport take up multiple lines of vertical space because `wrap()` was
  set on the `Paragraph` inside the scroll pane. Unfortunately, that broke
  our "scrollbar math" that assumed each `Line` contributes one line of
  height in the UI.
  
  This PR removes the `wrap()`, but introduces a new issue, which is that
  now you cannot see long lines without resizing your terminal window. For
  now, I filed an issue here:
  
  https://github.com/openai/codex/issues/869
  
  I think the long-term fix is to fix our math so it calculates the height
  of a `Line` after it is wrapped given the current width of the viewport.

  Michael Bolin · 2025-05-08 15:17:17 -07:00
  699ec5a87f
• Workspace lints and disallow unwrap (#855)

  Sets submodules to use workspace lints. Added denying unwrap as a
  workspace level lint, which found a couple of cases where we could have
  propagated errors. Also manually labeled ones that were fine by my eye.

  jcoens-openai · 2025-05-08 09:46:18 -07:00
  87cf120873
• fix: remove clap dependency from core crate (#860)
  Michael Bolin · 2025-05-07 19:33:09 -07:00
  9fdf2fa066
• feat: read model_provider and model_providers from config.toml (#853)

  This is the first step in supporting other model providers in the Rust
  CLI. Specifically, this PR adds support for the new entries in `Config`
  and `ConfigOverrides` to specify a `ModelProviderInfo`, which is the
  basic config needed for an LLM provider. This PR does not get us all the
  way there yet because `client.rs` still categorically appends
  `/responses` to the URL and expects the endpoint to support the OpenAI
  Responses API. Will fix that next!

  Michael Bolin · 2025-05-07 17:38:28 -07:00
  86022f097e
• fix: creating an instance of Codex requires a Config (#859)

  I discovered that I accidentally introduced a change in
  https://github.com/openai/codex/pull/829 where we load a fresh `Config`
  in the middle of `codex.rs`:
  
  
  https://github.com/openai/codex/blob/c3e10e180a341e719f61014ea508f6d9dbffe05b/codex-rs/core/src/codex.rs#L515-L522
  
  This is not good because the `Config` could differ from the one that has
  the user's overrides specified from the CLI. Also, in unit tests, it
  means the `Config` was picking up my personal settings as opposed to
  using a vanilla config, which was problematic.
  
  This PR cleans things up by moving the common case where
  `Op::ConfigureSession` is derived from `Config` (originally done in
  `codex_wrapper.rs`) and making it the standard way to initialize `Codex`
  by putting it in `Codex::spawn()`. Note this also eliminates quite a bit
  of boilerplate from the tests and relieves the caller of the
  responsibility of minting out unique IDs when invoking `submit()`.

  Michael Bolin · 2025-05-07 16:33:28 -07:00
  cfe50c7107
• fix: remove CodexBuilder and Recorder (#858)

  These abstractions were originally created exclusively for the REPL,
  which was removed in https://github.com/openai/codex/pull/754.
  Currently, the create some unnecessary Tokio tasks, so we are better off
  without them. (We can always bring this back if we have a new use case.)

  Michael Bolin · 2025-05-07 16:11:42 -07:00
  c3e10e180a
• feat: save session transcripts when using Rust CLI (#845)

  This adds support for saving transcripts when using the Rust CLI. Like
  the TypeScript CLI, it saves the transcript to `~/.codex/sessions`,
  though it uses JSONL for the file format (and `.jsonl` for the file
  extension) so that even if Codex crashes, what was written to the
  `.jsonl` file should generally still be valid JSONL content.

  Michael Bolin · 2025-05-07 13:49:15 -07:00
  42617f8726
• fix: add optional timeout to McpClient::send_request() (#852)

  We now impose a 10s timeout on the initial `tools/list` request to an
  MCP server. We do not apply a timeout for other types of requests yet,
  but we should start enforcing those, as well.

  Michael Bolin · 2025-05-07 12:56:38 -07:00
  9da6ebef3f
• feat: introduce the use of tui-markdown (#851)

  This introduces the use of the `tui-markdown` crate to parse an
  assistant message as Markdown and style it using ANSI for a better user
  experience. As shown in the screenshot below, it has support for syntax
  highlighting for _tagged_ fenced code blocks:
  
  <img width="907" alt="image"
  src="https://github.com/user-attachments/assets/900dc229-80bb-46e8-b1bb-efee4c70ba3c"
  />
  
  That said, `tui-markdown` is not as configurable (or stylish!) as
  https://www.npmjs.com/package/marked-terminal, which is what we use in
  the TypeScript CLI. In particular:
  
  * The styles are hardcoded and `tui_markdown::from_str()` does not take
  any options whatsoever. It uses "bold white" for inline code style which
  does not stand out as much as the yellow used by `marked-terminal`:
  
  
  https://github.com/joshka/tui-markdown/blob/65402cbda70325f34e7ddf6fe1ec629bcd9459cf/tui-markdown/src/lib.rs#L464
  
  I asked Codex to take a first pass at this and it came up with:
  
  https://github.com/joshka/tui-markdown/pull/80
  
  * If a fenced code block is not tagged, then it does not get
  highlighted. I would rather add some logic here:
  
  
  https://github.com/joshka/tui-markdown/blob/65402cbda70325f34e7ddf6fe1ec629bcd9459cf/tui-markdown/src/lib.rs#L262
  
  that uses something like https://pypi.org/project/guesslang/ to examine
  the value of `text` and try to use the appropriate syntax highlighter.
  
  * When we have a fenced code block, we do not want to show the opening
  and closing triple backticks in the output.
  
  To unblock ourselves, we might want to bundle our own fork of
  `tui-markdown` temporarily until we figure out what the shape of the API
  should be and then try to upstream it.

  Michael Bolin · 2025-05-07 10:46:32 -07:00
  0360b4d0d7
• Update submodules version to come from the workspace (#850)

  Tie the version of submodules to the workspace version.

  jcoens-openai · 2025-05-07 10:08:06 -07:00
  a080d7b0fd
• Update cargo to 2024 edition (#842)

  Some effects of this change:
  - New formatting changes across many files. No functionality changes
  should occur from that.
  - Calls to `set_env` are considered unsafe, since this only happens in
  tests we wrap them in `unsafe` blocks

  jcoens-openai · 2025-05-07 08:37:48 -07:00
  8a89d3aeda
• chore: introduce codex-common crate (#843)

  I started this PR because I wanted to share the `format_duration()`
  utility function in `codex-rs/exec/src/event_processor.rs` with the TUI.
  The question was: where to put it?
  
  `core` should have as few dependencies as possible, so moving it there
  would introduce a dependency on `chrono`, which seemed undesirable.
  `core` already had this `cli` feature to deal with a similar situation
  around sharing common utility functions, so I decided to:
  
  * make `core` feature-free
  * introduce `common`
  * `common` can have as many "special interest" features as it needs,
  each of which can declare their own deps
  * the first two features of common are `cli` and `elapsed`
  
  In practice, this meant updating a number of `Cargo.toml` files,
  replacing this line:
  
  ```toml
  codex-core = { path = "../core", features = ["cli"] }
  ```
  
  with these:
  
  ```toml
  codex-core = { path = "../core" }
  codex-common = { path = "../common", features = ["cli"] }
  ```
  
  Moving `format_duration()` into its own file gave it some "breathing
  room" to add a unit test, so I had Codex generate some tests and new
  support for durations over 1 minute.

  Michael Bolin · 2025-05-06 17:38:56 -07:00
  c577e94b67
• feat: show MCP tool calls in codex exec subcommand (#841)

  This is analogous to the change for the TUI in
  https://github.com/openai/codex/pull/836, but for `codex exec`.
  
  To test, I ran:
  
  ```
  cargo run --bin codex-exec -- 'what is the weather in wellesley ma tomorrow'
  ```
  
  and saw:
  
  
  ![image](https://github.com/user-attachments/assets/5714e07f-88c7-4dd9-aa0d-be54c1670533)

  Michael Bolin · 2025-05-06 16:52:43 -07:00
  7d8b38b37b
• feat: drop support for q in the Rust TUI since we already support ctrl+d (#799)

  Out of the box, we will make `/` the only official "escape sequence" for
  commands in the Rust TUI. We will look to support `q` (or any string you
  want to use as a "macro") via a plugin, but not make it part of the
  default experience.
  
  Existing `q` users will have to get by with `ctrl+d` for now.

  Michael Bolin · 2025-05-06 16:34:17 -07:00
  6f87f4c69f