27 Commits

• app-server: remove experimental persist_extended_history bool flag (#25712)

  ## Summary
  
  Remove the dead experimental `persistExtendedHistory` app-server flag
  and collapse rollout persistence to the single policy app-server already
  used.
  
  ## What Changed
  
  - Removed `persistExtendedHistory` from v2 thread start/resume/fork
  params and deleted its deprecation notice path.
  - Removed the persistence-mode enums and plumbing through core, rollout,
  and thread-store.
  - Made rollout filtering mode-free, keeping the existing limited
  persisted-history behavior.
  
  ## Test Plan
  
  - `just write-app-server-schema`
  - `cargo nextest run --no-fail-fast -p codex-app-server-protocol
  schema_fixtures`
  - `cargo nextest run --no-fail-fast -p codex-app-server
  thread_shell_command_history_responses_exclude_persisted_command_executions`
  - `cargo nextest run --no-fail-fast -p codex-rollout -p
  codex-thread-store`
  - final `rg` for removed flag/type names

  Owen Lin · 2026-06-01 23:33:42 +00:00
  11e0f3d3ae
• codex: use ThreadStore history for core review forks (#20577)

  - fork loaded parent threads from `ThreadStore` history in core agent
  control paths
  - migrate guardian review fork history to loaded session history instead
  of rereading rollout files
  
  ## Verification
  - `cargo test -p codex-core spawn_agent_fork`

  Tom · 2026-05-05 15:25:19 -07:00
  ee02cf26d6
• feat: split memories part 2 (#19860)

  Keep extracting memories out of core and moving the write trigger in the
  app-server
  This is temporary and it should move at the client level as a follow-up
  This makes core fully independant from `codex-memories-write`
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-28 13:03:28 +02:00
  431ebeaef7
• [codex] Route live thread writes through ThreadStore (#18882)

  Begin migrating the thread write codepaths to ThreadStore.
  
  This starts using ThreadStore inside of core session code, not only in
  the app server code.
  
  Rework the interfaces around thread recording/persistence. We're left
  with the following:
  
  * `ThreadManager`: owns the process-level registry of loaded threads and
  handles cross-thread orchestration: start, resume, fork, lookup, remove,
  and route ops to running CodexThreads.
  * `CodexThread`: represents one loaded/running thread from the outside.
  It is the handle app-server and callers use to submit ops, inspect
  session metadata, and shut the thread down.
  * `LiveThread`: session-owned persistence lifecycle handle for one
  active thread. Core session code uses it to append rollout items,
  materialize lazy persistence, flush, shutdown, discard init-failed
  writers, and load that thread’s persisted history.
  * `ThreadStore`: storage backend abstraction. It answers “how are
  threads persisted, read, listed, updated, archived?” Local and remote
  implementations live behind this trait.
  * `LocalThreadStore`: local ThreadStore implementation. It owns the
  file/sqlite-specific details and keeps RolloutRecorder as a local
  implementation detail.
  
  This is a few too many Thread abstractions for my liking, but they do
  all represent different concepts / needs / layers.
  
  Migration note: in places where the core code explicitly requires a
  path, rather than a thread ID, throw an error if we're running with a
  remote store.
  
  Cover the new local live-writer lifecycle with focused tests and
  preserve app-server thread-start behavior, including ephemeral pathless
  sessions.

  Tom · 2026-04-23 10:17:09 -07:00
  f1923a38b1
• Add sorting/backwardsCursor to thread/list and new thread/turns/list api (#17305)

  To improve performance of UI loads from the app, add two main
  improvements:
  1. The `thread/list` api now gets a `sortDirection` request field and a
  `backwardsCursor` to the response, which lets you paginate forwards and
  backwards from a window. This lets you fetch the first few items to
  display immediately while you paginate to fill in history, then can
  paginate "backwards" on future loads to catch up with any changes since
  the last UI load without a full reload of the entire data set.
  2. Added a new `thread/turns/list` api which also has sortDirection and
  backwardsCursor for the same behavior as `thread/list`, allowing you the
  same small-fetch for immediate display followed by background fill-in
  and resync catchup.

  David de Regt · 2026-04-17 11:49:02 -07:00
  eaf78e43f2
• [codex] Add local thread store listing (#17824)

  Builds on top of #17659 
  
  Move the filesystem + sqlite thread listing-related operations inside of
  a local ThreadStore implementation and call ThreadStore from the places
  that used to perform these filesystem/sqlite operations.
  
  This is the first of a series of PRs that will implement the rest of the
  local ThreadStore.
  
  Testing:
  - added unit tests for the thread store implementation
  - adjusted some unit tests in the realtime + personality packages whose
  callsites changed. Specifically I'm trying to hide ThreadMetadata inside
  of the local implementation and make ThreadMetadata a sqlite
  implementation detail concern rather than a public interface, preferring
  the more generate StoredThread interface instead
  - added a corner case test for the personality migration package that
  wasn't covered by the existing test suite
  - adjust the behavior of searched thread listing to run the existing
  local rollout repair/backfill pass _before_ querying SQLite results, so
  callers using ThreadStore::list_threads do not miss matches after a
  partial metadata warm-up

  Tom · 2026-04-15 11:34:27 -07:00
  cdfcd2ca92
• chore: merge name and title (#17116)

  Merge title and name concept to leverage the sqlite title column and
  have more efficient queries
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  jif-oai · 2026-04-09 18:44:26 +01:00
  12f0e0b0eb
• [codex] reduce module visibility (#16978)

  ## Summary
  - reduce public module visibility across Rust crates, preferring private
  or crate-private modules with explicit crate-root public exports
  - update external call sites and tests to use the intended public crate
  APIs instead of reaching through module trees
  - add the module visibility guideline to AGENTS.md
  
  ## Validation
  - `cargo check --workspace --all-targets --message-format=short` passed
  before the final fix/format pass
  - `just fix` completed successfully
  - `just fmt` completed successfully
  - `git diff --check` passed

  pakrym-oai · 2026-04-07 08:03:35 -07:00
  413c1e1fdf
• Extract rollout into its own crate (#15548)
  Ahmed Ibrahim · 2026-03-24 18:10:53 -07:00
  2e03d8b4d2
• core(rollout): extract rollout module, add listing API, and return file heads (#1634)

  - Move rollout persistence and listing into a dedicated module:
  rollout/{recorder,list}.
  - Expose lightweight conversation listing that returns file paths plus
  the first 5 JSONL records for preview.

  Ahmed Ibrahim · 2025-09-03 07:39:19 +00:00
  d77b33ded7
• chore: unify history loading (#2736)

  We have two ways of loading conversation with a previous history. Fork
  conversation and the experimental resume that we had before. In this PR,
  I am unifying their code path. The path is getting the history items and
  recording them in a brand new conversation. This PR also constraint the
  rollout recorder responsibilities to be only recording to the disk and
  loading from the disk.
  
  The PR also fixes a current bug when we have two forking in a row:
  History 1:
  <Environment Context>
  UserMessage_1
  UserMessage_2
  UserMessage_3
  
  **Fork with n = 1 (only remove one element)**
  History 2:
  <Environment Context>
  UserMessage_1
  UserMessage_2
  <Environment Context>
  
  **Fork with n = 1 (only remove one element)**
  History 2:
  <Environment Context>
  UserMessage_1
  UserMessage_2
  **<Environment Context>**
  
  This shouldn't happen but because we were appending the `<Environment
  Context>` after each spawning and it's considered as _user message_.
  Now, we don't add this message if restoring and old conversation.

  Ahmed Ibrahim · 2025-09-02 22:44:29 +00:00
  431a10fc50
• Following up on #2371 post commit feedback (#2852)

  - Introduce websearch end to complement the begin 
  - Moves the logic of adding the sebsearch tool to
  create_tools_json_for_responses_api
  - Making it the client responsibility to toggle the tool on or off 
  - Other misc in #2371 post commit feedback
  - Show the query:
  
  <img width="1392" height="151" alt="image"
  src="https://github.com/user-attachments/assets/8457f1a6-f851-44cf-bcca-0d4fe460ce89"
  />

  Ahmed Ibrahim · 2025-08-28 19:24:38 -07:00
  9dbe7284d2
• Move models.rs to protocol (#2595)

  Moving models.rs to protocol so we can use them in `Codex` operations

  Ahmed Ibrahim · 2025-08-22 22:18:54 +00:00
  097782c775
• [apply_patch] freeform apply_patch tool (#2576)

  ## Summary
  GPT-5 introduced the concept of [custom
  tools](https://platform.openai.com/docs/guides/function-calling#custom-tools),
  which allow the model to send a raw string result back, simplifying
  json-escape issues. We are migrating gpt-5 to use this by default.
  
  However, gpt-oss models do not support custom tools, only normal
  functions. So we keep both tool definitions, and provide whichever one
  the model family supports.
  
  ## Testing
  - [x] Tested locally with various models
  - [x] Unit tests pass

  Dylan · 2025-08-22 13:42:34 -07:00
  236c4f76a6
• chore: use one write call per item in rollout_writer() (#1679)

  Most of the time, we expect the `String` returned by
  `serde_json::to_string()` to have extra capacity, so `push('\n')` is
  unlikely to allocate, which seems cheaper than an extra `write(2)` call,
  on average?

  Michael Bolin · 2025-07-25 10:43:36 -07:00
  994c9a874d
• Record Git metadata to rollout (#1598)

  # Summary
  
  - Writing effective evals for codex sessions requires context of the
  overall repository state at the moment the session began
  - This change adds this metadata (git repository, branch, commit hash)
  to the top of the rollout of the session (if available - if not it
  doesn't add anything)
  - Currently, this is only effective on a clean working tree, as we can't
  track uncommitted/untracked changes with the current metadata set.
  Ideally in the future we may want to track unclean changes somehow, or
  perhaps prompt the user to stash or commit them.
  
  # Testing
  - Added unit tests
  - `cargo test && cargo clippy --tests && cargo fmt -- --config
  imports_granularity=Item`
  
  ### Resulting Rollout
  <img width="1243" height="127" alt="Screenshot 2025-07-17 at 1 50 00 PM"
  src="https://github.com/user-attachments/assets/68108941-f015-45b2-985c-ea315ce05415"
  />

  vishnu-oai · 2025-07-24 11:35:28 -07:00
  2437a8d17a
• Flaky CI fix (#1647)

  Flushing before sending `TaskCompleteEvent` and ending the submission
  loop to avoid race conditions.

  aibrahim-oai · 2025-07-23 15:03:26 -07:00
  b4ab7c1b73
• Always send entire request context (#1641)

  Always store the entire conversation history.
  Request encrypted COT when not storing Responses.
  Send entire input context instead of sending previous_response_id

  pakrym-oai · 2025-07-23 10:37:45 -07:00
  591cb6149a
• Add session loading support to Codex (#1602)

  ## Summary
  - extend rollout format to store all session data in JSON
  - add resume/write helpers for rollouts
  - track session state after each conversation
  - support `LoadSession` op to resume a previous rollout
  - allow starting Codex with an existing session via
  `experimental_resume` config variable
  
  We need a way later for exploring the available sessions in a user
  friendly way.
  
  ## Testing
  - `cargo test --no-run` *(fails: `cargo: command not found`)*
  
  ------
  https://chatgpt.com/codex/tasks/task_i_68792a29dd5c832190bf6930d3466fba
  
  This video is outdated. you should use `-c experimental_resume:<full
  path>` instead of `--resume <full path>`
  
  
  https://github.com/user-attachments/assets/7a9975c7-aa04-4f4e-899a-9e87defd947a

  aibrahim-oai · 2025-07-18 17:04:04 -07:00
  83eefb55fb
• Storing the sessions in a more organized way for easier look up. (#1596)

  now storing the sessions in `~/.codex/sessions/YYYY/MM/DD/<file>`

  aibrahim-oai · 2025-07-17 10:12:15 -07:00
  fcbcc40f51
• feat: add support for OpenAI tool type, local_shell (#961)

  The new `codex-mini-latest` model expects a new tool with `{"type":
  "local_shell"}`. Its contract is similar to the existing `function` tool
  with `"name": "shell"`, so this takes the `local_shell` tool call into
  `ExecParams` and sends it through the existing
  `handle_container_exec_with_params()` code path.
  
  This also adds the following logic when adding the default set of tools
  to a request:
  
  ```rust
  let default_tools = if self.model.starts_with("codex") {
      &DEFAULT_CODEX_MODEL_TOOLS
  } else {
      &DEFAULT_TOOLS
  };
  ```
  
  That is, if the model name starts with `"codex"`, we add `{"type":
  "local_shell"}` to the list of tools; otherwise, we add the
  aforementioned `shell` tool.
  
  To test this, I ran the TUI with `-m codex-mini-latest` and verified
  that it used the `local_shell` tool. Though I also had some entries in
  `[mcp_servers]` in my personal `config.toml`. The `codex-mini-latest`
  model seemed eager to try the tools from the MCP servers first, so I
  have personally commented them out for now, so keep an eye out if you're
  testing `codex-mini-latest`!
  
  Perhaps we should include more details with `{"type": "local_shell"}` or
  update the following:
  
  
  https://github.com/openai/codex/blob/fd0b1b020818dfe8aaf7eb68425f09e86ab1b819/codex-rs/core/prompt.md
  
  For reference, the corresponding change in the TypeScript CLI is
  https://github.com/openai/codex/pull/951.

  Michael Bolin · 2025-05-16 14:38:08 -07:00
  f48dd99f22
• chore: pin Rust version to 1.86 and use io::Error::other to prepare for 1.87 (#947)

  Previously, our GitHub actions specified the Rust toolchain as
  `dtolnay/rust-toolchain@stable`, which meant the version could change
  out from under us. In this case, the move from 1.86 to 1.87 introduced
  new clippy warnings, causing build failures.
  
  Because it will take a little time to fix all the new clippy warnings,
  this PR pins things to 1.86 for now to unbreak the build.
  
  It also replaces `io::Error::new(io::ErrorKind::Other)` with
  `io::Error::other()` in preparation for 1.87.

  Michael Bolin · 2025-05-15 14:07:16 -07:00
  ec5e82b77c
• chore: expose codex_home via Config (#941)
  Michael Bolin · 2025-05-15 00:30:13 -07:00
  5fc9fc3e3e
• fix: tighten up some logic around session timestamps and ids (#922)

  * update `SessionConfigured` event to include the UUID for the session
  * show the UUID in the Rust TUI
  * use local timestamps in log files instead of UTC
  * include timestamps in log file names for easier discovery

  Michael Bolin · 2025-05-13 19:22:16 -07:00
  e6c206d19d
• feat: include "reasoning" messages in Rust TUI (#892)

  As shown in the screenshot, we now include reasoning messages from the
  model in the TUI under the heading "codex reasoning":
  
  
  ![image](https://github.com/user-attachments/assets/d8eb3dc3-2f9f-4e95-847e-d24b421249a8)
  
  To ensure these are visible by default when using `o4-mini`, this also
  changes the default value for `summary` (formerly `generate_summary`,
  which is deprecated in favor of `summary` according to the docs) from
  unset to `"auto"`.

  Michael Bolin · 2025-05-10 21:43:27 -07:00
  b4785b5f88
• Workspace lints and disallow unwrap (#855)

  Sets submodules to use workspace lints. Added denying unwrap as a
  workspace level lint, which found a couple of cases where we could have
  propagated errors. Also manually labeled ones that were fine by my eye.

  jcoens-openai · 2025-05-08 09:46:18 -07:00
  87cf120873
• feat: save session transcripts when using Rust CLI (#845)

  This adds support for saving transcripts when using the Rust CLI. Like
  the TypeScript CLI, it saves the transcript to `~/.codex/sessions`,
  though it uses JSONL for the file format (and `.jsonl` for the file
  extension) so that even if Codex crashes, what was written to the
  `.jsonl` file should generally still be valid JSONL content.

  Michael Bolin · 2025-05-07 13:49:15 -07:00
  42617f8726