33 Commits

• turn metadata followups (#11782)

  some trivial simplifications from #11677

  pash-openai · 2026-02-13 14:59:16 -08:00
  a5e8e69d18
• turn metadata: per-turn non-blocking (#11677)
  pash-openai · 2026-02-13 12:48:29 -08:00
  6c0a924203
• Parse first order skill/connector mentions (#11547)

  This PR introduces a skill-expansion mechanism for mentions so nested or
  skill or connection mentions are expanded if present in skills invoked
  by the user. This keeps behavior aligned with existing mention handling
  while extending coverage to deeper scenarios. With these changes, users
  can create skills that invoke connectors, and skills that invoke other
  skills.
  
  Replaces #10863, which is not needed with the addition of
  [search_tool_bm25](https://github.com/openai/codex/issues/10657)

  canvrno-oai · 2026-02-12 10:55:22 -08:00
  22fa283511
• Fix flaky pre_sampling_compact switch test (#11573)

  Summary
  - address the nondeterministic behavior observed in
  `pre_sampling_compact_runs_on_switch_to_smaller_context_model` so it no
  longer fails intermittently during model switches
  - ensure the surrounding sampling logic consistently handles the
  smaller-context case that the test exercises
  
  Testing
  - Not run (not requested)

  jif-oai · 2026-02-12 11:40:48 +00:00
  3cd93c00ac
• Pre-sampling compact with previous model context (#11504)

  - Run pre-sampling compact through a single helper that builds
  previous-model turn context and compacts before the follow-up request
  when switching to a smaller context window.
  - Keep compaction events on the parent turn id and add compact suite
  coverage for switch-in-session and resume+switch flows.

  Ahmed Ibrahim · 2026-02-11 17:24:06 -08:00
  6938150c5e
• Hydrate previous model across resume/fork/rollback/task start (#11497)

  - Replace pending resume model state with persistent previous_model and
  hydrate it on resume, fork, rollback, and task end in spawn_task

  Ahmed Ibrahim · 2026-02-11 16:45:18 -08:00
  bb5dfd037a
• chore: persist turn_id in rollout session and make turn_id uuid based (#11246)

  Problem:
  1. turn id is constructed in-memory;
  2. on resuming threads, turn_id might not be unique;
  3. client cannot no the boundary of a turn from rollout files easily.
  
  This PR does three things:
  1. persist `task_started` and `task_complete` events;
  1. persist `turn_id` in rollout turn events;
  5. generate turn_id as unique uuids instead of incrementing it in
  memory.
  
  This helps us resolve the issue of clients wanting to have unique turn
  ids for resuming a thread, and knowing the boundry of each turn in
  rollout files.
  
  example debug logs
  ```
  2026-02-11T00:32:10.746876Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=8 turn=Turn { id: "019c4a07-d809-74c3-bc4b-fd9618487b4b", items: [UserMessage { id: "item-24", content: [Text { text: "hi", text_elements: [] }] }, AgentMessage { id: "item-25", text: "Hi. I’m in the workspace with your current changes loaded and ready. Send the next task and I’ll execute it end-to-end." }], status: Completed, error: None }
  2026-02-11T00:32:10.746888Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=9 turn=Turn { id: "019c4a18-1004-76c0-a0fb-a77610f6a9b8", items: [UserMessage { id: "item-26", content: [Text { text: "hello", text_elements: [] }] }, AgentMessage { id: "item-27", text: "Hello. Ready for the next change in `codex-rs`; I can continue from the current in-progress diff or start a new task." }], status: Completed, error: None }
  2026-02-11T00:32:10.746899Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=10 turn=Turn { id: "019c4a19-41f0-7db0-ad78-74f1503baeb8", items: [UserMessage { id: "item-28", content: [Text { text: "hello", text_elements: [] }] }, AgentMessage { id: "item-29", text: "Hello. Send the specific change you want in `codex-rs`, and I’ll implement it and run the required checks." }], status: Completed, error: None }
  ```
  
  backward compatibility:
  if you try to resume an old session without task_started and
  task_complete event populated, the following happens:
  - If you resume and do nothing: those reconstructed historical IDs can
  differ next time you resume.
  - If you resume and send a new turn: the new turn gets a fresh UUID from
  live submission flow and is persisted, so that new turn’s ID is stable
  on later resumes.
  I think this behavior is fine, because we only care about deterministic
  turn id once a turn is triggered.

  Celia Chen · 2026-02-11 03:56:01 +00:00
  641d5268fa
• feat: search_tool (#10657)

  **Why We Did This**
  - The goal is to reduce MCP tool context pollution by not exposing the
  full MCP tool list up front
  - It forces an explicit discovery step (`search_tool_bm25`) so the model
  narrows tool scope before making MCP calls, which helps relevance and
  lowers prompt/tool clutter.
  
  **What It Changed**
  - Added a new experimental feature flag `search_tool` in
  `core/src/features.rs:90` and `core/src/features.rs:430`.
  - Added config/schema support for that flag in
  `core/config.schema.json:214` and `core/config.schema.json:1235`.
  - Added BM25 dependency (`bm25`) in `Cargo.toml:129` and
  `core/Cargo.toml:23`.
  - Added new tool handler `search_tool_bm25` in
  `core/src/tools/handlers/search_tool_bm25.rs:18`.
  - Registered the handler and tool spec in
  `core/src/tools/handlers/mod.rs:11` and `core/src/tools/spec.rs:780` and
  `core/src/tools/spec.rs:1344`.
  - Extended `ToolsConfig` to carry `search_tool` enablement in
  `core/src/tools/spec.rs:32` and `core/src/tools/spec.rs:56`.
  - Injected dedicated developer instructions for tool-discovery workflow
  in `core/src/codex.rs:483` and `core/src/codex.rs:1976`, using
  `core/templates/search_tool/developer_instructions.md:1`.
  - Added session state to store one-shot selected MCP tools in
  `core/src/state/session.rs:27` and `core/src/state/session.rs:131`.
  - Added filtering so when feature is enabled, only selected MCP tools
  are exposed on the next request (then consumed) in
  `core/src/codex.rs:3800` and `core/src/codex.rs:3843`.
  - Added E2E suite coverage for
  enablement/instructions/hide-until-search/one-turn-selection in
  `core/tests/suite/search_tool.rs:72`,
  `core/tests/suite/search_tool.rs:109`,
  `core/tests/suite/search_tool.rs:147`, and
  `core/tests/suite/search_tool.rs:218`.
  - Refactored test helper utilities to support config-driven tool
  collection in `core/tests/suite/tools.rs:281`.
  
  **Net Behavioral Effect**
  - With `search_tool` **off**: existing MCP behavior (tools exposed
  normally).
  - With `search_tool` **on**: MCP tools start hidden, model must call
  `search_tool_bm25`, and only returned `selected_tools` are available for
  the next model call.

  Anton Panasenko · 2026-02-09 12:53:50 -08:00
  becc3a0424
• feat: do not close unified exec processes across turns (#10799)

  With this PR we do not close the unified exec processes (i.e. background
  terminals) at the end of a turn unless:
  * The user interrupt the turn
  * The user decide to clean the processes through `app-server` or
  `/clean`
  
  I made sure that `codex exec` correctly kill all the processes

  jif-oai · 2026-02-09 10:27:46 +00:00
  6cf61725d0
• Allow user shell commands to run alongside active turns (#10513)

  Summary
  - refactor user shell command execution into a shared helper and add
  modes for standalone vs active-turn execution
  - run user shell commands asynchronously when a turn is already active
  so they don’t replace or abort the current turn
  - extend the tests to cover the new behavior and add the generated Codex
  environment manifest
  
  Testing
  - Not run (not requested)

  jif-oai · 2026-02-05 11:11:00 +00:00
  97582ac52d
• Stop client from being state carrier (#10595)

  I'd like to make client session wide. This requires shedding all random
  state it has to carry.

  pakrym-oai · 2026-02-04 09:05:37 -08:00
  7f20357611
• chore: add phase to message responseitem (#10455)

  ### What
  
  add wiring for `phase` field on `ResponseItem::Message` to lay
  groundwork for differentiating model preambles and final messages.
  currently optional.
  
  follows pattern in #9698.
  
  updated schemas with `just write-app-server-schema` so we can see type
  changes.
  
  ### Tests
  Updated existing tests for SSE parsing and hydrating from history

  sayan-oai · 2026-02-03 02:52:26 +00:00
  fc05374344
• feat: reduce span exposition (#10171)

  This only avoids the creation of duplicates spans

  jif-oai · 2026-01-29 18:15:22 +00:00
  798c4b3260
• feat: adding thread ID to logs + filter in the client (#10150)
  jif-oai · 2026-01-29 16:53:30 +01:00
  89c5f3c4d4
• fix(app-server, core): defer initial context write to rollout file until first turn (#9950)

  ### Overview
  Currently calling `thread/resume` will always bump the thread's
  `updated_at` timestamp. This PR makes it the `updated_at` timestamp
  changes only if a turn is triggered.
  
  ### Additonal context
  What we typically do on resuming a thread is **always** writing “initial
  context” to the rollout file immediately. This initial context includes:
  - Developer instructions derived from sandbox/approval policy + cwd
  - Optional developer instructions (if provided)
  - Optional collaboration-mode instructions
  - Optional user instructions (if provided)
  - Environment context (cwd, shell, etc.)
  
  This PR defers writing the “initial context” to the rollout file until
  the first `turn/start`, so we don't inadvertently bump the thread's
  `updated_at` timestamp until a turn is actually triggered.
  
  This works even though both `thread/resume` and `turn/start` accept
  overrides (such as `model`, `cwd`, etc.) because the initial context is
  seeded from the effective `TurnContext` in memory, computed at
  `turn/start` time, after both sets of overrides have been applied.
  
  **NOTE**: This is a very short-lived solution until we introduce sqlite.
  Then we can remove this.

  Owen Lin · 2026-01-27 10:41:54 -08:00
  fc0fd85349
• chore: update interrupt message (#9925)
  jif-oai · 2026-01-26 19:07:54 +00:00
  09251387e0
• Support end_turn flag (#9698)

  Experimental flag that signals the end of the turn.

  pakrym-oai · 2026-01-22 17:27:48 +00:00
  b511c38ddb
• fix: prevent repeating interrupted turns (#9043)

  ## What
  Record a model-visible `<turn_aborted>` marker in history when a turn is
  interrupted, and treat it as a session prefix.
  
  ## Why
  When a turn is interrupted, Codex emits `TurnAborted` but previously did
  not persist anything model-visible in the conversation history. On the
  next user turn, the model can’t tell the previous work was aborted and
  may resume/repeat earlier actions (including duplicated side effects
  like re-opening PRs).
  
  Fixes: https://github.com/openai/codex/issues/9042
  
  ## How
  On `TurnAbortReason::Interrupted`, append a hidden user message
  containing a `<turn_aborted>…</turn_aborted>` marker and flush.
  Treat `<turn_aborted>` like `<environment_context>` for session-prefix
  filtering.
  Add a regression test to ensure follow-up turns don’t repeat side
  effects from an aborted turn.
  
  ## Testing
  `just fmt`
  `just fix -p codex-core`
  `cargo test -p codex-core -- --test-threads=1`
  `cargo test --all-features -- --test-threads=1`
  
  ---------
  
  Co-authored-by: Skylar Graika <sgraika127@gmail.com>
  Co-authored-by: jif-oai <jif@openai.com>
  Co-authored-by: Eric Traut <etraut@openai.com>

  Skylar Graika · 2026-01-20 13:07:28 -08:00
  b236f1c95d
• feat: timer total turn metrics (#9382)
  jif-oai · 2026-01-19 10:44:31 +00:00
  7ebe13f692
• renaming: task to turn (#8963)
  jif-oai · 2026-01-09 17:31:17 +00:00
  1aed01e99f
• chore: rename unified exec sessions (#8822)

  Renaming done by Codex

  jif-oai · 2026-01-07 16:12:47 +00:00
  4cef89a122
• Rename OpenAI models to models manager (#8346)

  # External (non-OpenAI) Pull Request Requirements
  
  Before opening this Pull Request, please read the dedicated
  "Contributing" markdown file or your PR may be closed:
  https://github.com/openai/codex/blob/main/docs/contributing.md
  
  If your PR conforms to our contribution guidelines, replace this text
  with a detailed and high quality description of your changes.
  
  Include a link to a bug report or enhancement request.

  Ahmed Ibrahim · 2025-12-19 16:20:05 -08:00
  f0dc6fd3c7
• feat: close unified_exec at end of turn (#8052)
  jif-oai · 2025-12-16 12:16:43 +00:00
  ae57e18947
• Migrate model family to models manager (#7565)

  This PR moves `ModelsFamily` to `openai_models`. It also propagates
  `ModelsManager` to session services and use it to drive model family. We
  also make `derive_default_model_family` private because it's a step
  towards what we want: one place that gives model configuration.
  
  This is a second step at having one source of truth for models
  information and config: `ModelsManager`.
  
  Next steps would be to remove `ModelsFamily` from config. That's massive
  because it's being used in 41 occasions mostly pre launching `codex`.
  Also, we need to make `find_family_for_model` private. It's also big
  because it's being used in 21 occasions ~ all tests.

  Ahmed Ibrahim · 2025-12-03 18:49:47 -08:00
  cee37a32b2
• Delegate review to codex instance (#5572)

  In this PR, I am exploring migrating task kind to an invocation of
  Codex. The main reason would be getting rid off multiple
  `ConversationHistory` state and streamlining our context/history
  management.
  
  This approach depends on opening a channel between the sub-codex and
  codex. This channel is responsible for forwarding `interactive`
  (`approvals`) and `non-interactive` events. The `task` is responsible
  for handling those events.
  
  This opens the door for implementing `codex as a tool`, replacing
  `compact` and `review`, and potentially subagents.
  
  One consideration is this code is very similar to `app-server` specially
  in the approval part. If in the future we wanted an interactive
  `sub-codex` we should consider using `codex-mcp`

  Ahmed Ibrahim · 2025-10-29 21:04:25 +00:00
  13e1d0362d
• feature: Add "!cmd" user shell execution (#2471)

  feature: Add "!cmd" user shell execution
  
  This change lets users run local shell commands directly from the TUI by
  prefixing their input with ! (e.g. !ls). Output is truncated to keep the
  exec cell usable, and Ctrl-C cleanly
    interrupts long-running commands (e.g. !sleep 10000).
  
  **Summary of changes**
  
  - Route Op::RunUserShellCommand through a dedicated UserShellCommandTask
  (core/src/tasks/user_shell.rs), keeping the task logic out of codex.rs.
  - Reuse the existing tool router: the task constructs a ToolCall for the
  local_shell tool and relies on ShellHandler, so no manual MCP tool
  lookup is required.
  - Emit exec lifecycle events (ExecCommandBegin/ExecCommandEnd) so the
  TUI can show command metadata, live output, and exit status.
  
  **End-to-end flow**
  
    **TUI handling**
  
  1. ChatWidget::submit_user_message (TUI) intercepts messages starting
  with !.
  2. Non-empty commands dispatch Op::RunUserShellCommand { command };
  empty commands surface a help hint.
  3. No UserInput items are created, so nothing is enqueued for the model.
  
    **Core submission loop**
  4. The submission loop routes the op to handlers::run_user_shell_command
  (core/src/codex.rs).
  5. A fresh TurnContext is created and Session::spawn_user_shell_command
  enqueues UserShellCommandTask.
  
    **Task execution**
  6. UserShellCommandTask::run emits TaskStartedEvent, formats the
  command, and prepares a ToolCall targeting local_shell.
    7. ToolCallRuntime::handle_tool_call dispatches to ShellHandler.
  
    **Shell tool runtime**
  8. ShellHandler::run_exec_like launches the process via the unified exec
  runtime, honoring sandbox and shell policies, and emits
  ExecCommandBegin/End.
  9. Stdout/stderr are captured for the UI, but the task does not turn the
  resulting ToolOutput into a model response.
  
    **Completion**
  10. After ExecCommandEnd, the task finishes without an assistant
  message; the session marks it complete and the exec cell displays the
  final output.
  
    **Conversation context**
  
  - The command and its output never enter the conversation history or the
  model prompt; the flow is local-only.
    - Only exec/task events are emitted for UI rendering.
  
  **Demo video**
  
  
  https://github.com/user-attachments/assets/fcd114b0-4304-4448-a367-a04c43e0b996

  Abhishek Bhardwaj · 2025-10-29 00:31:20 -07:00
  89591e4246
• nit: doc on session task (#5809)
  jif-oai · 2025-10-27 11:43:33 +00:00
  2338294b39
• feat: TUI undo op (#5629)
  jif-oai · 2025-10-27 10:55:29 +00:00
  afc4eaab8b
• feat: async ghost commit (#5618)
  jif-oai · 2025-10-27 10:09:10 +00:00
  e92c4f6561
• Pass TurnContext around instead of sub_id (#5421)

  Today `sub_id` is an ID of a single incoming Codex Op submition. We then
  associate all events triggered by this operation using the same
  `sub_id`.
  
  At the same time we are also creating a TurnContext per submission and
  we'd like to start associating some events (item added/item completed)
  with an entire turn instead of just the operation that started it.
  
  Using turn context when sending events give us flexibility to change
  notification scheme.

  pakrym-oai · 2025-10-21 08:04:16 -07:00
  789e65b9d2
• Add ItemStarted/ItemCompleted events for UserInputItem (#5306)

  Adds a new ItemStarted event and delivers UserMessage as the first item
  type (more to come).
  
  
  Renames `InputItem` to `UserInput` considering we're using the `Item`
  suffix for actual items.

  pakrym-oai · 2025-10-20 13:34:44 -07:00
  9c903c4716
• Support graceful agent interruption (#5287)
  pakrym-oai · 2025-10-17 18:52:57 +00:00
  c03e31ecf5
• ref: state - 2 (#4229)

  Extracting tasks in a module and start abstraction behind a Trait (more
  to come on this but each task will be tackled in a dedicated PR)
  The goal was to drop the ActiveTask and to have a (potentially) set of
  tasks during each turn

  jif-oai · 2025-09-26 13:49:08 +00:00
  1fc3413a46