64 Commits

• remove temporary ownership re-exports (#16626)

  Stacked on #16508.
  
  This removes the temporary `codex-core` / `codex-login` re-export shims
  from the ownership split and rewrites callsites to import directly from
  `codex-model-provider-info`, `codex-models-manager`, `codex-api`,
  `codex-protocol`, `codex-feedback`, and `codex-response-debug-context`.
  
  No behavior change intended; this is the mechanical import cleanup layer
  split out from the ownership move.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-04-03 00:33:34 -07:00
  af8a9d2d2b
• core: cut codex-core compile time 48% with native async SessionTask (#16631)

  ## Why
  
  This continues the compile-time cleanup from #16630. `SessionTask`
  implementations are monomorphized, but `Session` stores the task behind
  a `dyn` boundary so it can drive and abort heterogenous turn tasks
  uniformly. That means we can move the `#[async_trait]` expansion off the
  implementation trait, keep a small boxed adapter only at the storage
  boundary, and preserve the existing task lifecycle semantics while
  reducing the amount of generated async-trait glue in `codex-core`.
  
  One measurement caveat showed up while exploring this: a warm
  incremental benchmark based on `touch core/src/tasks/mod.rs && cargo
  check -p codex-core --lib` was basically flat, but that was the wrong
  benchmark for this change. Using package-clean `codex-core` rebuilds,
  like #16630, shows the real win.
  
  Relevant pre-change code:
  
  - [`SessionTask` with
  `#[async_trait]`](https://github.com/openai/codex/blob/3c7f013f9735e67796c70d95f75f436b7f97e3ec/codex-rs/core/src/tasks/mod.rs#L129-L182)
  - [`RunningTask` storing `Arc<dyn
  SessionTask>`](https://github.com/openai/codex/blob/3c7f013f9735e67796c70d95f75f436b7f97e3ec/codex-rs/core/src/state/turn.rs#L69-L77)
  
  ## What changed
  
  - Switched `SessionTask::{run, abort}` to native RPITIT futures with
  explicit `Send` bounds.
  - Added a private `AnySessionTask` adapter that boxes those futures only
  at the `Arc<dyn ...>` storage boundary.
  - Updated `RunningTask` to store `Arc<dyn AnySessionTask>` and removed
  `#[async_trait]` from the concrete task impls plus test-only
  `SessionTask` impls.
  
  ## Timing
  
  Benchmarked package-clean `codex-core` rebuilds with dependencies left
  warm:
  
  ```shell
  cargo check -p codex-core --lib >/dev/null
  cargo clean -p codex-core >/dev/null
  /usr/bin/time -p cargo +nightly rustc -p codex-core --lib -- \
    -Z time-passes \
    -Z time-passes-format=json >/dev/null
  ```
  
  | revision | rustc `total` | process `real` | `generate_crate_metadata`
  | `MIR_borrow_checking` | `monomorphization_collector_graph_walk` |
  | --- | ---: | ---: | ---: | ---: | ---: |
  | parent `3c7f013f9735` | 67.21s | 67.71s | 24.61s | 23.43s | 22.43s |
  | this PR `2cafd783ac22` | 35.08s | 35.60s | 8.01s | 7.25s | 7.15s |
  | delta | -47.8% | -47.4% | -67.5% | -69.1% | -68.1% |
  
  For completeness, the warm touched-file benchmark stayed flat (`1.96s`
  parent vs `1.97s` this PR), which is why that benchmark should not be
  used to evaluate this refactor.
  
  ## Verification
  
  - Ran `cargo test -p codex-core`; this change compiled and task-related
  tests passed before hitting the same unrelated 5
  `config::tests::*guardian*` failures already present on the parent
  stack.

  Michael Bolin · 2026-04-02 23:39:56 +00:00
  7a3eec6fdb
• core: remove cross-crate re-exports from lib.rs (#16512)

  ## Why
  
  `codex-core` was re-exporting APIs owned by sibling `codex-*` crates,
  which made downstream crates depend on `codex-core` as a proxy module
  instead of the actual owner crate.
  
  Removing those forwards makes crate boundaries explicit and lets leaf
  crates drop unnecessary `codex-core` dependencies. In this PR, this
  reduces the dependency on `codex-core` to `codex-login` in the following
  files:
  
  ```
  codex-rs/backend-client/Cargo.toml
  codex-rs/mcp-server/tests/common/Cargo.toml
  ```
  
  ## What
  
  - Remove `codex-rs/core/src/lib.rs` re-exports for symbols owned by
  `codex-login`, `codex-mcp`, `codex-rollout`, `codex-analytics`,
  `codex-protocol`, `codex-shell-command`, `codex-sandboxing`,
  `codex-tools`, and `codex-utils-path`.
  - Delete the `default_client` forwarding shim in `codex-rs/core`.
  - Update in-crate and downstream callsites to import directly from the
  owning `codex-*` crate.
  - Add direct Cargo dependencies where callsites now target the owner
  crate, and remove `codex-core` from `codex-rs/backend-client`.

  Michael Bolin · 2026-04-01 23:06:24 -07:00
  aa2403e2eb
• nit: update aborted line (#16318)
  jif-oai · 2026-03-31 13:06:00 +02:00
  2b8d29ac0d
• fix: ma2 (#16238)
  jif-oai · 2026-03-31 11:22:38 +02:00
  25fbd7e40e
• feat: add mailbox concept for wait (#16010)

  Add a mailbox we can use for inter-agent communication
  `wait` is now based on it and don't take target anymore

  jif-oai · 2026-03-30 11:47:20 +02:00
  213756c9ab
• feat: disable notifier v2 and start turn on agent interaction (#15624)

  Make the inter-agent communication start a turn
  
  As part of this, we disable the v2 notifier to prevent some odd
  behaviour where the agent restart working while you're talking to it for
  example

  jif-oai · 2026-03-24 17:01:24 +00:00
  b51d5f18c7
• Add fork snapshot modes (#15239)

  ## Summary
  - add `ForkSnapshotMode` to `ThreadManager::fork_thread` so callers can
  request either a committed snapshot or an interrupted snapshot
  - share the model-visible `<turn_aborted>` history marker between the
  live interrupt path and interrupted forks
  - update the small set of direct fork callsites to pass
  `ForkSnapshotMode::Committed`
  
  Note: this enables /btw to work similarly as Esc to interrupt (hopefully
  somewhat in distribution)
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Charley Cunningham · 2026-03-23 19:05:42 -07:00
  f547b79bd0
• feat: new op type for sub-agents communication (#15556)

  Add `InterAgentCommunication` for v2 agent communication

  jif-oai · 2026-03-23 21:09:00 +00:00
  18f1a08bc9
• feat: structured multi-agent output (#15515)

  Send input now sends messages as assistant message and with this format:
  
  ```
  author: /root/worker_a
  recipient: /root/worker_a/tester
  other_recipients: []
  Content: bla bla bla. Actual content. Only text for now
  ```

  jif-oai · 2026-03-23 18:53:54 +00:00
  37ac0c093c
• Split features into codex-features crate (#15253)

  - Split the feature system into a new `codex-features` crate.
  - Cut `codex-core` and workspace consumers over to the new config and
  warning APIs.
  
  Co-authored-by: Ahmed Ibrahim <219906144+aibrahim-oai@users.noreply.github.com>
  Co-authored-by: Codex <noreply@openai.com>

  Ahmed Ibrahim · 2026-03-19 20:12:07 -07:00
  2e22885e79
• Revert "Forward session and turn headers to MCP HTTP requests" (#15185)

  Reverts openai/codex#15011
  
  Codex merged by mistake before feedback applied

  nicholasclark-openai · 2026-03-19 10:38:53 -07:00
  5ec121ba12
• Forward session and turn headers to MCP HTTP requests (#15011)

  ## Summary
  - forward request-scoped task headers through MCP tool metadata lookups
  and tool calls
  - apply those headers to streamable HTTP initialize, tools/list, and
  tools/call requests
  - update affected rmcp/core tests for the new request_headers plumbing
  
  ## Testing
  - cargo test -p codex-rmcp-client
  - cargo test -p codex-core (fails on pre-existing unrelated error in
  core/src/auth_env_telemetry.rs: missing websocket_connect_timeout_ms in
  ModelProviderInfo initializer)
  - just fix -p codex-rmcp-client
  - just fix -p codex-core (hits the same unrelated auth_env_telemetry.rs
  error)
  - just fmt
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  nicholasclark-openai · 2026-03-18 21:29:37 -07:00
  b14689df3b
• [hooks] userpromptsubmit - hook before user's prompt is executed (#14626)

  - this allows blocking the user's prompts from executing, and also
  prevents them from entering history
  - handles the edge case where you can both prevent the user's prompt AND
  add n amount of additionalContexts
  - refactors some old code into common.rs where hooks overlap
  functionality
  - refactors additionalContext being previously added to user messages,
  instead we use developer messages for them
  - handles queued messages correctly
  
  Sample hook for testing - if you write "[block-user-submit]" this hook
  will stop the thread:
  
  example run
  ```
  › sup
  
  
  • Running UserPromptSubmit hook: reading the observatory notes
  
  UserPromptSubmit hook (completed)
    warning: wizard-tower UserPromptSubmit demo inspected: sup
    hook context: Wizard Tower UserPromptSubmit demo fired. For this reply only, include the exact
  phrase 'observatory lanterns lit' exactly once near the end.
  
  • Just riding the cosmic wave and ready to help, my friend. What are we building today? observatory
    lanterns lit
  
  
  › and [block-user-submit]
  
  
  • Running UserPromptSubmit hook: reading the observatory notes
  
  UserPromptSubmit hook (stopped)
    warning: wizard-tower UserPromptSubmit demo blocked the prompt on purpose.
    stop: Wizard Tower demo block: remove [block-user-submit] to continue.
  ```
  
  .codex/config.toml
  ```
  [features]
  codex_hooks = true
  ```
  
  .codex/hooks.json
  ```
  {
    "hooks": {
      "UserPromptSubmit": [
        {
          "hooks": [
            {
              "type": "command",
              "command": "/usr/bin/python3 .codex/hooks/user_prompt_submit_demo.py",
              "timeoutSec": 10,
              "statusMessage": "reading the observatory notes"
            }
          ]
        }
      ]
    }
  }
  ```
  
  .codex/hooks/user_prompt_submit_demo.py
  ```
  #!/usr/bin/env python3
  
  import json
  import sys
  from pathlib import Path
  
  
  def prompt_from_payload(payload: dict) -> str:
      prompt = payload.get("prompt")
      if isinstance(prompt, str) and prompt.strip():
          return prompt.strip()
  
      event = payload.get("event")
      if isinstance(event, dict):
          user_prompt = event.get("user_prompt")
          if isinstance(user_prompt, str):
              return user_prompt.strip()
  
      return ""
  
  
  def main() -> int:
      payload = json.load(sys.stdin)
      prompt = prompt_from_payload(payload)
      cwd = Path(payload.get("cwd", ".")).name or "wizard-tower"
  
      if "[block-user-submit]" in prompt:
          print(
              json.dumps(
                  {
                      "systemMessage": (
                          f"{cwd} UserPromptSubmit demo blocked the prompt on purpose."
                      ),
                      "decision": "block",
                      "reason": (
                          "Wizard Tower demo block: remove [block-user-submit] to continue."
                      ),
                  }
              )
          )
          return 0
  
      prompt_preview = prompt or "(empty prompt)"
      if len(prompt_preview) > 80:
          prompt_preview = f"{prompt_preview[:77]}..."
  
      print(
          json.dumps(
              {
                  "systemMessage": (
                      f"{cwd} UserPromptSubmit demo inspected: {prompt_preview}"
                  ),
                  "hookSpecificOutput": {
                      "hookEventName": "UserPromptSubmit",
                      "additionalContext": (
                          "Wizard Tower UserPromptSubmit demo fired. "
                          "For this reply only, include the exact phrase "
                          "'observatory lanterns lit' exactly once near the end."
                      ),
                  },
              }
          )
      )
      return 0
  
  
  if __name__ == "__main__":
      raise SystemExit(main())
  ```

  Andrei Eternal · 2026-03-17 22:09:22 -07:00
  6fef421654
• fix(core): prevent hanging turn/start due to websocket warming issues (#14838)

  ## Description
  
  This PR fixes a bad first-turn failure mode in app-server when the
  startup websocket prewarm hangs. Before this change, `initialize ->
  thread/start -> turn/start` could sit behind the prewarm for up to five
  minutes, so the client would not see `turn/started`, and even
  `turn/interrupt` would block because the turn had not actually started
  yet.
  
  Now, we:
  - set a (configurable) timeout of 15s for websocket startup time,
  exposed as `websocket_startup_timeout_ms` in config.toml
  - `turn/started` is sent immediately on `turn/start` even if the
  websocket is still connecting
  - `turn/interrupt` can be used to cancel a turn that is still waiting on
  the websocket warmup
  - the turn task will wait for the full 15s websocket warming timeout
  before falling back
  
  ## Why
  
  The old behavior made app-server feel stuck at exactly the moment the
  client expects turn lifecycle events to start flowing. That was
  especially painful for external clients, because from their point of
  view the server had accepted the request but then went silent for
  minutes.
  
  ## Configuring the websocket startup timeout
  Can set it in config.toml like this:
  ```
  [model_providers.openai]
  supports_websockets = true
  websocket_connect_timeout_ms = 15000
  ```

  Owen Lin · 2026-03-17 10:07:46 -07:00
  6ea041032b
• Apply argument comment lint across codex-rs (#14652)

  ## Why
  
  Once the repo-local lint exists, `codex-rs` needs to follow the
  checked-in convention and CI needs to keep it from drifting. This commit
  applies the fallback `/*param*/` style consistently across existing
  positional literal call sites without changing those APIs.
  
  The longer-term preference is still to avoid APIs that require comments
  by choosing clearer parameter types and call shapes. This PR is
  intentionally the mechanical follow-through for the places where the
  existing signatures stay in place.
  
  After rebasing onto newer `main`, the rollout also had to cover newly
  introduced `tui_app_server` call sites. That made it clear the first cut
  of the CI job was too expensive for the common path: it was spending
  almost as much time installing `cargo-dylint` and re-testing the lint
  crate as a representative test job spends running product tests. The CI
  update keeps the full workspace enforcement but trims that extra
  overhead from ordinary `codex-rs` PRs.
  
  ## What changed
  
  - keep a dedicated `argument_comment_lint` job in `rust-ci`
  - mechanically annotate remaining opaque positional literals across
  `codex-rs` with exact `/*param*/` comments, including the rebased
  `tui_app_server` call sites that now fall under the lint
  - keep the checked-in style aligned with the lint policy by using
  `/*param*/` and leaving string and char literals uncommented
  - cache `cargo-dylint`, `dylint-link`, and the relevant Cargo
  registry/git metadata in the lint job
  - split changed-path detection so the lint crate's own `cargo test` step
  runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes
  - continue to run the repo wrapper over the `codex-rs` workspace, so
  product-code enforcement is unchanged
  
  Most of the code changes in this commit are intentionally mechanical
  comment rewrites or insertions driven by the lint itself.
  
  ## Verification
  
  - `./tools/argument-comment-lint/run.sh --workspace`
  - `cargo test -p codex-tui-app-server -p codex-tui`
  - parsed `.github/workflows/rust-ci.yml` locally with PyYAML
  
  ---
  
  * -> #14652
  * #14651

  Michael Bolin · 2026-03-16 16:48:15 -07:00
  b77fe8fefe
• Preserve background terminals on interrupt and rename cleanup command to /stop (#14602)

  ### Motivation
  - Interrupting a running turn (Ctrl+C / Esc) currently also terminates
  long‑running background shells, which is surprising for workflows like
  local dev servers or file watchers.
  - The existing cleanup command name was confusing; callers expect an
  explicit command to stop background terminals rather than a UI clear
  action.
  - Make background‑shell termination explicit and surface a clearer
  command name while preserving backward compatibility.
  
  ### Description
  - Renamed the background‑terminal cleanup slash command from `Clean`
  (`/clean`) to `Stop` (`/stop`) and kept `clean` as an alias in the
  command parsing/visibility layer, updated the user descriptions and
  command popup wiring accordingly.
  - Updated the unified‑exec footer text and snapshots to point to `/stop`
  (and trimmed corresponding snapshot output to match the new label).
  - Changed interrupt behavior so `Op::Interrupt` (Ctrl+C / Esc interrupt)
  no longer closes or clears tracked unified exec / background terminal
  processes in the TUI or core cleanup path; background shells are now
  preserved after an interrupt.
  - Updated protocol/docs to clarify that `turn/interrupt` (or
  `Op::Interrupt`) interrupts the active turn but does not terminate
  background terminals, and that `thread/backgroundTerminals/clean` is the
  explicit API to stop those shells.
  - Updated unit/integration tests and insta snapshots in the TUI and core
  unified‑exec suites to reflect the new semantics and command name.
  
  ### Testing
  - Ran formatting with `just fmt` in `codex-rs` (succeeded). 
  - Ran `cargo test -p codex-protocol` (succeeded). 
  - Attempted `cargo test -p codex-tui` but the build could not complete
  in this environment due to a native build dependency that requires
  `libcap` development headers (the `codex-linux-sandbox` vendored build
  step); install `libcap-dev` / make `libcap.pc` available in
  `PKG_CONFIG_PATH` to run the TUI test suite locally.
  - Updated and accepted the affected `insta` snapshots for the TUI
  changes so visual diffs reflect the new `/stop` wording and preserved
  interrupt behavior.
  
  ------
  [Codex
  Task](https://chatgpt.com/codex/tasks/task_i_69b39c44b6dc8323bd133ae206310fae)

  friel-openai · 2026-03-15 22:17:25 -07:00
  ba463a9dc7
• [js_repl] Hard-stop active js_repl execs on explicit user interrupts (#13329)

  ## Summary
  - hard-stop `js_repl` only for `TurnAbortReason::Interrupted`,
  preserving the persistent REPL across replaced turns
  - track the current top-level exec by turn and only reset when the
  interrupted turn owns submitted work or a freshly started kernel for the
  current exec attempt
  - close both interrupt races: the write-window race by marking the exec
  as submitted before async pipe writes begin, and the startup-window race
  by tracking fresh-kernel ownership until submission
  - add regression coverage for interrupted in-flight execs and the
  pending-kernel-start window
  
  ## Why
  Stopping a turn previously surfaced `aborted by user after Xs` even
  though the underlying `js_repl` kernel could continue executing. Earlier
  fixes also risked resetting the session-scoped REPL too broadly or
  missing already-dispatched work. This change keeps cleanup scoped to
  explicit stop semantics and makes the interrupt path line up with both
  submitted execs and newly started kernels.
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-core`
  - `just fix -p codex-core`
  
  `cargo test -p codex-core` passes the updated `js_repl` coverage,
  including the new startup-window regression test, but still has
  unrelated integration failures in this environment outside `js_repl`.
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  aaronl-openai · 2026-03-12 17:51:56 -07:00
  d9a403a8c0
• fix: move inline codex-rs/core unit tests into sibling files (#14444)

  ## Why
  PR #13783 moved the `codex.rs` unit tests into `codex_tests.rs`. This
  applies the same extraction pattern across the rest of `codex-rs/core`
  so the production modules stay focused on runtime code instead of large
  inline test blocks.
  
  Keeping the tests in sibling files also makes follow-up edits easier to
  review because product changes no longer have to share a file with
  hundreds or thousands of lines of test scaffolding.
  
  ## What changed
  - replaced each inline `mod tests { ... }` in `codex-rs/core/src/**`
  with a path-based module declaration
  - moved each extracted unit test module into a sibling `*_tests.rs`
  file, using `mod_tests.rs` for `mod.rs` modules
  - preserved the existing `cfg(...)` guards and module-local structure so
  the refactor remains structural rather than behavioral
  
  ## Testing
  - `cargo test -p codex-core --lib` (`1653 passed; 0 failed; 5 ignored`)
  - `just fix -p codex-core`
  - `cargo fmt --check`
  - `cargo shear`

  Michael Bolin · 2026-03-12 08:16:36 -07:00
  0c8a36676a
• feat(core): emit turn metric for network proxy state (#14250)

  ## Summary
  - add a per-turn `codex.turn.network_proxy` metric constant
  - emit the metric from turn completion using the live managed proxy
  enabled state
  - add focused tests for active and inactive tag emission

  viyatb-oai · 2026-03-11 12:33:10 -07:00
  52a3bde6cc
• feat(otel): Centralize OTEL metric names and shared tag builders (#14117)

  This cleans up a bunch of metric plumbing that had started to drift.
  
  The main change is making `codex-otel` the canonical home for shared
  metric definitions and metric tag helpers. I moved the `turn/thread`
  metric names that were still duplicated into the OTEL metric registry,
  added a shared `metrics::tags` module for common tag keys and session
  tag construction, and updated `SessionTelemetry` to build its metadata
  tags through that shared path.
  
  On the codex-core side, TTFT/TTFM now use the shared metric-name
  constants instead of local string definitions. I also switched the
  obvious remaining turn/thread metric callsites over to the shared
  constants, and added a small helper so TTFT/TTFM can attach an optional
  sanitized client.name tag from TurnContext.
  
  This should make follow-on telemetry work less ad hoc:
  - one canonical place for metric names
  - one canonical place for common metric tag keys/builders
  - less duplication between `codex-core` and `codex-otel`

  Owen Lin · 2026-03-09 12:46:42 -07:00
  da991bdf3a
• Stabilize interrupted task approval cleanup (#14102)

  ## Summary
  - drain the active turn tasks before clearing pending approvals during
  interruption
  - keep the turn in hand long enough for interrupted tasks to observe
  cancellation first
  
  ## Why this fixes the flake
  Interrupted turns could clear pending approvals too early, which let an
  in-flight approval wait surface as a model-visible rejection before the
  turn emitted `TurnAborted`. Reordering the cleanup removes that race
  without changing the steady-state task model.

  Ahmed Ibrahim · 2026-03-09 11:22:51 -07:00
  ad57505ef5
• chore(otel): rename OtelManager to SessionTelemetry (#13808)

  ## Summary
  This is a purely mechanical refactor of `OtelManager` ->
  `SessionTelemetry` to better convey what the struct is doing. No
  behavior change.
  
  ## Why
  
  `OtelManager` ended up sounding much broader than what this type
  actually does. It doesn't manage OTEL globally; it's the session-scoped
  telemetry surface for emitting log/trace events and recording metrics
  with consistent session metadata (`app_version`, `model`, `slug`,
  `originator`, etc.).
  
  `SessionTelemetry` is a more accurate name, and updating the call sites
  makes that boundary a lot easier to follow.
  
  ## Validation
  
  - `just fmt`
  - `cargo test -p codex-otel`
  - `cargo test -p codex-core`

  Owen Lin · 2026-03-06 16:23:30 -08:00
  289ed549cf
• feat(otel, core): record turn TTFT and TTFM metrics in codex-core (#13630)

  ### Summary
  This adds turn-level latency metrics for the first model output and the
  first completed agent message.
  - `codex.turn.ttft.duration_ms` starts at turn start and records on the
  first output signal we see from the model. That includes normal
  assistant text, reasoning deltas, and non-text outputs like tool-call
  items.
  - `codex.turn.ttfm.duration_ms` also starts at turn start, but it
  records when the first agent message finishes streaming rather than when
  its first delta arrives.
  
  ### Implementation notes
  The timing is tracked in codex-core, not app-server, so the definition
  stays consistent across CLI, TUI, and app-server clients.
  
  I reused the existing turn lifecycle boundary that already drives
  `codex.turn.e2e_duration_ms`, stored the turn start timestamp in turn
  state, and record each metric once per turn.
  
  I also wired the new metric names into the OTEL runtime metrics summary
  so they show up in the same in-memory/debug snapshot path as the
  existing timing metrics.

  Owen Lin · 2026-03-06 10:23:48 -08:00
  3449e00bc9
• feat(core, tracing): add a span representing a turn (#13424)

  This is PR 3 of the app-server tracing rollout.
  
  PRs https://github.com/openai/codex/pull/13285 and
  https://github.com/openai/codex/pull/13368 gave us inbound request spans
  in app-server and propagated trace context through Submission. This
  change finishes the next piece in core: when a request actually starts a
  turn, we now create a core-owned long-lived span that stays open for the
  real lifetime of the turn.
  
  What changed:
  - `Session::spawn_task` can now optionally create a long-lived turn span
  and run the spawned task inside it
  - `turn/start` uses that path, so normal turn execution stays under a
  single core-owned span after the async handoff
  - `review/start` uses the same pattern
  - added a unit test that verifies the spawned turn task inherits the
  submission dispatch trace ancestry
  
  **Why**
  The app-server request span is intentionally short-lived. Once work
  crosses into core, we still want one span that covers the actual
  execution window until completion or interruption. This keeps that
  ownership where it belongs: in the layer that owns the runtime
  lifecycle.

  Owen Lin · 2026-03-04 11:09:17 -08:00
  27724f6ead
• feat: add metric for per-turn tool count and add tmp_mem flag (#13456)
  jif-oai · 2026-03-04 11:25:58 +00:00
  e07eaff0d3
• add metric for per-turn token usage (#13454)
  jif-oai · 2026-03-04 10:17:25 +00:00
  49634b7f9c
• tui: align pending steers with core acceptance (#12868)

  ## Summary
  - submit `Enter` steers immediately while a turn is already running
  instead of routing them through `queued_user_messages`
  - keep those submitted steers visible in the footer as `pending_steers`
  until core records them as a user message or aborts the turn
  - reconcile pending steers on `ItemCompleted(UserMessage)`, not
  `RawResponseItem`
  - emit user-message item lifecycle for leftover pending input at task
  finish, then remove the TUI `TurnComplete` fallback
  - keep `queued_user_messages` for actual queued drafts, rendered below
  pending steers
  
  ## Problem
  While the assistant was generating, pressing `Enter` could send the
  input into `queued_user_messages`. That queue only drains after the turn
  ends, so ordinary steers behaved like queued drafts instead of landing
  at the next core sampling boundary.
  
  The first version of this fix also used `RawResponseItem` to decide when
  a steer had landed. Review feedback was that this is the wrong
  abstraction for client behavior.
  
  There was also a late edge case in core: if pending steer input was
  accepted after the final sampling decision but before `TurnComplete`,
  core would record that user message into history at task finish without
  emitting `ItemStarted(UserMessage)` / `ItemCompleted(UserMessage)`. TUI
  had a fallback to paper over that gap locally.
  
  ## Approach
  - `Enter` during an active turn now submits a normal `Op::UserTurn`
  immediately
  - TUI keeps a local pending-steer preview instead of rendering that user
  message into history immediately
  - when core records the steer as `ItemCompleted(UserMessage)`, TUI
  matches and removes the corresponding pending preview, then renders the
  committed user message
  - core now emits the same user-message lifecycle when
  `on_task_finished(...)` drains leftover pending user input, before
  `TurnComplete`
  - with that lifecycle gap closed in core, TUI no longer needs to flush
  pending steers into history on `TurnComplete`
  - if the turn is interrupted, pending steers and queued drafts are both
  restored into the composer, with pending steers first
  
  ## Notes
  - `Tab` still uses the real queued-message path
  - `queued_user_messages` and `pending_steers` are separate state with
  separate semantics
  - the pending-steer matching key is built directly from `UserInput`
  - this removes the new TUI dependency on `RawResponseItem`
  
  ## Validation
  - `just fmt`
  - `cargo test -p codex-core
  task_finish_emits_turn_item_lifecycle_for_leftover_pending_user_input --
  --nocapture`
  - `cargo test -p codex-tui`

  Charley Cunningham · 2026-03-03 15:31:52 -08:00
  299b8ac445
• core: bundle settings diff updates into one dev/user envelope (#12417)

  ## Summary
  - bundle contextual prompt injection into at most one developer message
  plus one contextual user message in both:
    - per-turn settings updates
    - initial context insertion
  - preserve `<model_switch>` across compaction by rebuilding it through
  canonical initial-context injection, instead of relying on
  strip/reattach hacks
  - centralize contextual user fragment detection in one shared definition
  table and reuse it for parsing/compaction logic
  - keep `AGENTS.md` in its natural serialized format:
    - `# AGENTS.md instructions for {dirname}`
    - `<INSTRUCTIONS>...</INSTRUCTIONS>`
  - simplify related tests/helpers and accept the expected snapshot/layout
  updates from bundled multi-part messages
  
  ## Why
  The goal is to converge toward a simpler, more intentional prompt shape
  where contextual updates are consistently represented as one developer
  envelope plus one contextual user envelope, while keeping parsing and
  compaction behavior aligned with that representation.
  
  ## Notable details
  - the temporary `SettingsUpdateEnvelope` wrapper was removed; these
  paths now return `Vec<ResponseItem>` directly
  - local/remote compaction no longer rely on model-switch strip/restore
  helpers
  - contextual user detection is now driven by shared fragment definitions
  instead of ad hoc matcher assembly
  - AGENTS/user instructions are still the same logical context; only the
  synthetic `<user_instructions>` wrapper was replaced by the natural
  AGENTS text format
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-app-server
  codex_message_processor::tests::extract_conversation_summary_prefers_plain_user_messages
  -- --exact`
  - `cargo test -p codex-core
  compact::tests::collect_user_messages_filters_session_prefix_entries
  --lib -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact::snapshot_request_shape_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact_remote::snapshot_request_shape_remote_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_apps_guidance_as_developer_message_when_enabled'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_developer_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_user_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::resume_includes_initial_messages_and_sends_prior_items'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::review::review_input_isolated_from_parent_history' -- --exact`
  - `cargo test -p codex-exec --test all
  'suite::resume::exec_resume_last_respects_cwd_filter_and_all_flag' --
  --exact`
  - `cargo test -p core_test_support
  context_snapshot::tests::full_text_mode_preserves_unredacted_text --
  --exact`
  
  ## Notes
  - I also ran several targeted `compact`, `compact_remote`,
  `prompt_caching`, `model_visible_layout`, and `event_mapping` tests
  while iterating on prompt-shape changes.
  - I have not claimed a clean full-workspace `cargo test` from this
  environment because local sandbox/resource conditions have previously
  produced unrelated failures in large workspace runs.

  Charley Cunningham · 2026-02-26 00:12:08 -08:00
  07aefffb1f
• Fix compaction context reinjection and model baselines (#12252)

  ## Summary
  - move regular-turn context diff/full-context persistence into
  `run_turn` so pre-turn compaction runs before incoming context updates
  are recorded
  - after successful pre-turn compaction, rely on a cleared
  `reference_context_item` to trigger full context reinjection on the
  follow-up regular turn (manual `/compact` keeps replacement history
  summary-only and also clears the baseline)
  - preserve `<model_switch>` when full context is reinjected, and inject
  it *before* the rest of the full-context items
  - scope `reference_context_item` and `previous_model` to regular user
  turns only so standalone tasks (`/compact`, shell, review, undo) cannot
  suppress future reinjection or `<model_switch>` behavior
  - make context-diff persistence + `reference_context_item` updates
  explicit in the regular-turn path, with clearer docs/comments around the
  invariant
  - stop persisting local `/compact` `RolloutItem::TurnContext` snapshots
  (only regular turns persist `TurnContextItem` now)
  - simplify resume/fork previous-model/reference-baseline hydration by
  looking up the last surviving turn context from rollout lifecycle
  events, including rollback and compaction-crossing handling
  - remove the legacy fallback that guessed from bare `TurnContext`
  rollouts without lifecycle events
  - update compaction/remote-compaction/model-visible snapshots and
  compact test assertions (including remote compaction mock response
  shape)
  
  ## Why
  We were persisting incoming context items before spawning the regular
  turn task, which let pre-turn compaction requests accidentally include
  incoming context diffs without the new user message. Fixing that exposed
  follow-on baseline issues around `/compact`, resume/fork, and standalone
  tasks that could cause duplicate context injection or suppress
  `<model_switch>` instructions.
  
  This PR re-centers the invariants around regular turns:
  - regular turns persist model-visible context diffs/full reinjection and
  update the `reference_context_item`
  - standalone tasks do not advance those regular-turn baselines
  - compaction clears the baseline when replacement history may have
  stripped the referenced context diffs
  
  ## Follow-ups (TODOs left in code)
  - `TODO(ccunningham)`: fix rollback/backtracking baseline handling more
  comprehensively
  - `TODO(ccunningham)`: include pending incoming context items in
  pre-turn compaction threshold estimation
  - `TODO(ccunningham)`: inject updated personality spec alongside
  `<model_switch>` so some model-switch paths can avoid forced full
  reinjection
  - `TODO(ccunningham)`: review task turn lifecycle
  (`TurnStarted`/`TurnComplete`) behavior and emit task-start context
  diffs for task types that should have them (excluding `/compact`)
  
  ## Validation
  - `just fmt`
  - CI should cover the updated compaction/resume/model-visible snapshot
  expectations and rollout-hydration behavior
  - I did **not** rerun the full local test suite after the latest
  resume-lookup / rollout-persistence simplifications

  Charley Cunningham · 2026-02-20 23:13:08 -08:00
  bb0ac5be70
• feat: persist and restore codex app's tools after search (#11780)

  ### What changed
  1. Removed per-turn MCP selection reset in `core/src/tasks/mod.rs`.
  2. Added `SessionState::set_mcp_tool_selection(Vec<String>)` in
  `core/src/state/session.rs` for authoritative restore behavior (deduped,
  order-preserving, empty clears).
  3. Added rollout parsing in `core/src/codex.rs` to recover
  `active_selected_tools` from prior `search_tool_bm25` outputs:
     - tracks matching `call_id`s
     - parses function output text JSON
     - extracts `active_selected_tools`
     - latest valid payload wins
     - malformed/non-matching payloads are ignored
  4. Applied restore logic to resumed and forked startup paths in
  `core/src/codex.rs`.
  5. Updated instruction text to session/thread scope in
  `core/templates/search_tool/tool_description.md`.
  6. Expanded tests in `core/tests/suite/search_tool.rs`, plus unit
  coverage in:
     - `core/src/codex.rs`
     - `core/src/state/session.rs`
  
  ### Behavior after change
  1. Search activates matched tools.
  2. Additional searches union into active selection.
  3. Selection survives new turns in the same thread.
  4. Resume/fork restores selection from rollout history.
  5. Separate threads do not inherit selection unless forked.

  Anton Panasenko · 2026-02-15 19:18:41 -08:00
  02abd9a8ea
• turn metadata followups (#11782)

  some trivial simplifications from #11677

  pash-openai · 2026-02-13 14:59:16 -08:00
  a5e8e69d18
• turn metadata: per-turn non-blocking (#11677)
  pash-openai · 2026-02-13 12:48:29 -08:00
  6c0a924203
• Parse first order skill/connector mentions (#11547)

  This PR introduces a skill-expansion mechanism for mentions so nested or
  skill or connection mentions are expanded if present in skills invoked
  by the user. This keeps behavior aligned with existing mention handling
  while extending coverage to deeper scenarios. With these changes, users
  can create skills that invoke connectors, and skills that invoke other
  skills.
  
  Replaces #10863, which is not needed with the addition of
  [search_tool_bm25](https://github.com/openai/codex/issues/10657)

  canvrno-oai · 2026-02-12 10:55:22 -08:00
  22fa283511
• Fix flaky pre_sampling_compact switch test (#11573)

  Summary
  - address the nondeterministic behavior observed in
  `pre_sampling_compact_runs_on_switch_to_smaller_context_model` so it no
  longer fails intermittently during model switches
  - ensure the surrounding sampling logic consistently handles the
  smaller-context case that the test exercises
  
  Testing
  - Not run (not requested)

  jif-oai · 2026-02-12 11:40:48 +00:00
  3cd93c00ac
• Pre-sampling compact with previous model context (#11504)

  - Run pre-sampling compact through a single helper that builds
  previous-model turn context and compacts before the follow-up request
  when switching to a smaller context window.
  - Keep compaction events on the parent turn id and add compact suite
  coverage for switch-in-session and resume+switch flows.

  Ahmed Ibrahim · 2026-02-11 17:24:06 -08:00
  6938150c5e
• Hydrate previous model across resume/fork/rollback/task start (#11497)

  - Replace pending resume model state with persistent previous_model and
  hydrate it on resume, fork, rollback, and task end in spawn_task

  Ahmed Ibrahim · 2026-02-11 16:45:18 -08:00
  bb5dfd037a
• chore: persist turn_id in rollout session and make turn_id uuid based (#11246)

  Problem:
  1. turn id is constructed in-memory;
  2. on resuming threads, turn_id might not be unique;
  3. client cannot no the boundary of a turn from rollout files easily.
  
  This PR does three things:
  1. persist `task_started` and `task_complete` events;
  1. persist `turn_id` in rollout turn events;
  5. generate turn_id as unique uuids instead of incrementing it in
  memory.
  
  This helps us resolve the issue of clients wanting to have unique turn
  ids for resuming a thread, and knowing the boundry of each turn in
  rollout files.
  
  example debug logs
  ```
  2026-02-11T00:32:10.746876Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=8 turn=Turn { id: "019c4a07-d809-74c3-bc4b-fd9618487b4b", items: [UserMessage { id: "item-24", content: [Text { text: "hi", text_elements: [] }] }, AgentMessage { id: "item-25", text: "Hi. I’m in the workspace with your current changes loaded and ready. Send the next task and I’ll execute it end-to-end." }], status: Completed, error: None }
  2026-02-11T00:32:10.746888Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=9 turn=Turn { id: "019c4a18-1004-76c0-a0fb-a77610f6a9b8", items: [UserMessage { id: "item-26", content: [Text { text: "hello", text_elements: [] }] }, AgentMessage { id: "item-27", text: "Hello. Ready for the next change in `codex-rs`; I can continue from the current in-progress diff or start a new task." }], status: Completed, error: None }
  2026-02-11T00:32:10.746899Z DEBUG codex_app_server_protocol::protocol::thread_history: built turn from rollout items turn_index=10 turn=Turn { id: "019c4a19-41f0-7db0-ad78-74f1503baeb8", items: [UserMessage { id: "item-28", content: [Text { text: "hello", text_elements: [] }] }, AgentMessage { id: "item-29", text: "Hello. Send the specific change you want in `codex-rs`, and I’ll implement it and run the required checks." }], status: Completed, error: None }
  ```
  
  backward compatibility:
  if you try to resume an old session without task_started and
  task_complete event populated, the following happens:
  - If you resume and do nothing: those reconstructed historical IDs can
  differ next time you resume.
  - If you resume and send a new turn: the new turn gets a fresh UUID from
  live submission flow and is persisted, so that new turn’s ID is stable
  on later resumes.
  I think this behavior is fine, because we only care about deterministic
  turn id once a turn is triggered.

  Celia Chen · 2026-02-11 03:56:01 +00:00
  641d5268fa
• feat: search_tool (#10657)

  **Why We Did This**
  - The goal is to reduce MCP tool context pollution by not exposing the
  full MCP tool list up front
  - It forces an explicit discovery step (`search_tool_bm25`) so the model
  narrows tool scope before making MCP calls, which helps relevance and
  lowers prompt/tool clutter.
  
  **What It Changed**
  - Added a new experimental feature flag `search_tool` in
  `core/src/features.rs:90` and `core/src/features.rs:430`.
  - Added config/schema support for that flag in
  `core/config.schema.json:214` and `core/config.schema.json:1235`.
  - Added BM25 dependency (`bm25`) in `Cargo.toml:129` and
  `core/Cargo.toml:23`.
  - Added new tool handler `search_tool_bm25` in
  `core/src/tools/handlers/search_tool_bm25.rs:18`.
  - Registered the handler and tool spec in
  `core/src/tools/handlers/mod.rs:11` and `core/src/tools/spec.rs:780` and
  `core/src/tools/spec.rs:1344`.
  - Extended `ToolsConfig` to carry `search_tool` enablement in
  `core/src/tools/spec.rs:32` and `core/src/tools/spec.rs:56`.
  - Injected dedicated developer instructions for tool-discovery workflow
  in `core/src/codex.rs:483` and `core/src/codex.rs:1976`, using
  `core/templates/search_tool/developer_instructions.md:1`.
  - Added session state to store one-shot selected MCP tools in
  `core/src/state/session.rs:27` and `core/src/state/session.rs:131`.
  - Added filtering so when feature is enabled, only selected MCP tools
  are exposed on the next request (then consumed) in
  `core/src/codex.rs:3800` and `core/src/codex.rs:3843`.
  - Added E2E suite coverage for
  enablement/instructions/hide-until-search/one-turn-selection in
  `core/tests/suite/search_tool.rs:72`,
  `core/tests/suite/search_tool.rs:109`,
  `core/tests/suite/search_tool.rs:147`, and
  `core/tests/suite/search_tool.rs:218`.
  - Refactored test helper utilities to support config-driven tool
  collection in `core/tests/suite/tools.rs:281`.
  
  **Net Behavioral Effect**
  - With `search_tool` **off**: existing MCP behavior (tools exposed
  normally).
  - With `search_tool` **on**: MCP tools start hidden, model must call
  `search_tool_bm25`, and only returned `selected_tools` are available for
  the next model call.

  Anton Panasenko · 2026-02-09 12:53:50 -08:00
  becc3a0424
• feat: do not close unified exec processes across turns (#10799)

  With this PR we do not close the unified exec processes (i.e. background
  terminals) at the end of a turn unless:
  * The user interrupt the turn
  * The user decide to clean the processes through `app-server` or
  `/clean`
  
  I made sure that `codex exec` correctly kill all the processes

  jif-oai · 2026-02-09 10:27:46 +00:00
  6cf61725d0
• Allow user shell commands to run alongside active turns (#10513)

  Summary
  - refactor user shell command execution into a shared helper and add
  modes for standalone vs active-turn execution
  - run user shell commands asynchronously when a turn is already active
  so they don’t replace or abort the current turn
  - extend the tests to cover the new behavior and add the generated Codex
  environment manifest
  
  Testing
  - Not run (not requested)

  jif-oai · 2026-02-05 11:11:00 +00:00
  97582ac52d
• Stop client from being state carrier (#10595)

  I'd like to make client session wide. This requires shedding all random
  state it has to carry.

  pakrym-oai · 2026-02-04 09:05:37 -08:00
  7f20357611
• chore: add phase to message responseitem (#10455)

  ### What
  
  add wiring for `phase` field on `ResponseItem::Message` to lay
  groundwork for differentiating model preambles and final messages.
  currently optional.
  
  follows pattern in #9698.
  
  updated schemas with `just write-app-server-schema` so we can see type
  changes.
  
  ### Tests
  Updated existing tests for SSE parsing and hydrating from history

  sayan-oai · 2026-02-03 02:52:26 +00:00
  fc05374344
• feat: reduce span exposition (#10171)

  This only avoids the creation of duplicates spans

  jif-oai · 2026-01-29 18:15:22 +00:00
  798c4b3260
• feat: adding thread ID to logs + filter in the client (#10150)
  jif-oai · 2026-01-29 16:53:30 +01:00
  89c5f3c4d4
• fix(app-server, core): defer initial context write to rollout file until first turn (#9950)

  ### Overview
  Currently calling `thread/resume` will always bump the thread's
  `updated_at` timestamp. This PR makes it the `updated_at` timestamp
  changes only if a turn is triggered.
  
  ### Additonal context
  What we typically do on resuming a thread is **always** writing “initial
  context” to the rollout file immediately. This initial context includes:
  - Developer instructions derived from sandbox/approval policy + cwd
  - Optional developer instructions (if provided)
  - Optional collaboration-mode instructions
  - Optional user instructions (if provided)
  - Environment context (cwd, shell, etc.)
  
  This PR defers writing the “initial context” to the rollout file until
  the first `turn/start`, so we don't inadvertently bump the thread's
  `updated_at` timestamp until a turn is actually triggered.
  
  This works even though both `thread/resume` and `turn/start` accept
  overrides (such as `model`, `cwd`, etc.) because the initial context is
  seeded from the effective `TurnContext` in memory, computed at
  `turn/start` time, after both sets of overrides have been applied.
  
  **NOTE**: This is a very short-lived solution until we introduce sqlite.
  Then we can remove this.

  Owen Lin · 2026-01-27 10:41:54 -08:00
  fc0fd85349
• chore: update interrupt message (#9925)
  jif-oai · 2026-01-26 19:07:54 +00:00
  09251387e0
• Support end_turn flag (#9698)

  Experimental flag that signals the end of the turn.

  pakrym-oai · 2026-01-22 17:27:48 +00:00
  b511c38ddb
• fix: prevent repeating interrupted turns (#9043)

  ## What
  Record a model-visible `<turn_aborted>` marker in history when a turn is
  interrupted, and treat it as a session prefix.
  
  ## Why
  When a turn is interrupted, Codex emits `TurnAborted` but previously did
  not persist anything model-visible in the conversation history. On the
  next user turn, the model can’t tell the previous work was aborted and
  may resume/repeat earlier actions (including duplicated side effects
  like re-opening PRs).
  
  Fixes: https://github.com/openai/codex/issues/9042
  
  ## How
  On `TurnAbortReason::Interrupted`, append a hidden user message
  containing a `<turn_aborted>…</turn_aborted>` marker and flush.
  Treat `<turn_aborted>` like `<environment_context>` for session-prefix
  filtering.
  Add a regression test to ensure follow-up turns don’t repeat side
  effects from an aborted turn.
  
  ## Testing
  `just fmt`
  `just fix -p codex-core`
  `cargo test -p codex-core -- --test-threads=1`
  `cargo test --all-features -- --test-threads=1`
  
  ---------
  
  Co-authored-by: Skylar Graika <sgraika127@gmail.com>
  Co-authored-by: jif-oai <jif@openai.com>
  Co-authored-by: Eric Traut <etraut@openai.com>

  Skylar Graika · 2026-01-20 13:07:28 -08:00
  b236f1c95d
• feat: timer total turn metrics (#9382)
  jif-oai · 2026-01-19 10:44:31 +00:00
  7ebe13f692