29 Commits

• compaction (#10034)

  # External (non-OpenAI) Pull Request Requirements
  
  Before opening this Pull Request, please read the dedicated
  "Contributing" markdown file or your PR may be closed:
  https://github.com/openai/codex/blob/main/docs/contributing.md
  
  If your PR conforms to our contribution guidelines, replace this text
  with a detailed and high quality description of your changes.
  
  Include a link to a bug report or enhancement request.

  Ahmed Ibrahim · 2026-01-28 11:36:11 -08:00
  b7edeee8ca
• feat(core) update Personality on turn (#9644)

  ## Summary
  Support updating Personality mid-Thread via UserTurn/OverwriteTurn. This
  is explicitly unused by the clients so far, to simplify PRs - app-server
  and tui implementations will be follow-ups.
  
  ## Testing
  - [x] added integration tests

  Dylan Hurd · 2026-01-22 12:04:23 -08:00
  8b3521ee77
• Support end_turn flag (#9698)

  Experimental flag that signals the end of the turn.

  pakrym-oai · 2026-01-22 17:27:48 +00:00
  b511c38ddb
• Add collaboration_mode to TurnContextItem (#9583)

  ## Summary
  - add optional `collaboration_mode` to `TurnContextItem` in rollouts
  - persist the current collaboration mode when recording turn context
  (sampling + compaction)
  
  ## Rationale
  We already persist turn context data for resume logic. Capturing
  collaboration mode in the rollout gives us the mode context for each
  turn, enabling follow‑up work to diff mode instructions correctly on
  resume.
  
  ## Changes
  - protocol: add optional `collaboration_mode` field to `TurnContextItem`
  - core: persist collaboration mode alongside other turn context settings
  in rollouts

  charley-oai · 2026-01-21 14:14:21 -08:00
  fe641f759f
• fix: prevent repeating interrupted turns (#9043)

  ## What
  Record a model-visible `<turn_aborted>` marker in history when a turn is
  interrupted, and treat it as a session prefix.
  
  ## Why
  When a turn is interrupted, Codex emits `TurnAborted` but previously did
  not persist anything model-visible in the conversation history. On the
  next user turn, the model can’t tell the previous work was aborted and
  may resume/repeat earlier actions (including duplicated side effects
  like re-opening PRs).
  
  Fixes: https://github.com/openai/codex/issues/9042
  
  ## How
  On `TurnAbortReason::Interrupted`, append a hidden user message
  containing a `<turn_aborted>…</turn_aborted>` marker and flush.
  Treat `<turn_aborted>` like `<environment_context>` for session-prefix
  filtering.
  Add a regression test to ensure follow-up turns don’t repeat side
  effects from an aborted turn.
  
  ## Testing
  `just fmt`
  `just fix -p codex-core`
  `cargo test -p codex-core -- --test-threads=1`
  `cargo test --all-features -- --test-threads=1`
  
  ---------
  
  Co-authored-by: Skylar Graika <sgraika127@gmail.com>
  Co-authored-by: jif-oai <jif@openai.com>
  Co-authored-by: Eric Traut <etraut@openai.com>

  Skylar Graika · 2026-01-20 13:07:28 -08:00
  b236f1c95d
• Persist text elements through TUI input and history (#9393)

  Continuation of breaking up this PR
  https://github.com/openai/codex/pull/9116
  
  ## Summary
  - Thread user text element ranges through TUI/TUI2 input, submission,
  queueing, and history so placeholders survive resume/edit flows.
  - Preserve local image attachments alongside text elements and rehydrate
  placeholders when restoring drafts.
  - Keep model-facing content shapes clean by attaching UI metadata only
  to user input/events (no API content changes).
  
  ## Key Changes
  - TUI/TUI2 composer now captures text element ranges, trims them with
  text edits, and restores them when submission is suppressed.
  - User history cells render styled spans for text elements and keep
  local image paths for future rehydration.
  - Initial chat widget bootstraps accept empty `initial_text_elements` to
  keep initialization uniform.
  - Protocol/core helpers updated to tolerate the new InputText field
  shape without changing payloads sent to the API.

  charley-oai · 2026-01-19 23:49:34 -08:00
  eb90e20c0b
• fix(core) Preserve base_instructions in SessionMeta (#9427)

  ## Summary
  This PR consolidates base_instructions onto SessionMeta /
  SessionConfiguration, so we ensure `base_instructions` is set once per
  session and should be (mostly) immutable, unless:
  - overridden by config on resume / fork
  - sub-agent tasks, like review or collab
  
  
  In a future PR, we should convert all references to `base_instructions`
  to consistently used the typed struct, so it's less likely that we put
  other strings there. See #9423. However, this PR is already quite
  complex, so I'm deferring that to a follow-up.
  
  ## Testing
  - [x] Added a resume test to assert that instructions are preserved. In
  particular, `resume_switches_models_preserves_base_instructions` fails
  against main.
  
  Existing test coverage thats assert base instructions are preserved
  across multiple requests in a session:
  - Manual compact keeps baseline instructions:
  core/tests/suite/compact.rs:199
  - Auto-compact keeps baseline instructions:
  core/tests/suite/compact.rs:1142
  - Prompt caching reuses the same instructions across two requests:
  core/tests/suite/prompt_caching.rs:150 and
  core/tests/suite/prompt_caching.rs:157
  - Prompt caching with explicit expected string across two requests:
  core/tests/suite/prompt_caching.rs:213 and
  core/tests/suite/prompt_caching.rs:222
  - Resume with model switch keeps original instructions:
  core/tests/suite/resume.rs:136
  - Compact/resume/fork uses request 0 instructions for later expected
  payloads: core/tests/suite/compact_resume_fork.rs:215

  Dylan Hurd · 2026-01-19 21:59:36 -08:00
  675f165c56
• Act on reasoning-included per turn (#9402)

  - Reset reasoning-included flag each turn and update compaction test

  Ahmed Ibrahim · 2026-01-19 11:23:25 -08:00
  b11e96fb04
• Add collaboration developer instructions (#9424)

  - Add additional instructions when they are available
  - Make sure to update them on change either UserInput or UserTurn

  Ahmed Ibrahim · 2026-01-18 01:31:14 +00:00
  1478a88eb0
• Add text element metadata to types (#9235)

  Initial type tweaking PR to make the diff of
  https://github.com/openai/codex/pull/9116 smaller
  
  This should not change any behavior, just adds some fields to types

  charley-oai · 2026-01-14 16:41:50 -08:00
  4a9c2bcc5a
• Reuse websocket connection (#9127)

  Reuses the connection but still sends full requests.

  pakrym-oai · 2026-01-13 03:30:09 +00:00
  d75626ad99
• Add model client sessions (#9102)

  Maintain a long-running session.

  pakrym-oai · 2026-01-13 01:15:56 +00:00
  490c1c1fdd
• renaming: task to turn (#8963)
  jif-oai · 2026-01-09 17:31:17 +00:00
  1aed01e99f
• clean: all history cloning (#8916)
  jif-oai · 2026-01-08 18:17:18 +00:00
  69898e3dba
• chore: unify conversation with thread name (#8830)

  Done and verified by Codex + refactor feature of RustRover

  jif-oai · 2026-01-07 17:04:53 +00:00
  116059c3a0
• chore: save more about turn context in rollout log file (#8458)

  ### Motivation
  - Persist richer per-turn configuration in rollouts so resumed/forked
  sessions and tooling can reason about the exact instruction inputs and
  output constraints used for a turn.
  
  ### Description
  - Extend `TurnContextItem` to include optional `base_instructions`,
  `user_instructions`, and `developer_instructions`.
  - Record the optional `final_output_json_schema` associated with a turn.
  - Add an optional `truncation_policy` to `TurnContextItem` and populate
  it when writing turn-context rollout items.
  - Introduce a protocol-level `TruncationPolicy` representation and
  convert from core truncation policy when recording.
  
  ### Testing
  - `cargo test -p codex-protocol` (pass)

  Anton Panasenko · 2025-12-22 19:51:07 -08:00
  cbc5fb9acf
• Remote compact for API-key users (#7835)
  pakrym-oai · 2025-12-12 10:05:02 -08:00
  b3ddd50eee
• fix: Features should be immutable over the lifetime of a session/thread (#7540)

  I noticed that `features: Features` was defined on `struct
  SessionConfiguration`, which is commonly owned by `SessionState`, which
  is in turn owned by `Session`.
  
  Though I do not believe that `Features` should be allowed to be modified
  over the course of a session (if the feature state is not invariant, it
  makes it harder to reason about), which argues that it should live on
  `Session` rather than `SessionState` or `SessionConfiguration`.
  
  This PR moves `Features` to `Session` and updates all call sites. It
  appears the only place we were mutating `Features` was:
  
  - in tests
  - the sub-agent config for a review task:
  
  
  https://github.com/openai/codex/blob/3ef76ff29d5eed258fb6b8550e0e2b973d0dca21/codex-rs/core/src/tasks/review.rs#L86-L89
  
  Note this change also means it is no longer an `async` call to check the
  state of a feature, eliminating the possibility of a
  [TOCTTOU](https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use)
  error between checking the state of a feature and acting on it:
  
  
  https://github.com/openai/codex/blob/3ef76ff29d5eed258fb6b8550e0e2b973d0dca21/codex-rs/core/src/codex.rs#L1069-L1076

  Michael Bolin · 2025-12-03 16:12:31 -08:00
  1cfc967eb8
• feat: add compaction event (#7289)
  jif-oai · 2025-11-25 16:12:14 +00:00
  9ba27cfa0a
• [app-server & core] introduce new codex error code and v2 app-server error events (#6938)

  This PR does two things:
  1. populate a new `codex_error_code` protocol in error events sent from
  core to client;
  2. old v1 core events `codex/event/stream_error` and `codex/event/error`
  will now both become `error`. We also show codex error code for
  turncompleted -> error status.
  
  new events in app server test:
  ```
  < {
  <   "method": "codex/event/stream_error",
  <   "params": {
  <     "conversationId": "019aa34c-0c14-70e0-9706-98520a760d67",
  <     "id": "0",
  <     "msg": {
  <       "codex_error_code": {
  <         "response_stream_disconnected": {
  <           "http_status_code": 401
  <         }
  <       },
  <       "message": "Reconnecting... 2/5",
  <       "type": "stream_error"
  <     }
  <   }
  < }
  
   {
  <   "method": "error",
  <   "params": {
  <     "error": {
  <       "codexErrorCode": {
  <         "responseStreamDisconnected": {
  <           "httpStatusCode": 401
  <         }
  <       },
  <       "message": "Reconnecting... 2/5"
  <     }
  <   }
  < }
  
  < {
  <   "method": "turn/completed",
  <   "params": {
  <     "turn": {
  <       "error": {
  <         "codexErrorCode": {
  <           "responseTooManyFailedAttempts": {
  <             "httpStatusCode": 401
  <           }
  <         },
  <         "message": "exceeded retry limit, last status: 401 Unauthorized, request id: 9a1b495a1a97ed3e-SJC"
  <       },
  <       "id": "0",
  <       "items": [],
  <       "status": "failed"
  <     }
  <   }
  < }
  ```

  Celia Chen · 2025-11-20 23:06:55 +00:00
  9bce050385
• Revert "[core] add optional status_code to error events (#6865)" (#6955)

  This reverts commit c2ec477d93.
  
  # External (non-OpenAI) Pull Request Requirements
  
  Before opening this Pull Request, please read the dedicated
  "Contributing" markdown file or your PR may be closed:
  https://github.com/openai/codex/blob/main/docs/contributing.md
  
  If your PR conforms to our contribution guidelines, replace this text
  with a detailed and high quality description of your changes.
  
  Include a link to a bug report or enhancement request.

  Celia Chen · 2025-11-20 01:26:14 +00:00
  72a1453ac5
• [core] add optional status_code to error events (#6865)

  We want to better uncover error status code for clients. Add an optional
  status_code to error events (thread error, error, stream error) so app
  server could uncover the status code from the client side later.
  
  in event log:
  ```
  < {
  <   "method": "codex/event/stream_error",
  <   "params": {
  <     "conversationId": "019a9a32-f576-7292-9711-8e57e8063536",
  <     "id": "0",
  <     "msg": {
  <       "message": "Reconnecting... 5/5",
  <       "status_code": 401,
  <       "type": "stream_error"
  <     }
  <   }
  < }
  < {
  <   "method": "codex/event/error",
  <   "params": {
  <     "conversationId": "019a9a32-f576-7292-9711-8e57e8063536",
  <     "id": "0",
  <     "msg": {
  <       "message": "exceeded retry limit, last status: 401 Unauthorized, request id: 9a0cb03a485067f7-SJC",
  <       "status_code": 401,
  <       "type": "error"
  <     }
  <   }
  < }
  ```

  Celia Chen · 2025-11-19 19:51:21 +00:00
  c2ec477d93
• chore: consolidate compaction token usage (#6894)
  jif-oai · 2025-11-19 11:26:01 +00:00
  3e9e1d993d
• Run remote auto compaction (#6879)
  pakrym-oai · 2025-11-19 00:43:58 -08:00
  75f38f16dd
• Add the utility to truncate by tokens (#6746)

  - This PR is to make it on path for truncating by tokens. This path will
  be initially used by unified exec and context manager (responsible for
  MCP calls mainly).
  - We are exposing new config `calls_output_max_tokens`
  - Use `tokens` as the main budget unit but truncate based on the model
  family by Introducing `TruncationPolicy`.
  - Introduce `truncate_text` as a router for truncation based on the
  mode.
  
  In next PRs:
  - remove truncate_with_line_bytes_budget
  - Add the ability to the model to override the token budget.

  Ahmed Ibrahim · 2025-11-18 11:36:23 -08:00
  3de8790714
• feat: remote compaction (#6795)

  Co-authored-by: pakrym-oai <pakrym@openai.com>

  jif-oai · 2025-11-18 16:51:16 +00:00
  838531d3e4
• Improve compact (#6692)

  This PR does the following:
  - Add compact prefix to the summary
  - Change the compaction prompt
  - Allow multiple compaction for long running tasks
  - Filter out summary messages on the following compaction
  
  Considerations:
  - Filtering out the summary message isn't the most clean
  - Theoretically, we can end up in infinite compaction loop if the user
  messages > compaction limit . However, that's not possible in today's
  code because we have hard cap on user messages.
  - We need to address having multiple user messages because it confuses
  the model.
  
  Testing:
  - Making sure that after compact we always end up with one user message
  (task) and one summary, even on multiple compaction.

  Ahmed Ibrahim · 2025-11-15 07:17:51 +00:00
  0b28e72b66
• feat: proxy context left after compaction (#6597)
  jif-oai · 2025-11-13 16:54:03 +01:00
  2a417c47ac
• Move compact (#6454)
  jif-oai · 2025-11-10 11:59:48 +00:00
  50a77dc138