Commit Graph
80 Commits
Author SHA1 Message Date
Charley CunninghamandGitHub ec4a2d07e4 Plan mode: stream proposed plans, emit plan items, and render in TUI (#9786)
## Summary
- Stream proposed plans in Plan Mode using `<proposed_plan>` tags parsed
in core, emitting plan deltas plus a plan `ThreadItem`, while stripping
tags from normal assistant output.
- Persist plan items and rebuild them on resume so proposed plans show
in thread history.
- Wire plan items/deltas through app-server protocol v2 and render a
dedicated proposed-plan view in the TUI, including the “Implement this
plan?” prompt only when a plan item is present.

## Changes

### Core (`codex-rs/core`)
- Added a generic, line-based tag parser that buffers each line until it
can disprove a tag prefix; implements auto-close on `finish()` for
unterminated tags. `codex-rs/core/src/tagged_block_parser.rs`
- Refactored proposed plan parsing to wrap the generic parser.
`codex-rs/core/src/proposed_plan_parser.rs`
- In plan mode, stream assistant deltas as:
  - **Normal text** → `AgentMessageContentDelta`
  - **Plan text** → `PlanDelta` + `TurnItem::Plan` start/completion  
  (`codex-rs/core/src/codex.rs`)
- Final plan item content is derived from the completed assistant
message (authoritative), not necessarily the concatenated deltas.
- Strips `<proposed_plan>` blocks from assistant text in plan mode so
tags don’t appear in normal messages.
(`codex-rs/core/src/stream_events_utils.rs`)
- Persist `ItemCompleted` events only for plan items for rollout replay.
(`codex-rs/core/src/rollout/policy.rs`)
- Guard `update_plan` tool in Plan Mode with a clear error message.
(`codex-rs/core/src/tools/handlers/plan.rs`)
- Updated Plan Mode prompt to:  
  - keep `<proposed_plan>` out of non-final reasoning/preambles  
  - require exact tag formatting  
  - allow only one `<proposed_plan>` block per turn  
  (`codex-rs/core/templates/collaboration_mode/plan.md`)

### Protocol / App-server protocol
- Added `TurnItem::Plan` and `PlanDeltaEvent` to core protocol items.
(`codex-rs/protocol/src/items.rs`, `codex-rs/protocol/src/protocol.rs`)
- Added v2 `ThreadItem::Plan` and `PlanDeltaNotification` with
EXPERIMENTAL markers and note that deltas may not match the final plan
item. (`codex-rs/app-server-protocol/src/protocol/v2.rs`)
- Added plan delta route in app-server protocol common mapping.
(`codex-rs/app-server-protocol/src/protocol/common.rs`)
- Rebuild plan items from persisted `ItemCompleted` events on resume.
(`codex-rs/app-server-protocol/src/protocol/thread_history.rs`)

### App-server
- Forward plan deltas to v2 clients and map core plan items to v2 plan
items. (`codex-rs/app-server/src/bespoke_event_handling.rs`,
`codex-rs/app-server/src/codex_message_processor.rs`)
- Added v2 plan item tests.
(`codex-rs/app-server/tests/suite/v2/plan_item.rs`)

### TUI
- Added a dedicated proposed plan history cell with special background
and padding, and moved “• Proposed Plan” outside the highlighted block.
(`codex-rs/tui/src/history_cell.rs`, `codex-rs/tui/src/style.rs`)
- Only show “Implement this plan?” when a plan item exists.
(`codex-rs/tui/src/chatwidget.rs`,
`codex-rs/tui/src/chatwidget/tests.rs`)

<img width="831" height="847" alt="Screenshot 2026-01-29 at 7 06 24 PM"
src="https://github.com/user-attachments/assets/69794c8c-f96b-4d36-92ef-c1f5c3a8f286"
/>

### Docs / Misc
- Updated protocol docs to mention plan deltas.
(`codex-rs/docs/protocol_v1.md`)
- Minor plumbing updates in exec/debug clients to tolerate plan deltas.
(`codex-rs/debug-client/src/reader.rs`, `codex-rs/exec/...`)

## Tests
- Added core integration tests:
  - Plan mode strips plan from agent messages.
  - Missing `</proposed_plan>` closes at end-of-message.  
  (`codex-rs/core/tests/suite/items.rs`)
- Added unit tests for generic tag parser (prefix buffering, non-tag
lines, auto-close). (`codex-rs/core/src/tagged_block_parser.rs`)
- Existing app-server plan item tests in v2.
(`codex-rs/app-server/tests/suite/v2/plan_item.rs`)

## Notes / Behavior
- Plan output no longer appears in standard assistant text in Plan Mode;
it streams via `PlanDelta` and completes as a `TurnItem::Plan`.
- The final plan item content is authoritative and may diverge from
streamed deltas (documented as experimental).
- Reasoning summaries are not filtered; prompt instructs the model not
to include `<proposed_plan>` outside the final plan message.

## Codex Author
`codex fork 019bec2d-b09d-7450-b292-d7bcdddcdbfb`
2026-01-30 18:59:30 +00:00
Ahmed IbrahimandGitHub b7edeee8ca compaction (#10034)
# External (non-OpenAI) Pull Request Requirements

Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md

If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.

Include a link to a bug report or enhancement request.
2026-01-28 11:36:11 -08:00
Dylan HurdandGitHub 8b3521ee77 feat(core) update Personality on turn (#9644)
## Summary
Support updating Personality mid-Thread via UserTurn/OverwriteTurn. This
is explicitly unused by the clients so far, to simplify PRs - app-server
and tui implementations will be follow-ups.

## Testing
- [x] added integration tests
2026-01-22 12:04:23 -08:00
pakrym-oaiandGitHub b511c38ddb Support end_turn flag (#9698)
Experimental flag that signals the end of the turn.
2026-01-22 17:27:48 +00:00
charley-oaiandGitHub fe641f759f Add collaboration_mode to TurnContextItem (#9583)
## Summary
- add optional `collaboration_mode` to `TurnContextItem` in rollouts
- persist the current collaboration mode when recording turn context
(sampling + compaction)

## Rationale
We already persist turn context data for resume logic. Capturing
collaboration mode in the rollout gives us the mode context for each
turn, enabling follow‑up work to diff mode instructions correctly on
resume.

## Changes
- protocol: add optional `collaboration_mode` field to `TurnContextItem`
- core: persist collaboration mode alongside other turn context settings
in rollouts
2026-01-21 14:14:21 -08:00
b236f1c95d fix: prevent repeating interrupted turns (#9043)
## What
Record a model-visible `<turn_aborted>` marker in history when a turn is
interrupted, and treat it as a session prefix.

## Why
When a turn is interrupted, Codex emits `TurnAborted` but previously did
not persist anything model-visible in the conversation history. On the
next user turn, the model can’t tell the previous work was aborted and
may resume/repeat earlier actions (including duplicated side effects
like re-opening PRs).

Fixes: https://github.com/openai/codex/issues/9042

## How
On `TurnAbortReason::Interrupted`, append a hidden user message
containing a `<turn_aborted>…</turn_aborted>` marker and flush.
Treat `<turn_aborted>` like `<environment_context>` for session-prefix
filtering.
Add a regression test to ensure follow-up turns don’t repeat side
effects from an aborted turn.

## Testing
`just fmt`
`just fix -p codex-core`
`cargo test -p codex-core -- --test-threads=1`
`cargo test --all-features -- --test-threads=1`

---------

Co-authored-by: Skylar Graika <sgraika127@gmail.com>
Co-authored-by: jif-oai <jif@openai.com>
Co-authored-by: Eric Traut <etraut@openai.com>
2026-01-20 13:07:28 -08:00
charley-oaiandGitHub eb90e20c0b Persist text elements through TUI input and history (#9393)
Continuation of breaking up this PR
https://github.com/openai/codex/pull/9116

## Summary
- Thread user text element ranges through TUI/TUI2 input, submission,
queueing, and history so placeholders survive resume/edit flows.
- Preserve local image attachments alongside text elements and rehydrate
placeholders when restoring drafts.
- Keep model-facing content shapes clean by attaching UI metadata only
to user input/events (no API content changes).

## Key Changes
- TUI/TUI2 composer now captures text element ranges, trims them with
text edits, and restores them when submission is suppressed.
- User history cells render styled spans for text elements and keep
local image paths for future rehydration.
- Initial chat widget bootstraps accept empty `initial_text_elements` to
keep initialization uniform.
- Protocol/core helpers updated to tolerate the new InputText field
shape without changing payloads sent to the API.
2026-01-19 23:49:34 -08:00
Dylan HurdandGitHub 675f165c56 fix(core) Preserve base_instructions in SessionMeta (#9427)
## Summary
This PR consolidates base_instructions onto SessionMeta /
SessionConfiguration, so we ensure `base_instructions` is set once per
session and should be (mostly) immutable, unless:
- overridden by config on resume / fork
- sub-agent tasks, like review or collab


In a future PR, we should convert all references to `base_instructions`
to consistently used the typed struct, so it's less likely that we put
other strings there. See #9423. However, this PR is already quite
complex, so I'm deferring that to a follow-up.

## Testing
- [x] Added a resume test to assert that instructions are preserved. In
particular, `resume_switches_models_preserves_base_instructions` fails
against main.

Existing test coverage thats assert base instructions are preserved
across multiple requests in a session:
- Manual compact keeps baseline instructions:
core/tests/suite/compact.rs:199
- Auto-compact keeps baseline instructions:
core/tests/suite/compact.rs:1142
- Prompt caching reuses the same instructions across two requests:
core/tests/suite/prompt_caching.rs:150 and
core/tests/suite/prompt_caching.rs:157
- Prompt caching with explicit expected string across two requests:
core/tests/suite/prompt_caching.rs:213 and
core/tests/suite/prompt_caching.rs:222
- Resume with model switch keeps original instructions:
core/tests/suite/resume.rs:136
- Compact/resume/fork uses request 0 instructions for later expected
payloads: core/tests/suite/compact_resume_fork.rs:215
2026-01-19 21:59:36 -08:00
Ahmed IbrahimandGitHub b11e96fb04 Act on reasoning-included per turn (#9402)
- Reset reasoning-included flag each turn and update compaction test
2026-01-19 11:23:25 -08:00
Ahmed IbrahimandGitHub 1478a88eb0 Add collaboration developer instructions (#9424)
- Add additional instructions when they are available
- Make sure to update them on change either UserInput or UserTurn
2026-01-18 01:31:14 +00:00
charley-oaiandGitHub 4a9c2bcc5a Add text element metadata to types (#9235)
Initial type tweaking PR to make the diff of
https://github.com/openai/codex/pull/9116 smaller

This should not change any behavior, just adds some fields to types
2026-01-14 16:41:50 -08:00
pakrym-oaiandGitHub d75626ad99 Reuse websocket connection (#9127)
Reuses the connection but still sends full requests.
2026-01-13 03:30:09 +00:00
pakrym-oaiandGitHub 490c1c1fdd Add model client sessions (#9102)
Maintain a long-running session.
2026-01-13 01:15:56 +00:00
jif-oaiandGitHub 1aed01e99f renaming: task to turn (#8963) 2026-01-09 17:31:17 +00:00
jif-oaiandGitHub 69898e3dba clean: all history cloning (#8916) 2026-01-08 18:17:18 +00:00
jif-oaiandGitHub 116059c3a0 chore: unify conversation with thread name (#8830)
Done and verified by Codex + refactor feature of RustRover
2026-01-07 17:04:53 +00:00
Anton PanasenkoandGitHub cbc5fb9acf chore: save more about turn context in rollout log file (#8458)
### Motivation
- Persist richer per-turn configuration in rollouts so resumed/forked
sessions and tooling can reason about the exact instruction inputs and
output constraints used for a turn.

### Description
- Extend `TurnContextItem` to include optional `base_instructions`,
`user_instructions`, and `developer_instructions`.
- Record the optional `final_output_json_schema` associated with a turn.
- Add an optional `truncation_policy` to `TurnContextItem` and populate
it when writing turn-context rollout items.
- Introduce a protocol-level `TruncationPolicy` representation and
convert from core truncation policy when recording.

### Testing
- `cargo test -p codex-protocol` (pass)
2025-12-22 19:51:07 -08:00
pakrym-oaiandGitHub b3ddd50eee Remote compact for API-key users (#7835) 2025-12-12 10:05:02 -08:00
Michael BolinandGitHub 1cfc967eb8 fix: Features should be immutable over the lifetime of a session/thread (#7540)
I noticed that `features: Features` was defined on `struct
SessionConfiguration`, which is commonly owned by `SessionState`, which
is in turn owned by `Session`.

Though I do not believe that `Features` should be allowed to be modified
over the course of a session (if the feature state is not invariant, it
makes it harder to reason about), which argues that it should live on
`Session` rather than `SessionState` or `SessionConfiguration`.

This PR moves `Features` to `Session` and updates all call sites. It
appears the only place we were mutating `Features` was:

- in tests
- the sub-agent config for a review task:


https://github.com/openai/codex/blob/3ef76ff29d5eed258fb6b8550e0e2b973d0dca21/codex-rs/core/src/tasks/review.rs#L86-L89

Note this change also means it is no longer an `async` call to check the
state of a feature, eliminating the possibility of a
[TOCTTOU](https://en.wikipedia.org/wiki/Time-of-check_to_time-of-use)
error between checking the state of a feature and acting on it:


https://github.com/openai/codex/blob/3ef76ff29d5eed258fb6b8550e0e2b973d0dca21/codex-rs/core/src/codex.rs#L1069-L1076
2025-12-03 16:12:31 -08:00
jif-oaiandGitHub 9ba27cfa0a feat: add compaction event (#7289) 2025-11-25 16:12:14 +00:00
Celia ChenandGitHub 9bce050385 [app-server & core] introduce new codex error code and v2 app-server error events (#6938)
This PR does two things:
1. populate a new `codex_error_code` protocol in error events sent from
core to client;
2. old v1 core events `codex/event/stream_error` and `codex/event/error`
will now both become `error`. We also show codex error code for
turncompleted -> error status.

new events in app server test:
```
< {
<   "method": "codex/event/stream_error",
<   "params": {
<     "conversationId": "019aa34c-0c14-70e0-9706-98520a760d67",
<     "id": "0",
<     "msg": {
<       "codex_error_code": {
<         "response_stream_disconnected": {
<           "http_status_code": 401
<         }
<       },
<       "message": "Reconnecting... 2/5",
<       "type": "stream_error"
<     }
<   }
< }

 {
<   "method": "error",
<   "params": {
<     "error": {
<       "codexErrorCode": {
<         "responseStreamDisconnected": {
<           "httpStatusCode": 401
<         }
<       },
<       "message": "Reconnecting... 2/5"
<     }
<   }
< }

< {
<   "method": "turn/completed",
<   "params": {
<     "turn": {
<       "error": {
<         "codexErrorCode": {
<           "responseTooManyFailedAttempts": {
<             "httpStatusCode": 401
<           }
<         },
<         "message": "exceeded retry limit, last status: 401 Unauthorized, request id: 9a1b495a1a97ed3e-SJC"
<       },
<       "id": "0",
<       "items": [],
<       "status": "failed"
<     }
<   }
< }
```
2025-11-20 23:06:55 +00:00
Celia ChenandGitHub 72a1453ac5 Revert "[core] add optional status_code to error events (#6865)" (#6955)
This reverts commit c2ec477d93.

# External (non-OpenAI) Pull Request Requirements

Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md

If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.

Include a link to a bug report or enhancement request.
2025-11-20 01:26:14 +00:00
Celia ChenandGitHub c2ec477d93 [core] add optional status_code to error events (#6865)
We want to better uncover error status code for clients. Add an optional
status_code to error events (thread error, error, stream error) so app
server could uncover the status code from the client side later.

in event log:
```
< {
<   "method": "codex/event/stream_error",
<   "params": {
<     "conversationId": "019a9a32-f576-7292-9711-8e57e8063536",
<     "id": "0",
<     "msg": {
<       "message": "Reconnecting... 5/5",
<       "status_code": 401,
<       "type": "stream_error"
<     }
<   }
< }
< {
<   "method": "codex/event/error",
<   "params": {
<     "conversationId": "019a9a32-f576-7292-9711-8e57e8063536",
<     "id": "0",
<     "msg": {
<       "message": "exceeded retry limit, last status: 401 Unauthorized, request id: 9a0cb03a485067f7-SJC",
<       "status_code": 401,
<       "type": "error"
<     }
<   }
< }
```
2025-11-19 19:51:21 +00:00
jif-oaiandGitHub 3e9e1d993d chore: consolidate compaction token usage (#6894) 2025-11-19 11:26:01 +00:00
pakrym-oaiandGitHub 75f38f16dd Run remote auto compaction (#6879) 2025-11-19 00:43:58 -08:00
Ahmed IbrahimandGitHub 3de8790714 Add the utility to truncate by tokens (#6746)
- This PR is to make it on path for truncating by tokens. This path will
be initially used by unified exec and context manager (responsible for
MCP calls mainly).
- We are exposing new config `calls_output_max_tokens`
- Use `tokens` as the main budget unit but truncate based on the model
family by Introducing `TruncationPolicy`.
- Introduce `truncate_text` as a router for truncation based on the
mode.

In next PRs:
- remove truncate_with_line_bytes_budget
- Add the ability to the model to override the token budget.
2025-11-18 11:36:23 -08:00
838531d3e4 feat: remote compaction (#6795)
Co-authored-by: pakrym-oai <pakrym@openai.com>
2025-11-18 16:51:16 +00:00
Ahmed IbrahimandGitHub 0b28e72b66 Improve compact (#6692)
This PR does the following:
- Add compact prefix to the summary
- Change the compaction prompt
- Allow multiple compaction for long running tasks
- Filter out summary messages on the following compaction

Considerations:
- Filtering out the summary message isn't the most clean
- Theoretically, we can end up in infinite compaction loop if the user
messages > compaction limit . However, that's not possible in today's
code because we have hard cap on user messages.
- We need to address having multiple user messages because it confuses
the model.

Testing:
- Making sure that after compact we always end up with one user message
(task) and one summary, even on multiple compaction.
2025-11-15 07:17:51 +00:00
jif-oaiandGitHub 2a417c47ac feat: proxy context left after compaction (#6597) 2025-11-13 16:54:03 +01:00
jif-oaiandGitHub 50a77dc138 Move compact (#6454) 2025-11-10 11:59:48 +00:00