32 Commits

• Preserve image detail in app-server inputs (#20693)

  ## Summary
  
  - Add optional image detail to user image inputs across core, app-server
  v2, thread history/event mapping, and the generated app-server
  schemas/types.
  - Preserve requested detail when serializing Responses image inputs:
  omitted detail stays on the existing `high` default, while explicit
  `original` keeps local images on the original-resolution path.
  - Support `high`/`original` consistently for tool image outputs,
  including MCP `codex/imageDetail`, code-mode image helpers, and
  `view_image`.

  Curtis 'Fjord' Hawthorne · 2026-05-15 15:04:04 -07:00
  8543e39885
• Organize context fragments (#18794)

  Organize context fragments under `core/context`. Implement same trait on
  all of them.

  pakrym-oai · 2026-04-20 22:39:17 -07:00
  4c2e730488
• Update image outputs to default to high detail (#18386)

  Do not assume the default `detail`.

  pakrym-oai · 2026-04-18 11:01:12 -07:00
  53b1570367
• Trim pre-turn context updates during rollback (#15577)

  ## Summary
  - trim contiguous developer/contextual-user pre-turn updates when
  rollback cuts back to a user turn
  - add a focused history regression test for the trim behavior
  - update the rollback request-boundary snapshots to show the fixed
  non-duplicating context shape
  
  ---------
  
  Co-authored-by: Codex <noreply@openai.com>

  Charley Cunningham · 2026-03-24 12:43:53 -07:00
  2d61357c76
• feat: structured multi-agent output (#15515)

  Send input now sends messages as assistant message and with this format:
  
  ```
  author: /root/worker_a
  recipient: /root/worker_a/tester
  other_recipients: []
  Content: bla bla bla. Actual content. Only text for now
  ```

  jif-oai · 2026-03-23 18:53:54 +00:00
  37ac0c093c
• [hooks] use a user message > developer message for prompt continuation (#14867)

  ## Summary
  
  Persist Stop-hook continuation prompts as `user` messages instead of
  hidden `developer` messages + some requested integration tests
  
  This is a followup to @pakrym 's comment in
  https://github.com/openai/codex/pull/14532 to make sure stop-block
  continuation prompts match training for turn loops
  
  - Stop continuation now writes `<hook_prompt hook_run_id="...">stop
  hook's user prompt<hook_prompt>`
  - Introduces quick-xml dependency, though we already indirectly depended
  on it anyway via syntect
  - This PR only has about 500 lines of actual logic changes, the rest is
  tests/schema
  
  ## Testing
  
  Example run (with a sessionstart hook and 3 stop hooks) - this shows
  context added by session start, then two stop hooks sending their own
  additional prompts in a new turn. The model responds with a single
  message addressing both. Then when that turn ends, the hooks detect that
  they just ran using `stop_hook_active` and decide not to infinite loop
  
  test files for this (unzip, move codex -> .codex):
  [codex.zip](https://github.com/user-attachments/files/26075806/codex.zip)
  
  ```
  › cats
  
  
  • Running SessionStart hook: lighting the observatory
  
  SessionStart hook (completed)
    warning: Hi, I'm a session start hook for wizard-tower (startup).
    hook context: A wimboltine stonpet is an exotic cuisine from hyperspace
  
  • Cats are tiny zen wizards, my friend: equal parts nap, mystery, and chaos. If you want, we can talk
    cat facts, cat breeds, cat names, or build something cat-themed in this repo.
  
  • Running Stop hook: checking the tower wards
  
  • Running Stop hook: sacking the guards
  
  • Running Stop hook: hiring the guards
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook reviewed the completed reply (177 chars).
  
  Stop hook (blocked)
    warning: Wizard Tower Stop hook continuing conversation
    feedback: cook the stonpet
  
  Stop hook (blocked)
    warning: Wizard Tower Stop hook continuing conversation
    feedback: eat the cooked stonpet
  
  • Stonpet’s cooked, aloha style: flash-seared over a blue quasiflame, glazed with nebula salt, and
    rested until the hyperspace juices settle.
  
    Now we eat with gratitude, my friend. One mindful bite in, and the flavor is pure cosmic surf:
    smoky, bright, and totally out of this dimension.
  
  • Running Stop hook: checking the tower wards
  
  • Running Stop hook: sacking the guards
  
  • Running Stop hook: hiring the guards
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook reviewed the completed reply (285 chars).
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook saw a second pass and stayed calm to avoid a loop.
  
  Stop hook (completed)
    warning: Wizard Tower Stop hook saw a second pass and stayed calm to avoid a loop.
  ```

  Andrei Eternal · 2026-03-19 10:53:08 -07:00
  267499bed8
• feat: add memory citation to agent message (#14821)

  Client side to come

  jif-oai · 2026-03-18 10:03:38 +00:00
  a265d6043e
• fix: move inline codex-rs/core unit tests into sibling files (#14444)

  ## Why
  PR #13783 moved the `codex.rs` unit tests into `codex_tests.rs`. This
  applies the same extraction pattern across the rest of `codex-rs/core`
  so the production modules stay focused on runtime code instead of large
  inline test blocks.
  
  Keeping the tests in sibling files also makes follow-up edits easier to
  review because product changes no longer have to share a file with
  hundreds or thousands of lines of test scaffolding.
  
  ## What changed
  - replaced each inline `mod tests { ... }` in `codex-rs/core/src/**`
  with a path-based module declaration
  - moved each extracted unit test module into a sibling `*_tests.rs`
  file, using `mod_tests.rs` for `mod.rs` modules
  - preserved the existing `cfg(...)` guards and module-local structure so
  the refactor remains structural rather than behavioral
  
  ## Testing
  - `cargo test -p codex-core --lib` (`1653 passed; 0 failed; 5 ignored`)
  - `just fix -p codex-core`
  - `cargo fmt --check`
  - `cargo shear`

  Michael Bolin · 2026-03-12 08:16:36 -07:00
  0c8a36676a
• Enabling CWD Saving for Image-Gen (#13607)

  Codex now saves the generated image on to your current working
  directory.

  Won Park · 2026-03-06 00:47:21 -08:00
  ee1a20258a
• image-gen-event/client_processing (#13512)

  enabling client-side to process with image-generation capabilities
  (setting app-server)

  Won Park · 2026-03-04 16:54:38 -08:00
  229e6d0347
• core: bundle settings diff updates into one dev/user envelope (#12417)

  ## Summary
  - bundle contextual prompt injection into at most one developer message
  plus one contextual user message in both:
    - per-turn settings updates
    - initial context insertion
  - preserve `<model_switch>` across compaction by rebuilding it through
  canonical initial-context injection, instead of relying on
  strip/reattach hacks
  - centralize contextual user fragment detection in one shared definition
  table and reuse it for parsing/compaction logic
  - keep `AGENTS.md` in its natural serialized format:
    - `# AGENTS.md instructions for {dirname}`
    - `<INSTRUCTIONS>...</INSTRUCTIONS>`
  - simplify related tests/helpers and accept the expected snapshot/layout
  updates from bundled multi-part messages
  
  ## Why
  The goal is to converge toward a simpler, more intentional prompt shape
  where contextual updates are consistently represented as one developer
  envelope plus one contextual user envelope, while keeping parsing and
  compaction behavior aligned with that representation.
  
  ## Notable details
  - the temporary `SettingsUpdateEnvelope` wrapper was removed; these
  paths now return `Vec<ResponseItem>` directly
  - local/remote compaction no longer rely on model-switch strip/restore
  helpers
  - contextual user detection is now driven by shared fragment definitions
  instead of ad hoc matcher assembly
  - AGENTS/user instructions are still the same logical context; only the
  synthetic `<user_instructions>` wrapper was replaced by the natural
  AGENTS text format
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-app-server
  codex_message_processor::tests::extract_conversation_summary_prefers_plain_user_messages
  -- --exact`
  - `cargo test -p codex-core
  compact::tests::collect_user_messages_filters_session_prefix_entries
  --lib -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact::snapshot_request_shape_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact_remote::snapshot_request_shape_remote_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_apps_guidance_as_developer_message_when_enabled'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_developer_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_user_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::resume_includes_initial_messages_and_sends_prior_items'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::review::review_input_isolated_from_parent_history' -- --exact`
  - `cargo test -p codex-exec --test all
  'suite::resume::exec_resume_last_respects_cwd_filter_and_all_flag' --
  --exact`
  - `cargo test -p core_test_support
  context_snapshot::tests::full_text_mode_preserves_unredacted_text --
  --exact`
  
  ## Notes
  - I also ran several targeted `compact`, `compact_remote`,
  `prompt_caching`, `model_visible_layout`, and `event_mapping` tests
  while iterating on prompt-shape changes.
  - I have not claimed a clean full-workspace `cargo test` from this
  environment because local sandbox/resource conditions have previously
  produced unrelated failures in large workspace runs.

  Charley Cunningham · 2026-02-26 00:12:08 -08:00
  07aefffb1f
• fix(tui): conditionally restore status indicator using message phase (#10947)

  TLDR: use new message phase field emitted by preamble-supported models
  to determine whether an AgentMessage is mid-turn commentary. if so,
  restore the status indicator afterwards to indicate the turn has not
  completed.
  
  ### Problem
  `commit_tick` hides the status indicator while streaming assistant text.
  For preamble-capable models, that text can be commentary mid-turn, so
  hiding was correct during streaming but restore timing mattered:
  - restoring too aggressively caused jitter/flashing
  - not restoring caused indicator to stay hidden before subsequent work
  (tool calls, web search, etc.)
  
  ### Fix
  - Add optional `phase` to `AgentMessageItem` and propagate it from
  `ResponseItem::Message`
  - Keep indicator hidden during streamed commit ticks, restore only when:
    - assistant item completes as `phase=commentary`, and
    - stream queues are idle + task is still running.
  - Treat `phase=None` as final-answer behavior (no restore) to keep
  existing behavior for non-preamble models
  
  ### Tests
  Add/update tests for:
  - no idle-tick restore without commentary completion
  - commentary completion restoring status before tool begin
  - snapshot coverage for preamble/status behavior
  
  ---------
  
  Co-authored-by: Josh McKinney <joshka@openai.com>

  sayan-oai · 2026-02-07 02:39:52 +00:00
  5d2702f6b8
• chore: add phase to message responseitem (#10455)

  ### What
  
  add wiring for `phase` field on `ResponseItem::Message` to lay
  groundwork for differentiating model preambles and final messages.
  currently optional.
  
  follows pattern in #9698.
  
  updated schemas with `just write-app-server-schema` so we can see type
  changes.
  
  ### Tests
  Updated existing tests for SSE parsing and hydrating from history

  sayan-oai · 2026-02-03 02:52:26 +00:00
  fc05374344
• add missing fields to WebSearchAction and update app-server types (#10276)

  - add `WebSearchAction` to app-server v2 types
  - add `queries` to `WebSearchAction::Search` type
  
  Updated tests.

  sayan-oai · 2026-01-30 16:37:56 -08:00
  eb86663dcb
• fix: handle all web_search actions and in progress invocations (#9960)

  ### Summary
  - Parse all `web_search` tool actions (`search`, `find_in_page`,
  `open_page`).
  - Previously we only parsed + displayed `search`, which made the TUI
  appear to pause when the other actions were being used.
  - Show in progress `web_search` calls as `Searching the web`
    - Previously we only showed completed tool calls
  
  <img width="308" height="149" alt="image"
  src="https://github.com/user-attachments/assets/90a4e8ff-b06a-48ff-a282-b57b31121845"
  />
  
  ### Tests
  Added + updated tests, tested locally
  
  ### Follow ups
  Update VSCode extension to display these as well

  sayan-oai · 2026-01-27 03:33:48 +00:00
  86adf53235
• Support end_turn flag (#9698)

  Experimental flag that signals the end of the turn.

  pakrym-oai · 2026-01-22 17:27:48 +00:00
  b511c38ddb
• fix: prevent repeating interrupted turns (#9043)

  ## What
  Record a model-visible `<turn_aborted>` marker in history when a turn is
  interrupted, and treat it as a session prefix.
  
  ## Why
  When a turn is interrupted, Codex emits `TurnAborted` but previously did
  not persist anything model-visible in the conversation history. On the
  next user turn, the model can’t tell the previous work was aborted and
  may resume/repeat earlier actions (including duplicated side effects
  like re-opening PRs).
  
  Fixes: https://github.com/openai/codex/issues/9042
  
  ## How
  On `TurnAbortReason::Interrupted`, append a hidden user message
  containing a `<turn_aborted>…</turn_aborted>` marker and flush.
  Treat `<turn_aborted>` like `<environment_context>` for session-prefix
  filtering.
  Add a regression test to ensure follow-up turns don’t repeat side
  effects from an aborted turn.
  
  ## Testing
  `just fmt`
  `just fix -p codex-core`
  `cargo test -p codex-core -- --test-threads=1`
  `cargo test --all-features -- --test-threads=1`
  
  ---------
  
  Co-authored-by: Skylar Graika <sgraika127@gmail.com>
  Co-authored-by: jif-oai <jif@openai.com>
  Co-authored-by: Eric Traut <etraut@openai.com>

  Skylar Graika · 2026-01-20 13:07:28 -08:00
  b236f1c95d
• Persist text elements through TUI input and history (#9393)

  Continuation of breaking up this PR
  https://github.com/openai/codex/pull/9116
  
  ## Summary
  - Thread user text element ranges through TUI/TUI2 input, submission,
  queueing, and history so placeholders survive resume/edit flows.
  - Preserve local image attachments alongside text elements and rehydrate
  placeholders when restoring drafts.
  - Keep model-facing content shapes clean by attaching UI metadata only
  to user input/events (no API content changes).
  
  ## Key Changes
  - TUI/TUI2 composer now captures text element ranges, trims them with
  text edits, and restores them when submission is suppressed.
  - User history cells render styled spans for text elements and keep
  local image paths for future rehydration.
  - Initial chat widget bootstraps accept empty `initial_text_elements` to
  keep initialization uniform.
  - Protocol/core helpers updated to tolerate the new InputText field
  shape without changing payloads sent to the API.

  charley-oai · 2026-01-19 23:49:34 -08:00
  eb90e20c0b
• chore(core) Create instructions module (#9422)

  ## Summary
  We have a variety of things we refer to as instructions in the code
  base: our current canonical terms are:
  - base instructions (raw string)
  - developer instructions (has a type in protocol)
  - user instructions
  
  We also have `instructions` floating around in various places. We should
  standardize on the above, and start using types to prevent them from
  ending up in the wrong place. There will be additional PRs, but I'm
  going to keep these small so we can easily follow them!
  
  ## Testing
  - [x] Tests pass, this is purely a file move

  Dylan Hurd · 2026-01-17 16:01:26 -08:00
  bffe9b33e9
• Add text element metadata to types (#9235)

  Initial type tweaking PR to make the diff of
  https://github.com/openai/codex/pull/9116 smaller
  
  This should not change any behavior, just adds some fields to types

  charley-oai · 2026-01-14 16:41:50 -08:00
  4a9c2bcc5a
• Label attached images so agent can understand in-message labels (#8950)

  Agent wouldn't "see" attached images and would instead try to use the
  view_file tool:
  <img width="1516" height="504" alt="image"
  src="https://github.com/user-attachments/assets/68a705bb-f962-4fc1-9087-e932a6859b12"
  />
  
  In this PR, we wrap image content items in XML tags with the name of
  each image (now just a numbered name like `[Image #1]`), so that the
  model can understand inline image references (based on name). We also
  put the image content items above the user message which the model seems
  to prefer (maybe it's more used to definitions being before references).
  
  We also tweak the view_file tool description which seemed to help a bit
  
  Results on a simple eval set of images:
  
  Before
  <img width="980" height="310" alt="image"
  src="https://github.com/user-attachments/assets/ba838651-2565-4684-a12e-81a36641bf86"
  />
  
  After
  <img width="918" height="322" alt="image"
  src="https://github.com/user-attachments/assets/10a81951-7ee6-415e-a27e-e7a3fd0aee6f"
  />
  
  ```json
  [
    {
      "id": "single_describe",
      "prompt": "Describe the attached image in one sentence.",
      "images": ["image_a.png"]
    },
    {
      "id": "single_color",
      "prompt": "What is the dominant color in the image? Answer with a single color word.",
      "images": ["image_b.png"]
    },
    {
      "id": "orientation_check",
      "prompt": "Is the image portrait or landscape? Answer in one sentence.",
      "images": ["image_c.png"]
    },
    {
      "id": "detail_request",
      "prompt": "Look closely at the image and call out any small details you notice.",
      "images": ["image_d.png"]
    },
    {
      "id": "two_images_compare",
      "prompt": "I attached two images. Are they the same or different? Briefly explain.",
      "images": ["image_a.png", "image_b.png"]
    },
    {
      "id": "two_images_captions",
      "prompt": "Provide a short caption for each image (Image 1, Image 2).",
      "images": ["image_c.png", "image_d.png"]
    },
    {
      "id": "multi_image_rank",
      "prompt": "Rank the attached images from most colorful to least colorful.",
      "images": ["image_a.png", "image_b.png", "image_c.png"]
    },
    {
      "id": "multi_image_choice",
      "prompt": "Which image looks more vibrant? Answer with 'Image 1' or 'Image 2'.",
      "images": ["image_b.png", "image_d.png"]
    }
  ]
  ```

  charley-oai · 2026-01-09 21:33:45 -08:00
  6709ad8975
• Inject SKILL.md when it's explicitly mentioned. (#7763)

  1. Skills load once in core at session start; the cached outcome is
  reused across core and surfaced to TUI via SessionConfigured.
  2. TUI detects explicit skill selections, and core injects the matching
  SKILL.md content into the turn when a selected skill is present.

  xl-openai · 2025-12-10 13:59:17 -08:00
  b36ecb6c32
• Support all types of search actions (#7061)

  Fixes the 
  
  ```
  {
    "error": {
      "message": "Invalid value: 'other'. Supported values are: 'search', 'open_page', and 'find_in_page'.",
      "type": "invalid_request_error",
      "param": "input[150].action.type",
      "code": "invalid_value"
    }
  ```
  error.
  
  
  The actual-actual fix here is supporting absent `query` parameter.

  pakrym-oai · 2025-11-20 20:45:28 -08:00
  ab5972d447
• Add user command event types (#6246)

  adding new user command event, logic in TUI to render user command
  events

  zhao-oai · 2025-11-10 19:18:45 +00:00
  980886498c
• Update user instruction message format (#6010)
  pakrym-oai · 2025-10-30 18:44:02 -07:00
  2371d771cc
• Add item streaming events (#5546)

  Adds AgentMessageContentDelta, ReasoningContentDelta,
  ReasoningRawContentDelta item streaming events while maintaining
  compatibility for old events.
  
  ---------
  
  Co-authored-by: Owen Lin <owen@openai.com>

  pakrym-oai · 2025-10-29 22:33:57 +00:00
  3429e82e45
• Add new thread items and rewire event parsing to use them (#5418)

  1. Adds AgentMessage,  Reasoning,  WebSearch items.
  2. Switches the ResponseItem parsing to use new items and then also emit
  3. Removes user-item kind and filters out "special" (environment) user
  items when returning to clients.

  pakrym-oai · 2025-10-22 10:14:50 -07:00
  3c90728a29
• Use assert_matches (#4756)

  assert_matches is soon to be in std but is experimental for now.

  pakrym-oai · 2025-10-05 21:12:31 +00:00
  5c42419b02
• chore: enable clippy::redundant_clone (#3489)

  Created this PR by:
  
  - adding `redundant_clone` to `[workspace.lints.clippy]` in
  `cargo-rs/Cargol.toml`
  - running `cargo clippy --tests --fix`
  - running `just fmt`
  
  Though I had to clean up one instance of the following that resulted:
  
  ```rust
  let codex = codex;
  ```

  Michael Bolin · 2025-09-11 11:59:37 -07:00
  bec51f6c05
• Added images to UserMessageEvent (#3400)

  This PR adds an `images` field to the existing `UserMessageEvent` so we
  can encode zero or more images associated with a user message. This
  allows images to be restored when conversations are restored.

  Eric Traut · 2025-09-10 10:18:43 -07:00
  39db113cc9
• Dividing UserMsgs into categories to send it back to the tui (#3127)

  This PR does the following:
  
  - divides user msgs into 3 categories: plain, user instructions, and
  environment context
  - Centralizes adding user instructions and environment context to a
  degree
  - Improve the integration testing
  
  Building on top of #3123
  
  Specifically this
  [comment](https://github.com/openai/codex/pull/3123#discussion_r2319885089).
  We need to send the user message while ignoring the User Instructions
  and Environment Context we attach.

  Ahmed Ibrahim · 2025-09-04 05:34:50 +00:00
  2b96f9f569
• Replay EventMsgs from Response Items when resuming a session with history. (#3123)

  ### Overview
  
  This PR introduces the following changes:
  	1.	Adds a unified mechanism to convert ResponseItem into EventMsg.
  2. Ensures that when a session is initialized with initial history, a
  vector of EventMsg is sent along with the session configuration. This
  allows clients to re-render the UI accordingly.
  	3. 	Added integration testing
  
  ### Caveats
  
  This implementation does not send every EventMsg that was previously
  dispatched to clients. The excluded events fall into two categories:
  	•	“Arguably” rolled-out events
  Examples include tool calls and apply-patch calls. While these events
  are conceptually rolled out, we currently only roll out ResponseItems.
  These events are already being handled elsewhere and transformed into
  EventMsg before being sent.
  	•	Non-rolled-out events
  Certain events such as TurnDiff, Error, and TokenCount are not rolled
  out at all.
  
  ### Future Directions
  
  At present, resuming a session involves maintaining two states:
  	•	UI State
  Clients can replay most of the important UI from the provided EventMsg
  history.
  	•	Model State
  The model receives the complete session history to reconstruct its
  internal state.
  
  This design provides a solid foundation. If, in the future, more precise
  UI reconstruction is needed, we have two potential paths:
  1. Introduce a third data structure that allows us to derive both
  ResponseItems and EventMsgs.
  2. Clearly divide responsibilities: the core system ensures the
  integrity of the model state, while clients are responsible for
  reconstructing the UI.

  Ahmed Ibrahim · 2025-09-04 04:47:00 +00:00
  f2036572b6