24 Commits

• Enabling CWD Saving for Image-Gen (#13607)

  Codex now saves the generated image on to your current working
  directory.

  Won Park · 2026-03-06 00:47:21 -08:00
  ee1a20258a
• image-gen-event/client_processing (#13512)

  enabling client-side to process with image-generation capabilities
  (setting app-server)

  Won Park · 2026-03-04 16:54:38 -08:00
  229e6d0347
• core: bundle settings diff updates into one dev/user envelope (#12417)

  ## Summary
  - bundle contextual prompt injection into at most one developer message
  plus one contextual user message in both:
    - per-turn settings updates
    - initial context insertion
  - preserve `<model_switch>` across compaction by rebuilding it through
  canonical initial-context injection, instead of relying on
  strip/reattach hacks
  - centralize contextual user fragment detection in one shared definition
  table and reuse it for parsing/compaction logic
  - keep `AGENTS.md` in its natural serialized format:
    - `# AGENTS.md instructions for {dirname}`
    - `<INSTRUCTIONS>...</INSTRUCTIONS>`
  - simplify related tests/helpers and accept the expected snapshot/layout
  updates from bundled multi-part messages
  
  ## Why
  The goal is to converge toward a simpler, more intentional prompt shape
  where contextual updates are consistently represented as one developer
  envelope plus one contextual user envelope, while keeping parsing and
  compaction behavior aligned with that representation.
  
  ## Notable details
  - the temporary `SettingsUpdateEnvelope` wrapper was removed; these
  paths now return `Vec<ResponseItem>` directly
  - local/remote compaction no longer rely on model-switch strip/restore
  helpers
  - contextual user detection is now driven by shared fragment definitions
  instead of ad hoc matcher assembly
  - AGENTS/user instructions are still the same logical context; only the
  synthetic `<user_instructions>` wrapper was replaced by the natural
  AGENTS text format
  
  ## Testing
  - `just fmt`
  - `cargo test -p codex-app-server
  codex_message_processor::tests::extract_conversation_summary_prefers_plain_user_messages
  -- --exact`
  - `cargo test -p codex-core
  compact::tests::collect_user_messages_filters_session_prefix_entries
  --lib -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact::snapshot_request_shape_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::compact_remote::snapshot_request_shape_remote_pre_turn_compaction_strips_incoming_model_switch'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_apps_guidance_as_developer_message_when_enabled'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_developer_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::includes_user_instructions_message_in_request' --
  --exact`
  - `cargo test -p codex-core --test all
  'suite::client::resume_includes_initial_messages_and_sends_prior_items'
  -- --exact`
  - `cargo test -p codex-core --test all
  'suite::review::review_input_isolated_from_parent_history' -- --exact`
  - `cargo test -p codex-exec --test all
  'suite::resume::exec_resume_last_respects_cwd_filter_and_all_flag' --
  --exact`
  - `cargo test -p core_test_support
  context_snapshot::tests::full_text_mode_preserves_unredacted_text --
  --exact`
  
  ## Notes
  - I also ran several targeted `compact`, `compact_remote`,
  `prompt_caching`, `model_visible_layout`, and `event_mapping` tests
  while iterating on prompt-shape changes.
  - I have not claimed a clean full-workspace `cargo test` from this
  environment because local sandbox/resource conditions have previously
  produced unrelated failures in large workspace runs.

  Charley Cunningham · 2026-02-26 00:12:08 -08:00
  07aefffb1f
• fix(tui): conditionally restore status indicator using message phase (#10947)

  TLDR: use new message phase field emitted by preamble-supported models
  to determine whether an AgentMessage is mid-turn commentary. if so,
  restore the status indicator afterwards to indicate the turn has not
  completed.
  
  ### Problem
  `commit_tick` hides the status indicator while streaming assistant text.
  For preamble-capable models, that text can be commentary mid-turn, so
  hiding was correct during streaming but restore timing mattered:
  - restoring too aggressively caused jitter/flashing
  - not restoring caused indicator to stay hidden before subsequent work
  (tool calls, web search, etc.)
  
  ### Fix
  - Add optional `phase` to `AgentMessageItem` and propagate it from
  `ResponseItem::Message`
  - Keep indicator hidden during streamed commit ticks, restore only when:
    - assistant item completes as `phase=commentary`, and
    - stream queues are idle + task is still running.
  - Treat `phase=None` as final-answer behavior (no restore) to keep
  existing behavior for non-preamble models
  
  ### Tests
  Add/update tests for:
  - no idle-tick restore without commentary completion
  - commentary completion restoring status before tool begin
  - snapshot coverage for preamble/status behavior
  
  ---------
  
  Co-authored-by: Josh McKinney <joshka@openai.com>

  sayan-oai · 2026-02-07 02:39:52 +00:00
  5d2702f6b8
• chore: add phase to message responseitem (#10455)

  ### What
  
  add wiring for `phase` field on `ResponseItem::Message` to lay
  groundwork for differentiating model preambles and final messages.
  currently optional.
  
  follows pattern in #9698.
  
  updated schemas with `just write-app-server-schema` so we can see type
  changes.
  
  ### Tests
  Updated existing tests for SSE parsing and hydrating from history

  sayan-oai · 2026-02-03 02:52:26 +00:00
  fc05374344
• add missing fields to WebSearchAction and update app-server types (#10276)

  - add `WebSearchAction` to app-server v2 types
  - add `queries` to `WebSearchAction::Search` type
  
  Updated tests.

  sayan-oai · 2026-01-30 16:37:56 -08:00
  eb86663dcb
• fix: handle all web_search actions and in progress invocations (#9960)

  ### Summary
  - Parse all `web_search` tool actions (`search`, `find_in_page`,
  `open_page`).
  - Previously we only parsed + displayed `search`, which made the TUI
  appear to pause when the other actions were being used.
  - Show in progress `web_search` calls as `Searching the web`
    - Previously we only showed completed tool calls
  
  <img width="308" height="149" alt="image"
  src="https://github.com/user-attachments/assets/90a4e8ff-b06a-48ff-a282-b57b31121845"
  />
  
  ### Tests
  Added + updated tests, tested locally
  
  ### Follow ups
  Update VSCode extension to display these as well

  sayan-oai · 2026-01-27 03:33:48 +00:00
  86adf53235
• Support end_turn flag (#9698)

  Experimental flag that signals the end of the turn.

  pakrym-oai · 2026-01-22 17:27:48 +00:00
  b511c38ddb
• fix: prevent repeating interrupted turns (#9043)

  ## What
  Record a model-visible `<turn_aborted>` marker in history when a turn is
  interrupted, and treat it as a session prefix.
  
  ## Why
  When a turn is interrupted, Codex emits `TurnAborted` but previously did
  not persist anything model-visible in the conversation history. On the
  next user turn, the model can’t tell the previous work was aborted and
  may resume/repeat earlier actions (including duplicated side effects
  like re-opening PRs).
  
  Fixes: https://github.com/openai/codex/issues/9042
  
  ## How
  On `TurnAbortReason::Interrupted`, append a hidden user message
  containing a `<turn_aborted>…</turn_aborted>` marker and flush.
  Treat `<turn_aborted>` like `<environment_context>` for session-prefix
  filtering.
  Add a regression test to ensure follow-up turns don’t repeat side
  effects from an aborted turn.
  
  ## Testing
  `just fmt`
  `just fix -p codex-core`
  `cargo test -p codex-core -- --test-threads=1`
  `cargo test --all-features -- --test-threads=1`
  
  ---------
  
  Co-authored-by: Skylar Graika <sgraika127@gmail.com>
  Co-authored-by: jif-oai <jif@openai.com>
  Co-authored-by: Eric Traut <etraut@openai.com>

  Skylar Graika · 2026-01-20 13:07:28 -08:00
  b236f1c95d
• Persist text elements through TUI input and history (#9393)

  Continuation of breaking up this PR
  https://github.com/openai/codex/pull/9116
  
  ## Summary
  - Thread user text element ranges through TUI/TUI2 input, submission,
  queueing, and history so placeholders survive resume/edit flows.
  - Preserve local image attachments alongside text elements and rehydrate
  placeholders when restoring drafts.
  - Keep model-facing content shapes clean by attaching UI metadata only
  to user input/events (no API content changes).
  
  ## Key Changes
  - TUI/TUI2 composer now captures text element ranges, trims them with
  text edits, and restores them when submission is suppressed.
  - User history cells render styled spans for text elements and keep
  local image paths for future rehydration.
  - Initial chat widget bootstraps accept empty `initial_text_elements` to
  keep initialization uniform.
  - Protocol/core helpers updated to tolerate the new InputText field
  shape without changing payloads sent to the API.

  charley-oai · 2026-01-19 23:49:34 -08:00
  eb90e20c0b
• chore(core) Create instructions module (#9422)

  ## Summary
  We have a variety of things we refer to as instructions in the code
  base: our current canonical terms are:
  - base instructions (raw string)
  - developer instructions (has a type in protocol)
  - user instructions
  
  We also have `instructions` floating around in various places. We should
  standardize on the above, and start using types to prevent them from
  ending up in the wrong place. There will be additional PRs, but I'm
  going to keep these small so we can easily follow them!
  
  ## Testing
  - [x] Tests pass, this is purely a file move

  Dylan Hurd · 2026-01-17 16:01:26 -08:00
  bffe9b33e9
• Add text element metadata to types (#9235)

  Initial type tweaking PR to make the diff of
  https://github.com/openai/codex/pull/9116 smaller
  
  This should not change any behavior, just adds some fields to types

  charley-oai · 2026-01-14 16:41:50 -08:00
  4a9c2bcc5a
• Label attached images so agent can understand in-message labels (#8950)

  Agent wouldn't "see" attached images and would instead try to use the
  view_file tool:
  <img width="1516" height="504" alt="image"
  src="https://github.com/user-attachments/assets/68a705bb-f962-4fc1-9087-e932a6859b12"
  />
  
  In this PR, we wrap image content items in XML tags with the name of
  each image (now just a numbered name like `[Image #1]`), so that the
  model can understand inline image references (based on name). We also
  put the image content items above the user message which the model seems
  to prefer (maybe it's more used to definitions being before references).
  
  We also tweak the view_file tool description which seemed to help a bit
  
  Results on a simple eval set of images:
  
  Before
  <img width="980" height="310" alt="image"
  src="https://github.com/user-attachments/assets/ba838651-2565-4684-a12e-81a36641bf86"
  />
  
  After
  <img width="918" height="322" alt="image"
  src="https://github.com/user-attachments/assets/10a81951-7ee6-415e-a27e-e7a3fd0aee6f"
  />
  
  ```json
  [
    {
      "id": "single_describe",
      "prompt": "Describe the attached image in one sentence.",
      "images": ["image_a.png"]
    },
    {
      "id": "single_color",
      "prompt": "What is the dominant color in the image? Answer with a single color word.",
      "images": ["image_b.png"]
    },
    {
      "id": "orientation_check",
      "prompt": "Is the image portrait or landscape? Answer in one sentence.",
      "images": ["image_c.png"]
    },
    {
      "id": "detail_request",
      "prompt": "Look closely at the image and call out any small details you notice.",
      "images": ["image_d.png"]
    },
    {
      "id": "two_images_compare",
      "prompt": "I attached two images. Are they the same or different? Briefly explain.",
      "images": ["image_a.png", "image_b.png"]
    },
    {
      "id": "two_images_captions",
      "prompt": "Provide a short caption for each image (Image 1, Image 2).",
      "images": ["image_c.png", "image_d.png"]
    },
    {
      "id": "multi_image_rank",
      "prompt": "Rank the attached images from most colorful to least colorful.",
      "images": ["image_a.png", "image_b.png", "image_c.png"]
    },
    {
      "id": "multi_image_choice",
      "prompt": "Which image looks more vibrant? Answer with 'Image 1' or 'Image 2'.",
      "images": ["image_b.png", "image_d.png"]
    }
  ]
  ```

  charley-oai · 2026-01-09 21:33:45 -08:00
  6709ad8975
• Inject SKILL.md when it's explicitly mentioned. (#7763)

  1. Skills load once in core at session start; the cached outcome is
  reused across core and surfaced to TUI via SessionConfigured.
  2. TUI detects explicit skill selections, and core injects the matching
  SKILL.md content into the turn when a selected skill is present.

  xl-openai · 2025-12-10 13:59:17 -08:00
  b36ecb6c32
• Support all types of search actions (#7061)

  Fixes the 
  
  ```
  {
    "error": {
      "message": "Invalid value: 'other'. Supported values are: 'search', 'open_page', and 'find_in_page'.",
      "type": "invalid_request_error",
      "param": "input[150].action.type",
      "code": "invalid_value"
    }
  ```
  error.
  
  
  The actual-actual fix here is supporting absent `query` parameter.

  pakrym-oai · 2025-11-20 20:45:28 -08:00
  ab5972d447
• Add user command event types (#6246)

  adding new user command event, logic in TUI to render user command
  events

  zhao-oai · 2025-11-10 19:18:45 +00:00
  980886498c
• Update user instruction message format (#6010)
  pakrym-oai · 2025-10-30 18:44:02 -07:00
  2371d771cc
• Add item streaming events (#5546)

  Adds AgentMessageContentDelta, ReasoningContentDelta,
  ReasoningRawContentDelta item streaming events while maintaining
  compatibility for old events.
  
  ---------
  
  Co-authored-by: Owen Lin <owen@openai.com>

  pakrym-oai · 2025-10-29 22:33:57 +00:00
  3429e82e45
• Add new thread items and rewire event parsing to use them (#5418)

  1. Adds AgentMessage,  Reasoning,  WebSearch items.
  2. Switches the ResponseItem parsing to use new items and then also emit
  3. Removes user-item kind and filters out "special" (environment) user
  items when returning to clients.

  pakrym-oai · 2025-10-22 10:14:50 -07:00
  3c90728a29
• Use assert_matches (#4756)

  assert_matches is soon to be in std but is experimental for now.

  pakrym-oai · 2025-10-05 21:12:31 +00:00
  5c42419b02
• chore: enable clippy::redundant_clone (#3489)

  Created this PR by:
  
  - adding `redundant_clone` to `[workspace.lints.clippy]` in
  `cargo-rs/Cargol.toml`
  - running `cargo clippy --tests --fix`
  - running `just fmt`
  
  Though I had to clean up one instance of the following that resulted:
  
  ```rust
  let codex = codex;
  ```

  Michael Bolin · 2025-09-11 11:59:37 -07:00
  bec51f6c05
• Added images to UserMessageEvent (#3400)

  This PR adds an `images` field to the existing `UserMessageEvent` so we
  can encode zero or more images associated with a user message. This
  allows images to be restored when conversations are restored.

  Eric Traut · 2025-09-10 10:18:43 -07:00
  39db113cc9
• Dividing UserMsgs into categories to send it back to the tui (#3127)

  This PR does the following:
  
  - divides user msgs into 3 categories: plain, user instructions, and
  environment context
  - Centralizes adding user instructions and environment context to a
  degree
  - Improve the integration testing
  
  Building on top of #3123
  
  Specifically this
  [comment](https://github.com/openai/codex/pull/3123#discussion_r2319885089).
  We need to send the user message while ignoring the User Instructions
  and Environment Context we attach.

  Ahmed Ibrahim · 2025-09-04 05:34:50 +00:00
  2b96f9f569
• Replay EventMsgs from Response Items when resuming a session with history. (#3123)

  ### Overview
  
  This PR introduces the following changes:
  	1.	Adds a unified mechanism to convert ResponseItem into EventMsg.
  2. Ensures that when a session is initialized with initial history, a
  vector of EventMsg is sent along with the session configuration. This
  allows clients to re-render the UI accordingly.
  	3. 	Added integration testing
  
  ### Caveats
  
  This implementation does not send every EventMsg that was previously
  dispatched to clients. The excluded events fall into two categories:
  	•	“Arguably” rolled-out events
  Examples include tool calls and apply-patch calls. While these events
  are conceptually rolled out, we currently only roll out ResponseItems.
  These events are already being handled elsewhere and transformed into
  EventMsg before being sent.
  	•	Non-rolled-out events
  Certain events such as TurnDiff, Error, and TokenCount are not rolled
  out at all.
  
  ### Future Directions
  
  At present, resuming a session involves maintaining two states:
  	•	UI State
  Clients can replay most of the important UI from the provided EventMsg
  history.
  	•	Model State
  The model receives the complete session history to reconstruct its
  internal state.
  
  This design provides a solid foundation. If, in the future, more precise
  UI reconstruction is needed, we have two potential paths:
  1. Introduce a third data structure that allows us to derive both
  ResponseItems and EventMsgs.
  2. Clearly divide responsibilities: the core system ensures the
  integrity of the model state, while clients are responsible for
  reconstructing the UI.

  Ahmed Ibrahim · 2025-09-04 04:47:00 +00:00
  f2036572b6