42 Commits

• fix: handle all web_search actions and in progress invocations (#9960)

  ### Summary
  - Parse all `web_search` tool actions (`search`, `find_in_page`,
  `open_page`).
  - Previously we only parsed + displayed `search`, which made the TUI
  appear to pause when the other actions were being used.
  - Show in progress `web_search` calls as `Searching the web`
    - Previously we only showed completed tool calls
  
  <img width="308" height="149" alt="image"
  src="https://github.com/user-attachments/assets/90a4e8ff-b06a-48ff-a282-b57b31121845"
  />
  
  ### Tests
  Added + updated tests, tested locally
  
  ### Follow ups
  Update VSCode extension to display these as well

  sayan-oai · 2026-01-27 03:33:48 +00:00
  86adf53235
• feat: support proxy for ws connection (#9719)

  reapply websocket changes without changing tls lib.

  Anton Panasenko · 2026-01-22 15:23:15 -08:00
  e117a3ff33
• feat(core) update Personality on turn (#9644)

  ## Summary
  Support updating Personality mid-Thread via UserTurn/OverwriteTurn. This
  is explicitly unused by the clients so far, to simplify PRs - app-server
  and tui implementations will be follow-ups.
  
  ## Testing
  - [x] added integration tests

  Dylan Hurd · 2026-01-22 12:04:23 -08:00
  8b3521ee77
• Revert "feat: support proxy for ws connection" (#9693)

  Reverts openai/codex#9409

  pakrym-oai · 2026-01-22 15:57:18 +00:00
  4d48d4e0c2
• feat: support proxy for ws connection (#9409)

  unfortunately tokio-tungstenite doesn't support proxy configuration
  outbox, while https://github.com/snapview/tokio-tungstenite/pull/370 is
  in review, we can depend on source code for now.

  Anton Panasenko · 2026-01-20 09:36:30 -08:00
  7b27aa7707
• Turn-state sticky routing per turn (#9332)

  - capture the header from SSE/WS handshakes, store it per
  ModelClientSession using `Oncelock`, echo it on turn-scoped requests,
  and add SSE+WS integration tests for within-turn persistence +
  cross-turn reset.
  
  - keep `x-codex-turn-state` sticky within a user turn to maintain
  routing continuity for retries/tool follow-ups.

  Ahmed Ibrahim · 2026-01-16 09:30:11 -08:00
  ebdd8795e9
• Fix flakiness in WebSocket tests (#9169)

  The connection was being added to the list after the WebSocket response
  was sent.
  
  So the test can sometimes race and observe connections before the list
  was updated.
  
  After this change, connection and request is added to the list before
  the response is sent.

  pakrym-oai · 2026-01-13 15:09:59 -08:00
  9f8d3c14ce
• Support response.done and add integration tests (#9129)

  The agent loop using a persistent incremental web socket connection.

  pakrym-oai · 2026-01-13 16:12:30 +00:00
  2d56519ecd
• Add model client sessions (#9102)

  Maintain a long-running session.

  pakrym-oai · 2026-01-13 01:15:56 +00:00
  490c1c1fdd
• Add 5s timeout to models list call + integration test (#8942)

  - Enforce a 5s timeout around the remote models refresh to avoid hanging
  /models calls.

  Ahmed Ibrahim · 2026-01-08 18:06:10 -08:00
  81caee3400
• remove get_responses_requests and get_responses_request_bodies to use in-place matcher (#8858)
  Ahmed Ibrahim · 2026-01-08 13:57:48 -08:00
  0d3e673019
• Refresh on models etag mismatch (#8491)

  - Send models etag
  - Refresh models on 412
  - This wires `ModelsManager` to `ModelFamily` so we don't mutate it
  mid-turn

  Ahmed Ibrahim · 2026-01-01 11:41:16 -08:00
  66b7c673e9
• make model optional in config (#7769)

  - Make Config.model optional and centralize default-selection logic in
  ModelsManager, including a default_model helper (with
  codex-auto-balanced when available) so sessions now carry an explicit
  chosen model separate from the base config.
  - Resolve `model` once in `core` and `tui` from config. Then store the
  state of it on other structs.
  - Move refreshing models to be before resolving the default model

  Ahmed Ibrahim · 2025-12-10 11:19:00 -08:00
  cb9a189857
• load models from disk and set a ttl and etag (#7722)

  # External (non-OpenAI) Pull Request Requirements
  
  Before opening this Pull Request, please read the dedicated
  "Contributing" markdown file or your PR may be closed:
  https://github.com/openai/codex/blob/main/docs/contributing.md
  
  If your PR conforms to our contribution guidelines, replace this text
  with a detailed and high quality description of your changes.
  
  Include a link to a bug report or enhancement request.

  Ahmed Ibrahim · 2025-12-08 13:43:04 -08:00
  222a491570
• Add models endpoint (#7603)

  - Use the codex-api crate to introduce models endpoint. 
  - Add `models` to codex core tests helpers
  - Add `ModelsInfo` for the endpoint return type

  Ahmed Ibrahim · 2025-12-04 12:57:54 -08:00
  903b7774bc
• feat: retroactive image placeholder to prevent poisoning (#6774)

  If an image can't be read by the API, it will poison the entire history,
  preventing any new turn on the conversation.
  This detect such cases and replace the image by a placeholder

  jif-oai · 2025-12-03 11:35:56 +00:00
  51307eaf07
• fix(apply_patch) tests for shell_command (#7307)

  ## Summary
  Adds test coverage for invocations of apply_patch via shell_command with
  heredoc, to validate behavior.
  
  ## Testing
  - [x] These are tests

  Dylan Hurd · 2025-12-01 15:09:22 -08:00
  5b25915d7e
• Account for encrypted reasoning for auto compaction (#7113)

  - The total token used returned from the api doesn't account for the
  reasoning items before the assistant message
  - Account for those for auto compaction
  - Add the encrypted reasoning effort in the common tests utils
  - Add a test to make sure it works as expected

  Ahmed Ibrahim · 2025-11-22 03:06:45 +00:00
  b519267d05
• Migrate coverage to shell_command (#7042)
  pakrym-oai · 2025-11-21 03:44:00 +00:00
  767b66f407
• Run remote auto compaction (#6879)
  pakrym-oai · 2025-11-19 00:43:58 -08:00
  75f38f16dd
• feat: remote compaction (#6795)

  Co-authored-by: pakrym-oai <pakrym@openai.com>

  jif-oai · 2025-11-18 16:51:16 +00:00
  838531d3e4
• tests: replace mount_sse_once_match with mount_sse_once for SSE mocking (#6640)
  pakrym-oai · 2025-11-13 18:04:05 -08:00
  6c384eb9c6
• Promote shared helpers for suite tests (#6460)

  ## Summary
  - add `TestCodex::submit_turn_with_policies` and extend the response
  helpers with reusable tool-call utilities
  - update the grep_files, read_file, list_dir, shell_serialization, and
  tools suites to rely on the shared helpers instead of local copies
  - make the list_dir helper return `anyhow::Result` so clippy no longer
  warns about `expect`
  
  ## Testing
  - `just fix -p codex-core`
  - `cargo test -p codex-core --test all
  suite::grep_files::grep_files_tool_collects_matches`
  - `cargo test -p codex-core
  suite::grep_files::grep_files_tool_collects_matches -- --ignored`
  (filter requests ignored tests so nothing runs, but the build stays
  clean)
  
  
  ------
  [Codex
  Task](https://chatgpt.com/codex/tasks/task_i_69112d53abac83219813cab4d7cb6446)

  Ahmed Ibrahim · 2025-11-13 17:12:10 -08:00
  2a6e9b20df
• [App-server] add new v2 events:item/reasoning/delta, item/agentMessage/delta & item/reasoning/summaryPartAdded (#6559)

  core event to app server event mapping:
  1. `codex/event/reasoning_content_delta` ->
  `item/reasoning/summaryTextDelta`.
  2. `codex/event/reasoning_raw_content_delta` ->
  `item/reasoning/textDelta`
  3. `codex/event/agent_message_content_delta` →
  `item/agentMessage/delta`.
  4. `codex/event/agent_reasoning_section_break` ->
  `item/reasoning/summaryPartAdded`.
  
  Also added a change in core to pass down content index, summary index
  and item id from events.
  
  Tested with the `git checkout owen/app_server_test_client && cargo run
  -p codex-app-server-test-client -- send-message-v2 "hello"` and verified
  that new events are emitted correctly.

  Celia Chen · 2025-11-14 00:25:01 +00:00
  b8ec97c0ef
• chore(core) Consolidate apply_patch tests (#6545)

  ## Summary
  Consolidates our apply_patch tests into one suite, and ensures each test
  case tests the various ways the harness supports apply_patch:
  1. Freeform custom tool call
  2. JSON function tool
  3. Simple shell call
  4. Heredoc shell call
  
  There are a few test cases that are specific to a particular variant,
  I've left those alone.
  
  ## Testing
  - [x] This adds a significant number of tests

  Dylan Hurd · 2025-11-13 15:52:39 -08:00
  2c1b693da4
• Fix otel tests (#6541)

  Mount responses only once, remove unneeded retries and add a final
  assistant messages to complete the turn.

  pakrym-oai · 2025-11-12 16:35:34 +00:00
  7d9ad3effd
• Add user command event types (#6246)

  adding new user command event, logic in TUI to render user command
  events

  zhao-oai · 2025-11-10 19:18:45 +00:00
  980886498c
• test: undo (#6034)
  jif-oai · 2025-10-31 14:46:24 +00:00
  0508823075
• Add item streaming events (#5546)

  Adds AgentMessageContentDelta, ReasoningContentDelta,
  ReasoningRawContentDelta item streaming events while maintaining
  compatibility for old events.
  
  ---------
  
  Co-authored-by: Owen Lin <owen@openai.com>

  pakrym-oai · 2025-10-29 22:33:57 +00:00
  3429e82e45
• Filter out reasoning items from previous turns (#5857)

  Reduces request size and prevents 400 errors when switching between API
  orgs.
  
  Based on Responses API behavior described in
  https://cookbook.openai.com/examples/responses_api/reasoning_items#caching

  pakrym-oai · 2025-10-28 11:39:34 -07:00
  1b8f2543ac
• Handle cancelling/aborting while processing a turn (#5543)

  Currently we collect all all turn items in a vector, then we add it to
  the history on success. This result in losing those items on errors
  including aborting `ctrl+c`.
  
  This PR:
  - Adds the ability for the tool call to handle cancellation
  - bubble the turn items up to where we are recording this info
  
  Admittedly, this logic is an ad-hoc logic that doesn't handle a lot of
  error edge cases. The right thing to do is recording to the history on
  the spot as `items`/`tool calls output` come. However, this isn't
  possible because of having different `task_kind` that has different
  `conversation_histories`. The `try_run_turn` has no idea what thread are
  we using. We cannot also pass an `arc` to the `conversation_histories`
  because it's a private element of `state`.
  
  That's said, `abort` is the most common case and we should cover it
  until we remove `task kind`

  Ahmed Ibrahim · 2025-10-23 08:47:10 -07:00
  f59978ed3d
• Move changing turn input functionalities to ConversationHistory (#5473)

  We are doing some ad-hoc logic while dealing with conversation history.
  Ideally, we shouldn't mutate `vec[responseitem]` manually at all and
  should depend on `ConversationHistory` for those changes.
  
  Those changes are:
  - Adding input to the history
  - Removing items from the history
  - Correcting history
  
  I am also adding some `error` logs for cases we shouldn't ideally face.
  For example, we shouldn't be missing `toolcalls` or `outputs`. We
  shouldn't hit `ContextWindowExceeded` while performing `compact`
  
  This refactor will give us granular control over our context management.

  Ahmed Ibrahim · 2025-10-22 13:08:46 -07:00
  273819aaae
• Add new thread items and rewire event parsing to use them (#5418)

  1. Adds AgentMessage,  Reasoning,  WebSearch items.
  2. Switches the ResponseItem parsing to use new items and then also emit
  3. Removes user-item kind and filters out "special" (environment) user
  items when returning to clients.

  pakrym-oai · 2025-10-22 10:14:50 -07:00
  3c90728a29
• Auto compact at ~90% (#5292)

  Users now hit a window exceeded limit and they usually don't know what
  to do. This starts auto compact at ~90% of the window.

  Ahmed Ibrahim · 2025-10-20 11:29:49 -07:00
  049a61bcfc
• Simplify request body assertions (#4845)

  We'll have a lot more test like these

  pakrym-oai · 2025-10-07 09:56:39 +01:00
  35a770e871
• Add helper for response created SSE events in tests (#4758)

  ## Summary
  - add a reusable `ev_response_created` helper that builds
  `response.created` SSE events for integration tests
  - update the exec and core integration suites to use the new helper
  instead of repeating manual JSON literals
  - keep the streaming fixtures consistent by relying on the shared helper
  in every touched test
  
  ## Testing
  - `just fmt`
  
  
  ------
  https://chatgpt.com/codex/tasks/task_i_68e1fe885bb883208aafffb94218da61

  pakrym-oai · 2025-10-05 21:11:43 +00:00
  aecbe0f333
• Surface context window error to the client (#4675)

  In the past, we were treating `input exceeded context window` as a
  streaming error and retrying on it. Retrying on it has no point because
  it won't change the behavior. In this PR, we surface the error to the
  client without retry and also send a token count event to indicate that
  the context window is full.
  
  <img width="650" height="125" alt="image"
  src="https://github.com/user-attachments/assets/c26b1213-4c27-4bfc-90f4-51a270a3efd5"
  />

  Ahmed Ibrahim · 2025-10-05 01:40:06 +00:00
  90ef94d3b3
• OpenTelemetry events (#2103)

  ### Title
  
  ## otel
  
  Codex can emit [OpenTelemetry](https://opentelemetry.io/) **log events**
  that
  describe each run: outbound API requests, streamed responses, user
  input,
  tool-approval decisions, and the result of every tool invocation. Export
  is
  **disabled by default** so local runs remain self-contained. Opt in by
  adding an
  `[otel]` table and choosing an exporter.
  
  ```toml
  [otel]
  environment = "staging"   # defaults to "dev"
  exporter = "none"          # defaults to "none"; set to otlp-http or otlp-grpc to send events
  log_user_prompt = false    # defaults to false; redact prompt text unless explicitly enabled
  ```
  
  Codex tags every exported event with `service.name = "codex-cli"`, the
  CLI
  version, and an `env` attribute so downstream collectors can distinguish
  dev/staging/prod traffic. Only telemetry produced inside the
  `codex_otel`
  crate—the events listed below—is forwarded to the exporter.
  
  ### Event catalog
  
  Every event shares a common set of metadata fields: `event.timestamp`,
  `conversation.id`, `app.version`, `auth_mode` (when available),
  `user.account_id` (when available), `terminal.type`, `model`, and
  `slug`.
  
  With OTEL enabled Codex emits the following event types (in addition to
  the
  metadata above):
  
  - `codex.api_request`
    - `cf_ray` (optional)
    - `attempt`
    - `duration_ms`
    - `http.response.status_code` (optional)
    - `error.message` (failures)
  - `codex.sse_event`
    - `event.kind`
    - `duration_ms`
    - `error.message` (failures)
    - `input_token_count` (completion only)
    - `output_token_count` (completion only)
    - `cached_token_count` (completion only, optional)
    - `reasoning_token_count` (completion only, optional)
    - `tool_token_count` (completion only)
  - `codex.user_prompt`
    - `prompt_length`
    - `prompt` (redacted unless `log_user_prompt = true`)
  - `codex.tool_decision`
    - `tool_name`
    - `call_id`
  - `decision` (`approved`, `approved_for_session`, `denied`, or `abort`)
    - `source` (`config` or `user`)
  - `codex.tool_result`
    - `tool_name`
    - `call_id`
    - `arguments`
    - `duration_ms` (execution time for the tool)
    - `success` (`"true"` or `"false"`)
    - `output`
  
  ### Choosing an exporter
  
  Set `otel.exporter` to control where events go:
  
  - `none` – leaves instrumentation active but skips exporting. This is
  the
    default.
  - `otlp-http` – posts OTLP log records to an OTLP/HTTP collector.
  Specify the
    endpoint, protocol, and headers your collector expects:
  
    ```toml
    [otel]
    exporter = { otlp-http = {
      endpoint = "https://otel.example.com/v1/logs",
      protocol = "binary",
      headers = { "x-otlp-api-key" = "${OTLP_TOKEN}" }
    }}
    ```
  
  - `otlp-grpc` – streams OTLP log records over gRPC. Provide the endpoint
  and any
    metadata headers:
  
    ```toml
    [otel]
    exporter = { otlp-grpc = {
      endpoint = "https://otel.example.com:4317",
      headers = { "x-otlp-meta" = "abc123" }
    }}
    ```
  
  If the exporter is `none` nothing is written anywhere; otherwise you
  must run or point to your
  own collector. All exporters run on a background batch worker that is
  flushed on
  shutdown.
  
  If you build Codex from source the OTEL crate is still behind an `otel`
  feature
  flag; the official prebuilt binaries ship with the feature enabled. When
  the
  feature is disabled the telemetry hooks become no-ops so the CLI
  continues to
  function without the extra dependencies.
  
  ---------
  
  Co-authored-by: Anton Panasenko <apanasenko@openai.com>

  vishnu-oai · 2025-09-29 11:30:55 -07:00
  04c1782e52
• Actually mount sse once (#4264)

  Mock server was responding with the same result many times.

  pakrym-oai · 2025-09-26 01:17:51 +00:00
  a10403d697
• Add codex exec testing helpers (#4254)

  Add a shortcut to create working directories and run codex exec with
  fake server.

  pakrym-oai · 2025-09-25 17:12:45 -07:00
  8e3a048fec
• Use helpers instead of fixtures (#3888)

  Move to using test helper method everywhere.

  pakrym-oai · 2025-09-19 06:46:25 -07:00
  9b18875a42
• Move responses mocking helpers to a shared lib (#3878)

  These are generally useful

  pakrym-oai · 2025-09-18 17:53:14 -07:00
  881c7978f1