63 Commits

• feat: add shell snapshot for shell command (#7786)
  jif-oai · 2025-12-11 13:46:43 +00:00
  29381ba5c2
• Inject SKILL.md when it's explicitly mentioned. (#7763)

  1. Skills load once in core at session start; the cached outcome is
  reused across core and surfaced to TUI via SessionConfigured.
  2. TUI detects explicit skill selections, and core injects the matching
  SKILL.md content into the turn when a selected skill is present.

  xl-openai · 2025-12-10 13:59:17 -08:00
  b36ecb6c32
• make model optional in config (#7769)

  - Make Config.model optional and centralize default-selection logic in
  ModelsManager, including a default_model helper (with
  codex-auto-balanced when available) so sessions now carry an explicit
  chosen model separate from the base config.
  - Resolve `model` once in `core` and `tui` from config. Then store the
  state of it on other structs.
  - Move refreshing models to be before resolving the default model

  Ahmed Ibrahim · 2025-12-10 11:19:00 -08:00
  cb9a189857
• load models from disk and set a ttl and etag (#7722)

  # External (non-OpenAI) Pull Request Requirements
  
  Before opening this Pull Request, please read the dedicated
  "Contributing" markdown file or your PR may be closed:
  https://github.com/openai/codex/blob/main/docs/contributing.md
  
  If your PR conforms to our contribution guidelines, replace this text
  with a detailed and high quality description of your changes.
  
  Include a link to a bug report or enhancement request.

  Ahmed Ibrahim · 2025-12-08 13:43:04 -08:00
  222a491570
• Fix unified_exec on windows (#7620)

  Fix unified_exec on windows
  
  Requires removal of PSUEDOCONSOLE_INHERIT_CURSOR flag so child processed
  don't attempt to wait for cursor position response (and timeout).
  
  
  https://github.com/wezterm/wezterm/compare/main...pakrym:wezterm:PSUEDOCONSOLE_INHERIT_CURSOR?expand=1
  
  ---------
  
  Co-authored-by: pakrym-oai <pakrym@openai.com>

  Pavel Krymets · 2025-12-05 20:09:43 +00:00
  f48d88067e
• feat(core) Add login to shell_command tool (#6846)

  ## Summary
  Adds the `login` parameter to the `shell_command` tool - optional,
  defaults to true.
  
  ## Testing
  - [x] Tested locally

  Dylan Hurd · 2025-12-05 11:03:25 -08:00
  a8cbbdbc6e
• Wire with_remote_overrides to construct model families (#7621)

  - This PR wires `with_remote_overrides` and make the
  `construct_model_families` an async function
  - Moves getting model family a level above to keep the function `sync`
  - Updates the tests to local, offline, and `sync` helper for model
  families

  Ahmed Ibrahim · 2025-12-05 10:40:15 -08:00
  d08efb1743
• Add models endpoint (#7603)

  - Use the codex-api crate to introduce models endpoint. 
  - Add `models` to codex core tests helpers
  - Add `ModelsInfo` for the endpoint return type

  Ahmed Ibrahim · 2025-12-04 12:57:54 -08:00
  903b7774bc
• remove model_family from `config (#7571)

  - Remove `model_family` from `config`
  - Make sure to still override config elements related to `model_family`
  like supporting reasoning

  Ahmed Ibrahim · 2025-12-04 11:57:58 -08:00
  9b2055586d
• Migrate model family to models manager (#7565)

  This PR moves `ModelsFamily` to `openai_models`. It also propagates
  `ModelsManager` to session services and use it to drive model family. We
  also make `derive_default_model_family` private because it's a step
  towards what we want: one place that gives model configuration.
  
  This is a second step at having one source of truth for models
  information and config: `ModelsManager`.
  
  Next steps would be to remove `ModelsFamily` from config. That's massive
  because it's being used in 41 occasions mostly pre launching `codex`.
  Also, we need to make `find_family_for_model` private. It's also big
  because it's being used in 21 occasions ~ all tests.

  Ahmed Ibrahim · 2025-12-03 18:49:47 -08:00
  cee37a32b2
• feat: retroactive image placeholder to prevent poisoning (#6774)

  If an image can't be read by the API, it will poison the entire history,
  preventing any new turn on the conversation.
  This detect such cases and replace the image by a placeholder

  jif-oai · 2025-12-03 11:35:56 +00:00
  51307eaf07
• fix(apply_patch) tests for shell_command (#7307)

  ## Summary
  Adds test coverage for invocations of apply_patch via shell_command with
  heredoc, to validate behavior.
  
  ## Testing
  - [x] These are tests

  Dylan Hurd · 2025-12-01 15:09:22 -08:00
  5b25915d7e
• chore: add cargo-deny configuration (#7119)

  - add GitHub workflow running cargo-deny on push/PR
  - document cargo-deny allowlist with workspace-dep notes and advisory
  ignores
  - align workspace crates to inherit version/edition/license for
  consistent checks

  Josh McKinney · 2025-11-24 12:22:18 -08:00
  ec49b56874
• Account for encrypted reasoning for auto compaction (#7113)

  - The total token used returned from the api doesn't account for the
  reasoning items before the assistant message
  - Account for those for auto compaction
  - Add the encrypted reasoning effort in the common tests utils
  - Add a test to make sure it works as expected

  Ahmed Ibrahim · 2025-11-22 03:06:45 +00:00
  b519267d05
• Migrate coverage to shell_command (#7042)
  pakrym-oai · 2025-11-21 03:44:00 +00:00
  767b66f407
• Run remote auto compaction (#6879)
  pakrym-oai · 2025-11-19 00:43:58 -08:00
  75f38f16dd
• feat: remote compaction (#6795)

  Co-authored-by: pakrym-oai <pakrym@openai.com>

  jif-oai · 2025-11-18 16:51:16 +00:00
  838531d3e4
• chore(core) Add shell_serialization coverage (#6810)

  ## Summary
  Similar to #6545, this PR updates the shell_serialization test suite to
  cover the various `shell` tool invocations we have. Note that this does
  not cover unified_exec, which has its own suite of tests. This should
  provide some test coverage for when we eventually consolidate
  serialization logic.
  
  ## Testing
  - [x] These are tests

  Dylan Hurd · 2025-11-17 19:10:56 -08:00
  2b7378ac77
• tests: replace mount_sse_once_match with mount_sse_once for SSE mocking (#6640)
  pakrym-oai · 2025-11-13 18:04:05 -08:00
  6c384eb9c6
• Promote shared helpers for suite tests (#6460)

  ## Summary
  - add `TestCodex::submit_turn_with_policies` and extend the response
  helpers with reusable tool-call utilities
  - update the grep_files, read_file, list_dir, shell_serialization, and
  tools suites to rely on the shared helpers instead of local copies
  - make the list_dir helper return `anyhow::Result` so clippy no longer
  warns about `expect`
  
  ## Testing
  - `just fix -p codex-core`
  - `cargo test -p codex-core --test all
  suite::grep_files::grep_files_tool_collects_matches`
  - `cargo test -p codex-core
  suite::grep_files::grep_files_tool_collects_matches -- --ignored`
  (filter requests ignored tests so nothing runs, but the build stays
  clean)
  
  
  ------
  [Codex
  Task](https://chatgpt.com/codex/tasks/task_i_69112d53abac83219813cab4d7cb6446)

  Ahmed Ibrahim · 2025-11-13 17:12:10 -08:00
  2a6e9b20df
• [App-server] add new v2 events:item/reasoning/delta, item/agentMessage/delta & item/reasoning/summaryPartAdded (#6559)

  core event to app server event mapping:
  1. `codex/event/reasoning_content_delta` ->
  `item/reasoning/summaryTextDelta`.
  2. `codex/event/reasoning_raw_content_delta` ->
  `item/reasoning/textDelta`
  3. `codex/event/agent_message_content_delta` →
  `item/agentMessage/delta`.
  4. `codex/event/agent_reasoning_section_break` ->
  `item/reasoning/summaryPartAdded`.
  
  Also added a change in core to pass down content index, summary index
  and item id from events.
  
  Tested with the `git checkout owen/app_server_test_client && cargo run
  -p codex-app-server-test-client -- send-message-v2 "hello"` and verified
  that new events are emitted correctly.

  Celia Chen · 2025-11-14 00:25:01 +00:00
  b8ec97c0ef
• chore(core) Consolidate apply_patch tests (#6545)

  ## Summary
  Consolidates our apply_patch tests into one suite, and ensures each test
  case tests the various ways the harness supports apply_patch:
  1. Freeform custom tool call
  2. JSON function tool
  3. Simple shell call
  4. Heredoc shell call
  
  There are a few test cases that are specific to a particular variant,
  I've left those alone.
  
  ## Testing
  - [x] This adds a significant number of tests

  Dylan Hurd · 2025-11-13 15:52:39 -08:00
  2c1b693da4
• Migrate prompt caching tests to test_codex (#6605)

  To hopefully fix the flakiness

  pakrym-oai · 2025-11-13 09:19:38 -08:00
  041d6ad902
• Set verbosity to low for 5.1 (#6568)

  And improve test coverage

  pakrym-oai · 2025-11-13 01:40:52 +00:00
  f97874093e
• Fix otel tests (#6541)

  Mount responses only once, remove unneeded retries and add a final
  assistant messages to complete the turn.

  pakrym-oai · 2025-11-12 16:35:34 +00:00
  7d9ad3effd
• Add user command event types (#6246)

  adding new user command event, logic in TUI to render user command
  events

  zhao-oai · 2025-11-10 19:18:45 +00:00
  980886498c
• [App-server] v2 for account/updated and account/logout (#6175)

  V2 for `account/updated` and `account/logout` for app server. correspond
  to old `authStatusChange` and `LogoutChatGpt` respectively. Followup PRs
  will make other v2 endpoints call `account/updated` instead of
  `authStatusChange` too.

  Celia Chen · 2025-11-03 22:01:33 -08:00
  d3187dbc17
• test: undo (#6034)
  jif-oai · 2025-10-31 14:46:24 +00:00
  0508823075
• chore: testing on freeform apply_patch (#5952)

  ## Summary
  Duplicates the tests in `apply_patch_cli.rs`, but tests the freeform
  apply_patch tool as opposed to the function call path. The good news is
  that all the tests pass with zero logical tests, with the exception of
  the heredoc, which doesn't really make sense in the freeform tool
  context anyway.
  
  @jif-oai since you wrote the original tests in #5557, I'd love your
  opinion on the right way to DRY these test cases between the two. Happy
  to set up a more sophisticated harness, but didn't want to go down the
  rabbit hole until we agreed on the right pattern
  
  ## Testing
  - [x] These are tests

  Dylan Hurd · 2025-10-30 10:40:48 -07:00
  4a55646a02
• Add item streaming events (#5546)

  Adds AgentMessageContentDelta, ReasoningContentDelta,
  ReasoningRawContentDelta item streaming events while maintaining
  compatibility for old events.
  
  ---------
  
  Co-authored-by: Owen Lin <owen@openai.com>

  pakrym-oai · 2025-10-29 22:33:57 +00:00
  3429e82e45
• feat: deprecation warning (#5825)

  <img width="955" height="311" alt="Screenshot 2025-10-28 at 14 26 25"
  src="https://github.com/user-attachments/assets/99729b3d-3bc9-4503-aab3-8dc919220ab4"
  />

  jif-oai · 2025-10-29 12:29:28 +00:00
  060637b4d4
• Filter out reasoning items from previous turns (#5857)

  Reduces request size and prevents 400 errors when switching between API
  orgs.
  
  Based on Responses API behavior described in
  https://cookbook.openai.com/examples/responses_api/reasoning_items#caching

  pakrym-oai · 2025-10-28 11:39:34 -07:00
  1b8f2543ac
• chore: testing on apply_path (#5557)
  jif-oai · 2025-10-23 17:00:48 +01:00
  6745b12427
• Handle cancelling/aborting while processing a turn (#5543)

  Currently we collect all all turn items in a vector, then we add it to
  the history on success. This result in losing those items on errors
  including aborting `ctrl+c`.
  
  This PR:
  - Adds the ability for the tool call to handle cancellation
  - bubble the turn items up to where we are recording this info
  
  Admittedly, this logic is an ad-hoc logic that doesn't handle a lot of
  error edge cases. The right thing to do is recording to the history on
  the spot as `items`/`tool calls output` come. However, this isn't
  possible because of having different `task_kind` that has different
  `conversation_histories`. The `try_run_turn` has no idea what thread are
  we using. We cannot also pass an `arc` to the `conversation_histories`
  because it's a private element of `state`.
  
  That's said, `abort` is the most common case and we should cover it
  until we remove `task kind`

  Ahmed Ibrahim · 2025-10-23 08:47:10 -07:00
  f59978ed3d
• Move changing turn input functionalities to ConversationHistory (#5473)

  We are doing some ad-hoc logic while dealing with conversation history.
  Ideally, we shouldn't mutate `vec[responseitem]` manually at all and
  should depend on `ConversationHistory` for those changes.
  
  Those changes are:
  - Adding input to the history
  - Removing items from the history
  - Correcting history
  
  I am also adding some `error` logs for cases we shouldn't ideally face.
  For example, we shouldn't be missing `toolcalls` or `outputs`. We
  shouldn't hit `ContextWindowExceeded` while performing `compact`
  
  This refactor will give us granular control over our context management.

  Ahmed Ibrahim · 2025-10-22 13:08:46 -07:00
  273819aaae
• Add new thread items and rewire event parsing to use them (#5418)

  1. Adds AgentMessage,  Reasoning,  WebSearch items.
  2. Switches the ResponseItem parsing to use new items and then also emit
  3. Removes user-item kind and filters out "special" (environment) user
  items when returning to clients.

  pakrym-oai · 2025-10-22 10:14:50 -07:00
  3c90728a29
• Add a baseline test for resume initial messages (#5466)
  pakrym-oai · 2025-10-21 11:45:01 -07:00
  5cd8803998
• Add ItemStarted/ItemCompleted events for UserInputItem (#5306)

  Adds a new ItemStarted event and delivers UserMessage as the first item
  type (more to come).
  
  
  Renames `InputItem` to `UserInput` considering we're using the `Item`
  suffix for actual items.

  pakrym-oai · 2025-10-20 13:34:44 -07:00
  9c903c4716
• Auto compact at ~90% (#5292)

  Users now hit a window exceeded limit and they usually don't know what
  to do. This starts auto compact at ~90% of the window.

  Ahmed Ibrahim · 2025-10-20 11:29:49 -07:00
  049a61bcfc
• test: reduce time dependency on test harness (#5053)

  Tightened the CLI integration tests to stop relying on wall-clock
  sleeps—new fs watcher helper waits for session files instead of timing
  out, and SSE mocks/fixtures make the flows deterministic.

  jif-oai · 2025-10-15 09:56:59 +01:00
  8fed0b53c4
• fix: apply_patch shell_serialization tests (#4786)

  ## Summary
  Adds additional shell_serialization tests specifically for apply_patch
  and other cases.
  
  ## Test Plan
  - [x] These are all tests

  Dylan · 2025-10-14 13:00:49 -07:00
  0a0a10d8b3
• Simplify request body assertions (#4845)

  We'll have a lot more test like these

  pakrym-oai · 2025-10-07 09:56:39 +01:00
  35a770e871
• Make output assertions more explicit (#4784)

  Match using precise regexes.

  pakrym-oai · 2025-10-05 16:01:38 -07:00
  b2d81a7cac
• Add helper for response created SSE events in tests (#4758)

  ## Summary
  - add a reusable `ev_response_created` helper that builds
  `response.created` SSE events for integration tests
  - update the exec and core integration suites to use the new helper
  instead of repeating manual JSON literals
  - keep the streaming fixtures consistent by relying on the shared helper
  in every touched test
  
  ## Testing
  - `just fmt`
  
  
  ------
  https://chatgpt.com/codex/tasks/task_i_68e1fe885bb883208aafffb94218da61

  pakrym-oai · 2025-10-05 21:11:43 +00:00
  aecbe0f333
• Surface context window error to the client (#4675)

  In the past, we were treating `input exceeded context window` as a
  streaming error and retrying on it. Retrying on it has no point because
  it won't change the behavior. In this PR, we surface the error to the
  client without retry and also send a token count event to indicate that
  the context window is full.
  
  <img width="650" height="125" alt="image"
  src="https://github.com/user-attachments/assets/c26b1213-4c27-4bfc-90f4-51a270a3efd5"
  />

  Ahmed Ibrahim · 2025-10-05 01:40:06 +00:00
  90ef94d3b3
• [MCP] Add support for MCP Oauth credentials (#4517)

  This PR adds oauth login support to streamable http servers when
  `experimental_use_rmcp_client` is enabled.
  
  This PR is large but represents the minimal amount of work required for
  this to work. To keep this PR smaller, login can only be done with
  `codex mcp login` and `codex mcp logout` but it doesn't appear in `/mcp`
  or `codex mcp list` yet. Fingers crossed that this is the last large MCP
  PR and that subsequent PRs can be smaller.
  
  Under the hood, credentials are stored using platform credential
  managers using the [keyring crate](https://crates.io/crates/keyring).
  When the keyring isn't available, it falls back to storing credentials
  in `CODEX_HOME/.credentials.json` which is consistent with how other
  coding agents handle authentication.
  
  I tested this on macOS, Windows, WSL (ubuntu), and Linux. I wasn't able
  to test the dbus store on linux but did verify that the fallback works.
  
  One quirk is that if you have credentials, during development, every
  build will have its own ad-hoc binary so the keyring won't recognize the
  reader as being the same as the write so it may ask for the user's
  password. I may add an override to disable this or allow
  users/enterprises to opt-out of the keyring storage if it causes issues.
  
  <img width="5064" height="686" alt="CleanShot 2025-09-30 at 19 31 40"
  src="https://github.com/user-attachments/assets/9573f9b4-07f1-4160-83b8-2920db287e2d"
  />
  <img width="745" height="486" alt="image"
  src="https://github.com/user-attachments/assets/9562649b-ea5f-4f22-ace2-d0cb438b143e"
  />

  Gabriel Peal · 2025-10-03 13:43:12 -04:00
  1d17ca1fa3
• chore: refactor tool handling (#4510)

  # Tool System Refactor
  
  - Centralizes tool definitions and execution in `core/src/tools/*`:
  specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
  registry/dispatch (`registry.rs`), and shared context (`context.rs`).
  One registry now builds the model-visible tool list and binds handlers.
  - Router converts model responses to tool calls; Registry dispatches
  with consistent telemetry via `codex-rs/otel` and unified error
  handling. Function, Local Shell, MCP, and experimental `unified_exec`
  all flow through this path; legacy shell aliases still work.
  - Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
  make adding tools predictable and testable.
  
  Example: `read_file`
  - Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
  registered by `build_specs`).
  - Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
  1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
  - E2E test: `core/tests/suite/read_file.rs` validates the tool returns
  the requested lines.
  
  ## Next steps:
  - Decompose `handle_container_exec_with_params` 
  - Add parallel tool calls

  jif-oai · 2025-10-03 13:21:06 +01:00
  33d3ecbccc
• Support CODEX_API_KEY for codex exec (#4615)

  Allows to set API key per invocation of `codex exec`

  pakrym-oai · 2025-10-02 09:59:45 -07:00
  2f6fb37d72
• OpenTelemetry events (#2103)

  ### Title
  
  ## otel
  
  Codex can emit [OpenTelemetry](https://opentelemetry.io/) **log events**
  that
  describe each run: outbound API requests, streamed responses, user
  input,
  tool-approval decisions, and the result of every tool invocation. Export
  is
  **disabled by default** so local runs remain self-contained. Opt in by
  adding an
  `[otel]` table and choosing an exporter.
  
  ```toml
  [otel]
  environment = "staging"   # defaults to "dev"
  exporter = "none"          # defaults to "none"; set to otlp-http or otlp-grpc to send events
  log_user_prompt = false    # defaults to false; redact prompt text unless explicitly enabled
  ```
  
  Codex tags every exported event with `service.name = "codex-cli"`, the
  CLI
  version, and an `env` attribute so downstream collectors can distinguish
  dev/staging/prod traffic. Only telemetry produced inside the
  `codex_otel`
  crate—the events listed below—is forwarded to the exporter.
  
  ### Event catalog
  
  Every event shares a common set of metadata fields: `event.timestamp`,
  `conversation.id`, `app.version`, `auth_mode` (when available),
  `user.account_id` (when available), `terminal.type`, `model`, and
  `slug`.
  
  With OTEL enabled Codex emits the following event types (in addition to
  the
  metadata above):
  
  - `codex.api_request`
    - `cf_ray` (optional)
    - `attempt`
    - `duration_ms`
    - `http.response.status_code` (optional)
    - `error.message` (failures)
  - `codex.sse_event`
    - `event.kind`
    - `duration_ms`
    - `error.message` (failures)
    - `input_token_count` (completion only)
    - `output_token_count` (completion only)
    - `cached_token_count` (completion only, optional)
    - `reasoning_token_count` (completion only, optional)
    - `tool_token_count` (completion only)
  - `codex.user_prompt`
    - `prompt_length`
    - `prompt` (redacted unless `log_user_prompt = true`)
  - `codex.tool_decision`
    - `tool_name`
    - `call_id`
  - `decision` (`approved`, `approved_for_session`, `denied`, or `abort`)
    - `source` (`config` or `user`)
  - `codex.tool_result`
    - `tool_name`
    - `call_id`
    - `arguments`
    - `duration_ms` (execution time for the tool)
    - `success` (`"true"` or `"false"`)
    - `output`
  
  ### Choosing an exporter
  
  Set `otel.exporter` to control where events go:
  
  - `none` – leaves instrumentation active but skips exporting. This is
  the
    default.
  - `otlp-http` – posts OTLP log records to an OTLP/HTTP collector.
  Specify the
    endpoint, protocol, and headers your collector expects:
  
    ```toml
    [otel]
    exporter = { otlp-http = {
      endpoint = "https://otel.example.com/v1/logs",
      protocol = "binary",
      headers = { "x-otlp-api-key" = "${OTLP_TOKEN}" }
    }}
    ```
  
  - `otlp-grpc` – streams OTLP log records over gRPC. Provide the endpoint
  and any
    metadata headers:
  
    ```toml
    [otel]
    exporter = { otlp-grpc = {
      endpoint = "https://otel.example.com:4317",
      headers = { "x-otlp-meta" = "abc123" }
    }}
    ```
  
  If the exporter is `none` nothing is written anywhere; otherwise you
  must run or point to your
  own collector. All exporters run on a background batch worker that is
  flushed on
  shutdown.
  
  If you build Codex from source the OTEL crate is still behind an `otel`
  feature
  flag; the official prebuilt binaries ship with the feature enabled. When
  the
  feature is disabled the telemetry hooks become no-ops so the CLI
  continues to
  function without the extra dependencies.
  
  ---------
  
  Co-authored-by: Anton Panasenko <apanasenko@openai.com>

  vishnu-oai · 2025-09-29 11:30:55 -07:00
  04c1782e52
• Actually mount sse once (#4264)

  Mock server was responding with the same result many times.

  pakrym-oai · 2025-09-26 01:17:51 +00:00
  a10403d697