58 Commits

• feat: sqlite 1 (#10004)

  Add a `.sqlite` database to be used to store rollout metatdata (and
  later logs)
  This PR is phase 1:
  * Add the database and the required infrastructure
  * Add a backfill of the database
  * Persist the newly created rollout both in files and in the DB
  * When we need to get metadata or a rollout, consider the `JSONL` as the
  source of truth but compare the results with the DB and show any errors

  jif-oai · 2026-01-28 15:29:14 +01:00
  3878c3dc7c
• Warn users on enabling underdevelopment features (#9954)

  <img width="938" height="73" alt="image"
  src="https://github.com/user-attachments/assets/a2d5ac46-92c5-4828-b35e-0965c30cdf36"
  />

  Ahmed Ibrahim · 2026-01-27 01:58:05 +00:00
  c900de271a
• feat(core) ModelInfo.model_instructions_template (#9597)

  ## Summary
  #9555 is the start of a rename, so I'm starting to standardize here.
  Sets up `model_instructions` templating with a strongly-typed object for
  injecting a personality block into the model instructions.
  
  ## Testing
  - [x] Added tests
  - [x] Ran locally

  Dylan Hurd · 2026-01-21 18:11:18 -08:00
  96a72828be
• Feat: request user input tool (#9472)

  ### Summary
  * Add `requestUserInput` tool that the model can use for gather
  feedback/asking question mid turn.
  
  
  ### Tool input schema
  ```
  {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "requestUserInput input",
    "type": "object",
    "additionalProperties": false,
    "required": ["questions"],
    "properties": {
      "questions": {
        "type": "array",
        "description": "Questions to show the user (1-3). Prefer 1 unless multiple independent decisions block progress.",
        "minItems": 1,
        "maxItems": 3,
        "items": {
          "type": "object",
          "additionalProperties": false,
          "required": ["id", "header", "question"],
          "properties": {
            "id": {
              "type": "string",
              "description": "Stable identifier for mapping answers (snake_case)."
            },
            "header": {
              "type": "string",
              "description": "Short header label shown in the UI (12 or fewer chars)."
            },
            "question": {
              "type": "string",
              "description": "Single-sentence prompt shown to the user."
            },
            "options": {
              "type": "array",
              "description": "Optional 2-3 mutually exclusive choices. Put the recommended option first and suffix its label with \"(Recommended)\". Only include \"Other\" option if we want to include a free form option. If the question is free form in nature, do not include any option.",
              "minItems": 2,
              "maxItems": 3,
              "items": {
                "type": "object",
                "additionalProperties": false,
                "required": ["value", "label", "description"],
                "properties": {
                  "value": {
                    "type": "string",
                    "description": "Machine-readable value (snake_case)."
                  },
                  "label": {
                    "type": "string",
                    "description": "User-facing label (1-5 words)."
                  },
                  "description": {
                    "type": "string",
                    "description": "One short sentence explaining impact/tradeoff if selected."
                  }
                }
              }
            }
          }
        }
      }
    }
  }
  ```
  
  ### Tool output schema
  ```
  {
    "$schema": "http://json-schema.org/draft-07/schema#",
    "title": "requestUserInput output",
    "type": "object",
    "additionalProperties": false,
    "required": ["answers"],
    "properties": {
      "answers": {
        "type": "object",
        "description": "Map of question id to user answer.",
        "additionalProperties": {
          "type": "object",
          "additionalProperties": false,
          "required": ["selected"],
          "properties": {
            "selected": {
              "type": "array",
              "items": { "type": "string" }
            },
            "other": {
              "type": ["string", "null"]
            }
          }
        }
      }
    }
  }
  ```

  Shijie Rao · 2026-01-19 10:17:30 -08:00
  57ec3a8277
• Add collaboration developer instructions (#9424)

  - Add additional instructions when they are available
  - Make sure to update them on change either UserInput or UserTurn

  Ahmed Ibrahim · 2026-01-18 01:31:14 +00:00
  1478a88eb0
• Renew cache ttl on etag match (#9174)

  so we don't do unnecessary fetches

  Ahmed Ibrahim · 2026-01-14 01:21:41 +00:00
  4d787a2cc2
• Support response.done and add integration tests (#9129)

  The agent loop using a persistent incremental web socket connection.

  pakrym-oai · 2026-01-13 16:12:30 +00:00
  2d56519ecd
• Send message by default mid turn. queue messages by tab (#9077)

  https://github.com/user-attachments/assets/03838730-4ddc-44df-a2c7-cb8ecda78660

  Ahmed Ibrahim · 2026-01-12 23:06:35 -08:00
  cbca43d57a
• Add model client sessions (#9102)

  Maintain a long-running session.

  pakrym-oai · 2026-01-13 01:15:56 +00:00
  490c1c1fdd
• Assemble sandbox/approval/network prompts dynamically (#8961)

  - Add a single builder for developer permissions messaging that accepts
  SandboxPolicy and approval policy. This builder now drives the developer
  “permissions” message that’s injected at session start and any time
  sandbox/approval settings change.
  - Trim EnvironmentContext to only include cwd, writable roots, and
  shell; removed sandbox/approval/network duplication and adjusted XML
  serialization and tests accordingly.
  
  Follow-up: adding a config value to replace the developer permissions
  message for custom sandboxes.

  Ahmed Ibrahim · 2026-01-12 23:12:59 +00:00
  87f7226cca
• Add some tests for image attachments (#9080)

  Some extra tests for https://github.com/openai/codex/pull/8950

  charley-oai · 2026-01-12 13:41:50 -08:00
  d7cdcfc302
• Add hierarchical agent prompt (#8996)
  pakrym-oai · 2026-01-09 13:47:37 -08:00
  acfd94f625
• Add feature for optional request compression (#8767)

  Adds a new feature
  `enable_request_compression` that will compress using zstd requests to
  the codex-backend. Currently only enabled for codex-backend so only enabled for openai providers when using chatgpt::auth even when the feature is enabled
  
  Added a new info log line too for evaluating the compression ratio and
  overhead off compressing before requesting. You can enable with
  `RUST_LOG=$RUST_LOG,codex_client::transport=info`
  
  ```
  2026-01-06T00:09:48.272113Z  INFO codex_client::transport: Compressed request body with zstd pre_compression_bytes=28914 post_compression_bytes=11485 compression_duration_ms=0
  ```

  Channing Conger · 2026-01-07 13:21:40 -08:00
  21c6d40a44
• Override truncation policy at model info level (#8856)

  We used to override truncation policy by comparing model info vs config
  value in context manager. A better way to do it is to construct model
  info using the config value

  Ahmed Ibrahim · 2026-01-07 13:06:20 -08:00
  187924d761
• chore: unify conversation with thread name (#8830)

  Done and verified by Codex + refactor feature of RustRover

  jif-oai · 2026-01-07 17:04:53 +00:00
  116059c3a0
• add web_search_cached flag (#8795)

  Add `web_search_cached` feature to config. Enables `web_search` tool
  with access only to cached/indexed results (see
  [docs](https://platform.openai.com/docs/guides/tools-web-search#live-internet-access)).
  
  This takes precedence over the existing `web_search_request`, which
  continues to enable `web_search` over live results as it did before.
  
  `web_search_cached` is disabled for review mode, as `web_search_request`
  is.

  sayan-oai · 2026-01-06 14:53:59 -08:00
  54ded1a3c0
• Refresh on models etag mismatch (#8491)

  - Send models etag
  - Refresh models on 412
  - This wires `ModelsManager` to `ModelFamily` so we don't mutate it
  mid-turn

  Ahmed Ibrahim · 2026-01-01 11:41:16 -08:00
  66b7c673e9
• fix: enable resume_warning that was missing from mod.rs (#8333)

  This test was introduced in https://github.com/openai/codex/pull/6507,
  but was not included in `mod.rs`. It does not appear that it was getting
  compiled?

  Michael Bolin · 2025-12-19 19:21:47 +00:00
  0a7021de72
• Inject SKILL.md when it's explicitly mentioned. (#7763)

  1. Skills load once in core at session start; the cached outcome is
  reused across core and surfaced to TUI via SessionConfigured.
  2. TUI detects explicit skill selections, and core injects the matching
  SKILL.md content into the turn when a selected skill is present.

  xl-openai · 2025-12-10 13:59:17 -08:00
  b36ecb6c32
• feat: shell snapshotting (#7641)
  jif-oai · 2025-12-09 18:36:58 +00:00
  7836aeddae
• Add remote models feature flag (#7648)

  # External (non-OpenAI) Pull Request Requirements
  
  Before opening this Pull Request, please read the dedicated
  "Contributing" markdown file or your PR may be closed:
  https://github.com/openai/codex/blob/main/docs/contributing.md
  
  If your PR conforms to our contribution guidelines, replace this text
  with a detailed and high quality description of your changes.
  
  Include a link to a bug report or enhancement request.

  Ahmed Ibrahim · 2025-12-07 09:47:48 -08:00
  53a486f7ea
• feat(core) Add login to shell_command tool (#6846)

  ## Summary
  Adds the `login` parameter to the `shell_command` tool - optional,
  defaults to true.
  
  ## Testing
  - [x] Tested locally

  Dylan Hurd · 2025-12-05 11:03:25 -08:00
  a8cbbdbc6e
• chore(core): test apply_patch_cli on Windows (#7554)

  ## Summary
  These tests pass on windows, let's enable them.
  
  ## Testing
  - [x] These are more tests

  Dylan Hurd · 2025-12-04 10:39:45 -08:00
  37c36024c7
• Migrate model preset (#7542)

  - Introduce `openai_models` in `/core`
  - Move `PRESETS` under it
  - Move `ModelPreset`, `ModelUpgrade`, `ReasoningEffortPreset`,
  `ReasoningEffortPreset`, and `ReasoningEffortPreset` to `protocol`
  - Introduce `Op::ListModels` and `EventMsg::AvailableModels`
  
  Next steps:
  - migrate `app-server` and `tui` to use the introduced Operation

  Ahmed Ibrahim · 2025-12-03 20:30:43 +00:00
  71504325d3
• Fix: Improve text encoding for shell output in VSCode preview (#6178) (#6182)

  ## 🐛 Problem
  
  Users running commands with non-ASCII characters (like Russian text
  "пример") in Windows/WSL environments experience garbled text in
  VSCode's shell preview window, with Unicode replacement characters (�)
  appearing instead of the actual text.
  
  **Issue**: https://github.com/openai/codex/issues/6178
  
  ## 🔧 Root Cause
  
  The issue was in `StreamOutput<Vec<u8>>::from_utf8_lossy()` method in
  `codex-rs/core/src/exec.rs`, which used `String::from_utf8_lossy()` to
  convert shell output bytes to strings. This function immediately
  replaces any invalid UTF-8 byte sequences with replacement characters,
  without attempting to decode using other common encodings.
  
  In Windows/WSL environments, shell output often uses encodings like:
  
  - Windows-1252 (common Windows encoding)
  - Latin-1/ISO-8859-1 (extended ASCII)
  
  ## 🛠️ Solution
  
  Replaced the simple `String::from_utf8_lossy()` call with intelligent
  encoding detection via a new `bytes_to_string_smart()` function that
  tries multiple encoding strategies:
  
  1. **UTF-8** (fast path for valid UTF-8)
  2. **Windows-1252** (handles Windows-specific characters in 0x80-0x9F
  range)
  3. **Latin-1** (fallback for extended ASCII)
  4. **Lossy UTF-8** (final fallback, same as before)
  
  ## 📁 Changes
  
  ### New Files
  
  - `codex-rs/core/src/text_encoding.rs` - Smart encoding detection module
  - `codex-rs/core/tests/suite/text_encoding_fix.rs` - Integration tests
  
  ### Modified Files
  
  - `codex-rs/core/src/lib.rs` - Added text_encoding module
  - `codex-rs/core/src/exec.rs` - Updated StreamOutput::from_utf8_lossy()
  - `codex-rs/core/tests/suite/mod.rs` - Registered new test module
  
  ## ✅ Testing
  
  - **5 unit tests** covering UTF-8, Windows-1252, Latin-1, and fallback
  scenarios
  - **2 integration tests** simulating the exact Issue #6178 scenario
  - **Demonstrates improvement** over the previous
  `String::from_utf8_lossy()` approach
  
  All tests pass:
  
  ```bash
  cargo test -p codex-core text_encoding
  cargo test -p codex-core test_shell_output_encoding_issue_6178
  ```
  
  ## 🎯 Impact
  
  - ✅ **Eliminates garbled text** in VSCode shell preview for non-ASCII
  content
  - ✅ **Supports Windows/WSL environments** with proper encoding detection
  - ✅ **Zero performance impact** for UTF-8 text (fast path)
  - ✅ **Backward compatible** - UTF-8 content works exactly as before
  - ✅ **Handles edge cases** with robust fallback mechanism
  
  ## 🧪 Test Scenarios
  
  The fix has been tested with:
  
  - Russian text ("пример")
  - Windows-1252 quotation marks (""test")
  - Latin-1 accented characters ("café")
  - Mixed encoding content
  - Invalid byte sequences (graceful fallback)
  
  ## 📋 Checklist
  
  - [X] Addresses the reported issue
  - [X] Includes comprehensive tests
  - [X] Maintains backward compatibility
  - [X] Follows project coding conventions
  - [X] No breaking changes
  
  ---------
  
  Co-authored-by: Josh McKinney <joshka@openai.com>

  LIHUA · 2025-11-20 11:04:11 -08:00
  397279d46e
• execpolicy2 core integration (#6641)

  This PR threads execpolicy2 into codex-core.
  
  activated via feature flag: exec_policy (on by default)
  
  reads and parses all .codexpolicy files in `codex_home/codex`
  
  refactored tool runtime API to integrate execpolicy logic
  
  ---------
  
  Co-authored-by: Michael Bolin <mbolin@openai.com>

  zhao-oai · 2025-11-19 16:50:43 -08:00
  65c13f1ae7
• feat: remote compaction (#6795)

  Co-authored-by: pakrym-oai <pakrym@openai.com>

  jif-oai · 2025-11-18 16:51:16 +00:00
  838531d3e4
• chore(core) Consolidate apply_patch tests (#6545)

  ## Summary
  Consolidates our apply_patch tests into one suite, and ensures each test
  case tests the various ways the harness supports apply_patch:
  1. Freeform custom tool call
  2. JSON function tool
  3. Simple shell call
  4. Heredoc shell call
  
  There are a few test cases that are specific to a particular variant,
  I've left those alone.
  
  ## Testing
  - [x] This adds a significant number of tests

  Dylan Hurd · 2025-11-13 15:52:39 -08:00
  2c1b693da4
• Use codex-linux-sandbox in unified exec (#6480)

  Unified exec isn't working on Linux because we don't provide the correct
  arg0.
  
  The library we use for pty management doesn't allow setting arg0
  separately from executable. Use the same aliasing strategy we use for
  `apply_patch` for `codex-linux-sandbox`.
  
  Use `#[ctor]` hack to dispatch codex-linux-sandbox calls.
  
  
  Addresses https://github.com/openai/codex/issues/6450

  pakrym-oai · 2025-11-10 17:17:09 -08:00
  6c36318bd8
• Don't retry "insufficient_quota" errors (#6340)

  This PR makes an "insufficient quota" error fatal so we don't attempt to
  retry it multiple times in the agent loop.
  
  We have multiple bug reports from users about intermittent retry
  behaviors, and this could explain some of them. With this change, we'll
  eliminate the retries and surface a clear error message.
  
  The PR is a nearly identical copy of [this
  PR](https://github.com/openai/codex/pull/4837) contributed by
  @abimaelmartell. The original PR has gone stale. Rather than wait for
  the contributor to resolve merge conflicts, I wanted to get this change
  in.

  Eric Traut · 2025-11-06 15:12:01 -08:00
  0c647bc566
• Improved token refresh handling to address "Re-connecting" behavior (#6231)

  Currently, when the access token expires, we attempt to use the refresh
  token to acquire a new access token. This works most of the time.
  However, there are situations where the refresh token is expired,
  exhausted (already used to perform a refresh), or revoked. In those
  cases, the current logic treats the error as transient and attempts to
  retry it repeatedly.
  
  This PR changes the token refresh logic to differentiate between
  permanent and transient errors. It also changes callers to treat the
  permanent errors as fatal rather than retrying them. And it provides
  better error messages to users so they understand how to address the
  problem. These error messages should also help us further understand why
  we're seeing examples of refresh token exhaustion.
  
  Here is the error message in the CLI. The same text appears within the
  extension.
  
  <img width="863" height="38" alt="image"
  src="https://github.com/user-attachments/assets/7ffc0d08-ebf0-4900-b9a9-265064202f4f"
  />
  
  I also correct the spelling of "Re-connecting", which shouldn't have a
  hyphen in it.
  
  Testing: I manually tested these code paths by adding temporary code to
  programmatically cause my refresh token to be exhausted (by calling the
  token refresh endpoint in a tight loop more than 50 times). I then
  simulated an access token expiration, which caused the token refresh
  logic to be invoked. I confirmed that the updated logic properly handled
  the error condition.
  
  Note: We earlier discussed the idea of forcefully logging out the user
  at the point where token refresh failed. I made several attempts to do
  this, and all of them resulted in a bad UX. It's important to surface
  this error to users in a way that explains the problem and tells them
  that they need to log in again. We also previously discussed deleting
  the auth.json file when this condition is detected. That also creates
  problems because it effectively changes the auth status from logged in
  to logged out, and this causes odd failures and inconsistent UX. I think
  it's therefore better not to delete auth.json in this case. If the user
  closes the CLI or VSCE and starts it again, we properly detect that the
  access token is expired and the refresh token is "dead", and we force
  the user to go through the login flow at that time.
  
  This should address aspects of #6191, #5679, and #5505

  Eric Traut · 2025-11-05 10:51:57 -08:00
  c4ebe4b078
• test: undo (#6034)
  jif-oai · 2025-10-31 14:46:24 +00:00
  0508823075
• chore: testing on freeform apply_patch (#5952)

  ## Summary
  Duplicates the tests in `apply_patch_cli.rs`, but tests the freeform
  apply_patch tool as opposed to the function call path. The good news is
  that all the tests pass with zero logical tests, with the exception of
  the heredoc, which doesn't really make sense in the freeform tool
  context anyway.
  
  @jif-oai since you wrote the original tests in #5557, I'd love your
  opinion on the right way to DRY these test cases between the two. Happy
  to set up a more sophisticated harness, but didn't want to go down the
  rabbit hole until we agreed on the right pattern
  
  ## Testing
  - [x] These are tests

  Dylan Hurd · 2025-10-30 10:40:48 -07:00
  4a55646a02
• Delegate review to codex instance (#5572)

  In this PR, I am exploring migrating task kind to an invocation of
  Codex. The main reason would be getting rid off multiple
  `ConversationHistory` state and streamlining our context/history
  management.
  
  This approach depends on opening a channel between the sub-codex and
  codex. This channel is responsible for forwarding `interactive`
  (`approvals`) and `non-interactive` events. The `task` is responsible
  for handling those events.
  
  This opens the door for implementing `codex as a tool`, replacing
  `compact` and `review`, and potentially subagents.
  
  One consideration is this code is very similar to `app-server` specially
  in the approval part. If in the future we wanted an interactive
  `sub-codex` we should consider using `codex-mcp`

  Ahmed Ibrahim · 2025-10-29 21:04:25 +00:00
  13e1d0362d
• feat: deprecation warning (#5825)

  <img width="955" height="311" alt="Screenshot 2025-10-28 at 14 26 25"
  src="https://github.com/user-attachments/assets/99729b3d-3bc9-4503-aab3-8dc919220ab4"
  />

  jif-oai · 2025-10-29 12:29:28 +00:00
  060637b4d4
• feature: Add "!cmd" user shell execution (#2471)

  feature: Add "!cmd" user shell execution
  
  This change lets users run local shell commands directly from the TUI by
  prefixing their input with ! (e.g. !ls). Output is truncated to keep the
  exec cell usable, and Ctrl-C cleanly
    interrupts long-running commands (e.g. !sleep 10000).
  
  **Summary of changes**
  
  - Route Op::RunUserShellCommand through a dedicated UserShellCommandTask
  (core/src/tasks/user_shell.rs), keeping the task logic out of codex.rs.
  - Reuse the existing tool router: the task constructs a ToolCall for the
  local_shell tool and relies on ShellHandler, so no manual MCP tool
  lookup is required.
  - Emit exec lifecycle events (ExecCommandBegin/ExecCommandEnd) so the
  TUI can show command metadata, live output, and exit status.
  
  **End-to-end flow**
  
    **TUI handling**
  
  1. ChatWidget::submit_user_message (TUI) intercepts messages starting
  with !.
  2. Non-empty commands dispatch Op::RunUserShellCommand { command };
  empty commands surface a help hint.
  3. No UserInput items are created, so nothing is enqueued for the model.
  
    **Core submission loop**
  4. The submission loop routes the op to handlers::run_user_shell_command
  (core/src/codex.rs).
  5. A fresh TurnContext is created and Session::spawn_user_shell_command
  enqueues UserShellCommandTask.
  
    **Task execution**
  6. UserShellCommandTask::run emits TaskStartedEvent, formats the
  command, and prepares a ToolCall targeting local_shell.
    7. ToolCallRuntime::handle_tool_call dispatches to ShellHandler.
  
    **Shell tool runtime**
  8. ShellHandler::run_exec_like launches the process via the unified exec
  runtime, honoring sandbox and shell policies, and emits
  ExecCommandBegin/End.
  9. Stdout/stderr are captured for the UI, but the task does not turn the
  resulting ToolOutput into a model response.
  
    **Completion**
  10. After ExecCommandEnd, the task finishes without an assistant
  message; the session marks it complete and the exec cell displays the
  final output.
  
    **Conversation context**
  
  - The command and its output never enter the conversation history or the
  model prompt; the flow is local-only.
    - Only exec/task events are emitted for UI rendering.
  
  **Demo video**
  
  
  https://github.com/user-attachments/assets/fcd114b0-4304-4448-a367-a04c43e0b996

  Abhishek Bhardwaj · 2025-10-29 00:31:20 -07:00
  89591e4246
• Centralize truncation in conversation history (#5652)

  move the truncation logic to conversation history to use on any tool
  output. This will help us in avoiding edge cases while truncating the
  tool calls and mcp calls.

  Ahmed Ibrahim · 2025-10-27 14:05:35 -07:00
  7226365397
• chore: testing on apply_path (#5557)
  jif-oai · 2025-10-23 17:00:48 +01:00
  6745b12427
• Add a baseline test for resume initial messages (#5466)
  pakrym-oai · 2025-10-21 11:45:01 -07:00
  5cd8803998
• Add ItemStarted/ItemCompleted events for UserInputItem (#5306)

  Adds a new ItemStarted event and delivers UserMessage as the first item
  type (more to come).
  
  
  Renames `InputItem` to `UserInput` considering we're using the `Item`
  suffix for actual items.

  pakrym-oai · 2025-10-20 13:34:44 -07:00
  9c903c4716
• chore: rework tools execution workflow (#5278)

  Re-work the tool execution flow. Read `orchestrator.rs` to understand
  the structure

  jif-oai · 2025-10-20 20:57:37 +01:00
  5e4f3bbb0b
• Expand approvals integration coverage (#5358)

  Improve approval coverage

  pakrym-oai · 2025-10-20 17:11:43 +01:00
  540abfa05e
• feat: grep_files as a tool (#4820)

  Add `grep_files` to be able to perform more action in parallel

  jif-oai · 2025-10-08 11:02:50 +01:00
  f52320be86
• feat: list_dir tool (#4817)

  Add a tool to list_dir. It is useful because we can mark it as
  non-mutating and so use it in parallel

  jif-oai · 2025-10-07 19:33:19 +01:00
  226215f36d
• feat: parallel tool calls (#4663)

  Add parallel tool calls. This is configurable at model level and tool
  level

  jif-oai · 2025-10-05 16:10:49 +00:00
  dc3c6bf62a
• feat: Freeform apply_patch with simple shell output (#4718)

  ## Summary
  This PR is an alternative approach to #4711, but instead of changing our
  storage, parses out shell calls in the client and reserializes them on
  the fly before we send them out as part of the request.
  
  What this changes:
  1. Adds additional serialization logic when the
  ApplyPatchToolType::Freeform is in use.
  2. Adds a --custom-apply-patch flag to enable this setting on a
  session-by-session basis.
  
  This change is delicate, but is not meant to be permanent. It is meant
  to be the first step in a migration:
  1. (This PR) Add in-flight serialization with config
  2. Update model_family default
  3. Update serialization logic to store turn outputs in a structured
  format, with logic to serialize based on model_family setting.
  4. Remove this rewrite in-flight logic.
  
  ## Test Plan
  - [x] Additional unit tests added
  - [x] Integration tests added
  - [x] Tested locally

  Dylan · 2025-10-04 19:16:36 -07:00
  4764fc1ee7
• chore: refactor tool handling (#4510)

  # Tool System Refactor
  
  - Centralizes tool definitions and execution in `core/src/tools/*`:
  specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
  registry/dispatch (`registry.rs`), and shared context (`context.rs`).
  One registry now builds the model-visible tool list and binds handlers.
  - Router converts model responses to tool calls; Registry dispatches
  with consistent telemetry via `codex-rs/otel` and unified error
  handling. Function, Local Shell, MCP, and experimental `unified_exec`
  all flow through this path; legacy shell aliases still work.
  - Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
  make adding tools predictable and testable.
  
  Example: `read_file`
  - Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
  registered by `build_specs`).
  - Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
  1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
  - E2E test: `core/tests/suite/read_file.rs` validates the tool returns
  the requested lines.
  
  ## Next steps:
  - Decompose `handle_container_exec_with_params` 
  - Add parallel tool calls

  jif-oai · 2025-10-03 13:21:06 +01:00
  33d3ecbccc
• OpenTelemetry events (#2103)

  ### Title
  
  ## otel
  
  Codex can emit [OpenTelemetry](https://opentelemetry.io/) **log events**
  that
  describe each run: outbound API requests, streamed responses, user
  input,
  tool-approval decisions, and the result of every tool invocation. Export
  is
  **disabled by default** so local runs remain self-contained. Opt in by
  adding an
  `[otel]` table and choosing an exporter.
  
  ```toml
  [otel]
  environment = "staging"   # defaults to "dev"
  exporter = "none"          # defaults to "none"; set to otlp-http or otlp-grpc to send events
  log_user_prompt = false    # defaults to false; redact prompt text unless explicitly enabled
  ```
  
  Codex tags every exported event with `service.name = "codex-cli"`, the
  CLI
  version, and an `env` attribute so downstream collectors can distinguish
  dev/staging/prod traffic. Only telemetry produced inside the
  `codex_otel`
  crate—the events listed below—is forwarded to the exporter.
  
  ### Event catalog
  
  Every event shares a common set of metadata fields: `event.timestamp`,
  `conversation.id`, `app.version`, `auth_mode` (when available),
  `user.account_id` (when available), `terminal.type`, `model`, and
  `slug`.
  
  With OTEL enabled Codex emits the following event types (in addition to
  the
  metadata above):
  
  - `codex.api_request`
    - `cf_ray` (optional)
    - `attempt`
    - `duration_ms`
    - `http.response.status_code` (optional)
    - `error.message` (failures)
  - `codex.sse_event`
    - `event.kind`
    - `duration_ms`
    - `error.message` (failures)
    - `input_token_count` (completion only)
    - `output_token_count` (completion only)
    - `cached_token_count` (completion only, optional)
    - `reasoning_token_count` (completion only, optional)
    - `tool_token_count` (completion only)
  - `codex.user_prompt`
    - `prompt_length`
    - `prompt` (redacted unless `log_user_prompt = true`)
  - `codex.tool_decision`
    - `tool_name`
    - `call_id`
  - `decision` (`approved`, `approved_for_session`, `denied`, or `abort`)
    - `source` (`config` or `user`)
  - `codex.tool_result`
    - `tool_name`
    - `call_id`
    - `arguments`
    - `duration_ms` (execution time for the tool)
    - `success` (`"true"` or `"false"`)
    - `output`
  
  ### Choosing an exporter
  
  Set `otel.exporter` to control where events go:
  
  - `none` – leaves instrumentation active but skips exporting. This is
  the
    default.
  - `otlp-http` – posts OTLP log records to an OTLP/HTTP collector.
  Specify the
    endpoint, protocol, and headers your collector expects:
  
    ```toml
    [otel]
    exporter = { otlp-http = {
      endpoint = "https://otel.example.com/v1/logs",
      protocol = "binary",
      headers = { "x-otlp-api-key" = "${OTLP_TOKEN}" }
    }}
    ```
  
  - `otlp-grpc` – streams OTLP log records over gRPC. Provide the endpoint
  and any
    metadata headers:
  
    ```toml
    [otel]
    exporter = { otlp-grpc = {
      endpoint = "https://otel.example.com:4317",
      headers = { "x-otlp-meta" = "abc123" }
    }}
    ```
  
  If the exporter is `none` nothing is written anywhere; otherwise you
  must run or point to your
  own collector. All exporters run on a background batch worker that is
  flushed on
  shutdown.
  
  If you build Codex from source the OTEL crate is still behind an `otel`
  feature
  flag; the official prebuilt binaries ship with the feature enabled. When
  the
  feature is disabled the telemetry hooks become no-ops so the CLI
  continues to
  function without the extra dependencies.
  
  ---------
  
  Co-authored-by: Anton Panasenko <apanasenko@openai.com>

  vishnu-oai · 2025-09-29 11:30:55 -07:00
  04c1782e52
• [MCP] Introduce an experimental official rust sdk based mcp client (#4252)

  The [official Rust
  SDK](https://github.com/modelcontextprotocol/rust-sdk/tree/57fc428c578a1a3fe851ee0838bf068bda120eb3)
  has come a long way since we first started our mcp client implementation
  5 months ago and, today, it is much more complete than our own
  stdio-only implementation.
  
  This PR introduces a new config flag `experimental_use_rmcp_client`
  which will use a new mcp client powered by the sdk instead of our own.
  
  To keep this PR simple, I've only implemented the same stdio MCP
  functionality that we had but will expand on it with future PRs.
  
  ---------
  
  Co-authored-by: pakrym-oai <pakrym@openai.com>

  Gabriel Peal · 2025-09-26 13:13:37 -04:00
  e555a36c6a
• ref: state - 2 (#4229)

  Extracting tasks in a module and start abstraction behind a Trait (more
  to come on this but each task will be tackled in a dedicated PR)
  The goal was to drop the ActiveTask and to have a (potentially) set of
  tasks during each turn

  jif-oai · 2025-09-26 13:49:08 +00:00
  1fc3413a46