codex

feat: retroactive image placeholder to prevent poisoning (#6774 )

If an image can't be read by the API, it will poison the entire history,
preventing any new turn on the conversation.
This detect such cases and replace the image by a placeholder

jif-oai · 2025-12-03 11:35:56 +00:00

51307eaf07

fix(unified_exec): use platform default shell when unified_exec shell… (#7486 )

# Unified Exec Shell Selection on Windows

## Problem

reference issue #7466

The `unified_exec` handler currently deserializes model-provided tool
calls into the `ExecCommandArgs` struct:

```rust
#[derive(Debug, Deserialize)]
struct ExecCommandArgs {
    cmd: String,
    #[serde(default)]
    workdir: Option<String>,
    #[serde(default = "default_shell")]
    shell: String,
    #[serde(default = "default_login")]
    login: bool,
    #[serde(default = "default_exec_yield_time_ms")]
    yield_time_ms: u64,
    #[serde(default)]
    max_output_tokens: Option<usize>,
    #[serde(default)]
    with_escalated_permissions: Option<bool>,
    #[serde(default)]
    justification: Option<String>,
}
```

The `shell` field uses a hard-coded default:

```rust
fn default_shell() -> String {
    "/bin/bash".to_string()
}
```

When the model returns a tool call JSON that only contains `cmd` (which
is the common case), Serde fills in `shell` with this default value.
Later, `get_command` uses that value as if it were a model-provided
shell path:

```rust
fn get_command(args: &ExecCommandArgs) -> Vec<String> {
    let shell = get_shell_by_model_provided_path(&PathBuf::from(args.shell.clone()));
    shell.derive_exec_args(&args.cmd, args.login)
}
```

On Unix, this usually resolves to `/bin/bash` and works as expected.
However, on Windows this behavior is problematic:

- The hard-coded `"/bin/bash"` is not a valid Windows path.
- `get_shell_by_model_provided_path` treats this as a model-specified
shell, and tries to resolve it (e.g. via `which::which("bash")`), which
may or may not exist and may not behave as intended.
- In practice, this leads to commands being executed under a non-default
or non-existent shell on Windows (for example, WSL bash), instead of the
expected Windows PowerShell or `cmd.exe`.

The core of the issue is that **"model did not specify `shell`" is
currently interpreted as "the model explicitly requested `/bin/bash`"**,
which is both Unix-specific and wrong on Windows.

## Proposed Solution

Instead of hard-coding `"/bin/bash"` into `ExecCommandArgs`, we should
distinguish between:

1. **The model explicitly specifying a shell**, e.g.:

   ```json
   {
     "cmd": "echo hello",
     "shell": "pwsh"
   }
   ```

In this case, we *do* want to respect the model’s choice and use
`get_shell_by_model_provided_path`.

2. **The model omitting the `shell` field entirely**, e.g.:

   ```json
   {
     "cmd": "echo hello"
   }
   ```

In this case, we should *not* assume `/bin/bash`. Instead, we should use
`default_user_shell()` and let the platform decide.

To express this distinction, we can:

1. Change `shell` to be optional in `ExecCommandArgs`:

   ```rust
   #[derive(Debug, Deserialize)]
   struct ExecCommandArgs {
       cmd: String,
       #[serde(default)]
       workdir: Option<String>,
       #[serde(default)]
       shell: Option<String>,
       #[serde(default = "default_login")]
       login: bool,
       #[serde(default = "default_exec_yield_time_ms")]
       yield_time_ms: u64,
       #[serde(default)]
       max_output_tokens: Option<usize>,
       #[serde(default)]
       with_escalated_permissions: Option<bool>,
       #[serde(default)]
       justification: Option<String>,
   }
   ```

Here, the absence of `shell` in the JSON is represented as `shell:
None`, rather than a hard-coded string value.

Robby He · 2025-12-02 21:49:25 -08:00

f3989f6092

feat: intercept apply_patch for unified_exec (#7446 )

jif-oai · 2025-12-02 17:54:02 +00:00

72b95db12f

chore: review everywhere (#7444 )

jif-oai · 2025-12-02 11:26:27 +00:00

4b78e2ab09

feat: experimental support for skills.md (#7412 )

This change prototypes support for Skills with the CLI. This is an
**experimental** feature for internal testing.

---------

Co-authored-by: Gav Verma <gverma@openai.com>

Thibault Sottiaux · 2025-12-01 20:22:35 -08:00

a8d5ad37b8

fix(apply_patch) tests for shell_command (#7307 )

## Summary
Adds test coverage for invocations of apply_patch via shell_command with
heredoc, to validate behavior.

## Testing
- [x] These are tests

Dylan Hurd · 2025-12-01 15:09:22 -08:00

5b25915d7e

fix: disable review rollout filtering (#7371 )

jif-oai · 2025-12-01 09:04:13 +00:00

a421eba31f

feat: detached review (#7292 )

jif-oai · 2025-11-28 11:34:57 +00:00

aaec8abf58

feat: update process ID for event handling (#7261 )

jif-oai · 2025-11-25 14:21:05 -08:00

28ff364c3a

feat: add compaction event (#7289 )

jif-oai · 2025-11-25 16:12:14 +00:00

9ba27cfa0a

fix: don't store early exit sessions (#7263 )

jif-oai · 2025-11-24 21:14:24 +00:00

fc2ff624ac

fix(windows) support apply_patch parsing in powershell (#7221 )

## Summary
Support powershell parsing of apply_patch

## Testing
- [x] Enable apply_patch unit tests

---------

Co-authored-by: jif-oai <jif@openai.com>

Dylan Hurd · 2025-11-24 19:32:47 +00:00

1e832b1438

fix: flaky test (#7257 )

jif-oai · 2025-11-24 18:45:41 +00:00

35d89e820f

feat: unified exec basic pruning strategy (#7239 )

LRU + exited sessions first

jif-oai · 2025-11-24 17:22:32 +00:00

b2cddec3d7

Account for encrypted reasoning for auto compaction (#7113 )

- The total token used returned from the api doesn't account for the
reasoning items before the assistant message
- Account for those for auto compaction
- Add the encrypted reasoning effort in the common tests utils
- Add a test to make sure it works as expected

Ahmed Ibrahim · 2025-11-22 03:06:45 +00:00

b519267d05

refactor: inline sandbox type lookup in process_exec_tool_call (#7122 )

`process_exec_tool_call()` was taking `SandboxType` as a param, but in
practice, the only place it was constructed was in
`codex_message_processor.rs` where it was derived from the other
`sandbox_policy` param, so this PR inlines the logic that decides the
`SandboxType` into `process_exec_tool_call()`.



---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/7122).
* #7112
* __->__ #7122

Michael Bolin · 2025-11-21 22:53:05 +00:00

67975ed33a

Use use_model (#7121 )

pakrym-oai · 2025-11-21 22:10:52 +00:00

e52cc38dfd

enable unified exec for experiments (#7118 )

Ahmed Ibrahim · 2025-11-21 13:10:01 -08:00

d5f661c91d

Revert "fix: read max_output_tokens param from config" (#7088 )

Reverts openai/codex#4139

jif-oai · 2025-11-21 11:40:02 +01:00

bce030ddb5

fix: read max_output_tokens param from config (#4139 )

Request param `max_output_tokens` is documented in
`https://github.com/openai/codex/blob/main/docs/config.md`,
but nowhere uses the item in config, this commit read it from config for
GPT responses API.

see https://github.com/openai/codex/issues/4138 for issue report.

Signed-off-by: Yorling <shallowcloud@yeah.net>

Yorling · 2025-11-20 22:46:34 -08:00

c9e149fd5c

Fixed two tests that can fail in some environments that have global git rewrite rules (#7068 )

This fixes https://github.com/openai/codex/issues/7044

Eric Traut · 2025-11-20 22:45:40 -08:00

bacdc004be

Support all types of search actions (#7061 )

Fixes the 

```
{
  "error": {
    "message": "Invalid value: 'other'. Supported values are: 'search', 'open_page', and 'find_in_page'.",
    "type": "invalid_request_error",
    "param": "input[150].action.type",
    "code": "invalid_value"
  }
```
error.


The actual-actual fix here is supporting absent `query` parameter.

pakrym-oai · 2025-11-20 20:45:28 -08:00

ab5972d447

Migrate coverage to shell_command (#7042 )

pakrym-oai · 2025-11-21 03:44:00 +00:00

767b66f407

fix flaky tool_call_output_exceeds_limit_truncated_chars_limit (#7043 )

I am suspecting this is flaky because of the wall time can become 0,
0.1, or 1.

Ahmed Ibrahim · 2025-11-20 16:36:29 -08:00

1388e99674

feat: update process_exec_tool_call() to take a cancellation token (#6972 )

This updates `ExecParams` so that instead of taking `timeout_ms:
Option<u64>`, it now takes a more general cancellation mechanism,
`ExecExpiration`, which is an enum that includes a
`Cancellation(tokio_util::sync::CancellationToken)` variant.

If the cancellation token is fired, then `process_exec_tool_call()`
returns in the same way as if a timeout was exceeded.

This is necessary so that in #6973, we can manage the timeout logic
external to the `process_exec_tool_call()` because we want to "suspend"
the timeout when an elicitation from a human user is pending.








---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/6972).
* #7005
* #6973
* __->__ #6972

Michael Bolin · 2025-11-20 16:29:57 -08:00

f56d1dc8fc

migrate collect_tool_identifiers_for_model to test_codex (#7041 )

Maybe it solved flakiness

Ahmed Ibrahim · 2025-11-20 16:02:50 -08:00

9be310041b

core: make shell behavior portable on FreeBSD (#7039 )

- Use /bin/sh instead of /bin/bash on FreeBSD/OpenBSD in the process
group timeout test to avoid command-not-found failures.

- Accept /usr/local/bin/bash as a valid SHELL path to match common
FreeBSD installations.

- Switch the shell serialization duration test to /bin/sh for improved
portability across Unix platforms.

With this change, `cargo test -p codex-core --lib` runs and passes on
FreeBSD.

Xiao-Yong Jin · 2025-11-20 16:01:35 -08:00

0fbcdd77c8

Attempt to fix unified_exec_formats_large_output_summary flakiness (#7029 )

second attempt to fix this test after
https://github.com/openai/codex/pull/6884. I think this flakiness is
happening because yield_time is too small for a 10,000 step loop in
python.

Ahmed Ibrahim · 2025-11-20 14:38:04 -08:00

54ee302a06

Delete shell_command feature (#7024 )

pakrym-oai · 2025-11-20 14:14:56 -08:00

856f97f449

Delete tiktoken-rs (#7018 )

pakrym-oai · 2025-11-20 11:15:04 -08:00

52d0ec4cd8

Fix: Improve text encoding for shell output in VSCode preview (#6178 ) (#6182 )

## 🐛 Problem

Users running commands with non-ASCII characters (like Russian text
"пример") in Windows/WSL environments experience garbled text in
VSCode's shell preview window, with Unicode replacement characters (�)
appearing instead of the actual text.

**Issue**: https://github.com/openai/codex/issues/6178

## 🔧 Root Cause

The issue was in `StreamOutput<Vec<u8>>::from_utf8_lossy()` method in
`codex-rs/core/src/exec.rs`, which used `String::from_utf8_lossy()` to
convert shell output bytes to strings. This function immediately
replaces any invalid UTF-8 byte sequences with replacement characters,
without attempting to decode using other common encodings.

In Windows/WSL environments, shell output often uses encodings like:

- Windows-1252 (common Windows encoding)
- Latin-1/ISO-8859-1 (extended ASCII)

## 🛠️ Solution

Replaced the simple `String::from_utf8_lossy()` call with intelligent
encoding detection via a new `bytes_to_string_smart()` function that
tries multiple encoding strategies:

1. **UTF-8** (fast path for valid UTF-8)
2. **Windows-1252** (handles Windows-specific characters in 0x80-0x9F
range)
3. **Latin-1** (fallback for extended ASCII)
4. **Lossy UTF-8** (final fallback, same as before)

## 📁 Changes

### New Files

- `codex-rs/core/src/text_encoding.rs` - Smart encoding detection module
- `codex-rs/core/tests/suite/text_encoding_fix.rs` - Integration tests

### Modified Files

- `codex-rs/core/src/lib.rs` - Added text_encoding module
- `codex-rs/core/src/exec.rs` - Updated StreamOutput::from_utf8_lossy()
- `codex-rs/core/tests/suite/mod.rs` - Registered new test module

## ✅ Testing

- **5 unit tests** covering UTF-8, Windows-1252, Latin-1, and fallback
scenarios
- **2 integration tests** simulating the exact Issue #6178 scenario
- **Demonstrates improvement** over the previous
`String::from_utf8_lossy()` approach

All tests pass:

```bash
cargo test -p codex-core text_encoding
cargo test -p codex-core test_shell_output_encoding_issue_6178
```

## 🎯 Impact

- ✅ **Eliminates garbled text** in VSCode shell preview for non-ASCII
content
- ✅ **Supports Windows/WSL environments** with proper encoding detection
- ✅ **Zero performance impact** for UTF-8 text (fast path)
- ✅ **Backward compatible** - UTF-8 content works exactly as before
- ✅ **Handles edge cases** with robust fallback mechanism

## 🧪 Test Scenarios

The fix has been tested with:

- Russian text ("пример")
- Windows-1252 quotation marks (""test")
- Latin-1 accented characters ("café")
- Mixed encoding content
- Invalid byte sequences (graceful fallback)

## 📋 Checklist

- [X] Addresses the reported issue
- [X] Includes comprehensive tests
- [X] Maintains backward compatibility
- [X] Follows project coding conventions
- [X] No breaking changes

---------

Co-authored-by: Josh McKinney <joshka@openai.com>

LIHUA · 2025-11-20 11:04:11 -08:00

397279d46e

Always fallback to real shell (#6953 )

Either cmd.exe or `/bin/sh`.

pakrym-oai · 2025-11-20 10:58:46 -08:00

30ca89424c

Allow unified_exec to early exit (if the process terminates before yield_time_ms) (#6867 )

Thread through an `exit_notify` tokio `Notify` through to the
`UnifiedExecSession` so that we can return early if the command
terminates before `yield_time_ms`.

As Codex review correctly pointed out below 🙌 we also need a
`exit_signaled` flag so that commands which finish before we start
waiting can also exit early.

Since the default `yield_time_ms` is now 10s, this means that we don't
have to wait 10s for trivial commands like ls, sed, etc (which are the
majority of agent commands 😅)

---------

Co-authored-by: jif-oai <jif@openai.com>

hanson-openai · 2025-11-20 13:34:41 +01:00

b5dd189067

execpolicy2 core integration (#6641 )

This PR threads execpolicy2 into codex-core.

activated via feature flag: exec_policy (on by default)

reads and parses all .codexpolicy files in `codex_home/codex`

refactored tool runtime API to integrate execpolicy logic

---------

Co-authored-by: Michael Bolin <mbolin@openai.com>

zhao-oai · 2025-11-19 16:50:43 -08:00

65c13f1ae7

storing credits (#6858 )

Expand the rate-limit cache/TUI: store credit snapshots alongside
primary and secondary windows, render “Credits” when the backend reports
they exist (unlimited vs rounded integer balances)

zhao-oai · 2025-11-19 10:49:35 -08:00

72af589398

fix: don't truncate at new lines (#6907 )

Ahmed Ibrahim · 2025-11-19 17:05:48 +00:00

d62cab9a06

feat: arcticfox in the wild (#6906 )

<img width="485" height="600" alt="image"
src="https://github.com/user-attachments/assets/4341740d-dd58-4a3e-b69a-33a3be0606c5"
/>

---------

Co-authored-by: jif-oai <jif@openai.com>

Ahmed Ibrahim · 2025-11-19 16:31:06 +00:00

d5dfba2509

fix(core) Support changing /approvals before conversation (#6836 )

## Summary
Setting `/approvals` before the start of a conversation was not updating
the environment_context for a conversation. Not sure exactly when this
problem was introduced, but this should reduce model confusion
dramatically.

## Testing
- [x] Added unit test to reproduce bug, confirmed fix with update
- [x] Tested locally

Dylan Hurd · 2025-11-19 11:32:48 +00:00

15b5eb30ed

Move shell to use truncate_text (#6842 )

Move shell to use the configurable `truncate_text`

---------

Co-authored-by: pakrym-oai <pakrym@openai.com>

Ahmed Ibrahim · 2025-11-19 01:56:08 -08:00

efebc62fb7

Run remote auto compaction (#6879 )

pakrym-oai · 2025-11-19 00:43:58 -08:00

75f38f16dd

flaky-unified_exec_formats_large_output_summary (#6884 )

# External (non-OpenAI) Pull Request Requirements

Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md

If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.

Include a link to a bug report or enhancement request.

Ahmed Ibrahim · 2025-11-19 00:00:37 -08:00

0440a3f105

shell_command returns freeform output (#6860 )

Instead of returning structured out and then re-formatting it into
freeform, return the freeform output from shell_command tool.

Keep `shell` as the default tool for GPT-5.

pakrym-oai · 2025-11-18 23:38:43 -08:00

ee0484a98c

fix: typos in model picker (#6859 )

# External (non-OpenAI) Pull Request Requirements

Before opening this Pull Request, please read the dedicated
"Contributing" markdown file or your PR may be closed:
https://github.com/openai/codex/blob/main/docs/contributing.md

If your PR conforms to our contribution guidelines, replace this text
with a detailed and high quality description of your changes.

Include a link to a bug report or enhancement request.

Ahmed Ibrahim · 2025-11-19 06:29:02 +00:00

793063070b

fix: local compaction (#6844 )

jif-oai · 2025-11-18 22:18:10 +00:00

c56d0c159b

feat: review in app server (#6613 )

jif-oai · 2025-11-18 21:58:54 +00:00

8ddae8cde3

chore(config) enable shell_command (#6843 )

## Summary
Enables shell_command as default for `gpt-5*` and `codex-*` models.

## Testing
- [x] Updated unit tests

Dylan Hurd · 2025-11-18 12:46:02 -08:00

29ca89c414

Add the utility to truncate by tokens (#6746 )

- This PR is to make it on path for truncating by tokens. This path will
be initially used by unified exec and context manager (responsible for
MCP calls mainly).
- We are exposing new config `calls_output_max_tokens`
- Use `tokens` as the main budget unit but truncate based on the model
family by Introducing `TruncationPolicy`.
- Introduce `truncate_text` as a router for truncation based on the
mode.

In next PRs:
- remove truncate_with_line_bytes_budget
- Add the ability to the model to override the token budget.

Ahmed Ibrahim · 2025-11-18 11:36:23 -08:00

3de8790714

feat: remote compaction (#6795 )

Co-authored-by: pakrym-oai <pakrym@openai.com>

jif-oai · 2025-11-18 16:51:16 +00:00

838531d3e4

fix(windows) shell_command on windows, minor parsing (#6811 )

## Summary
Enables shell_command for windows users, and starts adding some basic
command parsing here, to at least remove powershell prefixes. We'll
follow this up with command parsing but I wanted to land this change
separately with some basic UX.

**NOTE**: This implementation parses bash and powershell on both
platforms. In theory this is possible, since you can use git bash on
windows or powershell on linux. In practice, this may not be worth the
complexity of supporting, so I don't feel strongly about the current
approach vs. platform-specific branching.

## Testing
- [x] Added a bunch of tests 
- [x] Ran on both windows and os x

Dylan Hurd · 2025-11-17 22:23:53 -08:00

28ebe1c97a

chore(core) Add shell_serialization coverage (#6810 )

## Summary
Similar to #6545, this PR updates the shell_serialization test suite to
cover the various `shell` tool invocations we have. Note that this does
not cover unified_exec, which has its own suite of tests. This should
provide some test coverage for when we eventually consolidate
serialization logic.

## Testing
- [x] These are tests

Dylan Hurd · 2025-11-17 19:10:56 -08:00

2b7378ac77

249 Commits