codex

fix: start publishing @openai/codex-shell-tool-mcp to npm (#7123 )

Start publishing as part of the normal release process.

Michael Bolin · 2025-11-21 15:03:50 -08:00

af63e6eccc

bypass sandbox for policy approved commands (#7110 )

allowing cmds greenlit by execpolicy to bypass sandbox + minor refactor
for a world where we have execpolicy rules with specific sandbox
requirements

zhao-oai · 2025-11-21 18:03:23 -05:00

87b211709e

refactor: inline sandbox type lookup in process_exec_tool_call (#7122 )

`process_exec_tool_call()` was taking `SandboxType` as a param, but in
practice, the only place it was constructed was in
`codex_message_processor.rs` where it was derived from the other
`sandbox_policy` param, so this PR inlines the logic that decides the
`SandboxType` into `process_exec_tool_call()`.



---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/7122).
* #7112
* __->__ #7122

Michael Bolin · 2025-11-21 22:53:05 +00:00

67975ed33a

support MCP elicitations (#6947 )

No support for request schema yet, but we'll at least show the message
and allow accept/decline.

<img width="823" height="551" alt="Screenshot 2025-11-21 at 2 44 05 PM"
src="https://github.com/user-attachments/assets/6fbb892d-ca12-4765-921e-9ac4b217534d"
/>

Jeremy Rose · 2025-11-21 14:44:53 -08:00

7561a6aaf0

fix(tui): Fail when stdin is not a terminal (#6382 )

Piping to codex fails to do anything useful and locks up the process.
We currently check for stdout, but not stdin

```
❯ echo foo|just c
cargo run --bin codex -- "$@"
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.21s
     Running `target/debug/codex`
Error: stdin is not a terminal
error: Recipe `codex` failed on line 10 with exit code 1
```

Josh McKinney · 2025-11-21 14:17:40 -08:00

3ea33a0616

feat: support login as an option on shell-tool-mcp (#7120 )

The unified exec tool has a `login` option that defaults to `true`:


https://github.com/openai/codex/blob/3bdcbc72927acd3e0071da3df3247e1da7ec578c/codex-rs/core/src/tools/handlers/unified_exec.rs#L35-L36

This updates the `ExecParams` for `shell-tool-mcp` to support the same
parameter. Note it is declared as `Option<bool>` to ensure it is marked
optional in the generated JSON schema.

Michael Bolin · 2025-11-21 22:14:41 +00:00

e8ef6d3c16

Use use_model (#7121 )

pakrym-oai · 2025-11-21 22:10:52 +00:00

e52cc38dfd

Windows: flag some invocations that launch browsers/URLs as dangerous (#7111 )

Prevent certain Powershell/cmd invocations from reaching the sandbox
when they are trying to launch a browser, or run a command with a URL,
etc.

iceweasel-oai · 2025-11-21 13:36:17 -08:00

3bdcbc7292

[app-server] doc: approvals (#7105 )

Add documentation for shell and apply_patch approvals

Owen Lin · 2025-11-21 21:27:54 +00:00

a0434bbdb4

enable unified exec for experiments (#7118 )

Ahmed Ibrahim · 2025-11-21 13:10:01 -08:00

d5f661c91d

feat: Add exp model to experiment with the tools (#7115 )

Ahmed Ibrahim · 2025-11-21 12:44:47 -08:00

8ecaad948b

[app-server] feat: expose gitInfo/cwd/etc. on Thread (#7060 )

Port the new additions from https://github.com/openai/codex/pull/6337 on
the legacy API to v2. Mainly need `gitInfo` and `cwd` for VSCE.

Owen Lin · 2025-11-21 10:37:12 -08:00

aa4e0d823e

fix(scripts) next_minor_version should reset patch number (#7050 )

## Summary
When incrementing the minor version, we should reset patch to 0, rather
than keeping it.

## Testing
- [x] tested locally with dry_run and `get_latest_release_version`
mocked out

---------

Co-authored-by: Michael Bolin <mbolin@openai.com>

Dylan Hurd · 2025-11-21 10:17:12 -08:00

0e051644a9

fix: clear out duplicate entries for bash in the GitHub release (#7103 )

https://github.com/openai/codex/pull/7005 introduced a new part of the
release process that added multiple files named `bash` in the `dist/`
folder used as the basis of the GitHub Release. I believe that all file
names in a GitHub Release have to be unique, which is why the recent
release build failed:

https://github.com/openai/codex/actions/runs/19577669780/job/56070183504

Based on the output of the **List** step, I believe these are the
appropriate artifacts to delete as a quick fix.

Michael Bolin · 2025-11-21 09:59:30 -08:00

40d14c0756

chore: drop model_max_output_tokens (#7100 )

jif-oai · 2025-11-21 17:42:54 +00:00

af65666561

[app-server] feat: add Declined status for command exec (#7101 )

Add a `Declined` status for when we request an approval from the user
and the user declines. This allows us to distinguish from commands that
actually ran, but failed.

This behaves similarly to apply_patch / FileChange, which does the same
thing.

Owen Lin · 2025-11-21 09:19:39 -08:00

2ae1f81d84

feat: codex-shell-tool-mcp (#7005 )

This adds a GitHub workflow for building a new npm module we are
experimenting with that contains an MCP server for running Bash
commands. The new workflow, `shell-tool-mcp`, is a dependency of the
general `release` workflow so that we continue to use one version number
for all artifacts across the project in one GitHub release.

`.github/workflows/shell-tool-mcp.yml` is the primary workflow
introduced by this PR, which does the following:

- builds the `codex-exec-mcp-server` and `codex-execve-wrapper`
executables for both arm64 and x64 versions of Mac and Linux (preferring
the MUSL version for Linux)
- builds Bash (dynamically linked) for a [comically] large number of
platforms (both x64 and arm64 for most) with a small patch specified by
`shell-tool-mcp/patches/bash-exec-wrapper.patch`:
  - `debian-11`
  - `debian-12`
  - `ubuntu-20.04`
  - `ubuntu-22.04`
  - `ubuntu-24.04`
  - `centos-9`
  - `macos-13` (x64 only)
  - `macos-14` (arm64 only)
  - `macos-15` (arm64 only)
- builds the TypeScript for the [new] Node module declared in the
`shell-tool-mcp/` folder, which creates `bin/mcp-server.js`
- adds all of the native binaries to `shell-tool-mcp/vendor/` folder;
`bin/mcp-server.js` does a runtime check to determine which ones to
execute
- uses `npm pack` to create the `.tgz` for the module
- if `publish: true` is set, invokes the `npm publish` call with the
`.tgz`

The justification for building Bash for so many different operating
systems is because, since it is dynamically linked, we want to increase
our confidence that the version we build is compatible with the glibc
whatever OS we end up running on. (Note this is less of a concern with
`codex-exec-mcp-server` and `codex-execve-wrapper` on Linux, as they are
statically linked.)

This PR also introduces the code for the npm module in `shell-tool-mcp/`
(the proposed module name is `@openai/codex-shell-tool-mcp`). Initially,
I intended the module to be a single file of vanilla JavaScript (like
[`codex-cli/bin/codex.js`](https://github.com/openai/codex/blob/ab5972d447da78d3e4dd8461cf7d43a22e5d2acb/codex-cli/bin/codex.js)),
but some of the logic seemed a bit tricky, so I decided to port it to
TypeScript and add unit tests.

`shell-tool-mcp/src/index.ts` defines the `main()` function for the
module, which performs runtime checks to determine the clang triple to
find the path to the Rust executables within the `vendor/` folder
(`resolveTargetTriple()`). It uses a combination of `readOsRelease()`
and `resolveBashPath()` to determine the correct Bash executable to run
in the environment. Ultimately, it spawns a command like the following:

```
codex-exec-mcp-server \
    --execve codex-execve-wrapper \
    --bash custom-bash "$@"
```

Note `.github/workflows/shell-tool-mcp-ci.yml` defines a fairly standard
CI job for the module (`format`/`build`/`test`).

To test this PR, I pushed this branch to my personal fork of Codex and
ran the CI job there:

https://github.com/bolinfest/codex/actions/runs/19564311320

Admittedly, the graph looks a bit wild now:

<img width="5115" height="2969" alt="Screenshot 2025-11-20 at 11 44
58 PM"
src="https://github.com/user-attachments/assets/cc5ef306-efc1-4ed7-a137-5347e394f393"
/>

But when it finished, I was able to download `codex-shell-tool-mcp-npm`
from the **Artifacts** for the workflow in an empty temp directory,
unzip the `.zip` and then the `.tgz` inside it, followed by `xattr -rc
.` to remove the quarantine bits. Then I ran:

```shell
npx @modelcontextprotocol/inspector node /private/tmp/foobar4/package/bin/mcp-server.js
```

which launched the MCP Inspector and I was able to use it as expected!
This bodes well that this should work once the package is published to
npm:

```shell
npx @modelcontextprotocol/inspector npx @openai/codex-shell-tool-mcp
```

Also, to verify the package contains what I expect:

```shell
/tmp/foobar4/package$ tree
.
├── bin
│   └── mcp-server.js
├── package.json
├── README.md
└── vendor
    ├── aarch64-apple-darwin
    │   ├── bash
    │   │   ├── macos-14
    │   │   │   └── bash
    │   │   └── macos-15
    │   │       └── bash
    │   ├── codex-exec-mcp-server
    │   └── codex-execve-wrapper
    ├── aarch64-unknown-linux-musl
    │   ├── bash
    │   │   ├── centos-9
    │   │   │   └── bash
    │   │   ├── debian-11
    │   │   │   └── bash
    │   │   ├── debian-12
    │   │   │   └── bash
    │   │   ├── ubuntu-20.04
    │   │   │   └── bash
    │   │   ├── ubuntu-22.04
    │   │   │   └── bash
    │   │   └── ubuntu-24.04
    │   │       └── bash
    │   ├── codex-exec-mcp-server
    │   └── codex-execve-wrapper
    ├── x86_64-apple-darwin
    │   ├── bash
    │   │   └── macos-13
    │   │       └── bash
    │   ├── codex-exec-mcp-server
    │   └── codex-execve-wrapper
    └── x86_64-unknown-linux-musl
        ├── bash
        │   ├── centos-9
        │   │   └── bash
        │   ├── debian-11
        │   │   └── bash
        │   ├── debian-12
        │   │   └── bash
        │   ├── ubuntu-20.04
        │   │   └── bash
        │   ├── ubuntu-22.04
        │   │   └── bash
        │   └── ubuntu-24.04
        │       └── bash
        ├── codex-exec-mcp-server
        └── codex-execve-wrapper

26 directories, 26 files
```

Michael Bolin · 2025-11-21 08:16:36 -08:00

d363a0968e

Revert "fix: read max_output_tokens param from config" (#7088 )

Reverts openai/codex#4139

jif-oai · 2025-11-21 11:40:02 +01:00

bce030ddb5

Windows Sandbox: support network_access and exclude_tmpdir_env_var (#7030 )

iceweasel-oai · 2025-11-20 22:59:55 -08:00

f4af6e389e

Fixed the deduplicator github action (#7070 )

It stopped working (found zero duplicates) starting three days ago when
the model was switched from `gpt-5` to `gpt-5.1`. I'm not sure why it
stopped working. This is an attempt to get it working again by using the
default model for the codex action (which is presumably
`gpt-5.1-codex-max`).

Eric Traut · 2025-11-20 22:46:55 -08:00

b315b22f7b

fix: read max_output_tokens param from config (#4139 )

Request param `max_output_tokens` is documented in
`https://github.com/openai/codex/blob/main/docs/config.md`,
but nowhere uses the item in config, this commit read it from config for
GPT responses API.

see https://github.com/openai/codex/issues/4138 for issue report.

Signed-off-by: Yorling <shallowcloud@yeah.net>

Yorling · 2025-11-20 22:46:34 -08:00

c9e149fd5c

Fixed two tests that can fail in some environments that have global git rewrite rules (#7068 )

This fixes https://github.com/openai/codex/issues/7044

Eric Traut · 2025-11-20 22:45:40 -08:00

bacdc004be

Support all types of search actions (#7061 )

Fixes the 

```
{
  "error": {
    "message": "Invalid value: 'other'. Supported values are: 'search', 'open_page', and 'find_in_page'.",
    "type": "invalid_request_error",
    "param": "input[150].action.type",
    "code": "invalid_value"
  }
```
error.


The actual-actual fix here is supporting absent `query` parameter.

pakrym-oai · 2025-11-20 20:45:28 -08:00

ab5972d447

Migrate coverage to shell_command (#7042 )

pakrym-oai · 2025-11-21 03:44:00 +00:00

767b66f407

Support full powershell paths in is_safe_command (#7055 )

New shell implementation always uses full paths.

pakrym-oai · 2025-11-20 19:29:15 -08:00

830ab4ce20

fix(app-server) remove www warning (#7046 )

### Summary
After #7022, we no longer need this warning. We should also clean up the
schema for the notification, but this is a quick fix to just stop the
behavior in the VSCE

## Testing
- [x] Ran locally

Dylan Hurd · 2025-11-20 19:18:39 -08:00

3f73e2c892

feat(tui): default reasoning selection to medium (#7040 )

## Summary
- allow selection popups to request an initial highlighted row
- begin the /models reasoning selector focused on the default effort

## Testing
- just fmt
- just fix -p codex-tui
- cargo test -p codex-tui



https://github.com/user-attachments/assets/b322aeb1-e8f3-4578-92f7-5c2fa5ee4c98



------
[Codex
Task](https://chatgpt.com/codex/tasks/task_i_691f75e8fc188322a910fbe2138666ef)

Dylan Hurd · 2025-11-20 17:06:04 -08:00

1822ffe870

[app-server] update doc with codex error info (#6941 )

Document new codex error info. Also fixed the name from
`codex_error_code` to `codex_error_info`.

Celia Chen · 2025-11-21 01:02:37 +00:00

7e2165f394

feat: waiting for an elicitation should not count against a shell tool timeout (#6973 )

Previously, we were running into an issue where we would run the `shell`
tool call with a timeout of 10s, but it fired an elicitation asking for
user approval, the time the user took to respond to the elicitation was
counted agains the 10s timeout, so the `shell` tool call would fail with
a timeout error unless the user is very fast!

This PR addresses this issue by introducing a "stopwatch" abstraction
that is used to manage the timeout. The idea is:

- `Stopwatch::new()` is called with the _real_ timeout of the `shell`
tool call.
- `process_exec_tool_call()` is called with the `Cancellation` variant
of `ExecExpiration` because it should not manage its own timeout in this
case
- the `Stopwatch` expiration is wired up to the `cancel_rx` passed to
`process_exec_tool_call()`
- when an elicitation for the `shell` tool call is received, the
`Stopwatch` pauses
- because it is possible for multiple elicitations to arrive
concurrently, it keeps track of the number of "active pauses" and does
not resume until that counter goes down to zero

I verified that I can test the MCP server using
`@modelcontextprotocol/inspector` and specify `git status` as the
`command` with a timeout of 500ms and that the elicitation pops up and I
have all the time in the world to respond whereas previous to this PR,
that would not have been possible.

---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/6973).
* #7005
* __->__ #6973
* #6972

Michael Bolin · 2025-11-20 16:45:38 -08:00

8e5f38c0f0

fix flaky tool_call_output_exceeds_limit_truncated_chars_limit (#7043 )

I am suspecting this is flaky because of the wall time can become 0,
0.1, or 1.

Ahmed Ibrahim · 2025-11-20 16:36:29 -08:00

1388e99674

feat: update process_exec_tool_call() to take a cancellation token (#6972 )

This updates `ExecParams` so that instead of taking `timeout_ms:
Option<u64>`, it now takes a more general cancellation mechanism,
`ExecExpiration`, which is an enum that includes a
`Cancellation(tokio_util::sync::CancellationToken)` variant.

If the cancellation token is fired, then `process_exec_tool_call()`
returns in the same way as if a timeout was exceeded.

This is necessary so that in #6973, we can manage the timeout logic
external to the `process_exec_tool_call()` because we want to "suspend"
the timeout when an elicitation from a human user is pending.








---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/6972).
* #7005
* #6973
* __->__ #6972

Michael Bolin · 2025-11-20 16:29:57 -08:00

f56d1dc8fc

migrate collect_tool_identifiers_for_model to test_codex (#7041 )

Maybe it solved flakiness

Ahmed Ibrahim · 2025-11-20 16:02:50 -08:00

9be310041b

core: make shell behavior portable on FreeBSD (#7039 )

- Use /bin/sh instead of /bin/bash on FreeBSD/OpenBSD in the process
group timeout test to avoid command-not-found failures.

- Accept /usr/local/bin/bash as a valid SHELL path to match common
FreeBSD installations.

- Switch the shell serialization duration test to /bin/sh for improved
portability across Unix platforms.

With this change, `cargo test -p codex-core --lib` runs and passes on
FreeBSD.

Xiao-Yong Jin · 2025-11-20 16:01:35 -08:00

0fbcdd77c8

[app-server & core] introduce new codex error code and v2 app-server error events (#6938 )

This PR does two things:
1. populate a new `codex_error_code` protocol in error events sent from
core to client;
2. old v1 core events `codex/event/stream_error` and `codex/event/error`
will now both become `error`. We also show codex error code for
turncompleted -> error status.

new events in app server test:
```
< {
<   "method": "codex/event/stream_error",
<   "params": {
<     "conversationId": "019aa34c-0c14-70e0-9706-98520a760d67",
<     "id": "0",
<     "msg": {
<       "codex_error_code": {
<         "response_stream_disconnected": {
<           "http_status_code": 401
<         }
<       },
<       "message": "Reconnecting... 2/5",
<       "type": "stream_error"
<     }
<   }
< }

 {
<   "method": "error",
<   "params": {
<     "error": {
<       "codexErrorCode": {
<         "responseStreamDisconnected": {
<           "httpStatusCode": 401
<         }
<       },
<       "message": "Reconnecting... 2/5"
<     }
<   }
< }

< {
<   "method": "turn/completed",
<   "params": {
<     "turn": {
<       "error": {
<         "codexErrorCode": {
<           "responseTooManyFailedAttempts": {
<             "httpStatusCode": 401
<           }
<         },
<         "message": "exceeded retry limit, last status: 401 Unauthorized, request id: 9a1b495a1a97ed3e-SJC"
<       },
<       "id": "0",
<       "items": [],
<       "status": "failed"
<     }
<   }
< }
```

Celia Chen · 2025-11-20 23:06:55 +00:00

9bce050385

add deny ACEs for world writable dirs (#7022 )

Our Restricted Token contains 3 SIDs (Logon, Everyone, {WorkspaceWrite
Capability || ReadOnly Capability})

because it must include Everyone, that left us vulnerable to directories
that allow writes to Everyone. Even though those directories do not have
ACEs that enable our capability SIDs to write to them, they could still
be written to even in ReadOnly mode, or even in WorkspaceWrite mode if
they are outside of a writable root.

A solution to this is to explicitly add *Deny* ACEs to these
directories, always for the ReadOnly Capability SID, and for the
WorkspaceWrite SID if the directory is outside of a workspace root.

Under a restricted token, Windows always checks Deny ACEs before Allow
ACEs so even though our restricted token would allow a write to these
directories due to the Everyone SID, it fails first because of the Deny
ACE on the capability SID

iceweasel-oai · 2025-11-20 14:50:33 -08:00

3f92ad4190

Attempt to fix unified_exec_formats_large_output_summary flakiness (#7029 )

second attempt to fix this test after
https://github.com/openai/codex/pull/6884. I think this flakiness is
happening because yield_time is too small for a 10,000 step loop in
python.

Ahmed Ibrahim · 2025-11-20 14:38:04 -08:00

54ee302a06

fix flaky test: approval_matrix_covers_all_modes (#7028 )

looks like it sometimes flake around 30. let's give it more time.

Ahmed Ibrahim · 2025-11-20 14:37:42 -08:00

44fa06ae36

Delete shell_command feature (#7024 )

pakrym-oai · 2025-11-20 14:14:56 -08:00

856f97f449

execpolicycheck command in codex cli (#7012 )

adding execpolicycheck tool onto codex cli

this is useful for validating policies (can be multiple) against
commands.

it will also surface errors in policy syntax:
<img width="1150" height="281" alt="Screenshot 2025-11-19 at 12 46
21 PM"
src="https://github.com/user-attachments/assets/8f99b403-564c-4172-acc9-6574a8d13dc3"
/>

this PR also changes output format when there's no match in the CLI.
instead of returning the raw string `noMatch`, we return
`{"noMatch":{}}`

this PR is a rewrite of: https://github.com/openai/codex/pull/6932 (due
to the numerous merge conflicts present in the original PR)

---------

Co-authored-by: Michael Bolin <mbolin@openai.com>

zhao-oai · 2025-11-20 16:44:31 -05:00

fe7a3f0c2b

increasing user shell timeout to 1 hour (#7025 )

setting user shell timeout to an unreasonably high value since there
isn't an easy way to have a command run without timeouts

currently, user shell commands timeout is 10 seconds

zhao-oai · 2025-11-20 13:39:16 -08:00

c30ca0d5b6

fix: route feedback issue links by category (#6840 )

## Summary
- TUI feedback note now only links to the bug-report template when the
category is bug/bad result.
- Good result/other feedback shows a thank-you+thread ID instead of
funneling people to file a bug.
- Added a helper + unit test so future changes keep the behavior
consistent.

## Testing
  - just fmt
  - just fix -p codex-tui
  - cargo test -p codex-tui

  Fixes #6839

Weiller Carvalho · 2025-11-20 13:20:03 -08:00

a8a6cbdd1c

codex-exec: allow resume --last to read prompt #6717 (#6719 )

### Description

- codex exec --json resume --last "<prompt>" bailed out because clap
treated the prompt as SESSION_ID. I removed the conflicts_with flag and
reinterpret that positional as a prompt when
--last is set, so the flow now keeps working in JSON mode.
(codex-rs/exec/src/cli.rs:84-104, codex-rs/exec/src/lib.rs:75-130)
- Added a regression test that exercises resume --last in JSON mode to
ensure the prompt is accepted and the rollout file is updated.
(codex-rs/exec/tests/suite/resume.rs:126-178)

### Testing

  - just fmt
  - cargo test -p codex-exec
  - just fix -p codex-exec
  - cargo test -p codex-exec

#6717

Signed-off-by: Dmitri Khokhlov <dkhokhlov@cribl.io>

Dmitri Khokhlov · 2025-11-20 13:10:49 -08:00

e4257f432e

tui: centralize markdown styling and make inline code cyan (#7023 )

<img width="762" height="271" alt="Screenshot 2025-11-20 at 12 54 06 PM"
src="https://github.com/user-attachments/assets/10021d63-27eb-407b-8fcc-43740e3bfb0f"
/>

Jeremy Rose · 2025-11-20 21:06:22 +00:00

2c793083f4

Bumped number of fuzzy search results from 8 to 20 (#7013 )

I just noticed that in the VSCode / Codex extension when you type @ the
number of results is around 70:

- small video of searching for `mod.rs` inside `codex` repository:
https://github.com/user-attachments/assets/46e53d31-adff-465e-b32b-051c4c1c298c

- while in the CLI the number of results is currently of 8 which is
quite small:
<img width="615" height="439" alt="Screenshot 2025-11-20 at 09 42 04"
src="https://github.com/user-attachments/assets/1c6d12cb-3b1f-4d5b-9ad3-6b12975eaaec"
/>

I bumped it to 20. I had several cases where I wanted a file and did not
find it because the number of results was too small

Signed-off-by: lionel-oai <lionel@openai.com>
Co-authored-by: lionel-oai <lionel@openai.com>

Lionel Cheng · 2025-11-20 12:33:12 -08:00

e150798baf

fix(cli): correct mcp add usage order (#6827 )

## Summary
- add an explicit `override_usage` string to `AddArgs` so clap prints
`<NAME>` before the command/url choice, matching the actual parser and
docs

### Before

Usage: codex mcp add [OPTIONS] <COMMAND|--url <URL>> <NAME>


### After

Usage: codex mcp add [OPTIONS] <NAME> [--url <URL> | -- <COMMAND>...]

---------

Signed-off-by: kyuheon-kr <kyuheon.kr@gmail.com>

Kyuheon Kim · 2025-11-20 12:32:12 -08:00

33a6cc66ab

Delete tiktoken-rs (#7018 )

pakrym-oai · 2025-11-20 11:15:04 -08:00

52d0ec4cd8

Fix: Improve text encoding for shell output in VSCode preview (#6178 ) (#6182 )

## 🐛 Problem

Users running commands with non-ASCII characters (like Russian text
"пример") in Windows/WSL environments experience garbled text in
VSCode's shell preview window, with Unicode replacement characters (�)
appearing instead of the actual text.

**Issue**: https://github.com/openai/codex/issues/6178

## 🔧 Root Cause

The issue was in `StreamOutput<Vec<u8>>::from_utf8_lossy()` method in
`codex-rs/core/src/exec.rs`, which used `String::from_utf8_lossy()` to
convert shell output bytes to strings. This function immediately
replaces any invalid UTF-8 byte sequences with replacement characters,
without attempting to decode using other common encodings.

In Windows/WSL environments, shell output often uses encodings like:

- Windows-1252 (common Windows encoding)
- Latin-1/ISO-8859-1 (extended ASCII)

## 🛠️ Solution

Replaced the simple `String::from_utf8_lossy()` call with intelligent
encoding detection via a new `bytes_to_string_smart()` function that
tries multiple encoding strategies:

1. **UTF-8** (fast path for valid UTF-8)
2. **Windows-1252** (handles Windows-specific characters in 0x80-0x9F
range)
3. **Latin-1** (fallback for extended ASCII)
4. **Lossy UTF-8** (final fallback, same as before)

## 📁 Changes

### New Files

- `codex-rs/core/src/text_encoding.rs` - Smart encoding detection module
- `codex-rs/core/tests/suite/text_encoding_fix.rs` - Integration tests

### Modified Files

- `codex-rs/core/src/lib.rs` - Added text_encoding module
- `codex-rs/core/src/exec.rs` - Updated StreamOutput::from_utf8_lossy()
- `codex-rs/core/tests/suite/mod.rs` - Registered new test module

## ✅ Testing

- **5 unit tests** covering UTF-8, Windows-1252, Latin-1, and fallback
scenarios
- **2 integration tests** simulating the exact Issue #6178 scenario
- **Demonstrates improvement** over the previous
`String::from_utf8_lossy()` approach

All tests pass:

```bash
cargo test -p codex-core text_encoding
cargo test -p codex-core test_shell_output_encoding_issue_6178
```

## 🎯 Impact

- ✅ **Eliminates garbled text** in VSCode shell preview for non-ASCII
content
- ✅ **Supports Windows/WSL environments** with proper encoding detection
- ✅ **Zero performance impact** for UTF-8 text (fast path)
- ✅ **Backward compatible** - UTF-8 content works exactly as before
- ✅ **Handles edge cases** with robust fallback mechanism

## 🧪 Test Scenarios

The fix has been tested with:

- Russian text ("пример")
- Windows-1252 quotation marks (""test")
- Latin-1 accented characters ("café")
- Mixed encoding content
- Invalid byte sequences (graceful fallback)

## 📋 Checklist

- [X] Addresses the reported issue
- [X] Includes comprehensive tests
- [X] Maintains backward compatibility
- [X] Follows project coding conventions
- [X] No breaking changes

---------

Co-authored-by: Josh McKinney <joshka@openai.com>

LIHUA · 2025-11-20 11:04:11 -08:00

397279d46e

Always fallback to real shell (#6953 )

Either cmd.exe or `/bin/sh`.

pakrym-oai · 2025-11-20 10:58:46 -08:00

30ca89424c

Added feature switch to disable animations in TUI (#6870 )

This PR adds support for a new feature flag `tui.animations`. By
default, the TUI uses animations in its welcome screen, "working"
spinners, and "shimmer" effects. This animations can interfere with
screen readers, so it's good to provide a way to disable them.

This change is inspired by [a
PR](https://github.com/openai/codex/pull/4014) contributed by @Orinks.
That PR has faltered a bit, but I think the core idea is sound. This
version incorporates feedback from @aibrahim-oai. In particular:
1. It uses a feature flag (`tui.animations`) rather than the unqualified
CLI key `no-animations`. Feature flags are the preferred way to expose
boolean switches. They are also exposed via CLI command switches.
2. It includes more complete documentation.
3. It disables a few animations that the other PR omitted.

Eric Traut · 2025-11-20 10:40:08 -08:00

d909048a85

fix: command formatting for user commands (#7002 )

jif-oai · 2025-11-20 17:29:15 +01:00

888c6dd9e7

2126 Commits