Commit Graph
57 Commits
Author SHA1 Message Date
Alex HornbyandGitHub b877a2041e fix unified_exec::tests::unified_exec_timeouts to use a more unique match value (#9414)
Fix unified_exec_timeouts to use a unique variable value rather than
"codex" which was causing false positives when running tests locally
(presumably from my bash prompts). Discovered while running tests to
validate another change.

Fixes https://github.com/openai/codex/issues/9413

Test Plan:

Ran test locally on my fedora 43 x86_64 machine with:
```
cd codex/cargo-rs
cargo nextest run --all-features --no-fail-fast unified_exec::tests::unified_exec_timeouts
```

Before, unified_exec_timeouts fails:
```
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.38s
────────────
 Nextest run ID fa2b4949-a66c-408c-8002-32c52c70ec4f with nextest profile: default
    Starting 1 test across 107 binaries (3211 tests skipped)
        FAIL [   5.667s] codex-core unified_exec::tests::unified_exec_timeouts
  stdout ───

    running 1 test
    test unified_exec::tests::unified_exec_timeouts ... FAILED

    failures:

    failures:
        unified_exec::tests::unified_exec_timeouts

    test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 774 filtered out; finished in 5.66s

  stderr ───

    thread 'unified_exec::tests::unified_exec_timeouts' (459601) panicked at core/src/unified_exec/mod.rs:381:9:
    timeout too short should yield incomplete output
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

────────────
     Summary [   5.677s] 1 test run: 0 passed, 1 failed, 3211 skipped
        FAIL [   5.667s] codex-core unified_exec::tests::unified_exec_timeouts
error: test run failed

```

After, works:
```
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.34s
────────────
 Nextest run ID f49e9004-e30b-4049-b0ff-283b543a1cd7 with nextest profile: default
    Starting 1 test across 107 binaries (3211 tests skipped)
        SLOW [> 15.000s] codex-core unified_exec::tests::unified_exec_timeouts
        PASS [  17.666s] codex-core unified_exec::tests::unified_exec_timeouts
────────────
     Summary [  17.676s] 1 test run: 1 passed (1 slow), 3211 skipped
```
2026-01-17 09:05:53 -08:00
jif-oaiandGitHub 1668ca726f chore: close pipe on non-pty processes (#9369)
Closing the STDIN of piped process when starting them to avoid commands
like `rg` to wait for content on STDIN and hangs for ever
2026-01-16 15:54:32 +01:00
jif-oaiandGitHub 7fc49697dd feat: CODEX_CI (#9366) 2026-01-16 13:52:16 +00:00
jif-oaiandGitHub 577e1fd1b2 feat: adding piped process to replace PTY when needed (#8797) 2026-01-14 18:44:04 +00:00
jif-oaiandGitHub 32b1795ff4 chore: clamp min yield time for empty write_stdin (#9156)
After evals, 0 impact on performance
2026-01-14 16:25:40 +00:00
jif-oaiandGitHub 7532f34699 fix: drop double waiting header in TUI (#9145) 2026-01-14 09:52:34 +00:00
jif-oaiandGitHub 4cef89a122 chore: rename unified exec sessions (#8822)
Renaming done by Codex
2026-01-07 16:12:47 +00:00
jif-oaiandGitHub 32db8ea5ca feat: add head-tail buffer for unified_exec (#8735) 2026-01-06 15:48:44 +00:00
jif-oaiandGitHub 8858012fd1 chore: emit unified exec begin only when PTY exist (#8780) 2026-01-06 13:12:54 +00:00
jif-oaiandGitHub fabb797097 chore: GH pager (#8747) 2026-01-05 18:40:34 +00:00
Shijie RaoandGitHub fb24c47bea fix: limit output size for exec command in unified exec (#8460)
### Issue
[Investigation
thread](https://openai.slack.com/archives/C095U48JNL9/p1766426234975789)

Github issue: [issue one](https://github.com/openai/codex/issues/8197),
[issue two](https://github.com/openai/codex/issues/8358), [issue
three](https://github.com/openai/codex/issues/7585)
Commonality: working in monorepo and large projects. Multiple threads
going and showing sluggishness ending with a crash and grey background.
Potential high usage of context.
How to reproduce:
* Open the whole monorepo with cursor or VSC on the latest extension.
* Run ls -R in current CWD.

### Change
In unified exec, we do not have max output check to the delta we ouput
for shell commands. This causes issue with our VSCE UI which take the
delta and construct the shell output.
2025-12-23 14:36:15 -08:00
pakrym-oaiandGitHub 96fdbdd434 Add ExecPolicyManager (#8349)
Move exec policy management into services to keep turn context
immutable.
2025-12-22 09:59:32 -08:00
Michael BolinandGitHub 3d4ced3ff5 chore: migrate from Config::load_from_base_config_with_overrides to ConfigBuilder (#8276)
https://github.com/openai/codex/pull/8235 introduced `ConfigBuilder` and
this PR updates all call non-test call sites to use it instead of
`Config::load_from_base_config_with_overrides()`.

This is important because `load_from_base_config_with_overrides()` uses
an empty `ConfigRequirements`, which is a reasonable default for testing
so the tests are not influenced by the settings on the host. This method
is now guarded by `#[cfg(test)]` so it cannot be used by business logic.

Because `ConfigBuilder::build()` is `async`, many of the test methods
had to be migrated to be `async`, as well. On the bright side, this made
it possible to eliminate a bunch of `block_on_future()` stuff.
2025-12-18 16:12:52 -08:00
jif-oaiandGitHub 0abaf1b57c nit: prevent race in event rendering (#8181) 2025-12-17 13:24:02 +00:00
zhao-oaiandGitHub e0fb3ca1db refactoring with_escalated_permissions to use SandboxPermissions instead (#7750)
helpful in the future if we want more granularity for requesting
escalated permissions:
e.g when running in readonly sandbox, model can request to escalate to a
sandbox that allows writes
2025-12-10 17:18:48 +00:00
jif-oaiandGitHub 463249eff3 fix: flaky test 2 (#7818) 2025-12-10 16:35:28 +00:00
jif-oaiandGitHub 0ad54982ae chore: rework unified exec events (#7775) 2025-12-10 10:30:38 +00:00
pakrym-oaiandGitHub ac5fa6baf8 Do not emit start/end events for write stdin (#7561) 2025-12-08 15:23:02 -08:00
jif-oaiandGitHub 36edb412b1 fix: release session ID when not used (#7592) 2025-12-04 17:42:16 +00:00
pakrym-oaiandGitHub f1b7cdc3bd Use shared check sandboxing (#7547) 2025-12-04 08:34:09 -08:00
pakrym-oaiandGitHub c4e18f1b63 Slightly better status display for unified exec (#7563)
Trim bash -lc
2025-12-04 08:32:54 -08:00
zhao-oaiandGitHub e925a380dc whitelist command prefix integration in core and tui (#7033)
this PR enables TUI to approve commands and add their prefixes to an
allowlist:
<img width="708" height="605" alt="Screenshot 2025-11-21 at 4 18 07 PM"
src="https://github.com/user-attachments/assets/56a19893-4553-4770-a881-becf79eeda32"
/>

note: we only show the option to whitelist the command when 
1) command is not multi-part (e.g `git add -A && git commit -m 'hello
world'`)
2) command is not already matched by an existing rule
2025-12-03 23:17:02 -08:00
jif-oaiandGitHub 8d0f023fa9 chore: update unified exec sandboxing detection (#7541)
No integration test for now because it would make them flaky. Tracking
it in my todos to add some once we have a clock based system for
integration tests
2025-12-03 20:06:47 +00:00
Michael BolinandGitHub ec93b6daf3 chore: make create_approval_requirement_for_command an async fn (#7501)
I think this might help with https://github.com/openai/codex/pull/7033
because `create_approval_requirement_for_command()` will soon need
access to `Session.state`, which is a `tokio::sync::Mutex` that needs to
be accessed via `async`.
2025-12-02 15:01:15 -08:00
jif-oaiandGitHub 9ee855ec57 feat: add warning message for the model (#7445)
Add a warning message as a user turn to the model if the model does not
behave as expected (here, for example, if the model opens too many
`unified_exec` sessions)
2025-12-02 11:56:00 +00:00
jif-oaiandGitHub 457c9fdb87 chore: better session recycling (#7368) 2025-11-30 12:42:26 -08:00
jif-oaiandGitHub 28ff364c3a feat: update process ID for event handling (#7261) 2025-11-25 14:21:05 -08:00
jif-oaiandGitHub 2845e2c006 fix: drop conversation when /new (#7297) 2025-11-25 17:20:25 +00:00
jif-oaiandGitHub 37d83e075e feat: add custom env for unified exec process (#7286) 2025-11-25 10:35:35 +00:00
jif-oaiandGitHub fc2ff624ac fix: don't store early exit sessions (#7263) 2025-11-24 21:14:24 +00:00
jif-oaiandGitHub b2cddec3d7 feat: unified exec basic pruning strategy (#7239)
LRU + exited sessions first
2025-11-24 17:22:32 +00:00
b5dd189067 Allow unified_exec to early exit (if the process terminates before yield_time_ms) (#6867)
Thread through an `exit_notify` tokio `Notify` through to the
`UnifiedExecSession` so that we can return early if the command
terminates before `yield_time_ms`.

As Codex review correctly pointed out below 🙌 we also need a
`exit_signaled` flag so that commands which finish before we start
waiting can also exit early.

Since the default `yield_time_ms` is now 10s, this means that we don't
have to wait 10s for trivial commands like ls, sed, etc (which are the
majority of agent commands 😅)

---------

Co-authored-by: jif-oai <jif@openai.com>
2025-11-20 13:34:41 +01:00
65c13f1ae7 execpolicy2 core integration (#6641)
This PR threads execpolicy2 into codex-core.

activated via feature flag: exec_policy (on by default)

reads and parses all .codexpolicy files in `codex_home/codex`

refactored tool runtime API to integrate execpolicy logic

---------

Co-authored-by: Michael Bolin <mbolin@openai.com>
2025-11-19 16:50:43 -08:00
efebc62fb7 Move shell to use truncate_text (#6842)
Move shell to use the configurable `truncate_text`

---------

Co-authored-by: pakrym-oai <pakrym@openai.com>
2025-11-19 01:56:08 -08:00
Ahmed IbrahimandGitHub 3de8790714 Add the utility to truncate by tokens (#6746)
- This PR is to make it on path for truncating by tokens. This path will
be initially used by unified exec and context manager (responsible for
MCP calls mainly).
- We are exposing new config `calls_output_max_tokens`
- Use `tokens` as the main budget unit but truncate based on the model
family by Introducing `TruncationPolicy`.
- Introduce `truncate_text` as a router for truncation based on the
mode.

In next PRs:
- remove truncate_with_line_bytes_budget
- Add the ability to the model to override the token budget.
2025-11-18 11:36:23 -08:00
Ahmed IbrahimandGitHub 94dfb211af Refactor truncation helpers into its own file (#6683)
That's to centralize the truncation in one place. Next step would be to
make only two methods public: one with bytes/lines and one with tokens.
2025-11-15 06:44:23 +00:00
jif-oaiandGitHub 63c8c01f40 feat: better UI for unified_exec (#6515)
<img width="376" height="132" alt="Screenshot 2025-11-12 at 17 36 22"
src="https://github.com/user-attachments/assets/ce693f0d-5ca0-462e-b170-c20811dcc8d5"
/>
2025-11-14 16:31:12 +01:00
pakrym-oaiandGitHub 7b027e7536 Revert "Revert "Overhaul shell detection and centralize command generation for unified exec"" (#6607)
Reverts openai/codex#6606
2025-11-13 16:45:17 -08:00
pakrym-oaiandGitHub 0792a7953d Update default yield time (#6610)
10s for exec and 250ms for write_stdin
2025-11-13 10:24:41 -08:00
pakrym-oaiandGitHub e6995174c1 Revert "Overhaul shell detection and centralize command generation for unified exec" (#6606)
Reverts openai/codex#6577
2025-11-13 08:43:00 -08:00
d28e912214 Overhaul shell detection and centralize command generation for unified exec (#6577)
This fixes command display for unified exec. All `cd`s and `ls`es are
now parsed.

<img width="452" height="237" alt="image"
src="https://github.com/user-attachments/assets/ce92d81f-f74c-485a-9b34-1eaa29290ec6"
/>

Deletes a ton of tests that were doing nothing from shell.rs.

---------

Co-authored-by: Pavel Krymets <pavel@krymets.com>
2025-11-13 08:28:09 -08:00
pakrym-oaiandGitHub 807e2c27f0 Add unified exec escalation handling and tests (#6492)
Similar implementation to the shell tool
2025-11-11 08:19:35 -08:00
pakrym-oaiandGitHub 6c36318bd8 Use codex-linux-sandbox in unified exec (#6480)
Unified exec isn't working on Linux because we don't provide the correct
arg0.

The library we use for pty management doesn't allow setting arg0
separately from executable. Use the same aliasing strategy we use for
`apply_patch` for `codex-linux-sandbox`.

Use `#[ctor]` hack to dispatch codex-linux-sandbox calls.


Addresses https://github.com/openai/codex/issues/6450
2025-11-10 17:17:09 -08:00
jif-oaiandGitHub f01f2ec9ee feat: add workdir to unified_exec (#6466) 2025-11-10 19:53:36 +00:00
jif-oaiandGitHub a6b9471548 feat: end events on unified exec (#5551) 2025-10-23 18:51:34 +01:00
jif-oaiandGitHub 00b1e130b3 chore: align unified_exec (#5442)
Align `unified_exec` with b implementation
2025-10-22 11:50:18 +01:00
jif-oaiandGitHub 4bd68e4d9e feat: emit events for unified_exec (#5448) 2025-10-21 17:32:39 +01:00
pakrym-oaiandGitHub 789e65b9d2 Pass TurnContext around instead of sub_id (#5421)
Today `sub_id` is an ID of a single incoming Codex Op submition. We then
associate all events triggered by this operation using the same
`sub_id`.

At the same time we are also creating a TurnContext per submission and
we'd like to start associating some events (item added/item completed)
with an entire turn instead of just the operation that started it.

Using turn context when sending events give us flexibility to change
notification scheme.
2025-10-21 08:04:16 -07:00
jif-oaiandGitHub 5e4f3bbb0b chore: rework tools execution workflow (#5278)
Re-work the tool execution flow. Read `orchestrator.rs` to understand
the structure
2025-10-20 20:57:37 +01:00
jif-oaiandGitHub 3ddd4d47d0 fix: lagged output in unified_exec buffer (#4992)
Handle `Lagged` error when the broadcast buffer of the unified_exec is
full
2025-10-09 16:06:07 +00:00