Commit Graph

77 Commits

  • utils/pty: add streaming spawn and terminal sizing primitives (#13695)
    Enhance pty utils:
    * Support closing stdin
    * Separate stderr and stdout streams to allow consumers differentiate them
    * Provide compatibility helper to merge both streams back into combined one
    * Support specifying terminal size for pty, including on-demand resizes while process is already running
    * Support terminating the process while still consuming its outputs
  • refactor: prepare unified exec for zsh-fork backend (#13392)
    ## Why
    
    `shell_zsh_fork` already provides stronger guarantees around which
    executables receive elevated permissions. To reuse that machinery from
    unified exec without pushing Unix-specific escalation details through
    generic runtime code, the escalation bootstrap and session lifetime
    handling need a cleaner boundary.
    
    That boundary also needs to be safe for long-lived sessions: when an
    intercepted shell session is closed or pruned, any in-flight approval
    workers and any already-approved escalated child they spawned must be
    torn down with the session, and the inherited escalation socket must not
    leak into unrelated subprocesses.
    
    ## What Changed
    
    - Extracted a reusable `EscalationSession` and
    `EscalateServer::start_session(...)` in `shell-escalation` so callers
    can get the wrapper/socket env overlay and keep the escalation server
    alive without immediately running a one-shot command.
    - Documented that `EscalationSession::env()` and
    `ShellCommandExecutor::run(...)` exchange only that env overlay, which
    callers must merge into their own base shell environment.
    - Clarified the prepared-exec helper boundary in `core` by naming the
    new helper APIs around `ExecRequest`, while keeping the legacy
    `execute_env(...)` entrypoints as thin compatibility wrappers for
    existing callers that still use the older naming.
    - Added a small post-spawn hook on the prepared execution path so the
    parent copy of the inheritable escalation socket is closed immediately
    after both the existing one-shot shell-command spawn and the
    unified-exec spawn.
    - Made session teardown explicit with session-scoped cancellation:
    dropping an `EscalationSession` or canceling its parent request now
    stops intercept workers, and the server-spawned escalated child uses
    `kill_on_drop(true)` so teardown cannot orphan an already-approved
    child.
    - Added `UnifiedExecBackendConfig` plumbing through `ToolsConfig`, a
    `shell::zsh_fork_backend` facade, and an opaque unified-exec
    spawn-lifecycle hook so unified exec can prepare a wrapped `zsh -c/-lc`
    request without storing `EscalationSession` directly in generic
    process/runtime code.
    - Kept the existing `shell_command` zsh-fork behavior intact on top of
    the new bootstrap path. Tool selection is unchanged in this PR: when
    `shell_zsh_fork` is enabled, `ShellCommand` still wins over
    `exec_command`.
    
    ## Verification
    
    - `cargo test -p codex-shell-escalation`
      - includes coverage for `start_session_exposes_wrapper_env_overlay`
      - includes coverage for `exec_closes_parent_socket_after_shell_spawn`
    - includes coverage for
    `dropping_session_aborts_intercept_workers_and_kills_spawned_child`
    - `cargo test -p codex-core
    shell_zsh_fork_prefers_shell_command_over_unified_exec`
    - `cargo test -p codex-core --test all
    shell_zsh_fork_prompts_for_skill_script_execution`
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/13392).
    * #13432
    * __->__ #13392
  • chore: migrate additional permissions to PermissionProfile (#12731)
    This PR replaces the old `additional_permissions.fs_read/fs_write` shape
    with a shared `PermissionProfile`
    model and wires it through the command approval, sandboxing, protocol,
    and TUI layers. The schema is adopted from the
    `SkillManifestPermissions`, which is also refactored to use this unified
    struct. This helps us easily expose permission profiles in app
    server/core as a follow-up.
  • feat(core) Introduce Feature::RequestPermissions (#11871)
    ## Summary
    Introduces the initial implementation of Feature::RequestPermissions.
    RequestPermissions allows the model to request that a command be run
    inside the sandbox, with additional permissions, like writing to a
    specific folder. Eventually this will include other rules as well, and
    the ability to persist these permissions, but this PR is already quite
    large - let's get the core flow working and go from there!
    
    <img width="1279" height="541" alt="Screenshot 2026-02-15 at 2 26 22 PM"
    src="https://github.com/user-attachments/assets/0ee3ec0f-02ec-4509-91a2-809ac80be368"
    />
    
    ## Testing
    - [x] Added tests
    - [x] Tested locally
    - [x] Feature
  • Use Arc-based ToolCtx in tool runtimes (#12583)
    ## Why
    Tool handlers and runtimes needed to pass the same turn/session context
    for shell and non-shell workflows without duplicative ownership churn.
    Using shared pointers avoids temporary lifetimes and keeps existing
    behavior unchanged while simplifying call sites.
    
    ## What changed
    - Converted `ToolCtx` to store shared context handles (`Arc`-based),
    including updates across shell, apply-patch, and unified-exec paths.
    - Updated orchestrator/runtime call sites to consume the shared context
    consistently and remove brittle move/borrow patterns.
    - Kept behavior unchanged while preparing the type surface for the new
    shell escalation integration in the next stack commit.
    
    ## Verification
    - Validated this commit stack point with `just clippy` and confirmed
    workspace compiles cleanly in this stack state.
    
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/12583).
    * #12584
    * __->__ #12583
    * #12556
  • Refactor network approvals to host/protocol/port scope (#12140)
    ## Summary
    Simplify network approvals by removing per-attempt proxy correlation and
    moving to session-level approval dedupe keyed by (host, protocol, port).
    Instead of encoding attempt IDs into proxy credentials/URLs, we now
    treat approvals as a destination policy decision.
    
    - Concurrent calls to the same destination share one approval prompt.
    - Different destinations (or same host on different ports) get separate
    prompts.
    - Allow once approves the current queued request group only.
    - Allow for session caches that (host, protocol, port) and auto-allows
    future matching requests.
    - Never policy continues to deny without prompting.
    
    Example:
    - 3 calls: 
      - a.com (line 443)
      - b.com (line 443)
      - a.com (line 443)
    => 2 prompts total (a, b), second a waits on the first decision.
    - a.com:80 is treated separately from a.com line 443
    
    ## Testing
    - `just fmt` (in `codex-rs`)
    - `cargo test -p codex-core tools::network_approval::tests`
    - `cargo test -p codex-core` (unit tests pass; existing
    integration-suite failures remain in this environment)
  • feat: add configurable write_stdin timeout (#12228)
    Add max timeout as config for `write_stdin`. This is only used for empty
    `write_stdin`.
    
    Also increased the default value from 30s to 5mins.
  • feat: use shell policy in shell snapshot (#11759)
    Honor `shell_environment_policy.set` even after a shell snapshot
  • feat(core): add structured network approval plumbing and policy decision model (#11672)
    ### Description
    #### Summary
    Introduces the core plumbing required for structured network approvals
    
    #### What changed
    - Added structured network policy decision modeling in core.
    - Added approval payload/context types needed for network approval
    semantics.
    - Wired shell/unified-exec runtime plumbing to consume structured
    decisions.
    - Updated related core error/event surfaces for structured handling.
    - Updated protocol plumbing used by core approval flow.
    - Included small CLI debug sandbox compatibility updates needed by this
    layer.
    
    #### Why
    establishes the minimal backend foundation for network approvals without
    yet changing high-level orchestration or TUI behavior.
    
    #### Notes
    - Behavior remains constrained by existing requirements/config gating.
    - Follow-up PRs in the stack handle orchestration, UX, and app-server
    integration.
    
    ---------
    
    Co-authored-by: Codex <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
  • Remove test-support feature from codex-core and replace it with explicit test toggles (#11405)
    ## Why
    
    `codex-core` was being built in multiple feature-resolved permutations
    because test-only behavior was modeled as crate features. For a large
    crate, those permutations increase compile cost and reduce cache reuse.
    
    ## Net Change
    
    - Removed the `test-support` crate feature and related feature wiring so
    `codex-core` no longer needs separate feature shapes for test consumers.
    - Standardized cross-crate test-only access behind
    `codex_core::test_support`.
    - External test code now imports helpers from
    `codex_core::test_support`.
    - Underlying implementation hooks are kept internal (`pub(crate)`)
    instead of broadly public.
    
    ## Outcome
    
    - Fewer `codex-core` build permutations.
    - Better incremental cache reuse across test targets.
    - No intended production behavior change.
  • Remove deterministic_process_ids feature to avoid duplicate codex-core builds (#11393)
    ## Why
    
    `codex-core` enabled `deterministic_process_ids` through a self
    dev-dependency.
    That forced a second feature-resolved build of the same crate, which
    increased
    compile time and test latency.
    
    ## What Changed
    
    - Removed the `deterministic_process_ids` feature from
    `codex-rs/core/Cargo.toml`.
    - Removed the self dev-dependency on `codex-core` that enabled that
    feature.
    - Removed the Bazel `deterministic_process_ids` crate feature for
    `codex-core`.
    - Added a test-only `AtomicBool` override in unified exec process-id
    allocation.
    - Added a test-support setter for that override and re-exported it from
    `codex-core`.
    - Enabled deterministic process IDs in integration tests via
    `core_test_support` ctor.
    
    ## Behavior
    
    - Production behavior remains random process IDs.
    - Unit tests remain deterministic via `cfg(test)`.
    - Integration tests remain deterministic via explicit test-support
    initialization.
    
    ## Validation
    
    - `just fmt`
    - `cargo test -p codex-core unified_exec::`
    - `cargo test -p codex-core --test all unified_exec -- --test-threads=1`
    - `cargo tree -p codex-core -e features` (verified the removed feature
    path)
  • feat(sandbox): enforce proxy-aware network routing in sandbox (#11113)
    ## Summary
    - expand proxy env injection to cover common tool env vars
    (`HTTP_PROXY`/`HTTPS_PROXY`/`ALL_PROXY`/`NO_PROXY` families +
    tool-specific variants)
    - harden macOS Seatbelt network policy generation to route through
    inferred loopback proxy endpoints and fail closed when proxy env is
    malformed
    - thread proxy-aware Linux sandbox flags and add minimal bwrap netns
    isolation hook for restricted non-proxy runs
    - add/refresh tests for proxy env wiring, Seatbelt policy generation,
    and Linux sandbox argument wiring
  • feat: include NetworkConfig through ExecParams (#11105)
    This PR adds the following field to `Config`:
    
    ```rust
    pub network: Option<NetworkProxy>,
    ```
    
    Though for the moment, it will always be initialized as `None` (this
    will be addressed in a subsequent PR).
    
    This PR does the work to thread `network` through to `execute_exec_env()`, `process_exec_tool_call()`, and `UnifiedExecRuntime.run()` to ensure it is available whenever we span a process.
  • Removed "exec_policy" feature flag (#10851)
    This is no longer needed because it's on by default
  • Inject CODEX_THREAD_ID into the terminal environment (#10096)
    Inject CODEX_THREAD_ID (when applicable) into the terminal environment
    so that the agent (and skills) can refer to the current thread / session
    ID.
    
    Discussion:
    https://openai.slack.com/archives/C095U48JNL9/p1769542492067109
  • feat(core) RequestRule (#9489)
    ## Summary
    Instead of trying to derive the prefix_rule for a command mechanically,
    let's let the model decide for us.
    
    ## Testing
    - [x] tested locally
  • fix unified_exec::tests::unified_exec_timeouts to use a more unique match value (#9414)
    Fix unified_exec_timeouts to use a unique variable value rather than
    "codex" which was causing false positives when running tests locally
    (presumably from my bash prompts). Discovered while running tests to
    validate another change.
    
    Fixes https://github.com/openai/codex/issues/9413
    
    Test Plan:
    
    Ran test locally on my fedora 43 x86_64 machine with:
    ```
    cd codex/cargo-rs
    cargo nextest run --all-features --no-fail-fast unified_exec::tests::unified_exec_timeouts
    ```
    
    Before, unified_exec_timeouts fails:
    ```
        Finished `test` profile [unoptimized + debuginfo] target(s) in 0.38s
    ────────────
     Nextest run ID fa2b4949-a66c-408c-8002-32c52c70ec4f with nextest profile: default
        Starting 1 test across 107 binaries (3211 tests skipped)
            FAIL [   5.667s] codex-core unified_exec::tests::unified_exec_timeouts
      stdout ───
    
        running 1 test
        test unified_exec::tests::unified_exec_timeouts ... FAILED
    
        failures:
    
        failures:
            unified_exec::tests::unified_exec_timeouts
    
        test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 774 filtered out; finished in 5.66s
    
      stderr ───
    
        thread 'unified_exec::tests::unified_exec_timeouts' (459601) panicked at core/src/unified_exec/mod.rs:381:9:
        timeout too short should yield incomplete output
        note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
    
    ────────────
         Summary [   5.677s] 1 test run: 0 passed, 1 failed, 3211 skipped
            FAIL [   5.667s] codex-core unified_exec::tests::unified_exec_timeouts
    error: test run failed
    
    ```
    
    After, works:
    ```
        Finished `test` profile [unoptimized + debuginfo] target(s) in 0.34s
    ────────────
     Nextest run ID f49e9004-e30b-4049-b0ff-283b543a1cd7 with nextest profile: default
        Starting 1 test across 107 binaries (3211 tests skipped)
            SLOW [> 15.000s] codex-core unified_exec::tests::unified_exec_timeouts
            PASS [  17.666s] codex-core unified_exec::tests::unified_exec_timeouts
    ────────────
         Summary [  17.676s] 1 test run: 1 passed (1 slow), 3211 skipped
    ```
  • chore: close pipe on non-pty processes (#9369)
    Closing the STDIN of piped process when starting them to avoid commands
    like `rg` to wait for content on STDIN and hangs for ever
  • chore: clamp min yield time for empty write_stdin (#9156)
    After evals, 0 impact on performance
  • fix: limit output size for exec command in unified exec (#8460)
    ### Issue
    [Investigation
    thread](https://openai.slack.com/archives/C095U48JNL9/p1766426234975789)
    
    Github issue: [issue one](https://github.com/openai/codex/issues/8197),
    [issue two](https://github.com/openai/codex/issues/8358), [issue
    three](https://github.com/openai/codex/issues/7585)
    Commonality: working in monorepo and large projects. Multiple threads
    going and showing sluggishness ending with a crash and grey background.
    Potential high usage of context.
    How to reproduce:
    * Open the whole monorepo with cursor or VSC on the latest extension.
    * Run ls -R in current CWD.
    
    ### Change
    In unified exec, we do not have max output check to the delta we ouput
    for shell commands. This causes issue with our VSCE UI which take the
    delta and construct the shell output.
  • Add ExecPolicyManager (#8349)
    Move exec policy management into services to keep turn context
    immutable.
  • chore: migrate from Config::load_from_base_config_with_overrides to ConfigBuilder (#8276)
    https://github.com/openai/codex/pull/8235 introduced `ConfigBuilder` and
    this PR updates all call non-test call sites to use it instead of
    `Config::load_from_base_config_with_overrides()`.
    
    This is important because `load_from_base_config_with_overrides()` uses
    an empty `ConfigRequirements`, which is a reasonable default for testing
    so the tests are not influenced by the settings on the host. This method
    is now guarded by `#[cfg(test)]` so it cannot be used by business logic.
    
    Because `ConfigBuilder::build()` is `async`, many of the test methods
    had to be migrated to be `async`, as well. On the bright side, this made
    it possible to eliminate a bunch of `block_on_future()` stuff.
  • refactoring with_escalated_permissions to use SandboxPermissions instead (#7750)
    helpful in the future if we want more granularity for requesting
    escalated permissions:
    e.g when running in readonly sandbox, model can request to escalate to a
    sandbox that allows writes
  • whitelist command prefix integration in core and tui (#7033)
    this PR enables TUI to approve commands and add their prefixes to an
    allowlist:
    <img width="708" height="605" alt="Screenshot 2025-11-21 at 4 18 07 PM"
    src="https://github.com/user-attachments/assets/56a19893-4553-4770-a881-becf79eeda32"
    />
    
    note: we only show the option to whitelist the command when 
    1) command is not multi-part (e.g `git add -A && git commit -m 'hello
    world'`)
    2) command is not already matched by an existing rule
  • chore: update unified exec sandboxing detection (#7541)
    No integration test for now because it would make them flaky. Tracking
    it in my todos to add some once we have a clock based system for
    integration tests
  • chore: make create_approval_requirement_for_command an async fn (#7501)
    I think this might help with https://github.com/openai/codex/pull/7033
    because `create_approval_requirement_for_command()` will soon need
    access to `Session.state`, which is a `tokio::sync::Mutex` that needs to
    be accessed via `async`.
  • feat: add warning message for the model (#7445)
    Add a warning message as a user turn to the model if the model does not
    behave as expected (here, for example, if the model opens too many
    `unified_exec` sessions)