Commit Graph

19 Commits

  • Move string truncation helpers into codex-utils-string (#15572)
    - move the shared byte-based middle truncation logic from `core` into
    `codex-utils-string`
    - keep token-specific truncation in `codex-core` so rollout can reuse
    the shared helper in the next stacked PR
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Code mode on v8 (#15276)
    Moves Code Mode to a new crate with no dependencies on codex. This
    create encodes the code mode semantics that we want for lifetime,
    mounting, tool calling.
    
    The model-facing surface is mostly unchanged. `exec` still runs raw
    JavaScript, `wait` still resumes or terminates a `cell_id`, nested tools
    are still available through `tools.*`, and helpers like `text`, `image`,
    `store`, `load`, `notify`, `yield_control`, and `exit` still exist.
    
    The major change is underneath that surface:
    
    - Old code mode was an external Node runtime.
    - New code mode is an in-process V8 runtime embedded directly in Rust.
    - Old code mode managed cells inside a long-lived Node runner process.
    - New code mode manages cells in Rust, with one V8 runtime thread per
    active `exec`.
    - Old code mode used JSON protocol messages over child stdin/stdout plus
    Node worker-thread messages.
    - New code mode uses Rust channels and direct V8 callbacks/events.
    
    This PR also fixes the two migration regressions that fell out of that
    substrate change:
    
    - `wait { terminate: true }` now waits for the V8 runtime to actually
    stop before reporting termination.
    - synchronous top-level `exit()` now succeeds again instead of surfacing
    as a script error.
    
    ---
    
    - `core/src/tools/code_mode/*` is now mostly an adapter layer for the
    public `exec` / `wait` tools.
    - `code-mode/src/service.rs` owns cell sessions and async control flow
    in Rust.
    - `code-mode/src/runtime/*.rs` owns the embedded V8 isolate and
    JavaScript execution.
    - each `exec` spawns a dedicated runtime thread plus a Rust
    session-control task.
    - helper globals are installed directly into the V8 context instead of
    being injected through a source prelude.
    - helper modules like `tools.js` and `@openai/code_mode` are synthesized
    through V8 module resolution callbacks in Rust.
    
    ---
    
    Also added a benchmark for showing the speed of init and use of a code
    mode env:
    ```
    $ cargo bench -p codex-code-mode --bench exec_overhead -- --samples 30 --warm-iterations 25 --tool-counts 0,32,128
    Finished [`bench` profile [optimized]](https://doc.rust-lang.org/cargo/reference/profiles.html#default-profiles) target(s) in 0.18s
         Running benches/exec_overhead.rs (target/release/deps/exec_overhead-008c440d800545ae)
    exec_overhead: samples=30, warm_iterations=25, tool_counts=[0, 32, 128]
    scenario       tools samples    warmups      iters      mean/exec       p95/exec       rssΔ p50       rssΔ max
    cold_exec          0      30          0          1         1.13ms         1.20ms        8.05MiB        8.06MiB
    warm_exec          0      30          1         25       473.43us       512.49us      912.00KiB        1.33MiB
    cold_exec         32      30          0          1         1.03ms         1.15ms        8.08MiB        8.11MiB
    warm_exec         32      30          1         25       509.73us       545.76us      960.00KiB        1.30MiB
    cold_exec        128      30          0          1         1.14ms         1.19ms        8.30MiB        8.34MiB
    warm_exec        128      30          1         25       575.08us       591.03us      736.00KiB      864.00KiB
    memory uses a fresh-process max RSS delta for each scenario
    ```
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Split features into codex-features crate (#15253)
    - Split the feature system into a new `codex-features` crate.
    - Cut `codex-core` and workspace consumers over to the new config and
    warning APIs.
    
    Co-authored-by: Ahmed Ibrahim <219906144+aibrahim-oai@users.noreply.github.com>
    Co-authored-by: Codex <noreply@openai.com>
  • Return image URL from view_image tool (#15072)
    Cleanup image semantics in code mode.
    
    `view_image` now returns `{image_url:string, details?: string}` 
    
    `image()` now allows both string parameter and `{image_url:string,
    details?: string}`
  • Propagate tool errors to code mode (#15075)
    Clean up error flow to push the FunctionCallError all the way up to
    dispatcher and allow code mode to surface as exception.
  • Add notify to code-mode (#14842)
    Allows model to send an out-of-band notification.
    
    The notification is injected as another tool call output for the same
    call_id.
  • Rename exec_wait tool to wait (#14983)
    Summary
    - document that code mode only exposes `exec` and the renamed `wait`
    tool
    - update code mode tool spec and descriptions to match the new tool name
    - rename tests and helper references from `exec_wait` to `wait`
    
    Testing
    - Not run (not requested)
  • Apply argument comment lint across codex-rs (#14652)
    ## Why
    
    Once the repo-local lint exists, `codex-rs` needs to follow the
    checked-in convention and CI needs to keep it from drifting. This commit
    applies the fallback `/*param*/` style consistently across existing
    positional literal call sites without changing those APIs.
    
    The longer-term preference is still to avoid APIs that require comments
    by choosing clearer parameter types and call shapes. This PR is
    intentionally the mechanical follow-through for the places where the
    existing signatures stay in place.
    
    After rebasing onto newer `main`, the rollout also had to cover newly
    introduced `tui_app_server` call sites. That made it clear the first cut
    of the CI job was too expensive for the common path: it was spending
    almost as much time installing `cargo-dylint` and re-testing the lint
    crate as a representative test job spends running product tests. The CI
    update keeps the full workspace enforcement but trims that extra
    overhead from ordinary `codex-rs` PRs.
    
    ## What changed
    
    - keep a dedicated `argument_comment_lint` job in `rust-ci`
    - mechanically annotate remaining opaque positional literals across
    `codex-rs` with exact `/*param*/` comments, including the rebased
    `tui_app_server` call sites that now fall under the lint
    - keep the checked-in style aligned with the lint policy by using
    `/*param*/` and leaving string and char literals uncommented
    - cache `cargo-dylint`, `dylint-link`, and the relevant Cargo
    registry/git metadata in the lint job
    - split changed-path detection so the lint crate's own `cargo test` step
    runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes
    - continue to run the repo wrapper over the `codex-rs` workspace, so
    product-code enforcement is unchanged
    
    Most of the code changes in this commit are intentionally mechanical
    comment rewrites or insertions driven by the lint itself.
    
    ## Verification
    
    - `./tools/argument-comment-lint/run.sh --workspace`
    - `cargo test -p codex-tui-app-server -p codex-tui`
    - parsed `.github/workflows/rust-ci.yml` locally with PyYAML
    
    ---
    
    * -> #14652
    * #14651
  • Add exit helper to code mode scripts (#14851)
    - **Summary**
    - expose `exit` through the code mode bridge and module so scripts can
    stop mid-flight
      - surface the helper in the description documentation
      - add a regression test ensuring `exit()` terminates execution cleanly
    - **Testing**
      - Not run (not requested)
  • Add code_mode_only feature (#14617)
    Summary
    - add the code_mode_only feature flag/config schema and wire its
    dependency on code_mode
    - update code mode tool descriptions to list nested tools with detailed
    headers
    - restrict available tools for prompt and exec descriptions when
    code_mode_only is enabled and test the behavior
    
    Testing
    - Not run (not requested)
  • code_mode: Move exec params from runtime declarations to @pragma (#14511)
    This change moves code_mode exec session settings out of the runtime API
    and into an optional first-line pragma, so instead of calling runtime
    helpers like set_yield_time() or set_max_output_tokens_per_exec_call(),
    the model can write // @exec: {"yield_time_ms": ...,
    "max_output_tokens": ...} at the top of the freeform exec source. Rust
    now parses that pragma before building the source, validates it, and
    passes the values directly in the exec start message to the code-mode
    broker, which applies them at session start without any worker-runtime
    mutation path. The @openai/code_mode module no longer exposes those
    setter functions, the docs and grammar were updated to describe the
    pragma form, and the existing code_mode tests were converted to use
    pragma-based configuration instead.
  • Expose code-mode tools through globals (#14517)
    Summary
    - make all code-mode tools accessible as globals so callers only need
    `tools.<name>`
    - rename text/image helpers and key globals (store, load, ALL_TOOLS,
    etc.) to reflect the new shared namespace
    - update the JS bridge, runners, descriptions, router, and tests to
    follow the new API
    
    Testing
    - Not run (not requested)
  • Rename exec session IDs to cell IDs (#14510)
    - Update the code-mode executor, wait handler, and protocol plumbing to
    use cell IDs instead of session IDs for node communication
    - Switch tool metadata, wait description, and suite tests to refer to
    cell IDs so user-visible messages match the new terminology
    
    **Testing**
    - Not run (not requested)
  • Fix MCP tool calling (#14491)
    Properly escape mcp tool names and make tools only available via
    imports.
  • Reuse tool runtime for code mode worker (#14496)
    ## Summary
    - create the turn-scoped `ToolCallRuntime` before starting the code mode
    worker so the worker reuses the same runtime and router
    - thread the shared runtime through the code mode service/worker path
    and use it for nested tool calls
    - model aborted tool calls as a concrete `ToolOutput` so aborted
    responses still produce valid tool output shapes
    
    ## Testing
    - `just fmt`
    - `cargo test -p codex-core` (still running locally)
  • Add default code-mode yield timeout (#14484)
    Summary
    - expose the default yield timeout through code mode runtime so the
    handler, wait tool, and protocol share the same 10s value that matches
    unified exec
    - document the timeout change in the tool descriptions and propagate the
    value all the way into the runner metadata
    - adjust Cargo.lock to keep the dependency tree in sync with the added
    code mode tool dependency
    
    Testing
    - Not run (not requested)
  • Cleanup code_mode tool descriptions (#14480)
    Move to separate files and clarify a bit.
  • Move code mode tool files under tools/code_mode and split functionality (#14476)
    - **Summary**
    - migrate the code mode handler, service, worker, process, runner, and
    bridge assets into the `tools/code_mode` module tree
    - split Execution, protocol, and handler logic into dedicated files and
    relocate the tool definition into `code_mode/spec.rs`
    - update core references and tests to stitch the new organization
    together
    - **Testing**
      - Not run (not requested)