Commit Graph

531 Commits

  • Make output assertions more explicit (#4784)
    Match using precise regexes.
  • add codex sandbox {linux|macos} (#4782)
    ## Summary
    - add a `codex sandbox` subcommand with macOS and Linux targets while
    keeping the legacy `codex debug` aliases
    - update documentation to highlight the new sandbox entrypoints and
    point existing references to the new command
    - clarify the core README about the linux sandbox helper alias
    
    ## Testing
    - just fmt
    - just fix -p codex-cli
    - cargo test -p codex-cli
    
    
    ------
    https://chatgpt.com/codex/tasks/task_i_68e2e00ca1e8832d8bff53aa0b50b49e
  • Use response helpers when mounting SSE test responses (#4783)
    ## Summary
    - replace manual wiremock SSE mounts in the compact suite with the
    shared response helpers
    - simplify the exec auth_env integration test by using the
    mount_sse_once_match helper
    - rely on mount_sse_sequence plus server request collection to replace
    the bespoke SeqResponder utility in tests
    
    ## Testing
    - just fmt
    
    ------
    https://chatgpt.com/codex/tasks/task_i_68e2e238f2a88320a337f0b9e4098093
  • Use assert_matches (#4756)
    assert_matches is soon to be in std but is experimental for now.
  • Add helper for response created SSE events in tests (#4758)
    ## Summary
    - add a reusable `ev_response_created` helper that builds
    `response.created` SSE events for integration tests
    - update the exec and core integration suites to use the new helper
    instead of repeating manual JSON literals
    - keep the streaming fixtures consistent by relying on the shared helper
    in every touched test
    
    ## Testing
    - `just fmt`
    
    
    ------
    https://chatgpt.com/codex/tasks/task_i_68e1fe885bb883208aafffb94218da61
  • chore: drop read-file for gpt-5-codex (#4739)
    Drop `read_file` for gpt-5-codex (will do the same for parallel tool
    call) and add `codex-` as internal model for this kind of feature
  • feat: parallel tool calls (#4663)
    Add parallel tool calls. This is configurable at model level and tool
    level
  • chore: update tool config (#4755)
    ## Summary
    Updates tool config for gpt-5-codex
    
    ## Test Plan
    - [x] Ran locally
    - [x]  Updated unit tests
  • Use wait_for_event helpers in tests (#4753)
    ## Summary
    - replace manual event polling loops in several core test suites with
    the shared wait_for_event helpers
    - keep prior assertions intact by using closure captures for stateful
    expectations, including plan updates, patch lifecycles, and review flow
    checks
    - rely on wait_for_event_with_timeout where longer waits are required,
    simplifying timeout handling
    
    ## Testing
    - just fmt
    
    
    ------
    https://chatgpt.com/codex/tasks/task_i_68e1d58582d483208febadc5f90dd95e
  • Add truncation hint on truncated exec output. (#4740)
    When truncating output, add a hint of the total number of lines
  • feat: Freeform apply_patch with simple shell output (#4718)
    ## Summary
    This PR is an alternative approach to #4711, but instead of changing our
    storage, parses out shell calls in the client and reserializes them on
    the fly before we send them out as part of the request.
    
    What this changes:
    1. Adds additional serialization logic when the
    ApplyPatchToolType::Freeform is in use.
    2. Adds a --custom-apply-patch flag to enable this setting on a
    session-by-session basis.
    
    This change is delicate, but is not meant to be permanent. It is meant
    to be the first step in a migration:
    1. (This PR) Add in-flight serialization with config
    2. Update model_family default
    3. Update serialization logic to store turn outputs in a structured
    format, with logic to serialize based on model_family setting.
    4. Remove this rewrite in-flight logic.
    
    ## Test Plan
    - [x] Additional unit tests added
    - [x] Integration tests added
    - [x] Tested locally
  • Surface context window error to the client (#4675)
    In the past, we were treating `input exceeded context window` as a
    streaming error and retrying on it. Retrying on it has no point because
    it won't change the behavior. In this PR, we surface the error to the
    client without retry and also send a token count event to indicate that
    the context window is full.
    
    <img width="650" height="125" alt="image"
    src="https://github.com/user-attachments/assets/c26b1213-4c27-4bfc-90f4-51a270a3efd5"
    />
  • fix: exec commands that blows up context window. (#4706)
    We truncate the output of exec commands to not blow the context window.
    However, some cases we weren't doing that. This caused reports of people
    with 76% context window left facing `input exceeded context window`
    which is weird.
  • [MCP] Don't require experimental_use_rmcp_client for no-auth http servers (#4689)
    The `experimental_use_rmcp_client` flag is still useful to:
    1. Toggle between stdio clients
    2. Enable oauth beacuse we want to land
    https://github.com/modelcontextprotocol/rust-sdk/pull/469,
    https://github.com/openai/codex/pull/4677, and binary signing before we
    enable it by default
    
    However, for no-auth http servers, there is only one option so we don't
    need the flag and it seems to be working pretty well.
  • set gpt-5 as default model for Windows users (#4676)
    Codex isn’t great yet on Windows outside of WSL, and while we’ve merged
    https://github.com/openai/codex/pull/4269 to reduce the repetitive
    manual approvals on readonly commands, we’ve noticed that users seem to
    have more issues with GPT-5-Codex than with GPT-5 on Windows.
    
    This change makes GPT-5 the default for Windows users while we continue
    to improve the CLI harness and model for GPT-5-Codex on Windows.
  • add(core): managed config (#3868)
    ## Summary
    
    - Factor `load_config_as_toml` into `core::config_loader` so config
    loading is reusable across callers.
    - Layer `~/.codex/config.toml`, optional `~/.codex/managed_config.toml`,
    and macOS managed preferences (base64) with recursive table merging and
    scoped threads per source.
    
    ## Config Flow
    
    ```
    Managed prefs (macOS profile: com.openai.codex/config_toml_base64)
                                   ▲
                                   │
    ~/.codex/managed_config.toml   │  (optional file-based override)
                                   ▲
                                   │
                    ~/.codex/config.toml (user-defined settings)
    ```
    
    - The loader searches under the resolved `CODEX_HOME` directory
    (defaults to `~/.codex`).
    - Managed configs let administrators ship fleet-wide overrides via
    device profiles which is useful for enforcing certain settings like
    sandbox or approval defaults.
    - For nested hash tables: overlays merge recursively. Child tables are
    merged key-by-key, while scalar or array values replace the prior layer
    entirely. This lets admins add or tweak individual fields without
    clobbering unrelated user settings.
  • [MCP] Add support for MCP Oauth credentials (#4517)
    This PR adds oauth login support to streamable http servers when
    `experimental_use_rmcp_client` is enabled.
    
    This PR is large but represents the minimal amount of work required for
    this to work. To keep this PR smaller, login can only be done with
    `codex mcp login` and `codex mcp logout` but it doesn't appear in `/mcp`
    or `codex mcp list` yet. Fingers crossed that this is the last large MCP
    PR and that subsequent PRs can be smaller.
    
    Under the hood, credentials are stored using platform credential
    managers using the [keyring crate](https://crates.io/crates/keyring).
    When the keyring isn't available, it falls back to storing credentials
    in `CODEX_HOME/.credentials.json` which is consistent with how other
    coding agents handle authentication.
    
    I tested this on macOS, Windows, WSL (ubuntu), and Linux. I wasn't able
    to test the dbus store on linux but did verify that the fallback works.
    
    One quirk is that if you have credentials, during development, every
    build will have its own ad-hoc binary so the keyring won't recognize the
    reader as being the same as the write so it may ask for the user's
    password. I may add an override to disable this or allow
    users/enterprises to opt-out of the keyring storage if it causes issues.
    
    <img width="5064" height="686" alt="CleanShot 2025-09-30 at 19 31 40"
    src="https://github.com/user-attachments/assets/9573f9b4-07f1-4160-83b8-2920db287e2d"
    />
    <img width="745" height="486" alt="image"
    src="https://github.com/user-attachments/assets/9562649b-ea5f-4f22-ace2-d0cb438b143e"
    />
  • Fix flaky test (#4672)
    This issue was due to the fact that the timeout is not always sufficient
    to have enough character for truncation + a race between synthetic
    timeout and process kill
  • feat: add beta_supported_tools (#4669)
    Gate the new read_file tool behind a new `beta_supported_tools` flag and
    only enable it for `gpt-5-codex`
  • chore: refactor tool handling (#4510)
    # Tool System Refactor
    
    - Centralizes tool definitions and execution in `core/src/tools/*`:
    specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
    registry/dispatch (`registry.rs`), and shared context (`context.rs`).
    One registry now builds the model-visible tool list and binds handlers.
    - Router converts model responses to tool calls; Registry dispatches
    with consistent telemetry via `codex-rs/otel` and unified error
    handling. Function, Local Shell, MCP, and experimental `unified_exec`
    all flow through this path; legacy shell aliases still work.
    - Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
    make adding tools predictable and testable.
    
    Example: `read_file`
    - Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
    registered by `build_specs`).
    - Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
    1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
    - E2E test: `core/tests/suite/read_file.rs` validates the tool returns
    the requested lines.
    
    ## Next steps:
    - Decompose `handle_container_exec_with_params` 
    - Add parallel tool calls
  • chore: sandbox refactor 2 (#4653)
    Revert the revert and fix the UI issue
  • Separate interactive and non-interactive sessions (#4612)
    Do not show exec session in VSCode/TUI selector.
  • fix: handle JSON Schema in additionalProperties for MCP tools (#4454)
    Fixes #4176
    
    Some common tools provide a schema (even if just an empty object schema)
    as the value for `additionalProperties`. The parsing as it currently
    stands fails when it encounters this. This PR updates the schema to
    accept a schema object in addition to a boolean value, per the JSON
    Schema spec.
  • Support CODEX_API_KEY for codex exec (#4615)
    Allows to set API key per invocation of `codex exec`
  • fix: update the gpt-5-codex prompt to be more explicit that it should always used fenced code blocks info tags (#4569)
    We get spurrious reports that the model writes fenced code blocks
    without an info tag which then causes auto-language detection in the
    extension to incorrectly highlight the code and show the wrong language.
    The model should really always include a tag when it can.
  • Include request ID in the error message (#4572)
    To help with issue debugging
    <img width="1414" height="253" alt="image"
    src="https://github.com/user-attachments/assets/254732df-44ac-4252-997a-6c5e0927355b"
    />
  • canonicalize display of Agents.md paths on Windows. (#4577)
    Canonicalize path on Windows to 
    - remove unattractive path prefixes such as `\\?\`
    - simplify it (`../AGENTS.md` vs
    `C:\Users\iceweasel\code\coded\Agents.md`)
    before: <img width="1110" height="45" alt="Screenshot 2025-10-01 123520"
    src="https://github.com/user-attachments/assets/48920ae6-d89c-41b8-b4ea-df5c18fb5fad"
    />
    
    after: 
    <img width="585" height="46" alt="Screenshot 2025-10-01 123612"
    src="https://github.com/user-attachments/assets/70a1761a-9d97-4836-b14c-670b6f13e608"
    />
  • Fall back to configured instruction files if AGENTS.md isn't available (#4544)
    Allow users to configure an agents.md alternative to consume, but warn
    the user it may degrade model performance.
    
    Fixes #4376
  • implement command safety for PowerShell commands (#4269)
    Implement command safety for PowerShell commands on Windows
    
    This change adds a new Windows-specific command-safety module under
    `codex-rs/core/src/command_safety/windows_safe_commands.rs` to strictly
    sanitise PowerShell invocations. Key points:
    
    - Introduce `is_safe_command_windows()` to only allow explicitly
    read-only PowerShell calls.
    - Parse and split PowerShell invocations (including inline `-Command`
    scripts and pipelines).
    - Block unsafe switches (`-File`, `-EncodedCommand`, `-ExecutionPolicy`,
    unknown flags, call operators, redirections, separators).
    - Whitelist only read-only cmdlets (`Get-ChildItem`, `Get-Content`,
    `Select-Object`, etc.), safe Git subcommands (`status`, `log`, `show`,
    `diff`, `cat-file`), and ripgrep without unsafe options.
    - Add comprehensive unit tests covering allowed and rejected command
    patterns (nested calls, side effects, chaining, redirections).
    
    This ensures Codex on Windows can safely execute discover-only
    PowerShell workflows without risking destructive operations.
  • chore: sanbox extraction (#4286)
    # Extract and Centralize Sandboxing
    - Goal: Improve safety and clarity by centralizing sandbox planning and
    execution.
      - Approach:
    - Add planner (ExecPlan) and backend registry (Direct/Seatbelt/Linux)
    with run_with_plan.
    - Refactor codex.rs to plan-then-execute; handle failures/escalation via
    the plan.
    - Delegate apply_patch to the codex binary and run it with an empty env
    for determinism.
  • fix: remove mcp-types from app server protocol (#4537)
    We continue the separation between `codex app-server` and `codex
    mcp-server`.
    
    In particular, we introduce a new crate, `codex-app-server-protocol`,
    and migrate `codex-rs/protocol/src/mcp_protocol.rs` into it, renaming it
    `codex-rs/app-server-protocol/src/protocol.rs`.
    
    Because `ConversationId` was defined in `mcp_protocol.rs`, we move it
    into its own file, `codex-rs/protocol/src/conversation_id.rs`, and
    because it is referenced in a ton of places, we have to touch a lot of
    files as part of this PR.
    
    We also decide to get away from proper JSON-RPC 2.0 semantics, so we
    also introduce `codex-rs/app-server-protocol/src/jsonrpc_lite.rs`, which
    is basically the same `JSONRPCMessage` type defined in `mcp-types`
    except with all of the `"jsonrpc": "2.0"` removed.
    
    Getting rid of `"jsonrpc": "2.0"` makes our serialization logic
    considerably simpler, as we can lean heavier on serde to serialize
    directly into the wire format that we use now.
  • chore: prompt update to enforce good usage of apply_patch (#3846)
    Update prompt to prevent codex to use Python script or fancy commands to
    edit files.
    
    ## Testing:
    3 scenarios have been considered:
    1. Rename codex to meca_code. Proceed to the whole refactor file by
    file. Don't ask for approval at each step
    2. Add a description to every single function you can find in the repo
    3. Rewrite codex.rs in a more idiomatic way. Make sure to touch ONLY
    this file and that clippy does not complain at the end
    
    Before this update, 22% (estimation as it's sometimes hard to find all
    the creative way the model find to edit files) of the file editions
    where made using something else than a raw `apply_patch`
    
    After this update, not a single edition without `apply_patch` was found
    
    [EDIT]
    I managed to have a few `["bash", "-lc", "apply_path"]` when reaching <
    10% context left
  • Add cloud tasks (#3197)
    Adds a TUI for managing, applying, and creating cloud tasks
  • Set originator for codex exec (#4485)
    Distinct from the main CLI.
  • [Core]: add tail in the rollout data (#4461)
    This will help us show the conversation tail and last updated timestamp.
  • Parse out frontmatter for custom prompts (#4456)
    [Cherry picked from https://github.com/openai/codex/pull/3565]
    
    Removes the frontmatter description/args from custom prompt files and
    only includes body.
  • OpenTelemetry events (#2103)
    ### Title
    
    ## otel
    
    Codex can emit [OpenTelemetry](https://opentelemetry.io/) **log events**
    that
    describe each run: outbound API requests, streamed responses, user
    input,
    tool-approval decisions, and the result of every tool invocation. Export
    is
    **disabled by default** so local runs remain self-contained. Opt in by
    adding an
    `[otel]` table and choosing an exporter.
    
    ```toml
    [otel]
    environment = "staging"   # defaults to "dev"
    exporter = "none"          # defaults to "none"; set to otlp-http or otlp-grpc to send events
    log_user_prompt = false    # defaults to false; redact prompt text unless explicitly enabled
    ```
    
    Codex tags every exported event with `service.name = "codex-cli"`, the
    CLI
    version, and an `env` attribute so downstream collectors can distinguish
    dev/staging/prod traffic. Only telemetry produced inside the
    `codex_otel`
    crate—the events listed below—is forwarded to the exporter.
    
    ### Event catalog
    
    Every event shares a common set of metadata fields: `event.timestamp`,
    `conversation.id`, `app.version`, `auth_mode` (when available),
    `user.account_id` (when available), `terminal.type`, `model`, and
    `slug`.
    
    With OTEL enabled Codex emits the following event types (in addition to
    the
    metadata above):
    
    - `codex.api_request`
      - `cf_ray` (optional)
      - `attempt`
      - `duration_ms`
      - `http.response.status_code` (optional)
      - `error.message` (failures)
    - `codex.sse_event`
      - `event.kind`
      - `duration_ms`
      - `error.message` (failures)
      - `input_token_count` (completion only)
      - `output_token_count` (completion only)
      - `cached_token_count` (completion only, optional)
      - `reasoning_token_count` (completion only, optional)
      - `tool_token_count` (completion only)
    - `codex.user_prompt`
      - `prompt_length`
      - `prompt` (redacted unless `log_user_prompt = true`)
    - `codex.tool_decision`
      - `tool_name`
      - `call_id`
    - `decision` (`approved`, `approved_for_session`, `denied`, or `abort`)
      - `source` (`config` or `user`)
    - `codex.tool_result`
      - `tool_name`
      - `call_id`
      - `arguments`
      - `duration_ms` (execution time for the tool)
      - `success` (`"true"` or `"false"`)
      - `output`
    
    ### Choosing an exporter
    
    Set `otel.exporter` to control where events go:
    
    - `none` – leaves instrumentation active but skips exporting. This is
    the
      default.
    - `otlp-http` – posts OTLP log records to an OTLP/HTTP collector.
    Specify the
      endpoint, protocol, and headers your collector expects:
    
      ```toml
      [otel]
      exporter = { otlp-http = {
        endpoint = "https://otel.example.com/v1/logs",
        protocol = "binary",
        headers = { "x-otlp-api-key" = "${OTLP_TOKEN}" }
      }}
      ```
    
    - `otlp-grpc` – streams OTLP log records over gRPC. Provide the endpoint
    and any
      metadata headers:
    
      ```toml
      [otel]
      exporter = { otlp-grpc = {
        endpoint = "https://otel.example.com:4317",
        headers = { "x-otlp-meta" = "abc123" }
      }}
      ```
    
    If the exporter is `none` nothing is written anywhere; otherwise you
    must run or point to your
    own collector. All exporters run on a background batch worker that is
    flushed on
    shutdown.
    
    If you build Codex from source the OTEL crate is still behind an `otel`
    feature
    flag; the official prebuilt binaries ship with the feature enabled. When
    the
    feature is disabled the telemetry hooks become no-ops so the CLI
    continues to
    function without the extra dependencies.
    
    ---------
    
    Co-authored-by: Anton Panasenko <apanasenko@openai.com>
  • [MCP] Add experimental support for streamable HTTP MCP servers (#4317)
    This PR adds support for streamable HTTP MCP servers when the
    `experimental_use_rmcp_client` is enabled.
    
    To set one up, simply add a new mcp server config with the url:
    ```
    [mcp_servers.figma]
    url = "http://127.0.0.1:3845/mcp"
    ```
    
    It also supports an optional `bearer_token` which will be provided in an
    authorization header. The full oauth flow is not supported yet.
    
    The config parsing will throw if it detects that the user mixed and
    matched config fields (like command + bearer token or url + env).
    
    The best way to review it is to review `core/src` and then
    `rmcp-client/src/rmcp_client.rs` first. The rest is tests and
    propagating the `Transport` struct around the codebase.
    
    Example with the Figma MCP:
    <img width="5084" height="1614" alt="CleanShot 2025-09-26 at 13 35 40"
    src="https://github.com/user-attachments/assets/eaf2771e-df3e-4300-816b-184d7dec5a28"
    />
  • update composer + user message styling (#4240)
    Changes:
    
    - the composer and user messages now have a colored background that
    stretches the entire width of the terminal.
    - the prompt character was changed from a cyan `▌` to a bold `›`.
    - the "working" shimmer now follows the "dark gray" color of the
    terminal, better matching the terminal's color scheme
    
    | Terminal + Background        | Screenshot |
    |------------------------------|------------|
    | iTerm with dark bg | <img width="810" height="641" alt="Screenshot
    2025-09-25 at 11 44 52 AM"
    src="https://github.com/user-attachments/assets/1317e579-64a9-4785-93e6-98b0258f5d92"
    /> |
    | iTerm with light bg | <img width="845" height="540" alt="Screenshot
    2025-09-25 at 11 46 29 AM"
    src="https://github.com/user-attachments/assets/e671d490-c747-4460-af0b-3f8d7f7a6b8e"
    /> |
    | iTerm with color bg | <img width="825" height="564" alt="Screenshot
    2025-09-25 at 11 47 12 AM"
    src="https://github.com/user-attachments/assets/141cda1b-1164-41d5-87da-3be11e6a3063"
    /> |
    | Terminal.app with dark bg | <img width="577" height="367"
    alt="Screenshot 2025-09-25 at 11 45 22 AM"
    src="https://github.com/user-attachments/assets/93fc4781-99f7-4ee7-9c8e-3db3cd854fe5"
    /> |
    | Terminal.app with light bg | <img width="577" height="367"
    alt="Screenshot 2025-09-25 at 11 46 04 AM"
    src="https://github.com/user-attachments/assets/19bf6a3c-91e0-447b-9667-b8033f512219"
    /> |
    | Terminal.app with color bg | <img width="577" height="367"
    alt="Screenshot 2025-09-25 at 11 45 50 AM"
    src="https://github.com/user-attachments/assets/dd7c4b5b-342e-4028-8140-f4e65752bd0b"
    /> |
  • reject dangerous commands for AskForApproval::Never (#4307)
    If we detect a dangerous command but approval_policy is Never, simply
    reject the command.
  • /status followup (#4304)
    - Render `send a message to load usage data` in the beginning of the
    session
    - Render `data not available yet` if received no rate limits 
    - nit case
    - Deleted stall snapshots that were moved to
    `codex-rs/tui/src/status/snapshots`