Commit Graph

480 Commits

  • [MCP] Add an enabled config field (#4917)
    This lets users more easily toggle MCP servers.
  • feat: truncate on compact (#4942)
    Truncate the message during compaction if it is just too large
    Do it iteratively as tokenization is basically free on server-side
  • feat: grep_files as a tool (#4820)
    Add `grep_files` to be able to perform more action in parallel
  • [MCP] Add support for streamable http servers with codex mcp add and replace bearer token handling (#4904)
    1. You can now add streamable http servers via the CLI
    2. As part of this, I'm also changing the existing bearer_token plain
    text config field with ane env var
    
    ```
    mcp add github --url https://api.githubcopilot.com/mcp/ --bearer-token-env-var=GITHUB_PAT
    ```
  • [MCP] Add the ability to explicitly specify a credentials store (#4857)
    This lets users/companies explicitly choose whether to force/disallow
    the keyring/fallback file storage for mcp credentials.
    
    People who develop with Codex will want to use this until we sign
    binaries or else each ad-hoc debug builds will require keychain access
    on every build. I don't love this and am open to other ideas for how to
    handle that.
    
    
    ```toml
    mcp_oauth_credentials_store = "auto"
    mcp_oauth_credentials_store = "file"
    mcp_oauth_credentials_store = "keyrung"
    ```
    Defaults to `auto`
  • Set codex SDK TypeScript originator (#4894)
    ## Summary
    - ensure the TypeScript SDK sets CODEX_INTERNAL_ORIGINATOR_OVERRIDE to
    codex_sdk_ts when spawning the Codex CLI
    - extend the responses proxy test helper to capture request headers for
    assertions
    - add coverage that verifies Codex threads launched from the TypeScript
    SDK send the codex_sdk_ts originator header
    
    ## Testing
    - Not Run (not requested)
    
    
    ------
    https://chatgpt.com/codex/tasks/task_i_68e561b125248320a487f129093d16e7
  • Remove instruction hack for /review (#4896)
    We use to put the review prompt in the first user message as well to
    bypass statsig overrides, but now that's been resolved and instructions
    are being respected, so we're duplicating the review instructions.
  • feat: list_dir tool (#4817)
    Add a tool to list_dir. It is useful because we can mark it as
    non-mutating and so use it in parallel
  • Simplify parallel (#4829)
    make tool processing return a future and then collect futures.
    handle cleanup on Drop
  • Use assert_matches (#4756)
    assert_matches is soon to be in std but is experimental for now.
  • chore: drop read-file for gpt-5-codex (#4739)
    Drop `read_file` for gpt-5-codex (will do the same for parallel tool
    call) and add `codex-` as internal model for this kind of feature
  • feat: parallel tool calls (#4663)
    Add parallel tool calls. This is configurable at model level and tool
    level
  • chore: update tool config (#4755)
    ## Summary
    Updates tool config for gpt-5-codex
    
    ## Test Plan
    - [x] Ran locally
    - [x]  Updated unit tests
  • Add truncation hint on truncated exec output. (#4740)
    When truncating output, add a hint of the total number of lines
  • feat: Freeform apply_patch with simple shell output (#4718)
    ## Summary
    This PR is an alternative approach to #4711, but instead of changing our
    storage, parses out shell calls in the client and reserializes them on
    the fly before we send them out as part of the request.
    
    What this changes:
    1. Adds additional serialization logic when the
    ApplyPatchToolType::Freeform is in use.
    2. Adds a --custom-apply-patch flag to enable this setting on a
    session-by-session basis.
    
    This change is delicate, but is not meant to be permanent. It is meant
    to be the first step in a migration:
    1. (This PR) Add in-flight serialization with config
    2. Update model_family default
    3. Update serialization logic to store turn outputs in a structured
    format, with logic to serialize based on model_family setting.
    4. Remove this rewrite in-flight logic.
    
    ## Test Plan
    - [x] Additional unit tests added
    - [x] Integration tests added
    - [x] Tested locally
  • Surface context window error to the client (#4675)
    In the past, we were treating `input exceeded context window` as a
    streaming error and retrying on it. Retrying on it has no point because
    it won't change the behavior. In this PR, we surface the error to the
    client without retry and also send a token count event to indicate that
    the context window is full.
    
    <img width="650" height="125" alt="image"
    src="https://github.com/user-attachments/assets/c26b1213-4c27-4bfc-90f4-51a270a3efd5"
    />
  • fix: exec commands that blows up context window. (#4706)
    We truncate the output of exec commands to not blow the context window.
    However, some cases we weren't doing that. This caused reports of people
    with 76% context window left facing `input exceeded context window`
    which is weird.
  • [MCP] Don't require experimental_use_rmcp_client for no-auth http servers (#4689)
    The `experimental_use_rmcp_client` flag is still useful to:
    1. Toggle between stdio clients
    2. Enable oauth beacuse we want to land
    https://github.com/modelcontextprotocol/rust-sdk/pull/469,
    https://github.com/openai/codex/pull/4677, and binary signing before we
    enable it by default
    
    However, for no-auth http servers, there is only one option so we don't
    need the flag and it seems to be working pretty well.
  • set gpt-5 as default model for Windows users (#4676)
    Codex isn’t great yet on Windows outside of WSL, and while we’ve merged
    https://github.com/openai/codex/pull/4269 to reduce the repetitive
    manual approvals on readonly commands, we’ve noticed that users seem to
    have more issues with GPT-5-Codex than with GPT-5 on Windows.
    
    This change makes GPT-5 the default for Windows users while we continue
    to improve the CLI harness and model for GPT-5-Codex on Windows.
  • add(core): managed config (#3868)
    ## Summary
    
    - Factor `load_config_as_toml` into `core::config_loader` so config
    loading is reusable across callers.
    - Layer `~/.codex/config.toml`, optional `~/.codex/managed_config.toml`,
    and macOS managed preferences (base64) with recursive table merging and
    scoped threads per source.
    
    ## Config Flow
    
    ```
    Managed prefs (macOS profile: com.openai.codex/config_toml_base64)
                                   ▲
                                   │
    ~/.codex/managed_config.toml   │  (optional file-based override)
                                   ▲
                                   │
                    ~/.codex/config.toml (user-defined settings)
    ```
    
    - The loader searches under the resolved `CODEX_HOME` directory
    (defaults to `~/.codex`).
    - Managed configs let administrators ship fleet-wide overrides via
    device profiles which is useful for enforcing certain settings like
    sandbox or approval defaults.
    - For nested hash tables: overlays merge recursively. Child tables are
    merged key-by-key, while scalar or array values replace the prior layer
    entirely. This lets admins add or tweak individual fields without
    clobbering unrelated user settings.
  • [MCP] Add support for MCP Oauth credentials (#4517)
    This PR adds oauth login support to streamable http servers when
    `experimental_use_rmcp_client` is enabled.
    
    This PR is large but represents the minimal amount of work required for
    this to work. To keep this PR smaller, login can only be done with
    `codex mcp login` and `codex mcp logout` but it doesn't appear in `/mcp`
    or `codex mcp list` yet. Fingers crossed that this is the last large MCP
    PR and that subsequent PRs can be smaller.
    
    Under the hood, credentials are stored using platform credential
    managers using the [keyring crate](https://crates.io/crates/keyring).
    When the keyring isn't available, it falls back to storing credentials
    in `CODEX_HOME/.credentials.json` which is consistent with how other
    coding agents handle authentication.
    
    I tested this on macOS, Windows, WSL (ubuntu), and Linux. I wasn't able
    to test the dbus store on linux but did verify that the fallback works.
    
    One quirk is that if you have credentials, during development, every
    build will have its own ad-hoc binary so the keyring won't recognize the
    reader as being the same as the write so it may ask for the user's
    password. I may add an override to disable this or allow
    users/enterprises to opt-out of the keyring storage if it causes issues.
    
    <img width="5064" height="686" alt="CleanShot 2025-09-30 at 19 31 40"
    src="https://github.com/user-attachments/assets/9573f9b4-07f1-4160-83b8-2920db287e2d"
    />
    <img width="745" height="486" alt="image"
    src="https://github.com/user-attachments/assets/9562649b-ea5f-4f22-ace2-d0cb438b143e"
    />
  • feat: add beta_supported_tools (#4669)
    Gate the new read_file tool behind a new `beta_supported_tools` flag and
    only enable it for `gpt-5-codex`
  • chore: refactor tool handling (#4510)
    # Tool System Refactor
    
    - Centralizes tool definitions and execution in `core/src/tools/*`:
    specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
    registry/dispatch (`registry.rs`), and shared context (`context.rs`).
    One registry now builds the model-visible tool list and binds handlers.
    - Router converts model responses to tool calls; Registry dispatches
    with consistent telemetry via `codex-rs/otel` and unified error
    handling. Function, Local Shell, MCP, and experimental `unified_exec`
    all flow through this path; legacy shell aliases still work.
    - Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
    make adding tools predictable and testable.
    
    Example: `read_file`
    - Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
    registered by `build_specs`).
    - Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
    1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
    - E2E test: `core/tests/suite/read_file.rs` validates the tool returns
    the requested lines.
    
    ## Next steps:
    - Decompose `handle_container_exec_with_params` 
    - Add parallel tool calls
  • chore: sandbox refactor 2 (#4653)
    Revert the revert and fix the UI issue
  • Separate interactive and non-interactive sessions (#4612)
    Do not show exec session in VSCode/TUI selector.
  • fix: handle JSON Schema in additionalProperties for MCP tools (#4454)
    Fixes #4176
    
    Some common tools provide a schema (even if just an empty object schema)
    as the value for `additionalProperties`. The parsing as it currently
    stands fails when it encounters this. This PR updates the schema to
    accept a schema object in addition to a boolean value, per the JSON
    Schema spec.
  • Support CODEX_API_KEY for codex exec (#4615)
    Allows to set API key per invocation of `codex exec`
  • Include request ID in the error message (#4572)
    To help with issue debugging
    <img width="1414" height="253" alt="image"
    src="https://github.com/user-attachments/assets/254732df-44ac-4252-997a-6c5e0927355b"
    />
  • canonicalize display of Agents.md paths on Windows. (#4577)
    Canonicalize path on Windows to 
    - remove unattractive path prefixes such as `\\?\`
    - simplify it (`../AGENTS.md` vs
    `C:\Users\iceweasel\code\coded\Agents.md`)
    before: <img width="1110" height="45" alt="Screenshot 2025-10-01 123520"
    src="https://github.com/user-attachments/assets/48920ae6-d89c-41b8-b4ea-df5c18fb5fad"
    />
    
    after: 
    <img width="585" height="46" alt="Screenshot 2025-10-01 123612"
    src="https://github.com/user-attachments/assets/70a1761a-9d97-4836-b14c-670b6f13e608"
    />
  • Fall back to configured instruction files if AGENTS.md isn't available (#4544)
    Allow users to configure an agents.md alternative to consume, but warn
    the user it may degrade model performance.
    
    Fixes #4376
  • implement command safety for PowerShell commands (#4269)
    Implement command safety for PowerShell commands on Windows
    
    This change adds a new Windows-specific command-safety module under
    `codex-rs/core/src/command_safety/windows_safe_commands.rs` to strictly
    sanitise PowerShell invocations. Key points:
    
    - Introduce `is_safe_command_windows()` to only allow explicitly
    read-only PowerShell calls.
    - Parse and split PowerShell invocations (including inline `-Command`
    scripts and pipelines).
    - Block unsafe switches (`-File`, `-EncodedCommand`, `-ExecutionPolicy`,
    unknown flags, call operators, redirections, separators).
    - Whitelist only read-only cmdlets (`Get-ChildItem`, `Get-Content`,
    `Select-Object`, etc.), safe Git subcommands (`status`, `log`, `show`,
    `diff`, `cat-file`), and ripgrep without unsafe options.
    - Add comprehensive unit tests covering allowed and rejected command
    patterns (nested calls, side effects, chaining, redirections).
    
    This ensures Codex on Windows can safely execute discover-only
    PowerShell workflows without risking destructive operations.
  • chore: sanbox extraction (#4286)
    # Extract and Centralize Sandboxing
    - Goal: Improve safety and clarity by centralizing sandbox planning and
    execution.
      - Approach:
    - Add planner (ExecPlan) and backend registry (Direct/Seatbelt/Linux)
    with run_with_plan.
    - Refactor codex.rs to plan-then-execute; handle failures/escalation via
    the plan.
    - Delegate apply_patch to the codex binary and run it with an empty env
    for determinism.
  • fix: remove mcp-types from app server protocol (#4537)
    We continue the separation between `codex app-server` and `codex
    mcp-server`.
    
    In particular, we introduce a new crate, `codex-app-server-protocol`,
    and migrate `codex-rs/protocol/src/mcp_protocol.rs` into it, renaming it
    `codex-rs/app-server-protocol/src/protocol.rs`.
    
    Because `ConversationId` was defined in `mcp_protocol.rs`, we move it
    into its own file, `codex-rs/protocol/src/conversation_id.rs`, and
    because it is referenced in a ton of places, we have to touch a lot of
    files as part of this PR.
    
    We also decide to get away from proper JSON-RPC 2.0 semantics, so we
    also introduce `codex-rs/app-server-protocol/src/jsonrpc_lite.rs`, which
    is basically the same `JSONRPCMessage` type defined in `mcp-types`
    except with all of the `"jsonrpc": "2.0"` removed.
    
    Getting rid of `"jsonrpc": "2.0"` makes our serialization logic
    considerably simpler, as we can lean heavier on serde to serialize
    directly into the wire format that we use now.
  • Add cloud tasks (#3197)
    Adds a TUI for managing, applying, and creating cloud tasks
  • Set originator for codex exec (#4485)
    Distinct from the main CLI.
  • [Core]: add tail in the rollout data (#4461)
    This will help us show the conversation tail and last updated timestamp.
  • Parse out frontmatter for custom prompts (#4456)
    [Cherry picked from https://github.com/openai/codex/pull/3565]
    
    Removes the frontmatter description/args from custom prompt files and
    only includes body.
  • OpenTelemetry events (#2103)
    ### Title
    
    ## otel
    
    Codex can emit [OpenTelemetry](https://opentelemetry.io/) **log events**
    that
    describe each run: outbound API requests, streamed responses, user
    input,
    tool-approval decisions, and the result of every tool invocation. Export
    is
    **disabled by default** so local runs remain self-contained. Opt in by
    adding an
    `[otel]` table and choosing an exporter.
    
    ```toml
    [otel]
    environment = "staging"   # defaults to "dev"
    exporter = "none"          # defaults to "none"; set to otlp-http or otlp-grpc to send events
    log_user_prompt = false    # defaults to false; redact prompt text unless explicitly enabled
    ```
    
    Codex tags every exported event with `service.name = "codex-cli"`, the
    CLI
    version, and an `env` attribute so downstream collectors can distinguish
    dev/staging/prod traffic. Only telemetry produced inside the
    `codex_otel`
    crate—the events listed below—is forwarded to the exporter.
    
    ### Event catalog
    
    Every event shares a common set of metadata fields: `event.timestamp`,
    `conversation.id`, `app.version`, `auth_mode` (when available),
    `user.account_id` (when available), `terminal.type`, `model`, and
    `slug`.
    
    With OTEL enabled Codex emits the following event types (in addition to
    the
    metadata above):
    
    - `codex.api_request`
      - `cf_ray` (optional)
      - `attempt`
      - `duration_ms`
      - `http.response.status_code` (optional)
      - `error.message` (failures)
    - `codex.sse_event`
      - `event.kind`
      - `duration_ms`
      - `error.message` (failures)
      - `input_token_count` (completion only)
      - `output_token_count` (completion only)
      - `cached_token_count` (completion only, optional)
      - `reasoning_token_count` (completion only, optional)
      - `tool_token_count` (completion only)
    - `codex.user_prompt`
      - `prompt_length`
      - `prompt` (redacted unless `log_user_prompt = true`)
    - `codex.tool_decision`
      - `tool_name`
      - `call_id`
    - `decision` (`approved`, `approved_for_session`, `denied`, or `abort`)
      - `source` (`config` or `user`)
    - `codex.tool_result`
      - `tool_name`
      - `call_id`
      - `arguments`
      - `duration_ms` (execution time for the tool)
      - `success` (`"true"` or `"false"`)
      - `output`
    
    ### Choosing an exporter
    
    Set `otel.exporter` to control where events go:
    
    - `none` – leaves instrumentation active but skips exporting. This is
    the
      default.
    - `otlp-http` – posts OTLP log records to an OTLP/HTTP collector.
    Specify the
      endpoint, protocol, and headers your collector expects:
    
      ```toml
      [otel]
      exporter = { otlp-http = {
        endpoint = "https://otel.example.com/v1/logs",
        protocol = "binary",
        headers = { "x-otlp-api-key" = "${OTLP_TOKEN}" }
      }}
      ```
    
    - `otlp-grpc` – streams OTLP log records over gRPC. Provide the endpoint
    and any
      metadata headers:
    
      ```toml
      [otel]
      exporter = { otlp-grpc = {
        endpoint = "https://otel.example.com:4317",
        headers = { "x-otlp-meta" = "abc123" }
      }}
      ```
    
    If the exporter is `none` nothing is written anywhere; otherwise you
    must run or point to your
    own collector. All exporters run on a background batch worker that is
    flushed on
    shutdown.
    
    If you build Codex from source the OTEL crate is still behind an `otel`
    feature
    flag; the official prebuilt binaries ship with the feature enabled. When
    the
    feature is disabled the telemetry hooks become no-ops so the CLI
    continues to
    function without the extra dependencies.
    
    ---------
    
    Co-authored-by: Anton Panasenko <apanasenko@openai.com>
  • [MCP] Add experimental support for streamable HTTP MCP servers (#4317)
    This PR adds support for streamable HTTP MCP servers when the
    `experimental_use_rmcp_client` is enabled.
    
    To set one up, simply add a new mcp server config with the url:
    ```
    [mcp_servers.figma]
    url = "http://127.0.0.1:3845/mcp"
    ```
    
    It also supports an optional `bearer_token` which will be provided in an
    authorization header. The full oauth flow is not supported yet.
    
    The config parsing will throw if it detects that the user mixed and
    matched config fields (like command + bearer token or url + env).
    
    The best way to review it is to review `core/src` and then
    `rmcp-client/src/rmcp_client.rs` first. The rest is tests and
    propagating the `Transport` struct around the codebase.
    
    Example with the Figma MCP:
    <img width="5084" height="1614" alt="CleanShot 2025-09-26 at 13 35 40"
    src="https://github.com/user-attachments/assets/eaf2771e-df3e-4300-816b-184d7dec5a28"
    />
  • reject dangerous commands for AskForApproval::Never (#4307)
    If we detect a dangerous command but approval_policy is Never, simply
    reject the command.
  • /status followup (#4304)
    - Render `send a message to load usage data` in the beginning of the
    session
    - Render `data not available yet` if received no rate limits 
    - nit case
    - Deleted stall snapshots that were moved to
    `codex-rs/tui/src/status/snapshots`
  • [MCP] Introduce an experimental official rust sdk based mcp client (#4252)
    The [official Rust
    SDK](https://github.com/modelcontextprotocol/rust-sdk/tree/57fc428c578a1a3fe851ee0838bf068bda120eb3)
    has come a long way since we first started our mcp client implementation
    5 months ago and, today, it is much more complete than our own
    stdio-only implementation.
    
    This PR introduces a new config flag `experimental_use_rmcp_client`
    which will use a new mcp client powered by the sdk instead of our own.
    
    To keep this PR simple, I've only implemented the same stdio MCP
    functionality that we had but will expand on it with future PRs.
    
    ---------
    
    Co-authored-by: pakrym-oai <pakrym@openai.com>