Commit Graph

20 Commits

  • Adjust MCP tool approval handling for custom servers (#11787)
    Summary
    This PR expands MCP client-side approval behavior beyond codex_apps and
    tightens elicitation capability signaling.
    
    - Removed the codex_apps-only gate in MCP tool approval checks, so
    local/custom MCP servers are now eligible for the same client-side
    approval prompt flow when tool annotations indicate side effects.
    - Updated approval memory keying to support tools without a connector ID
    (connector_id: Option<String>), allowing “Approve this Session” to be
    remembered even when connector metadata is missing.
    - Updated prompt text for non-codex_apps tools to identify origin as The
    <server> MCP server instead of This app.
    - Added MCP initialization capability policy so only codex_apps
    advertises MCP elicitation capability; other servers advertise no
    elicitation support.
    - Added regression tests for:
    server-specific prompt copy behavior
    codex-apps-only elicitation capability advertisement
    
    Testing
    - Not run (not requested)
  • support app usage analytics (#11687)
    Emit app mentioned and app used events. Dedup by (turn_id, connector_id)
    
    Example event params:
    {
        "event_type": "codex_app_used",
        "connector_id": "asdk_app_xxx",
        "thread_id": "019c5527-36d4-xxx",
        "turn_id": "019c552c-cd17-xxx",
        "app_name": "Slack (OpenAI Internal)",
        "product_client_id": "codex_cli_rs",
        "invoke_type": "explicit",
        "model_slug": "gpt-5.3-codex"
    }
  • Sanitize MCP image output for text-only models (#11346)
    - Replace image blocks in MCP tool results with a text placeholder when
    the active model does not accept image input.
    - Add an e2e rmcp test to verify sanitized tool output is what gets sent
    back to the model.
  • feat(app-server, core): allow text + image content items for dynamic tool outputs (#10567)
    Took over the work that @aaronl-openai started here:
    https://github.com/openai/codex/pull/10397
    
    Now that app-server clients are able to set up custom tools (called
    `dynamic_tools` in app-server), we should expose a way for clients to
    pass in not just text, but also image outputs. This is something the
    Responses API already supports for function call outputs, where you can
    pass in either a string or an array of content outputs (text, image,
    file):
    https://platform.openai.com/docs/api-reference/responses/create#responses_create-input-input_item_list-item-function_tool_call_output-output-array-input_image
    
    So let's just plumb it through in Codex (with the caveat that we only
    support text and image for now). This is implemented end-to-end across
    app-server v2 protocol types and core tool handling.
    
    ## Breaking API change
    NOTE: This introduces a breaking change with dynamic tools, but I think
    it's ok since this concept was only recently introduced
    (https://github.com/openai/codex/pull/9539) and it's better to get the
    API contract correct. I don't think there are any real consumers of this
    yet (not even the Codex App).
    
    Old shape:
    `{ "output": "dynamic-ok", "success": true }`
    
    New shape:
    ```
    {
        "contentItems": [
          { "type": "inputText", "text": "dynamic-ok" },
          { "type": "inputImage", "imageUrl": "data:image/png;base64,AAA" }
        ]
      "success": true
    }
    ```
  • Add option to approve and remember MCP/Apps tool usage (#10584)
    This PR adds a new approval option for app/MCP tool calls: “Allow and
    remember” (session-scoped).
    When selected, Codex stores a temporary approval and auto-approves
    matching future calls for the rest of the session.
    
    Added a session-scoped approval key (`server`, `connector_id`,
    `tool_name`) and persisted it in `tool_approvals` as
    `ApprovedForSession`.
    On subsequent matching calls, approval is skipped and treated as
    accepted.
    - Updated the approval question options to conditionally include:
    - Accept
    - Allow and remember (conditional)
    - Decline
    - Cancel
    
    The new “Allow and remember” option is only shown when all of these are
    true:
    
    1. The call is routed through the Codex Apps MCP server (codex_apps).
    2. The tool requires approval based on annotations:
    - read_only_hint == false, and
    - destructive_hint == true or open_world_hint == true.
    3. The tool includes a connector_id in metadata (used to build the
    remembered approval key).
    
    If no `connector_id` is present, the prompt still appears (when approval
    is required), but only with the existing choices (Accept / Decline /
    Cancel). Approval prompting in this path has an explicit early return
    unless server == `codex_apps`.
  • Stop client from being state carrier (#10595)
    I'd like to make client session wide. This requires shedding all random
    state it has to carry.
  • feat: replace custom mcp-types crate with equivalents from rmcp (#10349)
    We started working with MCP in Codex before
    https://crates.io/crates/rmcp was mature, so we had our own crate for
    MCP types that was generated from the MCP schema:
    
    
    https://github.com/openai/codex/blob/8b95d3e082376f4cb23e92641705a22afb28a9da/codex-rs/mcp-types/README.md
    
    Now that `rmcp` is more mature, it makes more sense to use their MCP
    types in Rust, as they handle details (like the `_meta` field) that our
    custom version ignored. Though one advantage that our custom types had
    is that our generated types implemented `JsonSchema` and `ts_rs::TS`,
    whereas the types in `rmcp` do not. As such, part of the work of this PR
    is leveraging the adapters between `rmcp` types and the serializable
    types that are API for us (app server and MCP) introduced in #10356.
    
    Note this PR results in a number of changes to
    `codex-rs/app-server-protocol/schema`, which merit special attention
    during review. We must ensure that these changes are still
    backwards-compatible, which is possible because we have:
    
    ```diff
    - export type CallToolResult = { content: Array<ContentBlock>, isError?: boolean, structuredContent?: JsonValue, };
    + export type CallToolResult = { content: Array<JsonValue>, structuredContent?: JsonValue, isError?: boolean, _meta?: JsonValue, };
    ```
    
    so `ContentBlock` has been replaced with the more general `JsonValue`.
    Note that `ContentBlock` was defined as:
    
    ```typescript
    export type ContentBlock = TextContent | ImageContent | AudioContent | ResourceLink | EmbeddedResource;
    ```
    
    so the deletion of those individual variants should not be a cause of
    great concern.
    
    Similarly, we have the following change in
    `codex-rs/app-server-protocol/schema/typescript/Tool.ts`:
    
    ```
    - export type Tool = { annotations?: ToolAnnotations, description?: string, inputSchema: ToolInputSchema, name: string, outputSchema?: ToolOutputSchema, title?: string, };
    + export type Tool = { name: string, title?: string, description?: string, inputSchema: JsonValue, outputSchema?: JsonValue, annotations?: JsonValue, icons?: Array<JsonValue>, _meta?: JsonValue, };
    ```
    
    so:
    
    - `annotations?: ToolAnnotations` ➡️ `JsonValue`
    - `inputSchema: ToolInputSchema` ➡️ `JsonValue`
    - `outputSchema?: ToolOutputSchema` ➡️ `JsonValue`
    
    and two new fields: `icons?: Array<JsonValue>, _meta?: JsonValue`
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/10349).
    * #10357
    * __->__ #10349
    * #10356
  • MCP tool call approval (simplified version) (#10200)
    Add elicitation approval request for MCP tool call requests.
  • [MCP] Render MCP tool call result images to the model (#5600)
    It's pretty amazing we have gotten here without the ability for the
    model to see image content from MCP tool calls.
    
    This PR builds off of 4391 and fixes #4819. I would like @KKcorps to get
    adequete credit here but I also want to get this fix in ASAP so I gave
    him a week to update it and haven't gotten a response so I'm going to
    take it across the finish line.
    
    
    This test highlights how absured the current situation is. I asked the
    model to read this image using the Chrome MCP
    <img width="2378" height="674" alt="image"
    src="https://github.com/user-attachments/assets/9ef52608-72a2-4423-9f5e-7ae36b2b56e0"
    />
    
    After this change, it correctly outputs:
    > Captured the page: image dhows a dark terminal-style UI labeled
    `OpenAI Codex (v0.0.0)` with prompt `model: gpt-5-codex medium` and
    working directory `/codex/codex-rs`
    (and more)  
    
    Before this change, it said:
    > Took the full-page screenshot you asked for. It shows a long,
    horizontally repeating pattern of stylized people in orange, light-blue,
    and mustard clothing, holding hands in alternating poses against a white
    background. No text or other graphics-just rows of flat illustration
    stretching off to the right.
    
    Without this change, the Figma, Playwright, Chrome, and other visual MCP
    servers are pretty much entirely useless.
    
    I tested this change with the openai respones api as well as a third
    party completions api
  • Pass TurnContext around instead of sub_id (#5421)
    Today `sub_id` is an ID of a single incoming Codex Op submition. We then
    associate all events triggered by this operation using the same
    `sub_id`.
    
    At the same time we are also creating a TurnContext per submission and
    we'd like to start associating some events (item added/item completed)
    with an entire turn instead of just the operation that started it.
    
    Using turn context when sending events give us flexibility to change
    notification scheme.
  • [MCP] Render full MCP errors to the model (#5298)
    Previously, the model couldn't see why MCP tool calls failed, many of
    which were the model using the parameters incorrectly. A common failure
    is the model stringifying the json for the notion-update-page tool which
    it then couldn't correct.
    
    I want to do some system prompt massaging around this as well. However,
    it is crucial that the model sees the error so it can fix it.
    
    Before:
    <img width="2984" height="832" alt="CleanShot 2025-10-17 at 13 02 36"
    src="https://github.com/user-attachments/assets/709a3d27-b71b-4d8d-87b6-9b2d7fe4e6f2"
    />
    
    After:
    <img width="2488" height="1550" alt="CleanShot 2025-10-17 at 13 01 18"
    src="https://github.com/user-attachments/assets/13a0b7dc-fdad-4996-bf2d-0772872c34fc"
    />
    
    🎉 
    <img width="1078" height="568" alt="CleanShot 2025-10-17 at 13 09 30"
    src="https://github.com/user-attachments/assets/64cde8be-9e6c-4e61-b971-c2ba22504292"
    />
    
    
    Fixes #4707
  • timeouts for mcp tool calls (#3959)
    defaults to 60sec, overridable with MCP_TOOL_TIMEOUT or on a per-server
    basis in the config.
  • Move models.rs to protocol (#2595)
    Moving models.rs to protocol so we can use them in `Codex` operations
  • remove conversation history widget (#1727)
    this widget is no longer used.
  • fix: introduce ResponseInputItem::McpToolCallOutput variant (#1151)
    The output of an MCP server tool call can be one of several types, but
    to date, we treated all outputs as text by showing the serialized JSON
    as the "tool output" in Codex:
    
    
    https://github.com/openai/codex/blob/25a9949c49194d5a64de54a11bcc5b4724ac9bd5/codex-rs/mcp-types/src/lib.rs#L96-L101
    
    This PR adds support for the `ImageContent` variant so we can now
    display an image output from an MCP tool call.
    
    In making this change, we introduce a new
    `ResponseInputItem::McpToolCallOutput` variant so that we can work with
    the `mcp_types::CallToolResult` directly when the function call is made
    to an MCP server.
    
    Though arguably the more significant change is the introduction of
    `HistoryCell::CompletedMcpToolCallWithImageOutput`, which is a cell that
    uses `ratatui_image` to render an image into the terminal. To support
    this, we introduce `ImageRenderCache`, cache a
    `ratatui_image::picker::Picker`, and `ensure_image_cache()` to cache the
    appropriate scaled image data and dimensions based on the current
    terminal size.
    
    To test, I created a minimal `package.json`:
    
    ```json
    {
      "name": "kitty-mcp",
      "version": "1.0.0",
      "type": "module",
      "description": "MCP that returns image of kitty",
      "main": "index.js",
      "dependencies": {
        "@modelcontextprotocol/sdk": "^1.12.0"
      }
    }
    ```
    
    with the following `index.js` to define the MCP server:
    
    ```js
    #!/usr/bin/env node
    
    import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
    import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js";
    import { readFile } from "node:fs/promises";
    import { join } from "node:path";
    
    const IMAGE_URI = "image://Ada.png";
    
    const server = new McpServer({
      name: "Demo",
      version: "1.0.0",
    });
    
    server.tool(
      "get-cat-image",
      "If you need a cat image, this tool will provide one.",
      async () => ({
        content: [
          { type: "image", data: await getAdaPngBase64(), mimeType: "image/png" },
        ],
      })
    );
    
    server.resource("Ada the Cat", IMAGE_URI, async (uri) => {
      const base64Image = await getAdaPngBase64();
      return {
        contents: [
          {
            uri: uri.href,
            mimeType: "image/png",
            blob: base64Image,
          },
        ],
      };
    });
    
    async function getAdaPngBase64() {
      const __dirname = new URL(".", import.meta.url).pathname;
      // From https://github.com/benjajaja/ratatui-image/blob/9705ce2c59ec669abbce2924cbfd1f5ae22c9860/assets/Ada.png
      const filePath = join(__dirname, "Ada.png");
      const imageData = await readFile(filePath);
      const base64Image = imageData.toString("base64");
      return base64Image;
    }
    
    const transport = new StdioServerTransport();
    await server.connect(transport);
    ```
    
    With the local changes from this PR, I added the following to my
    `config.toml`:
    
    ```toml
    [mcp_servers.kitty]
    command = "node"
    args = ["/Users/mbolin/code/kitty-mcp/index.js"]
    ```
    
    Running the TUI from source:
    
    ```
    cargo run --bin codex -- --model o3 'I need a picture of a cat'
    ```
    
    I get:
    
    <img width="732" alt="image"
    src="https://github.com/user-attachments/assets/bf80b721-9ca0-4d81-aec7-77d6899e2869"
    />
    
    Now, that said, I have only tested in iTerm and there is definitely some
    funny business with getting an accurate character-to-pixel ratio
    (sometimes the `CompletedMcpToolCallWithImageOutput` thinks it needs 10
    rows to render instead of 4), so there is still work to be done here.
  • fix: change EventMsg enum so every variant takes a single struct (#925)
    https://github.com/openai/codex/pull/922 did this for the
    `SessionConfigured` enum variant, and I think it is generally helpful to
    be able to work with the values as each enum variant as their own type,
    so this converts the remaining variants and updates all of the
    callsites.
    
    Added a simple unit test to verify that the JSON-serialized version of
    `Event` does not have any unexpected nesting.
  • fix: add optional timeout to McpClient::send_request() (#852)
    We now impose a 10s timeout on the initial `tools/list` request to an
    MCP server. We do not apply a timeout for other types of requests yet,
    but we should start enforcing those, as well.
  • fix: make all fields of Session struct private again (#840)
    https://github.com/openai/codex/pull/829 noted it introduced a circular
    dep between `codex.rs` and `mcp_tool_call.rs`. This attempts to clean
    things up: the circular dep still exists, but at least all the fields of
    `Session` are private again.
  • feat: support mcp_servers in config.toml (#829)
    This adds initial support for MCP servers in the style of Claude Desktop
    and Cursor. Note this PR is the bare minimum to get things working end
    to end: all configured MCP servers are launched every time Codex is run,
    there is no recovery for MCP servers that crash, etc.
    
    (Also, I took some shortcuts to change some fields of `Session` to be
    `pub(crate)`, which also means there are circular deps between
    `codex.rs` and `mcp_tool_call.rs`, but I will clean that up in a
    subsequent PR.)
    
    `codex-rs/README.md` is updated as part of this PR to explain how to use
    this feature. There is a bit of plumbing to route the new settings from
    `Config` to the business logic in `codex.rs`. The most significant
    chunks for new code are in `mcp_connection_manager.rs` (which defines
    the `McpConnectionManager` struct) and `mcp_tool_call.rs`, which is
    responsible for tool calls.
    
    This PR also introduces new `McpToolCallBegin` and `McpToolCallEnd`
    event types to the protocol, but does not add any handlers for them.
    (See https://github.com/openai/codex/pull/836 for initial usage.)
    
    To test, I added the following to my `~/.codex/config.toml`:
    
    ```toml
    # Local build of https://github.com/hideya/mcp-server-weather-js
    [mcp_servers.weather]
    command = "/Users/mbolin/code/mcp-server-weather-js/dist/index.js"
    args = []
    ```
    
    And then I ran the following:
    
    ```
    codex-rs$ cargo run --bin codex exec 'what is the weather in san francisco'
    [2025-05-06T22:40:05] Task started: 1
    [2025-05-06T22:40:18] Agent message: Here’s the latest National Weather Service forecast for San Francisco (downtown, near 37.77° N, 122.42° W):
    
    This Afternoon (Tue):
    • Sunny, high near 69 °F
    • West-southwest wind around 12 mph
    
    Tonight:
    • Partly cloudy, low around 52 °F
    • SW wind 7–10 mph
    ...
    ```
    
    Note that Codex itself is not able to make network calls, so it would
    not normally be able to get live weather information like this. However,
    the weather MCP is [currently] not run under the Codex sandbox, so it is
    able to hit `api.weather.gov` and fetch current weather information.
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/829).
    * #836
    * __->__ #829