Commit Graph

12 Commits

  • [tool_suggest] Improve tool_suggest triggering conditions. (#20091)
    ## Summary
    - Tighten `tool_suggest` guidance so it prefers explicit plugin install
    requests, while still allowing a connector install when the relevant
    plugin is already installed and a needed connector from that plugin is
    missing.
    - Tell the model not to call `tool_suggest` in parallel with other
    tools.
    
    ## Testing
    - `cargo test -p codex-tools tool_suggest`
    - `cargo test -p codex-core tool_suggest`
  • [tool search] support namespaced deferred dynamic tools (#18413)
    Deferred dynamic tools need to round-trip a namespace so a tool returned
    by `tool_search` can be called through the same registry key that core
    uses for dispatch.
    
    This change adds namespace support for dynamic tool specs/calls,
    persists it through app-server thread state, and routes dynamic tool
    calls by full `ToolName` while still sending the app the leaf tool name.
    Deferred dynamic tools must provide a namespace; non-deferred dynamic
    tools may remain top-level.
    
    It also introduces `LoadableToolSpec` as the shared
    function-or-namespace Responses shape used by both `tool_search` output
    and dynamic tool registration, so dynamic tools use the same wrapping
    logic in both paths.
    
    Validation:
    - `cargo test -p codex-tools`
    - `cargo test -p codex-core tool_search`
    
    ---------
    
    Co-authored-by: Sayan Sisodiya <sayan@openai.com>
  • enable tool search over dynamic tools (#18263)
    ## Summary
    
    - Normalize deferred MCP and dynamic tools into `ToolSearchEntry` values
    before constructing `ToolSearchHandler`.
    - Move the tool-search entry adapter out of `tools/handlers` and into
    `tools/tool_search_entry.rs` so the handlers directory stays focused on
    handlers.
    - Keep `ToolSearchHandler` operating over one generic entry list for
    BM25 search, namespace grouping, and per-bucket default limits.
    
    ## Why
    
    Follow-up cleanup for #17849. The dynamic tool-search support made the
    handler juggle source-specific MCP and dynamic tool lists, index
    arithmetic, output conversion, and namespace emission. This keeps source
    adaptation outside the handler so the search loop itself is smaller and
    source-agnostic.
    
    ## Validation
    
    - `just fmt`
    - `cargo test -p codex-core tools::handlers::tool_search::tests`
    - `git diff --check`
    - `cargo test -p codex-core` currently fails in unrelated
    `plugins::manager::tests::list_marketplaces_ignores_installed_roots_missing_from_config`;
    rerunning that single test fails the same way at
    `core/src/plugins/manager_tests.rs:1692`.
    
    ---------
    
    Co-authored-by: pash <pash@openai.com>
  • Fix empty tool descriptions (#17946)
    ## Summary
    - Ensure direct namespaced MCP tool groups are emitted with a non-empty
    namespace description even when namespace metadata is missing or blank.
    - Add regression coverage for missing MCP namespace descriptions.
    
    ## Cause
    Latest `main` can serialize a direct namespaced MCP tool group with an
    empty top-level `description`. The namespace description path used
    `unwrap_or_default()` when `tool_namespaces` did not include metadata
    for that namespace, so the outbound Responses API payload could contain
    a tool like `{"type":"namespace","description":""}`. The Responses API
    rejects that because namespace tool descriptions must be a non-empty
    string.
    
    ## Fix
    - Add a fallback namespace description: `Tools in the <namespace>
    namespace.`
    - Preserve provided namespace descriptions after trimming, but treat
    blank descriptions as missing.
    
    ### Issue I am seeing
    This is what I am seeing on the local build.
    <img width="1593" height="488" alt="Screenshot 2026-04-15 at 10 55 55
    AM"
    src="https://github.com/user-attachments/assets/bab668ba-bf17-4c71-be4e-b102202fce57"
    />
    
    ---------
    
    Co-authored-by: Sayan Sisodiya <sayan@openai.com>
  • register all mcp tools with namespace (#17404)
    stacked on #17402.
    
    MCP tools returned by `tool_search` (deferred tools) get registered in
    our `ToolRegistry` with a different format than directly available
    tools. this leads to two different ways of accessing MCP tools from our
    tool catalog, only one of which works for each. fix this by registering
    all MCP tools with the namespace format, since this info is already
    available.
    
    also, direct MCP tools are registered to responsesapi without a
    namespace, while deferred MCP tools have a namespace. this means we can
    receive MCP `FunctionCall`s in both formats from namespaces. fix this by
    always registering MCP tools with namespace, regardless of deferral
    status.
    
    make code mode track `ToolName` provenance of tools so it can map the
    literal JS function name string to the correct `ToolName` for
    invocation, rather than supporting both in core.
    
    this lets us unify to a single canonical `ToolName` representation for
    each MCP tool and force everywhere to use that one, without supporting
    fallbacks.
  • preserve search results order in tool_search_output (#17263)
    we used to alpha-sort tool search results because we were using
    `BTreeMap`, which threw away the actual search result ordering.
    
    Now we use a vec to preserve it.
    
    ### Tests
    Updated tests
  • [mcp] Expand tool search to custom MCPs. (#16944)
    - [x] Expand tool search to custom MCPs.
    - [x] Rename several variables/fields to be more generic.
    
    Updated tool & server name lifecycles:
    
    **Raw Identity**
    
    ToolInfo.server_name is raw MCP server name.
    ToolInfo.tool.name is raw MCP tool name.
    MCP calls route back to raw via parse_tool_name() returning
    (tool.server_name, tool.tool.name).
    mcpServerStatus/list now groups by raw server and keys tools by
    Tool.name: mod.rs:599
    App-server just forwards that grouped raw snapshot:
    codex_message_processor.rs:5245
    
    **Callable Names**
    
    On list-tools, we create provisional callable_namespace / callable_name:
    mcp_connection_manager.rs:1556
    For non-app MCP, provisional callable name starts as raw tool name.
    For codex-apps, provisional callable name is sanitized and strips
    connector name/id prefix; namespace includes connector name.
    Then qualify_tools() sanitizes callable namespace + name to ASCII alnum
    / _ only: mcp_tool_names.rs:128
    Note: this is stricter than Responses API. Hyphen is currently replaced
    with _ for code-mode compatibility.
    
    **Collision Handling**
    
    We do initially collapse example-server and example_server to the same
    base.
    Then qualify_tools() detects distinct raw namespace identities behind
    the same sanitized namespace and appends a hash to the callable
    namespace: mcp_tool_names.rs:137
    Same idea for tool-name collisions: hash suffix goes on callable tool
    name.
    Final list_all_tools() map key is callable_namespace + callable_name:
    mcp_connection_manager.rs:769
    
    **Direct Model Tools**
    
    Direct MCP tool declarations use the full qualified sanitized key as the
    Responses function name.
    The raw rmcp Tool is converted but renamed for model exposure.
    
    **Tool Search / Deferred**
    
    Tool search result namespace = final ToolInfo.callable_namespace:
    tool_search.rs:85
    Tool search result nested name = final ToolInfo.callable_name:
    tool_search.rs:86
    Deferred tool handler is registered as "{namespace}:{name}":
    tool_registry_plan.rs:248
    When a function call comes back, core recombines namespace + name, looks
    up the full qualified key, and gets the raw server/tool for MCP
    execution: codex.rs:4353
    
    **Separate Legacy Snapshot**
    
    collect_mcp_snapshot_from_manager_with_detail() still returns a map
    keyed by qualified callable name.
    mcpServerStatus/list no longer uses that; it uses
    McpServerStatusSnapshot, which is raw-inventory shaped.
  • Support anyOf and enum in JsonSchema (#16875)
    This brings us into better alignment with the JSON schema subset that is
    supported in
    <https://developers.openai.com/api/docs/guides/structured-outputs#supported-schemas>,
    and also allows us to render richer function signatures in code mode
    (e.g., anyOf{null, OtherObjectType})
  • Extract tool-search output helpers into codex-tools (#16497)
    ## Why
    
    This is the next straight-refactor step in the `codex-tools` migration
    that follows #16493.
    
    `codex-rs/core` still owned a chunk of pure tool-discovery metadata and
    response shaping even though the corresponding `tool_search` /
    `tool_suggest` specs already live in `codex-rs/tools`. Per the guidance
    in `AGENTS.md`, this moves that crate-agnostic logic out of `codex-core`
    so the handler crate keeps only the BM25 ranking/orchestration and
    runtime glue.
    
    ## What changed
    
    - Moved the canonical `tool_search` / `tool_suggest` tool names and the
    `tool_search` default limit into `codex-rs/tools/src/tool_discovery.rs`.
    - Added `ToolSearchResultSource` and
    `collect_tool_search_output_tools()` in `codex-tools` so namespace
    grouping and deferred Responses API tool serialization happen outside
    `codex-core`.
    - Rewired `ToolSearchHandler`, `ToolSuggestHandler`, and
    `core/src/tools/spec.rs` to consume those exports directly from
    `codex-tools`.
    - Ported the existing `tool_search` serializer tests from
    `core/src/tools/handlers/tool_search_tests.rs` to
    `tools/src/tool_discovery_tests.rs` without adding new behavior
    coverage.
    
    ## Validation
    
    ```shell
    cargo test -p codex-tools
    cargo test -p codex-core tools::spec::tests
    just argument-comment-lint
    ```
  • Extract tool discovery helpers into codex-tools (#16477)
    ## Why
    
    Follow-up to #16379 and #16471.
    
    `codex-rs/core/src/tools/spec.rs` still owned the pure discovery-shaping
    helpers that turn app metadata and discoverable tool metadata into the
    inputs used by `tool_search` and `tool_suggest`. Those helpers do not
    need `codex-core` runtime state, so keeping them in `codex-core`
    continued to blur the crate boundary this migration is trying to
    tighten.
    
    This change keeps pushing spec-only logic behind the `codex-tools` API
    so `codex-core` can focus on wiring runtime handlers to the resulting
    tool definitions.
    
    ## What Changed
    
    - Added `collect_tool_search_app_infos` and
    `collect_tool_suggest_entries` to
    `codex-rs/tools/src/tool_discovery.rs`.
    - Added a small `ToolSearchAppSource` adapter type in `codex-tools` so
    `codex-core` can pass app metadata into that shared helper logic without
    exposing `ToolInfo` across the crate boundary.
    - Re-exported the new discovery helpers from
    `codex-rs/tools/src/lib.rs`, which remains exports-only.
    - Updated `codex-rs/core/src/tools/spec.rs` to use those `codex-tools`
    helpers instead of maintaining local `tool_search_app_infos` and
    `tool_suggest_entries` functions.
    - Removed the now-redundant helper implementations from `codex-core`.
    
    ## Testing
    
    - `cargo test -p codex-tools`
    - `cargo test -p codex-core tools::spec::tests`
  • codex-tools: extract discoverable tool models (#16254)
    ## Why
    
    `#16193` moved the pure `tool_search` and `tool_suggest` spec builders
    into `codex-tools`, but `codex-core` still owned the shared
    discoverable-tool model that those builders and the `tool_suggest`
    runtime both depend on. This change continues the migration by moving
    that reusable model boundary out of `codex-core` as well, so the
    discovery/suggestion stack uses one shared set of types and
    `core/src/tools` no longer needs its own `discoverable.rs` module.
    
    ## What changed
    
    - Moved `DiscoverableTool`, `DiscoverablePluginInfo`, and
    `filter_tool_suggest_discoverable_tools_for_client()` into
    `codex-rs/tools/src/tool_discovery.rs` alongside the extracted
    discovery/suggestion spec builders.
    - Added `codex-app-server-protocol` as a `codex-tools` dependency so the
    shared discoverable-tool model can own the connector-side `AppInfo`
    variant directly.
    - Updated `core/src/tools/handlers/tool_suggest.rs`,
    `core/src/tools/spec.rs`, `core/src/tools/router.rs`,
    `core/src/connectors.rs`, and `core/src/codex.rs` to consume the shared
    `codex-tools` model instead of the old core-local declarations.
    - Changed `core/src/plugins/discoverable.rs` to return
    `DiscoverablePluginInfo` directly, moved the pure client-filter coverage
    into `tool_discovery_tests.rs`, and deleted the old
    `core/src/tools/discoverable.rs` module.
    - Updated `codex-rs/tools/README.md` so the crate boundary documents
    that `codex-tools` now owns the discoverable-tool models in addition to
    the discovery/suggestion spec builders.
    
    ## Test plan
    
    - `cargo test -p codex-tools`
    - `CARGO_TARGET_DIR=/tmp/codex-core-discoverable-model cargo test -p
    codex-core --lib tools::handlers::tool_suggest::`
    - `CARGO_TARGET_DIR=/tmp/codex-core-discoverable-model cargo test -p
    codex-core --lib tools::spec::`
    - `CARGO_TARGET_DIR=/tmp/codex-core-discoverable-model cargo test -p
    codex-core --lib plugins::discoverable::`
    - `just bazel-lock-check`
    - `just argument-comment-lint`
    
    ## References
    
    - #16193
    - #16154
    - #15923
    - #15928
    - #15944
    - #15953
    - #16031
    - #16047
    - #16129
    - #16132
    - #16138
    - #16141
  • codex-tools: extract discovery tool specs (#16193)
    ## Why
    
    `core/src/tools/spec.rs` still owned the pure `tool_search` and
    `tool_suggest` spec builders even though that logic no longer needed
    `codex-core` runtime state. This change continues the `codex-tools`
    migration by moving the reusable discovery and suggestion spec
    construction out of `codex-core` so `spec.rs` is left with the
    core-owned policy decisions about when these tools are exposed and what
    metadata is available.
    
    ## What changed
    
    - Added `codex-rs/tools/src/tool_discovery.rs` with the shared
    `tool_search` and `tool_suggest` spec builders, plus focused unit tests
    in `tool_discovery_tests.rs`.
    - Moved the shared `DiscoverableToolAction` and `DiscoverableToolType`
    declarations into `codex-tools` so the `tool_suggest` handler and the
    extracted spec builders use the same wire-model enums.
    - Updated `core/src/tools/spec.rs` to translate `ToolInfo` and
    `DiscoverableTool` values into neutral `codex-tools` inputs and delegate
    the actual spec building there.
    - Removed the old template-based description rendering helpers from
    `core/src/tools/spec.rs` and deleted the now-dead helper methods in
    `core/src/tools/discoverable.rs`.
    - Updated `codex-rs/tools/README.md` to document that discovery and
    suggestion models/spec builders now live in `codex-tools`.
    
    ## Test plan
    
    - `cargo test -p codex-tools`
    - `CARGO_TARGET_DIR=/tmp/codex-core-discovery-specs cargo test -p
    codex-core --lib tools::spec::`
    - `CARGO_TARGET_DIR=/tmp/codex-core-discovery-specs cargo test -p
    codex-core --lib tools::handlers::tool_suggest::`
    - `just argument-comment-lint`
    
    ## References
    
    - #16154
    - #15923
    - #15928
    - #15944
    - #15953
    - #16031
    - #16047
    - #16129
    - #16132
    - #16138
    - #16141