Celia Chen
b89d91f6ff
## Why
Some connector golden schemas use JSON Schema composition keywords
beyond `anyOf`, specifically top-level or nested `oneOf` and `allOf`.
Codex currently needs to preserve those shapes when parsing MCP tool
input schemas so connector tools do not lose valid schema structure
during normalization.
To prevent an increased Responses API error rate, this PR will be merged
after the Responses API supports top-level `oneOf`/`allOf`.
## What Changed
- Adds `oneOf` and `allOf` support to `JsonSchema`, matching the
existing `anyOf` handling.
- Traverses `oneOf` and `allOf` anywhere schema children are visited,
including sanitization, definition reachability, description stripping,
and deep schema compaction.
- Adds a final large-schema compaction pass that prunes schema objects
containing `anyOf`, `oneOf`, or `allOf` to `{}` if earlier compaction
passes still leave the schema over budget.
## Validation
Golden schema token validation over `2,025` schemas under
`golden_schemas`, all parsed successfully. Token count is `o200k_base`
over compact JSON from `parse_tool_input_schema`.
| Percentile | Before PR | After oneOf/allOf | After pruning |
|---|---:|---:|---:|
| p0 | 9 | 9 | 9 |
| p10 | 63 | 64 | 64 |
| p25 | 86 | 87 | 87 |
| p50 | 125 | 128 | 128 |
| p75 | 203 | 206 | 206 |
| p90 | 327 | 333 | 333 |
| p95 | 460 | 473 | 473 |
| p99 | 763 | 779 | 779 |
| max | 891 | 955 | 955 |
Totals:
| Parser state | Total tokens |
|---|---:|
| Before PR | 345,713 |
| After oneOf/allOf | 352,686 |
| After pruning | 352,686 |
The pruning column matches the oneOf/allOf column for this corpus
because no parsed compact golden schema remains over the `4,000`
compact-byte budget after the earlier compaction passes.
codex-tools
codex-tools is the shared support crate for building, adapting, and executing
model-visible tools outside codex-core.
Today this crate owns the host-facing tool models and helpers that no longer
need to live in core/src/tools/spec.rs or core/src/client_common.rs:
- aggregate host models such as
ToolSpec,ConfiguredToolSpec,LoadableToolSpec,ResponsesApiNamespace, andResponsesApiNamespaceTool - host discovery models used while assembling tool sets, including discoverable-tool models and request-plugin-install helpers
- host adapters such as schema sanitization, MCP/dynamic conversion, code-mode augmentation, and image-detail normalization
- shared executable-tool contracts such as
ToolExecutor,ToolCall, andToolOutput
That extraction is the first step in a longer migration. The goal is not to
move all of core/src/tools into this crate in one shot. Instead, the plan is
to peel off reusable pieces in reviewable increments while keeping
compatibility-sensitive orchestration in codex-core until the surrounding
boundaries are ready.
Vision
Over time, this crate should hold host-side tool machinery that is shared by multiple consumers, for example:
- host-visible aggregate tool models
- tool-set planning and discovery helpers
- MCP and dynamic-tool adaptation into Responses API shapes
- code-mode compatibility shims that do not depend on
codex-core - other narrowly scoped host utilities that multiple crates need
The corresponding non-goals are just as important:
- do not move
codex-coreorchestration here prematurely - do not pull
Session/TurnContext/ approval flow / runtime execution logic into this crate unless those dependencies have first been split into stable shared interfaces - do not turn this crate into a grab-bag for unrelated helper code
Migration approach
The expected migration shape is:
- Keep extension-owned executable-tool authoring in
codex-extension-api. - Move host-side planning/adaptation helpers here when they no longer need to
stay coupled to
codex-core. - Leave compatibility-sensitive adapters in
codex-corewhile downstream call sites are updated. - Only extract higher-level host infrastructure after the crate boundaries are clear and independently testable.
Crate conventions
This crate should start with stricter structure than core/src/tools so it
stays easy to grow:
src/lib.rsshould remain exports-only.- Business logic should live in named module files such as
foo.rs. - Unit tests for
foo.rsshould live in a siblingfoo_tests.rs. - The implementation file should wire tests with:
#[cfg(test)]
#[path = "foo_tests.rs"]
mod tests;
If this crate starts accumulating code that needs runtime state from
codex-core, that is a sign to revisit the extraction boundary before adding
more here.