Commit Graph

1 Commits

  • feat(core, mcp): cache codex_apps tools in memory (#29003)
    ## Description
    
    This makes Codex Apps tool reads use a shared in-memory snapshot instead
    of rereading the disk cache every time `list_all_tools()` runs. Disk
    still seeds the cache on startup and gets updated after successful
    fetches, but it is no longer the live read path.
    
    The core change is that `McpManager` now owns a process-scoped
    `CodexAppsToolsCache`. Codex threads in the same app-server process now
    share this Codex Apps in-memory tools snapshot. The snapshot is keyed by
    the Codex home plus the Codex Apps identity: the active Codex auth
    user/workspace and the effective Codex Apps MCP source config.
    
    There's already code to hard-refresh the cache, so we respect it in this
    PR.
    
    ## Local benchmark
    
    I ran a local steady-state microbenchmark of the exact repeated Codex
    Apps cached-tools read this PR removes, using the same real local cache
    payload in both trees: `3,678,138` bytes and `381` tools. The cache file
    was already warm in the OS page cache, so this measures same-process
    reread/deserialization work rather than cold-disk latency or full turn
    latency. Each run is 25 iterations (mimicking a turn that makes 25
    inference calls).
    
    | Version | Run 1 | Run 2 | Avg |
    |---|---:|---:|---:|
    | `origin/main` disk read + JSON deserialize + `filter_tools` | `50.755
    ms` | `52.894 ms` | `51.825 ms` |
    | This branch in-memory `current_tools` + `filter_tools` | `0.740 ms` |
    `0.778 ms` | `0.759 ms` |
    
    That removes about `51 ms` from each repeated Codex Apps cached-tools
    read on this machine, roughly `68x` faster for that subpath. It is
    useful evidence for the hot path this PR changes, but not a claim that
    every production turn gets `51 ms` faster; end-to-end impact also
    depends on the rest of `list_all_tools()` and tool-payload construction.
    
    This is on my M2 Max macbook, so with a slower disk this would be much
    worse (and indeed we did see this really blew up turn runtime with a
    slow disk).