Commit Graph

23 Commits

  • [codex] Cache plugin metadata for tool suggestions (#27812)
    ## Why
    
    `built_tools` runs for every sampling request, and local plugin
    discovery was repeatedly rereading plugin manifests, skills, MCP
    configuration, and app declarations to build the same tool-suggest
    metadata.
    
    That source-derived metadata is stable until the existing plugin manager
    reloads its cache. Runtime eligibility still needs to reflect the
    current install, disable, policy, app-overlap, and authentication state.
    
    ## What changed
    
    - Add a bounded, in-memory tool-suggest metadata cache owned by
    `PluginsManager`.
    - Key cached metadata by plugin identity and source, while applying
    authentication routing each time the metadata is projected.
    - Invalidate the metadata alongside the existing loaded-plugin cache,
    including its normal configuration, marketplace refresh, and
    remote-installed-plugin invalidation paths.
    - Guard against an in-flight load repopulating stale metadata after
    invalidation.
    - Keep marketplace membership and all runtime eligibility filtering live
    rather than introducing a separate catalog or revision model.
    
    ## Impact
    
    Repeated sampling requests reuse already-loaded plugin capability
    metadata while retaining the existing plugin-manager lifecycle as the
    single freshness boundary.
    
    ## Validation
    
    - `just test -p codex-core-plugins` — 252 passed
    - Added focused coverage for cache invalidation and authentication
    reprojection.
  • [codex] [3/4] Activate endpoint plugin recommendations (#27704)
    Summary\n- Await endpoint recommendation selection while constructing
    each authenticated turn, removing the first-turn cache race.\n- Snapshot
    and filter endpoint candidates once per turn, then use that same set for
    the bounded contextual user fragment, tool exposure, and exact install
    validation.\n- Keep recommendation selection ephemeral: do not persist
    recommendation state in or gate resumed threads on prior context.\n-
    Hide the legacy list tool in endpoint mode and preserve legacy discovery
    unchanged when the endpoint is disabled or unavailable.\n- Keep remote
    plugin and connector app identities out of model-visible context and
    attach them only to Codex-owned elicitation metadata.\n\nStack\n- 3/4,
    based on #28400.\n- Endpoint client and cache: #28399.\n- Generalized
    suggestion presentation: #28400.\n- Install-schema follow-up:
    #28403.\n\nValidation\n- \n- \n- \n- \n- Full : 2,649 passed and 88
    environment-dependent tests failed because this sandbox cannot write ,
    nest Seatbelt, or locate auxiliary test binaries.
  • [codex] [1/4] Add recommended plugin endpoint cache (#28399)
    Summary
    - Add authenticated parsing for `/ps/plugins/suggested?scope=GLOBAL`,
    including remote plugin and connector app identities.
    - Validate, deduplicate, sort, and cap endpoint candidates before
    caching them by backend and account identity.
    - Deduplicate concurrent cache misses and warm recommendations from the
    existing remote-installed-plugin refresh path used at startup and after
    account changes.
    - Keep endpoint results model-invisible in this PR; failures and
    responses without `enabled: true` resolve to legacy mode.
    
    Stack
    - 1/3. Follow-up: #28400 generalizes plugin suggestion presentation
    without activating endpoint recommendations.
    - Final activation: #27704.
    
    Validation
    - `just test -p codex-core-plugins recommended_plugins`
    - `just fix -p codex-core-plugins`
    - `just fmt`
    - `git diff --check`
  • [codex] Load API curated marketplace by auth (#28383)
    ## Summary
    - choose the local OpenAI curated marketplace manifest based on auth:
    Codex backend auth gets the existing marketplace, direct provider auth
    gets `api_marketplace.json`
    - include Bedrock API key auth in the direct-provider API marketplace
    path
    - safely skip the API marketplace when `api_marketplace.json` is absent
    
    ## Validation
    - `just fmt`
    - `git diff --check origin/main...HEAD`
    - CI should run the full validation
    
    ## Manual Testing
    
    ### - New api marketplace not available for API key sign
    1. Safely not display anything from api marketplace
    <img width="1161" height="289" alt="Screenshot 2026-06-15 at 21 37 43"
    src="https://github.com/user-attachments/assets/a5f16642-8a20-4ac1-a0de-1274a4c7b5b2"
    />
    
    ### - New api marketplace for API key sign in
    1. Setup api_marketplace.json
    ```
    {
      "name": "openai-curated",
      "interface": {
        "displayName": "Codex official"
      },
      "plugins": [
        {
          "name": "linear",
          "source": {
            "source": "local",
            "path": "./plugins/linear"
          },
          "policy": {
            "installation": "AVAILABLE",
            "authentication": "ON_INSTALL"
          },
          "category": "Productivity"
        }
      ]
    }
    ```
    
    2. Log in with API key, observe that only the defined plugin from
    api_marketplace.json is available from "Codex Official" (outside of
    local testing marketplaces)
    <img width="1167" height="446" alt="Screenshot 2026-06-15 at 21 16 53"
    src="https://github.com/user-attachments/assets/7cf61477-d826-4ef6-bc05-0a23ac1c0259"
    />
    
    also checked functionality on codex app
    
    ### - SiWC users 
    Still uses 'default' marketplace.json and renders all plugins
    <img width="1171" height="502" alt="Screenshot 2026-06-15 at 21 40 25"
    src="https://github.com/user-attachments/assets/d212ea9b-0aa5-470b-8ea4-450efe65bb2b"
    />
    
    also checked functionality on codex app
    
    
    ## Notes
    - `just test -p codex-core-plugins` was started locally before splitting
    branches, but I stopped relying on local tests per follow-up and left
    final validation to PR CI.
  • [codex] Centralize plugin auth capability filtering (#27902)
    ## Summary
    
    This is the first step in making plugin auth routing consistent. The
    rule should not live as one-off checks in every place that loads or
    displays plugin capabilities.
    
    This PR introduces a small resolver for the auth-level policy: given a
    plugin's declared apps, MCP servers, current auth mode, and active
    state, return the capabilities that are actually usable in that context.
    
    ## Why
    
    Product rule:
    - SiWC auth can use app connectors, so app declarations stay available.
    - API-key/direct auth cannot use app connectors, so app declarations are
    removed.
    - When an active plugin has both an app and an MCP server with the same
    name, the app route wins for Codex-backed auth and the conflicting MCP
    server is hidden.
    
    Putting that rule in `capabilities.rs` gives the rest of the stack one
    place to ask instead of duplicating auth checks in loader, manager,
    marketplace, and details code.
    
    ## Validation
    
    - `cargo fmt`
    - `cargo test -p codex-core-plugins`
  • Discover stdio MCP servers from selected executor plugins (#27870)
    ## Why
    
    **In short:** this PR discovers MCP registrations by reading a selected
    plugin's `.mcp.json` on its executor. #27884 then resolves those
    registrations in the shared catalog.
    
    `thread/start.selectedCapabilityRoots` can select a plugin root owned by
    an executor, and Codex can resolve that package through the executor
    filesystem. MCP declarations inside the selected plugin are still
    ignored.
    
    This PR adds the source-specific discovery layer on top of the
    selected-plugin catalog boundary in #27884:
    
    ```text
    selected capability root
            |
            v
    resolve the plugin through its executor filesystem
            |
            v
    read and normalize its MCP config through the same filesystem
            |
            v
    contribute stdio registrations bound to that environment ID
    ```
    
    The existing MCP launcher and connection manager remain unchanged. MCP
    config parsing is shared with local plugins through #27863.
    
    ## What changed
    
    - Added an executor plugin MCP provider in the MCP extension.
    - Retained only the exact filesystem capability used for package
    resolution and reused it for the selected plugin's MCP config, with no
    host-filesystem fallback or unrelated process/HTTP authority.
    - Read either the manifest-declared MCP config or the default
    `.mcp.json`; a missing default file means the plugin has no MCP servers.
    - Accepted stdio servers only for this first vertical. Executor-owned
    HTTP declarations are skipped with a warning until their placement
    semantics are defined.
    - Normalized stdio registrations with the owning environment's stable
    logical ID and plugin-root working directory.
    - Resolved environment-variable names on the owning executor and
    rejected explicit local forwarding for non-local plugins.
    - Froze discovered declarations once per active thread runtime, then
    applied current managed plugin and MCP requirements when contributing
    them.
    - Carried the selected root ID, display name, and selection order into
    the catalog contribution defined by #27884.
    
    ## Behavior and scope
    
    There is intentionally no production behavior change yet. This PR
    provides the executor provider and contribution boundary, but app-server
    does not install it in this change. Existing local plugin MCP loading is
    unchanged, and no MCP process is launched by this PR alone.
    
    ## Assumptions
    
    - The selected root ID is the plugin policy identity; the manifest
    display name is presentation metadata.
    - An environment ID is a stable logical authority. Reconnection or
    replacement under the same ID does not change ownership.
    - Selected plugin packages and their manifests are trusted inputs.
    - The selected package and MCP discovery snapshot remain frozen for the
    active thread runtime.
    
    ## Follow-up
    
    The next PR installs this contributor in app-server and adds an
    end-to-end test proving that a selected plugin MCP tool launches on its
    owning executor, can be called by the model, survives an explicit MCP
    refresh, and is invisible when its root was not selected.
    
    Resume, fork, environment removal or ID changes, dynamic catalog reload,
    and executor-owned HTTP MCP placement remain separate lifecycle
    decisions.
    
    ## Verification
    
    Focused tests cover executor-only filesystem reads, missing and
    malformed config, stdio filtering and normalization, managed
    requirements, package attribution, and selection order. CI owns
    execution of the test suite.
  • Add executor-owned plugin resolution (#27692)
    ## Why
    
    CCA can select a capability root that lives in an executor environment,
    but
    Codex only had a host-filesystem plugin loader. Before selected executor
    plugins can contribute MCP servers, we need a small package boundary
    that can
    answer:
    
    > Does this selected root contain a plugin, and if so, what does its
    manifest
    > declare?
    
    The answer must come from the selected environment's filesystem. A
    failed
    executor lookup must never fall back to the orchestrator filesystem.
    
    ## What this changes
    
    This PR introduces:
    
    ```rust
    PluginProvider::resolve(root)
        -> Result<Option<ResolvedPlugin>, Error>
    ```
    
    `ExecutorPluginProvider` resolves one `SelectedCapabilityRoot` through
    its
    exact `environment_id`. It checks the recognized manifest locations,
    reads the
    manifest through that environment's `ExecutorFileSystem`, and returns an
    inert
    `ResolvedPlugin` containing:
    
    - the opaque selected-root ID;
    - the environment-bound plugin root;
    - the authority-bound manifest resource;
    - parsed metadata and authority-bound component locators.
    
    Descriptor construction rejects manifest or component paths outside the
    selected package root, so consumers cannot accidentally lose the package
    boundary when they receive a resolved plugin.
    
    If the root has no plugin manifest, resolution returns `None`, allowing
    the
    caller to treat it as a standalone capability such as a skill.
    
    ```text
    selected root: repo -> env-1:/workspace/repo
                             |
                             | env-1 filesystem only
                             v
                 .codex-plugin/plugin.json
                             |
                             v
            ResolvedPlugin { authority, root, manifest }
    ```
    
    The existing host loader and the new executor provider now share the
    same
    manifest parser. Existing `codex-core-plugins::manifest` type paths
    remain
    available through re-exports, so host behavior and callers are
    unchanged.
    
    ## Scope
    
    This is intentionally a non-user-visible package-resolution PR. It does
    not:
    
    - parse or register plugin MCP server configurations;
    - activate skills, connectors, hooks, or MCP servers;
    - change app-server wiring;
    - introduce host fallback, caching, or lifecycle behavior.
    
    #27670 has merged, and this PR is now based directly on `main`. Together
    with
    the resolved MCP catalog from #27634, it establishes the inputs needed
    for the
    executor stdio MCP vertical without changing the existing MCP runtime.
    
    ## Follow-up
    
    The next PR will consume `ResolvedPlugin`, read its declared/default MCP
    config
    through the same executor filesystem, bind supported stdio servers to
    that
    environment, and feed those registrations into the resolved MCP catalog.
    An
    app-server E2E will prove that selecting an executor plugin exposes and
    invokes
    its tool on the owning executor.
    
    Resume/fork semantics, dynamic environment replacement, and non-stdio
    placement remain separate lifecycle decisions.
    
    ## Validation
    
    - `just fmt`
    - `cargo check --tests -p codex-plugin -p codex-core-plugins`
    - `just bazel-lock-check`
    - `git diff --check`
    
    Test targets were compiled but not executed locally; CI will run the
    test and
    Clippy suites.
  • Use cached remote plugin catalog for plugin list (#26932)
    ## Summary
    
    This changes the default remote plugin marketplace listing to use the
    cached global remote catalog when it is already present on disk. The
    foreground `plugin/list` response can then return from the local catalog
    cache instead of waiting on `/ps/plugins/list`.
    
    When a cached global catalog was present at the start of the request,
    `plugin/list` still schedules a background refresh through the existing
    plugin-list background task path so the disk cache is updated for future
    requests. Cache misses keep the existing synchronous remote fetch path
    and write the cache, and they do not schedule an extra duplicate
    background `/ps/plugins/list` refresh.
    
    Installed/enabled state continues to come from the existing remote
    installed overlay path. This change only affects the global remote
    catalog directory data used by `plugin/list`.
    
    ## Testing
    
    - `just fmt`
    - `just test -p codex-app-server
    plugin_list_uses_cached_global_remote_catalog_and_refreshes_it`
    - `just test -p codex-core-plugins`
    - `git diff --check`
  • [codex] Remove legacy remote plugin startup sync (#25936)
    ## Summary
    
    - Remove the legacy startup remote plugin sync path that called
    `/plugins/list` and reconciled curated plugin cache/config.
    - Remove the `sync_plugins_from_remote` API, its result/error types,
    startup marker task, and tests that expected the legacy request.
    - Keep the current remote installed bundle sync and remote catalog flows
    (`/ps/plugins/installed` and `/ps/plugins/list`) intact.
    
    ## Validation
    
    - `just fmt`
    - `git diff --check`
    - `env HOME=/private/tmp/codex-xin-build-home
    USERPROFILE=/private/tmp/codex-xin-build-home just test -p
    codex-core-plugins`
    - Searched for legacy `/plugins/list` sync references; remaining matches
    are `/ps/plugins/list` catalog tests/code.
    
    ## Notes
    
    - `just test -p codex-app-server plugin_list` is currently blocked
    before running filtered tests by an unrelated compile error in
    `app-server/tests/suite/v2/image_generation.rs`:
    `app_test_support::McpProcess` is not exported.
  • Load plugin hooks without other plugin capabilities (#26272)
    ## Summary
    
    `hooks/list` only consumes plugin hook declarations, but previously
    loaded every enabled plugin's skills, MCP configuration, apps, and
    capability summary before discarding them.
    
    In a local benchmark, this reduced `hooks/list` latency by over 100ms
    (e.g., from 594 to 467ms on startup, and 168 to 16ms when making a
    `hooks/list` call later in the same TUI session). This is on the
    critical path to rendering the TUI, so every 10s of ms should be eyed
    skeptically (IMO).
    
    This change adds a hook-specific plugin loading path that preserves
    plugin enablement, remote/local conflict resolution, deterministic
    ordering, manifest resolution, and hook-loading warnings while skipping
    unrelated capabilities. (I think there's room for a more general design
    here that allows you to project the capabilities you need at load-time,
    but that seems unnecessary right now.)
  • [codex] Move plugin discoverable logic into core-plugins (#25783)
    ## Summary
    - Move plugin discoverable recommendation filtering from `codex-core`
    into `codex-core-plugins` behind `ToolSuggestPluginDiscoveryInput`.
    - Keep `codex-core` as a thin adapter from `Config` to the core-plugins
    API and back to `DiscoverablePluginInfo`.
    - Keep the existing discoverable allowlist private to the core-plugins
    implementation.
    
    ## Validation
    - `just fmt`
    - `just test -p codex-core list_tool_suggest_discoverable_plugins`
    - `git diff --check`
    - Read-only subagent review: no findings
  • [codex] Cache remote plugin catalog for suggestions (#25457)
    ## Summary
    - cache the global remote plugin catalog when remote plugin listing runs
    and warm it during startup
    - use the cached remote catalog in plugin install recommendations with
    canonical `plugin@openai-curated-remote` ids
    - reuse the session `PluginsManager` for plugin recommendations so
    remote cache state is visible on the recommend path
    - skip core installed-state verification for remote plugin install
    suggestions while leaving local plugin and connector verification
    unchanged
    
    ## Testing
    - `just fmt`
    - `git diff --check`
    - `cargo test -p codex-core
    list_tool_suggest_discoverable_plugins_includes_cached_remote_global_plugins`
    - `cargo test -p codex-core
    remote_plugin_install_suggestions_skip_core_installed_verification`
    - `cargo test -p codex-app-server
    plugin_list_includes_remote_marketplaces_when_remote_plugin_enabled`
    
    Earlier focused checks during the same branch: codex-tools TUI filter
    test, request_plugin_install tests, and codex-app-server build.
  • fix: plugin bundle archive handling for upload and install (#23983)
    Move plugin tar.gz packing and unpacking into a shared core-plugins
    archive helper so uploaded bundles are decoded through the same tar
    handling used for installs. This removes duplicate archive logic,
    supports GNU long-name entries on extraction, and keeps size, traversal,
    link, and entry-type checks in one place.
  • [codex] Add OpenAI Developers to tool suggest allowlist (#21423)
    ## Summary
    
    Add `openai-developers@openai-curated` to
    `TOOL_SUGGEST_DISCOVERABLE_PLUGIN_ALLOWLIST` so the OpenAI Developers
    plugin can be surfaced through tool suggestions once it is available in
    the Built by OpenAI marketplace.
    
    Update the discoverable plugin test fixture to assert the plugin is
    returned from the curated marketplace allowlist path.
    
    ## Validation
    
    - `cargo fmt --check` passed; rustfmt emitted the existing
    stable-channel warnings about `imports_granularity`.
    - `cargo test -p codex-core
    list_tool_suggest_discoverable_plugins_returns_uninstalled_curated_plugins`
    passed.
  • [plugin] Add Canva to suggesteable list. (#20474)
    - [x] Add Canva to suggesteable list.
  • [Extension] Allowlist Chrome Extension in the tool_suggest tool (#20458)
    ### Summary
    Allowlist chrome extension in tool_suggest tool
    
    ### Screenshot
    Allowlist chrome extension in tool_suggest tool
    <img width="808" height="309" alt="chrome_internal"
    src="https://github.com/user-attachments/assets/ed769d77-b635-4a40-a0c5-fbff05af3036"
    />
  • [plugins] Allow MSFT curated plugins in tool_suggest (#20304)
    ## Summary
    - [x] Move the allowlist out of core crate
    - [x] Add Teams, SharePoint, Outlook Email, and Outlook Calendar to the
    tool_suggest discoverable plugin allowlist
    - [x] Add focused coverage for Microsoft curated plugin discovery
    
    ## Testing
    - just fmt
    - cargo test -p codex-core-plugins
    - cargo test -p codex-core
    list_tool_suggest_discoverable_plugins_returns_
  • feat: Cache remote plugin bundles on install (#19914)
    Remote installs now fetch, validate, download, and cache the plugin
    bundle locally
  • Move marketplace add/remove and startup sync out of core. (#19099)
    Move more things to core-plugins.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • feat: Support remote plugin list/read. (#18452)
    Add a temporary internal remote_plugin feature flag that merges remote
    marketplaces into plugin/list and routes plugin/read through the remote
    APIs when needed, while keeping pure local marketplaces working as
    before.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Auto-upgrade configured marketplaces (#17425)
    ## Summary
    - Add best-effort auto-upgrade for user-configured Git marketplaces
    recorded in `config.toml`.
    - Track the last activated Git revision with `last_revision` so
    unchanged marketplace sources skip clone work.
    - Trigger the upgrade from plugin startup and `plugin/list`, while
    preserving existing fail-open plugin behavior with warning logs rather
    than new user-visible errors.
    
    ## Details
    - Remote configured marketplaces use `git ls-remote` to compare the
    source/ref against the recorded revision.
    - Upgrades clone into a staging directory, validate that
    `.agents/plugins/marketplace.json` exists and that the manifest name
    matches the configured marketplace key, then atomically activate the new
    root.
    - Local `.agents/plugins/marketplace.json` marketplaces remain live
    filesystem state and are not auto-pulled.
    - Existing non-curated plugin cache refresh is kicked after successful
    marketplace root upgrades.
    
    ## Validation
    - `just write-config-schema`
    - `cargo test -p codex-core marketplace_upgrade`
    - `cargo check -p codex-cli -p codex-app-server`
    - `just fix -p codex-core`
    
    Did not run the complete `cargo test` suite because the repo
    instructions require asking before a full core workspace run.
  • Extract plugin loading and marketplace logic into codex-core-plugins (#18070)
    Split plugin loading, marketplace, and related infrastructure out of
    core into codex-core-plugins, while keeping the core-facing
    configuration and orchestration flow in codex-core.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>