Commit Graph

459 Commits

  • register all mcp tools with namespace (#17404)
    stacked on #17402.
    
    MCP tools returned by `tool_search` (deferred tools) get registered in
    our `ToolRegistry` with a different format than directly available
    tools. this leads to two different ways of accessing MCP tools from our
    tool catalog, only one of which works for each. fix this by registering
    all MCP tools with the namespace format, since this info is already
    available.
    
    also, direct MCP tools are registered to responsesapi without a
    namespace, while deferred MCP tools have a namespace. this means we can
    receive MCP `FunctionCall`s in both formats from namespaces. fix this by
    always registering MCP tools with namespace, regardless of deferral
    status.
    
    make code mode track `ToolName` provenance of tools so it can map the
    literal JS function name string to the correct `ToolName` for
    invocation, rather than supporting both in core.
    
    this lets us unify to a single canonical `ToolName` representation for
    each MCP tool and force everywhere to use that one, without supporting
    fallbacks.
  • [codex] Support local marketplace sources (#17756)
    ## Summary
    
    - Port marketplace source support into the shared core marketplace-add
    flow
    - Support local marketplace directory sources
    - Support direct `marketplace.json` URL sources
    - Persist the new source types in config/schema and cover them in CLI
    and app-server tests
    
    ## Validation
    
    - `cargo test -p codex-core marketplace_add`
    - `cargo test -p codex-cli marketplace_add`
    - `cargo test -p codex-app-server marketplace_add`
    - `just write-config-schema`
    - `just fmt`
    - `just fix -p codex-core`
    - `just fix -p codex-cli`
    
    ## Context
    
    Current `main` moved marketplace-add behavior into shared core code and
    still assumed only git-backed sources. This change keeps that structure
    but restores support for local directories and direct manifest URLs in
    the shared path.
  • Make skill loading filesystem-aware (#17720)
    Migrates skill loading to support reading repo skills from the remote
    environment.
  • fix: apply patch bin refresh (#17808)
    Make sure the link to apply patch binary (i.e. codex) does not die in
    case of an update
    
    Fix this:
    https://openai.slack.com/archives/C08MGJXUCUQ/p1776183247771849
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Spread AbsolutePathBuf (#17792)
    Mechanical change to promote absolute paths through code.
  • [codex-analytics] enable general analytics by default (#17389)
    ## Summary
    - Make GeneralAnalytics stable and enabled by default.
    - Update feature tests and app-server lifecycle fixtures for explicit
    general_analytics=false.
    - Keep app-server integration tests isolated from host managed config so
    explicit feature fixtures are deterministic.
    
    ## Validation
    - cargo test -p codex-features
    - cargo test -p codex-app-server general_analytics (matched 0 tests)
    - cargo test -p codex-app-server thread_start_
    - cargo test -p codex-app-server thread_fork_
    - cargo test -p codex-app-server thread_resume_
    - cargo test -p codex-app-server
    config_read_includes_system_layer_and_overrides
  • Moving updated-at timestamps to unique millisecond times (#17489)
    To allow the ability to have guaranteed-unique cursors, we make two
    important updates:
    * Add new updated_at_ms and created_at_ms columns that are in
    millisecond precision
    * Guarantee uniqueness -- if multiple items are inserted at the same
    millisecond, bump the new one by one millisecond until it becomes unique
    
    This lets us use single-number cursors for forwards and backwards paging
    through resultsets and guarantee that the cursor is a fixed point to do
    (timestamp > cursor) and get new items only.
    
    This updated implementation is backwards-compatible since multiple
    appservers can be running and won't handle the previous method well.
  • Add realtime output modality and transcript events (#17701)
    - Add outputModality to thread/realtime/start and wire text/audio output
    selection through app-server, core, API, and TUI.\n- Rename the realtime
    transcript delta notification and add a separate transcript done
    notification that forwards final text from item done without correlating
    it with deltas.
  • [codex-analytics] feature plumbing and emittance (#16640)
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/16640).
    * #16870
    * #16706
    * #16641
    * __->__ #16640
  • [codex] Refactor marketplace add into shared core flow (#17717)
    ## Summary
    
    Move `codex marketplace add` onto a shared core implementation so the
    CLI and app-server path can use one source of truth.
    
    This change:
    - adds shared marketplace-add orchestration in `codex-core`
    - switches the CLI command to call that shared implementation
    - removes duplicated CLI-only marketplace add helpers
    - preserves focused parser and add-path coverage while moving the shared
    behavior into core tests
    
    ## Why
    
    The new `marketplace/add` RPC should reuse the same underlying
    marketplace-add flow as the CLI. This refactor lands that consolidation
    first so the follow-up app-server PR can be mostly protocol and handler
    wiring.
    
    ## Validation
    
    - `cargo test -p codex-core marketplace_add`
    - `cargo test -p codex-cli marketplace_cmd`
    - `just fix -p codex-core`
    - `just fix -p codex-cli`
    - `just fmt`
  • [codex] Add symlink flag to fs metadata (#17719)
    Add `is_symlink` to FsMetadata struct.
  • Add turn item injection API (#17703)
    ## Summary
    - Add `turn/inject_items` app-server v2 request support for appending
    raw Responses API items to a loaded thread history without starting a
    turn.
    - Generate JSON schema and TypeScript protocol artifacts for the new
    params and empty response.
    - Document the new endpoint and include a request/response example.
    - Preserve compatibility with the typo alias `turn/injet_items` while
    returning the canonical method name.
    
    ## Testing
    - Not run (not requested)
  • change realtime tool description (#17699)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • app-server: Only unload threads which were unused for some time (#17398)
    Currently app-server may unload actively running threads once the last
    connection disconnects, which is not expected.
    Instead track when was the last active turn & when there were any
    subscribers the last time, also add 30 minute idleness/no subscribers
    timer to reduce the churn.
  • Add MCP tool wall time to model output (#17406)
    Include MCP wall time in the output so the model is aware of how long
    it's calls are taking.
  • Expose instruction sources (AGENTS.md) via app server (#17506)
    Addresses #17498
    
    Problem: The TUI derived /status instruction source paths from the local
    client environment, which could show stale <none> output or incorrect
    paths when connected to a remote app server.
    
    Solution: Add an app-server v2 instructionSources snapshot to thread
    start/resume/fork responses, default it to an empty list when older
    servers omit it, and render TUI /status from that server-provided
    session data.
    
    Additional context: The app-server field is intentionally named
    instructionSources rather than AGENTS.md-specific terminology because
    the loaded instruction sources can include global instructions, project
    AGENTS.md files, AGENTS.override.md, user-defined instruction files, and
    future dynamic sources.
  • Mirror user text into realtime (#17520)
    - Let typed user messages submit while realtime is active and mirror
    accepted text into the realtime text stream.
    - Add integration coverage and snapshot for outbound realtime text.
  • [mcp] Support MCP Apps part 3 - Add mcp tool call support. (#17364)
    - [x] Add a new app-server method so that MCP Apps can call their own
    MCP server directly.
  • app-server: add pipelined config rpc regression test (#17371)
    ### Summary
    Adds regression coverage for pipelined config RPC reads after writes
    ### Testing
    These are new tests
  • Revert "Option to Notify Workspace Owner When Usage Limit is Reached" (#17391)
    Reverts openai/codex#16969
    
    #sev3-2026-04-10-accountscheckversion-500s-for-openai-workspace-7300
  • Support clear SessionStart source (#17073)
    ## Motivation
    
    The `SessionStart` hook already receives `startup` and `resume` sources,
    but sessions created from `/clear` previously looked like normal startup
    sessions. This makes it impossible for hook authors to distinguish
    between these with the matcher.
    
    ## Summary
    
    - Add `InitialHistory::Cleared` so `/clear`-created sessions can be
    distinguished from ordinary startup sessions.
    - Add `SessionStartSource::Clear` and wire it through core, app-server
    thread start params, and TUI clear-session flow.
    - Update app-server protocol schemas, generated TypeScript, docs, and
    related tests.
    
    
    https://github.com/user-attachments/assets/9cae3cb4-41c7-4d06-b34f-966252442e5c
  • Queue Realtime V2 response.create while active (#17306)
    Builds on #17264.
    
    - queues Realtime V2 `response.create` while an active response is open,
    then flushes it after `response.done` or `response.cancelled`
    - requests `response.create` after background agent final output and
    steering acknowledgements
    - adds app-server integration coverage for all `response.create` paths
    
    Validation:
    - `just fmt`
    - `cargo check -p codex-app-server --tests`
    - `git diff --check`
    - CI green
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • fix: MCP leaks in app-server (#17223)
    The disconnect path now reuses the same teardown flow as explicit
    unsubscribe, and the thread-state bookkeeping consistently reports only
    threads that lost their last subscriber
    
    https://github.com/openai/codex/issues/16895
  • Stream Realtime V2 background agent progress (#17264)
    Stream Realtime V2 background agent updates while the background agent
    task is still running, then send the final tool output when it
    completes. User input during an active V2 handoff is acknowledged back
    to realtime as a steering update.
    
    Stack:
    - Depends on #17278 for the background_agent rename.
    - Depends on #17280 for the input task handler refactor.
    
    Coverage:
    - Adds an app-server integration regression test that verifies V2
    progress is sent before the final function-call output.
    
    Validation:
    - just fmt
    - cargo check -p codex-core
    - cargo check -p codex-app-server --tests
    - git diff --check
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Rename Realtime V2 tool to background_agent (#17278)
    Rename the Realtime V2 delegation tool and parser constant to
    background_agent, and update the tool description and fixtures to match.
    
    Validation: just fmt; cargo check -p codex-api; git diff --check
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Option to Notify Workspace Owner When Usage Limit is Reached (#16969)
    ## Summary
    - Replace the manual `/notify-owner` flow with an inline confirmation
    prompt when a usage-based workspace member hits a credits-depleted
    limit.
    - Fetch the current workspace role from the live ChatGPT
    `accounts/check/v4-2023-04-27` endpoint so owner/member behavior matches
    the desktop and web clients.
    - Keep owner, member, and spend-cap messaging distinct so we only offer
    the owner nudge when the workspace is actually out of credits.
    
    ## What Changed
    - `backend-client`
    - Added a typed fetch for the current account role from
    `accounts/check`.
      - Mapped backend role values into a Rust workspace-role enum.
    - `app-server` and protocol
      - Added `workspaceRole` to `account/read` and `account/updated`.
    - Derived `isWorkspaceOwner` from the live role, with a fallback to the
    cached token claim when the role fetch is unavailable.
    - `tui`
      - Removed the explicit `/notify-owner` slash command.
    - When a member is blocked because the workspace is out of credits, the
    error now prompts:
    - `Your workspace is out of credits. Request more from your workspace
    owner? [y/N]`
      - Choosing `y` sends the existing owner-notification request.
    - Choosing `n`, pressing `Esc`, or accepting the default selection
    dismisses the prompt without sending anything.
    - Selection popups now honor explicit item shortcuts, which is how the
    `y` / `n` interaction is wired.
    
    ## Reviewer Notes
    - The main behavior change is scoped to usage-based workspace members
    whose workspace credits are depleted.
    - Spend-cap reached should not show the owner-notification prompt.
    - Owners and admins should continue to see `/usage` guidance instead of
    the member prompt.
    - The live role fetch is best-effort; if it fails, we fall back to the
    existing token-derived ownership signal.
    
    ## Testing
    - Manual verification
      - Workspace owner does not see the member prompt.
    - Workspace member with depleted credits sees the confirmation prompt
    and can send the nudge with `y`.
    - Workspace member with spend cap reached does not see the
    owner-notification prompt.
    
    ### Workspace member out of usage
    
    https://github.com/user-attachments/assets/341ac396-eff4-4a7f-bf0c-60660becbea1
    
    ### Workspace owner
    <img width="1728" height="1086" alt="Screenshot 2026-04-09 at 11 48
    22 AM"
    src="https://github.com/user-attachments/assets/06262a45-e3fc-4cc4-8326-1cbedad46ed6"
    />
  • Omit empty app-server instruction overrides (#17258)
    ## Summary
    - omit serialized Responses instructions when an app-server base
    instruction override is empty
    - skip empty developer instruction messages and add v2 coverage for the
    empty-override request shape
    
    ## Validation
    - just fmt
    - git diff --check
  • [mcp] Expand tool search to custom MCPs. (#16944)
    - [x] Expand tool search to custom MCPs.
    - [x] Rename several variables/fields to be more generic.
    
    Updated tool & server name lifecycles:
    
    **Raw Identity**
    
    ToolInfo.server_name is raw MCP server name.
    ToolInfo.tool.name is raw MCP tool name.
    MCP calls route back to raw via parse_tool_name() returning
    (tool.server_name, tool.tool.name).
    mcpServerStatus/list now groups by raw server and keys tools by
    Tool.name: mod.rs:599
    App-server just forwards that grouped raw snapshot:
    codex_message_processor.rs:5245
    
    **Callable Names**
    
    On list-tools, we create provisional callable_namespace / callable_name:
    mcp_connection_manager.rs:1556
    For non-app MCP, provisional callable name starts as raw tool name.
    For codex-apps, provisional callable name is sanitized and strips
    connector name/id prefix; namespace includes connector name.
    Then qualify_tools() sanitizes callable namespace + name to ASCII alnum
    / _ only: mcp_tool_names.rs:128
    Note: this is stricter than Responses API. Hyphen is currently replaced
    with _ for code-mode compatibility.
    
    **Collision Handling**
    
    We do initially collapse example-server and example_server to the same
    base.
    Then qualify_tools() detects distinct raw namespace identities behind
    the same sanitized namespace and appends a hash to the callable
    namespace: mcp_tool_names.rs:137
    Same idea for tool-name collisions: hash suffix goes on callable tool
    name.
    Final list_all_tools() map key is callable_namespace + callable_name:
    mcp_connection_manager.rs:769
    
    **Direct Model Tools**
    
    Direct MCP tool declarations use the full qualified sanitized key as the
    Responses function name.
    The raw rmcp Tool is converted but renamed for model exposure.
    
    **Tool Search / Deferred**
    
    Tool search result namespace = final ToolInfo.callable_namespace:
    tool_search.rs:85
    Tool search result nested name = final ToolInfo.callable_name:
    tool_search.rs:86
    Deferred tool handler is registered as "{namespace}:{name}":
    tool_registry_plan.rs:248
    When a function call comes back, core recombines namespace + name, looks
    up the full qualified key, and gets the raw server/tool for MCP
    execution: codex.rs:4353
    
    **Separate Legacy Snapshot**
    
    collect_mcp_snapshot_from_manager_with_detail() still returns a map
    keyed by qualified callable name.
    mcpServerStatus/list no longer uses that; it uses
    McpServerStatusSnapshot, which is raw-inventory shaped.
  • Forward app-server turn clientMetadata to Responses (#16009)
    ## Summary
    App-server v2 already receives turn-scoped `clientMetadata`, but the
    Rust app-server was dropping it before the outbound Responses request.
    This change keeps the fix lightweight by threading that metadata through
    the existing turn-metadata path rather than inventing a new transport.
    
    ## What we're trying to do and why
    We want turn-scoped metadata from the app-server protocol layer,
    especially fields like Hermes/GAAS run IDs, to survive all the way to
    the actual Responses API request so it is visible in downstream
    websocket request logging and analytics.
    
    The specific bug was:
    - app-server protocol uses camelCase `clientMetadata`
    - Responses transport already has an existing turn metadata carrier:
    `x-codex-turn-metadata`
    - websocket transport already rewrites that header into
    `request.request_body.client_metadata["x-codex-turn-metadata"]`
    - but the Rust app-server never parsed or stored `clientMetadata`, so
    nothing from the app-server request was making it into that existing
    path
    
    This PR fixes that without adding a new header or a second metadata
    channel.
    
    ## How we did it
    ### Protocol surface
    - Add optional `clientMetadata` to v2 `TurnStartParams` and
    `TurnSteerParams`
    - Regenerate the JSON schema / TypeScript fixtures
    - Update app-server docs to describe the field and its behavior
    
    ### Runtime plumbing
    - Add a dedicated core op for app-server user input carrying turn-scoped
    metadata: `Op::UserInputWithClientMetadata`
    - Wire `turn/start` and `turn/steer` through that op / signature path
    instead of dropping the metadata at the message-processor boundary
    - Store the metadata in `TurnMetadataState`
    
    ### Transport behavior
    - Reuse the existing serialized `x-codex-turn-metadata` payload
    - Merge the new app-server `clientMetadata` into that JSON additively
    - Do **not** replace built-in reserved fields already present in the
    turn metadata payload
    - Keep websocket behavior unchanged at the outer shape level: it still
    sends only `client_metadata["x-codex-turn-metadata"]`, but that JSON
    string now contains the merged fields
    - Keep HTTP fallback behavior unchanged except that the existing
    `x-codex-turn-metadata` header now includes the merged fields too
    
    ### Request shape before / after
    Before, a websocket `response.create` looked like:
    ```json
    {
      "type": "response.create",
      "client_metadata": {
        "x-codex-turn-metadata": "{\"session_id\":\"...\",\"turn_id\":\"...\"}"
      }
    }
    ```
    Even if the app-server caller supplied `clientMetadata`, it was not
    represented there.
    
    After, the same request shape is preserved, but the serialized payload
    now includes the new turn-scoped fields:
    ```json
    {
      "type": "response.create",
      "client_metadata": {
        "x-codex-turn-metadata": "{\"session_id\":\"...\",\"turn_id\":\"...\",\"fiber_run_id\":\"fiber-start-123\",\"origin\":\"gaas\"}"
      }
    }
    ```
    
    ## Validation
    ### Targeted tests added / updated
    - protocol round-trip coverage for `clientMetadata` on `turn/start` and
    `turn/steer`
    - protocol round-trip coverage for `Op::UserInputWithClientMetadata`
    - `TurnMetadataState` merge test proving client metadata is added
    without overwriting reserved built-in fields
    - websocket request-shape test proving outbound `response.create`
    contains merged metadata inside
    `client_metadata["x-codex-turn-metadata"]`
    - app-server integration tests proving:
    - `turn/start` forwards `clientMetadata` into the outbound Responses
    request path
      - websocket warmup + real turn request both behave correctly
      - `turn/steer` updates the follow-up request metadata
    
    ### Commands run
    - `just write-app-server-schema`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-protocol`
    - `cargo test -p codex-core
    turn_metadata_state_merges_client_metadata_without_replacing_reserved_fields
    --lib`
    - `cargo test -p codex-core --test all
    responses_websocket_preserves_custom_turn_metadata_fields`
    - `cargo test -p codex-app-server --test all client_metadata`
    - `cargo test -p codex-app-server --test all
    turn_start_forwards_client_metadata_to_responses_websocket_request_body_v2
    -- --nocapture`
    - `just fmt`
    - `just fix -p codex-core -p codex-protocol -p codex-app-server-protocol
    -p codex-app-server`
    - `just fix -p codex-exec -p codex-tui-app-server`
    - `just argument-comment-lint`
    
    ### Full suite note
    `cargo test` in `codex-rs` still fails in:
    -
    `suite::v2::turn_interrupt::turn_interrupt_resolves_pending_command_approval_request`
    
    I verified that same failure on a clean detached `HEAD` worktree with an
    isolated `CARGO_TARGET_DIR`, so it is not caused by this patch.
  • chore: merge name and title (#17116)
    Merge title and name concept to leverage the sqlite title column and
    have more efficient queries
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Default realtime startup to v2 model (#17183)
    - Default realtime sessions to v2 and gpt-realtime-1.5 when no override
    is configured.
    - Add Op::RealtimeConversationStart integration coverage and keep
    v1-specific tests explicit.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add realtime voice selection (#17176)
    - Add realtime voice selection for realtime/start.
    - Expose the supported v1/v2 voice lists and cover explicit, configured,
    default, and invalid voice paths.
  • Move default realtime prompt into core (#17165)
    - Adds a core-owned realtime backend prompt template and preparation
    path.
    - Makes omitted realtime start prompts use the core default, while null
    or empty prompts intentionally send empty instructions.
    - Covers the core realtime path and app-server v2 path with integration
    coverage.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add WebRTC realtime app-server e2e tests (#17093)
    Summary:
    - add app-server WebRTC realtime e2e harness
    - cover v1 handoff and v2 codex tool delegation over sideband
    
    Validation:
    - just fmt
    - git diff --check
    - local tests not run; relying on PR CI
  • Attach WebRTC realtime starts to sideband websocket (#17057)
    Summary:
    - parse the realtime call Location header and join that call over the
    direct realtime WebSocket
    - keep WebRTC starts alive on the existing realtime conversation path
    
    Validation:
    - just fmt
    - git diff --check
    - cargo check -p codex-api
    - cargo check -p codex-core --tests
    - local cargo tests not run; relying on PR CI
  • Use model metadata for Fast Mode status (#16949)
    Fast Mode status was still tied to one model name in the TUI and
    model-list plumbing. This changes the model metadata shape so a model
    can advertise additional speed tiers, carries that field through the
    app-server model list, and uses it to decide when to show Fast Mode
    status.
    
    For people using Codex, the behavior is intended to stay the same for
    existing models. Fast Mode still requires the existing signed-in /
    feature-gated path; the difference is that the UI can now recognize any
    model the model list marks as Fast-capable, instead of requiring a new
    client-side slug check.
  • Add WebRTC transport to realtime start (#16960)
    Adds WebRTC startup to the experimental app-server
    `thread/realtime/start` method with an optional transport enum. The
    websocket path remains the default; WebRTC offers create the realtime
    session through the shared start flow and emit the answer SDP via
    `thread/realtime/sdp`.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • app-server: Move watch_id to request of fs/watch (#17026)
    It's easier for clients to maintain watchers if they define the watch
    id, so move it into the request.
    It's not used yet, so should be a safe change.
  • [codex] Make AbsolutePathBuf joins infallible (#16981)
    Having to check for errors every time join is called is painful and
    unnecessary.
  • Preserve null developer instructions (#16976)
    Preserve explicit null developer-instruction overrides across app-server
    resume and fork flows.
  • Honor null thread instructions (#16964)
    - Treat explicit null thread instructions as a blank-slate override
    while preserving omitted-field fallback behavior.
    - Preserve null through rollout resume/fork and keep explicit empty
    strings distinct.
    - Add app-server v2 start/fork coverage for the tri-state instruction
    params.
  • [mcp] Support MCP Apps part 1. (#16082)
    - [x] Add `mcpResource/read` method to read mcp resource.
  • Refactor config types into a separate crate (#16962)
    Move config types into a separate crate because their macros expand into
    a lot of new code.