Commit Graph

351 Commits

  • Propagate safety buffering treatment metadata (#29473)
    ## Summary
    
    - read the request-scoped safety-buffering treatment from HTTP response
    headers and per-turn WebSocket metadata through one shared header parser
    - combine that treatment with Responses API safety-buffering signals
    - propagate `showBufferingUi` and nullable `fasterModel` through the
    existing `model/safetyBuffering/updated` app-server notification
    - update the app-server documentation and generated JSON and TypeScript
    schemas
    
    The public implementation contains no model mapping or real model
    identifier. Tests and protocol examples use generic `current-model` and
    `faster-model` placeholders only.
    
    ## Dependencies
    
    - server-side treatment evaluation:
    https://github.com/openai/openai/pull/1060247
    - initial Responses API safety-buffering propagation:
    https://github.com/openai/codex/pull/29371
    - Codex App UI: https://github.com/openai/openai/pull/1057789
    
    ## Validation
    
    - Codex API tests: 129 passed
    - focused Codex core safety-buffering integration test passed
    - app-server protocol tests passed after regenerating schema fixtures
    - Clippy fix and repository formatting completed successfully
    
    The broader app-server run compiled all changed crates and completed
    with 1,269 passing tests. Its remaining failures were unrelated
    environment limitations: macOS sandbox application was denied, one
    expected test binary was unavailable, and several existing subprocess
    tests timed out as a result.
  • [codex] reject remote images at app-server ingress (#29419)
    ## Stack
    
    Stacked on #29417. Review and land that PR first.
    
    ## Summary
    
    - reject HTTP(S) image URLs in the handlers for `turn/start` and
    `turn/steer`
    - validate `thread/inject_items` after its existing
    JSON-to-`ResponseItem` conversion, so each item is deserialized once
    - turn invalid dynamic-tool image responses into the existing
    unsuccessful text fallback; the model receives the validation message as
    the function output
    - leave `thread/resume.history` compatible with legacy history; #29417
    replaces remote images before model input
    - continue accepting inline data URLs and `localImage` inputs
    - keep this policy in app-server; this PR does not add a shared protocol
    API or change core image preparation
    
    ## Test plan
    
    - `just test -p codex-app-server -E
    'test(/request_handlers_reject_remote_image_urls|dynamic_tool_remote_image_response_becomes_model_visible_error|dynamic_tool_call_round_trip_sends_content_items_to_model|turn_start_tracks_turn_event_analytics|standalone_image_edit_uses_recent_pathless_image/)'`
    (5 passed)
    - `just fix -p codex-app-server`
    - `just fmt`
  • permission profiles: expose availability to clients (#26678)
    ## Why
    
    `permissionProfile/list` currently advertises every built-in and
    configured profile even when effective enterprise requirements prevent
    selecting it. That forces each client to reconstruct policy from
    lower-level requirement fields, which is easy to miss and difficult to
    keep consistent.
    
    The catalog should remain complete so clients can explain that an option
    was disabled by an administrator, while also reporting whether each
    profile is selectable.
    
    ## What
    
    - Add an `allowed` field to each permission profile summary.
    - Build a shared catalog from the effective config and current
    requirements, including `allowed_sandbox_modes`, `allowed_permissions`,
    and filesystem restrictions.
    - Use the shared catalog in app-server and the TUI so disallowed
    profiles remain visible but cannot be selected.
    - Use the canonical `:danger-full-access` profile ID in the TUI.
    - Update the app-server schemas, API documentation, behavioral tests,
    and TUI snapshots.
    
    ## Scope
    
    This PR targets `main` directly and is independent of #24852. It
    preserves the current behavior where built-in profiles are constrained
    by sandbox-mode requirements and `allowed_permissions` applies to
    configured profiles.
    
    ## Testing
    
    - `just test -p codex-core
    permission_profile_catalog_marks_profiles_disallowed_by_requirements`
    - `just test -p codex-app-server permission_profile_list`
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-tui profile_permissions`
    - `just fix -p codex-core`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-app-server`
    - `just fix -p codex-tui`
    - `just fmt`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
    Co-authored-by: Joey Trasatti <joey.trasatti@openai.com>
  • Allow ChatGPT accounts without email (#28991)
    # Summary
    
    Codex required every ChatGPT account to have an email address. A
    service-account personal access token can return valid account metadata
    without one, so PAT login failed while decoding the metadata response.
    
    This change makes email optional in the account metadata type that owns
    it and preserves that absence through authentication, provider account
    state, the app-server API, generated clients, and TUI bootstrap.
    Existing accounts with email addresses keep the same behavior.
    
    ## Behavior-changing call sites
    
    | Call site | Behavior after this change |
    | --- | --- |
    | `login/src/auth/personal_access_token.rs` | PAT metadata accepts a
    missing or null email and retains `None`. |
    | `agent-identity/src/lib.rs` | Agent Identity JWT claims accept an
    omitted email. |
    | `login/src/auth/storage.rs` and `login/src/auth/agent_identity.rs` |
    Stored and managed Agent Identity records carry `Option<String>`.
    Deserialization maps the legacy empty-string sentinel to `None`. |
    | `login/src/auth/manager.rs` | `get_account_email` returns the stored
    option, and managed identity bootstrap no longer converts `None` to an
    empty string. |
    | `model-provider/src/provider.rs` and `protocol/src/account.rs` | A
    ChatGPT provider account requires a plan type but may carry no email. |
    | `app-server-protocol/src/protocol/v2/account.rs` | `account/read`
    keeps the `email` field on the wire and returns `null` when the account
    has no email. Generated TypeScript and JSON schemas describe a required,
    nullable field. |
    | `sdk/python/src/openai_codex/generated/v2_all.py` | The generated
    Python `ChatgptAccount` model accepts `None` for email. |
    | `tui/src/app_server_session.rs` | Email-less ChatGPT accounts
    bootstrap normally, keep external feedback routing, omit account-email
    telemetry, and display the plan in account status. |
    
    ## Design decisions
    
    - Missing email remains `None` at every layer. The code never uses an
    empty string as a substitute.
    - The app-server response includes `"email": null` instead of omitting
    the field. Clients retain a stable response shape.
    - Plan type remains required for provider account state. This change
    relaxes only the email assumption.
    
    ## Testing
    
    Tests: affected test targets compile, scoped Clippy and formatting pass,
    a focused TUI snapshot covers plan-only account status, real
    before/after PAT login smoke covers metadata without email, app-server
    smoke covers `account/read` with `email: null`, and a regression smoke
    covers an existing email-bearing PAT. Unit tests run in CI.
    
    ## Evidence
    
    Visual smoke evidence will be attached here.
  • Add workspace messages app-server API (#29001)
    ## Summary
    
    - Add backend-client types and fetch support for active workspace
    messages.
    - Add the app-server v2 `account/workspaceMessages/read` method,
    generated schemas, and README documentation.
    - Delegate workspace-message eligibility to the Codex backend feature
    gate; map a backend 404 to `featureEnabled: false`.
    
    ## Testing
    
    - `just write-app-server-schema`
    - `just test -p codex-backend-client`
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server workspace_messages`
    - `just fix -p codex-backend-client -p codex-app-server-protocol -p
    codex-app-server`
    - `just fmt`
    
    ## Stack
    
    - Base PR for #28232, which adds the TUI status-line integration.
  • Simplify multi-agent mode controls (#29324)
    ## Why
    
    Multi-agent delegation policy was split across `multiAgentMode`,
    `features.multi_agent_mode`, and `usage_hint_enabled`. These controls
    could disagree: a requested mode could be downgraded by the feature
    flag, and disabling usage hints also disabled mode instructions.
    
    Some clients also need multi-agent tools without adding
    delegation-policy text to model context. The previous two-mode API could
    not express that directly.
    
    ## What changed
    
    `multiAgentMode` is now the only live delegation-policy control:
    
    | Mode | Behavior |
    | --- | --- |
    | `none` | Keep multi-agent tools available without adding mode
    instructions. |
    | `explicitRequestOnly` | Only delegate after an explicit user request.
    |
    | `proactive` | Delegate when parallel work materially improves speed or
    quality. |
    
    - new threads default to `explicitRequestOnly`; omitting the mode on
    later turns keeps the current value
    - thread start, resume, fork, and settings responses always report the
    concrete current mode instead of `null`
    - mode selection remains sticky across turns and resume
    - usage-hint text no longer controls whether mode instructions apply
    - `features.multi_agent_mode` and `usage_hint_enabled` remain accepted
    as ignored compatibility settings so existing configs continue to load
    - app-server documentation and generated schemas describe the three-mode
    API
    
    ## Tests
    
    - `just test -p codex-core multi_agent_mode`
    - `just test -p codex-core multi_agent_v2_config_from_feature_table`
    - `just test -p codex-core spawn_agent_description`
    - `just test -p codex-features`
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server multi_agent_mode`
  • Propagate safety buffering events to app-server clients (#29371)
    Responses API safety buffering metadata currently stops at the transport
    boundary, so app-server clients cannot render the in-progress safety
    review state.
    
    This change:
    - decodes and deduplicates `safety_buffering` metadata from Responses
    API SSE and WebSocket events without suppressing the original response
    event
    - emits a typed core event containing the requested model plus backend
    use cases and reasons
    - forwards that event as `turn/safetyBuffering/updated` through
    app-server v2 and updates generated protocol schemas
    - keeps the side-channel event out of persisted rollouts and turn timing
    
    This supports the Codex Apps buffering UX and depends on the Responses
    API backend work in https://github.com/openai/openai/pull/1044569 and
    https://github.com/openai/openai/pull/1044571.
    
    Validation:
    - focused `codex-core` safety-buffering integration test passes
    - `cargo check -p codex-core -p codex-app-server -p
    codex-app-server-protocol`
    - `just fix -p codex-api -p codex-protocol -p codex-core -p
    codex-app-server-protocol -p codex-app-server -p codex-rollout -p
    codex-rollout-trace -p codex-otel`
    - `just fmt`
    - broad package test run: 4,430/4,492 passed; 62 unrelated
    local-environment/concurrency failures involved unavailable test
    binaries, MCP subprocess setup, and app-server timeouts
  • Expose thread-level multi-agent mode (#28792)
    ## Why
    
    Once multi-agent mode can be selected per turn, clients also need to
    choose the initial selection when creating a thread and observe that
    selection through lifecycle and settings APIs.
    
    The selected value is intentionally distinct from the effective
    model-visible value: no client selection is represented as `null`, even
    though an eligible multi-agent v2 turn derives `explicitRequestOnly` as
    its effective default.
    
    ## What changed
    
    - Add the optional experimental `thread/start.multiAgentMode` parameter
    and pass it through thread creation.
    - Preserve an omitted initial value as an unset selection rather than
    eagerly storing `explicitRequestOnly`.
    - Apply an explicit `thread/start` selection to the first turn through
    the session configuration established at thread creation.
    - Restore the latest persisted effective mode as the selected baseline
    on cold resume when rollout history contains one.
    - Inherit the optional selected mode from a loaded parent when creating
    related runtime threads.
    - Return the current selected `multiAgentMode` from `thread/start`,
    `thread/resume`, `thread/fork`, and thread settings, using `null` when
    no mode is selected.
    - Keep lifecycle reporting independent from model capability and feature
    eligibility; core turn construction remains responsible for calculating
    and persisting the effective mode.
    
    ## Not covered
    
    - Clearing an existing loaded-session selection back to unset through
    `turn/start`; omitted or `null` currently retains the session's
    selection.
    - A TUI control, slash command, or `config.toml` preference.
    
    ## Verification
    
    - `CARGO_INCREMENTAL=0 just test -p codex-app-server-protocol`
    - `CARGO_INCREMENTAL=0 just test -p codex-app-server multi_agent_mode`
    
    The focused app-server coverage verifies explicit `thread/start`
    initialization, first-turn prompting, nullable reporting for an omitted
    selection, and retention of selections that are not currently
    runtime-eligible.
    
    ## Stack
    
    Stacked on #28685. This PR contains only the thread initialization and
    lifecycle/settings API layer.
  • Add per-turn multi-agent mode (#28685)
    ## Why
    
    Multi-agent v2 currently carries an explicit-request-only delegation
    rule in its static usage hint. That provides a safe default, but it
    prevents clients from selecting proactive delegation per turn without
    changing static guidance or rewriting prior model context.
    
    This change makes delegation mode a session selection that can be
    updated through `turn/start`, while deriving the effective model-visible
    mode separately for each turn. Eligible multi-agent v2 turns remain
    explicit-request-only unless proactive mode is both selected and
    enabled.
    
    ## What changed
    
    - Add the experimental `turn/start.multiAgentMode` parameter with
    `explicitRequestOnly` and `proactive` values. Omission retains the
    loaded session's current optional selection.
    - Add the default-off `features.multi_agent_mode` feature gate. Eligible
    multi-agent v2 turns use the selected mode when enabled; an unset
    selection or disabled gate resolves to `explicitRequestOnly`.
    - Treat mode prompting as inapplicable for multi-agent v1 and other
    unsupported session configurations, producing no multi-agent mode
    developer message rather than rejecting the turn.
    - Move the explicit-request-only rule out of the static v2 usage hint
    and into a bounded, tagged developer context fragment.
    - Emit the effective mode in initial context and only when that
    effective mode changes on later turns.
    - Persist the effective mode in `TurnContextItem` as the durable
    baseline for resume and context-update comparisons.
    
    Historical rollout items are not rewritten. Later mode developer
    messages establish the current rule incrementally.
    
    ## Not covered
    
    - Initial selection through `thread/start` and selected-mode reporting
    from thread lifecycle/settings APIs; those are isolated in the stacked
    #28792.
    - A TUI control or slash command for selecting the mode.
    - Persisting a preferred mode to `config.toml`; selection remains
    session/turn scoped.
    - Changes to multi-agent concurrency limits, tool availability, or model
    catalog capability declarations.
    - Rewriting historical rollout prompt items. Cold resume restores the
    latest persisted effective mode when available while leaving historical
    developer messages intact.
    
    ## Verification
    
    - `CARGO_INCREMENTAL=0 just test -p codex-core multi_agent_mode`
    - Focused app-server coverage verifies that `turn/start.multiAgentMode`
    produces proactive developer instructions for an eligible v2 turn.
    
    ## Stack
    
    Followed by #28792, which adds `thread/start` initialization and
    lifecycle/settings observability.
  • [3/3] app-server: configure environment connection timeout (#29025)
    ## Why
    
    Remote environments registered through `environment/add` currently use
    the fixed 10-second WebSocket connection timeout. Slow-starting
    executors need a caller-selected connection window, but this should not
    add retry policy or couple exec-server behavior to Core’s
    `deferred_executor` feature.
    
    Make the timeout an optional part of the existing experimental request.
    Existing clients continue using the current default, while callers that
    know an executor may take longer can request a larger window explicitly.
    
    Depends on #28683.
    
    ## What changed
    
    - Add optional `connectTimeoutMs` to `EnvironmentAddParams` and document
    it in the app-server README.
    - Pass the optional timeout through `EnvironmentRequestProcessor` into
    one `EnvironmentManager::upsert_environment()` path; the manager applies
    the existing default when it is omitted.
    - Preserve the existing single-attempt lifecycle. The configured value
    controls WebSocket connection and handshake time for both initial
    connection and later reconnects; initialization retains its separate
    timeout.
    - Add an app-server integration test that sends the real JSON-RPC
    request and verifies a stalled handshake observes the requested timeout.
    
    ## Test plan
    
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-exec-server`
    - `just test -p codex-app-server
    environment_add_applies_connect_timeout`
    
    ## Rollout
    
    This is additive and does not enable `deferred_executor`. Callers should
    send a non-default timeout only after a compatible app-server is
    deployed; omitted or `null` values retain the existing 10-second
    default.
  • Always use AVAS for realtime WebRTC calls (#28856)
    ## Summary
    
    - Remove the realtime `architecture` selector from core protocol,
    app-server protocol, config parsing, generated schemas, and callers.
    - Always create WebRTC realtime calls with the AVAS query params:
    `intent=quicksilver&architecture=avas`.
    - Keep direct websocket realtime behavior on the existing config/default
    path, while WebRTC starts without an explicit version now default to
    realtime v1 because AVAS requires v1.
    
    ## Notes
    
    - WebRTC realtime now means AVAS. If a caller explicitly asks to start
    WebRTC with realtime v2, Codex rejects that request because the AVAS
    WebRTC path only supports realtime v1. Websocket realtime is separate
    and can still use realtime v2.
    - The old `[realtime] architecture = "realtimeapi" | "avas"` config knob
    is removed. Local configs that still set it will need to delete that
    line.
    - Some app-server tests that were only trying to exercise realtime v2
    protocol behavior now use websocket transport, because WebRTC is
    intentionally locked to AVAS/v1. Separate WebRTC tests cover the AVAS
    query params, v1 startup, SDP flow, and sideband join.
    
    ## Validation
    
    - Merged fresh `origin/main` at `83e6a786a2`.
    - `just fmt`
    - `just write-config-schema`
    - `just write-app-server-schema`
    - `git diff --check`
    - `just test -p codex-api -p codex-core -p codex-app-server-protocol -p
    codex-app-server realtime` (176 passed)
    - `just test -p codex-protocol -p codex-config` (413 passed)
  • core: load AGENTS.md from foreign environments (#28958)
    ## Why
    
    Make it possible to load AGENTS.md from remote exec-servers whose OS is
    different than app-server.
    
    ## What
    
    - keep `AGENTS.md` discovery and provenance as `PathUri`, with
    root-aware parent and ancestor traversal
    - expose lifecycle instruction sources as legacy app-server path strings
    in events while retaining `PathUri` internally
    - preserve and test mixed POSIX and Windows paths in model context and
    TUI status output
    - cover remote Windows loading end to end by seeding the Wine prefix
    through host filesystem APIs
    - fix bug in `PathUri`'s parent() implementation that would erase
    Windows drive letters
  • Emit Trusted MCP App Identity on Tool-Call Items (#27132)
    ## Summary
    
    - Add optional `appContext` to app-server MCP tool-call items with
    trusted `connectorId`, `linkId`, and `mcpAppResourceUri` metadata.
    - Preserve that context across tool-call events, persisted history,
    reconnects, and thread resume.
    - Keep the deprecated top-level `mcpAppResourceUri` temporarily for
    client migration.
    
    The consumer contract is `{ appContext: { connectorId, linkId,
    mcpAppResourceUri }, tool }`.
    
    ## Validation
    
    - Full GitHub Actions suite passes, including CLA, Bazel tests, clippy,
    release builds, and argument-comment lint.
    
    ---------
    
    Co-authored-by: martinauyeung-oai <280153141+martinauyeung-oai@users.noreply.github.com>
  • Add app-server current-time impl (varlatency 3/n) (#28835)
    ## What
    
    Server should request:
    
    ```
    {
      "id": 42,
      "method": "currentTime/read",
      "params": {
        "threadId": "11111111-1111-1111-1111-aaaaafdc2c11"
      }
    }
    ```
    
    Client should respond with something like:
    
    ```rust
    {
      "id": 42,
      "result": {
        "currentTimeAt": 1781717655
      }
    }
    ```
    
    ## Why
    
    Sessions configured with `clock_source = "external"` need a
    thread-specific external time source before inference. The system clock
    remains the default production provider.
    
    ## Validation
    
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-app-server --test all
    current_time_read_round_trip_adds_reminder_to_model_input`
    - `cargo test -p codex-app-server
    first_attestation_capable_connection_for_thread_only_uses_thread_subscribers`
    - `cargo test -p codex-analytics`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-app-server`
    
    Stacked on #28824.
  • Support openai/form extended form elicitations (#27500)
    # Summary
    Allow App Server clients to opt into `openai/form` MCP elicitations.
  • [codex] Support assistant realtime append text (#28836)
    ## Why
    
    Frontend realtime voice continuity needs to replay a tiny
    previous-session overlap as actual conversation items, including
    assistant text. The app-server `thread/realtime/appendText` API already
    carries a role through to the Rust realtime websocket layer, but the
    shared role enum only accepted `user` and `developer`.
    
    ## What Changed
    
    - Added `assistant` to `ConversationTextRole` and regenerated the
    app-server schema/type fixtures.
    - Added `output_text` as a realtime conversation content type.
    - Updated realtime websocket item creation so assistant appendText emits
    `content: [{ type: "output_text", text }]`, while user and developer
    continue to emit `input_text`.
    - Updated app-server docs and tests to cover assistant appendText
    alongside the existing developer role behavior.
    
    ## Validation
    
    - `just write-app-server-schema`
    - `just fmt` (first sandboxed attempt failed because `uv` could not
    access `~/.cache/uv`; reran with filesystem access and passed)
    - `just test -p codex-api` passed: 126/126
    - `just test -p codex-app-server-protocol` passed: 239/239, including
    generated JSON/TypeScript fixture checks
    - `just test -p codex-app-server` was started locally but stopped per
    request after unrelated local sandbox/Seatbelt failures (`sandbox-exec:
    sandbox_apply: Operation not permitted`) and one missing local `codex`
    binary failure; CI should be faster and more authoritative for the full
    suite.
  • [codex] control automatic realtime handoff delivery (#27986)
    ## What
    
    Built on the realtime speech-control plumbing merged in #27917.
    
    - Add optional `codexResponseHandoffPrefix` to `thread/realtime/start`.
    - Apply that prefix only to automatic V1 commentary sent through
    `conversation.handoff.append`; final answers remain unprefixed.
    - Add opt-in `clientManagedHandoffs`. When true, core suppresses
    automatic response handoffs and completion output so delivery is
    controlled by explicit client append APIs.
    - Preserve existing automatic behavior by default.
    `codexResponsesAsItems: true` continues to select item routing when
    client-managed mode is disabled.
    
    ## Why
    
    Voice clients need two delivery policies: automatic background context
    with silent commentary instructions and fully client-owned handoffs.
    Phase-aware prefixing keeps routine commentary silent without
    suppressing the final answer, while client-managed mode lets an app
    decide exactly which updates to append.
    
    ## Validation
    
    - `just fmt`
    - `cargo test -p codex-app-server-protocol
    serialize_thread_realtime_start`
    - `RUST_MIN_STACK=16777216 cargo test -p codex-core --test all
    conversation_handoff_persists_across_item_done_until_turn_complete`
    - `RUST_MIN_STACK=16777216 cargo test -p codex-app-server --test all
    webrtc_v1_client_managed_handoffs_disable_automatic_output`
    - `RUST_MIN_STACK=16777216 cargo test -p codex-app-server --test all
    webrtc_v1_final_automatic_handoff_omits_silent_prefix`
    - `cargo build -p codex-cli --bin codex`
    - Local Codex Apps compatibility check: 43 focused webview tests passed,
    and a live voice session routed through the source-built app-server.
    
    The explicit `RUST_MIN_STACK` avoids a macOS Tokio test-worker stack
    overflow seen with the default test environment.
  • [codex] Track plugin install and import telemetry failures (#28731)
    ## Summary
    - Track plugin install failures through the unified
    `codex_plugin_install_failed` event for local installs, remote install
    preflight failures, bundle failures, and remote catalog/backend
    failures.
    - Send classified `error_type` values in plugin install failure
    analytics instead of raw error strings.
    - Stop sending raw external-agent import errors in analytics while
    preserving raw failure details in app-facing import
    notifications/history.
    - Keep raw plugin/migration diagnostics in `tracing::warn!` logs.
    - Keep remote failure plugin names as the existing local placeholder
    (`unknown`) and remove the extra telemetry plugin-name override.
    - Change `ExternalAgentConfigImportParams.source` from a generated enum
    to `string | null`, with legacy `claudeCode` / `claudeCowork` inputs
    normalized to existing analytics values.
    
    ## Testing
  • [codex] Restore thread recency with compatible migration history (#28671)
    ## Summary
    
    - Revert #28655, restoring the thread `recencyAt` behavior introduced by
    #27910.
    - Move `threads_recency_at` to migration 0039 so it no longer collides
    with `external_agent_config_imports` at version 0038.
    - Repair databases that already applied the recency migration as version
    38 by moving the matching migration-history row to version 39 before
    SQLx validation. The current version-38 migration can then apply
    normally.
    
    ## Validation
    
    - `just test -p codex-state
    migrations::tests::repairs_recency_migration_that_was_applied_as_version_38`
    - `just test -p codex-state -p codex-rollout -p codex-thread-store -p
    codex-app-server-protocol -p codex-tui`: 3,439 passed; six TUI tests
    could not open the machine's existing read-only incident database at
    `~/.codex/sqlite/state_5.sqlite`.
    - `just fix -p codex-state`
    - `just fmt`
    - Verified that state migration versions are unique.
  • Scope command approvals by execution environment (#28738)
    ## Why
    
    Command approval cache keys included the command and working directory,
    but not the execution environment. An approval for `/workspace` locally
    could therefore be reused for the same command and path on an executor.
    
    ## What changed
    
    - Include the selected environment ID in shell and unified-exec approval
    cache keys.
    - Carry that ID through the normal command approval request so clients
    can show which environment is being approved.
    - Expose the environment through app-server as a required nullable
    `environmentId` and show it in the inline TUI approval prompt.
    - Keep older recorded approval events compatible when the environment is
    absent.
    
    For example, `echo ok` in local `/workspace` and `echo ok` in executor
    `/workspace` now produce different approval keys and separate prompts.
    
    ## Scope
    
    This PR does not change network approvals, Guardian review actions, MCP
    elicitation, full-screen TUI rendering, or environment-ID validation.
    Remote `shell_command` execution itself remains in #28722; this PR only
    makes its approval key environment-aware.
  • [ez][codex-rs] Support apps._default.default_tools_approval_mode (#27965)
    [from codex]
    
    ## Summary
    
    - add `default_tools_approval_mode` to `[apps._default]` and expose it
    through app-server v2 `config/read`
    - apply it after managed, per-tool, and per-app approval settings,
    before the built-in `auto` fallback
    - document the precedence, regenerate config/app-server schemas, and add
    unit plus end-to-end approval coverage
    
    ## Configuration
    
    ```toml
    [apps._default]
    default_tools_approval_mode = "prompt"
    ```
    
    The effective precedence is managed requirements, tool-specific
    `approval_mode`, app-specific `default_tools_approval_mode`,
    `apps._default.default_tools_approval_mode`, then `auto`.
    
    ## Test plan
    
    - `just write-config-schema`
    - `just write-app-server-schema`
    - `just write-app-server-schema --experimental`
    - `just test -p codex-core app_tool_policy`
    - `just test -p codex-core mcp_turn_metadata`
    - `just test -p codex-config`
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server config_read_includes_apps`
    - `just fix -p codex-config -p codex-core -p codex-app-server-protocol
    -p codex-app-server`
    - `just fmt`
  • Revert thread recencyAt for sidebar ordering (#28655)
    ## Why
    
    Revert #27910 to remove the newly introduced thread `recencyAt`
    persistence and API behavior from `main`.
    
    ## What changed
    
    This reverts commit `fac3158c2a783095768076489815f361fa9b0db4`,
    including the state migration, thread-store propagation, app-server API
    surface, generated schemas, and related tests.
    
    ## Validation
    
    Not run before opening; relying on CI for the initial fast signal.
  • Add thread recencyAt for sidebar ordering (#27910)
    ## Summary
    
    Add a server-owned `recencyAt` timestamp and `recency_at` thread-list
    sort key for product recency ordering while preserving the existing
    meaning of `updatedAt` as the latest persisted thread mutation.
    
    This is the server-side alternative to #27697. Rather than narrowing
    `updatedAt`, clients can sort the sidebar by `recency_at` and continue
    treating `updatedAt` as mutation time.
    
    Paired Codex Apps PR:
    [openai/openai#1024599](https://github.com/openai/openai/pull/1024599)
    
    ## Contract
    
    - `recencyAt` initializes when a thread is created.
    - A turn start advances `recencyAt` monotonically.
    - Commentary, agent output, tool results, token/accounting updates, turn
    completion, archive, unarchive, resume, and generic metadata writes do
    not advance it.
    - `updatedAt` retains its existing behavior and continues to advance for
    persisted thread mutations.
    - Current servers populate `recencyAt`; the response field is optional
    in generated TypeScript so clients connected to older servers can fall
    back to `updatedAt`.
    - Filesystem-only fallback uses existing updated/mtime ordering when
    SQLite is unavailable.
    
    ## Persistence and compatibility
    
    Migration 0038 adds second- and millisecond-precision recency columns,
    backfills them from the existing updated timestamp, creates list
    indexes, and includes an insert trigger so older binaries writing to a
    migrated database seed recency without causing later mutations to
    advance it.
    
    Generic metadata upserts preserve existing recency values. Turn-start
    updates use a dedicated monotonic touch, and process-local allocation
    keeps millisecond cursor values unique. State DB list, search, read,
    filtered-list repair, rollout fallback propagation, and app-server
    conversions all carry the new field.
    
    ## API
    
    `Thread` responses include:
    
    ```ts
    recencyAt?: number
    ```
    
    `thread/list` and `thread/search` accept:
    
    ```json
    { "sortKey": "recency_at" }
    ```
    
    Generated TypeScript and JSON schemas are included.
    
    ## Validation
    
    - `just test -p codex-state` — 146 passed
    - `just test -p codex-rollout` — 69 passed
    - `just test -p codex-thread-store` — 81 passed
    - `just test -p codex-app-server-protocol` — 231 passed
    - Focused app-server list ordering, response mapping, archive/unarchive,
    and resume lifecycle tests passed
    - Scoped `just fix` for state, rollout, thread-store,
    app-server-protocol, and app-server
    - `just fmt`
    - `git diff --check`
    - Independent correctness, simplicity, elegance, security, and
    test-quality reviews; actionable ordering, lifecycle, query-projection,
    and timestamp-uniqueness findings were addressed
  • [codex] expose Bedrock credential source in account/read (#27751)
    ## Why
    
    `account/read` currently reports only `type: "amazonBedrock"`, so
    clients cannot distinguish a Codex-managed Bedrock API key from
    credentials supplied by AWS. The app UI needs that distinction to render
    the appropriate account state without duplicating provider-auth logic.
    
    Credential-source selection belongs to the Bedrock model provider
    because it already owns the precedence between managed Bedrock auth and
    the external AWS credential path. This builds on #27443 and #27689.
    
    ## What changed
    
    - Added `AmazonBedrockCredentialSource` with `codexManaged` and
    `awsManaged` values.
    - Included the selected credential source in
    `ProviderAccount::AmazonBedrock` and the app-server `Account` response.
    - Made `AmazonBedrockModelProvider::account_state()` classify the source
    from its managed-auth state.
    - Regenerated the app-server JSON and TypeScript schemas.
    - Updated app-server account documentation and downstream TUI matches.
    
    `codexManaged` means the provider found a managed Bedrock API key.
    `awsManaged` identifies the provider's external AWS credential path; it
    does not assert that the AWS credential chain has been validated.
    
    ## Testing
    
    - Added model-provider coverage for Codex-managed precedence and
    AWS-managed fallback.
    - Added app-server protocol serialization coverage for both wire values.
    - Added app-server integration coverage for both `account/read`
    responses.
    - `just test -p codex-protocol -p codex-model-provider -p
    codex-app-server-protocol` (497 tests passed).
    
    After rebasing onto #27711, the `codex-app-server` test target compiled
    past the image-generation `PathUri` migration. Local linking was then
    interrupted by disk exhaustion (`No space left on device`).
  • [codex] Add interruptible sleep tool (#28429)
    ## Why
    
    Models sometimes need to pause briefly while waiting for external work,
    but using a shell command for that delay ties the wait to a process and
    does not naturally resume when new turn input arrives.
    
    ## What changed
    
    - add a built-in `sleep` tool behind the under-development `sleep_tool`
    feature
    - accept a bounded `duration_ms` argument, matching the millisecond
    convention used by unified exec
    - end the sleep early when either steered user input or mailbox input
    arrives
    - include elapsed wall-clock time in completed and interrupted outputs
    - emit a dedicated core `SleepItem` through `item/started` and
    `item/completed`
    - expose the sleep item as app-server v2 `ThreadItem::Sleep` and retain
    it in reconstructed thread history
    - regenerate the configuration schema for the new feature flag
    - regenerate app-server JSON and TypeScript schema fixtures
    
    ## Test plan
    
    - `just test -p codex-core sleep_tool_follows_feature_gate`
    - `just test -p codex-core any_new_input_interrupts_sleep`
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server
    sleep_emits_started_and_completed_items`
  • Add a toggle for realtime startup context (#28405)
    ## Summary
    - Add `includeStartupContext` to realtime start requests so callers can
    explicitly skip Codex startup context while keeping the backend prompt
    - Thread the new flag through protocol types, request processing, and
    realtime session config
    - Update app-server docs and coverage for the new default and opt-out
    behavior
    
    ## Testing
    - Added protocol serialization coverage for `includeStartupContext`
    - Added realtime integration coverage for starting a session with
    startup context disabled
  • Add realtime speech append control (#27917)
    ## Why
    
    Realtime voice harness tuning needs app-side control over what backend
    Codex text is spoken. Backend orchestrator text is written for a reading
    UI, so automatically speaking every preamble, progress update, or final
    assistant message can make the realtime voice model too chatty.
    
    For experimentation, clients need two simple controls: keep app/client
    text-item injection on the existing item-create path, and add an
    explicit speakable path that app code can call only when it wants
    realtime to speak. Automatic Codex output also needs an opt-in way to
    switch from the protocol's default speakable path to regular realtime
    items, with a caller-provided prefix so prompt wording can be tuned
    outside core.
    
    The default remains unchanged: if a client omits the new start fields
    and never calls `appendSpeech`, automatic backend output continues down
    the existing speakable path for the selected realtime protocol.
    
    ## What Changed
    
    - Adds experimental `thread/realtime/appendSpeech` for app-provided
    speakable text.
    - Keeps existing `thread/realtime/appendText` as the item-create API for
    app-provided realtime text items.
    - Adds `codexResponsesAsItems` / `codex_responses_as_items` on
    `thread/realtime/start` to send automatic Codex responses with
    `conversation.item.create` instead of the protocol's default speakable
    output path.
    - Adds `codexResponseItemPrefix` / `codex_response_item_prefix` so
    clients can prepend experiment instructions to those automatic Codex
    response items.
    - Keeps literal `conversation.handoff.append` routing scoped to the v1
    speakable path; v2 default speech uses its item/function-output plus
    `response.create` behavior.
    - Removes the earlier public silent-context API and hardcoded
    silent-context prefix.
    - Updates realtime tests to cover default automatic speakable behavior,
    opt-in automatic item-create behavior, and explicit `appendSpeech`
    behavior.
    
    ## Validation
    
    - `cargo check -p codex-core -p codex-app-server -p codex-api`
    - `just test -p codex-app-server realtime_conversation`
    - `just test -p codex-core realtime_conversation` (50/51 passed in the
    filtered parallel run; the lone failure passed when rerun in isolation)
    - `just test -p codex-core
    conversation_mirrors_assistant_message_text_to_realtime_handoff`
    - `just test -p codex-api
    e2e_connect_and_exchange_events_against_mock_ws_server`
    - `just fix -p codex-core`
    - `just fix -p codex-app-server`
    - `cargo build -p codex-cli`
  • [codex] Add created-by-me remote plugin marketplace (#28203)
    ## Summary
    - add the `created-by-me-remote` marketplace backed by paginated
    `scope=USER` plugin directory and installed-plugin requests
    - include USER plugins in installed-plugin caching, bundle sync, and
    stale-cache cleanup without client-side discoverability filtering
    - expose the marketplace through app-server v2 and regenerate the
    protocol schemas
    
    ## Testing
    - `cargo build -p codex-app-server --bin codex-app-server`
    - production-auth `plugin/list` smoke test for `created-by-me-remote`
    (returned the expected USER plugin as installed and enabled)
    - `just test -p codex-core-plugins` (221 passed)
    - `just test -p codex-app-server-protocol` (231 passed)
    - `just test -p codex-app-server suite::v2::plugin_list::` (37 passed)
    - `just fix -p codex-core-plugins -p codex-app-server-protocol -p
    codex-app-server`
    - `just fmt`
  • feat(app-server): expose rate-limit reset credits (#28143)
    ## Why
    
    Codex users can earn personal rate-limit reset credits, but app-server
    clients do not currently have an API for reading or redeeming them. This
    adds the backend and protocol foundation used by the `/usage` TUI flow
    in #28154.
    
    ## What changed
    
    - Extend `account/rateLimits/read` with a nullable
    `rateLimitResetCredits` summary sourced from the existing usage
    response.
    - Add backend-client and app-server support for consuming a reset with a
    caller-generated idempotency key. A UUID is recommended, and clients
    reuse the same key when retrying the same logical reset.
    - Return only the consume `outcome`; clients refetch
    `account/rateLimits/read` for updated window state.
    - Document the response field and each consume outcome, and regenerate
    the JSON and TypeScript schema fixtures.
    - Clarify in `AGENTS.md` that new app-server string enum values use
    camelCase on the wire.
    - Update the existing TUI response fixture for the expanded protocol
    shape.
    - Add coverage for authentication, response mapping, backend failures,
    consume outcomes, and request timeout behavior.
    
    ## Validation
    
    - `just test -p codex-app-server-protocol` — 231 passed.
    - `just test -p codex-backend-client` — 14 passed.
    - Focused `codex-app-server` reset-credit tests — 5 passed.
    - Focused `codex-tui` protocol response fixture test — passed.
    - `just fix -p codex-backend-client -p codex-app-server-protocol -p
    codex-app-server` — passed.
    - `just fmt` — passed.
  • [codex] Add external agent import result accounting (#28008)
    ## Why
    
    External-agent imports can complete synchronously or continue in the
    background for plugins/sessions. Clients need a stable import id to
    correlate the immediate response with the eventual completion
    notification, and the completion payload needs enough accounting to show
    which artifact types succeeded or failed without hiding partial
    failures.
    
    ## What Changed
    
    - `externalAgentConfig/import` now returns an `importId`;
    `externalAgentConfig/import/completed` includes the same `importId` plus
    type-level `itemResults`.
    - Completed `itemResults` report `successCount`, `errorCount`,
    `successes`, and `rawErrors` for each migrated item type.
    - Added protocol/schema/TypeScript types for import successes, raw
    errors, and type-level results. No progress notification is included in
    the final PR.
    - `ExternalAgentConfigService::import` now returns an outcome object
    with synchronous item results and pending plugin imports.
    - Plugin import outcomes track succeeded/failed marketplaces, plugin
    ids, and raw errors. Plugin failures can be reported in completed
    accounting while later migration items continue.
    - Non-plugin synchronous import failures still fail the request, so
    invalid config/skills-style failures are not reported as a successful
    import response.
    - Session imports now return item results. Successful imports include
    the source session path and imported thread id; prepare, persist,
    ledger, and source-validation failures become raw errors in completion
    accounting where the import can continue.
    - The request processor generates the `importId`, aggregates synchronous
    results with background plugin/session results, and sends a single
    completed notification when all selected work is done.
    - App-server docs and generated schema fixtures were updated for the new
    response/completed payload shapes.
    
    ## Validation
    
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server-client event_requires_delivery`
    - `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-review-sync-error
    just test -p codex-app-server
    external_agent_config_import_returns_error_for_failed_sync_import`
    - `CODEX_SQLITE_HOME=/private/tmp/codex-app-server-review-external-agent
    just test -p codex-app-server external_agent_config`
    
    Note: local sandbox validation used `CODEX_SQLITE_HOME` because the
    default sqlite state path is read-only in this environment.
  • Expose explicit dynamic tool namespaces in thread start (#27371)
    Stacked on #27365.
    
    ## Stack note
    
    [#27365](https://github.com/openai/codex/pull/27365) kept `thread/start`
    unchanged and converted its input in `thread_processor`. This PR updates
    `thread/start` to accept explicit functions and namespaces directly.
    
    Legacy per-tool arrays are still accepted and converted while reading
    the request. As a result, `thread_processor` can validate and pass the
    tools through directly, which is why some code added in #27365 is
    removed here.
    
    ## Why
    
    `thread/start.dynamicTools` still repeats namespace data on each
    function even though core now stores explicit namespace groups. The
    request API should use the same shape so each namespace has one
    description and one member list.
    
    ## What changed
    
    - Accept top-level functions and explicit namespace objects in
    `dynamicTools`.
    - Continue accepting fully legacy flat arrays, including
    `exposeToContext`.
    - Reject arrays that mix legacy and canonical entries.
    - Reuse the protocol types directly and remove the temporary app-server
    adapter.
    - Update validation, docs, the test client, and generated schemas.
    
    ## Test plan
    
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server
    dynamic_tool_call_round_trip_sends_text_content_items_to_model`
    - `just test -p codex-app-server
    thread_start_normalizes_legacy_dynamic_tools_into_model_request`
    - `just test -p codex-app-server
    thread_start_rejects_mixed_dynamic_tool_formats`
    - `just test -p codex-app-server
    thread_start_rejects_hidden_dynamic_tools_without_namespace`
  • Activate selected executor plugin MCPs in app-server (#27893)
    ## Why
    
    #27870 teaches the MCP extension how to discover stdio MCP servers
    declared by a selected executor plugin, but app-server does not yet
    install that contributor or initialize its per-thread state. As a
    result, `thread/start.selectedCapabilityRoots` can select the plugin
    while its MCP servers remain inactive.
    
    This PR closes that app-server wiring gap:
    
    ```text
    thread/start(selectedCapabilityRoots)
        -> initialize the thread's selected-plugin MCP snapshot
        -> read the selected plugin's .mcp.json through its environment
        -> start declared stdio servers in that environment
        -> expose their tools only on the selected thread
    ```
    
    ## What changed
    
    - Install the selected-executor-plugin MCP contributor in app-server
    using the existing shared `EnvironmentManager`.
    - Initialize its frozen thread snapshot when `thread/start` includes
    selected capability roots.
    - Document that selected plugin stdio MCPs are activated in their owning
    environment.
    - Add an app-server E2E covering the complete selection-to-tool-call
    path.
    
    The E2E verifies that:
    
    - the selected MCP process receives an executor-only environment value,
    proving the tool runs through the selected environment;
    - the MCP tool is advertised to the model and can be called;
    - a normal MCP config reload does not discard the thread's frozen
    selected-plugin registration;
    - another thread without the selected root does not see the MCP server.
    
    ## Scope
    
    - Existing sessions without `selectedCapabilityRoots` are unchanged.
    - Only stdio MCP declarations are activated. HTTP declarations remain
    inactive.
    - This does not change selected-root persistence across resume/fork or
    add hosted-plugin behavior.
    
    ## Verification
    
    - Focused app-server E2E:
    `selected_executor_plugin_exposes_its_stdio_mcp_only_to_that_thread`
    
    ## Stack
    
    Stacked on #27870.
  • feat(app-server): filter threads by parent (#26662)
    ## Why
    
    Clients that display or coordinate spawned subagents need an
    authoritative snapshot of a thread's immediate spawned children when
    they connect to app-server or recover after missing live events.
    `thread/list` cannot query by parent, so clients must otherwise scan
    unrelated threads or reconstruct relationships from rollout history and
    transient events.
    
    The direct spawn relationship already exists in persisted
    `thread_spawn_edges` state. Review and Guardian threads do not
    participate in that lifecycle and are intentionally outside this
    filter's scope.
    
    ## What changed
    
    This adds an experimental `parentThreadId` filter to `thread/list`.
    Parent-filtered requests return direct spawned children from persisted
    state while preserving the existing response shape, explicit filters,
    sorting, and timestamp-only cursor behavior. The lookup does not read
    rollout transcripts or recursively return descendants.
    
    Supersedes #25112 with the narrower `thread/list` filter approach.
    
    ## How it works
    
    1. An experimental client passes a valid thread ID as `parentThreadId`.
    2. App-server routes the list through the existing thread-store and
    state-database boundaries.
    3. SQLite selects threads whose IDs have a direct persisted spawn edge
    from that parent.
    4. Omitted provider and source filters include all values; explicit
    filters keep ordinary `thread/list` semantics.
    5. Grandchildren, Review threads, and Guardian threads are excluded.
    
    ## Verification
    
    State (144 tests), rollout (69 tests), and focused app-server
    thread-list (31 tests) suites passed. Scoped Clippy checks and
    repository formatting also passed. Coverage includes direct spawned
    children, omitted grandchildren, pagination, malformed IDs, mixed source
    kinds, explicit filters, and operation without rollout files.
  • feat(app-server): enforce managed remote control disable (#27961)
    ## Why
    
    Managed deployments need a reliable deny gate for remote control.
    Persisted enablement and explicit startup requests currently remain able
    to start the transport, while the removed `features.remote_control` key
    is intentionally only a compatibility no-op.
    
    This adds a dedicated requirement that administrators can use to force
    remote control off without deleting the user's persisted preference.
    Removing the requirement and restarting restores the prior choice.
    
    ## What Changed
    
    - Added top-level `allow_remote_control` requirements parsing, sourced
    layer precedence, debug output, and `configRequirements/read` exposure
    as `allowRemoteControl`.
    - Added a typed transport policy captured from the startup requirements
    snapshot. Managed disable forces the initial state to disabled and
    prevents enrollment, refresh, connection, and persisted-preference
    mutation.
    - Rejected every `remoteControl/*` RPC before parameter deserialization
    with JSON-RPC `-32600` and `remote control is disabled by managed
    requirements`.
    - Preserved the existing disabled status notification and the previous
    behavior when the requirement is `true` or omitted.
    - Regenerated app-server protocol schemas and documented the new
    requirement.
    
    ## Verification
    
    - Confirmed all remote-control RPCs, including a malformed request,
    return the managed-policy error while the initial status notification
    remains `disabled`.
    - Confirmed explicit ephemeral startup and persisted enablement make no
    backend connection and leave the SQLite preference unchanged.
    - Confirmed `allow_remote_control = true` does not enable or block
    remote control and `configRequirements/read` returns
    `allowRemoteControl: false` for the deny policy.
    
    Related issue: N/A (managed-policy hardening).
  • [codex] add roles to realtime append text (#27936)
    ## Summary
    
    Add an explicit `user` or `developer` role to
    `thread/realtime/appendText` and propagate it through the realtime input
    queue into `conversation.item.create`. Older JSON clients that omit the
    field continue to default to `user`.
    
    This lets app-provided context such as memory retain developer authority
    without bypassing app-server through a renderer-owned data channel. The
    app-server schemas, API documentation, and focused protocol and
    websocket coverage are updated with the new contract.
    
    The Codex Apps consumer is tracked in
    [openai/openai#1025261](https://github.com/openai/openai/pull/1025261).
  • [codex] expose remote plugin share URL (#27890)
    ## Summary
    
    - expose the remote plugin detail endpoint's `share_url` as nullable
    `PluginDetail.shareUrl`
    - preserve existing `PluginSummary.shareContext` behavior for local and
    workspace sharing flows
    - regenerate the app-server TypeScript and JSON schema fixtures
    
    ## Why
    
    The remote plugin detail response already includes a canonical
    `share_url`, but that value was not surfaced by `plugin/read` for global
    plugins. Global plugins intentionally have no `shareContext`, so using
    that model for the URL would change the semantics consumed by the
    existing share modal.
    
    ## User impact
    
    Codex clients can use `PluginDetail.shareUrl` for a remote plugin's
    copy-link action, including when the plugin is disabled by an
    administrator, without changing existing share-modal or ownership
    behavior.
    
    ## Validation
    
    - `cargo test -p codex-app-server
    plugin_read_includes_share_url_for_admin_disabled_remote_plugin`
    - `cargo test -p codex-app-server-protocol
    typescript_schema_fixtures_match_generated`
    - `cargo test -p codex-app-server-protocol
    json_schema_fixtures_match_generated`
    - `cargo fmt --all`
  • [ez][codex-rs] Support approvals reviewer in app defaults (#27075)
    [from codex]
    
    ## Summary
    
    - add `approvals_reviewer` support to `[apps._default]`
    - resolve connected-app reviewers in per-app, app-default, then global
    order
    - expose the setting through the v2 config API and regenerate schema
    fixtures
    
    ## Context
    
    PR #25167 added `apps.<connector_id>.approvals_reviewer`, but the shared
    app defaults table could not specify the reviewer. This extends the same
    behavior to `[apps._default]` while preserving per-app overrides.
    
    Managed `allowed_approvals_reviewers` requirements still constrain both
    default and per-app values. A disallowed app value falls back to the
    global reviewer, and non-app MCP servers continue using the global
    reviewer.
    
    ## Testing
    
    - `just write-config-schema`
    - `just write-app-server-schema`
    - `just fmt`
    - `just test -p codex-config`
    - `just test -p codex-core app_approvals_reviewer`
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server config_read_includes_apps`
  • feat(app-server): persist remote-control desired state (#27445)
    ## Why
    
    Remote-control runtime enablement and persisted enrollment preference
    were represented by separate flags. That made startup rehydration, RPC
    persistence, and new-enrollment seeding race with one another, and it
    did not cleanly distinguish runtime-only CLI or daemon starts from
    durable app-server RPC changes.
    
    ## What Changed
    
    - Replace the parallel enablement, seed, and rehydration flags with one
    transport-owned `RemoteControlDesiredState`.
    - Add nullable enrollment-scoped persistence and preserve existing
    preferences during enrollment upserts.
    - Rehydrate plain startup only after auth and client scope resolve,
    without overwriting a concurrent RPC transition.
    - Make ordinary `remoteControl/enable` and `remoteControl/disable`
    durable while retaining `ephemeral: true` for runtime-only callers.
    - Have the daemon explicitly request ephemeral enablement and regenerate
    the app-server schemas.
    
    ## Verification
    
    - Covered migration and `NULL`/`0`/`1` persistence round trips.
    - Covered plain-start rehydration and runtime-only versus durable
    enrollment seeding.
    - Covered durable enable, durable disable, and ephemeral enable through
    app-server RPC.
    - Covered the daemon's exact `{ "ephemeral": true }` request payload.
    
    Related issue: N/A (internal remote-control persistence architecture
    change).
  • [codex] Remove redundant plugin app auth state (#27465)
    ## Summary
    
    - remove the redundant `needsAuth` field from `AppSummary` and generated
    app-server schemas
    - stop `plugin/read` from querying Apps MCP solely to hydrate unused
    connector auth state
    - preserve `plugin/install.appsNeedingAuth` membership and
    `app/list.isAccessible` as the authentication signals
    
    ## Why
    
    Codex App and TUI do not consume `plugin/read.plugin.apps[].needsAuth`.
    Hydrating it could establish an Apps MCP connection and discover tools
    on a cold `plugin/read` request, adding avoidable latency. The plugin
    APIs are still marked under development, so removing this wire field is
    preferable to retaining a misleading default.
    
    ## Verification
    
    - `just write-app-server-schema`
    - `just fmt`
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server
    plugin_install_uses_remote_apps_needing_auth_response`
    - `just test -p codex-app-server
    plugin_install_returns_apps_needing_auth`
    - `just test -p codex-app-server
    plugin_read_returns_plugin_details_with_bundle_contents`
    - `just test -p codex-tui
    plugin_detail_popup_snapshot_shows_install_actions_and_capability_summaries`
    - `$xin-build` simplify and debug reviews
  • Add app-server thread/delete API (#25018)
    ## Why
    
    Clients can archive and unarchive threads today, but there is no
    app-server API for permanently removing a thread. Deletion also needs to
    cover the full session tree: deleting a main thread should remove
    spawned subagent threads and the related local metadata instead of
    leaving orphaned rollout files, goals, or subagent state behind.
    
    ## What
    
    - Adds the v2 `thread/delete` request and `thread/deleted` notification,
    with the response shape kept consistent with `thread/archive`.
    - Implements local hard delete for active and archived rollout files.
    - Deletes the requested thread's state DB row as the commit point, then
    best-effort cleans associated state including spawned descendants,
    goals, spawn edges, logs, dynamic tools, and agent job assignments.
    - Updates app-server API docs and generated protocol schema/TypeScript
    fixtures.
  • Add app-server background terminal process APIs (#26041)
    ## Summary
    
    Codex Apps needs app-server as the source of truth for chat-started
    background terminals instead of guessing from local process trees.
    
    This PR adds experimental v2 APIs to list and terminate background
    terminals for a loaded thread using app-server process ids, so clients
    can manage background terminals without local PID discovery.
    
    ## Changes
    
    - `thread/backgroundTerminals/list` returns paginated background
    terminal records with `itemId`, app-server `processId`, `command`,
    `cwd`, nullable `osPid`, nullable `cpuPercent`, and nullable `rssKb`.
    - `thread/backgroundTerminals/terminate` terminates one running
    background terminal by app-server `processId` and returns whether a
    process was terminated.
    - Background terminal list and terminate operations use unified-exec
    process manager state as their source of truth.
  • Add per-session realtime model and version overrides (#24999)
    ## Why
    
    Clients need to select a realtime session configuration for an
    individual start without rewriting persisted configuration or restarting
    the app-server process.
    
    ## What Changed
    
    - Add optional `model` and `version` fields to `thread/realtime/start`
    - Forward those optional values through the realtime start operation and
    apply them only for that session
    - Preserve existing configured/default behavior when the new fields are
    omitted
    - Update generated protocol schema and app-server documentation
    
    ## Validation
    
    - Added/updated protocol serialization coverage for the new optional
    request fields
    - Added focused core coverage for a session override taking precedence
    over configured realtime selection
    - Added focused app-server coverage that a request override reaches the
    realtime WebSocket handshake
  • Load selected executor skills through extensions (#27184)
    ## Why
    
    CCA is moving toward a split runtime where the orchestrator may not have
    a filesystem, while executors can expose preinstalled plugins and
    skills. A thread therefore needs to select capabilities without asking
    app-server or core to interpret executor-owned paths through the
    orchestrator's filesystem.
    
    The longer-term model is broader than executor skills:
    
    - A plugin is a bundle of skills, MCP servers, connectors/apps, and
    hooks.
    - A plugin root can be local, executor-owned, or hosted by a backend.
    - Components inside one plugin can use different access and execution
    mechanisms. A skill may be read from a filesystem or through backend
    tools; an HTTP MCP server can run without an executor; a stdio MCP
    server or hook needs an execution environment.
    - Core should carry generic extension initialization data. The extension
    that owns a component should discover it, expose it to the model, and
    invoke it through the appropriate runtime.
    
    This PR establishes that architecture through one complete vertical:
    selecting a root on an executor, discovering the skills beneath it,
    exposing those skills to the model, and reading an explicitly invoked
    `SKILL.md` through the same executor.
    
    ## Contract
    
    `thread/start` gains an experimental `selectedCapabilityRoots` field:
    
    ```json
    {
      "selectedCapabilityRoots": [
        {
          "id": "deploy-plugin@1",
          "location": {
            "type": "environment",
            "environmentId": "workspace",
            "path": "/opt/codex/plugins/deploy"
          }
        }
      ]
    }
    ```
    
    The root is intentionally not classified as a "plugin" or "skill" in the
    API. It can point at a standalone skill, a directory containing several
    skills, or a plugin containing skills and other components. This PR only
    teaches the skills extension how to consume it; later extensions can
    resolve MCP, connector, and hook components from the same selection.
    
    The platform-supplied `id` is stable selection identity. The location
    says which runtime owns the root and gives that runtime an opaque path.
    App-server does not inspect or canonicalize the path.
    
    ## What changed
    
    ### Generic thread extension initialization
    
    App-server converts selected roots into `ExtensionDataInit`. Core
    carries that generic initialization value until the final thread ID is
    known, then creates thread-scoped `ExtensionData` before lifecycle
    contributors run.
    
    This keeps `Session` and core independent of the capability-selection
    contract. The initialization value is consumed during construction; it
    is not retained as another long-lived `Session` field.
    
    ### Executor-backed skills
    
    The skills extension now owns an `ExecutorSkillProvider` that:
    
    - resolves the selected environment through `EnvironmentManager`
    - discovers, canonicalizes, and reads skills through that environment's
    `ExecutorFileSystem`
    - contributes the bounded selected-skill catalog as stable developer
    context
    - reads an explicitly invoked skill body through the authority that
    listed it
    - warns when an environment or root is unavailable
    - never falls back to the orchestrator filesystem for an executor-owned
    root
    
    Skill catalog and instruction fragments have hard byte bounds, which
    also bound them below the 10K-token per-item context limit. If a
    selected executor skill has the same name as a legacy local skill, the
    executor selection owns that invocation and the local body is not
    injected a second time.
    
    Existing local and bundled skill loading remains in place. Omitting
    `selectedCapabilityRoots` therefore preserves current local-only
    behavior.
    
    ## Current semantics
    
    - Only environment-owned locations are represented in this first
    contract.
    - Roots are resolved by the destination extension, not by app-server or
    core.
    - An unavailable executor or invalid root produces a warning and no
    capabilities from that root; it does not trigger a local-filesystem
    fallback.
    - Selection applies to a newly started active thread.
    - MCP servers, connectors, and hooks beneath a selected plugin root are
    not activated yet.
    - Selection is not yet persisted or inherited across resume, fork, or
    subagent creation. Existing local capabilities continue to behave as
    they do today in those flows.
    
    ## Planned vertical follow-ups
    
    1. **Hosted HTTP MCP:** add an extension-backed HTTP MCP source that
    works without an executor, then replace the special-purpose MCP plugins
    loader with that implementation.
    2. **Executor MCP:** register and execute stdio MCP servers through the
    environment that owns the selected plugin root.
    3. **Backend skills:** add a hosted skill source whose catalog and
    bodies are accessed through extension tools rather than a filesystem.
    4. **Connectors and hooks:** activate those components through their
    owning extensions, using the same selected-root boundary and
    component-specific runtime.
    5. **Durable selection:** define the desired-selection lifecycle,
    persist it, and make resume, fork, and subagent inheritance explicit
    rather than accidental.
    6. **Local convergence:** incrementally route existing local plugin,
    skill, and MCP loading through the same extension model while preserving
    current local behavior.
    
    Each follow-up remains reviewable as an end-to-end capability. The
    platform selects roots, generic thread extension data carries the
    selection, and the owning extension resolves and operates its component.
    
    ## Verification
    
    Coverage added for:
    
    - app-server end-to-end discovery and explicit invocation of a skill
    inside an executor-selected plugin root
    - exclusive invocation when a selected executor skill collides with a
    local skill name
    - executor filesystem authority for discovery, canonicalization, and
    reads
    - thread extension initialization before lifecycle contributors run
    - stable executor catalog context, explicit invocation, context
    rebuilding, hidden skills, and preserved host/remote catalog behavior
    
    Targeted protocol, core-skills, skills-extension, core lifecycle, and
    app-server executor-skill tests were run during development.
  • fix(tui): scope MCP startup status by thread (#26639)
    ## Why
    
    MCP startup failures from spawned subagents were rendered as global
    notifications, so a child thread's failure could pollute the visible
    parent transcript. Routing the notification to the child exposed two
    related replay problems: session refresh could discard the buffered
    event, and a newly created child `ChatWidget` did not know the expected
    MCP server set, which could leave its startup spinner running after
    every server had settled.
    
    MCP startup diagnostics should remain visible in the thread that owns
    the startup without affecting other transcripts. The protocol also needs
    to support a future app-scoped MCP lifecycle where startup is not owned
    by any thread.
    
    ## Reported Behavior
    
    The [originating Slack
    report](https://openai.slack.com/archives/C08JZTV654K/p1780604538859939)
    called out that using subagents could turn MCP startup failures into a
    wall of yellow CLI warnings because repeated failures were not
    deduplicated. The intended behavior is for those diagnostics to remain
    visible once in the thread that owns the startup, without polluting the
    parent transcript.
    
    ## What Changed
    
    - add nullable `threadId` ownership to `mcpServer/startupStatus/updated`
    - populate it from the app-server conversation ID for the current
    thread-scoped lifecycle and regenerate the protocol schema and
    TypeScript artifacts
    - treat a missing or null `threadId` as app-scoped without injecting it
    into the active chat transcript
    - route and buffer thread-owned MCP startup notifications by thread in
    the TUI
    - preserve buffered MCP startup events across child session refresh
    - seed expected MCP servers before replaying a thread snapshot so
    startup reaches its terminal state
    - suppress an identical repeated failure warning for the same server
    within one startup round
    
    The owning thread still renders the detailed failure and final `MCP
    startup incomplete (...)` summary.
    
    ## How to Test
    
    1. Configure an optional MCP server named `smoke` that exits during
    initialization.
    2. Launch the TUI with multi-agent support enabled.
    3. Confirm the main thread's own startup failure renders one detailed
    `smoke` warning and one incomplete-startup summary.
    4. Spawn exactly one subagent.
    5. Confirm the parent transcript does not receive the subagent's MCP
    startup failure.
    6. Switch to the subagent thread and confirm it contains exactly one
    detailed `smoke` failure and one incomplete-startup summary.
    7. Confirm the subagent's MCP startup spinner disappears and the thread
    remains usable.
    8. Switch between the parent and subagent and confirm the warnings
    neither move nor duplicate.
    
    Targeted tests:
    
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server
    thread_start_emits_mcp_server_status_updated_notifications`
    - `just test -p codex-tui mcp_startup`
    
    The parent/child behavior and spinner completion were also exercised
    manually in tmux. `just argument-comment-lint` was attempted but blocked
    by an unrelated local Bazel LLVM empty-glob failure; touched Rust
    callsites were inspected manually.
  • permissions: enforce managed permission profile allowlists (#24852)
    ## Why
    
    Permission profile allowlists are an enterprise security boundary, but
    they also need to compose across the managed requirements layers added
    in #24620.
    
    A map representation lets each requirements layer add, allow, or revoke
    individual profiles without replacing an entire array.
    
    ## Managed Contract
    
    Administrators configure the mergeable allow map with
    `allowed_permission_profiles`. A recommended enterprise configuration
    explicitly lists every built-in and custom profile users should be able
    to select:
    
    ```toml
    default_permissions = "review_only"
    
    [allowed_permission_profiles]
    ":read-only" = true
    ":workspace" = true
    review_only = true
    # ":danger-full-access" is intentionally omitted, so it is denied.
    
    [permissions.review_only]
    extends = ":read-only"
    ```
    
    - Profiles whose effective merged value is `true` are allowed.
    - Missing profiles and profiles set to `false` are denied.
    - This is a closed allowlist: built-in profiles and profiles introduced
    in future versions are denied unless explicitly allowed.
    - Explicitly list each built-in profile the enterprise wants to make
    available. Omit built-ins such as `:danger-full-access` when they should
    remain unavailable.
    - Set `default_permissions` explicitly to the allowed profile users
    should receive when they have no local selection.
    - Higher-precedence layers override only the profile keys they define.
    - `false` is only needed when a higher-precedence layer must revoke a
    `true` inherited from a lower layer.
    - Explicit keys must refer to known built-in or managed profiles.
    
    A custom or narrowed allowlist requires an allowed
    `default_permissions`. For compatibility, if both `:workspace` and
    `:read-only` are explicitly allowed, an omitted default resolves to
    `:workspace`; customer configurations should still set the intended
    default explicitly.
    
    When `allowed_permission_profiles` is absent, existing implicit
    permission and legacy `sandbox_mode` behavior is unchanged.
    
    ## What Changed
    
    - Add `allowed_permission_profiles` as a `BTreeMap<String, bool>` that
    merges per profile across requirements layers.
    - Enforce managed defaults, strict denial of omitted profiles, and the
    explicitly allowed standard-pair fallback.
    - Expose `allowedPermissionProfiles` through `configRequirements/read`
    and regenerate its schemas.
    - Add regression coverage for map composition and revocation, managed
    defaults, strict denial of omitted built-ins, and API output.
    
    ## Verification
    
    - Focused `codex-config` coverage for layered map composition and
    revocation
    - Focused `codex-core` coverage for managed defaults, invalid defaults,
    strict denial of omitted built-ins, and the standard built-in pair
    - Focused `codex-app-server` coverage for requirements API output
    - Scoped Clippy for `codex-config`, `codex-core`,
    `codex-app-server-protocol`, and `codex-app-server`
    
    ## Documentation
    
    The managed `requirements.toml` documentation should introduce
    `allowed_permission_profiles` as a closed permission-profile allowlist
    before this setting is published on developers.openai.com.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [codex-rs] support v2 personal access tokens (#25731)
    ## Summary
    
    - add v2 personal access token support for `codex login
    --with-access-token` and `CODEX_ACCESS_TOKEN`
    - classify opaque `at-` tokens separately from legacy Agent Identity
    JWTs
    - hydrate required ChatGPT account metadata through AuthAPI
    `/v1/user-auth-credential/whoami`
    - use PATs directly as bearer tokens while preserving existing ChatGPT
    account surfaces
    - expose PAT-backed auth as the explicit `personalAccessToken`
    app-server auth mode
    
    ## Implementation
    
    PAT auth is intentionally small and stateless. Loading a PAT performs
    one AuthAPI metadata request, stores the hydrated metadata in the
    in-memory auth object, and redacts the secret from debug output. Legacy
    Agent Identity JWT handling remains unchanged. The shared access-token
    classifier lives in a private neutral module because it dispatches
    between both credential types.
    
    PAT hydration fails closed when AuthAPI omits any required metadata,
    including email. Hydrated metadata is intentionally not persisted:
    startup performs a live `whoami` preflight so revoked tokens or changed
    account metadata are not accepted from a stale cache.
    
    ## Workspace restriction scope
    
    This change intentionally does **not** apply
    `forced_chatgpt_workspace_id` to PAT authentication. The setting is a
    client-side config guardrail, not an authorization boundary, and PAT
    does not currently require workspace-ID parity. The PAT login and
    `CODEX_ACCESS_TOKEN` paths therefore validate through AuthAPI without
    threading workspace-restriction state through access-token loading.
    Existing workspace checks for non-PAT auth remain on their established
    paths.
    
    ## App-server compatibility
    
    The public app-server `AuthMode` is shared across v1 and v2, and
    PAT-backed auth reports `personalAccessToken` through both APIs.
    Following human review, this intentionally removes the temporary v1
    compatibility mapping that reported PATs as `chatgpt`; the deprecated v1
    API is kept in parity with v2 rather than maintaining a separate closed
    enum. Clients with exhaustive auth-mode handling in either API version
    must add the new case and should generally treat it as ChatGPT-backed
    unless they need PAT-specific behavior.
    
    The v1 auth-status response still omits the raw PAT when `includeToken`
    is requested because that response cannot carry the account metadata
    needed to reuse the credential safely. Persisted PAT auth also omits the
    new enum value so older Codex builds can deserialize `auth.json` and
    infer PAT auth from the credential field after a rollback.
    
    ## Validation
    
    Latest review-fix validation:
    
    - `CARGO_INCREMENTAL=0 just test -p codex-login` (126 passed)
    - `CARGO_INCREMENTAL=0 just test -p codex-cli` (263 passed)
    - `CARGO_INCREMENTAL=0 just test -p codex-cli
    stored_auth_validation_handles_personal_access_token`
    - `CARGO_INCREMENTAL=0 just test -p codex-app-server-protocol` (226
    passed)
    - `CARGO_INCREMENTAL=0 just test -p codex-models-manager
    refresh_available_models_uses_remote_only_catalog_for_chatgpt_auth`
    - `CARGO_INCREMENTAL=0 just test -p codex-tui
    existing_non_oauth_chatgpt_login_counts_as_signed_in`
    - `CARGO_INCREMENTAL=0 just fix -p codex-login -p
    codex-app-server-protocol -p codex-models-manager -p codex-tui -p
    codex-cli`
    - `just fmt`
    - `git diff --check`
    
    The broader `codex-tui` suite previously compiled and ran 2,834 tests.
    Three unrelated environment-sensitive guardian/IDE-socket tests failed
    after retries; the PAT-relevant TUI coverage passed.
  • Make runtime workspace roots absolute in app-server API (#26552)
    Stacked on #26532.
    
    ## Why
    
    #26532 moves cwd normalization to the app-server/core boundary.
    `runtimeWorkspaceRoots` still accepted raw paths in v2 requests and in
    `ConfigOverrides`, which left core responsible for interpreting those
    roots later. This makes runtime workspace roots follow the same
    absolute-path boundary as cwd.
    
    ## What
    
    - Change v2 `runtimeWorkspaceRoots` request fields for `thread/start`,
    `thread/resume`, `thread/fork`, and `turn/start` to `AbsolutePathBuf`.
    - Deduplicate already-absolute runtime roots in app-server handlers and
    pass them through `ConfigOverrides.workspace_roots` as
    `AbsolutePathBuf`.
    - Update TUI and exec client request builders to pass absolute runtime
    roots directly.
    - Update app-server docs, schema fixtures, and focused tests for
    absolute runtime roots.
    
    ## Testing
    
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server runtime_workspace_roots`
    - `just test -p codex-core
    session_permission_profile_rebinds_runtime_workspace_roots`
    - `just test -p codex-tui app_server_session`
    - `just test -p codex-exec`
  • feat(app-server): add remote control pairing status RPC (#26450)
    ## What
    
    Exposes the pairing status transport as experimental app-server v2 RPC
    `remoteControl/pairing/status`.
    
    - Adds request/response protocol types for exactly one lookup key:
    `pairingCode` or `manualPairingCode`, returning `{ claimed }`.
    - Registers the RPC with `global_shared_read("remote-control-pairing")`.
    - Wires the method through `MessageProcessor` and
    `RemoteControlRequestProcessor`.
    - Validates missing/conflicting pairing-code params as invalid requests.
    - Documents the RPC in `app-server/README.md`.
    - Adds processor, protocol export, and JSON-RPC integration coverage for
    both code paths.
    
    ## Why
    
    This is the app-server surface the desktop app can poll while the
    QR/manual pairing modal is active.
    
    Depends on https://github.com/openai/codex/pull/26449
    Related backend change: https://github.com/openai/openai/pull/990244
    
    ## Verification
    
    - `cargo test --manifest-path app-server-protocol/Cargo.toml
    remote_control`
    - `cargo test --manifest-path app-server/Cargo.toml remote_control`
    - `cargo fmt --all --check`
    - `git diff --check`
  • feat(app-server): expose account token usage [1 of 2] (#25344)
    ## Why
    
    Token activity is useful account-level context, but terminal clients
    need a supported app-server path to fetch it without reaching into
    ChatGPT backend details directly. The API should also live under the
    broader account usage umbrella so future usage surfaces can be added
    without proliferating user-facing concepts.
    
    ## What Changed
    
    - Add `codex-backend-client` support for the ChatGPT profile token-usage
    payload.
    - Add the v2 `account/usage/read` app-server RPC.
    - Map lifetime usage, peak daily usage, streak, longest task duration,
    and daily buckets into app-server protocol types.
    - Gate the request on Codex-backend auth, which supports ChatGPT auth
    tokens and AgentIdentity.
    - Regenerate the app-server JSON and TypeScript schema fixtures.
    
    ## Token Count Source
    
    `account/usage/read` returns the token-usage aggregate supplied by the
    ChatGPT profile backend. App-server maps that backend-owned aggregate
    into protocol fields; it does not recompute cached-token treatment,
    usage multipliers, or raw input/output totals locally.
    
    ## Stack
    
    1. feat(app-server): expose account token usage [1 of 2] (this PR)
    2. [#25345](https://github.com/openai/codex/pull/25345) feat(tui): add
    token activity command [2 of 2]
    
    ## How to Test
    
    1. Start an app-server client from this branch while authenticated with
    ChatGPT or AgentIdentity.
    2. Call `account/usage/read`.
    3. Confirm the response includes `summary` and `dailyUsageBuckets`.
    4. Also verify a session without Codex-backend auth receives the
    existing auth error path.
    
    Targeted tests:
    - `just test -p codex-backend-client -p codex-app-server-protocol -p
    codex-app-server`
    - `just write-app-server-schema`
  • [codex] Forward turn moderation metadata through app-server (#25710)
    ## Why
    First-party backends can supply turn-scoped moderation metadata that
    app-server clients need for client-side presentation. Exposing this as
    an experimental typed notification lets opted-in clients consume it
    without interpreting raw Responses API events.
    
    ## What changed
    - forward `response.metadata.openai_chatgpt_moderation_metadata` from
    Responses API SSE and WebSocket streams as turn-scoped moderation
    metadata
    - emit the experimental app-server v2 `turn/moderationMetadata`
    notification with `{ threadId, turnId, metadata }`
    - add app-server integration coverage for the typed moderation metadata
    notification
    
    ## Testing
    - `just test -p codex-core
    build_ws_client_metadata_includes_window_lineage_and_turn_metadata`
    - `just test -p codex-core` (fails locally: 46 failures and 1 timeout,
    primarily missing `test_stdio_server` and shell snapshot timeouts)
    - `just test -p codex-app-server-protocol`
    - `just test -p codex-app-server
    turn_moderation_metadata_emits_typed_notification_v2`
    - `just test -p codex-app-server` (fails locally: 792 passed, 10 failed,
    and 5 timed out; failures are in existing environment-sensitive tests,
    primarily because nested macOS `sandbox-exec` is not permitted)
    - `just write-app-server-schema --experimental --schema-root
    /tmp/codex-app-server-schema-experimental`