Commit Graph

14 Commits

  • feat(app-server): include turns page on thread resume (#23534)
    ## Summary
    
    The client currently calls `thread/resume` to establish live updates and
    immediately follows it with `thread/turns/list` to hydrate recent turns.
    This lets `thread/resume` return that page directly, eliminating a round
    trip and the ordering/deduplication gap between the two calls.
    
    Experimental clients opt in with `initialTurnsPage: { limit,
    sortDirection, itemsView }`. The response returns `initialTurnsPage` as
    a `TurnsPage`, including cursors for paging further back in history.
    Keeping the controls in a nested opt-in object provides the useful
    `thread/turns/list` knobs without spreading page-specific parameters
    across `thread/resume`.
    
    ## Verification
    
    - `just fmt`
    - `just write-app-server-schema --experimental`
    - `just write-app-server-schema`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-app-server
    thread_resume_initial_turns_page_matches_requested_turns_list_page
    --tests`
    - `cargo test -p codex-app-server
    thread_resume_rejoins_running_thread_even_with_override_mismatch
    --tests`
    - `just fix -p codex-app-server-protocol -p codex-app-server`
  • Add experimental turn additional context (#24154)
    ## Summary
    
    Adds experimental `additionalContext` support to `turn/start` and
    `turn/steer` so clients can provide ephemeral external context, such as
    browser or automation state, without turning that plumbing into a
    visible user prompt or triggering user-prompt lifecycle behavior.
    
    ## API Shape
    
    The parameter shape is:
    
    ```ts
    additionalContext?: Record<string, {
      value: string
      kind: "untrusted" | "application"
    }> | null
    ```
    
    Example:
    
    ```json
    {
      "additionalContext": {
        "browser_info": {
          "value": "Active tab is CI failures.",
          "kind": "untrusted"
        },
        "automation_info": {
          "value": "CI rerun is in progress.",
          "kind": "application"
        }
      }
    }
    ```
    
    The keys are opaque and caller-defined.
    
    ## Context Injection
    
    When provided, accepted entries are inserted into model context as
    hidden contextual message items, not as visible thread user-message
    items.
    
    `kind: "untrusted"` entries are inserted with role `user`:
    
    ```text
    <external_${key}>${value}</external_${key}>
    ```
    
    `kind: "application"` entries are inserted with role `developer`:
    
    ```text
    <${key}>${value}</${key}>
    ```
    
    Values are not escaped. Each value is truncated to 1k approximate tokens
    before wrapping.
    
    For `turn/start`, accepted additional context is inserted before normal
    user input. For `turn/steer`, additional context is merged only when the
    steer includes non-empty user input; context-only steers still reject as
    empty input.
    
    ## Dedupe Strategy
    
    `AdditionalContextStore` lives on session state and stores the latest
    complete additional-context map.
    
    Each `turn/start` or non-empty `turn/steer` treats its
    `additionalContext` as the current complete set of values. Entries are
    injected only when the key is new or the exact entry for that key
    changed, including `value` or `kind`. After merging, the store is
    replaced with the provided map, so omitted keys are removed from the
    retained set and can be injected again later if reintroduced.
    
    Omitting `additionalContext`, passing `null`, or passing an empty object
    resets the store to empty and injects nothing.
    
    ## What Changed
    
    - Threads experimental v2 `additionalContext` through app-server into
    core turn start and steer handling.
    - Adds separate contextual fragment types for untrusted user-role
    context and application developer-role context.
    - Uses pending response input items so additional context can be
    combined with normal user input without treating it as prompt text.
    - Adds integration coverage for start/steer flow, role routing,
    dedupe/reset behavior, deletion/re-add behavior, hook-blocked input
    behavior, empty context-only steer rejection, external-fragment marker
    matching, and truncation.
  • app-server: stop returning thread permission profiles (#22792)
    ## Why
    
    The app-server thread lifecycle API should no longer expose the full
    `PermissionProfile` value. After the permissions-profile migration,
    clients should round-trip only the active profile identity through
    `activePermissionProfile` and `permissions` when that identity is known.
    
    The full profile is server-side config. Treating a response-derived
    legacy sandbox projection as a new local profile can lose named-profile
    restrictions and accidentally widen permissions on the next turn. The
    legacy `sandbox` response field remains only as the
    compatibility/display fallback.
    
    ## What Changed
    
    - Removed `permissionProfile` from `ThreadStartResponse`,
    `ThreadResumeResponse`, and `ThreadForkResponse`.
    - Stopped populating that field in app-server thread start/resume/fork
    responses.
    - Updated embedded exec/TUI response mapping to derive display
    permission state from local config or the legacy sandbox fallback
    instead of a response profile value.
    - Added a TUI turn override shape that distinguishes preserving server
    permissions, selecting an active profile id, and sending a legacy
    sandbox for an explicit local override.
    - Preserved remote app-server permissions across turns by sending
    `permissions` only when an `activePermissionProfile` id is known, and
    otherwise sending no sandbox override unless the user selected a local
    override.
    - Kept embedded `thread/resume` hydration server-authored when
    `activePermissionProfile` is absent, which matches the live-thread
    attach path where the server ignores requested overrides.
    - Updated the app-server README to remove the obsolete lifecycle
    response `permissionProfile` reference. The remaining
    `permissionProfile` README references are request-side permission
    overrides.
    - Regenerated app-server JSON schema and TypeScript fixtures.
    - Kept the generated typed response enum exempt from
    `large_enum_variant`, matching the existing payload enum exemption after
    the lifecycle response variants shrank.
    
    ## How To Review
    
    Start with `codex-rs/app-server-protocol/src/protocol/v2/thread.rs` to
    confirm the response shape, then check the response construction in
    `codex-rs/app-server/src/request_processors`. The generated schema and
    TypeScript fixture changes are mechanical follow-through from the
    protocol removal.
    
    The TUI behavior is the delicate part: review
    `codex-rs/tui/src/app_server_session.rs` for response hydration and
    turn-start override projection, then
    `codex-rs/tui/src/app/thread_routing.rs` for the decision about whether
    the next turn should preserve the server snapshot, send an active
    profile id, or send a legacy sandbox for an explicit local override.
    
    ## Verification
    
    - `just write-app-server-schema`
    - `cargo test -p codex-app-server-protocol
    thread_lifecycle_responses_default_missing_optional_fields`
    - `cargo test -p codex-exec
    session_configured_from_thread_response_uses_permission_profile_from_config`
    - `cargo test -p codex-tui --lib thread_response`
    - `cargo test -p codex-tui turn_permissions_`
    - `cargo test -p codex-tui
    resume_response_restores_turns_from_thread_items`
    - `cargo test -p codex-analytics
    track_response_only_enqueues_analytics_relevant_responses`
    - `just fix -p codex-analytics`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-tui`
    - `just argument-comment-lint`
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/22792).
    * #22795
    * __->__ #22792
  • app-server: use permission ids and runtime workspace roots (#22611)
    ## Why
    
    This PR builds on [#22610](https://github.com/openai/codex/pull/22610)
    and is the app-server side of the migration from mutable per-turn
    `SandboxPolicy` replacement toward selecting immutable permission
    profiles by id plus mutable runtime workspace roots.
    
    Once permission profiles can carry their own immutable
    `workspace_roots`, app-server no longer needs to mutate the selected
    `PermissionProfile` just to represent thread-specific filesystem
    context. The mutable part now lives on the thread as explicit
    `runtimeWorkspaceRoots`, while `:workspace_roots` remains symbolic until
    the sandbox is realized for a turn.
    
    ## What Changed
    
    - Replaced the v2 permission-selection wrapper surface with plain
    profile ids for `thread/start`, `thread/resume`, `thread/fork`, and
    `turn/start`.
    - Removed the API surface for profile modifications
    (`PermissionProfileSelectionParams`,
    `PermissionProfileModificationParams`,
    `ActivePermissionProfileModification`).
    - Added experimental `runtimeWorkspaceRoots` fields to the thread
    lifecycle and turn-start APIs.
    - Threaded runtime workspace roots through core session/thread
    snapshots, turn overrides, app-server request handling, and command
    execution permission resolution.
    - Kept session permission state symbolic so later runtime root updates
    and cwd-only implicit-root retargeting rebind `:workspace_roots`
    correctly.
    - Updated the embedded clients just enough to send and restore the new
    thread state.
    - Refreshed the generated schema/TypeScript artifacts and the app-server
    README to match the new contract.
    
    ## Verification
    
    Targeted coverage for this layer lives in:
    
    - `codex-rs/app-server-protocol/src/protocol/v2/tests.rs`
    - `codex-rs/app-server/tests/suite/v2/thread_start.rs`
    - `codex-rs/app-server/tests/suite/v2/thread_resume.rs`
    - `codex-rs/app-server/tests/suite/v2/turn_start.rs`
    - `codex-rs/core/src/session/tests.rs`
    
    The key regression checks exercise that:
    
    - `runtimeWorkspaceRoots` resolve against the effective cwd on thread
    start.
    - Profile-declared workspace roots are excluded from the runtime
    workspace roots returned by app-server.
    - A turn-level runtime workspace-root update persists onto the thread
    and is returned by `thread/resume`.
    - A named permission profile selected on one turn remains symbolic so a
    later runtime-root-only turn update changes the actual sandbox writes.
    - A cwd-only turn update retargets the implicit runtime cwd root while
    preserving additional runtime roots.
    - The protocol fixtures and generated client artifacts stay in sync with
    the string-based permission selection contract.
    
    
    
    
    
    
    
    
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/22611).
    * #22612
    * __->__ #22611
  • Stop uploading accepted line fingerprints (#22180)
    ## Summary
    - keep accepted-line diff parsing and fingerprint hashing logic locally
    - stop uploading path/line hash fingerprints in the accepted-line
    analytics event payload
    - keep aggregate accepted added/deleted line counts in the event
    
    ## Testing
    - just fmt
    - cargo test -p codex-analytics
    - just fix -p codex-analytics
  • Emit accepted line fingerprint analytics (#21601)
    ## Why
    
    Codex assisted-code attribution needs a client-side accepted-code source
    that does not upload raw code. This adds a hash-only analytics event
    derived from the turn diff so downstream attribution can compare
    accepted Codex lines against commit or PR diffs.
    
    ## What Changed
    
    - Parse accepted/effective added lines from the final turn diff and emit
    `codex_accepted_line_fingerprints` analytics.
    - Hash repo, path, and normalized line content before upload; raw code
    and raw diffs are not included in the event.
    - Chunk large fingerprint payloads and send accepted-line fingerprint
    events in isolated requests while preserving normal batching for other
    analytics events.
    - Canonicalize Git remote URLs before repo hashing so SSH/HTTPS GitHub
    remotes join to the same repo hash.
    - Add parser coverage for unified diff hunk lines that look like `+++`
    or `---` file headers.
    
    ## Verification
    
    - `cargo test -p codex-analytics`
    - `cargo test -p codex-git-utils canonicalize_git_remote_url`
    - `just fix -p codex-analytics`
    - `just bazel-lock-check`
    - `git diff --check`
  • feat(app-server): move v2 sessionId onto Thread (#21336)
    ## Why
    
    `session_id` and `thread_id` are separate identities after #20437, but
    app-server only surfaced `sessionId` on the `thread/start`,
    `thread/resume`, and `thread/fork` response envelopes. Other
    thread-bearing surfaces such as `thread/list`, `thread/read`,
    `thread/started`, `thread/rollback`, `thread/metadata/update`, and
    `thread/unarchive` either lacked the grouping key or forced clients to
    special-case those three responses.
    
    Making `sessionId` part of the reusable `Thread` payload gives every v2
    API surface one place to expose session-tree identity.
    
    ## Mental model
      1. thread.sessionId lives on `Thread`
    2. It is a view/runtime identity for the current live session tree, not
    durable stored lineage metadata
    3. When app-server has a live loaded thread, it copies the real value
    from core’s session_configured.session_id
    4. When it only has stored/unloaded data, it falls back to
    thread.sessionId = thread.id
    
    ## What changed
    
    - Added `sessionId` to the v2
    [`Thread`](https://github.com/openai/codex/blob/8fc9e9b4cf81b6f61d432e71f1eb266f6f104b63/codex-rs/app-server-protocol/src/protocol/v2/thread_data.rs#L105-L109).
    - Removed the duplicate top-level `sessionId` fields from
    `thread/start`, `thread/resume`, and `thread/fork`; clients should now
    read `response.thread.sessionId`.
    - Populated `thread.sessionId` when building live thread responses,
    replaying loaded threads, and returning stored-thread summaries so the
    field is present across start, resume, fork, list, read, rollback,
    metadata-update, unarchive, and `thread/started` paths. See
    [`load_thread_from_resume_source_or_send_internal`](https://github.com/openai/codex/blob/8fc9e9b4cf81b6f61d432e71f1eb266f6f104b63/codex-rs/app-server/src/request_processors/thread_processor.rs#L2824-L2918)
    and
    [`thread_from_stored_thread`](https://github.com/openai/codex/blob/8fc9e9b4cf81b6f61d432e71f1eb266f6f104b63/codex-rs/app-server/src/request_processors/thread_processor.rs#L3671-L3719).
    - Preserved the stored-thread fallback: if a thread has not been loaded
    into a live session tree yet, `thread.sessionId` falls back to
    `thread.id`; once the thread is live again, the field reports the active
    session tree root.
    - Regenerated the JSON/TypeScript schemas and updated the app-server
    README examples to show
    [`thread.sessionId`](https://github.com/openai/codex/blob/8fc9e9b4cf81b6f61d432e71f1eb266f6f104b63/codex-rs/app-server/README.md#L306-L310)
    on the thread object.
  • feat: return session ID from thread/fork (#21332)
    ## Why
    
    `thread/start` and `thread/resume` already return `sessionId`, but
    `thread/fork` only returned the new thread. That left clients to infer
    the forked thread's session identity from `thread.id`, which kept the
    new `session_id` / `thread_id` split implicit at one lifecycle boundary.
    Follow-up to #20437.
    
    ## What changed
    
    - Add `sessionId` to `ThreadForkResponse`.
    - Populate it from the forked session configuration.
    - Regenerate the v2 JSON/TypeScript schema fixtures and update the
    app-server docs/example.
    - Extend the fork integration test to assert the returned `sessionId`.
    
    ## Verification
    
    - Added coverage in `thread_fork_creates_new_thread_and_emits_started`
    for the new response field.
  • feat: add session_id (#20437)
    ## Summary
    
    Related to
    https://openai.slack.com/archives/C095U48JNL9/p1777537279707449
    TLDR:
    We update the meaning of session ids and thread ids:
    * thread_id stays as now
    * session_id become a shared id between every thread under a /root
    thread (i.e. every sub-agent share the same session id)
    
    This PR introduces an explicit `SessionId` and threads it through the
    protocol/client boundary so `session_id` and `thread_id` can diverge
    when they need to, while preserving compatibility for older serialized
    `session_configured` events.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [codex-analytics] rework thread_source for thread analytics (#20949)
    ## Summary
    - make `thread_source` an explicit optional thread-level field on
    `thread/start`, `thread/fork`, and returned thread payloads
    - persist `thread_source` in rollout/session metadata so resumed live
    threads retain the original value
    - replace the old best-effort `session_source` -> `thread_source`
    mapping with an explicit caller-supplied analytics classification
    
    ## Why
    Before this change, analytics `thread_source` was populated by a
    best-effort mapping from `session_source`. `session_source` describes
    the runtime/client surface, not the actual thread-level origin, so that
    projection was not accurate enough to distinguish cases such as `user`,
    `subagent`, `memory_consolidation`, and future thread origins reliably.
    
    Making `thread_source` explicit keeps one thread-level analytics field
    while letting callers provide the real classification directly instead
    of recovering it indirectly from `session_source`.
    
    ## Impact
    For new analytics events, `thread_source` now reflects the explicit
    thread-level classification supplied by the caller rather than an
    inferred value derived from `session_source`. Existing protocol fields
    remain optional; callers that omit `threadSource` now produce `null`
    instead of a best-effort inferred value.
    
    ## Validation
    - `just write-app-server-schema`
    - `cargo test -p codex-analytics -p codex-core -p
    codex-app-server-protocol --no-run`
    - `cargo test -p codex-app-server-protocol
    generated_ts_optional_nullable_fields_only_in_params`
    - `cargo test -p codex-analytics
    thread_initialized_event_serializes_expected_shape`
    - `cargo test -p codex-core
    resume_stopped_thread_from_rollout_preserves_thread_source`
  • add turn items view to app-server turns (#21063)
    ## Why
    
    `Turn.items` currently overloads an empty array to mean either that no
    items exist or that the server intentionally did not load them for this
    response. That ambiguity blocks future lazy-loading work where clients
    need to distinguish unloaded, summary, and fully hydrated turn payloads.
    
    ## What changed
    
    - add a new `TurnItemsView` enum with `notLoaded`, `summary`, and `full`
    variants
    - add required `itemsView` metadata to app-server `Turn` payloads
    - mark reconstructed persisted history as `full` and live shell-style
    turn payloads as `notLoaded`
    - keep current `thread/turns/list` behavior unchanged and document that
    it still returns `full` turns today
    - regenerate the JSON and TypeScript protocol fixtures
    
    ## Verification
    
    - `just write-app-server-schema`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-app-server thread_read_can_include_turns`
    - `cargo test -p codex-app-server
    thread_turns_list_can_page_backward_and_forward`
    - `cargo test -p codex-app-server
    thread_resume_rejects_history_when_thread_is_running`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-app-server`
    - `just fmt`
  • [app-server] centralize client response analytics (#20059)
    ## Why
    
    The precursor PR keeps successful client responses typed until
    app-server's outgoing response seam. This follow-up uses that seam to
    move successful client-response analytics out of individual handlers and
    into the shared sender path, while keeping filtering decisions inside
    `codex-analytics`.
    
    ## What changed
    
    - Emit successful client-response analytics centrally from
    `OutgoingMessageSender::send_response`.
    - Remove duplicate handler-local response tracking for the current
    thread/turn lifecycle responses.
    - Keep analytics ingestion selective inside `AnalyticsEventsClient`, so
    unrelated client traffic is ignored before cloning or boxing.
    - Collapse client-response analytics facts onto one typed path and
    normalize payloads in the reducer.
    - Add direct client-filter coverage plus sender-level coverage for the
    centralized forwarding path.
    
    ## Verification
    
    - `cargo test -p codex-analytics`
    - `cargo test -p codex-app-server outgoing_message::tests --lib`
  • [app-server] type client response payloads (#20050)
    ## Why
    
    `pr17088` adds typed server-originated request/response plumbing, but
    successful client responses are still erased into bare JSON-RPC `result`
    values before app-server can make any typed decision about them.
    
    This precursor PR keeps successful client responses typed until the
    outgoing response seam. It is intentionally limited to
    protocol/app-server plumbing so the analytics behavior change can review
    separately on top.
    
    ## What changed
    
    - Add `ClientResponsePayload` as the pre-serialization client response
    body type.
    - Route app-server successful response paths through the typed payload
    seam while preserving existing handler-local analytics behavior.
    - Keep `InterruptConversation` JSON-RPC-only because it has no
    `ClientResponse` variant.
    - Move the new payload conversion tests into a dedicated protocol test
    module.
    
    ## Verification
    
    - `cargo check -p codex-app-server`
    - `cargo test -p codex-app-server-protocol`