Commit Graph

14 Commits

  • feat: render typed envelopes for multi-agent v2 messages (#28368)
    ## Why
    
    Multi-agent v2 messages need a consistent, model-visible envelope that
    identifies what kind of interaction occurred, who sent it, and which
    agent it targets. Previously, encrypted deliveries exposed only
    `encrypted_content`, while child completion used the legacy
    `<subagent_notification>` shape. That meant the client could not
    consistently present `NEW_TASK`, `MESSAGE`, and `FINAL_ANSWER` using the
    same format.
    
    This change adds the routing envelope as plaintext while keeping task
    and message payloads encrypted. No new Responses API field is required:
    an encrypted delivery is represented as an `input_text` header
    immediately followed by its existing `encrypted_content` item.
    
    Every envelope now follows this shape:
    
    ```text
    Message Type: <NEW_TASK | MESSAGE | FINAL_ANSWER>
    Task name: <recipient agent path>
    Sender: <author agent path>
    Payload:
    <message payload>
    ```
    
    ## Message types
    
    ### `NEW_TASK`
    
    `NEW_TASK` is used when the recipient should begin a new turn, including
    an initial `spawn_agent` task and a later `followup_task`.
    
    For a root agent spawning `/root/worker`, the request contains a
    plaintext envelope followed by the encrypted task:
    
    ```json
    {
      "type": "agent_message",
      "author": "/root",
      "recipient": "/root/worker",
      "content": [
        {
          "type": "input_text",
          "text": "Message Type: NEW_TASK\nTask name: /root/worker\nSender: /root\nPayload:\n"
        },
        {
          "type": "encrypted_content",
          "encrypted_content": "<encrypted task payload>"
        }
      ]
    }
    ```
    
    Conceptually, the model receives:
    
    ```text
    Message Type: NEW_TASK
    Task name: /root/worker
    Sender: /root
    Payload:
    Review the authentication changes and report any regressions.
    ```
    
    ### `MESSAGE`
    
    `MESSAGE` is used for a queued `send_message` delivery. It communicates
    with an existing agent without starting a new turn.
    
    For `/root/worker` reporting progress to the root agent, the request
    contains:
    
    ```json
    {
      "type": "agent_message",
      "author": "/root/worker",
      "recipient": "/root",
      "content": [
        {
          "type": "input_text",
          "text": "Message Type: MESSAGE\nTask name: /root\nSender: /root/worker\nPayload:\n"
        },
        {
          "type": "encrypted_content",
          "encrypted_content": "<encrypted message payload>"
        }
      ]
    }
    ```
    
    Conceptually, the model receives:
    
    ```text
    Message Type: MESSAGE
    Task name: /root
    Sender: /root/worker
    Payload:
    The protocol tests pass; I am checking the resume path now.
    ```
    
    ### `FINAL_ANSWER`
    
    `FINAL_ANSWER` is emitted when a child agent reaches a terminal state
    and reports its result to its parent. Completion payloads are already
    available locally, so the complete envelope is represented as plaintext
    rather than as a plaintext header plus encrypted content.
    
    For `/root/worker` completing work for the root agent, the request
    contains:
    
    ```json
    {
      "type": "agent_message",
      "author": "/root/worker",
      "recipient": "/root",
      "content": [
        {
          "type": "input_text",
          "text": "Message Type: FINAL_ANSWER\nTask name: /root\nSender: /root/worker\nPayload:\nNo regressions found."
        }
      ]
    }
    ```
    
    The model-visible form is:
    
    ```text
    Message Type: FINAL_ANSWER
    Task name: /root
    Sender: /root/worker
    Payload:
    No regressions found.
    ```
    
    Errored, shut down, and missing agents also use `FINAL_ANSWER`, with a
    terminal-status description in the payload.
    
    ## What changed
    
    - Render `NEW_TASK` or `MESSAGE` in
    `InterAgentCommunication::to_model_input_item`, based on whether the
    encrypted delivery starts a turn.
    - Replace the multi-agent v2 `<subagent_notification>` completion
    payload with a model-visible `FINAL_ANSWER` envelope.
    - Document `Task name`, `Sender`, and `Payload` consistently in the
    multi-agent developer instructions.
    - Prevent local-only history projections from treating an encrypted
    message's plaintext header as the complete assistant message.
    - Preserve rollout-trace interaction edges when an agent message
    contains both plaintext and encrypted content.
    
    Legacy multi-agent behavior remains unchanged.
    
    ## Verification
    
    - `just test -p codex-protocol`
    - `just test -p codex-rollout-trace`
    - `just test -p codex-web-search-extension`
    - `just test -p codex-core
    encrypted_multi_agent_v2_spawn_sends_agent_message_to_child`
    - `just test -p codex-core
    plaintext_multi_agent_v2_completion_sends_agent_message`
    - `just test -p codex-core
    multi_agent_v2_followup_task_completion_notifies_parent_on_every_turn`
    - `just test -p codex-core
    multi_agent_v2_completion_queues_message_for_direct_parent`
  • skills: make backend plugin skills invocable without an executor (#27387)
    ## Why
    
    #27198 made the extension-owned `codex_apps` MCP connection the hosted
    plugin runtime, but its `mcp/skill` resources still bypassed the skills
    extension. App-server could list and read those resources through
    generic MCP APIs, but a thread with no selected environment did not
    expose them in the model's skills catalog or load their `SKILL.md`
    through `$skill`.
    
    Hosted skills should stay remote while using the same typed catalog,
    source authority, deduplication, bounded contextual catalog, and
    selected-skill prompt injection as host and executor skills. They should
    not be downloaded or exposed as ambient filesystem paths.
    
    ## What changed
    
    - Add a session-scoped `McpResourceClient` over the replaceable MCP
    connection manager so resource list/read calls follow startup and
    refresh replacements.
    - Add a `BackendSkillProvider` that pages `codex_apps` resources,
    accepts bounded and validated `mcp/skill` entries, and reads a selected
    skill's `SKILL.md` through the same MCP connection.
    - Register the remote provider in app-server and include it in the
    skills catalog even when a thread has no selected capability roots or
    executor.
    - Contribute hosted skill metadata through the bounded
    `AvailableSkillsInstructions` developer-context path, exclude remote
    entries from per-turn catalog injection, and classify `<skills>`
    messages as contextual developer content so rollback can trim and
    rebuild them correctly.
    
    ## Testing
    
    - Extend the app-server MCP resource integration test with
    `environments: []` to exercise two-page discovery, filter a
    non-`mcp/skill` resource, verify the escaped developer catalog entry and
    user-role `<skill>` fragment containing the fetched `SKILL.md`, and
    preserve generic MCP resource reads.
    - Add core event-mapping coverage that classifies `<skills>` developer
    messages as contextual history.
  • [codex] Add token budget context feature (#27438)
    ## Why
    
    The model should be able to see bounded context-window budget metadata
    when the `token_budget` feature is enabled. The full-window message is
    only injected with full context, while normal turns get a smaller
    follow-up only when reported usage first crosses a budget threshold.
    
    ## What changed
    
    - Added the `TokenBudget` feature flag.
    - Added `<token_budget>` developer fragments for full context-window
    metadata and current-window remaining tokens.
    - Inserted the threshold message during normal turn handling by
    comparing token usage before and after sampling, avoiding persistent
    threshold bookkeeping.
    - Added core integration coverage for full-context-only metadata and
    25/50/75 percent threshold messages.
    
    ## Verification
    
    - `just test -p codex-core token_budget`
    - `git diff --check`
  • Add saved image path hint to standalone image generation (#25947)
    ## Why
    
    Standalone image generation returns image bytes to the model, but the
    model also needs the host artifact path to reference the generated file
    in follow-up work.
    
    ## What changed
    
    - Append the default saved-image path hint alongside the generated image
    tool output.
    - Reuse the existing core image-generation hint text.
    - Pass the thread ID and Codex home directory needed to compute the
    artifact path.
    - Add app-server and extension coverage for the model-visible hint.
    
    ## Validation
    
    - `just fmt`
    - `just bazel-lock-check`
    - `just test -p codex-app-server
    standalone_image_generation_returns_saved_path_hint_to_model`
  • chore: extract context fragments into dedicated crate (#26122)
    ## Why
    
    `codex-core` currently owns the generic contextual-fragment trait and
    several reusable fragment implementations. That makes it harder for
    other crates to share the same host-owned model-input abstraction
    without depending on all of `codex-core`.
    
    This change extracts the reusable fragment machinery into a small
    `codex-context-fragments` crate so future extension and skills work can
    depend on the fragment abstraction directly.
    
    ## What Changed
    
    - Added the `codex-context-fragments` crate with:
      - `ContextualUserFragment`
      - `FragmentRegistration` / `FragmentRegistrationProxy`
      - additional-context fragment types
    - Moved `SkillInstructions` into `codex-core-skills`, since
    skill-specific rendering belongs with skills rather than generic core
    context machinery.
    - Kept `codex-core` re-exporting the fragment types it still uses
    internally, so existing call sites keep the same shape.
    - Updated Cargo and Bazel workspace metadata for the new crate.
    
    ## Verification
    
    - `cargo metadata --locked --format-version 1 --no-deps`
    - `just bazel-lock-update`
    - `just bazel-lock-check`
  • Use internal model context fragments for goal steering (#24918)
    ## Why
    
    Goal steering is one form of runtime-owned model context, but the old
    `<goal_context>` wrapper made the contextual-fragment hiding path
    goal-specific. Using a source-labeled internal context fragment gives
    core and extensions a shared shape for hidden model steering while
    keeping those prompts out of visible turn history.
    
    The change also keeps legacy `<goal_context>` messages recognized as
    hidden contextual input so existing stored history does not start
    rendering old goal-steering prompts as user-visible turn items.
    
    ## What Changed
    
    - Replaces `GoalContext` with `InternalModelContextFragment` plus a
    validated `InternalContextSource`.
    - Renders goal steering as `<codex_internal_context
    source="goal">...</codex_internal_context>`.
    - Updates core goal steering and `ext/goal` steering to inject the new
    internal-context fragment.
    - Updates contextual-fragment, event-mapping, goal, and session tests
    for the new wrapper.
    
    ## Test Coverage
    
    - Adds coverage for detecting the new internal model context fragment.
    - Preserves coverage for hiding legacy `<goal_context>` fragments.
    - Verifies invalid internal context sources are rejected and arbitrary
    context tags are not hidden.
    - Updates goal steering/session assertions to expect the new
    `source="goal"` wrapper.
  • Add experimental turn additional context (#24154)
    ## Summary
    
    Adds experimental `additionalContext` support to `turn/start` and
    `turn/steer` so clients can provide ephemeral external context, such as
    browser or automation state, without turning that plumbing into a
    visible user prompt or triggering user-prompt lifecycle behavior.
    
    ## API Shape
    
    The parameter shape is:
    
    ```ts
    additionalContext?: Record<string, {
      value: string
      kind: "untrusted" | "application"
    }> | null
    ```
    
    Example:
    
    ```json
    {
      "additionalContext": {
        "browser_info": {
          "value": "Active tab is CI failures.",
          "kind": "untrusted"
        },
        "automation_info": {
          "value": "CI rerun is in progress.",
          "kind": "application"
        }
      }
    }
    ```
    
    The keys are opaque and caller-defined.
    
    ## Context Injection
    
    When provided, accepted entries are inserted into model context as
    hidden contextual message items, not as visible thread user-message
    items.
    
    `kind: "untrusted"` entries are inserted with role `user`:
    
    ```text
    <external_${key}>${value}</external_${key}>
    ```
    
    `kind: "application"` entries are inserted with role `developer`:
    
    ```text
    <${key}>${value}</${key}>
    ```
    
    Values are not escaped. Each value is truncated to 1k approximate tokens
    before wrapping.
    
    For `turn/start`, accepted additional context is inserted before normal
    user input. For `turn/steer`, additional context is merged only when the
    steer includes non-empty user input; context-only steers still reject as
    empty input.
    
    ## Dedupe Strategy
    
    `AdditionalContextStore` lives on session state and stores the latest
    complete additional-context map.
    
    Each `turn/start` or non-empty `turn/steer` treats its
    `additionalContext` as the current complete set of values. Entries are
    injected only when the key is new or the exact entry for that key
    changed, including `value` or `kind`. After merging, the store is
    replaced with the provided map, so omitted keys are removed from the
    retained set and can be injected again later if reintroduced.
    
    Omitting `additionalContext`, passing `null`, or passing an empty object
    resets the store to empty and injects nothing.
    
    ## What Changed
    
    - Threads experimental v2 `additionalContext` through app-server into
    core turn start and steer handling.
    - Adds separate contextual fragment types for untrusted user-role
    context and application developer-role context.
    - Uses pending response input items so additional context can be
    combined with normal user input without treating it as prompt text.
    - Adds integration coverage for start/steer flow, role routing,
    dedupe/reset behavior, deletion/re-add behavior, hook-blocked input
    behavior, empty context-only steer rejection, external-fragment marker
    matching, and truncation.
  • [codex] Steer budget-limited goal extension turns (#23718)
    ## What
    - Add a small extension capability for injecting model-visible response
    items into the active turn
    - Have the goal extension inject hidden goal-context steering when
    tool-finish accounting reaches `BudgetLimited`
    - Cover the extension backend path with an assertion on the injected
    steering item
    
    ## Why
    PR #23696 persists and emits the budget-limited goal update from
    tool-finish accounting, but it leaves the model unaware of that
    transition. The existing core runtime steers the model to wrap up in
    this case; the extension path should do the same through an explicit
    host capability.
    
    ## Testing
    - `just fmt`
    - `cargo test -p codex-goal-extension`
    - `cargo test -p codex-extension-api`
  • [codex] Filter legacy warning messages during compaction (#22243)
    ## Why
    
    Older sessions can contain model-warning records persisted as `user`
    messages, including the unified exec process-limit warning, the
    `apply_patch`-via-`exec_command` warning, and the model-mismatch
    high-risk cyber fallback warning. Those warnings are no longer produced
    as conversation history items, but when old sessions compact they should
    still be recognized as injected context rather than preserved as real
    user turns.
    
    ## What changed
    
    - Removed `record_model_warning` and the production paths that emitted
    these warning messages into conversation history.
    - Added `LegacyUnifiedExecProcessLimitWarning`,
    `LegacyApplyPatchExecCommandWarning`, and `LegacyModelMismatchWarning`
    contextual fragments that are used only for matching old persisted
    messages.
    - Registered the legacy fragments with contextual user message detection
    so compaction filters them through the existing fragment path.
    - Added focused compaction coverage for old warning messages being
    dropped during compacted-history processing.
    
    ## Testing
    
    - `cargo test -p codex-core warning`
    - `just fix -p codex-core`
  • Improve goal continuation based on feedback (#22045)
    ## Summary
    
    This PR updates the goal continuation prompt to address feedback from
    early adopters. There are two primary changes:
    
    1. Goal continuation and budget-limit steering prompts now use hidden
    user-context messages instead of hidden developer messages.
    2. The goal continuation prompt is refined to improve the model's
    ability to fully complete the active goal rather than stop at a smaller
    or merely passing subset.
    
    The user-message transition is important for two reasons. First, it
    eliminates an issue where older steering messages could be responded to
    again after a new turn. Second, it works better with compaction because
    user messages are treated differently from developer messages during
    compaction.
    
    The prompt refinements make persistence explicit, ground work in current
    evidence, encourage `update_plan` for multi-step progress visibility,
    and require stronger completion audits before calling `update_goal`. It
    also removes the elapsed-time reporting in the prompt; I saw evidence
    that this was causing the model to shortcut work as it became nervous
    about time.
    
    These changes were tested with evals. Chriss4123 has also been running
    independent evals in
    [#19910](https://github.com/openai/codex/issues/19910), and many of the
    improvements in this PR were suggested by him.
    
    ## Verification
    
    - Tested with evals.
    - Added and updated focused `codex-core` coverage for hidden goal user
    context, continuation and budget-limit request shape, prompt rendering,
    and objective delimiter escaping.
  • feat: split memories part 2 (#19860)
    Keep extracting memories out of core and moving the write trigger in the
    app-server
    This is temporary and it should move at the client level as a follow-up
    This makes core fully independant from `codex-memories-write`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • feat: drop spawned-agent context instructions (#19127)
    ## Why
    
    MultiAgentV2 children should not receive an extra model-visible
    developer fragment just because they were spawned. The parent/configured
    developer instructions should carry through normally, but the dedicated
    `<spawned_agent_context>` block is no longer desired.
    
    ## What changed
    
    - Removed the `SpawnAgentInstructions` context fragment and its
    `<spawned_agent_context>` wrapper.
    - Stopped appending spawned-agent instructions in
    `codex-rs/core/src/tools/handlers/multi_agents_v2/spawn.rs`.
    - Updated subagent notification coverage to assert inherited parent
    developer instructions without expecting the spawned-agent wrapper.
    
    ## Verification
    
    - `cargo test -p codex-core --test all
    spawned_multi_agent_v2_child_inherits_parent_developer_context --
    --nocapture`
    - `cargo test -p codex-core --test all
    skills_toggle_skips_instructions_for_parent_and_spawned_child --
    --nocapture`
    - `cargo test -p codex-core --test all subagent_notifications --
    --nocapture`
  • Split DeveloperInstructions into individual fragments. (#18813)
    Split DeveloperInstructions into individual fragments.
  • Organize context fragments (#18794)
    Organize context fragments under `core/context`. Implement same trait on
    all of them.