Commit Graph

437 Commits

  • [feat] persist dynamic tools in session rollout file (#10130)
    Add dynamic tools to rollout file for persistence & read from rollout on
    resume. Ran a real example and spotted the following in the rollout
    file:
    ```
    {"timestamp":"2026-01-29T01:27:57.468Z","type":"session_meta","payload":{"id":"019c075d-3f0b-77e3-894e-c1c159b04b1e","timestamp":"2026-01-29T01:27:57.451Z","...."dynamic_tools":[{"name":"demo_tool","description":"Demo dynamic tool","inputSchema":{"additionalProperties":false,"properties":{"city":{"type":"string"}},"required":["city"],"type":"object"}}],"git":{"commit_hash":"ebc573f15c01b8af158e060cfedd401f043e9dfa","branch":"dev/cc/dynamic-tools","repository_url":"https://github.com/openai/codex.git"}}}
    ```
  • Better handling skill depdenencies on ENV VAR. (#9017)
    An experimental flow for env var skill dependencies. Skills can now
    declare required env vars in SKILL.md; if missing, the CLI prompts the
    user to get the value, and Core will store it in memory (eventually to a
    local persistent store)
    <img width="790" height="169" alt="image"
    src="https://github.com/user-attachments/assets/cd928918-9403-43cb-a7e7-b8d59bcccd9a"
    />
  • Fall back to http when websockets fail (#10139)
    I expect not all proxies work with websockets, fall back to http if
    websockets fail.
  • [connectors] Support connectors part 2 - slash command and tui (#9728)
    - [x] Support `/apps` slash command to browse the apps in tui.
    - [x] Support inserting apps to prompt using `$`.
    - [x] Lots of simplification/renaming from connectors to apps.
  • Ensure auto-compaction starts after turn started (#10129)
    Start auto-compaction only after TurnStarted is emitted.\nAdd an
    integration test for deterministic ordering.
  • chore: deprecate old web search feature flags (#10097)
    deprecate all old web search flags and aliases, including:
    - `[features].web_search_request` and `[features].web_search_cached`
    - `[tools].web_search`
    - `[features].web_search`
    
    slightly rework `legacy_usages` to enable pointing to non-features from
    deprecated features; we need to point to `web_search` (not under
    `[features]`) from things like `[features].web_search_cached` and
    `[features].web_search_request`.
    
    Added integration tests to confirm deprecation notice is shown on
    explicit enablement and disablement of deprecated flags.
  • feat: sqlite 1 (#10004)
    Add a `.sqlite` database to be used to store rollout metatdata (and
    later logs)
    This PR is phase 1:
    * Add the database and the required infrastructure
    * Add a backfill of the database
    * Persist the newly created rollout both in files and in the DB
    * When we need to get metadata or a rollout, consider the `JSONL` as the
    source of truth but compare the results with the DB and show any errors
  • feat(core) RequestRule (#9489)
    ## Summary
    Instead of trying to derive the prefix_rule for a command mechanically,
    let's let the model decide for us.
    
    ## Testing
    - [x] tested locally
  • [skills] Auto install MCP dependencies when running skils with dependency specs. (#9982)
    Auto install MCP dependencies when running skils with dependency specs.
  • fix: enable per-turn updates to web search mode (#10040)
    web_search can now be updated per-turn, for things like changes to
    sandbox policy.
    
    `SandboxPolicy::DangerFullAccess` now sets web_search to `live`, and the
    default is still `cached`.
    
    Added integration tests.
  • remove sandbox globals. (#9797)
    Threads sandbox updates through OverrideTurnContext for active turn
    Passes computed sandbox type into safety/exec
  • fix(app-server, core): defer initial context write to rollout file until first turn (#9950)
    ### Overview
    Currently calling `thread/resume` will always bump the thread's
    `updated_at` timestamp. This PR makes it the `updated_at` timestamp
    changes only if a turn is triggered.
    
    ### Additonal context
    What we typically do on resuming a thread is **always** writing “initial
    context” to the rollout file immediately. This initial context includes:
    - Developer instructions derived from sandbox/approval policy + cwd
    - Optional developer instructions (if provided)
    - Optional collaboration-mode instructions
    - Optional user instructions (if provided)
    - Environment context (cwd, shell, etc.)
    
    This PR defers writing the “initial context” to the rollout file until
    the first `turn/start`, so we don't inadvertently bump the thread's
    `updated_at` timestamp until a turn is actually triggered.
    
    This works even though both `thread/resume` and `turn/start` accept
    overrides (such as `model`, `cwd`, etc.) because the initial context is
    seeded from the effective `TurnContext` in memory, computed at
    `turn/start` time, after both sets of overrides have been applied.
    
    **NOTE**: This is a very short-lived solution until we introduce sqlite.
    Then we can remove this.
  • make cached web_search client-side default (#9974)
    [Experiment](https://console.statsig.com/50aWbk2p4R76rNX9lN5VUw/experiments/codex_web_search_rollout/summary)
    for default cached `web_search` completed; cached chosen as default.
    
    Update client to reflect that.
  • fix: handle all web_search actions and in progress invocations (#9960)
    ### Summary
    - Parse all `web_search` tool actions (`search`, `find_in_page`,
    `open_page`).
    - Previously we only parsed + displayed `search`, which made the TUI
    appear to pause when the other actions were being used.
    - Show in progress `web_search` calls as `Searching the web`
      - Previously we only showed completed tool calls
    
    <img width="308" height="149" alt="image"
    src="https://github.com/user-attachments/assets/90a4e8ff-b06a-48ff-a282-b57b31121845"
    />
    
    ### Tests
    Added + updated tests, tested locally
    
    ### Follow ups
    Update VSCode extension to display these as well
  • feat: dynamic tools injection (#9539)
    ## Summary
    Add dynamic tool injection to thread startup in API v2, wire dynamic
    tool calls through the app server to clients, and plumb responses back
    into the model tool pipeline.
    
    ### Flow (high level)
    - Thread start injects `dynamic_tools` into the model tool list for that
    thread (validation is done here).
    - When the model emits a tool call for one of those names, core raises a
    `DynamicToolCallRequest` event.
    - The app server forwards it to the client as `item/tool/call`, waits
    for the client’s response, then submits a `DynamicToolResponse` back to
    core.
    - Core turns that into a `function_call_output` in the next model
    request so the model can continue.
    
    ### What changed
    - Added dynamic tool specs to v2 thread start params and protocol types;
    introduced `item/tool/call` (request/response) for dynamic tool
    execution.
    - Core now registers dynamic tool specs at request time and routes those
    calls via a new dynamic tool handler.
    - App server validates tool names/schemas, forwards dynamic tool call
    requests to clients, and publishes tool outputs back into the session.
    - Integration tests
  • feat: ephemeral threads (#9765)
    Add ephemeral threads capabilities. Only exposed through the
    `app-server` v2
    
    The idea is to disable the rollout recorder for those threads.
  • change collaboration mode to struct (#9793)
    Shouldn't cause behavioral change
  • [connectors] Support connectors part 1 - App server & MCP (#9667)
    In order to make Codex work with connectors, we add a built-in gateway
    MCP that acts as a transparent proxy between the client and the
    connectors. The gateway MCP collects actions that are accessible to the
    user and sends them down to the user, when a connector action is chosen
    to be called, the client invokes the action through the gateway MCP as
    well.
    
     - [x] Add the system built-in gateway MCP to list and run connectors.
     - [x] Add the app server methods and protocol
  • feat(core) update Personality on turn (#9644)
    ## Summary
    Support updating Personality mid-Thread via UserTurn/OverwriteTurn. This
    is explicitly unused by the clients so far, to simplify PRs - app-server
    and tui implementations will be follow-ups.
    
    ## Testing
    - [x] added integration tests
  • Support end_turn flag (#9698)
    Experimental flag that signals the end of the turn.
  • Fix: Lower log level for closed-channel send (#9653)
    ## What?
    - Downgrade the closed-channel send error log to debug in
    `codex-rs/core/src/codex.rs`.
    
    ## Why?
    - `async_channel::Sender::send` only fails when the channel is closed,
    so the current error-level log is noisy during normal shutdown. See
    issue #9652.
    
    ## How?
    - Replace the error log with a debug log on send failure.
    
    ## Tests
    - `just fmt`
    - `just fix -p codex-core`
    - `cargo test -p codex-core`
  • Add UI for skill enable/disable. (#9627)
    "/skill" will now allow you to enable/disable skills:
    <img width="658" height="199" alt="image"
    src="https://github.com/user-attachments/assets/bf8994c8-d6c1-462f-8bbb-f1ee9241caa4"
    />
  • feat(core) ModelInfo.model_instructions_template (#9597)
    ## Summary
    #9555 is the start of a rename, so I'm starting to standardize here.
    Sets up `model_instructions` templating with a strongly-typed object for
    injecting a personality block into the model instructions.
    
    ## Testing
    - [x] Added tests
    - [x] Ran locally
  • Add collaboration_mode to TurnContextItem (#9583)
    ## Summary
    - add optional `collaboration_mode` to `TurnContextItem` in rollouts
    - persist the current collaboration mode when recording turn context
    (sampling + compaction)
    
    ## Rationale
    We already persist turn context data for resume logic. Capturing
    collaboration mode in the rollout gives us the mode context for each
    turn, enabling follow‑up work to diff mode instructions correctly on
    resume.
    
    ## Changes
    - protocol: add optional `collaboration_mode` field to `TurnContextItem`
    - core: persist collaboration mode alongside other turn context settings
    in rollouts
  • Added "codex." prefix to "conversation.turn.count" metric name (#9594)
    All other metrics names start with "codex.", so I presume this was an
    unintended omission.
  • Reject ask user question tool in Execute and Custom (#9560)
    ## Summary
    - Keep `request_user_input` in the tool list but reject it at runtime in
    Execute/Custom modes with a clear model-facing error.
    - Add a session accessor for current collaboration mode and enforce the
    gate in the request_user_input handler.
    - Update core/app-server tests to use Plan mode for success and add
    Execute/Custom rejection coverage.
  • feat: rename experimental_instructions_file to model_instructions_file (#9555)
    A user who has `experimental_instructions_file` set will now see this:
    
    <img width="888" height="660" alt="image"
    src="https://github.com/user-attachments/assets/51c98312-eb9b-4881-81f1-bea6677e158d"
    />
    
    And a `codex exec` would include this warning:
    
    <img width="888" height="660" alt="image"
    src="https://github.com/user-attachments/assets/a89f62be-1edf-4593-a75e-e0b4a762ed7d"
    />
  • fix: prevent repeating interrupted turns (#9043)
    ## What
    Record a model-visible `<turn_aborted>` marker in history when a turn is
    interrupted, and treat it as a session prefix.
    
    ## Why
    When a turn is interrupted, Codex emits `TurnAborted` but previously did
    not persist anything model-visible in the conversation history. On the
    next user turn, the model can’t tell the previous work was aborted and
    may resume/repeat earlier actions (including duplicated side effects
    like re-opening PRs).
    
    Fixes: https://github.com/openai/codex/issues/9042
    
    ## How
    On `TurnAbortReason::Interrupted`, append a hidden user message
    containing a `<turn_aborted>…</turn_aborted>` marker and flush.
    Treat `<turn_aborted>` like `<environment_context>` for session-prefix
    filtering.
    Add a regression test to ensure follow-up turns don’t repeat side
    effects from an aborted turn.
    
    ## Testing
    `just fmt`
    `just fix -p codex-core`
    `cargo test -p codex-core -- --test-threads=1`
    `cargo test --all-features -- --test-threads=1`
    
    ---------
    
    Co-authored-by: Skylar Graika <sgraika127@gmail.com>
    Co-authored-by: jif-oai <jif@openai.com>
    Co-authored-by: Eric Traut <etraut@openai.com>
  • feat(personality) introduce model_personality config (#9459)
    ## Summary
    Introduces the concept of a config model_personality. I would consider
    this an MVP for testing out the feature. There are a number of
    follow-ups to this PR:
    
    - More sophisticated templating with validation
    - In-product experience to manage this
    
    ## Testing
    - [x] Testing locally
  • Persist text elements through TUI input and history (#9393)
    Continuation of breaking up this PR
    https://github.com/openai/codex/pull/9116
    
    ## Summary
    - Thread user text element ranges through TUI/TUI2 input, submission,
    queueing, and history so placeholders survive resume/edit flows.
    - Preserve local image attachments alongside text elements and rehydrate
    placeholders when restoring drafts.
    - Keep model-facing content shapes clean by attaching UI metadata only
    to user input/events (no API content changes).
    
    ## Key Changes
    - TUI/TUI2 composer now captures text element ranges, trims them with
    text edits, and restores them when submission is suppressed.
    - User history cells render styled spans for text elements and keep
    local image paths for future rehydration.
    - Initial chat widget bootstraps accept empty `initial_text_elements` to
    keep initialization uniform.
    - Protocol/core helpers updated to tolerate the new InputText field
    shape without changing payloads sent to the API.
  • fix(core) Preserve base_instructions in SessionMeta (#9427)
    ## Summary
    This PR consolidates base_instructions onto SessionMeta /
    SessionConfiguration, so we ensure `base_instructions` is set once per
    session and should be (mostly) immutable, unless:
    - overridden by config on resume / fork
    - sub-agent tasks, like review or collab
    
    
    In a future PR, we should convert all references to `base_instructions`
    to consistently used the typed struct, so it's less likely that we put
    other strings there. See #9423. However, this PR is already quite
    complex, so I'm deferring that to a follow-up.
    
    ## Testing
    - [x] Added a resume test to assert that instructions are preserved. In
    particular, `resume_switches_models_preserves_base_instructions` fails
    against main.
    
    Existing test coverage thats assert base instructions are preserved
    across multiple requests in a session:
    - Manual compact keeps baseline instructions:
    core/tests/suite/compact.rs:199
    - Auto-compact keeps baseline instructions:
    core/tests/suite/compact.rs:1142
    - Prompt caching reuses the same instructions across two requests:
    core/tests/suite/prompt_caching.rs:150 and
    core/tests/suite/prompt_caching.rs:157
    - Prompt caching with explicit expected string across two requests:
    core/tests/suite/prompt_caching.rs:213 and
    core/tests/suite/prompt_caching.rs:222
    - Resume with model switch keeps original instructions:
    core/tests/suite/resume.rs:136
    - Compact/resume/fork uses request 0 instructions for later expected
    payloads: core/tests/suite/compact_resume_fork.rs:215
  • Act on reasoning-included per turn (#9402)
    - Reset reasoning-included flag each turn and update compaction test
  • Feat: request user input tool (#9472)
    ### Summary
    * Add `requestUserInput` tool that the model can use for gather
    feedback/asking question mid turn.
    
    
    ### Tool input schema
    ```
    {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "title": "requestUserInput input",
      "type": "object",
      "additionalProperties": false,
      "required": ["questions"],
      "properties": {
        "questions": {
          "type": "array",
          "description": "Questions to show the user (1-3). Prefer 1 unless multiple independent decisions block progress.",
          "minItems": 1,
          "maxItems": 3,
          "items": {
            "type": "object",
            "additionalProperties": false,
            "required": ["id", "header", "question"],
            "properties": {
              "id": {
                "type": "string",
                "description": "Stable identifier for mapping answers (snake_case)."
              },
              "header": {
                "type": "string",
                "description": "Short header label shown in the UI (12 or fewer chars)."
              },
              "question": {
                "type": "string",
                "description": "Single-sentence prompt shown to the user."
              },
              "options": {
                "type": "array",
                "description": "Optional 2-3 mutually exclusive choices. Put the recommended option first and suffix its label with \"(Recommended)\". Only include \"Other\" option if we want to include a free form option. If the question is free form in nature, do not include any option.",
                "minItems": 2,
                "maxItems": 3,
                "items": {
                  "type": "object",
                  "additionalProperties": false,
                  "required": ["value", "label", "description"],
                  "properties": {
                    "value": {
                      "type": "string",
                      "description": "Machine-readable value (snake_case)."
                    },
                    "label": {
                      "type": "string",
                      "description": "User-facing label (1-5 words)."
                    },
                    "description": {
                      "type": "string",
                      "description": "One short sentence explaining impact/tradeoff if selected."
                    }
                  }
                }
              }
            }
          }
        }
      }
    }
    ```
    
    ### Tool output schema
    ```
    {
      "$schema": "http://json-schema.org/draft-07/schema#",
      "title": "requestUserInput output",
      "type": "object",
      "additionalProperties": false,
      "required": ["answers"],
      "properties": {
        "answers": {
          "type": "object",
          "description": "Map of question id to user answer.",
          "additionalProperties": {
            "type": "object",
            "additionalProperties": false,
            "required": ["selected"],
            "properties": {
              "selected": {
                "type": "array",
                "items": { "type": "string" }
              },
              "other": {
                "type": ["string", "null"]
              }
            }
          }
        }
      }
    }
    ```
  • Add collaboration developer instructions (#9424)
    - Add additional instructions when they are available
    - Make sure to update them on change either UserInput or UserTurn
  • chore(instructions) Remove unread SessionMeta.instructions field (#9423)
    ### Description
    - Remove the now-unused `instructions` field from the session metadata
    to simplify SessionMeta and stop propagating transient instruction text
    through the rollout recorder API. This was only saving
    user_instructions, and was never being read.
    - Stop passing user instructions into the rollout writer at session
    creation so the rollout header only contains canonical session metadata.
    
    ### Testing
    
    - Ran `just fmt` which completed successfully.
    - Ran `just fix -p codex-protocol`, `just fix -p codex-core`, `just fix
    -p codex-app-server`, `just fix -p codex-tui`, and `just fix -p
    codex-tui2` which completed (Clippy fixes applied) as part of
    verification.
    - Ran `cargo test -p codex-protocol` which passed (28 tests).
    - Ran `cargo test -p codex-core` which showed failures in a small set of
    tests (not caused by the protocol type change directly):
    `default_client::tests::test_create_client_sets_default_headers`,
    several `models_manager::manager::tests::refresh_available_models_*`,
    and `shell_snapshot::tests::linux_sh_snapshot_includes_sections` (these
    tests failed in this CI run).
    - Ran `cargo test -p codex-app-server` which reported several failing
    integration tests (including
    `suite::codex_message_processor_flow::test_codex_jsonrpc_conversation_flow`,
    `suite::output_schema::send_user_turn_*`, and
    `suite::user_agent::get_user_agent_returns_current_codex_user_agent`).
    - `cargo test -p codex-tui` and `cargo test -p codex-tui2` were
    attempted but aborted due to disk space exhaustion (`No space left on
    device`).
    
    ------
    [Codex
    Task](https://chatgpt.com/codex/tasks/task_i_696bd8ce632483228d298cf07c7eb41c)
  • chore(core) Create instructions module (#9422)
    ## Summary
    We have a variety of things we refer to as instructions in the code
    base: our current canonical terms are:
    - base instructions (raw string)
    - developer instructions (has a type in protocol)
    - user instructions
    
    We also have `instructions` floating around in various places. We should
    standardize on the above, and start using types to prevent them from
    ending up in the wrong place. There will be additional PRs, but I'm
    going to keep these small so we can easily follow them!
    
    ## Testing
    - [x] Tests pass, this is purely a file move
  • Support enable/disable skill via config/api. (#9328)
    In config.toml:
    ```
    [[skills.config]]
    path = "/Users/xl/.codex/skills/my_skill/SKILL.md"
    enabled = false
    ```
    
    API:
    skills/list, skills/config/write
  • Introduce collaboration modes (#9340)
    - Merge `model` and `reasoning_effort` under collaboration modes.
    - Add additional instructions for custom collaboration mode
    - Default to Custom to not change behavior
  • feat: show forked from session id in /status (#9330)
    Summary:
    - Add forked_from to SessionMeta/SessionConfiguredEvent and persist it
    for forked sessions.
    - Surface forked_from in /status for tui + tui2 and add snapshots.
  • rename model turn to sampling request (#9336)
    We have two type of turns now: model and user turns. It's always
    confusing to refer to either. Model turn is basically a sampling
    request.
  • Add text element metadata to protocol, app server, and core (#9331)
    The second part of breaking up PR
    https://github.com/openai/codex/pull/9116
    
    Summary:
    
    - Add `TextElement` / `ByteRange` to protocol user inputs and user
    message events with defaults.
    - Thread `text_elements` through app-server v1/v2 request handling and
    history rebuild.
    - Preserve UI metadata only in user input/events (not `ContentItem`)
    while keeping local image attachments in user events for rehydration.
    
    Details:
    
    - Protocol: `UserInput::Text` carries `text_elements`;
    `UserMessageEvent` carries `text_elements` + `local_images`.
    Serialization includes empty vectors for backward compatibility.
    - app-server-protocol: v1 defines `V1TextElement` / `V1ByteRange` in
    camelCase with conversions; v2 uses its own camelCase wrapper.
    - app-server: v1/v2 input mapping includes `text_elements`; thread
    history rebuilds include them.
    - Core: user event emission preserves UI metadata while model history
    stays clean; history replay round-trips the metadata.