Commit Graph

540 Commits

  • chore(config) Rename config setting to personality (#10314)
    ## Summary
    Let's make the setting name consistent with the SlashCommand!
    
    ## Testing
    - [x] Updated tests
  • Add websocket telemetry metrics and labels (#10316)
    Summary
    - expose websocket telemetry hooks through the responses client so
    request durations and event processing can be reported
    - record websocket request/event metrics and emit runtime telemetry
    events that the history UI now surfaces
    - improve tests to cover websocket telemetry reporting and guard runtime
    summary updates
    
    
    <img width="824" height="79" alt="Screenshot 2026-01-31 at 5 28 12 PM"
    src="https://github.com/user-attachments/assets/ea9a7965-d8b4-4e3c-a984-ef4fdc44c81d"
    />
  • feat(core,tui,app-server) personality migration (#10307)
    ## Summary
    Keep existing users on Pragmatic, to preserve behavior while new users
    default to Friendly
    
    ## Testing
    - [x] Tested locally
    - [x] add integration tests
  • chore(core) Default to friendly personality (#10305)
    ## Summary
    Update default personality to friendly
    
    ## Testing
    - [x] Unit tests pass
  • feat(core) Smart approvals on (#10286)
    ## Summary
    Turn on Smart Approvals by default
    
    ## Testing
     - [x] Updated unit tests
  • add missing fields to WebSearchAction and update app-server types (#10276)
    - add `WebSearchAction` to app-server v2 types
    - add `queries` to `WebSearchAction::Search` type
    
    Updated tests.
  • fix: dont auto-enable web_search for azure (#10266)
    seeing issues with azure after default-enabling web search: #10071,
    #10257.
    
    need to work with azure to fix api-side, for now turning off
    default-enable of web_search for azure.
    
    diff is big because i moved logic to reuse
  • chore: rename ChatGpt -> Chatgpt in type names (#10244)
    When using ChatGPT in names of types, we should be consistent, so this
    renames some types with `ChatGpt` in the name to `Chatgpt`. From
    https://rust-lang.github.io/api-guidelines/naming.html:
    
    > In `UpperCamelCase`, acronyms and contractions of compound words count
    as one word: use `Uuid` rather than `UUID`, `Usize` rather than `USize`
    or `Stdin` rather than `StdIn`. In `snake_case`, acronyms and
    contractions are lower-cased: `is_xid_start`.
    
    This PR updates existing uses of `ChatGpt` and changes them to
    `Chatgpt`. Though in all cases where it could affect the wire format, I
    visually inspected that we don't change anything there. That said, this
    _will_ change the codegen because it will affect the spelling of type
    names.
    
    For example, this renames `AuthMode::ChatGPT` to `AuthMode::Chatgpt` in
    `app-server-protocol`, but the wire format is still `"chatgpt"`.
    
    This PR also updates a number of types in `codex-rs/core/src/auth.rs`.
  • Plan mode: stream proposed plans, emit plan items, and render in TUI (#9786)
    ## Summary
    - Stream proposed plans in Plan Mode using `<proposed_plan>` tags parsed
    in core, emitting plan deltas plus a plan `ThreadItem`, while stripping
    tags from normal assistant output.
    - Persist plan items and rebuild them on resume so proposed plans show
    in thread history.
    - Wire plan items/deltas through app-server protocol v2 and render a
    dedicated proposed-plan view in the TUI, including the “Implement this
    plan?” prompt only when a plan item is present.
    
    ## Changes
    
    ### Core (`codex-rs/core`)
    - Added a generic, line-based tag parser that buffers each line until it
    can disprove a tag prefix; implements auto-close on `finish()` for
    unterminated tags. `codex-rs/core/src/tagged_block_parser.rs`
    - Refactored proposed plan parsing to wrap the generic parser.
    `codex-rs/core/src/proposed_plan_parser.rs`
    - In plan mode, stream assistant deltas as:
      - **Normal text** → `AgentMessageContentDelta`
      - **Plan text** → `PlanDelta` + `TurnItem::Plan` start/completion  
      (`codex-rs/core/src/codex.rs`)
    - Final plan item content is derived from the completed assistant
    message (authoritative), not necessarily the concatenated deltas.
    - Strips `<proposed_plan>` blocks from assistant text in plan mode so
    tags don’t appear in normal messages.
    (`codex-rs/core/src/stream_events_utils.rs`)
    - Persist `ItemCompleted` events only for plan items for rollout replay.
    (`codex-rs/core/src/rollout/policy.rs`)
    - Guard `update_plan` tool in Plan Mode with a clear error message.
    (`codex-rs/core/src/tools/handlers/plan.rs`)
    - Updated Plan Mode prompt to:  
      - keep `<proposed_plan>` out of non-final reasoning/preambles  
      - require exact tag formatting  
      - allow only one `<proposed_plan>` block per turn  
      (`codex-rs/core/templates/collaboration_mode/plan.md`)
    
    ### Protocol / App-server protocol
    - Added `TurnItem::Plan` and `PlanDeltaEvent` to core protocol items.
    (`codex-rs/protocol/src/items.rs`, `codex-rs/protocol/src/protocol.rs`)
    - Added v2 `ThreadItem::Plan` and `PlanDeltaNotification` with
    EXPERIMENTAL markers and note that deltas may not match the final plan
    item. (`codex-rs/app-server-protocol/src/protocol/v2.rs`)
    - Added plan delta route in app-server protocol common mapping.
    (`codex-rs/app-server-protocol/src/protocol/common.rs`)
    - Rebuild plan items from persisted `ItemCompleted` events on resume.
    (`codex-rs/app-server-protocol/src/protocol/thread_history.rs`)
    
    ### App-server
    - Forward plan deltas to v2 clients and map core plan items to v2 plan
    items. (`codex-rs/app-server/src/bespoke_event_handling.rs`,
    `codex-rs/app-server/src/codex_message_processor.rs`)
    - Added v2 plan item tests.
    (`codex-rs/app-server/tests/suite/v2/plan_item.rs`)
    
    ### TUI
    - Added a dedicated proposed plan history cell with special background
    and padding, and moved “• Proposed Plan” outside the highlighted block.
    (`codex-rs/tui/src/history_cell.rs`, `codex-rs/tui/src/style.rs`)
    - Only show “Implement this plan?” when a plan item exists.
    (`codex-rs/tui/src/chatwidget.rs`,
    `codex-rs/tui/src/chatwidget/tests.rs`)
    
    <img width="831" height="847" alt="Screenshot 2026-01-29 at 7 06 24 PM"
    src="https://github.com/user-attachments/assets/69794c8c-f96b-4d36-92ef-c1f5c3a8f286"
    />
    
    ### Docs / Misc
    - Updated protocol docs to mention plan deltas.
    (`codex-rs/docs/protocol_v1.md`)
    - Minor plumbing updates in exec/debug clients to tolerate plan deltas.
    (`codex-rs/debug-client/src/reader.rs`, `codex-rs/exec/...`)
    
    ## Tests
    - Added core integration tests:
      - Plan mode strips plan from agent messages.
      - Missing `</proposed_plan>` closes at end-of-message.  
      (`codex-rs/core/tests/suite/items.rs`)
    - Added unit tests for generic tag parser (prefix buffering, non-tag
    lines, auto-close). (`codex-rs/core/src/tagged_block_parser.rs`)
    - Existing app-server plan item tests in v2.
    (`codex-rs/app-server/tests/suite/v2/plan_item.rs`)
    
    ## Notes / Behavior
    - Plan output no longer appears in standard assistant text in Plan Mode;
    it streams via `PlanDelta` and completes as a `TurnItem::Plan`.
    - The final plan item content is authoritative and may diverge from
    streamed deltas (documented as experimental).
    - Reasoning summaries are not filtered; prompt instructs the model not
    to include `<proposed_plan>` outside the final plan message.
    
    ## Codex Author
    `codex fork 019bec2d-b09d-7450-b292-d7bcdddcdbfb`
  • Conversation naming (#8991)
    Session renaming:
    - `/rename my_session`
    - `/rename` without arg and passing an argument in `customViewPrompt`
    - AppExitInfo shows resume hint using the session name if set instead of
    uuid, defaults to uuid if not set
    - Names are stored in `CODEX_HOME/sessions.jsonl`
    
    Session resuming:
    - codex resume <name> lookup for `CODEX_HOME/sessions.jsonl` first entry
    matching the name and resumes the session
    
    ---------
    
    Co-authored-by: jif-oai <jif@openai.com>
  • Chore: plan mode do not include free form question and always include isOther (#10210)
    We should never ask a freeform question when planning and we should
    always include isOther as an escape hatch.
  • chore(personality) new schema with fallbacks (#10147)
    ## Summary
    Let's dial in this api contract in a bit more with more robust fallback
    behavior when model_instructions_template is false.
    
    Switches to a more explicit template / variables structure, with more
    fallbacks.
    
    ## Testing
    - [x] Adding unit tests
    - [x] Tested locally
  • [feat] persist dynamic tools in session rollout file (#10130)
    Add dynamic tools to rollout file for persistence & read from rollout on
    resume. Ran a real example and spotted the following in the rollout
    file:
    ```
    {"timestamp":"2026-01-29T01:27:57.468Z","type":"session_meta","payload":{"id":"019c075d-3f0b-77e3-894e-c1c159b04b1e","timestamp":"2026-01-29T01:27:57.451Z","...."dynamic_tools":[{"name":"demo_tool","description":"Demo dynamic tool","inputSchema":{"additionalProperties":false,"properties":{"city":{"type":"string"}},"required":["city"],"type":"object"}}],"git":{"commit_hash":"ebc573f15c01b8af158e060cfedd401f043e9dfa","branch":"dev/cc/dynamic-tools","repository_url":"https://github.com/openai/codex.git"}}}
    ```
  • feat(app-server): support external auth mode (#10012)
    This enables a new use case where `codex app-server` is embedded into a
    parent application that will directly own the user's ChatGPT auth
    lifecycle, which means it owns the user’s auth tokens and refreshes it
    when necessary. The parent application would just want a way to pass in
    the auth tokens for codex to use directly.
    
    The idea is that we are introducing a new "auth mode" currently only
    exposed via app server: **`chatgptAuthTokens`** which consist of the
    `id_token` (stores account metadata) and `access_token` (the bearer
    token used directly for backend API calls). These auth tokens are only
    stored in-memory. This new mode is in addition to the existing `apiKey`
    and `chatgpt` auth modes.
    
    This PR reuses the shape of our existing app-server account APIs as much
    as possible:
    - Update `account/login/start` with a new `chatgptAuthTokens` variant,
    which will allow the client to pass in the tokens and have codex
    app-server use them directly. Upon success, the server emits
    `account/login/completed` and `account/updated` notifications.
    - A new server->client request called
    `account/chatgptAuthTokens/refresh` which the server can use whenever
    the access token previously passed in has expired and it needs a new one
    from the parent application.
    
    I leveraged the core 401 retry loop which typically triggers auth token
    refreshes automatically, but made it pluggable:
    - **chatgpt** mode refreshes internally, as usual.
    - **chatgptAuthTokens** mode calls the client via
    `account/chatgptAuthTokens/refresh`, the client responds with updated
    tokens, codex updates its in-memory auth, then retries. This RPC has a
    10s timeout and handles JSON-RPC errors from the client.
    
    Also some additional things:
    - chatgpt logins are blocked while external auth is active (have to log
    out first. typically clients will pick one OR the other, not support
    both)
    - `account/logout` clears external auth in memory
    - Ensures that if `forced_chatgpt_workspace_id` is set via the user's
    config, we respect it in both:
    - `account/login/start` with `chatgptAuthTokens` (returns a JSON-RPC
    error back to the client)
    - `account/chatgptAuthTokens/refresh` (fails the turn, and on next
    request app-server will send another `account/chatgptAuthTokens/refresh`
    request to the client).
  • Remove WebSocket wire format (#10179)
    I'd like WireApi to go away (when chat is removed) and WebSockets is
    still responses API just over a different transport.
  • Fall back to http when websockets fail (#10139)
    I expect not all proxies work with websockets, fall back to http if
    websockets fail.
  • chore: unify log queries (#10152)
    Unify log queries to only have SQLX code in the runtime and use it for
    both the log client and for tests
  • Ensure auto-compaction starts after turn started (#10129)
    Start auto-compaction only after TurnStarted is emitted.\nAdd an
    integration test for deterministic ordering.
  • [bazel] Improve runfiles handling (#10098)
    we can't use runfiles directory on Windows due to path lengths, so swap
    to manifest strategy. Parsing the manifest is a bit complex and the
    format is changing in Bazel upstream, so pull in the official Rust
    library (via a small hack to make it importable...) and cleanup all the
    associated logic to work cleanly in both bazel and cargo without extra
    confusion
  • default enable compression, update test helpers (#10102)
    set `enable_request_compression` flag to default-enabled.
    
    update integration test helpers to decompress `zstd` if flag set.
  • compaction (#10034)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • chore: deprecate old web search feature flags (#10097)
    deprecate all old web search flags and aliases, including:
    - `[features].web_search_request` and `[features].web_search_cached`
    - `[tools].web_search`
    - `[features].web_search`
    
    slightly rework `legacy_usages` to enable pointing to non-features from
    deprecated features; we need to point to `web_search` (not under
    `[features]`) from things like `[features].web_search_cached` and
    `[features].web_search_request`.
    
    Added integration tests to confirm deprecation notice is shown on
    explicit enablement and disablement of deprecated flags.
  • feat: sqlite 1 (#10004)
    Add a `.sqlite` database to be used to store rollout metatdata (and
    later logs)
    This PR is phase 1:
    * Add the database and the required infrastructure
    * Add a backfill of the database
    * Persist the newly created rollout both in files and in the DB
    * When we need to get metadata or a rollout, consider the `JSONL` as the
    source of truth but compare the results with the DB and show any errors
  • feat(core) RequestRule (#9489)
    ## Summary
    Instead of trying to derive the prefix_rule for a command mechanically,
    let's let the model decide for us.
    
    ## Testing
    - [x] tested locally
  • fix: enable per-turn updates to web search mode (#10040)
    web_search can now be updated per-turn, for things like changes to
    sandbox policy.
    
    `SandboxPolicy::DangerFullAccess` now sets web_search to `live`, and the
    default is still `cached`.
    
    Added integration tests.
  • remove sandbox globals. (#9797)
    Threads sandbox updates through OverrideTurnContext for active turn
    Passes computed sandbox type into safety/exec
  • make cached web_search client-side default (#9974)
    [Experiment](https://console.statsig.com/50aWbk2p4R76rNX9lN5VUw/experiments/codex_web_search_rollout/summary)
    for default cached `web_search` completed; cached chosen as default.
    
    Update client to reflect that.
  • fix: handle all web_search actions and in progress invocations (#9960)
    ### Summary
    - Parse all `web_search` tool actions (`search`, `find_in_page`,
    `open_page`).
    - Previously we only parsed + displayed `search`, which made the TUI
    appear to pause when the other actions were being used.
    - Show in progress `web_search` calls as `Searching the web`
      - Previously we only showed completed tool calls
    
    <img width="308" height="149" alt="image"
    src="https://github.com/user-attachments/assets/90a4e8ff-b06a-48ff-a282-b57b31121845"
    />
    
    ### Tests
    Added + updated tests, tested locally
    
    ### Follow ups
    Update VSCode extension to display these as well
  • Use test_codex more (#9961)
    Reduces boilderplate.
  • Reject request_user_input outside Plan/Pair (#9955)
    ## Context
    
    Previous work in https://github.com/openai/codex/pull/9560 only rejected
    `request_user_input` in Execute and Custom modes. Since then, additional
    modes
    (e.g., Code) were added, so the guard should be mode-agnostic.
    
    ## What changed
    
    - Switch the handler to an allowlist: only Plan and PairProgramming are
    allowed
    - Return the same error for any other mode (including Code)
    - Add a Code-mode rejection test alongside the existing Execute/Custom
    tests
    
    ## Why
    
    This prevents `request_user_input` from being used in modes where it is
    not
    intended, even as new modes are introduced.
  • Add MCP server scopes config and use it as fallback for OAuth login (#9647)
    ### Motivation
    - Allow MCP OAuth flows to request scopes defined in `config.toml`
    instead of requiring users to always pass `--scopes` on the CLI.
    CLI/remote parameters should still override config values.
    
    ### Description
    - Add optional `scopes: Option<Vec<String>>` to `McpServerConfig` and
    `RawMcpServerConfig`, and propagate it through deserialization and the
    built config types.
    - Serialize `scopes` into the MCP server TOML via
    `serialize_mcp_server_table` in `core/src/config/edit.rs` and include
    `scopes` in the generated config schema (`core/config.schema.json`).
    - CLI: update `codex-rs/cli/src/mcp_cmd.rs` `run_login` to fall back to
    `server.scopes` when the `--scopes` flag is empty, with explicit CLI
    scopes still taking precedence.
    - App server: update
    `codex-rs/app-server/src/codex_message_processor.rs`
    `mcp_server_oauth_login` to use `params.scopes.or_else(||
    server.scopes.clone())` so the RPC path also respects configured scopes.
    - Update many test fixtures to initialize the new `scopes` field (set to
    `None`) so test code builds with the new struct field.
    
    ### Testing
    - Ran config tooling and formatters: `just write-config-schema`
    (succeeded), `just fmt` (succeeded), and `just fix -p codex-core`, `just
    fix -p codex-cli`, `just fix -p codex-app-server` (succeeded where
    applicable).
    - Ran unit tests for the CLI: `cargo test -p codex-cli` (passed).
    - Ran unit tests for core: `cargo test -p codex-core` (ran; many tests
    passed but several failed, including model refresh/403-related tests,
    shell snapshot/timeouts, and several `unified_exec` expectations).
    - Ran app-server tests: `cargo test -p codex-app-server` (ran; many
    integration-suite tests failed due to mocked/remote HTTP 401/403
    responses and wiremock expectations).
    
    If you want, I can split the tests into smaller focused runs or help
    debug the failing integration tests (they appear to be unrelated to the
    config change and stem from external HTTP/mocking behaviors encountered
    during the test runs).
    
    ------
    [Codex
    Task](https://chatgpt.com/codex/tasks/task_i_69718f505914832ea1f334b3ba064553)
  • Add thread/unarchive to restore archived rollouts (#9843)
    ## Summary
    - Adds a new `thread/unarchive` RPC to move archived thread rollouts
    back into the active `sessions/` tree.
    
    ## What changed
    - **Protocol**
      - Adds `thread/unarchive` request/response types and wiring.
    - **Server**
      - Implements `thread_unarchive` in the app server.
      - Validates the archived rollout path and thread ID.
    - Restores the rollout to `sessions/YYYY/MM/DD/...` based on the rollout
    filename timestamp.
    - **Core**
    - Adds `find_archived_thread_path_by_id_str` helper for archived
    rollouts.
    - **Docs**
      - Documents the new RPC and usage example.
    - **Tests**
      - Adds an end-to-end server test that:
        1) starts a thread,
        2) archives it,
        3) unarchives it,
        4) asserts the file is restored to `sessions/`.
    
    ## How to use
    ```json
    { "method": "thread/unarchive", "id": 24, "params": { "threadId": "<thread-id>" } }
    ```
    
    ## Author Codex Session
    
    `codex resume 019bf158-54b6-7960-a696-9d85df7e1bc1` (soon I'll make this
    kind of session UUID forkable by anyone with the right
    `session_object_storage_url` line in their config, but for now just
    pasting it here for my reference)
  • Feat: add isOther to question returned by request user input tool (#9890)
    ### Summary
    Add `isOther` to question object from request_user_input tool input and
    remove `other` option from the tool prompt to better handle tool input.
  • chore(core) move model_instructions_template config (#9871)
    ## Summary
    Move `model_instructions_template` config to the experimental slug while
    we iterate on this feature
    
    ## Testing
    - [x] Tested locally, unit tests still pass
  • feat(tui) /personality (#9718)
    ## Summary
    Adds /personality selector in the TUI, which leverages the new core
    interface in #9644
    
    Notes:
    - We are doing some of our own state management for model_info loading
    here, but not sure if that's ideal. open to opinions on simpler
    approach, but would like to avoid blocking on a larger refactor
    - Right now, the `/personality` selector just hides when the model
    doesn't support it. we can update this behavior down the line
    
    ## Testing
    - [x] Tested locally
    - [x] Added snapshot tests
  • feat: ephemeral threads (#9765)
    Add ephemeral threads capabilities. Only exposed through the
    `app-server` v2
    
    The idea is to disable the rollout recorder for those threads.
  • change collaboration mode to struct (#9793)
    Shouldn't cause behavioral change
  • feat: support proxy for ws connection (#9719)
    reapply websocket changes without changing tls lib.
  • feat(core) update Personality on turn (#9644)
    ## Summary
    Support updating Personality mid-Thread via UserTurn/OverwriteTurn. This
    is explicitly unused by the clients so far, to simplify PRs - app-server
    and tui implementations will be follow-ups.
    
    ## Testing
    - [x] added integration tests
  • Support end_turn flag (#9698)
    Experimental flag that signals the end of the turn.
  • feat(core) ModelInfo.model_instructions_template (#9597)
    ## Summary
    #9555 is the start of a rename, so I'm starting to standardize here.
    Sets up `model_instructions` templating with a strongly-typed object for
    injecting a personality block into the model instructions.
    
    ## Testing
    - [x] Added tests
    - [x] Ran locally
  • Reduce burst testing flake (#9549)
    ## Summary
    
    - make paste-burst tests deterministic by injecting explicit timestamps
    instead of relying on wall clock timing
    - add time-aware helpers for input/submission paths so tests can drive
    the burst heuristic precisely
    - update burst-related tests to flush using computed timeouts while
    preserving behavior assertions
    - increase timeout slack in
    shell_tools_start_before_response_completed_when_stream_delayed to
    reduce flakiness
  • Add collaboration_mode to TurnContextItem (#9583)
    ## Summary
    - add optional `collaboration_mode` to `TurnContextItem` in rollouts
    - persist the current collaboration mode when recording turn context
    (sampling + compaction)
    
    ## Rationale
    We already persist turn context data for resume logic. Capturing
    collaboration mode in the rollout gives us the mode context for each
    turn, enabling follow‑up work to diff mode instructions correctly on
    resume.
    
    ## Changes
    - protocol: add optional `collaboration_mode` field to `TurnContextItem`
    - core: persist collaboration mode alongside other turn context settings
    in rollouts
  • Chore: update plan mode output in prompt (#9592)
    ### Summary
    * Update plan prompt output
    * Update requestUserInput response to be a single key value pair
    `answer: String`.
  • chore: defensive shell snapshot (#9609)
    This PR adds 2 defensive mechanisms for shell snapshotting:
    * Filter out invalid env variables (containing `-` for example) without
    dropping the whole snapshot
    * Validate the snapshot before considering it as valid by running a mock
    command with a shell snapshot