Commit Graph

1279 Commits

  • feat: replace custom mcp-types crate with equivalents from rmcp (#10349)
    We started working with MCP in Codex before
    https://crates.io/crates/rmcp was mature, so we had our own crate for
    MCP types that was generated from the MCP schema:
    
    
    https://github.com/openai/codex/blob/8b95d3e082376f4cb23e92641705a22afb28a9da/codex-rs/mcp-types/README.md
    
    Now that `rmcp` is more mature, it makes more sense to use their MCP
    types in Rust, as they handle details (like the `_meta` field) that our
    custom version ignored. Though one advantage that our custom types had
    is that our generated types implemented `JsonSchema` and `ts_rs::TS`,
    whereas the types in `rmcp` do not. As such, part of the work of this PR
    is leveraging the adapters between `rmcp` types and the serializable
    types that are API for us (app server and MCP) introduced in #10356.
    
    Note this PR results in a number of changes to
    `codex-rs/app-server-protocol/schema`, which merit special attention
    during review. We must ensure that these changes are still
    backwards-compatible, which is possible because we have:
    
    ```diff
    - export type CallToolResult = { content: Array<ContentBlock>, isError?: boolean, structuredContent?: JsonValue, };
    + export type CallToolResult = { content: Array<JsonValue>, structuredContent?: JsonValue, isError?: boolean, _meta?: JsonValue, };
    ```
    
    so `ContentBlock` has been replaced with the more general `JsonValue`.
    Note that `ContentBlock` was defined as:
    
    ```typescript
    export type ContentBlock = TextContent | ImageContent | AudioContent | ResourceLink | EmbeddedResource;
    ```
    
    so the deletion of those individual variants should not be a cause of
    great concern.
    
    Similarly, we have the following change in
    `codex-rs/app-server-protocol/schema/typescript/Tool.ts`:
    
    ```
    - export type Tool = { annotations?: ToolAnnotations, description?: string, inputSchema: ToolInputSchema, name: string, outputSchema?: ToolOutputSchema, title?: string, };
    + export type Tool = { name: string, title?: string, description?: string, inputSchema: JsonValue, outputSchema?: JsonValue, annotations?: JsonValue, icons?: Array<JsonValue>, _meta?: JsonValue, };
    ```
    
    so:
    
    - `annotations?: ToolAnnotations` ➡️ `JsonValue`
    - `inputSchema: ToolInputSchema` ➡️ `JsonValue`
    - `outputSchema?: ToolOutputSchema` ➡️ `JsonValue`
    
    and two new fields: `icons?: Array<JsonValue>, _meta?: JsonValue`
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/10349).
    * #10357
    * __->__ #10349
    * #10356
  • make codex better at git (#10145)
    adds basic git context to the session prefix so the model can anchor git
    actions and be a bit more version-aware. structured it in a
    multiroot-friendly shape even though we only have one root today
  • feat: Read personal skills from .agents/skills (#10437)
    - Issue: https://github.com/agentskills/agentskills/issues/15
    - Follow-up to https://github.com/openai/codex/pull/10317 (for team/repo
    skills)
    - This change now also loads personal/user skills from
    `$HOME/.agents/skills` (or `~/.agents/skills`) in addition to loading
    from `.agents/skills` inside of git repos.
    - The location of `.system` skills remains unchanged.
    - Keeping backwards compatibility with `~/.codex/skills` for now until
    we fully deprecate.
    
    With skills in both personal folders:
    <img width="831" height="421" alt="image"
    src="https://github.com/user-attachments/assets/ad8ac918-bfe6-4a2d-8a8e-d608c9d3d701"
    />
    
    We load from both places:
    <img width="607" height="236" alt="image"
    src="https://github.com/user-attachments/assets/480f4db0-ae64-4dc1-bdf5-c5de98c16f5c"
    />
  • [feat] persist thread_dynamic_tools in db (#10252)
    Persist thread_dynamic_tools in sqlite and read first from it. Fall back
    to rollout files if it's not found. Persist dynamic tools to both sqlite
    and rollout files.
    
    Saw that new sessions get populated to db correctly & old sessions get
    backfilled correctly at startup:
    ```
    celia@com-92114 codex-rs % sqlite3 ~/.codex/state.sqlite \      "select thread_id, position,name,description,input_schema from thread_dynamic_tools;"
    019c0cad-ec0d-74b2-a787-e8b33a349117|0|geo_lookup|lookup a city|{"properties":{"city":{"type":"string"}},"required":["city"],"type":"object"}
    ....
    019c10ca-aa4b-7620-ae40-c0919fbd7ea7|0|geo_lookup|lookup a city|{"properties":{"city":{"type":"string"}},"required":["city"],"type":"object"}
    ```
  • emit a separate metric when the user cancels UAT during elevated setup (#10399)
    Currently this shows up as elevated setup failure, which isn't quite
    accurate.
  • fix: unsafe auto-approval of git commands (#10258)
    fixes https://github.com/openai/codex/issues/10160 and some more.
    
    ## Description
    
    Hardens Git command safety to prevent approval bypasses for destructive
    or write-capable invocations (branch delete, risky push forms,
    output/config-override flags), so these commands no longer auto-run as
    “safe.”
    
    - `git branch -d` variants (especially in worktrees / with global
    options like -C / -c)
    - `git show|diff|log --output` ... style file-write flags
    - risky Git config override flags (-c, --config-env) that can trigger
    external execution
    - dangerous push forms that weren’t fully caught (`--force*`,
    `--delete`, `+refspec`, `:refspec`)
    - grouped short-flag delete forms (e.g. stacked branch flags containing
    `d/D`)
    
    will fast follow with a common git policy to bring windows to parity.
    
    ---------
    
    Co-authored-by: Eric Traut <etraut@openai.com>
  • Session picker shows thread_name if set (#10340)
    - shows names of threads in the ResumePicker used by `/resume` and
    `codex resume` if set, default to preview (previous behaviour) if none
    - adds a `find_thread_names_by_ids` that maps names to IDs in
    `codex-rs/core/src/rollout/session_index.rs`. It reads sequentially in
    normal (instead of reverse order in `codex resume <name>`) the index
    mapping file. This function is called from a list of session (default
    page is 25, pages loaded depends of height of terminal), for which most
    of them will always have at least one session unnamed and require the
    whole file to be read therefore. Could be better and sqlite integration
    will make this better
    - those reads won't be needed when leveraging sqlite
     
    
    Opened questions:
    - We could rename the TUI "Conversation" column to "Name" or "Thread"
    that would feel more accurate. Could be a fast-follow if we implement
    auto-naming as it'll always be a name instead?
  • fix(rules) Limit rules listed in conversation (#10351)
    ## Summary
    We should probably warn users that they have a million rules, and help
    clean them up. But for now, we should handle this unbounded case.
    
    Limit rules listed in conversations, with shortest / broadest rules
    first.
    
    ## Testing
    - [x] Updated unit tests
  • fix: System skills marker includes nested folders recursively (#10350)
    Updated system skills bundled with Codex were not correctly replacing
    the user's skills in their .system folder.
    
    - Fix `.codex-system-skills.marker` not updating by hashing embedded
    system skills recursively (nested dirs + file contents), so updates
    trigger a reinstall.
    - Added a build Cargo hook to rerun if there are changes in
    `src/skills/assets/samples/*`, ensuring embedded skill updates rebuild
    correctly under caching.
    - Add a small unit test to ensure nested entries are included in the
    fingerprint.
  • chore(features) Personality => Stable (#10310)
    ## Summary
    Bump `/personality` to stable
    
    ## Testing
     - [x] unit tests pass
  • chore(config) Rename config setting to personality (#10314)
    ## Summary
    Let's make the setting name consistent with the SlashCommand!
    
    ## Testing
    - [x] Updated tests
  • Add websocket telemetry metrics and labels (#10316)
    Summary
    - expose websocket telemetry hooks through the responses client so
    request durations and event processing can be reported
    - record websocket request/event metrics and emit runtime telemetry
    events that the history UI now surfaces
    - improve tests to cover websocket telemetry reporting and guard runtime
    summary updates
    
    
    <img width="824" height="79" alt="Screenshot 2026-01-31 at 5 28 12 PM"
    src="https://github.com/user-attachments/assets/ea9a7965-d8b4-4e3c-a984-ef4fdc44c81d"
    />
  • Make skills prompt explicit about relative-path lookup (#10282)
    Fix cases where the model tries to locate skill scripts from the cwd and
    fails.
  • feat: Support loading skills from .agents/skills (#10317)
    This PR adds support for loading
    [skills](https://developers.openai.com/codex/skills) from
    `.agents/skills/`.
    - Issue: https://github.com/agentskills/agentskills/issues/15
    - Motivation: When skills live on the filesystem, sharing them across
    agents is awkward and often ends up requiring symlinks/duplication. A
    single location under `.agents/` makes it easier to share skills.
    - Loading from `.codex/skills/` will remain but will be deprecated soon.
    The change only applies to the [REPO
    scope](https://developers.openai.com/codex/skills#where-to-save-skills).
    - Documentation will be updated before this change is live.
    
    Testing with skills in two locations of this repo:
    <img width="960" height="152" alt="image"
    src="https://github.com/user-attachments/assets/28975ff9-7363-46dd-ad40-f4c7bfdb8234"
    />
    
    When starting Codex with CWD in `$repo_root` (should only pick up at
    root):
    <img width="513" height="143" alt="image"
    src="https://github.com/user-attachments/assets/389e1ea7-020c-481e-bda0-ce58562db59f"
    />
    
    When starting Codex with CWD in `$repo_root/codex-rs` (should pick up at
    cwd and crawl up to root):
    <img width="552" height="177" alt="image"
    src="https://github.com/user-attachments/assets/a5beb8de-11b4-45ed-8660-80707c77006a"
    />
  • enable plan mode (#10313)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • feat(core,tui,app-server) personality migration (#10307)
    ## Summary
    Keep existing users on Pragmatic, to preserve behavior while new users
    default to Friendly
    
    ## Testing
    - [x] Tested locally
    - [x] add integration tests
  • chore(core) Default to friendly personality (#10305)
    ## Summary
    Update default personality to friendly
    
    ## Testing
    - [x] Unit tests pass
  • chore(features) remove Experimental tag from UTF8 (#10296)
    ## Summary
    This has been default on for some time, it should now be the default.
    
    ## Testing
    - [x] Existing tests pass
  • display promo message in usage error (#10285)
    If a promo message is attached to a rate limit response, then display it
    in the error message.
  • feat: show runtime metrics in console (#10278)
    Summary of changes:
    
    - Adds a new feature flag: runtime_metrics
      - Declared in core/src/features.rs
      - Added to core/config.schema.json
      - Wired into OTEL init in core/src/otel_init.rs
    
    - Enables on-demand runtime metric snapshots in OTEL
      - Adds runtime_metrics: bool to otel/src/config.rs
      - Enables experimental custom reader features in otel/Cargo.toml
      - Adds snapshot/reset/summary APIs in:
        - otel/src/lib.rs
        - otel/src/metrics/client.rs
        - otel/src/metrics/config.rs
        - otel/src/metrics/error.rs
    
    - Defines metric names and a runtime summary builder
      - New files:
        - otel/src/metrics/names.rs
        - otel/src/metrics/runtime_metrics.rs
      - Summarizes totals for:
        - Tool calls
        - API requests
        - SSE/streaming events
    
    - Instruments metrics collection in OTEL manager
      - otel/src/traces/otel_manager.rs now records:
        - API call counts + durations
        - SSE event counts + durations (success/failure)
        - Tool call metrics now use shared constants
    
    - Surfaces runtime metrics in the TUI
      - Resets runtime metrics at turn start in tui/src/chatwidget.rs
    - Displays metrics in the final separator line in
    tui/src/history_cell.rs
    
    - Adds tests
      - New OTEL tests:
        - otel/tests/suite/snapshot.rs
        - otel/tests/suite/runtime_summary.rs
      - New TUI test:
    - final_message_separator_includes_runtime_metrics in
    tui/src/history_cell.rs
    
    Scope:
    - 19 files changed
    - ~652 insertions, 38 deletions
    
    
    <img width="922" height="169" alt="Screenshot 2026-01-30 at 4 11 34 PM"
    src="https://github.com/user-attachments/assets/1efd754d-a16d-4564-83a5-f4442fd2f998"
    />
  • feat(core) Smart approvals on (#10286)
    ## Summary
    Turn on Smart Approvals by default
    
    ## Testing
     - [x] Updated unit tests
  • add missing fields to WebSearchAction and update app-server types (#10276)
    - add `WebSearchAction` to app-server v2 types
    - add `queries` to `WebSearchAction::Search` type
    
    Updated tests.
  • Add enforce_residency to requirements (#10263)
    Add `enforce_residency` to requirements.toml and thread it through to a
    header on `default_client`.
  • Wire up cloud reqs in exec, app-server (#10241)
    We're fetching cloud requirements in TUI in
    https://github.com/openai/codex/pull/10167.
    
    This adds the same fetching in exec and app-server binaries also.
  • Validate CODEX_HOME before resolving (#10249)
    Summary
    - require `CODEX_HOME` to point to an existing directory before
    canonicalizing and surface clear errors otherwise
    - share the same helper logic in both `core` and `rmcp-client` and add
    unit tests that cover missing, non-directory, valid, and default paths
    
    This addresses #9222
  • fix: dont auto-enable web_search for azure (#10266)
    seeing issues with azure after default-enabling web search: #10071,
    #10257.
    
    need to work with azure to fix api-side, for now turning off
    default-enable of web_search for azure.
    
    diff is big because i moved logic to reuse
  • file-search: multi-root walk (#10240)
    Instead of a separate walker for each root in a multi-root walk, use a
    single walker.
  • core: prevent shell_snapshot from inheriting stdin (#9735)
    Fixes #9559.
    
    When `shell_snapshot` runs, it may execute user startup files (e.g.
    `.bashrc`). If those files read from stdin (or if stdin is an
    interactive TTY under job control), the snapshot subprocess can block or
    receive `SIGTTIN` (as reported over SSH).
    
    This change explicitly sets `stdin` to `Stdio::null()` for the snapshot
    subprocess, so it can't read from the terminal.
    
    Regression test added that would hang/timeout without this change.
    Tests: `ulimit -n 4096 && cargo test -p codex-core`.
    
    cc @dongdongbh @etraut-openai
    
    ---------
    
    Co-authored-by: Skylar Graika <sgraika127@gmail.com>
  • Skip loading codex home as project layer (#10207)
    Summary:
    - Fixes issue #9932: https://github.com/openai/codex/issues/9932
    - Prevents `$CODEX_HOME` (typically `~/.codex`) from being discovered as
    a project `.codex` layer by skipping it during project layer traversal.
    We compare both normalized absolute paths and best-effort canonicalized
    paths to handle symlinks.
    - Adds regression tests for home-directory invocation and for the case
    where `CODEX_HOME` points to a project `.codex` directory (e.g.,
    worktrees/editor integrations).
    
    Testing:
    - `cargo build -p codex-cli --bin codex`
    - `cargo build -p codex-rmcp-client --bin test_stdio_server`
    - `cargo test -p codex-core`
    - `cargo test --all-features`
    - Manual: ran `target/debug/codex` from `~` and confirmed the
    disabled-folder warning and trust prompt no longer appear.
  • chore: rename ChatGpt -> Chatgpt in type names (#10244)
    When using ChatGPT in names of types, we should be consistent, so this
    renames some types with `ChatGpt` in the name to `Chatgpt`. From
    https://rust-lang.github.io/api-guidelines/naming.html:
    
    > In `UpperCamelCase`, acronyms and contractions of compound words count
    as one word: use `Uuid` rather than `UUID`, `Usize` rather than `USize`
    or `Stdin` rather than `StdIn`. In `snake_case`, acronyms and
    contractions are lower-cased: `is_xid_start`.
    
    This PR updates existing uses of `ChatGpt` and changes them to
    `Chatgpt`. Though in all cases where it could affect the wire format, I
    visually inspected that we don't change anything there. That said, this
    _will_ change the codegen because it will affect the spelling of type
    names.
    
    For example, this renames `AuthMode::ChatGPT` to `AuthMode::Chatgpt` in
    `app-server-protocol`, but the wire format is still `"chatgpt"`.
    
    This PR also updates a number of types in `codex-rs/core/src/auth.rs`.
  • Plan mode: stream proposed plans, emit plan items, and render in TUI (#9786)
    ## Summary
    - Stream proposed plans in Plan Mode using `<proposed_plan>` tags parsed
    in core, emitting plan deltas plus a plan `ThreadItem`, while stripping
    tags from normal assistant output.
    - Persist plan items and rebuild them on resume so proposed plans show
    in thread history.
    - Wire plan items/deltas through app-server protocol v2 and render a
    dedicated proposed-plan view in the TUI, including the “Implement this
    plan?” prompt only when a plan item is present.
    
    ## Changes
    
    ### Core (`codex-rs/core`)
    - Added a generic, line-based tag parser that buffers each line until it
    can disprove a tag prefix; implements auto-close on `finish()` for
    unterminated tags. `codex-rs/core/src/tagged_block_parser.rs`
    - Refactored proposed plan parsing to wrap the generic parser.
    `codex-rs/core/src/proposed_plan_parser.rs`
    - In plan mode, stream assistant deltas as:
      - **Normal text** → `AgentMessageContentDelta`
      - **Plan text** → `PlanDelta` + `TurnItem::Plan` start/completion  
      (`codex-rs/core/src/codex.rs`)
    - Final plan item content is derived from the completed assistant
    message (authoritative), not necessarily the concatenated deltas.
    - Strips `<proposed_plan>` blocks from assistant text in plan mode so
    tags don’t appear in normal messages.
    (`codex-rs/core/src/stream_events_utils.rs`)
    - Persist `ItemCompleted` events only for plan items for rollout replay.
    (`codex-rs/core/src/rollout/policy.rs`)
    - Guard `update_plan` tool in Plan Mode with a clear error message.
    (`codex-rs/core/src/tools/handlers/plan.rs`)
    - Updated Plan Mode prompt to:  
      - keep `<proposed_plan>` out of non-final reasoning/preambles  
      - require exact tag formatting  
      - allow only one `<proposed_plan>` block per turn  
      (`codex-rs/core/templates/collaboration_mode/plan.md`)
    
    ### Protocol / App-server protocol
    - Added `TurnItem::Plan` and `PlanDeltaEvent` to core protocol items.
    (`codex-rs/protocol/src/items.rs`, `codex-rs/protocol/src/protocol.rs`)
    - Added v2 `ThreadItem::Plan` and `PlanDeltaNotification` with
    EXPERIMENTAL markers and note that deltas may not match the final plan
    item. (`codex-rs/app-server-protocol/src/protocol/v2.rs`)
    - Added plan delta route in app-server protocol common mapping.
    (`codex-rs/app-server-protocol/src/protocol/common.rs`)
    - Rebuild plan items from persisted `ItemCompleted` events on resume.
    (`codex-rs/app-server-protocol/src/protocol/thread_history.rs`)
    
    ### App-server
    - Forward plan deltas to v2 clients and map core plan items to v2 plan
    items. (`codex-rs/app-server/src/bespoke_event_handling.rs`,
    `codex-rs/app-server/src/codex_message_processor.rs`)
    - Added v2 plan item tests.
    (`codex-rs/app-server/tests/suite/v2/plan_item.rs`)
    
    ### TUI
    - Added a dedicated proposed plan history cell with special background
    and padding, and moved “• Proposed Plan” outside the highlighted block.
    (`codex-rs/tui/src/history_cell.rs`, `codex-rs/tui/src/style.rs`)
    - Only show “Implement this plan?” when a plan item exists.
    (`codex-rs/tui/src/chatwidget.rs`,
    `codex-rs/tui/src/chatwidget/tests.rs`)
    
    <img width="831" height="847" alt="Screenshot 2026-01-29 at 7 06 24 PM"
    src="https://github.com/user-attachments/assets/69794c8c-f96b-4d36-92ef-c1f5c3a8f286"
    />
    
    ### Docs / Misc
    - Updated protocol docs to mention plan deltas.
    (`codex-rs/docs/protocol_v1.md`)
    - Minor plumbing updates in exec/debug clients to tolerate plan deltas.
    (`codex-rs/debug-client/src/reader.rs`, `codex-rs/exec/...`)
    
    ## Tests
    - Added core integration tests:
      - Plan mode strips plan from agent messages.
      - Missing `</proposed_plan>` closes at end-of-message.  
      (`codex-rs/core/tests/suite/items.rs`)
    - Added unit tests for generic tag parser (prefix buffering, non-tag
    lines, auto-close). (`codex-rs/core/src/tagged_block_parser.rs`)
    - Existing app-server plan item tests in v2.
    (`codex-rs/app-server/tests/suite/v2/plan_item.rs`)
    
    ## Notes / Behavior
    - Plan output no longer appears in standard assistant text in Plan Mode;
    it streams via `PlanDelta` and completes as a `TurnItem::Plan`.
    - The final plan item content is authoritative and may diverge from
    streamed deltas (documented as experimental).
    - Reasoning summaries are not filtered; prompt instructs the model not
    to include `<proposed_plan>` outside the final plan message.
    
    ## Codex Author
    `codex fork 019bec2d-b09d-7450-b292-d7bcdddcdbfb`
  • Load exec policy rules from requirements (#10190)
    `requirements.toml` should be able to specify rules which always run. 
    
    My intention here was that these rules could only ever be restrictive,
    which means the decision can be "prompt" or "forbidden" but never
    "allow". A requirement of "you must always allow this command" didn't
    make sense to me, but happy to be gaveled otherwise.
    
    Rules already applies the most restrictive decision, so we can safely
    merge these with rules found in other config folders.
  • chore(feature) Experimental: Smart Approvals (#10211)
    ## Summary
    Let's start getting feedback on this feature 😅 
    
    ## Testing
    - [x] existing tests pass
  • chore(feature) Experimental: Personality (#10212)
    ## Summary
    Let users start opting in to trying out personalities
    
    ## Testing
    - [x] existing tests pass
  • feat: refactor CodexAuth so invalid state cannot be represented (#10208)
    Previously, `CodexAuth` was defined as follows:
    
    
    https://github.com/openai/codex/blob/d550fbf41afc09d7d7b5ac813aea38de07b2a73f/codex-rs/core/src/auth.rs#L39-L46
    
    But if you looked at its constructors, we had creation for
    `AuthMode::ApiKey` where `storage` was built using a nonsensical path
    (`PathBuf::new()`) and `auth_dot_json` was `None`:
    
    
    https://github.com/openai/codex/blob/d550fbf41afc09d7d7b5ac813aea38de07b2a73f/codex-rs/core/src/auth.rs#L212-L220
    
    By comparison, when `AuthMode::ChatGPT` was used, `api_key` was always
    `None`:
    
    
    https://github.com/openai/codex/blob/d550fbf41afc09d7d7b5ac813aea38de07b2a73f/codex-rs/core/src/auth.rs#L665-L671
    
    https://github.com/openai/codex/pull/10012 took things further because
    it introduced a new `ChatgptAuthTokens` variant to `AuthMode`, which is
    important in when invoking `account/login/start` via the app server, but
    most logic _internal_ to the app server should just reason about two
    `AuthMode` variants: `ApiKey` and `ChatGPT`.
    
    This PR tries to clean things up as follows:
    
    - `LoginAccountParams` and `AuthMode` in `codex-rs/app-server-protocol/`
    both continue to have the `ChatgptAuthTokens` variant, though it is used
    exclusively for the on-the-wire messaging.
    - `codex-rs/core/src/auth.rs` now has its own `AuthMode` enum, which
    only has two variants: `ApiKey` and `ChatGPT`.
    - `CodexAuth` has been changed from a struct to an enum. It is a
    disjoint union where each variant (`ApiKey`, `ChatGpt`, and
    `ChatGptAuthTokens`) have only the associated fields that make sense for
    that variant.
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/10208).
    * #10224
    * __->__ #10208