Commit Graph

22 Commits

  • Another round of improvements for config error messages (#9746)
    In a [recent PR](https://github.com/openai/codex/pull/9182), I made some
    improvements to config error messages so errors didn't leave app server
    clients in a dead state. This is a follow-on PR to make these error
    messages more readable and actionable for both TUI and GUI users. For
    example, see #9668 where the user was understandably confused about the
    source of the problem and how to fix it.
    
    The improved error message:
    1. Clearly identifies the config file where the error was found (which
    is more important now that we support layered configs)
    2. Provides a line and column number of the error
    3. Displays the line where the error occurred and underlines it
    
    For example, if my `config.toml` includes the following:
    ```toml
    [features]
    collaboration_modes = "true"
    ```
    
    Here's the current CLI error message:
    ```
    Error loading config.toml: invalid type: string "true", expected a boolean in `features`
    ```
    
    And here's the improved message:
    ```
    Error loading config.toml:
    /Users/etraut/.codex/config.toml:43:23: invalid type: string "true", expected a boolean
       |
    43 | collaboration_modes = "true"
       |                       ^^^^^^
    ```
    
    The bulk of the new logic is contained within a new module
    `config_loader/diagnostics.rs` that is responsible for calculating the
    text range for a given toml path (which is more involved than I would
    have expected).
    
    In addition, this PR adds the file name and text range to the
    `ConfigWarningNotification` app server struct. This allows GUI clients
    to present the user with a better error message and an optional link to
    open the errant config file. This was a suggestion from @.bolinfest when
    he reviewed my previous PR.
  • Print warning if we skip config loading (#9611)
    https://github.com/openai/codex/pull/9533 silently ignored config if
    untrusted. Instead, we still load it but disable it. Maybe we shouldn't
    try to parse it either...
    
    <img width="939" height="515" alt="Screenshot 2026-01-21 at 14 56 38"
    src="https://github.com/user-attachments/assets/e753cc22-dd99-4242-8ffe-7589e85bef66"
    />
  • feat: add auto refresh on thread listeners (#9105)
    This PR is in the scope of multi-agent work. 
    
    An agent (=thread) can now spawn other agents. Those other agents are
    not attached to any clients. We need a way to make sure that the clients
    are aware of the new threads to look at (for approval for example). This
    PR adds a channel to the `ThreadManager` that pushes the ID of those
    newly created agents such that the client (here the app-server) can also
    subscribe to those ones.
  • Improve handling of config and rules errors for app server clients (#9182)
    When an invalid config.toml key or value is detected, the CLI currently
    just quits. This leaves the VSCE in a dead state.
    
    This PR changes the behavior to not quit and bubble up the config error
    to users to make it actionable. It also surfaces errors related to
    "rules" parsing.
    
    This allows us to surface these errors to users in the VSCE, like this:
    
    <img width="342" height="129" alt="Screenshot 2026-01-13 at 4 29 22 PM"
    src="https://github.com/user-attachments/assets/a79ffbe7-7604-400c-a304-c5165b6eebc4"
    />
    
    <img width="346" height="244" alt="Screenshot 2026-01-13 at 4 45 06 PM"
    src="https://github.com/user-attachments/assets/de874f7c-16a2-4a95-8c6d-15f10482e67b"
    />
  • feat(app-server): add an --analytics-default-enabled flag (#9118)
    Add a new `codex app-server --analytics-default-enabled` CLI flag that
    controls whether analytics are enabled by default.
    
    Analytics are disabled by default for app-server. Users have to
    explicitly opt in
    via the `analytics` section in the config.toml file.
    
    However, for first-party use cases like the VSCode IDE extension, we
    default analytics
    to be enabled by default by setting this flag. Users can still opt out
    by setting this
    in their config.toml:
    
    ```toml
    [analytics]
    enabled = false
    ```
    
    See https://developers.openai.com/codex/config-advanced/#metrics for
    more details.
  • fix(app-server): set originator header from initialize JSON-RPC request (#8873)
    **Motivation**
    The `originator` header is important for codex-backend’s Responses API
    proxy because it identifies the real end client (codex cli, codex vscode
    extension, codex exec, future IDEs) and is used to categorize requests
    by client for our enterprise compliance API.
    
    Today the `originator` header is set by either:
    - the `CODEX_INTERNAL_ORIGINATOR_OVERRIDE` env var (our VSCode extension
    does this)
    - calling `set_default_originator()` which sets a global immutable
    singleton (`codex exec` does this)
    
    For `codex app-server`, we want the `initialize` JSON-RPC request to set
    that header because it is a natural place to do so. Example:
    ```json
    {
      "method": "initialize",
      "id": 0,
      "params": {
        "clientInfo": {
          "name": "codex_vscode",
          "title": "Codex VS Code Extension",
          "version": "0.1.0"
        }
      }
    }
    ```
    and when app-server receives that request, it can call
    `set_default_originator()`. This is a much more natural interface than
    asking third party developers to set an env var.
    
    One hiccup is that `originator()` reads the global singleton and locks
    in the value, preventing a later `set_default_originator()` call from
    setting it. This would be fine but is brittle, since any codepath that
    calls `originator()` before app-server can process an `initialize`
    JSON-RPC call would prevent app-server from setting it. This was
    actually the case with OTEL initialization which runs on boot, but I
    also saw this behavior in certain tests.
    
    Instead, what we now do is:
    - [unchanged] If `CODEX_INTERNAL_ORIGINATOR_OVERRIDE` env var is set,
    `originator()` would return that value and `set_default_originator()`
    with some other value does NOT override it.
    - [new] If no env var is set, `originator()` would return the default
    value which is `codex_cli_rs` UNTIL `set_default_originator()` is called
    once, in which case it is set to the new value and becomes immutable.
    Later calls to `set_default_originator()` returns
    `SetOriginatorError::AlreadyInitialized`.
    
    **Other notes**
    - I updated `codex_core::otel_init::build_provider` to accepts a service
    name override, and app-server sends a hardcoded `codex_app_server`
    service name to distinguish it from `codex_cli_rs` used by default (e.g.
    TUI).
    
    **Next steps**
    - Update VSCE to set the proper value for `clientInfo.name` on
    `initialize` and drop the `CODEX_INTERNAL_ORIGINATOR_OVERRIDE` env var.
    - Delete support for `CODEX_INTERNAL_ORIGINATOR_OVERRIDE` in codex-rs.
  • Attach more tags to feedback submissions (#8688)
    Attach more tags to sentry feedback so it's easier to classify and debug
    without having to scan through logs.
    
    Formatting isn't amazing but it's a start.
    <img width="1234" height="276" alt="image"
    src="https://github.com/user-attachments/assets/521a349d-f627-4051-b511-9811cd5cd933"
    />
  • chore: cleanup Config instantiation codepaths (#8226)
    This PR does various types of cleanup before I can proceed with more
    ambitious changes to config loading.
    
    First, I noticed duplicated code across these two methods:
    
    
    https://github.com/openai/codex/blob/774bd9e432fa2e0f4e059e97648cf92216912e19/codex-rs/core/src/config/mod.rs#L314-L324
    
    
    https://github.com/openai/codex/blob/774bd9e432fa2e0f4e059e97648cf92216912e19/codex-rs/core/src/config/mod.rs#L334-L344
    
    This has now been consolidated in
    `load_config_as_toml_with_cli_overrides()`.
    
    Further, I noticed that `Config::load_with_cli_overrides()` took two
    similar arguments:
    
    
    https://github.com/openai/codex/blob/774bd9e432fa2e0f4e059e97648cf92216912e19/codex-rs/core/src/config/mod.rs#L308-L311
    
    The difference between `cli_overrides` and `overrides` was not
    immediately obvious to me. At first glance, it appears that one should
    be able to be expressed in terms of the other, but it turns out that
    some fields of `ConfigOverrides` (such as `cwd` and
    `codex_linux_sandbox_exe`) are, by design, not configurable via a
    `.toml` file or a command-line `--config` flag.
    
    That said, I discovered that many callers of
    `Config::load_with_cli_overrides()` were passing
    `ConfigOverrides::default()` for `overrides`, so I created two separate
    methods:
    
    - `Config::load_with_cli_overrides(cli_overrides: Vec<(String,
    TomlValue)>)`
    - `Config::load_with_cli_overrides_and_harness_overrides(cli_overrides:
    Vec<(String, TomlValue)>, harness_overrides: ConfigOverrides)`
    
    The latter has a long name, as it is _not_ what should be used in the
    common case, so the extra typing is designed to draw attention to this
    fact. I tried to update the existing callsites to use the shorter name,
    where possible.
    
    Further, in the cases where `ConfigOverrides` is used, usually only a
    limited subset of fields are actually set, so I updated the declarations
    to leverage `..Default::default()` where possible.
  • [App server] add mcp tool call item started/completed events (#6642)
    this PR does two things:
    1. refactor `apply_bespoke_event_handling` into a separate file as it's
    getting kind of long;
    2. add mcp tool call `item/started` and `item/completed` events. To roll
    out app server events asap we didn't properly migrate mcp core events to
    use TurnItem for mcp tool calls - this will be a follow-up PR.
    
    real events generated in log:
    ```
    {
      "method": "codex/event/mcp_tool_call_end",
      "params": {
        "conversationId": "019a8021-26af-7c20-83db-21ca81e44d68",
        "id": "0",
        "msg": {
          "call_id": "call_7EjRQkD9HnfyMWf7tGrT9FKA",
          "duration": {
            "nanos": 92708,
            "secs": 0
          },
          "invocation": {
            "arguments": {
              "server": ""
            },
            "server": "codex",
            "tool": "list_mcp_resources"
          },
          "result": {
            "Ok": {
              "content": [
                {
                  "text": "{\"resources\":[]}",
                  "type": "text"
                }
              ],
              "isError": false
            }
          },
          "type": "mcp_tool_call_end"
        }
      }
    }
    
    {
      "method": "item/completed",
      "params": {
        "item": {
          "arguments": {
            "server": ""
          },
          "error": null,
          "id": "call_7EjRQkD9HnfyMWf7tGrT9FKA",
          "result": {
            "content": [
              {
                "text": "{\"resources\":[]}",
                "type": "text"
              }
            ],
            "structuredContent": null
          },
          "server": "codex",
          "status": "completed",
          "tool": "list_mcp_resources",
          "type": "mcpToolCall"
        }
      }
    }
    ```
  • [app-server] model/list API (#5382)
    Adds a `model/list` paginated API that returns the list of models
    supported by Codex.
  • add(core): managed config (#3868)
    ## Summary
    
    - Factor `load_config_as_toml` into `core::config_loader` so config
    loading is reusable across callers.
    - Layer `~/.codex/config.toml`, optional `~/.codex/managed_config.toml`,
    and macOS managed preferences (base64) with recursive table merging and
    scoped threads per source.
    
    ## Config Flow
    
    ```
    Managed prefs (macOS profile: com.openai.codex/config_toml_base64)
                                   ▲
                                   │
    ~/.codex/managed_config.toml   │  (optional file-based override)
                                   ▲
                                   │
                    ~/.codex/config.toml (user-defined settings)
    ```
    
    - The loader searches under the resolved `CODEX_HOME` directory
    (defaults to `~/.codex`).
    - Managed configs let administrators ship fleet-wide overrides via
    device profiles which is useful for enforcing certain settings like
    sandbox or approval defaults.
    - For nested hash tables: overlays merge recursively. Child tables are
    merged key-by-key, while scalar or array values replace the prior layer
    entirely. This lets admins add or tweak individual fields without
    clobbering unrelated user settings.
  • fix: remove mcp-types from app server protocol (#4537)
    We continue the separation between `codex app-server` and `codex
    mcp-server`.
    
    In particular, we introduce a new crate, `codex-app-server-protocol`,
    and migrate `codex-rs/protocol/src/mcp_protocol.rs` into it, renaming it
    `codex-rs/app-server-protocol/src/protocol.rs`.
    
    Because `ConversationId` was defined in `mcp_protocol.rs`, we move it
    into its own file, `codex-rs/protocol/src/conversation_id.rs`, and
    because it is referenced in a ton of places, we have to touch a lot of
    files as part of this PR.
    
    We also decide to get away from proper JSON-RPC 2.0 semantics, so we
    also introduce `codex-rs/app-server-protocol/src/jsonrpc_lite.rs`, which
    is basically the same `JSONRPCMessage` type defined in `mcp-types`
    except with all of the `"jsonrpc": "2.0"` removed.
    
    Getting rid of `"jsonrpc": "2.0"` makes our serialization logic
    considerably simpler, as we can lean heavier on serde to serialize
    directly into the wire format that we use now.
  • fix: separate codex mcp into codex mcp-server and codex app-server (#4471)
    This is a very large PR with some non-backwards-compatible changes.
    
    Historically, `codex mcp` (or `codex mcp serve`) started a JSON-RPC-ish
    server that had two overlapping responsibilities:
    
    - Running an MCP server, providing some basic tool calls.
    - Running the app server used to power experiences such as the VS Code
    extension.
    
    This PR aims to separate these into distinct concepts:
    
    - `codex mcp-server` for the MCP server
    - `codex app-server` for the "application server"
    
    Note `codex mcp` still exists because it already has its own subcommands
    for MCP management (`list`, `add`, etc.)
    
    The MCP logic continues to live in `codex-rs/mcp-server` whereas the
    refactored app server logic is in the new `codex-rs/app-server` folder.
    Note that most of the existing integration tests in
    `codex-rs/mcp-server/tests/suite` were actually for the app server, so
    all the tests have been moved with the exception of
    `codex-rs/mcp-server/tests/suite/mod.rs`.
    
    Because this is already a large diff, I tried not to change more than I
    had to, so `codex-rs/app-server/tests/common/mcp_process.rs` still uses
    the name `McpProcess` for now, but I will do some mechanical renamings
    to things like `AppServer` in subsequent PRs.
    
    While `mcp-server` and `app-server` share some overlapping functionality
    (like reading streams of JSONL and dispatching based on message types)
    and some differences (completely different message types), I ended up
    doing a bit of copypasta between the two crates, as both have somewhat
    similar `message_processor.rs` and `outgoing_message.rs` files for now,
    though I expect them to diverge more in the near future.
    
    One material change is that of the initialize handshake for `codex
    app-server`, as we no longer use the MCP types for that handshake.
    Instead, we update `codex-rs/protocol/src/mcp_protocol.rs` to add an
    `Initialize` variant to `ClientRequest`, which takes the `ClientInfo`
    object we need to update the `USER_AGENT_SUFFIX` in
    `codex-rs/app-server/src/message_processor.rs`.
    
    One other material change is in
    `codex-rs/app-server/src/codex_message_processor.rs` where I eliminated
    a use of the `send_event_as_notification()` method I am generally trying
    to deprecate (because it blindly maps an `EventMsg` into a
    `JSONNotification`) in favor of `send_server_notification()`, which
    takes a `ServerNotification`, as that is intended to be a custom enum of
    all notification types supported by the app server. So to make this
    update, I had to introduce a new variant of `ServerNotification`,
    `SessionConfigured`, which is a non-backwards compatible change with the
    old `codex mcp`, and clients will have to be updated after the next
    release that contains this PR. Note that
    `codex-rs/app-server/tests/suite/list_resume.rs` also had to be update
    to reflect this change.
    
    I introduced `codex-rs/utils/json-to-toml/src/lib.rs` as a small utility
    crate to avoid some of the copying between `mcp-server` and
    `app-server`.