Commit Graph

373 Commits

  • [app-server] add events to readme (#6690)
    add table of contents, lifecycle and events to readme.
  • [App server] add mcp tool call item started/completed events (#6642)
    this PR does two things:
    1. refactor `apply_bespoke_event_handling` into a separate file as it's
    getting kind of long;
    2. add mcp tool call `item/started` and `item/completed` events. To roll
    out app server events asap we didn't properly migrate mcp core events to
    use TurnItem for mcp tool calls - this will be a follow-up PR.
    
    real events generated in log:
    ```
    {
      "method": "codex/event/mcp_tool_call_end",
      "params": {
        "conversationId": "019a8021-26af-7c20-83db-21ca81e44d68",
        "id": "0",
        "msg": {
          "call_id": "call_7EjRQkD9HnfyMWf7tGrT9FKA",
          "duration": {
            "nanos": 92708,
            "secs": 0
          },
          "invocation": {
            "arguments": {
              "server": ""
            },
            "server": "codex",
            "tool": "list_mcp_resources"
          },
          "result": {
            "Ok": {
              "content": [
                {
                  "text": "{\"resources\":[]}",
                  "type": "text"
                }
              ],
              "isError": false
            }
          },
          "type": "mcp_tool_call_end"
        }
      }
    }
    
    {
      "method": "item/completed",
      "params": {
        "item": {
          "arguments": {
            "server": ""
          },
          "error": null,
          "id": "call_7EjRQkD9HnfyMWf7tGrT9FKA",
          "result": {
            "content": [
              {
                "text": "{\"resources\":[]}",
                "type": "text"
              }
            ],
            "structuredContent": null
          },
          "server": "codex",
          "status": "completed",
          "tool": "list_mcp_resources",
          "type": "mcpToolCall"
        }
      }
    }
    ```
  • [app-server] small fixes for JSON schema export and one-of types (#6614)
    A partner is consuming our generated JSON schema bundle for app-server
    and identified a few issues:
    - not all polymorphic / one-of types have a type descriminator
    - `"$ref": "#/definitions/v2/SandboxPolicy"` is missing
    - "Option<>" is an invalid schema name, and also unnecessary
    
    This PR:
    - adds the type descriminator to the various types that are missing it
    except for `SessionSource` and `SubAgentSource` because they are
    serialized to disk (adding this would break backwards compat for
    resume), and they should not be necessary to consume for an integration
    with app-server.
    - removes the special handling in `export.rs` of various types like
    SandboxPolicy, which turned out to be unnecessary and incorrect
    - filters out `Option<>` which was auto-generated for request params
    that don't need a body
    
    For context, we currently pull in wayyy more types than we need through
    the `EventMsg` god object which we are **not** planning to expose in API
    v2 (this is how I suspect `SessionSource` and `SubAgentSource` are being
    pulled in). But until we have all the necessary v2 notifications in
    place that will allow us to remove `EventMsg`, we will keep exporting it
    for now.
  • [App-server] add new v2 events:item/reasoning/delta, item/agentMessage/delta & item/reasoning/summaryPartAdded (#6559)
    core event to app server event mapping:
    1. `codex/event/reasoning_content_delta` ->
    `item/reasoning/summaryTextDelta`.
    2. `codex/event/reasoning_raw_content_delta` ->
    `item/reasoning/textDelta`
    3. `codex/event/agent_message_content_delta` →
    `item/agentMessage/delta`.
    4. `codex/event/agent_reasoning_section_break` ->
    `item/reasoning/summaryPartAdded`.
    
    Also added a change in core to pass down content index, summary index
    and item id from events.
    
    Tested with the `git checkout owen/app_server_test_client && cargo run
    -p codex-app-server-test-client -- send-message-v2 "hello"` and verified
    that new events are emitted correctly.
  • Reasoning level update (#6586)
    Automatically update reasoning levels when migrating between models
  • Change model picker to include gpt5.1 (#6569)
    - Change the presets
    - Change the tests that make sure we keep the list of tools updated
    - Filter out deprecated models
  • [app-server] feat: thread/resume supports history, path, and overrides (#6483)
    This updates `thread/resume` to be at parity with v1's
    `ResumeConversationParams`. Turns out history is useful for codex cloud
    and path is useful for the VSCode extension. And config overrides are
    always useful.
  • [app-server] add item started/completed events for turn items (#6517)
    This one should be quite straightforward, as it's just a translation of
    TurnItem events we already emit to ThreadItem that app-server exposes to
    customers.
    
    To test, cp my change to owen/app_server_test_client and do the
    following:
    ```
    cargo build -p codex-cli
    RUST_LOG=codex_app_server=info CODEX_BIN=target/debug/codex cargo run -p codex-app-server-test-client -- send-message-v2 "hello"
    ```
    
    example event before (still kept there for backward compatibility):
    ```
    {
    <   "method": "codex/event/item_completed",
    <   "params": {
    <     "conversationId": "019a74cc-fad9-7ab3-83a3-f42827b7b074",
    <     "id": "0",
    <     "msg": {
    <       "item": {
    <         "Reasoning": {
    <           "id": "rs_03d183492e07e20a016913a936eb8c81a1a7671a103fee8afc",
    <           "raw_content": [],
    <           "summary_text": [
    <             "Hey! What would you like to work on? I can explore the repo, run specific tests, or implement a change. Let's keep it short and straightforward. There's no need for a lengthy introduction or elaborate planning, just a friendly greeting and an open offer to help. I want to make sure the user feels welcomed and understood right from the start. It's all about keeping the tone friendly and concise!"
    <           ]
    <         }
    <       },
    <       "thread_id": "019a74cc-fad9-7ab3-83a3-f42827b7b074",
    <       "turn_id": "0",
    <       "type": "item_completed"
    <     }
    <   }
    < }
    ```
    
    after (v2):
    ```
    < {
    <   "method": "item/completed",
    <   "params": {
    <     "item": {
    <       "id": "rs_03d183492e07e20a016913a936eb8c81a1a7671a103fee8afc",
    <       "text": "Hey! What would you like to work on? I can explore the repo, run specific tests, or implement a change. Let's keep it short and straightforward. There's no need for a lengthy introduction or elaborate planning, just a friendly greeting and an open offer to help. I want to make sure the user feels welcomed and understood right from the start. It's all about keeping the tone friendly and concise!",
    <       "type": "reasoning"
    <     }
    <   }
    < }
    ```
  • [hygiene][app-server] have a helper function for duplicate code in turn APIs (#6488)
    turn_start and turn_interrupt have some logic that can be shared. have a
    helper function for it.
  • [app-server] feat: add command to generate json schema (#6406)
    Add a `codex generate-json-schema` command for generating a JSON schema
    bundle of app-server types, analogous to the existing `codex
    generate-ts` command for Typescript.
  • [App-server] Add auth v2 doc & update codex mcp interface auth section (#6353)
    Added doc for auth v2 endpoints. Updated the auth section in Codex MCP
    interface doc too.
  • [app-server] feat: expose additional fields on Thread (#6338)
    Add the following fields to Thread:
    
    ```
        pub preview: String,
        pub model_provider: String,
        pub created_at: i64,
    ```
    
    Will prob need another PR once this lands:
    https://github.com/openai/codex/pull/6337
  • [App-server] Implement account/read endpoint (#6336)
    This PR does two things:
    1. add a new function in core that maps the core-internal plan type to
    the external plan type;
    2. implement account/read that get account status (v2 of
    `getAuthStatus`).
  • [App Server] Add more session metadata to listConversations (#6337)
    This unlocks a few new product experience for app server consumers
  • chore: move relevant tests to app-server/tests/suite/v2 (#6289)
    These are technically app-server v2 APIs, so move them to the same
    directory as the others.
  • [app-server] feat: v2 Turn APIs (#6216)
    Implements:
    ```
    turn/start
    turn/interrupt
    ```
    
    along with their integration tests. These are relatively light wrappers
    around the existing core logic, and changes to core logic are minimal.
    
    However, an improvement made for developer ergonomics:
    - `turn/start` replaces both `SendUserMessage` (no turn overrides) and
    `SendUserTurn` (can override model, approval policy, etc.)
  • [App-server] Add account/login/cancel v2 endpoint (#6288)
    Add `account/login/cancel` v2 endpoint for auth. this is similar
    implementation to `cancelLoginChatgpt` v1 endpoint.
  • [App-server] Implement v2 for account/login/start and account/login/completed (#6183)
    This PR implements `account/login/start` and `account/login/completed`.
    Instead of having separate endpoints for login with chatgpt and api, we
    have a single enum handling different login methods. For sync auth
    methods like sign in with api key, we still send a `completed`
    notification back to be compatible with the async login flow.
  • Fixes intermittent test failures in CI (#6282)
    I'm seeing two tests fail intermittently in CI. This PR attempts to
    address (or at least mitigate) the flakiness.
    
    * summarize_context_three_requests_and_instructions - The test snapshots
    server.received_requests() immediately after observing TaskComplete.
    Because the OpenAI /v1/responses call is streamed, the HTTP request can
    still be draining when that event fires, so wiremock occasionally
    reports only two captured requests. Fix is to wait for async activity to
    complete.
    * archive_conversation_moves_rollout_into_archived_directory - times out
    on a slow CI run. Mitigation is to increase timeout value from 10s to
    20s.
  • [app-server] feat: v2 Thread APIs (#6214)
    Implements:
    ```
    thread/list
    thread/start
    thread/resume
    thread/archive
    ```
    
    along with their integration tests. These are relatively light wrappers
    around the existing core logic, and changes to core logic are minimal.
    
    However, an improvement made for developer ergonomics:
    - `thread/start` and `thread/resume` automatically attaches a
    conversation listener internally, so clients don't have to make a
    separate `AddConversationListener` call like they do today.
    
    For consistency, also updated `model/list` and `feedback/upload` (naming
    conventions, list API params).
  • [app-server] feat: export.rs supports a v2 namespace, initial v2 notifications (#6212)
    **Typescript and JSON schema exports**
    While working on Thread/Turn/Items type definitions, I realize we will
    run into name conflicts between v1 and v2 APIs (e.g. `RateLimitWindow`
    which won't be reusable since v1 uses `RateLimitWindow` from `protocol/`
    which uses snake_case, but we want to expose camelCase everywhere, so
    we'll define a V2 version of that struct that serializes as camelCase).
    
    To set us up for a clean and isolated v2 API, generate types into a
    `v2/` namespace for both typescript and JSON schema.
    - TypeScript: v2 types emit under `out_dir/v2/*.ts`, and root index.ts
    now re-exports them via `export * as v2 from "./v2"`;.
    - JSON Schemas: v2 definitions bundle under `#/definitions/v2/*` rather
    than the root.
    
    The location for the original types (v1 and types pulled from
    `protocol/` and other core crates) haven't changed and are still at the
    root. This is for backwards compatibility: no breaking changes to
    existing usages of v1 APIs and types.
    
    **Notifications**
    While working on export.rs, I:
    - refactored server/client notifications with macros (like we already do
    for methods) so they also get exported (I noticed they weren't being
    exported at all).
    - removed the hardcoded list of types to export as JSON schema by
    leveraging the existing macros instead
    - and took a stab at API V2 notifications. These aren't wired up yet,
    and I expect to iterate on these this week.
  • [App-server] v2 for account/updated and account/logout (#6175)
    V2 for `account/updated` and `account/logout` for app server. correspond
    to old `authStatusChange` and `LogoutChatGpt` respectively. Followup PRs
    will make other v2 endpoints call `account/updated` instead of
    `authStatusChange` too.
  • Fix "archive conversation" on Windows (#6124)
    Addresses issue https://github.com/openai/codex/issues/3582 where an
    "archive conversation" command in the extension fails on Windows.
    
    The problem is that the `archive_conversation` api server call is not
    canonicalizing the path to the rollout path when performing its check to
    verify that the rollout path is in the sessions directory. This causes
    it to fail 100% of the time on Windows.
    
    Testing: I was able to repro the error on Windows 100% prior to this
    change. After the change, I'm no longer able to repro.
  • [Hygiene] Remove include_view_image_tool config (#5976)
    There's still some debate about whether we want to expose
    `tools.view_image` or `feature.view_image` so those are left unchanged
    for now, but this old `include_view_image_tool` config is good-to-go.
    Also updated the doc to reflect that `view_image` tool is now by default
    true.
  • [app-server] remove serde(skip_serializing_if = "Option::is_none") annotations (#5939)
    We had this annotation everywhere in app-server APIs which made it so
    that fields get serialized as `field?: T`, meaning if the field as
    `None` we would omit the field in the payload. Removing this annotation
    changes it so that we return `field: T | null` instead, which makes
    codex app-server's API more aligned with the convention of public OpenAI
    APIs like Responses.
    
    Separately, remove the `#[ts(optional_fields = nullable)]` annotations
    that were recently added which made all the TS types become `field?: T |
    null` which is not great since clients need to handle undefined and
    null.
    
    I think generally it'll be best to have optional types be either:
    - `field: T | null` (preferred, aligned with public OpenAI APIs)
    - `field?: T` where we have to, such as types generated from the MCP
    schema:
    https://github.com/modelcontextprotocol/modelcontextprotocol/blob/main/schema/2025-06-18/schema.ts
    (see changes to `mcp-types/`)
    
    I updated @etraut-openai's unit test to check that all generated TS
    types are one or the other, not both (so will error if we have a type
    that has `field?: T | null`). I don't think there's currently a good use
    case for that - but we can always revisit.
  • [codex] add developer instructions (#5897)
    we are using developer instructions for code reviews, we need to pass
    them in cli as well.
  • feat: compaction prompt configurable (#5959)
    ```
     codex -c compact_prompt="Summarize in bullet points"
     ```
  • chore: config editor (#5878)
    The goal is to have a single place where we actually write files
    
    In a follow-up PR, will move everything config related in a dedicated
    module and move the helpers in a dedicated file
  • Add a wrapper around raw response items (#5923)
    We currently have nested enums when sending raw response items in the
    app-server protocol. This makes downstream schemas confusing because we
    need to embed `type`-discriminated enums within each other.
    
    This PR adds a small wrapper around the response item so we can keep the
    schemas separate
  • [App Server] Allow fetching or resuming a conversation summary from the conversation id (#5890)
    This PR adds an option to app server to allow conversation summaries to
    be fetched from just the conversation id rather than rollout path for
    convenience at the cost of some latency to discover the rollout path.
    
    This convenience is non-trivial as it allows app servers to simply
    maintain conversation ids rather than rollout paths and the associated
    platform (Windows) handling associated with storing and encoding them
    correctly.
  • chore: use anyhow::Result for all app-server integration tests (#5836)
    There's a lot of visual noise in app-server's integration tests due to
    the number of `.expect("<some_msg>")` lines which are largely redundant
    / not very useful. Clean them up by using `anyhow::Result` + `?`
    consistently.
    
    Replaces the existing pattern of:
    ```
        let codex_home = TempDir::new().expect("create temp dir");
        create_config_toml(codex_home.path()).expect("write config.toml");
    
        let mut mcp = McpProcess::new(codex_home.path())
            .await
            .expect("spawn mcp process");
        timeout(DEFAULT_READ_TIMEOUT, mcp.initialize())
            .await
            .expect("initialize timeout")
            .expect("initialize request");
    ```
    
    With:
    ```
        let codex_home = TempDir::new()?;
        create_config_toml(codex_home.path())?;
    
        let mut mcp = McpProcess::new(codex_home.path()).await?;
        timeout(DEFAULT_READ_TIMEOUT, mcp.initialize()).await??;
    ```
  • [Auth] Choose which auth storage to use based on config (#5792)
    This PR is a follow-up to #5591. It allows users to choose which auth
    storage mode they want by using the new
    `cli_auth_credentials_store_mode` config.
  • [Auth] Introduce New Auth Storage Abstraction for Codex CLI (#5569)
    This PR introduces a new `Auth Storage` abstraction layer that takes
    care of read, write, and load of auth tokens based on the
    AuthCredentialsStoreMode. It is similar to how we handle MCP client
    oauth
    [here](https://github.com/openai/codex/blob/main/codex-rs/rmcp-client/src/oauth.rs).
    Instead of reading and writing directly from disk for auth tokens, Codex
    CLI workflows now should instead use this auth storage using the public
    helper functions.
    
    This PR is just a refactor of the current code so the behavior stays the
    same. We will add support for keyring and hybrid mode in follow-up PRs.
    
    I have read the CLA Document and I hereby sign the CLA
  • feat: introduce GetConversationSummary RPC (#5803)
    This adds an RPC to the app server to the the `ConversationSummary` via
    a rollout path. Now that the VS Code extension supports showing the
    Codex UI in an editor panel where the URI of the panel maps to the
    rollout file, we need to be able to get the `ConversationSummary` from
    the rollout file directly.
  • feat: update NewConversationParams to take an optional model_provider (#5793)
    An AppServer client should be able to use any (`model_provider`, `model`) in the user's config. `NewConversationParams` already supported specifying the `model`, but this PR expands it to support `model_provider`, as well.
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/5793).
    * #5803
    * __->__ #5793
  • feat: annotate conversations with model_provider for filtering (#5658)
    Because conversations that use the Responses API can have encrypted
    reasoning messages, trying to resume a conversation with a different
    provider could lead to confusing "failed to decrypt" errors. (This is
    reproducible by starting a conversation using ChatGPT login and resuming
    it as a conversation that uses OpenAI models via Azure.)
    
    This changes `ListConversationsParams` to take a `model_providers:
    Option<Vec<String>>` and adds `model_provider` on each
    `ConversationSummary` it returns so these cases can be disambiguated.
    
    Note this ended up making changes to
    `codex-rs/core/src/rollout/tests.rs` because it had a number of cases
    where it expected `Some` for the value of `next_cursor`, but the list of
    rollouts was complete, so according to this docstring:
    
    
    https://github.com/openai/codex/blob/bcd64c7e7231d6316a2377d1525a0fa74f21b783/codex-rs/app-server-protocol/src/protocol.rs#L334-L337
    
    If there are no more items to return, then `next_cursor` should be
    `None`. This PR updates that logic.
    
    
    
    
    
    
    ---
    [//]: # (BEGIN SAPLING FOOTER)
    Stack created with [Sapling](https://sapling-scm.com). Best reviewed
    with [ReviewStack](https://reviewstack.dev/openai/codex/pull/5658).
    * #5803
    * #5793
    * __->__ #5658
  • Fixed flaky unit test (#5654)
    This PR fixes a test that is sporadically failing in CI.
    
    The problem is that two unit tests (the older `login_and_cancel_chatgpt`
    and a recently added
    `login_chatgpt_includes_forced_workspace_query_param`) exercise code
    paths that start the login server. The server binds to a hard-coded
    localhost port number, so attempts to start more than one server at the
    same time will fail. If these two tests happen to run concurrently, one
    of them will fail.
    
    To fix this, I've added a simple mutex. We can use this same mutex for
    future tests that use the same pattern.
  • Added model summary and risk assessment for commands that violate sandbox policy (#5536)
    This PR adds support for a model-based summary and risk assessment for
    commands that violate the sandbox policy and require user approval. This
    aids the user in evaluating whether the command should be approved.
    
    The feature works by taking a failed command and passing it back to the
    model and asking it to summarize the command, give it a risk level (low,
    medium, high) and a risk category (e.g. "data deletion" or "data
    exfiltration"). It uses a new conversation thread so the context in the
    existing thread doesn't influence the answer. If the call to the model
    fails or takes longer than 5 seconds, it falls back to the current
    behavior.
    
    For now, this is an experimental feature and is gated by a config key
    `experimental_sandbox_command_assessment`.
    
    Here is a screen shot of the approval prompt showing the risk assessment
    and summary.
    
    <img width="723" height="282" alt="image"
    src="https://github.com/user-attachments/assets/4597dd7c-d5a0-4e9f-9d13-414bd082fd6b"
    />
  • fix: resume lookup for gitignored CODEX_HOME (#5311)
    Walk the sessions tree instead of using file_search so gitignored
    CODEX_HOME directories can resume sessions. Add a regression test that
    covers a .gitignore'd sessions directory.
    
    Fixes #5247
    Fixes #5412
    
    ---------
    
    Co-authored-by: Owen Lin <owen@openai.com>
  • [app-server] add new account method API stubs (#5527)
    These are the schema definitions for the new JSON-RPC APIs associated
    with accounts. These are not wired up to business logic yet and will
    currently throw an internal error indicating these are unimplemented.