Commit Graph

131 Commits

  • Set verbosity to low for 5.1 (#6568)
    And improve test coverage
  • Don't retry "insufficient_quota" errors (#6340)
    This PR makes an "insufficient quota" error fatal so we don't attempt to
    retry it multiple times in the agent loop.
    
    We have multiple bug reports from users about intermittent retry
    behaviors, and this could explain some of them. With this change, we'll
    eliminate the retries and surface a clear error message.
    
    The PR is a nearly identical copy of [this
    PR](https://github.com/openai/codex/pull/4837) contributed by
    @abimaelmartell. The original PR has gone stale. Rather than wait for
    the contributor to resolve merge conflicts, I wanted to get this change
    in.
  • Improved token refresh handling to address "Re-connecting" behavior (#6231)
    Currently, when the access token expires, we attempt to use the refresh
    token to acquire a new access token. This works most of the time.
    However, there are situations where the refresh token is expired,
    exhausted (already used to perform a refresh), or revoked. In those
    cases, the current logic treats the error as transient and attempts to
    retry it repeatedly.
    
    This PR changes the token refresh logic to differentiate between
    permanent and transient errors. It also changes callers to treat the
    permanent errors as fatal rather than retrying them. And it provides
    better error messages to users so they understand how to address the
    problem. These error messages should also help us further understand why
    we're seeing examples of refresh token exhaustion.
    
    Here is the error message in the CLI. The same text appears within the
    extension.
    
    <img width="863" height="38" alt="image"
    src="https://github.com/user-attachments/assets/7ffc0d08-ebf0-4900-b9a9-265064202f4f"
    />
    
    I also correct the spelling of "Re-connecting", which shouldn't have a
    hyphen in it.
    
    Testing: I manually tested these code paths by adding temporary code to
    programmatically cause my refresh token to be exhausted (by calling the
    token refresh endpoint in a tight loop more than 50 times). I then
    simulated an access token expiration, which caused the token refresh
    logic to be invoked. I confirmed that the updated logic properly handled
    the error condition.
    
    Note: We earlier discussed the idea of forcefully logging out the user
    at the point where token refresh failed. I made several attempts to do
    this, and all of them resulted in a bad UX. It's important to surface
    this error to users in a way that explains the problem and tells them
    that they need to log in again. We also previously discussed deleting
    the auth.json file when this condition is detected. That also creates
    problems because it effectively changes the auth status from logged in
    to logged out, and this causes odd failures and inconsistent UX. I think
    it's therefore better not to delete auth.json in this case. If the user
    closes the CLI or VSCE and starts it again, we properly detect that the
    access token is expired and the refresh token is "dead", and we force
    the user to go through the login flow at that time.
    
    This should address aspects of #6191, #5679, and #5505
  • Parse the Azure OpenAI rate limit message (#5956)
    Fixes #4161
    
    Currently Codex uses a regex to parse the "Please try again in 1.898s"
    OpenAI-style rate limit message, so that it can wait the correct
    duration before retrying. Azure OpenAI returns a different error that
    looks like "Rate limit exceeded. Try again in 35 seconds."
    
    This PR extends the regex and parsing code to match in a more fuzzy
    manner, handling anything matching the pattern "try again in
    \<duration>\<unit>".
  • override verbosity for gpt-5-codex (#6007)
    we are seeing [reports](https://github.com/openai/codex/issues/6004) of
    users having verbosity in their config.toml and facing issues.
    gpt-5-codex doesn't accept other values rather than medium for
    verbosity.
  • Send delegate header (#5942)
    Send delegate type header
  • Add item streaming events (#5546)
    Adds AgentMessageContentDelta, ReasoningContentDelta,
    ReasoningRawContentDelta item streaming events while maintaining
    compatibility for old events.
    
    ---------
    
    Co-authored-by: Owen Lin <owen@openai.com>
  • Delegate review to codex instance (#5572)
    In this PR, I am exploring migrating task kind to an invocation of
    Codex. The main reason would be getting rid off multiple
    `ConversationHistory` state and streamlining our context/history
    management.
    
    This approach depends on opening a channel between the sub-codex and
    codex. This channel is responsible for forwarding `interactive`
    (`approvals`) and `non-interactive` events. The `task` is responsible
    for handling those events.
    
    This opens the door for implementing `codex as a tool`, replacing
    `compact` and `review`, and potentially subagents.
    
    One consideration is this code is very similar to `app-server` specially
    in the approval part. If in the future we wanted an interactive
    `sub-codex` we should consider using `codex-mcp`
  • Made token refresh code resilient to missing id_token (#5782)
    This PR does the following:
    1. Changes `try_refresh_token` to handle the case where the endpoint
    returns a response without an `id_token`. The OpenID spec indicates that
    this field is optional and clients should not assume it's present.
    2. Changes the `attempt_stream_responses` to propagate token refresh
    errors rather than silently ignoring them.
    3. Fixes a typo in a couple of error messages (unrelated to the above,
    but something I noticed in passing) - "reconnect" should be spelled
    without a hyphen.
    
    This PR does not implement the additional suggestion from @pakrym-oai
    that we should sign out when receiving `refresh_token_expired` from the
    refresh endpoint. Leaving this as a follow-on because I'm undecided on
    whether this should be implemented in `try_refresh_token` or its
    callers.
  • Added model summary and risk assessment for commands that violate sandbox policy (#5536)
    This PR adds support for a model-based summary and risk assessment for
    commands that violate the sandbox policy and require user approval. This
    aids the user in evaluating whether the command should be approved.
    
    The feature works by taking a failed command and passing it back to the
    model and asking it to summarize the command, give it a risk level (low,
    medium, high) and a risk category (e.g. "data deletion" or "data
    exfiltration"). It uses a new conversation thread so the context in the
    existing thread doesn't influence the answer. If the call to the model
    fails or takes longer than 5 seconds, it falls back to the current
    behavior.
    
    For now, this is an experimental feature and is gated by a config key
    `experimental_sandbox_command_assessment`.
    
    Here is a screen shot of the approval prompt showing the risk assessment
    and summary.
    
    <img width="723" height="282" alt="image"
    src="https://github.com/user-attachments/assets/4597dd7c-d5a0-4e9f-9d13-414bd082fd6b"
    />
  • Log more types of request IDs (#5645)
    Different services return different sets of IDs, log all of them to
    simplify debugging.
  • Add CodexHttpClient wrapper with request logging (#5564)
    ## Summary
    - wrap the default reqwest::Client inside a new
    CodexHttpClient/CodexRequestBuilder pair and log the HTTP method, URL,
    and status for each request
    - update the auth/model/provider plumbing to use the new builder helpers
    so headers and bearer auth continue to be applied consistently
    - add the shared `http` dependency that backs the header conversion
    helpers
    
    ## Testing
    - `CODEX_SANDBOX=seatbelt CODEX_SANDBOX_NETWORK_DISABLED=1 cargo test -p
    codex-core`
    - `CODEX_SANDBOX=seatbelt CODEX_SANDBOX_NETWORK_DISABLED=1 cargo test -p
    codex-chatgpt`
    - `CODEX_SANDBOX=seatbelt CODEX_SANDBOX_NETWORK_DISABLED=1 cargo test -p
    codex-tui`
    
    ------
    https://chatgpt.com/codex/tasks/task_i_68fa5038c17483208b1148661c5873be
  • feat: add experimental_bearer_token option to model provider definition (#5467)
    While we do not want to encourage users to hardcode secrets in their
    `config.toml` file, it should be possible to pass an API key
    programmatically. For example, when using `codex app-server`, it is
    possible to pass a "bag of configuration" as part of the
    `NewConversationParams`:
    
    https://github.com/openai/codex/blob/682d05512f2992dd0657f1078d4146f15c744d7a/codex-rs/app-server-protocol/src/protocol.rs#L248-L251
    
    When using `codex app-server`, it's not practical to change env vars of
    the `codex app-server` process on the fly (which is how we usually read
    API key values), so this helps with that.
  • [app-server] read rate limits API (#5302)
    Adds a `GET account/rateLimits/read` API to app-server. This calls the
    codex backend to fetch the user's current rate limits.
    
    This would be helpful in checking rate limits without having to send a
    message.
    
    For calling the codex backend usage API, I generated the types and
    manually copied the relevant ones into `codex-backend-openapi-types`.
    It'll be nice to extend our internal openapi generator to support Rust
    so we don't have to run these manual steps.
    
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
  • chore: rework tools execution workflow (#5278)
    Re-work the tool execution flow. Read `orchestrator.rs` to understand
    the structure
  • Use int timestamps for rate limit reset_at (#5383)
    The backend will be returning unix timestamps (seconds since epoch)
    instead of RFC 3339 strings. This will make it more ergonomic for
    developers to integrate against - no string parsing.
  • Auto compact at ~90% (#5292)
    Users now hit a window exceeded limit and they usually don't know what
    to do. This starts auto compact at ~90% of the window.
  • fix: switch rate limit reset handling to timestamps (#5304)
    This change ensures that we store the absolute time instead of relative
    offsets of when the primary and secondary rate limits will reset.
    Previously these got recalculated relative to current time, which leads
    to the displayed reset times to change over time, including after doing
    a codex resume.
    
    For previously changed sessions, this will cause the reset times to not
    show due to this being a breaking change:
    <img width="524" height="55" alt="Screenshot 2025-10-17 at 5 14 18 PM"
    src="https://github.com/user-attachments/assets/53ebd43e-da25-4fef-9c47-94a529d40265"
    />
    
    Fixes https://github.com/openai/codex/issues/4761
  • Improve error decoding response body error (#5263)
    Split Reqwest error into separate error:
    1. One for streaming response
    2. One for initial connection failing
    
    Include request_id where possible.
    
    <img width="1791" height="116" alt="image"
    src="https://github.com/user-attachments/assets/549aa330-acfa-496a-9898-77fa58436316"
    />
  • [codex][otel] propagate user email in otel events (#5223)
    include user email into otel events for proper user-level attribution in
    case of workspace setup
  • feat: add header for task kind (#5142)
    Add a header in the responses API request for the task kind (compact,
    review, ...) for observability purpose
    The header name is `codex-task-type`
  • Use assert_matches (#4756)
    assert_matches is soon to be in std but is experimental for now.
  • feat: parallel tool calls (#4663)
    Add parallel tool calls. This is configurable at model level and tool
    level
  • Surface context window error to the client (#4675)
    In the past, we were treating `input exceeded context window` as a
    streaming error and retrying on it. Retrying on it has no point because
    it won't change the behavior. In this PR, we surface the error to the
    client without retry and also send a token count event to indicate that
    the context window is full.
    
    <img width="650" height="125" alt="image"
    src="https://github.com/user-attachments/assets/c26b1213-4c27-4bfc-90f4-51a270a3efd5"
    />
  • Include request ID in the error message (#4572)
    To help with issue debugging
    <img width="1414" height="253" alt="image"
    src="https://github.com/user-attachments/assets/254732df-44ac-4252-997a-6c5e0927355b"
    />
  • fix: remove mcp-types from app server protocol (#4537)
    We continue the separation between `codex app-server` and `codex
    mcp-server`.
    
    In particular, we introduce a new crate, `codex-app-server-protocol`,
    and migrate `codex-rs/protocol/src/mcp_protocol.rs` into it, renaming it
    `codex-rs/app-server-protocol/src/protocol.rs`.
    
    Because `ConversationId` was defined in `mcp_protocol.rs`, we move it
    into its own file, `codex-rs/protocol/src/conversation_id.rs`, and
    because it is referenced in a ton of places, we have to touch a lot of
    files as part of this PR.
    
    We also decide to get away from proper JSON-RPC 2.0 semantics, so we
    also introduce `codex-rs/app-server-protocol/src/jsonrpc_lite.rs`, which
    is basically the same `JSONRPCMessage` type defined in `mcp-types`
    except with all of the `"jsonrpc": "2.0"` removed.
    
    Getting rid of `"jsonrpc": "2.0"` makes our serialization logic
    considerably simpler, as we can lean heavier on serde to serialize
    directly into the wire format that we use now.
  • OpenTelemetry events (#2103)
    ### Title
    
    ## otel
    
    Codex can emit [OpenTelemetry](https://opentelemetry.io/) **log events**
    that
    describe each run: outbound API requests, streamed responses, user
    input,
    tool-approval decisions, and the result of every tool invocation. Export
    is
    **disabled by default** so local runs remain self-contained. Opt in by
    adding an
    `[otel]` table and choosing an exporter.
    
    ```toml
    [otel]
    environment = "staging"   # defaults to "dev"
    exporter = "none"          # defaults to "none"; set to otlp-http or otlp-grpc to send events
    log_user_prompt = false    # defaults to false; redact prompt text unless explicitly enabled
    ```
    
    Codex tags every exported event with `service.name = "codex-cli"`, the
    CLI
    version, and an `env` attribute so downstream collectors can distinguish
    dev/staging/prod traffic. Only telemetry produced inside the
    `codex_otel`
    crate—the events listed below—is forwarded to the exporter.
    
    ### Event catalog
    
    Every event shares a common set of metadata fields: `event.timestamp`,
    `conversation.id`, `app.version`, `auth_mode` (when available),
    `user.account_id` (when available), `terminal.type`, `model`, and
    `slug`.
    
    With OTEL enabled Codex emits the following event types (in addition to
    the
    metadata above):
    
    - `codex.api_request`
      - `cf_ray` (optional)
      - `attempt`
      - `duration_ms`
      - `http.response.status_code` (optional)
      - `error.message` (failures)
    - `codex.sse_event`
      - `event.kind`
      - `duration_ms`
      - `error.message` (failures)
      - `input_token_count` (completion only)
      - `output_token_count` (completion only)
      - `cached_token_count` (completion only, optional)
      - `reasoning_token_count` (completion only, optional)
      - `tool_token_count` (completion only)
    - `codex.user_prompt`
      - `prompt_length`
      - `prompt` (redacted unless `log_user_prompt = true`)
    - `codex.tool_decision`
      - `tool_name`
      - `call_id`
    - `decision` (`approved`, `approved_for_session`, `denied`, or `abort`)
      - `source` (`config` or `user`)
    - `codex.tool_result`
      - `tool_name`
      - `call_id`
      - `arguments`
      - `duration_ms` (execution time for the tool)
      - `success` (`"true"` or `"false"`)
      - `output`
    
    ### Choosing an exporter
    
    Set `otel.exporter` to control where events go:
    
    - `none` – leaves instrumentation active but skips exporting. This is
    the
      default.
    - `otlp-http` – posts OTLP log records to an OTLP/HTTP collector.
    Specify the
      endpoint, protocol, and headers your collector expects:
    
      ```toml
      [otel]
      exporter = { otlp-http = {
        endpoint = "https://otel.example.com/v1/logs",
        protocol = "binary",
        headers = { "x-otlp-api-key" = "${OTLP_TOKEN}" }
      }}
      ```
    
    - `otlp-grpc` – streams OTLP log records over gRPC. Provide the endpoint
    and any
      metadata headers:
    
      ```toml
      [otel]
      exporter = { otlp-grpc = {
        endpoint = "https://otel.example.com:4317",
        headers = { "x-otlp-meta" = "abc123" }
      }}
      ```
    
    If the exporter is `none` nothing is written anywhere; otherwise you
    must run or point to your
    own collector. All exporters run on a background batch worker that is
    flushed on
    shutdown.
    
    If you build Codex from source the OTEL crate is still behind an `otel`
    feature
    flag; the official prebuilt binaries ship with the feature enabled. When
    the
    feature is disabled the telemetry hooks become no-ops so the CLI
    continues to
    function without the extra dependencies.
    
    ---------
    
    Co-authored-by: Anton Panasenko <apanasenko@openai.com>
  • /status followup (#4304)
    - Render `send a message to load usage data` in the beginning of the
    session
    - Render `data not available yet` if received no rate limits 
    - nit case
    - Deleted stall snapshots that were moved to
    `codex-rs/tui/src/status/snapshots`
  • fix (#4251)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
  • chore: refactor attempt_stream_responses() out of stream_responses() (#4194)
    I would like to be able to swap in a different way to resolve model
    sampling requests, so this refactoring consolidates things behind
    `attempt_stream_responses()` to make that easier. Ideally, we would
    support an in-memory backend that we can use in our integration tests,
    for example.
  • chore: upgrade to Rust 1.90 (#4124)
    Inspired by Dependabot's attempt to do this:
    https://github.com/openai/codex/pull/4029
    
    The new version of Clippy found some unused structs that are removed in
    this PR.
    
    Though nothing stood out to me in the Release Notes in terms of things
    we should start to take advantage of:
    https://blog.rust-lang.org/2025/09/18/Rust-1.90.0/.
  • Add Reset in for rate limits (#4111)
    - Parse the headers
    - Reorganize the struct because it's getting too long
    - show the resets at in the tui
    
    <img width="324" height="79" alt="image"
    src="https://github.com/user-attachments/assets/ca15cd48-f112-4556-91ab-1e3a9bc4683d"
    />
  • Send limits when getting rate limited (#4102)
    Users need visibility on rate limits when they are rate limited.
  • Add exec output-schema parameter (#4079)
    Adds structured output to `exec` via the `--structured-output`
    parameter.
  • chore: clippy on redundant closure (#4058)
    Add redundant closure clippy rules and let Codex fix it by minimising
    FQP
  • Forward Rate limits to the UI (#3965)
    We currently get information about rate limits in the response headers.
    We want to forward them to the clients to have better transparency.
    UI/UX plans have been discussed and this information is needed.
  • Add Azure Responses API workaround (#3528)
    Azure Responses API doesn't work well with store:false and response
    items.
    
    If store = false and id is sent an error is thrown that ID is not found
    If store = false and id is not sent an error is thrown that ID is
    required
    
    Add detection for Azure urls and add a workaround to preserve reasoning
    item IDs and send store:true
  • feat: context compaction (#3446)
    ## Compact feature:
    1. Stops the model when the context window become too large
    2. Add a user turn, asking for the model to summarize
    3. Build a bridge that contains all the previous user message + the
    summary. Rendered from a template
    4. Start sampling again from a clean conversation with only that bridge
  • feat: reasoning effort as optional (#3527)
    Allow the reasoning effort to be optional
  • Log cf-ray header in client traces (#3488)
    ## Summary
    - log the `cf-ray` header when tracing HTTP responses in the Codex
    client
    - keep existing response status logging unchanged
    
    ## Testing
    - just fmt
    - just fix -p codex-core
    - cargo test -p codex-core *(fails:
    suite::client::azure_overrides_assign_properties_used_for_responses_url,
    suite::client::env_var_overrides_loaded_auth)*
    
    ------
    https://chatgpt.com/codex/tasks/task_i_68c31640dacc83209be131baf91611cd
  • Use PlanType enum when formatting usage-limit CTA (#3495)
    - Started using Play type struct
    - Added CTA for team/business 
    - Refactored a bit to unify the logic