Commit Graph

1348 Commits

  • Fix: cap aggregated exec output consistently (#9759)
    ## WHAT?
    - Bias aggregated output toward stderr under contention (2/3 stderr, 1/3
    stdout) while keeping the 1 MiB cap.
    - Rebalance unused stderr share back to stdout when stderr is tiny to
    avoid underfilling.
    - Add tests for contention, small-stderr rebalance, and under-cap
    ordering (stdout then stderr).
    
    ## WHY?
    - Review feedback requested stderr priority under contention.
    - Avoid underfilled aggregated output when stderr is small while
    preserving a consistent cap across exec paths.
    
    ## HOW?
    - Update `aggregate_output` to compute stdout/stderr shares, then
    reassign unused capacity to the other stream.
    - Use the helper in both Windows and async exec paths.
    - Add regression tests for contention/rebalance and under-cap ordering.
    
    ## BEFORE
    ```rust
    // Best-effort aggregate: stdout then stderr (capped).
    let mut aggregated = Vec::with_capacity(
        stdout
            .text
            .len()
            .saturating_add(stderr.text.len())
            .min(EXEC_OUTPUT_MAX_BYTES),
    );
    append_capped(&mut aggregated, &stdout.text, EXEC_OUTPUT_MAX_BYTES);
    append_capped(&mut aggregated, &stderr.text, EXEC_OUTPUT_MAX_BYTES);
    let aggregated_output = StreamOutput {
        text: aggregated,
        truncated_after_lines: None,
    };
    ```
    
    ## AFTER
    ```rust
    fn aggregate_output(
        stdout: &StreamOutput<Vec<u8>>,
        stderr: &StreamOutput<Vec<u8>>,
    ) -> StreamOutput<Vec<u8>> {
        let total_len = stdout.text.len().saturating_add(stderr.text.len());
        let max_bytes = EXEC_OUTPUT_MAX_BYTES;
        let mut aggregated = Vec::with_capacity(total_len.min(max_bytes));
    
        if total_len <= max_bytes {
            aggregated.extend_from_slice(&stdout.text);
            aggregated.extend_from_slice(&stderr.text);
            return StreamOutput {
                text: aggregated,
                truncated_after_lines: None,
            };
        }
    
        // Under contention, reserve 1/3 for stdout and 2/3 for stderr; rebalance unused stderr to stdout.
        let want_stdout = stdout.text.len().min(max_bytes / 3);
        let want_stderr = stderr.text.len();
        let stderr_take = want_stderr.min(max_bytes.saturating_sub(want_stdout));
        let remaining = max_bytes.saturating_sub(want_stdout + stderr_take);
        let stdout_take = want_stdout + remaining.min(stdout.text.len().saturating_sub(want_stdout));
    
        aggregated.extend_from_slice(&stdout.text[..stdout_take]);
        aggregated.extend_from_slice(&stderr.text[..stderr_take]);
    
        StreamOutput {
            text: aggregated,
            truncated_after_lines: None,
        }
    }
    ```
    
    ## TESTS
    - [x] `just fmt`
    - [x] `just fix -p codex-core`
    - [x] `cargo test -p codex-core aggregate_output_`
    - [x] `cargo test -p codex-core`
    - [x] `cargo test --all-features`
    
    ## FIXES
    Fixes #9758
  • Fixing main and make plan mode reasoning effort medium (#9980)
    It's overthinking so much on high and going over the context window.
  • make plan prompt less detailed (#9977)
    This was too much to ask for
  • make cached web_search client-side default (#9974)
    [Experiment](https://console.statsig.com/50aWbk2p4R76rNX9lN5VUw/experiments/codex_web_search_rollout/summary)
    for default cached `web_search` completed; cached chosen as default.
    
    Update client to reflect that.
  • plan prompt (#9975)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • prompt (#9970)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • prompt final (#9969)
    hopefully final this time (at least tonight) >_<
  • Improve plan mode prompt (#9968)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • plan prompt v7 (#9966)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • fix: handle all web_search actions and in progress invocations (#9960)
    ### Summary
    - Parse all `web_search` tool actions (`search`, `find_in_page`,
    `open_page`).
    - Previously we only parsed + displayed `search`, which made the TUI
    appear to pause when the other actions were being used.
    - Show in progress `web_search` calls as `Searching the web`
      - Previously we only showed completed tool calls
    
    <img width="308" height="149" alt="image"
    src="https://github.com/user-attachments/assets/90a4e8ff-b06a-48ff-a282-b57b31121845"
    />
    
    ### Tests
    Added + updated tests, tested locally
    
    ### Follow ups
    Update VSCode extension to display these as well
  • Use test_codex more (#9961)
    Reduces boilderplate.
  • Reject request_user_input outside Plan/Pair (#9955)
    ## Context
    
    Previous work in https://github.com/openai/codex/pull/9560 only rejected
    `request_user_input` in Execute and Custom modes. Since then, additional
    modes
    (e.g., Code) were added, so the guard should be mode-agnostic.
    
    ## What changed
    
    - Switch the handler to an allowlist: only Plan and PairProgramming are
    allowed
    - Return the same error for any other mode (including Code)
    - Add a Code-mode rejection test alongside the existing Execute/Custom
    tests
    
    ## Why
    
    This prevents `request_user_input` from being used in modes where it is
    not
    intended, even as new modes are introduced.
  • Add composer config and shared menu surface helpers (#9891)
    Centralize built-in slash-command gating and extract shared menu-surface
    helpers.
    
    - Add bottom_pane::slash_commands and reuse it from composer + command
    popup.
    - Introduce ChatComposerConfig + shared menu surface rendering without
    changing default behavior.
  • plan prompt (#9943)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • Add MCP server scopes config and use it as fallback for OAuth login (#9647)
    ### Motivation
    - Allow MCP OAuth flows to request scopes defined in `config.toml`
    instead of requiring users to always pass `--scopes` on the CLI.
    CLI/remote parameters should still override config values.
    
    ### Description
    - Add optional `scopes: Option<Vec<String>>` to `McpServerConfig` and
    `RawMcpServerConfig`, and propagate it through deserialization and the
    built config types.
    - Serialize `scopes` into the MCP server TOML via
    `serialize_mcp_server_table` in `core/src/config/edit.rs` and include
    `scopes` in the generated config schema (`core/config.schema.json`).
    - CLI: update `codex-rs/cli/src/mcp_cmd.rs` `run_login` to fall back to
    `server.scopes` when the `--scopes` flag is empty, with explicit CLI
    scopes still taking precedence.
    - App server: update
    `codex-rs/app-server/src/codex_message_processor.rs`
    `mcp_server_oauth_login` to use `params.scopes.or_else(||
    server.scopes.clone())` so the RPC path also respects configured scopes.
    - Update many test fixtures to initialize the new `scopes` field (set to
    `None`) so test code builds with the new struct field.
    
    ### Testing
    - Ran config tooling and formatters: `just write-config-schema`
    (succeeded), `just fmt` (succeeded), and `just fix -p codex-core`, `just
    fix -p codex-cli`, `just fix -p codex-app-server` (succeeded where
    applicable).
    - Ran unit tests for the CLI: `cargo test -p codex-cli` (passed).
    - Ran unit tests for core: `cargo test -p codex-core` (ran; many tests
    passed but several failed, including model refresh/403-related tests,
    shell snapshot/timeouts, and several `unified_exec` expectations).
    - Ran app-server tests: `cargo test -p codex-app-server` (ran; many
    integration-suite tests failed due to mocked/remote HTTP 401/403
    responses and wiremock expectations).
    
    If you want, I can split the tests into smaller focused runs or help
    debug the failing integration tests (they appear to be unrelated to the
    config change and stem from external HTTP/mocking behaviors encountered
    during the test runs).
    
    ------
    [Codex
    Task](https://chatgpt.com/codex/tasks/task_i_69718f505914832ea1f334b3ba064553)
  • Aligned feature stage names with public feature maturity stages (#9929)
    We've recently standardized a [feature maturity
    model](https://developers.openai.com/codex/feature-maturity) that we're
    using in our docs and support forums to communicate expectations to
    users. This PR updates the internal stage names and descriptions to
    match.
    
    This change involves a simple internal rename and updates to a few
    user-visible strings. No functional change.
  • Add thread/unarchive to restore archived rollouts (#9843)
    ## Summary
    - Adds a new `thread/unarchive` RPC to move archived thread rollouts
    back into the active `sessions/` tree.
    
    ## What changed
    - **Protocol**
      - Adds `thread/unarchive` request/response types and wiring.
    - **Server**
      - Implements `thread_unarchive` in the app server.
      - Validates the archived rollout path and thread ID.
    - Restores the rollout to `sessions/YYYY/MM/DD/...` based on the rollout
    filename timestamp.
    - **Core**
    - Adds `find_archived_thread_path_by_id_str` helper for archived
    rollouts.
    - **Docs**
      - Documents the new RPC and usage example.
    - **Tests**
      - Adds an end-to-end server test that:
        1) starts a thread,
        2) archives it,
        3) unarchives it,
        4) asserts the file is restored to `sessions/`.
    
    ## How to use
    ```json
    { "method": "thread/unarchive", "id": 24, "params": { "threadId": "<thread-id>" } }
    ```
    
    ## Author Codex Session
    
    `codex resume 019bf158-54b6-7960-a696-9d85df7e1bc1` (soon I'll make this
    kind of session UUID forkable by anyone with the right
    `session_object_storage_url` line in their config, but for now just
    pasting it here for my reference)
  • prompt (#9928)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • fix: attempt to reduce high cpu usage when using collab (#9776)
    Reproduce with a prompt like this with collab enabled:
    ```
    Examine the code at <some subdirectory with a deeply nested project>.  Find the most urgent issue to resolve and describe it to me.
    ```
    
    Existing behavior causes the top-level agent to busy wait on subagents.
  • Fix flakey shell snapshot test (#9919)
    Sometimes fails with:
    
    ```
    failures:
    
      ---- shell_snapshot::tests::timed_out_snapshot_shell_is_terminated stdout ----
    
      thread 'shell_snapshot::tests::timed_out_snapshot_shell_is_terminated' panicked at codex-rs/core/src/shell_snapshot.rs:588:9:
      expected timeout error, got Failed to execute sh
    
      Caused by:
          Text file busy (os error 26)
    
    
      failures:
          shell_snapshot::tests::timed_out_snapshot_shell_is_terminated
    
      test result: FAILED. 815 passed; 1 failed; 4 ignored; 0 measured; 0 filtered out; finished in 18.00s
    ```
  • Feat: add isOther to question returned by request user input tool (#9890)
    ### Summary
    Add `isOther` to question object from request_user_input tool input and
    remove `other` option from the tool prompt to better handle tool input.
  • Fix up config disabled err msg (#9916)
    **Before:**
    <img width="745" height="375" alt="image"
    src="https://github.com/user-attachments/assets/d6c23562-b87f-4af9-8642-329aab8e594d"
    />
    
    **After:**
    <img width="1042" height="354" alt="image"
    src="https://github.com/user-attachments/assets/c9a2413c-c945-4c34-8b7e-c6c9b8fbf762"
    />
    
    Two changes:
    1. only display if there is a `config.toml` that is skipped (i.e. if
    there is just `.codex/skills` but no `.codex/config.toml` we do not
    display the error)
    2. clarify the implications and the fix in the error message.
  • feat: dynamic tools injection (#9539)
    ## Summary
    Add dynamic tool injection to thread startup in API v2, wire dynamic
    tool calls through the app server to clients, and plumb responses back
    into the model tool pipeline.
    
    ### Flow (high level)
    - Thread start injects `dynamic_tools` into the model tool list for that
    thread (validation is done here).
    - When the model emits a tool call for one of those names, core raises a
    `DynamicToolCallRequest` event.
    - The app server forwards it to the client as `item/tool/call`, waits
    for the client’s response, then submits a `DynamicToolResponse` back to
    core.
    - Core turns that into a `function_call_output` in the next model
    request so the model can continue.
    
    ### What changed
    - Added dynamic tool specs to v2 thread start params and protocol types;
    introduced `item/tool/call` (request/response) for dynamic tool
    execution.
    - Core now registers dynamic tool specs at request time and routes those
    calls via a new dynamic tool handler.
    - App server validates tool names/schemas, forwards dynamic tool call
    requests to clients, and publishes tool outputs back into the session.
    - Integration tests
  • chore(core) move model_instructions_template config (#9871)
    ## Summary
    Move `model_instructions_template` config to the experimental slug while
    we iterate on this feature
    
    ## Testing
    - [x] Tested locally, unit tests still pass
  • feat(tui) /personality (#9718)
    ## Summary
    Adds /personality selector in the TUI, which leverages the new core
    interface in #9644
    
    Notes:
    - We are doing some of our own state management for model_info loading
    here, but not sure if that's ideal. open to opinions on simpler
    approach, but would like to avoid blocking on a larger refactor
    - Right now, the `/personality` selector just hides when the model
    doesn't support it. we can update this behavior down the line
    
    ## Testing
    - [x] Tested locally
    - [x] Added snapshot tests
  • Plan prompt (#9877)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • Prompt (#9874)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • feat: cap number of agents (#9855)
    Adding more guards to agent:
    * Max depth or 1 (i.e. a sub-agent can't spawn another one)
    * Max 12 sub-agents in total
  • Mark collab as beta (#9834)
    Co-authored-by: jif-oai <jif@openai.com>
  • Use collaboration mode masks without mutating base settings (#9806)
    Keep an unmasked base collaboration mode and apply the active mask on
    demand. Simplify the TUI mask helpers and update tests/docs to match the
    mask contract.
  • feat: ephemeral threads (#9765)
    Add ephemeral threads capabilities. Only exposed through the
    `app-server` v2
    
    The idea is to disable the rollout recorder for those threads.
  • Another round of improvements for config error messages (#9746)
    In a [recent PR](https://github.com/openai/codex/pull/9182), I made some
    improvements to config error messages so errors didn't leave app server
    clients in a dead state. This is a follow-on PR to make these error
    messages more readable and actionable for both TUI and GUI users. For
    example, see #9668 where the user was understandably confused about the
    source of the problem and how to fix it.
    
    The improved error message:
    1. Clearly identifies the config file where the error was found (which
    is more important now that we support layered configs)
    2. Provides a line and column number of the error
    3. Displays the line where the error occurred and underlines it
    
    For example, if my `config.toml` includes the following:
    ```toml
    [features]
    collaboration_modes = "true"
    ```
    
    Here's the current CLI error message:
    ```
    Error loading config.toml: invalid type: string "true", expected a boolean in `features`
    ```
    
    And here's the improved message:
    ```
    Error loading config.toml:
    /Users/etraut/.codex/config.toml:43:23: invalid type: string "true", expected a boolean
       |
    43 | collaboration_modes = "true"
       |                       ^^^^^^
    ```
    
    The bulk of the new logic is contained within a new module
    `config_loader/diagnostics.rs` that is responsible for calculating the
    text range for a given toml path (which is more involved than I would
    have expected).
    
    In addition, this PR adds the file name and text range to the
    `ConfigWarningNotification` app server struct. This allows GUI clients
    to present the user with a better error message and an optional link to
    open the errant config file. This was a suggestion from @.bolinfest when
    he reviewed my previous PR.
  • change collaboration mode to struct (#9793)
    Shouldn't cause behavioral change
  • Fix resume picker when user event appears after head (#9512)
    Fixes #9501
    
    Contributing guide:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    ## Summary
    The resume picker requires a session_meta line and at least one
    user_message event within the initial head scan. Some rollout files
    contain multiple session_meta entries before the first user_message, so
    the user event can fall outside the default head window and the session
    is omitted from the picker even though it is resumable by ID.
    
    This PR keeps the head summary bounded but extends scanning for a
    user_message once a session_meta has been observed. The summary still
    caps stored head entries, but we allow a small, bounded extra scan to
    find the first user event so valid sessions are not filtered out.
    
    ## Changes
    - Continue scanning past the head limit (bounded) when session_meta is
    present but no user_message has been seen yet.
    - Mark session_meta as seen even if the head summary buffer is already
    full.
    - Add a regression test with multiple session_meta lines before the
    first user_message.
    
    ## Why This Is Safe
    - The head summary remains bounded to avoid unbounded memory usage.
    - The extra scan is capped (USER_EVENT_SCAN_LIMIT) and only triggers
    after a session_meta is seen.
    - Behavior is unchanged for typical files where the user_message appears
    early.
    
    ## Testing
    - cargo test -p codex-core --lib
    test_list_threads_scans_past_head_for_user_event
  • Select default model from filtered presets (#9782)
    Pick the first available preset after auth filtering for default
    selection.
  • Print warning if we skip config loading (#9611)
    https://github.com/openai/codex/pull/9533 silently ignored config if
    untrusted. Instead, we still load it but disable it. Maybe we shouldn't
    try to parse it either...
    
    <img width="939" height="515" alt="Screenshot 2026-01-21 at 14 56 38"
    src="https://github.com/user-attachments/assets/e753cc22-dd99-4242-8ffe-7589e85bef66"
    />