Commit Graph

73 Commits

  • feat(app-server, core): allow text + image content items for dynamic tool outputs (#10567)
    Took over the work that @aaronl-openai started here:
    https://github.com/openai/codex/pull/10397
    
    Now that app-server clients are able to set up custom tools (called
    `dynamic_tools` in app-server), we should expose a way for clients to
    pass in not just text, but also image outputs. This is something the
    Responses API already supports for function call outputs, where you can
    pass in either a string or an array of content outputs (text, image,
    file):
    https://platform.openai.com/docs/api-reference/responses/create#responses_create-input-input_item_list-item-function_tool_call_output-output-array-input_image
    
    So let's just plumb it through in Codex (with the caveat that we only
    support text and image for now). This is implemented end-to-end across
    app-server v2 protocol types and core tool handling.
    
    ## Breaking API change
    NOTE: This introduces a breaking change with dynamic tools, but I think
    it's ok since this concept was only recently introduced
    (https://github.com/openai/codex/pull/9539) and it's better to get the
    API contract correct. I don't think there are any real consumers of this
    yet (not even the Codex App).
    
    Old shape:
    `{ "output": "dynamic-ok", "success": true }`
    
    New shape:
    ```
    {
        "contentItems": [
          { "type": "inputText", "text": "dynamic-ok" },
          { "type": "inputImage", "imageUrl": "data:image/png;base64,AAA" }
        ]
      "success": true
    }
    ```
  • feat(personality) introduce model_personality config (#9459)
    ## Summary
    Introduces the concept of a config model_personality. I would consider
    this an MVP for testing out the feature. There are a number of
    follow-ups to this PR:
    
    - More sophisticated templating with validation
    - In-product experience to manage this
    
    ## Testing
    - [x] Testing locally
  • fix(core) Preserve base_instructions in SessionMeta (#9427)
    ## Summary
    This PR consolidates base_instructions onto SessionMeta /
    SessionConfiguration, so we ensure `base_instructions` is set once per
    session and should be (mostly) immutable, unless:
    - overridden by config on resume / fork
    - sub-agent tasks, like review or collab
    
    
    In a future PR, we should convert all references to `base_instructions`
    to consistently used the typed struct, so it's less likely that we put
    other strings there. See #9423. However, this PR is already quite
    complex, so I'm deferring that to a follow-up.
    
    ## Testing
    - [x] Added a resume test to assert that instructions are preserved. In
    particular, `resume_switches_models_preserves_base_instructions` fails
    against main.
    
    Existing test coverage thats assert base instructions are preserved
    across multiple requests in a session:
    - Manual compact keeps baseline instructions:
    core/tests/suite/compact.rs:199
    - Auto-compact keeps baseline instructions:
    core/tests/suite/compact.rs:1142
    - Prompt caching reuses the same instructions across two requests:
    core/tests/suite/prompt_caching.rs:150 and
    core/tests/suite/prompt_caching.rs:157
    - Prompt caching with explicit expected string across two requests:
    core/tests/suite/prompt_caching.rs:213 and
    core/tests/suite/prompt_caching.rs:222
    - Resume with model switch keeps original instructions:
    core/tests/suite/resume.rs:136
    - Compact/resume/fork uses request 0 instructions for later expected
    payloads: core/tests/suite/compact_resume_fork.rs:215
  • Merge Modelfamily into modelinfo (#8763)
    - Merge ModelFamily into ModelInfo
    - Remove logic for adding instructions to apply patch
    - Add compaction limit and visible context window to `ModelInfo`
  • add web_search_cached flag (#8795)
    Add `web_search_cached` feature to config. Enables `web_search` tool
    with access only to cached/indexed results (see
    [docs](https://platform.openai.com/docs/guides/tools-web-search#live-internet-access)).
    
    This takes precedence over the existing `web_search_request`, which
    continues to enable `web_search` over live results as it did before.
    
    `web_search_cached` is disabled for review mode, as `web_search_request`
    is.
  • Rename OpenAI models to models manager (#8346)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • Chore: limit find family visability (#7891)
    a little bit more code quality of life
  • Migrate model family to models manager (#7565)
    This PR moves `ModelsFamily` to `openai_models`. It also propagates
    `ModelsManager` to session services and use it to drive model family. We
    also make `derive_default_model_family` private because it's a step
    towards what we want: one place that gives model configuration.
    
    This is a second step at having one source of truth for models
    information and config: `ModelsManager`.
    
    Next steps would be to remove `ModelsFamily` from config. That's massive
    because it's being used in 41 occasions mostly pre launching `codex`.
    Also, we need to make `find_family_for_model` private. It's also big
    because it's being used in 21 occasions ~ all tests.
  • fix: read max_output_tokens param from config (#4139)
    Request param `max_output_tokens` is documented in
    `https://github.com/openai/codex/blob/main/docs/config.md`,
    but nowhere uses the item in config, this commit read it from config for
    GPT responses API.
    
    see https://github.com/openai/codex/issues/4138 for issue report.
    
    Signed-off-by: Yorling <shallowcloud@yeah.net>
  • feat: arcticfox in the wild (#6906)
    <img width="485" height="600" alt="image"
    src="https://github.com/user-attachments/assets/4341740d-dd58-4a3e-b69a-33a3be0606c5"
    />
    
    ---------
    
    Co-authored-by: jif-oai <jif@openai.com>
  • Move shell to use truncate_text (#6842)
    Move shell to use the configurable `truncate_text`
    
    ---------
    
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • shell_command returns freeform output (#6860)
    Instead of returning structured out and then re-formatting it into
    freeform, return the freeform output from shell_command tool.
    
    Keep `shell` as the default tool for GPT-5.
  • fix: typos in model picker (#6859)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • Update defaults to gpt-5.1 (#6652)
    ## Summary
    - update documentation, example configs, and automation defaults to
    reference gpt-5.1 / gpt-5.1-codex
    - bump the CLI and core configuration defaults, model presets, and error
    messaging to the new models while keeping the model-family/tool coverage
    for legacy slugs
    - refresh tests, fixtures, and TUI snapshots so they expect the upgraded
    defaults
    
    ## Testing
    - `cargo test -p codex-core
    config::tests::test_precedence_fixture_with_gpt5_profile`
    
    
    ------
    [Codex
    Task](https://chatgpt.com/codex/tasks/task_i_6916c5b3c2b08321ace04ee38604fc6b)
  • fix(core) serialize shell_command (#6744)
    ## Summary
    Ensures we're serializing calls to `shell_command`
    
    ## Testing
    - [x] Added unit test
  • [App-server] add new v2 events:item/reasoning/delta, item/agentMessage/delta & item/reasoning/summaryPartAdded (#6559)
    core event to app server event mapping:
    1. `codex/event/reasoning_content_delta` ->
    `item/reasoning/summaryTextDelta`.
    2. `codex/event/reasoning_raw_content_delta` ->
    `item/reasoning/textDelta`
    3. `codex/event/agent_message_content_delta` →
    `item/agentMessage/delta`.
    4. `codex/event/agent_reasoning_section_break` ->
    `item/reasoning/summaryPartAdded`.
    
    Also added a change in core to pass down content index, summary index
    and item id from events.
    
    Tested with the `git checkout owen/app_server_test_client && cargo run
    -p codex-app-server-test-client -- send-message-v2 "hello"` and verified
    that new events are emitted correctly.
  • Add item streaming events (#5546)
    Adds AgentMessageContentDelta, ReasoningContentDelta,
    ReasoningRawContentDelta item streaming events while maintaining
    compatibility for old events.
    
    ---------
    
    Co-authored-by: Owen Lin <owen@openai.com>
  • Delegate review to codex instance (#5572)
    In this PR, I am exploring migrating task kind to an invocation of
    Codex. The main reason would be getting rid off multiple
    `ConversationHistory` state and streamlining our context/history
    management.
    
    This approach depends on opening a channel between the sub-codex and
    codex. This channel is responsible for forwarding `interactive`
    (`approvals`) and `non-interactive` events. The `task` is responsible
    for handling those events.
    
    This opens the door for implementing `codex as a tool`, replacing
    `compact` and `review`, and potentially subagents.
    
    One consideration is this code is very similar to `app-server` specially
    in the approval part. If in the future we wanted an interactive
    `sub-codex` we should consider using `codex-mcp`
  • chore: rework tools execution workflow (#5278)
    Re-work the tool execution flow. Read `orchestrator.rs` to understand
    the structure
  • feat: parallel tool calls (#4663)
    Add parallel tool calls. This is configurable at model level and tool
    level
  • feat: Freeform apply_patch with simple shell output (#4718)
    ## Summary
    This PR is an alternative approach to #4711, but instead of changing our
    storage, parses out shell calls in the client and reserializes them on
    the fly before we send them out as part of the request.
    
    What this changes:
    1. Adds additional serialization logic when the
    ApplyPatchToolType::Freeform is in use.
    2. Adds a --custom-apply-patch flag to enable this setting on a
    session-by-session basis.
    
    This change is delicate, but is not meant to be permanent. It is meant
    to be the first step in a migration:
    1. (This PR) Add in-flight serialization with config
    2. Update model_family default
    3. Update serialization logic to store turn outputs in a structured
    format, with logic to serialize based on model_family setting.
    4. Remove this rewrite in-flight logic.
    
    ## Test Plan
    - [x] Additional unit tests added
    - [x] Integration tests added
    - [x] Tested locally
  • chore: refactor tool handling (#4510)
    # Tool System Refactor
    
    - Centralizes tool definitions and execution in `core/src/tools/*`:
    specs (`spec.rs`), handlers (`handlers/*`), router (`router.rs`),
    registry/dispatch (`registry.rs`), and shared context (`context.rs`).
    One registry now builds the model-visible tool list and binds handlers.
    - Router converts model responses to tool calls; Registry dispatches
    with consistent telemetry via `codex-rs/otel` and unified error
    handling. Function, Local Shell, MCP, and experimental `unified_exec`
    all flow through this path; legacy shell aliases still work.
    - Rationale: reduce per‑tool boilerplate, keep spec/handler in sync, and
    make adding tools predictable and testable.
    
    Example: `read_file`
    - Spec: `core/src/tools/spec.rs` (see `create_read_file_tool`,
    registered by `build_specs`).
    - Handler: `core/src/tools/handlers/read_file.rs` (absolute `file_path`,
    1‑indexed `offset`, `limit`, `L#: ` prefixes, safe truncation).
    - E2E test: `core/tests/suite/read_file.rs` validates the tool returns
    the requested lines.
    
    ## Next steps:
    - Decompose `handle_container_exec_with_params` 
    - Add parallel tool calls
  • Send limits when getting rate limited (#4102)
    Users need visibility on rate limits when they are rate limited.
  • Add exec output-schema parameter (#4079)
    Adds structured output to `exec` via the `--structured-output`
    parameter.
  • Forward Rate limits to the UI (#3965)
    We currently get information about rate limits in the response headers.
    We want to forward them to the clients to have better transparency.
    UI/UX plans have been discussed and this information is needed.
  • fix: some nit Rust reference issues (#3849)
    Fix some small references issue. No behavioural change. Just making the
    code cleaner
  • fix: model family and apply_patch consistency (#3603)
    ## Summary
    Resolves a merge conflict between #3597 and #3560, and adds tests to
    double check our apply_patch configuration.
    
    ## Testing
    - [x] Added unit tests
    
    ---------
    
    Co-authored-by: dedrisian-oai <dedrisian@openai.com>
  • Add per-model-family prompts (#3597)
    Allows more flexibility in defining prompts.
  • Always request encrypted cot (#3539)
    Otherwise future requests will fail with 500
  • Review Mode (Core) (#3401)
    ## 📝 Review Mode -- Core
    
    This PR introduces the Core implementation for Review mode:
    
    - New op `Op::Review { prompt: String }:` spawns a child review task
    with isolated context, a review‑specific system prompt, and a
    `Config.review_model`.
    - `EnteredReviewMode`: emitted when the child review session starts.
    Every event from this point onwards reflects the review session.
    - `ExitedReviewMode(Option<ReviewOutputEvent>)`: emitted when the review
    finishes or is interrupted, with optional structured findings:
    
    ```json
    {
      "findings": [
        {
          "title": "<≤ 80 chars, imperative>",
          "body": "<valid Markdown explaining *why* this is a problem; cite files/lines/functions>",
          "confidence_score": <float 0.0-1.0>,
          "priority": <int 0-3>,
          "code_location": {
            "absolute_file_path": "<file path>",
            "line_range": {"start": <int>, "end": <int>}
          }
        }
      ],
      "overall_correctness": "patch is correct" | "patch is incorrect",
      "overall_explanation": "<1-3 sentence explanation justifying the overall_correctness verdict>",
      "overall_confidence_score": <float 0.0-1.0>
    }
    ```
    
    ## Questions
    
    ### Why separate out its own message history?
    
    We want the review thread to match the training of our review models as
    much as possible -- that means using a custom prompt, removing user
    instructions, and starting a clean chat history.
    
    We also want to make sure the review thread doesn't leak into the parent
    thread.
    
    ### Why do this as a mode, vs. sub-agents?
    
    1. We want review to be a synchronous task, so it's fine for now to do a
    bespoke implementation.
    2. We're still unclear about the final structure for sub-agents. We'd
    prefer to land this quickly and then refactor into sub-agents without
    rushing that implementation.
  • feat: reasoning effort as optional (#3527)
    Allow the reasoning effort to be optional
  • Never store requests (#3212)
    When item ids are sent to Responses API it will load them from the
    database ignoring the provided values. This adds extra latency.
    
    Not having the mode to store requests also allows us to simplify the
    code.
    
    ## Breaking change
    
    The `disable_response_storage` configuration option is removed.
  • Dividing UserMsgs into categories to send it back to the tui (#3127)
    This PR does the following:
    
    - divides user msgs into 3 categories: plain, user instructions, and
    environment context
    - Centralizes adding user instructions and environment context to a
    degree
    - Improve the integration testing
    
    Building on top of #3123
    
    Specifically this
    [comment](https://github.com/openai/codex/pull/3123#discussion_r2319885089).
    We need to send the user message while ignoring the User Instructions
    and Environment Context we attach.
  • chore: Clean up verbosity config (#3056)
    ## Summary
    It appears that #2108 hit a merge conflict with #2355 - I failed to
    notice the path difference when re-reviewing the former. This PR
    rectifies that, and consolidates it into the protocol package, in line
    with our philosophy of specifying types in one place.
    
    ## Testing
    - [x] Adds config test for model_verbosity
  • Improve gpt-oss compatibility (#2461)
    The gpt-oss models require reasoning with subsequent Chat Completions
    requests because otherwise the model forgets why the tools were called.
    This change fixes that and also adds some additional missing
    documentation around how to handle context windows in Ollama and how to
    show the CoT if you desire to.
  • Following up on #2371 post commit feedback (#2852)
    - Introduce websearch end to complement the begin 
    - Moves the logic of adding the sebsearch tool to
    create_tools_json_for_responses_api
    - Making it the client responsibility to toggle the tool on or off 
    - Other misc in #2371 post commit feedback
    - Show the query:
    
    <img width="1392" height="151" alt="image"
    src="https://github.com/user-attachments/assets/8457f1a6-f851-44cf-bcca-0d4fe460ce89"
    />
  • Add web search tool (#2371)
    Adds web_search tool, enabling the model to use Responses API web_search
    tool.
    - Disabled by default, enabled by --search flag
    - When --search is passed, exposes web_search_request function tool to
    the model, which triggers user approval. When approved, the model can
    use the web_search tool for the remainder of the turn
    <img width="1033" height="294" alt="image"
    src="https://github.com/user-attachments/assets/62ac6563-b946-465c-ba5d-9325af28b28f"
    />
    
    ---------
    
    Co-authored-by: easong-openai <easong@openai.com>
  • Move models.rs to protocol (#2595)
    Moving models.rs to protocol so we can use them in `Codex` operations
  • [apply_patch] freeform apply_patch tool (#2576)
    ## Summary
    GPT-5 introduced the concept of [custom
    tools](https://platform.openai.com/docs/guides/function-calling#custom-tools),
    which allow the model to send a raw string result back, simplifying
    json-escape issues. We are migrating gpt-5 to use this by default.
    
    However, gpt-oss models do not support custom tools, only normal
    functions. So we keep both tool definitions, and provide whichever one
    the model family supports.
    
    ## Testing
    - [x] Tested locally with various models
    - [x] Unit tests pass
  • feat(gpt5): add model_verbosity for GPT‑5 via Responses API (#2108)
    **Summary**
    - Adds `model_verbosity` config (values: low, medium, high).
    - Sends `text.verbosity` only for GPT‑5 family models via the Responses
    API.
    - Updates docs and adds serialization tests.
    
    **Motivation**
    - GPT‑5 introduces a verbosity control to steer output length/detail
    without pro
    mpt surgery.
    - Exposing it as a config knob keeps prompts stable and makes behavior
    explicit
    and repeatable.
    
    **Changes**
    - Config:
      - Added `Verbosity` enum (low|medium|high).
    - Added optional `model_verbosity` to `ConfigToml`, `Config`, and
    `ConfigProfi
    le`.
    - Request wiring:
      - Extended `ResponsesApiRequest` with optional `text` object.
    - Populates `text.verbosity` only when model family is `gpt-5`; omitted
    otherw
    ise.
    - Tests:
    - Verifies `text.verbosity` serializes when set and is omitted when not
    set.
    - Docs:
      - Added “GPT‑5 Verbosity” section in `codex-rs/README.md`.
      - Added `model_verbosity` section to `codex-rs/config.md`.
    
    **Usage**
    - In `~/.codex/config.toml`:
      - `model = "gpt-5"`
      - `model_verbosity = "low"` (or `"medium"` default, `"high"`)
    - CLI override example:
      - `codex -c model="gpt-5" -c model_verbosity="high"`
    
    **API Impact**
    - Requests to GPT‑5 via Responses API include: `text: { verbosity:
    "low|medium|h
    igh" }` when configured.
    - For legacy models or Chat Completions providers, `text` is omitted.
    
    **Backward Compatibility**
    - Default behavior unchanged when `model_verbosity` is not set (server
    default “
    medium”).
    
    **Testing**
    - Added unit tests for serialization/omission of `text.verbosity`.
    - Ran `cargo fmt` and `cargo test --all-features` (all green).
    
    **Docs**
    - `README.md`: new “GPT‑5 Verbosity” note under Config with example.
    - `config.md`: new `model_verbosity` section.
    
    **Out of Scope**
    - No changes to temperature/top_p or other GPT‑5 parameters.
    - No changes to Chat Completions wiring.
    
    **Risks / Notes**
    - If OpenAI changes the wire shape for verbosity, we may need to update
    `Respons
    esApiRequest`.
    - Behavior gated to `gpt-5` model family to avoid unexpected effects
    elsewhere.
    
    **Checklist**
    - [x] Code gated to GPT‑5 family only
    - [x] Docs updated (`README.md`, `config.md`)
    - [x] Tests added and passing
    - [x] Formatting applied
    
    Release note: Add `model_verbosity` config to control GPT‑5 output verbosity via the Responses API (low|medium|high).
  • [apply-patch] Clean up apply-patch tool definitions (#2539)
    ## Summary
    We've experienced a bit of drift in system prompting for `apply_patch`:
    - As pointed out in #2030 , our prettier formatting started altering
    prompt.md in a few ways
    - We introduced a separate markdown file for apply_patch instructions in
    #993, but currently duplicate them in the prompt.md file
    - We added a first-class apply_patch tool in #2303, which has yet
    another definition
    
    This PR starts to consolidate our logic in a few ways:
    - We now only use
    `apply_patch_tool_instructions.md](https://github.com/openai/codex/compare/dh--apply-patch-tool-definition?expand=1#diff-d4fffee5f85cb1975d3f66143a379e6c329de40c83ed5bf03ffd3829df985bea)
    for system instructions
    - We no longer include apply_patch system instructions if the tool is
    specified
    
    I'm leaving the definition in openai_tools.rs as duplicated text for now
    because we're going to be iterated on the first-class tool soon.
    
    ## Testing
    - [x] Added integration tests to verify prompt stability
    - [x] Tested locally with several different models (gpt-5, gpt-oss,
    o4-mini)
  • consolidate reasoning enums into one (#2428)
    We have three enums for each of reasoning summaries and reasoning effort
    with same values. They can be consolidated into one.
  • Fix #2296 Add "minimal" reasoning effort for GPT 5 models (#2326)
    This pull request resolves #2296; I've confirmed if it works by:
    
    1. Add settings to ~/.codex/config.toml:
    ```toml
    model_reasoning_effort = "minimal"
    ```
    
    2. Run the CLI:
    ```
    cd codex-rs
    cargo build && RUST_LOG=trace cargo run --bin codex
    /status
    tail -f ~/.codex/log/codex-tui.log
    ```
    
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • Added allow-expect-in-tests / allow-unwrap-in-tests (#2328)
    This PR:
    * Added the clippy.toml to configure allowable expect / unwrap usage in
    tests
    * Removed as many expect/allow lines as possible from tests
    * moved a bunch of allows to expects where possible
    
    Note: in integration tests, non `#[test]` helper functions are not
    covered by this so we had to leave a few lingering `expect(expect_used`
    checks around