Commit Graph

348 Commits

  • feat(app-server): thread/rollback API (#8454)
    Add `thread/rollback` to app-server to support IDEs undo-ing the last N
    turns of a thread.
    
    For context, an IDE partner will be supporting an "undo" capability
    where the IDE (the app-server client) will be responsible for reverting
    the local changes made during the last turn. To support this well, we
    also need a way to drop the last turn (or more generally, the last N
    turns) from the agent's context. This is what `thread/rollback` does.
    
    **Core idea**: A Thread rollback is represented as a persisted event
    message (EventMsg::ThreadRollback) in the rollout JSONL file, not by
    rewriting history. On resume, both the model's context (core replay) and
    the UI turn list (app-server v2's thread history builder) apply these
    markers so the pruned history is consistent across live conversations
    and `thread/resume`.
    
    Implementation notes:
    - Rollback only affects agent context and appends to the rollout file;
    clients are responsible for reverting files on disk.
    - If a thread rollback is currently in progress, subsequent
    `thread/rollback` calls are rejected.
    - Because we use `CodexConversation::submit` and codex core tracks
    active turns, returning an error on concurrent rollbacks is communicated
    via an `EventMsg::Error` with a new variant
    `CodexErrorInfo::ThreadRollbackFailed`. app-server watches for that and
    sends the BAD_REQUEST RPC response.
    
    Tests cover thread rollbacks in both core and app-server, including when
    `num_turns` > existing turns (which clears all turns).
    
    **Note**: this explicitly does **not** behave like `/undo` which we just
    removed from the CLI, which does the opposite of what `thread/rollback`
    does. `/undo` reverts local changes via ghost commits/snapshots and does
    not modify the agent's context / conversation history.
  • feat: agent controller (#8783)
    Added an agent control plane that lets sessions spawn or message other
    conversations via `AgentControl`.
    
    `AgentBus` (core/src/agent/bus.rs) keeps track of the last known status
    of a conversation.
    
    ConversationManager now holds shared state behind an Arc so AgentControl
    keeps only a weak back-reference, the goal is just to avoid explicit
    cycle reference.
    
    Follow-ups:
    * Build a small tool in the TUI to be able to see every agent and send
    manual message to each of them
    * Handle approval requests in this TUI
    * Add tools to spawn/communicate between agents (see related design)
    * Define agent types
  • Use ConfigLayerStack for skills discovery. (#8497)
    Use ConfigLayerStack to get all folders while loading skills.
  • feat: expose outputSchema to user_turn/turn_start app_server API (#8377)
    What changed
    - Added `outputSchema` support to the app-server APIs, mirroring `codex
    exec --output-schema` behavior.
    - V1 `sendUserTurn` now accepts `outputSchema` and constrains the final
    assistant message for that turn.
    - V2 `turn/start` now accepts `outputSchema` and constrains the final
    assistant message for that turn (explicitly per-turn only).
    
    Core behavior
    - `Op::UserTurn` already supported `final_output_json_schema`; now V1
    `sendUserTurn` forwards `outputSchema` into that field.
    - `Op::UserInput` now carries `final_output_json_schema` for per-turn
    settings updates; core maps it into
    `SessionSettingsUpdate.final_output_json_schema` so it applies to the
    created turn context.
    - V2 `turn/start` does NOT persist the schema via `OverrideTurnContext`
    (it’s applied only for the current turn). Other overrides
    (cwd/model/etc) keep their existing persistent behavior.
    
    API / docs
    - `codex-rs/app-server-protocol/src/protocol/v1.rs`: add `output_schema:
    Option<serde_json::Value>` to `SendUserTurnParams` (serialized as
    `outputSchema`).
    - `codex-rs/app-server-protocol/src/protocol/v2.rs`: add `output_schema:
    Option<JsonValue>` to `TurnStartParams` (serialized as `outputSchema`).
    - `codex-rs/app-server/README.md`: document `outputSchema` for
    `turn/start` and clarify it applies only to the current turn.
    - `codex-rs/docs/codex_mcp_interface.md`: document `outputSchema` for v1
    `sendUserTurn` and v2 `turn/start`.
    
    Tests added/updated
    - New app-server integration tests asserting `outputSchema` is forwarded
    into outbound `/responses` requests as `text.format`:
      - `codex-rs/app-server/tests/suite/output_schema.rs`
      - `codex-rs/app-server/tests/suite/v2/output_schema.rs`
    - Added per-turn semantics tests (schema does not leak to the next
    turn):
      - `send_user_turn_output_schema_is_per_turn_v1`
      - `turn_start_output_schema_is_per_turn_v2`
    - Added protocol wire-compat tests for the merged op:
      - serialize omits `final_output_json_schema` when `None`
      - deserialize works when field is missing
      - serialize includes `final_output_json_schema` when `Some(schema)`
    
    Call site updates (high level)
    - Updated all `Op::UserInput { .. }` constructions to include
    `final_output_json_schema`:
      - `codex-rs/app-server/src/codex_message_processor.rs`
      - `codex-rs/core/src/codex_delegate.rs`
      - `codex-rs/mcp-server/src/codex_tool_runner.rs`
      - `codex-rs/tui/src/chatwidget.rs`
      - `codex-rs/tui2/src/chatwidget.rs`
      - plus impacted core tests.
    
    Validation
    - `just fmt`
    - `cargo test -p codex-core`
    - `cargo test -p codex-app-server`
    - `cargo test -p codex-mcp-server`
    - `cargo test -p codex-tui`
    - `cargo test -p codex-tui2`
    - `cargo test -p codex-protocol`
    - `cargo clippy --all-features --tests --profile dev --fix -- -D
    warnings`
  • fix: /review to respect session cwd (#8738)
    Fixes /review base-branch prompt resolution to use the session/turn cwd
    (respecting runtime cwd overrides) so merge-base/diff guidance is
    computed from the intended repo; adds a regression test for cwd
    overrides; tested with cargo test -p codex-core --test all
    review_uses_overridden_cwd_for_base_branch_merge_base.
  • Attach more tags to feedback submissions (#8688)
    Attach more tags to sentry feedback so it's easier to classify and debug
    without having to scan through logs.
    
    Formatting isn't amazing but it's a start.
    <img width="1234" height="276" alt="image"
    src="https://github.com/user-attachments/assets/521a349d-f627-4051-b511-9811cd5cd933"
    />
  • hard disable remote models refresh when feature is disabled (#8687)
    We were refreshing on tag mismatch even with feature is disabled
  • Account for last token count on resume (#8677)
    last token count in context manager is initialized to 0. Gets populated
    only on events from server.
    
    This PR populates it on resume so we can decide if we need to compact or
    not.
  • Refresh on models etag mismatch (#8491)
    - Send models etag
    - Refresh models on 412
    - This wires `ModelsManager` to `ModelFamily` so we don't mutate it
    mid-turn
  • [chore] add additional_details to StreamErrorEvent + wire through (#8307)
    ### What
    
    Builds on #8293.
    
    Add `additional_details`, which contains the upstream error message, to
    relevant structures used to pass along retryable `StreamError`s.
    
    Uses the new TUI status indicator's `details` field (shows under the
    status header) to display the `additional_details` error to the user on
    retryable `Reconnecting...` errors. This adds clarity for users for
    retryable errors.
    
    Will make corresponding change to VSCode extension to show
    `additional_details` as expandable from the `Reconnecting...` cell.
    
    Examples:
    <img width="1012" height="326" alt="image"
    src="https://github.com/user-attachments/assets/f35e7e6a-8f5e-4a2f-a764-358101776996"
    />
    
    <img width="1526" height="358" alt="image"
    src="https://github.com/user-attachments/assets/0029cbc0-f062-4233-8650-cc216c7808f0"
    />
  • chore: save more about turn context in rollout log file (#8458)
    ### Motivation
    - Persist richer per-turn configuration in rollouts so resumed/forked
    sessions and tooling can reason about the exact instruction inputs and
    output constraints used for a turn.
    
    ### Description
    - Extend `TurnContextItem` to include optional `base_instructions`,
    `user_instructions`, and `developer_instructions`.
    - Record the optional `final_output_json_schema` associated with a turn.
    - Add an optional `truncation_policy` to `TurnContextItem` and populate
    it when writing turn-context rollout items.
    - Introduce a protocol-level `TruncationPolicy` representation and
    convert from core truncation policy when recording.
    
    ### Testing
    - `cargo test -p codex-protocol` (pass)
  • feat: load ExecPolicyManager from ConfigLayerStack (#8453)
    https://github.com/openai/codex/pull/8354 added support for in-repo
    `.config/` files, so this PR updates the logic for loading `*.rules`
    files to load `*.rules` files from all relevant layers. The main change
    to the business logic is `load_exec_policy()` in
    `codex-rs/core/src/exec_policy.rs`.
    
    Note this adds a `config_folder()` method to `ConfigLayerSource` that
    returns `Option<AbsolutePathBuf>` so that it is straightforward to
    iterate over the sources and get the associated config folder, if any.
  • Add ExecPolicyManager (#8349)
    Move exec policy management into services to keep turn context
    immutable.
  • Rename OpenAI models to models manager (#8346)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • feat: support allowed_sandbox_modes in requirements.toml (#8298)
    This adds support for `allowed_sandbox_modes` in `requirements.toml` and
    provides legacy support for constraining sandbox modes in
    `managed_config.toml`. This is converted to `Constrained<SandboxPolicy>`
    in `ConfigRequirements` and applied to `Config` such that constraints
    are enforced throughout the harness.
    
    Note that, because `managed_config.toml` is deprecated, we do not add
    support for the new `external-sandbox` variant recently introduced in
    https://github.com/openai/codex/pull/8290. As noted, that variant is not
    supported in `config.toml` today, but can be configured programmatically
    via app server.
  • feat: make ConstraintError an enum (#8330)
    This will make it easier to test for expected errors in unit tests since
    we can compare based on the field values rather than the message (which
    might change over time). See https://github.com/openai/codex/pull/8298
    for an example.
    
    It also ensures more consistency in the way a `ConstraintError` is
    constructed.
  • chore: migrate from Config::load_from_base_config_with_overrides to ConfigBuilder (#8276)
    https://github.com/openai/codex/pull/8235 introduced `ConfigBuilder` and
    this PR updates all call non-test call sites to use it instead of
    `Config::load_from_base_config_with_overrides()`.
    
    This is important because `load_from_base_config_with_overrides()` uses
    an empty `ConfigRequirements`, which is a reasonable default for testing
    so the tests are not influenced by the settings on the host. This method
    is now guarded by `#[cfg(test)]` so it cannot be used by business logic.
    
    Because `ConfigBuilder::build()` is `async`, many of the test methods
    had to be migrated to be `async`, as well. On the bright side, this made
    it possible to eliminate a bunch of `block_on_future()` stuff.
  • Support skills shortDescription. (#8278)
    Allow SKILL.md to specify a more human-readable short description as
    skill metadata.
  • Reintroduce feature flags for skills. (#8244)
    1. Reintroduce feature flags for skills;
    2. UI tweaks (truncate descriptions, better validation error display).
  • Support SYSTEM skills. (#8220)
    1. Remove PUBLIC skills and introduce SYSTEM skills embedded in the
    binary and installed into $CODEX_HOME/skills/.system at startup.
    2. Skills are now always enabled (feature flag removed).
    3. Update skills/list to accept forceReload and plumb it through (not
    used by clients yet).
  • fix: session downgrade (#8196)
    The problem is that the `tokio` task own an `Arc` reference of the
    session and that this task only exit with the broadcast channel get
    closed. But this never get closed if the session is not dropped. So it's
    a snake biting his tail basically
    
    The most notable result was that non of the `Drop` implementation were
    triggered (temporary files, shell snapshots, session cleaning etc etc)
    when closing the session (through a `/new` for example)
    
    The fix is just to weaken the `Arc` and upgrade it on the fly
  • feat: Constrain values for approval_policy (#7778)
    Constrain `approval_policy` through new `admin_policy` config.
    
    This PR will:
    1. Add a `admin_policy` section to config, with a single field (for now)
    `allowed_approval_policies`. This list constrains the set of
    user-settable `approval_policy`s.
    2. Introduce a new `Constrained<T>` type, which combines a current value
    and a validator function. The validator function ensures disallowed
    values are not set.
    3. Change the type of `approval_policy` on `Config` and
    `SessionConfiguration` from `AskForApproval` to
    `Constrained<AskForApproval>`. The validator function is set by the
    values passed into `allowed_approval_policies`.
    4. `GenericDisplayRow`: add a `disabled_reason: Option<String>`. When
    set, it disables selection of the value and indicates as such in the
    menu. This also makes it unselectable with arrow keys or numbers. This
    is used in the `/approvals` menu.
    
    Follow ups are:
    1. Do the same thing to `sandbox_policy`.
    2. Propagate the allowed set of values through app-server for the
    extension (though already this should prevent app-server from setting
    this values, it's just that we want to disable UI elements that are
    unsettable).
    
    Happy to split this PR up if you prefer, into the logical numbered areas
    above. Especially if there are parts we want to gavel on separately
    (e.g. admin_policy).
    
    Disabled full access:
    <img width="1680" height="380" alt="image"
    src="https://github.com/user-attachments/assets/1fb61c8c-1fcb-4dc4-8355-2293edb52ba0"
    />
    
    Disabled `--yolo` on startup:
    <img width="749" height="76" alt="image"
    src="https://github.com/user-attachments/assets/0a1211a0-6eb1-40d6-a1d7-439c41e94ddb"
    />
    
    CODEX-4087
  • Add public skills + improve repo skill discovery and error UX (#8098)
    1. Adds SkillScope::Public end-to-end (core + protocol) and loads skills
    from the public cache directory
    2. Improves repo skill discovery by searching upward for the nearest
    .codex/skills within a git repo
    3. Deduplicates skills by name with deterministic ordering to avoid
    duplicates across sources
    4. Fixes garbled “Skill errors” overlay rendering by preventing pending
    history lines from being injected during the modal
    5. Updates the project docs “Skills” intro wording to avoid hardcoded
    paths
  • nit: trace span for regular task (#8053)
    Logs are too spammy
    
    ---------
    
    Co-authored-by: Anton Panasenko <apanasenko@openai.com>
  • Reimplement skills loading using SkillsManager + skills/list op. (#7914)
    refactor the way we load and manage skills:
    1. Move skill discovery/caching into SkillsManager and reuse it across
    sessions.
    2. Add the skills/list API (Op::ListSkills/SkillsListResponse) to fetch
    skills for one or more cwds. Also update app-server for VSCE/App;
    3. Trigger skills/list during session startup so UIs preload skills and
    handle errors immediately.
  • fix: restore MCP startup progress messages in TUI (fixes #7827) (#7828)
    ## Problem
    
    The introduction of `notify_sandbox_state_change()` in #7112 caused a
    regression where the blocking call in `Session::new()` waits for all MCP
    servers to fully initialize before returning. This prevents the TUI
    event loop from starting, resulting in `McpStartupUpdateEvent` messages
    being emitted but never consumed or displayed. As a result, the app
    appears to hang during startup, and users do not see the expected
    "Booting MCP server: {name}" status line.
    
    Issue: [#7827](https://github.com/openai/codex/issues/7827)
    
    ## Solution
    This change moves sandbox state notification into each MCP server's
    background initialization task. The notification is sent immediately
    after the server transitions to the Ready state. This approach:
    - Avoids blocking `Session::new()`, allowing the TUI event loop to start
    promptly.
    - Ensures each MCP server receives its sandbox state before handling any
    tool calls.
    - Restores the display of "Booting MCP server" status lines during
    startup.
    
    ## Key Changes
    - Added `ManagedClient::notify_sandbox_state()` method.
    - Passed sandbox_state to `McpConnectionManager::initialize()`.
    - Sends sandbox state notification in the background task after the
    server reaches Ready status.
    - Removed blocking notify_sandbox_state_change() methods.
    - Added a chatwidget snapshot test for the "Booting MCP server" status
    line.
    
    ## Regression Details
    
    Regression was bisected to #7112, which introduced the blocking
    behavior.
    
    ---------
    
    Co-authored-by: Michael Bolin <bolinfest@gmail.com>
    Co-authored-by: Michael Bolin <mbolin@openai.com>
  • Added deprecation notice for "chat" wire_api (#7897)
    This PR adds a deprecation notice that appears once per invocation of
    codex (not per conversation) when a conversation is started using a
    custom model provider configured with the "chat" wire_api. We have
    [announced](https://github.com/openai/codex/discussions/7782) that this
    feature is deprecated and will be removed in early Feb 2026, so we want
    to notify users of this fact.
    
    The deprecation notice was added in a way that works with the
    non-interactive "codex exec", the TUI, and with the extension. Screen
    shots of each are below.
    
    <img width="1000" height="89" alt="image"
    src="https://github.com/user-attachments/assets/72cc08bb-d158-4a89-b3c8-7a896abd016f"
    />
    
    <img width="1000" height="38" alt="Screenshot 2025-12-11 at 2 22 29 PM"
    src="https://github.com/user-attachments/assets/7b2128ca-9afc-48be-9ce1-2ce81bc00fcb"
    />
    
    <img width="479" height="106" alt="Screenshot 2025-12-11 at 2 21 26 PM"
    src="https://github.com/user-attachments/assets/858ec1cc-ebfc-4c99-b22b-63015154d752"
    />
  • Inject SKILL.md when it's explicitly mentioned. (#7763)
    1. Skills load once in core at session start; the cached outcome is
    reused across core and surfaced to TUI via SessionConfigured.
    2. TUI detects explicit skill selections, and core injects the matching
    SKILL.md content into the turn when a selected skill is present.
  • make model optional in config (#7769)
    - Make Config.model optional and centralize default-selection logic in
    ModelsManager, including a default_model helper (with
    codex-auto-balanced when available) so sessions now carry an explicit
    chosen model separate from the base config.
    - Resolve `model` once in `core` and `tui` from config. Then store the
    state of it on other structs.
    - Move refreshing models to be before resolving the default model
  • Removed experimental "command risk assessment" feature (#7799)
    This experimental feature received lukewarm reception during internal
    testing. Removing from the code base.
  • refactoring with_escalated_permissions to use SandboxPermissions instead (#7750)
    helpful in the future if we want more granularity for requesting
    escalated permissions:
    e.g when running in readonly sandbox, model can request to escalate to a
    sandbox that allows writes
  • Remove legacy ModelInfo and merge it with ModelFamily (#7748)
    This is a step towards removing the need to know `model` when
    constructing config. We firstly don't need to know `model_info` and just
    respect if the user has already set it. Next step, we don't need to know
    `model` unless the user explicitly set it in `config.toml`
  • fix: refine the warning message and docs for deprecated tools config (#7685)
    Issue #7661 revealed that users are confused by deprecation warnings
    like:
    > `tools.web_search` is deprecated. Use `web_search_request` instead.
    
    This message misleadingly suggests renaming the config key from
    `web_search` to `web_search_request`, when the actual required change is
    to **move and rename the configuration from the `[tools]` section to the
    `[features]` section**.
    
    This PR clarifies the warning messages and documentation to make it
    clear that deprecated `[tools]` configurations should be moved to
    `[features]`. Changes made:
    - Updated deprecation warning format in `codex-rs/core/src/codex.rs:520`
    to include `[features].` prefix
    - Updated corresponding test expectations in
    `codex-rs/core/tests/suite/deprecation_notice.rs:39`
    - Improved documentation in `docs/config.md` to clarify upfront that
    `[tools]` options are deprecated in favor of `[features]`
  • Add remote models feature flag (#7648)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • Wire with_remote_overrides to construct model families (#7621)
    - This PR wires `with_remote_overrides` and make the
    `construct_model_families` an async function
    - Moves getting model family a level above to keep the function `sync`
    - Updates the tests to local, offline, and `sync` helper for model
    families
  • fix: taking plan type from usage endpoint instead of thru auth token (#7610)
    pull plan type from the usage endpoint, persist it in session state /
    tui state, and propagate through rate limit snapshots
  • Call models endpoint in models manager (#7616)
    - Introduce `with_remote_overrides` and update
    `refresh_available_models`
    - Put `auth_manager` instead of `auth_mode` on `models_manager`
    - Remove `ShellType` and `ReasoningLevel` to use already existing
    structs
  • chore: refactor to move Arc<RwLock> concern outside exec_policy_for (#7615)
    The caller should decide whether wrapping the policy in `Arc<RwLock>` is
    necessary. This should make https://github.com/openai/codex/pull/7609 a
    bit smoother.
    
    - `exec_policy_for()` -> `load_exec_policy_for_features()`
    - introduce `load_exec_policy()` that does not take `Features` as an arg
    - both return `Result<Policy, ExecPolicyError>` instead of
    Result<Arc<RwLock<Policy>>, ExecPolicyError>`
    
    This simplifies the tests as they have no need for `Arc<RwLock>`.
  • Inline response recording and remove process_items indirection (#7310)
    - Inline response recording during streaming: `run_turn` now records
    items as they arrive instead of building a `ProcessedResponseItem` list
    and post‑processing via `process_items`.
    - Simplify turn handling: `handle_output_item_done` returns the
    follow‑up signal + optional tool future; `needs_follow_up` is set only
    there, and in‑flight tool futures are drained once at the end (errors
    logged, no extra state writes).
    - Flattened stream loop: removed `process_items` indirection and the
    extra output queue
    - - Tests: relaxed `tool_parallelism::tool_results_grouped` to allow any
    completion order while still requiring matching call/output IDs.
  • remove model_family from `config (#7571)
    - Remove `model_family` from `config`
    - Make sure to still override config elements related to `model_family`
    like supporting reasoning
  • Refactor execpolicy fallback evaluation (#7544)
    ## Refactor of the `execpolicy` crate
    
    To illustrate why we need this refactor, consider an agent attempting to
    run `apple | rm -rf ./`. Suppose `apple` is allowed by `execpolicy`.
    Before this PR, `execpolicy` would consider `apple` and `pear` and only
    render one rule match: `Allow`. We would skip any heuristics checks on
    `rm -rf ./` and immediately approve `apple | rm -rf ./` to run.
    
    To fix this, we now thread a `fallback` evaluation function into
    `execpolicy` that runs when no `execpolicy` rules match a given command.
    In our example, we would run `fallback` on `rm -rf ./` and prevent
    `apple | rm -rf ./` from being run without approval.
  • whitelist command prefix integration in core and tui (#7033)
    this PR enables TUI to approve commands and add their prefixes to an
    allowlist:
    <img width="708" height="605" alt="Screenshot 2025-11-21 at 4 18 07 PM"
    src="https://github.com/user-attachments/assets/56a19893-4553-4770-a881-becf79eeda32"
    />
    
    note: we only show the option to whitelist the command when 
    1) command is not multi-part (e.g `git add -A && git commit -m 'hello
    world'`)
    2) command is not already matched by an existing rule