Commit Graph

1499 Commits

  • Persist complete TurnContextItem state via canonical conversion (#11656)
    ## Summary
    
    This PR delivers the first small, shippable step toward model-visible
    state diffing by making
    `TurnContextItem` more complete and standardizing how it is built.
    
    Specifically, it:
    - Adds persisted network context to `TurnContextItem`.
    - Introduces a single canonical `TurnContext -> TurnContextItem`
    conversion path.
    - Routes existing rollout write sites through that canonical conversion
    helper.
    
    No context injection/diff behavior changes are included in this PR.
    
    ## Why this change
    
    The design goal is to make `TurnContextItem` the canonical source of
    truth for context-diff
    decisions.
    Before this PR:
    - `TurnContextItem` did not include all TurnContext-derived environment
    inputs needed for v1
    completeness.
    - Construction was duplicated at multiple write sites.
    
    This PR addresses both with a minimal, reviewable change.
    
    ## Changes
    
    ### 1) Extend `TurnContextItem` with network state
    - Added `TurnContextNetworkItem { allowed_domains, denied_domains }`.
    - Added `network: Option<TurnContextNetworkItem>` to `TurnContextItem`.
    - Kept backward compatibility by making the new field optional and
    skipped when absent.
    
    Files:
    - `codex-rs/protocol/src/protocol.rs`
    
    ### 2) Canonical conversion helper
    - Added `TurnContext::to_turn_context_item(collaboration_mode)` in core.
    - Added internal helper to derive network fields from
    `config_layer_stack.requirements().network`.
    
    Files:
    - `codex-rs/core/src/codex.rs`
    
    ### 3) Use canonical conversion at rollout write sites
    - Replaced ad hoc `TurnContextItem { ... }` construction with
    `to_turn_context_item(...)` in:
      - sampling request path
      - compaction path
    
    Files:
    - `codex-rs/core/src/codex.rs`
    - `codex-rs/core/src/compact.rs`
    
    ### 4) Update fixtures/tests for new optional field
    - Updated existing `TurnContextItem` literals in tests to include
    `network: None`.
    - Added protocol tests for:
      - deserializing old payloads with no `network`
      - serializing when `network` is present
    
    Files:
    - `codex-rs/core/tests/suite/resume_warning.rs`
    - No replay/diff logic changes.
    - Persisted rollout `TurnContextItem` now carries additional network
    context when available.
    - Older rollout lines without `network` remain readable.
  • Add new apps_mcp_gateway (#11630)
    Adds a new apps_mcp_gateway flag to route Apps MCP calls through
    https://api.openai.com/v1/connectors/mcp/ when enabled, while keeping
    legacy MCP routing as default.
  • [apps] Add is_enabled to app info. (#11417)
    - [x] Add is_enabled to app info and the response of `app/list`.
    - [x] Update TUI to have Enable/Disable button on the app detail page.
  • Add js_repl_tools_only model and routing restrictions (#10671)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
    
    
    #### [git stack](https://github.com/magus/git-stack-cli)
    -  `1` https://github.com/openai/codex/pull/10674
    -  `2` https://github.com/openai/codex/pull/10672
    - 👉 `3` https://github.com/openai/codex/pull/10671
    -  `4` https://github.com/openai/codex/pull/10673
    -  `5` https://github.com/openai/codex/pull/10670
  • [feat] add seatbelt permission files (#11639)
    Add seatbelt permission extension abstraction as permission files for
    seatbelt profiles. This should complement our current sandbox policy
  • feat: introduce Permissions (#11633)
    ## Why
    We currently carry multiple permission-related concepts directly on
    `Config` for shell/unified-exec behavior (`approval_policy`,
    `sandbox_policy`, `network`, `shell_environment_policy`,
    `windows_sandbox_mode`).
    
    Consolidating these into one in-memory struct makes permission handling
    easier to reason about and sets up the next step: supporting named
    permission profiles (`[permissions.PROFILE_NAME]`) without changing
    behavior now.
    
    This change is mostly mechanical: it updates existing callsites to go
    through `config.permissions`, but it does not yet refactor those
    callsites to take a single `Permissions` value in places where multiple
    permission fields are still threaded separately.
    
    This PR intentionally **does not** change the on-disk `config.toml`
    format yet and keeps compatibility with legacy config keys.
    
    ## What Changed
    - Introduced `Permissions` in `core/src/config/mod.rs`.
    - Added `Config::permissions` and moved effective runtime permission
    fields under it:
      - `approval_policy`
      - `sandbox_policy`
      - `network`
      - `shell_environment_policy`
      - `windows_sandbox_mode`
    - Updated config loading/building so these effective values are still
    derived from the same existing config inputs and constraints.
    - Updated Windows sandbox helpers/resolution to read/write via
    `permissions`.
    - Threaded the new field through all permission consumers across core
    runtime, app-server, CLI/exec, TUI, and sandbox summary code.
    - Updated affected tests to reference `config.permissions.*`.
    - Renamed the struct/field from
    `EffectivePermissions`/`effective_permissions` to
    `Permissions`/`permissions` and aligned variable naming accordingly.
    
    ## Verification
    - `just fix -p codex-core -p codex-tui -p codex-cli -p codex-app-server
    -p codex-exec -p codex-utils-sandbox-summary`
    - `cargo build -p codex-core -p codex-tui -p codex-cli -p
    codex-app-server -p codex-exec -p codex-utils-sandbox-summary`
  • chore(core) Deprecate approval_policy: on-failure (#11631)
    ## Summary
    In an effort to start simplifying our sandbox setup, we're announcing
    this approval_policy as deprecated. In general, it performs worse than
    `on-request`, and we're focusing on making fewer sandbox configurations
    perform much better.
    
    ## Testing
    - [x] Tested locally
    - [x] Existing tests pass
  • add a slash command to grant sandbox read access to inaccessible directories (#11512)
    There is an edge case where a directory is not readable by the sandbox.
    In practice, we've seen very little of it, but it can happen so this
    slash command unlocks users when it does.
    
    Future idea is to make this a tool that the agent knows about so it can
    be more integrated.
  • Add js_repl host helpers and exec end events (#10672)
    ## Summary
    
    This PR adds host-integrated helper APIs for `js_repl` and updates model
    guidance so the agent can use them reliably.
    
    ### What’s included
    
    - Add `codex.tool(name, args?)` in the JS kernel so `js_repl` can call
    normal Codex tools.
    - Keep persistent JS state and scratch-path helpers available:
      - `codex.state`
      - `codex.tmpDir`
    - Wire `js_repl` tool calls through the standard tool router path.
    - Add/align `js_repl` execution completion/end event behavior with
    existing tool logging patterns.
    - Update dynamic prompt injection (`project_doc`) to document:
      - how to call `codex.tool(...)`
      - raw output behavior
    - image flow via `view_image` (`codex.tmpDir` +
    `codex.tool("view_image", ...)`)
    - stdio safety guidance (`console.log` / `codex.tool`, avoid direct
    `process.std*`)
    
    ## Why
    
    - Standardize JS-side tool usage on `codex.tool(...)`
    - Make `js_repl` behavior more consistent with existing tool execution
    and event/logging patterns.
    - Give the model enough runtime guidance to use `js_repl` safely and
    effectively.
    
    ## Testing
    
    - Added/updated unit and runtime tests for:
      - `codex.tool` calls from `js_repl` (including shell/MCP paths)
      - image handoff flow via `view_image`
      - prompt-injection text for `js_repl` guidance
      - execution/end event behavior and related regression coverage
    
    
    
    
    #### [git stack](https://github.com/magus/git-stack-cli)
    -  `1` https://github.com/openai/codex/pull/10674
    - 👉 `2` https://github.com/openai/codex/pull/10672
    -  `3` https://github.com/openai/codex/pull/10671
    -  `4` https://github.com/openai/codex/pull/10673
    -  `5` https://github.com/openai/codex/pull/10670
  • feat(app-server): experimental flag to persist extended history (#11227)
    This PR adds an experimental `persist_extended_history` bool flag to
    app-server thread APIs so rollout logs can retain a richer set of
    EventMsgs for non-lossy Thread > Turn > ThreadItems reconstruction (i.e.
    on `thread/resume`).
    
    ### Motivation
    Today, our rollout recorder only persists a small subset (e.g. user
    message, reasoning, assistant message) of `EventMsg` types, dropping a
    good number (like command exec, file change, etc.) that are important
    for reconstructing full item history for `thread/resume`, `thread/read`,
    and `thread/fork`.
    
    Some clients want to be able to resume a thread without lossiness. This
    lossiness is primarily a UI thing, since what the model sees are
    `ResponseItem` and not `EventMsg`.
    
    ### Approach
    This change introduces an opt-in `persist_full_history` flag to preserve
    those events when you start/resume/fork a thread (defaults to `false`).
    
    This is done by adding an `EventPersistenceMode` to the rollout
    recorder:
    - `Limited` (existing behavior, default)
    - `Extended` (new opt-in behavior)
    
    In `Extended` mode, persist additional `EventMsg` variants needed for
    non-lossy app-server `ThreadItem` reconstruction. We now store the
    following ThreadItems that we didn't before:
    - web search
    - command execution
    - patch/file changes
    - MCP tool calls
    - image view calls
    - collab tool outcomes
    - context compaction
    - review mode enter/exit
    
    For **command executions** in particular, we truncate the output using
    the existing `truncate_text` from core to store an upper bound of 10,000
    bytes, which is also the default value for truncating tool outputs shown
    to the model. This keeps the size of the rollout file and command
    execution items returned over the wire reasonable.
    
    And we also persist `EventMsg::Error` which we can now map back to the
    Turn's status and populates the Turn's error metadata.
    
    #### Updates to EventMsgs
    To truly make `thread/resume` non-lossy, we also needed to persist the
    `status` on `EventMsg::CommandExecutionEndEvent` and
    `EventMsg::PatchApplyEndEvent`. Previously it was not obvious whether a
    command failed or was declined (similar for apply_patch). These
    EventMsgs were never persisted before so I made it a required field.
  • Parse first order skill/connector mentions (#11547)
    This PR introduces a skill-expansion mechanism for mentions so nested or
    skill or connection mentions are expanded if present in skills invoked
    by the user. This keeps behavior aligned with existing mention handling
    while extending coverage to deeper scenarios. With these changes, users
    can create skills that invoke connectors, and skills that invoke other
    skills.
    
    Replaces #10863, which is not needed with the addition of
    [search_tool_bm25](https://github.com/openai/codex/issues/10657)
  • Add cwd to memory files (#11591)
    Add cwd to memory files so that model can deal with multi cwd memory
    better.
    
    ---------
    
    Co-authored-by: jif-oai <jif@openai.com>
  • fix(core) model_info preserves slug (#11602)
    ## Summary
    Preserve the specified model slug when we get a prefix-based match
    
    ## Testing
    - [x] added unit test
    
    ---------
    
    Co-authored-by: Ahmed Ibrahim <aibrahim@openai.com>
  • chore: drop and clean from phase 1 (#11605)
    This PR is mostly cleaning and simplifying phase 1 of memories
  • Fix config test on macOS (#11579)
    When running these tests locally, you may have system-wide config or
    requirements files. This makes the tests ignore these files.
  • Ensure list_threads drops stale rollout files (#11572)
    Summary
    - trim `state_db::list_threads_db` results to entries whose rollout
    files still exist, logging and recording a discrepancy for dropped rows
    - delete stale metadata rows from the SQLite store so future calls don’t
    surface invalid paths
    - add regression coverage in `recorder.rs` to verify stale DB paths are
    dropped when the file is missing
  • feat: mem drop cot (#11571)
    Drop CoT and compaction for memory building
  • Fix flaky pre_sampling_compact switch test (#11573)
    Summary
    - address the nondeterministic behavior observed in
    `pre_sampling_compact_runs_on_switch_to_smaller_context_model` so it no
    longer fails intermittently during model switches
    - ensure the surrounding sampling logic consistently handles the
    smaller-context case that the test exercises
    
    Testing
    - Not run (not requested)
  • feat: mem slash commands (#11569)
    Add 2 slash commands for memories:
    * `/m_drop` delete all the memories
    * `/m_update` update the memories with phase 1 and 2
  • fix: stop inheriting rate-limit limit_name (#11557)
    When we carry over values from partial rate-limit, we should only do so
    for the same limit_id.
  • Hide the first websocket retry (#11548)
    Sometimes connection needs to be quickly reestablished, don't produce an
    error for that.
  • fix: make project_doc skill-render tests deterministic (#11545)
    ## Why
    `project_doc::tests::skills_are_appended_to_project_doc` and
    `project_doc::tests::skills_render_without_project_doc` were assuming a
    single synthetic skill in test setup, but they called
    `load_skills(&cfg)`, which loads from repo/user/system roots.
    
    That made the assertions environment-dependent. After
    [#11531](https://github.com/openai/codex/pull/11531) added
    `.codex/skills/test-tui/SKILL.md`, the repo-scoped `test-tui` skill
    began appearing in these test outputs and exposed the flake.
    
    ## What Changed
    - Added a test-only helper in `codex-rs/core/src/project_doc.rs` that
    loads skills from an explicit root via `load_skills_from_roots`.
    - Scoped that root to `codex_home/skills` with `SkillScope::User`.
    - Updated both affected tests to use this helper instead of
    `load_skills(&cfg)`:
      - `skills_are_appended_to_project_doc`
      - `skills_render_without_project_doc`
    
    This keeps the tests focused on the fixture skills they create,
    independent of ambient repo/home skills.
    
    ## Verification
    - `cargo test -p codex-core
    project_doc::tests::skills_render_without_project_doc -- --exact`
    - `cargo test -p codex-core
    project_doc::tests::skills_are_appended_to_project_doc -- --exact`
  • feat: make sandbox read access configurable with ReadOnlyAccess (#11387)
    `SandboxPolicy::ReadOnly` previously implied broad read access and could
    not express a narrower read surface.
    This change introduces an explicit read-access model so we can support
    user-configurable read restrictions in follow-up work, while preserving
    current behavior today.
    
    It also ensures unsupported backends fail closed for restricted-read
    policies instead of silently granting broader access than intended.
    
    ## What
    
    - Added `ReadOnlyAccess` in protocol with:
      - `Restricted { include_platform_defaults, readable_roots }`
      - `FullAccess`
    - Updated `SandboxPolicy` to carry read-access configuration:
      - `ReadOnly { access: ReadOnlyAccess }`
      - `WorkspaceWrite { ..., read_only_access: ReadOnlyAccess }`
    - Preserved existing behavior by defaulting current construction paths
    to `ReadOnlyAccess::FullAccess`.
    - Threaded the new fields through sandbox policy consumers and call
    sites across `core`, `tui`, `linux-sandbox`, `windows-sandbox`, and
    related tests.
    - Updated Seatbelt policy generation to honor restricted read roots by
    emitting scoped read rules when full read access is not granted.
    - Added fail-closed behavior on Linux and Windows backends when
    restricted read access is requested but not yet implemented there
    (`UnsupportedOperation`).
    - Regenerated app-server protocol schema and TypeScript artifacts,
    including `ReadOnlyAccess`.
    
    ## Compatibility / rollout
    
    - Runtime behavior remains unchanged by default (`FullAccess`).
    - API/schema changes are in place so future config wiring can enable
    restricted read access without another policy-shape migration.
  • Update context window after model switch (#11520)
    - Update token usage aggregation to refresh model context window after a
    model change.
    - Add protocol/core tests, including an e2e model-switch test that
    validates switching to a smaller model updates telemetry.
  • Pre-sampling compact with previous model context (#11504)
    - Run pre-sampling compact through a single helper that builds
    previous-model turn context and compacts before the follow-up request
    when switching to a smaller context window.
    - Keep compaction events on the parent turn id and add compact suite
    coverage for switch-in-session and resume+switch flows.
  • change model cap to server overload (#11388)
    # External (non-OpenAI) Pull Request Requirements
    
    Before opening this Pull Request, please read the dedicated
    "Contributing" markdown file or your PR may be closed:
    https://github.com/openai/codex/blob/main/docs/contributing.md
    
    If your PR conforms to our contribution guidelines, replace this text
    with a detailed and high quality description of your changes.
    
    Include a link to a bug report or enhancement request.
  • Consolidate search_tool feature into apps (#11509)
    ## Summary
    - Remove `Feature::SearchTool` and the `search_tool` config key from the
    feature registry/schema.
    - Gate `search_tool_bm25` exposure via `Feature::Apps` in
    `core/src/tools/spec.rs`.
    - Update MCP selection logic in `core/src/codex.rs` to use
    `Feature::Apps` for search-tool behavior.
    - Update `core/tests/suite/search_tool.rs` to enable `Feature::Apps`.
    - Regenerate `core/config.schema.json` via `just write-config-schema`.
    
    ## Testing
    - `just fmt`
    - `cargo test -p codex-core --test all suite::search_tool::`
    
    ## Tickets
    - None
  • Hydrate previous model across resume/fork/rollback/task start (#11497)
    - Replace pending resume model state with persistent previous_model and
    hydrate it on resume, fork, rollback, and task end in spawn_task
  • Added seatbelt policy rule to allow os.cpus (#11277)
    I don't think this policy change increases the risk, other than
    potentially exposing the caller to bugs in these kernel calls, which are
    unlikely.
    
    Without this change, some tools are silently failing or making incorrect
    decisions about the processor type (e.g. installing x86 binaries rather
    than Apple silicon binaries).
    
    This addresses #11210
    
    ---------
    
    Co-authored-by: viyatb-oai <viyatb@openai.com>
  • Add AfterToolUse hook (#11335)
    Not wired up to config yet. (So we can change the name if we want)
    
    An example payload:
    
    ```
    {
      "session_id": "019c48b7-7098-7b61-bc48-32e82585d451",
      "cwd": "/Users/gt/code/codex/codex-rs",
      "triggered_at": "2026-02-10T18:02:31Z",
      "hook_event": {
        "event_type": "after_tool_use",
        "turn_id": "4",
        "call_id": "call_iuo4DqWgjE7OxQywnL2UzJUE",
        "tool_name": "apply_patch",
        "tool_kind": "custom",
        "tool_input": {
          "input_type": "custom",
          "input": "*** Begin Patch\n*** Update File: README.md\n@@\n-# Codex CLI hello (Rust Implementation)\n+# Codex CLI (Rust Implementation)\n*** End Patch\n"
        },
        "executed": true,
        "success": true,
        "duration_ms": 37,
        "mutating": true,
        "sandbox": "none",
        "sandbox_policy": "danger-full-access",
        "output_preview": "{\"output\":\"Success. Updated the following files:\\nM README.md\\n\",\"metadata\":{\"exit_code\":0,\"duration_seconds\":0.0}}"
      }
    }
    ```
  • Increased file watcher debounce duration from 1s to 10s (#11494)
    Users were reporting that when they were actively editing a skill file,
    they would see frequent errors (one per second) across all of their
    active session until they fixed all frontmatter parse errors. This
    change will reduce the chatter at the expense of a slightly longer delay
    before skills are updated in the UI.
    
    This addresses #11385
  • nit: memory truncation (#11479)
    Use existing truncation for memories
  • Add feature-gated freeform js_repl core runtime (#10674)
    ## Summary
    
    This PR adds an **experimental, feature-gated `js_repl` core runtime**
    so models can execute JavaScript in a persistent REPL context across
    tool calls.
    
    The implementation integrates with existing feature gating, tool
    registration, prompt composition, config/schema docs, and tests.
    
    ## What changed
    
    - Added new experimental feature flag: `features.js_repl`.
    - Added freeform `js_repl` tool and companion `js_repl_reset` tool.
    - Gated tool availability behind `Feature::JsRepl`.
    - Added conditional prompt-section injection for JS REPL instructions
    via marker-based prompt processing.
    - Implemented JS REPL handlers, including freeform parsing and pragma
    support (timeout/reset controls).
    - Added runtime resolution order for Node:
      1. `CODEX_JS_REPL_NODE_PATH`
      2. `js_repl_node_path` in config
      3. `PATH`
    - Added JS runtime assets/version files and updated docs/schema.
    
    ## Why
    
    This enables richer agent workflows that require incremental JavaScript
    execution with preserved state, while keeping rollout safe behind an
    explicit feature flag.
    
    ## Testing
    
    Coverage includes:
    
    - Feature-flag gating behavior for tool exposure.
    - Freeform parser/pragma handling edge cases.
    - Runtime behavior (state persistence across calls and top-level `await`
    support).
    
    ## Usage
    
    ```toml
    [features]
    js_repl = true
    ```
    
    Optional runtime override:
    
    - `CODEX_JS_REPL_NODE_PATH`, or
    - `js_repl_node_path` in config.
    
    #### [git stack](https://github.com/magus/git-stack-cli)
    - 👉 `1` https://github.com/openai/codex/pull/10674
    -  `2` https://github.com/openai/codex/pull/10672
    -  `3` https://github.com/openai/codex/pull/10671
    -  `4` https://github.com/openai/codex/pull/10673
    -  `5` https://github.com/openai/codex/pull/10670
  • Promote Windows Sandbox (#11341)
    1. Move Windows Sandbox NUX to right after trust directory screen
    2. Don't offer read-only as an option in Sandbox NUX.
    Elevated/Legacy/Quit
    3. Don't allow new untrusted directories. It's trust or quit
    4. move experimental sandbox features to `[windows]
    sandbox="elevated|unelevatd"`
    5. Copy tweaks = elevated -> default, non-elevated -> non-admin
  • feat: panic if Constrained<WebSearchMode> does not support Disabled (#11470)
    If this happens, this is a logical error on our part and we should fix
    it.
  • Extract codex-config from codex-core (#11389)
    `codex-core` had accumulated config loading, requirements parsing,
    constraint logic, and config-layer state handling in a single crate.
    This change extracts that subsystem into `codex-config` to reduce
    `codex-core` rebuild/test surface area and isolate future config work.
    
    ## What Changed
    
    ### Added `codex-config`
    
    - Added new workspace crate `codex-rs/config` (`codex-config`).
    - Added workspace/build wiring in:
      - `codex-rs/Cargo.toml`
      - `codex-rs/config/Cargo.toml`
      - `codex-rs/config/BUILD.bazel`
    - Updated lockfiles (`codex-rs/Cargo.lock`, `MODULE.bazel.lock`).
    - Added `codex-core` -> `codex-config` dependency in
    `codex-rs/core/Cargo.toml`.
    
    ### Moved config internals from `core` into `config`
    
    Moved modules to `codex-rs/config/src/`:
    
    - `core/src/config/constraint.rs` -> `config/src/constraint.rs`
    - `core/src/config_loader/cloud_requirements.rs` ->
    `config/src/cloud_requirements.rs`
    - `core/src/config_loader/config_requirements.rs` ->
    `config/src/config_requirements.rs`
    - `core/src/config_loader/fingerprint.rs` -> `config/src/fingerprint.rs`
    - `core/src/config_loader/merge.rs` -> `config/src/merge.rs`
    - `core/src/config_loader/overrides.rs` -> `config/src/overrides.rs`
    - `core/src/config_loader/requirements_exec_policy.rs` ->
    `config/src/requirements_exec_policy.rs`
    - `core/src/config_loader/state.rs` -> `config/src/state.rs`
    
    `codex-config` now re-exports this surface from `config/src/lib.rs` at
    the crate top level.
    
    ### Updated `core` to consume/re-export `codex-config`
    
    - `core/src/config_loader/mod.rs` now imports/re-exports config-loader
    types/functions from top-level `codex_config::*`.
    - Local moved modules were removed from `core/src/config_loader/`.
    - `core/src/config/mod.rs` now re-exports constraint types from
    `codex_config`.