Commit Graph

13 Commits

  • config: add strict config parsing (#20559)
    ## Why
    
    Codex intentionally ignores unknown `config.toml` fields by default so
    older and newer config files keep working across versions. That leniency
    also makes typo detection hard because misspelled or misplaced keys
    disappear silently.
    
    This change adds an opt-in strict config mode so users and tooling can
    fail fast on unrecognized config fields without changing the default
    permissive behavior.
    
    This feature is possible because `serde_ignored` exposes the exact
    signal Codex needs: it lets Codex run ordinary Serde deserialization
    while recording fields Serde would otherwise ignore. That avoids
    requiring `#[serde(deny_unknown_fields)]` across every config type and
    keeps strict validation opt-in around the existing config model.
    
    ## What Changed
    
    ### Added strict config validation
    
    - Added `serde_ignored`-based validation for `ConfigToml` in
    `codex-rs/config/src/strict_config.rs`.
    - Combined `serde_ignored` with `serde_path_to_error` so strict mode
    preserves typed config error paths while also collecting fields Serde
    would otherwise ignore.
    - Added strict-mode validation for unknown `[features]` keys, including
    keys that would otherwise be accepted by `FeaturesToml`'s flattened
    boolean map.
    - Kept typed config errors ahead of ignored-field reporting, so
    malformed known fields are reported before unknown-field diagnostics.
    - Added source-range diagnostics for top-level and nested unknown config
    fields, including non-file managed preference source names.
    
    ### Kept parsing single-pass per source
    
    - Reworked file and managed-config loading so strict validation reuses
    the already parsed `TomlValue` for that source.
    - For actual config files and managed config strings, the loader now
    reads once, parses once, and validates that same parsed value instead of
    deserializing multiple times.
    - Validated `-c` / `--config` override layers with the same
    base-directory context used for normal relative-path resolution, so
    unknown override keys are still reported when another override contains
    a relative path.
    
    ### Scoped `--strict-config` to config-heavy entry points
    
    - Added support for `--strict-config` on the main config-loading entry
    points where it is most useful:
      - `codex`
      - `codex resume`
      - `codex fork`
      - `codex exec`
      - `codex review`
      - `codex mcp-server`
      - `codex app-server` when running the server itself
      - the standalone `codex-app-server` binary
      - the standalone `codex-exec` binary
    - Commands outside that set now reject `--strict-config` early with
    targeted errors instead of accepting it everywhere through shared CLI
    plumbing.
    - `codex app-server` subcommands such as `proxy`, `daemon`, and
    `generate-*` are intentionally excluded from the first rollout.
    - When app-server strict mode sees invalid config, app-server exits with
    the config error instead of logging a warning and continuing with
    defaults.
    - Introduced a dedicated `ReviewCommand` wrapper in `codex-rs/cli`
    instead of extending shared `ReviewArgs`, so `--strict-config` stays on
    the outer config-loading command surface and does not become part of the
    reusable review payload used by `codex exec review`.
    
    ### Coverage
    
    - Added tests for top-level and nested unknown config fields, unknown
    `[features]` keys, typed-error precedence, source-location reporting,
    and non-file managed preference source names.
    - Added CLI coverage showing invalid `--enable`, invalid `--disable`,
    and unknown `-c` overrides still error when `--strict-config` is
    present, including compound-looking feature names such as
    `multi_agent_v2.subagent_usage_hint_text`.
    - Added integration coverage showing both `codex app-server
    --strict-config` and standalone `codex-app-server --strict-config` exit
    with an error for unknown config fields instead of starting with
    fallback defaults.
    - Added coverage showing unsupported command surfaces reject
    `--strict-config` with explicit errors.
    
    ## Example Usage
    
    Run Codex with strict config validation enabled:
    
    ```shell
    codex --strict-config
    ```
    
    Strict config mode is also available on the supported config-heavy
    subcommands:
    
    ```shell
    codex --strict-config exec "explain this repository"
    codex review --strict-config --uncommitted
    codex mcp-server --strict-config
    codex app-server --strict-config --listen off
    codex-app-server --strict-config --listen off
    ```
    
    For example, if `~/.codex/config.toml` contains a typo in a key name:
    
    ```toml
    model = "gpt-5"
    approval_polic = "on-request"
    ```
    
    then `codex --strict-config` reports the misspelled key instead of
    silently ignoring it. The path is shortened to `~` here for readability:
    
    ```text
    $ codex --strict-config
    Error loading config.toml:
    ~/.codex/config.toml:2:1: unknown configuration field `approval_polic`
      |
    2 | approval_polic = "on-request"
      | ^^^^^^^^^^^^^^
    ```
    
    Without `--strict-config`, Codex keeps the existing permissive behavior
    and ignores the unknown key.
    
    Strict config mode also validates ad-hoc `-c` / `--config` overrides:
    
    ```text
    $ codex --strict-config -c foo=bar
    Error: unknown configuration field `foo` in -c/--config override
    
    $ codex --strict-config -c features.foo=true
    Error: unknown configuration field `features.foo` in -c/--config override
    ```
    
    Invalid feature toggles are rejected too, including values that look
    like nested config paths:
    
    ```text
    $ codex --strict-config --enable does_not_exist
    Error: Unknown feature flag: does_not_exist
    
    $ codex --strict-config --disable does_not_exist
    Error: Unknown feature flag: does_not_exist
    
    $ codex --strict-config --enable multi_agent_v2.subagent_usage_hint_text
    Error: Unknown feature flag: multi_agent_v2.subagent_usage_hint_text
    ```
    
    Unsupported commands reject the flag explicitly:
    
    ```text
    $ codex --strict-config cloud list
    Error: `--strict-config` is not supported for `codex cloud`
    ```
    
    ## Verification
    
    The `codex-cli` `strict_config` tests cover invalid `--enable`, invalid
    `--disable`, the compound `multi_agent_v2.subagent_usage_hint_text`
    case, unknown `-c` overrides, app-server strict startup failure through
    `codex app-server`, and rejection for unsupported commands such as
    `codex cloud`, `codex mcp`, `codex remote-control`, and `codex
    app-server proxy`.
    
    The config and config-loader tests cover unknown top-level fields,
    unknown nested fields, unknown `[features]` keys, source-location
    reporting, non-file managed config sources, and `-c` validation for keys
    such as `features.foo`.
    
    The app-server test suite covers standalone `codex-app-server
    --strict-config` startup failure for an unknown config field.
    
    ## Documentation
    
    The Codex CLI docs on developers.openai.com/codex should mention
    `--strict-config` as an opt-in validation mode for supported
    config-heavy entry points once this ships.
  • Disable empty Cargo test targets (#21584)
    ## Summary
    
    `cargo test` has entails both running standard Rust tests and doctests.
    It turns out that the doctest discovery is fairly slow, and it's a cost
    you pay even for crates that don't include any doctests.
    
    This PR disables doctests with `doctest = false` for crates that lack
    any doctests.
    
    For the collection of crates below, this speeds up test execution by
    >4x.
    
    E.g., before this PR:
    
    ```
    Benchmark 1: cargo test     -p codex-utils-absolute-path     -p codex-utils-cache     -p codex-utils-cli     -p codex-utils-home-dir     -p codex-utils-output-truncation     -p codex-utils-path     -p codex-utils-string     -p codex-utils-template     -p codex-utils-elapsed     -p codex-utils-json-to-toml
      Time (mean ± σ):      1.849 s ±  4.455 s    [User: 0.752 s, System: 1.367 s]
      Range (min … max):    0.418 s … 14.529 s    10 runs
    ```
    
    And after:
    
    ```
    Benchmark 1: cargo test     -p codex-utils-absolute-path     -p codex-utils-cache     -p codex-utils-cli     -p codex-utils-home-dir     -p codex-utils-output-truncation     -p codex-utils-path     -p codex-utils-string     -p codex-utils-template     -p codex-utils-elapsed     -p codex-utils-json-to-toml
      Time (mean ± σ):     428.6 ms ±   6.9 ms    [User: 187.7 ms, System: 219.7 ms]
      Range (min … max):   418.0 ms … 436.8 ms    10 runs
    ```
    
    For a single crate, with >2x speedup, before:
    
    ```
    Benchmark 1: cargo test -p codex-utils-string
      Time (mean ± σ):     491.1 ms ±   9.0 ms    [User: 229.8 ms, System: 234.9 ms]
      Range (min … max):   480.9 ms … 512.0 ms    10 runs
    ```
    
    And after:
    
    ```
    Benchmark 1: cargo test -p codex-utils-string
      Time (mean ± σ):     213.9 ms ±   4.3 ms    [User: 112.8 ms, System: 84.0 ms]
      Range (min … max):   206.8 ms … 221.0 ms    13 runs
    ```
    
    Co-authored-by: Codex <noreply@openai.com>
  • Refactor exec-server filesystem API into codex-file-system (#19892)
    ## Summary
    - Extracted the shared filesystem types and `ExecutorFileSystem` trait
    into a new `codex-file-system` crate
    - Switched `codex-config` and `codex-git-utils` to depend on that crate
    instead of `codex-exec-server`
    - Kept `codex-exec-server` re-exporting the same API for existing
    callers
    
    ## Testing
    - Ran `cargo test -p codex-file-system`
    - Ran `cargo test -p codex-git-utils`
    - Ran `cargo test -p codex-config`
    - Ran `cargo test -p codex-exec-server`
    - Ran `just fix -p codex-file-system`, `just fix -p codex-git-utils`,
    `just fix -p codex-config`, `just fix -p codex-exec-server`
    - Ran `just fmt`
    - Updated and verified the Bazel module lockfile
  • [codex] Move config loading into codex-config (#19487)
    ## Why
    
    Config loading had become split across crates: `codex-config` owned the
    config types and merge logic, while `codex-core` still owned the loader
    that assembled the layer stack. This change consolidates that
    responsibility in `codex-config`, so the crate that defines config
    behavior also owns how configs are discovered and loaded.
    
    To make that move possible without reintroducing the old dependency
    cycle, the shell-environment policy types and helpers that
    `codex-exec-server` needs now live in `codex-protocol` instead of
    flowing through `codex-config`.
    
    This also makes the migrated loader tests more deterministic on machines
    that already have managed or system Codex config installed by letting
    tests override the system config and requirements paths instead of
    reading the host's `/etc/codex`.
    
    ## What Changed
    
    - moved the config loader implementation from `codex-core` into
    `codex-config::loader` and deleted the old `core::config_loader` module
    instead of leaving a compatibility shim
    - moved shell-environment policy types and helpers into
    `codex-protocol`, then updated `codex-exec-server` and other downstream
    crates to import them from their new home
    - updated downstream callers to use loader/config APIs from
    `codex-config`
    - added test-only loader overrides for system config and requirements
    paths so loader-focused tests do not depend on host-managed config state
    - cleaned up now-unused dependency entries and platform-specific cfgs
    that were surfaced by post-push CI
    
    ## Testing
    
    - `cargo test -p codex-config`
    - `cargo test -p codex-core config_loader_tests::`
    - `cargo test -p codex-protocol -p codex-exec-server -p
    codex-cloud-requirements -p codex-rmcp-client --lib`
    - `cargo test --lib -p codex-app-server-client -p codex-exec`
    - `cargo test --no-run --lib -p codex-app-server`
    - `cargo test -p codex-linux-sandbox --lib`
    - `cargo shear`
    - `just bazel-lock-check`
    
    ## Notes
    
    - I did not chase unrelated full-suite failures outside the migrated
    loader surface.
    - `cargo test -p codex-core --lib` still hits unrelated proxy-sensitive
    failures on this machine, and Windows CI still shows unrelated
    long-running/timeouting test noise outside the loader migration itself.
  • Add remote thread config loader protos (#18892)
    ## Why
    
    Thread-scoped config needs a stable boundary between the app/session
    owner and the config stack. Instead of having call sites manually copy
    thread config fields into individual overrides, this adds the proto and
    Rust plumbing needed for a `ThreadConfigLoader` implementation to return
    typed sources that can be translated into ordinary config layer entries.
    
    Keeping the remote payload typed also makes precedence easier to reason
    about: session-owned thread config maps back to the existing session
    config source, while user-owned thread config is represented separately
    without introducing a new config-layer source until it has TOML-backed
    fields.
    
    ## What changed
    
    - Added the `codex.thread_config.v1` protobuf service and generated Rust
    module for loading thread config sources.
    - Added `RemoteThreadConfigLoader`, which calls the gRPC service, parses
    `SessionThreadConfig` / `UserThreadConfig`, and validates provider
    fields such as `wire_api`, auth timeout, and absolute auth cwd.
    - Added proto generation tooling under
    `config/scripts/generate-proto.sh` and
    `config/examples/generate-proto.rs`.
    - Added `ThreadConfigLoader::load_config_layers`, plus static/no-op
    loader helpers, so tests and callers can use the same typed loader
    interface while config-layer translation stays centralized.
    
    ## Verification
    
    - `cargo test -p codex-config thread_config`
  • Add remote_sandbox_config to our config requirements (#18763)
    ## Why
    
    Customers need finer-grained control over allowed sandbox modes based on
    the host Codex is running on. For example, they may want stricter
    sandbox limits on devboxes while keeping a different default elsewhere.
    
    Our current cloud requirements can target user/account groups, but they
    cannot vary sandbox requirements by host. That makes remote development
    environments awkward because the same top-level `allowed_sandbox_modes`
    has to apply everywhere.
    
    ## What
    
    Adds a new `remote_sandbox_config` section to `requirements.toml`:
    
    ```toml
    allowed_sandbox_modes = ["read-only"]
    
    [[remote_sandbox_config]]
    hostname_patterns = ["*.org"]
    allowed_sandbox_modes = ["read-only", "workspace-write"]
    
    [[remote_sandbox_config]]
    hostname_patterns = ["*.sh", "runner-*.ci"]
    allowed_sandbox_modes = ["read-only", "danger-full-access"]
    ```
    
    During requirements resolution, Codex resolves the local host name once,
    preferring the machine FQDN when available and falling back to the
    cleaned kernel hostname. This host classification is best effort rather
    than authenticated device proof.
    
    Each requirements source applies its first matching
    `remote_sandbox_config` entry before it is merged with other sources.
    The shared merge helper keeps that `apply_remote_sandbox_config` step
    paired with requirements merging so new requirements sources do not have
    to remember the extra call.
    
    That preserves source precedence: a lower-precedence requirements file
    with a matching `remote_sandbox_config` cannot override a
    higher-precedence source that already set `allowed_sandbox_modes`.
    
    This also wires the hostname-aware resolution through app-server,
    CLI/TUI config loading, config API reads, and config layer metadata so
    they all evaluate remote sandbox requirements consistently.
    
    ## Verification
    
    - `cargo test -p codex-config remote_sandbox_config`
    - `cargo test -p codex-config host_name`
    - `cargo test -p codex-core
    load_config_layers_applies_matching_remote_sandbox_config`
    - `cargo test -p codex-core
    system_remote_sandbox_config_keeps_cloud_sandbox_modes`
    - `cargo test -p codex-config`
    - `cargo test -p codex-core` unit tests passed; `tests/all.rs`
    integration matrix was intentionally stopped after the relevant focused
    tests passed
    - `just fix -p codex-config`
    - `just fix -p codex-core`
    - `cargo check -p codex-app-server`
  • Add session config loader interface (#18208)
    ## Why
    
    Cloud-hosted sessions need a way for the service that starts or manages
    a thread to provide session-owned config without treating all config as
    if it came from the same user/project/workspace TOML stack.
    
    The important boundary is ownership: some values should be controlled by
    the session/orchestrator, some by the authenticated user, and later some
    may come from the executor. The earlier broad config-store shape made
    that boundary too fuzzy and overlapped heavily with the existing
    filesystem-backed config loader. This PR starts with the smaller piece
    we need now: a typed session config loader that can feed the existing
    config layer stack while preserving the normal precedence and merge
    behavior.
    
    ## What Changed
    
    - Added `ThreadConfigLoader` and related typed payloads in
    `codex-config`.
    - `SessionThreadConfig` currently supports `model_provider`,
    `model_providers`, and feature flags.
    - `UserThreadConfig` is present as an ownership boundary, but does not
    yet add TOML-backed fields.
    - `NoopThreadConfigLoader` preserves existing behavior when no external
    loader is configured.
      - `StaticThreadConfigLoader` supports tests and simple callers.
    
    - Taught thread config sources to produce ordinary `ConfigLayerEntry`
    values so the existing `ConfigLayerStack` remains the place where
    precedence and merging happen.
    
    - Wired the loader through `ConfigBuilder`, the config loader, and
    app-server startup paths so app-server can provide session-owned config
    before deriving a thread config.
    
    - Added coverage for:
      - translating typed thread config into config layers,
    - inserting thread config layers into the stack at the right precedence,
    - applying session-provided model provider and feature settings when
    app-server derives config from thread params.
    
    ## Follow-Ups
    
    This intentionally stops short of adding the remote/service transport.
    The next pieces are expected to be:
    
    1. Define the proto/API shape for this interface.
    2. Add a client implementation that can source session config from the
    service side.
    
    ## Verification
    
    - Added unit coverage in `codex-config` for the loader and layer
    conversion.
    - Added `codex-core` config loader coverage for thread config layer
    precedence.
    - Added app-server coverage that verifies session thread config wins
    over request-provided config for model provider and feature settings.
  • Refactor config loading to use filesystem abstraction (#18209)
    Initial pass propagating FileSystem through config loading.
  • Refactor config types into a separate crate (#16962)
    Move config types into a separate crate because their macros expand into
    a lot of new code.
  • [codex] Move config types into codex-config (#16523)
    ## Why
    
    `codex-rs/core/src/config/types.rs` is a plain config-type module with
    no dependency on `codex-core`. Moving it into `codex-config` shrinks the
    core crate and gives config-only consumers a more natural dependency
    boundary.
    
    ## What Changed
    
    - Added `codex_config::types` with the moved structs, enums, constants,
    and unit tests.
    - Kept `codex_core::config::types` as a compatibility re-export to avoid
    a broad call-site migration in this PR.
    - Switched notice-table writes in `core/src/config/edit.rs` to a local
    `NOTICE_TABLE_KEY` constant.
    - Added the `wildmatch` runtime dependency and `tempfile` test
    dependency to `codex-config`.
  • Extract codex-core-skills crate (#15749)
    ## Summary
    - move skill loading and management into codex-core-skills
    - leave codex-core with the thin integration layer and shared wiring
    
    ## Testing
    - CI
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • chore: move config diagnostics out of codex-core (#12427)
    ## Why
    
    Compiling `codex-rs/core` is a bottleneck for local iteration, so this
    change continues the ongoing extraction of config-related functionality
    out of `codex-core` and into `codex-config`.
    
    The goal is not just to move code, but to reduce `codex-core` ownership
    and indirection so more code depends on `codex-config` directly.
    
    ## What Changed
    
    - Moved config diagnostics logic from
    `core/src/config_loader/diagnostics.rs` into
    `config/src/diagnostics.rs`.
    - Updated `codex-core` to use `codex-config` diagnostics types/functions
    directly where possible.
    - Removed the `core/src/config_loader/diagnostics.rs` shim module
    entirely; the remaining `ConfigToml`-specific calls are in
    `core/src/config_loader/mod.rs`.
    - Moved `CONFIG_TOML_FILE` into `codex-config` and updated existing
    references to use `codex_config::CONFIG_TOML_FILE` directly.
    - Added a direct `codex-config` dependency to `codex-cli` for its
    `CONFIG_TOML_FILE` use.
  • Extract codex-config from codex-core (#11389)
    `codex-core` had accumulated config loading, requirements parsing,
    constraint logic, and config-layer state handling in a single crate.
    This change extracts that subsystem into `codex-config` to reduce
    `codex-core` rebuild/test surface area and isolate future config work.
    
    ## What Changed
    
    ### Added `codex-config`
    
    - Added new workspace crate `codex-rs/config` (`codex-config`).
    - Added workspace/build wiring in:
      - `codex-rs/Cargo.toml`
      - `codex-rs/config/Cargo.toml`
      - `codex-rs/config/BUILD.bazel`
    - Updated lockfiles (`codex-rs/Cargo.lock`, `MODULE.bazel.lock`).
    - Added `codex-core` -> `codex-config` dependency in
    `codex-rs/core/Cargo.toml`.
    
    ### Moved config internals from `core` into `config`
    
    Moved modules to `codex-rs/config/src/`:
    
    - `core/src/config/constraint.rs` -> `config/src/constraint.rs`
    - `core/src/config_loader/cloud_requirements.rs` ->
    `config/src/cloud_requirements.rs`
    - `core/src/config_loader/config_requirements.rs` ->
    `config/src/config_requirements.rs`
    - `core/src/config_loader/fingerprint.rs` -> `config/src/fingerprint.rs`
    - `core/src/config_loader/merge.rs` -> `config/src/merge.rs`
    - `core/src/config_loader/overrides.rs` -> `config/src/overrides.rs`
    - `core/src/config_loader/requirements_exec_policy.rs` ->
    `config/src/requirements_exec_policy.rs`
    - `core/src/config_loader/state.rs` -> `config/src/state.rs`
    
    `codex-config` now re-exports this surface from `config/src/lib.rs` at
    the crate top level.
    
    ### Updated `core` to consume/re-export `codex-config`
    
    - `core/src/config_loader/mod.rs` now imports/re-exports config-loader
    types/functions from top-level `codex_config::*`.
    - Local moved modules were removed from `core/src/config_loader/`.
    - `core/src/config/mod.rs` now re-exports constraint types from
    `codex_config`.