Commit Graph

13 Commits

  • Build remote exec env from exec-server policy (#17216)
    ## Summary
    - add an exec-server `envPolicy` field; when present, the server starts
    from its own process env and applies the shell environment policy there
    - keep `env` as the exact environment for local/embedded starts, but
    make it an overlay for remote unified-exec starts
    - move the shell-environment-policy builder into `codex-config` so Core
    and exec-server share the inherit/filter/set/include behavior
    - overlay only runtime/sandbox/network deltas from Core onto the
    exec-server-derived env
    
    ## Why
    Remote unified exec was materializing the shell env inside Core and
    forwarding the whole map to exec-server, so remote processes could
    inherit the orchestrator machine's `HOME`, `PATH`, etc. This keeps the
    base env on the executor while preserving Core-owned runtime additions
    like `CODEX_THREAD_ID`, unified-exec defaults, network proxy env, and
    sandbox marker env.
    
    ## Validation
    - `just fmt`
    - `git diff --check`
    - `cargo test -p codex-exec-server --lib`
    - `cargo test -p codex-core --lib unified_exec::process_manager::tests`
    - `cargo test -p codex-core --lib exec_env::tests`
    - `cargo test -p codex-core --lib exec_env_tests` (compile-only; filter
    matched 0 tests)
    - `cargo test -p codex-config --lib shell_environment` (compile-only;
    filter matched 0 tests)
    - `just bazel-lock-update`
    
    ## Known local validation issue
    - `just bazel-lock-check` is not runnable in this checkout: it invokes
    `./scripts/check-module-bazel-lock.sh`, which is missing.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
    Co-authored-by: pakrym-oai <pakrym@openai.com>
  • Add marketplace command (#17087)
    Added a new top-level `codex marketplace add` command for installing
    plugin marketplaces into Codex’s local marketplace cache.
    
    This change adds source parsing for local directories, GitHub shorthand,
    and git URLs, supports optional `--ref` and git-only `--sparse` checkout
    paths, stages the source in a temp directory, validates the marketplace
    manifest, and installs it under
    `$CODEX_HOME/marketplaces/<marketplace-name>`
    
    Included tests cover local install behavior in the CLI and marketplace
    discovery from installed roots in core. Scoped formatting and fix passes
    were run, and targeted CLI/core tests passed.
  • Refactor config types into a separate crate (#16962)
    Move config types into a separate crate because their macros expand into
    a lot of new code.
  • [codex] Move config types into codex-config (#16523)
    ## Why
    
    `codex-rs/core/src/config/types.rs` is a plain config-type module with
    no dependency on `codex-core`. Moving it into `codex-config` shrinks the
    core crate and gives config-only consumers a more natural dependency
    boundary.
    
    ## What Changed
    
    - Added `codex_config::types` with the moved structs, enums, constants,
    and unit tests.
    - Kept `codex_core::config::types` as a compatibility re-export to avoid
    a broad call-site migration in this PR.
    - Switched notice-table writes in `core/src/config/edit.rs` to a local
    `NOTICE_TABLE_KEY` constant.
    - Added the `wildmatch` runtime dependency and `tempfile` test
    dependency to `codex-config`.
  • Extract MCP into codex-mcp crate (#15919)
    - Split MCP runtime/server code out of `codex-core` into the new
    `codex-mcp` crate. New/moved public structs/types include `McpConfig`,
    `McpConnectionManager`, `ToolInfo`, `ToolPluginProvenance`,
    `CodexAppsToolsCacheKey`, and the `McpManager` API
    (`codex_mcp::mcp::McpManager` plus the `codex_core::mcp::McpManager`
    wrapper/shim). New/moved functions include `with_codex_apps_mcp`,
    `configured_mcp_servers`, `effective_mcp_servers`,
    `collect_mcp_snapshot`, `collect_mcp_snapshot_from_manager`,
    `qualified_mcp_tool_name_prefix`, and the MCP auth/skill-dependency
    helpers. Why: this creates a focused MCP crate boundary and shrinks
    `codex-core` without forcing every consumer to migrate in the same PR.
    
    - Move MCP server config schema and persistence into `codex-config`.
    New/moved structs/enums include `AppToolApproval`,
    `McpServerToolConfig`, `McpServerConfig`, `RawMcpServerConfig`,
    `McpServerTransportConfig`, `McpServerDisabledReason`, and
    `codex_config::ConfigEditsBuilder`. New/moved functions include
    `load_global_mcp_servers` and
    `ConfigEditsBuilder::replace_mcp_servers`/`apply`. Why: MCP TOML
    parsing/editing is config ownership, and this keeps config
    validation/round-tripping (including per-tool approval overrides and
    inline bearer-token rejection) in the config crate instead of
    `codex-core`.
    
    - Rewire `codex-core`, app-server, and plugin call sites onto the new
    crates. Updated `Config::to_mcp_config(&self, plugins_manager)`,
    `codex-rs/core/src/mcp.rs`, `codex-rs/core/src/connectors.rs`,
    `codex-rs/core/src/codex.rs`,
    `CodexMessageProcessor::list_mcp_server_status_task`, and
    `utils/plugins/src/mcp_connector.rs` to build/pass the new MCP
    config/runtime types. Why: plugin-provided MCP servers still merge with
    user-configured servers, and runtime auth (`CodexAuth`) is threaded into
    `with_codex_apps_mcp` / `collect_mcp_snapshot` explicitly so `McpConfig`
    stays config-only.
  • chore: refactor network permissions to use explicit domain and unix socket rule maps (#15120)
    ## Summary
    
    This PR replaces the legacy network allow/deny list model with explicit
    rule maps for domains and unix sockets across managed requirements,
    permissions profiles, the network proxy config, and the app server
    protocol.
    
    Concretely, it:
    
    - introduces typed domain (`allow` / `deny`) and unix socket permission
    (`allow` / `none`) entries instead of separate `allowed_domains`,
    `denied_domains`, and `allow_unix_sockets` lists
    - updates config loading, managed requirements merging, and exec-policy
    overlays to read and upsert rule entries consistently
    - exposes the new shape through protocol/schema outputs, debug surfaces,
    and app-server config APIs
    - rejects the legacy list-based keys and updates docs/tests to reflect
    the new config format
    
    ## Why
    
    The previous representation split related network policy across multiple
    parallel lists, which made merging and overriding rules harder to reason
    about. Moving to explicit keyed permission maps gives us a single source
    of truth per host/socket entry, makes allow/deny precedence clearer, and
    gives protocol consumers access to the full rule state instead of
    derived projections only.
    
    ## Backward Compatibility
    
    ### Backward compatible
    
    - Managed requirements still accept the legacy
    `experimental_network.allowed_domains`,
    `experimental_network.denied_domains`, and
    `experimental_network.allow_unix_sockets` fields. They are normalized
    into the new canonical `domains` and `unix_sockets` maps internally.
    - App-server v2 still deserializes legacy `allowedDomains`,
    `deniedDomains`, and `allowUnixSockets` payloads, so older clients can
    continue reading managed network requirements.
    - App-server v2 responses still populate `allowedDomains`,
    `deniedDomains`, and `allowUnixSockets` as legacy compatibility views
    derived from the canonical maps.
    - `managed_allowed_domains_only` keeps the same behavior after
    normalization. Legacy managed allowlists still participate in the same
    enforcement path as canonical `domains` entries.
    
    ### Not backward compatible
    
    - Permissions profiles under `[permissions.<profile>.network]` no longer
    accept the legacy list-based keys. Those configs must use the canonical
    `[domains]` and `[unix_sockets]` tables instead of `allowed_domains`,
    `denied_domains`, or `allow_unix_sockets`.
    - Managed `experimental_network` config cannot mix canonical and legacy
    forms in the same block. For example, `domains` cannot be combined with
    `allowed_domains` or `denied_domains`, and `unix_sockets` cannot be
    combined with `allow_unix_sockets`.
    - The canonical format can express explicit `"none"` entries for unix
    sockets, but those entries do not round-trip through the legacy
    compatibility fields because the legacy fields only represent allow/deny
    lists.
    ## Testing
    `/target/debug/codex sandbox macos --log-denials /bin/zsh -c 'curl
    https://www.example.com' ` gives 200 with config
    ```
    [permissions.workspace.network.domains]
    "www.example.com" = "allow"
    ```
    and fails when set to deny: `curl: (56) CONNECT tunnel failed, response
    403`.
    
    Also tested backward compatibility path by verifying that adding the
    following to `/etc/codex/requirements.toml` works:
    ```
    [experimental_network]
    allowed_domains = ["www.example.com"]
    ```
  • Extract codex-core-skills crate (#15749)
    ## Summary
    - move skill loading and management into codex-core-skills
    - leave codex-core with the thin integration layer and shared wiring
    
    ## Testing
    - CI
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Override local apps settings with requirements.toml settings (#14304)
    This PR changes app and connector enablement when `requirements.toml` is
    present locally or via remote configuration.
    
    For apps.* entries:
    - `enabled = false` in `requirements.toml` overrides the user’s local
    `config.toml` and forces the app to be disabled.
    - `enabled = true` in `requirements.toml` does not re-enable an app the
    user has disabled in config.toml.
    
    This behavior applies whether or not the user has an explicit entry for
    that app in `config.toml`. It also applies to cloud-managed policies and
    configurations when the admin sets the override through
    `requirements.toml`.
    
    Scenarios tested and verified:
    - Remote managed, user config (present) override
    - Admin-defined policies & configurations include a connector override:
      `[apps.<appID>]
    enabled = false`
    - User's config.toml has the same connector configured with `enabled =
    true`
      - TUI/App should show connector as disabled
      - Connector should be unavailable for use in the composer
      
    - Remote managed, user config (absent) override
    - Admin-defined policies & configurations include a connector override:
      `[apps.<appID>]
    enabled = false`
      - User's config.toml has no entry for the the same connector
      - TUI/App should show connector as disabled
      - Connector should be unavailable for use in the composer
      
    - Locally managed, user config (present) override
      - Local requirements.toml includes a connector override:
      `[apps.<appID>]
    enabled = false`
    - User's config.toml has the same connector configured with `enabled =
    true`
      - TUI/App should show connector as disabled
      - Connector should be unavailable for use in the composer
    
    - Locally managed, user config (absent) override
      - Local requirements.toml includes a connector override:
      `[apps.<appID>]
    enabled = false`
      - User's config.toml has no entry for the the same connector
      - TUI/App should show connector as disabled
      - Connector should be unavailable for use in the composer
    
    
    
    
    <img width="1446" height="753" alt="image"
    src="https://github.com/user-attachments/assets/61c714ca-dcca-4952-8ad2-0afc16ff3835"
    />
    <img width="595" height="233" alt="image"
    src="https://github.com/user-attachments/assets/7c8ab147-8fd7-429a-89fb-591c21c15621"
    />
  • Refactor cloud requirements error and surface in JSON-RPC error (#14504)
    Refactors cloud requirements error handling to carry structured error
    metadata and surfaces that metadata through JSON-RPC config-load
    failures, including:
    * adds typed CloudRequirementsLoadErrorCode values plus optional
    statusCode
    * marks thread/start, thread/resume, and thread/fork config failures
    with structured cloud-requirements error data
  • config: enforce enterprise feature requirements (#13388)
    ## Why
    
    Enterprises can already constrain approvals, sandboxing, and web search
    through `requirements.toml` and MDM, but feature flags were still only
    configurable as managed defaults. That meant an enterprise could suggest
    feature values, but it could not actually pin them.
    
    This change closes that gap and makes enterprise feature requirements
    behave like the other constrained settings. The effective feature set
    now stays consistent with enterprise requirements during config load,
    when config writes are validated, and when runtime code mutates feature
    flags later in the session.
    
    It also tightens the runtime API for managed features. `ManagedFeatures`
    now follows the same constraint-oriented shape as `Constrained<T>`
    instead of exposing panic-prone mutation helpers, and production code
    can no longer construct it through an unconstrained `From<Features>`
    path.
    
    The PR also hardens the `compact_resume_fork` integration coverage on
    Windows. After the feature-management changes,
    `compact_resume_after_second_compaction_preserves_history` was
    overflowing the libtest/Tokio thread stacks on Windows, so the test now
    uses an explicit larger-stack harness as a pragmatic mitigation. That
    may not be the ideal root-cause fix, and it merits a parallel
    investigation into whether part of the async future chain should be
    boxed to reduce stack pressure instead.
    
    ## What Changed
    
    Enterprises can now pin feature values in `requirements.toml` with the
    requirements-side `features` table:
    
    ```toml
    [features]
    personality = true
    unified_exec = false
    ```
    
    Only canonical feature keys are allowed in the requirements `features`
    table; omitted keys remain unconstrained.
    
    - Added a requirements-side pinned feature map to
    `ConfigRequirementsToml`, threaded it through source-preserving
    requirements merge and normalization in `codex-config`, and made the
    TOML surface use `[features]` (while still accepting legacy
    `[feature_requirements]` for compatibility).
    - Exposed `featureRequirements` from `configRequirements/read`,
    regenerated the JSON/TypeScript schema artifacts, and updated the
    app-server README.
    - Wrapped the effective feature set in `ManagedFeatures`, backed by
    `ConstrainedWithSource<Features>`, and changed its API to mirror
    `Constrained<T>`: `can_set(...)`, `set(...) -> ConstraintResult<()>`,
    and result-returning `enable` / `disable` / `set_enabled` helpers.
    - Removed the legacy-usage and bulk-map passthroughs from
    `ManagedFeatures`; callers that need those behaviors now mutate a plain
    `Features` value and reapply it through `set(...)`, so the constrained
    wrapper remains the enforcement boundary.
    - Removed the production loophole for constructing unconstrained
    `ManagedFeatures`. Non-test code now creates it through the configured
    feature-loading path, and `impl From<Features> for ManagedFeatures` is
    restricted to `#[cfg(test)]`.
    - Rejected legacy feature aliases in enterprise feature requirements,
    and return a load error when a pinned combination cannot survive
    dependency normalization.
    - Validated config writes against enterprise feature requirements before
    persisting changes, including explicit conflicting writes and
    profile-specific feature states that normalize into invalid
    combinations.
    - Updated runtime and TUI feature-toggle paths to use the constrained
    setter API and to persist or apply the effective post-constraint value
    rather than the requested value.
    - Updated the `core_test_support` Bazel target to include the bundled
    core model-catalog fixtures in its runtime data, so helper code that
    resolves `core/models.json` through runfiles works in remote Bazel test
    environments.
    - Renamed the core config test coverage to emphasize that effective
    feature values are normalized at runtime, while conflicting persisted
    config writes are rejected.
    - Ran `compact_resume_after_second_compaction_preserves_history` inside
    an explicit 8 MiB test thread and Tokio runtime worker stack, following
    the existing larger-stack integration-test pattern, to keep the Windows
    `compact_resume_fork` test slice from aborting while a parallel
    investigation continues into whether some of the underlying async
    futures should be boxed.
    
    ## Verification
    
    - `cargo test -p codex-config`
    - `cargo test -p codex-core feature_requirements_ -- --nocapture`
    - `cargo test -p codex-core
    load_requirements_toml_produces_expected_constraints -- --nocapture`
    - `cargo test -p codex-core
    compact_resume_after_second_compaction_preserves_history -- --nocapture`
    - `cargo test -p codex-core compact_resume_fork -- --nocapture`
    - Re-ran the built `codex-core` `tests/all` binary with
    `RUST_MIN_STACK=262144` for
    `compact_resume_after_second_compaction_preserves_history` to confirm
    the explicit-stack harness fixes the deterministic low-stack repro.
    - `cargo test -p codex-core`
    - This still fails locally in unrelated integration areas that expect
    the `codex` / `test_stdio_server` binaries or hit existing `search_tool`
    wiremock mismatches.
    
    ## Docs
    
    `developers.openai.com/codex` should document the requirements-side
    `[features]` table for enterprise and MDM-managed configuration,
    including that it only accepts canonical feature keys and that
    conflicting config writes are rejected.
  • Make cloud_requirements fail close (#13063)
    Make it fail-close only for CLI for now
    Will extend this for app-server later
  • chore: move config diagnostics out of codex-core (#12427)
    ## Why
    
    Compiling `codex-rs/core` is a bottleneck for local iteration, so this
    change continues the ongoing extraction of config-related functionality
    out of `codex-core` and into `codex-config`.
    
    The goal is not just to move code, but to reduce `codex-core` ownership
    and indirection so more code depends on `codex-config` directly.
    
    ## What Changed
    
    - Moved config diagnostics logic from
    `core/src/config_loader/diagnostics.rs` into
    `config/src/diagnostics.rs`.
    - Updated `codex-core` to use `codex-config` diagnostics types/functions
    directly where possible.
    - Removed the `core/src/config_loader/diagnostics.rs` shim module
    entirely; the remaining `ConfigToml`-specific calls are in
    `core/src/config_loader/mod.rs`.
    - Moved `CONFIG_TOML_FILE` into `codex-config` and updated existing
    references to use `codex_config::CONFIG_TOML_FILE` directly.
    - Added a direct `codex-config` dependency to `codex-cli` for its
    `CONFIG_TOML_FILE` use.
  • Extract codex-config from codex-core (#11389)
    `codex-core` had accumulated config loading, requirements parsing,
    constraint logic, and config-layer state handling in a single crate.
    This change extracts that subsystem into `codex-config` to reduce
    `codex-core` rebuild/test surface area and isolate future config work.
    
    ## What Changed
    
    ### Added `codex-config`
    
    - Added new workspace crate `codex-rs/config` (`codex-config`).
    - Added workspace/build wiring in:
      - `codex-rs/Cargo.toml`
      - `codex-rs/config/Cargo.toml`
      - `codex-rs/config/BUILD.bazel`
    - Updated lockfiles (`codex-rs/Cargo.lock`, `MODULE.bazel.lock`).
    - Added `codex-core` -> `codex-config` dependency in
    `codex-rs/core/Cargo.toml`.
    
    ### Moved config internals from `core` into `config`
    
    Moved modules to `codex-rs/config/src/`:
    
    - `core/src/config/constraint.rs` -> `config/src/constraint.rs`
    - `core/src/config_loader/cloud_requirements.rs` ->
    `config/src/cloud_requirements.rs`
    - `core/src/config_loader/config_requirements.rs` ->
    `config/src/config_requirements.rs`
    - `core/src/config_loader/fingerprint.rs` -> `config/src/fingerprint.rs`
    - `core/src/config_loader/merge.rs` -> `config/src/merge.rs`
    - `core/src/config_loader/overrides.rs` -> `config/src/overrides.rs`
    - `core/src/config_loader/requirements_exec_policy.rs` ->
    `config/src/requirements_exec_policy.rs`
    - `core/src/config_loader/state.rs` -> `config/src/state.rs`
    
    `codex-config` now re-exports this surface from `config/src/lib.rs` at
    the crate top level.
    
    ### Updated `core` to consume/re-export `codex-config`
    
    - `core/src/config_loader/mod.rs` now imports/re-exports config-loader
    types/functions from top-level `codex_config::*`.
    - Local moved modules were removed from `core/src/config_loader/`.
    - `core/src/config/mod.rs` now re-exports constraint types from
    `codex_config`.