Commit Graph

27 Commits

  • Bridge host-loaded skills into the skills extension (#26172)
    ## Why
    
    The skills extension needs to become the path that exposes local host
    skills without losing the behavior already owned by core skill loading.
    Host skill discovery is not just `$CODEX_HOME/skills`: it also includes
    config layers, bundled-skill settings, plugin roots, runtime extra
    roots, and the filesystem for the selected primary environment.
    
    Rather than making the extension reload host skills and risk drifting
    from that authoritative load, this PR bridges the already-loaded
    per-turn skills outcome into the extension. That lets the extension
    advertise host skills and inject explicit `$skill` prompts while
    preserving the same roots, disabled/hidden state, rendered paths, and
    environment-backed file reads that the legacy path uses.
    
    ## What Changed
    
    - Adds `HostLoadedSkills` in `core-skills` to wrap the turn's
    `SkillLoadOutcome` and read `SKILL.md` through the filesystem that
    loaded that skill.
    - Stores `HostLoadedSkills` in turn extension data for normal turns and
    review turns, so the skills extension can consume the loaded host
    catalog without reloading it.
    - Adds `HostSkillProvider` under `ext/skills/src/provider/host.rs`,
    mapping host-loaded skill metadata into the skills-extension
    catalog/read contract.
    - Registers the host provider by default from
    `codex_skills_extension::install()`.
    - Preserves host skill metadata such as dependencies, disabled state,
    hidden-from-prompt policy, and slash-normalized display paths.
    - Passes host-loaded skills through `SkillListQuery` and
    `SkillReadRequest` so explicit skill invocation reads only resources
    from the loaded host catalog.
    - Adds integration coverage for a real legacy
    `$CODEX_HOME/skills/.../SKILL.md` skill being listed and injected
    through the installed extension.
    
    ## Testing
    
    - Added `installed_extension_loads_host_skills_from_legacy_roots` in
    `ext/skills/tests/skills_extension.rs`.
    - `just test -p codex-skills-extension`
  • skills: resolve per-turn catalogs from turn input context (#26106)
    ## Why
    
    The skills extension needs the resolved turn environments to build a
    real per-turn `SkillListQuery`. The previous `TurnLifecycleContributor`
    hook only had a turn id, so it could only seed a placeholder query and
    never carry the executor authorities that executor-scoped skill routing
    will need.
    
    Moving catalog resolution onto `TurnInputContributor` puts the skills
    extension on the same turn-preparation path that already has the
    environment ids and working directories for the submitted turn, while
    keeping the actual prompt injection work for follow-up changes.
    
    ## What changed
    
    - switch `ext/skills` from `TurnLifecycleContributor` to
    `TurnInputContributor`
    - build `executor_authorities` from `TurnInputContext.environments` and
    pass them through `SkillListQuery`
    - keep storing the resolved catalog in `SkillsTurnState`, but drop the
    placeholder query helper that no longer matches the real data flow
    - update the extension TODOs to reflect that per-turn catalog resolution
    now happens in the turn-input contributor, and that prompt/context
    injection still needs to move later
    
    ## Testing
    
    - Not run locally.
  • chore: extract context fragments into dedicated crate (#26122)
    ## Why
    
    `codex-core` currently owns the generic contextual-fragment trait and
    several reusable fragment implementations. That makes it harder for
    other crates to share the same host-owned model-input abstraction
    without depending on all of `codex-core`.
    
    This change extracts the reusable fragment machinery into a small
    `codex-context-fragments` crate so future extension and skills work can
    depend on the fragment abstraction directly.
    
    ## What Changed
    
    - Added the `codex-context-fragments` crate with:
      - `ContextualUserFragment`
      - `FragmentRegistration` / `FragmentRegistrationProxy`
      - additional-context fragment types
    - Moved `SkillInstructions` into `codex-core-skills`, since
    skill-specific rendering belongs with skills rather than generic core
    context machinery.
    - Kept `codex-core` re-exporting the fragment types it still uses
    internally, so existing call sites keep the same shape.
    - Updated Cargo and Bazel workspace metadata for the new crate.
    
    ## Verification
    
    - `cargo metadata --locked --format-version 1 --no-deps`
    - `just bazel-lock-update`
    - `just bazel-lock-check`
  • [codex] Validate plugin skill base names (#25782)
    ## Summary
    
    - Validate skill base name length before plugin namespacing.
    - Bound the composed `plugin:skill` qualified name to 128 characters.
    - Keep plugin skill runtime names in the existing `plugin:skill` form.
    - Add regression tests for the max qualified-name boundary and rejection
    path.
    
    ## Root Cause
    
    Plugin skills are represented as `plugin_name:skill_name`, but the
    loader previously applied the 64-character skill name limit after adding
    the plugin namespace. Moving that check to the base name fixes valid
    plugin skills with longer namespaces, and the separate 128-character
    qualified-name limit keeps model-visible skill names bounded.
    
    ## Validation
    
    - `just fmt`
    - `just test -p codex-core-skills plugin_skill_name_length_limit`
    - `git diff --check`
  • Add cloud-managed config layer support (#24620)
    ## Summary
    
    PR 3 of 5 in the cloud-managed config client stack.
    
    Adds enterprise-managed cloud config as a first-class config layer
    source. The layer metadata is preserved through config loading,
    diagnostics, debug output, hook attribution, and app-server protocol
    surfaces.
    
    ## Details
    
    - Enterprise-managed config becomes a normal config layer source with
    backend-supplied `id` and display `name` attached for provenance.
    - These layers are designed to behave like non-file managed config: they
    can surface syntax/type diagnostics by layer name even though there is
    no physical config file.
    - Relative path settings are resolved from a stored config base so
    cloud-delivered config remains consistent with existing MDM-delivered
    config semantics.
    - Hook attribution distinguishes config-delivered hooks from
    requirements-delivered hooks via `HookSource::CloudManagedConfig`.
    - This remains pull-based and snapshot-oriented; the PR adds layer
    identity/diagnostics, not dynamic reload behavior.
    
    ## Validation
    
    Validated through the targeted stack checks after rebasing onto current
    `main`:
    
    - Rust crate tests for
    config/hooks/cloud-config/backend-client/app-server-protocol
    - Filtered `codex-core` and `codex-app-server` `cloud_config_bundle`
    tests
    - Python generated-file contract test
    - `cargo shear --deny-warnings`
    - Targeted `argument-comment-lint` for config/hooks
  • Add runtime extra skill roots API (#24977)
    ## Summary
    - Add v2 `skills/extraRoots/set` to replace app-server process-local
    standalone skill roots. The setting is not persisted, accepts missing
    roots, and `extraRoots: []` clears the runtime set.
    - Wire runtime roots into core skill discovery for `skills/list` and
    turn loads, clear skill caches on set, and register the roots with the
    skills watcher so later filesystem changes emit `skills/changed`.
    - Update app-server docs, generated JSON/TypeScript schemas, and
    coverage for serialization, missing roots, empty clears, and restart
    behavior.
    
    ## Testing
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-core-skills`
    - `cargo test -p codex-app-server
    skills_extra_roots_set_updates_process_runtime_roots`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-core-skills`
    - `just fix -p codex-app-server`
  • [codex] Add user input client ids (#24653)
    ## Summary
    
    Adds an optional `clientId` field to app-server v2 `UserInput` and
    carries it through the core `UserInput` model so clients can correlate
    echoed user input items without relying on payload equality.
    
    ## Details
    
    - Adds `client_id: Option<String>` to core `UserInput` variants.
    - Exposes the v2 app-server field as `clientId` on the wire and in
    generated TypeScript.
    - Preserves the id when converting between app-server v2 and core
    protocol types.
    - Regenerates app-server schema fixtures.
    
    ## Validation
    
    - `just fmt`
    - `just write-app-server-schema`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-protocol`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-protocol`
    - `git diff --check`
  • fix: Allow plugin skills to share plugin-level icon assets (#23776)
    Thread the plugin root through plugin skill loading so skill interface
    icons can reference shared plugin assets, such as ../../assets/logo.svg.
  • cleanup: Remove skill env var dependency prompting (#22721)
    Deletes the skill env var dependency prompt feature and its runtime
    path. env_var entries in skill dependency metadata are now silently
    ignored during skill loading.
  • feat: add layered --profile-v2 config files (#17141)
    ## Why
    
    `--profile-v2 <name>` gives launchers and runtime entry points a named
    profile config without making each profile duplicate the base user
    config. The base `$CODEX_HOME/config.toml` still loads first, then
    `$CODEX_HOME/<name>.config.toml` layers above it and becomes the active
    writable user config for that session.
    
    That keeps shared defaults, plugin/MCP setup, and managed/user
    constraints in one place while letting a named profile override only the
    pieces that need to differ.
    
    ## What Changed
    
    - Added the shared `--profile-v2 <name>` runtime option with validated
    plain names, now represented by `ProfileV2Name`.
    - Extended config layer state so the base user config and selected
    profile config are both `User` layers; APIs expose the active user layer
    and merged effective user config.
    - Threaded profile selection through runtime entry points: `codex`,
    `codex exec`, `codex review`, `codex resume`, `codex fork`, and `codex
    debug prompt-input`.
    - Made user-facing config writes go to the selected profile file when
    active, including TUI/settings persistence, app-server config writes,
    and MCP/app tool approval persistence.
    - Made plugin, marketplace, MCP, hooks, and config reload paths read
    from the merged user config so base and profile layers both participate.
    - Updated app-server config layer schemas to mark profile-backed user
    layers.
    
    ## Limits
    
    `--profile-v2` is still rejected for config-management subcommands such
    as feature, MCP, and marketplace edits. Those paths remain tied to the
    base `config.toml` until they have explicit profile-selection semantics.
    
    Some adjacent background writes may still update base or global state
    rather than the selected profile:
    
    - marketplace auto-upgrade metadata
    - automatic MCP dependency installs from skills
    - remote plugin sync or uninstall config edits
    - personality migration marker/default writes
    
    ## Verification
    
    Added targeted coverage for profile name validation, layer
    ordering/merging, selected-profile writes, app-server config writes,
    session hot reload, plugin config merging, hooks/config fixture updates,
    and MCP/app approval persistence.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Remove skills list extra roots (#21485)
    ## Summary
    - Remove `perCwdExtraUserRoots` / `SkillsListExtraRootsForCwd` from the
    `skills/list` app-server API.
    - Drop Rust app-server and `codex-core-skills` extra-root plumbing so
    skill scans are keyed by the normal cwd/user/plugin roots only.
    - Regenerate app-server schemas and update docs/tests that only existed
    for the removed extra-roots behavior.
    
    ## Validation
    - `just write-app-server-schema`
    - `just fmt`
    - `cargo test -p codex-app-server-protocol`
    - `cargo test -p codex-core-skills`
    - `just fix -p codex-app-server-protocol`
    - `just fix -p codex-core-skills`
    - `just fix -p codex-app-server`
    - `just fix -p codex-tui`
    
    ## Notes
    - `cargo test -p codex-app-server --test all skills_list` ran the edited
    skills-list cases, but the full filtered run ended on existing
    `skills_changed_notification_is_emitted_after_skill_change` timeout
    after a websocket `401`.
    - `cargo test -p codex-tui --lib` compiled the changed TUI callers, then
    failed two unrelated status permission tests because local
    `/etc/codex/requirements.toml` forbids `DangerFullAccess`.
    - Source-truth check found the OpenAI monorepo still has
    generated/app-server-kit mirror references to the removed field; those
    should be cleaned up when generated app-server types are synced or in a
    companion OpenAI cleanup.
  • Add plugin ID to skill analytics (#20923)
    ## Summary
    - thread plugin skill roots through the skills loader with their plugin
    ID
    - store plugin ID on loaded skill metadata for plugin-provided skills
    - include plugin ID on skill invocation analytics events
    
    ## Test plan
    - cargo check -p codex-core-skills
    - cargo check -p codex-core -p codex-core-plugins -p codex-analytics
    - cargo check -p codex-tui
    - cargo check -p codex-plugin -p codex-core -p codex-core-plugins -p
    codex-analytics
    - cargo check -p codex-app-server
    - cargo test -p codex-analytics
    - HOME=/private/tmp/codex-empty-home cargo test -p codex-core-skills
    - just fix -p codex-core-skills
    - just fix -p codex-analytics
    - just fix -p codex-core-plugins
    - just fix -p codex-core
    - just fmt
    - git diff --check
  • Soften skill description budget warnings (#20112)
    Updates skill description budget messaging to be less alarming
  • feat: Compress skill paths with root aliases (#19098)
    Add skill root tracking so model-visible skill lists can use short path
    aliases when absolute paths would exceed the metadata budget.
  • refactor: route Codex auth through AuthProvider (#18811)
    ## Summary
    
    This PR moves Codex backend request authentication from direct
    bearer-token handling to `AuthProvider`.
    
    The new `codex-auth-provider` crate defines the shared request-auth
    trait. `CodexAuth::provider()` returns a provider that can apply all
    headers needed for the selected auth mode.
    
    This lets ChatGPT token auth and AgentIdentity auth share the same
    callsite path:
    - ChatGPT token auth applies bearer auth plus account/FedRAMP headers
    where needed.
    - AgentIdentity auth applies AgentAssertion plus account/FedRAMP headers
    where needed.
    
    Reference old stack: https://github.com/openai/codex/pull/17387/changes
    
    ## Callsite Migration
    
    | Area | Change |
    | --- | --- |
    | backend-client | accepts an `AuthProvider` instead of a raw
    token/header |
    | chatgpt client/connectors | applies auth through
    `CodexAuth::provider()` |
    | cloud tasks | keeps Codex-backend gating, applies auth through
    provider |
    | cloud requirements | uses Codex-backend auth checks and provider
    headers |
    | app-server remote control | applies provider headers for backend calls
    |
    | MCP Apps/connectors | gates on `uses_codex_backend()` and keys caches
    from generic account getters |
    | model refresh | treats AgentIdentity as Codex-backend auth |
    | OpenAI file upload path | rejects non-Codex-backend auth before
    applying headers |
    | core client setup | keeps model-provider auth flow and allows
    AgentIdentity through provider-backed OpenAI auth |
    
    ## Stack
    
    1. https://github.com/openai/codex/pull/18757: full revert
    2. https://github.com/openai/codex/pull/18871: isolated Agent Identity
    crate
    3. https://github.com/openai/codex/pull/18785: explicit AgentIdentity
    auth mode and startup task allocation
    4. This PR: migrate Codex backend auth callsites through AuthProvider
    5. https://github.com/openai/codex/pull/18904: accept AgentIdentity JWTs
    and load `CODEX_AGENT_IDENTITY`
    
    ## Testing
    
    Tests: targeted Rust checks, cargo-shear, Bazel lock check, and CI.
  • feat: Fairly trim skill descriptions within context budget (#18925)
    Preserve skill name/path entries whenever possible and trim descriptions
    first, using round-robin character allocation so short descriptions do
    not waste budget.
  • fix: fully revert agent identity runtime wiring (#18757)
    ## Summary
    
    This PR fully reverts the previously merged Agent Identity runtime
    integration from the old stack:
    https://github.com/openai/codex/pull/17387/changes
    
    It removes the Codex-side task lifecycle wiring, rollout/session
    persistence, feature flag plumbing, lazy `auth.json` mutation,
    background task auth paths, and request callsite changes introduced by
    that stack.
    
    This leaves the repo in a clean pre-AgentIdentity integration state so
    the follow-up PRs can reintroduce the pieces in smaller reviewable
    layers.
    
    ## Stack
    
    1. This PR: full revert
    2. https://github.com/openai/codex/pull/18871: move Agent Identity
    business logic into a crate
    3. https://github.com/openai/codex/pull/18785: add explicit
    AgentIdentity auth mode and startup task allocation
    4. https://github.com/openai/codex/pull/18811: migrate auth callsites
    through AuthProvider
    
    ## Testing
    
    Tests: targeted Rust checks, cargo-shear, Bazel lock check, and CI.
  • Split DeveloperInstructions into individual fragments. (#18813)
    Split DeveloperInstructions into individual fragments.
  • Organize context fragments (#18794)
    Organize context fragments under `core/context`. Implement same trait on
    all of them.
  • [codex] Use background agent task auth for backend calls (#18094)
    ## Summary
    
    Introduces a single background/control-plane agent task for ChatGPT
    backend requests that do not have a thread-scoped task, with
    `AuthManager` owning the default ChatGPT backend authorization decision.
    
    Callers now ask `AuthManager` for the default ChatGPT backend
    authorization header. `AuthManager` decides whether that is bearer or
    background AgentAssertion based on config/internal state, while
    low-level bootstrap paths can explicitly request bearer-only auth.
    
    This PR is stacked on PR4 and focuses on the shared background task auth
    plumbing plus the first tranche of backend/control-plane consumers. The
    remaining callsite wiring is split into PR4.2 to keep review size down.
    
    ## Stack
    
    - PR1: https://github.com/openai/codex/pull/17385 - add
    `features.use_agent_identity`
    - PR2: https://github.com/openai/codex/pull/17386 - register agent
    identities when enabled
    - PR3: https://github.com/openai/codex/pull/17387 - register agent tasks
    when enabled
    - PR3.1: https://github.com/openai/codex/pull/17978 - persist and
    prewarm registered tasks per thread
    - PR4: https://github.com/openai/codex/pull/17980 - use task-scoped
    `AgentAssertion` for downstream calls
    - PR4.1: this PR - introduce AuthManager-owned background/control-plane
    `AgentAssertion` auth
    - PR4.2: https://github.com/openai/codex/pull/18260 - use background
    task auth for additional backend/control-plane calls
    
    ## What Changed
    
    - add background task registration and assertion minting inside
    `codex-login`
    - persist `agent_identity.background_task_id` separately from
    per-session task state
    - make `BackgroundAgentTaskManager` private to `codex-login`; call sites
    do not instantiate or pass it around
    - teach `AuthManager` the ChatGPT backend base URL and feature-derived
    background auth mode from resolved config
    - expose bearer-only helpers for bootstrap/registration/refresh-style
    paths that must not use AgentAssertion
    - wire `AuthManager` default ChatGPT authorization through app listing,
    connector directory listing, remote plugins, MCP status/listing,
    analytics, and core-skills remote calls
    - preserve bearer fallback when the feature is disabled, the backend
    host is unsupported, or background task registration is not available
    
    ## Validation
    
    - `just fmt`
    - `cargo check -p codex-core -p codex-login -p codex-analytics -p
    codex-app-server -p codex-cloud-requirements -p codex-cloud-tasks -p
    codex-models-manager -p codex-chatgpt -p codex-model-provider -p
    codex-mcp -p codex-core-skills`
    - `cargo test -p codex-login agent_identity`
    - `cargo test -p codex-model-provider bearer_auth_provider`
    - `cargo test -p codex-core agent_assertion`
    - `cargo test -p codex-app-server remote_control`
    - `cargo test -p codex-cloud-requirements fetch_cloud_requirements`
    - `cargo test -p codex-models-manager manager::tests`
    - `cargo test -p codex-chatgpt`
    - `cargo test -p codex-cloud-tasks`
    - `just fix -p codex-core -p codex-login -p codex-analytics -p
    codex-app-server -p codex-cloud-requirements -p codex-cloud-tasks -p
    codex-models-manager -p codex-chatgpt -p codex-model-provider -p
    codex-mcp -p codex-core-skills`
    - `just fix -p codex-app-server`
    - `git diff --check`
  • feat: Budget skill metadata and surface trimming as a warning (#18298)
    Cap the model-visible skills section to a small share of the context
    window, with a fallback character budget, and keep only as many implicit
    skills as fit within that budget.
    
    Emit a non-fatal warning when enabled skills are omitted, and add a new
    app-server warning notification
    
    Record thread-start skill metrics for total enabled skills, kept skills,
    and whether truncation happened
    
    ---------
    
    Co-authored-by: Matthew Zeng <mzeng@openai.com>
    Co-authored-by: Codex <noreply@openai.com>
  • Make skill loading filesystem-aware (#17720)
    Migrates skill loading to support reading repo skills from the remote
    environment.
  • Spread AbsolutePathBuf (#17792)
    Mechanical change to promote absolute paths through code.
  • chore: clean up argument-comment lint and roll out all-target CI on macOS (#16054)
    ## Why
    
    `argument-comment-lint` was green in CI even though the repo still had
    many uncommented literal arguments. The main gap was target coverage:
    the repo wrapper did not force Cargo to inspect test-only call sites, so
    examples like the `latest_session_lookup_params(true, ...)` tests in
    `codex-rs/tui_app_server/src/lib.rs` never entered the blocking CI path.
    
    This change cleans up the existing backlog, makes the default repo lint
    path cover all Cargo targets, and starts rolling that stricter CI
    enforcement out on the platform where it is currently validated.
    
    ## What changed
    
    - mechanically fixed existing `argument-comment-lint` violations across
    the `codex-rs` workspace, including tests, examples, and benches
    - updated `tools/argument-comment-lint/run-prebuilt-linter.sh` and
    `tools/argument-comment-lint/run.sh` so non-`--fix` runs default to
    `--all-targets` unless the caller explicitly narrows the target set
    - fixed both wrappers so forwarded cargo arguments after `--` are
    preserved with a single separator
    - documented the new default behavior in
    `tools/argument-comment-lint/README.md`
    - updated `rust-ci` so the macOS lint lane keeps the plain wrapper
    invocation and therefore enforces `--all-targets`, while Linux and
    Windows temporarily pass `-- --lib --bins`
    
    That temporary CI split keeps the stricter all-targets check where it is
    already cleaned up, while leaving room to finish the remaining Linux-
    and Windows-specific target-gated cleanup before enabling
    `--all-targets` on those runners. The Linux and Windows failures on the
    intermediate revision were caused by the wrapper forwarding bug, not by
    additional lint findings in those lanes.
    
    ## Validation
    
    - `bash -n tools/argument-comment-lint/run.sh`
    - `bash -n tools/argument-comment-lint/run-prebuilt-linter.sh`
    - shell-level wrapper forwarding check for `-- --lib --bins`
    - shell-level wrapper forwarding check for `-- --tests`
    - `just argument-comment-lint`
    - `cargo test` in `tools/argument-comment-lint`
    - `cargo test -p codex-terminal-detection`
    
    ## Follow-up
    
    - Clean up remaining Linux-only target-gated callsites, then switch the
    Linux lint lane back to the plain wrapper invocation.
    - Clean up remaining Windows-only target-gated callsites, then switch
    the Windows lint lane back to the plain wrapper invocation.
  • skills: remove unused skill permission metadata (#15900)
    ## Why
    
    Skill metadata accepted a `permissions` block and stored the result on
    `SkillMetadata`, but that data was never consumed by runtime behavior.
    Leaving the dead parsing path in place makes it look like skills can
    widen or otherwise influence execution permissions when, in practice,
    declared skill permissions are ignored.
    
    This change removes that misleading surface area so the skill metadata
    model matches what the system actually uses.
    
    ## What changed
    
    - removed `permission_profile` and `managed_network_override` from
    `core-skills::SkillMetadata`
    - stopped parsing `permissions` from skill metadata in
    `core-skills/src/loader.rs`
    - deleted the loader tests that only exercised the removed permissions
    parsing path
    - cleaned up dependent `SkillMetadata` constructors in tests and TUI
    code that were only carrying `None` for those fields
    
    ## Testing
    
    - `cargo test -p codex-core-skills`
    - `cargo test -p codex-tui
    submission_prefers_selected_duplicate_skill_path`
    - `just argument-comment-lint`
  • Extract codex-core-skills crate (#15749)
    ## Summary
    - move skill loading and management into codex-core-skills
    - leave codex-core with the thin integration layer and shared wiring
    
    ## Testing
    - CI
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>