Commit Graph

212 Commits

  • Remove remaining custom prompt support (#16115)
    ## Summary
    - remove protocol and core support for discovering and listing custom
    prompts
    - simplify the TUI slash-command flow and command popup to built-in
    commands only
    - delete obsolete custom prompt tests, helpers, and docs references
    - clean up downstream event handling for the removed protocol events
  • codex-tools: extract shared tool schema parsing (#15923)
    ## Why
    
    `parse_tool_input_schema` and the supporting `JsonSchema` model were
    living in `core/src/tools/spec.rs`, but they already serve callers
    outside `codex-core`.
    
    Keeping that shared schema parsing logic inside `codex-core` makes the
    crate boundary harder to reason about and works against the guidance in
    `AGENTS.md` to avoid growing `codex-core` when reusable code can live
    elsewhere.
    
    This change takes the first extraction step by moving the schema parsing
    primitive into its own crate while keeping the rest of the tool-spec
    assembly in `codex-core`.
    
    ## What changed
    
    - added a new `codex-tools` crate under `codex-rs/tools`
    - moved the shared tool input schema model and sanitizer/parser into
    `tools/src/json_schema.rs`
    - kept `tools/src/lib.rs` exports-only, with the module-level unit tests
    split into `json_schema_tests.rs`
    - updated `codex-core` to use `codex-tools::JsonSchema` and re-export
    `parse_tool_input_schema`
    - updated `codex-app-server` dynamic tool validation to depend on
    `codex-tools` directly instead of reaching through `codex-core`
    - wired the new crate into the Cargo workspace and Bazel build graph
  • [codex] import token_data from codex-login directly (#15903)
    ## Why
    `token_data` is owned by `codex-login`, but `codex-core` was still
    re-exporting it. That let callers pull auth token types through
    `codex-core`, which keeps otherwise unrelated crates coupled to
    `codex-core` and makes `codex-core` more of a build-graph bottleneck.
    
    ## What changed
    - remove the `codex-core` re-export of `codex_login::token_data`
    - update the remaining `codex-core` internals that used
    `crate::token_data` to import `codex_login::token_data` directly
    - update downstream callers in `codex-rs/chatgpt`,
    `codex-rs/tui_app_server`, `codex-rs/app-server/tests/common`, and
    `codex-rs/core/tests` to import `codex_login::token_data` directly
    - add explicit `codex-login` workspace dependencies and refresh lock
    metadata for crates that now depend on it directly
    
    ## Validation
    - `cargo test -p codex-chatgpt --locked`
    - `just argument-comment-lint`
    - `just bazel-lock-update`
    - `just bazel-lock-check`
    
    ## Notes
    - attempted `cargo test -p codex-core --locked` and `cargo test -p
    codex-core auth_refresh --locked`, but both ran out of disk while
    linking `codex-core` test binaries in the local environment
  • Extract codex-core-skills crate (#15749)
    ## Summary
    - move skill loading and management into codex-core-skills
    - leave codex-core with the thin integration layer and shared wiring
    
    ## Testing
    - CI
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Extract codex-analytics crate (#15748)
    ## Summary
    - move the analytics events client into codex-analytics
    - update codex-core and app-server callsites to use the new crate
    
    ## Testing
    - CI
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Move string truncation helpers into codex-utils-string (#15572)
    - move the shared byte-based middle truncation logic from `core` into
    `codex-utils-string`
    - keep token-specific truncation in `codex-core` so rollout can reuse
    the shared helper in the next stacked PR
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Move git utilities into a dedicated crate (#15564)
    - create `codex-git-utils` and move the shared git helpers into it with
    file moves preserved for diff readability
    - move the `GitInfo` helpers out of `core` so stacked rollout work can
    depend on the shared crate without carrying its own git info module
    
    ---------
    
    Co-authored-by: Ahmed Ibrahim <219906144+aibrahim-oai@users.noreply.github.com>
    Co-authored-by: Codex <noreply@openai.com>
  • core: Make FileWatcher reusable (#15093)
    ### Summary
    Make `FileWatcher` a reusable core component which can be built upon.
    Extract skills-related logic into a separate `SkillWatcher`.
    Introduce a composable `ThrottledWatchReceiver` to throttle filesystem
    events, coalescing affected paths among them.
    
    ### Testing
    Updated existing unit tests.
  • Move macOS sandbox builders into codex-sandboxing (#15593)
    ## Summary
    - move macOS permission merging/intersection logic and tests from
    `codex-core` into `codex-sandboxing`
    - move seatbelt policy builders, permissions logic, SBPL assets, and
    their tests into `codex-sandboxing`
    - keep `codex-core` owning only the seatbelt spawn wrapper and switch
    call sites to import the moved APIs directly
    
    ## Notes
    - no re-exports added
    - moved the seatbelt tests with the implementation so internal helpers
    could stay private
    - local verification is still finishing while this PR is open
  • Add fork snapshot modes (#15239)
    ## Summary
    - add `ForkSnapshotMode` to `ThreadManager::fork_thread` so callers can
    request either a committed snapshot or an interrupted snapshot
    - share the model-visible `<turn_aborted>` history marker between the
    live interrupt path and interrupted forks
    - update the small set of direct fork callsites to pass
    `ForkSnapshotMode::Committed`
    
    Note: this enables /btw to work similarly as Esc to interrupt (hopefully
    somewhat in distribution)
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Split features into codex-features crate (#15253)
    - Split the feature system into a new `codex-features` crate.
    - Cut `codex-core` and workspace consumers over to the new config and
    warning APIs.
    
    Co-authored-by: Ahmed Ibrahim <219906144+aibrahim-oai@users.noreply.github.com>
    Co-authored-by: Codex <noreply@openai.com>
  • Move auth code into login crate (#15150)
    - Move the auth implementation and token data into codex-login.
    - Keep codex-core re-exporting that surface from codex-login for
    existing callers.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Move terminal module to terminal-detection crate (#15216)
    - Move core/src/terminal.rs and its tests into a standalone
    terminal-detection workspace crate.
    - Update direct consumers to depend on codex-terminal-detection and
    import terminal APIs directly.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • feat(tui): restore composer history in app-server tui (#14945)
    ## Problem
    
    The app-server TUI (`tui_app_server`) lacked composer history support.
    Pressing Up/Down to recall previous prompts hit a stub that logged a
    warning and displayed "Not available in app-server TUI yet." New
    submissions were silently dropped from the shared history file, so
    nothing persisted for future sessions.
    
    ## Mental model
    
    Codex maintains a single, append-only history file
    (`$CODEX_HOME/history.jsonl`) shared across all TUI processes on the
    same machine. The legacy (in-process) TUI already reads/writes this file
    through `codex_core::message_history`. The app-server TUI delegates most
    operations to a separate process over RPC, but history is intentionally
    *not* an RPC concern — it's a client-local file.
    
    This PR makes the app-server TUI access the same history file directly,
    bypassing the app-server process entirely. The composer's Up/Down
    navigation and submit-time persistence now follow the same code paths as
    the legacy TUI, with the only difference being *where* the call is
    dispatched (locally in `App`, rather than inside `CodexThread`).
    
    The branch is rebuilt directly on top of `upstream/main`, so it keeps
    the
    existing app-server restore architecture intact.
    `AppServerStartedThread`
    still restores transcript history from the server `Thread` snapshot via
    `thread_snapshot_events`; this PR only adds composer-history support.
    
    ## Non-goals
    
    - Adding history support to the app-server protocol. History remains
    client-local.
    - Changing the on-disk format or location of `history.jsonl`.
    - Surfacing history I/O errors to the user (failures are logged and
    silently swallowed, matching the legacy TUI).
    
    ## Tradeoffs
    
    | Decision | Why | Risk |
    |----------|-----|------|
    | Widen `message_history` from `pub(crate)` to `pub` | Avoids
    duplicating file I/O logic; the module already has a clean, minimal API
    surface. | Other workspace crates can now call these functions — the
    contract is no longer crate-private. However, this is consistent with
    recent precedent: `590cfa617` exposed `mention_syntax` for TUI
    consumption, `752402c4f` exposed plugin APIs (`PluginsManager`), and
    `14fcb6645`/`edacbf7b6` widened internal core APIs for other crates.
    These were all narrow, intentional exposures of specific APIs — not
    broad "make internals public" moves. `1af2a37ad` even went the other
    direction, reducing broad re-exports to tighten boundaries. This change
    follows the same pattern: a small, deliberate API surface (3 functions)
    rather than a wholesale visibility change. |
    | Intercept `AddToHistory` / `GetHistoryEntryRequest` in `App` before
    RPC fallback | Keeps history ops out of the "unsupported op" error path
    without changing app-server protocol. | This now routes through a single
    `submit_thread_op` entry point, which is safer than the original
    duplicated dispatch. The remaining risk is organizational: future
    thread-op submission paths need to keep using that shared entry point. |
    | `session_configured_from_thread_response` is now `async` | Needs
    `await` on `history_metadata()` to populate real `history_log_id` /
    `history_entry_count`. | Adds an async file-stat + full-file newline
    scan to the session bootstrap path. The scan is bounded by
    `history.max_bytes` and matches the legacy TUI's cost profile, but
    startup latency still scales with file size. |
    
    ## Architecture
    
    ```
    User presses Up                     User submits a prompt
           │                                    │
           ▼                                    ▼
    ChatComposerHistory                 ChatWidget::do_submit_turn
      navigate_up()                       encode_history_mentions()
           │                                    │
           ▼                                    ▼
      AppEvent::CodexOp                  Op::AddToHistory { text }
      (GetHistoryEntryRequest)                  │
           │                                    ▼
           ▼                            App::try_handle_local_history_op
      App::try_handle_local_history_op    message_history::append_entry()
        spawn_blocking {                        │
          message_history::lookup()             ▼
        }                                $CODEX_HOME/history.jsonl
           │
           ▼
      AppEvent::ThreadEvent
      (GetHistoryEntryResponse)
           │
           ▼
      ChatComposerHistory::on_entry_response()
    ```
    
    ## Observability
    
    - `tracing::warn` on `append_entry` failure (includes thread ID).
    - `tracing::warn` on `spawn_blocking` lookup join error.
    - `tracing::warn` from `message_history` internals on file-open, lock,
    or parse failures.
    
    ## Tests
    
    - `chat_composer_history::tests::navigation_with_async_fetch` — verifies
    that Up emits `Op::GetHistoryEntryRequest` (was: checked for stub error
    cell).
    - `app::tests::history_lookup_response_is_routed_to_requesting_thread` —
    verifies multi-thread composer recall routes the lookup result back to
    the originating thread.
    -
    `app_server_session::tests::resume_response_relies_on_snapshot_replay_not_initial_messages`
    — verifies app-server session restore still uses the upstream
    thread-snapshot path.
    -
    `app_server_session::tests::session_configured_populates_history_metadata`
    — verifies bootstrap sets nonzero `history_log_id` /
    `history_entry_count` from the shared local history file.
  • [hooks] userpromptsubmit - hook before user's prompt is executed (#14626)
    - this allows blocking the user's prompts from executing, and also
    prevents them from entering history
    - handles the edge case where you can both prevent the user's prompt AND
    add n amount of additionalContexts
    - refactors some old code into common.rs where hooks overlap
    functionality
    - refactors additionalContext being previously added to user messages,
    instead we use developer messages for them
    - handles queued messages correctly
    
    Sample hook for testing - if you write "[block-user-submit]" this hook
    will stop the thread:
    
    example run
    ```
    › sup
    
    
    • Running UserPromptSubmit hook: reading the observatory notes
    
    UserPromptSubmit hook (completed)
      warning: wizard-tower UserPromptSubmit demo inspected: sup
      hook context: Wizard Tower UserPromptSubmit demo fired. For this reply only, include the exact
    phrase 'observatory lanterns lit' exactly once near the end.
    
    • Just riding the cosmic wave and ready to help, my friend. What are we building today? observatory
      lanterns lit
    
    
    › and [block-user-submit]
    
    
    • Running UserPromptSubmit hook: reading the observatory notes
    
    UserPromptSubmit hook (stopped)
      warning: wizard-tower UserPromptSubmit demo blocked the prompt on purpose.
      stop: Wizard Tower demo block: remove [block-user-submit] to continue.
    ```
    
    .codex/config.toml
    ```
    [features]
    codex_hooks = true
    ```
    
    .codex/hooks.json
    ```
    {
      "hooks": {
        "UserPromptSubmit": [
          {
            "hooks": [
              {
                "type": "command",
                "command": "/usr/bin/python3 .codex/hooks/user_prompt_submit_demo.py",
                "timeoutSec": 10,
                "statusMessage": "reading the observatory notes"
              }
            ]
          }
        ]
      }
    }
    ```
    
    .codex/hooks/user_prompt_submit_demo.py
    ```
    #!/usr/bin/env python3
    
    import json
    import sys
    from pathlib import Path
    
    
    def prompt_from_payload(payload: dict) -> str:
        prompt = payload.get("prompt")
        if isinstance(prompt, str) and prompt.strip():
            return prompt.strip()
    
        event = payload.get("event")
        if isinstance(event, dict):
            user_prompt = event.get("user_prompt")
            if isinstance(user_prompt, str):
                return user_prompt.strip()
    
        return ""
    
    
    def main() -> int:
        payload = json.load(sys.stdin)
        prompt = prompt_from_payload(payload)
        cwd = Path(payload.get("cwd", ".")).name or "wizard-tower"
    
        if "[block-user-submit]" in prompt:
            print(
                json.dumps(
                    {
                        "systemMessage": (
                            f"{cwd} UserPromptSubmit demo blocked the prompt on purpose."
                        ),
                        "decision": "block",
                        "reason": (
                            "Wizard Tower demo block: remove [block-user-submit] to continue."
                        ),
                    }
                )
            )
            return 0
    
        prompt_preview = prompt or "(empty prompt)"
        if len(prompt_preview) > 80:
            prompt_preview = f"{prompt_preview[:77]}..."
    
        print(
            json.dumps(
                {
                    "systemMessage": (
                        f"{cwd} UserPromptSubmit demo inspected: {prompt_preview}"
                    ),
                    "hookSpecificOutput": {
                        "hookEventName": "UserPromptSubmit",
                        "additionalContext": (
                            "Wizard Tower UserPromptSubmit demo fired. "
                            "For this reply only, include the exact phrase "
                            "'observatory lanterns lit' exactly once near the end."
                        ),
                    },
                }
            )
        )
        return 0
    
    
    if __name__ == "__main__":
        raise SystemExit(main())
    ```
  • Prefer websockets when providers support them (#13592)
    Remove all flags and model settings.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • Add auth env observability (#14905)
    CXC-410 Emit Env Var Status with `/feedback` report
    
    Add more observability on top of #14611 
    
    [Unset](https://openai.sentry.io/issues/7340419168/?project=4510195390611458&query=019cfa8d-c1ba-7002-96fa-e35fc340551d&referrer=issue-stream)
    
    [Set](https://openai.sentry.io/issues/7340426331/?project=4510195390611458&query=019cfa91-aba1-7823-ab7e-762edfbc0ed4&referrer=issue-stream)
    <img width="1063" height="610" alt="image"
    src="https://github.com/user-attachments/assets/937ab026-1c2d-4757-81d5-5f31b853113e"
    />
    
    
    ###### Summary
    - Adds auth-env telemetry that records whether key auth-related env
    overrides were present on session start and request paths.
    - Threads those auth-env fields through `/responses`, websocket, and
    `/models` telemetry and feedback metadata.
    - Buckets custom provider `env_key` configuration to a safe
    `"configured"` value instead of emitting raw config text.
    - Keeps the slice observability-only: no raw token values or raw URLs
    are emitted.
    
    ###### Rationale (from spec findings)
    - 401 and auth-path debugging needs a way to distinguish env-driven auth
    paths from sessions with no auth env override.
    - Startup and model-refresh failures need the same auth-env diagnostics
    as normal request failures.
    - Feedback and Sentry tags need the same auth-env signal as OTel events
    so reports can be triaged consistently.
    - Custom provider config is user-controlled text, so the telemetry
    contract must stay presence-only / bucketed.
    
    ###### Scope
    - Adds a small `AuthEnvTelemetry` bundle for env presence collection and
    threads it through the main request/session telemetry paths.
    - Does not add endpoint/base-url/provider-header/geo routing attribution
    or broader telemetry API redesign.
    
    ###### Trade-offs
    - `provider_env_key_name` is bucketed to `"configured"` instead of
    preserving the literal configured env var name.
    - `/models` is included because startup/model-refresh auth failures need
    the same diagnostics, but broader parity work remains out of scope.
    - This slice keeps the existing telemetry APIs and layers auth-env
    fields onto them rather than redesigning the metadata model.
    
    ###### Client follow-up
    - Add the separate endpoint/base-url attribution slice if routing-source
    diagnosis is still needed.
    - Add provider-header or residency attribution only if auth-env presence
    proves insufficient in real reports.
    - Revisit whether any additional auth-related env inputs need safe
    bucketing after more 401 triage data.
    
    ###### Testing
    - `cargo test -p codex-core emit_feedback_request_tags -- --nocapture`
    - `cargo test -p codex-core
    collect_auth_env_telemetry_buckets_provider_env_key_name -- --nocapture`
    - `cargo test -p codex-core
    models_request_telemetry_emits_auth_env_feedback_tags_on_failure --
    --nocapture`
    - `cargo test -p codex-otel
    otel_export_routing_policy_routes_api_request_auth_observability --
    --nocapture`
    - `cargo test -p codex-otel
    otel_export_routing_policy_routes_websocket_connect_auth_observability
    -- --nocapture`
    - `cargo test -p codex-otel
    otel_export_routing_policy_routes_websocket_request_transport_observability
    -- --nocapture`
    - `cargo test -p codex-core --no-run --message-format short`
    - `cargo test -p codex-otel --no-run --message-format short`
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • fix(core): prevent hanging turn/start due to websocket warming issues (#14838)
    ## Description
    
    This PR fixes a bad first-turn failure mode in app-server when the
    startup websocket prewarm hangs. Before this change, `initialize ->
    thread/start -> turn/start` could sit behind the prewarm for up to five
    minutes, so the client would not see `turn/started`, and even
    `turn/interrupt` would block because the turn had not actually started
    yet.
    
    Now, we:
    - set a (configurable) timeout of 15s for websocket startup time,
    exposed as `websocket_startup_timeout_ms` in config.toml
    - `turn/started` is sent immediately on `turn/start` even if the
    websocket is still connecting
    - `turn/interrupt` can be used to cancel a turn that is still waiting on
    the websocket warmup
    - the turn task will wait for the full 15s websocket warming timeout
    before falling back
    
    ## Why
    
    The old behavior made app-server feel stuck at exactly the moment the
    client expects turn lifecycle events to start flowing. That was
    especially painful for external clients, because from their point of
    view the server had accepted the request but then went silent for
    minutes.
    
    ## Configuring the websocket startup timeout
    Can set it in config.toml like this:
    ```
    [model_providers.openai]
    supports_websockets = true
    websocket_connect_timeout_ms = 15000
    ```
  • Add auth 401 observability to client bug reports (#14611)
    CXC-392
    
      [With
      401](https://openai.sentry.io/issues/7333870443/?project=4510195390611458&query=019ce8f8-560c-7f10-a00a-c59553740674&referrer=issue-stream)
      <img width="1909" height="555" alt="401 auth tags in Sentry"
      src="https://github.com/user-attachments/assets/412ea950-61c4-4780-9697-15c270971ee3"
      />
    
    
      - auth_401_*: preserved facts from the latest unauthorized response snapshot
      - auth_*: latest auth-related facts from the latest request attempt
      - auth_recovery_*: unauthorized recovery state and follow-up result
    
    
      Without 401
      <img width="1917" height="522" alt="happy-path auth tags in Sentry"
      src="https://github.com/user-attachments/assets/3381ed28-8022-43b0-b6c0-623a630e679f"
      />
    
      ###### Summary
      - Add client-visible 401 diagnostics for auth attachment, upstream auth classification, and 401 request id / cf-ray correlation.
      - Record unauthorized recovery mode, phase, outcome, and retry/follow-up status without changing auth behavior.
      - Surface the highest-signal auth and recovery fields on uploaded client bug reports so they are usable in Sentry.
      - Preserve original unauthorized evidence under `auth_401_*` while keeping follow-up result tags separate.
    
      ###### Rationale (from spec findings)
      - The dominant bucket needed proof of whether the client attached auth before send or upstream still classified the request as missing auth.
      - Client uploads needed to show whether unauthorized recovery ran and what the client tried next.
      - Request id and cf-ray needed to be preserved on the unauthorized response so server-side correlation is immediate.
      - The bug-report path needed the same auth evidence as the request telemetry path, otherwise the observability would not be operationally useful.
    
      ###### Scope
      - Add auth 401 and unauthorized-recovery observability in `codex-rs/core`, `codex-rs/codex-api`, and `codex-rs/otel`, including feedback-tag surfacing.
      - Keep auth semantics, refresh behavior, retry behavior, endpoint classification, and geo-denial follow-up work out of this PR.
    
      ###### Trade-offs
      - This exports only safe auth evidence: header presence/name, upstream auth classification, request ids, and recovery state. It does not export token values or raw upstream bodies.
      - This keeps websocket connection reuse as a transport clue because it can help distinguish stale reused sessions from fresh reconnects.
      - Misroute/base-url classification and geo-denial are intentionally deferred to a separate follow-up PR so this review stays focused on the dominant auth 401 bucket.
    
      ###### Client follow-up
      - PR 2 will add misroute/provider and geo-denial observability plus the matching feedback-tag surfacing.
      - A separate host/app-server PR should log auth-decision inputs so pre-send host auth state can be correlated with client request evidence.
      - `device_id` remains intentionally separate until there is a safe existing source on the feedback upload path.
    
      ###### Testing
      - `cargo test -p codex-core refresh_available_models_sorts_by_priority`
      - `cargo test -p codex-core emit_feedback_request_tags_`
      - `cargo test -p codex-core emit_feedback_auth_recovery_tags_`
      - `cargo test -p codex-core auth_request_telemetry_context_tracks_attached_auth_and_retry_phase`
      - `cargo test -p codex-core extract_response_debug_context_decodes_identity_headers`
      - `cargo test -p codex-core identity_auth_details`
      - `cargo test -p codex-core telemetry_error_messages_preserve_non_http_details`
      - `cargo test -p codex-core --all-features --no-run`
      - `cargo test -p codex-otel otel_export_routing_policy_routes_api_request_auth_observability`
      - `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_connect_auth_observability`
      - `cargo test -p codex-otel otel_export_routing_policy_routes_websocket_request_transport_observability`
  • Add openai_base_url config override for built-in provider (#12031)
    We regularly get bug reports from users who mistakenly have the
    `OPENAI_BASE_URL` environment variable set. This PR deprecates this
    environment variable in favor of a top-level config key
    `openai_base_url` that is used for the same purpose. By making it a
    config key, it will be more visible to users. It will also participate
    in all of the infrastructure we've added for layered and managed
    configs.
    
    Summary
    - introduce the `openai_base_url` top-level config key, update
    schema/tests, and route the built-in openai provider through it while
    - fall back to deprecated `OPENAI_BASE_URL` env var but warn user of
    deprecation when no `openai_base_url` config key is present
    - update CLI, SDK, and TUI code to prefer the new config path (with a
    deprecated env-var fallback) and document the SDK behavior change
  • Add plugin usage telemetry (#14531)
    adding metrics including: 
    * plugin used
    * plugin installed/uninstalled
    * plugin enabled/disabled
  • [elicitation] User-friendly tool call messages. (#14403)
    - [x] Add a curated set of tool call messages and human-readable tool
    param names.
  • Let models opt into original image detail (#14175)
    ## Summary
    
    This PR narrows original image detail handling to a single opt-in
    feature:
    
    - `image_detail_original` lets the model request `detail: "original"` on
    supported models
    - Omitting `detail` preserves the default resized behavior
    
    The model only sees `detail: "original"` guidance when the active model
    supports it:
    
    - JS REPL instructions include the guidance and examples only on
    supported models
    - `view_image` only exposes a `detail` parameter when the feature and
    model can use it
    
    The image detail API is intentionally narrow and consistent across both
    paths:
    
    - `view_image.detail` supports only `"original"`; otherwise omit the
    field
    - `codex.emitImage(..., detail)` supports only `"original"`; otherwise
    omit the field
    - Unsupported explicit values fail clearly at the API boundary instead
    of being silently reinterpreted
    - Unsupported explicit `detail: "original"` requests fall back to normal
    behavior when the feature is disabled or the model does not support
    original detail
  • add(core): arc_monitor (#13936)
    ## Summary
    - add ARC monitor support for MCP tool calls by serializing MCP approval
    requests into the ARC action shape and sending the relevant
    conversation/policy context to the `/api/codex/safety/arc` endpoint
    - route ARC outcomes back into MCP approval flow so `ask-user` falls
    back to a user prompt and `steer-model` blocks the tool call, with
    guardian/ARC tests covering the new request shape
    - update the TUI approval copy from “Approve Once” to “Allow” / “Allow
    for this session” and refresh the related
      snapshots
    
    ---------
    
    Co-authored-by: Fouad Matin <fouad@openai.com>
    Co-authored-by: Fouad Matin <169186268+fouad-openai@users.noreply.github.com>
  • chore: use @plugin instead of $plugin for plaintext mentions (#13921)
    change plaintext plugin-mentions from `$plugin` to `@plugin`, ensure TUI
    can correctly decode these from history.
    
    tested locally, added/updated tests.
  • Add guardian approval MVP (#13692)
    ## Summary
    - add the guardian reviewer flow for `on-request` approvals in command,
    patch, sandbox-retry, and managed-network approval paths
    - keep guardian behind `features.guardian_approval` instead of exposing
    a public `approval_policy = guardian` mode
    - route ordinary `OnRequest` approvals to the guardian subagent when the
    feature is enabled, without changing the public approval-mode surface
    
    ## Public model
    - public approval modes stay unchanged
    - guardian is enabled via `features.guardian_approval`
    - when that feature is on, `approval_policy = on-request` keeps the same
    approval boundaries but sends those approval requests to the guardian
    reviewer instead of the user
    - `/experimental` only persists the feature flag; it does not rewrite
    `approval_policy`
    - CLI and app-server no longer expose a separate `guardian` approval
    mode in this PR
    
    ## Guardian reviewer
    - the reviewer runs as a normal subagent and reuses the existing
    subagent/thread machinery
    - it is locked to a read-only sandbox and `approval_policy = never`
    - it does not inherit user/project exec-policy rules
    - it prefers `gpt-5.4` when the current provider exposes it, otherwise
    falls back to the parent turn's active model
    - it fail-closes on timeout, startup failure, malformed output, or any
    other review error
    - it currently auto-approves only when `risk_score < 80`
    
    ## Review context and policy
    - guardian mirrors `OnRequest` approval semantics rather than
    introducing a separate approval policy
    - explicit `require_escalated` requests follow the same approval surface
    as `OnRequest`; the difference is only who reviews them
    - managed-network allowlist misses that enter the approval flow are also
    reviewed by guardian
    - the review prompt includes bounded recent transcript history plus
    recent tool call/result evidence
    - transcript entries and planned-action strings are truncated with
    explicit `<guardian_truncated ... />` markers so large payloads stay
    bounded
    - apply-patch reviews include the full patch content (without
    duplicating the structured `changes` payload)
    - the guardian request layout is snapshot-tested using the same
    model-visible Responses request formatter used elsewhere in core
    
    ## Guardian network behavior
    - the guardian subagent inherits the parent session's managed-network
    allowlist when one exists, so it can use the same approved network
    surface while reviewing
    - exact session-scoped network approvals are copied into the guardian
    session with protocol/port scope preserved
    - those copied approvals are now seeded before the guardian's first turn
    is submitted, so inherited approvals are available during any immediate
    review-time checks
    
    ## Out of scope / follow-ups
    - the sandbox-permission validation split was pulled into a separate PR
    and is not part of this diff
    - a future follow-up can enable `serde_json` preserve-order in
    `codex-core` and then simplify the guardian action rendering further
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • app-server: Add streaming and tty/pty capabilities to command/exec (#13640)
    * Add an ability to stream stdin, stdout, and stderr
    * Streaming of stdout and stderr has a configurable cap for total amount
    of transmitted bytes (with an ability to disable it)
    * Add support for overriding environment variables
    * Add an ability to terminate running applications (using
    `command/exec/terminate`)
    * Add TTY/PTY support, with an ability to resize the terminal (using
    `command/exec/resize`)
  • feat(otel, core): record turn TTFT and TTFM metrics in codex-core (#13630)
    ### Summary
    This adds turn-level latency metrics for the first model output and the
    first completed agent message.
    - `codex.turn.ttft.duration_ms` starts at turn start and records on the
    first output signal we see from the model. That includes normal
    assistant text, reasoning deltas, and non-text outputs like tool-call
    items.
    - `codex.turn.ttfm.duration_ms` also starts at turn start, but it
    records when the first agent message finishes streaming rather than when
    its first delta arrives.
    
    ### Implementation notes
    The timing is tracked in codex-core, not app-server, so the definition
    stays consistent across CLI, TUI, and app-server clients.
    
    I reused the existing turn lifecycle boundary that already drives
    `codex.turn.e2e_duration_ms`, stored the turn start timestamp in turn
    state, and record each metric once per turn.
    
    I also wired the new metric names into the OTEL runtime metrics summary
    so they show up in the same in-memory/debug snapshot path as the
    existing timing metrics.
  • Refine realtime startup context formatting (#13560)
    ## Summary
    - group recent work by git repo when available, otherwise by directory
    - render recent work as bounded user asks with per-thread cwd context
    - exclude hidden files and directories from workspace trees
  • app-server: Update thread/name/set to support not-loaded threads (#13282)
    Currently `thread/name/set` does only work for loaded threads.
    Expand the scope to also support persisted but not-yet-loaded ones for a
    more predictable API surface.
    This will make it possible to rename threads discovered via
    `thread/list` and similar operations.
  • feat: load from plugins (#12864)
    Support loading plugins.
    
    Plugins can now be enabled via [plugins.<name>] in config.toml. They are
    loaded as first-class entities through PluginsManager, and their default
    skills/ and .mcp.json contributions are integrated into the existing
    skills and MCP flows.
  • core: bundle settings diff updates into one dev/user envelope (#12417)
    ## Summary
    - bundle contextual prompt injection into at most one developer message
    plus one contextual user message in both:
      - per-turn settings updates
      - initial context insertion
    - preserve `<model_switch>` across compaction by rebuilding it through
    canonical initial-context injection, instead of relying on
    strip/reattach hacks
    - centralize contextual user fragment detection in one shared definition
    table and reuse it for parsing/compaction logic
    - keep `AGENTS.md` in its natural serialized format:
      - `# AGENTS.md instructions for {dirname}`
      - `<INSTRUCTIONS>...</INSTRUCTIONS>`
    - simplify related tests/helpers and accept the expected snapshot/layout
    updates from bundled multi-part messages
    
    ## Why
    The goal is to converge toward a simpler, more intentional prompt shape
    where contextual updates are consistently represented as one developer
    envelope plus one contextual user envelope, while keeping parsing and
    compaction behavior aligned with that representation.
    
    ## Notable details
    - the temporary `SettingsUpdateEnvelope` wrapper was removed; these
    paths now return `Vec<ResponseItem>` directly
    - local/remote compaction no longer rely on model-switch strip/restore
    helpers
    - contextual user detection is now driven by shared fragment definitions
    instead of ad hoc matcher assembly
    - AGENTS/user instructions are still the same logical context; only the
    synthetic `<user_instructions>` wrapper was replaced by the natural
    AGENTS text format
    
    ## Testing
    - `just fmt`
    - `cargo test -p codex-app-server
    codex_message_processor::tests::extract_conversation_summary_prefers_plain_user_messages
    -- --exact`
    - `cargo test -p codex-core
    compact::tests::collect_user_messages_filters_session_prefix_entries
    --lib -- --exact`
    - `cargo test -p codex-core --test all
    'suite::compact::snapshot_request_shape_pre_turn_compaction_strips_incoming_model_switch'
    -- --exact`
    - `cargo test -p codex-core --test all
    'suite::compact_remote::snapshot_request_shape_remote_pre_turn_compaction_strips_incoming_model_switch'
    -- --exact`
    - `cargo test -p codex-core --test all
    'suite::client::includes_apps_guidance_as_developer_message_when_enabled'
    -- --exact`
    - `cargo test -p codex-core --test all
    'suite::client::includes_developer_instructions_message_in_request' --
    --exact`
    - `cargo test -p codex-core --test all
    'suite::client::includes_user_instructions_message_in_request' --
    --exact`
    - `cargo test -p codex-core --test all
    'suite::client::resume_includes_initial_messages_and_sends_prior_items'
    -- --exact`
    - `cargo test -p codex-core --test all
    'suite::review::review_input_isolated_from_parent_history' -- --exact`
    - `cargo test -p codex-exec --test all
    'suite::resume::exec_resume_last_respects_cwd_filter_and_all_flag' --
    --exact`
    - `cargo test -p core_test_support
    context_snapshot::tests::full_text_mode_preserves_unredacted_text --
    --exact`
    
    ## Notes
    - I also ran several targeted `compact`, `compact_remote`,
    `prompt_caching`, `model_visible_layout`, and `event_mapping` tests
    while iterating on prompt-shape changes.
    - I have not claimed a clean full-workspace `cargo test` from this
    environment because local sandbox/resource conditions have previously
    produced unrelated failures in large workspace runs.
  • feat: adding stream parser (#12666)
    Add a stream parser to extract citations (and others) from a stream.
    This support cases where markers are split in differen tokens.
    
    Codex never manage to make this code work so everything was done
    manually. Please review correctly and do not touch this part of the code
    without a very clear understanding of it
  • Support external agent config detect and import (#12660)
    Migration Behavior
    
    * Config
      *  Migrates settings.json into config.toml
    * Only adds fields when config.toml is missing, or when those fields are
    missing from the existing file
      *  Supported mappings:
        env -> shell_environment_policy
         sandbox.enabled = true -> sandbox_mode = "workspace-write"
    
    * Skills
      *  Copies home and repo .claude/skills into .agents/skills
      *  Existing skill directories are not overwritten
      *  SKILL.md content is rewritten from Claude-related terms to Codex
    
    * AgentsMd
      *  Repo only
      *  Migrates CLAUDE.md into AGENTS.md
    * Detect/import only proceed when AGENTS.md is missing or present but
    empty
      *  Content is rewritten from Claude-related terms to Codex
  • feat: run zsh fork shell tool via shell-escalation (#12649)
    ## Why
    
    This PR switches the `shell_command` zsh-fork path over to
    `codex-shell-escalation` so the new shell tool can use the shared
    exec-wrapper/escalation protocol instead of the `zsh_exec_bridge`
    implementation that was introduced in
    https://github.com/openai/codex/pull/12052. `zsh_exec_bridge` relied on
    UNIX domain sockets, which is not as tamper-proof as the FD-based
    approach in `codex-shell-escalation`.
    
    ## What Changed
    
    - Added a Unix zsh-fork runtime adapter in `core`
    (`core/src/tools/runtimes/shell/unix_escalation.rs`) that:
    - runs zsh-fork commands through
    `codex_shell_escalation::run_escalate_server`
      - bridges exec-policy / approval decisions into `ShellActionProvider`
    - executes escalated commands via a `ShellCommandExecutor` that calls
    `process_exec_tool_call`
    - Updated `ShellRuntime` / `ShellCommandHandler` / tool spec wiring to
    select a `shell_command` backend (`classic` vs `zsh-fork`) while leaving
    the generic `shell` tool path unchanged.
    - Removed the `zsh_exec_bridge`-based session service and deleted
    `core/src/zsh_exec_bridge/mod.rs`.
    - Moved exec-wrapper entrypoint dispatch to `arg0` by handling the
    `codex-execve-wrapper` arg0 alias there, and removed the old
    `codex_core::maybe_run_zsh_exec_wrapper_mode()` hooks from `cli` and
    `app-server` mains.
    - Added the needed `codex-shell-escalation` dependencies for `core` and
    `arg0`.
    
    ## Tests
    
    - `cargo test -p codex-core
    shell_zsh_fork_prefers_shell_command_over_unified_exec`
    - `cargo test -p codex-app-server turn_start_shell_zsh_fork --
    --nocapture`
    - verifies zsh-fork command execution and approval flows through the new
    backend
    - includes subcommand approve/decline coverage using the shared zsh
    DotSlash fixture in `app-server/tests/suite/zsh`
    - To test manually, I added the following to `~/.codex/config.toml`:
    
    ```toml
    zsh_path = "/Users/mbolin/code/codex3/codex-rs/app-server/tests/suite/zsh"
    
    [features]
    shell_zsh_fork = true
    ```
    
    Then I ran `just c` to run the dev build of Codex with these changes and
    sent it the message:
    
    ```
    run `echo $0`
    ```
    
    And it replied with:
    
    ```
      echo $0 printed:
    
      /Users/mbolin/code/codex3/codex-rs/app-server/tests/suite/zsh
    
      In this tool context, $0 reflects the script path used to invoke the shell, not just zsh.
    ```
    
    so the tool appears to be wired up correctly.
    
    ## Notes
    
    - The zsh subcommand-decline integration test now uses `rm` under a
    `WorkspaceWrite` sandbox. The previous `/usr/bin/true` scenario is
    auto-allowed by the new `shell-escalation` policy path, which no longer
    produces subcommand approval prompts.
  • Prefer v2 websockets if available (#12428)
    And also cleanup settings flow to avoid reading many separate flags.
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • fix: codex-arg0 no longer depends on codex-core (#12434)
    ## Why
    
    `codex-rs/arg0` only needed two things from `codex-core`:
    
    - the `find_codex_home()` wrapper
    - the special argv flag used for the internal `apply_patch`
    self-invocation path
    
    That made `codex-arg0` depend on `codex-core` for a very small surface
    area. This change removes that dependency edge and moves the shared
    `apply_patch` invocation flag to a more natural boundary
    (`codex-apply-patch`) while keeping the contract explicitly documented.
    
    ## What Changed
    
    - Moved the internal `apply_patch` argv[1] flag constant out of
    `codex-core` and into `codex-apply-patch`.
    - Renamed the constant to `CODEX_CORE_APPLY_PATCH_ARG1` and documented
    that it is part of the Codex core process-invocation contract (even
    though it now lives in `codex-apply-patch`).
    - Updated `arg0`, the core apply-patch runtime, and the `codex-exec`
    apply-patch test to import the constant from `codex-apply-patch`.
    - Updated `codex-rs/arg0` to call
    `codex_utils_home_dir::find_codex_home()` directly instead of
    `codex_core::config::find_codex_home()`.
    - Removed the `codex-core` dependency from `codex-rs/arg0` and added the
    needed direct dependency on `codex-utils-home-dir`.
    - Added `codex-apply-patch` as a dev-dependency for `codex-rs/exec`
    tests (the apply-patch test now imports the moved constant directly).
    
    ## Verification
    
    - `cargo test -p codex-apply-patch`
    - `cargo test -p codex-arg0`
    - `cargo test -p codex-core --lib apply_patch`
    - `cargo test -p codex-exec
    test_standalone_exec_cli_can_use_apply_patch`
    - `cargo shear`
  • chore: remove codex-core public protocol/shell re-exports (#12432)
    ## Why
    
    `codex-rs/core/src/lib.rs` re-exported a broad set of types and modules
    from `codex-protocol` and `codex-shell-command`. That made it easy for
    workspace crates to import those APIs through `codex-core`, which in
    turn hides dependency edges and makes it harder to reduce compile-time
    coupling over time.
    
    This change removes those public re-exports so call sites must import
    from the source crates directly. Even when a crate still depends on
    `codex-core` today, this makes dependency boundaries explicit and
    unblocks future work to drop `codex-core` dependencies where possible.
    
    ## What Changed
    
    - Removed public re-exports from `codex-rs/core/src/lib.rs` for:
    - `codex_protocol::protocol` and related protocol/model types (including
    `InitialHistory`)
      - `codex_protocol::config_types` (`protocol_config_types`)
    - `codex_shell_command::{bash, is_dangerous_command, is_safe_command,
    parse_command, powershell}`
    - Migrated workspace Rust call sites to import directly from:
      - `codex_protocol::protocol`
      - `codex_protocol::config_types`
      - `codex_protocol::models`
      - `codex_shell_command`
    - Added explicit `Cargo.toml` dependencies (`codex-protocol` /
    `codex-shell-command`) in crates that now import those crates directly.
    - Kept `codex-core` internal modules compiling by using `pub(crate)`
    aliases in `core/src/lib.rs` (internal-only, not part of the public
    API).
    - Updated the two utility crates that can already drop a `codex-core`
    dependency edge entirely:
      - `codex-utils-approval-presets`
      - `codex-utils-cli`
    
    ## Verification
    
    - `cargo test -p codex-utils-approval-presets`
    - `cargo test -p codex-utils-cli`
    - `cargo check --workspace --all-targets`
    - `just clippy`
  • Wire realtime api to core (#12268)
    - Introduce `RealtimeConversationManager` for realtime API management 
    - Add `op::conversation` to start conversation, insert audio, insert
    text, and close conversation.
    - emit conversation lifecycle and realtime events.
    - Move shared realtime payload types into codex-protocol and add core
    e2e websocket tests for start/replace/transport-close paths.
    
    Things to consider:
    - Should we use the same `op::` and `Events` channel to carry audio? I
    think we should try this simple approach and later we can create
    separate one if the channels got congested.
    - Sending text updates to the client: we can start simple and later
    restrict that.
    - Provider auth isn't wired for now intentionally
  • feat(core): zsh exec bridge (#12052)
    zsh fork PR stack:
    - https://github.com/openai/codex/pull/12051 
    - https://github.com/openai/codex/pull/12052 👈 
    
    ### Summary
    This PR introduces a feature-gated native shell runtime path that routes
    shell execution through a patched zsh exec bridge, removing MCP-specific
    behavior from the shell hot path while preserving existing
    CommandExecution lifecycle semantics.
    
    When shell_zsh_fork is enabled, shell commands run via patched zsh with
    per-`execve` interception through EXEC_WRAPPER. Core receives wrapper
    IPC requests over a Unix socket, applies existing approval policy, and
    returns allow/deny before the subcommand executes.
    
    ### What’s included
    **1) New zsh exec bridge runtime in core**
    - Wrapper-mode entrypoint (maybe_run_zsh_exec_wrapper_mode) for
    EXEC_WRAPPER invocations.
    - Per-execution Unix-socket IPC handling for wrapper requests/responses.
    - Approval callback integration using existing core approval
    orchestration.
    - Streaming stdout/stderr deltas to existing command output event
    pipeline.
    - Error handling for malformed IPC, denial/abort, and execution
    failures.
    
    **2) Session lifecycle integration**
    SessionServices now owns a `ZshExecBridge`.
    Session startup initializes bridge state; shutdown tears it down
    cleanly.
    
    **3) Shell runtime routing (feature-gated)**
    When `shell_zsh_fork` is enabled:
    - Build execution env/spec as usual.
    - Add wrapper socket env wiring.
    - Execute via `zsh_exec_bridge.execute_shell_request(...)` instead of
    the regular shell path.
    - Non-zsh-fork behavior remains unchanged.
    
    **4) Config + feature wiring**
    - Added `Feature::ShellZshFork` (under development).
    - Added config support for `zsh_path` (optional absolute path to patched
    zsh):
    - `Config`, `ConfigToml`, `ConfigProfile`, overrides, and schema.
    - Session startup validates that `zsh_path` exists/usable when zsh-fork
    is enabled.
    - Added startup test for missing `zsh_path` failure mode.
    
    **5) Seatbelt/sandbox updates for wrapper IPC**
    - Extended seatbelt policy generation to optionally allow outbound
    connection to explicitly permitted Unix sockets.
    - Wired sandboxing path to pass wrapper socket path through to seatbelt
    policy generation.
    - Added/updated seatbelt tests for explicit socket allow rule and
    argument emission.
    
    **6) Runtime entrypoint hooks**
    - This allows the same binary to act as the zsh wrapper subprocess when
    invoked via `EXEC_WRAPPER`.
    
    **7) Tool selection behavior**
    - ToolsConfig now prefers ShellCommand type when shell_zsh_fork is
    enabled.
    - Added test coverage for precedence with unified-exec enabled.
  • feat(core): add structured network approval plumbing and policy decision model (#11672)
    ### Description
    #### Summary
    Introduces the core plumbing required for structured network approvals
    
    #### What changed
    - Added structured network policy decision modeling in core.
    - Added approval payload/context types needed for network approval
    semantics.
    - Wired shell/unified-exec runtime plumbing to consume structured
    decisions.
    - Updated related core error/event surfaces for structured handling.
    - Updated protocol plumbing used by core approval flow.
    - Included small CLI debug sandbox compatibility updates needed by this
    layer.
    
    #### Why
    establishes the minimal backend foundation for network approvals without
    yet changing high-level orchestration or TUI behavior.
    
    #### Notes
    - Behavior remains constrained by existing requirements/config gating.
    - Follow-up PRs in the stack handle orchestration, UX, and app-server
    integration.
    
    ---------
    
    Co-authored-by: Codex <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
  • Report syntax errors in rules file (#11686)
    Currently, if there are syntax errors detected in the starlark rules
    file, the entire policy is silently ignored by the CLI. The app server
    correctly emits a message that can be displayed in a GUI.
    
    This PR changes the CLI (both the TUI and non-interactive exec) to fail
    when the rules file can't be parsed. It then prints out an error message
    and exits with a non-zero exit code. This is consistent with the
    handling of errors in the config file.
    
    This addresses #11603
  • [feat] add seatbelt permission files (#11639)
    Add seatbelt permission extension abstraction as permission files for
    seatbelt profiles. This should complement our current sandbox policy
  • add a slash command to grant sandbox read access to inaccessible directories (#11512)
    There is an edge case where a directory is not readable by the sandbox.
    In practice, we've seen very little of it, but it can happen so this
    slash command unlocks users when it does.
    
    Future idea is to make this a tool that the agent knows about so it can
    be more integrated.