9 Commits

  • Let image generation extension hosts control output persistence (#29711)
    ## Why
    
    Some extension hosts need generated images returned without writing them
    to the local filesystem or giving the model a local path.
    
    ## What changed
    
    **tl;dr**: we now conduct all extension operations in the image gen
    extension
    
    - Let hosts provide an optional image save root when installing the
    extension.
    - Save images and return path hints only when a save root is configured.
    - Return image data without saving or adding a path hint when no save
    root is configured.
    - Preserve the extension-provided `saved_path` instead of persisting
    extension images again in core.
    - Leave built-in image generation unchanged.
    
    ## Validation
    
    - `just test -p codex-image-generation-extension`
    - `just test -p codex-app-server
    standalone_image_generation_returns_saved_path_hint_to_model`
    - `just test -p codex-core
    extension_tool_uses_granted_turn_permissions_without_local_persistence`
    - `just test -p codex-core tools::handlers::extension_tools::tests`
    - tested on CODEX CLI on both save_root: CODEX_HOME and None 
    - tested on CODEX APP on both as well
  • [codex] replace remote images with model-visible error text (#29417)
    ## What
    
    This PR will extend the existing centralized image-preparation path to
    replace HTTP(S) image inputs with a model visible error message. It
    won't "ruin" and break existing rollouts, but it will deprecate support
    for the pathway. App server clients should no longer use HTTP image urls
    if they'd like to upgrade.
    
    The HTTP image url pathway is currently resolved in the responsesapi. It
    is slow and not reccomended.
    
    ## Behavior
    
    - HTTP(S) image URL: replace with `input_text`
    - data URL: use the existing decode and resize path
    - other image URL schemes: leave unchanged
    
    This intentionally does not change app-server ingress. That validation
    remains a follow-up.
    
    ## Test plan
    
    - `just test -p codex-core -E
    'test(/image_preparation|prepares_image_failures_before_history_insertion|prepares_resumed_history_before_installing_it|responses_lite_prepares_images/)'`
    — 7 passed
    - `just fix -p codex-core`
    - `just fmt`
  • remove flag for image preparation (#29429)
    ## What
    
    - make Fjord's centralized response-item image preparation unconditional
    for new and resumed history
    - have local user images and `view_image` outputs always defer decoding
    and resizing to that path
    - retain `resize_all_images` as an ignored, removed compatibility key
    for released clients
    - delete the flag-off producer paths and obsolete policy-specific tests
    
    ## Why
    
    Centralized preparation is now the intended image path. Keeping the
    runtime feature checks also kept two image-processing implementations
    alive and allowed client config to select the legacy behavior.
    
    This is a clean replacement for #28975, rebuilt from the latest `main`.
    
    ## How
    
    `prepare_response_items` now runs whenever items enter history and
    whenever persisted history is reconstructed. Producers emit deferred
    image data, so malformed images become the existing model-visible
    placeholder instead of failing the session at the producer.
    
    ## Test plan
    
    - `just fmt`
    - `just fix -p codex-core -p codex-features`
    - `just test -p codex-features` — 52 passed
    - focused affected `codex-core` set — 20 passed
    - `just test -p codex-core handle_accepts_explicit_high_detail` — 1
    passed
    - full `just test -p codex-core` attempt — 2,723 passed; 88 unrelated
    environment failures from read-only `~/.codex` SQLite state and
    unavailable integration helper binaries
  • Handle standalone image generation failures as terminal items (#27920)
    ## Why
    
    Standalone image generation emitted a started item but no terminal item
    when the backend failed. Clients could leave the operation unresolved or
    render it as successful.
    
    ## What changed
    
    - Emit a terminal image-generation item with `status: "failed"` when
    generation or editing fails.
    - Skip image persistence for failed terminal items.
    - Render failed image generation distinctly in TUI history.
    - Preserve the status when handling live and replayed terminal items.
    
    ## Looks for TUI, App-Side change needed 
    
    <img width="867" height="89" alt="image"
    src="https://github.com/user-attachments/assets/9e32342f-a982-411e-8498-456639fc468a"
    />
    
    ## Validation
    
    - `just test -p codex-image-generation-extension`
    - App-server image-generation tests
    - Core stream-event tests
    - TUI image-generation lifecycle and snapshot tests
    - Scoped Clippy and formatting
  • Route image extension reads through turn environments v2 (#27498)
    ## Why
    
    Image generation used `std::fs::read` for referenced image paths, which
    did not support environment-backed filesystems or their sandbox context.
    
    ## What changed
    
    - Expose optional turn environments to extension tool calls.
    - Include each environment’s ID, working directory, filesystem, and
    sandbox context.
    - Read referenced images through the selected environment filesystem.
    - Keep sandbox usage at the extension call site so extensions can choose
    the appropriate access mode.
    - Consolidate image request construction into one async function.
    - Add coverage for successful environment reads and read failures.
    
    ## Validation
    
    - `cargo check -p codex-image-generation-extension --tests`
    - `just fmt`
    - `just bazel-lock-update`
    - `just bazel-lock-check`
    
    `just test -p codex-image-generation-extension` could not complete
    because the build exhausted available disk space.
  • Route image edits through referenced file paths (#26486)
    ## Why
    
    Image edits should use the exact images selected by the model instead of
    inferring edit inputs from conversation history.
    
    ## What changed
    
    - Replaced the image tool's `action` argument with optional
    `referenced_image_paths`.
    - Treats omitted or empty references as generation and populated
    references as editing.
    - Reads referenced absolute image paths and packages them as image data
    URLs for the edit request.
    - Removed the previous history-selection and image-count heuristics.
    - Updated direct and code-mode tool instructions and calls.
    - Added an app-server integration test covering an attached image routed
    to the image edit endpoint.
    
    ## Validation
    - Tested end-to-end on local `just codex` with copy pasted image,
    attached image, etc.
    - `just test -p codex-image-generation-extension`
    - `just test -p codex-app-server
    standalone_image_edit_uses_attached_model_visible_image`
    - `just fix -p codex-image-generation-extension`
    - `just bazel-lock-check`
  • Add saved image path hint to standalone image generation (#25947)
    ## Why
    
    Standalone image generation returns image bytes to the model, but the
    model also needs the host artifact path to reference the generated file
    in follow-up work.
    
    ## What changed
    
    - Append the default saved-image path hint alongside the generated image
    tool output.
    - Reuse the existing core image-generation hint text.
    - Pass the thread ID and Codex home directory needed to compute the
    artifact path.
    - Add app-server and extension coverage for the model-visible hint.
    
    ## Validation
    
    - `just fmt`
    - `just bazel-lock-check`
    - `just test -p codex-app-server
    standalone_image_generation_returns_saved_path_hint_to_model`
  • Restore Windows coverage for code-mode image generation exposure (#25960)
    ## Summary
    
    Restore Windows coverage for standalone image generation in code mode.
    
    The previous test executed a V8-backed code-mode cell on Windows CI,
    where that runtime path is intentionally excluded because it is
    unreliable. The test was then ignored entirely on Windows, removing
    useful coverage.
    
    This splits the test into two checks:
    
    - All platforms verify that `image_gen__imagegen` is exposed to the
    model when image generation is configured for code mode only.
    - Non-Windows platforms continue to execute the full V8-backed flow and
    verify that the nested image-generation call succeeds.
    
    ## Verification
    
    - `just fmt`
    - `git diff --check`
    - `just test -p codex-app-server standalone_image_generation`
    
    Result: 3 tests passed, plus the required bench smoke check.
  • Expose standalone image generation in code mode (#25923)
    ## Why
    
    Standalone image generation remained top-level-only in code-mode
    sessions.
    
    ## What changed
    
    - Change imagegen exposure from `DirectModelOnly` to `Direct`.
    - Keep direct-mode access while enabling nested code-mode access.
    - Add a focused regression test for the exposure contract.
    
    ## Validation
    
    - `just test -p codex-image-generation-extension`