9 Commits

  • Let image generation extension hosts control output persistence (#29711)
    ## Why
    
    Some extension hosts need generated images returned without writing them
    to the local filesystem or giving the model a local path.
    
    ## What changed
    
    **tl;dr**: we now conduct all extension operations in the image gen
    extension
    
    - Let hosts provide an optional image save root when installing the
    extension.
    - Save images and return path hints only when a save root is configured.
    - Return image data without saving or adding a path hint when no save
    root is configured.
    - Preserve the extension-provided `saved_path` instead of persisting
    extension images again in core.
    - Leave built-in image generation unchanged.
    
    ## Validation
    
    - `just test -p codex-image-generation-extension`
    - `just test -p codex-app-server
    standalone_image_generation_returns_saved_path_hint_to_model`
    - `just test -p codex-core
    extension_tool_uses_granted_turn_permissions_without_local_persistence`
    - `just test -p codex-core tools::handlers::extension_tools::tests`
    - tested on CODEX CLI on both save_root: CODEX_HOME and None 
    - tested on CODEX APP on both as well
  • Fix image extension PathUri conversion (#27711)
    ## Why
    
    `main` stopped compiling when #27498 passed an `AbsolutePathBuf` to the
    `ExecutorFileSystem` API migrated to `PathUri` by #27653.
    
    ## What
    
    Convert referenced image paths to `PathUri` before filesystem reads,
    declare the internal path-URI dependency, and refresh `Cargo.lock`.
  • Route image extension reads through turn environments v2 (#27498)
    ## Why
    
    Image generation used `std::fs::read` for referenced image paths, which
    did not support environment-backed filesystems or their sandbox context.
    
    ## What changed
    
    - Expose optional turn environments to extension tool calls.
    - Include each environment’s ID, working directory, filesystem, and
    sandbox context.
    - Read referenced images through the selected environment filesystem.
    - Keep sandbox usage at the extension call site so extensions can choose
    the appropriate access mode.
    - Consolidate image request construction into one async function.
    - Add coverage for successful environment reads and read failures.
    
    ## Validation
    
    - `cargo check -p codex-image-generation-extension --tests`
    - `just fmt`
    - `just bazel-lock-update`
    - `just bazel-lock-check`
    
    `just test -p codex-image-generation-extension` could not complete
    because the build exhausted available disk space.
  • [codex] Remove async_trait from ToolExecutor (#27304)
    ## Why
    
    We're now [discouraging use of
    `async_trait`](https://github.com/openai/codex/pull/20242).
    
    Removing use of `async_trait` from `ToolExecutor` yields a `codex_core`
    debug test build speedup of ~78% (from 227.5s to 50.3s) on my machine.
    
    Stacked on #27299, this PR applies the trait change after the handler
    bodies have been outlined.
    
    ## What
    
    Changed `ToolExecutor::handle` to return an explicit boxed
    `ToolExecutorFuture` instead of using `async_trait`.
    
    Updated ToolExecutor implementors to return `Box::pin(...)`, reexported
    the future alias through `codex-tools` and `codex-extension-api`, and
    removed `codex-tools` direct `async-trait` dependency.
  • Route image edits through referenced file paths (#26486)
    ## Why
    
    Image edits should use the exact images selected by the model instead of
    inferring edit inputs from conversation history.
    
    ## What changed
    
    - Replaced the image tool's `action` argument with optional
    `referenced_image_paths`.
    - Treats omitted or empty references as generation and populated
    references as editing.
    - Reads referenced absolute image paths and packages them as image data
    URLs for the edit request.
    - Removed the previous history-selection and image-count heuristics.
    - Updated direct and code-mode tool instructions and calls.
    - Added an app-server integration test covering an attached image routed
    to the image edit endpoint.
    
    ## Validation
    - Tested end-to-end on local `just codex` with copy pasted image,
    attached image, etc.
    - `just test -p codex-image-generation-extension`
    - `just test -p codex-app-server
    standalone_image_edit_uses_attached_model_visible_image`
    - `just fix -p codex-image-generation-extension`
    - `just bazel-lock-check`
  • [codex] Use standalone tools for Responses Lite (#26490)
    ## Summary
    
    Responses Lite does not execute hosted Responses tools, so models using
    it must route web search and image generation through Codex-owned
    executors & standalone Response's API endpoints.
    
    This PR is stacked on #26487.
    
    ## Validation
    
    - `cargo test -p codex-core responses_lite_ --lib`
    - `cargo test -p codex-core
    standalone_executors_remain_hidden_without_flags_or_responses_lite
    --lib`
    - `cargo test -p codex-core
    hosted_tools_follow_provider_auth_model_and_config_gates --lib`
    - `cargo test -p codex-web-search-extension -p
    codex-image-generation-extension`
    - `cargo test -p codex-app-server --test all standalone_`
    - `cargo fmt --all -- --check`
  • Add saved image path hint to standalone image generation (#25947)
    ## Why
    
    Standalone image generation returns image bytes to the model, but the
    model also needs the host artifact path to reference the generated file
    in follow-up work.
    
    ## What changed
    
    - Append the default saved-image path hint alongside the generated image
    tool output.
    - Reuse the existing core image-generation hint text.
    - Pass the thread ID and Codex home directory needed to compute the
    artifact path.
    - Add app-server and extension coverage for the model-visible hint.
    
    ## Validation
    
    - `just fmt`
    - `just bazel-lock-check`
    - `just test -p codex-app-server
    standalone_image_generation_returns_saved_path_hint_to_model`
  • Route extension image generation through the native image completion pipeline (#24972)
    ## Why
    
    The standalone `image_gen.imagegen` extension should behave like native
    image generation for artifact persistence and UI completion, while
    returning its save-location guidance as part of the tool result instead
    of injecting a developer message.
    
    ## What Changed
    
    - Added an image-generation completion hook for extension tools so core
    can persist generated images and emit the existing `ImageGeneration`
    lifecycle events.
    - Reused core image artifact persistence for extension output and
    removed extension-local save-path/file-writing logic.
    - Split shared image persistence from built-in finalization so native
    image generation keeps its existing developer-message instruction
    behavior.
    - Returned the generated image save-location instruction through the
    extension `FunctionCallOutput`, alongside the generated image input for
    model follow-up.
    - Preserved the existing image-generation event shape for current UI and
    replay compatibility.
    - Avoided cloning the full generated-image base64 payload when emitting
    the in-progress image item.
    - Removed dependencies no longer needed after moving persistence out of
    the extension crate.
    
    ## Fast Follow
    - Adjust the existing Extension API and add a general `TurnItem`
    finalization path for re-usability of code
    
    ## Validation
    
    - Ran `just fmt`.
    - Ran `just bazel-lock-update`.
    - Ran `just bazel-lock-check`.
    - Ran `just test -p codex-tools -p codex-extension-api -p
    codex-image-generation-extension`.
    - Ran `just test -p codex-core
    image_generation_publication_is_finalized_by_core`.
    - Ran `just test -p codex-core
    handle_output_item_done_records_image_save_history_message`.
    - Ran `just fix -p codex-tools -p codex-extension-api -p codex-core -p
    codex-image-generation-extension`.
  • Add feature-gated standalone image generation extension (#24723)
    ## Why
    
    Add a standalone image generation path that can be exercised
    independently of hosted Responses image generation, while retaining the
    hosted tool as fallback unless the extension is actually available to
    the model.
    
    ## What changed
    
    - Added the `codex-image-generation-extension` crate with standalone
    generate/edit execution, prior-image selection for edits, model-visible
    image output, and local generated-image persistence.
    - Installed the extension in app-server behind the disabled-by-default
    `imagegenext` feature and backend eligibility checks.
    - Updated core tool planning so eligible `image_gen.imagegen` exposure
    replaces hosted `image_generation`, while unavailable configurations
    retain hosted fallback.
    - Added coverage for extension behavior, edit history reuse, feature
    gating, auth eligibility, and hosted-tool replacement.
    - The extension is installed through app-server only in this PR; other
    execution paths retain hosted image generation because hosted
    replacement occurs only when the standalone executor is actually
    registered and model-visible.
    - The initial extension contract intentionally fixes the image model to
    `gpt-image-2` and uses automatic image parameters.
    - Native generated-image history/card parity and rollout persistence
    cleanup are intentionally deferred follow-up work.
    
    ## Validation
    
    - `just test -p codex-image-generation-extension`
    - `just test -p codex-features`
    - `just test -p codex-core
    hosted_tools_follow_provider_auth_model_and_config_gates`
    - `just test -p codex-app-server`
    - `just fix -p codex-image-generation-extension -p codex-features -p
    codex-core -p codex-app-server`
    - `just fmt`
    - `just bazel-lock-update`
    - `just bazel-lock-check`
    
    ---------
    
    Co-authored-by: jif-oai <jif@openai.com>