Commit Graph

12 Commits

  • bound prompt image cache retention (#28294)
    ## Why
    
    The prompt image cache was bounded to 32 entries, but not by the size of
    those entries. A set of large encoded images could therefore retain
    substantially more memory than intended. Cache hits also cloned the full
    encoded payload.
    
    ## What changed
    
    - cap the cache at 64 MiB of encoded image data while preserving its
    existing 32-entry limit
    - skip caching an image that exceeds the entire byte budget
    - evict least-recently-used entries until the cache is back within its
    byte budget
    - share cached encoded bytes with `Arc<[u8]>` so cache hits do not
    deep-clone image payloads
    
    ## Validation
    
    - `just test -p codex-utils-image`
  • image: preserve metadata when resizing prompt images (#27266)
    ## Summary
    
    - Preserve ICC profiles and EXIF metadata when resizing and re-encoding
    prompt images.
    - Retain EXIF orientation metadata without rotating or otherwise
    modifying the pixel data locally.
    - Support metadata preservation for PNG, JPEG, and WebP outputs.
    - Continue returning the original bytes when an image does not require
    re-encoding.
    
    This intentionally preserves the metadata most important for rendering
    prompt images faithfully. Other format-specific metadata is not copied.
    
    ## Motivation
    
    Client-side resizing previously discarded image metadata during
    re-encoding. This could lose color-profile information and EXIF
    orientation needed by downstream image consumers.
    
    
    #### [git stack](https://github.com/magus/git-stack-cli)
    -  `1` https://github.com/openai/codex/pull/27245
    -  `2` https://github.com/openai/codex/pull/27247
    -  `3` https://github.com/openai/codex/pull/27246
    - 👉 `4` https://github.com/openai/codex/pull/27266
  • core: resize all history images behind a feature flag (#27247)
    ## Summary
    
    Adds complete client-side image preparation behind the default-off
    `resize_all_images` feature flag.
    
    When enabled, local image producers defer decoding and resizing. Images
    are prepared centrally before insertion into conversation history,
    covering user input, `view_image`, and structured tool-output images.
    
    ## Behavior
    
    - Processes base64 `data:` images in messages and function/custom tool
    outputs.
    - Leaves non-data URLs, including HTTP(S) URLs, unchanged.
    - Applies image-detail budgets:
      - `high` and omitted: 2048px maximum dimension and 2.5K 32px patches.
      - `original`: 6000px maximum dimension and 10K 32px patches.
      - `auto`: uses the same 2048px / 2.5K-patch budget as high.
      - `low`: unsupported and replaced with an actionable placeholder.
    - Preserves original image bytes when no resize or format conversion is
    needed.
    - Enforces the shared 1 GiB encoded and decoded data-URL sanity limits.
    - Replaces only an image that fails preparation, preserving sibling
    content and tool-output metadata.
    - Uses bounded placeholders distinguishing generic processing failures,
    oversized images, and unsupported `low` detail.
    - Prepares resumed and forked history before installing it as live
    history without modifying persisted rollouts.
    
    ## Flag-Off Behavior
    
    When `resize_all_images` is disabled:
    
    - Existing local user-input and `view_image` processing remains
    unchanged.
    - Existing decoding and error behavior remains unchanged.
    - Arbitrary tool-output images are not processed.
    - HTTP(S) image URLs continue to be forwarded unchanged.
    
    
    #### [git stack](https://github.com/magus/git-stack-cli)
    -  `1` https://github.com/openai/codex/pull/27245
    - 👉 `2` https://github.com/openai/codex/pull/27247
    -  `3` https://github.com/openai/codex/pull/27246
    -  `4` https://github.com/openai/codex/pull/27266
  • image: add shared data URL preparation utilities (#27245)
    ## Summary
    
    Add shared image-processing primitives needed for centralized image
    preparation in a follow-up PR.
    
    - Add `load_data_url_for_prompt` for decoding and preparing base64 image
    data URLs.
    - Add configurable maximum-dimension and 32px patch-budget resizing.
    - Enforce a 1 GiB sanity limit on both encoded and decoded data-URL
    representations.
    - Preserve original PNG, JPEG, and WebP bytes when resizing is
    unnecessary.
    - Preserve the existing GIF-to-PNG behavior.
    - Move image utility tests into the existing sidecar test module.
    
    ## Behavior
    
    This PR is intended to be runtime behavior-preserving.
    
    Existing production callers continue using
    `PromptImageMode::ResizeToFit` and `PromptImageMode::Original` with
    their existing semantics. The new data-URL entrypoint and configurable
    resize mode have no production callers in this PR; they are used by the
    next PR in the stack.
    
    This PR does not change user-input handling, `view_image`, history
    insertion, request construction, HTTP image URL forwarding, or
    app-server behavior.
    
    
    #### [git stack](https://github.com/magus/git-stack-cli)
    - 👉 `1` https://github.com/openai/codex/pull/27245
    -  `2` https://github.com/openai/codex/pull/27247
    -  `3` https://github.com/openai/codex/pull/27246
    -  `4` https://github.com/openai/codex/pull/27266
  • Update image resizing to fit 2048 square bounds (#18384)
    We don't have to downsize to 768 height.
  • feat: single app-server bootstrap in TUI (#16582)
    Before this, the TUI was starting 2 app-server. One to check the login
    status and one to actually start the session
    
    This PR make only one app-server startup and defer the login check in
    async, outside of the frame rendering path
    
    ---------
    
    Co-authored-by: Codex <noreply@openai.com>
  • [codex] reduce module visibility (#16978)
    ## Summary
    - reduce public module visibility across Rust crates, preferring private
    or crate-private modules with explicit crate-root public exports
    - update external call sites and tests to use the intended public crate
    APIs instead of reaching through module trees
    - add the module visibility guideline to AGENTS.md
    
    ## Validation
    - `cargo check --workspace --all-targets --message-format=short` passed
    before the final fix/format pass
    - `just fix` completed successfully
    - `just fmt` completed successfully
    - `git diff --check` passed
  • Return image URL from view_image tool (#15072)
    Cleanup image semantics in code mode.
    
    `view_image` now returns `{image_url:string, details?: string}` 
    
    `image()` now allows both string parameter and `{image_url:string,
    details?: string}`
  • Add FS abstraction and use in view_image (#14960)
    Adds an environment crate and environment + file system abstraction.
    
    Environment is a combination of attributes and services specific to
    environment the agent is connected to:
    File system, process management, OS, default shell.
    
    The goal is to move most of agent logic that assumes environment to work
    through the environment abstraction.
  • Add under-development original-resolution view_image support (#13050)
    ## Summary
    
    Add original-resolution support for `view_image` behind the
    under-development `view_image_original_resolution` feature flag.
    
    When the flag is enabled and the target model is `gpt-5.3-codex` or
    newer, `view_image` now preserves original PNG/JPEG/WebP bytes and sends
    `detail: "original"` to the Responses API instead of using the legacy
    resize/compress path.
    
    ## What changed
    
    - Added `view_image_original_resolution` as an under-development feature
    flag.
    - Added `ImageDetail` to the protocol models and support for serializing
    `detail: "original"` on tool-returned images.
    - Added `PromptImageMode::Original` to `codex-utils-image`.
      - Preserves original PNG/JPEG/WebP bytes.
      - Keeps legacy behavior for the resize path.
    - Updated `view_image` to:
    - use the shared `local_image_content_items_with_label_number(...)`
    helper in both code paths
      - select original-resolution mode only when:
        - the feature flag is enabled, and
        - the model slug parses as `gpt-5.3-codex` or newer
    - Kept local user image attachments on the existing resize path; this
    change is specific to `view_image`.
    - Updated history/image accounting so only `detail: "original"` images
    use the docs-based GPT-5 image cost calculation; legacy images still use
    the old fixed estimate.
    - Added JS REPL guidance, gated on the same feature flag, to prefer JPEG
    at 85% quality unless lossless is required, while still allowing other
    formats when explicitly requested.
    - Updated tests and helper code that construct
    `FunctionCallOutputContentItem::InputImage` to carry the new `detail`
    field.
    
    ## Behavior
    
    ### Feature off
    - `view_image` keeps the existing resize/re-encode behavior.
    - History estimation keeps the existing fixed-cost heuristic.
    
    ### Feature on + `gpt-5.3-codex+`
    - `view_image` sends original-resolution images with `detail:
    "original"`.
    - PNG/JPEG/WebP source bytes are preserved when possible.
    - History estimation uses the GPT-5 docs-based image-cost calculation
    for those `detail: "original"` images.
    
    
    #### [git stack](https://github.com/magus/git-stack-cli)
    - 👉 `1` https://github.com/openai/codex/pull/13050
    -  `2` https://github.com/openai/codex/pull/13331
    -  `3` https://github.com/openai/codex/pull/13049
  • feat: image resizing (#5446)
    Add image resizing on the client side to reduce load on the API