Commit Graph

14 Commits

  • feat(tui): syntax highlighting via syntect with theme picker (#11447)
    ## Summary
    
    Adds syntax highlighting to the TUI for fenced code blocks in markdown
    responses and file diffs, plus a `/theme` command with live preview and
    persistent theme selection. Uses syntect (~250 grammars, 32 bundled
    themes, ~1 MB binary cost) — the same engine behind `bat`, `delta`, and
    `xi-editor`. Includes guardrails for large inputs, graceful fallback to
    plain text, and SSH-aware clipboard integration for the `/copy` command.
    
    <img width="1554" height="1014" alt="image"
    src="https://github.com/user-attachments/assets/38737a79-8717-4715-b857-94cf1ba59b85"
    />
    
    <img width="2354" height="1374" alt="image"
    src="https://github.com/user-attachments/assets/25d30a00-c487-4af8-9cb6-63b0695a4be7"
    />
    
    ## Problem
    
    Code blocks in the TUI (markdown responses and file diffs) render
    without syntax highlighting, making it hard to scan code at a glance.
    Users also have no way to pick a color theme that matches their terminal
    aesthetic.
    
    ## Mental model
    
    The highlighting system has three layers:
    
    1. **Syntax engine** (`render::highlight`) -- a thin wrapper around
    syntect + two-face. It owns a process-global `SyntaxSet` (~250 grammars)
    and a `RwLock<Theme>` that can be swapped at runtime. All public entry
    points accept `(code, lang)` and return ratatui `Span`/`Line` vectors or
    `None` when the language is unrecognized or the input exceeds safety
    guardrails.
    
    2. **Rendering consumers** -- `markdown_render` feeds fenced code blocks
    through the engine; `diff_render` highlights Add/Delete content as a
    whole file and Update hunks per-hunk (preserving parser state across
    hunk lines). Both callers fall back to plain unstyled text when the
    engine returns `None`.
    
    3. **Theme lifecycle** -- at startup the config's `tui.theme` is
    resolved to a syntect `Theme` via `set_theme_override`. At runtime the
    `/theme` picker calls `set_syntax_theme` to swap themes live; on cancel
    it restores the snapshot taken at open. On confirm it persists `[tui]
    theme = "..."` to config.toml.
    
    ## Non-goals
    
    - Inline diff highlighting (word-level change detection within a line).
    - Semantic / LSP-backed highlighting.
    - Theme authoring tooling; users supply standard `.tmTheme` files.
    
    ## Tradeoffs
    
    | Decision | Upside | Downside |
    | ------------------------------------------------ |
    ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    |
    -----------------------------------------------------------------------------------------------------------------------
    |
    | syntect over tree-sitter / arborium | ~1 MB binary increase for ~250
    grammars + 32 themes; battle-tested crate powering widely-used tools
    (`bat`, `delta`, `xi-editor`). tree-sitter would add ~12 MB for 20-30
    languages or ~35 MB for full coverage. | Regex-based; less structurally
    accurate than tree-sitter for some languages (e.g. language injections
    like JS-in-HTML). |
    | Global `RwLock<Theme>` | Enables live `/theme` preview without
    threading Theme through every call site | Lock contention risk
    (mitigated: reads vastly outnumber writes, single UI thread) |
    | Skip background / italic / underline from themes | Terminal BG
    preserved, avoids ugly rendering on some themes | Themes that rely on
    these properties lose fidelity |
    | Guardrails: 512 KB / 10k lines | Prevents pathological stalls on huge
    diffs or pastes | Very large files render without color |
    
    ## Architecture
    
    ```
    config.toml  ─[tui.theme]─>  set_theme_override()  ─>  THEME (RwLock)
                                                                  │
                      ┌───────────────────────────────────────────┘
                      │
      markdown_render ─── highlight_code_to_lines(code, lang) ─> Vec<Line>
      diff_render     ─── highlight_code_to_styled_spans(code, lang) ─> Option<Vec<Vec<Span>>>
                      │
                      │   (None ⇒ plain text fallback)
                      │
      /theme picker   ─── set_syntax_theme(theme)    // live preview swap
                      ─── current_syntax_theme()      // snapshot for cancel
                      ─── resolve_theme_by_name(name) // lookup by kebab-case
    ```
    
    Key files:
    
    - `tui/src/render/highlight.rs` -- engine, theme management, guardrails
    - `tui/src/diff_render.rs` -- syntax-aware diff line wrapping
    - `tui/src/theme_picker.rs` -- `/theme` command builder
    - `tui/src/bottom_pane/list_selection_view.rs` -- side content panel,
    callbacks
    - `core/src/config/types.rs` -- `Tui::theme` field
    - `core/src/config/edit.rs` -- `syntax_theme_edit()` helper
    
    ## Observability
    
    - `tracing::warn` when a configured theme name cannot be resolved.
    - `Config::startup_warnings` surfaces the same message as a TUI banner.
    - `tracing::error` when persisting theme selection fails.
    
    ## Tests
    
    - Unit tests in `highlight.rs`: language coverage, fallback behavior,
    CRLF stripping, style conversion, guardrail enforcement, theme name
    mapping exhaustiveness.
    - Unit tests in `diff_render.rs`: snapshot gallery at multiple terminal
    sizes (80x24, 94x35, 120x40), syntax-highlighted wrapping, large-diff
    guardrail, rename-to-different-extension highlighting, parser state
    preservation across hunk lines.
    - Unit tests in `theme_picker.rs`: preview rendering (wide + narrow),
    dim overlay on deletions, subtitle truncation, cancel-restore, fallback
    for unavailable configured theme.
    - Unit tests in `list_selection_view.rs`: side layout geometry, stacked
    fallback, buffer clearing, cancel/selection-changed callbacks.
    - Integration test in `lib.rs`: theme warning uses the final
    (post-resume) config.
    
    ## Cargo Deny: Unmaintained Dependency Exceptions
    
    This PR adds two `cargo deny` advisory exceptions for transitive
    dependencies pulled in by `syntect v5.3.0`:
    
    | Advisory | Crate | Status |
    |----------|-------|--------|
    | RUSTSEC-2024-0320 | `yaml-rust` | Unmaintained (maintainer
    unreachable) |
    | RUSTSEC-2025-0141 | `bincode` | Unmaintained (development ceased;
    v1.3.3 considered complete) |
    
    **Why this is safe in our usage:**
    
    - Neither advisory describes a known security vulnerability. Both are
    "unmaintained" notices only.
    - `bincode` is used by syntect to deserialize pre-compiled syntax sets.
    Again, these are **static vendored artifacts** baked into the binary at
    build time. No user-supplied bincode data is ever deserialized. - Attack
    surface is zero for both crates; exploitation would require a
    supply-chain compromise of our own build artifacts.
    - These exceptions can be removed when syntect migrates to `yaml-rust2`
    and drops `bincode`, or when alternative crates are available upstream.
  • chore: upgrade to Rust 1.92.0 (#8860)
    **Summary**
    - Upgrade Rust toolchain used by CI to 1.92.0.
    - Address new clippy `derivable_impls` warnings by deriving `Default`
    for enums across protocol, core, backend openapi models, and
    windows-sandbox setup.
    - Tidy up related test/config behavior (originator header handling, env
    override cleanup) and remove a now-unused assignment in TUI/TUI2 render
    layout.
    
    **Testing**
    - `just fmt`
    - `just fix -p codex-tui`
    - `just fix -p codex-tui2`
    - `just fix -p codex-windows-sandbox`
    - `cargo test -p codex-tui`
    - `cargo test -p codex-tui2`
    - `cargo test -p codex-windows-sandbox`
    - `cargo test -p codex-core --test all`
    - `cargo test -p codex-app-server --test all`
    - `cargo test -p codex-mcp-server --test all`
    - `cargo test --all-features`
  • tui: refactor ChatWidget and BottomPane to use Renderables (#5565)
    - introduce RenderableItem to support both owned and borrowed children
    in composite Renderables
    - refactor some of our gnarlier manual layouts, BottomPane and
    ChatWidget, to use ColumnRenderable
    - Renderable and friends now handle cursor_pos()
  • tui: fix wrapping in trust_directory (#5007)
    Refactor trust_directory to use ColumnRenderable & friends, thus
    correcting wrapping behavior at small widths. Also introduce
    RowRenderable with fixed-width rows.
    
    - fixed wrapping in trust_directory
    - changed selector cursor to match other list item selections
    - allow y/n to work as well as 1/2
    - fixed key_hint to be standard
    
    before:
    <img width="661" height="550" alt="Screenshot 2025-10-09 at 9 50 36 AM"
    src="https://github.com/user-attachments/assets/e01627aa-bee4-4e25-8eca-5575c43f05bf"
    />
    
    after:
    <img width="661" height="550" alt="Screenshot 2025-10-09 at 9 51 31 AM"
    src="https://github.com/user-attachments/assets/cb816cbd-7609-4c83-b62f-b4dba392d79a"
    />
  • feat(tui): switch to tree-sitter-highlight bash highlighting (#4666)
    use tree-sitter-highlight instead of custom logic over the tree-sitter
    tree to highlight bash.
  • tui: bring the transcript closer to display mode (#4848)
    before
    <img width="1161" height="836" alt="Screenshot 2025-10-06 at 3 06 52 PM"
    src="https://github.com/user-attachments/assets/7622fd6b-9d37-402f-8651-61c2c55dcbc6"
    />
    
    after
    <img width="1161" height="858" alt="Screenshot 2025-10-06 at 3 07 02 PM"
    src="https://github.com/user-attachments/assets/1498f327-1d1a-4630-951f-7ca371ab0139"
    />
  • dynamic width for line numbers in diffs (#4664)
    instead of always reserving 6 spaces for the line number and gutter, we
    now dynamically adjust to the width of the longest number.
    
    <img width="871" height="616" alt="Screenshot 2025-10-03 at 8 21 00 AM"
    src="https://github.com/user-attachments/assets/5f18eae6-7c85-48fc-9a41-31978ae71a62"
    />
    <img width="871" height="616" alt="Screenshot 2025-10-03 at 8 21 21 AM"
    src="https://github.com/user-attachments/assets/9009297d-7690-42b9-ae42-9566b3fea86c"
    />
    <img width="871" height="616" alt="Screenshot 2025-10-03 at 8 21 57 AM"
    src="https://github.com/user-attachments/assets/669096fd-dddc-407e-bae8-d0c6626fa0bc"
    />
  • rework patch/exec approval UI (#4573)
    | Scenario | Screenshot |
    | ---------------------- |
    ----------------------------------------------------------------------------------------------------------------------------------------------------
    |
    | short patch | <img width="1096" height="533" alt="short patch"
    src="https://github.com/user-attachments/assets/8a883429-0965-4c0b-9002-217b3759b557"
    /> |
    | short command | <img width="1096" height="533" alt="short command"
    src="https://github.com/user-attachments/assets/901abde8-2494-4e86-b98a-7cabaf87ca9c"
    /> |
    | long patch | <img width="1129" height="892" alt="long patch"
    src="https://github.com/user-attachments/assets/fa799a29-a0d6-48e6-b2ef-10302a7916d3"
    /> |
    | long command | <img width="1096" height="892" alt="long command"
    src="https://github.com/user-attachments/assets/11ddf79b-98cb-4b60-ac22-49dfa7779343"
    /> |
    | viewing complete patch | <img width="1129" height="892" alt="viewing
    complete patch"
    src="https://github.com/user-attachments/assets/81666958-af94-420e-aa66-b60d0a42b9db"
    /> |
  • update composer + user message styling (#4240)
    Changes:
    
    - the composer and user messages now have a colored background that
    stretches the entire width of the terminal.
    - the prompt character was changed from a cyan `▌` to a bold `›`.
    - the "working" shimmer now follows the "dark gray" color of the
    terminal, better matching the terminal's color scheme
    
    | Terminal + Background        | Screenshot |
    |------------------------------|------------|
    | iTerm with dark bg | <img width="810" height="641" alt="Screenshot
    2025-09-25 at 11 44 52 AM"
    src="https://github.com/user-attachments/assets/1317e579-64a9-4785-93e6-98b0258f5d92"
    /> |
    | iTerm with light bg | <img width="845" height="540" alt="Screenshot
    2025-09-25 at 11 46 29 AM"
    src="https://github.com/user-attachments/assets/e671d490-c747-4460-af0b-3f8d7f7a6b8e"
    /> |
    | iTerm with color bg | <img width="825" height="564" alt="Screenshot
    2025-09-25 at 11 47 12 AM"
    src="https://github.com/user-attachments/assets/141cda1b-1164-41d5-87da-3be11e6a3063"
    /> |
    | Terminal.app with dark bg | <img width="577" height="367"
    alt="Screenshot 2025-09-25 at 11 45 22 AM"
    src="https://github.com/user-attachments/assets/93fc4781-99f7-4ee7-9c8e-3db3cd854fe5"
    /> |
    | Terminal.app with light bg | <img width="577" height="367"
    alt="Screenshot 2025-09-25 at 11 46 04 AM"
    src="https://github.com/user-attachments/assets/19bf6a3c-91e0-447b-9667-b8033f512219"
    /> |
    | Terminal.app with color bg | <img width="577" height="367"
    alt="Screenshot 2025-09-25 at 11 45 50 AM"
    src="https://github.com/user-attachments/assets/dd7c4b5b-342e-4028-8140-f4e65752bd0b"
    /> |
  • replace tui_markdown with a custom markdown renderer (#3396)
    Also, simplify the streaming behavior.
    
    This fixes a number of display issues with streaming markdown, and paves
    the way for better markdown features (e.g. customizable styles, syntax
    highlighting, markdown-aware wrapping).
    
    Not currently supported:
    - footnotes
    - tables
    - reference-style links
  • syntax-highlight bash lines (#3142)
    i'm not yet convinced i have the best heuristics for what to highlight,
    but this feels like a useful step towards something a bit easier to
    read, esp. when the model is producing large commands.
    
    <img width="669" height="589" alt="Screenshot 2025-09-03 at 8 21 56 PM"
    src="https://github.com/user-attachments/assets/b9cbcc43-80e8-4d41-93c8-daa74b84b331"
    />
    
    also a fairly significant refactor of our line wrapping logic.
  • tui: fix approval dialog for large commands (#3087)
    #### Summary
    - Emit a “Proposed Command” history cell when an ExecApprovalRequest
    arrives (parity with proposed patches).
    - Simplify the approval dialog: show only the reason/instructions; move
    the command preview into history.
    - Make approval/abort decision history concise:
      - Single line snippet; if multiline, show first line + " ...".
      - Truncate to 80 graphemes with ellipsis for very long commands.
    
    #### Details
    - History
    - Add `new_proposed_command` to render a header and indented command
    preview.
      - Use shared `prefix_lines` helper for first/subsequent line prefixes.
    - Approval UI
    - `UserApprovalWidget` no longer renders the command in the modal; shows
    optional `reason` text only.
      - Decision history renders an inline, dimmed snippet per rules above.
    - Tests (snapshot-based)
      - Proposed/decision flow for short command.
      - Proposed multi-line + aborted decision snippet with “ ...”.
      - Very long one-line command -> truncated snippet with “…”.
      - Updated existing exec approval snapshots and test reasons.
    
    <img width="1053" height="704" alt="Screenshot 2025-09-03 at 11 57
    35 AM"
    src="https://github.com/user-attachments/assets/9ed4c316-9daf-4ac1-80ff-7ae1f481dd10"
    />
    
    after approving:
    
    <img width="1053" height="704" alt="Screenshot 2025-09-03 at 11 58
    18 AM"
    src="https://github.com/user-attachments/assets/a44e243f-eb9d-42ea-87f4-171b3fb481e7"
    />
    
    rejection:
    
    <img width="1053" height="207" alt="Screenshot 2025-09-03 at 11 58
    45 AM"
    src="https://github.com/user-attachments/assets/a022664b-ae0e-4b70-a388-509208707934"
    />
    
    big command:
    
    
    https://github.com/user-attachments/assets/2dd976e5-799f-4af7-9682-a046e66cc494
  • Re-add markdown streaming (#2029)
    Wait for newlines, then render markdown on a line by line basis. Word wrap it for the current terminal size and then spit it out line by line into the UI. Also adds tests and fixes some UI regressions.