mirror of
https://github.com/pchuan98/codex.git
synced 2026-07-01 00:31:56 +08:00
baddb5e68632d3248750305be39179dae6c049dc
25
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  Felipe CouryandGitHub
|
4ca2e436e5 |
fix(tui): linkify complete bare URLs with tildes (#27088)
## Background Bare URLs containing `~` in their path are currently only clickable up to the tilde in the interactive TUI. For example, Codex renders the visible text for: `https://www.cs.tufts.edu/~nr/cs257/archive/olin-shivers/dissertation.pdf` but the OSC 8 destination stops at `https://www.cs.tufts.edu/`. This makes Cmd-click open the wrong location even though the terminal recognizes the complete URL outside Codex. Fixes #26774. ## Root Cause The URL scanner already accepts `~`. The truncation happens earlier: with strikethrough parsing enabled, `pulldown-cmark` splits this URL into adjacent decoded `Event::Text` values around the tilde. The Markdown renderer annotated each text event independently, so only the first event still looked like a complete URL with a supported scheme. The renderer now merges adjacent decoded text events before URL annotation. It preserves the combined source range while retaining parser-decoded contents, which avoids regressing entities such as `&`. ## Changes - Add a small iterator that merges adjacent decoded Markdown text events and their source ranges. - Apply it at the Markdown renderer boundary before hyperlink detection. - Add regression coverage for the reported URL in prose, wrapped table output, and entity-decoded URLs. ## How to Test 1. Run Codex with `just c`. 2. Ask the assistant to output this exact bare URL with no Markdown link syntax: `https://www.cs.tufts.edu/~nr/cs257/archive/olin-shivers/dissertation.pdf` 3. Hold Cmd and hover or click the URL. 4. Confirm the complete URL, including the suffix after `~`, is one destination. 5. Repeat with the URL inside a Markdown table and confirm wrapped portions retain the same complete destination. Targeted tests: - `just test -p codex-tui url_with_tilde` - `just test -p codex-tui merged_text_events_preserve_entity_decoding` The full `codex-tui` test run was also executed. Its only failures were the two existing Guardian feature-flag tests: - `app::tests::update_feature_flags_disabling_guardian_clears_review_policy_and_restores_default` - `app::tests::update_feature_flags_disabling_guardian_clears_manual_review_policy_without_history` |
||
 sayan-oaiandGitHub
|
090144e0ec |
[codex] Fix hyperlink-aware key-value table rendering (#24825)
## Why The key/value markdown table renderer added in #24636 still operates on `Line` values, while table cells and rendered table output now carry `HyperlinkLine`. That mismatch breaks `codex-tui` compilation on `main` and would risk losing semantic web-link annotations if corrected by flattening the values. ## What changed - Make key/value record rendering wrap and emit `HyperlinkLine` values consistently with the existing grid renderer. - Remap wrapped hyperlink ranges and shift them when value content is prefixed by record-mode indentation or labels. - Add focused coverage verifying key/value fallback output preserves web-link destinations. ## Verification - `just test -p codex-tui -E 'test(key_value_table_keeps_web_annotations) | test(/table_renders_(key_value_records_when_compact_fragmentation_is_systemic_snapshot|stacked_key_value_records_when_path_column_becomes_too_narrow_snapshot|records_when_multiple_prose_columns_are_starved_snapshot)/)'` |
||
|
Felipe CouryandGitHub
|
26c9502121 |
feat(tui): render cramped markdown tables as key-value records [2 of 2] (#24636)
## Stack - **Base: #24489 [1 of 2]** - render markdown tables in app style. - **Current: #24636 [2 of 2]** - render cramped markdown tables as key/value records. Review this PR against `fcoury/app-style-markdown-tables`; it contains only the fallback behavior for cramped tables. ## Why The row-separated markdown table rendering in #24489 remains readable while columns have usable room. Once long links or multiple prose-heavy columns are compressed into narrow allocations, however, the grid can turn words and paths into tall vertical strips that are difficult to scan. In those cases the content matters more than preserving the grid shape. ## What Changed <table> <tr><td> <p align="center"><b> Normal </b></p> <img width="1722" height="619" alt="CleanShot 2026-05-27 at 14 32 57" src="https://github.com/user-attachments/assets/d04f5fbd-6064-4acd-91bd-072d19b983df" /> </td></tr> <tr><td> <p align="center"><b> Narrow </b></p> <img width="863" height="1013" alt="CleanShot 2026-05-27 at 14 33 12" src="https://github.com/user-attachments/assets/6a7d2968-0a68-48fd-ab5d-209b3dbaf03e" /> </td></tr> <tr><td> <p align="center"><b> Very narrow </b></p> <img width="435" height="746" alt="CleanShot 2026-05-27 at 14 33 47" src="https://github.com/user-attachments/assets/f6a59e30-b1d2-4063-9c05-43933abc77d6" /> </td></tr> </table> - Detect tables whose grid allocation causes systemic token fragmentation or starves multiple prose-heavy columns. - Render those tables as repeated key/value records instead of retaining an unreadable grid. - Use aligned label/value records when there is useful horizontal room, and switch to a stacked narrow-record layout where each label is followed by a full-width value when width is especially constrained. - Preserve the themed label color, rich inline formatting, links, and the existing grid presentation for tables that remain readable. - Add snapshot coverage for path-heavy narrow tables, prose-heavy issue tables, systemic compact fragmentation, and a control case that should continue to render as a grid. ## How to Test 1. Start Codex from this branch and render a normal multi-column markdown table at a comfortable terminal width. Confirm it still appears as the styled row-separated grid from #24489. 2. Render a table containing a long linked record identifier or file-like value, then narrow the terminal until the grid would split the value into vertical fragments. Confirm it switches to key/value records, with labels above values at very narrow widths. 3. Render a table with multiple prose-heavy columns, such as an issue summary table with `Issue`, `Activity`, `Complexity`, and `Why start`. Confirm a cramped width switches to records rather than wrapping several columns into hard-to-read strips. 4. Render a compact table where only one value wraps mildly. Confirm it stays in grid form rather than switching prematurely. ## Validation - Ran `just test -p codex-tui` while developing the fallback and reviewed/accepted the intended new markdown-render snapshots. The command still reports two unrelated existing guardian feature-flag test failures outside this diff. - Ran `just fix -p codex-tui` and `just fmt` after the Rust changes were complete. - `just argument-comment-lint` cannot reach source linting locally because Bazel fails while resolving LLVM sanitizer headers; touched positional literal callsites were inspected manually and annotated where needed. |
||
|
Felipe CouryandGitHub
|
7a26497836 |
feat(tui): add OSC 8 web links to rich content (#24472)
## Why Wrapped URLs in rich TUI output, especially URLs rendered inside Markdown tables, are split across terminal rows. In terminals that support OSC 8 hyperlinks, treating each visible fragment as part of the complete destination enables reliable open-link and copy-link actions even after table layout wraps the URL. This addresses the semantic-link portion of #12200 and the behavior described in https://github.com/openai/codex/issues/12200#issuecomment-4535452980. It does not change ordinary drag-selection across bordered table rows. ## What Changed - Added shared TUI OSC 8 support that validates `http://` and `https://` destinations, sanitizes terminal payloads, and applies metadata separately from visible line width/layout. - Added semantic web-link annotations to assistant and proposed-plan Markdown, including explicit web links and bare web URLs in prose and table cells while excluding code and non-web Markdown destinations. - Preserved complete URL targets through table wrapping, narrow pipe fallback, streaming, transcript overlay rendering, history insertion, and resize replay. - Routed intentional Codex-owned links in notices, status/setup/app-link, feedback, onboarding, MCP/plugin help, memories, and update surfaces through the shared hyperlink handling. ## How to Test 1. Run Codex in a terminal with OSC 8 link support, such as Ghostty, and request an assistant response containing a Markdown table whose last column contains a long `https://` URL. 2. Make the terminal narrow enough for the URL to wrap across multiple bordered table rows. 3. Use the terminal's open-link or copy-link action on more than one wrapped URL fragment and confirm each fragment resolves to the complete original URL. 4. Resize the terminal after the table is rendered and repeat the link action to confirm the destination survives scrollback replay. 5. Open the transcript overlay while rich output is present and confirm web links remain interactive there. 6. As a regression check, render inline/fenced code containing URL text and a Markdown link such as `[https://example.com](mailto:support@example.com)`; confirm these do not acquire a web OSC 8 destination. Targeted automated coverage exercised Markdown links and exclusions, wrapped and pipe-fallback tables, streaming/transcript overlay propagation, status-link truncation, and rendered word-wrapping cell alignment. `just test -p codex-tui` was also run; it passed the hyperlink coverage and reproduced two unrelated existing guardian feature-flag test failures. |
||
|
Felipe CouryandGitHub
|
6b4b15a1ed |
feat(tui): render markdown tables in app style [1 of 2] (#24489)
## Stack - **Current: #24489 [1 of 2]** - render markdown tables in app style. - **Stacked follow-up: #24636 [2 of 2]** - render cramped markdown tables as key/value records. ## Why Markdown tables currently render as boxed terminal grids, which gives ordinary assistant output a heavier visual treatment than surrounding rich text. This row-separated layout is the best match for how the App renders tables, while accented headers remain distinguishable even when a terminal font renders bold subtly. <table> <tr><td> <p align="center">Codex CLI - Before</p> <img width="1722" height="742" alt="CleanShot 2026-05-25 at 18 46 17" src="https://github.com/user-attachments/assets/f673d92a-ebd8-46e2-b414-3d985e41b6a4" /> </td></tr> <tr><td> <p align="center">Codex CLI - After</p> <img width="1720" height="957" alt="image" src="https://github.com/user-attachments/assets/36a3d331-bea1-439b-b5be-e97b0731bd6f" /> </td></tr> <tr><td> <p align="center">Codex App</p> <img width="979" height="1293" alt="CleanShot 2026-05-25 at 18 45 04" src="https://github.com/user-attachments/assets/7d97cae0-9256-4f6e-a4b3-8b8f22b0d901" /> </td></tr> </table> ## What Changed - Render markdown tables as padded, aligned rows without an enclosing box. - Style table headers with the active syntax-theme accent plus bold text, while keeping separators low contrast and theme-aware. - Use a segmented heavy header rule and thin body-row rules, preserving wrapping, narrow-width fallback, streaming parity, and rich-history rendering. - Update focused assertions and snapshots for the final table layout. ## How to Test 1. Render a markdown table in the TUI with several rows and columns. 2. Confirm the header uses the active theme accent, rows use one-character interior padding, and the table has no enclosing box. 3. Confirm the header is followed by segmented `━` rules and multiple body rows are separated by muted segmented `─` rules. 4. Render the same table while streaming and in history/raw-mode toggles; the final rich layout should remain stable. 5. Render a narrow table with long content and verify wrapping or pipe fallback does not overflow. ## Validation - `just test -p codex-tui table` - `just test -p codex-tui streaming::controller::tests` - `just argument-comment-lint-from-source -p codex-tui -- --all-targets` - `just fix -p codex-tui` - `just fmt` `just test -p codex-tui` was also run after accepting the snapshots; it fails only in the unrelated existing guardian app tests `update_feature_flags_disabling_guardian_clears_review_policy_and_restores_default` and `update_feature_flags_disabling_guardian_clears_manual_review_policy_without_history`. |
||
|
Felipe CouryandGitHub
|
14d80e55cd |
fix(tui): improve multiline markdown list readability (#24351)
## Why Numbered Markdown findings become hard to scan when long items visually run together or when wrapped explanatory paragraphs lose their list indentation. This is especially visible in review output: the next number can look attached to the previous finding, and paragraph continuation rows can jump back toward the left margin instead of staying grouped beneath their item. <table><tr><td> <center>Before</center> <img width="1718" height="836" alt="CleanShot 2026-05-24 at 14 00 49" src="https://github.com/user-attachments/assets/f1ee0023-50fa-4f81-a641-ae08b17b99bd" /> </td></tr> <tr><td> <center>After</center> <img width="1714" height="906" alt="image" src="https://github.com/user-attachments/assets/b123a5e0-a232-47bf-96d5-c935295f7c0a" /> </td></tr> </table> ## What Changed - Insert a blank separator before a sibling list item when the previous item occupies more than one rendered line. - Preserve compact rendering for lists whose sibling items each render on one line. - Preserve list-body leading whitespace when transient streamed assistant rows require another wrapping pass for history display, so wrapped paragraphs stay aligned beneath their item. - Share the existing leading-whitespace prefix logic used by history insertion instead of introducing a second indentation rule. - Keep streamed Markdown output aligned with completed rendering and add snapshots for findings-style spacing and streamed paragraph indentation. ## How to Test 1. Start Codex from this branch and open the recorded repro session `019e563f-7d58-7ff2-8ec7-828f20fa61ca`. 2. Inspect the numbered `Findings` list whose items contain explanatory paragraphs. 3. Confirm each multiline finding is separated from the next numbered finding by one blank line. 4. Confirm wrapped rows of each indented paragraph remain aligned beneath the finding body, rather than returning to the left edge. 5. Render a short one-line numbered or unordered list and confirm its items remain compact without added blank rows. Targeted tests: - `just test -p codex-tui history_cell insert_history markdown_render markdown_stream streaming::controller` - `just argument-comment-lint-from-source -p codex-tui` ## Related Work PR #24346 changes Markdown table column allocation in parallel. This PR is intentionally limited to list-item readability and history wrapping; both branches touch `codex-rs/tui/src/markdown_render.rs`, so a small merge conflict may need resolution depending on merge order. |
||
|
Felipe CouryandGitHub
|
20d1b7674d |
fix(tui): improve markdown table column allocation (#24346)
## Why Markdown tables with a long path-heavy column could allocate almost all available width to that column and collapse neighboring prose columns to only a few characters. In rollout summaries this made `Unit` and `What It Adds` difficult to read, even though the long `Files` values were the content best suited to wrapping. The affected example also specified `Files` as right aligned in its markdown delimiter (`---:`). This change preserves that requested alignment while improving how width is distributed. | Before | After | |---|---| | <img width="1709" height="764" alt="image" src="https://github.com/user-attachments/assets/932ab21c-b72d-48a2-9aad-b69da87a0968" /> | <img width="1711" height="855" alt="image" src="https://github.com/user-attachments/assets/4028bd20-2228-4c2f-be8a-1866325b7f62" /> | ## What Changed - Classify table columns as narrative, token-heavy, or compact during width allocation. - Shrink token-heavy path and URL columns before shrinking narrative prose, while preserving compact counts and short labels longest. - Use readable soft floors for narrative and token-heavy content before falling back to tighter layouts. - Add snapshot coverage for a rollout-shaped table containing right-aligned file paths and prose columns. ## How to Test 1. Render a markdown table with `Unit`, right-aligned `Files`, `Adds`, `Removes`, and `What It Adds` columns at a constrained terminal width. 2. Put long repository paths in `Files` and sentence-length content in `Unit` and `What It Adds`. 3. Confirm that `Files` remains right aligned but wraps before the narrative columns become unreadable. 4. Confirm that the compact numeric columns remain easy to scan. Targeted tests: - `just test -p codex-tui markdown_render` Validation note: `just test -p codex-tui` was also attempted and reached two existing unrelated failures in `app::tests::update_feature_flags_disabling_guardian_*`; the markdown rendering regression test passes in the targeted run. |
||
|
Felipe CouryandGitHub
|
5248e3da2b |
feat(tui): render responsive Markdown tables in TUI (#22052)
## Why The TUI currently treats Markdown tables as ordinary wrapped text, which makes table-heavy responses hard to read and brittle across narrow panes and terminal resizes. This change teaches the TUI to render Markdown tables responsively while preserving the raw Markdown source needed to re-render streamed and finalized transcript content after width changes. The goal is to keep tables legible during streaming, after resize, and once a turn has finished, without corrupting scrollback ordering. ## What Changed - add table detection and responsive table rendering in the Markdown renderer - render standard tables with Unicode box-drawing borders when the pane is wide enough - add a vertical readability fallback for constrained or dense tables so narrow panes still show each row clearly - keep links and `<br>` content inside table cells instead of leaking text outside the table - avoid table normalization inside fenced or indented code blocks - preserve raw streamed Markdown source and keep the active table as a mutable tail until finalization - consolidate finalized streamed content into source-backed transcript cells so post-resize re-rendering stays correct - add snapshot and targeted streaming/resize regression coverage for the new table behavior ## How to Test 1. Start Codex TUI from this branch. 2. Paste this exact prompt: `This is a session to test codex, no need to do any thinking, just end different markdown tables, with columns exploring different markdown contents, like links, bold italic, code, etc. Make them different sizes, some 30+ rows, some not and intertwine them with some paragraphs with complex formatting as well.` 3. Confirm the response includes several Markdown tables mixed with richly formatted paragraphs. 4. Confirm wide-enough tables render with box-drawing borders instead of plain wrapped pipe text. 5. Resize the terminal narrower while the answer is still streaming and confirm the in-progress table stays coherent instead of duplicating headers or leaving broken scrollback behind. 6. Resize again after the turn finishes and confirm the finalized transcript re-renders cleanly at the new width. 7. In a narrow pane, verify dense tables fall back to the vertical per-row layout instead of producing unreadable wrapped columns. 8. Also verify pipe-heavy fenced code blocks still render as code, not as tables. Targeted tests: - `cargo test -p codex-tui table_readability_fallback --no-fail-fast` - `cargo test -p codex-tui markdown_render --no-fail-fast` - `cargo test -p codex-tui streaming::controller --no-fail-fast` - `cargo test -p codex-tui table_resize_lifecycle --no-fail-fast` ## Docs No developer docs update appears necessary. |
||
 Eric TrautandGitHub
|
52c06b8759 |
Preserve TUI markdown list spacing after code blocks (#19706)
## Why Fixes #19702. The TUI markdown renderer could visually attach the next list marker to a fenced code block inside the previous list item, even when the source markdown included a blank line before the next item. That made block-heavy loose lists harder to read, while the desired behavior is still to keep simple lists compact. ## What changed - Track whether the current rendered list item contains a code block. - Preserve one blank separator before the following list marker only when the previous item contained a code block. - Add regression coverage for both paths: code-block list items keep the separator, and simple loose list items stay compact. ## Verification - `cargo test -p codex-tui markdown_render` I also manually verified that the bug exists before and is fixed after. ## Before <img width="437" height="240" alt="Screenshot 2026-04-26 at 1 19 01 PM" src="https://github.com/user-attachments/assets/3bc9d64d-2dba-40d9-9d6b-a1d0b3c0f728" /> ## After <img width="410" height="269" alt="Screenshot 2026-04-26 at 1 18 54 PM" src="https://github.com/user-attachments/assets/19c15bee-da32-455e-a7cb-e05eb85f4ea0" /> |
||
|
Eric TrautandGitHub
|
54dbbb839e |
(tui): Decode percent-escaped bare local file links (#16810)
Addresses #16622 Problem: bare local file links in TUI markdown render percent-encoded path bytes literally, unlike file:// links. Solution: decode bare path targets before local-path expansion and add regression coverage for spaces and Unicode. |
||
|
Eric TrautandGitHub
|
61429a6c10 |
Rename tui_app_server to tui (#16104)
This is a follow-up to https://github.com/openai/codex/pull/15922. That previous PR deleted the old `tui` directory and left the new `tui_app_server` directory in place. This PR renames `tui_app_server` to `tui` and fixes up all references. |
||
|
Eric TrautandGitHub
|
d65deec617 |
Remove the legacy TUI split (#15922)
This is the part 1 of 2 PRs that will delete the `tui` / `tui_app_server` split. This part simply deletes the existing `tui` directory and marks the `tui_app_server` feature flag as removed. I left the `tui_app_server` feature flag in place for now so its presence doesn't result in an error. It is simply ignored. Part 2 will rename the `tui_app_server` directory `tui`. I did this as two parts to reduce visible code churn. |
||
 Michael BolinandGitHub
|
b77fe8fefe |
Apply argument comment lint across codex-rs (#14652)
## Why Once the repo-local lint exists, `codex-rs` needs to follow the checked-in convention and CI needs to keep it from drifting. This commit applies the fallback `/*param*/` style consistently across existing positional literal call sites without changing those APIs. The longer-term preference is still to avoid APIs that require comments by choosing clearer parameter types and call shapes. This PR is intentionally the mechanical follow-through for the places where the existing signatures stay in place. After rebasing onto newer `main`, the rollout also had to cover newly introduced `tui_app_server` call sites. That made it clear the first cut of the CI job was too expensive for the common path: it was spending almost as much time installing `cargo-dylint` and re-testing the lint crate as a representative test job spends running product tests. The CI update keeps the full workspace enforcement but trims that extra overhead from ordinary `codex-rs` PRs. ## What changed - keep a dedicated `argument_comment_lint` job in `rust-ci` - mechanically annotate remaining opaque positional literals across `codex-rs` with exact `/*param*/` comments, including the rebased `tui_app_server` call sites that now fall under the lint - keep the checked-in style aligned with the lint policy by using `/*param*/` and leaving string and char literals uncommented - cache `cargo-dylint`, `dylint-link`, and the relevant Cargo registry/git metadata in the lint job - split changed-path detection so the lint crate's own `cargo test` step runs only when `tools/argument-comment-lint/*` or `rust-ci.yml` changes - continue to run the repo wrapper over the `codex-rs` workspace, so product-code enforcement is unchanged Most of the code changes in this commit are intentionally mechanical comment rewrites or insertions driven by the lint itself. ## Verification - `./tools/argument-comment-lint/run.sh --workspace` - `cargo test -p codex-tui-app-server -p codex-tui` - parsed `.github/workflows/rust-ci.yml` locally with PyYAML --- * -> #14652 * #14651 |
||
 
|
da74da6684 |
render local file links from target paths (#13857)
Co-authored-by: Josh McKinney <joshka@openai.com> |
||
|
pash-openaiandGitHub
|
6acede5a28 |
tui: restore visible line numbers for hidden file links (#12870)
we recently changed file linking so the model uses markdown links when it wants something to be clickable. This works well across the GUI surfaces because they can render markdown cleanly and use the full absolute path in the anchor target. A previous pass hid the absolute path in the TUI (and only showed the label), but that also meant we could lose useful location info when the model put the line number or range in the anchor target instead of the label. This follow-up keeps the TUI behavior simple while making local file links feel closer to the old TUI file reference style. key changes: - Local markdown file links in the TUI keep the old file-ref feel: code styling, no underline, no visible absolute path. - If the hidden local anchor target includes a location suffix and the label does not already include one, we append that suffix to the visible label. - This works for single lines, line/column references, and ranges. - If the label already includes the location, we leave it alone. - normal web links keep the old TUI markdown-link behavior some examples: - `[foo.rs](/abs/path/foo.rs)` renders as `foo.rs` - `[foo.rs](/abs/path/foo.rs:45)` renders as `foo.rs:45` - `[foo.rs](/abs/path/foo.rs:45:3-48:9)` renders as `foo.rs:45:3-48:9` - `[foo.rs:45](/abs/path/foo.rs:45)` stays `foo.rs:45` - `[docs](https://example.com/docs)` still renders like a normal web link how it looks: <img width="732" height="813" alt="Screenshot 2026-02-26 at 9 27 55 AM" src="https://github.com/user-attachments/assets/d51bf236-653a-4e83-96e4-9427f0804471" /> |
||
|
pash-openaiandGitHub
|
9a96b6f509 |
Hide local file link destinations in TUI markdown (#12705)
## Summary - hide appended destinations for local path-style markdown links in the TUI renderer - keep web links rendering with their visible destination and style link labels consistently - add markdown renderer tests and a snapshot for the new file-link output ## Testing - just fmt - cargo test -p codex-tui <img width="1120" height="968" alt="image" src="https://github.com/user-attachments/assets/490e8eda-ae47-4231-89fa-b254a1f83eed" /> |
||
 Felipe CouryandGitHub
|
c4f1af7a86 |
feat(tui): syntax highlighting via syntect with theme picker (#11447)
## Summary Adds syntax highlighting to the TUI for fenced code blocks in markdown responses and file diffs, plus a `/theme` command with live preview and persistent theme selection. Uses syntect (~250 grammars, 32 bundled themes, ~1 MB binary cost) — the same engine behind `bat`, `delta`, and `xi-editor`. Includes guardrails for large inputs, graceful fallback to plain text, and SSH-aware clipboard integration for the `/copy` command. <img width="1554" height="1014" alt="image" src="https://github.com/user-attachments/assets/38737a79-8717-4715-b857-94cf1ba59b85" /> <img width="2354" height="1374" alt="image" src="https://github.com/user-attachments/assets/25d30a00-c487-4af8-9cb6-63b0695a4be7" /> ## Problem Code blocks in the TUI (markdown responses and file diffs) render without syntax highlighting, making it hard to scan code at a glance. Users also have no way to pick a color theme that matches their terminal aesthetic. ## Mental model The highlighting system has three layers: 1. **Syntax engine** (`render::highlight`) -- a thin wrapper around syntect + two-face. It owns a process-global `SyntaxSet` (~250 grammars) and a `RwLock<Theme>` that can be swapped at runtime. All public entry points accept `(code, lang)` and return ratatui `Span`/`Line` vectors or `None` when the language is unrecognized or the input exceeds safety guardrails. 2. **Rendering consumers** -- `markdown_render` feeds fenced code blocks through the engine; `diff_render` highlights Add/Delete content as a whole file and Update hunks per-hunk (preserving parser state across hunk lines). Both callers fall back to plain unstyled text when the engine returns `None`. 3. **Theme lifecycle** -- at startup the config's `tui.theme` is resolved to a syntect `Theme` via `set_theme_override`. At runtime the `/theme` picker calls `set_syntax_theme` to swap themes live; on cancel it restores the snapshot taken at open. On confirm it persists `[tui] theme = "..."` to config.toml. ## Non-goals - Inline diff highlighting (word-level change detection within a line). - Semantic / LSP-backed highlighting. - Theme authoring tooling; users supply standard `.tmTheme` files. ## Tradeoffs | Decision | Upside | Downside | | ------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------- | | syntect over tree-sitter / arborium | ~1 MB binary increase for ~250 grammars + 32 themes; battle-tested crate powering widely-used tools (`bat`, `delta`, `xi-editor`). tree-sitter would add ~12 MB for 20-30 languages or ~35 MB for full coverage. | Regex-based; less structurally accurate than tree-sitter for some languages (e.g. language injections like JS-in-HTML). | | Global `RwLock<Theme>` | Enables live `/theme` preview without threading Theme through every call site | Lock contention risk (mitigated: reads vastly outnumber writes, single UI thread) | | Skip background / italic / underline from themes | Terminal BG preserved, avoids ugly rendering on some themes | Themes that rely on these properties lose fidelity | | Guardrails: 512 KB / 10k lines | Prevents pathological stalls on huge diffs or pastes | Very large files render without color | ## Architecture ``` config.toml ─[tui.theme]─> set_theme_override() ─> THEME (RwLock) │ ┌───────────────────────────────────────────┘ │ markdown_render ─── highlight_code_to_lines(code, lang) ─> Vec<Line> diff_render ─── highlight_code_to_styled_spans(code, lang) ─> Option<Vec<Vec<Span>>> │ │ (None ⇒ plain text fallback) │ /theme picker ─── set_syntax_theme(theme) // live preview swap ─── current_syntax_theme() // snapshot for cancel ─── resolve_theme_by_name(name) // lookup by kebab-case ``` Key files: - `tui/src/render/highlight.rs` -- engine, theme management, guardrails - `tui/src/diff_render.rs` -- syntax-aware diff line wrapping - `tui/src/theme_picker.rs` -- `/theme` command builder - `tui/src/bottom_pane/list_selection_view.rs` -- side content panel, callbacks - `core/src/config/types.rs` -- `Tui::theme` field - `core/src/config/edit.rs` -- `syntax_theme_edit()` helper ## Observability - `tracing::warn` when a configured theme name cannot be resolved. - `Config::startup_warnings` surfaces the same message as a TUI banner. - `tracing::error` when persisting theme selection fails. ## Tests - Unit tests in `highlight.rs`: language coverage, fallback behavior, CRLF stripping, style conversion, guardrail enforcement, theme name mapping exhaustiveness. - Unit tests in `diff_render.rs`: snapshot gallery at multiple terminal sizes (80x24, 94x35, 120x40), syntax-highlighted wrapping, large-diff guardrail, rename-to-different-extension highlighting, parser state preservation across hunk lines. - Unit tests in `theme_picker.rs`: preview rendering (wide + narrow), dim overlay on deletions, subtitle truncation, cancel-restore, fallback for unavailable configured theme. - Unit tests in `list_selection_view.rs`: side layout geometry, stacked fallback, buffer clearing, cancel/selection-changed callbacks. - Integration test in `lib.rs`: theme warning uses the final (post-resume) config. ## Cargo Deny: Unmaintained Dependency Exceptions This PR adds two `cargo deny` advisory exceptions for transitive dependencies pulled in by `syntect v5.3.0`: | Advisory | Crate | Status | |----------|-------|--------| | RUSTSEC-2024-0320 | `yaml-rust` | Unmaintained (maintainer unreachable) | | RUSTSEC-2025-0141 | `bincode` | Unmaintained (development ceased; v1.3.3 considered complete) | **Why this is safe in our usage:** - Neither advisory describes a known security vulnerability. Both are "unmaintained" notices only. - `bincode` is used by syntect to deserialize pre-compiled syntax sets. Again, these are **static vendored artifacts** baked into the binary at build time. No user-supplied bincode data is ever deserialized. - Attack surface is zero for both crates; exploitation would require a supply-chain compromise of our own build artifacts. - These exceptions can be removed when syntect migrates to `yaml-rust2` and drops `bincode`, or when alternative crates are available upstream. |
||
|
Felipe CouryandGitHub
|
2ba2c57af4 |
fix(tui): preserve URL clickability across all TUI views (#12067)
## Problem Long URLs containing `/` and `-` characters are split across multiple terminal lines by `textwrap`'s default hyphenation rules. This breaks terminal link detection: emulators can no longer identify the URL as clickable, and copy-paste yields a truncated fragment. The issue affects every view that renders user or agent text — exec output, history cells, markdown, the app-link setup screen, and the VT100 scrollback path. A secondary bug compounds the first: `desired_height()` calculations count logical lines rather than viewport rows. When a URL overflows its line and wraps visually, the height budget is too small, causing content to clip or leave gaps. Here is how the complete URL is interpreted by the terminal before (first line only) and after (complete URL): | Before | After | |---|---| | <img width="777" height="1002" alt="Screenshot 2026-02-17 at 7 59 11 PM" src="https://github.com/user-attachments/assets/193a89a0-7e56-49c5-8b76-53499a76e7e3" /> | <img width="777" height="1002" alt="Screenshot 2026-02-17 at 7 58 40 PM" src="https://github.com/user-attachments/assets/0b9b4c14-aafb-439f-9ffe-f6bba556f95e" /> | ## Mental model The TUI now treats URL-like tokens as atomic units that must never be split by the wrapping engine. Every call site that previously used `word_wrap_*` has been migrated to `adaptive_wrap_*`, which inspects each line for URL-like tokens and switches wrapping strategy accordingly: - **Non-URL lines** follow the existing `textwrap` path unchanged (word boundaries, optional indentation, hyphenation). - **URL-only lines** (with at most decorative markers like `│`, `-`, `1.`) are emitted unwrapped so terminal link detection works; ratatui's `Wrap { trim: false }` handles the final character wrap at render time. - **Mixed lines** (URL + substantive non-URL prose) flow through `adaptive_wrap_line` so prose wraps naturally at word boundaries while URL tokens remain unsplit. Height measurement everywhere now delegates to `Paragraph::line_count(width)`, which accounts for the visual row cost of overflowed lines. This single source of truth replaces ad-hoc line counting in individual cells. For terminal scrollback (the VT100 path that prints history when the TUI exits), URL-only lines are emitted unwrapped so the terminal's own link detector can find them. Mixed URL+prose lines use adaptive wrapping so surrounding text wraps naturally. Continuation rows are pre-cleared to avoid stale content artifacts. ## Non-goals - Full RFC 3986 URL parsing. The detector is a conservative heuristic that covers `scheme://host`, bare domains (`example.com/path`), `localhost:port`, and IPv4 hosts. IPv6 (`[::1]:8080`) and exotic schemes are intentionally excluded from v1. - Changing wrapping behavior for non-URL content. - Reflowing or reformatting existing terminal scrollback on resize. ## Tradeoffs | Decision | Upside | Downside | |----------|--------|----------| | Heuristic URL detection vs. full parser | Fast, zero-alloc on the hot path; conservative enough to reject file paths like `src/main.rs` | False negatives on obscure URL formats (they get split as before) | | Adaptive (three-path) wrapping | Non-URL lines are untouched — no behavior change, no perf cost; mixed lines wrap prose naturally while preserving URLs | Three wrapping strategies to reason about when debugging layout | | Row-based truncation with line-unit ellipsis | Accurate viewport budget; stable "N lines omitted" count across terminal widths | `truncate_lines_middle` is more complex (must compute per-line row cost) | | Unwrapped URL-only lines in scrollback | Terminal emulators detect clickable links; copy-paste gets the full URL | TUI and scrollback formatting diverge for URL-only lines | | Default `desired_height` via `Paragraph::line_count` | DRY — most cells inherit correct measurement | Cells with custom layout must remember to override | ## Architecture ```mermaid flowchart TD A["adaptive_wrap_*()"] --> B{"line_contains_url_like?"} B -- No URL tokens --> C["word_wrap_line<br/>(textwrap default)"] B -- Has URL tokens --> D{"mixed URL + prose?"} D -- "URL-only<br/>(+ decorative markers)" --> E["emit unwrapped<br/>(terminal char-wraps)"] D -- "Mixed<br/>(URL + substantive text)" --> F["adaptive_wrap_line<br/>(AsciiSpace + custom WordSplitter)"] C --> G["Paragraph::line_count(w)<br/>(single height truth)"] E --> G F --> G ``` **Changed files:** | File | Role | |------|------| | `wrapping.rs` | URL detection heuristics, mixed-line detection, `adaptive_wrap_*` functions, custom `WordSplitter` | | `exec_cell/render.rs` | Row-aware `truncate_lines_middle`, adaptive wrapping for command/output display | | `history_cell.rs` | Migrate all cell types to `adaptive_wrap_*`; default `desired_height` via `Paragraph::line_count` | | `insert_history.rs` | Three-path scrollback wrapping (unwrapped URL-only, adaptive mixed, word-wrapped text); continuation row clearing | | `app_link_view.rs` | Adaptive wrapping for setup URL; `desired_height` via `Paragraph::line_count` | | `markdown_render.rs` | Adaptive wrapping in `finish_paragraph` | | `model_migration.rs` | Viewport-aware wrapping for narrow-pane markdown | | `pager_overlay.rs` | `Wrap { trim: false }` for transcript and streaming chunks | | `queued_user_messages.rs` | Migrate to `adaptive_wrap_lines` | | `status/card.rs` | Migrate to `adaptive_wrap_lines` | ## Observability - **Ellipsis message** in truncated exec output reports omitted count in logical lines (stable across resize) rather than viewport rows (fluctuates). - URL detection is deterministic and stateless — no hidden caching or memoization to go stale. - Height mismatch bugs surface immediately as visual clipping or gaps; the `Paragraph::line_count` path is the same code ratatui uses at render time, so measurement and rendering cannot diverge. ## Tests 26 new unit tests across 7 files, covering: - **URL integrity**: assert a URL-like token appears on exactly one rendered line (not split across two). - **Height accuracy**: compare `desired_height()` against `Paragraph::line_count()` for URL-containing content. - **Row-aware truncation**: verify ellipsis counts logical lines and output fits within the row budget. - **Scrollback rendering**: VT100 backend tests confirm prefix and URL land on the same row; continuation rows are cleared; mixed URL+prose lines wrap prose while preserving URL tokens. - **Mixed URL+prose detection**: `line_has_mixed_url_and_non_url_tokens` correctly distinguishes lines with substantive non-URL text from lines with only decorative markers alongside a URL. - **Heuristic correctness**: positive matches (`https://...`, `example.com/path`, `localhost:3000/api`, `192.168.1.1:8080/health`) and negative matches (`src/main.rs`, `foo/bar`, `hello-world`). ## Risks and open items 1. **URL-like tokens in code output** (e.g. `example.com/api` inside a JSON blob) will trigger URL-preserving wrap on that line. This is acceptable — the worst case is a slightly wider line, not broken output. 2. **Very long non-URL tokens on a URL line** can only break at character boundaries (the custom splitter emits all char indices for non-URL words). On extremely narrow terminals this could overflow, but narrow terminals already degrade gracefully. 3. **No IPv6 support** — `[::1]:8080/path` will be treated as a non-URL and may get split. Can be added later without API changes. Fixes #5457 |
||
|
Eric TrautandGitHub
|
1ad261d681 |
Changed default wrap algorithm from OptimalFit to FirstFit (#7960)
Codex identified this as the cause of a reported hang: https://github.com/openai/codex/issues/7822. Apparently, the wrapping algorithm we're using has known issues and bad worst-case behaviors when OptimalFit is used on certain strings. It recommended switching to FirstFit instead. |
||
 Jeremy RoseandGitHub
|
2c793083f4 |
tui: centralize markdown styling and make inline code cyan (#7023)
<img width="762" height="271" alt="Screenshot 2025-11-20 at 12 54 06 PM" src="https://github.com/user-attachments/assets/10021d63-27eb-407b-8fcc-43740e3bfb0f" /> |
||
|
Jeremy RoseandGitHub
|
39a2446716 |
tui: drop citation rendering (#4855)
We don't instruct the model to use citations, so it never emits them. Further, ratatui [doesn't currently support rendering links into the terminal with OSC 8](https://github.com/ratatui/ratatui/issues/1028), so even if we did parse citations, we can't correctly render them. So, remove all the code related to rendering them. |
||
|
Jeremy RoseandGitHub
|
01e6503672 |
wrap markdown at render time (#4506)
This results in correctly indenting list items with long lines. <img width="1006" height="251" alt="Screenshot 2025-09-30 at 10 00 48 AM" src="https://github.com/user-attachments/assets/0a076cf6-ca3c-4efb-b3af-dc07617cdb6f" /> |
||
 easong-openaiandGitHub
|
5b038135de |
Add cloud tasks (#3197)
Adds a TUI for managing, applying, and creating cloud tasks |
||
 jif-oaiandGitHub
|
be366a31ab |
chore: clippy on redundant closure (#4058)
Add redundant closure clippy rules and let Codex fix it by minimising FQP |
||
|
Jeremy RoseandGitHub
|
8068cc75f8 |
replace tui_markdown with a custom markdown renderer (#3396)
Also, simplify the streaming behavior. This fixes a number of display issues with streaming markdown, and paves the way for better markdown features (e.g. customizable styles, syntax highlighting, markdown-aware wrapping). Not currently supported: - footnotes - tables - reference-style links |