Commit Graph

18 Commits

  • feat(tui): render responsive Markdown tables in TUI (#22052)
    ## Why
    
    The TUI currently treats Markdown tables as ordinary wrapped text, which
    makes table-heavy responses hard to read and brittle across narrow panes
    and terminal resizes.
    
    This change teaches the TUI to render Markdown tables responsively while
    preserving the raw Markdown source needed to re-render streamed and
    finalized transcript content after width changes. The goal is to keep
    tables legible during streaming, after resize, and once a turn has
    finished, without corrupting scrollback ordering.
    
    ## What Changed
    
    - add table detection and responsive table rendering in the Markdown
    renderer
    - render standard tables with Unicode box-drawing borders when the pane
    is wide enough
    - add a vertical readability fallback for constrained or dense tables so
    narrow panes still show each row clearly
    - keep links and `<br>` content inside table cells instead of leaking
    text outside the table
    - avoid table normalization inside fenced or indented code blocks
    - preserve raw streamed Markdown source and keep the active table as a
    mutable tail until finalization
    - consolidate finalized streamed content into source-backed transcript
    cells so post-resize re-rendering stays correct
    - add snapshot and targeted streaming/resize regression coverage for the
    new table behavior
    
    ## How to Test
    
    1. Start Codex TUI from this branch.
    2. Paste this exact prompt:
    `This is a session to test codex, no need to do any thinking, just end
    different markdown tables, with columns exploring different markdown
    contents, like links, bold italic, code, etc. Make them different sizes,
    some 30+ rows, some not and intertwine them with some paragraphs with
    complex formatting as well.`
    3. Confirm the response includes several Markdown tables mixed with
    richly formatted paragraphs.
    4. Confirm wide-enough tables render with box-drawing borders instead of
    plain wrapped pipe text.
    5. Resize the terminal narrower while the answer is still streaming and
    confirm the in-progress table stays coherent instead of duplicating
    headers or leaving broken scrollback behind.
    6. Resize again after the turn finishes and confirm the finalized
    transcript re-renders cleanly at the new width.
    7. In a narrow pane, verify dense tables fall back to the vertical
    per-row layout instead of producing unreadable wrapped columns.
    8. Also verify pipe-heavy fenced code blocks still render as code, not
    as tables.
    
    Targeted tests:
    - `cargo test -p codex-tui table_readability_fallback --no-fail-fast`
    - `cargo test -p codex-tui markdown_render --no-fail-fast`
    - `cargo test -p codex-tui streaming::controller --no-fail-fast`
    - `cargo test -p codex-tui table_resize_lifecycle --no-fail-fast`
    
    ## Docs
    
    No developer docs update appears necessary.
  • Rename tui_app_server to tui (#16104)
    This is a follow-up to https://github.com/openai/codex/pull/15922. That
    previous PR deleted the old `tui` directory and left the new
    `tui_app_server` directory in place. This PR renames `tui_app_server` to
    `tui` and fixes up all references.
  • Remove the legacy TUI split (#15922)
    This is the part 1 of 2 PRs that will delete the `tui` /
    `tui_app_server` split. This part simply deletes the existing `tui`
    directory and marks the `tui_app_server` feature flag as removed. I left
    the `tui_app_server` feature flag in place for now so its presence
    doesn't result in an error. It is simply ignored.
    
    Part 2 will rename the `tui_app_server` directory `tui`. I did this as
    two parts to reduce visible code churn.
  • tui: drop citation rendering (#4855)
    We don't instruct the model to use citations, so it never emits them.
    Further, ratatui [doesn't currently support rendering links into the
    terminal with OSC 8](https://github.com/ratatui/ratatui/issues/1028), so
    even if we did parse citations, we can't correctly render them.
    
    So, remove all the code related to rendering them.
  • wrap markdown at render time (#4506)
    This results in correctly indenting list items with long lines.
    
    <img width="1006" height="251" alt="Screenshot 2025-09-30 at 10 00
    48 AM"
    src="https://github.com/user-attachments/assets/0a076cf6-ca3c-4efb-b3af-dc07617cdb6f"
    />
  • replace tui_markdown with a custom markdown renderer (#3396)
    Also, simplify the streaming behavior.
    
    This fixes a number of display issues with streaming markdown, and paves
    the way for better markdown features (e.g. customizable styles, syntax
    highlighting, markdown-aware wrapping).
    
    Not currently supported:
    - footnotes
    - tables
    - reference-style links
  • prefer ratatui Stylized for constructing lines/spans (#3068)
    no functional change, just simplifying ratatui styling and adding
    guidance in AGENTS.md for future.
  • fix (most) doubled lines and hanging list markers (#2789)
    This was mostly written by codex under heavy guidance via test cases
    drawn from logged session data and fuzzing. It also uncovered some bugs
    in tui_markdown, which will in some cases split a list marker from the
    list item content. We're not addressing those bugs for now.
  • Re-add markdown streaming (#2029)
    Wait for newlines, then render markdown on a line by line basis. Word wrap it for the current terminal size and then spit it out line by line into the UI. Also adds tests and fixes some UI regressions.
  • Streaming markdown (#1920)
    We wait until we have an entire newline, then format it with markdown and stream in to the UI. This reduces time to first token but is the right thing to do with our current rendering model IMO. Also lets us add word wrapping!
  • Stream model responses (#1810)
    Stream models thoughts and responses instead of waiting for the whole
    thing to come through. Very rough right now, but I'm making the risk call to push through.
  • chore: replace regex with regex-lite, where appropriate (#1200)
    As explained on https://crates.io/crates/regex-lite, `regex-lite` is a
    lighter alternative to `regex` and seems to be sufficient for our
    purposes.
  • chore: move types out of config.rs into config_types.rs (#1054)
    `config.rs` is already quite long without these definitions. Since they
    have no real dependencies of their own, let's move them to their own
    file so `config.rs` can focus on the business logic of loading a config.
  • fix: use text other than 'TODO' as test example (#969)
    I casually `rg TODO` to look for TODOs, so the use of TODO in a sample
    string in test output was throwing things off.
  • feat: add support for file_opener option in Rust, similiar to #911 (#957)
    This ports the enhancement introduced in
    https://github.com/openai/codex/pull/911 (and the fixes in
    https://github.com/openai/codex/pull/919) for the TypeScript CLI to the
    Rust one.
  • feat: introduce the use of tui-markdown (#851)
    This introduces the use of the `tui-markdown` crate to parse an
    assistant message as Markdown and style it using ANSI for a better user
    experience. As shown in the screenshot below, it has support for syntax
    highlighting for _tagged_ fenced code blocks:
    
    <img width="907" alt="image"
    src="https://github.com/user-attachments/assets/900dc229-80bb-46e8-b1bb-efee4c70ba3c"
    />
    
    That said, `tui-markdown` is not as configurable (or stylish!) as
    https://www.npmjs.com/package/marked-terminal, which is what we use in
    the TypeScript CLI. In particular:
    
    * The styles are hardcoded and `tui_markdown::from_str()` does not take
    any options whatsoever. It uses "bold white" for inline code style which
    does not stand out as much as the yellow used by `marked-terminal`:
    
    
    https://github.com/joshka/tui-markdown/blob/65402cbda70325f34e7ddf6fe1ec629bcd9459cf/tui-markdown/src/lib.rs#L464
    
    I asked Codex to take a first pass at this and it came up with:
    
    https://github.com/joshka/tui-markdown/pull/80
    
    * If a fenced code block is not tagged, then it does not get
    highlighted. I would rather add some logic here:
    
    
    https://github.com/joshka/tui-markdown/blob/65402cbda70325f34e7ddf6fe1ec629bcd9459cf/tui-markdown/src/lib.rs#L262
    
    that uses something like https://pypi.org/project/guesslang/ to examine
    the value of `text` and try to use the appropriate syntax highlighter.
    
    * When we have a fenced code block, we do not want to show the opening
    and closing triple backticks in the output.
    
    To unblock ourselves, we might want to bundle our own fork of
    `tui-markdown` temporarily until we figure out what the shape of the API
    should be and then try to upstream it.