20 Commits

  • feat(tui): render markdown tables in app style [1 of 2] (#24489)
    ## Stack
    
    - **Current: #24489 [1 of 2]** - render markdown tables in app style.
    - **Stacked follow-up: #24636 [2 of 2]** - render cramped markdown
    tables as key/value records.
    
    ## Why
    
    Markdown tables currently render as boxed terminal grids, which gives
    ordinary assistant output a heavier visual treatment than surrounding
    rich text. This row-separated layout is the best match for how the App
    renders tables, while accented headers remain distinguishable even when
    a terminal font renders bold subtly.
    
    <table>
    <tr><td>
    <p align="center">Codex CLI - Before</p>
    <img width="1722" height="742" alt="CleanShot 2026-05-25 at 18 46 17"
    src="https://github.com/user-attachments/assets/f673d92a-ebd8-46e2-b414-3d985e41b6a4"
    />
    </td></tr>
    <tr><td>
    <p align="center">Codex CLI - After</p>
    <img width="1720" height="957" alt="image"
    src="https://github.com/user-attachments/assets/36a3d331-bea1-439b-b5be-e97b0731bd6f"
    />
    </td></tr>
    <tr><td>
    <p align="center">Codex App</p>
    <img width="979" height="1293" alt="CleanShot 2026-05-25 at 18 45 04"
    src="https://github.com/user-attachments/assets/7d97cae0-9256-4f6e-a4b3-8b8f22b0d901"
    />
    </td></tr>
    </table>
    
    ## What Changed
    
    - Render markdown tables as padded, aligned rows without an enclosing
    box.
    - Style table headers with the active syntax-theme accent plus bold
    text, while keeping separators low contrast and theme-aware.
    - Use a segmented heavy header rule and thin body-row rules, preserving
    wrapping, narrow-width fallback, streaming parity, and rich-history
    rendering.
    - Update focused assertions and snapshots for the final table layout.
    
    ## How to Test
    
    1. Render a markdown table in the TUI with several rows and columns.
    2. Confirm the header uses the active theme accent, rows use
    one-character interior padding, and the table has no enclosing box.
    3. Confirm the header is followed by segmented `━` rules and multiple
    body rows are separated by muted segmented `─` rules.
    4. Render the same table while streaming and in history/raw-mode
    toggles; the final rich layout should remain stable.
    5. Render a narrow table with long content and verify wrapping or pipe
    fallback does not overflow.
    
    ## Validation
    
    - `just test -p codex-tui table`
    - `just test -p codex-tui streaming::controller::tests`
    - `just argument-comment-lint-from-source -p codex-tui -- --all-targets`
    - `just fix -p codex-tui`
    - `just fmt`
    
    `just test -p codex-tui` was also run after accepting the snapshots; it
    fails only in the unrelated existing guardian app tests
    `update_feature_flags_disabling_guardian_clears_review_policy_and_restores_default`
    and
    `update_feature_flags_disabling_guardian_clears_manual_review_policy_without_history`.
  • fix(tui): improve multiline markdown list readability (#24351)
    ## Why
    
    Numbered Markdown findings become hard to scan when long items visually
    run together or when wrapped explanatory paragraphs lose their list
    indentation. This is especially visible in review output: the next
    number can look attached to the previous finding, and paragraph
    continuation rows can jump back toward the left margin instead of
    staying grouped beneath their item.
    
    <table><tr><td>
    <center>Before</center>
    <img width="1718" height="836" alt="CleanShot 2026-05-24 at 14 00 49"
    src="https://github.com/user-attachments/assets/f1ee0023-50fa-4f81-a641-ae08b17b99bd"
    />
    </td></tr>
    <tr><td> 
    <center>After</center>
    <img width="1714" height="906" alt="image"
    src="https://github.com/user-attachments/assets/b123a5e0-a232-47bf-96d5-c935295f7c0a"
    />
    </td></tr>
    </table>
    
    ## What Changed
    
    - Insert a blank separator before a sibling list item when the previous
    item occupies more than one rendered line.
    - Preserve compact rendering for lists whose sibling items each render
    on one line.
    - Preserve list-body leading whitespace when transient streamed
    assistant rows require another wrapping pass for history display, so
    wrapped paragraphs stay aligned beneath their item.
    - Share the existing leading-whitespace prefix logic used by history
    insertion instead of introducing a second indentation rule.
    - Keep streamed Markdown output aligned with completed rendering and add
    snapshots for findings-style spacing and streamed paragraph indentation.
    
    ## How to Test
    
    1. Start Codex from this branch and open the recorded repro session
    `019e563f-7d58-7ff2-8ec7-828f20fa61ca`.
    2. Inspect the numbered `Findings` list whose items contain explanatory
    paragraphs.
    3. Confirm each multiline finding is separated from the next numbered
    finding by one blank line.
    4. Confirm wrapped rows of each indented paragraph remain aligned
    beneath the finding body, rather than returning to the left edge.
    5. Render a short one-line numbered or unordered list and confirm its
    items remain compact without added blank rows.
    
    Targeted tests:
    
    - `just test -p codex-tui history_cell insert_history markdown_render
    markdown_stream streaming::controller`
    - `just argument-comment-lint-from-source -p codex-tui`
    
    ## Related Work
    
    PR #24346 changes Markdown table column allocation in parallel. This PR
    is intentionally limited to list-item readability and history wrapping;
    both branches touch `codex-rs/tui/src/markdown_render.rs`, so a small
    merge conflict may need resolution depending on merge order.
  • feat(tui): render responsive Markdown tables in TUI (#22052)
    ## Why
    
    The TUI currently treats Markdown tables as ordinary wrapped text, which
    makes table-heavy responses hard to read and brittle across narrow panes
    and terminal resizes.
    
    This change teaches the TUI to render Markdown tables responsively while
    preserving the raw Markdown source needed to re-render streamed and
    finalized transcript content after width changes. The goal is to keep
    tables legible during streaming, after resize, and once a turn has
    finished, without corrupting scrollback ordering.
    
    ## What Changed
    
    - add table detection and responsive table rendering in the Markdown
    renderer
    - render standard tables with Unicode box-drawing borders when the pane
    is wide enough
    - add a vertical readability fallback for constrained or dense tables so
    narrow panes still show each row clearly
    - keep links and `<br>` content inside table cells instead of leaking
    text outside the table
    - avoid table normalization inside fenced or indented code blocks
    - preserve raw streamed Markdown source and keep the active table as a
    mutable tail until finalization
    - consolidate finalized streamed content into source-backed transcript
    cells so post-resize re-rendering stays correct
    - add snapshot and targeted streaming/resize regression coverage for the
    new table behavior
    
    ## How to Test
    
    1. Start Codex TUI from this branch.
    2. Paste this exact prompt:
    `This is a session to test codex, no need to do any thinking, just end
    different markdown tables, with columns exploring different markdown
    contents, like links, bold italic, code, etc. Make them different sizes,
    some 30+ rows, some not and intertwine them with some paragraphs with
    complex formatting as well.`
    3. Confirm the response includes several Markdown tables mixed with
    richly formatted paragraphs.
    4. Confirm wide-enough tables render with box-drawing borders instead of
    plain wrapped pipe text.
    5. Resize the terminal narrower while the answer is still streaming and
    confirm the in-progress table stays coherent instead of duplicating
    headers or leaving broken scrollback behind.
    6. Resize again after the turn finishes and confirm the finalized
    transcript re-renders cleanly at the new width.
    7. In a narrow pane, verify dense tables fall back to the vertical
    per-row layout instead of producing unreadable wrapped columns.
    8. Also verify pipe-heavy fenced code blocks still render as code, not
    as tables.
    
    Targeted tests:
    - `cargo test -p codex-tui table_readability_fallback --no-fail-fast`
    - `cargo test -p codex-tui markdown_render --no-fail-fast`
    - `cargo test -p codex-tui streaming::controller --no-fail-fast`
    - `cargo test -p codex-tui table_resize_lifecycle --no-fail-fast`
    
    ## Docs
    
    No developer docs update appears necessary.
  • fix(tui): reflow scrollback on terminal resize (#18575)
    Fixes multiple scrollback and terminal resize issues: #5538, #5576,
    #8352, #12223, #16165, and #15380.
    
    ## Why
    
    Codex writes finalized transcript output into terminal scrollback after
    wrapping it for the current viewport width. A later terminal resize
    could leave that scrollback shaped for the old width, so wider windows
    kept narrow output and narrower windows could show stale wrapping
    artifacts until enough new output replaced the visible area.
    
    This is also the foundation PR for responsive markdown tables. Table
    rendering needs finalized transcript content to be width-sensitive after
    insertion, not only while content is first streaming. Markdown table
    rendering itself stays in #18576.
    
    ## Stack
    
    - PR1: resize backlog reflow and interrupt cleanup
    - #18576: markdown table support
    
    ## What Changed
    
    - Rebuild source-backed transcript history when the terminal width
    changes. `terminal_resize_reflow` is introduced through the experimental
    feature system, but is enabled by default for this rollout so we can
    validate behavior across real terminals.
    - Preserve assistant and plan stream source so finalized streaming
    output can participate in resize reflow after consolidation.
    - Debounce resize work, but force a final source-backed reflow when a
    resize happened during active or unconsolidated streaming output.
    - Clear stale pending history lines on resize so old-width wrapped
    output is not emitted just before rebuilt scrollback.
    - Bound replay work with `[tui.terminal_resize_reflow].max_rows`:
    omitted uses terminal-specific defaults, `0` keeps all rendered rows,
    and a positive value sets an explicit cap. The cap applies both while
    initially replaying a resumed transcript into scrollback and when
    rebuilding scrollback after terminal resize.
    - Consolidate interrupted assistant streams before cleanup, then clear
    pending stream output and active-tail state consistently.
    - Move resize reflow and thread event buffering helpers out of `app.rs`
    into dedicated TUI modules.
    - Add focused coverage for resize reflow, feature-gated behavior,
    streaming source preservation, interrupted output cleanup,
    unicode-neutral text, terminal-specific row caps, and composer/layout
    stability.
    
    ## Runtime Bounds
    
    Resize reflow keeps only the most recent rendered rows when a row cap is
    active. The default is `auto`, which maps to the detected terminal's
    default scrollback size where Codex can identify it: VS Code `1000`,
    Windows Terminal `9001`, WezTerm `3500`, and Alacritty `10000`.
    Terminals without a dedicated mapping use the conservative fallback of
    `1000` rows. Users can override this with `[tui.terminal_resize_reflow]
    max_rows = N`, or set `max_rows = 0` to disable row limiting.
    
    ## Validation
    
    - `just fmt`
    - `git diff --check`
    - `cargo test --manifest-path codex-rs/Cargo.toml -p codex-tui reflow`
    - `cargo test --manifest-path codex-rs/Cargo.toml -p codex-tui
    transcript_reflow`
    - `just fix -p codex-tui`
    - PR CI in progress on the squashed branch
  • Rename tui_app_server to tui (#16104)
    This is a follow-up to https://github.com/openai/codex/pull/15922. That
    previous PR deleted the old `tui` directory and left the new
    `tui_app_server` directory in place. This PR renames `tui_app_server` to
    `tui` and fixes up all references.
  • Remove the legacy TUI split (#15922)
    This is the part 1 of 2 PRs that will delete the `tui` /
    `tui_app_server` split. This part simply deletes the existing `tui`
    directory and marks the `tui_app_server` feature flag as removed. I left
    the `tui_app_server` feature flag in place for now so its presence
    doesn't result in an error. It is simply ignored.
    
    Part 2 will rename the `tui_app_server` directory `tui`. I did this as
    two parts to reduce visible code churn.
  • tui: drop citation rendering (#4855)
    We don't instruct the model to use citations, so it never emits them.
    Further, ratatui [doesn't currently support rendering links into the
    terminal with OSC 8](https://github.com/ratatui/ratatui/issues/1028), so
    even if we did parse citations, we can't correctly render them.
    
    So, remove all the code related to rendering them.
  • add(core): managed config (#3868)
    ## Summary
    
    - Factor `load_config_as_toml` into `core::config_loader` so config
    loading is reusable across callers.
    - Layer `~/.codex/config.toml`, optional `~/.codex/managed_config.toml`,
    and macOS managed preferences (base64) with recursive table merging and
    scoped threads per source.
    
    ## Config Flow
    
    ```
    Managed prefs (macOS profile: com.openai.codex/config_toml_base64)
                                   ▲
                                   │
    ~/.codex/managed_config.toml   │  (optional file-based override)
                                   ▲
                                   │
                    ~/.codex/config.toml (user-defined settings)
    ```
    
    - The loader searches under the resolved `CODEX_HOME` directory
    (defaults to `~/.codex`).
    - Managed configs let administrators ship fleet-wide overrides via
    device profiles which is useful for enforcing certain settings like
    sandbox or approval defaults.
    - For nested hash tables: overlays merge recursively. Child tables are
    merged key-by-key, while scalar or array values replace the prior layer
    entirely. This lets admins add or tweak individual fields without
    clobbering unrelated user settings.
  • wrap markdown at render time (#4506)
    This results in correctly indenting list items with long lines.
    
    <img width="1006" height="251" alt="Screenshot 2025-09-30 at 10 00
    48 AM"
    src="https://github.com/user-attachments/assets/0a076cf6-ca3c-4efb-b3af-dc07617cdb6f"
    />
  • hide the status indicator when the answer stream starts (#4101)
    This eliminates a "bounce" at the end of streaming where we hide the
    status indicator at the end of the turn and the composer moves up two
    lines.
    
    Also, simplify streaming further by removing the HistorySink and
    inverting control, and collapsing a few single-element structures.
  • simplify StreamController (#3928)
    no intended functional change, just simplifying the code.
  • chore: more clippy rules 2 (#4057)
    The only file to watch is the cargo.toml
    All the others come from just fix + a few manual small fix
    
    The set of rules have been taken from the list of clippy rules
    arbitrarily while trying to optimise the learning and style of the code
    while limiting the loss of productivity
  • restyle thinking outputs (#3755)
    <img width="1205" height="930" alt="Screenshot 2025-09-16 at 2 23 18 PM"
    src="https://github.com/user-attachments/assets/bb2494f1-dd59-4bc9-9c4e-740605c999fd"
    />
  • replace tui_markdown with a custom markdown renderer (#3396)
    Also, simplify the streaming behavior.
    
    This fixes a number of display issues with streaming markdown, and paves
    the way for better markdown features (e.g. customizable styles, syntax
    highlighting, markdown-aware wrapping).
    
    Not currently supported:
    - footnotes
    - tables
    - reference-style links
  • fix (most) doubled lines and hanging list markers (#2789)
    This was mostly written by codex under heavy guidance via test cases
    drawn from logged session data and fuzzing. It also uncovered some bugs
    in tui_markdown, which will in some cases split a list marker from the
    list item content. We're not addressing those bugs for now.
  • hide CoT by default; show headers in status indicator (#2316)
    Plan is for full CoT summaries to be visible in a "transcript view" when
    we implement that, but for now they're hidden.
    
    
    https://github.com/user-attachments/assets/e8a1b0ef-8f2a-48ff-9625-9c3c67d92cdb
  • Re-add markdown streaming (#2029)
    Wait for newlines, then render markdown on a line by line basis. Word wrap it for the current terminal size and then spit it out line by line into the UI. Also adds tests and fixes some UI regressions.
  • Streaming markdown (#1920)
    We wait until we have an entire newline, then format it with markdown and stream in to the UI. This reduces time to first token but is the right thing to do with our current rendering model IMO. Also lets us add word wrapping!