11 Commits

  • [codex] simplify memory read metrics (#28164)
    ## Why
    
    Memory read telemetry currently reconstructs the executable shell
    command after a tool call finishes. That duplicates shell, login-policy,
    and cwd resolution owned by the tool handlers, and can diverge from the
    environment-specific command that unified exec actually ran.
    
    ## What changed
    
    - Expose the existing restricted shell-script parser directly for raw
    script text.
    - Parse `shell_command` and `exec_command` input into plain command argv
    before classifying memory reads.
    - Preserve all-or-nothing safe-command validation for multi-command
    scripts.
    - Remove cwd resolution, shell selection, and the unnecessary async
    boundary from memory read metric emission.
    
    ## Testing
    
    - `just test -p codex-shell-command`
    - `cargo check -p codex-core`
  • chore: move memory prompt builder into extension (#24558)
    ## Why
    
    The memories extension now owns the read-path developer instructions it
    injects at thread start. Keeping that prompt builder and template in
    `codex-memories-read` left the extension depending on a helper crate for
    extension-specific prompt assembly, and kept async template/truncation
    dependencies in the read crate after the remaining read surface no
    longer needed them.
    
    ## What changed
    
    - Moved `prompts.rs`, its tests, and `templates/memories/read_path.md`
    from `memories/read` into `ext/memories`.
    - Wired `MemoryExtension` to call the local prompt builder and added the
    moved templates to `ext/memories/BUILD.bazel` compile data.
    - Removed the now-unused prompt export and prompt-related dependencies
    from `codex-memories-read`.
    
    ## Testing
    
    - Not run locally.
  • Densify and version memory summaries (#23148)
    ## Why
    
    `memory_summary.md` is injected into every session, so its value depends
    on staying compact, navigational, and easy to regenerate when the
    expected shape changes. The previous consolidation prompt encouraged a
    broad actionable inventory and allowed older summary structures to be
    patched in place, which makes it easier for stale or overly verbose
    summaries to keep accumulating.
    
    This change makes the summary format explicitly versioned and biases
    Phase 2 memory consolidation toward denser prompt-loaded context.
    
    ## What changed
    
    - Require `memory_summary.md` to begin with an exact `v1` header.
    - Teach consolidation to regenerate `memory_summary.md` from scratch
    when the header is missing or incompatible, while still allowing
    incremental updates to `MEMORY.md`.
    - Tighten the `memory_summary.md` instructions so it acts as a compact
    routing/index layer instead of a second handbook.
    - Lower `MEMORY_TOOL_DEVELOPER_INSTRUCTIONS_SUMMARY_TOKEN_LIMIT` from
    `5_000` to `2_500` so the runtime prompt budget matches the denser
    summary target.
    
    ## Verification
    
    Not run; this is a prompt/template update plus a prompt budget constant
    change.
  • Disable empty Cargo test targets (#21584)
    ## Summary
    
    `cargo test` has entails both running standard Rust tests and doctests.
    It turns out that the doctest discovery is fairly slow, and it's a cost
    you pay even for crates that don't include any doctests.
    
    This PR disables doctests with `doctest = false` for crates that lack
    any doctests.
    
    For the collection of crates below, this speeds up test execution by
    >4x.
    
    E.g., before this PR:
    
    ```
    Benchmark 1: cargo test     -p codex-utils-absolute-path     -p codex-utils-cache     -p codex-utils-cli     -p codex-utils-home-dir     -p codex-utils-output-truncation     -p codex-utils-path     -p codex-utils-string     -p codex-utils-template     -p codex-utils-elapsed     -p codex-utils-json-to-toml
      Time (mean ± σ):      1.849 s ±  4.455 s    [User: 0.752 s, System: 1.367 s]
      Range (min … max):    0.418 s … 14.529 s    10 runs
    ```
    
    And after:
    
    ```
    Benchmark 1: cargo test     -p codex-utils-absolute-path     -p codex-utils-cache     -p codex-utils-cli     -p codex-utils-home-dir     -p codex-utils-output-truncation     -p codex-utils-path     -p codex-utils-string     -p codex-utils-template     -p codex-utils-elapsed     -p codex-utils-json-to-toml
      Time (mean ± σ):     428.6 ms ±   6.9 ms    [User: 187.7 ms, System: 219.7 ms]
      Range (min … max):   418.0 ms … 436.8 ms    10 runs
    ```
    
    For a single crate, with >2x speedup, before:
    
    ```
    Benchmark 1: cargo test -p codex-utils-string
      Time (mean ± σ):     491.1 ms ±   9.0 ms    [User: 229.8 ms, System: 234.9 ms]
      Range (min … max):   480.9 ms … 512.0 ms    10 runs
    ```
    
    And after:
    
    ```
    Benchmark 1: cargo test -p codex-utils-string
      Time (mean ± σ):     213.9 ms ±   4.3 ms    [User: 112.8 ms, System: 84.0 ms]
      Range (min … max):   206.8 ms … 221.0 ms    13 runs
    ```
    
    Co-authored-by: Codex <noreply@openai.com>
  • feat: seed ad-hoc memory extension instructions (#20606)
    ## Summary
    
    Ad-hoc memory notes are written under `memories/extensions/ad_hoc/`, but
    the consolidation agent only knows how to interpret an extension when
    the extension folder has an `instructions.md`. Seed those instructions
    from the memories write pipeline so an enabled memories startup creates
    the expected ad-hoc extension layout automatically.
    
    This also moves extension-specific write behavior behind a dedicated
    `memories/write/src/extensions/` module. `ad_hoc` owns the seeded
    instructions template, while the existing resource-retention cleanup
    lives in its own `prune` module so future memory extensions can add
    their own write-side setup without growing a flat helper file.
    
    ## Changes
    
    - Seed `memories/extensions/ad_hoc/instructions.md` during eligible
    memory startup without overwriting an existing file.
    - Store the ad-hoc instructions template under
    `memories/write/templates/extensions/ad_hoc/`, keeping ownership in
    `codex-memories-write`.
    - Split memory extension support into `extensions::ad_hoc` and
    `extensions::prune`.
    - Keep the existing old-resource pruning behavior unchanged.
    
    ## Verification
    
    - `cargo test -p codex-memories-write`
    - `bazel build //codex-rs/memories/write:write`
    
    ---------
    
    Co-authored-by: chatgpt-codex-connector[bot] <199175422+chatgpt-codex-connector[bot]@users.noreply.github.com>
  • feat: house-keeping memories 1 (#19998)
    Just move metrics in a dedicated file
  • chore: split memories part 1 (#19818)
    Extract memories into 2 different crates