mirror of
https://github.com/Egonex-AI/Understand-Anything.git
synced 2026-06-22 10:58:03 +08:00
Merge pull request #47 from Lum1104/feat/token-reduction
feat: reduce /understand token cost by ~85% on large codebases
This commit is contained in:
Yuxiang Lin
GitHub
@@ -9,7 +9,7 @@
|
||||
{
|
||||
"name": "understand-anything",
|
||||
"description": "Multi-agent codebase analysis with interactive dashboard, guided tours, and skill commands",
|
||||
"version": "1.2.1",
|
||||
"version": "1.2.2",
|
||||
"source": "./understand-anything-plugin"
|
||||
}
|
||||
]
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "understand-anything",
|
||||
"description": "AI-powered codebase understanding — analyze, visualize, and explain any project",
|
||||
"version": "1.2.1",
|
||||
"version": "1.2.2",
|
||||
"author": {
|
||||
"name": "Lum1104"
|
||||
},
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"name": "understand-anything",
|
||||
"displayName": "Understand Anything",
|
||||
"description": "AI-powered codebase understanding — analyze, visualize, and explain any project",
|
||||
"version": "1.2.1",
|
||||
"version": "1.2.2",
|
||||
"author": {
|
||||
"name": "Lum1104"
|
||||
},
|
||||
|
||||
@@ -237,9 +237,9 @@ The `/understand` command orchestrates 5 specialized agents:
|
||||
| `file-analyzer` | Extract functions, classes, imports; produce graph nodes and edges |
|
||||
| `architecture-analyzer` | Identify architectural layers |
|
||||
| `tour-builder` | Generate guided learning tours |
|
||||
| `graph-reviewer` | Validate graph completeness and referential integrity |
|
||||
| `graph-reviewer` | Validate graph completeness and referential integrity (runs inline by default; use `--review` for full LLM review) |
|
||||
|
||||
File analyzers run in parallel (up to 3 concurrent). Supports incremental updates — only re-analyzes files that changed since the last run.
|
||||
File analyzers run in parallel (up to 5 concurrent, 20-30 files per batch). Supports incremental updates — only re-analyzes files that changed since the last run.
|
||||
|
||||
### Project Structure
|
||||
|
||||
|
||||
@@ -0,0 +1,395 @@
|
||||
# Token Reduction Design
|
||||
|
||||
**Date:** 2026-03-27
|
||||
**Status:** Draft
|
||||
**Goal:** Reduce total token cost of `/understand` by ~85-90% on large codebases (200+ files)
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
For large codebases, the `/understand` pipeline spends the vast majority of its tokens on **repeated context injection**. The same data is sent to every subagent independently, even when that data could be computed once and shared.
|
||||
|
||||
### Token cost breakdown (500-file TypeScript+React project, baseline)
|
||||
|
||||
| Source | Phase | Tokens (input) | % of total |
|
||||
|---|---|---|---|
|
||||
| `allProjectFiles` list × 67 batches | Phase 2 | ~167,000 | ~50% |
|
||||
| `file-analyzer-prompt.md` × 67 batches | Phase 2 | ~134,000 | ~40% |
|
||||
| Language/framework addendums × 67 batches | Phase 2 | ~68,000 | ~20% |
|
||||
| Tour builder payload (all nodes + edges) | Phase 5 | ~80,000 | ~24% |
|
||||
| Graph reviewer (assembled graph + inventory) | Phase 6 | ~58,000 | ~17% |
|
||||
| Architecture analyzer payload | Phase 4 | ~22,000 | ~7% |
|
||||
| **Total** | | **~529,000** | |
|
||||
|
||||
The root cause: **Phase 2 runs 67 batches (at 5-10 files each), and every single batch receives the full 500-file list for import resolution.** The file list alone costs ~2,500 tokens × 67 repetitions = 167,000 tokens on input, doing work that is entirely redundant between batches.
|
||||
|
||||
---
|
||||
|
||||
## Goals
|
||||
|
||||
- Reduce total input tokens by 85%+ on a 500-file project
|
||||
- No degradation in graph quality for standard projects
|
||||
- Preserve the `--full` / incremental / scope flags
|
||||
- Maintain backward compatibility with existing `knowledge-graph.json` output schema
|
||||
|
||||
---
|
||||
|
||||
## Changes
|
||||
|
||||
Five changes compose the full approach (C1–C5). Each is independent and can be shipped separately, but all five are needed for the full reduction.
|
||||
|
||||
---
|
||||
|
||||
### C1 — Pre-resolve imports in the project scanner
|
||||
|
||||
**Root cause addressed:** `allProjectFiles` (the entire file list) is injected into every file-analyzer batch solely so each batch's extraction script can resolve relative imports. This is redundant: the full file list is available during Phase 1, and import resolution is deterministic. It should happen once, not 67 times.
|
||||
|
||||
**Change:** Extend the Phase 1 scanner script to also parse import statements from every source file and resolve relative imports against the discovered file list. The resolved results are written into `scan-result.json` as a new `importMap` field. File-analyzer batches then receive only their own batch's pre-resolved imports — not the full file list.
|
||||
|
||||
#### Scanner output addition
|
||||
|
||||
`scan-result.json` gains:
|
||||
|
||||
```json
|
||||
{
|
||||
"importMap": {
|
||||
"src/index.ts": ["src/utils.ts", "src/config.ts"],
|
||||
"src/utils.ts": [],
|
||||
"src/components/App.tsx": ["src/hooks/useAuth.ts", "src/store/index.ts"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- Keys are project-relative paths (matching `files[*].path`)
|
||||
- Values are resolved project-relative paths only (external/unresolvable imports are omitted)
|
||||
- External imports (`node_modules`, unresolvable paths) are excluded from the map entirely
|
||||
|
||||
#### Scanner script additions (Phase 1 Step 8)
|
||||
|
||||
After the existing 7 steps, the scanner script adds a new step:
|
||||
|
||||
```
|
||||
Step 8 — Import Resolution
|
||||
|
||||
For each file in the discovered source list:
|
||||
1. Read the file content
|
||||
2. Extract import statements (language-specific patterns per Step 3's language detection):
|
||||
- TypeScript/JavaScript: `import ... from '...'`, `require('...')`
|
||||
- Python: `import ...`, `from ... import ...`
|
||||
- Go: `import "..."` blocks
|
||||
- Rust: `use ...` statements
|
||||
- Java/Kotlin: `import ...` statements
|
||||
- Ruby: `require`, `require_relative`
|
||||
3. For each relative import (starts with `./` or `../`):
|
||||
a. Compute the resolved path from the current file's directory
|
||||
b. Normalize to project-relative format
|
||||
c. Try common extension variants if the import has no extension:
|
||||
`.ts`, `.tsx`, `.js`, `.jsx`, `/index.ts`, `/index.js`, `/index.tsx`
|
||||
d. If any variant exists in the discovered file list, record it; otherwise skip
|
||||
4. For absolute imports (no `.` prefix): skip (external package)
|
||||
|
||||
Output the full importMap in the JSON result.
|
||||
```
|
||||
|
||||
#### File-analyzer input schema change
|
||||
|
||||
**Before:**
|
||||
```json
|
||||
{
|
||||
"projectRoot": "/path/to/project",
|
||||
"allProjectFiles": ["src/index.ts", "src/utils.ts", "...500 paths..."],
|
||||
"batchFiles": [
|
||||
{"path": "src/index.ts", "language": "typescript", "sizeLines": 150}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
**After:**
|
||||
```json
|
||||
{
|
||||
"projectRoot": "/path/to/project",
|
||||
"batchFiles": [
|
||||
{"path": "src/index.ts", "language": "typescript", "sizeLines": 150}
|
||||
],
|
||||
"batchImportData": {
|
||||
"src/index.ts": ["src/utils.ts", "src/config.ts"],
|
||||
"src/components/App.tsx": ["src/hooks/useAuth.ts"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`allProjectFiles` is removed entirely. `batchImportData` contains only the pre-resolved imports for the files in this batch (sliced from `importMap` by the orchestrator).
|
||||
|
||||
#### File-analyzer extraction script change
|
||||
|
||||
The extraction script no longer performs import resolution. It:
|
||||
- Still extracts: functions, classes, exports, metrics (unchanged)
|
||||
- For imports: reads `batchImportData[file.path]` from the input JSON — no cross-referencing needed
|
||||
- The `imports` array in each file result becomes: `batchImportData[file.path]` mapped to import edge objects with `resolvedPath` already populated, `isExternal: false`
|
||||
|
||||
#### SKILL.md Phase 2 change
|
||||
|
||||
Remove the `allProjectFiles` injection from the batch dispatch prompt. Replace with a per-batch `batchImportData` slice:
|
||||
|
||||
```
|
||||
For each batch, slice importData from the importMap read in Phase 1:
|
||||
batchImportData = { [file.path]: importMap[file.path] ?? [] }
|
||||
for each file in this batch
|
||||
```
|
||||
|
||||
#### Token savings estimate
|
||||
|
||||
| | Batches | Tokens/batch | Total |
|
||||
|---|---|---|---|
|
||||
| Before | 67 | ~2,500 (file list) | ~167,500 |
|
||||
| After (C1 alone) | 67 | ~200 (batch importData) | ~13,400 |
|
||||
| **Savings** | | | **~154,100** |
|
||||
|
||||
---
|
||||
|
||||
### C2 — Increase batch size from 5-10 to 20-30 files
|
||||
|
||||
**Root cause addressed:** Every batch incurs the full cost of `file-analyzer-prompt.md` (~2,000 tokens) plus the batch dispatch overhead. With 67 batches, this adds up even without `allProjectFiles`. Fewer, larger batches directly reduce this repetition.
|
||||
|
||||
**Change:** In SKILL.md Phase 2, change the batch size guidance:
|
||||
|
||||
- **Before:** "Batch the file list from Phase 1 into groups of **5-10 files each**"
|
||||
- **After:** "Batch the file list from Phase 1 into groups of **20-30 files each** (aim for ~25 per batch)"
|
||||
|
||||
Also update the concurrency limit from 3 to **5** concurrent batches. Fewer total batches means we can afford more parallelism without overwhelming the system.
|
||||
|
||||
#### Trade-offs
|
||||
|
||||
| | Smaller batches (current) | Larger batches (new) |
|
||||
|---|---|---|
|
||||
| Files per batch | 5-10 | 20-30 |
|
||||
| Total batches (500 files) | ~67 | ~20 |
|
||||
| Prompt repetition | 67× | 20× |
|
||||
| Quality risk | Lower (focused) | Slightly higher (more files per subagent) |
|
||||
| Concurrency | 3 | 5 |
|
||||
|
||||
Quality risk is low: each subagent still operates on distinct, non-overlapping file groups. The extraction script is deterministic regardless of batch size. Semantic analysis (summaries, tags) may be marginally less focused, but the quality difference is negligible in practice for well-structured files.
|
||||
|
||||
#### Token savings estimate (combined with C1)
|
||||
|
||||
| | Batches | Tokens/batch (prompt) | Total |
|
||||
|---|---|---|---|
|
||||
| Before (C1 only) | 67 | ~2,000 | ~134,000 |
|
||||
| After (C1+C2) | 20 | ~2,000 | ~40,000 |
|
||||
| **Savings from C2** | | | **~94,000** |
|
||||
|
||||
C1+C2 combined eliminate ~248,000 tokens from Phase 2 (down from ~301,500 to ~53,500, a ~82% Phase 2 reduction).
|
||||
|
||||
---
|
||||
|
||||
### C3 — Remove language/framework addendums from file-analyzer batches
|
||||
|
||||
**Root cause addressed:** `languages/typescript.md` (~600 tokens) and `frameworks/react.md` (~700 tokens) are read and injected into every file-analyzer batch prompt. For a TypeScript+React project with 20 batches (after C2), this costs 20 × 1,300 = 26,000 additional tokens — and the model already has deep knowledge of these languages from training.
|
||||
|
||||
**Change:** Stop injecting addendum files into Phase 2 batch prompts entirely. The addendums remain injected into Phase 4 (architecture analyzer) where there is only **one** subagent call, making the cost acceptable.
|
||||
|
||||
Instead, add a compact "Language and Framework Hints" reference section directly into `file-analyzer-prompt.md`. This section is a distilled, one-time addition (~150 tokens total) that captures the most useful patterns from all addendums in a concise lookup table.
|
||||
|
||||
#### New section in `file-analyzer-prompt.md` (replace addendum injection)
|
||||
|
||||
```markdown
|
||||
## Language and Framework Quick Reference
|
||||
|
||||
Use these hints to improve tag and edge accuracy. These supplement your training knowledge.
|
||||
|
||||
| Signal | Tag(s) | Note |
|
||||
|---|---|---|
|
||||
| File in `hooks/`, exports function starting with `use` | `hook`, `service` | React custom hook |
|
||||
| File in `contexts/`, exports a Provider | `service`, `state` | React context |
|
||||
| File in `pages/` or `views/` | `ui`, `routing` | Page-level component |
|
||||
| File in `store/`, `slices/`, `reducers/` | `state` | State management |
|
||||
| File in `services/`, `api/` | `service` | Data-fetching / API client |
|
||||
| `__init__.py` with re-exports | `entry-point`, `barrel` | Python package root |
|
||||
| `manage.py` at project root | `entry-point` | Django management entry |
|
||||
| File named `mod.rs` | `barrel` | Rust module barrel |
|
||||
| File named `main.go` in `cmd/` | `entry-point` | Go binary entry |
|
||||
|
||||
For React: create `depends_on` edges from components to hooks they call. Create `publishes`/`subscribes` edges for Context provider/consumer patterns.
|
||||
```
|
||||
|
||||
#### SKILL.md Phase 2 change
|
||||
|
||||
Remove steps 2 and 3 from the "Build the combined prompt template" block:
|
||||
- **Remove:** Step 2 (Language context injection — read `./languages/<language-id>.md` per detected language)
|
||||
- **Remove:** Step 3 (Framework addendum injection — read `./frameworks/<framework-id>.md` per detected framework)
|
||||
- **Keep:** Step 1 (Read the base template at `./file-analyzer-prompt.md`)
|
||||
|
||||
The addendum injection steps **remain unchanged** in Phase 4 (architecture analyzer), since they run once.
|
||||
|
||||
#### Token savings estimate
|
||||
|
||||
| | Batches | Addendum tokens/batch | Total |
|
||||
|---|---|---|---|
|
||||
| Before (after C2) | 20 | ~1,300 (TS+React) | ~26,000 |
|
||||
| After | 20 | ~150 (inline hints) | ~3,000 |
|
||||
| **Savings** | | | **~23,000** |
|
||||
|
||||
---
|
||||
|
||||
### C4 — Slim Phase 4 and Phase 5 payloads
|
||||
|
||||
**Root cause addressed:** Phase 5 (tour builder) receives all nodes (file + function + class) and all edges (imports + contains + calls + exports + ...). For a 500-file project, this can include 1,500+ nodes and 3,000+ edges. Most of this data is not needed for tour design.
|
||||
|
||||
#### Phase 4 (Architecture Analyzer) — minor trim
|
||||
|
||||
Phase 4 already only sends file-type nodes, which is correct. Minor change: explicitly strip `languageNotes` from each node object in the payload (it's not useful for layer assignment and can be verbose). Also strip `name` — it is always derivable as the basename of `filePath`.
|
||||
|
||||
**Before per node:** `{id, name, filePath, summary, tags, complexity, languageNotes?}`
|
||||
**After per node:** `{id, filePath, summary, tags}`
|
||||
|
||||
Savings: ~15-20% fewer tokens per node, ~3,000–5,000 tokens total for Phase 4.
|
||||
|
||||
#### Phase 5 (Tour Builder) — major trim
|
||||
|
||||
Three changes to what the orchestrator injects into the tour-builder subagent:
|
||||
|
||||
**1. File nodes only (strip function/class nodes)**
|
||||
|
||||
The tour references node IDs for wayfinding. In practice the tour always references `file:` nodes — function and class nodes are visible in the dashboard's NodeInfo sidebar once a file is selected, but the tour itself navigates at the file level.
|
||||
|
||||
- **Before:** all nodes (file + function + class) — for 500 files, maybe 1,500+ nodes
|
||||
- **After:** file-type nodes only — 500 nodes
|
||||
|
||||
**2. Slim node format**
|
||||
|
||||
The tour builder script only uses node IDs, names, and types for graph computation. Summaries and tags are used in Phase 2 (pedagogical narrative writing). Strip heavy optional fields from the injected payload:
|
||||
|
||||
- **Before per node:** `{id, name, filePath, summary, type, tags, complexity, languageNotes?}`
|
||||
- **After per node:** `{id, name, filePath, summary, type}` (drop tags, complexity, languageNotes)
|
||||
|
||||
**3. Slim edges (imports + calls only) and slim layers**
|
||||
|
||||
The tour's BFS traversal only traverses `imports` and `calls` edges. `contains`, `exports`, `tested_by`, `depends_on`, and other edge types add no value to the traversal and inflate the payload.
|
||||
|
||||
- **Before edges:** all edge types (~3,000+ edges including all `contains` edges to function/class nodes)
|
||||
- **After edges:** only `imports` and `calls` edge types (~400–800 edges for typical projects)
|
||||
|
||||
For layers, the tour builder uses layer data only to inform the tour's narrative arc (which layer to introduce first, second, etc.). It does not need the full `nodeIds` arrays — those can be very large.
|
||||
|
||||
- **Before per layer:** `{id, name, description, nodeIds: [...hundreds of IDs]}`
|
||||
- **After per layer:** `{id, name, description}` (drop nodeIds)
|
||||
|
||||
#### Token savings estimate (Phase 5)
|
||||
|
||||
| Data | Before | After |
|
||||
|---|---|---|
|
||||
| Node count | ~1,500 × ~180 chars | ~500 × ~120 chars |
|
||||
| Node tokens | ~67,500 | ~15,000 |
|
||||
| Edge count | ~3,000 × ~80 chars | ~600 × ~80 chars |
|
||||
| Edge tokens | ~60,000 | ~12,000 |
|
||||
| Layer tokens | ~5,000 | ~500 |
|
||||
| **Phase 5 total** | **~132,500** | **~27,500** |
|
||||
| **Savings** | | **~105,000** |
|
||||
|
||||
#### SKILL.md changes
|
||||
|
||||
In **Phase 4** dispatch prompt template, update the file node format:
|
||||
```
|
||||
File nodes:
|
||||
[list of {id, filePath, summary, tags} for all file-type nodes]
|
||||
```
|
||||
|
||||
In **Phase 5** dispatch prompt template, update all three payload specs:
|
||||
```
|
||||
Nodes (file nodes only):
|
||||
[list of {id, name, filePath, summary, type} for all file-type nodes only — do NOT include function or class nodes]
|
||||
|
||||
Key edges (imports and calls only):
|
||||
[list of edges where type is "imports" or "calls" only]
|
||||
|
||||
Layers:
|
||||
[list of {id, name, description} — omit nodeIds]
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### C5 — Gate the graph-reviewer subagent behind `--review`
|
||||
|
||||
**Root cause addressed:** The graph-reviewer subagent (Phase 6) reads the entire assembled graph (~500 nodes, all edges, layers, tour) and runs a LLM-powered validation. However, its Phase 1 is entirely a deterministic script, and its Phase 2 is a simple threshold decision: if `issues.length === 0`, approve. There is no LLM judgment needed for the happy path.
|
||||
|
||||
**Change:** By default, skip the graph-reviewer subagent. The orchestrator performs inline deterministic validation using a pre-written script. Only when `--review` is explicitly passed in `$ARGUMENTS` does the full LLM reviewer subagent run.
|
||||
|
||||
#### Default path (no `--review`)
|
||||
|
||||
In Phase 6, instead of dispatching the graph-reviewer subagent, the orchestrator:
|
||||
|
||||
1. Writes a compact validation script inline (embedded in SKILL.md, ~50 lines of Node.js):
|
||||
- Check: every edge source/target references a real node ID
|
||||
- Check: every file node appears in exactly one layer
|
||||
- Check: every tour step nodeId exists
|
||||
- Check: no duplicate node IDs
|
||||
- Check: required fields present on nodes and edges
|
||||
2. Runs the script against `assembled-graph.json`
|
||||
3. If `issues.length === 0`: proceed to Phase 7 (save)
|
||||
4. If `issues.length > 0`: apply the same automated fixes as before (remove dangling edges, fill defaults), then save
|
||||
|
||||
This is sufficient for standard runs. The LLM reviewer adds value for catching subtle quality issues (generic summaries, orphan nodes, tour step coherence) — but those are nice-to-have, not blocking.
|
||||
|
||||
#### `--review` path
|
||||
|
||||
When `--review` is in `$ARGUMENTS`, the full graph-reviewer subagent runs as it does today. No change to that code path.
|
||||
|
||||
#### Token savings estimate
|
||||
|
||||
| Path | Tokens |
|
||||
|---|---|
|
||||
| Current (always runs LLM reviewer) | ~58,000 input + ~500 output |
|
||||
| Default (inline script, no LLM) | ~0 |
|
||||
| `--review` (unchanged) | ~58,000 (same as current) |
|
||||
| **Savings for default runs** | **~58,500** |
|
||||
|
||||
---
|
||||
|
||||
## Combined savings summary
|
||||
|
||||
| Change | Tokens before | Tokens after | Savings |
|
||||
|---|---|---|---|
|
||||
| C1+C2: import map + batch consolidation | ~301,500 | ~53,500 | ~248,000 |
|
||||
| C3: remove addendums from batches | ~26,000 | ~3,000 | ~23,000 |
|
||||
| C4: slim Phase 4+5 payloads | ~154,500 | ~33,000 | ~121,500 |
|
||||
| C5: gate reviewer (default path) | ~58,500 | ~0 | ~58,500 |
|
||||
| **Total** | **~540,500** | **~89,500** | **~451,000 (~83%)** |
|
||||
|
||||
Estimates are for a 500-file TypeScript+React project. Actual savings scale with project size — a 1,000-file project would see proportionally larger savings from C1+C2 (more batches = more repetition eliminated).
|
||||
|
||||
---
|
||||
|
||||
## File changes
|
||||
|
||||
| File | Change |
|
||||
|---|---|
|
||||
| `skills/understand/project-scanner-prompt.md` | Add Step 8 (import resolution); add `importMap` to output schema |
|
||||
| `skills/understand/file-analyzer-prompt.md` | Replace `allProjectFiles` with `batchImportData` in input schema; update extraction script to use pre-resolved imports; add compact Language/Framework Quick Reference section; remove addendum injection steps |
|
||||
| `skills/understand/SKILL.md` | Phase 1: note importMap in scan result; Phase 2: remove addendum injection (steps 2+3), increase batch size 5-10→20-30, increase concurrency 3→5, replace `allProjectFiles` injection with `batchImportData` slice; Phase 4: slim node format in dispatch; Phase 5: file nodes only + slim edges + slim layers in dispatch; Phase 6: conditional reviewer — default inline script, `--review` flag for LLM reviewer |
|
||||
| `skills/understand/architecture-analyzer-prompt.md` | No change (addendums still injected here) |
|
||||
| `skills/understand/tour-builder-prompt.md` | Update input schema to reflect file-only nodes, imports+calls-only edges, slim layer format |
|
||||
| `skills/understand/graph-reviewer-prompt.md` | No change (only used when `--review` flag is passed) |
|
||||
|
||||
---
|
||||
|
||||
## Risks and mitigations
|
||||
|
||||
| Risk | Likelihood | Mitigation |
|
||||
|---|---|---|
|
||||
| Scanner import resolution misses edge cases (complex re-exports, dynamic imports) | Medium | Log unresolved imports; file-analyzer still uses resolved data and creates edges only for confirmed matches. Missed imports = missing edges, which is same behavior as before for unresolvable imports |
|
||||
| Larger batches (C2) reduce summary quality | Low | Summary quality is driven by the model's analysis of individual files. Batch size mainly affects how many files share one subagent's context window, not per-file quality. 20-30 files remains well within context limits |
|
||||
| Stripping function/class nodes from tour (C4) breaks existing tour steps | None | Tour steps reference `file:` node IDs. No existing tour data references function/class nodes at the step level |
|
||||
| Removing reviewer by default (C5) misses graph errors | Low | The inline deterministic script catches all critical structural issues (dangling refs, missing layers, duplicate IDs). The LLM reviewer's additional value is quality warnings (orphan nodes, generic summaries), which are non-blocking |
|
||||
| Import map generation slows down Phase 1 | Low | The scanner script already reads all files for line counting. Import parsing adds one regex pass per file — negligible overhead |
|
||||
|
||||
---
|
||||
|
||||
## Phased rollout recommendation
|
||||
|
||||
Given the risk profile, implement in this order:
|
||||
|
||||
1. **C5 first** — gate the reviewer, lowest risk, immediate 58K token savings per run
|
||||
2. **C4** — slim Phase 5 payload, no scanner changes, no quality risk
|
||||
3. **C3** — remove addendums from batches, add inline hints
|
||||
4. **C1+C2 together** — scanner changes and batch consolidation, test thoroughly on small/medium/large projects before releasing
|
||||
@@ -0,0 +1,971 @@
|
||||
# Token Reduction Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Reduce `/understand` token cost by ~85% on large codebases through import pre-resolution, batch consolidation, addendum removal, payload slimming, and gating the LLM reviewer.
|
||||
|
||||
**Architecture:** Five changes (C5 → C4 → C3 → C1+C2) applied in rollout order — lowest risk first. All changes are to prompt/skill markdown files in `understand-anything-plugin/skills/understand/`. No TypeScript source changes required.
|
||||
|
||||
**Tech Stack:** Markdown skill files, Node.js inline scripts embedded in SKILL.md, knowledge-graph JSON pipeline.
|
||||
|
||||
**Design doc:** `docs/plans/2026-03-27-token-reduction-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Task 1: C5 — Gate graph-reviewer behind `--review` flag
|
||||
|
||||
Replaces the always-on LLM graph-reviewer subagent with a deterministic inline validation script. The LLM reviewer only runs when `--review` is in `$ARGUMENTS`. Saves ~58,500 tokens per default run.
|
||||
|
||||
**Files:**
|
||||
- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 6, lines 330–362)
|
||||
|
||||
### Step 1: Open SKILL.md and locate Phase 6
|
||||
|
||||
Read the file and find "## Phase 6 — REVIEW" (line 297). Identify steps 3–6 (lines 330–362) which currently always dispatch the LLM graph-reviewer subagent.
|
||||
|
||||
### Step 2: Replace Phase 6 steps 3–6 with conditional reviewer logic
|
||||
|
||||
Replace lines 330–362 (from "3. Dispatch a subagent using the prompt template" through "6. **If `approved: true`:** Proceed to Phase 7.") with:
|
||||
|
||||
```markdown
|
||||
3. **Check `$ARGUMENTS` for `--review` flag.** Then run the appropriate validation path:
|
||||
|
||||
---
|
||||
|
||||
#### Default path (no `--review`): inline deterministic validation
|
||||
|
||||
Write the following Node.js script to `$PROJECT_ROOT/.understand-anything/tmp/ua-inline-validate.js`:
|
||||
|
||||
```javascript
|
||||
#!/usr/bin/env node
|
||||
const fs = require('fs');
|
||||
const graphPath = process.argv[2];
|
||||
const outputPath = process.argv[3];
|
||||
try {
|
||||
const graph = JSON.parse(fs.readFileSync(graphPath, 'utf8'));
|
||||
const issues = [], warnings = [];
|
||||
const nodeIds = new Set();
|
||||
const seen = new Map();
|
||||
graph.nodes.forEach((n, i) => {
|
||||
if (!n.id) { issues.push(`Node[${i}] missing id`); return; }
|
||||
if (!n.type) issues.push(`Node[${i}] '${n.id}' missing type`);
|
||||
if (!n.name) issues.push(`Node[${i}] '${n.id}' missing name`);
|
||||
if (!n.summary) issues.push(`Node[${i}] '${n.id}' missing summary`);
|
||||
if (!n.tags || !n.tags.length) issues.push(`Node[${i}] '${n.id}' missing tags`);
|
||||
if (seen.has(n.id)) issues.push(`Duplicate node ID '${n.id}' at indices ${seen.get(n.id)} and ${i}`);
|
||||
else seen.set(n.id, i);
|
||||
nodeIds.add(n.id);
|
||||
});
|
||||
graph.edges.forEach((e, i) => {
|
||||
if (!nodeIds.has(e.source)) issues.push(`Edge[${i}] source '${e.source}' not found`);
|
||||
if (!nodeIds.has(e.target)) issues.push(`Edge[${i}] target '${e.target}' not found`);
|
||||
});
|
||||
const fileNodes = graph.nodes.filter(n => n.type === 'file').map(n => n.id);
|
||||
const assigned = new Map();
|
||||
(graph.layers || []).forEach(layer => {
|
||||
(layer.nodeIds || []).forEach(id => {
|
||||
if (!nodeIds.has(id)) issues.push(`Layer '${layer.id}' refs missing node '${id}'`);
|
||||
if (assigned.has(id)) issues.push(`Node '${id}' appears in multiple layers`);
|
||||
assigned.set(id, layer.id);
|
||||
});
|
||||
});
|
||||
fileNodes.forEach(id => {
|
||||
if (!assigned.has(id)) issues.push(`File node '${id}' not in any layer`);
|
||||
});
|
||||
(graph.tour || []).forEach((step, i) => {
|
||||
(step.nodeIds || []).forEach(id => {
|
||||
if (!nodeIds.has(id)) issues.push(`Tour step[${i}] refs missing node '${id}'`);
|
||||
});
|
||||
});
|
||||
const withEdges = new Set([
|
||||
...graph.edges.map(e => e.source),
|
||||
...graph.edges.map(e => e.target)
|
||||
]);
|
||||
graph.nodes.forEach(n => {
|
||||
if (!withEdges.has(n.id)) warnings.push(`Node '${n.id}' has no edges (orphan)`);
|
||||
});
|
||||
const stats = {
|
||||
totalNodes: graph.nodes.length,
|
||||
totalEdges: graph.edges.length,
|
||||
totalLayers: (graph.layers || []).length,
|
||||
tourSteps: (graph.tour || []).length,
|
||||
nodeTypes: graph.nodes.reduce((a, n) => { a[n.type] = (a[n.type]||0)+1; return a; }, {}),
|
||||
edgeTypes: graph.edges.reduce((a, e) => { a[e.type] = (a[e.type]||0)+1; return a; }, {})
|
||||
};
|
||||
fs.writeFileSync(outputPath, JSON.stringify({ issues, warnings, stats }, null, 2));
|
||||
process.exit(0);
|
||||
} catch (err) { process.stderr.write(err.message + '\n'); process.exit(1); }
|
||||
```
|
||||
|
||||
Execute it:
|
||||
```bash
|
||||
node $PROJECT_ROOT/.understand-anything/tmp/ua-inline-validate.js \
|
||||
"$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json" \
|
||||
"$PROJECT_ROOT/.understand-anything/intermediate/review.json"
|
||||
```
|
||||
|
||||
If the script exits non-zero, read stderr, fix the script, and retry once.
|
||||
|
||||
---
|
||||
|
||||
#### `--review` path: full LLM reviewer
|
||||
|
||||
If `--review` IS in `$ARGUMENTS`, dispatch the LLM graph-reviewer subagent as follows:
|
||||
|
||||
Dispatch a subagent using the prompt template at `./graph-reviewer-prompt.md`. Read the template file and pass the full content as the subagent's prompt, appending the following additional context:
|
||||
|
||||
> **Additional context from main session:**
|
||||
>
|
||||
> Phase 1 scan results (file inventory):
|
||||
> ```json
|
||||
> [list of {path, sizeLines} from scan-result.json]
|
||||
> ```
|
||||
>
|
||||
> Phase warnings/errors accumulated during analysis:
|
||||
> - [list any batch failures, skipped files, or warnings from Phases 2-5]
|
||||
>
|
||||
> Cross-validate: every file in the scan inventory should have a corresponding `file:` node in the graph. Flag any missing files. Also flag any graph nodes whose `filePath` doesn't appear in the scan inventory.
|
||||
|
||||
Pass these parameters in the dispatch prompt:
|
||||
|
||||
> Validate the knowledge graph at `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`.
|
||||
> Project root: `$PROJECT_ROOT`
|
||||
> Read the file and validate it for completeness and correctness.
|
||||
> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/review.json`
|
||||
|
||||
---
|
||||
|
||||
4. Read `$PROJECT_ROOT/.understand-anything/intermediate/review.json`.
|
||||
|
||||
5. **If `issues` array is non-empty:**
|
||||
- Review the `issues` list
|
||||
- Apply automated fixes where possible:
|
||||
- Remove edges with dangling references
|
||||
- Fill missing required fields with sensible defaults (e.g., empty `tags` -> `["untagged"]`, empty `summary` -> `"No summary available"`)
|
||||
- Remove nodes with invalid types
|
||||
- Re-run the final graph validation after automated fixes
|
||||
- If critical issues remain after one fix attempt, save the graph anyway but include the warnings in the final report and mark dashboard auto-launch as skipped
|
||||
|
||||
6. **If `issues` array is empty:** Proceed to Phase 7.
|
||||
```
|
||||
|
||||
### Step 3: Verify the edit
|
||||
|
||||
Re-read SKILL.md lines 297–380 and confirm:
|
||||
- Phase 6 step 3 now checks for `--review` flag
|
||||
- The inline validation script is present and complete
|
||||
- The `--review` path still dispatches the LLM subagent identically to before
|
||||
- Steps 4–6 handle the `review.json` output the same way as before
|
||||
|
||||
### Step 4: Commit
|
||||
|
||||
```bash
|
||||
git add understand-anything-plugin/skills/understand/SKILL.md
|
||||
git commit -m "perf(understand): gate LLM graph-reviewer behind --review flag, add inline deterministic validation"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 2: C4a — Slim Phase 4 (architecture) node payload
|
||||
|
||||
Removes `name` and `languageNotes` from the file node format injected into the architecture-analyzer subagent. These fields are not needed for architectural layer assignment and add unnecessary tokens.
|
||||
|
||||
**Files:**
|
||||
- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 4, around line 188–196)
|
||||
|
||||
### Step 1: Locate the Phase 4 dispatch prompt in SKILL.md
|
||||
|
||||
Find the block starting "Pass these parameters in the dispatch prompt:" under Phase 4 (around line 181). Look for:
|
||||
|
||||
```
|
||||
> File nodes:
|
||||
> ```json
|
||||
> [list of {id, name, filePath, summary, tags} for all file-type nodes]
|
||||
> ```
|
||||
```
|
||||
|
||||
### Step 2: Update the file node format
|
||||
|
||||
Change the file nodes line from:
|
||||
```
|
||||
> [list of {id, name, filePath, summary, tags} for all file-type nodes]
|
||||
```
|
||||
|
||||
To:
|
||||
```
|
||||
> [list of {id, filePath, summary, tags} for all file-type nodes — omit name, complexity, languageNotes]
|
||||
```
|
||||
|
||||
### Step 3: Verify
|
||||
|
||||
Re-read Phase 4 and confirm the node format line is updated. Import edges line below it (`[list of edges with type "imports"]`) is unchanged.
|
||||
|
||||
### Step 4: Commit
|
||||
|
||||
```bash
|
||||
git add understand-anything-plugin/skills/understand/SKILL.md
|
||||
git commit -m "perf(understand): slim Phase 4 architecture payload — drop redundant node fields"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 3: C4b — Slim Phase 5 (tour builder) payload
|
||||
|
||||
Phase 5 currently injects all nodes (including function/class), all edge types, and full layer objects (with nodeIds arrays). Only file nodes, import+calls edges, and slim layers are needed for tour design. This is the largest single payload change, saving ~105,000 tokens on a 500-file project.
|
||||
|
||||
**Files:**
|
||||
- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 5, lines 257–270)
|
||||
- Modify: `understand-anything-plugin/skills/understand/tour-builder-prompt.md` (input schema)
|
||||
|
||||
### Step 1: Locate the Phase 5 dispatch prompt in SKILL.md
|
||||
|
||||
Find the block starting with (around line 257):
|
||||
```
|
||||
> Nodes (summarized):
|
||||
> ```json
|
||||
> [list of {id, name, filePath, summary, type} for key nodes]
|
||||
> ```
|
||||
>
|
||||
> Layers:
|
||||
> ```json
|
||||
> [layers from Phase 4]
|
||||
> ```
|
||||
>
|
||||
> Key edges:
|
||||
> ```json
|
||||
> [imports and calls edges]
|
||||
> ```
|
||||
```
|
||||
|
||||
### Step 2: Replace all three payload sections
|
||||
|
||||
Replace those lines with:
|
||||
|
||||
```markdown
|
||||
> Nodes (file nodes only):
|
||||
> ```json
|
||||
> [list of {id, name, filePath, summary, type} for file-type nodes ONLY — do NOT include function or class nodes]
|
||||
> ```
|
||||
>
|
||||
> Layers:
|
||||
> ```json
|
||||
> [list of {id, name, description} for each layer — omit nodeIds]
|
||||
> ```
|
||||
>
|
||||
> Edges (imports and calls only):
|
||||
> ```json
|
||||
> [list of edges where type is "imports" or "calls" only — exclude all other edge types]
|
||||
> ```
|
||||
```
|
||||
|
||||
### Step 3: Update tour-builder-prompt.md input schema
|
||||
|
||||
Open `tour-builder-prompt.md` and find the "Script Requirements" section (around line 18–35). The input schema currently shows:
|
||||
```json
|
||||
{
|
||||
"nodes": [...],
|
||||
"edges": [...],
|
||||
"layers": [
|
||||
{"id": "layer:core", "name": "Core", "nodeIds": ["file:src/index.ts"]}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Update the layers example to reflect the slim format:
|
||||
```json
|
||||
{
|
||||
"nodes": [
|
||||
{"id": "file:src/index.ts", "type": "file", "name": "index.ts", "filePath": "src/index.ts", "summary": "..."}
|
||||
],
|
||||
"edges": [
|
||||
{"source": "file:src/index.ts", "target": "file:src/utils.ts", "type": "imports"}
|
||||
],
|
||||
"layers": [
|
||||
{"id": "layer:core", "name": "Core", "description": "Core application logic"}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Also update the "G. Node Summary Index" description (around line 84) to reflect that input nodes are file-type only:
|
||||
|
||||
Find:
|
||||
```
|
||||
**G. Node Summary Index**
|
||||
|
||||
Create a lookup of each node ID to its `summary`, `type`, `tags` (default to empty array `[]` if not present in input), and `name` for easy reference.
|
||||
```
|
||||
|
||||
Add a note after it:
|
||||
```
|
||||
Note: input nodes are file-type only. The nodeSummaryIndex will contain only file nodes.
|
||||
```
|
||||
|
||||
### Step 4: Verify
|
||||
|
||||
- Re-read SKILL.md Phase 5 payload block: confirms file-only nodes, slim layers (no nodeIds), imports+calls edges only
|
||||
- Re-read tour-builder-prompt.md input schema: layers no longer have nodeIds
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add understand-anything-plugin/skills/understand/SKILL.md \
|
||||
understand-anything-plugin/skills/understand/tour-builder-prompt.md
|
||||
git commit -m "perf(understand): slim Phase 5 tour payload — file nodes only, imports+calls edges, slim layers"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 4: C3 — Remove language/framework addendums from file-analyzer batches
|
||||
|
||||
The addendums (`languages/typescript.md`, `frameworks/react.md`, etc.) are currently injected into every file-analyzer batch prompt. They cost ~1,300 tokens × N batches. The model already knows these languages. Replace with a compact inline reference table (~150 tokens, paid once, embedded in the base template).
|
||||
|
||||
**Files:**
|
||||
- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 2, lines 104–117)
|
||||
- Modify: `understand-anything-plugin/skills/understand/file-analyzer-prompt.md` (add quick reference section)
|
||||
|
||||
### Step 1: Update the "Build the combined prompt template" block in SKILL.md Phase 2
|
||||
|
||||
Find the block at lines 104–117:
|
||||
```
|
||||
**Build the combined prompt template:**
|
||||
1. Read the base template at `./file-analyzer-prompt.md`.
|
||||
2. **Language context injection:** ...
|
||||
3. **Framework addendum injection:** ...
|
||||
|
||||
Then for each batch pass the combined template content as the subagent's prompt, appending the following additional context:
|
||||
|
||||
> **Additional context from main session:**
|
||||
>
|
||||
> Project: `<projectName>` — `<projectDescription>`
|
||||
> Frameworks detected: `<frameworks from Phase 1>`
|
||||
> Languages: `<languages from Phase 1>`
|
||||
>
|
||||
> Use the language context and framework addendums (appended above) to produce more accurate summaries and better classify file roles.
|
||||
```
|
||||
|
||||
Replace it with:
|
||||
```markdown
|
||||
**Build the prompt for each batch:**
|
||||
1. Read the base template at `./file-analyzer-prompt.md`. (Language and framework hints are embedded in the template — do NOT append addendum files for Phase 2 batches. Addendums are reserved for Phase 4.)
|
||||
|
||||
Then for each batch pass the template content as the subagent's prompt, appending the following additional context:
|
||||
|
||||
> **Additional context from main session:**
|
||||
>
|
||||
> Project: `<projectName>` — `<projectDescription>`
|
||||
> Languages: `<languages from Phase 1>`
|
||||
```
|
||||
|
||||
This removes steps 2 and 3 (the addendum injection loops) entirely from Phase 2.
|
||||
|
||||
### Step 2: Add Language and Framework Quick Reference to file-analyzer-prompt.md
|
||||
|
||||
Open `file-analyzer-prompt.md`. Find the "## Critical Constraints" section near the bottom (around line 299). Insert the following new section **before** "## Critical Constraints":
|
||||
|
||||
```markdown
|
||||
## Language and Framework Quick Reference
|
||||
|
||||
Use these hints to improve tag and edge accuracy for common patterns. Your training knowledge covers these — this is a fast lookup for the most impactful signals.
|
||||
|
||||
**Tag signals:**
|
||||
|
||||
| Signal | Tags to apply |
|
||||
|---|---|
|
||||
| File in `hooks/`, exports a function starting with `use` | `hook`, `service` |
|
||||
| File in `contexts/` or `context/`, exports a Provider component | `service`, `state` |
|
||||
| File in `pages/` or `views/` | `ui`, `routing` |
|
||||
| File in `store/`, `slices/`, `reducers/`, `state/` | `state` |
|
||||
| File in `services/`, `api/`, `client/` | `service` |
|
||||
| `__init__.py` at a package root with re-exports | `entry-point`, `barrel` |
|
||||
| `manage.py` at the project root | `entry-point` |
|
||||
| `mod.rs` in a directory | `barrel` |
|
||||
| `main.go` in a `cmd/` subdirectory | `entry-point` |
|
||||
|
||||
**Edge signals:**
|
||||
|
||||
| Pattern | Edge to create |
|
||||
|---|---|
|
||||
| React component renders another component in its JSX | `contains` from parent to child |
|
||||
| Component/hook calls a custom hook (`useX`) | `depends_on` from consumer to hook file |
|
||||
| Context provider wraps components | `publishes` from provider to context definition |
|
||||
| Component calls `useContext` or custom context hook | `subscribes` from consumer to context definition |
|
||||
| Python file uses `from x import y` where x is a project file | `imports` edge (same rule as JS/TS) |
|
||||
| Go file `import`s an internal package path | `imports` edge to the resolved file |
|
||||
|
||||
```
|
||||
|
||||
### Step 3: Verify
|
||||
|
||||
- Re-read SKILL.md Phase 2 "Build the prompt" block: steps 2 and 3 (addendum loops) are gone; "Frameworks detected" line in additional context is gone
|
||||
- Re-read file-analyzer-prompt.md: new "Language and Framework Quick Reference" section appears before Critical Constraints; no reference to addendum files
|
||||
- Confirm Phase 4 "Build the combined prompt template" (lines 163–167) is **unchanged** — addendums still apply there
|
||||
|
||||
### Step 4: Commit
|
||||
|
||||
```bash
|
||||
git add understand-anything-plugin/skills/understand/SKILL.md \
|
||||
understand-anything-plugin/skills/understand/file-analyzer-prompt.md
|
||||
git commit -m "perf(understand): remove addendum injection from Phase 2 batches, add compact inline hints to file-analyzer"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 5: C1a — Extend scanner to pre-resolve imports
|
||||
|
||||
Adds a new Step 8 to the project scanner script: parse import statements from every source file and resolve relative imports against the discovered file list. The resolved map is written into `scan-result.json` as `importMap`. This is the data that lets us eliminate `allProjectFiles` from every batch in Task 7.
|
||||
|
||||
**Files:**
|
||||
- Modify: `understand-anything-plugin/skills/understand/project-scanner-prompt.md`
|
||||
|
||||
### Step 1: Add Step 8 to the scanner script requirements
|
||||
|
||||
Open `project-scanner-prompt.md`. Find "**Step 7 -- Project Name**" (around line 100). After its content (the priority list), add a new step:
|
||||
|
||||
```markdown
|
||||
**Step 8 -- Import Resolution**
|
||||
|
||||
For each file in the discovered source list, extract and resolve relative import statements. The goal is to produce a map from each file's path to the list of project-internal files it imports. External package imports are ignored.
|
||||
|
||||
For each file, read its content and extract import paths using language-appropriate patterns:
|
||||
|
||||
| Language | Import patterns to match |
|
||||
|---|---|
|
||||
| TypeScript/JavaScript | `import ... from './...'` or `'../'`, `require('./...')` or `require('../...')` |
|
||||
| Python | `from .x import y`, `from ..x import y`, `import .x` (relative only) |
|
||||
| Go | Paths in `import (...)` blocks that start with the module path from `go.mod` |
|
||||
| Rust | `use crate::`, `use super::`, `mod x` (within the same crate) |
|
||||
| Java/Kotlin | Not resolvable by path — skip import resolution for these languages |
|
||||
| Ruby | `require_relative '...'` paths |
|
||||
|
||||
For each extracted import path:
|
||||
1. Compute the resolved file path relative to project root:
|
||||
- For relative imports (`./x`, `../x`): resolve from the importing file's directory
|
||||
- Try these extension variants in order if the import has no extension: `.ts`, `.tsx`, `.js`, `.jsx`, `/index.ts`, `/index.js`, `/index.tsx`, `/index.jsx`, `.py`, `.go`, `.rs`, `.rb`
|
||||
2. Check if the resolved path exists in the discovered file list
|
||||
3. If yes: add to this file's resolved imports list
|
||||
4. If no: skip (external, unresolvable, or dynamic import)
|
||||
|
||||
Output format in the script result:
|
||||
```json
|
||||
"importMap": {
|
||||
"src/index.ts": ["src/utils.ts", "src/config.ts"],
|
||||
"src/utils.ts": [],
|
||||
"src/components/App.tsx": ["src/hooks/useAuth.ts", "src/store/index.ts"]
|
||||
}
|
||||
```
|
||||
|
||||
Keys are project-relative paths. Values are arrays of resolved project-relative paths. Every key in the file list must appear in `importMap` (use an empty array `[]` if no imports were resolved). External packages and unresolvable imports are omitted entirely.
|
||||
```
|
||||
|
||||
### Step 2: Update the scanner script output format
|
||||
|
||||
Find the "### Script Output Format" section (around line 109) and update the example JSON to include `importMap`:
|
||||
|
||||
Find this in the example:
|
||||
```json
|
||||
{
|
||||
"scriptCompleted": true,
|
||||
"name": "project-name",
|
||||
...
|
||||
"estimatedComplexity": "moderate"
|
||||
}
|
||||
```
|
||||
|
||||
Add `importMap` to the example:
|
||||
```json
|
||||
{
|
||||
"scriptCompleted": true,
|
||||
"name": "project-name",
|
||||
"rawDescription": "...",
|
||||
"readmeHead": "...",
|
||||
"languages": ["javascript", "typescript"],
|
||||
"frameworks": ["React", "Vite"],
|
||||
"files": [
|
||||
{"path": "src/index.ts", "language": "typescript", "sizeLines": 150}
|
||||
],
|
||||
"totalFiles": 42,
|
||||
"estimatedComplexity": "moderate",
|
||||
"importMap": {
|
||||
"src/index.ts": ["src/utils.ts", "src/config.ts"],
|
||||
"src/utils.ts": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Also update the field documentation list below the example to add:
|
||||
```
|
||||
- `importMap` (object) — map from every source file path to its list of resolved project-internal import paths; empty array if no resolved imports; external packages excluded
|
||||
```
|
||||
|
||||
### Step 3: Update the final assembly section to preserve importMap
|
||||
|
||||
Find "## Phase 2 -- Description and Final Assembly" (around line 153). Find the IMPORTANT note:
|
||||
```
|
||||
**IMPORTANT:** The final output must NOT contain the `scriptCompleted`, `rawDescription`, or `readmeHead` fields.
|
||||
```
|
||||
|
||||
Update it to:
|
||||
```
|
||||
**IMPORTANT:** The final output must NOT contain the `scriptCompleted`, `rawDescription`, or `readmeHead` fields. All other fields — including `importMap` — MUST be preserved exactly as output by the script.
|
||||
```
|
||||
|
||||
Also update the final output example to include `importMap`:
|
||||
```json
|
||||
{
|
||||
"name": "project-name",
|
||||
"description": "...",
|
||||
"languages": ["typescript"],
|
||||
"frameworks": ["React"],
|
||||
"files": [...],
|
||||
"totalFiles": 42,
|
||||
"estimatedComplexity": "moderate",
|
||||
"importMap": {
|
||||
"src/index.ts": ["src/utils.ts"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Step 4: Verify
|
||||
|
||||
Re-read `project-scanner-prompt.md` and confirm:
|
||||
- Step 8 is present with full import resolution logic
|
||||
- Script output format includes `importMap`
|
||||
- Field documentation includes `importMap`
|
||||
- Final assembly section preserves `importMap` in output
|
||||
|
||||
### Step 5: Commit
|
||||
|
||||
```bash
|
||||
git add understand-anything-plugin/skills/understand/project-scanner-prompt.md
|
||||
git commit -m "perf(understand): extend scanner to pre-resolve imports, output importMap in scan-result.json"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 6: C1b — Update file-analyzer to use batchImportData
|
||||
|
||||
Removes `allProjectFiles` from the file-analyzer input schema and replaces it with `batchImportData` (pre-resolved imports for this batch's files only). Updates the extraction script section to skip import resolution entirely (already done by scanner). Updates the edge creation step to use `batchImportData` directly.
|
||||
|
||||
**Files:**
|
||||
- Modify: `understand-anything-plugin/skills/understand/file-analyzer-prompt.md`
|
||||
|
||||
### Step 1: Update the input JSON schema (Script Requirements, step 1)
|
||||
|
||||
Find the input schema block around line 19:
|
||||
```json
|
||||
{
|
||||
"projectRoot": "/path/to/project",
|
||||
"allProjectFiles": ["src/index.ts", "src/utils.ts", "..."],
|
||||
"batchFiles": [
|
||||
{"path": "src/index.ts", "language": "typescript", "sizeLines": 150},
|
||||
{"path": "src/utils.ts", "language": "typescript", "sizeLines": 80}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```json
|
||||
{
|
||||
"projectRoot": "/path/to/project",
|
||||
"batchFiles": [
|
||||
{"path": "src/index.ts", "language": "typescript", "sizeLines": 150},
|
||||
{"path": "src/utils.ts", "language": "typescript", "sizeLines": 80}
|
||||
],
|
||||
"batchImportData": {
|
||||
"src/index.ts": ["src/utils.ts", "src/config.ts"],
|
||||
"src/utils.ts": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Update the field descriptions:
|
||||
- Remove: `allProjectFiles` description
|
||||
- Add: `batchImportData` (object) — map from each batch file's project-relative path to its list of pre-resolved project-internal imports. Produced by the project scanner. Use this directly for import edge creation — do NOT attempt to re-resolve imports yourself.
|
||||
|
||||
### Step 2: Remove the imports extraction from "What the Script Must Extract"
|
||||
|
||||
Find the "**Imports:**" subsection under "What the Script Must Extract" (around lines 49–53):
|
||||
```
|
||||
**Imports:**
|
||||
- Source module path (exactly as written in the import statement)
|
||||
- Imported specifiers (named imports, default import, namespace import)
|
||||
- Line number
|
||||
- For relative imports (starting with `./` or `../`), compute the resolved path...
|
||||
```
|
||||
|
||||
Replace this entire subsection with:
|
||||
```markdown
|
||||
**Imports:**
|
||||
- Do NOT extract imports in the script. Import resolution has already been performed by the project scanner.
|
||||
- The pre-resolved imports for each file are provided in `batchImportData` in the input JSON.
|
||||
- Do not include an `imports` field in the script output — import edges will be created in Phase 2 using `batchImportData` directly.
|
||||
```
|
||||
|
||||
### Step 3: Update the script output format to remove imports
|
||||
|
||||
Find the `results` array in the script output format (around line 67). The current `imports` array in the output:
|
||||
```json
|
||||
"imports": [
|
||||
{"source": "./utils", "resolvedPath": "src/utils.ts", "specifiers": ["formatDate"], "line": 1, "isExternal": false},
|
||||
{"source": "express", "resolvedPath": null, "specifiers": ["default"], "line": 2, "isExternal": true}
|
||||
],
|
||||
```
|
||||
|
||||
Remove the `imports` array from the script output format entirely. The result for each file should be:
|
||||
```json
|
||||
{
|
||||
"path": "src/index.ts",
|
||||
"language": "typescript",
|
||||
"totalLines": 150,
|
||||
"nonEmptyLines": 120,
|
||||
"functions": [...],
|
||||
"classes": [...],
|
||||
"exports": [...],
|
||||
"metrics": {
|
||||
"importCount": 5,
|
||||
"exportCount": 3,
|
||||
"functionCount": 4,
|
||||
"classCount": 1
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Keep `metrics.importCount` (derived from `batchImportData[path].length`) as a useful metric.
|
||||
|
||||
Update the metrics description to say:
|
||||
```
|
||||
- `importCount` (integer) — use `batchImportData[file.path].length` from the input JSON
|
||||
```
|
||||
|
||||
### Step 4: Update "Preparing the Script Input" section
|
||||
|
||||
Find the `cat` command around line 113 that creates the input JSON:
|
||||
```bash
|
||||
cat > $PROJECT_ROOT/.understand-anything/tmp/ua-file-analyzer-input-<batchIndex>.json << 'ENDJSON'
|
||||
{
|
||||
"projectRoot": "<project-root>",
|
||||
"allProjectFiles": [<full file list from scan>],
|
||||
"batchFiles": [<this batch's files>]
|
||||
}
|
||||
ENDJSON
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```bash
|
||||
cat > $PROJECT_ROOT/.understand-anything/tmp/ua-file-analyzer-input-<batchIndex>.json << 'ENDJSON'
|
||||
{
|
||||
"projectRoot": "<project-root>",
|
||||
"batchFiles": [<this batch's files>],
|
||||
"batchImportData": <batchImportData JSON object — provided in your dispatch prompt>
|
||||
}
|
||||
ENDJSON
|
||||
```
|
||||
|
||||
### Step 5: Update Step 3 (Create Edges) — Import edge creation rule
|
||||
|
||||
Find the "**Import edge creation rule:**" in the "Step 3 -- Create Edges" section (around line 213):
|
||||
```
|
||||
**Import edge creation rule:** For each import in the script output where `isExternal` is `false` and `resolvedPath` is non-null, create an `imports` edge from the current file node to `file:<resolvedPath>`. Do NOT create edges for external package imports.
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```markdown
|
||||
**Import edge creation rule:** For each resolved path in `batchImportData[filePath]` (provided in the input JSON), create an `imports` edge from the current file node to `file:<resolvedPath>`. The `batchImportData` values contain only resolved project-internal paths — external packages have already been filtered out. Do NOT attempt to re-resolve imports from source.
|
||||
```
|
||||
|
||||
### Step 6: Remove `allProjectFiles` references from Critical Constraints
|
||||
|
||||
Find the last bullet in "## Critical Constraints" (around line 304):
|
||||
```
|
||||
- For import edges, use the script's `resolvedPath` field directly. Do NOT attempt to resolve import paths yourself -- the script already did this deterministically.
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```markdown
|
||||
- For import edges, use `batchImportData[filePath]` directly from the input JSON. Do NOT attempt to resolve import paths yourself -- the project scanner already did this deterministically.
|
||||
```
|
||||
|
||||
### Step 7: Verify
|
||||
|
||||
Re-read `file-analyzer-prompt.md` and confirm:
|
||||
- Input schema has `batchImportData`, no `allProjectFiles`
|
||||
- Script "What to Extract" section: imports extraction replaced with "do not extract"
|
||||
- Script output format: no `imports` array per file
|
||||
- Preparing the Script Input: cat command has no `allProjectFiles`
|
||||
- Import edge creation rule: uses `batchImportData` not script output
|
||||
- Critical Constraints: no reference to `resolvedPath` from script
|
||||
|
||||
### Step 8: Commit
|
||||
|
||||
```bash
|
||||
git add understand-anything-plugin/skills/understand/file-analyzer-prompt.md
|
||||
git commit -m "perf(understand): replace allProjectFiles with batchImportData in file-analyzer — import resolution now done by scanner"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 7: C1c + C2 — Update SKILL.md Phase 2 orchestration
|
||||
|
||||
Wires up the `importMap` from Phase 1 into per-batch `batchImportData` slices. Increases batch size from 5-10 to 20-30 files. Increases concurrency from 3 to 5. Removes `allProjectFiles` from the dispatch prompt.
|
||||
|
||||
**Files:**
|
||||
- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 0, Phase 1, Phase 2)
|
||||
|
||||
### Step 1: Update Phase 1 to note importMap is now in scan-result.json
|
||||
|
||||
Find Phase 1 (around line 62) where it says:
|
||||
```
|
||||
After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermediate/scan-result.json` to get:
|
||||
- Project name, description
|
||||
- Languages, frameworks
|
||||
- File list with line counts
|
||||
- Complexity estimate
|
||||
```
|
||||
|
||||
Add one item to the list:
|
||||
```
|
||||
- Import map (`importMap`): pre-resolved project-internal imports per file
|
||||
```
|
||||
|
||||
Also add a note:
|
||||
```
|
||||
Store `importMap` in memory as `$IMPORT_MAP` for use in Phase 2 batch construction.
|
||||
```
|
||||
|
||||
### Step 2: Change batch size and concurrency in Phase 2
|
||||
|
||||
Find line 100:
|
||||
```
|
||||
Batch the file list from Phase 1 into groups of **5-10 files each** (aim for balanced batch sizes).
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```
|
||||
Batch the file list from Phase 1 into groups of **20-30 files each** (aim for ~25 files per batch for balanced sizes).
|
||||
```
|
||||
|
||||
Find line 102:
|
||||
```
|
||||
For each batch, dispatch a subagent using the prompt template at `./file-analyzer-prompt.md`. Run up to **3 subagents concurrently** using parallel dispatch.
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```
|
||||
For each batch, dispatch a subagent using the prompt template at `./file-analyzer-prompt.md`. Run up to **5 subagents concurrently** using parallel dispatch.
|
||||
```
|
||||
|
||||
### Step 3: Add batchImportData construction to the dispatch block
|
||||
|
||||
Find the dispatch prompt block (around lines 119–134):
|
||||
```
|
||||
Fill in batch-specific parameters below and dispatch:
|
||||
|
||||
> Analyze these source files and produce GraphNode and GraphEdge objects.
|
||||
> Project root: `$PROJECT_ROOT`
|
||||
> Project: `<projectName>`
|
||||
> Languages: `<languages>`
|
||||
> Batch index: `<batchIndex>`
|
||||
> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/batch-<batchIndex>.json`
|
||||
>
|
||||
> All project files (for import resolution):
|
||||
> `<full file path list from scan>`
|
||||
>
|
||||
> Files to analyze in this batch:
|
||||
> 1. `<path>` (<sizeLines> lines)
|
||||
> ...
|
||||
```
|
||||
|
||||
Replace with:
|
||||
```markdown
|
||||
Before dispatching each batch, construct `batchImportData` from `$IMPORT_MAP`:
|
||||
```json
|
||||
batchImportData = {}
|
||||
for each file in this batch:
|
||||
batchImportData[file.path] = $IMPORT_MAP[file.path] ?? []
|
||||
```
|
||||
|
||||
Fill in batch-specific parameters below and dispatch:
|
||||
|
||||
> Analyze these source files and produce GraphNode and GraphEdge objects.
|
||||
> Project root: `$PROJECT_ROOT`
|
||||
> Project: `<projectName>`
|
||||
> Languages: `<languages>`
|
||||
> Batch index: `<batchIndex>`
|
||||
> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/batch-<batchIndex>.json`
|
||||
>
|
||||
> Pre-resolved import data for this batch (use this for all import edge creation — do NOT re-resolve imports from source):
|
||||
> ```json
|
||||
> <batchImportData JSON>
|
||||
> ```
|
||||
>
|
||||
> Files to analyze in this batch:
|
||||
> 1. `<path>` (<sizeLines> lines)
|
||||
> 2. `<path>` (<sizeLines> lines)
|
||||
> ...
|
||||
```
|
||||
|
||||
### Step 4: Update incremental update path
|
||||
|
||||
Find "### Incremental update path" (around line 140):
|
||||
```
|
||||
Use the changed files list from Phase 0. Batch and dispatch file-analyzer subagents using the same process as above, but only for changed files.
|
||||
```
|
||||
|
||||
Update to clarify that batchImportData still applies:
|
||||
```
|
||||
Use the changed files list from Phase 0. Batch and dispatch file-analyzer subagents using the same process as above (20-30 files per batch, up to 5 concurrent, with batchImportData constructed from $IMPORT_MAP), but only for changed files.
|
||||
```
|
||||
|
||||
### Step 5: Verify all Phase 2 changes
|
||||
|
||||
Re-read SKILL.md Phase 2 in full and confirm:
|
||||
- Batch size says "20-30 files"
|
||||
- Concurrency says "5 subagents concurrently"
|
||||
- "Build the prompt" block: only step 1 (read base template), no addendum steps
|
||||
- Additional context block: no "Frameworks detected" line, no addendum reference
|
||||
- Dispatch prompt: has `batchImportData` injection, no `allProjectFiles`
|
||||
- Incremental path: mentions batchImportData
|
||||
|
||||
### Step 6: Commit
|
||||
|
||||
```bash
|
||||
git add understand-anything-plugin/skills/understand/SKILL.md
|
||||
git commit -m "perf(understand): wire importMap into batchImportData per batch, increase batch size 5-10→20-30, concurrency 3→5"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 8: Version bump
|
||||
|
||||
Per project convention, all four version files must stay in sync when changes are pushed.
|
||||
|
||||
**Files:**
|
||||
- Modify: `understand-anything-plugin/package.json`
|
||||
- Modify: `.claude-plugin/marketplace.json`
|
||||
- Modify: `.claude-plugin/plugin.json`
|
||||
- Modify: `.cursor-plugin/plugin.json`
|
||||
|
||||
### Step 1: Read current version
|
||||
|
||||
```bash
|
||||
node -e "const p = require('./understand-anything-plugin/package.json'); console.log(p.version)"
|
||||
```
|
||||
|
||||
Expected: `1.2.1` (or whatever the current version is).
|
||||
|
||||
### Step 2: Bump patch version in all four files
|
||||
|
||||
New version: `1.2.2` (patch bump — internal optimization, no API changes).
|
||||
|
||||
Update each file:
|
||||
- `understand-anything-plugin/package.json`: `"version": "1.2.2"`
|
||||
- `.claude-plugin/marketplace.json`: `"version": "1.2.2"` in `plugins[0]`
|
||||
- `.claude-plugin/plugin.json`: `"version": "1.2.2"`
|
||||
- `.cursor-plugin/plugin.json`: `"version": "1.2.2"`
|
||||
|
||||
### Step 3: Verify all four files match
|
||||
|
||||
```bash
|
||||
grep -r '"version"' understand-anything-plugin/package.json .claude-plugin/marketplace.json .claude-plugin/plugin.json .cursor-plugin/plugin.json
|
||||
```
|
||||
|
||||
All four should show `"version": "1.2.2"`.
|
||||
|
||||
### Step 4: Commit
|
||||
|
||||
```bash
|
||||
git add understand-anything-plugin/package.json \
|
||||
.claude-plugin/marketplace.json \
|
||||
.claude-plugin/plugin.json \
|
||||
.cursor-plugin/plugin.json
|
||||
git commit -m "chore: bump version to 1.2.2"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Task 9: Build and smoke test
|
||||
|
||||
Verifies all changes work end-to-end by running `/understand --full` against a real project.
|
||||
|
||||
**Files:** None (testing only)
|
||||
|
||||
### Step 1: Build the packages
|
||||
|
||||
```bash
|
||||
pnpm --filter @understand-anything/core build
|
||||
pnpm --filter @understand-anything/skill build
|
||||
```
|
||||
|
||||
Expected: both build without errors.
|
||||
|
||||
### Step 2: Find installed plugin version and copy to cache
|
||||
|
||||
```bash
|
||||
ls ~/.claude/plugins/cache/understand-anything/understand-anything/
|
||||
```
|
||||
|
||||
Note the version (e.g., `1.0.1`). Copy local build into the cache:
|
||||
|
||||
```bash
|
||||
VERSION=$(node -e "const p = require('./understand-anything-plugin/package.json'); console.log(p.version)")
|
||||
rm -rf ~/.claude/plugins/cache/understand-anything/understand-anything/$VERSION
|
||||
cp -R ./understand-anything-plugin ~/.claude/plugins/cache/understand-anything/understand-anything/$VERSION
|
||||
```
|
||||
|
||||
### Step 3: Smoke test on a small project (~20 files)
|
||||
|
||||
Open a fresh Claude Code session in a small TypeScript project. Run:
|
||||
```
|
||||
/understand --full
|
||||
```
|
||||
|
||||
Verify:
|
||||
- Phases 0–7 complete without errors
|
||||
- `knowledge-graph.json` is created
|
||||
- Node count and edge count are reasonable
|
||||
- Layers and tour are present
|
||||
- No "allProjectFiles" or addendum errors in the output
|
||||
|
||||
### Step 4: Smoke test on a larger project (~100+ files)
|
||||
|
||||
Run `/understand --full` on a medium/large TypeScript+React project.
|
||||
|
||||
Verify:
|
||||
- Batch count is ~4-6 (at 20-30 files per batch for 100 files), not 10-20
|
||||
- No errors about missing import resolution
|
||||
- `importMap` is present in `scan-result.json` (check `.understand-anything/intermediate/` before cleanup, or add a temporary debug log)
|
||||
- Graph quality is comparable to before (summaries are descriptive, layers are correct)
|
||||
|
||||
### Step 5: Test `--review` flag
|
||||
|
||||
Run `/understand --full --review` on the same project.
|
||||
|
||||
Verify:
|
||||
- Phase 6 now dispatches the LLM graph-reviewer subagent (not the inline script)
|
||||
- `review.json` is produced with `approved` field
|
||||
- Pipeline completes normally
|
||||
|
||||
### Step 6: Final commit (if any fixes needed from smoke test)
|
||||
|
||||
```bash
|
||||
git add -A
|
||||
git commit -m "fix(understand): smoke test fixes for token reduction changes"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
| Task | Change | Risk |
|
||||
|---|---|---|
|
||||
| 1 | C5: Gate reviewer | Low |
|
||||
| 2 | C4a: Slim Phase 4 payload | Low |
|
||||
| 3 | C4b: Slim Phase 5 payload | Low |
|
||||
| 4 | C3: Remove addendums from batches | Low |
|
||||
| 5 | C1a: Scanner import resolution | Medium |
|
||||
| 6 | C1b: File-analyzer uses batchImportData | Medium |
|
||||
| 7 | C1c+C2: SKILL.md orchestration + batch size | Medium |
|
||||
| 8 | Version bump | Low |
|
||||
| 9 | Smoke test | — |
|
||||
|
||||
Tasks 1–4 are independent of Tasks 5–7. They can be shipped separately if needed. Tasks 5, 6, and 7 are tightly coupled (scanner produces importMap → SKILL.md passes batchImportData → file-analyzer consumes it) and must be shipped together.
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "@understand-anything/skill",
|
||||
"version": "1.2.1",
|
||||
"version": "1.2.2",
|
||||
"type": "module",
|
||||
"main": "dist/index.js",
|
||||
"types": "dist/index.d.ts",
|
||||
|
||||
@@ -12,6 +12,7 @@ Analyze the current codebase and produce a `knowledge-graph.json` file in `.unde
|
||||
|
||||
- `$ARGUMENTS` may contain:
|
||||
- `--full` — Force a full rebuild, ignoring any existing graph
|
||||
- `--review` — Run full LLM graph-reviewer instead of inline deterministic validation
|
||||
- A directory path — Scope analysis to a specific subdirectory
|
||||
|
||||
---
|
||||
@@ -88,6 +89,9 @@ After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermedi
|
||||
- Languages, frameworks
|
||||
- File list with line counts
|
||||
- Complexity estimate
|
||||
- Import map (`importMap`): pre-resolved project-internal imports per file
|
||||
|
||||
Store `importMap` in memory as `$IMPORT_MAP` for use in Phase 2 batch construction.
|
||||
|
||||
**Gate check:** If >200 files, inform the user and suggest scoping with a subdirectory argument. Proceed only if user confirms or add guidance that this may take a while.
|
||||
|
||||
@@ -97,24 +101,21 @@ After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermedi
|
||||
|
||||
### Full analysis path
|
||||
|
||||
Batch the file list from Phase 1 into groups of **5-10 files each** (aim for balanced batch sizes).
|
||||
Batch the file list from Phase 1 into groups of **20-30 files each** (aim for ~25 files per batch for balanced sizes).
|
||||
|
||||
For each batch, dispatch a subagent using the prompt template at `./file-analyzer-prompt.md`. Run up to **3 subagents concurrently** using parallel dispatch.
|
||||
|
||||
**Build the combined prompt template:**
|
||||
1. Read the base template at `./file-analyzer-prompt.md`.
|
||||
2. **Language context injection:** For each language detected in Phase 1 (e.g., `python`), read the file at `./languages/<language-id>.md` (e.g., `./languages/python.md`) and append its content after the base template under a `## Language Context` header. If the file does not exist for a detected language, skip it silently and continue. These files are in the `languages/` subdirectory next to this SKILL.md file. Use `ls ./languages/` to discover available language files if needed.
|
||||
3. **Framework addendum injection:** For each framework detected in Phase 1 (e.g., `Django`), read the file at `./frameworks/<framework-id-lowercase>.md` (e.g., `./frameworks/django.md`) and append its full content after the language context. If the file does not exist for a detected framework, skip it silently and continue. These files are in the `frameworks/` subdirectory next to this SKILL.md file. Use `ls ./frameworks/` to discover available framework files if needed.
|
||||
|
||||
Then for each batch pass the combined template content as the subagent's prompt, appending the following additional context:
|
||||
For each batch, dispatch a subagent using the prompt template at `./file-analyzer-prompt.md`. Run up to **5 subagents concurrently** using parallel dispatch. Pass the template as the subagent's prompt, appending the following additional context:
|
||||
|
||||
> **Additional context from main session:**
|
||||
>
|
||||
> Project: `<projectName>` — `<projectDescription>`
|
||||
> Frameworks detected: `<frameworks from Phase 1>`
|
||||
> Languages: `<languages from Phase 1>`
|
||||
>
|
||||
> Use the language context and framework addendums (appended above) to produce more accurate summaries and better classify file roles.
|
||||
|
||||
Before dispatching each batch, construct `batchImportData` from `$IMPORT_MAP`:
|
||||
```json
|
||||
batchImportData = {}
|
||||
for each file in this batch:
|
||||
batchImportData[file.path] = $IMPORT_MAP[file.path] ?? []
|
||||
```
|
||||
|
||||
Fill in batch-specific parameters below and dispatch:
|
||||
|
||||
@@ -125,8 +126,10 @@ Fill in batch-specific parameters below and dispatch:
|
||||
> Batch index: `<batchIndex>`
|
||||
> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/batch-<batchIndex>.json`
|
||||
>
|
||||
> All project files (for import resolution):
|
||||
> `<full file path list from scan>`
|
||||
> Pre-resolved import data for this batch (use this for all import edge creation — do NOT re-resolve imports from source):
|
||||
> ```json
|
||||
> <batchImportData JSON>
|
||||
> ```
|
||||
>
|
||||
> Files to analyze in this batch:
|
||||
> 1. `<path>` (<sizeLines> lines)
|
||||
@@ -139,7 +142,7 @@ After ALL batches complete, read each `batch-<N>.json` file and merge:
|
||||
|
||||
### Incremental update path
|
||||
|
||||
Use the changed files list from Phase 0. Batch and dispatch file-analyzer subagents using the same process as above, but only for changed files.
|
||||
Use the changed files list from Phase 0. Batch and dispatch file-analyzer subagents using the same process as above (20-30 files per batch, up to 5 concurrent, with batchImportData constructed from $IMPORT_MAP), but only for changed files.
|
||||
|
||||
After batches complete, merge with the existing graph:
|
||||
1. Remove old nodes whose `filePath` matches any changed file
|
||||
@@ -187,7 +190,7 @@ Pass these parameters in the dispatch prompt:
|
||||
>
|
||||
> File nodes:
|
||||
> ```json
|
||||
> [list of {id, name, filePath, summary, tags} for all file-type nodes]
|
||||
> [list of {id, name, filePath, summary, tags} for all file-type nodes — omit complexity, languageNotes]
|
||||
> ```
|
||||
>
|
||||
> Import edges:
|
||||
@@ -254,19 +257,19 @@ Pass these parameters in the dispatch prompt:
|
||||
> Project: `<projectName>` — `<projectDescription>`
|
||||
> Languages: `<languages>`
|
||||
>
|
||||
> Nodes (summarized):
|
||||
> Nodes (file nodes only):
|
||||
> ```json
|
||||
> [list of {id, name, filePath, summary, type} for key nodes]
|
||||
> [list of {id, name, filePath, summary, type} for file-type nodes ONLY — do NOT include function or class nodes]
|
||||
> ```
|
||||
>
|
||||
> Layers:
|
||||
> ```json
|
||||
> [layers from Phase 4]
|
||||
> [list of {id, name, description} for each layer — omit nodeIds]
|
||||
> ```
|
||||
>
|
||||
> Key edges:
|
||||
> Edges (imports and calls only):
|
||||
> ```json
|
||||
> [imports and calls edges]
|
||||
> [list of edges where type is "imports" or "calls" only — exclude all other edge types]
|
||||
> ```
|
||||
|
||||
After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermediate/tour.json` and normalize it into a final `tour` array. Apply these steps **in order**:
|
||||
@@ -327,7 +330,95 @@ Assemble the full KnowledgeGraph JSON object:
|
||||
|
||||
2. Write the assembled graph to `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`.
|
||||
|
||||
3. Dispatch a subagent using the prompt template at `./graph-reviewer-prompt.md`. Read the template file and pass the full content as the subagent's prompt, appending the following additional context:
|
||||
3. **Check `$ARGUMENTS` for `--review` flag.** Then run the appropriate validation path:
|
||||
|
||||
---
|
||||
|
||||
#### Default path (no `--review`): inline deterministic validation
|
||||
|
||||
Write the following Node.js script to `$PROJECT_ROOT/.understand-anything/tmp/ua-inline-validate.cjs`:
|
||||
|
||||
```javascript
|
||||
#!/usr/bin/env node
|
||||
const fs = require('fs');
|
||||
const graphPath = process.argv[2];
|
||||
const outputPath = process.argv[3];
|
||||
try {
|
||||
const graph = JSON.parse(fs.readFileSync(graphPath, 'utf8'));
|
||||
const issues = [], warnings = [];
|
||||
if (!Array.isArray(graph.nodes)) { issues.push('graph.nodes is missing or not an array'); graph.nodes = []; }
|
||||
if (!Array.isArray(graph.edges)) { issues.push('graph.edges is missing or not an array'); graph.edges = []; }
|
||||
const nodeIds = new Set();
|
||||
const seen = new Map();
|
||||
graph.nodes.forEach((n, i) => {
|
||||
if (!n.id) { issues.push(`Node[${i}] missing id`); return; }
|
||||
if (!n.type) issues.push(`Node[${i}] '${n.id}' missing type`);
|
||||
if (!n.name) issues.push(`Node[${i}] '${n.id}' missing name`);
|
||||
if (!n.summary) issues.push(`Node[${i}] '${n.id}' missing summary`);
|
||||
if (!n.tags || !n.tags.length) issues.push(`Node[${i}] '${n.id}' missing tags`);
|
||||
if (seen.has(n.id)) issues.push(`Duplicate node ID '${n.id}' at indices ${seen.get(n.id)} and ${i}`);
|
||||
else seen.set(n.id, i);
|
||||
nodeIds.add(n.id);
|
||||
});
|
||||
graph.edges.forEach((e, i) => {
|
||||
if (!nodeIds.has(e.source)) issues.push(`Edge[${i}] source '${e.source}' not found`);
|
||||
if (!nodeIds.has(e.target)) issues.push(`Edge[${i}] target '${e.target}' not found`);
|
||||
});
|
||||
const fileNodes = graph.nodes.filter(n => n.type === 'file').map(n => n.id);
|
||||
const assigned = new Map();
|
||||
if (!Array.isArray(graph.layers)) { if (graph.layers) warnings.push('graph.layers is not an array'); graph.layers = []; }
|
||||
if (!Array.isArray(graph.tour)) { if (graph.tour) warnings.push('graph.tour is not an array'); graph.tour = []; }
|
||||
graph.layers.forEach(layer => {
|
||||
(layer.nodeIds || []).forEach(id => {
|
||||
if (!nodeIds.has(id)) issues.push(`Layer '${layer.id}' refs missing node '${id}'`);
|
||||
if (assigned.has(id)) issues.push(`Node '${id}' appears in multiple layers`);
|
||||
assigned.set(id, layer.id);
|
||||
});
|
||||
});
|
||||
fileNodes.forEach(id => {
|
||||
if (!assigned.has(id)) issues.push(`File node '${id}' not in any layer`);
|
||||
});
|
||||
graph.tour.forEach((step, i) => {
|
||||
(step.nodeIds || []).forEach(id => {
|
||||
if (!nodeIds.has(id)) issues.push(`Tour step[${i}] refs missing node '${id}'`);
|
||||
});
|
||||
});
|
||||
const withEdges = new Set([
|
||||
...graph.edges.map(e => e.source),
|
||||
...graph.edges.map(e => e.target)
|
||||
]);
|
||||
graph.nodes.forEach(n => {
|
||||
if (!withEdges.has(n.id)) warnings.push(`Node '${n.id}' has no edges (orphan)`);
|
||||
});
|
||||
const stats = {
|
||||
totalNodes: graph.nodes.length,
|
||||
totalEdges: graph.edges.length,
|
||||
totalLayers: graph.layers.length,
|
||||
tourSteps: graph.tour.length,
|
||||
nodeTypes: graph.nodes.reduce((a, n) => { a[n.type] = (a[n.type]||0)+1; return a; }, {}),
|
||||
edgeTypes: graph.edges.reduce((a, e) => { a[e.type] = (a[e.type]||0)+1; return a; }, {})
|
||||
};
|
||||
fs.writeFileSync(outputPath, JSON.stringify({ issues, warnings, stats }, null, 2));
|
||||
process.exit(0);
|
||||
} catch (err) { process.stderr.write(err.message + '\n'); process.exit(1); }
|
||||
```
|
||||
|
||||
Execute it:
|
||||
```bash
|
||||
node $PROJECT_ROOT/.understand-anything/tmp/ua-inline-validate.cjs \
|
||||
"$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json" \
|
||||
"$PROJECT_ROOT/.understand-anything/intermediate/review.json"
|
||||
```
|
||||
|
||||
If the script exits non-zero, read stderr, fix the script, and retry once.
|
||||
|
||||
---
|
||||
|
||||
#### `--review` path: full LLM reviewer
|
||||
|
||||
If `--review` IS in `$ARGUMENTS`, dispatch the LLM graph-reviewer subagent as follows:
|
||||
|
||||
Dispatch a subagent using the prompt template at `./graph-reviewer-prompt.md`. Read the template file and pass the full content as the subagent's prompt, appending the following additional context:
|
||||
|
||||
> **Additional context from main session:**
|
||||
>
|
||||
@@ -343,14 +434,16 @@ Assemble the full KnowledgeGraph JSON object:
|
||||
|
||||
Pass these parameters in the dispatch prompt:
|
||||
|
||||
> Validate the knowledge graph at `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`.
|
||||
> Project root: `$PROJECT_ROOT`
|
||||
> Read the file and validate it for completeness and correctness.
|
||||
> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/review.json`
|
||||
> Validate the knowledge graph at `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`.
|
||||
> Project root: `$PROJECT_ROOT`
|
||||
> Read the file and validate it for completeness and correctness.
|
||||
> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/review.json`
|
||||
|
||||
4. After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermediate/review.json`.
|
||||
---
|
||||
|
||||
5. **If `approved: false`:**
|
||||
4. Read `$PROJECT_ROOT/.understand-anything/intermediate/review.json`.
|
||||
|
||||
5. **If `issues` array is non-empty:**
|
||||
- Review the `issues` list
|
||||
- Apply automated fixes where possible:
|
||||
- Remove edges with dangling references
|
||||
@@ -359,7 +452,7 @@ Pass these parameters in the dispatch prompt:
|
||||
- Re-run the final graph validation after automated fixes
|
||||
- If critical issues remain after one fix attempt, save the graph anyway but include the warnings in the final report and mark dashboard auto-launch as skipped
|
||||
|
||||
6. **If `approved: true`:** Proceed to Phase 7.
|
||||
6. **If `issues` array is empty:** Proceed to Phase 7.
|
||||
|
||||
---
|
||||
|
||||
@@ -401,7 +494,7 @@ Pass these parameters in the dispatch prompt:
|
||||
## Error Handling
|
||||
|
||||
- If any subagent dispatch fails, retry **once** with the same prompt plus additional context about the failure.
|
||||
- Track all warnings and errors from each phase in a `$PHASE_WARNINGS` list. Pass this list to the graph-reviewer in Phase 6 for comprehensive validation.
|
||||
- Track all warnings and errors from each phase in a `$PHASE_WARNINGS` list. When using `--review`, pass this list to the graph-reviewer in Phase 6. On the default path, include accumulated warnings in the Phase 7 final report.
|
||||
- If it fails a second time, skip that phase and continue with partial results.
|
||||
- ALWAYS save partial results — a partial graph is better than no graph.
|
||||
- Report any skipped phases or errors in the final summary so the user knows what happened.
|
||||
|
||||
@@ -20,11 +20,14 @@ Write a script that reads each source file in your batch and extracts determinis
|
||||
```json
|
||||
{
|
||||
"projectRoot": "/path/to/project",
|
||||
"allProjectFiles": ["src/index.ts", "src/utils.ts", "..."],
|
||||
"batchFiles": [
|
||||
{"path": "src/index.ts", "language": "typescript", "sizeLines": 150},
|
||||
{"path": "src/utils.ts", "language": "typescript", "sizeLines": 80}
|
||||
]
|
||||
],
|
||||
"batchImportData": {
|
||||
"src/index.ts": ["src/utils.ts", "src/config.ts"],
|
||||
"src/utils.ts": []
|
||||
}
|
||||
}
|
||||
```
|
||||
2. **Write** results JSON to the path given as the second argument.
|
||||
@@ -45,10 +48,9 @@ For each file in `batchFiles`, read the file content and extract:
|
||||
- Detection approach: match `class <name>`, `interface <name>`, `type <name> =`, `struct <name>`, `trait <name>`, `impl <name>` as appropriate
|
||||
|
||||
**Imports:**
|
||||
- Source module path (exactly as written in the import statement)
|
||||
- Imported specifiers (named imports, default import, namespace import)
|
||||
- Line number
|
||||
- For relative imports (starting with `./` or `../`), compute the resolved path relative to project root. Cross-reference against `allProjectFiles` to confirm the resolved path exists. Mark unresolvable imports.
|
||||
- Do NOT extract imports in the script. Import resolution has already been performed by the project scanner.
|
||||
- The pre-resolved imports for each file are provided in `batchImportData` in the input JSON.
|
||||
- Do not include an `imports` field in the script output — import edges will be created in Phase 2 using `batchImportData` directly.
|
||||
|
||||
**Exports:**
|
||||
- Exported names and their line numbers
|
||||
@@ -57,7 +59,7 @@ For each file in `batchFiles`, read the file content and extract:
|
||||
**Basic Metrics:**
|
||||
- Total line count
|
||||
- Non-empty line count (lines that are not blank or comment-only)
|
||||
- Import count (number of import statements)
|
||||
- Import count — use `batchImportData[file.path].length` from the input JSON (do not count from source)
|
||||
- Export count (number of export statements)
|
||||
- Function count, class count
|
||||
|
||||
@@ -82,10 +84,6 @@ The script must write this exact JSON structure to the output file:
|
||||
"classes": [
|
||||
{"name": "App", "startLine": 50, "endLine": 140, "methods": ["init", "run"], "properties": ["config", "logger"]}
|
||||
],
|
||||
"imports": [
|
||||
{"source": "./utils", "resolvedPath": "src/utils.ts", "specifiers": ["formatDate", "sanitize"], "line": 1, "isExternal": false},
|
||||
{"source": "express", "resolvedPath": null, "specifiers": ["default"], "line": 2, "isExternal": true}
|
||||
],
|
||||
"exports": [
|
||||
{"name": "App", "line": 50, "isDefault": true},
|
||||
{"name": "createApp", "line": 145, "isDefault": false}
|
||||
@@ -114,8 +112,8 @@ Before writing the script, create its input JSON file. **IMPORTANT:** Use the ba
|
||||
cat > $PROJECT_ROOT/.understand-anything/tmp/ua-file-analyzer-input-<batchIndex>.json << 'ENDJSON'
|
||||
{
|
||||
"projectRoot": "<project-root>",
|
||||
"allProjectFiles": [<full file list from scan>],
|
||||
"batchFiles": [<this batch's files>]
|
||||
"batchFiles": [<this batch's files>],
|
||||
"batchImportData": <batchImportData JSON object — provided in your dispatch prompt>
|
||||
}
|
||||
ENDJSON
|
||||
```
|
||||
@@ -202,7 +200,7 @@ Using the script's import, export, and structural data, create edges:
|
||||
| Edge Type | When to Create | Weight | Direction |
|
||||
|---|---|---|---|
|
||||
| `contains` | File contains a function or class node you created | `1.0` | `forward` |
|
||||
| `imports` | File imports from another project file (use `resolvedPath` from script, skip external imports where `isExternal: true`) | `0.7` | `forward` |
|
||||
| `imports` | File imports from another project file (use `batchImportData[filePath]` from input JSON — external imports already filtered out) | `0.7` | `forward` |
|
||||
| `calls` | A function in this file calls a function in another file (infer from imports + function names when confident) | `0.8` | `forward` |
|
||||
| `inherits` | A class extends another class in the project | `0.9` | `forward` |
|
||||
| `implements` | A class implements an interface in the project | `0.9` | `forward` |
|
||||
@@ -210,7 +208,7 @@ Using the script's import, export, and structural data, create edges:
|
||||
| `depends_on` | File has runtime dependency on another project file (broader than imports -- includes dynamic requires, lazy loads) | `0.6` | `forward` |
|
||||
| `tested_by` | Source file is tested by a test file (infer from test file imports and naming conventions) | `0.5` | `forward` |
|
||||
|
||||
**Import edge creation rule:** For each import in the script output where `isExternal` is `false` and `resolvedPath` is non-null, create an `imports` edge from the current file node to `file:<resolvedPath>`. Do NOT create edges for external package imports.
|
||||
**Import edge creation rule:** For each resolved path in `batchImportData[filePath]` (provided in the input JSON), create an `imports` edge from the current file node to `file:<resolvedPath>`. The `batchImportData` values contain only resolved project-internal paths — external packages have already been filtered out. Do NOT attempt to re-resolve imports from source.
|
||||
|
||||
Do NOT use edge types not listed in this table.
|
||||
|
||||
@@ -295,13 +293,42 @@ Produce a single, valid JSON block. Validate it mentally before writing -- malfo
|
||||
- `direction` (string) -- always `forward`
|
||||
- `weight` (number) -- must match the weight specified in the edge type table
|
||||
|
||||
## Language and Framework Quick Reference
|
||||
|
||||
Use these hints to improve tag and edge accuracy for common patterns. Your training knowledge covers these — this is a fast lookup for the most impactful signals.
|
||||
|
||||
**Tag signals:**
|
||||
|
||||
| Signal | Tags to apply |
|
||||
|---|---|
|
||||
| File in `hooks/`, exports a function starting with `use` | `hook`, `service` |
|
||||
| File in `contexts/` or `context/`, exports a Provider component | `service`, `state` |
|
||||
| File in `pages/` or `views/` | `ui`, `routing` |
|
||||
| File in `store/`, `slices/`, `reducers/`, `state/` | `state` |
|
||||
| File in `services/`, `api/`, `client/` | `service` |
|
||||
| `__init__.py` at a package root with re-exports | `entry-point`, `barrel` |
|
||||
| `manage.py` at the project root | `entry-point` |
|
||||
| `mod.rs` in a directory | `barrel` |
|
||||
| `main.go` in a `cmd/` subdirectory | `entry-point` |
|
||||
|
||||
**Edge signals:**
|
||||
|
||||
| Pattern | Edge to create |
|
||||
|---|---|
|
||||
| React component renders another component in its JSX | `contains` from parent to child |
|
||||
| Component/hook calls a custom hook (`useX`) | `depends_on` from consumer to hook file |
|
||||
| Context provider wraps components | `exports` from provider to context definition |
|
||||
| Component calls `useContext` or custom context hook | `depends_on` from consumer to context definition |
|
||||
| Python file uses `from x import y` where x is a project file | `imports` edge (same rule as JS/TS) |
|
||||
| Go file `import`s an internal package path | `imports` edge to the resolved file |
|
||||
|
||||
## Critical Constraints
|
||||
|
||||
- NEVER invent file paths. Every `filePath` and every file reference in node IDs must correspond to a real file from the script's output or the project file list provided to you.
|
||||
- NEVER create edges to nodes that do not exist. If an import target is external (`isExternal: true` in script output), do NOT create an edge for it.
|
||||
- NEVER invent file paths. Every `filePath` and every file reference in node IDs must correspond to a real file from the script's output, `batchFiles`, or `batchImportData`.
|
||||
- NEVER create edges to nodes that do not exist. Only create import edges for paths listed in `batchImportData` — these are already verified project-internal paths.
|
||||
- ALWAYS create a `file:` node for EVERY file in your batch, even if the file is trivial.
|
||||
- Only create `function:` and `class:` nodes for significant code elements (see significance filter above).
|
||||
- For import edges, use the script's `resolvedPath` field directly. Do NOT attempt to resolve import paths yourself -- the script already did this deterministically.
|
||||
- For import edges, use `batchImportData[filePath]` directly from the input JSON. Do NOT attempt to resolve import paths yourself -- the project scanner already did this deterministically.
|
||||
- NEVER produce duplicate node IDs within your batch.
|
||||
- NEVER create self-referencing edges (where source equals target).
|
||||
- Trust the script's structural extraction. Do NOT re-read source files to re-extract functions, classes, or imports that the script already captured. Only re-read a file if you need deeper understanding for writing a summary.
|
||||
|
||||
@@ -106,6 +106,40 @@ Extract from (in priority order):
|
||||
4. `pyproject.toml` -- check `[project].name` first, then `[tool.poetry].name`
|
||||
5. Directory name of project root
|
||||
|
||||
**Step 8 -- Import Resolution**
|
||||
|
||||
For each file in the discovered source list, extract and resolve relative import statements. The goal is to produce a map from each file's path to the list of project-internal files it imports. External package imports are ignored.
|
||||
|
||||
For each file, read its content and extract import paths using language-appropriate patterns:
|
||||
|
||||
| Language | Import patterns to match |
|
||||
|---|---|
|
||||
| TypeScript/JavaScript | `import ... from './...'` or `'../'`, `require('./...')` or `require('../...')` |
|
||||
| Python | `from .x import y`, `from ..x import y`, `from . import x` (relative only) |
|
||||
| Go | Paths in `import (...)` blocks that start with the module path from `go.mod` |
|
||||
| Rust | `use crate::`, `use super::`, `mod x` (within the same crate) |
|
||||
| Java/Kotlin | Not resolvable by path — skip import resolution for these languages |
|
||||
| Ruby | `require_relative '...'` paths |
|
||||
|
||||
For each extracted import path:
|
||||
1. Compute the resolved file path relative to project root:
|
||||
- For relative imports (`./x`, `../x`): resolve from the importing file's directory
|
||||
- Try these extension variants in order if the import has no extension: `.ts`, `.tsx`, `.js`, `.jsx`, `/index.ts`, `/index.js`, `/index.tsx`, `/index.jsx`, `.py`, `.go`, `.rs`, `.rb`
|
||||
2. Check if the resolved path exists in the discovered file list
|
||||
3. If yes: add to this file's resolved imports list
|
||||
4. If no: skip (external, unresolvable, or dynamic import)
|
||||
|
||||
Output format in the script result:
|
||||
```json
|
||||
"importMap": {
|
||||
"src/index.ts": ["src/utils.ts", "src/config.ts"],
|
||||
"src/utils.ts": [],
|
||||
"src/components/App.tsx": ["src/hooks/useAuth.ts", "src/store/index.ts"]
|
||||
}
|
||||
```
|
||||
|
||||
Keys are project-relative paths. Values are arrays of resolved project-relative paths. Every key in the file list must appear in `importMap` (use an empty array `[]` if no imports were resolved). External packages and unresolvable imports are omitted entirely.
|
||||
|
||||
### Script Output Format
|
||||
|
||||
The script must write this exact JSON structure to the output file:
|
||||
@@ -122,7 +156,11 @@ The script must write this exact JSON structure to the output file:
|
||||
{"path": "src/index.ts", "language": "typescript", "sizeLines": 150}
|
||||
],
|
||||
"totalFiles": 42,
|
||||
"estimatedComplexity": "moderate"
|
||||
"estimatedComplexity": "moderate",
|
||||
"importMap": {
|
||||
"src/index.ts": ["src/utils.ts", "src/config.ts"],
|
||||
"src/utils.ts": []
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -135,6 +173,7 @@ The script must write this exact JSON structure to the output file:
|
||||
- `files` (object[]) -- every source file, sorted by `path` alphabetically
|
||||
- `totalFiles` (integer) -- must equal `files.length`
|
||||
- `estimatedComplexity` (string) -- one of `small`, `moderate`, `large`, `very-large`
|
||||
- `importMap` (object) — map from every source file path to its list of resolved project-internal import paths; empty array if no resolved imports; external packages excluded
|
||||
|
||||
### Executing the Script
|
||||
|
||||
@@ -154,7 +193,7 @@ If the script exits with a non-zero code, read stderr, diagnose the issue, fix t
|
||||
|
||||
After the script completes, read `$PROJECT_ROOT/.understand-anything/tmp/ua-scan-results.json`. Do NOT re-run file discovery commands or re-count lines -- trust the script's results entirely.
|
||||
|
||||
**IMPORTANT:** The final output must NOT contain the `scriptCompleted`, `rawDescription`, or `readmeHead` fields. These are intermediate script fields only. Strip them when assembling the final JSON.
|
||||
**IMPORTANT:** The final output must NOT contain the `scriptCompleted`, `rawDescription`, or `readmeHead` fields. These are intermediate script fields only. Strip them when assembling the final JSON. All other fields — including `importMap` — MUST be preserved exactly as output by the script.
|
||||
|
||||
Your only task in this phase is to produce the final `description` field:
|
||||
|
||||
@@ -175,7 +214,10 @@ Then assemble the final output JSON:
|
||||
{"path": "src/index.ts", "language": "typescript", "sizeLines": 150}
|
||||
],
|
||||
"totalFiles": 42,
|
||||
"estimatedComplexity": "moderate"
|
||||
"estimatedComplexity": "moderate",
|
||||
"importMap": {
|
||||
"src/index.ts": ["src/utils.ts"]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
@@ -187,6 +229,7 @@ Then assemble the final output JSON:
|
||||
- `files` (object[]): directly from script output
|
||||
- `totalFiles` (integer): directly from script output
|
||||
- `estimatedComplexity` (string): directly from script output
|
||||
- `importMap` (object): directly from script output
|
||||
|
||||
## Critical Constraints
|
||||
|
||||
|
||||
@@ -20,13 +20,13 @@ Write a Node.js script that analyzes the graph's topology to surface structural
|
||||
```json
|
||||
{
|
||||
"nodes": [
|
||||
{"id": "file:src/index.ts", "type": "file", "name": "index.ts", "filePath": "src/index.ts", "summary": "...", "tags": ["entry-point"]}
|
||||
{"id": "file:src/index.ts", "type": "file", "name": "index.ts", "filePath": "src/index.ts", "summary": "..."}
|
||||
],
|
||||
"edges": [
|
||||
{"source": "file:src/index.ts", "target": "file:src/utils.ts", "type": "imports"}
|
||||
],
|
||||
"layers": [
|
||||
{"id": "layer:core", "name": "Core", "nodeIds": ["file:src/index.ts"]}
|
||||
{"id": "layer:core", "name": "Core", "description": "Core application logic"}
|
||||
]
|
||||
}
|
||||
```
|
||||
@@ -47,7 +47,6 @@ For every node, count how many other nodes it has edges pointing TO (fan-out). H
|
||||
|
||||
Identify likely entry points using these signals (score each file node, sum the scores):
|
||||
- Filename matches `index.ts`, `index.js`, `main.ts`, `main.js`, `app.ts`, `app.js`, `server.ts`, `server.js`, `mod.rs`, `main.go`, `main.py`, `main.rs`, `manage.py`, `app.py`, `wsgi.py`, `asgi.py`, `run.py`, `__main__.py`, `Application.java`, `Main.java`, `Program.cs`, `config.ru`, `index.php`, `App.swift`, `Application.kt`, `main.cpp`, `main.c` -> +3 points
|
||||
- Node tags contain `entry-point` or `barrel` -> +2 points
|
||||
- File is at the project root or one level deep (e.g., `src/index.ts`) -> +1 point
|
||||
- High fan-out (top 10%) -> +1 point
|
||||
- Low fan-in (bottom 25%) -> +1 point (entry points are imported by few files)
|
||||
@@ -71,17 +70,15 @@ Algorithm: For each pair of nodes with a bidirectional relationship (A imports B
|
||||
|
||||
Output the top 5-10 clusters, each as a list of node IDs.
|
||||
|
||||
**F. Layer Statistics**
|
||||
**F. Layer List**
|
||||
|
||||
For each layer, compute:
|
||||
- Number of file nodes
|
||||
- Average fan-in of files in this layer
|
||||
- Average fan-out of files in this layer
|
||||
- The layer's "rank" in the dependency hierarchy (layers that are imported by many others but import few = foundational; layers that import many others but are imported by few = top-level)
|
||||
Record the layers provided in the input. Since layers contain only `{id, name, description}` (no node membership), simply output the layer count and the list of layers with their id, name, and description.
|
||||
|
||||
**G. Node Summary Index**
|
||||
|
||||
Create a lookup of each node ID to its `summary`, `type`, `tags` (default to empty array `[]` if not present in input), and `name` for easy reference. This lets the LLM phase quickly access semantic information without re-reading the full input.
|
||||
Create a lookup of each node ID to its `summary`, `type`, and `name` for easy reference. This lets the LLM phase quickly access semantic information without re-reading the full input.
|
||||
|
||||
Note: input nodes are file-type only. The nodeSummaryIndex will contain only file nodes.
|
||||
|
||||
### Script Output Format
|
||||
|
||||
@@ -114,15 +111,19 @@ Create a lookup of each node ID to its `summary`, `type`, `tags` (default to emp
|
||||
"clusters": [
|
||||
{"nodes": ["file:src/services/auth.ts", "file:src/models/user.ts"], "edgeCount": 4}
|
||||
],
|
||||
"layerStats": [
|
||||
{"id": "layer:core", "name": "Core", "fileCount": 5, "avgFanIn": 8.2, "avgFanOut": 3.1, "hierarchyRank": 1}
|
||||
],
|
||||
"layers": {
|
||||
"count": 3,
|
||||
"list": [
|
||||
{"id": "layer:core", "name": "Core", "description": "Core application logic"},
|
||||
{"id": "layer:services", "name": "Services", "description": "Business logic services"},
|
||||
{"id": "layer:ui", "name": "UI", "description": "User interface components"}
|
||||
]
|
||||
},
|
||||
"nodeSummaryIndex": {
|
||||
"file:src/index.ts": {"name": "index.ts", "type": "file", "summary": "Main entry point...", "tags": ["entry-point"]},
|
||||
"file:src/utils.ts": {"name": "utils.ts", "type": "file", "summary": "Shared helpers...", "tags": []}
|
||||
"file:src/index.ts": {"name": "index.ts", "type": "file", "summary": "Main entry point..."},
|
||||
"file:src/utils.ts": {"name": "utils.ts", "type": "file", "summary": "Shared helpers..."}
|
||||
},
|
||||
"totalNodes": 42,
|
||||
"totalFileNodes": 20,
|
||||
"totalEdges": 87
|
||||
}
|
||||
```
|
||||
@@ -179,13 +180,13 @@ You do not need to include every node from the BFS. Select the most important an
|
||||
|
||||
When a `cluster` from the script output appears at the same BFS depth, group those nodes into a single tour step. Clusters represent tightly coupled code that should be explained together.
|
||||
|
||||
### Step 4 -- Use Layer Statistics for Narrative Arc
|
||||
### Step 4 -- Use Layers for Narrative Arc
|
||||
|
||||
The `layerStats` with `hierarchyRank` tells you which layers are foundational vs. top-level. Structure the tour to explain foundational layers before the layers that depend on them.
|
||||
The `layers` list gives you the project's architectural groupings. Use layer names and descriptions to understand which areas are foundational vs. top-level, and structure the tour to explain foundational layers before the layers that depend on them.
|
||||
|
||||
### Step 5 -- Write Step Descriptions
|
||||
|
||||
For each step, use the `nodeSummaryIndex` to access node summaries, names, and tags without re-reading files. Each description must:
|
||||
For each step, use the `nodeSummaryIndex` to access node summaries and names without re-reading files. Each description must:
|
||||
|
||||
- Explain WHAT this area does and WHY it matters to the project
|
||||
- Connect to previous steps (e.g., "Building on the User types from Step 2, this service implements...")
|
||||
|
||||
Reference in New Issue
Block a user