diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 140122a..6f2845b 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -9,7 +9,7 @@ { "name": "understand-anything", "description": "Multi-agent codebase analysis with interactive dashboard, guided tours, and skill commands", - "version": "1.2.1", + "version": "1.2.2", "source": "./understand-anything-plugin" } ] diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 5cabce8..375688e 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,7 +1,7 @@ { "name": "understand-anything", "description": "AI-powered codebase understanding — analyze, visualize, and explain any project", - "version": "1.2.1", + "version": "1.2.2", "author": { "name": "Lum1104" }, diff --git a/.cursor-plugin/plugin.json b/.cursor-plugin/plugin.json index 41f5046..76707fd 100644 --- a/.cursor-plugin/plugin.json +++ b/.cursor-plugin/plugin.json @@ -2,7 +2,7 @@ "name": "understand-anything", "displayName": "Understand Anything", "description": "AI-powered codebase understanding — analyze, visualize, and explain any project", - "version": "1.2.1", + "version": "1.2.2", "author": { "name": "Lum1104" }, diff --git a/README.md b/README.md index ea6d0ff..999fb49 100644 --- a/README.md +++ b/README.md @@ -237,9 +237,9 @@ The `/understand` command orchestrates 5 specialized agents: | `file-analyzer` | Extract functions, classes, imports; produce graph nodes and edges | | `architecture-analyzer` | Identify architectural layers | | `tour-builder` | Generate guided learning tours | -| `graph-reviewer` | Validate graph completeness and referential integrity | +| `graph-reviewer` | Validate graph completeness and referential integrity (runs inline by default; use `--review` for full LLM review) | -File analyzers run in parallel (up to 3 concurrent). Supports incremental updates — only re-analyzes files that changed since the last run. +File analyzers run in parallel (up to 5 concurrent, 20-30 files per batch). Supports incremental updates — only re-analyzes files that changed since the last run. ### Project Structure diff --git a/docs/plans/2026-03-27-token-reduction-design.md b/docs/plans/2026-03-27-token-reduction-design.md new file mode 100644 index 0000000..8bf8975 --- /dev/null +++ b/docs/plans/2026-03-27-token-reduction-design.md @@ -0,0 +1,395 @@ +# Token Reduction Design + +**Date:** 2026-03-27 +**Status:** Draft +**Goal:** Reduce total token cost of `/understand` by ~85-90% on large codebases (200+ files) + +--- + +## Problem + +For large codebases, the `/understand` pipeline spends the vast majority of its tokens on **repeated context injection**. The same data is sent to every subagent independently, even when that data could be computed once and shared. + +### Token cost breakdown (500-file TypeScript+React project, baseline) + +| Source | Phase | Tokens (input) | % of total | +|---|---|---|---| +| `allProjectFiles` list × 67 batches | Phase 2 | ~167,000 | ~50% | +| `file-analyzer-prompt.md` × 67 batches | Phase 2 | ~134,000 | ~40% | +| Language/framework addendums × 67 batches | Phase 2 | ~68,000 | ~20% | +| Tour builder payload (all nodes + edges) | Phase 5 | ~80,000 | ~24% | +| Graph reviewer (assembled graph + inventory) | Phase 6 | ~58,000 | ~17% | +| Architecture analyzer payload | Phase 4 | ~22,000 | ~7% | +| **Total** | | **~529,000** | | + +The root cause: **Phase 2 runs 67 batches (at 5-10 files each), and every single batch receives the full 500-file list for import resolution.** The file list alone costs ~2,500 tokens × 67 repetitions = 167,000 tokens on input, doing work that is entirely redundant between batches. + +--- + +## Goals + +- Reduce total input tokens by 85%+ on a 500-file project +- No degradation in graph quality for standard projects +- Preserve the `--full` / incremental / scope flags +- Maintain backward compatibility with existing `knowledge-graph.json` output schema + +--- + +## Changes + +Five changes compose the full approach (C1–C5). Each is independent and can be shipped separately, but all five are needed for the full reduction. + +--- + +### C1 — Pre-resolve imports in the project scanner + +**Root cause addressed:** `allProjectFiles` (the entire file list) is injected into every file-analyzer batch solely so each batch's extraction script can resolve relative imports. This is redundant: the full file list is available during Phase 1, and import resolution is deterministic. It should happen once, not 67 times. + +**Change:** Extend the Phase 1 scanner script to also parse import statements from every source file and resolve relative imports against the discovered file list. The resolved results are written into `scan-result.json` as a new `importMap` field. File-analyzer batches then receive only their own batch's pre-resolved imports — not the full file list. + +#### Scanner output addition + +`scan-result.json` gains: + +```json +{ + "importMap": { + "src/index.ts": ["src/utils.ts", "src/config.ts"], + "src/utils.ts": [], + "src/components/App.tsx": ["src/hooks/useAuth.ts", "src/store/index.ts"] + } +} +``` + +- Keys are project-relative paths (matching `files[*].path`) +- Values are resolved project-relative paths only (external/unresolvable imports are omitted) +- External imports (`node_modules`, unresolvable paths) are excluded from the map entirely + +#### Scanner script additions (Phase 1 Step 8) + +After the existing 7 steps, the scanner script adds a new step: + +``` +Step 8 — Import Resolution + +For each file in the discovered source list: + 1. Read the file content + 2. Extract import statements (language-specific patterns per Step 3's language detection): + - TypeScript/JavaScript: `import ... from '...'`, `require('...')` + - Python: `import ...`, `from ... import ...` + - Go: `import "..."` blocks + - Rust: `use ...` statements + - Java/Kotlin: `import ...` statements + - Ruby: `require`, `require_relative` + 3. For each relative import (starts with `./` or `../`): + a. Compute the resolved path from the current file's directory + b. Normalize to project-relative format + c. Try common extension variants if the import has no extension: + `.ts`, `.tsx`, `.js`, `.jsx`, `/index.ts`, `/index.js`, `/index.tsx` + d. If any variant exists in the discovered file list, record it; otherwise skip + 4. For absolute imports (no `.` prefix): skip (external package) + +Output the full importMap in the JSON result. +``` + +#### File-analyzer input schema change + +**Before:** +```json +{ + "projectRoot": "/path/to/project", + "allProjectFiles": ["src/index.ts", "src/utils.ts", "...500 paths..."], + "batchFiles": [ + {"path": "src/index.ts", "language": "typescript", "sizeLines": 150} + ] +} +``` + +**After:** +```json +{ + "projectRoot": "/path/to/project", + "batchFiles": [ + {"path": "src/index.ts", "language": "typescript", "sizeLines": 150} + ], + "batchImportData": { + "src/index.ts": ["src/utils.ts", "src/config.ts"], + "src/components/App.tsx": ["src/hooks/useAuth.ts"] + } +} +``` + +`allProjectFiles` is removed entirely. `batchImportData` contains only the pre-resolved imports for the files in this batch (sliced from `importMap` by the orchestrator). + +#### File-analyzer extraction script change + +The extraction script no longer performs import resolution. It: +- Still extracts: functions, classes, exports, metrics (unchanged) +- For imports: reads `batchImportData[file.path]` from the input JSON — no cross-referencing needed +- The `imports` array in each file result becomes: `batchImportData[file.path]` mapped to import edge objects with `resolvedPath` already populated, `isExternal: false` + +#### SKILL.md Phase 2 change + +Remove the `allProjectFiles` injection from the batch dispatch prompt. Replace with a per-batch `batchImportData` slice: + +``` +For each batch, slice importData from the importMap read in Phase 1: +batchImportData = { [file.path]: importMap[file.path] ?? [] } + for each file in this batch +``` + +#### Token savings estimate + +| | Batches | Tokens/batch | Total | +|---|---|---|---| +| Before | 67 | ~2,500 (file list) | ~167,500 | +| After (C1 alone) | 67 | ~200 (batch importData) | ~13,400 | +| **Savings** | | | **~154,100** | + +--- + +### C2 — Increase batch size from 5-10 to 20-30 files + +**Root cause addressed:** Every batch incurs the full cost of `file-analyzer-prompt.md` (~2,000 tokens) plus the batch dispatch overhead. With 67 batches, this adds up even without `allProjectFiles`. Fewer, larger batches directly reduce this repetition. + +**Change:** In SKILL.md Phase 2, change the batch size guidance: + +- **Before:** "Batch the file list from Phase 1 into groups of **5-10 files each**" +- **After:** "Batch the file list from Phase 1 into groups of **20-30 files each** (aim for ~25 per batch)" + +Also update the concurrency limit from 3 to **5** concurrent batches. Fewer total batches means we can afford more parallelism without overwhelming the system. + +#### Trade-offs + +| | Smaller batches (current) | Larger batches (new) | +|---|---|---| +| Files per batch | 5-10 | 20-30 | +| Total batches (500 files) | ~67 | ~20 | +| Prompt repetition | 67× | 20× | +| Quality risk | Lower (focused) | Slightly higher (more files per subagent) | +| Concurrency | 3 | 5 | + +Quality risk is low: each subagent still operates on distinct, non-overlapping file groups. The extraction script is deterministic regardless of batch size. Semantic analysis (summaries, tags) may be marginally less focused, but the quality difference is negligible in practice for well-structured files. + +#### Token savings estimate (combined with C1) + +| | Batches | Tokens/batch (prompt) | Total | +|---|---|---|---| +| Before (C1 only) | 67 | ~2,000 | ~134,000 | +| After (C1+C2) | 20 | ~2,000 | ~40,000 | +| **Savings from C2** | | | **~94,000** | + +C1+C2 combined eliminate ~248,000 tokens from Phase 2 (down from ~301,500 to ~53,500, a ~82% Phase 2 reduction). + +--- + +### C3 — Remove language/framework addendums from file-analyzer batches + +**Root cause addressed:** `languages/typescript.md` (~600 tokens) and `frameworks/react.md` (~700 tokens) are read and injected into every file-analyzer batch prompt. For a TypeScript+React project with 20 batches (after C2), this costs 20 × 1,300 = 26,000 additional tokens — and the model already has deep knowledge of these languages from training. + +**Change:** Stop injecting addendum files into Phase 2 batch prompts entirely. The addendums remain injected into Phase 4 (architecture analyzer) where there is only **one** subagent call, making the cost acceptable. + +Instead, add a compact "Language and Framework Hints" reference section directly into `file-analyzer-prompt.md`. This section is a distilled, one-time addition (~150 tokens total) that captures the most useful patterns from all addendums in a concise lookup table. + +#### New section in `file-analyzer-prompt.md` (replace addendum injection) + +```markdown +## Language and Framework Quick Reference + +Use these hints to improve tag and edge accuracy. These supplement your training knowledge. + +| Signal | Tag(s) | Note | +|---|---|---| +| File in `hooks/`, exports function starting with `use` | `hook`, `service` | React custom hook | +| File in `contexts/`, exports a Provider | `service`, `state` | React context | +| File in `pages/` or `views/` | `ui`, `routing` | Page-level component | +| File in `store/`, `slices/`, `reducers/` | `state` | State management | +| File in `services/`, `api/` | `service` | Data-fetching / API client | +| `__init__.py` with re-exports | `entry-point`, `barrel` | Python package root | +| `manage.py` at project root | `entry-point` | Django management entry | +| File named `mod.rs` | `barrel` | Rust module barrel | +| File named `main.go` in `cmd/` | `entry-point` | Go binary entry | + +For React: create `depends_on` edges from components to hooks they call. Create `publishes`/`subscribes` edges for Context provider/consumer patterns. +``` + +#### SKILL.md Phase 2 change + +Remove steps 2 and 3 from the "Build the combined prompt template" block: +- **Remove:** Step 2 (Language context injection — read `./languages/.md` per detected language) +- **Remove:** Step 3 (Framework addendum injection — read `./frameworks/.md` per detected framework) +- **Keep:** Step 1 (Read the base template at `./file-analyzer-prompt.md`) + +The addendum injection steps **remain unchanged** in Phase 4 (architecture analyzer), since they run once. + +#### Token savings estimate + +| | Batches | Addendum tokens/batch | Total | +|---|---|---|---| +| Before (after C2) | 20 | ~1,300 (TS+React) | ~26,000 | +| After | 20 | ~150 (inline hints) | ~3,000 | +| **Savings** | | | **~23,000** | + +--- + +### C4 — Slim Phase 4 and Phase 5 payloads + +**Root cause addressed:** Phase 5 (tour builder) receives all nodes (file + function + class) and all edges (imports + contains + calls + exports + ...). For a 500-file project, this can include 1,500+ nodes and 3,000+ edges. Most of this data is not needed for tour design. + +#### Phase 4 (Architecture Analyzer) — minor trim + +Phase 4 already only sends file-type nodes, which is correct. Minor change: explicitly strip `languageNotes` from each node object in the payload (it's not useful for layer assignment and can be verbose). Also strip `name` — it is always derivable as the basename of `filePath`. + +**Before per node:** `{id, name, filePath, summary, tags, complexity, languageNotes?}` +**After per node:** `{id, filePath, summary, tags}` + +Savings: ~15-20% fewer tokens per node, ~3,000–5,000 tokens total for Phase 4. + +#### Phase 5 (Tour Builder) — major trim + +Three changes to what the orchestrator injects into the tour-builder subagent: + +**1. File nodes only (strip function/class nodes)** + +The tour references node IDs for wayfinding. In practice the tour always references `file:` nodes — function and class nodes are visible in the dashboard's NodeInfo sidebar once a file is selected, but the tour itself navigates at the file level. + +- **Before:** all nodes (file + function + class) — for 500 files, maybe 1,500+ nodes +- **After:** file-type nodes only — 500 nodes + +**2. Slim node format** + +The tour builder script only uses node IDs, names, and types for graph computation. Summaries and tags are used in Phase 2 (pedagogical narrative writing). Strip heavy optional fields from the injected payload: + +- **Before per node:** `{id, name, filePath, summary, type, tags, complexity, languageNotes?}` +- **After per node:** `{id, name, filePath, summary, type}` (drop tags, complexity, languageNotes) + +**3. Slim edges (imports + calls only) and slim layers** + +The tour's BFS traversal only traverses `imports` and `calls` edges. `contains`, `exports`, `tested_by`, `depends_on`, and other edge types add no value to the traversal and inflate the payload. + +- **Before edges:** all edge types (~3,000+ edges including all `contains` edges to function/class nodes) +- **After edges:** only `imports` and `calls` edge types (~400–800 edges for typical projects) + +For layers, the tour builder uses layer data only to inform the tour's narrative arc (which layer to introduce first, second, etc.). It does not need the full `nodeIds` arrays — those can be very large. + +- **Before per layer:** `{id, name, description, nodeIds: [...hundreds of IDs]}` +- **After per layer:** `{id, name, description}` (drop nodeIds) + +#### Token savings estimate (Phase 5) + +| Data | Before | After | +|---|---|---| +| Node count | ~1,500 × ~180 chars | ~500 × ~120 chars | +| Node tokens | ~67,500 | ~15,000 | +| Edge count | ~3,000 × ~80 chars | ~600 × ~80 chars | +| Edge tokens | ~60,000 | ~12,000 | +| Layer tokens | ~5,000 | ~500 | +| **Phase 5 total** | **~132,500** | **~27,500** | +| **Savings** | | **~105,000** | + +#### SKILL.md changes + +In **Phase 4** dispatch prompt template, update the file node format: +``` +File nodes: +[list of {id, filePath, summary, tags} for all file-type nodes] +``` + +In **Phase 5** dispatch prompt template, update all three payload specs: +``` +Nodes (file nodes only): +[list of {id, name, filePath, summary, type} for all file-type nodes only — do NOT include function or class nodes] + +Key edges (imports and calls only): +[list of edges where type is "imports" or "calls" only] + +Layers: +[list of {id, name, description} — omit nodeIds] +``` + +--- + +### C5 — Gate the graph-reviewer subagent behind `--review` + +**Root cause addressed:** The graph-reviewer subagent (Phase 6) reads the entire assembled graph (~500 nodes, all edges, layers, tour) and runs a LLM-powered validation. However, its Phase 1 is entirely a deterministic script, and its Phase 2 is a simple threshold decision: if `issues.length === 0`, approve. There is no LLM judgment needed for the happy path. + +**Change:** By default, skip the graph-reviewer subagent. The orchestrator performs inline deterministic validation using a pre-written script. Only when `--review` is explicitly passed in `$ARGUMENTS` does the full LLM reviewer subagent run. + +#### Default path (no `--review`) + +In Phase 6, instead of dispatching the graph-reviewer subagent, the orchestrator: + +1. Writes a compact validation script inline (embedded in SKILL.md, ~50 lines of Node.js): + - Check: every edge source/target references a real node ID + - Check: every file node appears in exactly one layer + - Check: every tour step nodeId exists + - Check: no duplicate node IDs + - Check: required fields present on nodes and edges +2. Runs the script against `assembled-graph.json` +3. If `issues.length === 0`: proceed to Phase 7 (save) +4. If `issues.length > 0`: apply the same automated fixes as before (remove dangling edges, fill defaults), then save + +This is sufficient for standard runs. The LLM reviewer adds value for catching subtle quality issues (generic summaries, orphan nodes, tour step coherence) — but those are nice-to-have, not blocking. + +#### `--review` path + +When `--review` is in `$ARGUMENTS`, the full graph-reviewer subagent runs as it does today. No change to that code path. + +#### Token savings estimate + +| Path | Tokens | +|---|---| +| Current (always runs LLM reviewer) | ~58,000 input + ~500 output | +| Default (inline script, no LLM) | ~0 | +| `--review` (unchanged) | ~58,000 (same as current) | +| **Savings for default runs** | **~58,500** | + +--- + +## Combined savings summary + +| Change | Tokens before | Tokens after | Savings | +|---|---|---|---| +| C1+C2: import map + batch consolidation | ~301,500 | ~53,500 | ~248,000 | +| C3: remove addendums from batches | ~26,000 | ~3,000 | ~23,000 | +| C4: slim Phase 4+5 payloads | ~154,500 | ~33,000 | ~121,500 | +| C5: gate reviewer (default path) | ~58,500 | ~0 | ~58,500 | +| **Total** | **~540,500** | **~89,500** | **~451,000 (~83%)** | + +Estimates are for a 500-file TypeScript+React project. Actual savings scale with project size — a 1,000-file project would see proportionally larger savings from C1+C2 (more batches = more repetition eliminated). + +--- + +## File changes + +| File | Change | +|---|---| +| `skills/understand/project-scanner-prompt.md` | Add Step 8 (import resolution); add `importMap` to output schema | +| `skills/understand/file-analyzer-prompt.md` | Replace `allProjectFiles` with `batchImportData` in input schema; update extraction script to use pre-resolved imports; add compact Language/Framework Quick Reference section; remove addendum injection steps | +| `skills/understand/SKILL.md` | Phase 1: note importMap in scan result; Phase 2: remove addendum injection (steps 2+3), increase batch size 5-10→20-30, increase concurrency 3→5, replace `allProjectFiles` injection with `batchImportData` slice; Phase 4: slim node format in dispatch; Phase 5: file nodes only + slim edges + slim layers in dispatch; Phase 6: conditional reviewer — default inline script, `--review` flag for LLM reviewer | +| `skills/understand/architecture-analyzer-prompt.md` | No change (addendums still injected here) | +| `skills/understand/tour-builder-prompt.md` | Update input schema to reflect file-only nodes, imports+calls-only edges, slim layer format | +| `skills/understand/graph-reviewer-prompt.md` | No change (only used when `--review` flag is passed) | + +--- + +## Risks and mitigations + +| Risk | Likelihood | Mitigation | +|---|---|---| +| Scanner import resolution misses edge cases (complex re-exports, dynamic imports) | Medium | Log unresolved imports; file-analyzer still uses resolved data and creates edges only for confirmed matches. Missed imports = missing edges, which is same behavior as before for unresolvable imports | +| Larger batches (C2) reduce summary quality | Low | Summary quality is driven by the model's analysis of individual files. Batch size mainly affects how many files share one subagent's context window, not per-file quality. 20-30 files remains well within context limits | +| Stripping function/class nodes from tour (C4) breaks existing tour steps | None | Tour steps reference `file:` node IDs. No existing tour data references function/class nodes at the step level | +| Removing reviewer by default (C5) misses graph errors | Low | The inline deterministic script catches all critical structural issues (dangling refs, missing layers, duplicate IDs). The LLM reviewer's additional value is quality warnings (orphan nodes, generic summaries), which are non-blocking | +| Import map generation slows down Phase 1 | Low | The scanner script already reads all files for line counting. Import parsing adds one regex pass per file — negligible overhead | + +--- + +## Phased rollout recommendation + +Given the risk profile, implement in this order: + +1. **C5 first** — gate the reviewer, lowest risk, immediate 58K token savings per run +2. **C4** — slim Phase 5 payload, no scanner changes, no quality risk +3. **C3** — remove addendums from batches, add inline hints +4. **C1+C2 together** — scanner changes and batch consolidation, test thoroughly on small/medium/large projects before releasing diff --git a/docs/plans/2026-03-27-token-reduction-impl.md b/docs/plans/2026-03-27-token-reduction-impl.md new file mode 100644 index 0000000..848276b --- /dev/null +++ b/docs/plans/2026-03-27-token-reduction-impl.md @@ -0,0 +1,971 @@ +# Token Reduction Implementation Plan + +> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task. + +**Goal:** Reduce `/understand` token cost by ~85% on large codebases through import pre-resolution, batch consolidation, addendum removal, payload slimming, and gating the LLM reviewer. + +**Architecture:** Five changes (C5 → C4 → C3 → C1+C2) applied in rollout order — lowest risk first. All changes are to prompt/skill markdown files in `understand-anything-plugin/skills/understand/`. No TypeScript source changes required. + +**Tech Stack:** Markdown skill files, Node.js inline scripts embedded in SKILL.md, knowledge-graph JSON pipeline. + +**Design doc:** `docs/plans/2026-03-27-token-reduction-design.md` + +--- + +## Task 1: C5 — Gate graph-reviewer behind `--review` flag + +Replaces the always-on LLM graph-reviewer subagent with a deterministic inline validation script. The LLM reviewer only runs when `--review` is in `$ARGUMENTS`. Saves ~58,500 tokens per default run. + +**Files:** +- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 6, lines 330–362) + +### Step 1: Open SKILL.md and locate Phase 6 + +Read the file and find "## Phase 6 — REVIEW" (line 297). Identify steps 3–6 (lines 330–362) which currently always dispatch the LLM graph-reviewer subagent. + +### Step 2: Replace Phase 6 steps 3–6 with conditional reviewer logic + +Replace lines 330–362 (from "3. Dispatch a subagent using the prompt template" through "6. **If `approved: true`:** Proceed to Phase 7.") with: + +```markdown +3. **Check `$ARGUMENTS` for `--review` flag.** Then run the appropriate validation path: + +--- + +#### Default path (no `--review`): inline deterministic validation + +Write the following Node.js script to `$PROJECT_ROOT/.understand-anything/tmp/ua-inline-validate.js`: + +```javascript +#!/usr/bin/env node +const fs = require('fs'); +const graphPath = process.argv[2]; +const outputPath = process.argv[3]; +try { + const graph = JSON.parse(fs.readFileSync(graphPath, 'utf8')); + const issues = [], warnings = []; + const nodeIds = new Set(); + const seen = new Map(); + graph.nodes.forEach((n, i) => { + if (!n.id) { issues.push(`Node[${i}] missing id`); return; } + if (!n.type) issues.push(`Node[${i}] '${n.id}' missing type`); + if (!n.name) issues.push(`Node[${i}] '${n.id}' missing name`); + if (!n.summary) issues.push(`Node[${i}] '${n.id}' missing summary`); + if (!n.tags || !n.tags.length) issues.push(`Node[${i}] '${n.id}' missing tags`); + if (seen.has(n.id)) issues.push(`Duplicate node ID '${n.id}' at indices ${seen.get(n.id)} and ${i}`); + else seen.set(n.id, i); + nodeIds.add(n.id); + }); + graph.edges.forEach((e, i) => { + if (!nodeIds.has(e.source)) issues.push(`Edge[${i}] source '${e.source}' not found`); + if (!nodeIds.has(e.target)) issues.push(`Edge[${i}] target '${e.target}' not found`); + }); + const fileNodes = graph.nodes.filter(n => n.type === 'file').map(n => n.id); + const assigned = new Map(); + (graph.layers || []).forEach(layer => { + (layer.nodeIds || []).forEach(id => { + if (!nodeIds.has(id)) issues.push(`Layer '${layer.id}' refs missing node '${id}'`); + if (assigned.has(id)) issues.push(`Node '${id}' appears in multiple layers`); + assigned.set(id, layer.id); + }); + }); + fileNodes.forEach(id => { + if (!assigned.has(id)) issues.push(`File node '${id}' not in any layer`); + }); + (graph.tour || []).forEach((step, i) => { + (step.nodeIds || []).forEach(id => { + if (!nodeIds.has(id)) issues.push(`Tour step[${i}] refs missing node '${id}'`); + }); + }); + const withEdges = new Set([ + ...graph.edges.map(e => e.source), + ...graph.edges.map(e => e.target) + ]); + graph.nodes.forEach(n => { + if (!withEdges.has(n.id)) warnings.push(`Node '${n.id}' has no edges (orphan)`); + }); + const stats = { + totalNodes: graph.nodes.length, + totalEdges: graph.edges.length, + totalLayers: (graph.layers || []).length, + tourSteps: (graph.tour || []).length, + nodeTypes: graph.nodes.reduce((a, n) => { a[n.type] = (a[n.type]||0)+1; return a; }, {}), + edgeTypes: graph.edges.reduce((a, e) => { a[e.type] = (a[e.type]||0)+1; return a; }, {}) + }; + fs.writeFileSync(outputPath, JSON.stringify({ issues, warnings, stats }, null, 2)); + process.exit(0); +} catch (err) { process.stderr.write(err.message + '\n'); process.exit(1); } +``` + +Execute it: +```bash +node $PROJECT_ROOT/.understand-anything/tmp/ua-inline-validate.js \ + "$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json" \ + "$PROJECT_ROOT/.understand-anything/intermediate/review.json" +``` + +If the script exits non-zero, read stderr, fix the script, and retry once. + +--- + +#### `--review` path: full LLM reviewer + +If `--review` IS in `$ARGUMENTS`, dispatch the LLM graph-reviewer subagent as follows: + +Dispatch a subagent using the prompt template at `./graph-reviewer-prompt.md`. Read the template file and pass the full content as the subagent's prompt, appending the following additional context: + +> **Additional context from main session:** +> +> Phase 1 scan results (file inventory): +> ```json +> [list of {path, sizeLines} from scan-result.json] +> ``` +> +> Phase warnings/errors accumulated during analysis: +> - [list any batch failures, skipped files, or warnings from Phases 2-5] +> +> Cross-validate: every file in the scan inventory should have a corresponding `file:` node in the graph. Flag any missing files. Also flag any graph nodes whose `filePath` doesn't appear in the scan inventory. + +Pass these parameters in the dispatch prompt: + +> Validate the knowledge graph at `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`. +> Project root: `$PROJECT_ROOT` +> Read the file and validate it for completeness and correctness. +> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/review.json` + +--- + +4. Read `$PROJECT_ROOT/.understand-anything/intermediate/review.json`. + +5. **If `issues` array is non-empty:** + - Review the `issues` list + - Apply automated fixes where possible: + - Remove edges with dangling references + - Fill missing required fields with sensible defaults (e.g., empty `tags` -> `["untagged"]`, empty `summary` -> `"No summary available"`) + - Remove nodes with invalid types + - Re-run the final graph validation after automated fixes + - If critical issues remain after one fix attempt, save the graph anyway but include the warnings in the final report and mark dashboard auto-launch as skipped + +6. **If `issues` array is empty:** Proceed to Phase 7. +``` + +### Step 3: Verify the edit + +Re-read SKILL.md lines 297–380 and confirm: +- Phase 6 step 3 now checks for `--review` flag +- The inline validation script is present and complete +- The `--review` path still dispatches the LLM subagent identically to before +- Steps 4–6 handle the `review.json` output the same way as before + +### Step 4: Commit + +```bash +git add understand-anything-plugin/skills/understand/SKILL.md +git commit -m "perf(understand): gate LLM graph-reviewer behind --review flag, add inline deterministic validation" +``` + +--- + +## Task 2: C4a — Slim Phase 4 (architecture) node payload + +Removes `name` and `languageNotes` from the file node format injected into the architecture-analyzer subagent. These fields are not needed for architectural layer assignment and add unnecessary tokens. + +**Files:** +- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 4, around line 188–196) + +### Step 1: Locate the Phase 4 dispatch prompt in SKILL.md + +Find the block starting "Pass these parameters in the dispatch prompt:" under Phase 4 (around line 181). Look for: + +``` +> File nodes: +> ```json +> [list of {id, name, filePath, summary, tags} for all file-type nodes] +> ``` +``` + +### Step 2: Update the file node format + +Change the file nodes line from: +``` +> [list of {id, name, filePath, summary, tags} for all file-type nodes] +``` + +To: +``` +> [list of {id, filePath, summary, tags} for all file-type nodes — omit name, complexity, languageNotes] +``` + +### Step 3: Verify + +Re-read Phase 4 and confirm the node format line is updated. Import edges line below it (`[list of edges with type "imports"]`) is unchanged. + +### Step 4: Commit + +```bash +git add understand-anything-plugin/skills/understand/SKILL.md +git commit -m "perf(understand): slim Phase 4 architecture payload — drop redundant node fields" +``` + +--- + +## Task 3: C4b — Slim Phase 5 (tour builder) payload + +Phase 5 currently injects all nodes (including function/class), all edge types, and full layer objects (with nodeIds arrays). Only file nodes, import+calls edges, and slim layers are needed for tour design. This is the largest single payload change, saving ~105,000 tokens on a 500-file project. + +**Files:** +- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 5, lines 257–270) +- Modify: `understand-anything-plugin/skills/understand/tour-builder-prompt.md` (input schema) + +### Step 1: Locate the Phase 5 dispatch prompt in SKILL.md + +Find the block starting with (around line 257): +``` +> Nodes (summarized): +> ```json +> [list of {id, name, filePath, summary, type} for key nodes] +> ``` +> +> Layers: +> ```json +> [layers from Phase 4] +> ``` +> +> Key edges: +> ```json +> [imports and calls edges] +> ``` +``` + +### Step 2: Replace all three payload sections + +Replace those lines with: + +```markdown +> Nodes (file nodes only): +> ```json +> [list of {id, name, filePath, summary, type} for file-type nodes ONLY — do NOT include function or class nodes] +> ``` +> +> Layers: +> ```json +> [list of {id, name, description} for each layer — omit nodeIds] +> ``` +> +> Edges (imports and calls only): +> ```json +> [list of edges where type is "imports" or "calls" only — exclude all other edge types] +> ``` +``` + +### Step 3: Update tour-builder-prompt.md input schema + +Open `tour-builder-prompt.md` and find the "Script Requirements" section (around line 18–35). The input schema currently shows: +```json +{ + "nodes": [...], + "edges": [...], + "layers": [ + {"id": "layer:core", "name": "Core", "nodeIds": ["file:src/index.ts"]} + ] +} +``` + +Update the layers example to reflect the slim format: +```json +{ + "nodes": [ + {"id": "file:src/index.ts", "type": "file", "name": "index.ts", "filePath": "src/index.ts", "summary": "..."} + ], + "edges": [ + {"source": "file:src/index.ts", "target": "file:src/utils.ts", "type": "imports"} + ], + "layers": [ + {"id": "layer:core", "name": "Core", "description": "Core application logic"} + ] +} +``` + +Also update the "G. Node Summary Index" description (around line 84) to reflect that input nodes are file-type only: + +Find: +``` +**G. Node Summary Index** + +Create a lookup of each node ID to its `summary`, `type`, `tags` (default to empty array `[]` if not present in input), and `name` for easy reference. +``` + +Add a note after it: +``` +Note: input nodes are file-type only. The nodeSummaryIndex will contain only file nodes. +``` + +### Step 4: Verify + +- Re-read SKILL.md Phase 5 payload block: confirms file-only nodes, slim layers (no nodeIds), imports+calls edges only +- Re-read tour-builder-prompt.md input schema: layers no longer have nodeIds + +### Step 5: Commit + +```bash +git add understand-anything-plugin/skills/understand/SKILL.md \ + understand-anything-plugin/skills/understand/tour-builder-prompt.md +git commit -m "perf(understand): slim Phase 5 tour payload — file nodes only, imports+calls edges, slim layers" +``` + +--- + +## Task 4: C3 — Remove language/framework addendums from file-analyzer batches + +The addendums (`languages/typescript.md`, `frameworks/react.md`, etc.) are currently injected into every file-analyzer batch prompt. They cost ~1,300 tokens × N batches. The model already knows these languages. Replace with a compact inline reference table (~150 tokens, paid once, embedded in the base template). + +**Files:** +- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 2, lines 104–117) +- Modify: `understand-anything-plugin/skills/understand/file-analyzer-prompt.md` (add quick reference section) + +### Step 1: Update the "Build the combined prompt template" block in SKILL.md Phase 2 + +Find the block at lines 104–117: +``` +**Build the combined prompt template:** +1. Read the base template at `./file-analyzer-prompt.md`. +2. **Language context injection:** ... +3. **Framework addendum injection:** ... + +Then for each batch pass the combined template content as the subagent's prompt, appending the following additional context: + +> **Additional context from main session:** +> +> Project: `` — `` +> Frameworks detected: `` +> Languages: `` +> +> Use the language context and framework addendums (appended above) to produce more accurate summaries and better classify file roles. +``` + +Replace it with: +```markdown +**Build the prompt for each batch:** +1. Read the base template at `./file-analyzer-prompt.md`. (Language and framework hints are embedded in the template — do NOT append addendum files for Phase 2 batches. Addendums are reserved for Phase 4.) + +Then for each batch pass the template content as the subagent's prompt, appending the following additional context: + +> **Additional context from main session:** +> +> Project: `` — `` +> Languages: `` +``` + +This removes steps 2 and 3 (the addendum injection loops) entirely from Phase 2. + +### Step 2: Add Language and Framework Quick Reference to file-analyzer-prompt.md + +Open `file-analyzer-prompt.md`. Find the "## Critical Constraints" section near the bottom (around line 299). Insert the following new section **before** "## Critical Constraints": + +```markdown +## Language and Framework Quick Reference + +Use these hints to improve tag and edge accuracy for common patterns. Your training knowledge covers these — this is a fast lookup for the most impactful signals. + +**Tag signals:** + +| Signal | Tags to apply | +|---|---| +| File in `hooks/`, exports a function starting with `use` | `hook`, `service` | +| File in `contexts/` or `context/`, exports a Provider component | `service`, `state` | +| File in `pages/` or `views/` | `ui`, `routing` | +| File in `store/`, `slices/`, `reducers/`, `state/` | `state` | +| File in `services/`, `api/`, `client/` | `service` | +| `__init__.py` at a package root with re-exports | `entry-point`, `barrel` | +| `manage.py` at the project root | `entry-point` | +| `mod.rs` in a directory | `barrel` | +| `main.go` in a `cmd/` subdirectory | `entry-point` | + +**Edge signals:** + +| Pattern | Edge to create | +|---|---| +| React component renders another component in its JSX | `contains` from parent to child | +| Component/hook calls a custom hook (`useX`) | `depends_on` from consumer to hook file | +| Context provider wraps components | `publishes` from provider to context definition | +| Component calls `useContext` or custom context hook | `subscribes` from consumer to context definition | +| Python file uses `from x import y` where x is a project file | `imports` edge (same rule as JS/TS) | +| Go file `import`s an internal package path | `imports` edge to the resolved file | + +``` + +### Step 3: Verify + +- Re-read SKILL.md Phase 2 "Build the prompt" block: steps 2 and 3 (addendum loops) are gone; "Frameworks detected" line in additional context is gone +- Re-read file-analyzer-prompt.md: new "Language and Framework Quick Reference" section appears before Critical Constraints; no reference to addendum files +- Confirm Phase 4 "Build the combined prompt template" (lines 163–167) is **unchanged** — addendums still apply there + +### Step 4: Commit + +```bash +git add understand-anything-plugin/skills/understand/SKILL.md \ + understand-anything-plugin/skills/understand/file-analyzer-prompt.md +git commit -m "perf(understand): remove addendum injection from Phase 2 batches, add compact inline hints to file-analyzer" +``` + +--- + +## Task 5: C1a — Extend scanner to pre-resolve imports + +Adds a new Step 8 to the project scanner script: parse import statements from every source file and resolve relative imports against the discovered file list. The resolved map is written into `scan-result.json` as `importMap`. This is the data that lets us eliminate `allProjectFiles` from every batch in Task 7. + +**Files:** +- Modify: `understand-anything-plugin/skills/understand/project-scanner-prompt.md` + +### Step 1: Add Step 8 to the scanner script requirements + +Open `project-scanner-prompt.md`. Find "**Step 7 -- Project Name**" (around line 100). After its content (the priority list), add a new step: + +```markdown +**Step 8 -- Import Resolution** + +For each file in the discovered source list, extract and resolve relative import statements. The goal is to produce a map from each file's path to the list of project-internal files it imports. External package imports are ignored. + +For each file, read its content and extract import paths using language-appropriate patterns: + +| Language | Import patterns to match | +|---|---| +| TypeScript/JavaScript | `import ... from './...'` or `'../'`, `require('./...')` or `require('../...')` | +| Python | `from .x import y`, `from ..x import y`, `import .x` (relative only) | +| Go | Paths in `import (...)` blocks that start with the module path from `go.mod` | +| Rust | `use crate::`, `use super::`, `mod x` (within the same crate) | +| Java/Kotlin | Not resolvable by path — skip import resolution for these languages | +| Ruby | `require_relative '...'` paths | + +For each extracted import path: +1. Compute the resolved file path relative to project root: + - For relative imports (`./x`, `../x`): resolve from the importing file's directory + - Try these extension variants in order if the import has no extension: `.ts`, `.tsx`, `.js`, `.jsx`, `/index.ts`, `/index.js`, `/index.tsx`, `/index.jsx`, `.py`, `.go`, `.rs`, `.rb` +2. Check if the resolved path exists in the discovered file list +3. If yes: add to this file's resolved imports list +4. If no: skip (external, unresolvable, or dynamic import) + +Output format in the script result: +```json +"importMap": { + "src/index.ts": ["src/utils.ts", "src/config.ts"], + "src/utils.ts": [], + "src/components/App.tsx": ["src/hooks/useAuth.ts", "src/store/index.ts"] +} +``` + +Keys are project-relative paths. Values are arrays of resolved project-relative paths. Every key in the file list must appear in `importMap` (use an empty array `[]` if no imports were resolved). External packages and unresolvable imports are omitted entirely. +``` + +### Step 2: Update the scanner script output format + +Find the "### Script Output Format" section (around line 109) and update the example JSON to include `importMap`: + +Find this in the example: +```json +{ + "scriptCompleted": true, + "name": "project-name", + ... + "estimatedComplexity": "moderate" +} +``` + +Add `importMap` to the example: +```json +{ + "scriptCompleted": true, + "name": "project-name", + "rawDescription": "...", + "readmeHead": "...", + "languages": ["javascript", "typescript"], + "frameworks": ["React", "Vite"], + "files": [ + {"path": "src/index.ts", "language": "typescript", "sizeLines": 150} + ], + "totalFiles": 42, + "estimatedComplexity": "moderate", + "importMap": { + "src/index.ts": ["src/utils.ts", "src/config.ts"], + "src/utils.ts": [] + } +} +``` + +Also update the field documentation list below the example to add: +``` +- `importMap` (object) — map from every source file path to its list of resolved project-internal import paths; empty array if no resolved imports; external packages excluded +``` + +### Step 3: Update the final assembly section to preserve importMap + +Find "## Phase 2 -- Description and Final Assembly" (around line 153). Find the IMPORTANT note: +``` +**IMPORTANT:** The final output must NOT contain the `scriptCompleted`, `rawDescription`, or `readmeHead` fields. +``` + +Update it to: +``` +**IMPORTANT:** The final output must NOT contain the `scriptCompleted`, `rawDescription`, or `readmeHead` fields. All other fields — including `importMap` — MUST be preserved exactly as output by the script. +``` + +Also update the final output example to include `importMap`: +```json +{ + "name": "project-name", + "description": "...", + "languages": ["typescript"], + "frameworks": ["React"], + "files": [...], + "totalFiles": 42, + "estimatedComplexity": "moderate", + "importMap": { + "src/index.ts": ["src/utils.ts"] + } +} +``` + +### Step 4: Verify + +Re-read `project-scanner-prompt.md` and confirm: +- Step 8 is present with full import resolution logic +- Script output format includes `importMap` +- Field documentation includes `importMap` +- Final assembly section preserves `importMap` in output + +### Step 5: Commit + +```bash +git add understand-anything-plugin/skills/understand/project-scanner-prompt.md +git commit -m "perf(understand): extend scanner to pre-resolve imports, output importMap in scan-result.json" +``` + +--- + +## Task 6: C1b — Update file-analyzer to use batchImportData + +Removes `allProjectFiles` from the file-analyzer input schema and replaces it with `batchImportData` (pre-resolved imports for this batch's files only). Updates the extraction script section to skip import resolution entirely (already done by scanner). Updates the edge creation step to use `batchImportData` directly. + +**Files:** +- Modify: `understand-anything-plugin/skills/understand/file-analyzer-prompt.md` + +### Step 1: Update the input JSON schema (Script Requirements, step 1) + +Find the input schema block around line 19: +```json +{ + "projectRoot": "/path/to/project", + "allProjectFiles": ["src/index.ts", "src/utils.ts", "..."], + "batchFiles": [ + {"path": "src/index.ts", "language": "typescript", "sizeLines": 150}, + {"path": "src/utils.ts", "language": "typescript", "sizeLines": 80} + ] +} +``` + +Replace with: +```json +{ + "projectRoot": "/path/to/project", + "batchFiles": [ + {"path": "src/index.ts", "language": "typescript", "sizeLines": 150}, + {"path": "src/utils.ts", "language": "typescript", "sizeLines": 80} + ], + "batchImportData": { + "src/index.ts": ["src/utils.ts", "src/config.ts"], + "src/utils.ts": [] + } +} +``` + +Update the field descriptions: +- Remove: `allProjectFiles` description +- Add: `batchImportData` (object) — map from each batch file's project-relative path to its list of pre-resolved project-internal imports. Produced by the project scanner. Use this directly for import edge creation — do NOT attempt to re-resolve imports yourself. + +### Step 2: Remove the imports extraction from "What the Script Must Extract" + +Find the "**Imports:**" subsection under "What the Script Must Extract" (around lines 49–53): +``` +**Imports:** +- Source module path (exactly as written in the import statement) +- Imported specifiers (named imports, default import, namespace import) +- Line number +- For relative imports (starting with `./` or `../`), compute the resolved path... +``` + +Replace this entire subsection with: +```markdown +**Imports:** +- Do NOT extract imports in the script. Import resolution has already been performed by the project scanner. +- The pre-resolved imports for each file are provided in `batchImportData` in the input JSON. +- Do not include an `imports` field in the script output — import edges will be created in Phase 2 using `batchImportData` directly. +``` + +### Step 3: Update the script output format to remove imports + +Find the `results` array in the script output format (around line 67). The current `imports` array in the output: +```json +"imports": [ + {"source": "./utils", "resolvedPath": "src/utils.ts", "specifiers": ["formatDate"], "line": 1, "isExternal": false}, + {"source": "express", "resolvedPath": null, "specifiers": ["default"], "line": 2, "isExternal": true} +], +``` + +Remove the `imports` array from the script output format entirely. The result for each file should be: +```json +{ + "path": "src/index.ts", + "language": "typescript", + "totalLines": 150, + "nonEmptyLines": 120, + "functions": [...], + "classes": [...], + "exports": [...], + "metrics": { + "importCount": 5, + "exportCount": 3, + "functionCount": 4, + "classCount": 1 + } +} +``` + +Keep `metrics.importCount` (derived from `batchImportData[path].length`) as a useful metric. + +Update the metrics description to say: +``` +- `importCount` (integer) — use `batchImportData[file.path].length` from the input JSON +``` + +### Step 4: Update "Preparing the Script Input" section + +Find the `cat` command around line 113 that creates the input JSON: +```bash +cat > $PROJECT_ROOT/.understand-anything/tmp/ua-file-analyzer-input-.json << 'ENDJSON' +{ + "projectRoot": "", + "allProjectFiles": [], + "batchFiles": [] +} +ENDJSON +``` + +Replace with: +```bash +cat > $PROJECT_ROOT/.understand-anything/tmp/ua-file-analyzer-input-.json << 'ENDJSON' +{ + "projectRoot": "", + "batchFiles": [], + "batchImportData": +} +ENDJSON +``` + +### Step 5: Update Step 3 (Create Edges) — Import edge creation rule + +Find the "**Import edge creation rule:**" in the "Step 3 -- Create Edges" section (around line 213): +``` +**Import edge creation rule:** For each import in the script output where `isExternal` is `false` and `resolvedPath` is non-null, create an `imports` edge from the current file node to `file:`. Do NOT create edges for external package imports. +``` + +Replace with: +```markdown +**Import edge creation rule:** For each resolved path in `batchImportData[filePath]` (provided in the input JSON), create an `imports` edge from the current file node to `file:`. The `batchImportData` values contain only resolved project-internal paths — external packages have already been filtered out. Do NOT attempt to re-resolve imports from source. +``` + +### Step 6: Remove `allProjectFiles` references from Critical Constraints + +Find the last bullet in "## Critical Constraints" (around line 304): +``` +- For import edges, use the script's `resolvedPath` field directly. Do NOT attempt to resolve import paths yourself -- the script already did this deterministically. +``` + +Replace with: +```markdown +- For import edges, use `batchImportData[filePath]` directly from the input JSON. Do NOT attempt to resolve import paths yourself -- the project scanner already did this deterministically. +``` + +### Step 7: Verify + +Re-read `file-analyzer-prompt.md` and confirm: +- Input schema has `batchImportData`, no `allProjectFiles` +- Script "What to Extract" section: imports extraction replaced with "do not extract" +- Script output format: no `imports` array per file +- Preparing the Script Input: cat command has no `allProjectFiles` +- Import edge creation rule: uses `batchImportData` not script output +- Critical Constraints: no reference to `resolvedPath` from script + +### Step 8: Commit + +```bash +git add understand-anything-plugin/skills/understand/file-analyzer-prompt.md +git commit -m "perf(understand): replace allProjectFiles with batchImportData in file-analyzer — import resolution now done by scanner" +``` + +--- + +## Task 7: C1c + C2 — Update SKILL.md Phase 2 orchestration + +Wires up the `importMap` from Phase 1 into per-batch `batchImportData` slices. Increases batch size from 5-10 to 20-30 files. Increases concurrency from 3 to 5. Removes `allProjectFiles` from the dispatch prompt. + +**Files:** +- Modify: `understand-anything-plugin/skills/understand/SKILL.md` (Phase 0, Phase 1, Phase 2) + +### Step 1: Update Phase 1 to note importMap is now in scan-result.json + +Find Phase 1 (around line 62) where it says: +``` +After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermediate/scan-result.json` to get: +- Project name, description +- Languages, frameworks +- File list with line counts +- Complexity estimate +``` + +Add one item to the list: +``` +- Import map (`importMap`): pre-resolved project-internal imports per file +``` + +Also add a note: +``` +Store `importMap` in memory as `$IMPORT_MAP` for use in Phase 2 batch construction. +``` + +### Step 2: Change batch size and concurrency in Phase 2 + +Find line 100: +``` +Batch the file list from Phase 1 into groups of **5-10 files each** (aim for balanced batch sizes). +``` + +Replace with: +``` +Batch the file list from Phase 1 into groups of **20-30 files each** (aim for ~25 files per batch for balanced sizes). +``` + +Find line 102: +``` +For each batch, dispatch a subagent using the prompt template at `./file-analyzer-prompt.md`. Run up to **3 subagents concurrently** using parallel dispatch. +``` + +Replace with: +``` +For each batch, dispatch a subagent using the prompt template at `./file-analyzer-prompt.md`. Run up to **5 subagents concurrently** using parallel dispatch. +``` + +### Step 3: Add batchImportData construction to the dispatch block + +Find the dispatch prompt block (around lines 119–134): +``` +Fill in batch-specific parameters below and dispatch: + +> Analyze these source files and produce GraphNode and GraphEdge objects. +> Project root: `$PROJECT_ROOT` +> Project: `` +> Languages: `` +> Batch index: `` +> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/batch-.json` +> +> All project files (for import resolution): +> `` +> +> Files to analyze in this batch: +> 1. `` ( lines) +> ... +``` + +Replace with: +```markdown +Before dispatching each batch, construct `batchImportData` from `$IMPORT_MAP`: +```json +batchImportData = {} +for each file in this batch: + batchImportData[file.path] = $IMPORT_MAP[file.path] ?? [] +``` + +Fill in batch-specific parameters below and dispatch: + +> Analyze these source files and produce GraphNode and GraphEdge objects. +> Project root: `$PROJECT_ROOT` +> Project: `` +> Languages: `` +> Batch index: `` +> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/batch-.json` +> +> Pre-resolved import data for this batch (use this for all import edge creation — do NOT re-resolve imports from source): +> ```json +> +> ``` +> +> Files to analyze in this batch: +> 1. `` ( lines) +> 2. `` ( lines) +> ... +``` + +### Step 4: Update incremental update path + +Find "### Incremental update path" (around line 140): +``` +Use the changed files list from Phase 0. Batch and dispatch file-analyzer subagents using the same process as above, but only for changed files. +``` + +Update to clarify that batchImportData still applies: +``` +Use the changed files list from Phase 0. Batch and dispatch file-analyzer subagents using the same process as above (20-30 files per batch, up to 5 concurrent, with batchImportData constructed from $IMPORT_MAP), but only for changed files. +``` + +### Step 5: Verify all Phase 2 changes + +Re-read SKILL.md Phase 2 in full and confirm: +- Batch size says "20-30 files" +- Concurrency says "5 subagents concurrently" +- "Build the prompt" block: only step 1 (read base template), no addendum steps +- Additional context block: no "Frameworks detected" line, no addendum reference +- Dispatch prompt: has `batchImportData` injection, no `allProjectFiles` +- Incremental path: mentions batchImportData + +### Step 6: Commit + +```bash +git add understand-anything-plugin/skills/understand/SKILL.md +git commit -m "perf(understand): wire importMap into batchImportData per batch, increase batch size 5-10→20-30, concurrency 3→5" +``` + +--- + +## Task 8: Version bump + +Per project convention, all four version files must stay in sync when changes are pushed. + +**Files:** +- Modify: `understand-anything-plugin/package.json` +- Modify: `.claude-plugin/marketplace.json` +- Modify: `.claude-plugin/plugin.json` +- Modify: `.cursor-plugin/plugin.json` + +### Step 1: Read current version + +```bash +node -e "const p = require('./understand-anything-plugin/package.json'); console.log(p.version)" +``` + +Expected: `1.2.1` (or whatever the current version is). + +### Step 2: Bump patch version in all four files + +New version: `1.2.2` (patch bump — internal optimization, no API changes). + +Update each file: +- `understand-anything-plugin/package.json`: `"version": "1.2.2"` +- `.claude-plugin/marketplace.json`: `"version": "1.2.2"` in `plugins[0]` +- `.claude-plugin/plugin.json`: `"version": "1.2.2"` +- `.cursor-plugin/plugin.json`: `"version": "1.2.2"` + +### Step 3: Verify all four files match + +```bash +grep -r '"version"' understand-anything-plugin/package.json .claude-plugin/marketplace.json .claude-plugin/plugin.json .cursor-plugin/plugin.json +``` + +All four should show `"version": "1.2.2"`. + +### Step 4: Commit + +```bash +git add understand-anything-plugin/package.json \ + .claude-plugin/marketplace.json \ + .claude-plugin/plugin.json \ + .cursor-plugin/plugin.json +git commit -m "chore: bump version to 1.2.2" +``` + +--- + +## Task 9: Build and smoke test + +Verifies all changes work end-to-end by running `/understand --full` against a real project. + +**Files:** None (testing only) + +### Step 1: Build the packages + +```bash +pnpm --filter @understand-anything/core build +pnpm --filter @understand-anything/skill build +``` + +Expected: both build without errors. + +### Step 2: Find installed plugin version and copy to cache + +```bash +ls ~/.claude/plugins/cache/understand-anything/understand-anything/ +``` + +Note the version (e.g., `1.0.1`). Copy local build into the cache: + +```bash +VERSION=$(node -e "const p = require('./understand-anything-plugin/package.json'); console.log(p.version)") +rm -rf ~/.claude/plugins/cache/understand-anything/understand-anything/$VERSION +cp -R ./understand-anything-plugin ~/.claude/plugins/cache/understand-anything/understand-anything/$VERSION +``` + +### Step 3: Smoke test on a small project (~20 files) + +Open a fresh Claude Code session in a small TypeScript project. Run: +``` +/understand --full +``` + +Verify: +- Phases 0–7 complete without errors +- `knowledge-graph.json` is created +- Node count and edge count are reasonable +- Layers and tour are present +- No "allProjectFiles" or addendum errors in the output + +### Step 4: Smoke test on a larger project (~100+ files) + +Run `/understand --full` on a medium/large TypeScript+React project. + +Verify: +- Batch count is ~4-6 (at 20-30 files per batch for 100 files), not 10-20 +- No errors about missing import resolution +- `importMap` is present in `scan-result.json` (check `.understand-anything/intermediate/` before cleanup, or add a temporary debug log) +- Graph quality is comparable to before (summaries are descriptive, layers are correct) + +### Step 5: Test `--review` flag + +Run `/understand --full --review` on the same project. + +Verify: +- Phase 6 now dispatches the LLM graph-reviewer subagent (not the inline script) +- `review.json` is produced with `approved` field +- Pipeline completes normally + +### Step 6: Final commit (if any fixes needed from smoke test) + +```bash +git add -A +git commit -m "fix(understand): smoke test fixes for token reduction changes" +``` + +--- + +## Summary + +| Task | Change | Risk | +|---|---|---| +| 1 | C5: Gate reviewer | Low | +| 2 | C4a: Slim Phase 4 payload | Low | +| 3 | C4b: Slim Phase 5 payload | Low | +| 4 | C3: Remove addendums from batches | Low | +| 5 | C1a: Scanner import resolution | Medium | +| 6 | C1b: File-analyzer uses batchImportData | Medium | +| 7 | C1c+C2: SKILL.md orchestration + batch size | Medium | +| 8 | Version bump | Low | +| 9 | Smoke test | — | + +Tasks 1–4 are independent of Tasks 5–7. They can be shipped separately if needed. Tasks 5, 6, and 7 are tightly coupled (scanner produces importMap → SKILL.md passes batchImportData → file-analyzer consumes it) and must be shipped together. diff --git a/understand-anything-plugin/hooks/auto-update-prompt.md b/understand-anything-plugin/hooks/auto-update-prompt.md new file mode 100644 index 0000000..a9df20b --- /dev/null +++ b/understand-anything-plugin/hooks/auto-update-prompt.md @@ -0,0 +1,226 @@ +# Auto-Update Knowledge Graph (Internal — Hook-Triggered) + +Incrementally update the knowledge graph using deterministic structural fingerprinting to minimize token usage. This prompt is triggered automatically by the post-commit hook when `autoUpdate` is enabled. It is NOT a user-facing skill. + +**Key principle:** Spend zero LLM tokens when changes are cosmetic (formatting, internal logic). Only invoke LLM agents when structural changes (new/removed functions, classes, imports, exports) are detected. + +--- + +## Phase 0 — Pre-flight (Zero Token Cost) + +1. Set `PROJECT_ROOT` to the current working directory. + +2. Check that `$PROJECT_ROOT/.understand-anything/knowledge-graph.json` exists. + - If not: report "No existing knowledge graph found. Run `/understand` first to create one." and **STOP**. + +3. Check that `$PROJECT_ROOT/.understand-anything/meta.json` exists and read `gitCommitHash`. + - If not: report "No analysis metadata found. Run `/understand` to create a baseline." and **STOP**. + +4. Get current commit hash: + ```bash + git rev-parse HEAD + ``` + +5. If commit hashes match and `--force` is NOT in `$ARGUMENTS`: report "Knowledge graph is already up to date." and **STOP**. + +6. Get changed files: + ```bash + git diff ..HEAD --name-only + ``` + If no files changed: update `meta.json` with the new commit hash and **STOP**. + +7. Filter to source files only (`.ts`, `.tsx`, `.js`, `.jsx`, `.py`, `.go`, `.rs`, `.java`, `.rb`, `.cpp`, `.c`, `.h`, `.cs`, `.swift`, `.kt`, `.php`). + If no source files changed: update `meta.json` with the new commit hash, report "Only non-source files changed. Metadata updated." and **STOP**. + +8. Create intermediate directory: + ```bash + mkdir -p $PROJECT_ROOT/.understand-anything/intermediate + ``` + +--- + +## Phase 1 — Structural Fingerprint Check (Zero LLM Tokens) + +This phase runs a deterministic Node.js script that compares file structures against stored fingerprints. It costs **zero LLM tokens** — only the script execution cost. + +1. Write and execute a Node.js script (`$PROJECT_ROOT/.understand-anything/intermediate/fingerprint-check.mjs`): + +```javascript +// The script should: +// 1. Read fingerprints.json from .understand-anything/fingerprints.json +// 2. For each changed source file: +// a. Read the file content +// b. Compute SHA-256 content hash +// c. If content hash matches stored hash → NONE (skip) +// d. Extract structural elements via regex: +// - Functions: match patterns like `function NAME(`, `const NAME = (`, `export function NAME(` +// - Classes: match `class NAME`, `export class NAME` +// - Imports: match `import ... from '...'`, `import '...'` +// - Exports: match `export { ... }`, `export default`, `export function`, `export class`, `export const` +// e. Compare extracted elements against stored fingerprint +// f. Classify as NONE, COSMETIC, or STRUCTURAL +// 3. For new files (not in fingerprints.json): classify as STRUCTURAL +// 4. For deleted files (in fingerprints.json but not on disk): classify as STRUCTURAL +// 5. Determine overall decision: +// - All NONE/COSMETIC → action: "SKIP" +// - Some STRUCTURAL, ≤10 files, same directories → action: "PARTIAL_UPDATE" +// - New/deleted directories or >10 structural files → action: "ARCHITECTURE_UPDATE" +// - >30 structural files or >50% of graph → action: "FULL_UPDATE" +// 6. Write result to .understand-anything/intermediate/change-analysis.json +``` + +The output JSON should have this shape: +```json +{ + "action": "SKIP | PARTIAL_UPDATE | ARCHITECTURE_UPDATE | FULL_UPDATE", + "filesToReanalyze": ["src/new-feature.ts"], + "rerunArchitecture": false, + "rerunTour": false, + "reason": "1 file has structural changes (new function added)", + "fileChanges": [ + { "filePath": "src/utils.ts", "changeLevel": "COSMETIC", "details": ["internal logic changed"] }, + { "filePath": "src/new-feature.ts", "changeLevel": "STRUCTURAL", "details": ["new function: handleRequest"] } + ] +} +``` + +2. Read `.understand-anything/intermediate/change-analysis.json`. + +3. **Decision gate:** + + | Action | What to do | + |---|---| + | `SKIP` | Update `meta.json` with new commit hash. Report: "No structural changes detected. Graph metadata updated. Zero tokens spent." **STOP.** | + | `FULL_UPDATE` | Report: "Major structural changes detected (reason). Recommend running `/understand --full` for a complete rebuild." **STOP.** | + | `PARTIAL_UPDATE` | Proceed to Phase 2 with `filesToReanalyze` | + | `ARCHITECTURE_UPDATE` | Proceed to Phase 2 with `filesToReanalyze`, flag architecture re-run | + +--- + +## Phase 2 — Targeted Re-Analysis (Minimal Token Cost) + +Only re-analyze files with structural changes. This is the **only** phase that costs LLM tokens. + +1. Read the existing knowledge graph from `$PROJECT_ROOT/.understand-anything/knowledge-graph.json`. + +2. Batch the files from `filesToReanalyze` (from Phase 1). Use a single batch if ≤10 files, otherwise batch into groups of 5-10. + +3. For each batch, dispatch a subagent using the prompt template at `../skills/understand/file-analyzer-prompt.md`. Read the template file and pass the full content as the subagent's prompt, appending: + + > **Additional context from main session:** + > + > Project: `` — `` + > Frameworks detected: `` + > Languages: `` + > + > **IMPORTANT:** This is an incremental update. Only the files listed below have structural changes. Analyze them thoroughly but do not invent nodes for files not in this batch. + + Fill in batch-specific parameters: + + > Analyze these source files and produce GraphNode and GraphEdge objects. + > Project root: `$PROJECT_ROOT` + > Project: `` + > Languages: `` + > Batch index: `1` + > Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/batch-1.json` + > + > All project files (for import resolution): + > `` + > + > Files to analyze in this batch: + > 1. `` (`` lines) + > ... + +4. After batch(es) complete, read each `batch-.json` and merge results. + +5. **Merge with existing graph:** + - Remove old nodes whose `filePath` matches any file in `filesToReanalyze` or in the deleted files list + - Remove old edges whose `source` or `target` references a removed node + - Add new nodes and edges from the fresh analysis + - Deduplicate nodes by ID (keep latest), edges by `source + target + type` + - Remove any edge with dangling `source` or `target` references + +--- + +## Phase 3 — Conditional Architecture/Tour + Save + +### 3a. Architecture update (only if `rerunArchitecture === true`) + +If the change analysis flagged `ARCHITECTURE_UPDATE`: + +1. Dispatch a subagent using the prompt template at `../skills/understand/architecture-analyzer-prompt.md`, passing the full merged node set and import edges. Include previous layer definitions for naming consistency: + + > Previous layer definitions (for naming consistency): + > ```json + > [previous layers from existing graph] + > ``` + > Maintain the same layer names and IDs where possible. Only add/remove layers if the file structure has materially changed. + +2. After completion, read and normalize layers (same normalization as `/understand` Phase 4). + +3. Optionally re-run tour builder if layers changed significantly. + +### 3b. Lite layer update (if `rerunArchitecture === false`) + +If only a partial update: +1. For **new files**: assign them to the most likely existing layer based on directory path matching +2. For **deleted files**: remove their IDs from layer `nodeIds` arrays +3. Remove any layer that ends up with zero nodeIds + +### 3c. Lite validation + +Perform lightweight validation (no graph-reviewer agent): +1. Remove any edge with dangling `source` or `target` +2. Remove any layer `nodeIds` entry that doesn't exist in the node set +3. Ensure every file node appears in exactly one layer (add to a catch-all layer if missing) + +### 3d. Save + +1. Write the final knowledge graph to `$PROJECT_ROOT/.understand-anything/knowledge-graph.json`. + +2. Write updated metadata to `$PROJECT_ROOT/.understand-anything/meta.json`: + ```json + { + "lastAnalyzedAt": "", + "gitCommitHash": "", + "version": "1.0.0", + "analyzedFiles": + } + ``` + +3. **Update fingerprints:** Write and execute a Node.js script that: + - Reads the existing `fingerprints.json` + - For each re-analyzed file: computes new content hash and extracts structural elements via regex + - For deleted files: removes their entries + - Merges with existing fingerprints (keep unchanged files as-is) + - Writes updated `fingerprints.json` + +4. Clean up intermediate files: + ```bash + rm -rf $PROJECT_ROOT/.understand-anything/intermediate + ``` + +5. Report a summary: + - Files checked: N (total changed) + - Structural changes found: N files + - Cosmetic-only changes: N files (skipped) + - Nodes updated: N + - Action taken: PARTIAL_UPDATE / ARCHITECTURE_UPDATE + - Path to output: `$PROJECT_ROOT/.understand-anything/knowledge-graph.json` + +--- + +## Error Handling + +- If the fingerprint check script fails: fall back to treating all changed files as STRUCTURAL (conservative approach). +- If `fingerprints.json` doesn't exist: treat all changed files as STRUCTURAL and regenerate fingerprints after the update. +- If a subagent dispatch fails: retry once. If it fails again, save partial results and report the error. +- ALWAYS save partial results — a partially updated graph is better than no update. + +--- + +## Notes + +- This skill reuses the same `file-analyzer-prompt.md` and `architecture-analyzer-prompt.md` as `/understand` — no separate agent prompts needed. +- The fingerprint comparison in Phase 1 uses regex-based extraction (not tree-sitter) because it runs as a temporary Node.js script and doesn't need full AST accuracy — just signature-level detection. +- The authoritative fingerprints stored in `fingerprints.json` are generated by `/understand` Phase 7 using the core `fingerprint.ts` module (which uses tree-sitter for precise extraction). diff --git a/understand-anything-plugin/hooks/hooks.json b/understand-anything-plugin/hooks/hooks.json new file mode 100644 index 0000000..b8429bb --- /dev/null +++ b/understand-anything-plugin/hooks/hooks.json @@ -0,0 +1,25 @@ +{ + "hooks": { + "PostToolUse": [ + { + "matcher": "Bash", + "hooks": [ + { + "type": "command", + "command": "grep -qE 'git\\s+(commit|merge|cherry-pick|rebase)' <<< \"$TOOL_INPUT\" && [ -f .understand-anything/config.json ] && grep -q '\"autoUpdate\".*true' .understand-anything/config.json && [ -f .understand-anything/knowledge-graph.json ] && echo '[understand-anything] Commit detected with auto-update enabled. You MUST read the file at ${PLUGIN_DIR}/hooks/auto-update-prompt.md and execute its instructions to incrementally update the knowledge graph. Do not ask the user for confirmation — just do it.' || true" + } + ] + } + ], + "SessionStart": [ + { + "hooks": [ + { + "type": "command", + "command": "[ -f .understand-anything/config.json ] && grep -q '\"autoUpdate\".*true' .understand-anything/config.json && [ -f .understand-anything/meta.json ] && [ -f .understand-anything/knowledge-graph.json ] && [ \"$(node -p \"JSON.parse(require('fs').readFileSync('.understand-anything/meta.json','utf8')).gitCommitHash\")\" != \"$(git rev-parse HEAD 2>/dev/null)\" ] && echo '[understand-anything] Knowledge graph is stale. You MUST read the file at ${PLUGIN_DIR}/hooks/auto-update-prompt.md and execute its instructions to check for structural changes and update the graph. Do not ask the user for confirmation — just do it.' || true" + } + ] + } + ] + } +} diff --git a/understand-anything-plugin/package.json b/understand-anything-plugin/package.json index 56906b1..59ca16d 100644 --- a/understand-anything-plugin/package.json +++ b/understand-anything-plugin/package.json @@ -1,6 +1,6 @@ { "name": "@understand-anything/skill", - "version": "1.2.1", + "version": "1.2.2", "type": "module", "main": "dist/index.js", "types": "dist/index.d.ts", diff --git a/understand-anything-plugin/packages/core/src/__tests__/change-classifier.test.ts b/understand-anything-plugin/packages/core/src/__tests__/change-classifier.test.ts new file mode 100644 index 0000000..dd25373 --- /dev/null +++ b/understand-anything-plugin/packages/core/src/__tests__/change-classifier.test.ts @@ -0,0 +1,183 @@ +import { describe, it, expect } from "vitest"; +import { classifyUpdate } from "../change-classifier.js"; +import type { ChangeAnalysis } from "../fingerprint.js"; + +function makeAnalysis(overrides: Partial = {}): ChangeAnalysis { + return { + fileChanges: [], + newFiles: [], + deletedFiles: [], + structurallyChangedFiles: [], + cosmeticOnlyFiles: [], + unchangedFiles: [], + ...overrides, + }; +} + +describe("classifyUpdate", () => { + it("returns SKIP when all files are unchanged", () => { + const analysis = makeAnalysis({ + unchangedFiles: ["src/a.ts", "src/b.ts"], + }); + + const decision = classifyUpdate(analysis, 50); + + expect(decision.action).toBe("SKIP"); + expect(decision.filesToReanalyze).toHaveLength(0); + expect(decision.rerunArchitecture).toBe(false); + expect(decision.rerunTour).toBe(false); + }); + + it("returns SKIP when all changes are cosmetic", () => { + const analysis = makeAnalysis({ + cosmeticOnlyFiles: ["src/a.ts", "src/b.ts"], + }); + + const decision = classifyUpdate(analysis, 50); + + expect(decision.action).toBe("SKIP"); + expect(decision.reason).toContain("cosmetic-only"); + }); + + it("returns PARTIAL_UPDATE for a few structural changes", () => { + const analysis = makeAnalysis({ + structurallyChangedFiles: ["src/a.ts", "src/b.ts"], + newFiles: ["src/c.ts"], + cosmeticOnlyFiles: ["src/d.ts"], + }); + + // src/ already exists in the project, so adding src/c.ts is not a directory change + const allKnownFiles = ["src/a.ts", "src/b.ts", "src/d.ts", "lib/util.ts"]; + const decision = classifyUpdate(analysis, 50, allKnownFiles); + + expect(decision.action).toBe("PARTIAL_UPDATE"); + expect(decision.filesToReanalyze).toEqual(["src/a.ts", "src/b.ts", "src/c.ts"]); + expect(decision.rerunArchitecture).toBe(false); + expect(decision.rerunTour).toBe(false); + }); + + it("returns ARCHITECTURE_UPDATE when >10 structural files", () => { + const files = Array.from({ length: 12 }, (_, i) => `src/file${i}.ts`); + const analysis = makeAnalysis({ + structurallyChangedFiles: files, + }); + + const decision = classifyUpdate(analysis, 50); + + expect(decision.action).toBe("ARCHITECTURE_UPDATE"); + expect(decision.rerunArchitecture).toBe(true); + expect(decision.rerunTour).toBe(true); + }); + + it("returns ARCHITECTURE_UPDATE when new directories appear", () => { + const analysis = makeAnalysis({ + structurallyChangedFiles: ["src/existing.ts"], + newFiles: ["newdir/file.ts"], + }); + + const allKnownFiles = ["src/existing.ts", "src/other.ts", "lib/util.ts"]; + const decision = classifyUpdate(analysis, 50, allKnownFiles); + + expect(decision.action).toBe("ARCHITECTURE_UPDATE"); + expect(decision.rerunArchitecture).toBe(true); + }); + + it("returns ARCHITECTURE_UPDATE when directories are deleted", () => { + const analysis = makeAnalysis({ + structurallyChangedFiles: ["src/existing.ts"], + deletedFiles: ["olddir/removed.ts"], + }); + + const allKnownFiles = ["src/existing.ts", "src/other.ts"]; + const decision = classifyUpdate(analysis, 50, allKnownFiles); + + expect(decision.action).toBe("ARCHITECTURE_UPDATE"); + expect(decision.rerunArchitecture).toBe(true); + }); + + it("does NOT trigger ARCHITECTURE_UPDATE for new file in existing directory", () => { + const analysis = makeAnalysis({ + newFiles: ["src/newfile.ts"], + }); + + // src/ is already known via other files in the project + const allKnownFiles = ["src/a.ts", "src/b.ts", "lib/util.ts"]; + const decision = classifyUpdate(analysis, 50, allKnownFiles); + + expect(decision.action).toBe("PARTIAL_UPDATE"); + expect(decision.rerunArchitecture).toBe(false); + }); + + it("triggers ARCHITECTURE_UPDATE for new file in genuinely new directory", () => { + const analysis = makeAnalysis({ + newFiles: ["brand-new-pkg/index.ts"], + }); + + // allKnownFiles only contains src/ and lib/ — no brand-new-pkg/ + const allKnownFiles = ["src/a.ts", "src/b.ts", "lib/util.ts"]; + const decision = classifyUpdate(analysis, 50, allKnownFiles); + + expect(decision.action).toBe("ARCHITECTURE_UPDATE"); + expect(decision.rerunArchitecture).toBe(true); + }); + + it("returns FULL_UPDATE when >30 structural files", () => { + const files = Array.from({ length: 35 }, (_, i) => `src/file${i}.ts`); + const analysis = makeAnalysis({ + structurallyChangedFiles: files, + }); + + const decision = classifyUpdate(analysis, 100); + + expect(decision.action).toBe("FULL_UPDATE"); + expect(decision.rerunArchitecture).toBe(true); + expect(decision.rerunTour).toBe(true); + }); + + it("returns FULL_UPDATE when >50% of project is structurally changed", () => { + const files = Array.from({ length: 6 }, (_, i) => `src/file${i}.ts`); + const analysis = makeAnalysis({ + structurallyChangedFiles: files, + }); + + // 6 out of 10 files = 60% + const decision = classifyUpdate(analysis, 10); + + expect(decision.action).toBe("FULL_UPDATE"); + }); + + it("includes new and structural files in filesToReanalyze for PARTIAL", () => { + const analysis = makeAnalysis({ + structurallyChangedFiles: ["src/modified.ts"], + newFiles: ["src/added.ts"], + deletedFiles: ["src/removed.ts"], + }); + + const decision = classifyUpdate(analysis, 50); + + expect(decision.filesToReanalyze).toContain("src/modified.ts"); + expect(decision.filesToReanalyze).toContain("src/added.ts"); + // Deleted files shouldn't be re-analyzed + expect(decision.filesToReanalyze).not.toContain("src/removed.ts"); + }); + + it("handles empty analysis (no changes at all)", () => { + const analysis = makeAnalysis(); + const decision = classifyUpdate(analysis, 50); + + expect(decision.action).toBe("SKIP"); + expect(decision.reason).toContain("No changes detected"); + }); + + it("counts deleted files toward structural total", () => { + // 8 structural + 3 deleted = 11 total structural > 10 threshold + const analysis = makeAnalysis({ + structurallyChangedFiles: Array.from({ length: 8 }, (_, i) => `src/file${i}.ts`), + deletedFiles: ["src/old1.ts", "src/old2.ts", "src/old3.ts"], + }); + + const decision = classifyUpdate(analysis, 50); + + expect(decision.action).toBe("ARCHITECTURE_UPDATE"); + }); +}); diff --git a/understand-anything-plugin/packages/core/src/__tests__/fingerprint.test.ts b/understand-anything-plugin/packages/core/src/__tests__/fingerprint.test.ts new file mode 100644 index 0000000..6cd9533 --- /dev/null +++ b/understand-anything-plugin/packages/core/src/__tests__/fingerprint.test.ts @@ -0,0 +1,427 @@ +import { describe, it, expect, vi, beforeEach } from "vitest"; +import type { StructuralAnalysis } from "../types.js"; +import { + contentHash, + extractFileFingerprint, + compareFingerprints, + analyzeChanges, + type FileFingerprint, + type FingerprintStore, +} from "../fingerprint.js"; + +// Mock fs and path for analyzeChanges +vi.mock("node:fs", () => ({ + readFileSync: vi.fn(), + existsSync: vi.fn(), +})); + +import { readFileSync, existsSync } from "node:fs"; + +const mockedReadFileSync = vi.mocked(readFileSync); +const mockedExistsSync = vi.mocked(existsSync); + +beforeEach(() => { + vi.clearAllMocks(); +}); + +describe("contentHash", () => { + it("produces consistent SHA-256 hashes", () => { + const hash1 = contentHash("hello world"); + const hash2 = contentHash("hello world"); + expect(hash1).toBe(hash2); + expect(hash1).toMatch(/^[a-f0-9]{64}$/); + }); + + it("produces different hashes for different content", () => { + expect(contentHash("hello")).not.toBe(contentHash("world")); + }); +}); + +describe("extractFileFingerprint", () => { + it("extracts function fingerprints from analysis", () => { + const analysis: StructuralAnalysis = { + functions: [ + { name: "main", lineRange: [1, 20], params: ["config", "options"], returnType: "void" }, + { name: "helper", lineRange: [22, 30], params: [], returnType: "string" }, + ], + classes: [], + imports: [], + exports: [{ name: "main", lineNumber: 1 }], + }; + + const fp = extractFileFingerprint("src/index.ts", "const x = 1;\n".repeat(30), analysis); + + expect(fp.filePath).toBe("src/index.ts"); + expect(fp.functions).toHaveLength(2); + expect(fp.functions[0]).toEqual({ + name: "main", + params: ["config", "options"], + returnType: "void", + exported: true, + lineCount: 20, + }); + expect(fp.functions[1]).toEqual({ + name: "helper", + params: [], + returnType: "string", + exported: false, + lineCount: 9, + }); + }); + + it("extracts class fingerprints", () => { + const analysis: StructuralAnalysis = { + functions: [], + classes: [ + { name: "MyClass", lineRange: [1, 50], methods: ["doStuff", "init"], properties: ["name"] }, + ], + imports: [], + exports: [{ name: "MyClass", lineNumber: 1 }], + }; + + const fp = extractFileFingerprint("src/my-class.ts", "x\n".repeat(50), analysis); + + expect(fp.classes).toHaveLength(1); + expect(fp.classes[0]).toEqual({ + name: "MyClass", + methods: ["doStuff", "init"], + properties: ["name"], + exported: true, + lineCount: 50, + }); + }); + + it("extracts import and export fingerprints", () => { + const analysis: StructuralAnalysis = { + functions: [], + classes: [], + imports: [ + { source: "./utils", specifiers: ["format", "parse"], lineNumber: 1 }, + { source: "node:fs", specifiers: ["readFileSync"], lineNumber: 2 }, + ], + exports: [{ name: "main", lineNumber: 5 }, { name: "default", lineNumber: 10 }], + }; + + const fp = extractFileFingerprint("src/index.ts", "x\n", analysis); + + expect(fp.imports).toHaveLength(2); + expect(fp.imports[0]).toEqual({ source: "./utils", specifiers: ["format", "parse"] }); + expect(fp.exports).toEqual(["main", "default"]); + }); + + it("computes content hash and total lines", () => { + const content = "line1\nline2\nline3\n"; + const analysis: StructuralAnalysis = { + functions: [], + classes: [], + imports: [], + exports: [], + }; + + const fp = extractFileFingerprint("src/empty.ts", content, analysis); + + expect(fp.contentHash).toBe(contentHash(content)); + expect(fp.totalLines).toBe(4); // 3 lines + trailing newline = 4 elements + }); +}); + +describe("compareFingerprints", () => { + const baseFp: FileFingerprint = { + filePath: "src/index.ts", + contentHash: "abc123", + functions: [ + { name: "main", params: ["config"], returnType: "void", exported: true, lineCount: 20 }, + ], + classes: [], + imports: [{ source: "./utils", specifiers: ["format"] }], + exports: ["main"], + totalLines: 30, + hasStructuralAnalysis: true, + }; + + it("returns NONE when content hash is identical", () => { + const result = compareFingerprints(baseFp, { ...baseFp }); + expect(result.changeLevel).toBe("NONE"); + expect(result.details).toHaveLength(0); + }); + + it("returns COSMETIC when content changed but structure is identical", () => { + const newFp = { ...baseFp, contentHash: "different_hash" }; + const result = compareFingerprints(baseFp, newFp); + expect(result.changeLevel).toBe("COSMETIC"); + expect(result.details).toContain("internal logic changed (no structural impact)"); + }); + + it("detects new functions", () => { + const newFp: FileFingerprint = { + ...baseFp, + contentHash: "different", + functions: [ + ...baseFp.functions, + { name: "newFunc", params: [], exported: false, lineCount: 10 }, + ], + }; + const result = compareFingerprints(baseFp, newFp); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details).toContain("new function: newFunc"); + }); + + it("detects removed functions", () => { + const newFp: FileFingerprint = { + ...baseFp, + contentHash: "different", + functions: [], + }; + const result = compareFingerprints(baseFp, newFp); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details).toContain("removed function: main"); + }); + + it("detects parameter changes", () => { + const newFp: FileFingerprint = { + ...baseFp, + contentHash: "different", + functions: [ + { name: "main", params: ["config", "options"], returnType: "void", exported: true, lineCount: 20 }, + ], + }; + const result = compareFingerprints(baseFp, newFp); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details).toContain("params changed: main"); + }); + + it("detects export status changes", () => { + const newFp: FileFingerprint = { + ...baseFp, + contentHash: "different", + functions: [ + { name: "main", params: ["config"], returnType: "void", exported: false, lineCount: 20 }, + ], + }; + const result = compareFingerprints(baseFp, newFp); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details).toContain("export status changed: main"); + }); + + it("detects significant size changes (>50%)", () => { + const newFp: FileFingerprint = { + ...baseFp, + contentHash: "different", + functions: [ + { name: "main", params: ["config"], returnType: "void", exported: true, lineCount: 60 }, + ], + }; + const result = compareFingerprints(baseFp, newFp); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details.some((d) => d.includes("significant size change"))).toBe(true); + }); + + it("detects import changes", () => { + const newFp: FileFingerprint = { + ...baseFp, + contentHash: "different", + imports: [{ source: "./helpers", specifiers: ["doStuff"] }], + }; + const result = compareFingerprints(baseFp, newFp); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details).toContain("imports changed"); + }); + + it("detects export list changes", () => { + const newFp: FileFingerprint = { + ...baseFp, + contentHash: "different", + exports: ["main", "helper"], + }; + const result = compareFingerprints(baseFp, newFp); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details).toContain("exports changed"); + }); + + it("detects new and removed classes", () => { + const withClass: FileFingerprint = { + ...baseFp, + contentHash: "different", + classes: [{ name: "MyClass", methods: ["init"], properties: [], exported: true, lineCount: 30 }], + hasStructuralAnalysis: true, + }; + const result = compareFingerprints(baseFp, withClass); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details).toContain("new class: MyClass"); + }); + + it("detects class method changes", () => { + const oldFp: FileFingerprint = { + ...baseFp, + classes: [{ name: "Foo", methods: ["a", "b"], properties: [], exported: true, lineCount: 30 }], + hasStructuralAnalysis: true, + }; + const newFp: FileFingerprint = { + ...baseFp, + contentHash: "different", + classes: [{ name: "Foo", methods: ["a", "c"], properties: [], exported: true, lineCount: 30 }], + hasStructuralAnalysis: true, + }; + const result = compareFingerprints(oldFp, newFp); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details).toContain("methods changed: Foo"); + }); + + it("does NOT mutate input arrays (sort must use spread-copy)", () => { + const oldFp: FileFingerprint = { + ...baseFp, + classes: [{ name: "Foo", methods: ["b", "a"], properties: ["y", "x"], exported: true, lineCount: 30 }], + imports: [{ source: "./utils", specifiers: ["z", "a"] }], + hasStructuralAnalysis: true, + }; + const newFp: FileFingerprint = { + ...baseFp, + contentHash: "different", + classes: [{ name: "Foo", methods: ["b", "a"], properties: ["y", "x"], exported: true, lineCount: 30 }], + imports: [{ source: "./utils", specifiers: ["z", "a"] }], + hasStructuralAnalysis: true, + }; + + // Snapshot original order before comparison + const oldMethodsBefore = [...oldFp.classes[0].methods]; + const oldPropertiesBefore = [...oldFp.classes[0].properties]; + const oldSpecifiersBefore = [...oldFp.imports[0].specifiers]; + const newMethodsBefore = [...newFp.classes[0].methods]; + const newPropertiesBefore = [...newFp.classes[0].properties]; + const newSpecifiersBefore = [...newFp.imports[0].specifiers]; + + compareFingerprints(oldFp, newFp); + + // Arrays must remain in their original order (not sorted in-place) + expect(oldFp.classes[0].methods).toEqual(oldMethodsBefore); + expect(oldFp.classes[0].properties).toEqual(oldPropertiesBefore); + expect(oldFp.imports[0].specifiers).toEqual(oldSpecifiersBefore); + expect(newFp.classes[0].methods).toEqual(newMethodsBefore); + expect(newFp.classes[0].properties).toEqual(newPropertiesBefore); + expect(newFp.imports[0].specifiers).toEqual(newSpecifiersBefore); + }); + + it("classifies as STRUCTURAL when hasStructuralAnalysis is false (no tree-sitter)", () => { + const oldFp: FileFingerprint = { + filePath: "config.yaml", + contentHash: "hash_old", + functions: [], + classes: [], + imports: [], + exports: [], + totalLines: 10, + hasStructuralAnalysis: false, + }; + const newFp: FileFingerprint = { + filePath: "config.yaml", + contentHash: "hash_new", + functions: [], + classes: [], + imports: [], + exports: [], + totalLines: 12, + hasStructuralAnalysis: false, + }; + + const result = compareFingerprints(oldFp, newFp); + expect(result.changeLevel).toBe("STRUCTURAL"); + expect(result.details).toContain("no structural analysis available — conservative classification"); + }); +}); + +describe("analyzeChanges", () => { + const mockRegistry = { + analyzeFile: vi.fn(), + } as any; + + const existingStore: FingerprintStore = { + version: "1.0.0", + gitCommitHash: "abc123", + generatedAt: "2026-01-01T00:00:00.000Z", + files: { + "src/index.ts": { + filePath: "src/index.ts", + contentHash: "hash_a", + functions: [{ name: "main", params: [], exported: true, lineCount: 20 }], + classes: [], + imports: [], + exports: ["main"], + totalLines: 30, + hasStructuralAnalysis: true, + }, + "src/utils.ts": { + filePath: "src/utils.ts", + contentHash: "hash_b", + functions: [], + classes: [], + imports: [], + exports: [], + totalLines: 10, + hasStructuralAnalysis: true, + }, + }, + }; + + it("classifies new files as STRUCTURAL", () => { + mockedExistsSync.mockReturnValue(true); + mockedReadFileSync.mockReturnValue("new content"); + mockRegistry.analyzeFile.mockReturnValue({ + functions: [], + classes: [], + imports: [], + exports: [], + }); + + const result = analyzeChanges("/project", ["src/new-file.ts"], existingStore, mockRegistry); + + expect(result.newFiles).toContain("src/new-file.ts"); + expect(result.fileChanges[0].changeLevel).toBe("STRUCTURAL"); + }); + + it("classifies deleted files as STRUCTURAL", () => { + mockedExistsSync.mockReturnValue(false); + + const result = analyzeChanges("/project", ["src/utils.ts"], existingStore, mockRegistry); + + expect(result.deletedFiles).toContain("src/utils.ts"); + expect(result.fileChanges[0].changeLevel).toBe("STRUCTURAL"); + }); + + it("classifies unchanged content as NONE", () => { + mockedExistsSync.mockReturnValue(true); + // Return content that produces the same hash + const content = "test content"; + const hash = contentHash(content); + + const store: FingerprintStore = { + ...existingStore, + files: { + "src/index.ts": { + ...existingStore.files["src/index.ts"], + contentHash: hash, + }, + }, + }; + + mockedReadFileSync.mockReturnValue(content); + mockRegistry.analyzeFile.mockReturnValue({ + functions: [{ name: "main", lineRange: [1, 20], params: [] }], + classes: [], + imports: [], + exports: [{ name: "main", lineNumber: 1 }], + }); + + const result = analyzeChanges("/project", ["src/index.ts"], store, mockRegistry); + + expect(result.unchangedFiles).toContain("src/index.ts"); + }); + + it("ignores deleted files not in the store", () => { + mockedExistsSync.mockReturnValue(false); + + const result = analyzeChanges("/project", ["src/unknown.ts"], existingStore, mockRegistry); + + expect(result.deletedFiles).toHaveLength(0); + expect(result.fileChanges).toHaveLength(0); + }); +}); diff --git a/understand-anything-plugin/packages/core/src/change-classifier.ts b/understand-anything-plugin/packages/core/src/change-classifier.ts new file mode 100644 index 0000000..41660a6 --- /dev/null +++ b/understand-anything-plugin/packages/core/src/change-classifier.ts @@ -0,0 +1,143 @@ +import { dirname } from "node:path"; +import type { ChangeAnalysis } from "./fingerprint.js"; + +export interface UpdateDecision { + action: "SKIP" | "PARTIAL_UPDATE" | "ARCHITECTURE_UPDATE" | "FULL_UPDATE"; + filesToReanalyze: string[]; + rerunArchitecture: boolean; + rerunTour: boolean; + reason: string; +} + +/** + * Classify the type of graph update needed based on structural change analysis. + * + * Decision matrix: + * - SKIP: all files NONE or COSMETIC only + * - PARTIAL_UPDATE: some STRUCTURAL, same directories + * - ARCHITECTURE_UPDATE: new/deleted directories or >10 structural files + * - FULL_UPDATE: >30 structural files or >50% of total files changed structurally + */ +export function classifyUpdate( + analysis: ChangeAnalysis, + totalFilesInGraph: number, + allKnownFiles: string[] = [], +): UpdateDecision { + const { newFiles, deletedFiles, structurallyChangedFiles, cosmeticOnlyFiles, unchangedFiles } = analysis; + const structuralCount = structurallyChangedFiles.length + newFiles.length + deletedFiles.length; + + // No structural changes at all — skip + if (structuralCount === 0) { + const cosmeticCount = cosmeticOnlyFiles.length; + const reason = cosmeticCount > 0 + ? `${cosmeticCount} file(s) have cosmetic-only changes (no structural impact)` + : "No changes detected"; + + return { + action: "SKIP", + filesToReanalyze: [], + rerunArchitecture: false, + rerunTour: false, + reason, + }; + } + + // Too many structural changes — suggest full rebuild + const triggeredByCount = structuralCount > 30; + const triggeredByPercentage = totalFilesInGraph > 0 && structuralCount / totalFilesInGraph > 0.5; + if (triggeredByCount || triggeredByPercentage) { + const thresholdReason = + triggeredByCount && triggeredByPercentage + ? ">30 files and >50% of project" + : triggeredByCount + ? ">30 files" + : ">50% of project"; + return { + action: "FULL_UPDATE", + filesToReanalyze: [...structurallyChangedFiles, ...newFiles], + rerunArchitecture: true, + rerunTour: true, + reason: `${structuralCount} files have structural changes (${thresholdReason}) — full rebuild recommended`, + }; + } + + // Check if directory structure changed (new/deleted top-level directories) + const hasDirectoryChanges = detectDirectoryChanges(newFiles, deletedFiles, allKnownFiles); + + if (hasDirectoryChanges || structuralCount > 10) { + return { + action: "ARCHITECTURE_UPDATE", + filesToReanalyze: [...structurallyChangedFiles, ...newFiles], + rerunArchitecture: true, + rerunTour: true, + reason: hasDirectoryChanges + ? `Directory structure changed (${newFiles.length} new, ${deletedFiles.length} deleted files)` + : `${structuralCount} files have structural changes — architecture re-analysis needed`, + }; + } + + // Localized structural changes — partial update + return { + action: "PARTIAL_UPDATE", + filesToReanalyze: [...structurallyChangedFiles, ...newFiles], + rerunArchitecture: false, + rerunTour: false, + reason: `${structuralCount} file(s) have structural changes: ${summarizeChanges(analysis)}`, + }; +} + +/** + * Detect if the changes affect the directory structure (new or removed directories). + * Uses all known files in the project as the baseline for existing directories, + * then checks if any new/deleted files introduce or remove a top-level source directory. + */ +function detectDirectoryChanges( + newFiles: string[], + deletedFiles: string[], + allKnownFiles: string[], +): boolean { + const existingDirs = new Set( + allKnownFiles.map((f) => topDirectory(f)).filter(Boolean), + ); + + for (const f of newFiles) { + const dir = topDirectory(f); + if (dir && !existingDirs.has(dir)) return true; + } + + for (const f of deletedFiles) { + const dir = topDirectory(f); + if (dir && !existingDirs.has(dir)) return true; + } + + return false; +} + +/** + * Get the top-level directory of a file path (first path segment). + */ +function topDirectory(filePath: string): string | null { + const dir = dirname(filePath); + if (dir === "." || dir === "") return null; + const segments = dir.split("/"); + return segments[0] || null; +} + +/** + * Produce a concise human-readable summary of structural changes. + */ +function summarizeChanges(analysis: ChangeAnalysis): string { + const parts: string[] = []; + + if (analysis.newFiles.length > 0) { + parts.push(`${analysis.newFiles.length} new`); + } + if (analysis.deletedFiles.length > 0) { + parts.push(`${analysis.deletedFiles.length} deleted`); + } + if (analysis.structurallyChangedFiles.length > 0) { + parts.push(`${analysis.structurallyChangedFiles.length} modified`); + } + + return parts.join(", "); +} diff --git a/understand-anything-plugin/packages/core/src/fingerprint.ts b/understand-anything-plugin/packages/core/src/fingerprint.ts new file mode 100644 index 0000000..cc66dd0 --- /dev/null +++ b/understand-anything-plugin/packages/core/src/fingerprint.ts @@ -0,0 +1,385 @@ +import { createHash } from "node:crypto"; +import { readFileSync, existsSync } from "node:fs"; +import { join } from "node:path"; +import type { StructuralAnalysis } from "./types.js"; +import type { PluginRegistry } from "./plugins/registry.js"; + +// ---- Fingerprint types ---- + +export interface FunctionFingerprint { + name: string; + params: string[]; + returnType?: string; + exported: boolean; + lineCount: number; +} + +export interface ClassFingerprint { + name: string; + methods: string[]; + properties: string[]; + exported: boolean; + lineCount: number; +} + +export interface ImportFingerprint { + source: string; + specifiers: string[]; +} + +export interface FileFingerprint { + filePath: string; + contentHash: string; + functions: FunctionFingerprint[]; + classes: ClassFingerprint[]; + imports: ImportFingerprint[]; + exports: string[]; + totalLines: number; + hasStructuralAnalysis: boolean; +} + +export interface FingerprintStore { + version: "1.0.0"; + gitCommitHash: string; + generatedAt: string; + files: Record; +} + +export type ChangeLevel = "NONE" | "COSMETIC" | "STRUCTURAL"; + +export interface FileChangeResult { + filePath: string; + changeLevel: ChangeLevel; + details: string[]; +} + +export interface ChangeAnalysis { + fileChanges: FileChangeResult[]; + newFiles: string[]; + deletedFiles: string[]; + structurallyChangedFiles: string[]; + cosmeticOnlyFiles: string[]; + unchangedFiles: string[]; +} + +// ---- Core functions ---- + +/** + * Compute SHA-256 content hash for a file's content. + */ +export function contentHash(content: string): string { + return createHash("sha256").update(content).digest("hex"); +} + +/** + * Extract a structural fingerprint from a file using its tree-sitter analysis. + * The fingerprint captures only the elements that affect the knowledge graph + * (function/class/import/export signatures), not implementation details. + */ +export function extractFileFingerprint( + filePath: string, + content: string, + analysis: StructuralAnalysis, +): FileFingerprint { + const hash = contentHash(content); + const exportedNames = new Set(analysis.exports.map((e) => e.name)); + + const functions: FunctionFingerprint[] = analysis.functions.map((fn) => ({ + name: fn.name, + params: [...fn.params], + returnType: fn.returnType, + exported: exportedNames.has(fn.name), + lineCount: fn.lineRange[1] - fn.lineRange[0] + 1, + })); + + const classes: ClassFingerprint[] = analysis.classes.map((cls) => ({ + name: cls.name, + methods: [...cls.methods], + properties: [...cls.properties], + exported: exportedNames.has(cls.name), + lineCount: cls.lineRange[1] - cls.lineRange[0] + 1, + })); + + const imports: ImportFingerprint[] = analysis.imports.map((imp) => ({ + source: imp.source, + specifiers: [...imp.specifiers], + })); + + const exports = analysis.exports.map((e) => e.name); + + const totalLines = content.split("\n").length; + + return { + filePath, + contentHash: hash, + functions, + classes, + imports, + exports, + totalLines, + hasStructuralAnalysis: true, + }; +} + +/** + * Compare two file fingerprints and determine the change level. + * + * - NONE: content hash identical (file unchanged) + * - COSMETIC: content differs but structural signatures match (internal logic only) + * - STRUCTURAL: signature-level changes detected + */ +export function compareFingerprints( + oldFp: FileFingerprint, + newFp: FileFingerprint, +): FileChangeResult { + const details: string[] = []; + + // Fast path: identical content + if (oldFp.contentHash === newFp.contentHash) { + return { filePath: newFp.filePath, changeLevel: "NONE", details: [] }; + } + + // Conservative path: if either fingerprint lacks structural analysis, + // we cannot verify structure didn't change — classify as STRUCTURAL. + if (!oldFp.hasStructuralAnalysis || !newFp.hasStructuralAnalysis) { + return { + filePath: newFp.filePath, + changeLevel: "STRUCTURAL", + details: ["no structural analysis available — conservative classification"], + }; + } + + // Compare function signatures + const oldFuncNames = new Set(oldFp.functions.map((f) => f.name)); + const newFuncNames = new Set(newFp.functions.map((f) => f.name)); + + for (const name of newFuncNames) { + if (!oldFuncNames.has(name)) { + details.push(`new function: ${name}`); + } + } + for (const name of oldFuncNames) { + if (!newFuncNames.has(name)) { + details.push(`removed function: ${name}`); + } + } + + // Compare shared functions for signature changes + for (const newFn of newFp.functions) { + const oldFn = oldFp.functions.find((f) => f.name === newFn.name); + if (!oldFn) continue; + + if (JSON.stringify(oldFn.params) !== JSON.stringify(newFn.params)) { + details.push(`params changed: ${newFn.name}`); + } + if (oldFn.returnType !== newFn.returnType) { + details.push(`return type changed: ${newFn.name}`); + } + if (oldFn.exported !== newFn.exported) { + details.push(`export status changed: ${newFn.name}`); + } + // Flag large line count changes (>50% growth or shrink) + if (oldFn.lineCount > 0) { + const ratio = newFn.lineCount / oldFn.lineCount; + if (ratio > 1.5 || ratio < 0.5) { + details.push(`significant size change: ${newFn.name} (${oldFn.lineCount} → ${newFn.lineCount} lines)`); + } + } + } + + // Compare class signatures + const oldClassNames = new Set(oldFp.classes.map((c) => c.name)); + const newClassNames = new Set(newFp.classes.map((c) => c.name)); + + for (const name of newClassNames) { + if (!oldClassNames.has(name)) { + details.push(`new class: ${name}`); + } + } + for (const name of oldClassNames) { + if (!newClassNames.has(name)) { + details.push(`removed class: ${name}`); + } + } + + for (const newCls of newFp.classes) { + const oldCls = oldFp.classes.find((c) => c.name === newCls.name); + if (!oldCls) continue; + + if (JSON.stringify([...oldCls.methods].sort()) !== JSON.stringify([...newCls.methods].sort())) { + details.push(`methods changed: ${newCls.name}`); + } + if (JSON.stringify([...oldCls.properties].sort()) !== JSON.stringify([...newCls.properties].sort())) { + details.push(`properties changed: ${newCls.name}`); + } + if (oldCls.exported !== newCls.exported) { + details.push(`export status changed: ${newCls.name}`); + } + } + + // Compare imports + const oldImports = oldFp.imports.map((i) => `${i.source}:${[...i.specifiers].sort().join(",")}`).sort(); + const newImports = newFp.imports.map((i) => `${i.source}:${[...i.specifiers].sort().join(",")}`).sort(); + + if (JSON.stringify(oldImports) !== JSON.stringify(newImports)) { + details.push("imports changed"); + } + + // Compare exports + const oldExports = [...oldFp.exports].sort(); + const newExports = [...newFp.exports].sort(); + + if (JSON.stringify(oldExports) !== JSON.stringify(newExports)) { + details.push("exports changed"); + } + + if (details.length > 0) { + return { filePath: newFp.filePath, changeLevel: "STRUCTURAL", details }; + } + + // Content changed but structure is identical + return { + filePath: newFp.filePath, + changeLevel: "COSMETIC", + details: ["internal logic changed (no structural impact)"], + }; +} + +/** + * Build a fingerprint store for a set of files. + * Files without tree-sitter support get content-hash-only fingerprints + * (conservative: any change is treated as STRUCTURAL). + */ +export function buildFingerprintStore( + projectDir: string, + filePaths: string[], + registry: PluginRegistry, + gitCommitHash: string, +): FingerprintStore { + const files: Record = {}; + + for (const filePath of filePaths) { + const absolutePath = join(projectDir, filePath); + if (!existsSync(absolutePath)) continue; + + const content = readFileSync(absolutePath, "utf-8"); + const analysis = registry.analyzeFile(filePath, content); + + if (analysis) { + files[filePath] = extractFileFingerprint(filePath, content, analysis); + } else { + // No tree-sitter support: content hash only (conservative) + files[filePath] = { + filePath, + contentHash: contentHash(content), + functions: [], + classes: [], + imports: [], + exports: [], + totalLines: content.split("\n").length, + hasStructuralAnalysis: false, + }; + } + } + + return { + version: "1.0.0", + gitCommitHash, + generatedAt: new Date().toISOString(), + files, + }; +} + +/** + * Analyze changes between the current state of files and stored fingerprints. + * Returns a detailed breakdown of what changed and at what level. + */ +export function analyzeChanges( + projectDir: string, + changedFiles: string[], + existingStore: FingerprintStore, + registry: PluginRegistry, +): ChangeAnalysis { + const fileChanges: FileChangeResult[] = []; + const newFiles: string[] = []; + const deletedFiles: string[] = []; + const structurallyChangedFiles: string[] = []; + const cosmeticOnlyFiles: string[] = []; + const unchangedFiles: string[] = []; + + for (const filePath of changedFiles) { + const absolutePath = join(projectDir, filePath); + const existedBefore = filePath in existingStore.files; + const existsNow = existsSync(absolutePath); + + // File was deleted + if (!existsNow) { + if (existedBefore) { + deletedFiles.push(filePath); + fileChanges.push({ + filePath, + changeLevel: "STRUCTURAL", + details: ["file deleted"], + }); + } + continue; + } + + // File is new + if (!existedBefore) { + newFiles.push(filePath); + fileChanges.push({ + filePath, + changeLevel: "STRUCTURAL", + details: ["new file"], + }); + continue; + } + + // File exists in both — compare fingerprints + const content = readFileSync(absolutePath, "utf-8"); + const analysis = registry.analyzeFile(filePath, content); + const oldFp = existingStore.files[filePath]; + + let newFp: FileFingerprint; + if (analysis) { + newFp = extractFileFingerprint(filePath, content, analysis); + } else { + // No tree-sitter support: content hash only + newFp = { + filePath, + contentHash: contentHash(content), + functions: [], + classes: [], + imports: [], + exports: [], + totalLines: content.split("\n").length, + hasStructuralAnalysis: false, + }; + } + + const result = compareFingerprints(oldFp, newFp); + fileChanges.push(result); + + switch (result.changeLevel) { + case "NONE": + unchangedFiles.push(filePath); + break; + case "COSMETIC": + cosmeticOnlyFiles.push(filePath); + break; + case "STRUCTURAL": + structurallyChangedFiles.push(filePath); + break; + } + } + + return { + fileChanges, + newFiles, + deletedFiles, + structurallyChangedFiles, + cosmeticOnlyFiles, + unchangedFiles, + }; +} diff --git a/understand-anything-plugin/packages/core/src/index.ts b/understand-anything-plugin/packages/core/src/index.ts index 1dfcf66..3c6783e 100644 --- a/understand-anything-plugin/packages/core/src/index.ts +++ b/understand-anything-plugin/packages/core/src/index.ts @@ -71,3 +71,22 @@ export { cosineSimilarity, type SemanticSearchOptions, } from "./embedding-search.js"; +export { + extractFileFingerprint, + compareFingerprints, + analyzeChanges, + buildFingerprintStore, + contentHash, + type FunctionFingerprint, + type ClassFingerprint, + type ImportFingerprint, + type FileFingerprint, + type FingerprintStore, + type ChangeLevel, + type FileChangeResult, + type ChangeAnalysis, +} from "./fingerprint.js"; +export { + classifyUpdate, + type UpdateDecision, +} from "./change-classifier.js"; diff --git a/understand-anything-plugin/packages/core/src/persistence/index.ts b/understand-anything-plugin/packages/core/src/persistence/index.ts index 6c8fc67..cf1cab6 100644 --- a/understand-anything-plugin/packages/core/src/persistence/index.ts +++ b/understand-anything-plugin/packages/core/src/persistence/index.ts @@ -1,11 +1,14 @@ import { readFileSync, writeFileSync, mkdirSync, existsSync } from "node:fs"; -import { join } from "node:path"; -import type { KnowledgeGraph, AnalysisMeta } from "../types.js"; +import { join, isAbsolute, relative, basename } from "node:path"; +import type { KnowledgeGraph, AnalysisMeta, ProjectConfig } from "../types.js"; +import type { FingerprintStore } from "../fingerprint.js"; import { validateGraph } from "../schema.js"; const UA_DIR = ".understand-anything"; const GRAPH_FILE = "knowledge-graph.json"; const META_FILE = "meta.json"; +const FINGERPRINT_FILE = "fingerprints.json"; +const CONFIG_FILE = "config.json"; function ensureDir(projectRoot: string): string { const dir = join(projectRoot, UA_DIR); @@ -15,9 +18,68 @@ function ensureDir(projectRoot: string): string { return dir; } +/** + * Sanitise every node's filePath before writing to disk. + * + * The analysis agent produces absolute paths like: + * /Users/alice/company/src/auth.ts + * + * We convert them to paths relative to projectRoot: + * src/auth.ts + * + * Three cases are handled: + * 1. Path is inside projectRoot → make it relative + * 2. Path is absolute but outside → keep only the filename (last segment) + * 3. Path is already relative → leave it untouched + * + * This means the developer's home directory, username, and company + * directory layout are never written to knowledge-graph.json. + */ +function sanitiseFilePaths( + graph: KnowledgeGraph, + projectRoot: string, +): KnowledgeGraph { + const normalRoot = projectRoot.endsWith("/") + ? projectRoot + : projectRoot + "/"; + + const sanitisedNodes = graph.nodes.map((node) => { + if (typeof node.filePath !== "string") return node; + + const fp = node.filePath; + + if (!isAbsolute(fp)) { + // Already relative — nothing to do. + return node; + } + + if (fp.startsWith(normalRoot) || fp.startsWith(projectRoot)) { + // Inside the project root — make it relative. + return { ...node, filePath: relative(projectRoot, fp) }; + } + + // Absolute but outside the project root — use only the filename + // so we leak as little as possible. + return { ...node, filePath: basename(fp) }; + }); + + return { ...graph, nodes: sanitisedNodes }; +} + export function saveGraph(projectRoot: string, graph: KnowledgeGraph): void { const dir = ensureDir(projectRoot); - writeFileSync(join(dir, GRAPH_FILE), JSON.stringify(graph, null, 2), "utf-8"); + + // FIX — sanitise absolute file paths before persisting. + // Without this, absolute paths like /Users/alice/company/src/auth.ts + // are written verbatim into knowledge-graph.json and later served + // by the dashboard server, leaking the developer's directory layout. + const sanitised = sanitiseFilePaths(graph, projectRoot); + + writeFileSync( + join(dir, GRAPH_FILE), + JSON.stringify(sanitised, null, 2), + "utf-8", + ); } export function loadGraph( @@ -52,3 +114,35 @@ export function loadMeta(projectRoot: string): AnalysisMeta | null { if (!existsSync(filePath)) return null; return JSON.parse(readFileSync(filePath, "utf-8")) as AnalysisMeta; } + +export function saveFingerprints(projectRoot: string, store: FingerprintStore): void { + const dir = ensureDir(projectRoot); + writeFileSync(join(dir, FINGERPRINT_FILE), JSON.stringify(store, null, 2), "utf-8"); +} + +export function loadFingerprints(projectRoot: string): FingerprintStore | null { + const filePath = join(projectRoot, UA_DIR, FINGERPRINT_FILE); + if (!existsSync(filePath)) return null; + try { + return JSON.parse(readFileSync(filePath, "utf-8")) as FingerprintStore; + } catch { + return null; + } +} + +const DEFAULT_CONFIG: ProjectConfig = { autoUpdate: false }; + +export function saveConfig(projectRoot: string, config: ProjectConfig): void { + const dir = ensureDir(projectRoot); + writeFileSync(join(dir, CONFIG_FILE), JSON.stringify(config, null, 2), "utf-8"); +} + +export function loadConfig(projectRoot: string): ProjectConfig { + const filePath = join(projectRoot, UA_DIR, CONFIG_FILE); + if (!existsSync(filePath)) return { ...DEFAULT_CONFIG }; + try { + return JSON.parse(readFileSync(filePath, "utf-8")) as ProjectConfig; + } catch { + return { ...DEFAULT_CONFIG }; + } +} diff --git a/understand-anything-plugin/packages/core/src/persistence/persistence.test.ts b/understand-anything-plugin/packages/core/src/persistence/persistence.test.ts index 97681bc..cde784e 100644 --- a/understand-anything-plugin/packages/core/src/persistence/persistence.test.ts +++ b/understand-anything-plugin/packages/core/src/persistence/persistence.test.ts @@ -2,8 +2,10 @@ import { describe, it, expect, beforeEach, afterEach } from "vitest"; import { mkdtempSync, rmSync, existsSync } from "node:fs"; import { join } from "node:path"; import { tmpdir } from "node:os"; -import { saveGraph, loadGraph, saveMeta, loadMeta } from "./index.js"; +import { writeFileSync } from "node:fs"; +import { saveGraph, loadGraph, saveMeta, loadMeta, saveFingerprints, loadFingerprints, saveConfig, loadConfig } from "./index.js"; import type { KnowledgeGraph, AnalysisMeta } from "../types.js"; +import type { FingerprintStore } from "../fingerprint.js"; describe("persistence", () => { let tempDir: string; @@ -115,4 +117,70 @@ describe("persistence", () => { expect(loaded).toBeNull(); }); }); + + describe("saveFingerprints / loadFingerprints", () => { + const sampleFingerprints: FingerprintStore = { + version: "1.0.0", + gitCommitHash: "abc123", + generatedAt: "2026-03-14T00:00:00.000Z", + files: { + "src/index.ts": { + filePath: "src/index.ts", + contentHash: "deadbeef", + functions: [], + classes: [], + imports: [], + exports: [], + totalLines: 10, + hasStructuralAnalysis: false, + }, + }, + }; + + it("should round-trip fingerprints correctly", () => { + saveFingerprints(tempDir, sampleFingerprints); + const loaded = loadFingerprints(tempDir); + + expect(loaded).toEqual(sampleFingerprints); + }); + + it("should return null when no fingerprints file exists", () => { + const loaded = loadFingerprints(tempDir); + expect(loaded).toBeNull(); + }); + + it("should return null when fingerprints.json is corrupted", () => { + const dir = join(tempDir, ".understand-anything"); + // Ensure the directory exists by saving first, then overwrite with garbage + saveFingerprints(tempDir, sampleFingerprints); + writeFileSync(join(dir, "fingerprints.json"), "{{not valid json!!", "utf-8"); + + const loaded = loadFingerprints(tempDir); + expect(loaded).toBeNull(); + }); + }); + + describe("saveConfig / loadConfig", () => { + it("should round-trip config correctly", () => { + saveConfig(tempDir, { autoUpdate: true }); + const loaded = loadConfig(tempDir); + + expect(loaded).toEqual({ autoUpdate: true }); + }); + + it("should return default config when no file exists", () => { + const loaded = loadConfig(tempDir); + + expect(loaded).toEqual({ autoUpdate: false }); + }); + + it("should return default config when config.json is corrupted", () => { + saveConfig(tempDir, { autoUpdate: true }); + const dir = join(tempDir, ".understand-anything"); + writeFileSync(join(dir, "config.json"), "not json!!", "utf-8"); + + const loaded = loadConfig(tempDir); + expect(loaded).toEqual({ autoUpdate: false }); + }); + }); }); diff --git a/understand-anything-plugin/packages/core/src/types.ts b/understand-anything-plugin/packages/core/src/types.ts index f5fef33..819f17e 100644 --- a/understand-anything-plugin/packages/core/src/types.ts +++ b/understand-anything-plugin/packages/core/src/types.ts @@ -66,12 +66,24 @@ export interface KnowledgeGraph { tour: TourStep[]; } +// Theme configuration (for dashboard customization) +export interface ThemeConfig { + presetId: string; + accentId: string; +} + // AnalysisMeta (for persistence) export interface AnalysisMeta { lastAnalyzedAt: string; gitCommitHash: string; version: string; analyzedFiles: number; + theme?: ThemeConfig; +} + +// Project config (for auto-update opt-in) +export interface ProjectConfig { + autoUpdate: boolean; } // Plugin interfaces diff --git a/understand-anything-plugin/packages/dashboard/src/App.tsx b/understand-anything-plugin/packages/dashboard/src/App.tsx index 9eab49a..7f4f207 100644 --- a/understand-anything-plugin/packages/dashboard/src/App.tsx +++ b/understand-anything-plugin/packages/dashboard/src/App.tsx @@ -15,6 +15,16 @@ import KeyboardShortcutsHelp from "./components/KeyboardShortcutsHelp"; import WarningBanner from "./components/WarningBanner"; import { useKeyboardShortcuts } from "./hooks/useKeyboardShortcuts"; import type { KeyboardShortcut } from "./hooks/useKeyboardShortcuts"; +import { ThemeProvider } from "./themes/index.ts"; +import { ThemePicker } from "./components/ThemePicker.tsx"; +import type { ThemeConfig } from "./themes/index.ts"; + +// Extract the access token from the URL so protected endpoints can be fetched. +const ACCESS_TOKEN = new URLSearchParams(window.location.search).get("token"); + +function tokenUrl(path: string): string { + return ACCESS_TOKEN ? `${path}?token=${ACCESS_TOKEN}` : path; +} function App() { const graph = useDashboardStore((s) => s.graph); @@ -28,6 +38,16 @@ function App() { const [loadError, setLoadError] = useState(null); const [graphIssues, setGraphIssues] = useState([]); const [showKeyboardHelp, setShowKeyboardHelp] = useState(false); + const [metaTheme, setMetaTheme] = useState(null); + + useEffect(() => { + fetch(tokenUrl("/meta.json")) + .then((r) => (r.ok ? r.json() : null)) + .then((meta) => { + if (meta?.theme) setMetaTheme(meta.theme); + }) + .catch(() => {}); + }, []); // Define keyboard shortcuts const shortcuts = useMemo( @@ -120,7 +140,7 @@ function App() { useKeyboardShortcuts(shortcuts); useEffect(() => { - fetch("/knowledge-graph.json") + fetch(tokenUrl("/knowledge-graph.json")) .then((res) => res.json()) .then((data: unknown) => { const result = validateGraph(data); @@ -149,7 +169,7 @@ function App() { }, [setGraph]); useEffect(() => { - fetch("/diff-overlay.json") + fetch(tokenUrl("/diff-overlay.json")) .then((res) => { if (!res.ok) return null; return res.json(); @@ -187,6 +207,7 @@ function App() { ); return ( +
{/* Header */}
@@ -200,9 +221,10 @@ function App() {
+
+ ); } diff --git a/understand-anything-plugin/packages/dashboard/src/components/CodeViewer.tsx b/understand-anything-plugin/packages/dashboard/src/components/CodeViewer.tsx index d1df7c3..64fe249 100644 --- a/understand-anything-plugin/packages/dashboard/src/components/CodeViewer.tsx +++ b/understand-anything-plugin/packages/dashboard/src/components/CodeViewer.tsx @@ -27,8 +27,8 @@ export default function CodeViewer() { className="text-[10px] font-semibold uppercase tracking-wider px-2 py-0.5 rounded border" style={{ color: "var(--color-node-file)", - borderColor: "rgba(74,124,155,0.3)", - backgroundColor: "rgba(74,124,155,0.1)", + borderColor: "color-mix(in srgb, var(--color-node-file) 30%, transparent)", + backgroundColor: "color-mix(in srgb, var(--color-node-file) 10%, transparent)", }} > {node.type} @@ -56,14 +56,14 @@ export default function CodeViewer() {
{/* Summary */}
-

Summary

+

Summary

{node.summary}

{/* Language notes callout */} {node.languageNotes && ( -
-

Language Notes

+
+

Language Notes

{node.languageNotes}

)} @@ -71,7 +71,7 @@ export default function CodeViewer() { {/* Tags */} {node.tags.length > 0 && (
-

Tags

+

Tags

{node.tags.map((tag) => ( diff --git a/understand-anything-plugin/packages/dashboard/src/components/CustomNode.tsx b/understand-anything-plugin/packages/dashboard/src/components/CustomNode.tsx index 72faf64..1ec549e 100644 --- a/understand-anything-plugin/packages/dashboard/src/components/CustomNode.tsx +++ b/understand-anything-plugin/packages/dashboard/src/components/CustomNode.tsx @@ -20,7 +20,7 @@ const typeTextColors: Record = { const complexityColors: Record = { simple: "text-node-function", - moderate: "text-gold-dim", + moderate: "text-accent-dim", complex: "text-[#c97070]", }; @@ -53,17 +53,17 @@ function CustomNodeComponent({ let extraClass = ""; if (data.isSelected) { - extraClass = "ring-2 ring-gold node-glow"; + extraClass = "ring-2 ring-accent node-glow"; } else if (data.isTourHighlighted) { - extraClass = "ring-2 ring-gold-dim animate-gold-pulse"; + extraClass = "ring-2 ring-accent-dim animate-accent-pulse"; } else if (data.isHighlighted) { const score = data.searchScore ?? 1; if (score <= 0.1) { - extraClass = "ring-2 ring-gold-bright"; + extraClass = "ring-2 ring-accent-bright"; } else if (score <= 0.3) { - extraClass = "ring-2 ring-gold"; + extraClass = "ring-2 ring-accent"; } else { - extraClass = "ring-1 ring-gold-dim/60"; + extraClass = "ring-1 ring-accent-dim/60"; } } diff --git a/understand-anything-plugin/packages/dashboard/src/components/GraphView.tsx b/understand-anything-plugin/packages/dashboard/src/components/GraphView.tsx index f57ec8f..4f7350a 100644 --- a/understand-anything-plugin/packages/dashboard/src/components/GraphView.tsx +++ b/understand-anything-plugin/packages/dashboard/src/components/GraphView.tsx @@ -21,6 +21,7 @@ import PortalNode from "./PortalNode"; import type { PortalFlowNode } from "./PortalNode"; import Breadcrumb from "./Breadcrumb"; import { useDashboardStore } from "../store"; +import { useTheme } from "../themes/index.ts"; import { applyDagreLayout, NODE_WIDTH, @@ -425,6 +426,7 @@ function GraphViewInner() { const drillIntoLayer = useDashboardStore((s) => s.drillIntoLayer); const focusNodeId = useDashboardStore((s) => s.focusNodeId); const setFocusNode = useDashboardStore((s) => s.setFocusNode); + const { preset } = useTheme(); const overviewGraph = useOverviewGraph(); const detailGraph = useLayerDetailGraph(); @@ -510,18 +512,13 @@ function GraphViewInner() { fitViewOptions={{ minZoom: 0.01, padding: 0.1 }} minZoom={0.01} maxZoom={2} - colorMode="dark" + colorMode={preset.isDark ? "dark" : "light"} > - + diff --git a/understand-anything-plugin/packages/dashboard/src/components/KeyboardShortcutsHelp.tsx b/understand-anything-plugin/packages/dashboard/src/components/KeyboardShortcutsHelp.tsx index 22178f5..c9fcd1f 100644 --- a/understand-anything-plugin/packages/dashboard/src/components/KeyboardShortcutsHelp.tsx +++ b/understand-anything-plugin/packages/dashboard/src/components/KeyboardShortcutsHelp.tsx @@ -62,7 +62,7 @@ export default function KeyboardShortcutsHelp({
{Object.entries(groupedShortcuts).map(([category, categoryShortcuts]) => (
-

+

{category}

diff --git a/understand-anything-plugin/packages/dashboard/src/components/LearnPanel.tsx b/understand-anything-plugin/packages/dashboard/src/components/LearnPanel.tsx index 2534fd4..7e4344b 100644 --- a/understand-anything-plugin/packages/dashboard/src/components/LearnPanel.tsx +++ b/understand-anything-plugin/packages/dashboard/src/components/LearnPanel.tsx @@ -47,13 +47,13 @@ export default function LearnPanel() {
-

+

Steps

{tourSteps.map((step, i) => ( @@ -61,7 +61,7 @@ export default function LearnPanel() { key={step.order} className="flex items-start gap-2 text-xs bg-elevated rounded-lg px-3 py-2 border border-border-subtle" > - + {i + 1}. {step.title} @@ -86,7 +86,7 @@ export default function LearnPanel() { {/* Header with progress counter and exit */}
-

+

Tour

@@ -104,7 +104,7 @@ export default function LearnPanel() { {/* Progress bar */}
@@ -122,16 +122,16 @@ export default function LearnPanel() {

{children}

), strong: ({ children }) => ( - {children} + {children} ), code: ({ className, children }) => { const isBlock = className?.includes("language-"); return isBlock ? ( - + {children} ) : ( - + {children} ); @@ -154,8 +154,8 @@ export default function LearnPanel() { {/* Language lesson */} {step.languageLesson && ( -
-

+
+

Language Lesson

@@ -167,7 +167,7 @@ export default function LearnPanel() { {/* Referenced component pills */} {step.nodeIds.length > 0 && (

-

+

Referenced Components

@@ -198,7 +198,7 @@ export default function LearnPanel() { onClick={() => setTourStep(i)} className={`w-2 h-2 rounded-full transition-colors ${ i === currentTourStep - ? "bg-gold" + ? "bg-accent" : "bg-elevated hover:bg-surface" }`} aria-label={`Go to step ${i + 1}`} @@ -217,7 +217,7 @@ export default function LearnPanel() { diff --git a/understand-anything-plugin/packages/dashboard/src/components/NodeInfo.tsx b/understand-anything-plugin/packages/dashboard/src/components/NodeInfo.tsx index 2fbd521..48bdcde 100644 --- a/understand-anything-plugin/packages/dashboard/src/components/NodeInfo.tsx +++ b/understand-anything-plugin/packages/dashboard/src/components/NodeInfo.tsx @@ -11,7 +11,7 @@ const typeBadgeColors: Record = { const complexityBadgeColors: Record = { simple: "text-node-function border border-node-function/30 bg-node-function/10", - moderate: "text-gold-dim border border-gold-dim/30 bg-gold-dim/10", + moderate: "text-accent-dim border border-accent-dim/30 bg-accent-dim/10", complex: "text-[#c97070] border border-[#c97070]/30 bg-[#c97070]/10", }; @@ -212,7 +212,7 @@ export default function NodeInfo() {
diff --git a/understand-anything-plugin/packages/dashboard/src/components/SearchBar.tsx b/understand-anything-plugin/packages/dashboard/src/components/SearchBar.tsx index b932464..abc5eae 100644 --- a/understand-anything-plugin/packages/dashboard/src/components/SearchBar.tsx +++ b/understand-anything-plugin/packages/dashboard/src/components/SearchBar.tsx @@ -94,14 +94,14 @@ export default function SearchBar() { onChange={handleInputChange} onFocus={() => setDropdownOpen(true)} placeholder="Search nodes by name, summary, or tags..." - className="flex-1 bg-elevated text-text-primary text-sm rounded-lg px-3 py-1.5 border border-border-subtle focus:outline-none focus:border-gold/50 placeholder-text-muted" + className="flex-1 bg-elevated text-text-primary text-sm rounded-lg px-3 py-1.5 border border-border-subtle focus:outline-none focus:border-accent/50 placeholder-text-muted" />
+ + {open && ( +
+ {/* Presets */} +
+
+ Theme +
+
+ {PRESETS.map((p) => ( + + ))} +
+
+ + {/* Accent swatches */} +
+
+ Accent Color +
+
+ {preset.accentSwatches.map((swatch) => ( +
+
+
+ )} +
+ ); +} diff --git a/understand-anything-plugin/packages/dashboard/src/index.css b/understand-anything-plugin/packages/dashboard/src/index.css index a0f8ffa..95b5440 100644 --- a/understand-anything-plugin/packages/dashboard/src/index.css +++ b/understand-anything-plugin/packages/dashboard/src/index.css @@ -1,46 +1,77 @@ @import "tailwindcss"; @theme { - /* Dark luxury color palette */ + /* Base */ --color-root: #0a0a0a; --color-surface: #111111; --color-elevated: #1a1a1a; --color-panel: #141414; - /* Gold accent spectrum */ - --color-gold: #d4a574; - --color-gold-dim: #c9a96e; - --color-gold-bright: #e8c49a; + /* Accent */ + --color-accent: #d4a574; + --color-accent-dim: #c9a96e; + --color-accent-bright: #e8c49a; - /* Text hierarchy */ + /* Text */ --color-text-primary: #f5f0eb; --color-text-secondary: #a39787; --color-text-muted: #6b5f53; - /* Border tokens */ + /* Borders */ --color-border-subtle: rgba(212, 165, 116, 0.12); --color-border-medium: rgba(212, 165, 116, 0.25); - /* Node type colors (muted, refined) */ + /* Node types */ --color-node-file: #4a7c9b; --color-node-function: #5a9e6f; --color-node-class: #8b6fb0; --color-node-module: #c9a06c; --color-node-concept: #b07a8a; - /* Diff overlay colors */ + /* Diff */ --color-diff-changed: #e05252; --color-diff-affected: #d4a030; --color-diff-changed-dim: rgba(224, 82, 82, 0.25); --color-diff-affected-dim: rgba(212, 160, 48, 0.25); - /* Fonts */ + /* Glass */ + --glass-bg: rgba(20, 20, 20, 0.8); + --glass-bg-heavy: rgba(20, 20, 20, 0.95); + --glass-border: rgba(212, 165, 116, 0.1); + --glass-border-heavy: rgba(212, 165, 116, 0.15); + + /* Scrollbar */ + --scrollbar-thumb: rgba(212, 165, 116, 0.2); + --scrollbar-thumb-hover: rgba(212, 165, 116, 0.35); + + /* Glow */ + --glow-accent: rgba(212, 165, 116, 0.15); + --glow-accent-strong: rgba(212, 165, 116, 0.4); + --glow-accent-pulse: rgba(212, 165, 116, 0.6); + + /* Edges */ + --color-edge: rgba(212, 165, 116, 0.3); + --color-edge-dim: rgba(212, 165, 116, 0.08); + --color-edge-dot: rgba(212, 165, 116, 0.15); + + /* Accent overlays */ + --color-accent-overlay-bg: rgba(212, 165, 116, 0.05); + --color-accent-overlay-border: rgba(212, 165, 116, 0.25); + + /* Kbd */ + --kbd-bg: rgba(212, 165, 116, 0.1); + + /* Typography */ --font-serif: 'DM Serif Display', Georgia, serif; --font-mono: 'JetBrains Mono', 'Fira Code', monospace; --font-sans: 'Inter', system-ui, sans-serif; } /* Base styles */ +html { + transition: background-color 0.2s ease, color 0.2s ease; +} + body { font-family: var(--font-sans); background-color: var(--color-root); @@ -65,15 +96,15 @@ body { /* Glass utility */ .glass { - background: rgba(20, 20, 20, 0.8); - border: 1px solid rgba(212, 165, 116, 0.1); + background: var(--glass-bg); + border: 1px solid var(--glass-border); backdrop-filter: blur(12px); -webkit-backdrop-filter: blur(12px); } .glass-heavy { - background: rgba(20, 20, 20, 0.95); - border: 1px solid rgba(212, 165, 116, 0.15); + background: var(--glass-bg-heavy); + border: 1px solid var(--glass-border-heavy); backdrop-filter: blur(16px); -webkit-backdrop-filter: blur(16px); } @@ -89,11 +120,11 @@ body { font-family: var(--font-mono); font-size: 0.75rem; font-weight: 600; - color: var(--color-gold); - background: rgba(212, 165, 116, 0.1); - border: 1px solid rgba(212, 165, 116, 0.3); + color: var(--color-accent); + background: var(--kbd-bg); + border: 1px solid var(--color-border-medium); border-radius: 0.25rem; - box-shadow: 0 1px 0 rgba(212, 165, 116, 0.2); + box-shadow: 0 1px 0 var(--scrollbar-thumb); } /* Animation keyframes */ @@ -117,12 +148,12 @@ body { } } -@keyframes goldPulse { +@keyframes accentPulse { 0%, 100% { - box-shadow: 0 0 0 0 rgba(212, 165, 116, 0.4); + box-shadow: 0 0 8px var(--glow-accent-strong); } 50% { - box-shadow: 0 0 20px 4px rgba(212, 165, 116, 0.15); + box-shadow: 0 0 20px var(--glow-accent-pulse); } } @@ -135,13 +166,13 @@ body { animation: slideUp 0.3s ease-out forwards; } -.animate-gold-pulse { - animation: goldPulse 2s ease-in-out infinite; +.animate-accent-pulse { + animation: accentPulse 2s ease-in-out infinite; } /* Node selection glow */ .node-glow { - box-shadow: 0 0 20px rgba(212, 165, 116, 0.15); + box-shadow: 0 0 20px var(--glow-accent); } /* Diff overlay glow effects */ @@ -169,14 +200,37 @@ body { background: transparent; } ::-webkit-scrollbar-thumb { - background: rgba(212, 165, 116, 0.2); - border-radius: 3px; + background: var(--scrollbar-thumb); + border-radius: 4px; } ::-webkit-scrollbar-thumb:hover { - background: rgba(212, 165, 116, 0.35); + background: var(--scrollbar-thumb-hover); } /* Override React Flow dark theme */ .react-flow__background { background-color: var(--color-root) !important; } + +/* Light theme overrides */ +[data-theme="light"] { + color-scheme: light; +} + +[data-theme="light"] .diff-faded { + opacity: 0.35; +} + +[data-theme="light"] ::-webkit-scrollbar-track { + background: rgba(0, 0, 0, 0.05); +} + +[data-theme="light"] .warning-banner { + background: rgba(180, 130, 30, 0.1); + border-color: rgba(180, 130, 30, 0.3); + color: #92600a; +} + +[data-theme="dark"] { + color-scheme: dark; +} diff --git a/understand-anything-plugin/packages/dashboard/src/themes/ThemeContext.tsx b/understand-anything-plugin/packages/dashboard/src/themes/ThemeContext.tsx new file mode 100644 index 0000000..dc12fcc --- /dev/null +++ b/understand-anything-plugin/packages/dashboard/src/themes/ThemeContext.tsx @@ -0,0 +1,101 @@ +import { + createContext, + useCallback, + useContext, + useEffect, + useRef, + useState, + type ReactNode, +} from "react"; +import type { PresetId, ThemeConfig, ThemePreset } from "./types.ts"; +import { DEFAULT_THEME_CONFIG } from "./types.ts"; +import { getPreset } from "./presets.ts"; +import { applyTheme } from "./theme-engine.ts"; + +const STORAGE_KEY = "ua-theme"; + +interface ThemeContextValue { + config: ThemeConfig; + preset: ThemePreset; + setPreset: (presetId: PresetId) => void; + setAccent: (accentId: string) => void; +} + +const ThemeContext = createContext(null); + +function loadFromLocalStorage(): ThemeConfig | null { + try { + const raw = localStorage.getItem(STORAGE_KEY); + if (!raw) return null; + const parsed = JSON.parse(raw); + if (parsed && typeof parsed.presetId === "string" && typeof parsed.accentId === "string") { + return parsed as ThemeConfig; + } + return null; + } catch { + return null; + } +} + +function saveToLocalStorage(config: ThemeConfig): void { + try { + localStorage.setItem(STORAGE_KEY, JSON.stringify(config)); + } catch { + // Storage full or unavailable — ignore + } +} + +function resolveInitialTheme(metaTheme?: ThemeConfig | null): ThemeConfig { + return loadFromLocalStorage() ?? metaTheme ?? DEFAULT_THEME_CONFIG; +} + +interface ThemeProviderProps { + metaTheme?: ThemeConfig | null; + children: ReactNode; +} + +export function ThemeProvider({ metaTheme, children }: ThemeProviderProps) { + const [config, setConfig] = useState(() => resolveInitialTheme(metaTheme)); + const initialized = useRef(false); + + // Apply theme on mount and config changes + useEffect(() => { + applyTheme(config); + if (initialized.current) { + saveToLocalStorage(config); + } + initialized.current = true; + }, [config]); + + // Update if metaTheme arrives later (async fetch) and no localStorage preference exists + useEffect(() => { + if (metaTheme && !loadFromLocalStorage()) { + setConfig(metaTheme); + } + }, [metaTheme]); + + const setPreset = useCallback((presetId: PresetId) => { + setConfig((_prev) => { + const newPreset = getPreset(presetId); + return { presetId, accentId: newPreset.defaultAccentId }; + }); + }, []); + + const setAccent = useCallback((accentId: string) => { + setConfig((prev) => ({ ...prev, accentId })); + }, []); + + const preset = getPreset(config.presetId); + + return ( + + {children} + + ); +} + +export function useTheme(): ThemeContextValue { + const ctx = useContext(ThemeContext); + if (!ctx) throw new Error("useTheme must be used within ThemeProvider"); + return ctx; +} diff --git a/understand-anything-plugin/packages/dashboard/src/themes/index.ts b/understand-anything-plugin/packages/dashboard/src/themes/index.ts new file mode 100644 index 0000000..c033d59 --- /dev/null +++ b/understand-anything-plugin/packages/dashboard/src/themes/index.ts @@ -0,0 +1,5 @@ +export { ThemeProvider, useTheme } from "./ThemeContext.tsx"; +export { PRESETS, getPreset, getAccent } from "./presets.ts"; +export { applyTheme } from "./theme-engine.ts"; +export type { PresetId, ThemeConfig, ThemePreset, AccentSwatch } from "./types.ts"; +export { DEFAULT_THEME_CONFIG } from "./types.ts"; diff --git a/understand-anything-plugin/packages/dashboard/src/themes/presets.ts b/understand-anything-plugin/packages/dashboard/src/themes/presets.ts new file mode 100644 index 0000000..35e0517 --- /dev/null +++ b/understand-anything-plugin/packages/dashboard/src/themes/presets.ts @@ -0,0 +1,143 @@ +import type { AccentSwatch, ThemePreset } from "./types.ts"; + +const DARK_ACCENT_SWATCHES: AccentSwatch[] = [ + { id: "gold", name: "Gold", accent: "#d4a574", accentDim: "#c9a96e", accentBright: "#e8c49a" }, + { id: "ocean", name: "Ocean", accent: "#5ba4cf", accentDim: "#4e93ba", accentBright: "#7abce0" }, + { id: "emerald", name: "Emerald", accent: "#5ea67a", accentDim: "#4e9468", accentBright: "#78c492" }, + { id: "rose", name: "Rose", accent: "#cf7a8a", accentDim: "#b96e7e", accentBright: "#e094a4" }, + { id: "purple", name: "Purple", accent: "#9b7abf", accentDim: "#876bb0", accentBright: "#b494d4" }, + { id: "amber", name: "Amber", accent: "#c9963a", accentDim: "#b5862e", accentBright: "#ddb05c" }, + { id: "teal", name: "Teal", accent: "#4aab9a", accentDim: "#3d9686", accentBright: "#68c4b4" }, + { id: "silver", name: "Silver", accent: "#a0a8b0", accentDim: "#8e959c", accentBright: "#b8bfc6" }, +]; + +const LIGHT_ACCENT_SWATCHES: AccentSwatch[] = [ + { id: "indigo", name: "Indigo", accent: "#4a6fa5", accentDim: "#3d5f8f", accentBright: "#6088bf" }, + { id: "ocean", name: "Ocean", accent: "#3a8ab5", accentDim: "#2e7aa0", accentBright: "#55a0cc" }, + { id: "emerald", name: "Emerald", accent: "#3a8a5c", accentDim: "#2e7a4e", accentBright: "#55a878" }, + { id: "rose", name: "Rose", accent: "#a5566a", accentDim: "#8f4a5c", accentBright: "#bf6e82" }, + { id: "purple", name: "Purple", accent: "#6b5a9e", accentDim: "#5c4d8a", accentBright: "#8474b5" }, + { id: "amber", name: "Amber", accent: "#9e7a30", accentDim: "#8a6a28", accentBright: "#b5923e" }, + { id: "teal", name: "Teal", accent: "#2e8a7a", accentDim: "#267a6c", accentBright: "#45a595" }, + { id: "slate", name: "Slate", accent: "#5a6570", accentDim: "#4e5860", accentBright: "#6e7a85" }, +]; + +export const PRESETS: ThemePreset[] = [ + { + id: "dark-gold", + name: "Dark Gold", + isDark: true, + defaultAccentId: "gold", + accentSwatches: DARK_ACCENT_SWATCHES, + colors: { + root: "#0a0a0a", + surface: "#111111", + elevated: "#1a1a1a", + panel: "#141414", + "text-primary": "#f5f0eb", + "text-secondary": "#a39787", + "text-muted": "#6b5f53", + "node-file": "#4a7c9b", + "node-function": "#5a9e6f", + "node-class": "#8b6fb0", + "node-module": "#c9a06c", + "node-concept": "#b07a8a", + }, + }, + { + id: "dark-ocean", + name: "Dark Ocean", + isDark: true, + defaultAccentId: "ocean", + accentSwatches: DARK_ACCENT_SWATCHES, + colors: { + root: "#0a0e14", + surface: "#111820", + elevated: "#1a222c", + panel: "#141c24", + "text-primary": "#e8edf2", + "text-secondary": "#87939f", + "text-muted": "#536b7a", + "node-file": "#4a7c9b", + "node-function": "#5a9e6f", + "node-class": "#8b6fb0", + "node-module": "#c9a06c", + "node-concept": "#b07a8a", + }, + }, + { + id: "dark-forest", + name: "Dark Forest", + isDark: true, + defaultAccentId: "emerald", + accentSwatches: DARK_ACCENT_SWATCHES, + colors: { + root: "#0a100a", + surface: "#111811", + elevated: "#1a241a", + panel: "#141c14", + "text-primary": "#ebf0eb", + "text-secondary": "#87a38f", + "text-muted": "#536b5a", + "node-file": "#4a7c9b", + "node-function": "#5a9e6f", + "node-class": "#8b6fb0", + "node-module": "#c9a06c", + "node-concept": "#b07a8a", + }, + }, + { + id: "dark-rose", + name: "Dark Rose", + isDark: true, + defaultAccentId: "rose", + accentSwatches: DARK_ACCENT_SWATCHES, + colors: { + root: "#100a0a", + surface: "#181111", + elevated: "#221a1a", + panel: "#1c1414", + "text-primary": "#f2e8ea", + "text-secondary": "#9f8790", + "text-muted": "#6b535a", + "node-file": "#4a7c9b", + "node-function": "#5a9e6f", + "node-class": "#8b6fb0", + "node-module": "#c9a06c", + "node-concept": "#b07a8a", + }, + }, + { + id: "light-minimal", + name: "Light Minimal", + isDark: false, + defaultAccentId: "indigo", + accentSwatches: LIGHT_ACCENT_SWATCHES, + colors: { + root: "#f5f3f0", + surface: "#eae7e3", + elevated: "#ffffff", + panel: "#f0ede9", + "text-primary": "#1a1a1a", + "text-secondary": "#6b6b6b", + "text-muted": "#a0a0a0", + "node-file": "#3a6a87", + "node-function": "#488a5b", + "node-class": "#755d99", + "node-module": "#a88a56", + "node-concept": "#966674", + }, + }, +]; + +export function getPreset(id: string): ThemePreset { + return PRESETS.find((p) => p.id === id) ?? PRESETS[0]; +} + +export function getAccent(preset: ThemePreset, accentId: string): AccentSwatch { + return ( + preset.accentSwatches.find((s) => s.id === accentId) ?? + preset.accentSwatches.find((s) => s.id === preset.defaultAccentId) ?? + preset.accentSwatches[0] + ); +} diff --git a/understand-anything-plugin/packages/dashboard/src/themes/theme-engine.ts b/understand-anything-plugin/packages/dashboard/src/themes/theme-engine.ts new file mode 100644 index 0000000..23004ca --- /dev/null +++ b/understand-anything-plugin/packages/dashboard/src/themes/theme-engine.ts @@ -0,0 +1,56 @@ +import type { ThemeConfig } from "./types.ts"; +import { getAccent, getPreset } from "./presets.ts"; + +export function hexToRgb(hex: string): string { + const h = hex.replace("#", ""); + const n = parseInt(h, 16); + return `${(n >> 16) & 255}, ${(n >> 8) & 255}, ${n & 255}`; +} + +function deriveFromAccent(accentHex: string, isDark: boolean): Record { + const rgb = hexToRgb(accentHex); + return { + "color-border-subtle": `rgba(${rgb}, ${isDark ? 0.12 : 0.1})`, + "color-border-medium": `rgba(${rgb}, ${isDark ? 0.25 : 0.18})`, + "glass-bg": isDark ? "rgba(20, 20, 20, 0.8)" : "rgba(255, 255, 255, 0.8)", + "glass-bg-heavy": isDark ? "rgba(20, 20, 20, 0.95)" : "rgba(255, 255, 255, 0.95)", + "glass-border": `rgba(${rgb}, ${isDark ? 0.1 : 0.08})`, + "glass-border-heavy": `rgba(${rgb}, ${isDark ? 0.15 : 0.12})`, + "scrollbar-thumb": `rgba(${rgb}, 0.2)`, + "scrollbar-thumb-hover": `rgba(${rgb}, 0.35)`, + "glow-accent": `rgba(${rgb}, 0.15)`, + "glow-accent-strong": `rgba(${rgb}, 0.4)`, + "glow-accent-pulse": `rgba(${rgb}, 0.6)`, + "color-edge": `rgba(${rgb}, 0.3)`, + "color-edge-dim": `rgba(${rgb}, 0.08)`, + "color-edge-dot": `rgba(${rgb}, 0.15)`, + "color-accent-overlay-bg": `rgba(${rgb}, 0.05)`, + "color-accent-overlay-border": `rgba(${rgb}, 0.25)`, + "kbd-bg": `rgba(${rgb}, 0.1)`, + }; +} + +export function applyTheme(config: ThemeConfig): void { + const preset = getPreset(config.presetId); + const accent = getAccent(preset, config.accentId); + const style = document.documentElement.style; + + // 1. Apply base preset colors + for (const [key, value] of Object.entries(preset.colors)) { + style.setProperty(`--color-${key}`, value); + } + + // 2. Apply accent colors from swatch + style.setProperty("--color-accent", accent.accent); + style.setProperty("--color-accent-dim", accent.accentDim); + style.setProperty("--color-accent-bright", accent.accentBright); + + // 3. Apply derived values + const derived = deriveFromAccent(accent.accent, preset.isDark); + for (const [key, value] of Object.entries(derived)) { + style.setProperty(`--${key}`, value); + } + + // 4. Set data-theme for CSS-only selectors + document.documentElement.setAttribute("data-theme", preset.isDark ? "dark" : "light"); +} diff --git a/understand-anything-plugin/packages/dashboard/src/themes/types.ts b/understand-anything-plugin/packages/dashboard/src/themes/types.ts new file mode 100644 index 0000000..2d09590 --- /dev/null +++ b/understand-anything-plugin/packages/dashboard/src/themes/types.ts @@ -0,0 +1,33 @@ +export type PresetId = + | "dark-gold" + | "dark-ocean" + | "dark-forest" + | "dark-rose" + | "light-minimal"; + +export interface AccentSwatch { + id: string; + name: string; + accent: string; + accentDim: string; + accentBright: string; +} + +export interface ThemePreset { + id: PresetId; + name: string; + isDark: boolean; + colors: Record; + accentSwatches: AccentSwatch[]; + defaultAccentId: string; +} + +export interface ThemeConfig { + presetId: PresetId; + accentId: string; +} + +export const DEFAULT_THEME_CONFIG: ThemeConfig = { + presetId: "dark-gold", + accentId: "gold", +}; diff --git a/understand-anything-plugin/packages/dashboard/vite.config.ts b/understand-anything-plugin/packages/dashboard/vite.config.ts index aa60f93..cd43a83 100644 --- a/understand-anything-plugin/packages/dashboard/vite.config.ts +++ b/understand-anything-plugin/packages/dashboard/vite.config.ts @@ -3,8 +3,21 @@ import react from "@vitejs/plugin-react"; import tailwindcss from "@tailwindcss/vite"; import path from "path"; import fs from "fs"; +import crypto from "crypto"; + +// Generate a one-time token when the server process starts. +// This token is printed to the terminal and must be in the URL +// to fetch knowledge-graph.json or diff-overlay.json. +const ACCESS_TOKEN = crypto.randomBytes(16).toString("hex"); export default defineConfig({ + // FIX 1 — bind only to localhost, not 0.0.0.0 + // This blocks access from any other device on the same LAN / WiFi. + server: { + host: "127.0.0.1", + port: 5173, + }, + resolve: { alias: { "@understand-anything/core/schema": path.resolve(__dirname, "../core/dist/schema.js"), @@ -12,53 +25,119 @@ export default defineConfig({ "@understand-anything/core/types": path.resolve(__dirname, "../core/dist/types.js"), }, }, + plugins: [ react(), tailwindcss(), { name: "serve-knowledge-graph", configureServer(server) { + // Print the access URL once so the developer can open it. + server.httpServer?.once("listening", () => { + console.log( + `\n 🔑 Dashboard URL: http://127.0.0.1:5173?token=${ACCESS_TOKEN}\n` + ); + }); + server.middlewares.use((req, res, next) => { - if (req.url === "/knowledge-graph.json") { - // GRAPH_DIR env var points to the project being analyzed - // Falls back to monorepo root, then public/ (demo) - const graphDir = process.env.GRAPH_DIR; - const candidates = [ - ...(graphDir - ? [path.resolve(graphDir, ".understand-anything/knowledge-graph.json")] - : []), - path.resolve(process.cwd(), ".understand-anything/knowledge-graph.json"), - path.resolve(process.cwd(), "../../../.understand-anything/knowledge-graph.json"), - ]; - for (const candidate of candidates) { - if (fs.existsSync(candidate)) { - res.setHeader("Content-Type", "application/json"); - fs.createReadStream(candidate).pipe(res); - return; - } - } - } - if (req.url === "/diff-overlay.json") { - const graphDir = process.env.GRAPH_DIR; - const candidates = [ - ...(graphDir - ? [path.resolve(graphDir, ".understand-anything/diff-overlay.json")] - : []), - path.resolve(process.cwd(), ".understand-anything/diff-overlay.json"), - path.resolve(process.cwd(), "../../../.understand-anything/diff-overlay.json"), - ]; - for (const candidate of candidates) { - if (fs.existsSync(candidate)) { - res.setHeader("Content-Type", "application/json"); - fs.createReadStream(candidate).pipe(res); - return; - } - } - res.statusCode = 404; - res.end(); + const url = new URL(req.url ?? "/", "http://127.0.0.1:5173"); + const pathname = url.pathname; + const isProtectedEndpoint = + pathname === "/knowledge-graph.json" || + pathname === "/diff-overlay.json" || + pathname === "/meta.json"; + + if (!isProtectedEndpoint) { + next(); return; } - next(); + + // FIX 3 — require the one-time token on all data endpoints. + // Requests without a matching ?token= get a 403. + if (url.searchParams.get("token") !== ACCESS_TOKEN) { + res.statusCode = 403; + res.setHeader("Content-Type", "application/json"); + res.end(JSON.stringify({ error: "Forbidden: missing or invalid token" })); + return; + } + + const fileName = + pathname === "/diff-overlay.json" + ? "diff-overlay.json" + : pathname === "/meta.json" + ? "meta.json" + : "knowledge-graph.json"; + + const graphDir = process.env.GRAPH_DIR; + const candidates = [ + ...(graphDir + ? [path.resolve(graphDir, `.understand-anything/${fileName}`)] + : []), + path.resolve(process.cwd(), `.understand-anything/${fileName}`), + path.resolve( + process.cwd(), + `../../../.understand-anything/${fileName}` + ), + ]; + + for (const candidate of candidates) { + if (!fs.existsSync(candidate)) continue; + + // FIX 2 — sanitise absolute file paths before sending the JSON. + // Nodes can contain filePath values like /Users/alice/company/src/auth.ts. + // We convert those to relative paths (src/auth.ts) so the developer's + // home directory and company directory layout are not leaked. + try { + const raw = JSON.parse(fs.readFileSync(candidate, "utf-8")) as { + nodes?: Array>; + [key: string]: unknown; + }; + + // Derive the project root from the candidate path so we can + // make file paths relative to it. + const projectRoot = path.dirname( + candidate.replace( + `${path.sep}.understand-anything${path.sep}${fileName}`, + "" + ) + ); + + if (Array.isArray(raw.nodes)) { + raw.nodes = raw.nodes.map((node) => { + if (typeof node.filePath !== "string") return node; + const abs = node.filePath; + // Only relativise paths that actually sit inside projectRoot. + // Leave external or already-relative paths untouched. + const rel = abs.startsWith(projectRoot) + ? abs.slice(projectRoot.length).replace(/^[\\/]/, "") + : path.isAbsolute(abs) + ? path.basename(abs) // absolute but outside root — use filename only + : abs; // already relative — keep as-is + return { ...node, filePath: rel }; + }); + } + + res.setHeader("Content-Type", "application/json"); + res.end(JSON.stringify(raw)); + } catch (err) { + // If we cannot parse or sanitise the file, refuse to serve it + // rather than accidentally leaking raw content. + console.error("[understand-anything] Failed to sanitise graph file:", err); + res.statusCode = 500; + res.setHeader("Content-Type", "application/json"); + res.end(JSON.stringify({ error: "Failed to read graph file" })); + } + return; + } + + // No matching file found on disk. + res.statusCode = 404; + if (pathname === "/knowledge-graph.json") { + res.setHeader("Content-Type", "application/json"); + res.end(JSON.stringify({ error: "No knowledge graph found. Run /understand first." })); + } else { + res.end(); + } }); }, }, diff --git a/understand-anything-plugin/skills/understand-dashboard/SKILL.md b/understand-anything-plugin/skills/understand-dashboard/SKILL.md index e0614d7..4c20dbb 100644 --- a/understand-anything-plugin/skills/understand-dashboard/SKILL.md +++ b/understand-anything-plugin/skills/understand-dashboard/SKILL.md @@ -56,7 +56,7 @@ Start the Understand Anything dashboard to visualize the knowledge graph for the 5. Start the Vite dev server pointing at the project's knowledge graph: ```bash - cd && GRAPH_DIR= npx vite --open + cd && GRAPH_DIR= npx vite --host 127.0.0.1 --open ``` Run this in the background so the user can continue working. diff --git a/understand-anything-plugin/skills/understand/SKILL.md b/understand-anything-plugin/skills/understand/SKILL.md index 71f188f..5dff038 100644 --- a/understand-anything-plugin/skills/understand/SKILL.md +++ b/understand-anything-plugin/skills/understand/SKILL.md @@ -12,6 +12,9 @@ Analyze the current codebase and produce a `knowledge-graph.json` file in `.unde - `$ARGUMENTS` may contain: - `--full` — Force a full rebuild, ignoring any existing graph + - `--auto-update` — Enable automatic graph updates on commit (writes `autoUpdate: true` to `.understand-anything/config.json`) + - `--no-auto-update` — Disable automatic graph updates (writes `autoUpdate: false` to `.understand-anything/config.json`) + - `--review` — Run full LLM graph-reviewer instead of inline deterministic validation - A directory path — Scope analysis to a specific subdirectory --- @@ -30,6 +33,11 @@ Determine whether to run a full analysis or incremental update. mkdir -p $PROJECT_ROOT/.understand-anything/intermediate mkdir -p $PROJECT_ROOT/.understand-anything/tmp ``` +3.5. **Auto-update configuration:** + - If `--auto-update` is in `$ARGUMENTS`: write `{"autoUpdate": true}` to `$PROJECT_ROOT/.understand-anything/config.json` + - If `--no-auto-update` is in `$ARGUMENTS`: write `{"autoUpdate": false}` to `$PROJECT_ROOT/.understand-anything/config.json` + - These flags only set the config — analysis proceeds normally regardless. + 4. Check if `$PROJECT_ROOT/.understand-anything/knowledge-graph.json` exists. If it does, read it. 5. Check if `$PROJECT_ROOT/.understand-anything/meta.json` exists. If it does, read it to get `gitCommitHash`. 6. **Decision logic:** @@ -38,9 +46,12 @@ Determine whether to run a full analysis or incremental update. |---|---| | `--full` flag in `$ARGUMENTS` | Full analysis (all phases) | | No existing graph or meta | Full analysis (all phases) | - | Existing graph + unchanged commit hash | Report "Graph is up to date" and STOP | + | `--review` flag + existing graph + unchanged commit hash | Skip to Phase 6 (review-only — reuse existing assembled graph) | + | Existing graph + unchanged commit hash | Ask the user: "The graph is up to date at this commit. Would you like to: **(a)** run a full rebuild (`--full`), **(b)** run the LLM graph reviewer (`--review`), or **(c)** do nothing?" Then follow their choice. If they pick (c), STOP. | | Existing graph + changed files | Incremental update (re-analyze changed files only) | + **Review-only path:** Copy the existing `knowledge-graph.json` to `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`, then jump directly to Phase 6 step 3. + For incremental updates, get the changed file list: ```bash git diff ..HEAD --name-only @@ -88,6 +99,9 @@ After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermedi - Languages, frameworks - File list with line counts - Complexity estimate +- Import map (`importMap`): pre-resolved project-internal imports per file + +Store `importMap` in memory as `$IMPORT_MAP` for use in Phase 2 batch construction. **Gate check:** If >200 files, inform the user and suggest scoping with a subdirectory argument. Proceed only if user confirms or add guidance that this may take a while. @@ -97,24 +111,21 @@ After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermedi ### Full analysis path -Batch the file list from Phase 1 into groups of **5-10 files each** (aim for balanced batch sizes). +Batch the file list from Phase 1 into groups of **20-30 files each** (aim for ~25 files per batch for balanced sizes). -For each batch, dispatch a subagent using the prompt template at `./file-analyzer-prompt.md`. Run up to **3 subagents concurrently** using parallel dispatch. - -**Build the combined prompt template:** -1. Read the base template at `./file-analyzer-prompt.md`. -2. **Language context injection:** For each language detected in Phase 1 (e.g., `python`), read the file at `./languages/.md` (e.g., `./languages/python.md`) and append its content after the base template under a `## Language Context` header. If the file does not exist for a detected language, skip it silently and continue. These files are in the `languages/` subdirectory next to this SKILL.md file. Use `ls ./languages/` to discover available language files if needed. -3. **Framework addendum injection:** For each framework detected in Phase 1 (e.g., `Django`), read the file at `./frameworks/.md` (e.g., `./frameworks/django.md`) and append its full content after the language context. If the file does not exist for a detected framework, skip it silently and continue. These files are in the `frameworks/` subdirectory next to this SKILL.md file. Use `ls ./frameworks/` to discover available framework files if needed. - -Then for each batch pass the combined template content as the subagent's prompt, appending the following additional context: +For each batch, dispatch a subagent using the prompt template at `./file-analyzer-prompt.md`. Run up to **5 subagents concurrently** using parallel dispatch. Pass the template as the subagent's prompt, appending the following additional context: > **Additional context from main session:** > > Project: `` — `` -> Frameworks detected: `` > Languages: `` -> -> Use the language context and framework addendums (appended above) to produce more accurate summaries and better classify file roles. + +Before dispatching each batch, construct `batchImportData` from `$IMPORT_MAP`: +```json +batchImportData = {} +for each file in this batch: + batchImportData[file.path] = $IMPORT_MAP[file.path] ?? [] +``` Fill in batch-specific parameters below and dispatch: @@ -125,8 +136,10 @@ Fill in batch-specific parameters below and dispatch: > Batch index: `` > Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/batch-.json` > -> All project files (for import resolution): -> `` +> Pre-resolved import data for this batch (use this for all import edge creation — do NOT re-resolve imports from source): +> ```json +> +> ``` > > Files to analyze in this batch: > 1. `` ( lines) @@ -139,7 +152,7 @@ After ALL batches complete, read each `batch-.json` file and merge: ### Incremental update path -Use the changed files list from Phase 0. Batch and dispatch file-analyzer subagents using the same process as above, but only for changed files. +Use the changed files list from Phase 0. Batch and dispatch file-analyzer subagents using the same process as above (20-30 files per batch, up to 5 concurrent, with batchImportData constructed from $IMPORT_MAP), but only for changed files. After batches complete, merge with the existing graph: 1. Remove old nodes whose `filePath` matches any changed file @@ -187,7 +200,7 @@ Pass these parameters in the dispatch prompt: > > File nodes: > ```json -> [list of {id, name, filePath, summary, tags} for all file-type nodes] +> [list of {id, name, filePath, summary, tags} for all file-type nodes — omit complexity, languageNotes] > ``` > > Import edges: @@ -254,19 +267,19 @@ Pass these parameters in the dispatch prompt: > Project: `` — `` > Languages: `` > -> Nodes (summarized): +> Nodes (file nodes only): > ```json -> [list of {id, name, filePath, summary, type} for key nodes] +> [list of {id, name, filePath, summary, type} for file-type nodes ONLY — do NOT include function or class nodes] > ``` > > Layers: > ```json -> [layers from Phase 4] +> [list of {id, name, description} for each layer — omit nodeIds] > ``` > -> Key edges: +> Edges (imports and calls only): > ```json -> [imports and calls edges] +> [list of edges where type is "imports" or "calls" only — exclude all other edge types] > ``` After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermediate/tour.json` and normalize it into a final `tour` array. Apply these steps **in order**: @@ -327,7 +340,95 @@ Assemble the full KnowledgeGraph JSON object: 2. Write the assembled graph to `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`. -3. Dispatch a subagent using the prompt template at `./graph-reviewer-prompt.md`. Read the template file and pass the full content as the subagent's prompt, appending the following additional context: +3. **Check `$ARGUMENTS` for `--review` flag.** Then run the appropriate validation path: + +--- + +#### Default path (no `--review`): inline deterministic validation + +Write the following Node.js script to `$PROJECT_ROOT/.understand-anything/tmp/ua-inline-validate.cjs`: + +```javascript +#!/usr/bin/env node +const fs = require('fs'); +const graphPath = process.argv[2]; +const outputPath = process.argv[3]; +try { + const graph = JSON.parse(fs.readFileSync(graphPath, 'utf8')); + const issues = [], warnings = []; + if (!Array.isArray(graph.nodes)) { issues.push('graph.nodes is missing or not an array'); graph.nodes = []; } + if (!Array.isArray(graph.edges)) { issues.push('graph.edges is missing or not an array'); graph.edges = []; } + const nodeIds = new Set(); + const seen = new Map(); + graph.nodes.forEach((n, i) => { + if (!n.id) { issues.push(`Node[${i}] missing id`); return; } + if (!n.type) issues.push(`Node[${i}] '${n.id}' missing type`); + if (!n.name) issues.push(`Node[${i}] '${n.id}' missing name`); + if (!n.summary) issues.push(`Node[${i}] '${n.id}' missing summary`); + if (!n.tags || !n.tags.length) issues.push(`Node[${i}] '${n.id}' missing tags`); + if (seen.has(n.id)) issues.push(`Duplicate node ID '${n.id}' at indices ${seen.get(n.id)} and ${i}`); + else seen.set(n.id, i); + nodeIds.add(n.id); + }); + graph.edges.forEach((e, i) => { + if (!nodeIds.has(e.source)) issues.push(`Edge[${i}] source '${e.source}' not found`); + if (!nodeIds.has(e.target)) issues.push(`Edge[${i}] target '${e.target}' not found`); + }); + const fileNodes = graph.nodes.filter(n => n.type === 'file').map(n => n.id); + const assigned = new Map(); + if (!Array.isArray(graph.layers)) { if (graph.layers) warnings.push('graph.layers is not an array'); graph.layers = []; } + if (!Array.isArray(graph.tour)) { if (graph.tour) warnings.push('graph.tour is not an array'); graph.tour = []; } + graph.layers.forEach(layer => { + (layer.nodeIds || []).forEach(id => { + if (!nodeIds.has(id)) issues.push(`Layer '${layer.id}' refs missing node '${id}'`); + if (assigned.has(id)) issues.push(`Node '${id}' appears in multiple layers`); + assigned.set(id, layer.id); + }); + }); + fileNodes.forEach(id => { + if (!assigned.has(id)) issues.push(`File node '${id}' not in any layer`); + }); + graph.tour.forEach((step, i) => { + (step.nodeIds || []).forEach(id => { + if (!nodeIds.has(id)) issues.push(`Tour step[${i}] refs missing node '${id}'`); + }); + }); + const withEdges = new Set([ + ...graph.edges.map(e => e.source), + ...graph.edges.map(e => e.target) + ]); + graph.nodes.forEach(n => { + if (!withEdges.has(n.id)) warnings.push(`Node '${n.id}' has no edges (orphan)`); + }); + const stats = { + totalNodes: graph.nodes.length, + totalEdges: graph.edges.length, + totalLayers: graph.layers.length, + tourSteps: graph.tour.length, + nodeTypes: graph.nodes.reduce((a, n) => { a[n.type] = (a[n.type]||0)+1; return a; }, {}), + edgeTypes: graph.edges.reduce((a, e) => { a[e.type] = (a[e.type]||0)+1; return a; }, {}) + }; + fs.writeFileSync(outputPath, JSON.stringify({ issues, warnings, stats }, null, 2)); + process.exit(0); +} catch (err) { process.stderr.write(err.message + '\n'); process.exit(1); } +``` + +Execute it: +```bash +node $PROJECT_ROOT/.understand-anything/tmp/ua-inline-validate.cjs \ + "$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json" \ + "$PROJECT_ROOT/.understand-anything/intermediate/review.json" +``` + +If the script exits non-zero, read stderr, fix the script, and retry once. + +--- + +#### `--review` path: full LLM reviewer + +If `--review` IS in `$ARGUMENTS`, dispatch the LLM graph-reviewer subagent as follows: + +Dispatch a subagent using the prompt template at `./graph-reviewer-prompt.md`. Read the template file and pass the full content as the subagent's prompt, appending the following additional context: > **Additional context from main session:** > @@ -343,14 +444,16 @@ Assemble the full KnowledgeGraph JSON object: Pass these parameters in the dispatch prompt: - > Validate the knowledge graph at `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`. - > Project root: `$PROJECT_ROOT` - > Read the file and validate it for completeness and correctness. - > Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/review.json` +> Validate the knowledge graph at `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`. +> Project root: `$PROJECT_ROOT` +> Read the file and validate it for completeness and correctness. +> Write output to: `$PROJECT_ROOT/.understand-anything/intermediate/review.json` -4. After the subagent completes, read `$PROJECT_ROOT/.understand-anything/intermediate/review.json`. +--- -5. **If `approved: false`:** +4. Read `$PROJECT_ROOT/.understand-anything/intermediate/review.json`. + +5. **If `issues` array is non-empty:** - Review the `issues` list - Apply automated fixes where possible: - Remove edges with dangling references @@ -359,7 +462,7 @@ Pass these parameters in the dispatch prompt: - Re-run the final graph validation after automated fixes - If critical issues remain after one fix attempt, save the graph anyway but include the warnings in the final report and mark dashboard auto-launch as skipped -6. **If `approved: true`:** Proceed to Phase 7. +6. **If `issues` array is empty:** Proceed to Phase 7. --- @@ -377,6 +480,18 @@ Pass these parameters in the dispatch prompt: } ``` +2.5. **Generate structural fingerprints** for all analyzed files and save to `$PROJECT_ROOT/.understand-anything/fingerprints.json`. This creates the baseline for future automatic incremental updates. + + Write and execute a Node.js script that uses the core fingerprint module (tree-sitter-based, not regex): + ```javascript + import { buildFingerprintStore } from '@understand-anything/core'; + import { saveFingerprints } from '@understand-anything/core'; + + const store = await buildFingerprintStore('', sourceFilePaths); + saveFingerprints('', store); + ``` + Where `sourceFilePaths` is the list of all analyzed source file paths from Phase 1. This uses the same tree-sitter analysis pipeline as the main fingerprint engine, ensuring the baseline matches the comparison logic used during auto-updates. + 3. Clean up intermediate files: ```bash rm -rf $PROJECT_ROOT/.understand-anything/intermediate @@ -401,7 +516,7 @@ Pass these parameters in the dispatch prompt: ## Error Handling - If any subagent dispatch fails, retry **once** with the same prompt plus additional context about the failure. -- Track all warnings and errors from each phase in a `$PHASE_WARNINGS` list. Pass this list to the graph-reviewer in Phase 6 for comprehensive validation. +- Track all warnings and errors from each phase in a `$PHASE_WARNINGS` list. When using `--review`, pass this list to the graph-reviewer in Phase 6. On the default path, include accumulated warnings in the Phase 7 final report. - If it fails a second time, skip that phase and continue with partial results. - ALWAYS save partial results — a partial graph is better than no graph. - Report any skipped phases or errors in the final summary so the user knows what happened. diff --git a/understand-anything-plugin/skills/understand/file-analyzer-prompt.md b/understand-anything-plugin/skills/understand/file-analyzer-prompt.md index 27534a5..26df55f 100644 --- a/understand-anything-plugin/skills/understand/file-analyzer-prompt.md +++ b/understand-anything-plugin/skills/understand/file-analyzer-prompt.md @@ -20,11 +20,14 @@ Write a script that reads each source file in your batch and extracts determinis ```json { "projectRoot": "/path/to/project", - "allProjectFiles": ["src/index.ts", "src/utils.ts", "..."], "batchFiles": [ {"path": "src/index.ts", "language": "typescript", "sizeLines": 150}, {"path": "src/utils.ts", "language": "typescript", "sizeLines": 80} - ] + ], + "batchImportData": { + "src/index.ts": ["src/utils.ts", "src/config.ts"], + "src/utils.ts": [] + } } ``` 2. **Write** results JSON to the path given as the second argument. @@ -45,10 +48,9 @@ For each file in `batchFiles`, read the file content and extract: - Detection approach: match `class `, `interface `, `type =`, `struct `, `trait `, `impl ` as appropriate **Imports:** -- Source module path (exactly as written in the import statement) -- Imported specifiers (named imports, default import, namespace import) -- Line number -- For relative imports (starting with `./` or `../`), compute the resolved path relative to project root. Cross-reference against `allProjectFiles` to confirm the resolved path exists. Mark unresolvable imports. +- Do NOT extract imports in the script. Import resolution has already been performed by the project scanner. +- The pre-resolved imports for each file are provided in `batchImportData` in the input JSON. +- Do not include an `imports` field in the script output — import edges will be created in Phase 2 using `batchImportData` directly. **Exports:** - Exported names and their line numbers @@ -57,7 +59,7 @@ For each file in `batchFiles`, read the file content and extract: **Basic Metrics:** - Total line count - Non-empty line count (lines that are not blank or comment-only) -- Import count (number of import statements) +- Import count — use `batchImportData[file.path].length` from the input JSON (do not count from source) - Export count (number of export statements) - Function count, class count @@ -82,10 +84,6 @@ The script must write this exact JSON structure to the output file: "classes": [ {"name": "App", "startLine": 50, "endLine": 140, "methods": ["init", "run"], "properties": ["config", "logger"]} ], - "imports": [ - {"source": "./utils", "resolvedPath": "src/utils.ts", "specifiers": ["formatDate", "sanitize"], "line": 1, "isExternal": false}, - {"source": "express", "resolvedPath": null, "specifiers": ["default"], "line": 2, "isExternal": true} - ], "exports": [ {"name": "App", "line": 50, "isDefault": true}, {"name": "createApp", "line": 145, "isDefault": false} @@ -114,8 +112,8 @@ Before writing the script, create its input JSON file. **IMPORTANT:** Use the ba cat > $PROJECT_ROOT/.understand-anything/tmp/ua-file-analyzer-input-.json << 'ENDJSON' { "projectRoot": "", - "allProjectFiles": [], - "batchFiles": [] + "batchFiles": [], + "batchImportData": } ENDJSON ``` @@ -202,7 +200,7 @@ Using the script's import, export, and structural data, create edges: | Edge Type | When to Create | Weight | Direction | |---|---|---|---| | `contains` | File contains a function or class node you created | `1.0` | `forward` | -| `imports` | File imports from another project file (use `resolvedPath` from script, skip external imports where `isExternal: true`) | `0.7` | `forward` | +| `imports` | File imports from another project file (use `batchImportData[filePath]` from input JSON — external imports already filtered out) | `0.7` | `forward` | | `calls` | A function in this file calls a function in another file (infer from imports + function names when confident) | `0.8` | `forward` | | `inherits` | A class extends another class in the project | `0.9` | `forward` | | `implements` | A class implements an interface in the project | `0.9` | `forward` | @@ -210,7 +208,7 @@ Using the script's import, export, and structural data, create edges: | `depends_on` | File has runtime dependency on another project file (broader than imports -- includes dynamic requires, lazy loads) | `0.6` | `forward` | | `tested_by` | Source file is tested by a test file (infer from test file imports and naming conventions) | `0.5` | `forward` | -**Import edge creation rule:** For each import in the script output where `isExternal` is `false` and `resolvedPath` is non-null, create an `imports` edge from the current file node to `file:`. Do NOT create edges for external package imports. +**Import edge creation rule:** For each resolved path in `batchImportData[filePath]` (provided in the input JSON), create an `imports` edge from the current file node to `file:`. The `batchImportData` values contain only resolved project-internal paths — external packages have already been filtered out. Do NOT attempt to re-resolve imports from source. Do NOT use edge types not listed in this table. @@ -295,13 +293,42 @@ Produce a single, valid JSON block. Validate it mentally before writing -- malfo - `direction` (string) -- always `forward` - `weight` (number) -- must match the weight specified in the edge type table +## Language and Framework Quick Reference + +Use these hints to improve tag and edge accuracy for common patterns. Your training knowledge covers these — this is a fast lookup for the most impactful signals. + +**Tag signals:** + +| Signal | Tags to apply | +|---|---| +| File in `hooks/`, exports a function starting with `use` | `hook`, `service` | +| File in `contexts/` or `context/`, exports a Provider component | `service`, `state` | +| File in `pages/` or `views/` | `ui`, `routing` | +| File in `store/`, `slices/`, `reducers/`, `state/` | `state` | +| File in `services/`, `api/`, `client/` | `service` | +| `__init__.py` at a package root with re-exports | `entry-point`, `barrel` | +| `manage.py` at the project root | `entry-point` | +| `mod.rs` in a directory | `barrel` | +| `main.go` in a `cmd/` subdirectory | `entry-point` | + +**Edge signals:** + +| Pattern | Edge to create | +|---|---| +| React component renders another component in its JSX | `contains` from parent to child | +| Component/hook calls a custom hook (`useX`) | `depends_on` from consumer to hook file | +| Context provider wraps components | `exports` from provider to context definition | +| Component calls `useContext` or custom context hook | `depends_on` from consumer to context definition | +| Python file uses `from x import y` where x is a project file | `imports` edge (same rule as JS/TS) | +| Go file `import`s an internal package path | `imports` edge to the resolved file | + ## Critical Constraints -- NEVER invent file paths. Every `filePath` and every file reference in node IDs must correspond to a real file from the script's output or the project file list provided to you. -- NEVER create edges to nodes that do not exist. If an import target is external (`isExternal: true` in script output), do NOT create an edge for it. +- NEVER invent file paths. Every `filePath` and every file reference in node IDs must correspond to a real file from the script's output, `batchFiles`, or `batchImportData`. +- NEVER create edges to nodes that do not exist. Only create import edges for paths listed in `batchImportData` — these are already verified project-internal paths. - ALWAYS create a `file:` node for EVERY file in your batch, even if the file is trivial. - Only create `function:` and `class:` nodes for significant code elements (see significance filter above). -- For import edges, use the script's `resolvedPath` field directly. Do NOT attempt to resolve import paths yourself -- the script already did this deterministically. +- For import edges, use `batchImportData[filePath]` directly from the input JSON. Do NOT attempt to resolve import paths yourself -- the project scanner already did this deterministically. - NEVER produce duplicate node IDs within your batch. - NEVER create self-referencing edges (where source equals target). - Trust the script's structural extraction. Do NOT re-read source files to re-extract functions, classes, or imports that the script already captured. Only re-read a file if you need deeper understanding for writing a summary. diff --git a/understand-anything-plugin/skills/understand/project-scanner-prompt.md b/understand-anything-plugin/skills/understand/project-scanner-prompt.md index b80de31..8febcc8 100644 --- a/understand-anything-plugin/skills/understand/project-scanner-prompt.md +++ b/understand-anything-plugin/skills/understand/project-scanner-prompt.md @@ -106,6 +106,40 @@ Extract from (in priority order): 4. `pyproject.toml` -- check `[project].name` first, then `[tool.poetry].name` 5. Directory name of project root +**Step 8 -- Import Resolution** + +For each file in the discovered source list, extract and resolve relative import statements. The goal is to produce a map from each file's path to the list of project-internal files it imports. External package imports are ignored. + +For each file, read its content and extract import paths using language-appropriate patterns: + +| Language | Import patterns to match | +|---|---| +| TypeScript/JavaScript | `import ... from './...'` or `'../'`, `require('./...')` or `require('../...')` | +| Python | `from .x import y`, `from ..x import y`, `from . import x` (relative only) | +| Go | Paths in `import (...)` blocks that start with the module path from `go.mod` | +| Rust | `use crate::`, `use super::`, `mod x` (within the same crate) | +| Java/Kotlin | Not resolvable by path — skip import resolution for these languages | +| Ruby | `require_relative '...'` paths | + +For each extracted import path: +1. Compute the resolved file path relative to project root: + - For relative imports (`./x`, `../x`): resolve from the importing file's directory + - Try these extension variants in order if the import has no extension: `.ts`, `.tsx`, `.js`, `.jsx`, `/index.ts`, `/index.js`, `/index.tsx`, `/index.jsx`, `.py`, `.go`, `.rs`, `.rb` +2. Check if the resolved path exists in the discovered file list +3. If yes: add to this file's resolved imports list +4. If no: skip (external, unresolvable, or dynamic import) + +Output format in the script result: +```json +"importMap": { + "src/index.ts": ["src/utils.ts", "src/config.ts"], + "src/utils.ts": [], + "src/components/App.tsx": ["src/hooks/useAuth.ts", "src/store/index.ts"] +} +``` + +Keys are project-relative paths. Values are arrays of resolved project-relative paths. Every key in the file list must appear in `importMap` (use an empty array `[]` if no imports were resolved). External packages and unresolvable imports are omitted entirely. + ### Script Output Format The script must write this exact JSON structure to the output file: @@ -122,7 +156,11 @@ The script must write this exact JSON structure to the output file: {"path": "src/index.ts", "language": "typescript", "sizeLines": 150} ], "totalFiles": 42, - "estimatedComplexity": "moderate" + "estimatedComplexity": "moderate", + "importMap": { + "src/index.ts": ["src/utils.ts", "src/config.ts"], + "src/utils.ts": [] + } } ``` @@ -135,6 +173,7 @@ The script must write this exact JSON structure to the output file: - `files` (object[]) -- every source file, sorted by `path` alphabetically - `totalFiles` (integer) -- must equal `files.length` - `estimatedComplexity` (string) -- one of `small`, `moderate`, `large`, `very-large` +- `importMap` (object) — map from every source file path to its list of resolved project-internal import paths; empty array if no resolved imports; external packages excluded ### Executing the Script @@ -154,7 +193,7 @@ If the script exits with a non-zero code, read stderr, diagnose the issue, fix t After the script completes, read `$PROJECT_ROOT/.understand-anything/tmp/ua-scan-results.json`. Do NOT re-run file discovery commands or re-count lines -- trust the script's results entirely. -**IMPORTANT:** The final output must NOT contain the `scriptCompleted`, `rawDescription`, or `readmeHead` fields. These are intermediate script fields only. Strip them when assembling the final JSON. +**IMPORTANT:** The final output must NOT contain the `scriptCompleted`, `rawDescription`, or `readmeHead` fields. These are intermediate script fields only. Strip them when assembling the final JSON. All other fields — including `importMap` — MUST be preserved exactly as output by the script. Your only task in this phase is to produce the final `description` field: @@ -175,7 +214,10 @@ Then assemble the final output JSON: {"path": "src/index.ts", "language": "typescript", "sizeLines": 150} ], "totalFiles": 42, - "estimatedComplexity": "moderate" + "estimatedComplexity": "moderate", + "importMap": { + "src/index.ts": ["src/utils.ts"] + } } ``` @@ -187,6 +229,7 @@ Then assemble the final output JSON: - `files` (object[]): directly from script output - `totalFiles` (integer): directly from script output - `estimatedComplexity` (string): directly from script output +- `importMap` (object): directly from script output ## Critical Constraints diff --git a/understand-anything-plugin/skills/understand/tour-builder-prompt.md b/understand-anything-plugin/skills/understand/tour-builder-prompt.md index 8b1d318..0c0dda8 100644 --- a/understand-anything-plugin/skills/understand/tour-builder-prompt.md +++ b/understand-anything-plugin/skills/understand/tour-builder-prompt.md @@ -20,13 +20,13 @@ Write a Node.js script that analyzes the graph's topology to surface structural ```json { "nodes": [ - {"id": "file:src/index.ts", "type": "file", "name": "index.ts", "filePath": "src/index.ts", "summary": "...", "tags": ["entry-point"]} + {"id": "file:src/index.ts", "type": "file", "name": "index.ts", "filePath": "src/index.ts", "summary": "..."} ], "edges": [ {"source": "file:src/index.ts", "target": "file:src/utils.ts", "type": "imports"} ], "layers": [ - {"id": "layer:core", "name": "Core", "nodeIds": ["file:src/index.ts"]} + {"id": "layer:core", "name": "Core", "description": "Core application logic"} ] } ``` @@ -47,7 +47,6 @@ For every node, count how many other nodes it has edges pointing TO (fan-out). H Identify likely entry points using these signals (score each file node, sum the scores): - Filename matches `index.ts`, `index.js`, `main.ts`, `main.js`, `app.ts`, `app.js`, `server.ts`, `server.js`, `mod.rs`, `main.go`, `main.py`, `main.rs`, `manage.py`, `app.py`, `wsgi.py`, `asgi.py`, `run.py`, `__main__.py`, `Application.java`, `Main.java`, `Program.cs`, `config.ru`, `index.php`, `App.swift`, `Application.kt`, `main.cpp`, `main.c` -> +3 points -- Node tags contain `entry-point` or `barrel` -> +2 points - File is at the project root or one level deep (e.g., `src/index.ts`) -> +1 point - High fan-out (top 10%) -> +1 point - Low fan-in (bottom 25%) -> +1 point (entry points are imported by few files) @@ -71,17 +70,15 @@ Algorithm: For each pair of nodes with a bidirectional relationship (A imports B Output the top 5-10 clusters, each as a list of node IDs. -**F. Layer Statistics** +**F. Layer List** -For each layer, compute: -- Number of file nodes -- Average fan-in of files in this layer -- Average fan-out of files in this layer -- The layer's "rank" in the dependency hierarchy (layers that are imported by many others but import few = foundational; layers that import many others but are imported by few = top-level) +Record the layers provided in the input. Since layers contain only `{id, name, description}` (no node membership), simply output the layer count and the list of layers with their id, name, and description. **G. Node Summary Index** -Create a lookup of each node ID to its `summary`, `type`, `tags` (default to empty array `[]` if not present in input), and `name` for easy reference. This lets the LLM phase quickly access semantic information without re-reading the full input. +Create a lookup of each node ID to its `summary`, `type`, and `name` for easy reference. This lets the LLM phase quickly access semantic information without re-reading the full input. + +Note: input nodes are file-type only. The nodeSummaryIndex will contain only file nodes. ### Script Output Format @@ -114,15 +111,19 @@ Create a lookup of each node ID to its `summary`, `type`, `tags` (default to emp "clusters": [ {"nodes": ["file:src/services/auth.ts", "file:src/models/user.ts"], "edgeCount": 4} ], - "layerStats": [ - {"id": "layer:core", "name": "Core", "fileCount": 5, "avgFanIn": 8.2, "avgFanOut": 3.1, "hierarchyRank": 1} - ], + "layers": { + "count": 3, + "list": [ + {"id": "layer:core", "name": "Core", "description": "Core application logic"}, + {"id": "layer:services", "name": "Services", "description": "Business logic services"}, + {"id": "layer:ui", "name": "UI", "description": "User interface components"} + ] + }, "nodeSummaryIndex": { - "file:src/index.ts": {"name": "index.ts", "type": "file", "summary": "Main entry point...", "tags": ["entry-point"]}, - "file:src/utils.ts": {"name": "utils.ts", "type": "file", "summary": "Shared helpers...", "tags": []} + "file:src/index.ts": {"name": "index.ts", "type": "file", "summary": "Main entry point..."}, + "file:src/utils.ts": {"name": "utils.ts", "type": "file", "summary": "Shared helpers..."} }, "totalNodes": 42, - "totalFileNodes": 20, "totalEdges": 87 } ``` @@ -179,13 +180,13 @@ You do not need to include every node from the BFS. Select the most important an When a `cluster` from the script output appears at the same BFS depth, group those nodes into a single tour step. Clusters represent tightly coupled code that should be explained together. -### Step 4 -- Use Layer Statistics for Narrative Arc +### Step 4 -- Use Layers for Narrative Arc -The `layerStats` with `hierarchyRank` tells you which layers are foundational vs. top-level. Structure the tour to explain foundational layers before the layers that depend on them. +The `layers` list gives you the project's architectural groupings. Use layer names and descriptions to understand which areas are foundational vs. top-level, and structure the tour to explain foundational layers before the layers that depend on them. ### Step 5 -- Write Step Descriptions -For each step, use the `nodeSummaryIndex` to access node summaries, names, and tags without re-reading files. Each description must: +For each step, use the `nodeSummaryIndex` to access node summaries and names without re-reading files. Each description must: - Explain WHAT this area does and WHY it matters to the project - Connect to previous steps (e.g., "Building on the User types from Step 2, this service implements...")