From 231ef484a0268f1054f030b67ee57611a74cef7e Mon Sep 17 00:00:00 2001 From: Lum1104 Date: Sat, 28 Mar 2026 18:45:20 +0800 Subject: [PATCH] feat(agents): update project-scanner to discover all file types Remove the code-only file filter so the scanner discovers non-code files (.md, .yaml, .json, .sql, .tf, Dockerfile, etc.). Add fileCategory field to each discovered file (code/config/docs/infra/data/script/markup) with extension-based category detection logic. Expand language detection table to cover 26+ file types. Infrastructure tooling detection added to framework detection step. Co-Authored-By: Claude Opus 4.6 (1M context) --- .../understand/project-scanner-prompt.md | 120 +++++++++++++----- 1 file changed, 91 insertions(+), 29 deletions(-) diff --git a/understand-anything-plugin/skills/understand/project-scanner-prompt.md b/understand-anything-plugin/skills/understand/project-scanner-prompt.md index 8febcc8..0a0f264 100644 --- a/understand-anything-plugin/skills/understand/project-scanner-prompt.md +++ b/understand-anything-plugin/skills/understand/project-scanner-prompt.md @@ -2,7 +2,7 @@ > Used by `/understand` Phase 1. Dispatch as a subagent with this full content as the prompt. -You are a meticulous project inventory specialist. Your job is to scan a codebase directory and produce a precise, structured inventory of all source files, detected languages, frameworks, and estimated complexity. Accuracy is paramount -- every file path you report must actually exist on disk. +You are a meticulous project inventory specialist. Your job is to scan a codebase directory and produce a precise, structured inventory of all project files, detected languages, frameworks, and estimated complexity. Accuracy is paramount -- every file path you report must actually exist on disk. ## Task @@ -12,7 +12,7 @@ Scan the project directory provided in the prompt and produce a JSON inventory. ## Phase 1 -- Discovery Script -Write a script that discovers all source files, detects languages and frameworks, counts lines, and produces structured JSON. Choose the best language for this task (bash, Node.js, or Python -- whichever is available on the system). The script must handle errors gracefully and never crash on unexpected input. +Write a script that discovers all project files (including non-code files like configs, docs, and infrastructure), detects languages and frameworks, counts lines, and produces structured JSON. Choose the best language for this task (bash, Node.js, or Python -- whichever is available on the system). The script must handle errors gracefully and never crash on unexpected input. ### Script Requirements @@ -38,10 +38,19 @@ Remove ALL files matching these patterns: - **Binary/asset files:** `.png`, `.jpg`, `.jpeg`, `.gif`, `.svg`, `.ico`, `.woff`, `.woff2`, `.ttf`, `.eot`, `.mp3`, `.mp4`, `.pdf`, `.zip`, `.tar`, `.gz` - **Generated files:** `*.min.js`, `*.min.css`, `*.map`, `*.d.ts`, `*.generated.*` - **IDE/editor config:** paths containing `.idea/`, `.vscode/` -- **Config/doc files:** `*.md`, `*.txt`, `*.yml`, `*.yaml`, `*.toml`, `*.json`, `*.xml`, `*.lock`, `*.cfg`, `*.ini`, `Makefile`, `Dockerfile` - **Misc non-source:** `LICENSE`, `.gitignore`, `.editorconfig`, `.prettierrc`, `.eslintrc*`, `*.log` -The goal is to keep ONLY source code files (`.ts`, `.tsx`, `.js`, `.jsx`, `.py`, `.go`, `.rs`, `.java`, `.rb`, `.cpp`, `.cc`, `.cxx`, `.h`, `.hpp`, `.c`, `.cs`, `.swift`, `.kt`, `.php`, `.vue`, `.svelte`, `.sh`, `.bash`). +**IMPORTANT:** Do NOT exclude non-code project files. The following MUST be kept: +- Documentation: `*.md`, `*.rst`, `*.txt` (except `LICENSE`) +- Configuration: `*.yaml`, `*.yml`, `*.json`, `*.toml`, `*.xml`, `*.cfg`, `*.ini`, `*.env`, `*.env.example` +- Infrastructure: `Dockerfile`, `docker-compose.*`, `*.tf`, `Makefile`, `Jenkinsfile`, `Procfile`, `Vagrantfile` +- CI/CD: `.github/workflows/*`, `.gitlab-ci.yml`, `.circleci/*`, `Jenkinsfile` +- Data/Schema: `*.sql`, `*.graphql`, `*.gql`, `*.proto`, `*.prisma`, `*.schema.json` +- Web markup: `*.html`, `*.css`, `*.scss`, `*.sass`, `*.less` +- Shell scripts: `*.sh`, `*.bash`, `*.ps1`, `*.bat` +- Kubernetes: `*.k8s.yaml`, `*.k8s.yml`, paths containing `k8s/`, paths containing `kubernetes/` + +**Note on package manifests:** Config files read for framework detection (`package.json`, `tsconfig.json`, `Cargo.toml`, `go.mod`, `pyproject.toml`, etc.) should also appear in the file list with `fileCategory: "config"`. **Step 3 -- Language Detection** @@ -64,17 +73,48 @@ Map file extensions to language identifiers: | `.php` | `php` | | `.vue` | `vue` | | `.svelte` | `svelte` | -| `.sh`, `.bash` | `bash` | +| `.sh`, `.bash` | `shell` | +| `.md`, `.rst` | `markdown` | +| `.yaml`, `.yml` | `yaml` | +| `.json` | `json` | +| `.toml` | `toml` | +| `.sql` | `sql` | +| `.graphql`, `.gql` | `graphql` | +| `.proto` | `protobuf` | +| `.tf`, `.tfvars` | `terraform` | +| `.html`, `.htm` | `html` | +| `.css`, `.scss`, `.sass`, `.less` | `css` | +| `.xml` | `xml` | +| `.cfg`, `.ini`, `.env` | `config` | +| `Dockerfile` (no extension) | `dockerfile` | +| `Makefile` (no extension) | `makefile` | +| `Jenkinsfile` (no extension) | `groovy` | Collect unique languages, sorted alphabetically. -**Step 4 -- Line Counting** +**Step 4 -- File Category Detection** -For each source file, count lines using `wc -l`. For efficiency: +Assign a `fileCategory` to each discovered file based on its extension and path: + +| Pattern | Category | +|---|---| +| `.md`, `.rst`, `.txt` (except `LICENSE`) | `docs` | +| `.yaml`, `.yml`, `.json`, `.toml`, `.xml`, `.cfg`, `.ini`, `.env`, `tsconfig.json`, `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod` | `config` | +| `Dockerfile`, `docker-compose.*`, `.tf`, `.tfvars`, `Makefile`, `Jenkinsfile`, `Procfile`, `Vagrantfile`, `.github/workflows/*`, `.gitlab-ci.yml`, `.circleci/*`, `*.k8s.yaml`, `*.k8s.yml`, paths in `k8s/` or `kubernetes/` | `infra` | +| `.sql`, `.graphql`, `.gql`, `.proto`, `.prisma`, `*.schema.json`, `.csv` | `data` | +| `.sh`, `.bash`, `.ps1`, `.bat` | `script` | +| `.html`, `.htm`, `.css`, `.scss`, `.sass`, `.less` | `markup` | +| All other extensions (`.ts`, `.tsx`, `.js`, `.py`, `.go`, `.rs`, etc.) | `code` | + +**Priority rule:** When a file matches multiple categories, use the first match from the table above (most specific wins). For example, `docker-compose.yml` is `infra`, not `config`. + +**Step 5 -- Line Counting** + +For each file, count lines using `wc -l`. For efficiency: - If fewer than 500 files, count all of them - If 500+ files, count all of them but batch the `wc -l` calls (pass multiple files per invocation to avoid spawning thousands of processes) -**Step 5 -- Framework Detection** +**Step 6 -- Framework Detection** Read config files (if they exist) and extract framework information: - `package.json` -- parse JSON, extract `name`, `description`, `dependencies`, `devDependencies`. Match dependency names against known frameworks: `react`, `vue`, `svelte`, `@angular/core`, `express`, `fastify`, `koa`, `next`, `nuxt`, `vite`, `vitest`, `jest`, `mocha`, `tailwindcss`, `prisma`, `typeorm`, `sequelize`, `mongoose`, `redux`, `zustand`, `mobx` @@ -89,15 +129,23 @@ Read config files (if they exist) and extract framework information: - `Cargo.toml` dependencies -- if present, read `[dependencies]` and match crate names against known Rust frameworks: `actix-web`, `axum`, `rocket`, `diesel`, `tokio`, `serde`, `warp` - `pom.xml` / `build.gradle` / `build.gradle.kts` -- if present, confirms Java/Kotlin project; match dependency names against known JVM frameworks: `spring-boot`, `spring-web`, `spring-data`, `quarkus`, `micronaut`, `hibernate`, `jakarta`, `junit`, `ktor` -**Step 6 -- Complexity Estimation** +Also detect infrastructure tooling from discovered files: +- Presence of `Dockerfile` → add `Docker` to frameworks +- Presence of `docker-compose.yml` or `docker-compose.yaml` → add `Docker Compose` to frameworks +- Presence of `*.tf` files → add `Terraform` to frameworks +- Presence of `.github/workflows/*.yml` → add `GitHub Actions` to frameworks +- Presence of `.gitlab-ci.yml` → add `GitLab CI` to frameworks +- Presence of `Jenkinsfile` → add `Jenkins` to frameworks -Classify by source file count: -- `small`: 1-20 files -- `moderate`: 21-100 files -- `large`: 101-500 files +**Step 7 -- Complexity Estimation** + +Classify by total file count (including non-code files): +- `small`: 1-30 files +- `moderate`: 31-150 files +- `large`: 151-500 files - `very-large`: >500 files -**Step 7 -- Project Name** +**Step 8 -- Project Name** Extract from (in priority order): 1. `package.json` `name` field @@ -106,11 +154,13 @@ Extract from (in priority order): 4. `pyproject.toml` -- check `[project].name` first, then `[tool.poetry].name` 5. Directory name of project root -**Step 8 -- Import Resolution** +**Step 9 -- Import Resolution** -For each file in the discovered source list, extract and resolve relative import statements. The goal is to produce a map from each file's path to the list of project-internal files it imports. External package imports are ignored. +For each **code-category** file in the discovered list (`fileCategory === "code"`), extract and resolve relative import statements. The goal is to produce a map from each file's path to the list of project-internal files it imports. External package imports are ignored. -For each file, read its content and extract import paths using language-appropriate patterns: +**Non-code files** (config, docs, infra, data, script, markup) should have an empty array `[]` in the import map — they do not participate in code-level import resolution. + +For each code file, read its content and extract import paths using language-appropriate patterns: | Language | Import patterns to match | |---|---| @@ -134,6 +184,8 @@ Output format in the script result: "importMap": { "src/index.ts": ["src/utils.ts", "src/config.ts"], "src/utils.ts": [], + "README.md": [], + "Dockerfile": [], "src/components/App.tsx": ["src/hooks/useAuth.ts", "src/store/index.ts"] } ``` @@ -150,16 +202,22 @@ The script must write this exact JSON structure to the output file: "name": "project-name", "rawDescription": "Description from package.json or empty string", "readmeHead": "First 10 lines of README.md or empty string", - "languages": ["javascript", "typescript"], - "frameworks": ["React", "Vite", "Vitest"], + "languages": ["javascript", "markdown", "typescript", "yaml"], + "frameworks": ["React", "Vite", "Vitest", "Docker"], "files": [ - {"path": "src/index.ts", "language": "typescript", "sizeLines": 150} + {"path": "src/index.ts", "language": "typescript", "sizeLines": 150, "fileCategory": "code"}, + {"path": "README.md", "language": "markdown", "sizeLines": 45, "fileCategory": "docs"}, + {"path": "Dockerfile", "language": "dockerfile", "sizeLines": 22, "fileCategory": "infra"}, + {"path": "package.json", "language": "json", "sizeLines": 35, "fileCategory": "config"} ], "totalFiles": 42, "estimatedComplexity": "moderate", "importMap": { "src/index.ts": ["src/utils.ts", "src/config.ts"], - "src/utils.ts": [] + "src/utils.ts": [], + "README.md": [], + "Dockerfile": [], + "package.json": [] } } ``` @@ -170,10 +228,11 @@ The script must write this exact JSON structure to the output file: - `readmeHead` (string) -- first 10 lines of `README.md` or empty string if no README exists - `languages` (string[]) -- deduplicated, sorted alphabetically - `frameworks` (string[]) -- only confirmed frameworks; empty array if none detected -- `files` (object[]) -- every source file, sorted by `path` alphabetically +- `files` (object[]) -- every discovered file, sorted by `path` alphabetically +- `files[].fileCategory` (string) -- one of: `code`, `config`, `docs`, `infra`, `data`, `script`, `markup` - `totalFiles` (integer) -- must equal `files.length` - `estimatedComplexity` (string) -- one of `small`, `moderate`, `large`, `very-large` -- `importMap` (object) — map from every source file path to its list of resolved project-internal import paths; empty array if no resolved imports; external packages excluded +- `importMap` (object) -- map from every file path to its list of resolved project-internal import paths; empty array for non-code files and files with no resolved imports; external packages excluded ### Executing the Script @@ -208,10 +267,12 @@ Then assemble the final output JSON: { "name": "project-name", "description": "Brief description from README or package.json", - "languages": ["typescript", "javascript"], - "frameworks": ["React", "Vite", "Vitest"], + "languages": ["markdown", "typescript", "yaml"], + "frameworks": ["React", "Vite", "Vitest", "Docker"], "files": [ - {"path": "src/index.ts", "language": "typescript", "sizeLines": 150} + {"path": "src/index.ts", "language": "typescript", "sizeLines": 150, "fileCategory": "code"}, + {"path": "README.md", "language": "markdown", "sizeLines": 45, "fileCategory": "docs"}, + {"path": "Dockerfile", "language": "dockerfile", "sizeLines": 22, "fileCategory": "infra"} ], "totalFiles": 42, "estimatedComplexity": "moderate", @@ -226,7 +287,7 @@ Then assemble the final output JSON: - `description` (string): your synthesized 1-2 sentence description - `languages` (string[]): directly from script output - `frameworks` (string[]): directly from script output -- `files` (object[]): directly from script output +- `files` (object[]): directly from script output, including `fileCategory` per file - `totalFiles` (integer): directly from script output - `estimatedComplexity` (string): directly from script output - `importMap` (object): directly from script output @@ -237,7 +298,8 @@ Then assemble the final output JSON: - NEVER include files that do not exist on disk. - ALWAYS validate that `totalFiles` matches the actual length of the `files` array. - ALWAYS sort `files` by `path` for deterministic output. -- Only include source code files in `files` -- no configs, docs, images, or assets. +- Include ALL discovered project files in `files` -- code, configs, docs, infrastructure, and data files. Only exclude binaries, lock files, generated files, and dependency directories. +- Every file MUST have a `fileCategory` field with one of: `code`, `config`, `docs`, `infra`, `data`, `script`, `markup`. - Trust the script's output for all structural data. Your only contribution is the `description` field. ## Writing Results @@ -246,6 +308,6 @@ After producing the final JSON: 1. Create the output directory: `mkdir -p /.understand-anything/intermediate` 2. Write the JSON to: `/.understand-anything/intermediate/scan-result.json` -3. Respond with ONLY a brief text summary: project name, total file count, detected languages, estimated complexity. +3. Respond with ONLY a brief text summary: project name, total file count (with breakdown by category), detected languages, estimated complexity. Do NOT include the full JSON in your text response.