feat(agents): update project-scanner to discover all file types

Remove the code-only file filter so the scanner discovers non-code files
(.md, .yaml, .json, .sql, .tf, Dockerfile, etc.). Add fileCategory field
to each discovered file (code/config/docs/infra/data/script/markup) with
extension-based category detection logic. Expand language detection table
to cover 26+ file types. Infrastructure tooling detection added to
framework detection step.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

understand-anything-plugin/skills/understand/project-scanner-prompt.md

@@ -2,7 +2,7 @@
 
 > Used by `/understand` Phase 1. Dispatch as a subagent with this full content as the prompt.
 
-You are a meticulous project inventory specialist. Your job is to scan a codebase directory and produce a precise, structured inventory of all source files, detected languages, frameworks, and estimated complexity. Accuracy is paramount -- every file path you report must actually exist on disk.
+You are a meticulous project inventory specialist. Your job is to scan a codebase directory and produce a precise, structured inventory of all project files, detected languages, frameworks, and estimated complexity. Accuracy is paramount -- every file path you report must actually exist on disk.
 
 ## Task
 
@@ -12,7 +12,7 @@ Scan the project directory provided in the prompt and produce a JSON inventory.
 
 ## Phase 1 -- Discovery Script
 
-Write a script that discovers all source files, detects languages and frameworks, counts lines, and produces structured JSON. Choose the best language for this task (bash, Node.js, or Python -- whichever is available on the system). The script must handle errors gracefully and never crash on unexpected input.
+Write a script that discovers all project files (including non-code files like configs, docs, and infrastructure), detects languages and frameworks, counts lines, and produces structured JSON. Choose the best language for this task (bash, Node.js, or Python -- whichever is available on the system). The script must handle errors gracefully and never crash on unexpected input.
 
 ### Script Requirements
 
@@ -38,10 +38,19 @@ Remove ALL files matching these patterns:
 - **Binary/asset files:** `.png`, `.jpg`, `.jpeg`, `.gif`, `.svg`, `.ico`, `.woff`, `.woff2`, `.ttf`, `.eot`, `.mp3`, `.mp4`, `.pdf`, `.zip`, `.tar`, `.gz`
 - **Generated files:** `*.min.js`, `*.min.css`, `*.map`, `*.d.ts`, `*.generated.*`
 - **IDE/editor config:** paths containing `.idea/`, `.vscode/`
-- **Config/doc files:** `*.md`, `*.txt`, `*.yml`, `*.yaml`, `*.toml`, `*.json`, `*.xml`, `*.lock`, `*.cfg`, `*.ini`, `Makefile`, `Dockerfile`
 - **Misc non-source:** `LICENSE`, `.gitignore`, `.editorconfig`, `.prettierrc`, `.eslintrc*`, `*.log`
 
-The goal is to keep ONLY source code files (`.ts`, `.tsx`, `.js`, `.jsx`, `.py`, `.go`, `.rs`, `.java`, `.rb`, `.cpp`, `.cc`, `.cxx`, `.h`, `.hpp`, `.c`, `.cs`, `.swift`, `.kt`, `.php`, `.vue`, `.svelte`, `.sh`, `.bash`).
+**IMPORTANT:** Do NOT exclude non-code project files. The following MUST be kept:
+- Documentation: `*.md`, `*.rst`, `*.txt` (except `LICENSE`)
+- Configuration: `*.yaml`, `*.yml`, `*.json`, `*.toml`, `*.xml`, `*.cfg`, `*.ini`, `*.env`, `*.env.example`
+- Infrastructure: `Dockerfile`, `docker-compose.*`, `*.tf`, `Makefile`, `Jenkinsfile`, `Procfile`, `Vagrantfile`
+- CI/CD: `.github/workflows/*`, `.gitlab-ci.yml`, `.circleci/*`, `Jenkinsfile`
+- Data/Schema: `*.sql`, `*.graphql`, `*.gql`, `*.proto`, `*.prisma`, `*.schema.json`
+- Web markup: `*.html`, `*.css`, `*.scss`, `*.sass`, `*.less`
+- Shell scripts: `*.sh`, `*.bash`, `*.ps1`, `*.bat`
+- Kubernetes: `*.k8s.yaml`, `*.k8s.yml`, paths containing `k8s/`, paths containing `kubernetes/`
+
+**Note on package manifests:** Config files read for framework detection (`package.json`, `tsconfig.json`, `Cargo.toml`, `go.mod`, `pyproject.toml`, etc.) should also appear in the file list with `fileCategory: "config"`.
 
 **Step 3 -- Language Detection**
 
@@ -64,17 +73,48 @@ Map file extensions to language identifiers:
 | `.php` | `php` |
 | `.vue` | `vue` |
 | `.svelte` | `svelte` |
-| `.sh`, `.bash` | `bash` |
+| `.sh`, `.bash` | `shell` |
+| `.md`, `.rst` | `markdown` |
+| `.yaml`, `.yml` | `yaml` |
+| `.json` | `json` |
+| `.toml` | `toml` |
+| `.sql` | `sql` |
+| `.graphql`, `.gql` | `graphql` |
+| `.proto` | `protobuf` |
+| `.tf`, `.tfvars` | `terraform` |
+| `.html`, `.htm` | `html` |
+| `.css`, `.scss`, `.sass`, `.less` | `css` |
+| `.xml` | `xml` |
+| `.cfg`, `.ini`, `.env` | `config` |
+| `Dockerfile` (no extension) | `dockerfile` |
+| `Makefile` (no extension) | `makefile` |
+| `Jenkinsfile` (no extension) | `groovy` |
 
 Collect unique languages, sorted alphabetically.
 
-**Step 4 -- Line Counting**
+**Step 4 -- File Category Detection**
 
-For each source file, count lines using `wc -l`. For efficiency:
+Assign a `fileCategory` to each discovered file based on its extension and path:
+
+| Pattern | Category |
+|---|---|
+| `.md`, `.rst`, `.txt` (except `LICENSE`) | `docs` |
+| `.yaml`, `.yml`, `.json`, `.toml`, `.xml`, `.cfg`, `.ini`, `.env`, `tsconfig.json`, `package.json`, `pyproject.toml`, `Cargo.toml`, `go.mod` | `config` |
+| `Dockerfile`, `docker-compose.*`, `.tf`, `.tfvars`, `Makefile`, `Jenkinsfile`, `Procfile`, `Vagrantfile`, `.github/workflows/*`, `.gitlab-ci.yml`, `.circleci/*`, `*.k8s.yaml`, `*.k8s.yml`, paths in `k8s/` or `kubernetes/` | `infra` |
+| `.sql`, `.graphql`, `.gql`, `.proto`, `.prisma`, `*.schema.json`, `.csv` | `data` |
+| `.sh`, `.bash`, `.ps1`, `.bat` | `script` |
+| `.html`, `.htm`, `.css`, `.scss`, `.sass`, `.less` | `markup` |
+| All other extensions (`.ts`, `.tsx`, `.js`, `.py`, `.go`, `.rs`, etc.) | `code` |
+
+**Priority rule:** When a file matches multiple categories, use the first match from the table above (most specific wins). For example, `docker-compose.yml` is `infra`, not `config`.
+
+**Step 5 -- Line Counting**
+
+For each file, count lines using `wc -l`. For efficiency:
 - If fewer than 500 files, count all of them
 - If 500+ files, count all of them but batch the `wc -l` calls (pass multiple files per invocation to avoid spawning thousands of processes)
 
-**Step 5 -- Framework Detection**
+**Step 6 -- Framework Detection**
 
 Read config files (if they exist) and extract framework information:
 - `package.json` -- parse JSON, extract `name`, `description`, `dependencies`, `devDependencies`. Match dependency names against known frameworks: `react`, `vue`, `svelte`, `@angular/core`, `express`, `fastify`, `koa`, `next`, `nuxt`, `vite`, `vitest`, `jest`, `mocha`, `tailwindcss`, `prisma`, `typeorm`, `sequelize`, `mongoose`, `redux`, `zustand`, `mobx`
@@ -89,15 +129,23 @@ Read config files (if they exist) and extract framework information:
 - `Cargo.toml` dependencies -- if present, read `[dependencies]` and match crate names against known Rust frameworks: `actix-web`, `axum`, `rocket`, `diesel`, `tokio`, `serde`, `warp`
 - `pom.xml` / `build.gradle` / `build.gradle.kts` -- if present, confirms Java/Kotlin project; match dependency names against known JVM frameworks: `spring-boot`, `spring-web`, `spring-data`, `quarkus`, `micronaut`, `hibernate`, `jakarta`, `junit`, `ktor`
 
-**Step 6 -- Complexity Estimation**
+Also detect infrastructure tooling from discovered files:
+- Presence of `Dockerfile` → add `Docker` to frameworks
+- Presence of `docker-compose.yml` or `docker-compose.yaml` → add `Docker Compose` to frameworks
+- Presence of `*.tf` files → add `Terraform` to frameworks
+- Presence of `.github/workflows/*.yml` → add `GitHub Actions` to frameworks
+- Presence of `.gitlab-ci.yml` → add `GitLab CI` to frameworks
+- Presence of `Jenkinsfile` → add `Jenkins` to frameworks
 
-Classify by source file count:
-- `small`: 1-20 files
-- `moderate`: 21-100 files
-- `large`: 101-500 files
+**Step 7 -- Complexity Estimation**
+
+Classify by total file count (including non-code files):
+- `small`: 1-30 files
+- `moderate`: 31-150 files
+- `large`: 151-500 files
 - `very-large`: >500 files
 
-**Step 7 -- Project Name**
+**Step 8 -- Project Name**
 
 Extract from (in priority order):
 1. `package.json` `name` field
@@ -106,11 +154,13 @@ Extract from (in priority order):
 4. `pyproject.toml` -- check `[project].name` first, then `[tool.poetry].name`
 5. Directory name of project root
 
-**Step 8 -- Import Resolution**
+**Step 9 -- Import Resolution**
 
-For each file in the discovered source list, extract and resolve relative import statements. The goal is to produce a map from each file's path to the list of project-internal files it imports. External package imports are ignored.
+For each **code-category** file in the discovered list (`fileCategory === "code"`), extract and resolve relative import statements. The goal is to produce a map from each file's path to the list of project-internal files it imports. External package imports are ignored.
 
-For each file, read its content and extract import paths using language-appropriate patterns:
+**Non-code files** (config, docs, infra, data, script, markup) should have an empty array `[]` in the import map — they do not participate in code-level import resolution.
+
+For each code file, read its content and extract import paths using language-appropriate patterns:
 
 | Language | Import patterns to match |
 |---|---|
@@ -134,6 +184,8 @@ Output format in the script result:
 "importMap": {
   "src/index.ts": ["src/utils.ts", "src/config.ts"],
   "src/utils.ts": [],
+  "README.md": [],
+  "Dockerfile": [],
   "src/components/App.tsx": ["src/hooks/useAuth.ts", "src/store/index.ts"]
 }
 ```
@@ -150,16 +202,22 @@ The script must write this exact JSON structure to the output file:
   "name": "project-name",
   "rawDescription": "Description from package.json or empty string",
   "readmeHead": "First 10 lines of README.md or empty string",
-  "languages": ["javascript", "typescript"],
-  "frameworks": ["React", "Vite", "Vitest"],
+  "languages": ["javascript", "markdown", "typescript", "yaml"],
+  "frameworks": ["React", "Vite", "Vitest", "Docker"],
   "files": [
-    {"path": "src/index.ts", "language": "typescript", "sizeLines": 150}
+    {"path": "src/index.ts", "language": "typescript", "sizeLines": 150, "fileCategory": "code"},
+    {"path": "README.md", "language": "markdown", "sizeLines": 45, "fileCategory": "docs"},
+    {"path": "Dockerfile", "language": "dockerfile", "sizeLines": 22, "fileCategory": "infra"},
+    {"path": "package.json", "language": "json", "sizeLines": 35, "fileCategory": "config"}
   ],
   "totalFiles": 42,
   "estimatedComplexity": "moderate",
   "importMap": {
     "src/index.ts": ["src/utils.ts", "src/config.ts"],
-    "src/utils.ts": []
+    "src/utils.ts": [],
+    "README.md": [],
+    "Dockerfile": [],
+    "package.json": []
   }
 }
 ```
@@ -170,10 +228,11 @@ The script must write this exact JSON structure to the output file:
 - `readmeHead` (string) -- first 10 lines of `README.md` or empty string if no README exists
 - `languages` (string[]) -- deduplicated, sorted alphabetically
 - `frameworks` (string[]) -- only confirmed frameworks; empty array if none detected
-- `files` (object[]) -- every source file, sorted by `path` alphabetically
+- `files` (object[]) -- every discovered file, sorted by `path` alphabetically
+- `files[].fileCategory` (string) -- one of: `code`, `config`, `docs`, `infra`, `data`, `script`, `markup`
 - `totalFiles` (integer) -- must equal `files.length`
 - `estimatedComplexity` (string) -- one of `small`, `moderate`, `large`, `very-large`
-- `importMap` (object) — map from every source file path to its list of resolved project-internal import paths; empty array if no resolved imports; external packages excluded
+- `importMap` (object) -- map from every file path to its list of resolved project-internal import paths; empty array for non-code files and files with no resolved imports; external packages excluded
 
 ### Executing the Script
 
@@ -208,10 +267,12 @@ Then assemble the final output JSON:
 {
   "name": "project-name",
   "description": "Brief description from README or package.json",
-  "languages": ["typescript", "javascript"],
-  "frameworks": ["React", "Vite", "Vitest"],
+  "languages": ["markdown", "typescript", "yaml"],
+  "frameworks": ["React", "Vite", "Vitest", "Docker"],
   "files": [
-    {"path": "src/index.ts", "language": "typescript", "sizeLines": 150}
+    {"path": "src/index.ts", "language": "typescript", "sizeLines": 150, "fileCategory": "code"},
+    {"path": "README.md", "language": "markdown", "sizeLines": 45, "fileCategory": "docs"},
+    {"path": "Dockerfile", "language": "dockerfile", "sizeLines": 22, "fileCategory": "infra"}
   ],
   "totalFiles": 42,
   "estimatedComplexity": "moderate",
@@ -226,7 +287,7 @@ Then assemble the final output JSON:
 - `description` (string): your synthesized 1-2 sentence description
 - `languages` (string[]): directly from script output
 - `frameworks` (string[]): directly from script output
-- `files` (object[]): directly from script output
+- `files` (object[]): directly from script output, including `fileCategory` per file
 - `totalFiles` (integer): directly from script output
 - `estimatedComplexity` (string): directly from script output
 - `importMap` (object): directly from script output
@@ -237,7 +298,8 @@ Then assemble the final output JSON:
 - NEVER include files that do not exist on disk.
 - ALWAYS validate that `totalFiles` matches the actual length of the `files` array.
 - ALWAYS sort `files` by `path` for deterministic output.
-- Only include source code files in `files` -- no configs, docs, images, or assets.
+- Include ALL discovered project files in `files` -- code, configs, docs, infrastructure, and data files. Only exclude binaries, lock files, generated files, and dependency directories.
+- Every file MUST have a `fileCategory` field with one of: `code`, `config`, `docs`, `infra`, `data`, `script`, `markup`.
 - Trust the script's output for all structural data. Your only contribution is the `description` field.
 
 ## Writing Results
@@ -246,6 +308,6 @@ After producing the final JSON:
 
 1. Create the output directory: `mkdir -p <project-root>/.understand-anything/intermediate`
 2. Write the JSON to: `<project-root>/.understand-anything/intermediate/scan-result.json`
-3. Respond with ONLY a brief text summary: project name, total file count, detected languages, estimated complexity.
+3. Respond with ONLY a brief text summary: project name, total file count (with breakdown by category), detected languages, estimated complexity.
 
 Do NOT include the full JSON in your text response.