mirror of
https://github.com/Egonex-AI/Understand-Anything.git
synced 2026-06-22 10:58:03 +08:00
feat(merge): swap-then-supplement tested_by linker (#113)
Strip-and-rederive (current PR behaviour) drops real coverage signal on
projects whose test layout doesn't match a naming convention. On the
Google microservices-demo the LLM had emitted 7 valid tested_by edges
(3 with inverted direction); the strip pass dropped them and the path-
convention rederive could only re-pair 4 of them. Net: 7 → 4 edges,
3 production files lost their tested signal.
Replace strip-and-rederive with two-pass swap-then-supplement:
Pass 1 — walk LLM tested_by edges. Canonical (production → test)
edges pass through unchanged. Inverted (test → production) edges are
flipped in place; description gets a `[direction corrected]` audit
marker. Edges with no recoverable meaning (test↔test, prod↔prod,
orphan endpoint, duplicate pair) are dropped.
Pass 2 — for tests not yet paired by Pass 1, walk path-convention
candidates and emit a fresh production → test edge for the first
match. Pairs already covered by Pass 1 are skipped.
Tagging is consolidated into a final pass over all canonical edges so
production nodes get the "tested" tag whether the edge came from
Pass 1 (canonical / swapped) or Pass 2 (supplement).
Multi-language audit of production_candidates revealed three real-world
gaps surfaced by re-checking microservices-demo and common project
layouts:
- JS/TS walk-out only handled `__tests__/`. Extended to also walk out
of `<dir>/test/`, `<dir>/spec/`, and `<dir>/tests/` (some JS/TS
projects use these instead of __tests__/).
- Python walk-out only handled top-level `tests/`. Added in-package
`<pkg>/tests/test_<name>.py` → `<pkg>/<name>.py` (Django app style
and any project that colocates tests with the package).
- C# only had sibling fallback. Added two new mirrors:
* `<svc>/tests/X.cs` ↔ `<svc>/X.cs` and `<svc>/src/.../X.cs`
(microservices-demo cartservice exact layout).
* `<App>.Tests/Foo/BarTests.cs` ↔ `<App>/Foo/Bar.cs`
(.NET sibling-project convention).
Go is intentionally not changed — the "one _test.go covers several
.go files in the same package" pattern is now solved by Pass 1
(swapping LLM edges), not by trying to invent multi-pair path heuristics.
The file-analyzer prompt is updated: the `tested_by` row is restored
in the schema table because we now use those edges as evidence (Pass 1
canonicalizes the direction). The note explains direction will be
auto-corrected so the LLM doesn't need to be defensive about it.
link_tests now returns a 4-tuple (added, dropped, tagged, swapped);
the merge_and_normalize report distinguishes "edges produced
(supplement)" from "edges flipped" from "edges dropped".
Real-world validation on microservices-demo:
before: 7 tested_by edges, 3 inverted, 0 tagged
after PR: 4 tested_by edges, 0 inverted, 4 tagged ← strip-and-rederive
this: 7 tested_by edges, 0 inverted, 7 tagged ← swap-then-supplement
Tests: 47 pass (was 37). New cases cover all swap branches, the
shippingservice "one test, many sources" regression, and each new
language pattern.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -213,8 +213,9 @@ Using the script's structural data and file categories, create edges:
|
||||
| `implements` | A class implements an interface in the project | `0.9` | `forward` |
|
||||
| `exports` | File exports a function or class node you created (only for exported items — use IN ADDITION to `contains`, not instead of it) | `0.8` | `forward` |
|
||||
| `depends_on` | File has runtime dependency on another project file (broader than imports -- includes dynamic requires, lazy loads) | `0.6` | `forward` |
|
||||
| `tested_by` | Production file is exercised by a test file. Emit when you see the test importing/using the production file. Use direction `production → test` if you can; the merge script will flip inverted edges and dedupe. | `0.5` | `forward` |
|
||||
|
||||
**Note:** Do NOT emit `tested_by` edges. They are produced deterministically by the merge script (`merge-batch-graphs.py`) based on path conventions, in canonical `production → test` direction. Any `tested_by` edges you emit will be discarded.
|
||||
**Note on `tested_by`:** It's fine to emit even if you're unsure of the direction (you typically see the relationship while analyzing the *test* file, where the import points back at production). The merge script (`merge-batch-graphs.py`) canonicalizes direction to `production → test` and drops semantically broken edges (test↔test, prod↔prod, orphan endpoint). Path-convention pairing supplements anything you miss.
|
||||
|
||||
#### Edges for non-code files:
|
||||
|
||||
|
||||
@@ -278,7 +278,7 @@ This script reads all `batch-*.json` files from `$PROJECT_ROOT/.understand-anyth
|
||||
- Drops dangling edges referencing missing nodes
|
||||
- Logs all corrections and dropped items to stderr
|
||||
|
||||
The merge script also runs a deterministic `tested_by` linker that pairs production files with their tests by path convention (e.g. `X.ts` ↔ `X.test.ts`, `__tests__/`, mirrored `tests/` tree, Maven/Gradle `src/test/...` ↔ `src/main/...`). Production nodes that have a paired test get a `"tested"` tag. This produces canonical `production → test` direction for all `tested_by` edges; any LLM-emitted ones are dropped.
|
||||
The merge script also runs a `tested_by` linker that canonicalizes test-coverage edges in two passes. **Pass 1** walks LLM-emitted `tested_by` edges and flips inverted ones in place (the LLM systematically emits `test → production` because it sees the import only when analyzing the test file); semantically broken edges (test↔test, prod↔prod, orphan endpoints) are dropped. **Pass 2** supplements with path-convention pairings (`X.ts` ↔ `X.test.ts`, JS/TS `__tests__/` and `<dir>/test/` walk-out, Python in-package `tests/`, Go `_test.go` sibling, Maven/Gradle `src/test/...` ↔ `src/main/...`, .NET `<svc>/tests/` ↔ `<svc>/src/...` and `<App>.Tests/` ↔ `<App>/`). Production nodes that end up sourcing any `tested_by` edge get a `"tested"` tag. All resulting edges run `production → test`.
|
||||
|
||||
Output: `$PROJECT_ROOT/.understand-anything/intermediate/assembled-graph.json`
|
||||
|
||||
|
||||
@@ -221,11 +221,29 @@ def normalize_complexity(value: Any) -> tuple[str, str]:
|
||||
|
||||
# ── Deterministic tested_by linker ────────────────────────────────────────
|
||||
#
|
||||
# `tested_by` edges are produced here, not by the LLM. The LLM sees the
|
||||
# relationship only when analyzing a *test* file (production files don't
|
||||
# import their tests), so its emitted direction is unreliable across
|
||||
# batches. We strip every LLM-emitted `tested_by` edge and produce canonical
|
||||
# `production → test` edges from path conventions instead.
|
||||
# Two-pass linker. Both passes produce canonical `production → test` edges.
|
||||
#
|
||||
# Pass 1 — preserve LLM semantics, fix direction.
|
||||
# The LLM sees the relationship only when analyzing a *test* file
|
||||
# (production files don't import their tests), so its emitted direction
|
||||
# is systematically wrong: source = the file it was analyzing = a test.
|
||||
# We do NOT strip these edges — the *pairing* is real evidence (the LLM
|
||||
# saw an import / using / same-package call). We just flip direction
|
||||
# when source is test + target is production. Edges that are
|
||||
# semantically broken (test↔test, production↔production, orphan endpoints)
|
||||
# are dropped.
|
||||
#
|
||||
# Pass 2 — supplement with path-convention pairings.
|
||||
# For test files the LLM didn't link to anything, fall back to filename
|
||||
# conventions (sibling `_test.go`, JS/TS `__tests__/`, Maven `src/test/`,
|
||||
# etc.) to find a production counterpart. Pairs already covered by
|
||||
# Pass 1 are skipped.
|
||||
#
|
||||
# Why this beats strip-and-rederive: real projects often violate the
|
||||
# linker's naming conventions (one Go `_test.go` covering several `.go`
|
||||
# files in the same package, .NET `<svc>/tests/X.cs` against
|
||||
# `<svc>/src/Y/X.cs`). Stripping LLM edges drops that real-world coverage
|
||||
# signal entirely. Swapping preserves it.
|
||||
|
||||
def _path_segments(path: str) -> list[str]:
|
||||
"""Split a relative POSIX-style path into segments (ignoring empties)."""
|
||||
@@ -314,8 +332,11 @@ def production_candidates(test_path: str) -> list[str]:
|
||||
for c in _js_ts_sibling_candidates(dir_path, base_stem):
|
||||
_add_unique(candidates, c)
|
||||
|
||||
# 2. Walk out of __tests__/ — drop the trailing __tests__ segment.
|
||||
if dir_segs and dir_segs[-1] == "__tests__":
|
||||
# 2. Walk out of test-segregating subdir — drop the trailing
|
||||
# __tests__/test/spec/tests segment. Some JS/TS projects use
|
||||
# `<dir>/test/foo.spec.ts` or `<dir>/spec/foo.spec.ts` instead of
|
||||
# the more idiomatic `__tests__/`; treat them the same.
|
||||
if dir_segs and dir_segs[-1] in ("__tests__", "test", "spec", "tests"):
|
||||
parent_dir = "/".join(dir_segs[:-1])
|
||||
_add_unique(candidates, _join(parent_dir, f"{base_stem}{ext}"))
|
||||
for c in _js_ts_sibling_candidates(parent_dir, base_stem):
|
||||
@@ -346,6 +367,13 @@ def production_candidates(test_path: str) -> list[str]:
|
||||
# Sibling
|
||||
_add_unique(candidates, _join(dir_path, f"{base_stem}.py"))
|
||||
|
||||
# Walk out of an in-package tests/ or test/ directory:
|
||||
# `mypkg/tests/test_bar.py` → `mypkg/bar.py`. Common in Django apps
|
||||
# and any project that colocates tests with the package they cover.
|
||||
if dir_segs and dir_segs[-1] in ("tests", "test"):
|
||||
parent_dir = "/".join(dir_segs[:-1])
|
||||
_add_unique(candidates, _join(parent_dir, f"{base_stem}.py"))
|
||||
|
||||
# Mirrored: tests/foo/test_bar.py → src/foo/bar.py (and variants)
|
||||
if dir_segs and dir_segs[0] in ("tests", "test"):
|
||||
tail_path = "/".join(dir_segs[1:])
|
||||
@@ -392,7 +420,46 @@ def production_candidates(test_path: str) -> list[str]:
|
||||
for suffix in ("Tests", "Test"):
|
||||
if stem.endswith(suffix):
|
||||
base_stem = stem[: -len(suffix)]
|
||||
# Sibling fallback (e.g. `Foo.Tests/BarTests.cs` ↔ same dir
|
||||
# is rare but cheap to try).
|
||||
_add_unique(candidates, _join(dir_path, f"{base_stem}.cs"))
|
||||
|
||||
# Walk out of an in-service `tests/` directory and search
|
||||
# the sibling `src/` subtree. Handles layouts like
|
||||
# `src/<svc>/tests/BarTests.cs` ↔ `src/<svc>/src/.../Bar.cs`
|
||||
# (microservices-demo cartservice) and bare
|
||||
# `<proj>/tests/BarTests.cs` ↔ `<proj>/src/Bar.cs`.
|
||||
tests_idx = None
|
||||
for i in range(len(dir_segs) - 1, -1, -1):
|
||||
if dir_segs[i].lower() in ("tests", "test"):
|
||||
tests_idx = i
|
||||
break
|
||||
if tests_idx is not None:
|
||||
parent_segs = dir_segs[:tests_idx]
|
||||
tail_segs = dir_segs[tests_idx + 1 :]
|
||||
parent_dir = "/".join(parent_segs)
|
||||
# `<parent>/<base_stem>.cs` (drop `tests/` entirely).
|
||||
_add_unique(
|
||||
candidates,
|
||||
_join(parent_dir, f"{base_stem}.cs"),
|
||||
)
|
||||
# `<parent>/src/<tail>/<base_stem>.cs` (mirror through src/).
|
||||
src_dir = "/".join([*parent_segs, "src", *tail_segs])
|
||||
_add_unique(candidates, _join(src_dir, f"{base_stem}.cs"))
|
||||
|
||||
# `.NET`-style sibling-project mirror: `My.App.Tests/...` ↔
|
||||
# `My.App/...`. The test project's top dir typically ends in
|
||||
# `.Tests`. Strip it and try the same tail under the sibling.
|
||||
if dir_segs:
|
||||
top = dir_segs[0]
|
||||
if top.endswith(".Tests") or top.endswith(".Test"):
|
||||
sibling = top[: -len(".Tests")] if top.endswith(".Tests") else top[: -len(".Test")]
|
||||
if sibling:
|
||||
mirror_dir = "/".join([sibling, *dir_segs[1:]])
|
||||
_add_unique(
|
||||
candidates,
|
||||
_join(mirror_dir, f"{base_stem}.cs"),
|
||||
)
|
||||
break
|
||||
|
||||
# ── C/C++ ─────────────────────────────────────────────────────────
|
||||
@@ -420,35 +487,63 @@ def _file_node_path(node: dict[str, Any]) -> str | None:
|
||||
return nid[len("file:"):]
|
||||
|
||||
|
||||
def _ensure_tested_tag(node: dict[str, Any]) -> bool:
|
||||
"""Append "tested" to `node["tags"]`, coercing malformed `tags` to a
|
||||
fresh list. Returns True if the tag was newly added.
|
||||
|
||||
`tags` from raw LLM batch JSON may be missing, None, a string, or
|
||||
another non-list value — the TypeScript autoFixGraph normalizer that
|
||||
handles this runs downstream of this script, so we defend here.
|
||||
"""
|
||||
tags = node.get("tags")
|
||||
if not isinstance(tags, list):
|
||||
tags = []
|
||||
node["tags"] = tags
|
||||
if "tested" in tags:
|
||||
return False
|
||||
tags.append("tested")
|
||||
return True
|
||||
|
||||
|
||||
def link_tests(
|
||||
nodes_by_id: dict[str, dict[str, Any]],
|
||||
edges: list[dict[str, Any]],
|
||||
) -> tuple[int, int, int]:
|
||||
"""Strip LLM-emitted `tested_by` edges, then link production files to
|
||||
their tests deterministically.
|
||||
) -> tuple[int, int, int, int]:
|
||||
"""Canonicalize `tested_by` edges and link unmatched test files.
|
||||
|
||||
Mutates node values via `nodes_by_id` (adds `tested` tag) and `edges`
|
||||
(drops LLM `tested_by`, appends deterministic ones).
|
||||
Two passes (see module-level "Deterministic tested_by linker" comment
|
||||
for the rationale):
|
||||
|
||||
Returns (added, dropped, tagged):
|
||||
added: number of deterministic tested_by edges appended
|
||||
dropped: number of pre-existing tested_by edges removed
|
||||
tagged: number of production nodes newly tagged "tested"
|
||||
1. Walk every existing `tested_by` edge. Keep canonical
|
||||
(production → test) edges as-is. Flip inverted (test → production)
|
||||
edges so the swap preserves the LLM's pairing evidence with the
|
||||
right direction. Drop edges that don't classify cleanly as
|
||||
file ↔ file or where one endpoint is missing — they have no
|
||||
recoverable meaning.
|
||||
2. For every test file not yet paired by Pass 1, walk path-convention
|
||||
candidates and emit a fresh `production → test` edge for the first
|
||||
match.
|
||||
|
||||
Tagging happens once per production node that ends up on the source
|
||||
side of any `tested_by` edge (canonical, swapped, or supplemented).
|
||||
|
||||
Mutates `nodes_by_id` (adds "tested" tag) and `edges` (rewrites
|
||||
in place: drops semantically broken edges, swaps inverted ones, appends
|
||||
supplements).
|
||||
|
||||
Returns (added, dropped, tagged, swapped):
|
||||
added: path-convention supplemental edges appended in Pass 2
|
||||
dropped: pre-existing `tested_by` edges removed (unsalvageable)
|
||||
tagged: production nodes newly tagged "tested"
|
||||
swapped: pre-existing `tested_by` edges flipped (test → production
|
||||
became production → test)
|
||||
"""
|
||||
# 1. Strip every existing tested_by edge — the LLM's direction is
|
||||
# unreliable, so we discard and replace.
|
||||
dropped = 0
|
||||
write_idx = 0
|
||||
for edge in edges:
|
||||
if edge.get("type") == "tested_by":
|
||||
dropped += 1
|
||||
continue
|
||||
edges[write_idx] = edge
|
||||
write_idx += 1
|
||||
del edges[write_idx:]
|
||||
|
||||
# 2. Index file nodes by relative path; classify each as test or production.
|
||||
# ── Index file nodes by relative path; classify each as test/production.
|
||||
# `is_prod` here means "is a known file node AND is not a test by
|
||||
# path convention" — used both to validate edge endpoints and to drive
|
||||
# path-convention candidate matching.
|
||||
file_paths_to_nodes: dict[str, dict[str, Any]] = {}
|
||||
node_id_to_classification: dict[str, str] = {} # id → "test" | "prod"
|
||||
test_nodes: list[tuple[str, dict[str, Any]]] = []
|
||||
for node in nodes_by_id.values():
|
||||
path = _file_node_path(node)
|
||||
@@ -456,13 +551,71 @@ def link_tests(
|
||||
continue
|
||||
file_paths_to_nodes[path] = node
|
||||
if is_test_path(path):
|
||||
node_id_to_classification[node["id"]] = "test"
|
||||
test_nodes.append((path, node))
|
||||
else:
|
||||
node_id_to_classification[node["id"]] = "prod"
|
||||
|
||||
# 3. For each test, walk its candidate production paths and take the
|
||||
# first one that exists AND is itself classified as production.
|
||||
# ── Pass 1: walk existing tested_by edges, canonicalize or drop.
|
||||
# `covered` tracks (production_id, test_id) pairs already represented
|
||||
# by a tested_by edge after this pass — used to suppress duplicate
|
||||
# supplements in Pass 2 and to deduplicate within Pass 1 itself.
|
||||
covered: set[tuple[str, str]] = set()
|
||||
swapped = 0
|
||||
dropped = 0
|
||||
write_idx = 0
|
||||
for edge in edges:
|
||||
if edge.get("type") != "tested_by":
|
||||
edges[write_idx] = edge
|
||||
write_idx += 1
|
||||
continue
|
||||
|
||||
src = edge.get("source", "")
|
||||
tgt = edge.get("target", "")
|
||||
src_class = node_id_to_classification.get(src)
|
||||
tgt_class = node_id_to_classification.get(tgt)
|
||||
|
||||
# Both endpoints must be known file nodes; one test, one production.
|
||||
# Anything else (orphan, test↔test, prod↔prod, non-file endpoint)
|
||||
# has no recoverable meaning — drop it.
|
||||
if (src_class, tgt_class) == ("prod", "test"):
|
||||
pair = (src, tgt)
|
||||
if pair in covered:
|
||||
# Duplicate canonical edge — drop the dup, keep the first.
|
||||
dropped += 1
|
||||
continue
|
||||
covered.add(pair)
|
||||
edges[write_idx] = edge
|
||||
write_idx += 1
|
||||
elif (src_class, tgt_class) == ("test", "prod"):
|
||||
pair = (tgt, src)
|
||||
if pair in covered:
|
||||
dropped += 1
|
||||
continue
|
||||
covered.add(pair)
|
||||
# Flip in place; mark provenance so reviewers can audit.
|
||||
edge["source"] = tgt
|
||||
edge["target"] = src
|
||||
edge["direction"] = "forward"
|
||||
prev = edge.get("description")
|
||||
edge["description"] = (
|
||||
"Direction corrected (was test → production)"
|
||||
if not prev
|
||||
else f"{prev} [direction corrected]"
|
||||
)
|
||||
swapped += 1
|
||||
edges[write_idx] = edge
|
||||
write_idx += 1
|
||||
else:
|
||||
dropped += 1
|
||||
del edges[write_idx:]
|
||||
|
||||
# ── Pass 2: path-convention supplement for tests not yet paired.
|
||||
paired_test_ids = {test_id for (_prod_id, test_id) in covered}
|
||||
added = 0
|
||||
tagged = 0
|
||||
for test_path, test_node in test_nodes:
|
||||
if test_node["id"] in paired_test_ids:
|
||||
continue
|
||||
for cand_path in production_candidates(test_path):
|
||||
prod_node = file_paths_to_nodes.get(cand_path)
|
||||
if prod_node is None:
|
||||
@@ -470,6 +623,9 @@ def link_tests(
|
||||
if is_test_path(cand_path):
|
||||
# Don't link a test to another test even if naming aligns.
|
||||
continue
|
||||
pair = (prod_node["id"], test_node["id"])
|
||||
if pair in covered:
|
||||
continue
|
||||
edges.append({
|
||||
"source": prod_node["id"],
|
||||
"target": test_node["id"],
|
||||
@@ -478,21 +634,21 @@ def link_tests(
|
||||
"weight": 0.5,
|
||||
"description": "Path-based pairing (deterministic)",
|
||||
})
|
||||
covered.add(pair)
|
||||
added += 1
|
||||
# `tags` may be missing, None, or a malformed string from raw LLM
|
||||
# batch JSON — the TypeScript autoFixGraph layer that normalizes it
|
||||
# only runs downstream of this script. Coerce to a fresh list so a
|
||||
# bad batch can't crash the whole merge.
|
||||
tags = prod_node.get("tags")
|
||||
if not isinstance(tags, list):
|
||||
tags = []
|
||||
prod_node["tags"] = tags
|
||||
if "tested" not in tags:
|
||||
tags.append("tested")
|
||||
tagged += 1
|
||||
break
|
||||
|
||||
return added, dropped, tagged
|
||||
# ── Tag every production node that ended up sourcing a tested_by edge
|
||||
# (covers Pass 1 canonical + swapped + Pass 2 supplements in one place).
|
||||
tagged = 0
|
||||
for prod_id, _test_id in covered:
|
||||
prod_node = nodes_by_id.get(prod_id)
|
||||
if prod_node is None:
|
||||
continue
|
||||
if _ensure_tested_tag(prod_node):
|
||||
tagged += 1
|
||||
|
||||
return added, dropped, tagged, swapped
|
||||
|
||||
|
||||
# ── Main merge + normalize ────────────────────────────────────────────────
|
||||
@@ -580,7 +736,7 @@ def merge_and_normalize(batches: list[dict[str, Any]]) -> tuple[dict[str, Any],
|
||||
|
||||
# ── Step 5b: Deterministic tested_by linker ──────────────────────
|
||||
# See module-level "Deterministic tested_by linker" section above.
|
||||
tested_by_added, tested_by_dropped, tested_by_tagged = link_tests(
|
||||
tested_by_added, tested_by_dropped, tested_by_tagged, tested_by_swapped = link_tests(
|
||||
nodes_by_id, all_edges
|
||||
)
|
||||
|
||||
@@ -622,12 +778,22 @@ def merge_and_normalize(batches: list[dict[str, Any]]) -> tuple[dict[str, Any],
|
||||
fixed_lines.append(f" {edges_rewritten:>4} × edge references rewritten after ID normalization")
|
||||
if duplicate_count:
|
||||
fixed_lines.append(f" {duplicate_count:>4} × duplicate node IDs removed (kept last)")
|
||||
if tested_by_swapped:
|
||||
fixed_lines.append(f" {tested_by_swapped:>4} × tested_by edges flipped (test → production became production → test)")
|
||||
if tested_by_dropped:
|
||||
fixed_lines.append(f" {tested_by_dropped:>4} × LLM-emitted tested_by edges dropped (direction unreliable)")
|
||||
fixed_lines.append(f" {tested_by_dropped:>4} × tested_by edges dropped (orphan endpoint or test↔test / prod↔prod pair)")
|
||||
|
||||
if fixed_lines:
|
||||
report.append("")
|
||||
report.append(f"Fixed ({sum(id_fix_patterns.values()) + sum(complexity_fix_patterns.values()) + edges_rewritten + duplicate_count + tested_by_dropped} corrections):")
|
||||
total_fixes = (
|
||||
sum(id_fix_patterns.values())
|
||||
+ sum(complexity_fix_patterns.values())
|
||||
+ edges_rewritten
|
||||
+ duplicate_count
|
||||
+ tested_by_swapped
|
||||
+ tested_by_dropped
|
||||
)
|
||||
report.append(f"Fixed ({total_fixes} corrections):")
|
||||
report.extend(fixed_lines)
|
||||
|
||||
# Tested-by linker section — separate from Fixed since these are net-new
|
||||
@@ -635,7 +801,7 @@ def merge_and_normalize(batches: list[dict[str, Any]]) -> tuple[dict[str, Any],
|
||||
if tested_by_added or tested_by_tagged:
|
||||
report.append("")
|
||||
report.append("Tested-by linker:")
|
||||
report.append(f" {tested_by_added:>4} × tested_by edges produced (production → test)")
|
||||
report.append(f" {tested_by_added:>4} × tested_by edges produced (path-convention supplement, production → test)")
|
||||
report.append(f" {tested_by_tagged:>4} × production nodes tagged \"tested\"")
|
||||
|
||||
# Could not fix section — unknown patterns (grouped) + individual details
|
||||
|
||||
@@ -199,6 +199,49 @@ class ProductionCandidatesTests(unittest.TestCase):
|
||||
cands = mbg.production_candidates("src/test/kotlin/com/foo/BarTest.kt")
|
||||
self.assertIn("src/main/kotlin/com/foo/Bar.kt", cands)
|
||||
|
||||
def test_js_ts_test_subdir_walkout(self) -> None:
|
||||
# Some JS/TS projects use `<dir>/test/` or `<dir>/spec/` instead of
|
||||
# the more idiomatic `__tests__/`. Walk out of either.
|
||||
cands_test = mbg.production_candidates("src/foo/test/X.test.ts")
|
||||
self.assertIn("src/foo/X.ts", cands_test)
|
||||
cands_spec = mbg.production_candidates("src/foo/spec/X.spec.ts")
|
||||
self.assertIn("src/foo/X.ts", cands_spec)
|
||||
|
||||
def test_python_in_package_tests_walkout(self) -> None:
|
||||
# `mypkg/tests/test_bar.py` (Django-app style) should pair with
|
||||
# `mypkg/bar.py` — walk out of the in-package tests/ dir.
|
||||
cands = mbg.production_candidates("mypkg/tests/test_bar.py")
|
||||
self.assertIn("mypkg/bar.py", cands)
|
||||
# Also nested:
|
||||
cands_nested = mbg.production_candidates("a/b/test/test_bar.py")
|
||||
self.assertIn("a/b/bar.py", cands_nested)
|
||||
|
||||
def test_csharp_tests_subdir_mirror_to_src(self) -> None:
|
||||
# Real case from microservices-demo cartservice:
|
||||
# `src/cartservice/tests/CartServiceTests.cs` ↔
|
||||
# `src/cartservice/src/services/CartService.cs`. The candidate list
|
||||
# only knows the basename; the matcher must produce a parent-level
|
||||
# candidate that the linker can verify against the actual file index.
|
||||
cands = mbg.production_candidates(
|
||||
"src/cartservice/tests/CartServiceTests.cs"
|
||||
)
|
||||
# Drop tests/ entirely:
|
||||
self.assertIn("src/cartservice/CartService.cs", cands)
|
||||
# Mirror through `src/`:
|
||||
self.assertIn("src/cartservice/src/CartService.cs", cands)
|
||||
# Sibling fallback retained:
|
||||
self.assertIn("src/cartservice/tests/CartService.cs", cands)
|
||||
|
||||
def test_csharp_dotnet_sibling_project_mirror(self) -> None:
|
||||
# `.NET` convention: `MyApp.Tests/Foo/BarTests.cs` ↔
|
||||
# `MyApp/Foo/Bar.cs`. Strip the `.Tests` suffix from the top dir
|
||||
# and try the same tail under the sibling project.
|
||||
cands = mbg.production_candidates("MyApp.Tests/Foo/BarTests.cs")
|
||||
self.assertIn("MyApp/Foo/Bar.cs", cands)
|
||||
# Also `.Test` (singular) is sometimes used.
|
||||
cands_singular = mbg.production_candidates("MyApp.Test/BarTest.cs")
|
||||
self.assertIn("MyApp/Bar.cs", cands_singular)
|
||||
|
||||
def test_priority_underscore_tests_sibling_before_walkup(self) -> None:
|
||||
# When a test sits in `src/__tests__/`, the sibling-de-infix path
|
||||
# (same directory) ranks before the walk-out path (parent directory).
|
||||
@@ -232,11 +275,12 @@ class LinkTestsTests(unittest.TestCase):
|
||||
}
|
||||
edges: list[dict[str, Any]] = []
|
||||
|
||||
added, dropped, tagged = mbg.link_tests(nodes_by_id, edges)
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(added, 1)
|
||||
self.assertEqual(dropped, 0)
|
||||
self.assertEqual(tagged, 1)
|
||||
self.assertEqual(swapped, 0)
|
||||
self.assertEqual(len(edges), 1)
|
||||
edge = edges[0]
|
||||
self.assertEqual(edge["source"], "file:src/foo.ts")
|
||||
@@ -254,18 +298,21 @@ class LinkTestsTests(unittest.TestCase):
|
||||
}
|
||||
edges: list[dict[str, Any]] = []
|
||||
|
||||
added, dropped, tagged = mbg.link_tests(nodes_by_id, edges)
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(added, 0)
|
||||
self.assertEqual(tagged, 0)
|
||||
self.assertEqual(swapped, 0)
|
||||
self.assertEqual(len(edges), 0)
|
||||
|
||||
def test_strips_existing_llm_tested_by_edges(self) -> None:
|
||||
def test_inverted_llm_edge_is_swapped_not_stripped(self) -> None:
|
||||
# The LLM systematically emits tested_by edges as test → production
|
||||
# (it sees the import only when analyzing the test file). The pairing
|
||||
# is real evidence; we keep it and flip the direction in place.
|
||||
nodes_by_id = {
|
||||
"file:src/foo.ts": _file_node("src/foo.ts"),
|
||||
"file:src/foo.test.ts": _file_node("src/foo.test.ts"),
|
||||
}
|
||||
# Existing inverted LLM edge: test → production (wrong direction)
|
||||
edges: list[dict[str, Any]] = [
|
||||
{
|
||||
"source": "file:src/foo.test.ts",
|
||||
@@ -277,24 +324,212 @@ class LinkTestsTests(unittest.TestCase):
|
||||
},
|
||||
]
|
||||
|
||||
added, dropped, tagged = mbg.link_tests(nodes_by_id, edges)
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(added, 1)
|
||||
self.assertEqual(dropped, 1)
|
||||
# No supplement needed (the LLM edge already covers this pair).
|
||||
self.assertEqual(added, 0)
|
||||
self.assertEqual(swapped, 1)
|
||||
self.assertEqual(dropped, 0)
|
||||
self.assertEqual(tagged, 1)
|
||||
|
||||
tested_by_edges = [e for e in edges if e["type"] == "tested_by"]
|
||||
self.assertEqual(len(tested_by_edges), 1)
|
||||
self.assertEqual(tested_by_edges[0]["source"], "file:src/foo.ts")
|
||||
self.assertEqual(tested_by_edges[0]["target"], "file:src/foo.test.ts")
|
||||
edge = tested_by_edges[0]
|
||||
self.assertEqual(edge["source"], "file:src/foo.ts")
|
||||
self.assertEqual(edge["target"], "file:src/foo.test.ts")
|
||||
# Provenance recorded so reviewers can audit the swap.
|
||||
self.assertIn("direction corrected", edge["description"].lower())
|
||||
|
||||
def test_unrelated_edges_survive_strip(self) -> None:
|
||||
def test_canonical_llm_edge_kept_unchanged(self) -> None:
|
||||
# An LLM edge already in canonical direction should pass through
|
||||
# untouched (no swap, no drop), and Pass 2 must not produce a
|
||||
# duplicate.
|
||||
nodes_by_id = {
|
||||
"file:src/foo.ts": _file_node("src/foo.ts"),
|
||||
"file:src/foo.test.ts": _file_node("src/foo.test.ts"),
|
||||
}
|
||||
edges: list[dict[str, Any]] = [
|
||||
{
|
||||
"source": "file:src/foo.ts",
|
||||
"target": "file:src/foo.test.ts",
|
||||
"type": "tested_by",
|
||||
"direction": "forward",
|
||||
"weight": 0.5,
|
||||
"description": "original",
|
||||
},
|
||||
]
|
||||
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual((added, dropped, swapped), (0, 0, 0))
|
||||
self.assertEqual(tagged, 1)
|
||||
tested_by_edges = [e for e in edges if e["type"] == "tested_by"]
|
||||
self.assertEqual(len(tested_by_edges), 1)
|
||||
self.assertEqual(tested_by_edges[0]["description"], "original")
|
||||
|
||||
def test_drops_test_to_test_edge(self) -> None:
|
||||
# An LLM edge between two test files has no recoverable meaning.
|
||||
nodes_by_id = {
|
||||
"file:src/foo.test.ts": _file_node("src/foo.test.ts"),
|
||||
"file:src/bar.test.ts": _file_node("src/bar.test.ts"),
|
||||
}
|
||||
edges: list[dict[str, Any]] = [
|
||||
{
|
||||
"source": "file:src/foo.test.ts",
|
||||
"target": "file:src/bar.test.ts",
|
||||
"type": "tested_by",
|
||||
"direction": "forward",
|
||||
"weight": 0.5,
|
||||
},
|
||||
]
|
||||
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(added, 0)
|
||||
self.assertEqual(swapped, 0)
|
||||
self.assertEqual(dropped, 1)
|
||||
self.assertEqual(tagged, 0)
|
||||
tested_by_edges = [e for e in edges if e["type"] == "tested_by"]
|
||||
self.assertEqual(tested_by_edges, [])
|
||||
|
||||
def test_drops_orphan_endpoint_edge(self) -> None:
|
||||
# Endpoint references a node that doesn't exist in nodes_by_id —
|
||||
# nothing to canonicalize against, drop it.
|
||||
nodes_by_id = {
|
||||
"file:src/foo.ts": _file_node("src/foo.ts"),
|
||||
}
|
||||
edges: list[dict[str, Any]] = [
|
||||
{
|
||||
"source": "file:src/foo.ts",
|
||||
"target": "file:src/missing.test.ts",
|
||||
"type": "tested_by",
|
||||
"direction": "forward",
|
||||
"weight": 0.5,
|
||||
},
|
||||
]
|
||||
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual((added, dropped, tagged, swapped), (0, 1, 0, 0))
|
||||
self.assertEqual([e for e in edges if e["type"] == "tested_by"], [])
|
||||
|
||||
def test_drops_duplicate_canonical_edges(self) -> None:
|
||||
# Two LLM edges describing the same (production, test) pair — keep
|
||||
# one, drop the other.
|
||||
nodes_by_id = {
|
||||
"file:src/foo.ts": _file_node("src/foo.ts"),
|
||||
"file:src/foo.test.ts": _file_node("src/foo.test.ts"),
|
||||
}
|
||||
edges: list[dict[str, Any]] = [
|
||||
{
|
||||
"source": "file:src/foo.ts",
|
||||
"target": "file:src/foo.test.ts",
|
||||
"type": "tested_by",
|
||||
"direction": "forward",
|
||||
"weight": 0.5,
|
||||
},
|
||||
{
|
||||
"source": "file:src/foo.test.ts",
|
||||
"target": "file:src/foo.ts",
|
||||
"type": "tested_by",
|
||||
"direction": "forward",
|
||||
"weight": 0.5,
|
||||
},
|
||||
]
|
||||
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(added, 0)
|
||||
# First edge was canonical; second was inverted but described the
|
||||
# same pair → dropped as a duplicate (not a swap).
|
||||
self.assertEqual(dropped, 1)
|
||||
self.assertEqual(swapped, 0)
|
||||
self.assertEqual(tagged, 1)
|
||||
self.assertEqual(len([e for e in edges if e["type"] == "tested_by"]), 1)
|
||||
|
||||
def test_supplement_skips_pair_already_covered_by_llm(self) -> None:
|
||||
# If the LLM (after swap) already covers a (production, test) pair
|
||||
# that a path-convention candidate would also produce, Pass 2 must
|
||||
# not emit a duplicate.
|
||||
nodes_by_id = {
|
||||
"file:src/foo.ts": _file_node("src/foo.ts"),
|
||||
"file:src/foo.test.ts": _file_node("src/foo.test.ts"),
|
||||
"file:src/bar.ts": _file_node("src/bar.ts"),
|
||||
"file:src/bar.test.ts": _file_node("src/bar.test.ts"),
|
||||
}
|
||||
# LLM only emitted (and inverted) the foo pair. The bar pair is
|
||||
# covered by Pass 2 (path convention).
|
||||
edges: list[dict[str, Any]] = [
|
||||
{
|
||||
"source": "file:src/foo.test.ts",
|
||||
"target": "file:src/foo.ts",
|
||||
"type": "tested_by",
|
||||
"direction": "forward",
|
||||
"weight": 0.5,
|
||||
},
|
||||
]
|
||||
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(swapped, 1)
|
||||
self.assertEqual(added, 1) # only bar; foo is already covered
|
||||
self.assertEqual(dropped, 0)
|
||||
self.assertEqual(tagged, 2)
|
||||
tested_by_edges = sorted(
|
||||
[e for e in edges if e["type"] == "tested_by"],
|
||||
key=lambda e: e["source"],
|
||||
)
|
||||
self.assertEqual(len(tested_by_edges), 2)
|
||||
|
||||
def test_swap_recovers_real_world_one_test_many_production(self) -> None:
|
||||
# Real case from microservices-demo: shippingservice_test.go does
|
||||
# not have a `shippingservice.go` sibling — it tests `main.go`,
|
||||
# `tracker.go`, and `quote.go`. Path convention can't pair these,
|
||||
# but the LLM saw the same-package usage and emitted the edges
|
||||
# (with wrong direction). Swap should recover them.
|
||||
nodes_by_id = {
|
||||
"file:src/shippingservice/main.go": _file_node("src/shippingservice/main.go"),
|
||||
"file:src/shippingservice/tracker.go": _file_node("src/shippingservice/tracker.go"),
|
||||
"file:src/shippingservice/quote.go": _file_node("src/shippingservice/quote.go"),
|
||||
"file:src/shippingservice/shippingservice_test.go": _file_node("src/shippingservice/shippingservice_test.go"),
|
||||
}
|
||||
edges: list[dict[str, Any]] = [
|
||||
{
|
||||
"source": "file:src/shippingservice/shippingservice_test.go",
|
||||
"target": "file:src/shippingservice/main.go",
|
||||
"type": "tested_by",
|
||||
"direction": "forward",
|
||||
"weight": 0.5,
|
||||
},
|
||||
{
|
||||
"source": "file:src/shippingservice/shippingservice_test.go",
|
||||
"target": "file:src/shippingservice/tracker.go",
|
||||
"type": "tested_by",
|
||||
"direction": "forward",
|
||||
"weight": 0.5,
|
||||
},
|
||||
]
|
||||
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(swapped, 2)
|
||||
# Pass 2 fallback: the test file with no shippingservice.go sibling
|
||||
# produces no path-convention candidate — we rely entirely on swap.
|
||||
self.assertEqual(added, 0)
|
||||
self.assertEqual(dropped, 0)
|
||||
# main.go and tracker.go were tagged; quote.go was not (LLM didn't
|
||||
# emit an edge for it, and there's no path-convention pair).
|
||||
self.assertEqual(tagged, 2)
|
||||
self.assertIn("tested", nodes_by_id["file:src/shippingservice/main.go"]["tags"])
|
||||
self.assertIn("tested", nodes_by_id["file:src/shippingservice/tracker.go"]["tags"])
|
||||
self.assertNotIn("tested", nodes_by_id["file:src/shippingservice/quote.go"]["tags"])
|
||||
|
||||
def test_unrelated_edges_pass_through(self) -> None:
|
||||
nodes_by_id = {
|
||||
"file:src/foo.ts": _file_node("src/foo.ts"),
|
||||
"file:src/foo.test.ts": _file_node("src/foo.test.ts"),
|
||||
}
|
||||
edges: list[dict[str, Any]] = [
|
||||
# LLM tested_by edge that gets stripped
|
||||
{
|
||||
"source": "file:src/foo.test.ts",
|
||||
"target": "file:src/foo.ts",
|
||||
@@ -302,7 +537,6 @@ class LinkTestsTests(unittest.TestCase):
|
||||
"direction": "forward",
|
||||
"weight": 0.5,
|
||||
},
|
||||
# Unrelated edge — should survive untouched
|
||||
{
|
||||
"source": "file:src/foo.ts",
|
||||
"target": "file:src/foo.test.ts",
|
||||
@@ -331,7 +565,7 @@ class LinkTestsTests(unittest.TestCase):
|
||||
}
|
||||
edges: list[dict[str, Any]] = []
|
||||
|
||||
added, dropped, tagged = mbg.link_tests(nodes_by_id, edges)
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(added, 3)
|
||||
for edge in edges:
|
||||
@@ -355,12 +589,14 @@ class LinkTestsTests(unittest.TestCase):
|
||||
edges: list[dict[str, Any]] = []
|
||||
|
||||
mbg.link_tests(nodes_by_id, edges)
|
||||
# Second invocation must not duplicate edges or tags.
|
||||
added2, dropped2, tagged2 = mbg.link_tests(nodes_by_id, edges)
|
||||
# Second invocation must not duplicate edges or tags. The first run
|
||||
# added a canonical supplement edge; the second sees it as canonical
|
||||
# in Pass 1 and keeps it without flipping or duplicating.
|
||||
added2, dropped2, tagged2, swapped2 = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
# On the second pass: existing deterministic edge gets stripped (it is
|
||||
# a tested_by edge) and re-added; tag is already there. So edge count
|
||||
# stays at 1 and tags has exactly one "tested".
|
||||
self.assertEqual((added2, dropped2, swapped2), (0, 0, 0))
|
||||
# Tag was already present, so tagged counter for second call is 0.
|
||||
self.assertEqual(tagged2, 0)
|
||||
tested_by_edges = [e for e in edges if e["type"] == "tested_by"]
|
||||
self.assertEqual(len(tested_by_edges), 1)
|
||||
tags = nodes_by_id["file:src/foo.ts"]["tags"]
|
||||
@@ -377,7 +613,7 @@ class LinkTestsTests(unittest.TestCase):
|
||||
}
|
||||
edges: list[dict[str, Any]] = []
|
||||
|
||||
added, dropped, tagged = mbg.link_tests(nodes_by_id, edges)
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(added, 1)
|
||||
self.assertEqual(tagged, 1)
|
||||
@@ -397,7 +633,7 @@ class LinkTestsTests(unittest.TestCase):
|
||||
}
|
||||
edges: list[dict[str, Any]] = []
|
||||
|
||||
added, dropped, tagged = mbg.link_tests(nodes_by_id, edges)
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual(added, 0)
|
||||
self.assertEqual(tagged, 0)
|
||||
@@ -419,8 +655,8 @@ class LinkTestsTests(unittest.TestCase):
|
||||
|
||||
def test_empty_input(self) -> None:
|
||||
edges: list[dict[str, Any]] = []
|
||||
added, dropped, tagged = mbg.link_tests({}, edges)
|
||||
self.assertEqual((added, dropped, tagged), (0, 0, 0))
|
||||
added, dropped, tagged, swapped = mbg.link_tests({}, edges)
|
||||
self.assertEqual((added, dropped, tagged, swapped), (0, 0, 0, 0))
|
||||
self.assertEqual(edges, [])
|
||||
|
||||
def test_node_without_filepath_falls_back_to_id(self) -> None:
|
||||
@@ -436,9 +672,9 @@ class LinkTestsTests(unittest.TestCase):
|
||||
nodes_by_id = {prod["id"]: prod, test["id"]: test}
|
||||
edges: list[dict[str, Any]] = []
|
||||
|
||||
added, dropped, tagged = mbg.link_tests(nodes_by_id, edges)
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual((added, dropped, tagged), (1, 0, 1))
|
||||
self.assertEqual((added, dropped, tagged, swapped), (1, 0, 1, 0))
|
||||
self.assertEqual(edges[0]["source"], "file:src/foo.ts")
|
||||
self.assertEqual(edges[0]["target"], "file:src/foo.test.ts")
|
||||
self.assertIn("tested", prod["tags"])
|
||||
@@ -460,9 +696,9 @@ class LinkTestsTests(unittest.TestCase):
|
||||
nodes_by_id = {prod["id"]: prod, test["id"]: test}
|
||||
edges: list[dict[str, Any]] = []
|
||||
|
||||
added, dropped, tagged = mbg.link_tests(nodes_by_id, edges)
|
||||
added, dropped, tagged, swapped = mbg.link_tests(nodes_by_id, edges)
|
||||
|
||||
self.assertEqual((added, dropped, tagged), (1, 0, 1))
|
||||
self.assertEqual((added, dropped, tagged, swapped), (1, 0, 1, 0))
|
||||
self.assertEqual(prod["tags"], ["tested"])
|
||||
|
||||
|
||||
|
||||
Reference in New Issue
Block a user