From 0e6dd755a6d5a4ff63e69b593a77d19e1fbddad1 Mon Sep 17 00:00:00 2001 From: Filipe Chagas Date: Thu, 6 Aug 2026 08:23:00 -0300 Subject: [PATCH 1/5] fix(extract): scope member-call resolvers to the sources they own MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Swift, Python and TypeScript member-call resolvers consumed every raw call in the corpus. Only the cpp, csharp, java and objc extractors stamp a `lang` tag, and those three languages carry none, so they mined each other's data: a TypeScript `Lead.search({})` reached the Python resolver's capitalized-receiver class arm and minted an EXTRACTED edge into a Python method with no TypeScript `Lead` anywhere in the corpus. Each resolver now consumes only raw calls written in the source files it owns (`_raw_call_is_owned`, the form `ruby_resolution._ruby_raw_calls` already uses) — a positive suffix filter that is closed by construction rather than a list of languages to exclude. The tagged languages keep matching on `lang`, because C++ and ObjC share `.h` and a suffix cannot tell their raw calls apart. The Java and C# receiver-type indexes are language-scoped for the same reason, which cuts both ways: a Java `Lead lead; lead.search()` bound to a Python `class Lead` at INFERRED, and a foreign class merely SHARING a short name pushed the single-definition guard to 2 and silently suppressed the correct same-language edge. Polyglot corpora therefore lose always-wrong cross-language edges and gain Java/C# edges a name collision previously deleted; single-language corpora are unaffected. The same index exposure in the C++, ObjC, Swift, TypeScript and Python resolvers is untouched. Per-resolver suffix tuples are shared between the filters and the `LanguageResolver` registrations so the two cannot drift. Adapted from https://github.com/lawnstarter/graphify/pull/28. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 1 + graphify/extract.py | 85 ++++++- tests/test_mixed_corpus_member_calls.py | 292 ++++++++++++++++++++++++ 3 files changed, 371 insertions(+), 7 deletions(-) create mode 100644 tests/test_mixed_corpus_member_calls.py diff --git a/CHANGELOG.md b/CHANGELOG.md index 03d264a115..ccd0fd1f7d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu ## 0.9.34 (unreleased) - Fix: C# receiver typing no longer drops a true call when a same-named variable is declared untypeably elsewhere in the method (#2472, thanks @JensD-git). Receiver types are now tracked per lexical declaration scope and resolved by the call's position, so a typed `static` local-function parameter keeps resolving even when an `out var` reuses the name in the enclosing body. This fixes a regression from 0.9.32 (#2346). Cross-method independence (#2299) and field-conflict poisoning are unchanged; an `out var` receiver itself remains untyped. +- Fix: a member-call resolver no longer mints edges out of another language's data, which is two fixes. First, the Swift, Python and TypeScript resolvers consumed every raw call in the corpus: only the cpp, csharp, java and objc extractors stamp a `lang` tag, and those three languages carry none, so a TypeScript `Lead.search({})` reached the Python resolver's capitalized-receiver class arm and minted an EXTRACTED edge into a Python method with no TypeScript `Lead` anywhere in the corpus. Each now consumes only raw calls written in the source files it owns, a positive suffix filter that is closed by construction rather than a list of languages to exclude; the tagged languages keep matching on `lang`, because C++ and ObjC share `.h` and a suffix cannot tell their raw calls apart. Second, the Java and C# receiver-type indexes are now scoped to their own sources: a Java `Lead lead; lead.search()` bound to a Python `class Lead` at INFERRED. Polyglot corpora therefore LOSE cross-language member-call edges that were always wrong — including some labelled EXTRACTED, the strongest confidence — and GAIN Java and C# edges that a foreign class merely sharing a short name previously deleted. Single-language corpora are unaffected: every filter added here admits everything such a corpus contains. The same index exposure in the C++, ObjC, Swift, TypeScript and Python resolvers is untouched and left as a follow-up. - Fix: `graphify path` (and the MCP `shortest_path` tool) now respect edge direction by default instead of running on an undirected view, so a returned path no longer traverses edges backwards (#2487, thanks @luliaz0601). Direction is recovered from the stored `_src`/`_tgt` markers. Pass `--undirected` (CLI) or `undirected=true` (MCP) to search ignoring direction; when no directed path exists the command says so instead of silently returning a reversed one. - Fix: semantic extraction no longer aborts at merge with a `TypeError` when a hyperedge carries dict-shaped members (#2486, thanks @adminwat). Members are normalized to ids (or dropped with a warning) so a malformed hyperedge can no longer destroy a completed extraction. - Fix: `graphify merge-graphs` no longer drops hyperedges (#2484, thanks @sortakool, and @oleksii-tumanov for the approach in #1691). Hyperedge member ids and ids are now relabeled with the per-repo prefix, both inputs' hyperedges are unioned instead of one clobbering the other, and they are written to both the top-level and nested slots. diff --git a/graphify/extract.py b/graphify/extract.py index dc7540d5fa..78426aae8a 100644 --- a/graphify/extract.py +++ b/graphify/extract.py @@ -2250,6 +2250,27 @@ def _assembly_of_node(nid: str) -> str: rc["caller_nid"] = remap[cn] +# Source suffixes each member-call resolver owns. Used BOTH to register the +# resolver and to scope what it consumes — one definition so the two cannot drift. +_SWIFT_RESOLVER_SUFFIXES = (".swift",) +_PYTHON_RESOLVER_SUFFIXES = (".py",) +_TYPESCRIPT_RESOLVER_SUFFIXES = (".ts", ".tsx", ".mts", ".cts", ".js", ".jsx") +_CSHARP_RESOLVER_SUFFIXES = (".cs",) +_JAVA_RESOLVER_SUFFIXES = (".java",) + + +def _raw_call_is_owned(rc: dict, suffixes: tuple[str, ...]) -> bool: + """True when ``rc`` was written in a source file with one of ``suffixes``. + + How the three untagged resolvers (Swift, Python, TypeScript) claim their raw + calls. The tagged languages (cpp, csharp, java, objc) match on the + extractor-stamped ``lang`` instead, because C++ and ObjC share `.h` and a + suffix alone cannot tell their raw calls apart. Prior art for the suffix + form: ``ruby_resolution._ruby_raw_calls``. + """ + return str(rc.get("source_file") or "").lower().endswith(suffixes) + + def _resolve_swift_member_calls( per_file: list[dict], all_nodes: list[dict], @@ -2312,6 +2333,13 @@ def _key(label: str) -> str: existing_pairs = {(e.get("source"), e.get("target")) for e in all_edges} for rc in all_raw_calls: + # Consume only raw calls written in Swift sources. A raw call carries no + # marker of its language unless the extractor stamped `lang` — and only + # cpp, csharp, java and objc do — so Python and TypeScript raw calls + # flowed straight into the arms below. A positive suffix filter is closed + # by construction, where excluding tagged languages one by one is not. + if not _raw_call_is_owned(rc, _SWIFT_RESOLVER_SUFFIXES): + continue if not rc.get("is_member_call"): continue receiver = rc.get("receiver") @@ -2473,6 +2501,12 @@ def _emit_call(caller: str, target_nid: "str | None", rc: dict) -> None: }) for rc in all_raw_calls: + # Consume only raw calls written in Python sources. Nothing separated + # this resolver's raw calls from the other untagged languages': a + # TypeScript `Lead.search({})` reached the class arm below and minted an + # EXTRACTED edge into a Python method with no TS `Lead` in the corpus. + if not _raw_call_is_owned(rc, _PYTHON_RESOLVER_SUFFIXES): + continue if not rc.get("is_member_call"): continue receiver = rc.get("receiver") @@ -2557,6 +2591,12 @@ def _key(label: str) -> str: existing_pairs = {(e.get("source"), e.get("target")) for e in all_edges} for rc in all_raw_calls: + # Consume only raw calls written in TS/JS sources. A Python or Swift raw + # call otherwise reached the arms below — and this resolver's + # `references` fallback minted a call-context edge onto the receiver's + # type even when no method matched. + if not _raw_call_is_owned(rc, _TYPESCRIPT_RESOLVER_SUFFIXES): + continue if not rc.get("is_member_call"): continue receiver = rc.get("receiver") @@ -2771,11 +2811,19 @@ def _key(label: str) -> str: contained = {e.get("target") for e in all_edges if e.get("relation") == "contains"} + # Scoped to C# sources: this index is the fallback for a name the + # namespace/using scoping below knows nothing about, and unscoped it matched + # a C# receiver type against classes written in ANY language. That cut both + # ways — a Python `class Lead` could type the receiver, and a Python class + # merely SHARING the name pushed the single-definition guard to 2 and + # silently suppressed the correct C# edge. type_def_nids: dict[str, list[str]] = {} node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - if n.get("source_file") and n.get("id") in contained and _is_type_like_definition(n): + sf = str(n.get("source_file") or "").lower() + if (sf.endswith(_CSHARP_RESOLVER_SUFFIXES) + and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) # Namespace/using/alias-aware simple-name resolution, shared with the C# @@ -2955,10 +3003,13 @@ def key(label: str) -> str: if edge.get("relation") == "contains"} node_by_id = {node.get("id"): node for node in all_nodes} + # Scoped to Java sources: an unscoped index let a Python `class Lead` type + # the receiver of `Lead lead; lead.search()` at INFERRED, and let a foreign + # class merely sharing the name suppress the correct Java edge. type_def_nids: dict[str, list[str]] = {} for node in all_nodes: if ( - node.get("source_file") + str(node.get("source_file") or "").lower().endswith(_JAVA_RESOLVER_SUFFIXES) and node.get("id") in contained and _is_type_like_definition(node) ): @@ -3140,10 +3191,18 @@ def _key(label: str) -> str: # by adding one register() call below — no edits to extract()'s body. Order # preserved from the prior inlined wiring: Swift (#1356) before Python (#1446). register_language_resolver( - LanguageResolver("swift_member_calls", frozenset({".swift"}), _resolve_swift_member_calls) + LanguageResolver( + "swift_member_calls", + frozenset(_SWIFT_RESOLVER_SUFFIXES), + _resolve_swift_member_calls, + ) ) register_language_resolver( - LanguageResolver("python_member_calls", frozenset({".py"}), _resolve_python_member_calls) + LanguageResolver( + "python_member_calls", + frozenset(_PYTHON_RESOLVER_SUFFIXES), + _resolve_python_member_calls, + ) ) # Ruby type-aware member-call resolution (Class.new + typed var.method). Lives in # graphify.ruby_resolution; registered here as a second consumer of the framework. @@ -3151,7 +3210,11 @@ def _key(label: str) -> str: LanguageResolver("ruby_member_calls", frozenset({".rb", ".rake"}), resolve_ruby_member_calls) ) register_language_resolver( - LanguageResolver("typescript_member_calls", frozenset({".ts", ".tsx", ".mts", ".cts", ".js", ".jsx"}), _resolve_typescript_member_calls) + LanguageResolver( + "typescript_member_calls", + frozenset(_TYPESCRIPT_RESOLVER_SUFFIXES), + _resolve_typescript_member_calls, + ) ) # C++ (#1547) and ObjC (#1556) receiver-typed member-call resolution. `.h` is in # both suffix sets because it routes to extract_cpp or extract_objc by content; the @@ -3173,10 +3236,18 @@ def _key(label: str) -> str: # C# receiver-typed member-call resolution (#1609): `field/param/local.Method()` # bound to the receiver's declared type instead of a bare same-named match. register_language_resolver( - LanguageResolver("csharp_member_calls", frozenset({".cs"}), _resolve_csharp_member_calls) + LanguageResolver( + "csharp_member_calls", + frozenset(_CSHARP_RESOLVER_SUFFIXES), + _resolve_csharp_member_calls, + ) ) register_language_resolver( - LanguageResolver("java_member_calls", frozenset({".java"}), _resolve_java_member_calls) + LanguageResolver( + "java_member_calls", + frozenset(_JAVA_RESOLVER_SUFFIXES), + _resolve_java_member_calls, + ) ) # Pascal/Delphi cross-file inherited-method-call resolution: a call from a # manual descendant class to a method it inherits from an ancestor declared diff --git a/tests/test_mixed_corpus_member_calls.py b/tests/test_mixed_corpus_member_calls.py new file mode 100644 index 0000000000..c88465bcf8 --- /dev/null +++ b/tests/test_mixed_corpus_member_calls.py @@ -0,0 +1,292 @@ +"""Mixed-corpus isolation for the member-call resolvers. + +A corpus that mixes languages must not let one language's data mint an edge +through a different language's member-call resolver. Two independent mechanisms +have to hold for that: + +* **Raw-call ownership** -- a resolver consumes only raw calls from source files + it owns. The cpp/csharp/java/objc resolvers claim theirs by the + extractor-stamped ``lang``; Swift, Python and TypeScript raw calls carry no + tag, so those three filter by source-file suffix instead. +* **Definition-index scoping** -- the receiver-type index a resolver builds + holds only types declared in its own sources. + +Every test goes through the public ``extract()`` seam, and the Python class here +doubles as the cross-language decoy: it owns an identically named method, so a +bare method-name match cannot tell it apart from the target. +""" +from __future__ import annotations + +from pathlib import Path + +from graphify.extract import extract + + +def _calls(tmp_path: Path, files: dict[str, str]): + paths = [] + for name, body in files.items(): + path = tmp_path / name + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(body, encoding="utf-8") + paths.append(path) + result = extract(paths, cache_root=tmp_path / "graphify-out") + calls = { + (edge["source"], edge["target"]): edge + for edge in result["edges"] + if edge.get("relation") == "calls" + } + return calls, result + + +def _nid(result: dict, label: str, file_suffix: str) -> str: + return next( + node["id"] + for node in result["nodes"] + if node.get("label") == label + and str(node.get("source_file") or "").endswith(file_suffix) + ) + + +def _call_context_pairs(result: dict) -> set[tuple[str, str]]: + """Every ``context: "call"`` edge as (source, target) pairs. + + Wider than ``_calls``: the Swift and TypeScript resolvers fall back to a + ``references`` edge onto the receiver's TYPE when the type has no such + method, so a leak through either of them can surface as ``references`` + rather than ``calls``. + """ + return { + (edge["source"], edge["target"]) + for edge in result["edges"] + if edge.get("context") == "call" + } + + +# A Python class whose method name collides with the other languages' callee. +# Nothing written in another language may ever bind to it. +_PY_DECOY = ( + "class Lead:\n" + " def search(self):\n" + " return []\n" +) + + +# ── Untagged resolvers consume each other's raw calls ──────────────────────── +# +# The extractor stamps `lang` only on cpp/csharp/java/objc raw calls. Swift, +# Python and TypeScript raw calls carry none, so those three resolvers had +# nothing at all separating their own raw calls from each other's: a TypeScript +# receiver reached the Python resolver's capitalized-receiver class arm and +# minted a cross-language edge at EXTRACTED. Only a positive source-file suffix +# filter closes that by construction. +# +# Each test below is built so that exactly one resolver can be the miner: the +# resolver that legitimately owns the raw call refuses it on its own terms, so +# any surviving edge is another language's resolver reaching across. + +_TS_TYPED_RECEIVER = ( + "class Dep { go() { return 1; } }\n" + "export class Widget {\n" + " constructor(private dep: Dep) {}\n" + " run() { return this.dep.go(); }\n" + "}\n" +) +"""A TS constructor parameter property. Produces the `ts_type_table` the +TypeScript resolver requires -- without one it returns early and cannot leak.""" + +_SWIFT_TYPED_RECEIVER = ( + "class Dep { func go() {} }\n" + "class Widget {\n" + " let dep: Dep = Dep()\n" + " func run() { dep.go() }\n" + "}\n" +) +"""The Swift twin: produces the `swift_type_table` the Swift resolver requires.""" + +# Calls a method the Python class does not own, so the Python resolver itself +# refuses (its method index misses) -- any edge onto `Lead` is foreign. +_PY_CALLER_MISSING_METHOD = ( + "from svc import Lead\n" + "\n" + "\n" + "def run():\n" + " Lead.missing()\n" +) + + +def test_typescript_receiver_mints_no_edge_into_a_python_method(tmp_path: Path): + """No TypeScript `Lead` exists anywhere in the corpus. + + `Lead.search({})` is a TypeScript raw call. Read as a Python raw call it + hits the Python resolver's capitalized-receiver class arm and binds to the + Python `Lead.search` at EXTRACTED -- the strongest confidence label -- for + a call written in a file the Python resolver does not own. + """ + calls, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "runner.ts": "class Runner {\n go() { return Lead.search({}); }\n}\n", + }) + + go = _nid(result, ".go()", "runner.ts") + py_search = _nid(result, ".search()", "svc.py") + assert (go, py_search) not in calls, \ + "a TypeScript raw call minted an edge into a Python method" + + +def test_python_raw_call_is_not_mined_by_the_typescript_resolver(tmp_path: Path): + """The reverse direction: a Python raw call reaching the TS resolver. + + `Lead.missing()` is refused by the Python resolver -- the Python `Lead` has + no `missing`. The TypeScript resolver, handed the same raw call, resolves + the receiver type and falls back to a `references` edge onto the class, + inventing a call-context edge out of Python source. + """ + _, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "caller.py": _PY_CALLER_MISSING_METHOD, + "widget.ts": _TS_TYPED_RECEIVER, + }) + pairs = _call_context_pairs(result) + + run = _nid(result, "run()", "caller.py") + py_lead = _nid(result, "Lead", "svc.py") + assert (run, py_lead) not in pairs, \ + "the TypeScript resolver mined a Python raw call" + # Positive control: the TS resolver still resolves its own typed receiver. + ts_run = _nid(result, ".run()", "widget.ts") + ts_go = _nid(result, ".go()", "widget.ts") + assert (ts_run, ts_go) in pairs, "the TypeScript resolver stopped resolving" + + +def test_python_raw_call_is_not_mined_by_the_swift_resolver(tmp_path: Path): + """Same shape, Swift twin: `Lead.missing()` is Python's raw call to refuse.""" + _, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "caller.py": _PY_CALLER_MISSING_METHOD, + "Widget.swift": _SWIFT_TYPED_RECEIVER, + }) + pairs = _call_context_pairs(result) + + run = _nid(result, "run()", "caller.py") + py_lead = _nid(result, "Lead", "svc.py") + assert (run, py_lead) not in pairs, \ + "the Swift resolver mined a Python raw call" + # Positive control: the Swift resolver still resolves its own typed receiver. + swift_run = _nid(result, ".run()", "Widget.swift") + swift_go = _nid(result, ".go()", "Widget.swift") + assert (swift_run, swift_go) in pairs, "the Swift resolver stopped resolving" + + +def test_swift_raw_call_is_not_mined_by_the_python_resolver(tmp_path: Path): + """A Swift raw call the Swift resolver refuses must not reach Python. + + `Bundle` is in `_LANGUAGE_BUILTIN_GLOBALS`, so the Swift resolver skips the + receiver outright (#2147) rather than binding a same-named user symbol. The + Python resolver has no such guard, so the Swift raw call flowed into its + class arm and bound to a Python `class Bundle` at EXTRACTED. + """ + calls, result = _calls(tmp_path, { + "svc.py": "class Bundle:\n def load(self):\n return []\n", + "Runner.swift": "class Runner {\n func go() { Bundle.load() }\n}\n", + "Widget.swift": _SWIFT_TYPED_RECEIVER, + }) + + go = _nid(result, ".go()", "Runner.swift") + py_load = _nid(result, ".load()", "svc.py") + assert (go, py_load) not in calls, \ + "a Swift raw call minted an edge into a Python method" + # Positive control, with a decoy: the Swift resolver still resolves its own + # typed receiver and does not fall back to a bare method-name match. + swift_run = _nid(result, ".run()", "Widget.swift") + swift_go = _nid(result, ".go()", "Widget.swift") + assert (swift_run, swift_go) in calls, "the Swift resolver stopped resolving" + assert (swift_run, go) not in calls, "the decoy Swift class received an edge" + + +# ── Language-scoped Java and C# receiver type indexes ──────────────────────── +# +# Both resolvers built `type_def_nids` from every type-like node in the corpus, +# so a Java `Lead lead; lead.search()` bound to a Python `class Lead` at +# INFERRED -- and, in the other direction, a foreign class merely SHARING the +# short name pushed the single-definition guard to 2 and silently suppressed +# the correct same-language edge. + + +def test_java_receiver_type_does_not_match_a_python_class(tmp_path: Path): + """Defect 1, Java: no Java `class Lead` exists, only a Python one.""" + calls, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "Runner.java": ( + "class Runner {\n" + " private Lead lead;\n" + " void go() { lead.search(); }\n" + "}\n" + ), + }) + + go = _nid(result, ".go()", "Runner.java") + py_search = _nid(result, ".search()", "svc.py") + assert (go, py_search) not in calls, \ + "a Java receiver type must not resolve against a Python class" + + +def test_java_receiver_resolves_despite_a_same_named_python_class(tmp_path: Path): + """Defect 2, Java: the cross-language name collision pushed the + single-definition guard to 2 and suppressed the legitimate Java edge.""" + calls, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "Lead.java": "class Lead { void search() {} }\n", + "Runner.java": ( + "class Runner {\n" + " private Lead lead;\n" + " void go() { lead.search(); }\n" + "}\n" + ), + }) + + go = _nid(result, ".go()", "Runner.java") + java_search = _nid(result, ".search()", "Lead.java") + py_search = _nid(result, ".search()", "svc.py") + assert (go, java_search) in calls, \ + "a same-named class in another language suppressed the real Java edge" + assert (go, py_search) not in calls, "the Python decoy received an edge" + assert calls[(go, java_search)]["confidence"] == "INFERRED" + + +def test_csharp_receiver_type_does_not_match_a_python_class(tmp_path: Path): + """Defect 1, C#: no C# `class Lead` exists, only a Python one.""" + calls, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "Runner.cs": ( + "public class Runner {\n" + " private Lead lead;\n" + " public void Go() { lead.search(); }\n" + "}\n" + ), + }) + + go = _nid(result, ".Go()", "Runner.cs") + py_search = _nid(result, ".search()", "svc.py") + assert (go, py_search) not in calls, \ + "a C# receiver type must not resolve against a Python class" + + +def test_csharp_receiver_resolves_despite_a_same_named_python_class(tmp_path: Path): + """Defect 2, C#: the legitimate C#-to-C# edge survives the collision.""" + calls, result = _calls(tmp_path, { + "svc.py": "class Lead:\n def Search(self):\n return []\n", + "Lead.cs": "public class Lead { public void Search() {} }\n", + "Runner.cs": ( + "public class Runner {\n" + " private Lead lead;\n" + " public void Go() { lead.Search(); }\n" + "}\n" + ), + }) + + go = _nid(result, ".Go()", "Runner.cs") + cs_search = _nid(result, ".Search()", "Lead.cs") + py_search = _nid(result, ".Search()", "svc.py") + assert (go, cs_search) in calls, \ + "a same-named class in another language suppressed the real C# edge" + assert (go, py_search) not in calls, "the Python decoy received an edge" From 7ddf65c7ef37bb37aa53d3a2bd7d27851d753868 Mon Sep 17 00:00:00 2001 From: Filipe Chagas Date: Thu, 6 Aug 2026 08:30:11 -0300 Subject: [PATCH 2/5] fix(extract): scope the remaining receiver-type indexes to their own sources MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The C++, Objective-C, Swift and TypeScript member-call resolvers each built `type_def_nids` from every type-like node in the corpus, so the receiver's declared type name was matched against class definitions written in ANY language — the last copies of the shape already fixed for Java and C#. It cut both ways in every one of them. A Python `class Lead` could answer the receiver type behind `Lead lead; lead.search()` (C++, INFERRED), `[Lead search]` (ObjC, EXTRACTED), `let lead: Lead` (Swift, INFERRED) and `private lead: Lead` (TypeScript, EXTRACTED); and in the other direction a foreign class merely SHARING the short name pushed the single-definition guard to 2 and silently deleted the correct same-language edge. Raw-call ownership cannot close this: the raw call being resolved is genuinely the resolver's own, and the leak is in what its index offers up. `.h` is scoped into both the C++ and the ObjC index, because it routes to either extractor by content and a C++ class and an ObjC @interface both live in one. The two are therefore isolated from every other language but not from each other, which no suffix can fix. The Python resolver's class index is a different shape (built from `method` edges, not source-scoped nodes) and is left alone. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 2 +- graphify/extract.py | 49 ++++++- tests/test_mixed_corpus_member_calls.py | 184 ++++++++++++++++++++++++ 3 files changed, 227 insertions(+), 8 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ccd0fd1f7d..a6d6ad3654 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu ## 0.9.34 (unreleased) - Fix: C# receiver typing no longer drops a true call when a same-named variable is declared untypeably elsewhere in the method (#2472, thanks @JensD-git). Receiver types are now tracked per lexical declaration scope and resolved by the call's position, so a typed `static` local-function parameter keeps resolving even when an `out var` reuses the name in the enclosing body. This fixes a regression from 0.9.32 (#2346). Cross-method independence (#2299) and field-conflict poisoning are unchanged; an `out var` receiver itself remains untyped. -- Fix: a member-call resolver no longer mints edges out of another language's data, which is two fixes. First, the Swift, Python and TypeScript resolvers consumed every raw call in the corpus: only the cpp, csharp, java and objc extractors stamp a `lang` tag, and those three languages carry none, so a TypeScript `Lead.search({})` reached the Python resolver's capitalized-receiver class arm and minted an EXTRACTED edge into a Python method with no TypeScript `Lead` anywhere in the corpus. Each now consumes only raw calls written in the source files it owns, a positive suffix filter that is closed by construction rather than a list of languages to exclude; the tagged languages keep matching on `lang`, because C++ and ObjC share `.h` and a suffix cannot tell their raw calls apart. Second, the Java and C# receiver-type indexes are now scoped to their own sources: a Java `Lead lead; lead.search()` bound to a Python `class Lead` at INFERRED. Polyglot corpora therefore LOSE cross-language member-call edges that were always wrong — including some labelled EXTRACTED, the strongest confidence — and GAIN Java and C# edges that a foreign class merely sharing a short name previously deleted. Single-language corpora are unaffected: every filter added here admits everything such a corpus contains. The same index exposure in the C++, ObjC, Swift, TypeScript and Python resolvers is untouched and left as a follow-up. +- Fix: a member-call resolver no longer mints edges out of another language's data, which is two fixes. First, the Swift, Python and TypeScript resolvers consumed every raw call in the corpus: only the cpp, csharp, java and objc extractors stamp a `lang` tag, and those three languages carry none, so a TypeScript `Lead.search({})` reached the Python resolver's capitalized-receiver class arm and minted an EXTRACTED edge into a Python method with no TypeScript `Lead` anywhere in the corpus. Each now consumes only raw calls written in the source files it owns, a positive suffix filter that is closed by construction rather than a list of languages to exclude; the tagged languages keep matching on `lang`, because C++ and ObjC share `.h` and a suffix cannot tell their raw calls apart. Second, every receiver-type index is now scoped to its own sources — Java, C#, C++, Objective-C, Swift and TypeScript — where each previously matched the receiver's declared type against class definitions written in ANY language. That cut both ways, so it is itself two fixes: a Java `Lead lead; lead.search()` bound to a Python `class Lead` at INFERRED, and a foreign class merely SHARING a short name pushed the single-definition guard to 2 and silently suppressed the correct same-language edge. Polyglot corpora therefore LOSE cross-language member-call edges that were always wrong — including some labelled EXTRACTED, the strongest confidence — and GAIN edges that a foreign short-name collision previously deleted. Single-language corpora are unaffected: every filter added here admits everything such a corpus contains. `.h` is scoped into both the C++ and the Objective-C index, because it routes to either extractor by content; the two are therefore isolated from every other language but not from each other. The Python resolver's own class index is not suffix-scoped and is left as a follow-up. - Fix: `graphify path` (and the MCP `shortest_path` tool) now respect edge direction by default instead of running on an undirected view, so a returned path no longer traverses edges backwards (#2487, thanks @luliaz0601). Direction is recovered from the stored `_src`/`_tgt` markers. Pass `--undirected` (CLI) or `undirected=true` (MCP) to search ignoring direction; when no directed path exists the command says so instead of silently returning a reversed one. - Fix: semantic extraction no longer aborts at merge with a `TypeError` when a hyperedge carries dict-shaped members (#2486, thanks @adminwat). Members are normalized to ids (or dropped with a warning) so a malformed hyperedge can no longer destroy a completed extraction. - Fix: `graphify merge-graphs` no longer drops hyperedges (#2484, thanks @sortakool, and @oleksii-tumanov for the approach in #1691). Hyperedge member ids and ids are now relabeled with the per-repo prefix, both inputs' hyperedges are unioned instead of one clobbering the other, and they are written to both the top-level and nested slots. diff --git a/graphify/extract.py b/graphify/extract.py index 78426aae8a..b3ad71be03 100644 --- a/graphify/extract.py +++ b/graphify/extract.py @@ -2257,6 +2257,16 @@ def _assembly_of_node(nid: str) -> str: _TYPESCRIPT_RESOLVER_SUFFIXES = (".ts", ".tsx", ".mts", ".cts", ".js", ".jsx") _CSHARP_RESOLVER_SUFFIXES = (".cs",) _JAVA_RESOLVER_SUFFIXES = (".java",) +# `.h` routes to extract_cpp or extract_objc by content, so it appears in both +# the C++ and ObjC sets. Their raw calls are claimed by the extractor-stamped +# `lang`, never by suffix; only the DEFINITION index is scoped by suffix, where +# including `.h` is correct — a C++ class and an ObjC @interface both live in +# one. The consequence is that C++ and ObjC are isolated from every other +# language but not from each other, which no suffix can fix. +_CPP_RESOLVER_SUFFIXES = ( + ".cpp", ".cc", ".cxx", ".hpp", ".cu", ".cuh", ".metal", ".h", +) +_OBJC_RESOLVER_SUFFIXES = (".m", ".mm", ".h") def _raw_call_is_owned(rc: dict, suffixes: tuple[str, ...]) -> bool: @@ -2309,12 +2319,17 @@ def _key(label: str) -> str: contained = {e.get("target") for e in all_edges if e.get("relation") == "contains"} # Type name -> definition node ids (real, source-backed, type-like defs only). - # len != 1 is the god-node guard: an ambiguous type name bails. + # len != 1 is the god-node guard: an ambiguous type name bails. Scoped to + # Swift sources: unscoped, a Python `class Lead` could type `let lead: Lead` + # and mint a cross-language edge, and a foreign class merely SHARING the + # short name pushed the guard to 2 and deleted the correct Swift edge. type_def_nids: dict[str, list[str]] = {} node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - if n.get("source_file") and n.get("id") in contained and _is_type_like_definition(n): + sf = str(n.get("source_file") or "").lower() + if (sf.endswith(_SWIFT_RESOLVER_SUFFIXES) + and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) # (type_node_id, method_key) -> method_node_id, from `method` edges. @@ -2569,11 +2584,17 @@ def _key(label: str) -> str: contained = {e.get("target") for e in all_edges if e.get("relation") == "contains"} + # Scoped to TS/JS sources: unscoped, a Python `class Lead` could type + # `private lead: Lead` and mint a cross-language edge, and a foreign class + # merely SHARING the short name pushed the single-definition guard to 2 and + # deleted the correct TypeScript edge. type_def_nids: dict[str, list[str]] = {} node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - if n.get("source_file") and n.get("id") in contained and _is_type_like_definition(n): + sf = str(n.get("source_file") or "").lower() + if (sf.endswith(_TYPESCRIPT_RESOLVER_SUFFIXES) + and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) method_index: dict[tuple[str, str], str] = {} @@ -2681,11 +2702,18 @@ def _key(label: str) -> str: # excluding non-contained nodes keeps them from making a real type ambiguous. contained = {e.get("target") for e in all_edges if e.get("relation") == "contains"} + # Scoped to C++ sources: unscoped, a Python `class Lead` could type + # `Lead lead;` and mint a cross-language edge, and a foreign class merely + # SHARING the short name pushed the single-definition guard to 2 and deleted + # the correct C++ edge. `.h` is in the set (and in ObjC's), so the two stay + # distinguishable from every other language but not from each other. type_def_nids: dict[str, list[str]] = {} node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - if n.get("source_file") and n.get("id") in contained and _is_type_like_definition(n): + sf = str(n.get("source_file") or "").lower() + if (sf.endswith(_CPP_RESOLVER_SUFFIXES) + and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) # (type_node_id, method_key) -> method_node_id, and caller -> enclosing type @@ -3111,11 +3139,18 @@ def _key(label: str) -> str: contained = {e.get("target") for e in all_edges if e.get("relation") == "contains"} + # Scoped to ObjC sources: unscoped, a Python `class Lead` could answer the + # receiver type behind `[Lead search]` — at EXTRACTED, since a capitalized + # receiver names its type explicitly — and a foreign class merely SHARING + # the short name pushed the single-definition guard to 2 and deleted the + # correct ObjC edge. `.h` is in the set (and in C++'s); see the note there. type_def_nids: dict[str, list[str]] = {} node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - if n.get("source_file") and n.get("id") in contained and _is_type_like_definition(n): + sf = str(n.get("source_file") or "").lower() + if (sf.endswith(_OBJC_RESOLVER_SUFFIXES) + and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) method_index: dict[tuple[str, str], str] = {} @@ -3222,14 +3257,14 @@ def _key(label: str) -> str: register_language_resolver( LanguageResolver( "cpp_member_calls", - frozenset({".cpp", ".cc", ".cxx", ".hpp", ".cu", ".cuh", ".metal", ".h"}), + frozenset(_CPP_RESOLVER_SUFFIXES), _resolve_cpp_member_calls, ) ) register_language_resolver( LanguageResolver( "objc_member_calls", - frozenset({".m", ".mm", ".h"}), + frozenset(_OBJC_RESOLVER_SUFFIXES), _resolve_objc_member_calls, ) ) diff --git a/tests/test_mixed_corpus_member_calls.py b/tests/test_mixed_corpus_member_calls.py index c88465bcf8..2965b2690a 100644 --- a/tests/test_mixed_corpus_member_calls.py +++ b/tests/test_mixed_corpus_member_calls.py @@ -62,6 +62,25 @@ def _call_context_pairs(result: dict) -> set[tuple[str, str]]: } +def _cross_language_targets(result: dict, caller: str, file_suffix: str) -> set[str]: + """Call-context targets of ``caller`` that live in ``file_suffix``. + + Wider than naming one node: an index leak surfaces as ``calls`` onto the + foreign METHOD, or — when the callee name misses on the foreign type — as a + ``references`` edge onto the foreign TYPE itself. Both are the same bug. + """ + foreign = { + node["id"] + for node in result["nodes"] + if str(node.get("source_file") or "").endswith(file_suffix) + } + return { + target + for source, target in _call_context_pairs(result) + if source == caller and target in foreign + } + + # A Python class whose method name collides with the other languages' callee. # Nothing written in another language may ever bind to it. _PY_DECOY = ( @@ -290,3 +309,168 @@ def test_csharp_receiver_resolves_despite_a_same_named_python_class(tmp_path: Pa assert (go, cs_search) in calls, \ "a same-named class in another language suppressed the real C# edge" assert (go, py_search) not in calls, "the Python decoy received an edge" + + +# ── Language-scoped C++, ObjC, Swift and TypeScript receiver type indexes ──── +# +# The remaining copies of the same shape. Each of these four resolvers built +# `type_def_nids` from every type-like node in the corpus, so the receiver's +# declared type name was matched against class definitions written in ANY +# language. Both directions of the defect are covered per language: the foreign +# class TYPING the receiver, and the foreign class merely SHARING the short name +# pushing the single-definition guard to 2 and deleting the correct edge. +# +# Raw-call ownership (above) cannot close this: the raw call being resolved is +# genuinely the resolver's own, and the leak is in what its INDEX offers up. + +_CPP_CALLER = ( + "class Runner {\n" + "public:\n" + " void go() { Lead lead; lead.search(); }\n" + "};\n" +) +"""`Lead lead;` is a local declaration, so the C++ `cpp_type_table` types the +receiver and the call resolves at INFERRED.""" + +_OBJC_CALLER = ( + "@interface Runner : NSObject\n" + "- (void)go;\n" + "@end\n" + "\n" + "@implementation Runner\n" + "- (void)go {\n" + " [Lead search];\n" + "}\n" + "@end\n" +) +"""A capitalized ObjC receiver names the type explicitly, so this arm resolves +at EXTRACTED -- the strongest confidence a leak can carry.""" + +_SWIFT_CALLER = ( + "class Runner {\n" + " let lead: Lead\n" + " func go() { lead.search() }\n" + "}\n" +) +"""Declared without an initializer on purpose: `= Lead()` would additionally be +picked up by the shared cross-file CALL pass, which is a different mechanism +and would muddy what this test pins down.""" + +_TS_CALLER = ( + "export class Runner {\n" + " constructor(private lead: Lead) {}\n" + " go() { return this.lead.search(); }\n" + "}\n" +) + + +def test_cpp_receiver_type_does_not_match_a_python_class(tmp_path: Path): + """No C++ `Lead` exists in the corpus, only a Python one.""" + _, result = _calls(tmp_path, {"svc.py": _PY_DECOY, "Runner.cpp": _CPP_CALLER}) + + go = _nid(result, ".go()", "Runner.cpp") + assert not _cross_language_targets(result, go, "svc.py"), \ + "a C++ receiver type must not resolve against a Python class" + + +def test_cpp_receiver_resolves_despite_a_same_named_python_class(tmp_path: Path): + """The same-named Python class must not suppress the real C++ edge.""" + calls, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "lead.cpp": "class Lead {\npublic:\n void search() {}\n};\n", + "Runner.cpp": _CPP_CALLER, + }) + + go = _nid(result, ".go()", "Runner.cpp") + cpp_search = _nid(result, ".search()", "lead.cpp") + py_search = _nid(result, ".search()", "svc.py") + assert (go, cpp_search) in calls, \ + "a same-named Python class suppressed the real C++ edge" + assert (go, py_search) not in calls, "the Python decoy received an edge" + assert calls[(go, cpp_search)]["confidence"] == "INFERRED" + + +def test_objc_receiver_type_does_not_match_a_python_class(tmp_path: Path): + """No ObjC `Lead` exists in the corpus, only a Python one.""" + _, result = _calls(tmp_path, {"svc.py": _PY_DECOY, "Runner.m": _OBJC_CALLER}) + + go = _nid(result, "-go", "Runner.m") + assert not _cross_language_targets(result, go, "svc.py"), \ + "an ObjC receiver type must not resolve against a Python class" + + +def test_objc_receiver_resolves_despite_a_same_named_python_class(tmp_path: Path): + """The same-named Python class must not suppress the real ObjC edge.""" + calls, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "Lead.m": ( + "@interface Lead : NSObject\n" + "+ (void)search;\n" + "@end\n" + "\n" + "@implementation Lead\n" + "+ (void)search {}\n" + "@end\n" + ), + "Runner.m": _OBJC_CALLER, + }) + + go = _nid(result, "-go", "Runner.m") + objc_search = _nid(result, "+search", "Lead.m") + py_search = _nid(result, ".search()", "svc.py") + assert (go, objc_search) in calls, \ + "a same-named Python class suppressed the real ObjC edge" + assert (go, py_search) not in calls, "the Python decoy received an edge" + assert calls[(go, objc_search)]["confidence"] == "EXTRACTED" + + +def test_swift_receiver_type_does_not_match_a_python_class(tmp_path: Path): + """No Swift `Lead` exists in the corpus, only a Python one.""" + _, result = _calls(tmp_path, {"svc.py": _PY_DECOY, "Runner.swift": _SWIFT_CALLER}) + + go = _nid(result, ".go()", "Runner.swift") + assert not _cross_language_targets(result, go, "svc.py"), \ + "a Swift receiver type must not resolve against a Python class" + + +def test_swift_receiver_resolves_despite_a_same_named_python_class(tmp_path: Path): + """The same-named Python class must not suppress the real Swift edge.""" + calls, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "Lead.swift": "class Lead { func search() {} }\n", + "Runner.swift": _SWIFT_CALLER, + }) + + go = _nid(result, ".go()", "Runner.swift") + swift_search = _nid(result, ".search()", "Lead.swift") + py_search = _nid(result, ".search()", "svc.py") + assert (go, swift_search) in calls, \ + "a same-named Python class suppressed the real Swift edge" + assert (go, py_search) not in calls, "the Python decoy received an edge" + assert calls[(go, swift_search)]["confidence"] == "INFERRED" + + +def test_typescript_receiver_type_does_not_match_a_python_class(tmp_path: Path): + """No TypeScript `Lead` exists in the corpus, only a Python one.""" + _, result = _calls(tmp_path, {"svc.py": _PY_DECOY, "runner.ts": _TS_CALLER}) + + go = _nid(result, ".go()", "runner.ts") + assert not _cross_language_targets(result, go, "svc.py"), \ + "a TypeScript receiver type must not resolve against a Python class" + + +def test_typescript_receiver_resolves_despite_a_same_named_python_class(tmp_path: Path): + """The same-named Python class must not suppress the real TypeScript edge.""" + calls, result = _calls(tmp_path, { + "svc.py": _PY_DECOY, + "lead.ts": "export class Lead { search() { return []; } }\n", + "runner.ts": _TS_CALLER, + }) + + go = _nid(result, ".go()", "runner.ts") + ts_search = _nid(result, ".search()", "lead.ts") + py_search = _nid(result, ".search()", "svc.py") + assert (go, ts_search) in calls, \ + "a same-named Python class suppressed the real TypeScript edge" + assert (go, py_search) not in calls, "the Python decoy received an edge" + assert calls[(go, ts_search)]["confidence"] == "EXTRACTED" From 0f417fc369400167d2b9068c195f7822e40db72d Mon Sep 17 00:00:00 2001 From: Filipe Chagas Date: Thu, 6 Aug 2026 08:36:58 -0300 Subject: [PATCH 3/5] fix(extract): scope the Python resolver's class and module indexes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The last unscoped member-call index, and the one shaped differently from all the others: the Python resolver builds its class index by walking `method` edges rather than by filtering source-backed nodes, and resolves `module.func()` through the caller's own `imports` edges. Both were corpus-wide. The class arm leaked both ways. `Lead.search()` bound to a Java `class Lead` at EXTRACTED with no Python `Lead` in the corpus, and a foreign class merely SHARING the name pushed the single-definition guard to 2 and deleted the correct Python edge. Only classes declared in Python sources are candidates now; `method_index` needs no scoping, since it is only ever keyed by a class id the scoped index already admitted. The module arm leaked one way: it matched any corpus file whose stem equalled the receiver, so `import lead` beside a `lead.ts` bound `lead.search()` to a TypeScript function at EXTRACTED. A candidate module must now be a Python file. Its suppression direction is unreachable and left alone — two same-stem files disambiguate the file node ids while the `imports` edge target stays the bare alias, so the arm sees zero candidates rather than an ambiguous two. That is a separate defect in the import-alias remap. Index scoping across all seven resolvers now goes through one `_is_owned_definition` helper, the definition-index twin of `_raw_call_is_owned`, replacing the inline suffix checks added alongside each. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 2 +- graphify/extract.py | 40 +++++++---- tests/test_mixed_corpus_member_calls.py | 95 ++++++++++++++++++++++++- 3 files changed, 121 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index a6d6ad3654..8384655984 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,7 +5,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu ## 0.9.34 (unreleased) - Fix: C# receiver typing no longer drops a true call when a same-named variable is declared untypeably elsewhere in the method (#2472, thanks @JensD-git). Receiver types are now tracked per lexical declaration scope and resolved by the call's position, so a typed `static` local-function parameter keeps resolving even when an `out var` reuses the name in the enclosing body. This fixes a regression from 0.9.32 (#2346). Cross-method independence (#2299) and field-conflict poisoning are unchanged; an `out var` receiver itself remains untyped. -- Fix: a member-call resolver no longer mints edges out of another language's data, which is two fixes. First, the Swift, Python and TypeScript resolvers consumed every raw call in the corpus: only the cpp, csharp, java and objc extractors stamp a `lang` tag, and those three languages carry none, so a TypeScript `Lead.search({})` reached the Python resolver's capitalized-receiver class arm and minted an EXTRACTED edge into a Python method with no TypeScript `Lead` anywhere in the corpus. Each now consumes only raw calls written in the source files it owns, a positive suffix filter that is closed by construction rather than a list of languages to exclude; the tagged languages keep matching on `lang`, because C++ and ObjC share `.h` and a suffix cannot tell their raw calls apart. Second, every receiver-type index is now scoped to its own sources — Java, C#, C++, Objective-C, Swift and TypeScript — where each previously matched the receiver's declared type against class definitions written in ANY language. That cut both ways, so it is itself two fixes: a Java `Lead lead; lead.search()` bound to a Python `class Lead` at INFERRED, and a foreign class merely SHARING a short name pushed the single-definition guard to 2 and silently suppressed the correct same-language edge. Polyglot corpora therefore LOSE cross-language member-call edges that were always wrong — including some labelled EXTRACTED, the strongest confidence — and GAIN edges that a foreign short-name collision previously deleted. Single-language corpora are unaffected: every filter added here admits everything such a corpus contains. `.h` is scoped into both the C++ and the Objective-C index, because it routes to either extractor by content; the two are therefore isolated from every other language but not from each other. The Python resolver's own class index is not suffix-scoped and is left as a follow-up. +- Fix: a member-call resolver no longer mints edges out of another language's data, which is two fixes. First, the Swift, Python and TypeScript resolvers consumed every raw call in the corpus: only the cpp, csharp, java and objc extractors stamp a `lang` tag, and those three languages carry none, so a TypeScript `Lead.search({})` reached the Python resolver's capitalized-receiver class arm and minted an EXTRACTED edge into a Python method with no TypeScript `Lead` anywhere in the corpus. Each now consumes only raw calls written in the source files it owns, a positive suffix filter that is closed by construction rather than a list of languages to exclude; the tagged languages keep matching on `lang`, because C++ and ObjC share `.h` and a suffix cannot tell their raw calls apart. Second, every receiver-type index is now scoped to its own sources — Java, C#, C++, Objective-C, Swift, TypeScript and Python — where each previously matched the receiver's declared type against class definitions written in ANY language. That cut both ways, so it is itself two fixes: a Java `Lead lead; lead.search()` bound to a Python `class Lead` at INFERRED, and a foreign class merely SHARING a short name pushed the single-definition guard to 2 and silently suppressed the correct same-language edge. Polyglot corpora therefore LOSE cross-language member-call edges that were always wrong — including some labelled EXTRACTED, the strongest confidence — and GAIN edges that a foreign short-name collision previously deleted. Single-language corpora are unaffected: every filter added here admits everything such a corpus contains. `.h` is scoped into both the C++ and the Objective-C index, because it routes to either extractor by content; the two are therefore isolated from every other language but not from each other. Python is scoped on both of its arms: a `ClassName.method()` receiver no longer binds to a class written in another language, and a `module.func()` receiver no longer resolves to a same-stem file that is not Python — `import lead` beside a `lead.ts` bound the call to a TypeScript function at EXTRACTED. - Fix: `graphify path` (and the MCP `shortest_path` tool) now respect edge direction by default instead of running on an undirected view, so a returned path no longer traverses edges backwards (#2487, thanks @luliaz0601). Direction is recovered from the stored `_src`/`_tgt` markers. Pass `--undirected` (CLI) or `undirected=true` (MCP) to search ignoring direction; when no directed path exists the command says so instead of silently returning a reversed one. - Fix: semantic extraction no longer aborts at merge with a `TypeError` when a hyperedge carries dict-shaped members (#2486, thanks @adminwat). Members are normalized to ids (or dropped with a warning) so a malformed hyperedge can no longer destroy a completed extraction. - Fix: `graphify merge-graphs` no longer drops hyperedges (#2484, thanks @sortakool, and @oleksii-tumanov for the approach in #1691). Hyperedge member ids and ids are now relabeled with the per-repo prefix, both inputs' hyperedges are unioned instead of one clobbering the other, and they are written to both the top-level and nested slots. diff --git a/graphify/extract.py b/graphify/extract.py index b3ad71be03..f80020168c 100644 --- a/graphify/extract.py +++ b/graphify/extract.py @@ -2281,6 +2281,16 @@ def _raw_call_is_owned(rc: dict, suffixes: tuple[str, ...]) -> bool: return str(rc.get("source_file") or "").lower().endswith(suffixes) +def _is_owned_definition(node: "dict | None", suffixes: tuple[str, ...]) -> bool: + """True when ``node`` was declared in a source file with one of ``suffixes``. + + The definition-index twin of ``_raw_call_is_owned``. Every member-call + resolver scopes its receiver-type index through this, so a receiver's + declared type can only ever bind to a type written in the same language. + """ + return str((node or {}).get("source_file") or "").lower().endswith(suffixes) + + def _resolve_swift_member_calls( per_file: list[dict], all_nodes: list[dict], @@ -2327,8 +2337,7 @@ def _key(label: str) -> str: node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - sf = str(n.get("source_file") or "").lower() - if (sf.endswith(_SWIFT_RESOLVER_SUFFIXES) + if (_is_owned_definition(n, _SWIFT_RESOLVER_SUFFIXES) and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) @@ -2435,7 +2444,11 @@ def _key(label: str) -> str: # A class owns methods: it is the source of one or more `method` edges. Index # class label -> owning class node ids (len != 1 is the god-node guard), and - # (class_node_id, method_key) -> method_node_id. + # (class_node_id, method_key) -> method_node_id. Only classes declared in + # Python sources are candidates: unscoped, `Lead.search()` bound to a Java + # `class Lead` at EXTRACTED, and a foreign class merely SHARING the name + # pushed the guard to 2 and deleted the correct Python edge. `method_index` + # needs no scoping — it is only ever keyed by a class id already admitted here. class_def_nids: dict[str, list[str]] = {} method_index: dict[tuple[str, str], str] = {} for e in all_edges: @@ -2443,7 +2456,7 @@ def _key(label: str) -> str: continue src, tgt = e.get("source"), e.get("target") cnode = node_by_id.get(src) - if cnode is not None: + if cnode is not None and _is_owned_definition(cnode, _PYTHON_RESOLVER_SUFFIXES): class_def_nids.setdefault(_key(cnode.get("label", "")), []).append(src) tnode = node_by_id.get(tgt) if tnode is not None: @@ -2543,11 +2556,16 @@ def _emit_call(caller: str, target_nid: "str | None", rc: dict) -> None: # never match), then to the single callable that module contains. A # receiver also matches the local alias bound on that import edge # (#2082), so an aliased import resolves the same as the bare name. + # A candidate module must itself be a Python file: matching on the + # stem alone, a `lead.ts` answered `import lead` and bound the call + # to a TypeScript function, and a `lead.ts` sitting beside the real + # `lead.py` made the pair ambiguous and deleted the true edge. rkey = _key(receiver) caller_file = file_of_node.get(caller) file_aliases = import_alias_by_filenode.get(caller_file, {}) mods = [t for t in imported_by_filenode.get(caller_file, ()) if t in contains_children + and _is_owned_definition(node_by_id.get(t), _PYTHON_RESOLVER_SUFFIXES) and (_module_stem_key(t) == rkey or file_aliases.get(t) == rkey)] if len(mods) != 1: # not an imported module, or ambiguous -> bail continue @@ -2592,8 +2610,7 @@ def _key(label: str) -> str: node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - sf = str(n.get("source_file") or "").lower() - if (sf.endswith(_TYPESCRIPT_RESOLVER_SUFFIXES) + if (_is_owned_definition(n, _TYPESCRIPT_RESOLVER_SUFFIXES) and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) @@ -2711,8 +2728,7 @@ def _key(label: str) -> str: node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - sf = str(n.get("source_file") or "").lower() - if (sf.endswith(_CPP_RESOLVER_SUFFIXES) + if (_is_owned_definition(n, _CPP_RESOLVER_SUFFIXES) and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) @@ -2849,8 +2865,7 @@ def _key(label: str) -> str: node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - sf = str(n.get("source_file") or "").lower() - if (sf.endswith(_CSHARP_RESOLVER_SUFFIXES) + if (_is_owned_definition(n, _CSHARP_RESOLVER_SUFFIXES) and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) @@ -3037,7 +3052,7 @@ def key(label: str) -> str: type_def_nids: dict[str, list[str]] = {} for node in all_nodes: if ( - str(node.get("source_file") or "").lower().endswith(_JAVA_RESOLVER_SUFFIXES) + _is_owned_definition(node, _JAVA_RESOLVER_SUFFIXES) and node.get("id") in contained and _is_type_like_definition(node) ): @@ -3148,8 +3163,7 @@ def _key(label: str) -> str: node_by_id: dict[str, dict] = {} for n in all_nodes: node_by_id[n.get("id")] = n - sf = str(n.get("source_file") or "").lower() - if (sf.endswith(_OBJC_RESOLVER_SUFFIXES) + if (_is_owned_definition(n, _OBJC_RESOLVER_SUFFIXES) and n.get("id") in contained and _is_type_like_definition(n)): type_def_nids.setdefault(_key(n.get("label", "")), []).append(n["id"]) diff --git a/tests/test_mixed_corpus_member_calls.py b/tests/test_mixed_corpus_member_calls.py index 2965b2690a..0c73344193 100644 --- a/tests/test_mixed_corpus_member_calls.py +++ b/tests/test_mixed_corpus_member_calls.py @@ -22,14 +22,14 @@ from graphify.extract import extract -def _calls(tmp_path: Path, files: dict[str, str]): +def _calls(tmp_path: Path, files: dict[str, str], cache_root: Path | None = None): paths = [] for name, body in files.items(): path = tmp_path / name path.parent.mkdir(parents=True, exist_ok=True) path.write_text(body, encoding="utf-8") paths.append(path) - result = extract(paths, cache_root=tmp_path / "graphify-out") + result = extract(paths, cache_root=cache_root or (tmp_path / "graphify-out")) calls = { (edge["source"], edge["target"]): edge for edge in result["edges"] @@ -474,3 +474,94 @@ def test_typescript_receiver_resolves_despite_a_same_named_python_class(tmp_path "a same-named Python class suppressed the real TypeScript edge" assert (go, py_search) not in calls, "the Python decoy received an edge" assert calls[(go, ts_search)]["confidence"] == "EXTRACTED" + + +# ── Language-scoped Python class and module indexes ────────────────────────── +# +# The Python resolver's two arms are indexed differently from every resolver +# above -- its class index is built by walking `method` edges rather than by +# filtering source-backed nodes, and its module arm resolves through the +# caller's own `imports` edges -- but both were corpus-wide all the same. +# +# The class arm is covered in both directions. The module arm is covered only +# for the leak: its suppression direction is unreachable, because two same-stem +# files disambiguate the FILE node ids (`lead_py_lead`, `lead_ts_lead`) while +# the `imports` edge target stays the bare alias `lead`, so the arm sees zero +# candidates rather than an ambiguous two. That is a separate defect in the +# import-alias remap, untouched here. + +_JAVA_DECOY = "class Lead { void search() {} }\n" + + +def test_python_class_receiver_does_not_match_a_java_class(tmp_path: Path): + """Class arm, defect 1: no Python `Lead` exists, only a Java one. + + `Lead.search()` is a Python raw call the Python resolver rightly owns. Its + class index held every class in the corpus, so the Java `Lead` answered for + the receiver and the call was minted at EXTRACTED -- the label reserved for + an explicitly named, unambiguous class reference. + """ + _, result = _calls(tmp_path, { + "caller.py": "def run():\n Lead.search()\n", + "Lead.java": _JAVA_DECOY, + }) + + run = _nid(result, "run()", "caller.py") + assert not _cross_language_targets(result, run, "Lead.java"), \ + "a Python class receiver must not resolve against a Java class" + + +def test_python_class_receiver_resolves_despite_a_same_named_java_class(tmp_path: Path): + """Class arm, defect 2: the Java decoy must not delete the Python edge.""" + calls, result = _calls(tmp_path, { + "caller.py": "from svc import Lead\n\n\ndef run():\n Lead.search()\n", + "svc.py": _PY_DECOY, + "Lead.java": _JAVA_DECOY, + }) + + run = _nid(result, "run()", "caller.py") + py_search = _nid(result, ".search()", "svc.py") + java_search = _nid(result, ".search()", "Lead.java") + assert (run, py_search) in calls, \ + "a same-named Java class suppressed the real Python edge" + assert (run, java_search) not in calls, "the Java decoy received an edge" + assert calls[(run, py_search)]["confidence"] == "EXTRACTED" + + +# The module arm resolves an `import` to the imported file's node, which only +# lines up when ids are relativized against the corpus root itself -- hence the +# explicit `cache_root`, matching the prior art in +# `tests/test_extract.py::test_python_module_qualified_call_resolves_extracted`. +_MOD_CALLER = "import lead\n\n\ndef run():\n lead.search()\n" +_TS_MODULE_DECOY = "export function search() { return []; }\n" + + +def test_python_module_receiver_does_not_match_a_typescript_file(tmp_path: Path): + """Module arm, defect 1: `import lead` must not reach `lead.ts`. + + No `lead.py` exists. The arm matched any corpus file whose stem equalled the + receiver, so a TypeScript file of the same stem satisfied it and + `lead.search()` bound to a TypeScript function at EXTRACTED -- an import + edge Python could not possibly have. + """ + _, result = _calls(tmp_path, { + "caller.py": _MOD_CALLER, + "lead.ts": _TS_MODULE_DECOY, + }, cache_root=tmp_path) + + run = _nid(result, "run()", "caller.py") + assert not _cross_language_targets(result, run, "lead.ts"), \ + "a Python module receiver must not resolve against a TypeScript file" + + +def test_python_module_receiver_still_resolves_its_own_module(tmp_path: Path): + """The module arm's positive control: scoping must not cost the real edge.""" + calls, result = _calls(tmp_path, { + "caller.py": _MOD_CALLER, + "lead.py": "def search():\n return []\n", + }, cache_root=tmp_path) + + run = _nid(result, "run()", "caller.py") + py_search = _nid(result, "search()", "lead.py") + assert (run, py_search) in calls, "the module arm stopped resolving" + assert calls[(run, py_search)]["confidence"] == "EXTRACTED" From 176428014aed0e022c91813cf7fa21f161deefa0 Mon Sep 17 00:00:00 2001 From: Filipe Chagas Date: Thu, 6 Aug 2026 08:42:21 -0300 Subject: [PATCH 4/5] fix(extract): keep Python import edges alive across a same-stem foreign sibling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A Python import edge targets the imported module's bare file-node id (`import lead` -> `lead`), which holds only while that id is unique. Drop a `lead.ts` beside `lead.py` and the two file nodes collide, so `_disambiguate_colliding_node_ids` salts them into `lead_py_lead` and `lead_ts_lead`. The edge's target salt is keyed by the IMPORTER's source_file, which matches neither, so the edge was left pointing at an id that no longer named anything — silently dropped, together with everything downstream of it. The disambiguator already accepts a `target_file` hint for exactly this shape (#1814), keying the target salt by that file instead. Python import edges now stamp it. A Python import can only ever mean a Python file, so the hint is unambiguous even when the colliding sibling belongs to another language; an id claimed by more than one Python file is left dangling, as before. The hint is transient and popped by its only reader, so it never reaches graph.json. `.pyi` is excluded: it has no extractor, so it mints no file node to point at. This is what made the Python resolver's `module.func()` arm look like it had an ambiguity guard firing on a foreign same-stem file. It did not — it was seeing zero candidates, not two. Both layers are now asserted. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 1 + graphify/extract.py | 57 +++++++++++++++++++++ tests/test_mixed_corpus_member_calls.py | 67 ++++++++++++++++++++++--- 3 files changed, 119 insertions(+), 6 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 8384655984..a878bb2b19 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,6 +6,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu - Fix: C# receiver typing no longer drops a true call when a same-named variable is declared untypeably elsewhere in the method (#2472, thanks @JensD-git). Receiver types are now tracked per lexical declaration scope and resolved by the call's position, so a typed `static` local-function parameter keeps resolving even when an `out var` reuses the name in the enclosing body. This fixes a regression from 0.9.32 (#2346). Cross-method independence (#2299) and field-conflict poisoning are unchanged; an `out var` receiver itself remains untyped. - Fix: a member-call resolver no longer mints edges out of another language's data, which is two fixes. First, the Swift, Python and TypeScript resolvers consumed every raw call in the corpus: only the cpp, csharp, java and objc extractors stamp a `lang` tag, and those three languages carry none, so a TypeScript `Lead.search({})` reached the Python resolver's capitalized-receiver class arm and minted an EXTRACTED edge into a Python method with no TypeScript `Lead` anywhere in the corpus. Each now consumes only raw calls written in the source files it owns, a positive suffix filter that is closed by construction rather than a list of languages to exclude; the tagged languages keep matching on `lang`, because C++ and ObjC share `.h` and a suffix cannot tell their raw calls apart. Second, every receiver-type index is now scoped to its own sources — Java, C#, C++, Objective-C, Swift, TypeScript and Python — where each previously matched the receiver's declared type against class definitions written in ANY language. That cut both ways, so it is itself two fixes: a Java `Lead lead; lead.search()` bound to a Python `class Lead` at INFERRED, and a foreign class merely SHARING a short name pushed the single-definition guard to 2 and silently suppressed the correct same-language edge. Polyglot corpora therefore LOSE cross-language member-call edges that were always wrong — including some labelled EXTRACTED, the strongest confidence — and GAIN edges that a foreign short-name collision previously deleted. Single-language corpora are unaffected: every filter added here admits everything such a corpus contains. `.h` is scoped into both the C++ and the Objective-C index, because it routes to either extractor by content; the two are therefore isolated from every other language but not from each other. Python is scoped on both of its arms: a `ClassName.method()` receiver no longer binds to a class written in another language, and a `module.func()` receiver no longer resolves to a same-stem file that is not Python — `import lead` beside a `lead.ts` bound the call to a TypeScript function at EXTRACTED. +- Fix: a Python `imports` edge no longer vanishes when a file of another language shares its stem. `import lead` targets the imported module's bare file-node id (`lead`), but a `lead.ts` beside `lead.py` makes the two file nodes collide, so id-disambiguation salts them into `lead_py_lead` and `lead_ts_lead` and the edge — keyed by the importer's own file rather than the target's — was left pointing at an id that no longer named anything. Python import edges now stamp the `target_file` hint the disambiguator already accepts for this, so the salt lands on the Python sibling; a Python import can only mean a Python file, so the choice is unambiguous even when the collider is another language's. Every consumer of those edges is affected, `module.func()` call resolution among them. An id claimed by more than one Python file is left dangling, as before. - Fix: `graphify path` (and the MCP `shortest_path` tool) now respect edge direction by default instead of running on an undirected view, so a returned path no longer traverses edges backwards (#2487, thanks @luliaz0601). Direction is recovered from the stored `_src`/`_tgt` markers. Pass `--undirected` (CLI) or `undirected=true` (MCP) to search ignoring direction; when no directed path exists the command says so instead of silently returning a reversed one. - Fix: semantic extraction no longer aborts at merge with a `TypeError` when a hyperedge carries dict-shaped members (#2486, thanks @adminwat). Members are normalized to ids (or dropped with a warning) so a malformed hyperedge can no longer destroy a completed extraction. - Fix: `graphify merge-graphs` no longer drops hyperedges (#2484, thanks @sortakool, and @oleksii-tumanov for the approach in #1691). Hyperedge member ids and ids are now relabeled with the per-repo prefix, both inputs' hyperedges are unioned instead of one clobbering the other, and they are written to both the top-level and nested slots. diff --git a/graphify/extract.py b/graphify/extract.py index f80020168c..df64d21748 100644 --- a/graphify/extract.py +++ b/graphify/extract.py @@ -256,6 +256,58 @@ def _repoint_python_package_imports(paths, all_nodes, all_edges, root) -> None: e["target"] = alias_map[tgt] +def _hint_python_import_targets(paths, all_edges, root) -> None: + """Tell id-disambiguation which file a Python import edge really targets. + + A Python import edge targets the bare file-node id of the imported module + (``import lead`` -> ``lead``), which works only while that id is unique. Add + a ``lead.ts`` and the two file nodes collide, so + ``_disambiguate_colliding_node_ids`` salts them apart into ``lead_py_lead`` + and ``lead_ts_lead`` — and the edge, keyed by the IMPORTER's source_file + rather than the target's, matches neither salt and is left pointing at an id + that no longer names anything. The import edge is dropped and every consumer + of it loses out, the Python resolver's ``module.func()`` arm among them. + + The disambiguator already accepts a ``target_file`` hint for exactly this + (#1814): stamp it and the target salt is keyed by that file instead. A + Python import can only ever mean a Python file, so the hint is unambiguous + even when the colliding sibling is another language's. Guards: never + overwrite a hint an emitter already stamped, and skip an id claimed by more + than one Python file (leave it dangling, as before). + + ``.pyi`` is excluded deliberately — it has no extractor, so it mints no file + node to point a hint at. Must run BEFORE ``_disambiguate_colliding_node_ids``. + """ + try: + root = Path(root).resolve() + except OSError: + root = Path(root) + file_id_to_paths: dict[str, set[str]] = {} + for p in paths: + if p.suffix.lower() != ".py": + continue + try: + rel = Path(p).resolve().relative_to(root) + except (ValueError, OSError): + continue + file_id_to_paths.setdefault(_file_node_id(rel), set()).add(str(p)) + hint_map = { + fid: next(iter(ps)) for fid, ps in file_id_to_paths.items() if len(ps) == 1 + } + if not hint_map: + return + for e in all_edges: + if ( + isinstance(e, dict) + and e.get("relation") in ("imports", "imports_from") + and not e.get("target_file") + and str(e.get("source_file", "")).lower().endswith((".py", ".pyi")) + ): + target_path = hint_map.get(e.get("target")) + if target_path: + e["target_file"] = target_path + + SEMANTIC_RELATIONS = frozenset({ "inherits", "implements", "mixes_in", "embeds", "references", "calls", "imports", "imports_from", "re_exports", "contains", "method", @@ -5427,6 +5479,11 @@ def _learn(e: dict) -> None: # (src/) package root before the resolver/import-evidence passes run, so the # graph is identical regardless of scan root (#2072). _repoint_python_package_imports(paths, all_nodes, all_edges, root) + # Then hint the disambiguator at each import's real target file, so a + # same-stem sibling in another language cannot strand the edge on a salted- + # away id. Must be after the repoint above (it rewrites some targets) and + # before disambiguation (the hint's only reader). + _hint_python_import_targets(paths, all_edges, root) _merge_swift_extensions(per_file, all_nodes, all_edges) _merge_csharp_partial_class_nodes(per_file, all_nodes, all_edges, paths, root) _disambiguate_colliding_node_ids(all_nodes, all_edges, all_raw_calls, root) diff --git a/tests/test_mixed_corpus_member_calls.py b/tests/test_mixed_corpus_member_calls.py index 0c73344193..c01fa55ea7 100644 --- a/tests/test_mixed_corpus_member_calls.py +++ b/tests/test_mixed_corpus_member_calls.py @@ -483,12 +483,12 @@ def test_typescript_receiver_resolves_despite_a_same_named_python_class(tmp_path # filtering source-backed nodes, and its module arm resolves through the # caller's own `imports` edges -- but both were corpus-wide all the same. # -# The class arm is covered in both directions. The module arm is covered only -# for the leak: its suppression direction is unreachable, because two same-stem -# files disambiguate the FILE node ids (`lead_py_lead`, `lead_ts_lead`) while -# the `imports` edge target stays the bare alias `lead`, so the arm sees zero -# candidates rather than an ambiguous two. That is a separate defect in the -# import-alias remap, untouched here. +# Both arms are covered in both directions. The module arm's suppression case +# turned out not to be an index defect at all: two same-stem files disambiguate +# the FILE node ids (`lead_py_lead`, `lead_ts_lead`) while the `imports` edge +# target stays the bare alias `lead`, so the arm saw ZERO candidates rather +# than an ambiguous two. That is fixed one layer down, in the import-alias +# hinting, and asserted at both layers below. _JAVA_DECOY = "class Lead { void search() {} }\n" @@ -565,3 +565,58 @@ def test_python_module_receiver_still_resolves_its_own_module(tmp_path: Path): py_search = _nid(result, "search()", "lead.py") assert (run, py_search) in calls, "the module arm stopped resolving" assert calls[(run, py_search)]["confidence"] == "EXTRACTED" + + +def test_python_module_receiver_resolves_despite_a_same_stem_typescript_file( + tmp_path: Path, +): + """Module arm, defect 2: a same-stem foreign file must not delete the edge. + + `lead.py` and `lead.ts` both derive the file node id `lead`, so + disambiguation salts them into `lead_py_lead` and `lead_ts_lead` — while the + `imports` edge target stays the bare alias `lead`, which now names nothing. + The arm then sees ZERO candidate modules rather than an ambiguous two, and + the one edge Python's own import genuinely supports disappears. + """ + calls, result = _calls(tmp_path, { + "caller.py": _MOD_CALLER, + "lead.py": "def search():\n return []\n", + "lead.ts": _TS_MODULE_DECOY, + }, cache_root=tmp_path) + + run = _nid(result, "run()", "caller.py") + py_search = _nid(result, "search()", "lead.py") + ts_search = _nid(result, "search()", "lead.ts") + assert (run, py_search) in calls, \ + "a same-stem TypeScript file suppressed the real Python edge" + assert (run, ts_search) not in calls, "the TypeScript decoy received an edge" + assert calls[(run, py_search)]["confidence"] == "EXTRACTED" + + +def test_python_import_edge_survives_a_same_stem_foreign_sibling(tmp_path: Path): + """The `imports` edge itself, one layer below the call edge above. + + Asserted separately because the module arm is only one consumer: any query + over Python imports lost this edge to the same dangling alias. + """ + _, result = _calls(tmp_path, { + "caller.py": _MOD_CALLER, + "lead.py": "def search():\n return []\n", + "lead.ts": _TS_MODULE_DECOY, + }, cache_root=tmp_path) + + node_ids = {node["id"] for node in result["nodes"]} + py_file = _nid(result, "lead.py", "lead.py") + ts_file = _nid(result, "lead.ts", "lead.ts") + imports = { + (edge["source"], edge["target"]) + for edge in result["edges"] + if edge["relation"] in ("imports", "imports_from") + } + caller_file = _nid(result, "caller.py", "caller.py") + assert (caller_file, py_file) in imports, \ + "the Python import edge dangled on the pre-disambiguation alias" + assert (caller_file, ts_file) not in imports, \ + "a Python import must never resolve to a TypeScript file" + for source, target in imports: + assert target in node_ids, f"import edge dangles: {source} -> {target}" From 1c774907c5348d46d12739261423220d0ce9571d Mon Sep 17 00:00:00 2001 From: Filipe Chagas Date: Thu, 6 Aug 2026 08:54:57 -0300 Subject: [PATCH 5/5] fix(extract): keep stem-named import edges alive across a same-stem sibling MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Generalizes the Python fix to every language with the same exposure, found by auditing each import emitter against a control/collision corpus pair. Rust, Zig, Elixir, PowerShell and Pascal all name an import's target by the imported file's bare stem id, which resolves only while that id is unique. Any same-stem file — a `lead.md` is enough — collides the two file nodes, so id-disambiguation salts them apart and the edge, keyed by the IMPORTER's file rather than the target's, matches neither salt and dangles. They now stamp the `target_file` hint the disambiguator already reads (#1814), through one corpus-level pass rather than per extractor: an extractor sees a single file and cannot know which same-stem candidate the id will end up naming. The hint only selects among salted variants of an id the edge already named, so it cannot change which node an edge resolves to, and a corpus with no collision is bit-identical. Bash was different, and stamping there would have been useless: its second producer of the same edge, in `resolve_bash_source_edges`, runs AFTER disambiguation and derives every id from the path formula, so under a collision it emitted a dangling duplicate beside the extractor's correct edge. It now receives the file node ids as they actually stand; the duplicate collapses back into one deduped edge, and the source-backed `calls` edges the same pass resolves stop pointing at renamed-away ids too. Audited and left alone: TypeScript/JavaScript stamp the hint already, C/C++/ObjC are covered by the header carve-out (#1475), and Julia/Fortran/Verilog target an importer-scoped node that cannot collide. Go's package imports dangle with or without a collision — a separate gap, not this one. Co-Authored-By: Claude Opus 5 --- CHANGELOG.md | 2 +- graphify/extract.py | 143 +++++++++++++++++----- graphify/symbol_resolution.py | 13 +- tests/test_import_alias_disambiguation.py | 141 +++++++++++++++++++++ 4 files changed, 264 insertions(+), 35 deletions(-) create mode 100644 tests/test_import_alias_disambiguation.py diff --git a/CHANGELOG.md b/CHANGELOG.md index a878bb2b19..178e66687c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ Full release notes with details on each version: [GitHub Releases](https://githu - Fix: C# receiver typing no longer drops a true call when a same-named variable is declared untypeably elsewhere in the method (#2472, thanks @JensD-git). Receiver types are now tracked per lexical declaration scope and resolved by the call's position, so a typed `static` local-function parameter keeps resolving even when an `out var` reuses the name in the enclosing body. This fixes a regression from 0.9.32 (#2346). Cross-method independence (#2299) and field-conflict poisoning are unchanged; an `out var` receiver itself remains untyped. - Fix: a member-call resolver no longer mints edges out of another language's data, which is two fixes. First, the Swift, Python and TypeScript resolvers consumed every raw call in the corpus: only the cpp, csharp, java and objc extractors stamp a `lang` tag, and those three languages carry none, so a TypeScript `Lead.search({})` reached the Python resolver's capitalized-receiver class arm and minted an EXTRACTED edge into a Python method with no TypeScript `Lead` anywhere in the corpus. Each now consumes only raw calls written in the source files it owns, a positive suffix filter that is closed by construction rather than a list of languages to exclude; the tagged languages keep matching on `lang`, because C++ and ObjC share `.h` and a suffix cannot tell their raw calls apart. Second, every receiver-type index is now scoped to its own sources — Java, C#, C++, Objective-C, Swift, TypeScript and Python — where each previously matched the receiver's declared type against class definitions written in ANY language. That cut both ways, so it is itself two fixes: a Java `Lead lead; lead.search()` bound to a Python `class Lead` at INFERRED, and a foreign class merely SHARING a short name pushed the single-definition guard to 2 and silently suppressed the correct same-language edge. Polyglot corpora therefore LOSE cross-language member-call edges that were always wrong — including some labelled EXTRACTED, the strongest confidence — and GAIN edges that a foreign short-name collision previously deleted. Single-language corpora are unaffected: every filter added here admits everything such a corpus contains. `.h` is scoped into both the C++ and the Objective-C index, because it routes to either extractor by content; the two are therefore isolated from every other language but not from each other. Python is scoped on both of its arms: a `ClassName.method()` receiver no longer binds to a class written in another language, and a `module.func()` receiver no longer resolves to a same-stem file that is not Python — `import lead` beside a `lead.ts` bound the call to a TypeScript function at EXTRACTED. -- Fix: a Python `imports` edge no longer vanishes when a file of another language shares its stem. `import lead` targets the imported module's bare file-node id (`lead`), but a `lead.ts` beside `lead.py` makes the two file nodes collide, so id-disambiguation salts them into `lead_py_lead` and `lead_ts_lead` and the edge — keyed by the importer's own file rather than the target's — was left pointing at an id that no longer named anything. Python import edges now stamp the `target_file` hint the disambiguator already accepts for this, so the salt lands on the Python sibling; a Python import can only mean a Python file, so the choice is unambiguous even when the collider is another language's. Every consumer of those edges is affected, `module.func()` call resolution among them. An id claimed by more than one Python file is left dangling, as before. +- Fix: an `imports` edge no longer vanishes when any same-stem file sits beside its target, in Python, Rust, Zig, Elixir, PowerShell, Pascal and Bash. Each of these names an import's target by the imported file's bare stem id (`import lead` -> `lead`), which resolves only while that id is unique: add a `lead.md` and the two file nodes collide, so id-disambiguation salts them into `lead_py_lead` and `lead_md_lead` while the edge — keyed by the importer's own file rather than the target's — was left pointing at an id that no longer named anything, and was dropped along with everything downstream of it (Python's `module.func()` call resolution among them). These edges now stamp the `target_file` hint the disambiguator already accepts for this, so the salt lands on the right file; an import written in one language can only mean a file of that language, so the choice stays unambiguous whatever the collider is. An id claimed by more than one importable file of the same language is left dangling, as before, and a corpus with no collision is unchanged. Bash additionally emitted the edge twice under a collision — once correct, once dangling — because its second producer in `resolve_bash_source_edges` derives ids from the path after disambiguation has already renamed them; that pass now reads the ids as they actually stand, which also repairs the source-backed `calls` edges it resolves. TypeScript/JavaScript and C/C++/Objective-C already had equivalent protection; Julia, Fortran and Verilog target an importer-scoped node and were never exposed. - Fix: `graphify path` (and the MCP `shortest_path` tool) now respect edge direction by default instead of running on an undirected view, so a returned path no longer traverses edges backwards (#2487, thanks @luliaz0601). Direction is recovered from the stored `_src`/`_tgt` markers. Pass `--undirected` (CLI) or `undirected=true` (MCP) to search ignoring direction; when no directed path exists the command says so instead of silently returning a reversed one. - Fix: semantic extraction no longer aborts at merge with a `TypeError` when a hyperedge carries dict-shaped members (#2486, thanks @adminwat). Members are normalized to ids (or dropped with a warning) so a malformed hyperedge can no longer destroy a completed extraction. - Fix: `graphify merge-graphs` no longer drops hyperedges (#2484, thanks @sortakool, and @oleksii-tumanov for the approach in #1691). Hyperedge member ids and ids are now relabeled with the per-repo prefix, both inputs' hyperedges are unioned instead of one clobbering the other, and they are written to both the top-level and nested slots. diff --git a/graphify/extract.py b/graphify/extract.py index df64d21748..ac2f4e776f 100644 --- a/graphify/extract.py +++ b/graphify/extract.py @@ -256,56 +256,130 @@ def _repoint_python_package_imports(paths, all_nodes, all_edges, root) -> None: e["target"] = alias_map[tgt] -def _hint_python_import_targets(paths, all_edges, root) -> None: - """Tell id-disambiguation which file a Python import edge really targets. +# Languages whose import edges name their target by the imported file's bare +# stem id, as (importer suffixes, importable target suffixes). Each pair is +# closed within one language: an import written in one of the first set can only +# ever mean a file from the second, which is what makes the hint below +# unambiguous even when the colliding sibling belongs to another language. +# +# Python's target set omits `.pyi` on purpose — it has no extractor, so it mints +# no file node to point a hint at, and listing it could only mask a real `.py` +# target behind a phantom ambiguity. +_IMPORT_STEM_LANGUAGES: tuple[tuple[tuple[str, ...], tuple[str, ...]], ...] = ( + ((".py", ".pyi"), (".py",)), + ((".rs",), (".rs",)), + ((".zig",), (".zig",)), + ((".ex", ".exs"), (".ex", ".exs")), + ((".ps1", ".psm1", ".psd1"), (".ps1", ".psm1", ".psd1")), + ((".sh", ".bash"), (".sh", ".bash")), + ( + (".pas", ".pp", ".dpr", ".dpk", ".inc"), + (".pas", ".pp", ".dpr", ".dpk", ".inc"), + ), +) - A Python import edge targets the bare file-node id of the imported module - (``import lead`` -> ``lead``), which works only while that id is unique. Add - a ``lead.ts`` and the two file nodes collide, so - ``_disambiguate_colliding_node_ids`` salts them apart into ``lead_py_lead`` - and ``lead_ts_lead`` — and the edge, keyed by the IMPORTER's source_file - rather than the target's, matches neither salt and is left pointing at an id - that no longer names anything. The import edge is dropped and every consumer - of it loses out, the Python resolver's ``module.func()`` arm among them. - The disambiguator already accepts a ``target_file`` hint for exactly this - (#1814): stamp it and the target salt is keyed by that file instead. A - Python import can only ever mean a Python file, so the hint is unambiguous - even when the colliding sibling is another language's. Guards: never - overwrite a hint an emitter already stamped, and skip an id claimed by more - than one Python file (leave it dangling, as before). - - ``.pyi`` is excluded deliberately — it has no extractor, so it mints no file - node to point a hint at. Must run BEFORE ``_disambiguate_colliding_node_ids``. +def _file_nids_by_path(all_nodes, all_edges, root) -> dict: + """Resolved source path -> the id its file node carries RIGHT NOW. + + For passes that run after ``_disambiguate_colliding_node_ids`` and therefore + cannot derive a file's id from its path: a same-stem sibling salts the id + away from whatever the path formula would produce. A file node is the one + that ``contains`` its file's other nodes; an empty file contains nothing, so + fall back to the node whose label is the file's own basename. """ try: root = Path(root).resolve() except OSError: root = Path(root) - file_id_to_paths: dict[str, set[str]] = {} - for p in paths: - if p.suffix.lower() != ".py": + contains_sources = { + e.get("source") for e in all_edges if e.get("relation") == "contains" + } + by_path: dict[Path, str] = {} + fallback: dict[Path, str] = {} + for n in all_nodes: + source_file, nid = n.get("source_file"), n.get("id") + if not source_file or not nid: continue + candidate = Path(str(source_file)) try: - rel = Path(p).resolve().relative_to(root) - except (ValueError, OSError): + resolved = ( + candidate if candidate.is_absolute() else root / candidate + ).resolve() + except OSError: continue - file_id_to_paths.setdefault(_file_node_id(rel), set()).add(str(p)) - hint_map = { - fid: next(iter(ps)) for fid, ps in file_id_to_paths.items() if len(ps) == 1 - } - if not hint_map: + if nid in contains_sources: + by_path.setdefault(resolved, nid) + elif n.get("label") == resolved.name: + fallback.setdefault(resolved, nid) + for resolved, nid in fallback.items(): + by_path.setdefault(resolved, nid) + return by_path + + +def _hint_import_targets(paths, all_edges, root) -> None: + """Tell id-disambiguation which file each stem-named import edge targets. + + The extractors in ``_IMPORT_STEM_LANGUAGES`` name an import's target by the + bare file-node id of the imported file (``import lead`` -> ``lead``), which + resolves only while that id is unique. Add ANY same-stem file — a ``lead.md`` + will do — and the two file nodes collide, so + ``_disambiguate_colliding_node_ids`` salts them apart into ``lead_py_lead`` + and ``lead_md_lead``. The edge's target salt is keyed by the IMPORTER's + source_file, which matches neither, so the edge is left pointing at an id + that no longer names anything: silently dropped, along with everything + downstream of it (Python's ``module.func()`` call resolution among them). + + The disambiguator already accepts a ``target_file`` hint for exactly this + (#1814), keying the target salt by that file instead. Stamp it here rather + than in each extractor, because an extractor sees one file and cannot know + which of the corpus's same-stem candidates the id will end up naming. + + This cannot change WHICH node an edge resolves to — the hint only selects + among the salted variants of an id the edge already named — so a corpus with + no collision is bit-identical. Guards: never overwrite a hint an emitter + already stamped, and skip an id claimed by more than one importable file of + the same language (leave it dangling, as before). + + Must run BEFORE ``_disambiguate_colliding_node_ids``, the hint's only reader. + """ + try: + root = Path(root).resolve() + except OSError: + root = Path(root) + hints: list[tuple[tuple[str, ...], dict[str, str]]] = [] + for importer_suffixes, target_suffixes in _IMPORT_STEM_LANGUAGES: + file_id_to_paths: dict[str, set[str]] = {} + for p in paths: + if p.suffix.lower() not in target_suffixes: + continue + try: + rel = Path(p).resolve().relative_to(root) + except (ValueError, OSError): + continue + file_id_to_paths.setdefault(_file_node_id(rel), set()).add(str(p)) + hint_map = { + fid: next(iter(ps)) for fid, ps in file_id_to_paths.items() if len(ps) == 1 + } + if hint_map: + hints.append((importer_suffixes, hint_map)) + if not hints: return for e in all_edges: - if ( + if not ( isinstance(e, dict) and e.get("relation") in ("imports", "imports_from") and not e.get("target_file") - and str(e.get("source_file", "")).lower().endswith((".py", ".pyi")) ): + continue + source_file = str(e.get("source_file", "")).lower() + for importer_suffixes, hint_map in hints: + if not source_file.endswith(importer_suffixes): + continue target_path = hint_map.get(e.get("target")) if target_path: e["target_file"] = target_path + break SEMANTIC_RELATIONS = frozenset({ @@ -5483,7 +5557,7 @@ def _learn(e: dict) -> None: # same-stem sibling in another language cannot strand the edge on a salted- # away id. Must be after the repoint above (it rewrites some targets) and # before disambiguation (the hint's only reader). - _hint_python_import_targets(paths, all_edges, root) + _hint_import_targets(paths, all_edges, root) _merge_swift_extensions(per_file, all_nodes, all_edges) _merge_csharp_partial_class_nodes(per_file, all_nodes, all_edges, paths, root) _disambiguate_colliding_node_ids(all_nodes, all_edges, all_raw_calls, root) @@ -5589,7 +5663,10 @@ def _looks_like_bash(result: object) -> bool: sh_paths = [p for _, p in sh_pairs] try: all_edges.extend( - resolve_bash_source_edges(sh_results, sh_paths, root, existing_edges=all_edges) + resolve_bash_source_edges( + sh_results, sh_paths, root, existing_edges=all_edges, + file_nids=_file_nids_by_path(all_nodes, all_edges, root), + ) ) except Exception as exc: import logging diff --git a/graphify/symbol_resolution.py b/graphify/symbol_resolution.py index 892f310650..9f442c323f 100644 --- a/graphify/symbol_resolution.py +++ b/graphify/symbol_resolution.py @@ -406,6 +406,7 @@ def resolve_bash_source_edges( paths: Sequence[Path], root: Path, existing_edges: list[dict] | None = None, + file_nids: dict[Path, str] | None = None, ) -> list[dict]: """Resolve Bash source/import edges and source-backed function calls. @@ -428,7 +429,17 @@ def resolve_bash_source_edges( Anything else is silently skipped. """ path_by_index = [Path(p).resolve() for p in paths] - file_nid_by_path = {p: _file_node_id_for_path(p, root) for p in path_by_index} # resolved paths only + # `file_nids` carries the file node ids as they actually stand in the graph. + # This pass runs AFTER id-disambiguation, so when a same-stem sibling made a + # file id collide, the salted id no longer matches what + # `_file_node_id_for_path` derives from the path — every edge built from the + # formula would then name a node that does not exist. Fall back to the + # formula for any path the caller did not resolve (and for direct callers + # that pass nothing), which is exactly the previous behavior. + known_nids = file_nids or {} + file_nid_by_path = { # resolved paths only + p: known_nids.get(p) or _file_node_id_for_path(p, root) for p in path_by_index + } functions_by_file: dict[str, dict[str, str]] = {} for result, path in zip(per_file, path_by_index): diff --git a/tests/test_import_alias_disambiguation.py b/tests/test_import_alias_disambiguation.py new file mode 100644 index 0000000000..650f642a8d --- /dev/null +++ b/tests/test_import_alias_disambiguation.py @@ -0,0 +1,141 @@ +"""An import edge must survive a same-stem file in another language. + +Several extractors name an import's target by the bare file-stem id of the +imported file (``import lead`` -> ``lead``). That works only while the id is +unique: add ANY same-stem file -- a ``lead.md`` will do -- and the two file +nodes collide, so ``_disambiguate_colliding_node_ids`` salts them apart into +``lead_ex_lead`` and ``lead_md_lead``. The import edge's target salt is keyed by +the IMPORTER's own source_file, which matches neither, so the edge is left +pointing at an id that no longer names anything and is silently dropped. + +The disambiguator already accepts a ``target_file`` hint for exactly this shape +(#1814), keying the target salt by that file instead. Every language below +stamps it now. + +Each case asserts BOTH directions, so a fixture that never produced an import +edge in the first place cannot pass by accident: the control corpus must +resolve, and the collision corpus must resolve to the SAME file. +""" +from __future__ import annotations + +from pathlib import Path + +import pytest + +from graphify.extract import extract + +_IMPORT_RELATIONS = ("imports", "imports_from", "re_exports") + +# A language-neutral collider: it shares the stem, mints a file node, and has +# nothing whatsoever to do with the import under test. +_COLLIDER = ("lead.md", "# Lead\n\nUnrelated notes.\n") + +# (case id, importer file, importer body, target file, target body) +_CASES = [ + ( + "powershell-dot-source", + "caller.ps1", ". ./lead.ps1\nfunction Run { Search }\n", + "lead.ps1", "function Search { return @() }\n", + ), + ( + "powershell-import-module", + "caller.ps1", "Import-Module ./lead.ps1\nfunction Run { Search }\n", + "lead.ps1", "function Search { return @() }\n", + ), + ( + "rust-use", + "caller.rs", "use crate::lead;\n\npub fn run() { lead::search(); }\n", + "lead.rs", "pub fn search() {}\n", + ), + ( + "pascal-uses", + "caller.pas", + "unit Caller;\ninterface\nuses Lead;\nimplementation\nend.\n", + "lead.pas", + "unit Lead;\ninterface\nprocedure Search;\n" + "implementation\nprocedure Search; begin end;\nend.\n", + ), + ( + "zig-at-import", + "caller.zig", + 'const lead = @import("lead.zig");\npub fn run() void { lead.search(); }\n', + "lead.zig", "pub fn search() void {}\n", + ), + ( + "elixir-import", + "caller.ex", + "defmodule Caller do\n import Lead\n def run, do: search()\nend\n", + "lead.ex", "defmodule Lead do\n def search, do: []\nend\n", + ), + ( + "bash-source", + "caller.sh", "source ./lead.sh\nrun() { search; }\n", + "lead.sh", "search() { echo hi; }\n", + ), +] + + +def _extract(tmp_path: Path, files: list[tuple[str, str]]) -> dict: + tmp_path.mkdir(parents=True, exist_ok=True) + paths = [] + for name, body in files: + path = tmp_path / name + path.write_text(body, encoding="utf-8") + paths.append(path) + return extract(paths, cache_root=tmp_path) + + +def _import_targets(result: dict) -> list[dict]: + """The node each import edge points at, or ``None`` where it dangles.""" + by_id = {node["id"]: node for node in result["nodes"]} + return [ + by_id.get(edge.get("target")) + for edge in result["edges"] + if edge.get("relation") in _IMPORT_RELATIONS + ] + + +@pytest.mark.parametrize( + ("importer", "importer_body", "target", "target_body"), + [case[1:] for case in _CASES], + ids=[case[0] for case in _CASES], +) +def test_import_edge_survives_a_same_stem_foreign_sibling( + tmp_path: Path, importer: str, importer_body: str, target: str, target_body: str, +): + corpus = [(importer, importer_body), (target, target_body)] + + control = _import_targets(_extract(tmp_path / "control", corpus)) + assert any( + node is not None and str(node.get("source_file", "")).endswith(target) + for node in control + ), f"fixture is inert: no import edge reached {target} even without a collider" + + collided = _import_targets(_extract(tmp_path / "collided", [*corpus, _COLLIDER])) + assert None not in collided, \ + "an import edge dangled on the pre-disambiguation stem id" + assert any( + str(node.get("source_file", "")).endswith(target) for node in collided + ), f"the import edge no longer reaches {target}" + assert not any( + str(node.get("source_file", "")).endswith(_COLLIDER[0]) for node in collided + ), "an import edge was repointed onto the unrelated collider" + + +def test_the_transient_target_file_hint_never_reaches_the_graph(tmp_path: Path): + """``target_file`` carries an absolute path and is popped by its only reader. + + Asserted across every case at once: a language that stamps the hint but is + somehow not reached by the disambiguator would ship the analysing machine's + filesystem layout inside `graph.json`. + """ + # Two cases share `caller.ps1`; dedupe by name (last wins). The point is + # breadth of emitters in one graph, not per-case isolation. + corpus = {_COLLIDER[0]: _COLLIDER[1]} + for _, importer, importer_body, target, target_body in _CASES: + corpus[importer] = importer_body + corpus[target] = target_body + result = _extract(tmp_path, list(corpus.items())) + + leaked = [edge for edge in result["edges"] if "target_file" in edge] + assert leaked == [], f"transient hint reached the graph: {leaked[:3]}"