Repository navigation
Conversation
…ir own QNs
In C-family code several different entities competed for one node, and
some had no node at all:
- A typedef name was never a definition: `type_definition` has no `name`
field, so the class path dropped it. With an anonymous
`typedef struct/enum {...} X;` the fields and enumerators went with it.
- A function behind an unknown leading macro (`API RetT name(...)`) was
named after its return type, became a Method `RetT.name`, or was
dropped, depending on the error-recovery shape.
- Every bodyless `struct X` / `enum X` (a parameter, a field type, a
forward declaration) minted a Class/Enum node. In one file, or in a
same-stem .h/.c pair, such a reference displaced the real definition.
- A `#define NAME` shared the QN of the function, type or enumerator
called NAME, and the later line won: a namespace-rename macro replaced
the typedef it renames, an `#else` stub macro replaced the function.
- Enumerators were named `<scope>.<Enum>.<CONST>`, although C and
unscoped C++ enums put them in the enclosing scope.
- A definition repeated in several `#if` branches kept one span; the
others left no trace.
- Braces split across `#if` branches lost whole functions, and a C
keyword could surface as a function name (`if`).
Extraction:
- typedef names are definitions: `Type`, or `Class`/`Enum` with members
for an anonymous `typedef struct/enum {...} X;`; an alias is dropped
when the same file defines the tag of that name
- macro-prefixed functions get their real name in all three recovery
shapes, consistently for defs, call scopes, parameters and the C LSP
- a bodyless tag is a reference and creates no node; a definition head
left as loose ERROR tokens (`struct NAME {`) is recovered
- a C keyword is never a function name
- a first-branch projection (one branch per `#if` group, positions
unchanged) is parsed when the raw parse lost structure wholesale, and
only functions missing from the raw result are adopted from it
- macro-wrapped enumerator lists yield one constant per slot
Identity (semantic index version 4; an existing index is rebuilt once):
- a C-preprocessor macro's QN is `<module>.<NAME>#macro` (C, C++, CUDA,
GLSL, Objective-C, ISPC); name and label are unchanged. Resolvers
prefer a definition and use the macro only when no definition of that
name is visible (LSP target lookup, trace_path, get_code_snippet)
- enumerators of C enums and of unscoped C++/Objective-C enums are
`<enclosing scope>.<CONST>`; `enum class` stays nested; `parent_class`
names the enum
- a node whose file defines its QN more than once under one label
carries `variants`, the line spans of all those definitions (`#if`
twins, per-platform macro redefinitions, C++ overloads)
Full-text search indexes a macro's QN without the fence, so macro rows
score as before.
Measured on redis and curl against the parent commit: nodes 38,625 ->
40,484 and 28,180 -> 29,750; Type nodes 0 -> 573 and 0 -> 193; phantom
Class nodes from references 202 and 821 removed; functions lost to a
same-named macro 45 and 41 restored; curl functions with no node 11 ->
0; nodes of non-C-family files unchanged by QN. Of 174 audited Doxygen
references in xxhash, 174 now reach the right entity (64 before). Index
time is unchanged within noise (+2% pipeline time for 5% more nodes).
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Isolate the watcher-disabled runtime, use a deterministic Python responder for the fuzz harness environment probe, and align the version metadata contract with the current Scoop manifest. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…-enumerators Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> # Conflicts: # internal/cbm/cbm.h # src/mcp/mcp.c
cppcheck 2.20 (the CI linter) reports variableScope for the count taken before the definitions walk, which only the C-family branch uses. The branch now takes the count, walks and drops the shadowed typedefs; the other languages walk as before. Behaviour unchanged: extraction and pipeline suites pass. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Conflicts: internal/cbm/extract_defs.c includes (union of both sides); src/pipeline/registry.c keeps main's lang-aware index_under_name call, and the fence comment combines both facts (Rust cfg twins are variants of one QN; a C macro's #macro QN never reaches the registry). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Main (#2458) named the anonymous struct/union/enum of `typedef struct { ... } Name;` in extract_class_def from the typedef's declarator. This branch's extract_c_typedef already emits that aggregate as Name from the type_definition, so on the merged tree Name was emitted twice under one label: its members twice and a false variants entry. extract_c_typedef stays the one place that names it; the extract_class_def case is removed. Both typedef test sets pass on the result: #2458's extract_c_anonymous_typedef_aggregate_is_named_by_its_typedef and member-access tests, and this branch's C1 typedef/variant tests (extraction, pipeline, registry, graph_buffer, store_search, mcp: 1,336 passed, 4 skipped). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Regression test for the merge with #2458: with its extract_class_def case restored, typedef struct { int count; } Tally; yields two Class defs named Tally (count_defs_named == 2, RED at test_extraction.c:1129); with the case removed it is one Class, one count field, one Shade enum, and no variants list (extraction 432 passed). Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Main's graph_buffer.c ("Definition variants") records a definition's
variant spans as [{"file_path","start_line","end_line"}], and the
test-impact engine reads that schema to find every TEST(...) branch of one
test. This branch's extractor wrote [{"start","end"}]. On the merge with
main, graph_buffer's same-QN merge found a non-empty variants list it could
not read and dropped every span, the definition's own included. The engine
then no longer found the second #if branch of TEST(alpha_uses_lib), so
test_impact_engine_variant_tests_are_one_test reported suite alpha
UNMAPPED, in CI on macOS and Windows and locally.
The extractor now writes the shared schema, with the file path
JSON-escaped. The C-preprocessor-only gate and the one-carrier rule are
unchanged, and the variants assertions use the shared format.
test_impact_engine, extraction, pipeline, graph_buffer and
conditional_variants: 857 passed.
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
…last-line rule Merging main brought #2340's virtual_newline field in CBMStringInput. cbm_rescue_defs_from_projection still built its input with the two-field initializer, a missing-field-initializers error under -Werror. The projection keeps the raw source's length and line structure, so it now goes through cbm_parse_source like the raw parse and the C++ branch views: the same virtual newline, and the same edit that removes it again. extraction pipeline graph_buffer store_search mcp registry edge_types_probe parse_coverage c_lsp test_impact_engine conditional_variants on the merge result: 2277 passed, 4 skipped. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
In C-family code several entities competed for one graph node, and some had no node at all. This PR fixes the extraction defects behind that and gives macros and enumerators identities of their own.
What was wrong
typedef struct/enum {...} X;the fields and enumerators were lost too.type_definitionhas nonamefield, so the class path dropped it.API RetT name(...)was named after its return type, became a MethodRetT.name, or vanished.struct X/enum Xminted a Class/Enum node. In one file, or in a same-stem.h/.cpair, a reference displaced the definition.#define NAMEtook the node of the function, type or enumerator calledNAME.<scope>.<Enum>.<CONST>.#ifbranches kept one span.#ifbranches split a brace; a C keyword surfaced as a function namedif.What changes
Extraction
Type, orClass/Enumwith members for an anonymoustypedef struct/enum {...} X;. An alias is dropped when the same file defines the tag of that name.struct NAME {) is recovered, spanning to its matching brace.#ifgroup and every byte position unchanged. Only functions the raw pass lost are adopted, under the same gates as the existing preprocessed rescue. Sampled on the Linux kernel this re-parses about 1% of C bytes.CURLOPT(CURLOPT_URL, ...)) yield one constant per slot instead of one per identifier.Identity
<module>.<NAME>#macro(C, C++, CUDA, GLSL, Objective-C, ISPC).nameand theMacrolabel are unchanged. The#fence is the convention the Rust#[cfg]twins already use. Whatever resolves a name or a plain QN prefers a definition and uses the macro only when no definition of that name is visible.<enclosing scope>.<CONST>;enum class/enum structstay<Enum>.<CONST>. In C and Objective-C an enum declared inside a struct or union puts its constants at file scope, as the language does; in C++ and CUDA the class is the scope. The existingparent_classproperty names the enum:MATCH (v:Variable) WHERE v.parent_class = '<Enum QN>' RETURN v.name.variants: a node whose file defines its QN more than once under one label carries the line spans of all those definitions, its own included:"variants":[{"start":30,"end":30},{"start":182,"end":227}]. That covers#iftwins, macros redefined per platform, and C++ overloads. C-preprocessor languages only; never truncated; definitions in other files are not listed.Consumers adapted
trace_pathname resolution andget_code_snippetapply the definition-before-macro rule;get_code_snippetstill finds a macro by its old QN and by its short name.detect_changes,check_index_coverage.Upgrade note (for the release notes)
The index is rebuilt in full once after upgrading (semantic index version 4). In C-family code a preprocessor macro's qualified name now ends in
#macro(mod.NAME#macro), the constants of C enums and unscoped C++/Objective-C enums arescope.CONSTinstead ofscope.Enum.CONST(their enum is in theparent_classproperty), and typedef names are nodes. Names are unchanged. An exact macro QN, or aqn_patternending in\.NAME$, must become\.NAME#macro$.Measured effect
redis (
4f20cb48) and curl (c345611d), parent commit vs this change.#definenames with a nodevariantsvariants,parent_class, docstring and signature.variantsandparent_classas a fresh full index.Tests
28 new tests in
extraction(16),pipeline(7),mcp(2),graph_buffer(2) andstore_search(1), each failing without the change on a behaviour assertion. Targeted reverts were checked one cause at a time (version constant, LSP macro retry, MCP macro rank, snippet tier, slot rule, each serialisation path, the language gate, the C struct-scope rule and its C++ counterpart).Verification
Local ladder on the final tree (both commits):
scripts/test.shscripts/test.shscripts/test.shgrammar_probe_dprinted its first test name and then nothing until the runner stopped it at 900 s. The same suite passed in this VM on the C1 commit on 2026-10-02, and the other six grammar-probe suites passed in this run in 66 to 205 s, so it is recorded as an unattributed hang of that run, not as a C1 failure; the hosted Windows CI is the gate. (An earlier attempt of this leg stopped on a scheduler infrastructure error before the suites.)main(2026-10-04)qn_sig_offfield of #2061 next tovariants; the macro tier placed afterget_code_snippet's new definition tiers, so a definition still wins over a macro of the same name). On the merged tree: extraction, pipeline, mcp, graph_buffer, store_search, registry 1,294 passed, 4 skipped, 0 failed.make lint-ci(cppcheck, clang-format)Known limits (not changed here)
variantslists what extraction finds. Two of curl's 123 groups are short because one variant's head is itself lost to an error region.#iftwins is still the last by start line;variantsnow exposes the other spans.#defines inside function, struct and enum bodies have no node, as before.Follow-ups (not in this PR)
.hand its same-stem.cshare one module QN, which is the root of the remaining cross-file collisions.unique_namefallback binds calls from other languages to C struct fields.detect_changescould usevariantsto seed a node from a change inside another variant's span.