Repository navigation
feat(profile): children per parent for every declared relationship - #1026
Merged
Merged
Conversation
When one call profiles both exposes of a semantics.relationships entry, without --bundle, the profile's existing relationships[] (PROFILE-SPEC section 5, format 1-0-0/1-1-0 unchanged) carries a marginalHistogram/1 with the parent as the unit: each parent row counts once, by its number of child rows, on the bins 0, 1, 2, 3-4, 5-9, 10-19 and 20 or more. The last bin ends at 2^53 - 1, the largest count a profile holds, so a parent with many children is counted where it belongs and no value is cut to fit. - keys are matched as the registry reads both columns, so 7 and "007" agree where the type is a number; a null key, on either side, matches none; - the histogram is formed by the registry's own definition (StatisticsBackend.count_histogram) and checked by check_edge_counts in a call of its own, as flux-stats 0.7.0 documents for a relationship's count of children, then spelled as a column's histogram is; - the child's foreign-key columns withhold their distinctCount, sum, min, max and quantiles (D) and form no mean/1: the parents with a child, beside the parents' row count, give those with none, and the key's own values describe each parent's children key by key; - a relationship whose target is not in the call, declares a validity window, or whose keys are read in different domains, or whose fromColumns and toColumns are not two lists of one length, is named and left out; - the key columns of a counted relationship take no default statistic of their own: a key's level histogram, histogram or quantiles count each key's rows, and the child key's rows are each parent's children; - relationships[] lists them in the contract's order, whatever order the call names the exposes in, so the document's bytes do not depend on it. The output check sees the parents' cells and their total only. A reader can know more: other tables over the same parents (an enum's level histogram, another relationship's cells) bound their number, and the children's own counts bound what they hold, each child row having at most one parent and each parent between its bin's edges. disclosure.pinned_children_cells takes the strongest reader, who knows the numbers of parents, of children and of parents whose key is null exactly, besides the published bands and the check's own rule (the withheld cells never total a small count). A withheld cell stands only where two tables that reader can hold true differ in it, found by moving one parent and, if the children's bounds break, as many more as repair them; the search errs toward withholding. runner._audited_children withholds the published cell of the highest edge until no cell is pinned, down to the whole histogram. The disclosure audit splits a band of children over a band of parents every way, children unbounded and none an orphan, under ACRO's rule: 875 and 4,700 worlds in the suite and up to 144,962 beside it pin nothing, where the check alone left 14 pins in the first. Keyed by the exact numbers of parents and children, up to 7 parents and 14 children pin nothing under every scaled rule, exact counts included, so the exact-count case is closed too. Tests cover an enum over the parents beside the relationship, two relationships into one parent, and enum and number keys. ProfileResult is read through its tables (and relationships) only: the one-table shortcuts kept beside them (plan, statistics, withheld_statistics, partly_withheld_statistics, left_out) go, and the tests read tables[0].
📄 Documentation ReminderThis PR appears to be missing a documentation reference. Our docs live in a separate repo. Please update the PR description with one of:
See the Contributing Guide for details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
fluid profilecounts children per parent for everysemantics.relationshipsentry whose two exposes are profiled in one call, without--bundle. The document's existing top-levelrelationships[](PROFILE-SPEC §5, format unchanged) carries amarginalHistogram/1with the parent as unit, on the bins 0, 1, 2, 3-4, 5-9, 10-19 and 20 or more (the last bin ends at 2^53 - 1, so no count is cut to fit).fromColumnsandtoColumnsare not two lists of one length is named in the summary and left out.distinctCount,sum,min,maxandquantiles(D) and forms nomean/1. The key columns of a counted relationship take no default statistic of their own.This was part of #1024 and moved to its own PR after the pre-merge re-check found a further disclosure channel. It sits on top of 0.36.0.
Disclosure
The output check sees the parents' cells and their total. A reader can know more. Other tables over the same parents (an
enumcolumn's level histogram, another relationship's cells) bound their number. The children's own counts bound what they hold, since each child row has at most one parent and each parent holds between its bin's edges.So
disclosure.pinned_children_cellsaudits every withheld cell against the strongest reader. That reader knows the exact number of parents, of children and of parents whose key is null, plus the published bands and the check's own rule (the withheld cells never total a small count). A withheld cell stands only where two tables that reader can hold true differ in it. The second table is found by moving one parent and, when the children's bounds break, as many more as repair them. The search errs toward withholding.runner._audited_childrenwithholds the published cell of the highest edge until no cell is pinned, down to the whole histogram, which is where parents with few children often land.The review found and closed four channels:
enumkey's level histogram, or a number key's histogram or quantiles, counted each parent's children;Tests cover the re-check's worlds (503 parents, 23 parents, an
enumover the parents beside the relationship, two relationships into one parent),enumand number keys, and the reader's two bounds, each with a test that fails when its clause is removed. With exact counts the children's total is exact too, and the case the earlier audit kept as an expected failure now passes.Reviews
enumkeys and once for number keys. Both are fixed. It gave SHIP at the final commit, with its one LOW folded in.Validation
scripts/mypy_strict.py, import contracts, license headers, the product guardrail, and a secret scan of the touched files. All pass.tests/profiler,tests/fluid_sql, the quality engine's column rules, the lazy command table, bootstrap): 2,616 passed, with only the three failures this shared environment has on main too.Follow-ups
fluxsim initdoes not readrelationships[]yet, and must accept an entry whose only group is withheld.