diff --git a/research/p104-main46-hop-merge/EXPECTED.json b/research/p104-main46-hop-merge/EXPECTED.json new file mode 100644 index 0000000000..a9811b577c --- /dev/null +++ b/research/p104-main46-hop-merge/EXPECTED.json @@ -0,0 +1 @@ +{"kappa_26": "0.003650971539360618360754"} diff --git a/research/p104-main46-hop-merge/NOTICE b/research/p104-main46-hop-merge/NOTICE new file mode 100644 index 0000000000..8d04d8fc38 --- /dev/null +++ b/research/p104-main46-hop-merge/NOTICE @@ -0,0 +1,14 @@ +Apache-2.0. DreamingOfClouds with Claude Opus 5.5 (Anthropic), via Claude Code. + +data/base-7608-word.json.gz and data/invoice-7608.json are the unmodified +research/p104-dual-scan-drop-902/data/word.json.gz and data/invoice-new.json +at #7608 head 24b082992 (our own package; Apache-2.0). #7608 is itself +aliromman's #7605 (Ali Romman, with OpenAI Codex assistance; Apache-2.0, +research/p104-timed-helper757) with one exact edit. Credit aliromman for +#7605, Hao Yan (FanPUYen) for #7604, SYLforge for #7603, ToikY for #7546 and +the native, price-engine and replay lineage, and our own #7608, #7601 and +#7542 beneath them, on Clear1nsight's #7334 and #7314, with Rohan Arun's +signed #7113 caller and all contributors named in those packages. + +check.py adapts the replay checks of our #7608 package; it imports no other +package and none of the retained authors' code was run. diff --git a/research/p104-main46-hop-merge/README.md b/research/p104-main46-hop-merge/README.md new file mode 100644 index 0000000000..38c9c1737d --- /dev/null +++ b/research/p104-main46-hop-merge/README.md @@ -0,0 +1,108 @@ +# CONDITIONAL κ = 0.00365097153936 — hosted main 46 merges its final unit hop on #7608 (premise: #7608's CLEAR #44 content) + +This P104 word is the word of our +[#7608](https://github.com/CrocSwap/integer-mult-bounds/pull/7608) (head +`24b082992`, `research/p104-dual-scan-drop-902/data/word.json.gz`, SHA256 +`37dc87b2…9118`, κ 0.00365074956598) with one exact edit. #7608 is aliromman's +[#7605](https://github.com/CrocSwap/integer-mult-bounds/pull/7605) with one +edit; the chain below it is FanPUYen's #7604, SYLforge's #7603, ToikY's #7546 +and our #7601 (see [NOTICE](NOTICE)). Result: **κ = 0.0036509715393606183607545417…, +×1.0000608 (+0.00608%) over #7608**. + +> **Premise (inherited, nothing new).** The claim is conditional on exactly +> the premise of #7608: owned CLEAR #44 (owned field 1557, chart 595) erases +> `{451:−1, 1495:1}` and `owned_erasure` must let a CLEAR discard that row. This +> word's CLEAR content is identical to #7608's at all 74 CLEARs, so it adds no +> premise. If #7608's premise is refused, this word is void with #7608. + +## The edit + +**Main 46** goes 914→127 in one MOVE, where #7608 has 914→245→127 (dimensions +7→9 instead of 7→8→9). Net +10 events per native call (one MOVE fewer, 11 ADDs +more): +- `MOVE 46 914→245`, `ADD 1456 += 46 @245`, `ADD 588 += −46 @245` and + `MOVE 46 245→127` are deleted, and `MOVE 46 914→127` is inserted in place + of the last; +- two corrections at chart 245: `588 += 818`, `1456 += −818`; +- 11 premix ADDs at chart 3 (frame {0}) at the start of the program: a 3-ADD + 588/1456 shear, then `588 += −(228, 434, 443)`, `588 += 444` with the paired + `1456` updates. + +[data/edits.json](data/edits.json) lists every event by #7608 index. The pair +914→127 is already used by #7608, so there is no new chart pair. All 1,569 final +rows, all end charts, the owned MOVEs (47), the 74 owned lifetimes, the peak +(50), the header, the signed #7113 caller data and the coefficient grid +(2^25, odd denominator 1125, 40 bits) are #7608's; only main 46's MOVE chain +and the ADDs above differ. + +## Price + +The complete invoice is #7608's with the native delta charged 17 times +(native paid delta {1:−2, 2:+1}): +- paid positions 110,349 → 110,332; productive positions 22,461; +- rank surplus S = 1,424 unchanged; zero empty-root credits; +- D0 392,993.2743 → 392,969.7073 (−23.57). + +The crossing of Σ(T−P)·r^(1−C) is **κ = 0.0036509715393606183607545417…**, +up from #7608's 0.0036507495659836771294008126936…. + +Official tools (`price.py`): +- `scripts/compare_recursive_invoices.py` classifies the change against #7608 + as `uniformly_better` (strictly negative for r^(1−C)), paid {1:−34, 2:+17}, + targets unchanged. +- `scripts/evaluate_recursive_power.py` (256 bits) gives `new_cost_lower` at + C = 0.0036509715393 and at #7608's 0.003650749565983, and `new_cost_higher` + at C = 0.0036509715394. + +## Checks + +```sh +python3.12 -B scripts/research_package.py --package research/p104-main46-hop-merge +``` + +- `check.py` (standard library, about 2 minutes) does the following: + - authenticates the vendored #7608 word and rebuilds the new word from it + and the edit list; + - replays both words over the rationals from arbitrary inputs, with the + legality checks listed in its docstring; + - requires the header, every row and end chart, the owned MOVEs, lifetimes + and peak to equal #7608's, and that only main 46 changes its MOVE chain; + - requires all 74 CLEARs to erase #7608's content (CLEAR #44 asserted as + `{451:−1, 1495:1}`); + - recomputes the census, invoice and κ, and reproduces #7608's crossing; + - runs eight rejecting controls (premix coefficient changed, premix ADD + dropped, correction dropped, the merge without premixes or corrections, + correction at the wrong chart, premix at the wrong chart, a reversed MOVE, + a MOVE from the wrong chart) and a tampered invoice. +- `price.py` runs the two repository tools above. + +The strictification gate (the repository's separation check) was run offline +by us and is not part of this package: γ 0.940941, δ 0.908149, both below +31/32 (#7608: γ 0.940932, δ 0.908149). + +## Conditional premises and what is not rerun + +On top of #7608's CLEAR #44 premise, the claim carries the framework +assumptions of #7605 (`native_all_columns`, `owned_erasure`, `compiler`, +`precision_tape`, `analytic_recovery`), the signed #7113 caller, and the BIT +premise of the base. This package does not rerun the signed caller, outer +predicates, precision/tape counts or ordinary bill (header, rows, grid and +owned data are unchanged). An independent read-only audit found the word sound +relative to #7608. No upstream maintainer has reviewed the package. + +## Method + +Single-stay drop scan (#7601's token-settled engine): merge two adjacent hops +of a main, then repair the changed rows with an exact linear premix. The one +merge that settles exactly was rebuilt and replayed on #7608's word. + +## Open next questions + +**The premise.** A ruling on `owned_erasure` (clear-form-gap Q1) decides #7608 and +this word together. See [TASK.json](TASK.json). + +## Credit + +The base word and all retained work are #7608, aliromman's #7605 and the +authors it credits (see [NOTICE](NOTICE)). DreamingOfClouds with Claude Opus 5.5 +(Anthropic), via Claude Code. Apache-2.0. diff --git a/research/p104-main46-hop-merge/SOURCES.json b/research/p104-main46-hop-merge/SOURCES.json new file mode 100644 index 0000000000..c484f02b5a --- /dev/null +++ b/research/p104-main46-hop-merge/SOURCES.json @@ -0,0 +1,30 @@ +{ + "schema_version": 1, + "base_commit": "e54523517d0f03e58352e647c403a5af3e525a57", + "files": [ + { + "path": "certificates/selected-result.json", + "usage": "provenance", + "git_blob": "75d77b328ea02ded535e7f23aee5b1e55d0bb1a8", + "sha256": "154c3d30455d778fd3343b6b4851f0fa4665f0f7625b0adb3675e2bafbbf6bf1" + }, + { + "path": "docs/research/agent-task-catalog.json", + "usage": "provenance", + "git_blob": "3404fbdd5e7529a0674eef26114b5071e4586cec", + "sha256": "ead6cdc9d0af6d58e8a49744d8ec8c7c15e13d2ba93c276c434bcf9b51b93a81" + }, + { + "path": "scripts/compare_recursive_invoices.py", + "usage": "input", + "git_blob": "fda2a852e9937e42fe40df0707694465ca7c1b48", + "sha256": "3930c22c11da7bc6042cfeb6e459ecb06e4478430037bee3469d034353855fc6" + }, + { + "path": "scripts/evaluate_recursive_power.py", + "usage": "input", + "git_blob": "ca50f8021fc3238a58fd420383f4619a8d008ace", + "sha256": "52b20f41fd0bb55fa88dccd6015af2ce358917817b9506720f3e60ce681cc3ad" + } + ] +} diff --git a/research/p104-main46-hop-merge/TASK.json b/research/p104-main46-hop-merge/TASK.json new file mode 100644 index 0000000000..cf93d22c40 --- /dev/null +++ b/research/p104-main46-hop-merge/TASK.json @@ -0,0 +1,117 @@ +{ + "schema_version": 1, + "task_id": "p104-side-correction", + "priority_revision": 8, + "base_commit": "e54523517d0f03e58352e647c403a5af3e525a57", + "original_card": { + "id": "p104-side-correction", + "question": "Can joint helper bases and a legal causal split improve the complete P104 productive/paid invoice?", + "starting_points": [ + "docs/research/current-priorities.md", + "docs/research/research-family-status.md", + "certificates/selected-result.json", + "docs/research/pr-intake-evidence/pr5599-completion/PROOF-REVIEW.md", + "research/p104-helper-subspace-splits/README.md", + "research/dyadic-helper-channel-capacity/README.md" + ], + "methods": [ + "Construct and independently check one source-bound instance of the stated first experiment.", + "Seek an exact counterexample or target-specific complete-budget obstruction to that same proposed change." + ], + "acceptance_gate": "An exact arbitrary-input identity, source/target frame availability and restoration at the chosen cut, plus the complete channel+correction+idle/hosting bill or an explicit favorable bound. The native channel alone is not a supplier. A scoped local result may be retained without a complete supplier. Expanding it into further construction rounds requires the family-level budget and obligation gate in research-family-status.md; an optimistic passing relaxation alone is not acceptance.", + "first_step": { + "command": [ + "python3", + "-B", + "scripts/research_selected_baseline.py" + ], + "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY", + "inspect": "Read revision 8, the selected review and live exact-source handoffs. Then follow this card's bounded first_experiment.", + "scope": "Source-bound orientation only, not a supplier replay or theorem proof." + }, + "first_experiment": "Start with the selected #5599 emitted word and review pending #5718 distinct input/output bases plus #5717/#5719 coverage before claiming work. Pick one uncovered source word, helper group and cut region. Prove the arbitrary-input relation and legal entry/exit corrections, then compare full target and paid profiles. Reject if completion, owned lifetimes or consumer costs repay the gain. A static pairing rank alone is not a physical split.", + "bottleneck_link": "Judge the actual current supplier contract and complete signed target/paid bill; frozen-profile ceilings require an applicability proof.", + "allowed_variations": [ + "One exact source head, helper/cut region or quantified proposition and changed hypothesis; check live handoffs first. Different seeds are not different scopes." + ], + "scope_change_triggers": [ + "Changing source head, helper/cut domain, operation alphabet or caller boundary without recording the new scope.", + "Treating an archived proposal or local numerical pass as complete supplier admission." + ], + "overlap_terms": [ + "P104", + "helper basis", + "helper split", + "5718", + "dyadic capacity" + ], + "existing_checks": [ + { + "scope": "PR #4133: literal-slot defect", + "evidence": "research/p104-literal-slot-defect/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "88666f1dee93fbd85b31fd5d6f91a219e111b5f4", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4136: uniform commutation gap", + "evidence": "research/p104-uniform-commutation-gap/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "281ce601a8be9db470de630759179c23e1dfaa88", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4140: uniform encoding gap", + "evidence": "research/p104-uniform-encoding-gap/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "de25c6525aa866962309df5d107b32d55e6e5628", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4153: source-chart shortcut exhausts its optimistic allowance", + "evidence": "research/p104-source-chart-shortcut/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "9123241fea53df77e51ffe30d9aca1625d4bf3a8", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4154: fiber-tag seam mismatch", + "evidence": "research/p104-fiber-tag-seam/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "4ffcdaee5d6255a9cea1fe53a5424435068089d0", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "Selected PR5599: complete helper-subspace construction and independent caller replay; five framework assumptions remain.", + "evidence": "docs/research/pr-intake-evidence/pr5599-completion/PROOF-REVIEW.md", + "remaining": "The next experiment must identify uncovered source-specific work. Successor archive intake does not transfer this admission.", + "status": "reviewed", + "source_commit": "09dd0965711ca7eeb169057a420abab1c9052137", + "selected_record_sha256": "2a0220e866417774606e0f991e41b9c1d26b1f43b97857238cf478dbc7d330b5" + } + ] + }, + "scope_status": "refined", + "actual_scope": "Exact #7605 word, public main 902 only: it climbs 3->618 directly instead of 3->595->618 (five events deleted, the F595 stay removed); rows, end charts, owned MOVEs, lifetimes, header and signed #7113 caller data are #7605's. Conditional: the content erased at owned CLEAR #44 (owned 1557, chart 595) differs from #7605's.", + "scope_change_reason": "The card asks for joint helper bases and causal splits; this is a narrower exact chain shortening on one main of the current word, found by running the token-settled drop scan on the frame-perp dual of the word. It changes one erased CLEAR row, so it is conditional on owned_erasure admitting that row.", + "bottleneck_link": "Reduces the complete signed paid invoice (17 rank-1 and 17 rank-2 jobs become 17 rank-3 jobs) without changing the productive profile.", + "budget_minutes": 90, + "stop_rule": "One literal word with exact arbitrary-input row equality, legality, the single changed erased row stated and asserted, complete invoice and a certified crossing.", + "outcome": "completed", + "evidence": "README.md, data/edits.json, check.py, price.py, VERIFY.json.", + "author": "DreamingOfClouds", + "ai_assistance": "Claude Opus 5.5 (Anthropic), via Claude Code.", + "open_questions": [ + { + "question": "Does owned_erasure admit owned CLEAR #44 (field 1557, chart 595) discarding {451:-1, 1495:1} instead of {451:-1, 902:-3/2, 985:-3/2, 1495:1}? (research/p104-6372-clear-form-gap, Q1)", + "why_it_matters": "The claimed kappa holds only if it does; if not, #7605's kappa stands and this word is void.", + "first_step": "Run check.py and read the PREMISE line; compare it with the erased-row form the contract allows.", + "blocked_by": "A ruling on owned_erasure; no zero-move repair exists (an exact search over extra ADDs among registers live at chart 595 is inconsistent), a repair needs a new MOVE or an extra owned lifetime." + } + ] +} diff --git a/research/p104-main46-hop-merge/VERIFY.json b/research/p104-main46-hop-merge/VERIFY.json new file mode 100644 index 0000000000..0efa452557 --- /dev/null +++ b/research/p104-main46-hop-merge/VERIFY.json @@ -0,0 +1,32 @@ +{ + "schema_version": 1, + "status": "ready", + "python": "3.12", + "requirements": [], + "inputs": [ + "scripts/compare_recursive_invoices.py", + "scripts/evaluate_recursive_power.py" + ], + "commands": [ + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/check.py" + ], + "timeout_seconds": 300, + "checks": "Vendored #7608 word bytes; word = #7608 + edit list; exact arbitrary-input replay and legality of both words; header identical; all 1,569 rows and end charts equal; owned MOVEs, lifetimes, peak unchanged; all 74 CLEARs erase #7608 content (CLEAR #44 row asserted as {451:-1, 1495:1}); only main 46 changes chain; no new chart pair; #7608 crossing reproduced; new invoice and kappa 0.003650971539360618360754...; eight rejecting controls. Prints PASS_MAIN46_HOP_MERGE_EXACT." + }, + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/price.py" + ], + "timeout_seconds": 120, + "checks": "Repository tools: uniformly_better against #7608; new_cost_lower at C=0.0036509715393 and at #7608's 0.003650749565983; new_cost_higher at C=0.0036509715394. Prints PASS_PRICE." + } + ] +} diff --git a/research/p104-main46-hop-merge/check.py b/research/p104-main46-hop-merge/check.py new file mode 100644 index 0000000000..cfa728c6c0 --- /dev/null +++ b/research/p104-main46-hop-merge/check.py @@ -0,0 +1,354 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Exact replay of the main-46 hop merge against #7608's word. + +Standard library only. Written for this package; it imports no other package. + +1. The vendored #7608 word has the SHA256 of research/p104-dual-scan-drop-902/ + data/word.json.gz at #7608 head 24b082992, and applying data/edits.json + to it gives data/word.json.gz event for event. +2. Both words are replayed over the rationals from arbitrary inputs (every + public field starts as its own symbol, owned fields start at INIT as 0). + Legality: every ADD/COPY/SCALE operand sits at the event's chart, every + MOVE starts at the register's current chart, public MOVEs strictly ascend + (span inclusion), owned MOVEs are nested and only on live owned fields, + COPY writes a zero field, no SCALE by 0, no self-ADD, every owned field + is cleared at the chart it sits in, and public fields end at `final`. +3. Header identical; all 1,569 rows and end charts equal #7608's; helpers end + at identity, owned rows end empty; owned MOVEs, lifetimes and peak are + unchanged. +4. The content erased at all 74 owned CLEARs equals #7608's exactly (so the + only premise is #7608's: CLEAR #44, owned 1557, erases {451:-1, 1495:1}). +5. Only main 46 changes its MOVE chain (914->245->127 becomes 914->127). +6. Census, complete invoice (17 native calls) and kappa are recomputed and + compared with data/invoice-*.json; #7608's crossing is reproduced. +7. Controls: corrupted words and a tampered invoice must be rejected. +""" +import collections +import copy +import decimal +import gzip +import hashlib +import json +import sys +from fractions import Fraction as Q +from pathlib import Path + +HERE = Path(__file__).resolve().parent +DATA = HERE / 'data' +BASE_SHA256 = '37dc87b2e4f22a730e3e0c0f85eec6f57f12d1279160435a8d09eb7033a49118' +CALLS = 17 + + +def require(cond, label): + if not cond: + raise AssertionError(label) + + +def load_gz(path): + return json.loads(gzip.decompress(path.read_bytes())) + + +def span(vectors): + s = {0} + for v in vectors: + require(type(v) is int and 0 < v < 512 and v not in s, 'frame vectors independent in F2^9') + s |= {x ^ v for x in s} + return frozenset(s) + + +def mains(W): + live = set(W['live']) + return live | {r + 112 for r in live} | set(W['hosted']['X2']) | set(W['hosted']['Y2']) + + +def apply_edits(program, E): + require(len(program) == E['base_events'], 'edit list base length') + dele = {} + for i, ev in E['delete']: + require(program[i] == ev, 'deleted event %d matches' % i) + dele[i] = ev + ins = collections.defaultdict(list) + for i, evs in E['insert_before']: + ins[i].extend(evs) + out = [] + for i, ev in enumerate(program): + out.extend(ins.get(i, [])) + if i not in dele: + out.append(ev) + out.extend(ins.get(len(program), [])) + return out + E['append'] + + +def replay(W): + """Exact replay; returns rows, end charts, MOVE chains and owned data. Raises on any illegality.""" + n, N = W['public'], W['public'] + W['owned_fields'] + sp = [span(f) for f in W['frames']] + rows = [{r: Q(1)} if r < n else {} for r in range(N)] + cur = list(W['initial']) + [None] * W['owned_fields'] + alive, chains, owned_moves = set(), collections.defaultdict(list), collections.Counter() + lifetimes, peak = collections.Counter(), 0 + init_chart = {} + stats = dict(max_den_exp=0, odd_den=1, max_num_bits=0) + + def note(q): + d = q.denominator + e = (d & -d).bit_length() - 1 + stats['max_den_exp'] = max(stats['max_den_exp'], e) + odd = d >> e + if odd > 1 and stats['odd_den'] % odd: + stats['odd_den'] = stats['odd_den'] * odd // __import__('math').gcd(stats['odd_den'], odd) + stats['max_num_bits'] = max(stats['max_num_bits'], abs(q.numerator).bit_length()) + + for i, e in enumerate(W['program']): + op, r = e[0], e[1] + require(type(r) is int and 0 <= r < N, '%d register index' % i) + if op == 'MOVE': + a, b, flag = e[2], e[3], e[4] + require(type(flag) is bool and flag == (r >= n), '%d MOVE ownership flag' % i) + require(cur[r] == a, '%d MOVE from the current chart' % i) + if r >= n: + require(r in alive, '%d owned MOVE on a live field' % i) + require(sp[a] < sp[b] or sp[b] < sp[a], '%d owned MOVE nested' % i) + owned_moves[(a, b)] += 1 + else: + require(sp[a] < sp[b], '%d public MOVE strictly ascends' % i) + cur[r] = b + chains[r].append((a, b)) + elif op == 'INIT': + require(r >= n and r not in alive and len(e) == 3, '%d INIT of a dead owned field' % i) + alive.add(r); rows[r] = {}; cur[r] = e[2]; init_chart[r] = e[2] + peak = max(peak, len(alive)) + elif op == 'CLEAR': + require(r >= n and r in alive and cur[r] == e[2], '%d CLEAR at the current chart' % i) + lifetimes[(r, init_chart.pop(r), e[2])] += 1 + alive.discard(r); rows[r] = {}; cur[r] = None + elif op == 'SCALE': + q = Q(e[2]) + require(q != 0, '%d SCALE by zero' % i) + require(cur[r] == e[3] and (r < n or r in alive), '%d SCALE chart' % i) + rows[r] = {k: v * q for k, v in rows[r].items()} + for v in rows[r].values(): + note(v) + elif op in ('ADD', 'COPY'): + s, q, f = e[2], Q(e[3]), e[4] + require(type(s) is int and 0 <= s < N and s != r, '%d operand' % i) + require(q != 0, '%d zero coefficient' % i) + require(cur[r] == f and cur[s] == f, '%d ADD/COPY operands at chart %d' % (i, f)) + for x in (r, s): + require(x < n or x in alive, '%d owned field live' % i) + if op == 'COPY': + require(not rows[r], '%d COPY into a zero field' % i) + R = rows[r] + for k, v in rows[s].items(): + nv = R.get(k, 0) + q * v + if nv: + R[k] = nv + note(nv) + else: + R.pop(k, None) + else: + raise AssertionError('%d unknown op %r' % (i, op)) + require(not alive, 'every owned field cleared') + require(cur[:n] == list(W['final']), 'public fields end at final charts') + return dict(rows=rows, cur=cur, chains=chains, owned_moves=owned_moves, + lifetimes=lifetimes, peak=peak, sp=sp, stats=stats) + + +def census(W, R): + """Per-native paid MOVE ranks and helper targets (the #7546 accounting).""" + dim = [len(f) for f in W['frames']] + paid = collections.Counter(abs(dim[e[3]] - dim[e[2]]) for e in W['program'] if e[0] == 'MOVE') + M, n = mains(W), W['public'] + helpers = collections.Counter(dim[W['final'][r]] - dim[W['initial'][r]] for r in range(n) if r not in M) + rises = collections.Counter(dim[e[3]] - dim[e[2]] for e in W['program'] + if e[0] == 'MOVE' and e[1] >= n and dim[e[3]] > dim[e[2]]) + hist = lambda c: {str(k): v for k, v in sorted(c.items()) if v} + return dict(events=len(W['program']), public=n, helpers=hist(helpers), paid=hist(paid), + lifetimes=sum(R['lifetimes'].values()), owned_peak=R['peak'], rises=hist(rises)) + + +def crossing(T, P, digits=40): + """Root of F(C) = sum (T_r - P_r) r^(1-C) on [0.003, 0.0045] by bisection in 80-digit decimals.""" + D = decimal.Decimal + with decimal.localcontext(decimal.Context(prec=digits + 40)) as ctx: + ranks = sorted({int(r) for r in T} | {int(r) for r in P}) + terms = [(D(T.get(str(r), 0) - P.get(str(r), 0)) * r, D(r).ln()) for r in ranks] + + def F(C): + return sum((c * (-C * l).exp() for c, l in terms), D(0)) + lo, hi = D('0.003'), D('0.0045') + require(F(lo) > 0 > F(hi), 'crossing bracketed') + for _ in range(160): + mid = (lo + hi) / 2 + if F(mid) > 0: + lo = mid + else: + hi = mid + return decimal.Context(prec=digits).create_decimal(lo) + + +def invoice_json(P, T): + return {'schema': 'integer-mult-recursive-invoice/v1', + 'contract': {'model': 'mixed-target-unnormalized-power-v1', + 'rank_semantics': 'H_r costs r^(1-C); debit surrendered productive fields', + 'ambient_rank': 145, + 'boundary': 'complete signed #7113 caller application with 17 native calls'}, + 'paid': {k: P[k] for k in sorted(P, key=int)}, + 'targets': {k: T[k] for k in sorted(T, key=int)}} + + +def clears(W): + """Row content erased at every owned CLEAR, in program order.""" + n = W['public'] + rows = [{r: Q(1)} if r < n else {} for r in range(n + W['owned_fields'])] + out = [] + for e in W['program']: + op, r = e[0], e[1] + if op == 'INIT': + rows[r] = {} + elif op == 'CLEAR': + out.append((r, tuple(sorted(rows[r].items())))) + rows[r] = {} + elif op == 'SCALE': + rows[r] = {k: v * Q(e[2]) for k, v in rows[r].items()} + elif op in ('ADD', 'COPY'): + R = rows[r] + for k, v in rows[e[2]].items(): + nv = R.get(k, 0) + Q(e[3]) * v + if nv: + R[k] = nv + else: + R.pop(k, None) + return out + + +def compare(B, RB, W): + """Everything that must hold between #7608's word B and a candidate W, except the erased-content premise. + Returns the candidate replay, the changed chains and the list of differing CLEARs (index, base, new).""" + require(all(B[k] == W[k] for k in B if k != 'program') and B.keys() == W.keys(), 'header identical') + R = replay(W) + n = W['public'] + bad = [r for r in range(len(R['rows'])) if R['rows'][r] != RB['rows'][r]] + require(not bad, 'all rows equal #7608 (first differing %s)' % bad[:3]) + require(R['cur'] == RB['cur'], 'all end charts equal #7608') + M = mains(W) + require(all(R['rows'][r] == {r: Q(1)} for r in range(n) if r not in M), 'helpers end at identity') + require(all(not R['rows'][r] for r in range(n, len(R['rows']))), 'owned rows end empty') + require(R['owned_moves'] == RB['owned_moves'], 'owned MOVE multiset unchanged') + require(R['lifetimes'] == RB['lifetimes'] and R['peak'] == RB['peak'], 'owned lifetimes and peak unchanged') + cb, cw = clears(B), clears(W) + require(len(cb) == len(cw), 'same number of CLEARs') + diff = [(i, x, y) for i, (x, y) in enumerate(zip(cb, cw)) if x != y] + require(not diff, 'CLEAR content identical to #7608 (differs at %s)' % [d[0] for d in diff[:5]]) + changed = sorted(r for r in set(R['chains']) | set(RB['chains']) if R['chains'][r] != RB['chains'][r]) + require(changed == [46], 'only main 46 changes its chain (got %s)' % changed[:5]) + require(all(r in M for r in changed), 'changed registers are mains') + for r in changed: + old, new = RB['chains'][r], R['chains'][r] + require(old[0][0] == new[0][0] and old[-1][1] == new[-1][1], 'register %d chain endpoints' % r) + return R, changed, diff + + +def main(): + base_raw = (DATA / 'base-7608-word.json.gz').read_bytes() + require(hashlib.sha256(base_raw).hexdigest() == BASE_SHA256, 'vendored #7608 word bytes') + B = json.loads(gzip.decompress(base_raw)) + W = load_gz(DATA / 'word.json.gz') + E = json.loads((DATA / 'edits.json').read_text()) + require(apply_edits(B['program'], E) == W['program'], 'word = #7608 word + edit list') + print('PASS word = #7608 word (%d events) + %d deletions, %d insertions = %d events' + % (len(B['program']), len(E['delete']), sum(len(x[1]) for x in E['insert_before']), len(W['program']))) + + RB = replay(B) + R, changed, diff = compare(B, RB, W) + dim = [len(f) for f in W['frames']] + for r in changed: + f = lambda ch: '->'.join([str(dim[ch[0][0]])] + [str(dim[b]) for a, b in ch]) + print('PASS main %d dims %s became %s' % (r, f(RB['chains'][r]), f(R['chains'][r]))) + print('PASS exact replay: legal, all %d rows and end charts equal #7608, helpers identity, owned empty;' + ' owned MOVEs (%d), lifetimes (%d) and peak (%d) unchanged' + % (len(R['rows']), sum(R['owned_moves'].values()), sum(R['lifetimes'].values()), R['peak'])) + + cl = clears(W) + require(len(cl) == 74 and not diff, 'all 74 CLEARs erase #7608 content') + row44 = dict(cl[44][1]) + require(cl[44][0] == 1557 and row44 == {451: Q(-1), 1495: Q(1)}, 'CLEAR #44 (owned 1557) erases {451:-1, 1495:1}') + print('PASS CLEAR content: all 74 CLEARs erase exactly #7608 content (CLEAR #44, owned 1557, erases {451:-1, 1495:1}: the #7608 premise row, unchanged)') + + key = lambda W_, i: tuple(W_['frames'][i]) + pairs = lambda W_: {(key(W_, e[2]), key(W_, e[3])) for e in W_['program'] if e[0] == 'MOVE'} + require(pairs(W) <= pairs(B), 'no new chart pair') + print('PASS no new chart pair: every MOVE of the new word is a chart pair #7608 already uses') + ops = lambda P: collections.Counter(e[0] for e in P) + d = ops(W['program']); d.subtract(ops(B['program'])) + require({k: v for k, v in d.items() if v} == {'ADD': 11, 'MOVE': -1}, 'event delta +11 ADD, -1 MOVE') + sb, sw = RB['stats'], R['stats'] + require(sw['max_den_exp'] <= sb['max_den_exp'] and sw['odd_den'] == sb['odd_den'] + and sw['max_num_bits'] <= sb['max_num_bits'], 'intermediate coefficient grid within #7608') + print('PASS intermediate coefficients within #7608 (2-adic exponent %d, odd denominator lcm %d, numerator bits %d)' + % (sw['max_den_exp'], sw['odd_den'], sw['max_num_bits'])) + print('PASS event delta per native call', {k: v for k, v in sorted(d.items()) if v}) + + cb, cw = census(B, RB), census(W, R) + require(cw['helpers'] == cb['helpers'] and cw['rises'] == cb['rises'], 'helper targets and owned rises unchanged') + dpaid = collections.Counter({int(k): v for k, v in cw['paid'].items()}) + dpaid.subtract({int(k): v for k, v in cb['paid'].items()}) + dpaid = {k: v for k, v in sorted(dpaid.items()) if v} + require(dpaid == {1: -2, 2: 1}, 'native paid delta {1:-2, 2:+1}') + print('PASS native paid delta per call', dpaid) + inv0 = json.loads((DATA / 'invoice-7608.json').read_text()) + P = collections.Counter({int(k): v for k, v in inv0['paid'].items()}) + T = dict(inv0['targets']) + S0 = sum(int(k) * v for k, v in T.items()) - sum(int(k) * v for k, v in inv0['paid'].items()) + paid0 = sum(P.values()) + for k, v in dpaid.items(): + P[k] += CALLS * v + Pn = {str(k): v for k, v in sorted(P.items()) if v} + S = sum(int(k) * v for k, v in T.items()) - sum(int(k) * v for k, v in Pn.items()) + require(S == S0 == 1424, 'rank surplus 1424 unchanged') + require(json.loads((DATA / 'invoice-new.json').read_text()) == invoice_json(Pn, T), 'invoice-new.json matches') + require(json.loads((DATA / 'invoice-zero.json').read_text()) == invoice_json({}, {}), 'zero invoice') + paid1 = sum(Pn.values()) + require((paid0, paid1) == (110349, 110332), 'paid positions') + print('PASS complete invoice: paid positions %d -> %d, productive %d, S = %d' % (paid0, paid1, sum(T.values()), S)) + + k0, k1 = crossing(inv0['targets'], inv0['paid']), crossing(T, Pn) + lo, hi = decimal.Decimal('0.0036507495659'), decimal.Decimal('0.0036507495660') + require(lo <= k0 <= hi, '#7608 published crossing reproduced') + print('PASS kappa #7608 %s (inside the published #7608 enclosure)' % k0) + print('PASS kappa new %s (ratio %s)' % (k1, +decimal.Context(prec=12).divide(k1, k0))) + expected = json.loads((HERE / 'EXPECTED.json').read_text()) + require(str(k1)[:26] == expected['kappa_26'], 'kappa digits as recorded') + + def rejected(label, mutate): + X = copy.deepcopy(W) + mutate(X['program']) + try: + compare(B, RB, X) + except AssertionError as exc: + print('PASS control rejected: %s (%s)' % (label, exc)) + return + raise AssertionError('control not rejected: ' + label) + P0 = W['program'] + mv = P0.index(['MOVE', 46, 914, 127, False]) + cor = P0.index(['ADD', 588, 818, '1', 245]) + rejected('premix coefficient 588 += -1*228 changed to 2', + lambda P: P.__setitem__(P.index(['ADD', 588, 228, '-1', 3]), ['ADD', 588, 228, '2', 3])) + rejected('one premix ADD dropped', lambda P: P.remove(['ADD', 588, 443, '-1', 3])) + rejected('correction 588 += 818 dropped', lambda P: P.pop(cor)) + rejected('merge without premixes and corrections on #7608', + lambda P: (P.__init__(copy.deepcopy(B['program'])), P.remove(['MOVE', 46, 914, 245, False]), + P.__setitem__(P.index(['MOVE', 46, 245, 127, False]), ['MOVE', 46, 914, 127, False]))) + rejected('correction moved to chart 127', lambda P: P.__setitem__(cor, ['ADD', 588, 818, '1', 127])) + rejected('premix moved to chart 245', lambda P: P.__setitem__(P.index(['ADD', 588, 228, '-1', 3]), ['ADD', 588, 228, '-1', 245])) + rejected('reversed public MOVE 127->914', lambda P: P.__setitem__(mv, ['MOVE', 46, 127, 914, False])) + rejected('MOVE from the wrong chart', lambda P: P.__setitem__(mv, ['MOVE', 46, 245, 127, False])) + bad = dict(Pn); bad['3'] -= 1 + require(json.loads((DATA / 'invoice-new.json').read_text()) != invoice_json(bad, T), 'tampered invoice differs') + print('PASS control rejected: invoice with one rank-3 job removed differs from the recomputed one') + print('PASS_MAIN46_HOP_MERGE_EXACT (conditional on the #7608 CLEAR #44 premise)') + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/research/p104-main46-hop-merge/data/base-7608-word.json.gz b/research/p104-main46-hop-merge/data/base-7608-word.json.gz new file mode 100644 index 0000000000..e3cd54b929 Binary files /dev/null and b/research/p104-main46-hop-merge/data/base-7608-word.json.gz differ diff --git a/research/p104-main46-hop-merge/data/edits.json b/research/p104-main46-hop-merge/data/edits.json new file mode 100644 index 0000000000..65a5109016 --- /dev/null +++ b/research/p104-main46-hop-merge/data/edits.json @@ -0,0 +1 @@ +{"base_events":76631,"delete":[[45367,["MOVE",46,914,245,false]],[45368,["ADD",1456,46,"1",245]],[45370,["ADD",588,46,"-1",245]],[47953,["MOVE",46,245,127,false]]],"insert_before":[[0,[["ADD",588,1456,"1",3],["ADD",1456,588,"-1",3],["ADD",588,1456,"-1",3],["ADD",588,228,"-1",3],["ADD",1456,228,"1",3],["ADD",588,434,"-1",3],["ADD",1456,434,"1",3],["ADD",588,443,"-1",3],["ADD",1456,443,"1",3],["ADD",588,444,"1",3],["ADD",1456,444,"-1",3]]],[45372,[["ADD",588,818,"1",245],["ADD",1456,818,"-1",245]]],[47953,[["MOVE",46,914,127,false]]]],"append":[]} \ No newline at end of file diff --git a/research/p104-main46-hop-merge/data/invoice-7608.json b/research/p104-main46-hop-merge/data/invoice-7608.json new file mode 100644 index 0000000000..0f629caf0c --- /dev/null +++ b/research/p104-main46-hop-merge/data/invoice-7608.json @@ -0,0 +1 @@ +{"schema":"integer-mult-recursive-invoice/v1","contract":{"model":"mixed-target-unnormalized-power-v1","rank_semantics":"H_r costs r^(1-C); debit surrendered productive fields","ambient_rank":145,"boundary":"complete signed #7113 caller application with 17 native calls"},"paid":{"1":68248,"2":14416,"3":14093,"4":3808,"5":5848,"6":272,"7":187,"8":741,"9":32,"16":416,"24":416,"32":416,"40":416,"48":416,"56":416,"64":208},"targets":{"1":306,"2":357,"3":799,"4":3825,"5":1360,"6":731,"7":323,"8":544,"9":10888,"17":416,"25":416,"33":416,"41":416,"49":416,"57":416,"65":416,"73":208,"137":208}} \ No newline at end of file diff --git a/research/p104-main46-hop-merge/data/invoice-new.json b/research/p104-main46-hop-merge/data/invoice-new.json new file mode 100644 index 0000000000..af150f9670 --- /dev/null +++ b/research/p104-main46-hop-merge/data/invoice-new.json @@ -0,0 +1 @@ +{"schema":"integer-mult-recursive-invoice/v1","contract":{"model":"mixed-target-unnormalized-power-v1","rank_semantics":"H_r costs r^(1-C); debit surrendered productive fields","ambient_rank":145,"boundary":"complete signed #7113 caller application with 17 native calls"},"paid":{"1":68214,"2":14433,"3":14093,"4":3808,"5":5848,"6":272,"7":187,"8":741,"9":32,"16":416,"24":416,"32":416,"40":416,"48":416,"56":416,"64":208},"targets":{"1":306,"2":357,"3":799,"4":3825,"5":1360,"6":731,"7":323,"8":544,"9":10888,"17":416,"25":416,"33":416,"41":416,"49":416,"57":416,"65":416,"73":208,"137":208}} \ No newline at end of file diff --git a/research/p104-main46-hop-merge/data/invoice-zero.json b/research/p104-main46-hop-merge/data/invoice-zero.json new file mode 100644 index 0000000000..6b9ea16bcf --- /dev/null +++ b/research/p104-main46-hop-merge/data/invoice-zero.json @@ -0,0 +1 @@ +{"schema": "integer-mult-recursive-invoice/v1", "contract": {"model": "mixed-target-unnormalized-power-v1", "rank_semantics": "H_r costs r^(1-C); debit surrendered productive fields", "ambient_rank": 145, "boundary": "complete signed #7113 caller application with 17 native calls"}, "paid": {}, "targets": {}} \ No newline at end of file diff --git a/research/p104-main46-hop-merge/data/word.json.gz b/research/p104-main46-hop-merge/data/word.json.gz new file mode 100644 index 0000000000..97095b69f3 Binary files /dev/null and b/research/p104-main46-hop-merge/data/word.json.gz differ diff --git a/research/p104-main46-hop-merge/price.py b/research/p104-main46-hop-merge/price.py new file mode 100644 index 0000000000..4334a89629 --- /dev/null +++ b/research/p104-main46-hop-merge/price.py @@ -0,0 +1,53 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Certify the price with the repository's own invoice tools. + +Runs scripts/compare_recursive_invoices.py (#7608 invoice against the new +one, concave order) and scripts/evaluate_recursive_power.py (new invoice +against the empty invoice at r^t, t = 1 - C) and requires: + - the new invoice is uniformly better than #7608's for every increasing + concave tariff, strictly for r^(1-C), 0 < C < 1; + - at C = 0.0036509715393 the new paid cost is strictly below the new + targets (F(C) > 0), so the crossing is above the claim; + - at C = 0.0036509715394 it is strictly above (F(C) < 0); + - at #7608's C = 0.003650749565983 the new invoice is strictly favourable. +The invoices themselves are recomputed from the words by check.py. +""" +import json +import subprocess +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parents[1] +DATA = HERE / 'data' +CLAIM, ABOVE, PR7608 = '0.0036509715393', '0.0036509715394', '0.003650749565983' + + +def tool(name, *args): + out = subprocess.run([sys.executable, '-I', '-B', str(ROOT / 'scripts' / name), *map(str, args)], + capture_output=True, text=True, check=True, timeout=120) + return json.loads(out.stdout) + + +def one_minus(c): + from decimal import Decimal + return str(1 - Decimal(c)) + + +def main(): + old, new, zero = DATA / 'invoice-7608.json', DATA / 'invoice-new.json', DATA / 'invoice-zero.json' + rep = tool('compare_recursive_invoices.py', old, new) + assert rep['classification'] == 'uniformly_better', rep['classification'] + assert rep['power_moment']['new_minus_old_sign'] == 'strictly_negative' + assert rep['delta_paid'] == {'1': -34, '2': 17} and rep['delta_targets'] == {} + print('PASS compare_recursive_invoices #7608 -> new: uniformly_better, paid delta', rep['delta_paid']) + for c, want in ((CLAIM, 'new_cost_lower'), (ABOVE, 'new_cost_higher'), (PR7608, 'new_cost_lower')): + r = tool('evaluate_recursive_power.py', zero, new, '--exponent', one_minus(c), '--bits', 256) + assert r['classification'] == want, (c, r['classification']) + print('PASS evaluate_recursive_power at C = %s (t = %s): %s' % (c, r['exponent'], want)) + print('PASS_PRICE kappa in (%s, %s)' % (CLAIM, ABOVE)) + + +if __name__ == '__main__': + sys.exit(main())