diff --git a/research/p104-dual-scan-drop-902/EXPECTED.json b/research/p104-dual-scan-drop-902/EXPECTED.json new file mode 100644 index 0000000000..4ae71cf434 --- /dev/null +++ b/research/p104-dual-scan-drop-902/EXPECTED.json @@ -0,0 +1 @@ +{"kappa_26": "0.003650749565983677129400"} diff --git a/research/p104-dual-scan-drop-902/NOTICE b/research/p104-dual-scan-drop-902/NOTICE new file mode 100644 index 0000000000..7313f911c4 --- /dev/null +++ b/research/p104-dual-scan-drop-902/NOTICE @@ -0,0 +1,13 @@ +Apache-2.0. DreamingOfClouds with Claude Opus 5.5 (Anthropic), via Claude Code. + +data/base-7605-word.json.gz and data/invoice-7605.json are the unmodified +research/p104-timed-helper757/data/word.json.gz and data/invoice-new.json at +#7605 head e1d2756eecf03e146338239fee81772a99ba8d4c (Ali Romman, aliromman, +with OpenAI Codex assistance; Apache-2.0). Credit aliromman for #7605, Hao +Yan (FanPUYen) for #7604, SYLforge for #7603, ToikY for #7546 and the native, +price-engine and replay lineage, and our own #7601 and #7542 beneath them, +on Clear1nsight's #7334 and #7314, with Rohan Arun's signed #7113 caller and +all contributors named in those packages. + +check.py adapts the replay checks of our #7601 package; it imports no other +package and none of the retained authors' code was run. diff --git a/research/p104-dual-scan-drop-902/README.md b/research/p104-dual-scan-drop-902/README.md new file mode 100644 index 0000000000..698e448fa1 --- /dev/null +++ b/research/p104-dual-scan-drop-902/README.md @@ -0,0 +1,116 @@ +# CONDITIONAL κ = 0.00365074956598 — hosted main 902 skips F595 on #7605 (premise: owned CLEAR #44 may discard a 2-term row) + +This P104 word is the word of aliromman's +[#7605](https://github.com/CrocSwap/integer-mult-bounds/pull/7605) (head +`e1d2756eecf03e146338239fee81772a99ba8d4c`, `research/p104-timed-helper757/data/word.json.gz`, +SHA256 `37358c49…467d`) with one exact edit. #7605 builds on FanPUYen's #7604, +SYLforge's #7603, ToikY's #7546 and our #7601 +(see [NOTICE](NOTICE)). The claim is **conditional on one premise that the +earlier packages did not need**, stated first: + +> **Premise.** Owned CLEAR #44 (owned field 1557, chart 595, event 42713 of +> the new word) erases `{451:−1, 1495:1}` where #7605 erases +> `{451:−1, 902:−3/2, 985:−3/2, 1495:1}`. The word is valid only if +> `owned_erasure` lets a CLEAR discard that different (two-term) row. The +> other 73 of the 74 CLEARs erase exactly #7605's content. Every earlier link +> of the chain (#7546 → #7601 → #7604 → #7605) changed no erased content. + +If the premise is refused, #7605's κ 0.0036504442446 stands and this word +is void. Whether `owned_erasure` admits a different erased row is an open +upstream question: `research/p104-6372-clear-form-gap`, Q1. + +## The edit + +**Main 902** climbs 3→618 in one MOVE, where #7605 has 3→595→618 (chain 0→3→7→8 instead +of 0→1→3→7→8 in dimensions). Six events are deleted and one inserted (net −5; per native +call: 4 ADDs and 1 MOVE fewer): +- `MOVE 902 3→595` becomes `MOVE 902 3→618`; +- the ADDs `1532 += 2·902`, `1479 += −½·902`, `1255 += −1·902`, + `1495 += −3/2·902`, all at chart 595, and `MOVE 902 595→618` are deleted. + +[data/edits.json](data/edits.json) lists every event by #7605 index. The pair +3→618 is already used by #7605, so there is no new chart pair. All 1,569 final +rows, all end charts, the owned MOVEs (47), the 74 owned lifetimes, the peak +(50), the header, the signed #7113 caller data and the coefficient grid are +#7605's; only main 902's MOVE chain differs. + +The deleted `1495 += −3/2·902` is what fed owned 1557 at event 39612. That is +why CLEAR #44 changes. 1557 lives only at chart 595 (events 39611–42713) and +902 no longer visits 595, so no local repair exists. An exact linear search +over extra ADDs among registers live at 595 (4,673 candidates, 50 times) is +inconsistent. A repair needs a new MOVE that carries x902+x985 to 595, which +repays the 17 saved positions, or an extra owned lifetime. + +## Price + +The complete invoice is #7605's with the native delta charged 17 times +(native paid delta {1:−1, 2:−1, 3:+1}): +- paid positions 110,366 → 110,349; +- productive positions 22,461; +- rank surplus S = 1,424; zero empty-root credits. + +The crossing of Σ(T−P)·r^(1−C) is + +- **κ = 0.0036507495659836771294008126936…**, up from #7605's + 0.0036504442447168877637990594571… (×1.0000836, +0.00836%). + +The repository's own tools certify the price: +- `scripts/compare_recursive_invoices.py` classifies the change against #7605 + as `uniformly_better` (strict for r^(1−C)), paid {1:−17, 2:−17, 3:+17}. +- `scripts/evaluate_recursive_power.py` gives `new_cost_lower` at + C = 0.0036507495659 (and at #7605's 0.003650444244716) and + `new_cost_higher` at C = 0.0036507495660. + +## Checks + +```sh +python3.12 -B scripts/research_package.py --package research/p104-dual-scan-drop-902 +``` + +- `check.py` (standard library, about 3 minutes) does the following: + - authenticates the vendored #7605 word and rebuilds the new word from it + and the edit list; + - replays both words over the rationals from arbitrary inputs, with the + legality checks listed in its docstring; + - requires the header, every row and end chart, the owned MOVEs, lifetimes + and peak to equal #7605's, and that only main 902 changes its MOVE chain; + - **prints and asserts the erased-content diff: exactly one CLEAR (#44, + owned 1557) differs, with the two rows above**; + - recomputes the census, invoice and κ, and reproduces #7605's crossing; + - runs seven rejecting controls (re-added ADD at the wrong chart, a changed + 1557 coefficient, a 3→595 MOVE without 595→618, an owned write at 595 + after 902 left, a descending MOVE, a partial drop on #7605, a tampered + invoice). +- `price.py` runs the two repository tools above. + +The strictification gate (the repository's separation check) was run offline +by us and is not part of this package: γ 0.940932, δ 0.908149, both below +31/32 (#7605: γ 0.940920, δ 0.908149). + +## Conditional premises and what is not rerun + +On top of the premise above, the claim carries the framework assumptions of +#7605 (`native_all_columns`, `owned_erasure`, `compiler`, `precision_tape`, +`analytic_recovery`), the signed #7113 caller, and the BIT premise of the base. +This package does not rerun #7605's signed caller, outer predicates, +precision/tape counts or ordinary bill (four fewer ADDs and one fewer MOVE per +native call). No human has reviewed the package. + +## Method + +Frame-perp duality: the dual of a word is a cost-identical admissible native. +Running the token-settled drop scan (#7601) on the dual explores the V-side +edit class, whose edits the primal scan does not see. This drop was found +that way and then rebuilt and replayed on the primal word, exactly. + +## Open next questions + +**The premise.** A ruling on `owned_erasure` (clear-form-gap Q1) decides this +word. Absent one, a repair that restores 1557's erased row without paying the +saved positions would turn it into an unconditional drop. See [TASK.json](TASK.json). + +## Credit + +The base word and all retained work are aliromman's #7605 and the authors it +credits (see [NOTICE](NOTICE)). DreamingOfClouds with Claude Opus 5.5 +(Anthropic), via Claude Code. Apache-2.0. diff --git a/research/p104-dual-scan-drop-902/SOURCES.json b/research/p104-dual-scan-drop-902/SOURCES.json new file mode 100644 index 0000000000..c484f02b5a --- /dev/null +++ b/research/p104-dual-scan-drop-902/SOURCES.json @@ -0,0 +1,30 @@ +{ + "schema_version": 1, + "base_commit": "e54523517d0f03e58352e647c403a5af3e525a57", + "files": [ + { + "path": "certificates/selected-result.json", + "usage": "provenance", + "git_blob": "75d77b328ea02ded535e7f23aee5b1e55d0bb1a8", + "sha256": "154c3d30455d778fd3343b6b4851f0fa4665f0f7625b0adb3675e2bafbbf6bf1" + }, + { + "path": "docs/research/agent-task-catalog.json", + "usage": "provenance", + "git_blob": "3404fbdd5e7529a0674eef26114b5071e4586cec", + "sha256": "ead6cdc9d0af6d58e8a49744d8ec8c7c15e13d2ba93c276c434bcf9b51b93a81" + }, + { + "path": "scripts/compare_recursive_invoices.py", + "usage": "input", + "git_blob": "fda2a852e9937e42fe40df0707694465ca7c1b48", + "sha256": "3930c22c11da7bc6042cfeb6e459ecb06e4478430037bee3469d034353855fc6" + }, + { + "path": "scripts/evaluate_recursive_power.py", + "usage": "input", + "git_blob": "ca50f8021fc3238a58fd420383f4619a8d008ace", + "sha256": "52b20f41fd0bb55fa88dccd6015af2ce358917817b9506720f3e60ce681cc3ad" + } + ] +} diff --git a/research/p104-dual-scan-drop-902/TASK.json b/research/p104-dual-scan-drop-902/TASK.json new file mode 100644 index 0000000000..cf93d22c40 --- /dev/null +++ b/research/p104-dual-scan-drop-902/TASK.json @@ -0,0 +1,117 @@ +{ + "schema_version": 1, + "task_id": "p104-side-correction", + "priority_revision": 8, + "base_commit": "e54523517d0f03e58352e647c403a5af3e525a57", + "original_card": { + "id": "p104-side-correction", + "question": "Can joint helper bases and a legal causal split improve the complete P104 productive/paid invoice?", + "starting_points": [ + "docs/research/current-priorities.md", + "docs/research/research-family-status.md", + "certificates/selected-result.json", + "docs/research/pr-intake-evidence/pr5599-completion/PROOF-REVIEW.md", + "research/p104-helper-subspace-splits/README.md", + "research/dyadic-helper-channel-capacity/README.md" + ], + "methods": [ + "Construct and independently check one source-bound instance of the stated first experiment.", + "Seek an exact counterexample or target-specific complete-budget obstruction to that same proposed change." + ], + "acceptance_gate": "An exact arbitrary-input identity, source/target frame availability and restoration at the chosen cut, plus the complete channel+correction+idle/hosting bill or an explicit favorable bound. The native channel alone is not a supplier. A scoped local result may be retained without a complete supplier. Expanding it into further construction rounds requires the family-level budget and obligation gate in research-family-status.md; an optimistic passing relaxation alone is not acceptance.", + "first_step": { + "command": [ + "python3", + "-B", + "scripts/research_selected_baseline.py" + ], + "expected": "PASS_RESEARCH_BASELINE_BINDINGS_ONLY", + "inspect": "Read revision 8, the selected review and live exact-source handoffs. Then follow this card's bounded first_experiment.", + "scope": "Source-bound orientation only, not a supplier replay or theorem proof." + }, + "first_experiment": "Start with the selected #5599 emitted word and review pending #5718 distinct input/output bases plus #5717/#5719 coverage before claiming work. Pick one uncovered source word, helper group and cut region. Prove the arbitrary-input relation and legal entry/exit corrections, then compare full target and paid profiles. Reject if completion, owned lifetimes or consumer costs repay the gain. A static pairing rank alone is not a physical split.", + "bottleneck_link": "Judge the actual current supplier contract and complete signed target/paid bill; frozen-profile ceilings require an applicability proof.", + "allowed_variations": [ + "One exact source head, helper/cut region or quantified proposition and changed hypothesis; check live handoffs first. Different seeds are not different scopes." + ], + "scope_change_triggers": [ + "Changing source head, helper/cut domain, operation alphabet or caller boundary without recording the new scope.", + "Treating an archived proposal or local numerical pass as complete supplier admission." + ], + "overlap_terms": [ + "P104", + "helper basis", + "helper split", + "5718", + "dyadic capacity" + ], + "existing_checks": [ + { + "scope": "PR #4133: literal-slot defect", + "evidence": "research/p104-literal-slot-defect/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "88666f1dee93fbd85b31fd5d6f91a219e111b5f4", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4136: uniform commutation gap", + "evidence": "research/p104-uniform-commutation-gap/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "281ce601a8be9db470de630759179c23e1dfaa88", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4140: uniform encoding gap", + "evidence": "research/p104-uniform-encoding-gap/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "de25c6525aa866962309df5d107b32d55e6e5628", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4153: source-chart shortcut exhausts its optimistic allowance", + "evidence": "research/p104-source-chart-shortcut/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "9123241fea53df77e51ffe30d9aca1625d4bf3a8", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "PR #4154: fiber-tag seam mismatch", + "evidence": "research/p104-fiber-tag-seam/README.md", + "remaining": "Independently validate the stated scope or change its producer/consumer premise and pay the complete boundary.", + "status": "author-checked", + "source_commit": "4ffcdaee5d6255a9cea1fe53a5424435068089d0", + "selected_record_sha256": "bb2eaa58647b353f88d0aec2eafb9d30df1be53589231b23e12ceef3e4bad129" + }, + { + "scope": "Selected PR5599: complete helper-subspace construction and independent caller replay; five framework assumptions remain.", + "evidence": "docs/research/pr-intake-evidence/pr5599-completion/PROOF-REVIEW.md", + "remaining": "The next experiment must identify uncovered source-specific work. Successor archive intake does not transfer this admission.", + "status": "reviewed", + "source_commit": "09dd0965711ca7eeb169057a420abab1c9052137", + "selected_record_sha256": "2a0220e866417774606e0f991e41b9c1d26b1f43b97857238cf478dbc7d330b5" + } + ] + }, + "scope_status": "refined", + "actual_scope": "Exact #7605 word, public main 902 only: it climbs 3->618 directly instead of 3->595->618 (five events deleted, the F595 stay removed); rows, end charts, owned MOVEs, lifetimes, header and signed #7113 caller data are #7605's. Conditional: the content erased at owned CLEAR #44 (owned 1557, chart 595) differs from #7605's.", + "scope_change_reason": "The card asks for joint helper bases and causal splits; this is a narrower exact chain shortening on one main of the current word, found by running the token-settled drop scan on the frame-perp dual of the word. It changes one erased CLEAR row, so it is conditional on owned_erasure admitting that row.", + "bottleneck_link": "Reduces the complete signed paid invoice (17 rank-1 and 17 rank-2 jobs become 17 rank-3 jobs) without changing the productive profile.", + "budget_minutes": 90, + "stop_rule": "One literal word with exact arbitrary-input row equality, legality, the single changed erased row stated and asserted, complete invoice and a certified crossing.", + "outcome": "completed", + "evidence": "README.md, data/edits.json, check.py, price.py, VERIFY.json.", + "author": "DreamingOfClouds", + "ai_assistance": "Claude Opus 5.5 (Anthropic), via Claude Code.", + "open_questions": [ + { + "question": "Does owned_erasure admit owned CLEAR #44 (field 1557, chart 595) discarding {451:-1, 1495:1} instead of {451:-1, 902:-3/2, 985:-3/2, 1495:1}? (research/p104-6372-clear-form-gap, Q1)", + "why_it_matters": "The claimed kappa holds only if it does; if not, #7605's kappa stands and this word is void.", + "first_step": "Run check.py and read the PREMISE line; compare it with the erased-row form the contract allows.", + "blocked_by": "A ruling on owned_erasure; no zero-move repair exists (an exact search over extra ADDs among registers live at chart 595 is inconsistent), a repair needs a new MOVE or an extra owned lifetime." + } + ] +} diff --git a/research/p104-dual-scan-drop-902/VERIFY.json b/research/p104-dual-scan-drop-902/VERIFY.json new file mode 100644 index 0000000000..5a8d9362d1 --- /dev/null +++ b/research/p104-dual-scan-drop-902/VERIFY.json @@ -0,0 +1,32 @@ +{ + "schema_version": 1, + "status": "ready", + "python": "3.12", + "requirements": [], + "inputs": [ + "scripts/compare_recursive_invoices.py", + "scripts/evaluate_recursive_power.py" + ], + "commands": [ + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/check.py" + ], + "timeout_seconds": 300, + "checks": "Vendored #7605 word bytes; word = #7605 + edit list; exact arbitrary-input replay and legality of both words; header identical; all 1,569 rows and end charts equal; owned MOVEs, lifetimes, peak unchanged; THE PREMISE: exactly one of 74 CLEARs (#44, owned 1557) erases different content, both rows printed and asserted; only main 902 changes chain; no new chart pair; #7605 crossing reproduced; new invoice and kappa 0.003650749565983677129400...; seven rejecting controls. Prints PASS_DUAL_SCAN_DROP_902_EXACT." + }, + { + "argv": [ + "{python}", + "-I", + "-B", + "{package}/price.py" + ], + "timeout_seconds": 120, + "checks": "Repository tools: uniformly_better against #7605; new_cost_lower at C=0.0036507495659 and at #7605's 0.003650444244716; new_cost_higher at C=0.0036507495660. Prints PASS_PRICE." + } + ] +} diff --git a/research/p104-dual-scan-drop-902/check.py b/research/p104-dual-scan-drop-902/check.py new file mode 100644 index 0000000000..88231f8f68 --- /dev/null +++ b/research/p104-dual-scan-drop-902/check.py @@ -0,0 +1,366 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Exact replay of the hosted-main-902 drop against #7605's word. + +Standard library only. Written for this package; it imports no other package. + +1. The vendored #7605 word has the SHA256 of research/p104-timed-helper757/ + data/word.json.gz at #7605 head e1d2756eecf03e146338239fee81772a99ba8d4c, + and applying data/edits.json to it gives data/word.json.gz event for event. +2. Both words are replayed over the rationals from arbitrary inputs (every + public field starts as its own symbol, owned fields start at INIT as 0). + Legality: every ADD/COPY/SCALE operand sits at the event's chart, every + MOVE starts at the register's current chart, public MOVEs strictly ascend + (span inclusion), owned MOVEs are nested and only on live owned fields, + COPY writes a zero field, no SCALE by 0, no self-ADD, every owned field + is cleared at the chart it sits in, and public fields end at `final`. +3. Header identical; all 1,569 rows and end charts equal #7605's; helpers end + at identity, owned rows end empty; owned MOVEs, lifetimes and peak are + unchanged. +4. THE PREMISE. The content erased at the 74 owned CLEARs is compared with + #7605's. Exactly one differs: CLEAR #44 (owned field 1557, chart 595, + event 42713 of the new word). #7605 erases {451:-1, 902:-3/2, 985:-3/2, + 1495:1}; the new word erases {451:-1, 1495:1}. The check prints both rows + and requires that this is the only difference. This is what makes the + claim conditional. +5. Only main 902 changes its MOVE chain (0->1->3 becomes 0->3), endpoints kept. +6. Census, complete invoice (17 native calls) and kappa are recomputed and + compared with data/invoice-*.json; #7605's crossing is reproduced. +7. Controls: corrupted words and a tampered invoice must be rejected. +""" +import collections +import copy +import decimal +import gzip +import hashlib +import json +import sys +from fractions import Fraction as Q +from pathlib import Path + +HERE = Path(__file__).resolve().parent +DATA = HERE / 'data' +BASE_SHA256 = '37358c493d18ebc6be386aebb4c17529b38883674731f7009ef4e96a5313467d' +CALLS = 17 + + +def require(cond, label): + if not cond: + raise AssertionError(label) + + +def load_gz(path): + return json.loads(gzip.decompress(path.read_bytes())) + + +def span(vectors): + s = {0} + for v in vectors: + require(type(v) is int and 0 < v < 512 and v not in s, 'frame vectors independent in F2^9') + s |= {x ^ v for x in s} + return frozenset(s) + + +def mains(W): + live = set(W['live']) + return live | {r + 112 for r in live} | set(W['hosted']['X2']) | set(W['hosted']['Y2']) + + +def apply_edits(program, E): + require(len(program) == E['base_events'], 'edit list base length') + dele = {} + for i, ev in E['delete']: + require(program[i] == ev, 'deleted event %d matches' % i) + dele[i] = ev + ins = collections.defaultdict(list) + for i, evs in E['insert_before']: + ins[i].extend(evs) + out = [] + for i, ev in enumerate(program): + out.extend(ins.get(i, [])) + if i not in dele: + out.append(ev) + out.extend(ins.get(len(program), [])) + return out + E['append'] + + +def replay(W): + """Exact replay; returns rows, end charts, MOVE chains and owned data. Raises on any illegality.""" + n, N = W['public'], W['public'] + W['owned_fields'] + sp = [span(f) for f in W['frames']] + rows = [{r: Q(1)} if r < n else {} for r in range(N)] + cur = list(W['initial']) + [None] * W['owned_fields'] + alive, chains, owned_moves = set(), collections.defaultdict(list), collections.Counter() + lifetimes, peak = collections.Counter(), 0 + init_chart = {} + stats = dict(max_den_exp=0, odd_den=1, max_num_bits=0) + + def note(q): + d = q.denominator + e = (d & -d).bit_length() - 1 + stats['max_den_exp'] = max(stats['max_den_exp'], e) + odd = d >> e + if odd > 1 and stats['odd_den'] % odd: + stats['odd_den'] = stats['odd_den'] * odd // __import__('math').gcd(stats['odd_den'], odd) + stats['max_num_bits'] = max(stats['max_num_bits'], abs(q.numerator).bit_length()) + + for i, e in enumerate(W['program']): + op, r = e[0], e[1] + require(type(r) is int and 0 <= r < N, '%d register index' % i) + if op == 'MOVE': + a, b, flag = e[2], e[3], e[4] + require(type(flag) is bool and flag == (r >= n), '%d MOVE ownership flag' % i) + require(cur[r] == a, '%d MOVE from the current chart' % i) + if r >= n: + require(r in alive, '%d owned MOVE on a live field' % i) + require(sp[a] < sp[b] or sp[b] < sp[a], '%d owned MOVE nested' % i) + owned_moves[(a, b)] += 1 + else: + require(sp[a] < sp[b], '%d public MOVE strictly ascends' % i) + cur[r] = b + chains[r].append((a, b)) + elif op == 'INIT': + require(r >= n and r not in alive and len(e) == 3, '%d INIT of a dead owned field' % i) + alive.add(r); rows[r] = {}; cur[r] = e[2]; init_chart[r] = e[2] + peak = max(peak, len(alive)) + elif op == 'CLEAR': + require(r >= n and r in alive and cur[r] == e[2], '%d CLEAR at the current chart' % i) + lifetimes[(r, init_chart.pop(r), e[2])] += 1 + alive.discard(r); rows[r] = {}; cur[r] = None + elif op == 'SCALE': + q = Q(e[2]) + require(q != 0, '%d SCALE by zero' % i) + require(cur[r] == e[3] and (r < n or r in alive), '%d SCALE chart' % i) + rows[r] = {k: v * q for k, v in rows[r].items()} + for v in rows[r].values(): + note(v) + elif op in ('ADD', 'COPY'): + s, q, f = e[2], Q(e[3]), e[4] + require(type(s) is int and 0 <= s < N and s != r, '%d operand' % i) + require(q != 0, '%d zero coefficient' % i) + require(cur[r] == f and cur[s] == f, '%d ADD/COPY operands at chart %d' % (i, f)) + for x in (r, s): + require(x < n or x in alive, '%d owned field live' % i) + if op == 'COPY': + require(not rows[r], '%d COPY into a zero field' % i) + R = rows[r] + for k, v in rows[s].items(): + nv = R.get(k, 0) + q * v + if nv: + R[k] = nv + note(nv) + else: + R.pop(k, None) + else: + raise AssertionError('%d unknown op %r' % (i, op)) + require(not alive, 'every owned field cleared') + require(cur[:n] == list(W['final']), 'public fields end at final charts') + return dict(rows=rows, cur=cur, chains=chains, owned_moves=owned_moves, + lifetimes=lifetimes, peak=peak, sp=sp, stats=stats) + + +def census(W, R): + """Per-native paid MOVE ranks and helper targets (the #7546 accounting).""" + dim = [len(f) for f in W['frames']] + paid = collections.Counter(abs(dim[e[3]] - dim[e[2]]) for e in W['program'] if e[0] == 'MOVE') + M, n = mains(W), W['public'] + helpers = collections.Counter(dim[W['final'][r]] - dim[W['initial'][r]] for r in range(n) if r not in M) + rises = collections.Counter(dim[e[3]] - dim[e[2]] for e in W['program'] + if e[0] == 'MOVE' and e[1] >= n and dim[e[3]] > dim[e[2]]) + hist = lambda c: {str(k): v for k, v in sorted(c.items()) if v} + return dict(events=len(W['program']), public=n, helpers=hist(helpers), paid=hist(paid), + lifetimes=sum(R['lifetimes'].values()), owned_peak=R['peak'], rises=hist(rises)) + + +def crossing(T, P, digits=40): + """Root of F(C) = sum (T_r - P_r) r^(1-C) on [0.003, 0.0045] by bisection in 80-digit decimals.""" + D = decimal.Decimal + with decimal.localcontext(decimal.Context(prec=digits + 40)) as ctx: + ranks = sorted({int(r) for r in T} | {int(r) for r in P}) + terms = [(D(T.get(str(r), 0) - P.get(str(r), 0)) * r, D(r).ln()) for r in ranks] + + def F(C): + return sum((c * (-C * l).exp() for c, l in terms), D(0)) + lo, hi = D('0.003'), D('0.0045') + require(F(lo) > 0 > F(hi), 'crossing bracketed') + for _ in range(160): + mid = (lo + hi) / 2 + if F(mid) > 0: + lo = mid + else: + hi = mid + return decimal.Context(prec=digits).create_decimal(lo) + + +def invoice_json(P, T): + return {'schema': 'integer-mult-recursive-invoice/v1', + 'contract': {'model': 'mixed-target-unnormalized-power-v1', + 'rank_semantics': 'H_r costs r^(1-C); debit surrendered productive fields', + 'ambient_rank': 145, + 'boundary': 'complete signed #7113 caller application with 17 native calls'}, + 'paid': {k: P[k] for k in sorted(P, key=int)}, + 'targets': {k: T[k] for k in sorted(T, key=int)}} + + +def clears(W): + """Row content erased at every owned CLEAR, in program order.""" + n = W['public'] + rows = [{r: Q(1)} if r < n else {} for r in range(n + W['owned_fields'])] + out = [] + for e in W['program']: + op, r = e[0], e[1] + if op == 'INIT': + rows[r] = {} + elif op == 'CLEAR': + out.append((r, tuple(sorted(rows[r].items())))) + rows[r] = {} + elif op == 'SCALE': + rows[r] = {k: v * Q(e[2]) for k, v in rows[r].items()} + elif op in ('ADD', 'COPY'): + R = rows[r] + for k, v in rows[e[2]].items(): + nv = R.get(k, 0) + Q(e[3]) * v + if nv: + R[k] = nv + else: + R.pop(k, None) + return out + + +PREMISE_BASE = (451, Q(-1)), (902, Q(-3, 2)), (985, Q(-3, 2)), (1495, Q(1)) +PREMISE_NEW = (451, Q(-1)), (1495, Q(1)) + + +def compare(B, RB, W): + """Everything that must hold between #7605's word B and a candidate W, except the erased-content premise. + Returns the candidate replay, the changed chains and the list of differing CLEARs (index, base, new).""" + require(all(B[k] == W[k] for k in B if k != 'program') and B.keys() == W.keys(), 'header identical') + R = replay(W) + n = W['public'] + bad = [r for r in range(len(R['rows'])) if R['rows'][r] != RB['rows'][r]] + require(not bad, 'all rows equal #7605 (first differing %s)' % bad[:3]) + require(R['cur'] == RB['cur'], 'all end charts equal #7605') + M = mains(W) + require(all(R['rows'][r] == {r: Q(1)} for r in range(n) if r not in M), 'helpers end at identity') + require(all(not R['rows'][r] for r in range(n, len(R['rows']))), 'owned rows end empty') + require(R['owned_moves'] == RB['owned_moves'], 'owned MOVE multiset unchanged') + require(R['lifetimes'] == RB['lifetimes'] and R['peak'] == RB['peak'], 'owned lifetimes and peak unchanged') + cb, cw = clears(B), clears(W) + require(len(cb) == len(cw), 'same number of CLEARs') + diff = [(i, x, y) for i, (x, y) in enumerate(zip(cb, cw)) if x != y] + changed = sorted(r for r in set(R['chains']) | set(RB['chains']) if R['chains'][r] != RB['chains'][r]) + require(changed == [902], 'only main 902 changes its chain (got %s)' % changed[:5]) + require(all(r in M for r in changed), 'changed registers are mains') + for r in changed: + old, new = RB['chains'][r], R['chains'][r] + require(old[0][0] == new[0][0] and old[-1][1] == new[-1][1], 'register %d chain endpoints' % r) + return R, changed, diff + + +def main(): + base_raw = (DATA / 'base-7605-word.json.gz').read_bytes() + require(hashlib.sha256(base_raw).hexdigest() == BASE_SHA256, 'vendored #7605 word bytes') + B = json.loads(gzip.decompress(base_raw)) + W = load_gz(DATA / 'word.json.gz') + E = json.loads((DATA / 'edits.json').read_text()) + require(apply_edits(B['program'], E) == W['program'], 'word = #7605 word + edit list') + print('PASS word = #7605 word (%d events) + %d deletions, %d insertions = %d events' + % (len(B['program']), len(E['delete']), sum(len(x[1]) for x in E['insert_before']), len(W['program']))) + + RB = replay(B) + R, changed, diff = compare(B, RB, W) + dim = [len(f) for f in W['frames']] + for r in changed: + f = lambda ch: '->'.join([str(dim[ch[0][0]])] + [str(dim[b]) for a, b in ch]) + print('PASS main %d dims %s became %s' % (r, f(RB['chains'][r]), f(R['chains'][r]))) + print('PASS exact replay: legal, all %d rows and end charts equal #7605, helpers identity, owned empty;' + ' owned MOVEs (%d), lifetimes (%d) and peak (%d) unchanged' + % (len(R['rows']), sum(R['owned_moves'].values()), sum(R['lifetimes'].values()), R['peak'])) + + fmt = lambda row: '{' + ', '.join('%d:%s' % (k, v) for k, v in row) + '}' + require(len(diff) == 1, 'exactly one CLEAR erases different content (got %d)' % len(diff)) + i, x, y = diff[0] + require(i == 44 and x[0] == y[0] == 1557, 'the differing CLEAR is #44 on owned field 1557') + require(x[1] == PREMISE_BASE and y[1] == PREMISE_NEW, 'erased rows as stated') + print('PREMISE CLEAR #%d of %d (owned %d): #7605 erases %s, new word erases %s; the other %d CLEARs are identical' + % (i, len(clears(B)), x[0], fmt(x[1]), fmt(y[1]), len(clears(B)) - 1)) + + key = lambda W_, i: tuple(W_['frames'][i]) + pairs = lambda W_: {(key(W_, e[2]), key(W_, e[3])) for e in W_['program'] if e[0] == 'MOVE'} + require(pairs(W) <= pairs(B), 'no new chart pair') + print('PASS no new chart pair: every MOVE of the new word is a chart pair #7605 already uses') + ops = lambda P: collections.Counter(e[0] for e in P) + d = ops(W['program']); d.subtract(ops(B['program'])) + require({k: v for k, v in d.items() if v} == {'ADD': -4, 'MOVE': -1}, 'event delta -4 ADD, -1 MOVE') + sb, sw = RB['stats'], R['stats'] + require(sw['max_den_exp'] <= sb['max_den_exp'] and sw['odd_den'] == sb['odd_den'] + and sw['max_num_bits'] <= sb['max_num_bits'], 'intermediate coefficient grid within #7605') + print('PASS intermediate coefficients within #7605 (2-adic exponent %d, odd denominator lcm %d, numerator bits %d)' + % (sw['max_den_exp'], sw['odd_den'], sw['max_num_bits'])) + print('PASS event delta per native call', {k: v for k, v in sorted(d.items()) if v}) + + cb, cw = census(B, RB), census(W, R) + require(cw['helpers'] == cb['helpers'] and cw['rises'] == cb['rises'], 'helper targets and owned rises unchanged') + dpaid = collections.Counter({int(k): v for k, v in cw['paid'].items()}) + dpaid.subtract({int(k): v for k, v in cb['paid'].items()}) + dpaid = {k: v for k, v in sorted(dpaid.items()) if v} + require(dpaid == {1: -1, 2: -1, 3: 1}, 'native paid delta {1:-1, 2:-1, 3:+1}') + print('PASS native paid delta per call', dpaid) + inv0 = json.loads((DATA / 'invoice-7605.json').read_text()) + P = collections.Counter({int(k): v for k, v in inv0['paid'].items()}) + T = dict(inv0['targets']) + S0 = sum(int(k) * v for k, v in T.items()) - sum(int(k) * v for k, v in inv0['paid'].items()) + paid0 = sum(P.values()) + for k, v in dpaid.items(): + P[k] += CALLS * v + Pn = {str(k): v for k, v in sorted(P.items()) if v} + S = sum(int(k) * v for k, v in T.items()) - sum(int(k) * v for k, v in Pn.items()) + require(S == S0 == 1424, 'rank surplus 1424 unchanged') + require(json.loads((DATA / 'invoice-new.json').read_text()) == invoice_json(Pn, T), 'invoice-new.json matches') + require(json.loads((DATA / 'invoice-zero.json').read_text()) == invoice_json({}, {}), 'zero invoice') + paid1 = sum(Pn.values()) + require((paid0, paid1) == (110366, 110349), 'paid positions') + print('PASS complete invoice: paid positions %d -> %d, productive %d, S = %d' % (paid0, paid1, sum(T.values()), S)) + + k0, k1 = crossing(inv0['targets'], inv0['paid']), crossing(T, Pn) + lo, hi = decimal.Decimal('0.0036504442446'), decimal.Decimal('0.0036504442448') + require(lo <= k0 <= hi, '#7605 published crossing reproduced') + print('PASS kappa #7605 %s (inside the published #7605 enclosure)' % k0) + print('PASS kappa new %s (ratio %s)' % (k1, +decimal.Context(prec=12).divide(k1, k0))) + expected = json.loads((HERE / 'EXPECTED.json').read_text()) + require(str(k1)[:26] == expected['kappa_26'], 'kappa digits as recorded') + + def rejected(label, mutate): + X = copy.deepcopy(W) + mutate(X['program']) + try: + R2, _, diff2 = compare(B, RB, X) + require(len(diff2) == 1 and diff2[0][0] == 44, 'only the premise CLEAR differs') + except AssertionError as exc: + print('PASS control rejected: %s (%s)' % (label, exc)) + return + raise AssertionError('control not rejected: ' + label) + P0 = W['program'] + i = P0.index(['MOVE', 902, 3, 618, False]) + rejected('re-add 1479 += -1/2*902 at chart 618 (operand absent)', + lambda P: P.insert(i + 1, ['ADD', 1479, 902, '-1/2', 618])) + rejected('1557 += 1*1495 coefficient changed to 2', + lambda P: P.__setitem__(P.index(['ADD', 1557, 1495, '1', 595]), ['ADD', 1557, 1495, '2', 595])) + rejected('3->618 MOVE replaced by 3->595 without 595->618', + lambda P: P.__setitem__(i, ['MOVE', 902, 3, 595, False])) + rejected('owned write at 595 after 902 has left', + lambda P: P.insert(i + 1, ['ADD', 1532, 902, '2', 595])) + rejected('descending public MOVE 902 618->3', lambda P: P.__setitem__(i, ['MOVE', 902, 618, 3, False])) + + def partial(P): # the drop on #7605 with only the 1495 ADD removed + P[:] = copy.deepcopy(B['program']) + P.remove(['ADD', 1495, 902, '-3/2', 595]) + rejected('partial drop on #7605 (only 1495 += -3/2*902 deleted)', partial) + bad = dict(Pn); bad['3'] -= 1 + require(json.loads((DATA / 'invoice-new.json').read_text()) != invoice_json(bad, T), 'tampered invoice differs') + print('PASS control rejected: invoice with one rank-3 job removed differs from the recomputed one') + print('PASS_DUAL_SCAN_DROP_902_EXACT (conditional on the CLEAR #44 premise)') + + +if __name__ == '__main__': + sys.exit(main()) diff --git a/research/p104-dual-scan-drop-902/data/base-7605-word.json.gz b/research/p104-dual-scan-drop-902/data/base-7605-word.json.gz new file mode 100644 index 0000000000..0527ed3076 Binary files /dev/null and b/research/p104-dual-scan-drop-902/data/base-7605-word.json.gz differ diff --git a/research/p104-dual-scan-drop-902/data/edits.json b/research/p104-dual-scan-drop-902/data/edits.json new file mode 100644 index 0000000000..eed30fa786 --- /dev/null +++ b/research/p104-dual-scan-drop-902/data/edits.json @@ -0,0 +1 @@ +{"base_events":76636,"delete":[[38292,["MOVE",902,3,595,false]],[38293,["ADD",1532,902,"2",595]],[38582,["ADD",1479,902,"-1/2",595]],[38583,["ADD",1255,902,"-1",595]],[38584,["ADD",1495,902,"-3/2",595]],[38586,["MOVE",902,595,618,false]]],"insert_before":[[38292,[["MOVE",902,3,618,false]]]],"append":[]} \ No newline at end of file diff --git a/research/p104-dual-scan-drop-902/data/invoice-7605.json b/research/p104-dual-scan-drop-902/data/invoice-7605.json new file mode 100644 index 0000000000..7f710a6a08 --- /dev/null +++ b/research/p104-dual-scan-drop-902/data/invoice-7605.json @@ -0,0 +1,47 @@ +{ + "schema": "integer-mult-recursive-invoice/v1", + "contract": { + "model": "mixed-target-unnormalized-power-v1", + "rank_semantics": "H_r costs r^(1-C); debit surrendered productive fields", + "ambient_rank": 145, + "boundary": "complete signed #7113 caller application with 17 native calls" + }, + "paid": { + "1": 68265, + "2": 14433, + "3": 14076, + "4": 3808, + "5": 5848, + "6": 272, + "7": 187, + "8": 741, + "9": 32, + "16": 416, + "24": 416, + "32": 416, + "40": 416, + "48": 416, + "56": 416, + "64": 208 + }, + "targets": { + "1": 306, + "2": 357, + "3": 799, + "4": 3825, + "5": 1360, + "6": 731, + "7": 323, + "8": 544, + "9": 10888, + "17": 416, + "25": 416, + "33": 416, + "41": 416, + "49": 416, + "57": 416, + "65": 416, + "73": 208, + "137": 208 + } +} diff --git a/research/p104-dual-scan-drop-902/data/invoice-new.json b/research/p104-dual-scan-drop-902/data/invoice-new.json new file mode 100644 index 0000000000..0f629caf0c --- /dev/null +++ b/research/p104-dual-scan-drop-902/data/invoice-new.json @@ -0,0 +1 @@ +{"schema":"integer-mult-recursive-invoice/v1","contract":{"model":"mixed-target-unnormalized-power-v1","rank_semantics":"H_r costs r^(1-C); debit surrendered productive fields","ambient_rank":145,"boundary":"complete signed #7113 caller application with 17 native calls"},"paid":{"1":68248,"2":14416,"3":14093,"4":3808,"5":5848,"6":272,"7":187,"8":741,"9":32,"16":416,"24":416,"32":416,"40":416,"48":416,"56":416,"64":208},"targets":{"1":306,"2":357,"3":799,"4":3825,"5":1360,"6":731,"7":323,"8":544,"9":10888,"17":416,"25":416,"33":416,"41":416,"49":416,"57":416,"65":416,"73":208,"137":208}} \ No newline at end of file diff --git a/research/p104-dual-scan-drop-902/data/invoice-zero.json b/research/p104-dual-scan-drop-902/data/invoice-zero.json new file mode 100644 index 0000000000..6b9ea16bcf --- /dev/null +++ b/research/p104-dual-scan-drop-902/data/invoice-zero.json @@ -0,0 +1 @@ +{"schema": "integer-mult-recursive-invoice/v1", "contract": {"model": "mixed-target-unnormalized-power-v1", "rank_semantics": "H_r costs r^(1-C); debit surrendered productive fields", "ambient_rank": 145, "boundary": "complete signed #7113 caller application with 17 native calls"}, "paid": {}, "targets": {}} \ No newline at end of file diff --git a/research/p104-dual-scan-drop-902/data/word.json.gz b/research/p104-dual-scan-drop-902/data/word.json.gz new file mode 100644 index 0000000000..e3cd54b929 Binary files /dev/null and b/research/p104-dual-scan-drop-902/data/word.json.gz differ diff --git a/research/p104-dual-scan-drop-902/price.py b/research/p104-dual-scan-drop-902/price.py new file mode 100644 index 0000000000..04c7429936 --- /dev/null +++ b/research/p104-dual-scan-drop-902/price.py @@ -0,0 +1,53 @@ +#!/usr/bin/env python3 +# SPDX-License-Identifier: Apache-2.0 +"""Certify the price with the repository's own invoice tools. + +Runs scripts/compare_recursive_invoices.py (#7605 invoice against the new +one, concave order) and scripts/evaluate_recursive_power.py (new invoice +against the empty invoice at r^t, t = 1 - C) and requires: + - the new invoice is uniformly better than #7605's for every increasing + concave tariff, strictly for r^(1-C), 0 < C < 1; + - at C = 0.0036507495659 the new paid cost is strictly below the new + targets (F(C) > 0), so the crossing is above the claim; + - at C = 0.0036507495660 it is strictly above (F(C) < 0); + - at #7605's C = 0.003650444244716 the new invoice is strictly favourable. +The invoices themselves are recomputed from the words by check.py. +""" +import json +import subprocess +import sys +from pathlib import Path + +HERE = Path(__file__).resolve().parent +ROOT = HERE.parents[1] +DATA = HERE / 'data' +CLAIM, ABOVE, PR7605 = '0.0036507495659', '0.0036507495660', '0.003650444244716' + + +def tool(name, *args): + out = subprocess.run([sys.executable, '-I', '-B', str(ROOT / 'scripts' / name), *map(str, args)], + capture_output=True, text=True, check=True, timeout=120) + return json.loads(out.stdout) + + +def one_minus(c): + from decimal import Decimal + return str(1 - Decimal(c)) + + +def main(): + old, new, zero = DATA / 'invoice-7605.json', DATA / 'invoice-new.json', DATA / 'invoice-zero.json' + rep = tool('compare_recursive_invoices.py', old, new) + assert rep['classification'] == 'uniformly_better', rep['classification'] + assert rep['power_moment']['new_minus_old_sign'] == 'strictly_negative' + assert rep['delta_paid'] == {'1': -17, '2': -17, '3': 17} and rep['delta_targets'] == {} + print('PASS compare_recursive_invoices #7605 -> new: uniformly_better, paid delta', rep['delta_paid']) + for c, want in ((CLAIM, 'new_cost_lower'), (ABOVE, 'new_cost_higher'), (PR7605, 'new_cost_lower')): + r = tool('evaluate_recursive_power.py', zero, new, '--exponent', one_minus(c), '--bits', 256) + assert r['classification'] == want, (c, r['classification']) + print('PASS evaluate_recursive_power at C = %s (t = %s): %s' % (c, r['exponent'], want)) + print('PASS_PRICE kappa in (%s, %s)' % (CLAIM, ABOVE)) + + +if __name__ == '__main__': + sys.exit(main())