From f90705e59a3dce30f34bf61490675f7d1917187f Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 00:19:38 -0700 Subject: [PATCH 01/35] docs(codec): specify bounded value extension protocols --- docs/maintainers/value-contract-prototypes.md | 27 +++++++++++++++++++ 1 file changed, 27 insertions(+) create mode 100644 docs/maintainers/value-contract-prototypes.md diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md new file mode 100644 index 00000000..e91a9021 --- /dev/null +++ b/docs/maintainers/value-contract-prototypes.md @@ -0,0 +1,27 @@ +# Value extension prototypes + +These rules are candidates for #337 and #339. They do not change the 0.11 codec policy. The prototypes live under `test/prototypes` and cannot enable either feature in shipped wrappers. + +## Exact integer option + +The generation option is `integerMode: 'bigint'` for one callable. It changes every Python `int` in that callable's parameters and result to TypeScript `bigint`. Safe integers use the same type as large integers in this mode. Python `bool` remains `boolean`. Calls without the option keep the safe `number` rule. + +The generated wrapper first calls `meta` and requires `valueCapabilities` to contain `exactIntegerDecimalV2`. The option then travels on each `call` request as `params.valuePolicy.integer = 'bigint-v2'`. A new bridge rejects an unknown value policy. An old bridge lacks the capability, so the wrapper rejects it before sending a tagged value. A new bridge sends tagged integers only for an opted call. Existing clients never opt in. + +The version 2 integer envelope is `{"__tywrap__":"integer","codecVersion":2,"encoding":"decimal","value":"18446744073709551617"}`. The `value` string must match `^(?:0|[1-9][0-9]*|-[1-9][0-9]*)$`. This rejects `-0`, leading zeros, a leading plus, spaces, exponents, and Unicode digits. It allows at most 4096 decimal digits, excluding the sign. The ASCII value occupies at most 4097 bytes. The existing request and response payload byte limits still apply to the whole message. Both decoders reject an unknown version, extra envelope fields, an invalid decimal, or an oversized value before calling `BigInt` or Python `int`. + +The request encoder tags every `bigint`, including safe values. The Python request decoder yields `int`. The Python response encoder tags every `int`, including safe values. The TypeScript response decoder yields `bigint`. Both walk nested arrays and string-keyed records within the existing depth and node limits. They reject cycles and unsupported values with paths. Arrow int64 keeps its existing Arrow encoding and decoded type. + +The generated return validator requires `bigint` at every integer node. It must reject an ordinary number and a raw tagged record. A wrapper with `integerMode: 'bigint'` therefore cannot expose `Promise` for a Python integer. + +The bounded prototype must round trip positive and negative values larger than 64 bits through nested arrays and records in both directions. It must test malformed envelopes, the 4096-digit boundary, whole-message byte limits, and old/new bridge combinations. Production work needs a separate review of capability negotiation, generated types, and migration cost. + +## Dataclass output option + +The generation option is `dataclassReturns: 'validated-v2'` for a callable with an annotated dataclass result. Dataclass inputs remain unsupported. An ordinary class instance does not qualify. The generator compiles `dataclasses.fields` into a structural TypeScript record. It includes fields with defaults and `init=False` because an instance returns them. `Optional[T]` permits `null`; a default does not make the returned property optional. + +The generated wrapper first calls `meta` and requires `valueCapabilities` to contain `dataclassFieldsV2`. It sends `params.valuePolicy.dataclass = 'fields-v2'` on each call. The Python encoder accepts only the compiled dataclass type. It emits a version 2 envelope with the canonical `module.qualname` identity and an ordered `fields` record. It rejects unsupported fields and cycles with paths. It uses the existing depth, node, and payload limits. It does not call `dataclasses.asdict` or create object handles. + +The TypeScript decoder validates the marker, version, encoding, type identity, and field record before it strips the envelope. It returns a plain record and stores its verified type identity in a module-private `WeakMap`. The generated return validator reads that `WeakMap` before it checks exact keys and field values. An ordinary record cannot claim dataclass origin, even if it has matching keys. A nested dataclass gets its own verified entry. A malformed or unknown version fails before the decoder returns a record. + +The bounded prototype must show a generated `Promise` wrapper returning `{x: 1, y: 2}` from the actual compiler model. It must reject missing, extra, and wrong typed fields. It must test optional/defaulted fields, nesting, cycles, unsupported fields, depth, and byte limits. Production work needs an independent review of identity, generated types, and migration cost. From f9bea9c06586d76ee4e638601372f92b522fb525 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 00:25:35 -0700 Subject: [PATCH 02/35] test(codec): prototype exact integers and dataclass output --- test/prototypes/value_extensions.py | 276 ++++++++++++++++++ .../python/test_value_extensions_prototype.py | 135 +++++++++ 2 files changed, 411 insertions(+) create mode 100644 test/prototypes/value_extensions.py create mode 100644 test/python/test_value_extensions_prototype.py diff --git a/test/prototypes/value_extensions.py b/test/prototypes/value_extensions.py new file mode 100644 index 00000000..68b8bb6d --- /dev/null +++ b/test/prototypes/value_extensions.py @@ -0,0 +1,276 @@ +"""Isolated wire prototypes for #337 and #339. Production never imports this file.""" + +from __future__ import annotations + +import dataclasses +import json +import math +import re +import sys +from collections.abc import Mapping +from dataclasses import dataclass +from typing import Any + + +MAX_DECIMAL_DIGITS = 4096 +MAX_DEPTH = 64 +MAX_NODES = 100_000 +MAX_PAYLOAD_BYTES = 10 * 1024 * 1024 +SAFE_INTEGER_MAX = 2**53 - 1 +DECIMAL = re.compile(r'(?:0|[1-9][0-9]*|-[1-9][0-9]*)\Z', re.ASCII) +INTEGER_FIELDS = frozenset({'__tywrap__', 'codecVersion', 'encoding', 'value'}) + + +class PrototypeError(ValueError): + """A value cannot satisfy the proposed extension contract.""" + + +@dataclass +class Budget: + nodes: int = 0 + + def visit(self, path: str, depth: int) -> None: + self.nodes += 1 + if depth > MAX_DEPTH: + raise PrototypeError(f'maximum depth {MAX_DEPTH} exceeded at {path}') + if self.nodes > MAX_NODES: + raise PrototypeError(f'maximum nodes {MAX_NODES} exceeded at {path}') + + +def _check_payload(value: object, max_payload_bytes: int) -> None: + encoded = json.dumps(value, allow_nan=False, separators=(',', ':')) + if len(encoded.encode('utf-8')) > max_payload_bytes: + raise PrototypeError(f'payload exceeds {max_payload_bytes} bytes') + + +def _child(path: str, key: str | int) -> str: + return f'{path}[{key}]' if isinstance(key, int) else f'{path}.{key}' + + +def require_capability(meta: Mapping[str, object], feature: str, policy: str) -> None: + capabilities = meta.get('valueCapabilities') + if not isinstance(capabilities, list) or feature not in capabilities: + raise PrototypeError(f'bridge lacks {feature} capability') + expected_policy = { + 'exactIntegerDecimalV2': 'bigint-v2', + 'dataclassFieldsV2': 'fields-v2', + }.get(feature) + if policy != expected_policy: + raise PrototypeError(f'unsupported per-call value policy {policy!r}') + + +def _canonical_decimal(value: object, path: str) -> int: + if not isinstance(value, str) or not DECIMAL.fullmatch(value): + raise PrototypeError(f'noncanonical integer decimal at {path}') + digits = len(value) - (1 if value.startswith('-') else 0) + if digits > MAX_DECIMAL_DIGITS: + raise PrototypeError(f'integer exceeds {MAX_DECIMAL_DIGITS} digits at {path}') + return int(value) + + +def _integer_envelope(value: int, path: str) -> dict[str, object]: + decimal = str(value) + _canonical_decimal(decimal, path) + return { + '__tywrap__': 'integer', + 'codecVersion': 2, + 'encoding': 'decimal', + 'value': decimal, + } + + +def encode_exact_integers( + value: object, + *, + max_payload_bytes: int = MAX_PAYLOAD_BYTES, +) -> object: + """Encode all Python integers as exact tagged values for an opted call.""" + budget = Budget() + active: set[int] = set() + + def visit(current: object, path: str, depth: int) -> object: + budget.visit(path, depth) + if current is None or type(current) in (bool, str): + return current + if type(current) is int: + return _integer_envelope(current, path) + if type(current) is float and math.isfinite(current): + return current + if isinstance(current, (list, tuple, dict)): + marker = id(current) + if marker in active: + raise PrototypeError(f'cycle at {path}') + active.add(marker) + try: + if isinstance(current, dict): + if any(not isinstance(key, str) for key in current): + raise PrototypeError(f'non-string record key at {path}') + return { + key: visit(item, _child(path, key), depth + 1) + for key, item in current.items() + } + return [ + visit(item, _child(path, index), depth + 1) + for index, item in enumerate(current) + ] + finally: + active.remove(marker) + raise PrototypeError(f'unsupported value at {path}: {type(current).__name__}') + + result = visit(value, 'result', 0) + _check_payload(result, max_payload_bytes) + return result + + +def decode_exact_integers( + value: object, + *, + max_payload_bytes: int = MAX_PAYLOAD_BYTES, +) -> object: + """Decode only version 2 integer tags for an opted call.""" + _check_payload(value, max_payload_bytes) + budget = Budget() + active: set[int] = set() + + def visit(current: object, path: str, depth: int) -> object: + budget.visit(path, depth) + if current is None or type(current) in (bool, str): + return current + if type(current) is int: + raise PrototypeError(f'untagged integer at {path}') + if type(current) is float and math.isfinite(current): + return current + if isinstance(current, (list, dict)): + marker = id(current) + if marker in active: + raise PrototypeError(f'cycle at {path}') + active.add(marker) + try: + if isinstance(current, dict): + if '__tywrap__' in current: + if ( + set(current) != INTEGER_FIELDS + or current.get('__tywrap__') != 'integer' + or current.get('codecVersion') != 2 + or current.get('encoding') != 'decimal' + ): + raise PrototypeError(f'invalid integer envelope at {path}') + return _canonical_decimal(current['value'], _child(path, 'value')) + if any(not isinstance(key, str) for key in current): + raise PrototypeError(f'non-string record key at {path}') + return { + key: visit(item, _child(path, key), depth + 1) + for key, item in current.items() + } + return [ + visit(item, _child(path, index), depth + 1) + for index, item in enumerate(current) + ] + finally: + active.remove(marker) + raise PrototypeError(f'unsupported value at {path}: {type(current).__name__}') + + return visit(value, 'args', 0) + + +def encode_dataclass( + value: object, + expected_type: type, + *, + max_payload_bytes: int = MAX_PAYLOAD_BYTES, +) -> dict[str, object]: + """Encode only an annotated dataclass result as a version 2 envelope.""" + if type(value) is not expected_type or not dataclasses.is_dataclass(value): + raise PrototypeError('result is not the declared dataclass type') + budget = Budget() + active: set[int] = set() + + def visit(current: object, path: str, depth: int) -> object: + budget.visit(path, depth) + if current is None or type(current) in (bool, str): + return current + if type(current) is int: + if current < -SAFE_INTEGER_MAX or current > SAFE_INTEGER_MAX: + raise PrototypeError(f'unsafe integer at {path}') + return current + if type(current) is float and math.isfinite(current): + return current + if dataclasses.is_dataclass(current) and not isinstance(current, type): + marker = id(current) + if marker in active: + raise PrototypeError(f'cycle at {path}') + active.add(marker) + try: + cls = type(current) + fields = { + field.name: visit( + getattr(current, field.name), + _child(path, field.name), + depth + 1, + ) + for field in dataclasses.fields(current) + } + return { + '__tywrap__': 'dataclass', + 'codecVersion': 2, + 'encoding': 'fields', + 'type': f'{cls.__module__}.{cls.__qualname__}', + 'fields': fields, + } + finally: + active.remove(marker) + if isinstance(current, (list, tuple, dict)): + marker = id(current) + if marker in active: + raise PrototypeError(f'cycle at {path}') + active.add(marker) + try: + if isinstance(current, dict): + if any(not isinstance(key, str) for key in current): + raise PrototypeError(f'non-string record key at {path}') + return { + key: visit(item, _child(path, key), depth + 1) + for key, item in current.items() + } + return [ + visit(item, _child(path, index), depth + 1) + for index, item in enumerate(current) + ] + finally: + active.remove(marker) + raise PrototypeError(f'unsupported value at {path}: {type(current).__name__}') + + result = visit(value, 'result', 0) + assert isinstance(result, dict) + _check_payload(result, max_payload_bytes) + return result + + +@dataclass +class Point: + x: int + y: int + + +def _main() -> None: + raw = sys.stdin.buffer.read(MAX_PAYLOAD_BYTES + 1) + if len(raw) > MAX_PAYLOAD_BYTES: + raise PrototypeError('input payload exceeds byte limit') + payload: Any = json.loads(raw) + action = sys.argv[1] + if action == 'roundtrip-integer': + require_capability( + payload['meta'], 'exactIntegerDecimalV2', payload['policy'] + ) + result = encode_exact_integers(decode_exact_integers(payload['value'])) + elif action == 'encode-integer': + result = encode_exact_integers(payload) + elif action == 'encode-point': + result = encode_dataclass(Point(**payload), Point) + else: + raise PrototypeError(f'unknown action {action!r}') + print(json.dumps(result, allow_nan=False, separators=(',', ':'))) + + +if __name__ == '__main__': + _main() diff --git a/test/python/test_value_extensions_prototype.py b/test/python/test_value_extensions_prototype.py new file mode 100644 index 00000000..12fd3543 --- /dev/null +++ b/test/python/test_value_extensions_prototype.py @@ -0,0 +1,135 @@ +"""Bounded design tests. These codecs are not part of the shipped bridge.""" + +from __future__ import annotations + +import sys +from dataclasses import dataclass, field +from pathlib import Path + +import pytest + + +sys.path.insert(0, str(Path(__file__).parent.parent / 'prototypes')) +import value_extensions as prototype # noqa: E402 + + +def test_nested_exact_integers_round_trip_beyond_int64() -> None: + value = { + 'outer': [2**80 + 1, {'negative': -(2**130 + 7)}], + 'safe': 7, + 'flag': True, + } + encoded = prototype.encode_exact_integers(value) + assert encoded['outer'][0]['value'] == str(2**80 + 1) + assert encoded['safe']['value'] == '7' + assert encoded['flag'] is True + assert prototype.decode_exact_integers(encoded) == value + + +@pytest.mark.parametrize('bad', ['-0', '+1', '01', '-01', '1.0', '1e2', ' 1', '١']) +def test_integer_decimal_rejects_noncanonical_values(bad: str) -> None: + envelope = { + '__tywrap__': 'integer', + 'codecVersion': 2, + 'encoding': 'decimal', + 'value': bad, + } + with pytest.raises(prototype.PrototypeError, match='noncanonical integer decimal'): + prototype.decode_exact_integers({'item': [envelope]}) + + +def test_integer_digit_and_payload_caps() -> None: + accepted = int('9' * prototype.MAX_DECIMAL_DIGITS) + assert prototype.decode_exact_integers( + prototype.encode_exact_integers(accepted) + ) == accepted + with pytest.raises(prototype.PrototypeError, match='4096 digits'): + prototype.encode_exact_integers(int('9' * (prototype.MAX_DECIMAL_DIGITS + 1))) + with pytest.raises(prototype.PrototypeError, match='payload exceeds 80 bytes'): + prototype.encode_exact_integers({'text': 'x' * 80}, max_payload_bytes=80) + + +def test_integer_envelope_and_capability_fail_closed() -> None: + encoded = prototype.encode_exact_integers(2**80) + for change in ( + {'codecVersion': 1}, + {'encoding': 'json'}, + {'unexpected': True}, + ): + with pytest.raises(prototype.PrototypeError, match='invalid integer envelope'): + prototype.decode_exact_integers({**encoded, **change}) + with pytest.raises(prototype.PrototypeError, match='untagged integer at args'): + prototype.decode_exact_integers(42) + with pytest.raises(prototype.PrototypeError, match='bridge lacks'): + prototype.require_capability({}, 'exactIntegerDecimalV2', 'bigint-v2') + with pytest.raises(prototype.PrototypeError, match='unsupported per-call'): + prototype.require_capability( + {'valueCapabilities': ['exactIntegerDecimalV2']}, + 'exactIntegerDecimalV2', + 'fields-v2', + ) + + +def test_integer_cycles_and_depth_reject_with_path() -> None: + cycle: list[object] = [] + cycle.append(cycle) + with pytest.raises(prototype.PrototypeError, match=r'cycle at result\[0\]'): + prototype.encode_exact_integers(cycle) + nested: object = 1 + for _ in range(prototype.MAX_DEPTH + 1): + nested = [nested] + with pytest.raises(prototype.PrototypeError, match='maximum depth'): + prototype.encode_exact_integers(nested) + + +def test_dataclass_point_fields_and_identity() -> None: + encoded = prototype.encode_dataclass(prototype.Point(1, 2), prototype.Point) + assert encoded == { + '__tywrap__': 'dataclass', + 'codecVersion': 2, + 'encoding': 'fields', + 'type': 'value_extensions.Point', + 'fields': {'x': 1, 'y': 2}, + } + + @dataclass + class Record: + name: str + optional: int | None = None + output: int = field(init=False, default=3) + + record = prototype.encode_dataclass(Record('a'), Record) + assert record['fields'] == {'name': 'a', 'optional': None, 'output': 3} + + +def test_dataclass_nested_values_and_failures() -> None: + @dataclass + class Container: + points: list[prototype.Point] + + encoded = prototype.encode_dataclass(Container([prototype.Point(1, 2)]), Container) + assert encoded['fields']['points'][0]['fields'] == {'x': 1, 'y': 2} + + with pytest.raises(prototype.PrototypeError, match='declared dataclass type'): + prototype.encode_dataclass(prototype.Point(1, 2), Container) + + @dataclass + class Unsupported: + value: object + + with pytest.raises(prototype.PrototypeError, match='unsupported value at result.value'): + prototype.encode_dataclass(Unsupported(object()), Unsupported) + + @dataclass + class Cyclic: + value: object = None + + cyclic = Cyclic() + cyclic.value = cyclic + with pytest.raises(prototype.PrototypeError, match='cycle at result.value'): + prototype.encode_dataclass(cyclic, Cyclic) + + with pytest.raises(prototype.PrototypeError, match='payload exceeds 60 bytes'): + prototype.encode_dataclass( + prototype.Point(1, 2), prototype.Point, max_payload_bytes=60 + ) From 4040b7473237a591565f7498bafe19bfb6ef168a Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 00:30:32 -0700 Subject: [PATCH 03/35] test(codec): prototype cross-language value extensions --- test/prototypes/value_extensions.ts | 241 ++++++++++++++++++++++++ test/value_extensions_prototype.test.ts | 132 +++++++++++++ 2 files changed, 373 insertions(+) create mode 100644 test/prototypes/value_extensions.ts create mode 100644 test/value_extensions_prototype.test.ts diff --git a/test/prototypes/value_extensions.ts b/test/prototypes/value_extensions.ts new file mode 100644 index 00000000..d297ef61 --- /dev/null +++ b/test/prototypes/value_extensions.ts @@ -0,0 +1,241 @@ +/** Isolated value extension prototype. Shipped codecs never import this file. */ + +export type PrototypeContract = + | { readonly kind: 'integer' } + | { readonly kind: 'safe-integer' } + | { readonly kind: 'float' } + | { readonly kind: 'boolean' } + | { readonly kind: 'string' } + | { readonly kind: 'null' } + | { readonly kind: 'nullable'; readonly value: PrototypeContract } + | { readonly kind: 'array'; readonly item: PrototypeContract } + | { readonly kind: 'record'; readonly fields: Readonly> } + | { + readonly kind: 'dataclass'; + readonly typeId: string; + readonly fields: Readonly>; + }; + +const MAX_DECIMAL_DIGITS = 4096; +const MAX_DEPTH = 64; +const MAX_NODES = 100_000; +const MAX_PAYLOAD_BYTES = 10 * 1024 * 1024; +const DECIMAL = /^(?:0|[1-9][0-9]*|-[1-9][0-9]*)$/; +const provenance = new WeakMap(); + +export class PrototypeError extends Error {} + +function isRecord(value: unknown): value is Record { + if (typeof value !== 'object' || value === null || Array.isArray(value)) { + return false; + } + const prototype = Object.getPrototypeOf(value); + return prototype === Object.prototype || prototype === null; +} + +function exactKeys( + value: Record, + expected: readonly string[], + path: string +): void { + const actual = Object.keys(value).sort(); + const required = [...expected].sort(); + if (actual.length !== required.length || actual.some((key, index) => key !== required[index])) { + throw new PrototypeError(`fields differ at ${path}`); + } +} + +function checkPayload(value: unknown, limit: number): void { + const encoded = JSON.stringify(value); + if (encoded === undefined || new TextEncoder().encode(encoded).length > limit) { + throw new PrototypeError(`payload exceeds ${limit} bytes`); + } +} + +function canonicalDecimal(value: unknown, path: string): bigint { + if (typeof value !== 'string' || !DECIMAL.test(value)) { + throw new PrototypeError(`noncanonical integer decimal at ${path}`); + } + const digits = value.startsWith('-') ? value.length - 1 : value.length; + if (digits > MAX_DECIMAL_DIGITS) { + throw new PrototypeError(`integer exceeds ${MAX_DECIMAL_DIGITS} digits at ${path}`); + } + return BigInt(value); +} + +export function requireCapability(meta: unknown, feature: string, policy: string): void { + const capabilities = isRecord(meta) ? meta.valueCapabilities : undefined; + if (!Array.isArray(capabilities) || !capabilities.includes(feature)) { + throw new PrototypeError(`bridge lacks ${feature} capability`); + } + const expected = + feature === 'exactIntegerDecimalV2' + ? 'bigint-v2' + : feature === 'dataclassFieldsV2' + ? 'fields-v2' + : undefined; + if (policy !== expected) { + throw new PrototypeError(`unsupported per-call value policy ${policy}`); + } +} + +interface WalkState { + nodes: number; + active: WeakSet; +} + +function visit(state: WalkState, value: unknown, path: string, depth: number): void { + state.nodes += 1; + if (depth > MAX_DEPTH) { + throw new PrototypeError(`maximum depth ${MAX_DEPTH} exceeded at ${path}`); + } + if (state.nodes > MAX_NODES) { + throw new PrototypeError(`maximum nodes ${MAX_NODES} exceeded at ${path}`); + } + if (typeof value === 'object' && value !== null) { + if (state.active.has(value)) { + throw new PrototypeError(`cycle at ${path}`); + } + state.active.add(value); + } +} + +function leave(state: WalkState, value: unknown): void { + if (typeof value === 'object' && value !== null) { + state.active.delete(value); + } +} + +function walk( + value: unknown, + contract: PrototypeContract, + mode: 'encode' | 'decode', + path: string, + depth: number, + state: WalkState +): unknown { + if (contract.kind === 'nullable') { + return value === null + ? walk(null, { kind: 'null' }, mode, path, depth, state) + : walk(value, contract.value, mode, path, depth, state); + } + visit(state, value, path, depth); + try { + if (contract.kind === 'null') { + if (value !== null) throw new PrototypeError(`expected null at ${path}`); + return null; + } + if (contract.kind === 'boolean' || contract.kind === 'string') { + if (typeof value !== contract.kind) { + throw new PrototypeError(`expected ${contract.kind} at ${path}`); + } + return value; + } + if (contract.kind === 'float') { + if (typeof value !== 'number' || !Number.isFinite(value)) { + throw new PrototypeError(`expected finite float at ${path}`); + } + return value; + } + if (contract.kind === 'safe-integer') { + if (typeof value !== 'number' || !Number.isSafeInteger(value)) { + throw new PrototypeError(`expected safe integer at ${path}`); + } + return value; + } + if (contract.kind === 'integer') { + if (mode === 'encode') { + if (typeof value !== 'bigint') { + throw new PrototypeError(`expected bigint at ${path}`); + } + const decimal = value.toString(); + canonicalDecimal(decimal, path); + return { __tywrap__: 'integer', codecVersion: 2, encoding: 'decimal', value: decimal }; + } + if (!isRecord(value)) { + throw new PrototypeError(`expected integer envelope at ${path}`); + } + exactKeys(value, ['__tywrap__', 'codecVersion', 'encoding', 'value'], path); + if ( + value.__tywrap__ !== 'integer' || + value.codecVersion !== 2 || + value.encoding !== 'decimal' + ) { + throw new PrototypeError(`invalid integer envelope at ${path}`); + } + return canonicalDecimal(value.value, `${path}.value`); + } + if (contract.kind === 'array') { + if (!Array.isArray(value)) throw new PrototypeError(`expected array at ${path}`); + return value.map((item, index) => + walk(item, contract.item, mode, `${path}[${index}]`, depth + 1, state) + ); + } + if (contract.kind === 'record') { + if (!isRecord(value)) throw new PrototypeError(`expected record at ${path}`); + exactKeys(value, Object.keys(contract.fields), path); + return Object.fromEntries( + Object.entries(contract.fields).map(([key, field]) => [ + key, + walk(value[key], field, mode, `${path}.${key}`, depth + 1, state), + ]) + ); + } + if (mode === 'encode') { + throw new PrototypeError(`dataclass inputs are unsupported at ${path}`); + } + if (!isRecord(value)) throw new PrototypeError(`expected dataclass envelope at ${path}`); + exactKeys(value, ['__tywrap__', 'codecVersion', 'encoding', 'type', 'fields'], path); + if ( + value.__tywrap__ !== 'dataclass' || + value.codecVersion !== 2 || + value.encoding !== 'fields' || + value.type !== contract.typeId || + !isRecord(value.fields) + ) { + throw new PrototypeError(`invalid dataclass identity at ${path}`); + } + exactKeys(value.fields, Object.keys(contract.fields), `${path}.fields`); + const decoded = Object.fromEntries( + Object.entries(contract.fields).map(([key, field]) => [ + key, + walk(value.fields[key], field, 'decode', `${path}.fields.${key}`, depth + 1, state), + ]) + ); + provenance.set(decoded, contract); + return decoded; + } finally { + leave(state, value); + } +} + +export function encodeExactRequest( + value: unknown, + contract: PrototypeContract, + maxPayloadBytes = MAX_PAYLOAD_BYTES +): unknown { + const encoded = walk(value, contract, 'encode', 'args', 0, { + nodes: 0, + active: new WeakSet(), + }); + checkPayload(encoded, maxPayloadBytes); + return encoded; +} + +export function decodeExactResponse( + value: unknown, + contract: PrototypeContract, + maxPayloadBytes = MAX_PAYLOAD_BYTES +): unknown { + checkPayload(value, maxPayloadBytes); + return walk(value, contract, 'decode', 'result', 0, { + nodes: 0, + active: new WeakSet(), + }); +} + +export function validateDataclassOrigin(value: unknown, contract: PrototypeContract): void { + if (contract.kind !== 'dataclass' || !isRecord(value) || provenance.get(value) !== contract) { + throw new PrototypeError('value lacks verified dataclass provenance'); + } +} diff --git a/test/value_extensions_prototype.test.ts b/test/value_extensions_prototype.test.ts new file mode 100644 index 00000000..afe09ac5 --- /dev/null +++ b/test/value_extensions_prototype.test.ts @@ -0,0 +1,132 @@ +import { execFileSync } from 'node:child_process'; +import { existsSync } from 'node:fs'; +import { join } from 'node:path'; +import { describe, expect, it } from 'vitest'; +import { PYTHON, PYTHON_AVAILABLE } from './helpers/python-probe.js'; +import { + decodeExactResponse, + encodeExactRequest, + requireCapability, + validateDataclassOrigin, + type PrototypeContract, +} from './prototypes/value_extensions.js'; + +const pythonScript = join(process.cwd(), 'test', 'prototypes', 'value_extensions.py'); +const pythonPath = PYTHON ?? 'python3'; + +function pythonAction(action: string, input: string): unknown { + return JSON.parse( + execFileSync(pythonPath, [pythonScript, action], { + input, + encoding: 'utf8', + maxBuffer: 1024 * 1024, + }) + ) as unknown; +} + +const integerContract: PrototypeContract = { + kind: 'record', + fields: { + positive: { kind: 'array', item: { kind: 'integer' } }, + negative: { kind: 'record', fields: { value: { kind: 'integer' } } }, + safe: { kind: 'integer' }, + flag: { kind: 'boolean' }, + }, +}; + +describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( + 'exact integer cross-language prototype', + () => { + it('round-trips nested positive and negative values beyond int64 in both directions', () => { + const positive = 2n ** 80n + 1n; + const negative = -(2n ** 130n + 7n); + const value = { positive: [positive], negative: { value: negative }, safe: 7n, flag: true }; + + const pythonWire = pythonAction( + 'encode-integer', + `{"positive":[${positive}],"negative":{"value":${negative}},"safe":7,"flag":true}` + ); + expect(decodeExactResponse(pythonWire, integerContract)).toEqual(value); + + requireCapability( + { valueCapabilities: ['exactIntegerDecimalV2'] }, + 'exactIntegerDecimalV2', + 'bigint-v2' + ); + const request = encodeExactRequest(value, integerContract); + const response = pythonAction( + 'roundtrip-integer', + JSON.stringify({ + meta: { valueCapabilities: ['exactIntegerDecimalV2'] }, + policy: 'bigint-v2', + value: request, + }) + ); + expect(decodeExactResponse(response, integerContract)).toEqual(value); + }); + } +); + +describe('exact integer rejection prototype', () => { + const integer: PrototypeContract = { kind: 'integer' }; + + it('rejects an old bridge, a number under bigint typing, and a version 1 tag', () => { + expect(() => requireCapability({}, 'exactIntegerDecimalV2', 'bigint-v2')).toThrow( + /bridge lacks/ + ); + expect(() => encodeExactRequest(42, integer)).toThrow(/expected bigint at args/); + expect(() => + decodeExactResponse( + { __tywrap__: 'integer', codecVersion: 1, encoding: 'decimal', value: '42' }, + integer + ) + ).toThrow(/invalid integer envelope/); + }); + + it.each(['-0', '+1', '01', '-01', '1.0', '1e2', ' 1', '١'])( + 'rejects noncanonical decimal %s', + decimal => { + expect(() => + decodeExactResponse( + { __tywrap__: 'integer', codecVersion: 2, encoding: 'decimal', value: decimal }, + integer + ) + ).toThrow(/noncanonical integer decimal/); + } + ); + + it('enforces digit and payload limits', () => { + const max = BigInt('9'.repeat(4096)); + expect(decodeExactResponse(encodeExactRequest(max, integer), integer)).toBe(max); + expect(() => encodeExactRequest(BigInt('9'.repeat(4097)), integer)).toThrow(/4096 digits/); + expect(() => encodeExactRequest(1n, integer, 20)).toThrow(/payload exceeds 20 bytes/); + }); +}); + +describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( + 'dataclass output prototype', + () => { + const point: PrototypeContract = { + kind: 'dataclass', + typeId: '__main__.Point', + fields: { x: { kind: 'safe-integer' }, y: { kind: 'safe-integer' } }, + }; + + it('decodes Point and preserves verified origin', () => { + const wire = pythonAction('encode-point', JSON.stringify({ x: 1, y: 2 })); + const decoded = decodeExactResponse(wire, point); + expect(decoded).toEqual({ x: 1, y: 2 }); + expect(() => validateDataclassOrigin(decoded, point)).not.toThrow(); + expect(() => validateDataclassOrigin({ x: 1, y: 2 }, point)).toThrow(/lacks verified/); + expect(() => + decodeExactResponse({ ...(wire as object), type: 'other.Point' }, point) + ).toThrow(/invalid dataclass identity/); + expect(() => decodeExactResponse({ ...(wire as object), fields: { x: 1 } }, point)).toThrow( + /fields differ/ + ); + expect(() => + decodeExactResponse({ ...(wire as object), fields: { x: '1', y: 2 } }, point) + ).toThrow(/expected safe integer/); + }); + } +); From e460a4f64aa1ad7f62b01b1406f762f42ae8e447 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 00:34:22 -0700 Subject: [PATCH 04/35] test(codec): reject ambiguous prototype envelopes --- docs/maintainers/value-contract-prototypes.md | 2 ++ test/prototypes/value_extensions.py | 5 +++++ test/prototypes/value_extensions.ts | 3 +++ test/python/test_value_extensions_prototype.py | 3 +++ test/value_extensions_prototype.test.ts | 10 ++++++++++ 5 files changed, 23 insertions(+) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index e91a9021..0ac8eb8d 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -12,6 +12,8 @@ The version 2 integer envelope is `{"__tywrap__":"integer","codecVersion":2,"enc The request encoder tags every `bigint`, including safe values. The Python request decoder yields `int`. The Python response encoder tags every `int`, including safe values. The TypeScript response decoder yields `bigint`. Both walk nested arrays and string-keyed records within the existing depth and node limits. They reject cycles and unsupported values with paths. Arrow int64 keeps its existing Arrow encoding and decoded type. +An ordinary record cannot use `__tywrap__` as a key in this mode. The encoder rejects it because the decoder reserves that key for envelopes. + The generated return validator requires `bigint` at every integer node. It must reject an ordinary number and a raw tagged record. A wrapper with `integerMode: 'bigint'` therefore cannot expose `Promise` for a Python integer. The bounded prototype must round trip positive and negative values larger than 64 bits through nested arrays and records in both directions. It must test malformed envelopes, the 4096-digit boundary, whole-message byte limits, and old/new bridge combinations. Production work needs a separate review of capability negotiation, generated types, and migration cost. diff --git a/test/prototypes/value_extensions.py b/test/prototypes/value_extensions.py index 68b8bb6d..d8d2b1e3 100644 --- a/test/prototypes/value_extensions.py +++ b/test/prototypes/value_extensions.py @@ -105,6 +105,8 @@ def visit(current: object, path: str, depth: int) -> object: if isinstance(current, dict): if any(not isinstance(key, str) for key in current): raise PrototypeError(f'non-string record key at {path}') + if '__tywrap__' in current: + raise PrototypeError(f'reserved record key at {path}.__tywrap__') return { key: visit(item, _child(path, key), depth + 1) for key, item in current.items() @@ -151,6 +153,7 @@ def visit(current: object, path: str, depth: int) -> object: if ( set(current) != INTEGER_FIELDS or current.get('__tywrap__') != 'integer' + or type(current.get('codecVersion')) is not int or current.get('codecVersion') != 2 or current.get('encoding') != 'decimal' ): @@ -228,6 +231,8 @@ def visit(current: object, path: str, depth: int) -> object: if isinstance(current, dict): if any(not isinstance(key, str) for key in current): raise PrototypeError(f'non-string record key at {path}') + if '__tywrap__' in current: + raise PrototypeError(f'reserved record key at {path}.__tywrap__') return { key: visit(item, _child(path, key), depth + 1) for key, item in current.items() diff --git a/test/prototypes/value_extensions.ts b/test/prototypes/value_extensions.ts index d297ef61..8742d71b 100644 --- a/test/prototypes/value_extensions.ts +++ b/test/prototypes/value_extensions.ts @@ -173,6 +173,9 @@ function walk( } if (contract.kind === 'record') { if (!isRecord(value)) throw new PrototypeError(`expected record at ${path}`); + if (Object.hasOwn(value, '__tywrap__')) { + throw new PrototypeError(`reserved record key at ${path}.__tywrap__`); + } exactKeys(value, Object.keys(contract.fields), path); return Object.fromEntries( Object.entries(contract.fields).map(([key, field]) => [ diff --git a/test/python/test_value_extensions_prototype.py b/test/python/test_value_extensions_prototype.py index 12fd3543..ecc80a59 100644 --- a/test/python/test_value_extensions_prototype.py +++ b/test/python/test_value_extensions_prototype.py @@ -53,6 +53,7 @@ def test_integer_envelope_and_capability_fail_closed() -> None: encoded = prototype.encode_exact_integers(2**80) for change in ( {'codecVersion': 1}, + {'codecVersion': 2.0}, {'encoding': 'json'}, {'unexpected': True}, ): @@ -60,6 +61,8 @@ def test_integer_envelope_and_capability_fail_closed() -> None: prototype.decode_exact_integers({**encoded, **change}) with pytest.raises(prototype.PrototypeError, match='untagged integer at args'): prototype.decode_exact_integers(42) + with pytest.raises(prototype.PrototypeError, match='reserved record key at result.__tywrap__'): + prototype.encode_exact_integers({'__tywrap__': 'ordinary'}) with pytest.raises(prototype.PrototypeError, match='bridge lacks'): prototype.require_capability({}, 'exactIntegerDecimalV2', 'bigint-v2') with pytest.raises(prototype.PrototypeError, match='unsupported per-call'): diff --git a/test/value_extensions_prototype.test.ts b/test/value_extensions_prototype.test.ts index afe09ac5..e35fc6f9 100644 --- a/test/value_extensions_prototype.test.ts +++ b/test/value_extensions_prototype.test.ts @@ -101,6 +101,16 @@ describe('exact integer rejection prototype', () => { expect(() => encodeExactRequest(BigInt('9'.repeat(4097)), integer)).toThrow(/4096 digits/); expect(() => encodeExactRequest(1n, integer, 20)).toThrow(/payload exceeds 20 bytes/); }); + + it('rejects a record key reserved for envelopes', () => { + const record: PrototypeContract = { + kind: 'record', + fields: { __tywrap__: { kind: 'string' } }, + }; + expect(() => encodeExactRequest({ __tywrap__: 'ordinary' }, record)).toThrow( + /reserved record key/ + ); + }); }); describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( From 30453fce2b0ba8863296d6559b48bdea02c61866 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 00:51:16 -0700 Subject: [PATCH 05/35] test(codec): decode exact requests with declared float nodes --- docs/maintainers/value-contract-prototypes.md | 78 ++++++++-- test/prototypes/value_extensions.py | 145 ++++++++++++++---- test/prototypes/value_extensions.ts | 14 ++ .../python/test_value_extensions_prototype.py | 74 ++++++++- test/value_extensions_prototype.test.ts | 43 +++++- 5 files changed, 302 insertions(+), 52 deletions(-) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index 0ac8eb8d..fa031299 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -1,29 +1,83 @@ # Value extension prototypes -These rules are candidates for #337 and #339. They do not change the 0.11 codec policy. The prototypes live under `test/prototypes` and cannot enable either feature in shipped wrappers. +These rules are candidates for #337 and #339. They do not change the 0.11 codec +policy. The prototypes live under `test/prototypes`. Shipped wrappers cannot +import them. ## Exact integer option -The generation option is `integerMode: 'bigint'` for one callable. It changes every Python `int` in that callable's parameters and result to TypeScript `bigint`. Safe integers use the same type as large integers in this mode. Python `bool` remains `boolean`. Calls without the option keep the safe `number` rule. +The generation option is `integerMode: 'bigint'` for one callable. It maps every +Python `int` parameter and result to TypeScript `bigint`. Safe integers use the +same type. Python `bool` remains `boolean`. Calls without the option keep the +safe `number` rule. -The generated wrapper first calls `meta` and requires `valueCapabilities` to contain `exactIntegerDecimalV2`. The option then travels on each `call` request as `params.valuePolicy.integer = 'bigint-v2'`. A new bridge rejects an unknown value policy. An old bridge lacks the capability, so the wrapper rejects it before sending a tagged value. A new bridge sends tagged integers only for an opted call. Existing clients never opt in. +The generated wrapper requires `exactIntegerDecimalV2` in the bridge's +`meta.valueCapabilities`. It sends `params.valuePolicy.integer = 'bigint-v2'` on +each call. A new bridge rejects unknown policies. An old bridge lacks the +capability, so the wrapper rejects the call before sending a tag. -The version 2 integer envelope is `{"__tywrap__":"integer","codecVersion":2,"encoding":"decimal","value":"18446744073709551617"}`. The `value` string must match `^(?:0|[1-9][0-9]*|-[1-9][0-9]*)$`. This rejects `-0`, leading zeros, a leading plus, spaces, exponents, and Unicode digits. It allows at most 4096 decimal digits, excluding the sign. The ASCII value occupies at most 4097 bytes. The existing request and response payload byte limits still apply to the whole message. Both decoders reject an unknown version, extra envelope fields, an invalid decimal, or an oversized value before calling `BigInt` or Python `int`. +The version 2 integer envelope is +`{"__tywrap__":"integer","codecVersion":2,"encoding":"decimal","value":"18446744073709551617"}`. +The decimal must match `^(?:0|[1-9][0-9]*|-[1-9][0-9]*)$`. This excludes `-0`, +leading zeros, plus signs, spaces, exponents, and Unicode digits. The value +permits at most 4096 digits, excluding the sign. It uses at most 4097 ASCII +bytes. Whole request and response byte limits still apply. -The request encoder tags every `bigint`, including safe values. The Python request decoder yields `int`. The Python response encoder tags every `int`, including safe values. The TypeScript response decoder yields `bigint`. Both walk nested arrays and string-keyed records within the existing depth and node limits. They reject cycles and unsupported values with paths. Arrow int64 keeps its existing Arrow encoding and decoded type. +The TypeScript encoder tags every `bigint`, including safe values. The Python +request decoder requires the tag at each declared integer node. It rejects raw +JSON numbers there. It uses the trusted resolved value contract to distinguish +integer nodes from float nodes. A generated wrapper must not let a client supply +or change that contract. -An ordinary record cannot use `__tywrap__` as a key in this mode. The encoder rejects it because the decoder reserves that key for envelopes. +`JSON.stringify(1.0)` emits `1`. The Python decoder converts that token to +`float` only at a declared float node. `JSON.stringify(-0)` emits `0`, so the +TypeScript encoder uses +`{"__tywrap__":"float","codecVersion":2,"encoding":"negative-zero"}` for +negative zero. The Python decoder accepts that tag only at a float node. It +rejects malformed float tags and nonfinite numbers. -The generated return validator requires `bigint` at every integer node. It must reject an ordinary number and a raw tagged record. A wrapper with `integerMode: 'bigint'` therefore cannot expose `Promise` for a Python integer. +The Python response encoder tags every `int`, including safe values. The +TypeScript response decoder returns `bigint` at declared integer nodes. The +generated return validator must require `bigint`. It must reject an ordinary +number and a raw tag. Arrow int64 keeps its current Arrow encoding and decoded +type. -The bounded prototype must round trip positive and negative values larger than 64 bits through nested arrays and records in both directions. It must test malformed envelopes, the 4096-digit boundary, whole-message byte limits, and old/new bridge combinations. Production work needs a separate review of capability negotiation, generated types, and migration cost. +Both prototypes walk nested arrays and string-keyed records within depth and +node limits. They reject unsupported values and malformed envelopes. Encoders +reject cycles with paths. Ordinary records cannot use `__tywrap__` as a key +because the decoder reserves it for envelopes. + +The bounded tests must round trip positive and negative values beyond 64 bits. +They must cover floats, booleans, nested records, malformed tags, digit limits, +byte limits, and old bridges. Production work needs separate review of +capability negotiation, generated types, and migration cost. ## Dataclass output option -The generation option is `dataclassReturns: 'validated-v2'` for a callable with an annotated dataclass result. Dataclass inputs remain unsupported. An ordinary class instance does not qualify. The generator compiles `dataclasses.fields` into a structural TypeScript record. It includes fields with defaults and `init=False` because an instance returns them. `Optional[T]` permits `null`; a default does not make the returned property optional. +The generation option is `dataclassReturns: 'validated-v2'` for a callable with +an annotated dataclass result. Dataclass inputs remain unsupported. Ordinary +class instances do not qualify. + +The generator compiles `dataclasses.fields` into a structural TypeScript record. +It includes fields with defaults and `init=False`. An instance returns those +fields. `Optional[T]` permits `null`; a default does not make a returned +property optional. -The generated wrapper first calls `meta` and requires `valueCapabilities` to contain `dataclassFieldsV2`. It sends `params.valuePolicy.dataclass = 'fields-v2'` on each call. The Python encoder accepts only the compiled dataclass type. It emits a version 2 envelope with the canonical `module.qualname` identity and an ordered `fields` record. It rejects unsupported fields and cycles with paths. It uses the existing depth, node, and payload limits. It does not call `dataclasses.asdict` or create object handles. +The generated wrapper requires `dataclassFieldsV2` in `meta.valueCapabilities`. +It sends `params.valuePolicy.dataclass = 'fields-v2'` on each call. The Python +encoder accepts only the compiled dataclass type. It emits a version 2 envelope +with `module.qualname` identity and an ordered `fields` record. It rejects +unsupported fields and cycles with paths. It applies depth, node, and byte +limits. It does not call `dataclasses.asdict` or create object handles. -The TypeScript decoder validates the marker, version, encoding, type identity, and field record before it strips the envelope. It returns a plain record and stores its verified type identity in a module-private `WeakMap`. The generated return validator reads that `WeakMap` before it checks exact keys and field values. An ordinary record cannot claim dataclass origin, even if it has matching keys. A nested dataclass gets its own verified entry. A malformed or unknown version fails before the decoder returns a record. +The TypeScript decoder checks marker, version, encoding, type identity, and +fields before it removes the envelope. It returns a plain record. A +module-private `WeakMap` stores verified type identity. The return validator +reads that entry before it checks field keys and values. A plain record cannot +claim dataclass origin. A nested dataclass gets its own entry. -The bounded prototype must show a generated `Promise` wrapper returning `{x: 1, y: 2}` from the actual compiler model. It must reject missing, extra, and wrong typed fields. It must test optional/defaulted fields, nesting, cycles, unsupported fields, depth, and byte limits. Production work needs an independent review of identity, generated types, and migration cost. +The bounded prototype still needs a generated `Promise` wrapper from the +actual compiler model. It must return `{x: 1, y: 2}` and reject missing, extra, +and wrong typed fields. It must test defaults, optional fields, nesting, cycles, +unsupported fields, depth, and byte limits. Production work needs independent +review of identity, generated types, and migration cost. diff --git a/test/prototypes/value_extensions.py b/test/prototypes/value_extensions.py index d8d2b1e3..ae17e146 100644 --- a/test/prototypes/value_extensions.py +++ b/test/prototypes/value_extensions.py @@ -126,54 +126,105 @@ def visit(current: object, path: str, depth: int) -> object: def decode_exact_integers( value: object, + contract: Mapping[str, Any], *, max_payload_bytes: int = MAX_PAYLOAD_BYTES, ) -> object: - """Decode only version 2 integer tags for an opted call.""" + """Decode an opted request with its trusted resolved value contract.""" _check_payload(value, max_payload_bytes) budget = Budget() active: set[int] = set() - def visit(current: object, path: str, depth: int) -> object: + def visit( + current: object, spec: Mapping[str, Any], path: str, depth: int + ) -> object: + kind = spec.get('kind') + if kind == 'nullable': + if current is None: + return visit(None, {'kind': 'null'}, path, depth) + return visit(current, spec['value'], path, depth) budget.visit(path, depth) - if current is None or type(current) in (bool, str): - return current - if type(current) is int: - raise PrototypeError(f'untagged integer at {path}') - if type(current) is float and math.isfinite(current): - return current - if isinstance(current, (list, dict)): - marker = id(current) + marker = id(current) if isinstance(current, (list, dict)) else None + if marker is not None: if marker in active: raise PrototypeError(f'cycle at {path}') active.add(marker) - try: + try: + if kind == 'null': + if current is not None: + raise PrototypeError(f'expected null at {path}') + return None + if kind == 'boolean': + if type(current) is not bool: + raise PrototypeError(f'expected boolean at {path}') + return current + if kind == 'string': + if type(current) is not str: + raise PrototypeError(f'expected string at {path}') + return current + if kind == 'integer': + if not isinstance(current, dict): + raise PrototypeError(f'untagged integer at {path}') + if ( + set(current) != INTEGER_FIELDS + or current.get('__tywrap__') != 'integer' + or type(current.get('codecVersion')) is not int + or current.get('codecVersion') != 2 + or current.get('encoding') != 'decimal' + ): + raise PrototypeError(f'invalid integer envelope at {path}') + return _canonical_decimal(current['value'], _child(path, 'value')) + if kind == 'safe-integer': + if ( + type(current) is not int + or current < -SAFE_INTEGER_MAX + or current > SAFE_INTEGER_MAX + ): + raise PrototypeError(f'expected safe integer at {path}') + return current + if kind == 'float': if isinstance(current, dict): - if '__tywrap__' in current: - if ( - set(current) != INTEGER_FIELDS - or current.get('__tywrap__') != 'integer' - or type(current.get('codecVersion')) is not int - or current.get('codecVersion') != 2 - or current.get('encoding') != 'decimal' - ): - raise PrototypeError(f'invalid integer envelope at {path}') - return _canonical_decimal(current['value'], _child(path, 'value')) - if any(not isinstance(key, str) for key in current): - raise PrototypeError(f'non-string record key at {path}') - return { - key: visit(item, _child(path, key), depth + 1) - for key, item in current.items() - } + if current != { + '__tywrap__': 'float', + 'codecVersion': 2, + 'encoding': 'negative-zero', + } or type(current.get('codecVersion')) is not int: + raise PrototypeError(f'invalid float envelope at {path}') + return -0.0 + if type(current) not in (int, float): + raise PrototypeError(f'expected finite float at {path}') + try: + result = float(current) + except OverflowError as exc: + raise PrototypeError(f'expected finite float at {path}') from exc + if not math.isfinite(result): + raise PrototypeError(f'expected finite float at {path}') + return result + if kind == 'array': + if not isinstance(current, list): + raise PrototypeError(f'expected array at {path}') return [ - visit(item, _child(path, index), depth + 1) + visit(item, spec['item'], _child(path, index), depth + 1) for index, item in enumerate(current) ] - finally: + if kind == 'record': + if not isinstance(current, dict): + raise PrototypeError(f'expected record at {path}') + fields = spec['fields'] + if '__tywrap__' in current: + raise PrototypeError(f'reserved record key at {path}.__tywrap__') + if set(current) != set(fields): + raise PrototypeError(f'fields differ at {path}') + return { + key: visit(current[key], field, _child(path, key), depth + 1) + for key, field in fields.items() + } + raise PrototypeError(f'unsupported contract kind {kind!r} at {path}') + finally: + if marker is not None: active.remove(marker) - raise PrototypeError(f'unsupported value at {path}: {type(current).__name__}') - return visit(value, 'args', 0) + return visit(value, contract, 'args', 0) def encode_dataclass( @@ -257,6 +308,34 @@ class Point: y: int +INTEGER_TEST_CONTRACT: dict[str, Any] = { + 'kind': 'record', + 'fields': { + 'positive': {'kind': 'array', 'item': {'kind': 'integer'}}, + 'negative': { + 'kind': 'record', + 'fields': {'value': {'kind': 'integer'}}, + }, + 'safe': {'kind': 'integer'}, + 'flag': {'kind': 'boolean'}, + 'whole': {'kind': 'float'}, + 'fraction': {'kind': 'float'}, + 'negativeZero': {'kind': 'float'}, + 'mixed': { + 'kind': 'array', + 'item': { + 'kind': 'record', + 'fields': { + 'quantity': {'kind': 'integer'}, + 'ratio': {'kind': 'float'}, + 'flag': {'kind': 'boolean'}, + }, + }, + }, + }, +} + + def _main() -> None: raw = sys.stdin.buffer.read(MAX_PAYLOAD_BYTES + 1) if len(raw) > MAX_PAYLOAD_BYTES: @@ -267,7 +346,9 @@ def _main() -> None: require_capability( payload['meta'], 'exactIntegerDecimalV2', payload['policy'] ) - result = encode_exact_integers(decode_exact_integers(payload['value'])) + result = encode_exact_integers( + decode_exact_integers(payload['value'], INTEGER_TEST_CONTRACT) + ) elif action == 'encode-integer': result = encode_exact_integers(payload) elif action == 'encode-point': diff --git a/test/prototypes/value_extensions.ts b/test/prototypes/value_extensions.ts index 8742d71b..68112808 100644 --- a/test/prototypes/value_extensions.ts +++ b/test/prototypes/value_extensions.ts @@ -132,9 +132,23 @@ function walk( return value; } if (contract.kind === 'float') { + if (mode === 'decode' && isRecord(value)) { + exactKeys(value, ['__tywrap__', 'codecVersion', 'encoding'], path); + if ( + value.__tywrap__ !== 'float' || + value.codecVersion !== 2 || + value.encoding !== 'negative-zero' + ) { + throw new PrototypeError(`invalid float envelope at ${path}`); + } + return -0; + } if (typeof value !== 'number' || !Number.isFinite(value)) { throw new PrototypeError(`expected finite float at ${path}`); } + if (mode === 'encode' && Object.is(value, -0)) { + return { __tywrap__: 'float', codecVersion: 2, encoding: 'negative-zero' }; + } return value; } if (contract.kind === 'safe-integer') { diff --git a/test/python/test_value_extensions_prototype.py b/test/python/test_value_extensions_prototype.py index ecc80a59..09f63e99 100644 --- a/test/python/test_value_extensions_prototype.py +++ b/test/python/test_value_extensions_prototype.py @@ -2,6 +2,7 @@ from __future__ import annotations +import math import sys from dataclasses import dataclass, field from pathlib import Path @@ -12,10 +13,13 @@ sys.path.insert(0, str(Path(__file__).parent.parent / 'prototypes')) import value_extensions as prototype # noqa: E402 +INTEGER = {'kind': 'integer'} + def test_nested_exact_integers_round_trip_beyond_int64() -> None: value = { - 'outer': [2**80 + 1, {'negative': -(2**130 + 7)}], + 'outer': [2**80 + 1, -(2**130 + 7)], + 'nested': {'negative': -(2**130 + 7)}, 'safe': 7, 'flag': True, } @@ -23,7 +27,16 @@ def test_nested_exact_integers_round_trip_beyond_int64() -> None: assert encoded['outer'][0]['value'] == str(2**80 + 1) assert encoded['safe']['value'] == '7' assert encoded['flag'] is True - assert prototype.decode_exact_integers(encoded) == value + contract = { + 'kind': 'record', + 'fields': { + 'outer': {'kind': 'array', 'item': INTEGER}, + 'nested': {'kind': 'record', 'fields': {'negative': INTEGER}}, + 'safe': INTEGER, + 'flag': {'kind': 'boolean'}, + }, + } + assert prototype.decode_exact_integers(encoded, contract) == value @pytest.mark.parametrize('bad', ['-0', '+1', '01', '-01', '1.0', '1e2', ' 1', '١']) @@ -35,13 +48,19 @@ def test_integer_decimal_rejects_noncanonical_values(bad: str) -> None: 'value': bad, } with pytest.raises(prototype.PrototypeError, match='noncanonical integer decimal'): - prototype.decode_exact_integers({'item': [envelope]}) + prototype.decode_exact_integers( + {'item': [envelope]}, + { + 'kind': 'record', + 'fields': {'item': {'kind': 'array', 'item': INTEGER}}, + }, + ) def test_integer_digit_and_payload_caps() -> None: accepted = int('9' * prototype.MAX_DECIMAL_DIGITS) assert prototype.decode_exact_integers( - prototype.encode_exact_integers(accepted) + prototype.encode_exact_integers(accepted), INTEGER ) == accepted with pytest.raises(prototype.PrototypeError, match='4096 digits'): prototype.encode_exact_integers(int('9' * (prototype.MAX_DECIMAL_DIGITS + 1))) @@ -58,9 +77,9 @@ def test_integer_envelope_and_capability_fail_closed() -> None: {'unexpected': True}, ): with pytest.raises(prototype.PrototypeError, match='invalid integer envelope'): - prototype.decode_exact_integers({**encoded, **change}) + prototype.decode_exact_integers({**encoded, **change}, INTEGER) with pytest.raises(prototype.PrototypeError, match='untagged integer at args'): - prototype.decode_exact_integers(42) + prototype.decode_exact_integers(42, INTEGER) with pytest.raises(prototype.PrototypeError, match='reserved record key at result.__tywrap__'): prototype.encode_exact_integers({'__tywrap__': 'ordinary'}) with pytest.raises(prototype.PrototypeError, match='bridge lacks'): @@ -73,6 +92,49 @@ def test_integer_envelope_and_capability_fail_closed() -> None: ) +def test_float_contract_recovers_integer_json_tokens_without_weakening_integer() -> None: + assert prototype.decode_exact_integers(1, {'kind': 'float'}) == 1.0 + assert type(prototype.decode_exact_integers(1, {'kind': 'float'})) is float + assert prototype.decode_exact_integers(1.5, {'kind': 'float'}) == 1.5 + zero = prototype.decode_exact_integers( + {'__tywrap__': 'float', 'codecVersion': 2, 'encoding': 'negative-zero'}, + {'kind': 'float'}, + ) + assert math.copysign(1, zero) == -1 + with pytest.raises(prototype.PrototypeError, match='untagged integer'): + prototype.decode_exact_integers(1, INTEGER) + with pytest.raises( + prototype.PrototypeError, + match=r'untagged integer at args\.mixed\[0\]\.quantity', + ): + prototype.decode_exact_integers( + {'mixed': [{'quantity': 1, 'ratio': 2, 'flag': True}]}, + { + 'kind': 'record', + 'fields': { + 'mixed': { + 'kind': 'array', + 'item': { + 'kind': 'record', + 'fields': { + 'quantity': INTEGER, + 'ratio': {'kind': 'float'}, + 'flag': {'kind': 'boolean'}, + }, + }, + }, + }, + }, + ) + with pytest.raises(prototype.PrototypeError, match='expected finite float'): + prototype.decode_exact_integers(True, {'kind': 'float'}) + with pytest.raises(prototype.PrototypeError, match='invalid float envelope'): + prototype.decode_exact_integers( + {'__tywrap__': 'float', 'codecVersion': 1, 'encoding': 'negative-zero'}, + {'kind': 'float'}, + ) + + def test_integer_cycles_and_depth_reject_with_path() -> None: cycle: list[object] = [] cycle.append(cycle) diff --git a/test/value_extensions_prototype.test.ts b/test/value_extensions_prototype.test.ts index e35fc6f9..5dc1b41a 100644 --- a/test/value_extensions_prototype.test.ts +++ b/test/value_extensions_prototype.test.ts @@ -31,6 +31,20 @@ const integerContract: PrototypeContract = { negative: { kind: 'record', fields: { value: { kind: 'integer' } } }, safe: { kind: 'integer' }, flag: { kind: 'boolean' }, + whole: { kind: 'float' }, + fraction: { kind: 'float' }, + negativeZero: { kind: 'float' }, + mixed: { + kind: 'array', + item: { + kind: 'record', + fields: { + quantity: { kind: 'integer' }, + ratio: { kind: 'float' }, + flag: { kind: 'boolean' }, + }, + }, + }, }, }; @@ -40,11 +54,20 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( it('round-trips nested positive and negative values beyond int64 in both directions', () => { const positive = 2n ** 80n + 1n; const negative = -(2n ** 130n + 7n); - const value = { positive: [positive], negative: { value: negative }, safe: 7n, flag: true }; + const value = { + positive: [positive], + negative: { value: negative }, + safe: 7n, + flag: true, + whole: 1, + fraction: 1.5, + negativeZero: -0, + mixed: [{ quantity: -9n, ratio: 2, flag: false }], + }; const pythonWire = pythonAction( 'encode-integer', - `{"positive":[${positive}],"negative":{"value":${negative}},"safe":7,"flag":true}` + `{"positive":[${positive}],"negative":{"value":${negative}},"safe":7,"flag":true,"whole":1.0,"fraction":1.5,"negativeZero":-0.0,"mixed":[{"quantity":-9,"ratio":2.0,"flag":false}]}` ); expect(decodeExactResponse(pythonWire, integerContract)).toEqual(value); @@ -54,6 +77,11 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( 'bigint-v2' ); const request = encodeExactRequest(value, integerContract); + expect((request as { negativeZero: unknown }).negativeZero).toEqual({ + __tywrap__: 'float', + codecVersion: 2, + encoding: 'negative-zero', + }); const response = pythonAction( 'roundtrip-integer', JSON.stringify({ @@ -63,6 +91,7 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( }) ); expect(decodeExactResponse(response, integerContract)).toEqual(value); + expect(Object.is((response as { negativeZero: number }).negativeZero, -0)).toBe(true); }); } ); @@ -111,6 +140,16 @@ describe('exact integer rejection prototype', () => { /reserved record key/ ); }); + + it('rejects malformed negative-zero envelopes', () => { + const float: PrototypeContract = { kind: 'float' }; + expect(() => + decodeExactResponse( + { __tywrap__: 'float', codecVersion: 1, encoding: 'negative-zero' }, + float + ) + ).toThrow(/invalid float envelope/); + }); }); describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( From 8ed94d996629d48ecfebfb6fa90476d22ee7d8c3 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:28:02 -0700 Subject: [PATCH 06/35] test(codec): align prototype Unicode and version boundaries --- docs/maintainers/value-contract-prototypes.md | 20 ++++-- test/prototypes/value_extensions.py | 34 ++++++++-- test/prototypes/value_extensions.ts | 20 +++++- .../python/test_value_extensions_prototype.py | 39 ++++++++++- test/value_extensions_prototype.test.ts | 65 ++++++++++++++++++- 5 files changed, 165 insertions(+), 13 deletions(-) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index fa031299..b6b66c27 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -18,10 +18,12 @@ capability, so the wrapper rejects the call before sending a tag. The version 2 integer envelope is `{"__tywrap__":"integer","codecVersion":2,"encoding":"decimal","value":"18446744073709551617"}`. -The decimal must match `^(?:0|[1-9][0-9]*|-[1-9][0-9]*)$`. This excludes `-0`, -leading zeros, plus signs, spaces, exponents, and Unicode digits. The value -permits at most 4096 digits, excluding the sign. It uses at most 4097 ASCII -bytes. Whole request and response byte limits still apply. +`codecVersion` uses the parsed JSON numeric value. The lexemes `2`, `2.0`, and +`2e0` all mean version 2. Booleans and strings do not. The decimal must match +`^(?:0|[1-9][0-9]*|-[1-9][0-9]*)$`. This excludes `-0`, leading zeros, plus +signs, spaces, exponents, and Unicode digits. The value permits at most 4096 +digits, excluding the sign. It uses at most 4097 ASCII bytes. Whole request and +response byte limits still apply. The TypeScript encoder tags every `bigint`, including safe values. The Python request decoder requires the tag at each declared integer node. It rejects raw @@ -47,6 +49,11 @@ node limits. They reject unsupported values and malformed envelopes. Encoders reject cycles with paths. Ordinary records cannot use `__tywrap__` as a key because the decoder reserves it for envelopes. +The prototype counts UTF-8 bytes of compact JSON. Python emits Unicode +characters directly, as `JSON.stringify` does. Both paths reject unpaired +surrogates. Production must check raw wire bytes before parsing; object-level +checks do not prove equal byte counts for alternate numeric JSON spellings. + The bounded tests must round trip positive and negative values beyond 64 bits. They must cover floats, booleans, nested records, malformed tags, digit limits, byte limits, and old bridges. Production work needs separate review of @@ -76,6 +83,11 @@ module-private `WeakMap` stores verified type identity. The return validator reads that entry before it checks field keys and values. A plain record cannot claim dataclass origin. A nested dataclass gets its own entry. +The Python prototype checks the declared root dataclass type. It serializes +nested dataclasses by their observed runtime type and does not check a compiled +field contract. The TypeScript decoder rejects unexpected nested identities or +field values. Server-side field validation remains open. + The bounded prototype still needs a generated `Promise` wrapper from the actual compiler model. It must return `{x: 1, y: 2}` and reject missing, extra, and wrong typed fields. It must test defaults, optional fields, nesting, cycles, diff --git a/test/prototypes/value_extensions.py b/test/prototypes/value_extensions.py index ae17e146..afb67a8f 100644 --- a/test/prototypes/value_extensions.py +++ b/test/prototypes/value_extensions.py @@ -17,6 +17,7 @@ MAX_NODES = 100_000 MAX_PAYLOAD_BYTES = 10 * 1024 * 1024 SAFE_INTEGER_MAX = 2**53 - 1 +MAX_INTEGER_MAGNITUDE = 10**MAX_DECIMAL_DIGITS DECIMAL = re.compile(r'(?:0|[1-9][0-9]*|-[1-9][0-9]*)\Z', re.ASCII) INTEGER_FIELDS = frozenset({'__tywrap__', 'codecVersion', 'encoding', 'value'}) @@ -38,11 +39,21 @@ def visit(self, path: str, depth: int) -> None: def _check_payload(value: object, max_payload_bytes: int) -> None: - encoded = json.dumps(value, allow_nan=False, separators=(',', ':')) - if len(encoded.encode('utf-8')) > max_payload_bytes: + try: + encoded = json.dumps( + value, allow_nan=False, ensure_ascii=False, separators=(',', ':') + ) + byte_count = len(encoded.encode('utf-8')) + except (UnicodeEncodeError, ValueError) as exc: + raise PrototypeError('payload is not finite UTF-8 JSON') from exc + if byte_count > max_payload_bytes: raise PrototypeError(f'payload exceeds {max_payload_bytes} bytes') +def _version_two(value: object) -> bool: + return type(value) in (int, float) and value == 2 + + def _child(path: str, key: str | int) -> str: return f'{path}[{key}]' if isinstance(key, int) else f'{path}.{key}' @@ -69,7 +80,12 @@ def _canonical_decimal(value: object, path: str) -> int: def _integer_envelope(value: int, path: str) -> dict[str, object]: - decimal = str(value) + if abs(value) >= MAX_INTEGER_MAGNITUDE: + raise PrototypeError(f'integer exceeds {MAX_DECIMAL_DIGITS} digits at {path}') + try: + decimal = str(value) + except ValueError as exc: + raise PrototypeError(f'integer conversion failed at {path}') from exc _canonical_decimal(decimal, path) return { '__tywrap__': 'integer', @@ -168,8 +184,7 @@ def visit( if ( set(current) != INTEGER_FIELDS or current.get('__tywrap__') != 'integer' - or type(current.get('codecVersion')) is not int - or current.get('codecVersion') != 2 + or not _version_two(current.get('codecVersion')) or current.get('encoding') != 'decimal' ): raise PrototypeError(f'invalid integer envelope at {path}') @@ -188,7 +203,7 @@ def visit( '__tywrap__': 'float', 'codecVersion': 2, 'encoding': 'negative-zero', - } or type(current.get('codecVersion')) is not int: + } or not _version_two(current.get('codecVersion')): raise PrototypeError(f'invalid float envelope at {path}') return -0.0 if type(current) not in (int, float): @@ -351,6 +366,13 @@ def _main() -> None: ) elif action == 'encode-integer': result = encode_exact_integers(payload) + elif action == 'roundtrip-single-integer': + require_capability( + payload['meta'], 'exactIntegerDecimalV2', payload['policy'] + ) + result = encode_exact_integers( + decode_exact_integers(payload['value'], {'kind': 'integer'}) + ) elif action == 'encode-point': result = encode_dataclass(Point(**payload), Point) else: diff --git a/test/prototypes/value_extensions.ts b/test/prototypes/value_extensions.ts index 68112808..67d8d4f4 100644 --- a/test/prototypes/value_extensions.ts +++ b/test/prototypes/value_extensions.ts @@ -46,7 +46,25 @@ function exactKeys( } function checkPayload(value: unknown, limit: number): void { - const encoded = JSON.stringify(value); + const assertUnicodeScalars = (text: string): void => { + for (let index = 0; index < text.length; index += 1) { + const code = text.charCodeAt(index); + if (code >= 0xd800 && code <= 0xdbff) { + const next = text.charCodeAt(index + 1); + if (!(next >= 0xdc00 && next <= 0xdfff)) { + throw new PrototypeError('payload contains an unpaired Unicode surrogate'); + } + index += 1; + } else if (code >= 0xdc00 && code <= 0xdfff) { + throw new PrototypeError('payload contains an unpaired Unicode surrogate'); + } + } + }; + const encoded = JSON.stringify(value, (key, item: unknown) => { + assertUnicodeScalars(key); + if (typeof item === 'string') assertUnicodeScalars(item); + return item; + }); if (encoded === undefined || new TextEncoder().encode(encoded).length > limit) { throw new PrototypeError(`payload exceeds ${limit} bytes`); } diff --git a/test/python/test_value_extensions_prototype.py b/test/python/test_value_extensions_prototype.py index 09f63e99..a0ebf7ae 100644 --- a/test/python/test_value_extensions_prototype.py +++ b/test/python/test_value_extensions_prototype.py @@ -3,6 +3,7 @@ from __future__ import annotations import math +import json import sys from dataclasses import dataclass, field from pathlib import Path @@ -64,15 +65,51 @@ def test_integer_digit_and_payload_caps() -> None: ) == accepted with pytest.raises(prototype.PrototypeError, match='4096 digits'): prototype.encode_exact_integers(int('9' * (prototype.MAX_DECIMAL_DIGITS + 1))) + with pytest.raises( + prototype.PrototypeError, match=r'4096 digits at result.big\[0\]' + ): + prototype.encode_exact_integers({'big': [10**5000]}) with pytest.raises(prototype.PrototypeError, match='payload exceeds 80 bytes'): prototype.encode_exact_integers({'text': 'x' * 80}, max_payload_bytes=80) +def test_unicode_payload_cap_counts_compact_utf8_bytes() -> None: + assert prototype.encode_exact_integers('é', max_payload_bytes=4) == 'é' + with pytest.raises(prototype.PrototypeError, match='payload exceeds 3 bytes'): + prototype.encode_exact_integers('é', max_payload_bytes=3) + + nested = {'café': ['🍵', 'é']} + expected_bytes = len('{"café":["🍵","é"]}'.encode('utf-8')) + assert prototype.encode_exact_integers( + nested, max_payload_bytes=expected_bytes + ) == nested + with pytest.raises(prototype.PrototypeError, match='payload exceeds'): + prototype.encode_exact_integers( + nested, max_payload_bytes=expected_bytes - 1 + ) + + +@pytest.mark.parametrize( + ('version_token', 'accepted'), + [('2', True), ('2.0', True), ('2e0', True), ('true', False), ('"2"', False), ('2.5', False), ('null', False)], +) +def test_integer_version_uses_json_numeric_value(version_token: str, accepted: bool) -> None: + envelope = json.loads( + '{"__tywrap__":"integer","codecVersion":' + + version_token + + ',"encoding":"decimal","value":"7"}' + ) + if accepted: + assert prototype.decode_exact_integers(envelope, INTEGER) == 7 + else: + with pytest.raises(prototype.PrototypeError): + prototype.decode_exact_integers(envelope, INTEGER) + + def test_integer_envelope_and_capability_fail_closed() -> None: encoded = prototype.encode_exact_integers(2**80) for change in ( {'codecVersion': 1}, - {'codecVersion': 2.0}, {'encoding': 'json'}, {'unexpected': True}, ): diff --git a/test/value_extensions_prototype.test.ts b/test/value_extensions_prototype.test.ts index 5dc1b41a..2b5df28b 100644 --- a/test/value_extensions_prototype.test.ts +++ b/test/value_extensions_prototype.test.ts @@ -1,4 +1,4 @@ -import { execFileSync } from 'node:child_process'; +import { execFileSync, spawnSync } from 'node:child_process'; import { existsSync } from 'node:fs'; import { join } from 'node:path'; import { describe, expect, it } from 'vitest'; @@ -131,6 +131,22 @@ describe('exact integer rejection prototype', () => { expect(() => encodeExactRequest(1n, integer, 20)).toThrow(/payload exceeds 20 bytes/); }); + it('counts compact UTF-8 JSON bytes for Unicode strings and nested records', () => { + const text: PrototypeContract = { kind: 'string' }; + expect(encodeExactRequest('é', text, 4)).toBe('é'); + expect(() => encodeExactRequest('é', text, 3)).toThrow(/payload exceeds 3 bytes/); + + const nested = { café: ['🍵', 'é'] }; + const nestedContract: PrototypeContract = { + kind: 'record', + fields: { café: { kind: 'array', item: text } }, + }; + const limit = new TextEncoder().encode('{"café":["🍵","é"]}').length; + expect(encodeExactRequest(nested, nestedContract, limit)).toEqual(nested); + expect(() => encodeExactRequest(nested, nestedContract, limit - 1)).toThrow(/payload exceeds/); + expect(() => encodeExactRequest('\ud800', text)).toThrow(/unpaired Unicode surrogate/); + }); + it('rejects a record key reserved for envelopes', () => { const record: PrototypeContract = { kind: 'record', @@ -152,6 +168,53 @@ describe('exact integer rejection prototype', () => { }); }); +describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( + 'cross-language version and Unicode policy', + () => { + it.each([ + ['2', true], + ['2.0', true], + ['2e0', true], + ['true', false], + ['"2"', false], + ['2.5', false], + ['null', false], + ] as const)('agrees on integer envelope version %s', (version, accepted) => { + const raw = `{"meta":{"valueCapabilities":["exactIntegerDecimalV2"]},"policy":"bigint-v2","value":{"__tywrap__":"integer","codecVersion":${version},"encoding":"decimal","value":"7"}}`; + const request = JSON.parse(raw) as { value: unknown }; + if (accepted) { + expect(decodeExactResponse(request.value, { kind: 'integer' })).toBe(7n); + } else { + expect(() => decodeExactResponse(request.value, { kind: 'integer' })).toThrow(); + } + const python = spawnSync(pythonPath, [pythonScript, 'roundtrip-single-integer'], { + input: raw, + encoding: 'utf8', + }); + expect(python.status === 0).toBe(accepted); + if (accepted) { + expect(JSON.parse(python.stdout)).toEqual({ + __tywrap__: 'integer', + codecVersion: 2, + encoding: 'decimal', + value: '7', + }); + } + }); + + it('agrees on nested Unicode and rejects unpaired surrogates', () => { + const nested = { café: ['🍵', 'é'] }; + expect(pythonAction('encode-integer', JSON.stringify(nested))).toEqual(nested); + const bad = spawnSync(pythonPath, [pythonScript, 'encode-integer'], { + input: JSON.stringify('\ud800'), + encoding: 'utf8', + }); + expect(bad.status).not.toBe(0); + expect(bad.stderr).toContain('PrototypeError'); + }); + } +); + describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( 'dataclass output prototype', () => { From 6e694a7a23550415762815f195bf511bb9495f64 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:28:47 -0700 Subject: [PATCH 07/35] test(codec): emit the counted Unicode wire form --- test/prototypes/value_extensions.py | 2 +- test/value_extensions_prototype.test.ts | 6 ++++++ 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/test/prototypes/value_extensions.py b/test/prototypes/value_extensions.py index afb67a8f..85bf3b2b 100644 --- a/test/prototypes/value_extensions.py +++ b/test/prototypes/value_extensions.py @@ -377,7 +377,7 @@ def _main() -> None: result = encode_dataclass(Point(**payload), Point) else: raise PrototypeError(f'unknown action {action!r}') - print(json.dumps(result, allow_nan=False, separators=(',', ':'))) + print(json.dumps(result, allow_nan=False, ensure_ascii=False, separators=(',', ':'))) if __name__ == '__main__': diff --git a/test/value_extensions_prototype.test.ts b/test/value_extensions_prototype.test.ts index 2b5df28b..80b4b5ba 100644 --- a/test/value_extensions_prototype.test.ts +++ b/test/value_extensions_prototype.test.ts @@ -205,6 +205,12 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( it('agrees on nested Unicode and rejects unpaired surrogates', () => { const nested = { café: ['🍵', 'é'] }; expect(pythonAction('encode-integer', JSON.stringify(nested))).toEqual(nested); + const encoded = spawnSync(pythonPath, [pythonScript, 'encode-integer'], { + input: JSON.stringify(nested), + encoding: 'utf8', + }); + expect(encoded.status).toBe(0); + expect(encoded.stdout.trim()).toBe(JSON.stringify(nested)); const bad = spawnSync(pythonPath, [pythonScript, 'encode-integer'], { input: JSON.stringify('\ud800'), encoding: 'utf8', From 0ad97ce8078b215c1e61bf4b51c4102af0f46434 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:32:42 -0700 Subject: [PATCH 08/35] test(codec): match prototype output bytes to checked limit --- test/prototypes/value_extensions.py | 9 ++++++++- test/value_extensions_prototype.test.ts | 17 +++++++++++++++++ 2 files changed, 25 insertions(+), 1 deletion(-) diff --git a/test/prototypes/value_extensions.py b/test/prototypes/value_extensions.py index 85bf3b2b..135c519b 100644 --- a/test/prototypes/value_extensions.py +++ b/test/prototypes/value_extensions.py @@ -366,6 +366,10 @@ def _main() -> None: ) elif action == 'encode-integer': result = encode_exact_integers(payload) + elif action == 'encode-integer-limited': + result = encode_exact_integers( + payload['value'], max_payload_bytes=payload['limit'] + ) elif action == 'roundtrip-single-integer': require_capability( payload['meta'], 'exactIntegerDecimalV2', payload['policy'] @@ -377,7 +381,10 @@ def _main() -> None: result = encode_dataclass(Point(**payload), Point) else: raise PrototypeError(f'unknown action {action!r}') - print(json.dumps(result, allow_nan=False, ensure_ascii=False, separators=(',', ':'))) + wire = json.dumps( + result, allow_nan=False, ensure_ascii=False, separators=(',', ':') + ).encode('utf-8') + sys.stdout.buffer.write(wire + b'\n') if __name__ == '__main__': diff --git a/test/value_extensions_prototype.test.ts b/test/value_extensions_prototype.test.ts index 80b4b5ba..53bd73b1 100644 --- a/test/value_extensions_prototype.test.ts +++ b/test/value_extensions_prototype.test.ts @@ -211,6 +211,23 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( }); expect(encoded.status).toBe(0); expect(encoded.stdout.trim()).toBe(JSON.stringify(nested)); + const wire = Buffer.from(JSON.stringify(nested), 'utf8'); + const exact = spawnSync(pythonPath, [pythonScript, 'encode-integer-limited'], { + input: JSON.stringify({ value: nested, limit: wire.length }), + }); + expect(exact.status).toBe(0); + expect(exact.stdout).toEqual(Buffer.concat([wire, Buffer.from('\n')])); + expect(exact.stdout.subarray(0, -1).length).toBe(wire.length); + const tooSmall = spawnSync(pythonPath, [pythonScript, 'encode-integer-limited'], { + input: JSON.stringify({ value: nested, limit: wire.length - 1 }), + }); + expect(tooSmall.status).not.toBe(0); + + const short = spawnSync(pythonPath, [pythonScript, 'encode-integer-limited'], { + input: JSON.stringify({ value: 'é', limit: 4 }), + }); + expect(short.stdout).toEqual(Buffer.from('"é"\n', 'utf8')); + expect(short.stdout.subarray(0, -1).length).toBe(4); const bad = spawnSync(pythonPath, [pythonScript, 'encode-integer'], { input: JSON.stringify('\ud800'), encoding: 'utf8', From 9a1a4c0403fbcdf32f9d2f74210f14dcb644e476 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:37:11 -0700 Subject: [PATCH 09/35] test(codec): reject ordinary class as dataclass output --- test/python/test_value_extensions_prototype.py | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/test/python/test_value_extensions_prototype.py b/test/python/test_value_extensions_prototype.py index a0ebf7ae..df3de564 100644 --- a/test/python/test_value_extensions_prototype.py +++ b/test/python/test_value_extensions_prototype.py @@ -215,6 +215,14 @@ class Container: with pytest.raises(prototype.PrototypeError, match='declared dataclass type'): prototype.encode_dataclass(prototype.Point(1, 2), Container) + class OrdinaryPoint: + def __init__(self, x: int, y: int) -> None: + self.x = x + self.y = y + + with pytest.raises(prototype.PrototypeError, match='declared dataclass type'): + prototype.encode_dataclass(OrdinaryPoint(1, 2), prototype.Point) + @dataclass class Unsupported: value: object From 9a0064061ebed72a2d7472e51ab94eb687676b7a Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:38:20 -0700 Subject: [PATCH 10/35] test(codec): show serializable unsupported dataclass field --- test/python/test_value_extensions_prototype.py | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/test/python/test_value_extensions_prototype.py b/test/python/test_value_extensions_prototype.py index df3de564..f212e6a1 100644 --- a/test/python/test_value_extensions_prototype.py +++ b/test/python/test_value_extensions_prototype.py @@ -223,6 +223,15 @@ def __init__(self, x: int, y: int) -> None: with pytest.raises(prototype.PrototypeError, match='declared dataclass type'): prototype.encode_dataclass(OrdinaryPoint(1, 2), prototype.Point) + @dataclass + class UnsupportedAnnotation: + x: object + y: int + + assert prototype.encode_dataclass( + UnsupportedAnnotation(1, 2), UnsupportedAnnotation + )['fields'] == {'x': 1, 'y': 2} + @dataclass class Unsupported: value: object From 80bfa77e88b93429fc34a43bb11f28d8b83761e0 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:40:19 -0700 Subject: [PATCH 11/35] test(codec): prove generated Point through dataclass adapter --- test/value_extensions_generated_point.test.ts | 281 ++++++++++++++++++ 1 file changed, 281 insertions(+) create mode 100644 test/value_extensions_generated_point.test.ts diff --git a/test/value_extensions_generated_point.test.ts b/test/value_extensions_generated_point.test.ts new file mode 100644 index 00000000..eb4d0f9e --- /dev/null +++ b/test/value_extensions_generated_point.test.ts @@ -0,0 +1,281 @@ +import { execFileSync } from 'node:child_process'; +import { existsSync } from 'node:fs'; +import { mkdtemp, rm, writeFile } from 'node:fs/promises'; +import { join } from 'node:path'; +import { pathToFileURL } from 'node:url'; +import ts from 'typescript'; +import { describe, expect, it } from 'vitest'; +import { + compileContract, + DEFAULT_CALLABLE_CAPABILITIES, + DEFAULT_VALUE_CONVERSION, + type ValueConversionDescription, +} from '../src/core/callable-compiler.js'; +import { CodeGenerator } from '../src/core/generator.js'; +import { validateIrContract } from '../src/core/ir-contract.js'; +import { transformIrToTsModel } from '../src/core/ir-model.js'; +import { + VALUE_CONTRACT_REVISION, + type ValueContractField, +} from '../src/contracts/value-contract.js'; +import { BridgeValidationError } from '../src/runtime/errors.js'; +import { clearRuntimeBridge, setRuntimeBridge } from 'tywrap/runtime'; +import type { PythonModule } from '../src/types/index.js'; +import { PYTHON, PYTHON_AVAILABLE } from './helpers/python-probe.js'; +import { + decodeExactResponse, + requireCapability, + validateDataclassOrigin, + type PrototypeContract, +} from './prototypes/value_extensions.js'; + +const pythonScript = join(process.cwd(), 'test', 'prototypes', 'value_extensions.py'); +const pointContract: PrototypeContract = { + kind: 'dataclass', + typeId: '__main__.Point', + fields: { x: { kind: 'safe-integer' }, y: { kind: 'safe-integer' } }, +}; + +const pointIr = { + ir_version: '0.4.0', + module: 'value_extensions', + functions: [ + { + name: 'make_point', + qualname: 'value_extensions.make_point', + docstring: null, + parameters: [], + returns: 'Point', + is_async: false, + is_generator: false, + type_params: [], + method_kind: 'instance', + overloads: [], + }, + ], + classes: [ + { + name: 'Point', + qualname: 'value_extensions.Point', + docstring: null, + bases: ['object'], + methods: [], + typed_dict: false, + total: null, + fields: [ + { name: 'x', kind: 'FIELD', annotation: 'int', default: false }, + { name: 'y', kind: 'FIELD', annotation: 'int', default: false }, + ], + is_protocol: false, + is_namedtuple: false, + is_dataclass: true, + is_pydantic: false, + type_params: [], + accessors: [], + }, + ], + constants: [], + type_aliases: [], + metadata: {}, + warnings: [], +}; + +function pointConversion(module: PythonModule): ValueConversionDescription { + return { + revision: VALUE_CONTRACT_REVISION, + resolve(request) { + if ( + request.direction !== 'output' || + request.logicalType.kind !== 'custom' || + request.logicalType.name !== 'Point' + ) { + return DEFAULT_VALUE_CONVERSION.resolve(request); + } + const fields: ValueContractField[] = []; + for (const property of module.classes[0]!.properties) { + const resolved = DEFAULT_VALUE_CONVERSION.resolve({ + ...request, + logicalType: property.type, + path: `${request.path}.${property.name}`, + }); + if (resolved.status !== 'supported') { + return { + status: 'unsupported', + reason: `No value contract for Point.${property.name}`, + guidance: 'Use a supported dataclass field annotation.', + }; + } + fields.push({ name: property.name, value: resolved.value, required: true }); + } + return { + status: 'supported', + value: { kind: 'record', wire: 'json', decodedAs: 'object', fields }, + }; + }, + }; +} + +function compilePoint(sourceIr: typeof pointIr, useAdapter: boolean, enableAdapter: boolean) { + const validated = validateIrContract(sourceIr, 'Point prototype IR'); + expect(validated.ok).toBe(true); + if (!validated.ok) throw new Error('Point prototype IR is invalid'); + const module = transformIrToTsModel(validated.contract); + return compileContract(validated.contract, { + module, + generator: new CodeGenerator(), + conversion: useAdapter ? pointConversion(module) : DEFAULT_VALUE_CONVERSION, + capabilities: DEFAULT_CALLABLE_CAPABILITIES.map(capability => + capability.name === 'dataclass-adapter' + ? { ...capability, available: enableAdapter } + : capability + ), + }); +} + +function pythonPoint(): unknown { + return JSON.parse( + execFileSync(PYTHON ?? 'python3', [pythonScript, 'encode-point'], { + input: JSON.stringify({ x: 1, y: 2 }), + encoding: 'utf8', + }) + ) as unknown; +} + +describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( + 'generated Point output prototype', + () => { + it('keeps defaults, missing capability, and unsupported fields unresolved', () => { + const defaultResult = compilePoint(pointIr, false, false); + expect(defaultResult.callables[0]?.result.resolution.status).toBe('unsupported'); + expect(defaultResult.generated.declaration).toContain('Promise'); + + const missingCapability = compilePoint(pointIr, true, false); + expect(missingCapability.callables[0]?.result.resolution.status).toBe('unsupported'); + + const unsupportedIr = { + ...pointIr, + classes: [ + { + ...pointIr.classes[0]!, + fields: [ + { ...pointIr.classes[0]!.fields[0]!, annotation: 'object' }, + pointIr.classes[0]!.fields[1]!, + ], + }, + ], + }; + const unsupported = compilePoint(unsupportedIr, true, true); + expect(unsupported.callables[0]?.result.resolution.status).toBe('unsupported'); + expect(unsupported.generated.declaration).toContain('Promise'); + }); + + it('executes a generated Promise through the Python prototype adapter', async () => { + const compiled = compilePoint(pointIr, true, true); + expect(compiled.diagnostics.filter(item => item.severity === 'error')).toEqual([]); + expect(compiled.callables[0]?.requiredCapabilities).toContain('dataclass-adapter'); + expect(compiled.generated.declaration).toContain('make_point(): Promise'); + expect(compiled.generated.declaration).toContain( + 'export type Point = { x: number; y: number; }' + ); + expect(compiled.generated.typescript).toContain('"constraint":"safe-integer"'); + + const temporary = await mkdtemp(join(process.cwd(), 'test', '.tywrap-point-')); + try { + const outputPath = join(temporary, 'value_extensions.generated.mjs'); + const declarationPath = join(temporary, 'value_extensions.generated.d.ts'); + const consumerPath = join(temporary, 'consumer.ts'); + const javascript = ts.transpileModule(compiled.generated.typescript, { + compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext }, + }).outputText; + await writeFile(outputPath, javascript, 'utf8'); + await writeFile(declarationPath, compiled.generated.declaration, 'utf8'); + await writeFile( + consumerPath, + `import { make_point, type Point } from './value_extensions.generated.js'; +const result: Promise = make_point(); +result.then(point => { + const x: number = point.x; + const y: number = point.y; + // @ts-expect-error Point.x is a number. + const wrong: string = point.x; + // @ts-expect-error Point has no z field. + point.z; + return x + y + wrong.length; +});`, + 'utf8' + ); + const program = ts.createProgram([consumerPath], { + module: ts.ModuleKind.NodeNext, + moduleResolution: ts.ModuleResolutionKind.NodeNext, + target: ts.ScriptTarget.ES2022, + strict: true, + noEmit: true, + skipLibCheck: true, + types: [], + }); + expect( + ts + .getPreEmitDiagnostics(program) + .map(diagnostic => ts.flattenDiagnosticMessageText(diagnostic.messageText, '\n')) + ).toEqual([]); + + let wire: unknown = pythonPoint(); + let bridgeMeta: unknown = { valueCapabilities: ['dataclassFieldsV2'] }; + setRuntimeBridge({ + async call( + moduleName: string, + functionName: string, + _args: unknown[], + _kwargs?: Record, + validate?: (result: T) => void + ): Promise { + expect([moduleName, functionName]).toEqual(['value_extensions', 'make_point']); + requireCapability(bridgeMeta, 'dataclassFieldsV2', 'fields-v2'); + const decoded = decodeExactResponse(wire, pointContract); + validateDataclassOrigin(decoded, pointContract); + validate?.(decoded as T); + return decoded as T; + }, + async dispose(): Promise {}, + }); + const generated = (await import(pathToFileURL(outputPath).href)) as { + make_point: () => Promise<{ x: number; y: number }>; + }; + + await expect(generated.make_point()).resolves.toEqual({ x: 1, y: 2 }); + bridgeMeta = {}; + await expect(generated.make_point()).rejects.toThrow(/bridge lacks dataclassFieldsV2/); + bridgeMeta = { valueCapabilities: ['dataclassFieldsV2'] }; + const valid = pythonPoint() as { fields: Record }; + for (const bad of [ + { ...valid, fields: { x: 1 } }, + { ...valid, fields: { x: 1, y: 2, z: 3 } }, + { ...valid, fields: { x: '1', y: 2 } }, + { x: 1, y: 2 }, + ]) { + wire = bad; + await expect(generated.make_point()).rejects.toThrow(); + } + + setRuntimeBridge({ + async call( + _module: string, + _functionName: string, + _args: unknown[], + _kwargs?: Record, + validate?: (result: T) => void + ): Promise { + const wrong = { x: '1', y: 2 } as unknown as T; + validate?.(wrong); + return wrong; + }, + async dispose(): Promise {}, + }); + await expect(generated.make_point()).rejects.toThrow(BridgeValidationError); + } finally { + clearRuntimeBridge(); + await rm(temporary, { recursive: true, force: true }); + } + }); + } +); From 0226fd95b93665b701a7dfa222693933c7cf0458 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:44:35 -0700 Subject: [PATCH 12/35] test: use generated camelCase Point callable --- test/value_extensions_generated_point.test.ts | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/test/value_extensions_generated_point.test.ts b/test/value_extensions_generated_point.test.ts index eb4d0f9e..838bd4f7 100644 --- a/test/value_extensions_generated_point.test.ts +++ b/test/value_extensions_generated_point.test.ts @@ -173,7 +173,7 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( const compiled = compilePoint(pointIr, true, true); expect(compiled.diagnostics.filter(item => item.severity === 'error')).toEqual([]); expect(compiled.callables[0]?.requiredCapabilities).toContain('dataclass-adapter'); - expect(compiled.generated.declaration).toContain('make_point(): Promise'); + expect(compiled.generated.declaration).toContain('makePoint(): Promise'); expect(compiled.generated.declaration).toContain( 'export type Point = { x: number; y: number; }' ); @@ -191,8 +191,8 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( await writeFile(declarationPath, compiled.generated.declaration, 'utf8'); await writeFile( consumerPath, - `import { make_point, type Point } from './value_extensions.generated.js'; -const result: Promise = make_point(); + `import { makePoint, type Point } from './value_extensions.generated.js'; +const result: Promise = makePoint(); result.then(point => { const x: number = point.x; const y: number = point.y; @@ -239,12 +239,12 @@ result.then(point => { async dispose(): Promise {}, }); const generated = (await import(pathToFileURL(outputPath).href)) as { - make_point: () => Promise<{ x: number; y: number }>; + makePoint: () => Promise<{ x: number; y: number }>; }; - await expect(generated.make_point()).resolves.toEqual({ x: 1, y: 2 }); + await expect(generated.makePoint()).resolves.toEqual({ x: 1, y: 2 }); bridgeMeta = {}; - await expect(generated.make_point()).rejects.toThrow(/bridge lacks dataclassFieldsV2/); + await expect(generated.makePoint()).rejects.toThrow(/bridge lacks dataclassFieldsV2/); bridgeMeta = { valueCapabilities: ['dataclassFieldsV2'] }; const valid = pythonPoint() as { fields: Record }; for (const bad of [ @@ -254,7 +254,7 @@ result.then(point => { { x: 1, y: 2 }, ]) { wire = bad; - await expect(generated.make_point()).rejects.toThrow(); + await expect(generated.makePoint()).rejects.toThrow(); } setRuntimeBridge({ @@ -271,7 +271,7 @@ result.then(point => { }, async dispose(): Promise {}, }); - await expect(generated.make_point()).rejects.toThrow(BridgeValidationError); + await expect(generated.makePoint()).rejects.toThrow(BridgeValidationError); } finally { clearRuntimeBridge(); await rm(temporary, { recursive: true, force: true }); From f76641ffe8027e4a9602ca8a6a5f2cd0b8fe784b Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:50:23 -0700 Subject: [PATCH 13/35] test: derive Point proof from analyzed importable fixture --- test/fixtures/python/value_contract_point.py | 14 ++ test/value_extensions_generated_point.test.ts | 163 ++++++++++-------- 2 files changed, 102 insertions(+), 75 deletions(-) create mode 100644 test/fixtures/python/value_contract_point.py diff --git a/test/fixtures/python/value_contract_point.py b/test/fixtures/python/value_contract_point.py new file mode 100644 index 00000000..6e3b4559 --- /dev/null +++ b/test/fixtures/python/value_contract_point.py @@ -0,0 +1,14 @@ +"""Importable Point fixture for the dataclass value proof.""" + +from dataclasses import dataclass + + +@dataclass +class Point: + x: int + y: int = 2 + + +def make_point() -> Point: + """Return a Point with its defaulted field present.""" + return Point(1) diff --git a/test/value_extensions_generated_point.test.ts b/test/value_extensions_generated_point.test.ts index 838bd4f7..2ee3f3a7 100644 --- a/test/value_extensions_generated_point.test.ts +++ b/test/value_extensions_generated_point.test.ts @@ -1,7 +1,7 @@ import { execFileSync } from 'node:child_process'; import { existsSync } from 'node:fs'; import { mkdtemp, rm, writeFile } from 'node:fs/promises'; -import { join } from 'node:path'; +import { delimiter, join } from 'node:path'; import { pathToFileURL } from 'node:url'; import ts from 'typescript'; import { describe, expect, it } from 'vitest'; @@ -30,69 +30,48 @@ import { } from './prototypes/value_extensions.js'; const pythonScript = join(process.cwd(), 'test', 'prototypes', 'value_extensions.py'); -const pointContract: PrototypeContract = { - kind: 'dataclass', - typeId: '__main__.Point', - fields: { x: { kind: 'safe-integer' }, y: { kind: 'safe-integer' } }, +const fixtureDir = join(process.cwd(), 'test', 'fixtures', 'python'); +const fixturePath = join(fixtureDir, 'value_contract_point.py'); +const pythonEnvironment = { + ...process.env, + PYTHONPATH: [ + fixtureDir, + join(process.cwd(), 'tywrap_ir'), + join(process.cwd(), 'test', 'prototypes'), + process.env.PYTHONPATH, + ] + .filter(Boolean) + .join(delimiter), }; -const pointIr = { - ir_version: '0.4.0', - module: 'value_extensions', - functions: [ - { - name: 'make_point', - qualname: 'value_extensions.make_point', - docstring: null, - parameters: [], - returns: 'Point', - is_async: false, - is_generator: false, - type_params: [], - method_kind: 'instance', - overloads: [], - }, - ], - classes: [ - { - name: 'Point', - qualname: 'value_extensions.Point', - docstring: null, - bases: ['object'], - methods: [], - typed_dict: false, - total: null, - fields: [ - { name: 'x', kind: 'FIELD', annotation: 'int', default: false }, - { name: 'y', kind: 'FIELD', annotation: 'int', default: false }, +function analyzedPointIr(): unknown { + return JSON.parse( + execFileSync( + PYTHON ?? 'python3', + [ + '-c', + "from tywrap_ir.ir import emit_ir_json; print(emit_ir_json('value_contract_point', pretty=False))", ], - is_protocol: false, - is_namedtuple: false, - is_dataclass: true, - is_pydantic: false, - type_params: [], - accessors: [], - }, - ], - constants: [], - type_aliases: [], - metadata: {}, - warnings: [], -}; + { env: pythonEnvironment, encoding: 'utf8' } + ) + ) as unknown; +} function pointConversion(module: PythonModule): ValueConversionDescription { + const point = module.classes.find(cls => cls.name === 'Point' && cls.kind === 'dataclass'); + if (!point) throw new Error('Analyzed Point dataclass is missing'); return { revision: VALUE_CONTRACT_REVISION, resolve(request) { if ( request.direction !== 'output' || request.logicalType.kind !== 'custom' || - request.logicalType.name !== 'Point' + request.logicalType.name.split('.').at(-1) !== point.name ) { return DEFAULT_VALUE_CONVERSION.resolve(request); } const fields: ValueContractField[] = []; - for (const property of module.classes[0]!.properties) { + for (const property of point.properties) { const resolved = DEFAULT_VALUE_CONVERSION.resolve({ ...request, logicalType: property.type, @@ -115,12 +94,17 @@ function pointConversion(module: PythonModule): ValueConversionDescription { }; } -function compilePoint(sourceIr: typeof pointIr, useAdapter: boolean, enableAdapter: boolean) { - const validated = validateIrContract(sourceIr, 'Point prototype IR'); +function compilePoint(sourceIr: unknown, useAdapter: boolean, enableAdapter: boolean) { + const validated = validateIrContract(sourceIr, 'analyzed Point IR'); expect(validated.ok).toBe(true); - if (!validated.ok) throw new Error('Point prototype IR is invalid'); - const module = transformIrToTsModel(validated.contract); - return compileContract(validated.contract, { + if (!validated.ok) throw new Error('Analyzed Point IR is invalid'); + const canonical = transformIrToTsModel(validated.contract); + const module = { + ...canonical, + functions: canonical.functions.filter(func => func.name === 'make_point'), + classes: canonical.classes.filter(cls => cls.name === 'Point'), + }; + const compiled = compileContract(validated.contract, { module, generator: new CodeGenerator(), conversion: useAdapter ? pointConversion(module) : DEFAULT_VALUE_CONVERSION, @@ -130,53 +114,82 @@ function compilePoint(sourceIr: typeof pointIr, useAdapter: boolean, enableAdapt : capability ), }); + return { compiled, ir: validated.contract }; } function pythonPoint(): unknown { return JSON.parse( - execFileSync(PYTHON ?? 'python3', [pythonScript, 'encode-point'], { - input: JSON.stringify({ x: 1, y: 2 }), - encoding: 'utf8', - }) + execFileSync( + PYTHON ?? 'python3', + [ + '-c', + "import json; from value_contract_point import Point, make_point; from value_extensions import encode_dataclass; print(json.dumps(encode_dataclass(make_point(), Point), separators=(',', ':')))", + ], + { + env: pythonEnvironment, + encoding: 'utf8', + } + ) ) as unknown; } -describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( +function pointContractFrom(result: ReturnType): PrototypeContract { + const point = result.ir.classes.find(cls => cls.name === 'Point'); + const resolution = result.compiled.callables.find(item => item.name === 'make_point')?.result + .resolution; + if ( + !point || + !resolution || + resolution.status !== 'supported' || + resolution.value.kind !== 'record' + ) { + throw new Error('Analyzed Point output did not resolve to a record'); + } + const fields: Record = {}; + for (const field of resolution.value.fields) { + if (field.value.kind !== 'integer' || field.value.constraint !== 'safe-integer') { + throw new Error(`Unsupported Point field ${field.name}`); + } + fields[field.name] = { kind: 'safe-integer' }; + } + return { kind: 'dataclass', typeId: point.qualname, fields }; +} + +describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript) || !existsSync(fixturePath))( 'generated Point output prototype', () => { it('keeps defaults, missing capability, and unsupported fields unresolved', () => { - const defaultResult = compilePoint(pointIr, false, false); + const pointIr = analyzedPointIr(); + const defaultResult = compilePoint(pointIr, false, false).compiled; expect(defaultResult.callables[0]?.result.resolution.status).toBe('unsupported'); expect(defaultResult.generated.declaration).toContain('Promise'); - const missingCapability = compilePoint(pointIr, true, false); + const missingCapability = compilePoint(pointIr, true, false).compiled; expect(missingCapability.callables[0]?.result.resolution.status).toBe('unsupported'); - const unsupportedIr = { - ...pointIr, - classes: [ - { - ...pointIr.classes[0]!, - fields: [ - { ...pointIr.classes[0]!.fields[0]!, annotation: 'object' }, - pointIr.classes[0]!.fields[1]!, - ], - }, - ], + const unsupportedIr = structuredClone(pointIr) as { + classes: Array<{ name: string; fields: Array<{ annotation: string }> }>; }; - const unsupported = compilePoint(unsupportedIr, true, true); + const point = unsupportedIr.classes.find(cls => cls.name === 'Point'); + if (!point) throw new Error('Analyzed Point dataclass is missing'); + point.fields[0]!.annotation = 'object'; + const unsupported = compilePoint(unsupportedIr, true, true).compiled; expect(unsupported.callables[0]?.result.resolution.status).toBe('unsupported'); expect(unsupported.generated.declaration).toContain('Promise'); }); it('executes a generated Promise through the Python prototype adapter', async () => { - const compiled = compilePoint(pointIr, true, true); + const pointIr = analyzedPointIr(); + const result = compilePoint(pointIr, true, true); + const { compiled } = result; + const pointContract = pointContractFrom(result); expect(compiled.diagnostics.filter(item => item.severity === 'error')).toEqual([]); expect(compiled.callables[0]?.requiredCapabilities).toContain('dataclass-adapter'); expect(compiled.generated.declaration).toContain('makePoint(): Promise'); expect(compiled.generated.declaration).toContain( 'export type Point = { x: number; y: number; }' ); + expect(result.ir.classes.find(cls => cls.name === 'Point')?.fields[1]?.default).toBe(true); expect(compiled.generated.typescript).toContain('"constraint":"safe-integer"'); const temporary = await mkdtemp(join(process.cwd(), 'test', '.tywrap-point-')); @@ -229,7 +242,7 @@ result.then(point => { _kwargs?: Record, validate?: (result: T) => void ): Promise { - expect([moduleName, functionName]).toEqual(['value_extensions', 'make_point']); + expect([moduleName, functionName]).toEqual(['value_contract_point', 'make_point']); requireCapability(bridgeMeta, 'dataclassFieldsV2', 'fields-v2'); const decoded = decodeExactResponse(wire, pointContract); validateDataclassOrigin(decoded, pointContract); From 20fa0f669cd75151f58c40e5a580206dba4f4918 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:45:28 -0700 Subject: [PATCH 14/35] docs: record exact integer migration costs --- docs/maintainers/value-contract-prototypes.md | 20 +++++++++++++++++++ 1 file changed, 20 insertions(+) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index b6b66c27..c35febc3 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -59,6 +59,26 @@ They must cover floats, booleans, nested records, malformed tags, digit limits, byte limits, and old bridges. Production work needs separate review of capability negotiation, generated types, and migration cost. +### Exact integer migration cost + +Opting in changes every declared Python `int` at that callable to TypeScript +`bigint`, including values within the safe number range. Callers must update +number arithmetic, comparisons, and stored JSON. `JSON.stringify` rejects a +plain `bigint`, so callers that persist results need an explicit encoding. + +Each tagged integer adds 73 JSON bytes beyond its decimal digits. For example, +`7` uses one byte as a JSON number and 74 bytes in the version 2 envelope. +Nested collections multiply that cost by their integer count. Teams must measure +representative call payloads against the byte limit before enabling the option. + +The client must check the bridge capability before it sends tagged values. This +requires a coordinated client and bridge rollout. Old bridges reject the opted +call. Arrow int64 columns keep their current type and wire format. + +Before production adoption, inventory generated call sites that use `int`, +measure payload growth, and typecheck downstream callers after the opt-in. The +prototype has not measured those application-specific costs. + ## Dataclass output option The generation option is `dataclassReturns: 'validated-v2'` for a callable with From f835f98b3f3a83c6b267e25592bbfd31ca5c0c1a Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:51:31 -0700 Subject: [PATCH 15/35] docs: separate proposed policy from prototype checks --- docs/maintainers/value-contract-prototypes.md | 29 +++++++++++-------- 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index c35febc3..5c241efa 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -11,10 +11,12 @@ Python `int` parameter and result to TypeScript `bigint`. Safe integers use the same type. Python `bool` remains `boolean`. Calls without the option keep the safe `number` rule. -The generated wrapper requires `exactIntegerDecimalV2` in the bridge's -`meta.valueCapabilities`. It sends `params.valuePolicy.integer = 'bigint-v2'` on -each call. A new bridge rejects unknown policies. An old bridge lacks the -capability, so the wrapper rejects the call before sending a tag. +The proposed generated wrapper must require `exactIntegerDecimalV2` in the +bridge's `meta.valueCapabilities`. It must send +`params.valuePolicy.integer = 'bigint-v2'` on each call. A new bridge must +reject unknown policies. An old bridge lacks the capability, so the wrapper must +reject the call before sending a tag. The prototype tests this policy in an +isolated adapter. The shipped wrapper does not implement it. The version 2 integer envelope is `{"__tywrap__":"integer","codecVersion":2,"encoding":"decimal","value":"18446744073709551617"}`. @@ -90,18 +92,21 @@ It includes fields with defaults and `init=False`. An instance returns those fields. `Optional[T]` permits `null`; a default does not make a returned property optional. -The generated wrapper requires `dataclassFieldsV2` in `meta.valueCapabilities`. -It sends `params.valuePolicy.dataclass = 'fields-v2'` on each call. The Python -encoder accepts only the compiled dataclass type. It emits a version 2 envelope +The proposed generated wrapper must require `dataclassFieldsV2` in +`meta.valueCapabilities`. It must send +`params.valuePolicy.dataclass = 'fields-v2'` on each call. The Python prototype +encoder accepts only the declared dataclass type. It emits a version 2 envelope with `module.qualname` identity and an ordered `fields` record. It rejects unsupported fields and cycles with paths. It applies depth, node, and byte limits. It does not call `dataclasses.asdict` or create object handles. -The TypeScript decoder checks marker, version, encoding, type identity, and -fields before it removes the envelope. It returns a plain record. A -module-private `WeakMap` stores verified type identity. The return validator -reads that entry before it checks field keys and values. A plain record cannot -claim dataclass origin. A nested dataclass gets its own entry. +The TypeScript prototype decoder checks marker, version, encoding, type +identity, and exact fields before it removes the envelope. It returns a plain +record. A module-private `WeakMap` stores verified type identity. The prototype +origin check reads that entry. A plain record cannot claim dataclass origin. A +nested dataclass gets its own entry. The current generated return validator +checks required field types but does not check origin or reject extra keys. The +adapter must perform those checks before it calls the validator. The Python prototype checks the declared root dataclass type. It serializes nested dataclasses by their observed runtime type and does not check a compiled From a84e9b9642fc3f0283263e0a99edb21033a06a45 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:57:16 -0700 Subject: [PATCH 16/35] test: bind generated Point call to explicit value policy adapter --- test/value_extensions_generated_point.test.ts | 94 ++++++++++++++++--- 1 file changed, 82 insertions(+), 12 deletions(-) diff --git a/test/value_extensions_generated_point.test.ts b/test/value_extensions_generated_point.test.ts index 2ee3f3a7..9de36e9b 100644 --- a/test/value_extensions_generated_point.test.ts +++ b/test/value_extensions_generated_point.test.ts @@ -44,6 +44,12 @@ const pythonEnvironment = { .join(delimiter), }; +interface PointInvocationDescriptor { + requiredCapability: 'dataclassFieldsV2'; + valuePolicy: { dataclass: 'fields-v2' }; + result: PrototypeContract; +} + function analyzedPointIr(): unknown { return JSON.parse( execFileSync( @@ -183,6 +189,11 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript) || !existsSync(fi const result = compilePoint(pointIr, true, true); const { compiled } = result; const pointContract = pointContractFrom(result); + const descriptor: PointInvocationDescriptor = { + requiredCapability: 'dataclassFieldsV2', + valuePolicy: { dataclass: 'fields-v2' }, + result: pointContract, + }; expect(compiled.diagnostics.filter(item => item.severity === 'error')).toEqual([]); expect(compiled.callables[0]?.requiredCapabilities).toContain('dataclass-adapter'); expect(compiled.generated.declaration).toContain('makePoint(): Promise'); @@ -191,20 +202,56 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript) || !existsSync(fi ); expect(result.ir.classes.find(cls => cls.name === 'Point')?.fields[1]?.default).toBe(true); expect(compiled.generated.typescript).toContain('"constraint":"safe-integer"'); + const bound = new CodeGenerator().generateModuleBindingTemplate(compiled.module); + expect(bound.declaration).toBe(compiled.generated.declaration); + expect(bound.typescript).toContain('__tywrapRuntimeProvider().call'); + expect(bound.typescript).not.toContain('getRuntimeBridge().call'); const temporary = await mkdtemp(join(process.cwd(), 'test', '.tywrap-point-')); try { - const outputPath = join(temporary, 'value_extensions.generated.mjs'); - const declarationPath = join(temporary, 'value_extensions.generated.d.ts'); + const outputPath = join(temporary, 'value_contract_point.generated.mjs'); + const declarationPath = join(temporary, 'value_contract_point.generated.d.ts'); const consumerPath = join(temporary, 'consumer.ts'); - const javascript = ts.transpileModule(compiled.generated.typescript, { + const binding = ` +import { getRuntimeBridge } from 'tywrap/runtime'; +type PolicyCall = ( + module: string, + functionName: string, + args: unknown[], + kwargs: Record | undefined, + validate: ((result: T) => void) | undefined, + descriptor: unknown +) => Promise; +function __tywrapRuntimeProvider() { + const bridge = getRuntimeBridge() as ReturnType & { + callWithValuePolicy?: PolicyCall; + }; + if (typeof bridge.callWithValuePolicy !== 'function') { + throw new Error('bridge lacks the Point value-policy adapter'); + } + return { + call( + module: string, + functionName: string, + args: unknown[], + kwargs?: Record, + validate?: (result: T) => void + ): Promise { + return bridge.callWithValuePolicy!( + module, functionName, args, kwargs, validate, ${JSON.stringify(descriptor)} + ); + }, + }; +} +`; + const javascript = ts.transpileModule(bound.typescript + binding, { compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext }, }).outputText; await writeFile(outputPath, javascript, 'utf8'); await writeFile(declarationPath, compiled.generated.declaration, 'utf8'); await writeFile( consumerPath, - `import { makePoint, type Point } from './value_extensions.generated.js'; + `import { makePoint, type Point } from './value_contract_point.generated.js'; const result: Promise = makePoint(); result.then(point => { const x: number = point.x; @@ -234,28 +281,47 @@ result.then(point => { let wire: unknown = pythonPoint(); let bridgeMeta: unknown = { valueCapabilities: ['dataclassFieldsV2'] }; - setRuntimeBridge({ - async call( + const adapterBridge = { + async call(): Promise { + throw new Error('the bound Point wrapper must use the policy adapter'); + }, + async callWithValuePolicy( moduleName: string, functionName: string, _args: unknown[], _kwargs?: Record, - validate?: (result: T) => void + validate?: (result: T) => void, + invocation?: PointInvocationDescriptor ): Promise { expect([moduleName, functionName]).toEqual(['value_contract_point', 'make_point']); - requireCapability(bridgeMeta, 'dataclassFieldsV2', 'fields-v2'); - const decoded = decodeExactResponse(wire, pointContract); - validateDataclassOrigin(decoded, pointContract); + expect(invocation).toEqual(descriptor); + if (!invocation) throw new Error('Point invocation descriptor is missing'); + requireCapability( + bridgeMeta, + invocation.requiredCapability, + invocation.valuePolicy.dataclass + ); + const decoded = decodeExactResponse(wire, invocation.result); + validateDataclassOrigin(decoded, invocation.result); validate?.(decoded as T); return decoded as T; }, async dispose(): Promise {}, - }); + }; + setRuntimeBridge(adapterBridge); const generated = (await import(pathToFileURL(outputPath).href)) as { makePoint: () => Promise<{ x: number; y: number }>; }; await expect(generated.makePoint()).resolves.toEqual({ x: 1, y: 2 }); + setRuntimeBridge({ + async call(): Promise { + throw new Error('ordinary bridge call must not run'); + }, + async dispose(): Promise {}, + }); + await expect(generated.makePoint()).rejects.toThrow(/lacks the Point value-policy adapter/); + setRuntimeBridge(adapterBridge); bridgeMeta = {}; await expect(generated.makePoint()).rejects.toThrow(/bridge lacks dataclassFieldsV2/); bridgeMeta = { valueCapabilities: ['dataclassFieldsV2'] }; @@ -264,6 +330,7 @@ result.then(point => { { ...valid, fields: { x: 1 } }, { ...valid, fields: { x: 1, y: 2, z: 3 } }, { ...valid, fields: { x: '1', y: 2 } }, + { ...valid, type: 'other.Point' }, { x: 1, y: 2 }, ]) { wire = bad; @@ -271,7 +338,10 @@ result.then(point => { } setRuntimeBridge({ - async call( + async call(): Promise { + throw new Error('ordinary bridge call must not run'); + }, + async callWithValuePolicy( _module: string, _functionName: string, _args: unknown[], From 5010dc027836fa6bd64110d600a367ff6432d599 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:59:04 -0700 Subject: [PATCH 17/35] docs: record generated Point proof and remaining boundary --- docs/maintainers/value-contract-prototypes.md | 17 ++++++++++++----- 1 file changed, 12 insertions(+), 5 deletions(-) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index 5c241efa..c08d4432 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -113,8 +113,15 @@ nested dataclasses by their observed runtime type and does not check a compiled field contract. The TypeScript decoder rejects unexpected nested identities or field values. Server-side field validation remains open. -The bounded prototype still needs a generated `Promise` wrapper from the -actual compiler model. It must return `{x: 1, y: 2}` and reject missing, extra, -and wrong typed fields. It must test defaults, optional fields, nesting, cycles, -unsupported fields, depth, and byte limits. Production work needs independent -review of identity, generated types, and migration cost. +The bounded Point proof in `test/value_extensions_generated_point.test.ts` uses +analyzer IR from an importable Python module. The compiler emits +`Promise` with required numeric `x` and `y`, including a defaulted `y`. A +bound prototype adapter returns `{x: 1, y: 2}` and rejects missing, extra, wrong +typed fields, and untagged records before the generated validator runs. The +generated validator independently rejects wrong field types. The binding rejects +a bridge without the Point policy adapter. + +The Python prototype tests optional fields, nesting, cycles, unsupported fields, +depth, and byte limits separately. Production still needs bridge capability +negotiation, a server-side compiled field contract, and an independent design +review. The default runtime has no dataclass adapter. From 58da68021380d93f728fb12055eb7aad29f53ef6 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:02:21 -0700 Subject: [PATCH 18/35] test: pin Point adapter rejection causes --- test/value_extensions_generated_point.test.ts | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/test/value_extensions_generated_point.test.ts b/test/value_extensions_generated_point.test.ts index 9de36e9b..6920ef2d 100644 --- a/test/value_extensions_generated_point.test.ts +++ b/test/value_extensions_generated_point.test.ts @@ -326,15 +326,15 @@ result.then(point => { await expect(generated.makePoint()).rejects.toThrow(/bridge lacks dataclassFieldsV2/); bridgeMeta = { valueCapabilities: ['dataclassFieldsV2'] }; const valid = pythonPoint() as { fields: Record }; - for (const bad of [ - { ...valid, fields: { x: 1 } }, - { ...valid, fields: { x: 1, y: 2, z: 3 } }, - { ...valid, fields: { x: '1', y: 2 } }, - { ...valid, type: 'other.Point' }, - { x: 1, y: 2 }, - ]) { + for (const [bad, error] of [ + [{ ...valid, fields: { x: 1 } }, /fields differ at result\.fields/], + [{ ...valid, fields: { x: 1, y: 2, z: 3 } }, /fields differ at result\.fields/], + [{ ...valid, fields: { x: '1', y: 2 } }, /expected safe integer at result\.fields\.x/], + [{ ...valid, type: 'other.Point' }, /invalid dataclass identity at result/], + [{ x: 1, y: 2 }, /fields differ at result/], + ] as const) { wire = bad; - await expect(generated.makePoint()).rejects.toThrow(); + await expect(generated.makePoint()).rejects.toThrow(error); } setRuntimeBridge({ From 18aa8defb3f34a39294993d1f61f7cd50d485e06 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 01:59:47 -0700 Subject: [PATCH 19/35] docs: mark bigint generator contract gap --- docs/maintainers/value-contract-prototypes.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index c08d4432..fc4f6af6 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -11,6 +11,10 @@ Python `int` parameter and result to TypeScript `bigint`. Safe integers use the same type. Python `bool` remains `boolean`. Calls without the option keep the safe `number` rule. +The current value-contract revision represents Python `int` only as a safe JSON +`number`. The compiler and return validator follow that rule. This prototype +does not generate `bigint` wrappers; that needs a deliberate contract extension. + The proposed generated wrapper must require `exactIntegerDecimalV2` in the bridge's `meta.valueCapabilities`. It must send `params.valuePolicy.integer = 'bigint-v2'` on each call. A new bridge must From 5b2dc3fdb4190150a124ca9ce0585beae7fa3c51 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:04:09 -0700 Subject: [PATCH 20/35] test: pin dataclass input and depth rejection --- test/python/test_value_extensions_prototype.py | 10 ++++++++++ test/value_extensions_prototype.test.ts | 3 +++ 2 files changed, 13 insertions(+) diff --git a/test/python/test_value_extensions_prototype.py b/test/python/test_value_extensions_prototype.py index f212e6a1..0ae5fef4 100644 --- a/test/python/test_value_extensions_prototype.py +++ b/test/python/test_value_extensions_prototype.py @@ -252,3 +252,13 @@ class Cyclic: prototype.encode_dataclass( prototype.Point(1, 2), prototype.Point, max_payload_bytes=60 ) + + @dataclass + class Nested: + value: object + + deep: object = 1 + for _ in range(prototype.MAX_DEPTH + 1): + deep = Nested(deep) + with pytest.raises(prototype.PrototypeError, match='maximum depth'): + prototype.encode_dataclass(deep, Nested) diff --git a/test/value_extensions_prototype.test.ts b/test/value_extensions_prototype.test.ts index 53bd73b1..fb55abb7 100644 --- a/test/value_extensions_prototype.test.ts +++ b/test/value_extensions_prototype.test.ts @@ -248,6 +248,9 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(pythonScript))( }; it('decodes Point and preserves verified origin', () => { + expect(() => encodeExactRequest({ x: 1, y: 2 }, point)).toThrow( + /dataclass inputs are unsupported at args/ + ); const wire = pythonAction('encode-point', JSON.stringify({ x: 1, y: 2 })); const decoded = decodeExactResponse(wire, point); expect(decoded).toEqual({ x: 1, y: 2 }); From c45aac010c6332bb26bfc60eb154caaf231ab9b4 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:07:56 -0700 Subject: [PATCH 21/35] test: add analyzed exact integer callable fixture --- test/fixtures/python/value_contract_exact_integer.py | 7 +++++++ 1 file changed, 7 insertions(+) create mode 100644 test/fixtures/python/value_contract_exact_integer.py diff --git a/test/fixtures/python/value_contract_exact_integer.py b/test/fixtures/python/value_contract_exact_integer.py new file mode 100644 index 00000000..82115bc9 --- /dev/null +++ b/test/fixtures/python/value_contract_exact_integer.py @@ -0,0 +1,7 @@ +"""Importable exact-integer fixture for the bounded v3 compiler proof.""" + + +def combine_exact(value: int, nested: list[int], flag: bool, ratio: float) -> int: + """Combine scalar and nested integers after the bridge decodes them.""" + total = value + sum(nested) + return total if flag and ratio >= 0 else -total From 684b9407eca80a07843a0ca31bc899b3d4b7cdf5 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:10:44 -0700 Subject: [PATCH 22/35] test: add private exact integer call adapter --- test/prototypes/value_exact_call.py | 52 +++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+) create mode 100644 test/prototypes/value_exact_call.py diff --git a/test/prototypes/value_exact_call.py b/test/prototypes/value_exact_call.py new file mode 100644 index 00000000..dff4b8c6 --- /dev/null +++ b/test/prototypes/value_exact_call.py @@ -0,0 +1,52 @@ +"""Private Python call adapter for the exact-integer compiler proof.""" + +from __future__ import annotations + +import json +import sys +from typing import Any + +from value_contract_exact_integer import combine_exact +from value_extensions import ( + MAX_PAYLOAD_BYTES, + PrototypeError, + decode_exact_integers, + encode_exact_integers, + require_capability, +) + + +def main() -> None: + trusted_contracts: list[dict[str, Any]] = json.loads(sys.argv[1]) + raw = sys.stdin.buffer.read(MAX_PAYLOAD_BYTES + 1) + if len(raw) > MAX_PAYLOAD_BYTES: + raise PrototypeError('input payload exceeds byte limit') + request: dict[str, Any] = json.loads(raw) + if (request.get('module'), request.get('functionName')) != ( + 'value_contract_exact_integer', + 'combine_exact', + ): + raise PrototypeError('unexpected exact-integer callable') + params = request.get('params') + if not isinstance(params, dict): + raise PrototypeError('missing exact-integer call parameters') + value_policy = params.get('valuePolicy') + policy = value_policy.get('integer') if isinstance(value_policy, dict) else None + require_capability(request.get('meta', {}), 'exactIntegerDecimalV2', policy) + args = params.get('args') + if not isinstance(args, list) or len(args) != len(trusted_contracts): + raise PrototypeError('exact-integer arguments differ from the contract') + decoded = [ + decode_exact_integers(value, contract) + for value, contract in zip(args, trusted_contracts, strict=True) + ] + result = combine_exact(*decoded) + encoded = encode_exact_integers(result) + wire = json.dumps( + encoded, allow_nan=False, ensure_ascii=False, separators=(',', ':') + ).encode('utf-8') + sys.stdout.buffer.write(wire + b'\n') + + +if __name__ == '__main__': + main() From bd9efc7e7e4fc91ff6cdd26057a5217b13ccf27c Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:13:33 -0700 Subject: [PATCH 23/35] test: keep adapter capability metadata outside request --- test/prototypes/value_exact_call.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/test/prototypes/value_exact_call.py b/test/prototypes/value_exact_call.py index dff4b8c6..6f55ab18 100644 --- a/test/prototypes/value_exact_call.py +++ b/test/prototypes/value_exact_call.py @@ -18,6 +18,7 @@ def main() -> None: trusted_contracts: list[dict[str, Any]] = json.loads(sys.argv[1]) + bridge_meta: dict[str, Any] = json.loads(sys.argv[2]) raw = sys.stdin.buffer.read(MAX_PAYLOAD_BYTES + 1) if len(raw) > MAX_PAYLOAD_BYTES: raise PrototypeError('input payload exceeds byte limit') @@ -32,7 +33,7 @@ def main() -> None: raise PrototypeError('missing exact-integer call parameters') value_policy = params.get('valuePolicy') policy = value_policy.get('integer') if isinstance(value_policy, dict) else None - require_capability(request.get('meta', {}), 'exactIntegerDecimalV2', policy) + require_capability(bridge_meta, 'exactIntegerDecimalV2', policy) args = params.get('args') if not isinstance(args, list) or len(args) != len(trusted_contracts): raise PrototypeError('exact-integer arguments differ from the contract') From 120832216b3c005905296c3455eb46cfc45d1f72 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:14:47 -0700 Subject: [PATCH 24/35] test: check exact call arguments as one bounded record --- test/prototypes/value_exact_call.py | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-) diff --git a/test/prototypes/value_exact_call.py b/test/prototypes/value_exact_call.py index 6f55ab18..d1fe0a74 100644 --- a/test/prototypes/value_exact_call.py +++ b/test/prototypes/value_exact_call.py @@ -17,7 +17,7 @@ def main() -> None: - trusted_contracts: list[dict[str, Any]] = json.loads(sys.argv[1]) + trusted_contract: dict[str, Any] = json.loads(sys.argv[1]) bridge_meta: dict[str, Any] = json.loads(sys.argv[2]) raw = sys.stdin.buffer.read(MAX_PAYLOAD_BYTES + 1) if len(raw) > MAX_PAYLOAD_BYTES: @@ -35,13 +35,18 @@ def main() -> None: policy = value_policy.get('integer') if isinstance(value_policy, dict) else None require_capability(bridge_meta, 'exactIntegerDecimalV2', policy) args = params.get('args') - if not isinstance(args, list) or len(args) != len(trusted_contracts): + fields = trusted_contract.get('fields') + if ( + trusted_contract.get('kind') != 'record' + or not isinstance(fields, dict) + or not isinstance(args, list) + or len(args) != len(fields) + ): raise PrototypeError('exact-integer arguments differ from the contract') - decoded = [ - decode_exact_integers(value, contract) - for value, contract in zip(args, trusted_contracts, strict=True) - ] - result = combine_exact(*decoded) + input_record = dict(zip(fields, args, strict=True)) + decoded = decode_exact_integers(input_record, trusted_contract) + assert isinstance(decoded, dict) + result = combine_exact(**decoded) encoded = encode_exact_integers(result) wire = json.dumps( encoded, allow_nan=False, ensure_ascii=False, separators=(',', ':') From 9aec49feece2ad9ea2dc41bdc3927553ae96a394 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:15:59 -0700 Subject: [PATCH 25/35] docs: measure representative exact integer payload growth --- docs/maintainers/value-contract-prototypes.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index fc4f6af6..331592aa 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -77,6 +77,12 @@ Each tagged integer adds 73 JSON bytes beyond its decimal digits. For example, Nested collections multiply that cost by their integer count. Teams must measure representative call payloads against the byte limit before enabling the option. +For a compact `{"args": [...]}` payload with three integers, a boolean, and a +float, the safe-value example grows from 28 to 247 UTF-8 bytes (8.82 times). +Using one 81-bit and one 131-bit integer grows the same shape from 91 to 310 +bytes (3.41 times). These figures exclude the surrounding RPC envelope and do +not estimate application traffic. + The client must check the bridge capability before it sends tagged values. This requires a coordinated client and bridge rollout. Old bridges reject the opted call. Arrow int64 columns keep their current type and wire format. From 8d437b7c3d2fc883f5a9f60c406b8cffee0e6379 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:17:16 -0700 Subject: [PATCH 26/35] test: make exact fixture distinguish signed zero --- test/fixtures/python/value_contract_exact_integer.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/test/fixtures/python/value_contract_exact_integer.py b/test/fixtures/python/value_contract_exact_integer.py index 82115bc9..6fd47a3c 100644 --- a/test/fixtures/python/value_contract_exact_integer.py +++ b/test/fixtures/python/value_contract_exact_integer.py @@ -1,7 +1,9 @@ """Importable exact-integer fixture for the bounded v3 compiler proof.""" +import math + def combine_exact(value: int, nested: list[int], flag: bool, ratio: float) -> int: """Combine scalar and nested integers after the bridge decodes them.""" total = value + sum(nested) - return total if flag and ratio >= 0 else -total + return total if flag and math.copysign(1.0, ratio) > 0 else -total From d8b29a451b444f7a0c0fd8932721235f227e4523 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:48:46 -0700 Subject: [PATCH 27/35] feat(contracts): define pinned exact integer revision 3 --- .../value-contract-fixtures.v3.json | 36 ++++++ docs/maintainers/value-contracts.v3.json | 50 +++++++++ src/contracts/value-contract.ts | 85 +++++++++----- test/value_contracts_v3.test.ts | 106 ++++++++++++++++++ 4 files changed, 247 insertions(+), 30 deletions(-) create mode 100644 docs/maintainers/value-contract-fixtures.v3.json create mode 100644 docs/maintainers/value-contracts.v3.json create mode 100644 test/value_contracts_v3.test.ts diff --git a/docs/maintainers/value-contract-fixtures.v3.json b/docs/maintainers/value-contract-fixtures.v3.json new file mode 100644 index 00000000..f5d419af --- /dev/null +++ b/docs/maintainers/value-contract-fixtures.v3.json @@ -0,0 +1,36 @@ +{ + "revision": 3, + "integerCases": [ + { + "logicalDecimal": "0", + "envelope": { "__tywrap__": "integer", "codecVersion": 2, "encoding": "decimal", "value": "0" } + }, + { + "logicalDecimal": "9007199254740991", + "envelope": { "__tywrap__": "integer", "codecVersion": 2, "encoding": "decimal", "value": "9007199254740991" } + }, + { + "logicalDecimal": "18446744073709551617", + "envelope": { "__tywrap__": "integer", "codecVersion": 2, "encoding": "decimal", "value": "18446744073709551617" } + }, + { + "logicalDecimal": "-18446744073709551617", + "envelope": { + "__tywrap__": "integer", + "codecVersion": 2, + "encoding": "decimal", + "value": "-18446744073709551617" + } + } + ], + "rejectedDecimals": ["-0", "+1", "01", "1.0", "1e3", "١", ""], + "floatNegativeZero": { + "envelope": { "__tywrap__": "float", "codecVersion": 2, "encoding": "negative-zero" }, + "decodedAs": "negative zero" + }, + "migration": { + "default": "Python int remains a safe JavaScript number under revision 2", + "opted": "Python int becomes bigint at every declared integer node under revision 3", + "caller": "bigint arithmetic and JSON persistence require explicit handling" + } +} diff --git a/docs/maintainers/value-contracts.v3.json b/docs/maintainers/value-contracts.v3.json new file mode 100644 index 00000000..014a2f39 --- /dev/null +++ b/docs/maintainers/value-contracts.v3.json @@ -0,0 +1,50 @@ +{ + "revision": 3, + "status": "bounded-internal-prototype", + "base": { + "file": "value-contracts.v2.json", + "revision": 2, + "sha256": "554f84b75126513250a7eb773294347fe4dc42262e4ac52f86e7aa19cc65bb4e" + }, + "transport": "tywrap/1 value RPC", + "valueEnvelopeCodecVersion": 2, + "callPolicy": "bigint-v2", + "requiredCapability": "exactIntegerDecimalV2", + "rules": { + "integer-exact": { + "logicalKind": "Python int, excluding bool, on an opted call", + "contract": { + "kind": "integer-exact", + "wire": "decimal-string-envelope", + "decodedAs": "bigint", + "constraint": "exact-integer" + }, + "wire": { + "__tywrap__": "integer", + "codecVersion": 2, + "encoding": "decimal", + "value": "canonical signed decimal string" + }, + "decimalGrammar": "^(?:0|[1-9][0-9]*|-[1-9][0-9]*)$", + "maximumDigitsWithoutSign": 4096, + "composition": "apply the exact integer rule at every selected sequence, tuple, union, and record node" + }, + "float-negative-zero": { + "logicalKind": "Python float equal to negative zero on an opted call", + "wire": { + "__tywrap__": "float", + "codecVersion": 2, + "encoding": "negative-zero" + }, + "decodedAs": "JavaScript negative zero", + "otherFiniteFloats": "JSON number under the inherited finite float rule" + } + }, + "compatibility": { + "revision2Default": "unchanged safe-number integer rule", + "unknownRevision": "reject", + "missingCapability": "reject before sending a tagged request", + "oldClientNewProvider": "revision 2 behavior unless the call opts in", + "arrowInt64": "keep the existing Arrow rule" + } +} diff --git a/src/contracts/value-contract.ts b/src/contracts/value-contract.ts index d8bc033b..63eb1db4 100644 --- a/src/contracts/value-contract.ts +++ b/src/contracts/value-contract.ts @@ -1,10 +1,17 @@ /** Internal value policy consumed by codecs and callable-contract generation. */ export const VALUE_CONTRACT_REVISION = 2 as const; +/** Exact integer mode uses a separate semantic contract revision. */ +export const VALUE_CONTRACT_V3_REVISION = 3 as const; + +export type ValueContractRevision = + | typeof VALUE_CONTRACT_REVISION + | typeof VALUE_CONTRACT_V3_REVISION; + /** JSON numbers can carry Python integers exactly only within this range. */ export const MAX_SAFE_JSON_INTEGER = 2 ** 53 - 1; -export type ValueContract = +type CommonValueContractLeaf = | { readonly kind: 'null'; readonly wire: 'json'; readonly decodedAs: 'null' } | { readonly kind: 'boolean'; readonly wire: 'json'; readonly decodedAs: 'boolean' } | { @@ -21,36 +28,11 @@ export type ValueContract = } | { readonly kind: 'string'; readonly wire: 'json'; readonly decodedAs: 'string' } | { readonly kind: 'bytes'; readonly wire: 'base64-envelope'; readonly decodedAs: 'Uint8Array' } - | { - readonly kind: 'sequence'; - readonly wire: 'json'; - readonly decodedAs: 'array'; - readonly item: ValueContract; - } - | { - readonly kind: 'tuple'; - readonly wire: 'json'; - readonly decodedAs: 'array'; - readonly items: readonly ValueContract[]; - } - | { - readonly kind: 'union'; - readonly wire: 'selected-option'; - readonly decodedAs: 'selected-option'; - readonly options: readonly [ValueContract, ValueContract, ...ValueContract[]]; - } - | { - readonly kind: 'record'; - readonly wire: 'json'; - readonly decodedAs: 'object'; - readonly fields: readonly ValueContractField[]; - readonly additionalValues?: ValueContract; - } | { readonly kind: 'ndarray-float16'; readonly wire: 'arrow' | 'json-fallback'; readonly decodedAs: 'nested-array-or-scalar'; - readonly element: Extract; + readonly element: Extract; readonly dtype: 'float16'; readonly rank?: number; } @@ -59,7 +41,7 @@ export type ValueContract = readonly wire: 'ndarray-envelope'; readonly decodedAs: 'tensor-record'; readonly dtype: 'torch.float16'; - readonly value: Extract; + readonly value: Extract; } | { readonly kind: 'unsupported'; @@ -69,9 +51,52 @@ export type ValueContract = readonly guidance: string; }; -export interface ValueContractField { +type CompositeValueContract = + | { + readonly kind: 'sequence'; + readonly wire: 'json'; + readonly decodedAs: 'array'; + readonly item: TValue; + } + | { + readonly kind: 'tuple'; + readonly wire: 'json'; + readonly decodedAs: 'array'; + readonly items: readonly TValue[]; + } + | { + readonly kind: 'union'; + readonly wire: 'selected-option'; + readonly decodedAs: 'selected-option'; + readonly options: readonly [TValue, TValue, ...TValue[]]; + } + | { + readonly kind: 'record'; + readonly wire: 'json'; + readonly decodedAs: 'object'; + readonly fields: readonly ValueContractField[]; + readonly additionalValues?: TValue; + }; + +/** Revision 2 remains the default safe-number value contract. */ +export type ValueContract = CommonValueContractLeaf | CompositeValueContract; + +export interface ExactIntegerValueContract { + readonly kind: 'integer-exact'; + readonly wire: 'decimal-string-envelope'; + readonly decodedAs: 'bigint'; + readonly constraint: 'exact-integer'; +} + +/** Revision 3 adds exact integers without changing revision 2. */ +export type ValueContractV3 = + | CommonValueContractLeaf + | ExactIntegerValueContract + | CompositeValueContract; + +export interface ValueContractField { readonly name: string; - readonly value: ValueContract; + readonly value: TValue; readonly required: boolean; } diff --git a/test/value_contracts_v3.test.ts b/test/value_contracts_v3.test.ts new file mode 100644 index 00000000..70db9b6e --- /dev/null +++ b/test/value_contracts_v3.test.ts @@ -0,0 +1,106 @@ +import { createHash } from 'node:crypto'; +import { readFileSync } from 'node:fs'; +import { describe, expect, it } from 'vitest'; +import { + VALUE_CONTRACT_REVISION, + VALUE_CONTRACT_V3_REVISION, + type ExactIntegerValueContract, + type ValueContractV3, +} from '../src/contracts/value-contract.js'; + +const specification = JSON.parse( + readFileSync(new URL('../docs/maintainers/value-contracts.v3.json', import.meta.url), 'utf8') +) as { + revision: number; + base: { file: string; revision: number; sha256: string }; + valueEnvelopeCodecVersion: number; + callPolicy: string; + requiredCapability: string; + rules: { + 'integer-exact': { + contract: ExactIntegerValueContract; + decimalGrammar: string; + maximumDigitsWithoutSign: number; + }; + }; +}; + +const fixtures = JSON.parse( + readFileSync( + new URL('../docs/maintainers/value-contract-fixtures.v3.json', import.meta.url), + 'utf8' + ) +) as { + revision: number; + integerCases: readonly { + logicalDecimal: string; + envelope: { __tywrap__: string; codecVersion: number; encoding: string; value: string }; + }[]; + rejectedDecimals: readonly string[]; +}; + +describe('exact integer value policy', () => { + it('pins revision 3 to the unchanged revision-2 source', () => { + const base = readFileSync( + new URL(`../docs/maintainers/${specification.base.file}`, import.meta.url) + ); + const baseRevision = JSON.parse(base.toString('utf8')) as { revision: number }; + expect(specification.revision).toBe(VALUE_CONTRACT_V3_REVISION); + expect(fixtures.revision).toBe(VALUE_CONTRACT_V3_REVISION); + expect(specification.base.revision).toBe(VALUE_CONTRACT_REVISION); + expect(baseRevision.revision).toBe(VALUE_CONTRACT_REVISION); + expect(createHash('sha256').update(base).digest('hex')).toBe(specification.base.sha256); + }); + + it('keeps semantic revision, envelope version, and call policy distinct', () => { + expect(specification.revision).toBe(3); + expect(specification.valueEnvelopeCodecVersion).toBe(2); + expect(specification.callPolicy).toBe('bigint-v2'); + expect(specification.requiredCapability).toBe('exactIntegerDecimalV2'); + expect(specification.rules['integer-exact'].contract).toEqual({ + kind: 'integer-exact', + wire: 'decimal-string-envelope', + decodedAs: 'bigint', + constraint: 'exact-integer', + }); + }); + + it('permits exact integer leaves inside the shared composite shapes', () => { + const nested: ValueContractV3 = { + kind: 'record', + wire: 'json', + decodedAs: 'object', + fields: [ + { + name: 'values', + required: true, + value: { + kind: 'sequence', + wire: 'json', + decodedAs: 'array', + item: specification.rules['integer-exact'].contract, + }, + }, + ], + }; + expect(nested.fields[0]?.value.kind).toBe('sequence'); + }); + + it('fixes the canonical decimal grammar and proof values', () => { + const rule = specification.rules['integer-exact']; + const decimal = new RegExp(rule.decimalGrammar); + expect(rule.maximumDigitsWithoutSign).toBe(4096); + for (const fixture of fixtures.integerCases) { + expect(decimal.test(fixture.logicalDecimal)).toBe(true); + expect(fixture.envelope).toEqual({ + __tywrap__: 'integer', + codecVersion: 2, + encoding: 'decimal', + value: fixture.logicalDecimal, + }); + } + for (const value of fixtures.rejectedDecimals) { + expect(decimal.test(value)).toBe(false); + } + }); +}); From 38c9b84c4eb520a72986ecb007888433deef031f Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:50:21 -0700 Subject: [PATCH 28/35] test: map resolved v3 nodes into prototype codec --- test/prototypes/value_contract_v3_adapter.ts | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+) create mode 100644 test/prototypes/value_contract_v3_adapter.ts diff --git a/test/prototypes/value_contract_v3_adapter.ts b/test/prototypes/value_contract_v3_adapter.ts new file mode 100644 index 00000000..17f9eb73 --- /dev/null +++ b/test/prototypes/value_contract_v3_adapter.ts @@ -0,0 +1,19 @@ +/** Translate resolved v3 nodes into the isolated cross-language test codec. */ + +import type { ValueContractV3 } from '../../src/contracts/value-contract.js'; +import type { PrototypeContract } from './value_extensions.js'; + +export function toExactPrototypeContract(value: ValueContractV3): PrototypeContract { + switch (value.kind) { + case 'integer-exact': + return { kind: 'integer' }; + case 'boolean': + return { kind: 'boolean' }; + case 'float': + return { kind: 'float' }; + case 'sequence': + return { kind: 'array', item: toExactPrototypeContract(value.item) }; + default: + throw new Error(`The exact call fixture cannot use ${value.kind}.`); + } +} From a3bdc13d3d8fac178932cd589478691faae570fc Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:56:02 -0700 Subject: [PATCH 29/35] feat(codegen): compile exact integers behind v3 capability --- docs/maintainers/value-contracts.v3.json | 2 +- src/contracts/value-contract.ts | 20 ++++ src/core/callable-compiler.ts | 92 ++++++++++++++--- src/core/generator.ts | 123 +++++++++++++++++++++-- src/runtime/validators.ts | 10 +- src/types/index.ts | 6 +- test/callable-compiler.test.ts | 91 +++++++++++++++++ test/runtime_return_validation.test.ts | 19 ++++ test/value_contracts_v3.test.ts | 4 +- 9 files changed, 335 insertions(+), 32 deletions(-) diff --git a/docs/maintainers/value-contracts.v3.json b/docs/maintainers/value-contracts.v3.json index 014a2f39..169add95 100644 --- a/docs/maintainers/value-contracts.v3.json +++ b/docs/maintainers/value-contracts.v3.json @@ -7,7 +7,7 @@ "sha256": "554f84b75126513250a7eb773294347fe4dc42262e4ac52f86e7aa19cc65bb4e" }, "transport": "tywrap/1 value RPC", - "valueEnvelopeCodecVersion": 2, + "exactIntegerEnvelopeCodecVersion": 2, "callPolicy": "bigint-v2", "requiredCapability": "exactIntegerDecimalV2", "rules": { diff --git a/src/contracts/value-contract.ts b/src/contracts/value-contract.ts index 63eb1db4..75c65aad 100644 --- a/src/contracts/value-contract.ts +++ b/src/contracts/value-contract.ts @@ -100,6 +100,26 @@ export interface ValueContractField { readonly required: boolean; } +export function containsExactInteger(value: ValueContractV3): boolean { + switch (value.kind) { + case 'integer-exact': + return true; + case 'sequence': + return containsExactInteger(value.item); + case 'tuple': + return value.items.some(containsExactInteger); + case 'union': + return value.options.some(containsExactInteger); + case 'record': + return ( + value.fields.some(field => containsExactInteger(field.value)) || + (value.additionalValues !== undefined && containsExactInteger(value.additionalValues)) + ); + default: + return false; + } +} + export function isSafeJsonInteger(value: number): boolean { return Number.isInteger(value) && Math.abs(value) <= MAX_SAFE_JSON_INTEGER; } diff --git a/src/core/callable-compiler.ts b/src/core/callable-compiler.ts index 9a0d6726..ebf637ea 100644 --- a/src/core/callable-compiler.ts +++ b/src/core/callable-compiler.ts @@ -7,8 +7,11 @@ import { VALUE_CONTRACT_REVISION, - type ValueContract, + VALUE_CONTRACT_V3_REVISION, + containsExactInteger, type ValueContractField, + type ValueContractRevision, + type ValueContractV3, } from '../contracts/value-contract.js'; import type { GeneratedCode, @@ -29,7 +32,8 @@ export type CallableCapability = | 'scientific-ndarray' | 'scientific-torch' | 'coroutine-execution' - | 'dataclass-adapter'; + | 'dataclass-adapter' + | 'exact-integer-adapter'; export interface CapabilityDescription { name: CallableCapability; @@ -51,7 +55,7 @@ export interface ValueConversionRequest { export interface SupportedValueResolution { status: 'supported'; - value: ValueContract; + value: ValueContractV3; } export interface UnsupportedValueResolution { @@ -76,7 +80,7 @@ export type ValueResolution = * The compiler does not infer wire behavior from a TypeScript spelling. */ export interface ValueConversionDescription { - revision: typeof VALUE_CONTRACT_REVISION; + revision: ValueContractRevision; resolve(request: ValueConversionRequest): ValueResolution; } @@ -170,6 +174,11 @@ export const DEFAULT_CALLABLE_CAPABILITIES: readonly CapabilityDescription[] = [ guidance: 'Use a TypedDict or an explicit record adapter until #339 defines dataclass conversion.', }, + { + name: 'exact-integer-adapter', + available: false, + guidance: 'Bind a revision-3 exact integer provider before generating this callable.', + }, ]; function leafName(type: PythonType): string | null { @@ -197,10 +206,14 @@ function firstUnresolvedChild( return resolutions.find(resolution => resolution.status !== 'supported') ?? null; } +function rootWireKinds(value: ValueContractV3): ValueContractV3['kind'][] { + return value.kind === 'union' ? value.options.flatMap(rootWireKinds) : [value.kind]; +} + function resolveSequence( item: PythonType, request: ValueConversionRequest, - conversion: ValueConversionDescription + conversion: Pick ): ValueResolution { const child = conversion.resolve({ ...request, @@ -220,7 +233,7 @@ function resolveSequence( function resolveRecord( valueType: PythonType, request: ValueConversionRequest, - conversion: ValueConversionDescription + conversion: Pick ): ValueResolution { const child = conversion.resolve({ ...request, @@ -246,7 +259,7 @@ function resolveRecord( function resolveTuple( items: readonly PythonType[], request: ValueConversionRequest, - conversion: ValueConversionDescription + conversion: Pick ): ValueResolution { if (items.length === 0) { return { @@ -289,7 +302,7 @@ function resolveTuple( function resolveUnion( options: readonly PythonType[], request: ValueConversionRequest, - conversion: ValueConversionDescription + conversion: Pick ): ValueResolution { if (options.length < 2 || options.length > 32) { return { @@ -320,6 +333,14 @@ function resolveUnion( guidance: 'Provide two explicit value alternatives.', }; } + const wireKinds = values.flatMap(entry => rootWireKinds(entry.value)); + if (wireKinds.includes('integer-exact') && wireKinds.includes('record')) { + return { + status: 'unsupported', + reason: 'An exact integer tag can overlap a record wire value.', + guidance: 'Use a disjoint tagged record instead of this union.', + }; + } return { status: 'supported', value: { @@ -393,9 +414,9 @@ export const DEFAULT_VALUE_CONVERSION: ValueConversionDescription = { revision: VALUE_CONTRACT_REVISION, resolve(request): ValueResolution { const { logicalType: type } = request; - const nestedConversion: ValueConversionDescription = request.resolveNested - ? { revision: VALUE_CONTRACT_REVISION, resolve: request.resolveNested } - : DEFAULT_VALUE_CONVERSION; + const nestedConversion: Pick = { + resolve: request.resolveNested ?? DEFAULT_VALUE_CONVERSION.resolve, + }; if ((request.depth ?? 0) > 64) { return { status: 'unsupported', @@ -526,10 +547,35 @@ export const DEFAULT_VALUE_CONVERSION: ValueConversionDescription = { }, }; -function capabilityNamesFor(value: ValueContract): CallableCapability[] { +/** Explicit revision-3 mode uses the revision-2 rules for every other value. */ +export const EXACT_INTEGER_VALUE_CONVERSION: ValueConversionDescription = { + revision: VALUE_CONTRACT_V3_REVISION, + resolve(request): ValueResolution { + if (request.logicalType.kind === 'primitive' && request.logicalType.name === 'int') { + return { + status: 'supported', + value: { + kind: 'integer-exact', + wire: 'decimal-string-envelope', + decodedAs: 'bigint', + constraint: 'exact-integer', + }, + }; + } + return DEFAULT_VALUE_CONVERSION.resolve({ + ...request, + resolveNested: request.resolveNested ?? EXACT_INTEGER_VALUE_CONVERSION.resolve, + }); + }, +}; + +function capabilityNamesFor(value: ValueContractV3): CallableCapability[] { const names = new Set(['value-rpc', 'return-validation']); - const collect = (current: ValueContract): void => { + const collect = (current: ValueContractV3): void => { switch (current.kind) { + case 'integer-exact': + names.add('exact-integer-adapter'); + break; case 'sequence': collect(current.item); break; @@ -613,7 +659,7 @@ function outputType(value: ResolvedCallableValue): PythonType { return value.resolution.status === 'supported' ? value.logicalType : UNKNOWN_TYPE; } -function valuesMayOverlap(left: ValueContract, right: ValueContract): boolean { +function valuesMayOverlap(left: ValueContractV3, right: ValueContractV3): boolean { if (left.kind === 'unsupported' || right.kind === 'unsupported') { return true; } @@ -809,6 +855,14 @@ function resolveCallable( ]; const requiredCapabilities = new Set(); for (const value of values) { + if (value.resolution.status === 'supported' && containsExactInteger(value.resolution.value)) { + if (conversion.revision !== VALUE_CONTRACT_V3_REVISION) { + throw new Error(`${value.path}: exact integers require value contract revision 3.`); + } + if (capabilities.get('exact-integer-adapter')?.available !== true) { + throw new Error(`${value.path}: exact-integer-adapter is unavailable.`); + } + } const diagnostic = diagnosticForResolution(value.resolution, value.path); if (diagnostic) { diagnostics.push(diagnostic); @@ -913,7 +967,7 @@ function resolveCallable( })), returnType: overloadResults[index] ? outputType(overloadResults[index]) : overload.returnType, })); - const supportedValue = (value: ResolvedCallableValue): ValueContract | undefined => + const supportedValue = (value: ResolvedCallableValue): ValueContractV3 | undefined => value.resolution.status === 'supported' ? value.resolution.value : undefined; const resolvedFunction: PythonFunction = { ...func, @@ -961,6 +1015,12 @@ export function compileContract( ir: ValidatedIrContract, options: CompileContractOptions ): CompiledContract { + if ( + options.conversion.revision !== VALUE_CONTRACT_REVISION && + options.conversion.revision !== VALUE_CONTRACT_V3_REVISION + ) { + throw new Error(`Unsupported value contract revision ${options.conversion.revision}.`); + } if (ir.module !== options.module.name) { throw new Error( `Compiled module ${options.module.name} does not match IR module ${ir.module}.` @@ -1025,7 +1085,7 @@ export function compileContract( guidance: 'Flatten the value or provide a bounded adapter.', }; } - const fields: ValueContractField[] = []; + const fields: ValueContractField[] = []; for (const property of typedDict.properties) { const field = conversion.resolve({ ...request, diff --git a/src/core/generator.ts b/src/core/generator.ts index 41cddf4e..493918cf 100644 --- a/src/core/generator.ts +++ b/src/core/generator.ts @@ -13,7 +13,7 @@ import type { GeneratedCode, TypescriptType, } from '../types/index.js'; -import type { ValueContract } from '../contracts/value-contract.js'; +import { containsExactInteger, type ValueContractV3 } from '../contracts/value-contract.js'; import type { OverloadReturnSchema, ReturnSchema } from '../runtime/validators.js'; import { globalCache } from '../utils/cache.js'; @@ -41,7 +41,7 @@ interface GenericRenderContext { type ReturnDefinitionNames = ReadonlySet & { readonly moduleName?: string }; -function valueContractToReturnSchema(value: ValueContract): ReturnSchema { +function valueContractToReturnSchema(value: ValueContractV3): ReturnSchema { switch (value.kind) { case 'null': return { kind: 'primitive', type: 'null' }; @@ -49,6 +49,8 @@ function valueContractToReturnSchema(value: ValueContract): ReturnSchema { return { kind: 'primitive', type: 'boolean' }; case 'integer': return { kind: 'primitive', type: 'number', constraint: value.constraint }; + case 'integer-exact': + return { kind: 'primitive', type: 'bigint' }; case 'float': return { kind: 'primitive', type: 'number', constraint: value.constraint }; case 'string': @@ -83,6 +85,77 @@ function valueContractToReturnSchema(value: ValueContract): ReturnSchema { } } +function valueContractToTsType(value: ValueContractV3): string { + switch (value.kind) { + case 'null': + return 'null'; + case 'boolean': + return 'boolean'; + case 'integer': + case 'float': + return 'number'; + case 'integer-exact': + return 'bigint'; + case 'string': + return 'string'; + case 'bytes': + return 'Uint8Array'; + case 'sequence': { + const item = valueContractToTsType(value.item); + return `${value.item.kind === 'union' ? `(${item})` : item}[]`; + } + case 'tuple': + return `[${value.items.map(valueContractToTsType).join(', ')}]`; + case 'union': + return value.options.map(valueContractToTsType).join(' | '); + case 'record': { + const fields = value.fields + .map( + field => + `${JSON.stringify(field.name)}${field.required ? '' : '?'}: ${valueContractToTsType(field.value)};` + ) + .join(' '); + const record = `{ ${fields} }`; + if (!value.additionalValues) { + return record; + } + const additional = `Record`; + return value.fields.length === 0 ? additional : `(${record} & ${additional})`; + } + case 'ndarray-float16': + return '__tywrapFloat16Value'; + case 'torch-float16': + return '__tywrapFloat16Tensor'; + case 'unsupported': + return 'unknown'; + } +} + +function containsScientificValue( + value: ValueContractV3, + kind: 'ndarray-float16' | 'torch-float16' +): boolean { + if (value.kind === kind) { + return true; + } + switch (value.kind) { + case 'sequence': + return containsScientificValue(value.item, kind); + case 'tuple': + return value.items.some(item => containsScientificValue(item, kind)); + case 'union': + return value.options.some(option => containsScientificValue(option, kind)); + case 'record': + return ( + value.fields.some(field => containsScientificValue(field.value, kind)) || + (value.additionalValues !== undefined && + containsScientificValue(value.additionalValues, kind)) + ); + default: + return false; + } +} + export interface CodeGeneratorOptions { /** Reports a generated annotation that cannot be represented by emitted declarations. */ onTypeDegrade?: (typeName: string) => void; @@ -449,10 +522,13 @@ export class CodeGenerator { } private resolvedReturnType( - contract: ValueContract | undefined, + contract: ValueContractV3 | undefined, logicalType: PythonType, ctx: GenericRenderContext ): string { + if (contract && containsExactInteger(contract)) { + return valueContractToTsType(contract); + } if (contract?.kind === 'ndarray-float16') { return '__tywrapFloat16Value'; } @@ -786,8 +862,28 @@ export class CodeGenerator { ); const typeParamDecl = genericContext.declaration; - const tsTypeForValue = (p: Parameter): string => - this.typeToTsFromPython(p.type, genericContext, 'value'); + const parameterContracts = new WeakMap(); + func.parameters.forEach((parameter, index) => { + const value = func.callableContract?.parameterValues[index]; + if (value) { + parameterContracts.set(parameter, value); + } + }); + func.overloads?.forEach((overload, overloadIndex) => { + overload.parameters.forEach((parameter, parameterIndex) => { + const value = + func.callableContract?.overloads[overloadIndex]?.parameterValues[parameterIndex]; + if (value) { + parameterContracts.set(parameter, value); + } + }); + }); + const tsTypeForValue = (p: Parameter): string => { + const value = parameterContracts.get(p); + return value && containsExactInteger(value) + ? valueContractToTsType(value) + : this.typeToTsFromPython(p.type, genericContext, 'value'); + }; const kwargsType = (() => { if (!needsKwargsParam) { @@ -1501,13 +1597,22 @@ ${migrationNote}${declarationMethodsSection} ) ), ]; - const resolvedReturns = emittedCallables.flatMap(func => [ + const resolvedValues = emittedCallables.flatMap(func => [ + ...(func.callableContract?.parameterValues ?? []), func.callableContract?.returnValue, - ...(func.callableContract?.overloads.map(overload => overload.returnValue) ?? []), + ...(func.callableContract?.overloads.flatMap(overload => [ + ...overload.parameterValues, + overload.returnValue, + ]) ?? []), ]); - const needsTorchFloat16 = resolvedReturns.some(value => value?.kind === 'torch-float16'); + const needsTorchFloat16 = resolvedValues.some( + value => value !== undefined && containsScientificValue(value, 'torch-float16') + ); const needsFloat16 = - needsTorchFloat16 || resolvedReturns.some(value => value?.kind === 'ndarray-float16'); + needsTorchFloat16 || + resolvedValues.some( + value => value !== undefined && containsScientificValue(value, 'ndarray-float16') + ); const scientificTypes = needsFloat16 ? `type __tywrapFloat16Value = number | __tywrapFloat16Value[];\n${ needsTorchFloat16 diff --git a/src/runtime/validators.ts b/src/runtime/validators.ts index cabbafda..c0b56bbb 100644 --- a/src/runtime/validators.ts +++ b/src/runtime/validators.ts @@ -26,7 +26,15 @@ export type ReturnSchema = | { kind: 'any' } | { kind: 'primitive'; - type: 'number' | 'string' | 'boolean' | 'null' | 'undefined' | 'Uint8Array' | 'object'; + type: + | 'number' + | 'bigint' + | 'string' + | 'boolean' + | 'null' + | 'undefined' + | 'Uint8Array' + | 'object'; constraint?: 'safe-integer' | 'finite'; } | { kind: 'literal'; value: string | number | boolean | null } diff --git a/src/types/index.ts b/src/types/index.ts index 90546f36..b717b0fb 100644 --- a/src/types/index.ts +++ b/src/types/index.ts @@ -2,7 +2,7 @@ * Core type definitions for tywrap */ -import type { ValueContract } from '../contracts/value-contract.js'; +import type { ValueContractV3 } from '../contracts/value-contract.js'; import type { DecodedProvenance } from '../runtime/decoded-provenance.js'; export interface PythonModule { @@ -72,8 +72,8 @@ export interface PythonFunctionOverload { /** A resolved value contract for a callable signature. */ export interface ResolvedCallableSignatureContract { - parameterValues: readonly (ValueContract | undefined)[]; - returnValue?: ValueContract; + parameterValues: readonly (ValueContractV3 | undefined)[]; + returnValue?: ValueContractV3; /** Original annotation used for best-effort validation when no value contract resolves. */ returnValidationType?: PythonType; } diff --git a/test/callable-compiler.test.ts b/test/callable-compiler.test.ts index e154674f..f255e745 100644 --- a/test/callable-compiler.test.ts +++ b/test/callable-compiler.test.ts @@ -9,6 +9,7 @@ import { compileContract, DEFAULT_CALLABLE_CAPABILITIES, DEFAULT_VALUE_CONVERSION, + EXACT_INTEGER_VALUE_CONVERSION, } from '../src/core/callable-compiler.js'; import { CodeGenerator } from '../src/core/generator.js'; import { parseAnnotationToPythonType } from '../src/core/annotation-parser.js'; @@ -1701,6 +1702,96 @@ describe('compileContract', () => { }); }); + it('uses revision 3 for explicit nested bigint callables only', () => { + const source = rawIr.functions[1]!; + const ir = validateIrContract( + { + ...rawIr, + functions: [ + { + ...source, + name: 'combine_exact', + qualname: 'fixture.combine_exact', + parameters: [ + { name: 'value', kind: 'POSITIONAL_OR_KEYWORD', annotation: 'int', default: false }, + { + name: 'nested', + kind: 'POSITIONAL_OR_KEYWORD', + annotation: 'list[int]', + default: false, + }, + { name: 'flag', kind: 'POSITIONAL_OR_KEYWORD', annotation: 'bool', default: false }, + { name: 'ratio', kind: 'POSITIONAL_OR_KEYWORD', annotation: 'float', default: false }, + ], + returns: 'int', + }, + ], + }, + 'exact integer callable contract' + ); + expect(ir.ok).toBe(true); + if (!ir.ok) { + return; + } + const module = { + ...moduleModel, + functions: [{ ...moduleModel.functions[1]!, name: 'combine_exact' }], + classes: [], + }; + const options = { + module, + generator: new CodeGenerator(), + conversion: EXACT_INTEGER_VALUE_CONVERSION, + capabilities: DEFAULT_CALLABLE_CAPABILITIES, + }; + expect(() => compileContract(ir.contract, options)).toThrow( + /exact-integer-adapter is unavailable/ + ); + const capabilities = DEFAULT_CALLABLE_CAPABILITIES.map(capability => + capability.name === 'exact-integer-adapter' ? { ...capability, available: true } : capability + ); + const exact = compileContract(ir.contract, { ...options, capabilities }); + expect(exact.callables[0]?.requiredCapabilities).toContain('exact-integer-adapter'); + expect(exact.callables[0]?.parameters[1]?.resolution).toMatchObject({ + status: 'supported', + value: { kind: 'sequence', item: { kind: 'integer-exact' } }, + }); + expect(exact.generated.declaration).toContain('value: bigint'); + expect(exact.generated.declaration).toContain('nested: bigint[]'); + expect(exact.generated.declaration).toContain('flag: boolean'); + expect(exact.generated.declaration).toContain('ratio: number'); + expect(exact.generated.declaration).toContain('Promise'); + expect(exact.generated.typescript).toContain('"kind":"primitive","type":"bigint"'); + + const defaultContract = compileContract(ir.contract, { + ...options, + conversion: DEFAULT_VALUE_CONVERSION, + }); + expect(defaultContract.generated.declaration).toContain('value: number'); + expect(defaultContract.generated.declaration).toContain('nested: number[]'); + expect(defaultContract.generated.declaration).toContain('Promise'); + }); + + it('keeps exact integer tags disjoint from record union options', () => { + expect( + EXACT_INTEGER_VALUE_CONVERSION.resolve({ + direction: 'output', + path: '$.returns', + logicalType: parseAnnotationToPythonType('int | dict[str, str]'), + }) + ).toMatchObject({ + status: 'unsupported', + reason: 'An exact integer tag can overlap a record wire value.', + }); + expect( + EXACT_INTEGER_VALUE_CONVERSION.resolve({ + direction: 'output', + path: '$.returns', + logicalType: parseAnnotationToPythonType('int | bool'), + }) + ).toMatchObject({ status: 'supported', value: { kind: 'union' } }); + }); + it('requires returned dataclass fields even when their constructor has defaults', () => { const point = rawIr.classes[0]!; const ir = validateIrContract( diff --git a/test/runtime_return_validation.test.ts b/test/runtime_return_validation.test.ts index ca35a82c..7a4594f2 100644 --- a/test/runtime_return_validation.test.ts +++ b/test/runtime_return_validation.test.ts @@ -41,6 +41,25 @@ describe('generated return validators', () => { expect(() => validator({ count: Number.MAX_SAFE_INTEGER + 1 })).toThrow(BridgeValidationError); }); + it('requires bigint at each exact integer position', () => { + const validator = createReturnValidator( + { + kind: 'record', + fields: { + values: { + schema: { kind: 'array', element: { kind: 'primitive', type: 'bigint' } }, + }, + }, + }, + 'fixture.exactValues' + ); + expect(validator({ values: [1n, 18446744073709551617n] })).toEqual({ + values: [1n, 18446744073709551617n], + }); + expect(() => validator({ values: [1, 2n] })).toThrow(BridgeValidationError); + expect(() => validator({ values: ['1'] })).toThrow(/expected bigint, received string/); + }); + it('requires per-call provenance for a scalar float16 ndarray', () => { const validate = createReturnValidator( { kind: 'marker', marker: 'ndarray', dims: 0, dtype: 'float16' }, diff --git a/test/value_contracts_v3.test.ts b/test/value_contracts_v3.test.ts index 70db9b6e..046f1cee 100644 --- a/test/value_contracts_v3.test.ts +++ b/test/value_contracts_v3.test.ts @@ -13,7 +13,7 @@ const specification = JSON.parse( ) as { revision: number; base: { file: string; revision: number; sha256: string }; - valueEnvelopeCodecVersion: number; + exactIntegerEnvelopeCodecVersion: number; callPolicy: string; requiredCapability: string; rules: { @@ -54,7 +54,7 @@ describe('exact integer value policy', () => { it('keeps semantic revision, envelope version, and call policy distinct', () => { expect(specification.revision).toBe(3); - expect(specification.valueEnvelopeCodecVersion).toBe(2); + expect(specification.exactIntegerEnvelopeCodecVersion).toBe(2); expect(specification.callPolicy).toBe('bigint-v2'); expect(specification.requiredCapability).toBe('exactIntegerDecimalV2'); expect(specification.rules['integer-exact'].contract).toEqual({ From d6f1bacf13aeaf709b5b974a7d34cf4c5ca58e93 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:59:29 -0700 Subject: [PATCH 30/35] test: prove generated exact bigint wrapper through bound provider --- ...value_extensions_generated_integer.test.ts | 380 ++++++++++++++++++ 1 file changed, 380 insertions(+) create mode 100644 test/value_extensions_generated_integer.test.ts diff --git a/test/value_extensions_generated_integer.test.ts b/test/value_extensions_generated_integer.test.ts new file mode 100644 index 00000000..1c54943d --- /dev/null +++ b/test/value_extensions_generated_integer.test.ts @@ -0,0 +1,380 @@ +import { execFileSync } from 'node:child_process'; +import { existsSync } from 'node:fs'; +import { mkdtemp, rm, writeFile } from 'node:fs/promises'; +import { delimiter, join } from 'node:path'; +import { pathToFileURL } from 'node:url'; +import ts from 'typescript'; +import { describe, expect, it } from 'vitest'; +import { + compileContract, + DEFAULT_CALLABLE_CAPABILITIES, + DEFAULT_VALUE_CONVERSION, + EXACT_INTEGER_VALUE_CONVERSION, + type CompiledContract, +} from '../src/core/callable-compiler.js'; +import { CodeGenerator } from '../src/core/generator.js'; +import { validateIrContract } from '../src/core/ir-contract.js'; +import { transformIrToTsModel } from '../src/core/ir-model.js'; +import { BridgeValidationError } from '../src/runtime/errors.js'; +import { clearRuntimeBridge, setRuntimeBridge } from 'tywrap/runtime'; +import type { PythonModule } from '../src/types/index.js'; +import { PYTHON, PYTHON_AVAILABLE } from './helpers/python-probe.js'; +import { + decodeExactResponse, + encodeExactRequest, + type PrototypeContract, +} from './prototypes/value_extensions.js'; +import { toExactPrototypeContract } from './prototypes/value_contract_v3_adapter.js'; + +const fixtureDir = join(process.cwd(), 'test', 'fixtures', 'python'); +const fixturePath = join(fixtureDir, 'value_contract_exact_integer.py'); +const pythonAdapter = join(process.cwd(), 'test', 'prototypes', 'value_exact_call.py'); +const pythonEnvironment = { + ...process.env, + PYTHONPATH: [ + fixtureDir, + join(process.cwd(), 'tywrap_ir'), + join(process.cwd(), 'test', 'prototypes'), + process.env.PYTHONPATH, + ] + .filter(Boolean) + .join(delimiter), +}; + +interface ExactInvocationDescriptor { + requiredCapability: 'exactIntegerDecimalV2'; + valuePolicy: { integer: 'bigint-v2' }; + args: PrototypeContract; + result: PrototypeContract; +} + +function analyzedIr(): unknown { + return JSON.parse( + execFileSync( + PYTHON ?? 'python3', + [ + '-c', + "from tywrap_ir.ir import emit_ir_json; print(emit_ir_json('value_contract_exact_integer', pretty=False))", + ], + { env: pythonEnvironment, encoding: 'utf8' } + ) + ) as unknown; +} + +function compileInteger(ir: unknown, exact: boolean, bound = false): CompiledContract { + const validated = validateIrContract(ir, 'analyzed exact integer IR'); + expect(validated.ok).toBe(true); + if (!validated.ok) throw new Error('Exact integer IR is invalid'); + const module = transformIrToTsModel(validated.contract); + const generator = new CodeGenerator(); + return compileContract(validated.contract, { + module, + generator: bound + ? { + generateModuleDefinition(model: PythonModule, annotatedJSDoc = false) { + return generator.generateModuleBindingTemplate(model, annotatedJSDoc); + }, + } + : generator, + conversion: exact ? EXACT_INTEGER_VALUE_CONVERSION : DEFAULT_VALUE_CONVERSION, + capabilities: DEFAULT_CALLABLE_CAPABILITIES.map(capability => + capability.name === 'exact-integer-adapter' ? { ...capability, available: exact } : capability + ), + }); +} + +function descriptorFrom(compiled: CompiledContract): ExactInvocationDescriptor { + const callable = compiled.callables.find(item => item.name === 'combine_exact'); + const functionModel = compiled.module.functions.find(item => item.name === 'combine_exact'); + if (!callable || !functionModel || callable.result.resolution.status !== 'supported') { + throw new Error('Exact callable result is unresolved'); + } + const fields: Record = {}; + for (const [index, parameter] of functionModel.parameters.entries()) { + const resolution = callable.parameters[index]?.resolution; + if (!resolution || resolution.status !== 'supported') { + throw new Error(`Exact parameter ${parameter.name} is unresolved`); + } + fields[parameter.name] = toExactPrototypeContract(resolution.value); + } + return { + requiredCapability: 'exactIntegerDecimalV2', + valuePolicy: { integer: 'bigint-v2' }, + args: { kind: 'record', fields }, + result: toExactPrototypeContract(callable.result.resolution.value), + }; +} + +describe.skipIf(!PYTHON_AVAILABLE || !existsSync(fixturePath) || !existsSync(pythonAdapter))( + 'generated exact integer proof', + () => { + it('keeps default numeric types and fails closed without exact adapter capability', () => { + const ir = analyzedIr(); + const ordinary = compileInteger(ir, false); + expect(ordinary.generated.declaration).toContain('value: number'); + expect(ordinary.generated.declaration).toContain('nested: number[]'); + expect(ordinary.generated.declaration).toContain('Promise'); + + const validated = validateIrContract(ir, 'exact integer capability control'); + expect(validated.ok).toBe(true); + if (!validated.ok) throw new Error('Exact integer IR is invalid'); + const module = transformIrToTsModel(validated.contract); + expect(() => + compileContract(validated.contract, { + module, + generator: new CodeGenerator(), + conversion: EXACT_INTEGER_VALUE_CONVERSION, + capabilities: DEFAULT_CALLABLE_CAPABILITIES, + }) + ).toThrow(/exact-integer-adapter is unavailable/); + }); + + it('executes a bound bigint wrapper with pre-send capability checks', async () => { + const compiled = compileInteger(analyzedIr(), true, true); + const descriptor = descriptorFrom(compiled); + expect(compiled.callables[0]?.requiredCapabilities).toContain('exact-integer-adapter'); + expect(compiled.generated.declaration).toContain('value: bigint'); + expect(compiled.generated.declaration).toContain('nested: bigint[]'); + expect(compiled.generated.declaration).toContain('flag: boolean'); + expect(compiled.generated.declaration).toContain('ratio: number'); + expect(compiled.generated.declaration).toContain('Promise'); + expect(compiled.generated.typescript).toContain('"kind":"primitive","type":"bigint"'); + expect(compiled.generated.typescript).toContain('__tywrapRuntimeProvider().call'); + + const temporary = await mkdtemp(join(process.cwd(), 'test', '.tywrap-integer-')); + try { + const outputPath = join(temporary, 'value_contract_exact_integer.generated.mjs'); + const declarationPath = join(temporary, 'value_contract_exact_integer.generated.d.ts'); + const consumerPath = join(temporary, 'consumer.ts'); + const binding = ` +import { getRuntimeBridge } from 'tywrap/runtime'; +type PolicyCall = ( + module: string, + functionName: string, + args: unknown[], + kwargs: Record | undefined, + validate: ((result: T) => void) | undefined, + descriptor: unknown +) => Promise; +function __tywrapRuntimeProvider() { + const bridge = getRuntimeBridge() as ReturnType & { + meta?: { valueCapabilities?: unknown }; + callWithValuePolicy?: PolicyCall; + }; + const capabilities = bridge.meta?.valueCapabilities; + if (!Array.isArray(capabilities) || !capabilities.includes('exactIntegerDecimalV2')) { + throw new Error('bridge lacks exactIntegerDecimalV2 capability'); + } + if (typeof bridge.callWithValuePolicy !== 'function') { + throw new Error('bridge lacks the exact integer value-policy adapter'); + } + return { + call( + module: string, + functionName: string, + args: unknown[], + kwargs?: Record, + validate?: (result: T) => void + ): Promise { + return bridge.callWithValuePolicy!( + module, functionName, args, kwargs, validate, ${JSON.stringify(descriptor)} + ); + }, + }; +} +`; + const javascript = ts.transpileModule(compiled.generated.typescript + binding, { + compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext }, + }).outputText; + await writeFile(outputPath, javascript, 'utf8'); + await writeFile(declarationPath, compiled.generated.declaration, 'utf8'); + await writeFile( + consumerPath, + `import { combineExact } from './value_contract_exact_integer.generated.js'; +const result: Promise = combineExact(1n, [2n], true, 1.5); +result.then(value => { + const valid: bigint = value; + // @ts-expect-error Exact result is bigint. + const wrong: number = value; + return valid + BigInt(wrong); +}); +// @ts-expect-error Exact input is bigint. +combineExact(1, [2n], true, 1.5); +// @ts-expect-error Nested exact input is bigint[]. +combineExact(1n, [2], true, 1.5);`, + 'utf8' + ); + const program = ts.createProgram([consumerPath], { + module: ts.ModuleKind.NodeNext, + moduleResolution: ts.ModuleResolutionKind.NodeNext, + target: ts.ScriptTarget.ES2022, + strict: true, + noEmit: true, + skipLibCheck: true, + types: [], + }); + expect( + ts + .getPreEmitDiagnostics(program) + .map(diagnostic => ts.flattenDiagnosticMessageText(diagnostic.messageText, '\n')) + ).toEqual([]); + + let sent = 0; + let responseOverride: unknown; + const bridge = { + meta: { valueCapabilities: ['exactIntegerDecimalV2'] }, + async call(): Promise { + throw new Error('the exact wrapper must use its policy adapter'); + }, + async callWithValuePolicy( + moduleName: string, + functionName: string, + args: unknown[], + _kwargs?: Record, + validate?: (result: T) => void, + invocation?: ExactInvocationDescriptor + ): Promise { + expect([moduleName, functionName]).toEqual([ + 'value_contract_exact_integer', + 'combine_exact', + ]); + expect(invocation).toEqual(descriptor); + if (!invocation || invocation.args.kind !== 'record') { + throw new Error('Exact invocation descriptor is missing'); + } + const names = Object.keys(invocation.args.fields); + const input = Object.fromEntries(names.map((name, index) => [name, args[index]])); + const encoded = encodeExactRequest(input, invocation.args) as Record; + const request = { + module: moduleName, + functionName, + params: { + valuePolicy: invocation.valuePolicy, + args: names.map(name => encoded[name]), + }, + }; + const raw = JSON.stringify(request); + if (new TextEncoder().encode(raw).length > 10 * 1024 * 1024) { + throw new Error('exact request exceeds byte limit'); + } + sent += 1; + const response = JSON.parse( + execFileSync( + PYTHON ?? 'python3', + [pythonAdapter, JSON.stringify(invocation.args), JSON.stringify(this.meta)], + { input: raw, env: pythonEnvironment, encoding: 'utf8', maxBuffer: 1024 * 1024 } + ) + ) as unknown; + const decoded = decodeExactResponse( + responseOverride === undefined ? response : responseOverride, + invocation.result + ); + validate?.(decoded as T); + return decoded as T; + }, + async dispose(): Promise {}, + }; + setRuntimeBridge(bridge); + const generated = (await import(pathToFileURL(outputPath).href)) as { + combineExact: ( + value: bigint, + nested: bigint[], + flag: boolean, + ratio: number + ) => Promise; + }; + + const positive = 2n ** 80n + 1n; + const negative = -(2n ** 130n + 7n); + await expect(generated.combineExact(positive, [7n, negative], true, 1.5)).resolves.toBe( + positive + 7n + negative + ); + await expect(generated.combineExact(7n, [2n], true, 0)).resolves.toBe(9n); + await expect(generated.combineExact(7n, [2n], true, -0)).resolves.toBe(-9n); + await expect(generated.combineExact(7n, [2n], false, 1.5)).resolves.toBe(-9n); + + const beforeMissing = sent; + bridge.meta = { valueCapabilities: [] }; + await expect(generated.combineExact(7n, [2n], true, 1.5)).rejects.toThrow( + /bridge lacks exactIntegerDecimalV2 capability/ + ); + expect(sent).toBe(beforeMissing); + bridge.meta = { valueCapabilities: ['exactIntegerDecimalV2'] }; + responseOverride = { + __tywrap__: 'integer', + codecVersion: 1, + encoding: 'decimal', + value: '9', + }; + await expect(generated.combineExact(7n, [2n], true, 1.5)).rejects.toThrow( + /invalid integer envelope/ + ); + responseOverride = 9; + await expect(generated.combineExact(7n, [2n], true, 1.5)).rejects.toThrow( + /expected integer envelope/ + ); + + const bypassBridge = { + meta: { valueCapabilities: ['exactIntegerDecimalV2'] }, + async call(): Promise { + throw new Error('ordinary call must not run'); + }, + async callWithValuePolicy( + _module: string, + _functionName: string, + _args: unknown[], + _kwargs?: Record, + validate?: (result: T) => void + ): Promise { + const wrong = 9 as unknown as T; + validate?.(wrong); + return wrong; + }, + async dispose(): Promise {}, + }; + setRuntimeBridge(bypassBridge); + await expect(generated.combineExact(7n, [2n], true, 1.5)).rejects.toThrow( + BridgeValidationError + ); + + const ordinary = compileInteger(analyzedIr(), false); + const ordinaryPath = join(temporary, 'value_contract_integer_default.generated.mjs'); + await writeFile( + ordinaryPath, + ts.transpileModule(ordinary.generated.typescript, { + compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ESNext }, + }).outputText, + 'utf8' + ); + let ordinaryCalls = 0; + let exactCalls = 0; + const compatibleBridge = { + async call(): Promise { + ordinaryCalls += 1; + return 9 as T; + }, + async callWithValuePolicy(): Promise { + exactCalls += 1; + throw new Error('default calls must not select the exact policy'); + }, + async dispose(): Promise {}, + }; + setRuntimeBridge(compatibleBridge); + const defaultGenerated = (await import(pathToFileURL(ordinaryPath).href)) as { + combineExact: ( + value: number, + nested: number[], + flag: boolean, + ratio: number + ) => Promise; + }; + await expect(defaultGenerated.combineExact(7, [2], true, 1.5)).resolves.toBe(9); + expect(ordinaryCalls).toBe(1); + expect(exactCalls).toBe(0); + } finally { + clearRuntimeBridge(); + await rm(temporary, { recursive: true, force: true }); + } + }); + } +); From 694bc001e0335f2cb8446264a390fe55dcfdb8ca Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 02:58:48 -0700 Subject: [PATCH 31/35] fix(contracts): make recursive revisions typecheck --- src/contracts/value-contract.ts | 76 +++++++++++++++++++++------------ 1 file changed, 48 insertions(+), 28 deletions(-) diff --git a/src/contracts/value-contract.ts b/src/contracts/value-contract.ts index 75c65aad..b8b8c207 100644 --- a/src/contracts/value-contract.ts +++ b/src/contracts/value-contract.ts @@ -51,35 +51,47 @@ type CommonValueContractLeaf = readonly guidance: string; }; -type CompositeValueContract = - | { - readonly kind: 'sequence'; - readonly wire: 'json'; - readonly decodedAs: 'array'; - readonly item: TValue; - } - | { - readonly kind: 'tuple'; - readonly wire: 'json'; - readonly decodedAs: 'array'; - readonly items: readonly TValue[]; - } - | { - readonly kind: 'union'; - readonly wire: 'selected-option'; - readonly decodedAs: 'selected-option'; - readonly options: readonly [TValue, TValue, ...TValue[]]; - } - | { - readonly kind: 'record'; - readonly wire: 'json'; - readonly decodedAs: 'object'; - readonly fields: readonly ValueContractField[]; - readonly additionalValues?: TValue; - }; +interface SequenceContract { + readonly kind: 'sequence'; + readonly wire: 'json'; + readonly decodedAs: 'array'; + readonly item: TValue; +} + +interface TupleContract { + readonly kind: 'tuple'; + readonly wire: 'json'; + readonly decodedAs: 'array'; + readonly items: readonly TValue[]; +} + +interface UnionContract { + readonly kind: 'union'; + readonly wire: 'selected-option'; + readonly decodedAs: 'selected-option'; + readonly options: readonly [TValue, TValue, ...TValue[]]; +} + +interface RecordContract { + readonly kind: 'record'; + readonly wire: 'json'; + readonly decodedAs: 'object'; + readonly fields: readonly ValueContractField[]; + readonly additionalValues?: TValue; +} + +interface V2SequenceContract extends SequenceContract {} +interface V2TupleContract extends TupleContract {} +interface V2UnionContract extends UnionContract {} +interface V2RecordContract extends RecordContract {} /** Revision 2 remains the default safe-number value contract. */ -export type ValueContract = CommonValueContractLeaf | CompositeValueContract; +export type ValueContract = + | CommonValueContractLeaf + | V2SequenceContract + | V2TupleContract + | V2UnionContract + | V2RecordContract; export interface ExactIntegerValueContract { readonly kind: 'integer-exact'; @@ -88,11 +100,19 @@ export interface ExactIntegerValueContract { readonly constraint: 'exact-integer'; } +interface V3SequenceContract extends SequenceContract {} +interface V3TupleContract extends TupleContract {} +interface V3UnionContract extends UnionContract {} +interface V3RecordContract extends RecordContract {} + /** Revision 3 adds exact integers without changing revision 2. */ export type ValueContractV3 = | CommonValueContractLeaf | ExactIntegerValueContract - | CompositeValueContract; + | V3SequenceContract + | V3TupleContract + | V3UnionContract + | V3RecordContract; export interface ValueContractField { readonly name: string; From ba138a49b56b9142bf0bd907405e789cda937884 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 03:03:57 -0700 Subject: [PATCH 32/35] docs: state bounded generated bigint proof and rollout gap --- docs/maintainers/value-contract-prototypes.md | 40 +++++++++++-------- 1 file changed, 24 insertions(+), 16 deletions(-) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index 331592aa..9e9546cc 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -6,21 +6,24 @@ import them. ## Exact integer option -The generation option is `integerMode: 'bigint'` for one callable. It maps every -Python `int` parameter and result to TypeScript `bigint`. Safe integers use the -same type. Python `bool` remains `boolean`. Calls without the option keep the -safe `number` rule. +The proposed public option is `integerMode: 'bigint'` for one callable. It would +map every Python `int` parameter and result to TypeScript `bigint`. Safe +integers use the same type. Python `bool` remains `boolean`. Calls without the +option keep the safe `number` rule. -The current value-contract revision represents Python `int` only as a safe JSON -`number`. The compiler and return validator follow that rule. This prototype -does not generate `bigint` wrappers; that needs a deliberate contract extension. +The default value-contract revision 2 represents Python `int` as a safe JSON +`number`. The bounded revision 3 compiler proof selects +`EXACT_INTEGER_VALUE_CONVERSION` for a module with one callable. It emits a +`bigint` wrapper and validator through a test-only runtime binding. A public +per-callable option and production bridge adapter are not implemented. The proposed generated wrapper must require `exactIntegerDecimalV2` in the bridge's `meta.valueCapabilities`. It must send `params.valuePolicy.integer = 'bigint-v2'` on each call. A new bridge must reject unknown policies. An old bridge lacks the capability, so the wrapper must -reject the call before sending a tag. The prototype tests this policy in an -isolated adapter. The shipped wrapper does not implement it. +reject the call before sending a tag. The bound proof checks the capability +before its private adapter encodes arguments. That adapter sends the policy to +the Python prototype. The default bridge does not implement this mode. The version 2 integer envelope is `{"__tywrap__":"integer","codecVersion":2,"encoding":"decimal","value":"18446744073709551617"}`. @@ -46,9 +49,9 @@ rejects malformed float tags and nonfinite numbers. The Python response encoder tags every `int`, including safe values. The TypeScript response decoder returns `bigint` at declared integer nodes. The -generated return validator must require `bigint`. It must reject an ordinary -number and a raw tag. Arrow int64 keeps its current Arrow encoding and decoded -type. +generated revision 3 return validator requires `bigint`. The prototype decoder +rejects an ordinary number and a raw tag. Arrow int64 keeps its current Arrow +encoding and decoded type. Both prototypes walk nested arrays and string-keyed records within depth and node limits. They reject unsupported values and malformed envelopes. Encoders @@ -60,10 +63,15 @@ characters directly, as `JSON.stringify` does. Both paths reject unpaired surrogates. Production must check raw wire bytes before parsing; object-level checks do not prove equal byte counts for alternate numeric JSON spellings. -The bounded tests must round trip positive and negative values beyond 64 bits. -They must cover floats, booleans, nested records, malformed tags, digit limits, -byte limits, and old bridges. Production work needs separate review of -capability negotiation, generated types, and migration cost. +The bounded tests round trip positive and negative values beyond 64 bits. They +cover floats, booleans, nested records, malformed tags, digit limits, byte +limits, and old bridges. `test/value_extensions_generated_integer.test.ts` uses +analyzer IR to check `bigint` input, `bigint[]` input, `Promise` output, +and the generated bigint return validator. It also checks signed zero and that +an incapable bridge receives no exact request. The same fixture under revision 2 +keeps `number` types and uses an ordinary call. Production still needs a public +opt-in selector, bridge capability negotiation, and application migration +measurements. ### Exact integer migration cost From d06e4fa3e3869504a4591852633e63336e6f8cbf Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 03:16:12 -0700 Subject: [PATCH 33/35] test: prove generated nested exact integer return and request caps --- docs/maintainers/value-contract-prototypes.md | 13 +-- .../python/value_contract_exact_integer.py | 5 + test/prototypes/value_exact_call.py | 11 ++- ...value_extensions_generated_integer.test.ts | 97 +++++++++++++++---- 4 files changed, 100 insertions(+), 26 deletions(-) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index 9e9546cc..9ff53d2a 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -66,12 +66,13 @@ checks do not prove equal byte counts for alternate numeric JSON spellings. The bounded tests round trip positive and negative values beyond 64 bits. They cover floats, booleans, nested records, malformed tags, digit limits, byte limits, and old bridges. `test/value_extensions_generated_integer.test.ts` uses -analyzer IR to check `bigint` input, `bigint[]` input, `Promise` output, -and the generated bigint return validator. It also checks signed zero and that -an incapable bridge receives no exact request. The same fixture under revision 2 -keeps `number` types and uses an ordinary call. Production still needs a public -opt-in selector, bridge capability negotiation, and application migration -measurements. +analyzer IR to check `bigint` and `bigint[]` inputs, `Promise` and +`Promise` outputs, and generated bigint return validation at a +nested index. It checks signed zero, a client request limit before dispatch, +the Python raw input limit, and that an incapable bridge receives no exact +request. The same fixture under revision 2 keeps `number` types and uses an +ordinary call. Production still needs a public opt-in selector, bridge +capability negotiation, and application migration measurements. ### Exact integer migration cost diff --git a/test/fixtures/python/value_contract_exact_integer.py b/test/fixtures/python/value_contract_exact_integer.py index 6fd47a3c..b9a2d431 100644 --- a/test/fixtures/python/value_contract_exact_integer.py +++ b/test/fixtures/python/value_contract_exact_integer.py @@ -7,3 +7,8 @@ def combine_exact(value: int, nested: list[int], flag: bool, ratio: float) -> in """Combine scalar and nested integers after the bridge decodes them.""" total = value + sum(nested) return total if flag and math.copysign(1.0, ratio) > 0 else -total + + +def echo_exact(value: int, nested: list[int]) -> list[list[int]]: + """Return nested values after the bridge decodes them.""" + return [[value], nested, [-value]] diff --git a/test/prototypes/value_exact_call.py b/test/prototypes/value_exact_call.py index d1fe0a74..fae0cdc4 100644 --- a/test/prototypes/value_exact_call.py +++ b/test/prototypes/value_exact_call.py @@ -6,7 +6,7 @@ import sys from typing import Any -from value_contract_exact_integer import combine_exact +from value_contract_exact_integer import combine_exact, echo_exact from value_extensions import ( MAX_PAYLOAD_BYTES, PrototypeError, @@ -19,13 +19,18 @@ def main() -> None: trusted_contract: dict[str, Any] = json.loads(sys.argv[1]) bridge_meta: dict[str, Any] = json.loads(sys.argv[2]) + function_name = sys.argv[3] + functions = {'combine_exact': combine_exact, 'echo_exact': echo_exact} + function = functions.get(function_name) + if function is None: + raise PrototypeError('unknown exact-integer callable') raw = sys.stdin.buffer.read(MAX_PAYLOAD_BYTES + 1) if len(raw) > MAX_PAYLOAD_BYTES: raise PrototypeError('input payload exceeds byte limit') request: dict[str, Any] = json.loads(raw) if (request.get('module'), request.get('functionName')) != ( 'value_contract_exact_integer', - 'combine_exact', + function_name, ): raise PrototypeError('unexpected exact-integer callable') params = request.get('params') @@ -46,7 +51,7 @@ def main() -> None: input_record = dict(zip(fields, args, strict=True)) decoded = decode_exact_integers(input_record, trusted_contract) assert isinstance(decoded, dict) - result = combine_exact(**decoded) + result = function(**decoded) encoded = encode_exact_integers(result) wire = json.dumps( encoded, allow_nan=False, ensure_ascii=False, separators=(',', ':') diff --git a/test/value_extensions_generated_integer.test.ts b/test/value_extensions_generated_integer.test.ts index 1c54943d..b9a0a7bf 100644 --- a/test/value_extensions_generated_integer.test.ts +++ b/test/value_extensions_generated_integer.test.ts @@ -83,9 +83,9 @@ function compileInteger(ir: unknown, exact: boolean, bound = false): CompiledCon }); } -function descriptorFrom(compiled: CompiledContract): ExactInvocationDescriptor { - const callable = compiled.callables.find(item => item.name === 'combine_exact'); - const functionModel = compiled.module.functions.find(item => item.name === 'combine_exact'); +function descriptorFrom(compiled: CompiledContract, name: string): ExactInvocationDescriptor { + const callable = compiled.callables.find(item => item.name === name); + const functionModel = compiled.module.functions.find(item => item.name === name); if (!callable || !functionModel || callable.result.resolution.status !== 'supported') { throw new Error('Exact callable result is unresolved'); } @@ -114,6 +114,7 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(fixturePath) || !existsSync(pyt expect(ordinary.generated.declaration).toContain('value: number'); expect(ordinary.generated.declaration).toContain('nested: number[]'); expect(ordinary.generated.declaration).toContain('Promise'); + expect(ordinary.generated.declaration).toContain('Promise'); const validated = validateIrContract(ir, 'exact integer capability control'); expect(validated.ok).toBe(true); @@ -131,13 +132,19 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(fixturePath) || !existsSync(pyt it('executes a bound bigint wrapper with pre-send capability checks', async () => { const compiled = compileInteger(analyzedIr(), true, true); - const descriptor = descriptorFrom(compiled); - expect(compiled.callables[0]?.requiredCapabilities).toContain('exact-integer-adapter'); + const descriptors = { + combine_exact: descriptorFrom(compiled, 'combine_exact'), + echo_exact: descriptorFrom(compiled, 'echo_exact'), + }; + for (const callable of compiled.callables) { + expect(callable.requiredCapabilities).toContain('exact-integer-adapter'); + } expect(compiled.generated.declaration).toContain('value: bigint'); expect(compiled.generated.declaration).toContain('nested: bigint[]'); expect(compiled.generated.declaration).toContain('flag: boolean'); expect(compiled.generated.declaration).toContain('ratio: number'); expect(compiled.generated.declaration).toContain('Promise'); + expect(compiled.generated.declaration).toContain('Promise'); expect(compiled.generated.typescript).toContain('"kind":"primitive","type":"bigint"'); expect(compiled.generated.typescript).toContain('__tywrapRuntimeProvider().call'); @@ -148,6 +155,7 @@ describe.skipIf(!PYTHON_AVAILABLE || !existsSync(fixturePath) || !existsSync(pyt const consumerPath = join(temporary, 'consumer.ts'); const binding = ` import { getRuntimeBridge } from 'tywrap/runtime'; +const __tywrapExactDescriptors = ${JSON.stringify(descriptors)} as Record; type PolicyCall = ( module: string, functionName: string, @@ -176,8 +184,12 @@ function __tywrapRuntimeProvider() { kwargs?: Record, validate?: (result: T) => void ): Promise { + const descriptor = __tywrapExactDescriptors[functionName]; + if (!descriptor) { + throw new Error('exact integer descriptor is missing'); + } return bridge.callWithValuePolicy!( - module, functionName, args, kwargs, validate, ${JSON.stringify(descriptor)} + module, functionName, args, kwargs, validate, descriptor ); }, }; @@ -190,18 +202,27 @@ function __tywrapRuntimeProvider() { await writeFile(declarationPath, compiled.generated.declaration, 'utf8'); await writeFile( consumerPath, - `import { combineExact } from './value_contract_exact_integer.generated.js'; + `import { combineExact, echoExact } from './value_contract_exact_integer.generated.js'; const result: Promise = combineExact(1n, [2n], true, 1.5); +const nested: Promise = echoExact(1n, [2n]); result.then(value => { const valid: bigint = value; // @ts-expect-error Exact result is bigint. const wrong: number = value; return valid + BigInt(wrong); }); +nested.then(rows => { + const valid: bigint = rows[0]![0]!; + // @ts-expect-error Nested exact result is bigint. + const wrong: number = rows[0]![0]!; + return valid + BigInt(wrong); +}); // @ts-expect-error Exact input is bigint. combineExact(1, [2n], true, 1.5); // @ts-expect-error Nested exact input is bigint[]. -combineExact(1n, [2], true, 1.5);`, +combineExact(1n, [2], true, 1.5); +// @ts-expect-error Nested exact input is bigint[]. +echoExact(1n, [2]);`, 'utf8' ); const program = ts.createProgram([consumerPath], { @@ -221,6 +242,7 @@ combineExact(1n, [2], true, 1.5);`, let sent = 0; let responseOverride: unknown; + let maxRequestBytes = 10 * 1024 * 1024; const bridge = { meta: { valueCapabilities: ['exactIntegerDecimalV2'] }, async call(): Promise { @@ -234,11 +256,9 @@ combineExact(1n, [2], true, 1.5);`, validate?: (result: T) => void, invocation?: ExactInvocationDescriptor ): Promise { - expect([moduleName, functionName]).toEqual([ - 'value_contract_exact_integer', - 'combine_exact', - ]); - expect(invocation).toEqual(descriptor); + expect(moduleName).toBe('value_contract_exact_integer'); + expect(['combine_exact', 'echo_exact']).toContain(functionName); + expect(invocation).toEqual(descriptors[functionName as keyof typeof descriptors]); if (!invocation || invocation.args.kind !== 'record') { throw new Error('Exact invocation descriptor is missing'); } @@ -254,14 +274,19 @@ combineExact(1n, [2], true, 1.5);`, }, }; const raw = JSON.stringify(request); - if (new TextEncoder().encode(raw).length > 10 * 1024 * 1024) { + if (new TextEncoder().encode(raw).length > maxRequestBytes) { throw new Error('exact request exceeds byte limit'); } sent += 1; const response = JSON.parse( execFileSync( PYTHON ?? 'python3', - [pythonAdapter, JSON.stringify(invocation.args), JSON.stringify(this.meta)], + [ + pythonAdapter, + JSON.stringify(invocation.args), + JSON.stringify(this.meta), + functionName, + ], { input: raw, env: pythonEnvironment, encoding: 'utf8', maxBuffer: 1024 * 1024 } ) ) as unknown; @@ -282,6 +307,7 @@ combineExact(1n, [2], true, 1.5);`, flag: boolean, ratio: number ) => Promise; + echoExact: (value: bigint, nested: bigint[]) => Promise; }; const positive = 2n ** 80n + 1n; @@ -292,6 +318,19 @@ combineExact(1n, [2], true, 1.5);`, await expect(generated.combineExact(7n, [2n], true, 0)).resolves.toBe(9n); await expect(generated.combineExact(7n, [2n], true, -0)).resolves.toBe(-9n); await expect(generated.combineExact(7n, [2n], false, 1.5)).resolves.toBe(-9n); + await expect(generated.echoExact(positive, [negative, 7n])).resolves.toEqual([ + [positive], + [negative, 7n], + [-positive], + ]); + + const beforeOversize = sent; + maxRequestBytes = 100; + await expect(generated.echoExact(7n, [2n])).rejects.toThrow( + /exact request exceeds byte limit/ + ); + expect(sent).toBe(beforeOversize); + maxRequestBytes = 10 * 1024 * 1024; const beforeMissing = sent; bridge.meta = { valueCapabilities: [] }; @@ -321,12 +360,13 @@ combineExact(1n, [2], true, 1.5);`, }, async callWithValuePolicy( _module: string, - _functionName: string, + functionName: string, _args: unknown[], _kwargs?: Record, validate?: (result: T) => void ): Promise { - const wrong = 9 as unknown as T; + const wrong = + functionName === 'echo_exact' ? ([[7n], [2, 3n]] as unknown as T) : (9 as T); validate?.(wrong); return wrong; }, @@ -336,6 +376,7 @@ combineExact(1n, [2], true, 1.5);`, await expect(generated.combineExact(7n, [2n], true, 1.5)).rejects.toThrow( BridgeValidationError ); + await expect(generated.echoExact(7n, [2n])).rejects.toThrow(BridgeValidationError); const ordinary = compileInteger(analyzedIr(), false); const ordinaryPath = join(temporary, 'value_contract_integer_default.generated.mjs'); @@ -371,6 +412,28 @@ combineExact(1n, [2], true, 1.5);`, await expect(defaultGenerated.combineExact(7, [2], true, 1.5)).resolves.toBe(9); expect(ordinaryCalls).toBe(1); expect(exactCalls).toBe(0); + + let serverLimitError = ''; + try { + execFileSync( + PYTHON ?? 'python3', + [ + pythonAdapter, + JSON.stringify(descriptors.echo_exact.args), + JSON.stringify(bridge.meta), + 'echo_exact', + ], + { + input: Buffer.alloc(10 * 1024 * 1024 + 1, 0x20), + env: pythonEnvironment, + encoding: 'utf8', + maxBuffer: 1024 * 1024, + } + ); + } catch (error) { + serverLimitError = String((error as { stderr?: string }).stderr ?? error); + } + expect(serverLimitError).toContain('input payload exceeds byte limit'); } finally { clearRuntimeBridge(); await rm(temporary, { recursive: true, force: true }); From 42ff1e1ff4fe74d3ef8ea64846eb07aa34bf52bb Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 03:18:49 -0700 Subject: [PATCH 34/35] test: reject malformed nested bigint envelope in bound call --- ...value_extensions_generated_integer.test.ts | 45 ++++++++++--------- 1 file changed, 23 insertions(+), 22 deletions(-) diff --git a/test/value_extensions_generated_integer.test.ts b/test/value_extensions_generated_integer.test.ts index b9a0a7bf..3a54053f 100644 --- a/test/value_extensions_generated_integer.test.ts +++ b/test/value_extensions_generated_integer.test.ts @@ -1,4 +1,4 @@ -import { execFileSync } from 'node:child_process'; +import { execFileSync, spawnSync } from 'node:child_process'; import { existsSync } from 'node:fs'; import { mkdtemp, rm, writeFile } from 'node:fs/promises'; import { delimiter, join } from 'node:path'; @@ -352,6 +352,11 @@ echoExact(1n, [2]);`, await expect(generated.combineExact(7n, [2n], true, 1.5)).rejects.toThrow( /expected integer envelope/ ); + responseOverride = [ + [{ __tywrap__: 'integer', codecVersion: 2, encoding: 'decimal', value: '7' }], + [{ __tywrap__: 'integer', codecVersion: 1, encoding: 'decimal', value: '2' }], + ]; + await expect(generated.echoExact(7n, [2n])).rejects.toThrow(/invalid integer envelope/); const bypassBridge = { meta: { valueCapabilities: ['exactIntegerDecimalV2'] }, @@ -413,27 +418,23 @@ echoExact(1n, [2]);`, expect(ordinaryCalls).toBe(1); expect(exactCalls).toBe(0); - let serverLimitError = ''; - try { - execFileSync( - PYTHON ?? 'python3', - [ - pythonAdapter, - JSON.stringify(descriptors.echo_exact.args), - JSON.stringify(bridge.meta), - 'echo_exact', - ], - { - input: Buffer.alloc(10 * 1024 * 1024 + 1, 0x20), - env: pythonEnvironment, - encoding: 'utf8', - maxBuffer: 1024 * 1024, - } - ); - } catch (error) { - serverLimitError = String((error as { stderr?: string }).stderr ?? error); - } - expect(serverLimitError).toContain('input payload exceeds byte limit'); + const serverGuard = spawnSync( + PYTHON ?? 'python3', + [ + pythonAdapter, + JSON.stringify(descriptors.echo_exact.args), + JSON.stringify(bridge.meta), + 'echo_exact', + ], + { + input: Buffer.alloc(10 * 1024 * 1024 + 1, 0x20), + env: pythonEnvironment, + encoding: 'utf8', + maxBuffer: 1024 * 1024, + } + ); + expect(serverGuard.status).not.toBe(0); + expect(serverGuard.stderr).toContain('input payload exceeds byte limit'); } finally { clearRuntimeBridge(); await rm(temporary, { recursive: true, force: true }); From 577800bcbc8d12a88bca6159d949c37855143589 Mon Sep 17 00:00:00 2001 From: Brett Bonner Date: Sun, 20 Sep 2026 03:28:49 -0700 Subject: [PATCH 35/35] docs: describe both analyzed exact integer callables --- docs/maintainers/value-contract-prototypes.md | 9 +++++---- 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/docs/maintainers/value-contract-prototypes.md b/docs/maintainers/value-contract-prototypes.md index 9ff53d2a..49ba6be9 100644 --- a/docs/maintainers/value-contract-prototypes.md +++ b/docs/maintainers/value-contract-prototypes.md @@ -12,10 +12,11 @@ integers use the same type. Python `bool` remains `boolean`. Calls without the option keep the safe `number` rule. The default value-contract revision 2 represents Python `int` as a safe JSON -`number`. The bounded revision 3 compiler proof selects -`EXACT_INTEGER_VALUE_CONVERSION` for a module with one callable. It emits a -`bigint` wrapper and validator through a test-only runtime binding. A public -per-callable option and production bridge adapter are not implemented. +`number`. The bounded revision 3 proof selects +`EXACT_INTEGER_VALUE_CONVERSION` for an analyzed Python module. It emits +`bigint` wrappers and validators for scalar and nested results through a +test-only runtime binding. A public per-callable option and production bridge +adapter are not implemented. The proposed generated wrapper must require `exactIntegerDecimalV2` in the bridge's `meta.valueCapabilities`. It must send