Repository navigation
fix(codec): decode Arrow float16 values for NumPy and Torch #332
Description
Arrow float16 payloads currently return storage bits as numeric values.
Reproduced on 6b84614: fixtures.library_torture.numpy_float16() returns [15872, 49280] through NodeBridge, although Python returns [1.5, -2.25]. The Torch fixture returns the same wrong data.
See the decoder and existing NumPy and Torch rows.
Acceptance criteria:
- Decode half-float storage by dtype before reshape and numeric validation.
- NumPy and Torch return the original finite numeric values through Arrow.
- Cover signed zero, subnormals, normal values, finite extrema, scalars, and multidimensional arrays.
- Non-finite values follow the documented rejection policy after conversion.
- The shared ndarray path handles nested Torch payloads and container composition.
- Preserve JSON fallback, Arrow int64, and existing bfloat16 upcast behavior.
- Both affected menagerie rows become EXPECTED_OK.
- Real NodeBridge tests and relevant performance gates pass with pinned scientific dependencies.
Preserve the envelope format, shape, and provenance. Avoid a general codec rewrite.
Architectural approach:
Implement float16 semantics through the shared ndarray conversion in #343 so Torch reuses the same rule. Use the independent numeric cases in #344. Coordinate through #345.
Activity
bbopen commentedon Sep 20, 2026
Arrow float16 payloads now decode as numeric half-precision values before reshape. Coverage includes NumPy and actual analyzed Torch wrappers, signed zero, subnormals, finite extrema, multidimensional and nested results, and non-finite rejection. The independent oracle compares every finite binary16 word with Python
struct.unpack. JSON, int64, and bfloat16 controls remain separate.Evidence:
src/utils/codec.ts,test/architecture_numeric_oracle.test.ts,test/value_contracts.test.ts, andtest/menagerie/torch_callable.test.ts.Verified release evidence:
0861ebeand mergef6f811dhave identical trees.The architecture decision records the accepted scope and prototype limits.
Closing with the implemented and verified 0.11 release outcome.