Repository navigation
prelude: vendor Ryu, make String.toFloat correctly rounded (fix WASM round-trip + TOML float parsing) - #1287
Merged
Merged
Conversation
jberdine
approved these changes
Jul 15, 2026
jberdine
left a comment
Contributor
There was a problem hiding this comment.
LGTM. Not sure if the CI job itself failed or there is something off.
Contributor
Author
|
I guess I just need to restart the CI (rebase needed anyways) |
mbouaziz
force-pushed
the
ryu-string-to-float
branch
2 times, most recently
from
November 13, 2026 10:42
9f691c5 to
bb93897
Compare
mbouaziz
enabled auto-merge
November 13, 2026 10:51
Vendored unmodified from https://github.com/ulfjack/ryu at commit 4c0618b0e44f7ef027ebae05d2cc7812048f7c8f (2026-02-09). Files copied from ryu/ directory: ryu.h, d2s.c, common.h, d2s_full_table.h, d2s_intrinsics.h, digit_table.h License files copied from repository root: LICENSE-Apache2, LICENSE-Boost Ryu converts IEEE 754 doubles to their shortest decimal string representation. It is self-contained C, dual-licensed under Apache License 2.0 and Boost Software License 1.0. No modifications applied — files are byte-identical to upstream. ## How to reproduce COMMIT=4c0618b0e44f7ef027ebae05d2cc7812048f7c8f DST=skiplang/prelude/runtime/ryu mkdir -p "$DST" for f in ryu.h d2s.c common.h d2s_full_table.h d2s_intrinsics.h digit_table.h; do curl -sL "https://raw.githubusercontent.com/ulfjack/ryu/$COMMIT/ryu/$f" -o "$DST/$f" done for f in LICENSE-Apache2 LICENSE-Boost; do curl -sL "https://raw.githubusercontent.com/ulfjack/ryu/$COMMIT/$f" -o "$DST/$f" done ## How to verify Compare vendored files against upstream checksums (sha256): common.h 0bbd71d26da6193e678d0776cf418f43f287c73d6fd6725353df0aadf70f2a19 d2s.c d24323c7eb77d63f1e52c415212b50060b776f0883525d907d04954fcd48cf64 d2s_full_table.h 2618f6e5fae6c4443899b184efe3d08295dd267dc9f1a994c983c7caca59ebe6 d2s_intrinsics.h 1d05702f2edacce428223d4b43dd3095c1bd1f3ad30128ce1761d84356dddc7d digit_table.h 8b782573abc0b8554d74163ae6c02f0beb5c30d1e63eaebdb0ddf2c98d817e01 ryu.h b7feab0ba1df5e9ef3d602f386592ad152491405e49b408fb21c0e0e9e6bfb16 LICENSE-Apache2 c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4 LICENSE-Boost c9bff75738922193e67fa726fa225535870d2aa1059f91452c411736284ad566 Run: sha256sum skiplang/prelude/runtime/ryu/* Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add Ryu-specific clang-format settings to minimize diff against upstream. Based on LLVM style with no column limit, cast spacing, and short-if-on-single-line to match Ryu's conventions. Add clang-format off/on guards around lookup tables to preserve upstream's aligned two-entries-per-line layout. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Guard libc headers behind #ifdef SKIP32 since WASM is compiled with -nostdlibinc (no libc available): - d2s.c: replace <assert.h>, <stdlib.h>, <string.h> with SKIP32 stubs (memcpy declaration + assert no-op); guard d2s() which uses malloc - common.h: replace <assert.h>, <string.h> with SKIP32 stubs - d2s_intrinsics.h: guard <assert.h> with #ifndef SKIP32 - ryu.h: guard <inttypes.h> with #ifndef SKIP32 Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add ryu/d2s.c to both the Makefile and build.sk so it's compiled for SKIP64 and SKIP32 targets, and list runtime/ryu/ in the prelude Makefile's RUNTIME_SRCS so that editing a Ryu file rebuilds the runtime archive. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Vendored from https://github.com/ulfjack/ryu at commit 4c0618b0e44f7ef027ebae05d2cc7812048f7c8f (the same commit as the d2s files), then reformatted with the vendored ryu/.clang-format. Files copied from the ryu/ directory: s2d.c, ryu_parse.h Ryu's s2d converts a decimal string to the nearest IEEE 754 double (correctly rounded). Upstream documents it as experimental: it supports up to 17 significant digits and not all input formats, returning a non-SUCCESS Status otherwise (callers fall back for those cases). s2d reuses the shared d2s tables/intrinsics already vendored; no new tables. Upstream sha256 (pre-format): s2d.c 9485e78f329ec1cf931b737333ef7bd85195864dfb9dd096413f0d336a7a46d3 ryu_parse.h da8ad9764b2701627cb75926b9d14b540fdfd7be693d671da9594eb16dabd27a Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Same treatment as d2s.c: guard libc headers behind #ifdef SKIP32 since WASM is compiled with -nostdlibinc (no libc): - replace <assert.h>, <stdlib.h>, <string.h> with SKIP32 stubs (memcpy declaration + assert no-op) - guard the s2d() convenience wrapper, which uses strlen; callers on SKIP32 use s2d_n() with an explicit length instead Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Compile ryu/s2d.c for both SKIP64 and SKIP32 (added to runtime/Makefile CFILES and prelude/build.sk srcs, alongside ryu/d2s.c). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
On SKIP32 (WASM) there is no libc strtod, so SKIP_String__toFloat_raw fell back to the in-tree atof, which reconstructs the value with repeated `* 0.1` and is not correctly rounded. This breaks round-trip: e.g. "0.3".toFloat() yielded 0.30000000000000004, and toString().toFloat() did not recover the original for most values. Use Ryu's s2d (correctly rounded) on SKIP32 for the inputs it supports (<=17 significant digits, standard formats), falling back to the existing atof for the rest (>17 digits, leading '+', etc.). SKIP64 keeps using libc atof, which is already correctly rounded. Verified: s2d round-trips all shortest-representation toString outputs (0 failures over 7053 sampled doubles, incl. subnormals and DBL_MAX), where the old atof failed ~85%. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
TOML's parseFloat reconstructed the value by hand with repeated `* 0.1`, which is not correctly rounded: e.g. "3.14" parsed to 3.1400000000000006 rather than the nearest double. The old 9-digit Float.toString masked this, but it breaks round-tripping and shows up once toString becomes exact (shortest-representation). Keep the existing validation walk (so malformed input is still rejected and the TOML "invalid" cases still fail), but delegate the actual value to String.toFloat (Ryu s2d on WASM, libc strtod on native), after stripping digit-group underscores and a leading '+'. All 331 toml tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
wasm32 defines __SIZEOF_INT128__, so Ryu's `HAS_UINT128` path used __uint128_t. Those 128-bit ops lower to compiler-rt calls (__multi3, __lshrti3) that are not available under -nostdlibinc, so the resulting wasm module failed to instantiate: LinkError: WebAssembly.instantiate(): Import "env" "__multi3": function import requires a callable Define Ryu's `RYU_ONLY_64_BIT_OPS` under `#ifdef SKIP32` in d2s_intrinsics.h to force the portable 64-bit path (used by both d2s and s2d). Verified: no 128-bit libcalls remain in either wasm object, and s2d still round-trips all 7053 sampled toString outputs. SKIP64 is unaffected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The runtime Makefile now builds ryu/d2s.c and ryu/s2d.c into ryu/*.o and ryu/*.bc, which the existing runtime/*.o and runtime/*.bc patterns do not cover (*.o is also ignored at the root, *.bc is not). Ignore both. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai> Co-Authored-By: Claude Opus 5.5 (Anthropic, via Mistral Vibe; thinking: high) <noreply@anthropic.com>
Build ryu/s2d.c the way the Wasm runtime does (SKIP32, -nostdlibinc) and check it bit for bit against JavaScript's correctly-rounded Number(): fixed edge cases (ties to even, subnormals, DBL_MAX, overflow, underflow), 100k random decimal strings and the shortest representations of 100k random doubles. Linking without --allow-undefined also catches 128-bit compiler-rt calls (__multi3, __lshrti3): with RYU_ONLY_64_BIT_OPS removed from d2s_intrinsics.h the link fails. Runs in the compiler job's "Run runtime regressions" step (~0.3s). Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai> Co-Authored-By: Claude Opus 5.5 (Anthropic, via Mistral Vibe; thinking: high) <noreply@anthropic.com>
The JSON fixtures compare Float.toString output, which has at most 9
significant digits, so they could not see the 1-ulp errors of the old
hand-rolled parser ("3.14" -> 3.1400000000000006). Compare the decoded
bits instead. Fails on the previous parseFloat, passes with String.toFloat.
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Co-Authored-By: Claude Opus 5.5 (Anthropic, via Mistral Vibe; thinking: high) <noreply@anthropic.com>
On SKIP32, String.toFloat inputs that Ryu's s2d rejects (more than 17 significant digits) fell back to an in-tree atof that: - ignored a leading '-': "-123456789012345678" parsed to 0 (before s2d, every negative number did, e.g. "-1.5" -> 0); - was a few ulps off: "3.14159265358979323846" -> 3.1415926535897967 instead of 3.141592653589793. Replace it with a new host import, SKIP_js_parse_float, implemented in sk_runtime.ts with Number(), which is correctly rounded for any input. s2d still handles the common case in Wasm (~20 ns vs ~100 ns per call into JS). Native builds keep libc atof. The new runtime regression test builds the real string.c for Wasm and checks both paths against Number(); it fails on the previous atof. Also checked end to end with the skdb npm package in Node: long FLOAT literals now insert as the nearest double. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai> Co-Authored-By: Claude Opus 5.5 (Anthropic, via Mistral Vibe; thinking: high) <noreply@anthropic.com>
Record the upstream commit, which files were taken, the SKIP32 patches on top, and how to update. Ryu is dual-licensed Apache 2.0 / Boost 1.0; state that we use it under the Boost license, which is MIT-compatible and only requires keeping the notices in source copies. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai> Co-Authored-By: Claude Opus 5.5 (Anthropic, via Mistral Vibe; thinking: high) <noreply@anthropic.com>
mbouaziz
force-pushed
the
ryu-string-to-float
branch
from
November 13, 2026 10:53
bb93897 to
d3a5cae
Compare
mbouaziz
added a commit
that referenced
this pull request
Nov 16, 2026
…tion) (#1288) ## What Replace the pure-Skip `Float.toString`, which printed at most 9 significant digits and so could not round-trip, with a C helper built on Ryu `d2s` (vendored in #1287). It produces the shortest decimal string that round-trips through the IEEE-754 double: `toString().toFloat() == value` for every finite value, e.g. `0.1` prints `"0.1"` and `0.1 + 0.2` prints `"0.30000000000000004"`. The layout is unchanged: fixed notation for 1e-4 <= value < 1e15, `.0` for integers, exponents `e+N`, `e-0N` and `e-NN`. Only the number of digits changes, e.g. `DBL_MAX` prints `1.7976931348623157e+308` instead of `1.797693135e+308`. **JS compatibility:** the digits are those of JS `Number.prototype.toString`. This fixes skdb's JS API, which passes `FLOAT` values to JS as `Float.toString()` text and `parseFloat`s them, so values lost everything past the 9th significant digit. ## How - `SKIP_Float_toStringShortest` (`string.c`) lays out Ryu's shortest digits in Skip's conventions; `floatToString` keeps handling `nan`, `inf` and the sign. - `skc` no longer constant-folds `Float.toString()`: folding uses the compiler's own `Float.toString`, i.e. that of the prelude the compiler was built with, so a constant could print differently from the same value at runtime. - Tests: `tests/Float.sk` expectations updated to full precision, and previously non-round-tripping values (`5e-324`, smallest normal, `DBL_MAX`, 2^53, 2^53 - 1) added to the round-trip test. The TOML golden for long pi now has all its digits. A new runtime regression (`runtime/tests/float_to_string.{c,mjs}`) builds the formatter for Wasm and checks it against JS for 200k values. - Versions: `std` 0.4.0 → 0.5.0 (visible output change), `skc` 0.4.0 → 0.4.1. ## Testing Float.toString with any compiler `skc` folds `toString()` on a Float literal at compile time with its *own* `Float.toString`, i.e. that of the prelude the compiler was built with. CI's candidate toolchain is built from the bootstrap IR, which still has the 9-digit formatter, so literal-based tests saw `5e-324` → `"4.940656458e-324"` whatever the runtime does. The `Float.toString` tests now parse their inputs with `String.toFloat` (never folded), so they test the runtime formatter with any compiler; with the previous formatter they fail as expected. Literal folding goes away for good once the bootstrap is bumped past this PR. `test_vtable_slots` compared function text including debug metadata numbers (`!dbg !N`), which are per LLVM module. With this PR's smaller prelude, the edited function lands in the same module as the checked one and renumbers it, while the vtable slot is unchanged. The test now strips `!dbg` attachments (with the existing `llvm_artifact_diff.without_dbg_attachments`) and still fails if the slot offset changes. ## Validation Rebased onto `main` `e46671a9` (the #1287 commits dropped; one conflict in `runtime/tests/README.md` with the new macOS section, both kept). The version bumps are on top of #1683, which already moved `std` and `skc` to 0.4.0. - Formatter fuzzed natively over 4M doubles (UBSan, stack protector): 0 round-trip failures, 0 layout differences from the old conventions, output at most 23 characters. The later simplification of the formatter (109 → 47 lines) produces byte-identical output on 18.6M values. - With a from-source stage1: `prelude` 131/131, `toml` 332/332, compiler `Runtime`/`Native`/`Syntax` tests 378/378 (all `Float*` tests), skdb native SQL suite against sqlite3 437/437, `runtime/tests/run.sh` pass. - CI's path (candidate stage0 from the bootstrap, fresh build cache): `prelude` 131/131; with the previous formatter, the Float tests fail. - Backend regressions, both shards, stage1: 30/30 modules (incl. `test_vtable_slots`, also passing on `main`). - `skdb` Wasm build: +27 KB (Ryu's `d2s` tables). ## AI assistance The original commits were written with Claude Code (Claude Opus 4.6). The rebase, review and the commits on top (rename, cleanup, tests, version bumps) were done with Mistral Vibe (Claude Opus 5.5, thinking: high); see the `Co-Authored-By` trailers.
mbouaziz
added a commit
to SkipLabs/skjs
that referenced
this pull request
Nov 18, 2026
Skip's String.toFloatOption is now correctly rounded and Float.toString prints the shortest round-trip digits (Ryu, SkipLabs/skip#1287/#1288), so normalizeJSNumber can name every literal by the String() form of its double directly: parse (radix digits converted to decimal first), print with jsNumberToString. This replaces the exact decimal pipeline (normalizeDecimalJS, parseRadixInt, normalizeUnsignedJSNumber) and the second beyond-precision pass (canonicalizeBeyondPrecision). Long zero-padded spellings that exceed the runtime parser's input limit are compacted first (compactDecimal), and a leading '.' gets a 0, so the pinned edge spellings keep their names. No behavior change: the full suite and a 2,000-literal differential run against node agree. Generated by Mistral Vibe. Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
mbouaziz
added a commit
to SkipLabs/skjs
that referenced
this pull request
Nov 18, 2026
…1073) Now that Skip parses floats correctly rounded and prints them with Ryu (SkipLabs/skip#1287, #1288), `normalizeJSNumber` names each literal directly: it parses the literal to a double, then prints that double with `jsNumberToString` (the JS `String()` form). This removes the exact decimal pipeline (`normalizeDecimalJS`, `parseRadixInt`, `normalizeUnsignedJSNumber`) and the second beyond-precision pass (`canonicalizeBeyondPrecision`). `src/JSNumericString.sk` goes from 276 changed lines to a net reduction of about 90 lines. Commits (tests first): 1. `test:` pins the spellings the runtime parser does not accept as written: `.5`, `5.`, zero padding past the 255-byte parser limit, and a literal that is still too long after the padding is dropped. 2. `refactor:` makes the switch. A leading `.` gets a `0` prepended, and long zero-padded spellings are compacted first (`compactDecimal`), so those pins keep their names. There is no behavior change, so there is no version bump. Both commits pass `make test` locally. A differential run on 2,000 random literals (decimal, leading/trailing dot, exponent, hex, binary, separators) matches node's `String(Number(x))` on every literal. — Mistral Vibe
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two pre-existing float-parsing bugs share one root cause — reconstructing a
double from its decimal digits with repeated
* 0.1, which is not correctlyrounded:
String.toFloat(SKIP_String__toFloat_raw's in-treeatofonSKIP32):
"0.3".toFloat()yielded0.30000000000000004, round-trip(
toString().toFloat() == value) failed for ~85% of sampled doubles, and theleading
-was ignored, so every negative number parsed to0(
"-1.5"→0).parseFloat:"3.14"parsed to3.1400000000000006instead of thenearest double.
Both were masked while
Float.toStringproduced only 9 significant digits; theysurface once toString becomes exact (#1288).
How
ryu/d2s.cdouble→string andryu/s2d.cstring→double) from ulfjack/ryu @
4c0618b0, with#ifdef SKIP32guards so itbuilds under
-nostdlibinc, andRYU_ONLY_64_BIT_OPSon SKIP32 so it needs no128-bit compiler-rt helpers (
__multi3,__lshrti3).ryu/README.mdrecordsthe origin, license and local changes. (
d2sis used by prelude: Float.toString via Ryu (round-trip-safe, shortest representation) #1288;s2dhere.)String.toFloatuses Ryus2d(correctly rounded, ~20 ns) for theinputs it supports (≤17 significant digits). For longer inputs it calls a new
host import,
SKIP_js_parse_float, implemented insk_runtime.tswithNumber()(correctly rounded for any input). The in-treeatofis removed.SKIP64 keeps libc
atof, already correct.parseFloatkeeps its validation but delegates the value toString.toFloatinstead of reconstructing it by hand.Wasm import: the module now imports
SKIP_js_parse_float. Every host(skdb,
@skipruntime/wasm) gets it fromsk_runtime.ts, which their buildscopy in next to the module, so they stay in sync. A host that loads a new
module with an old
sk_runtime.tswould fail to instantiate.Tests
runtime/tests/string_to_float.{c,mjs}:s2dbuilt with the Wasm runtime'sflags, checked bit for bit against
Number()(edge cases, 100k random decimalstrings, shortest representations of 100k random doubles). Links without
--allow-undefined, so it fails if__multi3& co. come back.runtime/tests/to_float_raw.{c,mjs}: the realstring.cbuilt for Wasm, withthe host import as in
sk_runtime.ts; checks thes2dpath and the hostpath against
Number(). Fails on the previousatof(
-123456789012345678→0).tomlfloat-bits: compares decoded bits, which the JSON fixtures cannot(they compare 9-digit
toStringoutput). Fails on the previousparseFloat.All three run in CI: the first two in the compiler job's runtime regressions
(<1s), the last in the
tomlpackage tests.Choice of Ryu, license
d2sgives the same shortest digits as JSNumber.prototype.toString(0 differences over 7M doubles, includingsubnormals and
DBL_MAX). Differences in layout (1.0vs1, exponentthresholds) belong to the prelude: Float.toString via Ryu (round-trip-safe, shortest representation) #1288 formatter.
is MIT-compatible and only requires keeping the notices in source copies.
Validation
Rebased onto
main(one conflict inprelude/build.sk:magic.cis nowWasm-only since the build-id change; kept that and added the Ryu sources and
.include("runtime")).main's incremental prelude build (#1637) tracksruntime/*.c/*.honly, soruntime/ryu/is added toRUNTIME_SRCS(checked: touching
ryu/s2d.crebuilds the archive).4c0618b0:d2sand licensesbyte-identical;
s2d.c/ryu_parse.hdiffer only by the clang-format pass.s2dvs libcstrtod: 0 mismatches over ~25M random inputs, with both thenative 128-bit path and the
RYU_ONLY_64_BIT_OPSpath used on Wasm (UBSantrap mode).
sk_runtime.ts): longFLOATliterals (-123456789012345678,3.14159265358979323846, …) insertas the nearest double.
skdbWasm module: the only new import isSKIP_js_parse_float; no__multi3/__lshrti3.@skip-wasm/stdbuilds and lints.runtime/tests/run.sh,prelude(131/131) andtoml(332/332) tests pass.make STAGE=1, the skdb Wasm Playwright suite (CI runs bothwith the candidate toolchain since the runtime changed).
After merge, the CI images need a
docker-publishrun (runtime change; seeHACKING.md).
AI assistance
The original commits were written with Claude Code (Claude Opus 4.6/4.8). The
rebase, review and the commits on top (tests,
.gitignore, the Wasm fallbackfix,
ryu/README.md) were done with Mistral Vibe (Claude Opus 5.5, thinking:high); see the
Co-Authored-Bytrailers.