Repository navigation
Conversation
…stepped fake clock Every server spec that writes a test/harness/__artifacts__ fixture now renders through one recorder (test/server/artifact-recorder.ts) that drives the render on a fake clock stepped 1 ms at a time — the fix #3849 applied to generic-hydration.gen.spec.tsx alone, for every artifact. Why: the fixtures' flush points are timers and the stream coalesces one event-loop turn into one chunk (`deferFlush`), so on real timers the chunking and the script order are the host's. Reproduced on untouched sources (full server suite, 3 runs, plus 20 targeted runs under 12 busy loops on 10 cores): welcome-status-{loaded,streamed}.json re-recorded on every run (the 5 ms usage/slot events vs the 15 ms `stats` promise, in either order), write-before-resume.json split its one 60 ms chunk into two on 3/23 runs (two timers created microseconds apart straddling a millisecond), and nav-before-resume.json once wrote the 60 ms `<Side>` fragment ahead of the route's 20 ms one. On the stepped clock every fixture timer fires at its own instant in creation order; timers due at the same instant flush together, distinct instants flush separately. With the fixture built inside the recorded render (welcome's `makeWelcome()` started its 15 ms timer at construction, on the real clock), the recording is a function of the fixture: 10/10 full-suite runs under load, every artifact byte-identical. Re-recorded: welcome-status-loaded.json and welcome-status-streamed.json (shell unchanged; `rest` now carries the 5 ms usage trace events and the slot chunk before the 15 ms stats value + fragment, the fixture's order — the committed form was a loaded-host recording). The other ~150 artifacts are byte-identical to what the recorder produces. Guard: `writeArtifact` still re-records locally, but under CI (the checkout IS the committed artifact) a recording that differs from the committed file — or an artifact with no committed file — fails the test with the first difference, instead of silently re-recording. `JSX_COMPILER=babel` runs are an A/B and only write. Tests / tooling only; no runtime change, no changeset. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Size (brotli, eager entry chunk)
|
Coverage Report for CI Build 37747106541Coverage remained the same at 76.43%Details
Uncovered ChangesNo uncovered changes found. Coverage RegressionsNo coverage regressions found. Coverage Stats
💛 - Coveralls |
Merging this PR will degrade performance by 13.51%
|
| Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|
| ❌ | memo + sync render effect only (reference) |
28 ms | 32.4 ms | -13.51% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing test/deterministic-artifact-recorder (8b2ed00) with next (8d23a5a)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every server spec that writes a
test/harness/__artifacts__fixture now renders through one recorder,packages/web/test/server/artifact-recorder.ts, which drives the render on a fake clock stepped 1 ms at a time — the fix #3849 applied togeneric-hydration.gen.spec.tsxalone, generalised to every artifact. Tests / tooling only: nopackages/*/srcchange, no changeset.The nondeterminism, as reproduced
The fixtures' flush points are timers (
sleep(5),sleep(15),sleep(60)…) and the stream coalesces whatever settles in one event-loop turn into one chunk (deferFlushinsrc/server.ts), so on real timers the chunking and the script order are the host's. Verified this week's symptom on untouchednext(8d23a5a13): the full server suite 3×, plus the three recorder specs 20× under 12 busy loops on a 10-core host and a concurrent hydrate-suite run.welcome-status-streamed.json— re-recorded on every run (23/23). The committedrestopened with the 15 msstatsvalue:while this host wrote the 5 ms usage-trace events and the slot chunk first:
(Same
shell;$Rindices renumber with the order.) Both orders are innext's history — #3849 committed the second, #3860 the first.welcome-status-loaded.json, the same fixture, is committed in the first order and happened to reproduce it here: the first render in the worker is cold enough that the 5 ms chain trails the 15 ms promise.write-before-resume.json— the two 60 ms boundaries (<Side>,<SyncSide>) are one chunk as committed; on 3/23 runs they were two:Two timers created microseconds apart straddle a millisecond (
Timeout._idleStartis a freshuv_now), fire in two turns, anddeferFlushgets two bursts.nav-before-resume.json— a third one, caught by the full-suite runs (1/3): the route's 20 ms fragment is committed as chunk 0 and the 60 ms<Side>as chunk 1; a stalled worker wrote them the other way round (the 60 ms timer was created ≥ 40 ms before the 20 ms one got its turn).generic-hydration-{ab,ba}.json(already on #3849's fake clock) was stable 23/23.The fix
recordStream(code, options)renders throughrenderToStreamwithsetTimeout/clearTimeout/setImmediate/clearImmediatefaked (vi.useFakeTimers, the sametoFakelist as #3849) and steps the clock 1 ms at a time withvi.advanceTimersByTimeAsync(1)until the stream ends; it returns{ shell, chunks, rest }.deferFlushcaptured the realsetImmediateat module load and keeps it, so each step yields to the real loop and a burst's microtask chain and then its flush run before the next timer fires. Two properties fall out, and both are now pinned by the specs' own assertions:tickAsyncqueues its own continuation (a realsetImmediate) before the burst'sdeferFlush, so the next same-instant timer fires before the flush, exactly as a real loop runs same-instant timers before its check phase (write-before-resume's two 60 ms boundaries → one chunk,<Side>first).Every artifact spec (
hydration-harness,welcome-status-parity,write-before-resume.gen,nav-before-resume.gen,live-shell-source-3764.gen,generic-hydration.gen,document-shell,document-live-channel-artifact,frame-live-document-artifact,frame-nonlive-document-3666,dynamic-component-parity,dynamic-async-loading-3666,hybrid-memo-handoff,hybrid-store-handoff-3574) now renders through it; their copy-pastedcollectChunksare gone (−521 lines).slot-hydration-3567(syncrenderToString) only switches its write to the guard. The recorder's one rule for fixtures — build them inside the recordedcode— surfaced on the first full-suite run:makeWelcome()starts its 15 msstatstimer at construction, which the spec did before the recording, on the real clock; the welcome spec now constructs it inside the render.Artifacts re-recorded (2), and why
welcome-status-loaded.json,welcome-status-streamed.json—shellbyte-identical;restmoves from the loaded-host order (15 ms stats first) to the fixture's order (5 ms usage events + slot chunk, then the 15 ms stats value and fragment), as shown above. Script-order only, no markup change; both hydrate specs replayrestwhole, andwelcome-status-{loaded,streamed,lazy}.spec.tsxare green against the new form.Every other artifact (~150) is byte-identical to what the recorder produces — including
write-before-resume.json(one chunk) andnav-before-resume.json(route chunk first), whose committed forms were the natural ones.The guard
writeArtifact(name, data)is the one write site. Locally it re-records, as before (the artifact is committed so the diff shows what the compiler + runtime changed). Under CI (process.env.CI) the checkout is the committed artifact and nothing changed between it and the run, so a recording that differs from the committed file — or an artifact with no committed file at all — fails the test with the first differing bytes, instead of silently re-recording. That is the loud failure for both "committed a change without re-recording" and "the recorder is nondeterministic for this fixture".JSX_COMPILER=babelruns are an A/B against the native compiler's artifacts and only write (checked: babel and native record byte-identical welcome artifacts). Exercised locally withCI=1: match → pass; a byte flipped in a committed artifact → fails naming the offset; artifact deleted → fails "not committed"; noCI→ writes silently.Stability
Full server suite (161 files) 10/10 under load — 12 busy loops on 10 cores plus a concurrent hydrate-suite run, load average up to 19 — every run green and the hash of all 152 artifacts identical to the baseline each time.
Suites
vitest run --config vite.config.server.mjs: 161 files, 1524 passed, 3 expected fail, 2 skipped (and 10/10 under load).vitest run --config vite.config.hydrate.mjs(test/hydration + test/consistency, the artifact readers): 93 files, 463 passed, 16 expected fail (thetest.failspins), 2 skipped — includingwelcome-status-{loaded,streamed,lazy},write-before-resume,nav-before-resume,parity-harness,document-shell, andtest/consistency/generic/{replay,preload-hold,campaign}.vitest run(client): 136 files, 1272 passed, 1 expected fail.tsc --project tsconfig.test.json: clean.Public API changes
None.