hydration: gather hydratable nodes once per adoption, not per occurrence

.changeset/perf-gather-hydratable-once-per-adoption.md

@@ -0,0 +1,5 @@
+---
+"@solidjs/web": patch
+---
+
+Frames (document adoption): gather an adopted boundary's hydration keys once per adoption, not once per occurrence. Each adopted occurrence's claim window used to select `[_hk^="<prefix>"]` over the whole hydration root — 37 ms on a 652-occurrence comment thread, more than the rest of its hydration. The boundary now indexes the `_hk` nodes under its element once (bucketed by occurrence prefix) and serves every window — the occurrence's and a streamed `<Loading>`'s resume inside it — from that index; a fragment revealed into the element later extends the index for the revealed parent, so late (post-done) claims still find their nodes. The page-level gather and the streamed boundary's resume path are unchanged.

packages/web/frames/src/client.ts

@@ -407,16 +407,98 @@ export function getFrameHost() {
  * its occurrences' claims (which may run long after, under the frame's hold
  * or at a fragment's reveal) gather against the root that holds the frame
  * and not whichever `hydrate()` root replaced the live pair since (#2917).
+ * The gather is the boundary's own (`claimScope`): an index of the keys
+ * under its element, built once, not the root's scan of the whole page.
  */
 type ClaimScope = { registry?: Map<string, object>; gather?: (key: string) => void };
 
+/**
+ * The claim scope of an adopted boundary: the registry the page's root
+ * gathered into, read now, and a gather over the keys under `el` served from
+ * an index built ONCE here — the root's `gather` selects
+ * `[_hk^="<prefix>"]` over the whole hydration root, which an adoption with
+ * hundreds of occurrences (a comment thread: 652 on the HN story page) paid
+ * once PER occurrence's window, more than the rest of its hydration put
+ * together. One `[_hk]` pass over the element buckets every keyed node by
+ * the occurrence it belongs to; a window's gather is a lookup.
+ *
+ * The bucket of a key is its producer prefix, `sc-<fid>-<occurrence>-`: the
+ * fid and the occurrence may carry dashes, the child path after the prefix
+ * never does (`formatId` spells it in `[0-9a-zA-Z]`), so it ends at the
+ * key's LAST dash. A window's id names the same bucket by the same rule —
+ * an occurrence's prefix is its own bucket; a streamed `<Loading>` inside a
+ * fill resumes under a boundary id (the prefix plus a child path), whose
+ * bucket is the occurrence's, narrowed to the keys under the id. Ids are
+ * prefix-closed (a child's id extends its owner's), so `startsWith` is
+ * containment, as the selector's `^=` was. A bare `_hk` (an event-slot
+ * consumer's replay stamp) lands in the empty bucket no id ever names.
+ *
+ * `gather` hands a bucket's keys to the registry as the root's gather did:
+ * each node under the id that the registry does not hold — so a key the
+ * window already claimed (gone from the registry) IS put back, exactly as
+ * the selector put it back; `claimRender`'s `bound` is still what keeps a
+ * fill's second window from gathering — and only nodes still in the
+ * document, which is what a live selector over the root saw. The last
+ * matters: a `<Loading>` the server left pending inside a fill renders its
+ * fallback under the SAME id as its content, so the fallback's keys and the
+ * content's collide by design; the swap removes the fallback before the
+ * content's claim gathers, and a removed node must not shadow the one that
+ * replaced it.
+ *
+ * The index is a snapshot; `index(root)` extends it with what a fragment's
+ * reveal landed under `root` (the parent `$dfr` announces — the adoption's
+ * `fr.subscribe`, before the reveal's re-sync mounts anything). That is how
+ * keyed nodes enter an adopted element after adoption: a stream's re-call
+ * ships bare marker pairs (no keys; it renders fresh), an occluded region
+ * materializes from data (never claimed). (A reveal group's fallback
+ * materialization, `$dfl`, announces nothing; a fill's keyed fallback it
+ * lands between the adoption and a held window is not indexed — the
+ * content that replaces it is, at its reveal.) A reveal AFTER
+ * hydration-done (a server `<Loading>`'s content under corollary 4) finds
+ * the index where the registry was cleared, so the late window claims from
+ * it. Nodes indexed twice (the reveal's parent re-scanned) dedupe at the
+ * gather, by key.
+ *
+ * Without a registry (no `hydrate()` pass has run — a client render
+ * adopting server markup) there is nothing to index for and nothing to
+ * gather into: the index stays empty and a window gathers nothing. (A
+ * window opened under such a scope has no registry to claim against either
+ * way; `claimRender` renders fresh while no root has gathered.)
+ */
+function claimScope(el: Element): ClaimScope & { index(root: ParentNode): void } {
+  const registry: Map<string, object> | undefined = sharedConfig.registry;
+  const buckets = new Map<string, Element[]>();
+  const bucketOf = (key: string) => key.slice(0, key.lastIndexOf("-") + 1);
+  const index = (root: ParentNode) => {
+    if (registry)
+      for (const n of root.querySelectorAll("[_hk]")) {
+        const p = bucketOf(n.getAttribute("_hk")!);
+        const b = buckets.get(p);
+        b ? b.push(n) : buckets.set(p, [n]);
+      }
+  };
+  index(el);
+  return {
+    registry,
+    gather(id) {
+      for (const n of buckets.get(bucketOf(id)) || []) {
+        const k = n.getAttribute("_hk")!;
+        if (k.startsWith(id) && n.isConnected && !registry!.has(k)) registry!.set(k, n);
+      }
+    },
+    index
+  };
+}
+
 /**
  * Hydration re-entry for one adopted slot range: the fill renders inside a
  * claim window — `sharedConfig.hydrateWindow`, the same window a streamed
  * boundary's resume opens — under an owner whose id chain reproduces the
  * document producer's keys (`sc-<fid>-<occurrence>-`). The window gathers
- * the range's keys by that prefix, so the fill's components take the
- * server-rendered nodes by key; the range is declared as the window's claim
+ * the range's keys by that prefix — from the boundary's index of its
+ * element's keys (`scope`, see `claimScope`), not a scan of the page — so
+ * the fill's components take the server-rendered nodes by key; the range
+ * is declared as the window's claim
  * roots because it may be DETACHED right now (an async slot fill renders
  * before its boundary re-inserts it) and the runtime's hydration guards
  * read connectivity to tell claimed SSR nodes from fresh clones. A fill
@@ -1303,6 +1385,11 @@ function adoptBoundary(
     }
   };
   IS_DEV && reportRegionFragments(el);
+  // The root this boundary adopts under (see ClaimScope / claimScope): its
+  // occurrences claim against this registry however late they mount, from
+  // an index of this element's keys built now — one pass, not one per
+  // occurrence.
+  const scope = claimScope(el);
   const fr = (globalThis as any)._$HY?.fr;
   // The adopting frame, bound below; the reveal cascade syncs it.
   let frame: ReturnType<typeof createFrame> | undefined;
@@ -1312,6 +1399,10 @@ function adoptBoundary(
         // (nested server async). Scoped to the revealed parent, so each
         // sweep is proportional to what just landed.
         const inside = !!parent && el.contains(parent as Node);
+        // What the reveal landed joins the claim index first: the re-sync
+        // below mounts the occurrences it carried, and their windows gather
+        // from the index.
+        inside && scope.index(parent!);
         IS_DEV && inside && reportRegionFragments(parent!);
         // A revealed fragment also brings its occurrences' ARGS RECORDS: a
         // slot invoked inside a server `<Loading>` ships its `sc:slot:`
@@ -1368,16 +1459,14 @@ function adoptBoundary(
   // streamed morphs — bind consumer cleanup to this boundary's owner (see
   // boundaryScope for the ambient-preserving rule).
   const owner = getOwner();
-  // The root this boundary adopts under (see ClaimScope): its occurrences
-  // claim against this pair however late they mount.
   const sc: any = sharedConfig;
   // The frame's error, announced to the address source below (`landing`).
   const [failed, onApply] = failing();
   frame = createFrame(el, {
     adopt: true,
     host,
     id: address,
-    slots: slotsFor(props, { registry: sc.registry, gather: sc.gather }),
+    slots: slotsFor(props, scope),
     ownerScope: boundaryScope(owner),
     reveal: revealSeam(owner),
     onApply,

packages/web/src/client.ts

@@ -3035,13 +3035,17 @@ function cleanChildren(parent, current, marker, replacement) {
 }
 
 function gatherHydratable(element, root) {
-  // A prefix-scoped gather (a boundary's late resume; an adopted frame
-  // occurrence's claim window) names exactly what it owns — collect wherever
-  // the keys sit, frame interiors included: keys are namespaced by their
-  // producer chain, so a nested frame's content can never match a foreign
-  // prefix. Selected natively: it runs once per resume or per occurrence,
-  // and a full `_hk` sweep filtered in JS each time is a cost per
-  // occurrence on the whole page.
+  // A prefix-scoped gather (a streamed boundary's late resume) names exactly
+  // what it owns — collect wherever the keys sit, frame interiors included:
+  // keys are namespaced by their producer chain, so a nested frame's content
+  // can never match a foreign prefix. Selected natively: it runs once per
+  // resume, and a full `_hk` sweep filtered in JS each time is a cost per
+  // resume on the whole page. An adopted frame's occurrences do NOT gather
+  // here: a scan of the root per occurrence is a cost per occurrence on the
+  // whole page (37 ms on a 652-occurrence thread), so the frames client
+  // indexes its element's keys once at adoption and serves its windows from
+  // that (`claimScope` in frames/src/client.ts) — only a window a frame
+  // opens with no root gathered yet falls through to this gather.
   const templates = element.querySelectorAll(
     root ? `[_hk^="${root.replace(/["\\]/g, "\\$&")}"]` : `*[_hk]`
   );

packages/web/src/server.ts

@@ -4992,9 +4992,11 @@ function slotMarker(position, entries) {
  * tier completes the element at the bind and replays (bind-tier.ts). Bare
  * (no key): the bootstrap tests presence, and no claim ever looks this
  * element up — the ambient `_hk` sweep skips frame interiors (`data-fid`
- * containment, client.ts `gatherHydratable`) and a prefix-scoped gather
- * (`[_hk^="sc-…"]`, a boundary id) cannot match the empty value, so the dev
- * completion sweep never reports it. Four bytes per element, document face
+ * containment, client.ts `gatherHydratable`), a prefix-scoped gather
+ * (`[_hk^="sc-…"]`, a boundary id) cannot match the empty value, and the
+ * adopted frame's own key index (frames/src/client.ts `claimScope`) buckets
+ * it under the empty prefix no window's id names, so the dev completion
+ * sweep never reports it. Four bytes per element, document face
  * only: the stream face arrives after hydration, when the bootstrap no
  * longer queues (its `events` buffer is gone with `_$HY.done`). Server
  * components render under NoHydration, so the element has no keyed `_hk`