Repository navigation
Release completed CloudFetch body-reader listeners - #634
Open
vuanhphung wants to merge 5 commits into
Open
vuanhphung wants to merge 5 commits into
vuanhphung wants to merge 5 commits into
Conversation
vuanhphung
temporarily deployed
to
azure-prod
November 19, 2026 15:51 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 19, 2026 15:51 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 19, 2026 15:51 — with
GitHub Actions
Inactive
vuanhphung
force-pushed
the
perf/result-memory
branch
from
November 19, 2026 17:14
36ee05a to
7b3e0d2
Compare
vuanhphung
temporarily deployed
to
azure-prod
November 19, 2026 17:14 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 19, 2026 17:14 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 19, 2026 17:14 — with
GitHub Actions
Inactive
vuanhphung
marked this pull request as ready for review
November 19, 2026 18:59
There was a problem hiding this comment.
🔀 Cross-reviewed: two independent reviews (Claude Opus 5.5 · GPT-6.1 Sol), checked by a judge model (Claude Sonnet 5.5).
Verdict: 1 Nit
Looks good, no blocking concerns. node-fetch v2 installs the data/end/error reader listeners synchronously inside buffer(), so the before/after listener diff removes exactly those, and the constructor's own error handler is kept for late stream errors. The tests cover the success, rejection and production HTTP paths.
Other findings
- ⚪ Nit — The new Unreleased CHANGELOG entry is a bare paragraph, but the entries after it are
-bullets. Prefix it with-so the list stays consistent. (Found by Claude Opus 5.5 · confirmed by the judge)
Collaborator
Author
|
Fixed the changelog-list nit in e44f0af: the new Unreleased entry now uses the existing bullet style. |
Detach only the listeners installed by node-fetch after each body read settles, preserving other listeners and late-error handling. Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
vuanhphung
force-pushed
the
perf/result-memory
branch
from
November 20, 2026 04:10
e44f0af to
cfbf0b2
Compare
vuanhphung
temporarily deployed
to
azure-prod
November 20, 2026 04:10 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 20, 2026 04:10 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 20, 2026 04:10 — with
GitHub Actions
Inactive
Resolve the main changes by merging (no force-push). Signed-off-by: Vu Anh Phung <vu.phung@databricks.com> Co-authored-by: Isaac <no-reply@databricks.com>
vuanhphung
temporarily deployed
to
azure-prod
November 20, 2026 04:30 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 20, 2026 04:30 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 20, 2026 04:30 — with
GitHub Actions
Inactive
Explain the Node core retention the listener cleanup works around: on a pooled TLS socket, https.Agent#createConnection listeners pin the first ClientRequest and, through node-fetch, the body readers and settled body. Add an HTTPS keep-alive test on the production download path that asserts the downloaded buffer is collectable (WeakRef + gc); it fails without the cleanup, while plain HTTP retains nothing either way. Share the per-run localhost certificate fixture with the HttpConnection proxy tests. Co-authored-by: Isaac <no-reply@databricks.com> Signed-off-by: Vu Anh Phung <vu.phung@databricks.com>
vuanhphung
temporarily deployed
to
azure-prod
November 20, 2026 07:55 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 20, 2026 07:55 — with
GitHub Actions
Inactive
vuanhphung
temporarily deployed
to
azure-prod
November 20, 2026 07:55 — with
GitHub Actions
Inactive
This branch was previously deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Completed Thrift CloudFetch bodies stay reachable from idle pooled TLS sockets. The anchor is in Node core:
https.Agent#createConnection's'session'/'close'socket listeners share a closure context withcb(captured by the proxy-tunnelcleanupAndPropagate), which pins each pooled TLS socket's firstClientRequest, then node-fetch's request'error'closure, theResponse, and the body-reader closures holding the settled body. Plain HTTP retains nothing, and later requests on the same socket aren't pinned. A live post-drain heap snapshot attributes 154.7 MB of large backing stores to that path after a 216 MB read. The workaround can go once Node no longer capturescbin those listeners.Remove only the listeners installed by each body read once it settles, preserving earlier/later listeners and late-error handling. Rebased onto
mainafter #627 merged. Concurrency, retries, telemetry, values and defaults are unchanged; kernel downloading is native and does not use node-fetch.Verification: 1809 unit tests (31 CloudFetch), lint/format/types/build, live Thrift 9/9 and kernel 7/7, and value/type/null/limit/close/cancel checksums pass. Late-error coverage runs independently of cwd. An HTTPS keep-alive test on the production download path (real node-fetch,
HttpRetryPolicy, keep-alivehttps.Agent, per-run localhost certificate shared with theHttpConnectionproxy tests) asserts the downloaded 1 MiB buffer is garbage-collectable (WeakRef+gc) while its socket stays pooled; it fails with the listener cleanup removed, while the same test over plain HTTP passes with or without it. Independent review reruns against4831873, fresh alternating processes on Node 24.22.1; medians:Independent repeats find no throughput change beyond noise: live concurrent elapsed p≈0.29, with client drain time also unchanged (p=0.92). The ~14 GB caps cluster in low-bandwidth periods and hit both revisions; no progressive code-related slowdown reproduces. This is a peak-RSS improvement, not a universal speedup. V8-256 materialization OOMs and largest-materialization caps remain; historical worker results and main-to-stack context are retained in the handoff.
Adversarial review: round 1 converged with no open high finding; memory gains reproduced, throughput is within noise, and the path/error-listener coverage nits are fixed.
This pull request and its description were written by Isaac.