Add new rate limiter

[itsibitzi]

This implements the multibucket client side rate limiter for blackbird. This is required because we made it so that different RPCs use different quotas (you need to be able to upload very quickly when uploading documents, but we dont want you to be able to request hundreds of new checkpoints per second).

It also brings the sliding timing and N window up to where the reference implementation is. This mainly just clears out a kinda pointless background task and adds a slow start.

This is not my code base so I may have messed up some stuff stylistically, feel free to push commits on top or let me know if there's anything you'd like different.

I did a test of uploading some things and was able to get around 100 document per second.

[itsibitzi]

@microsoft-github-policy-service agree company="GitHub"

[Copilot]

Pull request overview

Implements multi-bucket client-side throttling for GitHub API requests by learning a “quota bucket” from response headers and maintaining per-bucket throttlers, while also simplifying the sliding-window implementation by removing the background cleanup task.

Changes:

• Add support for reading a new x-github-quota-bucket-name response header.
• Replace the single global throttler with a per-quota-bucket throttler map (endpoint → bucket → throttler).
• Remove interval-based cleanup from the sliding window and perform cleanup on-demand during throttle decisions; adjust ramp-up logic.

Comments suppressed due to low confidence (1)

src/platform/github/common/githubApiFetcherService.ts:295

• quotaUsed parsing uses parseFloat(quotaUsedHeader) || 0, which coerces any non-numeric header value to 0 and then records it into the throttler. This can skew throttling decisions (and interacts with the get() === 0 fast-path) if the header is missing/empty/invalid. Prefer validating the parsed value (e.g. Number.isFinite) and skipping the sample when it is not a valid number.

			// Record quota usage for throttle calibration, keyed by bucket. If the bucket name is not in the headers use a
			// fake __global__ bucket.
			const bucketName = res.headers.get(githubHeaders.quotaBucketName) || '__global__';
			const quotaUsedHeader = res.headers.get(githubHeaders.totalQuotaUsed);
			if (quotaUsedHeader !== null) {
				const quotaUsed = parseFloat(quotaUsedHeader) || 0;
				this.updateThrottlers(options.method, options.url, bucketName, quotaUsed);
			}

[Copilot]

shouldSendRequest treats totalQuotaUsedWindow.get() === 0 as "no requests". Since recordQuotaUsed can be fed 0 (e.g. when the header value is "0" or if parsing fails and falls back to 0), this condition can incorrectly bypass the PID delay logic and effectively disable throttling after ramp-up. Consider checking totalQuotaUsedWindow.size() === 0 to detect the "no requests" case, and/or ignoring non-finite/zero quota samples if 0 is not a meaningful signal from the server.

This issue also appears on line 288 of the same file.

Suggested change

	if (this.totalQuotaUsedWindow.get() === 0 || this.sendPeriodWindow.size() === 0) {
	if (this.totalQuotaUsedWindow.size() === 0 || this.sendPeriodWindow.size() === 0) {

[Copilot]

endpointBuckets keys endpoints by the full url string ("METHOD url"). If callers include query strings or other per-request variants in options.url, this can lead to unbounded growth of endpointBuckets and also prevent reusing the learned quota bucket for what is logically the same RPC. Consider keying by a stable identifier (e.g. options.telemetry.urlId) and/or normalizing the URL (strip query, canonicalize path parameters) before using it as a map key.

[mjbvz]

Awesome! Thanks for handling this. I fixed the merge conflicts and opened a new PR here: #4395