Repository navigation
fix: keep block checks in budget and stop repo-name squats - #194
Merged
Merged
Conversation
sync_likes looked up /orgs/devstreamhq/blocks/{login} for every cached
trusted reactor, every hour. With a few thousand Likers that alone
spends the installation's hourly budget. The block list is now read
once per run and folded into the exclusions, which already read as
BLOCKED and skip the cache.
Entries were keyed by owner/name only. If an owner renamed or deleted their account, someone could register the old login, create a repo with the same name, and the health check would carry the listing and its Likes over to it. Every entry now stores GitHub's numeric repo_id, which never changes and is never reused. Submissions and seeds record it, validate_data and schema.json require it, and all 454 existing entries are backfilled. When the health check finds a different ID at the listed name, the entry gets a removal notice and a needs-review issue instead of being updated. A rename keeps its ID, so it still just follows the new name. Closes #36
The health check gave a replaced repo a removal notice, but the hourly Likes sync still read stars, avatar and language by name, so the stranger's repo showed on the card and could trend for a week. sync_likes now compares GitHub's databaseId to the entry's repo_id, keeps the last card on a mismatch and flags the entry. Also: blocked_logins read a 404 on the block listing as nobody blocked (paginate now takes must_exist), validate_data rejects a duplicate repo_id, rot warns on an entry with no repo_id, and the review issue text no longer claims more than the code does.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two fixes from the final review: one keeps the Likes sync inside the API budget, and one stops a name squat from taking over an entry.
Block checks were one API call per Liker
Every hour,
sync_likescalledGET /orgs/devstreamhq/blocks/{login}for each cached trusted reactor. With a few thousand Likers that alone uses up the installation's hourly budget, and then Likes freeze.Now the org's block list gets read once per run (
blocked_logins, which pages throughper_page=100) and folded into the exclusions. An excluded login already reads as BLOCKED and skips the cache, so a new block still takes effect on the very next run. Submit and reports use the same set. If the token can't read the list, the run still fails closed: a 404 on the listing raises (paginate(..., must_exist=True)) instead of reading as nobody blocked.A squatter's repo could inherit an entry (#36)
Entries were keyed only by
owner/name. Say an owner renames or deletes their account. Someone registers the old login and creates a repo with the same name, and the health check carries the listing, its Likes and the link over to the stranger's repo.repo_id, GitHub's numeric ID, which never changes and is never reused. Submissions and seeds record it from the repo response.validate_data.pyanddata/schema.jsonrequire it: a positive integer, never null. The validator also rejects two entries with the samerepo_id.gh api repos/{owner}/{name}over the file. Every entry resolved, aquasecurity/trivy included, and none of the names redirect.needs-reviewissue ("A different repo now answers to …"). The entry itself isn't updated. A real rename keeps its ID, so it still just follows the new name. An entry with norepo_idgets a warning in the log, and the run carries on.databaseIdalong with the card. If it doesn't matchrepo_id, the sync keeps the last card (the same way it handles a repo hidden by an IP allow list), logs a warning and flags the entry, so the stranger's stars and avatar never show up and the entry can't trend. Until the entry comes off the list it still links to that name, and the review issue says so.Known gap: the review issue is deduped against open issues only. If a maintainer closes it and the mismatch is still there, next week's run opens a new one.
Tests
python3.12 -m unittest discover -s scripts -t .: 546 pass. Block list: one listing, lowercased, 404 and 5xx still fail, and cache hits make no calls, and a 404 on the listing raises. Validator:repo_idis required, must be positive and can't repeat. Likes sync: a mismatched ID keeps the old card and flags the entry, for live and flagged entries alike. Health check: a mismatch gives a notice plus one review issue with no rename, and a matching ID follows the rename.npx prettier --checkpasses on both data files.Not in this PR: letting rot sleep through rate-limit resets or shard its run, and batching rot's reads through GraphQL (I5 b and c). Both change how rot is scheduled or how it fetches, so they need their own PR.