Skip to content

fix(reports): stop old and already-reviewed reports from hiding an entry - #201

Merged
jamesconwaydev merged 2 commits into
devstreamhq:mainfrom
foothillsdigital:fix/report-unhide-expiry
Nov 6, 2026
Merged

jamesconwaydev merged 2 commits into
devstreamhq:mainfrom
foothillsdigital:fix/report-unhide-expiry

Conversation

@foothillsdigital

@foothillsdigital foothillsdigital commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

What and why

Closes #41.

Un-hides are recorded. on_merge already runs hourly with a stats-branch token, so it now also keeps hidden.json there. Each hidden entry gets a record with seen_at, the last time a run saw it hidden. When an entry it saw hidden is visible again, the record gets an unhidden_at stamp. Entries that leave the list are dropped. Nothing in entries.json or its schema changes, so un-hiding is still the same hand edit. A failed hidden.json write is logged, and the Discussions sync still runs.

Only newer reports count. _reporters skips spam/other reports filed before a cutoff. The cutoff is the later of 30 days ago and the entry's unhidden_at. If hidden.json still says hidden but the entry is visible, and seen_at is under 2 hours old, the entry was just un-hidden, so the cutoff is now. Past 2 hours, on_merge is failing, so it falls back to the normal window rather than blocking hides. Timestamps are parsed strictly (ASCII YYYY-MM-DDTHH:MM:SSZ, then strptime with UTC attached). A malformed or naive one raises, and so does an unreadable hidden.json, so nothing gets hidden on bad data.

Known gaps, left on purpose:

  • No hidden.json yet means the plain 30-day window, which is today's behaviour.
  • A hide and un-hide inside the same hour never shows up in the snapshot.
  • Records are keyed by name, so a hidden entry renamed by rot loses its record. Both shortcuts are noted in the code.

Other changes:

  • The decide token in reports.yml gets contents: read to fetch hidden.json.
  • GOVERNANCE.md mentions the 30 days and the un-hide rule.
  • The 30 days is REPORT_MAX_AGE, and the 2 hours is JUST_UNHIDDEN.

Tests: 7 new report tests. 5 fail on main; the other two (a stale snapshot, a record for a different entry) pin behaviour main already has. Plus track_hidden tests and one where the hidden.json PUT fails and discussions.json is still written. Rebased onto main with #199.

Checks

  • The PR title follows Conventional Commits
  • python3 -m unittest discover -s scripts -t . passes (581)
  • npm --prefix site test passes (if you touched site/)
  • npx prettier --check . passes
  • This doesn't edit data/entries.json by hand (submissions go through the form)

@jamesconwaydev jamesconwaydev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The idea's right, and I like that un-hiding stays a plain edit. A few things before it goes in:

  1. track_hidden can stop the hourly sync. It runs before run() with nothing around it, so one failed hidden.json write (a 409 after three tries, or a 5xx) raises out of main() and discussions.json never syncs. #199 just went in to stop exactly that. Please wrap it so a failure is logged and run() still goes ahead, and add a test where the hidden.json PUT fails and discussions.json is still written.

  2. Dates need strict parsing. fromisoformat takes date-only and naive strings, and a naive one throws TypeError on the compare. We've settled on an ASCII re.fullmatch for YYYY-MM-DDTHH:MM:SSZ and then strptime(..., "%Y-%m-%dT%H:%M:%SZ") with UTC attached. (_established and _recent_checks have the same loose pattern; leave those, I'll get them separately.)

  3. Put a time limit on "hidden:true but visible means cutoff now". If on_merge keeps failing, that entry can't be hidden by reports at all, for as long as the failures last. Only treat it as "just un-hidden" when the record was stamped in the last 2 hours; past that, fall back to the 30-day window.

  4. Tests: unreadable hidden.json should raise and hide nothing, and a malformed or naive unhidden_at should raise. Also, the body says 3 new report tests fail on main, but 4 were added, and unhide_of_another_entry passes on main. Just tidy the wording.

  5. Note two gaps in the PR body and leave them: a missing hidden.json falls back to the plain 30-day window (fine, that's today's behaviour), and a hidden entry renamed by rot loses its record. A one-line comment next to the shortcut covers the rename case.

Then rebase onto main to pick up #199. It applies cleanly.

on_merge now keeps hidden.json on the stats branch: the entries that
are hidden, and when one was last un-hid. A spam or other report only
counts toward a hide if it was filed after that un-hide and within the
last 30 days. An entry that was hidden at the last hourly look but is
visible now counts no reports yet, so the window before on_merge sees
the un-hide is covered too.

The decide token gets contents: read to fetch hidden.json.
@foothillsdigital
foothillsdigital force-pushed the fix/report-unhide-expiry branch from 84fdbae to 39ae24e Compare November 6, 2026 15:30
@foothillsdigital

Copy link
Copy Markdown
Collaborator Author

Thanks, all five are in 39ae24e, rebased onto main:

  1. main() wraps track_hidden: a failure is logged, then run() goes ahead. New test: the hidden.json PUT fails and discussions.json is still written.
  2. _parse in reports.py is now the strict ASCII fullmatch + strptime with UTC. I left _established and _recent_checks alone as you said.
  3. Hidden records carry seen_at from each hourly run. "Visible but hidden at the last look" only counts as just un-hidden while seen_at is under 2 hours old (JUST_UNHIDDEN); after that it falls back to the 30-day window.
  4. Tests added: an unreadable hidden.json raises and hides nothing, and a malformed, naive or offset unhidden_at raises. The body now says 7 tests, 5 failing on main.
  5. Both gaps are in the body, and the rename one has a shortcut comment in track_hidden.

@jamesconwaydev jamesconwaydev left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's all five, thanks. Putting the 2-hour bound on seen_at is a neat way to do it, and the two shortcuts are documented right where someone will look.

@jamesconwaydev
jamesconwaydev merged commit 92d3c53 into devstreamhq:main Nov 6, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Remember un-hides and let old reports expire

2 participants