Repository navigation
Hardening: durable dedup + stuck-incident recovery, signed alerts, drop dead fields - #13
Merged
Merged
Conversation
- Open-incident dedup moves from an in-memory dict to a partial unique index on incidents.signature (status triggered/triaging), so it holds across restarts and workers. Rows from before the column existed stay NULL and never dedup, so no backfill is needed. - A triage that throws now ends `degraded` with the reason recorded. It used to revert to `triggered` and stay open with nothing in it. - At startup, open incidents with no diagnostics and no update for SENTINEL_STUCK_AFTER_S (default 300s) are re-triaged in a background thread. - Drops the unused app_logs table from the schema. Existing tables are left alone. - CI runs a Postgres service so the new SQL is tested against a real database (test_db_integration, opt-in via SENTINEL_DB_TESTS=1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
With SENTINEL_ALERT_SECRET set, /alert requires X-Sentinel-Signature: sha256=<HMAC-SHA256 of the raw body>, compared in constant time; anything else gets a 401. Unset, unsigned alerts are still accepted, with a startup warning, so the local demo works without setup. chaos_cli and the load generator sign through sandbox/alert_signing.py. /alert now reads the raw body (the signature covers exact bytes) and rejects non-object JSON with a 400. The handler runs in the threadpool so its DB call doesn't block the event loop. CORS origins come from SENTINEL_CORS_ORIGINS. An empty value falls back to http://localhost:3000 rather than allowing no origins. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Nothing upstream reports user ids, so the field could never be filled honestly. It is removed from the orchestrator, the contract schema, the dashboard type and the CLAUDE.md contract example. Old rows that still carry it validate, because the schema allows extra properties. CLAUDE.md's pipeline step 2 now says Sentinel reads the alert's stack trace; the Postgres log table it described never existed in practice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Commits
incidents.signature(status IN ('triggered','triaging')), so it holds across restarts and workers.degradedwith the reason recorded. It used to revert totriggeredand stay open with nothing in it.app_logstable is dropped from the schema.test_db_integration.py, opt-in viaSENTINEL_DB_TESTS=1so local runs never touch Neon).SENTINEL_ALERT_SECRETset,/alertrequiresX-Sentinel-Signature: sha256=<hmac>(constant-time compare); without it, unsigned alerts are allowed with a startup warning. The sandbox senders sign. CORS origins come fromSENTINEL_CORS_ORIGINS.estimated_affected_users: it was always null, and no data source exists to fill it honestly.Tests
Notes
signaturecolumn and index. Existing rows stay NULL and never dedup, so no backfill is needed.🤖 Generated with Claude Code