Skip to content

feat(webhooks): verifyAndParse* API for compressed payloads (CHA-3071)#391

Open
nijeesh-stream wants to merge 7 commits into
masterfrom
nijeeshjoshy/cha-3071-compress-webhook-payloads
Open

feat(webhooks): verifyAndParse* API for compressed payloads (CHA-3071)#391
nijeesh-stream wants to merge 7 commits into
masterfrom
nijeeshjoshy/cha-3071-compress-webhook-payloads

Conversation

@nijeesh-stream
Copy link
Copy Markdown
Contributor

@nijeesh-stream nijeesh-stream commented May 7, 2026

Summary

Adds first-class support for gzip-compressed webhook payloads (HTTP webhooks, SQS, SNS) and exposes a stable VerifyAndParse* API that mirrors the cross-SDK contract published in Webhooks Overview.

New public API

Primitives (in webhook.go, intended to be composable):

  • GunzipPayload(body) ([]byte, error) — gzip-magic-byte detection, no-op when not compressed
  • DecodeSqsPayload(body) ([]byte, error) — base64 decode then gunzip-if-magic
  • DecodeSnsPayload(notificationBody) ([]byte, error) — JSON-parse the SNS HTTP notification envelope, extract the inner Message, then run the SQS pipeline. Falls through to a pre-extracted Message string when the input is not a JSON envelope
  • VerifySignature(body, signature, secret) bool — HMAC-SHA256 over the uncompressed body, with a constant-time comparison (matters for the HTTP webhook path where the X-Signature header is exposed publicly; SQS / SNS deliveries arrive over AWS-internal transports where timing-attack resistance is not strictly required)
  • ParseEvent(payload) (*Event, error) — JSON → typed *Event

Composites (return a typed *Event):

  • VerifyAndParseWebhook(body, signature, secret) (*Event, error)
  • VerifyAndParseSqs(body, signature, secret) (*Event, error)
  • VerifyAndParseSns(body, signature, secret) (*Event, error)

Client-bound versions on *Client use the configured app secret automatically.

Backwards compatibility

Client.VerifyWebhook is preserved and delegates to VerifySignature. Existing callers continue to work unchanged for plain (uncompressed) bodies.

Tests

webhook_test.go covers plain / gzip / base64 / base64+gzip payloads, signature mismatches, malformed bytes, JSON parsing into *Event, and a backwards-compat path for Client.VerifyWebhook. Linked Linear ticket: CHA-3071.

Golden test fixtures (Tommaso)

Added shared reference fixtures to the test suite so future SDKs can sanity-check decoders against the same payloads:

aGVsbG93b3JsZA==                          -> helloworld   (base64)
H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA -> helloworld   (base64 + gzip)

Test plan

  • go vet ./... clean
  • go test -c . (test binary compiles clean)
  • Full go test ./... runs in CI (package init() requires STREAM_KEY / STREAM_SECRET)

…HA-3071)

Co-authored-by: Cursor <cursoragent@cursor.com>
Replaces VerifyAndDecodeWebhook / DecompressWebhookBody with the
cross-SDK contract documented at
https://getstream.io/chat/docs/node/webhooks_overview/.

Package-level helpers in stream_chat:

  Primitives:
    UngzipPayload      - gzip magic-byte detection + inflate
    DecodeSqsPayload   - base64 then ungzip-if-magic
    DecodeSnsPayload   - alias for DecodeSqsPayload
    VerifySignature    - constant-time HMAC-SHA256 comparison
    ParseEvent         - JSON -> *Event (typed, with ExtraData fallback
                         for unknown event types)

  Composite (return *Event):
    VerifyAndParseWebhook
    VerifyAndParseSqs
    VerifyAndParseSns

The composite functions auto-detect compression from body bytes, so
the same handler stays correct whether or not Stream is currently
compressing payloads, and behind middleware that auto-decompresses.

Client instance methods (Client.VerifyAndParse*) mirror the three
composite helpers and pull the API secret from the receiver.

The legacy Client.VerifyWebhook(body, signature) bool helper is
unchanged for backward compatibility.

ErrInvalidWebhookSignature is preserved as the sentinel error for
HMAC mismatch.

Co-authored-by: Cursor <cursoragent@cursor.com>
@nijeesh-stream nijeesh-stream changed the title feat(webhooks): add VerifyAndDecodeWebhook for compressed payloads feat(webhooks): verifyAndParse* API for compressed payloads (CHA-3071) May 8, 2026
nijeesh-stream and others added 3 commits May 8, 2026 16:53
RFC 1952 defines the gzip magic number as the two-byte sequence
1F 8B; the third byte (CM) is informational and not part of the
identifier. Trim the magic check from three bytes to two to match
the spec and stay consistent with the reference implementations
in the public docs.

Co-authored-by: Cursor <cursoragent@cursor.com>
The compression section referenced VerifyAndDecodeWebhook,
DecompressWebhookBody and a payloadEncoding argument that do not exist
in this SDK. Rewrite to document the actual public surface:
VerifyAndParseWebhook / VerifyAndParseSqs / VerifyAndParseSns plus the
lower-level UngzipPayload, VerifySignature and ParseEvent helpers.

CHA-3071

Co-authored-by: Cursor <cursoragent@cursor.com>
io.ReadAll followed by a deferred Close swallowed the trailing CRC and
length-trailer error that compress/gzip only reports when the reader
is closed. Replace the defer with explicit Close handling so a corrupt
or truncated gzip stream is reported as a gzip error instead of
falling through to a misleading invalid-signature failure.

CHA-3071

Co-authored-by: Cursor <cursoragent@cursor.com>
Copy link
Copy Markdown
Contributor

@mogita mogita left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cross-SDK review pass for CHA-3071. Two inline comments — see below.

Comment thread webhook.go Outdated
Comment thread webhook_test.go
…y verify

decode_sns_payload now JSON-parses the SNS HTTP notification envelope
({"Type":"Notification","Message":"..."}) and extracts the inner
Message field before running the SQS pipeline. Falls through to the
pre-extracted Message string when the input is not a JSON envelope so
existing call sites keep working.

Legacy Client.VerifyWebhook now uses hmac.Equal for constant-time
comparison instead of bytes.Equal, closing the same timing leak we
already closed in WebhookHelpers.VerifySignature.

Test fixture adds a realistic SNS HTTP notification body that all
SDKs in this rollout can share.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mogita
Copy link
Copy Markdown
Contributor

mogita commented May 11, 2026

Cross-SDK coordination: unifying webhook exception types

After the review pass across all 6 SDKs in this rollout and team discussion, we're consolidating the new webhook exception strategy to a single unified error type rather than the split (signature vs parse sentinels) being introduced in this PR.

The Webhook Handling Spec on Notion (CHA-2961) has been revised to reflect this — §5.2 / §5.3 / §7 now specify a single class.

Why unified: From a customer's perspective, all failure modes — signature mismatch, gzip decompression failure, base64 decode failure, SNS envelope failure, JSON parse failure, missing schema field — terminate at the same error-handling branch in customer code. A signature/parse split adds structural complexity without changing customer behavior. Customers who want to filter security logs for signature mismatches specifically can do so via error message text or errors.Unwrap cause-chain.

Sentinel name: ErrInvalidWebhook (+ struct type InvalidWebhookError) — "Invalid" covers all failure modes accurately and is consistent with Go stdlib naming (errors.ErrUnsupported, etc., where the prefix Err names the category).

Per-SDK naming across the rollout:

SDK Class name
JS InvalidWebhookError (extends Error)
Python InvalidWebhookError
Go sentinel ErrInvalidWebhook + struct InvalidWebhookError
Java InvalidWebhookException (extends existing StreamException)
PHP InvalidWebhookException (extends existing StreamException)
Ruby StreamChat::InvalidWebhookError (extends StandardError)
.NET StreamInvalidWebhookException (extends StreamBaseException)

Asks for this PR:

  1. Consolidate the existing sentinels (ErrInvalidSignature, ErrMalformedWebhook, etc.) into a single ErrInvalidWebhook sentinel, with concrete struct InvalidWebhookError carrying details
  2. Wrap all failure paths to return that single sentinel — signature mismatch, gzip failure, base64 failure, SNS envelope failure, JSON parse failure, missing type/schema failure — using fmt.Errorf("%w: <mode>: %v", ErrInvalidWebhook, cause) so errors.Is(err, ErrInvalidWebhook) matches and errors.Unwrap reaches the cause
  3. The wrapped message identifies which failure mode fired ("signature mismatch", "invalid base64", "missing type field") so customers can filter on substring
  4. Legacy Client.VerifyWebhook (returning bool) stays unchanged — back-compat preserved. Note: there's an existing review comment on AppClient.cs-equivalent in another SDK about the legacy method's bytes.Equal not being timing-safe; same applies to client.go:160, but that's separate from this exception-unification ask.
  5. Update unit tests to assert via errors.Is(err, stream.ErrInvalidWebhook); for mode-specific tests, also assert on error message substrings

This same comment is being posted on all 6 SDK PRs (JS / Go / Ruby / PHP / Java / .NET) for coordination. Happy to discuss naming or scope tradeoffs.

…n fixtures (CHA-3071)

Per Tommaso's suggestion, align the gzip helper with the GNU `gunzip`
command name. The function was added in this PR and not yet released,
so this is a straight rename with no back-compat alias.

Adds Tommaso's reference fixtures to the test suite as named cases so
future SDKs can sanity-check against the same payloads:

  aGVsbG93b3JsZA==                          -> helloworld   (base64)
  H4sIAGrYAWoAA8tIzcnJL88vykkBAK0g6/kKAAAA -> helloworld   (base64+gzip)

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants