Repository navigation
ZSTD_decompressStream skips the content-size check when the last block is empty #4880
Description
Describe the bug
ZSTD_decompressStream reports a frame as complete even though it decoded fewer bytes than the frame's Frame_Content_Size, if the frame ends with an empty last block. ZSTD_decompress rejects the same frame with Data corruption detected.
ZSTD_decompressContinue checks decodedSize against frameContentSize only in case ZSTDds_decompressLastBlock (L1377-L1382). An empty last block ends the frame earlier, in case ZSTDds_decodeBlockHeader (L1322-L1330), and that path has no check.
To Reproduce
The frame declares 1,000,000 bytes of content and holds a 5-byte raw block followed by an empty last raw block:
28 b5 2f fd a0 40 42 0f 00 28 00 00 68 65 6c 6c 6f 01 00 00
make -C lib libzstd.acc -Ilib poc.c lib/libzstd.a -o poc && ./poc(PoC below)- Output:
declared content size 1000000
ZSTD_decompress: Data corruption detected
ZSTD_decompressStream: success, 5 bytes, frame complete
poc.c
#include <stdio.h>
#include <stdlib.h>
#include "zstd.h"
int main(void)
{
/* declares 1,000,000 bytes of content; holds one 5-byte raw block, then an empty last block */
static const unsigned char frame[] = {
0x28, 0xb5, 0x2f, 0xfd, /* magic number */
0xa0, /* single segment, 4-byte content size, no checksum */
0x40, 0x42, 0x0f, 0x00, /* content size: 1,000,000 */
0x28, 0x00, 0x00, 'h', 'e', 'l', 'l', 'o', /* raw block of 5 bytes, not the last */
0x01, 0x00, 0x00, /* raw block of 0 bytes, the last */
};
unsigned long long declared = ZSTD_getFrameContentSize(frame, sizeof frame);
unsigned char *all = malloc((size_t) declared);
size_t one = ZSTD_decompress(all, (size_t) declared, frame, sizeof frame);
printf("declared content size %llu\n", declared);
printf("ZSTD_decompress: %s\n", ZSTD_isError(one) ? ZSTD_getErrorName(one) : "success");
ZSTD_DStream *ds = ZSTD_createDStream();
size_t cap = ZSTD_DStreamOutSize();
unsigned char *buf = malloc(cap);
ZSTD_inBuffer in = { frame, sizeof frame, 0 };
size_t total = 0, r = 1;
while (in.pos < in.size || r != 0) {
ZSTD_outBuffer out = { buf, cap, 0 };
size_t before = in.pos;
r = ZSTD_decompressStream(ds, &out, &in);
total += out.pos;
if (ZSTD_isError(r) || r == 0 || (in.pos == before && out.pos == 0))
break;
}
printf("ZSTD_decompressStream: %s, %zu bytes, %s\n", ZSTD_isError(r) ? ZSTD_getErrorName(r) : "success",
total, r == 0 ? "frame complete" : "frame incomplete");
ZSTD_freeDStream(ds);
free(buf);
free(all);
return 0;
}Expected behavior
ZSTD_decompressStream returns corruption_detected, as ZSTD_decompress does, and as streaming itself does when the last block is not empty.
Desktop:
- OS: Ubuntu 24.04.4 LTS
- Version: dev 49cf517; also v1.5.7, v1.5.6 and v1.4.9 (v1.4.9's one-shot error is
Corrupted block detected) - Compiler: gcc 13.3.0
- Flags: default
lib/Makefileflags, PoC built with-O2 - Other relevant hardware specs: x86_64
- Build system: Makefile
Additional context
Suggested fix: run the same check where an empty last block ends the frame. With it the PoC's streaming call returns Data corruption detected.
--- a/lib/decompress/zstd_decompress.c
+++ b/lib/decompress/zstd_decompress.c
@@ -1321,6 +1321,10 @@ size_t ZSTD_decompressContinue(ZSTD_DCtx* dctx, void* dst, size_t dstCapacity, c
}
/* empty block */
if (bp.lastBlock) {
+ RETURN_ERROR_IF(
+ dctx->fParams.frameContentSize != ZSTD_CONTENTSIZE_UNKNOWN
+ && dctx->decodedSize != dctx->fParams.frameContentSize,
+ corruption_detected, "");
if (dctx->fParams.checksumFlag) {
dctx->expected = 4;
dctx->stage = ZSTDds_checkChecksum;Found by genproof.com Formal Verification Agent. Reviewed and approved by a human.
Activity
[-]`ZSTD_decompressStream` accepts a frame shorter than its `Frame_Content_Size` when the last block is empty[/-][+]`ZSTD_decompressStream` skips the content-size check when the last block is empty[/+]vi-sa-vie commentedon Nov 6, 2026
I found this independently, while fuzzing a decoder by comparing one-shot and streaming results, and can confirm it with libzstd 1.5.7. Two details that may help:
The verdict also depends on the output buffer size, so the CLI is affected. With the whole frame in one call,
ZSTD_decompressStreamrejects the frame only when it takes the single-pass path inzdss_loadHeader, that is, when the output buffer can hold the declared size. With a smaller output buffer, or with input arriving in pieces, it goes throughZSTD_decompressContinueand accepts the frame. As a result,zstd -tpasses a frame whose declared size exceeds the CLI's output buffer:ZSTD_decompress28 b5 2f fd 24 a9 01 00 00 99 e9 d8 51(FCS 169)28 b5 2f fd a4 40 42 0f 00 01 00 00 99 e9 d8 51(FCS 1000000)The first frame differs from the frame
zstdwrites for an empty input only in the Frame_Content_Size byte (00→a9).The same empty-block branch also skips a second check. It treats a Compressed_Block with Block_Size 0 as empty too, although a compressed block always holds at least its section headers;
ZSTD_decompressrejects such a frame and the streaming decoder accepts it. #4885 adds only the content-size check, so I have filed that case separately as #4900; its fix would go in the same place.