Repository navigation
Description
Describe the bug
A frame containing a Compressed_Block whose Block_Size is 0 is rejected by ZSTD_decompress ("Data corruption detected") but accepted, as an empty block, by ZSTD_decompressStream whenever it decodes through ZSTD_decompressContinue instead of the single-pass path. That is the case for every frame without a Frame_Content_Size, and for any frame whose input arrives in pieces. zstd -t passes such a frame without an error.
A Compressed_Block's content is a Literals_Section followed by a Sequences_Section (RFC 8878, section 3.1.1.3), each beginning with a header of at least one byte, so Block_Size 0 is never valid for this block type.
To Reproduce
# 13 bytes: no Frame_Content_Size (Window_Descriptor 0), one last Compressed_Block
# with Block_Size 0, then the content checksum of the empty output
printf '\x28\xb5\x2f\xfd\x04\x00\x05\x00\x00\x99\xe9\xd8\x51' > empty-compressed-block.zst
zstd -t empty-compressed-block.zst # 1.5.7: "0 bytes", exit status 0 (no error)Results with libzstd 1.5.7:
| Frame | ZSTD_decompress |
ZSTD_decompressStream, whole frame in one call, 64 KiB output |
ZSTD_decompressStream, one input byte per call |
|---|---|---|---|
28 b5 2f fd 04 00 05 00 00 99 e9 d8 51 (no FCS, last Compressed_Block of size 0) |
Data corruption detected | complete, 0 bytes | complete, 0 bytes |
28 b5 2f fd 24 00 05 00 00 99 e9 d8 51 (FCS 0, same block) |
Data corruption detected | Data corruption detected | complete, 0 bytes |
28 b5 2f fd 24 00 04 00 00 01 00 00 99 e9 d8 51 (FCS 0, non-last Compressed_Block of size 0, then an empty last Raw_Block) |
Data corruption detected | Data corruption detected | complete, 0 bytes |
The second frame differs from the frame zstd writes for an empty input (28 b5 2f fd 24 00 01 00 00 99 e9 d8 51) only in the block type bits.
Expected behavior
ZSTD_decompressStream rejects these frames, as ZSTD_decompress does, whatever the buffer sizes.
Cause
In ZSTD_decompressContinue, case ZSTDds_decodeBlockHeader sends every block with cBlockSize == 0 down the /* empty block */ branch, whatever its type. ZSTD_getcBlockSize returns 1 for RLE blocks, so only Raw and Compressed blocks get there, and only a Raw_Block can be empty. The one-shot path decodes the empty compressed block like any other and fails on the missing literals header.
Suggested fix
Reject it at the top of the empty-block branch, the branch where #4885 adds the Frame_Content_Size check for #4880:
/* empty block */
RETURN_ERROR_IF(bp.blockType == bt_compressed, corruption_detected,
"Compressed_Block with Block_Size 0");Valid frames are unaffected: the encoder never writes a compressed block of size 0, and ZSTD_decompress already rejects one.
Desktop
- OS: Linux x86-64
- Version: libzstd and
zstd1.5.7 (distribution packages); the dev branch was not tested
Additional context
Found by fuzzing that compares one-shot and streaming decoding of mutated frames. Related: #4880 (the same branch skips the Frame_Content_Size check) and its fix #4885.
Activity