Skip to content

ZSTD_decompressStream accepts a Compressed_Block with Block_Size 0 that ZSTD_decompress rejects #4900

Open

Description

@vi-sa-vie

Describe the bug

A frame containing a Compressed_Block whose Block_Size is 0 is rejected by ZSTD_decompress ("Data corruption detected") but accepted, as an empty block, by ZSTD_decompressStream whenever it decodes through ZSTD_decompressContinue instead of the single-pass path. That is the case for every frame without a Frame_Content_Size, and for any frame whose input arrives in pieces. zstd -t passes such a frame without an error.

A Compressed_Block's content is a Literals_Section followed by a Sequences_Section (RFC 8878, section 3.1.1.3), each beginning with a header of at least one byte, so Block_Size 0 is never valid for this block type.

To Reproduce

# 13 bytes: no Frame_Content_Size (Window_Descriptor 0), one last Compressed_Block
# with Block_Size 0, then the content checksum of the empty output
printf '\x28\xb5\x2f\xfd\x04\x00\x05\x00\x00\x99\xe9\xd8\x51' > empty-compressed-block.zst
zstd -t empty-compressed-block.zst    # 1.5.7: "0 bytes", exit status 0 (no error)

Results with libzstd 1.5.7:

Frame ZSTD_decompress ZSTD_decompressStream, whole frame in one call, 64 KiB output ZSTD_decompressStream, one input byte per call
28 b5 2f fd 04 00 05 00 00 99 e9 d8 51 (no FCS, last Compressed_Block of size 0) Data corruption detected complete, 0 bytes complete, 0 bytes
28 b5 2f fd 24 00 05 00 00 99 e9 d8 51 (FCS 0, same block) Data corruption detected Data corruption detected complete, 0 bytes
28 b5 2f fd 24 00 04 00 00 01 00 00 99 e9 d8 51 (FCS 0, non-last Compressed_Block of size 0, then an empty last Raw_Block) Data corruption detected Data corruption detected complete, 0 bytes

The second frame differs from the frame zstd writes for an empty input (28 b5 2f fd 24 00 01 00 00 99 e9 d8 51) only in the block type bits.

Expected behavior

ZSTD_decompressStream rejects these frames, as ZSTD_decompress does, whatever the buffer sizes.

Cause

In ZSTD_decompressContinue, case ZSTDds_decodeBlockHeader sends every block with cBlockSize == 0 down the /* empty block */ branch, whatever its type. ZSTD_getcBlockSize returns 1 for RLE blocks, so only Raw and Compressed blocks get there, and only a Raw_Block can be empty. The one-shot path decodes the empty compressed block like any other and fails on the missing literals header.

Suggested fix

Reject it at the top of the empty-block branch, the branch where #4885 adds the Frame_Content_Size check for #4880:

            /* empty block */
            RETURN_ERROR_IF(bp.blockType == bt_compressed, corruption_detected,
                            "Compressed_Block with Block_Size 0");

Valid frames are unaffected: the encoder never writes a compressed block of size 0, and ZSTD_decompress already rejects one.

Desktop

  • OS: Linux x86-64
  • Version: libzstd and zstd 1.5.7 (distribution packages); the dev branch was not tested

Additional context

Found by fuzzing that compares one-shot and streaming decoding of mutated frames. Related: #4880 (the same branch skips the Frame_Content_Size check) and its fix #4885.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions