[WIP] Feature: Support TRUNCATE TABLE for Iceberg engine

[il9ue]

[Feature] Support TRUNCATE TABLE for Iceberg Engine

Overview

As part of the Antalya release, v26.1 needs to natively support the TRUNCATE TABLE command for the Iceberg database engine. Currently, upstream ClickHouse explicitly rejects this operation. As of PR ClickHouse#91713, executing TRUNCATE down-casts to StorageObjectStorage, where it immediately throws an ErrorCodes::NOT_IMPLEMENTED exception for Data Lake engines.

To support standard analytics workflows and testing pipelines without requiring users to DROP and recreate tables (which breaks catalog bindings), implementing a metadata-only truncation is essential.

Proposed Architecture

Unlike a standard MergeTree truncation that physically drops parts from the local disk, Iceberg truncation must be entirely logical. The implementation will leave physical file garbage collection to standard Iceberg maintenance operations and focus strictly on metadata manipulation.

Core Workflow:

1. Bypass the Upstream Block: Modify StorageObjectStorage::truncate to check data_lake_metadata->supportsTruncate().
2. Snapshot Generation: Generate a new Iceberg snapshot ID and increment the metadata version (v<N+1>.metadata.json).
3. Empty Avro Manifest List: The new snapshot cannot simply omit the manifest list. We must use ClickHouse's internal object storage and Avro APIs to generate and write a strictly typed, empty Avro manifest list to object storage.
4. Metadata Update: Attach the new empty manifest list to the new snapshot, append the snapshot to the snapshots array, and update the snapshot-log and current-snapshot-id.
5. Catalog Commit: Perform an atomic swap via the ICatalog interface (e.g., REST Catalog) to point the table to the newly generated metadata JSON.

Implementation Details

The required changes span the following internal abstractions:

• src/Storages/ObjectStorage/StorageObjectStorage.h/cpp: Override truncate and remove the hardcoded throw Exception. Delegate to IDataLakeMetadata.
• src/Storages/ObjectStorage/DataLakes/IDataLakeMetadata.h: Introduce supportsTruncate() and truncate(ContextPtr, ICatalog) virtual methods.
• src/Storages/ObjectStorage/DataLakes/Iceberg/IcebergMetadata.h/cpp: Implement the core truncation logic. Must safely obtain an IObjectStorage write buffer via context->getWriteSettings() to serialize the empty Avro file before committing the JSON metadata.

Acceptance Criteria

• (to be updated)

[github-actions]

Workflow [PR], commit [de68832]

[arthurpassos]

I haven't implemented anything for Iceberg yet, tho it is in my todo list. I left two small comments for now.

I also looked at the transactional model and it looks ok (assuming I understood it correctly).

My understanding of the Iceberg + catalog transactional model is that updating the catalog is the commit marker, and if it fails, the transaction isn't complete even if the new metadata files were already uploaded. Those become orphan and must be ignored. This also implies an Iceberg table should always be read through a catalog if one exists, otherwise it becomes hard to determine the latest metadata snapshot.

I'll read the code more carefully later.

[arthurpassos]

Isn't isDataLake() the same as the above configuration->isDataLakeconfiguration()? see

ClickHouse/src/Storages/ObjectStorage/StorageObjectStorage.h

Line 115 in b806ac5

bool isDataLake() const override { return configuration->isDataLakeConfiguration(); }

[arthurpassos]

I would not change this method simply to avoid merge conflicts with upstream later on