Conversation
| WHERE active | ||
| ) b | ||
| ON a.database = b.database AND a.table = b.table AND a.partition_id = b.partition_id | ||
| WHERE a.min_block_number IS NOT NULL |
There was a problem hiding this comment.
Do we really want to suggest removing all?
| Here is a quick way to find out if you have detached parts along with the reason why. | ||
| ### Monitoring of detached parts | ||
|
|
||
| You can find information in `clickhouse-server.log`, for what happened when the parts were detached during startup. If `clickhouse-server.log` is lost it might be impossible to figure out what happened and why the parts were detached. |
There was a problem hiding this comment.
part_log fragment disappeared.
There was a problem hiding this comment.
I removed it, because checked and RemovePart event_type is generated if there is a manual ALTER TABLE DETACH partition, but if clickhouse is detaching because of problem when starting up, it seems RemovePart event is not pushed to part_log. But I can recheck with codex again. Did that earlier when seeing that fragment about part_log.
There was a problem hiding this comment.
Codex:
In ClickHouse, system.part_log.event_type='RemovePart' is written when a part is actually removed via the normal part-removal pipeline (removePartsFinally()).
Main cases:
Old/outdated parts are physically cleaned up after merges/mutations/replace/drop workflows.
ALTER ... DROP PART|PARTITION paths that end with old part cleanup.
ALTER ... DETACH PART|PARTITION also results in RemovePart for the removed-from-working-set old part during cleanup.
Important exceptions:
Automatic startup detach of broken parts (broken-on-start) is usually just renameToDetached(...), so no RemovePart is written at that moment.
Other direct renames to detached/ (without passing final removal path) won’t emit RemovePart.
Code points:
Write of RemovePart: MergeTreeData.cpp (line 3437)
Removal pipeline call: MergeTreeData.cpp (line 3765)
Startup broken detach (no part_log add there): MergeTreeData.cpp (line 2461)
There was a problem hiding this comment.
So we could check for manual ops in part_log. I guess clickhouse-server log will provide more info than part_log here when there was a unexpected/broken part
There was a problem hiding this comment.
In most of the cases in the part log i'm not looking for the deletion event, but for what what happening with the part before (like when was created, merged, etc).
| Here is a query that can help with investigations. It looks for active parts containing the same data blocks as the detached parts and generates commands to drop the detached parts. | ||
|
|
||
| ```sql | ||
| SELECT a.*, |
There was a problem hiding this comment.
since we generating set of ALTER statements there, i would put all missing details to the comment of ALTER, to simplify copy and paste.
2 participants
Updated detached-parts KB with safer cleanup guidance and query fixes: added warnings before destructive actions, added asynchronous detached-part monitoring queries, fixed the bulk DROP DETACHED helper to use system.detached_parts block columns (validated locally), and shortened appendix/reference links for readability.
I have read the CLA Document and I hereby sign the CLA