Skip to content

Stats counts empty files (__init__.py, .gitkeep) as duplicates and wasted copies that no verb will ever delete, so it disagrees with DryRun #172

Closed

Description

@matt-edmondson

What's wrong

#151 made Scan, DryRun, PlanDeletions and DeleteDuplicates keep every copy in an empty-file group (Deduplicator.IsDeletable(group) is group.FileSize > 0). Verbs/Stats.cs was not updated, so it still reports empty files as redundant:

  • Stats.cs:63 computes duplicateFiles = fileHashes.Count - uniqueFiles over all groups, empty ones included.
  • CountRedundantCopiesByExtension (Stats.cs:117-126) treats only SelectFileToKeep's pick as the keeper and never checks IsDeletable, so for an empty group every copy but one is counted as a redundant copy. Its doc comment says it "leav[es] out the copy that is kept", but for an empty group every copy is kept.
  • The "Largest duplicate groups" list (Stats.cs:96) also includes empty groups.

Reproduction

Take a tree with these files:

  • a/__init__.py, b/__init__.py, c/__init__.py and .gitkeep, all empty
  • A.JPG and b/copy.jpg, with the same 2 bytes

Stats prints:

Duplicate files: 4
Duplicate groups: 2
Duplicate files by extension:
  .py: 3 file(s)
  .jpg: 1 file(s)
Largest duplicate groups (by wasted space):
  73cb3858a687... - 2 copies, 2 B each, 2 B wasted
  e3b0c44298fc... - 4 copies, 0 B each, 0 B wasted

DryRun on the same tree prints Files to delete: 1.

In a Python repository with dozens of __init__.py files, .py dominates the breakdown and "Duplicate files" is badly inflated, even though nothing will be removed.

Related: case variants of one extension are split

Stats.cs:125 uses Path.GetExtension as-is and counts into a case-sensitive dictionary. Three duplicate pairs a.jpg/a2.JPG, b.jpg/b2.jpg and c.JPG/c2.JPG print .JPG: 2 file(s) and .jpg: 1 file(s) rather than one line with 3. Camera and phone photo folders mix these routinely. It is the same method, so it fits in the same PR.

Suggested fix

  • In CountRedundantCopiesByExtension and the largest-groups list, skip groups that fail Deduplicator.IsDeletable.
  • Either compute "Duplicate files" from the deletable groups only, or report empty files on their own line (e.g. Empty files (never deleted): N). Either way, the extension breakdown should again sum to "Duplicate files".
  • Count extensions in a dictionary built with StringComparer.OrdinalIgnoreCase, or normalise them with ToLowerInvariant().
  • Add Stats tests next to the Deduplicate deletes all but one empty file (__init__.py, .gitkeep, py.typed) while reclaiming 0 bytes #151 tests: an empty-file group, and mixed-case extensions.

Acceptance criteria

  • On the tree above, Stats reports 1 redundant file (.jpg: 1), matching DryRun, and does not list the empty group among the largest groups.
  • .jpg and .JPG duplicates are counted under one extension.

Activity

  1. matt-edmondson commented on Oct 6, 2026

    @matt-edmondson
    ContributorAuthor

    Triage


    Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't working

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions