Skip to content

bin/vh pace: fixes from the review of #72 - #73

Merged
ZLHad merged 2 commits into
mainfrom
claude/pace-review-fixes
Oct 7, 2026
Merged

ZLHad merged 2 commits into
mainfrom
claude/pace-review-fixes

Conversation

@ZLHad

@ZLHad ZLHad commented Oct 7, 2026 •

Copy link
Copy Markdown
Owner

Follow-up to #72, which merged (auto-merge) before its independent review came back. This fixes what the review found.

Why

The reviewer found no path that loses the original take, but these defects:

  1. An ASR boundary off by more than ~0.1 s dropped syllables or moved one behind a pause. Line edges were found by walking from the ASR start/end while the signal stayed loud, so a 10 ms dip (a stop closure, a gap between syllables) stopped the walk. The split between two lines picked the nearest quiet run of ≥ 30 ms, which can be a gap between syllables.
  2. A traceback when a line has no audio (a skipped last line, or a timeline that reaches past the audio).
  3. --align gemini without a key wrote the take, then exited before the captions (the same for --align whisper without mlx-whisper).
  4. --restore --dry-run restored.
  5. A missing script.txt (or ids not in it) silently changed the result.
  6. --ref on a "……" line: ZeroDivisionError.
  7. The language could not follow the flags.
  8. The hop was 220 samples at 22.05 kHz while times assumed 10 ms (0.23% drift).
  9. "……" lines collapsed to a point.
  10. A take without word times got its .raw timeline rewritten with the transcription.

The --restore refusal also suggested deleting the .raw files, which may be the only copy of the original.

What changed

Line edges (1). The ASR start and end now only locate the silence between two lines: the quiet run nearest the boundary, within 0.2 s of it, at least 0.12 s long when there is one. Shorter dips are between syllables. A line's voice is everything voiced between the silences on either side, so nothing voiced can be dropped. A blip under 0.12 s at a line's edge, set off by 0.06 s of quiet (a breath), stays in the audio but no longer counts as speech in the pace and the pauses. The real take showed why this matters: a breath before a line made it measure 6.40 chars/s instead of 6.93.

Word times follow the audio. atempo stretches pauses and speech by different amounts. The loudness envelopes before and after are now aligned by dynamic time warping, within ±0.15 s of the even stretch. The first word of a line starts, and the last one ends, where the speech does.

The other findings:

  • No audio for a line → it is a point, marked "nothing heard" (2).
  • Key and mlx-whisper are checked before anything is written (3).
  • --restore --dry-run is a usage error (4).
  • A note when script.txt or some ids are missing (5).
  • --ref with no heard line is an error (6).
  • parse_intermixed_args (7).
  • A whole-sample hop (8).
  • A pause with a "……" line in it is kept as it is (9).
  • The transcription of a take without word times goes to audio/pace.<lang>.asr.json, keyed by the take's sha256. The .raw timeline stays byte-identical, and --dry-run writes nothing (10).
  • The refusal text says to keep the .raw files.

Docs: the README section, playbook/04 step 1, the module docstring and the CHANGELOG describe the new edges, the breath rule, "……" lines, the cache, --restore, and the one case it can't settle: two lines run together with < 0.12 s between them and an ASR boundary off by a syllable can put that syllable in the neighbouring line (nothing is lost).

How it was verified

The reviewer's repro scripts, on the synthetic take; syllables lost out of 40 trials:

ASR error before after
starts 0.1 s late 0/40 0/40
starts 0.2 / 0.3 / 0.5 s late 40/40 each 0/40 each
ends 0.2 s early 4/40 0/40
ends 0.3 / 0.5 s early 40/40 each 0/40 each
starts early, ends late 0/40 0/40

A boundary 0.1 s inside a syllable moved it behind a 0.39 s pause before; now it stays where it was (0.05–0.2 s tried).

Mapped word starts vs the real onsets on the synthetic take:

before after
48 kHz median 2 ms, max 24–38 ms median 2 ms, max 8 ms
22.05 kHz, a line slowed to ×0.92 up to 61 ms 27 ms

The 2-minute, 27-line Gemini take (scratch copy):

  • Pace 5.18–7.38 → 5.88–6.83 chars/s (sd 0.50 → 0.20), 13 lines' tempo untouched, 129.5 → 121.5 s, 1.4 s to run.
  • Mapped line starts sit on the voice onsets: median 0.000 s, max 0.032 s (before: within 0.034 s, mostly 0.03 s early).
  • Whisper re-alignment puts line starts 0.20 s early (median).
  • bin/vh qa click warnings: 54, against 57 in the original take.
  • The same take with its word times removed: --align whisper transcribes once into pace.zh.asr.json, and a later default run reuses it. The .raw timeline stays byte-identical, and the pace numbers are the same as with Gemini's word times, since pace is now measured from the waveform.

CI: pace_checks now uses a 22.05 kHz take with a "……" line, and adds:

  • ASR starts 0.2 s late and ends 0.3 s early;
  • a line break 0.1 s inside a syllable;
  • a line with no audio;
  • --restore --dry-run;
  • --align gemini without a key (nothing written);
  • --ref on a line without words;
  • voiceover.wav / timeline.json restored too.

Run against #72's pace.py, the new checks fail on the "……" pause, --restore --dry-run, the no-audio crash, the keyless write and the --ref crash.

tools/ci.sh --committed passes, also with VH_BASH=/bin/bash (shellcheck and pyflakes through the uv shim).

Second review (of this PR) and its fixes (b9ee72a)

An independent Sonnet review of 6d1b18d checked:

  • about 1000 random timelines at 11.025–48 kHz: no voiced frame dropped;
  • that the chunks tile exactly;
  • word-time mapping at odd rates (≤ 20 ms);
  • --restore and the cache flow.

It found these defects, now fixed:

Finding Fix
The breath rule peeled every short isolated syllable at a line edge, with no loudness test: a line opening with three short loud syllables measured 6.36 for a true 5.07 One blip per side, 15 dB under the line's speech. Now 5.17
A line with no audio in the middle got zero-width times and an extra clamped pause (0.99 → 1.24 s); a whispered line was replaced by silence The line stays inside the pause around it: that stretch of the take is copied as it is, the line gets a 0.1 s span, and the command names it
Spans that go back in time (a hand edit) repeated up to 3 s of speech Refused
A partial script.txt made a block break the note denied Lines missing from it count as one block
CI's 60 ms word bound passed with the DTW alignment reverted Now 35 ms (27 with the alignment, 61 without)
The breath rule, the cache, --target / --extra / the language after the flags, and the notes had no CI cases Cases added
The dense DTW matrix took 1.2 GB for one 120 s line Banded: 0.23 GB
The first transcription was unguarded Exits with a message
An unreachable 50 ms guard Removed

Documented rather than changed: a whole ASR boundary shifted ≥ 0.4 s one way can put a few syllables in the neighbouring line (nothing is lost); output is mono; a hand-edited paced timeline is rebuilt from .raw on the next run.

Against 6d1b18d's pace.py, the new CI cases fail: swapped spans not refused, the no-audio middle line at 33 syllables with its pause clamped, the breath case at 6.83 for a true 5.24. Against b9ee72a all pass. The real 27-line take is unchanged by this round (5.88–6.83 chars/s, line starts within 0.032 s of the voice, qa clicks 54). tools/ci.sh --committed passes, also with VH_BASH=/bin/bash.

An independent review of #72 (merged before its findings were in) found that an ASR boundary off by more than about
0.1 s could drop syllables or move one behind a pause, a crash when a line has no audio, a missing GEMINI_API_KEY found
only after the take was written, and smaller gaps (--restore --dry-run restored; --ref on a "……" line divided by zero;
a 22.05 kHz hop drift; "……" lines collapsed; the .raw timeline rewritten with a transcription).

A line's voice is now everything voiced between the silences that part it from its neighbours; the ASR start and end
only locate those silences, so nothing voiced can be dropped. Word times follow the audio through atempo's uneven
stretch (loudness envelopes aligned by DTW). Keys and mlx-whisper are checked before anything is written. tools/ci.sh
covers each case; run against #72's pace.py the new checks fail where the review said.
@ZLHad
ZLHad marked this pull request as ready for review October 7, 2026 16:08
An independent review of this PR found:
- the breath rule peeled every short isolated syllable at a line edge with no loudness test (a line opening with three
  short loud syllables measured 6.36 for a true 5.07); now one blip per side, 15 dB under the line's speech;
- a line with no audio in the middle got zero-width times and an extra clamped pause, and a whispered line was replaced
  by silence; now such a line stays inside the pause around it, that stretch of the take is copied as it is, the line
  gets a 0.1 s span and is named;
- spans that go back in time (a hand edit) repeated up to 3 s of audio; now refused;
- a partial script.txt made a block break the note denied; lines missing from it now count as one block;
- CI's 60 ms word bound could not catch a revert of the DTW alignment (now 35 ms: 27 with it, 61 without), and the
  breath rule, the cache, --target/--extra and the notes had no cases;
- the dense DTW matrix took 1.2 GB for one 120 s line (now banded, 0.23 GB); the first transcription was unguarded;
  an unreachable 50 ms guard.

Each new CI case fails against the previous commit and passes now.
@ZLHad
ZLHad enabled auto-merge (squash) October 7, 2026 18:10
@ZLHad
ZLHad merged commit 4d57730 into main Oct 7, 2026
2 checks passed
@ZLHad
ZLHad deleted the claude/pace-review-fixes branch October 7, 2026 18:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant