Skip to content

fix(rewrite): keep every repair iteration's output on the staged path - #354

Merged
binaryaaron merged 1 commit into
mainfrom
aagonzales/bugfix/rewrite-repair-overwrite
Nov 4, 2026
Merged

binaryaaron merged 1 commit into
mainfrom
aagonzales/bugfix/rewrite-repair-overwrite

Conversation

@binaryaaron

@binaryaaron binaryaaron commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

In Rewrite mode, every repair after the first was discarded. The evaluate-repair loop joined each repair result onto the failing rows without overwrite; from the second repair on, the rows already carried _rewritten_text__next, so the join kept the first repair's text. Re-evaluation scored that same text again, _repair_iterations still went up, and max_repair_iterations above 1 (the default is 3) only added model calls.

The fix

One line in src/anonymizer/engine/rewrite/rewrite_workflow.py, mirroring the re-evaluation join below it:

failing_rows = _join_new_columns(failing_rows, repaired, overwrite=True, seed_cols=repair_seed_cols)

overwrite=True lets the repair step's outputs (_leaked_privacy_items, _rewritten_text__next) replace the previous iteration's. seed_cols=repair_seed_cols protects the columns the step only read, which the adapter echoes back serialised (_privacy_qa would come back as a JSON string): the record id, original text, current rewrite, rewrite baseline, _privacy_qa, _privacy_qa_reanswer, leakage_mass, any_high_leaked, utility_score.

Behaviour

One repair: no change. More than one: each repair sees the previous repair's text and re-evaluation scores the new text; a row that passes after its second repair stops with _repair_iterations == 2. For rows repaired more than once, the trace columns _leaked_privacy_items and _rewritten_text__next show the last iteration; column set and dtypes are unchanged. The combined path (Rewrite(use_combined_graph=True)) is unaffected: it writes each iteration to its own columns.

Checks

  • make check passes. Full suite: 1,587 passed at f228263b on main (f294dc1e). Earlier validation also covered tests/engine with pandas 3.
  • New tests, each with adapter-shaped stubs (a step returns its seed plus only its own columns):
    • test_repair_loop_keeps_each_iterations_repaired_text: three repairs with distinct text, row never passes; final text is the third repair's, counter 3, each later step receives the previous repair's text. Without the fix the final text is the first repair's.
    • test_repair_join_keeps_seed_columns: _privacy_qa seeded as a JSON string stays a dict. Passes before and after; fails if seed_cols is dropped.
    • test_repair_loop_mixed_rows_keep_their_last_repair: rows passing after 0, 1, 2 or never; final text, counter, rows sent to each repair and output order all checked.
    • test_repair_loop_stops_after_successful_repair: passes after the second of three allowed repairs.
    • test_repair_loop_through_real_adapter_keeps_each_repair: through the real NddAdapter with deterministic custom columns, no model requests.
    • test_combined_and_legacy_paths_return_equivalent_repaired_result (strengthened): two distinct repairs; the combined result comes from the real graph finalizer through the real adapter.
  • Five of the six fail on previous main (the seed-guard test is meant to pass on both). Removing overwrite=True fails the same five; removing seed_cols fails the seed-guard and real-adapter tests.

Previously stacked on #353; it does not depend on that code and now targets main directly.

🤖 Generated with Claude Code

The staged evaluate-repair loop joined each repair result onto the failing
rows without overwrite. From the second repair on, the rows already carried
the first repair's `_rewritten_text__next`, so the new repair output was
dropped: the row kept its first repair's text, the re-evaluation scored that
same text again, and `_repair_iterations` still went up. Setting
`max_repair_iterations` above 1 only added model calls.

Join the repair result with `overwrite=True`, excluding the repair seed
columns as the re-evaluation join already does, so the adapter's serialised
copies of the seed never replace the frame's values. The repair step's own
outputs, `_leaked_privacy_items` and `_rewritten_text__next`, now land on
every iteration. Behaviour with a single repair is unchanged; the combined
path keeps each iteration in its own columns and was not affected.

Tests use adapter-shaped stubs with distinct text per repair, a run through
the real adapter with deterministic columns, and a staged-vs-combined
comparison that takes the combined result from the real graph finalizer.

🤖 Generated with Claude Code

Co-Authored-By: Claude <noreply@anthropic.com>
Signed-off-by: Aaron Gonzales <aagonzales@nvidia.com>
@binaryaaron
binaryaaron requested a review from a team as a code owner October 30, 2026 05:25
@greptile-apps

greptile-apps Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable issues were found.

Summary

The staged rewrite path now keeps each repair’s latest output while preserving the columns passed into the repair step.

  • Rewrite mode uses each repair’s text in the next attempt.

No actionable issues were found. Tests were inspected but not run during this review.

Reviews (4) · Last reviewed commit: "fix(rewrite): keep every repair iteratio..." · Reviewed by Greptile

@binaryaaron
binaryaaron requested a review from a team as a code owner October 30, 2026 16:49
@binaryaaron
binaryaaron changed the base branch from main to aagonzales/bugfix/rewrite-coverage-retention October 30, 2026 16:49

Copy link
Copy Markdown
Collaborator Author

agent (develop): The repair-loop fixtures now work with #353.

Published as c1d6867fa1e97e9ff921055d762871200e4596e5. The deterministic evaluators now produce an empty _retained_protected_entities list, matching the repair dependency introduced by #353. This fixes the missing-producer failure in test_repair_loop_through_real_adapter_keeps_each_repair when both PRs are combined.

The updated #353 prerequisite was merged without rewriting either branch, and this PR now targets its branch. Merge #353 first. The remaining diff is still the one-line production fix plus its tests.

Validation: 127 focused rewrite/combined/retention tests passed; full make test passed all 1,660 tests; required commit hooks, including make check, passed.

@asteier2026

Copy link
Copy Markdown
Contributor

I would like to approve the changes in this PR, but it's stacked on PR 353 which I don't think we want.

@binaryaaron
binaryaaron force-pushed the aagonzales/bugfix/rewrite-repair-overwrite branch from c1d6867 to f228263 Compare November 2, 2026 23:04
@binaryaaron
binaryaaron removed this pull request from stack #355 November 2, 2026 23:10
@binaryaaron
binaryaaron changed the base branch from aagonzales/bugfix/rewrite-coverage-retention to main November 2, 2026 23:10
@binaryaaron binaryaaron closed this Nov 2, 2026
@binaryaaron binaryaaron reopened this Nov 2, 2026
@binaryaaron

Copy link
Copy Markdown
Collaborator Author

This PR is now based on main. #353 was closed in favour of #333, and this branch was rebased onto main on 2026-11-02 (head f228263). The diff is the one-line overwrite=True fix plus its tests; all checks pass.

@andreatnvidia andreatnvidia left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. Overwriting the repair outputs while preserving the seed columns fixes the repeated-repair behavior and follows the existing re-evaluation pattern.

Comment thread tests/engine/test_rewrite_workflow.py Resolved

Copy link
Copy Markdown
Collaborator

One follow-up for the graph line: #319 and #365 still call run_columns(..., "rewrite-repair-...") without overwrite=True, so later repairs still leave the first repair’s text in place. Merging this into #319 conflicts in that loop. Could we carry the overwrite flag and these regression cases into the conflict resolution? Not blocking this PR against main.

@binaryaaron
binaryaaron merged commit 95b7770 into main Nov 4, 2026
53 of 59 checks passed
@binaryaaron
binaryaaron deleted the aagonzales/bugfix/rewrite-repair-overwrite branch November 4, 2026 19:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants