[lipikaramaswamy]

Related Issue

Fixes #262.

Plan Document

Hybrid regex entity detection plan

Summary

Adds regex-based entity detection as a first-class detection source alongside GLiNER and LLM detection.

• Enables built-in rules by default for requested credit/debit card, email, IPv4, IPv6, MAC address, and URL labels.
• Introduces one unified Detect.regex_rules configuration surface for built-in customization and user-defined rules.
• Supports optional local validators and per-rule LLM validation controls, defaulting LLM validation to enabled.
• Allows user-defined rules to replace built-in behavior for the same label without a separate overrides API.
• Adds safe bounded matching, stable content-derived rule IDs, deterministic source-aware merging, and DataDesigner serialization/plugin support.
• Documents the user experience and updates the bundled Anonymizer skill.

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Refactoring
• CI, release, or contributor workflow update

Contributor Checklist

• PR title follows Conventional Commits (type(scope): description).
• Related issue is linked.
• Plan document is linked.
• Tests cover the new behavior.
• Public API and documentation changes are included.
• Commits include a DCO sign-off.

Validation

• Commands run:
  • TMPDIR=/tmp make test — 1312 passed, 1 warning.
  • make format-check — passed.
  • Targeted ty check for changed files — passed.
  • Full commit hooks in a clean uv environment — passed, including format, typecheck, lock, and copyright checks.
  • make docs-build from a clean worktree at this commit — passed.
• Skipped checks or known failures: None.

Documentation and Artifacts

• Docs updated, or not needed.
• If docs changed: make docs-build passes locally.
• If tutorial sources changed: notebooks regenerated and checked.
• If end-to-end behavior changed: relevant e2e checks completed, or not applicable.

PR stack

Merge/review order: #305 → #267 → #335.

• refactor(detection): establish deterministic entity candidate lifecycle #305: candidate-lifecycle foundation, targeting main.
• fix(detection): augment plain source on candidate lifecycle #267: plain-source augmentation, stacked on refactor(detection): establish deterministic entity candidate lifecycle #305.
• fix(detection): clarify partial calendar date validation #335: partial-calendar-date guidance, stacked on fix(detection): augment plain source on candidate lifecycle #267.
• feat(detection): add regex entity detection #265: separate regex feature built on refactor(detection): establish deterministic entity candidate lifecycle #305; it is not a prerequisite or merge gate for refactor(detection): establish deterministic entity candidate lifecycle #305, fix(detection): augment plain source on candidate lifecycle #267, or fix(detection): clarify partial calendar date validation #335.

After a parent is squash-merged, restack the remaining child commits onto the updated parent/main and retarget their PRs as appropriate. These links describe dependency order, not approval or CI status.

[greptile-apps]

Confidence Score: 5/5

[High impact] The PR appears safe to merge, with no new actionable findings.

Summary

Adds regex detection alongside GLiNER and LLM detection.

• Anonymizer finds built-in and custom patterns in text.
• Regex findings join the existing detection and redaction flow.
• Exported detection workflows carry regex rules into DataDesigner.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  T[Input text] --> R[Regex rules]
  T --> G[GLiNER]
  R --> D[Direct acceptance]
  R --> V[LLM validation]
  G --> V
  D --> P[Label and exact-span checks]
  V --> P
  P --> A[Grounded augmentation]
  P --> M[Combine accepted candidates]
  A --> M
  M --> O[Resolve overlaps and publish entities]

Loading

_{Reviews (47) · Last reviewed commit: "test(detection): guard async regex wirin..." · Reviewed by Greptile}

[lipikaramaswamy]

/nvskills-ci

[andreatnvidia]

The overall direction makes sense, and regex plus GLiNER as independent Data Designer columns feels like a solid first step toward layered detection. I don't think this PR needs to build the full multi-detector framework. Keeping the public regex API and focusing here on correct two-source behavior seems like the right scope.

Could we capture a follow-up issue or design for the general framework? I'd include a shared candidate/outcome shape with multiple origins, one independently schedulable column per detector, generic fan-in, overlap resolution after validation, and per-detector preparation, resource, timeout, and failure contracts. A useful acceptance test would be adding a dummy third detector without changing fan-in, while producing identical results regardless of completion order. That follow-up could also replace the currently missing multi-pole design referenced by this plan.

[lipikaramaswamy]

/nvskills-ci

[binaryaaron]

agent (review-pr): Request changes — correctness fixes and integration work remain

local human-driven agent review found a few more bits, in addition to the outstanding comments:

1. Medium: URL validation accepts malformed hosts.
   regex_detection.py:_validate_url() accepts any IDNA-encodable host containing a dot, including http://-bad.com and http://999.999.999.999/path, with validate_with_llm=False. These false positives reduce detection precision and can cause unnecessary replacement of text. Define the intended hostname policy, validate IP-shaped hosts with ipaddress, and validate DNS-style labels before accepting them.

2. Low: callable serialization succeeds but produces an unusable configuration.
   RegexRule serializes a callable as module:qualname, but restored configurations cannot resolve that representation. Direct callables are documented as in-process only; serialization should fail clearly at that boundary instead of emitting a value that later fails as an unknown validator.

3. Previously reported: CASE\K still permits a zero-width match.
   Configuration accepts the pattern, then runtime matching produces (4, 4) and drops the row. Width analysis needs to account for core.Keep.

4. Previously reported: generated files still lack SPDX headers.
   Signing regeneration removed headers from BENCHMARK.md and skill-card.md, and the copyright check still fails. Fix the generator before restoring the headers. The restored files will have different hashes, so regenerate and sign skill.oms.sig after their final contents are settled.

The PR also conflicts with current main across detection code, configuration, tests, interface code, and skill artifacts. This is an integration gate. Conflict resolution must preserve both regex detection and the newer entity-exclusion behavior.

Local validation at 49ea1652: 1341 passed; format and type checks passed; copyright check failed on the two generated Markdown files.

Prompts for managed agents

Agents 1–3 own the fixes below; Agent 4 owns final integration. Each agent should preserve other agents’ changes and report the files changed, validation results, and any unresolved issues.

Agent 1 — URL validation

Fix built-in URL host validation in src/anonymizer/engine/detection/regex_detection.py. Reproduce acceptance of http://-bad.com and http://999.999.999.999/path, and establish the current underscore-host behavior, using BuiltinRegex(label="url", validate_with_llm=False). Define the intended hostname policy, validate IP-shaped hosts with ipaddress, and validate DNS-style labels consistently with IDNA support. Preserve valid Unicode domains, bracketed IPv6 hosts, ports, balanced delimiters, and exact offsets. Add focused positive and negative tests. Keep changes scoped to URL validation.

Agent 2 — regex configuration boundaries

Fix two configuration issues without enabling arbitrary imports. First, reject patterns such as r"CASE\K" during RegexRule validation by accounting for regex._regex_core.Keep semantics; retain the runtime zero-width backstop. Second, make JSON serialization of direct callable validators fail clearly, preserving the documented rule that only registered entry-point names are portable. Add tests covering RegexRule and Detect JSON round trips with registered names, serialization failures for top-level callables, and \K validation. Preserve validator trust boundaries.

Agent 3 — generated skill artifacts

Identify why NVSkills regeneration removes repository SPDX headers from skills/anonymizer/BENCHMARK.md and skills/anonymizer/skill-card.md. Fix the producing template or generation step so future runs retain the headers, then regenerate both files. Renew skills/anonymizer/skill.oms.sig only after the final bytes are stable, and run the copyright and skill-package checks. Coordinate with Agent 4 so any artifact changes during integration are followed by regeneration and signing.

Agent 4 — rebase and integration

Integrate the fixes and rebase PR 265 onto current main, resolving conflicts semantically. Preserve both regex detection and the newer entity-exclusion/denylist behavior through AnonymizerConfig, interface plumbing, detection workflow columns, postprocessing, and tests. Regenerate signed skill artifacts after source conflicts are resolved, and coordinate final signing with Agent 3. Run make test, make format-check, make typecheck, and make copyright-check. Report any remaining conflicts or failed checks without discarding either feature to complete the merge.

[lipikaramaswamy]

Addressed review items 1–3 in dd5bf2e:

• URL validation now distinguishes valid IP literals from invalid IP-shaped hosts and validates every IDNA-normalized DNS label, rejecting leading/trailing hyphens, underscores, empty labels, and malformed numeric addresses. Positive coverage preserves Unicode domains, IPv4, IPv6, and direct deterministic acceptance.
• Regex width validation now handles \\K by resetting sequence width and rejects CASE\\K at configuration time.
• Direct callable validators remain available in Python-mode configuration but now fail clearly on JSON serialization; registered validator names round-trip through both RegexRule and Detect.

Focused tests and the full 1,352-test suite pass; formatting, lint, and type checks pass.

[lipikaramaswamy]

Merge-resolution note on excluded_entity_labels + regex detection:

• Exclusions are applied to the effective label set before GLiNER, built-in regex rule selection, and LLM prompts are constructed. Custom regex labels are added to the effective scope first, then exclusions are applied.
• Exclusions are applied again after locally accepted regex entities and LLM-validated entities are merged. This second boundary is a safety net for regex rules using validate_with_llm=False, so that route cannot bypass excluded_entity_labels.
• The same final filtering remains in place during finalization to protect against reclassification or any later detection route producing an excluded label.

Added integration coverage for excluded custom-regex labels and locally accepted regex matches at both the pre-augmentation and finalization boundaries. The merged suite passes locally (1,467 tests), and the Python 3.11/3.12/3.13 CI jobs pass.

@memadi tagging you for visibility since you authored the excluded-labels change. The merge keeps its original early filtering behavior and extends the safety-net filtering to the new regex routes.

[binaryaaron]

agent (review-pr): Three reproducible correctness issues remain.

Verified against 39c31515. The stale-decision replay issue is fixed, including a passing regression. Three additional correctness issues remain reproducible: normalized entity identity, regex precedence through augmentation, and export checks on excluded rules. These need bounded internal fixes; no public API changes are required. The existing custom-column suite passes (52 tests), but four targeted cases still fail across these three findings.

[memadi-nv]

Left one nice-to-have suggestion, and two medium-priority issues that I think is better to be addressed before merge. Overall, the implementation looks good. Thanks, Lipika!

[greptile-apps]

URLs after colons disappear

The new start boundary rejects URLs immediately after an ASCII colon, such as url:https://example.org/private/alice. These matched before this change. With BuiltinRegex(label="url", detect_additional_matches=False), the models cannot recover the missed URL, so it remains unredacted. Distinguish unsupported scheme prefixes from ordinary prose or key-value separators instead of rejecting every preceding colon.

[lipikaramaswamy]

Pushed a deliberately narrow fix in 0937044e: a standalone, case-insensitive url: / URL: prefix can precede any supported network scheme or www. form without a space. The lookbehind excludes the prefix from the matched span, so offsets and redaction affect only the URL. Tests cover case variants, all supported locator forms, both LLM-validation settings, and exported offline preview/create through final redaction. myurl:, _url:, custom:url:, git+https://, and s3://www... remain rejected.

Scope decision: this is NOT a blanket allowance for preceding colons. Visit:https://... and custom:https://... remain outside the built-in's conservative scope; whitespace or a custom rule is required, and the limitation is documented. It fixes the reported url:https://... example but is narrower than your general prose/key-value request, so I am leaving the thread open for agreement on that policy.

Verification at 0937044e: 629 relevant config/public-API/skill/matcher/lifecycle/export/candidate/admission tests passed with the PR's locked dependencies; normal commit hooks, including aggregate make check and DCO, passed. This is local deterministic regression evidence, not a new model benchmark.

[lipikaramaswamy]

Luna-only benchmark results and interpretation

Results at commit 0a978a0ac844c1df98398d58aa668f49a95bc9c3 (now the PR head). This is an interim Luna report, not a claim that the full multi-model suite has finished.

Comparison and coverage

Both arms returned all 617/617 annotated records across the seven workloads below. The only intended configuration change was BuiltinRegex.validate_matches_with_llm: false → true for email, payment card, IPv4, IPv6, MAC address and URL. detect_additional_matches remained false for those six configured labels. Other labels still used LLM validation/augmentation in both arms; “OFF” does not mean all LLM processing was disabled. Inputs and keyed GLiNER caches were held fixed; LLM calls were fresh.

The following counts measure full annotated-target character coverage by final replacement spans, irrespective of predicted label. They are not label-aware recall or precision.

Workload	Matched records	Regex validation OFF	Regex validation ON
Synthetic propagation regression	17	8/19	7/19
RAT difficulty 1	100	553/589	546/589
RAT difficulty 2	100	81/89	81/89
RAT difficulty 3	100	9/9	9/9
Nemotron PII test	100	877/921	876/921
OpenPII validation	100	586/638	589/638
Privacy-Bench	100	189/197	190/197
Total	617	2,303/2,462	2,298/2,462

There were 16 target-coverage gains and 21 losses, rather than five identical records simply becoming worse. These totals include non-regex labels; fresh stochastic validation/augmentation can change those independently of the regex flag.

Both annotated FTP URL targets had accepted regex URL coverage with validation OFF and remained covered as URLs with validation ON (2/2 in each arm). Offline checks passed for source-text identity, exact source-span consistency and actual Annotate replacement application on the paired records; there were no skipped replacement spans. These mechanical checks do not prove semantic boundary correctness.

SWE is separate and has no gold targets: the OFF arm returned 95/100, with five failed deliveries; the ON arm was still running at the reporting snapshot. SWE is excluded from the coverage table. Failed deliveries are not counted as privacy leaks.

What the contextual audit found

The annotated cohort produced the same 274 regex candidate occurrences in both arms: 150 emails, 47 card-shaped numbers, 48 URLs, 21 IPv4, four IPv6 and four MAC addresses. These are candidate counts, not unique values or necessarily distinct final spans; overlap reconciliation can suppress nested candidates.

LLM validation rejected 14 email candidates and reclassified three card-shaped candidates to account_number, national_id and monetary_amount. The bank-account correction was supported by context; the other two interpretations need care because source wording, numeric form and benchmark labels do not all agree.

A single Codex-agent source-context audit found that all 14 rejected email candidates included an underlying contact email plus adjacent text. For example, a synthetic dialogue can contain alice@example.com.Doctor:; the candidate may include .Doctor. Multilingual examples similarly concatenate contact text with surrounding prose.

Under a strict, context-interpreted email-boundary rubric, exact agreement was 125/150 OFF versus 125/136 accepted ON. However, this is not independently human-verified precision, and it is not universal correctness: separator-free text can be ambiguous, and interpreting dialogue markers or multilingual boundaries is a user-policy decision. We should not infer a general fix from dataset-specific speaker names, a domain-ending shortlist, or ASCII-only restrictions.

The important end-to-end trade-off is that rejecting the larger candidate did not reliably preserve its sensitive portion: 12 of the 14 underlying email occurrences were uncovered in the final ON output. Two were recovered elsewhere under an email_address label. This also means exact configured-label restrictions should not be read as proof that every semantically equivalent label alias is excluded.

The broader review also found a URL candidate including closing citation punctuation and card-shaped candidates representing other identifier types. Many recognized URLs were public/generic references: being a correctly recognized URL does not establish that it should be anonymized under a privacy-sensitive-only policy.

Limits and takeaway

RAT annotations are selective; unannotated predictions are not automatically false positives. Boundary judgments depend on the chosen policy, ambiguous cases remain unresolved, and this was a single-agent audit rather than independent adjudication. There is no regex-disabled baseline here, so this comparison does not establish the overall benefit of adding regex detection.

Takeaway: the optional contextual validation path can reject or reclassify format-valid candidates, but keep/drop decisions can trade coverage for narrower interpretation. Report that trade-off explicitly; do not claim universal precision improvement or add dataset-specific boundary heuristics based on this cohort alone. Raw texts, identifiers and responses remain private.

[asteier2026]

Looks good to me.