Skip to content

fix(detection): include excluded entity types in augmenter instructions #369

Open

Description

@lipikaramaswamy

Problem

Follow-up from PR #305 review C7. This behavior is pre-existing on main, not a newly introduced #305 regression.

In permissive label mode (entity_labels=None), excluded_entity_labels reaches post-generation admission but not the augmentation prompt. Admission rejects exact normalized labels; it does not establish semantic equivalence between arbitrary labels. The augmenter can therefore return a close synonym of an excluded type and have it admitted.

Synthetic example:

  • Input: She works as a nurse.
  • Configuration: permissive labels, excluded_entity_labels=["occupation"].
  • Augmenter response labels nurse as job_title.
  • The exact-label exclusion does not reject job_title, so the text can be replaced contrary to the intended exclusion.

The consequence is over-protection/unwanted replacement, not a demonstrated privacy leak.

Proposed scope

  • Pass excluded types to augmentation prompt construction and tell the model not to emit those types or close synonyms.
  • Preserve deterministic normalized-label exclusion in admission as a separate safety boundary.
  • Cover both initial and corrective augmentation prompts, including compatibility with fix(detection): augment plain source on candidate lifecycle #267's planned plain-text input.
  • Do not claim prompt wording guarantees semantic synonym enforcement. Document the difference between exact label exclusions and semantic instructions; any deterministic alias taxonomy is a separate policy/design decision.

Validation

  • A prompt test confirms exclusions are present in permissive mode.
  • Retain tests for normalized exact-label rejection and explicitly allowed labels.
  • Add a mocked workflow case showing an excluded exact label is still rejected regardless of model compliance.
  • If making a live effectiveness claim about synonyms, use a separately approved evaluation rather than asserting prompt compliance from a mocked test.

This is a scoped follow-up, separate from #305's offset mismatch, output schema tolerance, and overlap-ranking tests.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions