Skip to content

docs(analyzer): add guidance on long texts and on running without an NLP model - #2518

Draft
valbarov wants to merge 3 commits into
data-privacy-stack:mainfrom
valbarov:docs/long-texts-and-no-op-nlp-engine
Draft

valbarov wants to merge 3 commits into
data-privacy-stack:mainfrom
valbarov:docs/long-texts-and-no-op-nlp-engine

Conversation

@valbarov

Copy link
Copy Markdown
Contributor

Change Description

This is item 3 of the direction agreed in #2446, the docs that follow #2517. It changes documentation only.

Draft until #2517 merges. The branch is built on top of #2517, because the new page links to the max_length section that #2517 adds. Until then the diff also shows the two commits of #2517. Only the last commit, docs(analyzer): add guidance on long texts and on running without an NLP model, belongs to this PR (3 files under docs/ and mkdocs.yml). After #2517 merges I will rebase this branch onto main and mark it ready.

New page "Analyzing long texts" (docs/analyzer/long_texts.md, added to the Presidio Analyzer navigation):

  • Presidio does not split a text on its own; split it before calling analyze.
  • A short example with the existing CharacterBasedTextChunker.predict_with_chunking, which shifts the results to the positions of the full text and removes duplicates from the overlap.
  • The limits of splitting: the chunk size depends on the model and the memory, entities at a chunk edge, context words in another chunk, texts without spaces, and speed.
  • Pointers to max_length and the 413 response from feat(analyzer): expose spaCy max_length in the NLP engine config #2517, to the NoOpNlpEngine section, and to text_chunker for the recognizers that run their own model.

New section "Running without an NLP model" in docs/analyzer/customizing_nlp_models.md:

  • When the NoOpNlpEngine fits, and how to set it up in YAML, on the server (NLP_CONF_FILE), and in code.
  • What is lost: the entities from the NLP model (PERSON, LOCATION, NRP with the default recognizers), and the score boost from context words in the text.
  • A table of scores with the default spaCy engine and with the NoOpNlpEngine.
  • The effect on score thresholds, and two things to do before moving a deployment: check the thresholds again, and pass known context with the context parameter.

How I checked the content:

  • I ran both Python examples as written.
  • The scores in the table come from a run with the default configuration on current main.
  • For the splitting example, I compared the results of the split run with one analyze call on a 23,000-character text with 958 results. With an overlap of 100 to 200 characters, 957 or all 958 results were the same. The one that differed was a date at a chunk edge that was detected only in part. Without overlap, 4 to 5 results at chunk edges were split in two. The page states these limits.
  • I checked the server setup by starting the app with NLP_CONF_FILE=presidio_analyzer/conf/no_op.yaml.
  • I did not build the docs site locally.

Behavior change: none, documentation only.

One thing to confirm: the page recommends CharacterBasedTextChunker for splitting at the analyzer level. It was written for the NER recognizers, but it is exported from presidio_analyzer.chunkers and it works for this use. If you prefer a plain loop over chunks in the example, I can change it.

Issue reference

Part of #2446 (item 3). Depends on #2517.

Checklist

  • I have reviewed the contribution guidelines
  • I agree to follow this project's Code of Conduct
  • I confirm that I have the right to submit this contribution and that it does not knowingly contain proprietary or confidential code.
  • My code includes unit tests
  • All unit tests and lint checks pass locally
  • My PR contains documentation updates / additions if required
  • My PR description states any change to existing behavior and any possible impact on detection accuracy or security
  • If I added or changed a recognizer, it has a test that enables it in a YAML configuration and loads it through RecognizerRegistryProvider

No code is changed, so there are no new tests.

A model entry in the spaCy NLP engine configuration can now set
`max_length`. SpacyNlpEngine sets it as `nlp.max_length` after it loads
the model, so spaCy rejects a longer text (E088) before the pipeline
runs. Without the key, spaCy's default is kept.

`/analyze` now returns 413 with a clear message when spaCy rejects
a text as too long, in place of the generic 500.

Refs data-privacy-stack#2446
Build the message without data from the exception, so that no part of
the exception text reaches the client. The exception is used only to
detect spaCy's E088.
…NLP model

- New page "Analyzing long texts": split the text before calling
  `analyze`, with an example and its limits, and pointers to
  `max_length` and to the no-op NLP engine.
- New section "Running without an NLP model" on `NoOpNlpEngine`: how to
  set it up, which entities are lost, that context words in the text no
  longer raise scores, and what this means for score thresholds.

Refs data-privacy-stack#2446

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant