Skip to content

Known subjects: caller-declared identities, an egress check, receipts, report-only - #141

Merged
askalf merged 39 commits into
mainfrom
subject-protection
Oct 14, 2026
Merged

askalf merged 39 commits into
mainfrom
subject-protection

Conversation

@askalf

@askalf askalf commented Oct 13, 2026 •

Copy link
Copy Markdown
Owner

Pattern detection finds what has a shape. It cannot know that "Marisol Quintero-Vance", "Quintero-Vance, M.", "mqvance77" and "1428 Elm St" are one person. A caller that already knows who a request is about can now say so.

What it adds

  • X-Cordon-Subject (base64url JSON): up to 8 subjects with names, emails, phones, addresses, dates of birth, IDs, handles and other literals, plus an opaque ref.

  • How they're matched:

    • names in any order, with initials, as handles, and alone when capitalised (common words such as Will or Grace are skipped when alone);
    • street-suffix and direction abbreviations;
    • any separators in phones and IDs;
    • numeric and spelled-out dates;
    • accents, apostrophes and hyphens.

    Matches become <SUBJECT<n>_<KIND>_…> and win over the pattern catalog.

  • Egress check. After redaction, every string and key in the outgoing body is read again, Unicode-normalised, without zero-width characters, case-folded and separator-blind. Any identifier still present refuses the request with 422 and the upstream is never called. The refusal names the kind, never the value.

  • Media: block, mask or allow (the stricter of the policy subjectMedia and the header media wins; the policy defaults to block). Mask reads each image on the server with an offline OCR engine (tesseract.js plus the English model from node_modules, no network), paints over the words of any subject identifier or catalog entity (sharp), then reads the masked image again. It is sent only if that second reading finds nothing, after up to two rounds; otherwise the request is refused. PDFs, audio, files, and images given by URL or file id are refused in mask mode. The three libraries are optional dependencies.

  • Receipts.

    • Audit records gain a ref digest, the egress verdict and a media count.
    • X-Cordon-Receipt returns the record's hash.
    • GET /admin/receipts?ref= verifies the chain first, then returns the records for that ref.
    • Records without the new fields hash exactly as before, so older logs still verify.
  • Policy.

    • requireSubject: requests without the header get a 400.
    • subjectMedia.
    • reportOnly: forward unchanged and record the verdict, for measuring a caller before enforcing.

    All three take part in the X-Tenant strictness rule.

  • Fail closed.

    • An unreadable header is a 400, never pattern-only redaction.
    • A detection or audit failure on a subject request is refused whatever FAIL_MODE says.
    • A subject request to a mode-off tenant is refused.

Tests

  • _test_subject.mjs: 69 unit checks (header validation, every name, contact, address, date and ID form, the common-word guard, catalog coexistence, the egress check, media counting, the ref digest).

  • _test_subject_proxy.mjs: 35 integration checks against the echo stub, covering:

    • Messages, Responses, chat and streaming, with nothing identifying reaching the stub and the reply restored;
    • the media, egress, mode-off, requireSubject and bad-header refusals, with the stub never called;
    • report-only;
    • receipts, including that no value or ref is written to the audit log.
  • _test_fuzz_subject.js: five fast-check properties:

    • no identifier survives redaction, in any rendering or field;
    • an identifier slipped back in after redaction is caught, however it is disguised (spacing, case, zero-width characters, full-width digits);
    • text without identifiers is never refused;
    • the header parser and the matcher are total on byte soup.
  • Mask mode (in _test_subject_proxy.mjs): a rendered screenshot of an invented person goes through mask mode. The image the stub receives reads back as Name: ███ Email: ███ Phone: ███ ... Last login from Atlanta. PDFs and image URLs are refused, and a header can tighten mask to block.

npm test: all suites green (262 + 70 + 18 fuzz + 5 subject fuzz + 156 + 42 + 94). The subject fuzz properties also pass at 2,000 runs each.

…, report-only

X-Cordon-Subject names the people a request is about. Their names, emails, phones,
addresses, dates of birth, IDs and handles are matched in the forms people write them
(name order, initials, handles, street-suffix abbreviations, any separators, accents,
apostrophes and hyphens) and replaced with SUBJECT<n>_<KIND> placeholders ahead of the
pattern catalog. The de-identified body is then read again, separator-blind and
Unicode-normalised, and refused (422, upstream never called) if any identifier is still
there. Media in a subject request is refused unless the policy and the header both allow
it. The audit gains a digest of the caller's ref and the egress verdict, X-Cordon-Receipt
returns the record's hash, and GET /admin/receipts?ref= serves a ref's records after
verifying the chain. Tenant policy adds requireSubject, subjectMedia and reportOnly.

Tests: a unit suite (69), an integration suite against the echo stub (35) and five
fast-check properties: no identifier survives in any form or field, a disguised
identifier slipped back in is caught, text without identifiers is never refused, and
parsing and matching are total on byte soup.
@github-actions github-actions Bot added documentation Improvements or additions to documentation detection What counts as PII, and how confidently proxy The fail-closed request path dashboard Dashboard and metrics audit Audit trail tests Test suite and CI size/XL 800+ hand-written lines labels Oct 13, 2026
Comment thread src/index.ts Dismissed

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching and receipt flow need changes before merge. The egress check can miss identifiers in ordinary metadata fields and phone numbers with longer separators, allowing both to reach the upstream with a verified verdict.

1. Blocking: src/subject.ts:389

MEDIA_PAYLOAD_KEYS.has(key) && (v.startsWith("data:") || node?.type === "base64" || key === "file_data" || key === "result" || (key === "data" && isMediaPart(node)));

The payload exemption applies outside media parts. For a Responses request with metadata: {result: "Marisol Quintero-Vance"} and that name declared as a subject, metadata is not redacted and leaves() skips its result value solely because of the key. countMedia() returns zero, so the request passes as verified and sends the identifier upstream. Ordinary string values starting with data: under these keys have the same problem.

Suggested fix:

Limit payload exemptions to actual binary fields within recognized media parts, carrying parent context where needed. Add an integration test with a declared identifier in metadata.result that must be refused before forwarding.

2. Blocking: src/subject.ts:142

const digitPattern = (d: string) => d.split("").join("[^\\p{L}\\p{N}]{0,3}");

With the subject phone 4045550187, a message containing "4 0 4 5 5 5 0 1 8 7" survives subject redaction because each gap exceeds three characters. The egress check reuses this same bounded pattern, and phone matchers have no folded needle, so it also returns no violation. This forwards the phone despite the documented separator-blind check.

Suggested fix:

Make the independent numeric egress check recognize longer separator runs while retaining the intended boundaries between unrelated numbers. Add a regression covering four or more spaces between every digit and assert that the upstream is never called.

rule:none

subjectMedia and the header's media now take block, mask or allow, and the stricter wins.
In mask mode each image is read on the server (tesseract.js with the English model from
node_modules, no network), the words of any subject identifier or catalog entity are
painted over (sharp), and the masked image is read again; it is sent only when that reading
finds nothing, after up to two rounds, otherwise the request is refused. PDFs, audio, files
and images by URL or file id cannot be read here and are refused in mask mode. The three
libraries are optional dependencies; without them mask mode refuses every image.

Tests: a rendered screenshot of an invented person goes through mask mode and the image
the stub receives reads back with no identifier and the rest still legible; a PDF and an
image URL are refused; a header can tighten mask to block.

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching, media handling, receipts, policy checks and added tests were reviewed. Two egress-check gaps still let declared identifiers reach the upstream while the request is recorded as verified.

1. Blocking: src/subject.ts:400

MEDIA_PAYLOAD_KEYS.has(key) && (v.startsWith("data:") || node?.type === "base64" || key === "file_data" || key === "result" || (key === "data" && isMediaPart(node)));

The payload exemption applies to arbitrary objects, not just media. For a subject named Marisol Quintero-Vance, an Anthropic tool_use input such as {"result":"m a r i s o l q u i n t e r o v a n c e"} survives the regular matcher. countMedia returns zero, and leaves skips the result value solely because of its key. The separator-blind check therefore never sees the identifier, and the request is forwarded with a verified verdict.

Suggested fix:

Only exempt payloads within validated media structures, carrying parent context where needed. Treat result, file_data and data-URL-looking strings in ordinary tool inputs as text. Add an integration test that puts an obfuscated subject name under result and asserts refusal before any upstream call.

2. Blocking: src/subject.ts:151

const digitPattern = (d: string) => d.split("").join("[^\\p{L}\\p{N}]{0,3}");

The egress check is not separator-blind for numeric identifiers. With phone 4045550187 declared and redactSystem:false, a system prompt containing "404 555 0187" is left untouched by redaction. Both the phone matcher and digitRes reject the four-space gaps, and PHONE has no folded needle. The egress check returns no hit and sends the phone upstream, despite the documented protection for identifiers in unredacted system prompts.

Suggested fix:

Use a separate egress matcher that tolerates arbitrary runs of separators for numeric identifiers, without weakening the normal redaction matcher if its limit is intentional. Add a proxy test with four or more separators in an unredacted system prompt and require a 422 with no upstream call.

rule:none

…een digits

A string under data, file_data, result, image_url or url was skipped anywhere in the body,
so a declared name in Responses metadata.result left marked verified. Payloads are now
skipped only below a recognised media part; the same key elsewhere is read like any text.

Digits spread wider than three separators slipped past both detection and the egress
check. Detection now spans up to eight separators, and the egress reading any run of
non-letter, non-digit characters, so a phone or ID spaced out further is refused before
the upstream is called while two numbers split by a word are still not joined.

Regressions: metadata.result with a declared name is refused with the stub never called;
four spaces between digits is redacted; twelve is refused with the stub never called.

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject checks, media masking, receipts and added tests cover the main request paths, but the egress check still permits declared identifiers in numeric tool arguments and spaced-out abbreviated addresses. Both cases need regression tests before this can safely enforce subject protection.

1. Blocking: src/subject.ts:450

leaves(deidBody, texts);

The check reads serialized tool arguments as one string without interpreting their numeric literals. With phones: ["4045550187"] and a chat tool call whose function.arguments is '{"phone":4.045550187e9}', applyRedaction detects the expanded number but drops the SUBJECT1_PHONE span because numeric slots only retain NUMERIC_PII types. This check then searches the exponent spelling, which matches neither the phone regex nor its digit needle. The request is marked verified and forwards the declared phone unchanged in a valid numeric representation.

Suggested fix:

Check decoded tool-argument literals using the existing exactDecimal handling, and allow whole-value declared-subject matches to redact numeric slots. Add an integration test for exponent-form subject phones or IDs that asserts the identifier cannot reach upstream.

2. Blocking: src/subject.ts:234

return { patterns, folded: [alnum(street)] };

Only the supplied street spelling becomes a separator-blind needle. For addresses: ["1428 Elm Street"], an unredacted system prompt containing "1428 E l m S t" passes egress: the address regex requires contiguous Elm and a contiguous suffix, while the flattened text is 1428elmst rather than 1428elmstreet. The declared address reaches upstream with a verified receipt, despite the advertised abbreviation and separator handling.

Suggested fix:

Apply the separator-blind egress reading to the supported address variants as well as the original spelling. Add a test combining an abbreviated suffix with spacing inside the street name in a field redaction leaves untouched, and assert refusal before forwarding.

rule:none

…able matchers

A declared phone sent as a JSON number in exponent form passed both checks: a numeric slot
kept only checksum types, and the egress reading saw 4.045550187e9. A numeric leaf that is
wholly a declared subject identifier is now redacted, and the egress check also reads each
number literal in a string at its exact decimal value.

Only the supplied street spelling was a separator-blind needle, so "1428 E l m S t" passed
for "1428 Elm Street". Every spelling the address patterns accept (suffix and direction
abbreviations) is now a needle.

The egress check's test() left a global regex's lastIndex past a hit, so detection run
later with the same matchers (the image re-read rounds) could start mid-text. lastIndex is
reset before every matchAll and after every test.

Regressions: exponent and decimal-point phones in tool arguments (unit and through the
proxy, never reaching upstream), a spaced abbreviated address in an unredacted system
prompt refused before forwarding, and detection after an egress hit.
@github-actions github-actions Bot added the redaction Redaction, pseudonyms, reidentification label Oct 13, 2026
Comment thread src/subject.ts Fixed

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching, egress checks, media handling and receipt paths need changes before merging. There are paths that forward declared identifiers unchecked, and single-token common names bypass the intended common-word guard.

1. Blocking: src/proxy.ts:69

return onRedactionError(r, res, e, r.subject ? "closed" : failMode, t0);

The subject-specific fail-closed override only covers the detection catch. With TENANT_SECRET unset, a tenant configured through /admin/tenant with consistentPseudonyms: true and failMode: "open" reaches the earlier pseudonym-secret guard, which passes failMode to onRedactionError. A request carrying X-Cordon-Subject then forwards its original body upstream before subject compilation or the egress check.

Suggested fix:

Apply the subject fail-closed override to the pseudonym-secret guard as well, or enforce it centrally in onRedactionError. Add an integration test for a subject request to this tenant and assert that the upstream receives no call.

2. Blocking: src/subject.ts:446

const media = inMedia || isMediaPart(node);

Media classification depends only on an arbitrary object's type field, not its position in the provider schema. With subjectMedia: "allow", an Anthropic tool_use input such as {"type":"image","data":"m a r i s o l q u i n t e r o v a n c e"} is ordinary tool data, but this walk treats it as media and skips data. Detection does not match the letter-spaced name, so the request is marked verified and sends the declared identifier upstream. Allowing actual images should not exempt tool input strings from the egress check.

Suggested fix:

Restrict payload exemptions to actual media parts in provider content locations, rather than any object with a matching type. Add a regression test with a media-shaped tool input and a spaced-out identifier.

3. Blocking: src/subject.ts:203

patterns.add(tokens.map(tok).join(S)); // as given

For a declared single-token name such as "Will", this adds the unbounded case-insensitive pattern /Will/ before the common-word guard. It redacts "will" in "we will help" and even the substring in "willing". If the same text appears in an unredacted system prompt, the egress check refuses the request instead. The later COMMON check only controls the additional exact patterns and cannot prevent either result.

Suggested fix:

Handle single-token names separately: apply the common-word, minimum-length, capitalisation and boundary checks before adding any pattern or folded needle. Add tests declaring "Will" alone, rather than only "Will Grace", and cover both redaction and egress.

rule:none

@askalf

askalf commented Oct 13, 2026

Copy link
Copy Markdown
Owner Author

Fix at a6a91ef

  • [1] The fail-open path could forward a subject request raw. In src/proxy.ts, onRedactionError now refuses with 422 whenever the request carries X-Cordon-Subject, even when failMode is open. The check sits in the one function that can forward a raw body, so it covers every stage that calls it: the pseudonym-secret guard, detection and audit. The override that was only in the detection catch is no longer needed and has been removed. _test_subject_proxy.mjs adds a subject request against the secret-less instance (:8811). That request goes to a tenant with consistentPseudonyms: true, failMode: "open", and the test asserts a 422 with no call to the stub.

  • [2] Any object with a media type field was treated as media. In src/subject.ts, a new mediaPartsOf finds media parts only where the provider schemas put them:

    • message content parts;
    • the content of tool_result and function_call_output;
    • Responses input items and their content.

    countMedia and the egress walk both use it. An object with the same shape inside tool_use input, function arguments or metadata is now read in full, and its data field is no longer skipped. New unit tests in _test_subject.mjs:

    • a media-shaped tool input with a spaced-out name is caught and is not counted as media;
    • an image inside a tool result is still counted, and its data is still skipped.

    A new integration test in _test_subject_proxy.mjs sends that tool input under subjectMedia: "allow" with header media: "allow". The request is refused and never reaches the stub.

  • [3] A one-word name added an unbounded pattern. In src/subject.ts, nameForms now adds the case-insensitive "as given" pattern only for names of two or more tokens. A one-word name is matched only by the word-bounded, capitalised exact form, which already applies the minimum length and the common-word check. Its separator-blind egress needle is now also dropped for common words. compileSubjects keeps a name that has only exact forms. New unit tests in _test_subject.mjs:

    • declaring "Will" alone leaves "We will help. Will you check? He is willing." untouched, and the same text in a system prompt is not an egress hit;
    • declaring "Okonkwo" alone redacts the capitalised word but not "okonkwoish".

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/proxy.ts, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject checks cover the main request paths, but two unchecked representations can still forward declared identifiers: images inside content-source documents and large numeric identifiers outside redacted fields. Both need regression coverage before merging.

1. Blocking: src/subject.ts:423

  else if (p?.type === "tool_result" || p?.type === "function_call_output") {
    parts(p.content);
    parts(p.output);
  }

A Messages document with source.type === "content" can contain image blocks in source.content, but this traversal never visits them. With such a document as the only media, countMedia returns zero, so both block and mask policies skip enforcement. Redaction leaves the image bytes intact, and egress scans their base64 spelling rather than the pixels. An image displaying a declared name can therefore go upstream unchanged with a verified receipt.

Suggested fix:

Traverse document content sources when locating media, and share those schema-aware locations with masking. Add integration cases for an image inside a content-source document under both block and mask policies, checking that it is refused or masked before forwarding.

2. Blocking: src/subject.ts:468

if (typeof node === "number" || typeof node === "bigint") {
out.push(String(node));
return;
}

Numeric leaves are not expanded to exact decimal form. For a declared ID "1000000000000000000000", a Responses body containing metadata.account: 1e21 leaves that field outside redaction. This branch reads it as "1e+21"; neither the ID matcher nor its digit needle matches, so egress reports verified and forwards the identifier. The exactDecimal handling above only applies to string leaves.

Suggested fix:

Apply exactDecimal to numeric leaves as well as number literals inside strings. Add a regression using an exponent-form numeric identifier in an unredacted body field and assert that egress refuses it before any upstream call.

rule:none

@askalf

askalf commented Oct 13, 2026

Copy link
Copy Markdown
Owner Author

Fix at 66593d8

  • [1] Images inside content-source documents (src/subject.ts, mediaPartsOf): the media walk now goes into source.content of a document whose source type is content, the same way it already goes into tool results. An image there now counts as media. Under block the request is refused (422, stage media). Under mask the existing maskMedia path runs, and its walk in src/media/mask.ts already reaches nested images, so the image is masked and read again before it is sent. The egress check skips that image's base64 payload, as it does for every other recognised media part, so the pixels are judged by the media policy and not by their base64 text.
    • Unit test in _test_subject.mjs: such an image is counted and its data is skipped.
    • Integration tests in _test_subject_proxy.mjs: under the default block policy the request is refused and the stub is never called. Under a mask tenant the rendered screenshot inside the document reaches the stub masked, and reading it back shows no identifier.
  • [2] Exponent-form numeric leaves (src/subject.ts, leaves): numeric leaves now go through exactDecimal too, so 1e21 is also read as 1000000000000000000000, the form the ID matcher and its digit needle expect.
    • Unit test in _test_subject.mjs: a declared ID sent as metadata.account: 1e21 is reported as SUBJECT1_ID.
    • Integration test in _test_subject_proxy.mjs: the same body on /v1/responses is refused with 422 and the stub is never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching, egress checks, receipts and tests cover several important request paths, but media traversal still has two correctness gaps. Images in MCP tool results can bypass blocking and masking, while ordinary tool data can be mistaken for media during masking.

1. Blocking: src/subject.ts:424

  else if (p?.type === "tool_result" || p?.type === "function_call_output") {

An Anthropic message containing an mcp_tool_result whose content contains a base64 image is not traversed here. With that image as the only media, countMedia returns zero, so both block and mask policies skip media enforcement. Text redaction leaves the image intact, and the egress scan reads its encoded bytes rather than its pixels. A screenshot of a declared person therefore reaches upstream with a verified verdict. The existing request text walker already recognises mcp_tool_result as a content container.

Suggested fix:

Traverse MCP tool result content when discovering media. Add integration cases with an image inside `mcp_tool_result` that prove block mode never calls upstream and mask mode processes or refuses the image.

2. Blocking: src/media/mask.ts:208

for (const v of Object.values(node)) mediaParts(v, out, depth + 1);

The masking walker recursively interprets every object as provider media, unlike countMedia's schema-aware traversal. For a mask-mode request containing a real image plus ordinary tool input such as { "type": "file", "name": "notes" }, countMedia triggers masking for the real image, but this walk also visits the tool input and returns a MediaRefusal for it. The request is rejected even though the tool object is ordinary data, not a file attachment. Image-shaped tool data can likewise be decoded and rewritten.

Suggested fix:

Share the schema-aware media discovery between counting, egress and masking, while retaining replacement access to actual image parts. Add a mask-mode test combining a real image with media-shaped tool input and verify that the tool data is neither refused as media nor rewritten.

rule:none

@askalf

askalf commented Oct 13, 2026

Copy link
Copy Markdown
Owner Author

Fix at 016296f

  • [1] Images in mcp_tool_result were not seen as media. In src/subject.ts, mediaPartsOf now looks inside mcp_tool_result content the same way it looks inside tool_result content. This matches the request text walker. An image there is now counted, so block mode refuses the request, mask mode masks the image, and the egress check skips only its base64 payload. New tests:
    • _test_subject.mjs: such an image is counted, and mask discovery finds it.
    • _test_subject_proxy.mjs: block mode refuses it (422) without calling upstream, and mask mode sends a masked image that reads back with no identifier.
  • [2] The mask walker took ordinary tool data for media. mediaPartsOf is now exported. mediaParts in src/media/mask.ts now uses it to find parts, then turns each part into an image it can replace or a refusal. It no longer walks every object in the body. Counting, the egress check and masking now find the same parts, so a media-shaped tool_use input is neither refused nor decoded and rewritten. New tests:
    • _test_subject.mjs: a body with a real image and a tool input containing { "type": "file", "name": "notes" } and an image-shaped object yields only the real image.
    • _test_subject_proxy.mjs: the same request in mask mode returns 200, and the tool input reaches upstream byte for byte.

One earlier full run had a single failure in a _test_fuzz_subject.js property about text redaction in a body with no media parts. This change does not touch that path. The failure did not recur over several thousand further runs or in the final suite run.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/media/mask.ts, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching, media discovery, receipts and proxy enforcement paths are covered by substantial tests, but the egress check still lets a declared date of birth through in an ordinary spaced numeric form. That gap needs a regression test before this can merge.

1. Blocking: src/subject.ts:268

return { patterns, digits: [${y}${mm}${dd}, ${mm}${dd}${y}] };

For a subject with dates: ["1986-07-14"], send a request to a tenant with redactSystem: false and system: "DOB: 7 / 14 / 1986". The date regexes require each slash to touch its digits, so none matches. The separator-blind digit needles require 19860714 or 07141986, neither of which matches the unpadded 7141986. There are no folded needles for dates. The egress check therefore reports verified and forwards the date unchanged, despite accepting the equivalent unspaced 7/14/1986 as this subject's date.

Suggested fix:

Include the supported unpadded and short-year numeric date forms in the separator-blind egress matchers, or apply separator-tolerant matching to all supported numeric date forms. Add a unit check and a proxy regression using an unredacted system prompt, asserting 422 and no upstream call.

rule:none

@askalf

askalf commented Oct 13, 2026

Copy link
Copy Markdown
Owner Author

Fix at fe2d24a

[1] spaced numeric date of birth passed the egress check (src/subject.ts)

  • The fix: dateForms now also returns egress-only matchers for the separated numeric forms. These are year-month-day and month-day-year, padded or not, with a four- or two-digit year.
  • How they match: at least one separator is required between each part, but any run of separators is accepted. So 7 / 14 / 1986, 7 - 14 - 86 and 1986 / 7 / 14 all match.
  • What they don't match: they don't run into neighbouring digits, and a bare short number like 71486 is not read as the date.
  • Where they're used: compileSubjects adds them to the matcher's digitRes, which the egress check already tests on the folded text. A date in one of these forms in an unredacted field is now reported as SUBJECT<n>_DOB and the request is refused. Detection patterns are unchanged.

Tests:

  • _test_subject.mjs checks that five spaced forms are caught, including DOB: 7 / 14 / 1986. It also checks that spaced dates one day or one year off are not a hit.
  • _test_subject_proxy.mjs sends system: "DOB: 7 / 14 / 1986" to the redactSystem: false tenant with a subject declaring 1986-07-14. It asserts a 422 naming SUBJECT1_DOB and that the stub was never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@askalf

askalf commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Fix at bbdf539

  • [1] Object keys skipped number-literal expansion (src/subject.ts): String leaves expanded number literals and object keys did not. That logic is now in one helper, pushText. It pushes the text as written, then the exact decimal value of every fraction or exponent literal in the text and in its folded spelling (full-width and zero-width-split). Both string leaves and caller-defined keys in leaves now go through it. A metadata key such as "1e40", "1e40" or "1\u200be\u200d40" is therefore read as the declared 41-digit ID, and the request is refused just as when the same value is in a metadata value. Protocol keys are still skipped as before.
  • Regressions:
    • _test_subject.mjs adds unit checks for exponent-form metadata keys: 1e40, 1e+40, full-width, and split by zero-width characters. A check that 1e39 and a full-width 1e41 as keys are not hits guards against false blocks.
    • _test_subject_proxy.mjs sends a Responses request with input: "hello" and each of these keys under metadata. Each one must get a 422, and the upstream stub must never be called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject checks, media handling, receipts and tests cover many representations, but the date matcher still allows an accepted date identifier to bypass the egress check when its letters are spaced out.

1. Blocking: src/subject.ts:309

else return { patterns: [loose(date)], digits: [], egress: [] };

A header with dates: ["July 14, 1986"] is accepted, but this branch supplies only a literal matcher and no separator-blind egress matcher. In a Responses request with input: "hello" and metadata.note: "J u l y 1 4 1 9 8 6", redaction leaves metadata untouched. Neither the literal matcher nor its normalised readings match the spaced month, and the empty digit and egress lists leave no further check. The request is marked verified and the declared date reaches upstream.

Suggested fix:

Give fallback date literals a separator-blind egress matcher, or reject unsupported date declarations rather than accepting them with weaker protection. Add unit and proxy tests using a spelled-out date declaration and its spaced-out form in an unredacted field, asserting refusal before upstream is called.

rule:none

@askalf

askalf commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Fix at ced33fb

  • [1] src/subject.ts dateForms: A spelled-out declared date is now parsed into year, month and day. This covers "July 14, 1986", "Jul. 14th 1986", "14 July 1986" and "14th of July, 1986". These dates now get the same numeric patterns, digit needles and separator-blind egress readings that an ISO date already had. So with dates: ["July 14, 1986"], the text "J u l y 1 4 1 9 8 6" in an unredacted field is refused. Numeric forms such as "7 / 1 4 / 1 9 8 6" are also caught. A declaration in any other form is still matched as written, and now also has an egress reading: word-bounded, with any separators between its letters and digits. No accepted date is left with only the literal matcher.
  • Tests:
    • _test_subject.mjs adds unit checks for three spelled-out declarations. Each one is caught spaced out and in numeric form, and a different date is not a hit. It also checks a declaration in an unparsed form.
    • _test_subject_proxy.mjs adds the case from the review: a Responses request with input: "hello", metadata.note: "J u l y 1 4 1 9 8 6" and dates: ["July 14, 1986"]. It asserts a 422 naming SUBJECT1_DOB, and that the stub was never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. Changes are needed because the egress check misses a declared phone number when its leading country code is attached and the remaining digits are widely spaced. The review covered subject matching, outbound checks, media handling, receipts, policy enforcement and the added tests.

1. Blocking: src/subject.ts:460

      ...digits.map((d) => new RegExp(`(?<!\\d)${egressDigitPattern(d)}(?!\\d)`, "u")),

Declare phones: ["404-555-0187"] and send a Responses request with metadata.note containing "1404" followed by each digit of "5550187" separated by twelve spaces. Metadata is not redacted. The detection regex cannot span twelve spaces, and this egress regex cannot start at the first 4 because it is preceded by the country-code digit 1. Phone matchers have no folded needle to catch it either, so the request is marked verified and the declared phone reaches upstream. This also affects numbers declared with +1, since compilation removes that prefix.

Suggested fix:

Give the phone-specific egress matcher the same optional leading country code as detection, while retaining unlimited separators and number boundaries. Add unit and proxy regression tests with an attached leading 1 and separator runs longer than detection accepts, asserting refusal before upstream is called.

rule:none

@askalf

askalf commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Fix at d21f6d8

  • [1] Phone egress missed an attached leading 1 with wide separators (src/subject.ts, PHONE case): phones now get their own egress matcher. Like detection, it accepts an optional leading country code 1 before the declared digits. Unlike detection, it allows any run of separators, both after the 1 and between digits, and keeps the number boundaries on both sides. A note like 1404 followed by 5550187 with each digit twelve spaces apart is now refused. This also covers numbers declared with +1, since that prefix is removed when the phone is compiled. A longer number that only contains the phone, such as 2140455501870, is still not a hit.
  • Tests:
    • _test_subject.mjs has three new unit checks: the finding's string for a phone declared as 404-555-0187, and the same string, with and without a leading +, for a phone declared as +1 404 555 0187. The two refusal checks fail with the new matcher removed and pass with it. A third check confirms a longer number around the phone is not a hit.
    • _test_subject_proxy.mjs sends the same string in Responses metadata.note (which redaction does not touch) for both declarations. It asserts a 422 and that the upstream stub was never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching, egress walk, media handling, receipts and added tests were reviewed. A short initial-and-surname form can still reach upstream unchanged while the egress check records it as verified.

1. Blocking: src/subject.ts:254

  } else for (const p of egressSpread(h)) egress.add(p);

For a subject declared as "Will Li", send "W . L i" in message text. The initial-and-surname detector requires the surname's letters to be contiguous, so it leaves this text unchanged. Here the initial form is "WLi", but egressSpread defaults to a minimum of four characters and returns no pattern. "Will" is excluded as a common word, "Li" is below the single-token minimum, and the full-name needles do not match "wli". The egress check therefore returns no violation and forwards the identifier with a verified verdict.

Suggested fix:

Include word-bounded separator-blind patterns for supported initial-and-surname forms shorter than four characters, without changing the common-word guard for standalone names. Add unit and proxy regressions for "Will Li" written as "W . L i" that assert refusal and no upstream call.

rule:none

@askalf

askalf commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Fix at 5e2c6c6

  • [1] src/subject.ts:254: Some name forms are too short to be a needle: initial and surname, surname and initial, and the middle-initial concatenations. The egress check now builds a word-bounded, separator-blind pattern for these forms however short they are, using egressSpread(h, 1) instead of the default four-character minimum. For "Will Li", "W . L i" and "L-i W" now refuse the request. The forms keep the word bounds that egressSpread already adds, so a longer word that contains them ("wlis", "awli") is still not a hit. The common-word guard for standalone tokens is unchanged, because those tokens still go through the separate t.length >= 3 && !COMMON.has(...) path.
  • Tests: _test_subject.mjs adds unit checks for "Will Li". Detection leaves "W . L i" unchanged and egress reports SUBJECT1_NAME. A surname-first "L-i W" is caught, and longer words that contain the form are not hits. _test_subject_proxy.mjs adds a proxy check that "W . L i" under a "Will Li" subject gets a 422 and the upstream stub is never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching, egress walk, media handling, receipts and added tests cover many request forms, but address matching still lets a declared street address through when the declaration includes an apartment without a comma.

1. Blocking: src/subject.ts:285

const street = addr.split(/,|\n/)[0].trim();

Declare addresses: ["1428 Elm Street Apt 4"] and send a Responses request with input: "hello" and metadata.note: "1428 Elm St". The street tokens retain "Apt 4", so every abbreviated pattern and folded spelling requires that unit too. The metadata is not redacted, and the egress check finds neither the full declaration nor a spelling containing the required unit. It marks the request verified and forwards the identifying street address. This also contradicts the nearby claim that unit numbers are covered by the street line.

Suggested fix:

Separate apartment/unit designators from the street portion even when no comma precedes them, and compile street-only forms alongside the complete address. Add a unit test and a proxy refusal test for a unit-bearing declaration with an unredacted street-only spelling.

rule:none

@askalf

askalf commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Fix at 4624ab0

  • [1] A unit declared without a comma kept the street from matching alone (src/subject.ts, addressForms). The street line now ends at the first unit word, even when no comma comes before it. The unit words are Apt, Apartment, Unit, Suite, Ste, Room, Fl, Bldg, Lot and similar, plus any word starting with #, and only from the third word on. Before this change, every abbreviated pattern and every egress needle required the unit. Now they are built from the street alone, so 1428 Elm St is redacted and the egress check refuses it when redaction cannot reach it. A second pattern with the unit is added when one is declared, so 1428 Elm St Apt 4 is still replaced in one piece. That makes the existing note that unit numbers are covered by the street line true again.
  • Tests:
    • _test_subject.mjs: four checks for 1428 Elm Street Apt 4:
      • the street alone is redacted in both spellings;
      • the abbreviated form with its unit is redacted whole;
      • 1428 Elm St in metadata is an egress hit;
      • a different house number with the same unit is not a hit.
    • _test_subject_proxy.mjs: a Responses request with input: "hello" and metadata.note: "1428 Elm St" under that declaration is refused with 422, and the upstream is never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching, egress walk, masking, receipts and regression tests cover many text and content-part paths. Media discovery still misses Chat Completions assistant audio references, allowing subject requests to bypass both block and mask policies.

1. Blocking: src/subject.ts:551

for (const m of Array.isArray(body.messages) ? body.messages : []) parts(m?.content);

Chat Completions can replay an assistant audio response through a message such as {"role":"assistant","content":null,"audio":{"id":"audio_123"}}. This walk only inspects content, so that request has a media count of zero. If the referenced audio contains a declared person's identifiers, neither block nor mask runs, and the egress walk only sees the opaque audio ID. The upstream receives and resolves the audio reference despite the subject media policy.

Suggested fix:

Include message-level assistant audio references in schema-aware media discovery. Count them as media and refuse them in mask mode because their bytes cannot be read locally. Add integration cases for both block and mask that assert the upstream is not called.

rule:none

@askalf

askalf commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Fix at 5a1f31e

  • [1] Chat Completions assistant audio references (src/subject.ts). mediaPartsOf now picks up a message's audio object as well as its content parts. A message like {"role":"assistant","content":null,"audio":{"id":"audio_123"}} now counts as media in a subject request:

    • block (the default): the request is refused with 422 at stage media.
    • mask: mediaPart in src/media/mask.ts returns a MediaRefusal for the reference, because its bytes cannot be read on this server, so the request is refused rather than forwarded.

    The egress walk uses the same set of media parts. It treats the reference as a media part and still reads the id, which is not a payload key.

  • Tests:

    • _test_subject.mjs checks that a replayed audio reference is counted as media and is refused by the mask path.
    • _test_subject_proxy.mjs sends a body with that message to /v1/chat/completions under block and under mask. Each case asserts a 422 and that the upstream stub is never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/media/mask.ts, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching, egress walk, media handling, receipts and added tests were reviewed. Declared identifiers need normalisation before compilation: an accepted phone written with full-width digits currently receives no protection.

1. Blocking: src/subject.ts:408

        let d = value.replace(/\D/g, "");

A header containing phones: ["4045550187"] passes validation, but this expression removes every digit. The length check then skips the identifier entirely. A request containing either "4045550187" or "4045550187" in unredacted Responses metadata consequently passes egress as verified and reaches upstream. Normalising outgoing text cannot help because no phone matcher was compiled.

Suggested fix:

Normalise declared phone values with NFKC before extracting digits, and apply the same principle to other identifier compilation paths that discard non-ASCII characters. Add unit and proxy tests with a full-width declared phone and both ASCII and full-width outgoing spellings in metadata, asserting refusal before upstream is called.

rule:none

@askalf

askalf commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Fix at d825586

  • [1] Full-width declared phone left unprotected (src/subject.ts): compileSubjects now applies NFKC to every declared identifier and drops zero-width characters before compiling it, using a new declared helper. A phone declared as 4045550187 therefore compiles from 4045550187 and gets a matcher instead of failing the length check. The same step covers the other compilation paths that discard non-ASCII characters: ID extraction ([^A-Za-z0-9]), the digit count used for exponent reading, and handles and other literals. When the declared value differs from its normalised form, it is also matched as written, so a full-width spelling in the request is redacted in place rather than only refused at egress.
  • Tests: _test_subject.mjs checks that a phone declared in full-width digits is caught at egress when written in ASCII (4045550187, 404-555-0187) and in full-width, and that both spellings are redacted. It also checks that an ID declared in full-width digits is caught when written in ASCII. _test_subject_proxy.mjs sends that phone with 4045550187 and 4045550187 each in Responses metadata and asserts a 422 with the upstream stub never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject checks, media handling, receipts and regression tests cover many identifier forms, but the ID compiler still silently changes non-ASCII identifiers and can forward them unchanged with a verified verdict.

1. Blocking: src/subject.ts:440

        const d = value.replace(/[^A-Za-z0-9]/g, "");

A valid declaration such as ids: ["ABé1234"] compiles as AB1234. If Responses metadata.note contains "ABé1234", redaction leaves that field alone. Neither the detection pattern nor the egress pattern matches the declared value, and the folded needle ab1234 is absent from the unaccented text abe1234. The request therefore receives egress=verified and sends the identifier upstream unchanged. NFKC does not remove this accented letter.

Suggested fix:

Preserve Unicode letters and numbers when compiling IDs, with consistent accent handling between declarations and outbound text, or explicitly reject unsupported ID characters rather than silently deleting them. Add unit and proxy tests for this declaration in an unredacted metadata field, asserting refusal before any upstream call.

rule:none

@askalf

askalf commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Fix at d0b2757

  • [1] src/subject.ts, ID compiler. IDs now keep Unicode letters and numbers (\p{L}\p{N}) instead of only A-Za-z0-9, so ABé1234 is no longer compiled as AB1234.
    • Detection matches the ID as declared. It also adds an unaccented form, so ABe1234 is redacted as well.
    • The egress check now builds its pattern with the same egressSpread helper as handles and other literals. That helper folds case and removes accents, and the check also reads outgoing text with accents removed. So ABé1234, abe1234 and spread-out forms such as A B É 1 2 3 4 are all caught, while AB1234 is not.
    • The long-ID needle comes from alnum(d), which handles accents the same way as the outgoing text.
    • IDs that are all ASCII compile exactly as before.
  • Tests.
    • _test_subject.mjs adds unit checks: the declared and unaccented spellings are redacted; egress catches the ID in Responses metadata.note (accented, unaccented and spread out); AB1234 is not flagged; a short non-ASCII ID is caught when spread out.
    • _test_subject_proxy.mjs sends ABé1234 and abe1234 in Responses metadata.note, which redaction does not touch, and checks for a 422 with the upstream never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: request changes. The subject matching, egress walk, media handling, receipts and regression tests cover many request forms, but the egress check still misses separator variants of accepted short addresses.

1. Blocking: src/subject.ts:429

        egress = f.folded.filter((x) => x.length < 8).flatMap((x) => egressSpread(x));

For a declared address "1 A St", addressForms produces the abbreviated spelling "1ast" and the expanded spelling, so those are covered. However, an accepted address "1 A" produces only "1a": it is excluded from folded at line 426 and egressSpread drops it because its default minimum is four characters. A Responses request with input "hello" and metadata.note "1 / A" therefore passes unchanged: metadata is not redacted, neither address detection pattern accepts the slash, and there is no separator-blind egress matcher. The request is marked verified despite carrying the declared address with separators changed.

Suggested fix:

Compile the short-address egress forms with a minimum that covers every accepted address, or explicitly reject addresses that cannot receive that protection. Add a unit check and an integration test for this short-address case, asserting refusal before upstream forwarding.

rule:none

@askalf

askalf commented Oct 14, 2026

Copy link
Copy Markdown
Owner Author

Fix at f6d4738

  • [1] short addresses at egress (src/subject.ts, ADDRESS case in compileSubjects): the address spellings too short to be needles were compiled with egressSpread's default minimum of four characters. That meant an accepted address such as "1 A" (folded to 1a) got no egress form at all. These spellings now use a minimum of one character, as handles, IDs and email local parts already do. Every accepted address therefore has a word-bounded, separator-blind egress pattern, and "1 / A" in an unredacted field is refused.
  • Tests:
    • _test_subject.mjs: a unit check that "1 / A", "1a" and "1-a" in Responses metadata are caught for a declared "1 A".
    • _test_subject.mjs: a check that "1 / A2", "2 / 1 / B" and "1 / Ab" are not hits.
    • _test_subject_proxy.mjs: an integration test that a Responses request with metadata.note "1 / A" gets a 422 and the upstream is never called.

Files: _test_subject.mjs, _test_subject_proxy.mjs, src/subject.ts

Tests: npm test exited 0 (5 pass, 0 fail)

@sprayberry-redline sprayberry-redline left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict: approve. The subject matchers feed redaction and a separate outbound check, with refusals before forwarding and receipt fields included in the audit hash. The added tests cover identifier variants, provider request shapes, media handling, report-only forwarding and receipt lookup; the reference header schema needs a small correction.

Minor:

  • docs/reference.md:75: The header schema omits "mask", although parseSubjectHeader accepts it and the policy documentation below describes it. Readers following this schema cannot tell that the header can tighten an allow policy to mask.

@askalf
askalf merged commit f46b7b7 into main Oct 14, 2026
12 of 13 checks passed
@askalf
askalf deleted the subject-protection branch October 14, 2026 14:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

audit Audit trail dashboard Dashboard and metrics detection What counts as PII, and how confidently documentation Improvements or additions to documentation proxy The fail-closed request path redaction Redaction, pseudonyms, reidentification size/XL 800+ hand-written lines tests Test suite and CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants