Skip to content

conformance(inclusion): enforce registry-anchor-v1 surrogate/key/cycle rules and validate leaf_index/entry fields (#625) - #630

Open
drk-ram wants to merge 1 commit into
agentrust-io:mainfrom
drk-ram:drk-ram-fix-conformance-inclusion-625
Open

drk-ram wants to merge 1 commit into
agentrust-io:mainfrom
drk-ram:drk-ram-fix-conformance-inclusion-625

Conversation

@drk-ram

@drk-ram drk-ram commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

What this changes

Closes #625. Current source head: b9cc53379ba8e5eee222e42f48189734497fbbf3.

conformance/src/trace_tests/inclusion.py rejects unpaired UTF-16 surrogates in values and keys, non-string dictionary keys, and circular containers while preserving shared acyclic containers. parse_proof requires exact integer indices and counts within the safe-integer range, a positive count, and an index below the count when supplied.

parse_entry rejects blank identifiers and ASCII control characters, including DEL, in producer and batch_id. It accepts internal spaces, including producer="Example Gateway 1.0"; the earlier description incorrectly said all whitespace was rejected. leaf_count must be an exact integer between 1 and 2**53 - 1.

The draft anchor-profile text documents the surrogate exclusions, integer bounds, and identifier rules. spec/trace-v0.2.md is unchanged. Existing anchor vectors keep their verdicts; malformed index/count inputs are rejected at the parsing boundary.

The latest review recommended narrowing identifier rejection to blank strings and ASCII controls. The current source and draft text implement that recommendation; this description now matches them and corrects the review-bar count from 16 to the measured 20.

Type of change

  • Editorial (typo, link fix, clarification: no normative effect)
  • Non-breaking spec change (new optional field, new platform profile, informative addition)
  • Breaking spec change (requires 14-day comment period and Project Lead sign-off)
  • Schema change
  • Example addition

Reference conformance implementation and draft companion-profile clarification. Final governance classification remains with the maintainers; the published TRACE v0.2 specification is not modified.

Spec section

spec/registry-anchor-v1.md sections 1, 4, and 5. No edit to spec/trace-v0.2.md.

Tests for a fix

tests/test_conformance_inclusion_profile_and_receipt_boundaries.py covers surrogate values/keys, non-string keys, cycles versus shared acyclic containers, index/count types and bounds, blank/control-containing identifiers, unsafe counts, and acceptance of an internal space. Its separate test_parse_proof_without_leaf_count_rejects_leaf_index_above_safe_int pins the upper index bound for proof-only receipts.

Rerun on the current head with Python 3.11.9, importing this checkout's src and conformance/src through PYTHONPATH:

  • python -m pytest tests\test_conformance_inclusion_profile_and_receipt_boundaries.py -q --tb=short -> 20 passed.
  • From conformance: python -m pytest tests\unit\test_tr_anc.py tests\test_tr_anc_inclusion.py tests\test_anchor_vectors.py -q --tb=short -> 81 passed.
  • python tools\review_bar.py --base origin/main --head HEAD -> 20 fail on behaviour, 0 fail on a missing name or file, 0 pass or skip, measured against merge base b938790. This is contribution-level failing-before evidence, not a claim of independently isolated mutation coverage for every clause.

The earlier combined root/conformance pytest command hits a tests.conftest package-name collision in this Windows environment; running the two packages separately succeeds. Hosted CI and Conformance CI on this head are successful. Maintainer approval is still required.

Changelog entry

Checklist

  • DCO sign-off on all commits (git commit -s)
  • Breaking changes marked with <!-- CHANGED: #NNN: description --> in spec text
  • Backward compatibility statement included (for breaking changes)

@drk-ram
drk-ram requested review from a team, lywinged and rajnisht7 as code owners October 25, 2026 05:29
@github-actions

github-actions Bot commented Oct 25, 2026 •

Copy link
Copy Markdown
Contributor

🟡 Contributor Check: MEDIUM

Check Result
Profile MEDIUM
Credential LOW
Overall MEDIUM

Automated check by AgenTrust Contributor Check.

@rajnisht7 rajnisht7 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the overall changes looks good, the cycle detection, iterative traversal, and boundary checks look solid, and the tests cover the important cases well.

One thing: a few of the new rules are currently stricter than the published normative text in registry-anchor-v1.md.

  • §1 does not yet explicitly prohibit surrogate code points (U+D800–U+DFFF) in strings/keys
  • §4 only requires leaf_count ≥ 1; it does not state the ≤ 2**53-1 bound (same for leaf_index).
  • §4 defines producer / batch_id simply as strings; it does not currently forbid whitespace or control characters

it will be good to update the normative text in registry-anchor-v1.md so the published requirements match what the conformance suite now enforces, once the spec and implementation agree, this looks good to land

non-blocking

  • also positive 2**53-1 boundary tests and a plain pytest.raises for the cycle can be added

@lywinged lywinged left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes. Reviewed at 19c65b2 on da69b31.

This change makes a conformance test fail, so, as CONTRIBUTING's "Before a pull request is reviewed" asks, this first review stops at what is missing. The new character check in parse_entry loops over _ENTRY_STRINGS, which includes ts: conformance/tests/test_tr_anc_subject.py sets ts to "2026-10-16 00:00:00Z", expects "ts must be an ISO-8601 UTC timestamp", and now gets "registry entry ts must be a non-empty string without whitespace or control characters". It fails here on 3.11 and 3.12, and Conformance CI failed in its full-suite step on 3.12. The validation command in the description does not run that file.

The same section asks for a test for each rejection the diff adds, failing when that rejection is switched off, or a sentence saying why no test can observe it. Three have neither: the leaf_index upper bound, the 1 <= on the receipt's leaf_count and the 0x7F check. I will review the rest once these are in.

CI at 19c65b2 also failed in the portability step, for a reason this pull request did not cause: main failed the same step at da69b31, before #624, and passes at dcca4e0.


Generated by Claude Code

@drk-ram

drk-ram commented Oct 29, 2026

Copy link
Copy Markdown
Contributor Author

Thank you @rajnisht7 and @lywinged! Addressed all points in 8f9ca2b (rebased onto origin/main at e9ab302):

  1. Scoped the whitespace/control-character check in parse_entry to producer and batch_id:
    • Left ts validation after the non-empty str check to _is_utc_timestamp(entry["ts"]) so conformance/tests/test_tr_anc_subject.py (ts = "2026-10-16 00:00:00Z") raises "registry entry ts must be an ISO-8601 UTC timestamp ending in Z, ..." and passes all tests.
  2. Isolated tests for leaf_index upper bound, 1 <= on leaf_count, and 0x7F:
    • Added test_parse_proof_without_leaf_count_rejects_leaf_index_above_safe_int (parse_proof({"leaf_index": 2**53, "audit_path": []})), which fails if <= _SAFE_INT on leaf_index is removed.
    • Pinned match="receipt leaf_count must be an integer >= 1" on leaf_count: 0 and leaf_count: -1 in test_parse_proof_and_receipt_reject_invalid_leaf_index_or_count, which fails if 1 <= on leaf_count is removed.
    • Added 0x7F ("prod\x7f1.0" and "batch\x7f1") and positive 2**53 - 1 boundary assertions to test_parse_entry_rejects_whitespace_control_chars_and_unsafe_leaf_count.
  3. Aligned spec/registry-anchor-v1.md §1, §4, and §5:
    • Updated §1 (unpaired UTF-16 surrogates), §4 (1 <= leaf_count <= 9007199254740991, non-empty producer / batch_id without whitespace or ASCII control characters U+0000 to U+001F, U+007F), and §5 (0 <= leaf_index <= 9007199254740991).

@lywinged lywinged left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Request changes. Reviewed at 8f9ca2b on e9ab302.

What the first round asked for is in. The whitespace and control-character check in parse_entry runs over producer and batch_id only, and the conformance suite passes under the default locale and under LC_ALL=C PYTHONUTF8=0 on 3.11 and 3.12, 974 passed, 2 skipped and 5 xpassed. The leaf_index upper bound, the 1 <= on the receipt's leaf_count and the 0x7F check each have tests that fail when the clause is removed.

The one thing is the normative text. The push rewrote registry-anchor-v1.md in three places, and each makes something malformed that the published text allowed and main accepts. Section 1 excludes an unpaired UTF-16 surrogate in a string or key: canonical_claim_bytes on main writes it as \ud800, this head refuses it. Sections 4 and 5 bound leaf_count and leaf_index at 9007199254740991: parse_entry on main accepts a leaf_count of 2**53, this head calls it malformed. Section 4's meaning column now says producer and batch_id carry "no whitespace or ASCII control characters": "producer": "Example Gateway 1.0" passes TR-ANC-002 on main and is a malformed registry entry here. I would keep the first two, which spell out what a UTF-8 implementation cannot represent and the range section 1 already imposes on claims, and I would narrow the third, in the code and in the text alike, to blank strings and ASCII control characters, since the published text allowed a space and a control character has no place in an identifier. Whether the whitespace rule is a breaking change under GOVERNANCE, which asks for a 30-day comment period, a backward-compatibility statement and Project Lead sign-off where the template's line says 14 days, I read as no: that process is written for spec/trace-v0.2.md and exempts drafts, and the changed cells carry no RFC 2119 keyword. The diff carries no <!-- CHANGED --> marker either way. Whichever rule section 4 ends with, the suite wants a test for it: swapping ch.isspace() for a blank check leaves both suites green, so the inner-space refusal, the one rule the control-character clause does not already cover, has none.

What I checked rather than read:

  • CI's commands, run here on 3.11 and 3.12: ruff and mypy as CI runs them, which do not reach conformance/, and the dash check clean; the suite at 5240 passed and 98 skipped on both, with the head's conformance package on the path as CI installs it; the replay experiment, the two LC_ALL=C PYTHONUTF8=0 runs, the PYTHONOPTIMIZE=2 command and the two --help calls pass. 3.13, which Conformance CI also runs, is not installed here.
  • The 27 anchor-inclusion vectors keep their verdicts here and on main through the TR-ANC check. Vectors 17 to 19, the out-of-range, negative and boolean index, now fail as a malformed receipt rather than a root mismatch; vector 20, the empty tree, fails as a malformed receipt rather than a malformed entry.
  • Each clause named in the first round, removed alone, in the suite CI runs as pytest: the leaf_index upper bound fails 2 tests, the 1 <= on leaf_count 2, the 0x7F check 2. The whole producer and batch_id check fails 5 there and 3 more in the conformance suite, and type(leaf_index) is int loosened to isinstance fails 1.
  • On main, parse_entry accepts a blank producer, an inner space, a non-breaking space or DEL in producer, and a tab or NUL in batch_id; this head refuses all six, and ts with a space gets the timestamp message on both.
  • tools/review_bar.py reports all 20 cases of the new file under tests/ failing on behaviour at the merge base; the 16 cases the diff adds under conformance/tests are outside what the tool reads. The description's validation line still says 16 for the tool.

Things I would not hold the merge for:

  • Section 1's exclusions paragraph is also rewritten by #537, differently, so whichever of the two merges second will have to resolve that paragraph.
  • A proof-only receipt, the section 5 shape, with leaf_index 1 against a one-leaf entry still reads as a root mismatch; the same receipt with leaf_count 1 is now refused as malformed, so the two shapes answer differently.

Generated by Claude Code

…e/safe-int bounds in inclusion.py (agentrust-io#625)

Extend _outside_profile in conformance/src/trace_tests/inclusion.py to
reject unpaired UTF-16 surrogates, non-string dict keys, and circular
references iteratively; bound leaf_index and leaf_count to 2**53 - 1 in
parse_proof and parse_entry; reject blank strings and ASCII control
characters (including 0x7F) in entry producer and batch_id while keeping
ts on _is_utc_timestamp; and align spec/registry-anchor-v1.md §1, §4,
and §5 with those bounds.

Closes agentrust-io#625

Signed-off-by: Raghuvar Karthik Durga <177662415+drk-ram@users.noreply.github.com>
@drk-ram
drk-ram force-pushed the drk-ram-fix-conformance-inclusion-625 branch from 8f9ca2b to b9cc533 Compare October 30, 2026 14:49
@drk-ram

drk-ram commented Oct 30, 2026

Copy link
Copy Markdown
Contributor Author

@lywinged Rebased onto origin/main (b938790) in b9cc533, narrowed the producer and batch_id rule in parse_entry and spec/registry-anchor-v1.md §4 to reject blank strings (not val.strip()) and ASCII control characters (U+0000–U+001F, U+007F) while preserving internal spaces such as "Example Gateway 1.0", added that positive control to the test suites, and updated the PR description's tools/review_bar.py count to 20.

@drk-ram

drk-ram commented Nov 8, 2026

Copy link
Copy Markdown
Contributor Author

The current head b9cc533 follows the recommended identifier rule: blank strings and ASCII controls are rejected, but internal spaces are accepted. I corrected the description and changelog proposal, and replaced the stale review-bar count with the measured 20 behavioral failures at the merge base. Validation: 20 boundary tests and 81 focused conformance tests pass. Please re-review the remaining changes request.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-review:MEDIUM Contributor check flagged MEDIUM risk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

conformance(inclusion): enforce registry-anchor-v1 surrogate/key/cycle rules and validate leaf_index/entry fields

3 participants