Skip to content

eTLD+1 binding does not IDNA-normalize, so a U-label agent URL fails to bind to an A-label brand domain #988

Closed

Description

@KonstantinMirin

Summary

adcp.signing.etld.host_from applies no IDNA conversion, so same_registrable_domain — the eTLD+1 gate the brand-authorization binding runs on — returns False when the two sides spell the same IDN host differently. A brand that publishes its brand.json under a U-label host and lists its agent under the A-label form (or the reverse) fails the binding.

Separately, the same function is asymmetric with itself on the trailing FQDN-root dot, and its docstring describes behaviour that only one of its two branches has.

Cause

src/adcp/signing/etld.py:73-82:

if "://" in value:
    parts = urlsplit(value)
    host = parts.hostname
    if not host:
        raise ValueError(f"URL has no host: {value!r}")
    return host.lower()                              # :78 — no dot strip, no IDNA
stripped = value.strip().rstrip(".").lower()         # :79 — dot stripped, still no IDNA

Neither branch performs UTS-46 / IDNA-2008 encoding, and tldextract does not do it either — it accepts a U-label and returns a U-label eTLD+1. So the two spellings never reduce to a common form.

The docstring at :63-64 claims the function "trims a single trailing dot (the FQDN root separator) so Example.COM. and example.com compare equal". The bare-host branch does that; the URL branch does not. The bare-host branch also uses rstrip("."), which strips all trailing dots, not one.

Reproduce

from adcp.signing.etld import host_from, registrable_domain, same_registrable_domain

# IDNA gap — the load-bearing one
registrable_domain("straße.de")            # 'straße.de'
registrable_domain("xn--strae-oqa.de")     # 'xn--strae-oqa.de'
same_registrable_domain("https://shop.straße.de/", "xn--strae-oqa.de")
# False   <- same host, two spellings, binding fails

# Trailing-dot asymmetry
host_from("https://Example.COM./")   # 'example.com.'
host_from("Example.COM.")            # 'example.com'
host_from("https://x.example.com..") # 'x.example.com..'
host_from("x.example.com..")         # 'x.example.com'

Impact

The IDNA gap is a real binding failure, fail-closed. src/adcp/signing/brand_authz.py:255 calls same_registrable_domain(agent_url, brand_host) as step 2a of the eTLD+1 binding. For brand_domain="bücher.example" and a brand.json listing https://ads.xn--bcher-kva.example/, the agent is found by _find_listed_agents (byte-equal URL match) but step 2a returns False, the authorized_operators[] fallback does not match either, and the result is BrandAuthorizationResult(False, reason="binding_failed") (brand_authz.py:278-283). A legitimate agent is refused; nothing is wrongly accepted. Scope is limited to brands on IDN hosts, which is a small population — but the failure is silent from the publisher's side and binding_failed points at the wrong thing.

The trailing-dot asymmetry does not currently affect the binding. tldextract strips trailing dots itself, so registrable_domain and same_registrable_domain return identical results for both spellings — verified above (registrable_domain("https://example.com.") and registrable_domain("example.com") both give example.com). The defect is observable only through host_from, which is a public export (src/adcp/signing/__init__.py:195,430), and through the docstring, which is wrong. Worth fixing as correctness and as a precondition for the IDNA change; not worth describing as a security issue.

The workaround already in the tree

src/adcp/adagents.py:378-383 does exactly the conversion host_from is missing, by hand, at its own call site:

def _idna_ascii_host(hostname: str, context: str) -> str:
    try:
        return idna.encode(hostname.rstrip("."), uts46=True).decode("ascii").lower()
    except idna.IDNAError as e:
        raise AdagentsValidationError(...) from e

and applies it to both arguments before calling same_registrable_domain (adagents.py:402-404). That is the redirect path working around this gap locally. It is also a hand-retyped copy of _idna_canonicalize.canonicalize_host steps 1/2/4 and diverges from it in three ways: no leading/trailing .strip(), transitional= left to the idna>=3 default rather than pinned the way the helper pins it, and no IP-literal short-circuit (so a public IPv6-literal redirect target raises "invalid IDNA hostname" rather than passing through). None of the three is reachable today — _normalize_domain and _check_safe_host run upstream of both call sites — so this is a maintenance duplicate rather than a live bug. It disappears entirely if host_from does the conversion.

Suggested direction

Route both branches of host_from through _idna_canonicalize.canonicalize_host, which already strips exactly one trailing dot, lowercases, short-circuits IP literals, and encodes with uts46=True, transitional_processing=False. canonical_formats/references.py:38 establishes that a non-signing module may import that helper, so adagents.py can drop _idna_ascii_host in the same change.

Two decisions the change has to make explicitly rather than incidentally:

  1. registrable_domain() return values change for IDN inputs. It is public API; today straße.de returns straße.de, after the change xn--strae-oqa.de. Both sides of same_registrable_domain move together so the predicate stays correct, but adopters logging or persisting the value see it change.
  2. canonicalize_host raises on underscore hosts and labels over 63 bytes, where host_from returns them unchanged today. The existing six callers are split between fail-open (ip_pinned_transport.py:119, key_origins.py) and fail-closed (jwks.py, revocation_fetcher.py), so "match the convention" is not available as an answer. Given etld is a comparison path and registrable_domain already documents a fail-closed None contract, mapping the raise onto None is probably right — but it should be stated in the diff.

Related

Activity

  1. added
    claude-triagingTriage routine is actively working on this issue (1-3 min)
    on Jul 29, 2026
  2. bokelley commented on Jul 29, 2026

    @bokelley
    Contributor

    Triage

    Classification: Bug
    Bucket(s): signing (label not in repo — flagged)
    Status: ready-for-human

    What the experts said:

    • ad-tech-protocol-expert: Non-breaking bug fix; ADCP spec #3690 mandates A-label normalization in the eTLD+1 path; routing host_from through canonicalize_host is correct; old U-label output was never a valid contract value, so the return-value change for IDN inputs is not a SemVer break; IDNAError → None in registrable_domain() is consistent with its documented fail-closed contract.
    • code-reviewer: Approach is sound; canonicalize_host is the right consolidation (already 4 consumers in signing); host_from() must not leak idna.IDNAError through its public contract — re-raise as ValueError in host_from or catch and map to None in registrable_domain(); removing _idna_ascii_host is safe but degrades error message at adagents.py:405; tag as semver-minor, not patch.
    • security-reviewer: No fail-open vector in the normalization itself (UTS-46 confusables produce distinct punycode, not ASCII collisions). Two exception-path mitigations required before shipping: (1) same_registrable_domain at brand_authz.py:255 is unguarded — an IDNA-invalid agent_url currently propagates an exception out of is_authorized() rather than returning BrandAuthorizationResult(False, ...); (2) if _idna_ascii_host is removed from adagents.py, the IDNAError → AdagentsValidationError wrapping at lines 402–403 must be preserved, because the caller at line 1051 catches only (AdagentsValidationError, AdagentsTimeoutError) and a raw IDNAError would escape uncaught.

    My take: Fix direction is clear and spec-mandated; all three experts agree on the approach. Flagging for human review because the two exception-path changes touch the live brand-authorization binding and the adagents redirect guard and warrant explicit sign-off before the change ships to IDN-facing deployments. @bokelley


    Triaged by Claude Code. Session: https://claude.ai/code/${CLAUDE_CODE_REMOTE_SESSION_ID}


    Generated by Claude Code

  3. added
    bugSomething isn't working
    and removed
    claude-triagingTriage routine is actively working on this issue (1-3 min)
    on Jul 29, 2026
  4. added a commit that references this issue on Jul 29, 2026
    827e4f4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions