Authenticated Received Chain - ARC sealing#33
Authenticated Received Chain - ARC sealing#33agrinchenko wants to merge 40 commits intoapache:masterfrom
Conversation
Co-authored-by: Benoit TELLIER <btellier@linagora.com>
Co-authored-by: Benoit TELLIER <btellier@linagora.com>
Co-authored-by: Benoit TELLIER <btellier@linagora.com>
Co-authored-by: Benoit TELLIER <btellier@linagora.com>
…dback - Introduced ArcValidationResult enum - Made ArcSealVerifyData immutable pojo - Removed dependency on geronimo.javamail - Removed dependency on Freemarker and simplified ARC templates - Updated versions in pom to pull them from the parent pom - Fixed missing licensing headers - Applied suggested style fixes
- Rewired DMARC into separate module - Added relaxed header alignment for DMARC using PSL list - Removed commons-codec dependency
- Added ARC validation outcome with details on the failure - Refactored logic such as `computeBTag` into separate methods - Added hard fail whenever multiple From headers detected - Rewrote tag extraction logic to do it all in one pass for efficiency - Simplified getTimeMeasure to use standard Java - Removed unnecessary Overrides
Removed typos
…ature b= is cryptographically invalid
…yptographically invalid
…chain structure is invalid. Also add cv_fail_i1_as_pass and cv_fail_i1_as_cv_fail tests: assert cv=fail when ARC-Seal cv= is invalid on first hop
…ature failures in a two-hop ARC chain
… Also add cv_pass_i2..i5 and cv_pass_i2_1_ams1_invalid tests
- Add ValiMail-derived AMS tag format validation coverage for whitespace around separators and trailing semicolons. Add a duplicate - ARC-Message-Signature i=1 ordering variant to ensure duplicate AMS instances are rejected regardless of header order.
- Add ValiMail-derived ARC tests for Authentication-Results merging, ARC-Authentication-Results structure validation, and ARC-Seal structure and tag-format handling. - Consolidate existing Authentication-Results headers for the configured authserv-id when building ARC-Authentication-Results. - Update ARC verification to select signed headers bottom-up, matching signing behavior; tolerate whitespace around tag-list equals separators.
- Add ValiMail-derived ARC-Seal field coverage for duplicate instance headers and malformed a=, cv=, d=, s=, and t= tags. Cover the valid case where ARC-Seal omits the optional t= timestamp by building a one-hop chain with a custom seal template. - Make ARC-Seal cv= validation null-safe so missing cv= is handled as cv=fail instead of risking a null dereference.
-Add ARCTest cases for ARC-Seal signature handling, including missing, empty, non-base64, modified, and whitespace-folded b= values. Cover seal integrity failures when signed AAR, AMS, or ARC-Seal fields are changed, and reject invalid ARC-Seal h= usage. -Harden ARC-Seal verification so missing or malformed b= values fail ARC validation instead of throwing, and treat invalid signature bytes as a validation failure.
- Added ValiMail public key records for 512/1024/2048-bit selector fixtures. - Hardened RSA key validation in ARCVerifier.java so RSA ARC keys below 1024 bits are rejected. - Updated ARCChainValidator.java so invalid/undersized keys fail ARC validation cleanly instead of escaping as exceptions.
-Add ARC-Message-Signature tests for duplicate instance handling, algorithm validation, signature b= parsing, and relaxed signed-header canonicalization. Cover missing, empty, unsupported, non-base64, and modified AMS signature values, plus ValiMail fixtures for signed header case, folding, whitespace, and tamper failures. -Reject AMS verification when a=rsa-sha256 is absent or b= is missing or empty so malformed signatures fail validation cleanly.
- Add ValiMail ARC validation coverage for no-chain base messages, single-hop pass fixtures, missing and malformed ARC public keys, and AMS/AS selector-domain independence. - Add AMS bh= validation coverage for whitespace-tolerant body hashes, simple and relaxed body canonicalization, missing/empty/non-base64 body hash tags, modified body hashes, and modified message bodies. - Verify AMS body hashes before accepting ARC-Message-Signature by decoding bh=, applying the AMS body canonicalization mode, and comparing the computed SHA-256 digest. Reject malformed or unsupported body hash inputs cleanly while preserving existing multipart fixture behavior when the Mime4j DOM parser cannot reconstruct a raw multipart body.
- Updated ARCVerifier to parse AMS canonicalization as header/body modes. - Added support for AMS simple header canonicalization in addition to relaxed. - Rejects empty/invalid c= values before accepting AMS.
- d= is required, non-empty, and must look like a valid domain. - s= is required, non-empty, and must look like a valid selector. - t= remains optional, but if present must be numeric.
- Add ValiMail fixture-backed tests for the remaining AMS h= signed-header cases, covering empty h=, whitespace and folded separators, case-insensitive header names, duplicate header selection, missing headers, omitted h=, header order failures, and ARC header inclusion behavior. - Reject ARC-Seal in the AMS h= signed-header list so messages that include arc-seal in AMS signing data fail validation as expected by the upstream test suite, while preserving the allowed arc-message-signature case. - Add explicit first-hop signing coverage for messages without an incoming ARC chain, asserting that generated ARC sets start at i=1 and seal with cv=none.
- Validate every ARC-Message-Signature in multi-hop chains and use the ARC-Seal key from the instance being verified. - Parse ARC i= tags numerically, enforce the valid 1..50 range, and avoid i=1 matching i=10. - Harden AMS verification for unsupported algorithms, x=/t= expiry, no-body bh= checks, and reject ARC-Seal h= tags. - Fix multipart body reconstruction to emit CRLF after the closing boundary. - Return DMARC permerror for malformed From headers instead of throwing. - Replace the bundled public suffix list with Guava InternetDomainName and add Guava as a managed DMARC dependency. - Remove the SPF thread context classloader workaround. - Add ARC and DMARC regression tests for the validation, canonicalization, and malformed input cases above. - Refresh README ARC/DMARC documentation while preserving upstream DKIM content.
- Include the failing ARC instance in previous-hop AMS validation errors. - Add injectable Clock support to ARCVerifier and use it for AMS timestamp checks. - Reject AMS t= timestamps in the future, even when x= is absent. - Remove empty multipart bh= bypass and always compare computed body hashes. - Fix parsed multipart reconstruction by reading the boundary from parent Content-Type. - Tighten ARC-Seal tag validation for required i/a/cv/d/s/b tags, cv values, d/s syntax, and h= rejection. - Convert DMARC PublicSuffixList to an injectable interface. - Add built-in default PSL resolver and optional Guava-backed implementation. - Mark Guava optional and add custom PSL injection coverage. - Add regression tests for diagnostics, timestamp validation, multipart bh= handling, and custom PSL wiring.
|
Hello @agrinchenko |
Agreed this is safer. Co-authored-by: Benoit TELLIER <btellier@linagora.com>
- Add configurable clock skew handling for AMS t=/x= validation, defaulting to 5 minutes. - Apply the same clock skew validation to ARC-Seal t= tags. - Guard ARC-Seal verification when signing data cannot be built. - Replace loose i= regex matching in ARCChainValidator with exact tag parsing. - Add regression coverage for AMS clock skew, ARC-Seal clock skew, null seal data handling, and exact i= tag matching.
|
Thanks again, @chibenwa for thorough reviews! Feel free to suggest cosmetic remarks or if you see something else. |
chibenwa
left a comment
There was a problem hiding this comment.
I think slightly more work would be needed in order to make the interface friendlier to use. I did make two comments in that direction, and paid attention to ask for minimal changes that should not impact much the code that was written but relieve library users... If you like the suggestion then it is great, otherwise I'm fine contributing those enhancements.
Also, this is a major contribution, we are forced to ask you to sign an ICLA https://www.apache.org/licenses/icla.pdf and send it to secretary@apache.org cf apache.org/licenses/contributor-agreements.html please send the email address used to sign the ICLA to private@james.apache.org. Do not Cc the list on your ICLA. We may track this onhttps://whimsy.apache.org/roster/icla/ with this info. With such a qualitative and impactful contribution this is a necessary measure.
Thanks again for your involvement
Best regards,
Benoit
| "pass client-ip=192.0.2.1; envelope-from=sender@example.org", | ||
| "example.org", | ||
| "pass", | ||
| "example.org"); |
There was a problem hiding this comment.
That is a lot of string arguments that I would be annoyend to distinguish from each other and to actually come up with. Suggestion: syntactic sugar
// I know no record but for interface writing the corresponding POJO is worth it.
enum DkimStatus {FAIL, PASS, NEUTRAL} // better discoverability that strings ! We can have a convenience parse method to bridge the gap, and getters.
record DmarcDataInput(Message message, String spfHeaderText, String spfDomain, String dkimResult, String dkimDomain);
// Then we could play with builders
dmarcVerifier.verify(DmarcDataInput.builder()
.message(message)
.spfHeaderValue("pass client-ip=192.0.2.1; envelope-from=sender@example.org")
.ehlo("exemple.org")
.dkimResult(PASS)
.dkimDToken("example.org")
.build());
Would already be a nice enhancement.
We could have a typed alternative for SPF. Also We likely should offer a wauy to work directly with DKIM signature records.
dmarcVerifier.verify(DmarcDataInput.builder()
.message(message)
.spfResult(spfResult)
.ehlo("exemple.org") // This eases the user in understanding how to get that horrible SPF string and not get it himself
.dkimSignatures(signatureRecords) // Again we would offweight the user from dealing himself with interpreting the DKIM results -> DMARC job. Type: List<SignatureRecord>
.build());
To be honnest I do not care (here) about the internal, just about the outer layers. IMO we can easily do the last kilometer to make it easy to use.
Does this make sense?
| "mail.example.org", | ||
| "sender@example.org", | ||
| "192.0.2.1", | ||
| keyRecordRetriever); |
There was a problem hiding this comment.
Here again some interface cosmetic fixes would go a long way. Same principal: builders, and POJO extracted...
| keyRecordRetriever); | |
| ArcSetBuilder arcSetBuilder = new ArcSetBuilder( | |
| privateKey, | |
| keyRecordRetriever, | |
| amsTemplate, | |
| sealTemplate, | |
| "mx.example.org", | |
| System.currentTimeMillis() / 1000); | |
| PublicKeyRetrieverArc keyRecordRetriever = null; | |
| record ArcSigningInput(Message message, String helo, String mailFrom, String ip); | |
| Map<String, String> arcSet = arcSetBuilder.buildArcSet(ArcSigningInput.builder() | |
| .message(message) | |
| .ehlo("mail.example.org") | |
| .mailFrom("sender@example.org") | |
| .ip("192.0.2.1") | |
| .build()); |
- IMO
keyRecordRetrieveris better handled as a class field as there is as far as I - Builder.
|
(Ps: those two comment address and I approuve) |
Replacement branch for PR #31, based on the current apache/master to resolve merge conflicts on PR #31.