Skip to content
James Conway edited this page Nov 19, 2026 · 2 revisions

parsec-url (crates/parsec-url) is the URL module: the WHATWG URL Standard with the same getters and setters the JavaScript URL has, URLSearchParams, and the host, IDNA, Punycode and Unicode normalization steps under them. It's a parser of Parsec's own, written from the standard's current text, with no dependencies. Every one of the 4,565 web-platform-tests URL cases passes, and on the links in real pages it's faster than Ada, the C++ parser in Node.js.

On this page: Using it · Url · SearchParams · Hosts, IDNA and Punycode · Unicode normalization · How it's built · Conformance · Speed · Next to Lexbor

Using it

It isn't on crates.io yet. From git:

[dependencies]
parsec-url = { git = "https://github.com/parsec-html/parsec" }
use parsec_url::Url;

let base = Url::parse("https://example.com/a/b?x=1").unwrap();
let mut url = Url::parse_with_base("../c?q=two words#top", &base).unwrap();
assert_eq!(url.href(), "https://example.com/c?q=two%20words#top");

let mut params = url.search_params();
params.append("page", "2");
url.set_search_params(&params);
assert_eq!(url.search(), "?q=two+words&page=2");

Url

Rust JavaScript Notes
Url::parse(input) new URL(input) Result<Url, ParseError>
Url::parse_with_base(input, &base) new URL(input, base) Parse the base once and reuse it.
Url::can_parse(input, Option<&Url>) URL.canParse()
href(), set_href(v) href set_href returns a Result, because the href setter throws.
origin() origin A String; "null" for an opaque origin. A blob: URL's origin is its inner URL's when that's http: or https:.
protocol(), set_protocol(v) protocol With the colon: "https:".
username(), set_username(v) username
password(), set_password(v) password
host(), set_host(v) host Host and port.
hostname(), set_hostname(v) hostname Host alone.
port(), set_port(v) port Empty when it's the scheme's default.
pathname(), set_pathname(v) pathname
search(), set_search(v) search With the ?, or empty.
hash(), set_hash(v) hash With the #, or empty.
search_params() searchParams A copy; see below.
set_search_params(&params) changing searchParams An empty list takes the ? away too.

Every setter but set_href behaves as the JavaScript one does: give it a value the spec rejects and the URL stays as it was, with no error. Url is Clone, Eq, Ord, Hash, Display and FromStr; two URLs are equal, and ordered, by their href.

searchParams in a browser is live: change it and the URL changes. A Rust value can't do that without shared ownership, so here it's a copy you write back with set_search_params, which does what the browser does on each change.

ParseError says why a parse failed: EmptyHost, IdnaError, InvalidPort, InvalidIpv4Address, InvalidIpv6Address, InvalidDomainCharacter, RelativeUrlWithoutBase, RelativeUrlWithCannotBeABaseBase, or Overflow for a URL of 4 GiB or more.

SearchParams

URLSearchParams: an ordered list of name-value pairs.

Rust JavaScript
SearchParams::parse(init) new URLSearchParams(init); one leading ? is dropped
iter().collect() from pairs new URLSearchParams([[n, v], …])
len(), is_empty() size
append(name, value) append
delete(name, Option<value>) delete
get(name) get, Option<&str>
get_all(name) getAll
has(name, Option<value>) has
set(name, value) set: the first pair gets the value, the rest go
sort() sort, stable, by UTF-16 code units as the spec says, so 🌈 sorts before ffi
to_string() toString()
iter() iteration

Parsing and serializing are application/x-www-form-urlencoded as the URL Standard defines it: + is a space, bytes that aren't UTF-8 after percent-decoding become U+FFFD, and only *-._ and ASCII letters and digits go out unescaped. tests/search_params.rs checks all of this with the cases from WPT's urlencoded-parser, urlsearchparams-sort and urlsearchparams-stringifier tests, which keep their data in JavaScript rather than JSON.

Hosts, IDNA and Punycode

Function What it is
domain_to_ascii(d) The standard's domain parser, as the host parser runs it. "Bücher.example" becomes "xn--bcher-kva.example"; Err(InvalidDomain) if it's rejected.
domain_to_unicode(d) Domain to Unicode, for showing a host to a person: "xn--bcher-kva.example" becomes "bücher.example". A domain UTS #46 finds an error in comes back as it was.
Host::parse(s) The host parser for special schemes: a domain, IPv4 or IPv6.
punycode::encode_str, punycode::decode_to_string RFC 3492 Punycode, one label, no xn--.

Since June 2026 the standard keeps an ASCII domain as it's written, lowercased: xn--a, which isn't valid Punycode, is a host now, where it used to be an error. So IDNA only runs on a domain with a non-ASCII code point in it, and most hosts never reach it. When it does run, it's UTS #46 with the options the standard picks: nontransitional, CheckBidi and CheckJoiners on, CheckHyphens, UseSTD3ASCIIRules and VerifyDnsLength off. CheckBidi is RFC 5893's six rules over every label once any label is right-to-left; CheckJoiners is RFC 5892's ContextJ rules for U+200C and U+200D.

Unicode normalization

parsec_url::normalize::nfc and nfd are Unicode Normalization Forms C and D (UAX #15). IDNA needs NFC, and they're public because other things need them too.

use parsec_url::normalize::{nfc, nfd};

assert_eq!(nfc("e\u{301}"), "\u{e9}");
assert_eq!(nfd("\u{e9}"), "e\u{301}");

They do it the way UAX #15 describes: full canonical decomposition, canonical ordering by combining class, then composition with the composition exclusions, and Hangul syllables by arithmetic. NFC checks the quick-check property first, so text that's already NFC, which is nearly all of it, comes back after one pass.

How it's built

A parsed Url is one String, the serialization, and the offsets where each component starts in it, the way Ada's url_aggregator is. Parsing writes the serialization as it goes, so a parse is one allocation, and every getter is a slice. The setters take the URL apart into those slices, change the one they're for, and write it out again.

The state machine is the standard's, walked over runs of bytes rather than a code point at a time: a state that only collects, like a path segment, is a scan to the byte that ends it, and runs that need no percent-encoding are copied whole. An ASCII host with no % is lowercased as it's copied, with one table lookup a byte deciding whether it needs anything else.

The Unicode data is generated: crates/parsec-url/scripts/gen-tables.py downloads Unicode 18.0's IdnaMappingTable, UnicodeData, DerivedNormalizationProps, DerivedBidiClass and DerivedJoiningType and writes src/tables.rs, sorted runs looked up by binary search. Runs that map each code point the same distance (A to Z onto a to z) or that alternate capital and small letters are one entry each, so the IDNA table is 5,411 runs for all 1,114,112 code points. Unicode 18.0 rather than 17.0 because WPT's IDNA tests at the pinned commit are generated from 18.0: seven of them use code points 18.0 added.

Conformance

crates/parsec-url/tests/wpt.rs runs the JSON files from WPT's url/resources, pinned in crates/parsec-url/tests/wpt at WPT commit 6dac967, the way WPT's own .any.js files use them. Each WPT test() counts once, and any failure fails the run. cargo test runs it, and CI publishes the counts with the others (Conformance).

WPT test Data Passed Failed Skipped
url-constructor urltestdata.json 904 0 0
url-origin urltestdata.json 421 0 0
url-setters setters_tests.json 284 0 0
toascii toascii.json 261 0 522
IdnaTestV2 IdnaTestV2.json 2,675 0 1
IdnaTestV2-removed IdnaTestV2-removed.json 20 0 0
All 4,565 0 523

The skipped ones: in toascii, the <a> and <area> halves, which test what HTML elements do with a bad URL, not URL parsing; in IdnaTestV2, the one empty input, which WPT skips too because it can't go through new URL().

crates/parsec-url/tests/normalization.rs runs Unicode 18.0's NormalizationTest.txt, pinned in tests/unicode: all 20,171 lines pass, each checked ten ways (NFC and NFD of all five columns), and so does the file's part 1 rule, that every code point it doesn't list normalizes to itself, checked for all 1,112,064 scalar values. That's 20,172 in the counts.

Before this parser, parsec-url wrapped Servo's url 2.5.8 and idna 1.1.0, which pass 3,690 of the 4,565. Most of the 875 they fail, 795, are the June 2026 ASCII-domain rule above; the rest are file: hosts before drive letters and empty path segments, blob: origins, spaces at the end of opaque paths, ^ in paths, and ///x against an http: base. None of them fails here.

Lone surrogates in the JSON become U+FFFD before parsing. new URL() takes a USVString, so a browser makes the same swap, and a Rust string can't hold one anyway. Not vendored: urltestdata-javascript-only.json, which is all lone-surrogate cases for JavaScript, and percent-encoding.json, which needs the legacy encodings (#25).

Speed

bench/ times parsec-url against Lexbor's lxb_url_parse and Ada 4.0.0. Benchmarks has the method and the full table.

List URLs Parsec Lexbor Ada
href and src on the 13 benchmark pages 83,113 107 ns 243 ns 142 ns
WPT urltestdata.json inputs 904 124 ns 183 ns 157 ns

On real links Parsec is 1.33 times as fast as Ada and 2.28 times as fast as Lexbor, and all three give the same href for every one of the 83,113. On the WPT inputs Ada's hrefs match Parsec's on all 904; Lexbor's differ on 16. The url crate this replaced took 181 ns on the real links, so that's 1.7 times as fast as before.

Next to Lexbor

Lexbor's URL module (url.h) has the same parts: URL, URLSearchParams, IDNA, Punycode and normalization. Its own tests are 287 cases of its own; it doesn't run WPT's or publish a count, and it has open conformance bugs (lexbor#438 to lexbor#442). PHP's Uri\WhatWg\Url carries about 18 patches on top of it.

See also

Clone this wiki locally