Repository navigation
URL
parsec-url (crates/parsec-url) is the URL module: the WHATWG URL Standard with the same getters and setters the JavaScript URL has, URLSearchParams, and the host, IDNA, Punycode and Unicode normalization steps under them. It's a parser of Parsec's own, written from the standard's current text, with no dependencies. Every one of the 4,565 web-platform-tests URL cases passes, and on the links in real pages it's faster than Ada, the C++ parser in Node.js.
On this page: Using it · Url · SearchParams · Hosts, IDNA and Punycode · Unicode normalization · How it's built · Conformance · Speed · Next to Lexbor
It isn't on crates.io yet. From git:
[dependencies]
parsec-url = { git = "https://github.com/parsec-html/parsec" }use parsec_url::Url;
let base = Url::parse("https://example.com/a/b?x=1").unwrap();
let mut url = Url::parse_with_base("../c?q=two words#top", &base).unwrap();
assert_eq!(url.href(), "https://example.com/c?q=two%20words#top");
let mut params = url.search_params();
params.append("page", "2");
url.set_search_params(¶ms);
assert_eq!(url.search(), "?q=two+words&page=2");| Rust | JavaScript | Notes |
|---|---|---|
Url::parse(input) |
new URL(input) |
Result<Url, ParseError> |
Url::parse_with_base(input, &base) |
new URL(input, base) |
Parse the base once and reuse it. |
Url::can_parse(input, Option<&Url>) |
URL.canParse() |
|
href(), set_href(v)
|
href |
set_href returns a Result, because the href setter throws. |
origin() |
origin |
A String; "null" for an opaque origin. A blob: URL's origin is its inner URL's when that's http: or https:. |
protocol(), set_protocol(v)
|
protocol |
With the colon: "https:". |
username(), set_username(v)
|
username |
|
password(), set_password(v)
|
password |
|
host(), set_host(v)
|
host |
Host and port. |
hostname(), set_hostname(v)
|
hostname |
Host alone. |
port(), set_port(v)
|
port |
Empty when it's the scheme's default. |
pathname(), set_pathname(v)
|
pathname |
|
search(), set_search(v)
|
search |
With the ?, or empty. |
hash(), set_hash(v)
|
hash |
With the #, or empty. |
search_params() |
searchParams |
A copy; see below. |
set_search_params(¶ms) |
changing searchParams
|
An empty list takes the ? away too. |
Every setter but set_href behaves as the JavaScript one does: give it a value the spec rejects and the URL stays as it was, with no error. Url is Clone, Eq, Ord, Hash, Display and FromStr; two URLs are equal, and ordered, by their href.
searchParams in a browser is live: change it and the URL changes. A Rust value can't do that without shared ownership, so here it's a copy you write back with set_search_params, which does what the browser does on each change.
ParseError says why a parse failed: EmptyHost, IdnaError, InvalidPort, InvalidIpv4Address, InvalidIpv6Address, InvalidDomainCharacter, RelativeUrlWithoutBase, RelativeUrlWithCannotBeABaseBase, or Overflow for a URL of 4 GiB or more.
URLSearchParams: an ordered list of name-value pairs.
| Rust | JavaScript |
|---|---|
SearchParams::parse(init) |
new URLSearchParams(init); one leading ? is dropped |
iter().collect() from pairs |
new URLSearchParams([[n, v], …]) |
len(), is_empty()
|
size |
append(name, value) |
append |
delete(name, Option<value>) |
delete |
get(name) |
get, Option<&str>
|
get_all(name) |
getAll |
has(name, Option<value>) |
has |
set(name, value) |
set: the first pair gets the value, the rest go |
sort() |
sort, stable, by UTF-16 code units as the spec says, so 🌈 sorts before ffi
|
to_string() |
toString() |
iter() |
iteration |
Parsing and serializing are application/x-www-form-urlencoded as the URL Standard defines it: + is a space, bytes that aren't UTF-8 after percent-decoding become U+FFFD, and only *-._ and ASCII letters and digits go out unescaped. tests/search_params.rs checks all of this with the cases from WPT's urlencoded-parser, urlsearchparams-sort and urlsearchparams-stringifier tests, which keep their data in JavaScript rather than JSON.
| Function | What it is |
|---|---|
domain_to_ascii(d) |
The standard's domain parser, as the host parser runs it. "Bücher.example" becomes "xn--bcher-kva.example"; Err(InvalidDomain) if it's rejected. |
domain_to_unicode(d) |
Domain to Unicode, for showing a host to a person: "xn--bcher-kva.example" becomes "bücher.example". A domain UTS #46 finds an error in comes back as it was. |
Host::parse(s) |
The host parser for special schemes: a domain, IPv4 or IPv6. |
punycode::encode_str, punycode::decode_to_string
|
RFC 3492 Punycode, one label, no xn--. |
Since June 2026 the standard keeps an ASCII domain as it's written, lowercased: xn--a, which isn't valid Punycode, is a host now, where it used to be an error. So IDNA only runs on a domain with a non-ASCII code point in it, and most hosts never reach it. When it does run, it's UTS #46 with the options the standard picks: nontransitional, CheckBidi and CheckJoiners on, CheckHyphens, UseSTD3ASCIIRules and VerifyDnsLength off. CheckBidi is RFC 5893's six rules over every label once any label is right-to-left; CheckJoiners is RFC 5892's ContextJ rules for U+200C and U+200D.
parsec_url::normalize::nfc and nfd are Unicode Normalization Forms C and D (UAX #15). IDNA needs NFC, and they're public because other things need them too.
use parsec_url::normalize::{nfc, nfd};
assert_eq!(nfc("e\u{301}"), "\u{e9}");
assert_eq!(nfd("\u{e9}"), "e\u{301}");They do it the way UAX #15 describes: full canonical decomposition, canonical ordering by combining class, then composition with the composition exclusions, and Hangul syllables by arithmetic. NFC checks the quick-check property first, so text that's already NFC, which is nearly all of it, comes back after one pass.
A parsed Url is one String, the serialization, and the offsets where each component starts in it, the way Ada's url_aggregator is. Parsing writes the serialization as it goes, so a parse is one allocation, and every getter is a slice. The setters take the URL apart into those slices, change the one they're for, and write it out again.
The state machine is the standard's, walked over runs of bytes rather than a code point at a time: a state that only collects, like a path segment, is a scan to the byte that ends it, and runs that need no percent-encoding are copied whole. An ASCII host with no % is lowercased as it's copied, with one table lookup a byte deciding whether it needs anything else.
The Unicode data is generated: crates/parsec-url/scripts/gen-tables.py downloads Unicode 18.0's IdnaMappingTable, UnicodeData, DerivedNormalizationProps, DerivedBidiClass and DerivedJoiningType and writes src/tables.rs, sorted runs looked up by binary search. Runs that map each code point the same distance (A to Z onto a to z) or that alternate capital and small letters are one entry each, so the IDNA table is 5,411 runs for all 1,114,112 code points. Unicode 18.0 rather than 17.0 because WPT's IDNA tests at the pinned commit are generated from 18.0: seven of them use code points 18.0 added.
crates/parsec-url/tests/wpt.rs runs the JSON files from WPT's url/resources, pinned in crates/parsec-url/tests/wpt at WPT commit 6dac967, the way WPT's own .any.js files use them. Each WPT test() counts once, and any failure fails the run. cargo test runs it, and CI publishes the counts with the others (Conformance).
| WPT test | Data | Passed | Failed | Skipped |
|---|---|---|---|---|
url-constructor |
urltestdata.json |
904 | 0 | 0 |
url-origin |
urltestdata.json |
421 | 0 | 0 |
url-setters |
setters_tests.json |
284 | 0 | 0 |
toascii |
toascii.json |
261 | 0 | 522 |
IdnaTestV2 |
IdnaTestV2.json |
2,675 | 0 | 1 |
IdnaTestV2-removed |
IdnaTestV2-removed.json |
20 | 0 | 0 |
| All | 4,565 | 0 | 523 |
The skipped ones: in toascii, the <a> and <area> halves, which test what HTML elements do with a bad URL, not URL parsing; in IdnaTestV2, the one empty input, which WPT skips too because it can't go through new URL().
crates/parsec-url/tests/normalization.rs runs Unicode 18.0's NormalizationTest.txt, pinned in tests/unicode: all 20,171 lines pass, each checked ten ways (NFC and NFD of all five columns), and so does the file's part 1 rule, that every code point it doesn't list normalizes to itself, checked for all 1,112,064 scalar values. That's 20,172 in the counts.
Before this parser, parsec-url wrapped Servo's url 2.5.8 and idna 1.1.0, which pass 3,690 of the 4,565. Most of the 875 they fail, 795, are the June 2026 ASCII-domain rule above; the rest are file: hosts before drive letters and empty path segments, blob: origins, spaces at the end of opaque paths, ^ in paths, and ///x against an http: base. None of them fails here.
Lone surrogates in the JSON become U+FFFD before parsing. new URL() takes a USVString, so a browser makes the same swap, and a Rust string can't hold one anyway. Not vendored: urltestdata-javascript-only.json, which is all lone-surrogate cases for JavaScript, and percent-encoding.json, which needs the legacy encodings (#25).
bench/ times parsec-url against Lexbor's lxb_url_parse and Ada 4.0.0. Benchmarks has the method and the full table.
| List | URLs | Parsec | Lexbor | Ada |
|---|---|---|---|---|
href and src on the 13 benchmark pages |
83,113 | 107 ns | 243 ns | 142 ns |
WPT urltestdata.json inputs |
904 | 124 ns | 183 ns | 157 ns |
On real links Parsec is 1.33 times as fast as Ada and 2.28 times as fast as Lexbor, and all three give the same href for every one of the 83,113. On the WPT inputs Ada's hrefs match Parsec's on all 904; Lexbor's differ on 16. The url crate this replaced took 181 ns on the real links, so that's 1.7 times as fast as before.
Lexbor's URL module (url.h) has the same parts: URL, URLSearchParams, IDNA, Punycode and normalization. Its own tests are 287 cases of its own; it doesn't run WPT's or publish a count, and it has open conformance bugs (lexbor#438 to lexbor#442). PHP's Uri\WhatWg\Url carries about 18 patches on top of it.
- Conformance, for every suite Parsec runs.
- Benchmarks, for the URL timings and how to run them.
- Comparison with Lexbor, module by module.
Parsec is Apache-2.0 and not yet released; these pages describe main. Found something wrong here? Open an issue or ask in Discussions.
Repository · Issues · Discussions · Roadmap · Security policy · Licence