Repository navigation
FAQ
Short answers to what people ask first about Parsec, each pointing at the page with the long one.
All 24 questions
- Can I use Parsec today?
- When is the first release?
- Is it a fork of Lexbor, or html5ever?
- Is it faster than Lexbor?
- How correct is it?
- Does it run JavaScript?
- Which
scriptingvalue should I use? - Can I parse bytes, or pages in other encodings?
- Why did my page parse in quirks mode?
- Why doesn't
.Foomatchclass="foo"? - Why doesn't
[href]find SVG'sxlink:href? - Why can't my selector see what's inside a
<template>? - Why is
:blank(or::before) an error? - How do I read one attribute?
- How do I set
innerHTML? - What happens to a node I remove?
- Can I share a
Documentbetween threads? - How do I protect a crawler from hostile pages?
- Can I get line and column numbers?
- Can I stream a page in as it downloads?
- Is there XPath?
- Can I use it from C, Python or JavaScript?
- Why does it need a CLA if it's Apache-2.0?
- Where do I ask something that isn't here?
Yes, from git; it isn't released. Add parsec-html = { git = "https://github.com/parsec-html/parsec" } to your dependencies and use parsec. The API can still change before 0.1.0, so pin a commit if that matters to you. Getting started has the details.
When milestones M1 to M4 are done and the benchmarks show Parsec ahead of Lexbor on every page in the corpus (#32). There's no date. Roadmap has what's in each milestone.
Neither. It's its own parser, written in Rust from the HTML Standard. It's developed inside Netweir as crates/netweir-html and synced here; Contributing explains how.
On real pages, yes. Against Lexbor 3.0.1's own C in one process, Parsec is 1.5 times as fast across 13 real pages and ahead on every one of them, with less memory. It's ahead on all eight hostile inputs as well. Benchmarks has the numbers and how to rerun them on your machine.
It passes all 7,042 html5lib tokenizer runs it can be given and all 3,898 runs of the WPT tree-construction cases that don't need JavaScript. Conformance says exactly what's counted and skipped.
No. The scripting flag only changes how <noscript> is parsed, as a browser with JavaScript on or off would parse it. See Parsing.
false, usually. Then <noscript> fallbacks become ordinary elements your selectors can find. Use true if you want the tree a JavaScript-enabled browser builds.
Yes. parsec_encoding::parse_bytes takes bytes and finds the encoding the way a browser does: byte order mark, the HTTP charset if you pass it, <meta charset>, then a fallback. Every encoding in the WHATWG Encoding Standard decodes, through encoding_rs. See Encoding.
Either it has no doctype, or it has an old one, or a tag or text other than whitespace comes before it. A byte order mark at the very start doesn't count: parse drops it, as a browser does. Quirks mode has the rules.
Class names and ids compare exactly, as in a browser, except in quirks-mode documents. A page with no doctype is in quirks mode, so there .foo would match. See Selectors.
Because xlink:href is in the XLink namespace and [href] means no namespace, as in a browser. Use [*|href].
The HTML Standard keeps template contents out of the tree, and Parsec follows it. Run the selector on doc.template_contents(id), or call doc.adopt_template_contents() once to move all contents into the tree, as lxml and Beautiful Soup show them. See The document tree.
Parsec refuses selectors it doesn't match yet, rather than parse them and silently match nothing. Each missing pseudo-class has an issue; pseudo-elements select parts of elements, not elements, so select can't return them. Selectors has the list.
let doc = parsec::parse("<a href=/x>x</a>", false);
let a = parsec::css::SelectorList::parse("a").unwrap().select(&doc, doc.root())[0];
let href = doc.attrs(a).find(|&(_, name, _)| name == "href").map(|(_, _, v)| v);
assert_eq!(href, Some("/x"));There's no single-call getter yet.
There's no setter. Parse the markup with parse_fragment in the element's context and copy the nodes in; Moving from Lexbor has a helper that does it.
That depends on how you took it out. remove is for good: the node and everything in it stay in the document's memory until it's dropped, but their ids read as nothing (no parent, children, attributes, text or markup), and inserting one is EditError::Removed. detach is for moving: the node leaves the tree with its subtree and you can insert it again. In C they're parsec_remove and parsec_detach. See Editing.
Yes. It's Send and Sync today, so you can read it from many threads and move it between them; edits need &mut. That isn't a promise yet: #11 makes it one. See Thread safety and memory safety.
Use parse_with with a budget, and check sizes before you parse. Limits and hostile input explains what the budget covers and where Parsec is and isn't linear.
Yes, for nodes, attributes and parse errors. Parse with Options { spans: true, .. } and Document::span, end_span, attr_span and line_col say where each was written; with errors: true, Document::errors() lists the parse errors with line and column. Parsing has both. They're in the Python, C and WASM bindings too.
Yes. For text, parsec::Parser::new(options), then feed(&str) as each piece arrives and finish(). For bytes straight off the network, parsec_encoding::ByteParser does the same and finds the encoding too: it waits only until it knows it (a byte order mark, your HTTP charset, a whole <meta>, or 1,024 bytes), then decodes and parses each piece as it comes. Either way you get the tree the whole page would give, however it was cut. See Encoding.
Yes, XPath 1.0, in parsec-xpath (XPath), and from Python (page.xpath("//a/@href"), answering as lxml does), C (parsec_xpath_evaluate) and JavaScript (doc.xpath()). There's no XML parser; Ns::Xml is for trees you build yourself.
From C and Python, yes. C API has the header, the libraries and how to link them, and Moving from Lexbor maps Lexbor's calls to it. bindings/python is a package with selectolax's API, built from git until its wheels go to PyPI with 0.1.0; Python has it all. WebAssembly isn't there yet: #24.
Because the parser is shared with Netweir, which is AGPL-3.0 and also sold under a commercial licence, so contributions here may go there too. You keep your copyright. Contributing has the details.
Discussions. Bugs go to Issues, and security problems to the private form in SECURITY.md.
- Getting started, for a first program.
- Glossary, for the terms these answers use.
- Roadmap, for anything that says "not yet".
Parsec is Apache-2.0 and not yet released; these pages describe main. Found something wrong here? Open an issue or ask in Discussions.
Repository · Issues · Discussions · Roadmap · Security policy · Licence