Skip to content

Docs: the Broker orders offers by relevance; AI search crawlers refused by default; the console's operator named - #76

Merged
EugeneDymo merged 61 commits into
mainfrom
docs/broker-relevance
Nov 16, 2026
Merged

EugeneDymo merged 61 commits into
mainfrom
docs/broker-relevance

Conversation

@KonstantinMirin

@KonstantinMirin KonstantinMirin commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

The docs stop describing the Broker as a price-comparison step, the bot-detection page matches what FORA Server does, and Publisher onboarding names the console's operator. Docs, two proto comments and one check; no wire change.

The Broker orders by relevance, and the agent decides. A Broker returns every Exchange's signed offers in its order of relevance to the agent's request, optionally narrowed to the kinds of license the agent accepts (acceptable_restrictions). It groups offers for the same content without dropping any, and never buys, resells or substitutes an offer. Price is not a signal: Pricing.unit_cost is there for the agent to compare offers under its own policy.

  • components/broker/selection-engine.mdx is rewritten on these lines (RelevancePolicy as the default, the selection log records the order and why).
  • content-dedup.mdx groups instead of keeping the cheapest; walkthroughs where a cheaper offer wins say it's the agent's own policy.
  • One-line fixes across Discovery Paths, the Agent SDK pages, How money flows (:78), For AI agents, For Exchange operators, Industry landscape, Role composition, Standards layering, Transaction flow, the architecture pages, the billing adapter and ext-news.
  • Two comments in fora.proto (Offer.pricing and the one-term-per-offer note) now say an agent compares the price, not that a Broker ranks on it. gen/ is regenerated (buf generate, buf build, scripts/gen-sdk-types.sh); the diff there is comment text only.
  • After this PR, "cheapest", "best deal" and price ranking appear only where the agent's own policy is described, in threat descriptions, and in the changelogs' history.

AI search crawlers are refused by default. components/edge-function/bot-detection.mdx described a bot_detection.ai_search_policy setting that FORA Server doesn't have. It now shows FORA Server's built-in allow and deny lists and how a provider replaces them (BOT_UA_ALLOW_JSON, BOT_UA_DENY_JSON): AI search crawlers such as PerplexityBot get a 403 on licensed paths by default, and Googlebot, Bingbot and Applebot are never refused. The /publishers/ FAQ, the AI crawlers resource, BotTable and the rules generator say the same.

Publisher onboarding names the console's operator. FORA is open-source software, not an operator. The console on this site is run by Postindustria, which charges publishers no Exchange fee beyond the payment system's transaction fees.

Conformance check. check-doc-conformance.sh matched the pre-rename org literal case-insensitively, so it refused the company's name in prose. It now matches case-sensitively: an org, module path or package name is lower case, and still fails the build.

Changelog. Entries under Unreleased in proto/CHANGELOG.md and the docs changelog.

Checks. npm run build (94 pages, links valid), npm test (pass), npm run routes (unchanged), routes:check (clean), check-doc-conformance.sh (clean), buf generate leaves gen/ in sync.


Revised 2026-11-16: this PR now also carries the Resources stack. #77 (content marketplaces, the standards map, About) and #79 (Web Bot Auth and agent identity) were merged into this branch. On top of them, all 27 Resources figures are redrawn in the site's diagram style: flat rounded boxes and typeset labels, matching the diagram components, square and laid out top to bottom for phones. The labels keep their wording. The wba-8 alt text changed because its timeline now runs down the side. The figures total 1.8 MB, down from 7.4 MB. Checks on the merged branch: build, 161 tests, routes, routes:check and doc conformance all pass.

The Broker returns every offer, each as the full package its Exchange
signed, in its order of relevance to the request. It can narrow offers to
the license kinds the agent states in acceptable_restrictions, groups
offers for the same content without dropping any, and applies the agent's
own budget and price caps as filters. Price is not part of the order: the
agent compares offers, unit_cost included, and decides what it buys.

Rewrites the Selection Engine and Content Deduplication pages to match,
and removes price selection from the walkthroughs, the discovery paths,
the Agent SDK pages and the getting-started pages. Where an agent picks
a cheaper offer, the walkthrough now says that the choice is the agent's.
FORA Server's edge function refuses AI search crawlers such as
PerplexityBot and OAI-SearchBot on licensed paths, as it refuses training
crawlers and live fetchers. Its built-in deny list names them, and its
generic bot, crawler and spider patterns catch most others. The provider
lets a crawler through by replacing the allow list with
BOT_UA_ALLOW_JSON; BOT_UA_DENY_JSON replaces the deny list.

The bot detection page now shows the built-in lists, the Cloudflare
verified-bot category and the two settings, in place of a
bot_detection.ai_search_policy setting that the edge function does not
have. The publishers FAQ, the AI crawlers page, its bot table and the
rules generator say the same.
src/data/content-marketplaces.ts holds the texts every page of the series
uses word for word: About FORA, the definitions, the search callout, the
eight comparison questions, the GlanceCard rows, FORA's fixed cells, the
status vocabulary and the series list. The three TollBit figures are
palette PNGs, 1024 px square.
New: GlanceCard, KeyFacts, Callout (seven kinds, edge colour by voice,
optional disclosure on narrow screens), StatusTag, Figure (with a text
version), StepList, Split, CompareTable, SectionDivider and SeriesNav.
Status tags and citation markers are global atoms in landing.css, with
their markup in src/lib/resource-markup.ts.

Extended without changing existing pages: Band gains --lp-surface, --lp-bg,
--lp-muted and --lp-fora, a compact option and top-aligned asides; CardGrid
reads the band variables and takes a status; PointList takes inline HTML and
a numbered checklist form; PullQuote has a small size; SourceList can number
its entries with id=src-N; LinkRow can stack; AudienceHero's badge can hold
a link.
A resource can name a section; the index lists entries without one as
before, then one band per section with its heading and intro. The series
lists a page once it is built: TollBit for now.
/resources/content-marketplaces/tollbit/, built from the editorial pattern
components, with numbered sources that the copy's markers link to.
A pull quote's edge is now grey: a quote is always someone else's words.
PointList gains tone="neutral" for points that are a vendor's or nobody's
in particular, set on the 'ahead' lists, the questions, PCM's column, the
Perplexity timeline and the standards map's status and differences.
Below 760px each player's card shows what's sold and who pays today; the
other columns fold behind a per-card More disclosure on the player's line.
The desktop table is unchanged.
Add /about/: the mission, what FORA is, Postindustria and the people
who build FORA, how the protocol is developed, and how to take part.

The header links to it: About joins LANDING_PAGES, so the site nav and
the docs header both list it. With five links the nav fits on one row
just above the menu breakpoint only with tighter gaps, so the nav sets
them between 769 and 960 px and keeps each link on one line.

The positioning statement and the tagline move to
src/data/content-marketplaces.ts, shared by /about/ and the
content-marketplaces page, which renders as before.
The section listed what exists today, a built-versus-not framing. It now
states what FORA consists of, and the Exchange operator line describes
running an Exchange on FORA Server under the operator's own rules and fees.
Selection Engine, Content Deduplication, Budget Enforcement and Deployment
described a design rather than FORA Server, and were off the sidebar. The
overview now states the order rule (relevance, the agent compares prices),
that offers for the same content are grouped and none dropped, that the
agent's budget is its own constraint, that a Broker's fee is never folded
into the signed price, and what an operator deploys to run FORA Server's
Broker. The four URLs redirect to the overview, and How Money Flows links
there.
The docs no longer explain FORA by OpenRTB, Prebid, SSPs or walled
gardens. The SSP-Alpha and SSP-Beta Exchanges in the examples are
Exchange Alpha and Exchange Beta, and the ssp hostnames are .example
hosts. The quote in the fair-collective evidence follows How Money Flows.
The page framed the operator as an ad-tech aggregator, projected revenue
and named real companies as example operators. It now describes running
an Exchange: anyone can run FORA Server, every FORA agent can buy from
it, and the operator sets its own rules and fee. What an operator deploys,
the calls the Exchange answers, the fee and agent policy, and how
publishers and agents join follow FORA Server. Role types replace the
named operators. The SSP mapping, the revenue model, the dispute tiers,
the NFR targets and the old billing interface are gone.
The page explained FORA by ad-tech fraud history and claimed controls
that neither the protocol nor FORA Server has: single-use URLs, per-tenant
signing keys, automatic dispute credits, quota tracking, and Exchange-side
fetching of License.uri. Each countermeasure now states what the protocol
defines and what FORA Server does, with the real field names. The example
lookalike domain is fictional, the SSRF table states each SDK's default,
and the taxonomy lists every threat once. A Security group in the sidebar
holds the page.
The Exchange table has three columns, and the two admin calls get a row
each.
A guide to Web Bot Auth for publishers and engineers: who signs, how much
is signed, what changes in your stack, what a signature proves, the allow
list, the argument over who gets in, the other ways to identify an agent,
and how FORA uses it. Five figures, 54 numbered sources.

Adds ResourceTable (a table that turns into cards on phones), the
Experimental and Preview status tags, and optional lead-ins for PointList
and Callout points.
…e credentials

A band after "What a signature proves": a token issued at sign-up, OAuth
or a JWT, and Web Bot Auth side by side, how permission is attached on top,
and how FORA keeps the two apart. ResourceTable can label every cell on
phones and sets even widths for four columns.
Web Bot Auth now opens by showing how a signature is checked, in both
directions, and gives key directories, rotation and revocation their own
band, with what to ask your CDN and your team. The other identity schemes
move to /resources/agent-identity/: four questions about any agent (who it
is, whom it acts for, what it may do, who pays), a signature against an
OAuth token, OAuth and MCP step by step, card networks, sign-in providers,
anonymous credentials, and the other schemes with whether each matters now.
…ly signers; registries and key registration corrected
@EugeneDymo
EugeneDymo merged commit 02d6b68 into main Nov 16, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants