Skip to content

Add the SearchApi adapter for Google web, news and Scholar search - #1776

Open
gabsferreira wants to merge 1 commit into
HelpCode-ai:mainfrom
gabsferreira:feat/searchapi-adapter
Open

gabsferreira wants to merge 1 commit into
HelpCode-ai:mainfrom
gabsferreira:feat/searchapi-adapter

Conversation

@gabsferreira

Copy link
Copy Markdown

Disclosure: I work at SearchApi (developer advocate). This adapter wraps our own API, so please read it with that in mind; I kept it to what the API verifiably does today.

Summary

Adds SearchApi (https://www.searchapi.io) to the catalog as a data adapter with three read-only tools, following the shape of the Serply adapter (#765): Google web search, Google News and Google Scholar through one endpoint, GET /api/v1/search, with a fixed engine per tool.

  • searchapi_google_search (engine=google): organic_results[] with position, title, link, snippet, source, domain, displayed_link, date, plus knowledge_graph, related_questions and related_searches when Google shows them.
  • searchapi_google_news (engine=google_news): articles with title, link, snippet, source, date, iso_date.
  • searchapi_google_scholar (engine=google_scholar): papers with title, link, snippet, publication, authors, cited_by and the direct document resource link when Scholar lists one.

Parameters are the ones the SearchApi docs list for these engines and that I verified live: q, num (1 to 10, higher values are clamped by the API), page (1-based), gl, hl (no gl on Scholar). The key goes out as Authorization: Bearer (BEARER_TOKEN, like the Tavily adapter), never in the URL. Response mappings trim search_metadata and search_parameters and keep the useful blocks, in the style of the SerpApi adapter. A new SearchApi account has 100 free requests; every call, including the install probe, spends one, and the instructions say so.

Files

File Change
packages/backend/src/adapters/intl/searchapi.json New adapter: 3 tools, envVarMeta, verifyHints (401, 429), probe, prerequisites, response mappings
packages/backend/src/adapters/intl/searchapi.live.spec.ts New spec: static assertions always, live calls behind RUN_SEARCHAPI_LIVE=1
packages/frontend/public/logos/connectors/searchapi.svg Official SearchApi mark (favicon.svg from searchapi.io) with role="img" and a <title> like the other connector logos; no scripts
packages/backend/src/adapters/catalog.ts Regenerated by scripts/regenerate-catalog.mjs (+2 lines)
README.md, README.de.md, README.ja.md, README.zh-CN.md Adapter count 877 to 878, where scripts/adapter-count.mjs --check pointed
CITATION.cff, glama.json, package.json, server.json Same count bump
llms.txt, llms-full.txt Regenerated by scripts/generate-llms-txt.mjs after the count change

No change under ee/, no change to workflows, no change to the generated docs/catalog/ pages (they come from the website repository).

How it mirrors #765 (Serply)

Same three search verticals (web, news, Scholar), same data category, intl region, API-key auth through requiredEnvVars, one probe on the web search tool, logo under public/logos/connectors/, catalog registration, count bump in the README translations, CITATION, glama, server.json and package.json. What is different is what the repo now asks for that did not exist in September: envVarMeta with a key pattern, verifyHints, prerequisites, response mappings, a spec that drives the real RestEngine with a mocked axios, and the regenerated llms.txt files, as in #1673 (Strava) and #1732 (OpenCart Bridge).

Tests

All run from a fresh npm ci on Node 24.13.0, following AGENTS.md:

Command (working directory) Result
node scripts/validate-adapters.mjs --warn (root) exit 0, no warning for searchapi.json (3 pre-existing warnings on other adapters)
node scripts/regenerate-catalog.mjs (root) 894 adapters across 21 regions, catalog.ts +2 lines, no further diff after the backend prebuild
node scripts/adapter-count.mjs --check (root) OK: every quoted count equals 878 adapters (46 keyless, 17785 tools).
node scripts/generate-llms-txt.mjs --check (root) exit 0
npm run lint:check (packages/backend) 0 errors, 11 warnings, all pre-existing, none in the new files
npx tsc --noEmit -p tsconfig.json (packages/backend) exit 0
npx jest src/adapters/intl/searchapi.live.spec.ts src/adapters/catalog*.spec.ts (packages/backend) Test Suites: 5 passed, 5 total / Tests: 4 skipped, 20912 passed, 20916 total
RUN_SEARCHAPI_LIVE=1 SEARCHAPI_API_KEY=... npx jest src/adapters/intl/searchapi.live.spec.ts (packages/backend) Tests: 21 passed, 21 total (17 static, 4 live: wrong key answers 401 Invalid API key., one real search per tool, 3 requests spent)
npm test (packages/backend, no local Postgres) Test Suites: 3 failed, 13 skipped, 1288 passed, 1291 of 1304 total / Tests: 25 failed, 2304 skipped, 43112 passed, 45441 total. The 3 failures are environmental and unrelated to adapters: auth/jwt-or-management-key.guard.spec.ts and ee/learning/learning.database.spec.ts need a real database (Database does not exist on the database server), and connectors/engines/download-follow.util.spec.ts hit its 5 s timeout under the full-suite load and passed on npx jest --onlyFailures. CI runs these with its Postgres and Redis services.
npm run build (packages/backend) exit 0, dist/src/main.js present
npm run lint and npx tsc --noEmit -p tsconfig.json (packages/frontend) 0 errors, 52 warnings (pre-existing) and exit 0

Not run: Playwright e2e and the Docker image boot (the change adds one static SVG on the frontend side and no dependency or Dockerfile change, so CI skips the image boot for this PR by its own rule).

A note on redirect links

On web and news results Google sometimes returns link as https://www.google.com/goto?url=<opaque token> instead of the publisher URL; in my live checks this hit most lower-ranked web rows and every news row. The row's domain and displayed_link (web) or source (news) still carry the real site. The adapter passes these links through unchanged and the instructions tell the model not to try to decode them; the static spec pins that behaviour with an example.com fixture. If you would rather have such rows dropped or the host surfaced differently, I am happy to change it.

Happy to adjust anything else too: tool names (searchapi_google_* versus Serply's shorter serply_*), the amount of trimming in the response mappings, or the wording of the instructions.

🤖 Generated with Claude Code

@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@github-actions

Copy link
Copy Markdown

👋 Welcome, @gabsferreira, and thanks for opening your first PR on AnythingMCP!

A few quick pointers:

  • Make sure CI is green before requesting review (Backend, Frontend, Playwright, CodeQL, Trivy).
  • If this is a new adapter, the parametrised catalog.spec.ts test will validate it automatically.
  • Sign off your commits if you can — it's not blocking, just nice to have.

Someone from the core team will look at this within ~48h. If you don't hear back, please ping us in Discussions / Q&A.

⭐ While you wait — if you find AnythingMCP useful, a star helps others discover it.

@gabsferreira
gabsferreira force-pushed the feat/searchapi-adapter branch from beca5c2 to 461d5f0 Compare November 17, 2026 18:37
@gabsferreira

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

github-actions Bot added a commit that referenced this pull request Nov 17, 2026
SearchApi (www.searchapi.io) returns Google results as JSON behind one
endpoint, GET /api/v1/search, with the engine as a parameter. The adapter
wraps three engines as read-only tools, mirroring the Serply adapter
(HelpCode-ai#765): searchapi_google_search (engine=google), searchapi_google_news
(engine=google_news) and searchapi_google_scholar (engine=google_scholar),
with q, num (1-10, clamped by the API), page (1-based), gl and hl.

The key travels as Authorization: Bearer (BEARER_TOKEN), never in the URL.
Response mappings drop search_metadata and search_parameters and keep the
organic rows with title, link, snippet, source, domain, displayed_link and
date (web), source and iso_date (news), publication, authors, cited_by and
the resource link (Scholar), plus knowledge_graph, related_questions and
related_searches on web search. Google sometimes answers lower-ranked web
rows and news rows with a google.com/goto?url= redirect instead of the
publisher link; the row's domain, displayed_link or source still name the
site, so links pass through unchanged and the instructions say so.

A new account has 100 free requests and every call spends one, which the
instructions and tool descriptions state. The spec pins the auth header,
the fixed engine per tool, every parameter reaching the query, read-only
annotations and the trimming against the shapes returned on 17 Nov 2026;
live calls run behind RUN_SEARCHAPI_LIVE=1.

Catalog regenerated, counts bumped to 878, llms.txt regenerated, logo
added under public/logos/connectors.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@gabsferreira
gabsferreira force-pushed the feat/searchapi-adapter branch from 461d5f0 to ff2430c Compare November 18, 2026 00:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant