Repository navigation
Add SubQ backend with per-backend request limits; release 0.10.0 - #30
Merged
Merged
Conversation
Add a subq backend for SubQ's decision model: judgments go to api.subq.ai under /v1/systemone with the key from SUBQ_API_KEY and the model subq-snap-preview, priced at $0.05 per million input tokens. Its model list is public, so listModels() there does not verify the key. SubQ accepts at most 16 questions in one request, a Choice of at least 2 options, and a Score of at most 10 levels, where the schema allows 32, 1, and 32. A registry entry or a caller-supplied endpoint can now declare these limits; a backend that declares none keeps the schema's bounds, and no limit can be above them. evaluate refuses a request over its backend's limits with a validation error before sending it, evaluateAll and chunkEvaluationRequest chunk to the backend's question limit and cap a larger maxQuestions, and client.limits and resolveBackend().limits expose the limits without a request.
DevMortimer
marked this pull request as ready for review
October 15, 2026 01:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
subqbackend for SubQ's decision model.createTypeSafe({ backend: "subq" })sends judgments tohttps://api.subq.ai/v1/systemonewithAuthorization: Bearer $SUBQ_API_KEYand the modelsubq-snap-preview. The usage ledger prices it at $0.05 per million input tokens. The model list at/v1/modelsis public, solistModels()does not verify the key there.BackendLimits:maxQuestions,minChoiceOptions,maxChoiceOptions,maxScoreLevels). A backend that declares none keeps the schema's bounds (DEFAULT_BACKEND_LIMITS: 32 questions, 1–64 options, 32 levels). No limit can be above the schema's bound.subqdeclares 16 questions, at least 2 Choice options, and at most 10 Score levels, per SubQ's Decisions API docs.evaluate(and soevaluateManyandevaluateAll) refuses a request over the backend's limits before it sends it, with the samevalidationerror kind the schema uses. The refused request does not count toward the request budget.evaluateAllchunks to the client's question limit.chunkEvaluationRequesttakes abackendoption and chunks to that backend's limit. AmaxQuestionsabove the limit is capped.client.limitsorresolveBackend(spec).limits. ABackendEndpointcan declarelimits, and they are validated like the other endpoint fields.package.json,package-lock.json,CHANGELOG.md), as the maintainer asked.Why
SubQ serves the same decisions protocol but accepts at most 16 questions in one request. Without limits, any caller that sends 17–32 questions would get a 422 from SubQ.
Behaviour of existing backends
typesafe,openrouter,commandcode, andliquiddeclare no limits. They keep the schema's bounds, so they accept the same requests as before. There is one change in chunking:chunkEvaluationRequest(request, { maxQuestions: 50 })now returns chunks of 32, not 50. Chunks of 50 were refused by the schema anyway.Verification
npm run check: build, typecheck, and 169 of 169 tests pass. That includes 12 new tests intests/subq.test.tsand new limit tests intests/backends.test.tsandtests/batch.test.ts.dist/with a dummySUBQ_API_KEYand sent requests to the real api.subq.ai:npm run test:livewas not run, because no key was available. The transport change is limited to one case: the model-list rewrite now also applies when a backend namesmodelsFieldwithoutmodelsPath, and onlysubqdoes that in the registry. The tests and the live 401 and model-list calls above cover it. Response validation is unchanged.