Skip to content

feat(decisions): support decisions modules and the decide query argument - #2188

Draft
jfrancoa wants to merge 3 commits into
mainfrom
jose/decisions-typesafeai
Draft

jfrancoa wants to merge 3 commits into
mainfrom
jose/decisions-typesafeai

Conversation

@jfrancoa

@jfrancoa jfrancoa commented Oct 8, 2026 •

Copy link
Copy Markdown
Contributor

What

Client support for the decisions-* module type and the decide query argument, starting with decisions-typesafeai.

Server side: weaviate/weaviate#13477 (module type), weaviate/weaviate#13408 (decisions-typesafeai), weaviate/weaviate#13484 (decide property, proto and decisions-dummy). This PR carries the regenerated decisions_pb2 stubs from that proto. Support for decisions-openai comes in a stacked PR (#2189), and rerank=Decide(...) in #2198.

Collection configuration

from weaviate.classes.config import Configure, Reconfigure, DecisionsOrder

client.collections.create(
    "Tickets",
    reranker_config=Configure.Reranker.cohere(),  # optional: a reranker next to the decisions module
    decisions_config=Configure.Decisions.typesafeai(
        min_probability=0.5,               # rerank: drop results below this probability
        order=DecisionsOrder.PROBABILITY,  # rerank order: by probability instead of search order
    ),
)
collection.config.update(decisions_config=Reconfigure.Decisions.typesafeai(min_probability=0.7))
collection.config.get().decisions_config  # DecisionsConfig(decisions="decisions-typesafeai", model={...})

A collection has at most one reranker and at most one decisions module, and an update replaces the module of its own kind only. Configure.Decisions.custom(name, settings) covers a module the client does not know. The settings are model, base_url, max_documents, min_probability and order; the module batches requests by itself.

Queries

from weaviate.classes.query import Decision

res = collection.query.hybrid(
    "refund not received",
    decide=[
        Decision.predicate("angry", prop="text", instructions="Is the customer angry?"),
        Decision.choice("team", prop="text", instructions="Which team should handle this?",
                        options={"billing": "money and invoices", "support": None}),
        Decision.score("severity", prop="text", instructions="How severe is the complaint?",
                       levels=["low", "medium", "high"]),
    ],
)
for o in res.objects:
    o.decisions["angry"].probability      # 0.91
    o.decisions["team"].choice            # "billing", plus .probabilities and .confidence
    o.decisions["severity"].score         # 2.0 (position between the levels), plus .probabilities
    o.decisions["angry"].refused          # True when the model declined to answer

decide= is accepted by every search method and by fetch_objects, and can be combined with rerank=. It never drops or reorders an object. It is not available with group_by.

Tests

  • Unit: configuration serialisation, a reranker next to a decisions module, update replacement within a kind, read-back, argument validation.
  • Mock: a Search reply with every answer kind is parsed into DecisionAnswer; the request carries the questions.
  • Integration (integration/test_collection_decisions.py): runs against the server's decisions-dummy module, whose answers are a function of the text; skips when the server lacks the module. Verified locally against a build of feat(modules): add the decide additional property weaviate#13484. ci/docker-compose.yml enables the module, so the tests start running once a server image ships it.

Weaviate is adding a `decisions-*` module type, starting with
decisions-typesafeai. A decisions module serves `rerank` like a reranker
and also answers questions about every object a query returns.

Collection configuration:
- `Configure.Decisions.typesafeai(...)` and `Configure.Decisions.custom(...)`
  build a `decisions_config`, accepted by `collections.create()` and
  `collection.config.update()`. A collection has either a reranker or a
  decisions module: passing both is rejected, and an update replaces
  whichever the collection has.
- `config.get()` returns `decisions_config` next to `reranker_config`.

Queries:
- `decide=` on every search method and on `fetch_objects` takes one or
  more questions built with `Decide.predicate`, `Decide.choice` or
  `Decide.score`.
- Each returned object carries `decisions`, a mapping of question name to
  `DecisionAnswer` with the probability, choice, score, per-option
  probabilities, confidence, or a refusal.

The integration tests run against the server's decisions-dummy module and
skip when the server does not have it.

@orca-security-eu orca-security-eu Bot left a comment •

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Orca Security Scan Summary

Status Check Issues by priority
Passed Passed Infrastructure as Code high 0   medium 0   low 0   info 0 View in Orca
Passed Passed SAST high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Secrets high 0   medium 0   low 0   info 0 View in Orca
Passed Passed Vulnerabilities high 0   medium 0   low 0   info 0 View in Orca

@codecov-commenter

codecov-commenter commented Oct 8, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 83.08271% with 45 lines in your changes missing coverage. Please review.
✅ Project coverage is 88.95%. Comparing base (95b5d76) to head (f1a6600).
⚠️ Report is 219 commits behind head on main.

Files with missing lines Patch % Lines
integration/test_collection_decisions.py 30.15% 44 Missing ⚠️
weaviate/collections/grpc/query.py 95.23% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2188      +/-   ##
==========================================
+ Coverage   86.64%   88.95%   +2.31%     
==========================================
  Files         300      310      +10     
  Lines       23172    24692    +1520     
==========================================
+ Hits        20077    21965    +1888     
+ Misses       3095     2727     -368     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

…; drop the cache and over-fetch settings

The server keeps at most one reranker and one decisions module per
collection and an update replaces a module of its own kind only, so the
client no longer rejects reranker_config next to decisions_config and
merge_with_existing clears the module of the same kind only.

decisions-typesafeai no longer has batchSize, fetchDepth, cache,
scoreLevels and minScore: the module batches by default, the answer
cache is gone and over-fetch and the rubric rerank are not in this
version.

The decide question factory is Decision (Decision.predicate, .choice,
.score); Decide becomes the rerank input of a decision model in a later
change.
Duplicate option or level names, an empty decide and decide with
group_by are rejected client-side with the reasons the server would
give. The decide docstrings and error name the Decision factory.
@jfrancoa
jfrancoa force-pushed the jose/decisions-typesafeai branch from e106f58 to 2275aaf Compare October 13, 2026 13:24

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants