feat(query): rerank by a decision model with Rerank.decide by jfrancoa · Pull Request #2198 · weaviate/weaviate-python-client

jfrancoa · 2026-10-12T21:59:25Z

What

rerank=Rerank.decide(...): rerank by a decision model (server: weaviate/weaviate#13528 and the RFC section "rerank with a decision model"). The collection's decisions module answers the query as a yes/no question about every candidate; the probability is metadata.rerank_score, candidates below min_probability are dropped and order keeps the search order or sorts by probability.

from weaviate.classes.query import DecisionsOrder, Rerank

res = collection.query.hybrid(
    "refund not received",
    limit=20,
    rerank=Rerank.decide(prop="text", query="The customer is angry.", min_probability=0.6, order=DecisionsOrder.PROBABILITY),
)
for o in res.objects:
    o.metadata.rerank_score  # the probability

collection.query.hybrid("refund", rerank=Rerank.decide(prop="text", query="The customer is angry."))  # class settings apply
collection.query.hybrid("refund", rerank=Rerank(prop="text", query="refund"))  # the reranker, if the collection has one

min_probability and order are optional; an option left unset takes the class setting (min_probability, order of Configure.Decisions.*). rerank=Rerank(...) keeps going to the collection's reranker, or to the decisions module as a relevance predicate when there is no reranker. rerank=Rerank.decide(...) on a collection without a decisions module is a server error. The page can hold fewer objects than limit: the model only removes candidates.

The gRPC stubs are regenerated with the Rerank.decision sub-message.

The rerank argument keeps the Rerank namespace (review of 2026-10-14): Rerank(prop, query) for a reranker, Rerank.decide(prop, query, min_probability, order) for a decision model. The factory returns a private subclass of Rerank, so every rerank: Optional[Rerank] signature accepts it; order is a DecisionsOrder, the same enum Configure.Decisions.* takes for the class default (exported from classes.config and classes.query), or its value. There is no RerankOrder: the option belongs to the decisions module, not to every reranker. The type Decision.predicate/choice/score return is DecisionQuestion, as its answer is DecisionAnswer (renamed from DecideQuestion here). The question setting of Configure.Decisions.openai is gone with the server's (weaviate/weaviate#13534): a plain Rerank(...) asks relevance, Rerank.decide(...) sends the statement.

Tests

Mock: the request carries decision with the set fields only; a plain Rerank carries none; min_probability out of [0, 1] and a blank query are rejected client-side; order given as a string is sent as the enum.
Integration (decisions-dummy, skips without it): class defaults, threshold, order by probability, the plain rerank fallback, class settings overridden by the request, the error on a reranker-only collection, and a collection with both a reranker and a decisions module.

Stacked on #2189.

orca-security-eu

Orca Security Scan Summary

Status	Check	Issues by priority
Passed	Secrets	0 0 0 0	View in Orca

rerank=Decide(prop, query, min_probability, order) sends the question to the collection's decisions module: the probability is the rerank score, candidates below min_probability are dropped and order keeps the search order or sorts by probability. An option left unset takes the class setting. rerank=Rerank(...) keeps going to the reranker, or to the decisions module as a relevance predicate when the collection has no reranker. The gRPC stubs carry the new Rerank.decision sub-message.

Rerank leaves the query optional for the reranker; a decision has nothing to judge without one, and the server only fails after the search.

The server would run the search and then fail with 'no query provided'.

codecov-commenter · 2026-10-13T17:25:04Z

Codecov Report

❌ Patch coverage is 72.16495% with 27 lines in your changes missing coverage. Please review.
⚠️ Please upload report for BASE (jose/decisions-openai@7cc18b0). Learn more about missing BASE report.

Files with missing lines	Patch %	Lines
integration/test_collection_decisions.py	40.00%	27 Missing ⚠️

Additional details and impacted files

@@                   Coverage Diff                    @@
##             jose/decisions-openai    #2198   +/-   ##
========================================================
  Coverage                         ?   88.78%           
========================================================
  Files                            ?      310           
  Lines                            ?    24807           
  Branches                         ?        0           
========================================================
  Hits                             ?    22024           
  Misses                           ?     2783           
  Partials                         ?        0

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

The rerank argument keeps the Rerank namespace: Rerank(prop, query) goes to the reranker, Rerank.decide(prop, query, min_probability, order) to the decisions module. The Decide class and DecideOrder are gone; the order enum is RerankOrder, as in the proto.

…uilt The user sees the factory's name in the error, not the private class. RerankOrder is a BaseEnum like the other enums; the Rerank docstring names the decisions-module fallback; a mock test sends the order as a string.

Copilot

🟡 Changes recommended

The public order type annotation excludes the supported raw-string enum values.

1 open finding

Include str in enum parameter annotation · New

What changed in this PR

Adds decision-model reranking through Rerank.decide, including validation, gRPC serialization, exports, documentation, and tests.

Changes:

Adds RerankOrder and the Rerank.decide factory.
Regenerates protobuf bindings with decision-reranking fields.
Adds mock and integration coverage.

File	Description
`weaviate/proto/v1/v6300/v1/search_get_pb2.pyi`	Adds decision-reranking types.
`weaviate/proto/v1/v6300/v1/search_get_pb2.py`	Updates generated descriptors.
`weaviate/proto/v1/v5261/v1/search_get_pb2.pyi`	Adds decision-reranking types.
`weaviate/proto/v1/v5261/v1/search_get_pb2.py`	Updates generated descriptors.
`weaviate/proto/v1/v4216/v1/search_get_pb2.pyi`	Adds decision-reranking types.
`weaviate/proto/v1/v4216/v1/search_get_pb2.py`	Updates generated descriptors.
`weaviate/collections/grpc/query.py`	Serializes decision rerank options.
`weaviate/collections/classes/internal.py`	Corrects decision terminology.
`weaviate/collections/classes/grpc.py`	Defines the public rerank API.
`weaviate/collections/classes/config.py`	Updates decisions configuration documentation.
`weaviate/classes/query.py`	Exports `RerankOrder`.
`mock_tests/test_decisions.py`	Tests validation and request serialization.
`integration/test_collection_decisions.py`	Tests end-to-end decision reranking.

Files not reviewed (3)

weaviate/proto/v1/v4216/v1/search_get_pb2.py: Generated file
weaviate/proto/v1/v5261/v1/search_get_pb2.py: Generated file
weaviate/proto/v1/v6300/v1/search_get_pb2.py: Generated file

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

+        query: str,
+        *,
+        min_probability: Optional[float] = None,
+        order: Optional[RerankOrder] = None,


DecisionsOrder is the order a decisions module reranks in: the class default of Configure.Decisions.* and the override of Rerank.decide take the same enum, exported from classes.config and classes.query. RerankOrder is gone: it read as an option of every reranker. The type a Decision.* factory returns is DecisionQuestion, as its answer is DecisionAnswer. The question setting of Configure.Decisions.openai is gone with the server's: a plain Rerank asks relevance, Rerank.decide sends the statement.

…op the question setting test/collection/test_config.py still imported DecisionsQuestion and failed at collection. Rerank.decide now accepts a DecisionsOrder or its value and raises WeaviateInvalidInputError for anything else, as it does for a blank query.

Copilot

🟡 Changes recommended

The public order type annotation rejects the documented and supported string values.

2 open findings

Files not reviewed (3)

weaviate/proto/v1/v4216/v1/search_get_pb2.py: Generated file
weaviate/proto/v1/v5261/v1/search_get_pb2.py: Generated file
weaviate/proto/v1/v6300/v1/search_get_pb2.py: Generated file

🧠 Review effort: Balanced

+        query: str,
+        *,
+        min_probability: Optional[float] = None,
+        order: Optional[DecisionsOrder] = None,


orca-security-eu Bot reviewed Oct 12, 2026

View reviewed changes

This was referenced Oct 12, 2026

feat(rerank): rerank by a decision model with rerank.decision weaviate/weaviate#13528

Draft

feat(decisions): support decisions modules and the decide query argument #2188

Draft

jfrancoa force-pushed the jose/rerank-decide-client branch from c2d2b86 to ff535fe Compare October 13, 2026 13:21

jfrancoa force-pushed the jose/decisions-openai branch from 941bd58 to 933bcc4 Compare October 13, 2026 13:21

jfrancoa added 3 commits October 13, 2026 15:23

fix(query): Decide needs the statement it judges

fb4f4b9

Rerank leaves the query optional for the reranker; a decision has nothing to judge without one, and the server only fails after the search.

test(query): read the plain rerank scores by text, not by position

b8cc752

jfrancoa force-pushed the jose/decisions-openai branch from 933bcc4 to 7cc18b0 Compare October 13, 2026 13:24

jfrancoa force-pushed the jose/rerank-decide-client branch from ff535fe to b8cc752 Compare October 13, 2026 13:24

jfrancoa mentioned this pull request Oct 13, 2026

docs(decisions): questions and Rerank.decide take an image in a blob property #2204

Draft

fix(query): an empty Decide query is refused too

0ae0430

The server would run the search and then fail with 'no query provided'.

jfrancoa changed the title ~~feat(query): rerank by a decision model with Decide~~ feat(query): rerank by a decision model with Rerank.decide Oct 14, 2026

jfrancoa requested a balanced review from Copilot October 14, 2026 12:48

Copilot started reviewing on behalf of jfrancoa October 14, 2026 12:50

View session

Copilot AI reviewed Oct 14, 2026

View reviewed changes

Comment thread weaviate/collections/classes/grpc.py Outdated

query: str,

*,

min_probability: Optional[float] = None,

order: Optional[RerankOrder] = None,

jfrancoa added 2 commits October 14, 2026 15:14

jfrancoa requested a balanced review from Copilot October 14, 2026 13:49

Copilot started reviewing on behalf of jfrancoa October 14, 2026 13:50

View session

Copilot AI reviewed Oct 14, 2026

View reviewed changes

Comment thread weaviate/collections/classes/grpc.py

query: str,

*,

min_probability: Optional[float] = None,

order: Optional[DecisionsOrder] = None,

Navigation Menu

Repository navigation

feat(query): rerank by a decision model with Rerank.decide - #2198

feat(query): rerank by a decision model with Rerank.decide#2198
jfrancoa wants to merge 8 commits into
jose/decisions-openaifrom
jose/rerank-decide-client

jfrancoa commented Oct 12, 2026 •

edited

Loading

Uh oh!

orca-security-eu Bot left a comment •

edited

Loading

Uh oh!

codecov-commenter commented Oct 13, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

jfrancoa commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Tests

Uh oh!

orca-security-eu Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Orca Security Scan Summary

Uh oh!

codecov-commenter commented Oct 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

jfrancoa commented Oct 12, 2026 •

edited

Loading

orca-security-eu Bot left a comment •

edited

Loading

codecov-commenter commented Oct 13, 2026 •

edited

Loading