Skip to content

FEAT: add raise-on-empty variants to true/false score aggregators - #2984

Merged
Roman Lutz (romanlutz) merged 6 commits into
microsoft:mainfrom
feiiiiii5:fix/tf-aggregator-raise-on-empty
Oct 10, 2026
Merged

Roman Lutz (romanlutz) merged 6 commits into
microsoft:mainfrom
feiiiiii5:fix/tf-aggregator-raise-on-empty

Conversation

@feiiiiii5

Copy link
Copy Markdown
Contributor

Description

Fixes #2883.

Aggregating an empty score list with TrueFalseScoreAggregator today returns value=False with a COMPLETE result, which is indistinguishable from a genuine "veto" verdict, and there is no way to opt into a strict check. The float-scale aggregators already solve this with raise_on_empty variants (AVERAGE_RAISE_ON_EMPTY, MAX_RAISE_ON_EMPTY, MIN_RAISE_ON_EMPTY in float_scale_score_aggregator.py); this adds the same three opt-in variants to the true/false family — AND_RAISE_ON_EMPTY, OR_RAISE_ON_EMPTY, MAJORITY_RAISE_ON_EMPTY — raising ValueError("No scores available for aggregation") on empty input. Default behavior is unchanged.

Tests and Documentation

Test: pytest tests/unit/score/test_true_false_score_aggregator.py — the 7 new tests fail on main (AttributeError: type object 'TrueFalseScoreAggregator' has no attribute 'AND_RAISE_ON_EMPTY') and pass on this branch; the existing tests, including the pinned empty-input default, are unchanged and pass. Also ran ruff format --check and ruff check (v0.16.9) on the touched files.

No JupyText notebooks affected; the float-scale *_RAISE_ON_EMPTY variants are likewise not listed in doc/code/scoring/3_combining_scorers.py.

@romanlutz Roman Lutz (romanlutz) self-assigned this Oct 9, 2026
Roman Lutz (romanlutz) and others added 5 commits October 9, 2026 06:36
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
…erings

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@romanlutz
Roman Lutz (romanlutz) added this pull request to the merge queue Oct 9, 2026
Merged via the queue into microsoft:main with commit de2f04a Oct 10, 2026
55 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Should the true/false aggregators get a strict-empty variant, like the float ones?

2 participants