Skip to content

feat(eval): surface final_response_match_v2 judge reasoning and pass rubric rationale through details - #279

Merged
krisztianfekete merged 1 commit into
mainfrom
feat/judge-reasoning-details
Nov 16, 2026
Merged

krisztianfekete merged 1 commit into
mainfrom
feat/judge-reasoning-details

Conversation

@krisztianfekete

Copy link
Copy Markdown
Member

final_response_match_v2 failures now explain themselves. ADK asks the judge for its reasoning and then drops it. A small subclass of ADK's FinalResponseMatchV2Evaluator keeps that reasoning and returns it as a rubric score (rubric_id: final_response_match). The score is still ADK's own label parsing. Remove the subclass once ADK returns the reasoning itself.

MetricResult.details now carries rubric scores and rationale for any metric that returns them, not only rubric_based_*. This also adds rationale to safety_v1 and multi_turn_*, which ADK already returns. Metrics that return no rubric scores keep details: null. No score, status, threshold or aggregation changes.

The CLI table and the UI do not show the rationale yet. Use -o json, the API or gRPC.

@krisztianfekete
krisztianfekete merged commit 5715283 into main Nov 16, 2026
9 checks passed
@krisztianfekete
krisztianfekete deleted the feat/judge-reasoning-details branch November 16, 2026 15:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant