You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
repo compare: baseline_mean and treatment_mean are None for finite scores whose plain sum overflows #866
sign_flip_summary (dokimi/dokimi/repos/compare.py) computes baseline_mean and treatment_mean with _mean, a plain sum(nums) / len(nums), and _round turns a non-finite result into None. Repro: sign_flip_summary([{"task": f"t{i}", "baseline": 1.7e308, "treatment": 1.7e308, "delta": 0.0} for i in range(3)], "metric:x") returns baseline_mean None and treatment_mean None although both means are 1.7e308 (the deltas are 0, so the checked_mean refusal of the deltas does not apply). Fix: average with statistics.mean when the plain sum is not finite and every value is, as eval/arm_summary.mean_in_range and eval/stats._mean do.
Activity