Skip to content

fix(cli): guard miner score _drain_logs against Warning-state crash#1374

Merged
anderdc merged 2 commits into
entrius:testfrom
statxc:fix/miner-score-drain-logs-warning-state
May 29, 2026
Merged

fix(cli): guard miner score _drain_logs against Warning-state crash#1374
anderdc merged 2 commits into
entrius:testfrom
statxc:fix/miner-score-drain-logs-warning-state

Conversation

@statxc
Copy link
Copy Markdown
Contributor

@statxc statxc commented May 26, 2026

Summary

gitt miner score --log-level warning crashed at shutdown with
statemachine.exceptions.TransitionNotAllowed: Can't disable_logging when in Warning.
This guards the _drain_logs cleanup path so every value Click already accepts in
--log-level completes cleanly.

Closes #1373

Root cause

bittensor's LoggingMachine defines disable_logging transitions from
Trace/Debug/Default/Disabled/Info but not from Warning — the only
shutdown transition out of Warning is disable_warning = Warning.to(Default)
(bittensor/utils/btlogging/loggingmachine.py:127-137). Any caller that lands in
Warning and then invokes bt.logging.off() raises TransitionNotAllowed.

gitt miner score exposes warning via click.Choice([...]), calls
_apply_log_level('warning')bt.logging.set_warning(), then unconditionally
calls bt.logging.off() in _drain_logs(). Deterministic crash, not flaky.

Fix

Single guarded transition in _drain_logs(): if the LoggingMachine is in
Warning, step down to Default via set_warning(on=False) first, then call
off() (which is defined from Default). This preserves the helper's existing
contract — before_disable_logging sets all stdlib loggers to CRITICAL — that
a naked skip-when-Warning would silently break.

State check follows bittensor's own internal idiom
(set_debug/set_trace/set_info/set_warning all compare
current_state_value before transitioning) and avoids using exceptions for
control flow.

A/B reproduction

Before:
info exit=0
debug exit=0
trace exit=0
warning exit=1 ← TransitionNotAllowed in _drain_logs

After:
info exit=0
debug exit=0
trace exit=0
warning exit=0

Tests

New TestDrainLogsAcrossAdvertisedLevels in tests/cli/test_miner_score.py:

  • Parametrized over all four advertised levels (warning/info/debug/
    trace) — pins the regression and prevents drift if Click's --log-level
    choices ever expand.
  • Disabled.to(Disabled) idempotency — a second drain after a successful
    drain must not raise.
  • Warning → Disabled contract — proves the fix doesn't just sidestep the
    crash; the machine actually ends in Disabled so logging is genuinely off.

A restore_bt_logging_state fixture snapshots current_state_value before each
test and restores it on teardown — the LoggingMachine is a process-wide
singleton and would otherwise leak into the 867 other tests.

Test plan

  • uv run --extra dev pytest tests/cli/test_miner_score.py -v → 21 passed
    (6 new + 15 existing)
  • uv run --extra dev pytest tests/ -x -q → 873 passed
  • uv run ruff check . / uv run ruff format --check . → clean
  • uv run --extra dev pyright gittensor tests → 0 errors
  • uv run --extra dev vulture → no findings
  • Manual repro: A/B across all four --log-level values, all exit 0

Copy link
Copy Markdown
Collaborator

@anderdc anderdc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guard in gittensor/cli/miner_commands/score.py is the fix and it's correct. The 50+ lines added to tests/cli/test_miner_score.py — the parametrized test class, the idempotency and lands-in-Disabled cases, and the process-wide restore_bt_logging_state singleton fixture — are disproportionate for a two-line guard. Keep: the score.py guard. Drop: the additions to tests/cli/test_miner_score.py.

@statxc statxc force-pushed the fix/miner-score-drain-logs-warning-state branch from b70809c to cd9aa85 Compare May 29, 2026 21:38
bittensor's LoggingMachine defines `disable_logging` transitions from
Trace/Debug/Default/Disabled/Info but NOT from Warning (only
`disable_warning = Warning.to(Default)`), so `gitt miner score
--log-level warning` crashed at shutdown when `_drain_logs` invoked
`bt.logging.off()`. Step Warning down to Default first to preserve the
helper's "logging off on return" contract while clearing the crash.
@statxc statxc force-pushed the fix/miner-score-drain-logs-warning-state branch from cd9aa85 to 799f061 Compare May 29, 2026 21:40
@statxc
Copy link
Copy Markdown
Contributor Author

statxc commented May 29, 2026

Hi, @anderdc, I've removed. Thanks for the review.

@anderdc anderdc merged commit b2f2d18 into entrius:test May 29, 2026
3 checks passed
@statxc
Copy link
Copy Markdown
Contributor Author

statxc commented May 29, 2026

Thanks @anderdc

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] gitt miner score --json --log-level warning crashes in _drain_logs()bt.logging.off() raises TransitionNotAllowed from Warning state

2 participants