Skip to content

fix(mqtt): let a reconnect follow the previous handshake instead of a fixed 15 s - #96

Merged
agessaman merged 3 commits into
observer-firmware-devfrom
fix/mqtt-reconnect-guard-release
Nov 12, 2026
Merged

agessaman merged 3 commits into
observer-firmware-devfrom
fix/mqtt-reconnect-guard-release

Conversation

@agessaman

Copy link
Copy Markdown
Owner

Draft until the source review is back.

What changes

After a link outage every slot is due to reconnect at once, but reconnects were spaced by the 15 s cross-slot guard, so a node took about (slots - 1) x 15 s to recover. First setups already start as soon as the previous slot's handshake is over (2 s minimum gap). A backoff reconnect now gets the same early release when the guard was armed by a first setup or by another reconnect whose handshake has finished: connected and its status published, failed, or the slot lost its client.

Unchanged:

  • Handshakes still run one at a time.
  • The guard still expires at 15 s for a handshake that hangs.
  • Token renewals, slot reconfigures and breaker probes keep the full 15 s guard, both as the armer and as the follower.
  • A first setup still only follows another first setup early.
  • Each slot's own backoff ladder and the breaker are untouched.

Testing

  • 749 native tests pass; four new policy tests cover the release, the minimum gap, an armer that is still connecting or publishing, a non-handshake armer, and clock rollover.
  • Hardware, using a scratch CLI hook that calls esp_wifi_disconnect() (not in this branch). Three drops per board on the V4 and G3, two on the V3. Time from WiFi link-up to the last slot connected:
Board Slots Before After
Heltec V4 (PSRAM) 5 64 s, 64 s, 64 s 18 s, 16 s, 16 s
Station G3 (PSRAM) 4 not measured 13 s, 12 s, 13 s
Heltec V3 (no PSRAM, stock TLS) 2 not measured 7 s, 7 s
  • V3 internal heap after the two cycles: 66 KB free, 48 KB largest block, 44 KB minimum; no TLS allocation failures.

Not tested

  • A reconnect following a handshake that fails or hangs, on hardware (host tests only).
  • Brokers dropping all sessions at once without a link outage.
  • Ethernet (ThinkNode M7).

… fixed 15 s

After a link outage every slot is due at once, but reconnects were spaced by
the 15 s cross-slot guard, so a five-slot node took about a minute to recover
against about 16 s at boot. A reconnect may now start once the handshake that
armed the guard (a first setup or another reconnect) is over and 2 s have
passed, the rule first setups already use. Renewals, reconfigures and breaker
probes keep the full guard, and it still expires at 15 s for a handshake that
hangs.
…attempt

The early release trusts the armer's pending flag, which carries no attempt
identity: the outcome of an older attempt could clear it for a reconnect that
was just accepted. Such a reconnect now arms the guard without an armer, so
the next slot waits the full 15 s as before. Comments updated to match.
… owed

A soft disconnect that times out leaves the old session's DISCONNECTED to arrive
later. If a reconnect was accepted first, that event cleared the new attempt's
pending flag, and the guard could release while its handshake was still running.
Record the owed event per slot and arm the full 15 s guard for a reconnect issued
while one is outstanding.
@agessaman
agessaman marked this pull request as ready for review November 12, 2026 03:42
@agessaman
agessaman merged commit 38cf17b into observer-firmware-dev Nov 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant