Skip to content

bug(scheduler): rust-preview can skip catch-up and lose deadline or retry wakes #6827

Open

Description

@kesonm

What happened

On rust-preview, scheduled tasks can miss catch-up execution or remain idle without a timer to reconsider them. This report groups three related scheduler paths, with separate evidence and acceptance cases below.

1. Resume ignores the selected catch-up policy. With Misfire::Latest, pausing an interval task before its first trigger and resuming after that trigger advances directly to a future occurrence. An overdue one-shot task instead returns schedule has no remaining trigger. In contrast, recover() on an active task honors Latest and leaves the missed occurrence claimable. Expected: resume should honor the selected policy, or explicitly document and expose a different pause/resume policy.

2. A deadline crossed during a scheduler tick can lose its timer wake. load_ready(at) selects deadlines at or before the tick's initial time. The final timer selection uses Excluded(now()) with a later time. A deadline between these two times is selected by neither operation. With no other events or future deadlines, the loop waits indefinitely even though an active task is due. Expected: an unprocessed overdue entry schedules an immediate next pass.

3. A transient pre-admission dispatch failure has no timed retry. Dispatch errors retain the original pending fire and are logged, but no retry deadline is added. A pending task without an expiry contributes no deadline to the ready index. After available change notifications are consumed, an otherwise idle owner can wait indefinitely even after the transient failure clears. Expected: bounded/backoff reconciliation of the original fire, preserving its identity and existing admission deduplication.

How to reproduce

1. Pure scheduler state machine — reproduced

Using the scheduler engine source at the commit below:

  1. Create a plan at timestamp 0, timezone UTC, with Schedule::Interval { every_seconds: 10, start_at: 1000 }, no expiry and no fire limit. Timestamps here are milliseconds.
  2. Set plan.misfire = Misfire::Latest.
  3. Call plan.pause(500).resume(25000).
  4. Inspect next_fire_at and call claim(25000).

Observed:

latest_resume next=Some(31000), due_now=false

Control: call recover(25000) on the same active plan instead of pause/resume:

latest_recover due_now=true

Repeat pause/resume with Schedule::Once { run_at: 1000 } and the same catch-up policy:

latest_once_resume=Err(Invalid("schedule has no remaining trigger"))

Validation boundary: the unmodified scheduler policy source was compiled in an isolated harness, with a dummy valid effect and stand-ins for external fact types (RunId and Patch) to avoid compiling the full application. This was not a Desktop UI test.

2. Deadline crossing — index logic reproduced; full owner test still needed

Use one authorized active task due at 1001, no pending fire and no expiry. Start an otherwise idle tick at 1000; let index/storage reading cross the deadline so final timer selection happens at 1002. No writes or unrelated notifications are required in this scenario.

The production selection expressions reduce to:

use std::collections::BTreeMap;
use std::ops::Bound::{Excluded, Unbounded};

let due = BTreeMap::from([(1001_i64, "task")]);
assert_eq!(due.range(..=1000).count(), 0); // initial load_ready
assert!(due.range((Excluded(1002), Unbounded)).next().is_none());
assert_eq!(due.len(), 1); // due task remains in the index

These assertions pass. The production delay == None branch waits for external notifications instead of scheduling another tick. A regression test should control the clock across the actual owner/storage boundary.

3. Transient dispatch failure — static control-flow finding

  1. Use one authorized non-reminder task, with no expiry or other scheduled deadlines.
  2. Let it be claimed, then inject a transient error before message admission, e.g. in the durable queue lookup.
  3. Keep the error through any immediate pass caused by the claim's own storage notification, until existing notifications are consumed.
  4. Clear the injected fault without changing storage or emitting a run/scheduler notification.

The error path preserves pending, due_at() yields no deadline, and the scheduler has no retry timer. The original fire is reconsidered only when another event wakes the owner (or it restarts). This scenario has not been fault-injected in the complete application; it is supported by the call chain below. It is not a claim about retrying failed model calls or replaying already-executed effects.

Environment

  • Branch: rust-preview
  • Commit: 7d572089bab6b54ee8cf051a5358775a9d7930f5 (upstream head verified November 12, 2026)
  • Surface: Agent scheduler engine and native owner
  • Probe environment: macOS 26.5.2 (25F84), rustc 1.99.1
  • Node.js: not used by the isolated probes
  • Full application integration tests were not run.

Logs, screenshots, or additional context

Source references at the audited commit:

Suggested regression coverage: Latest versus Skip on resume for interval/one-shot schedules; a deadline crossed during a read-only tick; transient pre-admission failure recovering without unrelated events and without duplicate admission.

Duplicate search covered open/closed issues using scheduler, scheduled task, misfire, 定时, rust schedule, and the relevant function/error names. Related reports #4133 (TypeScript recovery invariant), #4954 (TypeScript storage amplification), #5221 (snoozed reminder editing), and #5353 (JavaScript DST calculations) describe different defects.

Submitted by kesonm.

Activity

  1. Adarsh-Me commented on Nov 12, 2026

    @Adarsh-Me
    Contributor

    I'd like to take this one on. The three paths are related but separable, so my plan is focused PRs: starting with (2), where the fix is to schedule an immediate next pass when load_ready(at) selects a deadline that the final Excluded(now()) timer window then misses; then (1), resume honoring the selected Misfire policy; then (3), a bounded backoff reconciliation that preserves the original fire identity and admission dedup. Happy to reorder or split differently if that's preferred.

  2. kesonm commented on Nov 13, 2026

    @kesonm
    Author

    @Adarsh-Me Thanks for offering to help! I have already submitted PR #6855, which addresses all three bugs described in this issue, so there is no need to take these on separately. I wanted to let you know to avoid duplicating work. Thanks for your understanding!

  3. added a commit that references this issue on Nov 13, 2026
    73570d4
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions