Summary
The verif_scale_pyperf600 BPF selftest has been denylisted in CI for months, but
major BPF verifier improvements that have since landed (SCC analysis, backedge
propagation, live stack analysis) mean the test now passes. Local reproduction confirmed: all 4 pyperf600 variants
pass (4/0 PASSED, 0 SKIPPED, 0 FAILED), including the denylisted partial-unroll
variant which completes in ~10 seconds. The CI-side DENYLIST entry should be
removed to restore test coverage.
Failure Details
- Test / Component:
verif_scale_pyperf600 (BPF verifier scalability test)
- Frequency: Every run (test is unconditionally skipped via DENYLIST)
- Failure mode: Originally: verification failure or timeout for ~82k instruction BPF program
- Affected architectures: All (DENYLIST is not arch-specific)
- CI runs observed: Test shows as SKIPPED in all recent runs due to DENYLIST
Root Cause Analysis
pyperf600 is a BPF verifier scalability test that loads an ~82k instruction
program. The program (pyperf600.bpf.o) partially unrolls a 600-iteration loop
150 times, producing 4 ASM loops of ~16k instructions each, called 5 times from
on_event() (see tools/testing/selftests/bpf/progs/pyperf.h:346-356).
The test was denylisted in CI-side DENYLIST files (not in-tree), likely because
the verifier could not handle the program within time/instruction limits. Since
then, several major verifier improvements have landed:
- Live stack analysis (
815276dbfbb7, Sep 2025) — replaced path-sensitive
with path-insensitive live stack analysis, reducing verification state
- SCC analysis and backedge propagation (
ccaa6d2c9635, Dec 2025) — calls
to bpf_loop now have SCC and accumulate backedges, improving loop handling
- Relaxed frame limit (
ad95d3c758d8, Mar 2026) — only enforce 8-frame
call stack limit for all-static stacks
All other pyperf600 test variants pass consistently in CI:
verif_scale_pyperf600_nounroll — passes (no unrolling, ~1500 insns)verif_scale_pyperf600_bpf_loop — passes (uses bpf_loop helper)verif_scale_pyperf600_iter — passes (open-coded BPF iterator)verif_scale_pyperf_subprogs — passes (non-inlined subprogs)verif_scale_pyperf_global — passes (global subprogs)
The original pyperf600 variant (partial unroll, C loop with UNROLL_COUNT=150)
is the only one still denylisted. Given the verifier improvements above,
particularly the SCC and live stack analysis work that directly target loop
verification efficiency, this DENYLIST entry is stale.
Local reproduction confirms the test passes:
$ vmtest -k arch/x86/boot/bzImage -- ./test_progs -t verif_scale_pyperf600
WATCHDOG: test case verif_scale_pyperf600 executes for 10 seconds...
#538 verif_scale_pyperf600:OK
#539 verif_scale_pyperf600_bpf_loop:OK
#540 verif_scale_pyperf600_iter:OK
WATCHDOG: test case verif_scale_pyperf600_nounroll executes for 10 seconds...
#541 verif_scale_pyperf600_nounroll:OK
Summary: 4/0 PASSED, 0 SKIPPED, 0 FAILED
Proposed Fix
Remove verif_scale_pyperf600 from CI DENYLIST files in both:
kernel-patches/vmtest (ci/vmtest/configs/DENYLIST)libbpf/ci (ci/vmtest/configs/DENYLIST)
See attached patch files:
0001-ci-vmtest-Remove-stale-verif_scale_pyperf600-from-DENYLIST.patch (kernel-patches/vmtest)0002-ci-libbpf-Remove-stale-verif_scale_pyperf600-from-DENYLIST.patch (libbpf/ci)
If the test does fail after removal, re-add it with a comment explaining the
failure mode so future investigators can track the issue.
Impact
Without this fix, the pyperf600 verifier scalability test is permanently skipped
in CI, meaning regressions in BPF verifier handling of large partially-unrolled
loop programs would go undetected. This is particularly relevant as the verifier
continues to evolve with new optimizations.
References
- SCC analysis:
ccaa6d2c9635 ("Merge branch 'bpf-calls-to-bpf_loop-should-have-an-scc-and-accumulate-backedges'")
- Live stack analysis:
815276dbfbb7 ("Merge branch 'bpf-replace-path-sensitive-with-path-insensitive-live-stack-analysis'")
- Frame limit relaxation:
ad95d3c758d8 ("bpf: Only enforce 8 frame call stack limit for all-static stacks")
- Test source:
tools/testing/selftests/bpf/progs/pyperf600.c, tools/testing/selftests/bpf/progs/pyperf.h
- Test runner:
tools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c:113-122
Summary
The
verif_scale_pyperf600BPF selftest has been denylisted in CI for months, butmajor BPF verifier improvements that have since landed (SCC analysis, backedge
propagation, live stack analysis) mean the test now passes. Local reproduction confirmed: all 4 pyperf600 variants
pass (
4/0 PASSED, 0 SKIPPED, 0 FAILED), including the denylisted partial-unrollvariant which completes in ~10 seconds. The CI-side DENYLIST entry should be
removed to restore test coverage.
Failure Details
verif_scale_pyperf600(BPF verifier scalability test)Root Cause Analysis
pyperf600is a BPF verifier scalability test that loads an ~82k instructionprogram. The program (
pyperf600.bpf.o) partially unrolls a 600-iteration loop150 times, producing 4 ASM loops of ~16k instructions each, called 5 times from
on_event()(seetools/testing/selftests/bpf/progs/pyperf.h:346-356).The test was denylisted in CI-side DENYLIST files (not in-tree), likely because
the verifier could not handle the program within time/instruction limits. Since
then, several major verifier improvements have landed:
815276dbfbb7, Sep 2025) — replaced path-sensitivewith path-insensitive live stack analysis, reducing verification state
ccaa6d2c9635, Dec 2025) — callsto
bpf_loopnow have SCC and accumulate backedges, improving loop handlingad95d3c758d8, Mar 2026) — only enforce 8-framecall stack limit for all-static stacks
All other pyperf600 test variants pass consistently in CI:
verif_scale_pyperf600_nounroll— passes (no unrolling, ~1500 insns)verif_scale_pyperf600_bpf_loop— passes (uses bpf_loop helper)verif_scale_pyperf600_iter— passes (open-coded BPF iterator)verif_scale_pyperf_subprogs— passes (non-inlined subprogs)verif_scale_pyperf_global— passes (global subprogs)The original
pyperf600variant (partial unroll, C loop withUNROLL_COUNT=150)is the only one still denylisted. Given the verifier improvements above,
particularly the SCC and live stack analysis work that directly target loop
verification efficiency, this DENYLIST entry is stale.
Local reproduction confirms the test passes:
Proposed Fix
Remove
verif_scale_pyperf600from CI DENYLIST files in both:kernel-patches/vmtest(ci/vmtest/configs/DENYLIST)libbpf/ci(ci/vmtest/configs/DENYLIST)See attached patch files:
0001-ci-vmtest-Remove-stale-verif_scale_pyperf600-from-DENYLIST.patch(kernel-patches/vmtest)0002-ci-libbpf-Remove-stale-verif_scale_pyperf600-from-DENYLIST.patch(libbpf/ci)If the test does fail after removal, re-add it with a comment explaining the
failure mode so future investigators can track the issue.
Impact
Without this fix, the pyperf600 verifier scalability test is permanently skipped
in CI, meaning regressions in BPF verifier handling of large partially-unrolled
loop programs would go undetected. This is particularly relevant as the verifier
continues to evolve with new optimizations.
References
ccaa6d2c9635("Merge branch 'bpf-calls-to-bpf_loop-should-have-an-scc-and-accumulate-backedges'")815276dbfbb7("Merge branch 'bpf-replace-path-sensitive-with-path-insensitive-live-stack-analysis'")ad95d3c758d8("bpf: Only enforce 8 frame call stack limit for all-static stacks")tools/testing/selftests/bpf/progs/pyperf600.c,tools/testing/selftests/bpf/progs/pyperf.htools/testing/selftests/bpf/prog_tests/bpf_verif_scale.c:113-122