Skip to content

JEP-0019 + JEP-0020: Scale Testing - #1356

Open
mangelajo wants to merge 7 commits into
jumpstarter-dev:mainfrom
mangelajo:jep-controller-scale-testing
Open

mangelajo wants to merge 7 commits into
jumpstarter-dev:mainfrom
mangelajo:jep-controller-scale-testing

Conversation

@mangelajo

@mangelajo mangelajo commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

Summary

Propose jumpstarter-overdrive (alternative name: jumpstarter-tempest), an opt-in Python scale/performance testing tool for existing and disposable Jumpstarter clusters.

Phase 1 — JEP-0019: standalone Python tool

  • Independent lease-scheduling, client-churn, exporter-churn, and router-traffic scenarios.
  • Per-scenario load ranges and timing, using real protocol paths and runner-managed mock exporters.
  • Controller/router CPU, memory, and network collection, with preliminary report examples.
  • Small implementation increments: runner and lease scheduling first, additional scenarios next, then resource collection and readable reports.
  • No controller, operator, CRD, or provisioner changes; no Phase 2 scaffolding required.

Phase 2 — JEP-0020: separate extension work

  • A separate JEP referencing JEP-0019 defines the opt-in mock ExporterSet provisioner and scaling scenario.
  • Reuses the completed runner, metrics collector, and reporting to exercise the real ExporterSet framework without real target provisioning.
  • Includes ExporterSet controller metrics and scaling-specific acceptance criteria.
  • Not a prerequisite or acceptance criterion for Phase 1: the initial tool can ship independently.

Both design documents are included here; implementation is not part of this PR.

Validation

  • Validated required sections, local links, YAML/JSON/Python example syntax, aligned load ranges, and example throughput arithmetic.
  • Confirmed the Phase 1 configuration excludes the mock provisioner and ExporterSet scenario.
  • Staged whitespace checks passed.
  • A documentation build during initial drafting failed on an unrelated local, untracked document: docs/source/introduction/exporter-fsm-diagram.md is not included in a toctree. That file is not part of this PR.

@mangelajo mangelajo added the jep Jumpstarter Enhancement Proposals label Nov 19, 2026
@coderabbitai

coderabbitai Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

No actionable comments were generated in the recent review.

Walkthrough

The change adds two draft JEPs proposing a standalone scale-testing tool and an opt-in ExporterSet testing extension. The JEP index adds entries and links for both proposals.

Changes: Performance testing proposals

Layer / File(s) Summary
Define Phase 1 performance testing
docs/source/contributing/jeps/JEP-0019-scale-performance-testing.md
JEP-0019 proposes a standalone tool with four initial scenarios, load-sweep rules, measurement and reporting formats, and validation requirements. It separates ExporterSet testing into Phase 2.
Define the ExporterSet extension
docs/source/contributing/jeps/JEP-0020-mock-exporterset-performance-testing.md
JEP-0020 proposes an opt-in mock provisioner and an exporterset-scaling scenario. It describes the extension as separate follow-up work that uses the existing provisioner interface.
Describe Phase 2 operation and publish both JEPs
docs/source/contributing/jeps/JEP-0020-mock-exporterset-performance-testing.md, docs/source/contributing/jeps/index.md
JEP-0020 specifies the proposed test flow, measurements, cleanup, and validation. The index adds Draft entries and links for both JEPs.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Other

Suggested labels: documentation

Suggested reviewers: kirkbrauer

Merge Risk: Minimal · up to de3e1

This change adds two draft proposals for scale and performance testing, plus their index entries. It does not change runtime behavior. No merge-blocking issues remain.

Pre-merge checks: Passed 4

Check name Status Explanation
Title check Passed The title names both JEP documents added by this PR (JEP-0019 and JEP-0020) and the topic they address, scale testing. It is short and specific. It does not name the tool or the phase split, but it ac…
Description check Passed The description states the purpose of the PR: two design documents (JEP-0019 and JEP-0020) for the jumpstarter-overdrive scale and performance testing tool. It lists the Phase 1 and Phase 2 scope, the…
Linked Issues check Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check Passed Check skipped because no linked issues were found for this pull request.
Full details: Title check

Explanation

The title names both JEP documents added by this PR (JEP-0019 and JEP-0020) and the topic they address, scale testing. It is short and specific. It does not name the tool or the phase split, but it accurately covers the main change, which is two design documents for scale and performance testing.

Full details: Description check

Explanation

The description states the purpose of the PR: two design documents (JEP-0019 and JEP-0020) for the jumpstarter-overdrive scale and performance testing tool. It lists the Phase 1 and Phase 2 scope, the validation performed, and notes that implementation is not part of the PR. It matches the changeset closely.

Finishing Touches

Generate unit tests (beta)

  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

A rabbit reads the JEPs at dawn,
And nibbles through each testing plan.
Mock pods hop into the scaling scene,
While metrics mark what steps have been.
Two drafts now join the index row,
Then off through clover paths we go!

Recent review info

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 61d326f8-2043-4f6e-bf31-39ca0e70f747
Commits

Reviewing files that changed from the base of the PR and between 27ec21e and de3e1f7.

Files selected for processing: 3

  • docs/source/contributing/jeps/JEP-0019-scale-performance-testing.md
  • docs/source/contributing/jeps/JEP-0020-mock-exporterset-performance-testing.md
  • docs/source/contributing/jeps/index.md

Included review availability: This review used your included allowance. Your plan provides up to 2 included reviews per hour; 1 remain after this review.

@mangelajo mangelajo changed the title JEP-0019: Scale and Performance Testing JEP-0019 + JEP-0020: Scale Testing Nov 20, 2026
@mangelajo
mangelajo marked this pull request as ready for review November 20, 2026 10:09

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jep Jumpstarter Enhancement Proposals

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant