Skip to content

perf: import rarely used modules lazily on Python 3.15 via __lazy_modules__ - #7910

Draft
benedikt-bartscher wants to merge 2 commits into
reflex-dev:mainfrom
benedikt-bartscher:lazy-modules
Draft

benedikt-bartscher wants to merge 2 commits into
reflex-dev:mainfrom
benedikt-bartscher:lazy-modules

Conversation

@benedikt-bartscher

@benedikt-bartscher benedikt-bartscher commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

PEP 810 (Python 3.15) lets a module name other modules in __lazy_modules__. Module-level import x / from x import y statements for a listed module then bind a lazy proxy, and the module is imported the first time the name is used. Python 3.11–3.14 ignore the variable, so on those versions nothing changes.

This PR lists modules that are imported at module level but not used while the importing module executes, so on 3.15 they are only loaded when needed.

Listed modules

File __lazy_modules__ Used only in
reflex/app.py reflex.compiler, reflex.compiler.compiler compiling the app. Importing reflex.app, creating rx.App() and add_page don't need it. Deferring it skips 19 modules (compiler plugins, reflex_base.components.memo, reflex_base.plugins.compiler, the radix theme and document components).
reflex/utils/exec.py click parsing one env var in a function
reflex/utils/prerequisites.py, reflex/utils/js_runtimes.py packaging version checks in functions
reflex/utils/telemetry.py multiprocessing cpu_count() when telemetry is sent
reflex_base/utils/console.py rich.progress, rich.prompt, rich.table progress(), ask() and table output. rich.console stays eager because the module creates its Console at import time.
reflex_base/plugins/sitemap.py xml.etree.ElementTree sitemap generation

The comment next to each declaration says to keep these names out of import-time code. A module-level use would silently load the module eagerly again.

How I picked them

  • On 3.15.2 I recompiled reflex's own modules with every module-level import marked lazy and ran each workload below. Then I listed which import proxies were never resolved and whether their target got loaded anyway. Internal modules can't all be made lazy at once (the reflex_base.vars import order breaks), so I probed those one at a time.
  • I excluded modules whose deferral would skip import-time side effects:
    • reflex.utils.telemetry_accounting imports reflex.istate.shared, which registers SharedStateBaseInternal as a substate of State. This also rules out listing reflex.utils in reflex/app.py, because that one statement also imports prerequisites and js_runtimes.
    • reflex.istate.manager.redis records its pid and start time at import for its debug-log prefix.
  • I checked that reflex compile of a small app writes identical .web output and log messages (apart from durations) on 3.14 (eager) and 3.15.2 (lazy).
  • tests/units/test_lazy_modules.py (skipped before 3.15) imports each module in a fresh interpreter and asserts the exact set of names still bound to lazy proxies. It fails if a declaration is dropped or a listed name gets used at import time.
  • ruff doesn't flag the assignment before the imports: E402 already allows __lazy_modules__, so no noqa is needed.

Timings (Python 3.15.2)

Median of 40 paired samples per row, two independent runs shown as run 1 / run 2. Each pair runs upstream main and this branch back to back (order alternating) in fresh interpreters, from two copies of the source tree that differ only in these 7 files, in the same uv venv on Python 3.15.2. Import rows time only the import statement in-process (perf_counter), excluding interpreter start-up and shutdown. The CLI row is full-process wall time. "Minimal app" imports a module with import reflex as rx, one rx.State subclass with an event handler, one page, rx.App() and add_page.

Workload main (ms) this PR (ms) Paired diff (ms) Modules loaded
import reflex 2.8 / 2.8 2.8 / 2.8 0.0 / 0.0 52 → 52
import reflex.state 116.1 / 111.3 108.0 / 105.5 −6.5 / −5.8 479 → 455
import reflex.app 221.0 / 217.4 205.6 / 203.2 −16.1 / −14.2 911 → 865
import reflex.model 150.5 / 149.1 146.9 / 146.3 −4.3 / −2.6 534 → 510
Minimal app module 222.1 / 217.4 206.6 / 204.7 −14.3 / −12.9 916 → 870
reflex --help (wall) 70.3 / 69.8 64.7 / 64.4 −5.8 / −4.8 315 → 289

That is about 6–7% off importing an app and about 7–8% off CLI start-up. import reflex alone was already cheap on 3.15, since lazy_loader.attach binds native lazy imports there (#6930).

Deliberately left eager

  • socketio (~70 ms, and it pulls in redis, aiohttp and opentelemetry): EventNamespace(AsyncNamespace) in reflex/app.py subclasses it at import time, and App() creates the AsyncServer.
  • sqlalchemy / sqlmodel in reflex.model: Model subclasses SQLModel at import time.
  • pydantic (serializer registration), rich.console, starlette (reflex_base.utils.streaming_response subclasses StreamingResponse, reached from reflex.app through reflex._upload), and the reflex_base vars/event/component modules: all are used while reflex.state / reflex.app execute.
  • reflex.utils.exec in reflex/state.py and reflex/app.py: App() loads it anyway, and the gain for a bare import reflex.state was within noise once click was lazy.
  • hashlib, json, subprocess: under 1 ms each and spread over several files.
  • reflex/utils/console.py re-exports reflex_base.utils.console with import *, which resolves every lazy name. So the rich entries only help where nothing imports that shim, i.e. the CLI and reflex.model. An __all__ in reflex_base.utils.console could avoid that, but that's a code change I left out of this PR.

🤖 Generated with Claude Code

View guided diff

benedikt-bartscher and others added 2 commits November 15, 2026 21:33
List them in `__lazy_modules__` (PEP 810), so Python 3.15 imports them
the first time they are used. Older Pythons ignore the variable.

- reflex_base.utils.console: rich.progress, rich.prompt and rich.table
  are only used by progress(), ask() and print_table(). rich.console
  stays eager because the module creates its Console at import time.
- reflex_base.plugins.sitemap: xml.etree.ElementTree is only used to
  render the sitemap.

`reflex --help` on 3.15.2 no longer loads these modules (about 6 ms of
imports). The new test imports each module in a fresh interpreter and
checks which names are still lazy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hon 3.15

List them in `__lazy_modules__` (PEP 810), so Python 3.15 imports them
the first time they are used. Older Pythons ignore the variable.

- reflex.app: reflex.compiler and reflex.compiler.compiler. Importing
  reflex.app, creating rx.App() and add_page() don't compile, so the
  compiler and the 19 modules it pulls in now load when the app is
  compiled.
- reflex.utils.exec: click only parses GRANIAN_RESPAWN_FAILED_WORKERS in
  run_granian_backend_prod().
- reflex.utils.prerequisites, reflex.utils.js_runtimes: packaging is
  only used for version checks.
- reflex.utils.telemetry: multiprocessing is only used by
  get_cpu_count().

`reflex.utils` itself is not listed in reflex.app: the same statement
imports telemetry_accounting, which imports reflex.istate.shared, and
that module registers SharedStateBaseInternal as a substate of State.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Nov 15, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Summary

This PR delays selected imports until first use on Python 3.15+, reducing startup work without changing older Python versions.

  • App setup waits to load the compiler until compilation needs it.
  • Utility modules wait to load selected dependencies until their functions need them.
  • Console and sitemap tools wait to load their supporting modules until used.

No actionable issues found. Tests were not run because uv is unavailable in this environment.

Reviews (1) · Last reviewed commit: "perf: defer the compiler, click, packagi..." · Reviewed by Greptile

@codspeed

codspeed Bot commented Nov 15, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

1 warning

⚠️ 19 benchmarks are dominated by system calls

System calls cannot be simulated consistently and are left out of the measure. The reported value understates the real cost. Switch to the Walltime instrument to measure them.

Learn more about system calls

See affected benchmarks

✅ 143 untouched benchmarks
⏩ 8 skipped benchmarks1


Comparing benedikt-bartscher:lazy-modules (559b025) with main (8e5bb84)

Open in CodSpeed

Footnotes

  1. 8 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@masenf masenf added the perf Performance-improving changes label Nov 16, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

perf Performance-improving changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants