Make Devise work with lazy-loaded routes

[excid3]

Fixes #5830, #5844, #5752 and #5847.

Problem

Since Rails 8.0 routes are lazy-loaded in development and test (Rails::Engine::LazyRouteSet): they are only drawn once a request reaches the router. Devise configures Warden while routes are drawn (Devise::RouteSet#finalize!), but Warden::Manager runs earlier in the middleware stack and Warden::Proxy#initialize dups manager.config when it builds the proxy for the request. So the first request of a fresh process sees a Warden with no default scope, no strategies and no session serializers. That is what the reports describe: :default user is not logged in (#5830), the rememberable strategy not running after a restart (#5844), and login_as/sign_in storing the raw record in the session and overflowing the cookie (#5752).

Separately, Rails only uses lazy routes when Rails.env.local?. Elsewhere routes are drawn during initialization, and devise_for has to load the model class to know which routes to draw. Rails 8.2's load-hook guard (rails/rails#56201) reports that as a component loaded before initialization whenever eager loading is off, e.g. rake tasks in production where rake_eager_load is false (#5847).

Fix

• Devise::RoutesLoader, a two-line middleware inserted right before Warden::Manager, calls Rails.application.reload_routes_unless_loaded. That is the same call Devise.mappings has made since Make Devise.mappings work with lazy loaded routes. #5728 for code paths that are not requests (console, tests). The first request was going to draw the routes anyway, so nothing changes for boot time.
• A devise.lazy_routes initializer sets config.route_set_class = Rails::Engine::LazyRouteSet in every environment on Rails 8+, unless the app configured its own class. It runs before make_routes_lazy, which on current Rails main also instantiates the route set. When eager loading is on, Rails loads routes at boot either way (set_routes_reloader_hook), so production servers behave exactly as before; only non-eager boots like rake tasks stop drawing routes and loading models they never use.

Both are guarded on defined?(Rails::Engine::LazyRouteSet), so Rails 7.x is untouched.

Why not #5831 or #5853

#5831 turns lazy routes off, which is the opposite of where Rails is going and keeps loading every Devise model at boot. #5853 registers models up front, but is opt-in, regresses Devise.mappings (it no longer loads routes) and still trips the 8.2 guard in development. This keeps lazy routes working without the app having to know about it.

Tests

test/lazy_routes_test.rb checks the middleware order and boots the test app in a fresh process (test/scripts/first_request_probe.rb, since the suite has already loaded the routes) to assert that routes and the User model are not loaded at boot, that the first request configures Warden fully, and that the same holds when Rails.env.local? is false. The lazy assertions are gated on LazyRouteSet so the test runs on every supported Rails.

Also verified on a real Jumpstart-based app against Rails 8.2.0.alpha (rails/rails@4658033), fresh process each time:

	Devise main	this PR
Development, first request is POST /users/sign_in	500, default_scope=:default, no strategies, no failure app	303, default_scope=:user, [:rememberable, :database_authenticatable], session key [[id], salt]
Production rake task (rake_eager_load=false)	routes drawn and User loaded during boot	LazyRouteSet, neither loaded

Notes for Rails

While tracking this down, three things on the Rails side seemed worth mentioning:

• The underlying inconsistency is make_routes_lazy being gated on Rails.env.local?. If Rails used LazyRouteSet everywhere, the devise.lazy_routes initializer here could go away.
• RoutesReloader#execute_unless_loaded is not synchronized on 8.0/8.1 (it is on main). Two concurrent first requests could draw the routes twice. This middleware does not make that worse than Devise.mappings already does, but it is the reason the PR keeps the broad LazyRouteSet gate rather than version checks.
• Any gem that references ActionMailer::Base in an initializer that runs before action_mailer.set_configs (mailpace-rails does) makes that railtie's on_load(:action_mailer) block call app.routes before make_routes_lazy, so the app silently gets a plain RouteSet. config.exceptions_app = routes in config/application.rb has the same effect. Not Devise's to fix, but useful to know when an app does not reproduce these bugs.

CI

The full 60-job matrix is green on the fork with this change combined with #5866 and #5867: https://github.com/excid3/devise/actions/runs/37403424517. On its own, the Rails 7.0/7.1/8.0 jobs need #5866 (json 3.0) and the rails-main job needs #5867 (8.2 test deprecations); none of those failures are related to this change. #5856 is the companion fix for the remaining 8.2 deprecation in lib.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Fb2E5SnkxkicWdSSbqUN4y

[excid3]

Follow-up on the Rails side: rails/rails#58951 moves the LazyRouteSet default from the make_routes_lazy initializer into Rails::Engine::Configuration#initialize and removes that initializer. Rails ignores before:/after: references to a missing initializer name, so the before: :make_routes_lazy ordering here keeps booting unchanged if it lands; I'll drop that option and update the comment in a follow-up once Rails decides.

That PR only fixes the default being chosen too late. The Rails.env.local? gate that this PR's devise.lazy_routes initializer works around is still there, so this PR is needed either way.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Fb2E5SnkxkicWdSSbqUN4y