perf(group): count members in sql and preload group inbounds when syncing users

[FakharzadehH]

Summary

Adding an inbound to a group with many users (PUT /api/group/{id}) is very slow, and listing groups loads far more data than it needs. Two root causes:

1. Group summaries hydrate every member just to count them. Group.total_users was len(self.users) (app/db/models.py). get_group used selectinload(Group.users) (app/db/crud/group.py), and load_group_attrs awaited group.awaitable_attrs.users for get_group_by_id (used by get/modify/delete), create_group, modify_group and the bulk enable/disable path. Every one of those loaded all User rows for a number.
   • total_users is now a SQL COUNT: with_expression(Group._total_users_query, Group.total_users) in get_group / get_group_by_id, and a single COUNT in load_group_attrs. The total_users hybrid uses the loaded value and falls back to len(users), so API responses are unchanged. Same query_expression mechanism as User._reseted_usage_query.
2. Syncing a group's users to nodes issued one inbound query per user. GroupOperation.modify_group (app/operation/group.py) re-syncs every member. get_users only did selectinload(User.groups) (app/db/crud/user.py), so Group.inbounds was never loaded and User.inbounds() fell back to one SELECT DISTINCT inbounds.tag ... per user. The sync path also loaded usage_logs for every user, which it does not use.
   • New get_users(load_group_inbounds=False) nests selectinload(Group.inbounds) under User.groups. The group operation paths that feed sync_users (modify, remove, bulk remove by id, bulk enable/disable) now pass load_group_inbounds=True, load_usage_logs=False.

Type of change

• Bug fix
• New feature
• Breaking change
• Refactor / cleanup (performance)
• Documentation
• Tests / CI

Checklist

• I tested the change locally or explained why it cannot be tested.
• I added or updated tests for behavior changes.
• I updated documentation, translations, or examples if needed. (Not needed: no API or behavior change.)
• I checked database migrations when models or schema changed. (query_expression adds no column; alembic check reports no new upgrade operations.)
• I did not include secrets, tokens, private keys, or unrelated changes.

Testing

export TEST_FROM=github SQLALCHEMY_DATABASE_URL=sqlite+aiosqlite:///./test.db
uv run alembic upgrade head
uv run alembic check          # No new upgrade operations detected.
make test                     # 731 passed, 34 skipped
uv run pytest tests/test_group_user_loading.py -q   # new tests: 3 passed (3 failed on origin/dev code)
uv run ruff check <changed files> && uv run ruff format --check <changed files>   # clean

ruff check . reports 4 errors (app/routers/passkey.py x2, tests/api/test_user.py, tests/test_review_users_unit.py). They are identical on a clean origin/dev checkout and are not touched by this PR.

Only SQLite was run locally; the PostgreSQL and MariaDB jobs were not run.

New tests (tests/test_group_user_loading.py, crud level, in-memory SQLite) assert that get_group / get_group_by_id / load_group_attrs return correct total_users without loading Group.users or touching the users table, and that get_users(load_group_inbounds=True) leaves nothing for User.inbounds() to query.

Before / after

Measured on in-memory SQLite with one big group plus a second group sharing half its members, node dispatch stubbed, calling GroupOperation.modify_group and get_all_groups. SQLite hides network round trips and wide users rows, so real deployments should gain more.

Scenario	Users	Before	After
Add inbound (modify_group)	5,000	0.856 s, 2,542 queries	0.186 s, 32 queries
Add inbound (modify_group)	20,000	3.165 s, 10,132 queries	0.529 s, 92 queries
Group list	5,000	0.081 s, 4 queries	0.054 s, 3 queries
Group list	20,000	0.199 s, 4 queries	0.056 s, 3 queries (no User rows loaded)

total_users values and the modify response (inbound_tags, count, 20,000 users dispatched) were identical before and after.

Screenshots

Not applicable.

Notes for reviewers

• Refs Scale group summaries for large user counts #771. This PR covers its "populate total_users with a SQL count without loading Group.users" item.
• Overlap with perf(groups): add reverse group membership indexes #772: it also replaces the total_users hybrid on the model with a query_expression and adds the reverse association indexes. Both touch the same lines in models.py / crud/group.py, so one will need a rebase. This PR keeps the len(users) fallback in the hybrid and adds the count in load_group_attrs, which perf(groups): add reverse group membership indexes #772's with_expression approach is compatible with. Without perf(groups): add reverse group membership indexes #772's (groups_id, user_id) index the COUNT still scans users_groups_association.
• load_group_attrs now takes db first and load_total_users replaces load_users; get_group_by_id no longer takes load_users. All callers are updated (crud/group.py, operation/group.py).
• Out of scope, deliberately not changed here:
  • Moving the user/node sync to a background task so the request returns early, and skipping it on name-only edits (perf(groups): return updates before large user sync #827).
  • sync_users_allocations builds one IN (...) over all affected user ids. On PostgreSQL/asyncpg this could exceed the 32,767-parameter limit at roughly 33k+ users. Not tested.
  • remove_group and bulk_remove_groups_by_id first call get_users only to collect usernames, loading full users.
  • create_user_template / modify_user_template still load group members (perf(templates): avoid loading group members and batch list relations #944).

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 4859ccc6-15e4-4cce-bdf3-3395c4084182

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}