fix: track users reload separetely of server config#218
Draft
GrigoryPervakov wants to merge 1 commit into
Draft
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR addresses flakiness in config-reload behavior by ensuring the operator considers a replica “reloaded” only when both the server config and the users config have applied the same revision marker. It does this by adding a users-side reload marker (via a custom setting in an operator-owned settings profile) and by updating reconciliation logic to check both markers.
Changes:
- Add a custom users-profile setting (
custom_operator_reload_revision) that tracks the reload revision independently of the server config named collection revision. - Enable ClickHouse custom settings with
custom_settings_prefixes: custom_in the base server config template. - Update probe/reconcile logic to treat a replica as reloaded only when both the named-collection revision and the users-profile marker match the expected revision (plus update integration tests accordingly).
Reviewed changes
Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| internal/controller/clickhouse/templates/users.yaml.tmpl | Adds an operator settings profile with a custom setting to mark users reload revision; moves operator user onto that profile. |
| internal/controller/clickhouse/templates/base.yaml.tmpl | Enables custom_ custom settings so the users-profile marker can be applied/queried. |
| internal/controller/clickhouse/sync.go | Switches reload checks to require both server-config and users-config markers to match expected revision. |
| internal/controller/clickhouse/config.go | Adds constants and template params to wire the operator profile name + reload revision into generated users config. |
| internal/controller/clickhouse/commands.go | Extends replica probe to also read the users reload marker from system.settings_profile_elements and adds a Reloaded() helper. |
| internal/controller/clickhouse/commands_test.go | Updates integration test config and assertions to validate both reload markers are observed. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
ClickHouse updates users and server config in the background with the separate ConfigReloaders, so if server config is valid, but users are misconfigured, it will show a new revision in the named collection while failing on
SYSTEM RELOAD CONFIGThis makes the e2e test for config reload flaky
What
User custom config entry for the management user to track users' reload revision