Skip to content

fix: track users reload separetely of server config#218

Draft
GrigoryPervakov wants to merge 1 commit into
mainfrom
fix-users-hot-reload
Draft

fix: track users reload separetely of server config#218
GrigoryPervakov wants to merge 1 commit into
mainfrom
fix-users-hot-reload

Conversation

@GrigoryPervakov
Copy link
Copy Markdown
Member

Why

ClickHouse updates users and server config in the background with the separate ConfigReloaders, so if server config is valid, but users are misconfigured, it will show a new revision in the named collection while failing on SYSTEM RELOAD CONFIG
This makes the e2e test for config reload flaky

What

User custom config entry for the management user to track users' reload revision

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR addresses flakiness in config-reload behavior by ensuring the operator considers a replica “reloaded” only when both the server config and the users config have applied the same revision marker. It does this by adding a users-side reload marker (via a custom setting in an operator-owned settings profile) and by updating reconciliation logic to check both markers.

Changes:

  • Add a custom users-profile setting (custom_operator_reload_revision) that tracks the reload revision independently of the server config named collection revision.
  • Enable ClickHouse custom settings with custom_settings_prefixes: custom_ in the base server config template.
  • Update probe/reconcile logic to treat a replica as reloaded only when both the named-collection revision and the users-profile marker match the expected revision (plus update integration tests accordingly).

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

Show a summary per file
File Description
internal/controller/clickhouse/templates/users.yaml.tmpl Adds an operator settings profile with a custom setting to mark users reload revision; moves operator user onto that profile.
internal/controller/clickhouse/templates/base.yaml.tmpl Enables custom_ custom settings so the users-profile marker can be applied/queried.
internal/controller/clickhouse/sync.go Switches reload checks to require both server-config and users-config markers to match expected revision.
internal/controller/clickhouse/config.go Adds constants and template params to wire the operator profile name + reload revision into generated users config.
internal/controller/clickhouse/commands.go Extends replica probe to also read the users reload marker from system.settings_profile_elements and adds a Reloaded() helper.
internal/controller/clickhouse/commands_test.go Updates integration test config and assertions to validate both reload markers are observed.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants