Skip to content

CNTRLPLANE-3364: add the kms suite and migrate the kms tests of auth-o to ote#895

Open
sandeepknd wants to merge 1 commit into
openshift:masterfrom
sandeepknd:kms-suite-auth
Open

CNTRLPLANE-3364: add the kms suite and migrate the kms tests of auth-o to ote#895
sandeepknd wants to merge 1 commit into
openshift:masterfrom
sandeepknd:kms-suite-auth

Conversation

@sandeepknd
Copy link
Copy Markdown

@sandeepknd sandeepknd commented May 14, 2026

add the kms suite and migrate the kms encryption tests to ote

Summary by CodeRabbit

  • Tests
    • Added comprehensive end-to-end tests for KMS encryption functionality
    • Implemented test coverage for KMS encryption activation/deactivation scenarios
    • Added validation tests for encryption provider migration workflows

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 14, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

This PR adds end-to-end tests for KMS encryption in the cluster-authentication-operator. A new test suite is registered with serial execution constraints, and two test scenarios verify token encryption behavior during KMS enable/disable transitions and encryption provider migrations. The test logic is implemented in a new file with refactored thin entry points.

Changes

KMS Encryption E2E Tests

Layer / File(s) Summary
Test suite registration
cmd/cluster-authentication-operator-tests-ext/main.go
A new test suite openshift/cluster-authentication-operator/encryption-kms is registered with Parallelism: 1 and a qualifier filter that selects only tests with KMSEncryption in their names.
KMS encryption test scenarios
test/e2e-encryption-kms/encryption_kms.go, test/e2e-encryption-kms/encryption_kms_test.go
Two serial e2e test specs are defined: TestKMSEncryptionOnOff deploys a mock KMS plugin and verifies tokens are encrypted when KMS is enabled and unencrypted when disabled; TestKMSEncryptionProvidersMigration deploys the same plugin and verifies encryption status after each migration step while shuffling between KMS and a random static AES provider. Thin test entrypoints delegate to package-level helper functions.

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 9 | ❌ 3

❌ Failed checks (3 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 40.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Microshift Test Compatibility ⚠️ Warning Tests use config.openshift.io API which is unavailable on MicroShift. No protection mechanisms are present. Add [apigroup:config.openshift.io] tag to test names, or use [Skipped:MicroShift] label, or add IsMicroShiftCluster() runtime check with skip.
Ipv6 And Disconnected Network Test Compatibility ⚠️ Warning Tests require pulling quay.io/openshifttest/mock-kms-plugin image from external public registry, causing failure in disconnected environments. Pre-load mock KMS plugin image to cluster, mirror to internal registry, or add [Skipped:Disconnected] tag to tests to skip in disconnected CI environments.
✅ Passed checks (9 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed All Ginkgo test names are stable with no dynamic content. Test titles contain only static, descriptive strings.
Test Structure And Quality ✅ Passed Tests satisfy all requirements: single responsibility, proper timeouts, delegated assertions via library, consistent with test/e2e patterns.
Single Node Openshift (Sno) Test Compatibility ✅ Passed Tests do not assume multi-node clusters. KMS DaemonSet with control-plane node selector runs on SNO's single node. Tests verify encryption behavior only, not node topology.
Topology-Aware Scheduling Compatibility ✅ Passed PR adds only e2e test code and test suite registration (no deployment manifests, operator code, or controllers). No scheduling constraints are introduced. Not applicable to custom check scope.
Ote Binary Stdout Contract ✅ Passed PR adheres to OTE Binary Stdout Contract. main.go correctly sets klog.LogToStderr(true), contains no fmt/log stdout writes in process-level code, and test setup properly avoids stdout violations.
Title check ✅ Passed The title accurately describes the main changes: adding a KMS test suite and migrating KMS encryption tests to OTE, which aligns with all file modifications.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@sandeepknd
Copy link
Copy Markdown
Author

/assign @gangwgr

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
test/e2e-encryption-kms/encryption_kms.go (1)

5-5: ⚡ Quick win

Avoid nondeterministic provider selection in migration tests.

Line 69 randomizes provider choice and order, which makes failures harder to reproduce. Prefer a fixed provider list/order (or explicit deterministic subcases) for stable CI diagnostics.

Proposed deterministic variant
-	"math/rand/v2"
@@
-		EncryptionProviders:            library.ShuffleEncryptionProviders([]configv1.APIServerEncryption{{Type: configv1.EncryptionTypeKMS, KMS: librarykms.DefaultFakeKMSPluginConfig}, library.SupportedStaticEncryptionProviders[rand.IntN(len(library.SupportedStaticEncryptionProviders))]}),
+		EncryptionProviders: []configv1.APIServerEncryption{
+			{Type: configv1.EncryptionTypeKMS, KMS: librarykms.DefaultFakeKMSPluginConfig},
+			library.SupportedStaticEncryptionProviders[0],
+		},
As per coding guidelines, "Review Ginkgo test code for quality: ... (5) Consistency - follow existing repository patterns."

Also applies to: 69-69

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/e2e-encryption-kms/encryption_kms.go` at line 5, The test currently
randomizes the provider selection/order (the providers slice and its
rand.Shuffle/selection logic) which causes nondeterministic failures; replace
the randomization with a deterministic order—either hardcode the providers slice
in a stable order (e.g., []string{"aws","gcp","azure"}) or
sort.Strings(providers) before use, and remove any use of math/rand/v2 or
rand.Shuffle in the provider-selection code path so the migration tests run with
a stable, reproducible provider list.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/e2e-encryption-kms/encryption_kms.go`:
- Line 5: The test currently randomizes the provider selection/order (the
providers slice and its rand.Shuffle/selection logic) which causes
nondeterministic failures; replace the randomization with a deterministic
order—either hardcode the providers slice in a stable order (e.g.,
[]string{"aws","gcp","azure"}) or sort.Strings(providers) before use, and remove
any use of math/rand/v2 or rand.Shuffle in the provider-selection code path so
the migration tests run with a stable, reproducible provider list.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 6e62b6d6-dcfd-4fe7-a5e4-b453d6d4ca2c

📥 Commits

Reviewing files that changed from the base of the PR and between cdcee0b and c352356.

📒 Files selected for processing (4)
  • cmd/cluster-authentication-operator-tests-ext/main.go
  • pkg/dependencymagnet/dependencymagnet.go
  • test/e2e-encryption-kms/encryption_kms.go
  • test/e2e-encryption-kms/encryption_kms_test.go

@sandeepknd
Copy link
Copy Markdown
Author

/test e2e-aws-operator-encryption-kms-ote

1 similar comment
@sandeepknd
Copy link
Copy Markdown
Author

/test e2e-aws-operator-encryption-kms-ote

@sandeepknd
Copy link
Copy Markdown
Author

sandeepknd commented May 15, 2026

re-triggering the test e2e-aws-operator-encryption-kms-ote as it failed due to node not becoming ready.
e2e-aws-operator-encryption-kms-ote-nodes-readiness failed.


[INFO] - 6 ready nodes expected, found 7... Waiting 1min before retrying (timeout in 1min)...
108
[ERROR] Timeout reached. 6 ready nodes expected, found 7... Failing.

@sandeepknd
Copy link
Copy Markdown
Author

The tests (TestKMSEncryptionOnOff and TestKMSEncryptionProvidersMigration) got executed within the suite openshift/cluster-authentication-operator/encryption-kms.
Please find the log ,

Name: "openshift/cluster-authentication-operator/encryption-kms",
Parallelism: 1,
Qualifiers: []string{
`name.contains("KMSEncryptionProvider")`,
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"github.com/openshift/cluster-authentication-operator/pkg/version"

_ "github.com/openshift/cluster-authentication-operator/test/e2e"
_ "github.com/openshift/cluster-authentication-operator/test/e2e-encryption-kms"
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added only in this file. Referred other repo as well.

Comment thread test/e2e-encryption-kms/encryption_kms_test.go
Comment thread test/e2e-encryption-kms/encryption_kms_test.go
Comment thread test/e2e-encryption-kms/encryption_kms_test.go
Comment thread test/e2e-encryption-kms/encryption_kms.go Outdated
})
})

func testKMSEncryptionOnOff(t testing.TB) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

were there any changes to this test ?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the latest changes made to this test has been pulled in this PR as well.

})
}

func testKMSEncryptionProvidersMigration(t testing.TB) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

were there any changes to this test ?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the latest changes made to this test (multiline parameters) has been pulled in this PR as well.

@p0lyn0mial
Copy link
Copy Markdown
Contributor

/assign @gangwgr

@openshift-ci openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 15, 2026
@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 15, 2026

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from gangwgr. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label May 15, 2026
@sandeepknd sandeepknd changed the title add the kms suite and migrate the kms tests to ote [WIP]add the kms suite and migrate the kms tests to ote May 15, 2026
@openshift-ci openshift-ci Bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 15, 2026
@sandeepknd sandeepknd force-pushed the kms-suite-auth branch 2 times, most recently from 0b99d30 to 31d729b Compare May 15, 2026 16:49
@sandeepknd sandeepknd changed the title [WIP]add the kms suite and migrate the kms tests to ote add the kms suite and migrate the kms tests to ote May 15, 2026
@openshift-ci openshift-ci Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 15, 2026
@sandeepknd sandeepknd changed the title add the kms suite and migrate the kms tests to ote CNTRLPLANE-3364: add the kms suite and migrate the kms tests to ote May 15, 2026
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label May 15, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

openshift-ci-robot commented May 15, 2026

@sandeepknd: This pull request references CNTRLPLANE-3364 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "5.0.0" version, but no target version was set.

Details

In response to this:

add the kms suite and migrate the kms encryption tests to ote

Summary by CodeRabbit

  • Tests
  • Added comprehensive end-to-end tests for KMS encryption functionality
  • Implemented test coverage for KMS encryption activation/deactivation scenarios
  • Added validation tests for encryption provider migration workflows

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci
Copy link
Copy Markdown
Contributor

openshift-ci Bot commented May 15, 2026

@sandeepknd: all tests passed!

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@sandeepknd
Copy link
Copy Markdown
Author

job e2e-aws-operator-encryption-kms-ote got executed. Logs can be found here. It has successfully executed the tests TestKMSEncryptionOnOff and estKMSEncryptionProvidersMigration .

@sandeepknd
Copy link
Copy Markdown
Author

All tests have passed.
@p0lyn0mial Could you PTAL.
@gangwgr Could you PTAL.

@sandeepknd sandeepknd changed the title CNTRLPLANE-3364: add the kms suite and migrate the kms tests to ote CNTRLPLANE-3364: add the kms suite and migrate the kms tests of auth-o to ote May 16, 2026
@sandeepknd sandeepknd requested a review from p0lyn0mial May 16, 2026 03:22
@sandeepknd
Copy link
Copy Markdown
Author

/verified by CI

@openshift-ci-robot openshift-ci-robot added the verified Signifies that the PR passed pre-merge verification criteria label May 16, 2026
@openshift-ci-robot
Copy link
Copy Markdown
Contributor

@sandeepknd: This PR has been marked as verified by CI.

Details

In response to this:

/verified by CI

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. verified Signifies that the PR passed pre-merge verification criteria

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants