Skip to content

CF-1873 : Manage CMF secret mappings#3306

Open
Paras Negi (paras-negi-flink) wants to merge 7 commits intomainfrom
CF-1873
Open

CF-1873 : Manage CMF secret mappings#3306
Paras Negi (paras-negi-flink) wants to merge 7 commits intomainfrom
CF-1873

Conversation

@paras-negi-flink
Copy link
Copy Markdown

@paras-negi-flink Paras Negi (paras-negi-flink) commented Apr 5, 2026 •
edited
Loading

Release Notes

Breaking Changes

  • None.

New Features

  • Added confluent flink secrets create|list|describe|update|delete commands to manage CMF Secrets (sensitive credentials used by Flink SQL catalogs) on Confluent Platform.

Bug Fixes

  • None.

Checklist

  • I have successfully built and used a custom CLI binary, without linter issues from this PR.
  • I have clearly specified in the What section below whether this PR applies to Confluent Cloud, Confluent Platform, or both.
  • I have attached manual CLI verification results or screenshots in the Test & Review section below.
  • I have added appropriate CLI integration or unit tests for any new or updated commands and functionality.
  • I confirm that this PR introduces no breaking changes or backward compatibility issues.
  • I have indicated the potential customer impact if something goes wrong in the Blast Radius section below.
  • I have put checkmarks below confirming that the feature associated with this PR is enabled in:
    • Confluent Cloud prod
    • Confluent Cloud stag
    • Confluent Platform
    • Check this box if the feature is enabled for certain organizations only

What

This PR implements CF-1873 — Manage CMF secret mappings, targeting Confluent Platform / CP Flink (CMF on-prem):

  • Adds a new command group under confluent flink:

    • confluent flink secret-mapping create <resourceFilePath> --environment <envName>
    • confluent flink secret-mapping list --environment <envName>
    • confluent flink secret-mapping describe <name> --environment <envName>
    • confluent flink secret-mapping update <resourceFilePath> --environment <envName>
    • cconfluent flink secret-mapping delete <name> --environment <envName>

  • Secret mappings bind connection secret IDs (used in catalog database configs) to actual CMF Secrets per environment. When a SQL statement runs, CMF resolves these mappings to inject the correct credentials into the catalog connection properties at execution time.

  • Wires these commands to the existing CMF Environment Secret Mapping REST APIs:
    POST /cmf/api/v1/environments/{envName}/secret-mappings
    GET /cmf/api/v1/environments/{envName}/secret-mappings
    GET /cmf/api/v1/environments/{envName}/secret-mappings/{name}
    PUT /cmf/api/v1/environments/{envName}/secret-mappings/{name}
    DELETE /cmf/api/v1/environments/{envName}/secret-mappings/{name}

  • Introduces a CmfRestClient wrapper for environment secret-mapping operations and corresponding local types/output formatting, following existing patterns used for catalogs, compute pools, catalog databases, and CMF Secrets.

Blast Radius

  • Scope is limited to the new Flink secret-mapping commands; existing CLI behavior (including other confluent flink and confluent kafka commands) is unchanged.

  • If something goes wrong:

    • Impact is confined to users managing environment secret mappings via the new commands:
      • Secret-mapping create/update/delete may fail or surface CMF errors.
      • List/describe may not show the correct state for environment secret mappings.

  • There are no breaking changes to existing commands, flags, or APIs. Removing or reverting this feature is straightforward (command group + client wrapper).

References

Test & Review

Environment

  • Repo: confluentinc/cli
  • Branch: CF-1872
  • CMF: 2.3-SNAPSHOT (image: confluentinc/cp-cmf:c505ee8b) - Kubernetes: local cluster with CMF deployed (cmf-service exposed via kubectl port-forward svc/cmf-service 8080:80 -n e2e)

Manual CLI validation

Attached in the comment below

Copilot AI review requested due to automatic review settings April 5, 2026 16:03
@confluent-cla-assistant
Copy link
Copy Markdown

confluent-cla-assistant bot commented Apr 5, 2026

🎉 All Contributor License Agreements have been signed. Ready to merge.
Please push an empty commit if you would like to re-run the checks to verify CLA status for all contributors.

Copilot AI reviewed Apr 5, 2026
View reviewed changes
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR extends the on-prem Flink (CMF) CLI surface to support managing environment secret mappings, and adds an update command for Kafka catalogs, including CMF client support and on-prem integration test coverage.

Changes:

  • Add confluent flink secret-mapping command group (create/describe/list/update/delete) backed by new CMF REST client methods.
  • Add confluent flink catalog update command and CMF REST client UpdateCatalog.
  • Extend the on-prem test server router/handlers and add integration tests + fixtures for the new behaviors.

Reviewed changes

Copilot reviewed 52 out of 52 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
test/test-server/flink_onprem_router.go Adds CMF on-prem router endpoints for secret-mappings.
test/test-server/flink_onprem_handler.go Implements test-server handlers for secret-mappings and catalog update.
test/flink_onprem_test.go Adds integration tests for catalog update and secret-mapping commands (JSON/YAML + failures).
test/fixtures/output/flink/help-onprem.golden Adds secret-mapping to top-level flink help output (on-prem).
test/fixtures/output/flink/catalog/help-onprem.golden Adds update to flink catalog help (on-prem).
test/fixtures/output/flink/catalog/update-help-onprem.golden Adds help fixture for flink catalog update (on-prem).
test/fixtures/output/flink/catalog/update-success.golden Golden output for successful catalog update (human).
test/fixtures/output/flink/catalog/update-success-json.golden Golden output for successful catalog update (json).
test/fixtures/output/flink/catalog/update-success-yaml.golden Golden output for successful catalog update (yaml).
test/fixtures/output/flink/catalog/update-invalid-failure.golden Golden output for invalid catalog update failure.
test/fixtures/output/flink/secret-mapping/create-success.golden Golden output for secret-mapping create (human).
test/fixtures/output/flink/secret-mapping/create-success-json.golden Golden output for secret-mapping create (json).
test/fixtures/output/flink/secret-mapping/create-success-yaml.golden Golden output for secret-mapping create (yaml).
test/fixtures/output/flink/secret-mapping/create-invalid-failure.golden Golden output for secret-mapping create invalid failure.
test/fixtures/output/flink/secret-mapping/describe-success.golden Golden output for secret-mapping describe (human).
test/fixtures/output/flink/secret-mapping/describe-success-json.golden Golden output for secret-mapping describe (json).
test/fixtures/output/flink/secret-mapping/describe-success-yaml.golden Golden output for secret-mapping describe (yaml).
test/fixtures/output/flink/secret-mapping/describe-not-found.golden Golden output for secret-mapping describe not-found failure.
test/fixtures/output/flink/secret-mapping/list-success.golden Golden output for secret-mapping list (human).
test/fixtures/output/flink/secret-mapping/list-success-json.golden Golden output for secret-mapping list (json).
test/fixtures/output/flink/secret-mapping/list-success-yaml.golden Golden output for secret-mapping list (yaml).
test/fixtures/output/flink/secret-mapping/update-success.golden Golden output for secret-mapping update (human).
test/fixtures/output/flink/secret-mapping/update-success-json.golden Golden output for secret-mapping update (json).
test/fixtures/output/flink/secret-mapping/update-success-yaml.golden Golden output for secret-mapping update (yaml).
test/fixtures/output/flink/secret-mapping/update-invalid-failure.golden Golden output for secret-mapping update invalid failure.
test/fixtures/output/flink/secret-mapping/delete-single-successful.golden Golden output for secret-mapping delete (interactive confirm).
test/fixtures/output/flink/secret-mapping/delete-single-force.golden Golden output for secret-mapping delete (force).
test/fixtures/output/flink/secret-mapping/delete-non-exist-failure.golden Golden output for secret-mapping delete failure.
test/fixtures/input/flink/catalog/update-successful.json Input fixture for catalog update success (json).
test/fixtures/input/flink/catalog/update-successful.yaml Input fixture for catalog update success (yaml).
test/fixtures/input/flink/catalog/update-invalid-failure.json Input fixture for catalog update invalid case (json).
test/fixtures/input/flink/catalog/update-invalid-failure.yaml Input fixture for catalog update invalid case (yaml).
test/fixtures/input/flink/secret-mapping/create-successful.json Input fixture for secret-mapping create success (json).
test/fixtures/input/flink/secret-mapping/create-successful.yaml Input fixture for secret-mapping create success (yaml).
test/fixtures/input/flink/secret-mapping/create-invalid-failure.json Input fixture for secret-mapping create invalid case (json).
test/fixtures/input/flink/secret-mapping/create-invalid-failure.yaml Input fixture for secret-mapping create invalid case (yaml).
test/fixtures/input/flink/secret-mapping/update-successful.json Input fixture for secret-mapping update success (json).
test/fixtures/input/flink/secret-mapping/update-successful.yaml Input fixture for secret-mapping update success (yaml).
test/fixtures/input/flink/secret-mapping/update-invalid-failure.json Input fixture for secret-mapping update invalid case (json).
test/fixtures/input/flink/secret-mapping/update-invalid-failure.yaml Input fixture for secret-mapping update invalid case (yaml).
pkg/resource/resource.go Adds resource label constant for Flink secret mapping (used by deletion UX).
pkg/flink/cmf_rest_client.go Adds CMF REST client methods for catalog update + secret-mapping CRUD/list.
internal/flink/local_types.go Adds local types for serialized secret-mapping output.
internal/flink/command.go Registers the new secret-mapping command under flink.
internal/flink/command_secret_mapping.go Introduces secret-mapping command group + resource file parsing + output helpers.
internal/flink/command_secret_mapping_create.go Implements flink secret-mapping create.
internal/flink/command_secret_mapping_delete.go Implements flink secret-mapping delete with confirmation flow.
internal/flink/command_secret_mapping_describe.go Implements flink secret-mapping describe.
internal/flink/command_secret_mapping_list.go Implements flink secret-mapping list.
internal/flink/command_secret_mapping_update.go Implements flink secret-mapping update.
internal/flink/command_catalog.go Adds shared helpers for catalog output + resource-file parsing, and wires catalog update subcommand.
internal/flink/command_catalog_update.go Implements flink catalog update.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

internal/flink/command_catalog.go Outdated
Comment on lines +64 to +68
func readCatalogResourceFile(resourceFilePath string) (cmfsdk.KafkaCatalog, error) {
data, err := os.ReadFile(resourceFilePath)
if err != nil {
return cmfsdk.KafkaCatalog{}, fmt.Errorf("failed to read file: %w", err)
}
Copy link

Copilot AI Apr 5, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

readCatalogResourceFile introduces a shared parsing helper for catalog resource files, but catalog create (and other catalog commands) still inline similar parsing logic elsewhere. Consider reusing this helper across the catalog commands to avoid duplicated implementations that can drift in behavior/error handling over time.

Copilot uses AI. Check for mistakes.
@sonarqube-confluent
Copy link
Copy Markdown

sonarqube-confluent bot commented Apr 6, 2026

Quality Gate passed Quality Gate passed

Issues
10 New issues
0 Fixed issues
0 Accepted issues

Measures
0 Security Hotspots
90.6% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube

@paras-negi-flink
Copy link
Copy Markdown
Author

Paras Negi (paras-negi-flink) commented Apr 6, 2026

  1. Create Secret Mapping
parasnegi@C6V9RN9V2Y confluent_darwin_arm64_v8.0 % ./confluent flink secret-mapping create secret-mapping.yaml --url http://localhost:8080 --environment test
+---------------+--------------------------------+
| Creation Time | 2026-04-06T14:58:21.758923508Z |
| Name          | test-mapping                   |
| Secret Name   | test-secret                    |
+---------------+--------------------------------+
  1. List Secret Mapping (created above)
parasnegi@C6V9RN9V2Y confluent_darwin_arm64_v8.0 % ./confluent flink secret-mapping list --url http://localhost:8080 --environment test               
       Creation Time       |           Name           |   Secret Name     
---------------------------+--------------------------+-------------------
  2026-03-25T14:08:11.573Z | sec-kafka-conn-secret-id | sec-kafka-secret  
  2026-04-06T14:58:21.758Z | test-mapping             | test-secret
  1. Describe SecretMapping
parasnegi@C6V9RN9V2Y confluent_darwin_arm64_v8.0 % ./confluent flink secret-mapping describe test-mapping --url http://localhost:8080 --environment test --output json
{
  "apiVersion": "cmf.confluent.io/v1",
  "kind": "EnvironmentSecretMapping",
  "metadata": {
    "name": "test-mapping",
    "creationTimestamp": "2026-04-06T14:58:21.758Z",
    "updateTimestamp": "2026-04-06T14:58:21.758Z",
    "uid": "57b4a385-376c-472d-b777-571c36ef5bca",
    "labels": {},
    "annotations": {}
  },
  "spec": {
    "secretName": "test-secret"
  }
}
parasnegi@C6V9RN9V2Y confluent_darwin_arm64_v8.0 % ./confluent flink secret-mapping describe test-mapping --url http://localhost:8080 --environment test --output yaml
apiVersion: cmf.confluent.io/v1
kind: EnvironmentSecretMapping
metadata:
    name: test-mapping
    creationTimestamp: "2026-04-06T14:58:21.758Z"
    updateTimestamp: "2026-04-06T14:58:21.758Z"
    uid: 57b4a385-376c-472d-b777-571c36ef5bca
    labels: {}
    annotations: {}
spec:
    secretName: test-secret
  1. Update Secret-Mapping
parasnegi@C6V9RN9V2Y confluent_darwin_arm64_v8.0 % ./confluent flink secret-mapping update secret-mapping.yaml --url http://localhost:8080 --environment test --output json
{
  "apiVersion": "cmf.confluent.io/v1",
  "kind": "EnvironmentSecretMapping",
  "metadata": {
    "name": "test-mapping",
    "creationTimestamp": "2026-04-06T14:58:21.758Z",
    "updateTimestamp": "2026-04-06T15:07:36.432879126Z",
    "uid": "57b4a385-376c-472d-b777-571c36ef5bca",
    "labels": {
      "env": "test",
      "team": "platform"
    },
    "annotations": {}
  },
  "spec": {
    "secretName": "test-secret"
  }
}
  1. Delete Secret-Mapping
parasnegi@C6V9RN9V2Y confluent_darwin_arm64_v8.0 % ./confluent flink secret-mapping delete test-mapping --url http://localhost:8080 --environment test       
Are you sure you want to delete Flink secret mapping "test-mapping"? (y/n): y
Deleted Flink secret mapping "test-mapping".
parasnegi@C6V9RN9V2Y confluent_darwin_arm64_v8.0 % ./confluent flink secret-mapping list --url http://localhost:8080 --environment test                 
       Creation Time       |           Name           |   Secret Name     
---------------------------+--------------------------+-------------------
  2026-03-25T14:08:11.573Z | sec-kafka-conn-secret-id | sec-kafka-secret

@paras-negi-flink Paras Negi (paras-negi-flink) marked this pull request as ready for review April 7, 2026 17:01
@paras-negi-flink Paras Negi (paras-negi-flink) requested a review from a team as a code owner April 7, 2026 17:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

@rmetzger Robert Metzger (rmetzger) Awaiting requested review from rmetzger

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@paras-negi-flink