Repository navigation
Redact sensitive variables from bundle plan and saved plans - #7713
andrewnester wants to merge 6 commits into
Conversation
bundle plan no longer prints or saves sensitive values: new_state is redacted once planning is done. A saved plan does not carry them, so deploy --plan requires every sensitive variable of the bundle to be set with --var or BUNDLE_VAR_<name>, and restores each sensitive field from the config when the config sets it. One check runs before approval and reports all missing values in one error. The prepareChanges warning no longer prints the change values, which were visible before redaction. Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
a9836b9 to
614d281
Compare
|
Commit: Regression test reportSelected 2 changed tests (limit=20, 0 not selected) 4 tests: regression=4 unreleased=0 coverage=0 failing=0
TestAccept/bundle/variables/sensitive/plan-derived/DMS=: branch ✅ | main (36fdeb1) ❌ | latest (v1.27.0) ❌main (36fdeb1): latest (v1.27.0): same output as main (36fdeb1) TestAccept/bundle/variables/sensitive/plan-derived/DMS=true: branch ✅ | main (36fdeb1) ❌ | latest (v1.27.0) ❌main (36fdeb1): latest (v1.27.0): same output as main (36fdeb1) TestAccept/bundle/variables/sensitive/plan/DMS=: branch ✅ | main (36fdeb1) ❌ | latest (v1.27.0) ❌main (36fdeb1): latest (v1.27.0): same output as main (36fdeb1) TestAccept/bundle/variables/sensitive/plan/DMS=true: branch ✅ | main (36fdeb1) ❌ | latest (v1.27.0) ❌main (36fdeb1): latest (v1.27.0): same output as main (36fdeb1) |
Co-authored-by: Isaac <no-reply@databricks.com>
The acceptance test checks that the literals stay out of plan output, the saved plan and local state, that a saved plan needs the values passed again, that an empty value fails before any state-changing request, and that a rotated value reaches only the PATCH request. Co-authored-by: Isaac <no-reply@databricks.com>
614d281 to
2212f0c
Compare
✅ Integration test reportCommit: fcecb40
Top 3 slowest tests (at least 2 minutes):
|
|
/review |
denik
left a comment
There was a problem hiding this comment.
Design: the deploy value isn't tied to the plan value. deploy --plan now sends whatever value is supplied at deploy time, so it can differ from the planned one. For example, a plan updates a secret's comment, and at deploy time BUNDLE_VAR_x holds a different (wrong) value. The PATCH then sends a value nobody reviewed in the plan, and nothing warns. Would a salted fingerprint of each sensitive field in the plan work, with a check at deploy time that the supplied value matches and an error if not? That would keep "the saved plan is what gets applied" true. At minimum, the changelog should say this more strongly.
| return nil | ||
| } | ||
|
|
||
| if entry.NewState != nil { |
There was a problem hiding this comment.
The rule "the plan must not be serialized after this" is only stated in the comment, and nothing enforces it. If someone later re-serializes new_state from the cache after this point, the secrets leak silently. Could we assert it, or add a test that fails if new_state is non-redacted at output time?
There was a problem hiding this comment.
Not asserted in code, because makePlan has no unit-test harness in this package. The check is at output time instead: the variables/sensitive/plan acceptance case pipes bundle plan -o json and the saved plan file through a check that fails the test if a secret literal appears in them. A change that re-serializes unredacted new_state before output fails there. I can add a unit test on the serialized plan if you want one at that level.
deploy --plan rejected every sensitive variable that had no --var or BUNDLE_VAR_ value, including variables a resource never uses and variables set by a default, a target or the variable-overrides file. The check now looks only at the resolved config value of each sensitive field the plan sets, so every source of a value counts the same, and the variable-overrides file is no longer a special case. The check reports one error per problem, and it reads the adapter only for resource types with sensitive fields. The acceptance case variables/sensitive/plan-derived covers unused derived, literal-default and target-set sensitive variables. Co-authored-by: Isaac <no-reply@databricks.com>
|
Thanks for the review. The changes for the other comments are in fcecb40. On the design point (the deploy-time value is not tied to the plan-time value): this PR does not add a fingerprint. A fingerprint would change the saved plan format and store secret-derived data in the plan file, so it needs its own decision. Meanwhile the changelog and PR description now say explicitly that |
|
Follow-up on the design point: the deploy-time value stays as documented in this PR (the plan format does not change). If a fingerprint check is wanted later, it can come as a separate PR with a plan-format version bump. |
Changes
bundle planoutput (text and-o json) and the saved plan file no longer contain sensitive variable values. The plan'snew_stateis redacted once planning is done, alongsideremote_stateandchanges.bundle deploy --planrestores sensitive fields from the current config and fails before approval when a value is missing. For every sensitive field: a non-empty config value is restored; a non-zero planned value with an empty config value is an error; both empty is skipped (output-only fields).CheckSensitivePlanValues, reports one error per problem. It looks at the resolved config value of each sensitive field the plan sets, so a value from--var,BUNDLE_VAR_<name>, a default, a target or the variable-overrides file all count. Sensitive variables that no planned resource uses need no value.deploy --planapplies the sensitive values supplied at deploy time. They are not compared with the values used at plan time, so a saved plan can be applied with a different secret value than was planned.prepareChangeswarning no longer prints change values. It previously showed old, new and remote values before redaction at the default log level.Why
Fixes #7685.
bundle planandbundle deployshowed and stored sensitive variable values thatbundle validate -o jsonalready redacts. Since a saved plan no longer carries them,deploy --planneeds the values supplied again, and it applies the values given at deploy time, not those at plan time.Tests
bundle/direct/redact_test.go): the restore table covers marker and plaintext replaced, zero in plan but set in config restored, both zero skipped, empty config value error, missing resource error, and nested paths. The check table covers a plan without sensitive resources, a set or zero value passing, and an empty config value. A separate test checks one error per resource.bundle/variables/sensitive/plan-derived: unused sensitive variables with a derived default, a literal default, and a dev-target value need no value, and a saved plan deploys with the credential from the variable-overrides file.bundle/variables/sensitive/plan: literals are absent from plan output, the saved plan and local state; a saved-plan deploy requires the values again; an empty value fails with no state-changing request; a plain-path create with real values; a rotate step where only the PATCH carries the new value. The test fails on main without this change.This pull request and its description were written by Isaac.