Repository navigation
fix: a plan made from a --release-dir release applies, also from another checkout - #1022
Merged
Merged
Conversation
fluid ship --release-dir copied bundle.tgz without the provenance file fluid bundle writes beside it, so a plan made from the release's bundle recorded no source contract, and fluid apply refused a contract with a dbt build (apply_plan_digest_bundle_mismatch: the checkout to check it against cannot be found). publish() now writes bundle.tgz.provenance.json into the release with the same source contract, relative to the release directory (resolved as its readers resolve the bundle's directory), through bundle._write_provenance. release.json lists it in files; identity is unchanged. The file's format is bundle.py's, which borrows Helm's <chart>.tgz.prov pattern: a file beside the archive that names the archive's sha256.
A plan records the planning checkout's absolute paths. When a later CI job checks the repository out at another path, fluid apply plan.json --bundle B could not find a dbt build's project and refused the plan. When the recorded paths are gone, the plan now resolves its source contract through the bundle apply has just verified it against (its provenance file, relative to the bundle), for the dbt check, the build runners, binding paths and the IaC source. The recorded paths, under planDigest, still come first. The provenance file is unauthenticated, so resolve_source_contract's id check applies and the dbt files are still compared with the packed copy. Scoped per apply by a context variable, reset when the run returns. Pattern: locate the tree relative to the verified artifact, as Helm's .prov sidecar names files relative to its archive.
A bundle that names no usable source contract, or a release on another drive, still publishes without bundle.tgz.provenance.json, and a later apply of a plan made from it refuses a dbt build. fluid ship now says so where the release is made (release_provenance_missing). Tests now cover the apply side beyond the dbt check: _verify_plan_digests notes the verified bundle, and the binding-path anchor, the IaC source and the Spark workspace follow it while the run's scope holds. The changelog drops an unsourced GitLab claim and says the release must keep its place in the checkout.
📄 Documentation ReminderThis PR appears to be missing a documentation reference. Our docs live in a separate repo. Please update the PR description with one of:
See the Contributing Guide for details. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A CI release flow that publishes a packet with
fluid ship --release-dir, plans the packet's bundle in a later job, and applies that plan after approval could not reach apply for a contract with a dbt build. Two causes, two fixes:--release-dircopiedbundle.tgzwithoutbundle.tgz.provenance.json, so a plan made from the release's bundle recorded no source contract.fluid applythen refused withapply_plan_digest_bundle_mismatch("the checkout to check it against cannot be found").publish()now writes the file into the release, with the same contract relative to the release directory. It usesbundle._write_provenance, resolved as readers resolve a bundle's directory.release.jsonlists it infiles.identityis unchanged.fluid shipwarns (release_provenance_missing) when it cannot write one.planDigest, still come first._contract_loader._resolve_plan_source_contractresolves the contract through the provenance file beside the bundle that apply has just verified against the plan. A context variable carries that bundle, and_scopes_dbt_bindingresets it per apply.--no-verify-plan-bindingsets nothing.Reuse. The provenance file is the existing one (Helm's
.provpattern, already inbundle.py). The per-run scope mirrorsproject_files.set_bound_projects. Outside search: Terraform embeds a config snapshot in its saved plans so apply never needs the planning path. GitLab Runner's default build path holds the runner and a concurrency slot, so two jobs of one pipeline can check out at different paths. Here the plan's digest-covered path stays first, and a path relative to the verified artifact is the fallback.Validation
_verify_plan_digestsnotes the bundle, and the binding-path anchor, IaC source and Spark workspace follow it;--dry-run, then again in a moved checkout with the planning one gone, where an edited model is refused by name.apply_plan_digest_bundle_mismatch.contextvarswas already loaded on thefluid --helppath: 944 modules before and after.tofu init), and so does one applied from another checkout path.ripwire --quality-delta. The complexity regression is fixed. Rows left on purpose:
test_verify_databricks.py::contract_with_job;Reviews