Skip to content

docs(spark): removing a Spark build on AWS is not a data loss (forge-cli #1007) - #266

Merged
fas89 merged 1 commit into
mainfrom
claude/docs-spark-aws-wave13
Nov 19, 2026
Merged

fas89 merged 1 commit into
mainfrom
claude/docs-spark-aws-wave13

Conversation

@fas89

@fas89 fas89 commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Documents forge-cli #1007, on main (3a390592) after the 0.33.0 cut and in no release: removing a Spark build on AWS is no longer a data loss for the data-loss gate of fluid apply.

  • advanced/spark-iceberg-aws.md, "When a build leaves the contract": what the gate no longer counts (the build's runner and job config, jar copies, log group, role and policies, Glue job and connection, the product's own EMR Serverless application, the scheduler policy, the log key, the Lake Formation registration role), the line fluid plan --tofu and fluid apply print for them, what still counts (an output table, a bucket, a bucket's own key, and a stream's dead-letter table, which goes with its build), the refusal and its count, and the fail-closed rule.
  • cli/apply.md, "OpenTofu data-loss gate": one paragraph, since the list there is by resource type and this exemption is by type and by the end of the name.
  • upgrading.md: one bullet under "Not in a release yet".

Related CLI PRs

Areas Updated

  • Getting Started
  • CLI Reference
  • Provider Docs
  • Walkthroughs / Examples
  • Contributing / Community Docs
  • Other: advanced guide, upgrading

Decisions

  • The markers read *On forge-cli main, not yet in a release ([forge-cli #1007](...)).*: the pull request merged after the 0.33.0 cut. The pin stays at 0.32.0.
  • The two outputs are real but not from a tofu run: they are forge-cli's plan review (the code fluid plan --tofu and fluid apply run after tofu plan), given as the plan's deletes the resources that the page's example contract adds to the module with its build, computed the way forge-cli's own test computes them, and for the second output the contract's output table. The text says so. No state or AWS call was involved.
  • A build with a dead-letter table (sink: table) still needs --allow-data-loss: the build adds that table, and it goes with the build. The first sentence of the section, the apply.md paragraph and the upgrading bullet say "unless the build has a dead-letter table", and a run shows the table as gated.
  • fluid plan --tofu warns that the apply will refuse, and does not refuse itself; the text says so.

Checklist

  • python scripts/check_cli_docs.py (with forge-cli's live CHANGELOG.md), check_providers.py and gen_contract_reference.py --check pass
  • Site build, built-link check, offline link check and the linked anchors pass locally

Notes

  • A reviewer checked 46 sentences against the code and 17 runs. Its four findings are fixed: the lead sentence and the upgrading bullet did not say that a dead-letter table still counts, the pointer to "What forge provisions" left out the two sections that describe the jar copies and the registration role, and the apply.md paragraph did not name the log key next to its "a product's KMS key counts" sentence.
  • The code of #1007 on main equals the code the text was written and reviewed against (the pull request's head 984a06de) for all five files it touches.
  • Docs main (docs(spark): fluid apply stops a removed build's runs before it destroys what they run with (forge-cli #1008) #265) is the base.

…cli #1007)

The data-loss gate no longer counts what a removed build ran with. A stream's dead-letter table still counts.
@fas89
fas89 merged commit a44a842 into main Nov 19, 2026
9 checks passed

This branch was previously deployed

1 inactive deployment
github-pages — fc4f7753 Deployed Nov 19, 2026 by fas89 via deploy #415
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant