Skip to content

docs(spark): fluid apply stops a removed build's runs before it destroys what they run with (forge-cli #1008) - #265

Merged
fas89 merged 3 commits into
mainfrom
claude/docs-spark-aws-wave12
Nov 19, 2026
Merged

fas89 merged 3 commits into
mainfrom
claude/docs-spark-aws-wave12

Conversation

@fas89

@fas89 fas89 commented Nov 19, 2026

Copy link
Copy Markdown
Collaborator

Summary

Documents forge-cli #1008, on main (3492ca20) after the 0.33.0 cut and in no release: when a plan deletes the resources of a Spark build on AWS (the build left the contract, or the product went), fluid apply stops the build's runs before tofu apply destroys the role, the artifacts and the log group they run with.

  • advanced/spark-iceberg-aws.md: a new section, "When a build leaves the contract": the refusal of a policy that is still attached (with the detach commands), the stop of each build's runs, of a Glue job's runs and of the product's own EMR Serverless application, the refusal when a run is still going after FLUID_SPARK_WAIT_SECONDS, what --dry-run does, the three tags that say where a build runs (and the note for a build an earlier forge deployed), and the extra actions the identity that runs fluid apply needs. Two rows in the Errors table (spark-policy-attached, spark-destroy-runs-going).
  • advanced/environment-variables.md: the FLUID_SPARK_WAIT_SECONDS row also covers the destroy.
  • upgrading.md: a new section, "Not in a release yet", with one bullet. The earlier section of that name is "Changes in 0.33.0" since docs: every forge-cli change in 0.33.0 reads since 0.33.0, and the releases map is refreshed #263.

Related CLI PRs

Areas Updated

  • Getting Started
  • CLI Reference
  • Provider Docs
  • Walkthroughs / Examples
  • Contributing / Community Docs
  • Other: advanced guide, upgrading

Decisions

  • The markers read *On forge-cli main, not yet in a release ([forge-cli #1008](...)).*: the pull request merged after the 0.33.0 cut. The pin stays at 0.32.0.
  • The output is real but not from AWS: it is what forge-cli's destroy step, the code fluid apply calls with the plan's deletes before tofu apply, prints, with AWS answered by botocore's test stubs (which check each call against the real API model) and the plan's deletes built as forge-cli's own tests build them. The section says so. The wiring of the plan's deletes into that step was read from the code. Nothing was destroyed on AWS.
  • What was not proven offline, and is worded as forge-cli's own account: what a 0.32.0 destroy did on AWS (the code shows it had no step for this).
  • Limits the review found in the code are in the text: a Glue job's runs are the 25 most recent that the job lists, EMR on EKS is one page, a build that moves off EMR Serverless or EMR on EKS keeps its log group, so the apply does not look for its old runs, and a still-attached policy refuses a dry run as well.

Checklist

  • python scripts/check_cli_docs.py, check_providers.py and gen_contract_reference.py --check pass, with forge-cli's live CHANGELOG.md
  • Site build, built-link check, offline link check and the new anchors pass locally

Notes

@fas89
fas89 merged commit 85df142 into main Nov 19, 2026
9 checks passed

This branch was previously deployed

1 inactive deployment
github-pages — 8641f4cb Deployed Nov 19, 2026 by fas89 via deploy #413
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant