Skip to content

build: stop a stale stage0 from compiling the stdlib (and reporting it as an untracked error) - #1496

Merged
mbouaziz merged 3 commits into
mainfrom
fix-stale-stage-rebuild
Nov 19, 2026
Merged

mbouaziz merged 3 commits into
mainfrom
fix-stale-stage-rebuild

Conversation

@mbouaziz

@mbouaziz mbouaziz commented Jul 28, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #1408.

What this fixes

A stale stage0 silently compiles the current prelude and reports errors for language features it predates. A stage0/bin/skc older than @allow_tracked_call fails with 47 × Cannot call an untracked function from a tracked context, which reads exactly like the stdlib having been broken by the untracked migration. It has not been.

Two make-level defects combined to cause it (both diagnosed in #1408):

  1. stage0/lib/libstd.sklib had no dependency on stage0/bin/skc, so an outdated skc already sitting in stage0/bin was used as-is to compile the current prelude — and the bootstrap IR it should have been rebuilt from was never unpacked, so check-bootstrap never ran on this path either. Fixed on main by cb1adfc (stage0/lib/libstd.sklib: stage0/bin/skc <prelude srcs> …, a real file target).
  2. stage% names a directory and had no prerequisites, so as a file target make considers it up to date the moment the directory exists: on a warm tree make stageN (and plain make, make test, make install) printed "up to date"/"Nothing to be done" and recursed into nothing — reusing however stale a skc was in stageN/bin, surviving a git pull, a bootstrap bump and a rebuild. Fixed by this PR.

Rebase status (2026-11-19, onto main @ e3d2ba7, head 65f8b777)

Rebased from ef1d0618; four of the original seven commits are absorbed by main and were dropped:

Original commit Absorbed by
rebuild stage0/bin/skc before compiling the stdlib with it cb1adfc (real stage0/lib/libstd.sklib target with stage0/bin/skc prerequisite)
key the stage0 artifacts off their real inputs, not phony mtimes cb1adfc (PRELUDE_BUILD_SRCS; archive targets are real files now, so the order-only trick is unnecessary)
copy the prelude archives only when their contents change cb1adfc (archives are no longer .PHONY, so plain cp only fires on real changes; the remaining wasm .bc chain is #1495)
stop the one-recipe-two-outputs rules running their recipe twice 361d350 (&: grouped targets — one recipe, once even under -j, which also resolves the #1492 race)

What remains (3 commits):

  1. make the compiler stage targets phony — .PHONY: stage0 stage1 stage2 stage3 stageD with an explicit rule (phony targets skip implicit-rule search, so a stage% pattern would get "Nothing to be done"; naming them also turns make stage7 into an error instead of a silent no-op). Recursing every time is cheap: stage0's rules key off real inputs (cb1adfc) and stage1+ delegate freshness to skargo; under the default -j the sub-make inherits the jobserver and the make -j on skiplang/compiler races on shared stage target dir (state.db collisions, libbacktrace configure race) #1581 prerequisite ordering between stage skargo builds is unchanged.
  2. give stageD/bin/skc its sources, and stop it writing into stage$(STAGE) — no prerequisites meant it was never rebuilt after source edits; and with the default STAGE=1 its last line relocated stage1's skc.ll into stage1's deps dir (where make promote globs skc-*.ll) or aborted with mv: cannot stat after a successful compile. Now keyed on $(STAGED_SRC) $(PRELUDE_BUILD_SRCS) and names stageD literally.
  3. correct the bump_bootstrap.sh comment that justified make clean with stage-directory mtimes — the cleans stay, but for the real reasons: a true from-scratch verification and an unambiguous skc-*.ll glob.

Validation (rebased head 65f8b777, skiplabs/skip:latest, GNU Make 4.3, 8 cores)

  • Cold make STAGE=0: green. Warm make STAGE=0: 0.13s, recurses (phony) with every stage0 file target reported up to date — no clang++ relink, no stdlib recompile.
  • Contrast on the same warm tree with origin/main's Makefile: make stage0 → "'stage0' is up to date", make → "Nothing to be done" — no recursion at all.
  • Change detection still precise: touching a prelude src/*.sk fires the libstd.sklib recipe (plus the dependent skfmt/skargo relinks); rm stage0/bin/skc fires exactly the skc relink (real relink green); make -n stage7 → "No rule to make target 'stage7'".
  • Cold make STAGE=1: green in 9m26s; warm make STAGE=1 rerun: 24.8s (stage0 targets up to date in ~0.1s; the three stage1 skargo invocations re-run their own freshness check, everything Fresh, no relink); stage1/bin/skc --version runs.
  • shellcheck bin/bump_bootstrap.sh clean (make check-sh); no .sk/.c/.ts changes, so check-fmt is unaffected. Full CI re-runs on this head.
  • CI/Docker consumers are unaffected by construction: .circleci/base.yml (make STAGE=1 in a fresh container; the stage0 cache's mtime refresh makes the now-always-recursed file targets correctly up to date), skiplang/Dockerfile and root Dockerfile (make clean && make STAGE=0), bin/bump_bootstrap.sh (make clean before each stage) all build from clean trees, where phony vs directory target makes no difference.

Not in scope

#1408's third suggestion — a runtime diagnostic when const-init detects compiler/stdlib version skew — remains a backstop for skew this line of fixes prevents at the source. Follow-up.

The seven pre-existing same-class defects filed from the original review remain open and unaffected: #1490, #1491, #1492, #1493, #1494, #1495, #1498. (#1492's -j double-run hazard is now addressed on main by the &: grouped targets this branch no longer needs to split; #1495's wasm llc/llvm-ar chain is the one .PHONY-prerequisite case left.)


Original series written by Claude (Opus 5, 1M context) via Claude Code, on behalf of @mbouaziz. Rebased onto e3d2ba7 and re-validated with Mistral Vibe (Mistral Vibe CLI); commits carry the corresponding Co-Authored-By trailers. GNU make semantics verified empirically on GNU Make 4.3; build results from real runs inside skiplabs/skip:latest.

@mbouaziz
mbouaziz requested a review from jberdine July 28, 2026 15:32

Copy link
Copy Markdown
Contributor Author

Merged origin/main into this branch (7553c34) to clear the conflict — the PR no longer shows conflicts.

The only conflict was the stage0/bin/skc rule in skiplang/compiler/Makefile. Resolution keeps both sides: main's bootstrap_vtable prerequisite and -Wl,--build-id, plus this branch's order-only archive prerequisites, $(RUNTIME_SRC) as a real input, and an explicit link line ($^ would pass the runtime sources to clang++). The merge also picks up main's newer bootstrap submodule pointer (836f06e9, with the _vtables64 modules).

Verified in a clean worktree at 7553c34:

  • make clean && make STAGE=1 builds green end to end: stage0 skc relinked from the bootstrap IR incl. skc_vtables64.ll, stdlib compiled with that fresh skc, then stage1 via skargo; stage1/bin/skc --version reports skc 0.3.3 (2197c232).
  • Warm-tree make STAGE=1 rerun: ~7s, 0 clang++ relinks, and each recursive make -C ../prelude / skargo build invocation exactly once.
  • Touching build/skc_out64.ll (bootstrap bump) triggers exactly one skc relink — the stale-stage0 path from Build: stage0 compiler and stdlib can be built from inconsistent inputs, crashing at startup with no diagnostic #1408.
  • skargo test in skiplang/prelude with the stage1 toolchain: 131/131 pass.

Note: this environment needed LD_LIBRARY_PATH unset for clang/skfmt to run (agent-injected libstdc++ shadowing) — unrelated to this PR. CI should now run on the new head; review still needed from @jberdine.

@mbouaziz
mbouaziz removed the request for review from jberdine November 11, 2026 15:27
@mbouaziz
mbouaziz force-pushed the fix-stale-stage-rebuild branch from 7553c34 to df0f1ad Compare November 11, 2026 15:36

Copy link
Copy Markdown
Contributor Author

Rebased onto main (ef1d061) as the same 7 clean commits, no merge commit — new head df0f1ad (force-pushed; supersedes 7553c34 above).

The rebase had one conflict, in commit 2 (a4381e68) at the stage0/bin/skc rule: resolved by keeping both sides — main's bootstrap_vtable prerequisite and -Wl,--build-id, plus the order-only archive prerequisites, $(RUNTIME_SRC), and an explicit link line ($^ would pass the runtime sources to clang++). The final tree is byte-identical to 7553c34 (git diff 7553c342 df0f1ad2 is empty), so the earlier verification stands; additionally re-verified on the rebased head: full make STAGE=1 green, warm rerun ~8s with 0 relinks and each recursive make/skargo recipe exactly once, stage1/bin/skc reports skc 0.3.3 (df0f1ad2).

Still needs CI green + review from @jberdine.

mbouaziz and others added 3 commits November 19, 2026 11:18
…-opping

Each stage target names a directory and had no prerequisites, so as a file
target make considered it up to date the moment the directory existed. Once
you had built a stage, `make stageN` printed "'stageN' is up to date" and
recursed into nothing -- reusing whatever binaries were left in stageN/bin,
however old. That is how a stage0 built weeks earlier survived a git pull, a
bootstrap submodule bump and a rebuild: nothing ever asked make to look
inside the directory again.

Make the aggregates phony so they always recurse. Recursing every time is
cheap now: the stage0 rules key off real inputs (cb1adfc), and stages 1-3
delegate freshness to skargo, which tracks its own state. Under the default
-j the inner sub-make inherits the jobserver and the #1581 prerequisite
ordering between the stage skargo builds is unchanged. CI and Docker consumers
already build from clean trees (.circleci/base.yml, skiplang/Dockerfile,
bin/bump_bootstrap.sh), so this only changes behaviour on a warm local tree
-- which is the case it fixes: `make`, `make test` and `make install` now
re-check the stage artifacts instead of no-opping.

The stages are listed explicitly rather than matched with `stage%`: make
skips implicit-rule search for phony targets, so a phony target built by a
pattern rule gets "Nothing to be done" instead. Naming them also turns a
typo like `make stage7` into an error rather than a successful no-op.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
…e$(STAGE)

Two problems, both of the same silent-staleness kind the phony stage targets
fix:

`stageD/bin/skc` had no prerequisites at all, so once built it was never
rebuilt however much of the compiler or prelude changed. Its recipe already
computed the source list; hoist it into STAGED_SRC and use it (plus the
prelude build inputs, PRELUDE_BUILD_SRCS) as prerequisites, so the target is
genuinely tracked and the find runs once instead of twice.

The last line then moved the emitted IR out of `stage$(STAGE)`, not stageD.
That is only coherent under `make STAGE=D`; with the default STAGE=1 it
relocates stage1's skc.ll and plants a stray skc-d.ll in stage1's deps
directory, where `make promote` globs skc-*.ll and would match two files.
On a tree without a stage1 it instead aborts with `mv: cannot stat` after
the whole compile has already succeeded. Name stageD literally: identical
under STAGE=D, correct for `make stageD`.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
The comment justified the second `make clean` with "the stage rules key off
stage directory mtimes, so a partial clean would reuse stale stages". They no
longer do: the stage aggregates are phony and the stage rules key off real
inputs, so a warm tree rebuilds exactly what changed.

The cleans are still right, for the reason the first one already states: a
from-scratch bootstrap is the point of the verification, and it keeps the
skc-*.ll glob that `make promote` copies from unambiguous. Say that instead
of describing behaviour that no longer exists.

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
@mbouaziz
mbouaziz force-pushed the fix-stale-stage-rebuild branch from df0f1ad to 65f8b77 Compare November 19, 2026 11:34

Copy link
Copy Markdown
Contributor Author

Rebased onto main @ e3d2ba7 and force-pushed — new head 65f8b777, now 3 commits (was 7).

Commits 1, 2, 5 and 7 of the original series are absorbed by main (cb1adfcd incremental stage0/archive rules, 361d350e parallel stages with &: grouped targets) and were dropped; the remaining commits are the phony stage aggregates, the stageD/bin/skc sources + stage$(STAGE) write fix, and the bump_bootstrap.sh comment. One adaptation the rebase required: the stageD rule now keys off main's PRELUDE_BUILD_SRCS instead of the dropped commit's RUNTIME_SRC. Details and fresh validation (cold make STAGE=1 9m26s green, warm reruns recurse and no-op correctly, change-detection checks, old-vs-new contrast) are in the updated PR body. CI is re-running on the new head.

@mbouaziz
mbouaziz merged commit 9e9eab9 into main Nov 19, 2026
22 checks passed
@mbouaziz
mbouaziz deleted the fix-stale-stage-rebuild branch November 19, 2026 12:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Build: stage0 compiler and stdlib can be built from inconsistent inputs, crashing at startup with no diagnostic

1 participant