Skip to content

feat(cli): validate a cluster config file without planning or probes - #3189

Open
mvanhorn wants to merge 2 commits into
ashhart:mainfrom
mvanhorn:feat/config-validate
Open

mvanhorn wants to merge 2 commits into
ashhart:mainfrom
mvanhorn:feat/config-validate

Conversation

@mvanhorn

Copy link
Copy Markdown

feat(cli): validate a cluster config file without planning or probes

What this changes

Before this change there was no tensorfold config validate command. tensorfold cluster check still plans models and may need probes or memory_gb. After this change, tensorfold config validate FILE reads a cluster.json through the existing parser, prints a short confirmation (path, node count, model count) when the file is valid, or every problem the parser collected when it is not, and exits 0 / 1 / 2. It does not plan models, probe hosts, open an engine, or load a checkpoint. cluster check is unchanged.

  • Issue: n/a — not a new family, backend, or arithmetic; a parse-only CLI on the existing cluster.json schema.
  • Stacked on: main
  • atc overlap --json: ATC 0.9.6, run atc overlap --json --base origin/main at 3:56 AM PT, Fri Nov 13, 2026 (America/Los_Angeles) on head 1cf7bc7. The snapshot was incomplete (complete: false; limitations: otherPullRequestsLimit, pullFilesUnavailable, pullRequestsLimit, timeLimit). changedFiles: 10. myPullRequests: none. Two overlaps, both conflict: "unknown" (ATC could not run merge-tree): draft #2762 and draft #2763 each share zig/src/cli/cuda_main.zig. Manual review of those patches: they edit the --carveout usage and the carveout-open path; this change only adds a usage line for config validate and a dispatch next to the existing cluster-command branch. The regions do not overlap, so a merge would be clean. No overlap with open #2885 (serve --config FILE.toml).

Checks on this head

  • Head commit: 1cf7bc7f6606c95dc7758b84f71c9e3f96159161
  • Lean checker, python3 tools/zig/lean_check.py, on the base and on this head: 341 problems on origin/main (13d7ce0), 341 on this head, 0 new. None of the 10 changed files appear in the lean report.
  • Tests run, with the command and pass, fail and skip counts:
    • Fail before the change: zig build test-cluster-membership-host — validate runner 2 pass, 6 fail (8 total); the six new cases exited 2 from a stub run (nonzero).
    • Pass after: zig build test-cluster-membership-host — validate runner 10 pass, 0 fail, 0 skip; bringup runner 37 pass, 1 skip. Re-run on this head after review: the same step succeeded (Build Summary: 5/5 steps succeeded).
    • Host-only binary: zig build tf-config-validate, then tensorfold config validate FILE on a valid file (exit 0: fixtures/cluster.json: 6 nodes, 3 models), an unknown backend (exit 1: nodes[0] (a): backend must be "metal" or "cuda"), a wrong type (exit 1: nodes: must be an array), a conflicting layout (exit 1: tensor 3 does not divide 4 nodes, and mixed Metal/CUDA in one tensor-parallel group), a missing path (exit 1: no-such-cluster-9f3a.json: FileNotFound), and bad usage (exit 2: usage: tensorfold config validate FILE).
  • CI: will run on this pull request. Not run from this Linux host.
  • Not run, and why: GPU tests, a CUDA native product build, and test-cuda-host. This box has no GPU and the change never reaches a device.

Receipt

n/a — this command only reads a cluster.json and prints parse problems. It does not load a checkpoint, open an engine, or reach inference.

  • Main against branch: n/a (cannot reach inference)
  • Boxes it reaches: n/a (cannot reach inference)
  • New switch? none
  • Environment, with the TensorFold commit, Zig and backend toolchain, machine and GPU, checkpoint and revision: Zig 0.17.0 host, commit 1cf7bc7f6606c95dc7758b84f71c9e3f96159161, no GPU, no checkpoint
  • Paths and platforms the change reaches, which I ran and which I could not: Linux host zig build test-cluster-membership-host and zig build tf-config-validate. Metal tensorfold and the CUDA product binary were not built here. The command is also wired from zig/src/main.zig and zig/src/cli/cuda_main.zig.
  • Exactness, from tools/bench_concurrent.py --alone --serial or token_sha against "draft": false: n/a (cannot reach inference)
  • Main against this branch, same machine and checkpoint, posted before asking for a run on our machines (or why you can't): n/a (cannot reach inference)
  • Decode speed before and after, from tools/bench_openai.py: n/a (cannot reach inference)
  • Prompt speed before and after: n/a (cannot reach inference)
  • Licence of any new weights or drafter, or "n/a": n/a

Checklist

  • The title and description match the head commit, including the receipt and the stack.
  • New tests fail before the change, pass after it, and skip cleanly without their dependency.
  • Comments and docstrings are one line. No measurements or history in the source.
  • No personal data, machine names, internal hosts or local paths in the diff or its commits. No AI attribution lines.

Demo

Animated replay of real captured command output, not a live UI recording.

demo

`tensorfold config validate FILE` reads a cluster.json, prints every
problem the existing parser collected, and exits without planning models
or probing hosts.
@ashhart

ashhart commented Nov 14, 2026

Copy link
Copy Markdown
Owner

Thanks @mvanhorn, config validate is a handy addition. It reads the file with the same 16 MiB limit and config.parse the cluster commands use, so it accepts exactly what the server would load. It reports every problem, and its exit codes (0, 1, 2) are clear. The seven tests cover the useful cases.

One fix, in the docs:

  1. docs/evaluating.md:46 lists tensorfold config validate FILE among the smoke checks for the release binary, but the release binary doesn't have this command. Release archives ship tensorfold-native, and its entry point (zig/src/server/main.zig) has no config command. So on a release install the line prints the usage and exits 2. The smallest fix is to say the command comes from a source build (tensorfold built from source, or tf-cluster); wiring it into server/main.zig would work too.

Optional: the test "an unknown key exits 1 and names the key" actually checks an unknown backend value, because unknown keys are ignored, as the loader ignores them. A name like "an unknown backend exits 1 and names it" would match what it tests.

  • Synthena (Ash's Agent)

@mvanhorn

Copy link
Copy Markdown
Author

Good catch on the release binary. You're right that the archives only ship tensorfold-native and server/main.zig has no config command, so that smoke check line would print usage and exit 2 on a release install.

Of the two options I'd go with scoping the docs line to source builds (tensorfold from source, or tf-cluster). Wiring config into server/main.zig adds a command to the release binary's surface, which feels like its own change rather than something to slip into this PR.

Agreed on the test name too. It checks an unknown backend value, not an unknown key, so "an unknown backend exits 1 and names it" is the accurate name.

… test

The release binary, tensorfold-native, has no config command, so the
smoke check line now says the command comes from a source build. The
"unknown key" test checks an unknown backend value, so its name says so.
@mvanhorn

Copy link
Copy Markdown
Author

Pushed 66dfcd6: docs/evaluating.md now says config validate comes from a source build (tensorfold built from source, or tf-cluster) and that tensorfold-native doesn't have it. The backend test in validate_test.zig now carries the name "an unknown backend exits 1 and names it".

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants