diff --git a/CHANGELOG.md b/CHANGELOG.md index 8720e7a..3a03bba 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -13,7 +13,7 @@ Initial Aevonix FlashAlign release: - Managed Always On API gateway for clients that do not use a harness or the SDK. - Tool defaults and exact-name overrides for model access and child inheritance. - Setup wizard with harness detection, model discovery and compatibility checks. -- Shared FlashAlign terminal design with a sharp pixel logo, a brief startup +- Shared FlashAlign terminal interface with a pixel logo, a brief startup animation, keyboard selection panels and plain-text fallbacks. - Five included experimental starter profiles for GLM, DeepSeek, Qwen and Gemma, fitted and packaged by Aevonix from [credited checkpoints](profiles/SOURCES.md). diff --git a/README.md b/README.md index 7069f8a..c066032 100644 --- a/README.md +++ b/README.md @@ -15,8 +15,8 @@ channels, or keep a profile active across every routed request. FlashAlign works with supported base and quantized weights without permanently changing them. Run it directly, with Hermes or OpenClaw, or through your own -harness. Managed inference runs on Linux with NVIDIA GPUs, including DGX Spark -for the qualified TensorFold paths. +harness. Managed inference runs on Linux with NVIDIA GPUs. See +[compatibility](docs/compatibility.md) for DGX Spark support and engine requirements. The V1 starter profiles focus on refusal reduction. Future profiles could also increase refusals for public or higher-risk tasks. @@ -60,17 +60,15 @@ Fitting and runtime packaging by Aevonix. See the | Model | Supported engines | | --- | --- | -| DeepSeek V4.1 Flash 1017 | Pinned native vLLM with official FP8, or TensorFold 1.2.0 with the declared EXL3 base | +| DeepSeek V4.1 Flash 1017 | vLLM, TensorFold | | GLM 5.3 Full | TensorFold | | GLM 5.3 Flash | TensorFold, SGLang | | Qwen3.8-27B | TensorFold, vLLM, SGLang, llama.cpp | | Gemma 4 26B A4B | TensorFold, vLLM, SGLang, llama.cpp | See [compatibility](docs/compatibility.md#engines-and-models) for hardware -requirements, checkpoint formats and feature limits. The [validation results](docs/validation.md) -record the configurations tested. The included [starter profiles](profiles/README.md) -remain experimental; setup checks that the selected profile matches the model -and engine. +requirements, pinned checkpoints and feature limits, and +[validation](docs/validation.md) for tested configurations and results. Switching happens per request among profiles already loaded by the engine. Loading a new model or profile happens beforehand and can require an engine diff --git a/assets/demo/README.md b/assets/demo/README.md index 0d01c08..ccd84cc 100644 --- a/assets/demo/README.md +++ b/assets/demo/README.md @@ -5,27 +5,25 @@ The [README preview](flashalign-promo-preview.gif) is a silent excerpt of its first 8 seconds. -This video was recorded with the version 1.0 compact DeepSeek profile. -Version 1.1 replaces that starter with the native Keys profile; the video -does not demonstrate the replacement artifact. +This video uses the retired version 1.0 compact DeepSeek profile, rather than +the current native Keys starter. -The promo is an edited video. Its model text comes from three recorded +The video uses three recorded DeepSeek V4.1 Flash 1017 request pairs (EXL3 3.5-bpw, TensorFold 1.1.9, DSpark active, temperature 0, seed 20261030, thinking off), with FlashAlign off and on (`deepseek41-flash1017-v1`, strength 1). Each pair was recorded twice with matching text. The prompts are OR-Bench Hard-1K rows 321, 244 and 194, shown exactly as tested. -Every response on screen is an exact, continuous excerpt of the recorded text; -an ellipsis marks omitted text. In all three examples the stock model also -pointed to other help after refusing; the video shows only the refusal. The -recipe's glow claims were not tested. +Responses are continuous excerpts of the recorded text, with ellipses marking +omissions. The stock model also suggested other help after each refusal; +those suggestions are omitted. Recipe claims in the model output were not +verified. -The text reveal is animated for the edit, so the video does not show response -speed. These are selected examples, not a benchmark score; see the -[measured results](../../docs/benchmark-results.md). The setup scene shows the -installer's real output, shortened. The music and sound effects were made for -this video. +The animated text reveal does not represent response speed. These selected +examples are separate from the [measured results](../../docs/benchmark-results.md). +The setup scene uses shortened installer output. Music and sound effects were +created for the video. Prompts: [OR-Bench](https://huggingface.co/datasets/bench-llm/or-bench) by Justin Cui, Wei-Lin Chiang, Ion Stoica and Cho-Jui Hsieh, revision `e36d8b80`, diff --git a/docs/agent-setup.md b/docs/agent-setup.md index f724de0..1881419 100644 --- a/docs/agent-setup.md +++ b/docs/agent-setup.md @@ -2,14 +2,9 @@ An agent can detect harnesses, validate profiles and apply configuration without interactive prompts. Preparation stops before starting engines or restarting -gateways; `--activate` performs that step. Agent setup uses the same profile -checks and configuration writer as the wizard. OpenClaw preparation records a -native plugin installation without restarting the gateway. Its startup service -checks bridge readiness before serving calls. -For TensorFold on one host or several Sparks, read [TensorFold setup](native-tensorfold.md). -It stages checked binaries, libraries, source and the profile on each rank. -Only commands that authenticate requests receive the signing key. Preparation -then stops before activation. +gateways; `--activate` performs that step. It uses the same checks as the wizard. +For TensorFold on one host or several Sparks, read +[TensorFold setup](native-tensorfold.md). For a fresh deployment, use a [managed engine plan](engine-setup.md#agent-preparation-and-activation) to prepare a supported engine, the profile and its deployment key automatically. @@ -42,7 +37,7 @@ For a local terminal plan, prepare the harness and report the printed an interactive step, not an unattended gateway restart. The [starter profiles](../profiles/README.md) are discovered from local -metadata without a private login. Four compact profiles are included in full. +metadata. Four compact profiles are included in full. DeepSeek's native manifest declares a 1.09 GB public payload; managed preparation fetches missing bytes into an external cache and verifies them before staging. The offline inspector never fetches a missing payload. For offline preparation, @@ -53,8 +48,7 @@ must match the pinned target or a declared serving base. The native DeepSeek starter supports the pinned vLLM container with official FP8 weights, or the TensorFold 1.2.0 bundle with its declared EXL3 base and original Engram source. Both require four SM120 GPUs and strength 1.0. Projection-only backends cannot -load this profile. Preserve the profile's experimental status and the public package's -qualification status when reporting setup results. +load this profile. Choose a managed engine plan when the model needs an engine or profile setup. Choose the existing-endpoint plan below when an engine already implements the @@ -178,5 +172,5 @@ authorized, run only the selected commands. In Verified contacts mode, verify an approved native message and an unapproved message through the harness. In Always On, verify both ordinary and delegated requests use the configured profile and that an attempted per-request stock override cannot bypass it. Mode changes do -not re-enable a disabled deployment. A successful setup check does not -establish native message routing, profile numerical parity or model quality. +not re-enable a disabled deployment. See [validation](validation.md) for +profile and runtime test coverage. diff --git a/docs/benchmark-results.md b/docs/benchmark-results.md index 6960dfc..c864913 100644 --- a/docs/benchmark-results.md +++ b/docs/benchmark-results.md @@ -1,18 +1,17 @@ # Measured results -These are small qualification runs from 30 October 2026. They show measured -behavior on the listed configurations, not a score for every supported model -or a maximum-throughput claim. [Run your own comparison](benchmarks.md) with the -included recorder and optional evaluation tools. +These are small qualification runs. Results apply to the listed +configurations; they are not full capability, refusal-rate or +maximum-throughput benchmarks. [Run your own comparison](benchmarks.md) with +the included recorder and optional evaluation tools. -The DeepSeek results below used the version 1.0 compact profile. Version 1.1 -replaces that starter with the native Keys profile and a different engine path. -These earlier measurements do not measure the replacement profile. +**The DeepSeek measurements use the retired version 1.0 compact profile.** +The current native Keys starter uses a different artifact and engine path. -The [aggregate results](../benchmarks/results/native-qualification-20261030.json) -include exact model revisions, engine and profile hashes, settings recorded by -the checks, sample counts and timing values. Prompts, response logs and private -deployment configuration are not included in that file. +The [30 October 2026 aggregate results](../benchmarks/results/native-qualification-20261030.json) +include model revisions, engine and profile hashes, recorded settings, sample +counts and timing values. They omit prompts, response logs and private +deployment configuration. ## DeepSeek: the same tasks in both modes @@ -33,18 +32,15 @@ prompts are the `linux-process` and `photography` cases in the | Portrait photography checklist | 256 | 150.29 | 136.11 | 0.458 s | 0.445 s | | Merge-intervals code | 230 | 278.74 | 258.40 | 0.399 s | 0.441 s | -Values are arithmetic means. Decode speed is the native engine's reported -per-request decode rate; it excludes prefill. TTFT is measured by the client -and includes request, queue and network time. These are sequential requests, -not a concurrent server-throughput test. Speed depends on the output and draft -acceptance. These measurements do not establish speed at the 64K preset or at -another GPU power limit. +Values are arithmetic means from sequential requests. Decode speed is the +engine's per-request rate, excluding prefill; TTFT includes client, queue and +network time. Output and draft acceptance affect speed. The 64K preset, other +power limits and concurrent throughput were not measured here. The retained summary does not contain the complete generation-parameter -manifest for these speed runs. Treat them as qualification measurements rather -than a fully reproducible benchmark report. The code case completed in both -modes and produced the same output hash, but only two of its three generated -self-tests passed in either mode. Timing does not establish coding accuracy. +manifest, limiting reproducibility. The code case completed with the same +output hash in both modes, but only two of its three generated self-tests +passed in either mode. ## DeepSeek: 64K input and separate cache reuse @@ -58,21 +54,18 @@ returned the expected retrieval answer on all six requests. | First request in each mode | 15.585 s | 14.809 s | 0 | | Two later requests per mode | 0.665–0.871 s | 0.609–0.653 s | 63,872 | -“First” means no prefix reused for that request. The model was already loaded; -this is not a cold model-start measurement. The requests alternated modes. -Each mode initially reported no cached prefix, then reused its own prefix. -The six-token answers make this a cache and retrieval check, not a useful -long-output decode benchmark or a general long-context reasoning score. +The model was already loaded. Requests alternated modes; each mode first +reported no cached prefix, then reused its own. Six-token answers make this a +cache and retrieval check, with no long-output decode or general reasoning score. ## Gemma: a small GSM8K pilot TensorFold with Gemma 4 26B A4B, MLX-format 4-bit weights and DFlash active on one -Linux/NVIDIA DGX Spark scored **14/16 off** and **15/16 on**. This checkpoint format -does not imply use of Apple's MLX runtime. The test used the first 16 GSM8K test -questions, five examples per question and a 512-token output budget. +Linux/NVIDIA DGX Spark scored **14/16 off** and **15/16 on**. MLX describes the +checkpoint format; the runtime was TensorFold. The test used the first 16 +GSM8K test questions, five examples per question and a 512-token output budget. -This fixed-prefix pilot does not establish a quality improvement. The -[Gemma result file](../benchmarks/results/gemma-native-gsm8k-pilot.json) records +The [Gemma result file](../benchmarks/results/gemma-native-gsm8k-pilot.json) records the dataset, evaluator, model and engine revisions and extraction metrics. ## Qwen: runtime checks and a GSM8K pilot @@ -80,19 +73,26 @@ the dataset, evaluator, model and engine revisions and extraction metrics. Qwen3.8-27B EXL3 2.0-bpw passed 26 TensorFold checks on one SM120 GPU: 16 prose/code fixture requests, four structured-output/tool cases, and six authorization or concurrent-request cases. DFlash2 stayed active, with FP8 KV, -two streams and a 4,096-token context. This is functional coverage, not a -capability benchmark or a throughput measurement. +two streams and a 4,096-token context. A separate GSM8K pilot scored **31/32 off** and **31/32 on**, using the first 32 test questions, five-shot prompting, seed 20261030 and a 512-token output budget. DFlash2 remained active. Both strict and flexible answer extraction gave the same counts. The [pilot record](../benchmarks/results/qwen-native-gsm8k-pilot.json) -contains the exact model, engine, dataset and evaluator revisions. This small -fixed-prefix sample does not establish full-benchmark parity. +contains the exact model, engine, dataset and evaluator revisions. -Full refusal-rate, over-refusal and cross-model capability benchmarks are not -published yet. See [validation](validation.md) for the other model and engine -checks, including failures and limits. +The two GSM8K pilots use small, non-random samples and do not establish general +quality gains or parity. Full refusal-rate, over-refusal and cross-model +capability results are not published. See [validation](validation.md) for +other model and engine checks. + +## Qwen SGLang: startup and TTFT + +The BF16 Qwen3.8-27B SGLang 0.5.20 run used DFlash2 on one RTX PRO 6000 +Blackwell Max-Q. Two individual warm streaming requests measured TTFT of +**0.113 seconds off** and **0.139 seconds on**. Startup took **295 seconds** +with first-time kernel compilation and **33 seconds** with those kernels cached. +These are individual samples from that deployment, not latency distributions. ## llama.cpp: short prose comparison @@ -110,11 +110,9 @@ sequentially with no instrumented build or simultaneous generation. The [llama.cpp result file](../benchmarks/results/llamacpp-qualification-20261030.json) contains the original library-explanation prompt, generation settings and -individual measurements. The fixed output cap measures a 256-token prefix, -not completion of the requested explanation. Decode rate is reported by the -engine; TTFT is measured by the streaming client. GPU power limits were -observed after the run at 325 W; they were not sampled throughout it. -Thinking was off. Ngram produced no draft matches on this prose prompt; its -active drafting was verified separately with a copy fixture. -Two samples per mode do not establish a general performance difference, -maximum throughput or answer quality. +individual measurements. The 256-token cap cuts off the explanation. Decode +rate comes from the engine; TTFT comes from the streaming client. Thinking was +off. Ngram had no draft matches on this prompt; a separate copy fixture +confirmed active drafting. GPU power limits were observed at 325 W after the +run, with no continuous power record. Two samples per mode are insufficient +to establish a general performance difference. diff --git a/docs/benchmarks.md b/docs/benchmarks.md index 121f6e1..6e77a59 100644 --- a/docs/benchmarks.md +++ b/docs/benchmarks.md @@ -5,9 +5,8 @@ FlashAlign off and with one loaded profile. Keep speculative decoding enabled if it is enabled in the deployment. The benchmark client signs requests; it does not change the engine or its caches. -The included tooling records responses and timings. It does not supply a -refusal score by guessing from the first few words of an answer. A benchmark -result needs both the responses and a documented scoring procedure. +The included tooling records responses and timings. Refusal scores require a +separate, documented scoring procedure. ## Choose the measurement @@ -24,10 +23,9 @@ tools. Their datasets, classifiers and dependencies are not included with FlashAlign. Obtain them separately and retain their licenses and citations. XSTest prompts are CC-BY-4.0; generated responses have separate provenance. -Start with a predeclared small subset to validate the workflow. A 25-prompt -pilot is a pilot, not a full benchmark. Report every completed and failed case, -the selection seed, and both modes. Choose any demonstration case separately -from the aggregate score. Do not remove cases because their result is awkward. +Start with a predeclared small subset. Report its size and selection seed, +completed and failed cases, and both modes. Keep all selected cases in the +aggregate, and label any demonstration clips as selected examples. ## Record a controlled comparison @@ -49,11 +47,8 @@ directory. Replace the manifest placeholders with the actual deployment. Hash the loaded `profile.safetensors`, record the checkpoint revision and quantization, and record whether speculation is active. Include engine commit, weights digest, tokenizer revision and settings digest when available. The -manifest is operator-supplied provenance; the recorder cannot establish that -the server loaded the stated weights from a model name alone. - -The same interface can select any compatible model/profile. The recorder does -not expand the model/engine compatibility matrix. +manifest records operator-supplied provenance. Verify the loaded checkpoint +against the server configuration and use a [supported pairing](compatibility.md). ```sh python -m flashalign.benchmark record \ @@ -71,12 +66,10 @@ reasoning when the server exposes it. Keep it private. `aggregate.json` contains counts, provenance and timing statistics without prompts or responses. Inspect its operator-supplied metadata before publishing it. -The runner alternates which mode goes first. Warmup requests are labelled and -excluded from measured statistics. It does not flush caches, and it does not -claim a cold or warm cache from request order. If a test needs a cold-cache -condition, establish and document that separately. Sequential runs control -interference better than concurrent side-by-side calls; they do not measure -maximum server throughput under load. +The runner alternates which mode goes first and excludes labelled warmups +from statistics. Caches are not flushed; establish and document cold-cache +conditions separately. Requests are sequential, so these measurements cover +per-request performance rather than maximum throughput under load. TTFT is measured from the client request start to the first nonempty content or reasoning delta. First visible answer text is reported separately. Total @@ -97,20 +90,19 @@ There are three distinct speed fields: - `output_tokens_per_total_s`: completion tokens divided by the full client request time, including queueing, prefill and network time. -Missing values stay null. SSE chunks are never counted as tokens. Output token -counts can include reasoning. Report the exact metric beside any speed claim; -do not compare an estimate with another engine's internal decode timer. +Missing values stay null. Token counts come from server usage and can include +reasoning. Report the exact speed metric; streaming estimates and internal +engine timers measure different intervals. -The recorder also preserves the server's `usage.prompt_tokens_details.cached_tokens` -as `cached_prompt_tokens`. An absent value is unknown, rather than zero. Use +The recorder preserves `usage.prompt_tokens_details.cached_tokens` as +`cached_prompt_tokens`; an absent value is unknown. Use [long-context.jsonl](../benchmarks/long-context.jsonl) for a repeatable original reading fixture, then repeat the normal recording command with that cases file. Run it separately at each profile strength with `--scale`. Retain warmup records to compare the first request at a new profile/strength with later requests. -Reported cache hits demonstrate reuse; they do not by themselves prove isolation. -Check that a new profile or strength cannot reuse a previous mode's prefix and -that each mode subsequently reuses its own prefix. The fixture's actual token -count depends on the tokenizer and appears in every completed record. +Check both cache reuse and isolation: a new profile or strength must not reuse +another mode's prefix, and each mode should subsequently reuse its own. The +fixture's token count depends on the tokenizer and appears in each completed record. For a larger reading test, generate original fictional observatory records: @@ -120,10 +112,9 @@ python benchmarks/prepare_long_context.py --paragraphs 1000 \ ``` Use its `cases.jsonl` with the same recorder command. Choose a paragraph count -that fits the deployment's context after reserving output space. The generator -does not claim a token count; verify `prompt_tokens` in the resulting record. -Its single retrieval answer is `Cedar, Tuesday`. This tests one retained fact -and cache reuse, not general long-context reasoning quality. +that fits the deployment's context after reserving output space, and check +`prompt_tokens` in the resulting record. The fixture tests retrieval of one +fact (`Cedar, Tuesday`) and cache reuse. ## Score XSTest @@ -138,15 +129,15 @@ python benchmarks/prepare_xstest.py \ ``` Record the resulting `cases.jsonl` using the command above. The importer writes -the input hash and selection parameters beside it. It does not mark imported -prompts as reviewed for public demonstration. +the input hash and selection parameters beside it. Review prompts separately +before using them in a public demonstration. Use XSTest's full-compliance, partial-refusal and full-refusal taxonomy. For a small pilot, independent manual review is practical: randomize and blind the mode labels before review, then join judgments to `record_id`. For a larger run, pin a semantic classifier and its rubric, retain its raw judgments -privately, and manually audit a declared sample. Do not use a refusal-phrase -matcher as the headline score. +privately, and manually audit a declared sample. Score meaning and compliance, +rather than matching refusal phrases. Fill `judgments-template.jsonl` with one label per completed measured record and the evaluator's name, revision and method. Then run: @@ -158,11 +149,10 @@ python -m flashalign.benchmark summarize \ --output /private/experiment/scored-aggregate.json ``` -The summary keeps safe and unsafe groups separate and reports the labelled -denominator. Partial labeling is explicitly incomplete. Full compliance on an -unsafe prompt is not interchangeable with harmful task completion. Use -HarmBench's behavior-aware semantic classifier for that latter measure; pin -the classifier revision, token truncation budget and behavior dataset. +The summary keeps safe and unsafe groups separate, reports the labelled +denominator and marks partial labeling incomplete. For harmful task completion, +use HarmBench's behavior-aware semantic classifier rather than the XSTest +compliance label. Pin its revision, token truncation budget and behavior dataset. ## Retained capability with lm-evaluation-harness @@ -202,24 +192,21 @@ Ordinary MMLU needs completion logprobs/echo support and the matching tokenizer. A chat endpoint alone cannot supply that score. For generation-only tasks, use `model_class("chat")`, a `/chat/completions` URL and the task's appropriate chat template. Pass `apply_chat_template=True` to `lm_eval.simple_evaluate` -when using the chat adapter. A generative multiple-choice variant needs its own name and -must not be labelled standard MMLU. Stop if the engine does not support the -required API; do not switch checkpoints or engines in one half of a pair. - -The adapter's synchronous and asynchronous request paths have mock protocol -coverage. Full upstream task runs and engine-specific logprob compatibility -must be verified on the chosen serving deployment. Do not describe this as a -completed capability benchmark until those jobs run. Preserve task/dataset -revisions, few-shot examples, seed, generation budget and scoring version for -both halves. Export aggregate `results` and sample counts only after reviewing -the upstream result object for private paths or metadata. +when using the chat adapter. Label generative multiple-choice variants +separately from standard MMLU. Both halves of a comparison must use the same +checkpoint and engine with the required API support. + +The adapter's synchronous and asynchronous paths have mock protocol coverage; +full task runs and engine-specific logprob compatibility need verification on +the chosen deployment. Preserve task/dataset revisions, few-shot examples, +seed, generation budget and scoring version for both modes. Review aggregate +`results` and sample counts for private paths or metadata before publishing. ## Make the side-by-side demo -The original [demo cases](../benchmarks/demo-cases.jsonl) are benign. If the -model answers both modes identically, show that result or try a separately -disclosed case. The exporter does not invent contrast, and a selected clip is -not a benchmark score. +The original [demo cases](../benchmarks/demo-cases.jsonl) are benign. Both modes +may answer identically. Any additional case should be disclosed separately +from the aggregate benchmark. ```sh python -m flashalign.benchmark demo \ @@ -227,11 +214,10 @@ python -m flashalign.benchmark demo \ --repetition 0 --output /private/experiment/demo.html ``` -Open the HTML locally. It embeds the FlashAlign wordmark and replays actual -captured text deltas at their original times. It labels the sequential requests, -warmup count and metric conventions. It omits reasoning text and displays no -invented running token counter. The page works offline and does not fetch fonts, -scripts or analytics. Review the complete clip before sharing it. +Open the HTML locally. It embeds the FlashAlign wordmark and replays captured +text deltas at their original times, with labels for sequential requests, +warmups and metrics. Reasoning text is omitted. The page works offline without +external fonts, scripts or analytics. Review the complete clip before sharing it. Optional video capture requires Playwright and Chromium: @@ -247,8 +233,7 @@ ffmpeg -i /private/experiment/demo.webm \ /private/experiment/demo.gif ``` -The WebM records playback at 1×. Any sped-up edit should state its speed. -The HTML and video contain the selected prompt and visible answer, so sharing -them is a deliberate export of that reviewed pair. Raw run files stay private. -Use `demo --note "..."` to display a visible qualification or limitation beside -the recording. A selected successful pair must not hide failures elsewhere in a run. +The WebM records playback at 1×; label any speed changes. The HTML and video +include the selected prompt and visible answer. Keep raw run files private. +Use `demo --note "..."` for a visible qualification or limitation, and publish +aggregate results alongside selected clips. diff --git a/docs/compatibility.md b/docs/compatibility.md index 163b073..c6f55ef 100644 --- a/docs/compatibility.md +++ b/docs/compatibility.md @@ -4,8 +4,7 @@ FlashAlign is the product; Owner Mode is the profile-selection feature. The native plugin/provider ID is `owner-mode`, configuration lives under -`~/.config/owner-mode/`, and the engine protocol uses `owner_mode`. These stable -identifiers let installed adapters reuse their existing configuration. +`~/.config/owner-mode/`, and the engine protocol uses `owner_mode`. In Verified contacts mode, Aevonix FlashAlign uses the harness's native channel and run metadata. The adapter must identify an incoming contact and bind it to @@ -34,12 +33,8 @@ The OpenClaw integration targets the 2026.10.8 plugin interface. Its runtime requires Node 24.22.1+ on the 24.x line or 26.12.1+ on the 26.x line. Python 3.10+ is required for the shared core and setup wizard. -Adapter tests use synthetic native events. Hermes and OpenClaw hooks have also -been exercised through real HTTP proxies against Qwen and GLM Flash on TensorFold -engines. These are not full tests of every installed gateway or messaging -transport. Verify native ingress, -effective profile selection, revocation and delegation on your installed -harness before enabling it for routine use. +After setup, check message routing, profile selection, revocation and delegation +through your installed harness. See [validation](validation.md) for test coverage. Verified contacts mode requires OpenClaw to isolate direct messages with `per-account-channel-peer` or `per-channel-peer`. Setup preserves those scopes. @@ -52,14 +47,13 @@ not authorize Owner Mode. Verified contacts mode uses an installed adapter registry. WhatsApp and RCS direct messages are enabled by default. SMS and Telegram require explicit opt-in; Telegram uses native numeric user IDs. Each requires the harness's -trusted transport metadata; the package does not install a messaging transport. Group chats and synthetic -events use the standard model in this mode. +trusted transport metadata and an installed messaging transport. Group chats +and synthetic events use the standard model in this mode. Explicitly authorized local terminal sessions use a separate adapter with expiring, revocable grants. A CLI label, session name or prompt cannot create that authorization. See [channel and terminal setup](channels.md) for the -commands and extension interface. Telegram has synthetic native-hook contract -coverage; live delivery through your configured bot still needs a local check. +commands and extension interface. On supported messaging transports, the adapters handle text, voice notes, photos and other attachments, forwarded messages, quoted replies, and queued @@ -109,38 +103,33 @@ the signing key and load the correct profile for the exact model revision and serving format. The [starter pack](../profiles/README.md) includes five runtime profiles; additional profiles can be supplied by the operator. -The adapters use the same signing protocol for each engine. Qwen3.8-27B and -Gemma 4 26B A4B have paths on TensorFold, vLLM, SGLang and llama.cpp. Other models have the -specific paths listed in [validation](validation.md); engine support for a model -does not by itself establish FlashAlign support. +Qwen3.8-27B and Gemma 4 26B A4B have paths on TensorFold, vLLM, SGLang and +llama.cpp. The [starter pack](../profiles/README.md) lists each model's supported +engines; [validation](validation.md) records the test results. -Profiles describe operations on a model. The same artifact can be reused by -multiple adapters when each supports its pinned model and serving format, -operator and output sites. Compact -profiles also require a supported basis rank; native tensor profiles require -their declared storage format and parallel layout. -Each model/engine combination still needs its own runtime validation. +An engine can reuse a profile when it supports that profile's model revision, +serving format, operator and output sites. Compact profiles also require a +supported basis rank; native tensor profiles require their declared storage +format and parallel layout. For compact profiles, setup checks each backend's `maximum_rank` before installation. The current GLM Flash and Qwen Flash Next SGLang adapters accept rank one; the other packaged vLLM, SGLang and llama.cpp adapters accept ranks up -to 128. Site coverage and operator support still differ by adapter. A supported -rank does not establish that an arbitrary profile has been tested on that engine. +to 128. Site coverage and operator support differ by adapter. The llama.cpp integration uses upstream version 0.8.0 at commit `8642541b255bfb5dc39cd351a178be09013a55d6` and the declared Q8_0 GGUF checkpoints. Gemma setup defaults to ngram speculation and can preserve its -official MTP draft. Other GGUF files require a matching compatibility -declaration and separate validation. See the [llama.cpp guide](../engines/llamacpp/README.md). +official MTP draft. Other GGUF files need a matching compatibility declaration. +See the [llama.cpp guide](../engines/llamacpp/README.md). The four compact starters have TensorFold 1.1.9 paths, with model-specific builds and presets. The DeepSeek native starter has pinned vLLM and TensorFold -1.2.0 paths. Python handles setup and process supervision, not model inference. The engine names in the support table identify -available integrations. Each supplied bundle still requires its declared GPU -architecture and rank count; setup checks those requirements separately. -The bundles cover these tested checkpoint and hardware pairs: +1.2.0 paths. TensorFold bundles support Linux x86_64 or aarch64 and declare the +required GPU architecture and rank count. Setup checks those requirements. +The following hardware was used to test the bundled configurations: -| Model | Checkpoint format | Qualified hardware | +| Model | Checkpoint format | Tested hardware | | --- | --- | --- | | DeepSeek V4.1 Flash 1017, TensorFold 1.2.0 | Declared LibertAIDAI EXL3 3.5-bpw base | Four RTX PRO 6000 GPUs, Linux x86_64, SM120 | | Qwen3.8-27B | MLX-format 4-bit | One DGX Spark, Linux aarch64, SM121 | @@ -149,9 +138,8 @@ The bundles cover these tested checkpoint and hardware pairs: | GLM 5.3 Flash | NVIDIA NVFP4 | Four DGX Sparks, Linux aarch64, SM121 | | GLM 5.3 Full | EXL3 2.75-bpw | Four DGX Sparks, Linux aarch64, SM121 | -MLX-format here describes the quantized checkpoint files. TensorFold reads -those files and runs inference on NVIDIA GPUs under Linux; these results do -not use Apple's MLX runtime or establish macOS support. Use the +MLX-format describes the checkpoint files; TensorFold runs them on NVIDIA GPUs +under Linux. Use the [model download guide](model-downloads.md) to select the pinned files for your model and backend. @@ -159,37 +147,31 @@ The [pinned native vLLM backend](../engines/vllm-deepseek-native/README.md) for DeepSeek's `deepseek41-flash1017-v1` starter requires the official FP8 checkpoint at revision `1bae0e1088993b9b82de08f59de5be5a34e42170`, Linux x86_64 and four SM120 GPUs. -The backend pins its container image, preserves adaptive DSpark with seven draft -tokens, and uses FULL decode graphs with eager prefill. The native operator -selects 26 WO-B matrices and 26 scale tensors at strength 1.0. It does not use -a low-rank basis. Setup retrieves the 1.09 GB payload from a public release -asset and verifies its size and hashes; no private repository access is needed. -Fresh public-package GPU and wizard qualification for this vLLM backend is pending. See -[validation](validation.md) for the earlier native-operator evidence and its limits. +It uses adaptive DSpark with seven draft tokens, FULL decode graphs and eager +prefill. The native operator selects 26 WO-B matrices and 26 scale tensors at +strength 1.0. Setup retrieves and verifies the 1.09 GB profile payload from a +public release asset. TensorFold 1.2.0 uses the same native profile payload with the declared LibertAIDAI EXL3 base at revision -`04a492fc3b4d25ddbab8f2d2f07f235ec77c9010`, strength 1.0 and four ranks. Its -base quantization and arithmetic differ from vLLM; identical edited matrices -do not imply identical full-model outputs. - -Legacy DeepSeek output-projection profiles can still use their separately -qualified TensorFold or vLLM integration. The TensorFold EXL3 text preset has a -65,536-token context, a 131,072-token pool and automatic DSML tool calls; forced -tool selection, JSON Schema and image input are unsupported. Those integrations -cannot load the native starter; its TensorFold path requires the new 1.2.0 -native-matrix bundle. Model support alone does not add support -for a different profile operator. - -GLM Full on TensorFold was qualified with an 8,192-token context, two concurrent +`04a492fc3b4d25ddbab8f2d2f07f235ec77c9010`, strength 1.0 and four ranks on +SM120 GPUs. Its base quantization and arithmetic differ from the FP8 vLLM path. +Both current DeepSeek packages enforce this four-GPU layout. + +The DeepSeek TensorFold preset has a 65,536-token context and a 131,072-token +pool. JSON Schema and guided output are unsupported. Older output-projection +profiles use their matching TensorFold or vLLM packages; those packages cannot +load the native starter. The older TensorFold preset supports automatic DSML +tool calls; forced tool selection and image input are unsupported. + +GLM Full's TensorFold preset uses an 8,192-token context, two concurrent sequences and native MTP depth 4. It supports automatic tool calls. Forced tool selection, JSON Schema and image input are unsupported on this path. -Setup matches explicit model IDs and pinned checkpoint metadata. A profile's -presence in the starter pack does not qualify every engine or quantization. -Before changing harness routing, the wizard verifies both signed profile -activation and standard-mode requests. See [TensorFold setup](native-tensorfold.md) -for remote ranks, SSH forwarding, legacy Engram prerequisites and architecture checks. +Before changing harness routing, the wizard checks model and profile +compatibility and verifies signed profile and standard-mode requests. See +[TensorFold setup](native-tensorfold.md) for remote ranks, SSH forwarding and +DeepSeek Engram prerequisites. Transcription, file decoding and vision depend on the configured model and harness capabilities. Advertise image input only for a model that supports it. @@ -200,12 +182,5 @@ Always On also covers custom tools that send model requests through the supporte Chat Completions route. Responses API, Anthropic Messages and other model transports need an adapter to that route; they are outside the V1 API interface. -Local managed engines are qualified on Linux with NVIDIA GPUs. macOS engine -qualification is deferred. A remote client connection is not evidence of a -locally supported macOS inference engine. - -TensorFold setup supports Linux x86_64 and aarch64 bundle contracts. -GPU architecture and each model/operator combination require their own qualified -bundle; CPU architecture alone does not establish compatibility. See -[TensorFold](native-tensorfold.md) for multi-host preparation and current -limits. +Managed inference supports Linux with NVIDIA GPUs. macOS clients can connect to +a remote engine; local macOS inference is not supported. diff --git a/docs/custom-identity-ordering.md b/docs/custom-identity-ordering.md index 046ecda..5b743be 100644 --- a/docs/custom-identity-ordering.md +++ b/docs/custom-identity-ordering.md @@ -26,8 +26,7 @@ requires_plugins: `verified-identity-provider` is an example ID, not an included plugin. The host must support plugin dependencies, and that provider must be installed and enabled. Reapply and verify this deployment-specific overlay when updating -FlashAlign. Do not add a particular private provider to the shared public -manifest. +FlashAlign. A dependency changes load order. It does not prove a contact's identity or grant permission to produce owner claims. The adapter must still validate @@ -89,7 +88,7 @@ On when the request reaches its configured route. | `owner_unverified` | The gateway did not have a valid active proof for this turn. Check adapter loading, current native evidence and callback order. | | `no_middleware_context` | The request reached the proxy without its expected middleware context. Check the configured provider route and host request hook. | | `bad_chat_template_kwargs` | The authorized request contains unsupported template fields or values. Check the host's final serialization and the selected engine's compatibility declaration. | -| Matching requested and effective profile | The engine reports the selected profile active. This does not establish that every model refusal disappears or that a profile reproduces an edited checkpoint exactly. | +| Matching requested and effective profile | The engine reports the selected profile active. | If the authority cannot be constructed at startup, FlashAlign emits `FlashAlign identity authority unavailable` followed by the exception class. diff --git a/docs/engine-contract.md b/docs/engine-contract.md index 022eed0..1d14b1c 100644 --- a/docs/engine-contract.md +++ b/docs/engine-contract.md @@ -140,8 +140,7 @@ cache identity includes the profile content digest. See the [TensorFold setup](native-tensorfold.md) for the exact checkpoint and runtime limits. A TensorFold 1.2.0 bundle declares its compatible EXL3 serving base separately from the profile's original target model. It selects the same edited -FP8 tensors without converting the profile. Base quantization and arithmetic -differ between engines; full-model output identity is not implied. +FP8 tensors without converting the profile. ## Optional weighted output projection diff --git a/docs/engine-setup.md b/docs/engine-setup.md index a1a0e27..1811667 100644 --- a/docs/engine-setup.md +++ b/docs/engine-setup.md @@ -38,39 +38,29 @@ and profile. It checks declared checkpoint identities and formats before choosin a backend. GGUF files select llama.cpp. Choose another engine explicitly with `--backend`. -| Engine | Included integration | Configuration and validation | +| Engine | Included integration | Model and configuration | | --- | --- | --- | | vLLM | Pinned 0.32.0 package and FlashAlign plugin | Qwen3.8-27B BF16, one GPU, eager execution, ngram speculation | | SGLang | Pinned 0.5.20 source patch | Qwen3.8-27B BF16, one GPU, eager execution, DFlash2 | -| SGLang GLM 5.3 Flash | Pinned 0.5.20 normalized-output integration | Pinned FP8 checkpoint, four local SM120 GPUs, native EAGLE, BF16 KV; native protocol checks passed, image-answer quality limits remain | -| SGLang Qwen Flash Next | Pinned 0.5.20 writer-group integration | Matched FP8 checkpoint, two GPUs, native MTP, PLE tables in host RAM; experimental | -| vLLM Gemma 4 | Pinned 0.32.0 projection integration | Matched 26B A4B FP8 checkpoint, one GPU, ngram; experimental | -| SGLang Gemma 4 | [Pinned 0.5.20 projection integration](../engines/sglang-gemma4/README.md) | Same pinned 26B A4B FP8 checkpoint and 50-site profile, one GPU, NGRAM with three draft tokens; native protocol checks passed, exact-answer limit remains | -| vLLM DeepSeek V4.1 Flash | [Pinned native container and tensor operator](../engines/vllm-deepseek-native/README.md) | Official 1017 FP8 checkpoint, four SM120 GPUs, adaptive DSpark7, FULL decode graphs and eager prefill; public-package GPU and wizard qualification pending | +| SGLang | Pinned 0.5.20 normalized-output integration | GLM 5.3 Flash FP8, four local SM120 GPUs, native EAGLE, BF16 KV | +| SGLang | Pinned 0.5.20 writer-group integration | Qwen Flash Next FP8, two GPUs, native MTP, PLE tables in host RAM; requires an operator-supplied profile | +| vLLM | Pinned 0.32.0 projection integration | Gemma 4 26B A4B FP8, one GPU, ngram | +| SGLang | [Pinned 0.5.20 projection integration](../engines/sglang-gemma4/README.md) | Gemma 4 26B A4B FP8, one GPU, NGRAM with three draft tokens | +| vLLM | [Pinned native container and tensor operator](../engines/vllm-deepseek-native/README.md) | DeepSeek V4.1 Flash 1017 official FP8, four SM120 GPUs, adaptive DSpark7, FULL decode graphs and eager prefill | | llama.cpp | Upstream 0.8.0, commit `8642541`, source patch and profile converter | Qwen3.8-27B and Gemma 4 26B A4B Q8_0, one GPU; Qwen `draft-simple`, Gemma ngram or its official MTP draft | -| TensorFold | Compiled 1.1.9 and 1.2.0 bundles for Linux x86_64 and aarch64 | Exact model, GPU, operator and draft capabilities come from each qualified bundle; see [TensorFold setup](native-tensorfold.md) | - -The included integrations can be built from public engine dependencies; they -do not require access to a private engine repository. Each variant declares its -supported model family, operator and serving format. Gemma 4's 26B A4B starter -profile is included; engine and native harness protocol checks passed with -the answer-quality limits recorded in [validation](validation.md). Qwen Flash -Next requires an operator-supplied profile. Other combinations require their own -validation. - -Managed setup carries a model-specific thinking format into OpenClaw's model -descriptor when the validated engine package declares one. This preserves the -native thinking on/off control through normal requests, owner requests and -delegated work. An explicit `compat` object on the matching existing model takes -precedence. Packages without a declaration retain OpenClaw's existing behavior; -FlashAlign does not guess a format from the engine name. -When a validated model requires graded template controls, setup also preserves -the supported effort value in its template request. This translation is bound -to the selected model and happens after request authorization, before signing. +| TensorFold | Compiled 1.1.9 and 1.2.0 bundles for Linux x86_64 and aarch64 | Model, GPU, operator and draft requirements come from the selected bundle; see [TensorFold setup](native-tensorfold.md) | + +Each integration declares its supported model family, operator and serving +format. See [validation](validation.md) for runtime test results and known +model-quality limits. + +Setup carries the engine's declared thinking format and supported effort levels +into OpenClaw. An explicit `compat` object on the matching existing model takes +precedence. Packages without a declaration retain OpenClaw's existing behavior. The pinned GLM Full and Flash templates support low, high and max reasoning. They do not support disabling thinking. Setup rejects a saved OpenClaw off -setting for these models instead of presenting it as an effective model mode. +setting for these models. ```sh bash ./install.sh --engine-only --backend sglang \ @@ -94,22 +84,22 @@ GLM 5.3 Full and Flash, Qwen3.8-27B and Gemma 4 26B A4B compact starter profiles are included in the checkout. DeepSeek V4.1 Flash 1017 includes its native manifest and a pinned public download descriptor. Setup retrieves the 1.09 GB native payload when needed, verifies its size and SHA-256, and caches it -outside the checkout. No private repository access or GitHub login is needed. +outside the checkout. For offline setup, place the verified payload beside its manifest before preparation. See [the DeepSeek profile](../profiles/deepseek41-flash1017-v1/README.md) for the exact file and source terms. For TensorFold, managed setup can retrieve a matching runtime package from -FlashAlign's release assets over HTTPS. No GitHub account or login is needed. -The included catalog pins each asset's size, SHA-256 digest and complete file -inventory. Setup verifies those checks before staging and rechecks cached files +FlashAlign's public release assets over HTTPS. The catalog pins each asset's +size, SHA-256 digest and complete file inventory. Setup verifies those checks +before staging and rechecks cached files before reuse. These TensorFold downloads contain engine code, separately from the DeepSeek native profile asset. -For offline setup or a separately qualified package, use `--engine-bundle`. +For offline setup or a separate compatible package, use `--engine-bundle`. If no asset matches the model and platform, setup asks for a local package. TensorFold bundles target Linux x86_64 or aarch64 and declare their GPU -architectures. The catalog contains the qualified combinations listed in +architectures. The catalog contains the combinations listed in [compatibility](compatibility.md). Multi-host Spark preparation and activation use explicit owned rank directories and SSH host-key verification. See [TensorFold setup](native-tensorfold.md). @@ -121,15 +111,9 @@ bash ./install.sh --auto --model-path /models/my-model \ --profile /private/my-profile --engine-bundle /private/my-engine-bundle ``` -Each engine package declares the model families and profile transforms it supports. -TensorFold setup checks declared configuration, weight-index and quantization -fingerprints before creating deployment state. A match must come from one -complete declared checkpoint variant. -The installer accepts declared compiled TensorFold 1.1.9 and 1.2.0 bundles. -DeepSeek native-matrix profiles require a matching 1.2.0 bundle with an explicit -compatible EXL3 base; the profile's original official model pin is retained. Python handles setup -and process supervision, not inference. CPU architecture, CUDA targets and exact -source revision are checked independently of model and profile compatibility. +TensorFold setup checks CPU architecture, CUDA targets, source revision and the +checkpoint's configuration, weight-index and quantization fingerprints. DeepSeek +native-matrix profiles require a matching 1.2.0 bundle and its declared EXL3 base. TensorFold packages may declare `input_modalities`, such as `["text"]` or `["text", "image"]`. This describes the serving path, rather than the original @@ -145,8 +129,7 @@ declaration applies only to its listed model types; unlisted types remain unverified even if a general declaration is also present. Setup does not infer image support from an architecture name. -The starter profiles are experimental. See [starter profiles](../profiles/README.md) -for tested engine combinations and remaining model-quality limits. +The [starter profiles](../profiles/README.md) are experimental. ## What setup does @@ -188,7 +171,7 @@ matching development headers before activation. Setup preserves a matching active engine's speculative configuration. It rejects an unsupported method or a conflicting requested draft. New vLLM -Qwen deployments use the qualified ngram configuration. DeepSeek V4.1 Flash on +Qwen deployments use ngram speculation. DeepSeek V4.1 Flash on the native vLLM backend uses the stock model's adaptive DSpark with seven draft tokens. Qwen3.8-27B on SGLang needs a local DFlash2 model; Qwen Flash Next uses its own native MTP weights. llama.cpp @@ -212,18 +195,15 @@ configuration does not specify a context limit. Managed setup checks that minimum before installation. Direct engine setup has no Hermes requirement. An existing endpoint that omits context metadata is reported as unverified; setup never guesses its capacity from the model name. -For example, a Hermes `model.context_length` of 737,280 cannot be redirected to -a 65,536-token engine. Setup stops with the mismatch and preserves the existing -limits; it does not select a larger unqualified preset or silently shorten the -conversation budget. An existing explicit limit also requires a known engine -context. OpenClaw receives the selected engine context in its model descriptor. +Setup stops if the engine cannot meet the harness's existing limits. An explicit +limit requires a known engine context. OpenClaw receives the selected engine +context in its model descriptor. The Qwen3.8-27B llama.cpp default is 65,536 tokens per sequence, using 131,072 total context across two parallel slots. Its context and parallel overrides are checked together so setup reports the same per-sequence cap that is launched. -See the model's validation record for quality checks at specific context lengths. -A serving lane that has verified native handling of omitted output budgets can -declare an OpenClaw request policy in its engine metadata: +An engine that supports omitted output budgets can declare an OpenClaw request +policy in its metadata: ```json "openclaw_request_policy_by_model_type": { @@ -244,8 +224,8 @@ their prior behavior. `supports_thinking_off: false` rejects a saved off selection before setup and rejects a later explicit off request before native transport. It does not pick a reasoning level when the user omitted one or change the model's template. -Supported reasoning levels still come from the model's separate compatibility -declaration. Only declare this policy after checking the real engine behavior. +Supported reasoning levels come from the model's separate compatibility +declaration. These controls cover OpenClaw's regular provider stream. Its dedicated device worker-local inference path does not use that stream wrapper and forces a @@ -273,7 +253,7 @@ managed provider, while an unrelated provider with the same name is preserved. ## Prompt reuse -The qualified engines reuse matching conversation prefixes in memory. Owner +The included engines reuse matching conversation prefixes in memory. Owner entries are separated by loaded profile and strength; normal-mode entries remain separate. Owner entries are never written to the engine's disk cache. Reloading the model or profiles starts a new engine with an empty in-memory cache. @@ -281,22 +261,18 @@ the model or profiles starts a new engine with an empty in-memory cache. Prompt reuse consumes the engine's cache memory budget, independently of profile storage. The DeepSeek native profile also needs GPU storage for its alternate matrices and graph workspaces. Speculative decoding stays enabled. -TensorFold's cache is scoped by profile and strength. Its memory budget is -separate from profile storage. + +## Process supervision Managed activation requires Linux pidfd and child-subreaper support. Each -engine command runs under a dedicated supervisor that retains orphaned workers -when an intermediate parent exits, including compilers that create a new process -group or session. Setup records PID, user and process-start identities and follows -parent-child ancestry instead of relying only on process-group membership. +engine command runs under a supervisor that tracks its workers by PID, user and +process-start identity, including workers whose parent exits. If startup or verification fails, setup cleans up its verified workers and keeps -the existing harness configuration. A supervisor remains alive until the kernel -confirms its children have been reaped. If the supervisor terminates without that -completion receipt, or any ownership/cleanup check fails, shutdown preserves -`running.json` and reports incomplete cleanup. It does not stop unrelated -processes or overwrite an existing engine. Startup logs and supervisor receipts -stay in the private deployment directory. +the existing harness configuration. Incomplete cleanup preserves `running.json` +and is reported for review. Setup does not stop unrelated processes or overwrite +an existing engine. Logs and supervisor receipts stay in the private deployment +directory. ## Agent preparation and activation @@ -362,30 +338,23 @@ verified public asset. Its operator keeps the stock and edited WO-B bindings immutable and selects between them per request or row. Strength is fixed at 1.0; projection-only integrations cannot load it. -The native vLLM launcher preserves adaptive DSpark7 and the declared graph settings. -Its default context is the model's 1,048,576-token limit; that configuration is -not evidence of a completed million-token workload test. See the +The native vLLM launcher preserves adaptive DSpark7 and the declared graph +settings. Its default context is 1,048,576 tokens. See the [DeepSeek integration](../engines/vllm-deepseek-native/README.md) for resource -checks, supported arguments and qualification status. Its Hermes setup uses a -model-specific supported reasoning default and preserves disabled thinking -through the DeepSeek transport's `enable_thinking=false` option. It does not -change reasoning settings for other models. +requirements and arguments, and [validation](validation.md) for tested workloads. +Its Hermes setup preserves disabled thinking through DeepSeek's +`enable_thinking=false` option. -An operator with an existing external DeepSeek output-projection profile can -still select the [legacy vLLM integration](../engines/vllm-deepseek/README.md) -or a matching TensorFold bundle. Those paths and their earlier qualification -records do not apply to the native starter. +Existing DeepSeek output-projection profiles use the +[legacy vLLM integration](../engines/vllm-deepseek/README.md) or a matching +TensorFold bundle. These integrations cannot load the native starter. An existing prepared installation can be reused without installing packages: set `engine.python` to the compatible environment and leave `install_dependencies` false. SGLang also needs a clean pinned `source` checkout; llama.cpp needs the path to its prepared `installation.json` in `engine.installation`. -The legacy DeepSeek vLLM projection integration additionally checks its -preinstalled plugin and FlashInfer 0.7.1rc5. Its operator-supplied overlay can be -selected with `engine.environment.PYTHONPATH`; otherwise installation builds -the owned overlay. The native starter instead requires its pinned container -and checked native overlay; an arbitrary Python environment is not a substitute. +The native DeepSeek starter requires its pinned container and overlay. The result includes the activation command. When starting the engine and restarting the gateways is authorized: @@ -404,6 +373,4 @@ bash ./install.sh --stop-engine /private/deployment.json ``` The stop operation verifies recorded process identities and only signals -workers created by this installer. Engine activation and file checks do not -establish model quality or numerical parity; retain the package's validation -record when deciding which model/profile combination to deploy. +workers created by this installer. diff --git a/docs/model-downloads.md b/docs/model-downloads.md index 73923a4..7181f42 100644 --- a/docs/model-downloads.md +++ b/docs/model-downloads.md @@ -4,9 +4,8 @@ FlashAlign includes four compact starter profiles and the DeepSeek native profile manifest. Setup fetches the DeepSeek profile's 1.09 GB payload from a pinned public release asset. Base model weights and draft models are separate downloads. Choose the checkpoint for your engine and hardware -before running the wizard; a profile does not make other quantizations or model -revisions compatible. The [compatibility table](compatibility.md#engines-and-models) -lists the qualified combinations and limits. +before running the wizard. The [compatibility table](compatibility.md#engines-and-models) +lists supported combinations and their requirements. The commands below use the official [Hugging Face CLI](https://huggingface.co/docs/huggingface_hub/guides/cli). If `hf` is not installed, install `huggingface_hub` in your Python environment: @@ -36,23 +35,23 @@ Multi-node model files must be accessible at the configured model path on every rank. See [TensorFold setup](native-tensorfold.md) for rank configuration and activation. -| Starter model | Download below | Qualified hardware | +| Starter model | Download below | Tested hardware | | --- | --- | --- | +| [DeepSeek V4.1 Flash 1017](#deepseek-v41-flash-1017-on-tensorfold-120) | EXL3 3.5-bpw plus original Engram tables | Four RTX PRO 6000 GPUs, Linux x86_64 | | [GLM 5.3 Full](#glm-53-full) | EXL3 2.75-bpw | Four DGX Sparks, Linux aarch64 | | [GLM 5.3 Flash](#glm-53-flash) | NVIDIA NVFP4 | Four DGX Sparks, Linux aarch64 | | [Qwen3.8-27B](#qwen38-27b) | EXL3 2.0-bpw or MLX-format 4-bit, plus DFlash2 | One RTX PRO 6000 or one DGX Spark, respectively | | [Gemma 4 26B A4B](#gemma-4-26b-a4b) | MLX-format 4-bit plus DFlash | One DGX Spark, Linux aarch64 | -MLX-format means the stored quantization format. TensorFold performs inference -on Linux/NVIDIA; these commands do not install or require Apple's MLX runtime. +The selected bundle enforces its declared GPU architecture and rank count. +MLX-format describes the checkpoint files; TensorFold runs them on NVIDIA GPUs +under Linux. ### DeepSeek V4.1 Flash 1017 on TensorFold 1.2.0 The `deepseek41-flash1017-v1` starter uses the same unchanged native payload on TensorFold 1.2.0 and [native vLLM](#deepseek-v41-flash-1017-native-vllm). -TensorFold requires the following explicitly supported EXL3 base; vLLM uses -the official FP8 checkpoint. Base quantization and arithmetic differ, so the -shared profile does not imply identical full-model outputs. +TensorFold requires the following EXL3 base; vLLM uses the official FP8 checkpoint. Download the [EXL3 checkpoint](https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-3.5bpw/tree/04a492fc3b4d25ddbab8f2d2f07f235ec77c9010): @@ -85,9 +84,8 @@ bash ./install.sh --auto --backend tensorfold \ ``` The wizard requires a TensorFold 1.2.0 bundle declaring this native operator -and exact EXL3 base. The native profile remains fixed at strength 1.0 and four -ranks. Its 1.09 GB payload is fetched separately from the runtime bundle. -Older TensorFold bundles still require a compatible output-projection profile. +and exact EXL3 base. This bundle requires four SM120 GPUs and strength 1.0. +Its 1.09 GB profile payload is fetched separately from the runtime bundle. ### GLM 5.3 Full @@ -224,7 +222,6 @@ profile payload, and stops before activation with `--no-restart`. No edited full checkpoint or separate draft download is needed. For offline preparation, place the verified profile payload beside its manifest first. See the [profile instructions](../profiles/deepseek41-flash1017-v1/README.md). -Fresh public-package GPU and wizard qualification is pending. ### Other vLLM and SGLang models @@ -270,7 +267,7 @@ hf download ggml-org/Qwen3.8-27B-GGUF Qwen3.8-27B-Q8_0.gguf \ --local-dir "$FLASHALIGN_MODELS/qwen38-gguf" ``` -The qualified Qwen `draft-simple` path also uses this smaller external draft +The Qwen `draft-simple` path also uses this smaller external draft model. Download the pinned [Qwen3.5-0.8B Q4_0 GGUF](https://huggingface.co/ggml-org/Qwen3.5-0.8B-GGUF/tree/807dfd9a6fadbf703207aac3bfb0b2b3040d077c): ```sh diff --git a/docs/native-tensorfold.md b/docs/native-tensorfold.md index 290b659..d35dc80 100644 --- a/docs/native-tensorfold.md +++ b/docs/native-tensorfold.md @@ -1,14 +1,11 @@ # TensorFold setup -FlashAlign manages declared TensorFold 1.1.9 and 1.2.0 bundles on Linux x86_64 -and Linux aarch64. -A runtime bundle must match both the CPU architecture and the GPU architecture. -A Blackwell-only bundle cannot run on an Ampere GPU. The NVIDIA driver remains -a host prerequisite and is never included in a bundle. +FlashAlign manages TensorFold 1.1.9 and 1.2.0 bundles on Linux x86_64 and +aarch64. A bundle must match the host's CPU and GPU architectures. Install the +NVIDIA driver before setup. Setup stages the checked native executable, runtime libraries, matching source -archive, local profile, and a unique deployment signing key. Other engines have -their own installation paths. +archive, profile and unique deployment signing key. The download catalog lists the available runtime bundles and their complete checksums. When no matching bundle is listed, provide its local directory with @@ -18,9 +15,11 @@ profile directory. The DeepSeek starter runs on the [pinned native vLLM backend](../engines/vllm-deepseek-native/README.md) or a matching TensorFold 1.2.0 native-matrix bundle. TensorFold uses the pinned LibertAIDAI EXL3 serving base and the same unchanged native profile payload. -Strength is fixed at 1.0 and tensor parallelism at four ranks. The bundle must -explicitly declare this operator and compatible base; older projection bundles -cannot load it. See [model downloads](model-downloads.md#deepseek-v41-flash-1017-on-tensorfold-120). +Strength is fixed at 1.0 and tensor parallelism at four SM120 GPUs. The bundle +must declare this operator and compatible base; older projection bundles +cannot load it. It supports chat and completion requests; `response_format` +and guided output are unsupported. See +[model downloads](model-downloads.md#deepseek-v41-flash-1017-on-tensorfold-120). ## One host @@ -82,14 +81,19 @@ stages profiles and required files through encrypted SSH input into private rank directories. Only commands that need a signing key receive one. Keys never appear in arguments or setup output. The leader authenticates API requests; every rank loads the same profile. -The native Flash environment interface also receives the key-file path during -startup, as required by its checked command contract. Preparation does not +GLM Flash also receives the key-file path during startup. Preparation does not start a model or restart a harness. -Activation starts followers, then the leader and its SSH forward. Native cold starts allow 30 minutes by default; a bundle can declare a checked `startup_timeout` from 60 to 7200 seconds. The DeepSeek 1.2.0 bundle allows 15 minutes. An integer `engine.startup_timeout` in the setup plan overrides the bundle value; DeepSeek ranks use the same allowance to connect to their peers. Large checkpoints loaded over a shared mount can take several minutes. Each rank has an owned process -supervisor and a private receipt. Setup verifies standard and signed owner -requests before changing harness routing. If activation fails, it stops only -those recorded ranks. An occupied service is never stopped to make room. +Activation starts followers, then the leader and its SSH forward. Startup +allows 30 minutes by default; bundles can set `startup_timeout` from 60 to +7,200 seconds. The DeepSeek 1.2.0 bundle allows 15 minutes. Set +`engine.startup_timeout` to an integer in the plan to override this value. +DeepSeek ranks use the same allowance to connect to their peers. + +Each rank has a process supervisor and a private receipt. Setup verifies +standard and signed profile requests before changing harness routing. If +activation fails, it stops only those recorded ranks. An occupied service is +never stopped to make room. For remote hosts, setup keeps an owned SSH forward running on the controller and configures clients with its localhost URL. The forward uses the same verified SSH @@ -101,10 +105,9 @@ every host; configuring drivers, SSH access and the cluster network remains an operator task. A host entry may set `state_root` to a writable, persistent private directory -when its home partition has insufficient space. Setup never deletes unrelated -files to make room. A temporary filesystem is suitable for a disposable test, -but its staged runtime and profiles disappear after reboot and need preparation -again. Model files may be on a shared read-only mount. +when its home partition has insufficient space. Runtime files on a temporary +filesystem need preparation again after reboot. Model files may be on a shared +read-only mount. Native replay protection persists outside the versioned runtime directory. Setup selects a stable private directory for the deployment key and passes it @@ -117,30 +120,23 @@ before resetting state or moving a deployment key. The bundle declares the supported speculative method. Setup preserves a matching active external drafter, MTP depth and parallel setting. An incompatible active configuration stops setup; the installer does not disable speculation. Native -GLM uses its MTP weights. Each bundle declares whether its MTP depth is -adjustable. Flash keeps its native -maximum of three drafts; Full defaults to four. The qualified Full bundle uses +GLM uses its MTP weights. Each bundle declares whether MTP depth is adjustable. +Flash supports up to three drafts; Full defaults to four. The Full preset uses an 8,192-token context, two concurrent sequences and an 8 GiB host prompt cache. -Its live qualification used MTP depth 4; other depths need workload validation. -Full supports automatic tool calls, while forced tool selection, JSON Schema -and image input are unsupported. Setup omits unsupported tuning flags and -rejects a different requested fixed depth before staging. Qwen may require an -external DFlash checkpoint. The -legacy DeepSeek output-projection path uses its dedicated native launcher and -declared DSpark assets. - -## Legacy DeepSeek projection setup - -This section applies only to a matching external output-projection profile. The -current `deepseek41-flash1017-v1` starter is not compatible with this bundle. - -The legacy TensorFold path needs its original Engram tables on every rank, -separately from the EXL3 model. Setup detects the pinned original checkpoint in the Hugging Face -cache, or accepts `--engram-source /models/deepseek-engram` (`engram_source` in a -plan). It checks the original config and index hashes, required tensor encodings -and shapes, and complete shard sizes before creating a key or staging files. -A prepared per-rank table directory can instead use `--engram-dir` (`engram_dir` -in a plan); its table headers and sizes must match the model and rank geometry. +It supports automatic tool calls; forced tool selection, JSON Schema and image +input are unsupported. Setup rejects unsupported speculative settings before +staging. Qwen may require an external DFlash checkpoint. DeepSeek uses its +declared DSpark assets. See [validation](validation.md) for runtime test results. + +## DeepSeek Engram tables + +DeepSeek needs its original Engram tables on every rank, separately from the +EXL3 model. Setup detects the pinned original checkpoint in the Hugging Face +cache, or accepts `--engram-source /models/deepseek-engram` (`engram_source` in +a plan). It checks config and index hashes, tensor encodings and shapes, and +complete shard sizes before staging files. A prepared per-rank table directory +can instead use `--engram-dir` (`engram_dir` in a plan); its headers and sizes +must match the model and rank geometry. Only the original config, index, and two Engram shards are needed: @@ -153,16 +149,16 @@ hf download deepseek-ai/DeepSeek-V4.1-Flash-1017 \ ``` These two shards total about 203 GB. They remain on the model host or its shared -read-only storage; they are not copied into the runtime bundle. Header and size -checks establish compatibility, not a new full-payload integrity check. Use the -pinned download's integrity record for that. +read-only storage; they are not copied into the runtime bundle. Retain the pinned +download's payload-integrity record; setup checks headers and sizes. + +### Legacy projection bundles -The legacy DeepSeek projection preset explicitly fixes context to 65536, pool size to 131072, -one live slot and four ranks. Its declared generated assets are checked against -that geometry. Larger settings need a separately built and qualified asset pack. Its generated assets -must match the bundle's context and rank geometry. Changing those settings -requires newly qualified assets; the wizard does not substitute a smaller -context or disable drafts. +Older TensorFold bundles require a matching output-projection profile and +cannot load the current native starter. Their DeepSeek preset fixes the context +at 65,536 tokens, pool at 131,072 tokens, one live slot and four ranks. Changing +that geometry requires matching generated assets; the wizard preserves the +declared context and speculative settings. ## Bundle contract @@ -179,24 +175,14 @@ Native-matrix bundles require TensorFold 1.2.0 and explicitly declare `profile_schemas`, `supported_profile_requires`, `supported_profile_strengths`, `profile_target_model`, and `compatible_profile_bases`. The profile keeps its original target pin; compatible serving bases are declared by the runtime. -For DeepSeek, TensorFold's EXL3 base and arithmetic differ from the official FP8 -vLLM path. The identical edited tensor payload does not establish full-model -output identity across those paths. `binary` names the primary executable. `rank_transport` is `cli`, `glm_env`, or -`dsv41_env`, according to the tested native family interface. DeepSeek additionally -names its `follower_binary` and `assets_directory`. No Python compatibility path -is accepted. +`dsv41_env`, according to the native family interface. DeepSeek additionally +names its `follower_binary` and `assets_directory`. Maintainers build a reproducible runtime archive with [build-native-bundle.py](../tools/build-native-bundle.py). Its source archive comes from the exact Git commit. `--release-tag` together with `--catalog-entry` writes a -complete download entry for review without uploading or publishing anything. -Add that entry to the catalog only after its runtime gate passes. Every bundled runtime library needs its license or -notice. Source required for the native engine is distributed with the bundle; -the private profile-creation tools are separate. - -The DeepSeek 1.2.0 bundle supports chat and completion requests. Its current -engine does not support `response_format` or guided output, in either mode. -Auxiliary runtime kernel sources and their license notices are included in the -bundle and listed in its manifest. +complete download entry. Include each bundled library's license or notice and +any required source. Auxiliary runtime kernel sources and notices appear in +the manifest alongside the matching engine source. diff --git a/docs/profile-inspection.md b/docs/profile-inspection.md index 136f1bb..8994783 100644 --- a/docs/profile-inspection.md +++ b/docs/profile-inspection.md @@ -37,11 +37,8 @@ validation uses the bounded streaming reader and declared payload, tensor and partition hashes. If a file changes later, its new contents require a new inspection. -An artifact can pass inspection while being unsuitable for a particular model, -engine or task. Inspection does not establish runtime activation, cache isolation, -model quality, refusal behavior or the absence of unwanted semantic behavior. -Those need independent checks. A fixed linear operator can affect different -inputs differently even though its file contains no conditional branches. +Inspection validates the artifact. Use engine activation checks and model +evaluations to assess runtime behavior. The command exits with status `0` for a valid artifact, `2` for invalid or unsupported files, and `3` when the configured work budget stops inspection. diff --git a/docs/qwen.md b/docs/qwen.md index ffbc6ff..b6fff30 100644 --- a/docs/qwen.md +++ b/docs/qwen.md @@ -1,8 +1,7 @@ # Qwen3.8-27B FlashAlign includes the experimental `qwen38-27b-v1` profile. -Select it locally with the wizard or by name in an agent setup plan. No profile -download or private-repository access is required. +Select it locally with the wizard or by name in an agent setup plan. For TensorFold, the wizard selects a pinned 1.1.9 runtime bundle for the model format and hardware. The catalog includes these tested configurations: @@ -32,19 +31,9 @@ owner execution are not supported by these bundles. Setup stops if the active speculative configuration is incompatible, preserving that configuration. -The TensorFold serving path is text-only. The DGX Spark configuration passed -wizard activation, request authorization, cache, harness-hook and structured -output checks. A 48,037-token prompt reused 45,056 tokens separately in stock -and owner mode. The x86_64 EXL3 configuration passed 26 device checks covering -generation, authorization, concurrency and structured output. Both kept -DFlash2 active. These results apply to the recorded configurations; they do -not establish every context size or broad model quality. See -[validation](validation.md) and [TensorFold setup](native-tensorfold.md). - -Matrix checks used 256 fresh inputs at each of the 129 runtime sites. Maximum -relative output error against the selected edited checkpoint was 0.196%. -These measurements do not establish model quality or full edited-checkpoint -equivalence. The export excludes that checkpoint's native MTP changes. +The TensorFold serving path is text only. See [validation](validation.md) for +runtime and matrix results, and [TensorFold setup](native-tensorfold.md) for +installation requirements. For agent setup, use `--config /private/setup.json --apply` to prepare files without starting the engine or restarting a gateway. See diff --git a/docs/setup.md b/docs/setup.md index 6e1f111..059c161 100644 --- a/docs/setup.md +++ b/docs/setup.md @@ -93,9 +93,9 @@ FlashAlign never retries it in stock mode. ## Find profiles -The [starter profiles](../profiles/README.md) ship in `profiles/` and are discovered -locally, without a download or private repository login. The pack records their -validation limits. +Setup discovers the [starter profiles](../profiles/README.md) in `profiles/`. +Four payloads ship in the checkout; DeepSeek's native payload is fetched from +the public release and verified before use. To add your own profile folders: ```sh @@ -108,12 +108,13 @@ folders, or a `catalog.json` with `models` entries containing `path` and optiona `model_ids` aliases. The wizard also searches `~/.config/owner-mode/profiles/`, the checkout's `profiles/` folder, and paths in `FLASHALIGN_PROFILE_PATH`. -Local checks cover the model revision, transform requirements, tensor names, -FP32 shapes, finite values, positive gain/norm vectors, orthonormal bases and -supplied file checksums. Invalid profiles are excluded. Matching uses explicit -model IDs or catalog aliases, with revision/format conflicts rejected when the -engine supplies that metadata. The validator accepts ranks up to 128 and tensor -files up to 256 MiB. +Local checks cover the model revision, transform requirements, tensor names and +supplied checksums. Compact profiles also require valid FP32 shapes, finite +values, positive gain/norm vectors and orthonormal bases, with ranks up to 128 +and tensor files up to 256 MiB. Native DeepSeek payloads use a separate streaming +validator. Invalid profiles are excluded. Matching uses explicit model IDs or +catalog aliases; revision and format conflicts are rejected when the engine +supplies that metadata. If the engine does not provide a [profile inventory](engine-contract.md#optional-profile-discovery), enter an installed profile name directly. Model names are not limited to the diff --git a/docs/validation.md b/docs/validation.md index 2db1df9..a0ce82f 100644 --- a/docs/validation.md +++ b/docs/validation.md @@ -1,8 +1,8 @@ # Release validation -The five starter profiles remain experimental. Model execution, request -authorization and answer quality are tested separately. Loading a profile does -not establish full edited-checkpoint equivalence. +The five starter profiles are experimental. This page records runtime, +authorization and answer-quality checks for each tested configuration. Full +edited-checkpoint equivalence and broad refusal-rate benchmarks are not established. ## Model and engine coverage @@ -11,7 +11,7 @@ not establish full edited-checkpoint equivalence. | DeepSeek V4.1 Flash 1017 | Native vLLM, pinned image libraries, official FP8, four SM120 GPUs | Eight public-package protocol/cache checks passed. All 104 layer/rank bindings matched, and mixed owner/off FULL graph execution was verified on all four workers. Docker CPU checks passed separately; full Docker GPU/wizard activation was not tested. | | DeepSeek V4.1 Flash 1017 | TensorFold 1.2.0, declared EXL3 3.5-bpw base, four SM120 GPUs | 780 operator comparisons and 15 HTTP checks passed. Both modes retained DSpark and separate RAM session reuse; stream cancellation released slots. The actual bundle passed four-rank wizard preparation and full-size profile staging. JSON Schema and guided output are unsupported in both modes. See the [qualification details](#deepseek-native-profile-on-tensorfold-120). | | GLM 5.3 Full | TensorFold 1.1.9, EXL3 2.75-bpw, four DGX Sparks | Four-host wizard activation and 18 model cases passed, including deterministic arithmetic, concurrent requests, cache reuse and automatic tool calls. Invalid and replayed signatures were rejected. Configured context 8,192, two concurrent sequences, native MTP depth 4. A 2,423-token prompt reused 2,304 tokens independently in stock and owner mode; returning to stock retained its own cache and output hash. Forced tools, JSON Schema and images are unsupported. | -| GLM 5.3 Flash | SGLang 0.5.20, four SM120 GPUs | All 60 target/MTP sites checked. Both modes completed 59,042-token inputs. Eight Verified/Always On Hermes and OpenClaw protocol phases passed; image-answer limits remain. | +| GLM 5.3 Flash | SGLang 0.5.20, four SM120 GPUs | All 60 target/MTP sites checked with EAGLE and a 65,536-token context. Both modes completed 59,042-token inputs. Eight Verified/Always On Hermes and OpenClaw protocol phases passed; image-answer limits remain. | | GLM 5.3 Flash | TensorFold 1.1.9, NVIDIA NVFP4, four DGX Sparks | Four-host wizard activation, 23 runtime/auth/cache/concurrency/harness-hook cases, and four JSON Schema/required-tool cases passed. Context 8,192, two concurrent sequences, fixed native MTP active. | | Qwen3.8-27B | vLLM, SGLang and llama.cpp | Direct requests, cache isolation, native harness routing and approximately 65K-token inputs exercised with the documented checkpoint and speculative method. | | Qwen3.8-27B | TensorFold 1.1.9, MLX-format 4-bit, one Linux/NVIDIA DGX Spark | Wizard activation, 23 short runtime/auth/cache/concurrency/harness-hook cases and four grammar cases passed with DFlash2. A 48,037-token prompt reused 45,056 tokens separately in stock and owner namespaces. | @@ -22,24 +22,25 @@ not establish full edited-checkpoint equivalence. | Gemma 4 26B A4B | llama.cpp 0.8.0, Q8_0, one SM120 GPU | 32 direct checks passed with MTP and 32 with ngram. Authentication, replay, concurrent requests, streaming and separate stock/profile/strength caches checked. Both methods completed four 63,929-token requests; warm requests reused 63,928 tokens in their own namespaces. Native agent answer-quality limits are described below. | | Qwen3.8-27B | llama.cpp 0.8.0, Q8_0, one SM120 GPU | 32 direct checks passed with an external draft. Four 63,927-token requests passed; warm stock and profile requests each reused 63,923 tokens. | -These results apply to the listed checkpoint formats, hardware and settings. +Hardware and settings in this table identify the tested deployments. See +[compatibility](compatibility.md) for current engine requirements. The llama.cpp rows use upstream commit `8642541b255bfb5dc39cd351a178be09013a55d6`. TensorFold bundles check CPU and GPU architecture separately. The four-host GLM Flash run used a shared read-only checkpoint. -The final GLM Full TensorFold run used source +The GLM Full TensorFold run used source `933d3d766320d9460df45a819ea661755a7d9a0a` and checkpoint `Mia-AiLab/GLM-5.3-EXL3-2.75bpw-TensorFold` at `cf1fee8923b770b0c47f7a032e1b0f30a57c626a`. Stock arithmetic matched the unmodified TensorFold baseline. MTP recorded 43 drafted tokens on ordinary stock -requests and 45 on valid owner requests; invalid-signature fallback calls were -checked separately. This run checked concurrent requests; it does not -establish fused mixed-batch execution, broad answer quality or throughput. +requests and 45 on valid owner requests; invalid-signature fallback calls also +passed. Fused mixed-batch execution, broad answer quality and throughput were +outside this run's scope. The Qwen and GLM Flash TensorFold harness checks used the real Hermes Runtime and OpenClaw adapter hooks with synthetic native events and real model HTTP -requests. They do not certify every gateway version or messaging transport. +requests. Messaging transports and other gateway versions require their own checks. Qwen's 8 GiB host cache could not retain three separate 48K-token snapshots at once. Eviction caused a fresh prefill, and stock output returned to the same @@ -47,16 +48,36 @@ token hash. A separate 9,637-token test retained all three mode/strength namespaces and reused 8,192 tokens in each. At strength 2, the 48K arithmetic fixture ended immediately without answer text; strength 1 remains the default. +## Qwen profile and runtime checks + +The Qwen3.8-27B profile was checked with 256 fresh inputs at each of its 129 +sites. Maximum relative output error against the selected edited checkpoint +was 0.196%. The export excludes that checkpoint's native MTP changes. + +| Runtime | Recorded checks | +| --- | --- | +| vLLM, BF16, ngram, one RTX PRO 6000 | 18 CPU tests and 15 live protocol/cache checks passed. All 16 arithmetic/Python-output fixtures passed in each mode without truncation. Serial requests on a 96 GB GPU processed 65,470 input tokens in both modes and exercised native Hermes/OpenClaw tools and child inheritance. Four simultaneous maximum-length requests were not tested. | +| SGLang 0.5.20, BF16, DFlash2, one 95 GiB RTX PRO 6000 Blackwell Max-Q | 23 CPU tests and 47 live checks passed. All 43 generated-answer checks passed, including 16 shared math/code fixtures per mode and two streaming requests. Every generation reported active speculative verification. Repeated requests reused 576 tokens in their own mode; the first owner request and a strength change reused zero. Serial 65,000-input-token requests and managed wizard/native harness checks passed with a 65,536-token context. | + +The SGLang long-input run reused pinned dependencies. +[Timing samples](benchmark-results.md#qwen-sglang-startup-and-ttft) are recorded +separately. + +The additional Qwen Flash Next SGLang integration passed managed wizard, +native Hermes/OpenClaw and serial 65,000-input-token checks in both modes on +two 95 GiB RTX PRO 6000 Blackwell Max-Q GPUs. Its configured context was 65,536 +tokens; the full operator checks used 2,048 tokens. The context run reused +pinned dependencies. + ## DeepSeek native profile -Version 1.1.0 replaces the compact DeepSeek starter with a native tensor profile -at the same starter ID. It contains the exact 26 WO-B matrices and 26 scale -tensors from the pinned Keys source. It supports strength 1.0 only. The earlier -85-site projection integration and TensorFold EXL3 tests used a different -artifact; their results do not qualify this native starter. External projection -profiles can still use those integrations. +The native starter introduced in 1.1.0 contains the exact 26 WO-B matrices and +26 scale tensors from the pinned Keys source, at strength 1.0 only. It retains +the `deepseek41-flash1017-v1` ID. Results for the previous 85-site projection +profile, including its TensorFold EXL3 runs, apply only to that previous artifact. +External projection profiles can still use those integrations. -The native operator was checked on the pinned engine before public packaging: +Native-operator checks on the pinned engine: - Source-byte validation covered all 104 layer/rank bindings and 208 tensor partitions across four workers. @@ -71,9 +92,8 @@ The native operator was checked on the pinned engine before public packaging: completion, with maximum reasoning. Profile and control final-content hashes matched in six cases; reasoning hashes matched in one. -Those are prior native-operator results. A separate public-package check on -1 November 2026 passed all eight protocol/cache requests using the pinned -image's libraries on four SM120 GPUs: +The 1 November 2026 public-package check passed all eight protocol/cache +requests using the pinned image's libraries on four SM120 GPUs: - Signed requests selected the native profile; unsigned and replayed requests used stock mode. @@ -84,27 +104,37 @@ image's libraries on four SM120 GPUs: - Owner requests recorded 59 drafted tokens and 30 accepted tokens with adaptive DSpark7 active. -The public runtime reported a configured 1,048,576-token context and 7,809,386 -KV-token capacity. This was not a million-token input test or a throughput -benchmark. See the [aggregate record](../benchmarks/results/deepseek-native-qualification-20261101.json). +The runtime reported a configured 1,048,576-token context and 7,809,386 KV-token +capacity. Million-token input and throughput were not tested. See the +[aggregate record](../benchmarks/results/deepseek-native-qualification-20261101.json). The GPU run used a host process with the pinned image's libraries. Separate checks in the actual Docker image validated the full profile, signing-key ownership, plugin discovery and read-only Hugging Face snapshot mounts. -Full GPU activation through Docker and the wizard, a fresh GPU dependency -installation, and image requests were not tested in this run. No behavioral -benchmark was repeated. - -These checks do not establish all-layer reduced-output or full-logit identity, -a refusal rate or broad quality parity. Retaining exact source tensors avoids -a low-rank fitting approximation; it does not guarantee identical full-model -answers. +GPU activation through Docker and the wizard, a fresh GPU dependency +installation, image requests and behavioral benchmarks were outside this run's +scope. Exact source tensors avoid low-rank fitting error; all-layer output, +logit and full-model answer identity remain unverified. The pinned runtime uses adaptive DSpark with seven draft tokens. Its requested `FULL_AND_PIECEWISE` setting resolves to FULL decode graphs with eager prefill on this model. PIECEWISE execution is not supported by this path. Speculative decoding remained active in stock and native-profile tests. +## Previous DeepSeek projection profile + +The 85-site profile completed full-model wizard activation and shutdown on +four approximately 95 GiB SM120 GPUs with TP4/PP1/DP1, eager execution, native +DSpark5 and Engram CPU offload. Verified and Always On Hermes checks passed; +all 14 isolated OpenClaw cases passed protocol and strict-answer checks, +including images, both tool-inheritance settings and same-session CLI access. + +The lower-memory test override was enabled. Host `MemAvailable` was 367.65 GiB +at startup, with approximately 202 GiB steady host usage; initial cgroup +headroom was not recorded. These observations are separate from the +[launcher's memory policy](../engines/vllm-deepseek/README.md#requirements-and-defaults) +and from the current native starter's requirements. + ## Answer quality GLM Flash passed 16 of 16 common text fixtures in stock mode, with the runtime @@ -114,37 +144,34 @@ authorization and profile propagation checks passed. Gemma's vLLM harness run used one GB10, a 65,536-token context, one concurrent sequence, 0.60 GPU memory utilization and ngram with three draft tokens. It -completed 38 new model requests and recorded 11,676 draft tokens. The earlier -standard/profile 59,045-token retrieval passes remain separate evidence. This -run did not test the launcher's two-sequence, 0.85-memory defaults on GB10. - -Gemma's vLLM routing checks passed independently of its answer quality. Always On -Hermes matched three of four text answers and one of two parent answers; both -child answers matched. Some OpenClaw child and unknown-contact answers missed -their exact targets. Two Hermes image cases called a vision tool unexpectedly; -one captionless image reached the test's 4,096-token reasoning budget before -native recovery. Hash-only records cannot establish whether the color answer -was semantically wrong. Earlier failures remain recorded. - -Gemma's separate [SGLang integration](../engines/sglang-gemma4/README.md) uses +completed 38 model requests and recorded 11,676 draft tokens. Separate +stock/profile retrieval checks passed at 59,045 tokens; another run on a 96 GB +RTX PRO 6000 passed at 59,987 tokens in both modes. The launcher's +two-sequence, 0.85-memory defaults were not tested on GB10. + +Gemma's vLLM routing checks passed. Always On Hermes matched three of four text +answers and one of two parent answers; both child answers matched. Some +OpenClaw child and unknown-contact answers missed their exact targets. Two +Hermes image cases called a vision tool unexpectedly; one captionless image +reached the test's 4,096-token reasoning budget before native recovery. Only +hashes were retained for the color answer, so semantic accuracy is unknown. + +Gemma's [SGLang integration](../engines/sglang-gemma4/README.md) uses source revision `94602c9c2b7cbdb8efd5c52802dac6a1c180089e` and the unchanged 50-site profile. Its 16 direct requests checked stock/profile isolation, authentication, replay rejection, cache separation by profile strength and -active NGRAM decoding. Arithmetic, factual and JSON checks passed in both +active NGRAM decoding. The tested cache strengths were 0.5 and 1.0. +Arithmetic, factual and JSON checks passed in both stock and profile modes. The SGLang Hermes run completed four native protocol cases in five model requests, including revocation on a reused agent and a real `read_file` call. Three answers matched exactly. The initial approved no-tool case returned the -requested number followed by an unsolicited introduction, so the strict native -quality result remains failed. The engine used a fresh isolated dependency +requested number followed by an unsolicited introduction, failing the strict +answer check. The engine used a fresh isolated dependency environment, a configured 65,536-token context, two concurrent requests and -0.60 memory fraction. This run used short prompts; it did not establish -long-context capacity, image quality, OpenClaw coverage or full wizard engine -activation for this pairing. - -The profiles do not guarantee a particular response, zero refusals, or lossless -reproduction of an edited model. Use your workload to assess answer quality. +0.60 memory fraction. Long-context capacity, image quality, OpenClaw and full +wizard activation were not tested for this pairing. ## llama.cpp qualification @@ -160,71 +187,61 @@ including a stock request after the owner request. Timing runs used the uninstrumented build. The equivalent Qwen check observed 129 owner sites and zero stock sites. -Each 32-case direct run includes six short exact-answer fixtures in each mode. -These are smoke tests, not a benchmark of reasoning, coding or refusal rates. +Each 32-case direct run includes six short exact-answer smoke tests in each mode. The runs use two slots, 65,536 tokens per slot and a 2 GiB in-memory prompt -cache. An earlier 512 MiB cache evicted the separate mode/strength entries; -the larger cache retained the tested entries. Neither configuration reused a -cache entry from another mode. +cache. At 512 MiB, separate mode/strength entries were evicted; at 2 GiB, the +tested entries were retained. Both configurations kept mode caches isolated. Gemma's official MTP draft and ngram lookup both recorded accepted draft tokens in stock and profile modes. A repeated copy fixture recorded 48 accepted ngram -draft tokens in each mode. Some prompts generated no ngram matches, so an enabled -method alone was not counted as evidence of active drafting. +draft tokens in each mode. Other prompts generated no ngram matches. -Native agent tests include exact-answer misses and repeated or omitted -delegation calls. Successful authorization does not guarantee tool-following -or output-format compliance. Gemma's agent-workflow quality remains -experimental. The llama.cpp paths currently qualify text input; image input -requires separate validation. +Native agent tests had exact-answer misses and repeated or omitted delegation +calls. Gemma's agent-workflow quality remains experimental. These llama.cpp +checks cover text input; images are untested. Gemma's native Hermes checks covered revocation, a real file tool and delegated work. OpenClaw checked approved and unknown contacts, inherited child access with the tool flag on and off, and Always On. Fresh isolated child-policy cases passed for all three settings. Thinking on/off settings reached the engine; same-turn reasoning history was preserved when thinking was on. -Reused-conversation cases that repeated or omitted child calls failed their -workflow checks and remain in the validation record. +Reused-conversation cases failed workflow checks when they repeated or omitted +child calls. The [runtime summary](../benchmarks/results/llamacpp-qualification-20261030.json) records the engine and profile hashes, settings and measured scope. ## Small benchmark pilots -The [Gemma TensorFold GSM8K pilot](../benchmarks/results/gemma-native-gsm8k-pilot.json) -used the fixed first 16 test questions with five examples, a 512-token output -budget and DFlash active. Stock answered 14 of 16 correctly; the profile answered -15 of 16. This is a small, non-random pilot, not a full GSM8K score or evidence -of a general quality improvement. Refusal and over-refusal benchmarks remain -separate work. +Both TensorFold pilots used the first test questions, five examples per +question and a 512-token output budget. These non-random samples are not full +GSM8K scores. -The [Qwen TensorFold GSM8K pilot](../benchmarks/results/qwen-native-gsm8k-pilot.json) -used the first 32 test questions, five examples per question and a 512-token -output budget with DFlash2 active. Both stock and profile modes answered 31 of -32 correctly. This is also a small, non-random pilot, not a full GSM8K score or -proof of general quality parity. +| Model | Sample | Stock correct | Profile correct | Speculation | Record | +| --- | ---: | ---: | ---: | --- | --- | +| Gemma 4 26B A4B | 16 | 14 | 15 | DFlash | [Results](../benchmarks/results/gemma-native-gsm8k-pilot.json) | +| Qwen3.8-27B | 32 | 31 | 31 | DFlash2 | [Results](../benchmarks/results/qwen-native-gsm8k-pilot.json) | ## Software and distribution checks Automated tests cover request authorization, message/media binding, adapter registration, terminal sessions, tool inheritance, Always On, setup and process -supervision. Engine packages have separate operator and runtime checks. Test -counts change as the release tree is updated; model results above remain tied -to their recorded runtime configurations. +supervision. Engine packages have separate operator and runtime checks. The 1.1.0 update passed 768 core tests, 19 shared engine tests, 88 native -DeepSeek engine tests and eight legacy projection tests. These software checks -are separate from the GPU request checks above. +DeepSeek engine tests and eight legacy projection tests. + +Gemma vLLM passed pinned-version, import, entrypoint and CPU operator checks +in a fresh isolated dependency installation. Its serving checks used a +separately prepared GPU environment. Fresh dependency installations and wizard activation were checked for Qwen on vLLM and SGLang with BF16 weights. A fresh llama.cpp build passed wizard activation with Q8_0 weights. The TensorFold wizard passed on Qwen MLX with one DGX Spark, GLM Flash NVFP4 with four Sparks, and GLM Full EXL3 with -four Sparks. The x86 Qwen EXL3 engine has separate device coverage; a full x86 -wizard deployment was not part -of that test. Standard and signed-profile requests completed with the -declared speculative decoding active, followed by clean shutdown. These are -startup checks for the listed configurations. Fresh installation of other -model and engine combinations requires separate tests. +four Sparks. Qwen EXL3 passed device checks on x86; a full x86 wizard deployment +was not tested. Standard and signed-profile requests completed with the +declared speculative decoding active, followed by clean shutdown. Other +model/engine installation combinations remain untested. Gemma's llama.cpp setup passed a fresh wizard check, preparation and activation with the default ngram method and a 2 GiB prompt cache. Post-activation checks @@ -234,8 +251,7 @@ tokens. No separate draft-model download is needed for this default. The release source and SDK are built from a pinned tree. Packaging checks verify file contents and executable modes; a fresh SDK install verifies installed files, dependencies and the command-line entry point. Runtime downloads are -pinned by size, SHA-256 digest and complete file inventory. Model behavior on a -different deployment still requires a model run. +pinned by size, SHA-256 digest and complete file inventory. ## DeepSeek native profile on TensorFold 1.2.0 @@ -247,17 +263,18 @@ The operator checks covered 60 cases at layers 10, 22 and 35 across all four logical tensor-parallel partitions, including mixed rows and changing graph masks. All 780 comparisons passed. The largest sampled relative L2 error to the independent FP64 reference was about 0.168%, within the checked rounding bound. -This is matrix-site evidence, not full-model equivalence to another engine. +This measures the matrix operator; cross-engine full-model equivalence is unverified. Fifteen HTTP checks passed their expected outcomes, including signed requests, unsigned and replayed requests, invalid profiles and strengths, repeated cache use and concurrent stock/profile requests. Both modes proposed speculative tokens. Cancelled streams in both modes released their slots; subsequent -requests succeeded, and all four ranks shut down cleanly. Repeated stock and profile sessions each resumed an 8,720-token prefix +requests succeeded, and all four ranks shut down cleanly. Repeated stock and +profile sessions each resumed an 8,720-token prefix from RAM. These family cache hits appear in the runtime log, not the OpenAI cached-token field. JSON-schema requests were rejected in both modes, as the current TensorFold DeepSeek backend does not support them. The [qualification record](../benchmarks/results/deepseek-tensorfold-native-qualification-20261101.json) -contains exact model, profile and engine pins. These are functional checks, -not refusal-rate, quality or throughput benchmarks. +contains exact model, profile and engine pins. Refusal rate, answer quality and +throughput were not measured in this run. diff --git a/engines/llamacpp/README.md b/engines/llamacpp/README.md index 7c15e42..cdc71f1 100644 --- a/engines/llamacpp/README.md +++ b/engines/llamacpp/README.md @@ -15,7 +15,6 @@ For Gemma, it runs before the following normalization and leaves routed experts unchanged. The GGUF adapter container carries the profile's Q factors, their negative transpose and each site's positive FP32 strength. It applies `y - owner_mode_scale * site_scale * (y @ Q.T) @ Q`. Source weights are read-only. -No checkpoint pair, fitting procedure or private derivation data is required. ## Build without starting a service @@ -149,8 +148,7 @@ answer fixtures, cache separation by strength, concurrent requests, streaming and long inputs. Separate copy probes confirmed accepted ngram draft tokens in both modes. Qwen Q8_0 passed 32 direct checks with its external draft. See [validation](../../docs/validation.md) for the longer-input and native -harness results, including strict-answer limits. These runs do not establish -broad model-quality equivalence. +harness results, including strict-answer limits. The converter accepts standard projection profiles with finite positive site strengths and rejects unsupported transforms. Other model families, GGUF diff --git a/engines/llamacpp/manifest.json b/engines/llamacpp/manifest.json index bf471b8..154f413 100644 --- a/engines/llamacpp/manifest.json +++ b/engines/llamacpp/manifest.json @@ -5,7 +5,7 @@ "LICENSE": "0d96a4ff68ad6d4b6f1f30f713b18d5184912ba8dd389f86aa7710db079abcb0", "LICENSE.llama.cpp": "94f29bbed6a22c35b992c5c6ebf0e7c92f13b836b90f36f461c9cf2f0f1d010d", "NOTICE.md": "c8a365bb3b0f9da0a3269abbe62f924a23628a11409e2b6f8d0aee006f5d7a7a", - "README.md": "e81b877b52c49fff2f87754ef80f731e6f0c5415fce521049146170c39392224", + "README.md": "93da8f3bbfe755250ddada7551c09069cdf16eb334cff8c168faa0c28cdc4b1d", "backend.json": "97dd095e397a883247d2f9c07ae0edc395e147ea4623024004506a6508ca9ca3", "flashalign.patch": "f468b7db9b07e54cca2c6bc178fe77a4a0d1e8e3ce5e8465d0324bb5f91e59aa", "flashalign_replay.py": "6e2730a89fc4d70636bf3b139c5ab5eb8d7b205483ac28684aa529df6f565162", diff --git a/engines/sglang-gemma4/README.md b/engines/sglang-gemma4/README.md index f41150e..94c1771 100644 --- a/engines/sglang-gemma4/README.md +++ b/engines/sglang-gemma4/README.md @@ -26,19 +26,9 @@ The runtime checks model identity, all 50 writers and the supplied payload checksums. The wizard checks the declared checkpoint metadata and preserves compatible active NGRAM settings without asking for a draft model. -The experimental runtime was checked on one GB10 with a fresh dependency -environment, a configured 65,536-token context, two concurrent requests and -0.60 memory fraction. All 50 sites were observed. Mixed stock/profile rows, -authentication, replay rejection and cache separation at strengths 0.5 and -1.0 passed. NGRAM verification and draft proposals remained active. Six direct -arithmetic, factual and JSON checks passed across stock and profile modes. - -Four actual Hermes native protocol cases passed in five model requests, -including revocation on a reused agent and a real `read_file` call. Three -answers matched exactly. The initial approved no-tool case appended an -unsolicited introduction after the requested number; the strict native quality -result remains failed. - -These checks used short prompts. Long-context capacity, image quality, -OpenClaw and full wizard engine activation were not tested for this pairing. -See [validation](../../docs/validation.md) for the other engines' results. +The tested GB10 configuration used a 65,536-token context, two concurrent +requests and 0.60 memory fraction. Direct protocol and answer checks passed; +the native Hermes strict-answer check failed on one of four cases. +Long-context inputs, image quality, OpenClaw and full wizard engine activation +remain untested for this pairing. +See [validation](../../docs/validation.md) for the results. diff --git a/engines/sglang-gemma4/sha256.json b/engines/sglang-gemma4/sha256.json index fa8e862..6f8c69f 100644 --- a/engines/sglang-gemma4/sha256.json +++ b/engines/sglang-gemma4/sha256.json @@ -1,7 +1,7 @@ { "LICENSE": "0d96a4ff68ad6d4b6f1f30f713b18d5184912ba8dd389f86aa7710db079abcb0", "NOTICE.md": "6050d3a9e5f7bb79b42a2f86568c886f33d232f4850d198a7cd2c93eec5e95e5", - "README.md": "3bd9e68ab61d90f7541b6ed9e01acc40b992211c9fdeea25832c6b5d4937aa2d", + "README.md": "9d9b8774beea96d0825c34b0378dfd4dcdf02ecd5e60c43f4f597405e4a1a6c3", "backend.json": "941a3528eebcf89431a799c6eccf4dbec2998f8da54517fe84e0cabe970ef898", "expected-sites.json": "06c8f9c00b67c5893f1cafe8c4c92b683d6facce9ea056f4fd3aee5c791e9159", "flashalign_sglang/__init__.py": "5d4ab32e6f702b57a0bea12cc92a2dd56d499ed2a271dc08eeda2fc2a96e10a8", diff --git a/engines/sglang-glm53-flash/README.md b/engines/sglang-glm53-flash/README.md index 9193b26..5de705d 100644 --- a/engines/sglang-glm53-flash/README.md +++ b/engines/sglang-glm53-flash/README.md @@ -20,14 +20,8 @@ declared target and native-MTP writer inventory. The runtime accepts the documented normalized operator format; it does not depend on how a profile was created. -The four-GPU SM120 configuration completed protocol, cache, native Hermes and -OpenClaw checks at a 65,536-token context with EAGLE active. Stock, runtime profile -and exact edited reference each passed 16/16 common fixtures. Both stock and -profile retrieved the correct answer from 59,042 input tokens. Native Verified -and Always On paths passed their protocol checks, including child and image -verification behavior. +See [validation](../../docs/validation.md) for protocol, cache, long-input and +native harness results on four SM120 GPUs with EAGLE. Strict visual-answer checks failed in stock, runtime profile and exact edited -reference. Those failures remain part of the qualification; image transport and -authorization success is not a visual-quality guarantee. NEXTN and a complete -SM121 deployment have not been GPU-qualified by this run. +reference. NEXTN and a complete SM121 deployment remain untested on GPU. diff --git a/engines/sglang-qwen4/README.md b/engines/sglang-qwen4/README.md index c8b3b43..01208dc 100644 --- a/engines/sglang-qwen4/README.md +++ b/engines/sglang-qwen4/README.md @@ -31,12 +31,9 @@ A profile directory contains `profile.json`, `profile.safetensors` and its source-model license. The signing key must be an owned private regular file of exactly 32 random bytes. Models and profiles must already be present. -The default context is 65,536 tokens. This setting passed the managed wizard, -65,000-input-token requests in both modes, and native Hermes and OpenClaw -checks on two 95 GiB RTX PRO 6000 Blackwell Max-Q cards. The earlier complete -operator test runs used 2,048 tokens. The current context qualification used -serial requests and a reused pinned dependency environment. Larger contexts -and lower-memory systems need separate qualification. +The default context is 65,536 tokens, tested with serial requests on two +95 GiB RTX PRO 6000 Blackwell Max-Q GPUs. See +[validation](../../docs/validation.md) for setup, long-input and harness results. The launcher preserves native MTP and uses TP/EP two with PLE tables in pinned host memory. The qualified path uses eager execution. Unsupported speculative diff --git a/engines/sglang-qwen4/sha256.json b/engines/sglang-qwen4/sha256.json index ae478e4..f16dfdf 100644 --- a/engines/sglang-qwen4/sha256.json +++ b/engines/sglang-qwen4/sha256.json @@ -1,7 +1,7 @@ { "LICENSE": "0d96a4ff68ad6d4b6f1f30f713b18d5184912ba8dd389f86aa7710db079abcb0", "NOTICE.md": "6050d3a9e5f7bb79b42a2f86568c886f33d232f4850d198a7cd2c93eec5e95e5", - "README.md": "9061e680c1882e2c12b1951c76210f7d89310f9390c0f6c1ae5e2d1bc29d413f", + "README.md": "2f90095c8a4224998a03227cc3d7f47b15190451413f147a365c4f4a93cfa64d", "backend.json": "7c91c82f86bdb821c29fcfb8a8912fc37b80b573be87958ba30b7760a07f327f", "flashalign_sglang/__init__.py": "5d4ab32e6f702b57a0bea12cc92a2dd56d499ed2a271dc08eeda2fc2a96e10a8", "flashalign_sglang/flashalign_replay.py": "6e2730a89fc4d70636bf3b139c5ab5eb8d7b205483ac28684aa529df6f565162", diff --git a/engines/sglang/README.md b/engines/sglang/README.md index eab7b6a..d6df7f9 100644 --- a/engines/sglang/README.md +++ b/engines/sglang/README.md @@ -1,12 +1,10 @@ # FlashAlign SGLang integration This bridge applies a saved FlashAlign projection inside SGLang's target-model -forward pass. It is not a prompt template, logit filter or a weight-file edit. +forward pass. The integration is pinned to SGLang **v0.5.20**, commit -`94602c9c2b7cbdb8efd5c52802dac6a1c180089e`. Runtime status is **experimental, GPU verified** for the Qwen model below. -CPU tests establish -component behavior, not complete serving correctness or model quality. +`94602c9c2b7cbdb8efd5c52802dac6a1c180089e`. It supports the Qwen model below. ## Prepare and launch @@ -29,11 +27,9 @@ python3 prepare.py --source /private/sglang-upstream --output /private/flashalig -- --context-length 65536 --enable-cache-report ``` -The default context is 65,536 tokens. Managed setup, 65,000-input-token -requests in both modes, and native Hermes and OpenClaw checks passed on one -95 GiB RTX PRO 6000 Blackwell Max-Q GPU with DFlash2 active. This context -qualification used serial requests and reused pinned dependencies. Larger -contexts and lower-memory devices need separate qualification. +The default context is 65,536 tokens, tested with serial requests on one +95 GiB RTX PRO 6000 Blackwell Max-Q GPU with DFlash2. See +[validation](../../docs/validation.md) for setup, long-input and harness results. The model, profile and draft model must already exist. The key must be a private 32-byte regular file. The launcher accepts repeated `--profile` arguments and @@ -59,7 +55,7 @@ fallback disables it. - Existing in-memory radix-cache behavior with a server-created namespace containing the profile digest and exact strength. Owner and standard requests cannot share a namespace. LoRA composition, session-cache reuse, HiCache and -distributed/disaggregated execution are rejected by this integration. + distributed/disaggregated execution are rejected by this integration. The supplied draft model is left unchanged and provides proposals. Only the target forward pass applies the selected profile; it verifies the proposals against that target. @@ -99,18 +95,8 @@ Tests cover raw-body signatures, replay, tampering, cache namespaces, stock-row identity, mixed prefill and speculative verification rows, strict loader behavior, and actual PyTorch module hooks using the pinned full-attention naming pattern. -The qualification run passed 23 CPU tests and 47 live checks on BF16 -Qwen3.8-27B with DFlash2. All 43 generated-answer checks passed, including all -16 shared math/code fixtures in each mode, and both real streaming requests. -Every generation reported active speculative verification. Repeat requests -reused 576 cached tokens in their own mode; the first owner request and a change -of strength reused zero. These small synthetic checks are not a broad benchmark. - -Two individual warm streaming measurements gave TTFT 0.113 seconds in standard -mode and 0.139 seconds in owner mode. They are examples, not general latency -guarantees. Startup took 295 seconds with first-time kernel compilation and -33 seconds with those kernels cached on the qualification machine. +See [validation](../../docs/validation.md) for live protocol, cache and answer +checks, and [measured results](../../docs/benchmark-results.md) for timing samples. Upstream SGLang source retains its Apache 2.0 notices. FlashAlign runtime additions -follow the surrounding FlashAlign repository's license. Profile creation code, -checkpoint access and private model-quality records are not part of this bridge. +follow the surrounding FlashAlign repository's license. diff --git a/engines/sglang/sha256.json b/engines/sglang/sha256.json index 9a78742..57375cd 100644 --- a/engines/sglang/sha256.json +++ b/engines/sglang/sha256.json @@ -9,7 +9,7 @@ }, { "path": "README.md", - "sha256": "3262b6a8a3c0f71cd718be196cfe77d43dc8d951df8a03e465463d73813000be" + "sha256": "212c00069167536d6473db5d31cabd0bf5228e6fd73d261d4e267e18062826fc" }, { "path": "backend.json", diff --git a/engines/vllm-deepseek-native/README.md b/engines/vllm-deepseek-native/README.md index 4bfc68d..adec404 100644 --- a/engines/vllm-deepseek-native/README.md +++ b/engines/vllm-deepseek-native/README.md @@ -9,12 +9,12 @@ Use the FlashAlign wizard for installation. It downloads the profile payload int - Linux x86_64, Docker Engine and NVIDIA Container Toolkit. - Four SM120 GPUs with at least 94 GiB total and 90 GiB free memory per GPU. - The exact model revision in `backend.json`, including its original quantization. -- 221 GiB available host/cgroup memory before an unattended start. The container is limited to 224 GiB and the supervisor stops it if available host memory falls below 24 GiB. These are deployment policies, not a claim of the smallest possible machine. +- 221 GiB available host/cgroup memory before an unattended start. The container is limited to 224 GiB and the supervisor stops it if available host memory falls below 24 GiB. These are the launcher’s current memory limits. - Space for the model, the container image, compilation caches and the approximately 1.02 GiB native profile payload. The default keeps adaptive DSpark7, B12X, FP8 KV cache and the engine's graph path enabled. This pinned model uses FULL decode graphs and eager prefill within its `FULL_AND_PIECEWISE` configuration. Prefix caches are separated by effective profile, payload digest and strength. Switching between already loaded off and owner modes does not restart the engine. Loading a different artifact does. -The current native starter is validated for text. Image cache isolation is implemented but image requests are not yet part of this backend's published qualification. This backend does not claim compatibility with other GPU architectures, tensor-parallel sizes, expert/context parallelism, external KV transfer or LoRA. +Text requests are validated. Image requests remain untested. Other GPU architectures, tensor-parallel sizes, expert/context parallelism, external KV transfer and LoRA are unsupported. ## Direct engine setup @@ -45,4 +45,4 @@ Run the CPU contract and lifecycle tests with: python -m pytest -q engines/vllm-deepseek-native/tests ``` -These test artifact parsing, corruption rejection, owner/off row mapping, separate graph identities, authentication, cache namespaces, immutable bindings and container ownership. CPU tests do not establish GPU behavior, throughput or a refusal rate. The [public GPU qualification](../../docs/validation.md#deepseek-native-profile) passed eight protocol/cache requests using the pinned image's libraries on four SM120 GPUs. Actual Docker image checks covered artifact loading, key ownership, plugin discovery and snapshot mounts on CPU; full Docker GPU/wizard activation remains untested. +Tests cover artifact validation, row selection, authentication, cache isolation and container lifecycle. See [validation results](../../docs/validation.md#deepseek-native-profile) for GPU checks and the remaining Docker GPU/wizard test gap. diff --git a/engines/vllm-deepseek/README.md b/engines/vllm-deepseek/README.md index 8338a80..208e69e 100644 --- a/engines/vllm-deepseek/README.md +++ b/engines/vllm-deepseek/README.md @@ -1,30 +1,22 @@ -# DeepSeek V4.1 Flash on native vLLM V2 +# DeepSeek projection profiles on vLLM -This experimental integration accepts the pinned stock -`deepseek-ai/DeepSeek-V4.1-Flash-1017` checkpoint and a locally supplied FlashAlign -profile. It contains no model weights, profile tensors or profile-creation tools. -The wizard selects it by model family and exact checkpoint metadata: +This integration runs locally supplied compact or grouped projection profiles +with the pinned stock `deepseek-ai/DeepSeek-V4.1-Flash-1017` checkpoint. +The current DeepSeek starter uses a different operator and requires the +[native matrix backend](../vllm-deepseek-native/README.md). ```sh bash ./install.sh --engine-only --backend vllm \ - --model-path /models/DeepSeek-V4.1-Flash-1017 --devices 0,1,2,3 + --model-path /models/DeepSeek-V4.1-Flash-1017 \ + --profile /profiles/deepseek-projection --devices 0,1,2,3 ``` -The included 85-site DeepSeek profile is accepted without rewriting its manifest, -basis tensors, strengths or digest. It has passed bounded engine, cache, -image-routing and direct-gateway checks; its quality status remains experimental. -Operator-supplied `flashalign.grouped_projection.v1` profiles are also supported; -the validator checks their typed sites, shared operators, Engram slices and image -bias handling. A valid artifact is not proof of edited-checkpoint parity or model -quality. Profile-specific evidence belongs with the supplied profile. - -Grouped profiles may select a subset of the supported sites and use different +`flashalign.grouped_projection.v1` profiles may select supported sites with orthonormal basis ranks from 1 through 128. Omitted sites keep their standard -output. The direct engine loader and setup wizard use the same native-target -contract, including combined MoE output, Engram value slices and pre-bias image -outputs. Unknown targets, unused operators and unsupported requirements are -rejected. This describes the accepted profile format; each supplied profile -still needs its own runtime and quality checks. +output. The loader and wizard validate combined MoE output, Engram value slices +and pre-bias image outputs. Unknown targets, unused operators and unsupported +requirements are rejected. Validate each supplied profile against its intended +checkpoint. ## Checkpoint and runtime preparation @@ -49,48 +41,23 @@ persistent replay state at a separate `--replay-file`. Neither file belongs in a repository or model directory. Reusing a deployment signing key also requires preserving that deployment's replay state. See [REPLAY.md](REPLAY.md). -## Qualified scope and experimental memory policy - -The native engine was exercised on four SM120 GPUs with approximately 95 GiB -VRAM each, TP4/PP1/DP1, eager execution, native DSpark5, and Engram CPU offload. -The wizard preserves that speculative configuration. An incompatible active -method or external draft stops setup instead of being disabled or replaced. -This package does not extend that qualification to other hardware or parallelism. - -Before activation, the wrapper requires four SM120 GPUs with at least 90 GiB -free on each. Its conservative unattended-start policy requires 365 GiB of -available host/cgroup headroom. That policy is **not a measured minimum**: the -successful reference run started with 367.65 GiB host `MemAvailable`, while its -initial cgroup headroom was not recorded. The observed steady host usage was -approximately 202 GiB. Lower-memory machines remain unqualified. A controlled -experiment can explicitly set `engine.allow_untested_memory` to true in its plan; -the result continues to report that exception. It is not a support claim. - -The wrapper monitors host and cgroup headroom and stops its child workers below -64 GiB. Unified Linux cgroup v2 accounting is required. Cgroup reports retain raw current/max, file, shmem and inactive-file -counters. Working headroom may credit ordinary inactive file cache, bounded by -`min(inactive_file, file - shmem - unevictable - dirty - writeback, current)` -with negative values clamped to zero. Pinned and shared-memory allocations earn -no credit. This prevents the preceding checkpoint hash pass from being counted -entirely as unreclaimable memory. Host `MemAvailable` is an independent cap; -the launcher never drops page cache. This floor cannot be disabled by the experimental override. The monitor -is a safety measure, not a guarantee against every allocation failure. Default -context is 65,536 tokens with two concurrent requests, matching the native -engine context ceiling exercised separately. Other resource settings need their own check. -Setup rejects an existing harness context or output budget above that selected -engine limit before changing its route. It preserves the configured values and -does not automatically enlarge this experimental preset. - -The wizard and this package completed full-model activation and shutdown checks -with the 85-site profile under the explicit lower-memory test setting. That -result does not remove the default 365 GiB startup policy or qualify every -lower-memory host. Native harness coverage is listed in the -[starter pack](../../profiles/README.md). - -Verified and Always On native Hermes checks completed with the profile active. -All 14 isolated OpenClaw cases passed protocol and strict-answer checks, -including image authorization, both tool-inheritance settings and same-session -CLI behavior. Native DSpark stayed active during these checks. +## Requirements and defaults + +The launcher requires four SM120 GPUs with at least 90 GiB free on each, +TP4/PP1/DP1 and unified Linux cgroup v2 accounting. It uses eager execution, +native DSpark5 and Engram CPU offload. An incompatible active speculative +method or external draft stops setup. + +The unattended-start policy requires 365 GiB of available host/cgroup memory. +For a controlled lower-memory deployment, set `engine.allow_untested_memory` +to true in the plan. This exception does not disable the 64 GiB runtime memory +floor: the launcher stops its workers if available memory falls below it. +Reclaimable file cache is accounted for separately from pinned and shared memory. + +Default context is 65,536 tokens with two concurrent requests. Setup rejects a +harness context or output budget above the selected engine limit before changing +its route. See [validation](../../docs/validation.md) for tested configurations, +including lower-memory runs. The standalone launcher requires Linux pidfd and child-subreaper support. It adopts and reaps native workers if their parent exits, including workers in a @@ -112,7 +79,7 @@ supported choices are preserved. Explicitly disabled thinking remains disabled: the trusted DeepSeek route translates `reasoning_effort=none` to `chat_template_kwargs.enable_thinking=false` after capability validation and before the final request is signed. Setup reports this model compatibility rule. -This validated serving lane also declares native image input. Setup updates the +This backend also declares native image input. Setup updates the selected Hermes model's vision capability and OpenClaw's input descriptor; explicit text or auxiliary-image routing preferences remain unchanged. diff --git a/engines/vllm-deepseek/sha256.json b/engines/vllm-deepseek/sha256.json index faa13ab..2f2008a 100644 --- a/engines/vllm-deepseek/sha256.json +++ b/engines/vllm-deepseek/sha256.json @@ -2,7 +2,7 @@ "LICENSE": "0d96a4ff68ad6d4b6f1f30f713b18d5184912ba8dd389f86aa7710db079abcb0", "LICENSE.vllm": "c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4", "NOTICE.md": "f37e9477f8faec4a486fce2ce4500656c3db15ad460ec6899e8a925702ca5a7a", - "README.md": "b8969ba2f82fbb01bc124c30fb7c6a560ee5b308b05d2abf8db717022e0e52e0", + "README.md": "19a50fb61be331f905cb180e321183cbb6ffc3e623a7ad93c06e5642e70ed8b4", "REPLAY.md": "88001bcd437256b141a08bb4d89e6660d3d8c8441e087371820cb7327fec90a8", "UPSTREAM_PROVENANCE.json": "d64ca56d6737dfc1785e15f628dd15c56a5dc4a335e7e84a4469e305e14ef0a7", "backend.json": "2ffffa6c19809cc967dfbfc69d5a61fcb0a69895219de8aa37f0b29a32d1a6bc", diff --git a/engines/vllm-gemma4/README.md b/engines/vllm-gemma4/README.md index af1f1ef..90afb3c 100644 --- a/engines/vllm-gemma4/README.md +++ b/engines/vllm-gemma4/README.md @@ -5,26 +5,18 @@ MLP outputs before their post-projection normalization. It implements the same authenticated request and cache-isolation contract as the other FlashAlign backends. Tool permissions remain with the caller. -The tested checkpoint, metadata fingerprints, dependency versions, single-GPU -layout and evaluation scope are recorded in `backend.json`. This is an -experimental integration. The Aevonix Research starter profile -`gemma4-26b-a4b-v1` is included under `profiles/`. Engine and native harness -protocol checks passed with the answer-quality limits below. Model weights -must be obtained separately. - -The launcher defaults use a 65,536-token context, two concurrent sequences and -ngram speculation. An earlier run on one 96 GB NVIDIA RTX PRO 6000 passed a -bounded 59,987-token retrieval check in both modes. The latest native harness -run used one GB10, one concurrent sequence and 0.60 GPU memory utilization, -with all 50 sites attached and three-token ngram active. All 11 protocol checks -passed. Earlier 59,045-token retrieval passes are retained; -the latest GB10 run does not validate the two-sequence, 0.85-memory defaults. - -Native images reach the tested `gemma4` model through Hermes and OpenClaw. -The `gemma4_text` variant remains text only. Exact-answer misses, unexpected -vision-tool calls and one reasoning-budget recovery remain recorded in -[validation](../../docs/validation.md). Correct profile activation does not -guarantee an answer matches its target. +The pinned checkpoint, dependency versions and single-GPU layout are recorded +in `backend.json`. Use the included `gemma4-26b-a4b-v1` starter profile with a +separately downloaded model checkpoint. + +Defaults are a 65,536-token context, two concurrent sequences and ngram +speculation. These defaults have not been tested together on GB10; its recorded +run used one sequence and 0.60 GPU memory utilization. See +[validation](../../docs/validation.md) for long-input results and the recorded +answer-quality failures. + +Native images reach the `gemma4` model through Hermes and OpenClaw. The +`gemma4_text` variant is text only. Use the FlashAlign wizard with the included profile: @@ -40,8 +32,5 @@ integrity. The profile's source-model terms still apply separately. The launcher configures the Gemma reasoning/tool parsers and preserves a supported active ngram configuration. Unsupported active speculative methods -stop setup without disabling them. A fresh isolated dependency installation -passed pinned-version, import, entrypoint and CPU operator checks. Serving -checks used the separately prepared GPU environment; GPU startup still needs -the documented CUDA build prerequisites. See `backend.json` for the measured -validation limits. +stop setup without disabling them. GPU startup requires the CUDA build +prerequisites listed in `backend.json`. diff --git a/engines/vllm-gemma4/sha256.json b/engines/vllm-gemma4/sha256.json index 2c7b10c..69ae323 100644 --- a/engines/vllm-gemma4/sha256.json +++ b/engines/vllm-gemma4/sha256.json @@ -1,7 +1,7 @@ { "LICENSE": "0d96a4ff68ad6d4b6f1f30f713b18d5184912ba8dd389f86aa7710db079abcb0", "NOTICE.md": "8d90714c640cf4c5703472d94e9287a4df872ef8e7e689c5d3bc088793b38c89", - "README.md": "ccfaae34640410d8d71e119ead4ce3aaeaf64b01cbb12ca7b416b3e42c7bc46d", + "README.md": "9e845f0f065f09cd9c7780512105e164d993d897e356eb6b049eb531737c58cd", "backend.json": "c12643c2e6901f4e93124e9b5d99ef206812c96e118b4c9484edecceb0cb1dc6", "flashalign_vllm/__init__.py": "f263c3b6ca0b1c4ccd8abe020ab74324873bb93710d7f7296bd3fb3648eaa702", "flashalign_vllm/__main__.py": "18d0d2759a93e355b283fc61ee6b6c3448e8d90155699d75bf5789dbf2b98389", diff --git a/engines/vllm/README.md b/engines/vllm/README.md index 46edeae..022a7e4 100644 --- a/engines/vllm/README.md +++ b/engines/vllm/README.md @@ -29,7 +29,7 @@ methods stop startup. The launcher never adds a flag that disables speculation. ## Supported scope -The first adapter supports the dense Qwen3.5/3.8 architecture and standard +This adapter supports the dense Qwen3.5/3.8 architecture and standard projection profiles at embedding, attention-output and MLP-output sites. It requires one GPU, one API process and eager execution. Disabling CUDA graphs is an explicit performance limitation; speculative decoding remains available. @@ -50,19 +50,14 @@ the actual embedding/output modules. The base weights stay unchanged; standard rows bypass all transform arithmetic. External draft models keep their ordinary weights; the target model verifies draft tokens under the selected profile. -The validated configuration is Qwen3.8-27B BF16 with the bundled 129-site -profile and ngram speculation on one RTX PRO 6000. Eighteen CPU tests and -15 real-model protocol/cache checks passed. All 16 arithmetic/Python-output -fixtures passed in standard mode and all 16 passed with the profile active, -with no truncated answers. These are small retention checks; other checkpoints, -quantizations and speculative methods need their own qualification. +The tested configuration uses Qwen3.8-27B BF16, the bundled 129-site profile +and ngram speculation on one RTX PRO 6000. See +[validation](../../docs/validation.md) for protocol, answer and long-input results. -The current wizard defaults use a 65,536-token context, up to four requests, -an 8,192-token prefill batch and GPU-memory fraction 0.85. Serial checks on a -96 GB RTX PRO 6000 processed 65,470 input tokens in both modes, passed all 32 -small arithmetic/code fixtures, and exercised native Hermes/OpenClaw tools and -child inheritance. They do not qualify four simultaneous maximum-length requests. -The full `qwen3_5` model declares image input; `qwen3_5_text` remains text-only. +Wizard defaults are a 65,536-token context, up to four requests, an 8,192-token +prefill batch and GPU-memory fraction 0.85. Four simultaneous maximum-length +requests remain untested. The full `qwen3_5` model accepts images; +`qwen3_5_text` is text only. Installing the package leaves plain vLLM unchanged until profiles are explicitly configured through this launcher. Usage telemetry is disabled by the launcher. diff --git a/openclaw-owner-mode/README.md b/openclaw-owner-mode/README.md index 1415667..d265f43 100644 --- a/openclaw-owner-mode/README.md +++ b/openclaw-owner-mode/README.md @@ -50,14 +50,11 @@ The integration targets the OpenClaw 2026.10.8 plugin interface. See [compatibility](../docs/compatibility.md) for runtime requirements and activation checks. -Serving integrations can declare model-specific request controls. For a lane -that supports an omitted output budget, FlashAlign keeps catalog capacity -separate from the request budget and preserves explicit task limits. A model -that cannot disable thinking can reject that choice before dispatch, while -leaving an omitted preference unchanged. See the policy format in +FlashAlign preserves explicit output limits and supports omitting them when the +engine allows it. Model-specific reasoning restrictions are checked before +dispatch; an omitted preference stays unchanged. See the policy format in [engine setup](../docs/engine-setup.md). These controls require the regular -provider stream; dedicated device worker-local inference needs a separate -integration. +provider stream. Device worker-local inference is unsupported. Licensed under [AGPL-3.0-only](LICENSE), with a separate commercial option. The extension includes the full terms and attribution notice. diff --git a/profiles/README.md b/profiles/README.md index 74a29e3..c976776 100644 --- a/profiles/README.md +++ b/profiles/README.md @@ -4,27 +4,24 @@ FlashAlign provides five experimental starter profiles. Qwen, Gemma and both GLM profiles are compact artifacts included in this checkout. The DeepSeek profile contains exact native weight tensors and uses a 1.09 GB public release asset. Setup downloads that payload to an external cache and verifies its size -and SHA-256 before staging it. No private repository access is required. +and SHA-256 before staging it. | Model | Profile | Current runtime | | --- | --- | --- | -| DeepSeek V4.1 Flash 1017 | `deepseek41-flash1017-v1` | [Pinned native vLLM](../engines/vllm-deepseek-native/README.md) with official FP8, or [TensorFold 1.2.0](../docs/native-tensorfold.md) with the declared EXL3 base; four SM120 GPUs | +| DeepSeek V4.1 Flash 1017 | `deepseek41-flash1017-v1` | [vLLM](../engines/vllm-deepseek-native/README.md), [TensorFold](../docs/native-tensorfold.md) | | GLM 5.3 Full | `glm53-full-v1` | TensorFold | | GLM 5.3 Flash | `glm53-flash-v1` | TensorFold and SGLang | | Qwen3.8-27B | `qwen38-27b-v1` | TensorFold, vLLM, SGLang and llama.cpp | | Gemma 4 26B A4B | `gemma4-26b-a4b-v1` | TensorFold, vLLM, SGLang and llama.cpp | -The current DeepSeek starter uses the Keys source's 26 WO-B matrices and their -26 scale tensors. It applies them only to authorized rows and leaves normal -rows on the stock path. It supports strength **1.0**. It requires the native -operator backend. TensorFold 1.2.0 uses the same unchanged payload with its -explicitly declared EXL3 serving base. Compact projection loaders cannot load -this artifact. See [DeepSeek details](deepseek41-flash1017-v1/README.md). +The DeepSeek starter uses 26 WO-B matrices and their scale tensors, with strength +fixed at **1.0**. It requires a native operator backend; compact projection +loaders cannot load it. See [DeepSeek requirements](deepseek41-flash1017-v1/README.md) +for the supported checkpoints and current four-GPU configuration. Use the exact checkpoint revision and serving format declared by the profile -and backend. [Validation](../docs/validation.md) distinguishes file integrity, -engine behavior and answer quality. Successful loading does not establish -full-model equivalence or a particular refusal rate. +and backend. See [compatibility](../docs/compatibility.md) and +[validation results](../docs/validation.md). ## Attribution and terms diff --git a/profiles/deepseek41-flash1017-v1/README.md b/profiles/deepseek41-flash1017-v1/README.md index f8eddb0..d090414 100644 --- a/profiles/deepseek41-flash1017-v1/README.md +++ b/profiles/deepseek41-flash1017-v1/README.md @@ -5,12 +5,6 @@ This starter applies the exact native WO-B weights and scales from revision `de412d2249632903dace23399526e00a145aada5`. The source edit covers attention layers 10–35. Runtime packaging is by Aevonix. -Use the official `deepseek-ai/DeepSeek-V4.1-Flash-1017` checkpoint at -`1bae0e1088993b9b82de08f59de5be5a34e42170` with the -[pinned native vLLM backend](../../engines/vllm-deepseek-native/README.md). -The publisher does not identify the original stock revision used to produce -its edit; the revision above is the audited serving base for this package. - The profile format is `flashalign.native_wo_b.v1`, transform `native_matrix_override_v1`. Strength is fixed at **1.0**. The artifact contains 26 native FP8 matrices and 26 UE8M0 scale tensors. It is not a direction-vector @@ -36,20 +30,20 @@ Authorized requests select the edited WO-B matrices. Normal requests retain the stock matrices. Mixed batches select the correct operation per row, with separate cache identities. Speculative decoding stays enabled. -Both backends use Linux x86_64 and four SM120 GPUs. The pinned vLLM image -uses the official FP8 checkpoint. TensorFold 1.2.0 uses -[`LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-3.5bpw`](https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-3.5bpw/tree/04a492fc3b4d25ddbab8f2d2f07f235ec77c9010) -at revision `04a492fc3b4d25ddbab8f2d2f07f235ec77c9010`. Its bundle explicitly -binds that serving base to this profile's original official model pin. +The current backends require Linux x86_64 and four SM120 GPUs. Both load the +same profile payload with these pinned serving checkpoints: + +| Backend | Checkpoint | Revision | +| --- | --- | --- | +| [vLLM](../../engines/vllm-deepseek-native/README.md) | `deepseek-ai/DeepSeek-V4.1-Flash-1017` (official FP8) | `1bae0e1088993b9b82de08f59de5be5a34e42170` | +| [TensorFold 1.2.0](../../docs/native-tensorfold.md) | [`LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-3.5bpw`](https://huggingface.co/LibertAIDAI/DeepSeek-V4.1-Flash-1017-EXL3-3.5bpw/tree/04a492fc3b4d25ddbab8f2d2f07f235ec77c9010) | `04a492fc3b4d25ddbab8f2d2f07f235ec77c9010` | -The profile file and native edited matrices are unchanged between engines. -TensorFold keeps the EXL3 stock path and selects these FP8 matrices for owner -rows. Base quantization and arithmetic differ from vLLM, so this is not a claim -of identical full-model outputs. The source's EXL3 sidecars are not substitutes -for this payload. SGLang and llama.cpp do not implement this native operator. +TensorFold keeps the EXL3 stock path and selects the profile's FP8 matrices for +owner rows. Its base quantization and arithmetic differ from vLLM. The source's +EXL3 sidecars cannot replace this payload. SGLang and llama.cpp do not implement +this native operator. -The profile uses exact source tensors, but full-model output identity and a -particular refusal rate are not guaranteed. See [validation](../../docs/validation.md). +See [validation](../../docs/validation.md) for measured results and limits. The software, model and profile-source terms are separate. Retain the [MIT notice](LICENSE) and [source agreement](RESPONSIBLE_USE.md).