Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Initial Aevonix FlashAlign release:
- Managed Always On API gateway for clients that do not use a harness or the SDK.
- Tool defaults and exact-name overrides for model access and child inheritance.
- Setup wizard with harness detection, model discovery and compatibility checks.
- Shared FlashAlign terminal design with a sharp pixel logo, a brief startup
- Shared FlashAlign terminal interface with a pixel logo, a brief startup
animation, keyboard selection panels and plain-text fallbacks.
- Five included experimental starter profiles for GLM, DeepSeek, Qwen and Gemma,
fitted and packaged by Aevonix from [credited checkpoints](profiles/SOURCES.md).
Expand Down
12 changes: 5 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,8 +15,8 @@ channels, or keep a profile active across every routed request.

FlashAlign works with supported base and quantized weights without permanently
changing them. Run it directly, with Hermes or OpenClaw, or through your own
harness. Managed inference runs on Linux with NVIDIA GPUs, including DGX Spark
for the qualified TensorFold paths.
harness. Managed inference runs on Linux with NVIDIA GPUs. See
[compatibility](docs/compatibility.md) for DGX Spark support and engine requirements.

The V1 starter profiles focus on refusal reduction. Future profiles could also
increase refusals for public or higher-risk tasks.
Expand Down Expand Up @@ -60,17 +60,15 @@ Fitting and runtime packaging by Aevonix. See the

| Model | Supported engines |
| --- | --- |
| DeepSeek V4.1 Flash 1017 | Pinned native vLLM with official FP8, or TensorFold 1.2.0 with the declared EXL3 base |
| DeepSeek V4.1 Flash 1017 | vLLM, TensorFold |
| GLM 5.3 Full | TensorFold |
| GLM 5.3 Flash | TensorFold, SGLang |
| Qwen3.8-27B | TensorFold, vLLM, SGLang, llama.cpp |
| Gemma 4 26B A4B | TensorFold, vLLM, SGLang, llama.cpp |

See [compatibility](docs/compatibility.md#engines-and-models) for hardware
requirements, checkpoint formats and feature limits. The [validation results](docs/validation.md)
record the configurations tested. The included [starter profiles](profiles/README.md)
remain experimental; setup checks that the selected profile matches the model
and engine.
requirements, pinned checkpoints and feature limits, and
[validation](docs/validation.md) for tested configurations and results.

Switching happens per request among profiles already loaded by the engine.
Loading a new model or profile happens beforehand and can require an engine
Expand Down
24 changes: 11 additions & 13 deletions assets/demo/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,27 +5,25 @@
The [README preview](flashalign-promo-preview.gif) is a silent excerpt of its
first 8 seconds.

This video was recorded with the version 1.0 compact DeepSeek profile.
Version 1.1 replaces that starter with the native Keys profile; the video
does not demonstrate the replacement artifact.
This video uses the retired version 1.0 compact DeepSeek profile, rather than
the current native Keys starter.

The promo is an edited video. Its model text comes from three recorded
The video uses three recorded
DeepSeek V4.1 Flash 1017 request pairs (EXL3 3.5-bpw, TensorFold 1.1.9,
DSpark active, temperature 0, seed 20261030, thinking off), with FlashAlign
off and on (`deepseek41-flash1017-v1`, strength 1). Each pair was recorded
twice with matching text. The prompts are OR-Bench Hard-1K rows 321, 244 and
194, shown exactly as tested.

Every response on screen is an exact, continuous excerpt of the recorded text;
an ellipsis marks omitted text. In all three examples the stock model also
pointed to other help after refusing; the video shows only the refusal. The
recipe's glow claims were not tested.
Responses are continuous excerpts of the recorded text, with ellipses marking
omissions. The stock model also suggested other help after each refusal;
those suggestions are omitted. Recipe claims in the model output were not
verified.

The text reveal is animated for the edit, so the video does not show response
speed. These are selected examples, not a benchmark score; see the
[measured results](../../docs/benchmark-results.md). The setup scene shows the
installer's real output, shortened. The music and sound effects were made for
this video.
The animated text reveal does not represent response speed. These selected
examples are separate from the [measured results](../../docs/benchmark-results.md).
The setup scene uses shortened installer output. Music and sound effects were
created for the video.

Prompts: [OR-Bench](https://huggingface.co/datasets/bench-llm/or-bench) by
Justin Cui, Wei-Lin Chiang, Ion Stoica and Cho-Jui Hsieh, revision `e36d8b80`,
Expand Down
20 changes: 7 additions & 13 deletions docs/agent-setup.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,14 +2,9 @@

An agent can detect harnesses, validate profiles and apply configuration without
interactive prompts. Preparation stops before starting engines or restarting
gateways; `--activate` performs that step. Agent setup uses the same profile
checks and configuration writer as the wizard. OpenClaw preparation records a
native plugin installation without restarting the gateway. Its startup service
checks bridge readiness before serving calls.
For TensorFold on one host or several Sparks, read [TensorFold setup](native-tensorfold.md).
It stages checked binaries, libraries, source and the profile on each rank.
Only commands that authenticate requests receive the signing key. Preparation
then stops before activation.
gateways; `--activate` performs that step. It uses the same checks as the wizard.
For TensorFold on one host or several Sparks, read
[TensorFold setup](native-tensorfold.md).
For a fresh deployment, use a [managed engine plan](engine-setup.md#agent-preparation-and-activation)
to prepare a supported engine, the profile and its deployment key automatically.

Expand Down Expand Up @@ -42,7 +37,7 @@ For a local terminal plan, prepare the harness and report the printed
an interactive step, not an unattended gateway restart.

The [starter profiles](../profiles/README.md) are discovered from local
metadata without a private login. Four compact profiles are included in full.
metadata. Four compact profiles are included in full.
DeepSeek's native manifest declares a 1.09 GB public payload; managed preparation
fetches missing bytes into an external cache and verifies them before staging.
The offline inspector never fetches a missing payload. For offline preparation,
Expand All @@ -53,8 +48,7 @@ must match the pinned target or a declared serving base. The native DeepSeek
starter supports the pinned vLLM container with official FP8 weights, or the
TensorFold 1.2.0 bundle with its declared EXL3 base and original Engram source.
Both require four SM120 GPUs and strength 1.0. Projection-only backends cannot
load this profile. Preserve the profile's experimental status and the public package's
qualification status when reporting setup results.
load this profile.

Choose a managed engine plan when the model needs an engine or profile setup.
Choose the existing-endpoint plan below when an engine already implements the
Expand Down Expand Up @@ -178,5 +172,5 @@ authorized, run only the selected commands. In Verified contacts mode, verify an
approved native message and an unapproved message through the harness. In Always
On, verify both ordinary and delegated requests use the configured profile and
that an attempted per-request stock override cannot bypass it. Mode changes do
not re-enable a disabled deployment. A successful setup check does not
establish native message routing, profile numerical parity or model quality.
not re-enable a disabled deployment. See [validation](validation.md) for
profile and runtime test coverage.
90 changes: 44 additions & 46 deletions docs/benchmark-results.md
Original file line number Diff line number Diff line change
@@ -1,18 +1,17 @@
# Measured results

These are small qualification runs from 30 October 2026. They show measured
behavior on the listed configurations, not a score for every supported model
or a maximum-throughput claim. [Run your own comparison](benchmarks.md) with the
included recorder and optional evaluation tools.
These are small qualification runs. Results apply to the listed
configurations; they are not full capability, refusal-rate or
maximum-throughput benchmarks. [Run your own comparison](benchmarks.md) with
the included recorder and optional evaluation tools.

The DeepSeek results below used the version 1.0 compact profile. Version 1.1
replaces that starter with the native Keys profile and a different engine path.
These earlier measurements do not measure the replacement profile.
**The DeepSeek measurements use the retired version 1.0 compact profile.**
The current native Keys starter uses a different artifact and engine path.

The [aggregate results](../benchmarks/results/native-qualification-20261030.json)
include exact model revisions, engine and profile hashes, settings recorded by
the checks, sample counts and timing values. Prompts, response logs and private
deployment configuration are not included in that file.
The [30 October 2026 aggregate results](../benchmarks/results/native-qualification-20261030.json)
include model revisions, engine and profile hashes, recorded settings, sample
counts and timing values. They omit prompts, response logs and private
deployment configuration.

## DeepSeek: the same tasks in both modes

Expand All @@ -33,18 +32,15 @@ prompts are the `linux-process` and `photography` cases in the
| Portrait photography checklist | 256 | 150.29 | 136.11 | 0.458 s | 0.445 s |
| Merge-intervals code | 230 | 278.74 | 258.40 | 0.399 s | 0.441 s |

Values are arithmetic means. Decode speed is the native engine's reported
per-request decode rate; it excludes prefill. TTFT is measured by the client
and includes request, queue and network time. These are sequential requests,
not a concurrent server-throughput test. Speed depends on the output and draft
acceptance. These measurements do not establish speed at the 64K preset or at
another GPU power limit.
Values are arithmetic means from sequential requests. Decode speed is the
engine's per-request rate, excluding prefill; TTFT includes client, queue and
network time. Output and draft acceptance affect speed. The 64K preset, other
power limits and concurrent throughput were not measured here.

The retained summary does not contain the complete generation-parameter
manifest for these speed runs. Treat them as qualification measurements rather
than a fully reproducible benchmark report. The code case completed in both
modes and produced the same output hash, but only two of its three generated
self-tests passed in either mode. Timing does not establish coding accuracy.
manifest, limiting reproducibility. The code case completed with the same
output hash in both modes, but only two of its three generated self-tests
passed in either mode.

## DeepSeek: 64K input and separate cache reuse

Expand All @@ -58,41 +54,45 @@ returned the expected retrieval answer on all six requests.
| First request in each mode | 15.585 s | 14.809 s | 0 |
| Two later requests per mode | 0.665–0.871 s | 0.609–0.653 s | 63,872 |

“First” means no prefix reused for that request. The model was already loaded;
this is not a cold model-start measurement. The requests alternated modes.
Each mode initially reported no cached prefix, then reused its own prefix.
The six-token answers make this a cache and retrieval check, not a useful
long-output decode benchmark or a general long-context reasoning score.
The model was already loaded. Requests alternated modes; each mode first
reported no cached prefix, then reused its own. Six-token answers make this a
cache and retrieval check, with no long-output decode or general reasoning score.

## Gemma: a small GSM8K pilot

TensorFold with Gemma 4 26B A4B, MLX-format 4-bit weights and DFlash active on one
Linux/NVIDIA DGX Spark scored **14/16 off** and **15/16 on**. This checkpoint format
does not imply use of Apple's MLX runtime. The test used the first 16 GSM8K test
questions, five examples per question and a 512-token output budget.
Linux/NVIDIA DGX Spark scored **14/16 off** and **15/16 on**. MLX describes the
checkpoint format; the runtime was TensorFold. The test used the first 16
GSM8K test questions, five examples per question and a 512-token output budget.

This fixed-prefix pilot does not establish a quality improvement. The
[Gemma result file](../benchmarks/results/gemma-native-gsm8k-pilot.json) records
The [Gemma result file](../benchmarks/results/gemma-native-gsm8k-pilot.json) records
the dataset, evaluator, model and engine revisions and extraction metrics.

## Qwen: runtime checks and a GSM8K pilot

Qwen3.8-27B EXL3 2.0-bpw passed 26 TensorFold checks on one SM120 GPU:
16 prose/code fixture requests, four structured-output/tool cases, and six
authorization or concurrent-request cases. DFlash2 stayed active, with FP8 KV,
two streams and a 4,096-token context. This is functional coverage, not a
capability benchmark or a throughput measurement.
two streams and a 4,096-token context.

A separate GSM8K pilot scored **31/32 off** and **31/32 on**, using the first
32 test questions, five-shot prompting, seed 20261030 and a 512-token output
budget. DFlash2 remained active. Both strict and flexible answer extraction
gave the same counts. The [pilot record](../benchmarks/results/qwen-native-gsm8k-pilot.json)
contains the exact model, engine, dataset and evaluator revisions. This small
fixed-prefix sample does not establish full-benchmark parity.
contains the exact model, engine, dataset and evaluator revisions.

Full refusal-rate, over-refusal and cross-model capability benchmarks are not
published yet. See [validation](validation.md) for the other model and engine
checks, including failures and limits.
The two GSM8K pilots use small, non-random samples and do not establish general
quality gains or parity. Full refusal-rate, over-refusal and cross-model
capability results are not published. See [validation](validation.md) for
other model and engine checks.

## Qwen SGLang: startup and TTFT

The BF16 Qwen3.8-27B SGLang 0.5.20 run used DFlash2 on one RTX PRO 6000
Blackwell Max-Q. Two individual warm streaming requests measured TTFT of
**0.113 seconds off** and **0.139 seconds on**. Startup took **295 seconds**
with first-time kernel compilation and **33 seconds** with those kernels cached.
These are individual samples from that deployment, not latency distributions.

## llama.cpp: short prose comparison

Expand All @@ -110,11 +110,9 @@ sequentially with no instrumented build or simultaneous generation.

The [llama.cpp result file](../benchmarks/results/llamacpp-qualification-20261030.json)
contains the original library-explanation prompt, generation settings and
individual measurements. The fixed output cap measures a 256-token prefix,
not completion of the requested explanation. Decode rate is reported by the
engine; TTFT is measured by the streaming client. GPU power limits were
observed after the run at 325 W; they were not sampled throughout it.
Thinking was off. Ngram produced no draft matches on this prose prompt; its
active drafting was verified separately with a copy fixture.
Two samples per mode do not establish a general performance difference,
maximum throughput or answer quality.
individual measurements. The 256-token cap cuts off the explanation. Decode
rate comes from the engine; TTFT comes from the streaming client. Thinking was
off. Ngram had no draft matches on this prompt; a separate copy fixture
confirmed active drafting. GPU power limits were observed at 325 W after the
run, with no continuous power record. Two samples per mode are insufficient
to establish a general performance difference.
Loading