What fits, and what does not

Everything on this page was measured, not estimated, and everything here can be reproduced with the commands the documentation gives. It exists because a buyer’s first questions are “does it run on my hardware” and “where is it slower” — and the honest answer is more useful than a brochure.

Model size against the card

Card	The largest model that runs	Memory	Loading
6 GB	3B in mx8	3.15 GB	13.5 s
6 GB	3B in mx16	does not fit (5.77 GB at the peak)	—
6 GB	8B in any format	does not fit — 7.7 GB are needed	—
16 GB	8B in mx8	~4 GB	to be measured on a rented card
24 GB	8B in mx16, 14B in mx8	~8 GB	to be measured on a rented card

What does not fit in one card is not a dead end: it is what the cluster is for. An 8B split across two machines runs at 6.8 tokens/s and gives the same bits as a single machine — the split costs speed, never correctness.

The two “to be measured” rows are honest about being open: the machine this was developed on has 6 GB, and the continuous integration runs on a virtual M1. The plan for those classes is written down in the repository.

Where we are not the fastest

Against	Distance	Why
PyTorch with CUDA, exact arithmetic, one card	1.70× faster	tensor cores cannot be used without changing the bits
cuBLAS int8	we are ahead (21.2 against 19.2 TOPS on the same card)	integer matrix products are exact by construction
Mojo / MAX on Apple Silicon	they reach the matrix units natively; we reach them with f16 arithmetic only	their compiler emits Metal directly; WebGPU does not expose the exact path
llama.cpp	comparable on the MX formats	different formats, both quantized

None of this is a rounding error, and none of it is hidden: the number that matters for a customer is whether the result can be defended, and that is the one we optimise.

Where another stack is the better choice

If the goal is the shortest training run, the answer is a CUDA stack with tensor cores: this engine deliberately gives up 15% to 40% of throughput (and much more in half precision) to keep every value identical on every device.

If you already run Mojo, PyTorch or a closed simulator and need to defend its number, the answer is the other way round: keep your stack and certify its output. The certify command recomputes the same program here and reports the difference in ULP — identical, or how many values and how far — in a receipt that can be signed.

What is not supported yet

	Status
AMD GPUs	not tested on real hardware; the conformance kit is ready to run on one
Windows	runs (DX12 through WARP in the integration); not tested on a discrete GPU
f16 and bf16 arithmetic	deliberately not used: they would break the identical bits between vendors
float64	not available on the GPU (the shader language has no float64)
Tensor units on Apple Silicon	reachable only with f16 arithmetic, which is a different contract
Large-model training	not the goal: training is for small models and fine-tuning, where the receipt matters
Model families	Llama, Qwen and Mistral-shaped decoders, and GGUF files of those

How to check any of this on your own machine

vision devices                        # what the machine offers
vision variants --device 0            # which paths this device takes, and why
vision conformance                    # 60 checks against the frozen signatures
vision certify result.npy --program heat.vis --device numpy

The conformance kit is the one that answers “will my hardware give the same bits”: it compares this machine with the reference signatures, and a difference is a finding, not a failure of the kit.