What PXA does
Grouped by status: what you get in the current release, what 4.0 turns on by default, what you switch on yourself, and what is still being measured. Each item names its source: a ledger row, a message or a doc.
Ships today in v3.1
Default on. Download the release and you have these.
PXQN quants with measured quality classes
Every tier carries a measured KL class on Qwen3.8-27B: PXQN2 behaves like a classic 3-bit file at 2-bit size (9.6 GB, KL 0.161); PXQN4 is 6-bit class in 15.7 GB (KL 0.0079); PXQN5 matches Q6_K (0.0022 vs 0.0023) in 84% of the space.
Kernels written for sm_60, sm_61 and sm_70
P100, GTX 1080 Ti / P40 and V100 get their own decode, verify and attention paths, using the instructions these chips really have.
Settings measured per card and model
Split mode, batch sizes, context, flash attention, speculation and KV type come from one measured table. Every choice prints at boot as a opt-in preset line; a flag you type always wins.
MTP and n-gram speculation, chosen per model
A file with an MTP head uses it where it pays; others get the n-gram drafter. Sampled drafts are accepted with the lossless rule, so the output distribution is the model's own.
Tensor split
Two identical P100s or V100s (or four P100s) split every layer; the residual add, the two-card sum and the next norm run in one launch per card. Falls back to the layer split instead of stopping.
Expert cache for MoE files bigger than the card
Hot experts stay on the card, cold ones run from system RAM on a PXQN CPU matmul; the server profiles the routing once and can learn a better map per session. This is how a 32 GB Flash-Next file runs on one 16 GB card.
Flash-Next Overdrive on one P100
A measured preset: 26.91 to 30.70 t/s prose and 32.28 to 37.96 code (+14% / +18%).
Long context on one card
Checkpoint budgeting lets a 27B fit 131k context on one 16 GB card; a needle test at 125,000 tokens passes on one P100 and one V100.
PXA 4.0 optimizations
144.9 / 155.6 t/s prose / code on one V100 with the assistant drafter (-md).
Hot model swap
they wait in pinned host RAM and the request's model field picks which one is on the cards (Volta and newer).
Runs on old CPUs
No AVX2? The launchers pick a compatibility library on their own; card speed is unchanged.
One tarball, one container, one updater
The tarball carries its own CUDA 12.8 runtime; ghcr.io/poisonxa16/pxa:v3.1 carries the same binaries; pxa-update apply / rollback switches versions and keeps your models.
4.0: measured, gated, default on
On the integration branches now; every number is a ledger row on the Speed page.
Flash-Next 28k prefill, one V100
opt-in preset doubles the ubatch when experts stream over PCIe: 324.3 to 597.3 t/s at 28k (+84.1%). Det 12/12, needle 28k.
27B on two P100s: tensor split by default
38.00 / 38.04 vs 25.74 / 25.76 t/s on the layer split (+47.6%), identical output.
PXA 4.0 optimizations
Auto-picked: +7.4% / +5.3% MTP decode and +20.0% / +23.4% / +28.3% prompt reading at 8k / 16k / 30k over the layer split. Det 12/12, smoke, needle.
PXA 4.0 optimizations
2x V100 layer split: 63.31 / 74.37 vs 59.05 / 70.22 t/s (+7.2% / +5.9%), bit-identical.
PXA 4.0 optimizations
35B-A3B MTP at 138.11 t/s greedy on two P100s
V100 decode host and kernel family
PXA 4.0 optimizations
Opt-in levers
Measured wins you switch on yourself; each is documented with its switch and its number.
27B MTP preset for contexts up to 8k
Gated det 12/12, needle, smoke.
P100 one-card stack
25.61 / 25.62 to 27.77 / 27.83 t/s (+8.4% / +8.6%)
MoE long-prefill set (P100)
Det 12/12, needle 28k, KLD 0.0017.
Adaptive expert cache
opt-in preset: +4% prose / +8% code on Flash-Next one P100; off by default because a greedy answer then depends on earlier requests.
Weight streaming
opt-in preset keeps weights pinned in host RAM and streams them through a VRAM ring: a capacity lever for files that do not fit.
Self-selecting speculation
opt-in preset measures tokens per millisecond every round and picks no draft, the n-gram draft or the MTP head at a depth.
In progress
Measured candidates and open work. Nothing here is a promise.
V100 1x Overdrive recipe
one quiet rerun owed.
MTP head self-distillation
Retraining the 27B draft head on its own outputs to lift prose acceptance; generators run around the clock on two P100s.
GLM-5.3-Flash speed cell
No number yet: the RAM-heavy cell waits for a free P100 pair.
Windows, macOS and Android
Control builds for Windows x64, macOS (Intel) and Android (PWA + APK) are built and in testing; the Windows engine build is in progress.
PXQN tiers and what they are worth
Measured on Qwen3.8-27B: KL divergence of the quantized model's next-token probabilities from the original, on the assistant's tokens of a chat test set. Lower is closer.
| Tier | Size (27B) | Quality class | Drift (KL) | In plain words |
|---|---|---|---|---|
| PXQN1 | 6.5 GB | 1-bit class | 2.205 | Experimental, very lossy; for the biggest expert files. |
| PXQN2 | 9.6 GB | 3-bit class | 0.161 | A classic 3-bit file's quality at 2-bit size. |
| PXQN3 | 12.7 GB | 3.5-bit class | 0.035 | A good size and quality trade. |
| PXQN3bal | 13.5 GB | 4-bit class | 0.0238 | PXQN3 and PXQN4 mixed per tensor; fits one 16 GB card with room for context. |
| PXQN4 | 15.7 GB | 6-bit class | 0.0079 | As faithful as a classic 6-bit file (0.0076 at 18.8 GB). |
| PXQN5 | 18.8 GB | Q6_K class | 0.0022 | Matches Q6_K (0.0023) in 84% of the space. |
Source: the v3.1 README tier table. Old PXQ files keep loading.