Models

Does my model run, and how fast? Pick your cards, or read the best measured number for each family.

What can I run?

updated 2026-10-10

Card

How many

1Qwen3.8 27BPXQN2free file

uses 1 of your 1

decode 47.38 t/s code · 40.66 prose · prefill 238.2 t/s (4k prompt) · Single card · MTP

4.0 pre-release, 1× P100 16 GB, PXQN4: decode 26.42 t/s (code) · prefill 248.2 t/s (4k prompt, opt-in setting) #8514 #7438

bench site · PXA v3 · measured 2026-10-104.0 line: our ledger

2Qwen3.8 Flash-NextPXQN 32 GBfree file

uses 1 of your 1

decode 34.95 t/s code · 30.31 prose · prefill 321.4 t/s (4k prompt) · One card + system RAM · Overdrive

bench site · PXA v3.1 Overdrive · measured 2026-10-09

Not in the list: no measured row for that card yet. GLM-5.3-Flash is coming.

Best first. Speeds from the bench site's verified rows (PXA v3, refreshed when the site has newer ones) and our 4.0 ledger rows; each line names its source.

Model families

Qwen3.8 27B

PXQN4PXQN2

Dense model with its own MTP head; the one-card file fits a 16 GB card.

Best decode89.04 t/s1× V100, code #8444

Best prefill1,133 t/s1× V100, 4k prompt #7396

Measured on: 1× P100 · 1× V100 · 2× P100

27B one-card file · Swift 1.5

35B-A3B MoE

PXQN4PXQ4

Qwen3.6 35B-A3B and Ornith 1.5 with MTP; new in 4.0, 3.1 cannot run it.

Best decode138.1 t/s2× P100, greedy #7409

Best prefill1,953 t/s2× P100, 4k prompt #7499

Measured on: 2× P100

Ornith 1.5 · PXA Coder

Qwen3.8 Flash-Next

PXQN2

A very large MoE; the 32 GB file runs on one card plus system RAM.

Best decode74.37 t/s2× V100, code #7443

Best prefill777.5 t/s1× V100, 8k prompt #7400

Measured on: 1× V100 · 2× P100 · 2× V100

Swift 1.5, 32 GB

Gemma 4 26B-A4B

PXQN4

Mixture of experts, 4B active per token; contexts up to 64k on one 16 GB card.

Best decode118.0 t/s1× V100, prose #8623

Best prefill1,892 t/s1× V100, 4k prompt #8608

Measured on: 1× V100

All files

Nemotron

PXQN4

Nemotron, Nemotron-H and Nemotron-H MoE load and run; new in 4.0.

Best decode196.9 t/sNemotron 3 Nano 4B, 1× V100, 128-token answer #8129

Best prefill6,937 t/sNemotron 4 Mini 4B, 1× V100, 512-token prompt #8134

Measured on: 1× V100

All files

GLM-5.3-Flash

PXQN2coming

The file is ready; its 4.0 speed is the next measurement.

No 4.0 speed yet

Measured on: not yet

All files

What the engine does for every model

PXQN files with measured qualityEach tier carries a quality class measured against the original model.

Written for sm_60, sm_61 and sm_70P100, GTX 1080 Ti and P40, and V100 each get their own fast paths.

Settings measured per card and modelControl picks them from a measured table; a flag you type always wins.

Speculative decoding per modelMTP where the file has a head, n-gram drafts elsewhere; sampling keeps the model's own distribution.

Two or four cards as oneIdentical P100 or V100 cards share every layer, with a safe fallback.

MoE files bigger than the cardExperts that do not fit stay in system RAM; the busy ones stay on the card.

Long prompts on one 16 GB cardGemma 4 starts every context up to 64k on one 16 GB V100 or P100.

Hot model swapChange the model without restarting the server.

Runs on old CPUsAny x86-64 CPU; no AVX2 needed.

One tarball, one container, one updaterUpdate with one command, and roll back if you need to.