Benchmark table
Measured LLM decode speed, model × hardware × engine × quantization, with every row scored against the hardware's memory-bandwidth roofline. Serving runs under stated configs, run by us. Nothing third-party, nothing unreproduced.
45measured cells
9model architectures
3serving engines
125best single-stream tok/s · Qwen3.6-35B-A3B
2026-05-22to 2026-09-10
NVIDIA DGX Spark (GB10, Grace Blackwell)
- memory
- 122 GiB unified LPDDR5X, no separate VRAM, swap off
- spec bandwidth
- 273 GB/s
- STREAM copy
- 245.6 GB/s
- decode roofline divisor
- 187 GB/s
% roofline = measured ÷ (nodes × 187 ÷ active GB per token). 100% is the anchor: a clean single-stream, no-speculation run. Rows above 100% have speculation on or a better kernel path. Rows well below it are dequant-compute-bound or misconfigured. — means active bytes per token is not derived yet.
under 70%70 to 105%over 105%, speculation paying · click a header to sort · click a row for its notes
Single-stream decode concurrency 1
| model | engine | quant | spec | ctx | decode tok/s | roofline | % roofline | prefill tok/s | date |
|---|---|---|---|---|---|---|---|---|---|
| OpenAI gpt-oss-120bgpt-oss-120b MXFP4 GGUF | llama.cpp2026-05 build | MXFP4 | off | — | 40.6 | — | — | — | 2026-05-22 |
| 2026-05-22-gpt-oss-120b-mxfp4-llamacpp-c1 · c4 50.0 / c16 84.0 random prompts; c16 shared prefix 158.9 (~1.9x). | |||||||||
| Poolside Laguna-S 2.1 118B-A8BLaguna-S-2.1 Q4_K_M GGUF (90 GB — model card said 68) | llama.cppPoolside fork, branch laguna (2026-08) | Q4_K_M | off | — | 22.5 | 30.7 | — | 2026-08-19 | |
2026-08-19-laguna-s-q4km-llamacpp-c1 · Leaves ~22 GB free at -c 16384; cannot coexist with anything.-c 16384 | |||||||||
| Poolside Laguna-S 2.1 118B-A8BLaguna-S-2.1 Q4_K_M GGUF + DFlash draft | llama.cppPoolside fork, branch laguna (2026-08) | Q4_K_M | dflash | — | 27.3 | 30.7 | — | 2026-08-19 | |
2026-08-19-laguna-s-q4km-llamacpp-dflash-c1 · Best of a sweep: n-max 15 was WORSE than plain (20.9). DFlash draft-context init on the 90 GB target is I/O-bound, 5-7 min. llama.cpp's DFlash on GB10 is far less efficient than vLLM's.--spec-draft-n-max 6 | |||||||||
| Poolside Laguna-S 2.1 118B-A8Bpoolside/Laguna-S-2.1-NVFP4 @ 07614121b318 (15 shards, 71,916,714,120 B) + DFlash draft | vllm0.25.1 | NVFP4 | dflash | — | 50.6 | 41.2 | — | 2026-08-23 | |
2026-08-23-laguna-s-nvfp4-vllm-dflash-c1 · Median of 3 (48.2-53.5); 52.9 at util 0.85. DFlash mean acceptance length 4.5-6.8. Published single-stream for this model is 19-24 tok/s without DFlash. Pin the revision: the main branch of the same repo is 99.7 GB (experts of layers 40-47 left BF16).--gpu-memory-utilization 0.80, thinking off, code-generation prompt | |||||||||
| Poolside Laguna-S 2.1 118B-A8Bpoolside/Laguna-S-2.1-NVFP4 @ 07614121b318 (71,916,714,120 B), no DFlash | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | off | — | 18.3 | 41.2 | 2447@ 29010 | 2026-09-06 | |
2026-09-06-laguna-s-nvfp4-sglang-c1 · 44% of roofline, engine-independent (vLLM cell 19.6). SGLang's KV pool caps usable context at ~29.8k tokens at this fraction; 96k prefill failed x3. c4 32.9 aggregate / 14.0 per stream.--mem-fraction-static 0.65 | |||||||||
| Poolside Laguna-S 2.1 118B-A8Bpoolside/Laguna-S-2.1-NVFP4 @ 07614121b318, no DFlash | vllmv0.28.0 (upstream image) | NVFP4 | off | — | 19.6 | 41.2 | 2859@ 29010 | 2026-09-06 | |
2026-09-06-laguna-s-nvfp4-vllm-c1 · Engine-comparison cell. Prefill 3010.3 @4k / 2224.6 @96k (96k ran clean — no KV cap on vLLM at this fraction). Cold start 563 s. c4 41.7 aggregate / 15.7 per stream. Compare the 2026-08-23 row: DFlash takes this same checkpoint from ~19 to 50.6.--gpu-memory-utilization 0.65 | |||||||||
| MiniMax M2.7 230B-A10Bcyankiwi/MiniMax-M2.7-AWQ-4bit @ 70ed6a57 (130,495,823,656 B) | vllmv0.28.0 (upstream image) · 2× TP=2 over 200G RoCE | AWQ-int4-g32 + fp8 KV | off | — | 41.9 | 63.2 | 2486@ 28600 | 2026-09-07 | |
2026-09-07-minimax-m27-awq-vllm-tp2-c1 · Median of 3 on a 512-token coding prompt. Ships 3 MTP modules but vLLM 0.28.0 has no M2 MTP class, so this is plain decode. Two-box roofline 63.2 -> 66%; 1.33x the SINGLE-box roofline. Fabric < 1 Gb/s during decode: cross-node decode is allreduce-latency-bound, not bandwidth-bound. Prefill 3218 @3.6k / 1458 @85.9k. Cannot fit one box (121.5 GB weights on a 121.7 GiB machine). Two independent third-party 2x-Spark rigs published 39.4 and 38.3 on the same checkpoint.--gpu-memory-utilization 0.72, ctx 196608, fp8 KV, FlashInfer, CUDA graphs, PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True (load-bearing) | |||||||||
| Mistral Small 4 119B (2603)mistralai/Mistral-Small-4-119B-2603-NVFP4 @ b1a90485 (70,808,028,368 B), MLA + expert Linears NVFP4, rest BF16 | vllmv0.28.0 (upstream image) | NVFP4 | off | — | 31.6 | — | — | 2494@ 29010 | 2026-09-06 |
2026-09-06-mistral-small-4-nvfp4-vllm-c1 · Reviewer estimate 78-96% of roofline (no active-param figure derived, so no % here). Prefill 3328.3 @4k / 1247.3 @96k. c4 71.0 aggregate / 21.2 per stream. SGLang v0.5.18 loads this checkpoint then dies on a triton MLA kernel shape mismatch on GB10 — vLLM-only.--gpu-memory-utilization 0.65, --attention-backend TRITON_MLA | |||||||||
| NVIDIA Nemotron 3 Nano 30B-A3BNemotron-3-Nano-30B-A3B Q8_0 GGUF | llama.cpp2026-05 build | Q8_0 | off | — | 38.8 | — | — | — | 2026-05-22 |
| 2026-05-22-nemotron3-nano-q8-llamacpp-c1 · Early build; no active-GB derivation, so no roofline %. c4 48.1 / c16 71.3 random prompts; c16 with a shared prefix 87.8 (+23%) — llama.cpp's prompt cache already captures most of the prefix-reuse win. | |||||||||
| NVIDIA Nemotron 3.5 Lightning 30B-A3Bnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 @ cc84af2f (21,561,882,284 B), ModelOpt NVFP4 | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | — | 106.7 | 93.0 | 4713@ 29010 | 2026-09-06 | |
2026-09-06-nemotron35-nvfp4-sglang-nextn-c1 · No no-spec cell was run; backing out the ~1.2x speculation multiplier implies ~89 tok/s = 96% of roofline. NEXTN acceptance 0.65-0.85, 3.0-3.5 accepted. Prefill 4935.1 @4k / 3972.6 @96k. Cold start 426 s, MemAvailable after load 52.25 GiB. Coding eval 10/10 both modes (thinking-on costs 8.1x tokens). Bake-off winner on wall-per-task.--speculative-algorithm NEXTN (embedded head), --mem-fraction-static 0.50, quantization modelopt_fp4 | |||||||||
| NVIDIA Nemotron 3.5 Lightning 30B-A3Bnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 @ cc84af2f | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | 152505 | 105.3 | 93.0 | — | 2026-09-10 | |
| NVIDIA Nemotron 3.5 Lightning 30B-A3Bnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 @ cc84af2f | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | 1013 | 116.6 | 93.0 | — | 2026-09-10 | |
2026-09-10-nemotron35-nvfp4-sglang-nextn-ctx1k-c1 · Retention curve vs context: 1k 116.6 / 51k 119.0 / 152k 105.3 / 254k 99.7 / 399k 82.0. Only 6 of 52 layers are attention, which is why the curve is this flat. Raising the context limit cost no memory.--context-length 524288, --max-running-requests 16, >=1,100 output tokens forced | |||||||||
| NVIDIA Nemotron 3.5 Lightning 30B-A3Bnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 @ cc84af2f | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | 254145 | 99.7 | 93.0 | — | 2026-09-10 | |
| NVIDIA Nemotron 3.5 Lightning 30B-A3Bnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 @ cc84af2f | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | 399345 | 82.0 | 93.0 | — | 2026-09-10 | |
2026-09-10-nemotron35-nvfp4-sglang-nextn-ctx399k-c1 · Retrieval verified at 399,348 tokens. Cold prefill of the 400k prompt took 187 s. The falloff is accelerating (-14 pts 254k->400k vs -4 pts 152k->254k).--context-length 524288 | |||||||||
| Ornith-1.0-35BOrnith-1.0-35B, experts NVFP4 via llama-quantize RTN (data-free), attention Q8_0, no MTP head | llama.cpp2026-08 build | NVFP4-RTN-experts | off | — | 83.0 | 68.0 | — | 2026-08-08 | |
| 2026-08-08-ornith-nvfp4rtn-llamacpp-c1 · +48.7% over the same-day Q8_0 baseline (55.8) but coding eval fell 8/10 -> 5/10: REJECTED. Quant damage is per-model — the identical recipe was lossless on qwen3.6-heretic. | |||||||||
| Ornith-1.0-35BOrnith NVFP4 (AEON build, 23 GB) + DFlash drafter (905 MB) | vllm0.27.1+aeon.sm121a (community container) | NVFP4 | dflash | — | 95.3 | 68.0 | 5197@ 4096 | 2026-08-19 | |
2026-08-19-ornith-aeon-nvfp4-vllm-dflash-c1 · Reproduced the container author's decode claim (93.3) and DFlash 3.74 tok/step (45.6% acceptance). Did NOT reproduce their 9,661 prefill — 5,197 peak, decaying to 3,208 at 139k. Prefill 3.2x llama.cpp on the same prompts, same day. Active GB assumed equal to the NVFP4-experts GGUF footprint. --max-num-seqs > 16 with DFlash hard-crashes the box.--gpu-memory-utilization 0.6, --max-num-seqs 16, MARLIN weight-only kernel | |||||||||
| Ornith-1.0-35BOrnith-35B heretic APEX mixed-IQ quant (SC117), has MTP head | llama.cpp2026-08 build | APEX-mixed | draft-mtp | — | 70.8 | 78.1 | — | 2026-08-19 | |
| 2026-08-19-ornith-heretic-apex-llamacpp-c1 · THE DIAGNOSTIC CASE: smallest active footprint of the set yet slower than NVFP4 and 9% under its own size prediction even with speculation on — mixed IQ types are dequant-compute-bound, not bandwidth-bound. | |||||||||
| Ornith-1.0-35BOrnith-1.0-35B Q8_0 GGUF (no MTP head) | llama.cpp2026-08 build | Q8_0 | off | — | 58.5 | 58.5 | 1608@ 4138 | 2026-08-19 | |
| 2026-08-19-ornith-q8-llamacpp-c1 · THE ANCHOR: the only clean no-speculation cell on the box in Aug 2026; defines decode_bw_gbs = 187. llama-bench tg128 on the same model gives 59.1-59.6, so llama-server costs ~2%. | |||||||||
| Qwen3-Coder-Next 80B-A3BRedHatAI/Qwen3-Coder-Next-NVFP4 @ 27a8f16f (47,566,688,792 B), llm-compressor requant, no MTP head | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | off | — | 35.8 | 112.7 | 4750@ 29010 | 2026-09-06 | |
2026-09-06-qwen3-coder-next-nvfp4-sglang-c1 · THE UNDER-PERFORMER: 32% of roofline on a third-party requant (not ModelOpt/native). Dequant-compute-bound suspect; cause not dug. Prefill is fine (4812 @4k / 3480 @96k) — only decode is off. Coding eval 10/10.--mem-fraction-static 0.50 | |||||||||
| Qwen3.6-35B-A3BQwen3.6-35B-A3B heretic, experts NVFP4 via llama-quantize RTN (data-free), attention Q8_0 | llama.cpp2026-08 build | NVFP4-RTN-experts | draft-mtp | — | 80.5 | 68.0 | 1823@ 4138 | 2026-08-19 | |
| 2026-08-19-qwen36-heretic-nvfp4rtn-llamacpp-c1 · Quant was lossless on this model (coding eval 10/10 == Q8_0). Draft acceptance unchanged vs Q8_0 (51.3% vs 52.2%). | |||||||||
| Qwen3.6-35B-A3BQwen3.6-35B-A3B heretic (decensored) Q8_0 GGUF, native MTP head preserved | llama.cpp2026-08 build | Q8_0 | draft-mtp | — | 69.1 | 58.5 | — | 2026-08-19 | |
| 2026-08-19-qwen36-heretic-q8-llamacpp-c1 · Draft acceptance ~52%. Coding eval 10/10 (thinking off). | |||||||||
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e, repacked to GGUF (MTP head left BF16) | llama.cppv0.3.0 native | NVFP4 | off | — | 55.1 | 68.0 | 2263@ 29010 | 2026-09-04 | |
2026-09-04-qwen36-nvfp4-llamacpp-c1 · Prefill 2148.4 @4k / 1960.0 @96k. Cold start 6.7 s, MemAvailable after load 82.8 GiB. Same build in a container: within 1%, +8.1 s cold start.--jinja, -c CTX*NP | |||||||||
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e, repacked to GGUF (MTP head left BF16) | llama.cppv0.3.0 native | NVFP4 | draft-mtp | — | 89.0 | 68.0 | — | 2026-09-04 | |
2026-09-04-qwen36-nvfp4-llamacpp-mtp-c1 · 1.62x over no-spec at 85.9% acceptance. Costs 9-10% of prefill, ~4 GiB, +12 s cold start.--spec-type draft-mtp | |||||||||
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | off | — | 75.8 | 68.0 | 5792@ 29010 | 2026-09-04 | |
2026-09-04-qwen36-nvfp4-sglang-c1 · Prefill 6634.8 @4k / 3719.0 @96k — +17% over vLLM at 32k, a real engine difference; tie at 96k. Cold start 253.5 s, MemAvailable after load 51.06 GiB. Survived a reboot: boot-to-token 304 s.--mem-fraction-static 0.50, KV fp8 (engine default) | |||||||||
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | — | 123.2 | 68.0 | 5571@ 29010 | 2026-09-04 | |
2026-09-04-qwen36-nvfp4-sglang-nextn-c1 · 1.63x at 78.2% acceptance, 3.39/4 accepted. Costs 3.8-4.4% of prefill, ~1.2 GiB, +16 s cold start. All three engines land in the same 1.62-1.66x speculation band; higher acceptance did not mean a bigger speedup.--speculative-algorithm NEXTN (built-in MTP head) | |||||||||
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | vllmv0.28.0 (upstream image) | NVFP4 | off | — | 75.4 | 68.0 | 4967@ 29010 | 2026-09-04 | |
2026-09-04-qwen36-nvfp4-vllm-c1 · Prefill 5071.4 @4k / 3710.2 @96k. Cold start 325.9 s, MemAvailable after load 50.6 GiB. 111% of the llama.cpp-derived roofline = ~85% of STREAM copy. FLASHINFER_TRTLLM and FLASHINFER_CUTLASS FP4 MoE backends both refuse the device/scale scheme.KV bf16 (engine default), Marlin weight-only MoE (the only NVFP4 MoE path on GB10 in this version) | |||||||||
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | vllmv0.28.0 (upstream image) | NVFP4 + fp8 KV | off | — | 76.6 | 68.0 | 4740@ 29010 | 2026-09-04 | |
2026-09-04-qwen36-nvfp4-vllm-kvfp8-c1 · fp8 KV moved vLLM prefill the WRONG way (-4.6% @32k, -10.6% @96k), decode flat. So SGLang's prefill lead is not its fp8-KV default.--kv-cache-dtype fp8 | |||||||||
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | vllmv0.28.0 (upstream image) | NVFP4 | mtp | — | 124.9 | 68.0 | 4623@ 29010 | 2026-09-04 | |
2026-09-04-qwen36-nvfp4-vllm-mtp-c1 · 1.66x at 77.3% acceptance (4417/5715). Costs 7-8% of prefill and +102 s cold start.--speculative-config method=mtp | |||||||||
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | — | 122.2 | 68.0 | 5584@ 29010 | 2026-09-06 | |
2026-09-06-qwen36-nvfp4-sglang-nextn-c1 · Cross-phase continuity check: within 0.8% of the 2026-09-04 cell two days earlier. Prefill 6334.9 @4k / 3552.5 @96k. c4 257.3 aggregate / 66.9 per stream. Coding eval 9/10 off, 8/10 on.--speculative-algorithm NEXTN, --mem-fraction-static 0.50 | |||||||||
| Qwen3.6-35B-A3BQwen3.6-35B-A3B heretic Q8_0 GGUF, native MTP head preserved | llama.cppv0.3.0 native (llama-swap) | Q8_0 | draft-mtp | — | 68.1 | 58.5 | — | 2026-09-08 | |
2026-09-08-qwen36-heretic-q8-llamacpp-coresident-c1 · 86-token sample. Matches the solo 69.1 (2026-08-19): an idle co-tenant costs nothing on decode. MemAvailable with both resident 30.0-30.7 GiB. Cold load beside a resident model 87 s vs ~25-30 s into an empty box.-c 16384, thinking off; co-resident with SGLang serving nemotron35 at mem-fraction 0.35 | |||||||||
Fan-out decode concurrency > 1, aggregate tok/s
| model | engine | quant | spec | conc | aggregate tok/s | per stream | × single-stream roofline | date |
|---|---|---|---|---|---|---|---|---|
| OpenAI gpt-oss-120bgpt-oss-120b MXFP4 GGUF | llama.cpp2026-05 build | MXFP4 | off | 16 | 84.0 | — | — | 2026-05-22 |
| MiniMax M2.7 230B-A10Bcyankiwi/MiniMax-M2.7-AWQ-4bit @ 70ed6a57 | vllmv0.28.0 (upstream image) · 2× TP=2 over 200G RoCE | AWQ-int4-g32 + fp8 KV | off | 16 | 184.0 | 11.6 | 2026-09-07 | |
| 2026-09-07-minimax-m27-awq-vllm-tp2-c16 · Fan-out curve: c1 41.9 / c4 93 / c8 122 / c16 184. | ||||||||
| NVIDIA Nemotron 3 Nano 30B-A3BNemotron-3-Nano-30B-A3B Q8_0 GGUF | llama.cpp2026-05 build | Q8_0 | off | 16 | 71.3 | — | — | 2026-05-22 |
| NVIDIA Nemotron 3.5 Lightning 30B-A3Bnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 @ cc84af2f | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | 4 | 210.3 | 56.1 | 2026-09-06 | |
| NVIDIA Nemotron 3.5 Lightning 30B-A3Bnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 @ cc84af2f | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | 16 | 405.5 | 26.4 | 2026-09-07 | |
2026-09-07-nemotron35-nvfp4-sglang-nextn-c16 · TTFT 0.24 s. An identical uncapped run on the second box: c8 302 / c16 398.--max-running-requests 16 | ||||||||
| NVIDIA Nemotron 3.5 Lightning 30B-A3Bnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 @ cc84af2f | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | 8 | 290.0 | — | 2026-09-07 | |
2026-09-07-nemotron35-nvfp4-sglang-nextn-c8 · THE CONFIG CHANGE: raising max running requests 4 -> 16 took c16 from ~205 to 405 aggregate and TTFT from 15 s to 0.24 s, at -9% KV pool. 60-min c8 soak: 1,497 ok / 0 err.--max-running-requests 16 (was 4: capped aggregate at ~205 from c4 up, TTFT 4.7 s @c8) | ||||||||
| NVIDIA Nemotron 3.5 Lightning 30B-A3Bnvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-NVFP4 @ cc84af2f | sglangv0.5.18-cu130 · 2× DP=2 (independent replicas, client-side split) | NVFP4 + fp8 KV | nextn | 16 | 419.0 | — | 2026-09-07 | |
| 2026-09-07-nemotron35-nvfp4-sglang-nextn-dp2-c16 · DP=2 control arm: 418 @c8 / 419 @c16 — i.e. no gain over one uncapped box at c16 (405) because a single box already saturates at 16 streams; the win is that both boxes' streams are independent (no coupling). Beat TP=2 MiniMax-M2.7 2.3x aggregate / 4.7x per stream. | ||||||||
| Ornith-1.0-35BOrnith-1.0-35B, experts NVFP4 via llama-quantize RTN (data-free), attention Q8_0, no MTP head | llama.cpp2026-08 build | NVFP4-RTN-experts | off | 8 | 250.9 | — | 2026-08-08 | |
| Ornith-1.0-35BOrnith-1.0-35B Q8_0 GGUF (no MTP head) | llama.cpp2026-08 build | Q8_0 | off | 8 | 185.6 | — | 2026-08-08 | |
| 2026-08-08-ornith-q8-llamacpp-c8 · Same-day single-stream was 55.8 (vs 58.5 on 2026-08-19, the anchor). | ||||||||
| Qwen3-Coder-Next 80B-A3BRedHatAI/Qwen3-Coder-Next-NVFP4 @ 27a8f16f | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | off | 4 | 119.9 | 31.6 | 2026-09-06 | |
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e, repacked to GGUF | llama.cppv0.3.0 native | NVFP4 | off | 4 | 126.7 | 32.6 | 2026-09-04 | |
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | off | 4 | 189.2 | 48.1 | 2026-09-04 | |
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | sglangv0.5.18-cu130 | NVFP4 + fp8 KV | nextn | 4 | 256.8 | — | 2026-09-04 | |
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | vllmv0.28.0 (upstream image) | NVFP4 | off | 4 | 193.8 | 49.5 | 2026-09-04 | |
| Qwen3.6-35B-A3Bnvidia/Qwen3.6-35B-A3B-NVFP4 @ 491c2f1e | vllmv0.28.0 (upstream image) | NVFP4 | mtp | 4 | 285.5 | — | 2026-09-04 | |