1 GPU on the selected test

Qwen3.8-Flash-Next-NVFP4-NVIDIA

GPUs are compared only on one exact test of this model. Change the test to see other conditions.

Test conditions

Every GPU below ran exactly these conditions.

Task: Text generation · Text → Text
Workload: 8K → 2K tokens
Concurrency: 1 request
Format · Engine: NVFP4 · strata
Unit: tok/s

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Protocol details

wbench-fixed-http-v1 · synthetic-tokenizer-v1:repeats=3 · 8192in-2048out

GPUs on this test

Only one GPU configuration has matching measurements; a cross-hardware comparison is not yet available.

RTX PRO 6000 Blackwell Server Edition · 96 GB
Mean throughput
135tok/s
Observed GPU occupancy
71.2GiB
First token
1.3 s
Reports combined
1
Runs, parameters and setup ↗

Sortable table

Hardware ranking · 1 rows
Hardware	Mean throughput (tok/s)	P95 (s)	TTFT (ms)	Observed occupancy (GiB)	GPU power (W)	Throughput / W	VRAM per GPU (GB)	Reports combined
#1RTX PRO 6000 Blackwell Server Edition · 96 GBRuns, parameters and setup ↗	134.63	19.867	1,305.67	71.22	280.01	0.48	96	1

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

How results are grouped

Results merge successful runs of the same model, precision and workload. CPU thread counts and CUDA wheel builds may differ; actual PyTorch builds are shown. Other recorded protocol settings remain separate.

Updated 11/5/2026, 11:07:29 PM