32 GB · 33 reports

RTX 5090 · 32 GB

Qualified results first; original reports and full tables stay below.

Text generation · Qualified results

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

Qwen3.8-27B-NVFP4
RadixArk · 2 tested configurations
- NVFP4 · 32K → 2K tokens
  Output throughput · includes prefill · 1 request
  62.5tok/s
  Observed GPU occupancy
  27.3GiB
  First token
  3.6 s
  Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=2
  Workload
  32768in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 62.48 32.79 s 3.6 s 27.29 433.3 1 746bbc39 ↗
- 1 other test configurations on this GPU
  - NVFP4 · 8K → 2K tokens
    Output throughput · includes prefill · 1 request
    70.8tok/s
    Observed GPU occupancy
    27.3GiB
    First token
    633 ms
    Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=2
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 70.82 29.01 s 633 ms 27.29 405.5 1 bad294c0 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	62.48	32.79 s	3.6 s	27.29	433.3	1	746bbc39 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	70.82	29.01 s	633 ms	27.29	405.5	1	bad294c0 ↗

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Sortable tables by exact test condition · 2 rows

C: concurrency · B: batch · —: not measured

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v1	C1	70.82	70.82	633	29.01	27.29	405.5	70.82	0.17	1 ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v1	C1	62.48	62.48	3,553.5	32.792	27.29	433.3	62.48	0.14	1 ↗

Scaling curves and individual runs

One model and test protocol per curve; different workloads are never joined into one line.

RadixArk/Qwen3.8-27B-NVFP4 · NVFP4 · vllm · 8192in-2048out · wbench-fixed-http-v1

Mean throughput tok/s

One measured point; more levels are needed for a curve.

Mean run P95 latency s

One measured point; more levels are needed for a curve.

Mean GPU memory used GB

One measured point; more levels are needed for a curve.

Each point is the mean of recorded runs at that level. A source report may differ from the mean; all runs are listed below.

Uncached prefill is unavailable when prompts reuse cache or reliable phase timings are missing. Open a report for its recorded phase timings and cache counts.

Runs behind the curve · RadixArk/Qwen3.8-27B-NVFP4 · tok/s · 1 rows
Concurrency	Throughput	Decode (tok/s)	Uncached prefill (tok/s)	GPU power (W)	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	Test date	View report
1	70.82 tok/s	—	Uncached prefill unavailableView report	405.5	633	29.01	27.29	10/14/2026, 11:12:54 PM	bad294c0 ↗

Quality by dataset

Scores from different datasets and sample scopes are not directly comparable.

Quality by dataset · 10 rows
Model	Dataset	Format	Engine	Average score	Sample size	Runs
RadixArk/Qwen3.8-27B-NVFP4	aime2026-test-zero-shot-strict-v4-n30-mt16384-t0	NVFP4	vllm	76.67%	30 / 30 · Full dataset	1 runs 76.67% · 6a3bd68e ↗
RadixArk/Qwen3.8-27B-NVFP4	hle-test-self-judge-22ed307-v1-n5-mt16384-t0	NVFP4	vllm	20%	5 / 2500 · Subset only	1 runs 20% · af15c983 ↗
RadixArk/Qwen3.8-27B-NVFP4	gpqa-diamond-zero-shot-seed0-56686c0-v1-n198-mt16384-t0	NVFP4	vllm	70.2%	198 / 198 · Full dataset	1 runs 70.2% · 7e11f00c ↗
RadixArk/Qwen3.8-27B-NVFP4	gpqa-diamond-zero-shot-seed0-56686c0-v1-n3-mt16384-t0	NVFP4	vllm	66.67%	3 / 198 · Subset only	1 runs 66.67% · 933f4b5f ↗
RadixArk/Qwen3.8-27B-NVFP4	aime2026-test-zero-shot-strict-v3-n30-mt16384-t0	NVFP4	vllm	60%	30 / 30 · Full dataset	1 runs 60% · 28dfa67e ↗
RadixArk/Qwen3.8-27B-NVFP4	mmlu-pro-test-zero-shot-strict-v3-n3-mt16384-t0	NVFP4	vllm	66.67%	3 / 12032 · Subset only	1 runs 66.67% · cb193ebe ↗
RadixArk/Qwen3.8-27B-NVFP4	gsm8k-test-zero-shot-strict-v2-n3-mt4096-t0	NVFP4	vllm	100%	3 / 1319 · Subset only	1 runs 100% · 7c0c73ef ↗
RadixArk/Qwen3.8-27B-NVFP4	mmlu-pro-test-zero-shot-strict-v2-n3-mt4096-t0	NVFP4	vllm	0%	3 / 12032 · Subset only	1 runs 0% · 360177e6 ↗
RadixArk/Qwen3.8-27B-NVFP4	aime2026-test-zero-shot-strict-v2-n3-mt4096-t0	NVFP4	vllm	66.67%	3 / 30 · Subset only	1 runs 66.67% · c3de29e5 ↗
sambit9238/MiniCPM-V-4.7-35B-A3B-NVFP4	aime2026-test-zero-shot-strict-v4-n30-mt16384-t0	NVFP4	vllm	70%	30 / 30 · Full dataset	1 runs 70% · 787ba046 ↗

Measurements without a comparable protocol · 91 rows

These measured rates remain visible, but are excluded from comparison curves because the protocol is unknown.

Measurements without a comparable protocol · 91 rows
Model	Format	Engine	Workload	Concurrency	Throughput	GPU power (W)
Hunyuan-MT2-1.8B-FP8 ↗	FP8	vLLM	2K	1	352.71 tok/s	272.5
Hunyuan-MT2-1.8B-FP8 ↗	FP8	vLLM	2K	8	1,413.65 tok/s	212.31
Hunyuan-MT2-1.8B-FP8 ↗	FP8	vLLM	2K	32	3,085.27 tok/s	219.79
Hunyuan-MT2-1.8B-FP8 ↗	FP8	vLLM	2K	128	6,448.61 tok/s	228.61
Hunyuan-MT2-1.8B-FP8 ↗	FP8	vLLM	2K	1	328.21 tok/s	268.39
Hunyuan-MT2-1.8B-FP8 ↗	FP8	vLLM	2K	8	1,323.5 tok/s	211.84
Hunyuan-MT2-1.8B-FP8 ↗	FP8	vLLM	2K	32	3,336.45 tok/s	225.75
Hunyuan-MT2-1.8B-FP8 ↗	FP8	vLLM	2K	64	4,408.13 tok/s	232.94
Hunyuan-MT2-7B-FP8 ↗	FP8	vLLM	2K	1	146.47 tok/s	359.27
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1	556.11 tok/s	240.53
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	8	3,585.67 tok/s	319.08
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	32	8,669.18 tok/s	408.9
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	128	12,795.95 tok/s	494.21
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1	231.02 tok/s	186.59
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	8	1,599.75 tok/s	214.2
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	16	2,809.19 tok/s	247.26
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	32	4,168.46 tok/s	229.96
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	64	6,626.59 tok/s	256.69
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	128	9,674.65 tok/s	300.39
Qwen3-0.6B-Q8 ↗	Q8_0	vLLM	1K	1	817.27 tok/s	309.77
Qwen3-0.6B-Q8 ↗	Q8_0	vLLM	1K	2	1,269.2 tok/s	282.77
Qwen3-0.6B-Q8 ↗	Q8_0	vLLM	1K	4	1,760.95 tok/s	349.13
Qwen3-0.6B-Q8 ↗	Q8_0	vLLM	1K	8	2,564.52 tok/s	337.38
Qwen3-0.6B-Q8 ↗	Q8_0	vLLM	1K	16	2,628.18 tok/s	342.83
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	1	368.42 tok/s	305.09
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	8	2,512.57 tok/s	355.46
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	32	6,707.34 tok/s	430.2
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	128	10,639.28 tok/s	495.39
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	1	151.79 tok/s	233.1
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	8	1,079.26 tok/s	248.44
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	16	1,936.42 tok/s	271.51
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	32	3,010.79 tok/s	264.5
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	64	4,828.97 tok/s	288.73
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	128	6,878.18 tok/s	326.53
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	209.1 tok/s	337.08
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	1,470.27 tok/s	372.67
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	3,910.09 tok/s	446.95
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	6,679.67 tok/s	521.47
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	85.05 tok/s	262.33
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	623.99 tok/s	266.86
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	16	1,142.53 tok/s	291.69
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	1,910.41 tok/s	286.6
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	64	3,128.97 tok/s	307.24
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	5,047.76 tok/s	348.53
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	1	58.64 tok/s	301.11
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	8	401.88 tok/s	303.48
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	16	840.7 tok/s	321.44
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	32	1,427.58 tok/s	319.19
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	64	2,481.27 tok/s	333.75
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	128	2,977.9 tok/s	370.31
Qwen3-8B-NVFP4 ↗	NVFP4	vLLM	2K	1	213.52 tok/s	375.3
Qwen3-8B-NVFP4 ↗	NVFP4	vLLM	2K	8	1,442.46 tok/s	410.02
Qwen3-8B-NVFP4 ↗	NVFP4	vLLM	2K	32	3,892.02 tok/s	465.99
Qwen3-8B-NVFP4 ↗	NVFP4	vLLM	2K	128	7,565.13 tok/s	549.63
Qwen3-8B-NVFP4 ↗	NVFP4	vLLM	2K	1	215.04 tok/s	375.69
Qwen3-8B-NVFP4 ↗	NVFP4	vLLM	2K	8	1,462.34 tok/s	417.09
Qwen3-8B-NVFP4 ↗	NVFP4	vLLM	2K	32	3,900.83 tok/s	477.71
Qwen3-8B-NVFP4 ↗	NVFP4	vLLM	2K	64	5,678.73 tok/s	530.75
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	1	42.18 tok/s	352.01
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	8	263.17 tok/s	348.33
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	16	518.13 tok/s	358.4
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	32	901.48 tok/s	355.96
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	64	1,439.85 tok/s	370.38
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	128	2,290.9 tok/s	419.12
Qwen3-14B-NVFP4 ↗	NVFP4	vLLM	2K	1	140.9 tok/s	409.67
Qwen3-14B-NVFP4 ↗	NVFP4	vLLM	2K	8	917.66 tok/s	435.52
Qwen3-14B-NVFP4 ↗	NVFP4	vLLM	2K	32	2,690.61 tok/s	480.12
Qwen3-14B-NVFP4 ↗	NVFP4	vLLM	2K	128	5,684.8 tok/s	547.99
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	1	149.22 tok/s	230.23
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	8	940.47 tok/s	242.87
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	16	1,732.72 tok/s	267.19
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	32	2,666.66 tok/s	254.26
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	64	4,246.88 tok/s	282.59
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	128	6,170.24 tok/s	320.3
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	1	83.2 tok/s	256.49
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	8	583.88 tok/s	261.5
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	16	947.58 tok/s	278.07
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	32	1,609.89 tok/s	273.51
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	64	2,807.56 tok/s	293.93
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	128	4,281.57 tok/s	337.74
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	1	57.98 tok/s	296.09
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	8	368.23 tok/s	297.25
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	16	675.08 tok/s	312.39
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	32	1,189.46 tok/s	310.39
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	64	2,072.3 tok/s	325.38
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	128	3,221.98 tok/s	362.69
qwen3:0.6b ↗	Ollama	vLLM	2K	1	353.27 tok/s	191.38
qwen3:0.6b ↗	Ollama	vLLM	2K	2	561.67 tok/s	266.04
qwen3:0.6b ↗	Ollama	vLLM	2K	4	571.2 tok/s	273.27
qwen3:0.6b ↗	Ollama	vLLM	2K	8	593.3 tok/s	285.74
qwen3:0.6b ↗	Ollama	vLLM	2K	16	572.22 tok/s	276.98

Original reports

Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.

Original reports · 31 rows
Model	Test task	Format	Engine	Workload	Protocol	Quality score	Test date
Qwen3-0.6B-Q8 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	5/23/2026, 8:34:45 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	5/23/2026, 6:29:46 PM
Hunyuan-MT2-1.8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	5/23/2026, 6:17:06 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	5/23/2026, 6:07:36 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	5/23/2026, 5:58:21 PM
Qwen3-14B-NVFP4 ↗	Text generation	NVFP4	vLLM	2K	Method unknown	—	5/23/2026, 5:50:20 PM
Qwen3-8B-NVFP4 ↗	Text generation	NVFP4	vLLM	2K	Method unknown	—	5/23/2026, 5:31:13 PM
qwen3:0.6b ↗	Text generation	Ollama	vLLM	2K	Method unknown	—	5/23/2026, 9:59:11 AM
Hunyuan-MT2-7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	5/23/2026, 8:08:15 AM
Qwen3-8B-NVFP4 ↗	Text generation	NVFP4	vLLM	2K	Method unknown	—	5/23/2026, 7:52:06 AM
Hunyuan-MT2-1.8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	5/23/2026, 7:16:29 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 12:24:37 PM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 12:17:18 PM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 12:11:17 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 12:05:21 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 12:00:12 PM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 10:34:38 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 10:21:26 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 10:09:11 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	8192in-2048out	wbench-fixed-http-v1	—	10/14/2026, 11:12:54 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	32768in-2048out	wbench-fixed-http-v1	—	10/14/2026, 11:14:30 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	aime2026-test-zero-shot-strict-v2-n3-mt4096-t0	quality-v1	66.67%	10/14/2026, 11:16:34 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	aime2026-test-zero-shot-strict-v3-n30-mt16384-t0	quality-v1	60%	10/15/2026, 6:11:39 AM
MiniCPM-V-4.7-35B-A3B-NVFP4 ↗	Text generation	NVFP4	vllm	aime2026-test-zero-shot-strict-v4-n30-mt16384-t0	quality-v1	70%	10/16/2026, 7:41:21 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	aime2026-test-zero-shot-strict-v4-n30-mt16384-t0	quality-v1	76.67%	10/16/2026, 2:11:58 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	gpqa-diamond-zero-shot-seed0-56686c0-v1-n3-mt16384-t0	quality-v1	66.67%	10/15/2026, 3:55:03 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	gpqa-diamond-zero-shot-seed0-56686c0-v1-n198-mt16384-t0	quality-v1	70.2%	10/15/2026, 9:47:06 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	gsm8k-test-zero-shot-strict-v2-n3-mt4096-t0	quality-v1	100%	10/14/2026, 11:18:43 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	hle-test-self-judge-22ed307-v1-n5-mt16384-t0	quality-v1	20%	10/15/2026, 10:05:56 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	mmlu-pro-test-zero-shot-strict-v2-n3-mt4096-t0	quality-v1	0%	10/14/2026, 11:18:21 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	mmlu-pro-test-zero-shot-strict-v3-n3-mt16384-t0	quality-v1	66.67%	10/15/2026, 5:03:15 AM

Running benchmarks

0Updates automatically

No benchmarks are currently reported as running.

Methodology

Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.

Updated 11/5/2026, 11:53:04 PM