96 GB · 196 reports

RTX PRO 6000 Blackwell Server Edition · 96 GB

Qualified results first; original reports and full tables stay below.

Text generation · Qualified results

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

WeVisOne-VL-2B

tencent · 6 tested configurations

BF16 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 71tok/s
Best measured · 4 concurrent requests: 165tok/s
Observed GPU occupancy: 85.3GiB
First token: 268 ms

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	71.05	28.83 s	268 ms	85.25	356.6	1	1ff0c98a ↗
C2	89.43	45.82 s	323 ms	85.25	399.6	1	1ff0c98a ↗
C3	135.43	45.41 s	379 ms	85.25	470.9	1	1ff0c98a ↗
C4	165.29	49.61 s	420 ms	85.25	524	1	1ff0c98a ↗

5 other test configurations on this GPU

BF16 · 128K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 56.1tok/s
Best measured · 4 concurrent requests: 66.8tok/s
Observed GPU occupancy: 85.3GiB
First token: 8 s

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	56.09	36.63 s	8 s	85.25	418.9	1	2c661c7e ↗
C2	62.52	67.49 s	12 s	85.25	444.3	1	2c661c7e ↗
C3	63.59	109.31 s	13.5 s	85.25	446.8	1	2c661c7e ↗
C4	66.75	144.63 s	15 s	85.25	461.6	1	2c661c7e ↗

BF16 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 114tok/s
Best measured · 9 concurrent requests: 433tok/s
Observed GPU occupancy: 85.3GiB
First token: 135 ms

Use this recipe · 9 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 7 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	113.51	18.06 s	135 ms	85.25	341.3	1	5fee1315 ↗
C2	153.3	26.73 s	161 ms	85.25	369.5	1	5fee1315 ↗
C4	281.86	29.1 s	202 ms	85.25	483.8	1	5fee1315 ↗
C6	293.74	41.93 s	214 ms	85.25	459.2	1	5fee1315 ↗
C7	340.1	42.27 s	227 ms	85.25	496.1	1	5fee1315 ↗
C8	387.93	42.41 s	237 ms	85.25	534.9	1	5fee1315 ↗
C9	433.07	42.78 s	246 ms	85.25	575.8	1	5fee1315 ↗

BF16 · 64K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 101tok/s
Best measured · 8 concurrent requests: 151tok/s
Observed GPU occupancy: 85.3GiB
First token: 2.3 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 5 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	101.26	20.26 s	2.3 s	85.25	369.1	1	c5c9cf6f ↗
C2	123.11	33.41 s	3.6 s	85.25	404.8	1	c5c9cf6f ↗
C4	140.19	64.26 s	4.5 s	85.25	430	1	c5c9cf6f ↗
C6	142.08	97.79 s	5.3 s	85.25	430.4	1	c5c9cf6f ↗
C8	150.73	122.37 s	6.3 s	85.25	446.7	1	c5c9cf6f ↗

BF16 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 162tok/s
Best measured · 18 concurrent requests: 857tok/s
Observed GPU occupancy: 85.3GiB
First token: 71 ms

Use this recipe · 18 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 7 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	161.57	12.72 s	71 ms	85.25	327.8	1	75198c63 ↗
C2	269.43	15.21 s	86 ms	85.25	349.1	1	75198c63 ↗
C4	460.41	17.83 s	106 ms	85.25	427.3	1	75198c63 ↗
C8	644.45	25.52 s	127 ms	85.25	486.9	1	75198c63 ↗
C16	771.04	42.77 s	184 ms	85.25	533.9	1	75198c63 ↗
C17	812.79	43.14 s	195 ms	85.25	550.4	1	75198c63 ↗
C18	856.64	43.41 s	196 ms	85.25	568.9	1	75198c63 ↗

BF16 · 32K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 153tok/s
Best measured · 16 concurrent requests: 319tok/s
Observed GPU occupancy: 85.3GiB
First token: 753 ms

Use this recipe · 16 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	153.44	13.39 s	753 ms	85.25	343.4	1	d2914dd8 ↗
C2	208.65	19.64 s	1.1 s	85.25	360.4	1	d2914dd8 ↗
C4	259.19	33.21 s	1.6 s	85.25	398.9	1	d2914dd8 ↗
C8	295.37	59.5 s	2.1 s	85.25	422	1	d2914dd8 ↗
C12	311.33	86.14 s	2.7 s	85.25	432.4	1	d2914dd8 ↗
C16	319.17	112.4 s	3.3 s	85.25	438	1	d2914dd8 ↗

Qwen3.8-27B-NVFP4

unsloth · 11 tested configurations

NVFP4 · 128K → 2K tokens · No cache reuse
Output throughput · includes prefill · 1 request
31.5tok/s
Best measured · 2 concurrent requests
41.3tok/s
Observed GPU occupancy
87.4GiB
First token
24.9 s
Use this recipe · 2 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 concurrency levels
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
vllm · vLLM 0.28.0
Protocol
wbench-search-http-v1
Dataset
migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload
131072in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
C1 31.54 64.93 s 24.9 s 87.37 402.7 1 73fcb7c1 ↗
C2 41.27 111.14 s 34 s 87.37 426.3 1 73fcb7c1 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	31.54	64.93 s	24.9 s	87.37	402.7	1	73fcb7c1 ↗
C2	41.27	111.14 s	34 s	87.37	426.3	1	73fcb7c1 ↗

10 other test configurations on this GPU

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 54.7tok/s
Best measured · 8 concurrent requests: 340tok/s
Observed GPU occupancy: 87.4GiB
First token: 329 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	54.72	37.56 s	329 ms	87.37	322.4	1	d3f6b770 ↗
C2	99.95	40.99 s	522 ms	87.37	315.9	1	d3f6b770 ↗
C4	189.07	43.34 s	971 ms	87.37	337.8	1	d3f6b770 ↗
C6	264.7	46.44 s	1.3 s	87.37	364	1	d3f6b770 ↗
C7	303.08	47.32 s	1.4 s	87.37	373.4	1	d3f6b770 ↗
C8	339.61	48.27 s	1.7 s	87.37	381.2	1	d3f6b770 ↗

NVFP4 · 64K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 44.9tok/s
Best measured · 4 concurrent requests: 96.8tok/s
Observed GPU occupancy: 87.4GiB
First token: 8.6 s

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	44.86	45.67 s	8.6 s	87.37	360.5	1	d2ff32ec ↗
C2	67.59	63.21 s	12.7 s	87.37	379.1	1	d2ff32ec ↗
C3	84.95	84.07 s	14.6 s	87.37	398.5	1	d2ff32ec ↗
C4	96.77	105.62 s	16.3 s	87.37	415.8	1	d2ff32ec ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57tok/s
Best measured · 8 concurrent requests: 374tok/s
Observed GPU occupancy: 87.4GiB
First token: 222 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	56.96	36.17 s	222 ms	87.37	322.6	1	bced0642 ↗
C2	104.14	39.34 s	355 ms	87.37	305.9	1	bced0642 ↗
C4	198.42	41.3 s	704 ms	87.37	319.3	1	bced0642 ↗
C6	291.12	42.23 s	1 s	87.37	342.1	1	bced0642 ↗
C7	333.32	43.04 s	1.1 s	87.37	349.1	1	bced0642 ↗
C8	374.04	43.82 s	1.3 s	87.37	353.2	1	bced0642 ↗

NVFP4 · 32K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 52.8tok/s
Best measured · 8 concurrent requests: 215tok/s
Observed GPU occupancy: 87.4GiB
First token: 3.3 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.78	38.8 s	3.3 s	87.37	337.3	1	aee8b33e ↗
C2	87.2	48.71 s	4.9 s	87.37	343.4	1	aee8b33e ↗
C4	142.14	64.91 s	7 s	87.37	375.3	1	aee8b33e ↗
C6	187.69	79.85 s	8.1 s	87.37	402.8	1	aee8b33e ↗
C7	202.31	85.21 s	8.7 s	87.37	411.8	1	aee8b33e ↗
C8	215.05	94.52 s	9.3 s	87.37	421.4	1	aee8b33e ↗

NVFP4 · 128K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 31.5tok/s
Best measured · 8 concurrent requests: 57.7tok/s
Observed GPU occupancy: 86.6GiB
First token: 24.9 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=isolated:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	31.45	65.12 s	24.9 s	86.63	415.7	1	448b0777 ↗
C4	50.67	220.6 s	45.8 s	86.63	469.4	1	448b0777 ↗
C8	57.74	421.65 s	61.9 s	86.63	494.5	1	448b0777 ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 54.5tok/s
Best measured · 8 concurrent requests: 339tok/s
Observed GPU occupancy: 86.6GiB
First token: 359 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	54.54	37.55 s	359 ms	86.63	324.9	1	7d1a7c8e ↗
C4	188.48	43.48 s	878 ms	86.63	350.2	1	7d1a7c8e ↗
C8	338.65	48.43 s	1.6 s	86.63	396.9	1	7d1a7c8e ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.1tok/s
Best measured · 8 concurrent requests: 373tok/s
Observed GPU occupancy: 86.6GiB
First token: 231 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.09	35.88 s	231 ms	86.63	319	1	37f29252 ↗
C4	197.8	41.44 s	650 ms	86.63	330.2	1	37f29252 ↗
C8	372.88	43.98 s	1.3 s	86.63	367.3	1	37f29252 ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 50tok/s
Best measured · 8 concurrent requests: 287tok/s
Observed GPU occupancy: 86.4GiB
First token: 506 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	49.98	40.98 s	506 ms	86.36	322.7	1	68d1f2c6 ↗
C4	171.33	47.85 s	1 s	86.36	365.5	1	68d1f2c6 ↗
C8	287.31	57.12 s	1.9 s	86.36	425.4	1	68d1f2c6 ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 54tok/s
Best measured · 8 concurrent requests: 337tok/s
Observed GPU occupancy: 86.4GiB
First token: 361 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	54.04	38.07 s	361 ms	86.36	320.7	1	417f69be ↗
C4	187.12	43.81 s	879 ms	86.36	333.2	1	417f69be ↗
C8	336.95	48.66 s	1.6 s	86.36	379.7	1	417f69be ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 56.4tok/s
Best measured · 8 concurrent requests: 371tok/s
Observed GPU occupancy: 86.4GiB
First token: 244 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	56.4	36.51 s	244 ms	86.35	319.1	1	791b887d ↗
C4	196.45	41.73 s	654 ms	86.35	315.5	1	791b887d ↗
C8	370.68	44.24 s	1.3 s	86.36	348.1	1	791b887d ↗

Qwen3.8-27B-NVFP4

RadixArk · 12 tested configurations

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 53tok/s
Best measured · 4 concurrent requests: 181tok/s
Observed GPU occupancy: 86GiB
First token: 435 ms

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.97	38.72 s	435 ms	85.99	333.9	1	78fc908c ↗
C2	97.25	42.14 s	646 ms	85.99	340.8	1	78fc908c ↗
C3	138.66	44.36 s	822 ms	85.99	358.8	1	78fc908c ↗
C4	181.16	45.25 s	1.1 s	85.99	375.3	1	78fc908c ↗

11 other test configurations on this GPU

NVFP4 · 128K → 2K tokens · No cache reuse
Output throughput · includes prefill · 1 request
32.7tok/s
Best measured · 2 concurrent requests
42.5tok/s
Observed GPU occupancy
86GiB
First token
24.3 s
Use this recipe · 2 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 concurrency levels
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
vllm · vLLM 0.28.0
Protocol
wbench-search-http-v1
Dataset
migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload
131072in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
C1 32.72 62.59 s 24.3 s 85.99 406.5 1 28f51cf1 ↗
C2 42.54 106.24 s 33.5 s 85.99 428.9 1 28f51cf1 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	32.72	62.59 s	24.3 s	85.99	406.5	1	28f51cf1 ↗
C2	42.54	106.24 s	33.5 s	85.99	428.9	1	28f51cf1 ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.5tok/s
Best measured · 8 concurrent requests: 355tok/s
Observed GPU occupancy: 86GiB
First token: 326 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.45	35.8 s	326 ms	85.98	326.1	1	67a5e6d3 ↗
C2	104.7	39.14 s	515 ms	85.98	321	1	67a5e6d3 ↗
C4	198.72	41.24 s	954 ms	85.98	344.2	1	67a5e6d3 ↗
C6	277.65	44.28 s	1.3 s	85.98	358.8	1	67a5e6d3 ↗
C7	317.41	45.18 s	1.4 s	85.98	369.2	1	67a5e6d3 ↗
C8	354.95	46.19 s	1.6 s	85.98	377.1	1	67a5e6d3 ↗

NVFP4 · 64K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 47.1tok/s
Best measured · 4 concurrent requests: 101tok/s
Observed GPU occupancy: 86GiB
First token: 8.4 s

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	47.15	43.45 s	8.4 s	85.98	364.4	1	e5a201fa ↗
C2	70.39	60.71 s	12.3 s	85.98	381.3	1	e5a201fa ↗
C3	88.36	80.85 s	14.2 s	85.98	403.6	1	e5a201fa ↗
C4	100.66	101.65 s	15.8 s	85.98	419.2	1	e5a201fa ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 60.1tok/s
Best measured · 8 concurrent requests: 393tok/s
Observed GPU occupancy: 86GiB
First token: 217 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	60.1	34.34 s	217 ms	85.98	323	1	9cdf45b7 ↗
C2	109.3	37.48 s	349 ms	85.98	310.8	1	9cdf45b7 ↗
C4	209.23	39.18 s	686 ms	85.98	324.8	1	9cdf45b7 ↗
C6	306.94	40.06 s	992 ms	85.98	336	1	9cdf45b7 ↗
C7	351.24	40.83 s	1.1 s	85.98	342.6	1	9cdf45b7 ↗
C8	392.93	41.71 s	1.3 s	85.98	348.2	1	9cdf45b7 ↗

NVFP4 · 32K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 55.6tok/s
Best measured · 8 concurrent requests: 225tok/s
Observed GPU occupancy: 86GiB
First token: 3.2 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	55.59	36.91 s	3.2 s	85.98	341	1	dca51ce5 ↗
C2	91.37	46.48 s	4.6 s	85.98	342.9	1	dca51ce5 ↗
C4	149.21	62.63 s	6.5 s	85.98	372.9	1	dca51ce5 ↗
C6	196.73	76.24 s	7.8 s	85.98	398.9	1	dca51ce5 ↗
C7	211.87	81.41 s	8.3 s	85.98	408.7	1	dca51ce5 ↗
C8	224.61	90.5 s	8.9 s	85.98	414.8	1	dca51ce5 ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 52.9tok/s
Best measured · 8 concurrent requests: 301tok/s
Observed GPU occupancy: 86.9GiB
First token: 508 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.9	38.72 s	508 ms	86.86	338	1	7cea351c ↗
C4	179.81	45.65 s	1 s	86.86	387.5	1	7cea351c ↗
C8	301.16	54.48 s	1.9 s	86.86	443.7	1	7cea351c ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.7tok/s
Best measured · 8 concurrent requests: 356tok/s
Observed GPU occupancy: 86.9GiB
First token: 357 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.67	35.51 s	357 ms	86.86	325.8	1	fe8141d2 ↗
C4	197.42	41.52 s	870 ms	86.86	354.3	1	fe8141d2 ↗
C8	356.25	46.05 s	1.6 s	86.86	392.4	1	fe8141d2 ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 60.5tok/s
Best measured · 8 concurrent requests: 395tok/s
Observed GPU occupancy: 86.9GiB
First token: 228 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	60.47	33.87 s	228 ms	86.86	320.3	1	cbf1431c ↗
C4	207.86	39.44 s	636 ms	86.86	333.6	1	cbf1431c ↗
C8	394.52	41.6 s	1.3 s	86.86	361.6	1	cbf1431c ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 52.6tok/s
Best measured · 8 concurrent requests: 301tok/s
Observed GPU occupancy: 86.8GiB
First token: 509 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.59	38.95 s	509 ms	86.83	328.8	1	d14962c1 ↗
C4	179.42	45.72 s	1.1 s	86.83	371.9	1	d14962c1 ↗
C8	300.98	54.52 s	1.9 s	86.83	426	1	d14962c1 ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.1tok/s
Best measured · 8 concurrent requests: 356tok/s
Observed GPU occupancy: 86.8GiB
First token: 358 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.08	36.15 s	358 ms	86.82	326	1	275a096d ↗
C4	196.91	41.63 s	870 ms	86.83	338.7	1	275a096d ↗
C8	355.89	46.08 s	1.6 s	86.83	375.7	1	275a096d ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 59.7tok/s
Best measured · 8 concurrent requests: 394tok/s
Observed GPU occupancy: 86.8GiB
First token: 230 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	59.68	34.59 s	230 ms	86.82	323.3	1	83d37469 ↗
C4	207.31	39.54 s	664 ms	86.82	320	1	83d37469 ↗
C8	394.04	41.6 s	1.2 s	86.82	346.1	1	83d37469 ↗

Qwen3-4B-AWQ
Qwen · 2 tested configurations
- AWQ · 8K → 2K tokens · Prefix cache reuse
  Output throughput · includes prefill · 1 request
  223tok/s
  Best measured · 4 concurrent requests
  742tok/s
  Observed GPU occupancy
  86.3GiB
  First token
  28 ms
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 222.74 9.19 s 28 ms 86.28 288.8 1 adbfef8a ↗
  C4 741.85 11.05 s 35 ms 86.28 335 1 adbfef8a ↗
- 1 other test configurations on this GPU
  - AWQ · 8K → 2K tokens · No cache reuse
    Output throughput · includes prefill · 1 request
    215tok/s
    Best measured · 4 concurrent requests
    517tok/s
    Observed GPU occupancy
    86.3GiB
    First token
    298 ms
    Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 concurrency levels
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm · vLLM 0.29.0
    Protocol
    wbench-fixed-http-v3
    Dataset
    migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 214.6 9.63 s 298 ms 86.28 303.8 1 86d36e4f ↗
    C4 516.94 16.14 s 872 ms 86.28 339.5 1 86d36e4f ↗
Qwen3.8-Flash-Next-NVFP4-NVIDIA
4 tested configurations
- NVFP4 · 32K → 2K tokensCPU + GPU hybrid
  Output throughput · includes prefill · 1 request
  124tok/s
  Observed GPU occupancy
  71.2GiB
  First token
  4.1 s
  Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  strata
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=3
  Workload
  32768in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 124.44 18.26 s 4.1 s 71.22 319.5 1 4a41258c ↗
- 3 other test configurations on this GPU
  - NVFP4 · 8K → 2K tokensCPU + GPU hybrid
    Output throughput · includes prefill · 1 request
    151tok/s
    Observed GPU occupancy
    71.2GiB
    First token
    1.1 s
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    strata
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 151.39 15.46 s 1.1 s 71.22 292.2 1 60a45fd1 ↗
  - NVFP4 · 32K → 2K tokensCPU + GPU hybrid
    Output throughput · includes prefill · 1 request
    114tok/s
    Observed GPU occupancy
    71.2GiB
    First token
    4.8 s
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    strata
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    32768in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 114.25 20.98 s 4.8 s 71.22 316.1 1 4342000e ↗
  - NVFP4 · 8K → 2K tokensCPU + GPU hybrid
    Output throughput · includes prefill · 1 request
    135tok/s
    Observed GPU occupancy
    71.2GiB
    First token
    1.3 s
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    strata
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 134.63 19.87 s 1.3 s 71.22 280 1 f2b52c15 ↗
Qwen3.8-Flash-Next-UD-Q4_K_XL
2 tested configurations
- Q4_K_XL · 32K → 2K tokensCPU + GPU hybrid
  Output throughput · includes prefill · 1 request
  106tok/s
  Observed GPU occupancy
  79.7GiB
  First token
  6.7 s
  Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  strata
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=3
  Workload
  32768in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 106.11 23.97 s 6.7 s 79.71 308 1 66369e97 ↗
- 1 other test configurations on this GPU
  - Q4_K_XL · 8K → 2K tokensCPU + GPU hybrid
    Output throughput · includes prefill · 1 request
    125tok/s
    Observed GPU occupancy
    79.7GiB
    First token
    1.8 s
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    strata
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 125.06 23.75 s 1.8 s 79.71 279.5 1 18c8631f ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	222.74	9.19 s	28 ms	86.28	288.8	1	adbfef8a ↗
C4	741.85	11.05 s	35 ms	86.28	335	1	adbfef8a ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	214.6	9.63 s	298 ms	86.28	303.8	1	86d36e4f ↗
C4	516.94	16.14 s	872 ms	86.28	339.5	1	86d36e4f ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	124.44	18.26 s	4.1 s	71.22	319.5	1	4a41258c ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	151.39	15.46 s	1.1 s	71.22	292.2	1	60a45fd1 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	114.25	20.98 s	4.8 s	71.22	316.1	1	4342000e ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	134.63	19.87 s	1.3 s	71.22	280	1	f2b52c15 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	106.11	23.97 s	6.7 s	79.71	308	1	66369e97 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	125.06	23.75 s	1.8 s	79.71	279.5	1	18c8631f ↗

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Sortable tables by exact test condition · 131 rows

C: concurrency · B: batch · —: not measured

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen3.8-Flash-Next-NVFP4-NVIDIAScaling curves ↗	NVFP4	strata	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v1	C1	151.39	151.39	1,140	15.457	71.22	292.2	151.39	0.52	1 ↗

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen3.8-Flash-Next-NVFP4-NVIDIAScaling curves ↗	NVFP4	strata	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v1	C1	134.63	134.63	1,305.67	19.867	71.22	280	134.63	0.48	1 ↗

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen3.8-Flash-Next-UD-Q4_K_XLScaling curves ↗	Q4_K_XL	strata	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v1	C1	125.06	125.06	1,819	23.745	79.71	279.5	125.06	0.45	1 ↗

Text generation · 8K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen/Qwen3-4B-AWQScaling curves ↗	AWQ	vllm	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3	C1	214.6	214.6	298.33	9.627	86.28	303.8	214.6	0.71	1 ↗

Text generation · 8K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen/Qwen3-4B-AWQScaling curves ↗	AWQ	vllm	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3	C4	516.94	516.94	872.17	16.142	86.28	339.5	516.94	1.52	1 ↗

Text generation · 8K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen/Qwen3-4B-AWQScaling curves ↗	AWQ	vllm	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3	C1	222.74	222.74	27.67	9.194	86.28	288.8	222.74	0.77	1 ↗

Text generation · 8K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen/Qwen3-4B-AWQScaling curves ↗	AWQ	vllm	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3	C4	741.85	741.85	35	11.05	86.28	335	741.85	2.21	1 ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen3.8-Flash-Next-NVFP4-NVIDIAScaling curves ↗	NVFP4	strata	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v1	C1	124.44	124.44	4,136	18.261	71.22	319.5	124.44	0.39	1 ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen3.8-Flash-Next-NVFP4-NVIDIAScaling curves ↗	NVFP4	strata	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v1	C1	114.25	114.25	4,774.67	20.984	71.22	316.1	114.25	0.36	1 ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen3.8-Flash-Next-UD-Q4_K_XLScaling curves ↗	Q4_K_XL	strata	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v1	C1	106.11	106.11	6,671	23.973	79.71	308	106.11	0.34	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C1	153.44	153.44	753.33	13.392	85.25	343.4	153.44	0.45	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C1	52.78	52.78	3,332.67	38.801	87.37	337.3	52.78	0.16	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C1	55.59	55.59	3,188.33	36.913	85.98	341	55.59	0.16	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C2	208.65	208.65	1,143.5	19.644	85.25	360.4	208.65	0.58	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C2	87.2	87.2	4,857.17	48.708	87.37	343.4	87.2	0.25	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C2	91.37	91.37	4,639	46.482	85.98	342.9	91.37	0.27	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C4	259.19	259.19	1,572.17	33.205	85.25	398.9	259.19	0.65	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C4	142.14	142.14	6,978.17	64.914	87.37	375.3	142.14	0.38	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C4	149.21	149.21	6,485	62.629	85.98	372.9	149.21	0.4	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C6	187.69	187.69	8,132.33	79.851	87.37	402.8	187.69	0.47	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C6	196.73	196.73	7,789.22	76.243	85.98	398.9	196.73	0.49	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C7	202.31	202.31	8,682.81	85.213	87.37	411.8	202.31	0.49	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C7	211.87	211.87	8,317.9	81.411	85.98	408.7	211.87	0.52	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C8	295.37	295.37	2,138.5	59.5	85.25	422	295.37	0.7	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C8	215.05	215.05	9,335.96	94.521	87.37	421.4	215.05	0.51	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C8	224.61	224.61	8,934.33	90.496	85.98	414.8	224.61	0.54	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 12 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C12	311.33	311.33	2,680.67	86.142	85.25	432.4	311.33	0.72	1 ↗

Text generation · 32K → 2K tokens · No cache reuse · 16 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C16	319.17	319.17	3,296.31	112.396	85.25	438	319.17	0.73	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C1	161.57	161.57	71.33	12.719	85.25	327.8	161.57	0.49	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C1	56.96	56.96	222.33	36.172	87.37	322.6	56.96	0.18	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C1	60.1	60.1	217	34.343	85.98	323	60.1	0.19	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C1	57.09	57.09	230.67	35.875	86.63	319	57.09	0.18	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C1	60.47	60.47	228.33	33.87	86.86	320.3	60.47	0.19	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C1	56.4	56.4	244.33	36.506	86.35	319.1	56.4	0.18	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C1	59.68	59.68	230.33	34.585	86.82	323.3	59.68	0.18	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C2	269.43	269.43	86.33	15.207	85.25	349.1	269.43	0.77	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C2	104.14	104.14	355.17	39.341	87.37	305.9	104.14	0.34	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C2	109.3	109.3	348.67	37.484	85.98	310.8	109.3	0.35	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C4	460.41	460.41	106.17	17.828	85.25	427.3	460.41	1.08	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C4	198.42	198.42	703.92	41.299	87.37	319.3	198.42	0.62	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C4	209.23	209.23	686.08	39.178	85.98	324.8	209.23	0.64	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C4	197.8	197.8	650.08	41.439	86.63	330.2	197.8	0.6	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C4	207.86	207.86	636.25	39.444	86.86	333.6	207.86	0.62	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C4	196.45	196.45	653.75	41.732	86.35	315.5	196.45	0.62	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C4	207.31	207.31	663.92	39.543	86.82	320	207.31	0.65	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C6	291.12	291.12	1,001.28	42.226	87.37	342.1	291.12	0.85	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C6	306.94	306.94	991.61	40.059	85.98	336	306.94	0.91	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C7	333.32	333.32	1,130.05	43.039	87.37	349.1	333.32	0.95	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C7	351.24	351.24	1,112.43	40.832	85.98	342.6	351.24	1.03	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C8	644.45	644.45	127	25.517	85.25	486.9	644.45	1.32	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C8	374.04	374.04	1,268.29	43.823	87.37	353.2	374.04	1.06	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C8	392.93	392.93	1,250.17	41.709	85.98	348.2	392.93	1.13	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C8	372.88	372.88	1,281.71	43.976	86.63	367.3	372.88	1.02	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C8	394.52	394.52	1,251.75	41.597	86.86	361.6	394.52	1.09	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C8	370.68	370.68	1,281.96	44.239	86.36	348.1	370.68	1.06	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3	C8	394.04	394.04	1,243.54	41.604	86.82	346.1	394.04	1.14	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 16 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C16	771.04	771.04	184.02	42.765	85.25	533.9	771.04	1.44	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 17 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C17	812.79	812.79	195.27	43.138	85.25	550.4	812.79	1.48	1 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 18 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1	C18	856.64	856.64	195.8	43.409	85.25	568.9	856.64	1.51	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C1	101.26	101.26	2,339.33	20.264	85.25	369.1	101.26	0.27	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C1	44.86	44.86	8,631.33	45.668	87.37	360.5	44.86	0.12	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C1	47.15	47.15	8,364	43.446	85.98	364.4	47.15	0.13	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C2	123.11	123.11	3,556.67	33.412	85.25	404.8	123.11	0.3	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C2	67.59	67.59	12,736.67	63.213	87.37	379.1	67.59	0.18	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C2	70.39	70.39	12,314.33	60.714	85.98	381.3	70.39	0.18	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C3	84.95	84.95	14,624.56	84.074	87.37	398.5	84.95	0.21	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C3	88.36	88.36	14,170.56	80.853	85.98	403.6	88.36	0.22	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C4	140.19	140.19	4,518.33	64.264	85.25	430	140.19	0.33	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C4	96.77	96.77	16,298.33	105.615	87.37	415.8	96.77	0.23	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C4	100.66	100.66	15,765.08	101.653	85.98	419.2	100.66	0.24	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C6	142.08	142.08	5,332.72	97.785	85.25	430.4	142.08	0.33	1 ↗

Text generation · 64K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C8	150.73	150.73	6,287.21	122.368	85.25	446.7	150.73	0.34	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C1	113.51	113.51	134.67	18.063	85.25	341.3	113.51	0.33	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C1	54.72	54.72	329	37.559	87.37	322.4	54.72	0.17	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C1	57.45	57.45	325.67	35.8	85.98	326.1	57.45	0.18	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C1	54.54	54.54	359	37.553	86.63	324.9	54.54	0.17	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C1	57.67	57.67	356.67	35.509	86.86	325.8	57.67	0.18	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C1	54.04	54.04	361.33	38.065	86.36	320.7	54.04	0.17	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C1	57.08	57.08	357.67	36.147	86.82	326	57.08	0.18	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C2	153.3	153.3	160.67	26.728	85.25	369.5	153.3	0.41	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C2	99.95	99.95	522.17	40.991	87.37	315.9	99.95	0.32	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C2	104.7	104.7	515.33	39.136	85.98	321	104.7	0.33	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C4	281.86	281.86	201.83	29.1	85.25	483.8	281.86	0.58	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C4	189.07	189.07	971	43.343	87.37	337.8	189.07	0.56	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C4	198.72	198.72	953.5	41.236	85.98	344.2	198.72	0.58	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C4	188.48	188.48	878.08	43.484	86.63	350.2	188.48	0.54	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C4	197.42	197.42	870.08	41.52	86.86	354.3	197.42	0.56	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C4	187.12	187.12	878.92	43.809	86.36	333.2	187.12	0.56	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C4	196.91	196.91	869.5	41.63	86.83	338.7	196.91	0.58	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C6	293.74	293.74	213.72	41.934	85.25	459.2	293.74	0.64	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C6	264.7	264.7	1,279.5	46.444	87.37	364	264.7	0.73	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C6	277.65	277.65	1,254.28	44.283	85.98	358.8	277.65	0.77	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C7	340.1	340.1	227.1	42.269	85.25	496.1	340.1	0.69	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C7	303.08	303.08	1,447.95	47.318	87.37	373.4	303.08	0.81	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C7	317.41	317.41	1,446.57	45.184	85.98	369.2	317.41	0.86	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C8	387.93	387.93	236.71	42.41	85.25	534.9	387.93	0.73	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C8	339.61	339.61	1,674.71	48.265	87.37	381.2	339.61	0.89	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C8	354.95	354.95	1,632.08	46.193	85.98	377.1	354.95	0.94	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C8	338.65	338.65	1,606.17	48.432	86.63	396.9	338.65	0.85	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C8	356.25	356.25	1,596.79	46.052	86.86	392.4	356.25	0.91	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C8	336.95	336.95	1,612.13	48.661	86.36	379.7	336.95	0.89	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3	C8	355.89	355.89	1,592	46.079	86.83	375.7	355.89	0.95	1 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 9 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1	C9	433.07	433.07	246.07	42.778	85.25	575.8	433.07	0.75	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C1	56.09	56.09	8,031	36.628	85.25	418.9	56.09	0.13	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C1	31.54	31.54	24,870	64.932	87.37	402.7	31.54	0.08	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C1	32.72	32.72	24,314.67	62.594	85.99	406.5	32.72	0.08	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C1	31.45	31.45	24,908.33	65.121	86.63	415.7	31.45	0.08	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C2	62.52	62.52	12,019.67	67.494	85.25	444.3	62.52	0.14	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C2	41.27	41.27	33,977.33	111.136	87.37	426.3	41.27	0.1	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C2	42.54	42.54	33,482.83	106.239	85.99	428.9	42.54	0.1	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C3	63.59	63.59	13,461.56	109.305	85.25	446.8	63.59	0.14	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C4	66.75	66.75	15,049.25	144.628	85.25	461.6	66.75	0.14	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C4	50.67	50.67	45,800.67	220.602	86.63	469.4	50.67	0.11	1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C8	57.74	57.74	61,931.75	421.654	86.63	494.5	57.74	0.12	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C1	71.05	71.05	267.67	28.828	85.25	356.6	71.05	0.2	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C1	52.97	52.97	434.67	38.717	85.99	333.9	52.97	0.16	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C1	52.9	52.9	508.33	38.72	86.86	338	52.9	0.16	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C1	49.98	49.98	506.33	40.979	86.36	322.7	49.98	0.15	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C1	52.59	52.59	509.33	38.947	86.83	328.8	52.59	0.16	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C2	89.43	89.43	322.5	45.82	85.25	399.6	89.43	0.22	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C2	97.25	97.25	646	42.135	85.99	340.8	97.25	0.29	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C3	135.43	135.43	378.89	45.412	85.25	470.9	135.43	0.29	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C3	138.66	138.66	822.11	44.36	85.99	358.8	138.66	0.39	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
tencent/WeVisOne-VL-2BScaling curves ↗	BF16	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C4	165.29	165.29	420.33	49.613	85.25	524	165.29	0.32	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1	C4	181.16	181.16	1,056.42	45.249	85.99	375.3	181.16	0.48	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C4	179.81	179.81	1,037	45.651	86.86	387.5	179.81	0.46	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C4	171.33	171.33	1,032.67	47.851	86.36	365.5	171.33	0.47	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C4	179.42	179.42	1,069	45.721	86.83	371.9	179.42	0.48	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C8	301.16	301.16	1,852.29	54.479	86.86	443.7	301.16	0.68	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C8	287.31	287.31	1,878.92	57.121	86.36	425.4	287.31	0.68	1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗	NVFP4	vllm	131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3	C8	300.98	300.98	1,857.75	54.52	86.83	426	300.98	0.71	1 ↗

Scaling curves and individual runs

One model and test protocol per curve; different workloads are never joined into one line.

Benchmark configuration

tencent/WeVisOne-VL-2B · BF16 · vllm · 131072in-2048out · wbench-search-http-v1

Mean throughput tok/s

Measured points only. Select a point to open its source report.

Mean run P95 latency s

Measured points only. Select a point to open its source report.

Mean GPU memory used GB

Measured points only. Select a point to open its source report.

Each point is the mean of recorded runs at that level. A source report may differ from the mean; all runs are listed below.

Uncached prefill is unavailable when prompts reuse cache or reliable phase timings are missing. Open a report for its recorded phase timings and cache counts.

Runs behind the curve · tencent/WeVisOne-VL-2B · tok/s · 4 rows
Concurrency	Throughput	Decode (tok/s)	Uncached prefill (tok/s)	GPU power (W)	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	Test date	View report
1	71.05 tok/s	71.63	Uncached prefill unavailableView report	356.6	267.67	28.828	85.25	10/26/2026, 1:05:52 AM	1ff0c98a ↗
2	89.43 tok/s	44.99	Uncached prefill unavailableView report	399.62	322.5	45.82	85.25	10/26/2026, 1:05:52 AM	1ff0c98a ↗
3	135.43 tok/s	45.49	Uncached prefill unavailableView report	470.93	378.89	45.412	85.25	10/26/2026, 1:05:52 AM	1ff0c98a ↗
4	165.29 tok/s	41.65	Uncached prefill unavailableView report	524.02	420.33	49.613	85.25	10/26/2026, 1:05:52 AM	1ff0c98a ↗

Quality by dataset

Scores from different datasets and sample scopes are not directly comparable.

Quality by dataset · 20 rows
Model	Dataset	Format	Engine	Average score	Sample size	Runs
Qwen3.8-Flash-Next-NVFP4-NVIDIA	aime2026-test-zero-shot-strict-v1-n3-mt2048-t0	NVFP4	strata	0%	3 / 30 · Subset only	1 runs 0% · 7b6f15b0 ↗
Qwen3.8-Flash-Next-NVFP4-NVIDIA	aime2026-test-zero-shot-strict-v1-n3-mt2048-t0	NVFP4	strata	33.33%	3 / 30 · Subset only	1 runs 33.33% · f7729544 ↗
Qwen3.8-Flash-Next-UD-Q4_K_XL	aime2026-test-zero-shot-strict-v1-n3-mt2048-t0	Q4_K_XL	strata	0%	3 / 30 · Subset only	1 runs 0% · d5277889 ↗
hf:nvidia/Qwen3.8-27B-NVFP4	aime2026-test-zero-shot-strict-v2-n3-mt16384-t0	NVFP4	vllm	0%	3 / 30 · Subset only	1 runs 0% · c4b1903f ↗
Qwen3.8-Flash-Next-NVFP4-NVIDIA	gsm8k-test-zero-shot-strict-v1-n3-mt2048-t0	NVFP4	strata	100%	3 / 1319 · Subset only	1 runs 100% · 2ec541fd ↗
Qwen3.8-Flash-Next-NVFP4-NVIDIA	gsm8k-test-zero-shot-strict-v1-n3-mt2048-t0	NVFP4	strata	100%	3 / 1319 · Subset only	1 runs 100% · ed5862f7 ↗
Qwen3.8-Flash-Next-UD-Q4_K_XL	gsm8k-test-zero-shot-strict-v1-n3-mt2048-t0	Q4_K_XL	strata	100%	3 / 1319 · Subset only	1 runs 100% · b1cfdc40 ↗
hf:nvidia/Qwen3.8-27B-NVFP4	gsm8k-test-zero-shot-strict-v2-n3-mt16384-t0	NVFP4	vllm	0%	3 / 1319 · Subset only	1 runs 0% · 7d12a387 ↗
cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit	gsm8k-test-zero-shot-strict-v4-n32-mt2048-t0	AWQ	vllm	78.13%	32 / 1319 · Subset only	1 runs 78.13% · e4d4c4ae ↗
cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit	gsm8k-test-zero-shot-strict-v4-n32-mt2048-t0	AWQ	vllm	84.38%	32 / 1319 · Subset only	1 runs 84.38% · 0e9cce27 ↗
cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit	gsm8k-test-zero-shot-strict-v4-n32-mt2048-t0	AWQ	vllm	84.38%	32 / 1319 · Subset only	1 runs 84.38% · d42e0304 ↗
cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit	gsm8k-test-zero-shot-strict-v4-n32-mt2048-t0	AWQ	vllm	78.13%	32 / 1319 · Subset only	1 runs 78.13% · 2e5e23a9 ↗
Qwen3.8-Flash-Next-NVFP4-NVIDIA	mmlu-test-zero-shot-strict-v1-n3-mt2048-t0	NVFP4	strata	100%	3 / 14042 · Subset only	1 runs 100% · c7ef5e0c ↗
Qwen3.8-Flash-Next-NVFP4-NVIDIA	mmlu-test-zero-shot-strict-v1-n3-mt2048-t0	NVFP4	strata	100%	3 / 14042 · Subset only	1 runs 100% · 17483940 ↗
Qwen3.8-Flash-Next-UD-Q4_K_XL	mmlu-test-zero-shot-strict-v1-n3-mt2048-t0	Q4_K_XL	strata	100%	3 / 14042 · Subset only	1 runs 100% · b72eb056 ↗
hf:nvidia/Qwen3.8-27B-NVFP4	mmlu-test-zero-shot-strict-v2-n3-mt16384-t0	NVFP4	vllm	0%	3 / 14042 · Subset only	1 runs 0% · d29d7993 ↗
cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit	mmlu-test-zero-shot-strict-v4-n32-mt2048-t0	AWQ	vllm	78.13%	32 / 14042 · Subset only	1 runs 78.13% · f68a1a8d ↗
cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit	mmlu-test-zero-shot-strict-v4-n32-mt2048-t0	AWQ	vllm	81.25%	32 / 14042 · Subset only	1 runs 81.25% · e55835c0 ↗
cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit	mmlu-test-zero-shot-strict-v4-n32-mt2048-t0	AWQ	vllm	81.25%	32 / 14042 · Subset only	1 runs 81.25% · 23fe3c88 ↗
cyankiwi/Qwen3.6-35B-A3B-AWQ-4bit	mmlu-test-zero-shot-strict-v4-n32-mt2048-t0	AWQ	vllm	81.25%	32 / 14042 · Subset only	1 runs 81.25% · 8010df52 ↗

Measurements without a comparable protocol · 153 rows

These measured rates remain visible, but are excluded from comparison curves because the protocol is unknown.

Measurements without a comparable protocol · 153 rows
Model	Format	Engine	Workload	Concurrency	Throughput	GPU power (W)
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	1	406.75 tok/s	167.76
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	2	570.1 tok/s	210.36
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	4	578.85 tok/s	215.95
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	8	448.48 tok/s	197.41
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	16	580.86 tok/s	216.54
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	32	581.39 tok/s	215.74
Qwen2.5-1.5B ↗	Q4_K_M	Ollama	1K	1	314.88 tok/s	214.05
Qwen2.5-1.5B ↗	Q4_K_M	Ollama	1K	2	402.65 tok/s	241.67
Qwen2.5-1.5B ↗	Q4_K_M	Ollama	1K	4	434.05 tok/s	239.26
Qwen2.5-1.5B ↗	Q4_K_M	Ollama	1K	8	433.88 tok/s	234.26
Qwen2.5-1.5B ↗	Q4_K_M	Ollama	1K	16	411.66 tok/s	228.41
Qwen2.5-1.5B ↗	Q4_K_M	Ollama	1K	32	440.13 tok/s	222.51
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	1	251.72 tok/s	248.53
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	2	306.47 tok/s	262.35
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	4	310.41 tok/s	262.62
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	8	308.57 tok/s	263.5
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	16	308.6 tok/s	262.89
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	32	310.61 tok/s	263.09
Qwen2.5-7B ↗	Q4_K_M	Ollama	1K	1	203.51 tok/s	337.12
Qwen2.5-7B ↗	Q4_K_M	Ollama	1K	2	215.15 tok/s	345.8
Qwen2.5-7B ↗	Q4_K_M	Ollama	1K	4	212.45 tok/s	346.42
Qwen2.5-7B ↗	Q4_K_M	Ollama	1K	8	214.69 tok/s	347.88
Qwen2.5-7B ↗	Q4_K_M	Ollama	1K	16	213.85 tok/s	348.22
Qwen2.5-7B ↗	Q4_K_M	Ollama	1K	32	214.86 tok/s	347.52
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	1	32.11 tok/s	363.29
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	8	236.11 tok/s	374.64
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	16	256.92 tok/s	422.67
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	32	766.98 tok/s	479.71
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	64	1,067.57 tok/s	543.98
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	128	738.36 tok/s	561.85
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	256	1,395.03 tok/s	581.25
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	512	1,465 tok/s	581.24
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	1024	1,538.3 tok/s	580.76
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	1	31.4 tok/s	356.5
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	8	231.23 tok/s	365.07
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	16	440.89 tok/s	416.21
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	32	763.09 tok/s	472.48
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	64	690.88 tok/s	541.99
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	128	1,247.66 tok/s	554.24
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	256	1,393.92 tok/s	582.18
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	512	967.3 tok/s	578.86
Qwen3-0.6B ↗	BF16	vLLM	2K	1	19.73 tok/s	83.37
Qwen3-0.6B ↗	BF16	vLLM	2K	8	1,986.29 tok/s	235.61
Qwen3-0.6B ↗	BF16	vLLM	2K	16	3,212.79 tok/s	264.97
Qwen3-0.6B ↗	BF16	vLLM	2K	32	4,524.91 tok/s	257.8
Qwen3-0.6B ↗	BF16	vLLM	2K	64	5,971.43 tok/s	288.07
Qwen3-0.6B ↗	BF16	vLLM	2K	128	8,165.65 tok/s	294.81
Qwen3-0.6B ↗	BF16	vLLM	2K	256	9,394.89 tok/s	314.8
Qwen3-0.6B ↗	BF16	vLLM	2K	512	10,133.18 tok/s	318
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1	261.66 tok/s	180.93
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	8	1,806.06 tok/s	213.17
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	16	3,241.44 tok/s	220.84
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	32	4,712.67 tok/s	206.69
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	64	7,537.31 tok/s	224.29
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	128	10,619.38 tok/s	253.25
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	256	13,064.79 tok/s	290.66
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	512	14,809.02 tok/s	326.59
Qwen3-1.7B ↗	BF16	vLLM	2K	1	225.09 tok/s	292.97
Qwen3-1.7B ↗	BF16	vLLM	2K	8	1,349.38 tok/s	281.45
Qwen3-1.7B ↗	BF16	vLLM	2K	16	2,398.56 tok/s	296.63
Qwen3-1.7B ↗	BF16	vLLM	2K	32	3,571.94 tok/s	284.81
Qwen3-1.7B ↗	BF16	vLLM	2K	64	5,485.36 tok/s	297.75
Qwen3-1.7B ↗	BF16	vLLM	2K	128	6,746.53 tok/s	330.45
Qwen3-1.7B ↗	BF16	vLLM	2K	256	7,824.13 tok/s	333.74
Qwen3-1.7B ↗	BF16	vLLM	2K	512	9,361.48 tok/s	337.73
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	1	179.06 tok/s	208.4
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	8	1,245.98 tok/s	214.11
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	16	2,304.28 tok/s	233.01
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	32	3,542.33 tok/s	227.19
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	64	5,147.91 tok/s	245.44
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	128	7,544.12 tok/s	275.6
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	256	8,950.66 tok/s	314.39
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	512	9,498.06 tok/s	336.52
Qwen3-4B ↗	BF16	vLLM	2K	1	121.72 tok/s	359.47
Qwen3-4B ↗	BF16	vLLM	2K	8	799.81 tok/s	304.1
Qwen3-4B ↗	BF16	vLLM	2K	16	1,281.01 tok/s	299.17
Qwen3-4B ↗	BF16	vLLM	2K	32	2,482.46 tok/s	301.8
Qwen3-4B ↗	BF16	vLLM	2K	64	3,482.71 tok/s	329.03
Qwen3-4B ↗	BF16	vLLM	2K	128	5,184.54 tok/s	346.23
Qwen3-4B ↗	BF16	vLLM	2K	256	5,835.3 tok/s	387.88
Qwen3-4B ↗	BF16	vLLM	2K	512	6,909.38 tok/s	385
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	105.66 tok/s	232.53
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	643.63 tok/s	235.67
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	16	1,310.26 tok/s	253.02
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	2,276.18 tok/s	247.29
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	64	3,254.85 tok/s	264.59
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	5,611.52 tok/s	289.17
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	256	5,906.88 tok/s	335.17
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	512	7,272.85 tok/s	356.19
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	1	24.61 tok/s	338.6
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	8	178.45 tok/s	334.35
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	16	273.53 tok/s	341.98
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	32	483.54 tok/s	347.96
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	64	885.38 tok/s	358.14
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	128	1,533.95 tok/s	383.66
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	256	2,427.56 tok/s	466.41
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	1	24.62 tok/s	336.97
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	8	140.26 tok/s	332.58
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	16	249.29 tok/s	342.02
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	32	498.29 tok/s	341.89
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	64	914.64 tok/s	358.75
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	128	1,513.36 tok/s	382.52
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	256	2,466.97 tok/s	467.58
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	1	171.92 tok/s	218.33
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	8	1,124.53 tok/s	212.82
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	16	2,038.55 tok/s	227.6
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	32	3,183.95 tok/s	219.06
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	64	5,040.91 tok/s	238.57
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	128	6,806.73 tok/s	268.61
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	256	8,328.63 tok/s	310.02
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	512	9,245.88 tok/s	328.26
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	1	102.15 tok/s	229.98
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	8	703.94 tok/s	229.64
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	16	1,204.74 tok/s	244.82
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	32	2,001.12 tok/s	238.55
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	64	3,432.74 tok/s	255.4
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	128	5,021.53 tok/s	289.06
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	256	6,345.02 tok/s	340.49
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	512	7,385.79 tok/s	366.12
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	1	121.24 tok/s	201.34
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	8	501.51 tok/s	256.94
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	16	791.06 tok/s	285.02
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	32	1,195.07 tok/s	308.88
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	64	1,921.03 tok/s	301.01
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	128	2,929.04 tok/s	322.27
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	256	4,341.5 tok/s	350.73
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	512	5,718.49 tok/s	354.64
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	1024	7,023.62 tok/s	366.43
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	1	125.01 tok/s	203.37
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	8	502.95 tok/s	261.6
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	16	797.5 tok/s	284.65
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	32	1,214.35 tok/s	310.36
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	64	1,897.86 tok/s	304.25
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	128	2,984.25 tok/s	318.33
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	256	4,339.44 tok/s	346.93
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	512	5,618.23 tok/s	350.9
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	1	24.56 tok/s	336.78
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	8	150.89 tok/s	330.42
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	16	328.98 tok/s	338.71
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	32	585.96 tok/s	343.88
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	64	1,016.4 tok/s	352.32
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	128	1,458.68 tok/s	395.79
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	256	2,179.85 tok/s	475.22
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	512	2,351.76 tok/s	491.25
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	1024	3,036.65 tok/s	511.42
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	1	24.58 tok/s	335.46
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	8	164.11 tok/s	327.86
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	16	316.04 tok/s	336.36
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	32	526.67 tok/s	342.6
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	64	1,111.66 tok/s	353.51
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	128	1,809.55 tok/s	398.63
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	256	1,808.09 tok/s	477.82
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	512	2,799.31 tok/s	492.49

Original reports

Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.

Original reports · 161 rows
Model	Test task	Format	Engine	Workload	Protocol	Quality score	Test date
DeepSeek-R1-Distill-Qwen-1.5B ↗	Text generation	BF16	vLLM	1K	wbench-http	—	10/14/2026, 8:52:17 AM
Llama-3.1-8B-Instruct-NVFP4 ↗	Text generation	NVFP4	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:12:13 PM
Llama-3.1-8B-Instruct-NVFP4 ↗	Text generation	NVFP4	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:11:46 PM
Llama-3.1-8B-Instruct-NVFP4 ↗	Text generation	NVFP4	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:11:21 PM
NVIDIA-Nemotron-Nano-9B-v2-NVFP4 ↗	Text generation	NVFP4	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:16:58 PM
NVIDIA-Nemotron-Nano-9B-v2-NVFP4 ↗	Text generation	NVFP4	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:16:28 PM
NVIDIA-Nemotron-Nano-9B-v2-NVFP4 ↗	Text generation	NVFP4	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:15:49 PM
NVIDIA-Nemotron-Nano-12B-v2-VL-NVFP4-QAD ↗	Text generation	NVFP4	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:21:34 PM
NVIDIA-Nemotron-Nano-12B-v2-VL-NVFP4-QAD ↗	Text generation	NVFP4	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:20:57 PM
Qwen2.5-0.5B ↗	Text generation	Q4_K_M	Ollama	1K	Method unknown	—	10/14/2026, 8:27:58 AM
qwen2.5-0.5b-instruct-q4_k_m.gguf ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:06:17 AM
Qwen2.5-1.5B ↗	Text generation	Q4_K_M	Ollama	1K	Method unknown	—	10/14/2026, 8:32:19 AM
qwen2.5-1.5b-instruct-q4_k_m.gguf ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:08:55 AM
Qwen2.5-3B ↗	Text generation	Q4_K_M	Ollama	1K	Method unknown	—	10/14/2026, 8:37:06 AM
Qwen2.5-3B-Instruct ↗	Text generation	BF16	vllm	32K	vllm-official	—	10/26/2026, 6:36:29 AM
Qwen2.5-3B-Instruct ↗	Text generation	BF16	vllm	32K	vllm-official	—	10/26/2026, 6:32:34 AM
Qwen2.5-3B-Instruct ↗	Text generation	BF16	vllm	4K	vllm-official	—	10/26/2026, 6:31:24 AM
Qwen2.5-3B-Instruct ↗	Text generation	BF16	vllm	4K	vllm-official	—	10/26/2026, 6:30:25 AM
qwen2.5-3b-instruct-q4_k_m.gguf ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:11:33 AM
Qwen2.5-7B ↗	Text generation	Q4_K_M	Ollama	1K	Method unknown	—	10/14/2026, 8:42:41 AM
Qwen2.5-72B-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	1/13/2026, 1:08:31 AM
Qwen2.5-72B-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	1/12/2026, 9:06:29 PM
Qwen3-0.6B ↗	Text generation	BF16	vLLM	1K	wbench-http	—	10/14/2026, 8:46:02 AM
Qwen3-0.6B ↗	Text generation	BF16	vLLM	2K	Method unknown	—	1/11/2026, 3:59:24 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 7:18:04 PM
Qwen3-0.6B-GPTQ ↗	Text generation	GPTQ	vLLM	1K	wbench-http	—	10/14/2026, 8:25:21 AM
Qwen3-0.6B-Q8 ↗	Text generation	Q8_0	llama.cpp	1K	wbench-http	—	10/14/2026, 9:14:07 AM
Qwen3-1.7B ↗	Text generation	BF16	vLLM	1K	wbench-http	—	10/14/2026, 8:58:30 AM
Qwen3-1.7B ↗	Text generation	BF16	vLLM	2K	Method unknown	—	1/11/2026, 4:06:46 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	32K	vllm-official	—	10/26/2026, 6:27:28 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	32K	vllm-official	—	10/26/2026, 6:23:11 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	4K	vllm-official	—	10/26/2026, 6:22:18 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	4K	vllm-official	—	10/26/2026, 6:21:16 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 7:25:25 PM
Qwen3-1.7B-Q8 ↗	Text generation	Q8_0	llama.cpp	1K	wbench-http	—	10/14/2026, 9:16:40 AM
Qwen3-4B ↗	Text generation	BF16	vLLM	1K	wbench-http	—	10/14/2026, 9:04:46 AM
Qwen3-4B ↗	Text generation	BF16	vLLM	2K	Method unknown	—	1/11/2026, 4:14:58 PM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vllm	8192in-2048out	wbench-fixed-http-v3	—	10/23/2026, 6:41:45 AM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vllm	8192in-2048out	wbench-fixed-http-v3	—	10/23/2026, 6:39:18 AM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vLLM	1K	wbench-http	—	10/14/2026, 9:10:51 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vllm	32K	vllm-official	—	10/26/2026, 6:47:58 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vllm	32K	vllm-official	—	10/26/2026, 6:43:10 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vllm	4K	vllm-official	—	10/26/2026, 6:41:49 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vllm	4K	vllm-official	—	10/26/2026, 6:40:24 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 8:05:29 PM
Qwen3-4B-Q4 ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:19:13 AM
Qwen3-4B-Q8_0 ↗	Text generation	Q8_0	llama.cpp	1K	wbench-http	—	10/14/2026, 9:48:49 AM
Qwen3-8B ↗	Text generation	BF16	vllm	2K	vllm-official	—	10/14/2026, 6:16:09 AM
Qwen3-8B ↗	Text generation	BF16	vllm	2K	vllm-official	—	10/14/2026, 6:16:08 AM
Qwen3-8B ↗	Text generation	BF16	vllm	2K	vllm-official	—	10/14/2026, 6:16:07 AM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	1K	wbench-http	—	10/14/2026, 9:17:31 AM
Qwen3-8B-NVFP4 ↗	Text generation	NVFP4	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:08:42 PM
Qwen3-8B-NVFP4 ↗	Text generation	NVFP4	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:08:10 PM
Qwen3-8B-NVFP4 ↗	Text generation	NVFP4	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:07:42 PM
Qwen3-8B-Q4 ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:21:56 AM
Qwen3-14B-AWQ ↗	Text generation	AWQ	vLLM	1K	wbench-http	—	10/14/2026, 9:24:27 AM
Qwen3-14B-Q4 ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:24:54 AM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	1K	wbench-http	—	10/14/2026, 8:39:06 AM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 1:35:30 AM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 9:58:17 PM
Qwen3-32B-Q4 ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:29:00 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 7:57:22 PM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 8:13:21 PM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 1:45:48 AM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 10:17:10 PM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 1:19:39 AM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 9:30:01 PM
Qwen3.5-0.8B ↗	Text generation	BF16	vllm	32K	vllm-official	—	10/26/2026, 6:15:09 AM
Qwen3.5-0.8B ↗	Text generation	BF16	vllm	32K	vllm-official	—	10/26/2026, 6:13:41 AM
Qwen3.5-0.8B ↗	Text generation	BF16	vllm	4K	vllm-official	—	10/26/2026, 6:13:12 AM
Qwen3.5-0.8B ↗	Text generation	BF16	vllm	4K	vllm-official	—	10/26/2026, 6:12:41 AM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/22/2026, 2:21:46 AM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 6:16:25 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 6:14:59 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 6:10:03 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 6:08:52 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 6:05:25 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 6:04:00 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:59:16 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:58:06 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:54:40 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:53:26 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:48:23 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:47:15 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:43:49 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:42:31 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:37:23 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:36:06 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	mmlu-test-zero-shot-strict-v4-n32-mt2048-t0	quality-v1	78.13%	10/21/2026, 5:30:58 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	gsm8k-test-zero-shot-strict-v4-n32-mt2048-t0	quality-v1	78.13%	10/21/2026, 5:28:11 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:23:51 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:23:26 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:22:29 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:22:04 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	mmlu-test-zero-shot-strict-v4-n32-mt2048-t0	quality-v1	81.25%	10/21/2026, 5:19:15 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	gsm8k-test-zero-shot-strict-v4-n32-mt2048-t0	quality-v1	84.38%	10/21/2026, 5:16:42 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:12:28 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:12:04 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:11:08 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:10:44 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	mmlu-test-zero-shot-strict-v4-n32-mt2048-t0	quality-v1	81.25%	10/21/2026, 5:08:00 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	gsm8k-test-zero-shot-strict-v4-n32-mt2048-t0	quality-v1	84.38%	10/21/2026, 5:05:35 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:01:27 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 5:01:01 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 4:59:56 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 4:59:33 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 4:59:08 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 4:59:07 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	mmlu-test-zero-shot-strict-v4-n32-mt2048-t0	quality-v1	81.25%	10/21/2026, 4:56:50 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	gsm8k-test-zero-shot-strict-v4-n32-mt2048-t0	quality-v1	78.13%	10/21/2026, 4:54:23 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	8192in-128out	wbench-fixed-http-v3	—	10/21/2026, 4:49:46 PM
Qwen3.6-35B-A3B-AWQ-4bit ↗	Text generation	AWQ	vllm	1024in-128out	wbench-fixed-http-v3	—	10/21/2026, 4:48:19 PM
Qwen3.6-35B-A3B-FP8 ↗	Text generation	FP8	vllm	32K	vllm-official	—	10/26/2026, 5:57:30 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	131072in-2048out	wbench-search-http-v1	—	10/25/2026, 9:17:12 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	65536in-2048out	wbench-search-http-v1	—	10/25/2026, 9:03:28 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	65536in-2048out	wbench-search-http-v1	—	10/25/2026, 8:40:15 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	32768in-2048out	wbench-search-http-v1	—	10/25/2026, 8:18:10 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	32768in-2048out	wbench-search-http-v1	—	10/25/2026, 7:57:47 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	131072in-2048out	wbench-search-http-v1	—	10/25/2026, 7:24:36 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	131072in-2048out	wbench-search-http-v1	—	10/25/2026, 7:09:07 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	65536in-2048out	wbench-search-http-v1	—	10/25/2026, 6:55:57 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	65536in-2048out	wbench-search-http-v1	—	10/25/2026, 6:34:21 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	32768in-2048out	wbench-search-http-v1	—	10/25/2026, 6:13:45 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	32768in-2048out	wbench-search-http-v1	—	10/25/2026, 5:54:42 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	131072in-2048out	wbench-fixed-http-v3	—	10/19/2026, 9:20:04 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	65536in-2048out	wbench-fixed-http-v3	—	10/19/2026, 8:34:56 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	32768in-2048out	wbench-fixed-http-v3	—	10/19/2026, 8:07:59 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	131072in-2048out	wbench-fixed-http-v3	—	10/19/2026, 7:40:50 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	65536in-2048out	wbench-fixed-http-v3	—	10/19/2026, 6:58:25 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	32768in-2048out	wbench-fixed-http-v3	—	10/19/2026, 6:33:20 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	131072in-2048out	wbench-fixed-http-v3	—	10/19/2026, 2:00:54 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	65536in-2048out	wbench-fixed-http-v3	—	10/19/2026, 1:18:10 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	32768in-2048out	wbench-fixed-http-v3	—	10/19/2026, 12:52:39 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	131072in-2048out	wbench-fixed-http-v3	—	10/19/2026, 12:27:36 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	65536in-2048out	wbench-fixed-http-v3	—	10/18/2026, 11:46:41 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	32768in-2048out	wbench-fixed-http-v3	—	10/18/2026, 11:22:38 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	aime2026-test-zero-shot-strict-v2-n3-mt16384-t0	quality-v1	0%	10/14/2026, 1:10:28 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	gsm8k-test-zero-shot-strict-v2-n3-mt16384-t0	quality-v1	0%	10/14/2026, 1:10:26 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	mmlu-test-zero-shot-strict-v2-n3-mt16384-t0	quality-v1	0%	10/14/2026, 1:10:24 PM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	aime2026-test-zero-shot-strict-v1-n3-mt2048-t0	quality-v1	0%	10/14/2026, 10:50:49 AM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	mmlu-test-zero-shot-strict-v1-n3-mt2048-t0	quality-v1	100%	10/14/2026, 10:50:09 AM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	gsm8k-test-zero-shot-strict-v1-n3-mt2048-t0	quality-v1	100%	10/14/2026, 10:50:05 AM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	32768in-2048out	wbench-fixed-http-v1	—	10/14/2026, 10:49:58 AM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	8192in-2048out	wbench-fixed-http-v1	—	10/14/2026, 10:49:01 AM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	aime2026-test-zero-shot-strict-v1-n3-mt2048-t0	quality-v1	33.33%	10/14/2026, 10:42:50 AM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	mmlu-test-zero-shot-strict-v1-n3-mt2048-t0	quality-v1	100%	10/14/2026, 10:42:03 AM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	gsm8k-test-zero-shot-strict-v1-n3-mt2048-t0	quality-v1	100%	10/14/2026, 10:41:57 AM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	32768in-2048out	wbench-fixed-http-v1	—	10/14/2026, 10:41:46 AM
Qwen3.8-Flash-Next-NVFP4-NVIDIA ↗	Text generation	NVFP4	strata	8192in-2048out	wbench-fixed-http-v1	—	10/14/2026, 10:40:44 AM
Qwen3.8-Flash-Next-UD-Q4_K_XL ↗	Text generation	Q4_K_XL	strata	aime2026-test-zero-shot-strict-v1-n3-mt2048-t0	quality-v1	0%	10/14/2026, 10:19:21 AM
Qwen3.8-Flash-Next-UD-Q4_K_XL ↗	Text generation	Q4_K_XL	strata	mmlu-test-zero-shot-strict-v1-n3-mt2048-t0	quality-v1	100%	10/14/2026, 10:18:19 AM
Qwen3.8-Flash-Next-UD-Q4_K_XL ↗	Text generation	Q4_K_XL	strata	gsm8k-test-zero-shot-strict-v1-n3-mt2048-t0	quality-v1	100%	10/14/2026, 10:18:10 AM
Qwen3.8-Flash-Next-UD-Q4_K_XL ↗	Text generation	Q4_K_XL	strata	32768in-2048out	wbench-fixed-http-v1	—	10/14/2026, 10:17:57 AM
Qwen3.8-Flash-Next-UD-Q4_K_XL ↗	Text generation	Q4_K_XL	strata	8192in-2048out	wbench-fixed-http-v1	—	10/14/2026, 10:16:49 AM
WeVisOne-VL-2B ↗	Text generation	BF16	vllm	131072in-2048out	wbench-search-http-v1	—	10/26/2026, 1:05:52 AM
WeVisOne-VL-2B ↗	Text generation	BF16	vllm	131072in-2048out	wbench-search-http-v1	—	10/26/2026, 12:51:38 AM
WeVisOne-VL-2B ↗	Text generation	BF16	vllm	65536in-2048out	wbench-search-http-v1	—	10/26/2026, 12:25:56 AM
WeVisOne-VL-2B ↗	Text generation	BF16	vllm	65536in-2048out	wbench-search-http-v1	—	10/26/2026, 12:06:16 AM
WeVisOne-VL-2B ↗	Text generation	BF16	vllm	32768in-2048out	wbench-search-http-v1	—	10/25/2026, 11:42:06 PM
WeVisOne-VL-2B ↗	Text generation	BF16	vllm	32768in-2048out	wbench-search-http-v1	—	10/25/2026, 11:26:15 PM
WeVisOne-VL-2B ↗	Text generation	BF16	vllm	32768in-2048out	wbench-search-http-v1	—	10/25/2026, 10:40:36 PM

Running benchmarks

0Updates automatically

No benchmarks are currently reported as running.

RTX PRO 6000 Blackwell Server Edition · 96 GB · 403 queued · 0 running · 0 awaiting reassignment · 6 succeeded · 3 failed or exhausted

Methodology

Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.

Updated 11/5/2026, 9:54:22 PM

Text generation · Qualified results

WeVisOne-VL-2B

Qwen3.8-27B-NVFP4

Qwen3.8-27B-NVFP4

Qwen3-4B-AWQ

Qwen3.8-Flash-Next-NVFP4-NVIDIA

Qwen3.8-Flash-Next-UD-Q4_K_XL

Quality by dataset

Original reports

Running benchmarks