0 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Get your startup command

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

All tasks · RTX PRO 6000 Blackwell Workstation Edition · 96 GB

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

No qualified results yet · All tasks · RTX PRO 6000 Blackwell Workstation Edition · 96 GB

28 original reports are kept as evidence but do not qualify for comparison; each report states why. This does not show that the model or GPU cannot run it.

Check model launch support View original reports Measured alternatives for this task Test your own GPU

Measured alternatives on other GPUs

Measured on different GPUs with the same or smaller listed capacity. These are references for choosing hardware, not performance or compatibility evidence for your selected GPU.

WeVisOne-VL-2B

tencent · 6 tested configurations · 1 GPU

RTX PRO 6000 Blackwell Server Edition · 96 GBBF16 · 32K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 153tok/s
Best measured · 16 concurrent requests: 319tok/s
Observed GPU occupancy: 85.3GiB
First token: 753 ms

Use this recipe · 16 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	153.44	13.39 s	753 ms	85.25	343.4	1	d2914dd8 ↗
C2	208.65	19.64 s	1.1 s	85.25	360.4	1	d2914dd8 ↗
C4	259.19	33.21 s	1.6 s	85.25	398.9	1	d2914dd8 ↗
C8	295.37	59.5 s	2.1 s	85.25	422	1	d2914dd8 ↗
C12	311.33	86.14 s	2.7 s	85.25	432.4	1	d2914dd8 ↗
C16	319.17	112.4 s	3.3 s	85.25	438	1	d2914dd8 ↗

5 other test configurations on this GPU

BF16 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 162tok/s
Best measured · 18 concurrent requests: 857tok/s
Observed GPU occupancy: 85.3GiB
First token: 71 ms

Use this recipe · 18 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 7 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	161.57	12.72 s	71 ms	85.25	327.8	1	75198c63 ↗
C2	269.43	15.21 s	86 ms	85.25	349.1	1	75198c63 ↗
C4	460.41	17.83 s	106 ms	85.25	427.3	1	75198c63 ↗
C8	644.45	25.52 s	127 ms	85.25	486.9	1	75198c63 ↗
C16	771.04	42.77 s	184 ms	85.25	533.9	1	75198c63 ↗
C17	812.79	43.14 s	195 ms	85.25	550.4	1	75198c63 ↗
C18	856.64	43.41 s	196 ms	85.25	568.9	1	75198c63 ↗

BF16 · 64K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 101tok/s
Best measured · 8 concurrent requests: 151tok/s
Observed GPU occupancy: 85.3GiB
First token: 2.3 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 5 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	101.26	20.26 s	2.3 s	85.25	369.1	1	c5c9cf6f ↗
C2	123.11	33.41 s	3.6 s	85.25	404.8	1	c5c9cf6f ↗
C4	140.19	64.26 s	4.5 s	85.25	430	1	c5c9cf6f ↗
C6	142.08	97.79 s	5.3 s	85.25	430.4	1	c5c9cf6f ↗
C8	150.73	122.37 s	6.3 s	85.25	446.7	1	c5c9cf6f ↗

BF16 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 114tok/s
Best measured · 9 concurrent requests: 433tok/s
Observed GPU occupancy: 85.3GiB
First token: 135 ms

Use this recipe · 9 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 7 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	113.51	18.06 s	135 ms	85.25	341.3	1	5fee1315 ↗
C2	153.3	26.73 s	161 ms	85.25	369.5	1	5fee1315 ↗
C4	281.86	29.1 s	202 ms	85.25	483.8	1	5fee1315 ↗
C6	293.74	41.93 s	214 ms	85.25	459.2	1	5fee1315 ↗
C7	340.1	42.27 s	227 ms	85.25	496.1	1	5fee1315 ↗
C8	387.93	42.41 s	237 ms	85.25	534.9	1	5fee1315 ↗
C9	433.07	42.78 s	246 ms	85.25	575.8	1	5fee1315 ↗

BF16 · 128K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 56.1tok/s
Best measured · 4 concurrent requests: 66.8tok/s
Observed GPU occupancy: 85.3GiB
First token: 8 s

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	56.09	36.63 s	8 s	85.25	418.9	1	2c661c7e ↗
C2	62.52	67.49 s	12 s	85.25	444.3	1	2c661c7e ↗
C3	63.59	109.31 s	13.5 s	85.25	446.8	1	2c661c7e ↗
C4	66.75	144.63 s	15 s	85.25	461.6	1	2c661c7e ↗

BF16 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 71tok/s
Best measured · 4 concurrent requests: 165tok/s
Observed GPU occupancy: 85.3GiB
First token: 268 ms

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	71.05	28.83 s	268 ms	85.25	356.6	1	1ff0c98a ↗
C2	89.43	45.82 s	323 ms	85.25	399.6	1	1ff0c98a ↗
C3	135.43	45.41 s	379 ms	85.25	470.9	1	1ff0c98a ↗
C4	165.29	49.61 s	420 ms	85.25	524	1	1ff0c98a ↗

Qwen3.8-27B-NVFP4

unsloth · 11 tested configurations · 1 GPU

RTX PRO 6000 Blackwell Server Edition · 96 GBNVFP4 · 32K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 52.8tok/s
Best measured · 8 concurrent requests: 215tok/s
Observed GPU occupancy: 87.4GiB
First token: 3.3 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.78	38.8 s	3.3 s	87.37	337.3	1	aee8b33e ↗
C2	87.2	48.71 s	4.9 s	87.37	343.4	1	aee8b33e ↗
C4	142.14	64.91 s	7 s	87.37	375.3	1	aee8b33e ↗
C6	187.69	79.85 s	8.1 s	87.37	402.8	1	aee8b33e ↗
C7	202.31	85.21 s	8.7 s	87.37	411.8	1	aee8b33e ↗
C8	215.05	94.52 s	9.3 s	87.37	421.4	1	aee8b33e ↗

10 other test configurations on this GPU

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57tok/s
Best measured · 8 concurrent requests: 374tok/s
Observed GPU occupancy: 87.4GiB
First token: 222 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	56.96	36.17 s	222 ms	87.37	322.6	1	bced0642 ↗
C2	104.14	39.34 s	355 ms	87.37	305.9	1	bced0642 ↗
C4	198.42	41.3 s	704 ms	87.37	319.3	1	bced0642 ↗
C6	291.12	42.23 s	1 s	87.37	342.1	1	bced0642 ↗
C7	333.32	43.04 s	1.1 s	87.37	349.1	1	bced0642 ↗
C8	374.04	43.82 s	1.3 s	87.37	353.2	1	bced0642 ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.1tok/s
Best measured · 8 concurrent requests: 373tok/s
Observed GPU occupancy: 86.6GiB
First token: 231 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.09	35.88 s	231 ms	86.63	319	1	37f29252 ↗
C4	197.8	41.44 s	650 ms	86.63	330.2	1	37f29252 ↗
C8	372.88	43.98 s	1.3 s	86.63	367.3	1	37f29252 ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 56.4tok/s
Best measured · 8 concurrent requests: 371tok/s
Observed GPU occupancy: 86.4GiB
First token: 244 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	56.4	36.51 s	244 ms	86.35	319.1	1	791b887d ↗
C4	196.45	41.73 s	654 ms	86.35	315.5	1	791b887d ↗
C8	370.68	44.24 s	1.3 s	86.36	348.1	1	791b887d ↗

NVFP4 · 64K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 44.9tok/s
Best measured · 4 concurrent requests: 96.8tok/s
Observed GPU occupancy: 87.4GiB
First token: 8.6 s

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	44.86	45.67 s	8.6 s	87.37	360.5	1	d2ff32ec ↗
C2	67.59	63.21 s	12.7 s	87.37	379.1	1	d2ff32ec ↗
C3	84.95	84.07 s	14.6 s	87.37	398.5	1	d2ff32ec ↗
C4	96.77	105.62 s	16.3 s	87.37	415.8	1	d2ff32ec ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 54.7tok/s
Best measured · 8 concurrent requests: 340tok/s
Observed GPU occupancy: 87.4GiB
First token: 329 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	54.72	37.56 s	329 ms	87.37	322.4	1	d3f6b770 ↗
C2	99.95	40.99 s	522 ms	87.37	315.9	1	d3f6b770 ↗
C4	189.07	43.34 s	971 ms	87.37	337.8	1	d3f6b770 ↗
C6	264.7	46.44 s	1.3 s	87.37	364	1	d3f6b770 ↗
C7	303.08	47.32 s	1.4 s	87.37	373.4	1	d3f6b770 ↗
C8	339.61	48.27 s	1.7 s	87.37	381.2	1	d3f6b770 ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 54tok/s
Best measured · 8 concurrent requests: 337tok/s
Observed GPU occupancy: 86.4GiB
First token: 361 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	54.04	38.07 s	361 ms	86.36	320.7	1	417f69be ↗
C4	187.12	43.81 s	879 ms	86.36	333.2	1	417f69be ↗
C8	336.95	48.66 s	1.6 s	86.36	379.7	1	417f69be ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 54.5tok/s
Best measured · 8 concurrent requests: 339tok/s
Observed GPU occupancy: 86.6GiB
First token: 359 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	54.54	37.55 s	359 ms	86.63	324.9	1	7d1a7c8e ↗
C4	188.48	43.48 s	878 ms	86.63	350.2	1	7d1a7c8e ↗
C8	338.65	48.43 s	1.6 s	86.63	396.9	1	7d1a7c8e ↗

NVFP4 · 128K → 2K tokens · No cache reuse
Output throughput · includes prefill · 1 request
31.5tok/s
Best measured · 2 concurrent requests
41.3tok/s
Observed GPU occupancy
87.4GiB
First token
24.9 s
Use this recipe · 2 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 concurrency levels
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
vllm · vLLM 0.28.0
Protocol
wbench-search-http-v1
Dataset
migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload
131072in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
C1 31.54 64.93 s 24.9 s 87.37 402.7 1 73fcb7c1 ↗
C2 41.27 111.14 s 34 s 87.37 426.3 1 73fcb7c1 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	31.54	64.93 s	24.9 s	87.37	402.7	1	73fcb7c1 ↗
C2	41.27	111.14 s	34 s	87.37	426.3	1	73fcb7c1 ↗

NVFP4 · 128K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 31.5tok/s
Best measured · 8 concurrent requests: 57.7tok/s
Observed GPU occupancy: 86.6GiB
First token: 24.9 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=isolated:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	31.45	65.12 s	24.9 s	86.63	415.7	1	448b0777 ↗
C4	50.67	220.6 s	45.8 s	86.63	469.4	1	448b0777 ↗
C8	57.74	421.65 s	61.9 s	86.63	494.5	1	448b0777 ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 50tok/s
Best measured · 8 concurrent requests: 287tok/s
Observed GPU occupancy: 86.4GiB
First token: 506 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	49.98	40.98 s	506 ms	86.36	322.7	1	68d1f2c6 ↗
C4	171.33	47.85 s	1 s	86.36	365.5	1	68d1f2c6 ↗
C8	287.31	57.12 s	1.9 s	86.36	425.4	1	68d1f2c6 ↗

Models with reports but no qualified results · 27

Their original reports remain available on each model page. They did not qualify for comparison; each report states why.

Models with reports but no qualified results · 27 rows
Model	Test task
Qwen3-0.6BQwen/Qwen3-0.6B	Text generation
Qwen3-0.6B-FP8Qwen/Qwen3-0.6B-FP8	Text generation
Qwen3-1.7BQwen/Qwen3-1.7B	Text generation
Qwen3-1.7B-FP8Qwen/Qwen3-1.7B-FP8	Text generation
Qwen3-4BQwen/Qwen3-4B	Text generation
Qwen3-4B-FP8Qwen/Qwen3-4B-FP8	Text generation
Qwen3-4B-Instruct-2507-FP8Qwen/Qwen3-4B-Instruct-2507-FP8	Text generation
Qwen3-4B-Thinking-2507-FP8Qwen/Qwen3-4B-Thinking-2507-FP8	Text generation
Qwen3-8B/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-8B	Text generation
Qwen3-8B-FP8Qwen/Qwen3-8B-FP8	Text generation
Qwen3-14B/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-14B	Text generation
Qwen3-14B-FP8Qwen/Qwen3-14B-FP8	Text generation
Qwen3-30B-A3B/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-30B-A3B	Text generation
Qwen3-30B-A3B-FP8Qwen/Qwen3-30B-A3B-Instruct-2507-FP8	Text generation
Qwen3-30B-A3B-Thinking-FP8Qwen/Qwen3-30B-A3B-Thinking-2507-FP8	Text generation
Qwen3-32B/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-32B	Text generation
Qwen3-32B-FP8Qwen/Qwen3-32B-FP8	Text generation
Qwen3-VL-2B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-2B-Instruct	Text generation
Qwen3-VL-2B-Instruct-FP8Qwen/Qwen3-VL-2B-Instruct-FP8	Text generation
Qwen3-VL-4B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-4B-Instruct	Text generation
Qwen3-VL-4B-Instruct-FP8Qwen/Qwen3-VL-4B-Instruct-FP8	Text generation
Qwen3-VL-8B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-8B-Instruct	Text generation
Qwen3-VL-8B-Instruct-FP8Qwen/Qwen3-VL-8B-Instruct-FP8	Text generation
Qwen3-VL-30B-A3B-FP8Qwen/Qwen3-VL-30B-A3B-Instruct-FP8	Text generation
Qwen3-VL-30B-A3B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-30B-A3B-Instruct	Text generation
Qwen3-VL-32B-FP8Qwen/Qwen3-VL-32B-Instruct-FP8	Text generation
Qwen3-VL-32B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-32B-Instruct	Text generation