6 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Get your startup command

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

Text generation · All measured GPUs

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

WeVisOne-VL-2B

tencent · 6 tested configurations · 1 GPU

RTX PRO 6000 Blackwell Server Edition · 96 GBBF16 · 32K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 153tok/s
Best measured · 16 concurrent requests: 319tok/s
Observed GPU occupancy: 85.3GiB
First token: 753 ms

Use this recipe · 16 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	153.44	13.39 s	753 ms	85.25	343.4	1	d2914dd8 ↗
C2	208.65	19.64 s	1.1 s	85.25	360.4	1	d2914dd8 ↗
C4	259.19	33.21 s	1.6 s	85.25	398.9	1	d2914dd8 ↗
C8	295.37	59.5 s	2.1 s	85.25	422	1	d2914dd8 ↗
C12	311.33	86.14 s	2.7 s	85.25	432.4	1	d2914dd8 ↗
C16	319.17	112.4 s	3.3 s	85.25	438	1	d2914dd8 ↗

5 other test configurations on this GPU

BF16 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 162tok/s
Best measured · 18 concurrent requests: 857tok/s
Observed GPU occupancy: 85.3GiB
First token: 71 ms

Use this recipe · 18 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 7 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	161.57	12.72 s	71 ms	85.25	327.8	1	75198c63 ↗
C2	269.43	15.21 s	86 ms	85.25	349.1	1	75198c63 ↗
C4	460.41	17.83 s	106 ms	85.25	427.3	1	75198c63 ↗
C8	644.45	25.52 s	127 ms	85.25	486.9	1	75198c63 ↗
C16	771.04	42.77 s	184 ms	85.25	533.9	1	75198c63 ↗
C17	812.79	43.14 s	195 ms	85.25	550.4	1	75198c63 ↗
C18	856.64	43.41 s	196 ms	85.25	568.9	1	75198c63 ↗

BF16 · 64K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 101tok/s
Best measured · 8 concurrent requests: 151tok/s
Observed GPU occupancy: 85.3GiB
First token: 2.3 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 5 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	101.26	20.26 s	2.3 s	85.25	369.1	1	c5c9cf6f ↗
C2	123.11	33.41 s	3.6 s	85.25	404.8	1	c5c9cf6f ↗
C4	140.19	64.26 s	4.5 s	85.25	430	1	c5c9cf6f ↗
C6	142.08	97.79 s	5.3 s	85.25	430.4	1	c5c9cf6f ↗
C8	150.73	122.37 s	6.3 s	85.25	446.7	1	c5c9cf6f ↗

BF16 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 114tok/s
Best measured · 9 concurrent requests: 433tok/s
Observed GPU occupancy: 85.3GiB
First token: 135 ms

Use this recipe · 9 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 7 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	113.51	18.06 s	135 ms	85.25	341.3	1	5fee1315 ↗
C2	153.3	26.73 s	161 ms	85.25	369.5	1	5fee1315 ↗
C4	281.86	29.1 s	202 ms	85.25	483.8	1	5fee1315 ↗
C6	293.74	41.93 s	214 ms	85.25	459.2	1	5fee1315 ↗
C7	340.1	42.27 s	227 ms	85.25	496.1	1	5fee1315 ↗
C8	387.93	42.41 s	237 ms	85.25	534.9	1	5fee1315 ↗
C9	433.07	42.78 s	246 ms	85.25	575.8	1	5fee1315 ↗

BF16 · 128K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 56.1tok/s
Best measured · 4 concurrent requests: 66.8tok/s
Observed GPU occupancy: 85.3GiB
First token: 8 s

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	56.09	36.63 s	8 s	85.25	418.9	1	2c661c7e ↗
C2	62.52	67.49 s	12 s	85.25	444.3	1	2c661c7e ↗
C3	63.59	109.31 s	13.5 s	85.25	446.8	1	2c661c7e ↗
C4	66.75	144.63 s	15 s	85.25	461.6	1	2c661c7e ↗

BF16 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 71tok/s
Best measured · 4 concurrent requests: 165tok/s
Observed GPU occupancy: 85.3GiB
First token: 268 ms

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	71.05	28.83 s	268 ms	85.25	356.6	1	1ff0c98a ↗
C2	89.43	45.82 s	323 ms	85.25	399.6	1	1ff0c98a ↗
C3	135.43	45.41 s	379 ms	85.25	470.9	1	1ff0c98a ↗
C4	165.29	49.61 s	420 ms	85.25	524	1	1ff0c98a ↗

Qwen3.8-27B-NVFP4

unsloth · 11 tested configurations · 1 GPU

RTX PRO 6000 Blackwell Server Edition · 96 GBNVFP4 · 32K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 52.8tok/s
Best measured · 8 concurrent requests: 215tok/s
Observed GPU occupancy: 87.4GiB
First token: 3.3 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.78	38.8 s	3.3 s	87.37	337.3	1	aee8b33e ↗
C2	87.2	48.71 s	4.9 s	87.37	343.4	1	aee8b33e ↗
C4	142.14	64.91 s	7 s	87.37	375.3	1	aee8b33e ↗
C6	187.69	79.85 s	8.1 s	87.37	402.8	1	aee8b33e ↗
C7	202.31	85.21 s	8.7 s	87.37	411.8	1	aee8b33e ↗
C8	215.05	94.52 s	9.3 s	87.37	421.4	1	aee8b33e ↗

10 other test configurations on this GPU

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57tok/s
Best measured · 8 concurrent requests: 374tok/s
Observed GPU occupancy: 87.4GiB
First token: 222 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	56.96	36.17 s	222 ms	87.37	322.6	1	bced0642 ↗
C2	104.14	39.34 s	355 ms	87.37	305.9	1	bced0642 ↗
C4	198.42	41.3 s	704 ms	87.37	319.3	1	bced0642 ↗
C6	291.12	42.23 s	1 s	87.37	342.1	1	bced0642 ↗
C7	333.32	43.04 s	1.1 s	87.37	349.1	1	bced0642 ↗
C8	374.04	43.82 s	1.3 s	87.37	353.2	1	bced0642 ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.1tok/s
Best measured · 8 concurrent requests: 373tok/s
Observed GPU occupancy: 86.6GiB
First token: 231 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.09	35.88 s	231 ms	86.63	319	1	37f29252 ↗
C4	197.8	41.44 s	650 ms	86.63	330.2	1	37f29252 ↗
C8	372.88	43.98 s	1.3 s	86.63	367.3	1	37f29252 ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 56.4tok/s
Best measured · 8 concurrent requests: 371tok/s
Observed GPU occupancy: 86.4GiB
First token: 244 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	56.4	36.51 s	244 ms	86.35	319.1	1	791b887d ↗
C4	196.45	41.73 s	654 ms	86.35	315.5	1	791b887d ↗
C8	370.68	44.24 s	1.3 s	86.36	348.1	1	791b887d ↗

NVFP4 · 64K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 44.9tok/s
Best measured · 4 concurrent requests: 96.8tok/s
Observed GPU occupancy: 87.4GiB
First token: 8.6 s

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	44.86	45.67 s	8.6 s	87.37	360.5	1	d2ff32ec ↗
C2	67.59	63.21 s	12.7 s	87.37	379.1	1	d2ff32ec ↗
C3	84.95	84.07 s	14.6 s	87.37	398.5	1	d2ff32ec ↗
C4	96.77	105.62 s	16.3 s	87.37	415.8	1	d2ff32ec ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 54.7tok/s
Best measured · 8 concurrent requests: 340tok/s
Observed GPU occupancy: 87.4GiB
First token: 329 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	54.72	37.56 s	329 ms	87.37	322.4	1	d3f6b770 ↗
C2	99.95	40.99 s	522 ms	87.37	315.9	1	d3f6b770 ↗
C4	189.07	43.34 s	971 ms	87.37	337.8	1	d3f6b770 ↗
C6	264.7	46.44 s	1.3 s	87.37	364	1	d3f6b770 ↗
C7	303.08	47.32 s	1.4 s	87.37	373.4	1	d3f6b770 ↗
C8	339.61	48.27 s	1.7 s	87.37	381.2	1	d3f6b770 ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 54tok/s
Best measured · 8 concurrent requests: 337tok/s
Observed GPU occupancy: 86.4GiB
First token: 361 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	54.04	38.07 s	361 ms	86.36	320.7	1	417f69be ↗
C4	187.12	43.81 s	879 ms	86.36	333.2	1	417f69be ↗
C8	336.95	48.66 s	1.6 s	86.36	379.7	1	417f69be ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 54.5tok/s
Best measured · 8 concurrent requests: 339tok/s
Observed GPU occupancy: 86.6GiB
First token: 359 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	54.54	37.55 s	359 ms	86.63	324.9	1	7d1a7c8e ↗
C4	188.48	43.48 s	878 ms	86.63	350.2	1	7d1a7c8e ↗
C8	338.65	48.43 s	1.6 s	86.63	396.9	1	7d1a7c8e ↗

NVFP4 · 128K → 2K tokens · No cache reuse
Output throughput · includes prefill · 1 request
31.5tok/s
Best measured · 2 concurrent requests
41.3tok/s
Observed GPU occupancy
87.4GiB
First token
24.9 s
Use this recipe · 2 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 concurrency levels
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
vllm · vLLM 0.28.0
Protocol
wbench-search-http-v1
Dataset
migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload
131072in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
C1 31.54 64.93 s 24.9 s 87.37 402.7 1 73fcb7c1 ↗
C2 41.27 111.14 s 34 s 87.37 426.3 1 73fcb7c1 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	31.54	64.93 s	24.9 s	87.37	402.7	1	73fcb7c1 ↗
C2	41.27	111.14 s	34 s	87.37	426.3	1	73fcb7c1 ↗

NVFP4 · 128K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 31.5tok/s
Best measured · 8 concurrent requests: 57.7tok/s
Observed GPU occupancy: 86.6GiB
First token: 24.9 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=isolated:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	31.45	65.12 s	24.9 s	86.63	415.7	1	448b0777 ↗
C4	50.67	220.6 s	45.8 s	86.63	469.4	1	448b0777 ↗
C8	57.74	421.65 s	61.9 s	86.63	494.5	1	448b0777 ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 50tok/s
Best measured · 8 concurrent requests: 287tok/s
Observed GPU occupancy: 86.4GiB
First token: 506 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	49.98	40.98 s	506 ms	86.36	322.7	1	68d1f2c6 ↗
C4	171.33	47.85 s	1 s	86.36	365.5	1	68d1f2c6 ↗
C8	287.31	57.12 s	1.9 s	86.36	425.4	1	68d1f2c6 ↗

Qwen3.8-27B-NVFP4

RadixArk · 14 tested configurations · 2 GPUs

RTX 5090 · 32 GBNVFP4 · 8K → 2K tokens
Output throughput · includes prefill · 1 request
70.8tok/s
Observed GPU occupancy
27.3GiB
First token
633 ms
Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
Test details
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
vllm
Protocol
wbench-fixed-http-v1
Dataset
synthetic-tokenizer-v1:repeats=2
Workload
8192in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
C1 70.82 29.01 s 633 ms 27.29 405.5 1 bad294c0 ↗
1 other test configurations on this GPU
- NVFP4 · 32K → 2K tokens
  Output throughput · includes prefill · 1 request
  62.5tok/s
  Observed GPU occupancy
  27.3GiB
  First token
  3.6 s
  Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=2
  Workload
  32768in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 62.48 32.79 s 3.6 s 27.29 433.3 1 746bbc39 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	70.82	29.01 s	633 ms	27.29	405.5	1	bad294c0 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	62.48	32.79 s	3.6 s	27.29	433.3	1	746bbc39 ↗

RTX PRO 6000 Blackwell Server Edition · 96 GBNVFP4 · 32K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 55.6tok/s
Best measured · 8 concurrent requests: 225tok/s
Observed GPU occupancy: 86GiB
First token: 3.2 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	55.59	36.91 s	3.2 s	85.98	341	1	dca51ce5 ↗
C2	91.37	46.48 s	4.6 s	85.98	342.9	1	dca51ce5 ↗
C4	149.21	62.63 s	6.5 s	85.98	372.9	1	dca51ce5 ↗
C6	196.73	76.24 s	7.8 s	85.98	398.9	1	dca51ce5 ↗
C7	211.87	81.41 s	8.3 s	85.98	408.7	1	dca51ce5 ↗
C8	224.61	90.5 s	8.9 s	85.98	414.8	1	dca51ce5 ↗

11 other test configurations on this GPU

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 60.1tok/s
Best measured · 8 concurrent requests: 393tok/s
Observed GPU occupancy: 86GiB
First token: 217 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	60.1	34.34 s	217 ms	85.98	323	1	9cdf45b7 ↗
C2	109.3	37.48 s	349 ms	85.98	310.8	1	9cdf45b7 ↗
C4	209.23	39.18 s	686 ms	85.98	324.8	1	9cdf45b7 ↗
C6	306.94	40.06 s	992 ms	85.98	336	1	9cdf45b7 ↗
C7	351.24	40.83 s	1.1 s	85.98	342.6	1	9cdf45b7 ↗
C8	392.93	41.71 s	1.3 s	85.98	348.2	1	9cdf45b7 ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 59.7tok/s
Best measured · 8 concurrent requests: 394tok/s
Observed GPU occupancy: 86.8GiB
First token: 230 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	59.68	34.59 s	230 ms	86.82	323.3	1	83d37469 ↗
C4	207.31	39.54 s	664 ms	86.82	320	1	83d37469 ↗
C8	394.04	41.6 s	1.2 s	86.82	346.1	1	83d37469 ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 60.5tok/s
Best measured · 8 concurrent requests: 395tok/s
Observed GPU occupancy: 86.9GiB
First token: 228 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	60.47	33.87 s	228 ms	86.86	320.3	1	cbf1431c ↗
C4	207.86	39.44 s	636 ms	86.86	333.6	1	cbf1431c ↗
C8	394.52	41.6 s	1.3 s	86.86	361.6	1	cbf1431c ↗

NVFP4 · 64K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 47.1tok/s
Best measured · 4 concurrent requests: 101tok/s
Observed GPU occupancy: 86GiB
First token: 8.4 s

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	47.15	43.45 s	8.4 s	85.98	364.4	1	e5a201fa ↗
C2	70.39	60.71 s	12.3 s	85.98	381.3	1	e5a201fa ↗
C3	88.36	80.85 s	14.2 s	85.98	403.6	1	e5a201fa ↗
C4	100.66	101.65 s	15.8 s	85.98	419.2	1	e5a201fa ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.5tok/s
Best measured · 8 concurrent requests: 355tok/s
Observed GPU occupancy: 86GiB
First token: 326 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.45	35.8 s	326 ms	85.98	326.1	1	67a5e6d3 ↗
C2	104.7	39.14 s	515 ms	85.98	321	1	67a5e6d3 ↗
C4	198.72	41.24 s	954 ms	85.98	344.2	1	67a5e6d3 ↗
C6	277.65	44.28 s	1.3 s	85.98	358.8	1	67a5e6d3 ↗
C7	317.41	45.18 s	1.4 s	85.98	369.2	1	67a5e6d3 ↗
C8	354.95	46.19 s	1.6 s	85.98	377.1	1	67a5e6d3 ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.1tok/s
Best measured · 8 concurrent requests: 356tok/s
Observed GPU occupancy: 86.8GiB
First token: 358 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.08	36.15 s	358 ms	86.82	326	1	275a096d ↗
C4	196.91	41.63 s	870 ms	86.83	338.7	1	275a096d ↗
C8	355.89	46.08 s	1.6 s	86.83	375.7	1	275a096d ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.7tok/s
Best measured · 8 concurrent requests: 356tok/s
Observed GPU occupancy: 86.9GiB
First token: 357 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.67	35.51 s	357 ms	86.86	325.8	1	fe8141d2 ↗
C4	197.42	41.52 s	870 ms	86.86	354.3	1	fe8141d2 ↗
C8	356.25	46.05 s	1.6 s	86.86	392.4	1	fe8141d2 ↗

NVFP4 · 128K → 2K tokens · No cache reuse
Output throughput · includes prefill · 1 request
32.7tok/s
Best measured · 2 concurrent requests
42.5tok/s
Observed GPU occupancy
86GiB
First token
24.3 s
Use this recipe · 2 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 concurrency levels
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
vllm · vLLM 0.28.0
Protocol
wbench-search-http-v1
Dataset
migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload
131072in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
C1 32.72 62.59 s 24.3 s 85.99 406.5 1 28f51cf1 ↗
C2 42.54 106.24 s 33.5 s 85.99 428.9 1 28f51cf1 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	32.72	62.59 s	24.3 s	85.99	406.5	1	28f51cf1 ↗
C2	42.54	106.24 s	33.5 s	85.99	428.9	1	28f51cf1 ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 53tok/s
Best measured · 4 concurrent requests: 181tok/s
Observed GPU occupancy: 86GiB
First token: 435 ms

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.97	38.72 s	435 ms	85.99	333.9	1	78fc908c ↗
C2	97.25	42.14 s	646 ms	85.99	340.8	1	78fc908c ↗
C3	138.66	44.36 s	822 ms	85.99	358.8	1	78fc908c ↗
C4	181.16	45.25 s	1.1 s	85.99	375.3	1	78fc908c ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 52.9tok/s
Best measured · 8 concurrent requests: 301tok/s
Observed GPU occupancy: 86.9GiB
First token: 508 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.9	38.72 s	508 ms	86.86	338	1	7cea351c ↗
C4	179.81	45.65 s	1 s	86.86	387.5	1	7cea351c ↗
C8	301.16	54.48 s	1.9 s	86.86	443.7	1	7cea351c ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 52.6tok/s
Best measured · 8 concurrent requests: 301tok/s
Observed GPU occupancy: 86.8GiB
First token: 509 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.59	38.95 s	509 ms	86.83	328.8	1	d14962c1 ↗
C4	179.42	45.72 s	1.1 s	86.83	371.9	1	d14962c1 ↗
C8	300.98	54.52 s	1.9 s	86.83	426	1	d14962c1 ↗

Qwen3-4B-AWQ
Qwen · 5 tested configurations · 3 GPUs
- L4 · 24 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  55.3tok/s
  Best measured · 4 concurrent requests
  109tok/s
  Observed GPU occupancy
  19.5GiB
  First token
  1.6 s
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 55.3 37.17 s 1.6 s 19.54 71.9 1 9301a9ba ↗
  C4 109.43 78.88 s 3.1 s 19.55 71.9 1 9301a9ba ↗
- 1 other test configurations on this GPU
  - AWQ · 8K → 2K tokens · Prefix cache reuse
    Output throughput · includes prefill · 1 request
    57.4tok/s
    Best measured · 4 concurrent requests
    175tok/s
    Observed GPU occupancy
    19.5GiB
    First token
    60 ms
    Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 concurrency levels
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm · vLLM 0.29.0
    Protocol
    wbench-fixed-http-v3
    Dataset
    migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 57.39 35.7 s 60 ms 19.55 71.8 1 7e2cbfda ↗
    C4 175.39 46.75 s 114 ms 19.55 71.9 1 7e2cbfda ↗
- RTX PRO 6000 Blackwell Server Edition · 96 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  215tok/s
  Best measured · 4 concurrent requests
  517tok/s
  Observed GPU occupancy
  86.3GiB
  First token
  298 ms
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 214.6 9.63 s 298 ms 86.28 303.8 1 86d36e4f ↗
  C4 516.94 16.14 s 872 ms 86.28 339.5 1 86d36e4f ↗
- 1 other test configurations on this GPU
  - AWQ · 8K → 2K tokens · Prefix cache reuse
    Output throughput · includes prefill · 1 request
    223tok/s
    Best measured · 4 concurrent requests
    742tok/s
    Observed GPU occupancy
    86.3GiB
    First token
    28 ms
    Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 concurrency levels
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm · vLLM 0.29.0
    Protocol
    wbench-fixed-http-v3
    Dataset
    migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 222.74 9.19 s 28 ms 86.28 288.8 1 adbfef8a ↗
    C4 741.85 11.05 s 35 ms 86.28 335 1 adbfef8a ↗
- T4 · 16 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  15.4tok/s
  Best measured · 4 concurrent requests
  20.3tok/s
  Observed GPU occupancy
  12.7GiB
  First token
  28.5 s
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 15.39 133.36 s 28.5 s 12.74 67.9 1 fcfe2d64 ↗
  C4 20.31 468.13 s 53.1 s 12.74 67.5 1 fcfe2d64 ↗
Qwen3.8-27B-UD-Q4_K_M.gguf
unsloth/Qwen3.8-27B-GGUF · 2 tested configurations · 1 GPU
- L4 · 24 GBQ4_K_M · 8K → 2K tokens
  Output throughput · includes prefill · 1 request
  14.1tok/s
  Observed GPU occupancy
  17GiB
  First token
  279 ms
  Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  llamacpp
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 14.09 145.5 s 279 ms 17.05 71.9 1 4bc32a6c ↗
- 1 other test configurations on this GPU
  - Q4_K_M · 32K → 2K tokens
    Output throughput · includes prefill · 1 request
    12.9tok/s
    Observed GPU occupancy
    17GiB
    First token
    371 ms
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    llamacpp
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    32768in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 12.94 158.32 s 371 ms 17.05 71.8 1 d8a4b66b ↗
Qwen3.8-Flash-Next-NVFP4-NVIDIA
Qwen/Qwen3.8-Flash-Next · 6 tested configurations · 1 GPU
- RTX PRO 6000 Blackwell Server Edition · 96 GBQ4_K_XL · 8K → 2K tokensCPU + GPU hybrid
  Output throughput · includes prefill · 1 request
  125tok/s
  Observed GPU occupancy
  79.7GiB
  First token
  1.8 s
  Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  strata
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 125.06 23.75 s 1.8 s 79.71 279.5 1 18c8631f ↗
- 5 other test configurations on this GPU
  - NVFP4 · 8K → 2K tokensCPU + GPU hybrid
    Output throughput · includes prefill · 1 request
    151tok/s
    Observed GPU occupancy
    71.2GiB
    First token
    1.1 s
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    strata
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 151.39 15.46 s 1.1 s 71.22 292.2 1 60a45fd1 ↗
  - NVFP4 · 8K → 2K tokensCPU + GPU hybrid
    Output throughput · includes prefill · 1 request
    135tok/s
    Observed GPU occupancy
    71.2GiB
    First token
    1.3 s
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    strata
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 134.63 19.87 s 1.3 s 71.22 280 1 f2b52c15 ↗
  - NVFP4 · 32K → 2K tokensCPU + GPU hybrid
    Output throughput · includes prefill · 1 request
    114tok/s
    Observed GPU occupancy
    71.2GiB
    First token
    4.8 s
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    strata
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    32768in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 114.25 20.98 s 4.8 s 71.22 316.1 1 4342000e ↗
  - Q4_K_XL · 32K → 2K tokensCPU + GPU hybrid
    Output throughput · includes prefill · 1 request
    106tok/s
    Observed GPU occupancy
    79.7GiB
    First token
    6.7 s
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    strata
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    32768in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 106.11 23.97 s 6.7 s 79.71 308 1 66369e97 ↗
  - NVFP4 · 32K → 2K tokensCPU + GPU hybrid
    Output throughput · includes prefill · 1 request
    124tok/s
    Observed GPU occupancy
    71.2GiB
    First token
    4.1 s
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    strata
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    32768in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 124.44 18.26 s 4.1 s 71.22 319.5 1 4a41258c ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	55.3	37.17 s	1.6 s	19.54	71.9	1	9301a9ba ↗
C4	109.43	78.88 s	3.1 s	19.55	71.9	1	9301a9ba ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.39	35.7 s	60 ms	19.55	71.8	1	7e2cbfda ↗
C4	175.39	46.75 s	114 ms	19.55	71.9	1	7e2cbfda ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	214.6	9.63 s	298 ms	86.28	303.8	1	86d36e4f ↗
C4	516.94	16.14 s	872 ms	86.28	339.5	1	86d36e4f ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	222.74	9.19 s	28 ms	86.28	288.8	1	adbfef8a ↗
C4	741.85	11.05 s	35 ms	86.28	335	1	adbfef8a ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	15.39	133.36 s	28.5 s	12.74	67.9	1	fcfe2d64 ↗
C4	20.31	468.13 s	53.1 s	12.74	67.5	1	fcfe2d64 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	14.09	145.5 s	279 ms	17.05	71.9	1	4bc32a6c ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	12.94	158.32 s	371 ms	17.05	71.8	1	d8a4b66b ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	125.06	23.75 s	1.8 s	79.71	279.5	1	18c8631f ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	151.39	15.46 s	1.1 s	71.22	292.2	1	60a45fd1 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	134.63	19.87 s	1.3 s	71.22	280	1	f2b52c15 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	114.25	20.98 s	4.8 s	71.22	316.1	1	4342000e ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	106.11	23.97 s	6.7 s	79.71	308	1	66369e97 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	124.44	18.26 s	4.1 s	71.22	319.5	1	4a41258c ↗

Sortable tables by exact test condition · 141 rows

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX 5090 · 32 GB	NVFP4	vllm	70.82	29.01 s	633 ms	27.29	405.5	1	bad294c0 ↗

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-UD-Q4_K_M.gguf	L4 · 24 GB	Q4_K_M	llamacpp	14.09	145.5 s	279 ms	17.05	71.9	1	4bc32a6c ↗

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-Flash-Next-NVFP4-NVIDIA	RTX PRO 6000 Blackwell Server Edition · 96 GB	Q4_K_XL	strata	125.06	23.75 s	1.8 s	79.71	279.5	1	18c8631f ↗

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-Flash-Next-NVFP4-NVIDIA	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	strata	151.39	15.46 s	1.1 s	71.22	292.2	1	60a45fd1 ↗

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-Flash-Next-NVFP4-NVIDIA	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	strata	134.63	19.87 s	1.3 s	71.22	280	1	f2b52c15 ↗

Text generation · 8K → 2K tokens · No cache reuse · 1 request · tok/s · 3 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-4B-AWQ	RTX PRO 6000 Blackwell Server Edition · 96 GB	AWQ	vllmvLLM 0.29.0	214.6	9.63 s	298 ms	86.28	303.8	1	86d36e4f ↗
Qwen3-4B-AWQ	L4 · 24 GB	AWQ	vllmvLLM 0.29.0	55.3	37.17 s	1.6 s	19.54	71.9	1	9301a9ba ↗
Qwen3-4B-AWQ	T4 · 16 GB	AWQ	vllmvLLM 0.29.0	15.39	133.36 s	28.5 s	12.74	67.9	1	fcfe2d64 ↗

Text generation · 8K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 3 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-4B-AWQ	RTX PRO 6000 Blackwell Server Edition · 96 GB	AWQ	vllmvLLM 0.29.0	516.94	16.14 s	872 ms	86.28	339.5	1	86d36e4f ↗
Qwen3-4B-AWQ	L4 · 24 GB	AWQ	vllmvLLM 0.29.0	109.43	78.88 s	3.1 s	19.55	71.9	1	9301a9ba ↗
Qwen3-4B-AWQ	T4 · 16 GB	AWQ	vllmvLLM 0.29.0	20.31	468.13 s	53.1 s	12.74	67.5	1	fcfe2d64 ↗

Text generation · 8K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 2 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-4B-AWQ	RTX PRO 6000 Blackwell Server Edition · 96 GB	AWQ	vllmvLLM 0.29.0	222.74	9.19 s	28 ms	86.28	288.8	1	adbfef8a ↗
Qwen3-4B-AWQ	L4 · 24 GB	AWQ	vllmvLLM 0.29.0	57.39	35.7 s	60 ms	19.55	71.8	1	7e2cbfda ↗

Text generation · 8K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 2 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-4B-AWQ	RTX PRO 6000 Blackwell Server Edition · 96 GB	AWQ	vllmvLLM 0.29.0	741.85	11.05 s	35 ms	86.28	335	1	adbfef8a ↗
Qwen3-4B-AWQ	L4 · 24 GB	AWQ	vllmvLLM 0.29.0	175.39	46.75 s	114 ms	19.55	71.9	1	7e2cbfda ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX 5090 · 32 GB	NVFP4	vllm	62.48	32.79 s	3.6 s	27.29	433.3	1	746bbc39 ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-UD-Q4_K_M.gguf	L4 · 24 GB	Q4_K_M	llamacpp	12.94	158.32 s	371 ms	17.05	71.8	1	d8a4b66b ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-Flash-Next-NVFP4-NVIDIA	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	strata	114.25	20.98 s	4.8 s	71.22	316.1	1	4342000e ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-Flash-Next-NVFP4-NVIDIA	RTX PRO 6000 Blackwell Server Edition · 96 GB	Q4_K_XL	strata	106.11	23.97 s	6.7 s	79.71	308	1	66369e97 ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-Flash-Next-NVFP4-NVIDIA	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	strata	124.44	18.26 s	4.1 s	71.22	319.5	1	4a41258c ↗

Text generation · 32K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	153.44	13.39 s	753 ms	85.25	343.4	1	d2914dd8 ↗

Text generation · 32K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	52.78	38.8 s	3.3 s	87.37	337.3	1	aee8b33e ↗

Text generation · 32K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	55.59	36.91 s	3.2 s	85.98	341	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	208.65	19.64 s	1.1 s	85.25	360.4	1	d2914dd8 ↗

Text generation · 32K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	87.2	48.71 s	4.9 s	87.37	343.4	1	aee8b33e ↗

Text generation · 32K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	91.37	46.48 s	4.6 s	85.98	342.9	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	259.19	33.21 s	1.6 s	85.25	398.9	1	d2914dd8 ↗

Text generation · 32K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	142.14	64.91 s	7 s	87.37	375.3	1	aee8b33e ↗

Text generation · 32K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	149.21	62.63 s	6.5 s	85.98	372.9	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	187.69	79.85 s	8.1 s	87.37	402.8	1	aee8b33e ↗

Text generation · 32K → 2K tokens · No cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	196.73	76.24 s	7.8 s	85.98	398.9	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	202.31	85.21 s	8.7 s	87.37	411.8	1	aee8b33e ↗

Text generation · 32K → 2K tokens · No cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	211.87	81.41 s	8.3 s	85.98	408.7	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	295.37	59.5 s	2.1 s	85.25	422	1	d2914dd8 ↗

Text generation · 32K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	215.05	94.52 s	9.3 s	87.37	421.4	1	aee8b33e ↗

Text generation · 32K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	224.61	90.5 s	8.9 s	85.98	414.8	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 12 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	311.33	86.14 s	2.7 s	85.25	432.4	1	d2914dd8 ↗

Text generation · 32K → 2K tokens · No cache reuse · 16 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	319.17	112.4 s	3.3 s	85.25	438	1	d2914dd8 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	161.57	12.72 s	71 ms	85.25	327.8	1	75198c63 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	56.96	36.17 s	222 ms	87.37	322.6	1	bced0642 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	57.09	35.88 s	231 ms	86.63	319	1	37f29252 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	56.4	36.51 s	244 ms	86.35	319.1	1	791b887d ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	60.1	34.34 s	217 ms	85.98	323	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	59.68	34.59 s	230 ms	86.82	323.3	1	83d37469 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	60.47	33.87 s	228 ms	86.86	320.3	1	cbf1431c ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	269.43	15.21 s	86 ms	85.25	349.1	1	75198c63 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	104.14	39.34 s	355 ms	87.37	305.9	1	bced0642 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	109.3	37.48 s	349 ms	85.98	310.8	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	460.41	17.83 s	106 ms	85.25	427.3	1	75198c63 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	198.42	41.3 s	704 ms	87.37	319.3	1	bced0642 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	197.8	41.44 s	650 ms	86.63	330.2	1	37f29252 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	196.45	41.73 s	654 ms	86.35	315.5	1	791b887d ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	209.23	39.18 s	686 ms	85.98	324.8	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	207.31	39.54 s	664 ms	86.82	320	1	83d37469 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	207.86	39.44 s	636 ms	86.86	333.6	1	cbf1431c ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	291.12	42.23 s	1 s	87.37	342.1	1	bced0642 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	306.94	40.06 s	992 ms	85.98	336	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	333.32	43.04 s	1.1 s	87.37	349.1	1	bced0642 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	351.24	40.83 s	1.1 s	85.98	342.6	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	644.45	25.52 s	127 ms	85.25	486.9	1	75198c63 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	374.04	43.82 s	1.3 s	87.37	353.2	1	bced0642 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	372.88	43.98 s	1.3 s	86.63	367.3	1	37f29252 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	370.68	44.24 s	1.3 s	86.36	348.1	1	791b887d ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	392.93	41.71 s	1.3 s	85.98	348.2	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	394.04	41.6 s	1.2 s	86.82	346.1	1	83d37469 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	394.52	41.6 s	1.3 s	86.86	361.6	1	cbf1431c ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 16 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	771.04	42.77 s	184 ms	85.25	533.9	1	75198c63 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 17 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	812.79	43.14 s	195 ms	85.25	550.4	1	75198c63 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 18 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	856.64	43.41 s	196 ms	85.25	568.9	1	75198c63 ↗

Text generation · 64K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	101.26	20.26 s	2.3 s	85.25	369.1	1	c5c9cf6f ↗

Text generation · 64K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	44.86	45.67 s	8.6 s	87.37	360.5	1	d2ff32ec ↗

Text generation · 64K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	47.15	43.45 s	8.4 s	85.98	364.4	1	e5a201fa ↗

Text generation · 64K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	123.11	33.41 s	3.6 s	85.25	404.8	1	c5c9cf6f ↗

Text generation · 64K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	67.59	63.21 s	12.7 s	87.37	379.1	1	d2ff32ec ↗

Text generation · 64K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	70.39	60.71 s	12.3 s	85.98	381.3	1	e5a201fa ↗

Text generation · 64K → 2K tokens · No cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	84.95	84.07 s	14.6 s	87.37	398.5	1	d2ff32ec ↗

Text generation · 64K → 2K tokens · No cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	88.36	80.85 s	14.2 s	85.98	403.6	1	e5a201fa ↗

Text generation · 64K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	140.19	64.26 s	4.5 s	85.25	430	1	c5c9cf6f ↗

Text generation · 64K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	96.77	105.62 s	16.3 s	87.37	415.8	1	d2ff32ec ↗

Text generation · 64K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	100.66	101.65 s	15.8 s	85.98	419.2	1	e5a201fa ↗

Text generation · 64K → 2K tokens · No cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	142.08	97.79 s	5.3 s	85.25	430.4	1	c5c9cf6f ↗

Text generation · 64K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	150.73	122.37 s	6.3 s	85.25	446.7	1	c5c9cf6f ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	113.51	18.06 s	135 ms	85.25	341.3	1	5fee1315 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	54.72	37.56 s	329 ms	87.37	322.4	1	d3f6b770 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	54.04	38.07 s	361 ms	86.36	320.7	1	417f69be ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	54.54	37.55 s	359 ms	86.63	324.9	1	7d1a7c8e ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	57.45	35.8 s	326 ms	85.98	326.1	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	57.08	36.15 s	358 ms	86.82	326	1	275a096d ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	57.67	35.51 s	357 ms	86.86	325.8	1	fe8141d2 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	153.3	26.73 s	161 ms	85.25	369.5	1	5fee1315 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	99.95	40.99 s	522 ms	87.37	315.9	1	d3f6b770 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	104.7	39.14 s	515 ms	85.98	321	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	281.86	29.1 s	202 ms	85.25	483.8	1	5fee1315 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	189.07	43.34 s	971 ms	87.37	337.8	1	d3f6b770 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	187.12	43.81 s	879 ms	86.36	333.2	1	417f69be ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	188.48	43.48 s	878 ms	86.63	350.2	1	7d1a7c8e ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	198.72	41.24 s	954 ms	85.98	344.2	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	196.91	41.63 s	870 ms	86.83	338.7	1	275a096d ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	197.42	41.52 s	870 ms	86.86	354.3	1	fe8141d2 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	293.74	41.93 s	214 ms	85.25	459.2	1	5fee1315 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	264.7	46.44 s	1.3 s	87.37	364	1	d3f6b770 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	277.65	44.28 s	1.3 s	85.98	358.8	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	340.1	42.27 s	227 ms	85.25	496.1	1	5fee1315 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	303.08	47.32 s	1.4 s	87.37	373.4	1	d3f6b770 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	317.41	45.18 s	1.4 s	85.98	369.2	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	387.93	42.41 s	237 ms	85.25	534.9	1	5fee1315 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	339.61	48.27 s	1.7 s	87.37	381.2	1	d3f6b770 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	336.95	48.66 s	1.6 s	86.36	379.7	1	417f69be ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	338.65	48.43 s	1.6 s	86.63	396.9	1	7d1a7c8e ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	354.95	46.19 s	1.6 s	85.98	377.1	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	355.89	46.08 s	1.6 s	86.83	375.7	1	275a096d ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	356.25	46.05 s	1.6 s	86.86	392.4	1	fe8141d2 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 9 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	433.07	42.78 s	246 ms	85.25	575.8	1	5fee1315 ↗

Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	56.09	36.63 s	8 s	85.25	418.9	1	2c661c7e ↗

Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	31.54	64.93 s	24.9 s	87.37	402.7	1	73fcb7c1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	31.45	65.12 s	24.9 s	86.63	415.7	1	448b0777 ↗

Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	32.72	62.59 s	24.3 s	85.99	406.5	1	28f51cf1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	62.52	67.49 s	12 s	85.25	444.3	1	2c661c7e ↗

Text generation · 128K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	41.27	111.14 s	34 s	87.37	426.3	1	73fcb7c1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	42.54	106.24 s	33.5 s	85.99	428.9	1	28f51cf1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	63.59	109.31 s	13.5 s	85.25	446.8	1	2c661c7e ↗

Text generation · 128K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	66.75	144.63 s	15 s	85.25	461.6	1	2c661c7e ↗

Text generation · 128K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	50.67	220.6 s	45.8 s	86.63	469.4	1	448b0777 ↗

Text generation · 128K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	57.74	421.65 s	61.9 s	86.63	494.5	1	448b0777 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	71.05	28.83 s	268 ms	85.25	356.6	1	1ff0c98a ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	49.98	40.98 s	506 ms	86.36	322.7	1	68d1f2c6 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	52.97	38.72 s	435 ms	85.99	333.9	1	78fc908c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	52.9	38.72 s	508 ms	86.86	338	1	7cea351c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	52.59	38.95 s	509 ms	86.83	328.8	1	d14962c1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	89.43	45.82 s	323 ms	85.25	399.6	1	1ff0c98a ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	97.25	42.14 s	646 ms	85.99	340.8	1	78fc908c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	135.43	45.41 s	379 ms	85.25	470.9	1	1ff0c98a ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	138.66	44.36 s	822 ms	85.99	358.8	1	78fc908c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
WeVisOne-VL-2B	RTX PRO 6000 Blackwell Server Edition · 96 GB	BF16	vllmvLLM 0.28.0	165.29	49.61 s	420 ms	85.25	524	1	1ff0c98a ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	171.33	47.85 s	1 s	86.36	365.5	1	68d1f2c6 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	181.16	45.25 s	1.1 s	85.99	375.3	1	78fc908c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	179.81	45.65 s	1 s	86.86	387.5	1	7cea351c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	179.42	45.72 s	1.1 s	86.83	371.9	1	d14962c1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	287.31	57.12 s	1.9 s	86.36	425.4	1	68d1f2c6 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	301.16	54.48 s	1.9 s	86.86	443.7	1	7cea351c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	300.98	54.52 s	1.9 s	86.83	426	1	d14962c1 ↗

Models with reports but no qualified results · 131

Their original reports remain available on each model page. They did not qualify for comparison; each report states why.

Models with reports but no qualified results · 131 rows
Model	Test task
AutoGLM-Phone-9BZhipuAI/AutoGLM-Phone-9B	Text generation
DASD-4B-ThinkingAlibaba-Apsara/DASD-4B-Thinking	Text generation
DeepSeek-OCR-2deepseek-ai/DeepSeek-OCR-2	Text generation
DeepSeek-R1-Distill-Llama-8Bdeepseek-ai/DeepSeek-R1-Distill-Llama-8B	Text generation
DeepSeek-R1-Distill-Qwen-1.5Bdeepseek-ai/DeepSeek-R1-Distill-Qwen-1.5B	Text generation
DeepSeek-R1-Distill-Qwen-1.5B/root/autodl-tmp/.cache/modelscope/models/deepseek-ai--DeepSeek-R1-Distill-Qwen-1.5B/snapshots/master	Text generation
DeepSeek-R1-Distill-Qwen-7Bdeepseek-ai/DeepSeek-R1-Distill-Qwen-7B	Text generation
DeepSeek-R1-Qwen3-8Bdeepseek-ai/DeepSeek-R1-0528-Qwen3-8B	Text generation
dots.ocrrednote-hilab/dots.ocr	Text generation
Gemma-3-270mgoogle/gemma-3-270m	Text generation
Hunyuan-1.8B-FP8/root/.cache/modelscope/hub/models/Tencent-Hunyuan/HY-MT1___5-1___8B-FP8	Text generation
Hunyuan-1.8B-FP8/home/tiger/.cache/modelscope/hub/models/Tencent-Hunyuan/HY-MT1___5-1___8B-FP8	Text generation
Hunyuan-7B-FP8/root/.cache/modelscope/hub/models/Tencent-Hunyuan/HY-MT1___5-7B-FP8	Text generation
Hunyuan-MT2-1.8B-FP8Tencent-Hunyuan/Hy-MT2-1.8B-FP8	Text generation
Hunyuan-MT2-7B-FP8Tencent-Hunyuan/Hy-MT2-7B-FP8	Text generation
HunyuanOCRTencent-Hunyuan/HunyuanOCR	Text generation
HY-MT1.5-1.8B-Q4_K_M/root/.cache/modelscope/hub/models/Tencent-Hunyuan/HY-MT1___5-1___8B-GGUF/HY-MT1.5-1.8B-Q4_K_M.gguf	Text generation
HY-MT1.5-1.8B-Q6_KTencent-Hunyuan/HY-MT1.5-1.8B-GGUF/HY-MT1.5-1.8B-Q6_K.gguf	Text generation
HY-MT1.5-1.8B-Q8_0Tencent-Hunyuan/HY-MT1.5-1.8B-GGUF/HY-MT1.5-1.8B-Q8_0.gguf	Text generation
HY-MT1.5-7B-Q4_K_M/root/.cache/modelscope/hub/models/Tencent-Hunyuan/HY-MT1___5-7B-GGUF/HY-MT1.5-7B-Q4_K_M.gguf	Text generation
HY-MT1.5-7B-Q6_KTencent-Hunyuan/HY-MT1.5-7B-GGUF/HY-MT1.5-7B-Q6_K.gguf	Text generation
HY-MT1.5-7B-Q8_0Tencent-Hunyuan/HY-MT1.5-7B-GGUF/HY-MT1.5-7B-Q8_0.gguf	Text generation
LightOnOCR-2-1Blightonai/LightOnOCR-2-1B	Text generation
Llama-3.1-8B-Instruct-FP8nvidia/Llama-3.1-8B-Instruct-FP8	Text generation
Llama-3.1-8B-Instruct-NVFP4nvidia/Llama-3.1-8B-Instruct-NVFP4	Text generation
MAI-UI-8BTongyi-MAI/MAI-UI-8B	Text generation
MiniCPM-o-4_5openbmb/MiniCPM-o-4_5	Text generation
MiniCPM-V-4.7-35B-A3B-NVFP4sambit9238/MiniCPM-V-4.7-35B-A3B-NVFP4	Text generation
Ministral-3-14B-Instruct-2512mistralai/Ministral-3-14B-Instruct-2512	Text generation
Ministral-3-14B-Instruct-2512-FP8mistralai/Ministral-3-14B-Instruct-2512-FP8	Text generation
Ministral-3-3B-Instruct-2512mistralai/Ministral-3-3B-Instruct-2512	Text generation
Ministral-3-8B-Instruct-2512mistralai/Ministral-3-8B-Instruct-2512	Text generation
Nanbeige4-3B-Thinking-2511Nanbeige/Nanbeige4-3B-Thinking-2511	Text generation
NVIDIA-Nemotron-3-Nano-4B-FP8nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8	Text generation
NVIDIA-Nemotron-Nano-12B-v2-VL-FP8nvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-FP8	Text generation
NVIDIA-Nemotron-Nano-12B-v2-VL-NVFP4-QADnvidia/NVIDIA-Nemotron-Nano-12B-v2-VL-NVFP4-QAD	Text generation
NVIDIA-Nemotron-Nano-9B-v2-FP8nvidia/NVIDIA-Nemotron-Nano-9B-v2-FP8	Text generation
NVIDIA-Nemotron-Nano-9B-v2-NVFP4nvidia/NVIDIA-Nemotron-Nano-9B-v2-NVFP4	Text generation
PaddleOCR-VL-1.5PaddlePaddle/PaddleOCR-VL-1.5	Text generation
Qwen2.5-0.5Bqwen2.5:0.5b	Text generation
Qwen2.5-0.5B-Instruct-AWQ/root/.cache/modelscope/hub/models/Qwen/Qwen2___5-0___5B-Instruct-AWQ	Text generation
qwen2.5-0.5b-instruct-q4_k_m.ggufQwen/Qwen2.5-0.5B-Instruct-GGUF/qwen2.5-0.5b-instruct-q4_k_m.gguf	Text generation
Qwen2.5-1.5Bqwen2.5:1.5b	Text generation
Qwen2.5-1.5B-Instruct-AWQ/root/.cache/modelscope/hub/models/Qwen/Qwen2___5-1___5B-Instruct-AWQ	Text generation
qwen2.5-1.5b-instruct-q4_k_m.ggufQwen/Qwen2.5-1.5B-Instruct-GGUF/qwen2.5-1.5b-instruct-q4_k_m.gguf	Text generation
Qwen2.5-14B-AWQ/home/u20250324/.cache/modelscope/hub/models/Qwen/Qwen2___5-14B-Instruct-AWQ	Text generation
Qwen2.5-14B-Q4_K_MQwen/Qwen2.5-14B-Instruct-GGUF/qwen2.5-14b-instruct-q4_k_m.gguf	Text generation
Qwen2.5-32B-Q4_K_MQwen/Qwen2.5-32B-Instruct-GGUF/qwen2.5-32b-instruct-q4_k_m.gguf	Text generation
Qwen2.5-3Bqwen2.5:3b	Text generation
Qwen2.5-3B-AWQ/home/u20250324/.cache/modelscope/hub/models/Qwen/Qwen2___5-3B-Instruct-AWQ	Text generation
Qwen2.5-3B-InstructQwen/Qwen2.5-3B-Instruct	Text generation
qwen2.5-3b-instruct-q4_k_m.ggufQwen/Qwen2.5-3B-Instruct-GGUF/qwen2.5-3b-instruct-q4_k_m.gguf	Text generation
Qwen2.5-72B-AWQQwen/Qwen2.5-72B-Instruct-AWQ	Text generation
Qwen2.5-7Bqwen2.5:7b	Text generation
Qwen2.5-7B-AWQ/home/u20250324/.cache/modelscope/hub/models/Qwen/Qwen2___5-7B-Instruct-AWQ	Text generation
Qwen2.5-7B-Instruct-AWQ/root/.cache/modelscope/hub/models/Qwen/Qwen2___5-7B-Instruct-AWQ	Text generation
Qwen2.5-7B-Q4_K_MQwen/Qwen2.5-7B-Instruct-GGUF/qwen2.5-7b-instruct-q4_k_m.gguf	Text generation
Qwen3-0.6BQwen/Qwen3-0.6B	Text generation
Qwen3-0.6B/root/autodl-tmp/.cache/modelscope/models/Qwen--Qwen3-0.6B/snapshots/master	Text generation
Qwen3-0.6B/root/.cache/modelscope/models/Qwen--Qwen3-0.6B/snapshots/master	Text generation
Qwen3-0.6B-FP8Qwen/Qwen3-0.6B-FP8	Text generation
Qwen3-0.6B-FP8/root/autodl-tmp/.cache/modelscope/models/Qwen--Qwen3-0.6B-FP8/snapshots/master	Text generation
Qwen3-0.6B-GPTQ/root/autodl-tmp/.cache/modelscope/models/Qwen--Qwen3-0.6B-GPTQ-Int8/snapshots/master	Text generation
Qwen3-0.6B-GPTQ-Int8Qwen/Qwen3-0.6B-GPTQ-Int8	Text generation
Qwen3-0.6B-Q8Qwen/Qwen3-0.6B-GGUF/Qwen3-0.6B-Q8_0.gguf	Text generation
Qwen3-0.6B-Q8/root/.cache/modelscope/models/Qwen--Qwen3-0.6B-GGUF/snapshots/master/Qwen3-0.6B-Q8_0.gguf	Text generation
Qwen3-1.7BQwen/Qwen3-1.7B	Text generation
Qwen3-1.7B-FP8Qwen/Qwen3-1.7B-FP8	Text generation
Qwen3-1.7B-Q8Qwen/Qwen3-1.7B-GGUF/Qwen3-1.7B-Q8_0.gguf	Text generation
Qwen3-14B/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-14B	Text generation
Qwen3-14B-AWQQwen/Qwen3-14B-AWQ	Text generation
Qwen3-14B-FP8Qwen/Qwen3-14B-FP8	Text generation
Qwen3-14B-NVFP4RedHatAI/Qwen3-14B-NVFP4	Text generation
Qwen3-14B-Q4Qwen/Qwen3-14B-GGUF/Qwen3-14B-Q4_K_M.gguf	Text generation
Qwen3-14B-Q5_K_MQwen/Qwen3-14B-GGUF/Qwen3-14B-Q5_K_M.gguf	Text generation
Qwen3-14B-Q8_0Qwen/Qwen3-14B-GGUF/Qwen3-14B-Q8_0.gguf	Text generation
Qwen3-30B-A3B/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-30B-A3B	Text generation
Qwen3-30B-A3B-FP8Qwen/Qwen3-30B-A3B-Instruct-2507-FP8	Text generation
Qwen3-30B-A3B-FP8/root/.cache/modelscope/hub/models/Qwen/Qwen3-30B-A3B-FP8	Text generation
Qwen3-30B-A3B-Q4_K_MQwen/Qwen3-30B-A3B-GGUF/Qwen3-30B-A3B-Q4_K_M.gguf	Text generation
Qwen3-30B-A3B-Q8_0Qwen/Qwen3-30B-A3B-GGUF/Qwen3-30B-A3B-Q8_0.gguf	Text generation
Qwen3-30B-A3B-Thinking-FP8Qwen/Qwen3-30B-A3B-Thinking-2507-FP8	Text generation
Qwen3-32B/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-32B	Text generation
Qwen3-32B-AWQQwen/Qwen3-32B-AWQ	Text generation
Qwen3-32B-FP8Qwen/Qwen3-32B-FP8	Text generation
Qwen3-32B-Q4Qwen/Qwen3-32B-GGUF/Qwen3-32B-Q4_K_M.gguf	Text generation
Qwen3-32B-Q5_K_MQwen/Qwen3-32B-GGUF/Qwen3-32B-Q5_K_M.gguf	Text generation
Qwen3-4BQwen/Qwen3-4B	Text generation
Qwen3-4B-FP8Qwen/Qwen3-4B-FP8	Text generation
Qwen3-4B-Instruct-2507-FP8Qwen/Qwen3-4B-Instruct-2507-FP8	Text generation
Qwen3-4B-Q4Qwen/Qwen3-4B-GGUF/Qwen3-4B-Q4_K_M.gguf	Text generation
Qwen3-4B-Q5_K_MQwen/Qwen3-4B-GGUF/Qwen3-4B-Q5_K_M.gguf	Text generation
Qwen3-4B-Q8_0Qwen/Qwen3-4B-GGUF/Qwen3-4B-Q8_0.gguf	Text generation
Qwen3-4B-Thinking-2507-FP8Qwen/Qwen3-4B-Thinking-2507-FP8	Text generation
Qwen3-8BQwen/Qwen3-8B	Text generation
Qwen3-8B/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-8B	Text generation
Qwen3-8B-AWQQwen/Qwen3-8B-AWQ	Text generation
Qwen3-8B-FP8Qwen/Qwen3-8B-FP8	Text generation
Qwen3-8B-NVFP4RedHatAI/Qwen3-8B-NVFP4	Text generation
Qwen3-8B-Q4Qwen/Qwen3-8B-GGUF/Qwen3-8B-Q4_K_M.gguf	Text generation
Qwen3-8B-Q5_K_MQwen/Qwen3-8B-GGUF/Qwen3-8B-Q5_K_M.gguf	Text generation
Qwen3-8B-Q8_0Qwen/Qwen3-8B-GGUF/Qwen3-8B-Q8_0.gguf	Text generation
Qwen3-VL-2B/root/.cache/huggingface/hub/models--Qwen--Qwen3-VL-2B-Instruct/snapshots/89644892e4d85e24eaac8bacfd4f463576704203	Text generation
Qwen3-VL-2B/home/obscura/.cache/modelscope/hub/models/Qwen/Qwen3-VL-2B-Instruct	Text generation
Qwen3-VL-2B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-2B-Instruct	Text generation
Qwen3-VL-2B-Instruct-FP8Qwen/Qwen3-VL-2B-Instruct-FP8	Text generation
Qwen3-VL-30B-A3B-FP8Qwen/Qwen3-VL-30B-A3B-Instruct-FP8	Text generation
Qwen3-VL-30B-A3B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-30B-A3B-Instruct	Text generation
Qwen3-VL-32B-FP8Qwen/Qwen3-VL-32B-Instruct-FP8	Text generation
Qwen3-VL-32B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-32B-Instruct	Text generation
Qwen3-VL-4B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-4B-Instruct	Text generation
Qwen3-VL-4B-Instruct-FP8Qwen/Qwen3-VL-4B-Instruct-FP8	Text generation
Qwen3-VL-8B-Instruct/home/lyx/.cache/modelscope/hub/models/Qwen/Qwen3-VL-8B-Instruct	Text generation
Qwen3-VL-8B-Instruct-FP8Qwen/Qwen3-VL-8B-Instruct-FP8	Text generation
qwen3:0.6bqwen3:0.6b	Text generation
Qwen3.5-0.8BQwen/Qwen3.5-0.8B	Text generation
Qwen3.5-0.8B/root/autodl-tmp/.cache/modelscope/models/Qwen--Qwen3.5-0.8B/snapshots/master	Text generation
Qwen3.5-0.8B/root/.cache/modelscope/models/Qwen--Qwen3.5-0.8B/snapshots/master	Text generation
Qwen3.5-27B-FP8Qwen/Qwen3.5-27B-FP8	Text generation
Qwen3.5-2BQwen/Qwen3.5-2B	Text generation
Qwen3.5-2B/root/.cache/modelscope/models/Qwen--Qwen3.5-2B/snapshots/master	Text generation
Qwen3.5-4BQwen/Qwen3.5-4B	Text generation
Qwen3.5-4B/root/.cache/modelscope/models/Qwen--Qwen3.5-4B/snapshots/master	Text generation
Qwen3.5-9BQwen/Qwen3.5-9B	Text generation
Qwen3.6-27B-AWQ/root/.cache/modelscope/models/tclf90--Qwen3.6-27B-AWQ/snapshots/master	Text generation
Qwen3.6-35B-A3B-AWQ-4bitcyankiwi/Qwen3.6-35B-A3B-AWQ-4bit	Text generation
Qwen3.6-35B-A3B-FP8Qwen/Qwen3.6-35B-A3B-FP8	Text generation
Qwen3.8-27B-NVFP4hf:nvidia/Qwen3.8-27B-NVFP4	Text generation
Qwen3.8-27B-UD-Q4_K_M.ggufhf:unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.gguf	Text generation
Step3-VL-10Bstepfun-ai/Step3-VL-10B	Text generation
testtest	Text generation

Get your startup command

Text generation · All measured GPUs

WeVisOne-VL-2B

Qwen3.8-27B-NVFP4

Qwen3.8-27B-NVFP4

Qwen3-4B-AWQ

Qwen3.8-27B-UD-Q4_K_M.gguf

Qwen3.8-Flash-Next-NVFP4-NVIDIA