1 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Run this model

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

Text generation · All measured GPUs

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

Qwen3.8-27B-NVFP4

RadixArk · 14 tested configurations · 2 GPUs

RTX 5090 · 32 GBNVFP4 · 8K → 2K tokens
Output throughput · includes prefill · 1 request
70.8tok/s
Observed GPU occupancy
27.3GiB
First token
633 ms
Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
Test details
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
vllm
Protocol
wbench-fixed-http-v1
Dataset
synthetic-tokenizer-v1:repeats=2
Workload
8192in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
C1 70.82 29.01 s 633 ms 27.29 405.5 1 bad294c0 ↗
1 other test configurations on this GPU
- NVFP4 · 32K → 2K tokens
  Output throughput · includes prefill · 1 request
  62.5tok/s
  Observed GPU occupancy
  27.3GiB
  First token
  3.6 s
  Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=2
  Workload
  32768in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 62.48 32.79 s 3.6 s 27.29 433.3 1 746bbc39 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	70.82	29.01 s	633 ms	27.29	405.5	1	bad294c0 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	62.48	32.79 s	3.6 s	27.29	433.3	1	746bbc39 ↗

RTX PRO 6000 Blackwell Server Edition · 96 GBNVFP4 · 32K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 55.6tok/s
Best measured · 8 concurrent requests: 225tok/s
Observed GPU occupancy: 86GiB
First token: 3.2 s

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	55.59	36.91 s	3.2 s	85.98	341	1	dca51ce5 ↗
C2	91.37	46.48 s	4.6 s	85.98	342.9	1	dca51ce5 ↗
C4	149.21	62.63 s	6.5 s	85.98	372.9	1	dca51ce5 ↗
C6	196.73	76.24 s	7.8 s	85.98	398.9	1	dca51ce5 ↗
C7	211.87	81.41 s	8.3 s	85.98	408.7	1	dca51ce5 ↗
C8	224.61	90.5 s	8.9 s	85.98	414.8	1	dca51ce5 ↗

11 other test configurations on this GPU

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 60.1tok/s
Best measured · 8 concurrent requests: 393tok/s
Observed GPU occupancy: 86GiB
First token: 217 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	60.1	34.34 s	217 ms	85.98	323	1	9cdf45b7 ↗
C2	109.3	37.48 s	349 ms	85.98	310.8	1	9cdf45b7 ↗
C4	209.23	39.18 s	686 ms	85.98	324.8	1	9cdf45b7 ↗
C6	306.94	40.06 s	992 ms	85.98	336	1	9cdf45b7 ↗
C7	351.24	40.83 s	1.1 s	85.98	342.6	1	9cdf45b7 ↗
C8	392.93	41.71 s	1.3 s	85.98	348.2	1	9cdf45b7 ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 59.7tok/s
Best measured · 8 concurrent requests: 394tok/s
Observed GPU occupancy: 86.8GiB
First token: 230 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	59.68	34.59 s	230 ms	86.82	323.3	1	83d37469 ↗
C4	207.31	39.54 s	664 ms	86.82	320	1	83d37469 ↗
C8	394.04	41.6 s	1.2 s	86.82	346.1	1	83d37469 ↗

NVFP4 · 32K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 60.5tok/s
Best measured · 8 concurrent requests: 395tok/s
Observed GPU occupancy: 86.9GiB
First token: 228 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 32768in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	60.47	33.87 s	228 ms	86.86	320.3	1	cbf1431c ↗
C4	207.86	39.44 s	636 ms	86.86	333.6	1	cbf1431c ↗
C8	394.52	41.6 s	1.3 s	86.86	361.6	1	cbf1431c ↗

NVFP4 · 64K → 2K tokens · No cache reuse

Output throughput · includes prefill · 1 request: 47.1tok/s
Best measured · 4 concurrent requests: 101tok/s
Observed GPU occupancy: 86GiB
First token: 8.4 s

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	47.15	43.45 s	8.4 s	85.98	364.4	1	e5a201fa ↗
C2	70.39	60.71 s	12.3 s	85.98	381.3	1	e5a201fa ↗
C3	88.36	80.85 s	14.2 s	85.98	403.6	1	e5a201fa ↗
C4	100.66	101.65 s	15.8 s	85.98	419.2	1	e5a201fa ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.5tok/s
Best measured · 8 concurrent requests: 355tok/s
Observed GPU occupancy: 86GiB
First token: 326 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 6 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.45	35.8 s	326 ms	85.98	326.1	1	67a5e6d3 ↗
C2	104.7	39.14 s	515 ms	85.98	321	1	67a5e6d3 ↗
C4	198.72	41.24 s	954 ms	85.98	344.2	1	67a5e6d3 ↗
C6	277.65	44.28 s	1.3 s	85.98	358.8	1	67a5e6d3 ↗
C7	317.41	45.18 s	1.4 s	85.98	369.2	1	67a5e6d3 ↗
C8	354.95	46.19 s	1.6 s	85.98	377.1	1	67a5e6d3 ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.1tok/s
Best measured · 8 concurrent requests: 356tok/s
Observed GPU occupancy: 86.8GiB
First token: 358 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.08	36.15 s	358 ms	86.82	326	1	275a096d ↗
C4	196.91	41.63 s	870 ms	86.83	338.7	1	275a096d ↗
C8	355.89	46.08 s	1.6 s	86.83	375.7	1	275a096d ↗

NVFP4 · 64K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 57.7tok/s
Best measured · 8 concurrent requests: 356tok/s
Observed GPU occupancy: 86.9GiB
First token: 357 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 65536in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.67	35.51 s	357 ms	86.86	325.8	1	fe8141d2 ↗
C4	197.42	41.52 s	870 ms	86.86	354.3	1	fe8141d2 ↗
C8	356.25	46.05 s	1.6 s	86.86	392.4	1	fe8141d2 ↗

NVFP4 · 128K → 2K tokens · No cache reuse
Output throughput · includes prefill · 1 request
32.7tok/s
Best measured · 2 concurrent requests
42.5tok/s
Observed GPU occupancy
86GiB
First token
24.3 s
Use this recipe · 2 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 concurrency levels
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
vllm · vLLM 0.28.0
Protocol
wbench-search-http-v1
Dataset
migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
Workload
131072in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
C1 32.72 62.59 s 24.3 s 85.99 406.5 1 28f51cf1 ↗
C2 42.54 106.24 s 33.5 s 85.99 428.9 1 28f51cf1 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	32.72	62.59 s	24.3 s	85.99	406.5	1	28f51cf1 ↗
C2	42.54	106.24 s	33.5 s	85.99	428.9	1	28f51cf1 ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 53tok/s
Best measured · 4 concurrent requests: 181tok/s
Observed GPU occupancy: 86GiB
First token: 435 ms

Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-search-http-v1
Dataset: migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.97	38.72 s	435 ms	85.99	333.9	1	78fc908c ↗
C2	97.25	42.14 s	646 ms	85.99	340.8	1	78fc908c ↗
C3	138.66	44.36 s	822 ms	85.99	358.8	1	78fc908c ↗
C4	181.16	45.25 s	1.1 s	85.99	375.3	1	78fc908c ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 52.9tok/s
Best measured · 8 concurrent requests: 301tok/s
Observed GPU occupancy: 86.9GiB
First token: 508 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.9	38.72 s	508 ms	86.86	338	1	7cea351c ↗
C4	179.81	45.65 s	1 s	86.86	387.5	1	7cea351c ↗
C8	301.16	54.48 s	1.9 s	86.86	443.7	1	7cea351c ↗

NVFP4 · 128K → 2K tokens · Prefix cache reuse

Output throughput · includes prefill · 1 request: 52.6tok/s
Best measured · 8 concurrent requests: 301tok/s
Observed GPU occupancy: 86.8GiB
First token: 509 ms

Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 concurrency levels

Total output tokens divided by the full request window, including prefill and queueing.

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: vllm · vLLM 0.28.0
Protocol: wbench-fixed-http-v3
Dataset: synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
Workload: 131072in-2048out

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	52.59	38.95 s	509 ms	86.83	328.8	1	d14962c1 ↗
C4	179.42	45.72 s	1.1 s	86.83	371.9	1	d14962c1 ↗
C8	300.98	54.52 s	1.9 s	86.83	426	1	d14962c1 ↗

Sortable tables by exact test condition · 48 rows

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX 5090 · 32 GB	NVFP4	vllm	70.82	29.01 s	633 ms	27.29	405.5	1	bad294c0 ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX 5090 · 32 GB	NVFP4	vllm	62.48	32.79 s	3.6 s	27.29	433.3	1	746bbc39 ↗

Text generation · 32K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	55.59	36.91 s	3.2 s	85.98	341	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	91.37	46.48 s	4.6 s	85.98	342.9	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	149.21	62.63 s	6.5 s	85.98	372.9	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	196.73	76.24 s	7.8 s	85.98	398.9	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	211.87	81.41 s	8.3 s	85.98	408.7	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	224.61	90.5 s	8.9 s	85.98	414.8	1	dca51ce5 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	60.1	34.34 s	217 ms	85.98	323	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	59.68	34.59 s	230 ms	86.82	323.3	1	83d37469 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	60.47	33.87 s	228 ms	86.86	320.3	1	cbf1431c ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	109.3	37.48 s	349 ms	85.98	310.8	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	209.23	39.18 s	686 ms	85.98	324.8	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	207.31	39.54 s	664 ms	86.82	320	1	83d37469 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	207.86	39.44 s	636 ms	86.86	333.6	1	cbf1431c ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	306.94	40.06 s	992 ms	85.98	336	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	351.24	40.83 s	1.1 s	85.98	342.6	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	392.93	41.71 s	1.3 s	85.98	348.2	1	9cdf45b7 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	394.04	41.6 s	1.2 s	86.82	346.1	1	83d37469 ↗

Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	394.52	41.6 s	1.3 s	86.86	361.6	1	cbf1431c ↗

Text generation · 64K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	47.15	43.45 s	8.4 s	85.98	364.4	1	e5a201fa ↗

Text generation · 64K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	70.39	60.71 s	12.3 s	85.98	381.3	1	e5a201fa ↗

Text generation · 64K → 2K tokens · No cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	88.36	80.85 s	14.2 s	85.98	403.6	1	e5a201fa ↗

Text generation · 64K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	100.66	101.65 s	15.8 s	85.98	419.2	1	e5a201fa ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	57.45	35.8 s	326 ms	85.98	326.1	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	57.08	36.15 s	358 ms	86.82	326	1	275a096d ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	57.67	35.51 s	357 ms	86.86	325.8	1	fe8141d2 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	104.7	39.14 s	515 ms	85.98	321	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	198.72	41.24 s	954 ms	85.98	344.2	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	196.91	41.63 s	870 ms	86.83	338.7	1	275a096d ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	197.42	41.52 s	870 ms	86.86	354.3	1	fe8141d2 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	277.65	44.28 s	1.3 s	85.98	358.8	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	317.41	45.18 s	1.4 s	85.98	369.2	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	354.95	46.19 s	1.6 s	85.98	377.1	1	67a5e6d3 ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	355.89	46.08 s	1.6 s	86.83	375.7	1	275a096d ↗

Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	356.25	46.05 s	1.6 s	86.86	392.4	1	fe8141d2 ↗

Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	32.72	62.59 s	24.3 s	85.99	406.5	1	28f51cf1 ↗

Text generation · 128K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	42.54	106.24 s	33.5 s	85.99	428.9	1	28f51cf1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	52.97	38.72 s	435 ms	85.99	333.9	1	78fc908c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	52.9	38.72 s	508 ms	86.86	338	1	7cea351c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	52.59	38.95 s	509 ms	86.83	328.8	1	d14962c1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	97.25	42.14 s	646 ms	85.99	340.8	1	78fc908c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 3 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	138.66	44.36 s	822 ms	85.99	358.8	1	78fc908c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	181.16	45.25 s	1.1 s	85.99	375.3	1	78fc908c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	179.81	45.65 s	1 s	86.86	387.5	1	7cea351c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	179.42	45.72 s	1.1 s	86.83	371.9	1	d14962c1 ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	301.16	54.48 s	1.9 s	86.86	443.7	1	7cea351c ↗

Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
Model	Hardware	Format	Engine	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3.8-27B-NVFP4	RTX PRO 6000 Blackwell Server Edition · 96 GB	NVFP4	vllmvLLM 0.28.0	300.98	54.52 s	1.9 s	86.83	426	1	d14962c1 ↗