96 GB · 28 reports

RTX PRO 6000 Blackwell Workstation Edition · 96 GB

Qualified results first; original reports and full tables stay below.

Text generation · Qualified results

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

No qualified results yet · Text generation · RTX PRO 6000 Blackwell Workstation Edition · 96 GB

4 original reports are kept as evidence but do not qualify for comparison; each report states why. This does not show that the model or GPU cannot run it.

Check model launch support View original reports Measured alternatives for this task Test your own GPU

Measured alternatives on other GPUs

Measured on different GPUs with the same or smaller listed capacity. These are references for choosing hardware, not performance or compatibility evidence for your selected GPU.

Qwen3-4B-AWQ
Qwen · 5 tested configurations · 3 GPUs
- L4 · 24 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  55.3tok/s
  Best measured · 4 concurrent requests
  109tok/s
  Observed GPU occupancy
  19.5GiB
  First token
  1.6 s
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 55.3 37.17 s 1.6 s 19.54 71.9 1 9301a9ba ↗
  C4 109.43 78.88 s 3.1 s 19.55 71.9 1 9301a9ba ↗
- 1 other test configurations on this GPU
  - AWQ · 8K → 2K tokens · Prefix cache reuse
    Output throughput · includes prefill · 1 request
    57.4tok/s
    Best measured · 4 concurrent requests
    175tok/s
    Observed GPU occupancy
    19.5GiB
    First token
    60 ms
    Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 concurrency levels
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm · vLLM 0.29.0
    Protocol
    wbench-fixed-http-v3
    Dataset
    migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 57.39 35.7 s 60 ms 19.55 71.8 1 7e2cbfda ↗
    C4 175.39 46.75 s 114 ms 19.55 71.9 1 7e2cbfda ↗
Show 2 more GPUs
- RTX PRO 6000 Blackwell Server Edition · 96 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  215tok/s
  Best measured · 4 concurrent requests
  517tok/s
  Observed GPU occupancy
  86.3GiB
  First token
  298 ms
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 214.6 9.63 s 298 ms 86.28 303.8 1 86d36e4f ↗
  C4 516.94 16.14 s 872 ms 86.28 339.5 1 86d36e4f ↗
- 1 other test configurations on this GPU
  - AWQ · 8K → 2K tokens · Prefix cache reuse
    Output throughput · includes prefill · 1 request
    223tok/s
    Best measured · 4 concurrent requests
    742tok/s
    Observed GPU occupancy
    86.3GiB
    First token
    28 ms
    Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 concurrency levels
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm · vLLM 0.29.0
    Protocol
    wbench-fixed-http-v3
    Dataset
    migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 222.74 9.19 s 28 ms 86.28 288.8 1 adbfef8a ↗
    C4 741.85 11.05 s 35 ms 86.28 335 1 adbfef8a ↗
- T4 · 16 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  15.4tok/s
  Best measured · 4 concurrent requests
  20.3tok/s
  Observed GPU occupancy
  12.7GiB
  First token
  28.5 s
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 15.39 133.36 s 28.5 s 12.74 67.9 1 fcfe2d64 ↗
  C4 20.31 468.13 s 53.1 s 12.74 67.5 1 fcfe2d64 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	55.3	37.17 s	1.6 s	19.54	71.9	1	9301a9ba ↗
C4	109.43	78.88 s	3.1 s	19.55	71.9	1	9301a9ba ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.39	35.7 s	60 ms	19.55	71.8	1	7e2cbfda ↗
C4	175.39	46.75 s	114 ms	19.55	71.9	1	7e2cbfda ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	214.6	9.63 s	298 ms	86.28	303.8	1	86d36e4f ↗
C4	516.94	16.14 s	872 ms	86.28	339.5	1	86d36e4f ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	222.74	9.19 s	28 ms	86.28	288.8	1	adbfef8a ↗
C4	741.85	11.05 s	35 ms	86.28	335	1	adbfef8a ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	15.39	133.36 s	28.5 s	12.74	67.9	1	fcfe2d64 ↗
C4	20.31	468.13 s	53.1 s	12.74	67.5	1	fcfe2d64 ↗

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Measurements without a comparable protocol · 34 rows

These measured rates remain visible, but are excluded from comparison curves because the protocol is unknown.

Measurements without a comparable protocol · 34 rows
Model	Format	Engine	Workload	Concurrency	Throughput	GPU power (W)
Qwen3-4B ↗	Native	vLLM	2K	1	137.3 tok/s	495.88
Qwen3-4B ↗	Native	vLLM	2K	8	922.46 tok/s	385.95
Qwen3-4B ↗	Native	vLLM	2K	16	1,689.63 tok/s	423.75
Qwen3-4B ↗	Native	vLLM	2K	32	2,609.22 tok/s	438.64
Qwen3-4B ↗	Native	vLLM	2K	64	4,529.36 tok/s	495.93
Qwen3-4B ↗	Native	vLLM	2K	128	5,700.05 tok/s	502.06
Qwen3-4B ↗	Native	vLLM	2K	256	6,245.25 tok/s	545.8
Qwen3-4B ↗	Native	vLLM	2K	512	7,662.98 tok/s	526.72
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	123.74 tok/s	339.99
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	900.5 tok/s	348.42
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	16	1,689.18 tok/s	374.68
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	2,461.61 tok/s	357.75
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	64	4,589.22 tok/s	386.08
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	5,914.47 tok/s	420.27
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	256	6,818.11 tok/s	489.37
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	512	8,143.94 tok/s	505.71
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	1	123.78 tok/s	340.7
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	8	825.93 tok/s	349.18
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	16	1,526.88 tok/s	374.72
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	32	2,553.39 tok/s	358.2
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	64	4,192.4 tok/s	384.46
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	128	6,005.61 tok/s	424.98
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	256	6,759.05 tok/s	486.82
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	512	8,086.71 tok/s	515.9
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	1024	8,326.43 tok/s	528.37
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	1	124.1 tok/s	340.71
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	8	849.15 tok/s	349.68
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	16	1,605.45 tok/s	373.79
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	32	2,568.27 tok/s	356.3
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	64	4,391.43 tok/s	384.86
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	128	6,015.29 tok/s	425.21
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	256	6,784.95 tok/s	490.43
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	512	8,078.91 tok/s	517.34
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	1024	8,360.24 tok/s	505.72

Original reports

Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.

Original reports · 4 rows
Model	Test task	Format	Engine	Workload	Protocol	Quality score	Test date
Qwen3-4B ↗	Text generation	Native	vLLM	2K	Method unknown	—	1/13/2026, 4:29:35 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 2:47:05 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 7:49:18 PM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 7:35:40 PM

Running benchmarks

0Updates automatically

No benchmarks are currently reported as running.

Methodology

Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.

Updated 11/5/2026, 11:00:27 PM

Text generation · Qualified results

Measured alternatives on other GPUs

Qwen3-4B-AWQ

Original reports

Running benchmarks