32 GB · 5 reports

RTX 4080 SUPER · 32 GB

Qualified results first; original reports and full tables stay below.

Text generation · Qualified results

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

No qualified results yet · Text generation · RTX 4080 SUPER · 32 GB

5 original reports are kept as evidence but do not qualify for comparison; each report states why. This does not show that the model or GPU cannot run it.

Check model launch support View original reports Measured alternatives for this task Test your own GPU

Measured alternatives on other GPUs

Measured on different GPUs with the same or smaller listed capacity. These are references for choosing hardware, not performance or compatibility evidence for your selected GPU.

Qwen3.8-27B-NVFP4
RadixArk · 2 tested configurations · 1 GPU
- RTX 5090 · 32 GBNVFP4 · 8K → 2K tokens
  Output throughput · includes prefill · 1 request
  70.8tok/s
  Observed GPU occupancy
  27.3GiB
  First token
  633 ms
  Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=2
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 70.82 29.01 s 633 ms 27.29 405.5 1 bad294c0 ↗
- 1 other test configurations on this GPU
  - NVFP4 · 32K → 2K tokens
    Output throughput · includes prefill · 1 request
    62.5tok/s
    Observed GPU occupancy
    27.3GiB
    First token
    3.6 s
    Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=2
    Workload
    32768in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 62.48 32.79 s 3.6 s 27.29 433.3 1 746bbc39 ↗
Qwen3-4B-AWQ
Qwen · 3 tested configurations · 2 GPUs
- L4 · 24 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  55.3tok/s
  Best measured · 4 concurrent requests
  109tok/s
  Observed GPU occupancy
  19.5GiB
  First token
  1.6 s
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 55.3 37.17 s 1.6 s 19.54 71.9 1 9301a9ba ↗
  C4 109.43 78.88 s 3.1 s 19.55 71.9 1 9301a9ba ↗
- 1 other test configurations on this GPU
  - AWQ · 8K → 2K tokens · Prefix cache reuse
    Output throughput · includes prefill · 1 request
    57.4tok/s
    Best measured · 4 concurrent requests
    175tok/s
    Observed GPU occupancy
    19.5GiB
    First token
    60 ms
    Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 concurrency levels
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm · vLLM 0.29.0
    Protocol
    wbench-fixed-http-v3
    Dataset
    migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 57.39 35.7 s 60 ms 19.55 71.8 1 7e2cbfda ↗
    C4 175.39 46.75 s 114 ms 19.55 71.9 1 7e2cbfda ↗
Show 1 more GPUs
- T4 · 16 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  15.4tok/s
  Best measured · 4 concurrent requests
  20.3tok/s
  Observed GPU occupancy
  12.7GiB
  First token
  28.5 s
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 15.39 133.36 s 28.5 s 12.74 67.9 1 fcfe2d64 ↗
  C4 20.31 468.13 s 53.1 s 12.74 67.5 1 fcfe2d64 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	70.82	29.01 s	633 ms	27.29	405.5	1	bad294c0 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	62.48	32.79 s	3.6 s	27.29	433.3	1	746bbc39 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	55.3	37.17 s	1.6 s	19.54	71.9	1	9301a9ba ↗
C4	109.43	78.88 s	3.1 s	19.55	71.9	1	9301a9ba ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.39	35.7 s	60 ms	19.55	71.8	1	7e2cbfda ↗
C4	175.39	46.75 s	114 ms	19.55	71.9	1	7e2cbfda ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	15.39	133.36 s	28.5 s	12.74	67.9	1	fcfe2d64 ↗
C4	20.31	468.13 s	53.1 s	12.74	67.5	1	fcfe2d64 ↗

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Measurements without a comparable protocol · 30 rows

These measured rates remain visible, but are excluded from comparison curves because the protocol is unknown.

Measurements without a comparable protocol · 30 rows
Model	Format	Engine	Workload	Concurrency	Throughput	GPU power (W)
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	1	69.34 tok/s	179.57
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	8	484.06 tok/s	192.34
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	16	799.89 tok/s	187.47
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	32	1,329.45 tok/s	194.48
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	64	2,085.01 tok/s	208.44
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	128	2,699.25 tok/s	230.56
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	70.04 tok/s	181.14
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	525.11 tok/s	194.61
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	16	951.26 tok/s	193.69
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	1,374.31 tok/s	200.45
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	64	2,382.86 tok/s	213.72
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	2,564.63 tok/s	234.69
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	1	125.54 tok/s	161.92
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	8	804.03 tok/s	174.87
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	16	1,402.39 tok/s	168.72
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	32	2,184.92 tok/s	183.07
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	64	3,131.37 tok/s	199.34
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	128	3,504.71 tok/s	217.81
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	1	127.9 tok/s	163.46
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	8	921.6 tok/s	176.8
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	16	1,531.13 tok/s	172.42
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	32	2,418.5 tok/s	184.98
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	64	3,457.71 tok/s	204.14
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	128	4,093.06 tok/s	223.63
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1	215.55 tok/s	135.8
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	8	1,415.3 tok/s	159.52
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	16	2,142.71 tok/s	151.64
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	32	3,450.39 tok/s	168.04
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	64	5,030.83 tok/s	189.85
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	128	6,273.26 tok/s	215.47

Original reports

Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.

Original reports · 5 rows
Model	Test task	Format	Engine	Workload	Protocol	Quality score	Test date
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/11/2026, 8:51:03 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/11/2026, 8:56:18 PM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/11/2026, 9:08:49 PM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/11/2026, 9:02:11 PM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/11/2026, 9:15:02 PM

Running benchmarks

0Updates automatically

No benchmarks are currently reported as running.

Methodology

Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.

Updated 11/5/2026, 11:07:29 PM

Text generation · Qualified results

Measured alternatives on other GPUs

Qwen3.8-27B-NVFP4

Qwen3-4B-AWQ

Original reports

Running benchmarks