Run an AI model on your computer

Measured alternatives on other GPUs

Measured on different GPUs with the same or smaller listed capacity. These are references for choosing hardware, not performance or compatibility evidence for your selected GPU.

1. Qwen3.8-27B-NVFP4

   RadixArk · 14 tested configurations · 2 GPUs

   • RTX 5090 · 32 GBNVFP4 · 8K → 2K tokens

     Output throughput · includes prefill · 1 request

     70.8tok/s

     Observed GPU occupancy

     27.3GiB

     First token

     633 ms

     Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
     Test details

     Total output tokens divided by the full request window, including prefill and queueing.

     Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

     Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

     Engine

     vllm

     Protocol

     wbench-fixed-http-v1

     Dataset

     synthetic-tokenizer-v1:repeats=2

     Workload

     8192in-2048out

     Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
     C1	70.82	29.01 s	633 ms	27.29	405.5	1	bad294c0 ↗

   • 1 other test configurations on this GPU

     • NVFP4 · 32K → 2K tokens

       Output throughput · includes prefill · 1 request

       62.5tok/s

       Observed GPU occupancy

       27.3GiB

       First token

       3.6 s

       Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
       Test details

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm

       Protocol

       wbench-fixed-http-v1

       Dataset

       synthetic-tokenizer-v1:repeats=2

       Workload

       32768in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	62.48	32.79 s	3.6 s	27.29	433.3	1	746bbc39 ↗

   Show 1 more GPUs

   • RTX PRO 6000 Blackwell Server Edition · 96 GBNVFP4 · 32K → 2K tokens · No cache reuse

     Output throughput · includes prefill · 1 request

     55.6tok/s

     Best measured · 8 concurrent requests

     225tok/s

     Observed GPU occupancy

     86GiB

     First token

     3.2 s

     Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
     Test details · 6 concurrency levels

     Total output tokens divided by the full request window, including prefill and queueing.

     Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

     Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

     Engine

     vllm · vLLM 0.28.0

     Protocol

     wbench-search-http-v1

     Dataset

     migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3

     Workload

     32768in-2048out

     Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
     C1	55.59	36.91 s	3.2 s	85.98	341	1	dca51ce5 ↗
     C2	91.37	46.48 s	4.6 s	85.98	342.9	1	dca51ce5 ↗
     C4	149.21	62.63 s	6.5 s	85.98	372.9	1	dca51ce5 ↗
     C6	196.73	76.24 s	7.8 s	85.98	398.9	1	dca51ce5 ↗
     C7	211.87	81.41 s	8.3 s	85.98	408.7	1	dca51ce5 ↗
     C8	224.61	90.5 s	8.9 s	85.98	414.8	1	dca51ce5 ↗

   • 11 other test configurations on this GPU

     • NVFP4 · 32K → 2K tokens · Prefix cache reuse

       Output throughput · includes prefill · 1 request

       60.1tok/s

       Best measured · 8 concurrent requests

       393tok/s

       Observed GPU occupancy

       86GiB

       First token

       217 ms

       Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 6 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-search-http-v1

       Dataset

       migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3

       Workload

       32768in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	60.1	34.34 s	217 ms	85.98	323	1	9cdf45b7 ↗
       C2	109.3	37.48 s	349 ms	85.98	310.8	1	9cdf45b7 ↗
       C4	209.23	39.18 s	686 ms	85.98	324.8	1	9cdf45b7 ↗
       C6	306.94	40.06 s	992 ms	85.98	336	1	9cdf45b7 ↗
       C7	351.24	40.83 s	1.1 s	85.98	342.6	1	9cdf45b7 ↗
       C8	392.93	41.71 s	1.3 s	85.98	348.2	1	9cdf45b7 ↗

     • NVFP4 · 32K → 2K tokens · Prefix cache reuse

       Output throughput · includes prefill · 1 request

       59.7tok/s

       Best measured · 8 concurrent requests

       394tok/s

       Observed GPU occupancy

       86.8GiB

       First token

       230 ms

       Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 3 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-fixed-http-v3

       Dataset

       synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3

       Workload

       32768in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	59.68	34.59 s	230 ms	86.82	323.3	1	83d37469 ↗
       C4	207.31	39.54 s	664 ms	86.82	320	1	83d37469 ↗
       C8	394.04	41.6 s	1.2 s	86.82	346.1	1	83d37469 ↗

     • NVFP4 · 32K → 2K tokens · Prefix cache reuse

       Output throughput · includes prefill · 1 request

       60.5tok/s

       Best measured · 8 concurrent requests

       395tok/s

       Observed GPU occupancy

       86.9GiB

       First token

       228 ms

       Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 3 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-fixed-http-v3

       Dataset

       synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3

       Workload

       32768in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	60.47	33.87 s	228 ms	86.86	320.3	1	cbf1431c ↗
       C4	207.86	39.44 s	636 ms	86.86	333.6	1	cbf1431c ↗
       C8	394.52	41.6 s	1.3 s	86.86	361.6	1	cbf1431c ↗

     • NVFP4 · 64K → 2K tokens · No cache reuse

       Output throughput · includes prefill · 1 request

       47.1tok/s

       Best measured · 4 concurrent requests

       101tok/s

       Observed GPU occupancy

       86GiB

       First token

       8.4 s

       Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 4 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-search-http-v1

       Dataset

       migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3

       Workload

       65536in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	47.15	43.45 s	8.4 s	85.98	364.4	1	e5a201fa ↗
       C2	70.39	60.71 s	12.3 s	85.98	381.3	1	e5a201fa ↗
       C3	88.36	80.85 s	14.2 s	85.98	403.6	1	e5a201fa ↗
       C4	100.66	101.65 s	15.8 s	85.98	419.2	1	e5a201fa ↗

     • NVFP4 · 64K → 2K tokens · Prefix cache reuse

       Output throughput · includes prefill · 1 request

       57.5tok/s

       Best measured · 8 concurrent requests

       355tok/s

       Observed GPU occupancy

       86GiB

       First token

       326 ms

       Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 6 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-search-http-v1

       Dataset

       migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3

       Workload

       65536in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	57.45	35.8 s	326 ms	85.98	326.1	1	67a5e6d3 ↗
       C2	104.7	39.14 s	515 ms	85.98	321	1	67a5e6d3 ↗
       C4	198.72	41.24 s	954 ms	85.98	344.2	1	67a5e6d3 ↗
       C6	277.65	44.28 s	1.3 s	85.98	358.8	1	67a5e6d3 ↗
       C7	317.41	45.18 s	1.4 s	85.98	369.2	1	67a5e6d3 ↗
       C8	354.95	46.19 s	1.6 s	85.98	377.1	1	67a5e6d3 ↗

     • NVFP4 · 64K → 2K tokens · Prefix cache reuse

       Output throughput · includes prefill · 1 request

       57.1tok/s

       Best measured · 8 concurrent requests

       356tok/s

       Observed GPU occupancy

       86.8GiB

       First token

       358 ms

       Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 3 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-fixed-http-v3

       Dataset

       synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3

       Workload

       65536in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	57.08	36.15 s	358 ms	86.82	326	1	275a096d ↗
       C4	196.91	41.63 s	870 ms	86.83	338.7	1	275a096d ↗
       C8	355.89	46.08 s	1.6 s	86.83	375.7	1	275a096d ↗

     • NVFP4 · 64K → 2K tokens · Prefix cache reuse

       Output throughput · includes prefill · 1 request

       57.7tok/s

       Best measured · 8 concurrent requests

       356tok/s

       Observed GPU occupancy

       86.9GiB

       First token

       357 ms

       Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 3 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-fixed-http-v3

       Dataset

       synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3

       Workload

       65536in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	57.67	35.51 s	357 ms	86.86	325.8	1	fe8141d2 ↗
       C4	197.42	41.52 s	870 ms	86.86	354.3	1	fe8141d2 ↗
       C8	356.25	46.05 s	1.6 s	86.86	392.4	1	fe8141d2 ↗

     • NVFP4 · 128K → 2K tokens · No cache reuse

       Output throughput · includes prefill · 1 request

       32.7tok/s

       Best measured · 2 concurrent requests

       42.5tok/s

       Observed GPU occupancy

       86GiB

       First token

       24.3 s

       Use this recipe · 2 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 2 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-search-http-v1

       Dataset

       migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3

       Workload

       131072in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	32.72	62.59 s	24.3 s	85.99	406.5	1	28f51cf1 ↗
       C2	42.54	106.24 s	33.5 s	85.99	428.9	1	28f51cf1 ↗

     • NVFP4 · 128K → 2K tokens · Prefix cache reuse

       Output throughput · includes prefill · 1 request

       53tok/s

       Best measured · 4 concurrent requests

       181tok/s

       Observed GPU occupancy

       86GiB

       First token

       435 ms

       Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 4 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-search-http-v1

       Dataset

       migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3

       Workload

       131072in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	52.97	38.72 s	435 ms	85.99	333.9	1	78fc908c ↗
       C2	97.25	42.14 s	646 ms	85.99	340.8	1	78fc908c ↗
       C3	138.66	44.36 s	822 ms	85.99	358.8	1	78fc908c ↗
       C4	181.16	45.25 s	1.1 s	85.99	375.3	1	78fc908c ↗

     • NVFP4 · 128K → 2K tokens · Prefix cache reuse

       Output throughput · includes prefill · 1 request

       52.9tok/s

       Best measured · 8 concurrent requests

       301tok/s

       Observed GPU occupancy

       86.9GiB

       First token

       508 ms

       Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 3 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-fixed-http-v3

       Dataset

       synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3

       Workload

       131072in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	52.9	38.72 s	508 ms	86.86	338	1	7cea351c ↗
       C4	179.81	45.65 s	1 s	86.86	387.5	1	7cea351c ↗
       C8	301.16	54.48 s	1.9 s	86.86	443.7	1	7cea351c ↗

     • NVFP4 · 128K → 2K tokens · Prefix cache reuse

       Output throughput · includes prefill · 1 request

       52.6tok/s

       Best measured · 8 concurrent requests

       301tok/s

       Observed GPU occupancy

       86.8GiB

       First token

       509 ms

       Use this recipe · 8 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
       Test details · 3 concurrency levels

       Total output tokens divided by the full request window, including prefill and queueing.

       Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

       Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

       Engine

       vllm · vLLM 0.28.0

       Protocol

       wbench-fixed-http-v3

       Dataset

       synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3

       Workload

       131072in-2048out

       Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
       C1	52.59	38.95 s	509 ms	86.83	328.8	1	d14962c1 ↗
       C4	179.42	45.72 s	1.1 s	86.83	371.9	1	d14962c1 ↗
       C8	300.98	54.52 s	1.9 s	86.83	426	1	d14962c1 ↗