96 GB · 196 reports

RTX PRO 6000 Blackwell Server Edition · 96 GB

Qualified results first; original reports and full tables stay below.
Reset

Text generation · Qualified results

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.
  1. Qwen3.8-27B-NVFP4

    RadixArk · 12 tested configurations

    • NVFP4 · 128K → 2K tokens · Prefix cache reuse
      Output throughput · includes prefill · 1 request
      53tok/s
      Best measured · 4 concurrent requests
      181tok/s
      Observed GPU occupancy
      86GiB
      First token
      435 ms
      Test details · 4 concurrency levels

      Total output tokens divided by the full request window, including prefill and queueing.

      Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

      Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

      Engine
      vllm · vLLM 0.28.0
      Protocol
      wbench-search-http-v1
      Dataset
      migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
      Workload
      131072in-2048out
      ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
      C152.9738.72 s435 ms85.99333.9178fc908c ↗
      C297.2542.14 s646 ms85.99340.8178fc908c ↗
      C3138.6644.36 s822 ms85.99358.8178fc908c ↗
      C4181.1645.25 s1.1 s85.99375.3178fc908c ↗
    • 11 other test configurations on this GPU
      • NVFP4 · 128K → 2K tokens · No cache reuse
        Output throughput · includes prefill · 1 request
        32.7tok/s
        Best measured · 2 concurrent requests
        42.5tok/s
        Observed GPU occupancy
        86GiB
        First token
        24.3 s
        Test details · 2 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-search-http-v1
        Dataset
        migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
        Workload
        131072in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C132.7262.59 s24.3 s85.99406.5128f51cf1 ↗
        C242.54106.24 s33.5 s85.99428.9128f51cf1 ↗
      • NVFP4 · 64K → 2K tokens · Prefix cache reuse
        Output throughput · includes prefill · 1 request
        57.5tok/s
        Best measured · 8 concurrent requests
        355tok/s
        Observed GPU occupancy
        86GiB
        First token
        326 ms
        Test details · 6 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-search-http-v1
        Dataset
        migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
        Workload
        65536in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C157.4535.8 s326 ms85.98326.1167a5e6d3 ↗
        C2104.739.14 s515 ms85.98321167a5e6d3 ↗
        C4198.7241.24 s954 ms85.98344.2167a5e6d3 ↗
        C6277.6544.28 s1.3 s85.98358.8167a5e6d3 ↗
        C7317.4145.18 s1.4 s85.98369.2167a5e6d3 ↗
        C8354.9546.19 s1.6 s85.98377.1167a5e6d3 ↗
      • NVFP4 · 64K → 2K tokens · No cache reuse
        Output throughput · includes prefill · 1 request
        47.1tok/s
        Best measured · 4 concurrent requests
        101tok/s
        Observed GPU occupancy
        86GiB
        First token
        8.4 s
        Test details · 4 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-search-http-v1
        Dataset
        migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
        Workload
        65536in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C147.1543.45 s8.4 s85.98364.41e5a201fa ↗
        C270.3960.71 s12.3 s85.98381.31e5a201fa ↗
        C388.3680.85 s14.2 s85.98403.61e5a201fa ↗
        C4100.66101.65 s15.8 s85.98419.21e5a201fa ↗
      • NVFP4 · 32K → 2K tokens · Prefix cache reuse
        Output throughput · includes prefill · 1 request
        60.1tok/s
        Best measured · 8 concurrent requests
        393tok/s
        Observed GPU occupancy
        86GiB
        First token
        217 ms
        Test details · 6 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-search-http-v1
        Dataset
        migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
        Workload
        32768in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C160.134.34 s217 ms85.9832319cdf45b7 ↗
        C2109.337.48 s349 ms85.98310.819cdf45b7 ↗
        C4209.2339.18 s686 ms85.98324.819cdf45b7 ↗
        C6306.9440.06 s992 ms85.9833619cdf45b7 ↗
        C7351.2440.83 s1.1 s85.98342.619cdf45b7 ↗
        C8392.9341.71 s1.3 s85.98348.219cdf45b7 ↗
      • NVFP4 · 32K → 2K tokens · No cache reuse
        Output throughput · includes prefill · 1 request
        55.6tok/s
        Best measured · 8 concurrent requests
        225tok/s
        Observed GPU occupancy
        86GiB
        First token
        3.2 s
        Test details · 6 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-search-http-v1
        Dataset
        migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
        Workload
        32768in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C155.5936.91 s3.2 s85.983411dca51ce5 ↗
        C291.3746.48 s4.6 s85.98342.91dca51ce5 ↗
        C4149.2162.63 s6.5 s85.98372.91dca51ce5 ↗
        C6196.7376.24 s7.8 s85.98398.91dca51ce5 ↗
        C7211.8781.41 s8.3 s85.98408.71dca51ce5 ↗
        C8224.6190.5 s8.9 s85.98414.81dca51ce5 ↗
      • NVFP4 · 128K → 2K tokens · Prefix cache reuse
        Output throughput · includes prefill · 1 request
        52.9tok/s
        Best measured · 8 concurrent requests
        301tok/s
        Observed GPU occupancy
        86.9GiB
        First token
        508 ms
        Test details · 3 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-fixed-http-v3
        Dataset
        synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
        Workload
        131072in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C152.938.72 s508 ms86.8633817cea351c ↗
        C4179.8145.65 s1 s86.86387.517cea351c ↗
        C8301.1654.48 s1.9 s86.86443.717cea351c ↗
      • NVFP4 · 64K → 2K tokens · Prefix cache reuse
        Output throughput · includes prefill · 1 request
        57.7tok/s
        Best measured · 8 concurrent requests
        356tok/s
        Observed GPU occupancy
        86.9GiB
        First token
        357 ms
        Test details · 3 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-fixed-http-v3
        Dataset
        synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
        Workload
        65536in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C157.6735.51 s357 ms86.86325.81fe8141d2 ↗
        C4197.4241.52 s870 ms86.86354.31fe8141d2 ↗
        C8356.2546.05 s1.6 s86.86392.41fe8141d2 ↗
      • NVFP4 · 32K → 2K tokens · Prefix cache reuse
        Output throughput · includes prefill · 1 request
        60.5tok/s
        Best measured · 8 concurrent requests
        395tok/s
        Observed GPU occupancy
        86.9GiB
        First token
        228 ms
        Test details · 3 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-fixed-http-v3
        Dataset
        synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
        Workload
        32768in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C160.4733.87 s228 ms86.86320.31cbf1431c ↗
        C4207.8639.44 s636 ms86.86333.61cbf1431c ↗
        C8394.5241.6 s1.3 s86.86361.61cbf1431c ↗
      • NVFP4 · 128K → 2K tokens · Prefix cache reuse
        Output throughput · includes prefill · 1 request
        52.6tok/s
        Best measured · 8 concurrent requests
        301tok/s
        Observed GPU occupancy
        86.8GiB
        First token
        509 ms
        Test details · 3 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-fixed-http-v3
        Dataset
        synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
        Workload
        131072in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C152.5938.95 s509 ms86.83328.81d14962c1 ↗
        C4179.4245.72 s1.1 s86.83371.91d14962c1 ↗
        C8300.9854.52 s1.9 s86.834261d14962c1 ↗
      • NVFP4 · 64K → 2K tokens · Prefix cache reuse
        Output throughput · includes prefill · 1 request
        57.1tok/s
        Best measured · 8 concurrent requests
        356tok/s
        Observed GPU occupancy
        86.8GiB
        First token
        358 ms
        Test details · 3 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-fixed-http-v3
        Dataset
        synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
        Workload
        65536in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C157.0836.15 s358 ms86.823261275a096d ↗
        C4196.9141.63 s870 ms86.83338.71275a096d ↗
        C8355.8946.08 s1.6 s86.83375.71275a096d ↗
      • NVFP4 · 32K → 2K tokens · Prefix cache reuse
        Output throughput · includes prefill · 1 request
        59.7tok/s
        Best measured · 8 concurrent requests
        394tok/s
        Observed GPU occupancy
        86.8GiB
        First token
        230 ms
        Test details · 3 concurrency levels

        Total output tokens divided by the full request window, including prefill and queueing.

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        vllm · vLLM 0.28.0
        Protocol
        wbench-fixed-http-v3
        Dataset
        synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
        Workload
        32768in-2048out
        ConcurrencyMean throughput (tok/s)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        C159.6834.59 s230 ms86.82323.3183d37469 ↗
        C4207.3139.54 s664 ms86.82320183d37469 ↗
        C8394.0441.6 s1.2 s86.82346.1183d37469 ↗

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Sortable tables by exact test condition · 46 rows

C: concurrency · B: batch · —: not measured

Text generation · 32K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C155.5955.593,188.3336.91385.9834155.590.161 ↗
Text generation · 32K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C291.3791.374,63946.48285.98342.991.370.271 ↗
Text generation · 32K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C4149.21149.216,48562.62985.98372.9149.210.41 ↗
Text generation · 32K → 2K tokens · No cache reuse · 6 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C6196.73196.737,789.2276.24385.98398.9196.730.491 ↗
Text generation · 32K → 2K tokens · No cache reuse · 7 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C7211.87211.878,317.981.41185.98408.7211.870.521 ↗
Text generation · 32K → 2K tokens · No cache reuse · 8 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C8224.61224.618,934.3390.49685.98414.8224.610.541 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C160.160.121734.34385.9832360.10.191 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3C160.4760.47228.3333.8786.86320.360.470.191 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3C159.6859.68230.3334.58586.82323.359.680.181 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C2109.3109.3348.6737.48485.98310.8109.30.351 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C4209.23209.23686.0839.17885.98324.8209.230.641 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3C4207.86207.86636.2539.44486.86333.6207.860.621 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3C4207.31207.31663.9239.54386.82320207.310.651 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C6306.94306.94991.6140.05985.98336306.940.911 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C7351.24351.241,112.4340.83285.98342.6351.241.031 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-search-http-v1C8392.93392.931,250.1741.70985.98348.2392.931.131 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3C8394.52394.521,251.7541.59786.86361.6394.521.091 ↗
Text generation · 32K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v3C8394.04394.041,243.5441.60486.82346.1394.041.141 ↗
Text generation · 64K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C147.1547.158,36443.44685.98364.447.150.131 ↗
Text generation · 64K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C270.3970.3912,314.3360.71485.98381.370.390.181 ↗
Text generation · 64K → 2K tokens · No cache reuse · 3 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C388.3688.3614,170.5680.85385.98403.688.360.221 ↗
Text generation · 64K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C4100.66100.6615,765.08101.65385.98419.2100.660.241 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C157.4557.45325.6735.885.98326.157.450.181 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3C157.6757.67356.6735.50986.86325.857.670.181 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3C157.0857.08357.6736.14786.8232657.080.181 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C2104.7104.7515.3339.13685.98321104.70.331 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C4198.72198.72953.541.23685.98344.2198.720.581 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3C4197.42197.42870.0841.5286.86354.3197.420.561 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3C4196.91196.91869.541.6386.83338.7196.910.581 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 6 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C6277.65277.651,254.2844.28385.98358.8277.650.771 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 7 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C7317.41317.411,446.5745.18485.98369.2317.410.861 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-search-http-v1C8354.95354.951,632.0846.19385.98377.1354.950.941 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3C8356.25356.251,596.7946.05286.86392.4356.250.911 ↗
Text generation · 64K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm65536in-2048out65,536 → 2,048 tokenswbench-fixed-http-v3C8355.89355.891,59246.07986.83375.7355.890.951 ↗
Text generation · 128K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1C132.7232.7224,314.6762.59485.99406.532.720.081 ↗
Text generation · 128K → 2K tokens · No cache reuse · 2 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1C242.5442.5433,482.83106.23985.99428.942.540.11 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1C152.9752.97434.6738.71785.99333.952.970.161 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3C152.952.9508.3338.7286.8633852.90.161 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3C152.5952.59509.3338.94786.83328.852.590.161 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 2 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1C297.2597.2564642.13585.99340.897.250.291 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 3 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1C3138.66138.66822.1144.3685.99358.8138.660.391 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-search-http-v1C4181.16181.161,056.4245.24985.99375.3181.160.481 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3C4179.81179.811,03745.65186.86387.5179.810.461 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3C4179.42179.421,06945.72186.83371.9179.420.481 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3C8301.16301.161,852.2954.47986.86443.7301.160.681 ↗
Text generation · 128K → 2K tokens · Prefix cache reuse · 8 concurrent requests · tok/s · 1 rows
ModelFormatEngineWorkloadConcurrency / batchMean throughputBest measuredTTFT (ms)P95 (s)Observed occupancy (GiB)GPU power (W)Lowest measuredThroughput / WReports combined
RadixArk/Qwen3.8-27B-NVFP4Scaling curves ↗NVFP4vllm131072in-2048out131,072 → 2,048 tokenswbench-fixed-http-v3C8300.98300.981,857.7554.5286.83426300.980.711 ↗
Scaling curves and individual runs

One model and test protocol per curve; different workloads are never joined into one line.

RadixArk/Qwen3.8-27B-NVFP4 · NVFP4 · vllm · 131072in-2048out · wbench-search-http-v1

Mean throughput tok/s
Concurrency 1: 53 tok/s; Concurrency 2: 97.2 tok/s; Concurrency 3: 139 tok/s; Concurrency 4: 181 tok/s049.899.61491991234Concurrency · log₂

Measured points only. Select a point to open its source report.

Mean run P95 latency s
Concurrency 1: 38.7 s; Concurrency 2: 42.1 s; Concurrency 3: 44.4 s; Concurrency 4: 45.2 s012.424.937.349.81234Concurrency · log₂

Measured points only. Select a point to open its source report.

Mean GPU memory used GB
Concurrency 1: 86 GB; Concurrency 2: 86 GB; Concurrency 3: 86 GB; Concurrency 4: 86 GB023.647.370.994.61234Concurrency · log₂

Measured points only. Select a point to open its source report.

Each point is the mean of recorded runs at that level. A source report may differ from the mean; all runs are listed below.

Uncached prefill is unavailable when prompts reuse cache or reliable phase timings are missing. Open a report for its recorded phase timings and cache counts.

Runs behind the curve · RadixArk/Qwen3.8-27B-NVFP4 · tok/s · 4 rows
ConcurrencyThroughputDecode (tok/s)Uncached prefill (tok/s)GPU power (W)TTFT (ms)P95 (s)Observed occupancy (GiB)Test dateView report
152.97 tok/s53.52Uncached prefill unavailableView report333.9434.6738.71785.9910/25/2026, 7:24:36 AM78fc908c ↗
297.25 tok/s49.34Uncached prefill unavailableView report340.8464642.13585.9910/25/2026, 7:24:36 AM78fc908c ↗
3138.66 tok/s47.05Uncached prefill unavailableView report358.76822.1144.3685.9910/25/2026, 7:24:36 AM78fc908c ↗
4181.16 tok/s46.32Uncached prefill unavailableView report375.281,056.4245.24985.9910/25/2026, 7:24:36 AM78fc908c ↗

Original reports

Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.
Original reports · 12 rows
ModelTest taskFormatEngineWorkloadProtocolQuality scoreTest date
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm131072in-2048outwbench-search-http-v1—10/25/2026, 7:24:36 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm131072in-2048outwbench-search-http-v1—10/25/2026, 7:09:07 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm65536in-2048outwbench-search-http-v1—10/25/2026, 6:55:57 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm65536in-2048outwbench-search-http-v1—10/25/2026, 6:34:21 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm32768in-2048outwbench-search-http-v1—10/25/2026, 6:13:45 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm32768in-2048outwbench-search-http-v1—10/25/2026, 5:54:42 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm131072in-2048outwbench-fixed-http-v3—10/19/2026, 7:40:50 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm65536in-2048outwbench-fixed-http-v3—10/19/2026, 6:58:25 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm32768in-2048outwbench-fixed-http-v3—10/19/2026, 6:33:20 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm131072in-2048outwbench-fixed-http-v3—10/19/2026, 12:27:36 AM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm65536in-2048outwbench-fixed-http-v3—10/18/2026, 11:46:41 PM
Qwen3.8-27B-NVFP4 ↗Text generationNVFP4vllm32768in-2048outwbench-fixed-http-v3—10/18/2026, 11:22:38 PM

Running benchmarks

0Updates automatically

No benchmarks are currently reported as running.

RTX PRO 6000 Blackwell Server Edition · 96 GB · 403 queued · 0 running · 0 awaiting reassignment · 6 succeeded · 3 failed or exhausted

Methodology

Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.

Updated 11/5/2026, 9:56:22 PM