RTX PRO 6000 Blackwell Server Edition · 96 GB
Qwen3.8-27B-NVFP4
Open a report for complete measurements.
- Task
- Text generation · Text → Text
- Workload
- 128K → 2K tokens · Prefix cache reuse
- Concurrency
- 1 request
- Format · Engine
- NVFP4 · vllm · vLLM 0.28.0
- Unit
- tok/s
- Mean throughput
- 52.6tok/s
- Observed GPU occupancy
- 86.8 GiB
- First token
- 509 ms
- Reports combined
- 1
Launch this model
Open the measured report recipe to preserve its engine and MoE settings.
Use this recipeIndividual tests
Text → Text · Text generation · NVFP4 · vllm · 131072in-2048out · tok/s
Sortable table
Uncached prefill is unavailable when prompts reuse cache or reliable phase timings are missing. Open a report for its recorded phase timings and cache counts.
| Test date | Throughput | Concurrency / batch | P95 (s) | TTFT (ms) | Observed occupancy (GiB) | GPU power (W) | Decode (tok/s) | Uncached prefill (tok/s) | Parameters | View report |
|---|---|---|---|---|---|---|---|---|---|---|
| 10/19/2026, 12:27:36 AM | 52.59 tok/s | 1 | 38.947 | 509.33 | 86.83 | 328.78 | 53.23 | Uncached prefill unavailableView report | Parameters ↗ | d14962c1 ↗ |
Recorded test parameters
Configuration 1 · Parameters
- model
- RadixArk/Qwen3.8-27B-NVFP4
- engine
- vllm
- quant
- NVFP4
- method
- wbench-fixed-http-v3
- task
- llm
- dataset
- synthetic-tokenizer-v3:cache=prefix-reuse:repeats=3
- context
- 131072in-2048out
- input_tokens
- 131072
- output_tokens
- 2048
- duration
- 3600
- concurrency
- 1
- unit
- tok/s
- software
- { "kv_capacity": { "gpu_cache_tokens": 1888278, "maximum_concurrency": 14.13, "note": "Engine estimate, not measured saturation; hybrid cache token units may differ. Minimum reported worker bound.", "reference_tokens": 133632, "source": "vllm-startup-log" }, "packages": { "flash-attn": null, "flashinfer-python": "0.6.16.post3", "sgl-kernel": null, "sglang": null, "tokenizers": "0.23.3", "torch": "2.13.0", "transformers": "5.20.1", "triton": "3.7.1", "vllm": "0.28.0" }, "python_implementation": "cpython", "python_version": "3.12.3", "schema": "wbench.python-runtime.v1", "scope": "launch-environment-installed-distributions" }
- engine_provenance
- { "checkpoint_quantization": { "mixed_quantization": true, "quantized_weight_formats": [ "FP8", "NVFP4" ], "recipe_label": "NVFP4", "scope": "Declared quantized weight formats; unquantized modules and observed runtime kernels are separate evidence", "source": "checkpoint-metadata" }, "client_version": "1.1.48", "cpu_offload_gb": 0, "engine": "vllm", "gpu_driver_version": "580.82.07", "gpu_memory_utilization": 0.8999999761581421, "limit_mm_per_prompt": null, "model_config_sha256": "7ff41ec6f96ad50efea3c92751cd261b63839d39936eb6e6ffc9066db8672740", "model_revision": "319f741cce68d7914884900c138a1fbb70a42f30", "quantization_config_sha256": "0f39e8cd23abdfb79adc89ac1b19acad990aa6ac32973f9ab0a67d1e3449535c", "reasoning_parser": "qwen3", "reported_cuda_version": "13.0", "runtime": { "kv_capacity": { "gpu_cache_tokens": 1888278, "maximum_concurrency": 14.13, "note": "Engine estimate, not measured saturation; hybrid cache token units may differ. Minimum reported worker bound.", "reference_tokens": 133632, "source": "vllm-startup-log" }, "packages": { "flash-attn": null, "flashinfer-python": "0.6.16.post3", "sgl-kernel": null, "sglang": null, "tokenizers": "0.23.3", "torch": "2.13.0", "transformers": "5.20.1", "triton": "3.7.1", "vllm": "0.28.0" }, "python_implementation": "cpython", "python_version": "3.12.3", "schema": "wbench.python-runtime.v1", "scope": "launch-environment-installed-distributions" }, "shared_gpu": false }
- kernel_settings
- —
- precision
- —
- resolution
- —
- performance_task
- —
- performance_scope
- —
- batch_size
- —
- dimensions
- —
- weight_quantization
- —
- max_sequence_length
- —
- model_scope
- —
- performance_dataset
- —
- workload_mode
- —
- graph_mode
- —
- model_revision
- —
- client_version
- 1.1.48