24 GB · 111 reports
L4 · 24 GB
Qualified results first; original reports and full tables stay below.
Text generation · Qualified results
Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.
Qwen3-4B-AWQ
Qwen · 2 tested configurations
- AWQ · 8K → 2K tokens · Prefix cache reuse
- Output throughput · includes prefill · 1 request
- 57.4tok/s
- Best measured · 4 concurrent requests
- 175tok/s
- Observed GPU occupancy
- 19.5GiB
- First token
- 60 ms
Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUsTest details · 2 concurrency levels
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- vllm · vLLM 0.29.0
- Protocol
- wbench-fixed-http-v3
- Dataset
- migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
- Workload
- 8192in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report C1 57.39 35.7 s 60 ms 19.55 71.8 1 7e2cbfda ↗ C4 175.39 46.75 s 114 ms 19.55 71.9 1 7e2cbfda ↗ 1 other test configurations on this GPU
- AWQ · 8K → 2K tokens · No cache reuse
- Output throughput · includes prefill · 1 request
- 55.3tok/s
- Best measured · 4 concurrent requests
- 109tok/s
- Observed GPU occupancy
- 19.5GiB
- First token
- 1.6 s
Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUsTest details · 2 concurrency levels
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- vllm · vLLM 0.29.0
- Protocol
- wbench-fixed-http-v3
- Dataset
- migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
- Workload
- 8192in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report C1 55.3 37.17 s 1.6 s 19.54 71.9 1 9301a9ba ↗ C4 109.43 78.88 s 3.1 s 19.55 71.9 1 9301a9ba ↗
Qwen3.8-27B-UD-Q4_K_M.gguf
unsloth/Qwen3.8-27B-GGUF · 2 tested configurations
- Q4_K_M · 32K → 2K tokens
- Output throughput · includes prefill · 1 request
- 12.9tok/s
- Observed GPU occupancy
- 17GiB
- First token
- 371 ms
Test details
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- llamacpp
- Protocol
- wbench-fixed-http-v1
- Dataset
- synthetic-tokenizer-v1:repeats=3
- Workload
- 32768in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report C1 12.94 158.32 s 371 ms 17.05 71.8 1 d8a4b66b ↗ 1 other test configurations on this GPU
- Q4_K_M · 8K → 2K tokens
- Output throughput · includes prefill · 1 request
- 14.1tok/s
- Observed GPU occupancy
- 17GiB
- First token
- 279 ms
Test details
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- llamacpp
- Protocol
- wbench-fixed-http-v1
- Dataset
- synthetic-tokenizer-v1:repeats=3
- Workload
- 8192in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report C1 14.09 145.5 s 279 ms 17.05 71.9 1 4bc32a6c ↗
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Sortable tables by exact test condition · 6 rows
C: concurrency · B: batch · —: not measured
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.ggufScaling curves ↗ | Q4_K_M | llamacpp | 8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v1 | C1 | 14.09 | 14.09 | 278.67 | 145.499 | 17.05 | 71.9 | 14.09 | 0.2 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen/Qwen3-4B-AWQScaling curves ↗ | AWQ | vllm | 8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3 | C1 | 55.3 | 55.3 | 1,582.33 | 37.174 | 19.54 | 71.9 | 55.3 | 0.77 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen/Qwen3-4B-AWQScaling curves ↗ | AWQ | vllm | 8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3 | C4 | 109.43 | 109.43 | 3,149.58 | 78.879 | 19.55 | 71.9 | 109.43 | 1.52 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen/Qwen3-4B-AWQScaling curves ↗ | AWQ | vllm | 8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3 | C1 | 57.39 | 57.39 | 60 | 35.702 | 19.55 | 71.8 | 57.39 | 0.8 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen/Qwen3-4B-AWQScaling curves ↗ | AWQ | vllm | 8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3 | C4 | 175.39 | 175.39 | 114 | 46.748 | 19.55 | 71.9 | 175.39 | 2.44 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.ggufScaling curves ↗ | Q4_K_M | llamacpp | 32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v1 | C1 | 12.94 | 12.94 | 371 | 158.323 | 17.05 | 71.8 | 12.94 | 0.18 | 1 ↗ |
Scaling curves and individual runs
One model and test protocol per curve; different workloads are never joined into one line.
unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.gguf · Q4_K_M · llamacpp · 8192in-2048out · wbench-fixed-http-v1
One measured point; more levels are needed for a curve.
One measured point; more levels are needed for a curve.
One measured point; more levels are needed for a curve.
Each point is the mean of recorded runs at that level. A source report may differ from the mean; all runs are listed below.
Uncached prefill is unavailable when prompts reuse cache or reliable phase timings are missing. Open a report for its recorded phase timings and cache counts.
| Concurrency | Throughput | Decode (tok/s) | Uncached prefill (tok/s) | GPU power (W) | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | Test date | View report |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 14.09 tok/s | 14.12 | Uncached prefill unavailableView report | 71.92 | 278.67 | 145.499 | 17.05 | 10/14/2026, 12:56:36 PM | 4bc32a6c ↗ |
Quality by dataset
Scores from different datasets and sample scopes are not directly comparable.
| Model | Dataset | Format | Engine | Average score | Sample size | Runs |
|---|---|---|---|---|---|---|
| hf:unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.gguf | gsm8k-test-zero-shot-strict-v2-n3-mt16384-t0 | Q4_K_M | llamacpp | 100% | 3 / 1319 · Subset only | 1 runs100% · ab312fed ↗ |
| hf:unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.gguf | mmlu-test-zero-shot-strict-v2-n3-mt16384-t0 | Q4_K_M | llamacpp | 100% | 3 / 14042 · Subset only | 1 runs100% · 30bcfc00 ↗ |
| Qwen/Qwen3-0.6B-FP8 | aime2026-test-zero-shot-strict-v4-n30-mt16384-t0 | FP8 | vllm | 0% | 30 / 30 · Full dataset | 1 runs0% · 35ae8f96 ↗ |
Measurements without a comparable protocol · 16 rows
These measured rates remain visible, but are excluded from comparison curves because the protocol is unknown.
| Model | Format | Engine | Workload | Concurrency | Throughput | GPU power (W) |
|---|---|---|---|---|---|---|
| Qwen2.5-3B ↗ | Q4_K_M | Ollama | 1K | 1 | 82.39 tok/s | 69.71 |
| Qwen2.5-3B ↗ | Q4_K_M | Ollama | 1K | 2 | 83.62 tok/s | 70.71 |
| Qwen2.5-3B ↗ | Q4_K_M | Ollama | 1K | 4 | 84.03 tok/s | 70.48 |
| Qwen2.5-3B ↗ | Q4_K_M | Ollama | 1K | 8 | 83.62 tok/s | 70.64 |
| Qwen2.5-3B ↗ | Q4_K_M | Ollama | 1K | 16 | 83.96 tok/s | 70.15 |
| Qwen2.5-0.5B ↗ | Q4_K_M | Ollama | 1K | 1 | 180.69 tok/s | 52.28 |
| Qwen2.5-0.5B ↗ | Q4_K_M | Ollama | 1K | 2 | 197.1 tok/s | 55.46 |
| Qwen2.5-0.5B ↗ | Q4_K_M | Ollama | 1K | 4 | 92.45 tok/s | 39.96 |
| Qwen2.5-0.5B ↗ | Q4_K_M | Ollama | 1K | 8 | 150.14 tok/s | 47.71 |
| Qwen2.5-0.5B ↗ | Q4_K_M | Ollama | 1K | 16 | 197.53 tok/s | 58.26 |
| Qwen3-0.6B ↗ | Native | vLLM | 2K | 1 | 133.88 tok/s | 69.47 |
| Qwen3-0.6B ↗ | Native | vLLM | 2K | 8 | 548.24 tok/s | 58.33 |
| Qwen3-0.6B ↗ | Native | vLLM | 2K | 16 | 994.33 tok/s | 66.32 |
| Qwen3-0.6B ↗ | Native | vLLM | 2K | 32 | 1,418.76 tok/s | 69.6 |
| Qwen3-0.6B ↗ | Native | vLLM | 2K | 64 | 1,711.93 tok/s | 70.52 |
| Qwen3-0.6B ↗ | Native | vLLM | 2K | 128 | 1,826.9 tok/s | 70.28 |
Original reports
Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.
| Model | Test task | Format | Engine | Workload | Protocol | Quality score | Test date |
|---|---|---|---|---|---|---|---|
| Llama-3.1-8B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:46:40 PM |
| Llama-3.1-8B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:43:48 PM |
| Llama-3.1-8B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:41:11 PM |
| Ministral-3-3B-Instruct-2512 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:08:22 PM |
| Ministral-3-3B-Instruct-2512 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:06:35 PM |
| Ministral-3-3B-Instruct-2512 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:05:09 PM |
| Ministral-3-8B-Instruct-2512 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:32:15 PM |
| Ministral-3-8B-Instruct-2512 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:28:53 PM |
| Ministral-3-8B-Instruct-2512 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:25:59 PM |
| Ministral-3-14B-Instruct-2512 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:30:05 PM |
| Ministral-3-14B-Instruct-2512 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:25:16 PM |
| Ministral-3-14B-Instruct-2512 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:20:53 PM |
| Ministral-3-14B-Instruct-2512-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:17:56 PM |
| Ministral-3-14B-Instruct-2512-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:13:13 PM |
| Ministral-3-14B-Instruct-2512-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:08:50 PM |
| NVIDIA-Nemotron-3-Nano-4B-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:39:58 PM |
| NVIDIA-Nemotron-3-Nano-4B-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:39:46 PM |
| NVIDIA-Nemotron-3-Nano-4B-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:39:39 PM |
| NVIDIA-Nemotron-Nano-9B-v2-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:51:00 PM |
| NVIDIA-Nemotron-Nano-9B-v2-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:47:42 PM |
| NVIDIA-Nemotron-Nano-9B-v2-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:44:31 PM |
| NVIDIA-Nemotron-Nano-12B-v2-VL-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:10:50 PM |
| NVIDIA-Nemotron-Nano-12B-v2-VL-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:06:44 PM |
| NVIDIA-Nemotron-Nano-12B-v2-VL-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:02:46 PM |
| Qwen2.5-0.5B ↗ | Text generation | Q4_K_M | Ollama | 1K | Method unknown | — | 10/14/2026, 8:36:53 AM |
| qwen2.5-0.5b-instruct-q4_k_m.gguf ↗ | Text generation | Q4_K_M | llama.cpp | 1K | wbench-http | — | 10/14/2026, 9:06:56 AM |
| qwen2.5-1.5b-instruct-q4_k_m.gguf ↗ | Text generation | Q4_K_M | llama.cpp | 1K | wbench-http | — | 10/14/2026, 9:09:44 AM |
| Qwen2.5-3B ↗ | Text generation | Q4_K_M | Ollama | 1K | Method unknown | — | 10/14/2026, 8:42:44 AM |
| qwen2.5-3b-instruct-q4_k_m.gguf ↗ | Text generation | Q4_K_M | llama.cpp | 1K | wbench-http | — | 10/14/2026, 9:12:52 AM |
| Qwen3-0.6B ↗ | Text generation | Native | vLLM | 2K | Method unknown | — | 1/11/2026, 11:13:47 AM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v2 | — | 10/18/2026, 8:02:34 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v2 | — | 10/18/2026, 8:01:15 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v2 | — | 10/18/2026, 8:00:32 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:09:14 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:08:19 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:07:39 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:17:31 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:16:39 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:16:05 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | aime2026-test-zero-shot-strict-v4-n30-mt16384-t0 | quality-v1 | 0% | 10/18/2026, 1:22:12 PM |
| Qwen3-0.6B-FP8 ↗ | Text generation | FP8 | vllm | 512in-256out | wbench-fixed-http-v1 | — | 10/18/2026, 12:25:46 PM |
| Qwen3-0.6B-GPTQ-Int8 ↗ | Text generation | GPTQ | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:33:40 PM |
| Qwen3-0.6B-GPTQ-Int8 ↗ | Text generation | GPTQ | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:32:50 PM |
| Qwen3-0.6B-GPTQ-Int8 ↗ | Text generation | GPTQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:32:16 PM |
| Qwen3-1.7B-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:16:30 PM |
| Qwen3-1.7B-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:15:00 PM |
| Qwen3-1.7B-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:13:58 PM |
| Qwen3-1.7B-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:22:05 PM |
| Qwen3-1.7B-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:20:45 PM |
| Qwen3-1.7B-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:19:47 PM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 8192in-2048out | wbench-fixed-http-v3 | — | 10/23/2026, 6:48:39 AM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 8192in-2048out | wbench-fixed-http-v3 | — | 10/23/2026, 6:45:39 AM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:37:49 PM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:36:11 PM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:35:06 PM |
| Qwen3-4B-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:45:11 PM |
| Qwen3-4B-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:43:01 PM |
| Qwen3-4B-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:41:14 PM |
| Qwen3-4B-Instruct-2507-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:53:16 PM |
| Qwen3-4B-Instruct-2507-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:51:10 PM |
| Qwen3-4B-Instruct-2507-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:49:25 PM |
| Qwen3-4B-Thinking-2507-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:55:07 PM |
| Qwen3-4B-Thinking-2507-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:52:58 PM |
| Qwen3-4B-Thinking-2507-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:51:08 PM |
| Qwen3-8B-AWQ ↗ | Text generation | AWQ | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:42:12 PM |
| Qwen3-8B-AWQ ↗ | Text generation | AWQ | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:40:00 PM |
| Qwen3-8B-AWQ ↗ | Text generation | AWQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:38:09 PM |
| Qwen3-8B-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:12:18 PM |
| Qwen3-8B-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:09:03 PM |
| Qwen3-8B-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:06:15 PM |
| Qwen3-14B-AWQ ↗ | Text generation | AWQ | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:46:30 PM |
| Qwen3-14B-AWQ ↗ | Text generation | AWQ | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:43:18 PM |
| Qwen3-14B-AWQ ↗ | Text generation | AWQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:40:35 PM |
| Qwen3-14B-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:28:01 PM |
| Qwen3-14B-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:22:42 PM |
| Qwen3-14B-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:17:46 PM |
| Qwen3-VL-2B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:43:26 PM |
| Qwen3-VL-2B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:42:17 PM |
| Qwen3-VL-2B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:41:22 PM |
| Qwen3-VL-4B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:49:57 PM |
| Qwen3-VL-4B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:48:02 PM |
| Qwen3-VL-4B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 2:46:30 PM |
| Qwen3-VL-8B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:13:17 PM |
| Qwen3-VL-8B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:10:15 PM |
| Qwen3-VL-8B-Instruct-FP8 ↗ | Text generation | FP8 | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 3:07:45 PM |
| Qwen3.8-27B-UD-Q4_K_M.gguf ↗ | Text generation | Q4_K_M | llamacpp | gsm8k-test-zero-shot-strict-v2-n3-mt16384-t0 | quality-v1 | 100% | 10/14/2026, 1:13:34 PM |
| Qwen3.8-27B-UD-Q4_K_M.gguf ↗ | Text generation | Q4_K_M | llamacpp | mmlu-test-zero-shot-strict-v2-n3-mt16384-t0 | quality-v1 | 100% | 10/14/2026, 1:12:01 PM |
| Qwen3.8-27B-UD-Q4_K_M.gguf ↗ | Text generation | Q4_K_M | llamacpp | 32768in-2048out | wbench-fixed-http-v1 | — | 10/14/2026, 1:05:27 PM |
| Qwen3.8-27B-UD-Q4_K_M.gguf ↗ | Text generation | Q4_K_M | llamacpp | 8192in-2048out | wbench-fixed-http-v1 | — | 10/14/2026, 12:56:36 PM |
Running benchmarks
0Updates automaticallyNo benchmarks are currently reported as running.
L4 · 24 GB · 91 queued · 0 running · 0 awaiting reassignment · 22 succeeded · 7 failed or exhausted
Methodology
Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.
Updated 11/5/2026, 11:02:27 PM