16 GB · 37 reports
T4 · 16 GB
Qualified results first; original reports and full tables stay below.
All tasks · Qualified results
Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.
Qwen3-4B-AWQ
Qwen · 1 tested configuration
- AWQ · 8K → 2K tokens · No cache reuse
- Output throughput · includes prefill · 1 request
- 15.4tok/s
- Best measured · 4 concurrent requests
- 20.3tok/s
- Observed GPU occupancy
- 12.7GiB
- First token
- 28.5 s
Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUsTest details · 2 concurrency levels
Total output tokens divided by the full request window, including prefill and queueing.
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- vllm · vLLM 0.29.0
- Protocol
- wbench-fixed-http-v3
- Dataset
- migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
- Workload
- 8192in-2048out
Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report C1 15.39 133.36 s 28.5 s 12.74 67.9 1 fcfe2d64 ↗ C4 20.31 468.13 s 53.1 s 12.74 67.5 1 fcfe2d64 ↗
embeddinggemma-2
unsloth · 2 tested configurations
- fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 109vectors/sec
- Best measured · Batch 16
- 1,002vectors/sec
- Observed GPU occupancy
- 0.9GiB
- P95 latency
- 9 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 108.76 9 ms — 0.89 50.3 2 cfe051ff ↗ B4 365.82 11 ms — 0.89 58.1 2 cfe051ff ↗ B16 1,002.27 16 ms — 0.89 63.9 2 cfe051ff ↗ 1 other test configurations on this GPU
- nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 90.1vectors/sec
- Best measured · Batch 16
- 809vectors/sec
- Observed GPU occupancy
- 0.7GiB
- P95 latency
- 11 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 90.07 11 ms — 0.75 56.2 2 2f9465c4 ↗ B4 302.9 13 ms — 0.75 61.5 2 2f9465c4 ↗ B16 809.43 20 ms — 0.75 64 2 2f9465c4 ↗
embeddinggemma-2
google · 2 tested configurations
- fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 109vectors/sec
- Best measured · Batch 16
- 1,004vectors/sec
- Observed GPU occupancy
- 0.9GiB
- P95 latency
- 9 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 108.65 9 ms — 0.89 49.3 2 2846de88 ↗ B4 365.65 11 ms — 0.89 57.9 2 2846de88 ↗ B16 1,004.11 16 ms — 0.89 63.9 2 2846de88 ↗ 1 other test configurations on this GPU
- nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 90vectors/sec
- Best measured · Batch 16
- 815vectors/sec
- Observed GPU occupancy
- 0.7GiB
- P95 latency
- 11 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 90.02 11 ms — 0.75 54.6 2 43c67a74 ↗ B4 303.35 13 ms — 0.75 60.1 2 43c67a74 ↗ B16 814.64 20 ms — 0.75 64.1 2 43c67a74 ↗
bge-small-zh-v1.5
BAAI · 1 tested configuration
- nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 1,170vectors/sec
- Best measured · Batch 16
- 5,188vectors/sec
- Observed GPU occupancy
- 0.3GiB
- P95 latency
- 1 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 1,169.81 1 ms — 0.29 39 1 0fb03772 ↗ B4 2,301.06 2 ms — 0.29 32.5 1 0fb03772 ↗ B16 5,188.01 3 ms — 0.29 32.5 1 0fb03772 ↗
granite-embedding-97m-multilingual-r2
ibm-granite · 1 tested configuration
- fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 464vectors/sec
- Best measured · Batch 16
- 4,084vectors/sec
- Observed GPU occupancy
- 0.5GiB
- P95 latency
- 2 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 464.26 2 ms — 0.47 45.4 1 37c260e4 ↗ B4 1,713.26 2 ms — 0.47 49.4 1 37c260e4 ↗ B16 4,083.56 4 ms — 0.47 38.3 1 37c260e4 ↗
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Sortable tables by exact test condition · 20 rows
C: concurrency · B: batch · —: not measured
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B1 | 108.76 | 108.81 | — | 0.009 | 0.89 | 50.3 | 108.71 | 2.16 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B1 | 90.07 | 90.08 | — | 0.011 | 0.75 | 56.2 | 90.06 | 1.6 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B1 | 108.65 | 108.66 | — | 0.009 | 0.89 | 49.3 | 108.64 | 2.2 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B1 | 90.02 | 90.07 | — | 0.011 | 0.75 | 54.6 | 89.97 | 1.65 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BAAI/bge-small-zh-v1.5Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B1 | 1,169.81 | 1,169.81 | — | 0.001 | 0.29 | 39 | 1,169.81 | 29.97 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ibm-granite/granite-embedding-97m-multilingual-r2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B1 | 464.26 | 464.26 | — | 0.002 | 0.47 | 45.4 | 464.26 | 10.22 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B4 | 365.82 | 366 | — | 0.011 | 0.89 | 58.1 | 365.65 | 6.3 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B4 | 302.9 | 302.94 | — | 0.013 | 0.75 | 61.5 | 302.87 | 4.93 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B4 | 365.65 | 365.65 | — | 0.011 | 0.89 | 57.9 | 365.65 | 6.32 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B4 | 303.35 | 303.62 | — | 0.013 | 0.75 | 60.1 | 303.08 | 5.05 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BAAI/bge-small-zh-v1.5Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B4 | 2,301.06 | 2,301.06 | — | 0.002 | 0.29 | 32.5 | 2,301.06 | 70.76 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ibm-granite/granite-embedding-97m-multilingual-r2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B4 | 1,713.26 | 1,713.26 | — | 0.002 | 0.47 | 49.4 | 1,713.26 | 34.7 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B16 | 1,002.27 | 1,002.27 | — | 0.016 | 0.89 | 63.9 | 1,002.27 | 15.67 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B16 | 809.43 | 809.77 | — | 0.02 | 0.75 | 64 | 809.1 | 12.65 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B16 | 1,004.11 | 1,006.76 | — | 0.016 | 0.89 | 63.9 | 1,001.46 | 15.7 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B16 | 814.64 | 817.98 | — | 0.02 | 0.75 | 64.1 | 811.3 | 12.7 | 2 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| BAAI/bge-small-zh-v1.5Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B16 | 5,188.01 | 5,188.01 | — | 0.003 | 0.29 | 32.5 | 5,188.01 | 159.61 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ibm-granite/granite-embedding-97m-multilingual-r2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B16 | 4,083.56 | 4,083.56 | — | 0.004 | 0.47 | 38.3 | 4,083.56 | 106.72 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen/Qwen3-4B-AWQScaling curves ↗ | AWQ | vllm | 8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3 | C1 | 15.39 | 15.39 | 28,473.67 | 133.362 | 12.74 | 67.9 | 15.39 | 0.23 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen/Qwen3-4B-AWQScaling curves ↗ | AWQ | vllm | 8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3 | C4 | 20.31 | 20.31 | 53,099.5 | 468.128 | 12.74 | 67.5 | 20.31 | 0.3 | 1 ↗ |
Scaling curves and individual runs
One model and test protocol per curve; different workloads are never joined into one line.
Qwen/Qwen3-4B-AWQ · AWQ · vllm · 8192in-2048out · wbench-fixed-http-v3
Measured points only. Select a point to open its source report.
Measured points only. Select a point to open its source report.
Measured points only. Select a point to open its source report.
Each point is the mean of recorded runs at that level. A source report may differ from the mean; all runs are listed below.
Uncached prefill is unavailable when prompts reuse cache or reliable phase timings are missing. Open a report for its recorded phase timings and cache counts.
| Concurrency | Throughput | Decode (tok/s) | Uncached prefill (tok/s) | GPU power (W) | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | Test date | View report |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 15.39 tok/s | 19.56 | 288.03 | 67.87 | 28,473.67 | 133.362 | 12.74 | 10/23/2026, 6:56:43 AM | fcfe2d64 ↗ |
| 4 | 20.31 tok/s | 5.85 | 221.89 | 67.54 | 53,099.5 | 468.128 | 12.74 | 10/23/2026, 6:56:43 AM | fcfe2d64 ↗ |
Measurements without a comparable protocol · 12 rows
These measured rates remain visible, but are excluded from comparison curves because the protocol is unknown.
| Model | Format | Engine | Workload | Concurrency | Throughput | GPU power (W) |
|---|---|---|---|---|---|---|
| Qwen2.5-0.5B ↗ | Q4_K_M | Ollama | 1K | 1 | 74.68 tok/s | 44.44 |
| Qwen2.5-0.5B ↗ | Q4_K_M | Ollama | 1K | 2 | 153.16 tok/s | 62.92 |
| Qwen2.5-0.5B ↗ | Q4_K_M | Ollama | 1K | 4 | 157.58 tok/s | 63.46 |
| Qwen2.5-0.5B ↗ | Q4_K_M | Ollama | 1K | 8 | 155.79 tok/s | 66.23 |
| Qwen2.5-3B ↗ | Q4_K_M | Ollama | 1K | 1 | 66.07 tok/s | 64.75 |
| Qwen2.5-3B ↗ | Q4_K_M | Ollama | 1K | 2 | 65.56 tok/s | 66.71 |
| Qwen2.5-3B ↗ | Q4_K_M | Ollama | 1K | 4 | 64.08 tok/s | 65.03 |
| Qwen2.5-3B ↗ | Q4_K_M | Ollama | 1K | 8 | 62.28 tok/s | 64.4 |
| Qwen2.5-7B ↗ | Q4_K_M | Ollama | 1K | 1 | 35.25 tok/s | 69.12 |
| Qwen2.5-7B ↗ | Q4_K_M | Ollama | 1K | 2 | 34.23 tok/s | 66.07 |
| Qwen2.5-7B ↗ | Q4_K_M | Ollama | 1K | 4 | 32.67 tok/s | 66.07 |
| Qwen2.5-7B ↗ | Q4_K_M | Ollama | 1K | 8 | 31.77 tok/s | 65.93 |
Original reports
Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.
| Model | Test task | Format | Engine | Workload | Protocol | Quality score | Test date |
|---|---|---|---|---|---|---|---|
| Qwen2.5-0.5B ↗ | Text generation | Q4_K_M | Ollama | 1K | Method unknown | — | 10/14/2026, 8:36:27 AM |
| Qwen2.5-3B ↗ | Text generation | Q4_K_M | Ollama | 1K | Method unknown | — | 10/14/2026, 8:41:27 AM |
| Qwen2.5-7B ↗ | Text generation | Q4_K_M | Ollama | 1K | Method unknown | — | 10/14/2026, 8:47:49 AM |
| qwen2.5-0.5b-instruct-q4_k_m.gguf ↗ | Text generation | Q4_K_M | llama.cpp | 1K | wbench-http | — | 10/14/2026, 9:08:07 AM |
| qwen2.5-1.5b-instruct-q4_k_m.gguf ↗ | Text generation | Q4_K_M | llama.cpp | 1K | wbench-http | — | 10/14/2026, 9:11:29 AM |
| qwen2.5-3b-instruct-q4_k_m.gguf ↗ | Text generation | Q4_K_M | llama.cpp | 1K | wbench-http | — | 10/14/2026, 9:14:47 AM |
| Qwen3-0.6B-GPTQ-Int8 ↗ | Text generation | GPTQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:48:18 PM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:49:10 PM |
| Qwen3-0.6B-GPTQ-Int8 ↗ | Text generation | GPTQ | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:49:44 PM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:51:41 PM |
| Qwen3-0.6B-GPTQ-Int8 ↗ | Text generation | GPTQ | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:54:06 PM |
| Qwen3-8B-AWQ ↗ | Text generation | AWQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:55:56 PM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:58:09 PM |
| Qwen3-8B-AWQ ↗ | Text generation | AWQ | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 4:59:26 PM |
| Qwen3-14B-AWQ ↗ | Text generation | AWQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 5:05:05 PM |
| Qwen3-8B-AWQ ↗ | Text generation | AWQ | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 5:06:50 PM |
| Qwen3-14B-AWQ ↗ | Text generation | AWQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 5:46:36 PM |
| granite-embedding-97m-multilingual-r2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/18/2026, 5:47:32 PM |
| bge-small-zh-v1.5 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/18/2026, 5:50:19 PM |
| Qwen3-14B-AWQ ↗ | Text generation | AWQ | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 5:51:40 PM |
| Qwen3-0.6B-GPTQ-Int8 ↗ | Text generation | GPTQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:06:55 PM |
| Qwen3-0.6B-GPTQ-Int8 ↗ | Text generation | GPTQ | vllm | 2048in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:08:22 PM |
| Qwen3-0.6B-GPTQ-Int8 ↗ | Text generation | GPTQ | vllm | 8192in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:12:44 PM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 512in-512out | wbench-fixed-http-v1 | — | 10/18/2026, 6:19:09 PM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:18:38 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:24:52 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:29:40 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:35:05 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:40:07 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:45:33 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:50:06 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:55:37 AM |
| WeMM-Embedding-2B ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:00:33 AM |
| WeMM-Embedding-2B ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:05:43 AM |
| WeMM-Embedding-4B ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:11:34 AM |
| WeMM-Embedding-9B ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:22:08 AM |
| Qwen3-4B-AWQ ↗ | Text generation | AWQ | vllm | 8192in-2048out | wbench-fixed-http-v3 | — | 10/23/2026, 6:56:43 AM |
Running benchmarks
0Updates automaticallyNo benchmarks are currently reported as running.
T4 · 16 GB · 22 queued · 0 running · 0 awaiting reassignment · 3 succeeded · 8 failed or exhausted
Methodology
Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.
Updated 11/5/2026, 11:02:27 PM