24 GB · 111 reports
L4 · 24 GB
Qualified results first; original reports and full tables stay below.
Embedding · Qualified results
Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.
embeddinggemma-2
unsloth · 2 tested configurations
- nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 175vectors/sec
- Best measured · Batch 16
- 1,650vectors/sec
- Observed GPU occupancy
- 0.9GiB
- P95 latency
- 6 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 174.95 6 ms — 0.85 54.8 5 29b19870 ↗ B4 572.6 7 ms — 0.85 58.4 5 29b19870 ↗ B16 1,649.62 10 ms — 0.85 60.7 5 29b19870 ↗ 1 other test configurations on this GPU
- fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 177vectors/sec
- Best measured · Batch 16
- 2,036vectors/sec
- Observed GPU occupancy
- 1GiB
- P95 latency
- 6 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 177.21 6 ms — 0.99 50.5 4 d74f69cc ↗ B4 648.08 6 ms — 0.99 57.1 4 d74f69cc ↗ B16 2,035.67 8 ms — 0.99 58.1 4 d74f69cc ↗
embeddinggemma-2
google · 2 tested configurations
- fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 177vectors/sec
- Best measured · Batch 16
- 2,039vectors/sec
- Observed GPU occupancy
- 1GiB
- P95 latency
- 6 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 177.46 6 ms — 0.99 49.8 6 e0700e04 ↗ B4 650.16 6 ms — 0.99 56.9 6 e0700e04 ↗ B16 2,039.17 8 ms — 0.99 58.4 6 e0700e04 ↗ 1 other test configurations on this GPU
- nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 175vectors/sec
- Best measured · Batch 16
- 1,639vectors/sec
- Observed GPU occupancy
- 0.9GiB
- P95 latency
- 6 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 174.88 6 ms — 0.85 53.1 6 7ad6a821 ↗ B4 571.03 7 ms — 0.85 57 6 7ad6a821 ↗ B16 1,639.08 10 ms — 0.85 59.8 6 7ad6a821 ↗
granite-embedding-97m-multilingual-r2
ibm-granite · 1 tested configuration
- nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
- Throughput · Batch 1
- 792vectors/sec
- Best measured · Batch 16
- 8,881vectors/sec
- Observed GPU occupancy
- 0.5GiB
- P95 latency
- 1 ms
Test details · 3 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
- Engine
- sentence-transformers · PyTorch 2.12.1+cu126
- Protocol
- native-forward-v1
- Dataset
- embedding-mteb-tasks-v1
- Workload
- embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report B1 791.85 1 ms — 0.51 37.4 1 d396c54b ↗ B4 2,816.45 1 ms — 0.51 37.9 1 d396c54b ↗ B16 8,880.86 2 ms — 0.51 38.9 1 d396c54b ↗
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Sortable tables by exact test condition · 15 rows
C: concurrency · B: batch · —: not measured
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B1 | 174.95 | 175.2 | — | 0.006 | 0.85 | 54.8 | 174.56 | 3.19 | 5 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B1 | 177.21 | 177.49 | — | 0.006 | 0.99 | 50.5 | 176.49 | 3.51 | 4 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B1 | 177.46 | 177.6 | — | 0.006 | 0.99 | 49.8 | 177.32 | 3.56 | 6 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B1 | 174.88 | 175.5 | — | 0.006 | 0.85 | 53.1 | 173.2 | 3.3 | 6 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ibm-granite/granite-embedding-97m-multilingual-r2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B1 | 791.85 | 791.85 | — | 0.001 | 0.51 | 37.4 | 791.85 | 21.19 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B4 | 572.6 | 573.69 | — | 0.007 | 0.85 | 58.4 | 571.58 | 9.81 | 5 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B4 | 648.08 | 650.84 | — | 0.006 | 0.99 | 57.1 | 641.3 | 11.36 | 4 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B4 | 650.16 | 650.67 | — | 0.006 | 0.99 | 56.9 | 649.39 | 11.44 | 6 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B4 | 571.03 | 573.83 | — | 0.007 | 0.85 | 57 | 564.91 | 10.03 | 6 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ibm-granite/granite-embedding-97m-multilingual-r2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B4 | 2,816.45 | 2,816.45 | — | 0.001 | 0.51 | 37.9 | 2,816.45 | 74.36 | 1 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B16 | 1,649.62 | 1,658.11 | — | 0.01 | 0.85 | 60.7 | 1,646.49 | 27.17 | 5 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| unsloth/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B16 | 2,035.67 | 2,049.35 | — | 0.008 | 0.99 | 58.1 | 2,014.24 | 35.05 | 4 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | fp16 | sentence-transformers | embeddingnative-forward-v1 | B16 | 2,039.17 | 2,043.49 | — | 0.008 | 0.99 | 58.4 | 2,033.4 | 34.93 | 6 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| google/embeddinggemma-2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B16 | 1,639.08 | 1,654.37 | — | 0.01 | 0.85 | 59.8 | 1,612.71 | 27.44 | 6 ↗ |
| Model | Format | Engine | Workload | Concurrency / batch | Mean throughput | Best measured | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | GPU power (W) | Lowest measured | Throughput / W | Reports combined |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ibm-granite/granite-embedding-97m-multilingual-r2Scaling curves ↗ | nf4 | sentence-transformers | embeddingnative-forward-v1 | B16 | 8,880.86 | 8,880.86 | — | 0.002 | 0.51 | 38.9 | 8,880.86 | 228.46 | 1 ↗ |
Scaling curves and individual runs
One model and test protocol per curve; different workloads are never joined into one line.
unsloth/embeddinggemma-2 · nf4 · sentence-transformers · embedding · native-forward-v1
Measured points only. Select a point to open its source report.
Measured points only. Select a point to open its source report.
Measured points only. Select a point to open its source report.
Each point is the mean of recorded runs at that level. A source report may differ from the mean; all runs are listed below.
| Batch size | Throughput | Decode (tok/s) | Uncached prefill (tok/s) | GPU power (W) | TTFT (ms) | P95 (s) | Observed occupancy (GiB) | Test date | View report |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 175.2 vectors/sec | — | Uncached prefill unavailableView report | 54.76 | — | 0.006 | 0.85 | 10/19/2026, 9:24:09 AM | 29b19870 ↗ |
| 1 | 174.56 vectors/sec | — | Uncached prefill unavailableView report | 54.33 | — | 0.006 | 0.85 | 10/19/2026, 9:05:59 AM | d676e386 ↗ |
| 1 | 174.96 vectors/sec | — | Uncached prefill unavailableView report | 55.45 | — | 0.006 | 0.85 | 10/19/2026, 7:53:50 AM | bc9747cb ↗ |
| 1 | 175.13 vectors/sec | — | Uncached prefill unavailableView report | 54.97 | — | 0.006 | 0.85 | 10/19/2026, 7:35:58 AM | e0a15fd7 ↗ |
| 1 | 174.92 vectors/sec | — | Uncached prefill unavailableView report | 54.44 | — | 0.006 | 0.85 | 10/19/2026, 7:17:49 AM | 41a06ae6 ↗ |
| 4 | 573.09 vectors/sec | — | Uncached prefill unavailableView report | 58.66 | — | 0.007 | 0.85 | 10/19/2026, 9:24:09 AM | 29b19870 ↗ |
| 4 | 571.58 vectors/sec | — | Uncached prefill unavailableView report | 58.33 | — | 0.007 | 0.85 | 10/19/2026, 9:05:59 AM | d676e386 ↗ |
| 4 | 573.07 vectors/sec | — | Uncached prefill unavailableView report | 57.94 | — | 0.007 | 0.85 | 10/19/2026, 7:53:50 AM | bc9747cb ↗ |
| 4 | 573.69 vectors/sec | — | Uncached prefill unavailableView report | 58.16 | — | 0.007 | 0.85 | 10/19/2026, 7:35:58 AM | e0a15fd7 ↗ |
| 4 | 571.58 vectors/sec | — | Uncached prefill unavailableView report | 58.68 | — | 0.007 | 0.85 | 10/19/2026, 7:17:49 AM | 41a06ae6 ↗ |
| 16 | 1,646.9 vectors/sec | — | Uncached prefill unavailableView report | 60.8 | — | 0.01 | 0.85 | 10/19/2026, 9:24:09 AM | 29b19870 ↗ |
| 16 | 1,646.49 vectors/sec | — | Uncached prefill unavailableView report | 60.42 | — | 0.01 | 0.85 | 10/19/2026, 9:05:59 AM | d676e386 ↗ |
| 16 | 1,649.28 vectors/sec | — | Uncached prefill unavailableView report | 60.84 | — | 0.01 | 0.85 | 10/19/2026, 7:53:50 AM | bc9747cb ↗ |
| 16 | 1,658.11 vectors/sec | — | Uncached prefill unavailableView report | 60.74 | — | 0.01 | 0.85 | 10/19/2026, 7:35:58 AM | e0a15fd7 ↗ |
| 16 | 1,647.34 vectors/sec | — | Uncached prefill unavailableView report | 60.72 | — | 0.01 | 0.85 | 10/19/2026, 7:17:49 AM | 41a06ae6 ↗ |
Original reports
Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.
| Model | Test task | Format | Engine | Workload | Protocol | Quality score | Test date |
|---|---|---|---|---|---|---|---|
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 9:24:09 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 9:16:01 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 9:05:59 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 8:57:46 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 8:47:51 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 8:39:42 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 8:29:57 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 8:21:47 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 8:12:01 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 8:03:34 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:53:50 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:45:47 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:35:58 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:27:38 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:17:49 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 7:09:29 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:59:52 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:51:53 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:41:51 AM |
| embeddinggemma-2 ↗ | Embedding | fp16 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:33:03 AM |
| embeddinggemma-2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/19/2026, 6:22:26 AM |
| granite-embedding-97m-multilingual-r2 ↗ | Embedding | nf4 | sentence-transformers | embedding | native-forward-v1 | — | 10/18/2026, 5:01:39 PM |
Running benchmarks
0Updates automaticallyNo benchmarks are currently reported as running.
L4 · 24 GB · 91 queued · 0 running · 0 awaiting reassignment · 22 succeeded · 7 failed or exhausted
Methodology
Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.
Updated 11/5/2026, 9:50:21 PM