1 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Get your startup command

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

Embedding · vGPU-32GB · 32 GB

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

embeddinggemma-2

google · 5 tested configurations

bf16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s

Throughput · Batch 1: 251vectors/sec
Best measured · Batch 16: 2,710vectors/sec
Observed GPU occupancy: 1GiB
P95 latency: 4 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.13.0+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	251.11	4 ms	—	1.05	99.4	1	f489c748 ↗
B4	924.83	4 ms	—	1.05	111.5	1	f489c748 ↗
B16	2,710.19	6 ms	—	1.05	114.9	1	f489c748 ↗

4 other test configurations on this GPU

bf16 · Text → Vectors · embedding-sanity-v1 · Test duration 6 s
Throughput · Batch 1
38.3vectors/sec
Best measured · Batch 4
139vectors/sec
Observed GPU occupancy
0.8GiB
P95 latency
31 ms
Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers · PyTorch 2.13.0+cu130
Protocol
native-forward-v1
Dataset
embedding-sanity-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 38.33 31 ms — 0.82 50.5 4 2c17d254 ↗
B4 138.64 33 ms — 0.82 54.2 4 2c17d254 ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	38.33	31 ms	—	0.82	50.5	4	2c17d254 ↗
B4	138.64	33 ms	—	0.82	54.2	4	2c17d254 ↗

bf16 · Text → Vectors · embedding-sanity-v1 · Test duration 12 s

Throughput · Batch 1: 38.8vectors/sec
Best measured · Batch 64: 1,766vectors/sec
Observed GPU occupancy: 0.8GiB
P95 latency: 27 ms

Use this recipe · Batch 64 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.13.0+cu130
Protocol: native-forward-v1
Dataset: embedding-sanity-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	38.8	27 ms	—	0.82	48.8	1	dc5960cd ↗
B4	141.65	29 ms	—	0.82	52.4	1	dc5960cd ↗
B16	553.54	30 ms	—	0.87	56.4	1	dc5960cd ↗
B64	1,766.47	38 ms	—	1.03	65.3	1	dc5960cd ↗

bf16 · Text → Vectors · embedding-sanity-v1 · Test duration 6 s
Throughput · Batch 1
38.3vectors/sec
Best measured · Batch 4
138vectors/sec
Observed GPU occupancy
0.8GiB
P95 latency
32 ms
Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers
Protocol
native-forward-v1
Dataset
embedding-sanity-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 38.27 32 ms — 0.82 48.6 1 47f71b64 ↗
B4 137.82 35 ms — 0.82 51.9 1 47f71b64 ↗
bf16 · Text → Vectors · embedding-sanity-v1
Throughput · Batch 1
39.4vectors/sec
Best measured · Batch 4
120vectors/sec
Observed GPU occupancy
Not recorded
P95 latency
26 ms
Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers
Protocol
native-forward-v1
Dataset
embedding-sanity-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 39.4 26 ms — — — 1 2c97c869 ↗
B4 119.85 44 ms — — — 1 2c97c869 ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	38.27	32 ms	—	0.82	48.6	1	47f71b64 ↗
B4	137.82	35 ms	—	0.82	51.9	1	47f71b64 ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	39.4	26 ms	—	—	—	1	2c97c869 ↗
B4	119.85	44 ms	—	—	—	1	2c97c869 ↗

Sortable tables by exact test condition · 13 rows

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	251.11	4 ms	—	1.05	99.4	1	f489c748 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	924.83	4 ms	—	1.05	111.5	1	f489c748 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	2,710.19	6 ms	—	1.05	114.9	1	f489c748 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	39.4	26 ms	—	—	—	1	2c97c869 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	119.85	44 ms	—	—	—	1	2c97c869 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	38.33	31 ms	—	0.82	50.5	4	2c17d254 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	38.27	32 ms	—	0.82	48.6	1	47f71b64 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	138.64	33 ms	—	0.82	54.2	4	2c17d254 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	137.82	35 ms	—	0.82	51.9	1	47f71b64 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	38.8	27 ms	—	0.82	48.8	1	dc5960cd ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	141.65	29 ms	—	0.82	52.4	1	dc5960cd ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	553.54	30 ms	—	0.87	56.4	1	dc5960cd ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 64 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	1,766.47	38 ms	—	1.03	65.3	1	dc5960cd ↗

Models with reports but no qualified results · 1

Their original reports remain available on each model page. They did not qualify for comparison; each report states why.

Models with reports but no qualified results · 1 rows
Model	Test task
WeMM-Embedding-2Btencent/WeMM-Embedding-2B	Embedding