3 GPUs on the selected test

embeddinggemma-2

GPUs are compared only on one exact test of this model. Change the test to see other conditions.

Run this model

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

Test conditions

Every GPU below ran exactly these conditions.

Task: Embedding · Text → Vectors
Workload: Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s
Batch size: Batch 16
Format · Engine: nf4 · sentence-transformers · PyTorch 2.12.1+cu126
Unit: vectors/sec

Other test conditions

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Protocol details

native-forward-v1 · embedding-mteb-tasks-v1 · embedding

GPUs on this test

#1RTX PRO 6000 Blackwell Server Edition · 96 GBPyTorch 2.12.1+cu130
Mean throughput
2,572vectors/sec
Observed GPU occupancy
1.3GiB
P95 latency
7 ms
Reports combined
1
Runs, parameters and setup ↗
#2L4 · 24 GBPyTorch 2.12.1+cu126
Mean throughput
1,639vectors/sec
Observed GPU occupancy
0.9GiB
P95 latency
10 ms
Reports combined
6
Runs, parameters and setup ↗
#3T4 · 16 GBPyTorch 2.12.1+cu126
Mean throughput
815vectors/sec
Observed GPU occupancy
0.7GiB
P95 latency
20 ms
Reports combined
2
Runs, parameters and setup ↗

1 more GPUs ran this model under other tests

vGPU-32GB · 32 GBNo result on this exact test

Sortable table

Hardware ranking · 4 rows
Hardware	Mean throughput (vectors/sec)	P95 (s)	TTFT (ms)	Observed occupancy (GiB)	GPU power (W)	Throughput / W	VRAM per GPU (GB)	Reports combined
#1RTX PRO 6000 Blackwell Server Edition · 96 GBPyTorch 2.12.1+cu130Runs, parameters and setup ↗	2,571.53	0.007	—	1.33	162.75	15.8	96	1
#2L4 · 24 GBPyTorch 2.12.1+cu126Runs, parameters and setup ↗	1,639.08	0.01	—	0.85	59.75	27.44	24	6
#3T4 · 16 GBPyTorch 2.12.1+cu126Runs, parameters and setup ↗	814.64	0.02	—	0.75	64.13	12.7	16	2
vGPU-32GB · 32 GBNo matching measurements	—	—	—	—	—	—	32	—

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

How results are grouped

Results merge successful runs of the same model, precision and workload. CPU thread counts and CUDA wheel builds may differ; actual PyTorch builds are shown. Other recorded protocol settings remain separate.

Updated 11/5/2026, 11:02:27 PM