6 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Get your startup command

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

Embedding · All measured GPUs

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

embeddinggemma-2

google · 11 tested configurations · 4 GPUs

L4 · 24 GBnf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 175vectors/sec
Best measured · Batch 16: 1,639vectors/sec
Observed GPU occupancy: 0.9GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	174.88	6 ms	—	0.85	53.1	6	7ad6a821 ↗
B4	571.03	7 ms	—	0.85	57	6	7ad6a821 ↗
B16	1,639.08	10 ms	—	0.85	59.8	6	7ad6a821 ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 177vectors/sec
Best measured · Batch 16: 2,039vectors/sec
Observed GPU occupancy: 1GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	177.46	6 ms	—	0.99	49.8	6	e0700e04 ↗
B4	650.16	6 ms	—	0.99	56.9	6	e0700e04 ↗
B16	2,039.17	8 ms	—	0.99	58.4	6	e0700e04 ↗

RTX PRO 6000 Blackwell Server Edition · 96 GBnf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 189vectors/sec
Best measured · Batch 16: 2,572vectors/sec
Observed GPU occupancy: 1.3GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	189.23	6 ms	—	1.33	145.5	1	c673556b ↗
B4	713.32	6 ms	—	1.33	142.8	1	c673556b ↗
B16	2,571.53	7 ms	—	1.33	162.7	1	c673556b ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 207vectors/sec
Best measured · Batch 16: 2,939vectors/sec
Observed GPU occupancy: 1.5GiB
P95 latency: 5 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	206.79	5 ms	—	1.48	119.3	1	196678d3 ↗
B4	764.95	6 ms	—	1.48	141.4	1	196678d3 ↗
B16	2,938.63	6 ms	—	1.48	161.8	1	196678d3 ↗

T4 · 16 GBnf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 90vectors/sec
Best measured · Batch 16: 815vectors/sec
Observed GPU occupancy: 0.7GiB
P95 latency: 11 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	90.02	11 ms	—	0.75	54.6	2	43c67a74 ↗
B4	303.35	13 ms	—	0.75	60.1	2	43c67a74 ↗
B16	814.64	20 ms	—	0.75	64.1	2	43c67a74 ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 109vectors/sec
Best measured · Batch 16: 1,004vectors/sec
Observed GPU occupancy: 0.9GiB
P95 latency: 9 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	108.65	9 ms	—	0.89	49.3	2	2846de88 ↗
B4	365.65	11 ms	—	0.89	57.9	2	2846de88 ↗
B16	1,004.11	16 ms	—	0.89	63.9	2	2846de88 ↗

Show 1 more GPUs

vGPU-32GB · 32 GBbf16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s

Throughput · Batch 1: 251vectors/sec
Best measured · Batch 16: 2,710vectors/sec
Observed GPU occupancy: 1GiB
P95 latency: 4 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.13.0+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	251.11	4 ms	—	1.05	99.4	1	f489c748 ↗
B4	924.83	4 ms	—	1.05	111.5	1	f489c748 ↗
B16	2,710.19	6 ms	—	1.05	114.9	1	f489c748 ↗

4 other test configurations on this GPU

bf16 · Text → Vectors · embedding-sanity-v1 · Test duration 6 s
Throughput · Batch 1
38.3vectors/sec
Best measured · Batch 4
139vectors/sec
Observed GPU occupancy
0.8GiB
P95 latency
31 ms
Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers · PyTorch 2.13.0+cu130
Protocol
native-forward-v1
Dataset
embedding-sanity-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 38.33 31 ms — 0.82 50.5 4 2c17d254 ↗
B4 138.64 33 ms — 0.82 54.2 4 2c17d254 ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	38.33	31 ms	—	0.82	50.5	4	2c17d254 ↗
B4	138.64	33 ms	—	0.82	54.2	4	2c17d254 ↗

bf16 · Text → Vectors · embedding-sanity-v1 · Test duration 12 s

Throughput · Batch 1: 38.8vectors/sec
Best measured · Batch 64: 1,766vectors/sec
Observed GPU occupancy: 0.8GiB
P95 latency: 27 ms

Use this recipe · Batch 64 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.13.0+cu130
Protocol: native-forward-v1
Dataset: embedding-sanity-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	38.8	27 ms	—	0.82	48.8	1	dc5960cd ↗
B4	141.65	29 ms	—	0.82	52.4	1	dc5960cd ↗
B16	553.54	30 ms	—	0.87	56.4	1	dc5960cd ↗
B64	1,766.47	38 ms	—	1.03	65.3	1	dc5960cd ↗

bf16 · Text → Vectors · embedding-sanity-v1 · Test duration 6 s
Throughput · Batch 1
38.3vectors/sec
Best measured · Batch 4
138vectors/sec
Observed GPU occupancy
0.8GiB
P95 latency
32 ms
Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers
Protocol
native-forward-v1
Dataset
embedding-sanity-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 38.27 32 ms — 0.82 48.6 1 47f71b64 ↗
B4 137.82 35 ms — 0.82 51.9 1 47f71b64 ↗
bf16 · Text → Vectors · embedding-sanity-v1
Throughput · Batch 1
39.4vectors/sec
Best measured · Batch 4
120vectors/sec
Observed GPU occupancy
Not recorded
P95 latency
26 ms
Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers
Protocol
native-forward-v1
Dataset
embedding-sanity-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 39.4 26 ms — — — 1 2c97c869 ↗
B4 119.85 44 ms — — — 1 2c97c869 ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	38.27	32 ms	—	0.82	48.6	1	47f71b64 ↗
B4	137.82	35 ms	—	0.82	51.9	1	47f71b64 ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	39.4	26 ms	—	—	—	1	2c97c869 ↗
B4	119.85	44 ms	—	—	—	1	2c97c869 ↗

embeddinggemma-2

unsloth · 4 tested configurations · 2 GPUs

L4 · 24 GBnf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 175vectors/sec
Best measured · Batch 16: 1,650vectors/sec
Observed GPU occupancy: 0.9GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	174.95	6 ms	—	0.85	54.8	5	29b19870 ↗
B4	572.6	7 ms	—	0.85	58.4	5	29b19870 ↗
B16	1,649.62	10 ms	—	0.85	60.7	5	29b19870 ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 177vectors/sec
Best measured · Batch 16: 2,036vectors/sec
Observed GPU occupancy: 1GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	177.21	6 ms	—	0.99	50.5	4	d74f69cc ↗
B4	648.08	6 ms	—	0.99	57.1	4	d74f69cc ↗
B16	2,035.67	8 ms	—	0.99	58.1	4	d74f69cc ↗

T4 · 16 GBnf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 90.1vectors/sec
Best measured · Batch 16: 809vectors/sec
Observed GPU occupancy: 0.7GiB
P95 latency: 11 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	90.07	11 ms	—	0.75	56.2	2	2f9465c4 ↗
B4	302.9	13 ms	—	0.75	61.5	2	2f9465c4 ↗
B16	809.43	20 ms	—	0.75	64	2	2f9465c4 ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 109vectors/sec
Best measured · Batch 16: 1,002vectors/sec
Observed GPU occupancy: 0.9GiB
P95 latency: 9 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	108.76	9 ms	—	0.89	50.3	2	cfe051ff ↗
B4	365.82	11 ms	—	0.89	58.1	2	cfe051ff ↗
B16	1,002.27	16 ms	—	0.89	63.9	2	cfe051ff ↗

bge-small-zh-v1.5

BAAI · 1 tested configuration · 1 GPU

T4 · 16 GBnf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 1,170vectors/sec
Best measured · Batch 16: 5,188vectors/sec
Observed GPU occupancy: 0.3GiB
P95 latency: 1 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	1,169.81	1 ms	—	0.29	39	1	0fb03772 ↗
B4	2,301.06	2 ms	—	0.29	32.5	1	0fb03772 ↗
B16	5,188.01	3 ms	—	0.29	32.5	1	0fb03772 ↗

granite-embedding-97m-multilingual-r2

ibm-granite · 2 tested configurations · 2 GPUs

L4 · 24 GBnf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 792vectors/sec
Best measured · Batch 16: 8,881vectors/sec
Observed GPU occupancy: 0.5GiB
P95 latency: 1 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	791.85	1 ms	—	0.51	37.4	1	d396c54b ↗
B4	2,816.45	1 ms	—	0.51	37.9	1	d396c54b ↗
B16	8,880.86	2 ms	—	0.51	38.9	1	d396c54b ↗

T4 · 16 GBfp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 464vectors/sec
Best measured · Batch 16: 4,084vectors/sec
Observed GPU occupancy: 0.5GiB
P95 latency: 2 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	464.26	2 ms	—	0.47	45.4	1	37c260e4 ↗
B4	1,713.26	2 ms	—	0.47	49.4	1	37c260e4 ↗
B16	4,083.56	4 ms	—	0.47	38.3	1	37c260e4 ↗

Qwen3-Embedding-0.6B

Qwen · 5 tested configurations · 3 GPUs

RTX 4090 · 24 GBfp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 259vectors/sec
Best measured · Batch 16: 2,018vectors/sec
Observed GPU occupancy: 1.8GiB
P95 latency: 4 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	258.52	4 ms	—	1.84	151.6	1	3ba3d2cb ↗
B4	921.44	4 ms	—	1.84	159.5	1	3ba3d2cb ↗
B16	2,017.74	8 ms	—	1.84	133	1	3ba3d2cb ↗

RTX PRO 6000 Blackwell Server Edition · 96 GBfp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 232vectors/sec
Best measured · Batch 32: 3,763vectors/sec
Observed GPU occupancy: 2.2GiB
P95 latency: 5 ms

Use this recipe · Batch 32 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	232.18	5 ms	—	2.22	155	1	65594583 ↗
B8	1,601.21	5 ms	—	2.22	191.2	1	65594583 ↗
B32	3,763.07	9 ms	—	2.22	208.7	1	65594583 ↗

2 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s

Throughput · Batch 1: 218vectors/sec
Best measured · Batch 32: 3,617vectors/sec
Observed GPU occupancy: 2.2GiB
P95 latency: 5 ms

Use this recipe · Batch 32 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	217.92	5 ms	—	2.22	156.5	1	a7d85bcf ↗
B8	1,494.89	5 ms	—	2.22	185.4	1	a7d85bcf ↗
B32	3,616.74	9 ms	—	2.22	206.8	1	a7d85bcf ↗

nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s

Throughput · Batch 1: 196vectors/sec
Best measured · Batch 32: 3,091vectors/sec
Observed GPU occupancy: 1.6GiB
P95 latency: 5 ms

Use this recipe · Batch 32 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	195.8	5 ms	—	1.61	174.8	1	b9501833 ↗
B8	1,156.9	7 ms	—	1.61	196	1	b9501833 ↗
B32	3,091.08	10 ms	—	1.61	207.4	1	b9501833 ↗

V100-PCIE-32GB · 32 GBfp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 95.3vectors/sec
Best measured · Batch 16: 771vectors/sec
Observed GPU occupancy: 1.7GiB
P95 latency: 11 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	95.33	11 ms	—	1.75	70.8	1	f91802d1 ↗
B4	308.67	13 ms	—	1.75	90.1	1	f91802d1 ↗
B16	771.26	21 ms	—	1.75	99.2	1	f91802d1 ↗

Qwen3-Embedding-4B

Qwen · 4 tested configurations · 2 GPUs

RTX 4090 · 24 GBfp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 3 s
Throughput · Batch 16
498vectors/sec
Observed GPU occupancy
8.2GiB
P95 latency
33 ms
Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs
Test details
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers · PyTorch 2.12.1+cu126
Protocol
native-forward-v1
Dataset
embedding-mteb-tasks-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B16 497.84 33 ms — 8.22 262.8 1 0d6d756e ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B16	497.84	33 ms	—	8.22	262.8	1	0d6d756e ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 81.2vectors/sec
Best measured · Batch 16: 507vectors/sec
Observed GPU occupancy: 8.3GiB
P95 latency: 12 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	81.19	12 ms	—	8.34	258.9	1	4f6f3fbb ↗
B4	279.31	14 ms	—	8.34	270.5	1	4f6f3fbb ↗
B16	507.04	32 ms	—	8.34	264.4	1	4f6f3fbb ↗

V100-PCIE-32GB · 32 GBfp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s

Throughput · Batch 1: 41vectors/sec
Best measured · Batch 16: 229vectors/sec
Observed GPU occupancy: 8.2GiB
P95 latency: 25 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	40.96	25 ms	—	8.25	119.3	1	65a1050f ↗
B4	100.37	40 ms	—	8.25	149.9	1	65a1050f ↗
B16	228.92	70 ms	—	8.25	178.1	1	65a1050f ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 40.9vectors/sec
Best measured · Batch 16: 228vectors/sec
Observed GPU occupancy: 8.2GiB
P95 latency: 25 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	40.94	25 ms	—	8.25	123.4	1	cd3861d6 ↗
B4	100.23	40 ms	—	8.25	146.9	1	cd3861d6 ↗
B16	228.35	70 ms	—	8.25	180.4	1	cd3861d6 ↗

Sortable tables by exact test condition · 77 rows

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 3 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	RTX 4090 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	497.84	33 ms	—	8.22	262.8	1	0d6d756e ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	251.11	4 ms	—	1.05	99.4	1	f489c748 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	40.96	25 ms	—	8.25	119.3	1	65a1050f ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	924.83	4 ms	—	1.05	111.5	1	f489c748 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	100.37	40 ms	—	8.25	149.9	1	65a1050f ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	2,710.19	6 ms	—	1.05	114.9	1	f489c748 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	228.92	70 ms	—	8.25	178.1	1	65a1050f ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	217.92	5 ms	—	2.22	156.5	1	a7d85bcf ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	195.8	5 ms	—	1.61	174.8	1	b9501833 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 8 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	1,494.89	5 ms	—	2.22	185.4	1	a7d85bcf ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 8 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	1,156.9	7 ms	—	1.61	196	1	b9501833 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 32 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	3,616.74	9 ms	—	2.22	206.8	1	a7d85bcf ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 32 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	3,091.08	10 ms	—	1.61	207.4	1	b9501833 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 3 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	174.88	6 ms	—	0.85	53.1	6	7ad6a821 ↗
embeddinggemma-2	T4 · 16 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	90.02	11 ms	—	0.75	54.6	2	43c67a74 ↗
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	189.23	6 ms	—	1.33	145.5	1	c673556b ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 3 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	T4 · 16 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	108.65	9 ms	—	0.89	49.3	2	2846de88 ↗
embeddinggemma-2	L4 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	177.46	6 ms	—	0.99	49.8	6	e0700e04 ↗
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	206.79	5 ms	—	1.48	119.3	1	196678d3 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 2 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	174.95	6 ms	—	0.85	54.8	5	29b19870 ↗
embeddinggemma-2	T4 · 16 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	90.07	11 ms	—	0.75	56.2	2	2f9465c4 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 2 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	T4 · 16 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	108.76	9 ms	—	0.89	50.3	2	cfe051ff ↗
embeddinggemma-2	L4 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	177.21	6 ms	—	0.99	50.5	4	d74f69cc ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
bge-small-zh-v1.5	T4 · 16 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	1,169.81	1 ms	—	0.29	39	1	0fb03772 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
granite-embedding-97m-multilingual-r2	L4 · 24 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	791.85	1 ms	—	0.51	37.4	1	d396c54b ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
granite-embedding-97m-multilingual-r2	T4 · 16 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	464.26	2 ms	—	0.47	45.4	1	37c260e4 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX 4090 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	258.52	4 ms	—	1.84	151.6	1	3ba3d2cb ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	232.18	5 ms	—	2.22	155	1	65594583 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	95.33	11 ms	—	1.75	70.8	1	f91802d1 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	RTX 4090 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	81.19	12 ms	—	8.34	258.9	1	4f6f3fbb ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	40.94	25 ms	—	8.25	123.4	1	cd3861d6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 3 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	571.03	7 ms	—	0.85	57	6	7ad6a821 ↗
embeddinggemma-2	T4 · 16 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	303.35	13 ms	—	0.75	60.1	2	43c67a74 ↗
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	713.32	6 ms	—	1.33	142.8	1	c673556b ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 3 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	650.16	6 ms	—	0.99	56.9	6	e0700e04 ↗
embeddinggemma-2	T4 · 16 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	365.65	11 ms	—	0.89	57.9	2	2846de88 ↗
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	764.95	6 ms	—	1.48	141.4	1	196678d3 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 2 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	572.6	7 ms	—	0.85	58.4	5	29b19870 ↗
embeddinggemma-2	T4 · 16 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	302.9	13 ms	—	0.75	61.5	2	2f9465c4 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 2 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	648.08	6 ms	—	0.99	57.1	4	d74f69cc ↗
embeddinggemma-2	T4 · 16 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	365.82	11 ms	—	0.89	58.1	2	cfe051ff ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
bge-small-zh-v1.5	T4 · 16 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	2,301.06	2 ms	—	0.29	32.5	1	0fb03772 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
granite-embedding-97m-multilingual-r2	L4 · 24 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	2,816.45	1 ms	—	0.51	37.9	1	d396c54b ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
granite-embedding-97m-multilingual-r2	T4 · 16 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	1,713.26	2 ms	—	0.47	49.4	1	37c260e4 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX 4090 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	921.44	4 ms	—	1.84	159.5	1	3ba3d2cb ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	308.67	13 ms	—	1.75	90.1	1	f91802d1 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	RTX 4090 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	279.31	14 ms	—	8.34	270.5	1	4f6f3fbb ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	100.23	40 ms	—	8.25	146.9	1	cd3861d6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 8 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	1,601.21	5 ms	—	2.22	191.2	1	65594583 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 3 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	1,639.08	10 ms	—	0.85	59.8	6	7ad6a821 ↗
embeddinggemma-2	T4 · 16 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	814.64	20 ms	—	0.75	64.1	2	43c67a74 ↗
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	2,571.53	7 ms	—	1.33	162.7	1	c673556b ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 3 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	2,039.17	8 ms	—	0.99	58.4	6	e0700e04 ↗
embeddinggemma-2	T4 · 16 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	1,004.11	16 ms	—	0.89	63.9	2	2846de88 ↗
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	2,938.63	6 ms	—	1.48	161.8	1	196678d3 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 2 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	1,649.62	10 ms	—	0.85	60.7	5	29b19870 ↗
embeddinggemma-2	T4 · 16 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	809.43	20 ms	—	0.75	64	2	2f9465c4 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 2 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	L4 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	2,035.67	8 ms	—	0.99	58.1	4	d74f69cc ↗
embeddinggemma-2	T4 · 16 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	1,002.27	16 ms	—	0.89	63.9	2	cfe051ff ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
bge-small-zh-v1.5	T4 · 16 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	5,188.01	3 ms	—	0.29	32.5	1	0fb03772 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
granite-embedding-97m-multilingual-r2	L4 · 24 GB	nf4	sentence-transformersPyTorch 2.12.1+cu126	8,880.86	2 ms	—	0.51	38.9	1	d396c54b ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
granite-embedding-97m-multilingual-r2	T4 · 16 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	4,083.56	4 ms	—	0.47	38.3	1	37c260e4 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX 4090 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	2,017.74	8 ms	—	1.84	133	1	3ba3d2cb ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	771.26	21 ms	—	1.75	99.2	1	f91802d1 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	RTX 4090 · 24 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	507.04	32 ms	—	8.34	264.4	1	4f6f3fbb ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	228.35	70 ms	—	8.25	180.4	1	cd3861d6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 32 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	3,763.07	9 ms	—	2.22	208.7	1	65594583 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	39.4	26 ms	—	—	—	1	2c97c869 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	119.85	44 ms	—	—	—	1	2c97c869 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	38.33	31 ms	—	0.82	50.5	4	2c17d254 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	38.27	32 ms	—	0.82	48.6	1	47f71b64 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	138.64	33 ms	—	0.82	54.2	4	2c17d254 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	137.82	35 ms	—	0.82	51.9	1	47f71b64 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	38.8	27 ms	—	0.82	48.8	1	dc5960cd ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	141.65	29 ms	—	0.82	52.4	1	dc5960cd ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	553.54	30 ms	—	0.87	56.4	1	dc5960cd ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 64 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	1,766.47	38 ms	—	1.03	65.3	1	dc5960cd ↗

Models with reports but no qualified results · 3

Their original reports remain available on each model page. They did not qualify for comparison; each report states why.

Models with reports but no qualified results · 3 rows
Model	Test task
WeMM-Embedding-2Btencent/WeMM-Embedding-2B	Embedding
WeMM-Embedding-4Btencent/WeMM-Embedding-4B	Embedding
WeMM-Embedding-9Btencent/WeMM-Embedding-9B	Embedding