2 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Get your startup command

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

Embedding · RTX PRO 6000 Blackwell Server Edition · 96 GB

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

embeddinggemma-2

google · 2 tested configurations

nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 189vectors/sec
Best measured · Batch 16: 2,572vectors/sec
Observed GPU occupancy: 1.3GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	189.23	6 ms	—	1.33	145.5	1	c673556b ↗
B4	713.32	6 ms	—	1.33	142.8	1	c673556b ↗
B16	2,571.53	7 ms	—	1.33	162.7	1	c673556b ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 207vectors/sec
Best measured · Batch 16: 2,939vectors/sec
Observed GPU occupancy: 1.5GiB
P95 latency: 5 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	206.79	5 ms	—	1.48	119.3	1	196678d3 ↗
B4	764.95	6 ms	—	1.48	141.4	1	196678d3 ↗
B16	2,938.63	6 ms	—	1.48	161.8	1	196678d3 ↗

Qwen3-Embedding-0.6B

Qwen · 3 tested configurations

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 232vectors/sec
Best measured · Batch 32: 3,763vectors/sec
Observed GPU occupancy: 2.2GiB
P95 latency: 5 ms

Use this recipe · Batch 32 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	232.18	5 ms	—	2.22	155	1	65594583 ↗
B8	1,601.21	5 ms	—	2.22	191.2	1	65594583 ↗
B32	3,763.07	9 ms	—	2.22	208.7	1	65594583 ↗

2 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s

Throughput · Batch 1: 218vectors/sec
Best measured · Batch 32: 3,617vectors/sec
Observed GPU occupancy: 2.2GiB
P95 latency: 5 ms

Use this recipe · Batch 32 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	217.92	5 ms	—	2.22	156.5	1	a7d85bcf ↗
B8	1,494.89	5 ms	—	2.22	185.4	1	a7d85bcf ↗
B32	3,616.74	9 ms	—	2.22	206.8	1	a7d85bcf ↗

nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s

Throughput · Batch 1: 196vectors/sec
Best measured · Batch 32: 3,091vectors/sec
Observed GPU occupancy: 1.6GiB
P95 latency: 5 ms

Use this recipe · Batch 32 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	195.8	5 ms	—	1.61	174.8	1	b9501833 ↗
B8	1,156.9	7 ms	—	1.61	196	1	b9501833 ↗
B32	3,091.08	10 ms	—	1.61	207.4	1	b9501833 ↗

Sortable tables by exact test condition · 15 rows

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	217.92	5 ms	—	2.22	156.5	1	a7d85bcf ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	195.8	5 ms	—	1.61	174.8	1	b9501833 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 8 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	1,494.89	5 ms	—	2.22	185.4	1	a7d85bcf ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 8 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	1,156.9	7 ms	—	1.61	196	1	b9501833 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 32 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	3,616.74	9 ms	—	2.22	206.8	1	a7d85bcf ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 15 s · Batch 32 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	3,091.08	10 ms	—	1.61	207.4	1	b9501833 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	189.23	6 ms	—	1.33	145.5	1	c673556b ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	206.79	5 ms	—	1.48	119.3	1	196678d3 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	232.18	5 ms	—	2.22	155	1	65594583 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	713.32	6 ms	—	1.33	142.8	1	c673556b ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	764.95	6 ms	—	1.48	141.4	1	196678d3 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 8 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	1,601.21	5 ms	—	2.22	191.2	1	65594583 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	nf4	sentence-transformersPyTorch 2.12.1+cu130	2,571.53	7 ms	—	1.33	162.7	1	c673556b ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	2,938.63	6 ms	—	1.48	161.8	1	196678d3 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 32 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	RTX PRO 6000 Blackwell Server Edition · 96 GB	fp16	sentence-transformersPyTorch 2.12.1+cu130	3,763.07	9 ms	—	2.22	208.7	1	65594583 ↗