4 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Get your startup command

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

All tasks · V100-PCIE-32GB · 32 GB

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

Qwen3-Embedding-4B

Qwen · 2 tested configurations

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s

Throughput · Batch 1: 41vectors/sec
Best measured · Batch 16: 229vectors/sec
Observed GPU occupancy: 8.2GiB
P95 latency: 25 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	40.96	25 ms	—	8.25	119.3	1	65a1050f ↗
B4	100.37	40 ms	—	8.25	149.9	1	65a1050f ↗
B16	228.92	70 ms	—	8.25	178.1	1	65a1050f ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 40.9vectors/sec
Best measured · Batch 16: 228vectors/sec
Observed GPU occupancy: 8.2GiB
P95 latency: 25 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	40.94	25 ms	—	8.25	123.4	1	cd3861d6 ↗
B4	100.23	40 ms	—	8.25	146.9	1	cd3861d6 ↗
B16	228.35	70 ms	—	8.25	180.4	1	cd3861d6 ↗

Qwen3-Embedding-0.6B

Qwen · 1 tested configuration

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 95.3vectors/sec
Best measured · Batch 16: 771vectors/sec
Observed GPU occupancy: 1.7GiB
P95 latency: 11 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	95.33	11 ms	—	1.75	70.8	1	f91802d1 ↗
B4	308.67	13 ms	—	1.75	90.1	1	f91802d1 ↗
B16	771.26	21 ms	—	1.75	99.2	1	f91802d1 ↗

Kokoro-82M
hexgrad · 1 tested configuration
- fp32 · Text → Audio · kokoro-english-v1 · Test duration 6 s
  Throughput · Batch 1
  36audio-sec/sec
  Best measured · Batch 2
  37.8audio-sec/sec
  Observed GPU occupancy
  1.4GiB
  P95 latency
  105 ms
  Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 batch sizes
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  kokoro · PyTorch 2.12.1+cu126
  Protocol
  audio-native-v1
  Dataset
  kokoro-english-v1
  Workload
  tts
  Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 35.95 105 ms — 1.45 76.3 11 b40f8533 ↗
  B2 37.83 237 ms — 1.45 82.6 11 b40f8533 ↗
whisper-tiny
openai · 2 tested configurations
- fp16 · Audio → Text · asr_prompts · Test duration 1 s
  Throughput · Batch 1
  105audio-sec/sec
  Best measured · Batch 4
  156audio-sec/sec
  Observed GPU occupancy
  0.5GiB
  P95 latency
  Not recorded
  Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 batch sizes
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  faster-whisper · PyTorch 2.12.1+cu126
  Protocol
  audio-native-v1
  Dataset
  asr_prompts
  Workload
  asr
  Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 104.96 — — 0.51 51.5 1 64035c7b ↗
  B4 155.93 — — 0.52 50.1 1 64035c7b ↗
- 1 other test configurations on this GPU
  - fp16 · Audio → Text · asr_prompts · Test duration 5 s
    Throughput · Batch 1
    90.9audio-sec/sec
    Best measured · Batch 4
    132audio-sec/sec
    Observed GPU occupancy
    0.5GiB
    P95 latency
    Not recorded
    Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    faster-whisper · PyTorch 2.12.1+cu126
    Protocol
    audio-native-v1
    Dataset
    asr_prompts
    Workload
    asr
    Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 90.94 — — 0.51 51.5 10 e4ef00c1 ↗
    B4 132.16 — — 0.52 51.2 10 e4ef00c1 ↗

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	35.95	105 ms	—	1.45	76.3	11	b40f8533 ↗
B2	37.83	237 ms	—	1.45	82.6	11	b40f8533 ↗

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	104.96	—	—	0.51	51.5	1	64035c7b ↗
B4	155.93	—	—	0.52	50.1	1	64035c7b ↗

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	90.94	—	—	0.51	51.5	10	e4ef00c1 ↗
B4	132.16	—	—	0.52	51.2	10	e4ef00c1 ↗

Sortable tables by exact test condition · 15 rows

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	40.96	25 ms	—	8.25	119.3	1	65a1050f ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	100.37	40 ms	—	8.25	149.9	1	65a1050f ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	228.92	70 ms	—	8.25	178.1	1	65a1050f ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	40.94	25 ms	—	8.25	123.4	1	cd3861d6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	95.33	11 ms	—	1.75	70.8	1	f91802d1 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	100.23	40 ms	—	8.25	146.9	1	cd3861d6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	308.67	13 ms	—	1.75	90.1	1	f91802d1 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-4B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	228.35	70 ms	—	8.25	180.4	1	cd3861d6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Qwen3-Embedding-0.6B	V100-PCIE-32GB · 32 GB	fp16	sentence-transformersPyTorch 2.12.1+cu126	771.26	21 ms	—	1.75	99.2	1	f91802d1 ↗

Speech recognition · Audio → Text · asr_prompts · Test duration 1 s · Batch 1 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	V100-PCIE-32GB · 32 GB	fp16	faster-whisperPyTorch 2.12.1+cu126	104.96	—	—	0.51	51.5	1	64035c7b ↗

Speech recognition · Audio → Text · asr_prompts · Test duration 1 s · Batch 4 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	V100-PCIE-32GB · 32 GB	fp16	faster-whisperPyTorch 2.12.1+cu126	155.93	—	—	0.52	50.1	1	64035c7b ↗

Speech recognition · Audio → Text · asr_prompts · Test duration 5 s · Batch 1 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	V100-PCIE-32GB · 32 GB	fp16	faster-whisperPyTorch 2.12.1+cu126	90.94	—	—	0.51	51.5	10	e4ef00c1 ↗

Speech recognition · Audio → Text · asr_prompts · Test duration 5 s · Batch 4 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	V100-PCIE-32GB · 32 GB	fp16	faster-whisperPyTorch 2.12.1+cu126	132.16	—	—	0.52	51.2	10	e4ef00c1 ↗

Speech synthesis · Text → Audio · kokoro-english-v1 · Test duration 6 s · Batch 1 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Kokoro-82M	V100-PCIE-32GB · 32 GB	fp32	kokoroPyTorch 2.12.1+cu126	35.95	105 ms	—	1.45	76.3	11	b40f8533 ↗

Speech synthesis · Text → Audio · kokoro-english-v1 · Test duration 6 s · Batch 2 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Kokoro-82M	V100-PCIE-32GB · 32 GB	fp32	kokoroPyTorch 2.12.1+cu126	37.83	237 ms	—	1.45	82.6	11	b40f8533 ↗

Get your startup command

All tasks · V100-PCIE-32GB · 32 GB

Qwen3-Embedding-4B

Qwen3-Embedding-0.6B

Kokoro-82M

whisper-tiny