4 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Get your startup command

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

All tasks · vGPU-32GB · 32 GB

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

embeddinggemma-2

google · 5 tested configurations

bf16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s

Throughput · Batch 1: 251vectors/sec
Best measured · Batch 16: 2,710vectors/sec
Observed GPU occupancy: 1GiB
P95 latency: 4 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.13.0+cu130
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	251.11	4 ms	—	1.05	99.4	1	f489c748 ↗
B4	924.83	4 ms	—	1.05	111.5	1	f489c748 ↗
B16	2,710.19	6 ms	—	1.05	114.9	1	f489c748 ↗

4 other test configurations on this GPU

bf16 · Text → Vectors · embedding-sanity-v1 · Test duration 6 s
Throughput · Batch 1
38.3vectors/sec
Best measured · Batch 4
139vectors/sec
Observed GPU occupancy
0.8GiB
P95 latency
31 ms
Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers · PyTorch 2.13.0+cu130
Protocol
native-forward-v1
Dataset
embedding-sanity-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 38.33 31 ms — 0.82 50.5 4 2c17d254 ↗
B4 138.64 33 ms — 0.82 54.2 4 2c17d254 ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	38.33	31 ms	—	0.82	50.5	4	2c17d254 ↗
B4	138.64	33 ms	—	0.82	54.2	4	2c17d254 ↗

bf16 · Text → Vectors · embedding-sanity-v1 · Test duration 12 s

Throughput · Batch 1: 38.8vectors/sec
Best measured · Batch 64: 1,766vectors/sec
Observed GPU occupancy: 0.8GiB
P95 latency: 27 ms

Use this recipe · Batch 64 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.13.0+cu130
Protocol: native-forward-v1
Dataset: embedding-sanity-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	38.8	27 ms	—	0.82	48.8	1	dc5960cd ↗
B4	141.65	29 ms	—	0.82	52.4	1	dc5960cd ↗
B16	553.54	30 ms	—	0.87	56.4	1	dc5960cd ↗
B64	1,766.47	38 ms	—	1.03	65.3	1	dc5960cd ↗

bf16 · Text → Vectors · embedding-sanity-v1 · Test duration 6 s
Throughput · Batch 1
38.3vectors/sec
Best measured · Batch 4
138vectors/sec
Observed GPU occupancy
0.8GiB
P95 latency
32 ms
Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers
Protocol
native-forward-v1
Dataset
embedding-sanity-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 38.27 32 ms — 0.82 48.6 1 47f71b64 ↗
B4 137.82 35 ms — 0.82 51.9 1 47f71b64 ↗
bf16 · Text → Vectors · embedding-sanity-v1
Throughput · Batch 1
39.4vectors/sec
Best measured · Batch 4
120vectors/sec
Observed GPU occupancy
Not recorded
P95 latency
26 ms
Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
sentence-transformers
Protocol
native-forward-v1
Dataset
embedding-sanity-v1
Workload
embedding
Batch size Mean throughput (vectors/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 39.4 26 ms — — — 1 2c97c869 ↗
B4 119.85 44 ms — — — 1 2c97c869 ↗

Kokoro-82M
hexgrad · 3 tested configurations
- fp32 · Text → Audio · kokoro-english-v1 · Test duration 6 s
  Throughput · Batch 1
  43.4audio-sec/sec
  Best measured · Batch 4
  44.9audio-sec/sec
  Observed GPU occupancy
  1GiB
  P95 latency
  101 ms
  Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 batch sizes
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  kokoro
  Protocol
  audio-native-v1
  Dataset
  kokoro-english-v1
  Workload
  tts
  Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 43.43 101 ms — 0.99 64.1 1 d8e4b3ce ↗
  B4 44.88 396 ms — 1.03 69.1 1 d8e4b3ce ↗
- 2 other test configurations on this GPU
  - fp32 · Text → Audio · kokoro-english-v1 · Test duration 6 s
    Throughput · Batch 1
    40.4audio-sec/sec
    Best measured · Batch 4
    41.8audio-sec/sec
    Observed GPU occupancy
    1GiB
    P95 latency
    104 ms
    Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    kokoro · PyTorch 2.13.0+cu130
    Protocol
    audio-native-v1
    Dataset
    kokoro-english-v1
    Workload
    tts
    Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 40.36 104 ms — 0.99 60.5 1 c0488108 ↗
    B4 41.8 426 ms — 1.03 65.9 1 c0488108 ↗
  - fp32 · Text → Audio · kokoro-english-v1 · Test duration 6 s
    Throughput · Batch 1
    76.7audio-sec/sec
    Best measured · Batch 2
    79.8audio-sec/sec
    Observed GPU occupancy
    1GiB
    P95 latency
    45 ms
    Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    kokoro · PyTorch 2.13.0+cu130
    Protocol
    audio-native-v1
    Dataset
    kokoro-english-v1
    Workload
    tts
    Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 76.72 45 ms — 1 83.5 1 d90a5c29 ↗
    B2 79.84 100 ms — 1.03 100.8 1 d90a5c29 ↗
whisper-tiny
openai · 3 tested configurations
- fp16 · Audio → Text · asr_prompts
  Throughput · Batch 1
  73.7audio-sec/sec
  Observed GPU occupancy
  0.4GiB
  P95 latency
  Not recorded
  Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 batch sizes
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  faster-whisper · PyTorch 2.13.0+cu130
  Protocol
  audio-native-v1
  Dataset
  asr_prompts
  Workload
  asr
  Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 73.68 — — 0.38 46.8 1 7d182de2 ↗
  B4 68.2 — — 0.38 49.2 1 7d182de2 ↗
- 2 other test configurations on this GPU
  - fp16 · Audio → Text · asr_prompts · Test duration 4 s
    Throughput · Batch 1
    122audio-sec/sec
    Best measured · Batch 4
    168audio-sec/sec
    Observed GPU occupancy
    0.4GiB
    P95 latency
    Not recorded
    Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    faster-whisper
    Protocol
    audio-native-v1
    Dataset
    asr_prompts
    Workload
    asr
    Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 121.9 — — 0.42 73.6 1 7aaa1f0d ↗
    B4 168.25 — — 0.42 77.4 1 7aaa1f0d ↗
  - fp16 · Audio → Text · asr_prompts · Test duration 1 s
    Throughput · Batch 1
    137audio-sec/sec
    Best measured · Batch 4
    205audio-sec/sec
    Observed GPU occupancy
    0.4GiB
    P95 latency
    Not recorded
    Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    faster-whisper · PyTorch 2.13.0+cu130
    Protocol
    audio-native-v1
    Dataset
    asr_prompts
    Workload
    asr
    Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 137.01 — — 0.38 54.3 1 4b83f25d ↗
    B4 205.27 — — 0.38 66.3 1 4b83f25d ↗
clef-flash
Cloudflare · 4 tested configurations
- bf16 · Text → Structured data · typed-invoices-v1 · Test duration 1 s
  Throughput · Batch 1
  10.3decisions/sec
  Best measured · Batch 4
  10.9decisions/sec
  Observed GPU occupancy
  18.2GiB
  P95 latency
  104 ms
  Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 batch sizes
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  clef-native
  Protocol
  native-forward-v1
  Dataset
  typed-invoices-v1
  Workload
  decision
  Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 10.26 104 ms — 18.17 67.9 1 1d1661ad ↗
  B4 10.89 389 ms — 18.49 156 1 1d1661ad ↗
- 3 other test configurations on this GPU
  - bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
    Throughput · Batch 1
    10.6decisions/sec
    Best measured · Batch 4
    11decisions/sec
    Observed GPU occupancy
    18.2GiB
    P95 latency
    99 ms
    Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    clef-native · PyTorch 2.13.0+cu130
    Protocol
    native-forward-v1
    Dataset
    typed-invoices-v1
    Workload
    decision
    Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 10.56 99 ms — 18.17 206.1 2 63f44036 ↗
    B4 11.04 402 ms — 18.55 244.4 2 63f44036 ↗
  - bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
    Throughput · Batch 1
    10.6decisions/sec
    Best measured · Batch 4
    11.1decisions/sec
    Observed GPU occupancy
    18.2GiB
    P95 latency
    103 ms
    Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    clef-native
    Protocol
    native-forward-v1
    Dataset
    typed-invoices-v1
    Workload
    decision
    Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 10.56 103 ms — 18.17 196.8 1 4eb73a4d ↗
    B4 11.1 390 ms — 18.56 235.9 1 4eb73a4d ↗
  - bf16 · 1K → 1K tokens
    Throughput · Batch 1
    10.2decisions/sec
    Best measured · Batch 4
    10.9decisions/sec
    Observed GPU occupancy
    18.2GiB
    P95 latency
    104 ms
    Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vLLM
    Protocol
    native-forward-v1
    Dataset
    typed-invoices-v1
    Workload
    decision
    Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 10.19 104 ms — 18.17 69.8 1 caef7154 ↗
    B4 10.88 388 ms — 18.49 152.8 1 caef7154 ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	38.27	32 ms	—	0.82	48.6	1	47f71b64 ↗
B4	137.82	35 ms	—	0.82	51.9	1	47f71b64 ↗

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	39.4	26 ms	—	—	—	1	2c97c869 ↗
B4	119.85	44 ms	—	—	—	1	2c97c869 ↗

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	43.43	101 ms	—	0.99	64.1	1	d8e4b3ce ↗
B4	44.88	396 ms	—	1.03	69.1	1	d8e4b3ce ↗

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	40.36	104 ms	—	0.99	60.5	1	c0488108 ↗
B4	41.8	426 ms	—	1.03	65.9	1	c0488108 ↗

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	76.72	45 ms	—	1	83.5	1	d90a5c29 ↗
B2	79.84	100 ms	—	1.03	100.8	1	d90a5c29 ↗

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	73.68	—	—	0.38	46.8	1	7d182de2 ↗
B4	68.2	—	—	0.38	49.2	1	7d182de2 ↗

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	121.9	—	—	0.42	73.6	1	7aaa1f0d ↗
B4	168.25	—	—	0.42	77.4	1	7aaa1f0d ↗

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	137.01	—	—	0.38	54.3	1	4b83f25d ↗
B4	205.27	—	—	0.38	66.3	1	4b83f25d ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	10.26	104 ms	—	18.17	67.9	1	1d1661ad ↗
B4	10.89	389 ms	—	18.49	156	1	1d1661ad ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	10.56	99 ms	—	18.17	206.1	2	63f44036 ↗
B4	11.04	402 ms	—	18.55	244.4	2	63f44036 ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	10.56	103 ms	—	18.17	196.8	1	4eb73a4d ↗
B4	11.1	390 ms	—	18.56	235.9	1	4eb73a4d ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	10.19	104 ms	—	18.17	69.8	1	caef7154 ↗
B4	10.88	388 ms	—	18.49	152.8	1	caef7154 ↗

Sortable tables by exact test condition · 33 rows

Decision · 1K → 1K tokens · Batch 1 · decisions/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
clef-flash	vGPU-32GB · 32 GB	bf16	vLLM	10.19	104 ms	—	18.17	69.8	1	caef7154 ↗

Decision · 1K → 1K tokens · Batch 4 · decisions/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
clef-flash	vGPU-32GB · 32 GB	bf16	vLLM	10.88	388 ms	—	18.49	152.8	1	caef7154 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 1 s · Batch 1 · decisions/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
clef-flash	vGPU-32GB · 32 GB	bf16	clef-native	10.26	104 ms	—	18.17	67.9	1	1d1661ad ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 1 s · Batch 4 · decisions/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
clef-flash	vGPU-32GB · 32 GB	bf16	clef-native	10.89	389 ms	—	18.49	156	1	1d1661ad ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 1 · decisions/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
clef-flash	vGPU-32GB · 32 GB	bf16	clef-nativePyTorch 2.13.0+cu130	10.56	99 ms	—	18.17	206.1	2	63f44036 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 1 · decisions/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
clef-flash	vGPU-32GB · 32 GB	bf16	clef-native	10.56	103 ms	—	18.17	196.8	1	4eb73a4d ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 4 · decisions/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
clef-flash	vGPU-32GB · 32 GB	bf16	clef-nativePyTorch 2.13.0+cu130	11.04	402 ms	—	18.55	244.4	2	63f44036 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 4 · decisions/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
clef-flash	vGPU-32GB · 32 GB	bf16	clef-native	11.1	390 ms	—	18.56	235.9	1	4eb73a4d ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	251.11	4 ms	—	1.05	99.4	1	f489c748 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	924.83	4 ms	—	1.05	111.5	1	f489c748 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 9 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	2,710.19	6 ms	—	1.05	114.9	1	f489c748 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	39.4	26 ms	—	—	—	1	2c97c869 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	119.85	44 ms	—	—	—	1	2c97c869 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	38.33	31 ms	—	0.82	50.5	4	2c17d254 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	38.27	32 ms	—	0.82	48.6	1	47f71b64 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	138.64	33 ms	—	0.82	54.2	4	2c17d254 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 6 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformers	137.82	35 ms	—	0.82	51.9	1	47f71b64 ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 1 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	38.8	27 ms	—	0.82	48.8	1	dc5960cd ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 4 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	141.65	29 ms	—	0.82	52.4	1	dc5960cd ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 16 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	553.54	30 ms	—	0.87	56.4	1	dc5960cd ↗

Embedding · Text → Vectors · embedding-sanity-v1 · Test duration 12 s · Batch 64 · vectors/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
embeddinggemma-2	vGPU-32GB · 32 GB	bf16	sentence-transformersPyTorch 2.13.0+cu130	1,766.47	38 ms	—	1.03	65.3	1	dc5960cd ↗

Speech recognition · Audio → Text · asr_prompts · Batch 1 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	vGPU-32GB · 32 GB	fp16	faster-whisperPyTorch 2.13.0+cu130	73.68	—	—	0.38	46.8	1	7d182de2 ↗

Speech recognition · Audio → Text · asr_prompts · Batch 4 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	vGPU-32GB · 32 GB	fp16	faster-whisperPyTorch 2.13.0+cu130	68.2	—	—	0.38	49.2	1	7d182de2 ↗

Speech recognition · Audio → Text · asr_prompts · Test duration 1 s · Batch 1 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	vGPU-32GB · 32 GB	fp16	faster-whisperPyTorch 2.13.0+cu130	137.01	—	—	0.38	54.3	1	4b83f25d ↗

Speech recognition · Audio → Text · asr_prompts · Test duration 1 s · Batch 4 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	vGPU-32GB · 32 GB	fp16	faster-whisperPyTorch 2.13.0+cu130	205.27	—	—	0.38	66.3	1	4b83f25d ↗

Speech recognition · Audio → Text · asr_prompts · Test duration 4 s · Batch 1 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	vGPU-32GB · 32 GB	fp16	faster-whisper	121.9	—	—	0.42	73.6	1	7aaa1f0d ↗

Speech recognition · Audio → Text · asr_prompts · Test duration 4 s · Batch 4 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
whisper-tiny	vGPU-32GB · 32 GB	fp16	faster-whisper	168.25	—	—	0.42	77.4	1	7aaa1f0d ↗

Speech synthesis · Text → Audio · kokoro-english-v1 · Test duration 6 s · Batch 1 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Kokoro-82M	vGPU-32GB · 32 GB	fp32	kokoro	43.43	101 ms	—	0.99	64.1	1	d8e4b3ce ↗

Speech synthesis · Text → Audio · kokoro-english-v1 · Test duration 6 s · Batch 1 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Kokoro-82M	vGPU-32GB · 32 GB	fp32	kokoroPyTorch 2.13.0+cu130	40.36	104 ms	—	0.99	60.5	1	c0488108 ↗

Speech synthesis · Text → Audio · kokoro-english-v1 · Test duration 6 s · Batch 1 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Kokoro-82M	vGPU-32GB · 32 GB	fp32	kokoroPyTorch 2.13.0+cu130	76.72	45 ms	—	1	83.5	1	d90a5c29 ↗

Speech synthesis · Text → Audio · kokoro-english-v1 · Test duration 6 s · Batch 2 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Kokoro-82M	vGPU-32GB · 32 GB	fp32	kokoroPyTorch 2.13.0+cu130	79.84	100 ms	—	1.03	100.8	1	d90a5c29 ↗

Speech synthesis · Text → Audio · kokoro-english-v1 · Test duration 6 s · Batch 4 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Kokoro-82M	vGPU-32GB · 32 GB	fp32	kokoro	44.88	396 ms	—	1.03	69.1	1	d8e4b3ce ↗

Speech synthesis · Text → Audio · kokoro-english-v1 · Test duration 6 s · Batch 4 · audio-sec/sec · 1 rows
Model	Hardware	Format	Engine	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
Kokoro-82M	vGPU-32GB · 32 GB	fp32	kokoroPyTorch 2.13.0+cu130	41.8	426 ms	—	1.03	65.9	1	c0488108 ↗

Models with reports but no qualified results · 14

Their original reports remain available on each model page. They did not qualify for comparison; each report states why.

Models with reports but no qualified results · 14 rows
Model	Test task
DeepSeek-R1-Distill-Qwen-1.5B/root/autodl-tmp/.cache/modelscope/models/deepseek-ai--DeepSeek-R1-Distill-Qwen-1.5B/snapshots/master	Text generation
Qwen3-0.6BQwen/Qwen3-0.6B	Text generation
Qwen3-0.6B/root/autodl-tmp/.cache/modelscope/models/Qwen--Qwen3-0.6B/snapshots/master	Text generation
Qwen3-0.6B/root/.cache/modelscope/models/Qwen--Qwen3-0.6B/snapshots/master	Text generation
Qwen3-0.6B-FP8/root/autodl-tmp/.cache/modelscope/models/Qwen--Qwen3-0.6B-FP8/snapshots/master	Text generation
Qwen3-0.6B-GPTQ/root/autodl-tmp/.cache/modelscope/models/Qwen--Qwen3-0.6B-GPTQ-Int8/snapshots/master	Text generation
Qwen3-0.6B-Q8/root/.cache/modelscope/models/Qwen--Qwen3-0.6B-GGUF/snapshots/master/Qwen3-0.6B-Q8_0.gguf	Text generation
Qwen3.5-0.8B/root/autodl-tmp/.cache/modelscope/models/Qwen--Qwen3.5-0.8B/snapshots/master	Text generation
Qwen3.5-0.8B/root/.cache/modelscope/models/Qwen--Qwen3.5-0.8B/snapshots/master	Text generation
Qwen3.5-2B/root/.cache/modelscope/models/Qwen--Qwen3.5-2B/snapshots/master	Text generation
Qwen3.5-4B/root/.cache/modelscope/models/Qwen--Qwen3.5-4B/snapshots/master	Text generation
Qwen3.5-9BQwen/Qwen3.5-9B	Text generation
Qwen3.6-27B-AWQ/root/.cache/modelscope/models/tclf90--Qwen3.6-27B-AWQ/snapshots/master	Text generation
WeMM-Embedding-2Btencent/WeMM-Embedding-2B	Embedding

Get your startup command

All tasks · vGPU-32GB · 32 GB

embeddinggemma-2

Kokoro-82M

whisper-tiny

clef-flash