0 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Get your startup command

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

Speech recognition · B200 · 179 GB

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

No qualified results yet · Speech recognition · B200 · 179 GB

No reports for this task have been recorded here. This does not show that the model or GPU cannot run it.

Check model launch support Measured alternatives for this task Test your own GPU

Measured alternatives on other GPUs

Measured on different GPUs with the same or smaller listed capacity. These are references for choosing hardware, not performance or compatibility evidence for your selected GPU.

granite-speech-5.0-470m-turboctc

ibm-granite · 2 tested configurations · 1 GPU

RTX PRO 6000 Blackwell Server Edition · 96 GBNative · Audio → Text · asr_prompts · Test duration 16 s

Throughput · Batch 1: 2,662audio-sec/sec
Best measured · Batch 32: 9,194audio-sec/sec
Observed GPU occupancy: 1.6GiB
P95 latency: Not recorded

Evidence report and run availability ↗Same test on other GPUs

Test details · 11 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: transformers-ctc · PyTorch 2.13.0
Protocol: audio-native-v1
Dataset: asr_prompts
Workload: asr

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	2,662.33	—	—	1.64	122.1	1	3be7f67e ↗
B2	4,844.87	—	—	1.67	134.9	1	3be7f67e ↗
B4	6,237.58	—	—	1.75	148	1	3be7f67e ↗
B8	7,907.41	—	—	1.9	165.1	1	3be7f67e ↗
B16	9,083.29	—	—	2.33	194.8	1	3be7f67e ↗
B32	9,193.57	—	—	3.25	243.5	1	3be7f67e ↗
B64	8,622.98	—	—	5.09	345.5	1	3be7f67e ↗
B128	8,355.43	—	—	10.1	451	1	3be7f67e ↗
B256	8,343	—	—	20.09	477.6	1	3be7f67e ↗
B512	8,391.94	—	—	40.63	492.3	1	3be7f67e ↗
B1024	8,387.34	—	—	80.89	501.3	1	3be7f67e ↗

1 other test configurations on this GPU

Native · Audio → Text · asr_prompts

Throughput · Batch 1: 3,143audio-sec/sec
Best measured · Batch 16: 8,581audio-sec/sec
Observed GPU occupancy: 1.6GiB
P95 latency: Not recorded

Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: transformers-ctc · PyTorch 2.13.0
Protocol: audio-native-v1
Dataset: asr_prompts
Workload: asr

Batch size	Mean throughput (audio-sec/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	3,143.16	—	—	1.64	95.8	1	d77a0cfd ↗
B8	8,162.53	—	—	1.64	105.2	1	d77a0cfd ↗
B16	8,581.45	—	—	1.92	111.9	1	d77a0cfd ↗

whisper-tiny
openai · 7 tested configurations · 4 GPUs
- RTX 3080 Ti · 12 GBfp16 · Audio → Text · asr_prompts · Test duration 2 s
  Throughput · Batch 1
  65.7audio-sec/sec
  Best measured · Batch 4
  91.8audio-sec/sec
  Observed GPU occupancy
  0.4GiB
  P95 latency
  Not recorded
  Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 batch sizes
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  faster-whisper · PyTorch 2.12.1+cu130
  Protocol
  audio-native-v1
  Dataset
  asr_prompts
  Workload
  asr
  Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 65.65 — — 0.41 134.9 1 3f8b97e6 ↗
  B4 91.84 — — 0.4 143.6 1 3f8b97e6 ↗
Show 3 more GPUs
- RTX PRO 6000 Blackwell Server Edition · 96 GBfp16 · Audio → Text · asr_prompts · Test duration 2 s
  Throughput · Batch 1
  233audio-sec/sec
  Best measured · Batch 4
  364audio-sec/sec
  Observed GPU occupancy
  1.4GiB
  P95 latency
  Not recorded
  Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 batch sizes
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  faster-whisper · PyTorch 2.14.1
  Protocol
  audio-native-v1
  Dataset
  asr_prompts
  Workload
  asr
  Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 232.81 — — 1.42 113.5 2 ba5a45ac ↗
  B4 364.19 — — 1.43 115.8 2 ba5a45ac ↗
- V100-PCIE-32GB · 32 GBfp16 · Audio → Text · asr_prompts · Test duration 1 s
  Throughput · Batch 1
  105audio-sec/sec
  Best measured · Batch 4
  156audio-sec/sec
  Observed GPU occupancy
  0.5GiB
  P95 latency
  Not recorded
  Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 batch sizes
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  faster-whisper · PyTorch 2.12.1+cu126
  Protocol
  audio-native-v1
  Dataset
  asr_prompts
  Workload
  asr
  Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 104.96 — — 0.51 51.5 1 64035c7b ↗
  B4 155.93 — — 0.52 50.1 1 64035c7b ↗
- 1 other test configurations on this GPU
  - fp16 · Audio → Text · asr_prompts · Test duration 5 s
    Throughput · Batch 1
    90.9audio-sec/sec
    Best measured · Batch 4
    132audio-sec/sec
    Observed GPU occupancy
    0.5GiB
    P95 latency
    Not recorded
    Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    faster-whisper · PyTorch 2.12.1+cu126
    Protocol
    audio-native-v1
    Dataset
    asr_prompts
    Workload
    asr
    Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 90.94 — — 0.51 51.5 10 e4ef00c1 ↗
    B4 132.16 — — 0.52 51.2 10 e4ef00c1 ↗
- vGPU-32GB · 32 GBfp16 · Audio → Text · asr_prompts
  Throughput · Batch 1
  73.7audio-sec/sec
  Observed GPU occupancy
  0.4GiB
  P95 latency
  Not recorded
  Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 batch sizes
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  faster-whisper · PyTorch 2.13.0+cu130
  Protocol
  audio-native-v1
  Dataset
  asr_prompts
  Workload
  asr
  Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 73.68 — — 0.38 46.8 1 7d182de2 ↗
  B4 68.2 — — 0.38 49.2 1 7d182de2 ↗
- 2 other test configurations on this GPU
  - fp16 · Audio → Text · asr_prompts · Test duration 4 s
    Throughput · Batch 1
    122audio-sec/sec
    Best measured · Batch 4
    168audio-sec/sec
    Observed GPU occupancy
    0.4GiB
    P95 latency
    Not recorded
    Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    faster-whisper
    Protocol
    audio-native-v1
    Dataset
    asr_prompts
    Workload
    asr
    Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 121.9 — — 0.42 73.6 1 7aaa1f0d ↗
    B4 168.25 — — 0.42 77.4 1 7aaa1f0d ↗
  - fp16 · Audio → Text · asr_prompts · Test duration 1 s
    Throughput · Batch 1
    137audio-sec/sec
    Best measured · Batch 4
    205audio-sec/sec
    Observed GPU occupancy
    0.4GiB
    P95 latency
    Not recorded
    Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 batch sizes
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    faster-whisper · PyTorch 2.13.0+cu130
    Protocol
    audio-native-v1
    Dataset
    asr_prompts
    Workload
    asr
    Batch size Mean throughput (audio-sec/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 137.01 — — 0.38 54.3 1 4b83f25d ↗
    B4 205.27 — — 0.38 66.3 1 4b83f25d ↗