Skip to content
m.recipes
Find a configuration
Sign in
2 models with qualified results

Models

Measured configurations by model. Open a model to compare GPUs on one exact test.

Get your startup command

Choose your operating system and enter a model repository. Launch support is checked separately from performance results.

Reset

Speech recognition · RTX PRO 6000 Blackwell Server Edition · 96 GB

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

  1. granite-speech-5.0-470m-turboctc

    ibm-granite · 2 tested configurations

    • Native · Audio → Text · asr_prompts · Test duration 16 s
      Throughput · Batch 1
      2,662audio-sec/sec
      Best measured · Batch 32
      9,194audio-sec/sec
      Observed GPU occupancy
      1.6GiB
      P95 latency
      Not recorded
      Evidence report and run availability ↗Same test on other GPUs
      Test details · 11 batch sizes

      Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

      Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

      Engine
      transformers-ctc · PyTorch 2.13.0
      Protocol
      audio-native-v1
      Dataset
      asr_prompts
      Workload
      asr
      Batch sizeMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
      B12,662.33——1.64122.113be7f67e ↗
      B24,844.87——1.67134.913be7f67e ↗
      B46,237.58——1.7514813be7f67e ↗
      B87,907.41——1.9165.113be7f67e ↗
      B169,083.29——2.33194.813be7f67e ↗
      B329,193.57——3.25243.513be7f67e ↗
      B648,622.98——5.09345.513be7f67e ↗
      B1288,355.43——10.145113be7f67e ↗
      B2568,343——20.09477.613be7f67e ↗
      B5128,391.94——40.63492.313be7f67e ↗
      B10248,387.34——80.89501.313be7f67e ↗
    • 1 other test configurations on this GPU
      • Native · Audio → Text · asr_prompts
        Throughput · Batch 1
        3,143audio-sec/sec
        Best measured · Batch 16
        8,581audio-sec/sec
        Observed GPU occupancy
        1.6GiB
        P95 latency
        Not recorded
        Evidence report and run availability ↗Same test on other GPUs
        Test details · 3 batch sizes

        Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

        Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

        Engine
        transformers-ctc · PyTorch 2.13.0
        Protocol
        audio-native-v1
        Dataset
        asr_prompts
        Workload
        asr
        Batch sizeMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
        B13,143.16——1.6495.81d77a0cfd ↗
        B88,162.53——1.64105.21d77a0cfd ↗
        B168,581.45——1.92111.91d77a0cfd ↗

  2. whisper-tiny

    openai · 1 tested configuration

    • fp16 · Audio → Text · asr_prompts · Test duration 2 s
      Throughput · Batch 1
      233audio-sec/sec
      Best measured · Batch 4
      364audio-sec/sec
      Observed GPU occupancy
      1.4GiB
      P95 latency
      Not recorded
      Evidence report and run availability ↗Same test on other GPUs
      Test details · 2 batch sizes

      Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

      Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

      Engine
      faster-whisper · PyTorch 2.14.1
      Protocol
      audio-native-v1
      Dataset
      asr_prompts
      Workload
      asr
      Batch sizeMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
      B1232.81——1.42113.52ba5a45ac ↗
      B4364.19——1.43115.82ba5a45ac ↗
Sortable tables by exact test condition · 16 rows
Speech recognition · Audio → Text · asr_prompts · Batch 1 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.03,143.16——1.6495.81d77a0cfd ↗
Speech recognition · Audio → Text · asr_prompts · Batch 8 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.08,162.53——1.64105.21d77a0cfd ↗
Speech recognition · Audio → Text · asr_prompts · Batch 16 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.08,581.45——1.92111.91d77a0cfd ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 2 s · Batch 1 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
whisper-tinyRTX PRO 6000 Blackwell Server Edition · 96 GBfp16faster-whisperPyTorch 2.14.1232.81——1.42113.52ba5a45ac ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 2 s · Batch 4 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
whisper-tinyRTX PRO 6000 Blackwell Server Edition · 96 GBfp16faster-whisperPyTorch 2.14.1364.19——1.43115.82ba5a45ac ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 1 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.02,662.33——1.64122.113be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 2 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.04,844.87——1.67134.913be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 4 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.06,237.58——1.7514813be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 8 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.07,907.41——1.9165.113be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 16 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.09,083.29——2.33194.813be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 32 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.09,193.57——3.25243.513be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 64 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.08,622.98——5.09345.513be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 128 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.08,355.43——10.145113be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 256 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.08,343——20.09477.613be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 512 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.08,391.94——40.63492.313be7f67e ↗
Speech recognition · Audio → Text · asr_prompts · Test duration 16 s · Batch 1024 · audio-sec/sec · 1 rows
ModelHardwareFormatEngineMean throughput (audio-sec/sec)P95TTFTObserved occupancy (GiB)GPU power (W)Reports combinedView report
granite-speech-5.0-470m-turboctcRTX PRO 6000 Blackwell Server Edition · 96 GBNativetransformers-ctcPyTorch 2.13.08,387.34——80.89501.313be7f67e ↗