Run an AI model on your computer
Get a startup command, then explore measured performance for your task and GPU.
Get your startup command
Choose your operating system and enter a model repository. Launch support is checked separately from performance results.
Embedding · RTX 3080 Ti · 12 GB
0 models with qualified results. Speeds compare only within one exact test; use “Same test on other GPUs” to compare hardware.
No qualified results yet · Embedding · RTX 3080 Ti · 12 GB
No reports for this task have been recorded here. This does not show that the model or GPU cannot run it.
Qualified results for other tasks: Speech synthesis (1) · Speech recognition (2)
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Measured hardware
- L4 · 24 GB
- RTX 4090 · 24 GB
- RTX PRO 6000 Blackwell Server Edition · 96 GB
- T4 · 16 GB
- V100-PCIE-32GB · 32 GB
- vGPU-32GB · 32 GB
15 more GPUs without qualified results for this task
- 2 × RTX 5090 · 32 GB
- 7 × RTX 3080 · 10 GB
- A100-SXM4-40GB · 40 GB
- A100-SXM4-80GB · 80 GB
- Apple M2 Max GPU (38-core) · 48 GB
- B200 · 179 GB
- H100 80GB HBM3 · 80 GB
- RTX 3080 · 10 GB
- RTX 3080 Ti · 12 GB
- RTX 3090 · 24 GB
- RTX 4080 SUPER · 32 GB
- RTX 4090D · 48 GB
- RTX 5090 · 32 GB
- RTX PRO 6000 Blackwell Workstation Edition · 96 GB
- Unknown · ? GB
Running benchmarks
0Updates automaticallyNo benchmarks are currently reported as running.
RTX PRO 6000 Blackwell Server Edition · 96 GB · 403 queued · 0 running · 0 awaiting reassignment · 6 succeeded · 3 failed or exhausted
L4 · 24 GB · 91 queued · 0 running · 0 awaiting reassignment · 22 succeeded · 7 failed or exhausted
T4 · 16 GB · 22 queued · 0 running · 0 awaiting reassignment · 3 succeeded · 8 failed or exhausted