48 GB · 445 reports

RTX 4090D · 48 GB

Qualified results first; original reports and full tables stay below.

Text generation · Qualified results

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

No qualified results yet · Text generation · RTX 4090D · 48 GB

445 original reports are kept as evidence but do not qualify for comparison; each report states why. This does not show that the model or GPU cannot run it.

Check model launch support View original reports Measured alternatives for this task Test your own GPU

Measured alternatives on other GPUs

Measured on different GPUs with the same or smaller listed capacity. These are references for choosing hardware, not performance or compatibility evidence for your selected GPU.

Qwen3.8-27B-NVFP4
RadixArk · 2 tested configurations · 1 GPU
- RTX 5090 · 32 GBNVFP4 · 8K → 2K tokens
  Output throughput · includes prefill · 1 request
  70.8tok/s
  Observed GPU occupancy
  27.3GiB
  First token
  633 ms
  Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=2
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 70.82 29.01 s 633 ms 27.29 405.5 1 bad294c0 ↗
- 1 other test configurations on this GPU
  - NVFP4 · 32K → 2K tokens
    Output throughput · includes prefill · 1 request
    62.5tok/s
    Observed GPU occupancy
    27.3GiB
    First token
    3.6 s
    Use this recipe · 1 request ↗Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=2
    Workload
    32768in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 62.48 32.79 s 3.6 s 27.29 433.3 1 746bbc39 ↗
Qwen3-4B-AWQ
Qwen · 3 tested configurations · 2 GPUs
- L4 · 24 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  55.3tok/s
  Best measured · 4 concurrent requests
  109tok/s
  Observed GPU occupancy
  19.5GiB
  First token
  1.6 s
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 55.3 37.17 s 1.6 s 19.54 71.9 1 9301a9ba ↗
  C4 109.43 78.88 s 3.1 s 19.55 71.9 1 9301a9ba ↗
- 1 other test configurations on this GPU
  - AWQ · 8K → 2K tokens · Prefix cache reuse
    Output throughput · includes prefill · 1 request
    57.4tok/s
    Best measured · 4 concurrent requests
    175tok/s
    Observed GPU occupancy
    19.5GiB
    First token
    60 ms
    Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 concurrency levels
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm · vLLM 0.29.0
    Protocol
    wbench-fixed-http-v3
    Dataset
    migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 57.39 35.7 s 60 ms 19.55 71.8 1 7e2cbfda ↗
    C4 175.39 46.75 s 114 ms 19.55 71.9 1 7e2cbfda ↗
Show 1 more GPUs
- T4 · 16 GBAWQ · 8K → 2K tokens · No cache reuse
  Output throughput · includes prefill · 1 request
  15.4tok/s
  Best measured · 4 concurrent requests
  20.3tok/s
  Observed GPU occupancy
  12.7GiB
  First token
  28.5 s
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 15.39 133.36 s 28.5 s 12.74 67.9 1 fcfe2d64 ↗
  C4 20.31 468.13 s 53.1 s 12.74 67.5 1 fcfe2d64 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	70.82	29.01 s	633 ms	27.29	405.5	1	bad294c0 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	62.48	32.79 s	3.6 s	27.29	433.3	1	746bbc39 ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	55.3	37.17 s	1.6 s	19.54	71.9	1	9301a9ba ↗
C4	109.43	78.88 s	3.1 s	19.55	71.9	1	9301a9ba ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.39	35.7 s	60 ms	19.55	71.8	1	7e2cbfda ↗
C4	175.39	46.75 s	114 ms	19.55	71.9	1	7e2cbfda ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	15.39	133.36 s	28.5 s	12.74	67.9	1	fcfe2d64 ↗
C4	20.31	468.13 s	53.1 s	12.74	67.5	1	fcfe2d64 ↗

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Quality by dataset

Scores from different datasets and sample scopes are not directly comparable.

Quality by dataset · 7 rows
Model	Dataset	Format	Engine	Average score	Sample size	Runs
RadixArk/Qwen3.8-27B-NVFP4	hle-test-self-judge-22ed307-v1-n5-mt16384-t0	NVFP4	vllm	20%	5 / 2500 · Subset only	1 runs 20% · 7bbadab5 ↗
RadixArk/Qwen3.8-27B-NVFP4	simpleqa-verified-eval-self-judge-0dc97e0-v1-n1000-mt16384-t0	NVFP4	vllm	6.4%	1000 / 1000 · Full dataset	1 runs 6.4% · 404eb015 ↗
RadixArk/Qwen3.8-27B-NVFP4	simpleqa-verified-eval-self-judge-0dc97e0-v1-n5-mt16384-t0	NVFP4	vllm	0%	5 / 1000 · Subset only	1 runs 0% · 328fd742 ↗
RadixArk/Qwen3.8-27B-NVFP4	ifbench-test-official-loose-1c40f0c-v1-n300-mt16384-t0	NVFP4	vllm	67.67%	300 / 300 · Full dataset	1 runs 67.67% · 02a43d63 ↗
RadixArk/Qwen3.8-27B-NVFP4	ifbench-test-official-loose-1c40f0c-v1-n3-mt16384-t0	NVFP4	vllm	66.67%	3 / 300 · Subset only	1 runs 66.67% · 20ef4b9f ↗
RadixArk/Qwen3.8-27B-NVFP4	aime2026-test-zero-shot-strict-v4-n30-mt16384-t0	NVFP4	vllm	63.33%	30 / 30 · Full dataset	1 runs 63.33% · 87acbd23 ↗
RadixArk/Qwen3.8-27B-NVFP4	mmlu-pro-test-zero-shot-strict-v4-n3-mt16384-t0	NVFP4	vllm	66.67%	3 / 12032 · Subset only	1 runs 66.67% · 1d2bdd31 ↗

Measurements without a comparable protocol · 917 rows

These measured rates remain visible, but are excluded from comparison curves because the protocol is unknown.

Measurements without a comparable protocol · 917 rows
Model	Format	Engine	Workload	Concurrency	Throughput	GPU power (W)
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	1	16.09 tok/s	251
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	1	16.17 tok/s	262.49
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	1	16.2 tok/s	259.91
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	1	16.2 tok/s	258.68
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	1	16.21 tok/s	258.88
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	1	16.24 tok/s	261.07
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	1	16.27 tok/s	258.29
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	1	22.47 tok/s	335.22
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	1	22.51 tok/s	329.21
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	1	32.77 tok/s	238.03
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	1	32.8 tok/s	247.48
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	1	32.87 tok/s	246.53
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	1	32.94 tok/s	244.41
Gemma-3-270m ↗	BF16	vLLM	512	1	37.2 tok/s	269.12
Qwen3-32B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	38.27 tok/s	—
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	44.09 tok/s	—
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	44.09 tok/s	—
Qwen3-32B-AWQ ↗	AWQ	vLLM	2K	1	46.34 tok/s	323.63
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	1	51 tok/s	227.48
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	1	51.27 tok/s	225.16
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	1	51.42 tok/s	229.09
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	1	51.51 tok/s	219.25
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	1	51.58 tok/s	228.74
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	1	51.65 tok/s	226.69
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	1	59.01 tok/s	—
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	1	59.14 tok/s	—
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	1	59.16 tok/s	—
Qwen3-32B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	68.31 tok/s	—
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	1	73.01 tok/s	193.84
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	1	73.36 tok/s	193.18
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	73.4 tok/s	194.95
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	1	73.66 tok/s	198.29
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	1	73.83 tok/s	196.81
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	1	74.1 tok/s	197.73
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	74.14 tok/s	200.2
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	74.15 tok/s	196.81
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	74.22 tok/s	198.86
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	1	74.28 tok/s	199.45
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	1	74.29 tok/s	199.01
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	1	74.3 tok/s	192.1
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	77.87 tok/s	—
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	77.92 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	83.57 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	83.65 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	83.71 tok/s	—
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	1	85.71 tok/s	262.3
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	1	85.79 tok/s	263.43
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	1	85.9 tok/s	257.47
Qwen3-14B-AWQ ↗	AWQ	vLLM	2K	1	91.53 tok/s	305.5
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	95.08 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	95.08 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	95.17 tok/s	—
Qwen3-4B ↗	BF16	vLLM	2K	1	99.14 tok/s	302.52
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	1	102.83 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	1	102.84 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	1	102.85 tok/s	—
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	1	103.36 tok/s	184.14
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	1	103.88 tok/s	188.49
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	1	105.18 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	1	105.42 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	1	105.45 tok/s	—
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	1	105.51 tok/s	189.72
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	2	106.01 tok/s	—
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	1	106.03 tok/s	185.4
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	1	106.2 tok/s	192.05
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	1	106.2 tok/s	191.4
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	1	106.34 tok/s	191.64
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	1	106.38 tok/s	193.41
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	2	106.56 tok/s	—
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	1	106.67 tok/s	191.68
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	2	106.73 tok/s	—
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	8	110.42 tok/s	283.05
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	8	113.89 tok/s	284.17
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	8	116.06 tok/s	286.72
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	8	121.1 tok/s	280.53
Qwen3-32B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	125.49 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	1	130.15 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	1	130.28 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	1	130.45 tok/s	—
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	8	133.11 tok/s	279.95
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	1	134.43 tok/s	176.42
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	1	135.49 tok/s	180.04
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	1	135.58 tok/s	179.28
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	1	136.31 tok/s	178.28
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	1	136.82 tok/s	173.65
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	8	136.83 tok/s	283.86
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	1	136.85 tok/s	177.93
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	8	136.95 tok/s	280.71
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	1	136.96 tok/s	177.68
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	137.74 tok/s	—
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	138.01 tok/s	—
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	1	138.43 tok/s	184.6
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	1	138.45 tok/s	182.55
Qwen3-8B-AWQ ↗	AWQ	vLLM	2K	1	139.77 tok/s	282.76
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	141.38 tok/s	—
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	141.51 tok/s	—
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	141.66 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	143.79 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	144.16 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	144.19 tok/s	—
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	8	156.01 tok/s	329.45
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	158.85 tok/s	—
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	159.16 tok/s	—
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	159.55 tok/s	—
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	8	159.82 tok/s	331.5
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	161.17 tok/s	—
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	2	161.7 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	162 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	162.04 tok/s	—
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	2	162.73 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	164.97 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	165.13 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	165.14 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	1	170.23 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	1	170.27 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	1	170.39 tok/s	—
Qwen3-32B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	170.59 tok/s	—
Qwen3-32B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	174.51 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	2	175.17 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	2	175.36 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	2	175.67 tok/s	—
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	175.89 tok/s	—
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	176.07 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	2	179.71 tok/s	—
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	179.94 tok/s	—
Qwen3-32B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	180.01 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	2	180.67 tok/s	—
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	16	181.55 tok/s	279.15
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	1	181.77 tok/s	—
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	1	182.28 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	2	182.38 tok/s	—
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	16	182.58 tok/s	277.89
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	16	183.12 tok/s	284.63
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	183.24 tok/s	—
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	186.16 tok/s	—
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	16	186.28 tok/s	281.54
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	186.29 tok/s	—
Qwen3-1.7B ↗	BF16	vLLM	2K	1	196.71 tok/s	278.09
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	16	198.1 tok/s	278.39
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	4	200.4 tok/s	—
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	4	202.4 tok/s	—
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	4	202.73 tok/s	—
Qwen3-4B-AWQ ↗	AWQ	vLLM	2K	1	209.72 tok/s	253.3
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	16	213.03 tok/s	277.63
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	2	213.5 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	2	216.5 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	2	219.72 tok/s	—
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	8	223.92 tok/s	261.52
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	224.67 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	224.88 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	1	225.21 tok/s	—
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	227.39 tok/s	—
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	228.11 tok/s	—
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1	228.61 tok/s	145.63
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	228.82 tok/s	—
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1	229.63 tok/s	146.12
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	8	229.88 tok/s	267.57
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1	230.12 tok/s	143.62
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1	230.43 tok/s	140.25
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1	231.11 tok/s	144.22
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	239.02 tok/s	—
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	8	239.71 tok/s	265.89
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	240.92 tok/s	—
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	241.13 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	245.88 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	246.18 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	246.24 tok/s	—
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	8	248.37 tok/s	264.54
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	1	249.24 tok/s	211.56
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	1	249.26 tok/s	209.3
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	1	249.58 tok/s	208.11
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	250.76 tok/s	—
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	252.85 tok/s	—
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	252.98 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	259.11 tok/s	—
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	4	259.49 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	16K	1	260.85 tok/s	152.74
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	4	261.63 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	261.78 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	262 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	262.31 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	262.95 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	265.72 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	128	1	267.49 tok/s	146.51
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	32	267.6 tok/s	381.6
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	2	268.97 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	2	269.12 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	2	269.59 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	283.27 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	283.97 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	284.26 tok/s	—
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	16	285.03 tok/s	353.6
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	16	290.43 tok/s	356.86
Qwen3-32B-AWQ ↗	AWQ	vLLM	2K	8	292.18 tok/s	315.33
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	297.97 tok/s	—
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	302.62 tok/s	—
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	303.71 tok/s	—
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	1	313.55 tok/s	243.16
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	1	317.91 tok/s	—
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	1	318.22 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	319.52 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	319.79 tok/s	—
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	1	320.03 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	2	320.07 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	1	326.17 tok/s	240.98
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	4	326.41 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	Mixed	1	327.11 tok/s	173.29
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	4	328.42 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	4	328.96 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	4	329.71 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	Mixed	8	331.25 tok/s	228.1
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	1	331.88 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	1	332.24 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	1	332.49 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	4	335.84 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	4	339.75 tok/s	—
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	8	348.1 tok/s	—
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	8	350.9 tok/s	—
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	8	351.33 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	351.48 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	351.95 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	351.96 tok/s	—
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	16	352.72 tok/s	—
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	16	354.67 tok/s	—
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	8	355.07 tok/s	245.27
Qwen3-14B-Q8_0 ↗	Q8_0	vLLM	1K	16	355.1 tok/s	—
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	8	355.42 tok/s	241.84
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	8	356.8 tok/s	246.18
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	8	358.75 tok/s	249.5
Gemma-3-270m ↗	BF16	vLLM	Mixed	2	358.96 tok/s	184.1
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	361.12 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	361.66 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	362.14 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	1	364.04 tok/s	279.69
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	365.02 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	365.53 tok/s	—
Qwen3-14B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	365.54 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	1	367.11 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	1	367.88 tok/s	—
Qwen3-0.6B ↗	BF16	vLLM	2K	1	368.05 tok/s	220.2
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	1	368.57 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	4	369.93 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	370.11 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	370.35 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	370.9 tok/s	—
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	32	371.07 tok/s	284.36
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	32	372.39 tok/s	280.54
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	16	373.01 tok/s	262.2
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	373.91 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	373.95 tok/s	—
Qwen3-14B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	374.13 tok/s	—
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	32	378.01 tok/s	284.42
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	4	383.63 tok/s	—
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	8	387.07 tok/s	248.16
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	4	389.11 tok/s	—
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	16	393.18 tok/s	—
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	16	393.72 tok/s	259.58
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	8	394.28 tok/s	242.01
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	8	394.57 tok/s	247.15
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	8	396.76 tok/s	—
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	8	399.3 tok/s	249.16
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	8	399.46 tok/s	245.39
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	8	401.84 tok/s	249.09
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	8	402.53 tok/s	—
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	406.49 tok/s	—
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	32	406.64 tok/s	282.73
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	8	406.92 tok/s	245.31
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	16	407.24 tok/s	263.18
Qwen3-30B-A3B-Q8_0 ↗	Q8_0	vLLM	1K	16	408.28 tok/s	—
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	411.62 tok/s	—
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	411.71 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	418.02 tok/s	—
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	16	418.77 tok/s	260.19
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	420.66 tok/s	246.98
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	423.84 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	424.64 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	424.7 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	426.5 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	Mixed	4	431.84 tok/s	235.46
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	435.1 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	2K	1	436.43 tok/s	170.53
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	438.09 tok/s	—
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	32	438.17 tok/s	278.61
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	438.26 tok/s	—
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	2	439.37 tok/s	206.75
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	64	443.99 tok/s	286.02
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	2	445.88 tok/s	—
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	2	447.2 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	4K	1	447.36 tok/s	174.14
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	2	447.85 tok/s	—
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	8	450.16 tok/s	252.08
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	452.95 tok/s	—
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	8	453.56 tok/s	251.87
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	8	454.5 tok/s	249.55
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	456.01 tok/s	—
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	456.96 tok/s	—
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	8	458.38 tok/s	253.52
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	32	459.21 tok/s	277.09
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	462.24 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	2	464.24 tok/s	208.5
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	466.04 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	2	466.26 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	2	466.5 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	2	466.66 tok/s	—
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	64	468.56 tok/s	290.4
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	469.7 tok/s	—
Qwen3-30B-A3B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	470.99 tok/s	—
Qwen3-32B-AWQ ↗	AWQ	vLLM	2K	16	475.16 tok/s	332.09
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	4	486.39 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	8	487.05 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	4	487.93 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	4	488.85 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	2	489.7 tok/s	229.33
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	32	492.31 tok/s	391.28
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	2	494.33 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	2	494.42 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	2	495.99 tok/s	—
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	8	498.46 tok/s	207.7
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	16	499.3 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	16	510.31 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	16	519.12 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	526.72 tok/s	211.09
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	8	531.12 tok/s	212.55
Gemma-3-270m ↗	BF16	vLLM	32K	1	532.47 tok/s	189.12
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	532.7 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	533.33 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	534.4 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	536.37 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	32K	1	536.84 tok/s	190.17
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	16	539.62 tok/s	—
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	8	541.59 tok/s	210.75
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	8	542.49 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	8	543.25 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	544.52 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	16K	1	546.4 tok/s	186.78
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	16	549.67 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	8	550.14 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	8	550.2 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	550.33 tok/s	—
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	16	550.36 tok/s	259.99
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	552.39 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	552.4 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	4	552.43 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	8	553.33 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	8K	1	553.45 tok/s	184.86
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	16	553.91 tok/s	—
Qwen3-8B-Q8_0 ↗	Q8_0	vLLM	1K	16	554.33 tok/s	—
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	555.43 tok/s	—
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	16	555.94 tok/s	263.88
Gemma-3-270m ↗	BF16	vLLM	Mixed	1	558.43 tok/s	185.07
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	560.16 tok/s	—
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	8	560.55 tok/s	210.36
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	560.68 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	2K	1	562.52 tok/s	185.58
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	562.98 tok/s	—
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	564.72 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	1	565.03 tok/s	186.89
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	1	565.23 tok/s	190.9
Gemma-3-270m ↗	BF16	vLLM	1K	1	565.84 tok/s	184.72
Gemma-3-270m ↗	BF16	vLLM	4K	1	566.15 tok/s	185.97
Gemma-3-270m ↗	BF16	vLLM	2K	1	566.41 tok/s	184.31
Qwen3-8B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	566.83 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	1K	1	566.96 tok/s	184.81
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	8	567.5 tok/s	210.54
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	64	568.3 tok/s	385.25
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	569.16 tok/s	—
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	8	569.24 tok/s	214.92
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	1	569.79 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	1	569.89 tok/s	—
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	16	570.76 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	8	570.85 tok/s	—
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	64	571.37 tok/s	291.69
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	1	571.89 tok/s	—
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	8	572.12 tok/s	215.36
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	573.83 tok/s	—
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	574.69 tok/s	—
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	575.12 tok/s	—
Qwen3-14B-AWQ ↗	AWQ	vLLM	2K	8	576.77 tok/s	296.35
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	577.53 tok/s	—
Qwen3-8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	578.08 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	578.74 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	578.86 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	578.91 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	580.31 tok/s	—
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	8	580.67 tok/s	—
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	581.37 tok/s	—
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	8	582.69 tok/s	254.22
HY-MT1.5-7B-Q8_0 ↗	Q8_0	vLLM	1K	8	582.7 tok/s	—
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	64	583.13 tok/s	382.87
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	16	583.48 tok/s	257.81
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	256	591.39 tok/s	386.76
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	8	591.65 tok/s	257.54
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	8	591.72 tok/s	258.46
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	256	594.19 tok/s	392.55
HY-MT1.5-7B-Q6_K ↗	Q6_K	vLLM	1K	16	595.63 tok/s	—
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	512	597.36 tok/s	397.72
Gemma-3-270m ↗	BF16	vLLM	512	1	598.27 tok/s	189.67
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	16	599.75 tok/s	265.94
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	599.9 tok/s	216.27
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	16	607.23 tok/s	266.07
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	607.43 tok/s	211.72
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	607.45 tok/s	212.95
HY-MT1.5-7B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	607.92 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	512	1	608.88 tok/s	191.83
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	612.17 tok/s	216.25
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	8	612.74 tok/s	215.25
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	16	623.26 tok/s	265.26
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	16	627.02 tok/s	266.67
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	16	634.03 tok/s	259.33
Gemma-3-270m ↗	BF16	vLLM	256	1	640.38 tok/s	195.23
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	16	640.69 tok/s	240.22
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	128	642.1 tok/s	393.7
Qwen2.5-72B-AWQ ↗	AWQ	vLLM	2K	128	646.86 tok/s	398.54
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	16	647.25 tok/s	267.19
Gemma-3-270m ↗	BF16	vLLM	128	1	648.15 tok/s	196.02
Qwen3-4B ↗	BF16	vLLM	2K	8	654.49 tok/s	269.69
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	2	665.32 tok/s	155.08
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	2	665.58 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	2	665.75 tok/s	150.73
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	16	666.4 tok/s	244.49
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	2	667.55 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	2	667.91 tok/s	—
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	64	676.99 tok/s	287.63
Gemma-3-270m ↗	BF16	vLLM	128	1	681.68 tok/s	200.79
Gemma-3-270m ↗	BF16	vLLM	128	1	682.05 tok/s	200.71
Qwen3-32B-FP8 ↗	FP8	vLLM	2K	64	687.14 tok/s	289.26
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	16	687.22 tok/s	247.51
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	16	687.4 tok/s	243.73
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	64	689.51 tok/s	292.41
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	128	690.52 tok/s	331.63
Gemma-3-270m ↗	BF16	vLLM	128	1	692.79 tok/s	202.09
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	16	706.33 tok/s	241.71
Qwen3-32B-AWQ ↗	AWQ	vLLM	2K	32	706.34 tok/s	365.23
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	32	708.79 tok/s	266.13
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	32	718 tok/s	264.95
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	32	726.89 tok/s	267.84
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	32	748.12 tok/s	261.87
Qwen2.5-0.5B-Q4_K_M ↗	Q4_K_M	vLLM	1K	1	749.41 tok/s	—
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	16	760.97 tok/s	245.93
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	8	762.92 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	8	763.08 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	8	764.16 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	16	764.89 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	765.63 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	16	766.16 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	767.09 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	8	767.56 tok/s	—
Qwen3-4B-Q8_0 ↗	Q8_0	vLLM	1K	16	767.6 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	767.67 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	769.14 tok/s	—
Qwen3-4B-Q5_K_M ↗	Q5_K_M	vLLM	1K	16	771.32 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	776.25 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	777.88 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	778.01 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	778.28 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	778.97 tok/s	—
Qwen3-4B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	780.32 tok/s	—
Qwen2.5-0.5B-Q4_K_M ↗	Q4_K_M	vLLM	1K	2	794.85 tok/s	—
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	256	801.87 tok/s	346.85
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	4	803.48 tok/s	218.36
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	4	803.71 tok/s	216.61
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	4	810.27 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	4	810.48 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	4	811.46 tok/s	—
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	4	813.28 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	4	813.68 tok/s	—
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	4	815.93 tok/s	—
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	16	825.69 tok/s	212.27
Gemma-3-270m ↗	BF16	vLLM	32K	2	827.48 tok/s	173.12
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	32	838.91 tok/s	261.45
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	128	847.58 tok/s	331.13
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	16	847.59 tok/s	210.24
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	4	855.11 tok/s	—
Gemma-3-270m ↗	BF16	vLLM	16K	2	856.9 tok/s	169.84
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	32	860.65 tok/s	261.52
Gemma-3-270m ↗	BF16	vLLM	Mixed	2	861.15 tok/s	164.61
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	4	861.42 tok/s	—
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	16	865.73 tok/s	209.24
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	16	866.94 tok/s	252.35
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	4	871 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	4	872.37 tok/s	243.84
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	16	874.73 tok/s	201.65
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	32	882 tok/s	262.59
Qwen3-8B-AWQ ↗	AWQ	vLLM	2K	8	883.62 tok/s	273.55
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	256	889.78 tok/s	353.01
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	899.87 tok/s	—
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	16	901.53 tok/s	206.79
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	901.77 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	902.61 tok/s	—
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	16	903.49 tok/s	212.2
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	903.58 tok/s	226.18
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	16	918.65 tok/s	206.99
Qwen2.5-0.5B-Q4_K_M ↗	Q4_K_M	vLLM	1K	4	922.16 tok/s	—
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	8	926.86 tok/s	193.14
Qwen3-14B-AWQ ↗	AWQ	vLLM	2K	16	931.96 tok/s	301.54
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	16	933.97 tok/s	208.24
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	16	935.67 tok/s	212.32
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	32	942.71 tok/s	262.69
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	8	954.81 tok/s	193.76
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	8	967.34 tok/s	196.55
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	16	968.24 tok/s	252.7
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	32	968.6 tok/s	265.2
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	16	968.67 tok/s	205.21
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	16	972.07 tok/s	247.73
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	32	975.24 tok/s	268.89
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	8	975.63 tok/s	194.74
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	32	980.52 tok/s	268.33
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	32	994.12 tok/s	268.01
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	32	996.01 tok/s	267.35
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	16	1,006.86 tok/s	207.2
Qwen3-32B-AWQ ↗	AWQ	vLLM	2K	64	1,028 tok/s	369.03
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	8	1,050.25 tok/s	196.18
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	8	1,051.26 tok/s	196.45
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	8	1,051.67 tok/s	199.69
Qwen2.5-0.5B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	1,055 tok/s	—
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	8	1,058.96 tok/s	201.86
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	8	1,060.97 tok/s	201.32
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	16	1,063.11 tok/s	212.6
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	4	1,099.69 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	4	1,100.18 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	4	1,103.61 tok/s	160.07
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	4	1,104.22 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	4	1,105.47 tok/s	157.62
Qwen3-4B ↗	BF16	vLLM	2K	16	1,107.11 tok/s	263.53
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	8	1,138.09 tok/s	—
Qwen3-32B-AWQ ↗	AWQ	vLLM	2K	128	1,160.53 tok/s	369.83
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	32	1,162.31 tok/s	248.22
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,179.37 tok/s	—
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,180.43 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,183.28 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,185.95 tok/s	—
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,186.06 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,186.78 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,186.92 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,187.4 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,187.61 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	1,192.23 tok/s	—
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,193.6 tok/s	233.56
Qwen3-1.7B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,194.11 tok/s	237.08
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	1,201.19 tok/s	—
Qwen3-1.7B ↗	BF16	vLLM	2K	8	1,202.43 tok/s	250.26
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	1,205.36 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	1,205.87 tok/s	249.31
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	8	1,206.15 tok/s	258.49
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,213.14 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	8	1,215.96 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	1,216.08 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	1,218.67 tok/s	—
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	8	1,219.05 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	8	1,219.77 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	16	1,220.29 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	16	1,220.63 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	16	1,221.21 tok/s	—
HY-MT1.5-1.8B-Q6_K ↗	Q6_K	vLLM	1K	16	1,221.48 tok/s	256.97
HY-MT1.5-1.8B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	1,221.5 tok/s	249.53
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	64	1,221.79 tok/s	275.3
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,229.45 tok/s	—
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	32	1,229.75 tok/s	251.18
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,236.89 tok/s	238.01
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,240.99 tok/s	241.67
HY-MT1.5-1.8B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,241.06 tok/s	—
Qwen3-VL-32B-FP8 ↗	FP8	vLLM	2K	512	1,254.31 tok/s	349.3
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	64	1,270.87 tok/s	275.84
Qwen3-4B-AWQ ↗	AWQ	vLLM	2K	8	1,272.44 tok/s	253.07
Qwen3-8B-AWQ ↗	AWQ	vLLM	2K	16	1,273.62 tok/s	269.7
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	64	1,294.65 tok/s	274.85
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	32	1,301.63 tok/s	252.53
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	64	1,302.67 tok/s	272.62
Gemma-3-270m ↗	BF16	vLLM	512	8	1,339.85 tok/s	185.83
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	64	1,342.16 tok/s	275.16
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	64	1,352.57 tok/s	277.38
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	64	1,355.61 tok/s	274.82
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	32	1,361.86 tok/s	251.1
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	64	1,362.38 tok/s	273.38
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	64	1,372.82 tok/s	276.92
Qwen2.5-0.5B-Q4_K_M ↗	Q4_K_M	vLLM	1K	16	1,378.74 tok/s	—
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	64	1,389.92 tok/s	273.04
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	64	1,403.32 tok/s	275.51
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	32	1,426.13 tok/s	250.87
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	32	1,436.15 tok/s	251.81
Gemma-3-270m ↗	BF16	vLLM	2K	8	1,445.3 tok/s	164.85
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,456.73 tok/s	172.69
Gemma-3-270m ↗	BF16	vLLM	32K	8	1,458.27 tok/s	157.46
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,470.9 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,471.65 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,477.03 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,478.49 tok/s	170.53
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,478.81 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,481.7 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,483.75 tok/s	—
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	16	1,490.8 tok/s	170.77
Qwen3-0.6B-Q8_0 ↗	Q8_0	vLLM	1K	8	1,490.92 tok/s	169.76
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	64	1,516.6 tok/s	275.01
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	32	1,519.97 tok/s	214.53
Gemma-3-270m ↗	BF16	vLLM	32K	4	1,522.33 tok/s	192.63
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	8	1,527.27 tok/s	211.02
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	8	1,534.73 tok/s	214.07
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	1,535.83 tok/s	223.71
Qwen3-14B-AWQ ↗	AWQ	vLLM	2K	32	1,539.11 tok/s	339.88
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	8	1,547.49 tok/s	212.55
Gemma-3-270m ↗	BF16	vLLM	4K	8	1,552.03 tok/s	149.96
Gemma-3-270m ↗	BF16	vLLM	16K	8	1,552.35 tok/s	149.86
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	1,556.33 tok/s	223.4
Gemma-3-270m ↗	BF16	vLLM	Mixed	4	1,559.22 tok/s	178.04
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	16	1,562.54 tok/s	189.88
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	1,574.68 tok/s	221.34
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	16	1,575.67 tok/s	186.03
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	8	1,580.25 tok/s	171.1
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	32	1,590.69 tok/s	221.52
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	8	1,590.87 tok/s	170.77
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	8	1,591.62 tok/s	171.52
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	8	1,592.69 tok/s	171.19
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	64	1,596.36 tok/s	273.4
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	8	1,598.01 tok/s	174.09
Gemma-3-270m ↗	BF16	vLLM	1K	8	1,599.27 tok/s	149.04
Gemma-3-270m ↗	BF16	vLLM	4K	8	1,602.47 tok/s	142.08
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	32	1,607.47 tok/s	218.44
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	16	1,611.31 tok/s	188.05
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	16	1,612.34 tok/s	190.12
Qwen3-4B ↗	BF16	vLLM	2K	32	1,618.13 tok/s	281.54
Gemma-3-270m ↗	BF16	vLLM	16K	4	1,630.3 tok/s	187.19
Gemma-3-270m ↗	BF16	vLLM	256	8	1,632.06 tok/s	141.7
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	32	1,638.15 tok/s	212.69
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	32	1,645.04 tok/s	218.77
Gemma-3-270m ↗	BF16	vLLM	2K	8	1,649.66 tok/s	141.18
Gemma-3-270m ↗	BF16	vLLM	8K	8	1,651.81 tok/s	146.62
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	32	1,665.06 tok/s	223.52
Gemma-3-270m ↗	BF16	vLLM	1K	8	1,665.74 tok/s	141.35
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	32	1,678.88 tok/s	216.37
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	32	1,682.08 tok/s	266.2
Gemma-3-270m ↗	BF16	vLLM	1K	8	1,687.01 tok/s	140.83
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	32	1,687.45 tok/s	264.22
Gemma-3-270m ↗	BF16	vLLM	512	8	1,695.08 tok/s	138.95
Gemma-3-270m ↗	BF16	vLLM	128	8	1,699.55 tok/s	158.78
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	32	1,709.98 tok/s	266.64
Gemma-3-270m ↗	BF16	vLLM	512	8	1,713.17 tok/s	140.12
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	16	1,738.28 tok/s	191.06
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	16	1,738.4 tok/s	189.7
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	16	1,739.89 tok/s	190.26
Gemma-3-270m ↗	BF16	vLLM	256	8	1,747.17 tok/s	143.73
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	16	1,750.12 tok/s	196.14
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	16	1,752.59 tok/s	192.7
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	1,828.23 tok/s	219.23
Gemma-3-270m ↗	BF16	vLLM	128	8	1,846.76 tok/s	138.58
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	32	1,847.98 tok/s	224.12
Qwen3-4B-AWQ ↗	AWQ	vLLM	2K	16	1,903.17 tok/s	244.57
Gemma-3-270m ↗	BF16	vLLM	256	8	1,917.24 tok/s	142.39
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	128	1,925.13 tok/s	325.6
Qwen3-0.6B ↗	BF16	vLLM	2K	8	1,925.32 tok/s	222.84
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	128	1,927.99 tok/s	323.35
Qwen3-1.7B ↗	BF16	vLLM	2K	16	1,931.19 tok/s	236.68
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	128	1,937.23 tok/s	325.71
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	64	1,963.24 tok/s	262.14
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	64	1,975.17 tok/s	262.92
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	128	1,990.19 tok/s	325.32
Qwen3-14B-AWQ ↗	AWQ	vLLM	2K	64	2,074.24 tok/s	347.03
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	128	2,107.95 tok/s	290.21
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	128	2,108.81 tok/s	289.3
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	128	2,129.26 tok/s	286.14
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	128	2,156.26 tok/s	288.11
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	128	2,166.93 tok/s	285.3
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	128	2,198.82 tok/s	284.83
Gemma-3-270m ↗	BF16	vLLM	128	8	2,225.08 tok/s	146.97
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	64	2,268.81 tok/s	267.2
Qwen3-14B-AWQ ↗	AWQ	vLLM	2K	128	2,286.57 tok/s	344.42
Qwen3-8B-AWQ ↗	AWQ	vLLM	2K	32	2,299.42 tok/s	303.56
Gemma-3-270m ↗	BF16	vLLM	128	8	2,330.32 tok/s	153.08
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	64	2,330.53 tok/s	265.76
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	256	2,334.15 tok/s	296.53
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	64	2,351.02 tok/s	269.89
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	64	2,369.35 tok/s	267.96
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	128	2,461.2 tok/s	287.36
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	128	2,472.09 tok/s	285.22
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	64	2,477.56 tok/s	233.58
Gemma-3-270m ↗	BF16	vLLM	Mixed	8	2,479.12 tok/s	161.51
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	256	2,484.72 tok/s	343.34
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	16	2,485.11 tok/s	162.18
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	16	2,485.19 tok/s	163.23
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	16	2,490.38 tok/s	161.21
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	16	2,491.82 tok/s	161.07
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	16	2,500.48 tok/s	163.99
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	256	2,501.77 tok/s	337.38
Qwen3-14B-AWQ ↗	AWQ	vLLM	2K	256	2,503.73 tok/s	353.52
Qwen3-14B-FP8 ↗	FP8	vLLM	2K	256	2,516.02 tok/s	336.15
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	64	2,534.64 tok/s	232.52
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	64	2,554.3 tok/s	233.73
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	128	2,554.99 tok/s	305.69
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	64	2,562.45 tok/s	240.53
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	64	2,566.37 tok/s	238.09
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	32	2,590.63 tok/s	204.2
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	128	2,596.37 tok/s	302.9
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	128	2,599.07 tok/s	305.71
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	128	2,611.51 tok/s	304.15
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	32	2,623.08 tok/s	207.92
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	32	2,628.95 tok/s	206.37
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	32	2,636.18 tok/s	205.06
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	64	2,636.56 tok/s	242.16
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	64	2,653.11 tok/s	239.51
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	64	2,654.38 tok/s	293.61
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	64	2,665.72 tok/s	241.49
Gemma-3-270m ↗	BF16	vLLM	128	8	2,667.02 tok/s	161.76
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	64	2,668.28 tok/s	295.58
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	64	2,669.39 tok/s	242.11
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	16	2,671.61 tok/s	226.59
Qwen3-0.6B ↗	BF16	vLLM	2K	16	2,677.74 tok/s	201.02
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	32	2,679.18 tok/s	209.15
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	64	2,690.33 tok/s	238.98
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	64	2,690.73 tok/s	292.87
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	16	2,695.85 tok/s	231.18
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	16	2,706.58 tok/s	228.46
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	128	2,768.2 tok/s	302.62
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	128	2,800.56 tok/s	301.17
Gemma-3-270m ↗	BF16	vLLM	512	32	2,851.12 tok/s	205.79
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	256	2,851.94 tok/s	285.61
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	256	2,879.43 tok/s	288.82
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	64	2,885.55 tok/s	240.16
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	64	2,888.89 tok/s	239.39
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	32	2,896.36 tok/s	214.16
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	32	2,899.87 tok/s	209.93
Qwen3-VL-30B-A3B-FP8 ↗	FP8	vLLM	2K	512	2,901.88 tok/s	286.26
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	256	2,903.16 tok/s	294.06
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	32	2,903.83 tok/s	212.69
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	32	2,903.95 tok/s	213.06
Qwen3-4B-AWQ ↗	AWQ	vLLM	2K	32	2,922.56 tok/s	272.3
Qwen3-4B ↗	BF16	vLLM	2K	64	2,959.2 tok/s	298.78
Qwen3-8B-AWQ ↗	AWQ	vLLM	2K	64	2,999.88 tok/s	316.37
Qwen3-30B-A3B-Thinking-FP8 ↗	FP8	vLLM	2K	256	3,003.25 tok/s	292.33
Qwen3-8B-AWQ ↗	AWQ	vLLM	2K	128	3,016.02 tok/s	316.05
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	3,049.61 tok/s	277.57
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	128	3,075.68 tok/s	280.2
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	128	3,095.75 tok/s	277.3
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	128	3,096.26 tok/s	312.51
Qwen3-4B ↗	BF16	vLLM	2K	128	3,097.45 tok/s	343.47
Qwen3-1.7B ↗	BF16	vLLM	2K	32	3,118 tok/s	263.79
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	256	3,132.23 tok/s	309.61
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	128	3,150.16 tok/s	277.72
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	512	3,179.04 tok/s	292.35
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	128	3,194.65 tok/s	279.21
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	512	3,197.39 tok/s	289.54
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	3,210.93 tok/s	276.06
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	3,246.14 tok/s	280.48
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	3,249.3 tok/s	278.92
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	256	3,274.96 tok/s	309.95
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	128	3,292.09 tok/s	280.22
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	256	3,357.9 tok/s	290.56
Qwen3-8B-FP8 ↗	FP8	vLLM	2K	256	3,371.27 tok/s	311.44
Qwen3-30B-A3B-FP8 ↗	FP8	vLLM	2K	256	3,384.65 tok/s	290.25
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	128	3,404.39 tok/s	273.65
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	256	3,406.27 tok/s	311
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	256	3,411.53 tok/s	307.65
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	128	3,419.49 tok/s	270.99
Qwen3-8B-AWQ ↗	AWQ	vLLM	2K	256	3,432.32 tok/s	314.78
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	128	3,436.23 tok/s	272.94
Qwen3-4B-AWQ ↗	AWQ	vLLM	2K	64	3,470.2 tok/s	291.51
Qwen3-VL-8B-FP8 ↗	FP8	vLLM	2K	512	3,479.51 tok/s	309.82
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	128	3,515.54 tok/s	312.46
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	32	3,648.6 tok/s	214.73
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	32	3,680.94 tok/s	216.53
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	32	3,687.18 tok/s	217.51
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	64	3,715.75 tok/s	231.34
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	64	3,720.63 tok/s	230.27
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	64	3,769.38 tok/s	231.58
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	64	3,789.86 tok/s	233.5
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	256	3,833.43 tok/s	306.58
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	256	3,957.84 tok/s	293.33
Qwen3-4B ↗	BF16	vLLM	2K	256	3,982.47 tok/s	347.41
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	256	4,010.29 tok/s	308.72
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	256	4,015.68 tok/s	298.33
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	256	4,022.43 tok/s	297.43
Hunyuan-7B-FP8 ↗	FP8	vLLM	2K	512	4,024.58 tok/s	309.83
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	256	4,039.04 tok/s	294.68
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	256	4,073.56 tok/s	297.38
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	256	4,074.31 tok/s	293.27
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	256	4,086.05 tok/s	294.86
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	32	4,088.45 tok/s	185.66
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	32	4,090.68 tok/s	183.52
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	32	4,092.97 tok/s	186.86
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	32	4,094.54 tok/s	182.77
Qwen3-4B-AWQ ↗	AWQ	vLLM	2K	128	4,099.79 tok/s	304.82
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	32	4,118.01 tok/s	187.59
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	256	4,126.67 tok/s	292.11
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	256	4,139.24 tok/s	292.68
Qwen3-4B ↗	BF16	vLLM	2K	512	4,144.53 tok/s	348.51
Qwen3-0.6B ↗	BF16	vLLM	2K	32	4,179.65 tok/s	234.44
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	256	4,185.57 tok/s	293.15
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	512	4,199.45 tok/s	291.88
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	512	4,200.93 tok/s	298.09
Qwen3-4B-Thinking-FP8 ↗	FP8	vLLM	2K	512	4,205.28 tok/s	297.69
Qwen3-4B-AWQ ↗	AWQ	vLLM	2K	256	4,223.48 tok/s	302.05
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	512	4,223.99 tok/s	298.36
Qwen3-VL-4B-FP8 ↗	FP8	vLLM	2K	512	4,231.81 tok/s	292.9
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	512	4,248.29 tok/s	297.86
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	512	4,266.23 tok/s	292.16
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	512	4,279.09 tok/s	293.24
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	64	4,280.04 tok/s	242.05
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	64	4,280.71 tok/s	239.22
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	64	4,281.05 tok/s	236.55
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	64	4,281.55 tok/s	238.65
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	64	4,282.26 tok/s	236.48
Qwen3-4B-Instruct-FP8 ↗	FP8	vLLM	2K	512	4,301.45 tok/s	296.96
Qwen3-4B-FP8 ↗	FP8	vLLM	2K	256	4,510.88 tok/s	292.8
Qwen3-1.7B ↗	BF16	vLLM	2K	64	4,571.17 tok/s	276.8
Gemma-3-270m ↗	BF16	vLLM	4K	32	4,736.15 tok/s	188.5
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	128	5,022.91 tok/s	261.06
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	128	5,031.73 tok/s	263.84
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	64	5,035.17 tok/s	235.97
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	128	5,044.73 tok/s	264.21
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	64	5,049.38 tok/s	234.03
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	128	5,063.88 tok/s	263.3
Gemma-3-270m ↗	BF16	vLLM	1K	32	5,070.66 tok/s	185.03
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	64	5,112.76 tok/s	231.93
Gemma-3-270m ↗	BF16	vLLM	8K	32	5,116.35 tok/s	192.84
Gemma-3-270m ↗	BF16	vLLM	4K	512	5,229.81 tok/s	252.58
Gemma-3-270m ↗	BF16	vLLM	2K	32	5,235.34 tok/s	181.53
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	128	5,315.85 tok/s	268.63
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	128	5,317.53 tok/s	264.7
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	128	5,318.32 tok/s	265.05
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	128	5,319.29 tok/s	264.76
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	128	5,320.59 tok/s	265.27
Gemma-3-270m ↗	BF16	vLLM	4K	32	5,431.31 tok/s	184.86
Qwen3-1.7B ↗	BF16	vLLM	2K	128	5,461.19 tok/s	303.63
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	256	5,471.64 tok/s	277.59
Gemma-3-270m ↗	BF16	vLLM	2K	32	5,480.34 tok/s	181.78
Gemma-3-270m ↗	BF16	vLLM	2K	32	5,509.22 tok/s	179.5
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	256	5,519.06 tok/s	279.59
Gemma-3-270m ↗	BF16	vLLM	1K	32	5,519.73 tok/s	177.48
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	256	5,523.21 tok/s	277.02
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	256	5,534.9 tok/s	276.54
Gemma-3-270m ↗	BF16	vLLM	1K	32	5,586.19 tok/s	178.08
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	256	5,618.03 tok/s	278.51
Gemma-3-270m ↗	BF16	vLLM	256	32	5,676.02 tok/s	176.12
Qwen3-0.6B ↗	BF16	vLLM	2K	64	5,676.24 tok/s	261.53
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	512	5,709.94 tok/s	277.06
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	512	5,738.93 tok/s	278.59
Qwen3-VL-2B-FP8 ↗	FP8	vLLM	2K	512	5,754.84 tok/s	276.92
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	512	5,783.67 tok/s	280.65
Gemma-3-270m ↗	BF16	vLLM	512	32	5,806.54 tok/s	172.88
Gemma-3-270m ↗	BF16	vLLM	512	32	5,829.92 tok/s	171.14
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	512	5,969.07 tok/s	285.44
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	512	5,973.7 tok/s	280.9
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	256	5,987.98 tok/s	282.56
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	256	5,992.59 tok/s	279.28
Qwen3-1.7B ↗	BF16	vLLM	2K	512	5,993.48 tok/s	319.17
Qwen3-1.7B-FP8 ↗	FP8	vLLM	2K	256	5,995.28 tok/s	282.88
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	128	6,017.48 tok/s	250.6
Qwen3-1.7B ↗	BF16	vLLM	2K	256	6,019.91 tok/s	319.95
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	128	6,043.67 tok/s	254.45
Gemma-3-270m ↗	BF16	vLLM	256	32	6,067.09 tok/s	161.77
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	128	6,086.99 tok/s	252.24
Gemma-3-270m ↗	BF16	vLLM	256	128	6,102.07 tok/s	253.16
Gemma-3-270m ↗	BF16	vLLM	256	32	6,119.2 tok/s	169.44
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	64	6,224.49 tok/s	214.96
Gemma-3-270m ↗	BF16	vLLM	128	32	6,225.67 tok/s	157.92
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	64	6,241.98 tok/s	219.05
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	64	6,245.89 tok/s	216.88
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	64	6,246.15 tok/s	218.29
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	64	6,255.71 tok/s	218.61
Qwen3-0.6B ↗	BF16	vLLM	2K	128	6,268.94 tok/s	279.97
Gemma-3-270m ↗	BF16	vLLM	128	32	6,287.08 tok/s	156.81
Gemma-3-270m ↗	BF16	vLLM	128	32	6,305 tok/s	155.19
Gemma-3-270m ↗	BF16	vLLM	128	32	6,494.38 tok/s	156.56
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	256	6,523.32 tok/s	270.23
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	256	6,554.34 tok/s	271.02
Gemma-3-270m ↗	BF16	vLLM	128	32	6,590.82 tok/s	162.13
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	256	6,593.06 tok/s	270.02
Gemma-3-270m ↗	BF16	vLLM	2K	128	6,609.37 tok/s	263.43
Qwen3-0.6B ↗	BF16	vLLM	2K	512	6,633 tok/s	289.9
Qwen3-0.6B ↗	BF16	vLLM	2K	256	6,682.18 tok/s	290.79
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	512	6,772.9 tok/s	269.82
Hunyuan-1.8B-FP8 ↗	FP8	vLLM	2K	512	6,777 tok/s	271.56
Gemma-3-270m ↗	BF16	vLLM	4K	128	7,607.28 tok/s	243.69
Gemma-3-270m ↗	BF16	vLLM	2K	512	7,885.21 tok/s	266.09
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	128	8,087.53 tok/s	256.21
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	128	8,088.78 tok/s	262.62
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	128	8,090.46 tok/s	255.69
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	128	8,109.16 tok/s	260.79
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	128	8,111.09 tok/s	260.13
Gemma-3-270m ↗	BF16	vLLM	8K	512	8,452.13 tok/s	267.55
Gemma-3-270m ↗	BF16	vLLM	8K	128	8,584.2 tok/s	252.28
Gemma-3-270m ↗	BF16	vLLM	2K	128	8,913.21 tok/s	243.45
Gemma-3-270m ↗	BF16	vLLM	512	512	8,939.01 tok/s	261.82
Gemma-3-270m ↗	BF16	vLLM	4K	128	8,967.4 tok/s	240.86
Gemma-3-270m ↗	BF16	vLLM	1K	128	9,108.9 tok/s	248.31
Gemma-3-270m ↗	BF16	vLLM	4K	512	9,211.88 tok/s	261.53
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1024	9,270.05 tok/s	290.67
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	1024	9,293.75 tok/s	288.38
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	512	9,314.63 tok/s	288.79
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	512	9,322.34 tok/s	289.15
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	512	9,328.49 tok/s	285.28
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	256	9,373.14 tok/s	281.63
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	256	9,378.78 tok/s	286.19
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	256	9,380.9 tok/s	283.27
Qwen3-0.6B-FP8 ↗	FP8	vLLM	2K	256	9,389.58 tok/s	287.53
Gemma-3-270m ↗	BF16	vLLM	2K	128	9,516.01 tok/s	234.75
Gemma-3-270m ↗	BF16	vLLM	2K	512	9,695.96 tok/s	263.99
Gemma-3-270m ↗	BF16	vLLM	1K	128	9,781.99 tok/s	237.32
Gemma-3-270m ↗	BF16	vLLM	2K	512	9,785.93 tok/s	260.17
Gemma-3-270m ↗	BF16	vLLM	1K	128	9,836.61 tok/s	232.35
Gemma-3-270m ↗	BF16	vLLM	1K	512	10,115.03 tok/s	263.25
Gemma-3-270m ↗	BF16	vLLM	512	128	10,299.83 tok/s	228.27
Gemma-3-270m ↗	BF16	vLLM	512	128	10,407.53 tok/s	218.17
Gemma-3-270m ↗	BF16	vLLM	1K	512	10,434.61 tok/s	262.5
Gemma-3-270m ↗	BF16	vLLM	512	128	10,816.18 tok/s	219.66
Gemma-3-270m ↗	BF16	vLLM	512	512	11,978.52 tok/s	250.64
Gemma-3-270m ↗	BF16	vLLM	512	512	12,144.95 tok/s	252.42
Gemma-3-270m ↗	BF16	vLLM	256	128	12,219.3 tok/s	205.72
Gemma-3-270m ↗	BF16	vLLM	256	128	12,576.67 tok/s	211.73
Gemma-3-270m ↗	BF16	vLLM	128	128	12,831.27 tok/s	199.11
Gemma-3-270m ↗	BF16	vLLM	256	512	14,312.31 tok/s	236.52
Gemma-3-270m ↗	BF16	vLLM	256	512	14,354.97 tok/s	230.05
Gemma-3-270m ↗	BF16	vLLM	128	512	14,492.99 tok/s	231.59
Gemma-3-270m ↗	BF16	vLLM	128	128	14,741.29 tok/s	209.43
Gemma-3-270m ↗	BF16	vLLM	256	512	14,801.51 tok/s	230.89
Gemma-3-270m ↗	BF16	vLLM	128	128	14,844.36 tok/s	218.56
Gemma-3-270m ↗	BF16	vLLM	128	128	15,588.69 tok/s	212.36
Gemma-3-270m ↗	BF16	vLLM	128	512	18,723.26 tok/s	227.31
Gemma-3-270m ↗	BF16	vLLM	128	512	18,775.66 tok/s	242.81

Original reports

Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.

Original reports · 445 rows
Model	Test task	Format	Engine	Workload	Protocol	Quality score	Test date
AutoGLM-Phone-9B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 11:34:43 AM
DASD-4B-Thinking ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 5:25:35 PM
DeepSeek-OCR-2 ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 7:31:44 AM
DeepSeek-R1-Distill-Llama-8B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 6:28:03 PM
DeepSeek-R1-Distill-Qwen-1.5B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 4:37:38 PM
DeepSeek-R1-Distill-Qwen-1.5B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 5:42:06 AM
DeepSeek-R1-Distill-Qwen-7B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 5:56:49 PM
dots.ocr ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 7:54:51 AM
gemma-3-270m ↗	Text generation	Native	vLLM	2K	Method unknown	—	1/25/2026, 4:47:46 AM
gemma-3-270m ↗	Text generation	Native	vLLM	1K	Method unknown	—	1/25/2026, 4:33:01 AM
gemma-3-270m ↗	Text generation	Native	vLLM	512	Method unknown	—	1/25/2026, 4:23:12 AM
gemma-3-270m ↗	Text generation	Native	vLLM	256	Method unknown	—	1/25/2026, 4:14:45 AM
gemma-3-270m ↗	Text generation	Native	vLLM	128	Method unknown	—	1/25/2026, 4:07:02 AM
Gemma-3-270m ↗	Text generation	BF16	vLLM	Mixed	Method unknown	—	3/6/2026, 10:54:28 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	128K	Method unknown	—	3/6/2026, 10:51:17 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	64K	Method unknown	—	3/6/2026, 10:47:54 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	32K	Method unknown	—	3/6/2026, 10:43:08 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	16K	Method unknown	—	3/6/2026, 10:39:20 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	8K	Method unknown	—	3/6/2026, 10:29:22 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	4K	Method unknown	—	3/6/2026, 10:21:36 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	2K	Method unknown	—	3/6/2026, 10:15:08 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	1K	Method unknown	—	3/6/2026, 10:08:36 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	512	Method unknown	—	3/6/2026, 10:01:36 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	256	Method unknown	—	3/6/2026, 9:50:42 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	128	Method unknown	—	3/6/2026, 9:25:21 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	128	Method unknown	—	3/6/2026, 9:06:52 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	8K	Method unknown	—	3/6/2026, 9:01:54 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	Mixed	Method unknown	—	3/6/2026, 8:55:56 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	4K	Method unknown	—	3/6/2026, 8:55:28 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	128K	Method unknown	—	3/6/2026, 8:51:38 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	64K	Method unknown	—	3/6/2026, 8:48:24 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	2K	Method unknown	—	3/6/2026, 8:46:56 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	32K	Method unknown	—	3/6/2026, 8:45:10 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	16K	Method unknown	—	3/6/2026, 8:40:32 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	1K	Method unknown	—	3/6/2026, 8:39:50 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	8K	Method unknown	—	3/6/2026, 8:34:40 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	512	Method unknown	—	3/6/2026, 8:31:13 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	4K	Method unknown	—	3/6/2026, 8:26:23 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	2K	Method unknown	—	3/6/2026, 8:17:37 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	256	Method unknown	—	3/6/2026, 8:16:37 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	1K	Method unknown	—	3/6/2026, 8:09:47 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	512	Method unknown	—	3/6/2026, 8:02:55 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	128	Method unknown	—	3/6/2026, 7:53:02 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	256	Method unknown	—	3/6/2026, 7:52:24 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	128	Method unknown	—	3/6/2026, 7:43:39 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	128	Method unknown	—	3/6/2026, 4:29:01 PM
Gemma-3-270m ↗	Text generation	BF16	vLLM	1K	Method unknown	—	1/25/2026, 6:03:52 AM
Gemma-3-270m ↗	Text generation	BF16	vLLM	512	Method unknown	—	1/25/2026, 5:25:46 AM
Gemma-3-270m ↗	Text generation	BF16	vLLM	256	Method unknown	—	1/25/2026, 5:16:21 AM
Gemma-3-270m ↗	Text generation	BF16	vLLM	128	Method unknown	—	1/25/2026, 5:07:39 AM
Hunyuan-1.8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 10:28:46 AM
Hunyuan-1.8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 1:05:02 AM
Hunyuan-1.8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 3:53:51 PM
Hunyuan-7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 11:14:03 AM
Hunyuan-7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 1:34:54 AM
Hunyuan-7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 6:58:29 PM
HunyuanOCR ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 6:57:44 AM
HY-MT1.5-1.8B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/22/2026, 4:37:21 AM
HY-MT1.5-1.8B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 11:33:27 PM
HY-MT1.5-1.8B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 7:27:24 PM
HY-MT1.5-1.8B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 3:32:41 PM
HY-MT1.5-1.8B-Q6_K ↗	Text generation	Q6_K	vLLM	1K	Method unknown	—	1/22/2026, 4:39:13 AM
HY-MT1.5-1.8B-Q6_K ↗	Text generation	Q6_K	vLLM	1K	Method unknown	—	1/20/2026, 11:35:20 PM
HY-MT1.5-1.8B-Q6_K ↗	Text generation	Q6_K	vLLM	1K	Method unknown	—	1/20/2026, 7:29:13 PM
HY-MT1.5-1.8B-Q6_K ↗	Text generation	Q6_K	vLLM	1K	Method unknown	—	1/20/2026, 3:34:32 PM
HY-MT1.5-1.8B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/22/2026, 4:41:06 AM
HY-MT1.5-1.8B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 11:37:10 PM
HY-MT1.5-1.8B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 7:31:06 PM
HY-MT1.5-1.8B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 3:36:24 PM
HY-MT1.5-7B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 11:44:39 PM
HY-MT1.5-7B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 7:38:44 PM
HY-MT1.5-7B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 3:44:03 PM
HY-MT1.5-7B-Q6_K ↗	Text generation	Q6_K	vLLM	1K	Method unknown	—	1/20/2026, 11:46:34 PM
HY-MT1.5-7B-Q6_K ↗	Text generation	Q6_K	vLLM	1K	Method unknown	—	1/20/2026, 7:40:39 PM
HY-MT1.5-7B-Q6_K ↗	Text generation	Q6_K	vLLM	1K	Method unknown	—	1/20/2026, 3:46:40 PM
HY-MT1.5-7B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 11:48:28 PM
HY-MT1.5-7B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 7:42:35 PM
HY-MT1.5-7B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 3:50:25 PM
LightOnOCR-2-1B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 6:35:52 AM
LightOnOCR-2-1B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 5:30:22 AM
MAI-UI-8B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 9:06:04 AM
MiniCPM-o-4_5 ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 9:54:41 AM
Nanbeige4-3B-Thinking-2511 ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 4:56:30 PM
PaddleOCR-VL-1.5 ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 6:12:32 AM
PaddleOCR-VL-1.5 ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 6:06:59 AM
PaddleOCR-VL-1.5 ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 4:34:39 AM
Qwen2.5-0.5B-Instruct-AWQ ↗	Text generation	AWQ	vLLM	128	Method unknown	—	2/7/2026, 4:24:16 AM
Qwen2.5-0.5B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 11:24:33 AM
Qwen2.5-1.5B-Instruct-AWQ ↗	Text generation	AWQ	vLLM	128	Method unknown	—	2/7/2026, 6:52:03 AM
Qwen2.5-1.5B-Instruct-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	2/7/2026, 6:46:46 AM
Qwen2.5-1.5B-Instruct-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	2/7/2026, 6:36:02 AM
Qwen2.5-1.5B-Instruct-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	2/7/2026, 6:24:22 AM
Qwen2.5-7B-Instruct-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	2/6/2026, 5:25:17 AM
Qwen2.5-72B-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	1/14/2026, 8:15:40 AM
Qwen2.5-72B-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	1/12/2026, 8:12:35 PM
Qwen3-0.6B ↗	Text generation	BF16	vLLM	128	Method unknown	—	1/23/2026, 3:39:14 AM
Qwen3-0.6B ↗	Text generation	BF16	vLLM	256	Method unknown	—	1/22/2026, 10:04:49 PM
Qwen3-0.6B ↗	Text generation	BF16	vLLM	2K	Method unknown	—	1/15/2026, 3:37:15 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/11/2026, 2:14:25 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	128	Method unknown	—	2/10/2026, 10:28:01 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 9:45:17 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/6/2026, 6:41:49 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/6/2026, 6:30:47 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/3/2026, 1:19:22 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 3:41:45 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 3:32:08 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 3:25:08 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 3:23:28 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 3:17:13 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 3:14:02 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 3:07:30 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 3:06:33 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 3:00:09 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 2:57:39 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	1/26/2026, 2:25:35 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	128	Method unknown	—	1/26/2026, 1:29:36 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/25/2026, 3:49:49 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/25/2026, 3:33:08 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	1K	Method unknown	—	1/25/2026, 3:22:58 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/25/2026, 3:14:46 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	1/25/2026, 3:07:29 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	128	Method unknown	—	1/25/2026, 3:00:26 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/25/2026, 1:49:45 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 11:31:08 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 7:06:39 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 1:54:48 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 9:46:46 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 10:05:45 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 12:50:21 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 3:41:11 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/10/2026, 5:03:04 PM
Qwen3-0.6B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/22/2026, 4:33:38 AM
Qwen3-0.6B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/21/2026, 12:05:51 AM
Qwen3-0.6B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 11:29:48 PM
Qwen3-0.6B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 7:23:44 PM
Qwen3-0.6B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 3:28:52 PM
Qwen3-1.7B ↗	Text generation	BF16	vLLM	128	Method unknown	—	1/23/2026, 3:31:47 AM
Qwen3-1.7B ↗	Text generation	BF16	vLLM	1K	Method unknown	—	1/22/2026, 9:52:43 PM
Qwen3-1.7B ↗	Text generation	BF16	vLLM	2K	Method unknown	—	1/15/2026, 3:23:17 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 12:11:58 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/6/2026, 6:37:22 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/3/2026, 1:35:40 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 4:29:14 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 4:17:08 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 4:09:27 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 4:08:08 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 4:01:58 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 3:57:42 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 3:50:36 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 3:43:19 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	1K	Method unknown	—	1/25/2026, 5:14:18 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/25/2026, 5:01:45 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	1/25/2026, 4:49:37 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	128	Method unknown	—	1/25/2026, 4:37:27 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 11:47:23 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 7:26:47 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 2:14:41 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 10:01:35 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 10:17:22 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 12:58:00 AM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 3:47:52 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/10/2026, 5:07:24 PM
Qwen3-1.7B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/22/2026, 4:35:30 AM
Qwen3-1.7B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 11:31:38 PM
Qwen3-1.7B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 7:25:35 PM
Qwen3-1.7B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 3:30:49 PM
Qwen3-4B ↗	Text generation	BF16	vLLM	128	Method unknown	—	1/23/2026, 3:25:44 AM
Qwen3-4B ↗	Text generation	BF16	vLLM	1K	Method unknown	—	1/22/2026, 9:44:00 PM
Qwen3-4B ↗	Text generation	BF16	vLLM	2K	Method unknown	—	1/15/2026, 3:11:58 PM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vLLM	256	Method unknown	—	2/6/2026, 12:00:37 PM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vLLM	1K	Method unknown	—	1/25/2026, 6:32:49 AM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vLLM	512	Method unknown	—	1/25/2026, 6:24:57 AM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vLLM	256	Method unknown	—	1/25/2026, 6:17:50 AM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vLLM	128	Method unknown	—	1/25/2026, 6:10:53 AM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vLLM	128	Method unknown	—	1/23/2026, 2:57:30 AM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vLLM	1K	Method unknown	—	1/22/2026, 8:27:13 PM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	1/15/2026, 2:21:39 PM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 4:47:03 PM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 9:26:36 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 9:19:15 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 9:12:18 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 9:05:17 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 8:36:17 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 7:59:43 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 7:52:31 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 7:46:23 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 7:40:38 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 7:33:24 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 7:09:50 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	Mixed	Method unknown	—	2/7/2026, 4:15:18 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	128K	Method unknown	—	2/7/2026, 4:11:10 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	64K	Method unknown	—	2/7/2026, 4:07:59 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	32K	Method unknown	—	2/7/2026, 3:50:21 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	16K	Method unknown	—	2/7/2026, 3:34:14 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	2/7/2026, 2:59:36 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	2/7/2026, 2:46:49 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 2:40:19 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	1K	Method unknown	—	2/7/2026, 2:34:31 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	2/7/2026, 2:28:53 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/7/2026, 2:22:07 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	128	Method unknown	—	2/7/2026, 2:14:44 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/7/2026, 1:29:44 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 12:24:40 PM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 6:13:24 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 5:59:22 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 5:51:00 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 5:49:07 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 5:43:31 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 5:34:10 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 5:24:50 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 5:16:27 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/23/2026, 12:24:49 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 8:09:57 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 5:14:57 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 3:36:34 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 10:15:35 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 10:52:27 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 1:21:54 AM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 4:07:23 PM
Qwen3-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/10/2026, 5:12:11 PM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 5:10:28 PM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 12:43:43 PM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 9:03:58 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 8:50:14 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 8:42:16 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 8:35:09 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/23/2026, 4:00:03 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 8:47:46 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 5:52:24 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 4:14:27 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 10:38:39 AM
Qwen3-4B-Instruct-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/14/2026, 8:52:10 PM
Qwen3-4B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 11:39:04 PM
Qwen3-4B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 7:32:56 PM
Qwen3-4B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 3:38:19 PM
Qwen3-4B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 11:40:55 PM
Qwen3-4B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 7:34:50 PM
Qwen3-4B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 3:40:15 PM
Qwen3-4B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 11:42:47 PM
Qwen3-4B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 7:36:44 PM
Qwen3-4B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 3:42:08 PM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 5:02:40 PM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 12:37:08 PM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 8:07:12 AM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 7:53:24 AM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 7:44:54 AM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 7:36:56 AM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	128	Method unknown	—	1/23/2026, 3:53:56 AM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 8:34:36 AM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 5:39:27 AM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 4:01:14 AM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 10:26:57 AM
Qwen3-4B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/14/2026, 8:40:29 PM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	256	Method unknown	—	2/6/2026, 1:02:31 PM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	4K	Method unknown	—	1/25/2026, 7:35:48 AM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	1/25/2026, 7:20:41 AM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	1K	Method unknown	—	1/25/2026, 7:10:33 AM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	512	Method unknown	—	1/25/2026, 7:01:23 AM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	256	Method unknown	—	1/25/2026, 6:53:38 AM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	128	Method unknown	—	1/25/2026, 6:45:32 AM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	128	Method unknown	—	1/23/2026, 3:03:39 AM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	256	Method unknown	—	1/22/2026, 8:35:44 PM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	1/15/2026, 2:32:20 PM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 5:18:05 PM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 12:49:40 PM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 10:01:31 AM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 9:45:07 AM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 9:35:21 AM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 9:27:35 AM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 9:00:30 AM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 6:05:20 AM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 4:27:17 AM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 10:50:17 AM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 11:26:08 AM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 7:06:09 PM
Qwen3-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/10/2026, 5:17:29 PM
Qwen3-8B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 11:50:20 PM
Qwen3-8B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 7:44:28 PM
Qwen3-8B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 3:52:22 PM
Qwen3-8B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 11:52:14 PM
Qwen3-8B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 7:46:21 PM
Qwen3-8B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 3:54:34 PM
Qwen3-8B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 11:54:10 PM
Qwen3-8B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 7:48:15 PM
Qwen3-8B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 3:57:45 PM
Qwen3-14B-AWQ ↗	Text generation	AWQ	vLLM	256	Method unknown	—	2/6/2026, 1:14:50 PM
Qwen3-14B-AWQ ↗	Text generation	AWQ	vLLM	512	Method unknown	—	1/23/2026, 3:09:48 AM
Qwen3-14B-AWQ ↗	Text generation	AWQ	vLLM	128	Method unknown	—	1/22/2026, 8:42:05 PM
Qwen3-14B-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	1/15/2026, 2:44:05 PM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 5:35:46 PM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 1:08:55 PM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 11:58:25 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 11:43:46 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 11:32:13 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 11:24:10 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 9:28:52 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 6:33:27 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 4:55:39 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 11:19:25 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 11:51:54 AM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 7:22:25 PM
Qwen3-14B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/10/2026, 5:23:48 PM
Qwen3-14B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 11:56:08 PM
Qwen3-14B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 7:50:11 PM
Qwen3-14B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 4:02:01 PM
Qwen3-14B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 11:58:08 PM
Qwen3-14B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 7:52:11 PM
Qwen3-14B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 4:07:27 PM
Qwen3-14B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/21/2026, 12:00:07 AM
Qwen3-14B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 7:54:11 PM
Qwen3-14B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 4:13:41 PM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 5:53:56 PM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 1:27:00 PM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 2:47:10 PM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 2:31:20 PM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 2:18:48 PM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 2:08:49 PM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	1/23/2026, 4:14:18 AM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 10:01:13 AM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 5:28:14 AM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 11:33:06 AM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/14/2026, 9:04:30 PM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 1:35:23 PM
Qwen3-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/10/2026, 5:29:56 PM
Qwen3-30B-A3B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/21/2026, 12:02:04 AM
Qwen3-30B-A3B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 7:56:10 PM
Qwen3-30B-A3B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 4:22:46 PM
Qwen3-30B-A3B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 7:58:10 PM
Qwen3-30B-A3B-Q8_0 ↗	Text generation	Q8_0	vLLM	1K	Method unknown	—	1/20/2026, 4:56:37 PM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 6:03:20 PM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 1:32:34 PM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 3:31:53 PM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 3:15:33 PM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 3:03:08 PM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 2:54:22 PM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/23/2026, 3:46:57 AM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 10:16:20 AM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 5:43:44 AM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 10:55:10 AM
Qwen3-30B-A3B-Thinking-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/14/2026, 9:16:49 PM
Qwen3-32B-AWQ ↗	Text generation	AWQ	vLLM	256	Method unknown	—	2/6/2026, 1:49:37 PM
Qwen3-32B-AWQ ↗	Text generation	AWQ	vLLM	256	Method unknown	—	1/23/2026, 3:17:48 AM
Qwen3-32B-AWQ ↗	Text generation	AWQ	vLLM	4K	Method unknown	—	1/22/2026, 8:48:34 PM
Qwen3-32B-AWQ ↗	Text generation	AWQ	vLLM	2K	Method unknown	—	1/15/2026, 2:56:53 PM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 6:12:14 PM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 1:38:23 PM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 4:50:40 PM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 4:10:15 PM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 3:48:31 PM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 3:38:08 PM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 10:34:40 AM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 6:01:33 AM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 11:46:15 AM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/14/2026, 4:14:36 AM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 7:44:37 PM
Qwen3-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/10/2026, 5:40:41 PM
Qwen3-32B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 8:00:15 PM
Qwen3-32B-Q4_K_M ↗	Text generation	Q4_K_M	vLLM	1K	Method unknown	—	1/20/2026, 5:05:10 PM
Qwen3-32B-Q5_K_M ↗	Text generation	Q5_K_M	vLLM	1K	Method unknown	—	1/20/2026, 8:02:21 PM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 4:38:34 PM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 12:18:22 PM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 5:23:39 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 5:11:05 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 5:02:00 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 5:00:40 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 4:54:16 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 4:50:09 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 4:40:56 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 4:31:39 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/23/2026, 12:07:35 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 7:49:56 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 4:54:52 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 3:17:00 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/15/2026, 4:26:39 PM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 10:40:59 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 1:12:54 AM
Qwen3-VL-2B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 4:00:25 PM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 4:55:10 PM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 12:31:10 PM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 7:16:41 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 7:03:19 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 6:54:55 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 6:47:03 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 8:23:18 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 5:28:07 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 3:49:49 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 12:53:18 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 11:02:52 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 1:30:03 AM
Qwen3-VL-4B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 4:14:27 PM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	3/2/2026, 10:19:54 PM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	3/2/2026, 9:19:18 PM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	3/2/2026, 9:05:44 PM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	3/1/2026, 3:52:51 PM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 5:26:36 PM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 12:55:59 PM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 11:06:07 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 10:55:04 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 10:45:38 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 10:38:33 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/23/2026, 4:06:50 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 9:15:47 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 6:20:23 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 4:42:24 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 1:04:48 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 11:38:12 AM
Qwen3-VL-8B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 7:13:50 PM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 5:44:50 PM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 1:21:01 PM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 1:26:59 PM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 1:13:43 PM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 1:01:39 PM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 12:52:05 PM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	1/23/2026, 4:21:33 AM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 9:45:50 AM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 6:50:34 AM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 5:12:46 AM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 1:18:25 AM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/14/2026, 3:16:16 AM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/13/2026, 11:57:27 AM
Qwen3-VL-30B-A3B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 7:30:30 PM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/9/2026, 6:31:18 PM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	256	Method unknown	—	2/6/2026, 1:44:06 PM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	8K	Method unknown	—	1/26/2026, 5:43:05 PM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	4K	Method unknown	—	1/26/2026, 5:20:55 PM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/26/2026, 5:07:41 PM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	512	Method unknown	—	1/26/2026, 4:57:04 PM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/22/2026, 11:18:25 AM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/20/2026, 1:33:14 AM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/14/2026, 4:56:43 AM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/14/2026, 4:34:55 AM
Qwen3-VL-32B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	1/12/2026, 7:53:41 PM
Qwen3.5-0.8B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/3/2026, 9:16:20 AM
Qwen3.5-2B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/3/2026, 9:23:57 AM
Qwen3.5-4B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/3/2026, 9:32:08 AM
Qwen3.5-9B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/3/2026, 9:39:06 AM
Qwen3.5-27B-FP8 ↗	Text generation	FP8	vLLM	2K	Method unknown	—	2/27/2026, 12:41:19 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	hle-test-self-judge-22ed307-v1-n5-mt16384-t0	quality-v1	20%	10/17/2026, 2:30:19 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	simpleqa-verified-eval-self-judge-0dc97e0-v1-n1000-mt16384-t0	quality-v1	6.4%	10/17/2026, 1:50:19 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	simpleqa-verified-eval-self-judge-0dc97e0-v1-n5-mt16384-t0	quality-v1	0%	10/15/2026, 9:23:17 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	ifbench-test-official-loose-1c40f0c-v1-n300-mt16384-t0	quality-v1	67.67%	10/15/2026, 8:57:08 PM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	ifbench-test-official-loose-1c40f0c-v1-n3-mt16384-t0	quality-v1	66.67%	10/15/2026, 9:27:35 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	aime2026-test-zero-shot-strict-v4-n30-mt16384-t0	quality-v1	63.33%	10/15/2026, 9:07:41 AM
Qwen3.8-27B-NVFP4 ↗	Text generation	NVFP4	vllm	mmlu-pro-test-zero-shot-strict-v4-n3-mt16384-t0	quality-v1	66.67%	10/15/2026, 6:20:58 AM
Step3-VL-10B ↗	Text generation	Native	vLLM	2K	Method unknown	—	3/4/2026, 10:44:11 AM

Running benchmarks

0Updates automatically

No benchmarks are currently reported as running.

Methodology

Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.

Updated 11/5/2026, 9:54:22 PM

Text generation · Qualified results

Measured alternatives on other GPUs

Qwen3.8-27B-NVFP4

Qwen3-4B-AWQ

Quality by dataset

Original reports

Running benchmarks