24 GB · 111 reports

L4 · 24 GB

Qualified results first; original reports and full tables stay below.

All tasks · Qualified results

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

Qwen3-4B-AWQ
Qwen · 2 tested configurations
- AWQ · 8K → 2K tokens · Prefix cache reuse
  Output throughput · includes prefill · 1 request
  57.4tok/s
  Best measured · 4 concurrent requests
  175tok/s
  Observed GPU occupancy
  19.5GiB
  First token
  60 ms
  Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
  Test details · 2 concurrency levels
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  vllm · vLLM 0.29.0
  Protocol
  wbench-fixed-http-v3
  Dataset
  migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=prefix-reuse:repeats=3
  Workload
  8192in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 57.39 35.7 s 60 ms 19.55 71.8 1 7e2cbfda ↗
  C4 175.39 46.75 s 114 ms 19.55 71.9 1 7e2cbfda ↗
- 1 other test configurations on this GPU
  - AWQ · 8K → 2K tokens · No cache reuse
    Output throughput · includes prefill · 1 request
    55.3tok/s
    Best measured · 4 concurrent requests
    109tok/s
    Observed GPU occupancy
    19.5GiB
    First token
    1.6 s
    Use this recipe · 4 concurrent requests ↗Evidence report and run availability ↗Same test on other GPUs
    Test details · 2 concurrency levels
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    vllm · vLLM 0.29.0
    Protocol
    wbench-fixed-http-v3
    Dataset
    migration-analysis-v1:sha256=68590c248e43f32c04dfd8062464fd1e4856917795fe307225e5c6112a54fe94:cache=isolated:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 55.3 37.17 s 1.6 s 19.54 71.9 1 9301a9ba ↗
    C4 109.43 78.88 s 3.1 s 19.55 71.9 1 9301a9ba ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	57.39	35.7 s	60 ms	19.55	71.8	1	7e2cbfda ↗
C4	175.39	46.75 s	114 ms	19.55	71.9	1	7e2cbfda ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	55.3	37.17 s	1.6 s	19.54	71.9	1	9301a9ba ↗
C4	109.43	78.88 s	3.1 s	19.55	71.9	1	9301a9ba ↗

embeddinggemma-2

unsloth · 2 tested configurations

nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 175vectors/sec
Best measured · Batch 16: 1,650vectors/sec
Observed GPU occupancy: 0.9GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	174.95	6 ms	—	0.85	54.8	5	29b19870 ↗
B4	572.6	7 ms	—	0.85	58.4	5	29b19870 ↗
B16	1,649.62	10 ms	—	0.85	60.7	5	29b19870 ↗

1 other test configurations on this GPU

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 177vectors/sec
Best measured · Batch 16: 2,036vectors/sec
Observed GPU occupancy: 1GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	177.21	6 ms	—	0.99	50.5	4	d74f69cc ↗
B4	648.08	6 ms	—	0.99	57.1	4	d74f69cc ↗
B16	2,035.67	8 ms	—	0.99	58.1	4	d74f69cc ↗

embeddinggemma-2

google · 2 tested configurations

fp16 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 177vectors/sec
Best measured · Batch 16: 2,039vectors/sec
Observed GPU occupancy: 1GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	177.46	6 ms	—	0.99	49.8	6	e0700e04 ↗
B4	650.16	6 ms	—	0.99	56.9	6	e0700e04 ↗
B16	2,039.17	8 ms	—	0.99	58.4	6	e0700e04 ↗

1 other test configurations on this GPU

nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 175vectors/sec
Best measured · Batch 16: 1,639vectors/sec
Observed GPU occupancy: 0.9GiB
P95 latency: 6 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	174.88	6 ms	—	0.85	53.1	6	7ad6a821 ↗
B4	571.03	7 ms	—	0.85	57	6	7ad6a821 ↗
B16	1,639.08	10 ms	—	0.85	59.8	6	7ad6a821 ↗

granite-embedding-97m-multilingual-r2

ibm-granite · 1 tested configuration

nf4 · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s

Throughput · Batch 1: 792vectors/sec
Best measured · Batch 16: 8,881vectors/sec
Observed GPU occupancy: 0.5GiB
P95 latency: 1 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 3 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: sentence-transformers · PyTorch 2.12.1+cu126
Protocol: native-forward-v1
Dataset: embedding-mteb-tasks-v1
Workload: embedding

Batch size	Mean throughput (vectors/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	791.85	1 ms	—	0.51	37.4	1	d396c54b ↗
B4	2,816.45	1 ms	—	0.51	37.9	1	d396c54b ↗
B16	8,880.86	2 ms	—	0.51	38.9	1	d396c54b ↗

Qwen3.8-27B-UD-Q4_K_M.gguf
unsloth/Qwen3.8-27B-GGUF · 2 tested configurations
- Q4_K_M · 32K → 2K tokens
  Output throughput · includes prefill · 1 request
  12.9tok/s
  Observed GPU occupancy
  17GiB
  First token
  371 ms
  Evidence report and run availability ↗Same test on other GPUs
  Test details
  Total output tokens divided by the full request window, including prefill and queueing.
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  llamacpp
  Protocol
  wbench-fixed-http-v1
  Dataset
  synthetic-tokenizer-v1:repeats=3
  Workload
  32768in-2048out
  Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  C1 12.94 158.32 s 371 ms 17.05 71.8 1 d8a4b66b ↗
- 1 other test configurations on this GPU
  - Q4_K_M · 8K → 2K tokens
    Output throughput · includes prefill · 1 request
    14.1tok/s
    Observed GPU occupancy
    17GiB
    First token
    279 ms
    Evidence report and run availability ↗Same test on other GPUs
    Test details
    Total output tokens divided by the full request window, including prefill and queueing.
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    llamacpp
    Protocol
    wbench-fixed-http-v1
    Dataset
    synthetic-tokenizer-v1:repeats=3
    Workload
    8192in-2048out
    Concurrency Mean throughput (tok/s) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    C1 14.09 145.5 s 279 ms 17.05 71.9 1 4bc32a6c ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	12.94	158.32 s	371 ms	17.05	71.8	1	d8a4b66b ↗

Concurrency	Mean throughput (tok/s)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
C1	14.09	145.5 s	279 ms	17.05	71.9	1	4bc32a6c ↗

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Sortable tables by exact test condition · 21 rows

C: concurrency · B: batch · —: not measured

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/embeddinggemma-2Scaling curves ↗	nf4	sentence-transformers	embeddingnative-forward-v1	B1	174.95	175.2	—	0.006	0.85	54.8	174.56	3.19	5 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/embeddinggemma-2Scaling curves ↗	fp16	sentence-transformers	embeddingnative-forward-v1	B1	177.21	177.49	—	0.006	0.99	50.5	176.49	3.51	4 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
google/embeddinggemma-2Scaling curves ↗	fp16	sentence-transformers	embeddingnative-forward-v1	B1	177.46	177.6	—	0.006	0.99	49.8	177.32	3.56	6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
google/embeddinggemma-2Scaling curves ↗	nf4	sentence-transformers	embeddingnative-forward-v1	B1	174.88	175.5	—	0.006	0.85	53.1	173.2	3.3	6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 1 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
ibm-granite/granite-embedding-97m-multilingual-r2Scaling curves ↗	nf4	sentence-transformers	embeddingnative-forward-v1	B1	791.85	791.85	—	0.001	0.51	37.4	791.85	21.19	1 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/embeddinggemma-2Scaling curves ↗	nf4	sentence-transformers	embeddingnative-forward-v1	B4	572.6	573.69	—	0.007	0.85	58.4	571.58	9.81	5 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/embeddinggemma-2Scaling curves ↗	fp16	sentence-transformers	embeddingnative-forward-v1	B4	648.08	650.84	—	0.006	0.99	57.1	641.3	11.36	4 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
google/embeddinggemma-2Scaling curves ↗	fp16	sentence-transformers	embeddingnative-forward-v1	B4	650.16	650.67	—	0.006	0.99	56.9	649.39	11.44	6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
google/embeddinggemma-2Scaling curves ↗	nf4	sentence-transformers	embeddingnative-forward-v1	B4	571.03	573.83	—	0.007	0.85	57	564.91	10.03	6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 4 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
ibm-granite/granite-embedding-97m-multilingual-r2Scaling curves ↗	nf4	sentence-transformers	embeddingnative-forward-v1	B4	2,816.45	2,816.45	—	0.001	0.51	37.9	2,816.45	74.36	1 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/embeddinggemma-2Scaling curves ↗	nf4	sentence-transformers	embeddingnative-forward-v1	B16	1,649.62	1,658.11	—	0.01	0.85	60.7	1,646.49	27.17	5 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/embeddinggemma-2Scaling curves ↗	fp16	sentence-transformers	embeddingnative-forward-v1	B16	2,035.67	2,049.35	—	0.008	0.99	58.1	2,014.24	35.05	4 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
google/embeddinggemma-2Scaling curves ↗	fp16	sentence-transformers	embeddingnative-forward-v1	B16	2,039.17	2,043.49	—	0.008	0.99	58.4	2,033.4	34.93	6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
google/embeddinggemma-2Scaling curves ↗	nf4	sentence-transformers	embeddingnative-forward-v1	B16	1,639.08	1,654.37	—	0.01	0.85	59.8	1,612.71	27.44	6 ↗

Embedding · Text → Vectors · embedding-mteb-tasks-v1 · Test duration 90 s · Batch 16 · vectors/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
ibm-granite/granite-embedding-97m-multilingual-r2Scaling curves ↗	nf4	sentence-transformers	embeddingnative-forward-v1	B16	8,880.86	8,880.86	—	0.002	0.51	38.9	8,880.86	228.46	1 ↗

Text generation · 8K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.ggufScaling curves ↗	Q4_K_M	llamacpp	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v1	C1	14.09	14.09	278.67	145.499	17.05	71.9	14.09	0.2	1 ↗

Text generation · 8K → 2K tokens · No cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen/Qwen3-4B-AWQScaling curves ↗	AWQ	vllm	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3	C1	55.3	55.3	1,582.33	37.174	19.54	71.9	55.3	0.77	1 ↗

Text generation · 8K → 2K tokens · No cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen/Qwen3-4B-AWQScaling curves ↗	AWQ	vllm	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3	C4	109.43	109.43	3,149.58	78.879	19.55	71.9	109.43	1.52	1 ↗

Text generation · 8K → 2K tokens · Prefix cache reuse · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen/Qwen3-4B-AWQScaling curves ↗	AWQ	vllm	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3	C1	57.39	57.39	60	35.702	19.55	71.8	57.39	0.8	1 ↗

Text generation · 8K → 2K tokens · Prefix cache reuse · 4 concurrent requests · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Qwen/Qwen3-4B-AWQScaling curves ↗	AWQ	vllm	8192in-2048out8,192 → 2,048 tokenswbench-fixed-http-v3	C4	175.39	175.39	114	46.748	19.55	71.9	175.39	2.44	1 ↗

Text generation · 32K → 2K tokens · 1 request · tok/s · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.ggufScaling curves ↗	Q4_K_M	llamacpp	32768in-2048out32,768 → 2,048 tokenswbench-fixed-http-v1	C1	12.94	12.94	371	158.323	17.05	71.8	12.94	0.18	1 ↗

Scaling curves and individual runs

One model and test protocol per curve; different workloads are never joined into one line.

Benchmark configuration

unsloth/embeddinggemma-2 · nf4 · sentence-transformers · embedding · native-forward-v1

Mean throughput vectors/sec

Measured points only. Select a point to open its source report.

Mean run P95 latency s

Measured points only. Select a point to open its source report.

Mean GPU memory used GB

Measured points only. Select a point to open its source report.

Each point is the mean of recorded runs at that level. A source report may differ from the mean; all runs are listed below.

Runs behind the curve · unsloth/embeddinggemma-2 · vectors/sec · 15 rows
Batch size	Throughput	Decode (tok/s)	Uncached prefill (tok/s)	GPU power (W)	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	Test date	View report
1	175.2 vectors/sec	—	Uncached prefill unavailableView report	54.76	—	0.006	0.85	10/19/2026, 9:24:09 AM	29b19870 ↗
1	174.56 vectors/sec	—	Uncached prefill unavailableView report	54.33	—	0.006	0.85	10/19/2026, 9:05:59 AM	d676e386 ↗
1	174.96 vectors/sec	—	Uncached prefill unavailableView report	55.45	—	0.006	0.85	10/19/2026, 7:53:50 AM	bc9747cb ↗
1	175.13 vectors/sec	—	Uncached prefill unavailableView report	54.97	—	0.006	0.85	10/19/2026, 7:35:58 AM	e0a15fd7 ↗
1	174.92 vectors/sec	—	Uncached prefill unavailableView report	54.44	—	0.006	0.85	10/19/2026, 7:17:49 AM	41a06ae6 ↗
4	573.09 vectors/sec	—	Uncached prefill unavailableView report	58.66	—	0.007	0.85	10/19/2026, 9:24:09 AM	29b19870 ↗
4	571.58 vectors/sec	—	Uncached prefill unavailableView report	58.33	—	0.007	0.85	10/19/2026, 9:05:59 AM	d676e386 ↗
4	573.07 vectors/sec	—	Uncached prefill unavailableView report	57.94	—	0.007	0.85	10/19/2026, 7:53:50 AM	bc9747cb ↗
4	573.69 vectors/sec	—	Uncached prefill unavailableView report	58.16	—	0.007	0.85	10/19/2026, 7:35:58 AM	e0a15fd7 ↗
4	571.58 vectors/sec	—	Uncached prefill unavailableView report	58.68	—	0.007	0.85	10/19/2026, 7:17:49 AM	41a06ae6 ↗
16	1,646.9 vectors/sec	—	Uncached prefill unavailableView report	60.8	—	0.01	0.85	10/19/2026, 9:24:09 AM	29b19870 ↗
16	1,646.49 vectors/sec	—	Uncached prefill unavailableView report	60.42	—	0.01	0.85	10/19/2026, 9:05:59 AM	d676e386 ↗
16	1,649.28 vectors/sec	—	Uncached prefill unavailableView report	60.84	—	0.01	0.85	10/19/2026, 7:53:50 AM	bc9747cb ↗
16	1,658.11 vectors/sec	—	Uncached prefill unavailableView report	60.74	—	0.01	0.85	10/19/2026, 7:35:58 AM	e0a15fd7 ↗
16	1,647.34 vectors/sec	—	Uncached prefill unavailableView report	60.72	—	0.01	0.85	10/19/2026, 7:17:49 AM	41a06ae6 ↗

Quality by dataset

Scores from different datasets and sample scopes are not directly comparable.

Quality by dataset · 3 rows
Model	Dataset	Format	Engine	Average score	Sample size	Runs
hf:unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.gguf	gsm8k-test-zero-shot-strict-v2-n3-mt16384-t0	Q4_K_M	llamacpp	100%	3 / 1319 · Subset only	1 runs 100% · ab312fed ↗
hf:unsloth/Qwen3.8-27B-GGUF/Qwen3.8-27B-UD-Q4_K_M.gguf	mmlu-test-zero-shot-strict-v2-n3-mt16384-t0	Q4_K_M	llamacpp	100%	3 / 14042 · Subset only	1 runs 100% · 30bcfc00 ↗
Qwen/Qwen3-0.6B-FP8	aime2026-test-zero-shot-strict-v4-n30-mt16384-t0	FP8	vllm	0%	30 / 30 · Full dataset	1 runs 0% · 35ae8f96 ↗

Measurements without a comparable protocol · 16 rows

These measured rates remain visible, but are excluded from comparison curves because the protocol is unknown.

Measurements without a comparable protocol · 16 rows
Model	Format	Engine	Workload	Concurrency	Throughput	GPU power (W)
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	1	180.69 tok/s	52.28
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	2	197.1 tok/s	55.46
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	4	92.45 tok/s	39.96
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	8	150.14 tok/s	47.71
Qwen2.5-0.5B ↗	Q4_K_M	Ollama	1K	16	197.53 tok/s	58.26
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	1	82.39 tok/s	69.71
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	2	83.62 tok/s	70.71
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	4	84.03 tok/s	70.48
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	8	83.62 tok/s	70.64
Qwen2.5-3B ↗	Q4_K_M	Ollama	1K	16	83.96 tok/s	70.15
Qwen3-0.6B ↗	Native	vLLM	2K	1	133.88 tok/s	69.47
Qwen3-0.6B ↗	Native	vLLM	2K	8	548.24 tok/s	58.33
Qwen3-0.6B ↗	Native	vLLM	2K	16	994.33 tok/s	66.32
Qwen3-0.6B ↗	Native	vLLM	2K	32	1,418.76 tok/s	69.6
Qwen3-0.6B ↗	Native	vLLM	2K	64	1,711.93 tok/s	70.52
Qwen3-0.6B ↗	Native	vLLM	2K	128	1,826.9 tok/s	70.28

Original reports

Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.

Original reports · 111 rows
Model	Test task	Format	Engine	Workload	Protocol	Quality score	Test date
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 9:24:09 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 9:16:01 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 9:05:59 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 8:57:46 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 8:47:51 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 8:39:42 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 8:29:57 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 8:21:47 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 8:12:01 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 8:03:34 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 7:53:50 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 7:45:47 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 7:35:58 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 7:27:38 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 7:17:49 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 7:09:29 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 6:59:52 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 6:51:53 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 6:41:51 AM
embeddinggemma-2 ↗	Embedding	fp16	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 6:33:03 AM
embeddinggemma-2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/19/2026, 6:22:26 AM
granite-embedding-97m-multilingual-r2 ↗	Embedding	nf4	sentence-transformers	embedding	native-forward-v1	—	10/18/2026, 5:01:39 PM
Llama-3.1-8B-Instruct-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:46:40 PM
Llama-3.1-8B-Instruct-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:43:48 PM
Llama-3.1-8B-Instruct-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:41:11 PM
Ministral-3-3B-Instruct-2512 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:08:22 PM
Ministral-3-3B-Instruct-2512 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:06:35 PM
Ministral-3-3B-Instruct-2512 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:05:09 PM
Ministral-3-8B-Instruct-2512 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:32:15 PM
Ministral-3-8B-Instruct-2512 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:28:53 PM
Ministral-3-8B-Instruct-2512 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:25:59 PM
Ministral-3-14B-Instruct-2512 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:30:05 PM
Ministral-3-14B-Instruct-2512 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:25:16 PM
Ministral-3-14B-Instruct-2512 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:20:53 PM
Ministral-3-14B-Instruct-2512-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:17:56 PM
Ministral-3-14B-Instruct-2512-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:13:13 PM
Ministral-3-14B-Instruct-2512-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:08:50 PM
NVIDIA-Nemotron-3-Nano-4B-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:39:58 PM
NVIDIA-Nemotron-3-Nano-4B-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:39:46 PM
NVIDIA-Nemotron-3-Nano-4B-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:39:39 PM
NVIDIA-Nemotron-Nano-9B-v2-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:51:00 PM
NVIDIA-Nemotron-Nano-9B-v2-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:47:42 PM
NVIDIA-Nemotron-Nano-9B-v2-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:44:31 PM
NVIDIA-Nemotron-Nano-12B-v2-VL-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:10:50 PM
NVIDIA-Nemotron-Nano-12B-v2-VL-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:06:44 PM
NVIDIA-Nemotron-Nano-12B-v2-VL-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:02:46 PM
Qwen2.5-0.5B ↗	Text generation	Q4_K_M	Ollama	1K	Method unknown	—	10/14/2026, 8:36:53 AM
qwen2.5-0.5b-instruct-q4_k_m.gguf ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:06:56 AM
qwen2.5-1.5b-instruct-q4_k_m.gguf ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:09:44 AM
Qwen2.5-3B ↗	Text generation	Q4_K_M	Ollama	1K	Method unknown	—	10/14/2026, 8:42:44 AM
qwen2.5-3b-instruct-q4_k_m.gguf ↗	Text generation	Q4_K_M	llama.cpp	1K	wbench-http	—	10/14/2026, 9:12:52 AM
Qwen3-0.6B ↗	Text generation	Native	vLLM	2K	Method unknown	—	1/11/2026, 11:13:47 AM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v2	—	10/18/2026, 8:02:34 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v2	—	10/18/2026, 8:01:15 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v2	—	10/18/2026, 8:00:32 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:09:14 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:08:19 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:07:39 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:17:31 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:16:39 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:16:05 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	aime2026-test-zero-shot-strict-v4-n30-mt16384-t0	quality-v1	0%	10/18/2026, 1:22:12 PM
Qwen3-0.6B-FP8 ↗	Text generation	FP8	vllm	512in-256out	wbench-fixed-http-v1	—	10/18/2026, 12:25:46 PM
Qwen3-0.6B-GPTQ-Int8 ↗	Text generation	GPTQ	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:33:40 PM
Qwen3-0.6B-GPTQ-Int8 ↗	Text generation	GPTQ	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:32:50 PM
Qwen3-0.6B-GPTQ-Int8 ↗	Text generation	GPTQ	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:32:16 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:16:30 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:15:00 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 6:13:58 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:22:05 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:20:45 PM
Qwen3-1.7B-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:19:47 PM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vllm	8192in-2048out	wbench-fixed-http-v3	—	10/23/2026, 6:48:39 AM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vllm	8192in-2048out	wbench-fixed-http-v3	—	10/23/2026, 6:45:39 AM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:37:49 PM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:36:11 PM
Qwen3-4B-AWQ ↗	Text generation	AWQ	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:35:06 PM
Qwen3-4B-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:45:11 PM
Qwen3-4B-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:43:01 PM
Qwen3-4B-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:41:14 PM
Qwen3-4B-Instruct-2507-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:53:16 PM
Qwen3-4B-Instruct-2507-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:51:10 PM
Qwen3-4B-Instruct-2507-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:49:25 PM
Qwen3-4B-Thinking-2507-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:55:07 PM
Qwen3-4B-Thinking-2507-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:52:58 PM
Qwen3-4B-Thinking-2507-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:51:08 PM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:42:12 PM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:40:00 PM
Qwen3-8B-AWQ ↗	Text generation	AWQ	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:38:09 PM
Qwen3-8B-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:12:18 PM
Qwen3-8B-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:09:03 PM
Qwen3-8B-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:06:15 PM
Qwen3-14B-AWQ ↗	Text generation	AWQ	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:46:30 PM
Qwen3-14B-AWQ ↗	Text generation	AWQ	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:43:18 PM
Qwen3-14B-AWQ ↗	Text generation	AWQ	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:40:35 PM
Qwen3-14B-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:28:01 PM
Qwen3-14B-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:22:42 PM
Qwen3-14B-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 4:17:46 PM
Qwen3-VL-2B-Instruct-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:43:26 PM
Qwen3-VL-2B-Instruct-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:42:17 PM
Qwen3-VL-2B-Instruct-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:41:22 PM
Qwen3-VL-4B-Instruct-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:49:57 PM
Qwen3-VL-4B-Instruct-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:48:02 PM
Qwen3-VL-4B-Instruct-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 2:46:30 PM
Qwen3-VL-8B-Instruct-FP8 ↗	Text generation	FP8	vllm	8192in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:13:17 PM
Qwen3-VL-8B-Instruct-FP8 ↗	Text generation	FP8	vllm	2048in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:10:15 PM
Qwen3-VL-8B-Instruct-FP8 ↗	Text generation	FP8	vllm	512in-512out	wbench-fixed-http-v1	—	10/18/2026, 3:07:45 PM
Qwen3.8-27B-UD-Q4_K_M.gguf ↗	Text generation	Q4_K_M	llamacpp	gsm8k-test-zero-shot-strict-v2-n3-mt16384-t0	quality-v1	100%	10/14/2026, 1:13:34 PM
Qwen3.8-27B-UD-Q4_K_M.gguf ↗	Text generation	Q4_K_M	llamacpp	mmlu-test-zero-shot-strict-v2-n3-mt16384-t0	quality-v1	100%	10/14/2026, 1:12:01 PM
Qwen3.8-27B-UD-Q4_K_M.gguf ↗	Text generation	Q4_K_M	llamacpp	32768in-2048out	wbench-fixed-http-v1	—	10/14/2026, 1:05:27 PM
Qwen3.8-27B-UD-Q4_K_M.gguf ↗	Text generation	Q4_K_M	llamacpp	8192in-2048out	wbench-fixed-http-v1	—	10/14/2026, 12:56:36 PM

Running benchmarks

0Updates automatically

No benchmarks are currently reported as running.

L4 · 24 GB · 91 queued · 0 running · 0 awaiting reassignment · 22 succeeded · 7 failed or exhausted

Methodology

Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.

Updated 11/5/2026, 11:06:29 PM

All tasks · Qualified results

Qwen3-4B-AWQ

embeddinggemma-2

embeddinggemma-2

granite-embedding-97m-multilingual-r2

Qwen3.8-27B-UD-Q4_K_M.gguf

Quality by dataset

Original reports

Running benchmarks