96 GB · 196 reports

RTX PRO 6000 Blackwell Server Edition · 96 GB

Qualified results first; original reports and full tables stay below.

Decision · Qualified results

Speeds compare only within one exact test. Recipe availability is checked on each report against the published recipe executor.

StartLux-Decision-v1-0.8B
startlux-models · 2 tested configurations
- bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
  Throughput · Batch 1
  50.3decisions/sec
  Observed GPU occupancy
  2.3GiB
  P95 latency
  20 ms
  Use this recipe · Batch 1 ↗Evidence report and run availability ↗Same test on other GPUs
  Test details
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  startlux-native · PyTorch 2.11.0+cu130
  Protocol
  native-forward-v1
  Dataset
  typed-invoices-v1
  Workload
  decision
  Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 50.28 20 ms — 2.29 165 1 8c1985ea ↗
- 1 other test configurations on this GPU
  - bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
    Throughput · Batch 1
    50.3decisions/sec
    Observed GPU occupancy
    3.2GiB
    P95 latency
    20 ms
    Use this recipe · Batch 1 ↗Evidence report and run availability ↗Same test on other GPUs
    Test details
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    startlux-native · PyTorch 2.11.0+cu130
    Protocol
    native-forward-v1
    Dataset
    typed-invoices-v1
    Workload
    decision
    Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 50.25 20 ms — 3.17 161.3 1 df141700 ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	50.28	20 ms	—	2.29	165	1	8c1985ea ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	50.25	20 ms	—	3.17	161.3	1	df141700 ↗

Intern-Decision-0.8B

internlm · 4 tested configurations

bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
Throughput · Batch 1
50.8decisions/sec
Best measured · Batch 17
347decisions/sec
Observed GPU occupancy
2.4GiB
P95 latency
20 ms
Use this recipe · Batch 17 ↗Evidence report and run availability ↗Same test on other GPUs
Test details · 2 batch sizes
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
intern-decision-native · PyTorch 2.11.0+cu130
Protocol
native-forward-v1
Dataset
typed-invoices-v1
Workload
decision
Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 50.83 20 ms — 2.36 169 2 18b06f63 ↗
B17 346.72 50 ms — 3.07 378 1 6ccb367c ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	50.83	20 ms	—	2.36	169	2	18b06f63 ↗
B17	346.72	50 ms	—	3.07	378	1	6ccb367c ↗

3 other test configurations on this GPU

bf16 · Text → Structured data · typed-invoices-v1 · Test duration 396 s

Throughput · Batch 1: 52.8decisions/sec
Best measured · Batch 17: 350decisions/sec
Observed GPU occupancy: 2.4GiB
P95 latency: 19 ms

Use this recipe · Batch 17 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 21 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: intern-decision-native · PyTorch 2.11.0
Protocol: native-forward-v1
Dataset: typed-invoices-v1
Workload: decision

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	52.8	19 ms	—	2.36	172.8	1	5eef633e ↗
B2	95.72	21 ms	—	2.38	186.5	1	5eef633e ↗
B4	177.64	23 ms	—	2.43	242	1	5eef633e ↗
B8	306.42	26 ms	—	2.57	344.2	1	5eef633e ↗
B12	322.68	37 ms	—	37.46	371.9	1	5eef633e ↗
B13	336	39 ms	—	37.46	380.8	1	5eef633e ↗
B14	344.61	41 ms	—	37.46	392.6	1	5eef633e ↗
B15	345.75	44 ms	—	37.46	389.5	1	5eef633e ↗
B16	345.28	47 ms	—	2.85	383.9	1	5eef633e ↗
B17	349.83	49 ms	—	37.46	397.8	1	5eef633e ↗
B18	327.34	55 ms	—	37.46	383.4	1	5eef633e ↗
B20	333.21	60 ms	—	37.46	402.6	1	5eef633e ↗
B24	331.98	73 ms	—	37.46	401.2	1	5eef633e ↗
B28	327.01	86 ms	—	37.46	399.2	1	5eef633e ↗
B32	331.67	97 ms	—	3.41	397.7	1	5eef633e ↗
B48	326.46	148 ms	—	37.46	405.9	1	5eef633e ↗
B64	318.22	202 ms	—	4.51	403.2	1	5eef633e ↗
B128	298.77	430 ms	—	6.71	405.8	1	5eef633e ↗
B256	298.58	859 ms	—	11.11	407.3	1	5eef633e ↗
B512	299.67	1.71 s	—	19.9	411.2	1	5eef633e ↗
B1024	297.46	3.64 s	—	37.46	410.6	1	5eef633e ↗

bf16 · Text → Structured data · typed-invoices-v1 · Test duration 399 s

Throughput · Batch 1: 52decisions/sec
Best measured · Batch 16: 334decisions/sec
Observed GPU occupancy: 2.4GiB
P95 latency: 20 ms

Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 21 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: intern-decision-native · PyTorch 2.11.0
Protocol: native-forward-v1
Dataset: typed-invoices-v1
Workload: decision

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	52.04	20 ms	—	2.36	162.6	1	156910e7 ↗
B2	96.21	21 ms	—	2.38	213.4	1	156910e7 ↗
B4	177.08	23 ms	—	2.43	243.4	1	156910e7 ↗
B8	299.06	27 ms	—	2.57	341.2	1	156910e7 ↗
B12	306.24	40 ms	—	36.98	349.6	1	156910e7 ↗
B14	322.36	43 ms	—	36.98	365.9	1	156910e7 ↗
B15	328.14	46 ms	—	36.98	369.8	1	156910e7 ↗
B16	333.92	48 ms	—	3.19	368.4	1	156910e7 ↗
B17	331.9	52 ms	—	36.98	379.5	1	156910e7 ↗
B18	310.84	58 ms	—	36.98	365.4	1	156910e7 ↗
B20	316.13	64 ms	—	36.98	375.1	1	156910e7 ↗
B24	312.87	77 ms	—	36.98	377.5	1	156910e7 ↗
B28	309.61	91 ms	—	36.98	379.4	1	156910e7 ↗
B32	319.44	101 ms	—	3.44	376.3	1	156910e7 ↗
B40	307.24	131 ms	—	36.98	379.6	1	156910e7 ↗
B48	306.75	159 ms	—	36.98	380.4	1	156910e7 ↗
B64	304.06	213 ms	—	4.2	383.7	1	156910e7 ↗
B128	285.17	453 ms	—	6.23	387.8	1	156910e7 ↗
B256	284.61	904 ms	—	10.63	390.3	1	156910e7 ↗
B512	282.22	2.04 s	—	19.42	390.1	1	156910e7 ↗
B1024	286.53	3.58 s	—	36.98	396.2	1	156910e7 ↗

bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
Throughput · Batch 1
51.5decisions/sec
Observed GPU occupancy
2.6GiB
P95 latency
20 ms
Use this recipe · Batch 1 ↗Evidence report and run availability ↗Same test on other GPUs
Test details
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
intern-decision-native · PyTorch 2.11.0
Protocol
native-forward-v1
Dataset
typed-invoices-v1
Workload
decision
Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B1 51.51 20 ms — 2.6 164.2 1 7f2e07c0 ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	51.51	20 ms	—	2.6	164.2	1	7f2e07c0 ↗

clef-flash

Cloudflare · 2 tested configurations

bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
Throughput · Batch 16
47.7decisions/sec
Observed GPU occupancy
19.5GiB
P95 latency
338 ms
Use this recipe · Batch 16 ↗Evidence report and run availability ↗Same test on other GPUs
Test details
Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
Engine
clef-native · PyTorch 2.11.0+cu130
Protocol
native-forward-v1
Dataset
typed-invoices-v1
Workload
decision
Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
B16 47.71 338 ms — 19.55 499.5 2 82d98b36 ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B16	47.71	338 ms	—	19.55	499.5	2	82d98b36 ↗

1 other test configurations on this GPU

bf16 · Text → Structured data · typed-invoices-v1 · Test duration 33 s

Throughput · Batch 1: 31.8decisions/sec
Best measured · Batch 4: 39.3decisions/sec
Observed GPU occupancy: 18.9GiB
P95 latency: 32 ms

Use this recipe · Batch 4 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: clef-native · PyTorch 2.11.0
Protocol: native-forward-v1
Dataset: typed-invoices-v1
Workload: decision

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	31.83	32 ms	—	18.95	388.5	1	10c64aff ↗
B4	39.27	93 ms	—	18.95	449.5	1	10c64aff ↗
B16	5.41	8.21 s	—	19.28	233.4	1	10c64aff ↗
B64	15.43	9.66 s	—	21.89	324.3	1	10c64aff ↗

StartLux-Decision-v1.1-0.8B-Instant
startlux-models · 2 tested configurations
- bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
  Throughput · Batch 1
  18.7decisions/sec
  Observed GPU occupancy
  2.7GiB
  P95 latency
  54 ms
  Use this recipe · Batch 1 ↗Evidence report and run availability ↗Same test on other GPUs
  Test details
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  startlux-transformers · PyTorch 2.11.0+cu130
  Protocol
  native-forward-v1
  Dataset
  typed-invoices-v1
  Workload
  decision
  Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 18.69 54 ms — 2.73 134.5 2 a4a69815 ↗
- 1 other test configurations on this GPU
  - bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
    Throughput · Batch 1
    18.8decisions/sec
    Observed GPU occupancy
    2.5GiB
    P95 latency
    54 ms
    Use this recipe · Batch 1 ↗Evidence report and run availability ↗Same test on other GPUs
    Test details
    Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
    Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
    Engine
    startlux-transformers · PyTorch 2.11.0
    Protocol
    native-forward-v1
    Dataset
    typed-invoices-v1
    Workload
    decision
    Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
    B1 18.83 54 ms — 2.54 125.5 1 c342a9ca ↗
Intern-Decision-2B
internlm · 1 tested configuration
- bf16 · Text → Structured data · typed-invoices-v1 · Test duration 6 s
  Throughput · Batch 1
  50.3decisions/sec
  Observed GPU occupancy
  4.9GiB
  P95 latency
  21 ms
  Use this recipe · Batch 1 ↗Evidence report and run availability ↗Same test on other GPUs
  Test details
  Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.
  Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.
  Engine
  intern-decision-native · PyTorch 2.11.0
  Protocol
  native-forward-v1
  Dataset
  typed-invoices-v1
  Workload
  decision
  Batch size Mean throughput (decisions/sec) P95 TTFT Observed occupancy (GiB) GPU power (W) Reports combined View report
  B1 50.29 21 ms — 4.9 216.2 1 e4bcd6a3 ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	18.69	54 ms	—	2.73	134.5	2	a4a69815 ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	18.83	54 ms	—	2.54	125.5	1	c342a9ca ↗

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	50.29	21 ms	—	4.9	216.2	1	e4bcd6a3 ↗

clef-omni

Cloudflare · 1 tested configuration

bf16 · Text → Structured data · typed-invoices-v1 · Test duration 44 s

Throughput · Batch 1: 7.5decisions/sec
Best measured · Batch 64: 7.5decisions/sec
Observed GPU occupancy: 60.9GiB
P95 latency: 134 ms

Use this recipe · Batch 64 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: clef-native · PyTorch 2.11.0
Protocol: native-forward-v1
Dataset: typed-invoices-v1
Workload: decision

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	7.45	134 ms	—	60.88	232.8	1	16597f91 ↗
B4	7.47	537 ms	—	60.89	249.7	1	16597f91 ↗
B16	7.48	2.14 s	—	60.89	252.8	1	16597f91 ↗
B64	7.49	8.56 s	—	60.89	257.2	1	16597f91 ↗

clef

Cloudflare · 1 tested configuration

bf16 · Text → Structured data · typed-invoices-v1 · Test duration 46 s

Throughput · Batch 1: 12.3decisions/sec
Observed GPU occupancy: 52.4GiB
P95 latency: 82 ms

Use this recipe · Batch 1 ↗Evidence report and run availability ↗Same test on other GPUs

Test details · 4 batch sizes

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Reports combined counts reports, not inference requests. Request successes and failures are listed in each report.

Engine: clef-native · PyTorch 2.11.0
Protocol: native-forward-v1
Dataset: typed-invoices-v1
Workload: decision

Batch size	Mean throughput (decisions/sec)	P95	TTFT	Observed occupancy (GiB)	GPU power (W)	Reports combined	View report
B1	12.32	82 ms	—	52.4	469.1	1	243b7f64 ↗
B4	1.4	8.09 s	—	52.4	225.7	1	243b7f64 ↗
B16	4.33	9.1 s	—	52.98	308.5	1	243b7f64 ↗
B64	9.39	12.46 s	—	57.95	435.9	1	243b7f64 ↗

Occupancy includes engine buffers and reserved KV cache. It is not the minimum VRAM required by the model.

Sortable tables by exact test condition · 63 rows

C: concurrency · B: batch · —: not measured

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
startlux-models/StartLux-Decision-v1-0.8BScaling curves ↗	bf16	startlux-native	decisionnative-forward-v1	B1	50.28	50.28	—	0.02	2.29	165	50.28	0.3	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B1	50.83	50.86	—	0.02	2.36	169	50.8	0.3	2 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
startlux-models/StartLux-Decision-v1-0.8BScaling curves ↗	bf16	startlux-native	decisionnative-forward-v1	B1	50.25	50.25	—	0.02	3.17	161.3	50.25	0.31	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
startlux-models/StartLux-Decision-v1.1-0.8B-InstantScaling curves ↗	bf16	startlux-transformers	decisionnative-forward-v1	B1	18.69	18.77	—	0.054	2.73	134.5	18.61	0.14	2 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-2BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B1	50.29	50.29	—	0.021	4.9	216.2	50.29	0.23	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
startlux-models/StartLux-Decision-v1.1-0.8B-InstantScaling curves ↗	bf16	startlux-transformers	decisionnative-forward-v1	B1	18.83	18.83	—	0.054	2.54	125.5	18.83	0.15	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B1	51.51	51.51	—	0.02	2.6	164.2	51.51	0.31	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 16 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clef-flashScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B16	47.71	47.74	—	0.338	19.55	499.5	47.67	0.1	2 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 6 s · Batch 17 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B17	346.72	346.72	—	0.05	3.07	378	346.72	0.92	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 33 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clef-flashScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B1	31.83	31.83	—	0.032	18.95	388.5	31.83	0.08	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 33 s · Batch 4 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clef-flashScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B4	39.27	39.27	—	0.093	18.95	449.5	39.27	0.09	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 33 s · Batch 16 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clef-flashScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B16	5.41	5.41	—	8.208	19.28	233.4	5.41	0.02	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 33 s · Batch 64 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clef-flashScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B64	15.43	15.43	—	9.663	21.89	324.3	15.43	0.05	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 44 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clef-omniScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B1	7.45	7.45	—	0.134	60.88	232.8	7.45	0.03	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 44 s · Batch 4 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clef-omniScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B4	7.47	7.47	—	0.537	60.89	249.7	7.47	0.03	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 44 s · Batch 16 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clef-omniScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B16	7.48	7.48	—	2.139	60.89	252.8	7.48	0.03	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 44 s · Batch 64 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clef-omniScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B64	7.49	7.49	—	8.557	60.89	257.2	7.49	0.03	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 46 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clefScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B1	12.32	12.32	—	0.082	52.4	469.1	12.32	0.03	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 46 s · Batch 4 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clefScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B4	1.4	1.4	—	8.086	52.4	225.7	1.4	0.01	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 46 s · Batch 16 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clefScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B16	4.33	4.33	—	9.102	52.98	308.5	4.33	0.01	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 46 s · Batch 64 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
Cloudflare/clefScaling curves ↗	bf16	clef-native	decisionnative-forward-v1	B64	9.39	9.39	—	12.463	57.95	435.9	9.39	0.02	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B1	52.8	52.8	—	0.019	2.36	172.8	52.8	0.31	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 2 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B2	95.72	95.72	—	0.021	2.38	186.5	95.72	0.51	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 4 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B4	177.64	177.64	—	0.023	2.43	242	177.64	0.73	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 8 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B8	306.42	306.42	—	0.026	2.57	344.2	306.42	0.89	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 12 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B12	322.68	322.68	—	0.037	37.46	371.9	322.68	0.87	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 13 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B13	336	336	—	0.039	37.46	380.8	336	0.88	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 14 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B14	344.61	344.61	—	0.041	37.46	392.6	344.61	0.88	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 15 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B15	345.75	345.75	—	0.044	37.46	389.5	345.75	0.89	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 16 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B16	345.28	345.28	—	0.047	2.85	383.9	345.28	0.9	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 17 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B17	349.83	349.83	—	0.049	37.46	397.8	349.83	0.88	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 18 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B18	327.34	327.34	—	0.055	37.46	383.4	327.34	0.85	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 20 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B20	333.21	333.21	—	0.06	37.46	402.6	333.21	0.83	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 24 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B24	331.98	331.98	—	0.073	37.46	401.2	331.98	0.83	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 28 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B28	327.01	327.01	—	0.086	37.46	399.2	327.01	0.82	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 32 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B32	331.67	331.67	—	0.097	3.41	397.7	331.67	0.83	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 48 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B48	326.46	326.46	—	0.148	37.46	405.9	326.46	0.8	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 64 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B64	318.22	318.22	—	0.202	4.51	403.2	318.22	0.79	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 128 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B128	298.77	298.77	—	0.43	6.71	405.8	298.77	0.74	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 256 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B256	298.58	298.58	—	0.859	11.11	407.3	298.58	0.73	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 512 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B512	299.67	299.67	—	1.713	19.9	411.2	299.67	0.73	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 396 s · Batch 1024 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B1024	297.46	297.46	—	3.64	37.46	410.6	297.46	0.72	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 1 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B1	52.04	52.04	—	0.02	2.36	162.6	52.04	0.32	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 2 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B2	96.21	96.21	—	0.021	2.38	213.4	96.21	0.45	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 4 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B4	177.08	177.08	—	0.023	2.43	243.4	177.08	0.73	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 8 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B8	299.06	299.06	—	0.027	2.57	341.2	299.06	0.88	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 12 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B12	306.24	306.24	—	0.04	36.98	349.6	306.24	0.88	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 14 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B14	322.36	322.36	—	0.043	36.98	365.9	322.36	0.88	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 15 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B15	328.14	328.14	—	0.046	36.98	369.8	328.14	0.89	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 16 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B16	333.92	333.92	—	0.048	3.19	368.4	333.92	0.91	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 17 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B17	331.9	331.9	—	0.052	36.98	379.5	331.9	0.87	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 18 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B18	310.84	310.84	—	0.058	36.98	365.4	310.84	0.85	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 20 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B20	316.13	316.13	—	0.064	36.98	375.1	316.13	0.84	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 24 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B24	312.87	312.87	—	0.077	36.98	377.5	312.87	0.83	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 28 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B28	309.61	309.61	—	0.091	36.98	379.4	309.61	0.82	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 32 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B32	319.44	319.44	—	0.101	3.44	376.3	319.44	0.85	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 40 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B40	307.24	307.24	—	0.131	36.98	379.6	307.24	0.81	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 48 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B48	306.75	306.75	—	0.159	36.98	380.4	306.75	0.81	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 64 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B64	304.06	304.06	—	0.213	4.2	383.7	304.06	0.79	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 128 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B128	285.17	285.17	—	0.453	6.23	387.8	285.17	0.74	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 256 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B256	284.61	284.61	—	0.904	10.63	390.3	284.61	0.73	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 512 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B512	282.22	282.22	—	2.039	19.42	390.1	282.22	0.72	1 ↗

Decision · Text → Structured data · typed-invoices-v1 · Test duration 399 s · Batch 1024 · decisions/sec · 1 rows
Model	Format	Engine	Workload	Concurrency / batch	Mean throughput	Best measured	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	GPU power (W)	Lowest measured	Throughput / W	Reports combined
internlm/Intern-Decision-0.8BScaling curves ↗	bf16	intern-decision-native	decisionnative-forward-v1	B1024	286.53	286.53	—	3.583	36.98	396.2	286.53	0.72	1 ↗

Scaling curves and individual runs

One model and test protocol per curve; different workloads are never joined into one line.

Benchmark configuration

internlm/Intern-Decision-0.8B · bf16 · intern-decision-native · decision · native-forward-v1

Mean throughput decisions/sec

Measured points only. Select a point to open its source report.

Mean run P95 latency s

Measured points only. Select a point to open its source report.

Mean GPU memory used GB

Measured points only. Select a point to open its source report.

Each point is the mean of recorded runs at that level. A source report may differ from the mean; all runs are listed below.

Runs behind the curve · internlm/Intern-Decision-0.8B · decisions/sec · 21 rows
Batch size	Throughput	Decode (tok/s)	Uncached prefill (tok/s)	GPU power (W)	TTFT (ms)	P95 (s)	Observed occupancy (GiB)	Test date	View report
1	52.04 decisions/sec	—	Uncached prefill unavailableView report	162.58	—	0.02	2.36	10/23/2026, 10:02:56 AM	156910e7 ↗
2	96.21 decisions/sec	—	Uncached prefill unavailableView report	213.36	—	0.021	2.38	10/23/2026, 10:02:56 AM	156910e7 ↗
4	177.08 decisions/sec	—	Uncached prefill unavailableView report	243.38	—	0.023	2.43	10/23/2026, 10:02:56 AM	156910e7 ↗
8	299.06 decisions/sec	—	Uncached prefill unavailableView report	341.16	—	0.027	2.57	10/23/2026, 10:02:56 AM	156910e7 ↗
12	306.24 decisions/sec	—	Uncached prefill unavailableView report	349.6	—	0.04	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
14	322.36 decisions/sec	—	Uncached prefill unavailableView report	365.85	—	0.043	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
15	328.14 decisions/sec	—	Uncached prefill unavailableView report	369.77	—	0.046	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
16	333.92 decisions/sec	—	Uncached prefill unavailableView report	368.39	—	0.048	3.19	10/23/2026, 10:02:56 AM	156910e7 ↗
17	331.9 decisions/sec	—	Uncached prefill unavailableView report	379.47	—	0.052	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
18	310.84 decisions/sec	—	Uncached prefill unavailableView report	365.42	—	0.058	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
20	316.13 decisions/sec	—	Uncached prefill unavailableView report	375.07	—	0.064	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
24	312.87 decisions/sec	—	Uncached prefill unavailableView report	377.48	—	0.077	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
28	309.61 decisions/sec	—	Uncached prefill unavailableView report	379.37	—	0.091	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
32	319.44 decisions/sec	—	Uncached prefill unavailableView report	376.34	—	0.101	3.44	10/23/2026, 10:02:56 AM	156910e7 ↗
40	307.24 decisions/sec	—	Uncached prefill unavailableView report	379.55	—	0.131	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
48	306.75 decisions/sec	—	Uncached prefill unavailableView report	380.44	—	0.159	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗
64	304.06 decisions/sec	—	Uncached prefill unavailableView report	383.68	—	0.213	4.2	10/23/2026, 10:02:56 AM	156910e7 ↗
128	285.17 decisions/sec	—	Uncached prefill unavailableView report	387.85	—	0.453	6.23	10/23/2026, 10:02:56 AM	156910e7 ↗
256	284.61 decisions/sec	—	Uncached prefill unavailableView report	390.31	—	0.904	10.63	10/23/2026, 10:02:56 AM	156910e7 ↗
512	282.22 decisions/sec	—	Uncached prefill unavailableView report	390.09	—	2.039	19.42	10/23/2026, 10:02:56 AM	156910e7 ↗
1024	286.53 decisions/sec	—	Uncached prefill unavailableView report	396.25	—	3.583	36.98	10/23/2026, 10:02:56 AM	156910e7 ↗

Original reports

Every report for this task, including quality, diagnostic and legacy runs that do not enter comparisons.

Original reports · 19 rows
Model	Test task	Format	Engine	Workload	Protocol	Quality score	Test date
clef ↗	Decision	bf16	clef-native	decision	native-forward-v1	—	10/23/2026, 8:57:12 AM
clef-flash ↗	Decision	bf16	clef-native	decision	native-forward-v1	—	10/23/2026, 12:06:35 PM
clef-flash ↗	Decision	bf16	clef-native	decision	native-forward-v1	—	10/23/2026, 12:02:11 PM
clef-flash ↗	Decision	bf16	clef-native	decision	native-forward-v1	—	10/23/2026, 9:04:07 AM
clef-flash ↗	Decision	bf16	clef-native	decision	native-forward-v1	—	10/14/2026, 8:26:33 AM
clef-flash ↗	Decision	bf16	clef-native	decision	native-forward-v1	—	10/14/2026, 6:35:30 AM
clef-omni ↗	Decision	bf16	clef-native	decision	native-forward-v1	—	10/23/2026, 9:01:55 AM
Intern-Decision-0.8B ↗	Decision	bf16	intern-decision-native	decision	native-forward-v1	—	10/23/2026, 12:07:54 PM
Intern-Decision-0.8B ↗	Decision	bf16	intern-decision-native	decision	native-forward-v1	—	10/23/2026, 12:06:48 PM
Intern-Decision-0.8B ↗	Decision	bf16	intern-decision-native	decision	native-forward-v1	—	10/23/2026, 12:00:13 PM
Intern-Decision-0.8B ↗	Decision	bf16	intern-decision-native	decision	native-forward-v1	—	10/23/2026, 10:11:05 AM
Intern-Decision-0.8B ↗	Decision	bf16	intern-decision-native	decision	native-forward-v1	—	10/23/2026, 10:02:56 AM
Intern-Decision-0.8B ↗	Decision	bf16	intern-decision-native	decision	native-forward-v1	—	10/23/2026, 9:23:01 AM
Intern-Decision-2B ↗	Decision	bf16	intern-decision-native	decision	native-forward-v1	—	10/23/2026, 9:24:25 AM
StartLux-Decision-v1-0.8B ↗	Decision	bf16	startlux-native	decision	native-forward-v1	—	10/23/2026, 12:09:03 PM
StartLux-Decision-v1-0.8B ↗	Decision	bf16	startlux-native	decision	native-forward-v1	—	10/23/2026, 12:06:36 PM
StartLux-Decision-v1.1-0.8B-Instant ↗	Decision	bf16	startlux-transformers	decision	native-forward-v1	—	10/23/2026, 12:06:09 PM
StartLux-Decision-v1.1-0.8B-Instant ↗	Decision	bf16	startlux-transformers	decision	native-forward-v1	—	10/23/2026, 12:04:32 PM
StartLux-Decision-v1.1-0.8B-Instant ↗	Decision	bf16	startlux-transformers	decision	native-forward-v1	—	10/23/2026, 9:23:32 AM

Running benchmarks

0Updates automatically

No benchmarks are currently reported as running.

RTX PRO 6000 Blackwell Server Edition · 96 GB · 403 queued · 0 running · 0 awaiting reassignment · 6 succeeded · 3 failed or exhausted

Methodology

Average of error-free runs for the selected recorded configuration. Missing methods and fallback runs are not ranked. Performance index = average / fastest matching hardware × 100. Value index = performance per purchase cost / best comparable value × 100. Quality is separate by dataset and protocol. Prices are dated seller listings, not completed transactions or market averages.

Updated 11/5/2026, 11:13:32 PM

Decision · Qualified results

StartLux-Decision-v1-0.8B

Intern-Decision-0.8B

clef-flash

StartLux-Decision-v1.1-0.8B-Instant

Intern-Decision-2B

clef-omni

clef

Original reports

Running benchmarks