RTX PRO 6000 Blackwell Server Edition · 96 GB
Intern-Decision-0.8B
Open a report for complete measurements.
- Task
- Decision · Text → Structured data
- Workload
- Text → Structured data · typed-invoices-v1 · Test duration 6 s
- Batch size
- Batch 1
- Format · Engine
- bf16 · intern-decision-native · PyTorch 2.11.0+cu130
- Unit
- decisions/sec
- Mean throughput
- 50.8decisions/sec
- Observed GPU occupancy
- 2.4 GiB
- P95 latency
- 20 ms
- Reports combined
- 2
Launch this model
Open the decision recipe to start its API or repeat the measured batch.
Use this recipeIndividual tests
Text → Structured data · Decision · bf16 · intern-decision-native · decision · decisions/sec
Sortable table
| Test date | Throughput | Concurrency / batch | P95 (s) | TTFT (ms) | Observed occupancy (GiB) | GPU power (W) | Decode (tok/s) | Uncached prefill (tok/s) | Parameters | View report |
|---|---|---|---|---|---|---|---|---|---|---|
| 10/23/2026, 12:06:48 PM | 50.8 decisions/sec | 1 | 0.02 | — | 2.36 | 166.93 | — | Uncached prefill unavailableView report | Parameters ↗ | d7e888be ↗ |
| 10/23/2026, 12:07:54 PM | 50.86 decisions/sec | 1 | 0.02 | — | 2.36 | 170.99 | — | Uncached prefill unavailableView report | Parameters ↗ | 18b06f63 ↗ |
Recorded test parameters
Configuration 1 · Parameters
- model
- internlm/Intern-Decision-0.8B
- engine
- intern-decision-native
- quant
- bf16
- method
- native-forward-v1
- task
- decision
- dataset
- typed-invoices-v1
- context
- decision
- input_tokens
- 0
- output_tokens
- 0
- duration
- 6
- concurrency
- 1
- unit
- decisions/sec
- software
- { "torch": "2.11.0+cu130", "torchvision": "0.26.0+cu130", "transformers": "5.14.1", "tokenizers": "0.22.2", "causal-conv1d": "1.7.0", "flash-linear-attention": "0.5.2", "fla-core": "0.5.2", "accelerate": "1.15.0", "peft": "0.21.2", "huggingface-hub": "1.33.0", "pillow": "13.0.0", "numpy": "2.5.4", "requests": "2.34.2" }
- engine_provenance
- { "native_batch_adapter": "intern-marker-batch-v1", "padding": "right", "logit_positions": "union with per-request marker projection", "allow_bf16_reduced_precision_reduction": false, "attention_implementation": "sdpa", "use_cache": false, "max_length": 1024, "answer_symbol_encoding": "once per batch" }
- kernel_settings
- —
- precision
- bf16
- resolution
- —
- performance_task
- —
- performance_scope
- Structured text request, tokenization, GPU inference and typed probabilities; batch-specific warmup excluded; no image or audio inputs
- batch_size
- 1
- dimensions
- —
- weight_quantization
- —
- max_sequence_length
- —
- model_scope
- typed-decision-marker-readout
- performance_dataset
- —
- workload_mode
- single
- graph_mode
- —
- model_revision
- 42457fafd13d10d0b0d9be46cb30a23caa409994
- client_version
- python-native-bench-1.1.3