# Concurrency throughput

The plot uses the best tested native draft length at each concurrency: 32 at
1–4, 24 at 8–32, and 16 at 64–512. Each plotted choice has three full-set trials.
The server uses a fixed draft length.

Each point is the median of three full 2,078-page trials on one H200/BF16.
Shading shows observed min/max, not confidence intervals. Base, MTP and DFlash2 use the current matched concurrency and draft-length sweep.
The plot contains throughput only, with no per-point draft-length annotations.

Native closed-loop single-pass decoding, greedy temperature 0/top-k 1,
repetition penalty 1.1, 8,744-token cap, TP1, engine capacity 512, maximum context
16,384, batched-token budget 32,768, GPU-memory utilization 0.8, prefix caching
off, processor cache 0, async scheduling and chunked prefill on, FA3, compilation
off and full-decode CUDA graphs. Timing includes preprocessing, prefill, decode,
refill and drain; it excludes startup, image loading, warmup and serialization.
There is no HTTP. Token-limited pages remain visible in the downloadable data.
Do not combine these rates with the HTTP multipass pipeline measurements.

[JSON data](results.json) contains settings, ranges and source identities;
[CSV](results.csv) contains plotted values. The accompanying pass timings and
quality scores use their own explicitly documented protocols.