Bina 0.4 Koochik — Tract-INT4
Calibration-aware 4-bit derivative of Reza2kn/Bina-0.4-Koochik, using checkpoint 18,404. The architecture and 630,800,496 unique parameters are unchanged. Original training was not modified.
170 linear weight matrices use symmetric GPTQ codes with group size 32 and FP32 group scales, calibrated on 64 approved training-only regions with fresh 18,404 source sequences. Tied input/output embeddings, convolutions, positions, norms, biases and unused auxiliary heads keep higher precision. “4-bit” describes those linear codes, not every stored parameter or activation. The canonical packed candidate is 533,891,055 bytes, about 6.771 effective bits per original parameter including higher precision exceptions and serialization; this runtime package has a different footprint, including separate components and any duplicated tables. No native BitNet or universal speed claim is made.
Validation
Actual target-runtime results on the frozen 64-image / 14,127-source-character qualification set, using the included pixel-layout-v2 pipeline:
| Runtime | Corrected source character parity | Original whole-page parity | New caps / repetition flags |
|---|---|---|---|
| Tract CPU | 98.8745% | 97.3778% | 0 / 0 |
| Tract auto → Metal | 98.9170% | 97.4206% | 0 / 0 |
Character agreement is 1 - pooled NFC Levenshtein edits / pooled NFC source characters, unclipped. The corrected source is the immutable uncompressed BF16 cast of the original checkpoint, run on the same pixel regions with source decoding. Original whole-page results are reported separately. These are source agreement measurements, not human-annotated OCR accuracy, per-image guarantees, or an independent unseen OCR benchmark. The final set was used for runtime qualification, not quant fitting or candidate selection. Calibration and final source groups are disjoint.
Greedy decoding uses repetition penalty 1.1, at most 2,048 tokens per region and the source's at-most-one resolution retry. No passage-stopping guard, no-repeat constraint, output deduplication or hidden truncation improves these scores. Token caps and repetition are audited. The pixel-only layout covers all original pixels once. Small decorative text, logos, handwriting and complex layouts can still differ. Markup mode is available but was not separately corpus-qualified.
Tested with tract 0.23.9-pre CPU and the included repaired Metal runtime on an Apple M2 Mac. Included binaries are macOS ARM64. Other operating systems, CPUs and GPUs require a compatible native build and their own validation. The Metal softmax patch fixes masked all-negative-infinity vector handling; it is included with the runtime licenses. This is a custom Bina pipeline, not a sherpa-onnx adapter.
Use
The complete runtime download is approximately 1,086.7 MB (decimal), including higher precision exceptions, split components, tokenizer/position assets and runtime files. INT4 does not mean every parameter or activation is four bits.
Download this complete repo, including external model data, then run:
python -m pip install -r requirements.txt
chmod +x bin/bina-tract-runtime bin/bina-tract-metal-runtime-fixed
python run_ocr.py --format tract --device auto --image examples/demo.png --output result.json
--device auto is the default: on compatible macOS ARM64 it launches the pinned Metal runtime, checks its identity and runs deterministic synthetic vision, neck, prefill and two cached-decode probes. If Metal is unavailable or a capability check fails, it uses CPU. The automatic Metal run completed all 64 images / 71 regions without CPU fallback. The bundled CPU executable is also macOS ARM64; other operating systems still need compatible binaries built from the included source.
Pass --device cpu to force CPU or --device metal / --device gpu for strict GPU execution. The legacy --format tract-metal also remains strict Metal. For default auto, failed GPU initialization, interrupted decoding, nonfinite logits or a timed-out GPU transaction cause a fresh CPU process to restart the entire current region with new prefill/KV; partial GPU output is never concatenated with CPU output. The runner logs the reason and failed partial token IDs and stays on CPU for subsequent regions. Binary/probe integrity mismatches fail closed.
Four components stay loaded for the lifetime of a model instance, with a decoder KV cache per region. Startup probes add cold-start work; reuse the model instance across requests. result.json records the actual backend and fallback events, and selects the corpus-validation record for that backend. It does not assert parity for the new input. See validation/automatic-backends.json for the automatic GPU qualification and recovery checks.
The demo is newly authored synthetic text. Its independent-folder installation smoke reached natural EOS and transcribed both lines exactly. The private user letter, training examples and protected qualification images/transcriptions are not included. validation/character-parity.json contains aggregate evidence. artifact-manifest.json lists release hashes.
Source and license
The standalone parent contains the original FP32 checkpoint weights and complete upstream provenance. This derivative inherits OpenMDW 1.1; preserve LICENSE and third-party notices. See quantization.json for source/candidate hashes. Parent architecture: NVIDIA Nemotron-3.5-Parse. The binary/runtime versions and arithmetic profiles are part of the qualification; changing them requires another check.
Local latency diagnostic
On the same newly authored 400×104 PNG (34 generated tokens) on an Apple M2 Mac, the supplied loader settings produced warm median end-to-end times of tract: 2.653 s, tract-metal: 0.725 s across three real repeated generations. Only one inference process was active; normal desktop apps and the idle resident browser could remain present. These historical manual-loader measurements predate the automatic startup probes and are not current automatic cold-start timings, corpus throughput, mobile measurements or a universal speed claim. See validation/latency-demo-*.json.