transcriber-app/parakeet-tdt-0.6b-v2-mlx-4bit

parakeet-tdt-0.6b-v2 — MLX 4-bit

NVIDIA Parakeet TDT 0.6B v2, in the MLX layout of mlx-community/parakeet-tdt-0.6b-v2, quantized to 4-bit for on-device speech-to-text on Apple silicon. English only.

What changed from the source

This is a redistribution of a modified model. Starting from the fp32 weights of mlx-community/parakeet-tdt-0.6b-v2:

• every Linear and Embedding layer (attention and feed-forward projections, the pre-encode and joint projections, the prediction-network embedding — 221 modules) is quantized with mlx.nn.quantize to 4-bit, group size 64 (affine);
• every other tensor (convolutions, LSTM, normalization) is stored as bfloat16;
• config.json gains "quantization": {"group_size": 64, "bits": 4}.

Tensor names are unchanged from the source checkpoint, so loaders that read the mlx-community layout and honour the quantization block (quantizing the modules whose weights arrive with .scales) load it directly.

Files

• model.safetensors — quantized weights (467 MB, vs 2.5 GB fp32)
• config.json — the NeMo config with the quantization block
• vocab.txt, tokenizer.model, tokenizer.vocab — unchanged from the source

Accuracy

Checked against the fp32 build on English speech: transcripts matched apart from one word's spacing. No WER benchmark has been run on this build.

Credits and licence

• Model: NVIDIA, parakeet-tdt-0.6b-v2 — CC-BY-4.0.
• MLX conversion: mlx-community (via parakeet-mlx) — CC-BY-4.0.
• Quantization: this repository, released under the same CC-BY-4.0 licence.

Other builds: 8-bit, 6-bit.