transcriber-app/parakeet-tdt-0.6b-v3-mlx-4bit

parakeet-tdt-0.6b-v3 — MLX 4-bit

NVIDIA Parakeet TDT 0.6B v3, in the MLX layout of mlx-community/parakeet-tdt-0.6b-v3, quantized to 4-bit for on-device speech-to-text on Apple silicon. Multilingual: 25 European languages.

What changed from the source

This is a redistribution of a modified model. Starting from the fp32 weights of mlx-community/parakeet-tdt-0.6b-v3:

• every Linear and Embedding layer (attention and feed-forward projections, the pre-encode and joint projections, the prediction-network embedding — 221 modules) is quantized with mlx.nn.quantize to 4-bit, group size 64 (affine);
• every other tensor (convolutions, LSTM, normalization) is stored as bfloat16;
• config.json gains "quantization": {"group_size": 64, "bits": 4}.

Tensor names are unchanged from the source checkpoint, so loaders that read the mlx-community layout and honour the quantization block (quantizing the modules whose weights arrive with .scales) load it directly.

Files

• model.safetensors — quantized weights (472 MB, vs 2.5 GB fp32)
• config.json — the NeMo config with the quantization block
• vocab.txt, tokenizer.model, tokenizer.vocab — unchanged from the source

Accuracy

Measured on 1,000 Russian clips (500 FLEURS, 500 GOLOS), greedy decode, on an M4 Mac:

build	download	FLEURS WER	GOLOS WER	speed
fp32 (mlx-community/parakeet-tdt-0.6b-v3)	2.5 GB	5.57 %	3.42 %	64×
8-bit (transcriber-app/parakeet-tdt-0.6b-v3-mlx-8bit)	744 MB	5.67 %	3.46 %	90×
6-bit (transcriber-app/parakeet-tdt-0.6b-v3-mlx-6bit)	608 MB	5.61 %	3.38 %	88×
5-bit (not published)	540 MB	5.93 %	3.50 %	87×
4-bit (transcriber-app/parakeet-tdt-0.6b-v3-mlx-4bit)	472 MB	5.95 %	3.67 %	90×

4-bit costs about a third of a WER point against fp32. 6-bit is the smallest build that keeps fp32 accuracy.

Credits and licence

• Model: NVIDIA, parakeet-tdt-0.6b-v3 — CC-BY-4.0.
• MLX conversion: mlx-community (via parakeet-mlx) — CC-BY-4.0.
• Quantization: this repository, released under the same CC-BY-4.0 licence.

Other builds: 8-bit, 6-bit.