transcriber-app/whisper-large-v3-turbo-mlx-8bit

whisper-large-v3-turbo — MLX 8-bit

Whisper large-v3-turbo in the mlx-whisper layout, quantized to 8-bit for on-device speech-to-text on Apple silicon.

How it was made

From mlx-community/whisper-large-v3-turbo (fp16): every Linear and Embedding layer (233 modules) quantized with mlx.nn.quantize — affine, 8-bit, group size 64 — the recipe of mlx-examples' whisper/convert.py. config.json is the mlx-whisper "dims" config with a "quantization": {"group_size": 64, "bits": 8} block.

Files

• weights.safetensors — quantized weights (864 MB)
• config.json

The tokenizer is not included: use the standard openai/whisper-large-v3 tokenizer.

Why MLX

It loads straight onto the Metal GPU with no Neural Engine compile — about half a second from disk, where a Core ML Whisper spends minutes specializing on first launch and after every app install or update.

Accuracy

Measured on 1,000 Russian clips (500 FLEURS, 500 GOLOS), language auto-detected, greedy decode, on an M4 Mac:

build	download	FLEURS WER	GOLOS WER	speed
8-bit (transcriber-app/whisper-large-v3-turbo-mlx-8bit)	864 MB	4.60 %	17.64 %	6.7×
6-bit (not published)	664 MB	4.70 %	18.10 %	6.8×
5-bit (transcriber-app/whisper-large-v3-turbo-mlx-5bit)	564 MB	4.71 %	17.64 %	7.0×
4-bit (not published)	463 MB	5.23 %	17.39 %	6.9×
Core ML, Argmax's 4-bit palettized build (for reference)	633 MB	4.85 %	18.80 %	11.8×

8-bit is the most accurate build. The 5-bit build gives up about a tenth of a FLEURS point for a third less download; below 5 bits accuracy drops clearly.

Licence

MIT, as the original Whisper weights. Other build: 5-bit.