MiLMMT-46-1B-v0.1 — MLX 6-bit

Apple MLX weights for xiaomi-research/MiLMMT-46-1B-v0.1, Xiaomi's 46-language translation model (a translation fine-tune of Gemma 3 1B).

Quantization 6-bit, --q-bits 6 (6.501 bits per weight as reported by the converter)
Size on disk 1.0 GB
Runtime mlx-lm / mlx-swift-lm
model_type gemma3_text

This is the MLX build to use. The 4-bit sibling is 269 MB smaller but visibly degraded — see below.

Why 6-bit and not 4-bit

Naive 4-bit post-training quantization measurably damages this model. Every regression it introduces disappears at 6 bits, with no calibration step of any kind. Both builds were run on the same 16-sentence JA↔EN set, on the same machine, with a byte-identical prompt and greedy decoding, so the difference is attributable to the quantization rather than to the harness:

Source 4-bit 6-bit
来年度の予算編成 "fiscal 2022" — hallucinated year "next fiscal year's budget"
のぞみ1号 (JA→EN) "the Shinkansen" — train name lost "Nozomi 1"
"I hope this message finds you well" 「今週のご挨拶」— nonsense 「…心よりお祈り申し上げます」
"Nozomi No. 1" (EN→JA) 「ノズミ1号」— misspelt 「ノゾミ1号」

Small models are disproportionately sensitive to naive PTQ. Calibrated quantization (mlx_lm.dwq, mlx_lm.awq) might reach this quality at 4-bit size, but was not needed to fix the regressions and is untried here.

Versus the GGUF

At equal size, this build is competitive with penta2himajin/MiLMMT-46-1B-v0.1-Q4_K_M-GGUF on quality and throughput, and loses on cold load — MLX has no mmap equivalent, so it pays for the whole file on every load. Timings are from a battery-powered run; read them as ratios, not absolutes.

MLX 6-bit GGUF Q4_K_M
Disk 1.0 GB 1.01 GB
Cold load 1546 ms 366 ms
p50 latency 251 ms 248 ms
Throughput 204 chars/s 215 chars/s

On quality the two trade places sentence by sentence. One example where this build is the better of the two: "The cabinet approved the basic policy…" renders as 「内閣は…承認した」here, against the GGUF's ungrammatical 「閣議決定は…承認した」.

Pick MLX if you are already on an MLX stack; pick the GGUF if cold load matters (an app that evicts weights between uses, for instance).

Prompt format

MiLMMT ships no chat template. Do not apply one — prompt it raw, exactly as the upstream model card specifies:

Translate this from <source language name> to <target language name>:
<source language name>: <source sentence>
<target language name>:

Use the language names from the upstream card's supported-language list (e.g. Japanese, English). Recommended decoding: top_k=1, temperature=0.

pip install mlx-lm
PROMPT=$'Translate this from Japanese to English:\nJapanese: こんにちは、今日も一日頑張りましょう。\nEnglish:'
mlx_lm.generate --model penta2himajin/MiLMMT-46-1B-v0.1-6bit \
  --prompt "$PROMPT" --max-tokens 128 --temp 0.0

Because there is no turn marker, generation can run on into a second translation block. Stop on the string Translate this from, or trim at it.

Conversion

mlx_lm.convert --hf-path xiaomi-research/MiLMMT-46-1B-v0.1 \
  -q --q-bits 6 --mlx-path MiLMMT-46-1B-v0.1-6bit

Converted with mlx-lm 0.31.3.

License

Gemma Terms of Use, inherited from the base model — not Apache 2.0. MiLMMT-46-1B is derived from Gemma 3, so redistribution and use are governed by the Gemma Terms and the Gemma Prohibited Use Policy, which pass through to anyone who obtains these weights.

Citation

@misc{shang2026scalingmodeldatamultilingual,
      title={Scaling Model and Data for Multilingual Machine Translation with Open Large Language Models},
      author={Yuzhe Shang and Pengzhi Gao and Wei Liu and Jian Luan and Jinsong Su},
      year={2026},
      eprint={2602.11961},
      archivePrefix={arXiv},
      url={https://arxiv.org/abs/2602.11961},
}
Downloads last month
12
Safetensors
Model size
1B params
Tensor type
U32
·
BF16
·
MLX
Hardware compatibility
Log In to add your hardware

6-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for penta2himajin/MiLMMT-46-1B-v0.1-6bit

Paper for penta2himajin/MiLMMT-46-1B-v0.1-6bit