hudabey/Kimi-K3-GGUF

Kimi K3 — GGUF, native MXFP4 preserved byte-exactly

The released moonshotai/Kimi-K3 checkpoint converted to GGUF by theseus convert — not requantized, provably.

Kimi K3 ships its 2.72T routed-expert parameters in native MXFP4 (the model was quantization-aware-trained at this precision — there is no higher-precision release). Conversion to ggml's block_mxfp4 is a pure byte permutation: nibble re-pairing plus scale-byte relocation, no value is ever decoded or re-encoded. Every expert weight in these files is bit-identical to Moonshot's release. The BF16 skeleton (attention, shared experts, latent sandwich, router, norms, embeddings) is carried at source precision.

Verify it yourself

verification/ contains the script and the run log: it samples random (layer, expert, projection) triples, reads the MXFP4 pair straight from the original safetensors, repacks with the oracle (theseus.repack_mxfp4 — round-trip and dequant-equivalence tested), and requires byte equality with these files plus bitwise-equal float32 through both formats' own dequantization rules. SHA256SUMS covers all shards.

Contents

• 34 shards, ~1.56 TB total (≤45 GB each): Kimi-K3-896x60B-MXFP4_MOE-*.gguf
• architecture kimi-k3: 93 layers — 69 KDA (linear attention) + 24 gated MLA (every 4th layer, plus the final layer), fully NoPE, 1M context
• Stable LatentMoE: 896 routed experts/layer in a 3584-dim latent, 16 active, 2 shared experts; AttnRes cross-depth residuals (block size 12); SiTU activation (β=4, β_lin=25) — all carried as first-class tensors/metadata
• generation EOS is <|end_of_msg|> (163586), per generation_config.json — conversions that copy the tokenizer's [EOS] (163585) never stop in chat

Runtime status

These files are the stable contract for the kimi-k3 architecture (metadata keys documented in the theseus repo, theseus/_vendor/PATCHES.md). Runtime support is being built in theseus; mainline llama.cpp does not run kimi-k3 yet.

Provenance

Converted 2026-07-29 from moonshotai/Kimi-K3 (96 safetensors shards, ~1.55 TB) with theseus at commit 6c5407b. Conversion machinery vendored from llama.cpp @ 1a064ab with the theseus kimi-k3 patch. Original Moonshot license included as LICENSE.