PollardWeights/K2-Horizon-3.7B-Pollard

K2-Horizon-3.7B — Pollard

Pollard shrank this model: 10.12 GB (f16) → 2.75 GB — 73% smaller, 3.7× down.

The smallest rung here; larger, higher-fidelity rungs are listed below.

format	this model's size
f16	10.12 GB
Q8_0	~5.36 GB
Q6_K	~4.15 GB
Q4_K_M	~2.93 GB
PollardMix (this repo's IQ3_S)	2.75 GB

Pollard builds of IFM/K2-Horizon-3.7B made with Pollard Weights — a ladder of measured-allocation quants (bits placed by per-layer sensitivity, not a uniform crush).

These files need the MBZUAI-IFM llama.cpp fork. This model's architecture (k2-horizon) is not one upstream llama.cpp knows, so stock llama.cpp -- and therefore Ollama and LM Studio -- cannot load them whatever the quant types are. The quants themselves are ordinary K-quants.

Model details

Parameter count	~5.1B
Architecture	k2_horizon
Input support	text
imatrix	no
Perplexity measured	yes — table below

Which file should I choose?

Every rung is the same weights, sized to a different RAM budget by the measured allocation. Pick the largest one that fits your machine with room for context:

• ~6 GB RAM / VRAM → Q6_K (4.16 GB). (needs the MBZUAI-IFM llama.cpp fork) near-lossless (+0.08 over f16)
• ~5 GB RAM / VRAM → IQ4_XS (3.33 GB). (needs the MBZUAI-IFM llama.cpp fork) recommended default (+0.14)
• ~5 GB RAM / VRAM → IQ3_S (2.75 GB). (needs the MBZUAI-IFM llama.cpp fork) smallest (+1.24)

Available files (wikitext-2 test, ctx 512)

f16 reference PPL 11.2798...

file	PPL	size	Mean KLD	notes
K2-Horizon-3.7B-Pollard-IQ3_S.gguf	12.5164	2.75 GB	—	smallest (+1.24)
K2-Horizon-3.7B-Pollard-IQ4_XS.gguf	11.4211	3.33 GB	—	recommended default (+0.14)
K2-Horizon-3.7B-Pollard-Q6_K.gguf	11.3603	4.16 GB	—	near-lossless (+0.08 over f16)

Measured notes

f16 reference PPL 11.2798.

Measured notes

f16 reference PPL 11.2798..

Download a specific file

pip install -U "huggingface_hub[cli]"
hf download PollardWeights/K2-Horizon-3.7B-Pollard \
  --include "K2-Horizon-3.7B-Pollard-IQ4_XS.gguf" --local-dir ./

How to run

This model's architecture (k2-horizon) needs the MBZUAI-IFM llama.cpp fork, so every file here runs there:

llama-cli    -m K2-Horizon-3.7B-Pollard-IQ4_XS.gguf -ngl 99 -p "Explain why the sky is blue."
llama-server -m K2-Horizon-3.7B-Pollard-IQ4_XS.gguf -ngl 99

No rung in this repo loads in stock llama.cpp, so Ollama and LM Studio cannot run these files.

ARM / AVX

llama.cpp repacks weights into an interleaved layout at load time for faster inference on ARM and AVX machines — no special file needed, online repacking covers these quants. The old Q4_0_4_4/4_8/8_8 variants are not required.

Errata

• general.architecture is k2-horizon, which upstream llama.cpp does not implement, so these files load only in the MBZUAI-IFM llama.cpp fork — the quant types are ordinary and irrelevant to that. Checked with pollard-ggufcheck, which reads the architecture and the tensor types out of the header.
• Measured allocation places bits by per-layer sensitivity under a size budget.
• Single machine; replication invited.

Credits & license

• Base model: IFM/K2-Horizon-3.7B
• Quantization tooling: llama.cpp (ggml-org)
• Method + tooling: Pollard Weights — measure first, no claim before a number.
• License: apache-2.0, inherited from the base model.

Built with Pollard Weights — frontier models, small hardware, no compromise.