Smaller GGUF that runs on stock llama.cpp (/v1/systemone)
#4
by mike18 - opened
Hi, we've published GGUF builds of pplx-decider-v1.1-27b that load in stock llama-server (llama.cpp from 9 October 2026 or newer) and answer on /v1/systemone:
https://huggingface.co/turintech/pplx-decider-27B-Artemis-GGUF
The default file is 15.3 GB against 53.8 GB for the BF16, and gives the same answer as the original on 96.8% of 2,000 test decisions (NVIDIA A100). The card also has held-out results, the accuracy change, GPU memory, and a comparison with a plain Q4_K_M made from the original (16.55 GB, 95.7%).
These are independent conversions by TurinTech, not affiliated with Perplexity. Thanks for releasing the model under Apache-2.0. Happy to answer questions here.