--- license: mit license_link: https://huggingface.co/NaiveAI/Naive-N0.5-Flash/blob/main/LICENSE model_name: doth4580/Naive-N0.5-Flash-EXL3-3.5bpw base_model: NaiveAI/Naive-N0.5-Flash base_model_relation: quantized tags: - exl3 - exllamav3 - trellis - 3.5bpw - vllm - moe - long-context - dgx-spark - naive-n0.5-flash --- # Naive-N0.5-Flash — EXL3 3.5 bpw > **Original model:** [NaiveAI/Naive-N0.5-Flash](https://huggingface.co/NaiveAI/Naive-N0.5-Flash) by the > NaiveAI Team, [MIT License](https://huggingface.co/NaiveAI/Naive-N0.5-Flash/blob/main/LICENSE). > This artifact is quantized from the public FP8 release > [NaiveAI/Naive-N0.5-Flash-FP8](https://huggingface.co/NaiveAI/Naive-N0.5-Flash-FP8). > It contains **modified weights** (EXL3 trellis quantization); the original model is © its authors. > Tokenizer and chat template are unchanged from the original. > Served by the veloGB10 engine it is listed by `/v1/models` as **`doth4580/Naive-N0.5-Flash-EXL3-3.5bpw`** (the `model_name:` line of > this card); `base_model:` is the original model it was quantized from, not this pack. An **EXL3 (ExLlamaV3 trellis) 3.5 bpw** quantization of **Naive-N0.5-Flash** — 309B parameters, 15.5B active, native 1M-token context. The pack is **~137 GB (~128 GiB) against ~315 GB for the FP8 original**, which is what lets it fit across **two NVIDIA DGX Spark (GB10)** machines. Quantized with the [exllamav3](https://github.com/turboderp-org/exllamav3) tooling, adapted for this architecture, run on **8×H100** for speed. It is served on two Sparks with the recipe in **[sf-stav/Naive-N0.5-Flash-EXL3-2x-DGX-Spark](https://github.com/sf-stav/Naive-N0.5-Flash-EXL3-2x-DGX-Spark)** — see [Serving](#serving) below. ## What this pack is: sources and credits - **Original model:** [NaiveAI/Naive-N0.5-Flash](https://huggingface.co/NaiveAI/Naive-N0.5-Flash) — © the NaiveAI Team, [MIT](https://huggingface.co/NaiveAI/Naive-N0.5-Flash/blob/main/LICENSE); a copy ships here as `LICENSE`. Built on Xiaomi's [MiMo-V2.5](https://huggingface.co/XiaomiMiMo/MiMo-V2.5-Base) and DeepSeek Sparse Attention. - **This artifact:** EXL3 3.5 bpw quantization produced with [exllamav3](https://github.com/turboderp-org/exllamav3) via the [vcruz305](https://github.com/vcruz305/exllamav3) fork. Weights are modified; tokenizer and chat template are not. - **Recipe:** [sf-stav/Naive-N0.5-Flash-EXL3-2x-DGX-Spark](https://github.com/sf-stav/Naive-N0.5-Flash-EXL3-2x-DGX-Spark) (MIT). Note that [`vllm-exl3`](https://github.com/vcruz305/vllm-exl3) is **AGPL-3.0** — relevant if you expose the server to others. ## Running it with veloGB10 Download a release binary of [veloGB10](https://github.com/sf-stav/veloGB10) (one self-contained file; see its README), and the pack with `hf download doth4580/Naive-N0.5-Flash-EXL3-3.5bpw --local-dir /path/to/Naive-N0.5-Flash-EXL3-3.5bpw`. The server is OpenAI-compatible on `http://:9000/v1`. The flags below are the ones this pack was tested with; `--max-batch` sets how many requests run at once. **TP=2 and TP=4 only:** no single GB10 holds this model (the engine says so at boot). ### Two GB10s (TP=2) On the second machine (no model copy, no configuration): ```bash ./gb10_inference --node --port 29500 ``` On the head (the machine with the model folder): ```bash ./gb10_inference --server \ --model-dir /path/to/Naive-N0.5-Flash-EXL3-3.5bpw \ --tp 2 --nodes :29500 \ --host 0.0.0.0 --port 9000 \ --max-seq-len 262144 \ --max-batch 2 \ --max-tokens 65536 \ --prefix-cache on ``` ### Four GB10s (TP=4) Start `./gb10_inference --node --port 29500` on each of the three other machines, then on the head: ```bash ./gb10_inference --server \ --model-dir /path/to/Naive-N0.5-Flash-EXL3-3.5bpw \ --tp 4 --nodes :29500,:29500,:29500 \ --host 0.0.0.0 --port 9000 \ --max-seq-len 262144 \ --max-batch 2 \ --max-tokens 65536 \ --prefix-cache on ``` **The nodes need no copy of the model.** The head sends each node only its own share of the weights over the network and keeps it in a local cache, so a second start of the same model transfers nothing. On a box with two active RoCE port pairs, name the one to use with `--rdma-dev ,`; with one pair it is found automatically. **With the Naive S8 FP8 drafter** ([NaiveAI/Naive-N0.5-Flash-FP8-Draft](https://huggingface.co/NaiveAI/Naive-N0.5-Flash-FP8-Draft)): add `--draft-dir /path/to/Naive-N0.5-Flash-FP8-Draft` to the head's command line (any topology above). ## Measured with veloGB10 VeloBench runs against the served API on NVIDIA DGX Spark-class GB10 machines (2026-11-09, veloGB10 0.7.7 development builds), at **production sampling**: each request sends the pack's default sampler settings, thinking on, one request at a time for the single-request columns. Decode is in output tokens per second and includes speculative decoding. One run per cell; sampled runs vary by roughly ±10%. | Topology | Draft source | Code turn decode | Story turn decode | All-turn median decode | Prefill (tok/s) | 2 requests, aggregate | Draft acceptance | |---|---|---:|---:|---:|---:|---:|---:| | TP=2 | Naive S8 FP8 drafter | 40 | 31 | 33 | 761 | 46 | 28.9% | | TP=4 | Naive S8 FP8 drafter | 80 | 46 | 50 | 1,077 | 69 | 34.7% | ## Accuracy Measured 2026-10-19 with a forward-only gate in **exllamav3** (teacher-forced NLL on a 977-token technical text; next-token probability after two code/JSON prefixes), the unquantized model through the authors' `transformers` code as the reference, same prompts for every row: | model | source | NLL (nats/token) | p(8-space indent after `if n < 0:`) | p(`age`) on a JSON key | |---|---|---|---|---| | Naive-N0.5-Flash bf16, `transformers` | bf16 | 1.627 | 1.000 | 1.000 | | **this pack** (3.5 bpw) | FP8 | **1.619** | 1.000 | 1.000 | | sibling [EXL3 4 bpw](https://huggingface.co/doth4580/Naive-N0.5-Flash-EXL3-4bpw) (experts 4 / attention 6) | bf16 | 1.634 | 1.000 | 1.000 | On this text the pack is indistinguishable from the unquantized model (the FP8 release it was made from scores marginally below the bf16 master). Agentic tool use, measured with [tool-eval-bench](https://github.com/sf-stav/tool-eval-bench) v2.7.1 (69 scenarios, temperature 0, seed 42, thinking on) on this pack served by the veloGB10 engine at TP=4: **86 / 100** (118 / 138 points), with the layer-47 correction below applied. A runtime without that correction scores ~2.02 nats on the same text. > **Runtime requirement (all EXL3 packs of this model):** layer 47's routed `up_proj` is stored **divided by > 128** (exllamav3's `interm_div` for this architecture — that layer's expert activations overflow f16) and the > forward must scale that layer's routing weights by 128, as exllamav3's `BlockSparseMLP` does. A runtime that > applies the pack as stored produces a last-layer MoE 128× too small; the symptoms are repetition loops, lost > indentation and garbled tool calls, not a crash. The veloGB10 engine had exactly this defect until 2026-10-19, > and the vLLM recipe's `vllm-exl3` path shows the same degraded NLL, so treat it as affected until its fix lands. ## Specifications | | | |---|---| | Base model | [NaiveAI/Naive-N0.5-Flash](https://huggingface.co/NaiveAI/Naive-N0.5-Flash) (FP8 release as the direct source) | | Architecture | MoE, 309B total / 15.5B active, 48 layers, hidden 4096, 64 heads / 4 KV heads, head_dim 192 | | Attention | hybrid 39 SWA layers (window 128) + 9 DSA layers (top-2048 selection), GQA4, no full attention | | MoE | 256 routed experts, 8 per token, expert intermediate 2048 | | Context / vocab | 1,048,576 tokens / 152,576 | | Quantization | EXL3 trellis, `mul1` codebook — uniform 3.5 bpw, 6-bit `lm_head`, embeddings unquantized, calibration 250×2048 | | Format / size | `exl3` (`--quantization exl3`) — ~137 GB (~128 GiB), 24 shards | | License | MIT (inherited from the original model) | ## Files | File | Notes | |---|---| | `model-00001..00024-of-00024.safetensors` | the quantized weights | | `model.safetensors.index.json`, `quantization_config.json` | tensor index and the full EXL3 metadata | | `config.json` | carries the EXL3 `quantization_config` incl. the `non_routed_exl3` map (the vLLM-rewritten pack config) | | `config.json.native`, `model.safetensors.index.json.native` | the equivalent native **ExLlamaV3** pack metadata | | `modeling_naive_n05_flash.py`, `configuration_naive_n05_flash.py` | model code, loaded via `trust_remote_code` | | tokenizer + `chat_template.jinja`, `generation_config.json` | unchanged from the original | ## Notes - **Beyond 2,048 tokens the DSA layers are approximate.** The model's 9 DSA layers attend to the top-2,048 tokens picked by a learned indexer; the recipe runs them as *dense* attention, which is exact up to 2,048 tokens but past that sees more than the reference model does. Long-context answers may differ from the original; the 1M native context needs a sparse attention backend that is not implemented yet. - **This is an early recipe, not a tuned one.** Measured decode on 2× DGX Spark: ~34.9 tok/s on pure code, ~23.9 mixed, ~19.5 reasoning (~23.5 without speculation). Performance tables, the profile, and the limitations are in the recipe's README. - **One request at a time**, bf16 KV only, and keep the Sparks otherwise idle (`GPU_MEM_UTIL=0.80`). ## Intellectual property We do **not** claim any intellectual-property ownership of the model, its weights, its architecture or its original documentation. All rights stay with the original authors and with every contributor credited above, under their own licences, which apply to this repository unchanged. Our contribution is limited to what the "What this pack is" section says we did (quantization, packaging, testing and the veloGB10 instructions). If you are a rights holder and want something here changed or removed, open a discussion on this repository. --- # Original model card — [NaiveAI/Naive-N0.5-Flash](https://huggingface.co/NaiveAI/Naive-N0.5-Flash) Reproduced verbatim from the original repository (metadata header removed).
Naive-N0.5-Flash

Building Frontier AI with AI

[🏠 Homepage][website] · [📰 Technical Blog][blog] · [💻 GitHub][github]
## Introduction Naive-N0.5-Flash is an open-weight **309B MoE model with 15.5B active parameters**, built for **coding and AI R&D**. It supports a **native 1M-token context window** through a hybrid of Sliding-Window Attention (SWA) and lightweight DeepSeek Sparse Attention (DSA), with no full-attention layers. ### Key Features - **Native 1M context, without full attention.** Naive-N0.5-Flash combines Sliding-Window Attention (SWA) and lightweight DeepSeek Sparse Attention (DSA) with GQA4 at a predominantly 5:1 SWA–DSA layout. The entire network remains local or sparse, with no full-attention layers. - **AI-optimized inference up to 2,000 tokens/s.** NaiveRT, our inference system for Naive-N0.5-Flash, was built and optimized through AI-centered R&D. It combines mega-kernel fusion, Programmatic Dependent Launch (PDL), and speculative decoding, delivering 50 tokens/s per user in Standard mode and up to 2,000 tokens/s in Ultrafast mode. See the NaiveRT case study in the [technical blog][blog] for the implementation and optimization process. - **Open weights and API.** Model weights and inference code are released under the MIT license. API access will also be provided, with pricing set at $0.10 / $0.40 / $0.01 per million tokens for input, output, and cache reads, respectively. ## Model Architecture | Property | Specification | | --- | --- | | Architecture | Mixture-of-Experts (MoE) | | Total parameters | 309B | | Active parameters | 15.5B | | Context length | Native 1M tokens | | Transformer layers | 48 | | Attention-layer composition | 39 SWA layers + 9 DSA layers | | Attention mechanism | Hybrid SWA–DSA | | SWA window | 128 tokens | | DSA token selection | Top 2,048 tokens for backbone attention | | DSA KV groups | 4 (GQA4) | | Indexer query heads | 16 | ### Hybrid SWA–DSA Attention Naive-N0.5-Flash builds on the open-weight MiMo-V2.5 base model, which has a simple architecture with strong foundational capabilities in world knowledge and deep research. Most layers use Sliding-Window Attention (SWA), whose per-token decoding cost does not grow with context length, while a small number of global-attention layers preserve long-range information. At million-token context lengths, however, these global-attention layers account for much of the decoding overhead. Naive-N0.5-Flash replaces the global-attention layers with DeepSeek Sparse Attention (DSA). A lightweight indexer scores the full history, while the backbone computes attention only over a selected subset of tokens. Although the indexer still scans the full history and the full KV cache is retained, sparse attention substantially reduces attention computation and memory access. Adapting the model to this new attention structure was one objective of continued pretraining. ![Hybrid SWA–DSA architecture showing the network stack and the DSA attention module, including the 16-head indexer and top-2,048 token selection.][architecture-figure]

Figure 1. The hybrid attention stack and DSA module.

The network consists of **eight six-layer modules**. A standard module contains five SWA layers followed by one DSA layer, with the first layer of the first module also replaced by DSA. SWA uses a **128-token window**, while DSA selects the **top 2,048 tokens** for backbone attention. Both attention types incorporate sink bias. Unlike the original MLA-based DSA implementation, Naive-N0.5-Flash replaces MLA with grouped-query attention (GQA) using **four KV groups**. For the architecture design process and indexer efficiency comparison, see model architecture in the [technical blog][blog]. ### Training Overview Following the architectural changes, Naive-N0.5-Flash completed 3.25T tokens of multi-stage training with a native 1M-token context window: 50B tokens of Indexer Warmup, 3T tokens of Sparse Attention Training, and 200B tokens of Learning Rate Decay. This process adapted the model to its new sparse attention architecture while substantially improving its AI R&D and coding capabilities. See the [technical blog][blog] for training details. ## Evaluation Results [![Coding benchmarks comparing Naive-N0.5-Flash with other models across seven software engineering and agentic tasks.][coding-figure]][coding-pdf]

Figure 2. Coding and agentic task results. Naive-N0.5-Flash is highlighted in yellow.

[![AI R&D benchmarks covering PostTrainBench, MLE-bench-30, PaperBench, SOL-ExecBench, NanoChat AutoResearch, and NanoGPT SpeedRun.][ai-rd-figure]][ai-rd-pdf]

Figure 3. AI research and systems optimization results. Metric directions are indicated in the figure.

Evaluation setup and metric notes **Evaluation setup.** Unless otherwise noted, our evaluations of Naive-N0.5-Flash use Claude Code 2.1.207 with a 1M-token context window, temperature 1.0, and top-p 0.95. The harness exposes only basic file I/O and Bash tools. **Sources for reported benchmark scores are as follows:** - **GLM-5.3 and GLM-5.3-Flash:** [GLM-5.3 blog](https://z.ai/blog/glm-5.3) and [GLM-5.3-Flash blog](https://z.ai/blog/glm-5.3-flash), respectively. - **Kimi-K3:** [Kimi-K3 model page](https://huggingface.co/moonshotai/Kimi-K3). - **Qwen-3.8-Max:** [Qwen-3.8-Max blog](https://qwen.ai/blog?id=qwen3.8). - **Hy4-preview:** [Hy4-preview model page](https://huggingface.co/tencent/Hy4-preview). - **DeepSeek-V4.1-Flash:** [DeepSeek-V4.1-Flash model page](https://huggingface.co/deepseek-ai/DeepSeek-V4.1-Flash). - **Step-5-preview:** [Step-5-Preview-BF16 model page](https://huggingface.co/TypeSafeAI/Step-5-Preview-BF16). - **Fable-5 (w/ fallback):** [GLM-5.3 blog](https://z.ai/blog/glm-5.3). - **SWE-Bench Pro:** GPT-5.6-Sol, Opus-5, and Opus-5.5 scores are drawn from the [GPT-5.6 blog](https://openai.com/index/gpt-5-6/) and the [Claude Opus 5.5 System Card](https://www-cdn.anthropic.com/fc1b44717c85dc068bc6ba5024219938094694bd/Claude%20Opus%205.5%20System%20Card.pdf). - **DeepSWE v1.1:** The Muse-Spark-1.3 score comes from its [Muse-Spark-1.3 blog](https://research.meta.ai/static/muse-spark-1-3-multimodal-evaluation-methodology). GPT-5.6-Sol and Opus-5 scores come from the [DeepSWE v1.1 leaderboard](https://deepswe.datacurve.ai/). The Opus-5.5 score comes from the [Claude Opus 5.5 System Card](https://www-cdn.anthropic.com/fc1b44717c85dc068bc6ba5024219938094694bd/Claude%20Opus%205.5%20System%20Card.pdf). - **Terminal-Bench 2.1:** Muse-Spark-1.3, GPT-5.6-Sol, and Opus-5 scores come from the [Muse-Spark-1.3 blog](https://research.meta.ai/static/muse-spark-1-3-multimodal-evaluation-methodology). The GPT-6-Astra score comes from the [Terminal-Bench 2.1 leaderboard](https://www.tbench.ai/?version=2.1). - **ALE-CLI:** GPT-5.6-Sol, GPT-6-Astra, Muse-Spark-1.3, Opus-5, and Opus-5.5 scores come from the [ALE-CLI leaderboard](https://agents-last-exam.org/leaderboard). - **FrontierSWE v1:** We calculate the Dominance score using the competing systems’ results as of August 23, 2026. - **ProgramBench:** We report the Almost@1 score. GPT-5.6-Sol and Opus-5 scores come from the [ProgramBench leaderboard](https://programbench.com/). - **MLE-bench-30:** Gemini-3.5-Flash, Gemini-3.6-Flash, Grok-4.5, and GPT-5.6-Luna scores come from the [Gemini 3.6 Flash model card](https://deepmind.google/models/model-cards/gemini-3-6-flash/). Following its evaluation protocol, we report the average position score of Naive-N0.5-Flash. - **PaperBench:** MiniMax M3, Opus-4.7, GPT-5.5, and Gemini-3.1-Pro scores come from the [MiniMax M3 model page](https://huggingface.co/MiniMaxAI/MiniMax-M3). - **SOL-ExecBench, NanoChat AutoResearch, and NanoGPT SpeedRun:** Naive-N0.5-Flash scores were obtained using our in-house AutoResearch harness. Recursive Superintelligence Inc. scores come from its [research article](https://www.recursive.com/articles/first-steps-toward-automated-ai-research). Following the SOL-ExecBench update, we use its updated score from the [leaderboard](https://research.nvidia.com/benchmarks/sol-execbench).
## Deployment Naive-N0.5-Flash supports FP8 mixed-precision inference. For general use, we recommend setting the sampling parameters to `temperature=1.0` and `top_p=0.95`. ### Quick Start with Transformers Naive-N0.5-Flash requires FP8-capable NVIDIA GPUs. The model weights occupy approximately 315 GB; allow additional GPU memory for inference. ```bash pip install "transformers[torch,kernels]>=5.17.0" ``` ```python from transformers import AutoModelForCausalLM, AutoTokenizer model_id = "NaiveAI/Naive-N0.5-Flash-FP8" tokenizer = AutoTokenizer.from_pretrained(model_id, trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained( model_id, trust_remote_code=True, dtype="auto", device_map="auto", ) inputs = tokenizer.apply_chat_template( [{"role": "user", "content": "Hello!"}], add_generation_prompt=True, return_dict=True, return_tensors="pt", ).to(model.device) output = model.generate(**inputs, max_new_tokens=2048) response = tokenizer.decode( output[0, inputs["input_ids"].shape[1]:], skip_special_tokens=True, ) print(response) ``` ## License Naive-N0.5-Flash is released under the MIT License. ## Citation If you find Naive-N0.5-Flash useful in your research or work, please cite: ```bibtex @misc{naiveai2026naiven05flash, title = {Naive-N0.5-Flash: Building Frontier AI with AI}, author = {{NaiveAI Team}}, year = {2026}, url = {https://naive.ai/en/research/} } ``` ## Acknowledgments Naive-N0.5-Flash builds on the work of the open-source community and gives back to it. We thank the Xiaomi MiMo team for making their MiMo-V2.5 base model publicly available, the DeepSeek team for their work on DeepSeek Sparse Attention (DSA), and the SGLang team and community for their open-source inference infrastructure. ## Contact For questions, feedback, or collaboration, please contact us at contact@naive.ai or follow us on X at @naiveailab. You can also find our open-source projects and model releases on [GitHub](https://github.com/naiveai-labs) and [Hugging Face](https://huggingface.co/NaiveAI). [website]: https://naive.ai/ [blog]: https://naive.ai/en/research/ [github]: https://github.com/NaiveAI-Labs/Naive-N0.5-Flash [weights]: https://huggingface.co/NaiveAI/Naive-N0.5-Flash [discussions]: https://huggingface.co/NaiveAI/Naive-N0.5-Flash/discussions [base-model]: https://huggingface.co/XiaomiMiMo/MiMo-V2.5-Base [architecture-figure]: assets/hybrid-swa-dsa-architecture.svg [coding-figure]: assets/coding-benchmarks.png [coding-pdf]: assets/coding-benchmarks.pdf [ai-rd-figure]: assets/ai-rd-benchmarks.png [ai-rd-pdf]: assets/ai-rd-benchmarks.pdf