Initial selective AWQ v1.0 release

.gitattributes

@@ -1,35 +1,4 @@
-*.7z filter=lfs diff=lfs merge=lfs -text
-*.arrow filter=lfs diff=lfs merge=lfs -text
-*.bin filter=lfs diff=lfs merge=lfs -text
-*.bz2 filter=lfs diff=lfs merge=lfs -text
-*.ckpt filter=lfs diff=lfs merge=lfs -text
-*.ftz filter=lfs diff=lfs merge=lfs -text
-*.gz filter=lfs diff=lfs merge=lfs -text
-*.h5 filter=lfs diff=lfs merge=lfs -text
-*.joblib filter=lfs diff=lfs merge=lfs -text
-*.lfs.* filter=lfs diff=lfs merge=lfs -text
-*.mlmodel filter=lfs diff=lfs merge=lfs -text
-*.model filter=lfs diff=lfs merge=lfs -text
-*.msgpack filter=lfs diff=lfs merge=lfs -text
-*.npy filter=lfs diff=lfs merge=lfs -text
-*.npz filter=lfs diff=lfs merge=lfs -text
-*.onnx filter=lfs diff=lfs merge=lfs -text
-*.ot filter=lfs diff=lfs merge=lfs -text
-*.parquet filter=lfs diff=lfs merge=lfs -text
-*.pb filter=lfs diff=lfs merge=lfs -text
-*.pickle filter=lfs diff=lfs merge=lfs -text
-*.pkl filter=lfs diff=lfs merge=lfs -text
-*.pt filter=lfs diff=lfs merge=lfs -text
-*.pth filter=lfs diff=lfs merge=lfs -text
-*.rar filter=lfs diff=lfs merge=lfs -text
 *.safetensors filter=lfs diff=lfs merge=lfs -text
-saved_model/**/* filter=lfs diff=lfs merge=lfs -text
-*.tar.* filter=lfs diff=lfs merge=lfs -text
-*.tar filter=lfs diff=lfs merge=lfs -text
-*.tflite filter=lfs diff=lfs merge=lfs -text
-*.tgz filter=lfs diff=lfs merge=lfs -text
-*.wasm filter=lfs diff=lfs merge=lfs -text
-*.xz filter=lfs diff=lfs merge=lfs -text
-*.zip filter=lfs diff=lfs merge=lfs -text
-*.zst filter=lfs diff=lfs merge=lfs -text
-*tfevents* filter=lfs diff=lfs merge=lfs -text
+*.json text eol=lf
+*.md text eol=lf
+*.jinja text eol=lf

EVALUATION.md

@@ -0,0 +1,132 @@
+# Evaluation summary
+
+This file separates the checkpoint-quality measurements from runtime
+compatibility checks. It is not a reproduction of the base model's published
+software-engineering benchmark table.
+
+## Checkpoint quality against the frozen BF16 base
+
+The selected unscaled checkpoint was compared with
+`poolside/Laguna-S-2.1@e80da38da3ed4c4e56888cc1ba39582946a164ba` on 16 disjoint
+512-token held-out blocks, covering 8,176 scored next-token positions.
+
+| Metric | Selective AWQ | BF16 reference |
+| --- | ---: | ---: |
+| Final-hidden relative L2 | 0.0052877845 | 0 by definition |
+| Mean full-vocabulary KLD | 0.2344706438 nats | 0 by definition |
+| Perplexity | 23.73715897 | 23.17865143 |
+| Token top-1 agreement | 86.6683% | 100% by definition |
+| Small math/code/agent smoke suite | 29/30 | Not used as a public benchmark |
+
+The held-out token tensor SHA-256 is
+`77976641882fa641e26f998594aa935518acb7796e9907e7182656b1361015ce`.
+
+## V100 combined FP16 runtime correction
+
+The V100 add-on was tested on four Tesla V100 PCIe 32GB GPUs with tensor
+parallel size 4, 1Cat-vLLM 1.2.2, PyTorch 2.10.0+cu128, Transformers 5.14.1,
+driver 580.159.04, FP16 activations/KV, `TRITON_ATTN`, chunked prefill, prefix
+caching disabled, eager execution, and custom all-reduce disabled.
+
+The original add-on corrected the late-layer activation range but missed a
+separate Laguna/vLLM integration defect. In the pinned runtime, MoERunner's
+FP16 overflow-protection path returns the combined routed/shared MoE output at
+`1 / routed_scaling_factor` of Laguna's mathematical decoder result and
+requires a matching decoder compensation. The Laguna decoder omitted that
+multiply, so all 47 MoE layer outputs were approximately `0.4x` when the
+configured factor was `2.5`. BF16 does not use this branch.
+
+Rc4 combines the required FP16 MoE `x2.5` forward compensation with the
+existing local Layer-45 `/64` range repair. The unchanged load-time weight
+transform was checked against the materialized Layer-45 `/64` reference:
+
+- 3,882 Layers 45-47 tensors checked;
+- 1,545 transformed tensors bitwise identical;
+- 2,337 untouched tensors bitwise identical;
+- Layer 45 AWQ `qweight` and `qzeros` unchanged.
+
+After correcting the MoE semantics in both arms, the fixed 32-record runtime
+corpus and all 8,212 shared scored positions were re-run. The local repair was
+compared with an unscaled FP16 reference that included the same required MoE
+compensation:
+
+| Metric | Rc4 local repair versus corrected unscaled FP16 |
+| --- | ---: |
+| Mean coarsened KLD | 6.0291049932e-05 nats |
+| Maximum coarsened KLD | 0.0339052182 nats |
+| Token top-1 agreement | 99.9391% |
+| Actual-token logprob MAE | 0.00373973 |
+| Diagnostic PPL | 21.9160 versus 21.9363 reference |
+
+The runtime corpus uses 1Cat-vLLM prompt-logprob output only for same-position
+distribution comparisons. Its token attachment is observed to be shifted by
+one position for Laguna, so the large diagnostic PPL value from that harness is
+not presented as an accepted language-model perplexity result.
+
+## V100 functional checks
+
+| Test | Result |
+| --- | ---: |
+| Complete 512-token calibration blocks with top-5 logprobs | 256/256 finite |
+| Forced generation immediately after an internal EOS | 11/11 finite |
+| Histories with two or three explicitly injected EOS tokens | 4/4 finite |
+| Plain text generation | Passed |
+| Parsed reasoning response | Passed |
+| Forced structured tool call and JSON arguments | Passed |
+| 30-case math/code/agent smoke (`thinking=off`, 512-token cap) | 29/30; all 30 predictions matched the saved pre-rc4 run |
+| Chinese chat, English chat, Python code, raw Chinese completion | Passed; no prior corruption/repetition or replacement characters |
+| 4,095-token needle | Passed; 3.156 s TTFT; 1,297.36 tok/s end-to-end prefill |
+| 15,352-token needle | Passed; 11.028 s TTFT; 1,392.07 tok/s end-to-end prefill |
+| 259,989-token needle | Passed; exact needle recovered; 1,384.602 s TTFT; 187.77 tok/s end-to-end prefill |
+
+The 259,989-token request completed in 1,387.309 seconds including 23 output
+tokens. Peak sampled device memory was 31,968 MiB used and 527 MiB free on each
+V100; the server remained healthy with no OOM and no restart.
+
+The 30-case suite is a gross-regression smoke test, not a public benchmark or
+an exact numerical gate. Two exploratory runs with a 128-token cap were
+discarded because verbose answer/reasoning text was truncated before the final
+option. With a 512-token cap, rc4 reproduced all 30 saved predictions exactly.
+
+## Stock-vLLM BF16 compatibility probe
+
+The exact unscaled RC was copied to a second machine, passed its complete
+`SHA256SUMS` check, and was loaded by upstream vLLM 0.24.0 with
+`trust_remote_code=False`, `dtype=bfloat16`, and native
+`LagunaForCausalLM`/AutoAWQ support. The compatibility host used heterogeneous
+TP2 across an RTX 3090 24 GB (SM86) and RTX 5060 Ti 16 GB (SM120), with 36 GiB
+CPU offload configured per GPU, PyTorch 2.11.0+cu130, and Transformers 5.14.1.
+
+Both TP workers reported the same loaded parameter inventory: 917 tensors,
+comprising 624 BF16 tensors, 47 FP32 router-bias tensors, and 246 INT32 packed
+tensors. No loaded parameter was FP16. Representative embeddings, attention,
+protected prefix/tail experts, shared experts, final norm, and LM head were
+BF16; packed AWQ weights remained INT32 and their floating scales were BF16.
+
+A real 28-token forward containing EOS token id 2 at position 0 installed hooks
+on every decoder layer:
+
+| Check | Result |
+| --- | ---: |
+| Decoder layers observed | 48/48 on both TP workers |
+| Hidden/residual dtype and finiteness | BF16 and finite on every observed layer |
+| Loaded FP16 parameter tensors | 0 on both TP workers |
+| Layer 47 hidden maximum | 73,728, finite in BF16 and above FP16's 65,504 limit |
+| Final normalization | BF16, finite, max absolute value 15.125 |
+| One-token decode | Completed |
+
+This is deliberately a compatibility and dtype/range test. Heavy CPU offload
+and the mixed GPUs make its latency non-representative; 4K, 16K, and 256K
+performance were not re-measured on this host. The complete long-context and
+functional acceptance numbers above belong to the primary 4x V100 target.
+
+## Scope and untested paths
+
+- Laguna S 2.1 is text-to-text; it has no image or audio encoder, so no
+  multimodal test is applicable.
+- The exact unscaled main checkpoint has passed a stock-vLLM BF16 native load,
+  positive loaded-parameter probe, 48-layer activation probe, and short EOS
+  forward. A homogeneous, all-GPU BF16 performance run was not performed.
+- Transformers/AutoAWQ inference, SGLang, TensorRT-LLM, standalone RTX 3090,
+  DFlash, MTP, and the configured one-million-token maximum context are not
+  evaluated.

LICENSE.md

@@ -0,0 +1,49 @@
+OpenMDW License Agreement, version 1.1 (OpenMDW-1.1)
+
+By exercising rights granted to you under this agreement, you accept and agree
+to its terms.
+
+As used in this agreement, "Model Materials" means the materials provided to
+you under this agreement, consisting of: (1) one or more machine learning
+models (including architecture and parameters); and (2) all related artifacts
+(including associated data, documentation and software) that are provided to
+you hereunder.
+
+Subject to your compliance with this agreement, permission is hereby granted,
+free of charge, to deal in the Model Materials without restriction, including
+under all copyright, patent, database, and trade secret rights included or
+embodied therein.
+
+If you distribute any portion of the Model Materials, you shall retain in your
+distribution (1) a copy of this agreement, and (2) all copyright notices and
+other notices of origin included in the Model Materials that are applicable to
+your distribution.
+
+If you file, maintain, or voluntarily participate in a lawsuit against any
+person or entity asserting that the Model Materials directly or indirectly
+infringe any patent or copyright, then all rights and grants made to you
+hereunder are terminated, unless that lawsuit was in response to a
+corresponding lawsuit first brought against you.
+
+This agreement does not impose any restrictions or obligations with respect to
+any use, modification, or sharing of any outputs generated by using the Model
+Materials.
+
+THE MODEL MATERIALS ARE PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
+OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE, TITLE, NONINFRINGEMENT, ACCURACY, OR THE
+ABSENCE OF LATENT OR OTHER DEFECTS OR ERRORS, WHETHER OR NOT DISCOVERABLE, ALL
+TO THE GREATEST EXTENT PERMISSIBLE UNDER APPLICABLE LAW.
+
+YOU ARE SOLELY RESPONSIBLE FOR (1) CLEARING RIGHTS OF OTHER PERSONS THAT MAY
+APPLY TO THE MODEL MATERIALS OR ANY USE THEREOF, INCLUDING WITHOUT LIMITATION
+ANY PERSON'S COPYRIGHTS OR OTHER RIGHTS INCLUDED OR EMBODIED IN THE MODEL
+MATERIALS; (2) OBTAINING ANY NECESSARY CONSENTS, PERMISSIONS OR OTHER RIGHTS
+REQUIRED FOR ANY USE OF THE MODEL MATERIALS; OR (3) PERFORMING ANY DUE
+DILIGENCE OR UNDERTAKING ANY OTHER INVESTIGATIONS INTO THE MODEL MATERIALS OR
+ANYTHING INCORPORATED OR EMBODIED THEREIN.
+
+IN NO EVENT SHALL THE PROVIDERS OF THE MODEL MATERIALS BE LIABLE FOR ANY CLAIM,
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE MODEL MATERIALS, THE
+USE THEREOF OR OTHER DEALINGS THEREIN.

README.md

@@ -0,0 +1,268 @@
+---
+license: openmdw-1.1
+base_model: poolside/Laguna-S-2.1
+base_model_relation: quantized
+library_name: transformers
+pipeline_tag: text-generation
+tags:
+  - laguna
+  - awq
+  - selective-quantization
+  - w4a16
+  - moe
+  - bfloat16
+  - vllm
+---
+
+# Laguna S 2.1 Selective AWQ — W4A16 g32
+
+This project was built to make the 118B-parameter Laguna S 2.1 MoE model usable on **4× Tesla V100 32GB with tensor parallel size 4 and a checkpoint below 100 GB**. The size ceiling, W4A16 g32 asymmetric quantization, FP16 weight storage, and the protected routed-expert allocation—Layers 1–4 and 46–47—were all selected for that deployment target.
+
+The main checkpoint is the general form of the same work: one unscaled weight artifact configured for BF16 activations, intended to run directly on a BF16-capable SM80+ system with stock vLLM. The V100 runtime add-on reuses those exact weights; there is no second V100 checkpoint.
+
+This is an independently produced selective AWQ quantization of [poolside/Laguna-S-2.1](https://huggingface.co/poolside/Laguna-S-2.1). It is not an official Poolside release.
+
+## Critical dtype requirement for the main checkpoint
+
+**Run the unscaled main checkpoint with BF16 activations. Do not force FP16 (`float16`/`half`).**
+
+The original model can produce very large late-layer hidden states at BOS/EOS anchor positions. In our fixed calibration replay, the largest observed BF16 hidden value was approximately `950,272`, above the FP16 finite limit of `65,504`. Each of the 1,097 token-id-2 BOS/EOS anchor rows contained at least one component outside the FP16 range, while the largest observed non-anchor value was `17,152`. Normalized outputs and logits remained finite in BF16.
+
+This is an activation dynamic-range property, not an AWQ packing failure. A runtime that silently executes the main checkpoint with FP16 activations can produce non-finite logits at EOS positions.
+
+## V100 runtime add-on — primary deployment target
+
+The included `v100-1cat/` directory is a small, fail-closed runtime patch for the configuration that motivated this quantization:
+
+- 4× Tesla V100 PCIe 32GB;
+- tensor parallel size 4;
+- 1Cat-vLLM 1.2.2;
+- FP16 activations and FP16 KV cache;
+- `TRITON_ATTN`, chunked prefill, and prefix caching disabled.
+
+It is deliberately version-pinned, not a cross-version plugin. It uses the exact main-model revision without copying or permanently rewriting the approximately 94 GB checkpoint.
+
+Rc4 fixes two independent FP16 runtime problems. The pinned 1Cat/vLLM MoE runner's overflow-protection branch represents Laguna's combined routed/shared output at `1 / routed_scaling_factor` and requires a matching decoder compensation. The pinned Laguna decoder omitted that multiply, so each MoE layer returned approximately `0.4x` of the intended branch output for the configured factor `2.5`. Rc4 restores the required FP16 `x2.5` after every MoE layer. BF16 inference does not use this branch and is unaffected.
+
+The second problem is the late-layer BOS/EOS activation overflow. For that range repair, the add-on applies a local power-of-two reparameterization to Layers 45–47, while the forward path multiplies hidden/residual by `1/64` once before Layer 45:
+
+- attention `o_proj`: `× 1/64`;
+- routed/shared expert `up_proj`: `× 1/256`;
+- routed/shared expert `down_proj`: `× 4`.
+
+The net branch factor is `1/64`. Layer 45 remains W4; its `qweight` and `qzeros` are unchanged, while the relevant floating AWQ scale carriers are transformed. The add-on rejects an unknown model revision, quantization layout, or tensor count.
+
+`FLASH_ATTN_V100` is deliberately rejected by the add-on. On this model, the tested Flash-V100 path became non-finite after chunked prefill crossed its observed boundary. Triton is somewhat slower on short prompts, but it is the accepted V100 path from 512 tokens through long context.
+
+The same-weight loader-time implementation has reproduced the materialized reference:
+
+| Test | Loader-time add-on result |
+| --- | ---: |
+| Loader transform versus materialized reference | 1,545 transformed and 2,337 untouched tail tensors bitwise identical |
+| Complete EOS-containing calibration blocks | 256/256 finite |
+| Generation immediately after internal EOS | 11/11 finite |
+| Explicit multi-EOS histories | 4/4 finite |
+| Chinese/English chat, Python code, raw Chinese completion | Passed; no prior corruption/repetition or replacement characters |
+| Mean coarsened KLD versus corrected unscaled FP16 arm | `6.0291049932e-05` nats over 8,212 positions |
+| Token top-1 agreement versus corrected unscaled FP16 arm | 99.9391% |
+| Actual-token logprob MAE versus corrected unscaled FP16 arm | `0.00373973` |
+| Plain text, parsed reasoning, forced structured tool call | Passed |
+| 30-case math/code/agent smoke | 29/30; all 30 predictions matched the saved pre-rc4 run |
+| 4,095- and 15,352-token needles | Passed |
+| 259,989-token needle | Passed; exact needle recovered; 1,384.602 s TTFT; 187.77 tok/s end-to-end prefill |
+
+The non-zero KLD and 99.9391% top-1 figures compare rc4's Layer-45 `/64` repair with an unscaled FP16 runtime that includes the same required MoE `x2.5` correction. The diagnostic PPL changed from 21.9363 to 21.9160, but 1Cat's Laguna prompt-logprob token attachment is shifted by one position, so that value is not presented as a language-model perplexity result.
+
+The 30-case smoke used `thinking=off` with a 512-token cap. It is a small gross-regression check, not a public benchmark or an exact numerical gate.
+
+A standalone global `/32` weight-folding experiment was also tested and rejected: against the same unscaled baseline it produced `0.0661285` nats mean coarsened KLD and `91.0010%` token top-1 agreement because FP16 rounding accumulated through all 48 layers. The V100 release therefore uses the local runtime add-on.
+
+## Model summary
+
+Laguna S 2.1 is a text-to-text, 118B-total-parameter, approximately 8B-active-per-token Mixture-of-Experts model for agentic coding and long-horizon work. It is not a vision or audio model. The base model has 48 decoder layers, 256 routed experts plus one shared expert, and a configured context window of 1,048,576 tokens. See the [official base model card](https://huggingface.co/poolside/Laguna-S-2.1) for architecture, chat formatting, intended use, responsible-use guidance, and base-model benchmark results.
+
+Frozen base revision:
+
+```text
+poolside/Laguna-S-2.1@e80da38da3ed4c4e56888cc1ba39582946a164ba
+```
+
+## Quantization layout
+
+Layer numbers are zero-based Hugging Face names, `model.layers.N`.
+
+| Component | Stored format |
+| --- | --- |
+| Layer 0 dense MLP | FP16 |
+| Routed experts, Layers 1–4 | FP16 |
+| Routed experts, Layers 5–45 | AWQ W4A16, group size 32, asymmetric with zero-points |
+| Routed experts, Layers 46–47 | FP16 |
+| Attention, all layers | FP16 |
+| Routers and correction biases, all layers | FP16 |
+| Shared experts, all layers | FP16 |
+| Embeddings, normalization, and output head | FP16 |
+
+AWQ configuration:
+
+```json
+{
+  "bits": 4,
+  "group_size": 32,
+  "quant_method": "awq",
+  "version": "gemm",
+  "zero_point": true
+}
+```
+
+`W4A16` denotes packed 4-bit weights with 16-bit activations. The main path uses BF16 activations; the V100 add-on uses FP16 activations with the local range repair above.
+
+A single FP16-stored weight artifact serves both the BF16 main path and the V100 add-on. The precision effect of storing protected tensors in FP16 is already included in the KLD, perplexity, and top-1 measurements reported below.
+
+Some protected FP16 tensors participate in mathematically equivalent AWQ scale compensation. They remain FP16 and were not packed to W4, but they need not be byte-identical to the corresponding base-model tensors.
+
+`quantization_config.modules_to_not_convert` has been checked against this exact FP16/W4 layout and the machine-readable tensor audit. Stock vLLM 0.24.0 also loaded the mixed layout successfully; this is no longer inferred only from the JSON declaration.
+
+## Why this allocation?
+
+The protected prefix and tail were selected from actual packed-model comparisons under the sub-100-GB V100 budget.
+
+| FP16 routed-expert layers | Directory bytes | Final hidden rel-L2 | Mean full-vocab KLD | PPL | Top-1 agreement | 30-task smoke |
+| --- | ---: | ---: | ---: | ---: | ---: | ---: |
+| 1–3 and 45–47 | 94,311,697,448 | 0.006003 | 0.241694 | 24.3297 | 86.56% | 28/30 |
+| 1–4 and 45–47 | 97,746,514,057 | 0.005274 | 0.233194 | 23.7390 | 86.94% | 28/30 |
+| **1–4 and 46–47** | **94,311,700,368** | **0.005288** | **0.234471** | **23.7372** | **86.67%** | **29/30** |
+| 1–5 and 46–47 | 97,746,516,977 | 0.005844 | 0.245507 | 23.6633 | 86.77% | 28/30 |
+| 1–6 and 47 | 97,746,519,891 | 0.010638 | 0.249403 | 24.0867 | 86.48% | 29/30 |
+| 1–7 | 97,746,522,803 | 0.060652 | 0.246453 | 24.4271 | 86.35% | 29/30 |
+
+The selected layout protects four early and two final routed-expert layers while remaining below the 100,000,000,000-byte checkpoint limit. The 30-task column is a small deterministic regression smoke test, not a general benchmark.
+
+## Main-checkpoint usage
+
+The supported main path uses upstream vLLM's native Laguna implementation and does not depend on copied model remote code. Transformers inference is not currently a validated path.
+
+The command below is the intended BF16 serving form. The Poolside parser flags remain separately qualified below because the stock-vLLM probe exercised the native model and dtype path, not the HTTP parser stack:
+
+```bash
+vllm serve \
+  --model leoncca/Laguna-S-2.1-Selective-AWQ \
+  --tensor-parallel-size 4 \
+  --dtype bfloat16 \
+  --tool-call-parser poolside_v1 \
+  --reasoning-parser poolside_v1 \
+  --enable-auto-tool-choice \
+  --served-model-name laguna \
+  --default-chat-template-kwargs '{"enable_thinking": true}'
+```
+
+The parser and chat-template flags are copied from the base model's recommended vLLM setup. Reasoning and forced tool calls passed on the validated 1Cat V100 runtime; the same parser flags have not yet been re-run through the upstream-vLLM HTTP server.
+
+The accepted stock-vLLM probe positively verified the loaded dtype rather than relying on the absence of a cast warning. On both TP workers, the 917 exposed parameter tensors consisted of 624 BF16, 47 FP32 router-bias, and 246 INT32 packed tensors, with no FP16 parameter. Hooks observed all 48 decoder layers with BF16 finite hidden/residual tensors and a finite BF16 final norm.
+
+Upstream vLLM 0.24.0 contains a native [Laguna model implementation](https://github.com/vllm-project/vllm/blob/v0.24.0/vllm/model_executor/models/laguna.py) and its [AutoAWQ configuration accepts BF16 activations](https://github.com/vllm-project/vllm/blob/v0.24.0/vllm/model_executor/layers/quantization/auto_awq.py). The exact RC was loaded with `trust_remote_code=False`, `dtype=bfloat16`, and `quantization=auto_awq`.
+
+## Compatibility status
+
+| Environment | Status |
+| --- | --- |
+| Offline packed-model comparison against the frozen BF16 base | Validated |
+| Materialized 4× V100 tail-repair reference | Validated as development evidence |
+| Final same-weight V100 add-on rc4, 1Cat-vLLM 1.2.2 | Validated with corrected FP16 MoE semantics through EOS, 259,989 tokens, reasoning, tool calls, and fixed-corpus drift |
+| Exact main checkpoint, upstream vLLM 0.24.0, BF16 | Native load and positive 48-layer dtype/EOS forward validated on heterogeneous RTX 3090 + RTX 5060 Ti TP2 with CPU offload; not a performance test |
+| Transformers/AutoAWQ inference | Not evaluated |
+| SGLang | Not evaluated |
+| TensorRT-LLM | Not evaluated |
+| Standalone RTX 3090 | Not evaluated; the validated compatibility probe used RTX 3090 + RTX 5060 Ti together |
+| DFlash or MTP | Not evaluated / not included |
+
+The `library_name: transformers` metadata describes the Hugging Face configuration/tokenizer format and base architecture; it is not a claim that Transformers inference has been validated.
+
+## Quantization and evaluation data
+
+### Calibration
+
+| Field | Value |
+| --- | --- |
+| Dataset | `codeparrot/self-instruct-starcoder` |
+| Split | `curated` |
+| Dataset revision | `9598805f9276dc0a339e293e9f205ea8bbf41f9f` |
+| Shuffle seed | 42 |
+| Shape | 256 blocks × 512 tokens |
+| Token tensor SHA-256 | `24f5f0f831a6abe747940602e990c81d52a6f39e96c15e23bf8396fc47150b6c` |
+
+Calibration data was used for post-training quantization only; the base model was not retrained.
+
+### Held-out numerical comparison
+
+The main numerical comparison used 16 disjoint 512-token held-out blocks and 8,176 scored next-token positions against the fixed raw-BF16 base revision. Decoder replay used BF16, final normalization accumulated in FP32, and the protected FP16 `lm_head` used the FP16 logit-compute path.
+
+| Metric | Selective-AWQ | BF16 reference |
+| --- | ---: | ---: |
+| Final hidden relative-L2 | 0.0052877845 | 0 by definition |
+| Mean full-vocabulary KLD | 0.2344706438 nats | 0 by definition |
+| Perplexity | 23.73715897 | 23.17865143 |
+| Top-1 agreement with reference | 86.6683% | 100% by definition |
+| Deterministic regression smoke suite | 29/30 | Not reported as a public benchmark |
+
+Held-out token tensor SHA-256:
+
+```text
+77976641882fa641e26f998594aa935518acb7796e9907e7182656b1361015ce
+```
+
+These measurements describe quantization drift on a fixed corpus. They do not claim to reproduce the base model's published software-engineering benchmark scores.
+
+## Artifact integrity
+
+Frozen release checkpoint:
+
+| Field | Value |
+| --- | ---: |
+| Safetensors shards | 49 |
+| Indexed tensors | 99,745 |
+| Tensor payload bytes | 94,283,421,184 |
+| Checkpoint shard bytes | 94,295,590,320 (87.820 GiB) |
+| `model.safetensors.index.json` SHA-256 | `95d151973d8e56372a8f449237b665f08e29cbc249189c02898eaac663eccd62` |
+
+`Checkpoint shard bytes` counts only the 49 safetensors files. The shards and
+model index remain frozen. The release process regenerates the repository-root
+`SHA256SUMS` for the corrected rc4 add-on; a clean-download verification is
+required before the release tag is created.
+
+## Context length
+
+The checkpoint inherits the base model's configured maximum context of 1,048,576 tokens, but that length has not been validated for this exact quantized release. Rc4 passed 4,095-, 15,352-, and 259,989-token needle tests and exposes 395,822 KV tokens at the validated 262,144-token server profile. The 259,989-token run recovered the exact needle with 1,384.602 s TTFT (187.77 prompt tok/s); peak sampled memory was 31,968 MiB used and 527 MiB free on each V100. One-million-token context remains not evaluated.
+
+## Intended use and limitations
+
+This derivative is intended for local or hosted text generation and agentic coding workflows where the selective W4/FP16 trade-off is useful, particularly 4× V100 32GB TP4 deployments.
+
+- The main checkpoint requires BF16 activations; forcing FP16 can overflow at EOS positions.
+- The V100 path requires the version-pinned add-on and must not be assumed compatible with other runtimes.
+- Only routed experts in Layers 5–45 are W4, so this is larger than a uniform all-expert W4 checkpoint by design.
+- The stock-vLLM BF16 compatibility test used heterogeneous TP2 plus 36 GiB CPU offload per GPU at a 512-token profile. It validates native loading and dtype/range behavior, not production speed or long-context capacity on that machine.
+- Transformers/AutoAWQ, SGLang, TensorRT-LLM, standalone RTX 3090, DFlash, MTP, and 1M context are not evaluated.
+- Quantization can change generations even when aggregate numerical metrics remain close.
+
+Follow the original model card's chat-template, thinking-preservation, intended-use, and responsible-use guidance.
+
+## License and attribution
+
+The original model is `poolside/Laguna-S-2.1`, created by Poolside and released under OpenMDW-1.1. This repository retains the complete original `LICENSE.md`, copyright notices, and source notices required by that license.
+
+Quantization and packaging: `leon`
+
+Original model: [poolside/Laguna-S-2.1](https://huggingface.co/poolside/Laguna-S-2.1)
+
+Original license: [OpenMDW-1.1](https://huggingface.co/poolside/Laguna-S-2.1/blob/main/LICENSE.md)
+
+The V100 runtime patch retains the applicable upstream copyright notices and Apache-2.0 license in `v100-1cat/`.
+
+## Release information
+
+- Hugging Face publication target: `leoncca/Laguna-S-2.1-Selective-AWQ`
+- Release tag: `v1.0` after clean-download verification
+- Publisher/quantizer: `leon`
+- Main BF16 validation hardware/runtime: upstream vLLM 0.24.0; RTX 3090 24GB (SM86) + RTX 5060 Ti 16GB (SM120), TP2, 36 GiB CPU offload/GPU; PyTorch 2.11.0+cu130; Transformers 5.14.1; native `LagunaForCausalLM`; `trust_remote_code=False`
+- V100 patched Laguna runtime SHA-256: `144ca4515847be808c12ca06a9d8c563a6be8a15d0c90953c16cfd30c5667fa4`
+- Integrity file: repository-root `SHA256SUMS`; clean-download verification is required before `v1.0` is created

SHA256SUMS

@@ -0,0 +1,70 @@
+ac2ab20fdf15898c5585310c717a6da02ae3314a4690fc2c166f82e12b1babd4  .gitattributes
+49721fe21377cfb692747d273abf7ef74a1684f368b19609090cd0a200073677  EVALUATION.md
+2ab44b68365473c112f5092211a38f231cb23e50de68b75a13369adbd76a74df  LICENSE.md
+108c9d5ad4befda2b90e39ab94b9da6ffd5bc826083c341cab4fd53efaba9a5c  README.md
+5a8d47d38f6b65b640ceeb504266efd3571bb8db2a77bc50890961ffdb9b9d84  SOURCE.md
+444819b8ad4612870827ac05b9147fe9e3344d3850cae8c2790898fc514099ff  chat_template.jinja
+ee36d2434f4b821178937cf722960439702390fc74f8f9cb1a11baaa6baa1a73  config.json
+ef713922daeddd03571647b452f012813b08710a9a0c0529cf0a1077f61529d3  generation_config.json
+599441a338d126518dbdf3bc3787ca6acae1d8413eebe5f2ce7c3bae2160e6be  model-global.safetensors
+f7fbef02ed3ce5b37e35262527e68d7a0e9d8244aa4ad92cb488bdc4eb777f7d  model-layer-00000.safetensors
+6a7489a3a325a97eff14cce5e055280ec87a375647fce53a36ee90f1d32d55ff  model-layer-00001.safetensors
+312fc7ed6a8ad44b7e65bd8dd98ff34a141af7642c96922486da31c245690184  model-layer-00002.safetensors
+ddca9dde8160308a56be9ba2e77da659bb16e4a4cab0318a60e3609472e5039c  model-layer-00003.safetensors
+3ba9db730bf53e69aeaa29299401e9088cda760586e9f54affa364e6be144e8c  model-layer-00004.safetensors
+3330e92d09d1848735914d5529f5091bd44abd0d0c853a20ab8f15c1ecfdad7f  model-layer-00005.safetensors
+ec5316d9f95f4de95c22884540448b03a930e360c1b6c95937f73d245f6665c6  model-layer-00006.safetensors
+b40aea7c670a64acda0d03d57f3116c8fbf71b1b23558cd5864fa6e1ae924fdc  model-layer-00007.safetensors
+e06732c88415be2c5415a7a318426f38d466d7f1fc7510783485b34419765745  model-layer-00008.safetensors
+35caa322f492e0120d1ce4da4e8fb68a40f18ddb58d182d225c471c04fdbf4af  model-layer-00009.safetensors
+dafd5245384114a755c17e07ab245e5914c4b1b304aa32f695d6109f8a38f380  model-layer-00010.safetensors
+9f6c988963eb8c5a8e4af8b33daabe95a522ce531661590854f3e0daebce573b  model-layer-00011.safetensors
+36de4f026dba4169399b24d0179f54318611937c525273faba4df08471536161  model-layer-00012.safetensors
+50618336c04a2b98aa50961b274520b04d89cc2b9605b831288908bb889dacd5  model-layer-00013.safetensors
+3293758717abc17e88b026c978b44fb19888afca4e41519da2feb10ad876c61f  model-layer-00014.safetensors
+a3a41bc9bc7128c9ee3dbc2ce81004aede134ac045ed0c82804a17e69b0379d0  model-layer-00015.safetensors
+94d98597f8444a1a88c56fe76cf9ab2f88d3795605594a6d57a2434d108bb21c  model-layer-00016.safetensors
+8b2451a294a07df422c2f3d1132040d60d62b8ab459705e587d83d86a19e1db4  model-layer-00017.safetensors
+c4dbb15ca1ab682803079f8fbff90334810fefc19d79712523216fa795a758d3  model-layer-00018.safetensors
+729891450bf203b040fb305a12a0e9a7556b489521ad8c7ecfa59f60ff6587d6  model-layer-00019.safetensors
+95541d606cba24d44bd7879f11fda96653fbd55aecec0da61d1b972f841dc61c  model-layer-00020.safetensors
+6e857b6a3a5c43cc05ba3544ed8f5a1a1b4d97ad2da9fe8b2480c9b9e1b76fce  model-layer-00021.safetensors
+0fb6b8736529bf35c0aed4172b70913df1c2e31bb537f3c1903a305684f65026  model-layer-00022.safetensors
+f239f2e15d800e1040a4507d8d323f4e32d850e99abfd6e2512b59b6e0aa6d47  model-layer-00023.safetensors
+24f4882df110522917b72454f326aab6478b7743c9dbf18e91483065e215f561  model-layer-00024.safetensors
+3509305dd38aba8ab78431056eb7c740321f07e9f35a0009168e14a1d41f309b  model-layer-00025.safetensors
+2bfe7c15915efc3cdc1d24a0dda636ede0c7d2d93dba4c9b1912db8731819bb1  model-layer-00026.safetensors
+996c8a23236d34083cf560707468f4e70041e6f000f88b256896f17021ea5cfb  model-layer-00027.safetensors
+c2efb8d642c4c64faef4808d4d8bb3cd5fbc57414a1f6376f4a6a9d9b6413e8b  model-layer-00028.safetensors
+31f4b10ff17f6226ee42ee6904228c04f4866088c5dcf3bfa9967451572d3577  model-layer-00029.safetensors
+619174715bfb454f10243c45160827d0d16836ef9e795bbadc504f62d9fd78e9  model-layer-00030.safetensors
+37ce2303f98e6cb5dc41a38fc5b9a14c6bcbc1b7f2503c8fa9cdd3c609593f56  model-layer-00031.safetensors
+704274442ad16a153f7e0f427e963269481d895efe11dd319ea5955d41ff3ef7  model-layer-00032.safetensors
+fe8fe9aded6dcc74407d20ec76fc7ac71599a42c6441a6ce2680a4141c69289f  model-layer-00033.safetensors
+8cb23551b182bb92f0e49ab4bad1149e43782d476c12b9362689e9656c85c16a  model-layer-00034.safetensors
+f9d5668135419cbad391df2b405400cce2dfccdd1bb3925a5b369d580feb6186  model-layer-00035.safetensors
+ba147cf1e0307a6674497951cf240a4a4fc0e93efc7dc876e21c58911c696fd8  model-layer-00036.safetensors
+24475cbb782635197caee4cd33f7d4b2cc9509e8b14691f2fe87427eb83e0348  model-layer-00037.safetensors
+ec492bfa2931a787081b5cc07004598dcba30d11cf5cbf4cac7483ae1f238e36  model-layer-00038.safetensors
+8b20c1efb28901ff35c213eeb5831b12727b69de0ea9e30671034f47fba5a66c  model-layer-00039.safetensors
+36734192f4d1a4cdd4ac9c2677d47ed6bf01daaeada9c93e111bfcd4772f3171  model-layer-00040.safetensors
+9a9b049b86bd0efc36ef17ebf42958478359913d38e1ba22e8f7ae5deee84916  model-layer-00041.safetensors
+09b54bae43e54d8292b7e3a310f016057663130c43c9e17555754054a0f65e67  model-layer-00042.safetensors
+b4dc8d93733ec75dbe84502918665d5e689c97ba9199b50a3339bab2e2b76e1e  model-layer-00043.safetensors
+d8207b04d73b5f35208ccda6d8238e89bc2adee8516960c2bac30d7517a10b93  model-layer-00044.safetensors
+6072c9369453d199d597e5df3f4cc825b73d0ed197fc29a99efdb913453401d2  model-layer-00045.safetensors
+e4ff6bae1d7194051ae1632cf8b00cfae0746b78be6cb9c763c577a66f5662ca  model-layer-00046.safetensors
+361df0ee52f5e12cb9eb1123a577460e6af4881ad83aaa7164c327bb9ee05c42  model-layer-00047.safetensors
+95d151973d8e56372a8f449237b665f08e29cbc249189c02898eaac663eccd62  model.safetensors.index.json
+3a1d74c8ff1a35ed8e05f02185e2e8e1a60b59415513a9d28f2150a651a296b7  quantization-manifest.json
+794a71715a7116655f71822b84ccf3621722c51867705a06fc037ba2ccd0aa29  quantize_config.json
+466fc850e087d003f0a839d4904a7b4c9854746d7370978228b7238f3d477a9c  tokenizer.json
+36d813a77e02ab0c125e1d5a39d07f67457fb11d46c92fca6396b97e05d2c08e  tokenizer_config.json
+a4381701afa217fd48ff31d02db79fdbe94c336275ae7f822656c20d35b4147b  v100-1cat/Dockerfile
+e90841d1888117d8a7708ec4535958017ace489752f10ceea6100e4d7e2b50a8  v100-1cat/Dockerfile.base
+c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4  v100-1cat/LICENSE-APACHE-2.0
+766c281c1fee419f4acead250b2a383a573f3b71c39bab67ff9b74ed64b0ba94  v100-1cat/README.md
+01e8c414ba5207e07db1cad5e503e4fbb6ab5a37fb3b30d2a7838e9c8f1dc255  v100-1cat/entrypoint.sh
+0d2422eaa0802ed1dd00a6ce1f3c6ba4ec4b343600671b510eccc2ca7712aa08  v100-1cat/install_v100_patch.py
+77b333378e2b87b354411b0222cfd702fe98ab7a2dd8a138d5cc5a699e1eb466  v100-1cat/run.sh
+91ee2d43988c6bb40d6ddd1809773b786c66d2ec330619eff2f4a3a4c7385ea0  v100-1cat/verify_model.py

SOURCE.md

@@ -0,0 +1,14 @@
+# Source and attribution
+
+This repository is an independently produced selective AWQ quantization of
+[poolside/Laguna-S-2.1](https://huggingface.co/poolside/Laguna-S-2.1). It is not an official
+Poolside release.
+
+- Original creator: Poolside
+- Original model: `poolside/Laguna-S-2.1`
+- Frozen source revision: `e80da38da3ed4c4e56888cc1ba39582946a164ba`
+- Original license: OpenMDW-1.1; the complete license is retained as `LICENSE.md`
+- Quantization and packaging: `leon`
+
+The model weights are modified derivative Model Materials. The original source
+and the applicable notice of origin are retained here for redistribution.

chat_template.jinja

@@ -0,0 +1,94 @@
+{#- Iteration on laguna_glm_thinking_v8/chat_template.jinja -#}
+{#- No formatting instructions -#}
+{{- "〈|EOS|〉" -}}
+{%- set enable_thinking = enable_thinking | default(true) -%}
+{%- set add_generation_prompt = add_generation_prompt | default(false) -%}
+{%- set preserve_thinking = preserve_thinking | default(false) -%}
+
+{#- ───── header (system message) ───── -#}
+{#- A caller-supplied system message with empty content opts out of the default below, producing no <system> block — used to train without a system message. -#}
+{%- set system_message = "You are a helpful, conversationally-fluent assistant made by Poolside. You are here to be helpful to users through natural language conversations." -%}
+{%- if messages and messages[0].role == "system" -%}
+  {%- set system_message = messages[0].content -%}
+  {%- set messages = messages[1:] -%}
+{%- endif -%}
+
+{%- set has_sys = system_message and system_message.strip() -%}
+{%- if has_sys or tools or enable_thinking -%}
+  {{- "<system>" -}}
+
+  {%- if has_sys -%}
+    {{- system_message.rstrip() -}}
+    {%- if tools -%}{{- "\n\n" -}}{%- endif -%}
+  {%- endif -%}
+
+  {%- if tools -%}
+    {{- "### Tools\n\n" -}}
+    {{- "You may call functions to assist with the user query.\n" -}}
+    {{- "All available function signatures are listed below:\n" -}}
+    {{- "<available_tools>\n" -}}
+    {%- for tool in tools -%}
+      {{- (tool | tojson) ~ "\n" -}}
+    {%- endfor -%}
+    {{- "</available_tools>" -}}
+  {%- endif -%}
+
+  {{- "</system>\n" -}}
+{%- endif -%}
+
+{#- ───── main loop ───── -#}
+{%- for message in messages -%}
+  {%- set content = message.content if message.content is string else "" -%}
+  {%- if message.role == "user" -%}
+    {{- "<user>" + content + "</user>\n" -}}
+  {%- elif message.role == "assistant" -%}
+    {%- generation -%}
+      {{- "<assistant>" -}}
+      {#- Extract reasoning content from message.reasoning (vLLM field name) or message.reasoning_content -#}
+      {%- set reasoning_content = '' -%}
+      {%- if message.reasoning is string -%}
+        {%- set reasoning_content = message.reasoning -%}
+      {%- elif message.reasoning_content is string -%}
+        {%- set reasoning_content = message.reasoning_content -%}
+      {%- endif -%}
+      {#- Display reasoning content for all messages if enable_thinking -#}
+      {%- if enable_thinking or preserve_thinking -%}
+        {{- '<think>' + reasoning_content + '</think>' -}}
+      {%- else -%}
+        {{- '</think>' -}}
+      {%- endif -%}
+      {#- Display main content (trailing newline only when no tool_calls follow) -#}
+      {%- if content -%}
+        {{- content -}}
+      {%- endif -%}
+      {%- if message.tool_calls -%}
+        {%- for tool_call in message.tool_calls -%}
+          {%- set function_data = tool_call.function -%}
+          {{- '<tool_call>' + function_data.name -}}
+          {%- set _args = function_data.arguments -%}
+          {%- for k, v in _args.items() -%}
+            {{- "<arg_key>" ~ k ~ "</arg_key>" -}}
+            {{- "<arg_value>" -}}{{- v | tojson(ensure_ascii=False) if v is not string else v -}}{{- "</arg_value>" -}}
+          {%- endfor -%}
+          {{- "</tool_call>" -}}
+        {%- endfor -%}
+      {%- endif -%}
+      {{- "</assistant>\n" -}}
+    {%- endgeneration -%}
+  {%- elif message.role == "tool" -%}
+    {{- "<tool_response>" + content + "</tool_response>\n" -}}
+  {%- elif message.role == "system" -%}
+    {#- Render additional system messages (the first one, if any, is handled separately in the header and was sliced off above) -#}
+    {{- "<system>" + content + "</system>\n" -}}
+  {%- endif -%}
+{%- endfor -%}
+{#- ───── generation prompt ───── -#}
+{%- if add_generation_prompt -%}
+  {{- "<assistant>" -}}
+  {#- ───── Include reasoning mode directive ───── -#}
+  {%- if enable_thinking -%}
+    {{- '<think>' -}}
+  {%- else -%}
+    {{- '</think>' -}}
+  {%- endif -%}
+{%- endif -%}

config.json

@@ -0,0 +1,297 @@
+{
+  "architectures": [
+    "LagunaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 2,
+  "decoder_sparse_step": 1,
+  "dtype": "bfloat16",
+  "eos_token_id": [
+    2,
+    24
+  ],
+  "gating": "per-head",
+  "gating_types": [
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head",
+    "per_head"
+  ],
+  "head_dim": 128,
+  "hidden_act": "silu",
+  "hidden_size": 3072,
+  "initializer_range": 0.02,
+  "intermediate_size": 12288,
+  "layer_types": [
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "full_attention",
+    "sliding_attention",
+    "sliding_attention",
+    "sliding_attention"
+  ],
+  "max_position_embeddings": 1048576,
+  "mlp_layer_types": [
+    "dense",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse",
+    "sparse"
+  ],
+  "mlp_only_layers": [
+    0
+  ],
+  "model_type": "laguna",
+  "moe_apply_router_weight_on_input": false,
+  "moe_intermediate_size": 1024,
+  "moe_routed_scaling_factor": 2.5,
+  "moe_router_logit_softcapping": 0.0,
+  "norm_topk_prob": true,
+  "num_attention_heads": 48,
+  "num_attention_heads_per_layer": [
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72,
+    48,
+    72,
+    72,
+    72
+  ],
+  "num_experts": 256,
+  "num_experts_per_tok": 10,
+  "num_hidden_layers": 48,
+  "num_key_value_heads": 8,
+  "output_router_logits": false,
+  "pad_token_id": 9,
+  "partial_rotary_factor": null,
+  "qkv_bias": false,
+  "quantization_config": {
+    "bits": 4,
+    "group_size": 32,
+    "modules_to_not_convert": [
+      "model.embed_tokens",
+      "lm_head",
+      "model.norm",
+      "self_attn",
+      "input_layernorm",
+      "post_attention_layernorm",
+      "mlp.gate",
+      "e_score_correction_bias",
+      "shared_expert",
+      "model.layers.0.mlp",
+      "model.layers.1.mlp.experts",
+      "model.layers.2.mlp.experts",
+      "model.layers.3.mlp.experts",
+      "model.layers.4.mlp.experts",
+      "model.layers.46.mlp.experts",
+      "model.layers.47.mlp.experts"
+    ],
+    "quant_method": "awq",
+    "version": "gemm",
+    "zero_point": true
+  },
+  "rms_norm_eps": 1e-06,
+  "rope_parameters": {
+    "full_attention": {
+      "attention_factor": 1.4852030263919618,
+      "beta_fast": 32.0,
+      "beta_slow": 1.0,
+      "factor": 128.0,
+      "original_max_position_embeddings": 8192,
+      "partial_rotary_factor": 0.5,
+      "rope_theta": 500000.0,
+      "rope_type": "yarn"
+    },
+    "rope_theta": 10000.0,
+    "rope_type": "default",
+    "sliding_attention": {
+      "partial_rotary_factor": 1.0,
+      "rope_theta": 10000.0,
+      "rope_type": "default"
+    }
+  },
+  "router_aux_loss_coef": 0.0,
+  "shared_expert_intermediate_size": 1024,
+  "sliding_window": 512,
+  "swa_attention_sink_enabled": false,
+  "swa_rope_parameters": {
+    "partial_rotary_factor": 1.0,
+    "rope_theta": 10000.0,
+    "rope_type": "default"
+  },
+  "tie_word_embeddings": false,
+  "transformers_version": "5.14.1",
+  "use_cache": false,
+  "vocab_size": 100352
+}

generation_config.json

@@ -0,0 +1,25 @@
+{
+  "bos_token_id": 2,
+  "default_chat_template_kwargs": {
+    "enable_thinking": true
+  },
+  "do_sample": true,
+  "eos_token_id": [
+    2,
+    24
+  ],
+  "min_p": 0.0,
+  "pad_token_id": 9,
+  "reasoning_parser": "poolside_v1",
+  "speculative_config": {
+    "method": "dflash",
+    "model": "poolside/Laguna-S-2.1-DFlash",
+    "num_speculative_tokens": 15,
+    "source": "huggingface"
+  },
+  "temperature": 1.0,
+  "tool_call_parser": "poolside_v1",
+  "top_k": 20,
+  "top_p": 1.0,
+  "transformers_version": "5.14.1"
+}

model-global.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:599441a338d126518dbdf3bc3787ca6acae1d8413eebe5f2ce7c3bae2160e6be
+size 1233131840

model-layer-00000.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f7fbef02ed3ce5b37e35262527e68d7a0e9d8244aa4ad92cb488bdc4eb777f7d
+size 314881880

model-layer-00001.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6a7489a3a325a97eff14cce5e055280ec87a375647fce53a36ee90f1d32d55ff
+size 4978666704

model-layer-00002.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:312fc7ed6a8ad44b7e65bd8dd98ff34a141af7642c96922486da31c245690184
+size 4978666704

model-layer-00003.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ddca9dde8160308a56be9ba2e77da659bb16e4a4cab0318a60e3609472e5039c
+size 4978666704

model-layer-00004.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3ba9db730bf53e69aeaa29299401e9088cda760586e9f54affa364e6be144e8c
+size 4940770512

model-layer-00005.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3330e92d09d1848735914d5529f5091bd44abd0d0c853a20ab8f15c1ecfdad7f
+size 1543715888

model-layer-00006.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec5316d9f95f4de95c22884540448b03a930e360c1b6c95937f73d245f6665c6
+size 1543715888

model-layer-00007.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b40aea7c670a64acda0d03d57f3116c8fbf71b1b23558cd5864fa6e1ae924fdc
+size 1543715888

model-layer-00008.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e06732c88415be2c5415a7a318426f38d466d7f1fc7510783485b34419765745
+size 1505819696

model-layer-00009.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:35caa322f492e0120d1ce4da4e8fb68a40f18ddb58d182d225c471c04fdbf4af
+size 1543715832

model-layer-00010.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:dafd5245384114a755c17e07ab245e5914c4b1b304aa32f695d6109f8a38f380
+size 1543718152

model-layer-00011.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:9f6c988963eb8c5a8e4af8b33daabe95a522ce531661590854f3e0daebce573b
+size 1543718152

model-layer-00012.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:36de4f026dba4169399b24d0179f54318611937c525273faba4df08471536161
+size 1505821960

model-layer-00013.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:50618336c04a2b98aa50961b274520b04d89cc2b9605b831288908bb889dacd5
+size 1543718152

model-layer-00014.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3293758717abc17e88b026c978b44fb19888afca4e41519da2feb10ad876c61f
+size 1543718152

model-layer-00015.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a3a41bc9bc7128c9ee3dbc2ce81004aede134ac045ed0c82804a17e69b0379d0
+size 1543718152

model-layer-00016.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:94d98597f8444a1a88c56fe76cf9ab2f88d3795605594a6d57a2434d108bb21c
+size 1505821960

model-layer-00017.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b2451a294a07df422c2f3d1132040d60d62b8ab459705e587d83d86a19e1db4
+size 1543718152

model-layer-00018.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c4dbb15ca1ab682803079f8fbff90334810fefc19d79712523216fa795a758d3
+size 1543718152

model-layer-00019.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:729891450bf203b040fb305a12a0e9a7556b489521ad8c7ecfa59f60ff6587d6
+size 1543718152

model-layer-00020.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:95541d606cba24d44bd7879f11fda96653fbd55aecec0da61d1b972f841dc61c
+size 1505821960

model-layer-00021.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6e857b6a3a5c43cc05ba3544ed8f5a1a1b4d97ad2da9fe8b2480c9b9e1b76fce
+size 1543718152

model-layer-00022.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0fb6b8736529bf35c0aed4172b70913df1c2e31bb537f3c1903a305684f65026
+size 1543718152

model-layer-00023.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f239f2e15d800e1040a4507d8d323f4e32d850e99abfd6e2512b59b6e0aa6d47
+size 1543718152

model-layer-00024.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24f4882df110522917b72454f326aab6478b7743c9dbf18e91483065e215f561
+size 1505821960

model-layer-00025.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:3509305dd38aba8ab78431056eb7c740321f07e9f35a0009168e14a1d41f309b
+size 1543718152

model-layer-00026.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:2bfe7c15915efc3cdc1d24a0dda636ede0c7d2d93dba4c9b1912db8731819bb1
+size 1543718152

model-layer-00027.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:996c8a23236d34083cf560707468f4e70041e6f000f88b256896f17021ea5cfb
+size 1543718152

model-layer-00028.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c2efb8d642c4c64faef4808d4d8bb3cd5fbc57414a1f6376f4a6a9d9b6413e8b
+size 1505821960

model-layer-00029.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:31f4b10ff17f6226ee42ee6904228c04f4866088c5dcf3bfa9967451572d3577
+size 1543718152

model-layer-00030.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:619174715bfb454f10243c45160827d0d16836ef9e795bbadc504f62d9fd78e9
+size 1543718152

model-layer-00031.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:37ce2303f98e6cb5dc41a38fc5b9a14c6bcbc1b7f2503c8fa9cdd3c609593f56
+size 1543718152

model-layer-00032.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:704274442ad16a153f7e0f427e963269481d895efe11dd319ea5955d41ff3ef7
+size 1505821960

model-layer-00033.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:fe8fe9aded6dcc74407d20ec76fc7ac71599a42c6441a6ce2680a4141c69289f
+size 1543718152

model-layer-00034.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8cb23551b182bb92f0e49ab4bad1149e43782d476c12b9362689e9656c85c16a
+size 1543718152

model-layer-00035.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:f9d5668135419cbad391df2b405400cce2dfccdd1bb3925a5b369d580feb6186
+size 1543718152

model-layer-00036.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ba147cf1e0307a6674497951cf240a4a4fc0e93efc7dc876e21c58911c696fd8
+size 1505821960

model-layer-00037.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:24475cbb782635197caee4cd33f7d4b2cc9509e8b14691f2fe87427eb83e0348
+size 1543718152

model-layer-00038.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:ec492bfa2931a787081b5cc07004598dcba30d11cf5cbf4cac7483ae1f238e36
+size 1543718152

model-layer-00039.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:8b20c1efb28901ff35c213eeb5831b12727b69de0ea9e30671034f47fba5a66c
+size 1543718152