Add INT4 MLX model card and metadata

LICENSE

@@ -0,0 +1,84 @@
+MiniMax H3 COMMUNITY LICENSE AGREEMENT
+MiniMax H3 release date/License date: August 2, 2026.
+The scope of this License Agreement (this “Agreement”) is expressly limited to the “Applicable Territory” as defined below.
+By clicking to accept, or by using, reproducing, modifying, distributing, running, or displaying any portion or element of the MiniMax H3 Works (including through any Hosted Services) in any manner, you acknowledge and accept the terms of this Agreement, and this Agreement shall take immediate effect upon the occurrence of such act.
+I.  Definitions
+1. “Acceptable Use Policy” means the policy published by MiniMax in Exhibit A.
+2. “Agreement” means the terms and conditions set forth herein that govern the use, reproduction, distribution, modification, running, and display of the MiniMax H3 Works or any portion or element thereof.
+3. “Applicable Territory” means worldwide, excluding the Excluded Territories.
+4. “Documentation” means the specifications, manuals, and documentation concerning MiniMax H3 that are publicly released by MiniMax.
+5. “Excluded Territories” means the European Union, the United Kingdom, the Republic of Korea and the United States of America.  
+6. “MiniMax H3” means the video generation model, together with its software and algorithms, including trained model weights, parameters (including optimizer states), machine-learning model code, inference-supporting code, and other elements thereof made publicly available by Us, as released at https://huggingface.co/MiniMaxAI/MiniMax-H3.
+7. “MiniMax H3 Works” means (i) the Materials, (ii) the Model Derivatives, and (iii) all derivatives thereof.
+8. “Hosted Services” means hosted services provided via application programming interfaces (APIs), web access, or any other electronic or remote means.
+9. “Licensee,” “you,” or “your” means the natural or legal person exercising rights and/or using the MiniMax H3 Works for any purpose in any field of use under this Agreement.
+10. “Materials” means, collectively, MiniMax H3 and the Documentation (and any portion thereof), in each case as made available by MiniMax under this Agreement and proprietary to MiniMax.
+11. “Model Derivatives” means all of the following: (i) any modification of MiniMax H3 or any Model Derivative thereof; (ii) any work based on MiniMax H3 or any Model Derivative thereof; or (iii) any other machine learning model created by transferring the patterns of the weights, parameters, operational patterns, or Outputs of MiniMax H3 or any Model Derivative thereof to another model, such that the latter model exhibits behavior similar to MiniMax H3 or its Model Derivatives, including by distillation methods, methods using intermediate data representations, or methods based on training using synthetic-data Outputs generated by MiniMax H3 or its Model Derivatives. For the avoidance of doubt, Outputs are not deemed Model Derivatives.
+12. “Output” means any result of operating or otherwise using MiniMax H3 or any Model Derivatives (including through Hosted Services).
+13. “Third Party” means any natural or legal person that is not under common control with us or with you.
+14. “Including” means “including but not limited to.”
+15. “We,” “Us”  or  “MiniMax” means Nanonoble Pte. Ltd..
+II.  Grant of Rights
+Solely within the Applicable Territory, we grant you a non-exclusive, non-transferable, royalty-free, limited license to use, reproduce, distribute, create derivative works (including Model Derivatives), and modify the Materials in accordance with the terms of this Agreement and the Acceptable Use Policy, based on the intellectual property and other rights owned by MiniMax that are embodied in or used by the Materials. You shall not violate (or encourage or permit any person to violate) any term of this Agreement or the Acceptable Use Policy.
+We will continuously evaluate the applicable laws, regulations and compliance requirements for the Excluded Territories. In the meantime, should any person in such Excluded Territories be interested in deploying our models, you are welcome to contact us about obtaining a license, which will be granted based on robust controls and guardrails for purposes of complying with the laws, regulations and compliance requirements of the Excluded Territories.
+III.  Distribution and Redistribution
+Subject to and conditioned on your continuing compliance with this Agreement, including its territorial restrictions and the Acceptable Use Policy, and solely within the Applicable Territory, you may distribute or make available the MiniMax H3 Works to Third Parties within the Applicable Territory; provided, that all of the following conditions are met:
+1. You must provide a copy of this Agreement to all such Third Parties who receive the MiniMax H3 Works or use your products or services related thereto;
+2. You must cause any modified files to carry prominent notices stating that you have modified such files;
+3. You are encouraged to:
+  a. display a notice on any product or service developed using MiniMax H3 indicating that the product or service is “Powered by MiniMax H3”; 
+  b. add an AI-generation identifier to files produced using generative AI models including MiniMax H3; and
+  c. publish at least one technical blog post or a public statement describing your experience using MiniMax H3 Works;
+4. All distributions to Third Parties (other than through Hosted Services) must be accompanied by a “NOTICE” text file containing the following notice:
+  “MiniMax H3 is licensed under the MiniMax H3 Community License Agreement, Copyright © 2026 MiniMax. All Rights Reserved.”
+You may add your own copyright notices on your modifications; except as provided in this Section and in Section V, however, you may not impose additional or different terms and conditions on the use, reproduction, or distribution of your modifications or of any aggregate Model Derivatives, and your use, reproduction, modification, distribution, running, and display of the work must otherwise comply with the terms and conditions of this Agreement (including the provisions concerning the Applicable Territory). If you receive the MiniMax H3 Works from a Licensee as part of an integrated end-user product, the provisions of Section III of this Agreement do not apply to you, but Section V and Exhibit A remain applicable.
+IV.  Additional Commercial Terms
+1. You shall obtain a separate, prior written authorization from MiniMax by contacting api@minimax.io with the subject line “MiniMax H3 licensing - authorization request”, if your commercial products and services generate more than 20 million US dollars (or equivalent in other currencies) in yearly revenue.
+2. You shall prominently display “MiniMax H3”on the user interface of commercial product or service that uses MiniMax H3 or MiniMax H3 Works.
+V. Use Restrictions
+1. Your use of the MiniMax H3 Works must comply with applicable laws and regulations (including trade-compliance laws and regulations) and must comply with the Acceptable Use Policy for the MiniMax H3 Works, which is incorporated into this Agreement by reference. 
+2. Before providing access to the MiniMax H3 Works or any product, service, or Hosted Service incorporating them, you must bind each recipient or user to enforceable terms at least as protective as the use restrictions in this Section V and Exhibit A, and you must notify each recipient or user that those restrictions apply.
+3. You may not use the MiniMax H3 Works or any of their Outputs or results to improve any other artificial intelligence model (other than MiniMax H3 or its Model Derivatives).
+4. You may not use, reproduce, modify, distribute, or display the MiniMax H3 Works or any of their Outputs or results outside the Applicable Territory. Any such use outside the Applicable Territory is not authorized by this Agreement.
+5. If you provide or make available to any Third Party a product, service, or Hosted Service that permits the generation of Outputs using MiniMax H3 or any Model Derivative, you must, before making that product or service available and throughout its operation, implement, maintain, test, and periodically review reasonable and proportionate technical and organizational safeguards designed to prevent and mitigate access, uses, and Outputs that violate this Section V or Exhibit A, including uses or Outputs that infringe, misappropriate, or otherwise violate any Third Party’s intellectual-property or other rights. You must not knowingly disable, materially weaken, or permit the circumvention of those safeguards. You must maintain a reasonably accessible mechanism for reporting suspected violations. Upon receiving a good-faith report or otherwise obtaining actual knowledge of a violation, you must promptly investigate and take reasonable steps within your control to stop or mitigate the violation, including removing or disabling access to offending content or services and suspending or terminating repeat violators where appropriate. You are responsible for implementing and enforcing these requirements with respect to your products, services, systems, users, and downstream recipients.
+VI.  Intellectual Property
+1. Subject to MiniMax’s rights in the MiniMax H3 Works (and the intellectual property therein), and to your compliance with the terms and conditions of this Agreement, as between you and MiniMax, you will own the derivative works and modifications of the Materials that you have created or had created, as well as any Model Derivatives.
+2. Except for the limited license expressly granted in this paragraph, no trademark license is granted under this Agreement; with respect to MiniMax H3 Works, the Licensee may not use any name or mark owned by or associated with MiniMax or any of its affiliates, except as reasonably and customarily necessary to describe and distribute the MiniMax H3 Works. MiniMax hereby grants you a license to use the “MiniMax H3” mark (the “Mark”) within the Applicable Territory solely for the purpose of complying with Section III.3; provided, that you comply with all applicable trademark-protection laws. All goodwill arising from your use of the Mark shall inure to the benefit of MiniMax.
+3. If you bring or assert any suit or other legal proceeding (including a cross-claim or counterclaim in any action) against us or any other natural or legal person alleging that the Materials, any Output, or any portion of the foregoing infringes any intellectual property right or other right owned by you or for which you can obtain a license, all licenses granted to you under this Agreement will terminate as of the date such suit or proceeding is filed. You shall defend, indemnify, and hold us harmless against any Third-Party claim arising out of or related to the use or distribution of the MiniMax H3 Works by you or by any Third Party.
+4. MiniMax claims no rights over the Outputs you generate. You and your users are entirely responsible for the Outputs and any subsequent use thereof.
+VII.  Disclaimers and Limitations of Liability
+1. We have no obligation to support, update, provide training for, or develop any further version of the MiniMax H3 Works, or to grant any license with respect thereto.
+2. UNLESS AND ONLY TO THE EXTENT REQUIRED BY APPLICABLE LAW, THE MINIMAX H3 WORKS AND ANY OUTPUT AND RESULTS THEREFROM ARE PROVIDED “AS IS” WITHOUT ANY EXPRESS OR IMPLIED WARRANTIES OF ANY KIND INCLUDING ANY WARRANTIES OF TITLE, MERCHANTABILITY, NONINFRINGEMENT, COURSE OF DEALING, USAGE OF TRADE, OR FITNESS FOR A PARTICULAR PURPOSE. YOU ARE SOLELY RESPONSIBLE FOR DETERMINING THE APPROPRIATENESS OF USING, REPRODUCING, MODIFYING, PERFORMING, DISPLAYING OR DISTRIBUTING ANY OF THE MINIMAX H3 WORKS OR OUTPUTS AND ASSUME ANY AND ALL RISKS ASSOCIATED WITH YOUR OR A THIRD PARTY’S USE OR DISTRIBUTION OF ANY OF THE MINIMAX H3 WORKS OR OUTPUTS AND YOUR EXERCISE OF RIGHTS AND PERMISSIONS UNDER THIS AGREEMENT.
+3. TO THE FULLEST EXTENT PERMITTED BY APPLICABLE LAW, IN NO EVENT SHALL MINIMAX OR ITS AFFILIATES BE LIABLE UNDER ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, TORT, NEGLIGENCE, PRODUCTS LIABILITY, OR OTHERWISE, FOR ANY DAMAGES, INCLUDING ANY DIRECT, INDIRECT, SPECIAL, INCIDENTAL, EXEMPLARY, CONSEQUENTIAL OR PUNITIVE DAMAGES, OR LOST PROFITS OF ANY KIND ARISING FROM THIS AGREEMENT OR RELATED TO ANY OF THE MINIMAX H3 WORKS OR OUTPUTS, EVEN IF MINIMAX OR ITS AFFILIATES HAVE BEEN ADVISED OF THE POSSIBILITY OF ANY OF THE FOREGOING.
+VIII.  Term and Termination
+1. This Agreement is effective from the moment you accept this Agreement or begin accessing the Materials, and, subject to your compliance with its terms and conditions, will remain in effect until terminated as provided herein.
+2. If you breach any term or condition of this Agreement, we have the right to terminate this Agreement. Upon termination, you must immediately cease accessing, using, and distributing the MiniMax H3 Works; delete or destroy all copies within your possession or control; and notify each downstream recipient that your authorization has ended. The obligations in the preceding sentence and Sections VI.1, VI.3, VII, and IX survive termination.
+IX.  Governing Law and Jurisdiction
+1. This Agreement, and any dispute arising out of or related to this Agreement, shall be governed by the laws of the Hong Kong Special Administrative Region of the People’s Republic of China, without regard to its conflict-of-laws rules. The United Nations Convention on Contracts for the International Sale of Goods does not apply to this Agreement.
+2. Any dispute arising out of or related to this Agreement shall be subject to the exclusive jurisdiction of the courts of the Hong Kong Special Administrative Region of the People’s Republic of China with competent jurisdiction. Both MiniMax and the Licensee hereby consent to the exclusive jurisdiction of such courts for any such dispute.
+Additional Note: Please note that the encoder of MiniMax H3 uses Qwen3-VL-32B, which is licensed under Apache 2.0 License: https://github.com/QwenLM/Qwen3-VL/blob/main/LICENSE.
+
+Exhibit A — Acceptable Use Policy
+MiniMax reserves the right to update this Acceptable Use Policy from time to time. 
+Last revised: August 2, 2026.
+MiniMax is committed to promoting the safe and fair use of its tools and features, including MiniMax H3. You agree not to use MiniMax H3, any Model Derivatives, or any Output in any of the following ways:
+1. Use outside the Applicable Territory;
+2. Use in any manner that violates any applicable national, federal, state, local, or international law, regulation, or other legal requirement, or that infringes, misappropriates, or otherwise violates any Third Party’s intellectual-property or other proprietary rights, including through unauthorized reproduction, distribution, public display, public performance, or creation of derivative works;
+3. Use in any manner that may harm yourself or others;
+4. Use to repurpose or distribute the Outputs of MiniMax H3 or any Model Derivatives in order to harm yourself or others;
+5. Use to circumvent or bypass any safety guardrails or safeguards we have implemented;
+6. Use in any manner that exploits or harms, or intends to exploit or harm, minors;
+7. Use to generate or disseminate verifiably false information and/or content for the purpose of harming others or influencing elections;
+8. Use to manufacture or facilitate false online engagement, including fake reviews and other means of false online engagement;
+9. Use to intentionally defame, disparage, or otherwise harass others;
+10. Use to generate and/or disseminate malware (including ransomware) or any other content intended to damage electronic systems;
+11. Use to generate or disseminate personally identifiable information for the purpose of harming others;
+12. Use to generate or disseminate information (including images, code, posts, or articles) in or to any public environment (including via bot tweets or similar means) without clearly and prominently disclosing that such information and/or content is machine-generated;
+13. Use to impersonate another person without that person’s consent, authorization, or lawful right to do so;
+14. Use to make high-risk automated decisions in critical domains that affect individual safety, rights, or well-being (such as law enforcement, immigration, healthcare or medical services, critical-infrastructure management, product-safety components, essential services, credit, employment, housing, education, social scoring, or insurance);
+15. Use in any manner that violates or disregards the social, ethical, or moral standards of other countries or regions;
+16. Use to carry out, assist, threaten, incite, plan, advocate for, or encourage violent extremism or terrorism;
+17. Use for any purpose intended to discriminate against, or harm, individuals or groups based on protected characteristics or categories, online or offline social behavior, or known or predicted personality traits;
+18. Use to intentionally exploit the vulnerabilities of specific populations based on age, social, physical, or psychological characteristics, so as to materially distort the behavior of a member of that group in a manner that causes, or is likely to cause, physical or psychological harm to that person or to others;
+19. Use for military purposes;
+20. Use to engage in any unauthorized or unlicensed professional activity, including but not limited to financial, legal, medical or healthcare, or other professional practice.

README.md

@@ -0,0 +1,86 @@
+---
+license: other
+license_name: minimax-h3-community
+license_link: https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree/blob/main/LICENSE
+base_model: FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree
+library_name: fastvideo
+pipeline_tag: text-to-video
+tags:
+- mlx
+- apple-silicon
+- minimax-h3
+- fasth3
+- text-to-video
+- audio
+- quantized
+- int4
+---
+
+# FastH3 Preview v1 Dense DataFree MLX INT4
+
+This repository contains the ready-to-run **INT4 MLX DiT** for
+[FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree](https://huggingface.co/FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree). It avoids the local
+conversion step and its temporary unified-memory requirement on Apple silicon.
+
+The checkpoint is the dense-attention, step-1000, four-forward Preview v1 model.
+Quantization is affine, weight-only INT4 with group size 64. Activations
+remain BF16. This export is dense-only and does not support `--vsa`.
+
+## Files
+
+| File | Purpose |
+| --- | --- |
+| `mlx_h3_dit.safetensors` | Quantized H3 DiT weights, 10.74 GiB |
+| `mlx_h3_dit.json` | Architecture, quantization, and cached AdaLN schedule |
+| `conversion_manifest.json` | Source revision, converter version, validation, and checksums |
+
+Weights SHA-256: `dcee86c17197c621a0e3d3ba9541cb6ef5764cd0c258beb7b044fb12b217e948`
+
+## Download
+
+Download the shared H3 components once. They include the tokenizer, Qwen3-VL
+text encoder, video VAE, and audio VAE:
+
+```bash
+hf download FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree \
+  --revision f624f08c6c279ab43534c003e556fc5b295b6558 \
+  --local-dir ./FastH3-Preview-v1-Dense-DataFree
+```
+
+Then download this ready-to-run MLX DiT:
+
+```bash
+hf download FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree-MLX-INT4 \
+  --local-dir ./FastH3-MLX-INT4
+```
+
+## Run on Apple silicon
+
+From a current FastVideo checkout:
+
+```bash
+uv venv --python 3.12 --seed
+source .venv/bin/activate
+uv pip install -e ".[mlx]"
+```
+
+```bash
+python examples/inference/basic/mlx_fasth3.py \
+  --model-root ./FastH3-Preview-v1-Dense-DataFree \
+  --mlx-checkpoint ./FastH3-MLX-INT4 \
+  --prompt '(S1) A presenter says <d>[English] Fast H3 runs on Apple silicon.</d>' \
+  --height 480 --width 832 --num-frames 124 --steps 4 --seed 2026 \
+  --output-path ./outputs/fasth3_int4.mp4
+```
+
+The MLX runtime loads one heavyweight component at a time. The source model's
+MiniMax H3 Community License applies to this converted checkpoint.
+
+## Validation and provenance
+
+The conversion used FastVideo commit
+`cf6a00b9be4675602126d6aeab902ad9a74810ea` with MLX
+`0.32.2` on an Apple M4 Max. FastVideo
+verified all 13 source transformer shards, opened the safetensors header,
+validated 1,464 tensors and the quantization manifest, loaded the checkpoint,
+and completed a 124-frame 832x480 generation with the full H3 VAE.

conversion_manifest.json

@@ -0,0 +1,49 @@
+{
+  "schema_version": "fastvideo-fasth3-mlx-conversion-v1",
+  "published_repo_id": "FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree-MLX-INT4",
+  "source": {
+    "repo_id": "FastVideo/FastVideo-FastH3-4-step-Preview-v1-Dense-DataFree",
+    "revision": "f624f08c6c279ab43534c003e556fc5b295b6558",
+    "checkpoint_step": 1000,
+    "attention": "dense",
+    "checkpoint_content_aggregate_sha256": "3f56129acb9749f2ae4e52bab176c09406bf027d68c6f020cf5ff99e651b1609",
+    "transformer_shards_verified": 13
+  },
+  "converter": {
+    "fastvideo_commit": "cf6a00b9be4675602126d6aeab902ad9a74810ea",
+    "script": "scripts/checkpoint_conversion/convert_minimax_h3_mlx.py",
+    "command": "python scripts/checkpoint_conversion/convert_minimax_h3_mlx.py --model-root <snapshot>/transformer --out <output> --formats 'int8 int6 int4'",
+    "mlx_version": "0.32.2",
+    "python_version": "3.12.13",
+    "hardware": "Apple M4 Max",
+    "unified_memory_bytes": 38654705664,
+    "combined_wall_seconds": 164.62,
+    "process_max_rss_bytes": 11317051392,
+    "peak_memory_footprint_bytes": 27108979504
+  },
+  "format": {
+    "conversion_seconds": 46.1,
+    "peak_mlx_gib": 14.8,
+    "weights_file": "mlx_h3_dit.safetensors",
+    "weights_bytes": 11532219083,
+    "weights_sha256": "dcee86c17197c621a0e3d3ba9541cb6ef5764cd0c258beb7b044fb12b217e948",
+    "manifest_file": "mlx_h3_dit.json",
+    "manifest_sha256": "25c89f37f53292c4abd8bbb70098eaad54783a1b1fc5f0f55ed779a8c85efb56",
+    "quantization": {
+      "mode": "affine",
+      "bits": 4,
+      "group_size": 64
+    },
+    "tensor_count": 1464,
+    "vsa_capable": false
+  },
+  "validation": {
+    "source_shard_size_and_sha256": "passed, 13 of 13",
+    "safetensors_header_open": "passed",
+    "manifest_quantization": "passed",
+    "adaln_cache_timesteps": 8,
+    "generation": "passed: 832x480, 124 frames, full H3 VAE",
+    "runtime_checkpoint_load": "passed for int8, int6, and int4",
+    "runtime_structure": "50 transformer blocks, 2 refiner blocks, 50 AdaLN block tables"
+  }
+}

mlx_h3_dit.json

@@ -0,0 +1,1307 @@
+{
+  "format_version": 1,
+  "config": {
+    "_class_name": "MiniMaxH3Transformer3DModel",
+    "_diffusers_version": "0.36.0.dev0",
+    "num_attention_heads": 56,
+    "attention_head_dim": 128,
+    "hidden_size": 5376,
+    "num_layers": 50,
+    "num_refiner_layers": 2,
+    "ffn_dim": 14336,
+    "in_channels": 24,
+    "audio_in_channels": 32,
+    "patch_size": [
+      1,
+      2,
+      2
+    ],
+    "text_dim": 5120,
+    "freq_dim": 256,
+    "time_embed_hidden_dim": 5376,
+    "time_embed_dim": 2688,
+    "rope_freq_dim": 16,
+    "rope_theta": 10000.0,
+    "norm_eps": 1e-05,
+    "qk_norm_eps": 1e-05,
+    "final_norm_eps": 1e-05
+  },
+  "num_blocks": 50,
+  "num_refiner_blocks": 2,
+  "quantization": {
+    "mode": "affine",
+    "bits": 4,
+    "group_size": 64
+  },
+  "quantized_keys": {
+    "blocks.0.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.0.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.0.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.0.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.0.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.0.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.1.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.1.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.1.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.1.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.1.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.1.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.2.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.2.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.2.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.2.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.2.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.2.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.3.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.3.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.3.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.3.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.3.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.3.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.4.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.4.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.4.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.4.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.4.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.4.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.5.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.5.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.5.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.5.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.5.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.5.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.6.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.6.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.6.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.6.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.6.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.6.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.7.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.7.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.7.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.7.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.7.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.7.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.8.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.8.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.8.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.8.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.8.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.8.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.9.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.9.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.9.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.9.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.9.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.9.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.10.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.10.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.10.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.10.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.10.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.10.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.11.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.11.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.11.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.11.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.11.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.11.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.12.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.12.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.12.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.12.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.12.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.12.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.13.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.13.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.13.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.13.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.13.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.13.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.14.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.14.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.14.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.14.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.14.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.14.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.15.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.15.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.15.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.15.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.15.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.15.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.16.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.16.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.16.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.16.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.16.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.16.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.17.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.17.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.17.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.17.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.17.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.17.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.18.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.18.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.18.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.18.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.18.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.18.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.19.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.19.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.19.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.19.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.19.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.19.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.20.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.20.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.20.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.20.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.20.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.20.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.21.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.21.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.21.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.21.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.21.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.21.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.22.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.22.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.22.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.22.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.22.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.22.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.23.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.23.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.23.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.23.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.23.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.23.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.24.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.24.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.24.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.24.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.24.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.24.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.25.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.25.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.25.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.25.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.25.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.25.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.26.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.26.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.26.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.26.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.26.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.26.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.27.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.27.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.27.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.27.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.27.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.27.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.28.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.28.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.28.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.28.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.28.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.28.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.29.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.29.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.29.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.29.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.29.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.29.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.30.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.30.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.30.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.30.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.30.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.30.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.31.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.31.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.31.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.31.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.31.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.31.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.32.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.32.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.32.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.32.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.32.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.32.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.33.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.33.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.33.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.33.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.33.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.33.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.34.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.34.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.34.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.34.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.34.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.34.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.35.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.35.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.35.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.35.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.35.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.35.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.36.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.36.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.36.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.36.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.36.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.36.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.37.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.37.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.37.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.37.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.37.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.37.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.38.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.38.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.38.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.38.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.38.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.38.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.39.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.39.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.39.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.39.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.39.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.39.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.40.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.40.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.40.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.40.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.40.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.40.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.41.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.41.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.41.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.41.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.41.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.41.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.42.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.42.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.42.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.42.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.42.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.42.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.43.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.43.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.43.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.43.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.43.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.43.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.44.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.44.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.44.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.44.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.44.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.44.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.45.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.45.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.45.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.45.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.45.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.45.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.46.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.46.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.46.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.46.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.46.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.46.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.47.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.47.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.47.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.47.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.47.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.47.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.48.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.48.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.48.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.48.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.48.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.48.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.49.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.49.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.49.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.49.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.49.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "blocks.49.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.0.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.0.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.0.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.0.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.0.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.0.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.1.ff.net.2.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.1.attn.to_v.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.1.attn.to_out.0.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.1.attn.to_k.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.1.ff.net.0.proj.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    },
+    "refiner.1.attn.to_q.weight": {
+      "dequantized_dtype": "bf16",
+      "has_biases": true
+    }
+  },
+  "adaln_cache": {
+    "timesteps": [
+      0.0,
+      0.027027010917663574,
+      0.07692307233810425,
+      0.10000002384185791,
+      0.19999998807907104,
+      0.25,
+      0.5,
+      1.0
+    ],
+    "num_blocks": 50,
+    "tables_per_block": 6
+  },
+  "vsa": {
+    "capable": false,
+    "num_gate_matrices": 0,
+    "gate_key_suffix": "attn.to_gate_compress.weight",
+    "attention_activations": "bf16",
+    "note": "Gate projections are quantized with the checkpoint's affine INT8/INT6/INT4 weight-only grid. VSA attention Q/K/V remain unquantized activations."
+  }
+}