Update README.md
Browse files
README.md
CHANGED
|
@@ -1,3 +1,159 @@
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: apache-2.0
|
| 3 |
+
pipeline_tag: text-to-speech
|
| 4 |
+
library_name: ZONOS2
|
| 5 |
+
tags:
|
| 6 |
+
- zonos2
|
| 7 |
+
- text-to-speech
|
| 8 |
+
- voice-clone
|
| 9 |
+
- clone
|
| 10 |
+
- voice
|
| 11 |
+
- tts
|
| 12 |
+
- comfyui
|
| 13 |
+
- fp8
|
| 14 |
+
- mixed-fp8
|
| 15 |
+
- bf16
|
| 16 |
+
- e4m3
|
| 17 |
+
- safetensors
|
| 18 |
---
|
| 19 |
+
|
| 20 |
+
# ZONOS2-FP8
|
| 21 |
+
|
| 22 |
+
This repository provides a **mixed FP8 Safetensors conversion** of the original [Zyphra/ZONOS2](https://huggingface.co/Zyphra/ZONOS2) model for use with the [ZONOS2 TTS ComfyUI custom node](https://github.com/Saganaki22/Zonos2_TTS-ComfyUI).
|
| 23 |
+
|
| 24 |
+
The model was converted from the original PyTorch checkpoint format to `.safetensors` and quantized using a conservative mixed-precision policy. Only selected MoE expert projection weights were converted to FP8 E4M3, while the precision-sensitive parts of the model were kept in BF16 for stability and output quality.
|
| 25 |
+
|
| 26 |
+

|
| 27 |
+
|
| 28 |
+
## Original Project
|
| 29 |
+
|
| 30 |
+
ZONOS2 is a text-to-speech model from Zyphra trained on more than 6 million hours of varied multilingual speech. It supports expressive speech generation and high-fidelity voice cloning.
|
| 31 |
+
|
| 32 |
+

|
| 33 |
+
|
| 34 |
+
## ComfyUI Custom Node
|
| 35 |
+
|
| 36 |
+
This model package is intended for use with:
|
| 37 |
+
|
| 38 |
+
- https://github.com/Saganaki22/Zonos2_TTS-ComfyUI
|
| 39 |
+
|
| 40 |
+
The ComfyUI node provides native ZONOS2 text-to-speech, audio-only voice cloning, mixed FP8 loading, BF16 compute support, SDPA/FlashAttention inference, progress reporting, and ComfyUI/AIMDO model-management integration.
|
| 41 |
+
|
| 42 |
+
## Model File
|
| 43 |
+
|
| 44 |
+
Main model file:
|
| 45 |
+
|
| 46 |
+
- `zonos2-fp8-mixed.safetensors`
|
| 47 |
+
|
| 48 |
+
Direct download:
|
| 49 |
+
|
| 50 |
+
- https://huggingface.co/drbaph/ZONOS-FP8/resolve/main/zonos2-fp8-mixed.safetensors?download=true
|
| 51 |
+
|
| 52 |
+
## Model Storage Location
|
| 53 |
+
|
| 54 |
+
Place the model and required assets under:
|
| 55 |
+
|
| 56 |
+
ComfyUI/
|
| 57 |
+
βββ models/
|
| 58 |
+
βββ zonos2/
|
| 59 |
+
βββ zonos2-fp8-mixed.safetensors
|
| 60 |
+
βββ dac_44khz/
|
| 61 |
+
βββ speaker_encoder/
|
| 62 |
+
|
| 63 |
+
Expected layout:
|
| 64 |
+
|
| 65 |
+
ComfyUI/models/zonos2/
|
| 66 |
+
βββ zonos2-fp8-mixed.safetensors
|
| 67 |
+
βββ dac_44khz/
|
| 68 |
+
β βββ config.json
|
| 69 |
+
β βββ model.safetensors
|
| 70 |
+
β βββ preprocessor_config.json
|
| 71 |
+
βββ speaker_encoder/
|
| 72 |
+
βββ config.json
|
| 73 |
+
βββ model.safetensors
|
| 74 |
+
βββ preprocessor_config.json
|
| 75 |
+
|
| 76 |
+
If `download_if_missing` is enabled in the ComfyUI node, missing assets can be downloaded automatically.
|
| 77 |
+
|
| 78 |
+
## Usage
|
| 79 |
+
|
| 80 |
+
Install the ComfyUI custom node:
|
| 81 |
+
|
| 82 |
+
cd ComfyUI/custom_nodes
|
| 83 |
+
git clone https://github.com/Saganaki22/Zonos2_TTS-ComfyUI.git
|
| 84 |
+
|
| 85 |
+
Then restart ComfyUI and load the **ZONOS2 FP8 Mixed** model from the node loader.
|
| 86 |
+
|
| 87 |
+
Recommended dtype settings for this checkpoint:
|
| 88 |
+
|
| 89 |
+
- `dtype: auto`
|
| 90 |
+
- `dtype: bf16`
|
| 91 |
+
|
| 92 |
+
The mixed FP8 checkpoint does not use the `fp16` runtime option.
|
| 93 |
+
|
| 94 |
+
## Quantization Details
|
| 95 |
+
|
| 96 |
+
This checkpoint was quantized as a **mixed FP8/BF16 model**.
|
| 97 |
+
|
| 98 |
+
The quantization policy is deliberately conservative:
|
| 99 |
+
|
| 100 |
+
- **Converted to FP8 E4M3**
|
| 101 |
+
- MoE expert gate/up projection weights
|
| 102 |
+
- Specifically the expert `w13` projections
|
| 103 |
+
|
| 104 |
+
- **Left in BF16**
|
| 105 |
+
- Attention layers
|
| 106 |
+
- Dense feed-forward layers
|
| 107 |
+
- Expert-down projections, `w2`
|
| 108 |
+
- LM head
|
| 109 |
+
- Routers
|
| 110 |
+
- Token embeddings
|
| 111 |
+
- Speaker embeddings and speaker projections
|
| 112 |
+
- Normalization layers
|
| 113 |
+
- Biases
|
| 114 |
+
- Temperatures
|
| 115 |
+
- Other precision-sensitive paths
|
| 116 |
+
|
| 117 |
+
In short, the large MoE expert gate/up weights were quantized to FP8 E4M3, while the parts most likely to affect stability, routing, speaker identity, and generation quality were kept in BF16.
|
| 118 |
+
|
| 119 |
+
This reduces the main checkpoint from approximately **14.28 GiB** for the BF16 version to approximately **9.78 GiB** for the mixed FP8 version.
|
| 120 |
+
|
| 121 |
+
The mixed FP8 checkpoint is primarily a **memory-saving option**. It is not guaranteed to generate faster than BF16 on every GPU or ComfyUI setup.
|
| 122 |
+
|
| 123 |
+
## Notes
|
| 124 |
+
|
| 125 |
+
- This repository is a mixed FP8 Safetensors package of the original ZONOS2 model.
|
| 126 |
+
- The model architecture and original weights come from Zyphra/ZONOS2.
|
| 127 |
+
- This package is provided for ComfyUI compatibility and convenience.
|
| 128 |
+
- Mixed FP8 support requires the current ZONOS2 TTS ComfyUI custom node.
|
| 129 |
+
- Voice cloning should only be used with voices you own or have explicit permission to use.
|
| 130 |
+
|
| 131 |
+
## License
|
| 132 |
+
|
| 133 |
+
The original ZONOS2 model is released under the Apache License 2.0.
|
| 134 |
+
|
| 135 |
+
This converted mixed FP8 Safetensors package follows the same model license.
|
| 136 |
+
|
| 137 |
+
## Responsible Use
|
| 138 |
+
|
| 139 |
+
Do not use this model for malicious impersonation, fraud, deception, harassment, non-consensual voice cloning, or any use intended to cause harm.
|
| 140 |
+
|
| 141 |
+
Only clone voices you own or have explicit permission to use.
|
| 142 |
+
|
| 143 |
+
## Citation
|
| 144 |
+
|
| 145 |
+
If you find this model useful in an academic context, please cite the original ZONOS2 work:
|
| 146 |
+
|
| 147 |
+
@misc{zyphra2025zonos,
|
| 148 |
+
title = {Zonos V2 Technical Report},
|
| 149 |
+
author = {Gabriel Clark, Sofian Mejjoute, Mohamed Osman, George Close, Beren Millidge},
|
| 150 |
+
year = {2026},
|
| 151 |
+
}
|
| 152 |
+
|
| 153 |
+
## Credits
|
| 154 |
+
|
| 155 |
+
- Original model: https://github.com/Zyphra/ZONOS2
|
| 156 |
+
- Original Hugging Face repository: https://huggingface.co/Zyphra/ZONOS2
|
| 157 |
+
- Mixed FP8 Safetensors package: https://huggingface.co/drbaph/ZONOS-FP8
|
| 158 |
+
- BF16 Safetensors package: https://huggingface.co/drbaph/ZONOS2-BF16
|
| 159 |
+
- ComfyUI custom node: https://github.com/Saganaki22/Zonos2_TTS-ComfyUI
|