drbaph commited on
Commit
6b713d1
Β·
verified Β·
1 Parent(s): 7ce841b

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +156 -0
README.md CHANGED
@@ -1,3 +1,159 @@
1
  ---
2
  license: apache-2.0
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
3
  ---
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
  ---
2
  license: apache-2.0
3
+ pipeline_tag: text-to-speech
4
+ library_name: ZONOS2
5
+ tags:
6
+ - zonos2
7
+ - text-to-speech
8
+ - voice-clone
9
+ - clone
10
+ - voice
11
+ - tts
12
+ - comfyui
13
+ - fp8
14
+ - mixed-fp8
15
+ - bf16
16
+ - e4m3
17
+ - safetensors
18
  ---
19
+
20
+ # ZONOS2-FP8
21
+
22
+ This repository provides a **mixed FP8 Safetensors conversion** of the original [Zyphra/ZONOS2](https://huggingface.co/Zyphra/ZONOS2) model for use with the [ZONOS2 TTS ComfyUI custom node](https://github.com/Saganaki22/Zonos2_TTS-ComfyUI).
23
+
24
+ The model was converted from the original PyTorch checkpoint format to `.safetensors` and quantized using a conservative mixed-precision policy. Only selected MoE expert projection weights were converted to FP8 E4M3, while the precision-sensitive parts of the model were kept in BF16 for stability and output quality.
25
+
26
+ ![Screenshot 2026-06-12 214924](https://cdn-uploads.huggingface.co/production/uploads/63473b59e5c0717e6737b872/aWr3cHXPHERGSyYG2hvMT.png)
27
+
28
+ ## Original Project
29
+
30
+ ZONOS2 is a text-to-speech model from Zyphra trained on more than 6 million hours of varied multilingual speech. It supports expressive speech generation and high-fidelity voice cloning.
31
+
32
+ ![zonos2](https://cdn-uploads.huggingface.co/production/uploads/63473b59e5c0717e6737b872/Aiw3SXr27w4SVT4g-QSOt.gif)
33
+
34
+ ## ComfyUI Custom Node
35
+
36
+ This model package is intended for use with:
37
+
38
+ - https://github.com/Saganaki22/Zonos2_TTS-ComfyUI
39
+
40
+ The ComfyUI node provides native ZONOS2 text-to-speech, audio-only voice cloning, mixed FP8 loading, BF16 compute support, SDPA/FlashAttention inference, progress reporting, and ComfyUI/AIMDO model-management integration.
41
+
42
+ ## Model File
43
+
44
+ Main model file:
45
+
46
+ - `zonos2-fp8-mixed.safetensors`
47
+
48
+ Direct download:
49
+
50
+ - https://huggingface.co/drbaph/ZONOS-FP8/resolve/main/zonos2-fp8-mixed.safetensors?download=true
51
+
52
+ ## Model Storage Location
53
+
54
+ Place the model and required assets under:
55
+
56
+ ComfyUI/
57
+ └── models/
58
+ └── zonos2/
59
+ β”œβ”€β”€ zonos2-fp8-mixed.safetensors
60
+ β”œβ”€β”€ dac_44khz/
61
+ └── speaker_encoder/
62
+
63
+ Expected layout:
64
+
65
+ ComfyUI/models/zonos2/
66
+ β”œβ”€β”€ zonos2-fp8-mixed.safetensors
67
+ β”œβ”€β”€ dac_44khz/
68
+ β”‚ β”œβ”€β”€ config.json
69
+ β”‚ β”œβ”€β”€ model.safetensors
70
+ β”‚ └── preprocessor_config.json
71
+ └── speaker_encoder/
72
+ β”œβ”€β”€ config.json
73
+ β”œβ”€β”€ model.safetensors
74
+ └── preprocessor_config.json
75
+
76
+ If `download_if_missing` is enabled in the ComfyUI node, missing assets can be downloaded automatically.
77
+
78
+ ## Usage
79
+
80
+ Install the ComfyUI custom node:
81
+
82
+ cd ComfyUI/custom_nodes
83
+ git clone https://github.com/Saganaki22/Zonos2_TTS-ComfyUI.git
84
+
85
+ Then restart ComfyUI and load the **ZONOS2 FP8 Mixed** model from the node loader.
86
+
87
+ Recommended dtype settings for this checkpoint:
88
+
89
+ - `dtype: auto`
90
+ - `dtype: bf16`
91
+
92
+ The mixed FP8 checkpoint does not use the `fp16` runtime option.
93
+
94
+ ## Quantization Details
95
+
96
+ This checkpoint was quantized as a **mixed FP8/BF16 model**.
97
+
98
+ The quantization policy is deliberately conservative:
99
+
100
+ - **Converted to FP8 E4M3**
101
+ - MoE expert gate/up projection weights
102
+ - Specifically the expert `w13` projections
103
+
104
+ - **Left in BF16**
105
+ - Attention layers
106
+ - Dense feed-forward layers
107
+ - Expert-down projections, `w2`
108
+ - LM head
109
+ - Routers
110
+ - Token embeddings
111
+ - Speaker embeddings and speaker projections
112
+ - Normalization layers
113
+ - Biases
114
+ - Temperatures
115
+ - Other precision-sensitive paths
116
+
117
+ In short, the large MoE expert gate/up weights were quantized to FP8 E4M3, while the parts most likely to affect stability, routing, speaker identity, and generation quality were kept in BF16.
118
+
119
+ This reduces the main checkpoint from approximately **14.28 GiB** for the BF16 version to approximately **9.78 GiB** for the mixed FP8 version.
120
+
121
+ The mixed FP8 checkpoint is primarily a **memory-saving option**. It is not guaranteed to generate faster than BF16 on every GPU or ComfyUI setup.
122
+
123
+ ## Notes
124
+
125
+ - This repository is a mixed FP8 Safetensors package of the original ZONOS2 model.
126
+ - The model architecture and original weights come from Zyphra/ZONOS2.
127
+ - This package is provided for ComfyUI compatibility and convenience.
128
+ - Mixed FP8 support requires the current ZONOS2 TTS ComfyUI custom node.
129
+ - Voice cloning should only be used with voices you own or have explicit permission to use.
130
+
131
+ ## License
132
+
133
+ The original ZONOS2 model is released under the Apache License 2.0.
134
+
135
+ This converted mixed FP8 Safetensors package follows the same model license.
136
+
137
+ ## Responsible Use
138
+
139
+ Do not use this model for malicious impersonation, fraud, deception, harassment, non-consensual voice cloning, or any use intended to cause harm.
140
+
141
+ Only clone voices you own or have explicit permission to use.
142
+
143
+ ## Citation
144
+
145
+ If you find this model useful in an academic context, please cite the original ZONOS2 work:
146
+
147
+ @misc{zyphra2025zonos,
148
+ title = {Zonos V2 Technical Report},
149
+ author = {Gabriel Clark, Sofian Mejjoute, Mohamed Osman, George Close, Beren Millidge},
150
+ year = {2026},
151
+ }
152
+
153
+ ## Credits
154
+
155
+ - Original model: https://github.com/Zyphra/ZONOS2
156
+ - Original Hugging Face repository: https://huggingface.co/Zyphra/ZONOS2
157
+ - Mixed FP8 Safetensors package: https://huggingface.co/drbaph/ZONOS-FP8
158
+ - BF16 Safetensors package: https://huggingface.co/drbaph/ZONOS2-BF16
159
+ - ComfyUI custom node: https://github.com/Saganaki22/Zonos2_TTS-ComfyUI