# Training using FSDP, qLoRa on multinode

**URL:** <https://discuss.huggingface.co/t/training-using-fsdp-qlora-on-multinode/137923>\
**Category:** 🤗Accelerate\
**Created:** [January 29, 2025, 3:39pm UTC](https://discuss.huggingface.co/t/training-using-fsdp-qlora-on-multinode/137923 "2025-01-29T15:39:43Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yustas](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/yustas/32/40036_2.png) [@Yustas](https://discuss.huggingface.co/u/Yustas)\
**Post date:** [January 29, 2025, 3:39pm UTC](https://discuss.huggingface.co/t/training-using-fsdp-qlora-on-multinode/137923/1 "2025-01-29T15:39:43Z")

</div>

How to train a model like Llama3 using FSDP, qLoRa on two (or more) nodes if each node has one (or more) GPU?  
Can anyone provide a link to a similar example?  
I am grateful in advance for any help!