# Lora finetuning 35 B model error

**URL:** <https://discuss.huggingface.co/t/lora-finetuning-35-b-model-error/91591>\
**Category:** 🤗Accelerate\
**Created:** [June 11, 2024, 4:51pm UTC](https://discuss.huggingface.co/t/lora-finetuning-35-b-model-error/91591 "2024-06-11T16:51:19Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![arihant-neohumans](https://avatars.discourse-cdn.com/v4/letter/a/fbc32d/32.png) [@arihant-neohumans](https://discuss.huggingface.co/u/arihant-neohumans)\
**Post date:** [June 11, 2024, 4:51pm UTC](https://discuss.huggingface.co/t/lora-finetuning-35-b-model-error/91591/1 "2024-06-11T16:51:19Z")

</div>

Hi I am trying to finetune a 35B model using lora (r and alpha 64) . My batch size is 2 and grad accumulation is 2 . I am using 8 A100 80GB gpus with deepspeed zero2 . I estimated it would require 3 gpus to do this . But I am not even able to achieve this on 8GPUs . I keep on getting CUDA OOM. I am unable to figure out why this disceperancy exists. It will be great if someone can explain why this is happening.