# Does one need to load the model to GPU before calling train when using accelerate?

**URL:** <https://discuss.huggingface.co/t/does-one-need-to-load-the-model-to-gpu-before-calling-train-when-using-accelerate/56852>\
**Category:** Beginners\
**Created:** [September 29, 2023, 7:29pm UTC](https://discuss.huggingface.co/t/does-one-need-to-load-the-model-to-gpu-before-calling-train-when-using-accelerate/56852 "2023-09-29T19:29:33Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![brando](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/brando/32/30114_2.png) [@brando](https://discuss.huggingface.co/u/brando)\
**Post date:** [September 29, 2023, 7:29pm UTC](https://discuss.huggingface.co/t/does-one-need-to-load-the-model-to-gpu-before-calling-train-when-using-accelerate/56852/1 "2023-09-29T19:29:33Z")

</div>

Do I need to call:

```auto
        device = torch.device(f"cuda:{0}" if torch.cuda.is_available() else "cpu")
        model = model.to(device)

```

if I am using accelerate? e.g.,

```auto
accelerate launch --config_file hf_training.py

```

I was worried about the `accelerator.prepare(...)` not working as expected.

related question: [huggingface transformers - How to adapt LLaMA v2 model to less than 7B parameters? - Stack Overflow](https://stackoverflow.com/questions/76971761/how-to-adapt-llama-v2-model-to-less-than-7b-parameters)

cross: [huggingface transformers - Does one need to load the model to GPU before calling train when using accelerate? - Stack Overflow](https://stackoverflow.com/questions/77204403/does-one-need-to-load-the-model-to-gpu-before-calling-train-when-using-accelerat)

---

<div class="post-metadata">

**Author:** ![brando](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/brando/32/30114_2.png) [@brando](https://discuss.huggingface.co/u/brando)\
**Post date:** [September 29, 2023, 7:42pm UTC](https://discuss.huggingface.co/t/does-one-need-to-load-the-model-to-gpu-before-calling-train-when-using-accelerate/56852/2 "2023-09-29T19:42:25Z")

</div>

HF always loads your model to GPU if it’s available. With accelerate it does it too.

> The Trainer checks torch.cuda.is\_available() to detect if GPUs are available.

ref [https://github.com/huggingface/transformers/blob/bffac926ca6bc6c965a92bfbfd00c567a2c0fb90/src/transformers/trainer.py#L2298](https://github.com/huggingface/transformers/blob/bffac926ca6bc6c965a92bfbfd00c567a2c0fb90/src/transformers/trainer.py#L2298)

so

```auto
       # from accelerate import Accelerator
        # accelerator = Accelerator()
        # # self.is_deepspeed_enabled = getattr(accelerator.state, "deepspeed_plugin", None) is not None
        # is_fsdp_enabled = getattr(accelerator.state, "fsdp_plugin", None) is not None
        # if not is_fsdp_enabled: # not sure if this is needed but its for sure safer
        # # maybe figuring out how to run everything with accelerate would fix things...
        # # ref: https://stackoverflow.com/questions/77204403/does-one-need-to-load-the-model-to-gpu-before-calling-train-when-using-accelerat
        # device = torch.device(f"cuda:{0}" if torch.cuda.is_available() else "cpu")
        # model = model.to(device)

```

is not needed. Just load the model and trainer will take care of it.

---

<div class="post-metadata">

**Author:** ![brando](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/brando/32/30114_2.png) [@brando](https://discuss.huggingface.co/u/brando)\
**Post date:** [October 31, 2023, 8:38pm UTC](https://discuss.huggingface.co/t/does-one-need-to-load-the-model-to-gpu-before-calling-train-when-using-accelerate/56852/3 "2023-10-31T20:38:35Z")

</div>

related HF discord: [Discord](https://discord.com/channels/879548962464493619/1169012020721504347/1169012020721504347)