# TrainingArguments changing the GPU by iteslf

**URL:** <https://discuss.huggingface.co/t/trainingarguments-changing-the-gpu-by-iteslf/8001>\
**Category:** 🤗Transformers\
**Created:** [July 4, 2021, 3:00pm UTC](https://discuss.huggingface.co/t/trainingarguments-changing-the-gpu-by-iteslf/8001 "2021-07-04T15:00:30Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![sanjaysingh23](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sanjaysingh23/32/2161_2.png) [@sanjaysingh23](https://discuss.huggingface.co/u/sanjaysingh23)\
**Post date:** [July 4, 2021, 3:00pm UTC](https://discuss.huggingface.co/t/trainingarguments-changing-the-gpu-by-iteslf/8001/1 "2021-07-04T15:00:30Z")

</div>

I have 4 gpus available, out of which i have selected the second one using

```auto
if torch.cuda.is_available():
    torch.cuda.set_device(2)

```

However when i compute the `TrainingArgument()` command :  
`training_args = TrainingArguments('mydirectory')`

then `torch.cuda.current_device()`. is giving 0

Any idea why this is happening?

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [July 7, 2021, 12:19pm UTC](https://discuss.huggingface.co/t/trainingarguments-changing-the-gpu-by-iteslf/8001/2 "2021-07-07T12:19:13Z")

</div>

Yes, the training argument set the GPU corresponding to its `local_rank` value (for distributed training), so you have to make sure to pass along `local_rank=2` when you instantiate them.

Though to execute a script on a given GPU, you would be better off setting the global env variable `CUDA_VISIBLE_DEVICES`.