# AutoTrain Python automatically using MPS on Mac - How to switch to CPU

**URL:** <https://discuss.huggingface.co/t/autotrain-python-automatically-using-mps-on-mac-how-to-switch-to-cpu/116449>\
**Category:** Beginners\
**Created:** [November 8, 2024, 11:20pm UTC](https://discuss.huggingface.co/t/autotrain-python-automatically-using-mps-on-mac-how-to-switch-to-cpu/116449 "2024-11-08T23:20:16Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Lyreck](https://avatars.discourse-cdn.com/v4/letter/l/3be4f8/32.png) [@Lyreck](https://discuss.huggingface.co/u/Lyreck)\
**Post date:** [November 8, 2024, 11:20pm UTC](https://discuss.huggingface.co/t/autotrain-python-automatically-using-mps-on-mac-how-to-switch-to-cpu/116449/1 "2024-11-08T23:20:16Z")

</div>

Hello everyone,

I am trying to fine-tune cardiffnlp/twitter-xlm-roberta-base-sentiment with some additional text. I am using the Python API to setup the model.

Everything appears to work just fine, the only thing is that I encounter a memory shortage with MPS (I am working on a 2019 MacBook Pro with AMD Radeon):

`MPS backend out of memory (MPS allocated: 5.69 GB, other allocations: 1.00 GB, max allowed: 6.77 GB). Tried to allocate 96.00 MB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).`

From what I understand, this is a physical limitation from my computer (perhaps I’m wrong - it’s surprising since the model is supposedly less than 2GB large)). Would it be possible to switch to CPU training rather than GPU training ? I don’t see any parameter in the model that seems to control that.

Any help or idea is welcome !

Here is the code I am using:

````Python
from autotrain.params import TextClassificationParams
from autotrain.project import AutoTrainProject

HF_USERNAME = ""
HF_TOKEN = ""

params = TextClassificationParams(
    model="cardiffnlp/twitter-xlm-roberta-base-sentiment",
    data_path="data/", #"Lyreck/tiktok_brat_comments", # path to the dataset on huggingface hub
    text_column="text", # the column in the dataset that contains the text
    target_column="label", # the column in the dataset that contains the labels
    train_split="train",
    valid_split="validation",
    epochs=3,
    batch_size=8,
    max_seq_length=512,
    lr=1e-5,
    optimizer="adamw_torch",
    scheduler="linear",
    gradient_accumulation=1,
    #mixed_precision="fp16", #need graphic card for this (no mps available)
    project_name="finetuned-sentiment-model",
    log="tensorboard",
    push_to_hub=True,
    username=HF_USERNAME,
    token=HF_TOKEN,
)

project = AutoTrainProject(params=params, backend="local", process=True)
project.create()```

Thanks!
````

---

<div class="post-metadata">

**Author:** ![John6666](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/john6666/32/27664_2.png) [@John6666](https://discuss.huggingface.co/u/John6666)\
**Post date:** [November 9, 2024, 2:08am UTC](https://discuss.huggingface.co/t/autotrain-python-automatically-using-mps-on-mac-how-to-switch-to-cpu/116449/2 "2024-11-09T02:08:23Z")

</div>

It feels like a bug.

> <https://github.com/pytorch/pytorch/issues/105839>
>
> \### 🐛 Describe the bug
> 
> There appears to be something wrong the the MPS cache, I… appears that either its not releasing memory when it ideally should be, or the freeable memory in the cache is not being taken into account when the check for space occurs.
> The issue occurs on the currently nightly, see versions, and 2.0.1 
> 
> This issue affects performance at best and terminates an application at worse.
> 
> Here's an example...
> 
> \`\`\`
> from diffusers import KandinskyV22PriorPipeline, KandinskyV22Pipeline
> from torch import mps
> import torch
> import fp16fixes
> import gc
> 
> fp16fixes.fp16\_fixes()
> 
> pipe\_prior = KandinskyV22PriorPipeline.from\_pretrained("kandinsky-community/kandinsky-2-2-prior", torch\_dtype=torch.float16)
> pipe\_prior.to("mps")
> prompt = "A car exploding into colorful dust"
> out = pipe\_prior(prompt)
> image\_emb = out.image\_embeds
> zero\_image\_emb = out.negative\_image\_embeds
> 
> pipe\_prior = None
> gc.collect()
> mps.empty\_cache()
> 
> pipe = KandinskyV22Pipeline.from\_pretrained("kandinsky-community/kandinsky-2-2-decoder", torch\_dtype=torch.float16)
> pipe.to("mps")
> pipe.enable\_attention\_slicing()
> 
> image = pipe(
> image\_embeds=image\_emb,
> negative\_image\_embeds=zero\_image\_emb,
> height=1024,
> width=1024,
> num\_inference\_steps=30,
> ).images
> 
> image\[0\].save("cat.png")
> \`\`\`
> 
> This works on a 8GB M1 Mac Mini without issue the two models run at
> 
> \`\`\`
> 100%|████████| 25/25 \[00:07\<00:00, 3.15it/s\]
> 100%|████████| 30/30 \[04:24\<00:00, 8.82s/it\]
> \`\`\`
> 
> Remove the \`mps.empty\_cache()\` and it fails during the second model run
> 
> \`\`\`
> 0%| | 0/30 \[00:03\<?, ?it/s\]
> Traceback (most recent call last):
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/8GB\_M1\_Diffusers\_Scripts/sag/k2img.py", line 25, in \<module\>
> image = pipe(
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/torch/utils/\_contextlib.py", line 115, in decorate\_context
> return func(\*args, \*\*kwargs)
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/diffusers/pipelines/kandinsky2\_2/pipeline\_kandinsky2\_2.py", line 272, in \_\_call\_\_
> noise\_pred = self.unet(
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in \_wrapped\_call\_impl
> return self.\_call\_impl(\*args, \*\*kwargs)
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in \_call\_impl
> return forward\_call(\*args, \*\*kwargs)
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/diffusers/models/unet\_2d\_condition.py", line 905, in forward
> sample, res\_samples = downsample\_block(
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in \_wrapped\_call\_impl
> return self.\_call\_impl(\*args, \*\*kwargs)
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in \_call\_impl
> return forward\_call(\*args, \*\*kwargs)
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/diffusers/models/unet\_2d\_blocks.py", line 1662, in forward
> hidden\_states = attn(
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1518, in \_wrapped\_call\_impl
> return self.\_call\_impl(\*args, \*\*kwargs)
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1527, in \_call\_impl
> return forward\_call(\*args, \*\*kwargs)
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/diffusers/models/attention\_processor.py", line 321, in forward
> return self.processor(
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/diffusers/models/attention\_processor.py", line 1590, in \_\_call\_\_
> attn\_slice = attn.get\_attention\_scores(query\_slice, key\_slice, attn\_mask\_slice)
> File "/Volumes/Sabrent Media/Documents/Source/Python/Diffusers/lib/python3.10/site-packages/diffusers/models/attention\_processor.py", line 374, in get\_attention\_scores
> attention\_probs = attention\_scores.softmax(dim=-1)
> RuntimeError: MPS backend out of memory (MPS allocated: 3.90 GB, other allocations: 4.94 GB, max allowed: 9.07 GB). Tried to allocate 387.00 MB on private pool. Use PYTORCH\_MPS\_HIGH\_WATERMARK\_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).
> \`\`\`
> 
> If I reduce the height and width values to 512 it'll run to completion but the second model runs at 40 seconds per iter with a lot of swap file access. With the cache emptied manually it runs at around 2 seconds per iter.
> 
> the fp16fixes file is required to work around some issues with using fp16 on mps which fails with a broadcast error on 2.0.1 and fails with a bad image on the nightly I'm currently using. If I remove it the issue still occurs on the nightly.
> 
> \`\`\`
> % cat fp16fixes.py 
> import torch
> 
> def fp16\_fixes():
> if torch.backends.mps.is\_available():
> torch.empty = torch.zeros
> 
> \_torch\_layer\_norm = torch.nn.functional.layer\_norm
> def new\_layer\_norm(input, normalized\_shape, weight=None, bias=None, eps=1e-05):
> if input.device.type == "mps" and input.dtype == torch.float16:
> input = input.float()
> if weight is not None:
> weight = weight.float()
> if bias is not None:
> bias = bias.float()
> return \_torch\_layer\_norm(input, normalized\_shape, weight, bias, eps).half()
> else:
> return \_torch\_layer\_norm(input, normalized\_shape, weight, bias, eps)
> 
> torch.nn.functional.layer\_norm = new\_layer\_norm
> 
> 
> def new\_torch\_tensor\_permute(input, \*dims):
> result = torch.permute(input, tuple(dims))
> if input.device == "mps" and input.dtype == torch.float16:
> result = result.contiguous()
> return result
> 
> torch.Tensor.permute = new\_torch\_tensor\_permute
> \`\`\`
> 
> \### Versions
> 
> Collecting environment information...
> PyTorch version: 2.1.0.dev20230724
> Is debug build: False
> CUDA used to build PyTorch: None
> ROCM used to build PyTorch: N/A
> 
> OS: macOS 13.4.1 (arm64)
> GCC version: Could not collect
> Clang version: 14.0.3 (clang-1403.0.22.14.1)
> CMake version: version 3.24.4
> Libc version: N/A
> 
> Python version: 3.10.11 (main, Apr 8 2023, 02:11:11) \[Clang 14.0.0 (clang-1400.0.29.202)\] (64-bit runtime)
> Python platform: macOS-13.4.1-arm64-arm-64bit
> Is CUDA available: False
> CUDA runtime version: No CUDA
> CUDA\_MODULE\_LOADING set to: N/A
> GPU models and configuration: No CUDA
> Nvidia driver version: No CUDA
> cuDNN version: No CUDA
> HIP runtime version: N/A
> MIOpen runtime version: N/A
> Is XNNPACK available: True
> 
> CPU:
> Apple M1
> 
> Versions of relevant libraries:
> \[pip3\] numpy==1.25.1
> \[pip3\] torch==2.1.0.dev20230724
> \[pip3\] torchvision==0.15.2
> \[conda\] Could not collect
> 
> 
> cc @ezyang @gchanan @zou3519 @kulinseth @albanD @malfet @DenisVieriu97 @razarmehr @abhudev

> **[Accelerated PyTorch Training on Mac](https://huggingface.co/docs/accelerate/usage_guides/mps#how-it-works-out-of-the-box)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

> It is enabled by default on MacOs machines with MPS enabled Apple Silicon GPUs. To disable it, pass `--cpu` flag to `accelerate launch` command or answer the corresponding question when answering the `accelerate config` questionnaire.