# How to fine-tune merlinite 7B model in Python

**URL:** <https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863>\
**Category:** Beginners\
**Created:** [July 1, 2024, 6:02am UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863 "2024-07-01T06:02:36Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![sasadangelo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sasadangelo/32/19276_2.png) [@sasadangelo](https://discuss.huggingface.co/u/sasadangelo)\
**Post date:** [July 1, 2024, 6:02am UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/1 "2024-07-01T06:02:36Z")

</div>

I am new to LLM programming in Python and I am trying to fine-tune the [instructlab/merlinite-7b-lab](https://huggingface.co/instructlab/merlinite-7b-lab) model on my Mac M1. My goal is to teach this model to a new music composer **Xenobi Amilen** I have invented.

The text of this composer is [here](https://github.com/sasadangelo/llm-train/blob/main/xenobi_amilen.json).

Using the new Ilab CLI from RedHat I created [this training set](https://github.com/sasadangelo/llm-train/blob/main/train_merlinite_7b.jsonl) for the model. It is a JSONL file with 100 questions/answers about the invented composer.

I wrote [this Python script](https://github.com/sasadangelo/llm-train/blob/main/main.py) to train the model. I tested all the parts related to the tokenizer, datasets and it seems to work. However, the final train got this error:

```auto
RuntimeError: Placeholder storage has not been allocated on MPS device!
  0%| | 0/75 [00:00<?, ?it/s]                                                                                                                                        

```

I found a lot of articles about this error on Google and also StackOverflow [like this](https://stackoverflow.com/questions/74724120/pytorch-on-m1-mac-runtimeerror-placeholder-storage-has-not-been-allocated-on-m), for example. The problem seems that in addition to the model I have to send to mps also the input parameters, but it’s not clear to me how to change my code to do that.

I tried several fixes but had no luck. Can anyone can help?

---

<div class="post-metadata">

**Author:** ![sasadangelo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sasadangelo/32/19276_2.png) [@sasadangelo](https://discuss.huggingface.co/u/sasadangelo)\
**Post date:** [July 3, 2024, 6:22pm UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/3 "2024-07-03T18:22:53Z")

</div>

No one in this forum can help?

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [July 8, 2024, 8:27am UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/4 "2024-07-08T08:27:29Z")

</div>

Hi,

Have you looked at this thread? [RuntimeError: Placeholder storage has not been allocated on MPS device!](https://discuss.huggingface.co/t/runtimeerror-placeholder-storage-has-not-been-allocated-on-mps-device/42999)

---

<div class="post-metadata">

**Author:** ![sasadangelo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sasadangelo/32/19276_2.png) [@sasadangelo](https://discuss.huggingface.co/u/sasadangelo)\
**Post date:** [July 8, 2024, 9:44am UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/5 "2024-07-08T09:44:37Z")

</div>

Yes, I have, but it hasn’t worked for me. As you can see I have the flag enabled.

> <https://github.com/sasadangelo/llm-train/blob/017385be56e6b643ba66352aed17e0b12629536b/main.py#L82>

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [July 8, 2024, 12:33pm UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/6 "2024-07-08T12:33:01Z")

</div>

Yes but further down in the thread they also mention to pass `no_cuda=True`, could you try that?

---

<div class="post-metadata">

**Author:** ![sasadangelo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sasadangelo/32/19276_2.png) [@sasadangelo](https://discuss.huggingface.co/u/sasadangelo)\
**Post date:** [July 9, 2024, 5:12pm UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/7 "2024-07-09T17:12:15Z")

</div>

Hi @nielsr

I added the option as you suggested but now I get this error:

```auto
Traceback (most recent call last):
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/main.py", line 71, in <module>
    training_args = TrainingArguments(
                    ^^^^^^^^^^^^^^^^^^
  File "<string>", line 129, in __init__
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/venv/lib/python3.12/site-packages/transformers/training_args.py", line 1693, in __post_init__
    self.device
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/venv/lib/python3.12/site-packages/transformers/training_args.py", line 2171, in device
    return self._setup_devices
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/venv/lib/python3.12/site-packages/transformers/utils/generic.py", line 60, in __get__
    cached = self.fget(obj)
             ^^^^^^^^^^^^^^
  File "/Users/sasadangelo/github.com/sasadangelo/llm-train/venv/lib/python3.12/site-packages/transformers/training_args.py", line 2133, in _setup_devices
    raise ValueError(
ValueError: Either you do not have an MPS-enabled device on this machine or MacOS version is not 12.3+ or current PyTorch install was not built with MPS enabled.

```

I have MacOS Sonoma 14.5, MPS is enabled because this test works fine:

> <https://github.com/sasadangelo/llm-train/blob/017385be56e6b643ba66352aed17e0b12629536b/main.py#L31>

The only thing I don’t know if PyTorch has MPS enabled but I don’t know how to verify. On MacOS I installed with pip the pythorch version (it is a recent nightly build):

```auto
torch==2.5.0.dev20240704

```

I used also the stable release in my tests but I didn’t try with the option you suggested. What you suggest to do?

---

<div class="post-metadata">

**Author:** ![sasadangelo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sasadangelo/32/19276_2.png) [@sasadangelo](https://discuss.huggingface.co/u/sasadangelo)\
**Post date:** [July 9, 2024, 5:14pm UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/8 "2024-07-09T17:14:52Z")

</div>

To verify MPS Pytorch support I run this test:

> **[Accelerated PyTorch training on Mac - Metal - Apple Developer](https://developer.apple.com/metal/pytorch/)**
>
> PyTorch uses the new Metal Performance Shaders (MPS) backend for GPU training acceleration.

the result was:

```auto
tensor([1.], device='mps:0')

```

so I think everythin is OK

---

<div class="post-metadata">

**Author:** ![sasadangelo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sasadangelo/32/19276_2.png) [@sasadangelo](https://discuss.huggingface.co/u/sasadangelo)\
**Post date:** [July 9, 2024, 5:23pm UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/9 "2024-07-09T17:23:03Z")

</div>

FYI, no\_cuda has been deprecated and raplaced by use\_cpu that by default is already to False. So I don’t think the flag is the issue. I think it is something related to Pytorch.

---

<div class="post-metadata">

**Author:** ![mairindubh](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mairindubh/32/29661_2.png) [@mairindubh](https://discuss.huggingface.co/u/mairindubh)\
**Post date:** [July 21, 2024, 2:48am UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/10 "2024-07-21T02:48:09Z")

</div>

Hi @sasadangelo - I’m a member of the InstructLab project. I am sorry I missed this post when you initially made it.

I looked at your repo - I don’t see the knowledge YAML or md you would need using the InstructLab methodology to train, do you have those available to take a look at?

I’m also wondering why you used a custom script instead of `ilab train` to train the model? Were you just interested in the datagen component and not the full workflow?

---

<div class="post-metadata">

**Author:** ![sasadangelo](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sasadangelo/32/19276_2.png) [@sasadangelo](https://discuss.huggingface.co/u/sasadangelo)\
**Post date:** [October 9, 2024, 7:23am UTC](https://discuss.huggingface.co/t/how-to-fine-tune-merlinite-7b-model-in-python/94863/11 "2024-10-09T07:23:14Z")

</div>

Hi [mairindubh](https://discuss.huggingface.co/u/mairindubh)

The point is that I don’t want to use instructlab to train the model and his methodology with skill and knowledge. I want to understand how training works at a lower level. This is the reason why I implemented my script.