# Transfer learning

**URL:** <https://discuss.huggingface.co/t/transfer-learning/12153>\
**Category:** 🤗Transformers\
**Created:** [November 22, 2021, 12:38pm UTC](https://discuss.huggingface.co/t/transfer-learning/12153 "2021-11-22T12:38:09Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![Emilio](https://avatars.discourse-cdn.com/v4/letter/e/f475e1/32.png) [@Emilio](https://discuss.huggingface.co/u/Emilio)\
**Post date:** [November 22, 2021, 12:38pm UTC](https://discuss.huggingface.co/t/transfer-learning/12153/1 "2021-11-22T12:38:09Z")

</div>

Hello Everyone, I have a quick question. I trained a Mask Language Model starting from “AutoModelForMaskedLM.from\_pretrained” (like a continual pre-training). Then I save that model and I want to use it as starting point for a text classification task, then I use AutoModel.from\_pretrained to load the model architecture and finally I use a load\_from\_ckpt function to copy the weights from the pre-trained model into my new instantiated model. However, I see that the naming of the layers is slightly different from the two models, one has the prefix “bert” and the other does not. This cause a conflict when loading the weights since I check the names of the layers. What is the best way to address this situation?. What I am doing now is to use AutoModelForMaskedLM.from\_pretrained also to load the architecture of the fine-tuning, even though i am not training for MLM.  
Many thanks!

---

<div class="post-metadata">

**Author:** ![Emilio](https://avatars.discourse-cdn.com/v4/letter/e/f475e1/32.png) [@Emilio](https://discuss.huggingface.co/u/Emilio)\
**Post date:** [November 30, 2021, 8:18am UTC](https://discuss.huggingface.co/t/transfer-learning/12153/2 "2021-11-30T08:18:40Z")

</div>

@nielsr or @lhoestq any thoughts on this will be more than welcome!

---

<div class="post-metadata">

**Author:** ![nielsr](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/nielsr/32/39617_2.png) [@nielsr](https://discuss.huggingface.co/u/nielsr)\
**Post date:** [November 30, 2021, 8:35am UTC](https://discuss.huggingface.co/t/transfer-learning/12153/3 "2021-11-30T08:35:45Z")

</div>

Hi,

Normally you should only use the `from_pretrained()` method to load the weights into your new model. You don’t need a `load_from_ckpt` function.

---

<div class="post-metadata">

**Author:** ![Emilio](https://avatars.discourse-cdn.com/v4/letter/e/f475e1/32.png) [@Emilio](https://discuss.huggingface.co/u/Emilio)\
**Post date:** [November 30, 2021, 9:24am UTC](https://discuss.huggingface.co/t/transfer-learning/12153/4 "2021-11-30T09:24:28Z")

</div>

Many thanks @nielsr, the thing is that we use PytorchLightning and it saves the model in ckpt format automatically and then it has this load\_from\_ckpt function too. Do you know if there is compatibility between the ckpt format of PL and the HF format?