# Difference between CausalLM and LMHeadModel

**URL:** <https://discuss.huggingface.co/t/difference-between-causallm-and-lmheadmodel/17135>\
**Category:** Models\
**Created:** [April 24, 2022, 10:39pm UTC](https://discuss.huggingface.co/t/difference-between-causallm-and-lmheadmodel/17135 "2022-04-24T22:39:35Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Kirti](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/kirti/32/4580_2.png) [@Kirti](https://discuss.huggingface.co/u/Kirti)\
**Post date:** [April 24, 2022, 10:39pm UTC](https://discuss.huggingface.co/t/difference-between-causallm-and-lmheadmodel/17135/1 "2022-04-24T22:39:35Z")

</div>

What us the difference between CausalLM and LMHeadModel? Both returns the similar variables. Loss, logits etc…

Example: GPT2LMHeadModel.from\_pretrained(‘gpt2’) and AutoModelForCausalLM.from\_pretrained(‘gpt2’) has the same model structure.

---

<div class="post-metadata">

**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\
**Post date:** [April 25, 2022, 1:00pm UTC](https://discuss.huggingface.co/t/difference-between-causallm-and-lmheadmodel/17135/2 "2022-04-25T13:00:42Z")

</div>

The name `LMHeadModel` are old names we used before for some models, but we stopped as it’s not very informative on what kind of language model head we’re talking about. To avoid breaking changes, we won’t rename the old classes, but the auto API and all newer models should have `ForCausalLM` or `ForMaskedLM` or `ForSeq2SeqLM` depending on that kind of LM objective the model has.