# Adding attention mask into MLM

**URL:** <https://discuss.huggingface.co/t/adding-attention-mask-into-mlm/89415>\
**Category:** 🤗Transformers\
**Created:** [May 30, 2024, 12:41pm UTC](https://discuss.huggingface.co/t/adding-attention-mask-into-mlm/89415 "2024-05-30T12:41:22Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![dc12395](https://avatars.discourse-cdn.com/v4/letter/d/c6cbf5/32.png) [@dc12395](https://discuss.huggingface.co/u/dc12395)\
**Post date:** [May 30, 2024, 12:41pm UTC](https://discuss.huggingface.co/t/adding-attention-mask-into-mlm/89415/1 "2024-05-30T12:41:22Z")

</div>

Hi all, I’m using this .py file ([sentence-transformers/examples/unsupervised\_learning/MLM at master · UKPLab/sentence-transformers · GitHub](https://github.com/UKPLab/sentence-transformers/tree/master/examples/unsupervised_learning/MLM)) to carry out some unsupervised training.

It’s returning a message saying: ‘We strongly recommend passing in an `attention_mask` since your input\_ids may be padded. See [Troubleshoot](https://huggingface.co/docs/transformers/troubleshooting#incorrect-output-when-padding-tokens-arent-masked.)’

I’m a little unfamiliar with this process, but my guess is that it may be due to the fact that my training data text file has every row varying in length.

How would I adapt this to add in an attention mask? Thanks!

---

<div class="post-metadata">

**Author:** ![dc12395](https://avatars.discourse-cdn.com/v4/letter/d/c6cbf5/32.png) [@dc12395](https://discuss.huggingface.co/u/dc12395)\
**Post date:** [May 30, 2024, 12:47pm UTC](https://discuss.huggingface.co/t/adding-attention-mask-into-mlm/89415/2 "2024-05-30T12:47:20Z")

</div>

To add to this, my code is killed following the message above, and no config.json file is created in the sentence transformer