# Does the BERT model choose only the inputs it needs?

**URL:** <https://discuss.huggingface.co/t/does-the-bert-model-choose-only-the-inputs-it-needs/19843>\
**Category:** Beginners\
**Created:** [June 30, 2022, 11:37am UTC](https://discuss.huggingface.co/t/does-the-bert-model-choose-only-the-inputs-it-needs/19843 "2022-06-30T11:37:00Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![tomleung1996](https://avatars.discourse-cdn.com/v4/letter/t/91b2a8/32.png) [@tomleung1996](https://discuss.huggingface.co/u/tomleung1996)\
**Post date:** [June 30, 2022, 11:37am UTC](https://discuss.huggingface.co/t/does-the-bert-model-choose-only-the-inputs-it-needs/19843/1 "2022-06-30T11:37:00Z")

</div>

Hi, I am a new user of Transformers and I have successfully fine-tuned a BERT model following the tutorial.

However, I have one question about the features I send to the Trainer and those accepted by the BERT model.

Specifically, my original dataset contains two columns named “text” and “label”. After tokenizing the “text”, the dataset object now have three more columns named “input\_ids”, “token\_type\_ids”, and “attention\_mask”. I understand that these three columns are required by the BERT model, but I didn’t drop the original “text” column when I fed the dataset to the Trainer API.

So my question is, does BERT automatically ignore non-relevant features? (maybe this is achieved quietly by the Trainer API) Or should I remove these columns, leaving only “input\_ids”, “token\_type\_ids”, and “attention\_mask”?

Thank you!

---

<div class="post-metadata">

**Author:** ![BramVanroy](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bramvanroy/32/47360_2.png) [@BramVanroy](https://discuss.huggingface.co/u/BramVanroy)\
**Post date:** [July 1, 2022, 9:36am UTC](https://discuss.huggingface.co/t/does-the-bert-model-choose-only-the-inputs-it-needs/19843/2 "2022-07-01T09:36:51Z")

</div>

The Trainer will ignore irrelevant columns in your input data.

> <https://github.com/huggingface/transformers/blob/d0acc9537829e7d067edbb791473bbceb2ecf056/src/transformers/trainer_utils.py#L678-L692>