# Get "using the \`\_\_call\_\_\` method is faster" warning with DataCollatorWithPadding

**URL:** <https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924>\
**Category:** 🤗Tokenizers\
**Created:** [October 3, 2022, 8:43am UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924 "2022-10-03T08:43:17Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![SteveR](https://avatars.discourse-cdn.com/v4/letter/s/71e660/32.png) [@SteveR](https://discuss.huggingface.co/u/SteveR)\
**Post date:** [October 3, 2022, 8:43am UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924/1 "2022-10-03T08:43:17Z")

</div>

When I use the out-of-the-box **DataCollatorWithPadding** I get my output filled with the warning:

> You’re using a DebertaV2TokenizerFast tokenizer. Please note that with a fast tokenizer, using the ` __call__ ` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.

If I change to use a different, custom collator, then the warning goes away.  
Would anyone know what I could be doing wrong that’s causing this warning?

Or alternatively, if I can’t fix the problem that’s causing this warning, is there a way to hide it?  
I’ve tried a few different ways of turning off warnings, but so far I’ve had no luck and because it gets written out multiple times it starts to swamp the actual output from my training.

---

<div class="post-metadata">

**Author:** ![mnaylor](https://avatars.discourse-cdn.com/v4/letter/m/cc9497/32.png) [@mnaylor](https://discuss.huggingface.co/u/mnaylor)\
**Post date:** [October 5, 2022, 6:38pm UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924/2 "2022-10-05T18:38:03Z")

</div>

I’m getting the same thing using a `BertTokenizerFast` with `DataCollatorWithPadding` - the error appears once for each worker every time I loop over a DataLoader. I would prefer not to silence warnings in my training code, but here’s how I’m getting around it (based on [this line](https://github.com/huggingface/transformers/blob/main/src/transformers/tokenization_utils_base.py#L2886) in the `PretrainedTokenizerBase` class, referencing [this section in the custom logger](https://github.com/huggingface/transformers/blob/d9101b71bcd2e2a1f26cf07057ffd15572997105/src/transformers/utils/logging.py#L273-L281)):

```python
import os
os.environ['TRANSFORMERS_NO_ADVISORY_WARNINGS'] = 'true'

```

---

<div class="post-metadata">

**Author:** ![Bikash](https://avatars.discourse-cdn.com/v4/letter/b/ee59a6/32.png) [@Bikash](https://discuss.huggingface.co/u/Bikash)\
**Post date:** [October 7, 2022, 10:07am UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924/3 "2022-10-07T10:07:51Z")

</div>

Same situation - happened with usage of `DistilBertTokenizerFast` tokenizer on using `DataCollatorWithPadding`. I don’t know what does this warning actually mean? And if I wanted to follow the suggestion as hinted in the warning message, what should I do?

---

<div class="post-metadata">

**Author:** ![KarmaCST](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/karmacst/32/15362_2.png) [@KarmaCST](https://discuss.huggingface.co/u/KarmaCST)\
**Post date:** [November 24, 2022, 6:04am UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924/4 "2022-11-24T06:04:43Z")

</div>

I am getting following suggestion while using NllbTokenizerFast tokenize  
**You’re using a NllbTokenizerFast tokenizer. Please note that with a fast tokenizer, using the ` __call__ ` method is faster than using a method to encode the text followed by a call to the `pad` method to get a padded encoding.**

Can anyone explain the given suggestion?

---

<div class="post-metadata">

**Author:** ![fredguth](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/fredguth/32/2843_2.png) [@fredguth](https://discuss.huggingface.co/u/fredguth)\
**Post date:** [May 9, 2023, 3:29pm UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924/5 "2023-05-09T15:29:37Z")

</div>

I guess the problem most people in this thread have is not with the `warnings` but with the lack of knowledge (at least at my part) if there is a better way of doing what I am doing. I also use a preprocess function.

---

<div class="post-metadata">

**Author:** ![ollibolli](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/ollibolli/32/16121_2.png) [@ollibolli](https://discuss.huggingface.co/u/ollibolli)\
**Post date:** [May 13, 2023, 2:08pm UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924/6 "2023-05-13T14:08:26Z")

</div>

It would be nice to have an answer from someone on this?

So my guess is that doing `tokenizer(input)` is faster than wrapping the `datacollator` around?

---

<div class="post-metadata">

**Author:** ![filippobistaffa](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/filippobistaffa/32/19181_2.png) [@filippobistaffa](https://discuss.huggingface.co/u/filippobistaffa)\
**Post date:** [September 21, 2023, 8:45am UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924/7 "2023-09-21T08:45:26Z")

</div>

Any news on this? Personally I found this warning more annoying than informative, so I ended up commenting it out in the source code, since I couldn’t find any information.

---

<div class="post-metadata">

**Author:** ![d3nigma](https://avatars.discourse-cdn.com/v4/letter/d/3be4f8/32.png) [@d3nigma](https://discuss.huggingface.co/u/d3nigma)\
**Post date:** [February 1, 2024, 7:19pm UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924/8 "2024-02-01T19:19:21Z")

</div>

Hello,

the warning is confusing because it does not tell you **how** to use the `tokenizer(...)`.  
I think it just means, that one should use the padding feature of the tokenizer when tokenizing your dataset. So, instead of using a collator to pad your inputs, skip this step and use `tokenizer(..., padding=True)` to add padding while tokenizing. Apparently, this is faster for [FastTokenizers](https://huggingface.co/learn/nlp-course/chapter6/3) (i.e. tokenizers written in Rust).

**Remark** : In case you didn’t know when calling `tokenizer(...)` this internally calls `Tokenizer. __call__ `. This is a magic python method: [python-callable-instances](https://realpython.com/python-callable-instances/)

**Disclaimer** : That’s just my interpretation of what it could mean. I would be happy if someone could confirm this.

---

<div class="post-metadata">

**Author:** ![AtxTom](https://avatars.discourse-cdn.com/v4/letter/a/ed655f/32.png) [@AtxTom](https://discuss.huggingface.co/u/AtxTom)\
**Post date:** [June 3, 2024, 9:21pm UTC](https://discuss.huggingface.co/t/get-using-the-call-method-is-faster-warning-with-datacollatorwithpadding/23924/9 "2024-06-03T21:21:47Z")

</div>

You can turn off the warning with

```auto
tokenizer.deprecation_warnings["Asking-to-pad-a-fast-tokenizer"] = True

```

See [When using transformers.DataCollatorWithPadding normally, always get annoying warning · Issue #22638 · huggingface/transformers · GitHub](https://github.com/huggingface/transformers/issues/22638)