# Using Batch Encodings

**URL:** <https://discuss.huggingface.co/t/using-batch-encodings/20295>\
**Category:** 🤗Transformers\
**Created:** [July 12, 2022, 7:26pm UTC](https://discuss.huggingface.co/t/using-batch-encodings/20295 "2022-07-12T19:26:34Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![ddegenaro](https://avatars.discourse-cdn.com/v4/letter/d/e79b87/32.png) [@ddegenaro](https://discuss.huggingface.co/u/ddegenaro)\
**Post date:** [July 12, 2022, 7:26pm UTC](https://discuss.huggingface.co/t/using-batch-encodings/20295/1 "2022-07-12T19:26:34Z")

</div>

Hi there! I am trying to work with a corpus I built which contains batch encodings of size 256 (I passed a list of 256 sentences to a fast BERT tokenizer at a time, and pickled the outputs).

Unfortunately, when I try to pass this entire BatchEncoding object into my model to get prediction logits, the GPU runs out of memory.

I am now trying to “unpack” the BatchEncoding objects into single inputs that can be fed to the model one at a time, but I cannot figure out how to do this. Does anyone know how I can accomplish this?

Alternatively, does anyone know how much memory one of these BatchEncodings might be taking up? I am not sure it is reasonable for 256 BERT encodings to eat up 12 GB of GPU RAM.

Thanks!