# How to train model without shuffling data with multi-GPU

**URL:** <https://discuss.huggingface.co/t/how-to-train-model-without-shuffling-data-with-multi-gpu/16054>\
**Category:** 🤗Transformers\
**Created:** [March 25, 2022, 6:42am UTC](https://discuss.huggingface.co/t/how-to-train-model-without-shuffling-data-with-multi-gpu/16054 "2022-03-25T06:42:12Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Boxi](https://avatars.discourse-cdn.com/v4/letter/b/4da419/32.png) [@Boxi](https://discuss.huggingface.co/u/Boxi)\
**Post date:** [March 25, 2022, 6:42am UTC](https://discuss.huggingface.co/t/how-to-train-model-without-shuffling-data-with-multi-gpu/16054/1 "2022-03-25T06:42:12Z")

</div>

Hi, my project requires pretraining a large model from scratch using fixed-order data, and my code is modified from the official script “run\_mlm.py”.  
Is it possible to train the model without shuffling data with multi-GPU?