# Load shards as one dataset

**URL:** <https://discuss.huggingface.co/t/load-shards-as-one-dataset/73422>\
**Category:** 🤗Datasets\
**Created:** [February 16, 2024, 3:47am UTC](https://discuss.huggingface.co/t/load-shards-as-one-dataset/73422 "2024-02-16T03:47:35Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![chenchaozhao](https://avatars.discourse-cdn.com/v4/letter/c/ac91a4/32.png) [@chenchaozhao](https://discuss.huggingface.co/u/chenchaozhao)\
**Post date:** [February 16, 2024, 3:47am UTC](https://discuss.huggingface.co/t/load-shards-as-one-dataset/73422/1 "2024-02-16T03:47:35Z")

</div>

I processed the datasets into several shards. If I want to load them as one piece I can do concatenation but it will take some time to index all of the files. Is there a quicker way to load the dataset like a memory mapping from several dataset shards?