# Efficient way to concatenate DatasetDict objects

**URL:** <https://discuss.huggingface.co/t/efficient-way-to-concatenate-datasetdict-objects/42395>\
**Category:** 🤗Datasets\
**Created:** [June 7, 2023, 11:13am UTC](https://discuss.huggingface.co/t/efficient-way-to-concatenate-datasetdict-objects/42395 "2023-06-07T11:13:51Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![bhavitvyamalik](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/bhavitvyamalik/32/30215_2.png) [@bhavitvyamalik](https://discuss.huggingface.co/u/bhavitvyamalik)\
**Post date:** [June 7, 2023, 11:13am UTC](https://discuss.huggingface.co/t/efficient-way-to-concatenate-datasetdict-objects/42395/1 "2023-06-07T11:13:51Z")

</div>

Is there any efficient way to combine DatasetDict objects? Current concatenate method works for Dataset object only. Thanks!

---

<div class="post-metadata">

**Author:** ![mariosasko](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mariosasko/32/31548_2.png) [@mariosasko](https://discuss.huggingface.co/u/mariosasko)\
**Post date:** [June 12, 2023, 7:54pm UTC](https://discuss.huggingface.co/t/efficient-way-to-concatenate-datasetdict-objects/42395/2 "2023-06-12T19:54:10Z")

</div>

No, but if all of them have the same set of keys, this should work:

```python
dd = DatasetDict()
for key in dd_to_concat:
    dd[key] = datasets.concatenate_datasets([ddd[key] for ddd in dd_to_concat])

```

PS: our goal is to remove the DatasetDict API eventually, as explained in [Reduce friction in tabular dataset workflow by eliminating having splits when dataset is loaded · Issue #5189 · huggingface/datasets · GitHub](https://github.com/huggingface/datasets/issues/5189), so it’s unlikely that we will add support for `DatasetDict` objects in `concatenate_datasets`.