# How to merge two dataset objects? **URL:** \ **Category:** Beginners\ **Created:** [August 24, 2020, 4:48pm UTC](https://discuss.huggingface.co/t/how-to-merge-two-dataset-objects/844 "2020-08-24T16:48:13Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![putopavel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/putopavel/32/273_2.png) [@putopavel](https://discuss.huggingface.co/u/putopavel)\ **Post date:** [August 24, 2020, 4:48pm UTC](https://discuss.huggingface.co/t/how-to-merge-two-dataset-objects/844/1 "2020-08-24T16:48:13Z")
Hi everyone! I have two datasets, loaded as CSV files, which have the same features/columns. I would like to know if there is a way to merge both datasets into a larger one (like I would do with `pd.concat((df_1, df_2))`using pandas. In case that such method does not exist, would it be interesting to implement such functionality? Thanks in advance 🤗 ---
**Author:** ![valhalla](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/valhalla/32/4179_2.png) [@valhalla](https://discuss.huggingface.co/u/valhalla)\ **Post date:** [August 24, 2020, 7:18pm UTC](https://discuss.huggingface.co/t/how-to-merge-two-dataset-objects/844/2 "2020-08-24T19:18:55Z")
I would rather combine the csv’s 😁 ---
**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\ **Post date:** [August 24, 2020, 7:30pm UTC](https://discuss.huggingface.co/t/how-to-merge-two-dataset-objects/844/3 "2020-08-24T19:30:42Z")
Are you using 🤗[nlp](https://github.com/huggingface/nlp) ? If so, you could try `nlp.concatenate_datasets` 🙂 ---
**Author:** ![putopavel](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/putopavel/32/273_2.png) [@putopavel](https://discuss.huggingface.co/u/putopavel)\ **Post date:** [August 24, 2020, 9:06pm UTC](https://discuss.huggingface.co/t/how-to-merge-two-dataset-objects/844/4 "2020-08-24T21:06:34Z")
Thank you. That is exactly what I was looking for, but I couldn’t find it in the documentation (now that I now the method I can find it in the API when I autocomplete the code, but it doesn’t appear anywhere in the documentation). ---
**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\ **Post date:** [May 3, 2021, 5:53pm UTC](https://discuss.huggingface.co/t/how-to-merge-two-dataset-objects/844/5 "2021-05-03T17:53:03Z")
Quick update since I see that this thread still has views: `concatenate_datasets` is available through the `datasets` library [here](https://github.com/huggingface/datasets), since the library was renamed. ---
**Author:** ![mmiakashs](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mmiakashs/32/1959_2.png) [@mmiakashs](https://discuss.huggingface.co/u/mmiakashs)\ **Post date:** [June 25, 2021, 8:58am UTC](https://discuss.huggingface.co/t/how-to-merge-two-dataset-objects/844/6 "2021-06-25T08:58:21Z")
Hi @lhoestq , thanks for the solution. I follow that approach but getting errors to merge two datasets ```auto dataset_ar = load_dataset('wikipedia',language='ar', date='20210320', beam_runner='DirectRunner') dataset_bn = load_dataset('wikipedia',language='bn', date='20210320', beam_runner='DirectRunner') ``` I tried two ways to concatenate but both approaches give errors. Could you please help to find out what am I missing? Thanks **First Approach** ```auto dataset_cc = concatenate_datasets(dataset_ar, dataset_bn) ``` > Traceback (most recent call last): > File “”, line 1, in > File “/home/anaconda3/envs/nlp/lib/python3.9/site-packages/datasets/arrow\_dataset.py”, line 3135, in concatenate\_datasets > if axis == 0 and not all([dset.features.type == dsets[0].features.type for dset in dsets]): > File “/home/anaconda3/envs/nlp/lib/python3.9/site-packages/datasets/arrow\_dataset.py”, line 3135, in > if axis == 0 and not all([dset.features.type == dsets[0].features.type for dset in dsets]): > AttributeError: ‘str’ object has no attribute ‘features’ **Second Approach** ```auto dataset_cc = concatenate_datasets(dataset_ar['train'], dataset_bn['train']) ``` > Traceback (most recent call last): > File “”, line 1, in > File “/home/anaconda3/envs/nlp/lib/python3.9/site-packages/datasets/arrow\_dataset.py”, line 3135, in concatenate\_datasets > if axis == 0 and not all([dset.features.type == dsets[0].features.type for dset in dsets]): > File “/home/anaconda3/envs/nlp/lib/python3.9/site-packages/datasets/arrow\_dataset.py”, line 3135, in > if axis == 0 and not all([dset.features.type == dsets[0].features.type for dset in dsets]): > AttributeError: ‘dict’ object has no attribute ‘features’ ---
**Author:** ![mmiakashs](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/mmiakashs/32/1959_2.png) [@mmiakashs](https://discuss.huggingface.co/u/mmiakashs)\ **Post date:** [June 25, 2021, 9:03am UTC](https://discuss.huggingface.co/t/how-to-merge-two-dataset-objects/844/7 "2021-06-25T09:03:49Z")
I solve this issue, it was a silly mistake 😜 Solution: ```auto dataset_cc = concatenate_datasets([dataset_ar['train'], dataset_bn['train']]) ``` ---
**Author:** ![AletheiaChengWon](https://avatars.discourse-cdn.com/v4/letter/a/46a35a/32.png) [@AletheiaChengWon](https://discuss.huggingface.co/u/AletheiaChengWon)\ **Post date:** [February 28, 2024, 10:04am UTC](https://discuss.huggingface.co/t/how-to-merge-two-dataset-objects/844/8 "2024-02-28T10:04:00Z")
concatenate\_datasets([part1, part2, part3]) this way~