# Questions about Dataset.map()

**URL:** <https://discuss.huggingface.co/t/questions-about-dataset-map/102961>\
**Category:** 🤗Datasets\
**Created:** [August 19, 2024, 10:01am UTC](https://discuss.huggingface.co/t/questions-about-dataset-map/102961 "2024-08-19T10:01:59Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Zaquan](https://avatars.discourse-cdn.com/v4/letter/z/e95f7d/32.png) [@Zaquan](https://discuss.huggingface.co/u/Zaquan)\
**Post date:** [August 19, 2024, 10:01am UTC](https://discuss.huggingface.co/t/questions-about-dataset-map/102961/1 "2024-08-19T10:01:59Z")

</div>

I encountered a problem when I used **‘map()’** for data preprocessing, my preprocessing function returned a **4d numpy.ndarray** , but when I used the **‘map()’** function, I printed **‘dataset.features’** , and found that the features in the column **‘image’** were not a 4d numpy.ndarray but a nested **Sequence** , and if I promised that a certain sample’s shape, there will be an error: **List has no attribute shape**  
Here is the snap of my code and output of dataset.features

 ![code](https://us1.discourse-cdn.com/hellohellohello/original/3X/4/3/432ed9c11958982149f5e2fee35fd236b5b4b8d7.png)

dataset.features

{‘flair\_path’: Value(dtype=‘string’, id=None), ‘t1ce\_path’: Value(dtype=‘string’, id=None), ‘seg\_path’: Value(dtype=‘string’, id=None), ‘label’: Value(dtype=‘int64’, id=None), ‘image’: Sequence(feature=Sequence(feature=Sequence(feature=Sequence(feature=Value(dtype=‘float32’, id=None), length=-1, id=None), length=-1, id=None), length=-1, id=None), length=-1, id=None)}

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [August 19, 2024, 1:41pm UTC](https://discuss.huggingface.co/t/questions-about-dataset-map/102961/2 "2024-08-19T13:41:59Z")

</div>

Maybe you can use the Array4D feature type instead ?

---

<div class="post-metadata">

**Author:** ![Zaquan](https://avatars.discourse-cdn.com/v4/letter/z/e95f7d/32.png) [@Zaquan](https://discuss.huggingface.co/u/Zaquan)\
**Post date:** [August 19, 2024, 3:46pm UTC](https://discuss.huggingface.co/t/questions-about-dataset-map/102961/3 "2024-08-19T15:46:28Z")

</div>

I followed your advice, I artificially defined the type of ‘image’ to be Array4D, and when I print the features of dataset, the features of image is indeed Array4D, but the problem is that when I try to print the dataset[0][‘image’].shape, it still throws the same error,: **List has no attribute ‘shape’**

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [August 19, 2024, 8:22pm UTC](https://discuss.huggingface.co/t/questions-about-dataset-map/102961/4 "2024-08-19T20:22:15Z")

</div>

Can you share the stack trace ?

---

<div class="post-metadata">

**Author:** ![Zaquan](https://avatars.discourse-cdn.com/v4/letter/z/e95f7d/32.png) [@Zaquan](https://discuss.huggingface.co/u/Zaquan)\
**Post date:** [August 20, 2024, 2:52am UTC](https://discuss.huggingface.co/t/questions-about-dataset-map/102961/5 "2024-08-20T02:52:33Z")

</div>

Sure.

 ![code](https://us1.discourse-cdn.com/hellohellohello/original/3X/1/1/11b31d288835e53fe4b2ee416865105bc05c37f7.png)

Exception has occurred: AttributeError  
‘list’ object has no attribute ‘shape’  
File “/root/autodl-tmp/dataset/dataset.py”, line 90, in   
print(dataset[0][‘image’].shape)  
AttributeError: ‘list’ object has no attribute ‘shape’

**Another interesting point is that when I add the following lines of code：**

 ![src](https://us1.discourse-cdn.com/hellohellohello/original/3X/1/f/1fc1a16c23e5c0d93559a741cec212c32ea4ce33.png)

The output looks like this:

```auto
<class 'list'>
Image saved to /root/autodl-tmp/dataset/first_image.npy
Loaded image shape: (16, 160, 160, 2)

```

---

<div class="post-metadata">

**Author:** ![lhoestq](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lhoestq/32/52888_2.png) [@lhoestq](https://discuss.huggingface.co/u/lhoestq)\
**Post date:** [August 20, 2024, 1:26pm UTC](https://discuss.huggingface.co/t/questions-about-dataset-map/102961/6 "2024-08-20T13:26:10Z")

</div>

I see ! You’re getting pythons lists which is the default behavior, while you want to get numpy arrays

To get numpy arrays you can change the dataset format to “numpy”:

```python
dataset = dataset.with_format("numpy")

```

now all the examples will be returned as numpy arrays 🙂

---

<div class="post-metadata">

**Author:** ![Zaquan](https://avatars.discourse-cdn.com/v4/letter/z/e95f7d/32.png) [@Zaquan](https://discuss.huggingface.co/u/Zaquan)\
**Post date:** [August 20, 2024, 4:37pm UTC](https://discuss.huggingface.co/t/questions-about-dataset-map/102961/7 "2024-08-20T16:37:01Z")

</div>

That works. Thank you so much for your patience！
