# Saving checkpoints when using DeepSpeed is taking abnormally long

**URL:** <https://discuss.huggingface.co/t/saving-checkpoints-when-using-deepspeed-is-taking-abnormally-long/98698>\
**Category:** DeepSpeed\
**Created:** [July 22, 2024, 7:26am UTC](https://discuss.huggingface.co/t/saving-checkpoints-when-using-deepspeed-is-taking-abnormally-long/98698 "2024-07-22T07:26:58Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![seanswyi](https://avatars.discourse-cdn.com/v4/letter/s/6f9a4e/32.png) [@seanswyi](https://discuss.huggingface.co/u/seanswyi)\
**Post date:** [July 22, 2024, 7:26am UTC](https://discuss.huggingface.co/t/saving-checkpoints-when-using-deepspeed-is-taking-abnormally-long/98698/1 "2024-07-22T07:26:58Z")

</div>

I found a similar topic on this forum but didn’t find any helpful answers ([Saving checkpoint is too slow with deepspeed](https://discuss.huggingface.co/t/saving-checkpoint-is-too-slow-with-deepspeed/50099)).

I’m currently using DeepSpeed with a custom HuggingFace Trainer. The checkpoint saving takes around 10-12 minutes without DeepSpeed. With DeepSpeed it’s taking more than 25 minutes (I had to force-quit my process - it might take longer).

Is this normal behavior? I’m currently using ZeRO-0.