# HF accelerate DeepSpeed plugin does not use custom optimizer or scheduler

**URL:** <https://discuss.huggingface.co/t/hf-accelerate-deepspeed-plugin-does-not-use-custom-optimizer-or-scheduler/143459>\
**Category:** 🤗Accelerate\
**Created:** [February 28, 2025, 5:06pm UTC](https://discuss.huggingface.co/t/hf-accelerate-deepspeed-plugin-does-not-use-custom-optimizer-or-scheduler/143459 "2025-02-28T17:06:29Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![jpcorb20](https://avatars.discourse-cdn.com/v4/letter/j/f4b2a3/32.png) [@jpcorb20](https://discuss.huggingface.co/u/jpcorb20)\
**Post date:** [February 28, 2025, 5:06pm UTC](https://discuss.huggingface.co/t/hf-accelerate-deepspeed-plugin-does-not-use-custom-optimizer-or-scheduler/143459/1 "2025-02-28T17:06:29Z")

</div>

Hello,

I am trying to launch the training of a large model in multi-node/multi-gpu setting with “accelerate” using DeepSpeed plugin (no DS config file) with 8-bit adam and LR cosine annealing scheduler. Yet, deepspeed doesn’t seem to use the 8-bit adam from BnB set in my python script but rather regular AdamW, while the documentation seems to indicate that this should work for custom optimizer/scheduler… Any idea what’s happening here? Is there a specific setup for this?

thanks

---

<div class="post-metadata">

**Author:** ![jpcorb20](https://avatars.discourse-cdn.com/v4/letter/j/f4b2a3/32.png) [@jpcorb20](https://discuss.huggingface.co/u/jpcorb20)\
**Post date:** [March 1, 2025, 4:23pm UTC](https://discuss.huggingface.co/t/hf-accelerate-deepspeed-plugin-does-not-use-custom-optimizer-or-scheduler/143459/2 "2025-03-01T16:23:13Z")

</div>

looks like there is an implementation with the trainer by setting the training argument `optim="adam_bnb_8bit"` and this way it works … Not sure why the custom instantiation is not working …

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [March 2, 2025, 4:23am UTC](https://discuss.huggingface.co/t/hf-accelerate-deepspeed-plugin-does-not-use-custom-optimizer-or-scheduler/143459/3 "2025-03-02T04:23:14Z")

</div>

Closed on March 2, 2025, 4:23am UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.