# Question about gradient checkpointing and use\_cache

**URL:** <https://discuss.huggingface.co/t/question-about-gradient-checkpointing-and-use-cache/96301>\
**Category:** Beginners\
**Created:** [July 9, 2024, 6:02am UTC](https://discuss.huggingface.co/t/question-about-gradient-checkpointing-and-use-cache/96301 "2024-07-09T06:02:46Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![WenyangHui](https://avatars.discourse-cdn.com/v4/letter/w/0ea827/32.png) [@WenyangHui](https://discuss.huggingface.co/u/WenyangHui)\
**Post date:** [July 9, 2024, 6:02am UTC](https://discuss.huggingface.co/t/question-about-gradient-checkpointing-and-use-cache/96301/1 "2024-07-09T06:02:46Z")

</div>

I want to use the past\_key\_values to compute some losses during training, but gradient checkpointing forbids use\_cache in CLMs. I wonder why there is such a conflict.

---

<div class="post-metadata">

**Author:** ![RaushanTurganbay](https://avatars.discourse-cdn.com/v4/letter/r/848f3c/32.png) [@RaushanTurganbay](https://discuss.huggingface.co/u/RaushanTurganbay)\
**Post date:** [July 9, 2024, 2:12pm UTC](https://discuss.huggingface.co/t/question-about-gradient-checkpointing-and-use-cache/96301/2 "2024-07-09T14:12:44Z")

</div>

I think it’s because gradient checkpointing forces us to do several forward passes to compute the gradients again. If we have use past\_key\_values in such a scenario, the cache will be filled with repeting keys/values from extra forward calls.

---

<div class="post-metadata">

**Author:** ![system](https://us1.discourse-cdn.com/hellohellohello/original/2X/d/de4155eb4aa4108ecb32a1389d7cc37ae69f88b7.png) [@system](https://discuss.huggingface.co/u/system)\
**Post date:** [July 10, 2024, 4:11am UTC](https://discuss.huggingface.co/t/question-about-gradient-checkpointing-and-use-cache/96301/3 "2024-07-10T04:11:40Z")

</div>

Closed on July 10, 2024, 4:11am UTC

This topic was automatically closed 12 hours after the last reply. New replies are no longer allowed.