# Trainer with load\_best\_model\_at\_end doesn't work **URL:** \ **Category:** 🤗Transformers\ **Created:** [July 1, 2021, 3:42am UTC](https://discuss.huggingface.co/t/trainer-with-load-best-model-at-end-doesnt-work/7799 "2021-07-01T03:42:52Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![rlian](https://avatars.discourse-cdn.com/v4/letter/r/a88e57/32.png) [@rlian](https://discuss.huggingface.co/u/rlian)\ **Post date:** [July 1, 2021, 3:42am UTC](https://discuss.huggingface.co/t/trainer-with-load-best-model-at-end-doesnt-work/7799/1 "2021-07-01T03:42:52Z")
I was trying to save the checkpoint after each epoch by setting load\_best\_model\_at\_end=True with p3.16xlarge parallel training. However, the following error occurs: OSError Can’t load config for ‘bert\_model/checkpoint-156’. Make sure that: - ‘bert\_model/checkpoint-156’ is a correct model identifier listed on ‘[Hugging Face – The AI community building the future.](https://huggingface.co/models)’ - or ‘bert\_model/checkpoint-156’ is the correct path to a directory containing config.json file. The error occurs only if the output\_dir folder is empty but not occurs if there were checkpoints from last training. Does anyone face the same issue or have an idea on this? ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [July 1, 2021, 12:41pm UTC](https://discuss.huggingface.co/t/trainer-with-load-best-model-at-end-doesnt-work/7799/2 "2021-07-01T12:41:33Z")
What is the version of Transformers you are using? Also what were the training arguments for this training? ---
**Author:** ![rlian](https://avatars.discourse-cdn.com/v4/letter/r/a88e57/32.png) [@rlian](https://discuss.huggingface.co/u/rlian)\ **Post date:** [July 1, 2021, 1:48pm UTC](https://discuss.huggingface.co/t/trainer-with-load-best-model-at-end-doesnt-work/7799/3 "2021-07-01T13:48:19Z")
Transformers version is 4.3.3. The training arguments are: –train\_data\_file sample\_data.json –test\_data\_file sample\_data.json –output\_dir output –tokenizer\_name bert-base-uncased \ –model\_name\_or\_path bert-base-uncased \ –per\_device\_train\_batch\_size 16 \ –per\_device\_eval\_batch\_size 16 \ –do\_train \ –do\_predict –num\_train\_epochs 3 \ –learning\_rate 0.00001 \ –logging\_steps 100 \ –dataloader\_num\_workers 4 \ –evaluation\_strategy epoch \ –overwrite\_output\_dir –load\_best\_model\_at\_end True –logging\_dir logs ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [July 1, 2021, 2:20pm UTC](https://discuss.huggingface.co/t/trainer-with-load-best-model-at-end-doesnt-work/7799/4 "2021-07-01T14:20:28Z")
It’s probably a bug that has been fixed in the latest version, could you try again with v4.8.2? ---
**Author:** ![rlian](https://avatars.discourse-cdn.com/v4/letter/r/a88e57/32.png) [@rlian](https://discuss.huggingface.co/u/rlian)\ **Post date:** [July 1, 2021, 4:39pm UTC](https://discuss.huggingface.co/t/trainer-with-load-best-model-at-end-doesnt-work/7799/5 "2021-07-01T16:39:00Z")
I tried it with v4.8.2 but there was another error: File “/home/anaconda3/envs/pytorch\_p36/lib/python3.6/site-packages/transformers/trainer.py”, line 594, in \_get\_train\_sampler seed=self.args.seed, TypeError: **init** () got an unexpected keyword argument ‘seed’ subprocess.CalledProcessError: Command ‘[’/home/anaconda3/envs/pytorch\_p36/bin/python’, ‘-u’, ‘main.py’, ‘–local\_rank=7’, ‘–train\_file’, ‘…/sample\_data.json’, ‘–validation\_file’, ‘…/sample\_data.json’, ‘–output\_dir’, ‘output\_roberta’, ‘–model\_name\_or\_path’, ‘…/roberta\_base’, ‘–per\_device\_train\_batch\_size’, ‘16’, ‘–per\_device\_eval\_batch\_size’, ‘16’, ‘–do\_train’, ‘–num\_train\_epochs’, ‘1’, ‘–learning\_rate’, ‘2e-5’, ‘–weight\_decay’, ‘1e-4’, ‘–max\_seq\_length’, ‘128’, ‘–logging\_steps’, ‘100’, ‘–load\_best\_model\_at\_end’, ‘True’, ‘–dataloader\_num\_workers’, ‘4’, ‘–evaluation\_strategy’, ‘epoch’, ‘–overwrite\_output\_dir’, ‘–logging\_dir’, ‘logs’]’ returned non-zero exit status 1. My Trainer looks like: trainer = Trainer( model=model, args=training\_args, train\_dataset=train\_dataset, eval\_dataset=val\_dataset, tokenizer=tokenizer, data\_collator=data\_collator.collate\_batch ) Do you know how to fix it? ---
**Author:** ![sgugger](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/sgugger/32/2291_2.png) [@sgugger](https://discuss.huggingface.co/u/sgugger)\ **Post date:** [July 1, 2021, 10:01pm UTC](https://discuss.huggingface.co/t/trainer-with-load-best-model-at-end-doesnt-work/7799/6 "2021-07-01T22:01:06Z")
Can you give the whole stack trace and not just the end? It’s hard to see where your issue comes from otherwise. Also please put it between two ``` to format it properly: ``` stack\_trace ``` ---
**Author:** ![mmukh](https://avatars.discourse-cdn.com/v4/letter/m/258eb7/32.png) [@mmukh](https://discuss.huggingface.co/u/mmukh)\ **Post date:** [July 28, 2022, 7:57pm UTC](https://discuss.huggingface.co/t/trainer-with-load-best-model-at-end-doesnt-work/7799/7 "2022-07-28T19:57:51Z")
Hi, were you able to fix this error?