# Using hyperparameter-search in Trainer **URL:** \ **Category:** 🤗Transformers\ **Created:** [August 20, 2020, 12:58pm UTC](https://discuss.huggingface.co/t/using-hyperparameter-search-in-trainer/785 "2020-08-20T12:58:34Z")\ **Posts on this page:** 1\ **Showing post:** 25
**Author:** ![lewtun](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/lewtun/32/30995_2.png) [@lewtun](https://discuss.huggingface.co/u/lewtun)\ **Post date:** [October 22, 2020, 3:18pm UTC](https://discuss.huggingface.co/t/using-hyperparameter-search-in-trainer/785/25 "2020-10-22T15:18:22Z")
Hi @sgugger, in case you’re not aware of it, it seems the latest commit on [master](https://github.com/huggingface/transformers/commit/e174bfeb340d3d3468d9c8eebce95c42aa2dcf84) broke the [Colab notebook](https://twitter.com/GuggerSylvain/status/1318215507242291202) you shared on Twitter Trying to run that notebook, I hit the following error when trying to run ```auto best_run = trainer.hyperparameter_search(n_trials=10, direction="maximize") ``` with the optuna backend. Stack trace: ```auto /usr/local/lib/python3.6/dist-packages/sklearn/metrics/_classification.py:900: RuntimeWarning: invalid value encountered in double_scalars [W 2020-10-22 14:58:41,815] Trial 0 failed because of the following error: RuntimeError("Metrics contains more entries than just 'eval_loss', 'epoch' and 'total_flos', please provide your own compute_objective function.",) Traceback (most recent call last): File "/usr/local/lib/python3.6/dist-packages/optuna/study.py", line 799, in _run_trial result = func(trial) File "/usr/local/lib/python3.6/dist-packages/transformers/integrations.py", line 114, in _objective trainer.train(model_path=model_path, trial=trial) File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 803, in train self._maybe_log_save_evaluate(tr_loss, model, trial, epoch) File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 855, in _maybe_log_save_evaluate self._report_to_hp_search(trial, epoch, metrics) File "/usr/local/lib/python3.6/dist-packages/transformers/trainer.py", line 537, in _report_to_hp_search self.objective = self.compute_objective(metrics.copy()) File "/usr/local/lib/python3.6/dist-packages/transformers/trainer_utils.py", line 120, in default_compute_objective "Metrics contains more entries than just 'eval_loss', 'epoch' and 'total_flos', please provide your own compute_objective function." RuntimeError: Metrics contains more entries than just 'eval_loss', 'epoch' and 'total_flos', please provide your own compute_objective function. --------------------------------------------------------------------------- RuntimeError Traceback (most recent call last) in () ----> 1 best_run = trainer.hyperparameter_search(n_trials=10, direction="maximize") 10 frames /usr/local/lib/python3.6/dist-packages/transformers/trainer_utils.py in default_compute_objective(metrics) 118 if len(metrics) != 0: 119 raise RuntimeError( --> 120 "Metrics contains more entries than just 'eval_loss', 'epoch' and 'total_flos', please provide your own compute_objective function." 121 ) 122 return loss RuntimeError: Metrics contains more entries than just 'eval_loss', 'epoch' and 'total_flos', please provide your own compute_objective function. ``` I tried passing the `dict` produced by `trainer.evaluate()` in the `compute_objective` arg, but this complains that `TypeError: 'dict' object is not callable`. I’d be happy to fix the docs / code if you can give me some pointers on where to start! --- _[View the full topic](https://discuss.huggingface.co/t/using-hyperparameter-search-in-trainer/785)._