# Incorporate SARI score into run\_summarization.py example script

**URL:** <https://discuss.huggingface.co/t/incorporate-sari-score-into-run-summarization-py-example-script/29511>\
**Category:** 🤗Tokenizers\
**Created:** [January 13, 2023, 6:02pm UTC](https://discuss.huggingface.co/t/incorporate-sari-score-into-run-summarization-py-example-script/29511 "2023-01-13T18:02:24Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![brooklynsheppard](https://avatars.discourse-cdn.com/v4/letter/b/7cd45c/32.png) [@brooklynsheppard](https://discuss.huggingface.co/u/brooklynsheppard)\
**Post date:** [January 13, 2023, 6:02pm UTC](https://discuss.huggingface.co/t/incorporate-sari-score-into-run-summarization-py-example-script/29511/1 "2023-01-13T18:02:24Z")

</div>

I am working with the example summarization script provided by Huggingface [here](https://github.com/huggingface/transformers/blob/main/examples/pytorch/summarization/run_summarization.py). My task is not quite summarization, but a different seq2seq task where I would like to use the SARI metric to evaluate my models. The compute\_metrics function in this script takes only the predictions and labels as input, making it easy to calculate ROUGE or BLEU sores, but does not include the source text, which is required for computing the SARI score. The compute\_metrics function is then given to the Seq2SeqTrainer as an argument, and I’m not sure how it is used from there. Does anyone have any guidance on how I can edit this function to be able to take the source text as an argument as well? Thanks!