# Evaluate a fine-tune zero-shot Facebook model error

**URL:** <https://discuss.huggingface.co/t/evaluate-a-fine-tune-zero-shot-facebook-model-error/40146>\
**Category:** Beginners\
**Created:** [May 17, 2023, 8:56am UTC](https://discuss.huggingface.co/t/evaluate-a-fine-tune-zero-shot-facebook-model-error/40146 "2023-05-17T08:56:26Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![LidorPrototype](https://avatars.discourse-cdn.com/v4/letter/l/91b2a8/32.png) [@LidorPrototype](https://discuss.huggingface.co/u/LidorPrototype)\
**Post date:** [May 17, 2023, 8:56am UTC](https://discuss.huggingface.co/t/evaluate-a-fine-tune-zero-shot-facebook-model-error/40146/1 "2023-05-17T08:56:26Z")

</div>

I’m finetuning the zero-shot `facebook/bart-large-mnli` model.

These is the new metric I used ([from here](https://huggingface.co/docs/transformers/main/training#evaluate)):

```python
import numpy as np
import evaluate

metric = evaluate.load("accuracy")

def compute_metrics(eval_pred):
    logits, labels = eval_pred
    predictions = np.argmax(logits, axis=-1)
    return metric.compute(predictions=predictions, references=labels)

```

This is how I train it:

```python
training_args = TrainingArguments(
    output_dir=model_directory, # output directory
    num_train_epochs=1, # total number of training epochs - 3
    per_device_train_batch_size=1, # batch size per device during training - 16
    per_device_eval_batch_size=2, # batch size for evaluation - 64
    warmup_steps=50, # number of warmup steps for learning rate scheduler - 500
    weight_decay=0.01, # strength of weight decay
    evaluation_strategy="epoch"
)

model = BartForSequenceClassification.from_pretrained("facebook/bart-large-mnli") # , num_labels=len(label_to_int), ignore_mismatched_sizes=True

trainer = Trainer(
    model=model, # the instantiated 🤗 Transformers model to be trained
    args=training_args, # training arguments, defined above
    compute_metrics=compute_metrics, # a function to compute the metrics
    train_dataset=train_dataset, # training dataset
    eval_dataset=test_dataset # evaluation dataset
)

# Train the trainer
trainer.train()

```

And I get the following error:

```python
ValueError: could not broadcast input array from shape (132,3) into shape (132,)

```

It looks like I get this error after the training is almost over or something like that, so I also tried using:

```python
trainer.evaluate()

```

And I get the following:

```python
The following columns in the evaluation set don't have a corresponding argument in `BartForSequenceClassification.forward` and have been ignored: input_sentence. If input_sentence are not expected by `BartForSequenceClassification.forward`, you can safely ignore this message.
*****Running Evaluation*****
  Num examples = 132
  Batch size = 8
warning: Databricks notebooks do not support updating results across cells.
ValueError: could not broadcast input array from shape (132,3) into shape (132,)
``

Why is this? How can I fix this?
```