# Am I using evaluate correctly? eval\_accuracy is super low

**URL:** <https://discuss.huggingface.co/t/am-i-using-evaluate-correctly-eval-accuracy-is-super-low/52878>\
**Category:** Beginners\
**Created:** [August 30, 2023, 9:27pm UTC](https://discuss.huggingface.co/t/am-i-using-evaluate-correctly-eval-accuracy-is-super-low/52878 "2023-08-30T21:27:48Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![thenatefisher](https://sea2.discourse-cdn.com/hellohellohello/user_avatar/discuss.huggingface.co/thenatefisher/32/17153_2.png) [@thenatefisher](https://discuss.huggingface.co/u/thenatefisher)\
**Post date:** [August 30, 2023, 9:27pm UTC](https://discuss.huggingface.co/t/am-i-using-evaluate-correctly-eval-accuracy-is-super-low/52878/1 "2023-08-30T21:27:48Z")

</div>

I’m training codellama using PEFT/SFT, split my dataset between train/test, and set my compute\_metrics training param to the following. My train/test data is the same format, which is “an input sequence -\>: an output sequence ”.

```auto
metric = evaluate.load("accuracy")
def compute_metrics(eval_pred):
    
    eval_acc = 0.0
    predictions = np.argmax(eval_pred[0], axis=-1).astype("int32")
    
    accs = []
    for i in range(len(eval_pred[0])):
        acc = metric.compute(predictions=predictions[i], references=eval_pred[1][i].astype("int32"))
        accs.append(acc["accuracy"])
    
    return {"accuracy": np.mean(np.array(accs))}

```

Basically, I am comparing the predictions against the inputs and averaging them. However, after sevearl epochs of training, eval accuracy is still 0.00102. Does that seem right? Thanks for your insight, all-