# Evaluating multiple choices using BertForMultipleChoice

**URL:** <https://discuss.huggingface.co/t/evaluating-multiple-choices-using-bertformultiplechoice/7010>\
**Category:** Beginners\
**Created:** [June 20, 2021, 3:29pm UTC](https://discuss.huggingface.co/t/evaluating-multiple-choices-using-bertformultiplechoice/7010 "2021-06-20T15:29:01Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![eric-hu2728](https://avatars.discourse-cdn.com/v4/letter/e/8e8cbc/32.png) [@eric-hu2728](https://discuss.huggingface.co/u/eric-hu2728)\
**Post date:** [June 20, 2021, 3:29pm UTC](https://discuss.huggingface.co/t/evaluating-multiple-choices-using-bertformultiplechoice/7010/1 "2021-06-20T15:29:01Z")

</div>

Hello community.  
I want to use BertForMultipleChoice to, well, answer a multiple choice question.  
However, the example script provided on

> **[BERT](https://huggingface.co/docs/transformers/model_doc/bert)**
>
> We’re on a journey to advance and democratize artificial intelligence through open source and open science.

only allows for two choices. I want to evaluate the best answer of 4 choices. Here is my script and output (using cpu only)

Any help would be greatly appreciated. I followed the error messages and considered doing some library surgery, but could not figure out where to flip the switch from “only allows 2 input sentences” to “allows more than 2 input sentences”.

Input:  
from transformers import BertTokenizer, BertForMultipleChoice  
import tensorflow as tf  
import torch

tokenizer = BertTokenizer.from\_pretrained(‘cl-tohoku/bert-base-japanese’)  
model = BertForMultipleChoice.from\_pretrained(‘cl-tohoku/bert-base-japanese’)  
prompt = “人気作家A氏の講演会が無料[MASK]、多くのファンが詰めかけた。”  
choice0 = “にして”  
choice1 = “にあって”  
choice2 = “として”  
choice3 = “とあって”  
labels = torch.tensor(0).unsqueeze(0)  
encoding = tokenizer([[prompt, prompt, prompt, prompt], [choice0, choice1, choice2, choice3]], return\_tensors=‘pt’, padding=True)  
outputs = model(\*\*{k: v.unsqueeze(0) for k,v in encoding.items()}, labels=labels)  
loss = outputs.loss  
logits = outputs.logits  
print(logits)

Error Message:  
Traceback (most recent call last):  
File “pretrain.py”, line 13, in   
encoding = tokenizer([[prompt, prompt, prompt, prompt], [choice0, choice1, choice2, choice3]], return\_tensors=‘pt’, padding=True)  
File “/home/eo/anaconda3/lib/python3.8/site-packages/transformers/tokenization\_utils\_base.py”, line 2289, in **call**  
return self.batch\_encode\_plus(  
File “/home/eo/anaconda3/lib/python3.8/site-packages/transformers/tokenization\_utils\_base.py”, line 2474, in batch\_encode\_plus  
return self.\_batch\_encode\_plus(  
File “/home/eo/anaconda3/lib/python3.8/site-packages/transformers/tokenization\_utils.py”, line 543, in \_batch\_encode\_plus  
ids, pair\_ids = ids\_or\_pair\_ids  
ValueError: too many values to unpack (expected 2)