# Finetuning Bert for Question answering task without context

**URL:** <https://discuss.huggingface.co/t/finetuning-bert-for-question-answering-task-without-context/78680>\
**Category:** Models\
**Created:** [March 25, 2024, 2:02pm UTC](https://discuss.huggingface.co/t/finetuning-bert-for-question-answering-task-without-context/78680 "2024-03-25T14:02:13Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ma7modi](https://avatars.discourse-cdn.com/v4/letter/m/f08c70/32.png) [@Ma7modi](https://discuss.huggingface.co/u/Ma7modi)\
**Post date:** [March 25, 2024, 2:02pm UTC](https://discuss.huggingface.co/t/finetuning-bert-for-question-answering-task-without-context/78680/1 "2024-03-25T14:02:13Z")

</div>

I am fine-tuning Bert model for a question answering use case in a specific domain (medicine). My dataset has the following structure: question,answer. (it doesn’t have any context like in the squad dataset).

how should I prepare the input for the tokenizer, since I don’t have context?  
I tried to do it in the following way and I am receiving this error:

```auto
from datasets import Dataset
import pandas as pd
import torch
from transformers import BertForQuestionAnswering, TrainingArguments, Trainer
from transformers import BertTokenizer, DataCollatorWithPadding

# prepare dataset
model = BertForQuestionAnswering.from_pretrained("bert-base-uncased")
tokenizer = BertTokenizer.from_pretrained("bert-base-uncased")
data_collator = DataCollatorWithPadding(tokenizer=tokenizer)
path = "data/dataset_QA.csv"
dataset = pd.read_csv(path)

def prepare_dataset(dataset):
    # Convert the DataFrame to a Dataset
    dataset= Dataset.from_pandas(dataset)

    return dataset

# Tokenize the inputs and add padding
def tokenize_data(example):
    question = example["question"]
    context = example["answer"]
    tokenized_input = tokenizer(question, context, padding="max_length", truncation=True, max_length=512)
    return tokenized_input

dataset = prepare_dataset(dataset)
dataset = dataset.train_test_split(test_size =0.2)

print(dataset)

tokenized_dataset = dataset.map(tokenize_data,batched = True)

training_args = TrainingArguments(
    output_dir= "model_bert",
    overwrite_output_dir=True,
    logging_dir="./logs/logs_bert",
    per_device_train_batch_size=16,
    per_device_eval_batch_size=16,
    evaluation_strategy="epoch",
    logging_strategy="epoch",
    save_strategy="epoch",
    save_total_limit=2,
    num_train_epochs=3,
)
trainer = Trainer(
    model=model,
    args=training_args,
    train_dataset=tokenized_dataset["train"],
    eval_dataset=tokenized_dataset["test"],
)

trainer.train()

```

**I am receiving the following output with this error:**

DatasetDict({  
train: Dataset({  
features: [‘question’, ‘answer’],  
num\_rows: 509  
})  
test: Dataset({  
features: [‘question’, ‘answer’],  
num\_rows: 128  
})  
})

Map: 100%  
509/509 [00:01\<00:00, 334.08 examples/s]  
Map: 100%  
128/128 [00:00\<00:00, 337.44 examples/s]

raise ValueError(  
ValueError: The model did not return a loss from the inputs, only the following keys: start\_logits,end\_logits. For reference, the inputs it received are input\_ids,token\_type\_ids,attention\_mask.

Is my approach correct? If yes, how to fix this error? and If no, is there an alternative approach? Thanks in advance for your help!

---

<div class="post-metadata">

**Author:** ![cha56](https://avatars.discourse-cdn.com/v4/letter/c/8baadc/32.png) [@cha56](https://discuss.huggingface.co/u/cha56)\
**Post date:** [June 21, 2024, 11:36am UTC](https://discuss.huggingface.co/t/finetuning-bert-for-question-answering-task-without-context/78680/2 "2024-06-21T11:36:18Z")

</div>

Have you found the answer pleaase?