# SFTTrainer and wikitext dataset

**URL:** <https://discuss.huggingface.co/t/sfttrainer-and-wikitext-dataset/77679>\
**Category:** Beginners\
**Created:** [March 16, 2024, 2:27pm UTC](https://discuss.huggingface.co/t/sfttrainer-and-wikitext-dataset/77679 "2024-03-16T14:27:44Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![Khaledai](https://avatars.discourse-cdn.com/v4/letter/k/34f0e0/32.png) [@Khaledai](https://discuss.huggingface.co/u/Khaledai)\
**Post date:** [March 16, 2024, 2:27pm UTC](https://discuss.huggingface.co/t/sfttrainer-and-wikitext-dataset/77679/1 "2024-03-16T14:27:44Z")

</div>

I downloaded the wikitext dataset:

```auto
from datasets import load_dataset

dataset = load_dataset("wikitext",'wikitext-103-raw-v1')

```

After downloading it I used SFTtrainer:

```auto
from trl import SFTTrainer

max_seq_length = 512

trainer = SFTTrainer(
    model=model,
    train_dataset=dataset["train"],
    peft_config=peft_config,
    dataset_text_field="text",
    max_seq_length=max_seq_length,
    tokenizer=tokenizer,
    args=training_arguments,
)

```

After using `trainer.train()` it gave the error "IndexError: Invalid key: 1593817 is out of bounds for size 0  
"