madlad400-10b-mt-ct2-bfloat16

google/madlad400-10b-mt converted to the CTranslate2 format with bfloat16 weights.

MADLAD-400-10B-MT is a multilingual machine translation model covering over 400 languages, trained on 250 billion tokens. This repo changes the runtime format.

Files

File	Size	Purpose
model.bin	~21 GB	CTranslate2 model weights (bfloat16)
config.json	—	CTranslate2 runtime config
shared_vocabulary.json	—	CTranslate2 vocabulary
spiece.model, tokenizer.json, tokenizer_config.json	—	SentencePiece / T5 tokenizer

License and attribution

Apache 2.0, inherited from the original model. Please cite the original work:

@misc{kudugunta2023madlad400,
      title={MADLAD-400: A Multilingual And Document-Level Large Audited Dataset},
      author={Sneha Kudugunta and Isaac Caswell and Biao Zhang and Xavier Garcia and
              Christopher A. Choquette-Choo and Katherine Lee and Derrick Xin and
              Aditya Kusupati and Romi Stella and Ankur Bapna and Orhan Firat},
      year={2023},
      eprint={2309.04662},
      archivePrefix={arXiv},
      primaryClass={cs.CL}
}

Refer to the original model card for intended use, limitations, and evaluation details.