NAMAA-Space/alexandriax-arat5v2-base

AlexandriaX-2026 · Subtask 1 — AraT5v2-base, full fine-tune

English to dialectal Arabic dialogue translation over 13 Arabic varieties, from the NAMAA Community submission to AlexandriaX-2026 (ArabicNLP 2026 / EMNLP). Full fine-tune of UBC-NLP/AraT5v2-base-1024; the target dialect is selected with a natural-language prefix, not a language code.

This was the strongest of the team's five small constrained-track fine-tunes — a 368M encoder-decoder that beat every larger decoder-only fine-tune trained on the same data — and it was reused unchanged as a routing candidate inside the unconstrained-track ensemble, where it supplied the winning output for Mauritanian and Omani.

Task	AlexandriaX-2026 Subtask 1 (context-aware EN→DA dialogue translation)
Base model	UBC-NLP/AraT5v2-base-1024 (T5, 12+12 layers, d_model 768, 110,208-token SentencePiece vocab)
Parameters	approx. 368M, all trained (full fine-tune, no adapters)
Dialect control	text prefix — translate English to {Dialect} Arabic: {source}
Context	none — this system translates each turn in isolation
Dev (12,250 turns, 11 countries)	25.12 spBLEU · 40.66 chrF++
Blind test (14,459 turns, 13 countries)	23.26 spBLEU · 39.03 chrF++
Track	constrained (provided data only, ≤5B parameters)
License	Apache-2.0, inherited from AraT5v2

Usage

Use the tokenizer that ships in this repo, and do not force use_fast=False. The repo carries AraT5v2's real 110k-token SentencePiece vocabulary as a fast tokenizer, verified token-id-identical to spiece.model. The slow T5 tokenizer path (use_fast=False, or any tokenizer_class: T5Tokenizer config) raises argument 'vocab': 'dict' object cannot be converted to 'Sequence' on transformers 5.x — and in this project a silent fallback from that very error is what voided a sibling checkpoint. Keep the Arabic-output assertion below in your pipeline regardless.

import torch
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM

REPO = "NAMAA-Space/alexandriax-arat5v2-base"

tok = AutoTokenizer.from_pretrained(REPO)          # fast backend, 110,100 tokens
model = AutoModelForSeq2SeqLM.from_pretrained(REPO, torch_dtype=torch.float32)
model.tie_weights()
model.eval()

DIALECT = {"EG": "Egyptian", "JO": "Jordanian", "LB": "Lebanese", "LY": "Libyan",
           "MA": "Moroccan", "MR": "Mauritanian", "OM": "Omani", "PS": "Palestinian",
           "SA": "Saudi", "SD": "Sudanese", "SY": "Syrian", "TN": "Tunisian",
           "YE": "Yemeni"}

def translate(sentences, country, num_beams=5, max_new_tokens=128):
    prompts = [f"translate English to {DIALECT[country]} Arabic: {s}" for s in sentences]
    enc = tok(prompts, return_tensors="pt", padding=True, truncation=True, max_length=256)
    enc = {k: v.to(model.device) for k, v in enc.items()}
    with torch.no_grad():
        out = model.generate(**enc, num_beams=num_beams,
                             max_new_tokens=max_new_tokens, length_penalty=1.0)
    return tok.batch_decode(out, skip_special_tokens=True)

preds = translate(["Good morning. How much for the whole quantity?"], "EG")
print(preds)

# Sanity check: if this fails the tokenizer is wrong, not the weights.
assert any("؀" <= ch <= "ۿ" for ch in preds[0]), "output is not Arabic"

Tokenizer files in this repo, and what to do on transformers 4.x

File	Purpose
tokenizer.json	AraT5v2's 110,100-token fast tokenizer — verified to produce byte-identical ids to spiece.model + </s>, i.e. the same ids the model was trained on
spiece.model	the original SentencePiece model, kept for anyone who needs the slow path on transformers 4.x
tokenizer_config.json	declares both backend: tokenizers (transformers 5.x) and tokenizer_class: PreTrainedTokenizerFast (transformers 4.x), so AutoTokenizer works on either
special_tokens_map.json	<pad>=0, </s>=1, <unk>=2, plus the 100 <extra_id_*> sentinels

On transformers 4.x the slow path also works if you want it: T5Tokenizer.from_pretrained(REPO, legacy=True, use_fast=False). On 5.x it does not — use the default fast tokenizer above. Either way the ids are the same.

Reference decoding for every number in this card: beam search with 5 beams, length_penalty=1.0, max_new_tokens=128, source truncated at 256 tokens.

Intended use

Research on dialectal Arabic MT and on evaluation of it: reproducing the AlexandriaX-2026 Subtask-1 results, a fine-tuning baseline for country-level Arabic varieties, and a data-efficient counterpoint to prompted LLMs. Not intended for production translation without human review, and not a dialect identifier.

---

The shared task

AlexandriaX-2026 (ArabicNLP 2026 / EMNLP) — Context-Aware Dialectal Arabic MT and MT Evaluation. This model was built for Subtask 1: Context-Aware English-to-Dialectal Arabic Dialogue Translation.

Given one English dialogue turn together with its conversation history and metadata — target country/dialect, domain, participant roles, speaker, and speaker→addressee gender direction — the system must produce the turn in the requested country's spoken Arabic, preserving meaning while adapting lexical, morphological, pragmatic and sociolinguistic choices to that variety.

Two tracks: constrained (provided data only, ≤5B parameters) and unconstrained (any external data or model). Ranking is by spBLEU (primary) and chrF++ (secondary), each macro-averaged over countries.

Official data (UBC-NLP/alexandria)

Split sizes in turns, as published by the organisers:

Split	EG	JO	LB	LY	MA	MR	OM	PS	SA	SD	SY	TN	YE	Total
train	3,108	5,501	8,906	0	2,573	5,515	6,280	14,933	8,470	0	6,071	2,034	3,089	66,480
dev	1,113	1,113	1,118	0	1,110	1,114	1,109	1,110	1,110	0	1,119	1,116	1,118	12,250
public test	1,118	1,107	1,106	1,109	1,115	1,112	1,118	1,109	1,113	1,106	1,114	1,109	1,106	14,442
private (blind) test	1,113	1,109	1,110	1,309	1,111	1,119	1,107	1,111	1,114	915	1,114	1,114	1,113	14,459

Libyan (LY) and Sudanese (SD) appear only at test time — they are zero-shot for every system trained on this data.

Conversation-level counts: 21,146 train / 3,963 dev / 4,706 public-test conversations; mean 3.13 turns per conversation (range 1–5). Mean length 102 characters of English source, 74 characters of dialectal target.

Dialects (13 countries). Egyptian, Jordanian, Lebanese, Libyan, Moroccan, Mauritanian, Omani, Palestinian, Saudi, Sudanese, Syrian, Tunisian, Yemeni. Labels are country + sub-dialect, and several countries carry more than one: Palestinian 10 (Nabulsi and Albira urban, plus Falahi varieties of Surif, Kobar, Noba, Ni'lin, Shuqba, Aboud, Silwad, Ramallah), Omani 5 (Suri, Rustaqi, Al-Wafi, Ibri, Seebi), Saudi 3 (Southern, Hijazi, Khaleeji), Yemeni 3 (Taiz, San'ani, Central), Syrian 2 (Levantine Standard, Homsi). The remaining countries carry one label each (e.g. Egyptian Arabic (Cairene), Moroccan Standard Darija, Mauritanian Hassaniya, Libyan Arabic (Misrati/Central)).

Domains (11, near-uniform). Agriculture and farming, Commerce and transactions, Construction and real estate, Education and academia, Energy and resources, Everyday and social, Healthcare and medical, Legal and financial, Logistics and transportation, Professional and workplace, Tourism and hospitality.

Speaker direction (turns, train+dev+public test): female→male 30,636 · male→female 30,203 · male→male 20,465 · female→female 11,868. The corpus carries 76 distinct translator IDs and 44 reviewer IDs.

Code-switching in the gold is strongly dialect-specific — the share of gold turns containing Latin characters runs from TN 39.1% / MA 33.8% / LB 18.1% down to SY 1.2% / YE 0.8%. Systems that normalise every borrowing into Arabic script are penalised hardest on Maghrebi references (see Known limitations).

Evaluation protocol

• spBLEU — sacrebleu.BLEU(tokenize="flores200"), corpus-level per country, then averaged over countries.
• chrF++ — sacrebleu.CHRF(word_order=2), same averaging.
• Decoding is turn-by-turn: at turn n the conversation history contains the system's own previous outputs, never the gold ones. (An early evaluation harness in this project leaked gold previous-turn Arabic into the prompt and inflated scores by ≈2.4 spBLEU; every number reported here comes from the corrected, self-conditioned harness.)

---

Training data

The official Subtask-1 training conversations only — 63,130 English→dialect turn pairs over the 11 countries that have a train split (the run held roughly 5% of the official 66,480-turn train split aside as an internal dev set). No conversation context, no auxiliary monolingual corpus, no back-translation.

Turns per country as seen in training:

PS	LB	SA	OM	SY	MR	JO	YE	EG	MA	TN
14,183	8,464	8,035	5,965	5,760	5,234	5,224	2,946	2,943	2,443	1,933
22.5%	13.4%	12.7%	9.4%	9.1%	8.3%	8.3%	4.7%	4.7%	3.9%	3.1%

That 7:1 skew between Palestinian and Tunisian is what motivated the sibling alexandriax-arat5v2-balanced checkpoint. LY and SD are absent from training entirely and are produced zero-shot, from the prefix alone.

Training procedure — every hyperparameter

Extracted verbatim from the Colab notebook that produced this checkpoint (AlexandriaX_NB8_AraT5v2_Full.ipynb). The runnable single-file version is in this repo as train_arat5v2_base.py.

Model and data

Setting	Value	Note
Base model	UBC-NLP/AraT5v2-base-1024	T5 architecture
Regime	full fine-tune	every parameter updated — no LoRA, no quantisation
Trainable parameters	367,508,736 (100%)
Tokenizer	AutoTokenizer.from_pretrained(base)	110,208-token SentencePiece
max_length source	256 tokens	truncation, no padding at map time
max_length target	256 tokens
Label padding	-100	via DataCollatorForSeq2Seq(label_pad_token_id=-100)
Training examples	63,130 turn pairs	11 countries; LY/SD absent
Shuffle seed	42	datasets.Dataset.shuffle(seed=42)
model.config.use_cache	False during training	re-enabled for generation

Optimisation

Setting	Value	Note
Optimiser	adafactor	the standard T5 recipe; memory-light at this size
Learning rate	1e-3	Adafactor's usual T5 range, far above an AdamW LR
LR scheduler	linear
Warmup	200 steps
Epochs	10	AraT5v2 is MSA-leaning and needs more passes than NLLB/NileChat
per_device_train_batch_size	8	sized for a 16 GB T4
gradient_accumulation_steps	4
Effective batch	32	8 × 4
Optimiser steps	≈19,730	63,130 / 32 × 10 epochs
Weight decay	0.0 (default)	not tuned
Gradient clipping	1.0 (default)	not tuned
Label smoothing	none
Gradient checkpointing	off	not needed at 368M

Precision and hardware

Setting	Value	Note
Hardware	1 × Tesla T4, 16 GB	Colab
fp16	True	the released run
bf16	False	a T4 has no bfloat16
TF32	disabled	allow_tf32 = False — TF32 is Ampere-only
Model load dtype	float32	fp32 master weights, fp16 autocast
group_by_length	True	length-bucketed batches; a real speedup on 63k short turns

Bookkeeping

Setting	Value
logging_steps	25
save_steps	500
save_total_limit	2
eval_strategy	"no" (dev scored separately, after training)
save_safetensors	False — why this repo carries pytorch_model.bin, not model.safetensors
Resume	automatic from the highest checkpoint-* in the output dir
Library	transformers 4.43.4

Inference (used for every score in this card)

Setting	Value
Decoding	beam search, num_beams=5
length_penalty	1.0
max_new_tokens	128
Generation batch	32
Source truncation	256 tokens
Context	none — each turn decoded independently

Results

Per-country, official dev set (12,250 turns, 11 countries)

	EG	JO	LB	MA	MR	OM	PS	SA	SY	TN	YE	macro
spBLEU	25.83	30.28	27.51	19.75	15.21	23.67	26.98	27.32	34.22	24.76	20.84	25.12
chrF++	40.79	44.95	42.44	35.57	31.63	39.79	42.71	43.42	49.24	39.44	37.27	40.66

Per-country, private blind test (14,459 turns, 13 countries)

EG	JO	LB	LY*	MA	MR	OM	PS	SA	SD*	SY	TN	YE	macro
26.82	29.93	26.28	15.62	18.46	15.46	23.12	26.27	26.14	15.66	33.97	24.71	19.90	23.26

* zero-shot — no training or development data was released for LY/SD.

Bold marks the two countries where this 368M model beat both prompted frontier models and was selected by the ensemble's routing table: MR 15.46 (Claude Sonnet 4.5 5-shot 10.08, Gemini 2.5 Flash 5-shot 11.24) and OM 23.12 (22.04 / 22.52). Fine-tuning on in-domain targets pays off exactly where the frontier models have seen the least of the variety.

Reading the spread

Syrian is the easiest variety (34.2 dev spBLEU) and Mauritanian Hassaniya the hardest (15.2) — a 19-point spread that tracks distance from MSA and available data. The Maghrebi varieties (MA, TN) additionally lose points to the code-switching mismatch described below.

Known limitations

• Code-switching is a systematic loss. Gold Moroccan and Tunisian references keep French and English terms in Latin script (33.8% and 39.1% of gold turns contain Latin characters); this model normalises them into Arabic — producing الشهادة where the Moroccan reference writes le certificat. Defensible Arabic, wrong variety, and the n-gram credit is lost either way. Aggregated over the 11 dev dialects the gold code-switches on 9.9% of turns and this model on 9.6%, so only the per-dialect breakdown exposes the problem.
• No conversational context. Each turn is translated in isolation, so anaphora and politeness choices that depend on earlier turns are not modelled. Adding history to the decoder-only siblings did not help either (NileChat-3B lost 0.67 spBLEU with context), so this is partly a property of the reference style rather than of the architecture.
• LY and SD are extrapolated from the prefix alone; treat those outputs as untested.
• Sub-dialects are not addressed. One prefix per country, while the corpus labels up to 10 Palestinian and 5 Omani sub-dialects. Routing per sub-dialect was the single best combination strategy measured in this project (+0.28 spBLEU), so there is headroom here.
• Metric-only evaluation. Everything rests on spBLEU and chrF++: no human evaluation, and no neural metric (COMET and friends may rank dialectal fidelity differently).
• Not suitable for MSA-target translation, nor for safety-critical, legal or medical use without human review.

---

Where this model sits in the NAMAA system

All Subtask-1 systems built by the team, scored on the official 12,250-turn dev set (11 countries) and, where they were run, on the 14,459-turn private blind test (13 countries). Country-macro spBLEU / chrF++.

System	Params / arch.	dev spBLEU	dev chrF++	blind spBLEU	blind chrF++	Released
Gemma, beam search (submitted, constrained)	~3.1B, dec-only	—	—	27.413	42.58	no
Routed ensemble (submitted, unconstrained)	—	—	—	27.412	43.05	n/a
Gemini 2.5 Flash, 5-shot	API	—	—	26.68	42.49	n/a
Claude Sonnet 4.5, 5-shot	API	—	—	26.36	42.26	n/a
AraT5v2 full fine-tune	368M, enc–dec	25.12	40.66	23.26	39.03	alexandriax-arat5v2-base
Qwen2.5-1.5B LoRA	1.5B, dec-only	23.71	40.51	21.24	38.06	no
NileChat-3B QLoRA, context-free	3B, dec-only	23.54	39.68	—	—	alexandriax-nilechat-lora
NileChat-3B QLoRA, +context	3B, dec-only	22.87	39.11	—	—	no
NileChat-3B QLoRA, +context +back-translation	3B, dec-only	22.77	38.71	—	—	alexandriax-nilechat-ctx-aux
Gemma-3-1B LoRA	1B, dec-only	22.71	38.86	20.09	36.20	no
NLLB-200-1.3B QLoRA	1.3B, enc–dec	21.83	38.13	—	—	alexandriax-nllb-1.3b-lora
AraT5v2, dialect-rebalanced	368M, enc–dec	void run¹		—	—	alexandriax-arat5v2-balanced
mT5-large, dialect-rebalanced	1.23B, enc–dec	not evaluated²		—	—	alexandriax-mt5-large-balanced
MBR over 3 NileChat variants	—	23.57	39.86	—	—	n/a
MBR over 5 samples, one model	—	20.09	37.50	—	—	n/a
Linear adapter merge	—	19.90	35.33	—	—	n/a

¹ That run was trained against destroyed targets — a tokenizer fallback substituted t5-base (32,100 English tokens) for AraT5v2's 110,208-token vocabulary, so every Arabic character became <unk>. It scored 0.00 spBLEU and cannot be recovered without retraining; the post-mortem and a fixed training script are in its card.

² That run stopped at step 2,500 of a planned 31,568 (epoch 0.63 of 8) and was never decoded on the development set, so no score exists for it. Its card carries the full recovered configuration.

Two findings from this bank of models are worth carrying elsewhere.

1. Parameter count does not predict rank below the cap. The 368M encoder–decoder AraT5v2 beats every larger decoder-only fine-tune on identical data, and among the decoder-only models spBLEU falls Qwen2.5-1.5B > NileChat-3B > Gemma-3-1B — the reverse of their size order. A reading consistent with this: the metric rewards fidelity to the annotators' conventions over generative fluency. A translator fine-tuned on the provided targets acquires those conventions; a decoder-only model several times its size contributes fluency n-gram overlap does not credit.
2. Combination is not free. Fitted and evaluated on disjoint halves of the dev conversations: routing per country +0.07, per country + sub-dialect +0.28, per country + domain −0.32, MBR consensus over 5 systems −0.57, MBR over the top-2 per dialect −0.81 — against a best single system of 24.98. The per-turn oracle reaches 32.25 (+7.27), so the right output is usually in the pool and the failure is in selection: three NileChat variants agree with one another and outvote the single strongest system, so consensus weights model-family size rather than quality. The submitted system therefore routes per dialect under a ±0.40 spBLEU margin guard instead of voting.

---

The collection

All released artefacts live in NAMAA at AlexandriaX-2026:

Repo	What it is
alexandriax-arat5v2-base	AraT5v2-base full fine-tune — best small fine-tune, 25.12 dev / 23.26 blind spBLEU
alexandriax-arat5v2-balanced	the same recipe on a temperature-rebalanced dialect mixture — void run, released for the post-mortem and the fixed script
alexandriax-nilechat-lora	NileChat-3B QLoRA, context-free — best of the three NileChat variants, 23.54 dev spBLEU
alexandriax-nilechat-ctx-aux	NileChat-3B QLoRA, context + back-translation — the augmentation ablation, 22.77 dev spBLEU
alexandriax-nllb-1.3b-lora	NLLB-200-1.3B QLoRA with per-dialect language codes, 21.83 dev spBLEU
alexandriax-mt5-large-balanced	mT5-large on the rebalanced mixture — partial run (2,500/31,568 steps), never evaluated
alexandria-backtranslated-pairs	348,787 synthetic EN→dialect pairs over 14 varieties

Every model repo above carries a single-file train_*.py reproduction script with the exact hyperparameters that produced its checkpoint; the dataset repo carries build_backtranslated_pairs.py.

Official task data: UBC-NLP/alexandria. Base models: UBC-NLP/AraT5v2-base-1024, UBC-NLP/NileChat-3B-Base, facebook/nllb-200-1.3B, google/mt5-large.

---

Team

NAMAA Community — Fatimah Emad Eldin (Cairo University) · Omer Nacar (Tuwaiq Academy) · Khloud Al Jallad (Arab International University) · Mona Abdelazim (Ain Shams University).

Citation

Coming soon. The NAMAA system-description paper for AlexandriaX-2026 is under review for the ArabicNLP 2026 (EMNLP) proceedings; this card will be updated with the final ACL Anthology reference and DOI when the proceedings are published. Until then, please cite as:

@inproceedings{namaa-alexandriax-2026,
  title     = {{NAMAA} Community at {AlexandriaX-2026}: Prompting, Fine-Tuning and Agreement
               Voting for Dialectal Arabic Translation and Evaluation},
  author    = {Emad Eldin, Fatimah and Nacar, Omer and Al Jallad, Khloud and Abdelazim, Mona},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
               (ArabicNLP 2026)},
  year      = {2026},
  note      = {To appear. Citation coming soon.}
}

Please also cite the shared task and the base model:

@inproceedings{alexandriax2026,
  title     = {{AlexandriaX-2026} Shared Task: Context-Aware Dialectal Arabic Machine
               Translation and MT Evaluation},
  author    = {El Mekki, Abdellah and Elmadany, AbdelRahim A. and Magdy, Samar M. and
               Ezzini, Saad and El-Haj, Mo and Jarrar, Mustafa and El-Beltagy, Samhaa and
               Abbas, Mourad and Zaraket, Fadi and Al Mandhari, Salim and Alyafeai, Zaid and
               Ghanem, Bernard and Abdul-Mageed, Muhammad},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Conference
               (ArabicNLP 2026)},
  year      = {2026},
  note      = {Overview paper. Citation coming soon.}
}

Acknowledgements

Thanks to the AlexandriaX-2026 organisers for the data, the evaluation infrastructure and their responsiveness during the evaluation phases.