ATM2: Arabic Triplet Matryoshka V2

Compact Arabic sentence embeddings for semantic similarity, search, and domain adaptation.

ATM2 is a 135M-parameter Arabic embedding model fine-tuned from AraBERTv02 using the Sentence Transformers framework. It maps Arabic sentences and passages into 768-dimensional vectors for comparing meaning and retrieving related content.

ATM2 is described in the GATE research paper, published in Findings of EMNLP 2026.

Published paper: GATE — Findings of EMNLP 2026
Preprint: arXiv:2505.24581

Model overview

Property	Details
Model ID	`Omartificial-Intelligence-Space/Arabic-Triplet-Matryoshka-V2`
Language	Arabic
Backbone	`aubmindlab/bert-base-arabertv02`
Architecture	Transformer encoder
Parameters	Approximately 135 million
Embedding dimension	768
Pooling	Mean pooling
Context length	Up to 512 tokens; check `model.max_seq_length` for the loaded checkpoint's effective limit
Training objective	MatryoshkaLoss with MultipleNegativesRankingLoss
Framework	Sentence Transformers
License	Apache 2.0

Intended uses

Semantic similarity: Compare Arabic sentences and identify paraphrases.
Semantic search: Retrieve passages relevant to an Arabic query.
Retrieval-augmented generation: Supply candidate evidence to a separate generation model.
Clustering and classification: Use embeddings as features for grouping or labeling text.
Domain adaptation: Fine-tune the checkpoint on task-specific query–passage examples.

ATM2 produces embeddings, not generated answers. Classification requires an additional classifier or decision rule; question answering requires downstream retrieval and answer-generation components.

Quick start

Install Sentence Transformers:

pip install -U sentence-transformers

Encode Arabic text

from sentence_transformers import SentenceTransformer

model = SentenceTransformer(
    "Omartificial-Intelligence-Space/Arabic-Triplet-Matryoshka-V2"
)

sentences = [
    "يساعد الذكاء الاصطناعي في تحسين البحث عن المعلومات.",
    "تُستخدم تقنيات الذكاء الاصطناعي لتطوير أنظمة البحث.",
    "تشتهر المنطقة بزراعة النخيل وإنتاج التمور.",
]

embeddings = model.encode(sentences, normalize_embeddings=True)
print(embeddings.shape)  # (3, 768)

# Dot products between unit-normalized vectors equal cosine similarities.
similarities = embeddings @ embeddings.T
print(similarities)

Retrieve relevant passages

import numpy as np

query = "كيف يساعد الذكاء الاصطناعي في البحث عن المعلومات؟"
passages = [
    "تستخدم محركات البحث التمثيلات الدلالية لفهم معنى الاستعلام واسترجاع النصوص ذات الصلة.",
    "تحتاج أشجار النخيل إلى الري المنتظم والعناية بالتربة.",
    "يمكن استخدام نماذج اللغة لتحسين مطابقة الأسئلة بالمقاطع المناسبة.",
]

query_embedding = model.encode(query, normalize_embeddings=True)
passage_embeddings = model.encode(passages, normalize_embeddings=True)
scores = passage_embeddings @ query_embedding

for index in np.argsort(scores)[::-1]:
    print(f"{scores[index]:.4f} | {passages[index]}")

For larger collections, precompute passage embeddings and store them in a vector index. Split long documents into passages within the effective sequence limit. Similarity scores indicate relative semantic proximity; they are not calibrated probabilities of correctness.

Training

ATM2 was fine-tuned from AraBERTv02 using Arabic triplet supervision from akhooli/arabic-triplets-1m-curated-sims-len.

The training objective combines:

MultipleNegativesRankingLoss: A contrastive ranking objective that brings matched texts closer while separating them from competing examples.
MatryoshkaLoss: Applies the embedding objective at multiple prefix dimensions, encouraging useful representations at different vector sizes.

The original release reports 3 training epochs and a final training loss of 0.718. Training loss is an optimization diagnostic, not a benchmark score.

Training-size documentation: The original card contains conflicting sample counts: a dataset_size:75000 tag and a reference to 1 million samples in the training description. The source dataset identifier is retained here, but the exact number of examples used for this checkpoint is not asserted pending confirmation from the training configuration.

The full 768-dimensional representation is the default. Before using shorter vectors, confirm the dimensions used during training and evaluate the quality–storage trade-off on your target task. Renormalize embeddings after truncation when using cosine similarity.

Evaluation

Semantic textual similarity

The original ATM2 release reports the following Arabic STS results:

Benchmark	Reported score (0–100 scale)
STS17	85
STS22-v2	64
Arithmetic mean of the displayed scores	74.5

These are rounded historical release results, not a claim of current leaderboard leadership or an overall MTEB score. Consult the GATE paper and associated experimental artifacts for evaluation protocols and broader model-family comparisons. Results for other GATE checkpoints should not automatically be attributed to ATM2.

Limitations

Performance on specialized domains may require additional training and task-specific evaluation.
Arabic dialects, spelling variation, and historical language may differ from the training distribution.
Semantically similar passages may contain different facts, conditions, or conclusions. High similarity does not guarantee that a passage answers a question.
Long inputs may be truncated. Use an appropriate passage segmentation strategy.
Matryoshka training does not guarantee equal quality at every embedding dimension.
This card does not establish cross-lingual retrieval performance; evaluate multilingual applications separately.
The checkpoint does not provide factual verification, source validation, or calibrated confidence estimates.

Responsible use

Evaluate retrieval quality, representation gaps, and potential biases on the intended population and domain. For religious, legal, medical, or other consequential applications, retain source attribution and appropriate expert review. Retrieved passages should be assessed for relevance and contextual completeness before being used to support an answer.

Citation

If you use ATM2 or build on the GATE research, please cite the published paper:

@inproceedings{nacar-etal-2026-gate-general,
  title = "{GATE}: General {A}rabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Training",
  author = "Nacar, Omer and Koubaa, Anis and Sibaee, Serry and Alhabashi, Yasser and Ammar, Adel and Boulila, Wadii",
  editor = "Lee, Hung-Yi and Dev, Sunipa and Li, Jessy and Elliott, Desmond",
  booktitle = "Findings of the {A}ssociation for {C}omputational {L}inguistics: {EMNLP} 2026",
  month = oct,
  year = "2026",
  address = "Budapest, Hungary",
  publisher = "Association for Computational Linguistics",
  url = "https://aclanthology.org/2026.findings-emnlp.1952/",
  pages = "41623--41634",
  ISBN = "979-8-89176-462-0"
}

The earlier preprint remains available:

@article{nacar2025gate,
  title = {GATE: General Arabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Training},
  author = {Nacar, Omer and Koubaa, Anis and Sibaee, Serry and Al-Habashi, Yasser and Ammar, Adel and Boulila, Wadii},
  journal = {arXiv preprint arXiv:2505.24581},
  year = {2025}
}

Acknowledgements

We thank the AraBERT team for the pretrained backbone, akhooli for the Arabic triplet dataset, and the Sentence Transformers community for the embedding framework. We also acknowledge researchers who evaluate and extend ATM2 for new Arabic NLP applications.