usrnotfound101/anlp-a2-moe-moe-4e-top1

Decoder-only Transformer trained from scratch to translate Vietnamese and Japanese -> English (ANLP Assignment 2, Part 1). FFN variant: moe_4e_top1 - MoE: 4 experts (hidden d), top-1 routing - same total params as dense.

	value
total parameters	35,408,384
active parameters / token	25,971,200
FFN params (total / active)	12,595,200 / 3,158,016
layers / d_model / heads	6 / 512 / 8
training tokens (non-pad)	36,824,882
test perplexity (all / vi / ja)	7.665 / 6.414 / 9.159
test BLEU (all / vi / ja)	28.91 / 32.43 / 25.31
test chrF (all)	51.03

Prompt format: <vi> source <en> or <ja> source <en>; the model continues with the English translation and </s>. Tokenizer: joint SentencePiece unigram (mt_spm.model, byte fallback, no romanisation).

Files: model.safetensors, config.json, results.json, train_log.json, expert_usage.json, plots (*.png) and the model code (common.py). Load with:

from common import load_model_dir
model = load_model_dir("usrnotfound101/anlp-a2-moe-moe-4e-top1")