Zhuchenyang Liu
AI & ML interests
Recent Activity
Organizations
Thanks, and one clarification first: the 84-87% was the range across our two document tower variants, not across datasets. HiRes averages 86.9% of the teacher at 37 pages/s, Fast 84.4% at 99. I have edited the post to say that directly, since it was easy to read the other way.
On domain slices, you are asking for the right thing, and the spread is much wider than the headline. I have just added the full per-dataset breakdown to the write-up, section named Where the Student Holds Up, and Where It Does Not.
On tower compatibility, the rule is that the teacher and the output width must both match, which is why both now appear in the model names. The three 4096-width towers pair with each other e.g.(https://huggingface.co/nanovdr/NanoVDR-D-Fast-Qwen3VL8B-4096 for Doc tower with https://huggingface.co/nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL8B-4096-ML for Query tower); the 2048-d query towers from our earlier release do not pair with new document towers. Since both towers land in the same frozen teacher space, all four student and teacher combinations is usable, like following (number is avg. NDCG@5 for vidore v1, v2, v3):
| teacher docs | student docs | |
|---|---|---|
| teacher queries | 71.05 | 65.02 |
| student queries | 66.36 | 61.74 |
Two document tower variants, each paired with the same 70M text-only query tower and averaged over all 22 ViDoRe datasets: HiRes keeps 86.9% of the teacher's NDCG@5 and indexes at 37 pages/s, Fast keeps 84.4% at 99 pages/s, against the teacher's 5.4. Both store 16.4 GB per million pages instead of the 256 GB their multi-vector peers need.
We released the training nanovdr/distilling-the-document-tower and models nanovdr/NanoVDR-D-Fast-Qwen3VL8B-4096, nanovdr/NanoVDR-D-HiRes-Qwen3VL8B-4096, nanovdr/NanoVDR-Q-DistilBERT-Qwen3VL8B-4096-ML on our NanoVDR space.
Feel free to explore!