Upload README.md with huggingface_hub
Browse files
README.md
ADDED
|
@@ -0,0 +1,111 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
---
|
| 2 |
+
license: cc-by-nc-4.0
|
| 3 |
+
library_name: easywam
|
| 4 |
+
base_model: Wan-AI/Wan2.2-TI2V-5B
|
| 5 |
+
tags:
|
| 6 |
+
- robotics
|
| 7 |
+
- world-action-model
|
| 8 |
+
- imitation-learning
|
| 9 |
+
- wan2.2
|
| 10 |
+
pipeline_tag: robotics
|
| 11 |
+
---
|
| 12 |
+
|
| 13 |
+
# EasyWAM-MoT-Wan22
|
| 14 |
+
|
| 15 |
+
This repository hosts released EasyWAM-MoT checkpoints built on [Wan2.2-TI2V-5B](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B). EasyWAM-MoT uses separate Video DiT and Action DiT experts whose tokens interact through mixed self-attention. At inference time, it predicts actions without generating future video. The checkpoints are trained with the [EasyWAM](https://github.com/OpenMOSS/EasyWAM) codebase.
|
| 16 |
+
|
| 17 |
+
The LoRA checkpoint uses rank 128 (alpha 128) on the backbone Video DiT target modules. Other trainable EasyWAM modules are stored in the same checkpoint, so it must be loaded with the matching `_lora` task recipe.
|
| 18 |
+
|
| 19 |
+
> **LoRA requirement:** The LoRA checkpoint does not include the complete Wan2.2 backbone. Before loading or evaluating it, download the required backbone weights and configure their paths as described in the [EasyWAM Wan2.2 backbone guide](https://github.com/OpenMOSS/EasyWAM/blob/main/docs/instructions/backbone/wan22.md).
|
| 20 |
+
|
| 21 |
+
## LIBERO
|
| 22 |
+
|
| 23 |
+
### Results
|
| 24 |
+
|
| 25 |
+
Task success rate (%) under the EasyWAM LIBERO evaluation protocol. All results use `state_position: sequence`; higher is better. Models using the same Wan2.2-TI2V-5B backbone are shown together for comparison, and 🔥 marks the architecture released in this repository.
|
| 26 |
+
|
| 27 |
+
**Full-Parameter**
|
| 28 |
+
|
| 29 |
+
| Model | Spatial | Object | Goal | LIBERO-10 | Avg. |
|
| 30 |
+
| --- | :---: | :---: | :---: | :---: | :---: |
|
| 31 |
+
| EasyWAM-Unified | 98.4 | 98.8 | 99.2 | 98.0 | 98.6 |
|
| 32 |
+
| 🔥 EasyWAM-MoT | 97.0 | 99.2 | 96.6 | 94.0 | 96.7 |
|
| 33 |
+
| EasyWAM-MoT-Joint | 98.2 | 98.0 | 97.6 | 96.8 | 97.7 |
|
| 34 |
+
| EasyWAM-MoT-IDM | 99.0 | 99.2 | 98.8 | 97.4 | 98.6 |
|
| 35 |
+
| EasyWAM-Hidden | 99.2 | 100.0 | 97.8 | 98.2 | 98.8 |
|
| 36 |
+
|
| 37 |
+
**LoRA (Rank 128)**
|
| 38 |
+
|
| 39 |
+
| Model | Spatial | Object | Goal | LIBERO-10 | Avg. |
|
| 40 |
+
| --- | :---: | :---: | :---: | :---: | :---: |
|
| 41 |
+
| EasyWAM-Unified | 91.2 | 98.8 | 91.8 | 66.2 | 87.0 |
|
| 42 |
+
| 🔥 EasyWAM-MoT | 96.8 | 99.6 | 97.4 | 90.0 | 95.9 |
|
| 43 |
+
| EasyWAM-Hidden | 98.0 | 99.8 | 89.4 | 86.6 | 93.5 |
|
| 44 |
+
|
| 45 |
+
**LIBERO-Plus**
|
| 46 |
+
|
| 47 |
+
| Model | Background | Camera | Language | Layout | Light | Noise | Robot | Avg. |
|
| 48 |
+
| --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
|
| 49 |
+
| EasyWAM-Unified | 72.3 | 54.8 | 93.7 | 83.4 | 97.0 | 72.0 | 83.4 | 79.0 |
|
| 50 |
+
| 🔥 EasyWAM-MoT | 64.5 | 45.5 | 71.4 | 80.1 | 94.7 | 78.5 | 71.5 | 71.7 |
|
| 51 |
+
| EasyWAM-MoT-Joint | 60.9 | 47.3 | 89.7 | 80.5 | 92.4 | 68.9 | 75.9 | 73.3 |
|
| 52 |
+
| EasyWAM-MoT-IDM | 62.2 | 52.0 | 94.1 | 82.0 | 93.2 | 67.8 | 78.0 | 75.3 |
|
| 53 |
+
| EasyWAM-Hidden | 59.3 | 57.0 | 93.2 | 84.3 | 95.0 | 70.8 | 83.7 | 77.6 |
|
| 54 |
+
|
| 55 |
+
LIBERO-Plus results are reported for full-parameter checkpoints only. See the [complete EasyWAM benchmark table](https://github.com/OpenMOSS/EasyWAM/blob/main/docs/results/result.md) for source results and comparisons across backbones.
|
| 56 |
+
|
| 57 |
+
### Download
|
| 58 |
+
|
| 59 |
+
```bash
|
| 60 |
+
hf download OpenMOSS-Team/EasyWAM-MoT-Wan22 \
|
| 61 |
+
libero_mot_wan22.pt \
|
| 62 |
+
libero_mot_wan22_lora_128.pt \
|
| 63 |
+
libero_dataset_stats.json \
|
| 64 |
+
--local-dir ./checkpoints
|
| 65 |
+
```
|
| 66 |
+
|
| 67 |
+
### Evaluation
|
| 68 |
+
|
| 69 |
+
Install EasyWAM, prepare the Wan2.2-TI2V-5B dependencies using the [backbone guide](https://github.com/OpenMOSS/EasyWAM/blob/main/docs/instructions/backbone/wan22.md), and set up the simulator using the [LIBERO evaluation guide](https://github.com/OpenMOSS/EasyWAM/blob/main/docs/instructions/benchmark/libero.md). Then run:
|
| 70 |
+
|
| 71 |
+
```bash
|
| 72 |
+
# Full-parameter checkpoint
|
| 73 |
+
python experiments/libero/run_libero_manager.py \
|
| 74 |
+
task=libero_easywam_mot_wan22 \
|
| 75 |
+
ckpt=./checkpoints/libero_mot_wan22.pt \
|
| 76 |
+
EVALUATION.dataset_stats_path=./checkpoints/libero_dataset_stats.json
|
| 77 |
+
|
| 78 |
+
# LoRA checkpoint
|
| 79 |
+
python experiments/libero/run_libero_manager.py \
|
| 80 |
+
task=libero_easywam_mot_wan22_lora \
|
| 81 |
+
ckpt=./checkpoints/libero_mot_wan22_lora_128.pt \
|
| 82 |
+
EVALUATION.dataset_stats_path=./checkpoints/libero_dataset_stats.json
|
| 83 |
+
```
|
| 84 |
+
|
| 85 |
+
The default protocol evaluates all four LIBERO suites with 50 trials per task. Add `MULTIRUN.num_gpus=<gpu-count>` to distribute evaluation across multiple GPUs.
|
| 86 |
+
|
| 87 |
+
## Available Checkpoints
|
| 88 |
+
|
| 89 |
+
- LIBERO full-parameter checkpoint: `libero_mot_wan22.pt`
|
| 90 |
+
- LIBERO LoRA checkpoint: `libero_mot_wan22_lora_128.pt` (rank 128, alpha 128)
|
| 91 |
+
- LIBERO normalization statistics: `libero_dataset_stats.json`
|
| 92 |
+
- State-token placement: `sequence`
|
| 93 |
+
- Checkpoint format: EasyWAM PyTorch checkpoint (`.pt`)
|
| 94 |
+
|
| 95 |
+
## Project
|
| 96 |
+
|
| 97 |
+
- Project page: [http://openmoss.ai/EasyWAM/](http://openmoss.ai/EasyWAM/)
|
| 98 |
+
- Code: [https://github.com/OpenMOSS/EasyWAM](https://github.com/OpenMOSS/EasyWAM)
|
| 99 |
+
|
| 100 |
+
## License and Citation
|
| 101 |
+
|
| 102 |
+
EasyWAM code is released under the MIT License. Released checkpoints in this repository use CC BY-NC 4.0 and remain subject to the terms of their base models and training data. If EasyWAM is useful in your research, please cite:
|
| 103 |
+
|
| 104 |
+
```bibtex
|
| 105 |
+
@misc{easywam2026,
|
| 106 |
+
title = {EasyWAM: A Unified and Efficient Framework for Training and Evaluating World Action Models},
|
| 107 |
+
author = {EasyWAM-Team},
|
| 108 |
+
year = {2026},
|
| 109 |
+
url = {https://github.com/OpenMOSS/EasyWAM}
|
| 110 |
+
}
|
| 111 |
+
```
|