sinwang commited on
Commit
ece7d93
·
verified ·
1 Parent(s): 15e0c80

Upload README.md with huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +111 -0
README.md ADDED
@@ -0,0 +1,111 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ ---
2
+ license: cc-by-nc-4.0
3
+ library_name: easywam
4
+ base_model: Wan-AI/Wan2.2-TI2V-5B
5
+ tags:
6
+ - robotics
7
+ - world-action-model
8
+ - imitation-learning
9
+ - wan2.2
10
+ pipeline_tag: robotics
11
+ ---
12
+
13
+ # EasyWAM-MoT-Wan22
14
+
15
+ This repository hosts released EasyWAM-MoT checkpoints built on [Wan2.2-TI2V-5B](https://huggingface.co/Wan-AI/Wan2.2-TI2V-5B). EasyWAM-MoT uses separate Video DiT and Action DiT experts whose tokens interact through mixed self-attention. At inference time, it predicts actions without generating future video. The checkpoints are trained with the [EasyWAM](https://github.com/OpenMOSS/EasyWAM) codebase.
16
+
17
+ The LoRA checkpoint uses rank 128 (alpha 128) on the backbone Video DiT target modules. Other trainable EasyWAM modules are stored in the same checkpoint, so it must be loaded with the matching `_lora` task recipe.
18
+
19
+ > **LoRA requirement:** The LoRA checkpoint does not include the complete Wan2.2 backbone. Before loading or evaluating it, download the required backbone weights and configure their paths as described in the [EasyWAM Wan2.2 backbone guide](https://github.com/OpenMOSS/EasyWAM/blob/main/docs/instructions/backbone/wan22.md).
20
+
21
+ ## LIBERO
22
+
23
+ ### Results
24
+
25
+ Task success rate (%) under the EasyWAM LIBERO evaluation protocol. All results use `state_position: sequence`; higher is better. Models using the same Wan2.2-TI2V-5B backbone are shown together for comparison, and 🔥 marks the architecture released in this repository.
26
+
27
+ **Full-Parameter**
28
+
29
+ | Model | Spatial | Object | Goal | LIBERO-10 | Avg. |
30
+ | --- | :---: | :---: | :---: | :---: | :---: |
31
+ | EasyWAM-Unified | 98.4 | 98.8 | 99.2 | 98.0 | 98.6 |
32
+ | 🔥 EasyWAM-MoT | 97.0 | 99.2 | 96.6 | 94.0 | 96.7 |
33
+ | EasyWAM-MoT-Joint | 98.2 | 98.0 | 97.6 | 96.8 | 97.7 |
34
+ | EasyWAM-MoT-IDM | 99.0 | 99.2 | 98.8 | 97.4 | 98.6 |
35
+ | EasyWAM-Hidden | 99.2 | 100.0 | 97.8 | 98.2 | 98.8 |
36
+
37
+ **LoRA (Rank 128)**
38
+
39
+ | Model | Spatial | Object | Goal | LIBERO-10 | Avg. |
40
+ | --- | :---: | :---: | :---: | :---: | :---: |
41
+ | EasyWAM-Unified | 91.2 | 98.8 | 91.8 | 66.2 | 87.0 |
42
+ | 🔥 EasyWAM-MoT | 96.8 | 99.6 | 97.4 | 90.0 | 95.9 |
43
+ | EasyWAM-Hidden | 98.0 | 99.8 | 89.4 | 86.6 | 93.5 |
44
+
45
+ **LIBERO-Plus**
46
+
47
+ | Model | Background | Camera | Language | Layout | Light | Noise | Robot | Avg. |
48
+ | --- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |
49
+ | EasyWAM-Unified | 72.3 | 54.8 | 93.7 | 83.4 | 97.0 | 72.0 | 83.4 | 79.0 |
50
+ | 🔥 EasyWAM-MoT | 64.5 | 45.5 | 71.4 | 80.1 | 94.7 | 78.5 | 71.5 | 71.7 |
51
+ | EasyWAM-MoT-Joint | 60.9 | 47.3 | 89.7 | 80.5 | 92.4 | 68.9 | 75.9 | 73.3 |
52
+ | EasyWAM-MoT-IDM | 62.2 | 52.0 | 94.1 | 82.0 | 93.2 | 67.8 | 78.0 | 75.3 |
53
+ | EasyWAM-Hidden | 59.3 | 57.0 | 93.2 | 84.3 | 95.0 | 70.8 | 83.7 | 77.6 |
54
+
55
+ LIBERO-Plus results are reported for full-parameter checkpoints only. See the [complete EasyWAM benchmark table](https://github.com/OpenMOSS/EasyWAM/blob/main/docs/results/result.md) for source results and comparisons across backbones.
56
+
57
+ ### Download
58
+
59
+ ```bash
60
+ hf download OpenMOSS-Team/EasyWAM-MoT-Wan22 \
61
+ libero_mot_wan22.pt \
62
+ libero_mot_wan22_lora_128.pt \
63
+ libero_dataset_stats.json \
64
+ --local-dir ./checkpoints
65
+ ```
66
+
67
+ ### Evaluation
68
+
69
+ Install EasyWAM, prepare the Wan2.2-TI2V-5B dependencies using the [backbone guide](https://github.com/OpenMOSS/EasyWAM/blob/main/docs/instructions/backbone/wan22.md), and set up the simulator using the [LIBERO evaluation guide](https://github.com/OpenMOSS/EasyWAM/blob/main/docs/instructions/benchmark/libero.md). Then run:
70
+
71
+ ```bash
72
+ # Full-parameter checkpoint
73
+ python experiments/libero/run_libero_manager.py \
74
+ task=libero_easywam_mot_wan22 \
75
+ ckpt=./checkpoints/libero_mot_wan22.pt \
76
+ EVALUATION.dataset_stats_path=./checkpoints/libero_dataset_stats.json
77
+
78
+ # LoRA checkpoint
79
+ python experiments/libero/run_libero_manager.py \
80
+ task=libero_easywam_mot_wan22_lora \
81
+ ckpt=./checkpoints/libero_mot_wan22_lora_128.pt \
82
+ EVALUATION.dataset_stats_path=./checkpoints/libero_dataset_stats.json
83
+ ```
84
+
85
+ The default protocol evaluates all four LIBERO suites with 50 trials per task. Add `MULTIRUN.num_gpus=<gpu-count>` to distribute evaluation across multiple GPUs.
86
+
87
+ ## Available Checkpoints
88
+
89
+ - LIBERO full-parameter checkpoint: `libero_mot_wan22.pt`
90
+ - LIBERO LoRA checkpoint: `libero_mot_wan22_lora_128.pt` (rank 128, alpha 128)
91
+ - LIBERO normalization statistics: `libero_dataset_stats.json`
92
+ - State-token placement: `sequence`
93
+ - Checkpoint format: EasyWAM PyTorch checkpoint (`.pt`)
94
+
95
+ ## Project
96
+
97
+ - Project page: [http://openmoss.ai/EasyWAM/](http://openmoss.ai/EasyWAM/)
98
+ - Code: [https://github.com/OpenMOSS/EasyWAM](https://github.com/OpenMOSS/EasyWAM)
99
+
100
+ ## License and Citation
101
+
102
+ EasyWAM code is released under the MIT License. Released checkpoints in this repository use CC BY-NC 4.0 and remain subject to the terms of their base models and training data. If EasyWAM is useful in your research, please cite:
103
+
104
+ ```bibtex
105
+ @misc{easywam2026,
106
+ title = {EasyWAM: A Unified and Efficient Framework for Training and Evaluating World Action Models},
107
+ author = {EasyWAM-Team},
108
+ year = {2026},
109
+ url = {https://github.com/OpenMOSS/EasyWAM}
110
+ }
111
+ ```