Instructions to use galilai-group/LeVJEPA-1.5-Large with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use galilai-group/LeVJEPA-1.5-Large with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("video-classification", model="galilai-group/LeVJEPA-1.5-Large", trust_remote_code=True)# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("galilai-group/LeVJEPA-1.5-Large", trust_remote_code=True, device_map="auto") - Notebooks
- Google Colab
- Kaggle
LeVJEPA 1.5 Large
11/05 UPDATE: Due to an error we uploaded weights from a different pretraining run which is now fixed!
LeVJEPA 1.5 Large is a self-supervised video encoder. It reaches a mean score of 78.76% across 22 image and video datasets, while training on ~5,300 unique video hours and 151 exaflops of compute.
For comparison, V-JEPA 1 (79.77%) used 25.7× more data and 5.6× more compute, and V-JEPA 2 (81.63%) used 321× more data and ~28.5× more compute.
LeVJEPA trains a single encoder with an invariance loss over global and local views of a clip, regularized by SIGReg (LeJEPA's Sketched Isotropic Gaussian Regularizer). It uses no EMA target encoder, stop-gradient, predictor or pixel decoder. Code: MLO-lab/LeVJEPA · Paper: arXiv 2608.27395 · Project page: levjepa.github.io
Training budget
| LeVJEPA 1.5 Large | V-JEPA 1 | V-JEPA 2 | |
|---|---|---|---|
| Unique video hours | ~5,300 | ~136,000 | ~1,700,000 |
| Data relative to LeVJEPA | 1× | 25.7× | 321× |
| Training compute (EF) | 151 | 844 | ~4,300* |
| Compute relative to LeVJEPA | 1× | 5.6× | ~28.5× |
| Mean score, 22 datasets | 78.76 | 79.77 | 81.63 |
* V-JEPA 2 training compute is an estimate.
Results
All three encoders are evaluated frozen with attentive probing. Bold marks the highest score in each row.
Image · 15 datasets
| Dataset | LeVJEPA 1.5 Large | V-JEPA 1 | V-JEPA 2 |
|---|---|---|---|
| ImageNet-1K | 75.20 | 79.45 | 82.68 |
| STL-10 | 90.30 | 93.49 | 99.05 |
| Flowers102 | 88.23 | 92.28 | 95.37 |
| Aircraft | 61.75 | 55.30 | 67.78 |
| Oxford Pets | 82.47 | 82.31 | 90.92 |
| DTD | 65.85 | 67.02 | 72.55 |
| EuroSAT | 98.06 | 97.17 | 96.39 |
| DermaMNIST | 88.98 | 83.44 | 83.49 |
| BloodMNIST | 99.09 | 98.48 | 98.19 |
| BreastMNIST | 87.82 | 88.46 | 84.62 |
| RetinaMNIST | 62.25 | 64.00 | 69.25 |
| PneumoniaMNIST | 91.03 | 94.23 | 89.58 |
| OrganCMNIST | 88.22 | 87.03 | 85.15 |
| OrganSMNIST | 77.61 | 74.10 | 70.21 |
| OrganAMNIST | 93.31 | 88.66 | 88.27 |
| Mean | 83.34 | 83.03 | 84.90 |
Video · 7 datasets
| Dataset | LeVJEPA 1.5 Large | V-JEPA 1 | V-JEPA 2 |
|---|---|---|---|
| SSv2 | 60.10 | 69.55 | 70.00 |
| Kinetics-710 | 68.23 | 75.27 | 77.01 |
| Diving48 | 27.43 | 34.54 | 39.46 |
| KTH | 78.24 | 87.04 | 89.81 |
| DynTex++ | 99.03 | 98.33 | 98.33 |
| YUP++ | 75.28 | 79.63 | 82.50 |
| COVID-BLUES | 74.24 | 65.15 | 65.15 |
| Mean | 68.94 | 72.79 | 74.61 |
Overall · 22 datasets
| LeVJEPA 1.5 Large | V-JEPA 1 | V-JEPA 2 | |
|---|---|---|---|
| Mean | 78.76 | 79.77 | 81.63 |
Usage
The model takes ImageNet-normalized clips of 16 frames at 224×224, shaped (B, C, T, H, W), sampled at ~7.5 fps (16 frames ≈ 2 s). It returns the [cls] token followed by the patch tokens of every frame.
import torch
from transformers import AutoModel
model = AutoModel.from_pretrained("galilai-group/LeVJEPA-1.5-Large", trust_remote_code=True).eval()
video = torch.randn(1, 3, 16, 224, 224) # (B, C, T, H, W), ImageNet-normalized
with torch.no_grad():
out = model(pixel_values=video)
out.last_hidden_state # (1, 3137, 1024): [cls] + 16 * 14 * 14 patch tokens, in (frame, row, column) order
out["pooler_output"] # (1, 1024): the [cls] token
trust_remote_code=True is required because the modeling code (RoPE and block-causal attention) ships with the weights. Normalize with the ImageNet statistics: mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225].
- Images: repeat a single image along the time axis,
image.unsqueeze(2).repeat(1, 1, 16, 1, 1). This is the protocol used for the image results above. - Attention mode: leave
config.attn_modeat"block_causal". The weights were trained causal; full attention runs without an error but returns worse features.
Without transformers, the LeVJEPA repository loads the same weights with its own code:
from legacy.encoder import load_legacy_encoder
encoder = load_legacy_encoder("galilai-group/LeVJEPA-1.5-Large", name="vit_large", num_frames=16)
cls, patches, _ = encoder(clip) # clip (B, T, C, H, W) -> (B, 1024), (B, T, 196, 1024)
The repository also contains the V-JEPA 2 attentive-probe evaluation (python -m eval.eval model=levjepa_v1_vitl checkpoint=galilai-group/LeVJEPA-1.5-Large) and a notebook that visualizes the patch features.
Make sure to utilize the newest repository version (v2) and follow the hyperparameter guidelines for best probing results.
Model details
Architecture: ViT-L/16 video encoder: 24 layers, width 1024, 16 heads.
- One token per 16×16 patch per frame (tubelet size 1), with 3-D RoPE positions.
- Block-causal attention: bidirectional within a frame, causal across frames. Each frame's tokens depend only on the current and earlier frames.
- A
[cls]token reads the whole clip.
Parameters: 303M.
Input: 16 frames at 224×224 (14×14 patches per frame), sampled at ~7.5 fps, ImageNet-normalized. RoPE also accepts other resolutions and clip lengths.
Evaluation protocol: frozen encoder with the V-JEPA 2 attentive probe (4-layers) trained on its patch tokens. All three models are evaluated with the same protocol tuned for them.
Intended use: frozen feature extraction (attentive or linear probing, retrieval, or a backbone for downstream heads). The model has no classification head.
License: CC BY-NC 4.0 (non-commercial). The encoder code is adapted from Meta's V-JEPA.
Citation
@misc{kuhn2026levjepaefficientscalable,
title={LeVJEPA: Efficient & Scalable Video Pretraining without the Heuristics},
author={Lukas Kuhn and Lucas Maes and Giuseppe Serra and Quentin Le Lidec and Yann LeCun and Randall Balestriero and Florian Buettner},
year={2026},
eprint={2608.27395},
archivePrefix={arXiv},
primaryClass={cs.CV},
url={https://arxiv.org/abs/2608.27395},
}
- Downloads last month
- 340