[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-72212":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":9,"language":10,"languages":9,"totalLinesOfCode":9,"stars":11,"forks":12,"watchers":13,"openIssues":14,"contributorsCount":15,"subscribersCount":15,"size":15,"stars1d":16,"stars7d":17,"stars30d":18,"stars90d":15,"forks30d":15,"starsTrendScore":19,"compositeScore":20,"rankGlobal":9,"rankLanguage":9,"license":21,"archived":22,"fork":22,"defaultBranch":23,"hasWiki":22,"hasPages":22,"topics":24,"createdAt":9,"pushedAt":9,"updatedAt":25,"readmeContent":26,"aiSummary":27,"trendingCount":15,"starSnapshotCount":15,"syncStatus":28,"lastSyncTime":29,"discoverSource":30},72212,"vjepa2","facebookresearch\u002Fvjepa2","facebookresearch","PyTorch code and models for VJEPA2 self-supervised learning from video.",null,"Python",4127,505,47,66,0,46,91,252,138,30.11,"MIT License",false,"main",[],"2026-06-12 02:03:00","\n🆕 **[2026-03-16]:** :fire: V-JEPA 2.1 is released :fire: A new familly of models trained with a novel recipe that learns high quality and temporolly consistent dense features !!!\n\n**[2025-06-25]:** V-JEPA 2 is released. [[`Blog`](https:\u002F\u002Fai.meta.com\u002Fblog\u002Fv-jepa-2-world-model-benchmarks)]\n\n\n# V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning\n\n### [Meta FAIR](https:\u002F\u002Fai.meta.com\u002Fresearch\u002F)\n\nMahmoud Assran∗, Adrien Bardes∗, David Fan∗, Quentin Garrido∗, Russell Howes∗, Mojtaba\nKomeili∗, Matthew Muckley∗, Ammar Rizvi∗, Claire Roberts∗, Koustuv Sinha∗, Artem Zholus*,\nSergio Arnaud*, Abha Gejji*, Ada Martin*, Francois Robert Hogan*, Daniel Dugas*, Piotr\nBojanowski, Vasil Khalidov, Patrick Labatut, Francisco Massa, Marc Szafraniec, Kapil\nKrishnakumar, Yong Li, Xiaodong Ma, Sarath Chandar, Franziska Meier*, Yann LeCun*, Michael\nRabbat*, Nicolas Ballas*\n\n*Core Team\n\n[[`Paper`](https:\u002F\u002Farxiv.org\u002Fabs\u002F2506.09985)] [[`Blog`](https:\u002F\u002Fai.meta.com\u002Fblog\u002Fv-jepa-2-world-model-benchmarks)] [[`BibTex`](#Citation)]\n\nOfficial Pytorch codebase for V-JEPA 2, V-JEPA 2-AC, V-JEPA 2.1.\n\nV-JEPA 2 is a self-supervised approach to training video encoders, using internet-scale video data, that attains state-of-the-art performance on motion understanding and human action anticipation tasks. V-JEPA 2-AC is a latent action-conditioned world model post-trained from V-JEPA 2 (using a small amount of robot trajectory interaction data) that solves robot manipulation tasks without environment-specific data collection or task-specific training or calibration.\n\n\u003Cp align=\"center\">\n\t\u003Cimg src=\"assets\u002Fflowchart.png\" width=100%>\n\u003C\u002Fp>\n\n\n\n## V-JEPA 2.1 Pre-training\n\nLorenzo Mur-Labadia, Matthew Muckley, Amir Bar, Mahmoud Assran, Koustuv Sinha, Michael\nRabbat, Yann LeCun, Nicolas Ballas, Adrien Bardes\n\n[[`Paper`](https:\u002F\u002Farxiv.org\u002Fabs\u002FTODO)] [[`BibTex`](#Citation)]\n\nV-JEPA 2.1 improves the training recipe to focus on learning high-quality and temporally consistent dense features, as higlighted by PCA visualizations:\n\n\u003Cp align=\"center\">\n\t\u003Cimg src=\"assets\u002Fteaser_screenshot_5dice.png\" width=100%>\n\u003C\u002Fp>\n\nThe V-JEPA 2.1 approach leverages: (1) **Dense Predictive Loss**, a masking-based\nself-supervision objective where all tokens (both visible\u002Fcontext and masked tokens) contribute to the\nself-supervised training loss; (2) **Deep Self-Supervision**, which applies the self-supervised loss at multiple\nintermediate representations of the encoder models; (3) **Multi-Modal Tokenizers** for images and videos;\nand we show that our approach benefit from (4) **Model and data scaling**.\n\n\u003Cp align=\"center\">\n\t\u003Cimg src=\"assets\u002Farchitecture_vjepa2_1.jpg\" width=100%>\n\u003C\u002Fp>\n\nV-JEPA 2.1 performance across dense and global prediction tasks:\n\n\u003Cp align=\"center\">\n\t\u003Cimg src=\"assets\u002Fbars_teaser_tikz-1.png\" width=100%>\n\u003C\u002Fp>\n\n\n## V-JEPA 2 Pre-training\n\n**(Top)** The encoder and predictor are pre-trained through self-supervised learning from video using a masked latent feature prediction objective, leveraging abundant natural videos to bootstrap physical world understanding and prediction. **(Bottom)** Performance of V-JEPA 2 on downstream understanding and prediction tasks.\n\n\u003Cimg align=\"left\" src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002F914942d8-6a1e-409d-86ff-ff856b7346ab\" width=65%> \n\u003Ctable>\n  \u003Ctr>\n    \u003Cth colspan=\"1\">Benchmark\u003C\u002Fth>\n    \u003Cth colspan=\"1\">V-JEPA 2\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Previous Best\u003C\u002Fth>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>EK100\u003C\u002Ftd>\n    \u003Ctd>39.7%\u003C\u002Ftd>\n    \u003Ctd>27.6% (PlausiVL)\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>SSv2 (Probe)\u003C\u002Ftd>\n    \u003Ctd>77.3%\u003C\u002Ftd>\n    \u003Ctd>69.7% (InternVideo2-1B)\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>Diving48 (Probe)\u003C\u002Ftd>\n    \u003Ctd>90.2%\u003C\u002Ftd>\n    \u003Ctd>86.4% (InternVideo2-1B)\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>MVP (Video QA)\u003C\u002Ftd>\n    \u003Ctd>44.5%\u003C\u002Ftd>\n    \u003Ctd>39.9% (InternVL-2.5)\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>TempCompass (Video QA)\u003C\u002Ftd>\n    \u003Ctd>76.9%\u003C\u002Ftd>\n    \u003Ctd>75.3% (Tarsier 2)\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n## V-JEPA 2-AC Post-training\n\n**(Top)** After post-training with a small amount of robot data, we can deploy the model on a robot arm in new environments, and tackle foundational tasks like reaching, grasping, and pick-and-place by planning from image goals. **(Bottom)** Performance on robot manipulation tasks using a Franka arm, with input provided through a monocular RGB camera.\n\n\u003Cimg align=\"left\" src=\"https:\u002F\u002Fgithub.com\u002Fuser-attachments\u002Fassets\u002Fc5d42221-0102-4216-911d-061a4369a805\" width=65%> \n\u003Ctable>\n  \u003Ctr>\n    \u003Cth colspan=\"1\">\u003C\u002Fth>\n    \u003Cth colspan=\"1\">\u003C\u002Fth>\n    \u003Cth colspan=\"2\">Grasp\u003C\u002Fth>\n    \u003Cth colspan=\"2\">Pick-and-Place\u003C\u002Fth>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Cth colspan=\"1\">Method\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Reach\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Cup\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Box\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Cup\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Box\u003C\u002Fth>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>Octo\u003C\u002Ftd>\n    \u003Ctd>100%\u003C\u002Ftd>\n    \u003Ctd>10%\u003C\u002Ftd>\n    \u003Ctd>0%\u003C\u002Ftd>\n    \u003Ctd>10%\u003C\u002Ftd>\n    \u003Ctd>10%\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>Cosmos\u003C\u002Ftd>\n    \u003Ctd>80%\u003C\u002Ftd>\n    \u003Ctd>0%\u003C\u002Ftd>\n    \u003Ctd>20%\u003C\u002Ftd>\n    \u003Ctd>0%\u003C\u002Ftd>\n    \u003Ctd>0%\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>VJEPA 2-AC\u003C\u002Ftd>\n    \u003Ctd>100%\u003C\u002Ftd>\n    \u003Ctd>60%\u003C\u002Ftd>\n    \u003Ctd>20%\u003C\u002Ftd>\n    \u003Ctd>80%\u003C\u002Ftd>\n    \u003Ctd>50%\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n\n\n\n\n## Models\n\n### V-JEPA 2 and V-JEPA 2.1\n\n#### HuggingFace\n\nSee our HuggingFace [collection](https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Ffacebook\u002Fv-jepa-2-6841bad8413014e185b497a6) for V-JEPA 2.\n\n#### V-JEPA 2 Pretrained Checkpoints\n\n\u003Ctable>\n  \u003Ctr>\n    \u003Cth colspan=\"1\">Model\u003C\u002Fth>\n    \u003Cth colspan=\"1\">#Parameters\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Resolution\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Download Link\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Pretraining Config\u003C\u002Fth>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>ViT-L\u002F16\u003C\u002Ftd>\n    \u003Ctd>300M\u003C\u002Ftd>\n    \u003Ctd>256\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvitl.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Ftrain\u002Fvitl16\">configs\u003C\u002Fa>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>ViT-H\u002F16\u003C\u002Ftd>\n    \u003Ctd>600M\u003C\u002Ftd>\n    \u003Ctd>256\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvith.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Ftrain\u002Fvith16\u002F\">configs\u003C\u002Fa>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>ViT-g\u002F16\u003C\u002Ftd>\n    \u003Ctd>1B\u003C\u002Ftd>\n    \u003Ctd>256\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvitg.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Ftrain\u002Fvitg16\">configs\u003C\u002Fa>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>ViT-g\u002F16\u003Csub>384\u003C\u002Fsub>\u003C\u002Ftd>\n    \u003Ctd>1B\u003C\u002Ftd>\n    \u003Ctd>384\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvitg-384.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Ftrain\u002Fvitg16\">configs\u003C\u002Fa>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n#### V-JEPA 2.1 Pretrained Checkpoints\n\n\u003Ctable>\n  \u003Ctr>\n    \u003Cth colspan=\"1\">Model\u003C\u002Fth>\n    \u003Cth colspan=\"1\">#Parameters\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Resolution\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Download Link\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Pretraining Config\u003C\u002Fth>\n  \u003C\u002Ftr>\n\n  \u003Ctr>\n    \u003Ctd>ViT-B\u002F16\u003C\u002Ftd>\n    \u003Ctd>80M\u003C\u002Ftd>\n    \u003Ctd>384\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvjepa2_1_vitb_dist_vitG_384.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Ftrain_2_1\u002Fvitb16\">configs\u003C\u002Fa>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n  \u003Ctr>\n    \u003Ctd>ViT-L\u002F16\u003C\u002Ftd>\n    \u003Ctd>300M\u003C\u002Ftd>\n    \u003Ctd>384\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvjepa2_1_vitl_dist_vitG_384.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Ftrain_2_1\u002Fvitl16\">configs\u003C\u002Fa>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n  \u003Ctr>\n    \u003Ctd>ViT-g\u002F16\u003C\u002Ftd>\n    \u003Ctd>1B\u003C\u002Ftd>\n    \u003Ctd>384\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvjepa2_1_vitg_384.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Ftrain_2_1\u002Fvitg16\">configs\u003C\u002Fa>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\n  \u003Ctr>\n    \u003Ctd>ViT-G\u002F16\u003C\u002Ftd>\n    \u003Ctd>2B\u003C\u002Ftd>\n    \u003Ctd>384\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvjepa2_1_vitG_384.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Ftrain_2_1\u002FvitG16\">configs\u003C\u002Fa>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n\n#### Pretrained backbones (via PyTorch Hub)\n\nPlease install [Pytorch](https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F), [timm](https:\u002F\u002Fpypi.org\u002Fproject\u002Ftimm\u002F) and [einops](https:\u002F\u002Fpypi.org\u002Fproject\u002Feinops\u002F) locally, then run the following to load each model. Installing Pytorch with CUDA support is strongly recommended.\n\n```python\nimport torch\n\n# preprocessor\nprocessor = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_preprocessor')\n# models\n# V-JEPA 2\nvjepa2_vit_large = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_vit_large')\nvjepa2_vit_huge = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_vit_huge')\nvjepa2_vit_giant = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_vit_giant')\nvjepa2_vit_giant_384 = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_vit_giant_384')\n# V-JEPA 2.1\nvjepa2_1_vit_base_384 = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_1_vit_base_384')\nvjepa2_1_vit_large_384 = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_1_vit_large_384')\nvjepa2_1_vit_giant_384 = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_1_vit_giant_384')\nvjepa2_1_vit_gigantic_384 = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_1_vit_gigantic_384')\n\n```\n\n#### Pretrained checkpoints on Huggingface\n\nYou can also use our pretrained checkpoints on [Huggingface for V-JEPA 2](https:\u002F\u002Fhuggingface.co\u002Fcollections\u002Ffacebook\u002Fv-jepa-2-6841bad8413014e185b497a6).\n\n```python\nfrom transformers import AutoVideoProcessor, AutoModel\n\nhf_repo = \"facebook\u002Fvjepa2-vitg-fpc64-256\"\n# facebook\u002Fvjepa2-vitl-fpc64-256\n# facebook\u002Fvjepa2-vith-fpc64-256\n# facebook\u002Fvjepa2-vitg-fpc64-256\n# facebook\u002Fvjepa2-vitg-fpc64-384\n\nmodel = AutoModel.from_pretrained(hf_repo)\nprocessor = AutoVideoProcessor.from_pretrained(hf_repo)\n```\n\n#### Evaluation Attentive Probes\n\nWe share the trained attentive probes for two of our visual understanding evals (Something-Something v2 and Diving48) and the action anticipation eval EPIC-KITCHENS-100.\n\n\u003Ctable>\n  \u003Ctr>\n    \u003Cth colspan=\"1\">Model\u003C\u002Fth>\n    \u003Cth colspan=\"4\">SSv2\u003C\u002Fth>\n    \u003Cth colspan=\"4\">Diving48\u003C\u002Fth>\n    \u003Cth colspan=\"4\">EK100\u003C\u002Fth>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Cth colspan=\"1\">\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Checkpoint\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Training Config\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Inference Config\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Result\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Checkpoint\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Training Config\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Inference Config\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Result\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Checkpoint\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Training Config\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Inference Config\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Result\u003C\u002Fth>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>ViT-L\u002F16\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fevals\u002Fssv2-vitl-16x2x3.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Feval\u002Fvitl\u002Fssv2.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Finference\u002Fvitl\u002Fssv2.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>73.7%\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fevals\u002Fdiving48-vitl-256.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Feval\u002Fvitl\u002Fdiving48.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Finference\u002Fvitl\u002Fdiving48.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>89.0%\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fevals\u002Fek100-vitl-256.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Feval\u002Fvitl\u002Fek100.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Finference\u002Fvitl\u002Fek100.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>32.7 R@5\u003C\u002Ftd>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>ViT-g\u002F16\u003Csub>384\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fevals\u002Fssv2-vitg-384-64x2x3.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Feval\u002Fvitg-384\u002Fssv2.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Finference\u002Fvitg-384\u002Fssv2.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>77.3%\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fevals\u002Fdiving48-vitg-384-32x4x3.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Feval\u002Fvitg-384\u002Fdiving48.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Finference\u002Fvitg-384\u002Fdiving48.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>90.2%\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fevals\u002Fek100-vitg-384.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Feval\u002Fvitg-384\u002Fek100.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Finference\u002Fvitg-384\u002Fek100.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>39.7 R@5\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n### V-JEPA 2-AC\n\nOur action-conditioned checkpoint was trained from the ViT-g encoder.\n\u003Ctable>\n  \u003Ctr>\n    \u003Cth colspan=\"1\">Model\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Download Link\u003C\u002Fth>\n    \u003Cth colspan=\"1\">Training Config\u003C\u002Fth>\n  \u003C\u002Ftr>\n  \u003Ctr>\n    \u003Ctd>ViT-g\u002F16\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvjepa2-ac-vitg.pt\">checkpoint\u003C\u002Fa>\u003C\u002Ftd>\n    \u003Ctd>\u003Ca href=\"configs\u002Ftrain\u002Fvitg16\u002Fdroid-256px-8f.yaml\">config\u003C\u002Fa>\u003C\u002Ftd>\n  \u003C\u002Ftr>\n\u003C\u002Ftable>\n\n#### Pretrained action-conditioned backbone (via PyTorch Hub)\n\nPlease install [Pytorch](https:\u002F\u002Fpytorch.org\u002Fget-started\u002Flocally\u002F), [timm](https:\u002F\u002Fpypi.org\u002Fproject\u002Ftimm\u002F) and [einops](https:\u002F\u002Fpypi.org\u002Fproject\u002Feinops\u002F) locally, then run the following to load each model. Installing Pytorch with CUDA support is strongly recommended.\n\n```python\nimport torch\n\nvjepa2_encoder, vjepa2_ac_predictor = torch.hub.load('facebookresearch\u002Fvjepa2', 'vjepa2_ac_vit_giant')\n```\n\n\nSee [energy_landscape_example.ipynb](notebooks\u002Fenergy_landscape_example.ipynb) for an example notebook computing the energy landscape of the pretrained action-conditioned backbone using a robot trajectory collected from our lab.\nTo run this notebook, you'll need to additionally install [Jupyter](https:\u002F\u002Fjupyter.org\u002Finstall) and [Scipy](https:\u002F\u002Fscipy.org\u002Finstall\u002F) in your conda environment.\n\n\n\n## Getting Started\n\n### Setup\n\n```\nconda create -n vjepa2-312 python=3.12\nconda activate vjepa2-312\npip install .  # or `pip install -e .` for development mode\n```\n\n**Note to macOS users:** V-JEPA 2 relies on [`decord`](https:\u002F\u002Fgithub.com\u002Fdmlc\u002Fdecord), which does not support macOS (and, unfortunately, is also no longer under development). In order to run the V-JEPA 2 code on macOS, you will need a different `decord` implementation. We do not make specific recommendations, although some users have reported the use of [`eva-decord`](https:\u002F\u002Fgithub.com\u002Fgeorgia-tech-db\u002Feva-decord) (see [PR 1](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fvjepa2\u002Fpull\u002F1)) or [`decord2`](https:\u002F\u002Fgithub.com\u002Fjohnnynunez\u002Fdecord2) (see [PR 31](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fvjepa2\u002Fpull\u002F31)).  We leave the selection of the `decord` package up to the user's discretion.\n\n### Usage Demo\n\nSee [vjepa2_demo.ipynb](notebooks\u002Fvjepa2_demo.ipynb) [(Colab Link)](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Ffacebookresearch\u002Fvjepa2\u002Fblob\u002Fmain\u002Fnotebooks\u002Fvjepa2_demo.ipynb) or [vjepa2_demo.py](notebooks\u002Fvjepa2_demo.py) for an example of how to load both the HuggingFace and PyTorch V-JEPA 2 models and run inference on a sample video to get a sample classification result.\n\nThe script assumes the presence of downloaded model checkpoints so you will need to download the model weights and update the corresponding paths in the script. E.g.:\n```\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fvitg-384.pt -P YOUR_DIR\nwget https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fvjepa2\u002Fevals\u002Fssv2-vitg-384-64x2x3.pt -P YOUR_DIR\n\n# Then update your model paths in vjepa2_demo.py.\npt_model_path = YOUR_DIR\u002Fvitg-384.pt\nclassifier_model_path = YOUR_DIR\u002Fssv2-vitg-384-64x2x3.pt\n\n# Then run the script (assumes your machine has a GPU)\npython -m notebooks.vjepa2_demo\n```\n\n### Probe-based evaluation\n\nProbe-based evaluation consists in training an attentive probe on top of frozen V-JEPA 2 features. We provide training scripts for training your own probes, and checkpoints to run inference directly.\n\n#### Training probes\n\nEvaluations can be run either locally, or distributed via SLURM. (Running locally is useful for debugging and validation).\nThese sample commands launch Something-Something v2 video classification; other evals are launched by specifying the corresponding config.\nUse provided training configs under \"Evaluation Attentive Probes\". These configs allow to train multiple probes in parallel with various optimization parameters.\nChange filepaths as needed (e.g. `folder`, `checkpoint`, `dataset_train`, `dataset_val`) to match locations of data and downloaded checkpoints on your local filesystem.\nChange \\# nodes and local batch size as needed to not exceed available GPU memory.\n\n##### Local\n\nTo run locally, specify the GPUs to use on\n```\npython -m evals.main --fname configs\u002Feval\u002Fvitl16\u002Fssv2.yaml \\\n  --devices cuda:0 cuda:1\n```\n\n##### Distributed\n\n```\npython -m evals.main_distributed \\\n  --fname configs\u002Feval\u002Fvitl\u002Fssv2.yaml  \\\n  --time 8600 \\\n  --account my_account --qos=my_qos\n```\n\n#### Inference from existing probes\n\nUse provided inference configs under [Evaluation Attentive Probes](#evaluation-attentive-probes).\nDownload the corresponding checkpoint, rename it to 'latest.pt', and create a folder with the checkpoint inside, with the format matching the variables in the config:\n```\n[folder]\u002F[eval_name]\u002F[tag]\u002Flatest.pt\n```\nThen run inference, locally or distributed, using the same evaluation commands as above, but with configs from `configs\u002Finference`.\n\n### Pretraining\n\nLikewise, training can also be run locally or distributed. Pretraining and cooldown training phases are\nrun with the same command using different configs.\nThese sample commands launch initial training of a ViT-L model. Configs for cooldown (or action-conditioned) training\ncan be found in the same directory as the config for initial training.\n\n#### Local\n\n```\npython -m app.main --fname configs\u002Ftrain\u002Fvitl16\u002Fpretrain-256px-16f.yaml \\\n  --devices cuda:0\n```\n\n#### Distributed\n\n```\npython -m app.main_distributed \\\n  --fname configs\u002Ftrain\u002Fvitl16\u002Fpretrain-256px-16f.yaml\n  --time 6000\n  --account my_account --qos=my_qos\n```\n\n### Postraining\n\nPost-training of the action-conditioned model, starting from the pretrained VJEPA 2 backbone, also follows a similar interface, and can be run locally or distributed using [this config](configs\u002Ftrain\u002Fvitg16\u002Fdroid-256px-8f.yaml).\nWe post-train the model starting from the ViT-g\u002F16 backbone.\n\n#### Local\n\n```\npython -m app.main --fname configs\u002Ftrain\u002Fvitg16\u002Fdroid-256px-8f.yaml \\\n  --devices cuda:0\n```\n\n#### Distributed\n\n```\npython -m app.main_distributed \\\n  --fname configs\u002Ftrain\u002Fvitg16\u002Fdroid-256px-8f.yaml\n  --time 6000\n  --account my_account --qos=my_qos\n```\n\n\n## Code Structure\n\n```\n.\n├── app                              # training loops\n│   ├── vjepa                        #   V-JEPA 2 pre-training\n│   ├── vjepa_2_1                    #   V-JEPA 2.1 pre-training\n│   ├── vjepa_droid                  #   training the action-conditioned model\n│   ├── main_distributed.py          #   entrypoint for launch app on slurm cluster\n│   └── main.py                      #   entrypoint for launch app locally on your machine\n├── configs                          # config files with experiment params for training and evaluation\n│   ├── train                        #   pretraining with V-JEPA 2 (phase 1), cooldown (phase 2), and action-conditioned training\n│   ├── train_2_1                    #   pretraining with V-JEPA 2.1 (phase 1), cooldown (phase 2)\n│   └── eval                         #   frozen evaluations\n│   └── inference                    #   inference only frozen evaluations\n├── evals                            # evaluation loops training an attentive probe with frozen backbone...\n│   ├── action_anticipation_frozen   #   action anticipation\n│   ├── image_classification_frozen  #   image understanding\n│   ├── video_classification_frozen  #   video understanding\n│   ├── main_distributed.py          #   entrypoint for distributed evaluations\n│   └── main.py                      #   entrypoint for locally-run evaluations\n├── src                              # the package\n│   ├── datasets                     #   datasets, data loaders, ...\n│   ├── models                       #   model definitions\n│   ├── masks                        #   mask collators, masking utilities, ...\n│   └── utils                        #   shared utilities\n├── tests                            # unit tests for some modules in `src`\n\n```\n\n## License\n\nThe majority of V-JEPA 2 is licensed under MIT, however portions of the project are available under separate license terms:\n\n[src\u002Fdatasets\u002Futils\u002Fvideo\u002Frandaugment.py](src\u002Fdatasets\u002Futils\u002Fvideo\u002Frandaugment.py)\u003Cbr>\n[src\u002Fdatasets\u002Futils\u002Fvideo\u002Franderase.py](src\u002Fdatasets\u002Futils\u002Fvideo\u002Franderase.py)\u003Cbr>\n[src\u002Fdatasets\u002Futils\u002Fworker_init_fn.py](src\u002Fdatasets\u002Futils\u002Fworker_init_fn.py)\u003Cbr>\n\nare licensed under the Apache 2.0 license.\n\n\n## Citation\nIf you find this repository useful in your research, please consider giving a star :star: and cite the papers:\n\n```bibtex\n@article{assran2025vjepa2,\n  title={V-JEPA~2: Self-Supervised Video Models Enable Understanding, Prediction and Planning},\n  author={Assran, Mahmoud and Bardes, Adrien and Fan, David and Garrido, Quentin and Howes, Russell and\nKomeili, Mojtaba and Muckley, Matthew and Rizvi, Ammar and Roberts, Claire and Sinha, Koustuv and Zholus, Artem and\nArnaud, Sergio and Gejji, Abha and Martin, Ada and Robert Hogan, Francois and Dugas, Daniel and\nBojanowski, Piotr and Khalidov, Vasil and Labatut, Patrick and Massa, Francisco and Szafraniec, Marc and\nKrishnakumar, Kapil and Li, Yong and Ma, Xiaodong and Chandar, Sarath and Meier, Franziska and LeCun, Yann and\nRabbat, Michael and Ballas, Nicolas},\n  journal={arXiv preprint arXiv:2506.09985},\n  year={2025}\n}\n```\n\n```bibtex\n@article{murlabadia2026vjepa2_1,\n  title={V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning},\n  author={Mur-Labadia, Lorenzo and Muckley, Matthew and Bar, Amir and Assran, Mahmoud and\nSinha, Koustuv and Rabbat, Michael and LeCun, Yann and Ballas, Nicolas and Bardes, Adrien},\n  journal={arXiv preprint arXiv:2603.14482},\n  year={2026}\n}\n```\n","V-JEPA 2 是一个基于自监督学习的视频编码器训练方法，使用大规模互联网视频数据进行训练，在运动理解和人类行为预测任务中达到顶尖性能。其核心功能包括密集预测损失、深层自监督和多模态分词器等技术特点，能够学习高质量且时间一致的密集特征。V-JEPA 2 及其变体 V-JEPA 2-AC 和 V-JEPA 2.1 适用于需要对视频内容有深入理解的应用场景，如机器人操作任务、视频内容分析及预测等领域。此外，通过少量机器人轨迹交互数据的后训练，V-JEPA 2-AC 能够解决机器人操纵任务，无需特定环境的数据收集或任务特定训练。项目提供了官方 PyTorch 代码库和预训练模型，便于研究人员和开发者快速上手。",2,"2026-06-11 03:40:51","high_star"]