[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"project-71035":3},{"id":4,"name":5,"fullName":6,"owner":7,"repo":5,"description":8,"homepage":9,"htmlUrl":10,"language":11,"languages":10,"totalLinesOfCode":10,"stars":12,"forks":13,"watchers":14,"openIssues":15,"contributorsCount":16,"subscribersCount":16,"size":16,"stars1d":17,"stars7d":18,"stars30d":19,"stars90d":16,"forks30d":16,"starsTrendScore":20,"compositeScore":21,"rankGlobal":10,"rankLanguage":10,"license":22,"archived":23,"fork":24,"defaultBranch":25,"hasWiki":24,"hasPages":24,"topics":26,"createdAt":10,"pushedAt":10,"updatedAt":27,"readmeContent":28,"aiSummary":29,"trendingCount":16,"starSnapshotCount":16,"syncStatus":30,"lastSyncTime":31,"discoverSource":32},71035,"mae","facebookresearch\u002Fmae","facebookresearch","PyTorch implementation of MAE https\u002F\u002Farxiv.org\u002Fabs\u002F2111.06377","",null,"Python",8334,1347,8311,124,0,4,6,23,12,78.69,"Other",true,false,"main",[],"2026-06-12 04:00:58","## Masked Autoencoders: A PyTorch Implementation\n\n\u003Cp align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fuser-images.githubusercontent.com\u002F11435359\u002F146857310-f258c86c-fde6-48e8-9cee-badd2b21bd2c.png\" width=\"480\">\n\u003C\u002Fp>\n\n\nThis is a PyTorch\u002FGPU re-implementation of the paper [Masked Autoencoders Are Scalable Vision Learners](https:\u002F\u002Farxiv.org\u002Fabs\u002F2111.06377):\n```\n@Article{MaskedAutoencoders2021,\n  author  = {Kaiming He and Xinlei Chen and Saining Xie and Yanghao Li and Piotr Doll{\\'a}r and Ross Girshick},\n  journal = {arXiv:2111.06377},\n  title   = {Masked Autoencoders Are Scalable Vision Learners},\n  year    = {2021},\n}\n```\n\n* The original implementation was in TensorFlow+TPU. This re-implementation is in PyTorch+GPU.\n\n* This repo is a modification on the [DeiT repo](https:\u002F\u002Fgithub.com\u002Ffacebookresearch\u002Fdeit). Installation and preparation follow that repo.\n\n* This repo is based on [`timm==0.3.2`](https:\u002F\u002Fgithub.com\u002Frwightman\u002Fpytorch-image-models), for which a [fix](https:\u002F\u002Fgithub.com\u002Frwightman\u002Fpytorch-image-models\u002Fissues\u002F420#issuecomment-776459842) is needed to work with PyTorch 1.8.1+.\n\n### Catalog\n\n- [x] Visualization demo\n- [x] Pre-trained checkpoints + fine-tuning code\n- [x] Pre-training code\n\n### Visualization demo\n\nRun our interactive visualization demo using [Colab notebook](https:\u002F\u002Fcolab.research.google.com\u002Fgithub\u002Ffacebookresearch\u002Fmae\u002Fblob\u002Fmain\u002Fdemo\u002Fmae_visualize.ipynb) (no GPU needed):\n\u003Cp align=\"center\">\n  \u003Cimg src=\"https:\u002F\u002Fuser-images.githubusercontent.com\u002F11435359\u002F147859292-77341c70-2ed8-4703-b153-f505dcb6f2f8.png\" width=\"600\">\n\u003C\u002Fp>\n\n### Fine-tuning with pre-trained checkpoints\n\nThe following table provides the pre-trained checkpoints used in the paper, converted from TF\u002FTPU to PT\u002FGPU:\n\u003Ctable>\u003Ctbody>\n\u003C!-- START TABLE -->\n\u003C!-- TABLE HEADER -->\n\u003Cth valign=\"bottom\">\u003C\u002Fth>\n\u003Cth valign=\"bottom\">ViT-Base\u003C\u002Fth>\n\u003Cth valign=\"bottom\">ViT-Large\u003C\u002Fth>\n\u003Cth valign=\"bottom\">ViT-Huge\u003C\u002Fth>\n\u003C!-- TABLE BODY -->\n\u003Ctr>\u003Ctd align=\"left\">pre-trained checkpoint\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmae\u002Fpretrain\u002Fmae_pretrain_vit_base.pth\">download\u003C\u002Fa>\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmae\u002Fpretrain\u002Fmae_pretrain_vit_large.pth\">download\u003C\u002Fa>\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Ca href=\"https:\u002F\u002Fdl.fbaipublicfiles.com\u002Fmae\u002Fpretrain\u002Fmae_pretrain_vit_huge.pth\">download\u003C\u002Fa>\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">md5\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Ctt>8cad7c\u003C\u002Ftt>\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Ctt>b8b06e\u003C\u002Ftt>\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Ctt>9bdbb0\u003C\u002Ftt>\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\u003C\u002Ftable>\n\nThe fine-tuning instruction is in [FINETUNE.md](FINETUNE.md).\n\nBy fine-tuning these pre-trained models, we rank #1 in these classification tasks (detailed in the paper):\n\u003Ctable>\u003Ctbody>\n\u003C!-- START TABLE -->\n\u003C!-- TABLE HEADER -->\n\u003Cth valign=\"bottom\">\u003C\u002Fth>\n\u003Cth valign=\"bottom\">ViT-B\u003C\u002Fth>\n\u003Cth valign=\"bottom\">ViT-L\u003C\u002Fth>\n\u003Cth valign=\"bottom\">ViT-H\u003C\u002Fth>\n\u003Cth valign=\"bottom\">ViT-H\u003Csub>448\u003C\u002Fsub>\u003C\u002Fth>\n\u003Ctd valign=\"bottom\" style=\"color:#C0C0C0\">prev best\u003C\u002Ftd>\n\u003C!-- TABLE BODY -->\n\u003Ctr>\u003Ctd align=\"left\">ImageNet-1K (no external data)\u003C\u002Ftd>\n\u003Ctd align=\"center\">83.6\u003C\u002Ftd>\n\u003Ctd align=\"center\">85.9\u003C\u002Ftd>\n\u003Ctd align=\"center\">86.9\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>87.8\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">87.1\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctd colspan=\"5\">\u003Cfont size=\"1\">\u003Cem>following are evaluation of the same model weights (fine-tuned in original ImageNet-1K):\u003C\u002Fem>\u003C\u002Ffont>\u003C\u002Ftd>\n\u003Ctr>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">ImageNet-Corruption (error rate) \u003C\u002Ftd>\n\u003Ctd align=\"center\">51.7\u003C\u002Ftd>\n\u003Ctd align=\"center\">41.8\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>33.8\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\">36.8\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">42.5\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">ImageNet-Adversarial\u003C\u002Ftd>\n\u003Ctd align=\"center\">35.9\u003C\u002Ftd>\n\u003Ctd align=\"center\">57.1\u003C\u002Ftd>\n\u003Ctd align=\"center\">68.2\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>76.7\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">35.8\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">ImageNet-Rendition\u003C\u002Ftd>\n\u003Ctd align=\"center\">48.3\u003C\u002Ftd>\n\u003Ctd align=\"center\">59.9\u003C\u002Ftd>\n\u003Ctd align=\"center\">64.4\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>66.5\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">48.7\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">ImageNet-Sketch\u003C\u002Ftd>\n\u003Ctd align=\"center\">34.5\u003C\u002Ftd>\n\u003Ctd align=\"center\">45.3\u003C\u002Ftd>\n\u003Ctd align=\"center\">49.6\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>50.9\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">36.0\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctd colspan=\"5\">\u003Cfont size=\"1\">\u003Cem>following are transfer learning by fine-tuning the pre-trained MAE on the target dataset:\u003C\u002Fem>\u003C\u002Ffont>\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">iNaturalists 2017\u003C\u002Ftd>\n\u003Ctd align=\"center\">70.5\u003C\u002Ftd>\n\u003Ctd align=\"center\">75.7\u003C\u002Ftd>\n\u003Ctd align=\"center\">79.3\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>83.4\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">75.4\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">iNaturalists 2018\u003C\u002Ftd>\n\u003Ctd align=\"center\">75.4\u003C\u002Ftd>\n\u003Ctd align=\"center\">80.1\u003C\u002Ftd>\n\u003Ctd align=\"center\">83.0\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>86.8\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">81.2\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">iNaturalists 2019\u003C\u002Ftd>\n\u003Ctd align=\"center\">80.5\u003C\u002Ftd>\n\u003Ctd align=\"center\">83.4\u003C\u002Ftd>\n\u003Ctd align=\"center\">85.7\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>88.3\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">84.1\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">Places205\u003C\u002Ftd>\n\u003Ctd align=\"center\">63.9\u003C\u002Ftd>\n\u003Ctd align=\"center\">65.8\u003C\u002Ftd>\n\u003Ctd align=\"center\">65.9\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>66.8\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">66.0\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003Ctr>\u003Ctd align=\"left\">Places365\u003C\u002Ftd>\n\u003Ctd align=\"center\">57.9\u003C\u002Ftd>\n\u003Ctd align=\"center\">59.4\u003C\u002Ftd>\n\u003Ctd align=\"center\">59.8\u003C\u002Ftd>\n\u003Ctd align=\"center\">\u003Cb>60.3\u003C\u002Fb>\u003C\u002Ftd>\n\u003Ctd align=\"center\" style=\"color:#C0C0C0\">58.0\u003C\u002Ftd>\n\u003C\u002Ftr>\n\u003C\u002Ftbody>\u003C\u002Ftable>\n\n### Pre-training\n\nThe pre-training instruction is in [PRETRAIN.md](PRETRAIN.md).\n\n### License\n\nThis project is under the CC-BY-NC 4.0 license. See [LICENSE](LICENSE) for details.\n","该项目是Masked Autoencoders (MAE)的PyTorch实现，基于论文《Masked Autoencoders Are Scalable Vision Learners》。其核心功能在于通过掩码自编码器对图像进行预训练，然后可以在多种视觉任务上进行微调以达到优秀的性能。技术特点包括支持在GPU上运行、提供预训练模型和可视化工具，并且基于timm库（版本0.3.2）构建。适用于需要高效利用大规模未标注数据来提升模型泛化能力的计算机视觉场景，如图像分类、目标检测等。此外，项目还提供了详细的使用指南和预训练权重下载链接，方便研究者和开发者快速上手。",2,"2026-06-11 03:35:34","high_star"]