MAE

文章标题：Masked Autoencoders Are Scalable Vision Learners 作者：Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, Ross Girshick

[toc]

Vision Transformer (VIT)

文章标题：An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale 作者：Alexey Dosovitskiy; Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn,Xiaohua Zhai 发表时间：(ICLR 2021)

ALBEF

文章标题：Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

CLIP

文章标题：Learning Transferable Visual Models From Natural Language Supervision

VILT

文章标题：ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision

SwAV

文章标题：Unsupervised Learning of Visual Features by Contrasting Cluster Assignments