Transformer

文章标题：Attention Is All You Need 作者：Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin 发表时间：(NIPS 2017)

MAE

文章标题：Masked Autoencoders Are Scalable Vision Learners 作者：Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, Ross Girshick

[toc]

Vision Transformer (VIT)

文章标题：An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale 作者：Alexey Dosovitskiy; Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn,Xiaohua Zhai 发表时间：(ICLR 2021)

ALBEF

文章标题：Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

CLIP

文章标题：Learning Transferable Visual Models From Natural Language Supervision

VILT

文章标题：ViLT: Vision-and-Language Transformer Without Convolution or Region Supervision