BERT

文章标题：BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding 作者：Jacob Devlin, Ming-Wei Chang, Kenton Lee, Kristina Toutanova 发表时间：(NAACL-HLT 2019)

Transformer

文章标题：Attention Is All You Need 作者：Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, Illia Polosukhin 发表时间：(NIPS 2017)

MAE

文章标题：Masked Autoencoders Are Scalable Vision Learners 作者：Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, Ross Girshick

[toc]

Vision Transformer (VIT)

文章标题：An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale 作者：Alexey Dosovitskiy; Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn,Xiaohua Zhai 发表时间：(ICLR 2021)

ALBEF

文章标题：Align before Fuse: Vision and Language Representation Learning with Momentum Distillation

CLIP

文章标题：Learning Transferable Visual Models From Natural Language Supervision