Awesome Self-supervised papers in CV, NLP, Graph, Recommendation, ML and so on.

1_CV

• 2006 (Yann LeCun) (CVPR) Dimensionality reduction by learning an invariant mapping
  
• 2015 (CMU) (ICCV) Unsupervised Learning of Visual Representations Using Videos
  
• 2015 (CMU) (ICCV) Unsupervised Visual Representation Learning by Context Prediction
  
• 2016 (CVPR) Slow and steady feature analysis - higher order temporal coherence in video
  
• 2016 (ECCV) Unsupervised learning of visual representations by solving jigsaw puzzles
  
• 2016 (UCB) (CVPR) Context Encoders - Feature Learning by Inpainting
  
• 2016 (UCB) (ECCV) Colorful image colorization
  
• 2017 (Google) (ICRA) Time-Contrastive Networks - Self-Supervised Learning from Video
  
• 2017 (ICCV) Unsupervised Representation Learning by Sorting Sequences
  
• 2017 (Kaiming He) (ICCV) Transitive Invariance for Self-Supervised Visual Representation Learning
  
• 2017 (UCB) (CVPR) Split-Brain Autoencoders - Unsupervised Learning by Cross-Channel Prediction
  
• 2018 (DeepMind) (Arxiv) [CPC] Representation Learning with Contrastive Predictive Coding
  
• 2018 (ICLR) Unsupervised Representation Learning by Predicting Image Rotations
  
• 2019 (Arxiv) [CPC2] Data-Efficient Image Recognition with Contrastive Predictive Coding
  
• 2019 (Facebook) (Arxiv) Scaling and Benchmarking Self-Supervised Visual Representation Learning
  
• 2019 (Google) (ICCV) VideoBERT - A Joint Model for Video and Language Representation Learning
  
• 2019 (NIPS) [AMDIM] Learning representations by maximizing mutual information across views
  
• 2019 (Yoshua Bengio) (ICLR) [DIM] Learning deep representations by mutual information estimation and maximization
  
• 2020 (DeepMind) (Arxiv) [BYOL] Bootstrap Your Own Latent - A New Approach to Self-Supervised Learning
  
• 2020 (Google) (Arxiv) Supervised Contrastive Learning
  
• 2020 (Hinton) (Arxiv) Big Self-Supervised Models are Strong Semi-Supervised Learners
  
• 2020 (Hinton) (ICML) [SimCLR] A Simple Framework for Contrastive Learning of Visual Representations
  
• 2020 (Kaiming He) (CVPR) [MoCo] Momentum Contrast for Unsupervised Visual Representation Learning
  

2_NLP

• 2013 (Google) (NIPS) [word2vec] Distributed representations of words and phrases and their compositionality
  
• 2017 (Google) (NIPS) [Transformer] Attention is All you Need
  
• 2018 (Google) (NAACL) [BERT] Bert - Pre-training of deep bidirectional transformers for language understanding
  
• 2018 (OpenAI) (Arxiv) [GPT] Improving Language Understanding by Generative Pre-Training
  
• 2019 (ACL) Self-Supervised Dialogue Learning
  
• 2019 (OpenAI) (Arxiv) [GPT2] Language Models are Unsupervised Multitask Learners
  
• 2020 (DeepMind) (ICLR) A Mutual Information Maximization Perspective of Language Representation Learning
  
• 2020 (OpenAI) (Arxiv) [GPT3] Language Models are Few-Shot Learners
  
• 2020 (UW) (ACL) Don’t Stop Pretraining - Adapt Language Models to Domains and Tasks
  

3_Graph

• 2020 (Arxiv) Self-supervised Learning on Graphs - Deep Insights and New Direction
  
• 2020 (KDD) GPT-GNN - Generative Pre-Training of Graph Neural Networks
  
• 2020 (WWW) Graph Representation Learning via Graphical Mutual Information Maximization
  

4_Recommendation

• 2020 (Alibaba) (AAAI) [DMR] Deep Match to Rank Model for Personalized Click-Through Rate Prediction
  
• 2020 (Alibaba) (CIKM) [BERT4Rec] BERT4Rec - Sequential Recommendation with Bidirectional Encoder Representations from Transformer
  
• 2020 (Alibaba) (KDD) Disentangled Self-Supervision in Sequential Recommenders
  
• 2020 (Arxiv) UserBERT - Self-supervised User Representation Learning
  
• 2020 (Arxiv) [SGL] Self-supervised Graph Learning for Recommendation
  
• 2020 (CIKM) [S3Rec] S3-Rec - Self-Supervised Learning for Sequential Recommendation with Mutual Information Maximization
  
• 2020 (EMNLP) [PTUM] PTUM - Pre-training User Model from Unlabeled User Behaviors via Self-supervision
  
• 2020 (Google) (Arxiv) Self-supervised Learning for Large-scale Item Recommendations
  
• 2020 (SIGIR) Self-Supervised Reinforcement Learning for Recommender Systems
  
• 2020 (Xiangnan He) (Arxiv) Self-supervised Graph Learning for Recommendation
  
• 2021 (Alibaba) (Arxiv) [CLRec] Contrastive Learning for Debiased Candidate Generation in Large-Scale Recommender Systems
  
• 2021 (Alibaba) (WWW) Contrastive Pre-training for Sequential Recommendation
  
• 2021 (WSDM) [Prop] PROP - Pre-training with Representative Words Prediction for Ad-hoc Retrieval
  

5_ML

Multi-task_Self-supervised_Learning

• 2017 (DeepMind) (ICCV) Multi-task Self-Supervised Visual Learning
  
• 2018 (CVPR) Cross-domain Self-supervised Multi-task Feature Learning Using Synthetic Imagery
  

Reinforcement_Learning

• 2019 (Yoshua Bengio) (NIPS) Unsupervised State Representation Learning in Atari
  

Self-supervised_and_Semi-supervised_Learning

• 2019 (Facebook) (Arxiv) Billion-scale semi-supervised learning for image classification
  
• 2019 (Google) (ICCV) S​4L - Self-Supervised Semi-Supervised Learning
  

Self_training

• 2020 (Google) (CVPR) Self-Training With Noisy Student Improves ImageNet Classification
  
• 2020 (Google) (NIPS) Rethinking Pre-training and Self-training
  

Task_Related_Self-supervised_Learning

• 2019 (Arxiv) Rethinking Data Augmentation - Self-Supervision and Self-Distillation
  
• 2019 (ICCV) Boosting Few-Shot Visual Learning with Self-Supervision
  

6_Survey

• 2020 (Jie Tang) (Arxiv) Self-supervised Learning - Generative or Contrastive
  
• 2020 (T-PAMI) Self-supervised Visual Feature Learning with Deep Neural Networks - A Survey
  

7_Talks

• Self-Supervised Learning_Yann LeCun_AAAI 2020