Must-read papers on Natural Language Processing (NLP)

General

• Beyond Accuracy: Behavioral Testing of NLP Models with CheckList

Transformers

• Attention Is All You Need
• GPT: Improving Language Understanding by Generative Pre-Training
• BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
• GPT-2: Language Models are Unsupervised Multitask Learners
• Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context
• XLNet: Generalized Autoregressive Pretraining for Language Understanding
• XLM: Cross-lingual Language Model Pretraining
• RoBERTa: Robustly Optimized BERT Pretraining Approach
• DistilBERT: DistilBERT, a distilled version of BERT: smaller, faster, cheaper, and lighter
• CTRL: A Conditional Transformer Language Model for Controllable Generation
• CamemBERT: a Tasty French Language Model
• ALBERT: A Lite BERT for Self-supervised Learning of Language Representations
• T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
• XLM-RoBERTa: Unsupervised Cross-lingual Representation Learning at Scale
• MMBT: Supervised Multimodal Bitransformers for Classifying Images and Text
• FlauBERT: Unsupervised Language Model Pre-training for French
• BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension
• ELECTRA: Pre-training text encoders as discriminators rather than generators
• DialoGPT: Large-Scale Generative Pre-training for Conversational Response Generation
• Reformer: The Efficient Transformer
• Longformer: The Long-Document Transformer
• GPT3: Language Models are Few-Shot Learners
• Big Bird: Transformers for Longer Sequences

GAN and VAE

• SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient
• Generating Sentences from a Continuous Space
• Toward Controlled Generation of Text
• Adversarially Regularized Autoencoders

Dialouge Systems and Chatbots

• A Neural Conversational Model
• A Knowledge-Grounded Neural Conversation Model
• Neural Approaches to Conversational AI
• Wizard of Wikipedia: Knowledge-Powered Conversational agents
• The Dialogue Dodecathlon: Open-Domain Knowledge and Image Grounded Conversational Agents

Q/A (Question Answering)

• SQuAD: 100,000+ Questions for Machine Comprehension of Text
• Reading Wikipedia to Answer Open-Domain Questions

NER (Name Entity Recognition)

• A Survey on Recent Advances in Named Entity Recognition from Deep Learning models
• A Survey on Deep Learning for Named Entity Recognition
• Named Entity Recognition With Parallel Recurrent Neural Networks
• Evaluating the Utility of Hand-crafted Features in Sequence Labelling
• Fast and Accurate Entity Recognition with Iterated Dilated Convolutions
• Neural Adaptation Layers for Cross-domain Named Entity Recognition
• Neural Architectures for Named Entity Recognition

Sentiment Analysis

• Sentiment analysis using deep learning approaches: an overview

• Sentiment analysis using deep learning architectures: a review

Data Augmentation

• add paper name and link

Text Summarization

• add paper name and link

Reading Comprehension

• add paper name and link

NLI (Natural Language Inference)

• add paper name and link

Image Captioning

• add paper name and link

Emotion Recognition

• add paper name and link

Dependency Parsing

• add paper name and link

Semantic Parsing

• add paper name and link

POS Tagging (Part-Of-Speech Tagging)

• add paper name and link

Topic Modeling

• add paper name and link

SRL (Semantic Role Labeling)

• add paper name and link

Keyword Extraction

• add paper name and link

Semantic Similarity

• add paper name and link

Text Clustering

• add paper name and link

Thanks to

• Danial Alihosseini
• Mohammad Mahdi Samiei
• MohammadMahdi Aghajani