nlp-guide

nlp-guide is a tutorial for who is studying NLP(Natural Language Processing) using TensorFlow and Pytorch. This repository heavily refers graykode's nlp-tutorial

• You can implement applications by uploading data from data folder
• You can also download data here
• For BERT_Tensor.ipynb, you have to upload files in BERT-Base, Multilingual Cased to load pretrained model

Curriculum

0. FFNN(Feed Forward Neural Network) with one-hot BoW

• 0-1. FFNN - Binary Theme Classification
  • Colab - FFNN_Tensor.ipynb, FFNN_Torch.ipynb

1. Basic Embedding Model

• 1-1. Word2Vec(Skip-gram) - Embedding Words and Show Graph
  • Paper - Distributed Representations of Words and Phrases and their Compositionality(2013)
  • Colab - Word2Vec_Tensor.ipynb, Word2Vec_Torch.ipynb
• 1-2. Doc2Vec(Application Level) - Sentence Classification
  • Paper - Distributed Representations of Sentences and Documents(2014)
  • Colab - Doc2Vec_Tensor.ipynb, Doc2Vec_Torch.ipynb

2. RNN(Recurrent Neural Network)

• 2-1. TextRNN - Predict Next Step
  • Paper - Finding Structure in Time(1990)
  • Colab - TextRNN_Tensor.ipynb, TextRNN_Torch.ipynb
• 2-2. TextLSTM - Autocomplete
  • Paper - LONG SHORT-TERM MEMORY(1997)
  • Colab - TextLSTM_Tensor.ipynb, TextLSTM_Torch.ipynb
  • Application(Colab) - SamHangSi_Tensor.ipynb
• 2-3. biLSTM - Binary Sentiment Classification
  • Colab - biLSTM_Tensor.ipynb, biLSTM_Torch.ipynb

3. Attention Mechanism

• 3-1. Seq2Seq - Change Word
  • Paper - Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation(2014)
  • Colab - Seq2Seq_Tensor.ipynb, Seq2Seq_Torch.ipynb
  • Application(Colab) - Conversation_Tensor.ipynb
• 3-2. Seq2Seq with Attention - Translate
  • Paper - Neural Machine Translation by Jointly Learning to Align and Translate(2014)
  • Colab - Seq2Seq(Attention)_Tensor.ipynb, Seq2Seq(Attention)_Torch.ipynb

4. CNN(Convolutional Neural Network)

• 4-1. TextCNN - Binary Sentiment Classification
  • Paper - Convolutional Neural Networks for Sentence Classification(2014)
  • Colab - TextCNN_Tensor.ipynb, TextCNN_Torch.ipynb
  • Application(Colab) - Sentiment_Tensor.ipynb

5. Model based on Transformer

• 5-1. The Transformer - Translate
  • Paper - Attention Is All You Need(2017)
  • Colab - Transformer_Torch.ipynb, Transformer(Greedy_decoder)_Torch.ipynb
• 5-2. BERT - Sentiment Analysis / Classification Next Sentence
  • Paper - BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding(2018)
  • Colab - BERT_Tensor.ipynb, BERT_Torch.ipynb
• 5-3. OpenAI GPT-2 - Sample model-written texts
  • Paper - Language Models are Unsupervised Multitask Learners(2019)
  • Colab - GPT2_Torch.ipynb

Dependencies

• Python 3.5+
• Tensorflow 1.12.0+
• Pytorch 0.4.1+

Author

• Minho Ryu @bzantium
• Author Email : [email protected]