over-documented over-explained chatbot using original transformer as sequence-to-sequence model
My research interest is in incorporating the the latest research in natural language processing and reinforcement learning into palpable human interaction such as fully neural network based conversational dialogue.
chloe is an end to end neural network chatbot written in PyTorch based on the transformer. Accomplishing goals through conversation is a task we can all relate to, thus chatbots are an ideal agent through which to connect new research to our current understanding of AI and demonstrate new research. chloe is built from modules that are thoroughly explained and demonstrated using jupyter notebooks and toy examples that build from these basics.
- Python 3.6 to 3.8
- torch==1.7.1 (PyTorch)
- torchtext==0.4.0
- nltk==3.4.5 (Natural Language Toolkit)
- START_HERE introduction, summon and meet chloe
- Trainer Tokenization in Sequnce to Sequence learning, intuitive explaination of Loss Functions using a toy example of cross entropy
- Building Blocks of Neural Networks Word Vector Embeddings, Positional Encoding, Neural Networks, Attention, Masking
- The Network Architecture Assembling the building blocks into an encoder and decoder
- Talk How chloe chooses which words to say in a sentence and some basic probability
if you already have python 3.6 and virtual environments, create a python 3.6 virtual environment, here i used env36 for python3.6 but you can use anything
python3 -m venv env
or
$ python3.6 -m venv env
or
virtualenv --python=/usr/bin/python3.6 env
if python3.6 is your default version, then when you type python into your terminal then it should say python version 3.6.x, and for you making the virtual environment is as simple as
$ python -m venv env
otherwise it is simple to get python3.6 and virtual environments
how to install Python 3.6 on ubuntu
install virtual environment then
how to specify the Python executable you want to use
sudo add-apt-repository ppa:jonathonf/python-3.6
sudo apt-get update
sudo apt-get install python3.6
virtualenv --python=/usr/bin/python3.6 env36
When you want to run the code activate the virtual environment inside the same folder as your environment env using
$ source env/bin/activate
install dependencies
$ pip3 install -r requirements.txt
even with virtual environments, some troubleshoot might be needed
with enough google searches you can find an answer for almost any problem
save new dependences to requirements
$ pip3 freeze > requirements.txt
You can deactivate the virtual environment using the following command in your terminal:
$ deactivate
if you get a ImportError: No module named while at the same time in your Terminal you get
pip3 install import-ipynb
Requirement already satisfied: import-ipynb in /path/to/env/lib/python3.6/site-packages (0.1.3)
This can be fixed by providing your python interpreter with the path-to-your-module,the path
import sys
sys.path.append('/path/to/env/lib/python3.6/site-packages')
you can find this path listed here
$ python3 -m site