Deep Learning Tutorial

Topics (papers)

Modern CNNs

• ImageNet Classification with Deep Convolutional Neural Networks (AlexNet)
• Going Deeper with Convolutions (GoogLeNet)
• Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning
• Deep Residual Learning for Image Recognition
• Residual Networks are Exponential Ensembles of Relatively Shallow Networks
• Wide Residual Networks

Regularization

• Dropout- A Simple Way to Prevent Neural Networks from Overfitting
• Batch Normalization- Accelerating Deep Network Training by Reducing Internal Covariate Shift

Algorithms behind AlphaGo

• Mastering the game of Go with deep neural networks and tree search (AlphaGo)

Optimization Methods

• Momentum, NAG, AdaGrad, AdaDelta, RMSprop, AdaM
• ADAM: A Method For Stochastic Optimization

Restricted Boltzmann Machine

• A Practical Guide to Training Restricted Boltzmann Machines (RBM)

Semantic Segmentation

• Fully Convolutional Networks for Semantic Segmentation
• Semantic Image Segmentation with Deep Convolutional Nets and Fully Connected CRFs
• Learning Deconvolution Network for Semantic Segmentation
• DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

Weakly Supervised Localization

• Learning Deep Features for Discriminative Localization
• Is object localization for free? – Weakly-supervised learning with convolutional neural networks

Image detection methods

• Rich feature hierarchies for accurate object detection and semantic segmentation
• Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
• Fast R-CNN
• Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
• You Only Look Once: Unified, Real-Time Object Detection
• AttentionNet: Aggregating Weak Directions for Accurate Object Detection
• SSD: Single Shot MultiBox Detector

Visual Q&A

• Image Question Answering using Convolutional Neural Network with Dynamic Parameter Prediction
• Multimodal Compact Bilinear Pooling for VQA

Deep reinforcement learning

• Playing Atari with Deep Reinforcement Learning
• Deep Reinforcement Learning with Double Q-learning

Recurrent Neural Networks

• Generating Sequences With Recurrent Neural Networks

Word embedding

• Distributed Representations of Words and Phrases and their Compositionality

Image captioning

• Show and Tell: A Neural Image Caption Generator
• Show, Attend and Tell: Neural Image Caption Generation with Visual Attention
• DenseCap: Fully Convolutional Localization Networks for Dense Captioning

Neural Styles

• Texture Synthesis Using Convolutional Neural Networks
• Understanding Deep Image Representations by Inverting Them
• A Neural Algorithm of Artistic Style

Generative adversarial networks

• Generative Adversarial Networks
• Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
• Generative Adversarial Text to Image Synthesis
• Pixel Level Domain Transfer

and implementations

• Basic Python usage (numpy, matplotlib, ..)
• Handling MNIST
• Logistic regression
• Multilayer Perceptron
• Convolutional Neural Network
• Denoising Autoencoders (+Convolutional)
• Class Activation Map
• Semantic Segmentation
• Using Custom Dataset
• Recurrent Neural Network
• Char-RNN
• Word2Vec
• Neural Style