Deep Learning Resources

> Trying to organise the vast majority of    Deep Learning resources that I encounter.    

If you want to contribute, feel free to make a pull request.

The Readme currently gets generated based on the Lnkr API from Zero to Singularity at [https: // lnkr.zerotosingularity.com](https: // lnkr.zerotosingularity.com) which is currently not publicly available yet. Feel free to contact me at [email protected] if you would like to contribute.

Table of Contents

1. Benchmarking
2. Blogs
3. Books
4. Build Your Own Dl Machine
5. Communities
6. Conferences
7. Datasets
8. Docker
9. Frameworks
10. Gan
11. Github Repositories
12. Infrastructure
13. Journalism
14. Jupyter Notebooks
15. Math & Statistics
16. Miscellaneous
17. Models
18. Online Courses
19. Papers
20. Playgrounds
21. Python Libraries
22. Reinforcement Learning
23. Research
24. Resnet
25. Resources From Courses
26. Tips
27. Tools

Benchmarking

1. RedditSota/state-of-the-art-result-for-machine-learning-problems
2. EpistasisLab/penn-ml-benchmarks: A large, curated repository of benchmark datasets for evaluating supervised machine learning algorithms.
3. MLPerf
4. u39kun/deep-learning-benchmark: Deep Learning Benchmark for comparing the performance of DL frameworks, GPUs, and single vs half precision
5. Benchmarks: Deep Learning Nvidia P100 vs V100 GPU | Xcelerit

Blogs

1. Zero to Singularity
2. Google Research
3. Microsoft ML Blog
4. Apple ML Blog
5. Foldl
6. Jonas Degrave
7. xzh
8. Andrew Gibiansky
9. Otoro
10. No free hunch - The official Kaggle Blog
11. Calculated content
12. deeplearning.net
13. Andrej Karpathy
14. Colah
15. WildML
16. I am trask
17. Towards Data Science
18. Machine Learning, Data Science, Big Data, Analytics, AI
19. Research Blog
20. Distill — Latest articles about machine learning

Books

1. Deep Learning Book - Ian Goodfellow and Yoshua Bengio and Aaron Courville (11/2016)
2. Neural Networks and Deep Learning - Michael Nielsen (12/2017)
3. Hands-on Machine Learning with Scikit-Learn and Tensorflow - Aurélien Géron (3/2017)

Build Your Own Dl Machine

1. Build a Deep Learning Rig for $800
2. Building your own deep learning box
3. The $1700 great Deep Learning box: Assembly, setup and benchmarks
4. Build your 1st Deep Learning Rig
5. Which GPU(s) to Get for Deep Learning: My Experience and Advice for Using GPUs in Deep Learning (Tim Dettmers)
6. GPU Benchmark
7. Choosing Components for Personal Deep Learning Machine
8. Why building your own Deep Learning computer is 10x cheaper than AWS

Communities

1. Artificial Intelligence & Deep Learning
2. Deep Learning
3. AI & Deep Learning Enthusiasts Worldwide
4. Artificial Intelligence & Deep Learning - DeepNetGroup
5. Deep Learning / AI
6. Self-Driving car with Deep Learning
7. Artificial intelligence & Deep learning

Conferences

1. NIPS
2. ICML
3. CVPS
4. ECCV
5. ICLR - International Conference on Learning Representations
6. CVPR2018

Datasets

1. Deeplearning.net / Datasets
2. Google Street View House Numbers (SVHN)
3. MNIST
4. Tiny images
5. One Hundred Million Creative Commons Flickr Images for Research
6. Text REtrieval Conference (TREC) Data - English Test Questions (Topics) File List
7. Translation Task - EMNLP 2015 Tenth Workshop on Statistical Machine Translation
8. VGG-16
9. Pretrained CNNs - MatConvNet

Docker

1. Repository configuration | nvidia-docker
2. floydhub/dl-docker: An all-in-one Docker image for deep learning. Contains all the popular DL frameworks (TensorFlow, Theano, Torch, Caffe, etc.)
3. Explore - Docker Hub
4. Use nvidia-docker to create awesome Deep Learning Environments for R (or Python) PT I – Kai Lichtenberg

Frameworks

1. Tensorflow
2. Keras
3. Caffe
4. Caffe2
5. CNTK
6. Theano
7. PyTorch
8. Apache MXNet
9. Chainer
10. Deeplearning4j
11. Deeplearn.js
12. Fast.ai
13. Lore
14. Brain.js
15. XGBoost
16. Libsvm
17. SciKit Learn
18. Gluon
19. Knet
20. TensorLayer
21. Keras-Sharp
22. Pyro
23. GitHub - OlafenwaMoses/ImageAI
24. Darknet: Open Source Neural Networks in C
25. Create ML | Apple Developer Documentation
26. apple/turicreate: Turi Create simplifies the development of custom machine learning models.

Gan

1. Progressive Growing of GANs for Improved Quality, Stability, and Variation | Research
2. High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs
3. junyanz/CycleGAN: Software that can generate photos from paintings, turn horses into zebras, perform style transfer, and more.
4. junyanz/pytorch-CycleGAN-and-pix2pix: Image-to-image translation in PyTorch (e.g. horse2zebra, edges2cats, and more)
5. junyanz/BicycleGAN: [NIPS 2017] Toward Multimodal Image-to-Image Translation
6. deepfakes/faceswap: Non official project based on original /r/Deepfakes thread. Many thanks to him!
7. faceswap-GAN/notes at master · shaoanlu/faceswap-GAN
8. yunjey/StarGAN: PyTorch Implementation of StarGAN - CVPR 2018
9. Meow Generator – Alexia Jolicoeur-Martineau
10. Meow Generator – Alexia Jolicoeur-Martineau
11. Microsoft researchers build an AI that draws what you tell it to
12. junyanz/iGAN: Interactive Image Generation via Generative Adversarial Networks
13. [1701.07875] Wasserstein GAN
14. Generative Adversarial Networks — A Theoretical Walk-Through.

Github Repositories

1. Hands-on Machine Learning with Scikit-Learn & Tensorflow
2. Open.ai gym
3. deepmind/lab: A customisable 3D platform for agent-based AI research
4. DeepMind
5. fast.ai
6. OpenAI
7. deep-painterly-harmonization/README.md at master · luanfujun/deep-painterly-harmonization
8. pytorch/examples: A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.
9. NervanaSystems/nlp-architect: NLP Architect by Intel AI Lab: Python library for exploring the state-of-the-art deep learning topologies and techniques for natural language processing and natural language understanding
10. Nervana
11. zhixuhao/unet: unet for image segmentation
12. google/python-fire: Python Fire is a library for automatically generating command line interfaces (CLIs) from absolutely any Python object.
13. google/sentencepiece: Unsupervised text tokenizer for Neural Network-based text generation.
14. autogenerating test data along with arranging it in the directories that are required for lesson 163
15. salesforce/awd-lstm-lm: LSTM and QRNN Language Model Toolkit for PyTorch
16. KaimingHe/deep-residual-networks: Deep Residual Learning for Image Recognition
17. MichalDanielDobrzanski/DeepLearningPython35: neuralnetworksanddeeplearning.com integrated scripts for Python 3.5.2 and Theano with CUDA support
18. mnielsen/neural-networks-and-deep-learning: Code samples for my book "Neural Networks and Deep Learning"
19. deeppomf/DeepLearningAnimePapers: A list of papers and other resources on deep learning with anime style images.
20. FavoritePapers/image_generation.md at master · SeitaroShinagawa/FavoritePapers
21. PeterTor/sparse_convolution: sparse convolution Implementation
22. general-deep-image-completion/README.md at master · adamstseng/general-deep-image-completion
23. uber/horovod: Distributed training framework for TensorFlow, Keras, and PyTorch.
24. wookayin/gpustat: 📊 A simple command-line utility for querying and monitoring GPU status
25. Model Zoo · BVLC/caffe Wiki
26. ysh329/deep-learning-model-convertor: The convertor/conversion of deep learning models for different deep learning frameworks/softwares.
27. Microsoft/MMdnn: MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.
28. ROCm-Developer-Tools/HIP: HIP : Convert CUDA to Portable C++ Code

Infrastructure

1. Paperspace
2. Crestle
3. Microsoft Azure
4. AWS Amazon
5. Google Compute Engine
6. Infrastructure for Deep Learning

Journalism

1. Hello World - Bloomberg

Jupyter Notebooks

1. Deep Learning with Python Notebooks
2. Jupyter Docker Deploy

Math & Statistics

1. Hands-on Machine Learning with Scikit-Learn and Tensorflow - Aurélien Géron (3/2017)
2. Probability and Statistics for Programmers
3. Probabilistic Programming & Bayesian Methods for Hackers
4. Understanding Machine Learning: from Theory to Algorithms
5. Elements of statistical learning
6. An introduction to statistical learning
7. Foundations of data science
8. A programmer's guide to data Mining
9. Mining massive datasets
10. Machine learning yearning

Miscellaneous

1. ec2-spotter
2. Python styleguide
3. Cycle-gan
4. Some snippets

Models

1. ModelDepot

Online Courses

1. Machine Learning- Coursera - Andrew Ng
2. Deep Learning Specialization - Coursera
3. Advanced Machine Learning Specialization - Coursera
4. Fast.ai
5. Deep Learning Udacity
6. MIT 6.S094: Deep Learning for Self-Driving Cars
7. 6.S191: Introduction to Deep Learning
8. CS231n: Convolutional Neural Networks for Visual Recognition
9. Google's Machine learning crash course
10. Google.ai
11. Machine Learning with TensorFlow on Google Cloud Platform | Coursera
12. Machine Learning Crash Course  |  Google Developers
13. Self-Driving Car | Udacity
14. MIT 6.S094: Deep Learning for Self-Driving Cars
15. AI School
16. Reviews for Machine Learning from Coursera | Class Central
17. Reviews for 6.S191: Introduction to Deep Learning from Massachusetts Institute of Technology | Class Central
18. Salesforce Einstein Discovery - Easy AI and Machine Learning | Udemy
19. Artificial Intelligence (AI) | edX
20. Artificial Intelligence A-Z™: Learn How To Build An AI | Udemy
21. Lecture Collection | Convolutional Neural Networks for Visual Recognition (Spring 2017) - YouTube

Papers

1. Deep Learning Papers Reading Roadmap
2. Machine Theory of Mind - DeepMind
3. [1801.06146] Universal Language Model Fine-tuning for Text ... - arXiv
4. A disciplined approach to neural network hyper-parameters: Part 1 ...
5. Visualizing and Understanding Convolutional Networks
6. Cyclical Learning Rates for Training Neural Networks
7. 1704.00109.pdf
8. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
9. Deep Video Portraits
10. Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks
11. A guide to convolution arithmetic for deep learning
12. Generative Reversible Networks
13. Patch-Based Image Inpainting with Generative Adversarial Networks
14. 1712.00080.pdf
15. NVIDIA SPLATNet Research Paper Wins a Major CVPR 2018 Award - NVIDIA Developer News CenterNVIDIA Developer News Center
16. Adversarial Attacks on Face Detectors using Neural Net based Constrained Optimization
17. 1803.09820.pdf
18. Show, Attend and Tell: Neural Image CaptionGeneration with Visual Attention
19. Show, Attend and Tell: Neural Image CaptionGeneration with Visual Attention

Playgrounds

1. Tensorflow Playground
2. ConvNetJS
3. Google Collab
4. Azure Notebooks
5. Seedbank

Python Libraries

1. Numpy
2. Pandas: Python Data Analysis Library
3. seaborn: statistical data visualization
4. Scikit-Learn
5. PDL - Python Download library

Reinforcement Learning

1. Reinforcement Learning from scratch – Insight Data

Research

1. Facebook AI | Tools | Open Source Deep Learning Tools
2. Facebook AI Research – Facebook Research
3. AI & Deep Learning Publications & Researchers | NVIDIA Research
4. Berkeley Artificial Intelligence Research Lab
5. Artificial Intelligence Archives - NVIDIA Developer News CenterNVIDIA Developer News Center

Resnet

1. http://arxiv.org/abs/1512.03385
2. blog
3. code
4. http://image-net.org/challenges/LSVRC/2015/
5. COCO
6. https://github.com/KaimingHe/resnet-1k-layers
7. VGG-16
8. Training and investigating Residual Nets
9. Deep Residual Learning for Image Recognition
10. ResNet training in Torch
11. Deep Residual learning - paper implementation
12. implementation of the deep residual network used for cifar10
13. Residual networks in torch (MNIST 100 layers)
14. NOAA Right Whale Recognition, Winner's Interview
15. Using neon for Scene Recognition: Mini-Places2
16. Matlab (MatConvNet) implementation of "Deep Residual Learning for Image Recognition"
17. imagenet-resnet.py
18. keras-resnet
19. ImageNet ILSVRC classification
20. ResNet in TensorFlow

Resources From Courses

1. Fast.ai - Lesson 1

• [Introduction to Resnet models](https://github.com/KaimingHe/deep-residual-networks]
• [Cyclical learning rates for training neural networks (paper)](http://arxiv.org/abs/1506.01186]
• [Visually explained convolutions (site)](http://setosa.io/ev/image-kernels/]
• [Sigmoid explained (online book chapter](http://neuralnetworksanddeeplearning.com/chap4.html]
• [Visualizing and understanding convolutional networks (paper)](https://cs.nyu.edu/~fergus/papers/zeilerECCV2014.pdf]
• [Snapshot emsembles](https://arxiv.org/abs/1704.00109]
• [Kaggle-cli](https://github.com/floydwch/kaggle-cli]

2. Fast.ai - Lesson 2

• [Summary of lesson 2](https://medium.com/@apiltamang/case-study-a-world-class-image-classifier-for-dogs-and-cats-err-anything-9cf39ee4690e]

3. Fast.ai - Lesson 3

• [AWS Setup](https://github.com/reshamas/fastai_deeplearn_part1/blob/master/tools/aws_ami_gpu_setup.md]
• [Tmux](https://github.com/reshamas/fastai_deeplearn_part1/blob/master/tools/tmux.md]
• [Notes on fast.ai](https://github.com/reshamas/fastai_deeplearn_part1/tree/master/tools]
• [Summary of Lesson 2](https://medium.com/@apiltamang/case-study-a-world-class-image-classifier-for-dogs-and-cats-err-anything-9cf39ee4690e]
• [Learning rate finder](https://medium.com/@surmenok/estimating-optimal-learning-rate-for-a-deep-neural-network-ce32f2556ce0]
• [Convolutional Neural Networks in 5 minutes](https://medium.com/@init_27/convolutional-neural-network-in-5-minutes-8f867eb9ca39]
• [Visualizing learning rate vs batch rate](https://miguel-data-sc.github.io/2017-11-05-first/]
• [Decoding the ResNet architecture](http://teleported.in/posts/decoding-resnet-architecture/]
• [A practitioners' guide to PyTorch](https://medium.com/@radekosmulski/a-practitioners-guide-to-pytorch-1d0f6a238040]

4. Fast.ai - Lesson 4

• [Techburst: Improving the way we work with learning rate.](https://techburst.io/improving-the-way-we-work-with-learning-rate-5e99554f163b]
• [validation sets](http://www.fast.ai/2017/11/13/validation-sets/]
• [Teleported.in: The Cyclical Learning Rate technique](http://teleported.in/posts/cyclic-learning-rate/]
• [Transfer Learning Using Differential Rates](https://towardsdatascience.com/transfer-learning-using-differential-learning-rates-638455797f00]
• [Deep Learning, NLP, and Representations115](http://colah.github.io/posts/2014-07-NLP-RNNs-Representations/]

5. Fast.ai - Lesson 5

• [Structured Deep Learning](https://towardsdatascience.com/structured-deep-learning-b8ca4138b848]
• [Fun with small image data-sets](https://towardsdatascience.com/fun-with-small-image-data-sets-part-2-54d683ca8c96]

6. Fast.ai - Lesson 6

• [Optimization for Deep Learning Highlights in 2017](http://ruder.io/deep-learning-optimization-2017/]
• [Entity Embeddings of Categorical Variables](https://arxiv.org/abs/1604.06737]

Tips

1. 28 Jupyter Notebook tips, tricks, and shortcuts
2. Lesser known ways of using Jupyter Notebooks

Tools

1. NVIDIA Container Runtime and Orchestrators | NVIDIA Developer
2. Introducing Apex: PyTorch Extension with Tools to Realize the Power of Tensor Cores - NVIDIA Developer News CenterNVIDIA Developer News Center
3. Download DeepStream SDK 2.0 Today to Develop Scalable Video Analytics Applications - NVIDIA Developer News CenterNVIDIA Developer News Center
4. Announcing NVIDIA DALI and NVIDIA nvJPEG - NVIDIA Developer News CenterNVIDIA Developer News Center
5. NVIDIA Releases TensorRT 4 - NVIDIA Developer News CenterNVIDIA Developer News Center