This roadmap outlines the mathematical concepts and topics covered in various deep learning resources. It provides a structured path to understand the necessary mathematical foundations for deep learning.
- Linear Algebra
- Probability and Statistics
- Calculus
- Numerical Computation
- Machine Learning Fundamentals
- Deep Learning Basics
- Advanced Deep Learning Topics
- Vectors and Matrices: Vector operations, matrix operations, dot product, matrix-vector multiplication.
-
Vectors and Matrices by Math Is Fun
-
Vectors and Matrices Crash Course by BetterExplained
-
Vectors and Matrices in MATLAB
-
Introduction to Vectors and Matrices by Purplemath
-
Vector and Matrix Basics by MathBootCamps (YouTube video)
-
Linear Algebra: Vectors and Matrices by MIT OpenCourseWare
-
Linear Algebra: Foundations to Frontiers by edX
-
Linear Algebra Refresher Course by Khan Academy
-
Introduction to Linear Algebra by Gilbert Strang (Textbook)
-
Linear Algebra Done Right by Sheldon Axler (Textbook)
-
3Blue1Brown: Essence of Linear Algebra (YouTube series)
-
Matrix Operations: Addition, Subtraction, Scalar Multiplication by Math Easy Solutions
-
Matrix Operations: Multiplication, Transpose, and Determinant by Math CUE math
-
Matrix Operations by Purplemath
-
Linear Algebra Toolkit: Matrix Operations by MathsIsPower4U
-
Matrix Operations by MathOnlineSchool
-
Matrix Operations by Khan Academy
-
Matrix Operations by MathIsPower4U
-
Linear Algebra: Matrix Operations by MathDoctorBob
-
Matrix Operations by Krista King Math
-
Matrix Operations by MathPortal
-
Linear Independence and Span by Khan Academy
-
Rank of a Matrix by Math Is Fun
-
Linear Independence and Rank by MIT OpenCourseWare
-
Linear Independence and Rank by MathDoctorBob
-
Linear Algebra: Introduction to Linear Independence and Span by MathBootCamps
-
Linear Independence and Rank by MathOnlineSchool
-
Linear Independence and Rank by Krista King Math
-
Linear Algebra: Basis, Linear Independence, and Rank by MathTheBeautiful
-
Linear Independence and Rank by MathPortal
-
Matrix Inverse by Khan Academy
-
Pseudoinverse by Math Is Fun
-
Matrix Inverse and Pseudoinverse by MIT OpenCourseWare
-
Matrix Inverse and Pseudoinverse by MathDoctorBob
-
Matrix Inverse and Pseudoinverse by MathTheBeautiful
-
Matrix Inversion and Pseudoinverse by MathOnlineSchool
-
Pseudoinverse by Krista King Math
-
Matrix Inverse and Pseudoinverse by MathPortal
- Eigendecomposition and Diagonalization: Eigenvalues, eigenvectors, eigendecomposition, diagonalization.
-
Eigenvectors and Eigenvalues by Khan Academy
-
Eigendecomposition by Math Is Fun
-
Eigendecomposition and Diagonalization by MIT OpenCourseWare
-
Eigenvectors, Eigenvalues, and Diagonalization by MathDoctorBob
-
Eigenvalues and Eigenvectors by 3Blue1Brown
-
Eigendecomposition and Diagonalization by MathTheBeautiful
-
Eigenvectors and Eigenvalues by MathOnlineSchool
-
Eigendecomposition and Diagonalization by Krista King Math
-
Eigendecomposition and Diagonalization by MathPortal
-
Singular Value Decomposition (SVD) by Khan Academy
-
Singular Value Decomposition (SVD) by Math Is Fun
-
Singular Value Decomposition (SVD) by MIT OpenCourseWare
-
Singular Value Decomposition (SVD) by MathDoctorBob
-
Singular Value Decomposition (SVD) by MathTheBeautiful
-
Singular Value Decomposition (SVD) by MathOnlineSchool
-
Singular Value Decomposition (SVD) by Krista King Math
-
Singular Value Decomposition (SVD) by MathPortal
- Probability Basics: Sample space, events, probability axioms, conditional probability, Bayes' rule.
s
-
Probability Basics by Khan Academy
-
Introduction to Probability by MIT OpenCourseWare
-
Probability Basics by Math Is Fun
-
Probability Basics by Stat Trek
-
Probability Basics by CrashCourse
-
Introduction to Probability by Khan Academy
-
Probability Fundamentals by Udacity
-
Introduction to Probability and Data by Duke University (Coursera)
-
Probability Basics by Statistics How To
-
Probability Basics by Math Goodies
- Random Variables and Probability Distributions: Discrete and continuous random variables, probability mass function (PMF), probability density function (PDF).
-
Random Variables and Probability Distributions by Khan Academy
-
Random Variables and Probability Distributions by MIT OpenCourseWare
-
Random Variables and Probability Distributions by Stat Trek
-
Random Variables and Probability Distributions by Math Is Fun
-
Random Variables and Probability Distributions by CrashCourse
-
Random Variables and Probability Distributions by Khan Academy
-
Introduction to Random Variables and Probability Distributions by Rice University (Coursera)
-
Random Variables and Probability Distributions by CliffsNotes
-
Introduction to Random Variables and Probability Distributions by Study.com
-
Random Variables and Probability Distributions by Math Goodies
- Expectation, Variance, and Covariance: Expected value, variance, covariance, correlation coefficient.
-
Expectation, Variance, and Covariance by Khan Academy
-
Expectation, Variance, and Covariance by Math Is Fun
-
Expectation, Variance, and Covariance by Stat Trek
-
Expectation, Variance, and Covariance by MIT OpenCourseWare
-
Expectation, Variance, and Covariance by CrashCourse
-
Expectation, Variance, and Covariance by MathDoctorBob
-
Expectation, Variance, and Covariance by Krista King Math
-
Expectation, Variance, and Covariance by Math Goodies
- Common Probability Distributions: Uniform, Bernoulli, Binomial, Gaussian (Normal), Exponential, Poisson.
-
Common Probability Distributions by Khan Academy
-
Common Probability Distributions by Stat Trek
-
Common Probability Distributions by Math Is Fun
-
Common Probability Distributions by CrashCourse
-
Common Probability Distributions by Rice University (Coursera)
-
Common Probability Distributions by MIT OpenCourseWare
-
Probability Distributions by Krista King Math
-
Introduction to Probability Distributions by Study.com
-
Common Probability Distributions by Math Goodies
-
Bayes' Rule and Conditional Probability by Khan Academy
-
Bayes' Rule and Conditional Probability by Math Is Fun
-
Bayes' Rule and Conditional Probability by CrashCourse
-
Bayes' Rule and Conditional Probability by Stat Trek
-
Bayes' Rule and Conditional Probability by MIT OpenCourseWare
-
Bayes' Rule and Conditional Probability by MathDoctorBob
-
Conditional Probability and Bayes' Rule by Krista King Math
-
Bayes' Rule and Conditional Probability by Math Goodies
-
Bayes' Theorem by Better Explained
-
Bayes' Rule and Conditional Probability by Study.com
-
Information Theory by Khan Academy
-
Information Theory by MIT OpenCourseWare
-
Information Theory by Math Is Fun
-
Information Theory by CrashCourse
-
Information Theory by Stanford University (Coursera)
-
Information Theory by Krista King Math
-
An Introduction to Information Theory by John Watrous
-
Information Theory by All About Circuits
-
Information Theory by Math Goodies
-
Differential Calculus by Khan Academy
-
Differential Calculus by MIT OpenCourseWare
-
Differential Calculus by Math Is Fun
-
Differential Calculus by CrashCourse
-
Calculus 1: Differentiation by The Essence of Mathematics
-
Differential Calculus by Krista King Math
-
Differential Calculus by MathDoctorBob
-
Calculus I: Differentiation by UCI Open
-
Differential Calculus by Math Goodies
- Integral Calculus: Integrals, definite and indefinite integrals, multivariable calculus, gradients.
-
Integral Calculus by Khan Academy
-
Integral Calculus by MIT OpenCourseWare
-
Integral Calculus by Math Is Fun
-
Integral Calculus by CrashCourse
-
Calculus 2: Integration by The Essence of Mathematics
-
Integral Calculus by Krista King Math
-
Integral Calculus by MathDoctorBob
-
Calculus II: Integration by UCI Open
-
Integral Calculus by Math Goodies
- Optimization Techniques: Gradient descent, stochastic gradient descent (SGD), learning rate, convex optimization.
-
Optimization Techniques by Khan Academy
-
Optimization Techniques by MIT OpenCourseWare
-
Optimization Techniques by Math Is Fun
-
Optimization Techniques by CrashCourse
-
Optimization Techniques by Stanford University (Coursera)
-
Optimization Techniques by Krista King Math
-
Optimization Techniques by MathDoctorBob
-
Optimization Techniques by University of Washington (Coursera)
-
Optimization Techniques by Math Goodies
-
Floating Point Representation by Khan Academy
-
Floating Point Representation by Wikipedia
-
Floating Point Representation by Exploring Binary
-
IEEE 754 Floating Point Standard by Explained Visually
-
Floating Point Representation by Computerphile
-
Floating Point Representation by MathWorks
-
Floating Point Representation by GeeksforGeeks
-
Floating Point Representation by Math Goodies
- Numerical Stability: Stability issues in numerical computations, conditioning and ill-conditioning.
-
Numerical Stability by Wikipedia
-
Numerical Stability and Conditioning by Khan Academy
-
Numerical Stability by Numerical Tours
-
Numerical Stability by MathWorks
-
Numerical Stability by MIT OpenCourseWare
-
Numerical Stability and Conditioning by Numerical Methods for Engineers
-
Floating Point Arithmetic and Numerical Stability by Computational Physics with Python
-
Numerical Stability in Machine Learning by Towards Data Science
-
Numerical Stability in Deep Learning by Machine Learning Mastery
- Gradient-Based Optimization: Calculating gradients, optimization algorithms, learning rate tuning.
-
Gradient Descent by Khan Academy
-
Gradient-Based Optimization by Stanford University (Coursera)
-
Gradient-Based Optimization by Machine Learning Mastery
-
Gradient-Based Optimization by Andrew Ng
-
Gradient-Based Optimization by DeepLearning.AI
-
Gradient-Based Optimization by MathWorks
-
Gradient Descent Optimization Algorithms by Sebastian Ruder
-
Gradient-Based Optimization by Christopher Bishop
-
Gradient-Based Optimization by OpenAI Spinning Up
-
Automatic Differentiation by Khan Academy
-
Automatic Differentiation by Stanford University
-
Automatic Differentiation by DiffSharp
-
Symbolic Differentiation by Math Is Fun
-
Symbolic Differentiation by MIT OpenCourseWare
-
Automatic Differentiation by TensorFlow
-
Automatic Differentiation by PyTorch
-
Automatic Differentiation and Symbolic Differentiation by MathWorks
-
Automatic Differentiation and Symbolic Differentiation by UC Berkeley
- Linear Regression: Model representation, cost function, normal equation, gradient descent for linear regression.
-
Linear Regression by Khan Academy
-
Linear Regression by Stanford University (Coursera)
-
Linear Regression by Andrew Ng
-
Linear Regression by Towards Data Science
-
Linear Regression by StatQuest with Josh Starmer
-
Linear Regression by MathWorks
-
Linear Regression by Machine Learning Mastery
-
Linear Regression by Python Data Science Handbook
-
Linear Regression by OpenAI Spinning Up
- Logistic Regression: Sigmoid function, logistic regression model, binary and multiclass logistic regression.
-
Logistic Regression by Khan Academy
-
Logistic Regression by Stanford University (Coursera)
-
Logistic Regression by Andrew Ng
-
Logistic Regression by Towards Data Science
-
Logistic Regression by StatQuest with Josh Starmer
-
Logistic Regression by MathWorks
-
Logistic Regression by Machine Learning Mastery
-
Logistic Regression by Python Data Science Handbook
-
Logistic Regression by OpenAI Spinning Up
-
Support Vector Machines (SVM) by Khan Academy
-
Support Vector Machines (SVM) by Stanford University (Coursera)
-
Support Vector Machines (SVM) by Andrew Ng
-
Support Vector Machines (SVM) by StatQuest with Josh Starmer
-
Support Vector Machines (SVM) by Scikit-learn Documentation
-
Support Vector Machines (SVM) by Machine Learning Mastery
-
Support Vector Machines (SVM) by Python Data Science Handbook
-
Support Vector Machines (SVM) by OpenAI Spinning Up
-
Support Vector Machines (SVM) by LIBSVM
-
Decision Trees by Khan Academy
-
Decision Trees and Random Forests by Stanford University (Coursera)
-
Decision Trees by Andrew Ng
-
Decision Trees and Random Forests by StatQuest with Josh Starmer
-
Decision Trees and Random Forests by Scikit-learn Documentation
-
Decision Trees and Random Forests by Machine Learning Mastery
-
Decision Trees and Random Forests by Python Data Science Handbook
-
Decision Trees and Random Forests by OpenAI Spinning Up
-
Decision Trees and Random Forests by Scikit-learn
-
Evaluation Metrics for Machine Learning by Towards Data Science
-
Evaluation Metrics for Classification by Machine Learning Mastery
-
Evaluation Metrics for Regression by Machine Learning Mastery
-
Evaluation Metrics for Binary Classification by Scikit-learn Documentation
-
Evaluation Metrics for Multiclass Classification by Scikit-learn Documentation
-
Evaluation Metrics for Imbalanced Classification by Machine Learning Mastery
-
Evaluation Metrics for Clustering by Scikit-learn Documentation
-
Evaluation Metrics for Recommender Systems by Machine Learning Mastery
-
Evaluation Metrics for Natural Language Processing (NLP) by Machine Learning Mastery
-
Evaluation Metrics for Time Series Forecasting by Machine Learning Mastery
- Feedforward Neural Networks: Architecture, activation functions, forward propagation, backward propagation.
-
Neural Networks by Khan Academy
-
Feedforward Neural Networks by Stanford University (Coursera)
-
Feedforward Neural Networks by Andrew Ng
-
Feedforward Neural Networks by DeepLearning.AI
-
Feedforward Neural Networks by PyTorch
-
Feedforward Neural Networks by TensorFlow
-
Feedforward Neural Networks by Machine Learning Mastery
-
Feedforward Neural Networks by Python Data Science Handbook
-
Feedforward Neural Networks by OpenAI Spinning Up
-
Backpropagation Algorithm by Khan Academy
-
Backpropagation Algorithm by Stanford University (Coursera)
-
Backpropagation Algorithm by Andrew Ng
-
Backpropagation Algorithm by DeepLearning.AI
-
Backpropagation Algorithm by Machine Learning Mastery
-
Backpropagation Algorithm by Towards Data Science
-
Backpropagation Algorithm by Python Data Science Handbook
-
Backpropagation Algorithm by OpenAI Spinning Up
-
Backpropagation Algorithm by Deep Learning with PyTorch
-
Weight Initialization in Neural Networks by Machine Learning Mastery
-
Weight Initialization in Deep Learning by Deeplearning.AI
-
Weight Initialization in Neural Networks by TensorFlow
-
Weight Initialization in Neural Networks by PyTorch
-
Weight Initialization in Neural Networks by Deep Learning with Python book
-
Weight Initialization in Neural Networks by Towards Data Science
-
Weight Initialization in Neural Networks by Neural Designer
-
Weight Initialization in Neural Networks by Machine Learning Wiki
-
Weight Initialization in Neural Networks by OpenAI Spinning Up
- Gradient-Based Optimization Algorithms: Gradient descent, mini-batch gradient descent, stochastic gradient descent.
-
Gradient-Based Optimization Algorithms by Machine Learning Mastery
-
Gradient-Based Optimization Algorithms by Stanford University (Coursera)
-
Gradient-Based Optimization Algorithms by Andrew Ng
-
Gradient-Based Optimization Algorithms by DeepLearning.AI
-
Gradient-Based Optimization Algorithms by Sebastian Ruder
-
Gradient-Based Optimization Algorithms by PyTorch
-
Gradient-Based Optimization Algorithms by TensorFlow
-
Gradient-Based Optimization Algorithms by OpenAI Spinning Up
-
Gradient-Based Optimization Algorithms by Machine Learning Wiki
-
Gradient-Based Optimization Algorithms by Deep Learning with Python book
-
Regularization Techniques in Machine Learning by Machine Learning Mastery
-
Regularization Techniques in Deep Learning by Deeplearning.AI
-
Regularization Techniques in Machine Learning by Towards Data Science
-
Regularization Techniques in Neural Networks by DeepLearning.AI
-
Regularization Techniques in Machine Learning by Scikit-learn Documentation
-
Regularization Techniques in Deep Learning by TensorFlow
-
Regularization Techniques in Neural Networks by PyTorch
-
Regularization Techniques in Machine Learning by Sebastian Raschka
-
Regularization Techniques in Machine Learning by OpenAI Spinning Up
- Convolutional Neural Networks (CNNs): Convolutional layers, pooling layers, convolution arithmetic.
-
Convolutional Neural Networks by Stanford University (Coursera)
-
Convolutional Neural Networks (CNNs) by DeepLearning.AI
-
Convolutional Neural Networks (CNNs) by Andrew Ng
-
Convolutional Neural Networks by Machine Learning Mastery
-
Convolutional Neural Networks by TensorFlow
-
Convolutional Neural Networks by PyTorch
-
Convolutional Neural Networks by Deep Learning with Python book
-
Convolutional Neural Networks by Machine Learning Wiki
-
Convolutional Neural Networks by OpenAI Spinning Up
-
Recurrent Neural Networks by Stanford University (Coursera)
-
Recurrent Neural Networks (RNNs) by DeepLearning.AI
-
Recurrent Neural Networks (RNNs) by Andrew Ng
-
Recurrent Neural Networks by Machine Learning Mastery
-
Recurrent Neural Networks by TensorFlow
-
Recurrent Neural Networks by PyTorch
-
Recurrent Neural Networks by Deep Learning with Python book
-
Recurrent Neural Networks by Machine Learning Wiki
-
Recurrent Neural Networks by OpenAI Spinning Up
-
Generative Adversarial Networks (GANs) by Stanford University (Coursera)
-
Generative Adversarial Networks (GANs) by DeepLearning.AI
-
Generative Adversarial Networks (GANs) by Ian Goodfellow, et al. (Original GAN Paper)
-
Generative Adversarial Networks (GANs) by Machine Learning Mastery
-
Generative Adversarial Networks (GANs) by TensorFlow
-
Generative Adversarial Networks (GANs) by PyTorch
-
Generative Adversarial Networks (GANs) by Deep Learning with Python book
-
Generative Adversarial Networks (GANs) by Machine Learning Wiki
-
Generative Adversarial Networks (GANs) by OpenAI Spinning Up
-
Batch Normalization by Machine Learning Mastery
-
Batch Normalization by Stanford University (Coursera)
-
Batch Normalization by DeepLearning.AI
-
Batch Normalization by Andrew Ng
-
Batch Normalization by TensorFlow
-
Batch Normalization by PyTorch
-
Batch Normalization by Deep Learning with Python book
-
Batch Normalization by Machine Learning Wiki
-
Batch Normalization by OpenAI Spinning Up
-
Batch Normalization by Machine Learning Mastery
-
Batch Normalization by Stanford University (Coursera)
-
Batch Normalization by DeepLearning.AI
-
Batch Normalization by Andrew Ng
-
Batch Normalization by TensorFlow
-
Batch Normalization by PyTorch
-
Batch Normalization by Deep Learning with Python book
-
Batch Normalization by Machine Learning Wiki
-
Batch Normalization by OpenAI Spinning Up
-
Reinforcement Learning by David Silver (DeepMind)
-
Reinforcement Learning by Stanford University (Coursera)
-
Reinforcement Learning by DeepLearning.AI
-
Reinforcement Learning by OpenAI
-
Reinforcement Learning by Sutton and Barto (Book)
-
Reinforcement Learning by TensorFlow
-
Reinforcement Learning by PyTorch
-
Reinforcement Learning by Machine Learning Wiki
-
Reinforcement Learning by OpenAI Spinning Up
- Natural Language Processing (NLP): Word embeddings, recurrent neural networks for sequence modeling.
-
Natural Language Processing by Stanford University (Coursera)
-
Natural Language Processing with Deep Learning by DeepLearning.AI
-
Natural Language Processing (NLP) by Fast.ai
-
Natural Language Processing (NLP) by TensorFlow
-
Natural Language Processing (NLP) by PyTorch
-
Natural Language Processing (NLP) by NLTK (Natural Language Toolkit)
-
Natural Language Processing (NLP) by Machine Learning Mastery
-
Natural Language Processing (NLP) by Machine Learning Wiki
-
Natural Language Processing (NLP) by OpenAI Spinning Up
-
Time Series Analysis and Its Applications by Shumway and Stoffer (Book)
-
Practical Time Series Analysis by Aileen Nielsen
-
Time Series Analysis by Stanford University (Coursera)
-
Time Series Analysis by Kaggle
-
Time Series Analysis by Machine Learning Mastery
-
Time Series Analysis by TensorFlow
-
Time Series Analysis by PyTorch
-
Time Series Analysis by Machine Learning Wiki
-
Time Series Analysis by OpenAI Spinning Up
- Autoencoders and Variational Autoencoders (VAEs): Unsupervised learning, dimensionality reduction, generative models.
-
Autoencoders by Stanford University (Coursera)
-
Autoencoders and Variational Autoencoders (VAEs) by DeepLearning.AI
-
Autoencoders by TensorFlow
-
Variational Autoencoders (VAEs) by TensorFlow
-
Autoencoders and Variational Autoencoders (VAEs) by PyTorch
-
Autoencoders and Variational Autoencoders (VAEs) by Machine Learning Mastery
-
Autoencoders and Variational Autoencoders (VAEs) by Machine Learning Wiki
-
Autoencoders and Variational Autoencoders (VAEs) by OpenAI Spinning Up
-
Autoencoders and Variational Autoencoders (VAEs) by Christopher Olah
-
Variational Autoencoders (VAEs) by Carl Doersch
-
Autoencoders and Variational Autoencoders (VAEs) by OpenAI
-
Building Autoencoders in Keras by François Chollet (Keras Blog)
-
Autoencoders and Variational Autoencoders (VAEs) by Google Developers
-
Deep Learning Book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville
-
Autoencoders and Variational Autoencoders (VAEs) by Distill.pub
-
Variational Autoencoders (VAEs) by OpenAI Spinning Up
- Model Interpretability and Explainability: Techniques to interpret and explain deep learning models.
-
Interpretable Machine Learning by Christoph Molnar (Book)
-
Interpretable Machine Learning by Microsoft Research
-
Model Interpretability and Explainability by Google AI
-
Model Interpretability and Explainability by OpenAI
-
Model Interpretability and Explainability by scikit-learn (Python Library)
-
Explainable AI (XAI) by DARPA
-
Model Interpretability and Explainability by Machine Learning Wiki
-
Model Interpretability and Explainability by Towards Data Science
-
Model Interpretability and Explainability by OpenAI Spinning Up
-
A Unified Approach to Interpreting Model Predictions by Marco Tulio Ribeiro, et al.
-
SHAP (SHapley Additive exPlanations) by Lundberg and Lee
-
LIME (Local Interpretable Model-Agnostic Explanations) by Ribeiro, Singh, and Guestrin
-
Anchors: High-Precision Model-Agnostic Explanations by Ribeiro, Singh, and Guestrin
-
Interpretable Deep Learning with Python by Yuriy Guts
-
Explainable AI and Machine Learning Interpretability by IBM Developer
-
Interpretable Machine Learning in Python by Christoph Molnar
-
InterpretML: A Python Library for Model Interpretability by Microsoft
-
Model Interpretability and Explainability by OpenAI Spinning Up
This roadmap is a compilation of mathematical concepts covered in various deep learning resources, including the following:
- "Deep Learning" book by Ian Goodfellow, Yoshua Bengio, and Aaron Courville.
- "Deep Learning with Python, Second Edition" by François Chollet.
- "Grokking Deep Learning" by Andrew Trask.
- "Deep Learning: A Practitioner's Approach" by Josh Patterson and Adam Gibson.
- "Deep Learning for Coders with fastai and PyTorch" by Jeremy Howard and Sylvain Gugger.