Machine Learning Fundamentals

Probability

• Correlation Matrices: A plotly vizualization of the space of 3x3 correlation matrices using a nice parametrization

Classification

• Naive Bayes: Introduction, derivation and reconciliation of Naive Bayes - a baseline model for classification.

• Linear / Quadratic Discriminant Analysis: Introduction, derivation, properties and examples of LDA/QDA classification

• Logistic Regression: Definitions, Binary and multi-class case, Sigmoid and Softmax functions, Cross-Entropy Loss, Regularization, Examples

• Decision Tree Classifiers: Graph theory, binary rooted trees, impurity functions, minimal cost-complexity pruning

Advanced Regression Techniques

• Linear Regression: A recap of linear regression - a core fundamental of machine learning.

• Local Regression: Local regression is a refinement of linear regression that adapts the model at each point of the prediction.

• Gaussian Process Regression (GPR): An advanced regression technique that produces not only predictions, but also confidence bounds around them.

• Dynamically Controlled Kernel Estimation (DCKE): A combination of local regression, control variates and Gaussian process regression to estimate conditional expectations. The method is model free, data-driven and particularly suited for financial applications.

• Decision Tree Regressors: Decision trees can be used for regression as well

Neural Network Topologies

• Multilayer Perceptrons (MLP): Introduction to the most common form of artificial neural networks (ANN).

• Long-Term-Short-Term-Memory networks (LSTM): Introduction to LSTMs, a popular form of recurrent neural networks (RNNs).

• Network Topology Selection: A methodology to choose a topology for a neural network, e.g. the number of hidden layers and units.

Training Networks & Optimization Techniques

• Newton's Method: A recap of the Newton's method.

• Gradient Descent - Basics: Mathematical foundations and basics of gradient descent.

• Gradient Descent - Advanced [@lapislago]: Some illustrations, background and examples of gradient descent.

• Backpropagation: Derivation of the backpropagation algorithm.

• Adjoint Method: Relationship between backpropagation and the adjoint method.

Basic Examples

• Learning the Sine [@lapislago]: Simple example of how to use keras and tensorflow to learn a curve.

• Learning a 2D function [@lapislago]: A slightly more complex example of how to learn a surface.

Machine Learning & Quantitative Finance

• How deep are financial models?: Learn the pricing function of Black-Scholes and Heston model. Application of network topology selection.

• Neural Network Brain Surgery: Can the difference between the Black-Scholes and the Heston model be visualized as the brains of the networks that learn their pricing function?

• Calibrating Heston [@lapislago]: Learn the calibration function of a Heston model using a neural network.

• Calibrating Hull-White [@lapislago]: Learn the calibration function of a Hull-White model using a neural network.

• Autograd [@lapislago]: A small example on how to automatically differentiate the Black-Scholes pricing formula.

References