Awesome Optimization Courses

Welcome to the "Awesome Optimization Courses" repository! This is a curated list of (mostly) free and open educational resources for mathematical optimization.

This list tries to cover topics on discrete and combinatorial optimization, operations research, linear and nonlinear programming, integer programming, constraint programming, convex optimization, continuous optimization, or unconstrained optimization, you'll find valuable resources here to enhance your understanding of these subjects.

Table of Contents

1. Video Lectures and Courses
   1. Convex Optimization
   2. Discrete, Combinatorial, and Integer Optimization
   3. Operations Research
   4. Constraint Programming
   5. Continuous Optimization
   6. Stochastic Optimization and Control
2. Books and Lecture Notes
   1. Convex Optimization
   2. Operations Research
   3. Constraint Programming
   4. Combinatorial Optimization
   5. Stochastic Optimization
3. Solvers, Tools, and Libraries
   1. Licensed/Commercial Solvers
   2. Free Solvers
   3. Tools and Libraries
4. Other Resources
   1. Repositories
   2. Conferences
5. Contribution Guidelines

Video Lectures and Courses

Convex Optimization

• Stanford: Convex optimization I by Stephen Boyd: (YouTube) (Course Website) (edX)
  Concentrates on recognizing and solving convex optimization problems that arise in engineering. Convex sets, functions, and optimization problems. Basics of convex analysis. Least-squares, linear and quadratic programs, semidefinite programming, minimax, extremal volume, and other problems. Optimality conditions, duality theory, theorems of alternative, and applications. Interiorpoint methods. Applications to signal processing, control, digital and analog circuit design, computational geometry, statistics, and mechanical engineering.

• Stanford: Convex optimization II by Stephen Boyd: (YouTube) (Course Website) (edX)
  Continuation of Convex Optimization I. Subgradient, cutting-plane, and ellipsoid methods. Decentralized convex optimization via primal and dual decomposition. Alternating projections. Exploiting problem structure in implementation. Convex relaxations of hard problems, and global optimization via branch & bound. Robust optimization. Selected applications in areas such as control, circuit design, signal processing, and communications. Course requirements include a substantial project.

• CMU: Convex Optimization (Fall 2018): (YouTube) by Ryan Tibshirani

  • Fall 2019
  • Fall 2016 (YouTube)

• Convex Optimization by Ahmad Bazzi (YouTube Playlist) + CVXPy tutorials

• Convex Optimization - NPTEL by Joydeep Dutta

Discrete, Combinatorial, and Integer Optimization

• U Wisconsin-Madison: Integer Optimization - ISyE/Math/CS 728 by Alberto Del Pia

  • Spring 2023: Syllabus | Slides | YouTube | Website
  • Spring 2020: YouTube

• A Second Course in Algorithms (Stanford CS261, Winter 2016) by Tim Roughgarden

• Combinatorial Optimization Course by Constantine Caramanis at UT Austin

• CSE 550: Combinatorial Optimization and Intractability by Joshua J. Daymude from Arizona State University

• U Warwick: MA252 Combinatorial Optimization by Jonathan Noel

• Coursera: Discrete optimization (series) - The University of Melbourne

  • Discrete Optimization (YouTube version)
  • Basic Modeling for Discrete Optimization
  • Advanced modeling for discrete optimization
  • Solving algorithms for discrete optimization

• Discrete Optimization, MSRI Berkeley by Shmuel Onn

• Texas A&M: ISEN 668: Integer Programming (partial) by Sergiy Butenko

• U Illinois - Urbana Champaign: IE511 - Integer Programming by James Davis

• Overview of Linear and Mixed Integer Programming YouTube Series by Mike Wagner

Operations Research

• Penn State: Math484 Linear Programming - Summer 2020 by Wen Shen

  • 2016 version: YouTube

• U Wisconsin-Madison: Linear Optimization - ISyE/Math/CS/Stat 525 (Fall 2021) by Alberto Del Pia

• Texas A&M: ISEN 622: Linear Programming by Sergiy Butenko

• Colorado State U: Math 510 - Linear Programming and Network Flows (Fall 2020) by Henry Adams

• Linear Programming YouTube Series by Shokoufeh Mirzaei

• Advanced Operations Research - NPTEL by G. Srinivasan

• Coursera: Operations Research (series) - National Taiwan University

  • Operations Research (1): Models and applications
  • Operations Research (2): Optimization algorithms
  • Operations Research (3): Theory

• Optimization Techniques/Operation Research Playlist by MathPod Channel

• Operations Research I & II by Dedy Suryadi

  • OR I (YouTube)
  • OR II (YouTube)

Constraint Programming

• An Introduction To Constraint Programming - Jacob Allen

• CPAIOR 2020 Master Class: Constraint Programming

• EdX: Constraint Programming course - UCLouvain (LouvainX):
  Understand the constraint programming paradigm. Design and implement a modern constraint programming library. Model using the constraint programming. Extend the solver with new global constraints. Design custom and black-box searches. Approach Scheduling and Vehicle Routing problems with constraint programming.

Optimization Algorithms

• Optimization Algorithms by Constantine Caramanis at UT Austin

  • additional lectures

• Optimization Methods for Machine Learning and Engineering (KIT Winter Term 20/21) by Julius Pfrommer

  • slides and recordings

• Dimitri Bertsekas's Videos on Dynamic Programming, Reinforcement Learning, etc.

Continuous Optimization

• Mathified YouTube Channle

  1. Unconstrained Optimization Lecture
  2. Constrained Optimization Lecture I
  3. Constrained Optimization Lecture II

• Arizona Math Camp: Optimization
  Local and global optimization. Unconstrained and constrained optimization. Solution function and value function. Implicit Function Theorem. Envelope Theorem. KKT conditions. Kuhn-Tucker Theorem.

• Optimization - NPTEL by A. Goswami & Debjani Chakraborty

Stochastic Optimization and Control

• Basic Course on Stochastic Programming from Instituto de Matemática Pura e Aplicada

• Stochastic Programming by Anthony Papavasiliou

• CTU Prague: Optimal and robust control

• CMU Robotic Exploration Lab: Optimal Control 2022

Books and Lecture notes

Convex Optimization

• Convex Optimization by Boyd and Vandenberghe - PDF
• Convex Optimization Theory by Dimitri P. Bertsekas - PDF

Operations Research

• Operations Research An Introduction by Hamdy A. Taha - Pearson
• Introduction to Operations Research by Frederick Hillier and Gerald Lieberman - McGraw Hill
• Julia Programming for Operations Research by Changhyun Kwon - PDF - code
• Mathematical Programming and Operations Research: Modeling, Algorithms, and Complexity. Examples in Python and Julia. Edited by Robert Hildebrand - PDF
• A First Course in Linear Optimization by Jon Lee - PDF
• Decomposition Techniques in Mathematical Programming by Conejo , Castillo , Mínguez , and García-Bertrand - Springer
• Algorithms for Optimization by Mykel J. Kochenderfer and Tim A. Wheeler - PDF
• Model Building in Mathematical Programming - Introductory modeling book by H. Paul Williams - Wiley

Constraint Programming

• Handbook of Constraint Programming by F. Rossi, Peter van Beek and Toby Walsh - Amazon

Combinatorial Optimization

• Combinatorial Optimization: Algorithms and Complexity by by Christos H. Papadimitriou and Kenneth Steiglitz - Amazon
• Combinatorial Optimization: Theory and Algorithms by Bernhard Korte and Jens Vygen - Springer
• A First Course in Combinatorial Optimization by Jon Lee - Amazon

Stochastic Optimization

• Lectures on Stochastic Programming Modeling and Theory (SIAM) - by Shapiro, Dentcheva, and Ruszczynski - PDF
• Introductory Lectures on Stochastic Optimization by John C. Duchi - PDF

Check out More of Prof. Bertsekas's Books

Solvers, Tools, and Libraries

Solvers

Licensed/Commercial Solvers

• CPLEX - IBM
• Gurobi - Gurobi
• FICO Xpress - FICO
• LocalSolver - LocalSolver
• Knitro - Artelys
• Mosek - Mosek
• Concorde - UWaterloo

Free Solvers

• CBC - COIN-OR Branch and Cut
• GLPK - GNU Linear Programming Kit
• HiGHS - High Performance Software for Linear Optimization
• lp_solve - (Mixed Integer) Linear Programming solver
• SCIP - Solving Constraint Integer Programs

Benchmarks

• Hans Mittelmann's Benchmarks

Tools and Libraries

• Google OR-Tools | examples | repository
• COIN-OR | repository
• MiniZinc - Constraint modeling language.
• GAMS - High-level modeling system.
• AMPL - Algebraic modeling language, both linear/integer and constraint programming.
• JuMP - Open source modeling language for Julia.
• Pyomo - Open source modeling library for Python.
• PuLP - Open source modeling library for Python.
• Scipy Optimize

Other Resources

Repositories

• Open Optimization:
  a platform for sharing and collaborating on open educational resources for optimization related courses.
• Introduction to Linear Programming with Python:
  Introduction to Linear Programming with Python
• cvxpy:
  A Python-embedded modeling language for convex optimization problems.
• MIPLearn:
  Framework for solving discrete optimization problems using a combination of Mixed-Integer Linear Programming (MIP) and Machine Learning (ML)
• or-gym:
  nvironments for OR and RL Research
• ecole:
  Extensible Combinatorial Optimization Learning Environments
• pyconcorde:
  Python wrapper around the Concorde TSP solver
• BOTorch:
  Bayesian optimization in PyTorch
• EvoTorch:
  Advanced evolutionary computation library built directly on top of PyTorch, created at NNAISENSE.
• TorchOpt:
  efficient library for differentiable optimization built upon PyTorch.

Conferences

• CPAIOR - Integration of Constraint Programming, Artificial Intelligence, and Operations Research.
• CP Conference - Principles and Practice of Constraint Programming.
• INFORMS - Institute for Operations Research and the Management Sciences.
• CORS - Canadian Operational Research Society

Contribution Guidelines

We welcome contributions to this repository. If you have a course or resource that you'd like to add, please follow these guidelines:

1. Make a pull request, adding the course to the relevant section with a brief description and a link to the resource.
2. Use a consistent format for entries to maintain readability.

Thank you for your contributions to making this repository a valuable resource for optimization enthusiasts in the academic community!

TODO

1. add missing topics *
   • numerical optimization/methods
   • bayesian optimization
   • meta-heuristics
2. add books and lecture notes
   • open access books (at least one for each topic)
3. add github repos
4. complete solver/software list
5. add short description
6. better fomatting
   • tables instead of bullets