Skip to content

Latest commit

 

History

History
 
 

Lectures

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
 
 
 
 
 
 

Lectures

The lectures consist of basic courses and advanced courses.

In basic lectures, we aimed to help learners completely understand the computer system architecture that supports deep learning, and learn the system design under the full life cycle of deep learning through practical problems.

In advanced lectures, we introduced cutting-edge systems and artificial intelligence research work, including AI for Systems and Systems for AI, to help learners better find and define meaningful research questions.

Basic Courses

Course No. Lecture Name Remarks Download
1 Introduction Overview and system/AI basics PPT Show
2 System perspective of Systems for AI Systems for AI: a historic view; Fundamentals of neural networks; Fundamentals of Systems for AI PPT Show
3 Computation frameworks for DNN Backprop and AD, Tensor, DAG, Execution graph.
Papers and systems: PyTorch, TensorFlow
PPT Show
4 Computer architecture for Matrix computation Matrix computation, CPU/SIMD, GPGPU, ASIC/TPU
Papers and systems: Blas, TPU
PPT Show
5 Distributed training algorithms Data parallelism, model parallelism, distributed SGD
Papers and systems: PipeDream
PPT Show
6 Distributed training systems MPI, parameter servers, all-reduce, RDMA
Papers and systems: Horovod
PPT Show
7 Scheduling and resource management system Running dnn job on cluster: container, resource allocation, scheduling
Papers and systems: Kubeflow, OpenPAI,Gandiva, HiveD
PPT Show
8 Inference systems Efficiency, latency, throughput, and deployment
Papers and systems: TensorRT, TensorflowLite, ONNX
PPT Show

Advanced Courses

Course No. Course Name Remarks
9 Computation graph compilation and optimization IR, sub-graph pattern match, Matrix multiplication and memory optimization
Papers and systems: XLA, MLIR, TVM, NNFusion
PPT Show
10 Efficiency via compression and sparsity Model compression, Sparsity, Pruning PPT Show
11 AutoML systems Hyper parameter tuning, NAS
Papers and systems: Hyperband, SMAC, ENAS, AutoKeras, NNI
PPT Show
12 Reinforcement learning systems Theory of RL, systems for RL
Papers and systems: AC3, RLlib, AlphaZero
PPT Show
13 Security and Privacy Federated learning, security, privacy
Papers and systems: DeepFake
PPT Show
14 AI for systems AI for traditional systems problems, for system algorithms
Papers and systems: Learned Indexes, Learned query path
PPT Show