普渡大学2020硬核课程《鲁棒机器学习理论》课件与笔记，38讲173页pdf

会员服务 ·

普渡大学2020硬核课程《鲁棒机器学习理论》课件与笔记，38讲173页pdf

2020 年 3 月 28 日 专知

【导读】居家来学习这门硬核课程，来自普渡大学 Stanley Chan 教授的机器学习课程，讲授数学基础、监督学习、对抗鲁棒、学习理论，包含38讲和4大笔记173页pdf。

课程地址：

https://engineering.purdue.edu/ChanGroup/ECE595/index.html

课程内容：

数学基础：矩阵、向量、Lp范数、范数的几何、对称性、正确定性、特征分解。无约束最优化，graident下降法，凸函数，拉格朗日乘子，线性最小二乘法。概率空间，随机变量，联合分布，多维高斯。
线性分类器：线性判别分析，分离超平面，多类分类，贝叶斯决策规则，贝叶斯决策规则几何，线性回归，逻辑回归，感知机算法，支持向量机，非线性变换。
鲁棒性：对抗性攻击、定向攻击和非定向攻击、最小距离攻击、最大允许攻击、基于规则的攻击。通过纳微扰。支持向量机的鲁棒性。
学习理论：偏差和方差，训练和测试，泛化，PAC框架，Hoeffding不等式，VC维。

参考书籍：

Pattern Classification, by Duda, Hart and Stork, Wiley-Interscience; 2 edition, 2000.
Learning from Data, by Abu-Mostafa, Magdon-Ismail and Lin, AMLBook, 2012.
Elements of Statistical Learning, by Hastie, Tibshirani and Friedman, Springer, 2 edition, 2009.
Pattern Recognition and Machine Learning, by Bishop, Springer, 2006.

讲者： Stanley Chan 教授

https://engineering.purdue.edu/ChanGroup/stanleychan.html

课程目标：

您将能够应用基本的线性代数、概率和优化工具来解决机器学习问题

•你将了解一般监督学习方法的原理，并能评论它们的优缺点。

•你会知道处理数据不确定性的方法。

• 您将能够使用学习理论的概念运行基本的诊断。

• 您将获得机器学习算法编程的实际经验。

课程课件：

Week 1: Lecture 00 (PDF) 2020-01-13 Course overview (Video)

Part 1: Mathematical Background

通过线性回归的案例研究来理解机器学习管道的组成部分

Week 1: Lecture 01 (PDF) 2020-01-15 Linear Regression 1: Concepts and Geometry
Week 1: Lecture 02 (PDF) 2020-01-17 Linear Regression 2: Ridge and LASSO Regularization
Week 2: No Class: 2020-01-20 MLK Day
Week 2: Lecture 03 (PDF) 2020-01-22 Linear Regression 3: Nonlinear transform, Kernel trick (Video 1) (Video 2)
Week 2: Lecture 04 (PDF) 2020-01-24 Optimization 1: Optimality, Convexity, and Constraints (Video 1) (Video 2) (Video 3)
Week 3: Lecture 05 (PDF) 2020-01-27 Optimization 2: Gradient Descent and Stochastic Gradient Descent (Video 1) (Video 2)

Part 2: Supervised Learning Methods 了解常用的监督学习方法背后的原理

Week 3: Lecture 06 (PDF) 2020-01-29 Linear Separability (Video 1) (Video 2) (Video 3)
Week 3: Lecture 07 (PDF) 2020-01-31 Feature Extraction 1: Principal Component Analysis (Video 1) (Video 2)
Week 4: Lecture 08 (PDF) 2020-02-03 Feature Extraction 2: Hand-Crafted and Deep Features (Video 1) (Video 2) (Video 3)
Week 4: Lecture 09 (PDF) 2020-02-05 Generative Method 1: Bayesian Decision Rule (Video 1) (Video 2) (Video 3)
Week 4: Lecture 10 (PDF) 2020-02-07 Generative Method 2: Minimum Probility of Error Rule (Video)
Week 5: Lecture 11 (PDF) 2020-02-10 Generative Method 3: Estimating Parameters (Video 1) (Video 2)
Week 5: Lecture 12 (PDF) 2020-02-12 Generative Method 4: Bayesian Priors (Video)
Week 5: Lecture 13 (PDF) 2020-02-14 Generative Method 5: Connecting Bayesian Decisions with Linear Regression (Video)
Week 6: Lecture 14 (PDF) 2020-02-17 Logistic Regression 1: Loss and Convexity (Video)
Week 6: Lecture 15 (PDF) 2020-02-19 Logistic Regression 2: Algorithms and Interpretations (Video)
Week 6: Lecture 16 (PDF) 2020-02-21 Perceptron 1: Definitions and Concepts (Video)
Week 7: Lecture 17 (PDF) 2020-02-24 Perceptron 2: Algorithm and Prooperties (Video)
Week 7: Lecture 18 (PDF) 2020-02-26 Multi-Layer Perceptron and Back Propagation (Video)
Week 7: Lecture 19 (PDF) 2020-02-28 Support Vector Machine 1: Introduction (Video)
Week 8: Lecture 20 (PDF) 2020-03-02 Support Vector Machine 2: Duality (Video)
Week 8: Lecture 21 (PDF) 2020-03-04 Support Vector Machine 3: Soft SVM and Kernel Trick (Video)

Part 3: Learning Theory 了解机器学习算法的理论极限

Week 8: Lecture 22 (PDF) 2020-03-06 Is Learning Feasible? (Video)
Week 9: Lecture 23 (PDF) 2020-03-09 Probability Inequality (Video)
Week 9: Special Announcement about COVID-19. (PDF) 2020-03-11 (Video)
Week 9: Midterm (take home) See Instructions
Week 10: Lecture 24 (PDF) (Animated PDF) 2020-03-23 Probably Approximately Correct (Video 1) (Video 2)
Week 10: Lecture 25 (PDF) (Animated PDF) 2020-03-25 Generalization (Video 1) (Video 2)
Week 10: Lecture 26 (PDF) (Animated PDF) 2020-03-27 Growth Function (Video 1) (Video 2)
Week 11: Lecture 27 (PDF) (Animated PDF) 2020-03-30 VC Dimension (Video 1) (Video 2)
Week 11: Lecture 28 (PDF) (Animated PDF) 2020-04-01 Sample and Model Complexity (Video 1) (Video 2)
Week 11: Lecture 29 (PDF) (Animated PDF) 2020-04-03 Bias and Variance (Video 1) (Video 2)
Week 12: Lecture 30 (PDF) (Animated PDF) 2020-04-06 Overfit (Video 1)(Video 2)
Week 12: Lecture 31 (PDF) (Animated PDF) 2020-04-08 Regularization (Video 1) (Video 2)(Video 3)
Week 12: Lecture 32 (PDF) (Animated PDF) 2020-04-10 Validation (Video 1) (Video 2)

Part 4: Robust Machine Learning 了解机器学习算法的鲁棒性

Week 13: Lecture 33 2020-04-13 Overview of Adversarial Attacks (Video)
Week 13: Lecture 34 2020-04-15 Minimum Distance Attack (Video)
Week 13: Lecture 35 2020-04-17 Maximum Loss Attack and Regularized Attack (Video)
Week 14: Lecture 36 2020-04-20 Defending Adversarial Attacks (Video)
Week 14: Lecture 37 2020-04-22 Robustness and Accuracy Trade-Off (Video)
Week 14: Lecture 38 2020-04-24 Conclusion: Practical Advices (Video)

课程笔记：