VIP内容

本课程(以及本教材)的目标是为最广泛使用的学习架构展示学习理论的旧成果和新成果。本课程面向的是理论导向型的学生,以及那些想要获得基本数学理解的学生,这些学生在机器学习和相关领域中使用了大量的学习方法,如计算机视觉或自然语言处理。为了证明从第一性原理得出的许多结果,将作出特别的努力,同时使阐明尽可能简单。这将自然导致选择的关键结果,在简单但相关的实例中展示学习理论的重要概念。在没有证明的情况下,也将给出一些一般的结果。当然,第一性原理的概念是主观的,我将假定有良好的线性代数、概率论和微分的知识。

https://www.di.ens.fr/~fbach/learning_theory_class/index.html

目录内容:

无线数据学习 Learning with infinite data (population setting) -Decision theory (loss, risk, optimal predictors) -Decomposition of excess risk into approximation and estimation errors -No free lunch theorems -Basic notions of concentration inequalities (MacDiarmid, Hoeffding, Bernstein) 线性最小二乘回归 Liner Least-squares regression -Guarantees in the fixed design settings (simple in closed-form) -Ridge regression: dimension independent bounds -Guarantees in the random design settings -Lower bound of performance 经验风险最小化 Empirical risk minimization -Convexification of the risk -Risk decomposition -Estimation error: finite number of hypotheses and covering numbers -Rademacher complexity -Penalized problems 机器学习的优化 Optimization for machine learning -Gradient descent -Stochastic gradient descent -Generalization bounds through stochastic gradient descent 局部平均技术 Local averaging techniques -Partition estimators -Nadaraya-Watson estimators -K-nearest-neighbors -Universal consistency 核方法 Kernel methods -Kernels and representer theorems -Algorithms -Analysis of well-specified models -Sharp analysis of ridge regression -Universal consistency 模型选择 Model selection -L0 penalty -L1 penalty -High-dimensional estimation 神经网络 Neural networks -Single hidden layer neural networks

  • Estimation error
  • Approximation properties and universality 特别主题 Special topics -Generalization/optimization properties of infinitely wide neural networks -Double descent
成为VIP会员查看完整内容
0
45
Top