Due to the hierarchical structure of many machine learning problems, bilevel programming is becoming more and more important recently, however, the complicated correlation between the inner and outer problem makes it extremely challenging to solve. Although several intuitive algorithms based on the automatic differentiation have been proposed and obtained success in some applications, not much attention has been paid to finding the optimal formulation of the bilevel model. Whether there exists a better formulation is still an open problem. In this paper, we propose an improved bilevel model which converges faster and better compared to the current formulation. We provide theoretical guarantee and evaluation results over two tasks: Data Hyper-Cleaning and Hyper Representation Learning. The empirical results show that our model outperforms the current bilevel model with a great margin. \emph{This is a concurrent work with \citet{liu2020generic} and we submitted to ICML 2020. Now we put it on the arxiv for record.}
翻译:由于许多机器学习问题的等级结构,最近双级编程变得越来越重要,但是,最近,内外部问题之间的复杂关联使得它极难解决。虽然已经提出了若干基于自动区分的直观算法,并在一些应用中取得了成功,但对于找到双级模型的最佳拟订方法没有多少注意。是否有更好的配方仍然是一个尚未解决的问题。在本文件中,我们提出了一个更好的双级模型,与目前的配方相比,速度更快、更好。我们在两个任务上提供了理论保障和评价结果:数据超清晰度和超代表性学习。经验结果显示,我们的模型比目前的双级模型大差强。\emph{这是与\citet{liu2020genic}并同时开展的工作,我们提交给了ICML 2020。我们现在把它放在Arxiv上备案。}