通过减少内部差异实现多级综合综合斯托克优化 (Multi-Level Composite Stochastic Optimization via Nested Variance Reduction)

We consider multi-level composite optimization problems where each mapping in the composition is the expectation over a family of random smooth mappings or the sum of some finite number of smooth mappings. We present a normalized proximal approximate gradient (NPAG) method where the approximate gradients are obtained via nested stochastic variance reduction. In order to find an approximate stationary point where the expected norm of its gradient mapping is less than $\epsilon$, the total sample complexity of our method is $O(\epsilon^{-3})$ in the expectation case, and $O(N+\sqrt{N}\epsilon^{-2})$ in the finite-sum case where $N$ is the total number of functions across all composition levels. In addition, the dependence of our total sample complexity on the number of composition levels is polynomial, rather than exponential as in previous work.

翻译：我们考虑的是多层次综合优化问题,因为每次绘图在组成中都是对随机平滑绘图组合的预期值,或者对一定数量的平滑绘图总和的预期值。我们提出了一个正常的近似近似梯度(NPAG)方法,通过嵌套的随机差异减少来获得近似梯度。为了找到一个大约的固定点,其梯度绘图的预期标准低于美元,在预期的情况下,我们方法的总样本复杂性是O美元(EpsilonQQQ--3}),在有限总和中,美元(N ⁇ sqrt{N ⁇ epsilonQ ⁇ -2}),其中,美元是所有构成等级的函数总数。此外,我们总样本复杂性对组成级别数量的依赖性是多元的,而不是像以前的工作那样的指数性。