Zeroth-order optimization is the process of minimizing an objective $f(x)$, given oracle access to evaluations at adaptively chosen inputs $x$. In this paper, we present two simple yet powerful GradientLess Descent (GLD) algorithms that do not rely on an underlying gradient estimate and are numerically stable. We analyze our algorithm from a novel geometric perspective and present a novel analysis that shows convergence within an $\epsilon$-ball of the optimum in $O(kQ\log(n)\log(R/\epsilon))$ evaluations, for {\it any monotone transform} of a smooth and strongly convex objective with latent dimension $k < n$, where the input dimension is $n$, $R$ is the diameter of the input space and $Q$ is the condition number. Our rates are the first of its kind to be both 1) poly-logarithmically dependent on dimensionality and 2) invariant under monotone transformations. We further leverage our geometric perspective to show that our analysis is optimal. Both monotone invariance and its ability to utilize a low latent dimensionality are key to the empirical success of our algorithms, as demonstrated on BBOB and MuJoCo benchmarks.
翻译:零顺序优化是将目标f(x)美元最小化的过程, 给您在适应性选择的投入上获得评价的机会, 给您提供 $x。 在本文中, 我们展示了两种简单而有力的GLD算法, 这些算法不依赖于基底梯度估计值, 且在数值上稳定。 我们从新的几何角度分析了我们的算法, 并提出了一个新颖的分析, 显示在单体变换中, 美元( kçlog( n)\log( R/\ epsilon)) $( 美元) 的最佳组合。 我们进一步利用我们的几何角度来显示我们的分析是最佳的, 其潜在维度为 $ < nk < $, 美元, 美元是输入空间的直径, 美元是条件号。 我们的算法首先显示(1) 多元- 和 2 等量球中, 在单体变换中, 我们进一步利用我们的几何角度来显示我们的分析是最佳的。 单体的单质度, 和 将它作为核心的实验基准, 成功度 。