Many data-science problems can be formulated as an inverse problem, where the parameters are estimated by minimizing a proper loss function. When complicated black-box models are involved, derivative-free optimization tools are often needed. The ensemble Kalman filter (EnKF) is a particle-based derivative-free Bayesian algorithm originally designed for data assimilation. Recently, it has been applied to inverse problems for computational efficiency. The resulting algorithm, known as ensemble Kalman inversion (EKI), involves running an ensemble of particles with EnKF update rules so they can converge to a minimizer. In this article, we investigate EKI convergence in general nonlinear settings. To improve convergence speed and stability, we consider applying EKI with non-constant step-sizes and covariance inflation. We prove that EKI can hit critical points with finite steps in non-convex settings. We further prove that EKI converges to the global minimizer polynomially fast if the loss function is strongly convex. We verify the analysis presented with numerical experiments on two inverse problems.
翻译:许多数据-科学问题可以作为一个反向问题提出,其中参数可以通过尽量减少适当的损失功能来估计。当涉及到复杂的黑盒模型时,往往需要无衍生的优化工具。共振卡尔曼过滤器(EnKF)是一种最初设计用于数据同化的无粒子衍生衍生物无巴伊西亚算法。最近,它被应用于反向的计算效率问题。由此产生的算法,称为共振卡尔曼反转(EKI),涉及以 EnKF 更新规则运行一个粒子组合,以便它们能与最小化器汇合。在本篇文章中,我们调查一般非线性设置的 EKI 趋同。为了提高趋同速度和稳定性,我们考虑用非一致的步进尺寸和共变通货膨胀来应用EKI 。我们证明EKI 可以在非convex 设置中以有限的步骤达到临界点。我们进一步证明EKI如果损失功能强烈地交错的话,它就会与全球最小化极快。我们用数字实验对两种问题进行了核查。