Variational inference transforms posterior inference into parametric optimization thereby enabling the use of latent variable models where otherwise impractical. However, variational inference can be finicky when different variational parameters control variables that are strongly correlated under the model. Traditional natural gradients based on the variational approximation fail to correct for correlations when the approximation is not the true posterior. To address this, we construct a new natural gradient called the Variational Predictive Natural Gradient (VPNG). Unlike traditional natural gradients for variational inference, this natural gradient accounts for the relationship between model parameters and variational parameters. We demonstrate the insight with a simple example as well as the empirical value on a classification task, a deep generative model of images, and probabilistic matrix factorization for recommendation.
翻译:变式推断将后推推力转化为参数优化,从而能够在不切实际的情况下使用潜伏变量模型。然而,当不同的变异参数控制变量在模型下具有强烈关联性时,变异推力可能会是纤维化的。基于变异近似的传统自然梯度在近似不是真实的后推力时无法纠正相关关系。为此,我们建造了一个新的自然梯度,称为变异预测性自然梯度(VPNG)。与传统的变异推力自然梯度不同,这种自然梯度计算模型参数和变异参数之间的关系。我们用简单的例子以及分类任务的经验价值、图像的深基因化模型和推荐的概率矩阵要素等来显示洞察力。