Recent work in unsupervised learning has focused on efficient inference and learning in latent variables models. Training these models by maximizing the evidence (marginal likelihood) is typically intractable. Thus, a common approximation is to maximize the Evidence Lower BOund (ELBO) instead. Variational autoencoders (VAE) are a powerful and widely-used class of generative models that optimize the ELBO efficiently for large datasets. However, the VAE's default Gaussian choice for the prior imposes a strong constraint on its ability to represent the true posterior, thereby degrading overall performance. A Gaussian mixture model (GMM) would be a richer prior, but cannot be handled efficiently within the VAE framework because of the intractability of the Kullback-Leibler divergence for GMMs. We deviate from the common VAE framework in favor of one with an analytical solution for Gaussian mixture prior. To perform efficient inference for GMM priors, we introduce a new constrained objective based on the Cauchy-Schwarz divergence, which can be computed analytically for GMMs. This new objective allows us to incorporate richer, multi-modal priors into the autoencoding framework. We provide empirical studies on a range of datasets and show that our objective improves upon variational auto-encoding models in density estimation, unsupervised clustering, semi-supervised learning, and face analysis.
翻译:在不受监督的近期学习中,最近的工作侧重于对潜在变量模型的高效推断和学习。通过最大限度地增加证据(边际可能性)来培训这些模型通常是难以解决的。因此,一个共同的近似点是最大限度地增加证据下库(ELBO ) 。 动态自动读数器(VAE)是一个强大和广泛使用的基因模型类别,为大型数据集高效优化ELBO。然而,VAE对前一变量的默认Gausian选择对其代表真实的后背模型的能力施加了强烈的制约,从而降低了总体性能。高斯混合模型(GMMM)将更丰富,但无法在VAE框架内高效处理,因为Kullback-Lebeter差异对于GMMM(GMMM)来说是不可忽视的。我们偏离了通用VAE框架,而前者则倾向于对高斯混合混合物有一个分析解决方案。为了高效地推断GMOM(GM)之前的面面模型,我们引入了一个新的限制目标。我们可以在CASS-S-Schwarz 差异模型上进行新的分析性目标。