This paper proposes Dirichlet Variational Autoencoder (DirVAE) using a Dirichlet prior for a continuous latent variable that exhibits the characteristic of the categorical probabilities. To infer the parameters of DirVAE, we utilize the stochastic gradient method by approximating the Gamma distribution, which is a component of the Dirichlet distribution, with the inverse Gamma CDF approximation. Additionally, we reshape the component collapsing issue by investigating two problem sources, which are decoder weight collapsing and latent value collapsing, and we show that DirVAE has no component collapsing; while Gaussian VAE exhibits the decoder weight collapsing and Stick-Breaking VAE shows the latent value collapsing. The experimental results show that 1) DirVAE models the latent representation result with the best log-likelihood compared to the baselines; and 2) DirVAE produces more interpretable latent values with no collapsing issues which the baseline models suffer from. Also, we show that the learned latent representation from the DirVAE achieves the best classification accuracy in the semi-supervised and the supervised classification tasks on MNIST, OMNIGLOT, and SVHN compared to the baseline VAEs. Finally, we demonstrated that the DirVAE augmented topic models show better performances in most cases.
翻译:本文建议 Dirrichlet Variational Autencoder (DirVAE) 使用 Dirrichlet 前方的隐性变量来显示绝对概率的特性。 要推断DirVAE 的参数, 我们使用与 Gamma 分布相近的蒸馏梯度法( Gamma 分布是Drichlet 分布的一个部分, 与 Gamma CDF 近似值相反 ) 。 此外, 我们通过调查两个问题源( 即 解码器重量崩溃和潜值崩溃) 来重塑组成部分的崩溃问题, 我们显示 DirVAE 没有拆散部件; Gaussian VAE 显示解码器重量崩溃和粘粘粘粘粘粘粘粘粘附VAE 显示潜在值的崩溃。 实验结果表明:(1) DirVAE 模型的潜性表示潜在代表值与基准值相比为最佳的对数值; 2) DirVAE 生成了更易解释的潜值,而基线模型不会遇到任何问题。 另外, 我们显示DirVVE 从 DirVE 所学到的潜在代表在半MA- VOMT 和监管中, 展示了我们最终展示了最佳的基线任务。