大多数概率模型中, 计算后验边际或准确计算归一化常数都是很困难的. 变分推断(variational inference)是一个近似计算这两者的框架. 变分推断把推断看作优化问题: 我们尝试根据某种距离度量来寻找一个与真实后验尽可能接近的分布(或者类似分布的表示).

VIP内容

图神经网络(GNN)已经在广泛的应用领域取得了良好的效果。大多数对GNN的实证研究都直接将观察到的图作为输入,假设观察到的结构完美地描述了节点之间准确完整的关系。然而,现实世界中的图不可避免地是嘈杂的或不完整的,这甚至会恶化图表示的质量。本文从信息论的角度提出了一种新的变分信息瓶颈引导的图结构学习框架VIB-GSL。VIB-GSL提出了图结构学习的信息瓶颈(Information Bottleneck, IB)原则,为挖掘底层任务相关关系提供了一个更优雅和通用的框架。VIB-GSL学习了一种信息丰富的压缩图结构,为特定的下游任务提取可操作的信息。VIB-GSL对不规则图数据进行变分逼近,形成易处理的IB目标函数,有利于训练的稳定性。大量的实验结果表明,VIB-GSL具有良好的有效性和鲁棒性。

https://www.zhuanzhi.ai/paper/8f506a32b4b05f9ea8a5d651eb1b27f1

成为VIP会员查看完整内容
0
8

最新内容

This study presents PRISM, a probabilistic simplex component analysis approach to identifying the vertices of a data-circumscribing simplex from data. The problem has a rich variety of applications, the most notable being hyperspectral unmixing in remote sensing and non-negative matrix factorization in machine learning. PRISM uses a simple probabilistic model, namely, uniform simplex data distribution and additive Gaussian noise, and it carries out inference by maximum likelihood. The inference model is sound in the sense that the vertices are provably identifiable under some assumptions, and it suggests that PRISM can be effective in combating noise when the number of data points is large. PRISM has strong, but hidden, relationships with simplex volume minimization, a powerful geometric approach for the same problem. We study these fundamental aspects, and we also consider algorithmic schemes based on importance sampling and variational inference. In particular, the variational inference scheme is shown to resemble a matrix factorization problem with a special regularizer, which draws an interesting connection to the matrix factorization approach. Numerical results are provided to demonstrate the potential of PRISM.

0
0
下载
预览

最新论文

This study presents PRISM, a probabilistic simplex component analysis approach to identifying the vertices of a data-circumscribing simplex from data. The problem has a rich variety of applications, the most notable being hyperspectral unmixing in remote sensing and non-negative matrix factorization in machine learning. PRISM uses a simple probabilistic model, namely, uniform simplex data distribution and additive Gaussian noise, and it carries out inference by maximum likelihood. The inference model is sound in the sense that the vertices are provably identifiable under some assumptions, and it suggests that PRISM can be effective in combating noise when the number of data points is large. PRISM has strong, but hidden, relationships with simplex volume minimization, a powerful geometric approach for the same problem. We study these fundamental aspects, and we also consider algorithmic schemes based on importance sampling and variational inference. In particular, the variational inference scheme is shown to resemble a matrix factorization problem with a special regularizer, which draws an interesting connection to the matrix factorization approach. Numerical results are provided to demonstrate the potential of PRISM.

0
0
下载
预览
参考链接
Top
微信扫码咨询专知VIP会员