损失函数，在AI中亦称呼距离函数，度量函数。此处的距离代表的是抽象性的，代表真实数据与预测数据之间的误差。损失函数（loss function）是用来估量你模型的预测值f(x)与真实值Y的不一致程度，它是一个非负实值函数,通常使用L(Y, f(x))来表示，损失函数越小，模型的鲁棒性就越好。损失函数是经验风险函数的核心部分，也是结构风险函数重要组成部分。
VIP内容
近日，计算机科学与工程学院2021级硕士研究生潘尔林的论文“Multi-view Contrastive Graph Clustering”被第35届神经信息处理系统大会（Conference on Neural Information Processing Systems，NeurIPS）成功录用。潘尔林为论文第一作者，其导师康昭副教授为通讯作者。NeurIPS会议每年举办一次，作为公认的人工智能、机器学习领域国际顶级会议，也是中国计算机学会（CCF）推荐A类会议，具有广泛而深远的国际影响力，受到来自学术界和工业界的广泛关注。
多视图图数据样例
该篇论文是潘尔林同学在本科四年级期间完成的，文中提出了一种在多视图属性图数据上的聚类算法。该任务是从诸多现实问题中抽象出来的，具有较高的实际价值，目前国际上对该问题的研究还处在起步阶段。作者没用使用千篇一律的深度神经网络大模型，而是独辟蹊径，基于传统的浅层模型实现了对深度学习方法在该任务上性能的超越，该方法不依赖于大量的训练数据和高性能的硬件，方便人工智能的实际应用。文中首先使用了图滤波的方法从原始数据中得到更加平滑的数据特征表示，滤除高频的噪声，使得到的特征表示更加有利于后继任务。接着，该文利用数据的自表达性质以及一套自适应的权重分配机制，从原始的多图中学习得到一个高质量图。最后，受到自监督学习的启发，文中提出图上的对比学习正则项，拉进相似的数据点，提高图的聚类亲和性。该文指出了一个非常有潜力的方向，为应对深度学习的挑战打开了新的研究途径，具有巨大的研究潜力。
https://www.zhuanzhi.ai/paper/830a24619d09dec0dbf226cd42023ada
成为VIP会员查看完整内容
Multi-view Contrastive Graph Clustering
热门内容
最新内容
In this paper, we study the sharpness of a deep learning (DL) loss landscape around local minima in order to reveal systematic mechanisms underlying the generalization abilities of DL models. Our analysis is performed across varying network and optimizer hyper-parameters, and involves a rich family of different sharpness measures. We compare these measures and show that the low-pass filter-based measure exhibits the highest correlation with the generalization abilities of DL models, has high robustness to both data and label noise, and furthermore can track the double descent behavior for neural networks. We next derive the optimization algorithm, relying on the low-pass filter (LPF), that actively searches the flat regions in the DL optimization landscape using SGD-like procedure. The update of the proposed algorithm, that we call LPF-SGD, is determined by the gradient of the convolution of the filter kernel with the loss function and can be efficiently computed using MC sampling. We empirically show that our algorithm achieves superior generalization performance compared to the common DL training strategies. On the theoretical front, we prove that LPF-SGD converges to a better optimal point with smaller generalization error than SGD.
最新论文
In this paper, we study the sharpness of a deep learning (DL) loss landscape around local minima in order to reveal systematic mechanisms underlying the generalization abilities of DL models. Our analysis is performed across varying network and optimizer hyper-parameters, and involves a rich family of different sharpness measures. We compare these measures and show that the low-pass filter-based measure exhibits the highest correlation with the generalization abilities of DL models, has high robustness to both data and label noise, and furthermore can track the double descent behavior for neural networks. We next derive the optimization algorithm, relying on the low-pass filter (LPF), that actively searches the flat regions in the DL optimization landscape using SGD-like procedure. The update of the proposed algorithm, that we call LPF-SGD, is determined by the gradient of the convolution of the filter kernel with the loss function and can be efficiently computed using MC sampling. We empirically show that our algorithm achieves superior generalization performance compared to the common DL training strategies. On the theoretical front, we prove that LPF-SGD converges to a better optimal point with smaller generalization error than SGD.
子主题
微信扫码咨询专知VIP会员