Graph Neural Networks (graph NNs) are a promising deep learning approach for analyzing graph-structured data. However, it is known that they do not improve (or sometimes worsen) their predictive performance as we pile up many layers and add non-lineality. To tackle this problem, we investigate the expressive power of graph NNs via their asymptotic behaviors as the layer size tends to infinity. Our strategy is to generalize the forward propagation of a Graph Convolutional Network (GCN), which is a popular graph NN variant, as a specific dynamical system. In the case of a GCN, we show that when its weights satisfy the conditions determined by the spectra of the (augmented) normalized Laplacian, its output exponentially approaches the set of signals that carry information of the connected components and node degrees only for distinguishing nodes. Our theory enables us to relate the expressive power of GCNs with the topological information of the underlying graphs inherent in the graph spectra. To demonstrate this, we characterize the asymptotic behavior of GCNs on the Erd\H{o}s -- R\'{e}nyi graph. We show that when the Erd\H{o}s -- R\'{e}nyi graph is sufficiently dense and large, a broad range of GCNs on it suffers from the "information loss" in the limit of infinite layers with high probability. Based on the theory, we provide a principled guideline for weight normalization of graph NNs. We experimentally confirm that the proposed weight scaling enhances the predictive performance of GCNs in real data. Code is available at https://github.com/delta2323/gnn-asymptotics.
翻译:内建网络(GG NNs) 是分析图形结构数据的一个很有希望的深层次学习方法。 然而, 当我们堆积多层并增加非线性时, 人们知道它们并没有改善( 有时恶化)它们的预测性能。 为了解决这个问题, 我们调查了图形 NNPs 的表情力量, 因为它的表层大小往往不尽如人意。 我们的策略是将图23 平流网络(GCN) 的前向传播( GCN) 是一个流行的图形 NNNF 变方, 作为一种特定的动态系统。 在GCN 中, 当它的重量满足( 放大的) 分层和非线性化的光谱确定的条件时, 我们发现它的重量, 它的重量会增加 GCN 的数值。 当 GCN 的直径( 放大的) 直径( ) 直径) 直径( 直) 直径( RCN) 直径( 直径) ( 直) ( 直径) 直径( 直径) ( 直径) 直径( 直) ( 直) 直径) 直径) ( 直) ( 直) ( 直径直) ( 直) ( 直) ( 直) ( 直) ( 直) ( 直) ( 直) ( 直) ( 直) 直) ( 直) ( 直) ( 直径直) ( 直) ( ) ( ) ( 直) ( 直) ( 直) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( 直) ( 直) ( ) ( ) ( ) ( 根 根 ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) (