A current goal in the graph neural network literature is to enable transformers to operate on graph-structured data, given their success on language and vision tasks. Since the transformer's original sinusoidal positional encodings (PEs) are not applicable to graphs, recent work has focused on developing graph PEs, rooted in spectral graph theory or various spatial features of a graph. In this work, we introduce a new graph PE, Graph Automaton PE (GAPE), based on weighted graph-walking automata (a novel extension of graph-walking automata). We compare the performance of GAPE with other PE schemes on both machine translation and graph-structured tasks, and we show that it generalizes several other PEs. An additional contribution of this study is a theoretical and controlled experimental comparison of many recent PEs in graph transformers, independent of the use of edge features.
翻译：图神经网络领域的一个目标是使Transformer能够处理图结构数据，因为Transformer在语言和视觉任务上很成功。由于原始的正弦位置编码对于图不适用，最近的工作集中在开发基于图的位置编码(P.E.s) ，根据谱图理论或图的各种空间特征。在这项工作中，我们介绍了一种新的图PE方法，Graph Automaton PE (GAPE) ，它基于加权图自动机（图自动机自动识别和转换字符串，扩展到图），并将GAPE的性能与其他PE方法在机器翻译和图结构任务上进行比较，同时我们还展示了它可以泛化多个其他P.E.s。本研究的另一个贡献是，我们在没有使用边界特征的情况下，理论和控制实验比较了许多最近在图Transformer中使用的PE方法。
相关内容
微信扫码咨询专知VIP会员