Neural Processes (NPs) (Garnelo et al 2018a;b) approach regression by learning to map a context set of observed input-output pairs to a distribution over regression functions. Each function models the distribution of the output given an input, conditioned on the context. NPs have the benefit of fitting observed data efficiently with linear complexity in the number of context input-output pairs, and can learn a wide family of conditional distributions; they learn predictive distributions conditioned on context sets of arbitrary size. Nonetheless, we show that NPs suffer a fundamental drawback of underfitting, giving inaccurate predictions at the inputs of the observed data they condition on. We address this issue by incorporating attention into NPs, allowing each input location to attend to the relevant context points for the prediction. We show that this greatly improves the accuracy of predictions, results in noticeably faster training, and expands the range of functions that can be modelled.
翻译:神经过程(NPs)(Garnelo等人,2018年a;b) 采取回归方法,通过学习绘制一组观察到的输入输出对对的上下文图,绘制回归函数的分布。每种功能都以上下文为条件,对输入输出的分布进行模型。 NP的好处是,将观测数据与上下文输入输出对的线性复杂度相匹配,并能够学习一系列广泛的有条件分布;他们学习以任意大小的上下文为条件的预测分布。然而,我们表明,NPs在配差方面有一个根本性的缺陷,在所观察到的数据输入时作出不准确的预测。我们通过将注意力纳入NPs来解决这一问题,允许每个输入地点参加预测的相关背景点。我们表明,这大大提高了预测的准确性,导致明显更快的培训,并扩大了可以模拟的功能范围。