The current state-of-the-art end-to-end semantic role labeling (SRL) model is a deep neural network architecture with no explicit linguistic features. However, prior work has shown that gold syntax trees can dramatically improve SRL, suggesting that neural network models could see great improvements from explicit modeling of syntax. In this work, we present linguistically-informed self-attention (LISA): a new neural network model that combines multi-head self-attention with multi-task learning across dependency parsing, part-of-speech, predicate detection and SRL. For example, syntax is incorporated by training one of the attention heads to attend to syntactic parents for each token. Our model can predict all of the above tasks, but it is also trained such that if a high-quality syntactic parse is already available, it can be beneficially injected at test time without re-training our SRL model. In experiments on the CoNLL-2005 SRL dataset LISA achieves an increase of 2.5 F1 absolute over the previous state-of-the-art on newswire with predicted predicates and more than 2.0 F1 on out-of-domain data. On ConLL-2012 English SRL we also show an improvement of more than 3.0 F1, a 13% reduction in error.
翻译:目前最先进的端到端语义作用标签模式(SRL)是一个深层神经网络网络结构,没有明确的语言特征。然而,先前的工作表明,黄金语法树可以极大地改善SRL,这表明神经网络模型可以从明确的语法模型中看到巨大的改进。在这项工作中,我们展示了语言知情自省(LISA):一个新的神经网络模型,将多头自控与跨依赖分析、部分语音、上游探测和SRL的多任务学习结合起来。例如,通过培训一位关注对象的负责人参加每个符号的合成父母,将语法树大大改善SRL。我们的模型可以预测所有上述任务,但是它也经过培训,如果已经存在高质量的合成精度分析,在测试时可以有益地注入,而无需再培训我们的SRL模式。在CON-2005 SRL数据集的实验中,比前一号F1绝对值(F1)和前一号F1(F1)的S-LL)的预测值改进值增加了2.5 F1,在前一号(F1)中,在前一号(F-L1)中也比前一号(F-L1)的预测的直位(F-L1)更精确的L1,在前的F-L1号中增加了我们-S-S-LLLLA中的数据减少。