In many machine learning scenarios, supervision by gold labels is not available and consequently neural models cannot be trained directly by maximum likelihood estimation (MLE). In a weak supervision scenario, metric-augmented objectives can be employed to assign feedback to model outputs, which can be used to extract a supervision signal for training. We present several objectives for two separate weakly supervised tasks, machine translation and semantic parsing. We show that objectives should actively discourage negative outputs in addition to promoting a surrogate gold structure. This notion of bipolarity is naturally present in ramp loss objectives, which we adapt to neural models. We show that bipolar ramp loss objectives outperform other non-bipolar ramp loss objectives and minimum risk training (MRT) on both weakly supervised tasks, as well as on a supervised machine translation task. Additionally, we introduce a novel token-level ramp loss objective, which is able to outperform even the best sequence-level ramp loss on both weakly supervised tasks.
翻译:在许多机械学习情景中,没有黄金标签的监管,因此神经模型无法直接通过最大可能性估计来直接培训。在薄弱的监督情景中,可以使用量化目标来分配对模型产出的反馈,用于提取监督培训信号。我们提出了两项单独监督薄弱的任务的若干目标,即机器翻译和语义分解。我们表明,除了促进代金结构外,目标应积极抑制负面产出。这种双极性概念自然存在于斜坡损失目标中,我们适应神经模型。我们表明,双极斜坡损失目标优于其他非双极斜坡损失目标和最低风险培训,既适用于监督薄弱的任务,也适用于监督的机器翻译任务。此外,我们引入了一种新颖的象征性水平斜坡损失目标,它能够超越即使是监督薄弱的任务中的最佳序列级斜坡损失。