Machine learning algorithms typically assume independent and identically distributed samples in training and at test time. Much work has shown that high-performing ML classifiers can degrade significantly and provide overly-confident, wrong classification predictions, particularly for out-of-distribution (OOD) inputs. Conditional language models (CLMs) are predominantly trained to classify the next token in an output sequence, and may suffer even worse degradation on OOD inputs as the prediction is done auto-regressively over many steps. Furthermore, the space of potential low-quality outputs is larger as arbitrary text can be generated and it is important to know when to trust the generated output. We present a highly accurate and lightweight OOD detection method for CLMs, and demonstrate its effectiveness on abstractive summarization and translation. We also show how our method can be used under the common and realistic setting of distribution shift for selective generation (analogous to selective prediction for classification) of high-quality outputs, while automatically abstaining from low-quality ones, enabling safer deployment of generative language models.
翻译:机器学习算法通常在培训和测试时间假定独立和相同分布的样本。许多工作表明,高性能的 ML 分类法可以大幅降解,提供过于自信和错误的分类预测,特别是分配外投入。有条件语言模型(CLMS)主要经过培训,在产出序列中对下一个符号进行分类,OOD 输入的降解可能更严重,因为预测是自动反向的,在许多步骤之上进行。此外,潜在低质量产出的空间更大,因为可以生成任意的文本,了解何时可以信任生成的输出非常重要。我们为 CLMS 提出了一个高度准确和轻度的 OOD 检测方法,并展示了其在抽象的合成和翻译方面的有效性。我们还表明,如何在为高质量产出的选择性生成(与选择性的分类预测不同)分配转移这一共同和现实的环境下使用我们的方法,同时自动避免低质量的输出,从而能够更安全地部署基因化语言模型。</s>