Transformer-based models have demonstrated excellent capabilities of capturing patterns and structures in natural language generation and achieved state-of-the-art results in many tasks. In this paper we present a transformer-based model for multi-turn dialog response generation. Our solution is based on a hybrid approach which augments a transformer-based generative model with a novel retrieval mechanism, which leverages the memorized information in the training data via k-Nearest Neighbor search. Our system is evaluated on two datasets made by customer/assistant dialogs: the Taskmaster-1, released by Google and holding high quality, goal-oriented conversational data and a proprietary dataset collected from a real customer service call center. Both achieve better BLEU scores over strong baselines.
翻译:以变换器为基础的模型展示了在自然语言生成中捕捉模式和结构的出色能力,并在许多任务中取得了最先进的成果。在本文中,我们提出了一个基于变压器的多点对话响应生成模型。我们的解决办法是基于一种混合方法,通过一种新型的检索机制,扩大以变压器为基础的基因化模型,通过 k- Nearest Neearbearbor 搜索来利用培训数据中的记忆信息。我们的系统是根据客户/协助对话制作的两个数据集进行评估的:由谷歌发行的任务主管1号,拥有高质量的、面向目标的谈话数据和从真正的客户服务呼叫中心收集的专有数据集。两者都比强的基线获得更好的BLEU分数。