Explicating implicit reasoning (i.e. warrants) in arguments is a long-standing challenge for natural language understanding systems. While recent approaches have focused on explicating warrants via crowdsourcing or expert annotations, the quality of warrants has been questionable due to the extreme complexity and subjectivity of the task. In this paper, we tackle the complex task of warrant explication and devise various methodologies for collecting warrants. We conduct an extensive study with trained experts to evaluate the resulting warrants of each methodology and find that our methodologies allow for high-quality warrants to be collected. We construct a preliminary dataset of 6,000 warrants annotated over 600 arguments for 3 debatable topics. To facilitate research in related downstream tasks, we release our guidelines and preliminary dataset.
翻译:论证中的隐含推理(即授权令)是自然语言理解系统的长期挑战。虽然最近的方法侧重于通过众包或专家说明来解释授权书,但由于任务极其复杂和主观性,授权书的质量令人怀疑。在本文件中,我们处理授权书的复制这一复杂任务,并设计各种收集授权书的方法。我们与受过培训的专家进行广泛研究,评估所得出的每种方法的授权书,发现我们的方法允许收集高质量的授权书。我们为3个有争议的专题建立了一套6 000个初步数据集,对6 000个授权书附加了600多个参数。为了便利相关下游任务的研究,我们发布了我们的指导方针和初步数据集。