Automatic essay grading (AEG) is a process in which machines assign a grade to an essay written in response to a topic, called the prompt. Zero-shot AEG is when we train a system to grade essays written to a new prompt which was not present in our training data. In this paper, we describe a solution to the problem of zero-shot automatic essay grading, using cognitive information, in the form of gaze behaviour. Our experiments show that using gaze behaviour helps in improving the performance of AEG systems, especially when we provide a new essay written in response to a new prompt for scoring, by an average of almost 5 percentage points of QWK.
翻译:自动作文评分( AEG) 是一个过程, 机器在其中给针对一个题目的论文定了一个分数, 称为“ 即时” 。 零射AEG 是当我们训练一个系统, 将作文评分给一个新的即时, 这在我们的训练数据中并不存在。 在本文中, 我们描述了一种解决零射自动作文评分问题的办法, 使用认知信息, 采取凝视行为的形式。 我们的实验显示, 使用凝视行为 有助于改善 AEG 系统的表现, 特别是当我们针对新的评分提示提供一份新作文, 平均近5%的QWK 。