In informal learning scenarios the popularity of multimedia content, such as video tutorials or lectures, has significantly increased. Yet, the users' interactions, navigation behavior, and consequently learning outcome, have not been researched extensively. Related work in this field, also called search as learning, has focused on behavioral or text resource features to predict learning outcome and knowledge gain. In this paper, we investigate whether we can exploit features representing multimedia resource consumption to predict of knowledge gain (KG) during Web search from in-session data, that is without prior knowledge about the learner. For this purpose, we suggest a set of multimedia features related to image and video consumption. Our feature extraction is evaluated in a lab study with 113 participants where we collected data for a given search as learning task on the formation of thunderstorms and lightning. We automatically analyze the monitored log data and utilize state-of-the-art computer vision methods to extract features about the seen multimedia resources. Experimental results demonstrate that multimedia features can improve KG prediction. Finally, we provide an analysis on feature importance (text and multimedia) for KG prediction.
翻译:在非正式学习情景中,多媒体内容,如视频辅导或讲座的普及程度显著提高;然而,用户的互动、导航行为以及随后的学习成果等多媒体内容的普及程度却没有进行广泛的研究;这一领域的相关工作,又称为学习搜索,侧重于行为或文字资源特征,以预测学习成果和知识收益。在本文中,我们调查是否可以利用代表多媒体资源消耗的特征,在网上搜索时,从会期数据中预测知识收益(KG),这是没有事先对学习者的了解的。为此目的,我们建议了一套与图像和视频消费有关的多媒体特征。我们用实验室研究对13名参与者评估了我们的特征提取情况,我们收集了数据,作为关于雷暴和闪电形成的学习任务进行特定搜索。我们自动分析监测的日志数据,并使用最新的计算机视觉方法提取所看到的多媒体资源的特征。实验结果表明,多媒体特征可以改进KG的预测。最后,我们对KG预测的特征(文字和多媒体)的重要性进行了分析。