Effects of Optical Character Recognition (OCR) quality on historical information retrieval have so far been studied in data-oriented scenarios regarding the effectiveness of retrieval results. Such studies have either focused on the effects of artificially degraded OCR quality (see, e.g., [1-2]) or utilized test collections containing texts based on authentic low quality OCR data (see, e.g., [3]). In this paper the effects of OCR quality are studied in a user-oriented information retrieval setting. Thirty-two users evaluated subjectively query results of six topics each (out of 30 topics) based on pre-formulated queries using a simulated work task setting. To the best of our knowledge our simulated work task experiment is the first one showing empirically that users' subjective relevance assessments of retrieved documents are affected by a change in the quality of optically read text. Users of historical newspaper collections have so far commented effects of OCR'ed data quality mainly in impressionistic ways, and controlled user environments for studying effects of OCR quality on users' relevance assessments of the retrieval results have so far been missing. To remedy this The National Library of Finland (NLF) set up an experimental query environment for the contents of one Finnish historical newspaper, Uusi Suometar 1869-1918, to be able to compare users' evaluation of search results of two different OCR qualities for digitized newspaper articles. The query interface was able to present the same underlying document for the user based on two alternatives: either based on the lower OCR quality, or based on the higher OCR quality, and the choice was randomized. The users did not know about quality differences in the article texts they evaluated. The main result of the study is that improved optical character recognition quality affects perceived usefulness of historical newspaper articles significantly. The mean average evaluation score for the improved OCR results was 7.94% higher than the mean average evaluation score of the old OCR results.
翻译:光学字符识别(OCR)质量对历史信息检索的影响至今已在关于检索结果有效性的数据导向假设中进行了研究。这些研究要么侧重于人为退化的OCR质量的影响(例如,见[1-2]),要么侧重于人工降解的OCR质量(例如,见[1-2]),或者使用了含有基于真实的低质量OCR数据(例如,见,见,[3])的文本的测试收藏;在本文件中,以面向用户的信息检索设置对OCR质量的影响进行了研究。32个用户在使用模拟工作设置的预制查询基础上,对每个主题(共30个专题)的随机查询结果进行了主观评价。为了最了解我们的知识,我们的模拟工作任务实验是以实验方式显示,用户对所检索文件的主观相关性评估会受到影响(例如,见,见,见,见,见,见,见,见,见,见;历史报纸收藏的用户对OCR质量的影响迄今为止,主要是以印象为主,对OCR质量对用户质量评估的影响进行了控制下,对用户质量评估的较低环境进行了评估。为了纠正,对芬兰国家O-ral质量的图书馆,可以对18的原始文件进行一项分析,对结果进行一项不同的分析,对历史文章进行实验结果的评为不同的分析,对18的评为,结果进行不同的分析,对18的评为根据,对历史文章进行不同的分析,对结果环境进行不同的分析,对结果。对18种不同的分析,对结果为根据,对结果为根据,对18种不同的分析,对18种。