The study of IR evaluation metrics through axiomatic analysis enables a better understanding of their numerical properties. Some works have modelled the effectiveness of retrieval metrics with axioms that capture desirable properties on the set of rankings of documents. This paper formally explores the effect of these ranking axioms on the numerical values of some IR evaluation metrics. It focuses on the set of ranked lists of documents with multigrade relevance. The possible orderings in this set are derived from three commonly accepted ranking axioms on retrieval metrics; then, they are classified by their latticial properties. When relevant documents are prioritised, a subset of document rankings are identified: the join-irreducible elements, which have some resemblance to the concept of basis in vector space. It is possible to compute the precision, recall, RBP or DCG values of any ranking from their values in the join-irreducible elements. However this is not the case when the swapping of documents is considered.
翻译:通过法理分析对IR评价指标的研究有助于更好地了解其数值属性。有些工作模拟了具有轴值的检索指标的有效性,这些参数在文件的顺序排列中反映了可取的属性。本文件正式探讨了这些排序轴值对IR评价指标数值的影响。它侧重于具有多级相关性的分级文件清单集。这套文件中可能的命令来自三个共同接受的关于检索指标的等级轴值;然后,它们按其纬度属性进行分类。当相关文件被优先排序时,将确定文件排序的子集:组合式不可缩减要素,这些要素与矢量空间的基础概念有一些相似之处。有可能将任何排序的精确性、回顾、RBP或DCG值与它们合并式可减少要素中的数值相容。但考虑转换文件时,情况并非如此。