The notion of individual fairness requires that similar people receive similar treatment. However, this is hard to achieve in practice since it is difficult to specify the appropriate similarity metric. In this work, we attempt to learn such similarity metric from human annotated data. We gather a new dataset of human judgments on a criminal recidivism prediction (COMPAS) task. By assuming the human supervision obeys the principle of individual fairness, we leverage prior work on metric learning, evaluate the performance of several metric learning methods on our dataset, and show that the learned metrics outperform the Euclidean and Precision metric under various criteria. We do not provide a way to directly learn a similarity metric satisfying the individual fairness, but to provide an empirical study on how to derive the similarity metric from human supervisors, then future work can use this as a tool to understand human supervision.
翻译:个人公正的概念要求类似的人得到类似的待遇。然而,这在实践中很难实现,因为很难具体指明适当的相似度量度。在这项工作中,我们试图从人类附加说明的数据中学习类似的度量。我们收集了一套关于犯罪性累犯预测(COMPAS)任务的人类判决的新数据集。我们假设人类监督符合个人公平原则,就利用先前的衡量学习工作,评估我们数据集中若干衡量学习方法的绩效,并表明所学的计量标准超越了不同标准的Euclidean和精度度度度度度度。我们不提供直接学习一种符合个人公平性的类似度度度度量度的方法,而是就如何从人类监督者那里获取类似度量度提供经验性研究,然后,我们今后的工作可以作为理解人类监督的工具。