Data generated by audits of social media websites have formed the basis of our understanding of the biases presented in algorithmic content recommendation systems. As legislators around the world are beginning to consider regulating the algorithmic systems that drive online platforms, it is critical to ensure the correctness of these inferred biases. However, as we will show in this paper, doing so is a challenging task for a variety of reasons related to the complexity of configuration parameters associated with the audits that gather data from a specific platform. Focusing specifically on YouTube, we show that conducting audits to make inferences about YouTube's recommendation systems is more methodologically challenging than one might expect. There are many methodological decisions that need to be considered in order to obtain scientifically valid results, and each of these decisions incur costs. For example, should an auditor use (expensive to obtain) logged-in YouTube accounts while gathering recommendations from the algorithm to obtain more accurate inferences? We explore the impact of this and many other decisions and make some startling discoveries about the methodological choices that impact YouTube's recommendations. Taken all together, our research suggests auditing configuration compromises that YouTube auditors and researchers can use to reduce audit overhead, both economically and computationally, without sacrificing accuracy of their inferences. Similarly, we also identify several configuration parameters that have a significant impact on the accuracy of measured inferences and should be carefully considered.
翻译:通过审计社交媒体网站产生的数据,形成了我们对算法内容建议系统提出的偏差的理解基础。随着世界各地的立法者开始考虑监管驱动在线平台的算法系统,确保这些推断偏偏的正确性至关重要。然而,正如我们在本文件中将显示的那样,这样做是一项具有挑战性的任务,其原因很多,原因很多,与从特定平台收集数据的某个具体平台收集数据的审计所涉的复杂配置参数有关,因此,与收集从特定平台收集数据的审计所涉的复杂配置参数有关。我们特别侧重于YouTube,显示审计对YouTube建议系统作出推断的审计比人们可能预期的更具方法上的挑战性。由于世界各地立法者正开始考虑监管驱动在线平台的算法系统,因此需要考虑监管许多方法上的决定,以获得科学上有效的结果,而其中的每一项决定都需要成本。例如,审计员在收集从算法中提出的建议以获得更准确性更准确的推理时,是否应该使用(非常精准的)登录YouTuTuTube账户,同时不考虑在经济上和计算方面进行重大修改。</s>