Relevance estimators are algorithms used by major social media platforms to determine what content is shown to users and its presentation order. These algorithms aim to personalize the platforms' experience for users, increasing engagement and, therefore, platform revenue. However, at the large scale of many social media platforms, many have concerns that the relevance estimation and personalization algorithms are opaque and can produce outcomes that are harmful to individuals or society. Legislations have been proposed in both the U.S. and the E.U. that mandate auditing of social media algorithms by external researchers. But auditing at scale risks disclosure of users' private data and platforms' proprietary algorithms, and thus far there has been no concrete technical proposal that can provide such auditing. Our goal is to propose a new method for platform-supported auditing that can meet the goals of the proposed legislations. The first contribution of our work is to enumerate these challenges and the limitations of existing auditing methods to implement these policies at scale. Second, we suggest that limited, privileged access to relevance estimators is the key to enabling generalizable platform-supported auditing of social media platforms by external researchers. Third, we show platform-supported auditing need not risk user privacy nor disclosure of platforms' business interests by proposing an auditing framework that protects against these risks. For a particular fairness metric, we show that ensuring privacy imposes only a small constant factor increase (6.34x as an upper bound, and 4x for typical parameters) in the number of samples required for accurate auditing. Our technical contributions, combined with ongoing legal and policy efforts, can enable public oversight into how social media platforms affect individuals and society by moving past the privacy-vs-transparency hurdle.
翻译:相关性估计值是主要社交媒体平台用来确定向用户及其演示程序显示的内容的算法。这些算法旨在将平台的经验个人化,增加参与,从而增加平台收入。然而,在许多社交媒体平台规模庞大的情况下,许多人担心相关估计和个人化算法不透明,可能产生有害个人或社会的结果。美国和欧盟都提出了立法,要求外部研究人员对社交媒体算法进行审计。但是,在规模风险中审计平台影响披露用户的私人数据和平台的专有算法,迄今还没有提出能够提供此类审计的具体技术建议。我们的目标是提出一种新的方法,用于平台支持的审计,以达到拟议立法的目标。我们工作的第一个贡献是列举这些挑战和现有审计方法在规模上对个人或社会有害的局限性。第二,我们建议有限、最容易获得相关估计者(6个)是使外部研究人员能够对社交媒体平台进行普遍支持的审计的关键。第三,我们展示平台支持的透明性审计需要的是,在常规审计中,我们通过不断的保密性框架来保护客户的隐私风险。