Calibration is defined as the ratio of the average predicted click rate to the true click rate. The optimization of calibration is essential to many online advertising recommendation systems because it directly affects the downstream bids in ads auctions and the amount of money charged to advertisers. Despite its importance, calibration optimization often suffers from a problem called "maximization bias". Maximization bias refers to the phenomenon that the maximum of predicted values overestimates the true maximum. The problem is introduced because the calibration is computed on the set selected by the prediction model itself. It persists even if unbiased predictions can be achieved on every datapoint and worsens when covariate shifts exist between the training and test sets. To mitigate this problem, we theorize the quantification of maximization bias and propose a variance-adjusting debiasing (VAD) meta-algorithm in this paper. The algorithm is efficient, robust, and practical as it is able to mitigate maximization bias problems under covariate shifts, neither incurring additional online serving costs nor compromising the ranking performance. We demonstrate the effectiveness of the proposed algorithm using a state-of-the-art recommendation neural network model on a large-scale real-world dataset.
翻译:校准被定义为平均预测点击率与真实点击率之比。在许多在线广告推荐系统中优化校准是必不可少的,因为它直接影响拍卖中的下游竞标和向广告商收取的费用。尽管它非常重要,但校准优化经常受到一种称为“最大化偏差”的问题的困扰。最大化偏差是指预测值的最大值高估了真实最大值的现象。这个问题是在预测模型自己选择的集合上计算校准时引入的。即使可以在每个数据点上实现无偏预测,这个问题仍然存在,并且在训练集和测试集之间存在协变量漂移时会加剧。为了缓解这个问题,我们在本文中提出了最大化偏差量化理论,并提出了一种方差调整去偏(VAD)元算法。该算法高效、稳健、实用,能够缓解协变量漂移下的最大化偏差问题,既不会增加额外的在线服务费用,也不会影响排名性能。我们使用一种最先进的推荐神经网络模型在大型真实世界数据集上展示了所提出算法的有效性。