In classification, the de facto method for aggregating individual losses is the average loss. When the actual metric of interest is 0-1 loss, it is common to minimize the average surrogate loss for some well-behaved (e.g. convex) surrogate. Recently, several other aggregate losses such as the maximal loss and average top-$k$ loss were proposed as alternative objectives to address shortcomings of the average loss. However, we identify common classification settings, e.g. the data is imbalanced, has too many easy or ambiguous examples, etc., when average, maximal and average top-$k$ all suffer from suboptimal decision boundaries, even on an infinitely large training set. To address this problem, we propose a new classification objective called the close-$k$ aggregate loss, where we adaptively minimize the loss for points close to the decision boundary. We provide theoretical guarantees for the 0-1 accuracy when we optimize close-$k$ aggregate loss. We also conduct systematic experiments across the PMLB and OpenML benchmark datasets. Close-$k$ achieves significant gains in 0-1 test accuracy, improvements of $\geq 2$% and $p<0.05$, in over 25% of the datasets compared to average, maximal and average top-$k$. In contrast, the previous aggregate losses outperformed close-$k$ in less than 2% of the datasets.
翻译:在分类中,个人损失合计的实际方法为平均损失。当实际利息衡量标准为0-1损失时,将某种良好行为(如 convex)的替代损失平均代谢损失降到最低是常见的。最近,提出了其他一些总损失,例如最大损失和平均最高至最高损失,作为解决平均损失缺陷的替代目标。然而,我们确定了共同分类设置,例如数据不平衡,有太多容易或模糊的例子,等等,当平均、最高和平均最高至最高为美元都受到次优决定界限的影响时,即使是在无限大的培训中,也是常见的。为了解决这一问题,我们提出了一个新的分类目标,称为接近至k美元的总损失,我们适应性地尽量减少接近决定界限点的损失。我们在优化接近至总损失总额时,为0-1的准确性提供了理论保证。我们还在PMLB和OpML基准数据集中进行了系统化的实验。近千美元在近于0.1美元测试的准确度上取得了显著的收益,比25美元的平均数据改进到低于25美元。