Building robust deterministic neural networks remains a challenge. On the one hand, some approaches improve out-of-distribution detection at the cost of reducing classification accuracy in some situations. On the other hand, some methods simultaneously increase classification accuracy, uncertainty estimation, and out-of-distribution detection at the expense of reducing the inference efficiency. In this paper, we propose training deterministic neural networks using our DisMax loss, which works as a drop-in replacement for the usual SoftMax loss (i.e., the combination of the linear output layer, the SoftMax activation, and the cross-entropy loss). Starting from the IsoMax+ loss, we create each logit based on the distances to all prototypes, rather than just the one associated with the correct class. We also introduce a mechanism to combine images to construct what we call fractional probability regularization. Moreover, we present a fast way to calibrate the network after training. Finally, we propose a composite score to perform out-of-distribution detection. Our experiments show that DisMax usually outperforms current approaches simultaneously in classification accuracy, uncertainty estimation, and out-of-distribution detection while maintaining deterministic neural network inference efficiency. The code to reproduce the results is available at https://github.com/dlmacedo/distinction-maximization-loss.
翻译:建立稳健的确定性神经网络仍是一个挑战。一方面,有些方法改进分配外检测,以降低某些情况下的分类准确性为代价。另一方面,有些方法同时提高分类准确性、不确定性估计和分配外检测,以降低推断效率为代价。在本文中,我们提议培训确定性神经网络,使用我们的Dismax损失,作为常规SoftMax损失(即线性输出层、SoftMax激活和交叉体质损失的组合)的倒置替代。从Isomax+损失开始,我们根据与所有原型的距离,而不是仅仅根据与正确等级相关的距离,建立每项对账。我们还引入了一种机制,将图像结合起来,以构建我们所称的分数概率规范化。此外,我们提出了一个在培训后校准网络的快速方法。最后,我们提出一个复合分数,以进行分配外检测。我们的实验显示,Dismax通常在分类准确性、不确定性估计和超出当前方法的同时,在可确定性网络的代码化方面,在确定性检测结果方面,同时进行。