A t-distribution based operator for enhancing out of distribution robustness of

A t-distribution based operator for enhancing out of distribution robustness of neural network classifiers

Neural Network (NN) classifiers can assign extreme probabilities to samples that have not appeared during training (out-of-distribution samples) resulting in erroneous and unreliable predictions. One of the causes for this unwanted behaviour lies in the use of the standard softmax operator which pushes the posterior probabilities to be either zero or unity hence failing to model uncertainty.The statistical derivation of the softmax operator relies on the assumption that the distributions of the latent variables for a given class are Gaussian with known variance. However, it is possible to use different assumptions in the same derivation and attain from other families of distributions as well. This allows derivation of novel operators with more favourable properties. Here, a novel operator is proposed that is derived using $t$-distributions which are capable of providing a better description of uncertainty. It is shown that classifiers that adopt this novel operator can be more robust to out of distribution samples, often outperforming NNs that use the standard softmax operator. These enhancements can be reached with minimal changes to the NN architecture.

基于t分布的算子，用于增强神经网络分类器的分布外鲁棒性

神经网络（NN）分类器可以将极端概率分配给训练期间未出现的样本（分布样本），从而导致错误和不可靠的预测。造成此不良行为的原因之一在于使用标准的softmax运算符，该运算符将后验概率推为零或1，因此无法对不确定性进行建模。.. softmax运算符的统计推导基于这样的假设，即给定类的潜在变量的分布是具有已知方差的高斯分布。但是，可以在相同的推导中使用不同的假设，也可以从其他分布族中得出。这允许派生具有更有利特性的新颖算子。在此，提出了一种新颖的算子，该算子使用 Ť -能够更好地描述不确定性的分布。结果表明，采用这种新颖算子的分类器对于分发样本外的样本可能更健壮，通常优于使用标准softmax算子的神经网络。只需对NN体系结构进行最小的更改即可实现这些增强。 （阅读更多）