CURRENT ANOMALY DETECTORS ARE ANOMALOUS: ON SEMANTIC TREATMENT OF OOD INPUTS Anonymous

Abstract

Machine learning models have achieved impressive performance across different modalities. It is well known that these models are prone to making mistakes on out-of-distribution inputs. OOD detection has, therefore, gained a lot of attention recently. We observe that most existing detectors use the distribution estimated by the training dataset for OOD detection. This can be a serious impediment since faulty OOD detectors can potentially restrict utility of the model. Such detectors, tied to the bias in data collection process, can be impermeable to inputs lying outside the training distribution but with the same semantic information (e.g., class labels) as the training data. We argue that in-distribution should not be tied to just the training distribution but to the distribution of the semantic information contained in the training data. To support our argument, we perform OOD detection on semantic information extracted from the training data of MNIST and COCO datasets, and show that it not only reduces false alarms but also significantly improves detection of OOD inputs with spurious features from training data.

1. INTRODUCTION

Machine learning models have achieved remarkable success in accomplishing different tasks across modalities such as image classification (Gkioxari et al., 2015) , speech recognition (Hannun et al., 2014) , and natural language processing (Majumder et al., 2017) . It is however known, that such models are unreliable on samples which are less likely to occur, according to the model's in-distribution estimated from its training data (Guo et al., 2017; Hendrycks & Gimpel, 2016) . Detection of these out-of-distribution (OOD) inputs is important for the deployment of machine learning models in safety-critical domains such as autonomous driving (Bojarski et al., 2016) , and medical diagnosis (De Fauw et al., 2018) . OOD detection has, therefore, gained a lot of attention recently (Liang et al., 2017; Lee et al., 2018; Hendrycks et al., 2019; Kaur et al., 2021b) . Even though there is sufficient interest in OOD detection, to the best of our knowledge, its unclear what precisely entails an OOD input. Existing detectors estimate a distribution that is tied to the training dataset, and flagging inputs as OOD when the assigned probability according to the estimated distribution is low. The standard drill involves a set of in-distribution inputs drawn from a dataset such as CIFAR10, and detecting those inputs as OOD that are drawn from a different dataset such as SVHN (Hendrycks & Gimpel, 2016; Kaur et al., 2021a; Lee et al., 2018) . Such external inputs (from SVHN) would have non-overlapping class labels (from CIFAR10). Ming et al. (2022) show that the existing detectors are unfortunately tied to the sampling bias of the training dataset. This results in low detection on OOD inputs with spurious features such as background, color, etc. from the training data. The authors report low detection performance of existing detectors on two datasets: 1) Birds (Sagawa et al., 2019) with class labels in {waterbirds, landbirds}, and 2) CelebA (Liu et al., 2015) with class labels in {grey hair, non-grey hair}. Table 1 shows these results for OOD images containing water (or land) as spurious feature for waterbirds (or landbirds), and OOD images of bald male with male as spurious feature for grey hair. This means that even though the classifier might be able to generalize better, OOD detectors itself can stifle its utility. On the other hand these detectors can be permeable to real OOD inputs which ought to be rejected. With this in mind, we propose to treat intended distribution of images as in-distribution. Images containing semantic information relevant to the training classesfoot_0 . Inputs deficient of semantic information



We will be using the terms "in-distribution" and "intended distribution" exchangeably in the paper.1

