HOW USEFUL ARE GRADIENTS FOR OOD DETECTION REALLY?

Abstract

One critical challenge in deploying machine learning models in real-life applications is out of distribution (OOD) detection. Given a predictive model which is accurate on in distribution (ID) data, an OOD detection system can further equip the model with the option to defer prediction when the input is novel and the model has low confidence. Notably, there has been some recent interest in utilizing gradient information in pre-trained models for OOD detection. While these methods are competitive, we argue that previous works conflate their performance with the necessity of gradients. In this work, we provide an in-depth analysis and comparison of gradient based methods and elucidate the key components that warrant their OOD detection performance. We further demonstrate that a general, non-gradientbased family of OOD detection methods are just as competitive, casting doubt on the usefulness of gradients for OOD detection.

1. INTRODUCTION

Recent advances in algorithms, models, and training infrastructure have brought about unprecedented performance of machine learning (ML) methods, across a wide range of data types and tasks. Despite their demonstrated potential on benchmark settings and domains, one obstacle which limits ML methods' applicability in real-world applications is the uncertainty or confidence of the predictions. Without any deliberate mechanisms, ML models will output a prediction for any given input, and the question of whether this prediction can be trusted will be especially critical in many high-risk decision-making settings (e.g. self-driving cars (Agarwal et al., 2021 ), physical sciences (Char et al., 2021; Boyer et al., 2021), and healthcare (Zhou et al., 2020) ). This risk is further exacerbated with deep learning where the interpretability of models are often limited (Rudin, 2019)). It is unrealistic for one to expect to train a model that has perfect predictions for all possible inputs, partly because real-world datasets are limited in their scope. Thus in lieu of trying to make predictions for all test inputs, one can attempt to first detect whether the input is covered by the support of the training data. This is the motivation behind OOD detection. Among the diverse approaches to OOD detection for image recognition, a recent line of work has suggested utilizing the information in gradients to derive efficient and performant methods for OOD detection (Liang et al., 2017; Lee & AlRegib, 2020; Agarwal et al., 2020; Lee & AlRegib, 2021; Huang et al., 2021; Sun et al., 2022; Kokilepersaud et al., 2022) . We motivate our work by first exploring the claim that gradients are useful for OOD detection. Through a comparison with various extensions of gradient-based scores, we analyze the key components that actually drive the performance of these methods, and we argue that gradient computations are not essential in deriving performant post hoc OOD detectors. Rather, these methods ultimately rely on the magnitude of the learned feature embedding and the predicted output distribution. We thereby refute many of the intuitions that previous works motivate their methods with. Based on our analysis, we advocate for the study of a more general, non-gradient-based framework for producing performant score functions and provide a comprehensive empirical evaluation of various instantiations of the score within this framework. The rest of this paper is structured as follows. We first provide a formal statement of the problem setting and the related works in Section 2. We then introduce existing and new gradient-based detectors and discuss how both can be simplified into intuitive forms (Section 3). Following this, we perform empirical evaluations of the methods (Section 4) and discuss their implications in Section 5.

