OUT-OF-DISTRIBUTION DETECTION WITH IMPLICIT OUTLIER TRANSFORMATION

Abstract

Outlier exposure (OE) is powerful in out-of-distribution (OOD) detection, enhancing detection capability via model fine-tuning with surrogate OOD data. However, surrogate data typically deviate from test OOD data. Thus, the performance of OE, when facing unseen OOD data, can be weakened. To address this issue, we propose a novel OE-based approach that makes the model perform well for unseen OOD situations, even for unseen OOD cases. It leads to a min-max learning scheme-searching to synthesize OOD data that leads to worst judgments and learning from such OOD data for uniform performance in OOD detection. In our realization, these worst OOD data are synthesized by transforming original surrogate ones. Specifically, the associated transform functions are learned implicitly based on our novel insight that model perturbation leads to data transformation. Our methodology offers an efficient way of synthesizing OOD data, which can further benefit the detection model, besides the surrogate OOD data. We conduct extensive experiments under various OOD detection setups, demonstrating the effectiveness of our method against its advanced counterparts.

1. INTRODUCTION

Deep learning systems in the open world often encounter out-of-distribution (OOD) data whose label space is disjoint with that of the in-distribution (ID) samples. For many safety-critical applications, deep models should make reliable predictions for ID data, while OOD cases (Bulusu et al., 2020) should be reported as anomalies. It leads to the well-known OOD detection problem (Lee et al., 2018c; Fang et al., 2022) , which has attracted intensive attention in reliable machine learning. OOD detection remains non-trivial since deep models can be over-confident when facing OOD data (Nguyen et al., 2015; Bendale & Boult, 2016) , and many efforts have been made in pursuing reliable detection models (Yang et al., 2021; Salehi et al., 2021) . Building upon discriminative models, existing OOD detection methods can generally be attributed to two categories, namely, posthoc approaches and fine-tuning approaches. The post-hoc approaches assume a well-trained model on ID data with its fixed parameters, using model responses to devise various scoring functions to indicate ID and OOD cases (Hendrycks & Gimpel, 2017; Liang et al., 2018; Lee et al., 2018c; Liu et al., 2020; Sun et al., 2021; 2022; Wang et al., 2022) . By contrast, the fine-tuning methods allow the target model to be further adjusted, boosting its detection capability by regularization (Lee et al., 2018a; Hendrycks et al., 2019; Tack et al., 2020; Mohseni et al., 2020; Sehwag et al., 2021; Chen et al., 2021; Du et al., 2022; Ming et al., 2022; Bitterwolf et al., 2022) . Typically, fine-tuning approaches benefit from explicit knowledge of unknowns during training and thus generally reveal reliable performance across various real-world situations (Yang et al., 2021) . For the fine-tuning approaches, outlier exposure (OE) (Hendrycks et al., 2019) is among the most potent ones, engaging surrogate OOD data during training to discern ID and OOD patterns. By mak- † Correspondence to Bo Han (bhanml@comp.hkbu.edu.hk) and Junjie Ye (yejunjie4@huawei.com).

