CADET: FULLY SELF-SUPERVISED ANOMALY DETEC-TION WITH CONTRASTIVE LEARNING

Abstract

Handling out-of-distribution (OOD) samples has become a major stake in the realworld deployment of machine learning systems. This work explores the application of self-supervised contrastive learning to the simultaneous detection of two types of OOD samples: unseen classes and adversarial perturbations. Since in practice the distribution of such samples is not known in advance, we do not assume access to OOD examples. We show that similarity functions trained with contrastive learning can be leveraged with the maximum mean discrepancy (MMD) two-sample test to verify whether two independent sets of samples are drawn from the same distribution. Inspired by this approach, we introduce CADet (Contrastive Anomaly Detection), a method based on image augmentations to perform anomaly detection on single samples. CADet compares favorably to adversarial detection methods to detect adversarially perturbed samples on ImageNet. Simultaneously, it achieves comparable performance to unseen label detection methods on two challenging benchmarks: ImageNet-O and iNaturalist. CADet is fully self-supervised and requires neither labels for in-distribution samples nor access to OOD examples.

1. INTRODUCTION

While modern machine learning systems have achieved countless successful real-world applications, handling out-of-distribution (OOD) inputs remains a tough challenge of significant importance. The problem is especially acute for high-dimensional problems like image classification. Models are typically trained in a close-world setting but inevitably faced with novel input classes when deployed in the real world. The impact can range from displeasing customer experience to dire consequences in the case of safety-critical applications such as autonomous driving (Kitt et al., 2010) or medical analysis (Schlegl et al., 2017a) . Although achieving high accuracy against all meaningful distributional shifts is the most desirable solution, it is particularly challenging. An efficient method to mitigate the consequences of unexpected inputs is to perform anomaly detection, which allows the system to anticipate its inability to process unusual inputs and react adequately. Anomaly detection methods generally rely on one of three types of statistics: features, logits, and softmax probabilities, with some systems leveraging a mix of these (Wang et al., 2022 ). An anomaly score f (x) is computed, and then detection with threshold τ is performed based on whether f (x) > τ . The goal of a detection system is to find an anomaly score that efficiently discriminates between in-distribution and out-of-distribution samples. However, the common problem of these systems is that different distributional shifts will unpredictably affect these statistics. Accordingly, detection systems either achieve good performance on specific types of distributions or require tuning on OOD samples. In both cases, their practical use is severely limited. Motivated by these issues, recent work has tackled the challenge of designing detection systems for unseen classes without prior knowledge of the unseen label set or access to OOD samples (Winkens et al., 2020; Tack et al., 2020; Wang et al., 2022) . We first investigate the use of maximum mean discrepancy two-sample test (MMD) (Gretton et al., 2012) in conjunction with self-supervised contrastive learning to assess whether two sets of samples have been drawn from the same distribution. Motivated by the strong testing power of this method, we then introduce a statistic inspired by MMD and leveraging contrastive transformations. Based on this statistic, we propose CADet (Contrastive Anomaly Detection), which is able to detect OOD samples from single inputs and performs well on both label-based and adversarial detection benchmarks, without requiring access to any OOD samples to train or tune the method. Only a few works have addressed these tasks simultaneously. These works either focus on particular in-distribution data such as medical imaging for specific diseases (Uwimana1 & Senanayake, 2021) or evaluate their performances on datasets with very distant classes such as CIFAR10 (Krizhevsky, 2009) , SVHN (Netzer et al., 2011) , and LSUN (Yu et al., 2015) , resulting in simple benchmarks that do not translate to general real world applications (Lee et al., 2018) . Contributions: Our main contributions are as follows: • We use similarity functions learned by self-supervised contrastive learning with MMD to show that the test sets of CIFAR10 and CIFAR10.1 (Recht et al., 2019) have different distributions. • We propose a novel improvement to MMD and show it can also be used to confidently detect distributional shifts when given a small number of samples. • We introduce CADet, a fully self-supervised method for anomaly detection, and show it outperforms current methods in adversarial detection tasks while performing well on class-based OOD detection. The outline is as follows: in Section 2, we discuss relevant previous work. Section 3 describes the self-supervised contrastive method based on SimCLRv2 (Chen et al., 2020b) used in this work. Section 4 explores the application of learned similarity functions in conjunction with MMD to verify whether two independent sets of samples are drawn from the same distribution. Section 5 presents CADet and evaluates its empirical performance. Finally, we discuss results and limitations in Section 6.

2. RELATED WORK

We propose a self-supervised contrastive method for anomaly detection (both unknown classes and adversarial attacks) inspired by MMD. Thus, our work intersects with the MMD, label-based OOD detection, adversarial detection, and self-supervised contrastive learning literature. MMD two-sample test has been extensively studied (Gretton et al., 2012; Wenliang et al., 2019; Gretton et al., 2009; Sutherland et al., 2016; Chwialkowski et al., 2015; Jitkrittum et al., 2016) , though it is, to the best of our knowledge, the first time a similarity function trained via contrastive learning is used in conjunction with MMD. Liu et al. (2020a) uses MMD with a deep kernel trained on a fraction of the samples to argue that CIFAR10 and CIFAR10.1 have different test distributions. We build upon that work by confirming their finding with higher confidence levels, using fewer samples. Label-based OOD detection methods discriminate samples that differ from those in the training distribution. We focus on unsupervised OOD detection in this work, i.e., we do not assume access to data labeled as OOD. Unsupervised OOD detection methods include density-based (Zhai et al., 2016; Nalisnick et al., 2018; 2019; Choi et al., 2018; Du & Mordatch, 2019; Ren et al., 2019; Serrà et al., 2019; Grathwohl et al., 2019; Liu et al., 2020b; Dinh et al., 2016) , reconstruction-based (Schlegl et al., 2017b; Zong et al., 2018; Deecke et al., 2018; Pidhorskyi et al., 2018; Perera et al., 2019; Choi et al., 2018) , one-class classifiers (Schölkopf et al., 1999; Ruff et al., 2018) , self-supervised (Golan & El-Yaniv, 2018; Hendrycks et al., 2019b; Bergman & Hoshen, 2020; Tack et al., 2020) , and supervised approaches (Liang et al., 2017; Hendrycks & Gimpel, 2016) , though some works do not fall into any of these categories (Wang et al., 2022) . Adversarial detection discriminates adversarial samples from the original data. Adversarial samples are generated by minimally perturbing actual samples to produce a change in the model's output, such as a misclassification. Most works rely on the knowledge of some attacks for training (Abusnaina et al., 2021; Metzen et al., 2017; Feinman et al., 2017; Lust & Condurache, 2020; Zuo & Zeng, 2021; Papernot & McDaniel, 2018; Ma et al., 2018) , with the exception of Hu et al. (2019) . Self-supervised contrastive learning methods (Wu et al., 2018; He et al., 2020; Chen et al., 2020a; b) are commonly used to pre-train a model from unlabeled data to solve a downstream task such as image classification. Contrastive learning relies on instance discrimination trained with a contrastive loss (Hadsell et al., 2006) such as infoNCE (Gutmann & Hyvärinen, 2010) . Contrastive learning for OOD detection aims to find good representations for detecting OOD samples in a supervised (Liu & Abbeel, 2020; Khalid et al., 2022) or unsupervised (Winkens et al., 2020; Mohseni et al., 2020; Sehwag et al., 2021) setting. Perhaps the closest work in the literature is CSI (Tack et al., 2020) , which found SimCLR features to have good discriminative power for unknown classes detection and leveraged similarities between transformed samples in their score. However, this method is not well-suited for adversarial detection. CSI ignores the similarities between different transformations of a same sample, an essential component to perform adversarial detection (see Section 6.2). In addition, CSI scales their score with the norm of input representations. While efficient on samples with unknown classes, it is unreliable on adversarial perturbations, which typically increase representation norms.

3. CONTRASTIVE MODEL

We build our model on top of SimCLRv2 (Chen et al., 2020b) for its simplicity and efficiency. It is composed of an encoder backbone network f θ as well as a 3-layer contrastive head h θ ′ . Given an indistribution sample X , a similarity function sim, and a distribution of training transformations T train , the goal is to simultaneously maximize E x∼X ;t0,t1∼Ttrain [sim(h θ ′ • f θ (t 0 (x)), h θ ′ • f θ (t 1 (x)))] and minimize E x,y∼X ;t0,t1∼Ttrain [sim(h θ ′ • f θ (t 0 (x)), h θ ′ • f θ (t 1 (y)))], i.e., we want to learn representations in which random transformations of a same example are close while random transformations of different examples are distant. To achieve this, given an input batch {x i } i=1,...,N , we compute the set {x (j) i } j=0,1;i=1,...,N by applying two transformations independently sampled from T train to each x i . We then compute the embeddings z (j) i = h θ ′ • f θ (x (j) i ) and apply the following contrastive loss: L(z) = i=1,...,N -log e sim(z (0) i ,z i )/τ j∈{1,...,N } e sim(z (0) i ,z (1) j )/τ + j∈{1,...,N }\i e sim(z (0) i ,z (0) j )/τ , where τ is the temperature hyperparameter and sim(x, y) = ⟨x|y⟩ ∥x∥ 2 ∥y∥ 2 is the cosine similarity. Hyperparameters: We follow as closely as possible the setting from SimCLRv2 with a few modifications to adapt to hardware limitations. In particular, we use the LARS optimizer (You et al., 2017) with learning rate 1.2, momentum 0.9, and weight decay 10 -4 . Iteration-wise, we scale up the learning rate for the first 40 epochs linearly, then use an iteration-wise cosine decaying schedule until epoch 800, with temperature τ = 0.1. We train on 8 V 100 GPUs with an accumulated batch size of 1024. We compute the contrastive loss on all batch samples by aggregating the embeddings computed by each GPU. We use synchronized BatchNorm and fp32 precision and do not use a memory buffer. We use the same set of transformations, i.e., Gaussian blur and horizontal flip with probability 0.5, color jittering with probability 0.8, random crop with scale uniformly sampled in [0.08, 1], and grayscale with probability 0.2. For computational simplicity and comparison with previous work, we use a ResNet50 encoder architecture with final features of size 2048. Following SimCLRv2, we use a three-layer fully connected contrastive head with hidden layers of width 2048 using ReLU activation and batchNorm and set the last layer projection to dimension 128. For evaluation, we use the features produced by the encoder without the contrastive head. We do not, at any point, use supervised fine-tuning.

4. MMD TWO-SAMPLE TEST

The Maximum Mean Discrepancy (MMD) is a statistic used in the MMD two-sample test to assess whether two sets of samples S P and S Q are drawn from the same distribution. It estimates the expected difference between the intra-set distances and the across-sets distances. Definition 4.1 (Gretton et al. (2012) ). Let k : X × X → R be the kernel of a reproducing Hilbert space H k , with feature maps k( •, x) ∈ H k . Let X, X ′ ∼ P and Y, Y ′ ∼ Q. Under mild integrability conditions, M M D(P, Q; H k ) := sup f ∈H,∥f ∥ H k ≤1 |E [f (X)] -E [f (Y )]| (2) = E [k(X, X ′ ) + k(Y, Y ′ ) -2k(X, Y )]. Given two sets of n samples S P = {X i } i≤n and S Q = {Y i } i≤n , respectively drawn from P and Q, we can compute the following unbiased estimator Liu et al. (2020a) : M M D 2 u (S P , S Q ; k) := 1 n(n -1) i̸ =j (k(X i , X j ) + k(Y i , Y j ) -k(X i , Y j ) -k(Y i , X j )). (4) Under the null hypothesis h 0 : P = Q, this estimator follows a normal distribution of mean 0 (Gretton et al., 2012) . Its variance can be directly estimated (Gretton et al., 2009) , but it is simpler to perform a permutation test as suggested in Sutherland et al. (2016) , which directly yields a p-value for h 0 . The idea is to use random splits X, Y of the input sample sets to obtain n perm different (though not independent) samplings of M M D 2 u (X, Y ; k), which approximate the distribution of M M D 2 u (S P , S P ; k) under the null hypothesis. Liu et al. (2020a) train a deep kernel to maximize the test power of the MMD two-sample test on a training split of the sets of samples to test. We propose instead to use our learned similarity function without any fine-tuning. Note that we return the p-value 1 nperm+1 1 + nperm i=1 1 (p i ≥ est) instead of 1 nperm nperm i=1 1 (p i ≥ est). Indeed, under the null hypothesis P = Q, est and p i are drawn from the same distribution, so for j ∈ {0, 1, . . . , n perm }, the probability for est to be smaller than exactly j elements of {p i } is 1 nperm+1 . Therefore, the probability that j elements or less of {p i } i are larger than est is j i=0 1 nperm+1 = j+1 nperm+1 . While this change has a small impact for large values of n perm , it is essential to guarantee that we indeed return a correct p-value. Notably, the algorithm of Liu et al. (2020a) has a probability 1 nperm > 0 to return an output of 0.00 even under the null hypothesis. Additionnally, we propose an improvement of MMD called MMD-CC (MMD with Clean Calibration). Instead of computing p i based on random splits of S P S Q , we require as input two disjoint sets of samples drawn from P and compute p i based on random splits of S (1) P

S

(2) P (see Algorithm 1). This change requires to use twice as many samples from P, but reduces the variance induced by the random splits of S P S Q , which is significant when the number of samples is small.  P , S P , S Q , n perm , sim est ← M M D 2 u (S P , S Q ; sim) for i = 1, 2, . . . , n perm do Randomly split S (1) P (2) P into two disjoint sets X, Y of equal size p i ← M M D 2 u (X, Y ; sim) end for Output: p: 1 1+nperm 1 + nperm i=1 1 (p i ≥ est) After years of evaluation of popular supervised architectures on the test set of CIFAR-10 ( Krizhevsky, 2009) , modern models may overfit it through their hyperparameter tuning and structural choices. CIFAR-10.1 (Recht et al., 2019) was collected to verify the performances of these models on a truly independent sample from the training distribution. The authors note a consistent drop in accuracy across models and suggest it could be due to a distributional shift, though they could not demonstrate it. Recent work (Liu et al., 2020a) leveraged the two-sample test to provide strong evidence of distributional shifts between the test sets of CIFAR-10 and CIFAR-10.1. We run MMD-CC and MMD two-sample tests for 100 different samplings of S (1) P , S P , S Q , using every time n perm = 500, and rejecting h 0 when the obtained p-value is below the threshold α = 0.05. We also report results using cosine similarity applied to the features of supervised models as a comparative baseline. We report the results in Table 1 for a range of sample sizes. We compare the results to three Other methods in the literature do not use external data for pre-training, as we do with ImageNet, which makes a fair comparison difficult. However, it is noteworthy that our learned similarity can very confidently distinguish samples from the two datasets, even in settings with fewer samples available. Furthermore, while we achieve excellent results even with a supervised network, our model trained with contrastive learning outperforms the supervised alternative very significantly. We note however that with such high number of samples available, MMD-CC performs slightly worse than MMD. Finally, we believe the confidence obtained with our method decisively concludes that CIFAR10 and CIFAR10.1 have different distributions, which is likely the primary explanation for the significant drop in performances across models on CIFAR10.1, as conjectured by Recht et al. (2019) . The difference in distribution between CIFAR10 and CIFAR10.1 is neither based on label set nor adversarial perturbations, making it an interesting task.

4.2. DETECTION OF DISTRIBUTIONAL SHIFTS FROM SMALL NUMBER OF SAMPLES

Given a small set of samples with potential unknown classes or adversarial attacks, we can similarly use the two-sample test with our similarity function to verify whether these samples are in-distribution (Gao et al., 2021) . In particular, we test for samples drawn from ImageNet-O, iNaturalist, and PGD perturbations, with sample sizes ranging from 3 to 20. For these experiments, we sample S (1)

P and S

(2) P 5000 times across all of ImageNet's validation set and compare their MMD and MMD-CC estimators to the one obtained from S P and S Q . We report in Table 2 the AUROC of the resulting detection and compare it to the ones obtained with a supervised ResNet50 as the baseline. Such a setting where we use several samples assumed to be drawn from a same distribution to perform detection is uncommon, and we are not aware of prior baselines in the literature. Despite using very few samples (3 ≤ n ≤ 20), our method can detect OOD samples with high confidence. We observe particularly outstanding performances on iNaturalist, which is easily explained by the fact that the subset we are using (cf. Section 1) only contains plant species, logically inducing an abnormally high similarity within its samples. Furthermore, we observe that MMD-CC performs significantly better than MMD, especially on detecting samples perturbed by PGD. Although our method attains excellent detection rates for sufficient numbers of samples, the requirement to have a set of samples all drawn from the same distribution to perform the test makes it unpractical for real-world applications. In the following section, we present CADet, a detection method inspired by MMD but applicable to anomaly detection with single inputs.

5. CADET: CONTRASTIVE ANOMALY DETECTION

While the numbers in Section 4 demonstrate the reliability of two-sample test coupled with contrastive learning for identifying distributional shifts, it requires several samples from the same distribution, which is generally unrealistic for practical detection purposes. This section presents CADet, a method to leverage contrastive learning for anomaly detection on single samples from OOD distributions. Self-supervised contrastive learning trains a similarity function s to maximize the similarity between augmentations of the same sample, and minimize the similarity between augmentations of different samples. Given an input sample x test , we propose to leverage this property to perform anomaly detection on x test , taking inspiration from MMD two-sample test. More precisely, given a transformation distribution T val , we compute n trs random transformations x test i of x test , as well as n trs random transformations x (k) i on each sample x (k) of a held-out validation dataset X (1) val . We then compute the intra-similarity and out-similarity: m in (x test ) := i̸ =j s(x test i , x test j ) n trs (n trs + 1) , m out (x test ) := x (k) ∈X (1) val i,j s(x test i , x (k) j ) n 2 trs × |X (1) val | . ( ) We finally define the following statistic to perform detection: score C := m in + γm out . ( ) Calibration: since we do not assume knowledge of OOD samples, it is difficult a priori to tune γ, although crucial to balance information between intra-sample similarity and cross-sample similarity. As a workaround, we calibrate γ by equalizing the variance between m in and γm out on a second set of validation samples X (2) val : γ = V ar{m in (x), x ∈ X (2) val } V ar{m out (x), x ∈ X (2) val } . ( ) Rather than evaluating the false positive rate (FPR) for a range of possible thresholds τ , we use the hypothesis testing approach to compute the p-value: p value (x test ) = |{x ∈ X (2) val s.t. score C (x) < score C (x test )}| + 1 |{X (2) val }| + 1 . Algorithm 2 and Algorithm 3 detail the calibration and the testing steps, respectively. Setting a threshold p ∈ [0, 1] for the p value will result in a FPR of mean p, with a variance dependant of |X (2) val |. Section 5.1 further describes our experimental setting. Algorithm 2 CADet calibration step Input: X (1) val , X val , T eval , learned similarity function s, various hyperparameters used below; 1: for x (1) ∈ X (1) val do 2: for i = 1, 2, . . . , n trs do 3: Sample t from T eval 4: x (1) i ← t(x (1) ) 5: end for 6: end for 7: k ← 0 8: for x (2) ∈ X (2) val do 9: for i = 1, 2, . . . , n trs do 10: Sample t from T eval 11: x (2) i ← t(x (2) ) 12: end for 13: m in k ← ntrs i̸ =j s(x (2) i , x m out k ← x (1) ∈X k ← k + 1 16: end for 17: m in ← m in ntrs(ntrs+1) 18: m out ← m out n 2 trs #{X (1) val } 19: V in , V out = V ar(m in ), V ar(m out ) 20: γ ← Vin Vout 21: k ← 0 22: for x (2) ∈ X (2) val do 23: score k ← m in k + γ × m out k 24: k ← k + 1 25: end for Output: coefficient: γ, scores: score k , transformed samples: x (1) i

Algorithm 3 CADet testing step

Input: transformed samples: x i , scores: scores, test sample: x test , coefficient: γ, trasnformation set: T eval ; 1: for i = 1, 2, . . . , n trs do 2: Sample t from T eval 3: x test i ← t(x test ) 4: end for 5: m in ← ntrs i̸ =j s(x test i , x test j ) 6: m out ← x (1) ∈X (1) val ntrs i,j s(x test i , x ) 7: score te ← m in ntrs(ntrs+1) + γ m out n 2 trs #{X val } 8: rank ← 0 9: for score val ∈ scores do 10: if score val < score te then 11: rank ← rank + 1 12: end if 13: end for 14: p val ← rank+1

#{scores}+1

Output: p-value: p val

5.1. EXPERIMENTS

For all evaluations, we use the same transformations as SimCLRv2 except color jittering, Gaussian blur and grayscaling. We fix the random crop scale to 0.75. We use |{X val }| = 300 separate samples to compute cross-similarities, and 50 transformations per sample. We pre-train a ResNet50 with ImageNet as in-distribution. Unknown classes detection: we use two challenging benchmarks for the detection of unknown classes. iNaturalist using the subset in Huang & Li (2021) made of plants with classes that do not intersect ImageNet. Wang et al. (2022) noted that this dataset is particularly challenging due to proximity of its classes. We also evaluate on ImageNet-O (Hendrycks et al., 2021) ; explicitly designed to be challenging for OOD detection with ImageNet as in-distribution. We compare to recent works and report the AUROC scores in Table 3 . Adversarial detection: for adversarial detection, we generate adversarial attacks on the validation partition of ImageNet against a pre-trained ResNet50 using three popular attacks: PGD (Madry et al., 2017), CW (Carlini & Wagner, 2017) , and FGSM (Goodfellow et al., 2014) . We follow the tuning suggested by Abusnaina et al. (2021) , i.e. PGD: norm L ∞ , δ = 0.02, step size 0.002, 50 iterations; CW: norm L 2 , δ = 0.10, learning rate of 0.03, and 50 iterations; FGSM: norm L ∞ , δ = 0.05. We compare our results with ODIN (Liang et al., 2017) , which achieves good performances in Lee et al. (2018) despite not being designed for adversarial detection, and to Hu et al. (2019) . Most other existing adversarial detection methods assume access to adversarial samples during training (see Section 2). While both of these works use adversarial samples to tune hyperparameters, we additionally propose a modification to Hu et al. (2019) to perform auto-calibration based on the mean and variance of the criterions on clean data, similarly to CADet's calibration step. We report the AUROC scores in Table 4 and illustrate them with ROC curves against each anomaly type in Figure 1b .

6.1. RESULTS

CADet performs particularly well on adversarial detection, surpassing alternatives by a wide margin. We argue that self-supervised contrastive learning is a suitable mechanism for detecting classification attacks due to its inherent label-agnostic nature. Interestingly, Hu et al. (2019) also benefits from contrastive pre-training, achieving much higher performances than with a supervised backbone. However, it is very reliant on calibrating on adversarial samples, since we observe a significant drop in performances with auto-calibration. Simultaneously, CADet performs well on detecting unknown classes, although not beating the best existing methods on iNaturalist. Notably, applying CADet to a supervised network achieves state-of-the-art performances on iNaturalist with ResNet50 architecture, suggesting CADet can be a reasonable standalone detection method on some benchmarks, independently of contrastive learning. In addition, the poor performances of the supervised network on ImageNet-O and adversarial attacks show that contrastive learning is essential to address the trade-off between different type of anomalies. Overall, our results show CADet achieves an excellent trade-off when considering both adversarial and label-based OOD samples.

6.2. THE PREDICTIVE POWER OF IN-SIMILARITIES AND OUT-SIMILARITIES

Table 5 reports the mean and variance of m in and m out , and the rescaled mean γm out across all distributions. Interestingly, we see that out-similarities m out better discriminate label-based OOD samples, while in-similarities m in better discriminate adversarial perturbations. Combining insimilarities and out-similarities is thus an essential component to simultaneously detect adversarial perturbations and unknown classes. Computational cost: To perform detection with CADet, we need to compute the features for a certain number of transformations of the test sample, incurring significant overhead. Figure 2 shows that reducing the number of transformations to minimize computational cost may not significantly affect performances. While the calibration step can be expensive, we note that it only needs to be run once for a given in-distribution. The coefficient γ and scores are all one-dimensional values that can be easily stored, and we purposely use a small number of validation samples |{X

6.3. LIMITATIONS

(1) val }| = 300 to make their embedding easy to memorize. Architecture scale: as self-supervised contrastive learning is computationally expensive, we only evaluated our method on a ResNet50 architecture. In Wang et al. (2022) , the authors achieve significantly superior performances when using larger, recent architectures. The performances achieved with a ResNet50 are insufficient for real-world usage, and the question of how our method would scale to larger architectures remains open.

6.4. FUTURE DIRECTIONS

While spurious correlations with background features are a problem in supervised learning, it is aggravated in self-supervised contrastive learning, where background features are highly relevant to the training task. We conjecture the poor performances of CADet on iNaturalist OOD detection are explained by the background similarities with ImageNet images, obfuscating the differences in relevant features. A natural way to alleviate this issue is to incorporate background transformations to the training pipeline, as was successfully applied in Ma et al. (2018) . This process would come at the cost of being unable to detect shifts in background distributions, but such a case is generally less relevant to deployed systems. We leave to future work the exploration of how background transformations could affect the capabilities of CADet.

6.5. CONCLUSION

We have presented CADet, a method for both OOD and adversarial detection based on self-supervised contrastive learning. CADet achieves an excellent trade-off in detection power across different anomaly types. Additionally, we discussed how MMD could be leveraged with contrastive learning to assess distributional discrepancies between two sets of samples.



4.1 DISTRIBUTION SHIFT BETWEEN CIFAR-10 AND CIFAR-10.1 TEST SETS Algorithm 1 MMD-CC two-sample test

Figure 1: ROC curves for MMD-CC two-sample test on CIFAR10 vs CIFAR10.1 using different sample sizes (left) and for CADet anomaly detection on different out-distributions (right).

3 73.2 88.0 97.7 95.4 99.2 99.5 99.5 70.5 84.0 96.6 99.5 MMD + Supervised 62.7 69.7 83.2 96.4 91.8 98.7 99.5 99.5 20.0 22.5 33.0 57.5 MMD-CC + Supervised 62.6 71.0 85.5 97.2 98.0 99.5 99.5 99.5 57.4 61.3 70.5 85.8

Figure 2: AUROC score of CADet against the number of transformations.



AUROC for detection using two-sample test on 3 to 20 samples drawn from ImageNet and from ImageNet-O, iNaturalist or PGD perturbations, with a ResNet50 backbone.

AUROC for OOD detection on ImageNet-O and iNaturalist with ResNet50 backbone.

AUROC for adversarial detection on ImageNet against PGD, CW and FGSM attacks, with ResNet50 backbone.

Mean and variance of m in and m out . Var m in 8.3e-05 7.8e-05 1.0e-04 2.1e-04 2.0e-04 2.1e-04 m out 1.7e-03 7.0e-04 2.3e-03 7.0e-04 1.7e-03 1.1e-03

