TAKING A STEP BACK WITH KCAL: MULTI-CLASS KERNEL-BASED CALIBRATION FOR DEEP NEURAL NETWORKS

Abstract

Deep neural network (DNN) classifiers are often overconfident, producing miscalibrated class probabilities. In high-risk applications like healthcare, practitioners require fully calibrated probability predictions for decision-making. That is, conditioned on the prediction vector, every class' probability should be close to the predicted value. Most existing calibration methods either lack theoretical guarantees for producing calibrated outputs, reduce classification accuracy in the process, or only calibrate the predicted class. This paper proposes a new Kernel-based calibration method called KCal. Unlike existing calibration procedures, KCal does not operate directly on the logits or softmax outputs of the DNN. Instead, KCal learns a metric space on the penultimate-layer latent embedding and generates predictions using kernel density estimates on a calibration set. We first analyze KCal theoretically, showing that it enjoys a provable full calibration guarantee. Then, through extensive experiments across a variety of datasets, we show that KCal consistently outperforms baselines as measured by the calibration error and by proper scoring rules like the Brier Score. Our code is available at https://github.com/zlin7/KCal.

1. INTRODUCTION

The notable successes of Deep Neural Networks (DNNs) in complex classification tasks, such as object detection (Ouyang & Wang, 2013 ), speech recognition (Deng et al., 2013) , and medical diagnosis (Qiao et al., 2020; Biswal et al., 2017) , have made them essential ingredients within various critical decision-making pipelines. In addition to the classification accuracy, a classifier should ideally also generate reliable uncertainty estimates represented in the predicted probability vector. An influential study (Guo et al., 2017) reported that modern DNNs are often overconfident or miscalibrated, which could lead to severe consequences in high-stakes applications such as healthcare (Jiang et al., 2012) . Calibration is the process of closing the gap between the prediction and the ground truth distribution given this prediction. For a K-classification problem, with covariates X ∈ X and the label Y ∈ Y = [K], denote our classifier X → ∆ K-1 as p = [p 1 , . . . , pK ], with ∆ K-1 being (K-1)-simplex. Then, Definition 1. (Full Calibration (Vaicenavicius et al., 2019)  ) p is fully-calibrated if ∀k ∈ [K]: ∀q = [q1, . . . , qK ] ∈ ∆ K-1 , P{Y = k|p(X) = q} = q k . (1) It is worth noting that Def. ( 1) implies nothing about accuracy. In fact, ignoring X and simply predicting π, the class frequency vector, results in a fully calibrated but inaccurate classifier. As a result, our goal is always to improve calibration while maintaining accuracy. Another important requirement is that p ∈ ∆ K-1 . Many binary calibration methods such as Zadrozny & Elkan (2001; 2002) result in vectors that are not interpretable as probabilities, and have to be normalized. Many existing works only consider confidence calibration (Guo et al., 2017; Zhang et al., 2020; Wenger et al., 2020; Ma & Blaschko, 2021) , a much weaker notion than that encapsulated by Def. (1) and only calibrates the predicted class (Kull et al., 2019; Vaicenavicius et al., 2019) .

