RISK CONTROL FOR ONLINE LEARNING MODELS

Abstract

To provide rigorous uncertainty quantification for online learning models, we develop a framework for constructing uncertainty sets that provably control risksuch as coverage of confidence intervals, false negative rate, or F1 score-in the online setting. This extends conformal prediction to apply to a larger class of online learning problems. Our method guarantees risk control at any userspecified level even when the underlying data distribution shifts drastically, even adversarially, over time in an unknown fashion. The technique we propose is highly flexible as it can be applied with any base online learning algorithm (e.g., a deep neural network trained online), requiring minimal implementation effort and essentially zero additional computational cost. We further extend our approach to control multiple risks simultaneously, so the prediction sets we generate are valid for all given risks. To demonstrate the utility of our method, we conduct experiments on real-world tabular time-series data sets showing that the proposed method rigorously controls various natural risks. Furthermore, we show how to construct valid intervals for an online image-depth estimation problem that previous sequential calibration schemes cannot handle.

1. INTRODUCTION

To confidently deploy learning models in high-stakes applications, we need both high predictive accuracy and reliable safeguards to handle unanticipated changes in the underlying data-generating process. Reasonable accuracy on a fixed validation set is not enough, as raised by Sullivan (2015); we must also quantify uncertainty to correctly handle hard input points and take into account shifting distributions. For example, consider the application of autonomous driving, where we have a real-time view of the surroundings of the car. To successfully operate such an autonomous system, we should measure the distance between the car and close-by objects, e.g., via a sensor that outputs a depth image whose pixels represent the distance of the objects in the scene from the camera. Figure 1a displays a colored image of a road and Figure 1b presents its corresponding depth map. Since high-resolution depth measurements often require longer acquisition time compared to capturing a colored image, there were developed online estimation models to predict the depth map from a given RGB image (Patil et al., 2020; Zhang et al., 2020) . The goal of these methods is to artificially speed-up depth sensing acquisition time. However, making decisions solely based on an estimate of the depth map is insufficient as the predictive model may not be accurate enough. Furthermore, the distribution can vary greatly and drastically over time, rendering the online model to output highly inaccurate and unreliable predictions. In these situations, it is necessary to design a predictive system that reflects the range of plausible outcomes, reporting the uncertainty in the prediction. To this end, we encode uncertainty in a rigorous manner via prediction intervals/sets that augment point predictions and have a long-range error control. In the autonomous driving example, the uncertainty in the depth map estimate is represented by depth-valued uncertainty intervals. In this paper, we introduce a novel calibration framework that can wrap any online learning algorithm (e.g., an LSTM model trained online) to construct prediction sets with guaranteed validity. Formally, suppose an online learning setting where we are given data stream {(X t , Y t )} t∈N in a sequential fashion, where X t ∈ X is a feature vector and Y t ∈ Y is a target variable. In single-output regression settings Y = R, while in classification tasks Y is a finite set of all class labels. The input X t is commonly a feature vector, i.e., X = R p , although it may take different forms, as in the depth sensing task, where X t ∈ R M ×N ×3 is an RGB image and Y t ∈ R M ×N is the ground truth depth. Consider a loss function L(Y t , Ĉt (X t )) ∈ R that measures the error of the estimated prediction

