SCHEDULE-ROBUST ONLINE CONTINUAL LEARNING

Abstract

A continual learning (CL) algorithm learns from a non-stationary data stream. The non-stationarity is modeled by some schedule that determines how data is presented over time. Most current methods make strong assumptions on the schedule and have unpredictable performance when such requirements are not met. A key challenge in CL is thus to design methods robust against arbitrary schedules over the same underlying data, since in real-world scenarios schedules are often unknown and dynamic. In this work, we introduce the notion of schedule-robustness for CL and a novel approach satisfying this desirable property in the challenging online class-incremental setting. We also present a new perspective on CL, as the process of learning a schedule-robust predictor, followed by adapting the predictor using only replay data. Empirically, we demonstrate that our approach outperforms existing methods on CL benchmarks for image classification by a large margin.

1. INTRODUCTION

A hallmark of natural intelligence is its ability to continually absorb new knowledge while retaining and updating existing one. Achieving this objective in machines is the goal of continual learning (CL). Ideally, CL algorithms learn online from a never-ending and non-stationary stream of data, without catastrophic forgetting (McCloskey & Cohen, 1989; Ratcliff, 1990; French, 1999) . The non-stationarity of the data stream is modeled by some schedule that defines what data arrives and how its distribution evolves over time. Two family of schedules commonly investigated are task-based (De Lange et al., 2021) and task-free (Aljundi et al., 2019b) . The task-based setting assumes that new data arrives one task at a time and data distribution is stationary for each task. Many CL algorithms (e.g., Buzzega et al., 2020; Kirkpatrick et al., 2017; Hou et al., 2019) thus train offline, with multiple passes and shuffles over task data. The task-free setting does not assume the existence of separate tasks but instead expects CL algorithms to learn online from streaming data, with evolving sample distribution (Caccia et al., 2022; Shanahan et al., 2021) . In this work, we tackle the task-free setting with focus on class-incremental learning, where novel classes are observed incrementally and a single predictor is trained to discriminate all of them (Rebuffi et al., 2017) . Existing works are typically designed for specific schedules, since explicitly modeling and evaluating across all possible data schedules is intractable. Consequently, methods have often unpredictable performance when scheduling assumptions fail to hold (Farquhar & Gal, 2018; Mundt et al., 2022; Yoon et al., 2020) . This is a considerable issue for practical applications, where the actual schedule is either unknown or may differ from what these methods were designed for. This challenge calls for an ideal notion of schedule-robustness: CL methods should behave consistently when trained on different schedules over the same underlying data. To achieve schedule-robustness, we introduce a new strategy based on a two-stage approach: 1) learning online a schedule-robust predictor, followed by 2) adapting the predictor using only data from experience replay (ER) (Chaudhry et al., 2019b) . We will show that both stages are robust to diverse data schedules, making the whole algorithm schedule-robust. We refer to it as SChedule-Robust Online continuaL Learning (SCROLL). Specifically, we propose two online predictors that by design are robust against arbitrary data schedules and catastrophic forgetting. To learn appropriate priors for these predictors, we present a meta-learning perspective (Finn et al., 2017; Wang et al., 2021) and connect it to the pre-training strategies in CL (Mehta et al., 2021) . We show that pre-training offers an alternative and efficient procedure for learning predictor priors instead of directly solving the meta-learning formulation. This makes our method computationally competitive and at the same

