SIMULATING TASK-FREE CONTINUAL LEARNING STREAMS FROM EXISTING DATASETS

Abstract

Task-free continual learning is the subfield of machine learning that focuses on learning online from a stream whose distribution changes continuously over time. However, previous works evaluate task-free continual learning using streams with distributions that change only at a few distinct points in time. In order to address the discrepancy between the definition and evaluation of task-free continual learning, we propose a principled algorithm that can permute any labeled dataset into a stream that is continuously nonstationary. We empirically show that the streams generated by our algorithm are less structured than the ones conventionally used in the literature. Moreover, we use our simulated task-free streams to benchmark multiple methods applicable to the task-free setting. We hope that our work will make it more likely that task-free continual learning methods are able to better generalize to real-world problems.

1. INTRODUCTION

The dominant paradigm in the field of machine learning involves building a model using a static set of pre-collected data (Mitchell, 1997; LeCun et al., 2015) . Unfortunately, it might not be always possible to stick to this paradigm. For instance, animals and humans extract knowledge from their observations continually, and under changing circumstances (Parisi et al., 2019) . The field of continual learning studies exactly this problem-namely, how to train a machine learning model using data provided by a nonstationary distribution (Aljundi et al., 2019c; Chrysakis & Moens, 2020) . Within the continual learning literature, different underlying assumptions give rise to a number of distinct continual learning settings. Such assumptions might be about whether the data distribution is continuously nonstationary or not, or about whether the model optimization takes place online (with small minibatches of data) or offline (with large batches of data) (De Lange et al., 2021) . In this paper, we focus on task-free continual learning, which we consider to be the setting closest to how humans and animals learn. In task-free continual learning, the data distribution is assumed to be continuously nonstationary and the optimization takes place online (Aljundi et al., 2019b) . The observation that motivated this work is that there is a large gap between how task-free continual learning is defined and how it is evaluated. In fact, previous works evaluate task-free continual learning using streams with data distributions that are not continuously nonstationary, but change only at a few distinct moments in time and remain stationary otherwise (Aljundi et al., 2019b; Jin et al., 2021) . With this work, we aim to bridge the gap between the definition and the evaluation of task-free continual learning. Our contributions are the following. First, we provide a principled algorithm that can reorder any labeled dataset into a simulated task-free (STF) continual learning stream. This algorithm was designed with the goal of introducing as little design bias as possible to the streams it constructs. Second, we perform a detailed comparison between STF streams generated by the proposed algorithm and the type of streams conventionally used in previous works. Via this comparison, we detail a number of different ways the streams conventionally used are different to our STF streams. Third, we transform four well-known datasets into STF streams, and use them to benchmark a number of methods applicable to task-free continual learning. The remainder of the paper is structured as follows. In Section 2, we provide an introduction to continual learning and online continual learning, and extensively discuss the gap between the definition and evaluation of task-free continual learning. In Section 3, we present our algorithm for generating STF streams, and motivate its design. In Section 4, we present and discuss our experiments, and, finally, in Section 5, we summarize our work, discuss its limitations, and offer a future perspective.

2. BACKGROUND

2.1 CONTINUAL LEARNING In general, continual learning is defined as learning from data that are generated by a nonstationary distribution, that is to say, a distribution that changes over time (Zenke et al., 2017; Kurle et al., 2019; Chrysakis & Moens, 2020 ). An alternative definition of continual learning is the learning of a sequence of tasks over time ( Van de Ven & Tolias, 2019; Prabhu et al., 2020; De Lange et al., 2021) . But, what is a task? In the context of continual learning, the term task is generally used to describe a collection of data which the model observes in an independent and identically distributed (iid) manner. Tasks are often assumed to be class-disjoint, that is, if data from a particular class appear in a task, no data from the same class will be present in any other task (Van de Ven & Tolias, 2019; Prabhu et al., 2020) . Previous works sometimes assume access to task labels, which explicitly inform the learner to which task each data point belongs (Nguyen et al., 2018; Zenke et al., 2017) . The setting of class-incremental continual learning assumes that task labels are only given during training, while the setting of task-incremental continual learning assumes access to task labels both during training and during evaluation (Van de Ven & Tolias, 2019; De Lange et al., 2021; Lomonaco & Rish, 2021) . Besides access to task labels, another distinction can be made with regard to whether continual learning takes place online or offline. In the offline setting, the learner has access to all data from the present task and can perform multiple passes over these data (De Lange et al., 2021; Prabhu et al., 2020) . Conversely, in the online setting, the learner receives data from a nonstationary stream in the form of small minibatches, and only has access to one of those minibatches at a time (Aljundi et al., 2019c; Chrysakis & Moens, 2020; Cai et al., 2021) .

2.2. ONLINE CONTINUAL LEARNING SETTINGS

To avoid potential confusion, we offer precise definitions for online, task-agnostic, and task-free continual learning. First, online continual learning has evolvedfoot_0 to be an umbrella term that encompasses all settings in which a model should be trained online using small minibatches of data that are generated by a nonstationary stream (Aljundi et al., 2019a; Pham et al., 2020; Yin et al., 2021) . Task-agnostic and task-free continual learning are both types of online continual learning. In taskagnostic continual learning the stream is assumed to be a sequence of tasks but without task labels being available. In other words, the stream consists of a number contiguous iid sub-streams (each one corresponding to a task), and the distribution only changes when there is a transition from one sub-stream to the next. In this setting, however, it is relatively easy to infer task labels during training (Zeno et al., 2018; Kirichenko et al., 2021) . Finally, in task-free continual learning the concept of a data distribution that changes at distinct points during learning, is generalized to one that changes constantly over time (Aljundi et al., 2019b) . Therefore, in a task-free stream, there are no iid sub-streams, hence the concepts of tasks, task labels, and task boundaries cannot be defined.

2.3. TASK-FREE CONTINUAL LEARNING AND ITS EVALUATION

We argue that, in terms of its applicability, task-free continual learning is the most general continual learning setting. To understand why, we need to consider the various aforementioned settings in the context of the simplifying assumptions they make. The most widely adopted assumptions are a) the existence of tasks, b) task labels during training, c) concurrent access to all data from the present



We write evolved because online continual learning was originally defined to be a nonstationary online learning problem without access to task labels(Aljundi et al., 2019c). A number of subsequent works, however, study online continual learning and do assume access to task labels(Pham et al., 2020; Yin et al., 2021).

