SIMULATING TASK-FREE CONTINUAL LEARNING STREAMS FROM EXISTING DATASETS

Abstract

Task-free continual learning is the subfield of machine learning that focuses on learning online from a stream whose distribution changes continuously over time. However, previous works evaluate task-free continual learning using streams with distributions that change only at a few distinct points in time. In order to address the discrepancy between the definition and evaluation of task-free continual learning, we propose a principled algorithm that can permute any labeled dataset into a stream that is continuously nonstationary. We empirically show that the streams generated by our algorithm are less structured than the ones conventionally used in the literature. Moreover, we use our simulated task-free streams to benchmark multiple methods applicable to the task-free setting. We hope that our work will make it more likely that task-free continual learning methods are able to better generalize to real-world problems.

1. INTRODUCTION

The dominant paradigm in the field of machine learning involves building a model using a static set of pre-collected data (Mitchell, 1997; LeCun et al., 2015) . Unfortunately, it might not be always possible to stick to this paradigm. For instance, animals and humans extract knowledge from their observations continually, and under changing circumstances (Parisi et al., 2019) . The field of continual learning studies exactly this problem-namely, how to train a machine learning model using data provided by a nonstationary distribution (Aljundi et al., 2019c; Chrysakis & Moens, 2020) . Within the continual learning literature, different underlying assumptions give rise to a number of distinct continual learning settings. Such assumptions might be about whether the data distribution is continuously nonstationary or not, or about whether the model optimization takes place online (with small minibatches of data) or offline (with large batches of data) (De Lange et al., 2021) . In this paper, we focus on task-free continual learning, which we consider to be the setting closest to how humans and animals learn. In task-free continual learning, the data distribution is assumed to be continuously nonstationary and the optimization takes place online (Aljundi et al., 2019b) . The observation that motivated this work is that there is a large gap between how task-free continual learning is defined and how it is evaluated. In fact, previous works evaluate task-free continual learning using streams with data distributions that are not continuously nonstationary, but change only at a few distinct moments in time and remain stationary otherwise (Aljundi et al., 2019b; Jin et al., 2021) . With this work, we aim to bridge the gap between the definition and the evaluation of task-free continual learning. Our contributions are the following. First, we provide a principled algorithm that can reorder any labeled dataset into a simulated task-free (STF) continual learning stream. This algorithm was designed with the goal of introducing as little design bias as possible to the streams it constructs. Second, we perform a detailed comparison between STF streams generated by the proposed algorithm and the type of streams conventionally used in previous works. Via this comparison, we detail a number of different ways the streams conventionally used are different to our STF streams. Third, we transform four well-known datasets into STF streams, and use them to benchmark a number of methods applicable to task-free continual learning. The remainder of the paper is structured as follows. In Section 2, we provide an introduction to continual learning and online continual learning, and extensively discuss the gap between the definition 1

