

Abstract

While deep networks can learn complex functions such as classifiers, detectors, and trackers, many applications require models that continually adapt to changing input distributions, changing tasks, and changing environmental conditions. Indeed, this ability to continuously accrue knowledge and use past experience to learn new tasks quickly in continual settings is one of the key properties of an intelligent system. For complex and high-dimensional problems, simply updating the model continually with standard learning algorithms such as gradient descent may result in slow adaptation. Meta-learning can provide a powerful tool to accelerate adaptation yet is conventionally studied in batch settings. In this paper, we study how metalearning can be applied to tackle online problems of this nature, simultaneously adapting to changing tasks and input distributions and meta-training the model in order to adapt more quickly in the future. Extending meta-learning into the online setting presents its own challenges, and although several prior methods have studied related problems, they generally require a discrete notion of tasks, with known ground-truth task boundaries. Such methods typically adapt to each task in sequence, resetting the model between tasks, rather than adapting continuously across tasks. In many real-world settings, such discrete boundaries are unavailable, and may not even exist. To address these settings, we propose a Fully Online Meta-Learning (FOML) algorithm, which does not require any ground truth knowledge about the task boundaries and stays fully online without resetting to pre-trained weights. Our experiments show that FOML was able to learn new tasks faster than the state-of-the-art online learning methods on various datasets.

1. INTRODUCTION

Flexibility and rapid adaptation are a hallmark of intelligence: humans can not only solve complex problems, but they can also figure out how to solve them very rapidly, as compared to our current machine learning algorithms. Such rapid adaptation is crucial for both humans and computers: for humans, it is crucial for survival in changing natural environments, and it is also crucial for agents that classify photographs on the Internet, interpret text, control autonomous vehicles, and generally make accurate predictions with rapidly changing real-world data. While deep neural networks are remarkably effective for learning and representing accurate models He et al. ( 2015 Hospedales (2020) , where a constant stream of data from distinct tasks is used for both adaptation and meta-training. In this scheme, meta-training is used to accelerate how quickly the network can adapt to each new task it sees, and simultaneously use that data from each new task for meta-training. This further accelerates how quickly each subsequent task can be acquired. However, current online meta-learning methods fall short of the goal of creating an effective adaptation system for online data in several ways: (1) they typically require task boundaries in the data stream to be known, making them ill-suited to settings where task boundaries are ill-defined and tasks change or evolve gradually, a common tread in real-world; (2) as a result, they typically re-adapt from the meta-trained model



); Krizhevsky et al. (2012); Simonyan & Zisserman (2014); Szegedy et al. (2015), they are comparatively unimpressive when it comes to adaptability, due to their computational and data requirements. Meta-learning in principle mitigates this problem, by leveraging the generalization power of neural networks to accelerate adaptation to new tasks Finn et al. (2019); Li et al. (2017); Nichol et al. (2018); Nichol & Schulman (2018); Park & Oliva (2019); Antoniou et al. (2018). However, standard meta-learning algorithms operate in batch mode, making them poorly suited for continuously evolving environments. More recently, online meta-learning methods have been proposed with the goal of enabling continual adaptation Finn et al. (2019); Jerfel et al. (2018); Yao et al. (2020); Nagabandi et al. (2018); Li &

