THALAMUS: A BRAIN-INSPIRED ALGORITHM FOR BIOLOGICALLY-PLAUSIBLE CONTINUAL LEARNING AND DISENTANGLED REPRESENTATIONS

Abstract

Animals thrive in a constantly changing environment and leverage the temporal structure to learn well-factorized causal representations. In contrast, traditional neural networks suffer from forgetting in changing environments and many methods have been proposed to limit forgetting with different trade-offs. Inspired by the brain thalamocortical circuit, we introduce a simple algorithm that uses optimization at inference time to generate internal representations of the current task dynamically. The algorithm alternates between updating the model weights and a latent task embedding, allowing the agent to parse the stream of temporal experience into discrete events and organize learning about them. On a continual learning benchmark, it achieves competitive end average accuracy by mitigating forgetting, but importantly, by requiring the model to adapt through latent updates, it organizes knowledge into flexible structures with a cognitive interface to control them. Tasks later in the sequence can be solved through knowledge transfer as they become reachable within the well-factorized latent space. The algorithm meets many of the desiderata of an ideal continually learning agent in open-ended environments, and its simplicity suggests fundamental computations in circuits with abundant feedback control loops such as the thalamocortical circuits in the brain

1. INTRODUCTION

Animals thrive in a constantly changing environmental demands at many time scales. Biological brains seem capable of using these changes advantageously and leverage the temporal structure to learn causal and well-factorized representations (Collins & Koechlin, 2012; Yu et al., 2021; Herce Castañón et al., 2021) . In contrast, traditional neural networks suffer in such settings with sequential experience and display prominent interference between old and new learning limiting most training paradigms to using shuffled data (McCloskey & Cohen, 1989) 2021)). However, in addition to mitigating forgetting, several desirable properties in a continually learning agent have been recently suggested (Hadsell et al., 2020; Veniat et al., 2021) including: accuracy on many tasks at the end of a learning episode or at least fast adaptation and recovery of accuracy with minimal additional training. The ideal agent would also display knowledge transfer forward, to future tasks and backwards to previously learned tasks, but also transfer to tasks with slightly different computation and or slightly different input or output distributions (Veniat et al., 2021) . The algorithm should scale favorably with the number of tasks and maintain plasticity, or the capacity for further learning, Finally, the agent should ideally able to function unsupervised and not rely on access to task labels and task boundaries (Hadsell et al., 2020; Rao et al., 2019) . We argue for another critical feature: contextual behavioral, where the same inputs may require different responses at different times, a feature that might constrain the solution space to be of more relevance to brain function and to the full complexity of the world. A learning agent might struggle to identify reliable contextual signals in high dimensional input space, 1



. Many recent methods advanced the flexibility of neural networks (for recent reviews, see Parisi et al. (2019); Hadsell et al. (2020); Veniat et al. (

