TOWARDS UNDERSTANDING HOW MACHINES CAN LEARN CAUSAL OVERHYPOTHESES

Abstract

Recent work in machine learning and cognitive science has suggested that understanding causal information is essential to the development of intelligence. One of the key challenges for current machine learning algorithms is modeling and understanding causal overhypotheses: transferable abstract hypotheses about sets of causal relationships. In contrast, even young children spontaneously learn causal overhypotheses, and use these to guide their exploration or to generalize to new situations. This has been demonstrated in a variety of cognitive science experiments using the "blicket detector" environment. We present a causal learning benchmark adapting the "blicket" environment for machine learning agents and evaluate a range of state-of-the-art methods in this environment. We find that although most agents have no problem learning causal structures seen during training, they are unable to learn causal overhypotheses from these experiences, and thus cannot generalize to new settings.

1. INTRODUCTION

Over the past few years, deep learning has made impressive progress in many areas, including reinforcement learning, natural language processing, and computer vision. However, most stateof-the-art algorithms still pale in comparison to humans, even children, for out-of-distribution generalization and fast adaptation to new tasks. In contrast, causal modeling has long concerned itself not merely with accurate modeling of in-distribution data, but also the accurate recovery of underlying causal mechanisms (and their true graphical relations) capable of explaining out-of-distribution data. Hence causal modeling holds promise for achieving systematic generalization (Bengio et al., 2019; Schölkopf et al., 2021; Ke et al., 2021) . One of the key components of causal learning for humans are causal overhypotheses, which describe priors over causal graphs. For example, a causal hypothesis may be that a system is stochastic, or that it is conjuntive-i.e., more than one cause must be present in order for the effect to occur. Causal overhypotheses are important because they enable humans to learn causal models from a sparse amount of data (Griffiths & Tenenbaum, 2009) , by reducing the set of possible causal relationships to consider. Despite the recent surge of machine learning datasets and environment for causal inference and learning (Ahmed et al., 2020; McDuff et al., 2021; Wang et al., 2021a; Ke et al., 2021) , the causal overhypotheses for these environments and datasets are unclear. Thus we cannot use these existing benchmarks to evaluate how capable are agents are at learning and using causal overhypotheses. In this work, we seek to fill this gap, by focusing on introducing a benchmark environment that draws inspiration from recent cognitive science work using blicket detectors. This environment contains a virtual blicket detector, and has been used previously for understanding causal learning and exploration in children (Kosoy et al., 2022) and basic RL models, but not yet for state-of-the-art machine learning agents. A "blicket detector" is a machine that lights up and plays music when some combinations of objects but not others are placed on it (Gopnik & Sobel, 2000; Lucas et al., 2014) . The central question is whether an agent can learn that a particular set of causal events will lead to the lighting-up effect, and use that knowledge to design novel interventions on the machine. The causal relationship is entirely determined by the pattern of conditional dependencies and interventions, rather than requiring intuitive physics knowledge or visual understanding. Although these tasks may seem simple, and are easily mastered by children, we find that they are challenging for current learning algorithms. Several features of this environment and the tasks it allows make it particularly useful as a benchmark for machine learning algorithms. First, causal representations are more powerful and structured than mere statistical generalizations, though both can be systematically inferred from statistical information. Many researchers (e.g. Pearl, Spirtes et al., Bengio) have argued that such causal representations are crucial for both human and general AI intelligence. The goals of causal inference, namely learning which actions will alter the environment in particular ways, are similar to those of standard reinforcement learning, but adding causal representations and inferences makes such learning far more effective. Second, and unlike some existing causal environments (Ke et al., 2021; Wang et al., 2021a) , the blicket environment enables the inference of both specific causal structure and more general features of causal structure, such as whether causal systems are conjunctive or disjunctive, stochastic or deterministic. Learning these overhypotheses about causal structure (Griffiths & Tenenbaum, 2009) is especially important because such inferences can constrain the search for causal structure in the future, a search that can rapidly become unwieldy. Third, this environment allows for active learning and exploration in a way that is both sufficiently constrained to be practical and that also allows for informative interventions. Most significantly, and again unlike existing environments, research has already shown that even preschool children can easily manipulate and explore this environment, generate appropriate data, and rapidly learn both particular causal structure and overhypotheses about causal structure (Kosoy et al., 2022) . We can then directly compare both the overall performance and the behavior of various agents in these tasks to the performance and behavior of children. Young children are a particularly informative baseline group. They do not have the extensive education and experience of typical adults, which might make comparisons to artificial agents challenging, but they are nevertheless effective causal learners and able to make broad yet accurate generalizations from small sample sizes, in contrast to many current machine learning systems (Gopnik, 2012; Gopnik et al., 2017) . The work of Kosoy et al. (2022) first described a version of this environment and included data on children and a few learning algorithms. But the environment and benchmarks have much wider applications than in that preliminary study or the studies reported here. It allows multiple comparisons of children and agents on multiple tasks and allows direct comparison with ML systems for multiple kinds of causal inference tasks. In Kosoy et al. (2022) and in the experiments described here we focus on one such inference -to conjunctive versus disjunctive structure, and a few algorithms. But with very minor changes the environment would allow tests of inferences to many other kinds of overhypotheses and inductive biases, such as inferring whether systems are stochastic or deterministic or require sequential or unordered interventions, overhypotheses that are also important for causal inference. It also allows multiple measures of causal inference including interventions and counterfactuals as well as predictions. And, significantly, it would allow researchers to empirically record the exploration behavior that children use in solving causal problems, not yet done this in this paper, and compare it to a wide variety of increasingly influential exploration based ML procedures. In what follows, we look at a spectrum of algorithms ranging from reinforcement learning, to imitation learning, to the use of pre-trained language models, to better understand how these different methods perform on the proposed environment and tasks. We find that such algorithms, in contrast to children, only converge on a solution after an extensive number of trials or if they are given all the possible settings and outcomes as training data. This suggests that these tasks are an interesting challenge for machine learning algorithms. In order for machines to perform as well as children do, algorithms must reason about the sequence of observations seen, extract causal overhypotheses from those observations and use them for exploration-which current methods fall short of doing.

2. RELATED WORK

Exploration in Reinforcement Learning Recent exploration algorithms for deep reinforcement learning typically add an exploration bonus to the task reward; please refer to Amin et al. ( 2021) for a comprehensive survey. This bonus could be based on novelty (Bellemare et al., 2016; Ostrovski et al., 2017; Martin et al., 2017; Tang et al., 2017; Machado et al., 2018a) , dynamics prediction error (Schmidhuber, 1991; Pathak et al., 2017 ), uncertainty (Osband et al., 2016; Burda et al., 2018) , or disagreement (Pathak et al., 2019) . The vast majority of existing exploration methods do not involve causal reasoning, whereas our proposed environment requires algorithms to learn and use causal overhypotheses in order to explore effectively, to solve the task.

