ACTIVE LEARNING BASED STRUCTURAL INFERENCE

Abstract

In this paper, we propose an active-learning based framework, Active Learning based Structural Inference (ALaSI), to infer the existence of directed connections from observed agents' states over a time period in a dynamical system. With the help of deep active learning, ALaSI is competent in learning the representation of connections with relatively small pool of prior knowledge. Moreover, based on information theory, we propose inter-and out-of-scope message learning pipelines, which are remarkably beneficial to the structural inference for large dynamical systems. We evaluate ALaSI on various large datasets including simulated systems and real-world networks, to demonstrate that ALaSI is able to precisely infer the existence of connections in these systems under either supervised learning or unsupervised learning, with better performance than baseline methods.

1. INTRODUCTION

Dynamical systems are commonly observed in real-world, including physical systems (Kwapień & Drożdż, 2012; Ha & Jeong, 2021) , biological systems (Tsubaki et al., 2019; Pratapa et al., 2020) , and multi-agent systems (Brasó & Leal-Taixé, 2020; Li et al., 2022) . A dynamical system can be described as a set of three core elements: (a) the state of the system in a time period, including state of the individual agents; (b) the state-space of the system; and (c) the state-transition function (Irwin & Wang, 2017) . Knowing these core elements, we can describe and predict how a dynamical system behaves. Yet the three elements are not independent of each other, for example, the evolution of the state is affected by the state-transition function, which suggests that we may predict the future state based on its current state and the entities which affect the agents (i.e. connectivity). Moreover, the state-transition function is often deterministic (Katok & Hasselblatt, 1995) , which simplifies the derivation of the future state as a Markovian transition function. However, in most cases, we hardly have access to the connectivity within a given system, or only have limited knowledge about the connectivity. Is it possible to infer the connectivity from observed states of the agents over a time period? We formulate it as the problem of structural inference, and several machine learning frameworks have been proposed to address it (Kipf et al., 2018; Webb et al., 2019; Alet et al., 2019; Chen et al., 2021; Löwe et al., 2022; Wang & Pang, 2022) . Although these frameworks can accurately infer the connectivity, as they perform representation learning on a fully connected graph, these methods can only work for small systems (up to dozens of agents), and cannot scale well to real-world dynamical systems, for example, with hundreds of agents. Besides that, as we show in the experiment and appendix sections in this work, the integration of prior knowledge about partial connectivity of the system is quite problematic among these methods. On the other hand, deep active learning (DeepAL) is an emerging branch of research that is used to reduce the cost of annotation while retaining the powerful learning capabilities of deep learning (Ren et al., 2022) . This motivates us to explore DeepAL to solve the problem of structural inference. In order to perform structural inference on large dynamical systems, instead of building pools based on batches, we build pools based on agents, and expect the learning framework can consequently infer the existence of directed connections with a little prior knowledge of the connections. Therefore, in this work, based on DeepAL, we propose a novel structural inference framework, namely, Active Learning based Structural Inference (ALaSI), which is designed for the structural inference of large dynamical systems, and is suitable for the integration of prior knowledge. ALaSI leverages query strategy with dynamics for agent-wise selection to update the pool with the most informative partial system, which encourages ALaSI to infer the connections efficiently and accurately with partial prior knowledge on the connectivity (named as 'scope'). Furthermore, based on information theory, ALaSI learns both inter-scope and out-of-scope (OOS) messages from the current scope to distinguish the information which represents connections from agents within the scope and from agents out of the scope, which reserves redundancy when new agents come into scope. Moreover, with oracle such as partial information decomposition (PID) (Williams & Beer, 2010) , ALaSI can infer the connectivity even without prior knowledge and be trained in an unsupervised way. We show with extensive experiments that ALaSI can infer the directed connections of dynamical systems with up to 1.5K agents with either supervised learning or unsupervised learning.

2. RELATED WORK

Deep Active learning. Our framework ALaSI follows the strategy of DeepAL (Ren et al., 2022) , which attempts to combine the strong learning capability of deep learning in the context of highdimensional data processing, as well as the significant potential of active learning (AL) in effectively reducing labeling costs. Within the field of DeepAL, several methods (Gal et al., 2017; Pop & Fulop, 2018; Kirsch et al., 2019; Tran et al., 2019) 2020) used labeled and unlabeled datasets to combine supervised and semisupervised training with AL methods. There also exist a number of works on how to improve the batch sample query strategy (Shi & Yu, 2019; Kirsch et al., 2019; Zhdanov, 2019; Ash et al., 2020) . As we will show, by leveraging the advantages of DeepAL, ALaSI is competent in efficient and accurate inferring the existence of directed connections with a smaller labeled pool of prior knowledge. Structural inference. The aim of structural inference is to accurately reconstruct the connections between the agents in a dynamical system with observational agents' states. Among the wide variety of methods, neural relational inference (NRI) (Kipf et al., 2018) was the first to address the problem of structural inference based on observational agents' states with the help of a VAE operating on a fixed fully connected graph structure. Based on NRI, Webb et al. (2019) proposed factorized neural relational inference (fNRI), extending NRI to multi-interaction systems. Chen et al. ( 2021) proposed a method with efficient message passing mechanisms (MPM), to increase the accuracy of structural inference for complex systems. Moreover, Alet et al. (2019) proposed a modular meta-learningbased framework that jointly infers the connectivity with higher data efficiency. From the aspect of Granger-causality, amortized causality discovery (ACD) (Löwe et al., 2022) attempted to infer a latent posterior graph from temporal conditional dependence. In addition to the work mentioned above, several frameworks can also infer the connectivity, but with different problem settings. Various methods (Ivanovic & Pavone, 2019; Graber & Schwing, 2020; Li et al., 2022) were specially designed to infer the connections of dynamic graphs. ARNI (Casadiego et al., 2017) inferred the latent structure based on regression analysis and a careful choice of basis functions. Mutual information was also utilized to determine the existence of causal links and thus could infer the connectivity of dynamical systems (Schreiber, 2000; Wu et al., 2020) . Some approaches fitted a dynamics model and then produced a causal graph estimate of the model by using recurrent models (Tank et al., 2021; Khanna & Tan, 2020) , or inferred the connections by generating edges sequentially (Johnson, 2017; Li et al., 2018) and others independently pruned the generated edges from an over-complete graph (Selvan et al., 2018) . It is worth mentioning that there exists another branch of research called graph structure learning, which aims to jointly learn an optimized graph structure and corresponding graph representations for downstream tasks (Zhu et al., 2021; Fatemi et al., 2021; Jin et al., 2020) . Besides that, there is another series of work to reconstruct the structure of directed acyclic graphs (Zheng et al., 2018; Yu et al., 2019; Saeed et al., 2020; Yu et al., 2021) . However, because of various reasons, such as the fixed latent space of VAE, or exponential computational efficiency, most of the methods mentioned above are incapable of structural inference on large dynamical systems and have difficulties in the efficient utilization of prior knowledge.



applied Bayesian deep learning to deal with high-dimensional mini-batch samples with fewer queries in the AL context. To solve the problem of insufficient labeled sample data, Tran et al. (2019) leveraged generative networks for data augmentation, and Wang et al. (2016) assigned pseudo-labels to high-confidence samples to expand the labeled training set. Moreover, Hossain & Roy (2019) and Siméoni et al. (

