ON SINGLE-ENVIRONMENT EXTRAPOLATIONS IN GRAPH CLASSIFICATION AND REGRESSION TASKS Anonymous

Abstract

Extrapolation in graph classification/regression remains an underexplored area of an otherwise rapidly developing field. Our work contributes to a growing literature by providing the first systematic counterfactual modeling framework for extrapolations in graph classification/regression tasks. To show that extrapolation from a single training environment is possible, we develop a connection between certain extrapolation tasks on graph sizes and Lovász's characterization of graph limits. For these extrapolations, standard graph neural networks (GNNs) will fail, while classifiers using induced homomorphism densities succeed, but mostly on unattributed graphs. Generalizing these density features through a GNN subgraph decomposition allows them to also succeed in more complex attributed graph extrapolation tasks. Finally, our experiments validate our theoretical results and showcase some shortcomings of common (interpolation) methods in the literature.

1. INTRODUCTION

In some graph classification and regression applications, the graphs themselves are representations of a natural process rather than the true state of the process. Molecular graphs are built from a pairwise atom distance matrix by keeping edges whose distance is below a certain threshold and the choice impacts distinguishability between molecules (Klicpera et al., 2020) . Functional brain connectomes are derived from time series but researchers must choose a frequency range for the signals, which affects resulting graph structure (De Domenico et al., 2016) In this work, we refer to graph-processing environment (or just environment) as the collection of heuristics and other data curation processes that gave us the observed graph from the true state of the process under consideration. The true state alone defines the target variable. Our work is interested in what we refer as the graph extrapolation task: predict a target variable from a graph regardless of its environment. In this context, even graph sizes can be determined by the environment. Unsurprisingly, graph extrapolation tasks-a type of out-of-distribution prediction-are only feasible when we make assumptions about these environments. We define the graph extrapolation task as a counterfactual inference task that requires learning environment-invariant (E-invariant) representations. Unfortunately, graph datasets largely contain a single environment, while common E-invariant representation methods require training data from multiple environments, including Independence of causal mechanism (ICM) methods (Bengio et al., 2019; Besserve et al., 2018; Johansson et al., 2016; Louizos et al., 2017; Raj et al., 2020; Schölkopf, 2019; Arjovsky et al., 2019) , Causal Discovery from Change (CDC) methods (Tian & Pearl, 2001) , and representation disentanglement methods (Bengio et al., 2019; Goudet et al., 2017; Locatello et al., 2019) . Contributions. Our work contributes to a growing literature by providing, to the best of our knowledge, the first systematic counterfactual modeling framework for extrapolations in graph classification/regression tasks. Existing work, e.g., the parallel work of Xu et al. (2020) , define extrapolations geometrically and, thus, have a different scope. Our work connects Lovász's graph limit theory with graph-size extrapolation in a family of graph classification and regression tasks. Moreover, our experiments show that in these tasks, traditional graph classification/regression methods -including graph neural networks and graph kernels-are unable to extrapolate.



. Recent work (e.g. Knyazev et al. (2019); Bouritsas et al. (2020); Xu et al. (2020)) explore extrapolations in real-world tasks, showcasing a growing interest in the underexplored topic of graph extrapolation tasks.

