GG-GAN: A GEOMETRIC GRAPH GENERATIVE ADVERSARIAL NETWORK

Abstract

We study the fundamental problem of graph generation. Specifically, we treat graph generation from a geometric perspective by associating each node with a position in space and then connecting edges in-between based on a similarity function. We then provide new solutions to the key challenges that prevent the widespread application of this classical geometric interpretation: (1) modeling complex relations, (2) modeling isomorphic graphs consistently, and (3) fully exploiting the latent distribution. Our main contribution is dubbed as the geometric graph (GG) generative adversarial network (GAN), which is a Wasserstein GAN that addresses the above challenges. GG-GAN is permutation equivariant and easily scales to generate graphs of tens of thousands of nodes. GG-GAN also strikes a good trade-off between novelty and modeling the distribution statistics, being competitive or surpassing the state-of-the-art methods that are either slower or that are non-equivariant, or that exploit problem-specific knowledge.

1. INTRODUCTION

Learning distributions from empirical observations is a fundamental problem in machine learning and statistics (Goodfellow et al., 2014; 2016; Salakhutdinov, 2015; Foster, 2019) . A challenging variant entails modeling distributions over graphs-discrete objects with possibly complex relational structure (Simonovsky & Komodakis, 2018; You et al., 2018; De Cao & Kipf, 2018; Liao et al., 2019; Niu et al., 2020; Yang et al., 2019) . When successfully trained, deep graph generative models carry the potential to transform a wide range of application domains, for instance by finding novel chemical compounds for drug discovery (De Cao & Kipf, 2018) , designing proteins that do not exist in nature (Huang et al., 2016) , and automatically synthesizing circuits (Guo et al., 2019) . By and large, there are four properties that a graph generator g should possess: (1) Isomorphism consistency: g should assign isomorphic graphs the same probability-a property also referred to as permutation equivariance (Niu et al., 2020; Yang et al., 2019) . ( 2) Expressive power: g should be able to model local and global dependencies between nodes and graph edges, e.g., going beyond simple degree statistics and learning structural features and motifs. (3) Scalability: g should be able to synthesize graphs with tens of thousands of vertices. (4) Novelty: g should produce non-isomorphic graphs that are similar to (but not necessarily in) the training set. Property (1) is important since there exist exponentially many ways to represent the same graph as a vector, inconsistent methods effectively waste a large portion of their capacity in describing different ways to construct the same object. Properties (2) and ( 3) are critical in large-scale contemporary applications that require going beyond simple degree statistics as well as learning structural features and motifs towards simultaneously modeling local and global dependencies between nodes and graph edges (You et al., 2018) . Property (4) is natural since common failure modes for graph generators include memorizing the training set and repeatedly generating the same graphs.

1.1. GEOMETRIC GRAPH GENERATION

Aiming to satisfy these properties, we propose a geometric generator that represents graphs spatially by embedding each node in a high-dimensional metric space and then by connecting two nodes if their positions are sufficiently similar. There is precedence to our approach, as spatial representations of graphs have been heavily used to construct simple models of random graphs, such as random geometric graphs, unit-disc graphs, unit-distance graphs, and graphons (Huson & Sen, 1995; Penrose et al., 2003; Bollobás et al., 2007; Lovász, 2012; Alon & Kupavskii, 2014; Glasscock, 2015) .

