AUTOREGRESSIVE DIFFUSION MODEL FOR GRAPH GENERATION Anonymous authors Paper under double-blind review

Abstract

Diffusion-based graph generative models have recently obtained promising results for graph generation. However, existing diffusion-based graph generative models are all one-shot generative models that apply Gaussian diffusion in the dequantized adjacency matrix space. Such a strategy can suffer from difficulty in model training, slow sampling speed, and incapability of incorporating constraints. We propose an autoregressive diffusion model for graph generation. Unlike existing methods, we define a node-absorbing diffusion process that operates directly in the discrete graph space. For forward diffusion, we design a diffusion ordering network, which learns an optimal node absorbing ordering from graph topology. For reverse generation, we design a denoising network that uses the reverse node ordering to efficiently reconstruct the graph by predicting one row of the adjacency matrix at a time. Based on permutation invariance of graph generation, we show that the two networks can be jointly trained by optimizing a simple lower bound of data likelihood. Our experiments on six diverse datasets show that our model achieves better or comparable generation performance with previous stateof-the-art, and meanwhile enjoys fast generation speed.

1. INTRODUCTION

Generating graphs from a target distribution is a fundamental problem in many domains such as drug discovery (Li et al., 2018) , material design (Maziarka et al., 2020) , social network analysis (Grover et al., 2019) , and public health (Yu et al., 2020) . Deep generative models have recently led to promising advances in this problem. Different from traditional random graph models (Erdos et al., 1960; Albert & Barabási, 2002) , these methods fit graph data with powerful deep generative models including variational auto-encoders (Simonovsky & Komodakis, 2018), generative adversarial networks (Maziarka et al., 2020) , normalizing flows (Madhawa et al., 2019) , and energy-based models (Liu et al., 2021) . These models are learned to capture complex graph structural patterns and then generate new high-fidelity graphs with desired properties. Recently, the emergence of probabilistic diffusion models has led to interest in diffusion-based graph generation (Jo et al., 2022) . Diffusion models decompose the full complex transformation between noise and real data into many small steps of simple diffusion. Compared with prior deep generative models, diffusion models enjoy both flexibility in modeling architecture and tractability of the model's probability distributions. To the best of our knowledge, there are two existing works 



on diffusion-based graph generation: Niu et al. (2020) model the adjacency matrices using score matching at different noise scales, and uses annealed Langevin dynamics to sample new adjacency matrices for generation; Jo et al. (2022) propose a continuous-time graph diffusion model that jointly models adjacency matrices and node features through stochastic differential equations (SDEs). However, existing diffusion-based graph generative models suffer from three key drawbacks: (1) Generation Efficiency. The sampling processes of Niu et al. (2020); Jo et al. (2022) are slow, as Niu et al. (2020) requires a large number of diffusion noising levels and Jo et al. (2022) needs to solve a complex system SDEs. (2) Continuous Approximation. They convert discrete graphs to continuous state spaces by adding real-valued noise to graph adjacency matrices. Such dequantization can distort the distribution of the original discrete graph structures, thus increasing the difficulty of model training. (3) Incorporating constraints. They are both one-shot generation models and hence cannot easily incorporate constraints during the one-shot generation process.

