CYCLE TO CLIQUE (CY2C) GRAPH NEURAL NET-WORK: A SIGHT TO SEE BEYOND NEIGHBORHOOD AGGREGATION

Abstract

Graph neural networks have been successfully adapted for learning vector representations of graphs through various neighborhood aggregation schemes. Previous researches suggest, however, that they possess limitations in incorporating key non-Euclidean topological properties of graphs. This paper mathematically identifies the caliber of graph neural networks in classifying isomorphism classes of graphs with continuous node attributes up to their local topological properties. In light of these observations, we construct the Cycle to Clique graph neural network, a novel yet simple algorithm which topologically enriches the input data of conventional graph neural networks while preserving their architectural components. This method theoretically outperforms conventional graph neural networks in classifying isomorphism classes of graphs while ensuring comparable time complexity in representing random graphs. Empirical results further support that the novel algorithm produces comparable or enhanced results in classifying benchmark graph data sets compared to contemporary variants of graph neural networks.

1. INTRODUCTION

Graph neural networks (GNN) are prominent deep learning methods for learning vector representation of graphs. Research in GNNs explores their empirical capabilities and effectiveness in classifying node labels, classifying graphs, and predicting links by modifying the message passing layers or pooling methods. These experiments support that GNNs can achieve state-of-the-art performances in executing these tasks and ensure equivalent performance to that of the Weisfeiler-Lehman (WL) isomorphism test in representing graphs with discrete node labels Xu et al. (2018) . However, they have limited capabilities in incorporating global topological properties of graphs, thereby exhibiting restricted discriminative power in distinguishing isomorphism classes of graphs Bouritsas et al. (2022); Rieck et al. (2019) . To overcome these limitations, this paper presents a mathematical framework that examines which topological properties of graphs with continuous node attribute that conventional GNNs can encapsulate. Inspired by the works of Krebs and Verbitsky Krebs & Verbitsky (2015) and Xu et al Xu et al. (2018) , we use the theory of covering spaces to prove that under some constraints, a pair of graphs with continuous node attributes is distinguishable by GNNs if and only if there exist isomorphisms among the collection of their finite depth unfolding trees that induce equality of induced node attributes. This gives a universal formulation which pinpoints the discriminative power of a wide range of variants of GNNs and the topological enrichments these models endow over the graph data set. Such approaches include enriching node attributes, using persistent homological techniques, gluing high dimensional complexes, and keeping track of recurring subgraph structures. Among these candidates, we focus on the procedure of transforming the cycle bases of graphs to complete subgraphs or cliques. This operation can be easily implemented by adding suitable edges to transform a cyclic subgraph into a clique and masking any other edges not included in the subgraph. The adjacency matrices obtained from the induced cliques, denoted as clique adjacency matrices, allow GNNs to effectively process the bases of cycles, which are topological properties equivalent to the first homological invariants of graphs Paton (1969) . In particular, the operation can be thought as a straightforward pre-processing procedure independent from training dynamical filtration functions or attaching higher dimensional cells Horn et al. ( 2021 We thus propose the Cycle-to-Clique Graph Neural Network (Cy2C-GNN), a graph neural network whose message passing layers compute two types of hidden node attributes, each obtained from the adjacency matrix and the induced clique adjacency matrix of a graph. We confirm that Cy2C-GNN effectively processes cycle bases of graphs, thus surpassing the strengths of conventional GNNs. Experimental results support that Cy2C-GNN ensures comparable performance to GNNs that utilize persistent homological techniques with both fixed and arbitrary filtration functions. Furthermore, the simplicity of the architecture guarantees equivalent computational complexity to conventional GNNs in representing random graphs and the effective utilization of trainable parameters. Our main contributions can therefore be summarized as follows: 1. Theoretical Foundation: We use the theory of covering spaces to prove that conventional GNNs fail to effectively represent cyclic structures of graphs with continuous node attributes. (Theorem 3.3, Section 3) 2. A Simple yet Novel Network: We propose a novel algorithm called "Cy2C-GNN" which overcomes the theoretical limitations by enriching the topological properties of the input data admitted by GNNs with clique adjacency matrices, which does not require training filtration functions or attaching high-dimensional complexes. (Theorem 4.3, Section 4) 3. Efficient Enhancements: The proposed algorithm effectively incorporates cyclic structures of graph data sets while ensuring equivalent computational complexity to conventional GNNs for representing random graphs and adaptability to variants of GNNs. (Section 5)

2. RELATED WORKS

Graph Neural Networks (GNNs) We recall the construction of GNNs as suggested in Xu et al Xu et al. (2018) . Denote by GNN l the conventional GNN (GNN) comprised of composition of l neighborhood aggregating layers. Each m-th layer H (m) of the network constructs hidden node attributes of dimension k m , denoted as h (m) v , using the following composition of functions: h (m) v := COMBINE (m) h (m-1) v , AGGREGATE (m) v h (m-1) u | u ∈ N (v) h (0) v := X v (1) Here, X v is the initial node attribute at v, N (v) is the set of nodes adjacent to v ∈ V (G), AGGREGATE v is a function which aggregates features of nodes adjacency to v, and COMBINE (m) is a function which combines features of the node v with those of nodes adjacent to v. Denote by H (l) the final layer of the network. The K-dimensional vector representation of G, denoted as h G , is given by h G := READOUT (l) {{h (l) v | v ∈ V (G)}} where READOUT (l) is the graph readout function of node features updated from l hidden layers. We refer readers to Appendix A.2 for a rigorous definition of graph neural networks. A wide range of GNNs and graph representation techniques can be formulated in terms of the construction outlined above. For example, the WL test is a classical technique which consists of combining adjacent discrete node labels, substituting newly obtained labels, and constructing a complete histogram of updated labels. The test is equivalent to conventional GNNs whose aggregation and combination functions correspond to sums of multisets, and the graph readout function corresponds to a hashing function of discrete node labels. Other well-known networks whose architecture can be formulated using conventional GNNs from Section 2 include graph convolutional networks (GCN), graph attention networks (GAT), and graph isomorphism networks 



);Bouritsas et al. (2022); Bodnar et al. (2021b;a), which are previously carefully studied methods for encapsulating the cyclic structures of graphs.

(GIN) Kipf & Welling (2016); Veličković et al. (2017); Xu et al. (2018).

