HYPERSAGE: GENERALIZING INDUCTIVE REPRESENTATION LEARNING ON HYPERGRAPHS

Abstract

Graphs are the most ubiquitous form of structured data representation used in machine learning. They model, however, only pairwise relations between nodes and are not designed for encoding the higher-order relations found in many real-world datasets. To model such complex relations, hypergraphs have proven to be a natural representation. Learning the node representations in a hypergraph is more complex than in a graph as it involves information propagation at two levels: within every hyperedge and across the hyperedges. Most current approaches first transform a hypergraph structure to a graph for use in existing geometric deep learning algorithms. This transformation leads to information loss, and sub-optimal exploitation of the hypergraph's expressive power. We present HyperSAGE, a novel hypergraph learning framework that uses a two-level neural message passing strategy to accurately and efficiently propagate information through hypergraphs. The flexible design of HyperSAGE facilitates different ways of aggregating neighborhood information. Unlike the majority of related work which is transductive, our approach, inspired by the popular GraphSAGE method, is inductive. Thus, it can also be used on previously unseen nodes, facilitating deployment in problems such as evolving or partially observed hypergraphs. Through extensive experimentation, we show that HyperSAGE outperforms state-of-the-art hypergraph learning methods on representative benchmark datasets. We also demonstrate that the higher expressive power of HyperSAGE makes it more stable in learning node representations as compared to the alternatives.

1. INTRODUCTION

Graphs are considered the most prevalent structures for discovering useful information within a network, especially because of their capability to combine object-level information with the underlying inter-object relations (Wu et al., 2020) . However, most structures encountered in practical applications form groups and relations that cannot be properly represented using pairwise connections alone, hence a graph may fail to capture the collective flow of information across objects. In addition, the underlying data structure might be evolving and only partially observed. Such dynamic higher-order relations occur in various domains, such as social networks (Tan et al., 2011 ), computational chemistry (Gu et al., 2020 ), neuroscience (Gu et al., 2017 ) and visual arts (Arya et al., 2019) , among others. These relations can be readily represented with hypergraphs, where an edge can connect an arbitrary number of vertices as opposed to just two vertices in graphs. Hypergraphs thus provide a more flexible and natural framework to represent such multi-way relations (Wolf et al., 2016) , however, this requires a representation learning technique that exploits the full expressive power of hypergraphs and can generalize on unseen nodes from a partially observed hypergraph. Recent work in the field of geometric deep learning have presented formulations on graph structured data for the tasks of node classification (Kipf & Welling, 2016) , link prediction (Zhang & Chen, 2018), or the classification of graphs (Zhang et al., 2018b) . Subsequently, for data containing higher-order relations, a few recent papers have presented hypergraph-based learning approaches on similar tasks (Yadati et al., 2019; Feng et al., 2019) . A common implicit premise in these papers is that a hypergraph can be viewed as a specific type of regular graph. Therefore, reduction of hypergraph learning problem to that of a graph should suffice. Strategies to reduce a hypergraph to a graph include transforming the hyperedges into multiple edges using clique expansion (Feng et al., 2019; Jiang et al., 2019; Zhang et al., 2018a) , converting to a heterogeneous graph using star ). The second Fano plane is a copy of the first with nodes v 2 and v 3 permuted. These two hypergraphs cannot be differentiated when transformed to a graph using clique expansion. v 3 v 4 v 1 v 2 v 5 v 1 v 2 v 3 v 1 e 1 v 2 v 3 e 3 e 2 v 5 v 4 v 5 v 4 clique expansion star expansion Hypergraph (a) v 1 v 7 v 3 v 6 v 5 v 4 v 2 v 1 v 7 v 2 v 6 v 5 v 4 v 3 (b) expansion (Agarwal et al., 2006) , and replacing every hyperedge with an edge created using a certain predefined metric (Yadati et al., 2019) . Yet these methods are based on the wrong premise, motivated chiefly by a larger availability of graph-based approaches. By reducing a hypergraph to regular graph, these approaches make existing graph learning algorithms applicable to hypergraphs. However, hypergraphs are not a special case of regular graphs. The opposite is true, regular graphs are simply a specific type of hypergraph (Berge & Minieka, 1976) . Therefore, reducing the hypergraph problem to that of a graph cannot fully utilize the information available in hypergraph. Two schematic examples outlining this issue are shown in Fig. 1 . To address tasks based on complex structured data, a hypergraph-based formulation is needed that complies with the properties of a hypergraph. A major limitation of the existing hypergraph learning frameworks is their inherently transductive nature. This implies that these methods can only predict characteristics of nodes that were present in the hypergraph at training time, and fail to infer on previously unseen nodes. The transductive nature of existing hypegraph approaches makes them inapplicable in, for example, finding the most promising target audience for a marketing campaign or making movie recommendations with new movies appearing all the time. An inductive solution would pave the way to solve such problems using hypergraphs. The inductive learning framework must be able to identify both the node's local role in the hypergraph, as well as its global position (Hamilton et al., 2017) . This is important for generalizing the learned node embeddings that the algorithm has optimized on to a newly observed hypergraph comprising previously unseen nodes, thus, making inductive learning a far more complex problem compared to the transductive learning methods. In this paper, we address the above mentioned limitations of the existing hypergraph learning methods. We propose a simple yet effective inductive learning framework for hypergraphs that is readily applicable to graphs as well. Our approach relies on neural message passing techniques due to which it can be used on hypergraphs of any degree of cardinality without the need for reduction to graphs. The points below highlight the contributions of this paper: • We address the challenging problem of representation learning on hypergraphs by proposing HyperSAGE, comprising a message passing scheme which is capable of jointly capturing the intra-relations (within a hyperedge) as well as inter-relations (across hyperedges). • The proposed hypergraph learning framework is inductive, i.e. it can perform predictions on previously unseen nodes, and can thus be used to model evolving hypergraphs. • HyperSAGE facilitates neighborhood sampling and provides the flexibility in choosing different ways to aggregate information from the neighborhood. • HyperSAGE is more stable than state-of-the-art methods, thus provides more accurate results on node classification tasks on hypergraphs with reduced variance in the output.



Figure1: (a) Example showing reduction of a hypergraph to a graph using clique and star expansion methods. The clique expansion loses the unique information associated with the hyperedge defined by the set of nodes {v 2 , v 3 }, and it cannot distinguish it from the hyperedge defined by the nodes {v 1 , v 2 , v 3 }. Star expansion creates a heterogeneous graph that is difficult to handle using most well-studied graph methods(Hein et al., 2013). (b) Schematic representations of two Fano planes comprising 7 nodes and 7 hyperedges (6 straight lines and 1 circle.). The second Fano plane is a copy of the first with nodes v 2 and v 3 permuted. These two hypergraphs cannot be differentiated when transformed to a graph using clique expansion.

