DYHCN: DYNAMIC HYPERGRAPH CONVOLUTIONAL NETWORKS

Abstract

Hypergraph Convolutional Network (HCN) has become a default choice for capturing high-order relations among nodes, i.e., encoding the structure of a hypergraph. However, existing HCN models ignore the dynamic evolution of hypergraphs in the real-world scenarios, i.e., nodes and hyperedges in a hypergraph change dynamically over time. To capture the evolution of high-order relations and facilitate relevant analytic tasks, we formulate dynamic hypergraph and devise the Dynamic Hypergraph Convolutional Networks (DyHCN). In general, DyHCN consists of a Hypergraph Convolution (HC) to encode the hypergraph structure at a time point and a Temporal Evolution module (TE) to capture the varying of the relations. The HC is delicately designed by equipping inner attention and outer attention, which adaptively aggregate nodes' features to hyperedge and estimate the importance of each hyperedge connected to the centroid node, respectively. Extensive experiments on the Tiigo and Stocktwits datasets show that DyHCN achieves superior performance over existing methods, which implies the effectiveness of capturing the property of dynamic hypergraphs by HC and TE modules.

1. INTRODUCTION

Graph Convolutional Network (GCN) Scarselli et al. (2008) extends deep neural networks to process graph data, which encodes the relations between nodes via propagating node features over the graph structure. GCN has become a promising solution in a wide spectral of graph analytic tasks, such as relation detection Schlichtkrull et al. (2018) and recommendation Ying et al. (2018) . An emergent direction of GCN research is extending the graph covolution operations to hypergraphs, i.e., hypergraph convolutional networks Zhu et al. (2017); Zhou et al. (2007); Zhang et al. (2017); Feng et al. (2019b); Yadati et al. (2019) , where high-order node relations are represented as hyperedges (one hyperedge can connect multiple nodes). For instance, in a hypergraph of stocks, an financial event relevant to several stocks is represented as a hyperedge. While a surge of attention paid on hypergraph convolutional networks, most of them discard the dynamic property of hypergraphs in real-world applications, e.g., new hyperedges (i.e., events) emerge in the hypergraph of stocks (see Fig. 1 ), where the evolution of the hypergraph is crucial for the analytic tasks (e.g., stock price prediction). Aiming to bridge the gap, this work explore the central theme of dynamic hypergraph and the corresponding GCN. Formally, a hypergraph with n nodes and m hyperedges is represented as G = (V, E, A, H, X) where V and E denote the set of nodes and hyperedges respectively; A ∈ R n×m is an incidence matrix with binary value indicating the connectedness of nodes; H ∈ R m×c and X ∈ R n×d are features represent the hyperedges and nodes respectively. In order to account for the evolution, we first extend the concept of static hypergraph to dynamic hypergraph, which has two different formulations when treating the time as continuous value or discrete value. 1) Discrete-time formulation. A straightforward solution is to treat a time window with length of T (e.g., T days) as a sequence of time-steps and get a snapshot at each time-step. In this way, a dynamic hypergraph is defined as G D = [G 1 , • • • , G t , • • • , G T ] T where G t is a hypergraph dumped at time-step t. 2) Continuous formulation. By treating time as a continuous variable, the dynamic hypergraph can be defined as G C = (G 0 , U ) where G 0 is the initial status (a hypergraph) and U = {(p t , v t , a t )|t <= T } is a streaming of updates. p t denotes the target variable (e.g., a row of X) changed at time t; v t denotes the latest value of the target variable, a t denotes the action of change, including add, delete, A big challenge of capturing spatial-temporal dependency in a dynamic hypergraph is that it is tough to extract the features of those changing nodes or hyperedges in a unified manner for the sake of varied scales of nodes and hyperedges. Besides, how to absorb their dynamic properties is very important for various application tasks. Towards this end, we need to design the proper convolution operations on dynamic hypergraph. There are two challenging toughs: 1) at each time step, since there are various relations between hyperedges and nodes, it is important to update the node features by considering various relations in the hyperedges; 2) due to dynamically changes of the node features, modeling the temporal dependency needs to extract the corresponding temporal features. In this work, we propose a framework of Dynamic Hypergraph Convolutional Networks (DyHCN) to tackle the challenges, which has two modules: Hypergraph Convolution (HC) module and Temporal Evolution (TE) module. In a dynamic hypergraph, the set of hyperedges at each time step includes different hyperedge embeddings and each hyperedge contains different numbers of nodes. We exploit three submodules to update an node's embeddings in HC: inner attention, outer attention, and embeddings update. Firstly, inner attention transform node features along with its hyperedge into the node-hyperedge feature; and then outer attention utilizes attention mechanism to estimate the importance of each hyperedge and output the importance weights; and then we update the node's embeddings by aggregating node-hyperedge, hyperedge and node features with the weight of each hyperedge. Getting the nodes embeddings, we extract temporal features of nodes' embeddings and make a prediction by the TE module. Extensive experimental results on two real-world datasets validate the superior performance of DyHCN over the existing baselines which proves the effectiveness of DyHCN on dynamically hypergraphs. The rest of the paper is organized as follows. Section 2 introduces the preliminary knowledge about GCN and the hypergraph convolutional network. Section 3 explains the proposed DyHCN method. Section 4 introduces related work about GCN on the graph and hyperedge. Applications and experimental results are presented in Section 5. Finally, we conclude this work in Section 6.

2. PRELIMINARY

Graph Convolutional Network Given a graph G = (V, E) with N nodes v i ∈ V, edges (v i , v j ) ∈ E, an adjacency matrix A ∈ R N ×N and a degree matrix D ii = j A ij . With the input signal x, Kipf & Welling (2016) considers spectral convolutions on graphs with a filter g θ = diag(θ) in the Fourier domain, g θ x = Ug θ U T x, where U is the matrix of eigenvectors of the normalized graph Laplacian L = I N -D -1/2 AD -1/2 = UΛU T , with a diagonal matrix of eigenvalues Λ and the graph Fourier transform U T x. In order to reduce the computation complexity, g θ is approximated with Chebyshev polynomials T k (x) = 2xT k-1 (x) -T k-2 (x) Defferrard et al. ( 2016), which can be formulated as: g θ ≈ K k=0 θ k T k ( Λ), where Λ = 2 λmax Λ -I, λ max denotes the largest eigenvalue



Figure 1: The evolution of dynamic hypergraph.update. It should be noted that both formulations have pros and cons, e.g., the discrete-time formulation is more friendly to existing analytic techniques on static hypergraph such as HCN while the continuous-time formulation records the accurate time of changes. This work focuses on the discrete-time formulation and makes the first attempt to extend HCN to dynamic hypergraph.

