DIRECT EMBEDDING OF TEMPORAL NETWORK EDGES VIA TIME-DECAYED LINE GRAPHS

Abstract

Temporal networks model a variety of important phenomena involving timed interactions between entities. Existing methods for machine learning on temporal networks generally exhibit at least one of two limitations. First, many methods assume time to be discretized, so if the time data is continuous, the user must determine the discretization and discard precise time information. Second, edge representations can only be calculated indirectly from the nodes, which may be suboptimal for tasks like edge classification. We present a simple method that avoids both shortcomings: construct the line graph of the network, which includes a node for each interaction, and weigh the edges of this graph based on the difference in time between interactions. From this derived graph, edge representations for the original network can be computed with efficient classical methods. The simplicity of this approach facilitates explicit theoretical analysis: we can constructively show the effectiveness of our method's representations for a natural synthetic model of temporal networks. Empirical results on real-world networks demonstrate our method's efficacy and efficiency on both link classification and prediction.

1. INTRODUCTION

Temporal networks, which are graphs augmented with a time value for each edge, model a variety of important phenomena involving timed interactions between entities, including financial transactions, flights, and web browsing. Common tasks for machine learning on temporal networks include classification of the temporal edges, as well as temporal link prediction, which involves predicting future links given some links in the past. These tasks yield various applications, such as recommendation systems (Zhou et al., 2021) and detection of illicit financial transactions (Pareja et al., 2020) . As with most machine learning for graphs, the key to learning for temporal networks is creating effective vector representations, also called embeddings, for the network's components, namely the nodes and edges. These embeddings can either be made as part of an end-to-end framework, or created then passed to off-the-shelf classifiers for downstream tasks; for example, for the edge classification task, a logistic regression classifier can be trained using the training edges' embedding vectors and class labels, then applied at inference time on the test edges' vectors. The node embedding task, in particular, has seen great interest, and many methods for 'static' (i.e., non-temporal) networks have been proposed over the years (Belkin & Niyogi, 2001; Perozzi et al., 2014; Grover & Leskovec, 2016) . Edge embedding has seen less interest; there are some exceptions (Li et al., 2017b; Bandyopadhyay et al., 2019) , but generally, edge embeddings are created by first making node embeddings, then aggregating them, e.g., by taking the entrywise product of the two endpoint nodes' embeddings. There are a wide variety of methods for temporal network embedding, including ones based on matrix/tensor factorization (Dunlavy et al., 2011; Li et al., 2017a; Zhang et al., 2018 ), random walks (Yu et al., 2018; Nguyen et al., 2018) , graph convolutional networks (Pareja et al., 2020), and deep autoencoders (Goyal et al., 2018; Rahman et al., 2018; Goyal et al., 2020) , but generally, these methods can be seen as variants of those for static networks. These prior methods overall present some limitations, which we now summarize. With some exceptions, especially in more recent work (Nguyen et al., 2018; Trivedi et al., 2019; Rossi et al 2020; Wang et al., 2021) , these methods often do not work directly with the continuous-valued times of temporal edges, but rather assume that the times are discretized, yielding a sequence of static graphs. Since most datasets have continuous-timed edges, this assumption requires the user to manually determine the discretization and discard precise time information. Second, prior methods generally return embeddings of nodes rather than edges, so edge embeddings can only be calculated indirectly from the nodes, which may be suboptimal for tasks like edge classification. Given that timestamps are associated with edges rather than nodes, and that there are few public datasets where the nodes rather than edges are associated with a time-series of attributes or classes, it is intuitively more natural to derive edge embeddings directly. We present a simple but novel framework to address these issues: construct the line graph of the network, which converts each edge (timed interaction) of the network into a node, and connects interactions that share an endpoint node. Then, set the edge weights in this line graph based on differences in time between interactions, with interactions that occur closer together in time being connected with higher weights. From this derived graph, which directly represents topological proximity (i.e., adjacency of edges) and temporal proximity, temporal edge representations for the original network can be computed and exploited with efficient classical methods. To our knowledge, ours is the first method that directly forms embeddings for continuous-time temporal edges without supervision and without aggregating node embeddings -see Table 1 . Our method is significantly simpler than recent prior work, particularly compared to deep methods, allowing for more direct theoretical analysis: we propose the union of Gaussian-timed stochastic block models (UGT-SBM), which naturally extends the well-known stochastic block model (SBM) for static networks to temporal networks, and we show that our method can exploit time and community information in UGT-SBMs to form effective representations. Practically, our method's simplicity makes it easy to implement, yet it is accurate and efficient: in experiments on five benchmark real-world temporal networks, our approach achieves superior predictive and runtime performance relative to prior methods.

2. METHODOLOGY

Our approach starts with a sequence of timestamped edges: Definition 1 (Temporal Graph). A temporal graph G = (V, E) is a set of vertices V and a set of temporal edges E, where E ⊆ V × V × R. t is the time of the temporal edge (u, v, t). Our approach centers around our proposed notion of 'time-decayed line graphs' (TDLGs) which are derived from temporal graphs. Our notion of TDLGs extends the well-established idea of line graphs, which are derived from static (i.e., non-temporal), undirected graphs; the line graph of an undirected, unweighted graph is another undirected, unweighted graph that represents adjacencies between edges in the original graph. We propose to incorporate time information by using it to set the weights of the resulting line graph. Specifically, given a temporal graph G, we construct a TDLG, which is a static weighted line graph L TD (G), as follows in Definition 2. Definition 2 (Time-Decayed Line Graph). Given a temporal graph G = (V, E), the associated timedecayed line graph is L TD (G) = (V L , E L , w), where V L = E, E L = {((u, v, t 1 ), (v, z, t 2 )) : (u, v, t 1 ), (v, z, t 2 ) ∈ E}, and the edge weight function w : E L → R + evaluated on edge ((u, v, t 1 ), (v, z, t 2 )) is given by exp -1 2σ 2 t (t 1 -t 2 ) 2 for some fixed σ t > 0. Thus, proximity of two temporal edges in the TDLG incorporates both topological proximity in the original graph as well as proximity in time. The parameter σ t controls how quickly proximity in the TDLG decays as difference in time grows. For a graph with n nodes and m edges, we can construct the weighted adjacency matrix A ∈ R m×m + of the TDLG as follows. Given the incidence matrix



., Problem studied in this work.

