GRAPH-INFORMED NEURAL POINT PROCESSES WITH MONOTONIC NETS

Abstract

Multi-class event data is ubiquitous in real-world applications. The recent neural temporal point processes (Omi et al., 2019) have used monotonic nets to model the cumulative conditional intensity to avoid an intractable integration in the likelihood. While successful, they are restricted to single-type events and can easily sink to poor learning results. To address these limitations and to exploit valuable structural information within event participants, we develop a Graph-Informed Neural Point Process (GINPP) that can freely handle multiple event types, greatly improve learning efficiency, and effectively integrate the graph information to facilitate training. First, we find the bottleneck of the previous model arises from the standard softplus transformation over the output of the monotonic net, which enlarges the prediction variations of the monotonic net and increases the training challenge. We propose a shift-scale variant that can significantly reduce the variation and promote the learning efficiency. Second, we use a conditional mark distribution to model multiple event types, without the need for explicitly estimating the intensity for each type. The latter can be much more challenging. Third, we use random walk to collect the neighborhood of each event participant, and use an attention mechanism to update the hidden state of each participant according to the observed events of both the participant itself and its neighborhood. In this way, we can effectively leverage the graph knowledge, and scale up to large graphs. We have shown the advantage of our approach in both ablation studies and real-world applications.

1. Introduction

Real-world applications often involve multi-class events. For example, 911 calls seek for a variety of helps, traffic records include different types of accidents, and among social network users are various types of interactions (tweeting, following, poking, etc.) . Neural temporal point processes (e.g., (Du et al., 2016; Mei and Eisner, 2017; Zhang et al., 2020a; Zuo et al., 2020) ) are a family of powerful methods for event modeling and prediction, which use neural networks (NN) to model the intensity of events and can flexibly estimate the complex dependencies among the observed events. However, due to the use of NNs, the cumulative (i.e., integral of) conditional intensity in the point process likelihood is often analytically intractable, and demand a complex, expensive approximation. To bypass this issue, the recent work Omi et al. ( 2019) uses a monotonic net (Sill, 1997; Chilinski and Silva, 2020) to model the monotonically increasing cumulative intensity to avoid the integration, and the intensity is obtained by simply taking the derivative. To ensure the positiveness, a softplus transformation is applied to the output of the monotonic net. Despite the elegance and success, this method only supports single-type events. More important, it often suffers from inefficient learning and easily falls into poor performance. In this paper, we propose GINPP, a graph-informed neural point process model to overcome these problems, and to further utilize the valuable structural knowledge within the event participants, which is often available in practice. The major contributions of our work are listed as follows. • First, we investigate the learning challenge of (Omi et al., 2019) , and find the bottleneck arises from the softplus transformation over the monotonic net prediction to ensure positiveness. To obtain an output slightly above zero, the standard softplus demands the input, i.e., the monotonic net prediction, must be negative and have much greater scales. Hence, a small output range can cause a much wider input (monotonic net prediction) range, which

