EFFICIENT INFERENCE OF FLEXIBLE INTERACTION IN SPIKING-NEURON NETWORKS

Abstract

Hawkes process provides an effective statistical framework for analyzing the timedependent interaction of neuronal spiking activities. Although utilized in many real applications, the classic Hawkes process is incapable of modelling inhibitory interactions among neurons. Instead, the nonlinear Hawkes process allows for a more flexible influence pattern with excitatory or inhibitory interactions. In this paper, three sets of auxiliary latent variables (Pólya-Gamma variables, latent marked Poisson processes and sparsity variables) are augmented to make functional connection weights in a Gaussian form, which allows for a simple iterative algorithm with analytical updates. As a result, an efficient expectationmaximization (EM) algorithm is derived to obtain the maximum a posteriori (MAP) estimate. We demonstrate the accuracy and efficiency performance of our algorithm on synthetic and real data. For real neural recordings, we show our algorithm can estimate the temporal dynamics of interaction and reveal the interpretable functional connectivity underlying neural spike trains.

1. INTRODUCTION

One of the most important tracks in neuroscience is to examine the neuronal activity in the cerebral cortex under varying experimental conditions. Recordings of neuronal activity are represented through a series of action potentials or spike trains. The transmitted information and functional connection between neurons are considered to be primarily represented by spike trains (Kass et al., 2014; Kass & Ventura, 2001; Brown et al., 2004; 2002) . A spike train is a sequence of recorded times at which a neuron fires an action potential and each spike may be considered to be a timestamp. Spikes occur irregularly both within and across multiple trials, so it is reasonable to consider a spike train as a point process with the instantaneous firing rate being the intensity function of point processes (Perkel et al., 1967; Paninski, 2004; Eden et al., 2004) . An example of spike trains for multiple neurons is shown in Fig. 2a in the real data experiment. Despite many existing applications, the classic point process models, e.g., Poisson processes, neglect the time-dependent interaction within one neuron and between multiple neurons, so fail to capture the complex temporal dynamics of a neural population. In contrast, Hawkes process is one type of point processes which is able to model the self-exciting interaction between past and future events. Existing applications cover a wide range of domains including seismology (Ogata, 1998; 1999 ), criminology (Mohler et al., 2011;; Lewis et al., 2012 ), financial engineering (Bacry et al., 2015; Filimonov & Sornette, 2015) and epidemics (Saichev & Sornette, 2011; Rizoiu et al., 2018) . Unfortunately, due to the linearly additive intensity, the vanilla Hawkes process can only represent the purely excitatory interaction because a negative firing rate may exist with inhibitory interaction. This makes the vanilla version inappropriate in the neuroscience domain where the influence between neurons is a mixture of excitation and inhibition (Maffei et al., 2004; Mongillo et al., 2018) . In order to reconcile Hawkes process with inhibition, various nonlinear Hawkes process variants are proposed to allow for both excitatory and inhibitory interactions. The core point of nonlinear Hawkes process is a nonlinearity which maps the convolution of the spike train with a causal influential kernel to a nonnegative conditional intensity, such as rectifier (Reynaud-Bouret et al., 2013 ), exponential (Gerhard et al., 2017) and sigmoid (Linderman, 2016; Apostolopoulou et al., 2019) . The sigmoid mapping function has the advantage that the Pólya-Gamma augmentation scheme can be utilized to convert the likelihood into a Gaussian form, which makes the inference tractable. In Linderman (2016), a discrete-time model is proposed to convert the likelihood from a Poisson process to a Poisson distribution. Then Pólya-Gamma random variables are augmented on discrete observations to propose a Gibbs sampler. This method is further extended to a continuous-time regime in Apostolopoulou et al. ( 2019) by augmenting thinned points and Pólya-Gamma random variables to propose a Gibbs sampler. However, the influence function is limited to be purely exciting or inhibitive exponential decay. Besides, due to the nonconjugacy of the excitation parameter of exponential decay influence function, a Metropolis-Hastings sampling step has to be embedded into the Gibbs sampler making the Markov chain Monte Carlo (MCMC) algorithm further inefficient. To address the parametric and inefficient problems in aforementioned existing works, we develop a flexible sigmoid nonlinear multivariate Hawkes processes (SNMHP) model in the continuous-time regime, (1) which can represent the flexible excitation-inhibition-mixture temporal dynamics among the neural population, (2) with the efficient conjugate inference. An EM inference algorithm is proposed to fit neural spike trains. Inspired by Donner & Opper (2017; 2018) , three auxiliary latent variable sets: Pólya-Gamma variables, latent marked Poisson processes and sparsity variables are augmented to make functional connection weights in a Gaussian form. As a result, the EM algorithm has analytical updates with drastically improved efficiency. As shown in experiments, it is even more efficient than the maximum likelihood estimation (MLE) for the parametric Hawkes process in high dimensional cases.

2. OUR MODEL

Neurons communicate with each other by action potentials (spikes) and chemical neurotransmitters. A spike causes the pre-synaptic neuron to release a chemical neurotransmitter that induces impulse responses, either exciting or inhibiting the post-synaptic neuron from firing its own spikes. The addition of excitatory and inhibitory influence to a neuron determines whether a spike will occur. At the same time, the impulse response characterizes the temporal dynamics of the exciting or inhibiting influence which can be complex and flexible (Purves et al., 2014; Squire et al., 2012; Bassett & Sporns, 2017) . Arguably, the flexible nonlinear multivariate Hawkes processes are a suitable choice for representing the temporal dynamics of mutually excitatory or inhibitory interactions and functional connectivity of neuron networks.

2.1. MULTIVARIATE HAWKES PROCESSES

The vanilla multivariate Hawkes processes (Hawkes, 1971)  are sequences of timestamps D = {{t i n } Ni n=1 } M i=1 ∈ [0, T ] where t i n is the timestamp of n-th event on i-th dimension with N i being the number of points on i-th dimension, M the number of dimensions, T the observation window. The i-th dimensional conditional intensity, the probability of an event occurring on i-th dimension in [t, t + dt) given all dimensional history before t, is designed in a linear superposition form: λ i (t) = µ i + M j=1 t j n <t φ ij (t -t j n ), where µ i > 0 is the baseline rate of i-th dimension and φ ij (•) ≥ 0 is the causal influence function (impulse response) from j-th dimension to i-th dimension which is normally a parameterized function, e.g., exponential decay. The summation explains the self-and mutual-excitation phenomenon, i.e., the occurrence of previous events increases the intensity of events in the future. Unfortunately, one blemish is the vanilla multivariate Hawkes processes allow only nonnegative (excitatory) influence functions because negative (inhibitory) influence functions may yield a negative intensity which is meaningless. To reconcile the vanilla version with inhibitory effect and flexible influence function, we propose the SNMHP.

