Simulation and Calibration of a Fully Bayesian Marked Multidimensional Hawkes Process with Dissimilar Decays

Transcription

1 Simulation and Calibration of a Fully Bayesian Marked Multidimensional Hawkes Process with Dissimilar Decays Kar Wai Lim, Young Lee, Leif Hanlen, Hongbiao Zhao Australian National University Data61/CSIRO Xiamen University 1 Proceedings of the 8th Asian Conference in Machine Learning (ACML) / 19

2 Talk Outline Introduction on Hawkes Processes Simulation of Hawkes Processes Bayesian Inference for Hawkes 2 / 19

3 Background Poisson distributions Commonly used to model the number of times an event occurs in an interval of time or space. Textbook example: the number of cars passing an intersection in half an hour. Figure: (left) Probability mass functions (right) Observed histogram Additive Property: If X Poi(λ1), Y Poi(λ 2), then X + Y Poi(λ 1 + λ 2) 3 / 19

4 Background (Homogeneous) Poisson process It is a stochastic process that keep track of the running counts of an event over time (and space). For example, the number of cars passing an intersection is an evolution of counts with time: Call the evolution of counts as the counting process N(t) and the times of an event happening the event times t i. 4 / 19

5 Background Properties of Poisson process The counting process starts at zero: N(t = 0) = 0. Parameterised by the expected number of events per unit time, e.g. λ = 3 vehicles per minute. The counting process N(t) at time t follows Poi(λt). (number of events observed until time t) The difference (also called increment) in counting processes N(t) N(s) Poi(λ(t s)) t > s Superposition property: If N(t) PP(λ1), M(t) PP(λ 2), then N(t) + M(t) PP(λ 1 + λ 2) 5 / 19

6 Background What if some events are more frequent at certain times? More cars during peak hours! Instead of constant intensity, allow the intensity to vary with time: λ(t) becomes a function of time. 6 / 19

7 Background Extension: Inhomogeneous Poisson process Example λ(t): Piecewise linear; Piecewise polynomial; Cyclical functions such as sine curve. Figure: Generated data 7 / 19

8 Background Properties of inhomogeneous Poisson process (IPP) The counting process starts at zero: N(t = 0) = 0. Parameterised by intensity function λ(t). t The counting process at time t follows Poi( λ(u) du). 0 The difference (also called increment) in counting processes ( t ) N(t) N(s) Poi λ(u) du t > s s Superposition property still holds: If N(t) IPP(λ1(t)), M(t) IPP(λ 2(t)), then N(t) + M(t) IPP(λ 1(t) + λ 2(t)) 8 / 19

9 Hawkes Processes Hawkes process is a point process in which an occurrence of an event triggers future events (self-excitation). Our formulation of Hawkes (univariate): λ(t) = µ(t) + N(T ) i=1: t>t i Y i e δ(t ti ) Decaying background intensity: µ(t) = k + Y (0) e δ t Random self excitations: Y i i.i.d. Gamma Terminology: ti, i = 1,..., N(T ) is a sequence of non-negative random variables such that t i < t i+1, known as event times. i = t i t i 1 is called the inter-arrival time. 9 / 19

10 Multivariate Hawkes Captures multiple event types for which the events mutually excite one another. Our formulation (Bivariate Hawkes): λ 1 (t) = µ 1 (t) + N 1 (t) Y 1 1,j e δ1 1 t + N 2 (t) Y 2 1,j e δ2 1 t j=1: t t 1 j j=1: t t 2 j λ 2 (t) = µ 2 (t) + N 1 (t) Y 1 2,j e δ1 2 t + N 2 (t) Y 2 2,j e δ2 2 t j=1: t t 1 j j=1: t t 2 j where λ 1 (t) and λ 2 (t) are the intensity functions for events 1 and 2, respectively. Note that the decay parameters δ are different for each process. 10 / 19

11 Illustration of Multivariate 0 Hawkes Time Intensity Time Dimension 1 Dimension 2 Dimension 3 Counting Process Time 11 / 19

12 Detour: Stationarity of Hawkes process Due to self-excitation property, a Hawkes process is only stable (stationary) when certain condition is satisfied. The intensity process λ(t) explodes if this condition is not satisfied: Causing chain reactions: intensity increases more future events further increases in intensity... We present a theoretical result on the expected stationary intensities for our Hawkes formulation. [Extension of Hawkes (1971) and Bacry et al. (2015)] 12 / 19

13 Outline Introduction on Hawkes Processes Simulation of Hawkes Processes Bayesian Inference for Hawkes 13 / 19

14 Simulation of Hawkes Processes There are three categories of simulation methods. 1. Inverse Sampling (Ozaki, 1979) Derives cdf (cumulative distribution function) of inter-arrival times, then performs inverse sampling. Cdf cannot be inverted directly so approximation is needed. 2. Thinning (Lewis and Shedler, 1979; Ogata, 1981) Simulate samples from a Poisson process and then thin the samples. Akin to a rejection sampler. 3. Cluster method (Brix & Kendall, 2002; Møller & Rasmussen, 2005) Recast Hawkes using a Poisson cluster representation. Each observed event generates an IPP. Superposition of all of them forms a Hawkes process. Notable mention: exact sampler of Dassios & Zhao (2013) Performs inverse sampling without approximation by decomposing a variable into two need to satisfy a Markovian constraint. Our method: exploits superposition theory and first order statistics for efficient sampling. 14 / 19

15 Our Simulation Method in One Slide Illustration with bivariate Hawkes λ 1(t) = µ 1(t) + N 1 (t) Y 1 1,j e δ1 1 t + N 2 (t) Y 2 1,j e δ2 1 t j=1: t t 1 j j=1: t t 2 j λ 2(t) = µ 2(t) + N 1 (t) Y 1 2,j e δ1 2 t + N 2 (t) Y 2 2,j e δ2 2 t j=1: t t 1 j j=1: t t 2 j A Hawkes with intensity λ 1 (t) is a superposition of IPP (with intensities µ 1 etc). Inter-arrival times (a i, b i, c i...) for these IPP can be sampled easily. We show that the inter-arrival time i for a Hawkes process is a first order statistics of these inter-arrival times: i = min{a i, b i, c i...} No need to resort to approximation or satisfy Markovian constraint. 2 Note: efficient caching can be performed if the Hawkes is Markov. 15 / 19

16 Simulation Statistics We compare the simulated statistics against theoretical expectations (over 1 million simulation paths): Intensity Dimension 1 Dimension 2 Dimension Time Table 1: Comparison of simulated mean intensities against their theore ten fixed time t. The simulated mean intensities are comput simulated intensities over one million sample paths. The perce the simulated means against the theoretical expectations are m the correctness of the proposed simulation algorithm. [Maybe 2 for space] Process m = 1 Process m = 2 Pro Time Sim. Expt. %Diff. Sim. Expt. %Diff. Sim. E In addition, through the simulation of univariate Hawkes processe parison against the inverse sampling method of Ozaki (1979) and the cl of Brix and Kendall (2002). Since implementation on these methods we able, we implement them based on the algorithms outlined in the resp might not be fully optimised, however, the same can be said to the im own sampler. The simulation settings (subscripts m and i suppressed) Figure: Plot of simulated mean intensities vs the theoretical stationary average intensities of the three-dimensional Hawkes processes. Verifies that our algorithm and implementation is correct. See paper for other results / 19

17 Outline Introduction on Hawkes Processes Simulation of Hawkes Processes Bayesian Inference for Hawkes 17 / 19

18 Bayesian Inference in One Slide Fully Gibbs sampling achieved by Auxiliary variables augmentation introduce additional parameters called branching structures allow decoupling of existing parameters. Adaptive rejection sampling (ARS) for variables that do not have known posterior distributions, we show conditions for which the posteriors are log-concave, and sample via ARS. On simulated data, we demonstrate that the parameters learned using Bayesian inference is accurate and superior to MLE: See paper for application on modelling Dark Networks. 18 / 19

19 Summary Theoretical result on expected stationary intensities Simulation of multivariate Hawkes with superposition theory and first order statistics Bayesian inference on Hawkes with auxiliary variable augmentation and adaptive rejection sampling 19 / 19