Towards inference for skewed alpha stable Levy processes

Transcription

1 Towards inference for skewed alpha stable Levy processes Simon Godsill and Tatjana Lemke Signal Processing and Communications Lab. University of Cambridge www-sigproc.eng.cam.ac.uk/~sjg

2 Overview Motivation Continuous-time models Inference for models with jumps Inference for alpha-stable Levy processes Conclusions

3 Motivation Traditional tracking applications run in discrete time discrete time state space models: Dynamic models of behaviour: Hidden state (position/velocity ) Sensor (observation) models: Measurements (range, bearing, )

4 State Space Model: Optimal Filtering (sequential inference):

5 It is often more natural to model in continuous time: Observations can arrive at arbitrary timings, and out of sequence Object manoeuvres are carried out in continuous time without reference to observation times Real objects undergo rapid and random changes of regime, or `jumps, occurring at unknown times on the continuous-time scale: Variable rate - C =60, C =2, T =100, N Particles= To appear

6 Can approximate this type of behaviour with jump Markov (or semi-markov) discrete time systems, allowing a finite set of discrete time switching models (HMMs, etc.) However, seems more natural to model directly in continuous time, without limiting regimes to a finite set (although can extend models to allow in addition regime switching between discrete states) Bayesian Monte Carlo computational tools are well suited to inference in these continuous time models

7 Continuous time models

8 100.3 Variable rate - C =60, C =2, T =100, N Particles=400 Example: stochastic trend model with Gaussian jumps Now assume that {W(t)} is comprised of (scaled) Brownian motion {B(t)} plus a pure Gaussian jump process{z(t)}: See Christensen, Murphy and Godsill, 2012

9

10 Observation of the jump process Now assume discrete-time observations of the continuous-time process, e.g. Filtering/ smoothing and parameter estimation can now be carried out using Kalman-filter conditional likelihoods:

11 Fixed Interval Filtering and Smoothing Particle filter-smoother, See Godsill, Doucet, West (2004) and Bunch and Godsill (2012)

12 Tracking examples

13 Joint work with Tatjana Lemke, see Lemke and Godsill Proc. ICASSP 2011, ICASSP stable Levy processes Previously modelled jumps as a finite activity process Perhaps more realistic to model the jumps as an infinite collection of large/ small/tiny jumps occurring in each finite time interval -stable Levy processes provide a natural way to achieve this justified in many Communications, Signal Processing and Finance applications

14 Stable distributions

15

16

17 Approaches to inference can be made by scale mixtures of normals for symmetric case, and Euler discretisation Godsill and Kuruoglu 1998, Godsill and Yang 2006, Tsionas See also Buckle (1995) Our current work focuses on a powerful series expansion for the general asymmetric case:

18 Arrival times of a unit rate Poisson process iid random variables satisfying some simple moment conditions, e.g. A Gaussian would do. [Here stated in its simplest form for a random variable with <1. A similar form applies for the stochastic integral.] [Samorodnitsky and Taqqu 1994]

19 now has a `physical interpretation as the scale of the ith jump. [Samorodnitsky and Taqqu 1994] [See also Barndorff-Nielsen and Shephard (2001,2001)]

20 Now, taking W i to be iid N(¹ W,¾ W2 ), we have a Gaussian structure, conditional upon the terms i This can be readily handled using Bayesian Monte Carlo inference. As a significant bonus, (¹ W,¾ W2 ) can be analytically marginalised, leaving only to be sampled The trade-off is that we must also simulate an infinite series of i terms in practice we do this by truncating the series and approximating the residual.

21

22 Truncation of the Series In practice cannot compute the entire infinite series: Instead, truncate at a limit, i <c, and approximate the residual with a Gaussian matched to first two moments of the residual (analytically computed).

23

24

25 Inference Schemes Consider e.g. A MCMC inference algorithm for the parameters and the latent variables. Parameters are, ¾ W, ¹ W, {m,v} The challenging part is the latent variables (m,v), one for each data point.

26 Latent Variables (m,v) Can sample directly from the prior p(m,v) and use rejection sampling (unbounded envelope, but can fix (condition on) the residual terms and make it bounded). Not good enough for large values of X (slow in the tails!); instead, look at p(m,v):

27 ca Observe that for large m, V is approximately m 2 This means that we can force prior samples of m into the tails of the distribution ( -stable and hence pareto) and perform rejection sampling with likelihood constrained to the line V=m 2 This is a rare-event tail approximation for large X values, but the approximation appears very good and speeds up sampling dramatically. Can tune accuracy vs. speed by selecting the switchover point between optimal (slow) and tail approximation (fast).

28 State-space models and continuous time Incorporation into discrete-time statespace models is fairly straightforward likelihood computation can then be done by a mean- and scale-shifted Kalman filter Continuous time is also a fairly straightforward extension also need to sample the {V i }

29

30

31

32 Conclusion A general framework for inference of -stable distribution parameters, linear state-space models and continuous-time Levy processes Straightforward computations using conditionally Gaussian models (Shephard (1993 Biometrika), Carter and Kohn (1994 Biometrika)), and particle filters Currently exploring Particle MCMC methods for parameter estimation

33 References S. J. Godsill, J. Vermaak, W. Ng and J.F. Li. Variable rate particle filters for tracking applications in Proceedings of the IEEE, Special Issue on Large Scale Dynamical Systems, S. J. Godsill and J. Vermaak. Variable rate particle filters for tracking applications. In Proc. IEEE Stat. Sig. Proc., Bordeaux, S. J. Godsill and J. Vermaak, Models and algorithms for tracking using trans-dimensional sequential Monte Carlo. In Proc. IEEE ICASSP 2004 Barndorff-Nielsen, Ole E. and Neil Shephard (2001) "Normal modified stable processes", Theory of Probability and Mathematical Statistics, 2001, Barndorff-Nielsen, Ole E. and Neil Shephard (2001) "Non-Gaussian Ornstein-Uhlenbeck-based models and some of their uses in financial economics", (with discussion) Journal of the Royal Statistical Society, Series B, 63, Buckle, D. J. (1995), Bayesian inference for stable distributions, JASA, 90: S. J. Godsill. Inference in symmetric alpha-stable noise using MCMC and the slice sampler. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, volume VI, pages , ISBN S. J. Godsill. MCMC and EM-based methods for inference in heavy-tailed processes with alpha-stable innovations. In Proc. IEEE Signal processing workshop on higher-order statistics, June Caesarea, Israel.[ bib.ps ] S. J. Godsill and E. E. Kuruoglu. Bayesian inference for time series with heavy-tailed symmetric α-stable noise processes. In Proc. Applications of heavy tailed distributions in economics, engineering and statistics, June Washington DC, USA.[ bib.ps ] S.J. Godsill and L. Yang. Bayesian inference for continuous-time AR models driven by non-gaussian lévy processes. In Proc. IEEE International Conference on Acoustics, Speech and Signal Processing, Toulouse, France, May Tatjana Lemke, Simon J. Godsill: Enhanced Poisson sum representation for alpha-stable processes. ICASSP 2011: Tatjana Lemke, Simon J. Godsill: LINEAR GAUSSIAN COMPUTATIONS FOR NEAR-EXACT BAYESIAN MONTE CARLO INFERENCE IN SKEWED ALPHA-STABLE TIME SERIES MODELS, ICASSP 20112