Perfect simulation of repulsive point processes

Size: px

Start display at page:

Download "Perfect simulation of repulsive point processes"

Martha Flynn
5 years ago
Views:

1 Perfect simulation of repulsive point processes Mark Huber Department of Mathematical Sciences, Claremont McKenna College 29 November, 2011 Mark Huber, CMC Perfect simulation of repulsive point processes 1/56

2 Some repulsive things Spanish towns Pine trees Mark Huber, CMC Perfect simulation of repulsive point processes 2/56

3 Two data sets Locations: Spanish towns Locations: Swedish pines Mark Huber, CMC Perfect simulation of repulsive point processes 3/56

4 Points farther apart than under uniformity Locations: Spanish towns Locations: Uniform placement Mark Huber, CMC Perfect simulation of repulsive point processes 4/56

5 Modeling repulsion Mark Huber, CMC Perfect simulation of repulsive point processes 5/56

6 Modeling repulsion Two common modeling approaches 1 Give story about how process developed (time evolution) 2 Family of densities with respect to Poisson point process (density) Both based on Poisson point process # of points has Poisson distribution Given the # of points, place points uniformly independently Mean of Poisson is λ times size of region Mark Huber, CMC Perfect simulation of repulsive point processes 6/56

7 Today s talk Matérn type III process Time evolution Density Strauss process Density Time evolution Mark Huber, CMC Perfect simulation of repulsive point processes 7/56

8 Strauss process uses density Penalize PPP as follows Distance parameter R Penalty parameter γ (0, 1) Intensity parameter λ For points x, let r(x) = # of pairs less than R apart For points x, density f (x) = γ r(x) λ #x /Z (γ, λ) Parameter effects Big R spaces points farther apart Small γ means fewer points violate R distance Higher λ means more points Mark Huber, CMC Perfect simulation of repulsive point processes 8/56

9 Example of Strauss R =.02 f (x) = γ 6 λ 55 / Z Mark Huber, CMC Perfect simulation of repulsive point processes 9/56

10 Basic perfect simulation for Strauss Acceptance rejection Draw a PPP X with intensity λ Find r(x) Accept X as Strauss draw with probability γ r(x) Otherwise, reject and start over Takes too long unless λ small or γ big Mark Huber, CMC Perfect simulation of repulsive point processes 10/56

11 Matérn Type I Matérn is a storyteller Introduced three ways of explaining the repulsion Type I, II, and III Simulate a Matérn Type I as follows First simulate a Poisson point process (PPP) Remove any point within R distance of another point Mark Huber, CMC Perfect simulation of repulsive point processes 11/56

12 Matérn Type I example a b c Circles of radius R/2 Circles touch = points eliminated Points a, b, c eliminated d e Mark Huber, CMC Perfect simulation of repulsive point processes 12/56

13 Type I: After removal Call a, b, c ghost points a b c Call d, e seen points Ghost points exert invisible pressure d e Mark Huber, CMC Perfect simulation of repulsive point processes 13/56

14 Type I: Comments Problems Too many eliminations As λ, # of points 0 Need method that preserves some points Mark Huber, CMC Perfect simulation of repulsive point processes 14/56

15 Matérn Type II: Older points rule! Simulate a Matérn Type II as follows First simulate a Poisson point process (PPP) Assign each point a birthday in [0, ) Remove any point within R distance of an older point Mark Huber, CMC Perfect simulation of repulsive point processes 15/56

16 Type II: Picture a b c.84 Circles of radius R/2 Point a eliminates b Point b eliminates c d.01 e.52 Mark Huber, CMC Perfect simulation of repulsive point processes 16/56

17 Type II: After thinning Call b, c ghost points a b c Call a, d, e seen points Ghost points exert invisible pressure d e Mark Huber, CMC Perfect simulation of repulsive point processes 17/56

18 Type II: Comments Better, but not perfect First points survive Higher number of points than Type I Why should b take out c if already killed by a? Mark Huber, CMC Perfect simulation of repulsive point processes 18/56

19 Matérn Type III Simulate a Matérn Type III as follows First simulate a Poisson point process (PPP) Assign each point a birthday in [0, ) Run time forward Only allow point birth if not within R of older born point Mark Huber, CMC Perfect simulation of repulsive point processes 19/56

20 Type III: Picture a b c.84 Circles of radius R/2 Point a eliminates b d.01 e.52 Mark Huber, CMC Perfect simulation of repulsive point processes 20/56

21 Type III: After thinning Call c ghost point a b c Call a, c, d, e seen points Ghost points exert invisible pressure d e Mark Huber, CMC Perfect simulation of repulsive point processes 21/56

22 Type III: Comments The good The bad Very natural story Try to add towns/trees If too close to existing town/tree, dies off Density not in closed form nasty high dimensional integral Makes it difficult to do maximum likelihood estimate/posterior Mark Huber, CMC Perfect simulation of repulsive point processes 22/56

23 Using Matérn Type III for inference The plan First turn story of Matérn into density (as with Strauss) Build Markov chain for density Build perfect sampler around Markov chain Build product estimator to utilize samples effectively (Those last two are my area of research) Mark Huber, CMC Perfect simulation of repulsive point processes 23/56

24 Matérn Type III: Time evolution to density Mark Huber, CMC Perfect simulation of repulsive point processes 24/56

25 Casting shadows The seen points cast a shadow across time time 1 R = space Any ghost points must lie in shaded region More shadow = more space for ghost points Mark Huber, CMC Perfect simulation of repulsive point processes 25/56

26 From shadows to density Poisson process in spacetime Looks like PPP conditioned so that no points lie in shadow Parameters θ = (λ, R), region S Let A θ (x, t) be the area of the shadow in spacetime For seen points x with no shadow violations: f seen points (x, t θ) = Cλ #x exp(λa θ (x, t)) Mark Huber, CMC Perfect simulation of repulsive point processes 26/56

27 Larger shadow means more likely Alternate way of drawing Matérn Type III process Fix points in x Draw new PPP y to add to x If all of y lies in shadow of x, accept x as Matérn Type III Otherwise draw new y and repeat Mark Huber, CMC Perfect simulation of repulsive point processes 27/56

28 Acceptance/rejection leads to density What is chance of accepting a given y? Let S be size of region, A θ (x, t) size of shadow Probability no points outside of shadow exp( λ( S A θ (x, t))) = exp(λa θ (x, t)) exp( λ S ) Mark Huber, CMC Perfect simulation of repulsive point processes 28/56

29 The problem of unknown time of birth Big problem We do not see the t values! To get the density for just x integrate out t: g(x θ) = Cλ t [0,1] #x exp(λa θ (x, t)) #x Doing this integral directly extremely nasty Even calculating A θ (x, t) hard Mark Huber, CMC Perfect simulation of repulsive point processes 29/56

30 Monte Carlo methods to the rescue! Given seen points x, want to approximate g(x θ) Monte Carlo approach: treat t values as auxiliary variables Given x: randomly choose t from f (x, t θ, x) Allows use to estimate g(x θ) Mark Huber, CMC Perfect simulation of repulsive point processes 30/56

31 How to draw time stamps given locations? Markov chain Monte Carlo (MCMC) 1 Build chain so stationary distribution = target distribution 2 Under mild conditions (φ-irreducibility, aperiodicity) limiting distribution will equal stationary distribution 3 Run chain for a long time to mix, then get samples Mark Huber, CMC Perfect simulation of repulsive point processes 31/56

32 Metropolis protocol for building Markov chain One step of Metropolis 1 Propose moving from (x, t) to (x, t ) 2 Accept move with probability { f (x, t } θ) min f (x, t θ), 1 3 Otherwise stay where you are Mark Huber, CMC Perfect simulation of repulsive point processes 32/56

33 Even better... Perfect simulation techniques For some Markov chains, possible to do better Can draw exactly from stationary distribution Without worrying about mixing time Perfect sampling protocol Coupling from the past (Propp & Wilson 1994) CFTP converts Markov chains to perfect samplers A good property of Markov chain: monotonicity Monotonicity not necessary for CFTP, is sufficient Mark Huber, CMC Perfect simulation of repulsive point processes 33/56

34 Is Metropolis for Matérn type III monotonic? Propose changing time stamp for one point in x How does the shadow change? time 1 R = space Probability accept move = exp(-area of shadow change) Mark Huber, CMC Perfect simulation of repulsive point processes 34/56

35 How to flip an exp( µ) coin Fun facts about Poisson point processes For X PPP(B), P(#X = 0) = exp( µ(b)) For regions A B, X PPP(B) then X A PPP(A) To check if move in Metropolis Draw PPP(largest possible change in shadow) If PPP restricted to actual change in shadow empty, move Otherwise, stay at current time stamp Using PPP to flip exponential coins: First appears in Beskos, Papaspiliopoulos, Roberts (2006) Perfect simulation of diffusions Mark Huber, CMC Perfect simulation of repulsive point processes 35/56

36 Example time 1 R = space Mark Huber, CMC Perfect simulation of repulsive point processes 36/56

37 Example time 1 R = space Mark Huber, CMC Perfect simulation of repulsive point processes 36/56

38 Example time 1 R = space Mark Huber, CMC Perfect simulation of repulsive point processes 36/56

39 Montonicity of Markov chain step For t 1 t 2 : Run one step of Markov chain for t 1 and t 2 Use same auxiliary PPP in change of shadow Then after step, still have t 1 t 2 Immediately gives us perfect sampling! Mark Huber, CMC Perfect simulation of repulsive point processes 37/56

40 Example of Monotonicity time 1 R = 1.5 t 2 t 2 t 2 0 t 1 t 1 t 1 10 space Mark Huber, CMC Perfect simulation of repulsive point processes 38/56

41 Example of Monotonicity time 1 R = 1.5 t 2 t 2 t 2 0 t 1 t 1 t 1 10 space Mark Huber, CMC Perfect simulation of repulsive point processes 38/56

42 Example of Monotonicity time 1 R = 1.5 t 2 t 2 t 2 0 t 1 t 1 t 1 10 space t 2 accepts move Mark Huber, CMC Perfect simulation of repulsive point processes 38/56

43 Example of Monotonicity time 1 R = 1.5 t 2 t 2 t 2 0 t 1 t 1 t 1 10 space t 2 accepts move t 1 rejects move Mark Huber, CMC Perfect simulation of repulsive point processes 38/56

44 Monotonic CFTP flowchart CFTP(k) Output t 1 YES t 1 all zeros t 2 all ones save RNG seed Run Markov chain for k steps Does t 1 = t 2? RNG = Random Number Generator NO t 1 CFTP(2k) t 2 t 1 reset RNG seed Mark Huber, CMC Perfect simulation of repulsive point processes 39/56

45 Monotonic CFTP details Running CFTP(k) 1 Set t 1 to all 0 s, t 2 to all 1 s 2 Save seed to random number generator 3 Take k steps in the Markov chain 4 If t 1 = t 2, then set T k to be this common value 5 Else 1 Get T 0 by calling CFTP(2k) recursively 2 Reset seed to random number generator to what it was in step 2 3 Get T k by taking k steps in the Markov chain 6 Output T k Mark Huber, CMC Perfect simulation of repulsive point processes 40/56

46 What s the point? Why do we need samples? Ability to sample gives approximation of nasty integral Use TPA or IS+TPA to go from samples to integral Once you have that integral Gives density of data under Matérn model Basis for maximum likelihood...or posterior analysis Mark Huber, CMC Perfect simulation of repulsive point processes 41/56

47 Results: towns Mark Huber, CMC Perfect simulation of repulsive point processes 42/56

48 Results: trees Mark Huber, CMC Perfect simulation of repulsive point processes 43/56

49 Strauss: Density to time evolution Mark Huber, CMC Perfect simulation of repulsive point processes 44/56

50 Preston (1977) Birth-Death Chains Adding time dimension to Poisson point process time Gray bar = node lifespan length bar exp(1) birth birth birth 0 space birth birth birth time between births exp(λµ(s)) Mark Huber, CMC Perfect simulation of repulsive point processes 45/56

51 Preston Birth-Death Chains Points are born, and later die Rate of births is λ times area of region Each point dies at rate 1 (Rate is parameter of exponential random variable) Points alive at time 0 form PPP Mark Huber, CMC Perfect simulation of repulsive point processes 46/56

52 Preston for Strauss Recall Strauss density Penalty γ for pair of points within distance R When point is born... If point within distance R of point already born......point only born with probability γ Mark Huber, CMC Perfect simulation of repulsive point processes 47/56

53 Picture for Strauss γ = 0 time birth birth birth 0 space birth birth birth Mark Huber, CMC Perfect simulation of repulsive point processes 48/56

54 Observations Red points Strauss process subset of earlier points Kendall and J. Møller (2000): can find red points by looking backwards in time Mark Huber, CMC Perfect simulation of repulsive point processes 49/56

55 Adding a Swap move Swap move Broder (1986): swap move for perfect matchings Dyer and Greenhill (2000): swap move for independent sets of graphs Adding Swaps to Preston Huber (2011) When point not born, give chance to swap If only blocked by one point......remove blocker, allow birth Mark Huber, CMC Perfect simulation of repulsive point processes 50/56

56 Picture with swap time birth birth birth 0 space birth birth birth Mark Huber, CMC Perfect simulation of repulsive point processes 51/56

57 Swap move helps chain mix better Easier to find red points Verified experimentally on plane Points affect fewer points in future Need to be blocked by at least two points to affect future So effect of point on later points cut in half Mark Huber, CMC Perfect simulation of repulsive point processes 52/56

58 Running times with swap Strauss model on S = [0, 1] 2, γ = 0.5, R = x 106 Running time for swap and no swap chains 3 Average number of events needed per sample No swapping Always swap when possible β 1 λ Mark Huber, CMC Perfect simulation of repulsive point processes 53/56

59 Running times with swap Time for no swap divided by time for swap 5 No swap times divided by swap times 4.5 Average no swap divided by average swap β 1 λ Mark Huber, CMC Perfect simulation of repulsive point processes 54/56

60 Conclusions For Matérn type III models Built a density (not in closed form) Can approximate density using perfect MCMC methods Allows MLE or posterior analysis For Strauss process Already had a density (but not in closed form) Added new type of Markov chain move Seems to speed up chain in practice Mark Huber, CMC Perfect simulation of repulsive point processes 55/56

61 References HUBER, M.L. (2011). Spatial Birth-Death-Swap Chains. Bernoulli (forthcoming paper, available online) HUBER, M.L. AND R.L. WOLPERT (2009). Likelihood based inference for Matérn type III repulsive point processes. Advances in Applied Probability 41, MATÉRN, B. (1986). Spatial Variation, vol. 36 of Lecture Notes in Statistics. New York, NY: Springer-Verlag, 2nd ed. (first edition published 1960 by Statens Skogsforsningsinstitut, Stockholm). Mark Huber, CMC Perfect simulation of repulsive point processes 56/56

Perfect simulation for repulsive point processes

Perfect simulation for repulsive point processes Why swapping at birth is a good thing Mark Huber Department of Mathematics Claremont-McKenna College 20 May, 2009 Mark Huber (Claremont-McKenna College)