An introduction to spatial point processes

Transcription

1 An introduction to spatial point processes Jean-François Coeurjolly

2 1 Examples of spatial data 2 Intensities and Poisson p.p. 3 Summary statistics 4 Models for point processes

3 A very very brief introduction... The realization x, of a spatial point process defined on a space S is a (locally) finite set of objects x i S. x = {x 1,..., x n }, x i and n are random.

4 A very very brief introduction... The realization x, of a spatial point process defined on a space S is a (locally) finite set of objects x i S. x = {x 1,..., x n }, x i and n are random. Thank you for your attention!

5 Spatial data can be roughly and mainly classified into three categories : 1 Geostatistical data. 2 Lattice data. 3 Spatial point pattern

6 Geostatistical data observation of a (e.g.) continuous random variable at fixed locations meuse dataset (R package gstat) : topsoil heavy metal concentrations, at the observation locations, collected in a flood plain of the river Meuse. Main objective : interpolate the spatial data.

7 Lattice data (1) Percentage with blood group A in Eire Eire dataset (R package spdep) % of people with group A in eire, observed in 26 regions. The data are aggregated on the region random field on a network. under over 31.02

8 Lattice data (2) Lennon dataset (R package fields) Real-valued random field (gray scale image with values in [0, 1]). Defined on the network {1,..., 256}

9 Lattice data (3) Over-interpretation : xkcd.

10 Spatial point pattern (1) japanesepines Japanesepines dataset (R package spatstat) Locations of 65 trees on a bounded domain. S = R 2 (equipped with ). Questions of interest : Can we estimate the number of trees per unit volume? Homogeneous or inhomogeneous? Is there any independence, attraction or repulsion between trees?

11 Spatial point pattern (2) Longleaf dataset (R package spatstat) Locations of 584 trees observed with their diameter at breast height. S = R 2 R + (equipped with max(, )). longleaf Additional scientific questions : Can the mark explain the intensity of the number of trees? Does a large tree tend to have smaller trees close to it?

12 Spatial point pattern (3) Ants dataset (R package spatstat) Locations of 97 ants categorised into two species. S = R 2 {0, 1} (equipped with the metric max(, d M ) for any distance d M on the mark space). ants Questions of interest : Competition inside one specie? between the two species?

13 Spatial point pattern (4) 3604 locations of trees observed with spatial covariates (here the elevation field). S = R 2 (equipped with the metric ), z( ) R 2. Questions of interest : Can the elevation field explain the arrangement of trees? Among a large number of spatial covariates, which ones have the largest influence?

14 Spatial point pattern (4) 3604 locations of trees observed with spatial covariates (here the elevation field). S = R 2 (equipped with the metric ), z( ) R 2. Questions of interest : Can the elevation field explain the arrangement of trees? Among a large number of spatial covariates, which ones have the largest influence?

15 Spatial point pattern (5) Spatio-temporal point process on a complex space Daily observation of sunspots at the surface of the sun. can be viewed as the realization of a marked spatio-temporal point process on the sphere. S = S 2 R + R + (state, time, and mark).

16 Spatial point pattern (6) : eye-movement data Eye-movement (on an image or video) is composed of sacades : exploratory step, local, very quick 120ms. fixations (< 1 of oscillation) ; analysing fixations allows to understand how a subject explores an image ; locations of fixations as well as their number are random.

17 Spatial point pattern (6) : eye-movement data Eye-movement (on an image or video) is composed of sacades : exploratory step, local, very quick 120ms. fixations (< 1 of oscillation) ; analysing fixations allows to understand how a subject explores an image ; locations of fixations as well as their number are random. Oculo-nimbus project (Univ. Grenoble) : aim to understand mechanisms of newborns vision Dozens of images Newborns of 3-, 6-, 9- and 12-month + adults control group 40 subjects per age group fixations by subject

18 Examples of spatial data Intensities and Poisson p.p. Summary statistics Models for point processes

19 Mathematical definition of a spatial point process Do you really want to look at this? S : Polish state space of the point process (equipped with the σ-algebra of Borel sets B). A configuration of points is denoted x = {x 1,..., x n,...}. For B S : x B = x B. N lf : space of locally finite configurations, i.e. {x, n(x B ) <, B bounded S} equipped with N lf = σ ( {x N lf, n(x B ) = m}, B B, B bounded, m 1 ). Definition A point process X defined on S is a measurable application defined on some probability space (Ω, F, P) with values on N lf. Measurability of X N (B) is a r.v. for any bounded B B.

20 Theoretical characterization of the distribution of X Proposition The distribution of a point process X is determined 1 by the joint distribution of N (B 1 ),..., N (B m ) for any bounded B 1,..., B m B and any m 1.

21 Theoretical characterization of the distribution of X Proposition The distribution of a point process X is determined 1 by the joint distribution of N (B 1 ),..., N (B m ) for any bounded B 1,..., B m B and any m 1. 2 or by its void probabilities, i.e. by P(N (B) = 0), for bounded B B.

22 Theoretical characterization of the distribution of X Proposition The distribution of a point process X is determined 1 by the joint distribution of N (B 1 ),..., N (B m ) for any bounded B 1,..., B m B and any m 1. 2 or by its void probabilities, i.e. by P(N (B) = 0), for bounded B B. For the rest of the talk : let S = R 2 or a bounded domain of R 2. everything can ± be extended to marked spatial point processes, spatio-temporal point processes, manifold-values point processes.

23 Moment measures Moments (mean, variance, covariance,...) play an important role in the characterization of a random variable (or a time series, random field) ; For point processes : moment measures which are related to moments of counting variables ; Definition : for n 1 we define the n-th order (reduced) moment measure (defined on S n ) by α (n) (D) = E 1({u 1,..., u n } D), D S n. u 1,...,u n X where the sign means that the n points are pairwise distinct.

24 Intensity functions Often (always) assumed that moment measures are absolutely continuous w.r.t. Lebesgue measures, so instead of the moment measures, the quantitites of interest (i.e. the ones you should keep in mind!) are : 1 Intensity function : ρ( ) : R d R + P(one event in B(u, du)) ρ(u) = lim du 0 du 2 Second-order intensity function : ρ (2) (, ) : R d R d R + ρ (2) (u, v) = P(2 distinct events in B(u, du) and B(v, dv)) lim du, dv 0 du dv 3 k-th order intensity function...

25 Campbell formula (1) Valid for any point process (having an intensity function) Campbell Theorem For any measurable function h : R d R (such that......) E h(u) = h(u)ρ(u)du. Examples : u X h(u) = 1(u W ), for W R d ( ) EN (W ) = ρ(u)du = ρ W if ρ( ) = ρ, homogeneous case W h(u) = 1(u W )ρ(u) 1, for W R d E ρ(u) 1 = W. u X W

26 Campbell formula (2) Valid for any point process (having a 2-nd...) Campbell Theorem For any measurable function h : R d R d R (such that......) Examples : E u,v X h(u, v) = h(u, v)ρ (2) (u, v)dudv. h(u) = 1(u A, v B) for A, B R d s.t. A B = E 1(u A)1(v B) = E (N (A)N (B)) = ρ (2) (u, v)dudv. u,v X Modelling/estimating ρ (2) allows to understand/model covariances of counting variables. A B

27 Covariances of counting variables For A, B R d s.t. A B = Cov (N (A), N (B)) = = where the function g given by A A B B { ρ (2) (u, v) ρ(u)ρ(v) } dudv ρ(u)ρ(v) {g(u, v) 1} dudv. g(u, v) = ρ(2) (u, v) ρ(u)ρ(v) is called the pair correlation function. The departure of g to 1 will measure some kind of independence for a point process X (wait for a few slides). If X is isotropic (i.e the distribution of X in invariant under rotation), then g(u, v) = g( v u ).

28 Poisson point processes Intuitive definition : X Poisson(S, ρ) m 1, bounded and disjoint B 1,..., B m S, the r.v. X B1,..., X Bm are independent. N (B) P ( B ρ(u)du) for any bounded A S. Poisson process is the reference model for point processes. PPP model points without any interaction! If ρ( ) = ρ, X is said to be homogeneous which implies EN (B) = ρ B, VarN (B) = ρ B.

29 A few realizations on W u = (u 1, u 2 ) W ρ(u) = βe u 1 u 2 1.5u3 1, W = [0, 1] 2. ρ = 100, W = [0, 1] 2. ρ(u) = βe 2 sin(4πu 1u 2 ), W = [ 1, 1] 2. (β is adjusted s.t. the mean number of points in W, W ρ(u)du = 200.) 110 points

30 A few properties of Poisson point processes Proposition : if X Poisson(W, ρ) Void probabilities : v(b) = P(N (B) = 0) = e B (ρ(u)du). For any u, v R d, ρ (2) (u, v) = ρ(u)ρ(v) g(u, v) = ρ2 (u, v) ρ(u)ρ(v) = 1 (also valid for ρ (k), k 1) Hence, for a general point process X g(u, v) < 1 means that two points are less likely to appear at u, v than for the Poisson model. characteristic for repulsive patterns g(u, v) > 1 means that two points are more likely to appear at u, v than for the Poisson model. characteristic for clustered patterns

31 Statistical inference for a Poisson point process Simulation : homogeneous case : very simple non-homogeneous case : a thinning procedure can be efficiently done. Inference : consists in estimating ρ, ρ( ; β) or ρ(u) depending on the context. All these estimates can be used even if the spatial point process is not Poisson (wait for 2 slides) Asymptotic properties very simple to derive under the Poisson assumption. Goodness-of-fit tests : tests based on quadrats counting, based on the void probability,...

32 Parametric intensity estimation Problem : model ρ(u) = ρ(u; β) for β R p, p 1 and estimate β. Example : forestry dataset ρ(u; β) = exp (β 1 + β 2 Alt(u) + β 3 Slope(u)) where Alt(u) and Slope(u) are spatial covariates corresponding to maps of altitude and slope of elevation. Assume we have a Poisson point process with intensity ρ(u; β) observed in W. Poisson likelihood It can be shown that the log-likelihood for this model writes (up to a normalizing constant) l W (X; β) = log ρ(u; β) ρ(u; β)du W u X W

33 Towards estimating equations Rathburn and Cressie (98) : MLE is consistent and asymptotically normal when X=Poisson ;......but the procedure is acutally valid for much more general point processes Evaluate the score function (gradient vector with length p) l ρ (u; β) (X, β) = ρ(u; β) ρ (u; β)du. u X W Campbell form. (valid for any X) : E l (X, β) = 0 So, l (X, β) is a nice estimating equation (see RW s talk). (Properties for the resulting estimator require more techniques). W

34 Examples of spatial data Intensities and Poisson p.p. Back to Eye-movement data Summary statistics Models for point processes

35 Example of modelling log ρ(u, m; β) = β 1 Saliency(u) + where 4 m =1 ( β m 0 + β m 1 Ad(u)) 1(m = m) Saliency(u) : Deterministic model of intensity map. Ad(u) : binary map built from the top 5% of ρ Ad (u). β m 0 : can be, the different number of points per age group. β m 1 : parameters of interest ; the values are not interesting but β 1 1 < < β4 1 is the cognitive hypothesis to test.

36 Adjusted contrasts tests Significant differences for 5 out of the 6 images.

37 Objective and classification Objective : Define some descriptive statistics for s.p.p. (independently on any model so). Measure the abundance of points, the clustering or the repulsiveness of a spatial point pattern w.r.t. the Poisson point process. Classification : First-order type based on the intensity function. Second-order type statistics : pair correlation function, Ripley s K function. Statistics based on distances : empy space function F, nearest-neigbour G, J function. (We assume that ρ and ρ (2) exist in the rest of the talk)

38 Ripley s K function (isotropic and planar case) Definition : let r 0 K (r) =ρ 1 E ( number of extra events within distance r of a randomly chosen event ) =ρ 1 E ( N (B(0, r) \ 0) 0 X ) L(r) = K (r)/π Properties : Under the Poisson assumption, K (r) = πr 2 ; L(r) = r. If K (r) > πr 2 or L(r) > r (resp. K (r) < πr 2 or L(r) < r) we suspect clustering (regularity) at distances lower than r. Application in practice : define a grid r values : r 1,..., r I ; find an estimator of K (r i ) or L(r i ), say ˆK (r i ) and ˆL(r i ) ; Plot e.g. (r i, ˆL(r i )) and compare with the Poisson case.

39 Edge corrected estimation of the K function Definition We define the border-corrected estimate as K BC (r) = 1 ρ 1 {N (B(u, r)) 1} N (W r ) u W r where W r = {u W : B(u, r) W } is the erosion of W by r. the translation-corrected estimate as K TC (r) = 1 ρ 1(v u B) 2 W W u,v X v u W where W u = W + u = {u + v : v W }. Remark : everything extends to 2nd-order reweighted stationary point processes ; asymptotic properties depend on mixing conditions,...

40 Example of L function for a Poisson point pattern r Linhom(r) Linhom obs(r) Linhom(r) Linhom hi(r) Linhom lo(r) The enveloppes are constructed using a Monte-Carlo approach under the Poisson assumption. we don t reject the Poisson assumption.

41 Example of L function for a repulsive point pattern r Linhom(r) Linhom obs(r) Linhom(r) Linhom hi(r) Linhom lo(r) the point pattern does not come from the realization of a homogeneous Poisson point process. exhibits repulsion at short distances (r.05)

42 Example of L function for a clustered point pattern Xth Linhom(r) Linhom obs(r) Linhom(r) Linhom hi(r) Linhom lo(r) the point pattern does not come from the realization of a homogeneous Poisson point process. exhibits attraction at short distances (r.08). r

43 Statistics based on distances : F, G and J functions Assume X is stationary (definitions can be extended in the general case) Definition The empty space function is defined by F (r) = P(d(0, X ) r) = P(N (B(0, r)) > 0), r > 0. The nearest-neighbour distribution function is G(r) = P(d(0, X \ 0) r 0 X ) J -function : J (r) = (1 G(r))/(1 F (r)), r > 0. Poisson case : r > 0, F (r) = G(r) = 1 e πr 2, J (r) = 1. F (r) < F pois (r), G(r) > G pois (r), J (r) < 1 : attraction at dist. < r. F (r) > F pois (r), G(r) < G pois (r), J (r) > 1 : repulsion at dist. < r.

44 Non-parametric estimation of F, G and J As for the K and L functions, several edge corrections exist. We focus here only on the border correction. We assume that X is observed on a bounded window W with positive volume. Definition Let I W be a finite regular grid of points and n(i ) its cardinality. Then, the (border corrected) estimator of F is where I r = I W r. F (r) = 1 1(d(u, X ) r) n(i r ) u I r The (border corrected) estimator of G is Ĝ(r) = 1 1(d(u, X \ u) r) N (W r ) u X W r

45 Application to a clustered point pattern data Xth F(r) F obs(r) F(r) F hi(r) F lo(r) r G(r) G obs(r) G(r) G hi(r) G lo(r) J(r) J obs(r) J(r) J hi(r) J lo(r) r r

46 More realistic models than the Poisson point process We can distinguish several classes of models for spatial point processes. Among them : 1 Cox point processes (which include Poisson Cluster point processes,... ). 2 Gibbs point processes. Strong links with statistical physics 3 Determinantal point processes. Links with random matrices

47 An attempt to classify these models... Model Allows to model Are moments Density w.r.t. explicit? Poisson? Cox attraction yes no Gibbs repulsion no yes but also attraction Determinantal repulsion yes yes This classification is really important since the methodologies to infer these models will be based either on moment methods or on conditional densities w.r.t. Poisson point process. asymptotic results require different tools : e.g. CLT based on mixing conditions (for Cox, determinantal point process) or on a martingale-type condition for Gibbs point process.

48 Cox point processes (1) Definition Suppose that Z = {Z (u) : u S} is a nonnegative random field so that with probability one, u Z (u) is a locally integrable function. If the conditional distribution of X given Z is a Poisson process on S with intensity function Z, then X is said to be a Cox process driven by Z. It is straightforwardly seen that 1 Provided Z (u) has finite expectation and variance for any u S ρ(u) = EZ (u), ρ (2) (u, η) = E[Z (u)z (η)], g(u, η) = 2 The void probabilities are given by ( ) v(b) = E exp Z (u)du B for bounded B S. E[Z (u)z (η)]. ρ(u)ρ(η)

49 Cox point processes (2) : Neymann-Scott process Definition Let C Poisson(R d, κ). Conditional on C, let X c Poisson(R d, ρ c ) be independent Poisson processes for any c C where ρ c (u) = αk(u c) where α > 0 is a parameter and k is a kernel (i.e. for all c R d, u k(u c) is a density function). Then X = c C X c is a Neymann-Scott process with cluster centres C and clusters X c, c C.

50 Cox point processes (2) : Neymann-Scott process Definition Let C Poisson(R d, κ). Conditional on C, let X c Poisson(R d, ρ c ) be independent Poisson processes for any c C where ρ c (u) = αk(u c) where α > 0 is a parameter and k is a kernel (i.e. for all c R d, u k(u c) is a density function). Then X = c C X c is a Neymann-Scott process with cluster centres C and clusters X c, c C. X is a Cox process on R d driven by Z (u) = c C αk(u c). When k is the Gaussian kernel, X is called the Thomas process.

51 Four realizations of Thomas point processes κ = 50, σ = 0.03, α = 5 κ = 100, σ = 0.03, α = 5 κ = 50, σ = 0.01, α = 5 κ = 100, σ = 0.01, α = 5

52 Cox point processes (4) : Log-Gaussian Cox processes Definition Let X be a Cox process on R d driven by Z = exp Y where Y is a Gaussian random field. Then, X is said to be a log Gaussian Cox process (LGCP). Basic properties : let m and c denote the mean function and the covariance function of Y 1 the intensition function of X is ρ(u) = exp (m(u) + c(u, u)/2). 2 The pair correlation function g of X is g(u, η) = exp(c(u, u)).

53 Four realizations of (stationary) LGCP point processes with exponential correlation function (δ = 1). The mean m of the Gaussian process is such that ρ = exp(m + σ 2 /2). σ = 2.5, α = 0.01, ρ = 100 σ = 2.5, α = 0.005, ρ = 100 σ = 2.5, α = 0.01, ρ = 200 σ = 2.5, α = 0.005, ρ = 200

54 Cox point processes (5) : parametric estimation method For most of the models, the likelihood is not available but moments are accessible. Then the idea is then to estimate θ using a minimum contrast approach : i.e. define ˆθ as the minimizer of r2 r 1 K (r) q K θ (r) q 2 dr or r2 r 1 ĝ(r) q g θ (r) q 2 dr where K (r) and ĝ(r) are the nonparametric estimates of K (r) and g(r). where [r 1, r 2 ] is a set of r fixed values. q is a power parameter (adviced in the literature to be set to q = 1/4 or 1/2).

55 Gibbs point process (1) We focus on the case S bounded. Definition A finite point process X on a bounded domain S is said to be a Gibbs point process if it admits a density f w.r.t. a Poisson point process with unit rate, i.e. for any F N f e S P(X F ) = n! n 0 S... S 1({x 1,..., x n } F )f ({x 1,..., x n })dx 1... dx n where the term n = 0 is read as exp( S )1( F )f ( ). Gpp can be viewed as a perturbation of a point process. f is easily interpretable weight w.r.t. a Poisson process. f specified up to an unknown constant f = c 1 h with exp( S ) c =... h({x 1,..., x n })dx 1... dx n = E[h(Y )] n! n 0 S S

56 Gibbs point process (2) : the most well-known class Definition An istotropic and homogeneous parwise interaction point process has a density of the form (for any x N f ) f (x) β n(x) φ 2 ( v u ) {u,v} x where φ 2 : R + R + is called the interaction function. The main example is the Strauss point process defined by f (x) β n(x) γ sr(x) where s R (x) = 1( v u R) {u,v} x where β > 0, R <, γ is called the interaction parameter : γ = 1 : homogeneous Poisson point process with intensity β. 0 < γ < 1 : repulsive point process. γ = 0 : hard-core process with hard-core R. γ > 1 : the model is not well-defined.

57 Realizations of a Strauss point process (simulation of spatial Gibbs point processes can be done using spatial birth-and-death process or using MCMC with reversible jumps, see Møller and Waagepetersen for details) β = 100, γ = 0, R = β = 100, γ = 0.3, R = β = 100, γ = 0.6, R = β = 100, γ = 1, R = 0.075

58 Gibbs point processes (3) : inference Likelihood unavailable : normalizing constant unknown, moments not expressible (e.g. in the stationary case ρ = Eλ(0, X )).

59 Gibbs point processes (3) : inference Likelihood unavailable : normalizing constant unknown, moments not expressible (e.g. in the stationary case ρ = Eλ(0, X )). Models (even when S = R d ) can be defined through the Papangelou conditional intensity f (x u) λ(u, x) =, x N lf, u S. f (x) Key-concept since several alternatives methods exist based on λ (and not on f ) including the pseudo-likelihood LPL W (x; θ) = λ(x, x \ u; θ) λ(u, x; θ)du. u x W W

60 Gibbs point processes (3) : inference Likelihood unavailable : normalizing constant unknown, moments not expressible (e.g. in the stationary case ρ = Eλ(0, X )). Models (even when S = R d ) can be defined through the Papangelou conditional intensity f (x u) λ(u, x) =, x N lf, u S. f (x) Key-concept since several alternatives methods exist based on λ (and not on f ) including the pseudo-likelihood LPL W (x; θ) = λ(x, x \ u; θ) λ(u, x; θ)du. u x W W Approaches and diagnostic tools use Georgii-Nguyen-Zessin formula E h(u, X \ u) = E(h(u, X )λ(u, X ))du. u X

61 Conclusion The anaysis of spatial point pattern very large domain of research including probability, mathematical statistics, applied statistics own specific models, methodologies and software(s) to deal with. is involved in more and more applied fields : economy, biology, physics, hydrology, environmentrics,...

62 Conclusion The anaysis of spatial point pattern very large domain of research including probability, mathematical statistics, applied statistics own specific models, methodologies and software(s) to deal with. is involved in more and more applied fields : economy, biology, physics, hydrology, environmentrics,... Still a lot of challenges Modelling : the true model, problems of existence, phase transition. Many classical statistical methodologies need to be adapted (and proved) to s.p.p. : robust methods, resampling techniques, multiple hypothesis testing. High-dimensional problems : S = R d with d large, selection of variables, regularization methods,... Space-time point processes.

63 References A. Baddeley and R. Turner. Spatstat : an R package for analyzing spatial point patterns. Journal of Statistical Software, 12 :1 42, N. Cressie. Statistics for spatial data. John Wiley and Sons, Inc, P. J. Diggle. Statistical Analysis of Spatial Point Patterns. Arnold, London, second edition, X. Guyon. Random Fields on a Network. Springer-Verlag, New York, J. Illian, A. Penttinen, H. Stoyan, and D. Stoyan. Statistical Analysis and Modelling of Spatial Point Patterns. Statistics in Practice. Wiley, Chichester, J. Møller and R. P. Waagepetersen. Statistical Inference and Simulation for Spatial Point Processes. Chapman and Hall/CRC, Boca Raton, 2004.