Approximations for two-dimensional discrete scan statistics in some dependent models

Transcription

1 Approximations for two-dimensional discrete scan statistics in some dependent models Alexandru Am rioarei Cristian Preda Laboratoire de Mathématiques Paul Painlevé Département de Probabilités et Statistique Université de Lille 1, INRIA Modal Team IMS-China International Conference on Statistics and Probability 30 June - 4 July, 2013, Chengdu, China A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

2 Outline 1 Introduction Framework and Previous Work Block-Factor Type Model 2 Description of the method Main Idea and Tools The Approximation 3 Error Bound Approximation Error Simulation Error 4 Illustrative Examples Description of the Examples 5 References A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

3 Introduction Framework and Previous Work Outline 1 Introduction Framework and Previous Work Block-Factor Type Model 2 Description of the method Main Idea and Tools The Approximation 3 Error Bound Approximation Error Simulation Error 4 Illustrative Examples Description of the Examples 5 References A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

4 Introduction Framework and Previous Work The Two-Dimensional Discrete Scan Statistic Let N 1, N 2 be positive integers N 2 j 1 X i,j N 1 N 2 Rectangular region R = [0, N 1 ] [0, N 2 ] (X ij ) 1 i N 1 1 j N2 integer rv's Bernoulli(B(1, p)) Binomial(B(n, p)) Poisson(P(λ)) X ij number of observed events in the elementary subregion r ij = [i 1, i] [j 1, j] 1 i N 1 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

5 Introduction Framework and Previous Work The Two-Dimensional Discrete Scan Statistic Let m 1, m 2 be positive integers N 1 Dene for 1 i j N j m j + 1, i1+m1 1 Y = i 1i2 i2+m2 1 X ij i=i1 j=i2 m 2 Y i 1,i2 N 2 The two dimensional scan statistic, m 1 S m 1,m2 (N 1, N 2 ) = max Y i 1i2 1 i1 N1 m1+1 1 i2 N2 m2+1 Used for testing the null hypotheses of randomness against the alternative hypothesis of clustering A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

6 Introduction Framework and Previous Work Problem and related results Problem Approximate the distribution of two dimensional discrete scan statistic P (S m 1,m2 (N 1, N 2 ) n) The iid model: No exact formulas For Bernoulli case: product type approximations (Boutsikas and Koutras 2000) Poisson approximations (Chen and Glaz 1996) bounds (Boutsikas and Koutras 2003) For binomial and Poisson cases: (Glaz 2009) Product type approximation Lower bound Approximation and error bounds (Haiman 2006) Dependent model: no results! A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

7 Introduction Block-Factor Type Model Outline 1 Introduction Framework and Previous Work Block-Factor Type Model 2 Description of the method Main Idea and Tools The Approximation 3 Error Bound Approximation Error Simulation Error 4 Illustrative Examples Description of the Examples 5 References A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

8 Introduction Block-Factor Type Model Introducing the Model Let 1 c s Ñ s, s {1, 2} integers ( Xij) 1 i Ñ1 1 j Ñ2 iid rv's conguration matrix in (i, j) C (i,j) = ( C (i,j) (k, l) ) 1 k c2 1 l c1 X i,j+c 2 1 X i,j T Xi +c 1 1,j+c 2 1 X i +c 1 1,j C (i,j) (k, l) = Xi+l 1,j+c 2 k transformation T : M c 2,c1 (R) R X i,j Dene the block-factor model, N 1 = Ñ 1 c 1 + 1, N 2 = Ñ 2 c X i,j = T ( C (i,j) ), 1 i N1 1 j N2 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

9 Introduction Block-Factor Type Model Model: case c 1 = c 2 = 3 To simplify the presentation we consider c 1 = c 2 = 3 X i,j+2 Xi+1,j+2 Xi+2,j+2 X i,j+1 Xi+1,j+1 Xi+2,j+1 Ñ2 X i,j Xi+1,j Xi+2,j X i,j N2 j T j i Ñ1 1 i N1 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

10 Introduction Block-Factor Type Model Model: case c 1 = c 2 = 3 To simplify the presentation we consider c 1 = c 2 = 3 X i,j+2 Xi+1,j+2 Xi+2,j+2 X i,j+1 Xi+1,j+1 Xi+2,j+1 Ñ2 X i,j Xi+1,j Xi+2,j X i,j N2 j T j i Ñ1 1 i N1 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

11 Introduction Block-Factor Type Model Example: A game of minesweeper Model: X i,j B(p) (presence, absence of a mine) number of neighboring mines T (C (i,j) ) = (s,t) {0,1,2} 2 (s,t) (1,1) ) X i,j = T (C (i,j) X i+s,j+t A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

12 Description of the method Main Idea and Tools Outline 1 Introduction Framework and Previous Work Block-Factor Type Model 2 Description of the method Main Idea and Tools The Approximation 3 Error Bound Approximation Error Simulation Error 4 Illustrative Examples Description of the Examples 5 References A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

13 Description of the method Main Idea and Tools Key Idea Haiman(2000) proposed a dierent approach Main Observation The scan statistic rv can be viewed as a maximum of a sequence of 1-dependent stationary rv The idea: discrete and continuous one dimensional scan statistic: Haiman (2000,2007) discrete and continuous two dimensional scan statistic: Haiman and Preda (2002,2006) discrete three dimensional scan statistic: Amarioarei (2013) A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

14 Description of the method Main Idea and Tools Writing the Scan as an Extreme of 1-Dependent RV's Let N j = (L j + 1)(m j + 1) 2, j {1, 2} positive integers Dene for l {1, 2,, L 2 } N2 Z l = max Y i 1i2 1 i1 L1(m1+1) (l 1)(m2+1)+1 i2 l(m2+1) (Z l ) l is 1-dependent and stationary Observe S m 1,m2 (N 1, N 2 ) = max Z l 1 l L2 4m m m m2 m2 + 2 m2 1 m1 Z3 2 Z2 Z1 N1 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

15 Description of the method Main Idea and Tools Main Tool Let (Z j ) j 1 be a strictly stationary 1-dependent sequence of rv's and let q m = q m (x) = P(max(Z 1,, Z m ) x), with x < sup{u P(Z 1 u) < 1} Main Theorem (Haiman 1999, Amarioarei 2012) For x such that P(Z 1 > x) = 1 q 1 α < 01 and m > 3 we have q 6(q 1 q 2 ) 2 + 4q 3 3q 4 m (1 + q 1 q 2 + q 3 q 4 + 2q q2 2 5q 1q 2 ) m 1(1 q 1 ) 3, q 2q 1 q 2 m [1 + q 1 q 2 + 2(q 1 q 2 ) 2 ] m 2(1 q 1 ) 2, 1 = 1 (α, q 1, m) = Γ(α) + mk(α) [ 2 = me(α, q 1, m) = m m + K(α)(1 q 1) + Γ(α)(1 q 1) m ] Selected values for K(α) and Γ(α) A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

16 Description of the method The Approximation Outline 1 Introduction Framework and Previous Work Block-Factor Type Model 2 Description of the method Main Idea and Tools The Approximation 3 Error Bound Approximation Error Simulation Error 4 Illustrative Examples Description of the Examples 5 References A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

17 Description of the method The Approximation First Step Approximation Using Main Theorem we obtain Dene Q 2 = P(Z 1 k) Q 3 = P(Z 1 k, Z 2 k) If 1 Q 2 α 1 < 01 the (rst) approximation N 2 m 2 m 1 N 1 R P(S k) 2Q2 Q3 [1+Q2 Q3+2(Q2 Q3) 2 ] L 2 where S = S m 1,m2 (N 1, N 2 ) Approximation error L2E(α 1, L2)(1 Q2) 2 N2 m1 N1 N1 N2 3m2 + 1 Q 2 2m2 m1 Q 3 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

18 Description of the method The Approximation Second Step Approximation Q 2 : For s {1, 2,, L1} Z (2) s = max (s 1)(m1+1)+1 i1 s(m1+1) 1 i2 m2+1 ( ) Q2 = P max Z (2) s k 1 s L1 Dene Q 22 = P(Z (2) 1 k) Q32 = P(Z (2) 1 k, Z (2) 2 k) Approximation (1 Q22 α 2 ) Q2 Error 2Q22 Q32 [1+Q22 Q32+2(Q22 Q32) 2 ] L 1 L1E(α 2, L1)(1 Q22) 2 Y i 1i2 Q 3 : For s {1, 2,, L1} Z (3) s = max (s 1)(m1+1)+1 i1 s(m1+1) 1 i2 2(m2+1) ( ) Q3 = P max Z (3) s k 1 l L1 Dene Q 23 = P(Z (3) 1 k) Q33 = P(Z (3) 1 k, Z (3) 2 k) Approximation (1 Q23 α 2 ) Q3 Error 2Q23 Q33 [1+Q23 Q33+2(Q23 Q33) 2 ] L 1 L1E(α 2, L1)(1 Q23) 2 Y i 1i2 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

19 Description of the method The Approximation Illustration of the Approximation Process N1 N2 R m2 N1 m1 N1 N2 N2 Q2 2m2 Q3 3m2 + 1 m1 m1 N1 N1 N1 N1 N2 2m1 Q22 N2 2m2 Q32 3m1 + 1 N2 N2 2m2 Q23 3m2 + 1 Q33 3m m1 3m1 + 1 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

20 Error Bound Approximation Error Outline 1 Introduction Framework and Previous Work Block-Factor Type Model 2 Description of the method Main Idea and Tools The Approximation 3 Error Bound Approximation Error Simulation Error 4 Illustrative Examples Description of the Examples 5 References A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

21 Error Bound Approximation Error Theoretical Approximation Error Dene for s {2, 3} H(x, y, m) = 2x y [1 + x y + 2(x y) 2 ] m, α 1 = 1 Q 3, α 2 = 1 Q 23, F 1 = E(α 2, L 1 ), F 2 = E(α 1, L 2 ), R s = H (Q 2s, Q 3s, L 1 ), The approximation error where B 2 is given by E app = L 2 F 2 B L 1L 2 F 1 [ (1 Q22 ) 2 + (1 Q 23 ) 2] B 2 = 1 R 2 + L 1 F 1 (1 Q 22 ) 2 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

22 Error Bound Simulation Error Outline 1 Introduction Framework and Previous Work Block-Factor Type Model 2 Description of the method Main Idea and Tools The Approximation 3 Error Bound Approximation Error Simulation Error 4 Illustrative Examples Description of the Examples 5 References A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

23 Error Bound Simulation Error Simulation Error for Approximation Formula If ITER is the number of simulations, we can say, at 95% condence level, Q rt ˆQrt 196 ˆQrt(1 ˆQrt) ITER = β rt, r, t {2, 3} where ˆQrt is the simulated value Dene for r {2, 3}, ( ) ˆQ r = H ˆQ2r, ˆQ3r, L 1 The simulation error corresponding to the approximation formula E sf = L 1 L 2 (β 22 + β 23 + β 32 + β 33 ) A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

24 Error Bound Simulation Error Simulation Error for Approximation Error Introducing C 2r = 1 ˆQ2r + β 2r, r {2, 3}, C 2 = 1 ˆQ2 + L 1 (β 22 + β 32 ) + L 1 F 1 C 2 22, The simulation error corresponding to the approximation E sapp = L 2 F 2 C2 2 + [ ] L 1L 2 F 1 C C23 2 The total error E total = E app + E sf + E sapp A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

25 Illustrative Examples Description of the Examples Outline 1 Introduction Framework and Previous Work Block-Factor Type Model 2 Description of the method Main Idea and Tools The Approximation 3 Error Bound Approximation Error Simulation Error 4 Illustrative Examples Description of the Examples 5 References A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

26 Illustrative Examples Description of the Examples A Game of Minesweeper - Part 2 Recall the model: X i,j B(p) iid representing the absence/presence of a mine X i,j - number of neighboring mines corresponding to (i, j) X i,j = ( ) T C (i,j) = (s,t) {0,1,2} 2 (s,t) (1,1) X i+s,j+t X i,j : X i,j : T A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

27 Illustrative Examples Description of the Examples Numerical Results for P(S m1,m 2 (N 1, N 2 ) n) Table 1 : m 1 = 3,m 2 = 3,N 1 = 42,N 2 = 42,p = 01,ITER = 10 7 n Sim Approx E app E sim E total Sim Approx Dep Dep Indep Indep A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

28 Illustrative Examples Description of the Examples Graphical Illustration: p = ECDF for dependent and independent model: p=01 independent dependent P(S 5 n) n A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

29 Illustrative Examples Description of the Examples Numerical Results for P(S m1,m 2 (N 1, N 2 ) n) Table 2 : m 1 = 3,m 2 = 3,N 1 = 42,N 2 = 42,p = 05,ITER = 10 7 n Sim Approx E app E sim E total Sim Approx Dep Dep Indep Indep A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

30 Illustrative Examples Description of the Examples Graphical Illustration: p = ECDF for dependent and independent model: p=05 independent dependent P(S 5 n) n A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

31 Illustrative Examples Description of the Examples Numerical Results for P(S m1,m 2 (N 1, N 2 ) n) Table 3 : m 1 = 3,m 2 = 3,N 1 = 42,N 2 = 42,p = 07,ITER = 10 7 n Sim Approx E app E sim E total Sim Approx Dep Dep Indep Indep A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

32 Illustrative Examples Description of the Examples Graphical Illustration: p = ECDF for dependent and independent model: p=07 independent dependent P(S 5 n) n A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

33 Illustrative Examples Description of the Examples Dependence Eect PMF: p=01 dependent independent PMF: p=025 dependent independent P(S = n) P(S = n) n n PMF: p=05 dependent independent PMF: p=07 dependent independent P(S = n) P(S = n) n n A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

34 References Glaz, J, Naus, J, Wallenstein, S: Scan statistic Springer (2001) Glaz, J, Pozdnyakov, V, Wallenstein, S: Scan statistic: Methods and Applications Birkhauser (2009) Amarioarei, A: Approximation for the distribution of extremes of one dependent stationary sequences of random variables, arxiv: v1 (submitted) Amarioarei, A: Approximation for the Distribution of Three-dimensional Discrete Scan Statistic, rxiv: (submitted) Boutsikas, MV, Koutras, M: Reliability approximations for Markov chain imbeddable systems Methodol Comput Appl Probab 2 (2000), Boutsikas, M and Koutras, M Bounds for the distribution of two dimensional binary scan statistics, Probability in the Engineering and Information Sciences, 17, , 2003 A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

35 References Chen, J and Glaz, J Two-dimensional discrete scan statistics, Statistics and Probability Letters 31, 5968, 1996 Haiman, G: First passage time for some stationary sequence Stoch Proc Appl 80 (1999), Haiman, G: Estimating the distribution of scan statistics with high precision Extremes 3 (2000), Haiman, G, Preda, C: A new method for estimating the distribution of scan statistics for a two-dimensional Poisson process Methodol Comput Appl Probab 4 (2002), Haiman, G, Preda, C: Estimation for the distribution of two-dimensional scan statistics Methodol Comput Appl Probab 8 (2006), Haiman, G: Estimating the distribution of one-dimensional discrete scan statistics viewed as extremes of 1-dependent stationary sequences J Stat Plan Infer 137 (2007), A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35

36 Appendix Selected Values for K(α) and Γ(α) α K(α) Γ(α) Table 4 : Selected values for K(α) and Γ(α) Return A Am rioarei, C Preda (INRIA) Scan 2d block factor IMS-China / 35