Areal Unit Data Regular or Irregular Grids or Lattices Large Point-referenced Datasets

Areal Unit Data Regular or Irregular Grids or Lattices Large Point-referenced Datasets Is there spatial pattern? Chapter 3: Basics of Areal Data Models p. 1/18

Areal Unit Data Regular or Irregular Grids or Lattices Large Point-referenced Datasets Is there spatial pattern? Do we want to smooth the data? Chapter 3: Basics of Areal Data Models p. 1/18

Areal Unit Data Regular or Irregular Grids or Lattices Large Point-referenced Datasets Is there spatial pattern? Do we want to smooth the data? Inference for new areal units? Chapter 3: Basics of Areal Data Models p. 1/18

Areal Unit Data Regular or Irregular Grids or Lattices Large Point-referenced Datasets Is there spatial pattern? Do we want to smooth the data? Inference for new areal units? Descriptive/algorithmic vs. Model-based Chapter 3: Basics of Areal Data Models p. 1/18

Areal unit data Actual Transformed SIDS Rates <2.0 2.0-3.0 3.0-3.5 >3.5 Chapter 3: Basics of Areal Data Models p. 2/18

Proximity matrices W, entries w ij (with w ii = 0). Choices for w ij : w ij = 1 if i,j share a common boundary (possibly a common vertex) w ij is an inverse distance between units w ij = 1 if distance between units is K w ij = 1 for m nearest neighbors. W is typically symmetric, but need not be W : standardize row i by w i+ = j w ij (so matrix is now row stochastic, but probably no longer symmetric). Chapter 3: Basics of Areal Data Models p. 3/18

Measures of spatial association Moran s I: essentially an areal covariogram" I = n i j w ij(y i Ȳ )(Y j Ȳ ) ( i j w ij) i (Y i Ȳ )2 Chapter 3: Basics of Areal Data Models p. 4/18

Measures of spatial association Moran s I: essentially an areal covariogram" I = n i j w ij(y i Ȳ )(Y j Ȳ ) ( i j w ij) i (Y i Ȳ )2 Geary s C: essentially an areal variogram" C = (n 1) i j w ij(y i Y j ) 2 ( i j w ij) i (Y i Ȳ )2 Both are asymptotically normal if Y i are i.i.d.; Moran has mean 1/(n 1) 0, Geary has mean 1 Chapter 3: Basics of Areal Data Models p. 4/18

Measures of spatial association (cont d) <503 504-525 526-562 >563 Figure 1: Choropleth map of 1999 average verbal SAT scores, lower 48 U.S. states. Chapter 3: Basics of Areal Data Models p. 5/18

Measures of spatial association (cont d) For these data, the spatial.cor function in S+SpatialStats gives a Moran s I of 0.5833, with associated standard error estimate 0.0920 very strong evidence against H 0 : no spatial correlation spatial.cor also gives a Geary s C of 0.3775, with associated standard error estimate 0.1008 again, very strong evidence against H 0 (departure from 1) Warning: These data have not been adjusted for covariates, such as the proportion of students who take the exam (Midwestern colleges have historically relied on the ACT, not the SAT; only the best and brightest students in these states would bother taking the SAT) Chapter 3: Basics of Areal Data Models p. 6/18

Correlogram (via Moran s I) rho(d) -1.0-0.5 0.0 0.5 1.0 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 distance Replace w ij with w (1) ij taken from W (1) I (1) Chapter 3: Basics of Areal Data Models p. 7/18

Correlogram (via Moran s I) rho(d) -1.0-0.5 0.0 0.5 1.0 0.0 0.2 0.4 0.6 0.8 1.0 1.2 1.4 distance Replace w ij with w (1) ij taken from W (1) I (1) Replace w ij with w (2) ij taken from W (2) I (2), etc. Plot I (r) vs. r; an initial decline in r followed by variation around 0 spatial pattern! Chapter 3: Basics of Areal Data Models p. 7/18

Rasterized binary data map NORTH SOUTH land use classification non-forest forest Chapter 3: Basics of Areal Data Models p. 8/18

Binary data correlogram logodds 0 1 2 3 N 0 1 2 3 NW 0 50 100 150 200 0 20 40 60 distance 0 1 2 3 NE 0 1 2 3 E 0 20 40 60 0 10 20 30 40 50 A version for a binary map, using two-way tables and log odds ratios at the pixel level Chapter 3: Basics of Areal Data Models p. 9/18

Spatial smoothers To smooth Y i, replace with Ŷi = i w ijy j w i+ Chapter 3: Basics of Areal Data Models p. 10/18

Spatial smoothers To smooth Y i, replace with Ŷi = i w ijy j w i+ More generally, we could include the value actually observed for unit i, and revise our smoother to (1 α)y i + αŷi For 0 < α < 1, this is a linear (convex) combination in shrinkage" form Finally, we could try model-based smoothing, i.e., based on E(Y i Data), i.e., the mean of the predictive distribution. Smoothers then emerge as byproducts of the hierarchical spatial models we use to explain the Y i s Chapter 3: Basics of Areal Data Models p. 10/18

Markov random fields Consider Y = (Y 1,Y 2,...,Y n ) and the set {p(y i y j, j i)} Chapter 3: Basics of Areal Data Models p. 11/18

Markov random fields Consider Y = (Y 1,Y 2,...,Y n ) and the set {p(y i y j, j i)} We know p(y 1,y 2,...y n ) determines {p(y i y j,j i)} (the set of full conditional distributions) Chapter 3: Basics of Areal Data Models p. 11/18

Markov random fields Consider Y = (Y 1,Y 2,...,Y n ) and the set {p(y i y j, j i)} We know p(y 1,y 2,...y n ) determines {p(y i y j,j i)} (the set of full conditional distributions) Does {p(y i y j,j i)} determine p(y 1,y 2,...y n )??? If the full conditionals are compatible, then by Brook s Lemma, p(y 1,...,y n ) = p(y 1 y 2,...,y n ) p(y 10 y 2,...,y n ) p(y 2 y 10,y 3,...,y n ) p(y 20 y 10,y 3,...,y n )... p(y n y 10,...,y n 1,0 ) p(y n0 y 10,...,y n 1,0 ) p(y 10,...,y n0 ) If left side is proper, the fact that it integrates to 1 determines the normalizing constant! Chapter 3: Basics of Areal Data Models p. 11/18

Local modeling Suppose we specify the full conditionals such that p(y i y j,j i) = p(y i y j i ), where i is the set of neighbors of cell (region) i. When does {p(y i y j i )} determine p(y 1,y 2,...y n )? Def n: a clique is a set of cells such that each element is a neighbor of every other element Def n: a potential function of order k is a function of k arguments that is exchangeable in these arguments Def n: p(y 1,...,y n ) is a Gibbs distribution if, as a function of the y i, it is a product of potentials on cliques Chapter 3: Basics of Areal Data Models p. 12/18

Local modeling Cliques of size 1 independence Chapter 3: Basics of Areal Data Models p. 13/18

Local modeling Cliques of size 1 independence Cliques of size 2 pairwise difference form p(y 1,y 2,...y n ) exp 1 2τ 2 (y i y j ) 2 I(i j) and therefore p(y i y j,j i) = N( j i y j /m i,τ 2 /m i ), where m i is the number of neighbors of i Hammersley-Clifford Theorem: If we have a Markov Random Field (i.e., {p(y i y j i )} uniquely determine p(y 1,y 2,...y n )), then the latter is a Gibbs distribution i,j Chapter 3: Basics of Areal Data Models p. 13/18

Local modeling Cliques of size 1 independence Cliques of size 2 pairwise difference form p(y 1,y 2,...y n ) exp 1 2τ 2 (y i y j ) 2 I(i j) and therefore p(y i y j,j i) = N( j i y j /m i,τ 2 /m i ), where m i is the number of neighbors of i Hammersley-Clifford Theorem: If we have a Markov Random Field (i.e., {p(y i y j i )} uniquely determine p(y 1,y 2,...y n )), then the latter is a Gibbs distribution Geman and Geman result : If we have a joint Gibbs distribution, then we have a Markov Random Field i,j Chapter 3: Basics of Areal Data Models p. 13/18

Conditional autoregressive (CAR) model Gaussian (autonormal) case p(y i y j,j i) = N j b ij y j,τ 2 i Using Brook s Lemma we can obtain { p(y 1,y 2,...y n ) exp 1 } 2 y D 1 (I B)y where B = {b ij } and D is diagonal with D ii = τ 2 i. suggests a multivariate normal distribution with µ Y = 0 and Σ Y = (I B) 1 D Chapter 3: Basics of Areal Data Models p. 14/18

CAR Model (cont d) Returning to W, let b ij = w ij /w i+ and τ 2 i = τ 2 /w i+, so p(y 1,y 2,...y n ) exp { 1 } 2τ 2y (D w W)y where D w is diagonal with (D w ) ii = w i+ and thus p(y 1,y 2,...y n ) exp 1 2τ 2 w ij (y i y j ) 2 i j Intrinsic autoregressive (IAR) model! Chapter 3: Basics of Areal Data Models p. 15/18

CAR Model Issues With τ 2 unknown, appropriate exponent for τ 2? Hodges et al.: n I, where I is the # of islands Chapter 3: Basics of Areal Data Models p. 16/18

CAR Model Issues With τ 2 unknown, appropriate exponent for τ 2? Hodges et al.: n I, where I is the # of islands Proper version: replace D w W by D w ρw, and choose ρ so that Σ y = (D w ρw) 1 exists! This in turn implies Y i Y j i N(ρ j w ijy j,τ 2 /m i ) To ρ or not to ρ? (that is the question) Chapter 3: Basics of Areal Data Models p. 16/18

CAR Models with ρ parameter Disadvantages: why should we expect Y i to be a proportion of average of neighbors a sensible spatial interpretation? Chapter 3: Basics of Areal Data Models p. 17/18

CAR Models with ρ parameter Disadvantages: why should we expect Y i to be a proportion of average of neighbors a sensible spatial interpretation? calibration of ρ as a correlation, e.g., ρ = 0.80 yields 0.1 Moran s I 0.15, ρ = 0.90 yields 0.2 Moran s I 0.25, ρ = 0.99 yields Moran s I 0.5 Chapter 3: Basics of Areal Data Models p. 17/18

Comments The CAR specifies Σ 1 Y, not Σ Y (as in the point-level modeling case), so does not directly model association Chapter 3: Basics of Areal Data Models p. 18/18

Comments The CAR specifies Σ 1 Y, not Σ Y (as in the point-level modeling case), so does not directly model association (Σ 1 Y ) ii = 1/τ 2 i ; (Σ 1 Y ) ij = 0 cond l independence Chapter 3: Basics of Areal Data Models p. 18/18

Comments The CAR specifies Σ 1 Y, not Σ Y (as in the point-level modeling case), so does not directly model association (Σ 1 Y ) ii = 1/τ 2 i ; (Σ 1 Y ) ij = 0 cond l independence Prediction at new sites is ad hoc, in that if p(y 0 y 1,y 2,...y n ) = N( j w 0j y j /w 0+,τ 2 /w 0+ ) then p(y 0,y 1,...y n ) well-defined but not CAR Chapter 3: Basics of Areal Data Models p. 18/18