Global Spatial Autocorrelation Clustering

Size: px

Start display at page:

Download "Global Spatial Autocorrelation Clustering"

Amberly Montgomery
5 years ago
Views:

1 Global Spatial Autocorrelation Clustering Luc Anselin

2 join count statistics Moran s I Moran scatter plot non-parametric spatial autocorrelation

3 Join Count Statistics

4 Recap - Spatial Autocorrelation Statistic combination of attribute similarity and locational similarity f(x i, xj) and wij general form ΣiΣj f(xi,xj)wij

5 Spatial Autocorrelation for Binary Data only two values observed, e.g., presence and absence x i = 1, or B(lack) xi = 0, or W(hite) coding of 1 or 0 is arbitrary, typically choose 1 for the smaller category unlike event analysis (point patterns) all locations are observed with either 1 or 0 values

6 neighborhoods with burglaries (=1) Walnut Hills, Cincinnati, OH

7 Match Value-Location count the number of times similar values occur for neighbors = joins if x i = 1 and xj = 1 for i-j neighbors, count as a BB join if xi = 0 and xj = 0 for i-j neighbors, count as a WW join

8 Positive Spatial Autocorrelation similarity of neighbors BB = x ixj = 1 for a join, 0 otherwise WW = (1 - x i)(1 - xj) = 1 for a join, 0 otherwise

9 Negative Spatial Autocorrelation dissimilarity of neighbors BW = or (x i - xj) 2 = 1 for dissimilar neighbors 0 otherwise

10 Join Count Statistics binary spatial weights, w ij = 1 or 0 BB: (1/2) ΣiΣj wij.xi.xj WW: (1/2) ΣiΣj wij.(1-xi).(1-xj) BW: (1/2) ΣiΣj wij.(xi - xj) 2 BB + WW + BW = (1/2) S0 = (1/2) ΣiΣj wij

11 null hypothesis: spatial randomness

12 Actual Random 1 Random 2 BB WW BW Sum join count statistics - queen contiguity n = 457, S0 = 2968

13 Analytical Inference based on sampling theory binomial distribution, using sampling with replacement or without replacement moments (mean, variance, higher) can be computed uses a (poor) normal approximation Walnut Hills burglary BB = 130 E[BB] = 91.7, Var[BB] = 77.9 z-value = 4.34 p-value <

14 Permutation Inference recompute statistic (BB) builds a reference distribution reshuffle 0-1 values over locations compare observed statistic to reference distribution pseudo p-value = (M + 1)/(R + 1) M = values equal to or greater than statistic R = number of replications (e.g., 999)

15 reference distribution, BB = 130

16 reference distribution for spatially random values BB = 95

17 Moran s I 6

18 Moran s I the most commonly used of many spatial autocorrelation statistics I = [ Σi Σj wij zi.zj/s0 ]/[Σi zi 2 / N] with z i = yi - mx : deviations from mean cross product statistic (z i.zj) similar to a correlation coefficient value depends on weights (w ij)

19 Moran s I examined more closely scaling factors in numerator and denominator in numerator: S0 = Σi Σj wij the number of non-zero elements in the weights matrix, or the number of neighbor pairs (x2) in denominator: N the total number of observations

20 Inference how to assess whether computed value of Moran s I is significantly different from a value for a spatially random distribution compute analytically (assume normal distribution, etc.) computationally, compare value to a reference distribution obtained from a series of randomly permuted patterns

21 Approximate Inference assumes either normal distribution or randomization (each value equally likely to occur at any location) analytical derivation of moments (mean, variance) of the statistic under the null hypothesis (spatial randomness) z = (I - E[I]) / SD[I] approximate the resulting z-value by a standard normal distribution

22 Cleveland 2015 q4 house sales prices (in $1,000)

23 Normal Randomization MI E[MI] Var[MI] z-value p-value << << normal vs randomization inference for Moran s I queen weights

24 Queen K6 Distance MI E[MI] Var[MI] z-value p-value << << << Moran s I inference under randomization, different spatial weights

25 Permutation Approach as such, even a high Moran s I does not indicate significance need to construct a reference distribution randomly reshuffle observations and recompute Moran s I each time compare observed value to reference distribution

26 Standardized z-value standardize by subtracting mean and dividing by standard deviation, computed from the reference distribution z = [Observed I - Mean(I)] / Standard Deviation(I) z-values are comparable across variables and across spatial weights

27 999 permutations and reference distribution (queen weights) MI = Mean = s.d. = z-value = 7.15

28 Interpretation of Moran s I

29 Sign of Moran s I theoretical mean is - 1/(N - 1), essentially zero for large N positive and significant = clustering of like value NOT clustering of high or low could be either or a combination negative and significant = alternating values presence of spatial outliers spatial heterogeneity (checkerboard pattern)

30 Significance of Moran s I compute pseudo p-value compare standard z-value to standard normal (approximate only) only if Moran s I is significant is sign of coefficient meaningful

31 Comparing Moran s I Moran s I depends on spatial weights relative magnitude for same weights and different variables is meaningful but NOT for different spatial weights instead, use standardized z-value to compare

32 Queen K6 Distance MI E[MI] Var[MI] z-value p-value << << << Moran s I, different spatial weights MI is largest for K6, but z is largest for Distance

33 Clustering vs Clusters Moran s I is a global statistic, i.e., a single value for the whole spatial pattern Moran s I does NOT provide the location of clusters cluster detection requires a local statistic

34 True vs Apparent Contagion the indication of clustering does not provide an explanation for why the clustering occurs different processes can result in the same pattern true contagion: evidence of clustering due to spatial interaction (peer effects, mimicking, etc.) apparent contagion: evidence of clustering due to spatial heterogeneity (different spatial structures create local similarity)

35 Geary s c

36 Focus on Dissimilarity squared difference as measure of dissimilarity similar to notion of variogram (geostatistics) values between 0 and 2

37 Geary s c Statistic c = (N-1) Σi Σj wij (xi - xj) 2 / 2S0 Σ zi 2 alternatively c = [ Σi Σj wij (xi - xj) 2 / 2S0 ] / [ Σ zi 2 / (N-1) ] with z i in deviations from the mean S0 = Σi Σj wij sum of all the weights

38 Interpretation positive spatial autocorrelation c < 1 or z < 0 negative spatial autocorrelation c > 1 or z > 0 opposite sign of Moran s I

39 Inference same approach as for Moran s I analytical approximate (normal, randomization) permutation

40 Queen K6 Distance c E[c] Var[c] z-value p-value < << << geary s c inference under randomization, different spatial weights

41 999 permutations and reference distribution (queen weights) c = Mean = s.d. = z-value = -4.58

42 Moran Scatter Plot 6

43 Recap: Moran s I I = [ Σi Σj wij zi.zj/s0 ]/[Σi zi 2 / N] with z i = yi - mx : deviations from mean for row-standardized weights S0 = N I = Σi Σj wij zi.zj / Σi zi 2 = Σi zi (Σj wij.zj) / Σi zi 2 Moran s I is slope in a regression of Σj wij.zj on zi

44 Moran Scatter Plot scatter plot of [ zi, Σj wij.zj ] the value at i on the x-axis, its spatial lag (weighted average of neighboring values) on the y-axis slope of linear fit is Moran s I use local regression (Lowess) to identify possible structural breaks

45 outliers Moran scatter plot - Cleveland house prices - queen weights

46 outliers (in red)

47 Moran scatter plot without outliers

48 Categories of Local Spatial Autocorrelation four quadrants of the scatter plot upper right and lower left positive spatial autocorrelation clusters of like values locations are similar to their neighbors lower right and upper left negative spatial autocorrelation spatial outliers locations are different from their neighbors

49 Positive Spatial Autocorrelation all comparisons relative to the mean not absolute high or low high-high low-low

50 Negative Spatial Autocorrelation spatial outliers high-low low-high

51 Moran scatter plot with Lowess smoother

52 Nonparametric Spatial Covariance Function 6

53 Alternative Perspective - Nonparametric non-parametric approach no model for covariance let the data suggest the functional form based on sample autocorrelation covariance function must be positive definite

54 Sample Autocorrelation computed for each pair i, j ρij = ρ(zi,zj) = zi*zj* / (1/n) Σh (zh - zm) 2 z i* = zi - zm deviations from mean in practice, easier to use standardized zi incidental parameter problem one parameter for each pair i-j n(n-1)/2 individual values of ρij

55 Non-Parametric Principle spatial autocorrelation as an unspecified function of distance ρij = g(dij) how to fit the function? use kernel estimator

56 Kernel Estimator Hall-Patil (1994) ρ(d) = Σi Σj K(dij/h)(zi.zj) / Σi Σj K(dij/h) K is the kernel function (many choices) h is the bandwidth results depends critically on h, less so on K

57 Kernel Regression z izj = g(dij) local regression depends on choice of kernel function and bandwidth values of the estimated g(dij) do not necessarily result in a valid (positive semi-definite) variancecovariance matrix

58 correlogram - full distance range

59 correlogram - 20,000 ft max distance

60 Interpretation range of spatial autocorrelation alternative to specifying spatial weights sensitive to kernel fit may violate Tobler s law

Local Spatial Autocorrelation Clusters

Local Spatial Autocorrelation Clusters Luc Anselin http://spatial.uchicago.edu LISA principle local Moran local G statistics issues and interpretation LISA Principle Clustering vs Clusters global spatial