Review on Spatial Data

Transcription

1 Week 12 Lecture: Spatial Autocorrelation and Spatial Regression Introduction to Programming and Geoprocessing Using R GEO GEO Point data Review on Spatial Data Area/lattice data May be regular or irregular Recall the key components of spatial data: Spatial information (coordinates, CRS, etc.) Attributes Some of this lecture based on a workshop presented by Elisabeth Root, Department of Geography, University of Colorado, Boulder 1

2 Tobler s First Law of Geography "Everything is related to everything else, but near things are more related than distant things." Spatial autocorrelation is the formal property that measures the degree to which near and distant things are related The key: A statistical ttiti ltest tof match thbt between locational similarity and attribute similarity Positive, negative or zero relationship Spatial Regression Steps in determining the extent of spatial autocorrelation in your data and running a spatial regression: 1. Choose a neighborhood criterion Which areas are linked? 2. Assign weights to the areas that are linked Create a spatial weights matrix 3. Run statistical test to examine spatial autocorrelation 4. Run an OLS regression 5. Run a spatial regression(s) By applying weights matrices 2

3 Assessing Spatial Autocorrelation Is a spatial model worth the trouble? Spatial Weights Matrices Neighborhoods can be defined in multiple ways Contiguity it (common boundary) But what is a shared boundary? Distance (distance band, K nearest neighbors) How many neighbors to include, what distance do we use? General weights (social distance, exponential decay) 3

4 Importing shapefiles into R and constructing neighborhood sets STEP 1: CHOOSE A NEIGHBORHOOD CRITERION R Libraries for Spatial Analyses Can install our libraries individually using install.packages Or, we can install the entire Spatial set of libraries, called a view. > install.packages( ctv ) lib ( t ) > library( ctv ) > install.views( Spatial ) 4

5 Spatial View Maintained by Bivand, and described here: project.org/web/views/spatial.html Other views: project.org/web/views/ 5

6 R Libraries for Spatial Analyses If we install the entire Spatial view, then all our various spatial libraries i will then be available: > install.packages( ctv ) > library( ctv ) > install.views( Spatial ) > library(maptools) > library(rgdal) > library(spdep) 6

7 Loading an ESRI Shapefile > library(maptools) > getinfo.shape("c:/tmp/data/sids.shp") Shapefile type: Polygon, (5), # of Shapes: 100 > sids <- readshapepoly("c:/tmp/data/sids.shp") > class(sids) [1] "SpatialPolygonsDataFrame" attr(,"package") Loading a Shapefile With Projection Information > library(rgdal) > sids <- readogr( dsn="c:/tmp/data/", layer="sids ) OGR data source with driver: ESRI Shapefile Source: "C:/tmp/data/", layer: "sids" with 100 features and 18 fields Feature type: wkbpolygon with 2 dimensions > class(sids) [1] "SpatialPolygonsDataFrame" attr(,"package") [1] "sp" 7

8 Projection of the Shapefile If the shapefile has no.prj file associated with it, you need to assign a coordinate system: > proj4string(sids)<-crs("+proj=longlat ellps=wgs84") And then we can transform the map into any projection: > sids_nad <- sptransform(sids, CRS("+init=epsg:3358")) > sids_sp <- sptransform(sids, CRS("+init=ESRI:102719")) For a list of applicable CRS codes: Easiest with the EPSG and ESRI pre defined codes 8

9 Contiguity Based Neighbors Areas sharing any boundary point ( queen s ) are taken as neighbors, using the poly2nb function, which acceptsa a SpatialPolygonsDataFrame > library(spdep) > sids_nbq<-poly2nb(sids) If contiguity is defined as areas sharing more than one boundary point ( rook s ), the queen= argument is set to FALSE > sids_nbr<-poly2nb(sids, queen=false) > coords<-coordinates(sids) > plot(sids) > plot(sids_nbq, coords, add=t) Queen s Rook s 9

10 Distance Based Neighbors Using K Nearest Neighbors (KNN) > coords <- coordinates(sids_sp) > IDs <- row.names(as(sids_sp, "data.frame")) > sids_kn1<-knn2nb(knearneigh(coords, k=1), row.names=ids) > sids_kn2<-knn2nb(knearneigh(coords, k=2), row.names=ids) > sids_kn4<-knn2nb(knearneigh(coords, k=4), row.names=ids) > plot(sids_sp) > plot(sids_kn2, coords, add=t) k=2 k=3 k=1 k=1 k=2 k=4 10

11 Distance Based Neighbors We might also assign neighbors based on a specified distance: > dist <- unlist(nbdists(sids_kn1, coords)) > summary(dist) Min. 1st Qu. Median Mean 3rd Qu. Max > max_k1 <- max(dist) > sids_kd1<-dnearneigh(coords, d1=0, d2 = 0.75*max_k1, row.names=ids) > sids_kd2<-dnearneigh(coords, d1=0, d2 = 1*max_k1, row.names=ids) > sids_kd3<-dnearneigh(coords, dnearneigh(coords d1=0, d2 = 1.5*max_k1, row.names=ids) dist = 0.75*max_k1 dist = 1*max_k1 = dist = 1.5 * max_k1 11

12 Creating spatial weights matrices using neighborhood lists STEP 2: ASSIGN WEIGHTS TO THE AREAS THAT ARE LINKED Spatial weights matrices Once we define which observations are neighbors, we assign weights iht to each link Can be binary or variable Even when the values are binary (0/1), the issue ofwhat to do with no neighborneighbor observations arises 12

13 Row standardized Weights Matrix > sids_nbq_w<- nb2listw(sids_nbq, style="w") > sids_nbq_w Characteristics of weights list: Neighbour list object: Number of regions: 100 Number of nonzero links: 490 Percentage nonzero weights: 4.9 Average number of links: 4.9 Weights style: W Weights constants summary: n nn S0 S1 S2 W Row standardization is used to create proportional weights when features have unequal numbers of neighbors Divide each neighbor weight for a feature by the sum of all neighbor weights Obs i has 3 neighbors, each has a weight of 1/3 Obs j has 2 neighbors, each has a weight of 1/2 Must be used if you want comparable spatial parameters across different data sets with different connectivity structures > sids_nbq_wb <- nb2listw(sids_nbq, style="b") > sids_nbq_wb Characteristics ti of weights list: Neighbour list object: Number of regions: 100 Number of nonzero links: 490 Percentage nonzero weights: 4.9 Average number of links: 4.9 Binary weights Row standardised weights increase the influence of links from observations with few neighbours Weights style: B Weights constants summary: n nn S0 S1 S2 B Binary weights vary the influence ceof observations o s Those with many neighbours are upweighted compared to those with few 13

14 Binary vs. Row Standardized A binary weights matrix looks like: A1 A2 A3 A4 A A A A A row standardized matrix it looks like: A1 A2 A3 A4 A A A A Regions With No Neighbors Error in nb2listw(filename): Empty neighbor sets found You have some regions that have no neighbors Could be: A problem in the GIS topology (digitizing errors, slivers, etc) Induced by hard coded distance thresholds, and are true islands (e.g., Hawaii) May want to use k nearest neighbors Or add zero.policy=t to the nb2listw call > sids_nbq_w<-nb2listw(sids_nbq, zero.policy=t) 14

15 How to use the spatial weights, assess degree of spatial autocorrelation STEP 3: EXAMINE SPATIAL AUTOCORRELATION Spatial Autocorrelation Test for the presence of spatial autocorrelation ti Global Moran s I Geary s C Local (LISA Local Indicators of Spatial Autocorrelation) Local Moran s I and Getis G i * 15

16 Moran s I in R > moran.test(sids_nad$sidr79, listw=sids_nbq_w, alternative= two.sided ) two.sided H A : I I 0 greater H A : I > I 0 Moran's I test under randomisation data: sids_nad$sidr79 weights: sids_nbq_w Moran I statistic standard deviate = , p-value = alternative hypothesis: greater sample estimates: Moran I statistic Expectation Variance Moran s I in R > moran.test(sids_nad$sidr79, listw=sids_nbq_wb) Moran's I test under randomisation data: sids_nad$sidr79 weights: sids_nbq_wb Moran I statistic standard deviate = , p-value = alternative hypothesis: greater sample estimates: Moran I statistic Expectation Variance

17 Spatial Regression And how do we do it in R? Spatial Autocorrelation in Residuals: Spatial Error Model Incorporates spatial effects through error term Where: ε is λ is the spatial error coefficient y = xβ + ε ε = λwε + ξ the vector of error terms, spatially weighted using the weights matrix (W) ξ is a vector of uncorrelated error terms If there is no spatial correlation between the errors, then λ = 0 17

18 Spatial Autocorrelation in Obs. Values: Spatial Lag Model Incorporates spatial effects by including a spatially lagged dependent variable as an additional predictor Where: Wy is ε is a vector of error terms y = ρ Wy + xβ + ε the spatially lagged DVs for weights matrix W x is a matrix of observations on the explanatory variables ρ is the spatial coefficient If there is no spatial dependence, and y does no depend on neighboring y values, ρ = 0 Good books Bivand, R., et al. (2007) Applied Spatial Data Analysis with R. New York: Springer. Ward, M.D. and K.S. Gleditsch (2008) Spatial Regression Models. Thousand Oaks, CA: Sage. 18

19 CRIM Spatial Regression in R Example: Housing Prices in Boston per capita crime rate by town ZN proportion of residential land zoned for lots over 25,000 ft 2 INDUS proportion of non retail business acres per town CHAS Charles River dummy variable (=1 if tract bounds river; 0 otherwise) NOX Nitrogen oxide concentration (parts per 10 million) RM average number of rooms per dwelling AGE proportion of owner occupied units built prior to 1940 DIS weighted distances to five Boston employment centres RAD index of accessibility to radial highways TAX full value property tax rate per $10,000 PTRATIO pupil teacher ratio by town B 1000(Bk 0.63) 2 where Bk is the proportion of blacks by town LSTAT % lower status of the population MEDV Median value of owner occupied homes in $1000's Spatial Regression in R 1. Read in a shapefile (boston.shp) 2. Define neighbors (k nearest w/point data) 3. Create weights matrix 4. Moran s test of observed, Moran scatterplot 5. Run OLS regression 6. Check residuals for spatial dependence 7. Determine which SR model to use w/lm tests 8. Run spatial regression model 19

20 Define Neighbors and Create Weights Matrix > boston<-readogr(dsn= C:/tmp/data/",layer="boston") > class(boston) > boston$logmedv<-log(boston$cmedv) > coords<-coordinates(boston) > IDs<-row.names(as(boston, "data.frame")) > bost_ kd1<-dnearneigh(coords, d e ds, d1=0, d2=3.973, row.names=ids) s) > plot(boston) > plot(bost_kd1, coords, add=t) > bost_kd1_w<- nb2listw(bost_kd1) Moran s I on the Observed Median House Values > moran.test(boston$logmedv, listw=bost_kd1_w) Moran's I test under randomisation data: boston$logmedv weights: bost_kd1_w Moran I statistic standard deviate = , p-value < 2.2e-16 alternative hypothesis: greater sample estimates: Moran I statistic Expectation Variance

21 Moran Plot for the DV > moran.plot(boston$logmedv, bost_kd1_w, labels=as.character(boston$id)) OLS Regression bostlm<-lm(logmedv~rm + LSTAT + CRIM + ZN + CHAS + DIS, data=boston) Residuals: Min 1Q Median 3Q Max Coefficients: Estimate Std. Error t value Pr(> t ) (Intercept) < 2e-16 *** RM e-11 *** LSTAT < 2e-16 *** CRIM < 2e-16 *** ZN *** CHAS *** DIS e-06 *** --- Residual standard error: on 499 degrees of freedom Multiple R-squared: , Adjusted R-squared: F-statistic: on 6 and 499 DF, p-value: < 2.2e-16 21

22 Checking residuals for spatial autocorrelation > boston$lmresid<-residuals(bostlm) > lm.morantest(bostlm, bost_kd1_w) Global Moran's I for regression residuals Moran I statistic standard deviate = , p-value = 2.396e-09 alternative hypothesis: greater sample estimates: Observed Moran's I Expectation Variance Determining the type of dependence > lm.lmtests(bostlm, bost_kd1_w, test="all") Lagrange multiplier diagnostics for spatial dependence LMerr = , df = 1, p-value = 3.201e-07 LMlag = , df = 1, p-value = 8.175e-12 RLMerr = , df = 1, p-value = RLMlag = , df = 1, p-value = 4.096e-07 SARMA = , df = 2, p-value = 5.723e-12 Robust tests used to find a proper alternative Only use robust forms when BOTH LMErr and LMLag are significant 22

23 One more diagnostic > install.packages( lmtest ) > library(lmtest) > bptest(bostlm) studentized Breusch-Pagan test data: bostlm BP = , df = 6, p-value = 2.651e-13 Indicates errors are heteroskedastic Not surprising since we have spatial dependence Running a spatial lag model > bostlag<-lagsarlm(logmedv~rm + LSTAT + CRIM + ZN + CHAS + DIS, data=boston, bost_kd1_w) Type: lag Coefficients: (asymptotic standard errors) Estimate Std. Error z value Pr(> z ) ) (Intercept) < 2.2e-16 RM e-10 LSTAT < 2.2e-16 CRIM < 2.2e-16 ZN CHAS DIS e-11 Rho: , LR test value:37.426, p-value:9.4936e-10 Asymptotic standard error: z-value: , p-value: e-11 Wald statistic: , p-value: e-11 Log likelihood: for lag model ML residual variance (sigma squared): , (sigma: ) AIC: , (AIC for lm: ) 23

24 A few more diagnostics LM test for residual autocorrelation test value: , p-value: > bptest.sarlm(bostlag) studentized Breusch-Pagan test data: BP = , df = 6, p-value = 4.451e-11 LM test suggests there is no more spatial autocorrelation in the data BP test indicates remaining heteroskedasticity in the residuals Most likely due to misspecification Running a spatial error model > bosterr<-errorsarlm(logmedv~rm + LSTAT + CRIM + ZN + CHAS + DIS, data=boston, listw=bost_kd1_w) Type: error Coefficients: (asymptotic standard errors) Estimate Std. Error z value Pr(> z ) ) (Intercept) < 2.2e-16 RM e-09 LSTAT < 2.2e-16 CRIM < 2.2e-16 ZN CHAS DIS Lambda: , LR test value: , p-value: e-07 Asymptotic standard error: z-value: , p-value: e-12 Wald statistic: 46.35, p-value: e-12 Log likelihood: for error model ML residual variance (sigma squared): , (sigma: ) AIC: , (AIC for lm: ) 24