Exploratory Spatial Data Analysis (And Navigating GeoDa)

Transcription

1 Exploratory Spatial Data Analysis (And Navigating GeoDa) June 9, 2006 Stephen A. Matthews Associate Professor of Sociology & Anthropology, Geography and Demography Director of the Geographic Information Analysis Core Population Research Institute Stephen A. Matthews GISPopSci Friday June Slide 01

2 Outline 1. Exploratory Spatial Data Analysis 2. GeoDa Navigating GeoDa Stephen A. Matthews GISPopSci - Friday June Slide 02

3 What is Exploratory Spatial Data Analysis? "ESDA is a collection of techniques to describe and visualize spatial distributions, identify atypical locations or spatial outliers, discover patterns of spatial association, clusters or hot-spots, and suggest spatial regimes or other forms of spatial heterogeneity. Central to this conceptualization is the notion of spatial autocorrelation or spatial association, i.e., the phenomenon where locational similarity (observations in spatial proximity) is matched by value similarity (attribute correlation)" (Anselin, 1998, p ). Stephen A. Matthews GISPopSci - Friday June Slide 03

4 Exploratory Data Analysis (EDA) Graphical and visual methods/tools that are used to identify data properties for purposes of - pattern detection in data - hypothesis formulation from the data - aspects of models assessment (e.g., goodness-of-fit) EDA emphasize the interaction between human cognition and computation in the form of dynamic statistical graphics that allow the user to manipulate "views" of the data (via box-plots, scatterplots, etc). Stephen A. Matthews GISPopSci - Friday June Slide 04

5 Exploratory Spatial Data Analysis (ESDA) Graphical and is an extension of EDA to detect spatial properties of data. There is a need for additional techniques to - detect spatial patterns in data - formulate hypotheses based on the geography of the data - assessing spatial models In many instances it is important to be able to link numerical and graphical procedures with a map to answer questions such as "Where are those cases?" Stephen A. Matthews GISPopSci - Friday June Slide 05

6 Exploratory Spatial Data Analysis (ESDA) With modern graphical interfaces this is often done by "brushing" - where cases are selected from the relevant areas of a box-plot or scatterplot, and the related regions are identified on a map. First example of ESDA by Mark Monmonier (1989) Interactive Spatial Data Analysis Stephen A. Matthews GISPopSci - Friday June Slide 06

7 Exploratory Spatial Data Analysis (ESDA) These geographic brushing tools are now found in some GIS packages where graphical and tabular displays are dynamically linked, such that the selection of any subset of observations in a map or other data view is immediately reflected in all other views of displays. In addition to choropleth maps, other displays/views include histograms, box-plots, scatterplots (see GeoDa). True ESDA pays attention to both spatial and attribute association (Anselin, 1998, p.79). Stephen A. Matthews GISPopSci - Friday June Slide 07

8 ESDA and GIS The integration of ESDA and GIS can be traced back to Michael Goodchild's 1987 article "A spatial analytical perspective on geographical information systems" in the International Journal of Geographic Information Systems. Integration of EDSA (within statistical packages) and GIS took on various forms. Goodchild et al (1992) categorize the integrations as being either loose coupling or close coupling. Stephen A. Matthews GISPopSci - Friday June Slide 08

9 Stephen Matthews Spatial Demography Course Fall 2001 Spatial Analysis options in Arcview and Coupling Arcview with Spatial Analysis software packages Later in the course I will use/introduce: Arcview Geoprocessing Wizard (Week 6) Arcviw and DynESDA (Week 7) Spatial Analyst Extension (Week 8) Arcview and CrimeStat (Week 9) Arcview and S-plus / SpatialStats (Week 10) Arcview and SpaceStat (Week 11) Arcview interface options to SpaceStat and S+ S-Plus (Week 10) SpaceStat (Week 11) S-plus Spatial Data SpaceStat Statistics Stephen A. Matthews GISPopSci - Friday June Slide 09

10 Stephen Matthews Spatial Demography Course Fall 2001 Coupling GIS and Spatial Statistics ESDA & GIS: Loose or Close Coupling With loose coupling data and commands are passed back and forth between the packages by means of auxiliary files. With close coupling the commands in one software system are called from another system by means of a seamless inter-process communication. Close coupling is a recent development and is exemplified by the S+GISlink between ArcInfo and S-plus. In both cases however these integrations consist of adding statistical capabilities to a GIS and rarely are focused on spatial data analysis methods. Stephen A. Matthews GISPopSci - Friday June Slide 10

11 Stephen Matthews Spatial Demography Course Fall 2001 ESDA & GIS: Encompassing Coupling With ESDA and GIS another form of integration emerges, encompassing coupling (see Anselin and Getis, 1992). Encompassing consists of writing spatial data analysis routines into a system's macro or scripting language (e.g., Avenue or VBA for Arcview). Such an approach is fully integrated within the GIS interface and hides the linked nature of the spatial data routines from the user. For example, see Ding and Fotheringham 1992, Bao et at 1995 and Zhang and Griffith Anselin (1998b p. 84) comments that these implementations are computationally slow and limited in terms of the size of data set that can be used (for more on this see Anselin and Boa, 1997). David Wong s SEG ESDA & GIS: Modular Coupling Anselin identifies a hybrid model or modular coupling approach which consists of including spatial data analysis software - typically specially developed routines - in a collection of linked systems where the communication between the different systems is established by means of a combination of loose and close couplings. Examples: ArcInfo, SpaceStat, Xgobi and clustering software Zhang et al Arcview, Xgobi and XploRe - Symanzik et al ArcInfo and spatial statistics routines - Haining et al, 1996 Arcview and SpaceStat - Anselin and Bao, 1997 Luc Anselin s SpaceStat Stephen A. Matthews GISPopSci - Friday June Slide 11

12 S-plus/SpatialStats samples Local Indicators of Spatial Association Testing the hypothesis of no autocorrelation using the global Moran statistic assumes stationarity of variance through space. Using local indicators of spatial association (LISA) is not limited by this assumption. The Local Moran is applied to each individual point/area with the index revealing clustering or dispersion relative to the local neighborhood. Local Indicators of Spatial Association The local Moran s I measures and z- scores are now automatically attached to the.shp file and can now be visualized in the Arcview-View window. Geographic Information Analysis Core Population Research Institute Geographic Information Analysis Core Population Research Institute Local Indicators of Spatial Association LISA - local Moran map LISA results will be saved Results are automatically joined to.shp file Geographic Information Analysis Core Population Research Institute Geographic Information Analysis Core Population Research Institute Stephen A. Matthews GISPopSci - Friday June Slide 12

13 S-plus/SpatialStats samples Spatial Regression Models To estimate a spatial regression model choose Spatial Linear Regression from the Arcview-Spatial Statistics menu. Spatial Regression Models - output #2 The model output contains Moran s index of spatial association for the model residuals it appears that the spatially lagged model accounts for most of the spatial autocorrelation in the data since the residual spatial autocorrelation is not significant. Geographic Information Analysis Core Population Research Institute Geographic Information Analysis Core Population Research Institute Spatial Regression Models Spatial Regression Model 2 - spatial residual map Select the dependent variable and the independent variables. Spatial relations are specified. Automatic joins Various model types Display results Geographic Information Analysis Core Population Research Institute Geographic Information Analysis Core Population Research Institute Stephen A. Matthews GISPopSci - Friday June Slide 13

14 SpaceStat samples Arcview interface options to SpaceStat Stephen A. Matthews GISPopSci - Friday June Slide 14

15 SpaceStat samples Regress Module (spatial regression analysis) x x Stephen A. Matthews GISPopSci - Friday June Slide 15

16 GeoVista Studio (Penn State) ESTAT ESTAT Lab on Monday June 12, 2006 Stephen A. Matthews GISPopSci - Friday June Slide 16

17 GeoDa can be found at the CSISS site: Under Spatial Tools Stephen A. Matthews GISPopSci - Friday June Slide 17

18 GeoDa Webpage: Stephen A. Matthews GISPopSci - Friday June Slide 18

19 GeoDa Datasets \GeoDa\Sample Data Baltimore Columbus National SIDS (North Carolina) South St. Louis Stephen A. Matthews GISPopSci - Friday June Slide 19

20 GeoDa Datasets Columbus Variable Codebook HTML files Stephen A. Matthews GISPopSci - Friday June Slide 20

21 GeoDa Datasets National Consortium on Violence Research Codebook HTML files See paper by Baller et al (2001) Criminology 39, Stephen A. Matthews GISPopSci - Friday June Slide 21

22 GeoDa Basics and Some Tips Stephen A. Matthews GISPopSci - Friday June Slide 22

23 GeoDa: Data Formats To work with GeoDa, your data have to have the following characteristics: 1. Continuously (as opposed to categorically) distributed 2. Contain no missing values 3. Refer to discrete areal units. 4. Contain a unique ID Stephen A. Matthews GISPopSci - Friday June Slide 23

24 GeoDa: No Missing Data GeoDa cannot deal with missing values - it will fill blank fields with zeros or treat values such as 99, -1, etc. as observed. There is no easy solution to this problem. Some options include excluding missing observations, re-saving your shape file for only those areas without missing values or interpolating missing values. (Note: care needs to be taken that this interpolation is not based on the values of immediate neighbors, otherwise spatial autocorrelation is introduced by design). Stephen A. Matthews GISPopSci - Friday June Slide 24

25 Choice of a weights matrix The choice of your weights matrix is mainly a substantive rather than a technical choice, i.e., it depends on your theory/definition of who your relevant neighbors are. For instance, if you are exploring a phenomenon where you expect the spatial structure to be concentrated around particular locations rather than dispersed, you would want to choose a matrix that only defines your immediately bordering locations as neighbors, and vice versa. In choosing a weights matrix you want to ask which neighbors' values should be averaged in comparison to a particular location. Stephen A. Matthews GISPopSci - Friday June Slide 25

26 Choice of Weights Matrix If you do not have much prior theoretical grounds to go by, you can also create different weights matrices and explore how sensitive your outputs are to the differences in matrices. By linking and brushing the weights characteristics with choropleth maps, you can get a better sense of what weights matrices capture what neighborhood structure and which might be most appropriate for your purposes. Stephen A. Matthews GISPopSci - Friday June Slide 26

27 Choice of Weights Matrix Stephen A. Matthews GISPopSci - Friday June Slide 27

28 Choice of Weights Matrix Also, before embarking on the computation of spatial autocorrelation statistics, it is a good idea to check the spatial weights matrix for the presence of islands (unconnected observations) and other undesirable characteristics (see TOOLS>WEIGHTS>PROPERTIES) Stephen A. Matthews GISPopSci - Friday June Slide 28

29 Islands The problem with running spatial regressions with data islands is that by definition it does not make sense to assess a spatial relationship between units that are spatially isolated. If the isolated area is actually connected to other areas in a substantive sense (e.g., through transportation, trade, etc.), you can assign it as a neighbor to other areas by editing your contiguity weights matrix or by using distance weights. Stephen A. Matthews GISPopSci - Friday June Slide 29

30 GeoDa uses a Row-Standardized Weights Matrix The results in GeoDa's global and local measures of spatial autocorrelation, its spatial lag, and its spatial regressions are based on row-standardized weights. A weights matrix is row-standardized when the values of each of its rows sum to one. By convention, the location at the center of its neighbors is not included in the definition of neighbors and is therefore set to 0. Stephen A. Matthews GISPopSci - Friday June Slide 30

31 Consider this lattice (example courtesy of Paul Voss, Wisconsin) Stephen A. Matthews GISPopSci - Friday June Slide 31

32 j GISPopSci Workshop Penn State 2006 June 2006 Stephen A. Matthews GISPopSci - Friday June Slide 32

33 j GISPopSci Workshop Penn State 2006 June 2006 Stephen A. Matthews GISPopSci - Friday June Slide 33

34 Consider these y i values, i = 1,, Stephen A. Matthews GISPopSci - Friday June Slide 34

35 For i = 6, the spatial lag Wy i is given by: Wy = w y i ij j j 1 = = 4.9 Stephen A. Matthews GISPopSci - Friday June Slide 35

36 Distance based definition of the contiguity matrix K Nearest Neighbors (KNN) is a distance-based definition of neighbors where "k" refers to the number of neighbors of a location. It is computed as the distance between a point and the number (k) of nearest neighbor points (i.e. the distance between the central points of polygons). It is often applied when areas (counties) have different sizes to ensure that every location has the same number of neighbors, independently how large the neighboring areas are. Stephen A. Matthews GISPopSci - Friday June Slide 36

37 Distance based definition of the contiguity matrix K Nearest Neighbors weights matrices can be created in GeoDa. Note: These KNN weights matrices are asymmetric (e.g., point A is B's nearest neighbor but point B does not have to be point A's nearest neighbor). Because of this asymmetry, it is currently not possible to correctly estimate spatial lag or error models with KNN weights Stephen A. Matthews GISPopSci - Friday June Slide 37

38 Constructing Spatially Lagged Variables (see An introduction to Spatial Autocorrelation Analysis with GeoDa, Luc Anselin, 2003 pp.5-7) You can add spatial lags for any variable in your data set using the Table Calculation options. With a table active, right click and select Add Column (or use the Options menu). In the dialog, specify a meaningful name for a spatial lag variable (conventionally a W is placed in front of a variable name to indicate it is a spatially lagged variable). Note that before you can compute the lag, you must create an empty column in the table to contain it. Also, you must make sure a spatial weights file has been opened (click on the Open Weights toolbar button and specify the file name). Stephen A. Matthews GISPopSci - Friday June Slide 38

39 Constructing Spatially Lagged Variables In the table, right click and select Field Calculation. You will need to click on the third tab for Lag Operations. Stephen A. Matthews GISPopSci - Friday June Slide 39

40 Constructing Spatially Lagged Variables Next, select new W variable as the Result in the drop down list, make sure the correct weights file is specified and choose original Variable. Stephen A. Matthews GISPopSci - Friday June Slide 40

41 Constructing Spatially Lagged Variables Click on OK to create the new variable. Its values will be added in the new column. To permanently add the new field to the table, right-click on the table and go to Save Shape File As... Stephen A. Matthews GISPopSci - Friday June Slide 41

42 Adding Centroids (central points) to Tables Rightclick on map Stephen A. Matthews GISPopSci - Friday June Slide 42

43 Adding Centroids (central points) to Tables Stephen A. Matthews GISPopSci - Friday June Slide 43

44 Adding Centroids (central points) to Tables Stephen A. Matthews GISPopSci - Friday June Slide 44

45 Explore > Conditional Plots Conditional plots (Trellis graphs) are 3x3 micromap (or microplot) matrices. They visualize multivariate relationships (three or four variables in two dimensions). They consist of nine smaller plots or maps of one continuous variable (two for the scatter plot option), conditioned on two other variables. The interval breaks of the two other variables can be controlled in GeoDa. Stephen A. Matthews GISPopSci - Friday June Slide 45

46 Explore > Conditional Plots This multivariate analysis might reveal interaction effects masked by univariate exploratory analysis (interaction effects exist when the distribution in a sub-view differs from the rest). By manipulating the handles in the conditional plots, you can analyze the sensitivity of the main variable to the conditioning variables. Stephen A. Matthews GISPopSci - Friday June Slide 46

47 Explore > Conditional Plots Map View Stephen A. Matthews GISPopSci - Friday June Slide 47

48 Explore > Conditional Plots > Map View Stephen A. Matthews GISPopSci - Friday June Slide 48

49 Explore > Conditional Plots Scatter Plot Stephen A. Matthews GISPopSci - Friday June Slide 49

50 Explore > Conditional Plots > Scatter Plots Stephen A. Matthews GISPopSci - Friday June Slide 50

51 Explore > Conditional Plots > Scatter Plots Stephen A. Matthews GISPopSci - Friday June Slide 51

52 Explore > Conditional Plots > Scatter Plots Stephen A. Matthews GISPopSci - Friday June Slide 52

53 Box Plots Unemployment rate 1990 (NAT) Box plots are particularly useful to identify outliers and gain an overview of the spread of a distribution. The box plot (sometimes referred to as box and whisker plot) is a non-parametric method. For normally distributed data, the median corresponds to the mean and the inter-quartile range to the standard deviation. The box plot shows the median, first and third quartile of a distribution (the 50%, 25% and 75% points in the cumulative distribution) as well as outliers. Stephen A. Matthews GISPopSci - Friday June Slide 53

54 Box Plots Unemployment rate 1990 (NAT) An observation is classified as an outlier when it lies more than a given multiple of the inter-quartile range (the difference in value between the 75% and 25% observation) above or below respectively the value for the 75th percentile and 25th percentile. The standard multiples used are 1.5 and 3 times the inter-quartile range. The hinge, corresponding to the default criterion for the hinge is 1.5. Observations are shown as blue dots in 1 st and 4 th Quartiles IQR Hinge Median Stephen A. Matthews GISPopSci - Friday June Slide 54

55 Unemployment rate 1990 (NAT) outliers highest values (hinge = 1.5) Stephen A. Matthews GISPopSci - Friday June Slide 55

56 Unemployment rate 1990 (NAT) observations in 4 th Quartile (i.e., highest values) Stephen A. Matthews GISPopSci - Friday June Slide 56

57 Standard Deviational Map A standard deviational map highlights differences in standardized values from the mean. GeoDa's standard deviational map displays the data in 7 categories: The mean, and three standard deviational units above and below the mean. The standard deviational map is the parametric counterpart to the box map. Stephen A. Matthews GISPopSci - Friday June Slide 57

58 Bivariate Moran Scatter Plot In a bivariate Moran scatter plot, y and x are different variables. The neighboring values of one variable (y) are regressed on the values of another variable (x). In the bivariate scatter plot the standardized version of one variable (y) can be regressed on the lag of another variable (x). Stephen A. Matthews GISPopSci - Friday June Slide 58

59 Geoda Exercise Nepal Handout Page 32. Stephen A. Matthews GISPopSci - Friday June Slide 59

60 LISA Local Indicators of Spatial Association (LISA) indicate the presence or absence of significant spatial clusters or outliers for each location. A randomization approach is used to generate a spatially random reference distribution to assess statistical significance. The Local Moran statistic implemented in GeoDa is a special case of a LISA. The average of the Local Moran statistics is proportional to the Global Moran's I value. Stephen A. Matthews GISPopSci - Friday June Slide 60

61 LISA LISA maps are particularly useful to assess the hypothesis of spatial randomness and to identify local hot spots. However, since LISA maps are univariate, they may mask multivariate associations, variability related to scale mismatch, and other spatial heterogeneity. Stephen A. Matthews GISPopSci - Friday June Slide 61

62 Saving Lisa Results Stephen A. Matthews GISPopSci - Friday June Slide 62

63 Saving Lisa Results You can add the results of the LISA analysis to a table, by invoking Option > Save Results or right click on the map (select Save Results). The three options are: a) the Local Moran statistic for each location (LISA indices) b) the indicator for the type of spatial autocorrelation (Clusters) c) the significance or p-value for the Local Moran statistic. Stephen A. Matthews GISPopSci - Friday June Slide 63

64 Saving Lisa Results You can add the results of the LISA analysis to a table, by invoking Option > Save Results or right click on the map (select Save Results). The three options are: a) the Local Moran statistic for each location (LISA indices) b) the indicator for the type of spatial autocorrelation (Clusters) c) the significance or p-value for the Local Moran statistic. Stephen A. Matthews GISPopSci - Friday June Slide 64

65 Saving Lisa Results The cluster indicators take on five values: 0 for not significant, 1 for high-high, 2 for low-low, 3 for high-low and 4 for low-high. As before, these additions do not become permanent until the table has been saved. Stephen A. Matthews GISPopSci - Friday June Slide 65

66 This afternoon Spatial Regression Modeling using GeoDa Stephen A. Matthews GISPopSci - Friday June Slide 66

67 Stephen A. Matthews GISPopSci - Friday June The End