ENGRG 59910: Introduction to GIS

Transcription

1 ENGRG 59910: Introduction to GIS Lecture 08: Geocoding and Vector Analysis Acknowledgment: Michael Piasecki

2 Today Agenda Geocoding Understanding of Spatial Analysis Attribute-based Operation Statistical description Map Statistical description Reclassification and Classification Spatial-based Operation Selection Set algebra Boolean algebra Spatial selection GeoProcessing in ArcGIS Dissolve Merge/Append Clip Erase Intersect Union Update

3 Geocoding overview

4 Geocoding overview Process of creating geometric representations for locations (such as points) from descriptions of locations (such as street addresses) Uses a computer program called a geocoding engine that employs code tables and rules to standardize address components

5 5 Examples City s economic development department Maps technology businesses by street address to determine technology-rich areas in a city Hospital Maps patients to determine where to open a satellite clinic Retail store chain Maps store and customer locations, and compares to mapped competitor locations Others?

6 Tabular data Text file or database Street addresses ZIP Codes

7 Geocoding reference layers Street centerlines ZIP Code polygons

8 ZIP Code geocoding Method to map data whose geocode is for a polygon Assign each record to its polygon Count the records for each polygon Join the table to the corresponding polygon layer Symbolize using a choropleth map or graduated point symbols

9 ZIP Code geocoding

10 ZIP Code geocoding Points created at ZIP Code centroids

11 ZIP Code geocoding Points (attendees) spatially joined to ZIP Code polygons

12 ZIP Code geocoding Choropleth map created

13 Linear (street) geocoding

14 Linear geocoding (streets) TIGER (Census Bureau) street maps Four street address numbers, low to high for each side of a street segment 100 Oak Street

15 Address components Number 125 Oak St E, Apt. 2, Pittsburgh, PA Street name 125 Oak St E, Apt. 2, Pittsburgh, PA Street type 125 Oak St E, Apt. 2, Pittsburgh, PA Direction, suffix 125 Oak St E, Apt. 2, Pittsburgh, PA Direction, prefix 125 E Oak St, Apt. 2, Pittsburgh, PA Unit number 125 Oak St E, Apt. 2, Pittsburgh, PA Zone, city 125 Oak St E, Apt. 2, Pittsburgh, PA Zone, ZIP Code 125 Oak St E, Apt. 2, Pittsburgh, PA Items for single-number street address: Address Unit City ZIP Code 125 Oak St E Apt. 2 Pittsburgh 15213

16 Street Intersections Put street intersections in address field Forbes AV & Craig ST Grant ST & 5 th AV E North Star RD & Duncan AV Do not include street numbers 3999 Forbes Ave & 100 Craig ST Connectors Any unusual character (e.g., ) Just be consistent

17 Geocoding Flowchart Input Address Parse Address Yes Score Matches Matches? No Output No match Generate Soundex Key Best match >= 90? No Find Candidates: No Range & Soundex Key Yes Output Address

18 Geocoding steps Original address: 125 East Oak Street Address parsed: 125 East Oak Street Abbreviations standardized: 125 E Oak St Elements assigned to match keys: [HN]:125 [SN]:Oak[ST]:St [SD]:E [ZP]:15213 Index values calculated: [HN]:125 [SN]:Oak(Soundex #) [ST]:St [SD]:E [ZP]:15213 (Index #)

19 Soundex index Matches names based on how they sound (if indices match) Translates names to a 4-digit index of 1 letter and 3 numbers First character of name remains unchanged Adjacent letters in the name which have the same Soundex key are assigned a single digit If the end of the name is reached before filling 3 digits, use zeros to complete the code Key Letters 1 b f p v 2 c g j k q s x z 3 d t 4 l 5 m n 6 r disregard a e h i o u y w Oake = O-200, Oak = O-200 Smith = S-530, Smythe = S-530 Paine = P-500, Payne = P-500 Callahan = C-450, Calahan = C-450 Beadles = B-342, Beattles = B-342 Schultz = S-243, Shults = S

20 Scoring candidates Use a rule base to score source and reference matches Start with score of 100 Subtract points for each mismatch Examples from rule base Soundex indices match but street names do not (-2) Street type missing in source (-1) Street types do not match (-2)

21 Candidate streets Candidates identified: 125 East Oak Street From To Street Type Side Parity Direction Street_ 2 98 Oak St R E W Oak St L O W Oak St R E E Oak St L O E 4357 Candidates scored and filtered: From To Street Type Side Parity Direction Street_ Oak St R E E Oak St L O E 4357 Parity is used to locate a geocoded address on the correct side of the street (such as odd numbers on the left-hand side, even numbers on the right).

22 Address matched as point Best candidate matched From To Street Type Side Parity Direction Street_ Oak St L O E 4357 Oak St Pine Ave

23 Problems and solutions

24 Possible problems Variations in street names Fifth Avenue, Fifth Ave., 5th AV Saw Mill Run Blvd, Route 51 Data entry errors Fidth Avenue Sawmill Run Place names White House, Heinz Field, Empire State Building Intersections Fifth Avenue and Craig Street

25 Possible problems Zones 100 Main ST 15101, 100 Main ST P.O. boxes P.O. Box 125 Missing street data

26 Solutions Clean data before geocoding Purchase or build high-quality maps (field verification) Use postal address standards Assign house numbers in rural areas Use alias tables Alias White House Heinz Field Empire State Building Address 1600 Pennsylvania Avenue 100 Art Rooney Avenue 350 5th Ave

27 Alias table Alias CMU Carnegie Mellon Carnegie Mellon U Carnegie Mellon Univ Carnegie Mellon University Etc. Address 5000 Forbes Av 5000 Forbes Av 5000 Forbes Av 5000 Forbes Av 5000 Forbes Av

28 Vector Analysis

29 Spatial Analysis in GIS From Data to Information: beyond mapping, added value Spatial Analysis is the Geographical tool that unveils patterns, connections, and possible causes of variations in the spatial data What is a pattern? Lower 48 United States 1996 Data from the U.S. Census on gender Gender Ratio = females per 100 males Range is What does the spatial distribution look like?

30 Ratio of the Total Pop: 2005, Male per Female

31 Spatial Analysis: successive levels of sophistication Spatial data manipulation Classical GIS capabilities Spatial queries & measurement, buffering, map layer overlay Spatial data analysis Descriptive and exploratory Visualization through data manipulation and mapping Spatial statistical analysis hypothesis testing Are data to be expected or are they unexpected relative to some statistical model, usually of a random process Spatial modeling Constructing models (of processes) to predict spatial outcomes (patterns) What if analyses

32 Spatial Analyses: a sequence of spatial ops

33 Spatial Analysis: In term of Data Access Attribute-based Operation Describe with statistics Analyze with statistical hypothesis testing Spatial-based Operation Describe with maps Analyze with spatial analysis GIS data description answers the question: Where is it? GIS data analysis answers the question: Why is it there? GIS data description is different from other statistics because the results are mapped for visual analysis.

34 Statistical Analysis Range: min, max and range, Central tendency: mean, median, mode Variation: variance, standard deviation

35 Statistical Analysis Function in ArcGIS Table Statistics

36 Statistical Analysis Function in ArcGIS Spatial Statistics

37 Statistical Analysis Function in ArcGIS Raster Statistic More advanced: GeoStatistical Analyst Extension

38 Classification and Reclassification Attribute operation in which the attribute values change without altering the geometry Original Attributes Vegetation Types of the Colombian Highlands Reclassified Attributes Vegetation Types of the Colombian Highlands N Vegetation Types Andean Agriculture Basal Agriculture Settlement (capital) Settlement (municipality) Andean forest Reservoir Lake Páramo N Natural Vegetation NO YES Source: IDEAM, Kilometers Source: IDEAM, Kilometers 3/24/2017 ENGRG Intro to GIS 38

39 Reclassification 3/24/2017 ENGRG Intro to GIS 39

40 Examples of Reclassification Social application: Crime rate low or high Ecological application: Environmental vulnerability Economical application: unemployed rate high or low Soil application: Reduce soil classes LC/LU application: Reduce LC/LU types Hydrology application: reduce stream orders Transportation application: show only trains on time vs delayed Infrastructure application: only pipes over 10 yrs old 3/24/2017 ENGRG Intro to GIS 40

41 Classification: know your data Natural Break: identify breakpoints between classes using a statistic formula (Jenk s optimization; minimize each class s ave. deviation from class mean, while maxi each class s deviation from means of other groups) Quantile: Each class contains the same number of features (easy understand but easy misleading) Equal Area: Total area of polygons in each class is approximately the same Equal Interval: Sub range = range/#of classes Standard Deviations: find mean value then place breaks at interval: ¼, ½, 1, >3std. Dev, <-3 std. Dev

42 Example: Equal Area Equal Interval Quantile Natural Break Std. Deviation

43 Choose classification method Classification method is determined by map application purpose In general, the classification method chosen should accomplish two things: maximize the between-class differences minimize the within-class differences

44 Guidelines of Classification methods From ESRI Virtual Campus: Learning ArcGIS 9

45 Geoprocessing in ArcGIS A process consists of a geoprocessing tool. A tool requires an input dataset and produces an output dataset using specified parameters.

46 Geoprocessing Toolbox in ArcGIS

47 Geoprocessing Toolbox in ArcGIS

48 GeoProcessing in ArcGIS

49 Spatial Analysis Methods (Vector) Selection Set algebra Boolean algebra Spatial selection GeoProcessing in ArcGIS Dissolve Merge Clip Intersect Union Update

50 Selection Set Algebra (<, >, = ) Used to identify groups such as extreme value (minimum, maximum) or specified value Boolean Algebra (AND, OR, NOT) A B A B A AND B A OR B A NOT B A B Population < 20,000

51 Boolean Algebra

52 Selection: Set Algebra

53 Construct a query Select all houses in NY and NJ whose value are between $400,000 and $500,000, located in low crime neighborhood, and their distance to CCNY is less than 10 miles or their distance to I-95 is less than 2 miles. Table fields: state_name,house_value, crime (low, high), dist2ccny, dist2i95 Write down your query: SELECT * FROM yourdatabase WHERE STATE_NAME = New York AND New Jersey AND HOUSE_VALUE (>= 400,000 AND <= 500,000) AND CRIME = low AND DIST2CCNY <= 10 OR DIST2I95 <= 2

54 Spatial Selection Operation Intersect Are within a distance of Completely contain Are completely within Have their center in Share a line segment with Touch the boundary of Are identical to Are Crossed by the outline of Contain Are contained by

55 Selecting sites within distances Using Select by Location Within a distance of one mile The selection examples are borrowed from Dr. David Cowen, Univ. of South Carolina

56 Select By Location Are Completely Within misses two

57 Interactive Selection - Allows you to add to or modify the current selection -

58 Need to Add Sites that Intersect to Select all sites

59 Have their Center Misses One

60 Are Contained by Misses Two

61 One Step Set Buffer Distance

62 Are Crossed by the outline of

63 Spatial Operations: GeoProcessing in ArcGIS Dissolve The opposite of map overlay (polygon-polygon) Separate map layers Merge Append together layers of adjoining areas Clip - Only Attributes of Input Layer Extracting an area of interest Cookie cutter approach Opposite of Merge Erase: Opposite of Clip Intersect - Attributes of both Union - Boolean or ; Split Polygon and Attributes of both

64 Dissolve Example 171 polygons Can Select any one Each polygon has Total population

65 After Dissolve One record Per County

66 Another Dissolve Example

67 Dissolve in ArcGIS

68 Merge/Append operation

69 Clip Creates New Layer, Attributes of input only

70 Clip example: Set study area

71 Clip in ArcGIS

72 Erase: Opposite of Clip

73 Erase in ArcGIS

74 Intersection Attributes of Both

75 Union Split the Site and include attributes of both Outside the County Inside the County

76 Example of Union

77 In Summary

78 Buffer Define spatial proximity One or more polygons of a prescribed extent around points, lines, or areas They can be easily calculated with raster and vector GIS Roads Buffered Buffers can define Areas of Influence of roads

79 Buffer Wizard

80 Buffer Wizard

81 Vector Buffers

82 More Buffers

83 Spatial Join: It is also called as spatial relate It is very similar to attribute join/related; rather than join with common fields, spatial join based on common locations Spatial join examples: Find the closest feature to another feature. Find what's inside a feature. Find what intersects a feature.

84 Join in ArcGIS Step 1 Step 2

85 Map Overlay One thematic layer is superimposed onto another thematic layer The new theme has the attributes of both the original themes, combined in specific ways. A B C + = D A-C B-C A-D B-D The output layer units are called Integrated Terrain Units Works better with raster data for mathematical overlays, but it is widely used in vector data.

86 Map Overlay: Vector 3 Basic Types Polygon-Polygon Point-Polygon Line-Polygon A B C D A B A B R A-C B-C A-DB-D New polygons are created inheriting the attributes of the parent layers 1-A 2-A 3-B Points inherit the attributes of the polygons where they fall R-A R-B Lines are broken into segments. Each segment inherits the attributes of the polygon in which it falls

87 Map Overlay: Raster Each cell can be easily superimposed onto other cells Map Algebra; Map Calculator; Image Calculator Because values are stored directly in each cell, arithmetic and statistical operations are performed directly on the data = 7

88 Sliver problem in vector overlay Common boundary example: Parcel Land use School zone Reasons: different data from different source with different time, scale, and coordinate system Problems: extra useless polygon and attribute records Overcome: Use highest coordinate accuracy boundary to replace others (preprocessing) Manually identify/remove slivers Defining snap distance during overlay; using Cluster tolerance Minimum mapping units (postprocessing)

89 Summary Understanding of Spatial Analysis: Level, Scope Classification: 5 methods Attribute-based Operation Selection: Set algebra, Boolean algebra Spatial Operations: Dissolve, Merge, Clip, Erase, Intersect, Union Proximity Functions and Buffering Overlay