GEOCODING SELF-REPORTED ADDRESSES: LESSONS LEARNED. Karyn Backus CT Department of Public Health

Transcription

1 GEOCODING SELF-REPORTED ADDRESSES: LESSONS LEARNED Karyn Backus CT Department of Public Health

2 GEOCODING ADDRESSES The process of assigning a location to an address Collect the address information Use a geocoder to assign a location to the address Use the locational information for analysis

3 SELF-REPORTED ADDRESSES ARE DIRTY Self-reported addresses are of varying quality Pre-processing is pivotal Prep the addresses for the geocoder (e.g., parse, clean, correct) Improve overall success Better match rates Reduce post-processing time More accurate results Pre-processing should be automated Cyclical process where post-processing findings can be implemented during pre-processing Birth and death certificates are legal documents Birth addresses: 99% match rate* Death addresses: 90%-96% match* * reformatted, cleaned, processed through an external software, idiosyncratic edits applied, and hand matched

4 STREET CENTERLINES AND INTERPOLATION Geocoder uses a database of street segments for reference Street centerlines or street reference dataset Segments are associated with geographic locations Actual road network broken into segments Number range / street name elements / town / state / zip code Alternate street names are supplied CT uses Tele Atlas for the streets database Tele Atlas (TA) segments are Census-based polygons (TIGER-line) CT uses official CT DEEP-based polygons Census <> DEEP misaligned boundary lines Segments break at Census-based town and tract boundaries Segments require interpolation for placement Point location is estimated along the segment line Longer segments may have lower accuracy (rural areas) May not represent building location Can be problematic at boundaries (town, state, shoreline) Roof-top the point is placed where the building is located Emphasizes the actual building location Databases are growing but not available for all areas

5 LESSON LEARNED: BOUNDARY DIFFERENCES

6 LESSON LEARNED: INTERPOLATION 39 Alison Ave

7 GEOCODER AKA ADDRESS LOCATOR Address locator = address geocoder in ArcMap Complex pattern matching program Compares input addresses to your reference database Uses a style to define the pattern Produces a score to represent the confidence in a match Many parameters are user-defined Options for alternate street and place names

8 USE THE USER-DEFINED PARAMETERS TO YOUR BENEFIT A 90% minimum match score = preferred sensitivity/specificity rate for our health data Side offset is distance from the center of the road Beware that in urban settings, an offset of 30+ feet can place the point in a parcel behind the intended parcel or over a town/census boundary End offset prevents clustering at intersections Do not match if candidates tie! ESRI noted that there was no logic for how the geocoder would choose one candidate over any other Output x/y coordinates isn t that why you are geocoding? Coordinates are in your current projection Can add coordinates at any time with Add XY Coordinates Convert to NAD83 if you want to add X/Y as Lat/Lon Alternate street names significantly improve match rates! Example: Berlin Tpke = Route 5 = North Broad St Alternates names are provided in the street reference dataset

9 USE COMPOSITE LOCATORS Composite locator will cycle through individual locators Addresses are matched using the style and settings of each locator Not possible to re-run the locator on just a portion of the input dataset Thus composite locator allows the user to define a hierarchy of locators More than one style (e.g., odd/even numbering, mixed numbering) More than one reference database (e.g., roof top, centerlines) More than one zone field (e.g., town name, town code, postal name) Different sets of parameters (e.g., change in spelling sensitivities)

10 INTERACTIVE MATCH IN 10.0 ROCKS Review results or edit addresses to adjust match status Subset the data file for selective review Hand-match only unmatched address Review matched addresses (e.g., less than 100%) Use a data variable to subset addresses (e.g., QAS edit Y/N) Use the candidates button to map candidates This is where the base maps become valuable!

11 INTERACTIVE MEANS INTERACTIVE Overlay relevant base maps to help decision process Street network, town boundaries, tract boundaries, block group boundaries, zip code-town overlap, water bodies Edit the input address (sensibly) to adjust the match score Misspellings or other text errors Null a town name or zip code (when using CT-DPH custom locator) Some ties cannot be reconciled Two street names with the same number range exist in the same town or zip code

12 EXAMPLE: INTERACTIVE MATCH

13 LESSON LEARNED: MAILING ADDRESSES STREET ADDRESSES Self-reported addresses = mailing addresses street addresses Mailing addresses are a function of the USPS Mailing address may not reflect street centerlines PO Boxes, rural routes, housing developments Postal name is reported One town but several zip codes Storrs Mansfield / Mansfield Center / Mansfield Depot One zip code but several towns Stafford Springs, Jewett City Mystic, Moosup, Winsted Postal zones can change over time Street reference data is based on actual street centerlines Town is defined by actual town boundary Location on street defines number range Secondary information (apt, floor, unit) is typically irrelevant

14 LESSONS LEARNED: REPORTED TOWN ACTUAL TOWN Postal code name = town of residence name How do you know which was reported? What happens when the postal zone extends beyond the town border?* Colchester postal code extends into East Haddam town New Haven and East Haven zip codes bisect New Haven and East Haven towns What happens when the postal zone is the same name as another town?* Canaan postal code exists only in North Canaan town North Canton postal code exists only in Barkhamsted town What happens when a postal zone represents more than one town?* Mystic, Stafford springs, Jewett City *see GIS DAY 2011: Geocoding Vital Events poster

15 TOWN? ZIP CODE?

16 LESSONS LEARNED: REPORTED TOWN ACTUAL TOWN CONT. Town-level counts can be skewed when corrections are not made Electronic birth registry defaults Mystic zip code to Groton town CT Tumor Registry collapses Canaan and North Canaan when reporting stats Less populous geographies, like Colebrook, will be more affected by incorrect assignments

17 FINAL THOUGHTS Many lessons learned over 7 years Verify your assumptions Inspect your results Automate your processes Pre-preprocess your data Contact Information: Karyn Backus Epidemiologist CT Department of Public Health Karyn.backus@ct.gov

18 ADDENDUM: CUSTOM STYLE Adjusting the address locator styles In ArcGIS 10.0, these are written in.xml ESRI provides a white paper on customizing address locators in.xml The code is editable but be careful!!! Coding errors may not be obvious in your results!!! ESRI created a custom CT dual-streets style upon request Made the cost/penalty for town and zip equal so that one isn t prioritized Increased the penalty for misspelled town or zip so an exact match is required to meet the minimum 90% match score (E.g., Town code 100 <> Town code 10) Allows for a missing town or a missing zip with no penalty Used heavily in hand-matching areas where zip codes overlap into neighboring towns Useful for matching when a zip code was incorrect or outdated Increased the penalty for town name (E.g., North Main St <> Main St) Allowed for a swap between pre-directional and post-directional when one was blank (E.g., Middle Tpke W will now match to W Middle Tpke) Added local idiosyncrasies to the pattern matching (E.g., Ella T. Grasso) Significantly reduced the time spent hand-matching addresses I knew existed but failed to automatically match