FlexiCadastre User Conference 2013 GIS Data Verification & Challenges May 7, 2013 Amanda Blackmore GIS Specialist
OVERVIEW GIS Definitions & Terminology Spatial Data Common Issues or Concerns Data Verification Challenges Data preparation example 2
What is GIS? Definition Computer-based system for storing and processing geographic information; to efficiently handle information about geographic locations and events. Allows users to: organize & store, access & retrieve data visualize, analyze, interpret, and understand data, discover relationships, patterns, and trends within the data. http://www.pobonline.com/pob/home/images/pob0708_nils02_parcel_lg.jpg 3
What is GIS? GIS is not just software Network computers linked together, internet, intranet Hardware computers, cell phones, Software desktop (ArcGIS, MapInfo, Erdas Imagine, Geosoft, GeoMedia, GRASS, MicroStation, ); web mapping systems; spatial database systems Data various data sources & types (shapefiles, geodatabases (feature classes, etc.), imagery, TINs, GRIDs, spreadsheets, ) Procedures ensure high quality standards and meet organization needs People programming software; creating/ supplying/ updating data; analyzing / interpreting/using results Diagram from Longley, et al. (2011) 4
Definitions & Terminology General Overview of GIS-Related Terms Data types Coordinate systems Georeference Scale Mapping 5
Definitions & Terminology Two Types of Digital Data Represented in a GIS Vector points, lines, polygons (example: license boundaries are polygons); can hold attribute information (number or text). Examples: Cannot be more accurate than the original scale of the original data (for example, paper map). Apparent precision of vector data not real. Primary sources: GPS measurements, field survey measurements. Secondary sources: Topographic and other paper maps (digitized). 6
Definitions & Terminology Two Types of Digital Data Represented in a GIS Raster space is divided into rectangular array (usually square); each cell can hold attribute information (number or text). Examples: Lake Bog Beach Forest Subject to projection distortion (raster data are inherently flat). Resolution vs. space required limitation (more detail more voluminous; larger cells less detail). Primary sources: digital satellite imagery, digital aerial photographs. Secondary sources: scanned maps, digital elevation models from topographic map contours. 7
Definitions & Terminology Representing the Earth Geographic coordinate system Defined from the axis of the Earth s rotation, and plane through the centre of mass... Longitude: (N-S lines) slices through the Earth parallel to the axis (segments of orange); Latitude: (E-W lines) perpendicular to longitude and parallel to the equator; The latitude is dependent on the ellipsoid (model of shape of Earth) and datum (which defines the axis of said ellipsoid, as Earth is not a perfect sphere). Datum: generally refers to horizontal datum; however if elevation is a factor, vertical datum may also be defined. Image from: http://geographyworldonline.com 8
Definitions & Terminology Representing 3-D in 2-D Projection Mathematical transform to convert 3D to 2D (are innumerable types local, regional, world) Different types may preserve different properties of the data (area, scale, angle, shape none preserve all) A projected coordinate system is always based on a geographic coordinate system, which is based on an ellipsoid. Image from: www.geosoft.com/.../geographic-projections-using-proj... - United States 9
Definitions & Terminology Representing 3-D in 2-D Projection (continued) Projections do distort the Earth impossible for any flat map to be perfectly uniform. Projections may be classified by 1) the way they distort the Earth, or 2) by how the flat surface is related to the positions on the Earth s surface. 3 types of projections onto a flat surface Planar e.g., Stereographic Cylindrical (transverse) e.g., Universal Mercator Transverse (UTM) Conic e.g., Lambert Conformal Conic (LCC) 10
Definitions & Terminology Representing 3-D in 2-D Example: Geographic Coordinate System with a Mercator projection Line of Longitude Line of Latitude (0 degrees = equator) 11
Definitions & Terminology Representing 3-D in 2-D Projection Considerations: Need to determine most appropriate for particular use (e.g., need to calculate area? > need a projection that conserves area). Data can be in many different projections and datums need to know each one being used. May need to re-project to a standard projection within a project, or organization. Within one data layer if multiple data sources are in different projections & / or datums, know that adjacent features are not likely to line up as expected (may need further processing to align features as needed). 12
Definitions & Terminology Georeference Spatial reference: describing location of data within the real world x, y coordinates can be georeferenced with a geographic coordinate system or projection (can also be a vertical component) Georeferencing a data set or raster applying a real world spatial reference to the spatial data, such as through: Applying points of known location ( ground control points ) Using vector data of appropriate scale (e.g., road intersections, rivers, ) Using other, georeferenced, data (e.g., recognizable features in both the ungeoreferenced data and georeferenced or rectified satellite imagery) 13
Definitions & Terminology Scale Various meanings or usages of scale may include: Level of geographic detail or spatial resolution Geographic extent or scope of a project Large scale meaning the scope or geographic extent is large Scale of a map (cartographic) Representative fraction or ratio of distance on the map to distance on the ground Large scale - large fraction (closer to 1) more detailed or fine (e.g., 1: 50 000 is more detailed than 1: 1 000 000) Scale bar 0 1 2 4 Kilometers 14
Definitions & Terminology Scale Example of data at 1: 50 000 vs 1: 1 000 000 1: 1 000 000 roads in black 1: 50 000 roads in red 15
Definitions & Terminology Mapping Static maps portray a static view of the world fixed scale and extent example: historical claim map, for the township of Ogden (Ontario, Canada) 16
Definitions & Terminology Mapping Dynamic mapping interactive with data GIS software (e.g., mxd file with ArcGIS) Web Mapping Service (WMS) GIS-capability may be limited within WMS, but allows users to dynamically interact with data examples: GLAM/FlexiCadastre, Ontario CLAIMaps database 17
Definitions & Terminology Mapping Map Composition principles of composition purpose, availability of data, scale, limits symbolization of data layers (e.g., by selected type of attribute, field/column within attribute table, other criteria) other components north arrow, legend, text, labeling, etc. example (next slide): mapping Goldcorp project location data from MEG, by type of project 18
Goldcorp MEG Project Locations as of April 2012 19
Spatial Data Common Data Quality Concerns can Include: Availability How complete are the data? If incomplete/missing, can they be derived from other sources? Accuracy Are the data considered accurate (in terms of both spatial position and the attributes/information being mapped)? Trustworthy/known sources? History Date/source of original material known? Any information of how data were transformed from original source to current digital format (data lineage )? Disparity Dealing with data from multiple sources, with different projections, scales, etc.; need to determine priority and usefulness / limits of each. 20
Spatial Data Project-specific Issues may be Identified: Projection Which ones are being used? Re-projection needed? Determine appropriate projection(s) for project, use consistently. Scale What scale(s) might be appropriate for the determined use? May need to use data of many different scales users need to be aware of limitations / appropriateness. Contiguity Are the polygons / lines properly meeting together (or being snapped) properly? Are there slivers or overlaps that should not exist? How could it affect usage? Consistency Are the codes/labels/other attributes consistent, match with previously mapped data? Processing being conducted in a consistent way? Being documented? Missing spatial data are missing, requiring research into historical records; or available data are unreliable and need to be re-created from known sources (for further external validation). 21
Data Verification The process of data verification can be anything from very simple to complex. Even if basic stages are identified and used, going through the process may become quite involved or complicated. Different models for different situations / projects / data acquisition & sources. May need to change or adapt process over time. Example of general verification steps follows. 22
Data Verification General Process of Data Acquisition -> Import Data Capture: Acquire (from government sources, digitizing, etc.) shape data Data Comparison: Compare / check the data of different sources Correction: Fix shape data (attributes/geometry) Verify: Changes/progress (with LM, at intervals) Import: Correct/corrected shapes into GLAM Import Verify Data Capture Correction (Geometry, Attribute) Data Comparison 23
Data Verification Data Capture Spatial data sources have included: Downloadable data from websites (e.g., Natural Resources Canada / GeoGratis) Surveys Historical and other paper maps GIS vector data Legacy data in other, non-gis, formats (e.g., AutoCAD) Digitizing Data process: Convert raster data to vector data Scan paper maps, then georeference in a GIS Digitize (or with the mouse, draw the polygon shapes) the desired features, in a GIS 24
Data Verification Data Comparison Compare spatial data (difference sources, checking for suitability, etc.) Never really quite as straight-forward as you might want it to be Example of comparing georeferenced block maps with current government disposition boundaries (purple boundaries). Ontario government is the source for both. 25
Data Verification Correction Geometry Attributes Depending on severity, may be with land manager (where significant differences in boundary location are identified) Generally with land manager (may include anything from license codes to whatever else is being tracked status, type.) 26
Data Verification Verify (by issue, import, interval) Identifying issues & investigating with a Land Manager Listing issues in spreadsheet, reviewing with Land Manager, tracking progress of shape edits and uploads Currently 5680 entries completed or in progress in this spreadsheet, involving 9 regions (Canada/USA) After an import: verify individual imports (Search on missing shapes, export data and compare to other data, etc.) Export data at regular intervals (first of month) and do spreadsheet analysis on data By License Region, License Type, and License Status Compare to previous summary (tracking overall numbers) 27
Data Verification Import Import spatial data into GLAM Bulk imports (load shapefile / feature class / etc. into import template) Individual license shape import Documentation (e.g., shapes imported, import itself) 28
Challenges Many challenges have been encountered during shape data preparation for upload into GLAM, including (and not limited to): Data provided in problematic projection (such as a local projection, needing a realworld spatial reference), Data in raster format (or paper map to be scanned), Unknown accuracy, or even unknown history altogether Provided in non-gis format (such as AutoCAD), Conflicting data sources creating a question of which source should have priority. Example: GIS shape data preparation for Equity, British Columbia 29
Challenges Case of Equity (British Columbia, Canada) Data Preparation Situation: Data provided was AutoCAD vector data (last updated in early 2000 s), in a local projection, of unknown accuracy (requiring conversion). 3 files, multiple line/other layers 1 AutoCAD file did not have all the data in it to be able to georeference directly in a GIS. AutoCAD data could not be directly projected to a real-world projection (need two georeferencing points to project file in ArcMap). Data provided was not organized in consistent way (attribute tables, layers, labels). 30
Challenges Case of Equity (Canada) Data Preparation for GLAM Solution: Integrated all suitable data layers (license boundaries, lakes, streams, roads) into one file and exported to raster ( last resort method). Georeferenced this raster using roads and river data (vector data used for spatial reference from GeoGratis website) ( rubber-sheeted ). Digitized the license boundaries manually. Results: Compared initial digitized data to current government data found significant differences. Data submitted to government for consideration. 31
Text Government data (downloaded from website April 2013) in red (claims) and purple (leases); data digitized from legacy AutoCAD data in yellow (claims) and brown (leases). 32
Challenges Case of Equity (Canada) Data Preparation for GLAM Further complication: One specific area needed to be considered as part of Goldcorp property for monitoring purposes, which is included in original AutoCAD data, but does not appear to be on a Goldcorp claim as per current government data. Issue could have been addressed earlier or perhaps even prevented altogether if data conversion & comparison had taken places years ago (AutoCAD data dates from early 2000 s, and apparently not compared with government data until early 2012). Current status: We received a response back from the BC government (Dec 2012) regarding our initial request (Mar 2012), and are waiting further response regarding the apparent boundary discrepancies. 33
GIS Data Verification & Challenges In conclusion GIS in general can be a very useful tool for data verification & correction. Verifying data spatially can reveal issues and problems that may otherwise be difficult (if not impossible) to identify. Data verification process is key the data must be reliable (limits identified as best possible); process can be very involved or complicated. GIS is an invaluable tool for converting data into one standard format and spatial reference for book of record, such as GLAM. 34
References Longley, Goodchild, Maguire, and Rhind. 2011. Geographic Information Systems & Science, 3 rd Ed. John Wiley & Sons, Inc., USA. MEG project data from www.metalseconomics.com (Apr 13/12). 35
Thank You