Large Scale Elevation Data Assessment of SCOOP 2013 Deliverables. Version 1.1 March 2015

Transcription

1 Large Scale Elevation Data Assessment of SCOOP 2013 Deliverables Version 1.1 March 2015

2 ACKNOWLEDGEMENTS... 2 EXECUTIVE SUMMARY INTRODUCTION BACKGROUND SOUTH CENTRAL ONTARIO ORTHOPHOTOGRAPHY PROJECT ASPRS POSITIONAL ACCURACY STANDARDS FOR DIGITAL GEOSPATIAL DATA (2014) ELEVATION DATA PRODUCTION ACQUISITION TECHNIQUES Semi-Global Matching LiDAR DEM PRODUCTION SCOOP DEM LiDAR DEM ACCURACY ASSESSMENT PROCEDURES CHECKPOINT REQUIREMENTS TESTING ACCURACY IN GIS RESULTS VERTICAL ACCURACY REPORTING STATEMENTS LiDAR DEM Vertical Accuracy SCOOP DEM Vertical Accuracy DISCUSSION VERTICAL ACCURACY AND QUALITY OF ELEVATION DATA RECOMMENDATIONS REFERENCES APPENDIX A APPENDIX B

3 Acknowledgements Large Scale Elevation Data Assessment of SCOOP 2013 Deliverables was written by Ian Jeffrey, GIS/Remote Sensing Specialist of the Ganaraska Region Conservation Authority. Financial support of Large Scale Elevation Data Assessment of SCOOP 2013 was provided by the Ontario Ministry of Natural Resources and Forestry (OMNRF) as part of the Assessment of Provincial Conservation Authority Floodplain Mapping Status and Case Studies to Assess Large Scale Geospatial Data and Hydrology/Hydraulic Models for Floodplain Mapping. It is also acknowledged that the foundation of knowledge required to complete this report has been the result of continued technical collaboration between the Ganaraska Region Conservation Authority and the Grand River Conservation Authority as well as the Ontario Ministry of Natural Resources and Forestry. Executive Summary Conservation Authorities in Ontario are facing challenges in meeting large scale elevation data needs. The Ontario Imagery Strategy offers a cost effective option for obtaining high-resolution orthoimagery as well as large scale elevation data. The South Central Ontario Orthophotography Project (SCOOP) 2013 is the first phase in the current five-year Ontario Imagery Strategy refresh cycle. This report documents the findings of vertical accuracy assessments of large scale elevation data delivered as part of SCOOP 2013 using the newly released American Society of Photogrammetry and Remote Sensing (ASPRS) Positional Accuracy Standards for Digital Geospatial Data. Correct citation of this document: Ganaraska Region Conservation Authority Large Scale Elevation Data Assessment of SCOOP 2013 Deliverables. Ganaraska Region Conservation Authority. Port Hope, Ontario. 2

4 1.0 Introduction Assessing the accuracy of a map was for centuries a relatively straightforward endeavour. The practice involved a hard copy cartographic map, a map scale, a scale ruler, and a set of procedures whereby the map was assessed as a scaled down version of reality with true relative geometries. Information to be represented on the map was captured and assessed as planar coordinates using scale relative to local survey monuments. In this context, mapping was a practice of representing geometry as a function of scale. As a result of advancements in the way in which mapping information is acquired, this approach no longer suffices. Mapping information, known as geospatial data, can now be acquired and processed using new technologies which enable data applications and analyses previously impossible. Further, these technologies provide a previously unattainable level of reliability. Traditional survey methods are still fundamental, but can now be complemented with various methods of data acquisition involving Real-Time Kinematic Global Navigational Satellite Systems (RTK GNSS), Light Detection and Ranging (LiDAR), Synthetic Aperture RADAR (SAR), to name a few. As result of these modern data acquisition methods, data is now captured, projected, and assessed in terms of absolute accuracy. Observed locations are now referenced to navigational satellites which are in turn referenced to deep space quasars so distant that they are effectively stationary in relation to the Earth. Through the refinement of these positional relationships, the accuracy of a location on the Earth can be determined to the millimetre level in absolute terms. As mapping professionals, how do we reconcile this technological leap in data acquisition and modeling? How do we procure data with any level of guarantee that it will in fact meet the data requirements of any intended use? In November 2014, ASPRS released a set of standards which aims to provide a quantitative framework for testing and reporting the accuracy of geospatial datasets. Under the title of the ASPRS Positional Accuracy Standards for Digital Geospatial Data, herein referred to as 2014 ASPRS Standards, a detailed standard for how digital geospatial data ought to be tested for accuracy, as well as how to effectively report the test results, was released. In Spring 2013, a leaf-off orthoimagery acquisition was made for central Ontario. The results were delivered in early 2014 including the full suite of data captured and produced in effort to create processed orthoimagery. As part of this full suite delivery, an unclassified 3D point cloud extracted from the raw aerial image stereo pairs was received. The following report aims to apply relevant portions of the 2014 ASPRS Standards to a terrain model produced from the SCOOP-delivered 3D point cloud. 3

5 2.0 Background In Ontario, there is a current and pronounced business need for one type of geospatial data defined as large scale elevation data, herein referred to simply as elevation data. The high costs associated with acquiring and preparing elevation data present challenges in addressing these data needs. The Ontario Imagery Strategy offers a unique and cost effective approach to providing partners with high quality orthoimagery as well as the elevation data generated as part of the orthorectification process. Concurrent to the need for elevation data is the parallel need for detailed standards on how to test and report the accuracy of elevation data and geospatial data in general. The recently released 2014 ASPRS Standards offer a potential solution to fill this void. 2.1 South Central Ontario Orthophotography Project 2013 SCOOP2013 is the first in a multi-year phased acquisition of high-quality orthoimagery in southern, and parts of northern, Ontario. SCOOP2013, as well as four subsequent phases covering other areas of Ontario, are key components of the Ontario Imagery Strategy. The Strategy holds the mandate to facilitate a multi-year, cross-industry partnership model to provide high quality orthoimagery and its associated products in a cost effective manner. Projects of this magnitude are effectively cost-prohibitive for any one organization, thus reinforcing the need for a provincially-coordinated approach such as the Ontario Imagery Strategy. 2.2 ASPRS Positional Accuracy Standards for Digital Geospatial Data (2014) In response to challenges faced by the industry in terms of linking acquisition to end product, the American Society of Photogrammetry and Remote Sensing (ASPRS) released an updated and more comprehensive set of specifications called the ASPRS Positional Accuracy Standards for Digital Geospatial Data (2014). These specifications effectively represent the most thorough and up-to-date reference for measuring and communicating the accuracy of geospatial data. 4

6 3.0 Elevation Data Production Elevation data can be acquired in many ways, but once brought into a Geographic Information Systems (GIS) lab, it becomes geospatial data all the same. These points and lines are then processed into end products designed to meet defined business data requirements. Perhaps the most common elevation data product is the digital elevation model (DEM). The DEM is defined as a bare-earth gridded raster representation of an area of interest georeferenced to the Earth s surface. For the purpose of this study, two different DEMs will be assessed using the 2014 ASPRS Standards; one created from LiDAR (LiDAR DEM) and the other from a delivered point cloud created using Semi-Global Matching (SGM) and delivered as part of the SCOOP2013 deliverables (SCOOP DEM). 3.1 Acquisition Techniques The most effective way to acquire high-quality geospatial data over a large area is by way of remote sensing. Remote sensing obtaining data of objects indirectly, without physical contact can be used to acquire geospatial for use in GIS. There are three main forms of remote sensing geospatial data acquisition currently in widespread use in Ontario: Light Detection and Ranging (LiDAR), Semi-Global Matching (SGM), and Synthetic Aperture RADAR (SAR). This report will compare data recently acquired using LiDAR and SGM. Generally speaking, each different remote sensing method has its associated strengths and weaknesses across multiple business purposes. For the purpose of this study, the effectiveness of obtaining high-quality bare earth data representation will be the principal focus. It should be kept in mind that there exist many other uses for remote sensing, with the derivation of a bare earth DEM as just one use. With that being said, a balance must be struck in choosing appropriate, and cost effective acquisition method(s) for the intended purpose. A thorough understanding of the strengths and limitations of a given acquisition method enables accuracy requirements to be met but not significantly exceeded Semi-Global Matching A digital photogrammetric technique, SGM is a refined form of auto-extraction of precise and accurate 3D masspoint information from overlapping stereo imagery. SGM can be classified under the broader category of image matching techniques which also include pixel autocorrelation, or, simply, elevation data extraction. What sets SGM apart from these other approaches is that it combines concepts of global and local stereo methods for accurate, pixelwise matching at low runtime (Hirschmuller, 2011). After intensive analysis through computation, the SGM technique produces data in the form of a point cloud. The use of the SGM processing technqiue for SCOOP2013 was enabled by using a pushbroom sensor (Leica ADS80-SH2) for data acquisition instead of the traditional fixed-frame sensor used in previous Ontario Imagery Strategy acquisitions. 5

7 3.1.2 LiDAR LiDAR or Light Detection and Ranging is an efficient means of acquiring highly-accurate elevation data for a given area. Whereas RTK GNSS can be used to capture targeted and discrete features, LiDAR is essentially a scan of the Earth that results in a large dataset that can be used to derive elevation data products. The LiDAR system is comprised of a laser scanner mounted in a fixed or rotating-wing aircraft alongside an inertial measurement unit (IMU) linked to an atomic clock and survey-grade GNSS unit. In brief, the laser scanner scans the Earth from the belly of an aircraft, while the IMU records orientation of the aircraft in the sky and the GNSS simultaneously measures the aircraft location relative to a geodetic datum. The scanner measures the time it takes for each laser scan pulse emitted to return to the sensor. Given that the speed of light can be considered infinite due to the close relative proximity of the scanner to the Earth, a direct inference of time can be determined as a function of distance. What is retrieved from the aircraft upon mission completion is an irregularly-spaced mass of points known as a point cloud. 3.2 DEM Production SCOOP DEM For the purpose of this study, a digital surface model (DSM) point cloud was received from the SCOOP2013 vendor, created using the semi-global matching technique. This approach effectively produced an unclassified point cloud at 40 cm ground sample distance (GSD). This DSM point cloud was imported into the Ganaraska Region Conservation Authority (GRCA) Remote Sensing Lab and processed in effort to delineated ground points for the purpose of DEM production. Using Inpho DTMaster software, the DSM point cloud was classified by way of a three-step automated filtering process. After multiple iterations and manual inspection routines, a classified ground point cloud was produced. This point cloud was then imported into ESRI ArcGIS 10.2 software for DEM production. By way of the ESRI Terrain Dataset individual classified ground point tiles were fused together and interpolated to produce a bare earth raster DEM at 0.5 m ground resolution. *See Appendix A for a detailed step-by-step account of the SCOOP DEM production process LiDAR DEM A LiDAR-derived DEM acquired in 2006 at 0.5 m ground resolution of Midtown Creek Catchment in Cobourg, Ontario was used for comparison in this study. 6

8 4.0 Accuracy Assessment Procedures 4.1 Checkpoint Requirements As required by the 2014 ASPRS Standards, vertical accuracy is tested by comparing the elevations of the surface represented by the data set with elevations determined from an independent source of higher accuracy which is defined as at least three times more accurate than the required accuracy of the geospatial data set being tested. This requirement was satisfied by acquiring survey-grade checkpoints using Real Time Kinematic (RTK) GNSS data captured in the field at centimetre-level accuracy, and classified by vegetative land cover type. Data was captured in vegetated and non-vegetated terrain to enable the accuracy to be tested and to ensure the results would be effectively communicated in terms of the overall reliability of DEM across the entire area of interest. Remote sensing data capture can become a challenge in vegetated areas in that the bare earth is obstructed, often resulting in less data captured for bare earth representation. A DEM is a continuous, gradient dataset with no discrete positions, thereby ruling out measuring horizontal accuracy, with vertical accuracy as the sole indicator of its overall reliability. This context calls for a more robust statistical approach than simply averaging the differences between test and control datasets. In areas designated as Non-vegetated (bare soil, sand, rocks, and short grass) as well as urban terrain (asphalt and concrete surfaces), elevation errors can be assumed to follow a normal distribution. Given this assumption, accuracy statistics can be represented as Non-vegetated Vertical Accuracy (NVA) at a 95% confidence interval. In areas where vegetative cover is more prevalent (tall weeds and crops, brush lands, and fully forested areas), error cannot be assumed to follow a normal error distribution. Therefore, RMSEz-based statistics cannot be used to estimate vertical accuracy in these cases. Instead, accuracy must be calculated as Vegetated Vertical Accuracy (VVA) at the 95 th percentile of the absolute value of vertical errors. 4.2 Testing Accuracy in GIS Survey checkpoints were imported into the GIS lab in the form of a projected shapefile point dataset. Once datums and projections were verified to be consistent across all datasets, the checkpoint shapefile was used to extract coincident raster values for statistical analysis. 7

9 5.0 Results 5.1 Vertical Accuracy After statistical analysis of vertical accuracy assessment data, results were reported in the current NVA (n = 25) and VVA (n = 20) statistics as well as legacy statistics for the purpose of comparison. Test DEM RMSE (m) LMAS RMSE z Non- Veget ated (cm) NVA at 95% Confidenc e Level (cm) VVA at 95th Percentil e (cm) Vertical Accurac y Class 2014 Equivale nt Class 1 Contour Interval per ASPRS 1990 (cm) Equivale nt Class 2 Contour Interval per ASPRS 1990 (cm) Equivale nt contour interval per NMAS (cm) LiDAR N/A X DEM (NVA) SCOOP N/A X DEM (NVA) LiDAR N/A N/A X N/A N/A N/A N/A DEM (VVA) SCOOP DEM N/A N/A X N/A N/A N/A N/A (VVA) Table 1 Vertical Accuracy Test Results in Current and Legacy Statistical Representations 5.2 Reporting Statements FGDC National Standard for Spatial Data Accuracy - NSSDA - Vertical - 95% Confidenc e Level (cm) *Note: Project data was acquired before 2014 ASPRS Standards released. Therefore, data could not have been acquired to meet a specified accuracy class. For consistency, the reporting statements are printed in full with the specified accuracy class denote by X in all cases LiDAR DEM Vertical Accuracy This data was produced to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data (2014) for a X cm RMSEz Vertical Accuracy Class. Actual NVA accuracy was found to be RMSEz = 27.4 cm, equating to +/ cm at 95% confidence level. Actual VVA accuracy was found to be +/ cm at the 95th percentile SCOOP DEM Vertical Accuracy This data was produced to meet ASPRS Positional Accuracy Standards for Digital Geospatial Data (2014) for an X cm RMSEz Vertical Accuracy Class. Actual NVA accuracy was found to be RMSEz = 15.0 cm, equating to +/ cm at 95% confidence level. Actual VVA accuracy was found to be +/ cm at the 95th percentile. 8

10 6.0 Discussion 6.1 Vertical Accuracy and Quality of Elevation Data Vertical accuracy test results show the SCOOP-derived DEM outperforming the LiDAR-derived DEM in both NVA and VVA for the Midtown Creek Catchment in Cobourg, Ontario. With an NVA RMSEz value of 0.15 m, the SCOOP-derived DEM falls under the 15-cm Vertical Accuracy Class as defined in the 2014 ASPRS Guidelines. This class also corresponds with the Level 2 Accuracy Category as defined in the 2009 Ontario Imagery and Elevation Guidelines (LMAS = m). By meeting Level 2 mapping accuracy, SCOOP-derived DEM can be sufficient to use in densely to moderately populated urban areas that may or may not fall within the Regulated Flood Line (Ontario, 2009). Alternately, the LiDAR DEM with an NVA RMSEz of m and LMAS of m, falls under the Level 3 Accuracy Category. This category restricts appropriate use of this data to moderately or sparsely populated areas that are primarily surrounded by agricultural and/or forested lands. In terms defined by the 2014 ASPRS Standard, the LiDAR DEM falls under the 33.3 cm Vertical Accuracy Class. Table 2 Vertical Accuracy/Quality Classes for Digital Elevation Data (ASPRS, 2014) 9

11 6.2 Recommendations The results presented by this report provide sound evidence of the potential for using SCOOPderived elevation data for DEM production. Further research to other elements of the 2014 ASPRS Standards would be of benefit prior to applying all elements of the Standards across the whole suite of SCOOP2013 deliverables. Aerial triangulation standards are an intriguing component to these new standards, enabling users for the first time to define an aerial imagery acquisition in terms of the quantified level of accuracy a derivative elevation data products may be required to meet. Further exploration into automating data production routines to this extent could potentially yield great benefits in addressing costly imagery and elevation data needs. Further to the above point, additional research with regards to establishing appropriate use categories, similar to the 2009 Ontario Imagery and Elevation Guidelines, would make for a powerful complimentary document, particularly with links to the 2014 ASPRS Standards. GNSS technology, RTK or otherwise, allows for a thread to be drawn through acquisition, processing, accuracy testing, and metadata. This integrated approach suggests that multiple business areas can now also be served by coupled imagery/elevation acquisitions, further increasing Return on Investment. Lastly, remote sensing has proven to be a powerful means of acquiring high-quality topographic data, however, regardless of how far data capture technology progresses, there will still be areas deemed to be low confidence due to obstructions in birds-eye sight (among other causes). Steps to integrate remotely sensed data products with on-the-ground surveying be it traditional surveying or land-based laser scanning offers an effective solution to this persistent issue. The findings of this report provide a promising glimpse of what can be achieved by incorporating and employing modern geospatial data acquisition and modeling technologies to serve welldefined business data requirements, as well as address pronounced large scale elevation data gaps across the Province of Ontario. 10

12 7.0 References ASPRS, ASPRS Positional Accuracy Standards for Digital Geospatial Data, Photogrammetric Engineering & Remote Sensing, Volume 81, No. 3, 53 p., URL: Gehrke, S., Morin, K., Downey, M., Boehrer, N. and Fuchs, T., Semi-Global Matching: An Alternative to LiDAR for DSM Generation?, Canadian Geomatics Conference and Symposium of Commission I, ISPRS, June 2010, Calgary, Canada. Government of Ontario, Ontario Imagery and Elevation Acquisitions Guidelines, Queen s Printer of Ontario. Grand River Conservation Authority, DEM Development, Integrated Large Scale Hydrology Pilot Tutorials, Cambridge, ON. Hirschmüller, H., Semi-Global Matching Motivation, Developments and Applications, Invited Paper at the Photogrammetric Week, September 2011 in Stuttgart, Germany, pp

13 Appendix A SCOOP DEM Production Workflow Portions of the SCOOP DEM Production Workflow adapted from Grand River Conservation Authority (2011). 1. Inpho DTMaster: Classify LAZ point cloud 3 step filter process: o LiDAR_STRONG 1. Non-ground feature removal. o THINOUT 1. Removal of unnecessary data points. o GROSS_ERRORS 1. Removal of remaining outlier data points. Figure 1 Filtered point cloud in Inpho DTMaster (GRCA, 2015) 2. Inpho DTMaster: DSM output (Classified LAS Files) Export Raw Classified LAS 3. Inpho DTMaster: LAS to MP Feature Class Export Raw Classified SHP 12

14 Figure 2 Delineated ground points in ArcGIS 10.2 (GRCA, 2015) 4. ESRI ArcGIS 10.2: Create ArcGIS Terrain Create Temp Terrain from Raw Classified SHP Figure 3 Dynamic Temporary Terrain Dataset (GRCA, 2015). 13

15 5. ESRI ArcGIS 10.2: Temp Raster DEM/Hillshade Interpolate to Temp DEM/Hillshade Figure 4 Temporary DEM ready for visual inspection (GRCA, 2015). 6. ESRI ArcGIS 10.2: Create issue areas Manual inspection of 3D hillshade view. Manual digitization of apparent issue areas relative to SCOOP Orthoimagery 14

16 Figure 5 Issue areas digitized over Temporary Hillshade (top) and over associated orthoimagery (bottom), (GRCA, 2015) 15

17 7. LAStools: Remove issue areas from LAS Data points which intersect or overlap with issues areas deleted from Raw Classified LAS Produces Final Classified LAS, exported to Final Classified SHP 8. ESRI ArcGIS 10.2: Final Terrain Final Classified SHP imported into Final Terrain Dataset 9. ESRI ArcGIS 10.2: Interpolated to raster DEM/Hillshade Final Terrain Dataset interpolated to Final DEM Figure 6 Final DEM of Midtown Creek Catchment - Cobourg, Ontario (GRCA, 2015) 16

18 Appendix B SCOOP DEM Non-Vegetated Areas Accuracy Test Statistics PT_ID NORTHING EASTING ELEVATION DESCRIPTIO RASTERVALU Zdiff ZdiffSq SumDiffSq Avg RMSE QC-URB QC-URB QC-URB QC-URB QC-URB E QC-URB QC-URB QC-URB QC-URB QC-URB QC-OPEN QC-OPEN QC-OPEN QC-OPEN QC-OPEN E QC-URB QC-URB QC-URB QC-URB QC-URB QC-OPEN QC-OPEN QC-OPEN QC-OPEN QC-OPEN

19 SCOOP DEM Vegetated Areas Accuracy Test Statistics PT_ID NORTHING EASTING ELEVATION DESCRIPTIO RASTERVALU Zdiff ZdiffSq SumDiffSq Avg RMSE QC-LONG QC-LONG QC-LONG QC-LONG QC-LONG QC-BRUSH QC-BRUSH QC-BRUSH QC-BRUSH QC-BRUSH QC-LONG GRASS QC-LONG GRASS QC-LONG GRASS QC-LONG GRASS QC-LONG GRASS QC-LONG GRASS QC-FOR QC-FOR QC-FOR QC-FOR

20 LiDAR DEM Non-Vegetated Areas Accuracy Test Statistics PT_ID NORTHING EASTING ELEVATION DESCRIPTIO RASTERVALU Zdiff ZdiffSq SumDiffSq Avg RMSE QC-URB QC-URB QC-URB QC-URB QC-URB QC-URB QC-URB QC-URB QC-URB QC-URB QC-OPEN QC-OPEN QC-OPEN QC-OPEN QC-OPEN QC-URB QC-URB QC-URB QC-URB QC-URB QC-OPEN QC-OPEN QC-OPEN QC-OPEN QC-OPEN

21 LiDAR DEM Vegetated Areas Accuracy Test Statistics PT_ID NORTHING EASTING ELEVATION DESCRIPTIO RASTERVALU Zdiff ZdiffSq SumDiffSq Avg RMSE QC-LONG E QC-LONG QC-LONG E QC-LONG QC-LONG QC-BRUSH QC-BRUSH QC-BRUSH QC-BRUSH QC-BRUSH QC-LONG GRASS QC-LONG GRASS QC-LONG GRASS QC-LONG GRASS QC-LONG GRASS QC-LONG GRASS QC-FOR QC-FOR QC-FOR QC-FOR

22