Building a validation database for land cover products from high spatial resolution images The Land Cover project of the ESA Climate Change Initiative Bontemps Sophie 1, Achard Frédéric 2, Lamarche Céline 1, Mayaux Philippe 2, Seifert Frank-Martin 3, Arino Olivier 3, Defourny Pierre 1 1: UCLouvain, Belgium; 2: Joint Research Center, Italy; 3: ESA, ESRIN, Italy
Land cover as an operational product What about products quality?
Validation The process of assessing, by independent means, the quality of the data products derived from the system outputs (CEOS-WGCV definition) CEOS 4-level validation approach according to the temporal and spatial coverage of available reference data CEOS validation stage Stage 1 Stage 2 Stage 3 Stage 4 Characteristics Product accuracy is assessed from a small (typically < 30) set of locations and time periods by comparison with in-situ or other suitable reference data. Product accuracy is estimated over a significant set of locations and time periods by comparison with reference in situ or other suitable reference data. Spatial and temporal consistency of the product and with similar products has been evaluated over globally representative locations and time periods. Results are published in the peer-reviewed literature. Uncertainties in the product and its associated structure are well quantified from comparison with reference in situ or other suitable reference data. Uncertainties are characterized in a statistically robust way over multiple locations and time periods representing global conditions. Spatial and temporal consistency of the product and with similar products has been evaluated over globally representative locations and periods. Results are published in the peer-reviewed literature. Validation results for stage 3 are systematically updated when new product versions are released and as the time-series expands.
The CCI Land Cover project ESA Climate Change Initiative to provide a comprehensive and timely response to the need for long-term satellite-based products in the climate domain Land Cover project to generate global LC maps consistent over time, to match the needs of key users of the climate modelling community. Phase 1 (2011-2013) Phase 2 (2014-2016) http://www.esa-landcover-cci.org/
The CCI Land Cover project 3 consistent global LC maps Envisat MERIS Full & Reduced Resolution 2002-2012 300-1000m SPOT-Vegetation 1 & 2 1998-2012 1000m Envisat ASAR 2005-2012 Mainly Wide Swath Mode (150m)
CCI-LC validation strategy
CCI-LC quantitative accuracy assessment 1. Stratified random sampling of sites 2. Collection of high resolution imagery 3. Pre-processing of high resolution imagery 4. Development of a validation dedicated tool 5. Generation of the reference dataset (image interpretation) 6. Comparison between reference dataset and CCI land cover products
Stratified random sampling 1. Stratified random sampling of sites Sampling design : systematic + 2-stage stratification + random Systematic sampling of the FAO FRA RSS / JRC TREES + diminution of sample intensity with higher latitudes (for equal-probability of samples selection) + stratification for lower intensity in very homogenous desert areas
20 km 8 km Stratified random sampling 1. Stratified random sampling of high-resolution sites Sampling design : systematic + 2-stage stratification + random 8 km Final random selection => 2600 sampling units distributed globally in a systematic stratified random manner
Collection of high resolution imagery 1. Stratified random sampling of sites 2. Collection of high resolution imagery To allow the validation of the 3 epochs (2010, 2005, 2000) 2010 2005 2000 Relying on Google Earth facilities Collecting Landsat imagery for years 2000, 2005 and 2010 (GLS)
Imagery pre-processing 1. Stratified random sampling of sites 2. Collection of high resolution imagery 3. Pre-processing of high resolution imagery Per-object approach: selection of segmentation parameters (object size, compactness, etc.) suitable for a wide variety of landscapes to avoid under-and over-segmentation
Imagery segmentation Under segmentation
Imagery segmentation Over segmentation
Imagery segmentation Correct segmentation
Validation interface 1. Stratified random sampling of sites 2. Collection of high resolution imagery 3. Pre-processing of high resolution imagery 4. Development of a validation dedicated tool On-line interpretation interface derived from the GlobCover tool
Validation interface
Validation interface 1. Layer box 2. Zooms 3. Tools (navigation, NDVI, select object, paint class) 4. Legend 5. Comment
Validation interface Step 1: use all information for assign class in the 2010 period Step 2: indicate the level of certainty for the entire interpretation
Validation interface Step 3: evaluate the change between the 3 epochs, by «back-dating» the 2010 interpretation Step 4: indicate the level of certainty for the entire interpretation
Insular SE Asia: Forest Class
Thailand: mixed land use
Image interpretation 1. Stratified random sampling of sites 2. Collection of high resolution imagery 3. Pre-processing of high resolution imagery 4. Development of a validation dedicated tool 5. Generation of the reference dataset High-resolution image interpretation by experts in a standardized manner on the developed tool
Already a great success with great possibilities for future All experts involved in validation quite happy with the tool: friendly and easy to use, clear interpretation framework, on-line Resulting database: Plenty information that need to be fully exploited 2010 interpretation + level of certainty 2010-2005 - 2000 change information + level of certainty Each sample as a set of objects Homogenity of the sample Interpretation of all LC classes in the unit in a single LC class, in a mosaic, in LC fractions, etc. Used in the CCI-LC project but the use of VHR images coupled with a perobject should allow using it for other maps with lower-higher spatial resolution
Sentinel-2 opportunities Use of S2 imagery instead of Google Earth Less variability in the images to segment (sensor, dates, pre-processing performance, etc.) => better segments delineation Deriving NDVI time profiles from high resolution dataset (S2 being compatible with other HR missions Landsat 8), to use temporal information inside the sample unit Broadleaf or needleleaf? Rainfed or irrigated?
Perspectives Validation tool based on GLC2000, GlobCover, FAO-FRA RSS / JRC-TREES previous projects lessons learned Precursor for operational LC validation activities in collaboration with international scientific community As a concept: Per-object approach inside a given area ensure the scalability of the resulting database => could be used to validate Sentinel-2 LC maps Relies on the availability of a large amount of good quality reference imagery at high spatial resolution => boosted by the unique amount of Sentinel-2 data. As a tool: could be of use in many other types of activities (validation, insitu data collection, crowd-sourcing, etc.), all of them being in link with high spatial resolution imagery Sentinel-2 to make a significant difference as data source and as application
Thank you for your attention