A novel approach for validating raster datasets with categorical data

Transcription

1 A novel approach for validating raster datasets with categorical data E. Dobos, P. Vadnai, D. Bertóti, & K. Kovács University of Miskolc, Geography Institute, Miskolc-Egyetemváros, Hungary E. Micheli, V. Láng & M. Fuchs Szent István University, Dept. of Soil Science and Agrochemistry, Gödöllő, Páter K.U ABSTRACT: Digital soil mapping develops several soil property and classification maps using remotely sensed data and digital terrain models as input variables into the model. The outputs of these maps are often in raster format with block support, representing average values for the pixel area or the dominant class occurring within the pixels. At the end these pixels occur as pure spatial object, but in reality there is a great heterogeneity behind these overall values. This study aims to characterize this heterogeneity using the reference soil group level of WRB and report the development and use of a validation datasets and methodology to characterize the datasets by its pixel purity and accuracy. This task requires the quantification of the relationships, similarities between the taxonomic classes. A hybrid approach of taxonomic distance and expert knowledge based similarity calculation is presented as well. 1 INTRODUCTION Digital soil mapping has become a very efficient tool in soil science, and several applications have been published (McBratney et al., 2003, Lagacherie et al. 2006). Soil data is needed in a format that can be integrated into the existing digital models. The majority of soil data users require data in raster format with values of certain properties, like ph, clay content or soil organic matter content. Some of these variables are used as it is, as direct inputs into the model, while others are used to estimate complex soil features and properties, like diagnostic features and horizons. The presentation and validation of these raster based, categorical soil data are not well developed. The e- SOTER project developed a novel approach to present categorical information on a block support. The resulting dataset has several layers of occurrence probabilities of WRB diagnostic horizons/features/properties and an additional layer of the reference soil group (RSG) of the WRB system (IUSS Working Group WRB. 2007). However, no appropriate validation methodology and data exist so far. This paper describes a novel approach for the development of a validation database and its use for validating WRB reference soil groups and the occurrence probabilities of selected diagnostics. The sampling methodology combines an automated simple random sampling with slight adjustment for better accessibility and fit to the raster database and a systematic random sampling approach to populate the selected pixels with additional observations. At the end each validation sites pixel area - have 5 observations. Therefore the overall purity - defined as the proportion of the mapped area covered by a certain soil class can be predicted. 2 METHODS 2.1 The dataset to be validated The dataset has a 500 meter resolution with only one attribute namely the reference soil group name from WRB and covers the area of Hungary. It was developed using SRTM and MODIS data and several ground truth points for the classification following the e-soter methodology (Dobos et al. 2010, 2011, 2013). This 500 m pixel area is large enough to include several soil classes. However, only the dominating soil class is allocated to the pixel. The goal is to develop a validation dataset and a methodology to describe the overall quality of the dataset. E. Dobos, P. Vadnai, D. Bertóti, & K. Kovács E. Micheli, V. Láng & M. Fuchs A novel approach for validating raster datasets with categorical data. First GlobalSoilMap Conference, October 7-9. Orleans France.

2 2.2 Overall validation procedure There were several independent national datasets used for the digital soil mapping exercise to develop the WRB RSG layer. An external validation dataset was developed and used for predicting the accuracy of the results. The sites/pixels for validation were randomly selected. All sites had 5 observations falling within a 450 by 450 meters pixel area. Having these 5 observations, proportions of the RSG within the pixel can be approximated with 20, 40, 60, 80 and 100 percent coverage. This information is used in the last step as field/ground truth data. The validation is done considering the taxonomic adjacencies between the WRB RSG classes and defining similarity factors to express their relationship in the quantification of the level of misclassification/uncertainty. 2.3 Field work and database design Each sites had one profile opened in the centre of the selected pixel. This soil pit was described and all WRB diagnostic criteria, materials, horizons and features have been documented and the classification name was defined. After describing the pit, four additional augerings were deepened 100 meter North, East, South and West from the pit. The material taken out from the hole has been put into a 1 meter long tray keeping the original depth. This way a disturbed profile has been created and was taken back to the pit, where all four were put next to each other in a clockwise order starting from North and a documenting photo was taken from the trays and the pit as well. All four disturbed profiles have been described in the same way as the pit. Table 1. The example of the validation database Reference data Predicted data RSG % within the pixel RSG Taxonomy weighted overall accuracy ARENOSOL 100 Regosol 0,9 CHERNOZEM 100 Vertisol 0,7 CHERNOZEM 100 Vertisol 0,7 CHERNOZEM 100 Chernozem/ 1 CHERNOZEM 100 Chernozem/ 1 CALCISOL ARENOSOL CHERNOZEM CALCISOL Arenosol 0,92 Chernozem/ 0,94 CALCISOL 100 Arenosol 0,9 CHERNOZEM 100 Chernozem/ 1 REGOSOL 100 Hydromorphic 0 KASTANOZEM CALCISOL PHAEOZEM Chernozem/ 0,86 In some cases, where the disturbed material did not let us recognizing the diagnostic features, which were important for the classification, the existence or lacking of them was assumed based on the pit description. At the end a table was compiled with five observation and all diagnostic properties, features, horizons and material have listed for the observations. Based on the five observations, a table with the RSG classes and diagnostics were listed with an appropriate proportion rounded up to 20 percents, like 20, 40, and 100 percent. 100 percent was given for a certain diagnostic, when it could be found in all observations, 40 percent is given when 2 out of the five showed the certain feature. The RSG column lists all RSG observed in the site having the proportion list as well,

3 where the proportions are rounded in the same way as for the diagnostics and sums up to 100 percent to a site. Here only the RSG class is validated (Table 1.) 2.4 Site selection methodology 43 sites have been selected randomly for the country. These sites had to be moved to the closest pixel centers and checked for accessibility, potential disturbance or other restricting factors. Technosols were not included into our classification, and therefore potentially disturbed sites had to be skipped from the dataset. The whole site optimization procedure was programmed in ArcGIS, no personal bias could have a significant impact on the site selection. Required input data: - 45 random generated sampling points by ArcGIS random point generator (Fig. 1) - a result raster of the e-soter Central European window, in this case the WRB based Reference Soil Groups (RSG) raster (Fig. 1) - vector-based GIS databases of the settlements, road and railroad networks, water bodies, nature conservation and other protected areas of the country Theoretical steps of the optimization process First, a 5 pixel circle shape neighborhood around the selected point was selected for further processing as potential sampling pixels. Because neither the pits, nor the auger sites should be within settlements or on roads, railroads, or any similar locations or even close to them, a 50 meter limit was set as minimum distance from the lines or polygons symbolizing them in the vector databases. This limit was increased to 150 meter because the auger sites are 100 meter far from the profile pit, so to keep the minimum 50 meter distance in the case of the auger sites, the pit should be at least 150 meter far from the forbidden areas. Every points falling within a distance of 150 meter from roads, railroads, or settlements were deleted from the possible sampling points, just like the points that were closer to the water bodies, or protected areas then this limit. At the end an accessibility test was performed on the data. A 500 meter maximum allowed distance was set up from the closest road to make sure that the field sampling group does not have to spend too much time on approaching the points and transport the gears there. These two steps of filtering resulted a certain number of potentially selectable pixels. The last step was to select the closest pixel centre to the original randomly selected point. 2.5 Validation In this study only the RSG layer was validated using the validation dataset. The main goal was to demonstrate the potential add-on value of the validation dataset and the introduction of the taxonomic adjacency into the validation procedure. Validation of categorical information, like WRB RSG, is a complex problem. Congalton (1991) and Brus et al (2011) reviewed the most common tools and approaches. Taxonomic adjacency or genetic relationship within a certain set of soil forming factors makes a significant difference in the level of misclassification (Phillips (2013). Misclassifying a pixel to a related RSG or to a nonsense RSG does not mean the same level of uncertainty. Minasny et al (2009) has published an approach to quantify the differences between the soil classes by estimating the taxonomic distances for the WRB RSG classes. This approach is very promising to solve the problem of taxonomic adjacency and quantify the taxonomic differences. However, the variables and their weights used to calculate the taxonomic distances are needed to be further refined for a more realistic picture. Therefore, a modified table was developed and used for the calculation based on the WRB defined prefix and suffix list, introducing the 0,5 probability value as well the Minasny et al (2009) approach had only 0 and 1 probabilities. The results indicated that even the introduction of 0.5 probability was not enough, further refinement of the probabilities and the concerned variables are needed in the near future. There are several local aspects acting in the pilot window as well, which has to be taken into consideration and therefore the taxonomic distance based adjacency has been overwritten by expert knowledge based decisions as well (Table 2.). (The best example of them is the Arenosol-Calcisol confusion. Soils in the middle part of Hungary E. Dobos, P. Vadnai, D. Bertóti, & K. Kovács E. Micheli, V. Láng & M. Fuchs A novel approach for validating raster datasets with categorical data. First GlobalSoilMap Conference, October 7-9. Orleans France.

4 are sandy and have a significant amount of secondary carbonate accumulated in them. Therefore, both Calcaric Arenosol or Haplic CALCISOL (Arenic) apply for them, depending on the amount and origin of the secondary carbonate. There was quite intensive discussion on deciding about the RSG even on the field.) These similarities adjacencies have been translated into similarity weights, of 0.9 for very similar classes (or problematic to separate even in the field) and 0,7 for intergrading RSG classes, where one RSG intergrades to a neighboring one and the decision depends on the level of expression of certain properties, like Cambisol- Luvisol, where the significant question is the level of the argic horizon development. The taxonomy weighted overall accuracy is calculated using these weight. In case of a perfect match the value is 1, for not related classes the value is set to 0, very similar classes gets 0,9 value, while related classes gets 0.7. The final value for a site with 5 observations is the sum of these values multiplied by their proportion within the site. For example the last line of the Table 1 is calculated as 0,4*1+0,4*0,7+0,2*0,9 =0,86 ( is a perfect match with a weight of 1; Calcisol is only related, therefore gets 0,7 weight for the 40 percent coverage, while Phaeozem is very similar with a weight of 0,9 for the 20 percent share). These values are summed up and divided by the number of observations to calculate the final taxonomy weighted overall accuracy. Table 2. Taxonomic similarity matrix Similarity factor group RSG classes 0,9 0.7 Histosols - - Leptosols - Umbrisols Hydromorphic soils, Vertisols Solonetz Chernozem/, Phaeozem Salt effected Vertisol Hydromorphic, Fluvisol Hydromorphic soils - Fluvisol, Chernozem/, Phaeozem, Vertisol, Salt effected Chernozems/ Vertisols, Solonetz, Phaeozem Hydromorphic, Calcisol Chernozem/ Vertisols, Solonetz, Phaeozems Hydromorphic Calcisols Arenosol Chernozem/, Regosol Alisols Luvisol - Luvisols Alisol Cambisol Arenosols Calcisol, Regosol - Cambisols - Regosol, Luvisol Cambisol, Fluvisol, Regosols Arenosol Solonczak Umbrisol - Leptosol

5 Figure 1. The WRB Reference Soil Group class map for Hungary and the validation sites. 3 RESULTS AND DISCUSSION There are several approaches to measure map quality. Datasets derived from remote sensing images are representing average values of certain properties or dominant classes of categorical variables for the pixel area. These maps are on block support. Their content is often very heterogeneous, therefore naming them by the dominant or the average classes often misleading. However, no better solution has been found so far. Soil associations were used to attribute soil polygons in the traditional maps, but the usability of those were often limited due to the lack of spatial definition. Fuzzy membership is a similar approach with similar limitations of their use. Therefore dominant classes are still used and are very popular among the users, even if this approach is far from perfect. Dominant soil types indicate the soil forming environment and a soil scientist can understand the acting factors and the potential associated soil types. That is why these maps were used successfully for human interpretations. However, the GIS world of modelers are different, it is often difficult to integrate this add-on knowledge to the pure polygons or pixels. Therefore the appropriate characterization of the pixel accuracy and purity is crucial for the users. The validation table is also an appropriate tool for descriptive characterization of the dataset. Table 3 shows that number of RSG classes observed within the pixel based on the 5 observations. It clearly shows, that the majority more than half of the pixels are more or less pure. 56 percent of the validation pixels have one RSG class for all observation. 28% has two RSG classes, but even here, out of the 12 sites, 10 has percent shares of the two RSG classes, showing that one RSG is still dominates the pixel. Only 16 percent show real heterogeneity, where three RSG classes occur within the pixel. There is no any pixel found E. Dobos, P. Vadnai, D. Bertóti, & K. Kovács E. Micheli, V. Láng & M. Fuchs A novel approach for validating raster datasets with categorical data. First GlobalSoilMap Conference, October 7-9. Orleans France.

6 with 4 or 5 RSG classes, which suggest the conclusion of having 3 RSG classes as maximum occurring within a 20 hectars area. The results are based on a random sampling approach, therefore this distribution can be taken as a typical soil heterogeneity status within a 450 meter pixel size for the Hungarian soil forming environment. Other kind of soil forming environment may have different results. Table 3. The purity of the observation sites No. of RSG No. of sites Percent of all , , , The validation results show similar trend. Accuracy has been measured in two ways both cases the overall accuracies were measured. The first approach, where the overall accuracy was measured based on the match between the predicted and dominant RSG of the validation dataset, resulted a 54,7% overall accuracy. the second approach, where the taxonomy weighted overall accuracy was calculated resulted a slightly better results of 55,8 percent. This small different between the two approaches was not surprising, because almost 80 percent of all validation pixels were at least 80 percent, but mostly 100 percent belonging to the same RSG class, so not much difference could occur in the calculation due to the refined procedure accounting for the compositional heterogeneity of the pixels. However, it has to be noted here, that this approach is just a demonstration, several details has to be worked out to increase the sensitivity of the approach. The similarity factors are very arbitrariliy defined, a better approach can be developed based on the taxonomic distance calculation and the distribution of this values in the taxonomic space. 4 CONCLUSIONS Validation of pixel based categorical data, especially the soil classification categories are difficult for several reasons. Pixels are block supported spatial elements with often one value assigned to, average values of quantitative properties or dominant classes. In reality, pixels with relatively large size, large 500 meter resolutions, almost never represent a pure area, there is a significant level of heterogeneity behind. Traditional datasets handled this problem with using soil associations. Is this heterogeneity within 450 meter pixel really that much to worry about? The results proved that almost 80 percet of the pixels are quite homogeneous as far as the RSG is considered only. That is proved by the accuracy indicators as well. However, it may happen that areas with much more variable soil forming environment show different results. Soil taxomonic adjacency is a crucial factor for characterizing the accuracy of a dataset. The level of similarities has to be quantified in order to integrate this information into the accuracy assessment algorithms. Taxonomic distance is the most promising approach to express these similarities, but the variables and their probability of occurrence within the RSG classes have to be refined. It was also concluded, that local knowledge is still needed to revise the similarity matrix and adjust it to the local taxonomic and environmental settings. 5 ACKNOWLEDGEMENT Our work has been supported by FP7 project Regional pilot platform as EU contribution to a Global Soil Observing System Grant agreement no.: , financed by the European Commission ; by the Hungarian National Scientific Research Foundation (OTKA, Grant No. K105167); by the "Validation of the Central European Soil database" Strategic Grant of the Visegrád Fund, No , and by the BONUS-HU Grant No. OMFB-01251/2009 and by the Excellent Research Faculty Grant of the Hungarian Ministry of Human Resources (registration no.: /2013/TUDPOL).

7 6 REFERENCES Brus, D.J., Kempen B.,& Heuvelink G.B.M Sampling for validation of digital soil mapping. European Journal of Soil Science. 62, Congalton, R.G A review of assessing the accuracy of classifications of remotely sensed data. Remote sensing of Enviroment. 37, Dobos, E., Seres, A., Vadnai, P Az e-soter digitális talajtérképezés módszertana. Geográfia, V. Magyar Földrajzi Konferencia. Pécs november 4-6. Dobos, E., Seres, A., Vadnai, P., Micheli, E., Fuchs, M., Láng, V., Bertóti, R.D., & Kovács, K Soil parent material delineation using MODIS and SRTM data. Hungarian Geographical Bulletin. Vol 62. No Dobos, E., Vadnai, P., Micheli, E., Láng, V., Fuchs, M., Seres A Új generációs nemzetközi talajtérképek készítése, az e- SOTER módszertan. Térinformatikai Konferencia. Debrecen május IUSS Working Group WRB 'World reference base for soil resources 2006, update 2007, 2nd ed.' World Soil Resources Reports 103. (FAO: Rome) Lagacherie, P., McBratney, A. B., & Voltz, M., (eds.) Digital soil mapping: an introductory perspective. Amsterdam: Elsevier. p ISBN McBratney, A.B.; M.L. Mendonça Santos, B. Minasny, On digital soil mapping. Geoderma (Elsevier B.V., Amsterdam) 117 (1 2): Minasny, B., & McBratney, A. B Incorporating taxonomic distance into spatial prediction and digital mapping of soil classes, Geoderma New York. ISBN Phillips, J.D Evaluating taxonomic adjacency as a source of soil map uncertainty. European Journal of Soil Science. 64, E. Dobos, P. Vadnai, D. Bertóti, & K. Kovács E. Micheli, V. Láng & M. Fuchs A novel approach for validating raster datasets with categorical data. First GlobalSoilMap Conference, October 7-9. Orleans France.