PREDICTION OF SOIL ORGANIC CARBON CONTENT BY SPECTROSCOPY AT EUROPEAN SCALE USING A LOCAL PARTIAL LEAST SQUARE REGRESSION APPROACH

Transcription

1 Proceedings of the 13 th International Conference on Environmental Science and Technology Athens, Greece, 5-7 September 2013 PREDICTION OF SOIL ORGANIC CARBON CONTENT BY SPECTROSCOPY AT EUROPEAN SCALE USING A LOCAL PARTIAL LEAST SQUARE REGRESSION APPROACH MARCO NOCITA 1, ANTOINE STEVENS 2, GERGELY TOTH 1, BAS VAN WESEMAEL 2 AND LUCA MONTANARELLA 1 1 Joint Research Centre - European Commission, Italy; 2 Universitè Catholique de Louvain Corresponding author: marco.nocita@jrc.ec.europa.eu ABSTRACT Due to the large spatial variation of soil organic carbon (SOC) content, assessing the current state of SOC for large areas is costly and time consuming. Visible and Near Infrared Diffuse Reflectance Spectroscopy (Vis-NIR DRS) is a fast and cheap tool for measuring SOC based on empirical equations and spectral libraries. While the approach has been demonstrated to yield accurate predictions for databases containing samples belonging to soils with similar characteristics such as mineralogy, texture, iron and CaCO 3 content, spectroscopic calibrations have been less successful when applied to large and diverse soil spectral libraries. About 20,000 samples collected all over the European Union were analyzed for physical and chemical properties, and scanned with a Vis-NIR spectrometer in a single laboratory. We implemented a modified local partial least square regression approach that, in addition to the spectra, uses other soil covariates (geographical coordinates, texture) for predicting the SOC content. The results showed good prediction ability for mineral soils under cropland (RMSE = 3.6 g C kg -1 ) and grassland (RMSE = 7.2 g C kg -1 ). Predictions of mineral soils under woodland (RMSE = 11.9 g C kg -1 ) and organic soils (RMSE= 51.1 g C kg - 1 ) were less accurate. The best results were obtained when sand content was used as covariate. Keywords: soil organic carbon, Vis-NIR spectroscopy, LUCAS spectral library, local approach, European scale 1. INTRODUCTION Assessing the spatial variation of soil organic carbon (SOC) requires a sampling density that can be costly and time consuming. Visible and Near Infrared Diffuse Reflectance Spectroscopy (Vis-NIR DRS) has been shown to be a fast and cheap tool to infer SOC at local/field scale (Islam et al., 2003). Vis-NIR DRS was first applied for soil analysis in the 80 s and showed that the SOC content, amongst other properties, can be accurately predicted (Viscarra-Rossel et al., 2006). The prediction of soil properties requires the development of a digital library relating spectral with reference data. Such a library should be designed to represent the variation in soil properties of the soil types of interest. Multivariate calibrations are then used to infer properties (Shepherd & Walsh, 2002). The majority of the existent spectral libraries are limited to small areas, and only few large scale soil spectral libraries have been developed so far. Shepherd and Walsh (2002) inferred SOC combining Vis-NIR DRS and Multivariate Adaptive Regression Splines (MARS) on more than 1000 topsoil samples from eastern and southern Africa. Brown et al. (2006) applied Vis-NIR DRS to characterize various soil properties of more than 4500 samples from the United States (3768), Africa (125), Asia (104), South and Central America (75), and Europe (112). Due to (i) lack of uniformity, as constructed in several laboratories, with different instruments and protocols, and (ii) relatively low sampling density, these spectral libraries did not produce the

2 same level of accuracy achieved at small scale. The scope of our research was to predict SOC content at European scale using the LUCAS spectral library coupled with a local partial least square regression (LPLS) as multivariate regression method. 2. MATERIALS AND METHODS The LUCAS (Land Use/Cover Area frame Statistical Survey) sampling campaign was built collecting approximately 20,000 random soil samples distributed all over Europe. Thirteen soil properties, including SOC and Vis-NIR DRS, were analyzed in the same laboratory. Several pre-processing techniques were applied: transformation of absorbance (A) spectra into reflectance (R=(1/10 A )) and continuum removal, standard normal variate (SNV), Savitzky-Golay smoothing with a window size of 50 and 2nd order polynomial, first and second derivatives. The dataset was divided in mineral and organic soil materials due to the different characteristic of the spectral response of the two materials. Mineral soils were split in cropland, grassland, and woodland soils according to land cover classes of LUCAS database. For each subset a training (70%) and test (30%) sets were created to calibrate and validate the SOC prediction models using the Kennard-Stone algorithm (Kennard and Stone, 1969). Modified local partial least square regression: to predict the SOC content (Y) with the spectral matrix (X), we implemented a LPLS approach. In principle, The LOCAL algorithm was developed to select from a wide spectral library samples that are spectrally similar to the unknown sample. The predictors are then used to calibrate a specific prediction model for the unknown sample. In the LOCAL algorithm the selection of the predicting neighbours is controlled by the correlation coefficient between the spectra of the predictors and the spectrum of the unknown sample (Shenk et al., 1997). The algorithm used in this study proposed a modification of the selecting process of the predictors adding sand content or geographical coordinates to spectral similarity as co-variables to calculate the correlation coefficient between the predictors and the unknown sample. The parameters of the LPLS algorithm such as the number of neighbours and the distance metrics were globally optimized via a grid search approach (brute force). The LPLS algorithm was successively run over the test of (i) the following k-neighbours : 50, 100, 250, 500, 750; (ii) spectral distance metrics: we tested the Euclidean, mahalanobis, correlation, cosine and partial least square (pls) distances; and (iii) the distance weights: we tested different combinations of weights, where let w the weight vector with values {w1,w2,w3} corresponding to the spectral, geographical and sand distance vectors, respectively, we tested w = {1,0,0} (spectral distance only; spc), w = {1/2,1/2,0}(spectral and geographical distances; spc+geo), w = {1/2,0,1/2} (spectral and sand distances; spc+sand), and w = {1/3,1/3,1/3} (spectral, geographical, and sand distances; spc+sand+geo). The coefficient of determination (R 2 ) between measured and predicted values, the root mean square error (RMSE) of the test set, the ratio of prediction to deviation (RPD), and the ratio of performance to interquartile range (RPIQ) were used to evaluate and compare the spectroscopic models. 3. RESULTS AND DISCUSSION The distribution of points per country was not proportional to the total country area. For example Italy was characterized by a sampling density much lower than France. The SOC content was abundant in organic soils, low in cropland and grassland, and moderate in woodland soils (Fig.1). For all the 3 classes of mineral soils the minimum and maximum SOC contents were similar while the mean SOC was higher in woodland soils (46.4 g C kg -1 ) than

3 grassland (33.4 g C kg -1 ) and cropland (17.6 g C kg -1 ) soils. The organic soils were characterized by a SOC content range between 156 and 586 g C kg -1, but few samples covering the range, compared to the samples density of mineral soil. Woodland soils were also characterized by unfavorable ratio samples/soc content, while cropland and grassland presented a narrow SOC range and a higher samples density. Fig.1. Histogram of SOC content of LUCAS dataset, for cropland, grassland, woodland and organic soil. 3.1 SOC predictions using the LPLS For cropland, grassland, and woodland soils, models with 250 predictors yielded the best results, while the use of 500 predictors was the best option for organic soil (Fig.2). For cropland soils a decrease of mean and lowest RMSE were registered passing from 50 to 250 neighbours. Organic soils showed a trend similar to cropland soils with an accuracy decrease from 50 to 250 neighbours. The important effect of predictors number on organic soils confirmed the lower prediction ability of soil spectra for organic soils and the need to have a higher sampling density to build a calibration set able to cope with the spectral variation. The use of spectral distance (spc), or the combination of spectra with sand (spc+sand), geographical coordinates (spc+geo), and both sand and geographical coordinates (spc+sand+geo), in the selection of the LPLS predicting neighbours caused differences of accuracy among the datasets (Fig.3). While for cropland soils the difference among the four options was not important, grassland and woodland soils showed higher RMSE gaps for the different combinations. The SOC predictions for organic soils were in average better when the selection of the nearest neighbours was based just on spectra than adding geographical coordinates (sand was not tested as covariate due to the irrelevance of sand in organic soils). The combination spectra + sand gave the lowest RMSE and the highest RPIQ and

4 RPD for all mineral soils. Although sand is not a spectral chromophore, it produces a higher light scattering effect, which determines an amplification of the response of SOC absorption features to reflectance (Stenberg et al., 2010), and consequently a higher predictive ability of the LPLS. Fig.2. Boxplot of the RMSE of all the models as a function of the number of k-neighbours for cropland, grassland, woodland and organic soils. The RMSE registered for soils under cropland ( g C kg-1) was much lower than the RMSE of soils under grassland ( g C kg-1) and woodland ( g C kg-1) (Fig.4). This was due to the weaker calibrations caused by the lower sampling density, and the higher SOC range of grassland and woodland soils, which presented a complexity of the soil matrix more important than the more disturbed and easier to sample cropland soils. These results compare well with previous studies conducted on large spectral libraries (Shepherd and Walsh, 2002; Brown et al, 2006). Organic soils were predicted with a RMSE of more than 50 g C kg-1, but the RPIQ of 3.6 was the highest recorded, showing a very good prediction ability of the model. The improvement of the SOC predictions for organic soil should pass by the increase of the sampling density, with the awareness that the level of accuracy will never be the same observed for mineral soils, due to the saturation of SOM absorption features beyond 120 g C kg-1.

5 Fig. 3. Boxplot of the RMSE obtained using spectra (spc), spectra + sand (spc+sand), spectra +geographic coordinates (spc+geo), and spectra + sand + geographic coordinates (spc+sand+geo) to select the predicting neighbours of the unknown sample in the LPLS algorithm. Fig. 4. Predicted vs. observed SOC values of the best models using spectral distance (spc) or adding sand content (spc+sand) for cropland, grassland, woodland and organic soils

6 4. CONCLUSIONS This research represents a novelty in the Vis-NIR diffuse reflectance spectroscopy scenario due to the statistical approach, culminated in the development of a new local PLS regression algorithm and the analyzed data, characterized by the largest continental harmonized spectral library. The combination of Vis-NIR diffuse reflectance spectroscopy and a modified local partial least squared regression algorithm (LPLS) resulted in very good predictions for cropland soils (RMSE: 3.6 g C kg -1 ), while grassland, woodland and organic soils, with RMSE of 7.2, 11.9, and 51.1 g C kg -1, respectively, got reduced accuracies. For all the mineral soils the best predictions were achieved when sand content was used as covariable in the selection of the predicting neighbours. This study showed that the local regression approach might consistently improve the prediction capacity of a large scale soil spectral library due to its ability to better exploit the spectral variability caused by the spatial variation of a large population of soils. REFERENCES 1. Brown, D. J., Shepherd, K. D., Walsh, M. G., Dewayne Mays, M., & Reinsch, T. G. (2006). Global soil characterization with VNIR diffuse reflectance spectroscopy. Geoderma, 132(3 4), doi: /j.geoderma Islam, K., Singh, B., & McBratney, A. (2003). Simultaneous estimation of several soil properties by ultra-violet, visible, and near- infrared reflectance spectroscopy. Soil Research, 41(6), Kennard, R. W., & Stone, L. A. (1969). Computer Aided Design of Experiments. Technometrics, 11(1), doi: / Shenk, J., Westerhaus, M., & Berzaghi, P. (1997). Investigation of a LOCAL calibration procedure for near infrared instruments. Journal of Near Infrared Spectroscopy, 5(1), doi: /jnirs Shepherd, K. D., & Walsh, M. G. (2002). Development of Reflectance Spectral Libraries for Characterization of Soil Properties. Soil Sci. Soc. Am. J., 66(3), Stenberg, B., Viscarra Rossel, R. A., Mouazen, A. M., & Wetterlind, J. (2010). Visible and Near Infrared Spectroscopy in Soil Science. In Advances in Agronomy (Vol. 107, pp ). Elsevier. 7. Viscarra Rossel, R. A., Walvoort, D. J. J., McBratney, A. B., Janik, L. J., & Skjemstad, J. O. (2006). Visible, near infrared, mid infrared or combined diffuse reflectance spectroscopy for simultaneous assessment of various soil properties. Geoderma, 131(1 2), DOI: /j.geoderma