Transactions on Information and Communications Technologies vol 16, 1996 WIT Press, ISSN

Transcription

1 Artificial Neural Networks and Geostatistics for Environmental Mapping Kanevsky M. (), Arutyunyan R. (), Bolshov L. (), Demyanov V. (), Savelieva E. (), Haas T. (2), Maignan M.(3) () Institute of Nuclear Safety, B. Tulskaya 52, 39, Moscow, Russia. Fax: (095) (2) University of Wisconsin-Milwaukee, School of Business and Administration (3) Lausanne University, Switzerland Abstract The report presents several approaches and models that were used for spatial estimations of radioactively contaminated territories after Chernobyl nuclear accident followed by large scale environmental consequences: well known traditional deterministic models (inverse distance weighted algorithms, multiquadric equations), recently developed geostatistical models (Moving Window Regression Residual Kriging/CoKriging), and artificial neural networks (ANN). The models used differ in amount of preliminary analysis, levels of complexity and assumptions about the data behaviour. Introduction The problem of analysis, processing and presentation of spatially distributed and time dependent environmental data is actual and important both from scientific and practical point of view. Analysis of real data stimulates development of new adaptive techniques and algorithms. Real case studies are important in decision making process. At present time there are many different approaches to understand and solve such problems [,2]. It should be noted that methods and models to be used depend on quality and quantity of data, available knowledge, final objectives: spatial estimations - prediction and decision-oriented mapping, description of local spatial uncertainties around unknown values, stochastic simulations or preparing alternative, equally probable, high resolution models of the spatial distributions. There is no the best solution for all cases.

2 The present work deals with the analysis of spatially distributed data of radioecological monitoring. 0 years after Chernobyl nuclear accident with large scale environmental consequences this is still an important and vital problem in Russia, Belarus and Ukraine [3]. On April 26, 986 a serious nuclear accident occurred at the 4th reactor of Chernobyl Nuclear Power Plant. It was followed by an explosion and release of radioactive materials which contaminated vast territories in Eastern, Western and Northern Europe. The accident caused significant long term consequences concerning the environment, population, economics, politics, radiology, agriculture, natural resources, and many other fields of human activity. After the accident there have been collected oceans of spatially and time dependent data to be analysed for decision making [3]. The present work deals with an essential part of spatial data analysis - spatial interpolations/estimations/predictions - of soil contamination by Chernobyl radionuclides at regional scales (about 00 km). The main features of the data are as follows: high variability at different scales, uncertainty originating from measurement errors and small scale variability, nonhomogeneous monitoring/sampling networks, anisotropic wet and dry deposition, long-range trends, etc. Depending on final goals the whole spectrum of modern possibilities of spatial data analysis have to be used: spatial interpolations, nonparametric description of uncertainties, stochastic simulations, fractal theory, artificial intelligence algorithms, etc. One of the main objectives of the work is to use several spatial estimators and compare their advantages and disadvantages: widely used traditional deterministic models - inverse distance power weighting; multiquadric equations; geostatistical approach, especially recently developed Mowing Window Regression Residual Kriging/Cokriging - MWRRK/MWRRCK - models; artificial neural networks with their possibilities of function mapping. The models used differ in amount of preliminary analysis, levels of complexity and assumptions about the data behaviour. Traditional interpolators usually depend only on a few model-dependent parameters (e.g., search radius, number of neighbours). Geostatistical approaches (both estimations and simulations) are based on methodology, which explicitly takes into account spatial correlation structures described by covariograms and semivariograms and their manual or automatic modelling (usually fitting to known theoretical models). This stage of study needs a lot of time and geostatistical expert knowledge. Geostatistical approach provides models' error variance and uncertainty of the estimates. From a large variety of geostatistical predictors (family of kriging models) MWRRK/MWRRCK were chosen for spatial estimations. MWRRK is unbiased non-linear predictor which is used to treat complex non-stationary data applying moving window technique. It includes

3 linear and non-linear trend analysis. In a case of more then one variable coestimation is carried out (MWRRCK model). It is important to note that within the framework of geostatistics there is a well developed methodology of analysis, processing and presentation of spatially distribute data starting from exploratory data analysis, through structural analysis (variography), cross-validation up to prediction mapping. This approach is taken into account during data analysis with the help of artificial neural networks as well. At present time neural networks are used for the solution of different environmental problems [ 4,5,6,7,]. It was shown that ANN can be used for the solution of many applied problems. Despite of these and other preliminary successful results it seems that much more work have to done both for the development of ANN implementation and interpretability of results. 2 Description of data Two data sets have been used: data on soil contamination by CS37 in Gomel region (Belarus) and soil contamination be CS37 and SR90 in the most contaminated region of Russia (south-west Part of Briansk region). The main characteristic of the region and basic descriptive statistics of the data are as follows:. Gomel region. Geographical scale = 30 km (East-West) x 230 km (North- South). CS37 data samples range from 0.04 to 60.4 Ci/sq.km. Mean value = 4.99; median value =.92; skewness = 3.07; kurtosis = Briansk region. Geographical scale = 75 km (East-West) x 25 km (North- South). SR90 data samples range from 0.08 to.36 Ci/sq.km. Mean value = 0.29, median value = 0.23; skewness = 2.02; kurtosis = 8.2. Exploratory data analysis includes process garbage in - garbage out, analysis of outliers, paying attention to data magnitude and variability, minimising nonlinearities etc. Despite good performance of neural networks on non-linear problems, minimising nonlinearities leads to faster training, less complicated network, and better performance [8,9]. In case of spatially distributed data there is another problem with spatial clustering - as a rule data are sampled/collected on a nonhomogeneous (nonregular) monitoring networks. Usually in order to obtain representative descriptive (univariate) statistics usually different methods of spatial declustering are used [2]. Basic structure of the Gomel monitoring network is presented in figure. by using triangulation. Analysing data with the help of artificial neural network it is desirable to have three data sets: training data set, testing data set, and validation data set. We have used only two data sets: one was used for network training and the other one was used as independent data set for validation of the trained network. In order to split original data set we have used spatial declustering procedure - random selection method: original data have been covered by a regular grid; from each grid cell we have selected data randomly

4 to form the training data set. The left data were used as a validation data set. So, for training procedure we had more homogeneous and representative data set. In case of Gomel data bases about 20% of data were used as a validation data set and 80 % as a training data set. Other methods for splitting data can be proposed but it seems reasonable to take into account spatial nature of data and their clustering. The same procedure was carried out with the Briansk data. Figure : Monitoring network of the Gomel region, CS37. In case of ANN estimations original data were log-transformed and then scaled to the interval [0.,0.9] (ANN s transfer function ranges from 0 to ). 3 Spatial data analysis 3. Traditional interpolations There are many traditional interpolation techniques within deterministic approach. They usually do not require as complicated preliminary analysis as there is in geostatistical approach. Traditional interpolators usually compute quick results and do not go into modelling spatial data structure. Unlike geostatistical methods, they assume a determined spatial relationship between the data, e.g. inverse distance methods give dependence on separation distance between the data and estimation points, which is not supported by any observations or speculations. 3.2 Multiquadric Equations

5 Multiquadric equations (MQE) is a method of deriving continuous mathematical functions representing a topographic surface from a set of discrete elevation control points [3]. The resultant predictive equations model the surface as a summation of individual conical surfaces originated at the control points. The surface functions have the form: N Z = c d + K i= i where N - number of control points, d - distance between the ith control point and point to be estimated, - coefficient for ith control points, K - arbitrary constant controlling steepness of conical surfaces. The coefficients are obtained by solving the following equations: N j= cd j ij i = Z where Z i - surface value at the ith control point, d ij - distance between the ith and the jth control points. The function value in unsampled point can be calculated from the following relations: N Z = c q x, y, x, y i= i ( ) i i i Circular hyperboloid is used for the quadric form: ( ) ( ) ( ) qx, y, xy, = x x+ y y+ C i i i i where C is an arbitrary parameter, in a case of C=0 the surfaces makes a cone. Originally this method supposed the use of all sample points for estimations in the whole area. Usually we have a large amount of data points (hundreds) and it is impossible to solve such large system of linear equations. So we propose to use only the nearest (0-30) sample points for each estimation point and solve the system only if the group of neighbours changes. 3.3 Inverse Distance Method Inverse Distance Interpolation is the way to give more weight to the closest samples and less to those that are farthest. The weights can be calculated inversely proportional to any power of any distance. Z N * i = = N i = Z p d i d p i i

6 Different choices of the exponent p result in different estimates. As p approaches 0 and the weights become more similar, the inverse distance estimate approaches the simple average of nearby sample values. As p approaches infinity, the inverse distance estimate approaches the polygonal estimate, giving all the weight to the closest sample. In the present work p=2 was chosen, which made inverse distance squared method. The two described above methods were applied to CS37 contamination in Gomel region, Belarus. The contour maps of the estimates are presented in figures 2, 3. Figure 2: Inverse distance squared estimates of CS37 distribution in Gomel region, Belarus. Figure 3: Multiquadric equations estimates of CS37 distribution in Gomel region, Belarus.

7 3.4 Geostatistical spatial estimations Geostatistics, in general, is a collection of deterministic and statistical tools aimed at understanding and modelling spatial variability. The basic paradigm of predictive statistics is to characterise any unsampled (unknown) value z as a random variable Z, the probability distribution of which characterises the uncertainty about z [2]. It is assumed that data are realisations of a random field. In order to use geostatistics we have to determine correlation structure of this field. The most famous geostatistical model is kriging. Kriging belongs to the class of BLUE (best linear unbiased estimator) estimators. The most frequently used measure of spatial continuity (correlation structure) is a Semivariogram. It is a second order moment and in geostatistics is used under intrinsic hypothesis [,2]. If Z(x) is a random function and Z(x i ) are realisations in space, then experimental semivariogram for the set of N samples is: N( h) γ ( h) = ( Zx ( i) Zx ( i + h)) 2N( h) i= where h is a separation distance between the points. Experimental semivariogram is then modelled by using positively defined theoretical functions. One of the common functions is a Spherical model with a nugget: γ ( h) h h c + c 3 3 if h a = a 0 a 2 2 c0 + c if h > a where c 0 is a nugget - a random component in the data distribution;(c 0 + c) is a sill, range a is the effective distance at which correlation exist. Exploratory variography was carried out by using VarioWin software [4]. Exploring spatial correlation includes building variogram surfaces, which show global spatial structure of correlation - variogram surface in 2D = γ(hx, hy). According to variogram surface search radius and search ellipse direction are chosen in estimation procedure. It is important not only while using geostatistical predictors but also applying traditional interpolation methods. The last significantly depend on input model-dependent parameters, thus such preliminary analysis is necessary both to understand spatial continuity and to prepare valid input. A variogram surface for CS37 in Gomel region is shown in figure 4. It features long range (70 km) correlation and anisotropy in NW-SE direction due to spotty structure of the contamination. In a case of multivariate analysis, when there is more then one spatially distributed variable, cross-variogram is used to explore their joint spatial 2

8 correlation. An example of cross variogram surface for SR90 and CS37 is presented in figure 5. It shows anysotropic structure different from the ones that SR90 and CS37 have themselves. Figure 4: Variogram surface for CS37 distribution in Gomel region, Belarus. Figure 5: Variogram surface for SR90 distribution in Briansk region, Belarus.

9 3.3 Moving Window Regression Residual Kriging and Cokriging Moving Window Regression Residual Kriging (MWRRK) and Cokriging (MWRRCK) [5] are geostatistical predictors recently developed to overcome non-stationary and trend problems, which raise while using global correlation structure models in geostatistical predictors like ordinary kriging (OK)[,2]. Moving window technique implies choosing local areas (windows) for variography and estimation procedures. Let Y= ( Y, Y2) be the partitioned vector containing the within-window observed random variables from both processes located at x,, x n ( w ) and z,, z n ( w, ) respectively. Within this window, the model for Y 2 ( x 0 ) and Y 2 ( x 0 ) consists of a spatial trend component plus an error or stochastic component: Y ( x ) = µ x, β, Y ( x ) + R ( x ) ( ) ( ) Y2( x0) = µ 2 x0, β2 + R2( x0) where Y 2 ( x 0 ) is the prediction of the co-variable at location x 0. The vectors µ () and R are variable 's trend model values at the locations x,, x n ( w, ) and the associated vector of variable 's residual process random variables, respectively. Define µ 2( β 2 ) and R 2 similarly for the co-variable at z,, z n ( w. ) The vectors β and β 2 contain the linear parameters defining the 2 mean functions µ and µ 2 respectively [5]. Preliminary local (within the window) trend is modelled using polynomial models, residuals are used in further variography and kriging procedure. The process of building and modelling experimental variogram within each window is automated using gls fitting. Final trend and residual estimates are summarised and followed by prediction error of the residuals. Such methodology has significant advantages over estimators which concern only global distribution characteristics (ordinary kriging). MWRRK method allow to avoid bias of cross-validation residuals. In a case of more then one variable (correlated under - and oversampled sampled variables) cokriging of undersampled variable substitutes kriging within window estimation. Cokriging usually gives better results and reduces estimation error. MWRRK was used to predict CS37 spatial distribution in Gomel region of Belarus. The chosen area has a complex spotty contamination with a number of outliers. There are three spots of high contamination (up to 70 Ci/sq.km) in the South, north-west and north-east, and low (below 5 Ci/sq.km) polluted central regions. In the work [6] there were presented cross-validation results and fuzzy isolines which characterised uncertainty of the estimates. Important feature of MWRRK/CK is that their estimation errors do not depend only on sample data locations and cluster structure of the monitoring network, but also

10 demonstrate variability of the data. In figures 6,7 there are contour maps of MWRRK predictions and prediction errors for CS37 distribution in Gomel region, Belarus. Figure 6: MWRRK CS37 (Ci/sq.km) estimates in Gomel region, Belarus. MWRRCK was applied to SR90 distribution in Briansk region - the most contaminated region in Russia. There are fewer SR90 samples (286) then CS37 samples (680) due to measurement s cost reasons. SR90 and CS37 are correlated with correlation coefficient equals Thus cokriging approach allowed to use supplementary CS37 samples to clarify SR90 spatial distribution and move from interpolation to extrapolation in the areas where SR90 was not sampled. Figure. 7: MWRRK CS37 (Ci/sq.km) estimation errors in Gomel region, Belarus.

11 Spatial peculiarities of local variogram nugget distribution are of a separate interest as they present random and correlated components of spatial data distribution, and thus influence the estimation error distribution (see fig. 7,8). Figure. 8: Spatial distribution of nugget of local variograms, CS37, Gomel region, Belarus. Figure 9: Scatterplots of sorted standard cross-validation residuals for MWRRCK estimates of SR90 in Briansk region, Russia. Cross-validation is an important step before the estimation procedure is carried out on the desirable grid. Cross-validation (or live-one-out) estimates

12 allow to choose models parameters to get better results. In MWRRK/CK case the adjusted parameter is the window size (fraction of the data to be considered in computing each local prediction). Analysing cross-validation residuals (measured value - estimated) it is possible to discover and get rid of bias. Zscore and Half Zscore plots in fig. 9 show absence of bias in MWRRCK cross-validation residuals - the scatterplot is very close to the line x=y. MWRRCK estimates and estimation errors for SR90 in Briansk region are presented in figure 0. )LJXUH%ULDQVNUHJLRQ65&LVTNP0RYLQJ:LQGRZ5HJUHVVLRQ 5HVLGXDO&RNULJLQJ0:55&.HVWLPDWHVOHIWDQGHVWLPDWLRQHUURUV ULJKW 3.4 Artificial neural networks A variety of artificial intelligence methods are widely used nowadays in a great number of basic and applied sciences [4-2]. Artificial neural networks are analytical systems that address problems whose solutions have not been explicitly formulated. They estimate a function without a mathematical model of how output depend on input. Neural networks being model-free estimators represent the adaptive technique. They "learn from experience" with sample data. It is well known that ANN with hidden layers solves practical non-linear problems. In the current work multilayered feedforward ANN (FFNN) were used as a spatial estimators. It was rigorously established that FFNN with as few as one hidden layer using arbitrary squashing functions (e.g., logistic, hyperbolic tangent) are capable of approximating any Borel measurable

13 function from one finite dimensional space to another to any desired degree of accuracy, provided sufficiently many hidden units are available (see, for example [0]). In this sense multilayered feedforward networks are a class of universal approximators. As such, failures in applications of the FFNN can be attributed to inadequate training, inadequate number of hidden neurones, or the presence of a stochastic rather than deterministic relation between input and output. Mixed backprop learning algorithms with simulating annealing and genetic algorithms in order to escape from local minima were used [8,9]. Two spatial co-ordinates were used as the input and contamination level was obtained as the output (Figure ). Depending on the complexity of data one or two hidden layers were selected. Choosing an appropriate number of hidden layers is extremely important. Before using a trained network, its competence must be evaluated. Two separate data sets were prepared. One is a training set to train the network, and the other one is a validation data set. After training and validation network can be used for interpolations. The purpose of validation is to provide an unbiased indication of what can be expected from the network when it is applied to the spatial prediction (generalisation). Figure. : Examples of feedforward neural networks used in the study: A - univariate prediction (CS or SR), network [3-2-CS]; B - multivariate predictions (CS & SR), network [4-0-2].

14 FFNN were used within the framework of the approach discussed in [2] and which consists of several important phases. In short, they are as follows: ) Exploratory data analysis, preparing of training and testing data sets; 2) designing network structure - number of hidden layers (always it was or 2 layers) and number of neurones in each hidden layers; 3) training of the network (most part of the ANN work was carried out based on software and algorithms presented in [ ], where details can be found); 4) evaluating performance of the network with the help of different tools like jack-knife, cross-validation, analysis of residuals = (real data - estimates); 5) validation - estimating the FFNN ability to generalise by using validation data set; 6) operation phase - interpolations on a dense grid. Two case studies were analysed: spatial estimations of surface contamination by CS37 in Gomel region (Belarus) and surface contamination by SR90 in Briansk region (Russia). Gomel region is characterised by three spots. FFNN with as few as 5 hidden neurones can learn basic spatial patterns. Spatial prediction map obtained with the help of [0-0-CS] FFNN is presented in Fig.2. Briansk case study for SR90 is discussed below in more detail. Neural networks with different architectures were trained by using training data set. Training is a procedure of minimising mean square error between desired and ANN output. One of the most important question is what was learned by FFNN. We have used simple accuracy tests to check correlation between desired outputs (real data) and neural network response to training data set (Fig.3). Coefficient of correlation is high and equals Figure 2: Gomel region, CS37 (Ci/sq.km). [0-0-CS] ANN interpolation surface.

15 After training have been finished at each point we have residuals Res = (real/desired data - ANN response/estimates). In case of unbiased ANN model mean value of Res E{Res}=0. Histograms along with descriptive statistics of residuals for the two networks are presented in Fig. 4,5. It is evident that both ANN models are unbiased and have symmetric distribution (mean, median and coefficient of skewness are about zero). Figure 3: Briansk region, SR90. Accuracy test: estimates of the training data. Figure 4: Briansk region, SR90. Descriptive statistics of residuals (dataestimates of [5-0-SR] ANN, training data set).

16 Figure 5. Briansk region, SR90. Descriptive statistics of residuals (dataestimates of [5-0-SR] ANN, training data set). Trained neural networks have been validated by sing independent validation data set. Validation data were not used during training. In case of Briansk SR90 data validation data set was about 0 % of training data set. Scatterplots of validation for the several neural network architectures are presented if Fig. 6. Both real and estimated data presented in Fig.6 are scaled. Results of validation test for the deterministic spatial interpolation model (inverse distance squared) is presented in the same figure. It seems that results are acceptable. Figure 6: Briansk region, SR90. ANN and inverse distance squared (INV2) validation test.

17 Figure 7: Briansk region, SR90 (Ci/sq.km). [5-0-SR] ANN spatial estimations. After validation neural networks have been used for prediction mapping. Two estimated surfaces are presented in Figures 7 and 8. Figure 8. Briansk region, SR90 (Ci/sq.km). [5-0-SR] ANN spatial estimations.

18 It was shown [2] that analysis of residuals is an important and informative stage of the study. When FFNN is used as a superregression model residuals should be normally distributed and uncorrelated. In case of spatially correlated residuals it was proposed to do second step based on geostatistical analysis of second-order stationary residuals. This model was named neural network residual kriging (NNRK). Its development was stimulated by MWRRK and MWRRCK models. The first and preliminary results have shown that better results of validation and prediction mapping can be obtained. 4. Conclusion By using real data, traditional interpolators, geostatistical, and ANN models were compared by using cross-validation technique as well as validation data sets. Spatial declustering procedures were used to split original data sets (training data set, validation data set). Capability of ANN depends on network topology and quantity and quality of the data; as a super regression model feedforward ANN can be useful while analysing non-stationary data with a complex trend over the entire area of interest. Simple analysis of residuals after learning is presented and discussed. FFNN having only a few hidden neurones can learn complex spatial patterns of radioecological data. By using validation data sets it was shown that FFNN are as good as traditional interpolators used. The most important advantage of MWRRK/MWRRCK models is their powerful ability to quantify quality of final results (prediction maps+maps of errors). The application of artificial neural networks in environmental and Earth sciences seems to be promising. Opened questions for the future research, besides others, are as follows: analysis of spatially distributed and time dependent data of environmental monitoring, stochastic simulations with the help of ANN, development of mixed ANN & geostatistical models, investigations on adaptive automatic decision-oriented environmental mapping, etc. 5. Acknowledgements The research was supported in part by INTAS grant N: KEY WORDS: Radioactively contaminated territories, artificial neural networks, geostatistical interpolations, moving window regression residual kriging. References []. Cressie N. Statistics for Spatial Data, John Wiley & Sons, New York, 99.

19 [2]. Deutsch C. & Journel A. GSLIB Geostatistical Software Library and User s Guide. Oxford Univ. Press, Oxford, 992 [3]. Karaoglou A., Desmet G., Kelly G.N., Menzel H.G. (eds.) The radiological consequences of the Chernobyl accident. EUR N6544 EN. 92 p., 996. [4]. Wu X. & Zhou Y. Reserve estimation using neural network techniques, Computers & Geosciences, 993, 4, [5]. Dowd P.A. The use of neural networks for spatial simulation, in Geostatistics for the next century (ed. R. Dimitrakopoulos), pp , Kluwer Academic Publishers, Netherlands, 994. [6]. Rogers L.L. & Dowla F.U. Optimization of groundwater remediation using artificial neural networks with parallel solute transport modelling, Water Resources Research, 994, 2, [7]. Rizzo D.M. & Dougherty D.E. Characterisation of aquifer properties using artificial neural networks: Neural kriging, Water Resources Research, 994, 2, [8]. Masters T. Practical Neural Network Recipes in C++, Academic Press, New York, 993. [9]. Masters T. Advanced algorithms for neural networks, John Wiley & Sons, Inc., New York, 995. [0]. Haykin S. Neural Networks. A Comprehensive Foundation, Macmillan College Publishing Company, New York, 994. []. Kanevsky M. Use of artificial neural networks for the spatial interpolations of radioecological data, Izvestija Akademii Nauk. Energetika, 995, No.3, (in Russian). [2]. Kanevsky M., Arutyunyan R., Bolshov L., Demyanov V., and Maignan M. Artificial neural networks and spatial estimations of Chernobyl fallout. in IAMG 95 Annual Conference. Abstracts for Technical Programs. pp , 995. [3]. Hardy R.L. Theory and applications of the multiquadric-bijarmonic equations method. Computers Math. Appl., 990, v.23, pp [4]. Pannatier Y. Exploratory Variography. in Workshops on Statistics of Spatial Processes - Theory and Applications. Bari, Italy, 27-30, 993. [5]. Haas T. C. Multivariate Spatial Prediction in the Presence of Non-linear Trend and Covariance Nonstationarity. Environmetrics, 7(), (in press), 996. [6]. M. Kanevsky, R. Arutyunyan, L. Bolshov, V. Demyanov, T. Haas, L. Savel eva. Environmental Data Analysis. Case Study: Chernobyl Fallout. Book of Abstracts of the 0th International Congress on Mathematical and Computer Modeling and Scientific Computing. Boston, July 5-9, 995, p. 79, 995.