Robust estimation of the external drift and the variogram of spatial data

Transcription

1 Researh Colletion Conferene Paper Robust estimation of the external drift and the variogram of spatial data Author(s): Künsh, Hansruedi; Papritz, Andreas Jürg; Shwierz, Cornelia; Stahel, Werner A. Publiation Date: 2013 Permanent Link: Rights / Liense: In Copyright - Non-Commerial Use Permitted This page was generated automatially upon download from the ETH Zurih Researh Colletion. For more information please onsult the Terms of use. ETH Library

2 ISI 58th World Statistis Congress of the International Statistial Institute, Dublin, August Robust estimation of the external drift and the variogram of spatial data Künsh, Hans R. ETH Zurih, Seminar for Statistis Rämistrasse Zürih, Switzerland hans-ruedi.kuensh@stat.math.ethz.h Papritz, Andreas ETH Zurih, Institute of Terrestrial Eosystems Universitätstrasse Zürih, Switzerland papritz@env.ethz.h Shwierz, Cornelia ETH Zurih, Seminar for Statistis & Institute of Terrestrial Eosystems Rämistrasse Zürih, Switzerland ornelia.shwierz@stat.math.ethz.h Stahel, Werner A. ETH Zurih, Seminar for Statistis Rämistrasse Zürih, Switzerland stahel@stat.math.ethz.h Introdution Most of the geostatistial software tools, available either ommerially in GIS and in statistial software pakages or as open soure software (e.g. R, R Development Core Team, 2011) rely on non-robust algorithms. This is unfortunate, beause outlying observations are rather the rule than the exeption, in partiular in environmental data sets. Outlying observations may results from errors (e.g. in data transription) or from loal perturbations in the proesses that are responsible for a given pattern of spatial variation. As an example, the spatial distribution of some trae metal in the soils of a region may be distorted by emissions of loal anthropogeni soures. Outliers affet the modelling of the large-sale spatial variation, the so-alled external drift or trend, the estimation of the spatial dependene of the residual variation and the preditions by kriging. Identifying outliers manually is umbersome and requires expertise. A better approah is to use robust algorithms that prevent automatially that outlying observations have undue influene. Former studies on robust geostatistis foused on robust estimation of the sample variogram (f. Lark, 2000, for a review), kriging without external drift (Hawkins and Cressie, 1984) and on robust estimation of parameters of a linear model for the external drift (Militino and Ugarte, 1997). Furthermore, Rihardson and Welsh (1995) proposed a robustified version of (restrited) maximum likelihood ([REML) estimation for the variane omponents of a linear mixed model, whih was later used by Marhant and Lark (2007) for robust REML estimation of the variogram. We propose here a novel method for robust REML estimation of the variogram of a Gaussian random field that is possibly ontaminated by independent errors from a long-tailed distribution. Besides robust estimates of the parameters of the external drift and of the variogram, the method also provides robustified kriging

3 ISI 58th World Statistis Congress of the International Statistial Institute, Dublin, August preditions. Theory We use the model Y (s) = x(s) T β+z(s)+ε(s) where x(s) T β is the external drift, Z(s) is a stationary Gaussian field with mean zero and ovariane R(h; σ0 2, α), and ε(s) is an independently distributed error with a probability density funtion proportional to 1/σ exp( ρ(ε/σ)) so that ρ(x) = x 2 /2 gives the ase where ε(s) has a Gaussian distribution. σ 2 is the squared sale parameter of ε and is ommonly alled the nugget, and θ T = (σ0 2, α) are the sill and range parameters of the ovariane funtion (or variogram). For some variogram models θ ontains further parameters. Using the observations, y T = (y(s 1 ), y(s 2 ),..., y(s n )) we want to estimate β, σ 2 and θ and to predit Z(s) for some s where s an be equal to one of the sampled loations s i or different. The starting point for developing our new proedure is the Gaussian log-likelihood (1) l(β, σ 2, θ y) = 1 2 log ( det(σ 2 I + V θ )) 1 2 (y Xβ) T (σ 2 I + V θ ) 1 (y Xβ) that arises when ε(s) is Gaussian. V θ = σ 2 0 V α is the ovariane and V α the orrelation matrix of z T = ( z(s 1 ), z(s 2 ),..., z(s n ) ). We define the pseudo log-likelihood that depends in addition to the parameters on the latent variable z (2) l (β, σ 2, θ, z y) = 1 2 log ( det(σ 2 I + V θ ) ) 1 2 n ( y(si ) x(s i ) T ) 2 β z(s i ) 1 σ 2 zt V 1 θ z. i=1 The Gaussian log-likelihood may be onsidered as a profile log-likelihood of l (β, σ 2, θ, z y) that has been maximized with respet to z. Hene, l(β, σ 2, θ y) = l (β, σ 2, θ, z y) where ( z = argmax l (β, σ 2, θ, z y) ) = V θ (σ 2 I + V θ ) 1 (y Xβ). z Maximizing (1) with respet to β, σ 2 and θ is thus equivalent to solving the expanded system of ML estimating equations (3) (4) (5) (6) (7) = 1ˆσ y X ˆβ ẑ V 1 ẑ = 0, z ˆσ ˆθ β = y X ˆβ ẑ XT = 0, ˆσ σ 2 = trae ( (I + 1ˆσ 2 Vˆθ) 1) σ 2 0 ( n i=1 = n trae ( (I + 1ˆσ 2 Vˆθ) 1) ẑ T V 1 ẑ = 0, ˆθ α = trae( ( ˆσ2 ˆσ 2 0 I + Vˆα ) 1 V α α α=ˆα ) 1 ˆσ 2 0 ) 2 y(s i ) x(s i ) T ˆβ ẑ(si ) = 0, ˆσ ẑ T V 1 V α ˆα V 1 α ˆα ẑ = 0. α=ˆα Closed-form expressions exist in general only for the generalized least squares (GLS) estimates of β ˆβ = (X T (ˆσ 2 I + Vˆθ) 1 X) 1 X T (ˆσ 2 I + Vˆθ) 1 y, and the plug-in mean square preditions of z ẑ = Vˆθ(ˆσ 2 I + Vˆθ) 1 (y X ˆβ), whih are alled universal (UK) or external drift kriging preditions in geostatistis.

4 ISI 58th World Statistis Congress of the International Statistial Institute, Dublin, August Equations (3) (7) show that the Gaussian ML estimates and the UK preditions depend on the standardized residuals ˆr/ˆσ = (y X ˆβ ẑ)/ˆσ, and it is obvious that both estimates and preditions are sensitive to outliers in y. Welsh and Rihardson (1997) proposed several approahes to make ML estimates in linear mixed models robust to outlying observations. A first approah onsists of robustifying the Gaussian log-likelihood. One has then to replae (y(s i ) x(s i ) T β z(s i )) 2 /σ 2 in (2) by a funtion ρ ( ) of the standardized errors that grows more slowly than the quadrati ( denotes here a tuning parameter that ontrols the amount of robustness of the estimates). Initially, we tried suh a strategy and approximated the restrited likelihood funtion (Harville, 1974) of a model with long-tailed ε(s) by a Laplae approximation and maximized the approximation to get robust REML estimates of the drift and variogram parameters (Shwierz et al., 2010). However, to bound the influene of outliers, we had to use a bounded ρ ( ) funtion, whih made the numerial maximization of the restrited log-likelihood more diffiult beause the related system of estimating equations has then multiple roots. Furthermore, we had diffiulties to find appropriate bias orretions to obtain Fisher onsistent estimates for Gaussian ε(s). Therefore, we eventually pursued another suggestion by Welsh and Rihardson (1997) and robustified (3) (7) by replaing ˆr/ˆσ by a bounded and odd funtion ψ ( ) of the standardized residuals, yielding thereby a system of robustified estimating equations (8) (9) (10) (11) (12) trae ( (I + 1 σ 2 V θ) 1) n i=1 trae ( ( σ2 σ 0 2 I + V α ) 1 V α ) 1 α σ 0 2 α= α 1 σ ψ (y X β z ) V 1 z + a σ θ 1 = 0, X T (y X ψ β z ) + a2 σ = 0, ψ 2 (y(s i ) x(s i ) T β z(si )) + a3 σ = 0, n trae ( (I + 1 σ 2 V θ) 1) z T V θ 1 z + a 4 = 0, z T V α 1 V α V α 1 α z + a 5 = 0. α= α In the above equations, a 1 to a 5 are adjustments for Fisher onsisteny at the Gaussian model. They are determined by the ondition that the expetations of (8) (12) must vanish for Gaussian ε(s) (e.g. Maronna et al., 2006, p. 67). Hene, a 1 = a 2 = 0 sine ψ ( ) is odd, and (13) a 3 = a 3 trae ( (I + 1 σ 2 V θ) 1) (14) (15) a 4 = a 4 ( n trae ( (I + 1 σ 2 V θ) 1)), a 5 = a 5 trae ( ( σ2 σ 0 2 I + V α ) 1 V α ), α α= α where (using (8) to derive (16)) n a 3 = E [ψ 2 (y(s i ) x(s i ) T β z(si )) (16) = σ 2 trae ( V 2 σ θ E [ z, zt ), (17) (18) i=1 a 4 = E [ z T V θ 1 z = trae ( V 1 a 5 = E [ 1 σ 2 0 z T V 1 α θ E [ z, zt ), V α α α= α V 1 α z = 1 σ 2 0 trae ( V α 1 V α V α 1 α E [ z, zt ). α= α To orret the biases we thus need the ovariane matrix of z, whih we approximated in the following way: The starting point was a first-order Taylor series approximation for ψ ( r/σ) at ε/σ ψ ( r/σ) ψ (ε/σ) 1 σ diag( ψ (ε/σ) )( z z + X( β β) )

5 ISI 58th World Statistis Congress of the International Statistial Institute, Dublin, August where r T = y X β z and ε T = (ε(s 1 ), ε(s 2 ),..., ε(s n )). Then we replaed ψ (ε(s i )/σ) by its expetation, say b = E [ψ (ε(s i )/σ) (for Gaussian ε(s)), and substituted the modified Taylor approximation in (8) (9) for ψ ( r/σ, whih resulted in [ z β β M 1 [ I X T I X T [ b z σψ(ε/σ) where M = [ b I + σ 2 V 1 θ b X T b X b X T X. Sine z and ψ (ε/σ) are independent, and letting a = E [ψ 2 (ε(s)/σ) (for Gaussian ε(s)), we finally obtained the approximation [ Cov [ z, z T Cov [ z, (19) β T [ Cov [ β, z T Cov [ β, β T M 1 Λ ΛX X T Λ X T M 1, ΛX where Λ = b 2 V θ + a σ 2 I. Solving (8) (12) subjet to (13) (15) and (19) for the non-robust ase yields the ustomary REML estimates of σ 2 and θ and the related plug-in GLS estimate of β and the UK predition of z. Hene, apart from the Bayesian interpretation of the REML estimates (Harville, 1974) and the interpretation as ML estimates obtained from error ontrasts (Patterson and Thompson, 1971), the REML estimates an also be onsidered as ML estimates, fored to be Fisher onsistent for finite sample size. Estimation proedure We wrote an R funtion that solves the robustified estimating equations for z, β, σ 2 and θ. Instead of the popular Huber ψ -funtion (Maronna et al., 2006, p. 26), we used the ontinuously differentiable funtion (20) ψ (x) = exp( 2x/), whih orresponds to a shifted and saled logisti umulative distribution funtion. Note that lim ψ (x) = ±. We omputed the roots of the estimating equations by a ombination of iteratively re-weighted least squares (IRWLS, e.g. Maronna et al., 2006, p. 105) and a Broyden s sheme. x ± For given σ 2 and θ, z and β were determined by IRWLS. The estimates were then inserted into (10) (12), and the roots of these equations were omputed by Broyden s method as implemented in the R pakage nleqslv. The estimation proedure needs initial values for z and all the parameters. An initial guess of β was obtained by MM-estimation (Maronna et al., 2006, pp. 124). The robust regression residuals were then spatially smoothed by loess to get an initial z. The MAD of the loess residuals was taken as initial σ. An initial guess of θ was obtained by fitting the sill and range parameters of a parametri variogram model (with nugget fixed at the initial σ 2 ) by weighted non-linear least squares (Cressie, 1993, p. 96) to the sample variogram of the regression residuals that had been omputed by the MAD estimator (Dowd, 1984).

6 ISI 58th World Statistis Congress of the International Statistial Institute, Dublin, August Table 1: Statistis of relative errors (%) of the approximations of a 3, a 4 and a 5 in simulations of n = Y (s) (s uniformly distributed on the unit square) for tuning parameter = 1. The data were generated with a linear external drift in the oordinates and an exponential variogram with ranges varying from 0.01 to 0.2 and nugget:sill ratios from 0.25 to 4. minimum 1st quartile median 3rd quartile maximum a a a Simulation study We explored the properties of our robust REML estimation method by simulations. First, we heked approximation (19) for Cov [ z, z T. We simulated for several external drifts and exponential or ubi variograms (with various ombinations of nugget, sill and range) data sets onsisting of Gaussian Y (s) on grids or for uniformly distributed s in one- and two-dimensional domains. For eah senario, we simulated realizations and omputed for eah realization a 3, a 4 and a 5 using (19) and the true σ2 and θ. Then we ompared the mean of these a i values with the true value that was omputed with the empirial ovarianes of the estimates z, obtained by interept slope easting slope northing sill nugget range Figure 1: Boxplots of estimates β, σ 2 and θ omputed from 200 Gaussian data sets that were simulated with a linear external drift in the oordinates and an exponential variogram with σ 2 = 0.5, σ0 2 = 2, and α = 0.05 for n = 200 uniformly distributed s in the unit square (dotted lines mark true parameter values, for = 1000 estimates orrespond to ustomary REML estimates).

7 ISI 58th World Statistis Congress of the International Statistial Institute, Dublin, August interept Z ~ (s) at ontam. lo s=0.895 Z ~ (s) next to ontam. lo. (s=0.905) Z ~ (s) far from ontam. lo. (s=0.099) = 1000 = 2 = 1.67 = 1.33 = = 1000 = 2 = 1.67 = 1.33 = = 1000 = 2 = 1.67 = 1.33 = = 1000 = 2 = 1.67 = 1.33 = 1 sill nugget range = 1000 = 2 = 1.67 = 1.33 = = 1000 = 2 = 1.67 = 1.33 = = 1000 = 2 = 1.67 = 1.33 = 1 Figure 2: Standardized urves for one realization of the simulation senario 500 Gaussian Y (s) on entred regular grid in [0, 1; no trend (β = 0.5); exponential variogram with σ 2 = 0.5, σ0 2 = 2, α = The observation at s = was ontaminated by adding x ( 20, 19.75, 19.5,..., 19.75, 20). Then we estimated for eah x the model parameters and z, and plotted n( φ(x) φ(0)) vs x, where φ denotes a parameter or an element of z. Estimates and preditions orrespond for = 1000 to ustomary REML estimates and UK preditions. solving (8) (9) with the true ovariane parameters. Table 1 lists as example the relative errors of the approximations for the simulations with the spatially uniformly distributed s on the unit square. On average, the approximation underestimated the a i, but, exept for the very short ranges, the modulus of the relative errors rarely exeeded 10 %. Errors in the approximations of a 3 to a 5 lead to some bias in the parameter estimates. Figure 1 shows boxplots of the estimates β, σ 2 and θ for a simulation senario where the approximation resulted for = 1 in relatively large errors (a 3 : %, a 4 : -8.3 %, a %). The medians of the variane estimates deviate with dereasing inreasingly from the true values, but ompared to the biases that we faed when maximizing the robustified log-likelihood these systemati errors seem tolerable. Seond, we omputed for a single realization of one simulation senario urves (Maronna et al., 2006, p. 55). Figure 2 shows that the urves remained bounded for 2, both for the parameter estimates and the preditions. As expeted from theory, the dereased with dereasing. In ontrast, the non-robust estimates ( = 1000) got inreasingly orrupted by inreasing the severity of the outlier. Third, we simulated 200 realizations of a ontaminated Gaussian field where 5 % of the ε(s) had been replaed by normal variates with ten times as large standard deviation. The boxplots of the estimates are shown in Figure 3. The ontamination strongly inflated the non-robust ( = 1000)

8 ISI 58th World Statistis Congress of the International Statistial Institute, Dublin, August interept sill nugget range Figure 3: Boxplots of estimates β, σ 2 and θ omputed from 200 ontaminated Gaussian data sets (500 Y (s) on entred regular grid in [0, 1; no trend (β = 0.5); exponential variogram with σ 2 = 0.5, σ0 2 = 2, α = 0.05; 5 % of ε(s) randomly replaed by normal variates with mean zero and standard deviation 7.9) (dotted lines mark true parameter values, for = 1000 estimates orrespond to ustomary REML estimates). REML estimate of the nugget, biased (negatively) the REML estimate of θ and inreased the varianes of the estimates of all three ovariane parameters. The robust estimates of θ showed no systemati errors and σ 2 was muh less biased and more effiient than the REML estimate. The systemati error in σ 2 dereased with dereasing. Summary and onlusions We developed a novel robust REML method to estimate the external drift and the variogram parameters of Gaussian spatial data that are possibly ontaminated by outliers. The methods also provides robustified universal kriging preditions. Simulations showed that our proedure bounds the influene of outliers on parameter estimates and preditions, and yields approximately Fisher onsistent estimates at the Gaussian model. For large, the estimates and preditions oinide with the ustomary REML estimates and the plug-in UK preditions. So far, we did not yet hek whether (19) provides an aurate approximation of Cov [ β, β T and Cov [( z z), ( z z) T. These quantities are required for omputing the mean square predition error at non-sampled s. Furthermore, we have to develop robust proedures for testing hypotheses about β. Some of these further developments will be presented at the onferene, along with an analysis of data on heavy metals in the soils around a metal smelter in Switzerland where removal and displaement of ontaminated soil has led to unpreditable irregularities in the spatial distribution of the metals. Aknowledgements We gratefully aknowledge the support of the Swiss Federal Offie for the Environment (FOEN), whih funded a part of this researh. REFERENCES Cressie, N. A. C. (1993). Statistis for Spatial Data. John Wiley & Sons, New York, revised edition. Dowd, P. A. (1984). The variogram and kriging: Robust and resistant estimators. In G. Verly, M. David,

9 ISI 58th World Statistis Congress of the International Statistial Institute, Dublin, August A. Journel, and A. Maréhal, editors, Geostatistis for Natural Resoures Charaterization, volume 1, pages , Dordreht. D. Reidel Publishing Company. Harville, D. A. (1974). Bayesian inferene for variane omponents using only error ontrasts. Biometrika, 61, Hawkins, D. M. and Cressie, N. (1984). Robust kriging a proposal. Mathematial Geology, 16, Lark, R. M. (2000). A omparison of some robust estimators of the variogram for use in soil survey. European Journal of Soil Siene, 51, Marhant, B. P. and Lark, R. M. (2007). Robust estimation of the variogram by residual maximum likelihood. Geoderma, 140, Maronna, R. A., Martin, R. D., and Yohai, V. J. (2006). Robust Statistis Theory and Methods. John Wiley & Sons, Chihester. Militino, A. F. and Ugarte, M. D. (1997). A GM estimation of the loation parameters in a spatial linear model. Communiations in Statistis, Theory and Methods, 26(7), Patterson, H. D. and Thompson, R. (1971). Reovery of inter-blok information when blok sizes are unequal. Biometrika, 58(3), R Development Core Team (2011). R: A Language and Environment for Statistial Computing. R Foundation for Statistial Computing, Vienna, Austria. Rihardson, A. M. and Welsh, A. H. (1995). Robust restrited maximum likelihood in mixed linear models. Biometris, 51, Shwierz, C., Papritz, A., Stahel, W. A., and Künsh, H. R. (2010). Robust REML estimation of the variogram and robust kriging. In Proeedings of the International Conferene on Robust Statistis Prague 2010, Book of Abstrats, pages 91 92, Prague. Welsh, A. H. and Rihardson, A. M. (1997). Approahes to the robust estimation of mixed models. In Robust Inferene, volume 15 of Handbook of Statistis, pages Elsevier.