Model Selection for Geostatistical Models

Transcription

1 Model Selection for Geostatistical Models Richard A. Davis Colorado State University Joint work with: Jennifer A. Hoeting, Colorado State University Andrew Merton, Colorado State University Sandy E. Thompson, Pacific Northwest National ab Partially supported by an EPA funded project entitled: STARMAP: Space-Time Aquatic Resources Modeling and Analysis Program The work reported here was developed un STAR Research Assistance Agreements CR awarded by the U.S. Environmental Protection Agency EPA to Colorado State University. This presentation has not been formally reviewed by EPA. EPA does not endorse any products or commercial services mentioned in this presentation.

2 Muddy Creek- tributary to Sun River in Central Montana Muddy Creek: surveyed every 5.4 meters, total of 5456m; 358 measurements bed elevation Degree AIC c distance m

3 Muddy Creek: residuals from polyd=4 fit Minimum AIC c ARMA model: ARMA, residuals deg = distance m Y t =.574 Y t- ε t.3 ε t-, {ε t }~WN0,.0564 Some Noncausal theory: ARMA, model: Y t S =.743 estimates Y t- of ε t trend.3 parameters ε t- are asymptotically efficient. S estimates are asymptotically indep of cov parameter estimates. acf Blue = sample Red = model acf Blue = sample Red = model lag m lag m 3

4 Muddy Creek cont Summary of models fitted to Muddy Creek bed elevation: Degree AIC c ARMA AIC c, 59.67, 6.98, 6.30, 7.,.78,

5 Muddy Creek cont Simulated series: polynomial degree 4 ARMA,: bed elevation distance m 5

6 Recap of Traditional Model Fitting Suppose {Y t } follows the linear model with time series or spatial errors given by Y t = β 0 X t β X tp β p W t, where {W t } is a stationary ARMA time series. Estimate β 0,...,β p by ordinary least squares OS. Identify and estimate ARMA parameters using the estimated residuals, Y βˆ X βˆ X βˆ t 0 t tp p Re-estimate β 0,...,β p and ARMA parameters using full ME. imitations of this approach for model selection: Ignore potential confounding between explanatory variables and correlation in {W t }. Ignoring autocorrelation function can mask importance of explanatory variables. 6

7 PROGRAM Motivation Introduction Model Selection for Geostatistical Models Basic setup AIC for Geostatistical Models Basics Minimum Description ength MD for Geostatistical Models Simulation Results Prediction Application to Orange-Throated Whiptail izard 7

8 Introduction Model selection setup: Suppose {Zs, s D} is a random field where D is some subset of R. Spatial data: observations Zs,..., Zs n at locations s,..., s n. Explanatory variables covariates: p explanatory variables X s,..., X p s are available at each location s. inear model for Z: Zs = β 0 X s β X p s β p deterministic δs { stochastic where δs is a mean 0 stationary isotropic Gaussian random field. Model selection issue. Which explanatory variables should be included in the model? What family of covariance functions should be used to model δs? 8

9 Model Selection Problem: How does one choose the best set of covariates and family of covariance functions? Some Objectives of Model Selection.. Choose the correct order model consistency. - There exists a true model and the model selection procedure will choose the correct set of covariates and the right family of covariance functions as sample size increases.. Choose the model that performs best for prediction efficiency. - Find the model that predicts or interpolates well at unobserved locations. 3. Choose the model that maximizes data compression. - Find a model that summarizes the data in the most compact fashion, yet retains the salient features present in the data. 9

10 The Geostatistical Model Model : Zs = β 0 X s β X p s β p δs Xs =, X s,, X p s T is a vector of explanatory variables observed at location s. β = β 0, β,, β p T is a p dimensional parameter vector. δs is the unobserved regression error at location s. Assumptions on δs. δs is a stationary, isotropic Gaussian process with mean 0 and covariance function Covδs, δt = σ ρ s-t,θ. σ is the variance of the process s-t is the euclidean distance between locations s and t. ρ., θ is an isotropic correlation function parameterized by a k-dim l vector θ. 0

11 Autocorrelation functions Some standard autocorrelation functions these are a bit limiting.. Exponential. Gaussian 3. Matern where Κ θ. is the modified Bessel function. θ is the range parameter controlling the rate of decay of correlations. θ is the smoothness parameter controlling the smoothness of the random field. 0, / exp ; > θ = θ ρ d d d 0, / exp ; > θ = θ ρ d d d 0,, ; > θ θ Κ θ θ θ Γ = θ ρ θ θ θ d d d d

12 Autocorrelation functions cont correlation Exponential θ =.0 Gaussian θ = 4.0 Matern θ =.5, θ =.75 Matern θ =.5, θ =.0 Matern θ =3.5, θ = distance

13 Estimation og-likelihood: For the data Z = Zs,..., Zs n T log Z β,θ,σ =.5 log σ Ω.5 σ Z Xβ Τ Ω Z Xβ Τ, where Ω = ρ s i s j \\;θ is the matrix of correlations for the data vector Z. ME estimate: Note. ˆ, β θˆ, σˆ maximizes the log-likelihood. ME estimates can be difficult to compute for large sample sizes. Restricted maximum likelihood REM estimates often have more desirable sampling properties, but performance for model selection is not clear. 3

14 AIC for Geostatistical Models Background on AIC Burnham and Anderson 998 and McQuarrie and Tsai 998.: Suppose Z ~ f T {f. ;ψ, ψ Ψ} is a family of candidate probability density functions. The Kullback-eibler information between f. ;ψ and f T is f z I ψ log f ft z ψ = has the interpretation as a measure of the distance between f. ;ψ and f T z dz loss of information when f. ;ψ is used as the model instead of f T T 4

15 AIC for Geostatistical Models By Jensen s inequality, f z ψ I ψ log 0 ft z dz = ft z with equality holding if and only if f z ;ψ = f T z, a.e. Basic idea: minimize the Kullback-eibler index ψ = = E log f T log z ψ f Z ψ, T z dz where Z ψ is the likelihood based on the data Z. Problem: Cannot compute ψ or ψˆ, where ψˆ is the ME of ψ. Strategy: Search for an unbiased estimator of E ψ ψˆ and find the candidate model which minimizes this statistic as a function of the model. 5

16 AIC for Geostatistical Models Note: The expectation above can be written as the double expectation, E ψ ψˆ = E E log ψˆ Z = E log ψˆ, ψ T Y where Y is independent of Z with the same distribution. Thus log ˆ Y ψ is an unbiased estimate of ψˆ. But Y is unobserved and that is where the AIC correction factor comes into play. Applied to the Geostatistical model: Parameter vector ψ = β,θ,σ T and ψ Y log Y ˆ, β θˆ, σˆ = log Z ˆ, β θˆ, σˆ σˆ S Y ˆ, β θˆ n, where and S Y σˆ β, θ = Y = Y So just need to approximate E ψ ˆ Xβˆ σ S Xβ Y T Ω T Ω ˆ, β θˆ. ˆ θ Y θ Y Xβ Xβˆ / n. 6

17 7 AIC for Geostatistical Models In order to compute we assume standard asymptotics hold: is ANβ,θ T, I n -, where I n is the Fisher information. For large n, I n can be approximated by For n large, is approximately distributed as σ χ n p k. is approximately independent of Then ˆ, ˆ, ˆ θ β σ ψ Y S E ˆ ˆ, θ β.,,, /, θ β θ β θ β σ = θ β ψ T T S Y E V ˆ ˆ, ˆ θ β = σ Z S n ˆσ. ˆ ˆ, θ β. 3 ~ ˆ ~ ˆ ˆ, ˆ = σ σ σ σ θ β σ ψ ψ k p n n k p n n k p n k p n n E k p n n S E Y

18 AIC for Geostatistical Models The quantity, referred to as the corrected AIC, is then AIC c = log Z ˆ, β θˆ, σˆ p n k n. p k 3 The standard AIC statistic is given by AIC = log Z ˆ, β θˆ, σˆ p k. 8

19 Model Selection Using Minimum Description ength Basics of MD: Choose the model which maximizes the compression of the data or, equivalently, select the model that minimizes the code length of the data i.e., amount of memory required to encode the data. M = class of operating models for y = y,..., y n F y = code length of y relative to F M Typically, this term can be decomposed into two pieces two-part code, where y = Fˆ y eˆ Fˆ, F Fˆ y e ˆ Fˆ = code length of the fitted model for F = code length of the residuals based on the fitted model 9

20 0 Illustration Using a Simple Regression Model see T. ee `0 Encoding the data: x,y,..., x n,y n. Naïve case,,,, " " n n n n y y x x y y x x naive = = K K. inear model; suppose y i = a 0 a x i, i =,..., n. Then,,, " " 0 0 a a x x a a x x p n n = = = K 3. inear model with noise; suppose y i = a 0 a x i ε i, i =,..., n, where {ε i }~IID N0,σ. Then If A < y... y n, then p= encoding scheme dominates the naïve scheme K ˆ, ˆ, ˆ ˆ,, ˆ ˆ ˆ ˆ " " 0 0 σ ε ε σ = = a a a a x x p n n A

21 Model Selection Using Minimum Description ength cont Applied to the Geostatistical model: Zs = β 0 X s β X p s β p δs First term Fˆ y : Fˆ y = ˆ β, K, βˆ ˆ θ, K, θˆ 0 p k Encoding: integer I : log I bits if I unbounded log I U bits if I bounded by I U ME θˆ : ½ log N bits where N = number of observations used to compute θˆ ; Rissanen 989

22 So, Model Selection Using Minimum Description ength cont p k Fˆ y = log n log n = p k log j= 0 j= Second term eˆ Fˆ : Using Shannon s classical results on information theory, Rissanen demonstrates that the code length of ê can be approximated by the negative of the log-likelihood of the fitted model, i.e., by eˆ Fˆ log Z ˆ, β θˆ, σˆ MD for the model F is then MD / p klog n log ˆ, β θˆ, σˆ = Z The strategy is to find the model that minimizes MD. n Note: p k n log ˆ, β θˆ, σˆ MD = / Z log Penalty coefficient log n is larger than that for AIC.

23 Model Selection and Spatial Correlation Traditional approach to model selection:. Select explanatory variable to model the large scale variation. Here one might use AIC c assuming independence in the noise term.. Estimate correlation function parameters using residuals from model in previous step. 3. Re-estimate regression parameters using GS. 4. Iterate steps and 3 until convergence. imitations of this approach: Ignore potential confounding between explanatory variables and correlation in spatial process. Ignoring autocorrelation function can mask importance of explanatory variables. 3

24 Model Selection: Simulation Set-up Goal of simulation study: compare model selection performance of AIC versus traditional method.. Sampling design: 00 sites randomly located on [0,0] [0,0].. Explanatory variables: Five potential explanatory variables, X s j, X s j,..., X 5 s j, j =,..., 00, IID sqrt/0*t 3. Response variables: Z = 0.75 X 0.50 X 0.5 X 3 δ, where δ is Gaussian with mean 0, σ =50, and Matern autocorrelation function with parameters θ =4 and θ =. 4. Replicates: 500 replicates were simulated with a new Gaussian random field generated for each replicate. 5. AIC: Computed for 5 =3 possible models per replicate. 4

25 Model Selection: Random Pattern Sampling Design y x 5

26 Model Selection: Simulation Results for the Random Pattern Independent AIC and Spatial AIC report the percentage of simulations that each model was selected. Of the 3 possible models, the results given here include only those with 0% or more support for one of the models. Variables in Model Spatial AIC Independent AIC MD X, X, X X, X, X 3, X X, X, X 3, X X, X Intercept only X X

27 Independent Model AIC values AICC N U

28 Spatial Model AIC values AICC N U

29 Sampling Patterns Highly Clustered ightly Clustered Random Pattern y y y x x x Regular Pattern Grid Design y y x x 9

30 Effect of Sampling Design Highly Variables in Model Clustered ightly Clustered Random Regular Pattern Grid Design X, X, X X, X X, X, X 3, X X, X, X 3, X 4, X Each column reports the percentage of simulations that each model was selected. Of the 3 possible models, the results given here include only those with 0% or more support for one of the models. 30

31 Effect of Smoothness Parameter θ = 0.50 θ = 0.75 θ =.00 θ = 4.00 Variables in Model spat ind spat ind spat ind spat ind X, X, X X, X, X 3, X X, X, X 3, X X, X Intercept only X X Each column reports the percentage of simulations that each model was selected. Of the 3 possible models, the results given here include only those with 0% or more support for one of the models. 3

32 Effect of smoothness cont correlation Matern θ =4.0, θ =.5 Matern θ =4.0, θ =.75 Matern θ =4.0, θ =.0 Matern θ =4.0, θ = distance 3

33 Effect of Range θ = θ = 4 θ = 6 θ = 8 Variables in Model spat ind spat ind spat ind spat ind X, X, X X, X, X 3, X X, X, X 3, X X, X Intercept only X X Each column reports the percentage of simulations that each model was selected. Of the 3 possible models, the results given here include only those with 0% or more support for one of the models. 33

34 Effect of range cont correlation Matern θ =4.0, θ =.0 Matern θ =4.0, θ =.0 Matern θ =6.0, θ =.0 Matern θ =8.0, θ = distance 34

35 Prediction Efficient prediction: Time series Shibata 980, Brockwell and Davis 99. AIC is an efficient order selection procedure for autoregressive models. Regression see McQuarrie and Tsai 998. Other notions of efficiency, e.g., Kullback-eibler efficiency and efficiency see McQuarrie and Tsai

36 Prediction Error Simulations: Model selection and estimation using 00 observations. Used fitted model to predict at 00 additional locations. Evaluated performance using mean square prediction error MSPE = j= Z j Zˆ j, where Ẑ j is the universal kriging predictor for the j th prediction location. 36

37 MSPE Spatial AICC is 6.9% improvement-methods agreed only times. Spatial AICC is 39.6% better over independent AICC with indep noise. Predictive coverage was about the same.9 for spatial AIC,.95 for indep error AIC MSPE Spatial AICC Independent AICC 37

38 Example: izard abundance Abundance for the orange-throated whiptail lizard in southern California Ver Hoef, Cressie, Fisher, Case 00. Data: 47 locations Zs i = log average number of lizards caught per day at location s i Explanatory variables. 37 variables 37 =.374*0 models reduced to the following 6. ant abundance three levels log % sandy soils elevation barerock indicator % cover log% chaparral plants 60 possible models 38

39 ocations Southern California of Whiptail izards atitude degrees ongitude degrees 39

40 izard abundance cont Spatial Ind Predictors AIC Rank Rank Ant, % sand Ant, Ants, % sand Ant, % sand, % cover Ant, Ant, % sand, % cover, elevation, barerock, % chaparral 9. 4 Ant, Ant, % sand, elevation, barerock, % chaparral Ant, % sand, % cover, elevation, barerock Note: MD chooses the same model as AICC. 40

41 izard abundance simulation Simulation of model: use covariates Ant, % sand, with Matern covariance function Key: = ant, = ant, 3 = ant 3, 4 = % sand, 5 = elevation, 6 = barerock, 7 = cover, 8 = chaparral plants AICC AICC N U N U

42 Summary Remarks. Spatial correlation should not be ignored when selecting explanatory variables.. Model choice for prediction should involve joint selection of the explanatory variables and the form of the autocorrelation function. 3. Sampling patterns can severely impact model selection. 4. MD offers a nice philosophical alternative to AIC for modeling complex phenomena. 4