Chapter 4 - Fundamentals of spatial processes Lecture notes

Transcription

1 Chapter 4 - Fundamentals of spatial processes Lecture notes Geir Storvik January 21, 2013

2 STK Intro 2 Spatial processes Typically correlation between nearby sites Mostly positive correlation Negative correlation when competition Typically part of a space-time process Temporal snapshot Temporal aggregation Statistical analysis Incorporate spatial dependence into spatial statistical models Relatively new in statistics, active research field!

3 STK Intro 3 Hierarchical (statistical) models Data model Process model In time series setting - state space models

4 STK Intro 4 Hierarchical (statistical) models Data model Process model In time series setting - state space models Example Y t =αy t 1 + W t, t = 2, 3,... Z t =βy t + η t, t = 1, 2, 3,... W t ind (0, σ 2 W ) η t ind (0, σ 2 W ) Process model Data model We will do similar type of modelling now, separating the process model and the data model: Z(s i ) = Y (s i ) + ε(s i ), ε(s i ) iid

5 STK Intro 5 Prediction Typical aim: Observed Z = (Z(s 1 ),..., Z(s m )) Want to predict Y (s 0 ) Squared error loss function: Optimal E(Y (s 0 ) Z) Empirical methods: E(Y (s 0) Z) E(Y (s 0) Z, ˆθ) Bayesian methods: E(Y (s 0) Z) = θ E(Y (s0) Z, θ)p(θ Z)dθ Require E(Y (s 0 ) Z, θ) Z Y are independent, but dependent marginally Require model for {Y (s)}

6 Geostatistical models Assume {Y (s)} is a Gaussian process (Y (s 1 ),..., Y (s m )) is multivariate Gaussian for all m and s 1,..., s m Need to specify µ(s) = E(Y (s)) C Y (s, s ) = cov(y (s), Y (s )) Usually assume 2. order stationarity: µ(s) = µ, s C Y (s, s ) = C Y ( s s ), s, s Possible extension: µ(s) = x(s) T β Often: Z(s i ) Y (s i ), σε 2 ind.gau(y (s i ), σε) 2

7 STK Intro 7 Prediction (Z(s 1 ),..., Z(s m ), Y (s 0 )) is multivariate Gaussian Can use rules about conditional distributions: ( ) (( ) ( )) X1 µ1 Σ11 Σ MVN, 12 X 2 µ 2 Σ 21 Σ 22 Need E(X 2 X 1 ) =µ 1 + Σ 12 Σ 1 22 (X 2 µ 2 ) var(x 2 X 1 ) =Σ 11 Σ 12 Σ 1 22 Σ 21 Expectations: As for ordinary linear regression Covariances: New!

8 TK Intro 8 Variogram and covariance function Dependence can be specified through covariance functions Not always that such exist, but prediction can still be carried out More general concept: Variogram Note: 2γ Y (h) var[y (s + h) Y (s)] =var[y (s + h)] + var[y (s)] 2cov[Y (s + h), Y (s)] =2C Y (0) 2C Y (h) Variogram can exist even if var[y (s)] =! Often assumed in spatial statistics, thereby the use of variograms When 2γ Y (h) = 2γ Y ( h ), we get an isotrophic variogram

9 Isotropic covariance functions/variograms Matern covariance function C Y (h; θ) = σ1{2 2 θ2 1 Γ(θ 2 )} 1 { h /θ 1 } θ2 K θ2 ( h /θ 1 ) Powered-exponential C Y (h; θ) = σ1 2 exp{ ( h /θ 1 ) θ2 } Exponential C Y (h; θ) = σ1 2 exp{ ( h /θ 1 )} Gaussian C Y (h; θ) = σ1 2 exp{ ( h /θ 1 ) 2 }

10 TK Intro 10 Bochner s theorem A covariance function needs to be positive definite Theorem (Bochner, 1955) If C Y (h) dh <, then a valid real-valued covariance function can be written as C Y (h) = cos(ω T h)f Y (ω)dω where f Y (ω) 0 is symmetric about ω = 0. f Y (ω): Spectral density of C Y (h).

11 Nugget effect and Sill We have C Z (h) =cov[z(s), Z(s + h)] =cov[y (s) + ε(s), Y (s + h) + ε(s + h)] =cov[y (s), Y (s + h)] + cov[ε(s), ε(s + h)] =C Y (h) + σεi 2 (h = 0) Assume Then C Y (0) =σy 2 lim [C Y (0) C Y (h)] =0 h 0 C Z (0) = σy 2 + σε 2 lim [C Z (0) C Z (h)] = σε 2 = c 0 h 0 Possible to include nugget effect also in Y -process. Sill Nugget effect TK Intro 11

12 Nugget/sill STK Intro 12

13 STK Intro 13 Estimation of variogram/covariance function 2γ Z (h) var[z(s + h) Z(s)] Const expectation = E[Z(s + h) Z(s)] 2 Can estimate from all pairs having distance h between. Problem: Few/no pairs for all h Simplifications γ Z (h) = γ 0 Z ( h ) Use 2 γ 0 Z (h) = ave{(z(s i) Z(s j )) 2 ; s i s j T (h)} If covariates, use residuals

14 Boreality data Empirical variogram: Boreality data

15 STK Intro 15 Testing for independence If independence: γ 0 Z (h) = σ2 Z Test-statistic F = γ Z (h 1 )/ σ 2 Z, h 1 smallest observed distance Reject H 0 for F 1 large Permutation test: Recalculate F for all permutations of Z If observed F is above 97.5% percentile, reject H 0 Boreality example: P-value =

16 Kriging = Prediction Model Y =Xβ + δ Z =Y + ε Prediction of Y (s 0 ) Linear predictors {L T Z + k} Optimal predictor minimize MSPE(L, k) E[Y (s 0 ) L T Z k] 2 =var[y (s 0 ) L T Z k] + {E[Y (s 0 ) L T Z k]} 2 Note: Do not assume any distributional assumptions

17 Kriging MSPE(L, k) =var[y (s 0 ) L T Z k] + {E[Y (s 0 ) L T Z k]} 2 =var[y (s 0 ) L T Z k] + {µ Y (s 0 ) L T µ z k]} 2 Second term is zero if k = µ Y (s 0 ) L T µ Z. First term (c(s 0 ) = cov[z, Y (s 0 )]) var[y (s 0 ) L T Z k] =C Y (s 0, s 0 ) 2L T c(s 0 ) + L T C Z L Derivative wrt L T : giving 2c(s 0 ) + 2C Z L = 0 L = C 1 Z c(s 0) Y (s 0 ) =µ Y (s 0 ) + c(s 0 ) T C 1 Z [Z µ Z ] MSPE(L, k ) =C Y (s 0, s 0 ) c(s 0 ) T C 1 Z c(s 0)

18 Kriging c(s 0 ) =cov[z, Y (s 0 )] =cov[y, Y (s 0 )] =(C Y (s 0, s 1 ),..., C Y (s 0, s m )) =c Y (s 0 ) C Z ={C Z (s i, s j )} { C Y (s i, s i ) + σ 2 C Z (s i, s j ) = ε, C Y (s i, s j ), s i = s j s i s j Y (s 0 ) = µ Y (s 0 ) + c Y (s 0 ) T C 1 Z (Z µ Z )

19 Gaussian assumptions Assume now Y, Z are MVN ( ) (( Z µz = MVN Y(s 0 ) µ Y (s 0 ) Give ) ( )) CZ c, Y (s 0 ) T c Y (s 0 ) C Y (s 0, s 0 ) Y (s 0 ) Z N(µ Y (s 0 ) + c(s 0 ) T C 1 Z [Z µ Z ], C Y (s 0, s 0 ) c(s 0 ) T C 1 Z c(s 0)) Same as kriging! The book derive this directly without using the formula for conditional distribution

20 Kriging (cont) Assuming Then and Y (s 0 ) = µ Y (s 0 ) + c Y (s 0 ) T C 1 Z (Z µ Z ) E[Y (s)] =x(s) T β Z(s i ) Y (s i ), σ 2 ε ind.gau(y (s i ), σ 2 ε) µ Z =µ Y = Xβ C Z =Σ Y + σ 2 εi c Y (s 0 ) =(C Y (s 0, s 1 ),..., C Y (s 0, s m )) T Y (s 0 ) = x(s 0 )β + c Y (s 0 ) T [Σ Y + σ 2 εi] 1 (Z Xβ)

21 STK Intro 21 Summary previous time/plan Simple kriging Linear predictor Assume parameters known Equal to conditional expectation

22 STK Intro 22 Summary previous time/plan Simple kriging Linear predictor Assume parameters known Equal to conditional expectation Unknown parameters Ordinary kriging Plug-in estimates Bayesian approach Non-Gaussian models

23 STK Intro 23 Unknown parameters So far assumed parameters known, what if unknown? Plug-in estimate/empirical Bayes Bayesian approach Direct approach - Ordinary kriging

24 STK Intro 24 Estimation Likelihood L(θ) = p(z; θ) = p(z y; θ)p(y; θ)dθ Gaussian processes: Can be derived analytically Optimization can still be problematic Many (too many?) routines in R available y

25 Estimation Likelihood L(θ) = p(z; θ) = p(z y; θ)p(y; θ)dθ Gaussian processes: Can be derived analytically Optimization can still be problematic y Many (too many?) routines in R available > gls(bor~wet,correlation=corgaus(form=~x+y,nugget=true),data=boreality) Generalized least squares fit by REML Model: Bor ~ Wet Data: Boreality Log-restricted-likelihood: Coefficients: (Intercept) Wet Correlation Structure: Gaussian spatial correlation Formula: ~x + y Parameter estimate(s): range nugget Degrees of freedom: 533 total; 531 residual Residual standard error:

26 Bayesian approach Hierarchical model Variable Densities Notation in book Data model: Z p(z Y, θ) [Z Y, θ] Process model: Y p(y θ) [Y θ] Parameter: θ

27 Bayesian approach Hierarchical model Variable Densities Notation in book Data model: Z p(z Y, θ) [Z Y, θ] Process model: Y p(y θ) [Y θ] Parameter: θ Simultaneous model: p(y, z θ) = p(z y, θ)p(y θ) Marginal model: L(θ) = p(z θ) = y p(z, y θ)dy Inference: θ = argmax θ L(θ)

28 Bayesian approach Hierarchical model Variable Densities Notation in book Data model: Z p(z Y, θ) [Z Y, θ] Process model: Y p(y θ) [Y θ] Parameter: θ Simultaneous model: p(y, z θ) = p(z y, θ)p(y θ) Marginal model: L(θ) = p(z θ) = p(z, y θ)dy y Inference: θ = argmax θ L(θ) Bayesian approach: Include model on θ Variable Densities Notation in book Data model: Z p(z Y, θ) [Z Y, θ] Process model: Y p(y θ) [Y θ] Parameter model: θ p(θ) [θ] Simultaneous model: p(y, z, θ) Marginal model: p(z) = θ p(z, y θ)dydθ y Inference: θ = θ θp(θ z)dθ

29 Bayesian approach - example Assume z 1,..., z m are iid, z i = µ + ε i, ε i N(0, σ 2 ) σ 2 known, interest in estimating µ. ML: ˆµ ML = z, Var[ z] = σ 2 /m

30 Bayesian approach - example Assume z 1,..., z m are iid, z i = µ + ε i, ε i N(0, σ 2 ) σ 2 known, interest in estimating µ. ML: ˆµ ML = z, Var[ z] = σ 2 /m Bayesian approach: Assume µ N(µ 0, kσ 2 ) Can show that E[µ z] = k 1 µ 0 + m z k 1 + m σ2 Var[µ z] = k 1 + m k z k σ 2 m

31 TK Intro 31 Prediction - example Predict new point z 0. Plug-in-rule: ẑ 0 = z, var[ẑ 0 ] = σ 2

32 TK Intro 32 Prediction - example Predict new point z 0. Plug-in-rule: ẑ 0 = z, var[ẑ 0 ] = σ 2 Bayesian approach: E[z 0 z] =E[E[z 0 µ, z]] = E[µ z] = k 1 µ 0 + m z k 1 + m var[z 0 z] =E[var[z 0 µ, z]] + var[e[z 0 µ, z]] =σ 2 + var[µ z] =σ 2 + σ 2 k 1 + m k k z σ 2 + σ2 m Taking uncertainty in µ into account.

33 INLA INLA = Integrated nested Laplace approximation (software) C-code, R-interface Can be installed by source(" Both empirical Bayes and Bayes

34 Example - simulated data Assume Y (s) =β 0 + β 1 x(s) + σ δ δ(s), Z(s i ) =Y (s i ) + σ ε ε(s i ), {δ(s)} matern correlation {ε(s i )} independent Simulations: Simulation on grid {x(s)} sinus curve in horizontal direction Parametervalues (β 0, β 1 ) = (2, 0.3), σ δ = 1, σ ε = 0.3 and (θ 1, θ 2 ) = (2, 3) Show images Show code/results

35 STK Intro 35 Matern model - parametrization Model Range par Smoothness par Book { h /θ 1 } θ2 K θ2 ( h /θ 1 ) θ 1 θ 2 geor { h /φ} κ K κ ( h /φ) φ κ INLA { h κ} ν K ν ( h κ) 1/κ ν Note: INLA reports an estimate of another range, defined as 8 r = κ corresponding to a distance where the covariance function is approximately zero. An estimate of the range parameter can then be obtained by 1 κ = r 8 Note: INLA works with precisions τ y = σy 2, τ z = σz 2.

36 STK Intro 36 Output INLA Empirical Bayes Fixed effects: mean sd 0.025quant 0.5quant 0.975quant kld (Intercept) x Random effects: Name Model Max KLD delta Matern2D model Model hyperparameters: mean sd 0.025quant 0.5quant Precision for the Gaussian observations Precision for delta Range for delta quant Precision for the Gaussian observations Precision for delta Range for delta Marginal Likelihood: Warning: Interpret the marginal likelihood with care if the prior model is improper. Posterior marginals for linear predictor and fitted values computed

37 STK Intro 37 Result - INLA - empirical Bayes Parameter True value Estimate β β σ z = σ y = φ = 2.113

38 STK Intro 38 Ordinary kriging E[Y (s)] = µ, µ unknown, obs Z = (Z 1,..., Z m ) MSPE(λ) = E[Y (s 0 ) λ T Z) 2 Constraint: E[λ T Z] = E[Y (s 0 )] = µ E[λ T Z] = λ T E[Z] = λ T 1µ λ T 1 = 1 Lagrange multiplier: Minimize MSPE(λ) 2κ(λ T 1 1) MSPE(λ) = C Y (s 0, s 0 ) 2λ T c Y (s 0 ) + λ T C Z λ Differentiation wrt λ T : 2c Y (s 0 ) + 2C Z λ =2κ1 λ T 1 =1 λ =C 1 Z [c Y (s 0 ) + κ 1] κ = 1 c Y (s 0 ) T C 1 Z 1 1 T C 1 Z 1

39 Kriging Simple kriging, EY (s) = x(s) T β, known Y (s 0 ) = x(s 0 )β + c Y (s 0 ) T C 1 (Z Xβ) Z

40 Kriging Simple kriging, EY (s) = x(s) T β, known Y (s 0 ) = x(s 0 )β + c Y (s 0 ) T C 1 (Z Xβ) Ordinary kriging, EY (s) = µ, unknown Z Ŷ (s 0 ) ={c Y (s 0 ) + 1(1 1T C Z 1c y (s 0 )) 1 T C 1 Z 1 } T C 1 Z Z = µ gls + c Y (s 0 ) T C 1 Z (Z 1 µ gls) µ gls =[1 T C 1 Z 1] 1 1 T C 1 z Z

41 Kriging Simple kriging, EY (s) = x(s) T β, known Y (s 0 ) = x(s 0 )β + c Y (s 0 ) T C 1 (Z Xβ) Ordinary kriging, EY (s) = µ, unknown Z Ŷ (s 0 ) ={c Y (s 0 ) + 1(1 1T C Z 1c y (s 0 )) 1 T C 1 Z 1 } T C 1 Z Z = µ gls + c Y (s 0 ) T C 1 Z (Z 1 µ gls) µ gls =[1 T C 1 Z 1] 1 1 T C 1 z Z Universal kriging, EY (s) = x(s) T β, unknown Ŷ (s 0 ) =x(s 0 ) T βgls + c Y (s 0 ) T C 1 Z (Z X β gls ) β gls =[X T C 1 Z X] 1 X T C 1 z Z

42 TK Intro 42 Non-Gaussian data Counts, binary data: Gaussian assumption inappropriate Can still have Gaussian assumption on latent process, but non-gaussian data-distribution Y (s) =x(s) T β + ε(s), {ε(s)} Gaussian process Z(s i ) Y (s i ), θ 1 ind.f(y (s i ), θ 1 ) Best linear predictor still possible, but not reasonable (?) Conditional expectation E[Y (s 0 ) Z] still optimal under square loss Not easy to compute anymore Exponential-family model (EFM) f (z) = exp{(zη b(η))/a(θ 1 ) + c(z, θ 1 )} η =x T β Include Binomial, Poisson, Gaussian, Gamma

43 STK Intro 43 Estimation Likelihood L(θ) = p(z; θ) = p(z y; θ)p(y; θ)dθ Gaussian processes: Can be derived analytically Non-Gaussian: Can be problematic Monte Carlo Laplace approximations y

44 STK Intro 44 Computation of conditional expectation E[Y (s 0 ) Z] = E[E[Y (s 0 ) Y, Z]] Monte Carlo: E[E[Y (s 0 ) Y, Z]] 1 M M E[Y (s 0 ) Y m, Z] m=1 where Y m [Y Z] (MCMC) Laplace approximation: Approximate [Y Z] by a Gaussian Computer software available for both approahces. Also give uncertainty measures var[y (s 0 ) Z].

45 STK Intro 45 Laplace approximation L(θ) =p(z; θ) = p(z y; θ)p(y; θ)dy = exp{f (y; θ)}dy y y where f (y; θ) = log p(z y; θ) + log p(y; θ) Taylor approximation, y 0 max-point of f (y; θ) (depending on θ) f (y) f (y 0 ) 1 2 (y y 0) T H(y y 0 ), H = 2 y y T f (y) y=y 0 L(θ) exp{f (y 0 ) 1 2 (y y 0) T H(y y 0 )}dy y = exp{f (y 0 )}(2π) m/2 H 1/2

46 Example - simulated data Assume Y (s) =β 0 + β 1 x(s) + σ δ δ(s), Z(s i ) Poisson(exp(Y (s i )) {δ(s)} matern correlation Simulations: Simulation on grid {x(s)} sinus curve in horizontal direction Parametervalues (β 0, β 1 ) = (2, 0.3), σ δ = 1, σ ε = 0.3 and (θ 1, θ 2 ) = (2, 3) Show images Show code/results