Bayesian Transformed Gaussian Random Field: A Review

Transcription

1 Bayesian Transformed Gaussian Random Field: A Review Benjamin Kedem Department of Mathematics & ISR University of Maryland College Park, MD (Victor De Oliveira, David Bindel, Boris and Sandra Kozintsev) 1

2 Berger, De Oliveira, Sansó (2001). Objective Bayesian Analysis of Spatially Correlated Data. JASA, 96, ( ) Bindel, De Oliveira, Kedem (1997). An Implementation of the Bayesian Transformed Gaussian Spatial Prediction Model. bnk/btg page.html. Box, Cox (1964). An Analysis of Transformations (with discussion). JRSS, B 26, De Oliveira (2000). Bayesian Prediction of Clipped Gaussian Random Fields. Comp. Data Analysis, 34, ( ) De Oliveira, Kedem, Short (1997). Bayesian Prediction of Transformed Gaussian Random Fields. JASA, 92,

3 Handcock, Stein (1993). A Bayesian Analysis of Kriging. Technometrics, 35, ( ) De Oliveira, Ecker (2002). Bayesian Hot Spot Detection in the Presence of a Spatial Trend: Application to Total Nitrogen Concentration in the Chesapeake Bay. Environmetrics, 13, ( ) Kedem, Fokianos (2002). Regression Models for Time Series Analysis, New York: Wiley. Kozintsev, Kedem (2000). Generation of Similar Images From a Given Discrete Image. J. Comp. Graphical Stat., 2000, Vol. 9, No. 2, Kozintseva (1999). Comparison of Three Methods of Spatial Prediction. M.A. Thesis, Department of Mathematics, University of Maryland, College Park. 3

4 Spatial/temporal geostatistical data often display: Non-Gaussian skewed sampling distributions. Positive continuous data. Heavy right tails. Bounded support. Small data sets observed irregularly (gaps). 4

5 Possible remedies: Bayesian Transformed Gaussian (BTG): A Bayesian approach combined with parametric families of nonlinear transformations to Gaussian data. BTG provides a unified framework for inference and prediction/interpolation in a wide variety of models, Gaussian and non- Gaussian. Will describe BTG and illustrate it using spatial and temporal data. 5

6 Stationary isotropic Gaussian random field. Let {Z(s)}, s D R d, be a spatial process or a random field. A random field {Z(s)} is Gaussian if for all s 1,...,s n D, the vector (Z(s 1 ),..., Z(s n )) has a multivariate normal distribution. {Z(s)} is (second order) stationary when for s,s + h D we have ( ) E(Z(s)) = µ, ( ) Cov(Z(s + h), Z(s)) C(h). The function C( ) is called the covariogram or covariance function. 6

7 We shall assume that C(h) depends only on the distance h between the locations s + h and s but not on the direction of h. In this case the covariance function as well as the process are called isotropic. The corresponding isotropic correlation function is given by K(l) = C(l)/C(0), where l is the distance between points. Useful special case: Matérn correlation K θ (l) = ( ) 1 l θ2 ( ) 2 θ 2 1 Γ(θ 2 ) θ κ l 1 θ2 θ if l if l = 0 where θ 1 > 0, θ 2 > 0, and κ θ2 is a modified Bessel function of the third kind of order θ 2. 7

8 Z K θ (l) = ( ) 1 l θ2 ( ) 2 θ 2 1 Γ(θ 2 ) θ κ l 1 θ2 θ if l if l = 0 Matérn (θ 1 = 8, θ 2 = 3) Y X

9 Z Spherical correlation: K θ (l) = ( lθ ) ( lθ ) 3 if l θ 0 if l > θ where θ > 0 controls the correlation range. Spherical (θ = 120) Y X 9

10 Z Exponential correlation: θ 1 (0,1), θ 2 (0,2] K θ (l) = exp(l θ 2 log θ 1 ) Exponential (θ 1 = 0.5, θ 2 = 1) Y X

11 Z Rational quadratic: θ 1 > 0, θ 2 > 0. K θ (l) = ( ) θ2 1 + l2 θ1 2 Rational quadratic (θ 1 = 12, θ 2 = 8) Y X 11

12 Clipped, at 3 levels, realizations from Gaussian random fields. Top left: Matérn (8,3). Top right: spherical (120). Bottom left: exponential (0.5,1). Bottom right: rational quadratic (12,8). generate.cgi?4 Kozintsev(1999), Kozintsev and Kedem(2000). 12

13 Ordinary Kriging. Given the data Z (Z(s 1 ),..., Z(s n )) observed at locations {s 1,...,s n } in D, the problem is to predict (or estimate) Z(s 0 ) at location s 0 using the best linear unbiased predictor (BLUP) obtained by minimizing E(Z(s 0 ) n i=1 λ i Z(s i )) 2 subject to n i=1 λ i = 1 13

14 Define Then 1 = (1,1,...,1), 1 n vector c = (C(s 0 s 1 ),..., C(s 0 s n )) C = (C(s i s j )), i, j = 1,..., n λ = (λ 1, λ 2,..., λ n ) ( ˆλ = C 1 c C 1 ) c 1 C The ordinary kriging predictor is then Ẑ(s 0 ) = ˆλ Z. 14

15 Define, m = 1 1 C 1 c 1 C 1 1 and the kriging variance σ 2 k (s 0) = E(Z(s 0 ) Ẑ(s 0 )) 2 = C(0) ˆλ c + m. Under the Gaussian assumption, Ẑ(s 0 ) ± 1.96σ k (s 0 ) is a 95% prediction interval for Z(s 0 ). For non- Gaussian fields this may not hold. 15

16 Bayesian Spatial Prediction: The BTG Model. RF {Z(s), s D} observed at s 1,...,s n D Parametric family of monotone transformations G = {g λ (.) : λ Λ}. Assumption: Z(.) can be transformed into a Gaussian random field by a member of G. A useful parametric family of transformations often used in applications to normalize positive data is the Box-Cox (1964) family of power transformations, g λ (x) = { x λ 1 λ if λ 0 log(x) if λ = 0. 16

17 For some unknown transformation parameter λ Λ, {g λ (Z(s)), s D} is a Gaussian random field with E{g λ (Z(s))} = p j=1 β j f j (s), cov{g λ (Z(s)), g λ (Z(u))} = τ 1 K θ (s,u), Regression parameters: β = (β 1,..., β p ) Covariates: f(s) = (f 1 (s),..., f p (s)) Variance: τ 1 = var{g λ (Z(s))} Simplifying assumption: Isotropy, K θ (s,u) = K θ ( s u ) θ = (θ 1,..., θ q ) Θ R q. 17

18 Data: Z obs = (Z 1,obs,..., Z n,obs ) g λ (Z i,obs ) = g λ (Z(s i )) + ǫ i ; i = 1,..., n, ǫ 1,..., ǫ n are i.i.d. N(0, ξ τ ). Parameters: η = (β, τ, ξ, θ, λ). Prediction problem: Predict Z 0 = (Z(s 01 ),..., Z(s 0k )) from the predictive density function, defined by p(z o z obs ) = = Ω p(z o, η z obs )dη Ω p(z o η,z obs )p(η z obs )dη, where Ω = R p (0, ) 2 Θ Λ. 18

19 Notation: For a = (a 1,..., a n ), we write g λ (a) (g λ (a 1 ),..., g λ (a n )). The Likelihood: L(η;z obs ) = ( τ )n 2 Ψξ,θ 12 exp { τ2 } 2π Q J λ, Q = ( g λ (z obs ) Xβ ) Ψ 1 ξ,θ ( gλ (z obs ) Xβ ). X n p design matrix, X ij = f j (s i ). Ψ ξ,θ = Σ θ + ξi, n n matrix. Σ θ;ij = K θ (s i,s j ). I identity matrix. J λ = n i=1 g (z i,obs ), the Jacobian. 19

20 The Prior. Insightful arguments in Box and Cox(1964), De Oliveira, Kedem, Short (1997), as well as practical experience lead us to the prior p(η) p(ξ)p(θ)p(λ) p, τj n where p(ξ), p(θ) and p(λ) are the prior marginals of ξ, θ and λ, respectively, which are assumed to be proper. λ Unusual prior: it depends on the data through the Jacobian. For more on prior selection see Berger, De Oliveira, Sansó (2001). 20

21 Simplifying assumption: No measurement noise (ξ = 0). g λ (Z i,obs ) = g λ (Z(s i )), i = 1,..., n, p(β, τ, θ, λ) p(θ)p(λ) τj p/n λ (1) η = (β, τ, θ, λ). Also, write z = z obs 21

22 The Posterior. p(η z) = p(β, τ, θ, λ z) = p(β, τ θ, λ,z)p(θ, λ z). To get the first factor: (β τ, θ, λ,z) N p (ˆβ θ,λ, 1 τ (X Σ 1 θ X) 1 ) where (τ θ, λ,z) Ga( n p 2, 2 q θ,λ ) ˆβ θ,λ = (X Σ 1 θ X) 1 X Σ 1 θ g λ(z) q θ,λ = (g λ (z) Xˆβ θ,λ ) Σ 1 θ (g λ(z) Xˆβ θ,λ ). and p(β, τ θ, λ,z) = p(β τ, θ, λ,z)p(τ θ, λ,z) is Normal-Gamma. 22

23 To get the second factor: p(θ, λ z) Σ θ 1/2 X Σ 1 θ X 1/2 q n p 2 θ,λ J1 p n λ p(θ)p(λ) In addition to the joint posterior distribution p(η z) derived above, the predictive density p(z o z) also requires p(z o η,z). We have p(z o η,z) = ( τ 2π )k/2 D θ 1/2 g λ (z oj) j=1 { exp τ 2 (g λ(z o ) M β,θ,λ ) D 1 θ (g λ(z) M β,θ,λ ) where M β,θ,λ,d θ are known. k } 23

24 We now have the integrand p(z o η,z)p(η z) needed for p(z o z). By integrating out β and τ we obtain the simplified form of the predictive density: p(z o z) = where = Λ Λ Θ p(z o θ, λ,z)p(θ, λ z)dθdλ Θ p(z o θ, λ,z)p(z θ, λ)p(θ)p(λ)dθdλ p(z θ, λ)p(θ)p(λ)dθdλ Λ Θ p(z o θ, λ,z) = Γ(n p+k 2 ) k j=1 g λ (z oj) Γ( n p 2 )πk/2 q θ,λ C θ 1/2 and from Bayes theorem, [1 + (g λ (z o ) m θ,λ ) ( q θ,λ C θ ) 1 (g λ (z o ) m θ,λ )] n p+k 2 p(z θ, λ) Σ θ 1/2 X Σ 1 θ where m θ,λ,c θ are known. n p X 1/2 q 2 θ,λ J1 p n λ 24

25 BTG Algorithm: Predictive Density Approximation 1. Let S = {z o (j) : j = 1,..., r} be the set of values obtained by discretizing the effective range of Z Generate independently θ 1,..., θ m i.i.d. p(θ) and λ 1,..., λ m i.i.d. p(λ). 3. For z o S, the approximation to p(z o z) is given by mi=1 p(z o θ ˆp m (z o z) = i, λ i,z)p(z θ i, λ i ) mi=1 p(z θ i, λ i ) p(z o θ, λ,z) and p(z θ, λ) given above. ( ) Ẑ 0 = Median of (Z 0 Z) 25

26 Software: tkbtg application. Hybrid of C++, Tcl/Tk, and FORTRAN 77 (Bindel et al (1997)). The tkbtg Interface Layout 26

27 Example: Spatial Rainfall Prediction Rain gauge positions and weekly rainfall totals in mm, Darwin, Australia, y x 27

28 1. Use the Box-Cox transformation family. 2. λ Unif( 3,3). 3. m = Correlation: Matérn and exponential. 5. No covariate information. Assume constant regression: E{g λ (Z(s))} = β Data apparently not normal Mean=33.94 Median=29.73 SD= Total Rainfall in mm 28

29 No. z ẑ 95% PI (10.70, 56.78) (15.29, 53.35) (20.93, 52.49) (18.49, 48.29) (9.53, 56.45) (16.46, 61.79) (15.40, 33.59) (11.52, 34.66) (28.25, 100) (18.01, 52.30) (15.62, 33.40) (15.35, 37.54) (44.14, 100) (18.58, 62.63) (14.00, 30.63) (13.43, 29.81) (19.25, 73.84) (16.57, 42.46) (10.79, 27.21) (20.33, 54.39) (14.71, 31.22) (11.64, 33.74) (12.21, 40.91) (10.85, 32.81) 29

30 Z Spatial prediction and contour maps from the Darwin data using Matérn correlation Y X y x 30

31 Z Spatial prediction and contour maps from the Darwin data using exponential correlation Y X y x 31

32 Predictive densities, 95% prediction intervals, and cross-validation: Predicting a true value from the remaining 23 observations using Matérn correlation. The vertical line marks the location of the true value. p(x) True: Median: (15.60, 53.02) p(x) True: Median: (41.52, 100) x x p(x) True: Median: 22.1 (13.26, 30.94) p(x) True: Median: (14.84, 31.26) x x 32

33 Comparison of BTG With Kriging and Trans- Gaussian kriging (Kozintseva (1999)). Cross validation results using artificial data on grid. Data obtained by transforming a Gaussian (0,1) RF using inverse Box-Cox transformation. In Kriging and TG kriging λ, θ, were known. Not in BTG (!) λ = 0: Log-Normal. λ = 1: Normal. λ = 0.5: Between Normal and Log-Normal. In most cases BTG has more reliable but larger prediction intervals. BTG predicts at the original scale. TG kriging does not. 33

34 Matérn(1,10) λ KRG MSE TGK MSE BTG MSE KRG AvePI TGK AvePI BTG AvePI KRG % out 100% 48% 6% TGK % out 18% 8% 6% BTG % out 12% 6% 6% Exponential(e 0.03,1) λ KRG MSE TGK MSE BTG MSE KRG AvePI TGK AvePI BTG AvePI KRG % out 98% 64% 2% TGK % out 20% 4% 2% BTG % out 6% 2% 2% 34

35 Application of BTG to Time Series Prediction. Short time series observed irregularly. Set: s = (x, y) = (t,0). Can predict/interpolate as in state space prediction: k step prediction forward, backward, and in the middle. Example 1: Monthly data of unemployed women 20 years of age and older, Data source: Bureau of Labor Statistics. N = 48. Example 2: Monthly airline passenger data, Data source: Box-Jenkins (1976). Use only N = 36 out of 144 observations, t = 51,...,86. 35

36 Example: Prediction of Monthly number of unemployed women (Age 20), Data in hundreds of thousands. Cross validation and 95% prediction intervals. Observations at times t = 12,13,36 are outside their 95% PI s. N = 48 1 = 47. Unemployed Women No. Unemployed Women True Predicted Lower Upper t 36

37 Forward and backward one and two step prediction in the unemployed women series. In 2 step higher dispersion. One step forward Two step forward p(x) True: Median: (17.24, 26.33) p(x) True: Median: (16.36, 28.05) x x One step backward Two step backward p(x) True: Median: (21.94, 30.40) p(x) True: Median: (19.77, 29.61) x x 37

38 Example: Time series of monthly international airline passengers in thousands, January December N=144. Seasonal time series. No. Passengers t 38

39 BTG cross validation and prediction intervals for the monthly airline passengers series, t = 51,..., 86, using Matérn correlation. Observations at t = 62,63 are outside the PI. N = 36 1 = 35. No. Passengers True Predicted Lower Upper t 39

40 Application to Rainfall: Heuristic Argument. Let X n represent the area average rain rate over a region such that X n = mx n 1 + λ + ǫ n, n = 1,2,3,, where the noise {ǫ n } is a martingale difference. It can be argued that necessary conditions for X n LogNormal, n, are that m 1 and λ

41 The monotone increase in ˆm (Curve a) and the monotone decrease in ˆλ (Curve b) as a function of the square root of the area. Source: Kedem and Chiu(1987). This suggests that the lognormal distribution as a model for averages or rainfall amounts over large areas or long periods. 41

42 It is interesting to obtain the posterior p(λ z) of λ, the transformation parameter in the Box- Cox family, given the data, where the data are weekly rainfall totals from Darwin, Australia. With a uniform prior for λ, the medians of p(λ z) in 5 different weeks are: Week 1 median = 0.45 (to the left of 0). Week 2 median = 0.45 (to the right of 0). Week 3 median = 0.15 (Not far from 0). Week 4 median = 0.20 (Not far from 0). Week 5 median = 0.95 (Close to 1). 42

43 Weekly posterior p(λ z) of λ given rainfall totals from Darwin, Australia, for 5 different weeks. (a) (b) density MAE = density MAE = lambda lambda (c) (d) density MAE = 0.15 density MAE = lambda lambda (e) density MAE = lambda 43