Bayesian dynamic modeling for large space-time weather datasets using Gaussian predictive processes

Size: px

Start display at page:

Download "Bayesian dynamic modeling for large space-time weather datasets using Gaussian predictive processes"

Suzan Stanley
5 years ago
Views:

1 Bayesian dynamic modeling for large space-time weather datasets using Gaussian predictive processes Alan Gelfand 1 and Andrew O. Finley 2 1 Department of Statistical Science, Duke University, Durham, North Carolina. 2 Department of Forestry, Michigan State University, Lansing, Michigan. September 9,

2 Collaborators Collaborators Sudipto Banerjee, School of Public Health, University of Minnesota. Alan Gelfand, Department of Statistical Science, Duke University. This talk draws from: Finley, Banerjee, Gelfand (2012) Bayesian dynamic modeling for large space-time datasets using Gaussian predictive processes. Journal of Geographical Systems. 14: Special Issue: In Honor of James P. LeSage Graybill/ENVR Conference Short Course

3 Outline Collaborators 1 Opportunities and challenges in spatio-temporal data analysis 2 Typical examples 3 Dynamic space-time modeling 4 Dimension reduction and predictive spatial process models 5 Data analysis 6 Extensions to multivariate setting Graybill/ENVR Conference Short Course

4 Opportunities and challenges in spatio-temporal data analysis Challenges in spatio-temporal data analysis Data sets often exhibit: missingness and misalignment among outcomes space- and time-varying impact of covariates complex residual dependence structures nonstationarity among multiple outcomes across locations unknown time and perhaps space lags between outcomes and covariates We seek modeling frameworks that: incorporate many sources of space and time indexed data accommodate structured residual dependence propagate uncertainty through to predictions scale to effectively exploit information in massive datasets Graybill/ENVR Conference Short Course

5 Typical examples Discrete-time, continuous-space settings Data is viewed as arising from a time series of spatial processes. For example: US Environmental Protection Agency s Air Quality System which reports pollutants mean, minimum, and maximum at 8 and 24 hour intervals soil sensor array that measures moisture and CO 2 at 1 minute intervals climate model outputs of weather variables generated on hourly or daily intervals remotely sensed landuse/landcover change recorded at annual or decadal time steps Graybill/ENVR Conference Short Course

6 Typical examples NCDC weather station data from North-eastern US 1530 stations that recorded temperature between 1/1/2000 1/1/2005. The outcome variable is mean monthly temperature. Northing (km) Elevation (m) Northing (km) Jan 2000 Temperature degrees C Easting (km) Graybill/ENVR Conference Short Course

7 Typical examples NCDC weather station data from North-eastern US Of the n N t = 1530 station 61 months = 93,330 possible observations, 15,107 had missing outcomes. Station count Percent missing temperature observations Graybill/ENVR Conference Short Course

8 Dynamic space-time modeling Dynamic space-time modeling West and Harrison (1997); Tonellato; (1997); Stroud et al. (2001); Gelfand et al. (2005);... Measurement equation: Regression specification + space-time process + serially and spatially uncorrelated zero-centered Gaussian disturbances as measurement error Transition equation: Markovian dependence in time temporal component: akin to usual state-space model space-time component: modeled using Gaussian processes Graybill/ENVR Conference Short Course

9 Dynamic space-time modeling Dynamic space-time model y t (s) = x t (s) β t + u t (s) + ɛ t (s), β t = β t 1 + η t, u t (s) = u t 1 (s) + w t (s), ɛ t (s) ind N(0, τ 2 t ). η t iid N(0, Σ η ) ; t = 1, 2,..., N t w t (s) ind GP (0, C t (, θ t )), Assume β 0 N(m 0, Σ 0 ) and u 0 (s) Graybill/ENVR Conference Short Course

10 Dynamic space-time modeling Consider settings where the number of locations is large but the number of time points is manageable. For any S = {s 1, s 2,..., s n } R d. Then, w t = (w t (s 1 ), w t (s 2 ),..., w t (s n )) N(0, C t (θ t )), where C t (θ t ) = {C t (s i, s j ; θ t )}. Model parameter estimation requires iterative matrix decomposition for C t (θ t ) 1 and determinant, i.e., computation of order O(n 3 ) for every MCMC iteration! Conducting full Bayesian inference is computationally onerous! Graybill/ENVR Conference Short Course

11 Dimension reduction and predictive spatial process models Modeling large spatial datasets Covariance tapering (Furrer et al. 2006; Zhang and Du 2007; Du et al. 2009; Kaufman et al., 2009) Spectral domain: (Fuentes 2007; Paciorek 2007) Approximate using MRFs: INLA (Rue et al. 2009) Approximations using cond. indep. (Vecchia 1988; Stein et al. 2004) Low-rank approaches (Wahba 1990; Higdon 2002; Lin et al. 2000; Kamman & Wand, 2003; Paciorek 2007; Rasmussen & Williams 2006; Tokdar et al. 2007, 2011; Stein 2007, 2008; Cressie & Johannesson 2008; Banerjee et al. 2008, 2011; Finley et al ) Graybill/ENVR Conference Short Course

12 Dimension reduction and predictive spatial process models Predictive process models Wahba (1990), Lin et al. (2000), Rasmussen and Williams (2006), Tokdar (2007, 2011) and Banerjee, Finley et al. ( ). Parent process: w t (s) GP (0, C t ( ; θ t )) Knots : S = {s 1, s 2,..., s n} with n << n w t = (w t (s { 1 ), w t(s 2 ),... }, w t(s n )) N(0, C t (θ t )), where C t (θ t ) = C t (s i, s j ; θ t) w t (s) = E[w{ t (s) w t ] = c} t (s; θ t ) C t (θ t ) 1 w t, where c t (s; θ t ) = C t (s, s j ; θ t) Replace u t (s) by ũ t (s) = ũ t 1 (s) + w t (s). Now we only have order O(n 3 ) complexity for every MCMC iteration Graybill/ENVR Conference Short Course

13 Dimension reduction and predictive spatial process models y y Illustration of biased estimates of variance parameters from the predictive process (synthetic data). Parent process surface Predictive process surface x x Graybill/ENVR Conference Short Course

14 Dimension reduction and predictive spatial process models Predictive process model estimates (synthetic data) σ 2 τ 2 True parameter values σ 2 =5 and τ 2 =5. Increasing knot intensity reduces bias Graybill/ENVR Conference Short Course

15 Dimension reduction and predictive spatial process models Bias-adjusted predictive process w ɛ (s) ind N( w(s), C(s, s; θ) c(s; θ) C (θ) 1 c(s; θ)). Rectified predictive process model estimates (synthetic data) σ 2 τ 2 So, in our current setting replace u t (s) by ũ t (s) = ũ t 1 (s) + w t (s) + ɛ t (s), where again, ɛ(s) ind N(0, C t (s, s; θ) c t (s; θ t ) C t (θ t ) 1 c t (s; θ t )) Graybill/ENVR Conference Short Course

16 Dimension reduction and predictive spatial process models Hierarchical dynamic predictive process models N t N t p (θ t ) IG ( τt 2 ) a τ, b τ N(β0 m 0, Σ 0 ) IW (Σ η r η, Υ η ) t=1 t=1 N t N t N(β t β t 1, Σ η ) N(w t 0, C t (θ t )) t=1 t=1 N t N(ũ t ũ t 1 + c t (θ t ) C t (θ t ) 1 w t, D t ) t=1 N t N(y t X t β t + ũ t, τt 2 I n ). t= Graybill/ENVR Conference Short Course

17 Dimension reduction and predictive spatial process models Selection of knots How do we choose S Knot selection Regular grid? More knots near locations we have sampled more? formal spatial design paradigm: maximize information metrics geometric considerations: space-filling designs; various clustering algorithms adaptive modeling of knots using point processes, see, e.g., Guhaniyogi et al. (2011) In practice: compare performance of estimation of range and smoothness by varying knot intensity usually inference is quite robust to S. more important to capture loss of variability due to low rank Graybill/ENVR Conference Short Course

18 Data analysis Candidate models: NCDC weather station data analysis non-spatial, i.e., ũ t s = 0, but temporally dependent β t s full model with spatial predictive process using 5, 10, 25, and 50 knot intensities Assessment: model fit minimum posterior predictive loss approach predictive performance based on 1000 holdout observations RMSPE and CI coverage rate Graybill/ENVR Conference Short Course

19 Data analysis Elevation covariate Northing (km) Elevation (m) Easting (km) Graybill/ENVR Conference Short Course

20 Data analysis Knot locations Northing (km) Stations Predictive process knots Easting (km) Graybill/ENVR Conference Short Course

21 Data analysis Dynamic estimation of the intercept β 0,t (25 knot model) Beta Years Graybill/ENVR Conference Short Course

22 Data analysis Dynamic estimation of the impact of elevation β 1,t (25 knot model) Beta Years Graybill/ENVR Conference Short Course

23 Data analysis Non-spatial Predictive process knots 25 Σ η [1,1] (17.97, 36.65) (16.83, 35.45) Σ η [2,1] (-0.050, 0.054) (-0.050, 0.055) Σ η [2,2] (0.001, 0.002) (0.001, 0.002) Σ η parameter credible intervals, 50 (2.5, 97.5) percentiles, for the non-spatial and 25 knot predictive process models Graybill/ENVR Conference Short Course

24 Data analysis Dynamic estimation of spatial range θ 1 (25 knot model) Spatial range (km) Years Graybill/ENVR Conference Short Course

25 Data analysis Dynamic estimation of partial sill θ 2 (25 knot model) sigmaˆ Years Graybill/ENVR Conference Short Course

26 Data analysis Dynamic estimation of nugget τ 2 t (25 knot model) tauˆ Years Graybill/ENVR Conference Short Course

27 Data analysis Fitted values Mean Non-spatial model Mean Full model (25 knots) Observed Temp. Observed Temp Graybill/ENVR Conference Short Course

28 Data analysis Predictive process knots Non-spatial G P D RMSPE CI cover % Using a squared error loss function, G lack of fit, P predictive variance, D = G + P lower is better (Gelfand and Ghosh, 1998) Graybill/ENVR Conference Short Course

29 Data analysis Graybill/ENVR Conference Short Course

30 Extensions to multivariate setting Extending this model to consider m outcome variables, e.g., temperature and precipitation, at each location. Simply choose your favorite approach to modeling w t (s) = (w t,1 (s), w t,2 (s),..., w t,m (s)). LMC w t (s) Multivariate Matérn Constructive Kernel Convolution Characterize Latent Dimension Graybill/ENVR Conference Short Course

31 Summary Extensions to multivariate setting Challenge to meet modeling needs: predictive process models for large datasets use model-based adjustment to compensate for over-smoothing Future work: extend to multivariate response and spatially-varying coefficients make these models available in the R package spbayes Graybill/ENVR Conference Short Course

Bayesian dynamic modeling for large space-time weather datasets using Gaussian predictive processes

Bayesian dynamic modeling for large space-time weather datasets using Gaussian predictive processes Sudipto Banerjee 1 and Andrew O. Finley 2 1 Biostatistics, School of Public Health, University of Minnesota,