Fusing space-time data under measurement error for computer model output

Transcription

1 for computer model output SAMSI joint work with Alan E. Gelfand and David M. Holland

2 Introduction In many environmental disciplines data come from two sources: monitoring networks and numerical models Numerical models are deterministic mathematical models used to predict environmental spatio-temporal processes Describe the underlying physical and chemical processes via partial differential equations Equations solved via numerical methods by discretizing space and time Predictions are given in terms of averages over grid cells

3 Introduction (a) Observed ozone concentration (b) Predicted ozone concentration Sparse locations Missing data Essentially, true value Large spatial domains No missingness Calibration concerns

4 Our goal Fuse the two sources of data Address the following issues Spatial scale of outputs from numerical models Outputs from numerical models are given in terms of predictions over grid cells, but predictions at points are more useful Calibration of numerical models Correct outputs from numerical models Problem: Comparing averages over grid cells with points

5 Downscaler: main idea (a) Observed ozone concentration (b) Predicted ozone concentration

6 Downscaler: main idea (a) Observed ozone concentration (b) Predicted ozone concentration

7 Downscaler: main idea (a) Observed ozone concentration (b) Predicted ozone concentration To each point s in the domain S with observation Y (s) we associate the numerical model output at grid cell B, x(b), where B is such that s B: Y (s) x(b)

8 Downscaler Time t is fixed. Y (s) observation at point s, x(b) numerical model output at grid cell B. For s in B: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s) with β i (s) = β i + β i (s), i=0,1. β 0 (s) and β 1 (s) correlated mean-zero GP. ε(s) ind N(0,τ 2 )

9 Downscaler Time t is fixed. Y (s) observation at point s, x(b) numerical model output at grid cell B. For s in B: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s) with β i (s) = β i + β i (s), i=0,1. β 0 (s) and β 1 (s) correlated mean-zero GP. ε(s) ind N(0,τ 2 ) Extension to space-time: For s in B and for each t Y (s,t) = β 0 (s,t)+ β 1 (s,t)x(b,t)+ε(s,t) with β i (s,t) = β i,t + β i (s,t), i=0,1. Temporal dependence in β i,t and β i (s,t): dynamic or independent ε(s,t) ind N(0,τ 2 )

10 Downscaler Driven by true station data rather than uncalibrated model output Computationally feasible also for large spatial domains Allows local calibration of the numerical model output Endows the spatial process Y (s) with a non-stationary covariance structure Straightforward prediction at an unmonitored sites Better predictive performance than other methods (geostatistical and model-based)

11 Extending the downscaler Improve the input to the downscaler provided by the numerical model output accounting for uncertainty in the numerical model output accounting for uncertainty in the association x(b) Y (s) with s in B

12 Extending the downscaler Improve the input to the downscaler provided by the numerical model output accounting for uncertainty in the numerical model output accounting for uncertainty in the association x(b) Y (s) with s in B Will consider two possibilities: a Measurement Error Model (MEM) a Smoother with spatially-varying weights

13 Static setting

14 A MEM for outputs from computer model Downscaler: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s)

15 A MEM for outputs from computer model Downscaler: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s) Model the numerical model output as a stochastic process x(b) = Ṽ (B) + η(b) η(b) ind N(0,σ 2 ) with Ṽ (B) = µ + V (B) and V (B) ICAR(ξ2,W ).

16 A MEM for outputs from computer model Downscaler: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s) Model the numerical model output as a stochastic process x(b) = Ṽ (B) + η(b) η(b) ind N(0,σ 2 ) with Ṽ (B) = µ + V (B) and V (B) ICAR(ξ2,W ). For s B: Y (s) = β 0 (s) + β 1 Ṽ (B) + ε(s) ε(s) ind N(0,τ 2 ) with β 0 (s) = β 0 + β 0 (s) and β 0 (s) mean-zero GP with exponential covariance function.

17 A MEM for outputs from computer model Downscaler: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s) Model the numerical model output as a stochastic process x(b) = Ṽ (B) + η(b) η(b) ind N(0,σ 2 ) with Ṽ (B) = µ + V (B) and V (B) ICAR(ξ2,W ). For s B: Y (s) = β 0 (s) + β 1 Ṽ (B) + ε(s) ε(s) ind N(0,τ 2 ) with β 0 (s) = β 0 + β 0 (s) and β 0 (s) mean-zero GP with exponential covariance function. Then: X MEM Ṽ downscaler Y

18 A MEM for outputs from computer model (a) Observed ozone concentration (b) Predicted ozone concentration

19 A MEM for outputs from computer model (a) Observed ozone concentration (b) Predicted ozone concentration

20 A MEM for outputs from computer model (a) Observed ozone concentration (b) Predicted ozone concentration

21 A MEM for outputs from computer model (a) Observed ozone concentration (b) Predicted ozone concentration

22 A MEM for outputs from computer model /5 1/5 1/5 1/5 1/ (a) Observed ozone concentration (b) Predicted ozone concentration To each point s in the domain S with observation Y (s) we associate the latent Gaussian Markov Random Field Ṽ (B), where B is such that s B and Ṽ (B) also associated with x(b) Y (s) x(b ),B δb

23 Smoother with spatially-varying weights Downscaler: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s)

24 Smoother with spatially-varying weights Downscaler: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s) The numerical model output, {x(b)}, is NOT modeled as a stochastic process. For s B : Y (s) = β 0 (s) + β 1 x(s) + ε(s) ε(s) ind N(0,τ 2 ) with x(s) = g k=1 w k(s)x(b k ), and w k (s) random and spatially-varying.

25 Smoother with spatially-varying weights Downscaler: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s) The numerical model output, {x(b)}, is NOT modeled as a stochastic process. For s B : Y (s) = β 0 (s) + β 1 x(s) + ε(s) ε(s) ind N(0,τ 2 ) with x(s) = g k=1 w k(s)x(b k ), and w k (s) random and spatially-varying. Let r k, k = 1,...,g be the centroids of the numerical model grid cells B k, then w k (s) := K (s r k;ψ) exp(q(r k )) g l=1 K (s r l;ψ) exp(q(r l ))

26 Smoother with spatially-varying weights Downscaler: Y (s) = β 0 (s) + β 1 (s)x(b) + ε(s) The numerical model output, {x(b)}, is NOT modeled as a stochastic process. For s B : Y (s) = β 0 (s) + β 1 x(s) + ε(s) ε(s) ind N(0,τ 2 ) with x(s) = g k=1 w k(s)x(b k ), and w k (s) random and spatially-varying. Let r k, k = 1,...,g be the centroids of the numerical model grid cells B k, then w k (s) := K (s r k;ψ) exp(q(r k )) g l=1 K (s r l;ψ) exp(q(r l )) Q( ) mean-zero GP with variance σ 2 Q, exponential covariance function. K (s r k ;ψ) = exp( ψ s r k ).

27 Smoother with spatially-varying weights (a) Observed ozone concentration (b) Predicted ozone concentration

28 Smoother with spatially-varying weights (a) Observed ozone concentration (b) Predicted ozone concentration

29 Smoother with spatially-varying weights (a) Observed ozone concentration (b) Predicted ozone concentration

30 w_6676 w_6675 w_6674 w_6673 w_6672 w_6671 w_6670 w_6669 w_6688 w_6687 w_6686 w_6685 w_6684 w_6683 w_6682 w_6681 w_6680 w_6679 w_6678 w_6677 w_6864 w_6863 w_6862 w_6861 w_6860 w_6859 w_6858 w_6857 w_6876 w_6875 w_6874 w_6873 w_6872 w_6871 w_6870 w_6869 w_6868 w_6867 w_6866 w_6865 w_7053 w_7052 w_7051 w_7050 w_7049 w_7048 w_7047 w_7046 w_7045 w_7064 w_7063 w_7062 w_7061 w_7060 w_7059 w_7058 w_7057 w_7056 w_7055 w_7054 w_7242 w_7241 w_7240 w_7239 w_7238 w_7237 w_7236 w_7235 w_7234 w_7233 w_7252 w_7251 w_7250 w_7249 w_7248 w_7247 w_7246 w_7245 w_7244 w_7243 w_7430 w_7429 w_7428 w_7427 w_7426 w_7425 w_7424 w_7423 w_7422 w_7421 w_7440 w_7439 w_7438 w_7437 w_7436 w_7435 w_7434 w_7433 w_7432 w_7431 w_7609 w_7619 w_7618 w_7617 w_7616 w_7615 w_7614 w_7613 w_7612 w_7611 w_7610 w_7628 w_7627 w_7626 w_7625 w_7624 w_7623 w_7622 w_7621 w_7620 w_7798 w_7797 w_7807 w_7806 w_7805 w_7804 w_7803 w_7802 w_7801 w_7800 w_7799 w_7816 w_7815 w_7814 w_7813 w_7812 w_7811 w_7810 w_7809 w_7808 w_7986 w_7985 w_7995 w_7994 w_7993 w_7992 w_7991 w_7990 w_7989 w_7988 w_7987 w_8004 w_8003 w_8002 w_8001 w_8000 w_7999 w_7998 w_7997 w_7996 w_8176 w_8175 w_8174 w_8173 w_8184 w_8183 w_8182 w_8181 w_8180 w_8179 w_8178 w_8177 w_8192 w_8191 w_8190 w_8189 w_8188 w_8187 w_8186 w_8185 w_8364 w_8363 w_8362 w_8361 w_8372 w_8371 w_8370 w_8369 w_8368 w_8367 w_8366 w_8365 w_8380 w_8379 w_8378 w_8377 w_8376 w_8375 w_8374 w_8373 w_8552 w_8551 w_8550 w_8549 w_8560 w_8559 w_8558 w_8557 w_8556 w_8555 w_8554 w_8553 w_8568 w_8567 w_8566 w_8565 w_8564 w_8563 w_8562 w_8561 w_8742 w_8741 w_8740 w_8739 w_8738 w_8737 w_8749 w_8748 w_8747 w_8746 w_8745 w_8744 w_8743 w_8756 w_8755 w_8754 w_8753 w_8752 w_8751 w_8750 w_8930 w_8929 w_8928 w_8927 w_8926 w_8925 w_8937 w_8936 w_8935 w_8934 w_8933 w_8932 w_8931 w_8944 w_8943 w_8942 w_8941 w_8940 w_8939 w_8938 w_9118 w_9117 w_9116 w_9115 w_9114 w_9113 w_9125 w_9124 w_9123 w_9122 w_9121 w_9120 w_9119 w_9132 w_9131 w_9130 w_9129 w_9128 w_9127 w_9126 w_9306 w_9305 w_9304 w_9303 w_9302 w 9301 w_9313 w_9312 w_9311 w_9310 w_9309 w_9308 w_9307 w_9320 w_9319 w_9318 w_9317 w_9316 w_9315 w_9314 w_ w_94 w 948 w w w_ w_ w_9 w_94 w_949 Smoother with spatially-varying weights (a) Observed ozone concentration (b) Predicted ozone concentration To each point s in the domain S with observation Y (s) we associate x(s), where x(s)= g k=1 w k(s)x(b k ) Y (s) x(s)

31 Spatio-temporal setting

32 A MEM for outputs from computer model Y (s,t) observations at s and time t, x(b,t) numerical model output at grid cell B and time t x(b,t) = Ṽ (B,t) + η(b,t) with Ṽ (B,t) = µ t + V (B,t) and V t = {V (B,t)}. ind η(b,t) N(0,σ 2 )

33 A MEM for outputs from computer model Y (s,t) observations at s and time t, x(b,t) numerical model output at grid cell B and time t x(b,t) = Ṽ (B,t) + η(b,t) with Ṽ (B,t) = µ t + V (B,t) and V t = {V (B,t)}. For s B: ind η(b,t) N(0,σ 2 ) Y (s,t) = β 0 (s,t) + β 1,t Ṽ (B,t) + ε(s,t) ε(s,t) ind N(0,τ 2 ) with β 0 (s,t) = β 0,t + β 0 (s,t).

34 A MEM for outputs from computer model Y (s,t) observations at s and time t, x(b,t) numerical model output at grid cell B and time t x(b,t) = Ṽ (B,t) + η(b,t) with Ṽ (B,t) = µ t + V (B,t) and V t = {V (B,t)}. For s B: ind η(b,t) N(0,σ 2 ) Y (s,t) = β 0 (s,t) + β 1,t Ṽ (B,t) + ε(s,t) ε(s,t) ind N(0,τ 2 ) with β 0 (s,t) = β 0,t + β 0 (s,t). Temporal dependence in β i,t, i = 0,1, β 0 (s,t) and V t : either (i) independent in time, or (ii) dynamic.

35 Smoother with spatially-varying weights Y (s,t) observations at s and time t, x(b,t) numerical model output at grid cell B and time t For s B : Y (s,t) = β 0 (s,t) + β 1,t x(s,t) + ε(s,t) ε(s,t) ind N(0,τ 2 ) with β 0 (s,t) = β 0,t + β 0 (s,t), x(s,t) = g k=1 w k,t(s)x(b k,t), and w k,t (s) random and varying both in space and time.

36 Smoother with spatially-varying weights Y (s,t) observations at s and time t, x(b,t) numerical model output at grid cell B and time t For s B : Y (s,t) = β 0 (s,t) + β 1,t x(s,t) + ε(s,t) ε(s,t) ind N(0,τ 2 ) with β 0 (s,t) = β 0,t + β 0 (s,t), x(s,t) = g k=1 w k,t(s)x(b k,t), and w k,t (s) random and varying both in space and time. w k,t (s) := K (s r k;ψ) exp(q(r k,t)) g l=1 K (s r l;ψ) exp(q(r l,t))

37 Smoother with spatially-varying weights Y (s,t) observations at s and time t, x(b,t) numerical model output at grid cell B and time t For s B : Y (s,t) = β 0 (s,t) + β 1,t x(s,t) + ε(s,t) ε(s,t) ind N(0,τ 2 ) with β 0 (s,t) = β 0,t + β 0 (s,t), x(s,t) = g k=1 w k,t(s)x(b k,t), and w k,t (s) random and varying both in space and time. w k,t (s) := K (s r k;ψ) exp(q(r k,t)) g l=1 K (s r l;ψ) exp(q(r l,t)) Temporal dependence in β i,t, i = 0,1, β 0 (s,t) and Q(s,t): either (i) independent in time, or (ii) dynamic.

38 Application Training sites Validation sites Daily maximum 8-hour ozone concentration (ppb): observations and CMAQ model output. Model output on 12-km grid cells (g =40,044). Period: June 1- August 31, monitoring sites: 400 training sites and 400 validation sites Transformation to square root scale; predictions were back-transformed Time-varying parameters: independent in time

39 Results Assessed predictive performance at the 400 validation sites for the period June 1 - August 31, Coverage Avg. length Method MSPE MAPE 95% PI 95% PI Downscaler % 31.6 MEM % 29.1 Smoother % 28.8

40 Results Downscaler: x(b,t) Y (s,t) MEM: Ṽ (B,t) Y (s,t) Smoother: x(s,t) Y (s,t)

41 Results Downscaler: x(b,t) Y (s,t) MEM: Ṽ (B,t) Y (s,t) Smoother: x(s,t) Y (s,t) Computed the correlation between Y (s,t) and: (i) x(b,t) and the posterior predictive mean of (ii) Ṽ (B,t) and (iii) x(s,t).

42 Results Downscaler: x(b,t) Y (s,t) MEM: Ṽ (B,t) Y (s,t) Smoother: x(s,t) Y (s,t) Computed the correlation between Y (s,t) and: (i) x(b,t) and the posterior predictive mean of (ii) Ṽ (B,t) and (iii) x(s,t). Posterior mean of Posterior mean Day x(b,t) Ṽ (B,t) of x(s,t) July 4, July 20, August 9,

43 Results Posterior mean of the random and spatially and temporally- varying weights w k,t (s) for July 4, 2001 at four sites s3 s4 s2 s

44 Results Posterior mean of the random and spatially and temporally- varying weights w k,t (s) for July 4, 2001 at four sites

45 Results Posterior mean of the random and spatially and temporally- varying weights w k,t (s) for July 4, 2001, July 20, and August 9, 2001 at two sites

46 Conclusions Presented two methods to extend the downscaler model to fuse numerical model ouput and monitoring data Both are computationally feasibile even for large computer model output (smoother + predictive processes (Banerjee et al., JRSS B, 2008)) Smoother with spatially-varying weights yielded better predictions than other methods Clear directionality in the weights of the smoother model