NRCSE. Statistics, data, and deterministic models

Transcription

1 NRCSE Statistics, data, and deterministic models

2 Some issues in model assessment Spatiotemporal misalignment Grid boxes vs observations Types of error Measurement error and bias Model error Approximation error Manipulate data or model output? Two case studies: SARMAP kriging MODELS-3 Bayesian melding Other uses of Bayesian hierarchical models

3 Assessing the SARMAP model 60 days of hourly observations at 32 sites in Sacramento region Hourly model runs for three episodes

4 Task Estimate from data the ozone level at x s in a grid square. Use sum to estimate integral over grid square. Issues: Transformation Diurnal cycle Temporal dependence Spatial dependence Space-time interaction

5 Transformation Heterogeneous variability mean and variance positively related Square root transformation All modeling now on square root scale approximately normal

6 Diurnal cycle

7 Temporal dependence

8 Spatial dependence

9 V t (s) = Estimate Estimating a grid square average Z t (s) V t (s) = µ t (s) + W t (s) W t (s) =! 1 (s)w t"1 (s) +! 2 (s)w t" 2 (s) + Y t (s) 1 A! A V 2 t (s)ds using 1! M E { V (s t j )2 data from 1,...,t } (not averages of squares of kriging estimates on the square root scale)

10 Looking at an episode

11 Afternoon comparison

12 Nighttime comparison

13 A Bayesian approach SARMAP study spatially data rich If spatially sparse data, how estimate grid squares? P = βz + M + A + δ O = (γ)z + B + E P = process model output O = observations Z = truth Calculate (Z P,O) for prediction Calculate (O P, β = 1, M = S = A = 0) for model assessment

14 CASTNet and Models-3 CASTNet is a dry deposition network Models-3 sophisticated air quality model Average fluxes on 36x36 km 2 grid Weekly data and hourly output

15 Estimated model bias The multiplicatice bias β is taken spatially constant (= 0.5). The additive bias E(M+A+δ) is spatially distributed.

16 Assessing model fit Predict CASTNet observation O i from posterior mean of prediction using Models-3 output P i and remaining observations O -I. Average length of 90% credible intervals is 7 ppb Average length using only Models-3 is 3.5 ppb

17 Crossvalidation

18 The Bayesian hierarchical approach Three levels of modelling: Data model: f(data process, parameters) Process model: f(process parameters) Parameter model: f(parameters) Use Bayes theorem to compute posterior f(process, parameters data)

19 Some applications Data assimilation Satellite tracking Precipitation measurement Combination of data on different scales Image analysis Agricultural field trials

20 Application to Models-3 (Z(s 0 ) P,O)! M % $ 1 M f(z(s 0 P,O,"(i) ) i=1 # f(z(s 0 ) P,O,")f(" P,O)d" where θ (I) are samples from the posterior distribution of θ ME IL NC IN FL MI CASTNet Models Adjusted Models

21 Predictions

22 NCAR-GSP (IMAGe) Theme for 2005: Data Assimilation in the Geosciences

23 ENSO project El Niño/Southern Oscillation is driven by surface temperature in tropical Pacific Data 2 o x2 o monthly SST anomalies at 2261 locations; zonal 10m wind Previous work indicates EOFs of SST may develop in a Markovian fashion Forecast 7 months ahead uses data from Jan 70 through latest available. Cressie-Wikle-Berliner ( ~sses/collab_enso.php)

24 Model Data model: EOFs Z t =!a t + " t Process model: a t+! = µ t + H t a t + " t+! H t = H(I t,j t ) Parameter model: regimes winds The current state is a mixture over three regimes (determined by SOI), with mixing probabilities that depend on the wind statistic Standardize by subtracting climatology (monthly average )

25 Latest ENSO forecast

26 forecast Latest forecast with data data

27 Relative performance Performance measure for anomalies: ave((forecast - data) 2 } over all pixels in Niño3.4-region Relative Performance of Forecast A relative to Forecast B is RP(A,B)=log(Perf B / Perf A) RP(A,B)>0 indicates A better than B Persistence: Predict using data 7 months ago Climatology: Predict using 0

28 Comparison to climatology and persistence