Multi-resolution models for large data sets

Transcription

1 Multi-resolution models for large data sets Douglas Nychka, National Center for Atmospheric Research National Science Foundation NORDSTAT, Umeå, June, 2012

2 Credits Steve Sain, NCAR Tia LeRud, UC Davis Dorit Hammerling, U Michigan Soutir Bandyopadhyay, Lehigh Finn Lindgren, NTNU, Norway D. Nychka LatticeKrig 2

3 Outline A climate data set Kriging... Compact basis functions (Φ), Markov Random fields (H) The multi-resolution model Properties of the spatial process A climate example Key idea: Introduce sparse basis and precision matrices without compromising the spatial model. D. Nychka LatticeKrig 3

4 Observed mean summer precipitation 1720 stations reporting, mean for Observed JJA Precipitation (.1 mm) D. Nychka LatticeKrig 4

5 Kriging or Gaussian spatial process estimates D. Nychka LatticeKrig 5

6 Estimating a curve or surface. An additive statistical model: Given n pairs of observations (x i, y i ), i = 1,..., n y i = g(x i ) + ɛ i ɛ i s are random errors and g is an unknown, smooth realization of a Gaussian process. The goals: Estimate g(x) based on the observations Quantify the uncertainty in the estimate. D. Nychka LatticeKrig 6

7 Random Effects/Linear model for g {Φ j }: m basis functions g(x) = j Φ j (x)c j A linear model: y = Φc + ɛ Random effects: c MN(0, ρp ) and ɛ MN(0, σ 2 I) Implied Covariance: E[g(x)g(x )] = j,k Φ j (x)ρp j,k Φ k (x ) λ = σ 2 /ρ plays an important role as a parameter. D. Nychka LatticeKrig 7

8 Key ideas for large data sets Inverse of P chosen to be sparse. Basis functions have compact support. Still have a useful spatial model! Why this works Find c by: Ridge regression/ conditional expectation/blue ĝ(x) = E[g(x) y, P ] = k=1,n ĉ k Φ k (x) ĉ = ( Φ T Φ + λp 1) 1 Φ T y, λ = σ 2 /ρ Φ T, Φ T Φ, P 1 are sparse. D. Nychka LatticeKrig 8

9 Choosing the basis and P D. Nychka LatticeKrig 9

10 A recipe for radial basis functions Basis function j (x) = ϕ( x u j /θ) ϕ is a positive definite, compactly supported function a nice bump. {u j } basis centers on a regular grid θ scale set to provide some overlap 2-d Wendland bump function Standard Wendland (order=2) Radial 2-d function D. Nychka LatticeKrig 10

11 A recipe for P 1 Markov random field among coefficients: c is a spatial AR 1 (4 + κ 2 )c j l N c l = e j {e j } are uncorrelated N(0,1) and N is 4 nearest neighbors. Hc MN(0, I) or c MN(0, (H T H) 1 ) i.e. P = (H T H) 1 ) Weights in lattice format: (4 + κ 2 ) D. Nychka LatticeKrig 11

12 What about SPDEs? An alternative way to define Q = P 1 is Q j,k = LΦ j LΦ k dx where L is the differential operator associated with the SPDE. D. Nychka LatticeKrig 12

13 Realizations of c Simulated fields on a lattice κ =.5 κ =.1 κ =.01 Note: κ acts as range parameter D. Nychka LatticeKrig 13

14 Combining basis and coefficients g(x) = j Φ j (x)c j Coefficient field and evaluated surface Normalized to give a constant marginal variance: E[g(x) 2 )] = j Φ j (x) 2 P j,j = 1 Overlap set to 2.5 units of lattice. D. Nychka LatticeKrig 14

15 A convolution interpretation g(x) = j Φ j (x)c j = j ϕ((x j u j )/θ)c(u j ) = (1/m) j ϕ((x j u j )/θ)(m)c(u j ) ϕ((x u)/θ)c(u)du Note: this only makes sense if centers are on a uniform grid. D. Nychka LatticeKrig 15

16 Generalizing to a multi-resolution basis D. Nychka LatticeKrig 16

17 A 1-d Multi-Res cartoon... First level: 8 basis functions Second level: 16 basis function More levels: Increasing by factor of 2... D. Nychka LatticeKrig 17

18 Example of multi-resolution in 2d An example on the unit square starting with grid: First level: centers on grid scale of 2.5/10 = = 121 basis functions Second level: centers on unit grid scale of 2.5/ = 441 basis functions. Four level multi-resolution for this case has 8804 basis functions. D. Nychka LatticeKrig 18

19 Some assumptions: The multiresolution: g(x) = g 1 (x) + g 2 (x) g L (x) = Φ 1 j (x)c1 j + Φ 2 j (x)c2 j Φ L j (x)cl j (1) Coefficients at each level follow a Markov Random field Coefficients between levels are independent At least two parameters: (κ l, ρ l ) at each level Correlation functions for each level correlation log (1 correlation) 1e 05 1e 03 1e distance distance D. Nychka LatticeKrig 19

20 Properties of spatial process D. Nychka LatticeKrig 20

21 Flexibility of LatticeKrig model Fitting an exponential (minimizing mean squared error) First level resolution of levels, 4 levels, target exponential correlation Error distance Distance Also works well for approximating smoother covariances. D. Nychka LatticeKrig 21

22 More Flexibility of LatticeKrig model Fitting a mixture of exponentials First level resolution of levels, 4 levels, target:.4exp(.1) +.6Exp(3) Correlation Error Distance Distance D. Nychka LatticeKrig 22

23 Some Theory Switch to an infinite sum of independent convolution processes. g(x) = g 1 (x) + g 2 (x) + g 3 (x) +... g l (x) has marginal variance ρ l and the spatial correlation range is θ l. What class of covariances can be approximated by letting {ρ l } and {θ l } vary? θ l = 2 l gives the power of 2 scaling of the multi-resolution. ρ l = 2 l/2 gives a process with similar smoothness as an exponential (matches tail behavior of spectral density.) D. Nychka LatticeKrig 23

24 The details: A theorem θ l = 2 l and ρ l = θ β 1 l c l (u) follows a Matern process with smoothness (ν) 1 and range θ l. ϕ is a K th order Wendland function. β 1 > 0, (β 1 + 1) < 5 + 2K If S(r) is the spectral density for the process g then C 1 r 2(β 1+1) < S(r) < C 2 r 2(β 1+1) as r. Comments: θ l = 2 l gives the power of 2 scaling of the multi-resolution. β 1 = 1/2 gives an exponential covariance-like spectral density. The tail behavior is not directly related to the smoothness of the basis functions or of the lattice process! They just need to be smooth enough. Approximation is accurate after about 4-6 components. D. Nychka LatticeKrig 24

25 Benefits of the multi-resolution Mixture of different scaled covariance functions can approximate standard covariance famlies. A mixture has the flexibility to approximate more complex covariance functions For irregularly spaced observational data the distances among station may vary widely and a multi-scale covariance model will adjust to these differences. D. Nychka LatticeKrig 25

26 Back to climate data D. Nychka LatticeKrig 26

27 Some details Used log transformation and weighted by number of observations Used stereographic projection for locations Elevation included as linear fixed effect. Covariance parameters found by maximum likelihood using space filling designs and partial maximization over ρ and σ D. Nychka LatticeKrig 27

28 Estimated correlation functions Matern MLE Solid line 4 level model beginning with and a single κ Dotted line 1 level with basis functions Correlation In projected coordinates size of spatial domain roughly 1.0 units Distance Why these are so different is still uncertain. D. Nychka LatticeKrig 28

29 Predicted field and uncertainty D. Nychka LatticeKrig 29 For 4 level covariance model Mean Summer total rainfall (cm) pred. standard error/mean Standard errors found by conditional simulation of 100 fields.

30 Summary Multi-resolution can approximate standard covariance families (e.g. Matern) Computational efficiency gained by compact basis functions and sparse precision matrix. Flexibility in model to account for nonstationary spatial dependence. Transform to preserve nonnegative rainfall and add orographic covariates See LatticeKrig package in R D. Nychka LatticeKrig 30

31 Thank you! D. Nychka LatticeKrig 31