Introduction to data assimilation and least squares methods

Transcription

1 Introduction to data assimilation and least squares methods Eugenia Kalnay and many friends University of Maryland October 008 (part 1

2 Contents (1 Forecasting the weather - we are really getting better! Why: Better obs? Better models? Better data assimilation? It s all three together! Intro to data assim: a toy scalar example 1, we measure with two thermometers, and we want an accurate temperature. Another toy example, we measure radiance but we want an accurate temperature: we derive OI/KF, 3D- Var, 4D-Var and EnKF for the toy model.

3 Contents ( Review of toy example 1 Another toy example, we measure radiance but we want an accurate temperature: We derive OI/KF, 3D-Var, 4D-Var and EnKF for the toy model. Comparison of the toy and the real equations An example from JMA comparing 4D-Var and LETKF

4 Typical 6-hour analysis cycle Bayes interpretation: a forecast (the prior, is combined with the new observations, to create the Analysis (IC (the posterior

5 The observing system a few years ago Before 1979 only raobs Now we have even more satellite data

6 Typical distribution of observations in +/- 3hours Typical distribution of the observing systems in a 6 hour period: a real mess: different units, locations, times

7 Typical distribution of observations in +/- 3hours Typical distribution of the observing systems in a 6 hour period: a real mess: different units, locations, times

8 Model grid points (uniformly distributed and observations (randomly distributed. For the grid point i only observations within a radius of influence may be considered i k

9 Some statistics of NWP

10 Some comparisons

11 We are getting better (NCEP observational increments 500MB RMS FITS TO RAWINSONDES 6 HR FORECASTS M ( S E C N E R E F F I D S M R Southern Hemisphere 15 Northern Hemisphere YEAR

12 Comparisons of Northern and Southern Hemispheres

13 Satellite radiances are essential in the SH

14 More and more satellite radiances

15 Intro. to data assimilation: toy example 1 We want to measure the temperature in this room, and we have two thermometers that measure with errors: T 1 = T t +! 1 T = T t +! We assume that the errors are unbiased:! 1 =! = 0 that we know their variances! 1 = " 1! = " and the errors of the two thermometers are uncorrelated:! 1! = 0 The question is: how can we estimate the true temperature optimally? We call this optimal estimate the analysis of the temperature

16 Intro. to data assimilation: toy example 1 We try to estimate the analysis from a linear combination of the observations: = a 1 T 1 + a T and assume that the analysis errors are unbiased: This implies that = T t a 1 + a = 1

17 Intro. to data assimilation: toy example 1 We try to estimate the analysis from a linear combination of the observations: = a 1 T 1 + a T and assume that the analysis errors are unbiased: = T t This implies that a 1 + a = 1 will be the best estimate of if the coefficients a 1, a are chosen to minimize the mean squared error of : T t! a = ( " T t = [a 1 (T 1 " T t + (1 " a 1 (T " T t ]

18 Intro. to data assimilation: toy example 1 Replacing a = 1! a 1! a the minimization of with respect to gives a 1! a = ( " T t = [a 1 (T 1 " T t + (1 " a 1 (T " T t ]

19 Intro. to data assimilation: toy example 1 Replacing a = 1! a 1! a a 1 the minimization of with respect to gives!" a or! a = ( " T t = [a 1 (T 1 " T t + (1 " a 1 (T " T t ]!a 1 = 0 ==> a 1 = a 1 =!! 1 +! a = 1 /! 1 1 /! /! a =! 1! 1 +! 1 /! 1 /! /! The first formula says that the weight of obs 1 is given by the variance of obs divided by the total error. The second formula says that the weights of the observations are proportional to the "precision" or accuracy of the measurements (defined as the inverse of the variances of the observational errors.

20 Intro. to data assim: toy example 1 summary Two measurements and an optimal linear combination (analysis: = a 1 T 1 + a T Optimal coefficients (min! a a 1 =!! 1 +! a =! 1! 1 +! a 1 = 1 /! 1 1 /! /! a = Replacing, we get! a =! 1!! 1 +! <! 1,! or or 1 /! 1 /! /! 1! = 1 a! + 1 1!

21 Intro. to data assim: toy example 1 summary Two measurements and an optimal linear combination (analysis: = a 1 T 1 + a T Optimal coefficients (min! a a 1 =!! 1 +! a =! 1! 1 +! or Replacing, we get! a =! 1!! 1 +! a 1 = 1 /! 1 1 /! /! a =, or 1! = 1 a! + 1 1! 1 /! 1 /! /!

22 Intro. to data assim: toy example 1 summary Two measurements and an optimal linear combination (analysis: = a 1 T 1 + a T Optimal coefficients (min! a a 1 =!! 1 +! a =! 1! 1 +! or Replacing, we get! a =! 1!! 1 +! a 1 = 1 /! 1 1 /! /! a = or 1! = 1 a! + 1 1! 1 /! 1 /! /! Now assume that T 1 =T b (forecast and T =T o (observation. Then = a 1 T b + a T o = T b + a (T o! T b

23 Intro. to data assim: toy example 1 summary Two measurements and an optimal linear combination (analysis: = a 1 T 1 + a T Optimal coefficients (min! a a 1 =!! 1 +! a = Replacing, we get! 1! 1 +! or! a =! 1!! 1 +! a 1 = 1 /! 1 1 /! /! a = 1! = 1 a! + 1 1! 1 /! 1 /! /! Now assume that T 1 =T b (forecast and T =T o (observation. Then = a 1 T b + a T o = T b + a (T o! T b or or = T b +! b! b +! o (T o " T b = T b + w(t o " T b This is the form that is always used in analyses

24 Intro. to data assim: toy example 1 summary A forecast and an observation optimally combined (analysis: = T b +! b! b +! o (T o " T b with 1! = 1 a! + 1 b! o f the statistics of the errors are exact, and if the coefficients are optimal, then the "precision" of the analysis (defined as the inverse of the variance is the sum of the precisions of the measurements. Now we are going to see a second toy example of data assimilation including remote sensing. The importance of these toy examples is that the equations are identical to those obtained with big models and many obs.

25 Intro. to remote sensing and data assimilation: toy example Assume we have an object, a stone in space We want to estimate its temperature T ( o K accurately but we measure the radiance y (W/m that it emits. We have an obs. model, e.g.: y = h(t!!t 4

26 Intro. to remote sensing and data assimilation: toy example Assume we have an object, a stone in space We want to estimate its temperature T ( o K accurately but we measure the radiance y (W/m that it emits. We have an obs. model, e.g.: y = h(t!!t 4 We also have a forecast model for the temperature T (t i+1 = m[ T (t i ]; [ ] e.g., T (t i+1 = T (t i +!t SW heating+lw cooling

27 Intro. to remote sensing and data assimilation: toy example Assume we have an object, a stone in space We want to estimate its temperature T ( o K accurately but we measure the radiance y (W/m that it emits. We have an obs. model, e.g.: y = h(t!!t 4 We also have a forecast model for the temperature T (t i+1 = m[ T (t i ]; [ ] e.g., T (t i+1 = T (t i +!t SW heating+lw cooling We will derive the data assim eqs (KF and Var for this toy system (easy to understand!

28 Intro. to remote sensing and data assimilation: toy example Assume we have an object, a stone in space We want to estimate its temperature T ( o K accurately but we measure the radiance y (W/m that it emits. We have an obs. model, e.g.: y = h(t!!t 4 We also have a forecast model for the temperature T (t i+1 = m[ T (t i ]; [ ] e.g., T (t i+1 = T (t i +!t SW heating+lw cooling We will derive the data assim eqs (OI/KF and Var for this toy system (easy to understand! Will compare the toy and the real huge vector/matrix equations: they are exactly the same!

29 Toy temperature data assimilation, measure radiance We have a forecast T b (prior and a radiance obs y o = h(t t +! 0 The new information (or innovation is the observational increment: y o! h(t b

30 Toy temperature data assimilation, measure radiance We have a forecast T b (prior and a radiance obs y o = h(t t +! 0 The new information (or innovation is the observational increment: y o! h(t b The final formula used has the same form as = T b + w(y o! h(t b with the optimal weight w =! b H (! o + H! b H "1 Where does H come from?! = T b + b! b +! (T o " T b o

31 Toy temperature data assimilation, measure radiance We have a forecast T b (prior and a radiance obs y o = h(t t +! 0 The new information (or innovation is the observational increment: y o! h(t b We assume that the obs. and model errors are Gaussian The innovation can be written in terms of errors: y o! h(t b = h(t t + " 0! h(t b = " 0 + h(t t! h(t b = " 0! H" b where H =!h /!T includes changes of units and observation model nonlinearity, e.g., h(t =!T 4

32 Toy temperature data assimilation, measure radiance We have a forecast T b and a radiance obs y o = h(t t +! 0 y o! h(t b = " 0! H" b

33 Toy temperature data assimilation, measure radiance We have a forecast T b and a radiance obs y o = h(t t +! 0 y o! h(t b = " 0! H" b From an OI/KF (sequential point of view: = T b + w(y o! h(t b = T b + w(" 0! H" b or! a =! b + w(! 0 " H! b Now, the analysis error variance (over many cases is! a = " a

34 Toy temperature data assimilation, measure radiance We have a forecast T b and a radiance obs y o = h(t t +! 0 y o! h(t b = " 0! H" b From an OI/KF (sequential point of view: = T b + w(y o! h(t b = T b + w(" 0! H" b or! a =! b + w(! 0 " H! b In OI/KF we choose w to minimize the analysis error:! a = " a We compute! a =! b + w (! o + H! b H " w! b H assuming that! b,! 0 are uncorrelated

35 Toy temperature data assimilation, measure radiance We have a forecast T b and a radiance obs y o = h(t t +! 0 y o! h(t b = " 0! H" b From an OI/KF (sequential point of view: = T b + w(y o! h(t b = T b + w(" 0! H" b or! a =! b + w(! 0 " H! b In OI/KF we choose w to minimize the analysis error:! a = " a! a =! b + w (! o + H! b H " w! b H From!" a!w = 0 we obtain w =! b H (! o + H! b H "1

36 Toy temperature data assimilation, measure radiance Repeat: from an OI/KF point of view the analysis (posterior is: = T b + w(y o! h(t b = T b + w(" 0! H" b with w =! b H (! o +! b H "1 Note that the scaled weight wh is between 0 and 1 If! o >>! b H! T b! T t If! o <<! b H! T b + 1 [ H h(t " h(t ] t b! T t The analysis interpolates between the background and the observation, giving more weights to smaller error variances.

37 Toy temperature data assimilation, measure radiance Repeat: from an OI/KF point of view the analysis (posterior is: = T b + w(y o! h(t b = T b + w(" 0! H" b with w =! b H (! o +! b H "1 Subtracting T t from both sides we obtain! a =! b + w(! 0 " H! b Squaring the analysis error and averaging over many cases, we obtain! a = (1 " wh! b which can also be written as 1! = " 1 a! + H % # $ b! o & '

38 Toy temperature data assimilation, measure radiance Summary for OI/KF (sequential: = T b + w(y o! h(t b with w =! b H (! o +! b H "1 analysis optimal weight The analysis error is computed from! a = (1 " wh! b which can also be written as 1! = " 1 a! + H # $ b! o % & ' analysis precision= forecast precision + observation precision

39 Toy temperature data assimilation, variational approach We have a forecast T b and a radiance obs y o = h(t t +! 0 y o! h(t b Innovation: From a 3D-Var point of view, we want to find a that minimizes the cost function J: J( = (T! b + (h(t! y a o " b " o

40 Summary part 1 Data assimilation methods have contributed much to the improvements in NWP. A toy example is easy to understand, and the equations are the same as in a realistic huge system Observation operator: model variables => observed variables We assume no bias, no error correlation Analysis = forecast +optimal weight x (innovation Optimal weight = forecast error variance/total error variance Precision = 1/error variance Analysis precision=forecast precis. + obs. precis.