Statistical interpretation of Numerical Weather Prediction (NWP) output

Transcription

1 Statistical interpretation of Numerical Weather Prediction (NWP) output 1

2 2

3 3

4 Four types of errors: Systematic errors Model errors Representativeness Synoptic errors Non-systematic errors Small scale noise 4

5 Even when we get rid of systematic errors, make the synoptic forecast perfect and only verify against representative observations the meso-scale 5 noise will still yield non-perfect forecasts

6 The two neighbouring stations Potsdam and Lindenberg outside Berlin are just 75 kilometres apart and are situated in almost the same environment. How well would a forecast based on the other one s observation verify? Other nearby stations were also used (Magdeburg, Dresden, Poznan and Stettin). They provided, together with the previous two data to calculate an average temperature as forecast. 6

7 Stettin 3 m Magdeburg 84 m Potsdam 99 m Lindenberg 115 m Poznan 92 m Dresden 226 m 7

8 Four tests were conducted all with the objective to estimate ( forecast ) the temperature at Lindenberg: 1.Using the observation from Potsdam as forecast 2.Using an average of all five surrounding stations 3.The same but with weights proportional to the square of the distance from Lindenberg 4.The same, but without using the observation from nearby Potsdam 8

9 Other weightings RMSE Potsdam s day & night observations applied on Lindenberg All five surrounding weighted observations 9

10 Other weightings SDE Potsdam s day & night observations applied on Lindenberg All five surrounding weighted observations 10

11 Other weightings MAE Potsdam s day & night observations applied on Lindenberg All five surrounding weighted observations 11

12 all kind of Potsdam s observation applied on Lindenberg All five surrounding weighted observations MAE ECMWF 12UTC + 12h and +24 h 12

13 all kind of Rather 0.6 Error=0 at t=0? 13

14 Conclusions from this observation investigation: 1. During favourable conditions the lowest RMSE and SDE would be around 0.8ºK, for MAE 0.6ºK 2. During seasons when the temperature depends quite a lot on the clouds the values increase to around 2ºK resp. 1½ºK. 3. Verified against a specific site, the weighted area average (3) provided the best forecast, whereas the neighboring station observation method (1) provided the worst. 14

15 Conclusions for all kinds of forecasts beyond a few hours: 1. Due to micro-scale variability the 2 metre temperature is at present not possible to forecast with higher accuracy than 0.8ºK (RMSE,SDE) or 0.6ºK (MAE). 2. Provided homogenous environment an area average forecast, applied to a specific site, might be superior to a site specific. 3. Site specificness only has meaning if the site is not representative to the area, if its climate is different to the area as a whole. 15

16 True and false error curves 16

17 Four types of errors: Systematic errors Model errors Representativeness Synoptic errors Non-systematic errors Small scale noise 17

18 18

19 There is much more to say about this - at some other time 19

20 Four types of errors: Systematic errors Model errors Representativeness Synoptic errors Non-systematic errors Small scale noise 20

21 21

22 RMSE errors of raw T399 grid point +24h forecasts 2007 for Tromsö airport [T] T The main contributor to the large RMSE for inland grid points are mean errors between up to -5º 22

23 RMSE after Kalman-2 filtering makes the quality almost the same for all grid points T

24 Statistical correction, calibration or interpretation: A heavily biased temperature forecast Tromsø (northern Norway) 24

25 The EPS plume after statistical correction Tromsø (northern Norway) 25

26 The forecast ( ) varies more than reality. The adaptive statistical filtering corrects for both mean error and overvariability Tromsø (northern Norway) 26

27 No simple, straight bias. The mean error depends on the forecast Tromsø (northern Norway) 27

28 Four types of errors: Systematic errors Model errors Representativeness Synoptic errors Non-systematic errors Small scale noise 28

29 29

30 The systematic errors we want to correct for are not only 1-dimensional flat biases... 30

31 Obs-T fc = correction Bias? T fc Corr = A(t) 31

32 Obs-T fc = correction Bias? T fc Corr = A(t) It appears as if the old bias has abruptly changed into a new one 32

33 Obs-T fc = correction Systematic error T fc Corr = A(t) + B(t) T fc In reality the systematic error has stayed more or less the same, but defined by two coefficients, A and B 33

34 Err Flat true bias Err 2-D systematic errors T fc T fc Apparent non-systematic errors.. Err Err but when projected into an additional dimension they appear to be systematic T fc T fc T850 fc 34

35 A very, very brief introduction to the adaptive procedure...before the break 35

36 T fc - obs 2-dim error equation Expected error T fc Last NWP forecast 36

37 T fc - obs 2-dim error equation Obs error Last verified NWP forecast T fc 37

38 T fc - obs 2-dim error equation T fc 38

39 T fc - obs 2-dim error equation Slightly modified values of A and B Next forecast Expected error T fc 39

40 The historical background to and its classical application 2nd lecture RSHU 40

41 The origin of the Kalman filter 1960 launching intercontinental ballistic missiles Track uncertainty Corrected track Observation error Airborne radar Ground radar Estimated position 2nd lecture RSHU 41

42 Latest observed position Remote observer Intended position True position 2nd lecture RSHU Estimated position 42

43 2nd lecture RSHU 43

44 2nd lecture RSHU 44

45 1-D corrects for mean errors ( biases ) but can also illustrate the basic philosophy, here in three ways 1. Pictorial description 2. Mathematical derivation 3. Graphical illustration 2nd lecture RSHU 45

46 How is it done? The pictorial version 2nd lecture RSHU 46

47 Obs-T fc = correction The filter makes a cold start i.e. no correction is applied The assumed covariance of a cold start T fc 2nd lecture RSHU 47

48 Obs-T fc = correction The latest verified numerical forecast The ideal correction (=the inverse of the error) and its error T fc The forecast 2nd lecture RSHU 48

49 Obs-T fc = correction The latest verified numerical forecast T fc makes the filter change its value and the initial uncertainty is shrunk 2nd lecture RSHU 49

50 Obs-T fc = correction T fc The correction is defined by the relative size of the observation uncertainty and the filter uncertainty 2nd lecture RSHU 50

51 Obs-T fc = correction The ideal correction (=the inverse of the error) and its error T fc A new forecast makes the filter change its value and the initial uncertainty is shrunk 2nd lecture RSHU 51

52 Obs-T fc = correction T fc The correction is again defined from the relative size of the observation uncertainty and the filter uncertainty 2nd lecture RSHU 52

53 Obs-T fc = correction Finally we end up with the filter oscillating around a mean state with a certain variance (uncertainty) correction Bias T fc This uncertainty have a lower threshold and can never be = 0, which would lock the Kalman filter 2nd lecture RSHU 53

54 How is it done? The mathematical derivation 2nd lecture RSHU 54

55 Y τ =the observed forecast error at verification time τ Y Tfc Tobs Y τ is the sum of the ideal correction χ τ and the noise ν τ Y v 2nd lecture RSHU 55

56 We introduce the first guess values X τ/τ-1 = A τ X τ/τ-1 where A τ = 1-F 1, where F 1 << 1 and Q τ/τ-1 to be discussed later The difference between the first guess value X τ/τ-1 and the observed value Y τ must obviously affect how much we shall modify X τ/τ-1 2nd lecture RSHU 56

57 We now introduce δ τ (0 <δ τ < 1) which indicates how much of the difference between Y τ and X τ/τ-1 that shall modify X τ/τ-1 X X ( Y X 1) / / 1 / Assume that the error in our estimation of χ τ is ε τ X / which yields X ( Y X 1) / 1 / 2nd lecture RSHU 57

58 And with the noise term Y v we get X ( v X 1) / 1 / and after rearrangement of the terms X ) ( 1 )( / 1 v 2nd lecture RSHU 58

59 The (co) variance term which indicates the (un) certainty of our estimation The uncertainty of Y v depends on sub-grid turbulence, non-systematic synoptic errors or measurement errors, what we choose to call the observation error D τ cov( v ) D 2nd lecture RSHU 59

60 cov( ) cov( X ) / Q / and cov( X ) / 1 Q / 1 and cov( v ) D...yields: Q 1 ) 2 / ( Q / 1 2 D 2nd lecture RSHU 60

61 We differentiate Q 1 ) 2 / ( Q / 1 2 D with respect to δτ dq d / 2(1 ) Q / 1 2 D 2nd lecture RSHU 61

62 from dq d / 2(1 ) Q / 1 2 D we get min D ( 1 ) Q / 1 0 min D Q / 1 Q / 1 Which is the final result 2nd lecture RSHU 62

63 1 / 1 / min Q D Q 2 2 / / 1 ) 1 ( D Q Q / / ) 1 ( D Q Q with and the updated forward (co)variances become 2nd lecture RSHU 63

64 How is it done? The graphical illustration 2nd lecture RSHU 64

65 Assume an unknown process χ which can be 1-dim (the mean error or bias of NWP) or N- dim (the N coefficients in an error correction equation) τ-1 τ τ+1 χ 2nd lecture RSHU 65

66 We have at τ-1 an estimated value X τ-1/τ-1 of the unknown process χ with variance Q τ-1/τ-1 Q τ-1/τ-1 X τ-1/τ-1 τ-1 τ τ+1 χ 2nd lecture RSHU 66

67 We carry X forward in time by a linear model A, assuming that the variance increases slightly X τ-1/τ-1 Q τ-1/τ-1 +C A X τ-1/τ-1 τ-1 τ τ+1 χ 2nd lecture RSHU 67

68 We have a predicted estimate, X τ/τ-1 and Q τ/τ-1 similar to the first guess in numerical weather prediction Q τ/τ-1 X τ/τ-1 τ-1 τ τ+1 χ 2nd lecture RSHU 68

69 The observation Y τ, with variance D, of the unknown process χ will modify the first guess value X τ/τ-1 Q τ/τ-1 X τ/τ-1 Y τ D χ τ-1 τ τ+1 2nd lecture RSHU 69

70 The weighting together of D and Q τ/τ-1 yields a variance of the new estimation X τ/τ and Q τ/τ Q τ/τ-1 X τ/τ Q τ/τ D χ τ-1 τ τ+1 2nd lecture RSHU 70

71 The new estimation of X τ/τ and Q τ/τ Q τ/τ X τ/τ τ-1 τ τ+1 χ 2nd lecture RSHU 71

72 The new estimation starts with predicting X τ+1/τ and Q τ+1/τ χ X τ/τ A τ Q τ/τ +c X τ+1/τ Q τ+1/τ τ-1 τ τ+1 2nd lecture RSHU 72

73 ...and a new observation arrives χ A τ Q τ/τ +c X τ+1/τ Q τ+1/τ New Y τ τ-1 τ τ+1 2nd lecture RSHU 73

74 But there are fundamental differences between 1- dimensional filtering and multi-dimensional 74

75 24 hour 2 m temperature forecast for Kiruna in Lapland winter The verification yielded RMSE=5.0 2nd lecture RSHU 75

76 A 1-dimensional Kalman filter reduces an overall bias Correction out of step The corrections yielded a reduction of the mean error from 2.6 to 0.3 and RMSE from 5.0 to 4.2 2nd lecture RSHU 76

77 A 2-dimensional Kalman filter system also improves the forecasts of the extremes Correction not out of step Two good achievements: The Kalman filtering has reduced two systematic errors: a positive mean error and an underestimation of the variability 2nd lecture RSHU 77

78 Why does the improvement not show up in the verification? The corrections still yielded a reduction of the mean error from 2.6 to 0.3 but the RMSE from 5.0 only to 4.6 and not to 4.2 as with the 1-D 2nd lecture RSHU Is the 2-D worse than the 1-D?? 78

79 END of part I 79