ACCOUNTING FOR THE SITUATION-DEPENDENCE OF THE AMV OBSERVATION ERROR IN THE ECMWF SYSTEM Kirsti Salonen and Niels Bormann ECMWF, Shinfield Park, Reading RG2 9AX, United Kingdom Abstract This article reports the status of the work towards using situation dependent observation errors for Atmospheric Motion Vector (AMV) observations in the ECMWF system. The height assignment errors have been estimated from model best-fit pressure statistics, and the tracking errors from cases where the error due to error in height assignment is small. Impact studies with the new situation dependent observation errors show encouraging results. INTRODUCTION Numerical weather prediction (NWP) models provide the basis for weather forecasting by simulating the evolution of the atmospheric state. A good forecast requires that the initial state of the atmosphere is known accurately, and that the NWP model is a realistic representation of the atmosphere. Data assimilation methods are used to produce initial conditions for NWP models. The NWP model background field, typically a short-range forecast, is updated with observations in a statistically optimal way. A realistic specification of background and observation errors, and error correlations is essential as they determine to what extent the model background field is corrected to fit the observations. In this article, work towards accounting for the atmospheric motion vector (AMV) observation error characteristics in the ECMWF data assimilation system is discussed. AMV observation errors originate mainly from two sources, errors in the wind vector tracking and errors in the height assignment of the tracers. The latter can be very significant in regions where wind shear is strong, but is less relevant in areas where there is not much variation in wind speed with height. The observation errors applied in the operational ECMWF system for AMVs vary currently only with height. Thus, observation errors are independent of satellite, channel, and height assignment method as well as the prevailing atmospheric conditions. Forsythe and Saunders (28) have introduced an approach to estimate situation dependent AMV observation errors. The method is investigated in the ECMWF system. This article is organised as follows. First, estimation of the situation dependent observation errors in the ECMWF system is described. Second, results from model experiments are reported including evaluation of the new observation errors, and impact assessment. Finally, a short summary is given. ESTIMATION OF THE SITUATION DEPENDENT OBSERVATION ERRORS The Forsythe and Saunders (28) approach devides the AMV observation error into two parts, one originating from the AMV tracking and one originating from the error in the height assignment. [total u/v error] 2 = [Tracking error in u/v] 2 + [Error in u/v due to error in height assignment] 2. (1) 1
EBBT MET 9 GOES 11 MTSAT1 R H 2 O intercept CO 2 slicing 2 2 2 4 4 4 Pressure (hpa) 6 Pressure (hpa) 6 Pressure (hpa) 6 8 8 8 1 1 1 1 2 3 Pressure error (hpa) 1 2 3 Pressure error (hpa) 1 2 3 Pressure error (hpa) Figure 1: Pressure error estimates based on best-fit pressure statistics for Meteosat-9 (black solid line), GOES-11 (blue dashed line), and MTSAT1-R (red dash dotted line) infrared channel AMVs utilising EBBT (left panel), H 2O intercept (middle panel), and CO 2 slicing (right panel) height assignment methods, respectively. The advantage of the approach is that it allows to down-weight observations in regions with high vertical wind shear where errors in height assignment are problematic, and give greater weight to observations in regions where the height assignment error is less critical. Other errors may also contribute to the total AMV observation error, e.g. errors of representativeness, but these are not explicitly modelled here. In the ECMWF system, the height errors have been estimated based on model best-fit pressure statistics. The model best-fit pressure is defined as the height where the vector difference between the observed and the model background wind is the smallest. Another option would be to use producer provided estimates for the height errors. However, these are not yet operationally available. Use of the model best-fit pressure to characterise the AMV height assignment errors is discussed in more details in Salonen et al. (212) in these proceedings. Figure 1 shows the pressure errors as a function of height for Meteosat-9, GOES-11, and MTSAT1-R IR channel AMVs utilising the EBBT (left panel), the H 2 O intercept (middle panel), and the CO 2 slicing (right panel) height assignment methods, respectively, as an example of the results. The statistics have been examined separately for all satellites, channels, and height assignment methods. The height error estimates vary typically between 7 hpa and 12 hpa. The largest height error estimate of 26 hpa was found for GOES-13 cloudy water vapour AMVs at 4 6 hpa height, and the smallest height error estimate of 25 hpa for Meteosat-9 visible channel AMVs at 6 8 hpa height. A default value of 8 hpa is used, if a pre-defined height error estimate does not exist. The height error is converted to a wind error due to the error in height using equations 2 and 3 in each case (Forsythe and Saunders, 28) E vp = Wi (v i v n ) 2 Wi, (2) where W i = exp( (p i p n ) 2 2Ep 2 ) dp i. (3) 2
GEO POLAR 2 Pressure (hpa) 4 6 8 1 1 2 3 4 Error in u/v (ms 1 ) Figure 2: Tracking error estimates for AMVs from geostationary (solid line) and polar (dashed line) satellites. In 2 and 3 i is the model level, v i is the wind component on model level i, v n is the wind component at the observation location, p i is the pressure on model level i, p n is the pressure assigned to the AMV, E p is the height error, and dp i is the layer thickness. The formulation assumes a Gaussian distribution of height error, and E p defines the width of the weighting function. An upper limit for the weighting function is set to the height of the model tropopause. It is assumed that there are no clouds or water vapour features suitable for AMV tracking above that height. The tracking errors have been estimated from cases where the error due to the error in height is small. Also the tracking errors have been studied separately for all satellites, channels, and height assignment methods but as the differences were relatively small, at the moment the tracking errors are defined separately only for AMVs from geostationary, and polar orbiting satellites (Fig. 2). The tracking error estimates vary between 2. ms 1 and 3.2 ms 1. A default value of 2.5 ms 1 is used if a predefined value does not exist. Finally, the total observation error for each AMV observation is calculated by combining the tracking error and the wind error due to error in observation height with equation 1. EXPERIMENTATION In order to evaluate the realism of the situation dependent observation errors, and to study their impact on model forecasts, a set of model experiments for July - August 21 have been performed with the ECMWF Integrated Forecasting System cycle 37r2 at T511 ( 4 km) resolution, 91 vertical levels and 12 hour 4D-Var. All operationally assimilated conventional and satellite observations have been used. The control run is similar to the current operationally used setup, i.e. the AMV observation errors vary only with height. In the experiments the new observation errors are used, and the experiment setup is varied in order to answer the following questions: Are the new observation errors realistic? Can the first guess check be simplified? Can the observation error due to the error in height be used to exclude suspicious observations? 3
Mean OmB, WV cloudy, 1 hpa 4 hpa 15 6 o N 1 3 o N 5 o 3 o S 5 6 o S 1 18 o W 12 o W 6 o W o 6 o E 12 o E 18 o W Mean obs error, WV cloudy, 1 hpa 4 hpa 15 15 6 o N 3 o N 1 o 3 o S 5 6 o S 18 o W 12 o W 6 o W o 6 o E 12 o E 18 o W Figure 3: Mean OmB (upper panel), and mean observation error (lower panel) for cloudy water vapour AMV u- component at levels 1-4 hpa, 25th August 21, 12 UTC. What is the impact of using the new observation errors on model analysis and forecasts? Evaluation of the new observation errors Comparison of the situation dependent AMV observation errors and the operationally used observation errors which vary only with height indicate that on average the situation dependent observation errors are of the same magnitude, or slightly larger, than the current observation errors. To illustrate the situation dependence, Fig. 3 displays the mean observation minus background (OmB; upper panel) and the mean observation error (lower panel) for cloudy water vapour AMVs (u component) at levels 1-4 hpa at 25th August 21 12 UTC. Comparison of the panels shows that at same locations where there are significant differences between the observed and model wind speed, also the situation dependent observation errors reach higher values. Thus, the behaviour of the new observation errors is consistent with expectations. Figure 4 shows the OmB standard deviation as a function of the situation dependent observation errors for the u wind component for Meteosat-9 cloudy water vapour AMVs applying CO2 height assignment method at levels 1-4 hpa. The grey histograms show the number of observations. There is a good agreement between the observation errors and the OmB standard deviation. The OmB standard deviation has a contribution from the background error as well. A similar comparison as Fig. 4 but for the background error reveals that the magnitude of the background error is relatively constant as a function of the situation dependent observation errors (not shown). Thus, the increase in the OmB 4
Figure 4: OmB standard deviation as a function of situation dependent observation errors (black dots) for southern hemisphere extra tropics (left panel), tropics (middle panel), and northern hemisphere extra tropics (right panel) Meteosat-9 cloudy water vapour AMVs applying CO2 height assignment method at levels 1-4 hpa. The grey histograms show the number of observations. standard deviation is clearly related to AMVs with increased error in wind due to error in the height assignment. In an ideal case the observation errors in Fig. 4 would lie above the one-to-one line. Thus, the results indicate that the new observation errors are slightly overestimated. This behaviour is quite typical for other satellite, channel, and height assignment combinations as well. However, at the moment the spatial and temporal correlations of the AMV observation errors are not taken into account, but only compensated by inflating the observation errors. From that point of view the magnitude of the new observation errors is justified. Model first guess check The model first guess check compares observations y with the model background information Hx b 1 2 ([(Hx b y) 2 σb 2 + ] u + [ (Hx b y) 2 σ2 o σb 2 + ] v ) L. (4) σ2 o Observations which deviate from the background more than a pre-defined limit L are rejected. In eq. 4 σb 2 and σ2 o are the background and observation error variances, respectively. Traditionally the first guess check has been very strict for AMV observations. In the operational ECMWF system tight rejection limits are applied, typically L is by factor 1 smaller for AMV observations compared to limits used for conventional wind observations. In addition, the first guess check is assymmetric, i.e. an additional penalty is applied to AMV observations that under-report wind speed when compared with the first guess field. There are also some geographical dependencies in the rejection limits, the first guess check is slightly relaxed for the low level winds, and in the tropics. The new situation dependent observation errors allow to down-weight observations in areas where wind shear is strong and the error in the height assignment can have a drastic impact. Thus, it is important to revise the first guess check and carefully consider how it could be simplified and possibly relaxed. 5
Figure 5: Demonstration of the operation of the model first guess check. The left panel shows Meteosat-9 WV AMVs at 1-4 hpa heights after blacklisting. The upper right panel displays the AMVs after applying the first guess check used in the operational system, and the lower middle panel the after the modified first guess which is under investigations. The lower right panel shows AMVs after applying the criterion to limit the magnitude of the observation error due to height error to be smaller than twice the tracking error. Figure 5 illustrates how the first guess check operates. In the left, a scatter plot of observed wind speed versus first guess wind speed is shown for Meteosat-9 WV AMVs at 1-4 hpa heights. The upper panel shows the scatter plot for AMVs which have been accepted by the first guess check used in the operational system. Outliers have been removed very effectively, and also the impact of the asymmetric check is clearly seen. The lower panel illustrates the simplified first guess check that is under investigations. In the simplified first guess check the asymmetric part has been removed, and the same rejection limits are used independent of the geographical location of the AMV observation. In Fig. 5, the rejection limits have also been slightly relaxed compared to the operational ones. The modified first guess check rejects outliers as well but the spread in the scatter plot is notably wider compared to the operational first guess check. Another aspect under investigation, and illustrated in the low right panel of Fig. 5, is how the observation error due to the error in height could be used to exclude bad quality observations. A first trial has been to limit the magnitude of the observation error due to height error to be smaller than twice the tracking error. Excluding AMVs with large errors due to errors in height assignment is motivated by the fact that the height assignment errors are likely to be more correlated spatially, and such correlations are currently neglected. Impact assessment Next, results from an experiment where the situation dependent observation errors, the above described modified first guess check, and the criterion to limit the magnitude of the observation error due to height error to be smaller than twice the tracking error have been used are discussed. The control run is similar to the current operational setup. 6
Vector difference of mean wind analysis, Exps fmfg-fhrd LEV=2, 2171 to 21831 12 W 6 W 6 E 12 E 5 4.5 4 6 N 6 N3.5 3 2.5 3 N 3 N2 1.5 1.5 -.5-1 -1.5 3 S 3 S-2-2.5-3 6 S 6 S-3.5-4 -4.5-5 12 W 6 W 6 E 12 E.5 2.5m/s Figure 6: Difference in the mean wind analysis at 2 hpa between the experiment using situation dependent observation errors and the control. Shading indicates the difference in mean wind speed (ms 1 ). The considered period is 1 July - 31 August 21. Figure 6 shows the vector difference between the experiment and the control for the mean wind analysis at level 2 hpa. The most significant impact is seen in the tropics where the difference reaches values as high as 2.5 ms 1. In the mid-latitudes the magnitude of the changes is typically less than.5 ms 1. The vector difference is mainly positive, i.e. the mean wind is stronger in the experiment than in the control. At 3 hpa level the modifications tested in the experiment tend to weaken the mean wind field in the tropics, the highest differences being 1 ms 1 (not shown). At lower levels the differences in the mean wind analysis are typically less than ±.5 ms 1. The largest differences in the mean wind analysis are seen over sea where very few radiosonde or pilot observations are available, and it is difficult to assess whether the changes in these areas indicate positive or negative impact. Figure 7 shows the normalised difference in RMS error for 48-hour wind forecasts at 5 hpa level. Verification has been done against own analysis. The difference is calculated as experiment minus control, i.e. blue shades indicate positive impact and green and red shades negative impact. The overall impression is that using the situation dependent observation errors, and the modifications have a positive impact on the forecast. Positive impact can be seen on other levels and forecast ranges as well. However, at 2 hpa level a more mixed impact is found. The main findings from all performed experiments are: Using the situation dependent observation errors has generally a positive impact on model analysis and forecasts. Removing the assymetric part from the first guess check is possible without degrading the forecast quality. However, verification against the own analysis indicates that relaxing the first guess check limits results into negative forecast impact at high levels, and on high latitudes (north from 8 N and south from 8 S) for short forecast ranges at all levels. Criterion [Error in u/v due to error in height assignment] < n [Tracking error in u/v] is an effective tool to detect and reject suspicious observations. However n=2 seems to be too tight criterion and too many good quality observations are also rejected. This leads to negative forecast impact in some areas in the tropics at high levels. 7
Figure 7: Normalised difference (experiment - control) in RMS error for 48-hour wind forecasts at 5 hpa level. Ongoing experimentation is addressing the remaining open questions related to defining the rejection limits for the modified first guess check, and the possible use of the limiting criterion to detect bad quality observations. Operational implementation of the changes is planned after finalising the experimentation. SUMMARY The status of the work towards taking into account the AMV observation error characteristics in the ECMWF data assimilation system has been reported in this article. The two main sources of AMV observation errors are errors in the wind vector derivation, and errors in the height assignment of the tracers. An approach which allows to estimate situation dependent observation errors have been implemented in the system. Thus, observations in regions with high vertical wind shear where errors in height assignment are problematic can be down-weighted in the model analysis, and observations in regions where the height assignment error is less critical can get greater weight. Experiments indicate that the new situation dependent observation errors are on average of the same magnitude, or slightly larger, than the currently used observation errors which vary only with height. Comparisons with OmB standard deviation indicate good agreement, and results from impact studies show encouraging results. However, some further investigations are still required before making final conclusions and changes to the operational system. REFERENCES Forsythe, M., Saunders, R., 28. AMV errors: a new approach in NWP. Proceedings of the 9th International Wind Workshop, Annapolis, Maryland, USA, 14-18 April 28 EUMETSAT P.51. Salonen, K., Cotton, J., Bormann, N., Forsythe, M., 212. Characterising AMV height assignment error by comparing best-fit pressure statistics from the Met Office and ECMWF system. Proceedings of the 11th International Wind Workshop, Auckland, New Zealand, 2-24 February 212. 8