TC/PR/RB Lecture 3 - Simulation of Random Model Errors Roberto Buizza (buizza@ecmwf.int) European Centre for Medium-Range Weather Forecasts http://www.ecmwf.int Roberto Buizza (buizza@ecmwf.int) 1 ECMWF TC/PR/RB L3 Apr 02
Outline The importance of the simulation of physical processes in Numerical Weather Prediction (NWP): introduction. Stochastic physics: a simple stochastic approach to simulate the effect of random model errors due to the parameterized physical processes. Impact of stochastic physics on single deterministic integration. Impact of stochastic physics on the ECMWF Ensemble Prediction System (EPS). Roberto Buizza (buizza@ecmwf.int) 2 ECMWF TC/PR/RB L3 Apr 02
The ECMWF Numerical Weather Prediction (NWP) Model The behavior of the atmosphere is governed by a set of physical laws which express how the air moves, the process of heating and cooling, the role of moisture, and so on. Interactions between the atmosphere and the underlying land and ocean are important in determining the weather. Roberto Buizza (buizza@ecmwf.int) 3 ECMWF TC/PR/RB L3 Apr 02
The ECMWF NWP model Numerical models provide approximate solutions of the system equations: The atmosphere is divided into a 3- dimensional mesh of points: the current high-resolution model version has 60 vertical levels from the surface to a height of 65 km, each with grid points spaced 40 km apart (more than 20M points). Measurements are used to estimate the current state of the atmosphere (initial state). Numerical models solve an approximate forms of the equations and provide weather forecasts. Roberto Buizza (buizza@ecmwf.int) 4 ECMWF TC/PR/RB L3 Apr 02
The importance of spatial resolution A high spatial resolution is needed to achieve an accurate representation of the system physical processes. Similarly, the representation of the orography becomes more realistic with increased horizontal resolution. Roberto Buizza (buizza@ecmwf.int) 5 ECMWF TC/PR/RB L3 Apr 02
Parameterization of physical processes Each physical process has a characteristic spatial and temporal scale. Many processes occur on a spatial scale smaller than the model grid. For example, over land a 40km square may include different types of vegetation, bare soil or buildings. Each type, for example, reflects the incoming solar radiation and affects moist processes in a different way. Roberto Buizza (buizza@ecmwf.int) 6 ECMWF TC/PR/RB L3 Apr 02
Clouds in NWP models Clouds are not uniform 40-km wide blobs, but have a wide range of sizes and forms. They can occur on scales as small as a few hundred metres, which is much smaller than the model s grid. Roberto Buizza (buizza@ecmwf.int) 7 ECMWF TC/PR/RB L3 Apr 02
Clouds in NWP models Note how incredibly complicated the structure of the cloud is. Including this structure in the cloud model is only part of the problem. The many physically processes that determine how rain is formed from cloud droplets, or how quickly water freezes to ice particles or snow (microphysics) also have to be taken into account. Roberto Buizza (buizza@ecmwf.int) 8 ECMWF TC/PR/RB L3 Apr 02
Clouds in NWP models Radiation is the only way through which the earthatmosphere system can exchange energy with the rest of the universe. Clouds have large influence on many aspects of weather forecasts: reflection of the incoming short-wave solar radiation; reflection of the outgoing long-wave earth radiation; energy absorption/release related to moist processes. Roberto Buizza (buizza@ecmwf.int) 9 ECMWF TC/PR/RB L3 Apr 02
Surface processes Surface processes determine the sources and sinks of temperature and moisture (in terms of sensible and latent heat fluxes) at the lowest boundary of the atmosphere. As a consequence, over land, they define the state of the ground (warm, cold, freezing, dry or wet) and whether falling rain or snow precipitation will remain or subsequently disappear. It is the surface characteristics, such as the 2-m temperature, humidity and wind, that are some of the most important (and most difficult) to predict, since, after all, the lowest 2 metres is exactly the part of the atmosphere in which we spend most of our lives! Roberto Buizza (buizza@ecmwf.int) 10 ECMWF TC/PR/RB L3 Apr 02
Ocean waves Over the ocean, the wind at the lowest level of the model is used to diagnose the state of the sea through the wave height, which has a large consequence for the feedback to the atmospheric circulation, as well as its obvious importance as a forecast product for shipping. An example of the WAM wave height analysis of a severe storm in the North Atlantic, together with Satellite Altimeter measurements of wave heights and winds. Contours: WAM wave heights up to a maximum of over 12.5 m. Numbers: wind speed and wave height (e.g. 216 129 means wind speed 21.6 m/s, wave height 12.9 m). Roberto Buizza (buizza@ecmwf.int) 11 ECMWF TC/PR/RB L3 Apr 02
Sources of forecast errors: initial and model uncertainties The fact that numerical models describe the laws of physics only approximately (model uncertainties) contributes to the growth of forecast errors. This figure shows the effect of small random perturbations added to the tendencies due to the parameterized physical processes on a 4d forecast (MSLP). Roberto Buizza (buizza@ecmwf.int) 12 ECMWF TC/PR/RB L3 Apr 02
Stochastic physics: the rationale The parameterization scheme ( stochastic physics ) should be simple. It should simulate the sort of random errors in parameterized forcing which are coherent among the different parameterization models (moist-processes, radiation, turbulence,..). A way to take this into account is to apply the stochastic forcing on the total tendency. Model tendencies due to parameterized physical processes have a certain coherence on the space and time scales associated, for example, with organized convection. A way to simulate this is to impose space-time correlation on the random numbers. It should not affect the model climate. Roberto Buizza (buizza@ecmwf.int) 13 ECMWF TC/PR/RB L3 Apr 02
Roberto Buizza (buizza@ecmwf.int) 14 ECMWF TC/PR/RB L3 Apr 02 The EPS with perturbed physics Each ensemble member evolution is given by the time integration of the perturbed model equations starting from the perturbed initial conditions The model tendency perturbation is defined at each grid point by where r(x) is a random number. = + + = T t j j j j j dt t e P t e P t e A T e 0 )], ( ), ( ), ( [ ) ( δ (0) (0) 0) ( 0 j j e e e δ + = ),, ( ), ( ),, ( p P r p P j j j φ λ φ λ φ λ δ =
Selection of random numbers Random numbers can be selected with different spatial correlation scales. The top figure shows the case when different random numbers are used at each grid point. The bottom figure shows the case when the same random number is used inside 5-degree boxes. In this case the numbers have been selected inside the interval -0.5 r(x) 0.5: blue is for -0.5 r(x) -0.3; green is for -0.1 r(x) 0.1; red is for 0.3 r(x) 0.5. Roberto Buizza (buizza@ecmwf.int) 15 ECMWF TC/PR/RB L3 Apr 02
Selection of random numbers Random numbers can be selected with different temporal correlation scales. These two figures shows the effect on the amplitude of the perturbation tendency for two adjacent boxes when the random numbers are re-selected every time step (red) or every 4 time steps (green). In this case the numbers have been selected inside the interval -0.5 r(x) 0.5. Roberto Buizza (buizza@ecmwf.int) 16 ECMWF TC/PR/RB L3 Apr 02
Tuning of the relevant parameters: 5 cases Stochastic physics is controlled by 3 parameters: maximum amplitude A, spatial and temporal scales (D, T): r( λ, φ, p) =< ρ( λ, φ) > D, T Sensitivity experiments have been performed to tune these A Mean ACC skill parameters. This figure shows the mean (5-cases) skill (top) and the mean spread (bottom) for five settings: red IC only; blue DT only, A=1,D=1deg,T=3h; green black DT only, A=0.5,D=10deg,T=3h; DT only, A=0.5,D=5deg,T=3h; Mean ACC spread cyan DT only, A=1, D=1deg,T=1h. Roberto Buizza (buizza@ecmwf.int) 17 ECMWF TC/PR/RB L3 Apr 02
Tuning of the relevant parameters: 5 cases The 3 parameters (A, D, T) have been selected so that they have a measurable impact on the ensemble spread, they have a negligible impact on the performance of single deterministic forecasts and they improve ensemble scores. Sensitivity experiments indicated that Mean ACC skill a configuration with (A=0.5,D=10deg,T=6h) should be used. This figure shows the mean (5 cases) skill of three Results indicate that the average skill of the ensemble members with and without stochastic physics is similar. ensembles, with IC perturbations (IC, red), with only stochastic physics with (A=0.5,D=10deg,T=6h) (DT, blue) and with both ICDT (green). Roberto Buizza (buizza@ecmwf.int) 18 ECMWF TC/PR/RB L3 Apr 02
Tuning of the relevant parameters: 5 cases The impact on the skill of the ensemble-mean is also very small (top), while there is a detectable increase in ensemble spread (about 5% for Z500, bottom). Mean ACC skill ACC skill of ensemble-mean The top figure shows the average (5 cases) skill of the ensemblemean of ensembles run with IC perturbations only (IC, red), with stochastic physics (A=0.5,D=10deg,T=6h) only (DT, blue) and with both IC and DT (green). The bottom figure shows the ensemble spread. Mean ACC spread Roberto Buizza (buizza@ecmwf.int) 19 ECMWF TC/PR/RB L3 Apr 02
What is the impact of stochastic physics? Diagnostics of single deterministic integrations: The area-average unperturbed and perturbed tendencies should be very similar. This has been verified by comparing area-average model tendencies for different regions. Stochastic physics should not deteriorate the model performance. This has been verified by comparing e.g. total precipitation forecasts in a tropical region for which data were available. Stochastic physics should not affect the model climate. Roberto Buizza (buizza@ecmwf.int) 20 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on tendencies: single fcs The impact of stochastic forcing on the parameterized tendencies has been investigated for single deterministic forecasts started at 12UTC of 23 Oct 1996. This figure shows the 0-48 hour accumulated precipitation (top) and the Z500 height field (bottom). Average tendencies inside ten 10-degree boxes have been compared. Roberto Buizza (buizza@ecmwf.int) 21 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on tendencies: single fcs This figure shows the time variation of normalized wind, temperature and specific humidity tendencies for a dry region in the Pacific (140-130W;30-40N). Each tendency has been scaled by the 0-48h timeaverage value. Stochastic scaling factors are also shown. Solid (dashed) lines identify unperturbed (perturbed) tendencies. Roberto Buizza (buizza@ecmwf.int) 22 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on tendencies: single fcs The two top panels show the time variation of normalized specific humidity tendencies and stochastic scaling factors for a wet region in the Rockies (110-100W;40-50N). The two bottom panels show the temperature tendencies for a region in the Sahara (0-10W;20-30N). Note that the average Q-tendency is about 50 times larger than the tendency for the Pacific. Roberto Buizza (buizza@ecmwf.int) 23 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on precipitation: single fcs The daily values of total Daily area-average rainfall - TOGA COARE IFA precipitation accumulated between the 48- and 72-hour forecasts from the unperturbed (pink) and the perturbed (yellow) integrations have been compared with observations (blue, from Lin & Johnson 1996). Results indicate that 24-hour 24h rainfal (mm) 35 30 25 20 15 10 5 0 1 6 11 16 21 26 Date (December 1992) obs control stoch. Phys. rainfall-forecast differences are smaller than forecast errors. Observed and forecast area-average rainfall inside the TOGA- COARE intensive-flux area (0-10S;150-160E). Roberto Buizza (buizza@ecmwf.int) 24 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on precipitation: single fcs The daily-average values of total precipitation accumulated between the 144- and the 168-hour forecasts of the unperturbed (top right) and the perturbed (bottom right) integrations are very similar, with differences smaller than differences from verification (bottom left). Roberto Buizza (buizza@ecmwf.int) 25 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on precipitation: single fcs The error of the daily-average values of total precipitation accumulated between the 144- and the 168-hour forecasts from the unperturbed (top) and the perturbed (bottom) integrations are very similar. The error has been computed using the 0-24h forecast as a proxi for verification. Roberto Buizza (buizza@ecmwf.int) 26 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on the EPS: 14 cases Impact on precipitation forecasts Area under the ROC curve for the events 12h 5 mm accumulated precipitation larger than 5, 10 and 20 mm for the unperturbed (solid) and the stochasticallyperturbed (dashed) ensemble for the week 16-22 Dec 1997. Dotted and chain-dashed lines are for the 10 mm unperturbed and the stochastically-perturbed ensemble for the week 29 June-5 July 1997. Values refer to the NH. 20 mm Roberto Buizza (buizza@ecmwf.int) 27 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on the EPS: 14 cases Impact on ensemble spread T850hPa - NH Ensemble rms-spread for the 850hPa temperature for the unperturbed (solid) and the stochastically-perturbed (dashed) ensemble for the week 16-22 Dec 1997. Dotted and chaindashed lines are for the unperturbed and the stochastically-perturbed ensemble for the week T850hPa - Tropics 29 June-5 July 1997. Top panel is for the NH, bottom for the Tropics. Roberto Buizza (buizza@ecmwf.int) 28 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on the EPS: 14 cases Impact on 850hPa temperature Area under the ROC curve for the events 850hPa temperature warmer than climatology for the NH unperturbed (solid) and the stochasticallyperturbed (dashed) ensemble for the week 16-22 Dec 1997. Dotted and chain-dashed lines are for the unperturbed and the stochastically-perturbed ensemble for the week 29 June-5 July 1997. Top panel is for the NH, bottom for the Tropics. Tropics Roberto Buizza (buizza@ecmwf.int) 29 ECMWF TC/PR/RB L3 Apr 02
Stochastic physics as a source of spread in the tropics Currently, stochastic physics is the dominant source of ensemble spread in the tropical region (the SVs are optimized to grow north of 30N and south of 30S). This can be seen, e.g., by comparing the 24-h ensemble spread in terms of precipitation in a tropical region for ensembles run without (top) and with (bottom) stochastic physics. Roberto Buizza (buizza@ecmwf.int) 30 ECMWF TC/PR/RB L3 Apr 02
Stochastic physics as a source of spread in the tropics After 5-days of time integration, initial perturbations initiated north of 30N and south of 30S induce divergence also in the tropical region. Thus, the difference between the ensemble spread in terms of precipitation for ensembles run without (top) and with (bottom) stochastic physics is smaller. Roberto Buizza (buizza@ecmwf.int) 31 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics: the US storm case Between 25 and 26 Jan 2000 a very intense storm affected the US East Coast. Sensitivity experiments have indicated that stochastic physics had a key positive role in producing some skilful members. These figures show the MSLP analysis for 26 Jan 00UTC (top left) and t+60h ensemble forecasts (started on 23 Jan at 12UTC) run with and without stochastic physics. OPE NOST Roberto Buizza (buizza@ecmwf.int) 32 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics: the US storm case A summary of the performance of the ensembles run with (ope) and without (NOST) stochastic physics is given in terms of the number of ensemble members with MSLP intensity and position errors smaller than 5hPa/200km, 10hPa/400km and 15hPa/400km. Results indicate that the Number of ens-mem 50 45 40 35 30 25 20 15 10 5 0 20 Jan (+120h) NOST Verification 12 UTC on 25 Jan 2000 20 Jan (+120h) ope 21 Jan (+96h) NOST 21 Jan (+96h) ope 22 Jan (+72h) NOST 22 Jan (+72h) ope 23 Jan (+48h) NOST 23 Jan (+48h) ope IE/PE lt 5hPa/200km IE/PE 10hPa/400km IE/PE lt 15hPa/600km Verification 12 UTC on 26 Jan 2000 NOST ensembles are less able to correctly predict the intensity of the cyclone especially for forecast times longer than 48h. Number of ens-mem 50 45 40 35 30 25 20 15 10 5 0 21 Jan (+120h) NOST 21 Jan (+120h) ope 22 Jan (+96h) NOST 22 Jan (+96h) ope 23 Jan (+72h) NOST 23 Jan (+72h) ope 24 Jan (+48h) NOST 24 Jan (+48h) ope IE/PE lt 5hPa/200km IE/PE 10hPa/400km IE/PE lt 15hPa/600km Roberto Buizza (buizza@ecmwf.int) 33 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics: the US storm case MSLP 72-h forecasts started on the 23 Jan: top: spread (left) and error of EPS2 middle: spread (left) and error of NOST2 bottom: difference EPS2-NOST2 (left) and control error Stochastic physics has smaller impact than the initial perturbations (left panels). Stochastic physics intensifies the cyclonic circulation (bottom left) in the 72-h forecast and this reduces the forecast error (top and middle right). Roberto Buizza (buizza@ecmwf.int) 34 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics: the US storm case Vertical cross section of temperature differences averaged between 30-60N between 72- h forecasts started on the 23 Jan: top: spread (left) and error of EPS2 middle: spread (left) and error of NOST2 bottom: difference EPS2-NOST2 (left) and control error Stochastic physics induces some cooling in the lower troposphere around 70W (bottom left) in correspondence of the difference in MSLP. Roberto Buizza (buizza@ecmwf.int) 35 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on EPS QPP over the US Results are part of a study aimed to evaluate QPF Performance by EPS over the US (Mullen & Buizza 2001 MWR in press, also ECMWF TM 307). The study wanted to assess: Seasonal Differences between Summer & Winter Predictability of QPF with EPS Impact of System Upgrades on EPS performance Performance for Heavy Rainfall (>50 mm per day) Impact of Increased Resolution for Heavy Events Impact of Ensemble Size for Heavy Events Roberto Buizza (buizza@ecmwf.int) 36 ECMWF TC/PR/RB L3 Apr 02
Model Output and verification measures Model output. Ensemble forecasts have been Interpolated to 1.25 o x 1.25 o Lat-Lon Grid. Daily Forecasts from 1 January 1997 to 30 January 1999 have been considered. Winter is defined as November-March (5 months) and summer as May-September (5 months). Verification measures: Brier Skill Score (BSS) Ranked Probability Skill Score (RPSS) Relative Operating Characteristic (ROC) Verification of Rank 24 Hour Thresholds: 1, 10, 20 & 50 mm Region: Conterminous U.S. East of 105 o W Roberto Buizza (buizza@ecmwf.int) 37 ECMWF TC/PR/RB L3 Apr 02
Verifying Analyses Data from NOAA River Forecast Centers (~5,000 Stations over U.S.) NCEP Box Averaging Technique to Map Station Data to Uniform 1.25 o x 1.25 o Grid 5 Stations Minimum per Grid box Apply a Simple Quality Control to RFC Data to Eliminate Grossly Erroneous Reports Highest Density of Stations over East U.S. Also verify against station reports Roberto Buizza (buizza@ecmwf.int) 38 ECMWF TC/PR/RB L3 Apr 02
RFC Station Density Roberto Buizza (buizza@ecmwf.int) 39 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on EPS during winter over US Stochastic Physics was introduced in the operational EPS on 21 October 1998 (Buizza et al 1999a) following extensive experimentation. It should be noted than in March 1998 the EPS initial perturbations were changed to include evolved singular vectors (Barkmeijer et al 1999; Buizza et al 1999b). Comparison of Winters 1997/98 & 1998/99 (summer comparison not possible). Combined Impact of evolved singular vectors and Stochastic Physics. Results (not shown) have indicated that for precipitation prediction the impact of stochastic physics is dominant. The comparison is based on the old EPS system (51*TL159L31). Roberto Buizza (buizza@ecmwf.int) 40 ECMWF TC/PR/RB L3 Apr 02
Impact of stochastic physics on EPS during winter over US Brier Skill Score: 0.6 0.4 1 20 50 1 mm 1 day better 20 mm skillful 50 mm improved Skill 0.2 0.0-0.2-0.4-0.6 ROC area: Areas 0.1-0.2 larger 20 & 50 mm most improved -0.8 1.0 0.9 1 2 3 4 5 6 7 8 9 10 Fcst Day 1 20 50 Legend: colour lines for 1997/98 and black with Area 0.8 0.7 0.6 diamonds for 1998/99 (stochastic physics). 0.5 1 2 3 4 5 6 7 8 9 10 Fcst Day Roberto Buizza (buizza@ecmwf.int) 41 ECMWF TC/PR/RB L3 Apr 02
Conclusions A scheme to represent random errors due to parameterized physical processes has been described. The scheme is based on the assumption that model uncertainties due to parameterized physical processes are characterized by spatial and temporal scales similar to processes such as organized convection and cloud clusters (100-500km, 5-10h). The scheme is also based on the assumption that there is a certain degree of coherence between the errors due to different parameterization scheme. In its present configuration, the simple scheme is controlled by three parameters: overall maximum amplitude, space and time correlation. When used in the ECMWF EPS, the scheme increases the overall level of spread and improves substantially the quality probabilistic predictions of grid point weather variables such as precipitation. Roberto Buizza (buizza@ecmwf.int) 42 ECMWF TC/PR/RB L3 Apr 02
Bibliography On the ECMWF EPS: Molteni, F., Buizza, R., Palmer, T. N., & Petroliagis, T., 1996: The new ECMWF ensemble prediction system: methodology and validation. Q. J. R. Meteorol. Soc., 122, 73-119 (also EC TM 202). Buizza, R., Richardson, D., & Palmer, T. N., 2002: Global High-resolution (80 km) ensemble prediction. Q. J. R. Meteorol. Soc., submitted. On studies on initial and model uncertainties: Harrison, M. S. J., Palmer, T. N., Richardson, D. S., & Buizza, R., 1999: Analysis and model dependencies in mediumrange ensembles: two transplant case studies. Q. J. R. Meteorol. Soc., 126, 2487-2515. Houtekamer, P. L., Lefraive, L., Derome, J, Ritchie, H., & Mitchell, H., 1996: A system simulation approach to ensemble prediction. Mon. Wea. Rev., 124, 1225-1242. On the ECMWF stochastic simulation of random model errors due to physical parameterization: Buizza, R., Miller, M., & Palmer, T. N., 1999a: Stochastic representation of model uncertainties in the ECMWF Ensemble Prediction System. Q. J. R. Meteorol. Soc., 125, 2887-2908 (also EC TM 279). On the ECMWF EPS and precipitation prediction: Mullen, S., & Buizza, R., 2001: Quantitative precipitation forecasts over the United States by the ECMWF Ensemble Prediction System. Mon. Wea. Rev., 129, 638-663 (also EC TM 307). Buizza, R., Hollingsworth, A., Lalaurette, F., & Ghelli, A., 1999b: Probabilistic predictions of precipitation using the ECMWF Ensemble Prediction System. Weather and Forecasting, 14.,168-189 (also EC TM 248). Roberto Buizza (buizza@ecmwf.int) 43 ECMWF TC/PR/RB L3 Apr 02