Winter Weather Workshop August 2002 Ensemble Forecasting

Transcription

1 Winter Weather Workshop August 2002 Ensemble Forecasting Stephen Jascourt, COMET NWP resource at NCEP Much of this presentation based on slides prepared by Jun Du (NCEP) and Bill Bua (COMET), and their material included contributions from Steve Tracton (NCEP) and Zoltan Toth (NCEP)

2 OUTLINE 1. What is an ensemble? 2. What kind of information can an ensemble add to the forecast? 3. Global Forecast System 3 days to 2 weeks 4. Short Range Ensemble Forecast (SREF) 0-3 days 5. Types of products, how to use them 6. Cases 7. COMET

3 What is an ensemble? A set of multiple predictions valid at the same time generated from reasonably different initial conditions and/or with various credible versions of models improves skill through averaging among the forecasts, which eliminates nonpredictable components provides reliable information on forecast uncertainties (e.g., probabilities) from the spread (diversity) amongst ensemble members

4 If you ve used more than one model in the forecast process, you ve done ensemble forecasting! Example: An Ensemble of Three NWP Models AVN, UKMET, ECMWF All initialized 12Z 6 May hr forecasts valid 12Z 9 May 2002

5

6

7

8 How is an EPS made? In principle, can use one or a combination of: Different initial atmospheric states Different models or variations on the same model (perturbed physics, dynamics, numerics) Different lateral or lower boundary conditions

9 ENSEMBLE FORECASTING SPAGHETTI CLUSTERS SLIGHTLY DIFFERENT INITIAL CONDITIONS OR MODEL FORMULATIONS PRODUCE A NUMBER OF POSSIBLE FORECASTS AND THE SPREAD OF THESE FORECASTS QUANTIFIES UNCERTAINTY

10 Same model and same initial conditions, but Different Convection Schemes

11 What is the rationale for ensemble prediction? Atmosphere is essentially a chaotic system Result: Small perturbations (initial or forecast errors) can potentially grow into large differences in atmospheric evolution Uncertainties in analyses inevitable => even a perfect NWP model can yield large errors (uncertainties) in forecasts NWP models not perfect: only approximate dynamical and physical behavior => added source of error (uncertainties) in forecasts

12 How chaotic can the atmosphere be? 8 day forecast

13 Chaos can reign even in the short-range! 3.5 day forecast

14 How do we find the initial condition errors that will grow? Singular vectors (ECMWF) Seeks out non-linear growing atmospheric modes Breeding method for initial condition perturbations (NCEP, Toth and Kalnay, 1993) Works out mathematically and practically to be roughly equivalent to singular vector method, but at a much lower cost

15 2) Perturbation grows in forecast 1) Initial Random perturbation 4) repeat: grow perturbations, scale back 3) Grown perturbation rescaled to size of typical analysis error 5) Resulting perturbations have 4-dimensional = 3d shape plus juxtaposition of different fields structure that makes a rapidly growing disturbance

16 What kinds of information can an ensemble add to the forecast?

17 Initial uncertainty = distribution of possible initial states Best single deterministic forecast Ensemble members Forecast uncertainty = distribution of possible forecast states verification INFO on DISTRIBUTION of SCENARIOS: - how many scenarios - how likely is each - how sharply defined is each

18 Increased accuracy! Operational higher resolution climo Ensemble mean NCEP global model

19 CONSIDER!!! High-Resolution Mesoscale models allow us to see features not in coarser models even small timing and placement errors can be significant in attempt to accurately forecast details; orographic local winds and precip highly sensitive to synoptic wind direction (see Mass, et al., 3/02 BAMS!!!). Forecaster judgement could mitigate problem details One detailed mesoscale model based forecast could allow the user to make highly specific and detailed inaccurate forecasts. (after Grumm)

20 What do we want an ensemble prediction system (EPS) to do? Encompass the case dependent range of possible forecast scenarios by region, circulation system, sensible weather elements, etc. Provide the most skillful forecast probability distribution (PDF) within the range of possibilities Facilitate the communication of forecast uncertainty - probabilistic forecast products - to the end-users (public, emergency managers, government agencies, etc.)

21 NWS Vision 2005 Goals NWS will move towards adding probabilistic forecast products. Move from subjective model to model comparison to the use of more objective ensemble prediction systems American Meteorological Society (AMS) Statement - Enhancing Weather Information with Probability Forecasts (3/02 BAMS!) The AMS endorses probability forecasts and recommends their use be substantially increased. HOW RECONCILE THIS WITH IFPS?????

22 Global Forecast System

23 AVN 00, 06, 12, 18 UTC MRF with late data cutoff is gone, fields labeled MRF are from AVN Current New T170 L42 T62 L added hours 84h 180h 384h 3 ½ d 7 ½ d 16 d T254 L64 T170 L42 T126 L28 84h 180h 384h 3 ½ d 7 ½ d 16 d Ensembles T126 L28 T62 L28 T126 L28 T62 L28 84h 180h 384h 84h 180h 384h 3 ½ d 7 ½ d 16 d 3 ½ d 7 ½ d 16 d 11 members (1 control, 10 perturbations) 11 members (1 cont., 10 pert) 00 UTC, 12 UTC 00 UTC, 06 UTC, 12 UTC, 18 UTC

24 Short-Range Ensemble Forecasts (SREF) What? 5 Eta 48 km (control + 2 perturbation pairs) 5 Regional Spectral Model 48 km (control + 2 perturbation pairs) [RSM has old AVN/MRF physics, not upgraded version] 5 Eta members using Kain-Fritsch convective parameterization Soon 5 RUC members will be added Soon? 5 ARPS (CAPS at Oklahoma) members may be added When? 21, 09 UTC in time for your use with 00, 12 UTC Eta to 63 hours Status? To become officially operational NWS-wide fall 2002 Output might get into AWIPS sometime in 2003 New user-friendly web interface linked from SREF home page, which is

25

26 Types of Products, How to use

27

28 Mean and Spread: Interpretation spread = standard deviation. Highest lowest is bigger than spread

29 Global 3 day forecast from 00 UTC 11/2/01 Depth uncertainty - how strong will trough be? Phase uncertainty - where will the trough axis be? SD - meters

30 SREF Uncertainty in strength of system

31 Global Uncertainty in intensity 3 day forecast from 00 UTC 11/2/01 980, SD~14 Uncertainty in forward speed/location Hurricane Michelle SD - hpa

32 Mean and Spread: Advantages Compact communication Can see field over entire domain Ensemble mean on average has greater skill than any individual member Spread (sample standard deviation) quantifies the degree of uncertainty

33 Mean and Spread: Limitations Assumes normal distribution of forecasts (bell curve with maximum likelihood at mean) Mean may hide important details Bi- or multi-modal solutions Timing problems in prediction of features COMMON PROBLEM: cyclone/shortwave timing mismatches all members have wave/cyclone but shows up much weaker in ensemble mean (phase cancellation) Precipitation forecasts (particularly where convective precipitation is expected to be important) Can use spread as guide to where mean may not be communicating the correct information, and use additional tools to make further assessments

34

35 Spaghetti Diagram Interpretation Phase uncertainty Amplitude uncertainty

36 Spaghetti Diagram Interpretation: Clustering Ensemble mean Clustering

37 SREF /0000V63 SREFX-CMB 500MB; 5820M

38 3-day forecast from 00 UTC 11/2/01, spaghetti diagram for ensemble global Uncertain location of incoming western trough Uncertain amplitude of eastern trough From CDC web site:

39 SREF Spaghetti Diagrams: Interpretation

40 Spaghetti Diagrams: Advantages Avoids the assumption of normally distributed data and pitfalls thereof Can tell if ensemble mean, if present, is representative of the ensemble as a whole Shows spread among ensemble members and whether there is clustering of members around two or more forecasts Shows mode (I.e. most frequently occurring solution) Indicates outliers which may overly influence the ensemble mean and spread

41 Spaghetti Diagrams: Limitations Limited to one or only a few contours Cannot see full field of interest over the full domain May not choose the right contour (use ensemble mean/spread to make the best choice)

42 PROBABILITY CHARTS SREF Percentage of members with QPF >.25 /24h /0000V63 SREFX-CMB; 24HR PQPF OF.25

43 SREF Highest QPF at each point exceeded by 60% of the ensemble members /0000V63 SREFX-CMB; SHADED, IN AT LEAST 60% OF MEMBES

44 SREF Probability of LI < /0000V63 SREFX-CMB; LIFTED INDEX PROB 0F < -4

45 Probability of precipitation exceeding thresholds, day 3 from 11/2/01 global

46 Probability charts: Advantages Depicts probabilities for exceeding critical value in a compact manner Variable of interest is seen over the full domain Uses actual distribution of data from ensemble members to determine probabilities

47 Probability charts: Limitations Do not get information on full PDF Only know percentage of ensemble members that exceed the value (sampling problem of limited ensemble size) Need to use several threshold values for complete picture Does not depict maximum value

48 EXAMPLE OF POINT DATA BOX AND WHISKER PLOTS FHR=> The blue boxes represent values from the 25% quartile (bottom of the box) to the 75% quartile (top of the box) with the median of the ensemble as a horizontal line in the box. The whiskers extend to both the max and min values supplied by the EPS output.. This allows you to instantly ascertain the uncertainty and the median (not the mean) in one view.

49 SREF Sequence provides info on envelope of storm tracks /0000V63 SREFX-CMB; SFC LOWS

50 5-day forecasts from 12z 11/22/01 and 00z 11/23/01, Valid 12 UTC 11/28/01 COMPARE SPAGHETTI INFO vs MEAN/SPREAD Uncertain location, orientation, and strength of AK block Uncertain location/strength of western and Pacific troughs Uncertain location and amplitude of eastern ridge

51 Note: Spread here is SD among ensemble members How strong and where will the block be? Phase uncertainty - where will the trough axis be?

52 Reflecting Seasonality and Recent Model Performance in Ensemble Forecasts Ensemble mean and normalized spread Average ensemble spread at each grid point and forecast lead time over last 30 days Divide current ensemble spread at each grid point and lead time by 30-day values Takes into account recent skill of model and seasonality in forecast variability

53 Spread appears high. Spread appears small.

54 .but actually relatively low for this location / season!.but is actually larger than usual for this location/season

55 Relative Measure of Predictability (RMOP) Premise: The more ensemble members predict an event, the more likely it is that the event will occur Ten equally likely bins set up for each grid box and variable, using climatology from NCEP/NCAR Reanalysis Ensemble mean and member values placed in climo. bins at each point RMOP determined by number of ensemble members in same bin as ensemble mean compared to last 30 days, and frequency over same period that the ensemble mean verified at each grid point. Graphic: Shows ensemble mean and relative measure of predictablility, calibrated by ensemble mean verification

56 Block: Relatively predictable 90% of the time in the last 30 days, fewer ensemble members fell into the same bin as the ensemble mean (I.e. relatively high predictability as we ve defined it here), BUT only verified 43% of the time!

57 Unpredictable, Strong gradient Ridge/Trough Highly predictable

58 Is there predictability in this 132-h fcst verifying 12z 11/28/01 Ens. mean Only one ensemble member in same bin as ensemble mean Distribution is bimodal

59 SREF Behavior: Current Configuration RSM and Eta tend to group into separate clusters (especially for QPF) SREF Eta tends to have smaller spread than the RSM Illustration in next three slides Yellow=RSM Green=SREF Eta Bold contours = RSM and Eta means

60

61

62 outliers

63 SREF Behavior: The Eta-KF Ensemble Subset Increased ensemble spread Could be result of using KF/mass-flux scheme or use of 4 th order (less damping) diffusion scheme Sharper gradients, including in stability parameters Tendency for larger instability Consistent with later triggering of convective scheme than for BMJ, particularly in weakly capped regions

64 SREF Eta/BMJ SLP mean/spread

65 SREF Eta/KF SLP mean/spread Increased spread

66 SREF Eta/BMJ CAPE

67 SREF Eta/KF CAPE

68 HPC Winter Weather Experiment Products/Services 2 shifts per day 630AM/PM 430PM/AM Issuing 3 graphics Watch/Warning Guidance Graphic Storm Tracks Graphic Conditional Probability Graphic Using Chat Room software As WFO coordination tool Low track valid 12Z/06 00Z/08

69 Winter Weather Products directly from SREF (graphical) Probability of freezing rain for each 3, 6, 12 and 24 hour period Joint probability of freezing rain and PQPF exceeding specified criteria Mean, maximum and minimum snow amounts and freezing rain for 3, 6, 12 and 24 hour periods

70 SREF Conditional Probability Product Example of product directly from SREF that is available to HPC as guidance for generating collaborative winter weather products. Storm tracks include SREF and all other models

71 PRECIP TYPE DETERMINED BY BALDWIN- SCHICTEL ALGORITHM

72

73

74

75 Probability of exceeding Flash Flood Guidance

76 CASES

77 Ensemble Performance January 31, 2002 Prob Snow Prob Freezing Rain 27 hour forecasts valid 12Z January 31 9AM Radar Jan. 31, 2002

78 Ensemble Performance January 31, 2002 Dominant precipitation type 27 hour forecast valid 12Z January 31 9AM Radar January 31

79 9AM Satellite January 31, 2002 Where s the rain-snow line?

80 Official forecast => winter storm warning with of 3-6 inches predicted in DC and 5-10 inches in Balt. Reality: DC and Balt woke up on the morning of the 30 th with sunny skies and, surprise, no snow. Eta from 12 GMT 29 Dec 24 hour accuml precip ending 00GMT 31 Dec 2000

81 SREF 24hr spaghetti from 12Z 29 Dec for hr precip ending 12 GMT 30 Most ensemble members push heavier precip offshore

82 JAN 6-7, 2002, obs snowfall

83 SREF from just hours before the snow has precip as mostly rain. Earlier runs had precip south, hardly any over PA 1002 hpa 0.2 3hr QPF Rain/snow line

84 Raob wind, raob hght, analysis wind, hght Eta Analysis Just as bad

85 Even ensemble members, with their perturbations, did not match obs. Analysis so bad that solution not in ensemble envelope!

86 Convection over Gulf in sharp trough often trouble for analysis!

87 Ensembles more consistent run to run Yellow = operational MRF, same valid times Initial=00 UTC 8 April 2002 initial=00 UTC 9 April 2002

88

89 60 hr Eta vt 12Z Aug 6, 2001

90 63 hr SREF vt 12Z Aug 6, 2001

91 Out West we have no SREF cases examined How are the short-range ensembles doing? Topography more important what is tradeoff between 12-km resolution and 48-km ensembles? Suggestion: Ensembles help give larger scale picture so high-resolution runs can either be accepted with confidence or modified

92 Additional Ensemble Links - global ensemble MOS - CASES!!! - global ensemble home -SREF home - global anomaly - SREF anomaly -maps -maps - Pete Manousos web page ATION/CHAOS/Chaos.html

93 COMET Bill Bua is preparing a PCU-1 module on ensembles, probably to be followed by teletraining COMET cases will be including more ensemble cases: