HPC Ensemble Uses and Needs

Transcription

1 1 HPC Ensemble Uses and Needs David Novak Science and Operations Officer With contributions from Keith Brill, Mike Bodner, Tony Fracasso, Mike Eckert, Dan Petersen, Marty Rausch, Mike Schichtel, Kenneth James, Bob Kelly, Ed Danaher, and Jim Hoke 2011 NCEP Ensemble Users Workshop May 10, 2011

2 2 Thank you from HPC Continued excellent working relationships Responsive to inquiries and requests Support and improvements for operational models Support for NOAA Hydrometeorological Testbed HPC 16 ongoing HPC-EMC projects (parallels, CSTAR, testbeds) Testbed collaborations have been successful and continue to grow

3 HPC Operations Every forecast desk uses ensemble information QPF Winter Weather Medium Range HPC is dependent on ensemble information Alaska Med. Range Model Diagnostics Short Term Weather 3 Puerto Rico QPF International Tropical

4 4 Available Ensemble Systems System Members Parameters GEFS 21 All members, mean, spread, anomalies CMCE 21 Mean, spread NAEFS 42 Mean, median, spread, 10/90 th percentiles SREF 21 All members, mean, spread, anomalies ECMWF 51 All members, mean, spread, anomalies

5 5 How HPC Uses Ensembles: Subjective Uses

6 Subjective Use of Ensembles Spaghetti QPF isohyets Low Clusters Mean/Spread Assess predictability Establish forecaster confidence Identify outliers and clusters Identify preferred solutions 6

7 7 Subjective Use of Ensembles Identify preferred solutions Used as inputs to model blends (weights) selected by forecasters for first-guess forecast

8 8 Subjective Use of Ensembles Uncertainty information communicated via forecasts, discussions and user-interactions

9 9 How HPC Uses Ensembles: Objective Uses

10 10 Objective Use of Ensembles Role of the Forecaster 4 th NCEP Ensemble User s Workshop: Q: Can a forecaster skillfully edit PDFs? A:?? 5 th NCEP Ensemble User s Workshop: Q: Can a forecaster skillfully edit PDFs? A: Maybe. Workload makes this difficult for multiple thresholds. Testing approaches combining human forecasts with objective ensemble information to improve PDFs.

11 11 Probability HPC PQPF Method Modifies ensemble distribution such that HPC deterministic QPF is the mode, while allowing skew Ensemble Spread (SREF+NAM+GFS+ ECMWF) Based on Bi-Normal Method Toth and Szentimrey (1990) QPF HPC most likely deterministic value

12 12 Probability HPC PQPF Method Modifies ensemble distribution such that HPC deterministic QPF is the mode, while allowing skew Based on Bi-Normal Method Toth and Szentimrey (1990) QPF HPC most likely deterministic value

13 13 Objective Use of Ensembles Role of the Forecaster Combine manual deterministic forecast with ensemble distribution and spread to generate probabilities Deterministic Snow Deterministic QPF

14 14 Objective Use of Ensembles Role of the Forecaster Combine manual deterministic forecast with ensemble distribution and spread to generate probabilities Probability > 4 Snow Probability > 0.50 QPF

15 15 Preliminary Verification Day 1 Brier Skill Score April 2010 March 2011 HRMOS and Tulsa approaches best at lower thresholds HPC PQPF and ensemble best at higher thresholds HPC PQPF generally has comparable skill to ensemble Hard to account for bi-modal situations

16 Objective Use of Ensembles Generation of automated probabilistic products based entirely on ensemble output SREF-based winter weather impact graphics 16 Probability of Duration 24 h of Winter Accumulating Precip Probability of meeting Blizzard Criteria

17 Targeted Observing Objective: Improve Forecasts of Events Through Targeted Observations in Data Sparse regions in the h lead time Adaptive approach to collection of observational data: 1) Only prior to events of socioeconomic interest -Determined by human 2) Only where influencing forecast event -Determined using ETKF 17

18 18 Ensemble Needs

19 19 Ensemble Needs Improve skill Reliability and spread Resolution Warn-on-Forecast for deadly floods/hazards Resolve key features to improve reliability

20 Improve Skill Underdispersion Continue to see clustering of ensemble solutions by modeling center Humans can hedge toward one cluster or another CMC GEFS ECMWF h Forecast valid 12 UTC 2 May

21 21 Improve Skill Resolution Oklahoma City Flash Flood, June 14, hr forecast, ending 18 UTC 14 June 2010 SREF SSEF Observed Probability of 0.5 Probability of 2

22 Percentage Improve Skill Resolution Storm Scale Ensemble (SSEF) is a Transformational Improvement 100% 90% 80% 70% 60% 50% 40% 30% 20% 10% 0% 2010 HWT Spring Experiment SSEF Performance Compared to the SREF 26/40 SSEF Model 3/40 Courtesy Adam Clark, NSSL 22 improved guidance worse guidance

23 23 Ensemble Needs Improve skill Reliability and spread Resolution Warn-on-Forecast for deadly floods/hazards Resolve key features to improve reliability Improve Post-processing Calibrate output, including QPF Data visualization and organization to reduce forecaster data overload Cyclone Tracks, joint probabilities, cluster EOFs

24 24 Downscaled Bias-Corrected NAEFS Median and Mean are impressive: -improves over MOS. -very competitive with HPC. Behavior of 10 th and 90 th consistent with expectations 10 Improve Post-Processing Mode Alaska NAEFS Max Temp MAE July-October 2010 NDFD MOS Mode Mean Median HPC 2 Day 4 Day 5 Day 6 Day 7 Day 8 Need full suite of elements for Alaska & CONUS

25 25 Improve Post-Processing Feature-Specific Information Present uncertainty information for specific cyclone

26 26 Improve Post-Processing Cluster Information Mine ensemble for common weather scenarios 168-h forecast ECMWF clusters of 500-hPa heights and anomalies verifying 12 UTC 10 May

27 27 Ensemble Needs Improve skill Reliability and spread Resolution Warn-on-Forecast for deadly floods/hazards Resolve key features to improve reliability Improve Post-processing Calibrate output, including QPF Data visualization and organization to reduce forecaster data overload Cyclone Tracks, joint probabilities, cluster EOFs Translate probabilities into decision support Match reliable probabilities to user thresholds

28 F180 VT Dec 2010 Decision Support Dec 2010 Blizzard F072 VT Dec 2010 L 28 Solution verifying outside/edge envelop

29 29 New York City Report: Preliminary Review of City s Response to the December 2010 Blizzard From Introduction On December 26, 2010, a blizzard struck New York City and surrounding areas. Though earlier forecasts had called for only a light to moderate snow falls, the National Weather Service issued a blizzard warning at 3:55 PM on December 25th. At that time, between 11 and 16 inches of snow were predicted, along with high winds and low visibility. Even this forecast underestimated the storm s ferocity. Third of six key problems Insufficient and delayed deployment of City assets. The weather forecast for the storm got significantly worse rather quickly, culminating in a blizzard warning issued at 3:55pm on Christmas Day. Due to the late change in the forecast, as well as the fact that DSNY has adeptly handled large snowfalls so many times previously without assistance, agencies that are not typically involved in snow removal such as the Taxi and Limousine Commission were not mobilized expeditiously.

30 30 Decision Support In order to prepare society for low probability highimpact events: Develop user thresholds using cost-benefit Match reliable probability to user thresholds End-to-end improvements in process needed. F072 VT Dec 2010 L

31 Role of Testbeds 2010 Spring Experiment 3 components (Severe, Aviation, QPF) 5 week program (May 17- June 18) ~70 participants from research, academia, and operations Rotation thru desks Facilitator at each desk 31 OVERALL GOAL: Explore use of convectionallowing models/ensembles (~4 km) QPF Goal: Does convection-allowing guidance help with the warm season QPF problem?

32 32 Role of Testbeds HMT-HPC Winter Weather Experiment 14 participants (WFO, SPC, AWC, HPC, EMC, ESRL, and COMET) Can high-resolution models improve Day 1 forecasts of precipitation type and amount? Can we better quantify and communicate winter weather uncertainty for Day 1-2 forecasts? Experimental NAM HRW-ARW HRW-NMM Observed

33 Summary 33 Ensembles are essential to HPC operations Subjective and objective uses Top three Needs: Skill improvements Reliability and spread Storm Scale Ensemble for Warn-on-Forecast Post-processing improvements for sensible weather elements, including feature-based displays Translating information into decision support Testbed activities helping to improve end-to-end ensemble systems Spring Experiment HMT-HPC Winter Weather Experiment

34 % Improvement in Threat Score Brier Skill Score (x 100) HPC Winter Weather Desk Verification Ensemble provides critical starting point for HPC probabilistic forecast product human forecast shows value-added Deterministic SNOW HPC skill relative to automated superensemble Probabilistic SNOW Skill Relative to Automated Superensemble D2 4" 8" 12" " 8" 12" -20 Day 1 Day 2 Day 3 0 Day 1 Day 2 Day 3 34 Superensemble : NAM+GFS+ECMWF+ SREF mean +GEFS mean

35 35 Winter Weather Experiment Example Confidence Discussion Storm track: Fairly confident of storm track, though high resolution models seem to deepen low center more and pull it further north and west into the cold air. Confidence: above average confidence that an area near Chicago will receive up to 20" of snow. Snow Probs: >12": 80% >20": 20%

36 36 Improve Post-Processing Feature-Specific Information Objectoriented tools applied to highresolution ensembles to indentify mesoscale bands