SUMMARY OF & RECOMMENDATIONS FROM THE 4 TH ENSEMBLE USER WORKSHOP Zoltan Toth Environmental Modeling Center NOAA/NWS/NCEP Acknowledgements: Doug Hilderbrand, Louis Uccellini, Steve Lord & Workshop Participants http://wwwt.emc.ncep.noaa.gov/gmb/ens/index.html 1
4 th ENSEMBLE USER WORKSHOP Logistics May 13-15 2008, Laurel, MD, 70+ participants NWS Regions (7), Headquarters (15), NCEP (31) OAR (4), other government agencies (2), private (4), academic (4) sectors & international (3) For further info, see: http://wwwt.emc.ncep.noaa.gov/gmb/ens/userwkshop_may2008.html Main Theme How to support NWS in its transition from single value to probabilistic forecasting Goal is to convey forecast uncertainty in user relevant form 28 presentations Significant progress since last workshop Reports from NCEP Service Centers and Regions E.g., first numerical ensemble-based 2-day tornado and week 3-4 MJO outlook Working groups Operational Requirements - Corporate outreach Ensemble forecast systems - Statistical post-processing Data access / distribution - Display / manipulation tools Role of forecasters - Training Outcome / Recommendations Prepared draft blueprint for transition from single value to probabilistic forecasting Plan for immediate steps (interim solution to be implemented in 2-3 years) Outline for long term solution and resource requirements (5-10 years) WGs to refine & execute plan All activities to be coordinated under NWS Forecast Uncertainty Program (NFUSE)
BACKGROUND NRC Report on Completing the Forecast (2006) All major recommendations embraced NWS Forecast Uncertainty Program (NFUSE, 2007-) Corporate commitment and coordination AMS Ad Hoc Committee on Uncertainty in Forecasts (ACUF, 2007-) Enterprise-wide planning and coordination CURRENT STATUS Scientific and technical preparedness High due to major advances in past 50 years Past initiatives MDL PoP and other probabilistic products since 1971 NCEP ensemble forecasting since 1992 Numerous other guidance & products by WFOs & NCEP Corporate policy Good vision: Vision 2005 NWS Strategic Plan (1998) Deliver Better Products and Services Provide weather, water, & climate forecasts in probabilistic terms (2005) Sporadic efforts - No coordinated execution Example: Major NWS forecast database (NDFD) lacks basic uncertainty information Need overarching plan & Coordinated execution
DRAFT PLAN Goal Capability to answer any question related to future weather, climate, & water conditions, including forecast uncertainty Example What is joint probability of heavy precipitation and strong wind? Resource limitations Computing power & storage, telecommunication, workforce training, etc, etc Two-stage approach Interim stage Limited capability with approximations Achievable with reallocation of projected resources in 2-3 years Major enhancement of forecast system, leading toward long term solution Final stage Full capability Built on Stage-1 achievements Requires long term budget planning and commitments
RECOMMENDATIONS OPERATIONAL REQUIREMENTS Institute official requirement for probabilistic information for all NWS warnings & forecasts Expand NDFD midpoint estimate with 10 & 90 percentile values of forecast distribution Other single value probabilistic information derived via fitted distribution Provide 20+ member ensemble trajectories to Drive professional user applications Derive any weather, water, climate foecast information Maintain user relevant verification statistics (with level of uncertainty incl.)
RECOMMENDATIONS ENSEMBLE FORECAST SYSTEMS Recognize ensembles as backbone of probabilistic forecast system Significantly increase HPCS computing capacity for operations & research Increase THORPEX funding for related research, development, transition Encourage NSF funding of related research (eg, model uncertainty) Develop high-impact event local ensemble forecast system for operations Variety of applications (hurricane, severe storm, floods, fire, etc) Consider uncertainties related to boundary conditions Improve ensemble methods for coupled systems Ocean/Atmosphere/Land ssytem, eg, MJO application Link initial perturbation techniques with analysis uncertainty information Explore methods for representing model related uncertainty Continue use of multi-model approach (SREF, NAEFS) Explore alternative methods (stochastic approach, parameter uncertainty, etc) Continue hind-cast related activities Operationally implement limited but affordable system Carry out research on benefits of expanding hindcasts vs. improving frcst system Study optimal configuration for operational systems Possibly terminate single high resolution forecast beyond quasi-linear regime ~8 days for GFS? Provide 1/3/6 hourly output out to 1/8/35 days
RECOMMENDATIONS STATISTICAL POST-PROCESSING Analyze/compare performance of different methods Suggested approach Use common data / metrics Special focus on high impact events Components for evaluation Bias correction of model forecasts (relationship between analyzed & forecast model variables, including ensemble distribution) Downscaling (perfect prog relationships between model variables and observed quantities) Generation of observationally based analysis (used as a proxy for truth) Improve methods to increase forecast reliability while maintaining sharpness Study sensitivity/requirement of different user groups to/for various levels of forecast reliability Evaluate what sample size is required for desired level of reliability Develop methods to combine information from diverse sources into single pdf Ensemble, hires forecasts, climatology, latest observations, regime dependent biases, etc
RECOMMENDATIONS DATA STORAGE / ACCESS Data types Expanded NDFD data 10/50/90 percentiles (3-fold increase in volume, short term) Multi-model ensemble raw & bias corrected (~100-fold increase in volume, long term) Architecture Distributed data access with mirrored sites (WOC-type) All ensemble data archived at NCDC Access On demand pull (NOMADS-type, in contrast with on schedule push) Public access NOMADS AWIPS-2 Test with NOMADS until new software developed Link with display / interrogation tools below
RECOMMENDATIONS DISPLAY / MANIPULATION TOOLS Develop / adapt tools for probabilistic / ensemble guidance Graphical display of ensemble members / probabilistic info (spaghetti, stamps, etc) Migrate existing NAWIPS ensemble/probabilistic functionalities Statistical manipulation of uni-variate distributions 10/50/90 percentiles => Fitted distribution => Derive probability for any threshold & vice versa Choice of analytical distribution based on climatology Forecaster modification of 10/50/90 percentile fields Use existing, & develop more sophisticated ensemble-based field modification software Adjust ensemble distribution based on forecaster modification Use increment between distributions fitted to guidance vs. modified 1/90/90 percentiles Ensemble database interrogation To derive joint probabilities (and any other information) Diagnostics, perfect prog relationships (CAPE, helicity, etc), Verification software / results Use standardized software throughout forecast process / across organization Deploy necessary software in AWIPS2
RECOMMENDATIONS ROLE OF FORECASTERS Forecast process steps Ensemble guidance Statistical post-processing Forecaster modification Product derivation Interpretation Outreach Assess where forecaster can have maximum positive impact Which part of forecast process? High impact events? Use common internal format for probabilistic forecasts Internal format can be different from external products Common verification for evaluating value added at each step important Elevate role of forecaster Traditional role Make / edit forecasts New role Direct / QC forecast process Focus on high impact events Adaptively assign resources Interpretation of probabilistic forecasts, outreach
RECOMMENDATIONS TRAINING Universities Build into existing courses (hard to add new courses) On the job (for forecasters and sophisticated external users) Prepare inventory of, and provide easy access to existing training material NCEP, Regions, WFOs, COMET, MSC, WMO, etc. Develop online curriculum with multiple entry points and levels of advancement (WFO, RFC, Nantional Centers) Chaos, statistics, ensemble techniques/data, modification & interpretation of probabilistic guidance, user applications (eg., emergency management WCM) Case studies of high impact, large anomaly events for different regions Hosted by NWS, COMET, NOAA Learning Center? User feedback via blog? Provide mechanism for incorporation of locally developed material Periodically update training material Track training as part of official job requirement
RECOMMENDATIONS CORPORATE OUTREACH Use ACUF as NOAA s official interface with enterprise Utilize other channels NWS Partner s meeting, AMS/NWA conferences, workshops, journal articles, customer education Broadcaster conferences Assess end-user operational requirements Challenge as requirements evolve with availability of new products Survey focused on Emergency Management and Transportation needs Build on success of NURO survey Use WCMs expertise to bridge the gap between Expanding product/service suite & Evolving user needs
2-STAGE PLAN Stage 1 Interim capability (2-3 yrs) Reliable 10/50/90 in NDFD database Forecasters edit fields Official forecast All questions on uni-variate distributions can be answered via AWIPS-2 tools Guidance info available for Forecasters Web Stage 2 Full calability (5-10 yrs) ~100 bias corrected ensemble members in NOAA database Forecasters edit 10/50/90 corresponding to ensemble Forecaster edits back-propagated to ensemble All questions on future weather/climate can be answered via AWIPS-2 tools Ensemble trajectories consistent with official forecast for user applications
NEXT STEPS AFTER WORKSHOP Finalize recommendations Prepare short workshop report (BAMS) Report to Forecast Uncertainty oversight board Carry on WG activities Develop coordinated plan for next 2-3 yrs WG leads to coordinate activities within and across WGs under NFUSE Seek long term funding through PPBES Engage with ACUF example products? Plan for next workshop - Oct 2009
ISSUES FOR FURTHER DISCUSSION Midpoint estimate 50%, mean, or mode? Where to store forecast uncertainty info? NDFD Separate probabilistic NDFD dataset? Official forecast vs. guidance 10,90 part of expanded official forecast? Human reviewed / edited if needed Rest of derived products guidance? Forecaster edits 10,90 official forecast 25,75 May be easier to learn/train? Assembled training material Hosted by NWS, COMET, NOAA Learning Center?
BACKGROUND