Convection-Permitting Ensemble Forecasts at CAPS for Hazardous Weather Testbed (HWT)

Convection-Permitting Ensemble Forecasts at CAPS for Hazardous Weather Testbed (HWT) Ming Xue Center for Analysis and Prediction of Storms and School of Meteorology University of Oklahoma mxue@ou.edu August, 2010 ARPS Simulated Tornado

NOAA Hazardous Weather Testbed (HWT) HWT is a facility jointly managed by NSSL, SPC, and NWS Norman WFO To accelerate the transition of promising new technologies into forecasting and warning for hazardous weather. HWT organizes annual Spring Experiment that attracts about 100 researchers and forecasters each year. Provides forecasters with a first-hand look at the latest research concepts and potential future products, and immerses researchers in the challenges, needs, and constraints of forecasters.

HWT Spring Experiment Daily Discussions (pictures from spring 2007)

Storm-Scale Convection-Permitting Ensemble and Convection-Resolving Deterministic Forecasting CAPS/OU provided CONUS-scale 4-km ensemble & 1-2 km high-res forecasts for HWT Spring Experiment since 2007. NSSL, EMC, NCAR, and GSD provided additional 3-4 km deterministic forecasts.

Scientific Issues to Address The values and cost-benefits of storm-scale versus coarserresolution short-range ensembles versus even-higherresolution deterministic forecast; Suitable perturbation methods for storm-scale ensemble, e.g., IC, physics, and model perturbations; Proper handling and use of boundary perturbations; The value and impact of assimilating high-res data including those from radars; The most effective ensemble post-processing and most useful products at the convective scales; The impact of such unique products on forecasting and warning.

Forecast Configurations of Four Years Spring 2007: 10-member WRF-ARW, 4 km, 33 h, 21Z start time, NAM+SREF ICs. 5 members physics perturbations only, 5 with Phy+IC+LBC perturbations. Single 2 km grid. 2/3 CONUS (Xue et al.; Kong et al.; 2007 NWP conf.) Spring 2008: larger domain, 00Z start, Phy+IC+LBC pert for all. Radar Vr and Z data assimilation for 4 and 2 km grids! (Xue et al.; Kong et al. 2008 SLS Conf.) Spring 2009: 20 members, 4 km, 3 models (ARW, NMM, ARPS), mixed physics/ic/lbcs. Single 1 km grid. Radar DA on native grids. 30 h forecasts from 0Z (Xue et al.; Kong et al. 2009 NWP Conf.) Spring 2010: 26 4-km and one 1-km forecasts. Full CONUS domain. Some members with physics difference only, and 3 with storm-scale and mesoscale IC perturbations only for studying error growth and predictability. About 1.5 months each spring season from mid-april through early June http://forecast.caps.ou.edu.

Configuration of 2007 Ensemble WRF ARW Model at 4 km

Average Domain Total Precipitation Physics All members only members Native grids Ferrier/MYJ Thompson/YSU Stage II Obs WSM6/YSU (Schwartz et al. 2009a,b)

Areal Coverages Native grids WSM6/YSU Ferrier/MYJ Thompson/YSU

Domain-mean ensemble spread - averaged over 38 forecast dates from April 18 to June 7 (Kong et al. 2007) Local Standard Time: 18 00 06 12 18 00

Key Findings from 2007 Experiment Ferrier/MYJ schemes are associated with greater average precipitation YSU PBL scheme seems to be associated with relatively less precipitation, on average, in combination with WSM6 or Thompson microphysics Physics only members are under-dispersive for large-scale fields For precipitation, physics perturbations seem to generate as much spread as IC/LBC perturbations There is significant high bias for most members especially on the second day Convective-allowing ensemble clearly out-performs convectionparameterization ensemble in propagation, ETS, statistical consistency, ROC, etc. (Adam Clark s talk) 2 km forecasts didn t seem to provide much more value than 4 km forecasts for the second day guidance.

Forecast Configurations of Four Years Spring 2007: 10-member WRF-ARW, 4 km, 33 h, 21Z start time, NAM+SREF ICs. 5 members physics perturbations only, 5 with Phy+IC+LBC perturbations. Single 2 km grid. 2/3 CONUS (Xue et al.; Kong et al.; 2007 NWP conf.) Spring 2008: larger domain, 00Z start, Phy+IC+LBC pert for all. Radar Vr and Z data assimilation for 4 and 2 km grids! (Xue et al.; Kong et al. 2008 SLS Conf.) Spring 2009: 20 members, 4 km, 3 models (ARW, NMM, ARPS), mixed physics/ic/lbcs. Single 1 km grid. Radar DA on native grids. 30 h forecasts from 0Z (Xue et al.; Kong et al. 2009 NWP Conf.) Spring 2010: 26 4-km and one 1-km forecasts. Full CONUS domain. Some members with physics difference only, and 3 with storm-scale and mesoscale IC perturbations only for studying error growth and predictability (Xue et al. 2010; Kong et al. 2010 SLS conf.). About 1.5 months each spring season from mid-april through early June http://forecast.caps.ou.edu.

4 km ensemble and 2 km high-res domains 3600 x 2688 km

Movie of 2 km forecast v.s. observations 5 minute time intervals

Configuration of 4-km Ensemble

1-h accumulated precipitation 0.1in, t=12 h 0.01in, t=24 h 2008 2007 2008 2007

BIAS comparison 1-h accumulated precipitation 0.1 in 2008 2007

Bias correction based on first 12 days bias based on ranks for each hour (a) Sorted 1 h accumulated precipitation, and (b) differences between members and observation (bias) for the 24 h forecast, averaged over a 12-day period from April 16 to May 7, 2008. (Kong et al. 2008 SLS)

Bias corrected (for the later 15 days) >0.01 in/h >0.1 in/h Probability matching (Ebert 2001) >0.5 in/h > 1.0 in/h

Rank histogram of 1 h accumulated precipitation for 18 h, and 24 h, averaged over 15 days of bias corrected dates 18 h 24h (not much improvement to the reliability though)

ETS comparison 1-h accumulated precipitation 0.1 in 2008 2007

ARPS 3DVAR Analysis Grid 1 km grid: 3603 x 2691 x 51 WRF ARW (4 and 1 km) and ARPS forecast grid and common post-processing grid WRF NMM forecast grid

ETS for 3-hourly Precip. 0.5 in from HWT Spring Forecast Experiments 2008 (32-day) 2009 (26-day) With radar With radar no radar 12 km NAM no radar 12 km NAM Probability-matched score generally better than any ensemble member 2 km score no-better than the best 4-km ensemble member may be due to physics 1-km score better than any 4-km member and than the 4 km PM score.

BIAS for 1 h precip of 2009 (24-day average) 0.1 inch/h

Reliability diagram for precipitation probability forecast Reliability is improved by using multiple models 12 h forecast of 1 h accumulated precip. 0.1in

Object-Oriented Precipitation forecasts clusters (by Aaron Johnson) ARW ARPS NMM No Radar ARW ARPS NMM Microphysics PBL 4

2010 Spring Experiment Domains Full CONUS NMM 790x999 3DVAR 1200x780 ARW, ARPS & verification 1160x720

member IC BC Radar data microphy LSM PBL arw_cn 00Z ARPSa 00Z NAMf yes Thompson Noah MYJ arw_c0 00Z NAMa 00Z NAMf no Thompson Noah MYJ arw_m3 arw_cn + random pert 00Z NAMf yes Thompson Noah MYJ arw_m4 arw_cn + RF pert 00Z NAMf yes Thompson Noah MYJ arw_m5 arw_cn + em-p1 + RF pert 21Z SREF em-p1 yes Morrison RUC YSU arw_m6 arw_cn + em-p1_pert ARW member configuration (19) 21Z SREF em-p1 yes Morrison RUC YSU arw_m7 arw_cn + em-p2_pert 21Z SREF em-p2 yes Thompson Noah QNSE arw_m8 arw_cn nmm-p1_pert 21Z SREF nmm-p1 yes WSM6 RUC QNSE arw_m9 arw_cn + nmm-p2_pert 21Z SREF nmm-p2 yes WDM6 Noah MYNN arw_m10 arw_cn + rsmsas-n1_pert 21Z SREF rsmsas-n1 yes Ferrier RUC YSU arw_m11 arw_cn etakf-n1_pert 21Z SREF etakf-n1 yes Ferrier Noah YSU arw_m12 arw_cn + etakf-p1_pert 21Z SREF etakf-p1 yes WDM6 RUC QNSE arw_m13 arw_cn etabmj-n1_pert 21Z SREF etabmj-n1 yes WSM6 Noah MYNN arw_m14 arw_cn + etabmj-p1_pert 21Z SREF etabmj-p1 yes Thompson RUC MYNN arw_m15 00Z ARPSa 00Z NAMf yes WDM6 Noah MYJ arw_m16 00Z ARPSa 00Z NAMf yes WSM6 Noah MYJ arw_m17 00Z ARPSa 00Z NAMf yes Morrison Noah MYJ arw_m18 00Z ARPSa 00Z NAMf yes Thompson Noah QNSE arw_m19 00Z ARPSa 00Z NAMf yes Thompson Noah MYNN For all ARW members: ra_lw_physics= RRTM; ra_sw_physics=goddard; cu_physics=none

NMM member configuration (5) member IC BC Radar data mp_phy lw_phy sw-phy sf_phy nmm_cn 00Z ARPSa 00Z NAMf yes Ferrier GFDL GFDL Noah nmm_c0 00Z NAMa 00Z NAMf no Ferrier GFDL GFDL Noah nmm_m3 nmm_cn + nmmn1_pert 21Z SREF nmmn1 yes Thompson RRTM Dudhia Noah nmm_m4 nmm_cn + nmmn2_pert 21Z SREF nmmn2 yes WSM 6-class RRTM Dudhia RUC nmm_m5 nmm_cn + em-n1_pert 21Z SREF em-n1 yes Ferrier GFDL GFDL RUC For all NMM members: pbl_physics=myj; cu_physics=none ARPS member configuration (2) member IC BC Radar data Microphy. radiation sf_phy arps_cn 00Z ARPSa 00Z NAMf yes Lin Chou/Suarez Force-restore arps_c0 00Z NAMa 00Z NAMf no Lin Chou/Suarez Force-restore For all ARPS members: no cumulus parameterization Members in red contribute to the 15-member sub-ensemble for post-processed product

HWT images 12 18Z accumulated precipitation: 18h (June 14, 2010 OKC Flood Day) SSEF mean SSEF Prob match QPE SREF mean SREF Prob match NCEP 12 km NAM

HWT images 12 18Z accumulated precipitation: 18h (May 19, 2010) SSEF mean SSEF Prob match QPE SREF mean SREF Prob match NAM

Gilbert Skill Scores (ETSs) for CAPS s SSEF (4 and 1 km) ESRL/GSD s 3 km HRRR NCEP 12 km NAM From 2010 spring experiment

Referred publications from the data Schwartz, C., J. Kain, S. Weiss, M. Xue, D. Bright, F. Kong, K. Thomas, J. Levit, and M. Coniglio, 2009: Next-day convection-allowing WRF model guidance: A second look at 2 vs. 4 km grid spacing. Mon. Wea. Rev., 137, 3351-3372. Schwartz, C. S., J. S. Kain, S. J. Weiss, M. Xue, D. R. Bright, F. Kong, K. W.Thomas, J. J. Levit, M. C. Coniglio, and M. S. Wandishin, 2010: Toward improved convection-allowing ensembles: model physics sensitivities and optimizing probabilistic guidance with small ensemble membership. Wea. Forcasting, 25, 263-280. Clark, A. J., W. A. Gallus, Jr., M. Xue, and F. Kong, 2009: A comparison of precipitation forecast skill between small near-convection-permitting and large convection-parameterizing ensembles. Wea. and Forecasting, 24, 1121-1140. Clark, A. J., W. A. Gallus, Jr., M. Xue, and F. Kong, 2010: Growth of spread in convection-allowing and convection-parameterizing ensembles, In press. Clark, A. J., W. A. Gallus, Jr., M. Xue, and F. Kong, 2010: Convection-allowing and convectionparameterizing ensemble forecasts of a mesoscale convective vortex and associated severe weather. Wea. Forecasting, Accepted. Coniglio, M. C., K. L. Elmore, J. S. Kain, S. Weiss, and M. Xue, 2009: Evaluation of WRF model output for severe-weather forecasting from the 2008 NOAA Hazardous Weather Testbed Spring Experiment. Wea. Forcasting, Accepted. Kain, J. S., M. Xue, M. C. Coniglio, S. J. Weiss, F. Kong, T. L. Jensen, B. G. Brown, J. Gao, K. Brewster, K. W. Thomas, Y. Wang, C. S. Schwartz, and J. J. Levit, 2010: Assessing advances in the assimilation of radar data within a collaborative forecasting-research environment. Wea. Forecasting, Accepted.

Web links to papers and realtime products Xue, M., F. Kong, K. W. Thomas, J. Gao, Y. Wang, K. Brewster, K. K. Droegemeier, X. Wang, J. Kain, S. Weiss, D. Bright, M. Coniglio, and J. Du, 2009: CAPS realtime multi-model convection-allowing ensemble and 1-km convectionresolving forecasts for the NOAA Hazardous Weather Testbed 2009 Spring Experiment. 23nd Conf. Wea. Anal. Forecasting/19th Conf. Num. Wea. Pred., Omaha, NB, Amer. Meteor. Soc., Paper 16A.2. Kong, F. M. Xue, K. W. Thomas, J. Gao, Y. Wang, K. Brewster, K. K. Droegemeier, J. Kain, S. Weiss, D. Bright, M. Coniglio, and J. Du, 2009: A realtime storm-scale ensemble forecast system: 2009 spring experiment. 23nd Conf. Wea. Anal. Forecasting/19th Conf. Num. Wea. Pred., Omaha, NB, Amer. Meteor. Soc., Paper 16A.3. http://forecast.caps.ou.edu

Resources $125K/year CSTAR funding! NSF supercomputers. 18,000-core Cray XT-4 at NICS ~5 hours a day in 2010 All data archived (TBs/day) need to be fully exploited Collaboration in analyzing the data welcome.

Future Plan (in CSTAR Renewal Proposal) General direction: more emphasis on aviation weather (e.g., 3 weeks in June + May), more runs/day, shorter forecast ranges, fine-tuning of ensemble design, Multi-scale IC perturbations, ETKF perturbations, EnKF-based perturbations Land surface perturbations, Possible additional LBC perturbations, More intelligent choices of physics suites Addition of COAMPS Improved initial conditions via more advanced data assimilation Possible GSI analyses with target HRRR set up and other more experimental configurations/schemes Possible hybrid ensemble-gsi analysis Possible EnKF analysis Post-analysis and probabilistic products: e.g., calibration, bias removal, detailed performance evaluation, cost-benefit/trade off assessment, effective products for end users (e.g., those for aviation weather, severe storms); Integration/coordination with national mesoscale ensemble efforts (DTC/DET collaborations).

Probabilistic Warn-on-Forecast for Tornadoes - The ultimate challenge need ~100 m resolution Radar and Initial Forecast at 2100 CST Radar at 2130 CST: Accurate Forecast An ensemble of storm-scale NWP models predict the path of a potentially tornadic supercell during the next 1 hour. The ensemble is used to create probabilistic tornado guidance. Probabilistic tornado guidance: Forecast looks on track, storm circulation (hook echo) is tracking along centerline of highest tornadic probabilities Developing thunderstorm 50% 30% Most Likely Tornado Path 50% 30% Most Likely Tornado Path 70% 70% T=2200 CST T=2200 CST T=2120 CST T=2140 T=2130 T=2150 T=2120 CST T=2140 T=2130 T=2150 (Stensrud, Xue, et al. BAMS 2009) NSSL Warn on Forecast Briefing March 5, 2007

The Computers Used NICS, Kraken (~99K cores) For 2010: Exclusive use of a 18,000-core Cray XT-4 at NICS/University of Tennessee 6 hours a day PSC (4K cores) Thanks!