Development of Ensemble Based Probabilistic Guidance

Development of Ensemble Based Probabilistic Guidance Bo Cui and Yuejian Zhu Presentation for Ensemble User Workshop May 10 th 2011 1

Statistical Post-Processing Issues GOAL Improve reliability while maintaining resolution in NWP forecasts Reduce systematic errors (improve reliability) while Not increasing random errors (maintaining resolution) Retain all useful information in NWP forecast METHODOLOGY Use bias-free estimators of systematic error Need methods with fast convergence using small sample APPROACH Computational efficiency Bias Correction : remove lead-time dependent bias on model grid Working on coarser model grid allows use of more complex methods Feedback on systematic errors to model development Downscaling: downscale bias-corrected forecast to finer grid Further refinement/complexity added No dependence on lead time 2

Current NCEP/EMC Statistical Post-Processing System Bias corrected NCEP/CMC GEFS and NCEP/GFS forecast (up to 180 hrs), same bias correction algorithm Combine bias corrected NCEP/GFS and NCEP/GEFS ensemble forecasts Dual resolution ensemble approach for short lead time NCEP/GFS has higher weights at short lead time NAEFS products Combine NCEP/GEFS (20m) and CMC/GEFS (20m), FNMOC ens. will be in soon Produce Ensemble mean, spread, mode, 10% 50%(median) and 90% probability forecast at 1*1 degree resolution Climate anomaly (percentile) forecasts also generated for ens. mean Statistical downscaling Use RTMA as reference - NDGD resolution (5km/6km), CONUS and Alaska Generate mean, mode, 10%, 50%(median) and 90% probability forecasts 3

Bias Correction Method & Application Bias Correction Techniques array of methods Estimate/correct bias moment by moment (e.g., D. Unger et al.). Simple approach, implemented partially May be less applicable for extreme cases Bayesian approach (e.g., Roman Krzysztofovicz) Allows simultaneous adjustment of all modes considered, under development Moment-based method at NCEP: apply adaptive (Kalman Filter type) algorithm decaying averaging mean error = (1-w) * prior t.m.e + w * (f a) For separated cycles, each lead time and individual grid point, t.m.e = time mean error 6.6% 3.3% Test different decaying weights. 0.25%, 0.5%, 1%, 2%, 5% and 10%, respectively Toth, Z., and Y. Zhu, 2001 1.6% Decide to use 2% (~ 50 days) decaying accumulation bias estimation 4

NAEFS bias correction variables (bias correction) Variables pgrba_bc file Total 49 (13) GHT 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 10 (3) TMP 2m, 2mMax, 2mMin, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 13 (3) UGRD 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 11 (3) VGRD 10m, 10, 50, 100, 200, 250, 500, 700, 850, 925, 1000hPa 11 (3) VVEL 850hPa 1(1) PRES Surface, PRMSL 2(0) FLUX (top) ULWRF (toa - OLR) 1 (0) 13 new vars Notes 5

Statistical downscaling for NAEFS forecast Proxy for truth RTMA at 5km resolution Variables (surface pressure, 2-m temperature, and 10-meter wind) Downscaling vector Interpolate GDAS analysis to 5km resolution Compare difference between interpolated GDAS and RTMA Apply decaying weight to accumulate this difference downscaling vector Downscaled forecast Interpolate bias corrected 1*1 degree NAEFS to 5km resolution Add the downscaling vector to interpolated NAEFS forecast Application Ensemble mean, mode, 10%, 50%(median) and 90% forecasts 6

NAEFS downscaling parameters and products Last update: May 1 st 2010 (NDGD resolutions) Variables Domains Resolutions Total 4/8 Surface Pressure CONUS/Alaska 5km/6km 1/1 2-m temperature CONUS/Alaska 5km/6km 1/1 10-m U component CONUS/Alaska 5km/6km 1/1 10-m V component CONUS/Alaska 5km/6km 1/1 2-m maximum T Alaska 6km 0/1 2-m minimum T Alaska 6km 0/1 10-m wind speed Alaska 6km 0/1 10-m wind direction Alaska 6km 0/1 Note: Alaska products implementation: Dec. 7, 2010 All products at 1*1 (lat/lon) degree globally Ensemble mean, spread, 10%, 50%, 90% and mode 7

NCEP/GEFS raw forecast 4+ days gain from NAEFS NAEFS final products From Bias correction (NCEP, CMC) Dual-resolution (NCEP only) Combination of NCEP and CMC Down-scaling (NCEP, CMC) 8

NCEP/GEFS raw forecast 8+ days gain NAEFS final products From Bias correction (NCEP, CMC) Dual-resolution (NCEP only) Combination of NCEP and CMC Down-scaling (NCEP, CMC) 9

Alaska Downscaling 6km resolution Implemented by December 7 th 2010 10

Application for Alaska region and HPC Alaska desk 10-m U Max Temp Solid RMS error Dash - spread Bias (absolute value) 10-m wind speed Max Temp CRPS small is better Bias (absolute value) 11

MAE (Degrees F) Objective Evaluation Alaska NAEFS Maximum Temperature MAE July-October 2010 3.5 3 2.5 2 1.5 1 Mean, 50 th, and HPC best HPC NDFD GEMODE GEAVG GE50PT GMOS00 GMOS12 0.5 0 Day 4 Day 5 Day 6 Day 7 Day 8 Forecast Day Courtesy of Dave Novak 12

MAE (m/s) 3.5 Alaska NAEFS Wind Speed MAE July-October 2010 3 2.5 2 1.5 50 th (median) and mean are best HPC NDFD GEMODE GEAVG GE50PT GMOS00 GMOS12 1 0.5 0 D4 06Z D4 12Z D4 18Z D4 00Z D5 06Z D5 12Z D5 18Z D5 00Z D6 06Z D6 12Z D6 18Z Forecast time D6 00Z D7 06Z D7 12Z D7 18Z D7 00Z D8 06Z D8 12Z D8 18Z D8 00Z 13 Courtesy of Dave Novak

Upgrade for CONUS downscaling Plan for implementation - Q4FY2011 14

NAEFS downscaling parameters and products Plan: Q4FY2011 (NDGD resolutions) Variables Domains Resolutions Total 10/8 Surface Pressure CONUS/Alaska 5km/6km 1/1 2-m temperature CONUS/Alaska 5km/6km 1/1 10-m U component CONUS/Alaska 5km/6km 1/1 10-m V component CONUS/Alaska 5km/6km 1/1 2-m maximum T CONUS/Alaska 5km/6km 1/1 2-m minimum T CONUS/Alaska 5km/6km 1/1 10-m wind speed CONUS/Alaska 5km/6km 1/1 10-m wind direction CONUS/Alaska 5km/6km 1/1 2-m dew-point T CONUS 5km 1/0 2-m relative humidity CONUS 5km 1/0 Total cloud cover? All downscaled products based on 1*1 (lat/lon) degree globally Products include ensemble mean, spread, 10%, 50%, 90% and mode 15

Process to Downscale Tmax & Tmin for CONUS Based on 1*1 degree 6-hr bias corrected Tmax, Tmin and down-scaling vectors (DV) for T2m at each 6-hr cycle Definition of Tmax and Tmin for Conus region Tmax period: 11/12UTC(7am-local) 23/00UTC(7pm-local) EAST Tmin period: 23/00UTC(7pm-local) 12/13UTC(8am-local) EAST Definition of approximated period for Tmax and Tmin for giving initial cycle Mean DV of T2m for 6-hr period: weighted average of two instantaneous DVs Interpolating bias corr. 6-hr Tmax & Tmin (1X1) to 6km NDGD grid for Conus Detailed Process Apply mean DV to each grid point, each ens. member, and each 6-hr lead-time period, to produce down-scaled Tmax and Tmin for each 6-hr lead-time period Find out highest Tmax and lowest Tmin for approximated period For different grid points, different ens. members, highest Tmax could be in different 6-hr period, the same for lowest Tmin Only one down-scaled Tmax and Tmin for every 24-hr. fcst, up to 384 hours Calculate the mean, spread, mode, 10%, 50% and 90% based on above step 16

Process to Downscale Dew Point Temperature Output products: DPT, RH2m NCEP & CMC bias corrected DPT, RH2m on 1 degree NAEFS probabilistic forecasts on 1 degree: mean, spread, mode, 10%, 50% and 90% fcst Downscaled probabilistic forecasts on NDGD 5km NCEP/CMC Bias correction process on 1 degree Calculate DPT fcst. & analysis from T2m and RH2m, accumulate bias by applying decaying weight Calculated DPT from T2m and RH2m forecast, bias correct DPT Adjust bias corrected DPT, comparing with bias corrected T2m, smaller value as the DPT Bias correct RH2m by removing RH2m bias accumulation NCEP/CMC combination process on 1 degree Combine bias corrected NCEP/CMC DPT to generate probabilistic forecasts Compare DPT and T2m probabilistic forecasts, DPT are smaller than T2m Combine NCEP/CMC RH2m to generate probabilistic forecasts Adjust RH2m probabilistic forecasts, values not larger than 100% or smaller than 0% Downscaling process Get downscaling vectors (DV) for DPT, T2m and RH2m at each 6-hr cycle Apply DV to produce down-scaled DPT,T2m and RH2m for each 6-hr lead-time period Generate the mean, spread, mode, 10%, 50% and 90% based on above step 17

Schemes to develop T2m, Td2m, RH probabilistic products Products 1x1 degree resolution NDGD (5km) resolution Raw ensemble members T2m Td2m RH T2m Td2m RH Yes Yes Derived from t2m and RH Yes No No No Bias corrected ensemble members Yes Yes One end is bounded (T2m) Yes Two ends are bounded (0,100) No No No Ensemble mean, mode, 10%, 50% and 90% Yes Yes One end is bounded (T2m) Yes Two ends are bounded (0,100) Yes Apply DV RTMA - Yes Yes Apply DV RTMA - Yes Yes Apply DV RTMA - Yes One end is bounded Two ends are bounded Ensemble spread Yes Yes Yes Yes Interpolated Yes Interpolated Yes Interpolated bounded Probabilistic products and spread of T2m and Td2m are not compatible to RH 18

Tmax Tmin CRPS CRPS Latest evaluation for CONUS by apply : Temperature CRPS 1. Bias correction at 1*1 degree for NCEP GFS and GEFS, CMC/GEFS 2. Hybrid bias corrected NCEP/GFS and NCEP/GEFS 3. Combined all forecasts at 1*1 degree with adjustment 4. Apply statistical downscaling for joint forecast at 5*5 km (NDGD) grid - NAEFS 19

U 10m V 10m CRPS CRPS Wind speed Wind direction CRPS CRPS 20

Dew point T RH2m RMS & Spread RMS & Spread Dew Point T RH2m CRPS CRPS 21

Development Plan of Statistical Post-Processing for NAEFS Opportunities for improving the post-processor Utilization of additional input information More ensembles - high resolution control forecasts, SREF, GEFS Using reforecast information to improve week-2 and precipitation Improving analysis fields (such as RTMA and etc..) Improving calibration technique Calibration of higher moments (especially spread) Use of objective weighting in input fields combination Processing of additional variables with non-gaussian distribution Improve downscaling methods Future plan (overlook beyond 2-3 years) 22 Bias correct all model output variables (>200 which include precipitation)

Background 23

Downscaling CONUS Expansion Current status for downscaling - CONUS and Alaska Statistical downscaling methods implemented to NAEFS for CONUS, 2007 Statistical downscaling methods implemented to NAEFS for Alaska, Dec.7, 2010 Downscaling advantage and benefits Statistical down-scaling adds value Bias correction with downscaling adds significant value to the forecasts T2m, Tmax and Tmin are significant improved when compared to winds Downscaling CONUS expansion Tmax & Tmin: similar to process for Alaska choosing proper daily Tmax & Tmin period definition Wind speed and direction: similar to process for Alaska T2d, or RH: new method design 24

NAEFS status Multi-model, multi-center ensemble NCEP GFS high resolution deterministic NCEP GFS based ensembles (GEFS) CMC global ensembles Current status NCEP 20+1+1 members per cycle, 4 cycle per day CMC 20+1 members per cycle, twice per day Every 6 hours, out to 16 days Products and Benefits Bias corrected at 1*1 degree (35 variables, will be 49) Hybrid with NCEP bias corrected GFS forecast Combined (with adjusted) bias corrected CMC s ensemble Probabilistic products (10%, 50%, 90%, mean, mode and spread) Downscaled products for CONUS (T2m MAE and CRPS) and Alaska (Q1FY11 verification, HPC s evaluation, Tmax and Windspeed) Improving hurricane tracks Plan for future development Expand variables: T2d, RH, total cloud cover, PWAT? Expand domain??? Blending ECMWF ensemble in??? Improve downscale methods (smartini + downscaled vector)? 25

Outlines NAEFS status Current NCEP/EMC statistical post-processing system NAEFS downscaling parameters and products Downscaled products verification for CONUS and Alaska PQPF products Support Hanson Dam flood threat Special guidance, 4 times per days, out to 16 days http://www.emc.ncep.noaa.gov/gmb/wx20cb/pqpf_24hr/pqpf_24h.html Support general public Precipitation types, 4 times per days, out to 16 days http://www.emc.ncep.noaa.gov/gmb/yzhu/html/pqpf_6h.html Downscaling CONUS expansion Process to downscale Tmax & Tmin for CONUS Process to downscale wind speed & direction for CONUS Process to downscale dew point temperature Future plan and questions for discussion 26

Tmax and Tmin calculations for CONUS "Daytime is defined as 0700-1900 Local Standard Time and overnight is defined as 1900-0800 Local Standard Time." OUTPUT: (00UTC) Tmax: f06(no), f12(no), f18(no), f24(no), f30(yes:6-30hrs),f36(no), f42(no),f48(no), f52(yes:30-52hrs), Tmin: f06(no), f12(no), f18(yes:0-18hrs), f24(no), f30(no), f36(no), f42(yes:18-42hrs), f48(no), Initial 00UTC Tmax 00UTC 06UTC 12UTC 18UTC 00UTC 06UTC 12UTC 18UTC 00UTC 00UTC temperature forecast at valid time Tmax Tmin Tmin Tmin CONUS: Tmax period: 11/12UTC (7am-local) 23/00UTC (7pm-local) - EAST Tmin period: 23/00UTC(7pm-local) 12/13UTC (8am-local) - EAST OUTPUT: (06UTC) Tmax: f06(no), f12(no), f18(no), f24(yes:0-24hrs), f30(no), f36(no), f42(no), f48(yes:24-48hrs), f52(no), Tmin: f06(no), f12(no), f18(no), f24(no), f30(n0), f36(yes:12-36hrs), f42(no), f48(no), f52(no) Initial 06UTC Tmax Tmin Tmax Tmin 06UTC 12UTC 18UTC 00UTC 06UTC 12UTC 18UTC 00UTC 06UTC 06UTC temperature forecast at valid time 27

Proposed Dew Point Temperature & Relative Humidity Calculation Compute Dew Point Temperature (Bolton 1980): es = 6.112 * exp((17.67 * T)/(T + 243.5)); e = es * (RH/100.0); Td = log(e/6.112)*243.5/(17.67-log(e/6.112)); where: T = temperature in deg C; es = saturation vapor pressure in mb; e = vapor pressure in mb; RH = Relative Humidity in percent; Td = dew point in deg C Compute Relative Humidity (Bolton 1980): es = 6.112*exp((17.67*T)/(T + 243.5)); e = 6.112*exp((17.67*Td)/(Td + 243.5)); RH = 100.0 * (e/es); where: es = saturation vapor pressure in mb; e = vapor pressure in mb; RH = Relative Humidity in percent 28

Comparison between Operational and Proposed Methods Operational GEFS/GFS dew point temperature Only computes dew point temp with respect to water (Hui-Ya Chuang ) Operational dew point temperature calculation process Derive saturation vapor pressure from pressure & specific humidity Construct a table of dew point temp as a function of vapor pressure that range from rvp1(0.0001e0 ) and rvp2 (10.E0) centibar New operational post for GEFS dew point temperature Dew point temp is computed wrt ice when T<-20C Plan to be implemented in Sep. 2009 (Hui-Ya Chuang) Proposed strategy for dew point temperature Derive dew point temp from temperature and relative humidity following paper of Bolton (1980, MWR, p.1047) Consistent with method used in RTMA analysis and GEMPAK (Manuel Pondeca) Exchanged variables from CMC are temperature and relative humidity Two method comparison Two ways yield similar answers unless temperature very cold 29

Diff. between Operational Product and Proposed Method (Valid Time: 2009060906) New DPT warm than operational product 30

Hanson Dam flood threat special guidance http://www.emc.ncep.noaa.gov/gmb/wx20cb/pqpf_24hr/pqpf_24h.html 4 times per days (initial forecast) 24 hours PQPF for every 6-hours out to 16 days Covers most of CONUS, and east pacific Special thresholds for 1, 2, 4 and 6 inches 31

Map of PQPF and Precipitation Types: every 6 hours, 4 different thresholds 32

Alaska Down-Scaling Statistical Down-Scaling Standard variables Surface pressure, 2-m temperature, 10-meter wind component Work well using current operational technique for CONUS Derived variables Daily maximum and minimum temperature (Tmax, Tmin) Choose proper period definition Wind speed and wind direction There are many ways to interpolate winds Solution to avoid interpolation of wind direction Not bad, but difficult for wind direction improvement Alaska Down-Scaling evaluations Verification for all 8 variables for Alaska domain Preliminary experiments since early 2009 Real time parallel in early 2010 Implemented by December 7 2010 33

Probabilities (percent) 2-m temp 10/90 probability forecast verification Northern Hem, period of Dec. 2007 Feb. 2008 P10-expect P10-ncepraw P10-naefs P90-expect P90-ncepraw P90-naefs 100 90 90% 80 70 60 50 40 3-month verifications 30 20 10 0 10% 24 48 72 96 120 144 168 192 216 240 264 288 312 336 360 384 Lead time (hours) Top: 2-m temperature probabilistic forecast (10% and 90%) verification red: perfect, blue: raw, green: NAEFS Left: example of probabilistic forecasts (meteogram) for Washington DC, every 6-hr out to 16 days from 2008042300 34

Short-term Plan and Questions for Discussion Plan Working on variables Tmax/Tmin, wind speed and direction for CONUS Working on DPT and RH2m? Generating new climatological mean and standard deviation for about 20 variables Half-degree for global and 3 hourly Bo Yang Generating climatological mean and standard deviation for precipitation which based on CCPA Half-degree for CONUS and 3 hourly (?) Yan Luo Questions for discussion Products resolution: 5km, or 2.5km, or both? 5x5 km RTMA for CONUS and 6x6 km RTMA for Alaska 9 variables: surface pressure, T2m, 10m U & V, 10m wind speed & direction, dew point temp (DPT), specific humidity ( SPFH), surface height 2.5x2.5 km RTMA for CONUS only 9 variables same as the above Tmax and Tmin problem on NAEFS 1*1 degree, Tmax smaller than Tmin Bias corrected Tmax/Tmin of NAEFS probabilistic products are from model 6hr output Different process for NCEP and CMC ens. due to different available analysis Same bias for CMC ens. and different bias for NCEP Possible happen (not sure) if Tmax and Tmin are very close to each other, Bias of Tmax and Tmin are in opposite direction Downscaled Tmax/Tmin, no problem anymore due to different processing Other variables - PWAT (Precipitable Water), POP (Probability of Precipitation) 35