David Ruth Meteorological Development Laboratory National Weather Service, NOAA Measuring Forecast Continuity
National Digital Forecast Database (NDFD) Contains a seamless mosaic of NWS digital forecasts Is available to all users and partners public and private Allows users and partners to create wide range of text, graphic, and image products Measuring Forecast Continuity 2
NDFD Elements MaxT, MinT Weather, Wind, Wind Gusts Sky, PoP12, QPF, Snow Temp, Dew, AppT, RH Wave Height, Tropical Winds Hazards, Convection, Climate Outlooks Measuring Forecast Continuity 3
How is NDFD Produced? National Centers Model Guidance Grids Detailed Interactive Collaborative Local Digital Forecast Database Field Offices Collaborate Data and Science Focus National Centers NWS Automated Products User-Generated Products Text Digital National Digital Forecast Database Graphic Voice Measuring Forecast Continuity 4
MaxT Change Accumulation One week example of forecasts valid on January 31, 2006 Measuring Forecast Continuity 5
MinT Change Accumulation One week example of forecasts valid on January 31, 2006 Measuring Forecast Continuity 6
GMOS MaxT GMOS MaxT Accum Change GMOS MaxT Net Change Measuring Forecast Continuity 7
NDFD MaxT NDFD MaxT Accum Change NDFD MaxT Net Change Measuring Forecast Continuity 8
Forecast Convergence Score The Forecast Convergence Score (FCS) is an index that measures the number and size of swings in forecasts made over a series of forecast cycles for forecasts valid at the same time. The FCS has four terms: FCS = ( T1 + T2 ) ( T3 + T4 ) Measuring Forecast Continuity 9
Forecast Convergence Score FCS = ( T1 + T2 ) ( T3 + T4 ) The first term (T1) is the number of forecasts that changed insignificantly (less than a threshold) from the previous forecast Fi-1 OR moved closer to the next forecast Fi+1, where i varies from 2 to n. When i=n, the observation is used as the next forecast Fi+1. T 1 = n i= 2 1 if 1 if 0 F F if i i F F i 1 i+ 1 neither of < threshold < F i F above i 1 Measuring Forecast Continuity 10
Forecast Convergence Score FCS = ( T1 + T2 ) ( T3 + T4 ) The second term (T2) is the difference between the first and last forecast scaled by the significance threshold T = Fn F 2 1 threshold Measuring Forecast Continuity 11
Forecast Convergence Score FCS = ( T1 + T2 ) ( T3 + T4 ) The third term (T3) is the number of possible forecast changes. T3 = n 1 Measuring Forecast Continuity 12
Forecast Convergence Score FCS = ( T1 + T2 ) ( T3 + T4 ) The fourth term (T4) is the sum of forecast changes scaled by the significance threshold. n T4 = Fi Fi 1 i= 2 threshold Measuring Forecast Continuity 13
Forecast Convergence Score ( T1 + T2 ) FCS = ----------------- ( T3 + T4 ) # of swings + size of swings The T1 and T3 terms account for the actual and possible number of swings, respectively. The T2 and T4 terms account for the size of the swings. A significance threshold specifies the minimum change necessary to count as a swing The FCS can range from near 0 (many large swings away from the next forecast) to 1.0 (no swings). Measuring Forecast Continuity 14
Case Study - NDFD Measuring Forecast Continuity 15
Case Study - GMOS Measuring Forecast Continuity 16
Case Study from TABLE 1. Ruth et al. 2009 NDFD KSLC KORD KBOS Issuance NDFD MOS GMOS NDFD MOS GMOS NDFD MOS GMOS 0000 UTC 24 Apr F 1 79 74 76 67 72 69 66 62 62 1200 UTC 24 Apr F 2 76 72 75 67 73 72 66 64 64 0000 UTC 25 Apr F 3 76 77 77 70 71 72 70 58 62 1200 UTC 25 Apr F 4 74 77 79 74 77 75 70 64 64 0000 UTC 26 Apr F 5 80 83 81 74 78 77 65 70 70 1200 UTC 26 Apr F 6 80 79 80 78 78 78 65 64 68 0000 UTC 27 Apr F 7 81 79 81 78 82 80 67 66 67 1200 UTC 27 Apr F 8 81 81 82 84 78 78 67 62 65 0000 UTC 28 Apr F 9 82 78 81 82 74 75 64 64 65 1200 UTC 28 Apr F 10 81 78 80 80 76 75 67 67 66 0000 UTC 29 Apr F 11 78 78 78 79 72 74 67 61 64 1200 UTC 29 Apr F 12 78 80 79 76 76 75 67 61 63 0000 UTC 30 Apr F 13 79 82 81 76 72 74 68 63 64 1200 UTC 30 Apr F 14 80 81 81 72 73 73 68 65 64 Ob 82 82 82 67 67 67 71 71 71 Measuring Forecast Continuity 17
FCS Calculation - KSLC significant swing for threshold = 3 FCS= (T1 + T2) / (T3 + T4) NDFD FCS= (13 + 1/3) / (13 + 19/3) = 0.69 MOS FCS= (12 + 7/3) / (13 + 27/3) = 0.65 GMOS FCS= (13 + 5/3) / (13 + 17/3) = 0.79 Measuring Forecast Continuity 18
FCS Calculation - KORD significant swing for threshold = 3 FCS= (T1 + T2) / (T3 + T4) NDFD FCS= (13 + 5/3) / (13 + 29/3) = 0.65 MOS FCS= ( 9 + 1/3) / (13 + 37/3) = 0.37 GMOS FCS= (13 + 4/3) / (13 + 20/3) = 0.73 Measuring Forecast Continuity 19
FCS Calculation - KBOS significant swing for threshold = 3 FCS= (T1 + T2) / (T3 + T4) NDFD FCS= (13 + 2/3) / (13 + 18/3) = 0.72 MOS FCS= (10 + 3/3) / (13 + 47/3) = 0.38 GMOS FCS= (13 + 2/3) / (13 + 22/3) = 0.67 Measuring Forecast Continuity 20
FCS Significance Thresholds Measuring Forecast Continuity 21
FCS by NWS Region CR = Central Region ER = Eastern Region SR = Southern Region WR = Western Region Measuring Forecast Continuity 22
FCS by Month of Year Measuring Forecast Continuity 23
Summary Score measures continuity of forecasts from one issuance to the next Score was developed and used to improve forecast continuity of Gridded MOS NDFD, MOS, and Gridded MOS continue to be scored on a monthly basis Score is described in 2009 manuscript published in Weather and Forecasting Measuring Forecast Continuity 24