Schweizerische Eidgenossenschaft Confédération suisse Confederazione Svizzera Confederazium svizra

Transcription

1 Schweizerische Eidgenossenschaft Confédération suisse Confederazione Svizzera Confederazium svizra Federal Department of Home Affairs FDHA Federal Office of Meteorology and Climatology MeteoSwiss Swiss Confederation L. Mayoraz (1), J. Ambühl (1), R. Voisard (2), C. Voisard (1), M. Züger (2), H. Romang (1) (1) MeteoSwiss, Zurich, Switzerland, (2) University of Zurich, Switzerland

2 Goal of Project GenWarn Development of a semi automatic short term warning system for gale on Swiss lakes and regional aerodromes, sending warning proposals to forecasters, based on genetic programg. aviaswiss.xooit.com 2

3 Context / Current Situation Strong gusts ( 25 kt) = potential danger to aviation and maritime safety Gale warnings In Switzerland, gale warnings are issued for more than 50 lakes and aerodromes but not automated First wind gust frequently missed: Low hit rates! Benefit of GenWarn System: supports the forecasters in their ongoing weather surveillance and alerts them by proposing potential gale warnings 3

4 Method Development Phase (1X, with historical data) COSMO 2 Evolutionary Algorithm 20 Java Methods Verification Optimal Probability Threshold q* Predictor list (from a 2 year data set): from several observation stations from the COSMO 2 model Wind Gust 25 kt in the next 3 hours? t 0 1h t 0 t 0+0.5h t 0+1h t 0+2h t 0+3h Current time 4

5 Method Development Phase (1X, with historical data) COSMO 2 Evolutionary Algorithm 20 Java Methods Verification Optimal Probability Threshold q* Genetic Programg Machine learning technique inspired by the evolution theory of species used for optimization problems. 1) Creation of a random population of computer programs from the predictor list. (= gen. 0) 2) Evaluation of the programs. 20 X Fitness function = Hit Rate * (1 False Alarm Ratio) * 100 3) Selection of the best programs and application of crossing and mutation processes on the selection (= gen. 1) 4) Repetition of steps 2 to 3 until the maximum number of generations is reached. 5

6 Method Development Phase (1X, with historical data) COSMO 2 Evolutionary Algorithm 20 Java Methods Verification Optimal Probability Threshold q* Output of evolutionary algorithm 20 java methods forecasting the maximum wind gust + in the next hours Tree Representation Example of Java Method: max dmo plus.evaluate (max.evaluate (mmo, fxxs), sine.evaluate (us.evaluate mmo (dmo, max.evaluate (us.evaluate (pow.evaluate (.evaluate (qfdif, sine.evaluate (1.71)), sine.evaluate (multiply.evaluate (4.16, mmo))), qfdif log pow.evaluate (.evaluate (6.27, divide.evaluate (0.52, wshe)), * sin / log.evaluate (plus.evaluate (pow.evaluate (.evaluate (6.27, sin mmo wshe fxxs us.evaluate (dmo, max.evaluate (fxxs, 6.12))), log.evaluate 4.16 log (plus.evaluate(f00, ttt))), fxxs)))), mmo)))) sin fxxs max mmo ttt f00 dmo 6.12 max fxx 6

7 Method Development Phase (1X, with historical data) COSMO 2 Evolutionary Algorithm 20 Java Methods Verification Optimal Probability Threshold q* Output of evolutionary algorithm 20 java methods forecasting the maximum wind gust in the next hours + mmo max fxxs fxxs * ttt / mi n ma x mmo fxxs ma x / 6.5 cape dmo sin max Herd of java methods ensemble forecast * mi n qfdif * sin / qfdif mi n sin / 6.27 wshe mmo + sin 4.16 mmo sin mmo log wshe / qfdif sin * sin fxxs max mmo 4.16 fxxs wshe mmo sin max dmo log mmo 4.16 max dmo f00 ttt qfdif log 7

8 Method Development Phase (1X, with historical data) COSMO 2 Verification: ROC Curve Evolutionary Algorithm 20 Java Methods Verification Optimal Probability Threshold q* Event based On a 2 year independent data set Hit Rate Probability of occurrence in % False Alarm Ratio 8

9 Method Development Phase (1X, with historical data) COSMO 2 Verification: ROC Curve Evolutionary Algorithm q*: optimal probability threshold 20 Java Methods Verification Optimal Probability Threshold q* Event based On a 2 year independent data set Hit Rate Fitness Function = HR*(1 FAR) q*: probability of occurrence above which an alarm proposal is sent Probability of occurrence in % False Alarm Ratio 9

10 us.evaluate( ,.evaluate(divide.evaluate( , divide.evaluate(max.evaluate(pow.evaluat , in.evaluate(divide.evaluate( ,) divide.evaluate(max.evaluate(pow.evaluate( , fmo), tt40),.evaluate(divide.evaluate( , ddd), f20))) Method For each warning object: Development Phase (1X, with historical data) COSMO 2 Evolutionary Algorithm 20 Java Methods Verification Optimal Probability Threshold q* Operational Routine COSMO 2 every 10 Class Method 1 Method 2 Method 3 Method 4 Probability P that wind gust 25 kt If P q* Alarm Proposal 10

11 Results Variation of Meteorological Threshold Q Verification on the 2 year data set Threshold Q = 25 knots Threshold Q = 25 knots Threshold Q = 12 knots Maximum Fitness: If Q = 25 kt : ~ 30 If Q = 12 kt : ~ 45 A clear performance limit is reached at this point. (HR ~95%, FAR ~70%) Performance of system is higher for Q = 12 kt Storm events stronger than 25 kt are too rare for the system to detect them correctly (tendency of detecting too many events) 11

12 Results Comparison with Forecasters Performance Typical ROC Curve GenWarn Vs. Forecasters Performance Forecaster Experience GenWarn System Typical Performance Forecasters Performance per Warning Object Overall increase in HR induced by GenWarn System Contribution of GenWarn System variable, object dependent Role of forecaster: decrease the FAR 12

13 Conclusions System performance so far: Hit Rate ~95%, FAR ~70% General increase of hit rate when using the GenWarn System compared to the actual forecasters performance best solution: mix machine & forecaster to lower the FAR Outlook: operationalization, in situ tests 13

14 Thank You! Questions? Sebastien Marti/Scoopmobile 14

15 Method Development Phase (with historical data) COSMO 2 Evolutionary Algorithm 20 Java Methods Verification Optimal Probability Threshold q* Genetic Operations Depth syntax tree representing max(x+x, x+3*y) Example of sub tree crossover (Poli, Landon, & McPhee, 2008) 15

16 Results Variation of Storm Proportion in the Learning Data Results for the learn data Verification on the 2 year data set 50% Storm Prop. 25% Storm Prop. 12.5% Storm Prop. No Overrep. Hit Rate [%] 50% Storm Prop. 25% Storm Prop. 12.5% Storm Prop. No Overrep. Maximum Fitness 50% Storm 25% Storm 12.5% Storm No Overrep ~ 80 ~ 65 ~ 50 ~ 25 The system shows a better detection of strong wind gusts with a higher storm proportion in the learn data (system learns more easily). Maximum Fitness 50% Storm 25% Storm 12.5% Storm No Overrep ~ 25 ~ 30 ~ 30 ~ 25 But the verification on the 2 year data set shows a clear limitation of the performance quite similar for all four methods. (Best fitness of only 3000) 16

17 Results Comparison with Forecasters Performance Fitness Distribution of Forecasters Performance Winter Summer GenWarn System TypicalPerformance Forecasters Performance Improvement of performance higher in winter than in summer Forecasters can follow thunderstorm cells with radar data: clear advantage compared to GenWarn. 17

18 Genetic Programg (type of evolutionary algorithm, inspired by the theory of the evolution of species) Fitness Function Events: v max 25 kt Observed Events Yes No Forecasted Events Yes No A C B D Hit Rate= HR = False Alarm Ratio = FAR = False Alarm Rate = F = Fitness Function = GenWarn System / 18