Active Learning with Support Vector Machines for Tornado Prediction

Transcription

1 Actve Learnng wth Support Vector Machnes for Tornado Predcton Theodore B. Trafas, Indra Adranto, and Mchae B. Rchman Schoo of Industra Engneerng, Unversty of Okahoma, 0 West Boyd St, Room 4, Norman, OK 7309, USA ttrafas@ou.edu, adranto@ou.edu Schoo of Meteoroogy, Unversty of Okahoma, 0 Davd L. Boren Bvd, Sute 5900, Norman, OK 7307, USA mrchman@ou.edu Abstract. In ths paper, actve earnng wth support vector machnes (SVMs) s apped to the probem of tornado predcton. Ths method s used to predct whch storm-scae crcuatons yed tornadoes based on the radar derved Mesocycone Detecton Agorthm (MDA) and near-storm envronment (NSE) attrbutes. The man goa of actve earnng s to choose the nstances or data ponts that are mportant or have nfuence to our mode to be abeed and ncuded n the tranng set. We compare ths method to passve earnng wth SVMs where the next nstances to be ncuded to the tranng set are randomy seected. The premnary resuts show that actve earnng can acheve hgh performance and sgnfcanty reduce the sze of tranng set. Keywords: Actve earnng, support vector machnes, tornado predcton, machne earnng, weather forecastng. Introducton Most conventona earnng methods use statc data n the tranng set to construct a mode or cassfer. The abty of earnng methods to update the mode dynamcay, usng new ncomng data, s mportant. One method that has ths abty s actve earnng. The objectve of actve earnng for cassfcaton s to choose the nstances or data ponts to be abeed and ncuded n the tranng set. In many machne earnng tasks, coectng data and/or abeng data to create a tranng set s costy and tmeconsumng. Rather than seectng and abeng data randomy, t s better f we can abe the data that are mportant or have nfuence to our mode or cassfer. In tornado predcton, abeng data s consdered costy and tme consumng snce we need to verfy whch storm-scae crcuatons produce tornadoes n the ground. The tornado events can be verfed from facts n the ground ncudng photographs, vdeos, damage surveys, and eyewtness reports. Based on tornado verfcaton, we then determne and abe whch crcuatons produce tornadoes or not. Therefore, appyng actve earnng for tornado predcton to mnmze the need for the nstances and use the most nformatve nstances n the tranng set n order to update the cassfer woud be benefca. Y. Sh et a. (Eds.): ICCS 007, Part I, LNCS 4487, pp , 007. Sprnger-Verag Bern Hedeberg 007

2 Actve Learnng wth Support Vector Machnes for Tornado Predcton 3 In the terature, the Mesocycone Detecton Agorthm (MDA) attrbutes [] derved from Dopper radar veocty data have been used to detect tornado crcuatons. Marzban and Stumpf [] apped artfca neura networks (ANNs) to cassfy MDA detectons as tornadc or non-tornadc crcuatons. Addtonay, Lakshmanan et a. [] used ANNs and added the near-storm envronment (NSE) data nto the orgna MDA data set and determned that the sk mproved margnay. Appcaton of support vector machnes (SVMs) usng the same data set used by Marzban and Stumpf [] has been nvestgated by Trafas et a. [3]. Trafas et a. [3] compared SVMs wth other cassfcaton methods, such as ANNs and rada bass functon networks, concudng that SVMs provded better performance n tornado detecton. Moreover, a study by Adranto et a. [4] reveaed that the addton of NSE data nto the MDA data can mprove performance of the cassfers sgnfcanty. However, those experments n the terature were conducted usng statc data. In ths paper, we nvestgated the appcaton of actve earnng wth SVMs for tornado predcton usng the MDA and NSE data. We aso compared ths method to passve earnng wth SVMs usng these data where the next nstances to be added to the tranng set are randomy seected. Data and Anayss The orgna data set was comprsed of 3 attrbutes taken from the MDA agorthm []. These attrbutes measure radar-derved veocty parameters that descrbe varous aspects of the mesocycone. Subsequenty, 59 attrbutes from the NSE data [] were ncorporated to ths data set. The NSE data descrbed the pre-storm envronment of the atmosphere on a broader scae than the MDA data, as the MDA attrbutes are radar-based. Informaton on wnd speed, drecton, wnd shear, humdty apse rate and the predsposton of the atmosphere to acceerate ar rapdy upward over specfc heghts were measured by the NSE data. Therefore, the MDA+NSE data consst of 8 attrbutes. 3 Methodoogy 3. Support Vector Machnes The SVM agorthm was deveoped by Vapnk and has proferated nto a powerfu method n machne earnng [5-7]. Ths agorthm has been used n rea-word appcatons and s we known for ts superor practca resuts. In bnary cassfcaton probems, the SVM agorthm constructs a hyperpane that separates a set of tranng vectors nto two casses (Fg. ). The objectve of SVMs (the prma probem) s to maxmze the margn of separaton and to mnmze the mscassfcaton error. The SVM formuaton can be wrtten as foows [8]: mn w subject to y + C ξ = ( w x + b) ξ, ξ 0, =,..., ()

3 3 T.B. Trafas, I. Adranto, and M.B. Rchman where w s the weght vector perpendcuar to the separatng hyperpane, b s the bas of the separatng hyperpane, ξ s a sack varabe, and C s a user-specfed parameter whch represents a trade off between generazaton and mscassfcaton. Usng Lagrange mutpers α, the SVM dua formuaton becomes [8]: max Q( α) = subject to = = α α y = 0, = j= α α y y x x 0 α C, j j =,..., j () The optma souton of Eq. () s gven by w = α yx where α = ( α,..., α ) s the = optma souton of the optmzaton probem n Eq. (). The decson functon s defned as: g(x) = sgn ( f ( x)), where f (x) = w x + b (3) Support vectors Outsde the margn of separaton Insde the margn of separaton ξ Outsde the margn of separaton Mscassfcaton pont Support vectors Cass - Cass w x + b = w x + b = w x + b = 0 Separatng hyperpane Fg.. Iustraton of support vector machnes For sovng nonnear probems, the SVM agorthm maps the nput vector x nto a hgher-dmensona feature space through some nonnear mappng Φ and constructs an optma separatng hyperpane [7]. Suppose we map the vector x nto a vector n the feature space (Φ (x),,φ n (x), ), then an nner product n feature space has an equvaent representaton defned through a kerne functon K as K(x,x ) = <Φ(x ).Φ(x )> [8]. Therefore, we can ntroduce the nner-product kerne as K(x,x j ) = <Φ(x ).Φ(x j )> and substtute the dot-product <x. x j > n the dua probem n Eq. () wth ths kerne functon. The kerne functon used n ths study s the rada bass functon (RBF) wth K(x,x j ) = exp γ x j x where γ s the parameter that contros the wdth of RBF.

4 Actve Learnng wth Support Vector Machnes for Tornado Predcton 33 New unabeed data, U Batch of unabeed data Are nstances nsde the margn of separaton No Remove nstances Update Cassfer Query functon, f(l) Yes Request correct abes Update Labeed data, L 3. Actve Learnng wth SVMs Fg.. Actve earnng wth SVMs scheme Severa actve earnng agorthms wth SVMs have been proposed by Campbe et a. [9], Schohn and Cohn [0], and Tong and Koer []. Campbe et a. [9] suggested that the generazaton performance of a earnng machne can be mproved sgnfcanty wth actve earnng. Usng SVMs, the basc dea of the actve earnng agorthms s to choose the unabeed nstance for the next query cosest to the separatng hyperpane n the feature space whch s the nstance wth the smaest margn [9-]. In ths paper, we choose the nstances that are nsde the margn of separaton to be abeed and ncuded n the tranng set. Snce the separatng hyperpane es n the mdde of the margn of separaton, these nstances w have an effect on the souton. Thus, the nstances outsde the margn of separaton w be removed. Suppose we are gven an unabeed poo U and a set of abeed data L. The frst step s to fnd a query functon f(l) where, gven a set of abeed data L, determne whch nstances n U to query next. Ths dea s caed the poo-based actve earnng. Scheme of actve earnng can be found n Fg Measurng the Quaty of the Forecasts for Tornado Predcton In order to measure the performance of a tornado predcton cassfer, t s mportant to compute scaar forecast evauaton scores such as the Crtca Success Index (CSI), Probabty of Detecton (POD), Fase Aarm Rato (FAR), Bas, and Hedke Sk Score (HSS), based on a confuson matrx or contngency tabe (Tabe I). Those sk scores are defned as: CSI = a/(a+b+c), POD = a/(a+c), FAR = b/(a+b), Bas = (a+b)/(a+c), and HSS = (ad-bc)/[(a+c)(c+d)+(a+b)(b+d)]. It s mportant not to rey soey on a forecast evauaton statstc ncorporatng ce d from the confuson matrx, as tornadoes are rare events wth many correct nus. Ths s mportant as there s tte usefuness n forecastng no tornadoes every day. Indeed, the cam of sk assocated wth such forecasts ncudng correct nus for rare events has a notorous hstory n meteoroogy [3].The CSI measures the

5 34 T.B. Trafas, I. Adranto, and M.B. Rchman accuracy of a souton equa to the tota number of correct event forecasts (hts) dvded by the tota number of tornado forecasts pus the number of msses (hts + fase aarms + msses) []. It has a range of 0 to, where s a perfect vaue. The POD cacuates the fracton of observed events that are correcty forecast. It has a perfect score of and a range s 0 to [4]. The FAR measures the rato of fase aarms to the number of yes forecasts. It has a perfect score of 0 wth ts range of 0 to [4]. The Bas computes the tota number of event forecasts (hts + fase aarms) dvded by the tota number of observed events. It shows whether the forecast system s underforecast (Bas < ) or overforecast (Bas > ) events wth a range of 0 to + and perfect score of [4]. The HSS [5] s commony used n forecastng snce t consders a eements n the confuson matrx. It measures the reatve ncrease n forecast accuracy over some reference forecast. In the present formuaton, the reference forecast s a random guess. A sk vaue > 0 s more accurate than the reference. It has a perfect score of and a range of - to. Tabe. Confuson matrx Observaton Yes No Yes ht fase aarm Forecast a b No mss correct nu c d 4 Experments The data were dvded nto two sets: tranng and testng. In the tranng set, we had 38 tornadc nstances and 8 non-tornadc nstances. In order to perform onne settng experments, the tranng nstances were arranged n tme order. The testng set conssted of 387 tornadc nstances and 87 non-tornadc nstances. For both actve and passve earnng experments, the nta tranng set was the frst 0 nstances conssted of 5 tornadc nstances and 5 non-tornadc nstances. At each teraton, new data were njected n a batch of severa nstances. Two dfferent batch szes, 75 and 50 nstances, were used for comparson. In passve earnng wth SVMs, a ncomng data were abeed and ncuded n the tranng set. Conversey, actve earnng wth SVMs ony chooses the nstances from each batch whch are most nformatve for the cassfer. Therefore, the cassfer was updated dynamcay at each teraton. The performance of the cassfer can be measured by computng the scaar sk scores (Secton 3.3) on the testng set. The rada bass functon kerne wth γ = 0.0 and C = 0 was used n these experments. The experments were performed n the Matab envronment usng LIBSVM toobox [6]. Before tranng a cassfer, the data set needs to be normazed. We normazed the tranng set so that each attrbute has the mean of 0 and the standard devaton of. Then, we used the mean and standard devaton from each attrbute n the tranng set to normaze each attrbute n the testng set.

6 Actve Learnng wth Support Vector Machnes for Tornado Predcton 35 Fg. 3. (a) The resuts of CSI, POD, FAR, Bas, and HSS on the testng set usng actve and passve earnng at a teratons. (b) The ast teraton resuts wth 95% confdence ntervas on the testng set.

7 36 T.B. Trafas, I. Adranto, and M.B. Rchman 5 Resuts It can bee seen from Fg. 3a for a sk scores, CSI, POD, FAR, Bas, and HSS, actve earnng acheved reatvey the same scores as passve earnng usng ess tranng nstances. From the FAR dagram (Fg. 3a), we notced that at eary teraton the actve and passve earnng FAR wth the batch sze of 75 dropped suddeny. It happened because the forecast system was underforecast (Bas < ) at that stage. Utmatey, every method produced overforecastng. Furthermore, Fg. 3b showed the ast teraton resuts wth 95% confdence ntervas after conductng bootstrap resampng wth 000 repcatons [7]. The 95% confdence ntervas between actve and passve earnng resuts wth the batch szes of 75 and 50 overapped each other for each sk score, so the dfferences were not statstcay sgnfcant. These resuts ndcated that actve earnng possessed smar performance compared to passve earnng usng the MDA and NSE data set. The resuts n Fg. 4 showed that actve earnng sgnfcanty reduced the tranng set sze to attan reatvey the same sk scores as passve earnng. Usng the batch sze of 75 nstances, ony 57 abeed nstances were requred n actve earnng whereas n passve earnng 50 abeed nstances were needed (Fg. 4a). Ths experment reveas that about 6.6% reducton was reazed by actve earnng. Usng the batch sze of 50 nstances, actve earnng can reduce the tranng set sze by 60.5% snce t ony needed 596 abeed nstances whereas passve earnng requred 50 abeed nstances (Fg. 4b). Fg. 4. Dagrams of tranng set sze vs. teraton for the batch szes of (a) 75 and (b) 50 nstances 6 Concusons In ths paper, actve earnng wth SVMs was used to dscrmnate between mesocycones that do not become tornadc from those that do form tornadoes. The premnary resuts showed that actve earnng can sgnfcanty reduce the sze of tranng set and acheve reatvey smar sk scores compared to passve earnng. Snce abeng new data s consdered costy and tme consumng n tornado predcton, actve earnng woud be benefca n order to update the cassfer dynamcay.

8 Actve Learnng wth Support Vector Machnes for Tornado Predcton 37 Acknowedgments. Fundng for ths research was provded under the Natona Scence Foundaton Grant EIA and NOAA Grant NA7RJ7. References. Marzban, C., Stumpf, G.: A neura network for tornado predcton based on Dopper radarderved attrbutes. J. App. Meteoro. 35 (996) Lakshmanan, V., Stumpf, G., Wtt, A.: A neura network for detectng and dagnosng tornadc crcuatons usng the mesocycone detecton and near storm envronment agorthms. In: st Internatona Conference on Informaton Processng Systems, San Dego, CA, Amer. Meteor. Soc. (005) CD ROM J5. 3. Trafas, T.B., Ince, H., Rchman M.B. Tornado detecton wth support vector machnes. In: Soot PM et a. (eds). Computatona Scence-ICCS (003) 0-4. Adranto, I., Trafas, T.B., Rchman, M.B., Lakshmvarahan, S., Park, J.: Machne earnng cassfers for tornado detecton: senstvty anayss on tornado data sets. In: Dag C. Buczak, A., Enke, D., Embrechts, M., Ersoy, O. (eds.): Integent Engneerng Systems Through Artfca Neura Networks, Vo. 6. ASME Press (006) Boser, B.E., Guyon, I.M., Vapnk, V.N.: A tranng agorthm for optma margn cassfers. In: Hausser D (ed): 5th Annua ACM Workshop on COLT. ACM Press, Pttsburgh, PA (99) Vapnk, V.N.: The Nature of Statstca Learnng Theory. Sprnger Verag, New York (995) 7. Vapnk, V.N.: Statstca Learnng Theory. Sprnger Verag, New York (998) 8. Haykn S.: Neura Networks: A Comprehensve Foundaton. nd edn. Prentce Ha, New Jersey (999) 9. Campbe, C., Crstann, N., Smoa, A.: Query earnng wth arge margn cassfers. In: Proceedngs of ICML-000, 7th Internatona Conference on Machne Learnng. (000)-8 0. Schohn, G., Cohn, D.: Less s more: Actve earnng wth support vector machnes. In: ICML Proceedngs of ICML-000, 7th Internatona Conference on Machne Learnng, (000) Tong, S., Koer, D.: Support vector machne actve earnng wth appcatons to text cassfcaton. J. Mach. Learn. Res. (00) Donadson, R., Dyer, R., Krauss, M.: An objectve evauator of technques for predctng severe weather events. In: 9th Conference on Severe Loca Storms, Norman, OK, Amer. Meteor. Soc. (975) Murphy, A.H.: The Fney affar: a sgna event n the hstory of forecast verfcatons. Weather Forecast. (996) Wks, D.: Statstca Methods n Atmospherc Scences. Academc Press, San Dego, CA (995) 5. Hedke P.: Berechnung des erfoges und der gute der wndstarkvorhersagen m sturmwarnungsdenst, Geogr. Ann. 8 (96) Chang, C., Ln, C.: LIBSVM: a brary for support vector machnes. Software avaabe at < (00) 7. Efron, B., Tbshran, R.J.: An Introducton to the Bootstrap. Chapman & Ha, New York (993)