Runoff Prediction with a Combined Artificial Neural Network and Support Vector Regression

Transcription

1 Iteratioal Joural of Machie Learig ad Computig, Vol. 8, No. 1, February 2018 Ruoff Predictio with a Combied Artificial Neural Network ad Support Vector Regressio Ratipor Chakla, Nutawut Kaougku, Keerachart Suksut, Kittisak Kerdprasop, ad Nittaya Kerdprasop Abstract Water is a importat part of our daily lives: food, maufacture, agriculture, etc. Whe water is ot eough for all populatio, it leads to may udesirable impacts icludig drought, famie ad death. The solutio to this problem is the good maagemet of water resources. The maagemet of water resources is plaig ad desigig of projects related to water. The ruoff predictio is oe major part of plaig. It is a complex process ad it also eeds a adequate modelig techique for accurate predictio. Therefore, we propose to use combied algorithms to improve predictio performace. Our combiatio icludes the two powerful methods: Artificial Neural Network (ANN) ad Support Vector Regressio (SVR). The root mea square error (RMSE) ad the correlatio coefficiet (R) are two criteria that we use to evaluate the model performace regardig the compariso betwee actual ruoff ad the predictio made by our model. We also compare performace of our model agaist the other algorithms: Liear Regressio, ANN, ad Support Vector Machies. The compariso results show that our proposed method shows the best performace ad the combied model is also quite accurate o predictig the peak ruoff values durig heavy rai seaso. Idex Terms Ruoff predictio, artificial eural etwork, support vector regressio, Mu Basi. I. INTRODUCTION Curretly, people are experiecig both direct ad idirect impacts from droughts such that it results i water usage restrictios, more frequet forest fires, reduced crop yields, loss of livestock, ad may others [1]. O the cotrary, people also experiece floods i larger areas year by year. The efficiet maagemet of water resources is oe way to protect the two water problems: flood ad droughts. A kowledge to make accurate ruoff predictio ca obviously help plaers ad water maagemet policy makers to kow i advace the water volume as either eough or ot eough for the demad of people use ad also ca improve efficiecy o flood ad drought cotrol. Artificial eural etwork (ANN) shows good performace o predictig outcomes from may complex processes ad Mauscript received September 19, 2017; revised Jauary 20, This work was supported by grat from Suraaree Uiversity of Techology through the fudig of Kowledge ad Data Egieerig Research Uits. The authors are with the School of Computer Egieerig, Suraaree Uiversity of Techology (SUT), 111 Uiversity Aveue, Muag, Nakho Ratchasima 30000, Thailad. (correspodig author: R. Chakla; Tel: ; arc_agle@hotmail.com, utawut@sut.ac.th, mikaiterg@gmail.com, kerdpras@sut.ac.th, ittaya@sut.ac.th). recogizig patters [2]. It has also bee used to create computatioal model to predict several kids of outcomes i the hydrology research, such as stream flow forecastig, raifall-ruoff modelig, water quality ad maagemet modelig [3], [4]. Besides ANN, Support vector regressio (SVR) is aother accurate techique that has bee recetly applied to predict ruoff ad [5], [6]. SVR has bee kow to show high performace o learig solutio i complex problems [7]. I this work, we propose a ew method to predict mothly ruoff. I our method, the better performace ca be achieved through the combiatio of ANN ad SVR. The results tur out that our model ca predict both low ad high ruoff values. A literature ormally uses statistical values such as mea absolute percetage error (MAPE), coefficiet of determiatio (R 2 ), correlatio coefficiet (R), ad root mea squared error (RMSE) to compare performace of a predictio algorithm. I our work, we adopt two metrics: R ad RMSE. II. BACKGROUND THEORIES A. Artificial Neural Network ANN has bee developed base o operatios of the huma brai. It is the most widely used tool i hydrology. The etwork has three layers: iput layer, hidde layer ad output layer. The iput odes are ame ode i iput layer. The umber of attribute i data is equal to the umber of iput odes. The hidde odes are ame ode i hidde layer, the umber of odes is defied by a user, ad this layer ca be more tha oe layer. The output odes are ame ode i output layer, the umber of odes is equal to the umber of target o data. The etwork are coected betwee the odes with lie, ad each lie has weight. ANN learig is to fid proper weight o each lie i the etwork. The proper weight is the oe that ca best separatig traiig data ito the corrected target groups. There exist several architectures of ANN. I this work, we use feed-forward eural etworks. The trasfer fuctio i the hidde layer is a liear fuctio. B. Support Vector Regressio SVR works by trasformig the iput data ito a high-dimesioal feature space by liear or oliear mappig. SVR is the most popular applicatio form of Support Vector Machie (SVM). The ituitive idea of SVR o predictig future data is illustrated i Fig. 1. The goal of SVR is to fid a fuctio ƒ(x) that has at most ε deviatio from the actual target value y i for all the traiig data [8]. This liear fuctio f is show i equatio 1. A traiig data is a set of iput-target pairs, {(x 1, y 1 ),, (x i, y i )} doi: /ijmlc

2 Iteratioal Joural of Machie Learig ad Computig, Vol. 8, No. 1, February 2018 X R. f(x) = w, x + b (1) Whe a fuctio ƒ(x) is hyperplae, the size of ε is margi, the symbol, is the dot product i X, b ϵ R ad w ϵ X. The hyperplae has small margi i which the SVR has to fid it. This small margi tries to keep all the data lyig iside as much as possible. The margi ca be calculated as i equatio 2. miimize 1 2 w 2 subject to y i w, x i + b ε w, x i + b y i ε Through equatio 2, it is implicitly assumed that this is a fuctio ƒ that approximates all the pairs (x i, y i ) with precisio. If it is ot possible to keep all data iside the margi, the slack variables ξ i, ξ i ca be itroduced to solve the problem. This ca be stated accordig to the equatio 3. miimize 1 2 w 2 N + C ξ i + ξ i i=1 (2) RMSE = (T i O i ) 2 where N is the umber of all data, O i is the predicted value by the model, ad T i is the actual value, i = 1,2,, N. III. MATERIALS AND METHODS Our study area is Mu Basi (Fig. 2), the largest basi i the North-easter regio of Thailad.To build a predictive model, we use temperature data from the Natioal Statistical Office ( mothly raifall, ruoff ad the umber of raiy days data from the Meteorological Departmet ( ad Normalized Differece Vegetatio Idex (NDVI), from the NOAA STAR ( This research use RStudio as the aalysis tools i our experimets. The ruoff data are from two sources: ad statio. N (5) subject to y i w, x i + b ε + ξ i w, x i + b y i ε + ξ i (3) The costat C > 0 directs the choice betwee the flatess of ƒ ad the amout up to which deviatios larger tha ε are tolerated. Fig. 2. The study area: Mu Basi, Thailad. Fig. 1. Example of liear support vector regressio. C. Correlatio Coefficiet The correlatio coefficiet is deoted by R ad its umerical value is betwee -1 ad 1. The plus sig meas the correlatio of the two variables (x ad y) moves i the same directio, whereas the mius sig ifers opposite directio. The magitude expresses the stregth of the relatioship; the higher is the stroger regardless of the sig. No relatioship is the magitude that closes to 0. The correlatio coefficiet ca be computed usig equatio 4. R = i=1 x i y i i=1 x i i=1 y i x 2 i=1 i x 2 i=1 i y 2 i=1 i y i i=1 2 (4) where x i is the value of variable x or predicted value (whe i = 1, 2,, ), y i is the value of variable y or actual value, ad is the total umber of samples. D. Root Mea Squared Error The Root Mea Squared Error is deoted by RMSE ad used to measure performace of the model. The RMSE ca be calculated as i equatio 5. statio locates at Ba Wag Takhia, Amphur Pak Chog, i Nakho Ratchasima Provice, Thailad. We use the 18-year data durig 1998 to We spilt the 13-year data ( ) to be traiig data ad the remaiig 5-year data ( ) are for testig the model performace. M.173 statio locates at Ba No Sa-at, Amphur Chokchai, i Nakho Ratchasima Provice, Thailad. The traiig data is a te-year period ( ) ad the test data is the four-year period ( ). The, we use raifall, ruoff, the umber of raiy days, temperature ad NDVI with the laggig time 1-moth ad 2-moth. These data are iput ito two learig algorithms: ANN ad SVR. The, the average raifall is lagged 1-moth (average raifall t-1 ) to select the best subset of data from the iitial traiig set for appropriate algorithm. The average raifall ca be calculated as i equatio 6. average raifall t 1 = N i=1 a Where a is the average mothly raifall i lagged 1-moth over the traiig years, N is the umber of traiig data (whe i=1, 2,.., N), ad is the umber of years from traiig data. Our combied predictio model has a flow as show i Fig. 3. The selectio of either ANN or SVR model is based o the amout of rai. If the raifall i lagged 1-moth (raifall t-1 ) has value less tha average raifall t-1, the apply the SVR (6) 40

3 Iteratioal Joural of Machie Learig ad Computig, Vol. 8, No. 1, February 2018 model because based o our observatio this kid of model is good at predictio o ormal or drought situatio. But if the raifall i lagged 1-moth (raifall t-1 ), has a value higher tha the average raifall t-1, the apply the ANN model that is good i predictig the ear-floodig situatio. respectively. The result from our proposed combied model is preseted i Table III. TABLE II: RUNOFF PREDICTION PERFORMANCE FROM SVR MODEL Statio Kerel R RMSE liear sigmoid polyomial radial basis fuctio liear sigmoid polyomial radial basis fuctio TABLE III: RUNOFF PREDICTION PERFORMANCE FROM PROPOSED MODEL Fig. 3. The modelig process for ruoff predictio. The fial step is the performace evaluatio usig the two statistical values: Correlatio Coefficiet (R) ad Root Mea Squared Error (RMSE). The R ad RMSE are computed from experimetatio with the separate test data. IV. EXPERIMENTAL RESULTS TABLE I: RUNOFF PREDICTION PERFORMANCE FROM ANN MODEL Statio ANN topology R RMSE The trai ad test data have te iput variables (raifall, ruoff, the umber of raiy day, temperature ad NDVI with lagged 1-moth ad 2-moth). We use the iitial traiig set iput ito ANN ad SVR to create a combied model ad the test this model with the test data. The results of applyig ANN ad SVR aloe are show i Tables I ad II, Statio ANN topology Kerel R RMSE liear sigmoid polyomial RBF liear sigmoid polyomial RBF Our desig of ANN architecture is based o the suggestio the optimal size of the hidde layer is usually betwee the size of the iput ad size of the output layers [9]. We thus set the hidde layer to be i the rage 1-10 as show i Table I. The results reveal that at the statio the best etwork topology is (R=0.67, RMSE=8.17). At the statio, the best topology is (R=0.66, RMSE=58.73). O buildig the SVR model, we set parameters as follows: cost=1, gamma=1/(data dimesio)), degree=3 ad coefficiets of the support vector =0 for each kerel. The best ruoff predictio at the statio is the radial basis kerel (R=0.67, RMSE=8.98). At the statio, the liear kerel (R=0.54, RMSE=61.66) performs almost as good as the radial basis fuctio (R=0.55, RMSE=62.84). Whe we combie the power of both ANN ad SVR algorithms, it shows clearly good performace. I our proposed method, we use either ANN or SVR depedig o the amout of rai i each data istace. We therefore report both the ANN architecture ad the kerel fuctio i Table III. The best results are the oe highlighted i red bold fot. We also show graphical comparisos of ANN ad SVR methods for the statio (Fig. 4) ad the statio (Fig. 5). The actual versus predicted ruoff values usig our proposed combiatio ANN-SVR method i both statios is preseted i Fig. 6. Figs. 4 ad 5 clearly reveal stregth of the ANN ad SVR models o predictig ruoff values. It ca be oticed that the ANN performs well o some peak values, but perform poorly o some low ruoff values at the ad. O the cotrary, the SVR is good at predictig ruoff amout durig the water shortage situatio. But whe ruoff is excessive, the SVR model performs quite poor at both the ad statios. It is actually based o these experimetal observatio that we thus desig the proposed model combiig the stregth from each model usig raiig amout as a decisio criterio. 41

4 Iteratioal Joural of Machie Learig ad Computig, Vol. 8, No. 1, February 2018 Fig. 4. The predicted ad actual ruoff values made by ANN ad SVR models at the statio. Fig.4. The predicted ad actual ruoff values made by ANN ad SVR models at the statio. Fig.6. The predicted ad actual ruoff values at the ad statios made by our proposed method. Our proposed method use average mothly raifall to select a subset of data for each algorithm with the ituitive idea that raifall causes ruoff. It turs out that our model performs the best i both shortage ad excessive raifall. The proposed method ca also fid a good architecture for hidde ode layer of the ANN. I additio, we also create model base o Liear Regressio (LR). The predictio results are however poor (at the statio, R=0.67, RMSE=8.17; at the statio, R=0.51, RMSE=65.46). Therefore, we ca coclude from these experimetal results that our proposed method is the best combied model to predict ruoff (at the statio, R=0.69, RMSE=8.01; at the statio, R=0.71, RMSE=51.99). V. CONCLUSION I this work, we propose a ruoff predictio method that combie the advatages of ANN ad SVR to predict ruoff (raifall t ). The stregth of ANN is its high accuracy o predictig ruoff whe the amout of raifall is high. The stregth of SVR is o the cotrary that it is good at the low 42

5 Iteratioal Joural of Machie Learig ad Computig, Vol. 8, No. 1, February 2018 amout of raifall. We thus combie these observed stregths. O the combiatio step, we use average accumulative raifall with lagged 1-moth (average raifall t-1 ) to select a subset of data from the iitial traiig set, the split the iitial traiig set for ANN ad SVR to create models. To use our combied method to predict ruoff, the decisio criteria for choosig either ANN or SVR model is the amout of raifall o the previous moth. If this amout is higher tha our threshold value, choose the ANN model; otherwise, choose the SVR model. From the experimetal results, the proposed method shows good efficiecy to predict ruoff whe it is compared agaist other techique icludig ANN, SVM ad LR. These comparisos are based o R ad RMSE metrics usig test data from the two statios i the Mu Basi of Thailad. REFERENCES [1] S. Subak, Climate chage adaptatio i the U.K. water idustry: Maagers perceptios of past variability ad future scearios, Water Resources Maagemet, vol. 14, pp , Jauary [2] T. A. Sezi ad P. A. Johso., Precipitatio-Ruoff Modelig usig Artificial Neural Networks ad coceptual models, Joural of Hydrologic Egieerig, vol. 5, o. 2, pp , April [3] A. W. Mis ad M. J. Hall, Artificial eural etworks as raifall-ruoff models, Hydrological Scieces Joural, vol. 41, o. 3, pp , Jue [4] J. Morshed ad J. J. Kaluarachchi, Applicatio of artificial eural etwork ad geeric algorithm i flow ad trasport simulatios, Advaces i Water Resouces, vol. 22, o. 2, pp , [5] S. M. V Choubey, S. Padey, & Shukla, J. A Efficiet Approach of Support Vector Machie for Ruoff Forecastig, Iteratioal Joural of Scietific & Egieerig Research, vol. 5, o. 3, pp , March [6] C. L. Wu, K. W. Chau, ad Y. S. Li., River stage predictio based o a distributed support vector regressio, Joural of hydrology, vol. 358, o. 1-2, pp , [7] A. J. Smola ad B. Schölkopf, A tutorial o support vector regressio, Statistics ad Computig, vol. 1, o. 3, pp , [8] F. Graata, R. Gargao ad Giovai de Mariis, Support vector regressio for raifall-ruoff modelig i urba draiage: A compariso with the EPA s storm water maagemet model, Water, vol.8, o. 3, p. 69, [9] Itroductio to Neural Networks with Java, Heato Research, Heato Research, Ic., St. Louis, R. Chakla is curretly a doctoral studet with the School of Computer Egieerig, Suraaree Uiversity of Techology (SUT), Thailad. She received her bachelor degree i computer egieerig from SUT i 2013, master degree i computer egieerig from SUT i Her curret research of iterest icludes data classificatio, data miig applicatio, ad artificial itelligece. N. Kaougku is curretly a lecturer at School of Computer Egieerig, SUT, Thailad. He received his doctoral degree, master dgree, ad bachelor degree i computer egieerig from SUT, i 2015, 2013, ad 2012, respectively. His curret research icludes data miig, kowledge egieerig, ad sematic web. K. Suksut is curretly a doctoral studet with the School of Computer Egieerig, SUT, Thailad. He received his bachelor degree i computer egieerig from SUT i 2011, master degree i computer egieerig from SUT i His curret research of iterest icludes data miig, geetic algorithm, ad imbalaced data classificatio. K. Kerdprasop is a associate professor ad chair of the School of Computer Egieerig, SUT. He received his bachelor degree i mathematics from Sriakariwirot Uiversity, Thailad, i 1986, master degree i computer sciece from the Price of Sogkla Uiversity, Thailad, i 1991, ad doctoral degree i computer sciece from Nova Southeaster Uiversity, U.S.A., i His curret research icludes data miig, artificial itelligece, ad computatioal statistics. N. Kerdprasop is a associate professor at the School of Computer Egieerig, SUT. She received her bachelor degree i radiatio techiques from Mahidol Uiversity, Thailad, i 1985, master degree i computer sciece from the Price of Sogkla Uiversity, Thailad, i 1991, ad doctoral degree i Computer Sciece from Nova Southeaster Uiversity, U.S.A, i Her research of iterest icludes kowledge discovery i databases, artificial itelligece, ad itelliget databases. 43