Neural network models for river flow forecasting

Transcription

1 Neural network models for river flow forecasting Nguyen Tan Danh Huynh Ngoc Phien* and Ashim Das Guta Asian Institute of Technology PO Box 4 KlongLuang 220 Pathumthani Thailand Abstract In this study back roagation neural network (BPNN models were used to forecast daily river flows in two basins namely the Da Nhim and La Nga basins in the Central Highlands of Vietnam for comarison with the Tank Model. It was found that the develoed BPNN models rovided satisfactory forecast discharges for both basins. Moreover the discharges were also forecast from individual data of different stations within the La Nga Basin which were inut directly and searately to the considered model. In this case however the model took a longer time to run and the corresonding forecast discharges were not as accurate as those obtained when mean areal values of rainfall and evaoration were used instead. Introduction Forecasting of daily discharges has been one of the imortant roblems for hydrologists reservoir oerators and for flood rotection engineers. In this connection the relationshi between rainfall and runoff has been widely exloited in many concetual rainfall-runoff models of which the Tank Model of Sugawara (96; 979 aears to be a well-known examle (Phien and Pradhan 983; Phien and Danh 997. Besides these concetual models several black-box models (having little or no hysical considerations have also been used. These models include the time-series aroach using Box-Jenkins ARIMA models multile regression models (Phien et al. 990 etc. Recentlyback roagation neural networks (BPNNs a articular tye of neural network have been develoed and successfully used in many fields (Gorr et al. 994; Lachtermacher and Fuller 995 Maier and Dandy; 996. In this study BPNNs were used to develo models for forecasting daily discharges (with lead time equal to one day in two catchments namely the Da Nhim and La Nga Basins in the Central Highlands of Vietnam (Phien and Danh 997. Besides daily rainfall and evaoration data the inut to these models may include ast values of the discharge itself thanks to the structure of BPNNs. As a consequence of this flexibility in inut data the effect of the different combinations of inut data was also investigated. Back roagation neural networks Neural networks are mathematical models of theorised mind and brain activities which attemt to exloit the massively arallel local rocessing and distributed storage architecture of the human brain. The basic building block of the brain and nervous system is the neuron that sends/receives information to/from various arts of the body. Each neuron collects inuts from a single or multile sources and roduces a single outut in accordance with a certain redetermined non-linear function. A neural network model is created by interconnecting many of these simle neuron models in a known configuration. * To whom all corresondence should be addressed. ( ( ; fax ( ; hn@cs.ait.ac.th Received 3 November 997; acceted in revised form 9 July 998. Among the different neural network structures BPNNs introduced by Rumelhart et al. (986 are most oular because of their alicability in many different areas (see for examle Wasserman 989; Gershenfield and Weigend 993; Phien and Siang 993; Shamseldin 997. For forecasting uroses a tyical BPNN model consists of an inut layer one or two hidden layers and an outut layer that has only one node. Shown in Fig. is a tyical simle structure which is most commonly used in forecasting. In this case the inut layer has several nodes each reresenting an inut variable. The hidden layer also has several nodes and reresents the non-linearity of the network system. The outut layer has only one node which reresents the forecast value corresonding to each set of inut values. In rincile a BPNN may have several hidden layers but in ractice only one or two layers are used. The number of nodes in the hidden layer is determined mainly by trial and error. Several attemts have been made to arrive at some kind of otimal structure of a BPNN model (Fahlman and Lebiere 990; Lim and Hong 993. The back roagation method Back roagation is a systematic method for training (calibrating multilayer neural networks. It uses a set of airs of inut and outut values (called atterns. An inut attern is fed into the network to roduce an outut which is then comared with the actual outut attern. If there is no difference between the network outut and the actual outut then no learning is needed. Otherwise the weights - which exress the contribution of the inut nodes to the hidden nodes and the hidden nodes to the outut - are changed (backward from the outut layer through the hidden layer(s to the inut layer. Since the training makes use of the actual outut the back roagation method is referred to as a suervised training method. Notation The following notation system is used: W jim (n weight of the effect received by j th unit in layer m caused by i th unit in layer (m- at n th iteration. O outut of the jth element in layer m (m 2... L I i i th element of the inut. t j j th element of the desired outut (target number of units in the m th layer. n m Available on website htt:// ISSN Water SA Vol. 25 No. January

2 Outut layer Hidden layer Figure A simle back roagation neural network for forecasting The outut of a node is obtained as a non-decreasing and differentiable function (called an activation or transfer function of a linear combination (lus a bias term of the node inuts. In ractice the sigmoid activation function is usually used for back roagation: O f ( N N + e ( It has a simle derivative: where: f (N jjn O jn (- O j (2 nm N Wjim. Oim + θ (m 2.L (3 O io I i i : inut unit : a bias (similar to the constant term in a regression model. The outut from the last layer (the network outut is comared with the actual (desired outut and the error for attern is calculated as: n L 2 E ( t j Oj l (4 2 j and the total error is: E E (5 (which is equal to half of the sum of squared errors. It should be noted that for the case of Fig. the number of layers L is 3 and n L is. Training rocedure During the training (calibration stage the weights and bias factors are estimated. The standard training rocedure is summarised as follows with an exlanation of the notation following Eq. (:. Initialise all weights and bias factors to small random values. Forward ass: 2. Present inut vector (I I 2... I no and secify the desired outut (t t 2... t nl For layer m 2... L: a. Comute: nm N W. O + θ jim im i with: O io I i. b. Comute the outut of the j th unit in layer m: O + e N (6 ; j 2... n m. (7 3. Comare the final outut (O O 2... O nll with the desired outut (t t 2... t nl by comuting: n L E ( t j O 2 j L j so that the total error is: E E 2 where the sum extends all over data used in the training (calibration stage. If the value of E reaches a minimum the rocess is terminated and the system has learned. Otherwise continue to the next ste. Backward ass: 4. For layer m L L-... : a. For j 2...n m ; comute the weight errors: O ( O.( t O j when m L (outut layer (8 n jm O jm O m + ( Wkjm +. km + k when m hidden layers (9 b. Comute the weight increments: W jim (n+ η. O im- + α W jim (n (0 c. Comute the new values of the weights: W jim (n+ W jim (n + W jim (n+ ( 5. Go to ste ISSN Water SA Vol. 25 No. January 999 Available on website htt://

3 For an exlanation of the above rocedure it should be noted that the back roagation method tries to minimise the error E (or equivalently the sum of squared errors by adjusting the weights. Here: W. N W by the chain rule jim or in view of Eq. (6 : W jim jim.o im where: (2 For the nodes in the outut layer (m L: jl E O. OjL jl again by the chain rule jl or: ( t j O jl. f' (N jl from Eqs. (2 and (4. jl Similarly for the nodes in the hidden layer(s: O. O (by the chain rule By the chain rule the artial derivative of E with resect to O can be exressed as: O n m + k km+ O km+ Using Eqs. (2 (6 and (2 we obtain: n m+ ( k km+.w f' (N k+ So the weight changes are: Wji m ( n+ η. j m. O im (3 where: f'(n jl.(t j O jl m L n + m f'(n. W k+. km + m < L (4 k Here η is a constant which reresents the learning rate. The larger η the larger the changes in the weight thus the faster the desired weight is found. But if η is too large it may cause oscillations. Rumelhart et al. (986 roosed an additional term called the momentum which hels to increase the learning rate without leading to oscillations. With the addition of the momentum term the weight changes become: W jim (n+ η..o im- + α W jim (n which is exactly Eq. (0. Clearly α (the momentum is a constant which determines the effect of the ast weight changes on the current direction of the movement. There are several unresolved issues with BPNN models. Firstly there is no automatic method to arrive at an otimum architecture (reresented by the number of nodes in the inut layer the number of hidden layers and the number of nodes in hidden layers for a BPNN model. Trial and error rocedures have been used to arrive an accetable structure. Secondly the training rocedure converges very slowly. It may converge to a local minimum of the total error (or the sum of squared errors rather than the global minimum or it may not converge at all! The resent study still follows the standard rocedure introduced by Rumelhart et al. (986 but limited to the BPNN models in which there is only one hidden layer. It should also be noted that instead of trying to obtain the (global minimum value of the total error E the resent study adoted a more relaxed stoing rule as follows: E (Old - E(New E(Old 0.00 (5 where E(Old and E(New denote the values of E obtained in one iteration and the next iteration resectively. This stoing rule worked well in this study. Alications For comarison with the results obtained in the revious study (Phien and Danh 997 the back roagation method is alied to the forecasting of daily inflows to the Da Nhim Reservoir in the Da Nhim Basin (see Fig. Phien and Danh 997 and of the daily discharge at Phu Dien in the La Nga Basin (see Fig. 2 Phien and Danh 997 both in the Central Highlands of Vietnam. The forecasting lead time is equal to one day. The erformance indices introduced in the revious study (Phien and Danh 997 namely the efficiency index (EI the root mean squared error (RMSE the root mean squared error with resect to the mean (RMSEM and the standard deviation (RMSES and the mean absolute deviation (MAD are used again in this work. As BPNN models use the sigmoid function of which the outut values lie in the interval [0] all the inut values are transformed into the interval [ ] (instead of [0] because the logistic activation function aroaches 0 and asymtotically when the variable aroaches negative infinity and ositive infinity resectively. This is done for any (inut or outut variable X by using the following equation: X *(X- X max / (X max - X min (6 where X is the transformed variable X max and X min are its maximum and minimum values in the observed data. Once the outut values are obtained from a BPNN model the daily discharge values are obtained by transforming them back to the original scale using the inverse transformation of Eq.(6: X X min + (X max - X min * ( X /0.9 (7 There is obviously a constraint imosed on the outut by Eq. (7: as X 0.95 the values obtained from a BPNN model for the discharge are always less than or equal to X max : X X max. Similarly as X 0.05 the values obtained from a BPNN model are always greater than or equal to X min i.e. X X min. In other words the forecast values of the discharge will always fall in the range of its ast observed values. To avoid this constraint the values of X may be assumed to lie in the interval [0.8 X min.2 X max ] or in a wider interval. Available on website htt:// ISSN Water SA Vol. 25 No. January

4 Da Nhim Basin Data alication - For training (calibration: data in 7 years ( For testing (validation : data in 4 years ( The streamflow station under consideration is located at the Da Nhim Dam (see Fig. Phien and Danh 997. Calculation combinations TABLE ARCHITECTURE OF BPNN MODELS FOR THE DA NHIM BASIN Case No. of nodes No. of nodes No. of nodes in inut in hidden in outut layer layer layer Case.Discharge at time t+ is a function of rainfall and evaoration at time t Q t+ f (R t E t. Case 2.Discharge at time t+ is a function of rainfall and evaoration at time t as well as of the ast value of discharge at time t: Q t+ f (R t E t Q t Case 3.Discharge at time t+ is a function of rainfalls and evaoration at times t and t- and of ast values of discharge at times t t- and t-2: Q t+ f (R t E t R t- E t- Q t Q t- Q t-2 It should be noted that Case was used in making the Tank Model alicable to daily flow forecasting by Phien and Danh (997. Case 2 is similar to Case excet that the value of the discharge on the immediately receding day is also included among the inut factors. This is an obvious extension of Case thanks to the flexibility of the BPNN models as comared to concetual models such as the Tank Model. Case 3 corresonds to an extreme situation for a catchment area as small (about 775 km² as that of the Da Nhim Basin: the effect of rainfall and evaoration on the discharge should not last longer than 2 d while the ersistence in the discharge itself can be adequately reresented by its values in the last 3 d. TABLE 2 PERFORMANCE STATISTICS OF BPNN MODELS FOR THE DA NHIM BASIN Case Stage EI RMSE RMSEM MAD RMSES Calibration Validation Calibration Validation Calibration Validation Discussions l l The architecture for each case is shown in Table. In fact for all the cases trial and error was used to determine the number of nodes in the hidden layer because the number of nodes in the inut layer has been secified in the Cases considered : 2 for Case 3 for Case 2 and 7 for Case 3. An aroriate architecture of a network was found when satisfactory values of the erformance indices were obtained. During the training rocess the learning rate (η and the momentum (α were varied between 0. to.0. It was found that the most aroriate values for both of them are 0.5. The calculated erformance statistics are shown in Table 2. In three cases it was clear that Case 3 gives the best results. By examining the hydrograhs showing the observed discharges and their forecast values it was found that in Case baseflow in the Figure 2 Comarison between observed and forecast discharges for Da Nhim 987 (Case low-flow eriod was over-estimated while flow during the highflow eriod was under-estimated (Fig.2 for the year 987 calibration. In Cases 2 and 3 an imrovement of the forecast discharges has been achieved: values of both low-flow and eak eriods are well matched esecially in Case 3 (Fig. 3 for Case 2 and Fig. 4 for Case 3 both for the year 987. As seen in Table 2 the efficiency index for Case is equal to 0.67 for the calibration stage and 0.60 for the validation stage. Both are relatively smaller than the values obtained by the Tank Model (0.69 and 0.65 resectively indicating that the erform- 36 ISSN Water SA Vol. 25 No. January 999 Available on website htt://

5 ance of the BPNN model is a little worse than that of the Tank Model (see Table 4 Phien and Danh 997. Other erformance indices indicate the same thing. However BPNN models can readily be exanded to take into account other influencing factors as shown in Cases 2 and 3. Table 2 clearly shows that the resulting models are far better than the Tank Model. In fact by removing both E t and E the efficiency t- index value reduces very slightly. These results show imortant contributions of ast values of discharge toward its future values. La Nga Basin (see Fig. 2 Phien and Danh 997 TABLE 3 PERFORMANCE STATISTICS OF THE BPNN MODEL FOR LA NGA BASIN Case EI RMSE RMSEM MAD RMSES Calibration Verification Calibration Verification Calibration Verification The data on rainfall evaoration and discharge are available from 987 to 994. The data in the first three years ( were used for calibration and those in the last five years ( were used for validation. It should be noted that the discharge station under consideration is Phu Dien with a catchment area of km² which is much larger than that of the dam site station in the Da Nhim Basin. Calculation combinations Case.Q t+ f ([R t ] mean areal [E t ] mean areal. Case 2.Q t+ f ([R t ] mean areal [R t- ] mean areal [E t ] mean areal [E t- ] mean areal Q t Q t- Q t-2. Case 3. Q t+ f ([R t ] all stations [E t ] mean areal Q t Q t- Q t-2. It should be noted that [R t ] denotes the mean areal mean areal rainfall calculated in Phien and Danh (997 with the weights obtained by the method suggested by Sugawara (96 while [E t ] is the arithmetic mean taken all over mean areal the five stations located within the La Nga Basin. In Case 3 [R t ] all stations means individual values of rainfall at each of the five stations namely Bao Loc Di Linh Dai Nga Phu Dien and Ta Pao (see Fig. 2 Phien and Danh 997. As for the Da Nhim Basin Case is the same for the Tank Model. In Case 2 even though for this medium size catchment the effect of rainfall and evaoration on discharge may last longer than two days but it is taken as 2 d (to reduce the number of nodes in the inut layer while any remaining effect is assumed to be taken care of by the ast values of the discharge itself in the last 3 d. In Case 3 due to a large number of inut variables (being equal to 9 only the values at time t for rainfall and evaoration were used. Discussion For Case the results obtained from the BPNN model are satisfactory (see Table 3 but a little Figure 3 Comarison between observed and forecast discharges for Da Nhim 987 (Case 2 Figure 4 Comarison between observed and forecast discharges for Da Nhim 987 (Case 3 Available on website htt:// ISSN Water SA Vol. 25 No. January

6 Figure 5 Comarison between observed and forecast discharges for La Nga 994 (Case 2 Figure 5 Comarison between observed and forecast discharges for La Nga 994 (Case 3 worse than those obtained from the Tank Model. From these results as well as from those obtained for the Da Nhim Basin it aears that as long as the same inut variables were used the BPNN models can erform almost as well as the Tank Model. An imortant asect related to BPNN models is that they can use as inut data any meaningful combinations among the inut variables including their lagged variables. As such Cases 2 and 3 deserve more attention. From the hydrograhs of the observed and forecast discharges it was found that the discharges were very well forecast in Case 2. Both the low flows and high flows were accurately forecast and the forecast hydrograh follows the observed hydrograh very closely. Tyical results are illustrated in Fig. 5 for the year 994 of the validation stage. In Case 3 the forecast discharge hydrograh does not match the observed hydrograh as well as in Case 2. Discharges were overestimated during the lowflow eriod. Only the eaks were forecast satisfactorily (see Fig. 6 for the year 994. From the erformance statistics and the hydrograhs one can say that Case 2 is better than Case 3. Moreover since fewer inut units are introduced to the model in Case 2 the resulting model runs faster. It should be noted that the RMSE for Case 2 is about 0% of the mean value and about 9% of the standard deviation of the observed discharge indicating that a very satisfactory forecasting model is obtained. It was also found that as for the Da Nhim Basin the removal of evaoration from the cases considered only affects the erformance statistics slightly indicating that the contribution of evaoration to the daily discharge in the La Nga Basin is negligible. In fact the evaoration was considered in this study because the Tank Model and many concetual rainfall-runoff models include this variable as one of the imortant factors that roduce the daily discharge at any station. Conclusions The results obtained in this aer indicate the caability of BPNN models in forecasting of daily river flows using daily evaoration and rainfall data as inuts. It should be noted that the forecasting results were found to be better for the La Nga Basin which is the larger basin of the two. This confirms the common belief that forecasting of discharges for larger basins is often more accurate than for smaller basins. It was noted that the contribution of ast values of discharge was very imortant in BPNN models just as in regression models 38 ISSN Water SA Vol. 25 No. January 999 Available on website htt://

7 (Phien et al Perhas this fact is the main reason for the wide accetance of Box-Jenkins ARIMA models which are based uon the linear relationshi between successive observations of the discharge itself. For BPNN models again as for regression models mean areal values or individual values at all rainfall and evaoration stations within the basin under consideration can be used as inuts in forecasting the required discharges. However while individual inuts are useful in determining the contribution of each factor the results obtained in this work for the case of the La Nga Basin indicate that use of aroriately calculated mean values leads to better forecasting erformance. Moreover it was also found that the contribution of the evaoration is not imortant. References FAHLMAN SE and LEBIERE C (990 The Cascade-Correlation Learning Architecture. Technical Reort School of Comuter Science Carnegie Mellon University CMU-CS GERSHENFIELD NA and WEIGEND AS (993 The Future of Time Series. Technical Reort. Palo Alto Research Center. GORR WL NAGIN D and SZCZYPULA J (994 Comarative study of artificial neural networks and statistical models for redicting student grade oint average. Int. J. Forecasting LACHTERMACHER G and FULLER JD (995 Backroagation in time series forecasting. J. Forecasting LIM SF and HONG SB (993 Dynamic Creation of Hidden Units with Selective Pruning in Backroagation. Technical Reort Det. of Information Systems and Comuter Science National University of Singaore No. TRD4/93. MAIER HR and DANDY GC (996 The use of artificial neural networks for the rediction of water quality arameters. Water Resour. Res. 32 ( PHIEN HN and DANH NT (997 A hybrid model for daily flow forecasting. Water SA 23 ( PHIEN HN and PRADHAN PSS (983 The Tank Model in rainfall-runoff modelling. Water SA 9 ( PHIEN HN and SIANG JJ (993 Forecasting monthly flows of the Mekong River using back roagation. Proc. IASTED Int. Conf PHIEN HN HUONG BK and LOI PD (990 Daily flow forecasting with regression analysis. Water SA 6 ( RUMELHART DE HINTON GE and McCLELLAND JL (996 A general framwork for arallel distributed rocessing. In: RUMEL- HART DE and McCLELLAND (eds. Parallel Distributed Processing; Exlorations in the Microstructure of Cognition Vol. MIT Press Cambridge. SHAMSELDIN AY (997 Alication of a neural network technique to rainfall-runoff modelling. J. Hydrol. 99 ( SUGAWARA M (96 On the analysis of runoff structure about several Jaanese rivers. Jaanese J. Geohys. 2 (4-76. SUGAWARA M (979 Automatic calibration of the Tank Model. Hydrol. Sci. Bull. 24 ( WASSERMAN PD (989 Neural Comuting Theory and Practice. Van Nostrand Reinhold New York Available on website htt:// ISSN Water SA Vol. 25 No. January

8 40 ISSN Water SA Vol. 25 No. January 999 Available on website htt://