American ourna of Appied Sciences 7 (): 372-378, 2 ISSN 546-9239 2 Science Pubications Seasona Time Series Data Forecasting by Using Neura Networks Mutiscae Autoregressive Mode Suhartono, B.S.S. Uama and A.. Endharta Department of Statistics, Facuty of Mathematics and Natura Sciences, Institute Technoogy Sepuuh Nopember, Surabaya 6, Indonesia Abstract: Probem statement: The aim of this research was to study further some atest progress of waveet transform for time series forecasting, particuary about Neura Networks Mutiscae Autoregressive (NN-MAR). Approach: There were three main issues that be considered further in this research. The first was some properties of scae and waveet coefficients from Maxima Overap Discrete Waveet Transform (MODWT) decomposition, particuary at seasona time series data. The second focused on the deveopment of mode buiding procedures of NN-MAR based on the properties of scae and waveet coefficients. Then, the third was empirica study about the impementation of the proposed procedure and comparison study about the forecast accuracy of NN-MAR to other forecasting modes. Resuts: The resuts showed that MODWT at seasona time series data aso has seasona pattern for scae coefficient, whereas the waveet coefficients are stationer. The resut of mode buiding procedure deveopment yieded a new proposed procedure of NN-MAR mode for seasona time series forecasting. In genera, this procedure accommodated input ags of scae and waveet coefficients and other additiona seasona ags. In addition, the resut showed that the proposed procedure works we for determining the best NN-MAR mode for seasona time series forecasting. Concusion: The comparison study of forecast accuracy showed that the NN-MAR mode yieds better forecast than MAR and ARIMA modes. Key words: Neura networks, mutiscae, MODWT, NN-MAR, seasona, time series INTRODUCTION Recenty, neura network has been proposed in many researches about different kinds of statistica anaysis. There are many types of neura network appied to sove many probems. For exampes, Feedforward Neura Network (FFNN) is appied in eectricity demand forecasting Tayor et a. (26), Genera Regression Neura Network (GRNN) is used in exchange rates forecasting and Recurrent Neura Network (RNN) has been appied in detecting changes in autocoreated process for quaity monitoring. Different from those previous researches, here, the predictors or the inputs are not the ags of the variabes or the data variabes, but they are the coefficients from waveet transformation. A new deveopment reated with waveet transformation appication for time series anaysis is proposed. As an overview this can be seen in Nason and von Sachs (999). At the beginning, most waveet research for time series anaysis is focused on periodogram or scaogram anaysis of periodicities and cyces evauation (Priestey, 996; Morettin, 997; Gao, 997; Perciva and Waden, 2). Bjorn (995); Sotani et a. (2) and Renaud et a. (23) are some first researcher groups discussing waveet for time series prediction based on autoregressive mode. In this case, waveet transformation gives good decomposition from a signa or time series, so that the structure can be evauated by parametric or nonparametric modes. WNN is a neura network with waveet function used in processing in transfer function. In time series forecasting cases, input used in WNN is waveet coefficients in certain time and resoution. Recenty, there are some artices about WNN for time series forecasting and fitering, such as Bashir and E-Hawary (2); Renaud et a. (23); Murtagh et a. (24) and Chen et a. (26). Waveet transformation that is mosty used for time series forecasting is Maxima Overap Discrete Waveet Transform (MODWT). The use of MODWT is to sove the imitation of Discrete Waveet Transform (DWT), that requires N = 2 where is positive integer. In practice, time series data rarey Corresponding Author: Suhartono, Department of Statistics, Facuty of Mathematics and Natura Sciences, Institute Technoogy Sepuuh Nopember, Surabaya 6, Indonesia 372
fufi those numbers, which are two powered with a positive integer. Some present researches reated with WNN for time series forecasting usuay focus on how to determine the best WNN mode which is appropriate for time series forecasting. The aim of this research is to deveop an accurate procedure for WNN modeing of seasona time series data and to compare the forecast accuracy with Mutiscae Autoregressive (MAR) and ARIMA modes. MATERIALS AND METHODS Data: The number of tourist arrivas to Bai through Ngurah Rai airport, from anuary 986 unti Apri 28, is used as a case study. The in-sampes are first 26 observations and the ast 6 observations are used as the out-sampe dataset. The anaysis starts by appying MODWT decomposition to the data. Based on the scae and waveet coefficients pattern, then the proposed of WNN mode buiding procedure for time series data forecasting wi be deveoped. This procedure is the improvement of genera FFNN mode buiding procedure for time series data forecasting. In this new procedure, the determination of the inputs in WNN mode is done by using waveet coefficient ags and the boundary effects. Whereas, the seection of the best WNN mode is done by empoying a combination between the inferentia statistics for the addition contribution in forward scheme for seecting the optimum number of neurons in the hidden ayer and Wad test in backward scheme for determining the optimum input unit. Waveets and prediction: Waveet means sma wave, whereas by contrast, sinus and cosines are big waves (Perciva and Waden, 2). A function ψ (.) is defined as waveet if it satisfies: ψ (u)du = () ψ 2 (u)du = (2) Am.. Appied Sci., 7 (): 372-378, 2 Meyer waveet, Daubechies waveet, Mexican hat waveet, Coifet waveet and ast assymetric waveet (Daubechies, 992). Scae and waveet equations: Scae equation or diate equation shows scae function φ experiencing contraction and transation (Debnath, 2), which is written as: Commony, waveets are functions that have characteristic as in Eq.. If it is integrated on (, ) the resut is zero and the integration of the quadrate of function ψ (.) equas to as written in Eq. 2. There are two functions in waveet transform, i.e., scae function (father waveet) and mother waveet. These two functions give a function famiy that can be used for reconstructing a signa. Some waveet famiies are Haar waveet (the odest and simpest waveet), Transform (MODWT) term. 373 L (3) = φ (t) = 2 g φ(2t ) where, φ(2t ) is scae function φ(t) experiencing contraction or transation in time axis with steps with scae fiter coefficient g. Waveet function ψ is defined as: L (t) 2 ( ) g (2t L 2 ) = ψ = φ + + (4) Coefficient g must satisfy conditions: L L m g = 2 dan ( ) g = = = for m =,, L,(L / 2) and: (5) L gg+ 2m =, m for m =, L,(L / 2) (6) = and: L 2 = g = (7) The reationship between coefficients h and g is + h = ( ) g, or it can be written as g ( ) h. Maxima Overap Discrete Waveet Transform (MODWT): One of modifications from Discrete Waveet Transform (DWT) is Maxima Overap Discrete Waveet Transform (MODWT). MODWT has been discussed in waveet iteratures with some names, such as undecimated-discrete Waveet Transform (DWT), Shift invariant DWT, waveet frames, transation DWT, non decimated DWT. Perciva and Waden (2) stated that essentiay those names are the same with MODWT which have connotation as mod DWT or modified DWT. This is the reason of this research using Maxima Overap Discrete Waveet
DWT suppose the data satisfy 2 j. In rea word most time series data has the ength that is not foowing this mutipication. MODWT has the advantage, which can eiminate the presence of data reduction to the haf (down samping). So that in MODWT there are N waveet and scae coefficients in each eves of MODWT (Perciva and Waden, 2). If there is time series data x, with N-ength, the MODWT transformation wi give coumn vectors w,w 2,...,w and v, each with N-ength. Vector w contains scae coefficients. As in DWT, in MODWT the efficient cacuation is done by pyramid agorithm. The smoothing coefficient of signa X is obtained iterativey by mutipying X with scae fiter or ow pass (g) and waveet fiter or high pass (h). In order to abridge the reationship of DWT and MODWT, waveet fiter and scae fiter definitions given by: Definition : (Perciva and Waden, 2): MODWT waveet fiter {h % } through h % h / 2 and MODWT scae fiter {g% } through {g% }g% g / 2. So that MODWT waveet fiter must satisfy this equation: h % =, h % = and h % h % = (8) L L 2 + 2m = = 2 = and the scae fiter must accompish the foowing equation: g% =, g% = and g% g% = (9) L L 2 + 2m = = 2 = Time series prediction by using waveet: Generay, time series forecasting given by using waveet is a forecasting method that use data preprocessing through waveet transform, especiay through MODWT. By the presence of mutiscae decomposition ike waveet, the advantage is automaticay separating the data components, such as trend component and irreguar component in the data. Thereby, this method coud be used for forecasting of stationary data (contain ony irreguar components) or non-stationary data (contain trend and irreguar components). For exampe, suppose that stationary signa X = (X,X 2,,X t ) and assume that vaue X t+ wi be forecasted. The basic idea is to use coefficients that are constructed from the decomposition, i.e., (Renaud et a., 23): Am.. Appied Sci., 7 (): 372-378, 2 374 and: w for k,2,,a, j,2, K, = K j,t 2 (k ) j = v for k =,2, K,A,t 2 (k ) The first step that shoud be known is how many and which waveet coefficients that shoud be used in each scae. Renaud et a. (23) introduced a process to cacuate the forecast at time (t+) th by using waveet mode as iustrated in Fig.. Figure represents the common form of waveet modeing with eve = 4, order A j = 2 and N = 6. Fig. iustrates that if the 8th data wi be forecasted, the input variabes are waveet coefficients in eve at t = 7 and t = 5, eve 2 at t = 7 and t = 3, eve 3 at t = 7 and t = 9, eve 4 at t = 7 and t = and smooth coefficient in eve 4 at t = 7 and t =. Hence, we can concude that the second input at each eve is t-2 j. The basic idea of mutiscae decomposition is trend pattern infuences Low frequency (L) components that tend to be deterministic; whereas High frequency (H) component is sti stochastic. The second point in waveet modeing for forecasting is about the function used to process the inputs, i.e., waveet coefficients to forecast at (t+) th period. Generay, there are two kinds of function that can be used in this input-output processing, such as inear and noninear functions. Renaud et a. (23) deveoped a inear waveet mode known as Mutiscae Autoregressive (MAR) mode. Moreover, Renaud et a. (23) aso introduced the possibiity of the noninear mode use in inputoutput processing of waveet mode, especiay Feed- Forward Neura Network (FFNN). Furthermore the second mode is known as Waveet Neura Network (WNN) mode. These two approaches use the ags of waveet coefficients as the inputs, i.e. scae and smooth coefficients as in Fig.. Fig. : Waveet modeing iustration for = 4 and A j = 2 j
Am.. Appied Sci., 7 (): 372-378, 2 Mutiscae Autoregressive (MAR): An autoregressive process with order p which is known as AR(p) can be written as: Procedures: There are four proposed procedures for buiding WNN mode for forecasting non-stationary (in mean) time series, i.e.: ˆX p = φˆ X t + k t (k ) k = By using decomposition of waveet coefficients, Renaud et a. (23) expained that AR prediction in this way coud be expanded become Mutiscae Autoregressive (MAR) mode, i.e.: Aj t + j,k j,k j,t 2 (k ) +,t 2 (k ) j= k= k= A j () Xˆ = aˆ w + aˆ v Where: j = The numbers of eve (j =,2,,) A j = Order of MAR mode (k =,2,,A j ) w j,i = Waveet coefficient vaue v j,t = Scae coefficient vaue a j,k = MAR coefficient vaue Waveet neura network: Suppose that a stationary signa X = (X,X 2,,X t ) and assume that X t+ wi be predicted. The basic idea of waveet neura network mode is the coefficients that are cacuated by the decomposition as in Fig. are used as inputs at certain neura network architecture for obtaining the prediction of X t+. Renaud et a. (23) introduced Mutiayer Perceptron (MLP) neura network architecture or known as Feed-Forward Neura Network (FFNN) to process the waveet coefficients. The architecture of this FFNN consists of one hidden ayer with P neurons that is written as: ˆX N+ = P p= bˆ pg A+ A j j= k = k= â â w j,k,p j j,n 2 (k ) v +,k,p j j,n 2 (k ) + () where, g is an activation function in hidden ayer, which is usuay sigmoid ogistic. In this FFNN, the activation function in output ayer is inear. Furthermore, mode in Eq. is known as Waveet Neura Network (WNN) or Mutiresoution Neura Network (MNN). 375 The inputs are the ags of scae and waveet coefficients simiar to Renaud et a. (23) The inputs are the combination between the ags of scae and waveet coefficients proposed by Renaud et a. (23) and some additiona ags that are identified by using stepwise The inputs are the ags of scae and waveet coefficients proposed by Renaud et a. (23) from differencing data The inputs are the combination between the ags of scae and waveet coefficients proposed by Renaud et a. (23) and some additiona ags identified by using stepwise from differencing data In this research, the additiona ags are the seasona ags because of the data pattern. The first and second procedures are used for the stationary data, whereas the third and fourth procedures are used for data that contain a trend. This study ony iustrates the fourth procedure. Stepwise method is used to simpify the process in finding the significant inputs. After buiding WNN mode, the resuts at out-sampe dataset are compared to MAR and ARIMA modes to find the best mode for forecasting the number of tourist arrivas to Bai. At the proposed first new procedure, the seection of the best WNN mode is done firsty by determining an appropriate number of neurons in the hidden ayer. The starting step before appying the proposed procedure is the determination of the eves or in MODWT. In this case, a scae and waveet coefficient ags from MAR() and additiona seasona ags which are significant based on stepwise method are used as inputs. Different from inear waveet mode (MAR) that the modeing process was divided into two additive parts, namey modeing the trend by using waveet coefficients and MAR modeing for the residua by using the waveet and scae coefficient ags. In this proposed procedure, the modeing of WNN is done simutaneousy by using scae and waveet coefficient ags. This is based on the fact that WNN is noninear mode expected to be abe to catch data characteristics simutaneousy by using scae and waveet coefficients from MODWT. The first proposed procedure for WNN mode buiding for forecasting seasona time series data can be seen at Fig. 2.
Am.. Appied Sci., 7 (): 372-378, 2 Fig. 2: The procedure for WNN mode buiding for forecasting seasona time series data using inference combination of R 2 incrementa and Wad test 376
Am.. Appied Sci., 7 (): 372-378, 2 RESULTS AND DISCUSSION The time series pot of the number of tourist arrivas to Bai through Ngurah Rai airport is shown in Fig. 3. The pot shows that the data has seasona and trend patterns. These data have been anayzed by using MAR and ARIMA modes and the resuts showed that MAR( = 4;[2,36],[2,36],[36],[],[])-Haar yieded better forecast than ARIMA mode. As the starting step, the modeing focuses to determine an appropriate number of neurons in the hidden ayer. In this study, scae and waveet coefficient ag inputs are assumed as ag inputs in noninearity test in the first step. Every proposed procedure is begun by using noninearity test, i.e., White test and Terasvirta test. By using scae and waveet coefficient ags as the inputs as proposed by Renaud et a. (23), the resuts show that there is a noninear reationship between inputs and the output. Hence, it is correct to use a noninear mode as WNN for forecasting the data. The next step of the fourth procedure is to determine an appropriate number of neurons in the hidden ayer. This step is started from one neuron unti the additiona neuron show does not have significanty contribution. The resuts of the seection process of the number of neurons which is appropriate with WNN mode using ag inputs proposed by Renaud et a. (23) can be seen in Tabe for the Daubechies(4) waveet famiy or D(4) and Tabe 2 for Haar waveet famiy. Moreover, the resuts of forecast accuracy comparison between WNN and MAR coud be seen in Tabe 3. Based on the resuts in Tabe and 2, the first proposed procedure shows that the best WNN mode for forecasting the number of tourist arrivas to Bai consists one neuron in the hidden ayer for both D(4) and Haar waveet. In this architecture, the inputs are the ags of scae and waveet coefficients of MAR() and mutipicative seasona ags which are statisticay significant from stepwise methods. Fig. 3: Pot of the number of tourist arrivas to Bai Tabe : The resut of the first proposed procedure for determining an appropriate number of neurons, using D(4) waveet No. of neurons RMSE of in-sampe RMSE of out-sampe R 2 R 2 increment F p-vaue.4344658.9728382.5885 - - - 2.4257894.9758635.72394.225.85739.67869 3.4235.97657623.723696.23.834 4.4225858.977566.724948.3.4276 5.423324.97634864.72366 -.9 -.62 6.4259.9747289.7253 -.2 -.638 7.4254999.97469538.79797 -.3 -.383 8.42532.9746483.7276.4.439 9.423524.97854646.725838.47.3.4263965.97378698.7976 -.6 -.648 Tabe 2: The resut of first proposed procedure for determining an appropriate number of neurons, using Haar waveet No. of neurons RMSE of in-sampe RMSE of out-sampe R 2 R 2 increment F p-vaue.376594.9776267.6492 - - - 2.3466526.99448277.95389.3489.46254.33262 3.325475.9463364.296628.4274.528266.967427 4.3242228.94433753.2626.963.3447 5.32384239.94345.29797.354.224 6.32364476.9429925.2847.25.6823 7.323852.9455599.26665.482.543 8.32333238.944276.2457 -.2 -.644 9.3236779.9433862.2465 -.3 -.95.3229692.9443982.2772.625.7656 377
Am.. Appied Sci., 7 (): 372-378, 2 Tabe 3: The resut of forecast accuracy comparison for testing data Method Procedure RMSE of in-sampe RMSE of out-sampe Expanation about the best mode WNN 4 - Haar waveet.376.978 MAR()-Haar, neuron 4 - Daubechies waveet.434.973 MAR()-D(4), neuron MAR MAR.85,4 MAR( = 4;[2,36],[2,36],[36],,)-Haar If the seection of WNN mode is done based on cross-vaidation principe, then the best mode is the mode that yieds the minimum vaue of RMSE at testing dataset, i.e., the WNN mode that consists of one neuron in the hidden ayer both for D(4) and Haar waveets with RMSE.973 and.978 respectivey. Hence, WNN mode with one neuron in the hidden ayer that uses D(4) waveet is the best mode. In addition, the resut of forecast accuracy comparison between WNN and MAR modes at Tabe 3 shows that WNN mode with one hidden neuron that uses D(4) waveet famiy yieds the most accurate forecast than other modes. CONCLUSION Based on the resuts at the previous sections, it can be concuded that there is a difference pattern between scae and waveet coefficients of MODWT circuar decomposition. For non-stationary seasona time series data, the scae coefficients have non-stationary and seasona pattern, whereas the waveet coefficients in each decomposition eve tend to have a stationary pattern and the vaues are around zero. Then, new procedures for buiding NN-MAR based on these properties of scae and waveet coefficients are proposed. The empirica resuts by using data of the number of tourist arrivas to Bai show that the proposed procedure for buiding a WNN mode works we for determining appropriate mode architecture. Moreover, the forecast accuracy comparison shows that the proposed procedure using stepwise in the beginning step for determining the ag inputs yieds more parsimony mode and more accurate forecast than other procedures. REFERENCES Bashir, Z. and M.E. E-Hawary, 2. Short term oad forecasting by using waveet neura networks. Proceeding of the Canadian Conference on Eectrica and Computer Engineering, Mar. 7-, IEEE Xpore Press, Haifax, NS., Canada, pp: 63-66. DOI:.9/CCECE.2.84969 Bjorn, V., 995. Mutiresoution methods for financia time series prediction. Proceeding of the IEEE/IAFE 995 Conference Computationa Inteigence for Financia Engineering, Apr. 9-, IEEE Xpore Press, New York, USA., pp: 97-97. DOI:.9/CIFER.995.495258 378 Chen, Y., B. Yang and. Dong, 26. Time-series prediction using a oca waveet neura network. Neurocomputing, 69: 449-465. DOI:.6/j.neucom.25.2.6 Daubechies, I., 992. Ten Lectures on Waveets. st Edn., SIAM: Society for Industria and Appied Mathematics, USA., ISBN: 89872742, pp: 377. Debnath, L., 2. Waveet Transform and their Appication. st Edn., Birkhhauser Boston, Boston, ISBN: 8764248, pp: 565. Gao, H.Y., 997. Choice of threshods for waveet shrinkage estimate of the spectrum.. Time Ser. Ana., 8: 23-25. DOI:./467-9892.48 Morettin, P.A., 997. Waveets in statistics. Resenhas, 3: 2-272. Murtagh, F.,.L. Starckand and O. Renaud, 24. On neuro-waveet modeing. Dec. Support Syst., 37: 475-484. DOI:.6/S67-9236(3)92-7 Nason, G.P. and R. von Sachs, 999. Waveets in time series anaysis. Phi. Trans. R. Soc. Lond. A., 357: 25-2526. DOI:.98/rsta.999.445 Perciva, D.B. and A.T. Waden, 2. Waveets Methods for Time Series Anaysis. st Edn., Cambridge University Press, Cambridge, ISBN: 5264687, pp: 62. Priestey, M.B., 996. Waveets and time-dependent spectra anaysis.. Time Ser. Ana., 7: 85-4. DOI:./j.467-9892.996.tb266.x Renaud, O.,.L. Stark and F. Murtagh, 23. Prediction based on a mutiscae decomposition. Int.. Waveets Mutiresout. Inform. Process., : 27-232. Sotani, S., D. Boichu, P. Simard and S. Canu, 2. The ong-term memory prediction by mutiscae decomposition. Sign. Process., 8: 295-225. DOI:.6/S65-684()77-3 Tayor,.W., L.M. Menezes and P.E. McSharry, 26. A comparison of univariate methods for forecasting eectricity demand up to a day ahead. Int.. Forecast., 22: -6. DOI:.6/j.ijforecast.25.6.6