An artificial neural network hybrid with wavelet transform for short-term wind speed forecasting: A preliminary case study Moslem Yousefi *,1, Danial Hooshyar 2, Milad Yousefi 3 1 Center for Advanced Mechatronics and Robotics Universiti Tenaga Nasional Jalan IKRAM-UNITEN, Kajang, 43000, Selangor Malaysia 2 Department of Software Engineering, Faculty of Computer Science and Information Technology University of Malaya Kuala Lumpur, Malaysia 3 Departamento de Engenharia Mecânica, Universidade Federal de Minas Gerais UFMG, Minas Gerais, Brazil Abstract Given the importance of an accurate wind speed forecasting for efficient utilization of wind farms, and the volatile nature of wind speed data including its non-linear and uncertain nature, the wind speed forecasting has remained an active field of research. In this study, the non-linearity of wind speed is tackled using artificial neural network and its uncertainty by wavelet transform. To avoid trial-and-error process for selection the ANN structure, the results of auto correlation factor (ACF) and partial auto correlation factor (PACF) on the historical wind speed data are employed. Instead of forecasting the time series directly, a set of better-behaved components of the data is achieved by decomposing the data using wavelet transform and are forecasted separately using a feedforward neural network. Finally, using an inverse wavelet transform, the future time series is reconstructed and the wind speed could be forecasted. The historical hourly wind speed from ABEI weather station in Idaho, United States is used for assessing the performance of the proposed algorithm. This data set is merely selected due to its availability. The data is divided to three parts of 50%, 25% and 25% for training, testing and validation respectively. The testing part of data set will be merely used for assessing the performance of the neural network which guarantees that only unseen data is used to evaluate the forecasting performance of the network. On the other hand, validation data could be used for parametersetting of the network if required. The results shows that using wavelet transform can enhance the forecasting accuracy when it is compared with a regular neural network prediction algorithm. Keywords short-term wind speed forecasting, wavelet transform, signal processing, artificial neural network I. INTRODUCTION Due to the unexpected and continuous increase in the price of fuel-extracted energy and electricity, huge attention has been given to renewable energy resources including wind power, solar energy etc. Given the availability of wind powers, many countries are generating a considerable amount of their energy based on this source. The wind power is greatly affected by wind speed and therefore an accurate wind speed forecasting is Weria Khaksar 1, Khairul Salleh Mohamed Sahari 1, Firas B. Ismail Alnaimi 4 4 Power Generation Research Centre College of Engineering Universiti Tenaga Nasional Jalan IKRAM-UNITEN, Kajang, 43000, Selangor Malaysia utterly essential. Wind forecasting could be related to different time periods ranging from a few hours ahead, to a few days ahead or more. Normally a prediction for a horizon of more than three days is considered long-term prediction, whereas medium-term refers to a period of a few hours to a few days, normally three. A short-term forecasting, which is the subject of the current study, is associated with horizon of up to a few hours. Nevertheless, these definitions are not fixed and could vary. [1-3] The most prominent WS forecasting methods include statistical and intelligent modelling. In the former, historical data are explored in order to form a linear or non-linear relation between them and future speed values where Auto-Regressive and Moving Average (ARMA) is the most employed method of this type in forecasting. [4]. Intelligent modelling develops a high-dimension and non-linear function to relate all the historical data, which could influence the forecasting, to the future speed by minimizing a defined training error. Artificial Neural Networks (ANN) [5] and Support Vector Machines (SVM) are among the mostly implemented algorithms in this field [6]. The above mentioned methods could be integrated together and with available data analysis methods to improve the accuracy of the wind speed forecasting. As an example, preprocessing of wind data could result in a better forecasting result. For example, Empirical mode decomposition (EMD) was implemented in order of decomposing the wind speed data into various intrinsic mode functions (IMFs) [7]. Although the input to the neural network and its structure have a vital role on the accuracy of the forecast, in most of the published works, these are selected based on the expertise of the user rather than a systematic way. Partial autocorrelation function (PACF) was used for selecting the input variable in [7]. PACF is applied on a time series to determine the relation between input variables and wind speed. In another attempt, clustering approaches like self-organizing map (SOM) was implemented for clustering 978-1-4799-8386-5/15/$31.00 2015 IEEE 95
wind speed data and preparing the models based on data similarities [8]. In general, any supervised algorithm, including ANN, SVM and etc., could be used for forecasting a time series. The most important part of these algorithms is their training methods which could be modified for better performance. Such a job could be done by implementing evolutionary algorithms, like a conventional genetic algorithm (GA), Particle swarm optimization (PSO) or its variants, Artificial bee colony (ABC) etc., for tuning the weight of these algorithms [9-10]. Moreover, the uncertainty of the noise in the historical data could be managed in order to achieve better results. Among different tools, Wavelet transform has shown great effectiveness in the field of signal processing [11]. Generally, two categories of WTs are known, if the sampling of wavelets is discreet then the WT is called discrete wavelet transform(dwt) while for a continuous sampling the WT is referred to as continuous wavelet transform (CWT) WT are superior to Fourier transforms as they can capture information regarding both frequency and location. In this study, the preliminary aim is to develop a robust method for short-term forecasting of wind speed based on neural network and wavelet transform. A. Artificial neural network III. THE PROPOSED METHOD In the proposed forecasting method, a back propagation feedforward neural network is adopted for constructing the forecasting structure. This ANN class is a supervised structure where normally a defined error functions, which is typically mean square error, is minimized using a gradient descent method. A typical structure of a feedforward neural network is presented in Fig 1. Input, hidden and output layers are the main parts of this type of neural network. The neurons in each layer are connected through a link which is mathematically represented by a weight in the network. This weight is the measure of the connection between the two nodes. These weights are changed in different steps of learning in order to minimize a chosen error function which is generally mean squared error (MSE) and ultimately to make the network applicable to any unknown sample. Although there are many algorithms available for weight selection of the ANN, in this study a conventional Levenberg Marquardt algorithm (LMA) is used for weight selection of the ANN structure. Although a comprehensive study on the performance of different learning algorithms for training the ANN and their impact on the outcome of the forecasting is essential, in the present work, merely the Levenberg Marquardt algorithm (LMA) is used. And the comprehensive comparative study of learning algorithms would be performed in future works. Fig 1: Typical structure of a feedforward neural network II. DATA COLLECTION The efficiency of the proposed method is assessed on a set of data from ABEI, Aberdeen, Idaho, United States weather station available on the Internet. This station is chosen merely based on the availability of the data to the public. Hourly wind speed data of two months including 1420 targets are used. Another set of three day data set is used for an out of sample testing. The testing part of data set will be merely used for assessing the performance of the neural network which guarantees that only unseen data is used to evaluate the prediction performance of the network. Additionally, this could also help and guarantee a fair comparison for future studies when all the proposed algorithms could be tested on the same out-of-sample data set. B. Network performance evaluation A quantitative approach is required to test the performance of the model and compare it with existing forecasting methods. Among various indicators, mean squared error (MSE) is chosen for evaluating the performance and efficiency of the proposed algorithm. MSE indicators, presented in (1), is a statistical tool that have been widely used in previous studies. 1 In the above equation, the average of squared error of all observations is calculated. Logically, the lower the MSE the better the performance of the forecasting model. In Equation (1), the number of samples is n while E i and M i are representing the actual value of the time series and the forecasted one respectively. The MSE gives an overall performance measure based on point-by-point comparisons of the actual times series values and the forecasted ones. However, it does not take into account the correlation between the outputs and the targets. Therefore, another performance indicator called regression (R) is implemented in this study as well. R values are representing the correlation between forecasted values and actual time series ones. An R value of 1 indicates that there is a close relationship, while lower values of R, close to zero, show a random relationship. 96
C. Selection of the input variable The input variables are essential elements in the neural network since the selected features would affect prediction accuracy. The input of the ANN is conventionally selected based on a trial-and-error process. Meaning numerous ANNs should be constructed and tested using different number of lags as the input and then based on their performance the better performing ANN would be selected as the forecasting model. To avoid this trial-and-error process, we therefore select our input variables using two statistical measures namely, autocorrelation Function (ACF) and the Partial Autocorrelation Function (PACF). Both of these are statistic tools used for time series analysis. D. WT In this paper, DWT are employed for decomposing the wind speed time series to its constituents. A signal is decomposed, when WT is applied, to a single approximation component and many detail ones. The identity of a signal is stored in the approximation component which contains the low-frequency information while the detail components are revealing the flavor of a signal. The tree of a typical wavelet transform decomposition can be seen in Fig 2. forecasting of the time series. The performance of the proposed algorithm is tested on available hourly wind speed data from a weather station in Idaho, United States. Fig. 3 shows the wind speed hourly data of January and February 2010 of ABIE weather station. Fig. 4 shows the histogram of the data. The wind speed as can be seen is highly fluctuating and its histogram is also has a wide range. A regular observation does not provide any useful information regarding the regulation of the wind speed data. However, deeper statistical look can help better determining the correlation of the available data. The autocorrelation and partial autocorrelation analysis is carried out to determine the lags of historical data which have the highest correlation with the target wind speed. The ACF and PACF analysis is carried out on A1 part of the decomposed signal. It can be observed in Fig. 5 and Fig. 6 for ACF and PACF respectively that lag 1 and lag 2 significantly correlate to the future wind speed and therefore these two time steps are selected as the input of the neural network. Fig. 3. Mean hourly wind speed values of training data Fig 2: Wavelet decomposition process In the early stage, a detailed and approximate components of the available signal, S, are extracted and will be called A 1 and D 1 respectively. The decomposition process could be repeated on the approximation signal, A 1 to achieve another set of approximation and detail component labeled A 2 and D 2 respectively. Further decomposition will allow higher level resolution analysis of the signal. The process will stop when a proper level of levels is achieved. In this study, the original wind speed time series is decomposed to an approximation signal and a detail one. While approximation signal incorporate the main fluctuations of the wind speed, the detail signal contains the spikes and random instabilities of the original signal. IV. RESULTS In this study a forecasting model for short-term wind speed forecasting is developed based on smoothing of the wind speed signal by wavelet transform, selection of the input variables by autocorrelation function (ACF) and partial autocorrelation function (PACF) and an artificial neural network for the Fig. 4: Histogram of wind speed values of training data Fig 5. Autocorrelation of wind speed data with its 72 hour lag for A1 signal 97
TABLE I. THE MSE AND REGRESSION FACTORS FOR THE PROPOSED NETWORK ON BOTH TRAINING AND TESTING DATA SET Training data Testing data MSE 4.34618 2.693995 Regression, R 7.64049e-1 7.80041e-1 Fig. 6. Partial autocorrelation of wind speed data with its 72 hour lag for A1 signal Having chosen the input variables, the number of hidden layers of the ANN is chosen to be 3 based on a trial-and-error process as there is not any systematic way for choosing the number of hidden layers. Next, we divided our data into three sets for training, test and validation. The structure of the employed neural network is shown in Fig. 7. It should be noted that in the current study the correlation of the future wind speed with other weather parameters such as temperature, humidity and etc. is not taken into account and the forecasting is merely based on the historical wind speed data. The trained network is also tested on a set of data for the first three days of March 2010 which includes of 72 targets. Fig. 9 shows the forecasted and actual times series in that period. The wind speed is forecasted for 1 hour head using the previous two wind speeds as the input of the network. Fig. 9. The forecasted time series and the actual one for a period of first three days of March 2010 for AIBE weather station. Fig. 7. The structure of the employed ANN with two inputs, three hidden layers and one output 50% of the initial data is used for training the proposed hybrid neural network. Afterwards, the testing and evaluation are carried out on the rest of the data. The change of the MSE factor in different epochs of the network is shown in Fig. 8. Fig. 10. Histogram of error for testing data The histogram of error, which is the value of target- the value of the forecast, for this test and its autocorrelation with the lags are shown in Fig. 10 and Fig. 11 respectively. Fig. 11 indicates the proper selection of the input variables Fig. 8. The performance of the network for different epochs The MSE and regression factors for both training and test data sets are shown in Table 1. Fig. 11. Autocorrelation of error for the testing data with its lags 98
The performance of the proposed ARIMA-based neural network hybrid with wavelet transform is compared with a regular neural network using the same learning algorithm, number of input variables and hidden layers. The results on the same data is shown in Table 2. The results indicate that using wavelet transform have improved the performance of the neural network in forecasting the 1-hour ahead wind speed. be useful in determining the peak of the electricity consumption in the grids. ACKNOWLEDGMENT This study is funded by internal grant (UNITEN/RMC/1/14-41) provided by Universiti Tenaga Nasional. TABLE II. PERFORMANCE OF A REGULAR NEURAL NETWORK Training data Testing data MSE 5.00105 5.50105 Regression, R 7.64049e-1 7.44412e-1 V. CONCLUSIONS AND DISCUSSIONS In this study, the non-linearity of wind speed is tackled using artificial neural network and its uncertainty by wavelet transform. To avoid trial-and-error process for selection the ANN structure, the results of auto correlation factor (ACF) and partial auto correlation factor (PACF) on the historical wind speed data are employed. Instead of forecasting the time series directly, a set of better-behaved components of the data is achieved by decomposing the data using wavelet transform and are forecasted separately using a feedforward neural network. Finally, using an inverse wavelet transform, the future time series is reconstructed and the wind speed could be forecasted. In this study only one level decomposition is used, however a suitable number of levels should be decided more carefully in the future studies based on the similarity between the approximation and the original signal. This work could also be expanded in several directions. Firstly, the training algorithm for neural network could be improved to achieve better overall performance of the network. Moreover, the performance of the wavelet transform and its effect on the forecasting should be better studied in the future works. Different classes of wavelet transform should be tested to determine their applicability in the area of time series short term forecasting. Additionally, for the future studies, it is recommended that different learning algorithms for ANN weight training to be used and their performance to be compared for a more robust shot-term forecasting of times series including wind speed data. Moreover, additional research should be carried out to determine the hybrid capabilities of the proposed method with the current available forecasting models. It has been suggested in the literature that an ensemble of different forecasting models may result in a better performance since the behavior of the wind speed, especially in the short-term, is volatile and a single forecasting model may not be able to predict the time series. A Bayesian regulation is suggested to be used for handling the ensemble method due to its performance in previous studies [12]. The proposed model for Short-term forecasting of time series could be also used for prediction of any time series in the short term including blood Glucose forecasting for diabetics patients, short-term solar energy forecasting, traffic count forecasting in short-term horizons, wind power forecasting, and power consumption short-term forecasting, which could REFERENCES [1] Foley AM, Leahy PG, Marvuglia A, McKeogh EJ. Current methods and advances in forecasting of wind power generation. Renewable Energy 2012;37:1-8. [2] Zheng XX, Fu Y. Research of wind speed and wind power forecasting. Renewable and Sustainable Energy 2012;347-353(1-7):611-4. [3] Cao Qing, Ewing Bradley T, Thompson Mark A. Forecasting wind speed with recurrent neural networks. European Journal of Operational Research 2012;221:148-54. [4] Erdem E, Shi J. Arma based approaches for forecasting the tuple of wind speed and direction. Applied Eenergy 2011;88:1405-14 [5] Li Gong, Shi Jing. On comparing three artificial neural networks for wind speed forecasting. Applied Energy 2010;87:2313-20. [6] Ortiz-Garcia EG, Salcedo-Sanz S, Perez-Bellido AM, Gascon-Moreno J, Portilla-Figueras JA, Prieto L. Short-term wind speed prediction in wind farms based on banks of support vector machines. Wind Energy 2011;14:193-207. [7] Guo ZH, Zhao WG, Lu HY, Wang JZ. Multi-step forecasting for wind speed using a modified emd-based artificial neural network model. Renewable Energy 2012;37:241-9. [8] Niu Dongxiao, Liu Da, Wu Desheng Dash. A soft computing system for day-ahead electricity price forecasting. Applied Soft Computing 2010:868-75. [9] Jursa René, Rohrig Kurt. Short-term wind power forecasting using evolutionary algorithms for the automated specification of artificial intelligence models. International Journal of Forecasting 2008;24:694-709. [10] Liu Da, Niu Dongxiao, Xing Mian, Nie Qiaoping. Day-ahead price forecast with genetic-algorithm-optimized support vector machines based on garch error calibration. Automation of Electric Power Systems 2007:31-4. [11] Tae Woo Joo, Seoung Bum Kim, Time series forecasting based on wavelet filtering, Expert Systems with Applications, Volume 42, Issue 8, 15 May 2015, Pages 3868 3874 [12] Hooshyar, Danial; Ahmad, Rodina Binti; Fathi, Moein; Yousefi, Moslem; Hooshyar, Maral, "Flowchart-based Bayesian Intelligent Tutoring System for computer programming," in Smart Sensors and Application (ICSSA), 2015 International Conference on, 26-28 May 2015. Pages 150-154. 99