Multi-Step-Ahead Prediction of Stock Price Using a New Architecture of Neural Networks

Transcription

1 Journal of Computer & Robotcs 8(), Mult-Step-Ahead Predcton of Stoc Prce Usng a New Archtecture of Neural Networs Mohammad Taleb Motlagh *, Hamd Khaloozadeh Department of Systems and Control, Industral Control Center of Excellence, K.N.Toos Unversty of Technology, Tehran, Iran Receved December 04; accepted 8 January 05 Abstract Modellng and forecastng Stoc maret s a challengng tas for economsts and engneers snce t has a dynamc structure and nonlnear characterstc. Ths nonlnearty affects the effcency of the prce characterstcs. Usng an Artfcal Neural Networ (ANN) s a proper way to model ths nonlnearty and t has been used successfully n onestep-ahead and mult-step-ahead predcton of dfferent stocs prces. Several factors, such as nput varables, preparng data sets, networ archtectures and tranng procedures, have huge mpact on the accuracy of the neural networ predcton. The purpose of ths paper s to predct mult-step-ahead prces of the stoc maret and derve the method, based on Recurrent Neural Networs (RNN), Real-Tme Recurrent Learnng (RTRL) networs and Nonlnear Autoregressve model process wth exogenous nput (NARX). Ths model s traned and tested by Tehran Securtes Exchange data. Keywords: stoc prce; neural networ; predct; mult-step-ahead predcton.. Introducton Due to the absence of accurate nformaton nfluencng the stoc prce, lnear regresson models such as ARIMA models are unable to provde a useful predcton of prce. After 980, there has been a turnover n the feld of tme seres predcton, when t became clear that ths type of predcton s a sutable applcaton for an ANN. Over the last few decades, ANNs have become one of the successful technques that are beng used for modellng stoc maret behavour. In stoc tradng, t s very desrable that the models provde an accurate mult-step-ahead predcton. A smple neural networ s unable to provde a relable long-term predcton []. Lately, RNNs have attracted much attenton for multstep-ahead predctng. Due to ther dynamc nature, RNNs have been appled successfully to a wde varety of problems, such as fnancal tme seres [], pea power load forecastng [3] and stream-flow forecastng [4]. An archtectural approach of RNN wth embedded memory, NARX networ, shows promsng qualtes for dynamc system applcatons. Studes show that * Correspondng author. Emal: m_taleb@outloo.com

2 48 M. T. Motlagh et al. / Mult-Step-Ahead Predcton of Stoc Prce Usng a New Archtecture of Neural Networs NARX networs are often much better at dscoverng long term dependences than RNNs [5]. Most of the presented neural networs are statc, whch can only smulate the short-term memory structures wthn process; consequently some characterstcs could not be modelled well. The RTRL networs, whch represent dynamc nternal feedbac loops to store nformaton for later use and to enhance the effcency of learnng, provde a better way for modellng [6].. Neural Networs The basc dea for forecastng s usually the case that addtonal nformaton from prevous observed values and model outputs wll be useful to forecast the future values. When a neural networ s used to forecast, results tend to be closer to the true values, as the model can be teratvely adusted through model performance wth a reducton of errors... RNN Networs To predct the future behavour of a process, t s necessary to model ts dynamcs. RNNs are capable of representng arbtrary nonlnear dynamcal systems. The RNN shown n Fg. s used n [] for longterm predcton of stoc maret. Ths networ has an archtecture, whch s based on the error vector as an auxlary nput vector. In ths networ, each element of the output vector represents the daly prce whch s to be predcted. In fact, the predcted values of prce for several days ahead are estmated usng the avalable prce daly data. In the tranng procedure of ths networ, the prce values of the next T days are predcted usng prce values of the last N days and actual error of estmaton for the next T days. In the test procedure the error vector s consdered zero snce the actual prce of the next T days are not avalable. x x n x n x n RTRL Networs Fg.. RNN for long-term predcton xˆ xˆ Another RNN as shown n Fg. whch has used RTRL algorthm, s appled n [6] for two-step-ahead predcton of stream flow. Ths networ has a representaton of dynamc nternal feedbac loops to store nformaton for later use. Ths would enhance the effcency of learnng. As ndcated n the archtecture of ths networ, the outputs of the hdden layer are fed bac to the nput of the networ. In aforementoned RNN networ, all layers perform a smultaneous mappng process x t y t z t. However, n ths networ, each layer perform a onestep-ahead mappng process x t y t z t..3. NARX Networs NARX s a useful mathematcal model of dscretetme nonlnear systems. Nonlnear systems can be approxmated by a NARX networ, whch s a powerful dynamc model for tme seres predcton. ( Y () t f X ( t DX ),, X ( t ), X ( t), Y ( t DY ),, Y ( t ) X t and Where, Y t represent the nput and output of the networ at tmet, DX and DY are the nput and output orders, and the functon f s a nonlnear functon. The archtecture of a NARX networ based on a multlayer perceptron neural networ, conssts of DX antecedent values of ) xˆ ()

3 Journal of Computer & Robotcs 8(), exogenous nput vectors X t ; DY prevous actual values Y t, whch are tapped-delay nputs or fed bac from the model s output; and a sngle n-stepahead output Y t n. The NARX model for mult-step-ahead predcton, can be mplemented n many ways, but the smpler, seems to be by usng a feedforward neural networ wth the embedded memory, as shown n Fg. 3 and a delayed connecton from the output of the second layer to nput. Ths networ s used n [5] for predcton of chaotc tme seres. z(t+) K y(t+) Output layer 3. The proposed archtecture In ths secton a new archtecture of neural networ, as depcted n Fg. 4, s presented for twostep-ahead predcton of stoc prces. An earler verson of the proposed archtecture was presented n [8]. As the archtecture shows, outputs of the hdden layer and output errors are consdered as addtonal nputs. When the output errors are appled as nput as n [], the networ tends to learn the error dynamcs durng the tranng procedure. In the valdaton and test procedure, snce the output errors are not avalable, they are consdered zero. When the outputs of the hdden layer are fed bac, the networ would mantan a sort of state that could perform such tass as sequence predcton whch are beyond the power of a standard multlayer perceptron. It also enhances the effcency of learnng. N Processng layer 3.. Data Normalzaton u() u( ) N z z y(t) Tme delay M Input x(t) Input layer Fg.. RTRL for two-step-ahead predcton u( ) b y( ) z y(+) Ths process helps to normalze the mum and maxmum values of each data seres to a boundary. In ths case because of the nature of the sgmod functon, whch s used as the transfer functon of the networ, the nput data need to be n,. Logreturn s used for normalzng here. The ratonale s that the movement of the prce value s more mportant than absolute value of the movement, whether t s gong up or down and how much the movement s n percentage. The defnton of logreturn s: y( ) y() z z X normalzed log X t t X () Fg. 3. Archtecture of the R-NARX networ durng tranng and testng phases n the recurrent mode. Ths fact has been observed prevously, n the sense that gradent-descent learnng s more effectve n NARX networs than n RNN networ archtectures [7]. 3.. Networ topology There are nfnte ways to construct a neural networ. The archtecture of a neural networ defnes ts geometrcal structure.

4 50 M. T. Motlagh et al. / Mult-Step-Ahead Predcton of Stoc Prce Usng a New Archtecture of Neural Networs x x ( t ) x n () t v ( t ) v n () t x () n t () nk t y ( t) y J () t z ( t ) + z K + () t yt ( ) y ( ) J t Fg. 4. Proposed networ for mult-step-ahead predcton z ( t ) z ( ) K t z ( ) K t The followng factors have to be consdered n desgnng a proper neural networ: The number of nputs: n ths networ the followng matrx s used as nput X X V E Y (3) n n where, X n and V n are the normalzed closng prce and normalzed tradng volume, E s the actual error of estmaton as used n [] and Y s the output of the hdden layer. The prce and the volume of the two past days are used as nput. The number of hdden layers: Theoretcally, a neural networ that has one hdden layer and suffcent number of hdden neurons s capable of generalzng any contnuous functon [9]. In ths networ one hdden layer s used. The number of neurons n each layer: In order to fnd the optmal number of neurons, the networ s performed for dfferent numbers of neurons. The number of outputs: Snce the goal s twostep-ahead predcton, the networ has two outputs Tranng Procedure The process of tranng a neural networ s to let the networ learn patterns n the tranng data. The goal of ths process s to fnd the best set of weghts and bases. For ths networ, gradent-descent T bacpropagaton has been appled, to fnd the desred set of weghts and bas. 70% of the data are used for tranng, 5% for valdaton and 5% for testng. Hyperbolc tangent functon s used as the transfer functon of the hdden layer and purln s used for the output layer. The same method as used n [] and mentoned n.. s used here for tranng and testng. Fg. 4 shows the proposed networ for two-stepahead predcton. Specfyng how to adust weghts and bases, s the man ssue n the tranng procedure of ths networ. Snce the output of the hdden layer and the output layer are fed bac to the nput, the adustment formulas of the weghts and the bases should be recalculated for ths networ. Let A denote the set of ndces for whch X () t s the stoc prce, B denote the set of ndces for whch Y () t s the volume of the stoc trade, C denote the set of ndces for whch e () t s the error of the networ output and D denote the set of ndces for whch Y () t s the output of the hdden layer of the networ. Thus the nput s defned by t x t A v t B e t C y t D The net actvty and the output of the neuron and neuron are computed by net t w t t b t ABCD y t f net t (4) (5) ABCD net t v t y t b t z t f net t Where,,, respectvely represent the ndex of the each nput, each neuron of the hdden layer and

5 Journal of Computer & Robotcs 8(), each output. The error of the predcton for th day at tme t s e t d t z t (6) Where, d t s the desred output at tme t. Then the networ error at tme t and the total networ error are E t e t total T t E E t Snce the gradent descent s used for bac propagaton, the weghts and bases should be adusted as follows t t v t w b t b b v E t v E t w w E t b Where,,, v w t Et b t b t b and t (7) (8) b are the learnng rates of the weghts and bases of the layers. The partal dervatve terms can be obtaned by the chan rule for dfferentaton. For v we have E t v t v t e e e t t t t v t t t f net t e t v t e t For w we have E t w t w t t t t e v z v e net v f net t y t f net t y t e { e t t t t f net t t t e e w z w } t e t v t y w t t (9) (0) It could be assumed that the synaptc weghts do not rely on the ntal state of the networ at tme t 0, so we have

6 5 M. T. Motlagh et al. / Mult-Step-Ahead Predcton of Stoc Prce Usng a New Archtecture of Neural Networs 0 0 y 0 w y t net t f net t w t w t net t w t w t ABCD w t w t w w t 0 m & n w t other t t t () The value of m s, f and only f m ; otherwse 0. w t t t 0 0 e w y w t t t t e y t v t f net t w t w t A B C D m n () Snce s a member of C set, then s equal to,,..., K, where K s the total number of the outputs. Let a t t t y t y t a t w t w t y t y t a t w t w t y w (3) Thus a t can be calculated by the followng recursve equaton D [ w tv t a t f net t t C m D a t f net t w t a t Wth the ntal condton a (0) 0. ] (4) Snce s a member of D set n the term w t a t, then s equal to,,...,j, where s the total number of the neurons n the hdden layer. Then the weght adustment for computed by ( w t e t v t w ) f net t a t w t can be (5) Smlarly bas adustment relatons are obtaned by b ( b t e t f net t b (6) f net t ) a t b t e t v t However, for the bases, a t wll be obtaned by a t f net t [ C m D w t v t ] a t f net t w t a t (7)

7 Journal of Computer & Robotcs 8(), Applcaton and Results In order to verfy the performance of proposed networ, an experment has been carred out on real stoc data. The networ s appled to predct two-stepahead closng value of Saderat Ban exchange (BSDR) form June 0th of 009 to October 5th of 05. For comparson, RNN networ, NARX networ and RTRL networ are also performed. Mean Square Error (MSE), Root Mean Square Error (RMSE), Normalzed Mean Square Error (NMSE) and Mean Absolut Error (MAE) are used to compare the predcton precson of these models. The defnton of these error ndces are gven by N MSE z N d N MAE z N d N RMSE z d N z d NMSE d (8) Where, N represents the total number of data ponts, z s the predcton result of th day, d s the desred output of th day and e s the estmaton error. The smaller the value of these ndces, the more precse the predcton model wll be. Daly prce of Saderat Ban exchange form June 0th of 009 to October 5th of 05 s used for smulaton. Fg. 5 shows the closng prce and the tradng volume chart of BSDR for the gven date range. Fg. 5. BSDR daly closng prce and tradng volume 4.. RNN Networ The same networ used n [] whch s presented n II.A s used here wth nputs, hdden layer, 3 neurons n the hdden layer, output neurons, hyperbolc tangent functon as the transfer functon of the hdden layer and the output layer. 4.. NARX Networ A NARX networ as descrbed n.3. s used here wth days prce as nput, hdden layer wth 3 neurons, outputs, hyperbolc tangent functon as the transfer functon of the hdden layer and the output layer. Both nput and output orders ( DX, D Y ) are set to Proposed Archtecture The proposed networ s confgured as descrbed n III. The tranng procedure s performed n 800 epochs n all four of the networs Smulaton Results The networ s performed for dfferent values of learnng rate and as mentoned n 3.. for dfferent neuron numbers, n order to fnd the best parameters. Table shows the networs MSE and MAE errors for dfferent neuron numbers and dfferent learnng rate values. Ths table ndcates that the optmal values of these two parameters are 3 and 0..

8 54 M. T. Motlagh et al. / Mult-Step-Ahead Predcton of Stoc Prce Usng a New Archtecture of Neural Networs Table MSE and MAE errors of the networ for dfferent neuron numbers ( ) and dfferent learnng rate ( ) MAE MSE MAE MSE MAE MSE MAE MSE The performance of the four networs s evaluated by MSE, MAE, RMSE and NMSE. Table shows the performance of the four explaned networs. Accordng to the results, all four of the crterons ndcate that the proposed networ performs better than the other networs. Fg. 6 represents one-step-ahead and two-stepahead predctons of the proposed networ for BSDR. Real prce s shown by green lne, one-step-ahead predcton s shown by red dashed lne and two-stepahead predcton s shown by blue dashed lne. Table Predcton error of dfferent networs Proposed networ RNN NARX RTRL MSE MAE RMSE NMSE

9 Journal of Computer & Robotcs 8(), As the results ndcate, the proposed networ has a better performance and could be effectve n modelng such dynamc system. 5. Conclusons Decson mang under uncertanty problems, such as portfolo optmzaton, pea power load and flood warnng systems, whch have a dynamc structure, requre a proper mult-step-ahead predcton. In ths paper a method for two-step-ahead predcton has been proposed, whch could be extended for more future tme steps. For comparson, two-step-ahead forecastng by usng RNN networ, NARX networ and RTRL networ were also performed. The results showed that the proposed archtecture outperforms the aforementoned networs and yelds a predcton wth less error. References Fg. 6. One-step-ahead and two-step-ahead predcton of the proposed networ [] H. Khaloozadeh, A. K. Sedgh, 00. "Long term predcton of Tehran prce ndex (TEPIX) usng neural networs". In IFSA World Congress and 0th NAFIPS Internatonal Conference. Jont 9th (Vol., pp ). IEEE. [] E. W. Saad, D. V. Prohov, D. C. Wunch, 998. "Comparatve study of stoc trend predcton usng tme delay, recurrent and probablstc neural networs". IEEE Transactons on Neural Networs 9(6): [3] H. C. Huang, R. C. Hwang, J. G. Hseh, 00. "A new artfcal ntellgent pea power load forecaster based on non-fxed neural networs". Internatonal ournal of electrcal power & energy systems, 4(3), [4] P. Coulbaly, F. Anctl, B. Bobee, 00. "Multvarate reservor nflow forecastng usng temporal neural networs". Journal of Hydrologcal Engneerng 6(5): Fg. 7. Two-step-ahead predcton of the four networs Fg. 7 represents two-step-ahead predcton of the networs. [5] E. Daconescu, 008. "The use of NARX Neural Networs to predct Chaotc Tme Seres". WSEAS Transactons on Computer Research 3(3):8-9. [6] L. C. Chang, F. J. Chang, Y. M. Chang, 004. "A twostep-ahead recurrent neural networ for stream-flow forecastng". Hydrologcal Processes, 8(), 8-9.

10 56 M. T. Motlagh et al. / Mult-Step-Ahead Predcton of Stoc Prce Usng a New Archtecture of Neural Networs [7] T. Ln, B. G. Horne, P. Tňo, C. L. Gles, 996. "Learnng long-term dependences n NARX recurrent neural networs". Neural Networs, IEEE Transactons on, 7(6), [8] M. T. Motlagh, H. Khaloozadeh, 06. "A new archtecture for modellng and predcton of dynamc systems usng neural networs: applcaton n Tehran stoc exchange". In 4th Internatonal Conference on Control, Instrumentaton and Automaton (ICCIA), 06. [9] A. S. Chen, M. T. Leung, H. Daou, 003. "Applcaton of neural networs to and emergng fnancal maret: forecastng and tradng the Tawan stoc Index". Computers and Operatons Research, vol. 30.