Training Algorithms for Recurrent Neural Networks

Transcription

1 raining Algrithms fr Recurrent Neural Netwrks SUWARIN PAAAVORAKUN UYN NGOC PIEN Cmputer Science Infrmatin anagement Prgram Asian Institute f echnlgy P.O. Bx 4, Klng Luang, Pathumthani AILAND Abstract: -Recurrent neural netwrks (RNNs), in which activity patterns pass thrugh the netwrk mre than nce befre they generate an utput pattern, can learn extremely cmplex tempral sequences. In this paper, three imprtant architectures f RNNs were described, alng with five existing training algrithms ne prpsed. An empirical study was made t evaluate the perfrmance f the frecasting mdels based n these netwrks the algrithms cnsidered, using the daily clsing stck prices f five prminent cmpanies listed n the Securities Exchange f hail. Frm the simulated results, ne may cnclude that gd frecasting mdels can be based n RNNs, the prpsed algrithm can perfrm very satisfactrily in terms f bth the frecast accuracy the cmputatin time. Key-Wrds: - Recurrent Neural Netwrks, Fully Recurrent Neural Netwrk, Lcally Recurrent Neural Netwrks, Efficiency Index, Stck Price Frecasting. 1 Intrductin Feedfrward neural netwrks are universal functin apprximatrs gd at detecting nnlinearies, but suffer frm lng training time a very high number f alternatives as far as architectures parameters g. hey are als prne t verfitting data. Recurrent neural netwrks (RNNs), in which the input layer s activity patterns pass thrugh the netwrk mre than nce befre generating a new utput pattern, can learn extremely cmplex tempral patterns. Several researchers have cnfirmed the superirity f RNNs ver feedfrward netwrks when perfrming nnlinear time series predictin [1, 2] fr an applicatin f RNNs t recgnitin f stck price patterns. RNNs require substantially mre cnnectins mre memry in simulatin than stard backprpagatin netwrks. RNNs can yield gd results because f the repetitin f similar patterns present in stck price time series. One f the main reasns fr being interested in RNNs is that there are new prpsed algrithms which allw them t learn hw t interact with an envirnment in an apprpriate way. he architecture f a predictr underpins its capacity t represent the dynamic prperties f the input signal hence its ability t frecast sme future value. rever, the stck market data nrmally react rapidly t the interventin f human being. his study fcuses n the selected training algrithms fr ppular RNN structures fr the predictin f the prices f the stcks f imprtant cmpanies listed n the Stck Exchange f hail. 2 Recurrent Neural Netwrks A frecasting mdel based n an RNN invlves the cnstructin f tw separate cmpnents : ne r mre recurrent layers (t prvide the tempral cntext, referred t as shrt-term memry) a predictr (usually the feedback part f the netwrk). his study cnsiders three variatins f the RNNs cnsisting the fully RNN tw lcal feedback netwrks, namely the Narendra-Parthasarathey Franscni-Gri-Sda netwrks. 2.1 Fully Recurrent Neural Netwrks (FRNNs) Shwn in Fig.1 is a typical FRNN where all the utputs f the existing neurns are used fr feedback its neurns are fully cnnected. he simplest memry element is the unit time delay, which has the transfer functin, (Z)= Z.

2 3 raining Algrithms he bjective f a training algrithm is t update the weights cnnecting t every nde f the utput units in rder t reach the minimum errr f the system. Fig. 1 Fully recurrent neural netwrk 2.2 Narendra-Parthasarathy (N-P) Netwrk his is a partial cnnectin with feedback links frm each nde f the utput layer t all hidden ndes [3]. In this netwrk, utput values are fed back t the netwrk as inputs act as shrt-term memry. Fig. 2 A Narendra-Parthasarathy recurrent netwrk 2.3 Franscni-Gri-Sda (FGS) Netwrk his is a lcally recurrent netwrk, where a multilayer perceptin is augmented with lcal feed back arund each hidden nde [4]. A ptential advantage f this netwrk is that it can adapt its internal representatin. Fig. 3 A Frascni-Gri-Sda lcally recurrent netwrk 3.1 Ntatin In a FRNN, m(n)-p-r tplgy, let y(k) r y k dente the n-tuple f utputs f the units in the netwrk at time k, let u(k) r u k dente the m-tuple f external input signals t the netwrk at time k. Each neurn in bth the hidden utput layers has a feedback lp frm the neurn itself the recurrent links frm ther neurns. distinguish the cmpnents f y representing unit utput frm thse representing external input values, let O dente the set f indicies i such that y i is the utput f a unit in the netwrk I dente the set f indicies i fr which y i is an external input. y i (k) = u i (k), i I = 1, 2,.., m y i (k) i O = 1, 2,.., n Let w ij dente the weight n the cnnectin t the i th unit frm either the j th unit, if j O r the j th input unit if j I. Let n = p + r w ij be the weight f link frm nde j t nde i, where, i = 1,.., r, r+1,, n = r+p; j = 1,., r, r+1,, n = r+p, n+1,, n + m. - N-P Netwrk: the algrithms fr N&P netwrk are the same as fr FRNN, but when updating the weights, we set w ij = 0, i = 1, 2,., n j = 1, 2,.,p. - FGS Netwrk: the algrithms fr FGS netwrk can be btained frm thse fr the FRNN, but when updating the weight, we set w ij = 0, i = 1, 2,., n j = 1, 2,.,m. 3.2 Basic Algrithms Y-N Algrithm: his algrithm, btained by Yamamt Nikifruk [5], incrprates an Errr Back Prpagatin (EBP) methd fr btaining a fictitius target signal f utput f the hidden ndes. he weight parameters are btained by an Expnentially Weighted Least Squares (EWLS) methd. A-P Algrithm: Atiya Parls [6] btained this algrithm by apprximating the errr gradient. his is an nline algrithm which assumes a small change in the netwrk state finds the directin f weight change by apprximatin. When minimizing the errr, the errr f hidden nde is equal zer, the errr f utput nde is the difference

3 between desired mdel utput. he determinatin f the change in the weights will be accrding t the change in netwrk state. he basic idea f the A-P algrithm is t interchange the rles f the netwrk states y(k) the weight matrix W. he states are cnsidered as the cntrl variables, the change in weights is determined accrding t the change in y(k). First dified Y-N Algrithm, An nline training algrithm was btained by cmbining three techniques, namely Errr Back Prpagatin (EBP), Recursive Least Squares (RLS), Errr Self Recurrent (ESR). his algrithm, dented YNC t recgnize the wrk f Chairatanatrai [7], updates weights bth in hidden ndes utput ndes in the same prcedure while the Y-N algrithm updates the weights f hidden ndes utput ndes separately. rever, the errrs calculated frm the utput unit are fed back fr determining weight updates f utput unit ndes. his part is called ESR. Secnd dified Algrithm he algrithm, dented YNS [8], is btained by cmbining the methds f weight updating f the A- P algrithm the Y-N algrithm techniques t enhance the netwrk perfrmance. Firstly weight cnnectins f the hidden ndes are adapted befre weight cnnectins f utput ndes in rder t generate net signal fr each hidden nde. Secndly, the netwrk uses these signals t adjust weight cnnectins f utput ndes. Frward Prpagatin Algrithm Als called real time recurrent learning (RRL) [9], this algrithm where the weights are updated in an n-line fashin. he gradients f the netwrk states with regard t the weights at time instant k are btained in terms f thse at time instant k. Once these are evaluated, the errr gradients can be btained in a straightfrward way. Prpsed Algrithm he algrithm is btained by cmbining the methds f weight update f the A-P algrithm (t find the directin f weight change by apprximatin), the Y-N algrithm (t estimate fictitius target signals f hidden ndes in rder t update hidden weight separately frm utput weights), then by adding the errr self-recurrent (ESR) netwrk t speed up the cnvergence. he prpsed algrithm, dented YN-, can be summarized as fllws: 1. Assume that data sets (y(1), d(1),..., y( ), d( )) is given. 2. Initialize the value f t = 1, let dente sme mathematical ntatins in rder t find weight changes as fllws γ () = -D ( )e () + We ( ) (1) B() = γ (t)y (t ) (2) t = 1 Because f ill-cnditining prblems, Eq.3 must be added with the uter prduct matrix by a small matrix ε I, where ε is a small psitive cnstant. V() = εi + = y(t)y (t) t Updated weights f hidden ndes are expressed by W = ηb 0 ()V () (3) (4) W (new) = W (ld) + W (5) hen, all new hidden utputs are calculated based n the updated hidden weights. y (t) = W y (t) = (t)y(t ) (y (t), y (t), u(t) ) (6) Finally, updated weights f utput ndes are expressed by W = ηb ()V () (7) W (new) = W (ld) + W (8) 3. Calculate a vectr f errr self-recurrent netwrks t be used as the errr signals fr utput hidden ndes. se(t) = e(t) + µ e(t) where se(t) = [se (t), se h (t),..,se (n) (t)], n-tuples are a number f utput units, µ is a cnstant f the time delayed errrs called ne-step previus errr, x (t) s (t) e(t) = x h (t) s h (t) 4. ime step at t = t+1, calculate the mdel utput. y(t) = f(wy(t )) (9) hen, cmpute the errr signal e (t) by ESR methd then using these signal in rder t generate as in Eq. 1 γ(t) 5. Find weighted changes f hidden ndes, W (t)

4 W (t) = W (t ) + γ (t)y (t )V (t η V ) - B (t )V (t )y(t ) 1 + y (t )V (t )y(t ) [ V (t )y(t )] [ V (t )y(t )][ V (t )y(t )] (10) (t) = V (t ) - 1+ y (t )V (t )y(t ) (11) B (t) = B (t ) + γ (t)y (t ) (12) 6. All new hidden utput signals are calculated based n the updated hidden weights. y (t) W (t)y(t ) = y (t) = (y (t), y (t), u(t) ) (13) 7. Find weighted changes f utput ndes, W (t) W (t) = W (t ) + γ (t)y (t ) η V (t) = V B V (t ) y (t ) V (t )y (t ) (t ) - B (t )V (t )y (t ) [ V (t )y (t )] [ V (t )y (t )][ V (t )y (t )] 1+ y (t ) V (t) = B (t ) + γ (t)y (t ) (t )y (t ) (14) (15) (16) 8. Repeat steps 3, 4, 5, 6 7 until the end f given data. erminating f training is terminated when the system errr has reached an acceptable criterin. Nte that set W W t zer at the beginning f next iteratin. - Prperty Cnstructin/Building furnishing materials sectr: Siam City Cement Public Cmpany Limited (SCCC). Daily data n the stck prices vlumes in the market frm 01 July 1999 t 30 June 2002 were used fr training phase 01 July June 2003 fr testing phase. he stck market is clsed n weekends, s the cnsecutive data series are frm ndays t Fridays. he frecasting equatin[10] with lead time f ne day can be expressed as: k P V i P(t+1) = g(p(t), P(t), P(t-2), V(t), V(t), V(t-2)) + h(p i (k), P p (k), P (k)) = time step f the netwrk = stck price in baht = stck vlume in unit = hidden nde i ( i = 1, 2, p). = utput nde Befre being presented t the RNN, the data are transfrmed by a linear affine transfrmatin t the range [0.05, 0.95] using the equatins: [ 0.9(y - y )/(y - y )] 0.05 y = (17) t t min max min + [(y max - y min )(y t )/0.9] y min y t = + (18) where y t y t dente the transfrmed riginal data, ymax ymin are the maximum minimum f the riginal data in the calibratin phase. A stpping rule fr indicating the cnvergence f the algrithms cnsidered. It is based n the relatinship between the sum f squared errrs f tw cnsecutive iteratins, it is as fllws: 4 Applicatins Discussins SE(t + 1) - SE(t) SE(t) ε (19) 4.1 Data Emplyed Stck prices data in hai arket data were btained frm he Securities Exchange f hail (SE). Five imprtant cmpanies were selected: - Real Estate Sectr: L use Public Cmpany Limited (L) - Banking Sectr :Bangkk Bank Public Cmpany Limited (BBL) Kasikrn Bank Public Cmpany Limited (KBANK), - Cmmunicatin Sectr: Shin Crpratin Public Cmpany Limited (SIN), where SE(t+1) is the Sum f Squared Errrs at iteratin (t+1), ε is a small number, set equal t in the present study. measure the accuracy f frecasting values, the efficiency index (EI) prpsed by Nash Sutcliffee [11] can be used. SE EI = 1 (20) S 2 2 S = (yi - y), SE = = (yi - ŷ i ), i = 1 i 1

5 y = 1 i = 1 y i where S = tal variatin, SE = Sum f squared errrs, y i = Actual utput, i.e. bserved value at time i, y = ean value f the actual utput, yˆ i = del utput, i.e. frecast value at time i, = Number f data pints A value f EI clse t 1.0 indicates an excellent perfrmance in terms f the predictin accuracy. 4.2 Experimental Cases Cnditins he training algrithms n the three RNN mdels are evaluated in terms f the cmputatin time the frecast accuracy expressed by EI. Fr these data sets, the 6(3)-2 tplgy is used. his tplgy has 6 external inputs (3 inputs fr stck price, 3 inputs fr the vlume), 3 feedback lines frm hidden ndes ne utput nde. All feedback cnnectin weights were set equal 1.0. he sigmid slpe f sigmid functin was set equal t 1. he same initial weights in the range f [0, 1] were used Fr the Y-N algrithm, the frgetting factr (β) was set at 0.99, Fr the A-P YNS algrithms, parameter η was set at 0.009, parameter ε at 13.0 fr L, KBANK SIN at 8.5 fr BBL SCCC, Fr the YNC algrithm, the bias f ne-step previus errr (µ ) was set at 1.0, Fr the RRL algrithm, the learning rate (η) at 0.003, Fr the YN- algrithm, µ was set at 0.3, η at ε at 13.0 fr L, KBANK SIN at 8.5 fr BBL SCCC. 4.3 Results Discussins he values f the efficiency index btained in the training phase are cllected in able 1 thse f the cmputatin time in able 2. able 1. Cmputed values f efficiency index fr stck prices in hai market Stck FRNN Cmpany Y-N A-P YNC YNS RRL YN- L BBL KBANK SIN SCCC Stck N-P Cmpany Y-N A-P YNC YNS RRL YN- L BBL KBANK SIN SCCC Stck FGS Cmpany Y-N A-P YNC YNS RRL YN- L BBL KBANK SIN SCCC Frm able 1, bth the N-P FGS netwrks prduce the values f EI less than thse prduced by the FRNN. his may be due t less infrmatin fr capturing the variability (less weight cnnectins). - - Amng six algrithms, the perfrmance f the RRL algrithm, in terms f frecasting accuracy, is the lwest, while the YN- algrithm appears t be the best. he Y-N,YNC, YNS YN- algrithms result in quite high values f EI with the FRNN mdel, but slightly lwer with the N-P FGS netwrks. he prpsed algrithm, YN- appears t perfrm very well fr all netwrk architectures. - In view f the cmputatin time (able 2), the prpsed algrithm als perfrms very well. It is the least time cnsuming amng the algrithms cnsidered. It shuld be nted that the values used fr the parameters f the different algrithms d have great influence n the time cnsumptin. he different algrithms have different sensitive parameters which must be tuned first befre training the netwrks. In this study, the apprpriate parameter values f each algrithm were chsen fixed, then the netwrks were trained many times with different initial weights in the range frm 0 t 1. Afterwards, the same initial weights were fixed fr training all the netwrks.

6 able 2. Cmputatinal time (in secnds) Stck FRNN Cmpany Y-N A-P YNC YNS RRL YN- L BBL KBANK SIN SCCC Stck N-P Cmpany Y-N A-P YNC YNS RRL YN- L BBL KBANK SIN SCCC Stck FGS Cmpany Y-N A-P YNC YNS RRL YN- L BBL KBANK SIN SCCC Fr each algrithm, bth N-P FGS netwrks require lwer cmputatin time than the FRNN des. his is due t the fact that these tw netwrks have simpler architectures with limited feedback lps. As a trade-ff, these netwrks d nt prduce high values fr the efficiency index as mentined earlier. 5 Cnclusins A brief descriptin f three recurrent netwrk architecture was given in this study. It is fllwed by a brief descriptin f five existing training algrithms ne prpsed in this study. By an empirical study using the stck data frm five imprtant cmpanies listed n the Securities Exchange f hail, the fllwing cnclusins may be drawn: - Frecasting shrt-term stck prices based n the histrical daily clsing prices is pssible with the use f RNN mdels. - he prpsed algrithm appears t perfrm very well in terms f bth the frecast accuracy the cmputatin time. As such, it can be used fr training these RNN mdels. References: [1] K. Kamij, &. anigawa, Stck Price Pattem Recgnitin : A Recurrent Neural Netwrk Apprach. Prceedings f the Internatinal Jint Cnference n Neural Netwrks. 1990, [2] C.L. Giles, S.Lawrence & A.C. si, Rule Inference fr Financial Predictin using Recurrent Neural Netwrks, IEEE Cnference n Cmputatinal Intelligence fr Financial Engineering, IEEE Press, 1997, pp.253. [3] K.S.Narendra, & K. Parthasarathy, Identificatin Cntrl f Dynamical Systems using Neural Netwrks, IEEE ransactins n Neural Netwrks, Vl 1, N. 1, 1990, pp [4] P. Franscni,. Gri, & G. Sda, Lcal Feedback ultilayered Netwrks Neural Cmputatin, Vl. 4, N.1, 1992, pp [5] Y. Yamamt, & P.N. Nikifruk, A Learning Algrithm fr Recurrent Neural Netwrks its Applicatin t Nnlinear Identificatin, Prceedings f the 1999 IEEE Internatinal Sympsium n Cmputer Aided Cntrl System Design, awai, USA., August, pp [6] A.F. Atiya, & A.G. Parls, New Result n Recurrent Netwrk raining : Unifying the Algrithms Accelerating Cnvergence, IEEE ransactins n Neural Netwrks, Vl.11, N.3, pp [7] A. Chairatanatrai, A Cmparisn f Selected raining Algrithms fr Recurrent Neural Netwrks, aster hesis., Asian Institute f echnlgy, Pathumthani, hail, [8] S. Pattamavrakun &.N. Phien, A Futher Study n Recurrent Neural Neural Netwrks fr Frecasting, Prc.Cnf. he 8 th Wrld ulti- Cnference n Systemics Cybernetics Infrmics, USA, ( appear) [9] R.J. William, & P.Zipser, A Learning Algrithm fr Cntinually Running Fully Recurrent Neural Netwrks Neural Cmputatin, Vl. 1, N.12, 1989, pp [10] N.. Danh,.N.Phien & A.D. Gupta, Neural Netwrk dels fr River Flw Frecasting, Water SA, Vl.25 N.1, 33-39, [11] J.E. Nash, & J. V. Sutcliffee, River Flw Frecasting thrugh Cnceptual dels, Jurnal f ydrlgy, Vl. 10, 1970, pp