Adaptive LRBP Using Learning Automata for Neural Networks

Transcription

1 Adaptve LRBP Usng Learnng Automata for eura etworks *B. MASHOUFI, *MOHAMMAD B. MEHAJ (#, *SAYED A. MOTAMEDI and **MOHAMMAD R. MEYBODI *Eectrca Engneerng Department **Computer Engneerng Department Amrkabr Unversty of Technoogy Hafez Ave. 44, Tehran 594, IRA # Curreny wth depatment of eectrca and computer engneerng at okahoma state unversty. Abstract: - One of the bggest mtatons of BP agorthm s ts ow rate of convergence. In ths paper,varabe Learnng Rate (VLR agorthm and earnng automata based earnng rate adaptaton agorthms are descrbed and compared wth each other. Because the VLR parameters have mportant nfuence n ts performance, we use earnng automata for adustng them. In the proposed agorthm named as Adaptve Varabe Learnng Rate (AVLR agorthm, VLR parameters are changed dynamcay accordng to error changes by earnng automata. Smuaton resuts on a second order dscrete tme nonnear functon approxmaton probem hghght better the mert of the proposed AVLR. KeyWords:Mutayer eura etwork, Backpropagaton,Varabe Learnng Rate, Learnng Automata Introducton Consder Probem of backpropagaton earnng wth batch method. For the purpose of gettng an optma weght vector n an teratve manner, a descent type agorthm was frst deveoped by werbos n 974, and redscovered by Parker n 98 and once agan by Rumehart n 986. Unfortunatey t has been observed that convergence rate of BP s extremey sow, especay for the networks wth more than one hdden ayer. The ntrnsc reason behnd ths s the saturaton property of the actvaton functon used for the hdden and output unts. Once the output of a unt es n the saturaton area, the correspondng descent gradent woud take a very sma vaue f the output error s very arge. Ths w resut n very tte progress n the weght adustment f one takes a fxed sma earnng rate parameter. To avod ths undesred phenomenon, one may consder a reatve arge earnng rate. Ths woud be dangerous, however, because t may ead to dvergence of the teraton especay when the weght adustment happens to fa nto the surface regons wth arge steepness. Therefore an effcent earnng agorthm shoud be abe dynamcay vary ts earnng rate n accordance wth changes of the gradent vaues. Research nto dynamc change of the earnng rate of BP agorthm has been reported n [4]-[6]. Bascay they a dynamcay ncrease or decrease by a fxed factor the earnng rate and momentum based on the observaton of error sgnas. Some other acceeraton methods have aso been presented, ncudng modfcatons of optmzaton crteron and use of second order methods (e.g., the ewton method [7]- [8], the Broyden-Fetcher-Godfarb-Shanno and the Levenberg-Marquardt methods et a. [9]-[]. There are some earnng automata (LA based methods []- [9]. Athough M. L. Tsetn and hs co-workers started work on earnng automaton n the 96s n the Sovet Unon, varabe structure earnng automata (VSLA or fxed structure earnng automata (FSLA have recenty been used to fnd the approprate vaues of parameters for the BP tranng agorthm []-[9]. In ths paper dfferent methods of dynamc changng of earnng rate have been consdered. Varabe earnng rate (VLR agorthm and earnng automata based earnng rate adaptaton agorthms are descrbed and compared wth each other. Because the VLR parameters have mportant nfuence n ts performance, we use earnng automata for adustng them. In the proposed agorthm named as Adaptve Varabe Learnng Rate (AVLR agorthm, VLR parameters are changed dynamcay accordng to error changes by earnng automata. Smuaton resuts on a second order dscrete tme nonnear functon approxmaton probem hghght better the mert of the proposed AVLR. The rest of the paper s organzed as foows: Secton brefy presents the standard backpropagaon agorthm. An ntroducton to earnng automata s gven n secton 3. In secton 4 varabe earnng rate agorthm s descrbed. Secton 5 presents earnng automata based methods. In secton 6 a new agorthm named as adaptve varabe earnng rate s ntroduced. The smuaton resuts are gven n secton 7. Secton 8 concudes the paper.

2 Backpropagaton Agorthm BP s a systematc method for tranng mutayer neura networks. BP agorthm has two computatona phase. Frst phase s forward phase and the socond s backward phase[]. Forward phase: ths phase s descrbed by foowng equatons: a = pk ( a ( = F L ( W ( a + b (, =,,..., L a = a ( Actvaton functon act on a neurons, that s: F + ( n( = + + [ f ( n (... f ( n ( ] T Backward phase: n ths phase, senstvty vectors propagated from output ayer to nput ayer. The foowng equatons descrbe dynamc of backward phase: δ L ( = F.. L ( n e( δ ( = F ( n ( W T e( = t( a( In backward phase frst error vector s computed. Then, the error vector from rght to eft and from output ayer to nput ayer s propagated and oca gradents, neuron by neuron are computed. Parameter adustng: n ths step weght matrces and δ, s + = L,..., bases are adusted as foows. ( T W ( k + = W ( αδ ( a ( b k b k k L ( + = ( αδ (, =,,..., Stopng crteron: f average of error squares n each epoch (sum of square error for a of tranng patterns s smaer than from a predetermned vaue then BP agorthm s stopped. 3 Learnng Automata Learnng automata (LA can be cassfed nto two man groups, fxed and varabe structure earnng automata (FSLA and VSLA [6]-[9]. Exampes of the FSLA type are Tsetne, Krnsky and Kryov automata. A fxed structure earnng automaton s quntupe < α, Φ, β, F, G > where: α = ( α,..., α R s the set of acton that t must choose from. Φ = ( Φ,..., Φs s the set of states. 3 β ={, } s the set of nputs where represents a penaty and a reward. 4 F : Φ β Φ s a map caed the transton map. 5 G : Φ α s the out map The seected acton serves as the nput to the envronment that n turn emts a stochastc response β (n at the tme n. β (n s an eement of β = {,} and s the feedback response of the envronment to the automaton. The envronment penaze (.e., β (n = the automaton wth the penaty c, whch s the acton depent. On the bass of the response β (n, the state of the automaton Φ(n s updated and a new acton s chosen at the tme (n+. The nterconnecton of earnng automata and envronment s shown n fgure. In the foowng sectons, we w descrbe fxed structure and varabe structure earnng automata. α (n {c} Random Envronment Learnng Automaton { Φ, α, β, F, G} β (n Fgure : The nterconnecton of earnng automata and envronment 3. The two-state automaton ( L, Ths automata has two states, Φ and Φ and two actons α and α. The automata accepts nput from a set of {, } and swtches ts states upon encounterng an nput (unfavorabe response and remans n the same state on recevng an nput (favorabe response. State transton are shown n fgure. 3. The two-acton automaton wth memory Ths automata has states and two actons and attempts to ncorporate the past behavor of the system n ts decson rue for choosng the sequence of actons. Whe the automata L, keeps an account of the number of success and faures receved for each acton. It s ony when the number of faures exceeds the number of successes, or some maxmum vaue, the automata swtches from one acton to the another. For every favorabe response, the state of automata moves deeper nto the memory of correspondng acton, and for an unfavorabe response, moves out of t. The state transton graph of L, automata s shown n fgure 3.

3 Favorabe Φ s a set of nterna states, α a set of outputs, P denotes the state probabty vector determnng the choce of the state at each stage k, G s the output mappng, and T s earnng agorthm. The earnng agorthm s a recurrence reaton and s used to Unfavorabe Fgure : The state transton graph for L, - Favorabe Response Favorabe Response Unfavorabe Response + + Fgure 4: The state transton graph for Krnsky Automata Unfavorabe esponse Fgure 3: The state transton graph for L, 3.3 The Krnsky automaton Ths automata behaves exacty ke L, automata when the response of the envronment s unfavorabe, but for favorabe response, any state Φ (for =,..., passes to the state Φ and any state Φ (for = +,...,passes to the state Φ +. The state transton graph of Krnsky automaton s shown n fgure The Kryov automaton Ths automaton has state transton that s dentca to the L, automata when the output of the envronment s favorabe. However, when the response of the envronment s unfavorabe, a state Φ (,, +, passes to a state Φ + wth probabty.5 and to a stateφ wth probabty.5. When = or = +, Φ stays n the same state wth probabty.5 and moves to + wth the same probabty. When =, automata state moves Φ to Φ and wth the same probabty.5. When =, automata state moves Φ to Φ wth the same probabty.5. The state transton of Kryov automata s shown n fgure Varabe structure earnng automata Ths automata s represented by sextupe < β, Φ, α, P, G, T >, where β s a set of nputs actons, Φ Fgure 5: The state transton graph for Kryov Automata modfy the state probabty vector. Varous earnng agorthms have been reported n the terature [6]. Let α be the acton chosen at tme k as a sampe reazaton from dstrbuton p(. In near rewardpenaty ( LR P agorthm the recurrence equaton for updatng p s defned as: Favorabe response β ( n = p ( n + = + a[ ] p ( n + = ( a Unfavorabe response β ( n = p ( n + = ( b p ( n + = b ( r + ( b - Parameters a and b are caed step sze and determne amount of ncrement (decrement of the acton probabty. Another earnng agorthm that usuay used s near reward -nacton agorthm. In ( LR I ( LR I agorthm for favorabe response β ( n =, probabty correspondng to α ncrease and others decrease. But for unfavorabe response β ( n = probabtes are not changed. Recursve equaton for changng P s as foows: Favorabe response β ( n = Favorabe Responce / / / / - Unfavorabe Response

4 p ( n + = + a[ ] p ( n + = ( a Favorabe response β ( n = p ( n + = r In the above equatons r represents number of acton. α ( n {c} eura etwork Learnng Automaton β (n 4 Varabe Learnng Rate (VLR The performance of the steepest descent agorthm can be mproved f we aow the earnng rate to change durng the tranng process. An adaptve earnng rate w attempt to keep the earnng step sze as arge as possbe whe keepng earnng stabe. Frst, the nta network output and error are cacuated. At each epoch new weghts and bases are cacuated usng the current earnng rate. ew outputs and errors are then cacuated. If the new error exceeds the od error by more than a predefned rato max_perf_nc (typcay.4, the new weghts and bases are dscarded. In addton, the earnng rate s decreased (typcay by mutpyng by r_dec =.7. Otherwse the new weghts, etc. are kept. If the new error s ess than the od error, the earnng rate s ncreased (typcay by mutpyng by r_nc =.5. 5 Learnng Automata Based Methods In ths secton, we descrbe LA based methods for adaptaton of BP parameters. In these methods neura network acts as an envronment. The nterconnecton of neura network and earnng automata s shown n fg 6. Dfferent vaues of BP parameter acts as set of automata actons, n each step an acton s seected and feed to envronment. eura network uses ths parameter and runs BP agorthm tmes. Then a functon of network error s compared wth correspondng vaue n recent teraton. Ths functon for exampe can be mnmum of error n teraton. If we have a decrement n functon vaue the neura network generates favorabe response ( β ( n = and an ncrement unfavorabe response ( β ( n =. Usng network response, earnng automata changes acton probabty (n the case of LA wth varabe structure or changes state (n the case of LA wth fxed structure. 6 Adaptve Varabe Learnng Rate As before we say n varabe earnng rate secton, ths agorthm has three mportant parameters. These parameters are earnng rate ncrement coeffcent, earnng rate decrement coeffcent and maxmum error rato. These parameters have mportant nfuence on neura network earnng speed. The reason behnd ths s as foows: { Φ, α, β, F, G} Fgure 6: The nterconnecton of neura network and earnng automata It has been found when the error surface are from the form of quadratc bow, they usuay conssts of a arge amount of fat regons as we as ong and narrow extremey steep regons. Wth standard steepest descent, the earnng rate s hed constant throughout tranng. The performance of the agorthm s very senstve to the proper settng of the earnng rate. 8 6 E 4 P O C 8 H 6 S 4 Epochs versus Maxmum Error Rato Fgure 7: Epochs versus Maxmum Error Rato If the earnng rate s set too hgh, the agorthm may oscate and become unstabe. If the earnng rate s too sma, the agorthm w take too ong to converge. It s not practca to determne the optma settng for the earnng rate before tranng, and, n E P O C H S Fgure 8: Epochs versus Learnng Rate Increment Coeffcent Maxmum Error Rato Epochs versus Lr_Inc Learnng Rate Increament Coeffcent. 9

5 fact, the optma earnng rate changes durng the tranng process, as the agorthm moves across the performance surface. The performance of the steepest descent agorthm can be mproved f we aow the earnng rate to change durng the tranng process. E P O C H Epochs versus Lr_Dec..6 Learnng Rate Decreament Coeffcent Fgure 9: Epochs versus Learnng Rate Decrement Coeffcent An adaptve earnng rate w attempt to keep the earnng step sze as arge as possbe whe keepng earnng stabe. Frst, the nta network output and error are cacuated. At each epoch new weghts and bases are cacuated usng the current earnng rate. ew outputs and errors are then cacuated. If we are n a pont that have extremey steep regons, the new error exceeds the od error by more than a predefned rato max_perf_nc (typcay.4, n ths case the new weghts and bases are dscarded. In addton, the earnng rate s decreased (typcay by mutpyng by r_dec =.7. Otherwse the new weghts are kept. If sop of error surface s very arge, we must decrease the earnng rate wth hgh rate and f sop of error surface s not very arge, we can decrease earnng rate wth ow rate. So wth choosng dynamcay eerng rate decrement coeffcent, we can pass through speedy from regons wth hgh sops. On the other hand f we are n the regon that s fat, n ths case new error w smaer than od error and earnng rate s ncremented (typcay by mutpyng wth r_nc. If error surface s very fat then we must ncrease earnng rate wth hgh rate and f error surface s not very fat then we can ncrease earnng rate wth ow rate. So we see that wth choosng dynamcay ncrement coeffcent, we can pass wth hgh speed, through fat regons. In fgures 7 through 9, changes of earnng speed versus dfferent parameters of varabe earnng rate agorthm are shown. In ths paper we ntroduce a new agorthm named as Adaptve Varabe Learnng Rate (AVLR. In ths agorthm n spte of varabe earnng rate agorthm the agorthm parameters Max_Err_Rato, Lr_Dec and Lr_Inc are changed dynamcay durng earnng process. For adaptng parameters we use the fxed structure automata Tsetne. Smuaton resuts over varous probems show that, the proposed agorthm has the hghest earnng rate. AVLR agorthm s shown n fg.. Intaze automata parameters Intaze eura etwork Weghts and Bases Set Tranng Parameters for =:me Seect_VLR_Parameters for =:STEP_SIZE f SSE < eg, =-; break, Feedforward; Backward; Compute ew Weghts and Bases; Compute new_sse ; f new_sse > SSE* max_ perf _ nc r = r * Lr _ dec ; MC = ; ese f new_sse < SSE r = r * Lr _ nc ; w = new_w; b = new_b; a = new_a; w = new_w; b = new_b; a = new_a; e = new_e; SSE = new_sse; f new_mnmumofsse >= MnmumOfSSE*COEFFICIET EnvromentResponce= % penaze Automatan ese EnvromentResponce= % reward Automatan UpdateActveState_Tsetne Fgure : Adaptve Varabe Learnng Rate Agorthm 7 Smuaton Resuts In ths secton we compare dfferent methods of adaptng earnng rate. We mpement ths methods over Second order dscrete tme nonnear functon approxmaton Probem. 7. Second order dscrete tme nonnear functon approxmaton Probem Gven second order dscrete tme nonnear functon

6 .5y y y.35( y y. u k k k+ = + k + k + + yk + yk usng a three-ayer neura network we ke to smuate above functon wth reasonabe error. For ths purpose we generate nputs randomy between - and + and feed them to network. We aso choose nta condton randomy between - and +. As a resut a set of earnng patterns are generated that we use them for tranng neura network. In the above equaton uk and yk are nput and output at nstance k and yk and yk+ are outputs at nstance k- and k+. For approxmatng functon we use a three-ayer neura network wth 3 neuron n nput ayer, 8 neuron n hdden ayer and neuron n output ayer. ow we present the smuaton resut. Dfferent methods of adaptng earnng rate parameter are used for dfferent appcaton. In a experments momentum coeffcent are seected %98. Aso we use batch stye n a methods. In the foowng we descrbe each of methods. 7. Standard BP In ths method earnng rate are seected.. Batch stye are used are used for tranng. Momentum coeffcent has been chosen Varabe Learnng Rate(VLR Agorthm starts from nta earnng rate 4, and changes earnng rate dynamcay through earnng. 7.4 Varabe Structure Automata In ths method we use an automata wth varabe structure for adaptng earnng rate. Automata acton set whch are set of earnng rate are chosen as foows. α = {.94, } Acton probabty ncrement and decrement coeffcents are. and. respectvey. Step sze s 5. For an acton, the BP agorthm are repeated 5 tmes, then mnmum of error n prevous step compared wth mnmum of error n next step, f error ncrease neura network penaze automata ese reward. 7.5 Fxed Structure Automata We use Tsetne, Krnsky and Kryov automata for adaptng earnng rate. Smar to varabe structure automata, we choose 5 acton as foow. α = {., } Depth of memory and step sze are seected 3 and 5 respectvey. k 7.6 Adaptve Varabe Learnng Rate In ths method that ntroduced n ths paper we use Tsetne automata for adaptng varabe earnng rate parameter adaptaton. We choose canddates for ncrement, decrement and maxmum error rato coeffcent. Chosen vaues form automata acton set. Depth of memory and step sze are and 5 respectvey. Fgure shows resut of smuaton for second order dscrete tme nonnear functon approxmaton Probem. As shown n fgure, AVLR has maxmum speed and standard BP has mnmum speed. Usng AVLR earnng agorthm we approxmate second order nonnear dscrete functon:.5y k yk yk+ = +.35( yk + yk. u + k + yk + yk usng a three ayer neura network wth 3 neuron n nput ayer, 8 neuron n hdden ayer and neuron n output ayer. After 8 teratons neura network converge wth.898 error. For tranng network, we use random patterns n [-, +] duraton. For testng a c b Fgure : a nput b target output c actua output SSE a b c d e f g epochs Fgure : speed of convergence for dfferent method for Second order dscrete tme nonnear functon approxmaton Probem astandard BP bvlr cvsla(5 df_tsetne(5,3 ef_krnsky(5,3 ff_kryov(5,3 gavlr

7 network, we appy a snusoda nput. Fg shows nput, target output and actua output waveform. Target waveform and actua waveform are shown wth star and sod nes respectvey. As shown n fg., output of neura network concdes wth target waveform wth acceptabe precson. 8-Concuson One of the bggest mportant mtatons of BP agorthm, s the ow rate of convergence. For ncreasng speed of earnng some agorthms proposed whch, change earnng rate dynamcay accordng gradent varaton. Resut of smuaton show that, because VLR earnng agorthm has hgh facty for choosng earnng rate, hence have hgher speed than earnng automata based methods. Because The VLR parameters have mportant nfuence n ts performance, n ths paper we use earnng automata for adustng them. In the proposed agorthm named Adaptve Varabe Learnng Rate (AVLR agorthm, VLR parameters are changed dynamcay accordng to error changes by earnng automata. Smuaton resuts on second order dscrete tme nonnear functon approxmaton probem hghght better the mert of the proposed AVLR. References: [] D. E. Rumehart and J. L. Mc Ceand, Dstrbuted Processng: Expanaton n the Mcrostructureof Cognton, VOL. I, II and III. Cambrdge, MA: MIT Press, 986 and 987. [] B. Wdrow and M. A. Lehr," 3 Years of adaptve neura networks: Perceptron, madanes, and Back Propagaton," Proc. IEEE, Vo. 78, o. 9, pp , 99. [3] D. R. Hush and B. G. Horne, " Progress n supervsed neura networks," IEEE Sgna Processng- Mag., Vo., o., pp. 8-39, 993. [4] R. A. Jacobs, " Increased rates of convergence through earnng rate adaptaton," eura etworks, Vo., o. 4, pp , 988. [5] T. P. Voge et a., " Accearatng the convergenceof the backpropagaton method," Bo. Cybern., - Vo. 59, pp , 988. [6] L. G. Ared and G. E. Key, " Supervsed Learnng technques for backpropagaton networks,"n - Proc. Of IJC, Vo., SanDego, June 99, pp [7] L. P. Rcot, S. Ragazzn, and G. Martne, " Learnng of word stress n a uboptma second orderbackpropagaton neura networks," n Proc. st Int. Conf. eura etworks, Vo., pp , ew York, 988. [8] R. L. Watrous, " Learnng agorthms for connec- tonst networks: apped gradent methods of n- onnear optmzaton," n Proc. st Int. Conf. - eura etworks, Vo., pp , 987. [9] M. S. Moer, " A scaed conugate gradent agorthm for fast supervsed Learnng," eura etworks, Vo. 6, pp , o. 4, 993. [] A. R. Webb, D. Lowe, and M. D. Bedworth, "A- Comparson of nonnear optmzaton strategesfor feed forward adaptve ayered networks, "Roya Sgnas and Radar Estabshment, memorandum o. 457, uy 988. [] M. B. Menha and M. R. Meybod," Appcaton of Learnng Automata to eura etworks", Proc. Of Second Annua CSI Computer Conference CSIC 96, Tehran, Iran, pp. 9-, Dec [] M. B. Menha and M. R. Meybod, " A nove Learnng scheme for feedforward neura networks", Proc. Of ICEE-95, Unversty of scence and Technoogy, Tehran, 994. [3] M. B. Menha and M. R. Meybd, " Fexbe Sgmoda type functons for neura networks usng Game of automata", Proc. Of Second Annua - CSI Computer Conf. CSI 96, Tehran, Iran, pp. - 3, Dec 996. [4] M. B. Menha and M. R. Meybod, Usng Learnng Automata n backpropagaton agorthm wth momentum", Technca Report, Computer Eng. Department, Amrkabr Unversty of Technoogy, Tehran, Iran, 997. [5] H. Begy, M. R. Meybod, and M. B. Menha, "- Adaptaton of Learnng Rate n Backpropagaton Agorthm Usng Fxed Structure Learnng Automata", Pro. Of ICEE-98. [6] K. S. arra and M. A. L. Thathachar, Learnng Automata: An Introducton, Prentce Ha, E- ngewood cffs, 989. [7] M. R. Meybod and S. Lakshmvarahan,"Optma ty of a Genera Cass of Learnng Agorthm", - Informaton scence, Vo. 8, pp. -, 98. [8] M. R. Meybod and S. Lakshmvarahan," on a c- ass of Learnng Agorthm whch have a symmetrc Behavor under Success and Faure", Sprnger Vereg Lecture otes n Statstcs, pp , 984. [9] M. B. Menha, " Computatona Integence (vo. Fundamentas of eura etworks ", Professor Hessab Pubcaton, 998.