PDF hosted at the Radboud Repository of the Radboud University Nijmegen

Transcription

1 PDF hoted at the Radboud Repoitory of the Radboud Univerity Nijmegen The following full text i an author' verion which may differ from the publiher' verion. For additional information about thi publication click thi link. Pleae be advied that thi information wa generated on and may be ubject to change.

2 Stochatic of on-line backpropagation Tom eke Beckman Intitute and Department of Phyic, Univerity of Illinoi at Urbana-Champaign, 405 North Mathew Avenue, Urbana, Illinoi 680, U.S.A. Abtract We tudy on-line backpropagation and how that the exiting theoretical decription are trictly valid only on relatively hort time cale or in the vicinity of local minima of the backpropagation error potential. Qualitative global feature e.g., why i it much eaier to ecape from local minima than from global minima may alo be explained by thee local decription, but the current approache cannot give accurate quantitative prediction of global propertie e.g., how long doe it take to reach the global minimum tarting from a local minimum. Introduction On-line backpropagation tand for backpropagation where at each learning tep one of the training pattern i drawn at random from the training et and preented to the network. Thi i in contrat with batch-mode backpropagation where a weight change take place on account of the whole training et. The random pattern preentation in on-line backpropagation lead to a pecial kind of noie, which help to ecape from local minima in the error function. In the literature, everal uggetion have been made to decribe on-line backpropagation a a determinitic proce with uperimpoed noie [, 2, 3]. In thi paper we will tudy the validity of thi approach. We will dicu the uefulne of thi decription to explain global propertie of on-line backpropagation, uch a tationary olution and mean rt paage time. 2 Expanion of the mater equation At each learning tep, a training pattern x, with x denoting the combination of input vector and deired output vector, i drawn at random from the total training et and preented to the network. The weight change follow w = f(w x ) () with w the weight vector, which include the trength of all ynape and threhold, the learning parameter, and f(: :) the backpropagation learning rule. In the following we will ue one-dimenional notation for implicity. The learning proce () can be decribed by the mater equation [4,, 5] Z t P (w t) + P (w t) = dw 0 T (wjw 0 ) P (w 0 t) (2) with the tranition probability to go from an old tate w 0 to a new one w, T (wjw 0 ) = p px (w? w 0? f(w 0 x )) : = With a mart choice of the time-interval between ubequent adaptation, the mater equation (2) exactly decribe the learning proce () [5]. In general, thi mater equation cannot be Proceeding of the European Sympoium on Articial Neural Network '94, page 223{228

3 olved analytically. An option i to look for approximation valid for mall learning parameter. The rt tep in mot approximation cheme i to write the mater equation in the form of it completely equivalent Kramer-Moyal expanion (ee e.g. [6]) t P (w t) = X n= (?) n n! n w n [a n(w)p (w t)] with a n (w) p The Fokker-Planck equation ue only the drift a (w) and the diuion a 2 (w): t px = f n (w x ) : (3) P (w t) =? w [a (w)p (w t)] w [a 2(w)P (w t)] : (4) 2 It i often ued to tudy on-line backpropagation [, 2]. We will how that thi approach i trictly valid only on relatively hort time cale and/or to tudy local propertie of on-line backpropagation. A proper expanion i Van Kampen' \mall-uctuation expanion" [6]. It i baed on the Anatz that the evolution of w i given by a determinitic part (t) and uperimpoed noie with tandard deviation of order p : w = (t) + p : (5) Subtitution of thi Anatz into the Kramer-Moyal expanion (3) and collecting all term up to order lead to a et of three dierential equation [6, 5]: 8 >< >: d dt (t) = a ((t)) d dt hi t = a 0 ((t)) hi t d dt 2 t = 2a 0 ((t)) 2 t + a 2((t)) where h:i t tand for the enemble average over P (w t) and 0 denote dierentiation of a function with repect to it argument. From the et of equation (6) we conclude that Van Kampen' Anatz i valid if the derivative of the average learning rule a 0 () i negative or on time cale O(=). The generalization to higher dimenion i that the eian matrix (w), containing the econd derivative of the error potential E(w), mut be poitive denite. Each of thee o-called attraction region with poitive denite eian (w) contain one (local) minimum of the error potential E(w). So, the mall-uctuation Anatz (5) i valid in thee attraction region, but [on time cale > O(=)] not outide of thee attraction region. The mall noie approximation (6) can alo be obtained by ubtituting the Anatz (5) into the Fokker-Planck equation (4), i.e., all term O( 3 ) in the Kramer-Moyal expanion (3) vanih for mall. In thi ene the Fokker-Planck equation (4) i equivalent to Van Kampen' equation (6). owever, any (nonlinear) feature that arie from uing the Fokker-Planck equation beyond thi mall-noie approximation are puriou and cannot be taken eriouly [6]. (6) 3 Qualitative explanation of global feature An important dierence between on-line learning and imulated annealing or Langevin equation (ee e.g. [7]), i that the noie in on-line learning procee i intrinic and inhomogeneou, i.e., depend on the weight vector w, wherea the noie in imulated annealing and Langevin equation i articial and homogeneou, i.e., contant over the whole tate pace. If we dene

4 0.5 average error log[paage time] Figure : Average error (recaled uch that E global 0 and E local ) veru logarithm of mean rt paage time for on-line (lower line) and Langevin learning (upper line). On-line learning clearly yield a better performance. temperature a the average increae in error potential due to the local uctuation at a particular minimum w,we obtain [5] T (w ) he(w)? E(w )i I w Tr[a 2(w )] where the average i over the enemble of all network in the attraction region Iw of minimum w. So, the local temperature i proportional to the learning parameter and to the local diuion at the particular minimum. A an example, let u conider the XOR problem with an additional pattern (ee Appendix), which iknown to have deep local minima [8]. At the global minima, the trace of the diuion matrix i mall, ince all pattern are claied correctly. At the local minima, one of the pattern i miclaied, which lead to a much higher local temperature. Simulated annealing and Langevin equation have a \global temperature," i.e., the ame local temperature at all minima. Thi dierence ugget that the intrinic noie of on-line learning make it relatively more dicult to ecape from lower lying minima and i therefore favorable. To tet the validity of thi tatement, we will compare on-line learning with Langevin learning, a dicretized verion of the Langevin equation [6], where Gauian white noie i added to the gradient of the total error: w =?t re(w) + p 2T t : (7) All 500 learning network tart at a local minimum where four out of ve pattern are claied correctly. For dierent value of the learning parameter and temperature T (we keep t = and do not take into account that Langevin learning i about p = 5 time lower), we calculate the mean rt paage time into a region of weight pace where all ve pattern are claied correctly, i.e., where the output ha the correct ign for all ve pattern, and the average error for 0 t 20. For a fater ecape out of the local minimum, one would like to chooe a large learning parameter (high temperature), for a low aymptotic error a mall learning parameter (low temperature). A can be een from gure, on-line backpropagation i clearly better in dealing with thi conict: a lower (average) error can be reached in a horter time. 4 Quantitative prediction of global feature? In the previou ection we ued local expanion of the mater equation for a qualitative explanation of why on-line backpropagation might be a ueful global minimization trategy if compared

5 log[paage time] /learning parameter Figure 2: Logarithm of mean rt paage time, tarting from a local minimum into a region where all ve pattern are claied correctly, veru one over the learning parameter. On-line (upper line) and Langevin learning (lower line). Theory baed on only drift and diuion cannot predict thee curve. with adding homogeneou noie. Now we would like to invetigate whether we can apply any of thee approache to make quantitative tatement about global propertie. Therefore we ugget to again compare on-line learning with Langevin learning (7), but now with t = and inhomogeneou noie, choen uch that the drift vector and diuion matrice for both learning procedure are exactly equal. We tart with an enemble of 500 network at the local minimum and calculate the mean rt paage time into the region of weight pace where all ve pattern are claied correctly. Exiting approache (ee e.g. [, 5, 2, 3]) try to compute global propertie of on-line learning uing only the drift vector and the diuion matrix, i.e., cannot make a dierence between both learning procedure. A can be een from the reult in gure 2, where the logarithm of the mean rt paage time i plotted a a function of the reciprocal value of the learning parameter, the exiting approache are not ophiticated enough. The graph of on-line and Langevin learning do not have the ame lope (a uggeted in [5]), nor do they converge in the limit of mall learning parameter (a uggeted in [, 2, 3]). So, although Fokker-Planck approache, only baed on drift and diuion, can be ued for a quantitative analyi of local propertie of backpropagation (ection 2) and poibly alo for a qualitative explanation of global feature (ection 3), an application of thee approache to calculate global propertie of on-line backpropagation i doomed to fail (ection 4). Appendix The network, hown in gure 3(a), ha N =9 adaptive element, combined in the weight vector w =(w0 w w2 w20 w2 w22 w30 w3 w32) T, two variable input, x and x2, and xed input x0 = y0 =? to incorporate threhold. Output are given by (z j = x j for the hidden unit and z j = y j for the output unit) y i = tanh 2 X 4 2 j=0 w ij z j 3 5 : To prevent the exploion of the weight, we add a o-called bia (with =0:0 and =0:) to the quared backpropagation error: E(w) = 2p px 2X 2X [y3(w x x 2 )? x 3 ]2 + = 4 i=0 j=0 h w 2 ij? i 2 : (8)

6 (a)? w 0 w 20 c c? w w 2 2? w w22 c w 30???? w32 w 3 x x 2 (b) x 2????? x????????? Figure 3: a Network tructure. b XOR problem with one additional pattern. After [8], we chooe the et of p = 5training pattern ketched in gure 3(b). Circle indicate negative deired output x =?0:8, croe poitive output 3 x 3 =0:8. It i the uual XOR truth table with an additional pattern at the origin. Now the error potential (8) ha not only global minima, but alo deep local minima. The thick line in gure 3(b) how the eparation line of the hidden unit that lead to the optimal olution (all ve pattern correctly claied), the thin line thoe correponding to the local minima (one pattern miclaied). Acknowledgment Thi work wa upported by a grant from the National Intitute of ealth (P4RR05969) to Klau Schulten. Iwould like to thank Qing Sheng, Bert Kappen, and Klau Schulten for valuable comment and dicuion. Reference [] G. Radon,. Schuter, and D. Werner. Fokker-Planck decription of learning in backpropagation network. In International Neural Network Conference 90 Pari, page 993{996, Dordrecht, 990. Kluwer Academic. [2] T. Leen and G. Orr. Weight-pace probability denitie and convergence time for tochatic learning. In International Joint Conference on Neural Network. IEEE, 992. [3] L. anen, R. Pathria, and P. Salamon. Stochatic dynamic of upervied learning. Journal of Phyic A, 26:63{7, 993. [4]. Ritter and K. Schulten. Convergence propertie of Kohonen' topology conerving map: uctuation, tability, and dimenion election. Biological Cybernetic, 60:59{7, 988. [5] T. eke and B. Kappen. On-line learning procee in articial neural network. In J. Taylor, editor, Mathematical Foundation of Neural Network, page 99{233. Elevier, Amterdam, 993. [6] N. van Kampen. Stochatic Procee in Phyic and Chemitry. North-olland, Amterdam, 992. [7] T. Guillerm and N. Cotter. A diuion proce for global optimization in neural network. In IJCNN, volume, page 335{340, New York, 99. IEEE. [8] M. Gori and A. Tei. On the problem of local minima in backpropagation. IEEE Tranaction on PAMI, 4:76{86, 992.