The Method of Steepest Descent for Feedforward Artificial Neural Networks

Size: px

Start display at page:

Download "The Method of Steepest Descent for Feedforward Artificial Neural Networks"

Myra Campbell
6 years ago
Views:

1 IOSR Joural o Mahac (IOSR-JM) -ISSN: , p-issn:39-765x. Volu, Iu Vr. II. (F. 4), PP 53-6.oroural.org Th Mhod o Sp Dc or Fdorard Arcal Nural Nor Muhaad Ha, Md. Jah Udd ad Md Adul Al 3 Aoca Proor, Dpar o Appld Mahac, Noahal Scc ad Tcholog Uvr, Noahal 384, Bagladh. Lcurr, Dpar o Appld Mahac, Noahal Scc ad Tcholog Uvr, Noahal 384, Bagladh. 3 Aa Proor, Chagog Uvr, Chagog 433, Bagladh. Arac: I h papr, pl h hod o Sp Dc gl ad ullar dorard arcal ural or. I all prvou or, all h upda gh quao or gl or ullar dorard arcal ural or ha calculad choog a gl acvao uco or varou procg u h or. W, a r, calcula h oal rror uco paral or gl ad ullar dorard arcal ural or ad h calcula h hr upda gh quao or ag dr acvao uco dr procg u paral gl ad ullar dorard arcal ural or. A apl gv o ho uul o h plao. Kord: Fdorard Arcal Nural Nor, Bac propagao Larg, Acvao Fuco, Trag. I. Iroduco Fd-orard arcal ural or (FNN) [ -3] hav dl ud or varou a, uch a par rcogo, uco approao, dacal odlg, daa g, ad r orcag. Th rag o FNN al udra ug h ac-propagao (BP) [4-5 ] ad larg. Th Bacpropagao rag algorh or rag d-orard or a dvlopd Paul Wro[Paul Wro84], ad lar Parr[Parr85] ad Rulhar [Rulhar 94]. Th p o or cogurao h o coo u, du o a o rag. I ad ha ovr 8% o all ural or proc dvlop u acpropagao. Th rao or h a "acprogagao" ha h oupu rror ar "propagad ac" ro h oupu lar o h hdd lar, ad ar ud h upda quao or h hdd lar gh. Thr ar o pha larg ccl, o o propaga h pu par hrough h or ad h ohr o adap h oupu, chagg h gh h or. I h rror gal ha ar ac propagad h or oprao o h hdd lar(). Th poro o h rror gal ha a hdd-lar uro rcv h proc a a o h coruo o a parcular uro o h oupu rror. Adug o h a h gh o h coco, h quard rror, or o ohr rc, rducd ach ccl ad all zd, pol. A Bac-Propagao or co o a la hr lar o u: a pu lar, a la o rda hdd lar, ad a oupu lar. Tpcall, u ar cocd a d-orard aho h pu u ull cocd o u h hdd lar ad hdd u ull cocd o u h oupu lar. Wh a Bac- Propagao or ccld, a pu par propagad orard o h oupu u hrough h rvg pu-o-hdd ad hdd-o-oupu gh. Fgur Srucur o a dorardd arcal ural or..oroural.org 53 Pag

2 Th Mhod o Sp Dc or Fdorard Arcal Nural Nor Th hod o p dc [7-8] o popular aog a ahaca: vr pl, a o u, ad ach rpo a. Bu h gg advaag o h hod l h ac ha guarad o d h u hrough urou o rao a log a. Hovr, h hod alo ha o g la: I ud o a adl cald, ll d up gog hrough a ur o rao or locag h u, ad c ach o p a durg rao ar rl all, hu h covrgc pd pr lo, h proc ca lrall a orvr! Al- hough a largr p z ll cra h covrgc pd, u could alo rul a a h larg rror. For apl, hr a log ad arro vall h rror urac, h copo o h grad h drco ha po alog h a o h vall vr all hl h copo alog h vall all qu larg. Th rul oo or h drco o h all v hough hav o ov a log dac alog h a ad a all dac alog h all. I h papr, pl h hod, paral, gl ad ullar dorard arcal ural or ad hav or uul rul. Sgl-Lar Nor Codr a gl lar dorard ural or o uro h -h oupu a ho h Fgur. Thr ar pu Fgur. Sgl lar dorard ural or, hr...,., W = , h ghd ar h z ud o do h rgh o h coco ro h h pu o h h procg l.,, hr...,, ar h a or -h procg u,, hr...,, ar h pu or -h procg u, hr...,, ar h acvao uco or -h procg u ad, hr...,, ar h oupu or uro. No ha h,, ar al gh or - h procg u ad o h al pu gal. W alo codr, ad ar h arg(drd), rror gal ad Ma-Squar rror ( Su o Squard rror) uco rpcvl. W hav h ollog calculao N pu:... hr...,.oroural.org 54 Pag

3 Th Mhod o Sp Dc or Fdorard Arcal Nural Nor.oroural.org 55 Pag () Ba:... () Fro () ad () hav; (3) Nor Oupu: (4) rror Sgal: Whr,, ar arg oupu or gv uro h oupu lar. (5) Ma-Squar rror uco:

4 Th oal rror(or pu par): oal Th Mhod o Sp Dc or Fdorard Arcal Nural Nor (7) Mulpl-Lar Nor Codr a hr lar dorard ural or a ho h Fgur 3. (6) Fro pu o hdd lar: Thr ar pu Fgur 3. Thr lar dorard ural or, hr...,., , h ghd ar h z ud o do h rgh o h coco ro h h pu o h h procg l., Thr ar uro h hdd lar,, hr...,, ar h a or -h procg u,, hr...,, ar h pu or -h procg u or -h procg u ad, hr...,, hr...,,, ar al gh or -h procg u ad o W hav h ollog calculao N pu: Ba:..., ar h acvao uco, ar h oupu or uro. No ha h h al pu gal. (8).oroural.org 56 Pag

5 Th Mhod o Sp Dc or Fdorard Arcal Nural Nor.oroural.org 57 Pag.... (9) Fro (8) ad (9) hav; () Nor Oupu h hdd lar:. () Fro hdd lar o oupu lar: Thr ar pu, hr...,, , h ghd ar h z ud o do h rgh o h coco ro h h pu o h h procg l., Thr ar uro h oupu lar,, hr...,, ar h a or -h procg u,, hr...,, ar h pu or -h procg u, hr...,, ar h acvao uco or -h procg u ad, hr..., ar h oupu or uro. No ha h,, ar al gh or -h procg u ad h al pu gal. W hav h ollog calculao N pu:

6 Th Mhod o Sp Dc or Fdorard Arcal Nural Nor.oroural.org 58 Pag... () Ba:.... (3) Fro () ad (3) hav; (4) Nor Oupu h oupu lar:.. (5) rror Sgal:. Whr,, ar arg oupu or gv uro h oupu lar. (6) Ma-Squar rror uco: (7) Th oal rror h oupu lar(or pu para):

7 Th Mhod o Sp Dc or Fdorard Arcal Nural Nor.oroural.org 59 Pag oal (8) Iplao Sgl lar Nor Fro quao (7) hav oal (9) W hav; oal. () Ug Sp Dc Mhod hav h upda gh quao a p p. () Mulpl lar or Fro (8) hav oal () oal (3) Ug Sp Dc Mhod hav h upda gh quao a p p ad.. oal oal. Ug Sp Dc Mhod hav h upda gh quao a

8 Th Mhod o Sp Dc or Fdorard Arcal Nural Nor p p. (4) Nurcal apl Codr h gl lar dorard ural or a ho pu.,. 6, W =..3.oroural.org.3.4 gur. Suppo ha hr ar, h ordr ghd ar ud o do h rgh o h coco ro h pu o procg u., h a or o procg u ar,, h pu or o procg u ar,,, ar h acvao uco or h procg u ad,, ar h oupu or uro. No ha h.,. ar al gh or o procg u ad h al pu gal., Codr, ad ar h rror gal ad Ma-Squar rror (Su o Squard rror) uco rpcvl ad.7,. 8 ar h arg(drd). U a p z o =., W hav h ollog calculao N pu: Nor Oupu: rror Sgal: Ma-Squar rror uco:.9.3 Th oal rror (or pu par): oal To upda h gh, u quao () Th upda gh ar:. o 6 Pag

9 Th Mhod o Sp Dc or Fdorard Arcal Nural Nor W Nor Oupu: rror Sgal: Ma-Squar rror uco:.6.43 Th oal rror (or pu par): oal Ovoul, oal < oal ; ha, h acual oupu o h ural or ha co clor o h arg oupu a a rul o updag h gh. II. Cocluo A pl u cv dcrpo o dorard arcal ural or ha ad hr gvg phaz o h acpropagao algorh, c dl ud ad a ohr algorh ar drvd ro. W hav pld h p dc hod gl ad ulpl-lar dorard arcal ural org prol ad -up a urcal apl. Frl, W hav calculad h oal rror uco paral or gl ad ullar dorard arcal ural or ad h calculad h hr upda gh quao or ag dr acvao uco dr procg u paral gl ad ullar dorard arcal ural or. Th covrgc havor o our apl ho ha h rul o h acual or oupu a clo o our drd(arg) oupu. Acoldg Th Noahal Scc ad Tcholog Uvr ad h Uvr o Chagog, or provdg a ulag vro or rarch coco h h arcl. Rrc [] Ad, P.R. ad Dpr, M.A.H Iroduco o Opzao Mhod. Had Pr, N Yor. [] Ba, R. 99. Fr ad cod-ordr hod or larg: B Sp Dc ad No hod. Nural Copuao 4: [3] Johao,.M., Dola, F.U. ad Gooda, D.M. 99. Bacpropagao larg or ul-lar d-orard ural or ug h couga grad hod. Il. J. Nural S, : 9-3 Judd, J.S. 99. Nural Nor Dg ad h copl o Larg. MIT Pr, Cardg, MA [5] Muhaad Ha.. Opal o Ucorad Nolar Prograg Prol. M.Phl Th, Uvr o Chagog, Chagog, Bagladh. [6] Mhra] P., ad Wah, B. W. 99. Arcal ural or: cocp ad hor I Copu. Soc Pr,. [7]. X. Yu, M. O., ad O. Kaa.. A gral acpropagao algorh or d-orard ural or larg. I Tra. Nural Nor. 3:5 54. [8] X. H. Hu ad G. A. Ch.997. c acpropagao larg ug opal larg ra ad ou. Nural Nor : [9] Zurada, J. M.. Iroduco o arcal ural. M. G. Road, Mua: Jaco..oroural.org 6 Pag

Overview. Introduction Building Classifiers (2) Introduction Building Classifiers. Introduction. Introduction to Pattern Recognition and Data Mining

Overview. Introduction Building Classifiers (2) Introduction Building Classifiers. Introduction. Introduction to Pattern Recognition and Data Mining Ovrv Iroduco o ar Rcogo ad Daa Mg Lcur 4: Lar Dcra Fuco Irucor: Dr. Gova Dpar of Copur Egrg aa Clara Uvry Iroduco Approach o uldg clafr Lar dcra fuco: dfo ad urfac Lar paral ca rcpro crra Ohr hod Lar Dcra