Lcur 4 : Bacpropagaon Algorhm Pro. Sul Jung Inllgn Sm and moonal ngnrng Laboraor Chungnam Naonal Unvr
Inroducon o Bacpropagaon algorhm 969 Mn and Papr aac. 980 Parr and Wrbo dcovrd bac propagaon algorhm. 986, McCllland and Rumlhar ormulad ncl b ung dmodal uncon Dla rul and gradn dcn algorhm
Concp o gradn dcn algorhm Upda Proc Larnng Maurmn Proc Corrc d
Gradn algorhm Sp dcn algorhm g Th gradn g
Gradn algorhm g H Non algorhm H g Han mar
Gradn algorhm m m Han mar m m m m g H
Dla rul d d
Dla rul d d n n Dla rul Wgh upda
Gnralzd dla rul No n d
Gnralzd dla rul d d n n n Chan Rul
Gnralzd dla rul Funcon hould b drnabl b uncon Thn ' ' d ' So, h gh upda quaon bcom n
Sgmodal uncon Nonlnar uncon ' Hprbolc angn uncon Lnar uncon ' '
Bacpropagaon algorhm NI b NH b
Bacpropagaon algorhm No NINH NH NI NINO NO NH
Bacpropagaon algorhm3 d d ' ' ' NH b '
Bacpropagaon algorhm4 d d ' No No No ' ' No ' '
Calculaon o Bacpropagaon Algorhm5 Upda Proc Larnng Maurmn Proc Corrc No No b ' ' ' ' ' ' b
Malab nonlnar uncon poln lnar hardlm Sp uncon, 0, hardlm Sgnum, -, logg gmod radba Radal uncon ang Hprbolc angn alm Sauraon 0, alm Sauraon,
Summar o Bacpropagaon Algorhm5 ' b ' ' No ' b ' No '
Rul o humb: Larnng. Inal gh ar randoml lcd and normalzd.. Larnng ra lcd a 0 3. Momnum cocn lcd 0 Hghr momnum valu man o rl on mor prvou gh chang. 4. Slc h numbr o r hddn un largr han ha o h cond hddn lar. 5. Tran op hn - h magnud o h gradn ucnl mall - h obcv uncon all blo a d hrhold - a d numbr o raon - hr non longr an mprovmn n rror
Rul o humb: Hddn lar. On hddn lar prrrd or conrol applcaon. To or mor hddn lar ar prrrd or parn rcognon applcaon. 3. Th numbr o hddn un ar prrrd o b m o numbr o npu 4. Th numbr o hddn un grar ha on. 5. Th numbr o oupu un prrrd o b l han ha o hddn un. 6. Th numbr o hddn lar nod hould almo b much l han h numbr o ranng ampl. Ohr, rul n poor gnralzaon.
Rul o humb3. So, h hap o h nor ll b.on hould nvr u mor hddn lar nod ha ranng ampl.. For XOR problm, nc npu parn 4 h numbr o hddn un hould b l han 4. 3. Ba gh hould b ud and updad n am mar.
Tranng Bacpropagaon algorhm. Sar h randoml lcd gh 0. 0.. Appl parn p a npu 3. Calcula, 4. Compu rror d 5. A oupu lar, calcula ' 6. A hddn lar, calcula
Tranng Bacpropagaon algorhm 6. A hddn lar, calcula 7. Calcula gh chang 8. Parn b parn, go o 0 poch larnng, go o p 9 '
Tranng Bacpropagaon algorhm3 9. All parn ar d? allparn p I, go o p, no, go bac o p. 0. Parn b parn larnng, upda gh b b b b b b
Tranng Bacpropagaon algorhm4. Rpa ccl unl convrg. 0. Parn b parn larnng, upda gh allparn p p b b b b b b
Drabac o Bacpropagaon algorhm. Snc BP u a gradn dcn procdur. Convrgnc can b lo.. I ma b uc a h local mnma.
Drabac o Bacpropagaon algorhm 3.No rul or lcng opmal numbr o hddn nuron 4. No rul or lcng opmal numbr o hddn lar. 5. No rul or lcng opmal larnng ra. 6. Wha condon ar rqurd or larg ranng 7. Ho o avod ovr ng? Ovrng: h pon hr h nor ar o ovr h ranng daa hn h prormanc o h nor ar o dgrad.
Som oluon or Bacpropagaon algorhm. Prunng proc - I lmna unmporan gh. - Th nor rand h BP, and hn chc gh chang. - Dl h gh h no chang or mall chang. - Rran h rducd z nor.
Som oluon or Bacpropagaon algorhm. Wgh addng mhod - I r o g ou o local mnma. - Whn h rror do no convrg or a long m, m o b uc a h local mnma. - Thn add h nuron o acva proc o ump ovr o h global mnma. - I rqur a dnamc programmng ll.
Som oluon or Bacpropagaon algorhm 3. Adapv larnng ra - I r o pd up convrgnc. -
Unvral Appromaon Thorm Q. Wha h mnmum numbr o hddn lar? A. Unvral appromaon horm or nonlnar npu-oupu mappng. N O - F b N H F
Hom or # Drv BP algorhm or npu-hddn-hddn-oupu lar rucur o MLP. Du on Spmbr 4, 03 Hom or #3 Fnd and ud Han mar. Fnd and Sud h Qua-Non Mhod. Du on Ocobr, 03