Content. A Strange World. Clustering. Introduction. Unsupervised Learning Networks. What is Unsupervised Learning? Unsupervised Learning Networks

Transcription

1 Usupervsed Learg Newors Cluserg Coe Iroduco Ipora Usupervsed Learg NNs Hag Newors Kohoe s Self-Orgazg Feaure Maps Grossberg s AR Newors Couerpropagao Newors Adapve BAN Neocogro Cocluso Usupervsed Learg Newors Iroduco Wha s Usupervsed Learg? Learg whou a eacher. Lear o for classes/clusers of saple paers accordg o slares aog he No feedbac o dcae he desred oupus. he ewor us b self dscover he relaoshp of eres fro he pu daa. E.g., paers, feaures, regulares, correlaos, or caegores. raslae he dscovered relaoshp o oupu. Usupervsed Learg wh Arfcal Neural Newors A Srage World he ANN s gve a se of paers, P, fro space, S, bu lle/o forao abou her classfcao, evaluao, eresg feaures, ec. I us lear hese b self! ass Cluserg - Group paers based o slar Vecor Quazao - Full dvde up S o a sall se of regos (defed b codeboo vecors) ha also helps cluser P. Probabl Des Approxao - Fd sall se of pos whose dsrbuo aches ha of P. Feaure Exraco - Reduce desoal of S b reovg upora feaures (.e. hose ha do o help cluserg P)

2 Hegh Hegh Hegh Hegh Hegh Hegh Supervsed Learg Supervsed Learg A B A B C C he Probables of Populaos Usupervsed Learg A B C Usupervsed Learg Caegorze he pu paers o several classes based o he slar aog paers. Cluserg Aalss

3 Hegh Hegh Hegh Hegh Caegorze he pu paers o several classes based o he slar aog paers. Cluserg Aalss Caegorze he pu paers o several classes based o he slar aog paers. Cluserg Aalss clusers Caegorze he pu paers o several classes based o he slar aog paers. Cluserg Aalss 3 clusers Caegorze he pu paers o several classes based o he slar aog paers. Cluserg Aalss 4 clusers Usupervsed Learg Newors he Hag Newors he Neares Neghbor Classfer Suppose ha we have p proopes ceered a x (), x (),, x (p). Gve paer x, s assged o he class label of he h proope f ( ) arg ds( xx, ) Exaples of dsace easures clude he Hag dsace ad Eucldea dsace. 3

4 he Neares Neghbor Classfer he Neares Neghbor Classfer x () x () x () x () 3 4 x (3) x (4) 3?Class 4 x (3) x (4) Hag Newor he Hag Newors Sored a se of classes represeed b a se of bar proopes. Gve a coplee bar pu, fd he class o whch belogs. Use Hag dsace as he dsace easuree. Dsace vs. Slar. Hag ewor: e copues d bewee a pu ad each of he P vecors,, P of deso pu odes, P oupu odes, oe for each of P sored vecor p whose oupu = d(, p ) Weghs ad bas: W, P Oupu of he e: o W, P where o 0.5( ) s he egave dsace bewee ad he Hag Ne he Hag Dsace MAXNE Wer-ae-All = x = Slar Measuree x x x 4

5 he Hag Dsace = x = HD( x, ) (7 ) 3 he Hag Dsace = x = Su= he Hag Dsace he Hag Dsace (,,, ) {, } x ( x, x,, x ) x {, } HD( x, )? Slar( x, )? (,,, ) {, } x ( x, x,, x ) x {, } HD( x, ) ( x ) Slar( x, ) ( x ) x he Hag Ne he Hag Ne MAXNE Wer-ae-All W M =? MAXNE Wer-ae-All Slar Measuree W S =? Slar Measuree x x x x x x x x 5

6 W M =? he Sored Paers W S =? Slar Sored ( x, s ) MAXNE Wer-ae-All ( xs, ) x x x x paers s, s,, s ( s, s, s ) wh s {, }. Slar x s x s s Slar Measuree Sored paers s, s,, s ( s, s, s ) wh s {, }. he Sored Paers Slar( / s s x, s ) Slar( x, s )... s x x x x s Slar( xs, ) x s s Slar Measuree Sored paers s, s,, s ( s, s, s ) wh s {, }. Weghs for Sored Paers s Sored paers s, s,, s ( s, s, s ) wh s {, }. Weghs for Sored Paers s s s s s s s WS s s s W S =? x s x s Wx S x s Slar Measuree W S =? θ / / / / / Slar Measuree x x x x x x x x he MAXNE Weghs of MAXNE MAXNE Wer-ae-All MAXNE Wer-ae-All Slar Measuree x x x x 6

7 Weghs of MAXNE 0< < / Updag Rule 0< < / MAXNE Wer-ae-All MAXNE Wer-ae-All ε ε ε ε ε ε WM ε ε ε ε ε ε s s s 3 s ε ε ε ε ε ε WM ε ε ε ε ε ε a( W ) M 0,,, 0 s Updag Rule 0< < / MAXNE Wer-ae-All a w a ε a ( ε) ε s s s 3 s ε ε ε ε ε ε WM ε ε ε ε ε ε a( W ) M 0,,, 0 s Aalss Updag Rule a w a e Le a( e) 0 e 0 e 0 ε a ( ε) ε v If ow, v0 0 v 0 Aalss Updag Rule Exaple a w a e Le a( e) 0 If ow ax e 0 e 0 ε a ( ε) ε ax f

8 Usupervsed Learg Newors he Self-Orgazg Feaure Map SOM Wha s? he os popular ANN algorh he usupervsed learg caegor Covers relaoshps bewee hgh-desoal daa es o sple geoerc relaoshps o a low-desoal dspla Copresses forao whle preservg he os pora opologcal ad erc relaoshps of he prar daa es Daa vsualzao, feaure exraco, paer classfcao, adapve corol of robos, ec. Feaure Mappg Soaoopc Map Illusrao: he Houculus Map hgh-desoal pu sgals oo a lowerdesoal (usuall or D) srucure. Desoal Reduco Slar relaos prese he orgal daa are sll prese afer he appg. he relaoshp bewee bod surfaces ad he regos of he bra ha corol he. opolog-preservg Map Phooopc aps(kohoe,998) Phooopc aps huppla 8

9 Self-Orgazg Feaure Map he Srucure of SOM Ideas frs roduced b C. vo der Malsburg (973) Developed b professor Kohoe -98 Oe of he os popular eural ewor odels. Usupervsed learg. Copeve learg ewors. SOM: Algorh Self-Orgazg Maps. Ialze ap (weghs). Selec a saple (pu) 3. Deere eghbors 4. Chage weghs 5. Repea fro for a fe uber of seps Lace of euros ( odes ) acceps ad respods o se of pu sgals Resposes copared; wg euro seleced fro lace Seleced euro acvaed ogeher wh eghbourhood euros Adapve process chages weghs o ore closel reseble pus w w w 3 w d arra of euros Weghed sapses x x x 3... x Se of pu sgals (coeced o all euros lace) Exaple Local Excao, Dsal Ihbo 9

10 opologcal Neghborhood Sze Shrage Square Hex ) N (* ) N (* * ) N (* 3 Sze Shrage Learg Rule ) N (* ) N (* Slar Machg x wˆ x wˆ * * ) N (* 3 Updag ( ) ( ) ( ) ( ) w α x w [ ] ( ) w ( ) w N ( ) * oherwse Kohoe SOM (Self Orgazg Maps) Kohoe SOM (Self Orgazg Maps) Exaple 0

11 Kohoe SOM (Self Orgazg Maps) Exaple Exaple Exaple Exaple Exaple-II Deals Radol alzed ap 00 x 00 grd of euros, each coag a 3-desoal wegh vecor represeg s RGB value rag pu radol seleced fro 48 uque colors Gaussa eghborhood fuco

12 Malab SOM code Screeshos clear all; close all; %e = ewso(pr,[d,d,...],fc,dfc,olr,oseps,lr,s) C = rad(3,000); %3 x 000 rado arx %PR - Rx arx of ad ax values for R pu elees. C = C*sqr(0.); %D - Sze of h laer deso, defauls = [5 8]. %FCN - opolog fuco, defaul = 'hexop'. C = rad(3,000); %3 x 000 rado arx C = C*sqr(0.); e.rapara.epochs = 500; C = C+5; e = ra(e,p); ploso(e.w{,},e.laers{}.dsaces) %Plo he daa se le('som 3-D weghs'); plo3(c(,:), C(,:), C(3,:), 'g+') %PLOSOM(W,D,ND) aes hree argues, hold o % W - SxR wegh arx. plo3(c(,:), C(,:), C(3,:), 'r*') % D - SxS dsace arx. hold off % ND - Neghborhood dsace, defaul =. % ad plos he euro's wegh vecors wh %Creae SOM ad Plo he weghs coecos bewee wegh vecors whose euros are P = [C C]; wh a dsace of. e = ewso(ax(p),[0 0],'grdop'); %NEWSOM Creae a self-orgazg ap. %e = ewso(pr,[d,d,...],fc,dfc,olr,oseps,lr,s). Classfg World Pover Coclusos Advaages SOM s Algorh ha proecs hgh-desoal daa oo a wo-desoal ap. he proeco preserves he opolog of he daa so ha slar daa es wll be apped o earb locaos o he ap SOM sll have a praccal applcaos paer recogo, speech aalss, dusral ad edcal dagoscs, daa g Dsadvaages Dffcul o deere wha pu weghs o use Mappg ca resul dvded clusers Requres ha earb pos behave slarl