CS6: Iroducio o Compuig ih Neural Nes lecure- Pushpak Bhaacharyya Compuer Sciece ad Egieerig Deparme IIT Bombay
Tilig Algorihm repea A kid of divide ad coquer sraegy Give he classes i he daa, ru he percepro raiig algorihm If liearly separable, covergece ihou ay hidde layer If o, do as ell as you ca pocke algorihm This ill produce classes ih misclassified pois
Tilig Algorihm cod Take he class ih misclassified pois ad break io subclasses hich coai o ouliers Ru PTA agai afer recruiig he required umber of percepros Do his uil homogeous classes are obaied Apply he same procedure for he firs hidde layer o obai he secod hidde layer ad so o
Illusraio XOR problem Classes are,,,,
As bes a classificaio as possible, ve, -ve, -ve, ve
Classes ih error,,,, oulier
Ho o achieve his classificaio Give he labels as sho: eqv o a OR problem -,,,, oulier
The parially developed / Ge he firs euro i he hidde layer, hich compues OR h.5..
Break he icorrec class,,,,, oulier,, - Do care: Make
Solve classificaio for h,,,, - This is
Ne sage of he / Compues Compues h h -.5.5 -. -...
Geig he oupu layer Solve a ilig algo problem for he hidde layer h h y,,, -, AND problem
Fial / AND / y.5 Compues.. Compues Compues h h.5 -.5. -. -..
Lab eercise Impleme he ilig algorihm ad ru i for. XOR. Majoriy. IRIS daa
Hopfield e Ispired by associaive memory hich meas memory rerieval is o by address, bu by par of he daa. Cosiss of N euros fully coeced ih symmeric eigh sregh ij = ji No self coecio. So he eigh mari is - diagoal ad symmeric. Each compuig eleme or euro is a liear hreshold eleme ih hreshold =.
Compuaio Y Neuro i θ = W W W σ σ... σ Figure: A euro i he Hopfield Ne.
Eample = = 5 = = = = A ime = s = s = - s = Usable sae: Neuro ill flip. A sable paer is called a aracor for he e. 5 Figure: A eample Hopfield Ne
Sabiliy Asychroous mode of operaio: a ay isa a radomly seleced euro compares he e ipu ih he hreshold. I he sychroous mode of operaio all euros updae hemselves simulaeously a ay isa of ime. Sice here are feedback coecios i he Hopfield Ne he quesio of sabiliy arises. A every ime isa he eork evolves ad fially seles io a sable sae. Ho does he Hopfield Ne fucio as associaive memory? Oe eeds o sore or sabilize a vecor hich is he memory eleme.
Eergy cosideraio Sable paers correspod o miimum eergy saes. Eergy a sae <,,,, > E = -/ j j<>i ji i j Chage i eergy alays comes ou o be egaive i he asychroous mode of operaio. Eergy alays decreases. Sabiliy esured.
Hopfield Ne is a fully coeced eork.... 4 i h euro is coeced o - euros
Cocep of Eergy Eergy a sae s is give by he equaio: E s = [ K K M ]
Coecio mari of he eork, -diagoal ad symmeric j... k i... ij k diagoal
Sae Vecor Biary valued vecor: value is eiher or - X = < -.... > e.g. Various aribues of a sude ca be represeed by a sae vecor heigh address 5 roll umber hair color Ram = ~Ram = - 4
Relaio beee eigh vecor W ad sae vecor X W T X Weigh vecor Traspose of sae vecor 5-5 5 W = T X 5 5 = W = T X For eample, i figure, A ime =, sae of he eural eork is: s = <, -, > ad correspodig vecors are as sho. Fig.
W.X T gives he ipus o he euros a he e ime isa 5 5 W = T X = 7 =. sg T X W This shos ha he / ill chage sae
Eergy Cosideraio A ime =, sae of he eural eork is: s = <, -, > 5 - E = -[5**-** *-*] = 4 - The sae of he eural eork uder sabiliy is <-, -, -> 5 - - Esable sae = - -[5*-*-*-* -*-*-] = -
Sae Chage s = <, -, > s = compue by comparig ad summig ==sg[σ j= j j ] = if Σ j= j j > = - oherise
Theorem I he asychroous mode of operaio, he eergy of he Hopfield e alays decreases. Proof: M [ E = K K ]
Proof Le euro chage sae by summig ad comparig We ge folloig equaio for eergy [ E = K K ] M
Proof: oe ha oly euro chages sae [ ]} K [ ] = = j j j j {[ ] = K E E E = [ ][ ] = = j j j Sice oly euro chages sae, j = j, j=,, 4,, ad hece
= Proof coiued [ ][ ] j j j= S D Observaios: Whe he sae chages from - o, S has o be ve ad D is ve; so E becomes egaive. Whe he sae chages from o -, S has o be -ve ad D is ve; so E becomes egaive. Therefore, Eergy for ay sae chage alays decreases.
The Hopfield e has o coverge i he asychroous mode of operaio As he eergy E goes o decreasig, i has o hi he boom, sice he eigh ad he sae vecor have fiie values. Tha is, he Hopfield Ne has o coverge o a eergy miimum. Hece he Hopfield Ne reaches sabiliy.