HOPFIELD NETWORKS 9.1 INTRODUCTION

Transcription

1 UNI 9 HOPFIELD NEWORKS Structure Page No. 9. Introducton 9 Objectves 9. Related Defntons 0 9. Hopfeld Networs 9.4 Structure of Hopfeld Networs he Functonalty of Hopfeld Networs Storage Capacty of Hopfeld Networs Summary Solutons/Answers Practcal Assgnments INRODUCION One of the most mportant functons of human bran s the layng down and recall of memores. More over the studes n neurodynamcs ndcate the presence of shortterm and long-term memory. he short-term memory contrbutes n processng routne tass whle long-term memory helps n storng the lesson from the past experence. Our memores mostly functon n a manner better termed as assocatve or content-addressable. hat s, a memory does not exst n some solated fashon, located n a partcular set of neurons. All memores are n some sense strngs of memores. For example, we remember a place or a person by some event. Sometmes more than the facal features of a person t s the voce or the pecularty of the name that mpresses our memory. hus, memores are stored n assocaton wth one another. he neural networ researchers contend that there s a basc msmatch between the standard technology used by computers and the technology of the human bran for processng nformaton. Feed-forward neural networs are well studed for obvous reasons of smplcty n understandng and modellng the computaton. Feed forward networ s acyclc. he artfcal neurons or the perceptrons n a feed-forward neural networ model have no cyclc connectons. he nput data s processed and passed to the outputs but not vve-versa. Recurrent networs can have connectons that go bac from the output nodes and can also have arbtrary connectons between any nodes. hat s a recurrent neural networ has at least one feedbac loop. For example, n a sngle layer recurrent networ each neuron may have a feedbac connecton to each of the other neuron or t may even have self-feedbac loops. As learnng usng perceptrons s a process of modfyng the values of the synaptc weghts and the threshold or the actvaton functon usng samples of tranng data. A group of (connected) perceptrons are traned on sample nput-output pars untl t learns to compute the correct functon. Once a feed-forward networ s traned, ts state s fxed whch s not altered by any subsequent nput data untl the networ s retraned. hs ndcates that a feed-forward neural networ does not have memory. In a neurobologcal context memory refers to the relatvely endurng neural alteratons nduced by the nteracton of an organsm wth ts envronment. Learnng tass lead us to assume exstence of memory. herefore, the alteratons ndcate presence of memory and for ts usefulness memory needs to be accessble. Assocatve memores have a very fant smlarty to that of the human bran s ablty to assocate patterns. he archtecture of a neural networ wth assocatve memory

2 Applcatons of Neural Networs depends on ts capablty of assocaton. An assocate memory s a storehouse of assocated patterns encoded n some form. When ths storehouse s trggered or ncted wth a pattern the assocated pattern par s recalled or output. An assocatve memory networ s desgned by storng/recordng several deal patterns nto the networ s stable state. When a pattern (even wth nose or dstorted representaton of stored patterns) s nput to the storehouse, the networ s expected to reach one of the stored patterns and retreve t as an output. hese neural networ models are categorzed as assocaton models and are also nown as content addressable or autoassocatve neural networs. All the defntons wll be dscussed n Sec. 9.. In Sec. 9. we shall dscuss Hopfeld networs. In Sec. 9.4, we shall dscuss the structure of the Hopfeld networs, n detal and we shall extend our dscusson on the functonalty of the Hopfeld networs n Sec he storage capacty of ths networ s dscussed n Sec Objectves After readng ths unt you should be able to: dentfy and desgn general assocaton and recurrent models of neural networ; model Hopfeld networs; tran Hopfeld networs for a gven set of patterns; gve a traned networ to run the Hopfeld for a test pattern wth nose may requre rgorous computaton specally f dmenson of the vectors s hgh. 9. RELAED DEFINIIONS Let us recall varous defntons as gven below: Recurrent Networs: he nterconnecton scheme of the recurrent networs s feedbac connecton or loops as shown below n Fg.. Output Input Fg. : Recurrent networ Assocatve Memory: An assocatve memory s a bran-le dstrbuted memory that learns by assocaton. Assocaton s one of the basc features of the human memory and s prevalent n most models of cognton. Assocatve memory can be categorzed as auto assocatve memory and Hetero assocatve memory. hese are defned n the followng: 0

3 ) Autoassocaton: In autoassocaton a neural networ s requred to store a set of patterns (vectors) by repeatedly presentng them to the networ. he networ s subsequently presented a partal descrpton or nosy verson of an orgnal pattern stored n t, and the tas s to recall or more specfcally retreve that partcular pattern. ) Heteroassocaton: In heteroassocaton an arbtrary set of nput patterns s assocated (pared) wth another set of arbtrary set of output patterns. he tas of retreval of patterns however s smlar.e. on nput of a stored pattern or a dstorted verson of the already stored pattern the orgnal pattern coupled wth the gven nput s recalled. Hopfeld Networs Let x be the ey pattern (vector) appled to an assocatve memory and y be the memorzed pattern (vector). he pattern assocaton performed by the networ s descrbed by, x y, =,,, q where q s the number of patterns stored n the networ. he ey pattern x acts as the stmulus that not only determnes the storage locaton of the memorzed pattern y, but also holds the ey for ts retreval. Input Vector x Pattern Assocator Output vector y Fg. : Input-output relaton of the pattern assocator Auto-assocatve networs consst of one layer of neural elements (unts) that are all nterconnected, although self-connectons are usually dsallowed. he neural elements are nonlnear, wth states bounded from zero to one so called two-state-neurons. Any state, whether ntal or desred, wll be represented by some pattern of zeros and ones over the unts. Recurrence and nonlnearty le at the heart of auto-assocatve networ behavour. Recurrence allows desred states to emerge va nteractons among the unts, and the nonlnearty prevents the whole networ from runnng away, and nstead encourages the networ to settle nto a stable state. he auto-assocatve networ wll, n most cases, relax from any ntal state nto the desred state closest to t. he nterconnectons between unts can be traned accordng to Hebban rules (proposed n 949) - when one cell repeatedly asssts n frng another, the axon of the frst cell develops synaptc nobs (or enlarges them f they already exst) n contact wth the soma of the second cell. Hebban rules are local, n the sense that they depend only upon the actvty of neurons pre- and post-synaptc to any partcular connecton. he orgnal rule proposed by Hebb specfed an ncrease n synaptc strength whenever pre- and post-synaptc neurons were actve together. herefore n an assocaton net, f we compare two patterns components (vectors or pxels) wthn many patterns and fnd that they are frequently n the same state then the arc weght between the two perceptrons must be ncreased (should be postve) and f they are frequently n dfferent states, then the arc weght between the two perceptrons must be decreased (should be negatve). here are two phases nvolved n the operaton of an assocatve memory: Storage phase: hs refers to the tranng of the networ.

4 Applcatons of Neural Networs Recall/Retreval phase: hs nvolves the retreval of a memorzed pattern n response to the presentaton of a nosy or dstorted verson of a ey pattern to the networ. Matrx Representaton: For the computaton a matrx representaton s a practcal tool. Let X = matrx of nput patterns, where each ROW s a pattern. So x, = the -th bt of the -th pattern. Let Y = matrx of output patterns, where each ROW s a pattern. So y, j = the j-th bt of the -th pattern. hen, average correlaton between two nput patterns and j across all patterns s: w = / P(x y + x y + + x y ) (), j,, j,, j p, p, j All weghts for an assocatve memory networ model therefore can be calculated by the followng: W = X Y () herefore, for a set of nput patterns: IP, IP,, IPp, and output patters OP, OP,, OP, p IP x... x IP x... x X = : : : : IP x... x,,n,,n p p, p,n X IP IP... IP p x x... x : :... : x x... x,, p, =,, p, : :... : x x... x,n,n p,n OP y... y... y OP y... y y Y = : : : : OP y... y y,, j,n,,j...,n p p, p, j... p,n From the above sets of vectors (matrces) X and Y, t s obvous that Eqn. () s the dot product of -th row of X and the j-th column of Y. But snce the output vector n case of autoassocaton s one of the earler memorzed vectors t s one of vectors n X tself and therefore the weght calculaton autoassocaton networs s, W = X X ()

5 Consder the followng procedure to explan the assocatve memory networ model. Hopfeld Networs Input: Pattern (often nosy/corrupt) Output: Correspondng pattern (complete/relatvely nose-free) Process:. Load nput pattern onto hghly-nterconnected neurons (a traned networ on gven set of lbrary patterns).. Run the neurons untl they reach a steady state.. Read output off the states of the neurons. Subsequently Fg. and fg. 4 gve a graphcal llustraton of the procedure. Input Output Fg. : raned assocatve networ Input: ( 0 - -) Fg. 4: Runnng an nput pattern Output: ( - - -) Now, let us dscuss Hopfeld networs n the followng secton. 9. HOPFIELD NEWORKS he Hopfeld Net proposed by J.J. Hopfeld n 98 s a neural networ that s a lot smpler to understand than the Mult Layer Perceptron. For a start, t only has one layer of nodes. Unle the MLP, t doesn't mae one of ts outputs go hgh to show whch of the patterns most closely matches the nput. Instead t taes the nput pattern, whch s assumed to be one of the lbrary patterns whch has been changed or corrupted somehow, and tres to reconstruct the correct pattern from the nput that you gve t.

6 Applcatons of Neural Networs Hopfeld networs are Recurrent neural networ model that possesses auto-assocatve property. herefore the networ s fully nterconnected wth the excepton that no neuron has any connecton to tself. hus, the Hopfeld networ s a form of artfcal neural networ that serve as content-addressable memory systems wth bnary threshold unts. hey are guaranteed to converge to a stable state. he am of the Hopfeld networ s to recognze a partal or dstorted nput pattern as one of ts prevously memorzed vectors and to output the perfect and complete memorzed verson. hus the Hopfeld networs proposed as a theory of memory has the followng features:. Dstrbuted Representaton: A memory s stored as a pattern of actvaton across a set of processng elements. Further, the memores can be supermposed on one another; dfferent memores are represented by dfferent patterns over the same set of processng elements.. Dstrbuted Asynchronous Controls: Each processng element maes decsons based only on ts own local stuaton. All these local actons add up to a global soluton.. Content-addressable Memory: A number of patterns can be stored n a networ. o retreve a pattern, we need to only specfy a porton of t. he networ automatcally fnds the closest match. 4. Fault olerance: If a few processng elements msbehave or fals completely, the networ wll stll functon properly. he man concept underlyng the Hopfeld networ s that a sngle networ of nterconnected, bnary-valued neurons can store multple stable states, the lbrary patterns also called attractors, smlar to the bran the full pattern can be recovered f the networ s presented wth only partal nformaton just as descrbed n assocatve memores. Furthermore, there s a degree of stablty n the system, f just a few of the connectons between neurons are severed, the recalled memory s not too badly corrupted, the networ can respond wth a "best guess". he nodes n the Hopfeld networ are vast smplfcatons of real neurons, they can only exst n one of two possble "states" - {frng, not frng} or { 0, } or {, }. Every node s connected to every other node wth some strength (synaptc weghts). At any nstant of tme a node wll change ts state (.e. start or stop frng) dependng on the nputs t receves from the other nodes. If we start the system off wth any general pattern of frng and non-frng nodes then ths pattern wll n general change wth tme. o see ths thn of startng the networ wth just one frng node. hs wll send a sgnal to all the other nodes va ts connectons so that a short tme later some of these other nodes wll fre. hese new frng nodes wll then excte others after a further short tme nterval and a whole cascade of dfferent frng patterns wll occur. One mght magne that the frng pattern of the networ would change n a complcated perhaps random way wth tme. he crucal property of the Hopfeld networ whch renders t useful for smulatng memory recall s the followng: t guarantees that the pattern wll settle down after a long enough tme to some fxed pattern. Certan nodes wll be always "on" and others "off". Furthermore, t s possble to arrange that these stable frng patterns of the networ correspond to the desred memores we wsh to store. Example : Suppose we have traned our Hopfeld Net on the three patterns gven n fg. 5: 4

7 Hopfeld Networs Fg. 5 Next we nput a pattern whch s a bt le one of these, say the left most one gven n the Fg. 6. Leave the networ to run. It gradually alters the pattern we gve t untl t has reconstructed one of the orgnals ones, the rght most one. Fg. 6 he Hopfeld Net s left to terate (loop round and round) untl t doesn't change any more. hen t should match one of the nput patterns. he program should then compare the reconstructed pattern wth the lbrary of orgnals, and see whch one matches. 9.4 SRUCURE OF HOPFIELD NEWORKS he structure of the Hopfeld networs model s shown n Fg. 7. OUPUS (Vald after convergence) INPUS (Appled at tme zero) Fg. 7: Hopfeld Networ Model he nputs to the Hopfeld Net are at the bottom. he nodes produce an output whch comes out at bottom and s fed bac nto all the nodes except the one that produced t (.e. all the nodes refer to each other, but not to themselves) and uses ths to produce the next output. he fnal output of the nodes s extracted at the top. For a networ of N neurons, the state of the networ s denoted by the vector, s = (s, s,, s ) N Each j s = ±, the state of neuron j represents the one bt of nformaton, and the N state vector s represents a bnary word of N bts of nformaton. As the unts/neurons 5

8 Applcatons of Neural Networs have a bnary threshold unts, the dynamcal rule has the possblty of two defntons for a the actvaton of neuron : f w j js j θ, a = (4) otherwse. or, f w j js j θ, a = (5) 0 otherwse. where, w j s the connecton (synaptc) weght from neuron to neuron j as n Eqn. (), s j s the state of neuron (unt) j and θ s the threshold of the neuron. he above relaton n Eqns. (4) and (5) may be wrtten n the compact form, a = sgn[ Σ wj s j] for all j =,,,N where, sgn s the sgnum functon. If a s zero then the acton can be arbtrary. However, the neuron I may reman n the prevous state regardless of whether t s on or off. he feedbac networ wth no self-loopng, Hopfeld networs also have the followng two restrctons on due to the nds of nterconnecton that prevals: ) w = 0 for all neurons..e. there s no loop/connecton to tself, and ) wj w j = for all neurons..e. the connectons are symmetrc. he Hopfeld nets also have a scalar value assocated wth each state of the networ referred to as the energy, E of the networ. Energy s defned as below, E = w s s + θ s (6) < j j j he reason for ths s somewhat techncal but we can proceed by analogy. Imagne a ball rollng on some bumpy surface. We magne the poston of the ball at any nstant to represent the actvty of the nodes n the networ. Memores wll be represented by specal patterns of node actvty correspondng to wells n the surface. hus, f the ball s let go, t wll execute some complcated moton but we are certan that eventually t wll end up n one of the wells of the surface. We can thn of the heght of the surface as representng the energy of the ball. We now that the ball wll see to mnmze ts energy by seeng out the lowest spots on the surface-the wells. In the language of memory recall, f we start/ ntate the networ wth a pattern for frng, the networ must approxmate one of the "stable frng patterns" n the memores. However, t wll "under ts own steam" end up n a nearby (wth respect to the threshold) well n the energy surface thereby recallng the orgnal perfect memory. In the followng secton, we shall focus on the functonalty of Hopfeld networs. 9.5 HE FUNCIONALIY OF HOPFIELD NEWORKS As mentoned earler that Hopfeld nets are a specal case of assocatve networs or content-addressable memory nets. herefore, Hopfeld nets too have the correspondng two phases namely, Storage phase and the Retreval phase. 6

9 Storage Phase: Let us have M, N-dmensonal vectors to be stored n the Hopfeld nets, say P = {p =,,, M}. he m vectors compose the lbrary or the fundamental memores, representng the patterns to be memorzed by the networ. Let p, I denote the -th element of the fundamental memory p. Accordng to the Hebb s rule the synaptc weghts from neuron to j can be computed usng Eqn. ().e. Hopfeld Networs w = / N(p p + p p + + p p ) (7) j,, j,, j M, M, j and w = 0 for all neurons =,,, M. Let W denotes the N N synaptc weght matrx or connectvty matrx of the networ. M (8) W = p p MI N where, pp denoted the dot product of the -th vector and ts transpose. I s the dentty matrx. From the above computaton the followng three ponts are reconfrmed: he output of each neuron n the networ s fed bac to all other neurons. here s no self-loopng n the networ. he weght matrx of networ s symmetrc. Retreval Phase: hs phase s ntated on nput of an N-dmensonal vector as nput say p t, to the traned networ. hs s also called a probe. ypcally the probe has elements ± and s thought to be a nosy or ncomplete verson of any of the attractors already n the lbrary/ fundamental memory. he nformaton retreval starts wth a dynamcal rule n whch each neuron of the networ randomly but at some fxed rate examnes ts actvaton functon a as a result of the connectvty ts neghbourng neurons. Whle examnng f a s greater than zero then the state s changed to else reman n the prevous state. But f the value of a s less than zero then swtch state to. However, f the value of a s equal to zero then the state s left n ts prevous state regardless of whether t was n on or off state. he updatng of the states of the neurons s determnstc but selecton of the neurons for updatng states s done randomly n seral (asynchronous) or synchronous. Startng wth the test nput or the probe vector the networ must fnally reach a state of stablty, or produce tme nvarant state vector as the output pattern.e. y, whose ndvdual elements satsfy the followng stablty condton, N y = sgn w j,p t, j =,,, N. (9) = or the matrx form, y = sgn( Wp t ) (0) Example : Consder the followng example: ) Input Patterns M =, N = 4, 4-dmensonal vector. IP IP IP 7

10 Applcatons of Neural Networs ) he average correlaton across the three patterns to desgn and tran the Hopfeld networ. w IP IP IP Avg, j w /, w /, w, 4 / w, / w, 4 / w, 4 X = X = he weght for the auto assocaton networ s W = X X = Here t s clear that n the connectvty matrx w, 4 = w4, or n general the lower and the upper trangles of the product matrx represents the 6 weghts w = w. We can scale the weghts by dvdng them by p =. For the, j j, teratve purposes t s easer to scale by p at the end nstead of scalng the entre weght matrx W pror to testng. Now let us construct the Hopfeld net and tranng t wth the nputs IP, IP, IP. [ /] 4 [+/] s = 0 for =,,, 4 Fg. 8: Hopfeld networ 8

11 ) Now consder the testng nput = ( 0 0 ). Hopfeld Networs [- /] [+/] Fg. 9: Wth nput ( 0 0 ) 4 s = ; s = 0; s = 0; s 4 = - v) Synchronous teraton to update all the nodes. ( 0 0 ) = ( ) able Inputs Node 4 Output 0 0 -/ / 0 0 / -/ / he dagonal represents the values from the nput layer n the above table depctng the result of the synchronous teraton modeled by the precedng computaton. Now scalng the output by dvdng by p = and usng the threshold θ = 0, we obtan the vector ( / / / / ) ( ) the same as IP an attractor n the lbrary. Consder the followng fgure to dsplay the processng n the Hopfeld net to reach a stable state agan wth the output vector as computed earler. he graphcal representaton of the output s gven n Fg Fg. 0: ( 0 0 ) ( ) 9

12 Applcatons of Neural Networs Now, try some exercses. E) Run the above Hopfeld networ for the test nput vector ( 0 0 0). E) Draw the graph of the nput to output vector obtaned n E). he smart thng about the Hopfeld networ s that there exsts a rather smple way of settng up the connectons between nodes n such a way that any desred set of patterns can be made "stable frng patterns". hus any set of memores can be burned nto the networ at the begnnng. hen f we c the networ off wth any old set of node actvty we are guaranteed that a "memory" wll be recalled. Not too surprsngly, the memory that s recalled s the one whch s "closest" to the startng pattern. In other words, we can gve the networ a corrupted mage or memory and the networ wll "all by tself" try to reconstruct the perfect mage. Of course, f the nput mage s suffcently poor, t may recall the ncorrect memory the networ can become "confused" just le the human bran. We now that when we try to remember someone's telephone number we wll sometmes produce the wrong one! Notce also that the networ s reasonably robust f we change a few connecton strengths just a lttle the recalled mages are "roughly rght". We don't lose any of the mages completely. Example : Consder a Hopfeld networ whose weght matrx s gven by, 0 w = 0 0 Consder a test nput vectors pt = ( ) and pt = ( ). Usng the Eqn. (9) and then ts transpose results n the desred vector. herefore, Wpt 0 4 = 0 4 = 0 4 y = Sgn ( ) = Wpt Snce the CM or the weght matrx s symmetrc we can also drectly get the vector as by the product of vector and the matrx,.e. ptw 0 y sgn ( ) ( ) 0 = ptw = = ( ) 0 Eqn. (0) s also nown as algnment condton. he state vectors that satsfy the algnment condton are stable states. herefore, the gven patterns are stables state vector too. 40 If we consder the followng vectors as probes for ths example, ( ), (,, ) or ( ). he resultng output s ( ). Note that each of the probes had sngle error compared to the stored memory.

13 Smlarly, consder another set of test patterns to nput to the Hopfeld networ ( ), ( ) or ( ). he resultng output vector n ths case s computed to be ( ). In ths case too the test patterns are noted to have sngle error. Hopfeld Networs Now, try the followng exercses. E) Run the Hopfeld networ of Example for the test nput vector ( 0 0). Also, draw the graphcal representaton of the output. E4) Consder the set of pattern vectors P. Obtan the connectvty matrx (CM) for the patterns n P (four patterns) P = In the followng Secton, we shall dscuss the storage capacty Hopfeld Networs. 9.6 HOPFIELD NEWORKS: SORAGE CAPACIY here are two major lmtatons of Hopfeld networs. One s the tendency to local mnma, whch s good for assocaton but bad for optmsaton. he other lmtaton s on the networ capacty. For a networ of N bnary nodes, the capacty lmt s of N the order of N rather than. If the patterns or the vectors are N-dmensonal then there must be networ of N bnary nodes. However, the capacty s equal to the relatonshp between the number of patterns that can be stored & retreved wthout error to the sze of the networ. Let M be the number of patterns to be stored or fundamental memores. So long as the storage capacty s of the networ s not overloaded.e., M s small compared to N. M Capacty = N M or, Number of weghts If we use the followng defnton of 00% correct retreval, when any of the stored patterns s entered completely (no nose), then that same pattern s returned by the networ;.e. he pattern s a stable attractor. A detaled proof shows that a Hopfeld networ of N nodes can acheve 00% correct retreval on P patterns f: P < N /(4 *ln(n)). N Max P In general, as more patterns are added to a networ, the average correlatons wll be less lely to match the correlatons n any partcular pattern. Hence, the lelhood of retreval error wll ncrease. Just as t happens n case of human memory. When the nformaton s loaded/crammed nto a memory, or patterns to store recall s slower 4

14 Applcatons of Neural Networs than when there are less patterns to remember. he ey to perfect recall s selectve gnorance! Example 4: A three neuron networ traned wth three patterns. W, W, W, W, W, O A W, O B W, W, W, O C Let s say we d le to tran three patterns: Pattern number one: O ( ) = O A B( ) = OC ( ) = Pattern number two: O ( ) = O A B( ) = OC ( ) = Pattern number three: O ( ) = O A B( ) = OC( ) = W = 0, W = O O + O O + O O = + + = ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ), A B A B A B W = O O + O O + O O = + + =, A C A C A C W = 0, W = O O + O O + O O = + + = ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ), B A B A B A W = O O + O O + O O = + + =, B C B C B C W = 0, W = O O + O O + O O = + + = ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ), C A C A C A W = O O + O O + O O = + + =, C B C B C B 4

15 Now try the followng exercse. Hopfeld Networs E5) ran ths networ wth the three patterns shown. W, W, W, W, W, O A W, O B W, W, W, O C Patterns: Applcatons of the Hopfeld Networ. he Hopfeld networ can be used as an effectve nterface between analog and dgtal devces, where the nput sgnals to the networ are analog and the output sgnals are dscrete values.. Assocatve memory s a major applcaton of the Hopfeld networ.. he energy mnmzaton ablty of the Hopfeld networ s used to solve optmzaton problems. 4. he varous applcatons of Hopfeld networs are as gven below. Hopfeld networ remembers cues from the past and does not complcate the tranng procedure. 5. he Hopfeld networ can be used for convertng analog sgnals nto the dgtal format, for assocatve memory. Now, let us summarze ths unt. 9.7 SUMMARY he summary of ths unt ncludes the four basc operatons to desgn and run a Hopfeld networ. hese four operatons are learnng, ntalzaton (of the networ), teraton untl convergence, and fnally outputtng the output vector. 4

16 Applcatons of Neural Networs ) Learnng/ranng : Let p, p,, pm be the N-dmensonal patterns to be stored. Usng the dot product rule, the Hebban postulate of learnng, the synaptc weghts of the entre recurrent networ prohbtng self-loopng s computed by, ) ) v) w M = p,p,j j N 0 = j, j = Intalzaton: Let p t be the n-dmensonal vector probe or the test nput to the Hopfeld networ. he ntalzaton s carred out by, s (0) = p, =,,, N, t where s (0) s the state of the -th neuron at tme n = 0, and p,t the -th element of the test nput p t. Iteraton untl convergence: Update the elements of the state vector s(n) synchronously or asynchronously usng the rule, N s (n + ) = sgn wjs (n), =,,, N j= Repeat the teraton untl the state of the neurons do not change.e. the state vector s remans unchanged. Outputtng: Let s fxed denote the stable states computed at the end of above teratve step. he resultng pattern OP of the networ s, OP = s fxed he storage phase uses the learnng/tranng operaton. he subsequent operatons ntalzaton, teraton untl convergence and outputtng are appled n the retreval phase. 9.8 SOLUIONS/ANSWERS E) hs test pattern too has elements other than ±. Usng the Eqn. (4) or ptw and dvdng the resultant by the vector obtaned s ( ). he calculaton s gven as below. Input = ( 0 0 0) he correspondng graphcal representaton of the Hopfeld networ s gven as 4 44 he synchronous teraton to update the nodes s gven by

17 ( 0 0 0) Hopfeld Networs = ( ) Nodes 4 Output / / / Whch s a stable state and one of the nput patterns stored n the lbrary. hough the asynchronous update results n spurous output, the synchronous update gves a pattern from the fundamental memory. E) Input 4 Output 4 E) Frst of all the nput defntely has some serous nose therefore the value 0 whch s not there n the nput patterns s present n the probe. However, use the Eqn. (4) or ptw = ( 0 0), whch s a stable state, but not one of the nput patterns stored n the lbrary. E4) Here N = 0 and M = 4. Use ether the Eqn. (6) or Eqn. (7) to obtan the weghts for each w for all and j =,,, 0 but j. Substtute the values of, j P and P n Eqn. (7) we get 9.9 PRACICAL ASSIGNMEN Sesson 8 Wrte a program n C language to fnd CM = / 4P P 0I 0. ) he average correlaton matrx for gven nput patterns M and N to desgn and tran the Hopfeld Networ. ) he weght matrx. ) Output of the Hopfeld networ. Also test your program on the nput patterns gven n Example. 45