Unsupervsed Learnng Kevn Swngler What s Unsupervsed Learnng? Most smply, t can be thought of as learnng to recognse and recall thngs Recognton I ve seen that before Recall I ve seen that before and I can recall more about t from memory. There s no feedback or reward lke there s wth renforcement learnng There s no gven answer lke there s n supervsed learnng 1
In Context Supervsed learnng: These are apples, these are pears What s ths new thng (apple or pear?) Renforcement learnng: Lke the warmer, colder game actons are gven rewards or feedback, whch s learned for future use Unsupervsed learnng: Rememberng the route home n day lght and stll beng able to do t at nght, or n the snow when thngs look dfferent, or when parked cars have moved Content Addressable Memory One way to thnk about unsupervsed learnng s as a content addressable memory That s, you look thngs up, not by searchng, but by descrbng some aspects of the thng you are lookng for and havng that trgger other thngs about t 2
Assocatve Patterns The mportant aspect of assocatve memory s that t stores patterns, or thngs that go together Just as an exotc smell mght evoke memores of a holday, assocatve memores work by completng partal patterns A Smple Assocatve Memory The Hopfeld Network Stores patterns n an assocatve memory Can recall a complete pattern when gven only a part of that pattern as nput Robust under nose wll recall the nearest pattern t has to the nput stmulus Robust under damage remove a few of the connectons and t stll works 3
Characterstcs of a Hopfeld Network A collecton of nodes, whch we wll call neurons (though they are really ust smple mathematcal functons) Each neuron s connected to every other neuron n the network (but not tself) we call the connectons synapses Synapses have a weght that s ether exctatory (+ve) or nhbtory (-ve) Weghts are symmetrcal: W = W Neurons can be ether on or off represented as an output value of +1 or -1 The neurons are of the McCulloch and Ptts type that we have already seen. Recall n a Hopfeld Network The output (+1 or -1) from a neuron s calculated based on the ncomng weghts and the values carred along those weghts The output from each neuron s multpled by the weght on each connecton leavng that neuron to contrbute to the nput to ts destnaton node So the nput to a neuron s the sum of the product of the ncomng weghts and ther values 4
The Value of a Hopfeld Neuron u = w v + I 1 v = 1 u 0 u < 0. u s the sum of the product of the weghts, w and the outputs from ther pre-synaptc neurons, v plus the nput to the neuron tself (I) v, the value of the neuron (ts output) s ether +1 or -1 dependng on the sgn of u (+ve or ve) Convergence You mght thnk that as each neuron s output value changes, t affects the nput to, and so the output from other neurons, whch change all other neurons and the whole system keeps changng forever But t doesn t Gven an nput pattern, a Hopfeld network wll always settle on a fxed state after a number of teratons Ths state s known as the attractor for the gven nput and t represents the pattern that has been recalled 5
As the Network Settles As the network settles n ts steady state, output from the neurons becomes consstent wth the weghts Ths consstency can be measured by multplyng the values of each par of neurons wth the weght of the synapse between them and summng the result: w v v Smple Example Take a sngle par of neurons: 1 Wth a connecton strength of 1: 1 x 1 x 1 = 1-1 x -1 x 1 = 1-1 x 1 x 1 = -1 6
Stable States The sum of the product of the weghts and the values gets larger as the network approaches an attractor In the prevous example, 1,1 and -1,-1 one attractors for the gven network Set them as nputs, and the network wll not change Set -1, 1 or 1, -1 as nputs, and t wll change to 1,1 or -1,-1 Energy Functon A network n a stable state s sad to have low energy A network n transton has hgher energy So energy s somehow the opposte of the consstency measure we saw earler In fact, f we make that measure negatve so that low values correspond wth stable states, we get 1 E = 2 w v v We halve the value because we are countng the b-drectonal synapses twce So, recall n a Hopfeld network s the same as mnmsng the energy functon 7
Another Example Look at the smple network below -1 Stable states are 1,-1 and -1,1 Put t n 1,1 or -1,-1 and one neuron wll flp Ths s a so-called flp-flop network It s energy s Whch s Whch s plotted below: Flp Flop 1 E = ( v1v 2) + ( v2v1) 2 E = ( v v ) + ( v v ) / = v v 1 2 2 1 2 1 2 E 1,-1-1,1 8
Pattern Recall So a Hopfeld network mnmses ts energy functon and settles nto an attractor state, whch represents the full pattern that the nputs are closest to Learnng n a Hopfeld Network Now we turn to the queston of learnng n a Hopfeld network how do the weghts get set to store the memores? The goal s to ensure that the energy of the network at each of the patterns s locally mnmal (.e. the lowest near that pattern) 9
Hopfeld Learnng To mnmse the energy, we present a pattern p to a network and update the weghts thus: w w + p p Smply update the weght between two neurons by the product of ts two neurons values n each pattern So neurons that are both present n many patterns have larger weghts! Learnng Fnally, when all the patterns have been loaded, we dvde each weght by the number of patterns there are to normalse the weghts We also make sure the leadng dagonal of the weghts matrx contans zeros to ensure no neuron lnks to tself What does ths remnd you of? Hebb s rule when two neurons are on, strengthen the weght between them 10
Example Learnng Images Tran a network wth clean mages: Present corrupted text as nputs: Network produces clean characters as outputs: 11