A Perceptron is a binary classifier that maps its input x (a real-valued vector) to an output value y (y single binary value, 0 or 1; -1 or 1)

Transcription

1 Percepron

2 A Percepron i a inary claifier ha map i inpu (a real-valued vecor) o an oupu value y (y ingle inary value 0 or ; - or ) Roenla [Roe6] creaed many variaion of he percepron. One of he imple: ingle-layer neork hoe eigh and iae could e rained o produce a correc arge vecor hen preened ih he correponding inpu vecor. The raining echnique ued i called he percepron learning rule. The percepron generaed grea inere due o i ailiy o generalize from i raining vecor and learn from iniially randomly diriued connecion. Percepron are epecially uied for imple prolem in paern claificaion. They are fa and reliale neork for he prolem hey can olve.

3 Percepron Inpu vecor... N T Weigh vecor... N

4 0 0 0 if y 0 0 if y The inpu pace of a o-inpu hard limi neuron

5 The inpu pace of a o-inpu hard limi neuron L - deciion oundary For = 0 L pae hrough he origin 0 L W pick eigh and ia value o orien and move he dividing line o a o claify he inpu pace a deired 0 L: y y 0

6 Learning Rule (Training algorihm) A learning rule (raining algorihm) i defined a a procedure for modifying he eigh and iae of a neork. Supervied learning Unupervied learning In upervied learning he learning rule i provided ih a e of eample (raining e) of proper neork ehavior { { - i an inpu o he neork (vecor) - he correponding correc (arge) oupu q q Q Q } }... { }... A he inpu are applied o he neork he neork oupu are compared o he arge. The learning rule i hen ued o adju he eigh and iae of he neork in order o move he neork oupu cloer o he arge. The percepron learning rule fall in upervied learning caegory. { }

7 Learning Rule (Training algorihm) - con. The ojecive i o reduce he error e hich i he difference eeen he neuron repone y and he arge vecor. e = y CASE. If an inpu vecor i preened and he oupu of he neuron i correc (y = and e = y = 0) hen he eigh vecor i no alered. CASE. If he neuron oupu i 0 and hould have een (y = 0 and = and e = y = ) he inpu vecor i added o he eigh vecor. Thi make he eigh vecor poin cloer o he inpu vecor increaing he chance ha he inpu vecor ill e claified a a in he fuure. CASE. If he neuron oupu i and hould have een 0 (y = and = 0 and e = y = ) he inpu vecor i uraced from he eigh vecor. Thi make he eigh vecor poin farher aay from he inpu vecor increaing he chance ha he inpu vecor ill e claified a a 0 in he fuure.

8 Learning Rule (Training algorihm) - con. CASE. If e = 0 hen make a change Δ equal o 0. CASE. If e = hen make a change Δ equal o T. CASE. If e = hen make a change Δ equal o T. Δ = ( y) T = e T You can ge he epreion for change in a neuron ia y noing ha he ia i imply a eigh ha alay ha an inpu of : Δ =( y) = e The percepron learning rule can e ummarized a follo: ne ne old old e e T e y

9 Learning Rule (Training algorihm) - con. The proce of finding ne eigh (and iae) can e repeaed unil here are no error. The percepron learning rule i guaraneed o converge in a finie numer of ep for all prolem ha can e olved y a percepron. Thee include all claificaion prolem ha are linearly eparale. The ojec o e claified in uch cae can e eparaed y a ingle line nnd4pr mala demo - deciion oundarie - percepron rule

10 0 0 0 : y: e: : y: e: : y: e: Invaare dupa Invaare dupa [ ] [ ] [ ] [ 0.488] [ 0.55] [ ] [0.68.7] ( ) [ ] [ 0.55] [0.488] ( )

11 Prolema Se conideră un percepron cu două inrări și uiliza inr-o aplicaie de claificare inara. Penru inruirea percepronului e foloee un e de anrenare de dimeniune după cum urmează: La momenul iniial -au genera aleaor vecorul ponderilor = [ -0.5] și polarizarea = -. Percepronul ee anrena uilizand regula de invaare upervizaa a percepronului. a) Reprezenai grafic cei doi vecori de inrare și linia de eparare (deciion oundary) definia de percepronul iniial (neinrui) ) Cum un claificai cei doi vecori de inrare? c) Deerminai noile valori ale ponderilor i polarizarii percepronului in urma inruirii cu primul vecor din eul de anrenare. Reprezenai grafic noua linia de eparare (deciion oundary). Cum un claificai acum cei doi vecori de inrare? d) Deerminai noile valori ale ponderilor i polarizarii percepronului in urma inruirii percepronului oinu la puncul anerior uilizand al doilea vecor din eul de anrenare. e) Reprezenai grafic noua linia de eparare (deciion oundary) i deerminai daca cei doi vecori din eul de anrenare un claificai corec.