Week 1, Lecture 2. Neural Network Basics. Announcements: HW 1 Due on 10/8 Data sets for HW 1 are online Project selection 10/11. Suggested reading :

Transcription

1 ME 537: Learig-Based Cotrol Week 1, Lecture 2 Neural Network Basics Aoucemets: HW 1 Due o 10/8 Data sets for HW 1 are olie Proect selectio 10/11 Suggested readig : NN survey paper (Zhag Chap 1, 2 ad Sectios 4.1 to 4.5 i Passio Orego State Uiversity Learig From Data! y = f(x = a x + b Liear regressio (parametric

2 Learig From Data! y = f(x = a x + b???! y = f(x = a x 2 + b x + c Polyomial regressio (parametric Learig From Data! y = f(x = a x + a -1 x a 2 x 2 + a 1 x + a 0 Polyomial regressio (parametric

3 Learig From Data! y = f(x =?? Neural Networks for Noliear Cotrol! Motivatio:! Cotrol a system with oliear dyamics! Robot! Satellite! Air vehicle! Do we kow what the good cotrol strategies are?! Yes: teach eural etwork those strategies! Drive a car ad record good driver actios for each state! Fly a helicopter ad record good pilot actios for each state! No: have a eural etwork discover those strategies! Let car drive aroud ad provide feedback o performace

4 Neural Networks! Why Neural Networks?! Uits or Neuros! Neural Network Architectures! Activatio Fuctios! Sigle Layer Feed Forward Networks! Multi Layer Feed Forward Networks! Error Backpropagatio! Implemetatio Issues Why Neural Networks?! Neural Network: A massively parallel distributed processor made up of simple processig uits. It stores kowledge.! A artificial eural etwork is similar to the brai i that:! Kowledge is acquired by the etwork from its eviromet through a learig process! Itereuro coectio stregths (syaptic weights are used to store the acquired kowledge! A artificial eural etwork is differet from the brai i a thousad ways!! Thik of a eural etwork as a statistical tool.

5 Beefits of Neural Networks! Performs a iput/output mappig! Ca be traied from examples Noliear regressio ++ Fuctioal form of mappig eed ot be kow! Is adaptive to chagig eviromets! Provides probabilistic respose! Results i fault tolerat computig Track ostatioarity Cofidece i solutio Graceful degradatio Iput / Output Mappig! Supervised learig: learig with a set of labeled examples! Each example has:! A iput! A desired output! Traiig:! Preset iput! Compute output! Compare etwork output to desired output! Update etwork weights to miimize error! Whe weights are stable etwork has leared a iput/output mappig

6 Types of Learig! Learig Rules:! Hebbia! Memory Based! Competitive! Gradiet descet! Learig Paradigms:! Supervised! Critic (Reiforcemet Learig! Usupervised Hebbia Learig! If two euros are activated at the same time, stregthe the weight betwee them (Hebb, 1949! Properties:! Highly Local! Time depedet! Iteractive! Appeal: Evidece for biological plausibility

7 Memory Based Learig! Explicitly store experieces (patters i memory! Whe a ew patter is observed:! Fid stored patters i eighborhood of test patter! Example: Nearest Neighbor algorithm! For each ew usee patter, fid closest (or closest K patters i memory! Assig ew patter to class most frequetly represeted i the eighborhood! Slow recall (search through all stored patters Competitive Learig! Oly euros wiig some competitio are updated! Basic elemets:! All euros start the same! There is a limit o the total stregth of each euro! A mechaism for euros to compete. Wier is called wiertakes-all euro! Example: euros represet cocetratios of data! For each patter, the wiig euro is modified to be closer to that particular patter! Neuros form clumps to represet the differet data clusters

8 Gradiet Descet! Update weights to miimize error! Take steps proportioal to the egative of the derivative! More later Model of a Neuro! Each iput is a product of some sigal (output ad a weight! All icomig iputs are summed! Sum goes through a activatio fuctio! Output is set out to the etwork

9 Activatio Fuctios Neural Network Architectures

10 Neural Network Architectures Sigle Layer Feed Forward Networks! Iput x is a m elemet iput vector! Target t is the desired output (ca be a vector! Output y is respose to x! Error e is differece betwee desired ad etwork outputs x 1! x 2! x m! w 1! w 2! w m!?! y! w 0! e = t " y

11 Sigle Layer Feed Forward Networks! Liear Discrimiatio: m y = " w k x k + w 0 k x 1! x 2! x m! w 1! w 2! w m! " w 0! y!! Logistic Discrimiatio: m y = f (" w k x k k x 1! x 2! x m! w 1! w 2! w m! f (" y! Sigle Layer Feed Forward Networks! Liear Discrimiatio: m x 1! x 2! w 1! w 2! y = " w k x k + w 0 k w w 0! x m! m! " y!! Logistic Discrimiatio: m y = f (" w k x k + w 0 k x 1! x 2! x m! w 1! w 2! w m! f (" w 0! y!

12 Learig From Data: Represetatio x a!" y b 1! y = f(x = a x + b Learig From Data: Represetatio x a!" y x 2 b c 1 1 h(x a x!" y 1 b c h(x = x 2 1 w 1 h(x w 3! y = f(x = a x 2 + b x + c x w 5!" y w 2 1 w 6 h(x w 4 h(x = 1/(1+e -x w 0 1

13 Learig From Data: Represetatio w 1 h(x w 3! y = f(x =?? x w 5!" y w 2 1 w 6 h(x w 4 h(x = 1/(1+e -x w 0 1 Sigle Layer Feed Forward Networks! patters (x,t! Mea Square Error: E = 1 2 N # =1 (t " y 2

14 Sigle Layer Feed Forward Networks! Error o patter : e = t " y Sigle Layer Feed Forward Networks! Error o patter : e = t " y! Mea Square Error: E = 1 2 N # =1 (t " y 2

15 Sigle Layer Feed Forward Networks! Error o patter : e = t " y! Mea Square Error: E = 1 2 N # =1 (t " y 2! Least Mea Square algorithm: "E "w = "e e "w Sigle Layer Feed Forward Networks! Error o patter : e = t " y! Mea Square Error: E = 1 2 N # =1 (t " y 2! Least Mea Square algorithm:! Gradiet descet: "E "w = "e e "w "w = #$ %E %w

16 Gradiet Descet: Move i directio of egative derivative E(w Decreasig E(w w 1 d E(w/ dw 1 w 1 d E(w/dw 1 > 0 w 1 <= w 1 - # d E(w/dw 1 i.e., the rule decreases w 1 Gradiet Descet: Move i directio of egative derivative E(w Decreasig E(w w 1 d E(w/ dw 1 w 1 d E(w/dw 1 > 0 w 1 <= w 1 - # de(w/dw 1 i.e., the rule decreases w 1

17 Gradiet Descet: Move i directio of egative derivative E(w Decreasig E(w w 1 Gradiet Descet: Move i directio of egative derivative E(w Decreasig E(w w 1 d E(w/ dw 1 w 1 d E(w/dw 1 < 0 w 1 <= w 1 - # de(w/dw 1 i.e., the rule icreases w 1

18 Sigle Layer Feed Forward Networks! Liear activatio fuctio: x 1! x 2! w 1! w 2! " y! x m! w m! w 0! Sigle Layer Feed Forward Networks! Liear activatio fuctio: "E = e i "w i, x 1! x 2! x m! w 1! w 2! w m! " w 0! y! "e i "w i, = #e i "y i "w i, = #e i x

19 Sigle Layer Feed Forward Networks! Liear activatio fuctio: x 1! x 2! w 1! w 2! " y! "E "e = e i i "w i, "w i, = #e i "y i "w i, = #e i x x m! w m! w 0!! Weight update: "w i, = #$ %E %w i, = $ e i x ME 537: Learig-Based Cotrol Week 2, Lecture 1 Neural Network Basics II Aoucemets: HW 1 Due o 10/8 Data sets for HW 1 are olie Proect selectio 10/11 Suggested readig : Sectios 9.2 & 11.1 i Passio Orego State Uiversity

20 Sigle Layer Feed Forward Networks! Sigmoid activatio fuctio:! Derivative of sigmoid: f (a = 1 1+ e "a f "(a = f (a(1# f (a Sigle Layer Feed Forward Networks! Sigmoid activatio fuctio:! Derivative of sigmoid: f (a = 1 1+ e "a f "(a = f (a(1# f (a! Gradiet descet: "E "e = e i " i = #e i f ( $ w i, x "w i, "w i, "w i, = #e i f ( $ w i, x (1# f ( $ w i, x x = #e i y i (1# y i x

21 Sigle Layer Feed Forward Networks! Sigmoid activatio fuctio:! Derivative of sigmoid: f (a = 1 1+ e "a f "(a = f (a(1# f (a! Gradiet descet: "E "e = e i " i = #e i f ( $ w i, x "w i, "w i, "w i, = #e i f ( $ w i, x (1# f ( $ w i, x x = #e i y i (1# y i x Sigle Layer Feed Forward Networks! Sigmoid activatio fuctio:! Derivative of sigmoid: f (a = 1 1+ e "a f "(a = f (a(1# f (a! Gradiet descet: "E "e = e i " i = #e i f ( $ w i, x "w i, "w i, "w i, = #e i f ( $ w i, x (1# f ( $ w i, x x = #e i y i (1# y i x! Weight update: "w i, = #$ %E %w i, = $ e i y i (1# y i x

22 Illustratio of Gradiet Descet E(w w 1 w 0 Illustratio of Gradiet Descet E(w w 1 w 0

23 Illustratio of Gradiet Descet E(w w 1 Directio of steepest descet = directio of egative gradiet w 0 Illustratio of Gradiet Descet E(w w 1 Origial poit i weight space w 0 New poit i weight space

24 Multi Layer Feed Forward Networks x 1! v 1,1! h 1! x i! h! y k! v i,! w,k! Multi Layer Feed Forward Networks x 1! v 1,1! h 1! x i! h! y k! y k = f (" w,k h + w 0,k v i,! w,k! f (a = 1 1+ e "a = f (" w,k # f (" v i, x i + v 0, + w 0,k i

25 Weight updates! Derivative of error wrt weight,k: "E = e k "w,k "e k "w,k "f = #e k k "w,k = #e k f ( $ w,k h (1# f ( $ w,k h h = #e k y k (1# y k h Weight updates! Derivative of error wrt weight,k: "E = e k "w,k "e k "w,k "f = #e k k "w,k = #e k f ( $ w,k h (1# f ( $ w,k h h = #e k y k (1# y k h! Updatig hidde-output layer weights:! Hidde-output layer deltas: "w,k = #$ %E %w,k = $& k h " k = e k y k (1# y k

26 Multi Layer Feed Forward Networks x 1! v 1,1! h 1! x i! h! y k! v i,! w,k! h = f (" v i, x i + v 0, i Multi Layer Feed Forward Networks x 1! v 1,1! h 1! x i! h! y k! v i,! w,k! What are the errors for the hidde layer? h = f (" v i, x i + v 0, i We do t kow the Targets. Now what?

27 Error Backpropagatio! Updatig iput-hidde layer weights: "v i, = #$ x i! Delta: " = e f (# v i, x i (1$ f (# v i, x i = e h (1$ h # i i ( = w,k " k % h (1$ h k Error Backpropagatio! Updatig iput-hidde layer weights: "v i, = #$ x i! Delta: " = e f (# v i, x i (1$ f (# v i, x i = e h (1$ h # i ( = w,k " k % h (1$ h k i Errors for the hidde layer: Backpropagated deltas from output layer

28 Backpropagatio Summary:! For sigmoidal activatio fuctios, update ay weight coectig iput i to output : "w i, = #$ x i Backpropagatio Summary:! For sigmoidal activatio fuctios, update ay weight coectig iput i to output :! Deltas give by: "w i, = #$ x i! For output layer " = e y (1# y

29 Backpropagatio Summary:! For sigmoidal activatio fuctios, update ay weight coectig iput i to output :! Deltas give by: "w i, = #$ x i! For output layer " = e y (1# y! For hidde layer " = $ w,k " k # h (1% h k Backpropagatio Algorithm! For each epoch:! Preset patter x to etwork! Propagate sigal forward:! Compute hidde uit values! Compute output value! Fid error! Compute output layer deltas! Compute hidde layer deltas! Compute gradiet for each weight! Update each weight! Preset ext patter! Repeat this process util MSE is satisfactory

30 Radial Basis Fuctio Networks Key RBF differeces: x 1! R 1!!! Local activatio Liear output layer recommeded x i! R! y k!! Euclidea orm activatio! All hidde uits are differet fuctios w,k!! Oe hidde layer Radial Basis Fuctio Networks R 1! x 1! $ i! x i! R! y k! c i! w,k!! c is the ceter of the th radial basis fuctio c = {c,1,,c,i,, c,n }! s is the radius of the th radial basis fuctio

31 Radial Basis Fuctio Networks R 1! x 1! $ i! x i! R! y k! c i! w,k! $ R (x = exp " x " c 2 ' & & 2(# 2 % (! c is the ceter of the th radial basis fuctio c = {c,1,,c,i,, c,n }! $ is the radius of the th radial basis fuctio Radial Basis Fuctio Networks R 1! x 1! $ i! x i! R! y k! c i! w,k! y k = f (" w,k R + w 0,k & & = f w,k # exp $ x $ c 2 ( ( + ( 2(% w + " 0,k ( + ' ' * *

32 RBF Ceter Updates! For quadratic distace: x " c 2 = (x " c,1 2 +!+ (x " c,i 2 +!+ (x " c,n 2! Ceter updates: "R "c,k = % " exp # x # c 2 ( ' * ' 2($ 2 * & "c,k % = exp # x # c 2 ( ' * ' 2($ 2 * (#2(# 1 2($ & (x 2 k # c,k = R (x k # c,k ($ 2 RBF Ceter Updates! For sigle output (y liear output layer: "E = "E "e "y "R "c,k "e "y "R "c,k = e(#1w "R "c,k = e(#1w R (x k # c,k ($ 2! Updatig Ceters: "c,k = #$ %E (x = $ e w R k # c,k %c,k (& 2

33 Implemetatio issues! Traiig, Testig ad Validatio! Network architecture ad Traiig! Iitial weights & Parameter selectio! Local miima! Mometum term for weights! Network complexity! Covergece! Geeralizatio! Model complexity! Uiversal approximator theorem Traiig, Testig ad Validatio!!!!! Traiig: usig kow samples to set parameters Testig: Verifyig that leared mappig applies to usee samples Validatio: testig o the traiig samples to set parameters Geeralizatio: Ability to exted learig to ew samples Example: 1000 data poits! Use 600 for traiig: set parameters! Use 200 for validatio: check performace, adust parameters! Use 200 for testig: Geeralizatio performace! Cross-validatio: Trai ad validate o data partitios.! 4-fold cross validatio meas split data ito four ad trai o three quarters ad validate o oe fourth for each combiatio! All traiig data used for traiig (200 data poits ot used above! Validatio results still valid (four validatio sets above

34 Network Architecture ad Traiig! Architecture:! Feed forward etwork! 2 Layer FFN! Neuro selectio! Activatio fuctios! Learig Algorithm! Gradiet descet! How may hidde uits?! How log should traiig last? Iitial Weights! Iitial Weights! Radom! Seed special cocept! Clusterig (for RBF etworks

35 Local Miima! Ca have multiple local miima! Gradiet descet goes to the closest local miimum:! solutio: radom restarts from multiple places i weight space Mometum Term! Weight update chages too fast with: "w i, = #$ x i! Let each update be closer to last update. Give the gradiet mometum : "w i, = #"w $1 i, + %& x i

36 Covergece! Preset time:! Trai for 2000 epochs! Preset error criteria:! Trai util MSE reaches! Relative error criteria:! Trai till MSE chages by less tha.1% per epoch! Use some left out patters to validate traiig. Whe validatio error bottoms out, stop traiig. Geeralizatio error Traiig set error Test set error Traiig time! Traiig set error reduced cotiuously! Test set error (geeralizatio error icreases after a poit! Network starts to lear the oise i the traiig data

37 Model Complexity Uiversal Fuctio Approximatio How good a approximator is a multi layer feed forward etwork? Uiversal Approximatio Theorem: Uder some assumptios, for ay give costat e ad cotiuous fuctio f (x 1,...,x m, there exists a three layer MLP with the property that f (x 1,...,x m - H(x 1,...,x m < e where H ( x 1,..., x m =! i v i h (! w i x + b i h(. is oliear activatio fuctio