Artificial Neural Network II MATLAB Neural Network Toolbox

Transcription

1 Artificial Neural Network II MATLAB Neural Network Toolbox Werapon Chiracharit Department of Electronic and Telecommunication Engineering King Mongkut s University of Technology Thonburi Input Layer Hidden Layer Output Layer MATrixLABoratory >> A=[ 2 3; 4 5 6] >> B=[ 2; 3 4; 5 6] >> C=A +2*B >> D=A*B /08/ RMUTK 2

2 MATLAB Demos () >>demos Toolboxes\Neural Network\Simple Neuron and Transfer Function Run this demo nnd2n (M GUI) To understand Weight (slope) Bias (y intercept) Transfer function 7/08/ RMUTK 3 Run nnd2n2 To understand 2 input neuron MATLAB Demos (2) 7/08/ RMUTK 4 2

3 Run nnd4db To understand Decision boundary MATLAB Demos (3) 7/08/ RMUTK 5 Run nnd4pr To understand Perceptron MATLAB Demos (4) 7/08/ RMUTK 6 3

4 Collect data Neural Network Design Create a network Configure the network Initialize the weights and biases Train the network Validate the network Test the network 7/08/ RMUTK 7 Line Fitting () m x y y (2,5) c Known: x, y (training set) Unknown: m, c (,2) No. x y targett y calculated = purelin(mx+c) 2 m+c m+c m+c (4,6) x 7/08/ RMUTK 8 4

5 Line Fitting (2) Define cost function or error to be minimized, E = points [y target y calculated ] 2 = [2 (m+c)] 2 + [5 (2m+c)] 2 + [6 (4m+c)] 2 = [2 m c] 2 + [5 2m c] 2 + [6 4m c] 2 From gradient descent, E/ m = 2[2 m c] 4[5 2m c] 8[6 4m c] = m + 4c E/ c = 2[2 m c] 2[5 2m c] 2[6 4m c] = m + 6c 7/08/ RMUTK 9 Line Fitting (3) Adaption with learning rate 0 < < m(t+) = m(t) E/ m c(t+) = c(t) E/ c Initialize m(0) [,] and c(0) [,] randomly and repeat adaption for N iterations 7/08/ RMUTK 0 5

6 Line Fitting (4) % Initialize randomly slope and y intercept >> m=unifrnd(,) m = >> c=unifrnd(,) c = % Learning rate >> alpha=0.0; % Number of iterations >> iteration=0; 7/08/ RMUTK Line Fitting (5) >> x=[ 2 4]; y=[2 5 6]; plot( x, y, 'o'), hold on >> xlabel('x'), ylabel('y'), axis([ ]) 7/08/ RMUTK 2 6

7 Line Fitting (6) % Adaption with gradient descent >> for i=:iteration m=m alpha*( 72+42*m+4*c); c=c alpha*( 26+4*m+6*c); end % Final slope and y=intercept >> m, c m =.3224 c =.729 7/08/ RMUTK 3 Line Fitting (7) >> x2=0:0.0:5; y2=m*x2+c; plot(x2, y2) >> title(strcat('m=', num2str(m), ', c=', num2str(c), t() ', iteration=', ti num2str(iteration))) ti 7/08/ RMUTK 4 7

8 Line Fitting (8) Convergence for, 2, 4, 6, 8 and 0 iterations 7/08/ RMUTK 5 p p 2 Perceptron for OR Problem () b n Known: p, p 2, t Unknown: w, w 2, b a n < 0 p 2 n > 0 (0,) (,) (0,0) (,0) p p 2 t a = hardlim(w p +w 2 p 2 +b) hardlim(b) 0 hardlim(w 2 +b) 0 hardlim(w +b) hardlim(w +w 2 +b) 7/08/ RMUTK 6 p 8

9 Perceptron for OR Problem (2) Define error function, E = i= 4 (t i a i ) 2 = [ 0 hardlim(b) ] 2 + [ hardlim(w 2 +b) ] 2 + [ hardlim(w +b) ] 2 + [ hardlim(w +w 2 +b) ] 2 Gradient descent, E/ w = [ hardlim(w +b) ] 2[ hardlim(w +w 2 +b) ] = 2 hardlim(w +b) + 2 hardlim(w +w 2 +b) 4 E/ w 2 = 2 hardlim(w 2 +b) + 2 hardlim(w +w 2 +b) 4 7/08/ RMUTK 7 Perceptron for OR Problem (3) E/ b = 2 hardlim(b) + 2 hardlim(w 2 +b) + 2 hardlim(w +b) + 2 hardlim(w +w 2 +b) 6 Adaption with 0 < < w (t+) = w (t) E/ w w 2 (t+) = w 2 (t) E/ w 2 b(t+) = b(t) E/ b Initialize weights [,] and bias [,] randomly and repeat adaption until t a 7/08/ RMUTK 8 9

10 Perceptron for OR Problem (4) % Initialize weights and bias >> w=unifrnd(,), w2=unifrnd(,) w = w2 = >> b=unifrnd(,) b = >> alpha=0.; % Learning rate >> gamma=0; % Threshold >> plot( [0 ], [ 0 ], '+'), hold on 7/08/ RMUTK 9 Perceptron for OR Problem (5) >> plot(0, 0, 'o') >> axis([ ]), title('input space') >> xlabel('p()') >> ylabel('p(2)') % Define target and output >> T=[0 ]; >> y=[ ]; >> iteration=; 7/08/ RMUTK 20 0

11 Perceptron for OR Problem (6) % Adaption >> while sum(abs(t y)) > gamma a=hardlim(b); a2=hardlim(w2+b); a3=hardlim(w+b); a4=hardlim(w+w2+b); y=[a a2 a3 a4]; E(iteration)=sum((T y).^2); w=w alpha*(a3+a4 2); w2=w2 alpha*(a2+a4 2); b=b alpha*(a+a2+a3+a4 3); iteration=iteration+; 7/08/ end RMUTK 2 Perceptron for OR Problem (7) >> w, w2, b, iteration w = w2 = b = iteration = 8 >> p= 0.5:0.:.5; >> p2= (w*p+b)/w2; >> plot(p, p2), hold off 7/08/ RMUTK 22

12 Perceptron for OR Problem (8) % Error >> figure, plot(e, 'o ') >> xlabel('iteration'), lbl('it ti ') ylabel('mean lbl('m square error') 7/08/ RMUTK 23 Data Structures () Input/target data format for training and simulation e.g. 2 input neuron and 4 elements for each input p 2 p = p 2 = t = W 2 b n a = f(wp+b) f 7/08/ RMUTK 24 2

13 Data Structures (2) Batch training is to update the weights after each presenting the complete data set >> p=[0 0 ; 0 0 ]; >> T=[0 ]; Incremental training is to update the weights after the presentation of each single sample >> p={[0;0][0;] [;0] [;]}; >> T={[0] [] [][]}; 7/08/ RMUTK 25 Neural Network Function () e.g. NAND problem (command line in m file) % Define input P (4 input vectors) and target T >> P=[0 0 ; 0 0 ]; >> T=[ 0]; >> plotpv(p, T) % Plot perceptron vector 7/08/ RMUTK 26 3

14 Neural Network Function (2) % Create a perceptron network, input ranges N 2, number of neuron, transfer function >>net=newp([0 ; 0 ],, 'hardlim'); >> net.iw{} % Initial weight ans = 0 0 >> net.b{} % Initial bias ans = 0 7/08/ RMUTK 27 Neural Network Function (3) % Number of adaption passing through the entire sequence >> net.adaptparam.passes=5; p % Training with adaption algorithm, output, error >> [net, y, e]=adapt(net, P, T); >> y, e,mse(e) % percent of training y = 0 e = ans = 0 7/08/ RMUTK 28 4

15 Neural Network Function (4) % Plot perceptron classification by p() p(2)+ >> plotpc(net.iw{}, net.b{}) >> net.iw{} ans = % Final weight >> net.b{} ans = % Final bias 7/08/ RMUTK 29 Neural Network Function (5) % Simulation >> P2=[0.5; 0.5]; >> y2=sim(net, P2); >> plotpv(p2, y2) >> hold on >> plotpv(p, T) >> plotpc(net.iw{}, net.b{}) 7/08/ RMUTK 30 5

16 References MathWorks Neural Network Toolbox webpage, html 7/08/ RMUTK 3 Thank you for attention Q & A 6