Searching The Performance Surface

Transcription

1 5 Searching The Performance Surface Assoc. Prof. Dr. Peerapol Yuvapoositanon Dept. of Electronic Engineering ASP-1

2 A Single Weight Filter From Ch 3 ASP-2

3 Cost Function J ASP-3

4 The Square Error Cost Function J J ()() w w R w w min opt T opt T ASP-4

5 Multiple Weight J J ()() w w R w w J min min opt T opt [ w w w w w w ] 0 opt,0 1 opt,1 L1 opt, L1 r (0, 0)(0,1)(2, r 0)(0, r 1) r L xx xx xx xx w w r (1, 0)(0, 0)(0,1) r r xx xx xx r (2, 0)(1, 0)(0, r 0) r w w xx xx xx w w L r ( L 1, 0)(0, 0) r xx xx T T 0 opt,0 1 opt,1 1 opt, L1 ASP-5

6 Single Weight For single-weight J J ()() w w R w w min opt J r (0, 0)() w w min xx 0 opt,0 2 () min xx opt J r w w T opt 2 T ASP-6

7 Single Weight From J J r w w () min xx opt 2 For single variable case, the eigenvalue is r xx J J w w min () opt 2 ASP-7

8 Find Wopt from intial w w opt w 0 ASP-8

9 Finding Derivatives J (() J w w w w min 2() w wopt 2 J w 2 2 Constant opt 2 ASP-9

10 Weight at k We d like to find w at k+1 from w at k w w () 1 k k k ASP-10

11 Gradient at k k J w 2() w ww k k w opt ASP-11

12 Substitute the gradient w 2() 1 w w w k k k opt w (1 2) 2) 1 w w k k opt ASP-12

13 w (1 2) w opt w (1 2) w opt w (1 2) 2 3 w 2 w opt ASP-13

14 The Recursive Gradient Search Algorithm We then arrive at* Initial w w (1 2)() k w w k opt 0 opt Step-size (* See derivation in class) Eigenvalue ASP-14

15 Stability Criterion Choice of Step size We arrive at for stability*, the step-size must be 1 0 (* See derivation in class) ASP-15

16 Example 1 Input signal: 2 E{()} x n 1 Desired signal: 2 E{()} d n 4 E{()()} d n x n 1 ASP-16

17 Example 1: One-weight 2 2 E{()} e n {(()()) E d n} y n 2 2 E{()} d n 2()()() d n y n y n d () n 2 E {()()} d n x n w {()} w E x n 4 2(1)(1) w w 2 w 2w 4 rdx 1 rxx 1 2 ASP-17

18 Example 1: One-weight Minimise J 2 e w 2 ( w 2w 4) w w 2w 2 Set to zero: 0 2w 2 opt wopt 1 ASP-18

19 Derivation From opt For Single Weight 1 w R r 2 1 w E {()} x n {()()} E d n x n opt 1 xx dx xx 1 ()() r r r 1 (1)(1) ASP-19

20 Example 1: One-weight w opt =1 ASP-20

21 Plot of w k for various step-sizes Overdamped Critically damped Underdamped ASP-21

22 Effect of r value Over damped Stable Under damped Critically damped 1 ASP-22

23 1-weight Leaning Curves ASP-23

24 Ch5_l.m % ch5_l.m plots weights % 20Feb2016 clear all set(0,'defaultaxesfontsize',20); set(0,'defaultlinelinewidth',3); lambdamax=1.5; % Eigenvalue lambda = lambdamax; w0=[5;5]; mutemp = [ ]; wopt= [-2;2]; N=10; W=[];JN=[];JS=[]; R=1; Jmin =0; J0 = Jmin +(w0-wopt)'* 1*(w0-wopt); R =[1 0.5;0.5 1]; Q=(1/sqrt(2))*[1 1;1-1]; L = [1.5 0;0.5]; v0=w0-wopt; vp0= w0-wopt; for j=1:length(mutemp) mu = mutemp(j); for k=1:n-1 % Newton's Method Jn(k) =Jmin +(1-2*mu)^(2*k)*( v0'* R * v0); % Steepest Descent Method % Put your code here end % Newton's Method Cost Jt=[J0;Jn(:)]; JN =[JN Jt]; % Steepest Descent Cost % Put your code here end figure(1) % Plot Newton's Method Cost clf len=0:n-1; plot(len,jn(:,1),'--',len,jn(:,2),'-',len,jn(:,3),'-.') legend('\mu=0.1 ','\mu=0.5','\mu=0.6') xlabel('$k$','interpreter','latex') ylabel('$j_k$','interpreter','latex') title('newtons Method') figure(2) clf % Plot Steepest Descent Cost % Put your code here ASP-24

25 Two-weight System ASP-25

26 Two-weight System ASP-26

27 Example 2 Two-weight System Input signal: Desired signal: ASP-27

28 ASP-28

29 Surface: Newton s Method w +w0 0 w 1 +2 w 0-2 w 1 +w w w 0 ASP-29

30 Surface: Steepest Descent w +w0 0 w 1 +2 w 0-2 w 1 +w w w 0 ASP-30

31 Assignment Download ch5_l.m Put your codes to generate plots of Newton s Method and Steepest Method. Discuss all the difference between the two methods, i.e., convergence rate, behaviour of the convergence etc. ASP-31

32 Cost: Newton s Method Newtons Method =0.1 =0.5 = ASP-32

33 Cost: Steepest Descent Steepest Descent =0.1 =0.5 = ASP-33