ECE580 Solution to Problem Set 4

Transcription

1 ECE580 Fall 05 Solution to Problem Set 4 December 8, 05 ECE580 Solution to Problem Set 4 These problems are from the textbook by Chong and Zak, 4th edition, which is the textbook for the ECE580 Fall 05 semester. As such, many of the problem statements are taken verbatim from the text; however, others have been reworded for reasons of efficiency or instruction. Solutions are mine. Any errors are mine and should be reported to me, skoskie@iupui.edu, rather than to the textbook authors. Please note that in the interest of time, I have neglected the appropriate checks and algorithm terminations in the Matlab scripts. These have been presented in other homework solutions. 9. Let f : R R be given by f(x) = (x x 0 ) 4, where x 0 R is a constant. Apply Newton s method to identify a minimizer of f. (a) Determine the update equation. Solution: The update equation for Newton s algorithm is x (k+) = x (k) F(x (k) ) (x x (k) ). Thus, for this problem, the update equation is x (k+) = x (k) ((x (k) x 0 ) ) 4(x k x 0 ) 3, () = x (k) x(k) x 0, () 3 = (x (k) + x 0 )/3. (3) (b) Let y (k) = x (k) x 0. Show that integer k 0, y (k+) = y (k) /3. Solution: For arbitrary integer k y (k+) = x (k+) x 0 (4) = x (k) /3 + x 0 /3 x 0 = y (k) /3. (5) (c) Show that the sequence of x (k) values converges to x 0 for any x (0). Solution: The expression derived in part (b) for y (k) implies that the error at iteration k, y (k) = ( 3) k y (0), which goes to zero as k goes to infinity. (d) Show that the order of convergence of the sequence {x k } in part (b) is. Solution: Actually the sequence in part (b) is {y k }, which has convergence of order by Theorem 8.5. (e) Why does Theorem 9. not imply that convergence the sequence {x k } is of order? Solution: The theorem applies only when f C 3, f(x ) = 0 and F(x ) is invertible. While f(x ) = 0, F(x ) is zero, so not invertible.

2 ECE580 Fall 05 Solution to Problem Set 4 December 8, Application of Newton s Method Given (a) Determine the update equation. Solution: f(x) = x 4/3. Thus Newton s algorithm gives the update f (x) = 4 3 x/3 (6) F (x) = 4 9 x /3. (7) x (k+) = x (k) 9 4 (xk ) /3 4 3 (x(k) ) /3 = x (k) 3x (k) = x (k). (b) Show that unless x (0) = 0, the algorithm does not converge to the minimizer x = 0. Solution: Applying the algorithm repeatedly, we obtain x (k) = ( ) k k x (0), so the magnitude of x (k) increases without bound as k goes to infinity. 0.0 Given f(x) = 5 x + x + x x 3x x. (a) Express f(x) in terms of a quadratic matrix equation. Solution: f(x) = [ ] [ ] [ ] 5 x x x [ ] [ 3 x x x (b) Find the minimizer using the conjugate gradient algorithm with x (0) = 0. Solution: Using the Matlab script x0 = [0 0] ; Q = [5 ; ]; b = [3 ] ; xk = x0; Q*xk -b dk = -gk for index = :0, disp([ Index,numstr(index)]) ak = -(gk *dk)/(dk *Q*dk) xk = xk+ak*dk Q*xk - b bk = (gk *Q*dk)/(dk *Q*dk) dk = -gk+bk*dk end; ].

3 ECE580 Fall 05 Solution to Problem Set 4 December 8, 05 3 we obtain the following results, which we have truncated at the point that the gradient becomes small. >> p0p0-3 - dk = 3 Index ak = 0.74 xk = bk = dk =

4 ECE580 Fall 05 Solution to Problem Set 4 December 8, Index ak = xk = e-5 * Note that the minimizer has been found on the second iteration. (c) Find the minimizer analytically. Solution: The minimizer is simply [ x = Q b = 5 ] [ 3 ] = [.5 Use the rank-one correction algorithm to minimize the function f(x) == [ ] [ ] [ ] 0 x x x [ ] [ ] x 0 x x + 7, starting with x (0) = 0. Solution The Matlab script x0 = [0 0] ; Q = [ 0;0 ]; b = [ -] ; xk = x0; ].

5 ECE580 Fall 05 Solution to Problem Set 4 December 8, 05 5 H0 = eye(); Hk = H0; Q*xk -b for index = :3, disp([ Index,numstr(index)]) dk = -Hk*gk ak = -(gk *dk)/(dk *Q*dk) xk = xk+ak*dk dxk = ak*dk d Q*xk - b - gk Q*xk - b Hk = Hk + (dxk - Hk*dgk)*(dxk - Hk*dgk) /(dgk *(dxk - Hk*dgk)) end; yields the result >> pp5 - Index dk = - ak = xk =

6 ECE580 Fall 05 Solution to Problem Set 4 December 8, 05 6 dxk = d Hk = Index dk = ak = xk = dxk =

7 ECE580 Fall 05 Solution to Problem Set 4 December 8, d Again, the minimizer was found on the second iteration..8 Consider the function f(x) = x 4 /4 + x / x x + x x. (a) Plot the level sets of f at the levels 0.7, 0.6, 0., 0.5, and locate the minimizers from the plot. Solution: Using the Matlab commands >> [X,Y] = meshgrid(-:0.:, -:0.:3); >> Z = X.^4/4+Y.^/ -X.*Y +X-Y; >> V = [-0.7, -0.6, -0., 0.5, ]; >> contour(x, Y, Z, V) >> text(-.5,, c = ) >> text(-.5,, c = 0.5 ) >> text(-0.5,.5, c = -0. ) >> text(-,.5, c = -0.6 ) >> text(-,0, c = -0.7 ) >> title( Problem -8 ECE580 Fall 05 ) >> ylabel( f(x) ) >> ylabel( x_ ) >> xlabel( x_ ) >> text(-.7,.7, Level Sets of f(x) ) >> print -depsc pp8a >>

8 ECE580 Fall 05 Solution to Problem Set 4 December 8, 05 8 generates the plot shown in Figure. From these, I surmise that the minimizers are near (, 0), and (, ). 3 Problem 8 ECE580 Fall 05.5 Level Sets of f(x) c =.5 c = 0. x c = c = c = x Figure : Level Sets for Problem.8 (b) Use the DFP algorithm to minimize the function. A bare bones script that implements the DFP algorithm to solve this problem is given below: %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%% %%%%% ECE569 Fall 05, IUPUI %%%%% Comparison of DFP algorithm performance for two initial %%%%% condiitons Problem.8 of Chong and Zak 4th ed. %%%%% c. 05 S. Koskie %%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% clear all x0 = [0;0]; % initial condition H0 = eye(); alpk = 0.5; xk = x0; Hk = H0; x = xk(); x = xk(); [4*x^3/4 - x + ;... x - x - ]

9 ECE580 Fall 05 Solution to Problem Set 4 December 8, 05 9 index = 0 while norm(gk) > 0^(-5), index = index + ; disp([ Index,numstr(index)]) dk = -Hk*gk alpk = 0.5; d = dk(); d = dk(); for iindex = :5, % Newton s method used to find optimizal alpha disp([ Iindex,numstr(iindex)]) f = d - d + d*(x + alpk*d)^3 - d*(x + alpk*d)... - d*(x + alpk*d) + d*(x + alpk*d) F = 3*d^*(x + alpk*d)^ - *d*d + d^ alpk = alpk - f/f end; if alpk < 0, alpk = 0; end; xk = xk+alpk*dk x = xk(); x = xk(); dxk = alpk*dk d [4*x^3/4 - x + ;... x - x - ]-gk [4*x^3/4 - x + ;... x - x - ] Hk = Hk + (dxk*dxk )/(dxk *dgk) - ((Hk*dgk)*(Hk*dgk) )/(dgk *Hk*dgk) end; f = d - d + d*(x + alpk*d)^3 - d*(x + alpk*d)... - d*(x + alpk*d) + d*(x + alpk*d) disp([ alpha(k) =,numstr(alpk)]) disp([ (x*,x*) = (,numstr(x),,,numstr(x), ) ]) disp([ f(x) =,numstr(f)]) (i) Case x 0 = 0. Solution: The matlab script finds the miminizer (, 0). f(, 0) = 6.934e 09. [ ].5 (ii) Case x 0 =. Solution: The matlab script finds the miminizer (, ). f(, ) = 5.659e 0. c 05 S. Koskie