ECE580 Fall 2015 Solution to Problem Set 6 December 23 2015 1 ECE580 Solution to Problem Set 6 These problems are from the textbook by Chong and Zak 4th edition which is the textbook for the ECE580 Fall 2015 semester. As such many of the problem statements are taken verbatim from the text; however others have been reworded for reasons of efficiency or instruction. Solutions are mine. Any errors are mine and should be reported to me skoskie@iupui.edu rather than to the textbook authors. 12.7 Given two mixtures of metal: A contains 30% Au 40% Ag and 30% Pt; B contains 10best amounts to achieve as close as possible to 5 oz. Au 3 oz. Ag and 4 oz. Pt. Solution: We formulate this as a least squares problem as follows. Let.3.1 5 [ A =.4.2 b = 3 xa x =. x.3.7 4 B Because A is m n with m n and ρ(a) = n the optimal choice for x will be (per Matlab) [ 1.5665 x = (A T A) 1 A T b =.9606 The ratio is thus 11 A to 1 B. Notice that this best choice is far from achieving the desired mixture. Ax = 3.2660 4.4187 3.8424 meaning that we have 3.266 oz. of Au 4.4187 oz. of Ag and 3.8424 oz. of Pt as compared to the 5 of Au 3 of Ac and 4 of Pt that we wanted. 12.14 Consider the discrete-time linear system x k+1 = ax k + bu k. Suppose we apply u k = 1 for all k 0 and observe x 0 = 0 x 1 = 1 x 2 = 2 and x 3 = 8. Find the least squares approximation of a and b. Solution: We have the following matrices: 0 1 1 A = 1 1 b = 2 x = 2 1 8 [ a b.
ECE580 Fall 2015 Solution to Problem Set 6 December 23 2015 2 Again A is 3 2 with ρ(a) = 2 so x = (A T A) 1 A T b = [ 1 [ 5 3 0 1 2 3 3 1 1 1 = = [ 1/2 1/2 1/2 5/6 [ 7/2 1/6 1 [ 18 11 i.e. the best approximation is that a = 7/2 and b = 1/6. 21.3 Find the local minimizer for 1 2 8 subject to x 2 1 + x 2 2 x 2 1 + 2x 1 x 2 + x 2 2 = 1 x 2 1 x 2 0 Solution: First we put the system in standard form f(x) = x 2 1 + x 2 2 h(x) = x 2 1 + 2x 1 x 2 + x 2 2 1 = 0 f(x) = x 2 1 x 2 0. The gradients and Hessians are [ 2x1 f(x) = 2x 2 [ 2x1 + 2x h(x) = 2 2x 1 + 2x 2 [ 2x1 g(x) = 1 F (x) = [ 2 0 0 2 [ 2 2 H(x) = 2 2. [ 2 0 0 0 The SOSC requires the following conditions for a minimizer. 1. µ 0 2. Df(x ) + λ Dh(x ) + µ Dg(x ) = 0 3. µ g(x ) = 0 4. For all y T (x µ ) y 0 y T L(x λ µ )y > 0.
ECE580 Fall 2015 Solution to Problem Set 6 December 23 2015 3 We start with the constraint µ g(x ) = 0. We have which requires either µ ( x 2 1 x 2 ) 0 µ = 0 or x 12 = x 2. If µ = 0 then the gradient constraint becomes [ 2x 1 2x [ 2 + λ 2x 1 + 2x 2 2x 1 + 2x 2 = 0 i.e. x 1 + λ x 1 + λ x 2 = 0 x 2 + λ x 1 + λ x 2 = 0 which requires x 1 = x 2. Next we check to see whether h(x) = 0 is satisfied. We have h(x ) = x 12 + 2x 1x 2 + x 22 1 = 0 which after substituting x 1 for x 2 is 4x 1 2 = 1 so x 1 = ± 1 2 = x 2. We must have g(x ) = 0 so we see that only one of these two options work and ( 1 (x 1 x 2) = 2 1 ). 2 The corresponding value of λ can be found from (1) to be 1/2. To check the remaining condition we must find (a) the Hessian of the Lagrangian which is [ 2(1 + λ + µ) 2λ L(x λ µ) = F (x) + λh(x) + µg(x) = 2λ 2(1 + λ) and (b) a subspace T (x ) to be defined below. This subspace is the tangent space to the surface defined by the active constraints having positive µ i. To express this conveniently we need an index set J(x µ ) := {i : g i (x ) = 0 µ i > 0}. Then the subspace can be expressed as the intersection of several sets: T (x ) = {y : Dh(x )y = 0} {y : Dg i (x ) = 0} i J(x µ )
ECE580 Fall 2015 Solution to Problem Set 6 December 23 2015 4 In this case J(x µ ) is empty so T (x µ ) is just the ordinary tangent space T (x ). Thus T (x ) = {y : [ 2x 1 + 2x 2 2x 1 + 2x 2 = {y : y 2 = y 1 } y = 0} and we find k 2 [ 1 1 [ 2(1 + λ + µ) 2λ 2λ 2(1 + λ) [ 1 1 = 2(1 + µ) + 2 = 4 + 2µ = 4 > 0. so L(x λ µ) is positive definite on the tangent space at x. Thus x is a minimizer. A positive definite (F (x) > 0) quadratic function has a unique minimizer so we need not consider the other alternative. 21.7 Consider the problem of optimizing subject to f(x) = (x 1 2) 2 + (x 2 1) 2 g 1 (x) = x 2 x 2 1 0 g 2 (x) = 2 x 1 x 2 0 g 3 (x) = x 1 0. The point x = 0 satisfies the KKT conditions. (a) Does x satisfy the FONC for minimization or maximization? Solution: yes. The KKT conditions are the FONC. What are the KKT multipliers? Solution: With x = 0 the condition that requires that µ g(x ) = 0 µ 1(x 2 x 12 ) = 0 µ 2(2 x 1 x 2) = 0 µ 3x 1 = 0. The second of these requires that µ 2 = 0. The condition Df(x ) + λ Dh(x ) + µ T Dg(x ) = 0
ECE580 Fall 2015 Solution to Problem Set 6 December 23 2015 5 becomes since there are no equality constraints At x = 0 they are 2(x 1 2) 2x 1µ 1 µ 2 + µ 3 = 0 2(x 2 1) + µ 1 µ 2 = 0. 4 µ 2 + µ 3 = 0 2 + µ 1 µ 2 = 0. Thus with µ 2 = 0 we have µ 1 = 2 and µ 3 = 4. (b) Does x = 0 satisfy the SOSC? Solution: We need f g h C 2 (true of f and g and there is no h) µ 0 (true) µ g(x ) = 0 (true because we chose the µ to satisfy it) Df(x )+λ Dh(x )+µ Dg(x ) = 0 (true because we chose the µ ) to satisfy it) and positive definiteness on the tangent space to the surface defined by the active constraints having positive µ i. The index set J(x µ ) is given by Then subspace T (x µ ) is T (x µ ) = {y : Dh(x )y = 0} J(x µ ) := {i : g i (x ) = 0 µ i > 0} = {1 3}. i:g i (x )=0µ i >0 {y : Dg i (x )y = 0} = {y : g 1 (x ) = g 3 (x ) = 0 µ 1 µ 3 > 0 Dg 1 (x )y = Dg 3 (x )y = 0} = {y : y 1 = y 2 2} {y : y 1 = 0} = 0 Thus for all non-zero y in T (x µ ) y T L(x λ µ )y > 0. So the origin is a minimizer of the cost function f(x) subject to the given constraints. 21.19 Consider the problem min x 2 1 + x 2 2 subject to 4 x 1 x 2 2 0 3x 2 x 1 0 3x 2 x 1 0 Deduce from [Figure 22.3 of the text that the problem has two strict local maximizers and use the [SOSCs to verify the graphical solution. Solution: I will assume from the problem statement that we should use the information we obtain from the figure to reduce the amount of calculation
ECE580 Fall 2015 Solution to Problem Set 6 December 23 2015 6 required to solve the problem. I will also assume that maximizers should be minimizers since that is what they are. As usual I will start with the requirement µ T g(x ) = 0 to identify candidate cases then apply the conditions derived from the derivative constraint to identify the candidate points for each case. Considering that each of the three constraints could be active or not would result in a set of 8 possible cases to consider. However the only constraint that could lead to a symmetric pair of solutions is the first so let s assume that In standard form we have µ 1 0. f(x) = x 2 1 + x 2 2 g 1 (x) = 4 x 1 x 2 2 0 g 2 (x) = 3x 2 x 1 0 g 3 (x) = 3x 2 x 1 0 The gradients and Hessians are [ 2x1 f(x) = 2x 2 [ 1 g 1 (x) = 2x 2 [ 1 g 2 (x) = 3 [ 1 g 3 (x) = 3 [ 2 0 F (x) = 0 2 G 1 (x) = [ 0 0 0 2 G 2 (x) = 0 G 3 (x) = 0. The derivative constraint is [ 2x 1 2x 2 Df(x ) + λ Dh(x ) + µ Dg(x ) = 0 [ [ [ + µ 1 1 2x + µ 2 1 3 + µ 3 1 3 = 0 2x 1 µ 1 µ 2 µ 3 = 0 (1) 2x 2 2x 2µ 1 + 3µ 2 3µ 3 = 0. (2) Case: µ = (µ 1 0 0) If we assume that only µ 1 is nonzero we solve the simultaneous equations 2x 1 = µ 1 2x 2 = 2x 2µ 1
ECE580 Fall 2015 Solution to Problem Set 6 December 23 2015 7 to obtain µ 1 = 1 x 1 = 1/2 and x 2 = 7/2. This does not satisfy the constraints ±3x 2 x 1 0. Thus this is not the case does not provide a minimizer. Case: µ = (µ 1 0 µ 3) The conditions x 2 2 = 4 x 1 x 2 = x 1 /3 combine to yield x 1 {3 12} and x 2 { 1 4}. x = ( 12 4) does not satisfy constraint g 3 (x ) 0. We solve the simultaneous equations 2x 1 µ 1 µ 3 = 0 2x 2 2x 2µ 1 3µ 3 = 0. to obtain µ 1 = 4 and µ 3 = 2 when x = (3 1). Case: µ = (µ 1 µ 2 µ 3) The two requirements x 2 = x 1 /3 = x 1 /3 are satisfied only by the origin. Thus this is not the case does not provide a minimizer. Case: µ = (µ 1 µ 2 0) In this case we find that x 1 {3 12} so x 2 {1 4}. x = ( 12 4) does not satisfy constraint g 3 (x ) 0. For x = (3 1) we solve the simultaneous equations to obtain µ 1 = 8 and µ 2 = 2. 2x 1 µ 1 µ 2 = 0 2x 2 2x 2µ 1 + 3µ 2 = 0. In both cases that produced minimizer candidates x = (3 1) and x = ( 3 1) the requirement µ 0 is satisfied. f g C 2. All that remains is to show positive definiteness on the tangent space to the surface defined by the active constraints having positive µ i. The index sets are J 2 (x µ ) := {i : g i (x ) = 0 µ i > 0} = {1 2} J 3 (x µ ) := {i : g i (x ) = 0 µ i > 0} = {1 3} Then subspaces are T 2 (x µ ) = {y : Dh(x )y = 0} i {12} {y : Dg i (x )y = 0} = {y : g 1 (x ) = g 2 (x ) = 0 µ 1 µ 2 > 0 Dg 1 (x )y = Dg 2 (x )y = 0} = {y : y 1 = 2y 2 } {y : y 1 = 3y 2 } = 0 T 3 (x µ ) = {y : Dh(x )y = 0} {y : Dg i (x )y = 0} i {13} = {y : g 1 (x ) = g 3 (x ) = 0 µ 1 µ 3 > 0 Dg 1 (x )y = Dg 3 (x )y = 0} = {y : y 1 = 2y 2 } {y : y 1 = 3y 2 } = 0
ECE580 Fall 2015 Solution to Problem Set 6 December 23 2015 8 Thus for all non-zero y in T (x µ ) y T L(x λ µ )y > 0. So the the two minimizer candidates are both minimizers of the cost function f(x) subject to the given constraints. c 2015 S. Koskie