ECE580 Spring 2016 Solution to Problem Set 3 February 8, 2016 1 ECE580 Solution to Problem Set 3: Applications of the FONC, SONC, and SOSC These problems are from the textbook by Chong and Zak, 4th edition, which is the textbook for the ECE580 Spring 2016 semester As such, many of the problem statements are taken verbatim from the text; however, others have been reworded for reasons of efficiency or instruction Solutions are mine Any errors are mine and should be reported to me, skoskie@iupuiedu, rather than to the textbook authors 61 Consider the problem minimize f(x) subject to x Ω, where f C 2 For each of the following definitions of Ω, x, and f, determine whether x is, is not, or might be a local minimizer (a) f : R 2 R, Ω = {x : x 1 1}, x = [ 1 2 T, f(x ) = [ 1 1 T Solution: x is on the boundary so we must verify that d T f(x ) 0 for all feasible directions d To be feasible, d 1 must be nonnegative, and d 2 is arbitrary, so long as d 0 d T f(x ) = d 1 + d 2 Since d 2 is arbitrary, this quantity may be negative, thus x is not a local minimizer (b) f : R 2 R, Ω = {x : x 1 1, x 2 2}, x = [ 1 2 T, f(x ) = [ 1 0 T Solution: In this case, x is on the boundary at a corner Thus, to be feasible, d must have both d 1 and d 2 nonnegative d T f(x ) = d 1 0, so the FONC is satisfied and x could be a local minimizer We are not given the Hessian, so we cannot check the SONC nor the SOSC (c) f : R 2 R, Ω = {x : x 1 0, x 2 0}, x = [ 1 2 T, f(x ) = [ 0 0 T, F (x ) = I Solution: x is an interior point Since the gradient is zero at x, the FONC is satisfied Since the Hessian is positive definite, the SOSC is satisfied so x is a local minimizer (d) f : R 2 R, Ω = {x : x 1 1, x 2 2}, x = [ 1 2 T, f(x ) = [ 1 0 T [, 1 0 F (x ) = 0 1
ECE580 Spring 2016 Solution to Problem Set 3 February 8, 2016 2 Solution: x is on the boundary at a corner Both d 1 and d 2 must be nonnegative as before, and not both zero To satisfy the FONC, we need d T f(x ) = d 1 0, for all feasible d at x, which is true However, Applying the SONC with d = (0, 1), which is a feasible direction at x, we find that d T F (x )d = [ 0 1 [ [ 1 0 0 = 1 < 0 0 1 1 so the SONC is not satisfied and x is not a local minimizer 62 Find the minimizers and maximizers of f(x 1, x 2 ) = 1 3 x3 1 4x 1 + 1 3 x3 2 16x 2 Solution: Note that there is no restriction on the set of feasible points so Ω = R 2 First, we identify possible minimizers and maximizers by applying the first order necessary condition (FONC) Solving [ (x f(x 1, x 2) = 1 ) 2 4 (x 2) 2 = 0 16 leads to two possible values for x 1 and two for x 2, namely x 1 { 2, 2} and x 2 { 4, 4} Thus we have four points to be checked: ( 2, 4), (2, 4), ( 2, 4), (2, 4) Next we apply the second order necessary condition (SONC) for interior points (all points of R 2 are interior points) At each of the four points, we examine the Hessian, [ 2x1 0 F (x 1, x 2 ) = 0 2x 2 If both x 1 and x 2 are positive, the Hessian is positive definite, so (2, 4) is a strict local minimizer If both x 1 and x 2 are negative, the Hessian is negative definite, so ( 2, 4) is a strict local maximizer In the other two cases, the Hessian is indefinite so those points are neither minimizers nor maximizers 69 Consider the function f(x 1, x 2 ) = x 2 1x 2 + x 3 2x 1 (a) In what direction does f decrease most rapidly at the point x (0) = [2, 1 T? Solution: The rate of increase of f at a point x in the unit direction d is < f(x), d > (see p 84) The direction of maximum rate of increase of f at x is f(x) (see pp 71, 132)
ECE580 Spring 2016 Solution to Problem Set 3 February 8, 2016 3 The gradient is Evaluated at x (0), this becomes [ 2x1 x f(x 1, x 2 ) = 2 + x 3 2 x 2 1 + 3x 1 x 2 2 f(2, 1) = [ 5 10 Thus the unit vector in the direction of maximum rate of decrease of f at x (0) is [ f(2, 1) 1/ 5 ˆd = f(2, 1) = 2/ 5 (b) What is the rate of increase of f at the point x (0) in the direction of maximum decrease of f? Solution: the rate of increase of f at the point x (0) in the direction of maximum decrease of f is < f(x (0) ), ˆd > = [ 5 10 [ 1/ 5 2/ 5 = 5 + 4 5 = 5 5 (c) Find the rate of increase of f at the point x (0) in the direction d = [ 3 4 T Solution: First, we must obtain the unit vector [ 3/5 ˆd = d/ d = 4/5 Then we take the scalar product 613 Consider the problem < f(x (0) ), ˆd > = [ 5 10 [ 3/5 4/5 = 3 + 8 = 11 minimize f(x) (1) subject to x Ω (2) where f : R 2 R is given by f(x) = 3x 1, where x = [ x 1 Ω = {x : x 1 + x 2 2 2} x 2 T, and (a) Does the point x = [ 2 0 T satisfy the FONC? Solution: The gradient evaluated at x is [ T 3 f(2, 0) = 0
ECE580 Spring 2016 Solution to Problem Set 3 February 8, 2016 4 which is never zero, so no point in the interior of Ω satisfies the FONC OTOH, The point x = [ 2 0 T is on the boundary, so we must examine the feasible directions on the boundary (or more precisely at x ) The boundary is shown in Figure?? The Matlab code used to generate the figure is as follows: >> x2 = [0:01:4; >> x1 = 2 - x2^2; >> plot(x1,x2) >> hold Current plot held >> plot(x1,-x2) >> title( Feasible set \Omega for Problem 613 ) >> text(-11,15, \Omega = \{x: x_1 + x_2^2 \leq 2\} ) >> xlabel( x_1 ) >> ylabel( x_2 ) (b) Does the point x = [ 2 0 T satisfy the SONC? Solution: The condition d T f(x ) = 0 requires that d 1 = 0 However, the only feasible directions at x have d 1 < 0 so the SONC is trivially satisfied (c) Is the point x = [ 2 0 T a local minimizer? Solution: To determine this, we might imagine that we should apply the SOSC However, the SOSC that we have covered applies only in the interior case and x is on the boundary, so the SOSC is of no help Instead, let s look at the gradient [ T 3 f(2, 0) = 0 means that the function f is strictly increasing with decreasing x 1 The point x is the unique point for which x 1 = 2 (because if x 2 0, then x 1 < 2 x 2 < 2, so we have shown that x is the unique minimizer of f on Ω 614 Consider the problem minimize f(x) subject to x Ω where Ω = {x R 2 : x 2 1 + x 2 2 1} and f(x) = x 2 (a) Find all point(s) satisfying the FONC Solution The gradient of f is f(x) = [ 0 1
ECE580 Spring 2016 Solution to Problem Set 3 February 8, 2016 5 The set Ω consists of all points on and outside the unit circle We can reject all points outside the unit circle because these are interior points of Ω and the gradient is nonzero at all of these points For the points on the boundary, we must consider feasible directions The FONC that applies at boundary points requires that for any feasible direction Here we find that d T f(x ) 0 d T f(x ) = d 2, so the only points that can satisfy the condition are those for which there is a feasible direction with nonnegative d 2 This immediately eliminates all points on the for which x 2 < 0 However, it eliminates more than that For any point on the upper portion of the unit circle, with one exception, there are feasible directions for which d 2 < 0 The one exception is the point (0, 1) For this point, d 2 0, so the unique point that satisfies the FONC is x = (0, 1) For this x, the set of feasible directions is {d : d 2 0} (b) Which of these point(s) satisfying the SONC? Solution The SONC requires that the Hessian be positive semidefinite on the set of points x with feasible directions d at x such that d T f(x ) = 0 The point x has two feasible directions with d 2 = 0, namely d = (±1, 0) To check the SONC for these directions, we need the Hessian, which in this case is F (x) = 0 So the SONC is satisfied for any d at all, feasible or not (c) Which of these point(s) are local minimizers? Solution We cannot use the SOSC because the point (0, 1) is not an interior point of Ω Reasoning towards the answer, we note that f(0, 1) = 1 If we travel in a direction with increasing d 2, f increases; if we travel in a direction with constant d 2, f remains constant; if we travel in a direction with decreasing d 2, however, we obtain a smaller value of f Of course we must show that such a value exists in Ω for this argument to be valid Let ɛ be small and positive and x 2 = 1 ɛ Then let x 1 = 1 (1 ɛ) 2 = 1 (1 2ɛ + ɛ 2 = 2ɛ ɛ 2 We see that for arbitrarily small ɛ, the point ( ɛ(2 ɛ), 1 ɛ) is on the unit circle and the value of f is smaller than 1 so the point (0, 1) is not a local minimizer