ECE580 Exam 1 October 4, 2012 1 Name: Solution Score: /100 You must show ALL of your work for full credit. This exam is closed-book. Calculators may NOT be used. Please leave fractions as fractions, etc. I do not want the decimal equivalents. Cell phones and other electronic communication devices must be turned off and stowed under your desk. Please do not write on the back of the exam pages. Extra paper is available from the instructor. 1 2 3 4 5 6 7 8 9 10
ECE580 Exam 1 October 4, 2012 2 1. (8 points) Consider the following matrices. A 1 = 1 0 0 7 ], A 2 = 1 0 0 0 ], A 3 = 1 0 0 7 ], A 4 = 1 0 0 7 ]. (a) State the definition of positive definiteness of a quadratic form x T Ax. The quadratic form x T Ax is positive definite if for all x, x T Ax 0 and x T Ax = 0 iff x = 0. (b) Which (if any) of the matrices listed above are negative definite (and why)? A 4 is negative definite because it is symmetric and all of its eigenvalues are negative. (c) Which (if any) of the matrices are positive semidefinite (and why)? A 2 and A 3 are positive semidefinite because they are symmetric and all of its eigenvalues are nonnegative. (d) Which matrices (if any) are indefinite (and why)? A 1 is indefinite because it is symmetric and has both positive and negative eigenvalues.
ECE580 Exam 1 October 4, 2012 3 2. (12 points) Determine whether f(x) has one or more local minimizers x, and, if it does, determine the x. f(x 1, x 2 ) = x 3 1 + 2x 1 x 2 + 3x 2 2 + 3x 2 We are given no constraints on feasible values so Ω = R 2 and all points of Ω are interior points. Thus, in order to satisfy the first order necessary condition (FONC), we need f(x ) = 0. f(x ) = 12(x 1) 2 + 2x 2 + 1 2x 1 + 6x 2 requires (second row) x 2 = x 1/3, and (first row) Substituting for x 2, we have 12(x 1) 2 + 2x 2 + 1 = 0. 12(x 1) 2 2x 1/3 + 1 = 0. The roots of this quadratic equation are x 1 = 2/3 ± 4/9 48, 24 which are complex, so f(x) has no real minimizers.
ECE580 Exam 1 October 4, 2012 4 3. (8 points) Suppose we wish to use a fixed step gradient method to find the minimum of the function f(x) = x T Ax where (a) Give the formula for x (k+1). A = 2 1 0 0 2 0 0 0 1. x (k+1) = x (k) α f(x (k) ). (b) What is the upper bound on the step size α to guarantee convergence? The upper bound is given by α < 2 λ max (A) = 1.
ECE580 Exam 1 October 4, 2012 5 4. (12 points) Given the function f(x 1, x 2 ) = x 3 2 + x 2 1 x 2 + 1 (a) Find any points that satisfy the first order necessary condition. There are no restrictions on the set of feasible points Ω so Ω = R 2. Thus all points are interior points of Ω and the appropriate FONC requires that f(x ) = 0. The gradient, evaluated at x is f(x ) = 2x 1 3(x 2) 2 1 so we need x 1 = 0 and x 2 = ± 1/3., (b) Find any points that satisfy the second order sufficient condition. The second order necessary condition for interior points requires that in addition to satisfying the FONC, that the Hessian F (x ) be positive semidefinite. Here F (x ) = 2 0 0 6x 2 so we need x 2 0 which is satisfied only by the point x = (0, 1/3). (c) Identify any local minimizers. The second order sufficient condition requires the Hessian evaluated at x to be positive definite, which it is not, so the function has no minimizers.
ECE580 Exam 1 October 4, 2012 6 5. (12 points) If a constraint set Ω is bounded below by y = x 3 and above by y = x 2 + 2, determine whether each of the following points is an interior point, a boundary point, or is not in Ω. (Justify your answers.) (a) (1, 3) 1 3 = 2 < 3 and 1 2 + 2 = 1 < 3. This means that the point (1, 3) is above both curves, thus not in Ω. (b) (0, 3) 0 3 = 3 = 3 and 0 2 + 2 = 2 > 3. This means that the point (0, 2) is on the line y = x 3 and below y = x 2 +2, so the point is a boundary point of Ω. (c) ( 1, 3) 1 3 = 4 < 3 and 1 2 + 2 = 1 > 3. This means that the point ( 1, 3) is above the line and below y = x 2 + 2, so the point is an interior point of Ω.
ECE580 Exam 1 October 4, 2012 7 6. (12 points) Given a point x = 1 0 ] T and the function f(x 1, x 2 ) = x 3 2 + x 2 1 x 2 + 1. (a) Find the gradient at the given point. f 1 0 = 2(1) 3(0) 2 1 = 2 1 (b) Find the rate of increase of the function at the given point. The rate of increase of the function at the given point is f(x) T d d = 2d 1 d 2 d 2 1 + d 2 2 (c) Find the equation of the plane tangent to the surface at the given point. In the case that f(x) 0, equation of the tangent plane at a point x 0 = 1 0 ] T is f(x 0 ) T (x x 0 ) = 0. Thus the equation for the tangent plane at x 0 2 1 ] x 1 1 x 2 = 2(x 1 1) x 2 = 0. is
ECE580 Exam 1 October 4, 2012 8 7. (8 points) Given initial point x (0) = 2 and the function f(x) = x 3 x 2 x + 1, find x (1) using Newton s method for finding a minimizer of f. In order to apply Newton s method, we will need the first and second derivatives of f evaluated at x 0 : f (x 0 ) = 3x 2 0 2x 0 1 = 12 4 1 = 7, f (x 0 ) = 6x 0 2 = 12 2 = 10. Now applying Newton s method we obtain x (1) = x (0) f (x (0) ) f (x (0) ) = 2 7/10 = 13/10.
ECE580 Exam 1 October 4, 2012 9 8. (8 points) Given initial point x (0) = 2 and the equation f(x) = x 3 x 2 x + 1 = 0, find x (1) using Newton s method of tangents for finding a root of f. The formula for the update in the method of tangents is x (k+1) = x (k) f(x(k) ) f (x (k) ). The derivative is f (x) = 3x 2 2x 1. Thus x (1) = x (0) 23 2 2 2 + 1 3(2) 2 2(2) 1 = 2 3/7 = 11/7.
ECE580 Exam 1 October 4, 2012 10 9. (8 points) Given initial points x (0) = 2 and x ( 1) = 1 and the equation f(x) = x 3 x 2 x + 1, find x (1) using the secant method for finding a minimizer of f. The secant method update algorithm is The derivative is Thus x (1) x (k+1) = x (k) x (k) x (k 1) f (x (k) ) f (x (k 1) ) f (x (k) ). f (x) = 3x 2 2x 1. 2 1 = 2 (3(2) 2 2(2) 1) (3(1) 2 2(1) 1) (3(2)2 2(2) 1) = 2 1 7 0 7 = 2 1 = 1.
ECE580 Exam 1 October 4, 2012 11 10. (12 points) Given an initial point x (0) = 1 1 f(x 1, x 2 ) = x 3 2 + x 2 1 x 2 + 1, ] T and the equation find x (1) using the steepest descent method for finding a minimizer of f. Gradient descent methods have the form x (k+1) = x (k) α k f(x (k) ). The method of steepest descent uses α k = arg min α 0 f ( x (k) α f(x (k) ) ). The gradient evaluated at x (0) f x=x (0) = is 2x 1 3x 2 2 1 x=x (0) = 2 2. Evaluating the right-hand side of the expression for α k yields α k = arg min α 0 8α 2 8α 2 8α + 2. Taking the derivative and equating it to zero we find that the critical points occur at 1± 2 so the expression has no minimum. Thus we need to try a different x (0) or a different method.