too, of course, but is perhaps overkill here.

Transcription

1 LUNDS TEKNISKA HÖGSKOLA MATEMATIK LÖSNINGAR OPTIMERING kl a) CQ points are shown as A and B below. Graphically: they are the only points that share the same tangent line for both active constraints. Furthermore, the gradients of two active constraints are directed opposite to each other. Hence, they are CQ points. It is possible to solve the problem analytically as well, however, the calculation would take more time. b) The set of feasible directions are the cone between the tangent lines to two active constraints at ( 3, 0) (including the one line and ecluding the other one). The tangent lines go through the point ( 3, 0) and the point B : (0, 3) and the point (0, 3) respectively. The cone of all possible gradients f is between the outer normals (gradients to the active constraint functions at the point). Those gradients are parallel to the lines through the point and the point A : (0, 1) and (0, 1) respectively. Figure 1: Drawing in 1a) to the left and in 1b) to the right. Consider the optimization problem min + 4y subject to y 1. a) (1, 1) is feasible, add + 4y 5 to get a compact set and use the Weierstrass theorem. The problem is not conve (due to g(, y) 1 y). Simplied argument: the minimum is where the gradients to f and g are parallel (since g must be active at the minimum, otherwise we can go a bit towards the unconstrained min at (0, 0) and make the functional value smaller): y det 4y 0 ±y. 4y Together with y 1 we have two solutions ±(, 1 ). KKT method works too, of course, but is perhaps overkill here.

2 b) Add penalty e.g. q µ (, y) + 4y + µ ma0, 1 y}. The penalty is zero at (1, ). The function is quadratic, hence, the rst Newton step gives the unconstraint minimum (0, 0) for any µ (no need to actually do any calculations here). The origin is a stationary point for q µ for any µ q µ (, y) + 4y + µ(y 1) q µ + µ(y 1)y 8y + µ(y 1) hence, we do not move anywhere on the second step for any µ. c) Add barrier with ɛ 1, e.g. q(, y) + 4y ln(y 1) [ q µ (0, 0) or qq(, y) + 4y + 1 y 1 ]. 0, 0 Then q(1, ) (0, 15) T and d H 1 q(1, ) (0.3, 1.8) T. It points much better to the minimum. 3. a) Check that the point is feasible for the primal problem. Construct the dual problem e.g. min (y 1 + 6y + 4y 3 ) subject to 5y 1 + 9y + 3y 3 1, 6y 1 + 3y y 3, y 1 + y + y 3 4, y 1 0, y 3 0. CSP gives that y 1 y 3 0, hence y. Check that y (0,, 0) is dual feasible and conclude that both are optimal by CSP. b) The second system can be written as the rst Farkas alternative A T A T I y 0, b T y > 0. Then by Farkas theorem we have the second alternative as u A A I v b, w (u, v, w) 0. Setting u v we get A w b A b since w a) See the book, Lemma, p. 11, and Eercise 4.8, p. 18.

3 b) The rst function can be split as g(h()) where h() 1 + ( + y + z) conve ( + y + z is ane and the squaring is conve) and h() 1, and g(t) t ln t is conve and increasing for t 1 (as g (t) ln t and g (t) 1 t 0). The second function is not conve: take the line y z, then the restriction to the line is 8 3, which is not conve on R ((8 3 ) 48 < 0 for < 0). c) Divide y + y by y to get (as > 0, y > 0) the condition It is y the sublevel set of the conve function 1 + 1, thus, conve. y 5. Using conveity (the shortest solution): the objective function is conve (e.g. check the Hessian by Sylvester criterion) ( ) ] ( ) + y, + y 4 y y + y + y ( + y) 3 y, [ y +y (+y) +y +y (+y) the constraint y 1 is not conve, but if we divide it by > 0, it becomes 1 y 0, which is conve. Hence, with X > 0, y > 0}, the problem becomes conve. By sucient condition, any KKT point is the global minimum. From the KKT conditions y +y u 1 0, (+y) +y +y u 0, (+y) u ( 1 y) 0, u 0, > 0, y > 0, 1 y 0, y + y u (+y) 0, + y + y u( + y) 0, u(1 y) 0, u 0, > 0, y > 0, 1 y 0, we see that u 0 (otherwise 4y 0: not possible). Then y 1. It is quite easy to notice that y 1 is a KKT point. If you do not think so, take the rst equation minus the second one to get y + u( + y) (1 1 ) 0 4 (y) +u( + y) ( 1) 0. }} 1 Since 4 1 ( 1)( + 1) we can factor out the common factor 1 and obtain ) ( 1) (( + 1) + u( + y) 0. Since the second factor is stricktly positive, it makes 1, thus, 1 (as > 0) and y 1. It is a KKT point, therefore, the global minimum by the sucient KKT condition. Alternative solution (without conveity): prove that min eists. The strict constraints are never active, that is, the set is closed (you may draw the set to see that the boundary is y 1, and it is, indeed, in the set.) Then we need boundedness only. Take e.g. y and add + y ( + y) to the constraints. Complete the squares to see that the new set is bounded (the added constraint is a disc ( 1) + (y 1), i.e. bounded). Weierstrass gives the eistence. With X > 0, y > 0} we have only one eplicit constraint. No CQ points (verify!).

4 The single KKT point can be found similarly to the other solution: take the rst KKT equation minus the second one to get y +y uy (+y) 0, +y +y u (+y) 0, y + y u( + y) y 0, + y + y u( + y) 0, y + u( y)( + y) 0. Use y ( y)( + y) to factor out the common y ( y)(( + y) + u( + y) ) 0. The second factor has no zeros since u 0 and, y > 0, thus, no more KKT points ecept for ( y 0, y 1 ) y a) The problem is not the easiest one. Set up the Lagrange function L(, y, z, u 1, u ) z zu 1 ( + y) + u 1 ( + y ) + u (1 y) and minimise on z to get z u 1 ( + y). We are left with a quadratic function to minimize inf z L ( u u 1 ) + T Q + u y y where Q is the matri of the quadratic form a b Q 1 b a 4 ( ) ( u u 1 y + u 1 1 ) 4 u 1 y + u u1 (4 u 1 ) u u 1 u u. 1 u 1 (4 u 1 ) Note that if a < 0 (i.e. u 1 > 4) then inf L (take y 0, + ). Let us study the case a 0. We are to minimize q(, y) over > 0, y > 0 given a 0, b 0. T Q a by + ay y y Proposition 1 The following statements are equivalent: 1. Q is positive semidenite,. q(1, 1) 0, 3. b a.

5 Proof: 1 : trivially by denition of positive deniteness. 3: trivially by q(1, 1) a 1 b 1 + a 1 (a b) : if a 0 then b 0, hence, Q is the zero matri, and it is positive semidenite. If a > 0 then a + b > 0 and det Q a b (a b)(a + b) 0. Therefore, by Sylvester criterion Q is positive semidenite. The Proposition gives us directly the following result 0 if Q is pos.semidef., inf q(, y) >0, y>0 otherwise. Q.E.D. The otherwise-part: if Q is not positive semidenite then q(1, 1) < 0, and taking y + makes q. Moreover, Q is positive semidenite if and only if a b 0 u 1 (4 u 1 ) u u 1 0 u u 1 ( u 1 ). Back to the Lagrange function minimization. We have u if 0 u u 1 ( u 1 ), Θ(u 1, u ) inf L otherwise. Maimum is when (clearly) u u 1 ( u 1 ). Maimizing u 1 ( u 1 ) subject to 0 u 1 gives the optimal u 1 u 1. Unfortunately it does not give us unique, y values for inf L, so we have to plug the optimal u 1 u 1 and z +y into the Lagrange function to get all the candidates L 3 4 ( y) + 1. Then the minimum is when y. Together with the Complementary Slackness u 1 (y 1) 0, i.e. y 1, we nally obtain the candidate for the saddle point y 1. Testing shows no duality gap. A pour man's solution would be to take the candidates y 1 from the solution to Problem 5 and check the duality gap (as long as we have no gap, it is the optimal solution, and nobody cares how you manage to "guess" those values). b) See the book.