Exam: Continuous Optimisation 215 1. Let f : C R, C R n convex, be a convex function. Show that then the following holds: A local imizer of f on C is a global imizer on C. And a strict local imizer of f on C is a strict global imizer on C. Solution: for a local imizer x: Suppose x is not a global imiser. Then with some C we have f(x) > f(). Thus for < λ 1 we find with x λ := x + λ( x) using convexit of f: f(x λ ) f(x) + λ[f() f(x)] < f(x) So letting λ +, x cannot be a local imizer. for a strict local imizer x: Suppose it is not a strict global imiser. Then with some C, x we have f(x) f(). Thus for < λ 1 we find with x λ := x + λ( x) using convexit of f: f(x λ ) f(x) + λ[f() f(x)] f(x) So letting λ +, x cannot be a strict local imizer. 2. (a) Show that for d R n it holds: d T x x R n d =. (b) Let c, a i R n, i = 1,..., m (m 1). Show using the Farkas Lemma (lecture sheets, Th. 5.1) that precisel one of the following alternatives (I) or (II) is true: (I): c T x <, a T i x, i = 1,..., m has a solution x R n. (II): there exist µ 1,..., µ m such that: c + m µ ia i = Solution: (a) : d T x x R n ±d T e j j d T e j = j d = : d = d T x = x R n d T x x R n (b) Considering a m+1 = c and b = e m+1 R m+1 we have that (I) is equivalent to: (i): a T i x b i, i = 1,..., (m + 1) has a solution x. B Farkas Lemma, precisel one of either (i) or the following statement, 1
(ii), is true: (ii): R m+1 + such that = m+1 ia i, > b T. This is equivalent to: R m+1 + such that = m+1 c + m ia i, > m+1, which in turn is equivalent to (II). 3. Given is the problem (P ) x R 2 ( 2x 1 x 2 ) s.t. x 1, and (x 1 1) 2 (x 2 1) 2 + 2. (a) (b) (c) (d) Is (P ) a convex problem? Sketch the feasible set and the level set of f given b f(x) = f(x) with x =. Is LICQ (constraint qualification) satisfied at x? Show that the point x = is a KKT-point of (P ). Detere the corresponding Lagrangean multipliers. Show that x is a local imizer. What is the order of this imizer? Is it a global imizer? Consider now the program (objective f and constraint function g 2 interchanged): ( P ) x R 2 (x 1 1) 2 (x 2 1) 2 + 2 s.t. x 1, and 2x 1 x 2. Explain (without an further calculations) wh x = is also a local imizer of ( P ). Solution: (a) (P) is not a convex program since g 2 is not convex: 2 g 2 (x) = ( 2 2) is negative definite. Above is a sketch of the problem. The feasible set is coloured blue and the level curve is coloured red. 2
LICQ holds at x = : ( ) 1 g 1 (x) =, g 2 (x) = ( ) 2 2 are linearl independent Give a complete sketch. (b) The KKT condition for x = (g 1 and g 2 active) read: ( ) ( ) ( ) 2 1 2 + µ 1 + µ 2 = 1 2 With (unique) solution µ 1 = 1, µ 2 = 1/2. (c) Since the assumptions of Th 5.13 are satisfied, x = is a local imizer of order p = 1. It is not a global imizer since f(x) = and e.g. for feasible x = (, x 2 ), x 2 2 we have f(, x 2 ) for x 2. (d) The KKT condition at x = for (P) directl ields a corresponding KKT condition for ( P ) at x (feasible for ( P )!!) which again satisfies the assumption of Theorem 5.13 for ( P ). 4. Consider the (nonlinear) program: (P ) x f(x) s.t. x F := {x R n g j (x), j J with f, g j C 1, f, g : R n R, J = {1,..., m. Let d k be a strictl feasible descent direction for x k F. Show that for t >, small enough, it holds: f(x k + td k ) < f(x k ) and x k + td k F Solution: B using Talor around x k we find for j J xk (use g j (x k ) T d k < ; g j (x k ) = ): g j (x k +td k ) = g j (x k )+t g j (x k ) T d k +o(t) = t g j (x k ) T d k +o(t) < for t > small enough. B continuit also for j / J xk we have g j (x k + td k ) < for t > small enough. So x k + td k F. In view of f(x k ) T d k < we also find f(x k + td k ) = f(x k ) + t f(x k ) T d k + o(t) < f(x k ) for t > small enough. 3
5. For a given nonempt set A R n we define its conic hull, conic(a) b { m conic(a) := µ i x i : x i A, µ i for all i, m N. (a) (b) (c) Show that conic(a) is a convex cone. Show that if A B R n, with B being a convex cone, then conic(a) B. Show that conic(a) is full dimensional if and onl if there does not exist R n \ { such that, x = for all x A. [1 point] Solution: (a) B Theorem 7.2, equivalentl we want to show that for all u, v conic(a) and λ 1, λ 2 > we have λ 1 u + λ 2 v conic(a). Considering an arbitrar u, v conic(a) and λ 1, λ 2 > we have Therefore u = µ i x i, v = for some x 1,..., x m, 1,..., p A, λ 1 u + λ 2 v = µ 1,..., µ m, ν 1,..., ν p, p, m N. λ 1 µ {{ i x i + p ν i i, p λ 2 ν {{ i i conic(a). { k (b) For k N, let L k := µi x i : x i A, µ i for all i. We will prove b induction that L k B for all k N, and thus B k N Lk = conic(a). We start b proving the case of k = 1. If L 1 then = µx for some µ and x A. We thus have x B, and as B is a cone we have = µx B. We now suppose the statement is true for k, and show it is also true for k + 1. If L k+1 then = k+1 µi x i where x i A and µ i for all i. Letting z 1 = k 2µi x i L k B and z 2 = 2µ k+1 x k+1 L 1 B, the set B being convex implies that B 1 2 z1 + 1 2 z2 =. 4
Alternativel: { m conic(a) = µ i x i : x i A, µ i for all i, m N { m = { µ i x i : x i A, µ i for all i, m N, λ = = { { λ θ i x i : x i A, θ i for all i, m N, 1 = = { R ++ conv(a) = R + conv(a). As B is convex, we have conv(a) B. As B is a cone we then get B R + conv(a) = conic(a). (c) We will prove the equivalent statement that conic(a) is not full dimensional if and onl if there exists R n \ { such that, x = for all x A. ( ) Suppose conic(a) is not full-dimensional. Then b definition 7.8.3 there exists R n \ { such that, x = for all x conic A. We triviall have A conic(a) and thus, x = for all x A. ( ) Suppose there exists R n \ { such that, x = for all x A. Then for all z conic(a) we have z = m µi x i for some x i A and µ i for all i, m N, and thus, z = m µi, x i =. Therefore, b definition 7.8.3, we have that conic(a) is not fulldimensional. µ i > θ i, λ > 6. In this question we will consider the proper cone K R n+2 defined as x K = : R n, x, z R, 2 x, z. z (a) (b) (c) Consider a ra R = {c 1 a 1 R + with fixed a, c R n. We wish to find the distance between the origin and the closest point in this ra. Formulate this problem as a conic optimisation problem over K. Give an explicit characterisation of K. [Justification for our answer must be provided] What is the dual problem to our formulation in part (a)? [If ou were not able to answer parts (a) and (b) then instead find the dual to: c + a R n +. ] [1 point] 5
Solution: (a) This problem is equivalent to the following problems 1 c 1 a 2 1, 2 c 1 a 2 2, 1, 2 c 1 a 1 2 1 K max 1 2 1 c 1 a 2 K 1 The correct answer is either of the last two formulations, or equivalent. (b) We have that K = L n R +, and thus K = L n R + = L n R + = K. (c) Considering max 1 2 1 c 1 a 2 K 1 6
the dual problem is x,,z x c, z x a, = 1 z 1 x, = 1, z x K z This can be simplified to max x,,z c, z = a, which in turn is equivalent to x = 1, z, 2 x max c, a,, 2 1. Alternative question: The problem is equivalent to max c ( a) R n +. The dual to this is x c, x a, x = 1, x R n +, which is equivalent to max x c, x a, x = 1, x R n + 7. Consider the following optimisation problem: x 2x 2 2 + 5x 1 x 2 4x 2 2x 2 1 + x 1 + 3x 2 2 2x 1 x 2 = 3 (A) x R 2. Give the standard positive semidefinite approximation for this problem, the solution of which would provide a lower bound to the optimal value of problem (A). 7
Solution: This problem is equivalent to ( ) 5/2, xx T 4x x 5/2 2 2 ( ) 2 1, xx T + x 1 3 1 = 3 ( ) 1 x T x xx T PSD 3 x R 3, which can be relaxed to x,x ( ) 5/2, X 4x 5/2 2 2 ( ) 2 1, X + x 1 3 1 = 3 ( ) 1 x T PSD 3 x X x R 3 8. (Automatic additional points) [4 points] Question: 1 2 3 4 5 6 7 8 Total Points: 3 5 11 3 6 5 3 4 4 A cop of the lecture-sheets ma be used during the exaation. Good luck! 8