Continuous Optimisation, Chpt 7: Proper Cones

Transcription

1 Continuous Optimisation, Chpt 7: Proper Cones Peter J.C. Dickinson DMMP, University of Twente version: 10/11/17 Monday 13th November 2017 Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 1/25

2 Book Convex Optimization Boyd and Vandenberghe boyd/cvxbook/ Proper cones: Proper cones and generalized inequalities, p43 Conic Optimisation: 4.3 Linear optimization problems, p Second-order cone programming, p Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 2/25

3 Table of Contents 1 Introduction 2 Proper Cones Cones Convex Cones Conic Hull Pointed Full-dimensional Proper Cones 3 Conic Optimisation Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 3/25

4 Cones Definition 7.1 We say that a set A R n is a cone if for all x A and all µ > 0 we have µx A. {x R 2 x 1 x 2 } {x R 2 x 2 1, 2 x 2 x 1 } Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 4/25

5 Cones Definition 7.1 We say that a set A R n is a cone if for all x A and all µ > 0 we have µx A. {x R 2 x 1 x 2 } {x R 2 x 2 1, 2 x 2 x 1 } Cone Not a cone Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 4/25

6 (a) (b) (c) {x R 2 x 2 3 x 1 } {x R 2 3x 2 x 1 } { x R 2 : 2x 2 x 1 2x 1 x 2 } { (d) (e) (f) x R 2 : 2x } 2 x x 1 x {x R 2 x 1 2x 2 } {x R 2 x 2 1 x 2} {x R 2 x 2 1 x 2} Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 5/25

7 (a) Cone (b) Cone (c) Not cone {x R 2 x 2 3 x 1 } {x R 2 3x 2 x 1 } { x R 2 : 2x 2 x 1 2x 1 x 2 } { (d) Cone (e) Not cone (f) Not cone x R 2 : 2x } 2 x x 1 x {x R 2 x 1 2x 2 } {x R 2 x 2 1 x 2} {x R 2 x 2 1 x 2} Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 5/25

8 Convex Cones Theorem 7.2 A R n is a convex cone if and only if λ 1 x + λ 2 y A for all λ 1, λ 2 > 0 and all x, y A. Ex. 7.1 Prove Theorem 7.2. Ex. 7.2 For arbitrary a R n show that the following set is a convex cone: A = {x R n a, x 0}. N.B. This set is also closed. N.B. For u, v R n we have u, v = u T v = n i=1 u iv i. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 6/25

9 Second order cone Lemma 7.3 For arbitrary seminorm p : R n R, the following set is a closed convex cone: A = {(x 0, x) R R n p(x) x 0 }. Example This cone is most commonly considered with either the 1-norm, x 1 = n i=1 x i, (a.k.a. taxi-cab norm) the -norm, x = max i { x i : i = 1..., n}, or the Euclidean norm (2-norm), x 2 = n i=1 x 2 i. In this case the cone is called the Second Order Cone (a.k.a. Lorentz cone or Ice-cream cone), and we will denote it by L n. [local file] Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 7/25

10 Theorem 7.4 Suppose that K 1, K 2 R n are convex cones. Then so is K 1 + K 2 := {z R n x K 1, y K 2 s. t. z = x + y}. Theorem 7.5 Suppose that K 1, K 2 R n are convex cones. Then so is K 1 K 2 := {z R n z K 1, z K 2 }. Ex. 7.3 Prove Theorem 7.5. Example The set of Nonnegative Vectors is a closed convex cone: n R n + := {x R n x i 0 i} = {x R n e i, x 0}. i=1 Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 8/25

11 Conic Hull Definition 7.6 Let A R n be an arbitrary set. The conic hull of A is { k } conic(a) := λ i x i : x i A, λ i 0, k 0. i=1 This is the smallest convex cone containing {0} A. If A \ {0} is a compact set then conic(a) is closed. Example The set of nonnegative vectors is a closed convex cone: R n + = {x R n x i 0 i} = conic{e 1,..., e n }. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 9/25

12 Pointed Definition 7.7 A set K R n is pointed if x R n \ {0} such that ±x K. { x R 2 : 2x } 2 x 1 2x 1 x 2 Pointed Example R n + is pointed. {x R 2 x 1 2x 2 } Not pointed Ex. 7.4 For arbitrary norm, show that the following set is pointed: A = {(x 0, x) R R n x x 0 }. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 10/25

13 Full-dimensional Definition 7.8 We say that a convex cone K R n is full-dimensional (or solid) if one of the following equivalent conditions hold: 1 K has a nonempty interior, 2 There are linearly independent vectors x 1,..., x n K, 3 y R n \ {0} such that y, x = 0 for all x K. Example R n + is full-dimensional. Ex. 7.5 For an arbitrary seminorm, p : R n R, show that the set A = {(x 0, x) R R n p(x) x 0 } is full-dimensional. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 11/25

14 Proper Cones Definition 7.9 K R n is a proper cone if it is a cone which is closed, convex, pointed and full dimensional. Example R n + and {(x 0, x) R R n x x 0 } are proper cones. (For any norm,.) Theorem 7.10 If K 1 R n and K 2 R m are proper cones then K 1 K 2 := {(x, y) x K 1, y K 2 } is a proper cone. Ex. 7.6 Prove Theorem Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 12/25

15 Examples Ex. 7.7 Which categories do the sets A 1,..., A 5 fall into out of: (a) Not a convex cone, (b) Convex cone; Not pointed nor full-dimensional, (c) Convex cone; Pointed but not full-dimensional, (d) Convex cone; Full-dimensional but not pointed, (e) Proper cone (closed convex full-dimensional pointed cone). A 1 = {x R n a T 1 x 0}, A 2 = {x R n x 2 1}, A 3 = conic{a 1,..., a n }, A 4 = {λa 1 λ R}, A 5 = {λa 1 λ 0}, where n 2 and a 1,..., a n R n linearly independent. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 13/25

16 Nonnegative polynomials Ex. 7.8 For V R + with nonempty interior show that the following set is a proper cone: { } n K = a R n : a 1 + a i x i 1 0 for all x V i=2 Hint: For two polynomials p, q : R R and an open set A, we have p(x) = q(x) for all x A if and only if p and q are coefficientwise identical. Lemma 7.11 For a natural number d and a compact set V R n with nonempty interior, we have that the set of polynomials of degree less than or equal to d which are nonnegative over V is a proper cone (in the space of all polynomials of degree less than or equal to d). Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 14/25

17 Table of Contents 1 Introduction 2 Proper Cones 3 Conic Optimisation Conic Optimisation Linear Optimisation Second Order Cone Optimisation Second Order Cone Optimisation Extended formulations Polynomial Optimisation Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 15/25

18 Conic Optimisation Conic optimisation problems are optimisation problems with only affine functions and convex cone constraints. We shall consider instances with one of the following standard forms, where K R n is a convex cone, c, a 1,..., a m R n, b R m : min c, x s. t. a i, x = b i for all i = 1,..., m x K, max b T y s. t. c m y i a i K i=1 y R m, Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 16/25

19 Linear Optimisation Definition 7.12 A linear optimisation problem is an optimisation problem composed of affine functions, equality constraints and inequality constraints. Such a problem can then be converted to the standard form from the previous slide over the cone of nonnegative vectors. Solvers Commercial: CPLEX, GUROBI, MOSEK. Free: SCIP. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 17/25

20 Example In chapter 4 (Wolfe-dual), showed how for convex problems min x C {f (x) : g i(x) 0 for all i = 1,..., m} and for arbitrary fixed x C, the following linear optimisation problem gives a lower bound on this: { } m m max f ( x) + y i g i ( x) : f ( x) + y i g i ( x) = 0, y R m y +. i=1 This is equivalent to the problem: { f ( x) min c, y : aj, y = b j j = 1,..., n, y R m } y +, where g 1 ( x) e T j g 1( x) c =., a j =., b j = e T j f ( x). g m ( x) e T j g m( x) i=1 Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 18/25

21 Ex. 7.9 (Adapted from Q4.14, Convex Optimization, Boyd and Vandenberghe, 2004) Consider a system of canals which meet at n nodes. The nodes themselves cannot store water, so the amount of water flowing into them must equal the amount of water flowing out. We will let the variable x ij be the amount of water we allow to flow from node i to node j. The cost of this flow is c ij x ij (we may have to pay to pump the water uphill). Each canal has a lower bound l ij and an upper bound u ij for the amount of water that can flow from node i to node j. An external supply of water flowing into node i is b i (this may be negative if the water is flowing out of this node). We wish to minimize the total cost. Formulate this problem as a linear optimization problem (in both a simple and a standard form). Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 19/25

22 Converting 1-norm and -norms to linear optimisation Example 1-norm, x 1 := n i=1 x i, -norm, x := max{ x 1,..., x n }, 2-norm, x 2 := x, x. Recall for any norm {(x 0, x) R R n x x 0 } is a proper cone. Lemma 7.13 x x 0 x 0 x i x 0 for all i = 1,..., n. x 1 x 0 y R n s. t. y i x i y i for all i = 1,..., n and n i=1 y i x 0. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 20/25

23 Example norms min y m c y i a i i=1 min y,t s. t. ( ) T ( ) 1 t 0 y ( ) 0 t c ( 1 0 ) m ( ) 0 y i A, a i i=1 where A = {(z 0, z) R R n z z 0 }. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 21/25

24 Second Order Cone Optimisation min y m c y i a i i=1 2 min y,t s. t. ( ) T ( ) 1 t 0 y ( ) 0 t c ( 1 0 ) m ( ) 0 y i L a n, i i=1 where L n := {(z 0, z) R R n z 2 z 0 }. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 22/25

25 Second Order Cone optimisation solvers Commercial: CPLEX, GUROBI, MOSEK. Free: SDPT3, SEDUMI. Ex A new mobile mast needs to be built in order to service m villages. The coordinates of these villages are given by v 1,..., v m R 2. We consider two alternative problems for this: 1 We want to minimise the distance to the furthest village. 2 The mast cannot be more than R kilometers from any of the vilages and we want to minimise the sum of the distances to all of the villages. Formulate these two problems as second order cone optimisation problems. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 23/25

26 Extended formulations min x c x 2 : a T x = b, x R n +, { } = min x,z,t { } t : z 2 t, x + z = c, a T x = b, x R n + 0 x = min 1, t : x,z,t 0 z e i x 0, t = c i for all i = 1,..., n e i z a x x 0, t = b, t K 0 z z x where K = t R 2n+1 : x Rn +, z z 2 t = Rn + L n. Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 24/25

27 Polynomial Optimisation Consider a compact set V R n with nonempty interior and a polynomial f : R n R of degree d. Let K be the set of polynomials of degree less than or equal to d which are nonnegative over V. From Lemma 7.11 this is a proper cone. We have min {f (x) : x V} = max{λ : f (x) λ for all x V} x λ = max{λ : f (x) λ 0 for all x V} λ = max{λ : f (x) λ K}. λ Peter J.C. Dickinson CO17, Chpt 7: Proper Cones 25/25