4. Convex optimization problems (part 1: general)

Size: px

Start display at page:

Download "4. Convex optimization problems (part 1: general)"

Jonas Flowers
6 years ago
Views:

1 EE/AA 578, Univ of Washington, Fall Convex optimization problems (part 1: general) optimization problem in standard form convex optimization problems quasiconvex optimization 4 1

2 Optimization problem in standard form minimize f 0 (x) subject to f i (x) 0, i = 1,...,m h i (x) = 0, i = 1,...,p x R n is the optimization variable f 0 : R n R is the objective or cost function f i : R n R, i = 1,...,m, are the inequality constraint functions h i : R n R are the equality constraint functions optimal value: p = inf{f 0 (x) f i (x) 0, i = 1,...,m, h i (x) = 0, i = 1,...,p} p = if problem is infeasible (no x satisfies the constraints) p = if problem is unbounded below 4 2

3 Optimal and locally optimal points x is feasible if x domf 0 and it satisfies the constraints a feasible x is optimal if f 0 (x) = p ; X opt is the set of optimal points x is locally optimal if there is an R > 0 such that x is optimal for minimize (over z) f 0 (z) subject to f i (z) 0, i = 1,...,m, h i (z) = 0, i = 1,...,p z x 2 R examples (with n = 1, m = p = 0) f 0 (x) = 1/x, domf 0 = R ++ : p = 0, no optimal point f 0 (x) = logx, domf 0 = R ++ : p = f 0 (x) = xlogx, domf 0 = R ++ : p = 1/e, x = 1/e is optimal f 0 (x) = x 3 3x, p =, local optimum at x = 1 4 3

4 Implicit constraints the standard form optimization problem has an implicit constraint x D = m i=0 domf i p i=1 domh i, we call D the domain of the problem the constraints f i (x) 0, h i (x) = 0 are the explicit constraints a problem is unconstrained if it has no explicit constraints (m = p = 0) example: minimize f 0 (x) = k i=1 log(b i a T i x) is an unconstrained problem with implicit constraints a T i x < b i 4 4

5 Feasibility problem find x subject to f i (x) 0, i = 1,...,m h i (x) = 0, i = 1,...,p can be considered a special case of the general problem with f 0 (x) = 0: minimize 0 subject to f i (x) 0, i = 1,...,m h i (x) = 0, i = 1,...,p p = 0 if constraints are feasible; any feasible x is optimal p = if constraints are infeasible 4 5

6 Convex optimization problem standard form convex optimization problem minimize f 0 (x) subject to f i (x) 0, i = 1,...,m a T i x = b i, i = 1,...,p f 0, f 1,..., f m are convex; equality constraints are affine problem is quasiconvex if f 0 is quasiconvex (and f 1,..., f m convex) often written as minimize f 0 (x) subject to f i (x) 0, i = 1,...,m Ax = b important property: feasible set of a convex optimization problem is convex 4 6

7 example minimize f 0 (x) = x 2 1+x 2 2 subject to f 1 (x) = x 1 /(1+x 2 2) 0 h 1 (x) = (x 1 +x 2 ) 2 = 0 f 0 is convex; feasible set {(x 1,x 2 ) x 1 = x 2 0} is convex not a convex problem (according to our definition): f 1 is not convex, h 1 is not affine equivalent (but not identical) to the convex problem minimize x 2 1+x 2 2 subject to x 1 0 x 1 +x 2 = 0 4 7

8 Local and global optima any locally optimal point of a convex problem is (globally) optimal proof: suppose x is locally optimal and y is optimal with f 0 (y) < f 0 (x) x locally optimal means there is an R > 0 such that z feasible, z x 2 R = f 0 (z) f 0 (x) consider z = θy +(1 θ)x with θ = R/(2 y x 2 ) y x 2 > R, so 0 < θ < 1/2 z is a convex combination of two feasible points, hence also feasible z x 2 = R/2 and f 0 (z) θf 0 (x)+(1 θ)f 0 (y) < f 0 (x) which contradicts our assumption that x is locally optimal 4 8

9 Optimality criterion for differentiable f 0 x is optimal if and only if it is feasible and f 0 (x) T (y x) 0 for all feasible y f X x f 0 (x) if nonzero, f 0 (x) defines a supporting hyperplane to feasible set X at x 4 9

10 unconstrained problem: x is optimal if and only if x domf 0, f 0 (x) = 0 equality constrained problem minimize f 0 (x) subject to Ax = b x is optimal if and only if there exists a ν such that x domf 0, Ax = b, f 0 (x)+a T ν = 0 minimization over nonnegative orthant x is optimal if and only if minimize f 0 (x) subject to x 0 x domf 0, x 0, { f0 (x) i 0 x i = 0 f 0 (x) i = 0 x i >

11 Equivalent convex problems two problems are (informally) equivalent if the solution of one is readily obtained from the solution of the other, and vice-versa some common transformations that preserve convexity: eliminating equality constraints is equivalent to minimize f 0 (x) subject to f i (x) 0, i = 1,...,m Ax = b minimize (over z) f 0 (Fz +x 0 ) subject to f i (Fz +x 0 ) 0, i = 1,...,m where F and x 0 are such that Ax = b x = Fz +x 0 for some z 4 11

12 introducing equality constraints is equivalent to minimize f 0 (A 0 x+b 0 ) subject to f i (A i x+b i ) 0, i = 1,...,m minimize (over x, y i ) f 0 (y 0 ) subject to f i (y i ) 0, i = 1,...,m y i = A i x+b i, i = 0,1,...,m introducing slack variables for linear inequalities is equivalent to minimize f 0 (x) subject to a T i x b i, i = 1,...,m minimize (over x, s) f 0 (x) subject to a T i x+s i = b i, i = 1,...,m s i 0, i = 1,...m 4 12

13 epigraph form: standard form convex problem is equivalent to minimize (over x, t) t subject to f 0 (x) t 0 f i (x) 0, i = 1,...,m Ax = b minimizing over some variables is equivalent to where f 0 (x 1 ) = inf x2 f 0 (x 1,x 2 ) minimize f 0 (x 1,x 2 ) subject to f i (x 1 ) 0, i = 1,...,m minimize f0 (x 1 ) subject to f i (x 1 ) 0, i = 1,...,m 4 13

14 Quasiconvex optimization minimize f 0 (x) subject to f i (x) 0, i = 1,...,m Ax = b with f 0 : R n R quasiconvex, f 1,..., f m convex can have locally optimal points that are not (globally) optimal convex representation of sublevel sets of f 0 if f 0 is quasiconvex, there exists a family of functions φ t such that: φ t (x) is convex in x for fixed t t-sublevel set of f 0 is 0-sublevel set of φ t, i.e., f 0 (x) t φ t (x)

15 example f 0 (x) = p(x) q(x) with p convex, q concave, and p(x) 0, q(x) > 0 on domf 0 can take φ t (x) = p(x) tq(x): for t 0, φ t convex in x p(x)/q(x) t if and only if φ t (x)

16 quasiconvex optimization via convex feasibility problems φ t (x) 0, f i (x) 0, i = 1,...,m, Ax = b (1) for fixed t, a convex feasibility problem in x if feasible, we can conclude that t p ; if infeasible, t p Bisection method for quasiconvex optimization given l p, u p, tolerance ǫ > 0. repeat 1. t := (l + u)/2. 2. Solve the convex feasibility problem (1). 3. if (1) is feasible, u := t; else l := t. until u l ǫ. requires exactly log 2 ((u l)/ǫ) iterations (where u, l are initial values) 4 16

4. Convex optimization problems

Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization