Convex optimization problems. Optimization problem in standard form

Size: px
Start display at page:

Download "Convex optimization problems. Optimization problem in standard form"

Transcription

1 Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality constraints semidefinite programming vector optimization IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 1 Optimization problem in standard form minimize f 0 (x) subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p x R n is the optimization variable f 0 : R n R is the objective or cost function f i : R n R, i = 1,..., m, are the inequality constraint functions h i : R n R are the equality constraint functions optimal value: p = inf{f 0 (x) f i (x) 0, i = 1,..., m, h i (x) = 0, i = 1,..., p} p = if problem is infeasible (no x satisfies the constraints) p = if problem is unbounded below IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 2

2 Optimal and locally optimal points x is feasible if x dom f 0 and it satisfies the constraints a feasible x is optimal if f 0 (x) = p ; X opt is the set of optimal points x is locally optimal if there is an R > 0 such that x is optimal for minimize (over z) f 0 (z) subject to f i (z) 0, i = 1,..., m, h i (z) = 0, i = 1,..., p z x 2 R examples (with n = 1, m = p = 0) f 0 (x) = 1/x, dom f 0 = R ++ : p = 0, no optimal point f 0 (x) = log x, dom f 0 = R ++ : p = f 0 (x) = x log x, dom f 0 = R ++ : p = 1/e, x = 1/e is optimal f 0 (x) = x 3 3x, p =, local optimum at x = 1 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 3 Implicit constraints The standard form optimization problem has an implicit constraint m p x D = dom f i dom h i, i=0 i=1 we call D the domain of the problem the constraints f i (x) 0, h i (x) = 0 are the explicit constraints a problem is unconstrained if it has no explicit constraints (m = p = 0) example: minimize f 0 (x) = k i=1 log(b i a T i x) is an unconstrained problem with implicit constraints a T i x < b i, i = 1,..., k IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 4

3 Feasibility problem find x subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p can be considered a special case of the general problem with f 0 (x) = 0: minimize 0 subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p p = 0 if constraints are feasible; any feasible x is optimal p = if constraints are infeasible IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 5 Convex optimization problem standard form convex optimization problem minimize f 0 (x) subject to f i (x) 0, i = 1,..., m ai T x = b i, i = 1,..., p f 0, f 1,..., f m are convex; equality constraints are affine problem is quasiconvex if f 0 is quasiconvex (and f 1,..., f m convex) often written as minimize f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b important property: feasible set of a convex optimization problem is convex IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 6

4 example minimize f 0 (x) = x1 2 + x 2 2 subject to f 1 (x) = x 1 /(1 + x2 2) 0 h 1 (x) = (x 1 + x 2 ) 2 = 0 f 0 is convex; feasible set {(x 1, x 2 ) x 1 = x 2 0} is convex not a convex problem (according to our definition): f 1 is not convex, h 1 is not affine equivalent (but not identical) to the convex problem minimize x x 2 2 subject to x 1 0 x 1 + x 2 = 0 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 7 Local and global optima Any locally optimal point of a convex problem is (globally) optimal proof: suppose x is locally optimal and y is optimal with f 0 (y) < f 0 (x). x locally optimal means there is an R > 0 such that z feasible, z x 2 R = f 0 (z) f 0 (x). Consider z = θy + (1 θ)x with θ = R/(2 y x 2 ) y x 2 > R, so 0 < θ < 1/2 z is a convex combination of two feasible points, hence also feasible z x 2 = R/2 and f 0 (z) θf 0 (x) + (1 θ)f 0 (y) < f 0 (x), which contradicts our assumption that x is locally optimal IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 8

5 Optimality criterion for differentiable f 0 x is optimal if and only if it is feasible and f 0 (x) T (y x) 0 for all feasible y X x f 0 (x) if nonzero, f 0 (x) defines a supporting hyperplane to feasible set X at x IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 9 Optimality criterion: special cases unconstrained problem: x is optimal if and only if x dom f 0, f 0 (x) = 0 equality constrained problem minimize f 0 (x) subject to Ax = b x is optimal if and only if there exists a ν such that x dom f 0, Ax = b, f 0 (x) + A T ν = 0 minimization over nonnegative orthant x is optimal if and only if minimize f 0 (x) subject to x 0 x dom f 0, x 0, { f0 (x) i 0 x i = 0 f 0 (x) i = 0 x i > 0 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 10

6 Equivalent convex problems two problems are (informally) equivalent if the solution of one is readily obtained from the solution of the other, and vice-versa. Some common transformations that preserve convexity: eliminating equality constraints is equivalent to minimize f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b minimize (over z) f 0 (Fz + x 0 ) subject to f i (Fz + x 0 ) 0, i = 1,..., m where F and x 0 are such that Ax = b x = Fz + x 0 for some z IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 11 introducing equality constraints is equivalent to minimize f 0 (A 0 x + b 0 ) subject to f i (A i x + b i ) 0, i = 1,..., m minimize (over x, y i ) f 0 (y 0 ) subject to f i (y i ) 0, i = 1,..., m y i = A i x + b i, i = 0, 1,..., m introducing slack variables for linear inequalities is equivalent to minimize f 0 (x) subject to ai T x b i, i = 1,..., m minimize (over x, s) f 0 (x) subject to ai T x + s i = b i, i = 1,..., m s i 0, i = 1,... m IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 12

7 epigraph form: standard form convex problem is equivalent to minimize (over x, t) t subject to f 0 (x) t 0 f i (x) 0, i = 1,..., m Ax = b minimizing over some variables is equivalent to minimize f 0 (x 1, x 2 ) subject to f i (x 1 ) 0, i = 1,..., m minimize f0 (x 1 ) subject to f i (x 1 ) 0, i = 1,..., m where f 0 (x 1 ) = inf x2 f 0 (x 1, x 2 ) IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 13 Linear program (LP) minimize subject to c T x + d Gx h Ax = b convex problem with affine objective and constraint functions feasible set is a polyhedron c s P x IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 14

8 Examples diet problem: choose quantities x 1,..., x n of n foods one unit of food j costs c j, contains amount a ij of nutrient i healthy diet requires nutrient i in quantity at least b i to find cheapest healthy diet, minimize c T x subject to Ax b, x 0 piecewise-linear minimization minimize max i=1,...,m (ai T x + b i ) equivalent to an LP minimize t subject to ai T x + b i t, i = 1,..., m IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 15 Chebyshev center of a polyhedron Chebyshev center of P = {x a T i x b i, i = 1,..., m} is center of largest inscribed ball x cheb B = {x c + u u 2 r} s a T i x b i for all x B if and only if sup{a T i (x c + u) u 2 r} = a T i x c + r a i 2 b i hence, x c, r can be determined by solving the LP maximize xc,r r subject to ai T x c + r a i 2 b i, i = 1,..., m IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 16

9 Quadratic program (QP) minimize subject to (1/2)x T Px + q T x + r Gx h Ax = b P S n +, so objective is convex quadratic minimize a convex quadratic function over a polyhedron f 0 (x ) x P IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 17 Examples least-squares minimize Ax b 2 2 analytical solution x = A b (A is pseudo-inverse) can add linear constraints, e.g., l x u linear program with random cost minimize c T x + γx T Σx = E c T x + γ var(c T x) subject to Gx h, Ax = b c is random vector with mean c and covariance Σ hence, c T x is random variable with mean c T x and variance x T Σx γ > 0 is risk aversion parameter; controls the trade-off between expected cost and variance (risk) IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 18

10 Quadratically constrained quadratic program (QCQP) minimize (1/2)x T P 0 x + q0 T x + r 0 subject to (1/2)x T P i x + qi T x + r i 0, i = 1,..., m Ax = b P i S n +; objective and constraints are convex quadratic if P 1,..., P m S n ++, feasible region is intersection of m ellipsoids and an affine set IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 19 Second-order cone programming minimize f T x subject to A i x + b i 2 ci T x + d i, i = 1,..., m Fx = g (A i R n i n, F R p n ) inequalities are called second-order cone (SOC) constraints: (A i x + b i, c T i x + d i ) second-order cone in R n i +1 for n i = 0, reduces to an LP; if c i = 0, reduces to a QCQP more general than QCQP and LP IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 20

11 Robust linear programming the parameters in optimization problems are often uncertain, e.g., in an LP minimize c T x subject to ai T x b i, i = 1,..., m, there can be uncertainty in c, a i, b i two common approaches to handling uncertainty (in a i, for simplicity) deterministic model: constraints must hold for all a i E i minimize c T x subject to ai T x b i for all a i E i, i = 1,..., m, stochastic model: a i is random variable; constraints must hold with probability η minimize c T x subject to prob(ai T x b i ) η, i = 1,..., m IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 21 Deterministic approach via SOCP choose an ellipsoid as E i : E i = {ā i + P i u u 2 1} (ā i R n, P i R n n ) center is ā i, semi-axes determined by singular values/vectors of P i robust LP minimize c T x subject to ai T x b i a i E i, i = 1,..., m is equivalent to the SOCP minimize c T x subject to āi T x + Pi T x 2 b i, i = 1,..., m (follows from sup u 2 1(ā i + P i u) T x = ā T i x + P T i x 2 ) IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 22

12 Stochastic approach via SOCP assume a i is Gaussian with mean ā i, covariance Σ i (a i N (ā i, Σ i )) ai T x is Gaussian r.v. with mean āi T x, variance x T Σ i x; hence ( ) prob(ai T b i āi T x x b i ) = Φ Σ 1/2 i x 2 where Φ(x) = (1/ 2π) x e t2 /2 dt is CDF of N (0, 1) robust LP minimize c T x subject to prob(ai T x b i ) η, i = 1,..., m, with η 1/2, is equivalent to the SOCP minimize c T x subject to āi T x + Φ 1 (η) Σ 1/2 i x 2 b i, i = 1,..., m IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 23 Geometric programming monomial function f (x) = cx a 1 1 x a 2 2 x a n n, dom f = R n ++ with c > 0; exponent α i can be any real number posynomial function: sum of monomials f (x) = K k=1 geometric program (GP) c k x a 1k 1 x a 2k 2 x a nk n, dom f = R n ++ minimize f 0 (x) subject to f i (x) 1, i = 1,..., m h i (x) = 1, i = 1,..., p with f i posynomial, h i monomial IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 24

13 Geometric program in convex form change variables to y i = log x i, and take logarithm of cost, constraints monomial f (x) = cx a 1 1 x a n n transforms to log f (e y 1,..., e y n ) = a T y + b (b = log c) posynomial f (x) = K k=1 c kx a 1k 1 x a 2k 2 x a nk n ( K ) log f (e y 1,..., e y n ) = log e at k y+b k k=1 transforms to (b k = log c k ) geometric program transforms to convex problem ( K ) minimize log k=1 exp(at 0k y + b 0k) ( K ) subject to log k=1 exp(at ik y + b ik) 0, i = 1,..., m Gy + d = 0 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 25 Design of cantilever beam segment 4 segment 3 segment 2 segment 1 N segments with unit lengths, rectangular cross-sections of size w i h i given vertical force F applied at the right end design problem minimize subject to total weight upper & lower bounds on w i, h i upper bound & lower bounds on aspect ratios h i /w i upper bound on stress in each segment upper bound on vertical deflection at the end of the beam variables: w i, h i for i = 1,..., N IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 26 F

14 Objective and constraint functions total weight w 1 h w N h N is posynomial aspect ratio h i /w i and inverse aspect ratio w i /h i are monomials maximum stress in segment i is given by 6iF /(w i h 2 i ), a monomial the vertical deflection y i and slope v i of central axis at the right end of segment i are defined recursively as v i = F 12(i 1/2) Ew i hi 3 + v i+1 y i = F 6(i 1/3) Ew i hi 3 + v i+1 + y i+1 for i = N, N 1,..., 1, with v N+1 = y N+1 = 0 (E is Young s modulus) v i and y i are posynomial functions of w, h IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 27 Formulation as a GP note minimize subject to w 1 h w N h N w 1 maxw i 1, h 1 maxh i 1, w min w 1 i 1, i = 1,..., N h min h 1 i 1, i = 1,..., N S 1 maxw 1 i h i 1, S min w i h 1 i 1, i = 1,..., N 6iF σ 1 maxw 1 i y 1 maxy 1 1 h 2 i 1, i = 1,..., N we write w min w i w max and h min h i h max w min /w i 1, w i /w max 1, h min /h i 1, h i /h max 1 we write S min h i /w i S max as S min w i /h i 1, h i /(w i S max ) 1 The number of monomials appearing in y 1 grows approximately as N 2. IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 28

15 Minimizing spectral radius of nonnegative matrix Perron-Frobenius eigenvalue λ pf (A) Consider (elementwise) positive A R n n that is irreducible: (I + A) n 1 > 0. P-F Theorem: there is a real, positive eigenvalue of A, λ pf, equal to spectral radius max i λ i (A) determines asymptotic growth (decay) rate of A k : A k λ k pf as k alternative characterization: λ pf (A) = inf{λ Av λv for some v 0} minimizing spectral radius of matrix of posynomials minimize λ pf (A(x)), where the elements A(x) ij are posynomials of x equivalent geometric program: minimize subject to variables λ, v, x λ n j=1 A(x) ijv j /(λv i ) 1, i = 1,..., n IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 29 Quasiconvex optimization minimize f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b with f 0 : R n R quasiconvex, f 1,..., f m convex can have locally optimal points that are not (globally) optimal (x, f 0 (x)) IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 30

16 convex representation of sublevel sets of f 0 if f 0 is quasiconvex, there exists a family of functions φ t such that: φ t (x) is convex in x for fixed t t-sublevel set of f 0 is 0-sublevel set of φ t, i.e., f 0 (x) t φ t (x) 0 example f 0 (x) = p(x) q(x) with p convex, q concave, and p(x) 0, q(x) > 0 on dom f 0 can take φ t (x) = p(x) tq(x): for t 0, φ t convex in x p(x)/q(x) t if and only if φ t (x) 0 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 31 quasiconvex optimization via convex feasibility problems φ t (x) 0, f i (x) 0, i = 1,..., m, Ax = b (1) for fixed t, a convex feasibility problem in x if feasible, we can conclude that t p ; if infeasible, t p Bisection method for quasiconvex optimization given l p, u p, tolerance ɛ > 0. repeat 1. t := (l + u)/2. 2. Solve the convex feasibility problem (1). 3. if (1) is feasible, u := t; else l := t. until u l ɛ. requires exactly log 2 ((u l)/ɛ) iterations (where u, l are initial values) IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 32

17 (Generalized) linear-fractional program linear-fractional program minimize subject to f 0 (x) Gx h Ax = b f 0 (x) = ct x + d e T x + f, dom f 0(x) = {x e T x + f > 0} a quasiconvex optimization problem; can be solved by bisection also equivalent to the LP (variables y, z) minimize subject to c T y + dz Gy hz Ay = bz e T y + fz = 1 z 0 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 33 Generalized linear-fractional program f 0 (x) = max i=1,...,r ci T x + d i ei T, dom f 0 (x) = {x ei T x+f i > 0, i = 1,..., r} x + f i a quasiconvex optimization problem; can be solved by bisection example: Von Neumann model of a growing economy maximize (over x, x + ) min i=1,...,n x + i /x i subject to x + 0, Bx + Ax x, x + R n : activity levels of n sectors, in current and next period (Ax) i, (Bx + ) i : produced, resp. consumed, amounts of good i x + i /x i : growth rate of sector i allocate activity to maximize growth rate of slowest growing sector IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 34

18 Generalized inequality constraints convex problem with generalized inequality constraints minimize f 0 (x) subject to f i (x) Ki 0, i = 1,..., m Ax = b f 0 : R n R convex; f i : R n R k i cone K i K i -convex w.r.t. proper same properties as standard convex problem (convex feasible set, local optimum is global, etc.) conic form problem: special case with affine objective and constraints minimize c T x subject to Fx + g K 0 Ax = b extends linear programming (K = R m +) to nonpolyhedral cones IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 35 Semidefinite program (SDP) with F i, G S k minimize c T x subject to x 1 F 1 + x 2 F x n F n + G 0 Ax = b inequality constraint is called linear matrix inequality (LMI) includes problems with multiple LMI constraints: for example, x 1 ˆF x n ˆF n + Ĝ 0, x 1 F x n F n + G 0 is equivalent to single LMI x 1 [ ˆF F 1 ] +x 2 [ ˆF F 2 ] + +x n [ ˆF n 0 0 F n ] [ Ĝ G ] 0 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 36

19 LP and SOCP as SDP LP and equivalent SDP LP: minimize c T x subject to Ax b SDP: minimize c T x subject to diag(ax b) 0 (note different interpretation of generalized inequality ) SOCP and equivalent SDP SOCP: minimize f T x subject to A i x + b i 2 c T i x + d i, i = 1,..., m SDP: minimize f [ T x (c T subject to i x + d i )I A i x + b i (A i x + b i ) T ci T x + d i ] 0, i = 1,..., m IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 37 Eigenvalue minimization minimize λ max (A(x)) where A(x) = A 0 + x 1 A x n A n (with given A i S k ) equivalent SDP minimize subject to t A(x) ti variables x R n, t R follows from λ max (A) t A ti IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 38

20 Matrix norm minimization minimize A(x) 2 = ( λ max (A(x) T A(x)) ) 1/2 where A(x) = A 0 + x 1 A x n A n (with given A i S p q ) equivalent SDP minimize subject to t [ ti A(x) T A(x) ti ] 0 variables x R n, t R constraint follows from A 2 t A T A t 2 I, t 0 [ ] ti A A T 0 ti IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 39 Convexity of vector-valued functions f : R n R m is K-convex if dom f is convex and f (θx + (1 θ)y) K θf (x) + (1 θ)f (y) for x, y dom f, 0 θ 1 example f : S m S m, f (X ) = X 2 is S m +-convex proof: for fixed z R m, z T X 2 z = Xz 2 2 is convex in X, i.e., z T (θx + (1 θ)y ) 2 z θz T X 2 z + (1 θ)z T Y 2 z for X, Y S m, 0 θ 1 therefore (θx + (1 θ)y ) 2 θx 2 + (1 θ)y 2 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 40

21 Vector optimization general vector optimization problem minimize (w.r.t. K) f 0 (x) subject to f i (x) 0, i = 1,..., m h i (x) 0, i = 1,..., p vector objective f 0 : R n R q, minimized w.r.t. proper cone K R q convex vector optimization problem minimize (w.r.t. K) f 0 (x) subject to f i (x) 0, i = 1,..., m Ax = b with f 0 K-convex, f 1,..., f m convex IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 41 Optimal and Pareto optimal points set of achievable objective values O = {f 0 (x) x feasible} feasible x is optimal if f 0 (x) is a minimum value of O feasible x is Pareto optimal if f 0 (x) is a minimal value of O nts O P f 0 (x po ) O f 0 (x ) x is optimal x po is Pareto optimal IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 42

22 Multicriterion optimization vector optimization problem with K = R q + f 0 (x) = (F 1 (x),..., F q (x)) q different objectives F i ; roughly speaking we want all F i s to be small feasible x is optimal if y feasible = f 0 (x ) f 0 (y) if there exists an optimal point, the objectives are noncompeting feasible x po is Pareto optimal if y feasible, f 0 (y) f 0 (x po ) = f 0 (x po ) = f 0 (y) if there are multiple Pareto optimal values, there is a trade-off between the objectives IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 43 Regularized least-squares multicriterion problem with two objectives F 1 (x) = Ax b 2 2, F 2 (x) = x example with A R shaded region is O heavy line is formed by Pareto optimal points F2(x) = x F 1 (x) = Ax b 2 2 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 44

23 Risk return trade-off in portfolio optimization minimize (w.r.t. R 2 +) ( p T x, x T Σx) subject to 1 T x = 1, x 0 x R n is investment portfolio; x i is fraction invested in asset i p R n is vector of relative asset price changes; modeled as a random variable with mean p, covariance Σ p T x = E r is expected return; x T Σx = var r is return variance example mean return 15% 10% 5% allocation x x(4) x(3) x(2) x(1) 0% 0% 10% 20% standard deviation of return 0 0% 10% 20% standard deviation of return IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 45 Scalarization to find Pareto optimal points: choose λ K 0 and solve scalar problem minimize λ T f 0 (x) subject to f i (x) 0, i = 1,..., m h i (x) = 0, i = 1,..., p if x is optimal for scalar problem, then it is Pareto-optimal for vector optimization problem s f 0 (x 1 ) O f λ 0 (x 3 ) 1 f 0 (x 2 ) λ 2 for convex vector optimization problems, can find (almost) all Pareto optimal points by varying λ K 0 IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 46

24 examples for multicriterion problem, find Pareto optimal points by minimizing positive weighted sum λ T f 0 (x) = λ 1 F 1 (x) + + λ q F q (x) regularized least-squares of page 111 (with λ = (1, γ)) minimize Ax b γ x 2 2 for fixed γ > 0, a least-squares problem risk-return trade-off of page 112 (with λ = (1, γ)) minimize p T x + γx T Σx subject to 1 T x = 1, x 0 for fixed γ > 0, a QP IOE 611: Nonlinear Programming, Winter Convex optimization problems Page 4 47

4. Convex optimization problems

4. Convex optimization problems Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization

More information

Lecture: Convex Optimization Problems

Lecture: Convex Optimization Problems 1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization

More information

4. Convex optimization problems

4. Convex optimization problems Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization

More information

Convex Optimization. 4. Convex Optimization Problems. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University

Convex Optimization. 4. Convex Optimization Problems. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University Conve Optimization 4. Conve Optimization Problems Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2017 Autumn Semester SJTU Ying Cui 1 / 58 Outline Optimization problems

More information

4. Convex optimization problems (part 1: general)

4. Convex optimization problems (part 1: general) EE/AA 578, Univ of Washington, Fall 2016 4. Convex optimization problems (part 1: general) optimization problem in standard form convex optimization problems quasiconvex optimization 4 1 Optimization problem

More information

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics 1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history

More information

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics 1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history

More information

CSCI : Optimization and Control of Networks. Review on Convex Optimization

CSCI : Optimization and Control of Networks. Review on Convex Optimization CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one

More information

Convex Optimization and Modeling

Convex Optimization and Modeling Convex Optimization and Modeling Convex Optimization Fourth lecture, 05.05.2010 Jun.-Prof. Matthias Hein Reminder from last time Convex functions: first-order condition: f(y) f(x) + f x,y x, second-order

More information

A Brief Review on Convex Optimization

A Brief Review on Convex Optimization A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review

More information

Tutorial on Convex Optimization for Engineers Part II

Tutorial on Convex Optimization for Engineers Part II Tutorial on Convex Optimization for Engineers Part II M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de

More information

Convex Optimization Problems. Prof. Daniel P. Palomar

Convex Optimization Problems. Prof. Daniel P. Palomar Conve Optimization Problems Prof. Daniel P. Palomar The Hong Kong University of Science and Technology (HKUST) MAFS6010R- Portfolio Optimization with R MSc in Financial Mathematics Fall 2018-19, HKUST,

More information

12. Interior-point methods

12. Interior-point methods 12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity

More information

Lecture: Examples of LP, SOCP and SDP

Lecture: Examples of LP, SOCP and SDP 1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:

More information

Optimisation convexe: performance, complexité et applications.

Optimisation convexe: performance, complexité et applications. Optimisation convexe: performance, complexité et applications. Introduction, convexité, dualité. A. d Aspremont. M2 OJME: Optimisation convexe. 1/128 Today Convex optimization: introduction Course organization

More information

Convex Optimization and l 1 -minimization

Convex Optimization and l 1 -minimization Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l

More information

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics 1. Introduction mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history of convex optimization 1 1 Mathematical

More information

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics 1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history

More information

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics

1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics 1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history

More information

CS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems. Instructor: Shaddin Dughmi

CS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems. Instructor: Shaddin Dughmi CS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems Instructor: Shaddin Dughmi Outline 1 Convex Optimization Basics 2 Common Classes 3 Interlude: Positive Semi-Definite

More information

Exercises. Exercises. Basic terminology and optimality conditions. 4.2 Consider the optimization problem

Exercises. Exercises. Basic terminology and optimality conditions. 4.2 Consider the optimization problem Exercises Basic terminology and optimality conditions 4.1 Consider the optimization problem f 0(x 1, x 2) 2x 1 + x 2 1 x 1 + 3x 2 1 x 1 0, x 2 0. Make a sketch of the feasible set. For each of the following

More information

8. Geometric problems

8. Geometric problems 8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid

More information

Semidefinite Programming Basics and Applications

Semidefinite Programming Basics and Applications Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent

More information

8. Geometric problems

8. Geometric problems 8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set Löwner-John ellipsoid

More information

12. Interior-point methods

12. Interior-point methods 12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity

More information

Convex Optimization in Communications and Signal Processing

Convex Optimization in Communications and Signal Processing Convex Optimization in Communications and Signal Processing Prof. Dr.-Ing. Wolfgang Gerstacker 1 University of Erlangen-Nürnberg Institute for Digital Communications National Technical University of Ukraine,

More information

Chapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1)

Chapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1) Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33 Example of a Linear Program Remarks. minimize 2x 1 x 2 + 4x 3 subject to x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3

More information

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17 EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko

More information

Lecture 4: Linear and quadratic problems

Lecture 4: Linear and quadratic problems Lecture 4: Linear and quadratic problems linear programming examples and applications linear fractional programming quadratic optimization problems (quadratically constrained) quadratic programming second-order

More information

Convex Optimization M2

Convex Optimization M2 Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization

More information

HW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.

HW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given. HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard

More information

EE364 Review Session 4

EE364 Review Session 4 EE364 Review Session 4 EE364 Review Outline: convex optimization examples solving quasiconvex problems by bisection exercise 4.47 1 Convex optimization problems we have seen linear programming (LP) quadratic

More information

A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution:

A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: 1-5: Least-squares I A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: f (x) =(Ax b) T (Ax b) =x T A T Ax 2b T Ax + b T b f (x) = 2A T Ax 2A T b = 0 Chih-Jen Lin (National

More information

A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution:

A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: 1-5: Least-squares I A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: f (x) =(Ax b) T (Ax b) =x T A T Ax 2b T Ax + b T b f (x) = 2A T Ax 2A T b = 0 Chih-Jen Lin (National

More information

Geometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as

Geometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as Chapter 8 Geometric problems 8.1 Projection on a set The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as dist(x 0,C) = inf{ x 0 x x C}. The infimum here is always achieved.

More information

Convex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013

Convex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013 Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research

More information

5. Duality. Lagrangian

5. Duality. Lagrangian 5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

More information

I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010

I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010 I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0

More information

ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications

ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications ELE539A: Optimization of Communication Systems Lecture 6: Quadratic Programming, Geometric Programming, and Applications Professor M. Chiang Electrical Engineering Department, Princeton University February

More information

Convex Optimization. Newton s method. ENSAE: Optimisation 1/44

Convex Optimization. Newton s method. ENSAE: Optimisation 1/44 Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)

More information

3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions

3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions 3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions

More information

9. Geometric problems

9. Geometric problems 9. Geometric problems EE/AA 578, Univ of Washington, Fall 2016 projection on a set extremal volume ellipsoids centering classification 9 1 Projection on convex set projection of point x on set C defined

More information

ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications

ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March

More information

Convex Optimization in Classification Problems

Convex Optimization in Classification Problems New Trends in Optimization and Computational Algorithms December 9 13, 2001 Convex Optimization in Classification Problems Laurent El Ghaoui Department of EECS, UC Berkeley elghaoui@eecs.berkeley.edu 1

More information

Convex Optimization & Lagrange Duality

Convex Optimization & Lagrange Duality Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT

More information

Lecture: Duality.

Lecture: Duality. Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong

More information

Convex Optimization Boyd & Vandenberghe. 5. Duality

Convex Optimization Boyd & Vandenberghe. 5. Duality 5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

More information

Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization

Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Instructor: Farid Alizadeh Author: Ai Kagawa 12/12/2012

More information

Constrained Optimization and Lagrangian Duality

Constrained Optimization and Lagrangian Duality CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may

More information

Lecture Note 5: Semidefinite Programming for Stability Analysis

Lecture Note 5: Semidefinite Programming for Stability Analysis ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State

More information

EE5138R: Problem Set 5 Assigned: 16/02/15 Due: 06/03/15

EE5138R: Problem Set 5 Assigned: 16/02/15 Due: 06/03/15 EE538R: Problem Set 5 Assigned: 6/0/5 Due: 06/03/5. BV Problem 4. The feasible set is the convex hull of (0, ), (0, ), (/5, /5), (, 0) and (, 0). (a) x = (/5, /5) (b) Unbounded below (c) X opt = {(0, x

More information

CS295: Convex Optimization. Xiaohui Xie Department of Computer Science University of California, Irvine

CS295: Convex Optimization. Xiaohui Xie Department of Computer Science University of California, Irvine CS295: Convex Optimization Xiaohui Xie Department of Computer Science University of California, Irvine Course information Prerequisites: multivariate calculus and linear algebra Textbook: Convex Optimization

More information

Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization

Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The

More information

Convex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)

Convex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST) Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Definition convex function Examples

More information

subject to (x 2)(x 4) u,

subject to (x 2)(x 4) u, Exercises Basic definitions 5.1 A simple example. Consider the optimization problem with variable x R. minimize x 2 + 1 subject to (x 2)(x 4) 0, (a) Analysis of primal problem. Give the feasible set, the

More information

3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions

3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions 3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions

More information

The Q-parametrization (Youla) Lecture 13: Synthesis by Convex Optimization. Lecture 13: Synthesis by Convex Optimization. Example: Spring-mass System

The Q-parametrization (Youla) Lecture 13: Synthesis by Convex Optimization. Lecture 13: Synthesis by Convex Optimization. Example: Spring-mass System The Q-parametrization (Youla) Lecture 3: Synthesis by Convex Optimization controlled variables z Plant distubances w Example: Spring-mass system measurements y Controller control inputs u Idea for lecture

More information

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 18

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 18 EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 18 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 31, 2012 Andre Tkacenko

More information

MOSEK Modeling Cookbook Release 3.0. MOSEK ApS

MOSEK Modeling Cookbook Release 3.0. MOSEK ApS MOSEK Modeling Cookbook Release 3.0 MOSEK ApS 13 August 2018 Contents 1 Preface 1 2 Linear optimization 3 2.1 Introduction................................... 3 2.2 Linear modeling.................................

More information

minimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,

minimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1, 4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x

More information

Linear and non-linear programming

Linear and non-linear programming Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)

More information

Lecture: Duality of LP, SOCP and SDP

Lecture: Duality of LP, SOCP and SDP 1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:

More information

CS-E4830 Kernel Methods in Machine Learning

CS-E4830 Kernel Methods in Machine Learning CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This

More information

minimize x subject to (x 2)(x 4) u,

minimize x subject to (x 2)(x 4) u, Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for

More information

Lecture 10. ( x domf. Remark 1 The domain of the conjugate function is given by

Lecture 10. ( x domf. Remark 1 The domain of the conjugate function is given by 10-1 Multi-User Information Theory Jan 17, 2012 Lecture 10 Lecturer:Dr. Haim Permuter Scribe: Wasim Huleihel I. CONJUGATE FUNCTION In the previous lectures, we discussed about conve set, conve functions

More information

FRTN10 Multivariable Control, Lecture 13. Course outline. The Q-parametrization (Youla) Example: Spring-mass System

FRTN10 Multivariable Control, Lecture 13. Course outline. The Q-parametrization (Youla) Example: Spring-mass System FRTN Multivariable Control, Lecture 3 Anders Robertsson Automatic Control LTH, Lund University Course outline The Q-parametrization (Youla) L-L5 Purpose, models and loop-shaping by hand L6-L8 Limitations

More information

ICS-E4030 Kernel Methods in Machine Learning

ICS-E4030 Kernel Methods in Machine Learning ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This

More information

Nonlinear Programming Models

Nonlinear Programming Models Nonlinear Programming Models Fabio Schoen 2008 http://gol.dsi.unifi.it/users/schoen Nonlinear Programming Models p. Introduction Nonlinear Programming Models p. NLP problems minf(x) x S R n Standard form:

More information

Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications

Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications Chong-Yung Chi Institute of Communications Engineering & Department of Electrical Engineering National Tsing

More information

Analytic Center Cutting-Plane Method

Analytic Center Cutting-Plane Method Analytic Center Cutting-Plane Method S. Boyd, L. Vandenberghe, and J. Skaf April 14, 2011 Contents 1 Analytic center cutting-plane method 2 2 Computing the analytic center 3 3 Pruning constraints 5 4 Lower

More information

Chapter 2: Preliminaries and elements of convex analysis

Chapter 2: Preliminaries and elements of convex analysis Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-14-15.shtml Academic year 2014-15

More information

CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization

CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of

More information

Preliminaries Overview OPF and Extensions. Convex Optimization. Lecture 8 - Applications in Smart Grids. Instructor: Yuanzhang Xiao

Preliminaries Overview OPF and Extensions. Convex Optimization. Lecture 8 - Applications in Smart Grids. Instructor: Yuanzhang Xiao Convex Optimization Lecture 8 - Applications in Smart Grids Instructor: Yuanzhang Xiao University of Hawaii at Manoa Fall 2017 1 / 32 Today s Lecture 1 Generalized Inequalities and Semidefinite Programming

More information

Agenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)

Agenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP) Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in

More information

Conic Linear Programming. Yinyu Ye

Conic Linear Programming. Yinyu Ye Conic Linear Programming Yinyu Ye December 2004, revised October 2017 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture

More information

EE/AA 578, Univ of Washington, Fall Duality

EE/AA 578, Univ of Washington, Fall Duality 7. Duality EE/AA 578, Univ of Washington, Fall 2016 Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

More information

Lecture 9 Sequential unconstrained minimization

Lecture 9 Sequential unconstrained minimization S. Boyd EE364 Lecture 9 Sequential unconstrained minimization brief history of SUMT & IP methods logarithmic barrier function central path UMT & SUMT complexity analysis feasibility phase generalized inequalities

More information

Advances in Convex Optimization: Theory, Algorithms, and Applications

Advances in Convex Optimization: Theory, Algorithms, and Applications Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne

More information

Nonlinear Optimization: The art of modeling

Nonlinear Optimization: The art of modeling Nonlinear Optimization: The art of modeling INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org Nonlinear optimization c 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org)

More information

Convex functions. Definition. f : R n! R is convex if dom f is a convex set and. f ( x +(1 )y) < f (x)+(1 )f (y) f ( x +(1 )y) apple f (x)+(1 )f (y)

Convex functions. Definition. f : R n! R is convex if dom f is a convex set and. f ( x +(1 )y) < f (x)+(1 )f (y) f ( x +(1 )y) apple f (x)+(1 )f (y) Convex functions I basic properties and I operations that preserve convexity I quasiconvex functions I log-concave and log-convex functions IOE 611: Nonlinear Programming, Fall 2017 3. Convex functions

More information

CHAPTER 3: OPTIMIZATION

CHAPTER 3: OPTIMIZATION CHAPTER 3: OPTIMIZATION 1 Overview Optimization is the process of maximizing or minimizing an objective function, subject to constraints. Optimization is fundamental to nearly all academic fields, and

More information

EE Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko, JPL Third Term

EE Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko, JPL Third Term EE 150 - Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko JPL Third Term 2011-2012 Due on Thursday May 3 in class. Homework Set #4 1. (10 points) (Adapted

More information

What can be expressed via Conic Quadratic and Semidefinite Programming?

What can be expressed via Conic Quadratic and Semidefinite Programming? What can be expressed via Conic Quadratic and Semidefinite Programming? A. Nemirovski Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Abstract Tremendous recent

More information

Lecture 2: Convex functions

Lecture 2: Convex functions Lecture 2: Convex functions f : R n R is convex if dom f is convex and for all x, y dom f, θ [0, 1] f is concave if f is convex f(θx + (1 θ)y) θf(x) + (1 θ)f(y) x x convex concave neither x examples (on

More information

Problems in VLSI design

Problems in VLSI design Problems in VLSI design wire and transistor sizing signal delay in RC circuits transistor and wire sizing Elmore delay minimization via GP dominant time constant minimization via SDP placement problems

More information

Lecture 6: Conic Optimization September 8

Lecture 6: Conic Optimization September 8 IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions

More information

Lecture 7: Convex Optimizations

Lecture 7: Convex Optimizations Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1

More information

Lagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)

Lagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST) Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual

More information

Homework Set #6 - Solutions

Homework Set #6 - Solutions EE 15 - Applications of Convex Optimization in Signal Processing and Communications Dr Andre Tkacenko JPL Third Term 11-1 Homework Set #6 - Solutions 1 a The feasible set is the interval [ 4] The unique

More information

N. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form:

N. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form: 0.1 N. L. P. Katta G. Murty, IOE 611 Lecture slides Introductory Lecture NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP does not include everything

More information

LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009

LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009 LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix

More information

Part I Formulation of convex optimization problems

Part I Formulation of convex optimization problems Nonlinear Optimization Part I Formulation of convex optimization problems Instituto Superior Técnico Carnegie Mellon University jxavier@isr.ist.utl.pt 1 Convex sets 2 Definition (Convex set) Let V be a

More information

NONLINEAR. (Hillier & Lieberman Introduction to Operations Research, 8 th edition)

NONLINEAR. (Hillier & Lieberman Introduction to Operations Research, 8 th edition) NONLINEAR PROGRAMMING (Hillier & Lieberman Introduction to Operations Research, 8 th edition) Nonlinear Programming g Linear programming has a fundamental role in OR. In linear programming all its functions

More information

Conic Linear Programming. Yinyu Ye

Conic Linear Programming. Yinyu Ye Conic Linear Programming Yinyu Ye December 2004, revised January 2015 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture

More information

Additional Homework Problems

Additional Homework Problems Additional Homework Problems Robert M. Freund April, 2004 2004 Massachusetts Institute of Technology. 1 2 1 Exercises 1. Let IR n + denote the nonnegative orthant, namely IR + n = {x IR n x j ( ) 0,j =1,...,n}.

More information

Introduction and Math Preliminaries

Introduction and Math Preliminaries Introduction and Math Preliminaries Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Appendices A, B, and C, Chapter

More information

Convex Optimization. (EE227A: UC Berkeley) Lecture 28. Suvrit Sra. (Algebra + Optimization) 02 May, 2013

Convex Optimization. (EE227A: UC Berkeley) Lecture 28. Suvrit Sra. (Algebra + Optimization) 02 May, 2013 Convex Optimization (EE227A: UC Berkeley) Lecture 28 (Algebra + Optimization) 02 May, 2013 Suvrit Sra Admin Poster presentation on 10th May mandatory HW, Midterm, Quiz to be reweighted Project final report

More information

Course Outline. FRTN10 Multivariable Control, Lecture 13. General idea for Lectures Lecture 13 Outline. Example 1 (Doyle Stein, 1979)

Course Outline. FRTN10 Multivariable Control, Lecture 13. General idea for Lectures Lecture 13 Outline. Example 1 (Doyle Stein, 1979) Course Outline FRTN Multivariable Control, Lecture Automatic Control LTH, 6 L-L Specifications, models and loop-shaping by hand L6-L8 Limitations on achievable performance L9-L Controller optimization:

More information

Convex Optimization M2

Convex Optimization M2 Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial

More information

Convex Functions. Pontus Giselsson

Convex Functions. Pontus Giselsson Convex Functions Pontus Giselsson 1 Today s lecture lower semicontinuity, closure, convex hull convexity preserving operations precomposition with affine mapping infimal convolution image function supremum

More information