Convex Optimization. 4. Convex Optimization Problems. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University
|
|
- Sheryl Phillips
- 5 years ago
- Views:
Transcription
1 Conve Optimization 4. Conve Optimization Problems Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2017 Autumn Semester SJTU Ying Cui 1 / 58
2 Outline Optimization problems Conve optimization Linear optimization problems Quadratic optimization problems Geometric programg Semidefinite prograg Vector optimization SJTU Ying Cui 2 / 58
3 Optimization problem in standard form f 0 () s.t. f i () 0, i = 1,,m optimization variable: R n objective function: f 0 : R n R h i () = 0, i = 1,,p inequality constraint functions: f i : R n R, i = 1,,m equality constraint functions: h i : R n R, i = 1,,p domain: D = m i=0 domf i p i=0 domh i feasible point : D and satisfies all constraints feasible set or constraint set X: set of all feasible points feasible problem: problem with nonempty feasible set SJTU Ying Cui 3 / 58
4 Implicit constraints the standard form optimization problem has an implicit constraint m p D = domf i domh i i=0 i=1 eplicit constraints: f i () 0, i = 1,,m, h i () = 0,i = 1,,p a problem is unconstrained if it has no eplicit constraints (m = p = 0) eample: f 0 () = k log(b i ai T ) is an unconstrained problem with implicit constraints a T i < b i i=1 SJTU Ying Cui 4 / 58
5 Optimal and locally optimal points optimal value: p = inf{f 0 () f i () 0, i = 1,,m, h i () = 0,i = 1,,p} p = if problem is infeasible p = if problem is unbounded below (globally) optimal point : is feasible and f 0 () = p optimal set X opt : set of optimal points if Xopt is nonempty, the optimal value is achieved; otherwise not achieved (always occurs when problem unbounded below) locally optimal point : R > 0 such that is optimal for f 0 (z) z s.t. f i (z) 0, i = 1,,m h i (z) = 0, i = 1,,p z 2 R SJTU Ying Cui 5 / 58
6 Optimal and locally optimal points eamples (with n = 1, m = p = 0) f 0 () = 1/, domf 0 = R ++ : p = 0, no optimal point f 0 () = log, domf 0 = R ++ : p = f 0 () = log, domf 0 = R ++ : p = 1/e, unique optimal point = 1/e f 0 () = 3 3, p =, locally optimal point = 1 SJTU Ying Cui 6 / 58
7 Feasibility problems The feasibility problem is to detere whether the constraints are consistent, and if so, find a point that satisfies them, i.e., find s.t. f i () 0, i = 1,...,m h i () = 0, i = 1,...,p It can be considered a special case of the general problem with f 0 () = 0, i.e., 0 s.t. f i () 0, i = 1,...,m h i () = 0, i = 1,...,p p = 0 if constraints are feasible; any feasible is optimal p = if constraints are infeasible SJTU Ying Cui 7 / 58
8 Conve optimization problems in standard form s.t. f 0 () f i () 0, i = 1,...,m a T i = b i, i = 1,...,p (or A = b) objective function f 0 and inequality constraint functions f 1,...,f m are conve; equality constraints are affine problem is quasiconve if f0 is quasiconve and f 1,...,f m conve feasible set of a conve optimization problem is conve intersection of domain D = m i=0 domf i with m sub level sets { f i () 0} and p hyperplanes { a T i = b i } (all conve) SJTU Ying Cui 8 / 58
9 Abstract form conve optimization problem eample f 0 () = s.t. f 1 () = 1 /(1+2) 2 0 h 1 () = ( ) 2 = 0 imize a conve function over a conve set f0 is conve, feasible set {( 1, 2 ) 1 = 2 0} is conve not a conve optimization problem in standard form (according to our definition) f1 is not conve, h 1 is not affine equivalent (but not identical) to the conve problem s.t = 0 SJTU Ying Cui 9 / 58
10 Local and global optima any locally optimal point of a conve problem is (globally) optimal proof: Suppose is locally optimal, i.e., is feasible and f 0 () (a) = inf{f 0 (z) z feasible, z 2 R} for some R > 0. Suppose is not globally optimal, i.e., there eists a feasible y such that f 0 (y) < f 0 (). Evidently y 2 > R. Consider z = θy +(1 θ) with θ = Then, we have z 2 = R/2 < R, and by conveity of the feasible set, z is feasible. By conveity of f 0, we have which contradicts (a). f 0 (z) θf 0 (y)+(1 θ)f 0 () < f 0 () R 2 y 2. SJTU Ying Cui 10 / 58
11 Optimality criterion for differentiable f 0 is optimal iff it is feasible and f 0 () T (y ) 0 for all y X X f0() Figure 4.2 Geometric interpretation of the optimality condition (4.21). The feasible set X is shown shaded. Some level curves of f0 are shown as dashed lines. The point is optimal: f0() defines a supporting hyperplane (shown as a solid line) to X at. geometric interpretation: if f 0 () 0, f 0 () defines a supporting hyperplane to feasible set X at SJTU Ying Cui 11 / 58
12 Optimality criterion for differentiable f 0 unconstrained problem: f 0 () is optimal iff domf 0, f 0 () = 0 equality constrained problem: f 0 () s.t. A = b is optimal iff there eists a v such that domf 0, A = b, f 0 ()+Av = 0 imization over nonnegative orthant: f 0 () s.t. 0 is optimal iff domf 0, 0, f 0 () 0, ( f 0 ()) i i = 0 condition ( f0 ()) i i = 0 is called complementarity: the sparsity patterns (i.e., the set of indices corresponding to nonzero components) of the vectors and f 0 () are complementary (i.e., have empty intersection) ( { f0 () i 0 i =0 f 0 () i =0 i >0 SJTU Ying Cui 12 / 58 )
13 Equivalent conve problems two problems are (informally) equivalent if the solution of one is readily obtained from the solution of the other, and vice-versa some common transformations that preserve conveity SJTU Ying Cui 13 / 58
14 Equivalent conve problems eliating equality constraints s.t. f 0 () f i () 0, i = 1,...,m A = b is equivalent to z f 0 (Fz + 0 ) s.t. f i (Fz + 0 ) 0, i = 1,...,m where F and 0 are such that A = b = Fz + 0 for some z in principle, we can restrict our attention to conve optimization problems without equality constraints in many cases, however, it is better to retain equality constraints, for ease of analysis, or not to ruin efficiency of an algorithm that solves it SJTU Ying Cui 14 / 58
15 Equivalent conve problems introducing equality constraints is equivalent to f 0 (A 0 +b 0 ) s.t. f i (A i +b i ) 0, i = 1,...,m,y s.t. f 0 (y 0 ) f i (y i ) 0, i = 1,...,m y i = A i +b i, i = 0,...,m SJTU Ying Cui 15 / 58
16 Equivalent conve problems introducing slack variables for linear inequalities s.t. f 0 () a T i b i, i = 1,...,m is equivalent to,s s.t. f 0 () a T i +s i = b i, i = 1,...,m s i 0, i = 1,...,m s i is called the slack variable associated with a T i b i each inequality constraint is replaced with an equality constraint and a nonnegativity constraint SJTU Ying Cui 16 / 58
17 Equivalent conve problems epigraph problem form s.t. f 0 () f i () 0, i = 1,...,m A = b is equivalent to,t t s.t. f 0 () t 0 f i () 0, i = 1,...,m A = b an optimization problem in the graph space (, t): imize t over the epigraph of f 0, subject to the constraints on. a linear objective is universal for conve optimization can simplify theoretical analysis and algorithm development SJTU Ying Cui 17 / 58
18 Equivalent conve problems imizing over some variables is equivalent to f 0 ( 1, 2 ) 1, 2 s.t. f i ( 1 ) 0, i = 1,...,m 1 f i ( 2 ) 0, i = 1,...,m 2 1 f0 ( 1 ) s.t. f i ( 1 ) 0, i = 1,...,m 1 where f 0 ( 1 ) = inf{f 0 ( 1,z) f i (z) 0, i = 1,...,m 2 } SJTU Ying Cui 18 / 58
19 Quasiconve optimization p = s.t. f 0 () f i () 0, i = 1,...,m A = b where f 0 : R n R is quasiconve, and f 1,...,f m are conve locally optimal solutions and optimality conditions Quasiconve optimization problem can have locally optimal points that are not (globally) optimal (,f()) Figure 4.3 A quasiconve function f on R, with a locally optimal point that is not globally optimal. This eample shows that the simple optimality condition f () = 0, valid for conve functions, does not hold for quasiconve functions. SJTU Ying Cui 19 / 58
20 Quasiconve optimization conve representation of sublevel sets of f 0 if f 0 is quasiconve, there eists a family of functions φ t such that: φ t () is conve in for fied t t-sublevel set of f 0 is 0-sublevel set of φ t,i.e., f 0 () t φ t () 0 for each, φ t () is nondecreasing in t eample (conve over concave function) f 0 () = p() q() with p conve, q concave, and p() 0,q() > 0 on domf 0 can take φ t () = p() tq() : for t 0,φ t conve in p()/q() t if and only if φ t () 0 for each, φ t () is nondecreasing in t SJTU Ying Cui 20 / 58
21 Quasiconve optimization quasiconve optimization via conve feasibility problems conve feasibility problem in for any fied t find φ t () 0, f i () 0, i = 1,...,m A = b if feasible, t p ; if infeasible, t p Bisection method for quasiconve optimization ( log 2 ((u l)/ǫ) iterations) given l p,u p, tolerance ǫ > 0. repeat 1. t := (l +u)/2. 2. Solve the conve feasibility problem (1). 3. if feasible, u := t; else l := t. until u l ǫ SJTU Ying Cui 21 / 58
22 Linear program (LP) c T +d s.t. G h A = b where c R n, d R, G R m n, h R m, A R p n and b R p objective and constraint functions are all affine linear programs are, of course, conve optimization problems omit d, i.e., linear objective function (not affect X opt or X) feasible set is a polyhedron c P Figure 4.4 Geometric interpretation of an LP. The feasible set P, which is a polyhedron, is shaded. The objective c T is linear, so its level curves are hyperplanes orthogonal to c (shown as dashed lines). The point is optimal; it is the point in P as far as possible in the direction c. SJTU Ying Cui 22 / 58
23 Linear program (LP) Two special cases of LP: Standard form LPs c T s.t. A = b 0 only inequalities are component-wise nonnegativity constraints Inequality form LPs no equality constraints c T s.t. A b SJTU Ying Cui 23 / 58
24 Linear program (LP) converting LPs to standard form sometimes useful to transform a general LP to a standard form LP (e.g., in order to use an algorithm for standard form LPs) introduce slack variables s i,i = 1,,m for inequalities:,s c T +d s.t. G +s = h A = b s 0 epress + and, i.e., = +, +, 0: +,,s c T + c T +d s.t. G + G +s = h A + A = b + 0, 0, s 0 SJTU Ying Cui 24 / 58
25 LP-eamples diet problem: choose quantities 1,, n of n foods one unit of food j costs c j, contains amount a ij of nutrient i healthy diet requires nutrient i in quantity at least b i to find cheapest healthy diet: piecewise-linear imization: c T s.t. A b, 0 ma i=1,,m (at i +b i ) equivalent to an LP by first forg the epigraph problem and then epressing the inequality as a set of m separate inequalities,t t s.t. a T i +b i t, i = 1,,m SJTU Ying Cui 25 / 58
26 LP-eamples Chebyshev center of a polyhedron: find the center c of the largest Euclidean ball B = { c +u u 2 r} that lies in a polyhedron P = { a T i b i,i = 1,,m} cheb Figure 8.5 Chebyshev center of a polyhedron C, in the Euclidean norm. The center cheb is the deepest point inside C, in the sense that it is farthest from the eterior, or complement, of C. The center cheb is also the center of the largest Euclidean ball (shown lightly shaded) that lies inside C. a T i b i for all B iff sup{a T i ( c +u) u 2 r} = a T i c +sup{a T i u u 2 r} = a T i c +r a i 2 b i Chebyshev center of polyhedron P can be detered by solving the LP c,r r s.t. a T i c +r a i 2 b i, i = 1,,m SJTU Ying Cui 26 / 58
27 Linear-fractional program imize a ratio of affine functions over a polyhedron: f 0 () = ct +d e T +f s.t. G h A = b (domf 0 () = { e T +f > 0}) f 0 () is quasiconve (quasilinear) so linear-fractional programs are quasiconve problems and can be solved by bisection method Transforg to an LP: let y = and z = 1 e T +f e T +f y,z c T y +dz s.t. Gy hz Ay = bz e T y +fz = 1 z 0 SJTU Ying Cui 27 / 58
28 Linear-fractional program generalized linear-fractional program ci T +d i f 0 () = ma i=1,,r ei T, domf 0 () = { ei T +f i > 0,i = 1,,r} +f i f 0 () is the pointwise maimum of r quasiconve functions, and therefore quasiconve, so this problem is quasiconve and can be solved by bisection method eample Von Neumann growth problem: allocate activity to maimize growth rate of slowest growing sector ma, + ) i=1,,n + i / i s.t. + 0, B + A, + R n : activity levels of n sectors, in current, net period (A) i, (B + ) i : produced, consumed amounts of good i + i / i : growth rate of sector i SJTU Ying Cui 28 / 58
29 Quadratic program (QP) (1/2) T P +q T +r s.t. G h A = b where P S n +, q Rn, r R, G R m n and A R p n objective function is conve quadratic and constraints are affine imize a conve quadratic function over a polyhedron QPs include LPs as a special case, by taking P = 0 f0( ) P Figure 4.5 Geometric illustration of QP. The feasible set P, which is a polyhedron, SJTU is shown shaded. The contour Ying lines Cuiof the objective function, which 29 / 58 is conve quadratic, are shown as dashed curves. The point is optimal.
30 QP-eamples least-squares: unconstrained QP (A R k n ) A b 2 2 = T A T A 2b T A +b T b analytical solution = A b singular value decomposition of A: A = UΣV T pseudo-inverse of A: A = VΣ 1 U T R n k constrained least-squares: add linear constraints, e.g., upper and lower bounds on A b 2 2 s.t. l u SJTU Ying Cui 30 / 58
31 QP-eamples linear program with random cost: imize risk-sensitive cost (a linear combination of epected cost and cost variance), captureing a trade-off between small epected cost and small cost variance c T +γ T Σ = Ec T +γvar(c T ) s.t. G h, A = b c R n is random vector with mean c and covariance E(c c)(c c) T ) = Σ for given, c T R is random variable with mean c T and variance E(c T Ec T ) 2 = T Σ risk aversion parameter γ > 0: control the trade-off between epected cost and cost variance (risk) SJTU Ying Cui 31 / 58
32 Quadratically constrained quadratic program (QCQP) (1/2) T P 0 +q T 0 +r 0 s.t. (1/2) T P i +q T i +r i 0, i = 1,,m A = b where P i S n +,i = 1,,m objective and inequality constraint functions are conve quadratic if P 1,,P m S n ++, feasible region is intersection of m ellipsoids and an affine set QCQPs include LPs as a special case, by taking P i = 0,i = 1,,m SJTU Ying Cui 32 / 58
33 Second-order cone programg (SOCP) f T s.t. A i +b i 2 ci T +d i, i = 1,,m F = g where f,c i R n, A i R n i n, d i R, F R p n and g R p each inequality constraint is a second-order cone (SOC) constraint: (A i +b i,c T i +d i ) second-order cone in R n i+1 more general than QCQP (and of course LP) if ci = 0, SOCP reduces to QCQP, by squaring each inequality constraint if Ai = 0, SOCP reduces to LP SJTU Ying Cui 33 / 58
34 SOCP-eample Robust linear programg: when parameters c,a i,b i c T s.t. ai T b i, i = 1,,m can be uncertain, two common approaches to handling uncertainty (in a i, for simplicity) deteristic model: a T i b i must hold for all a i E i c T s.t. ai T b i,for all a i E i i = 1,,m stochastic model: a i is random variable, and each constraint must hold with probability η (chance constraint) c T s.t. prob(ai T b i ) η i = 1,,m SJTU Ying Cui 34 / 58
35 SOCP-eample deteristic approach via SOCP choose an ellipsoid E i with center ā i R n, semi-aes detered by singular values/vectors of P i R n n : E i = {ā i +P i u u 2 1} a T i b i for all a i E i iff sup u 2 1(ā i +P i u) T = ā T i +sup u 2 1(P i u) T = ā T i + P T i 2 < b i robust LP c T s.t. ai T b i for all a i E i, i = 1,,m is equivalent to the SOCP c T s.t. āi T + Pi T 2 b i, i = 1,,m SJTU Ying Cui 35 / 58
36 SOCP-eample stochastic approach via SOCP Assume a i R n is Gaussian with mean ā i and covariance Σ i, i.e., a i N(ā i,σ i ) ai T is Gaussian r.v. with mean āi T and variance T Σ i : ( ) prob(ai T b i āi T b i ) = Φ Σ 1/2 i 2 where Φ() = (1/ 2π) ep t2 /2 dt is CDF of N(0,1) robust LP c T s.t. prob(a T i b i ) η, i = 1,,m with η 1/2, is equivalent to the SOCP c T s.t. ā T i +Φ 1 (η) Σ 1/2 i 2 b i i = 1,,m SJTU Ying Cui 36 / 58
37 Geometric programg (GP) monomial function: f : R n R with c > 0 and a i R f() = c a 1 1 a 2 2 an n, dom f = R n ++ monomials are closed under multiplication and division posynomial function (sum of monomials): f : R n R f() = K k=1 c k a 1k 1 a 2k 2 a nk n, dom f = R n ++ posynomials are closed under addition, multiplication, and nonnegative scaling a posynomial multiplied by a monomial is a posynomial a posynomial divided by a monomial is a posynomial SJTU Ying Cui 37 / 58
38 Geometric programg (GP) geometric program (GP) f 0 () s.t. f i () 1, i = 1,,m h i () = 1, i = 1,,p with f i posynomial and h i monomial, implying domain D = R n ++ (i.e., implicit constraint 0) etensions of GP: f posinomial and h (h i ) monomial f() h() f()/h() 1 (f/h posinomial) f() a (a > 0) f()/a 1 (f/a posinomial) h 1 () = h 2 () h 1 ()/h 2 () 1 (h 1 /h 2 monomial) maimize a nonzero monomial objective function, by imizing its inverse (which is also a monomial) SJTU Ying Cui 38 / 58
39 Geometric program in conve form GPs are not (in general) but can be transformed to conve problems by changing variables to y i = log i and taking log monomial f() = c a 1 1 a 2 2 an n transforms to logf(e y 1,,e yn ) = a T y +b (b = logc) posynomial f() = K k=1 c k a 1k 1 a 2k 2 a nk n transforms to K logf(e y 1,,e yn ) = log( e at k y+b k ) (b k = logc k ) k=1 geometric program transforms to conve problem log( s.t. log( K ep(a0k T y +b 0k)) k=1 K ep(aik T y +b ik)) 0, i = 1,,m k=1 Gy +d = 0 SJTU Ying Cui 39 / 58
40 GP-eamples design of cantilever beam: N segments with unit lengths and rectangular cross-sections of width w i and height h i, and given vertical force F applied at the right end, causing the beam to deflect (downward) and inducing stress in each segment segment 4 segment 3 segment 2 segment 1 Figure 4.6 Segmented cantilever beam with 4 segments. Each segment has unit length and a rectangular profile. A vertical force F is applied at the right end of the beam. F imize total weight subject to upper & lower bounds on w i,h i upper bound & lower bounds on aspect ratios h i /w i upper bound on stress in each segment upper bound on vertical deflection at the end of the beam variables: w i,h i for i = 1,,N SJTU Ying Cui 40 / 58
41 GP-eamples objective and constraint functions total weight w 1 h 1 + +w N h N is a posynomial aspect ratio h i /w i and inverse aspect ratio w i /h i are monomials maimum stress in segment i given by 6iF/(w i hi 2) is a monomial vertical deflection y i and slope v i of central ais at the right end of segment i are defined recursively as F v i = 12(i 1/2) Ew i hi 3 +v i+1 F y i = 6(i 1/3) Ew i hi 3 +v i+1 +y i+1 for i = N,N 1,...,1 with v N+1 = y N+1 = 0 (E: Young s modulus) v i and y i can be shown to be posynomial functions of w,h by induction SJTU Ying Cui 41 / 58
42 GP-eamples formulation as a GP w,h s.t. w 1 h w N h N wmaw 1 i 1, w w 1 i 1, i = 1,...,N hmah 1 i 1, h h 1 i 1, i = 1,...,N Smaw 1 1 i h i 1, S w i h 1 i 1, i = 1,...,N 6iFσmaw 1 1 i h 2 i 1, i = 1,...,N ymay write w w i w ma and h h i h ma as w /w i 1, w i /w ma 1, h /h i 1, h i /h ma 1 write S h i /w i S ma as S w i /h i 1, h i /(w i S ma ) 1 SJTU Ying Cui 42 / 58
43 GP-eamples Minimizing spectral radius of nonnegative matri Perron-Frobenius eigenvalue: λ pf (A) suppose matri A R n n is (elementwise) positive and irreducible (i.e., matri (I +A) n 1 is elementwise positive) Perron-Frobenius theorem: A has a positive real eigenvalue λ pf (A) equal to its spectral radius, i.e., largest magnitude of its eigenvalues ma i λ i (A) λ pf (A) deteres asymptotic growth (decay) rate of A k (i.e., A k λ k pf ) as k and (1/λ pf)a) k converges as k alternative characterization: λ pf (A) = inf{λ Av λv for some v 0} Av λv can be epressed as n A() ij v j /(λv i ) 1, i = 1,...,n j=1 SJTU Ying Cui 43 / 58
44 GP-eamples imizing spectral radius of matri of posynomials: suppose entries of matri A are posynomial functions of some underlying variable R k imize λ pf (A()) possibly subject to posynomial inequalities on equivalent geometric program: λ,v, s.t. λ n A() ij v j /(λv i ) 1, i = 1,...,n j=1 SJTU Ying Cui 44 / 58
45 Generalized inequality constraints conve problem with generalized inequality constraints f 0 () s.t. f i () Ki 0, i = 1,...,m A = b f 0 : R n R conve; f i : R n R k i K i -conve w.r.t proper cone K i R k i results for ordinary conve optimization problems still hold the feasible set, any sublevel set, and the optimal set are conve any point that is locally optimal for the problem (4.48) is globally optimal. optimality condition for differentiable f0 : is optimal iff it is feasible and f 0 () T (y ) 0 for all y X SJTU Ying Cui 45 / 58
46 Conic form problems (cone programs) simplest conve optimization problems with generalized inequalities with a linear objective and one affine inequality constraint function c T s.t. F +g K 0 A = b generalize LPs (reduce to LPs when K = R m +) conic form problem in standard form c T s.t. A = b K 0 conic form problem in inequality form c T s.t. F +g K 0 SJTU Ying Cui 46 / 58
47 Semidefinite prograg (SDP) c T s.t. 1 F F n F n +G 0 A = b with G,F i S k and A R p n (note that K is S k +) inequality constraint is a linear matri inequality (LMI) G,F i are all diagonal, then the LMI is equivalent to a set of n linear inequalities, and the SDP reduces to an LP SJTU Ying Cui 47 / 58
48 Semidefinite prograg (SDP) common to refer to a problem with linear objective, linear equality and inequality constraints, and several LMI constraints, i.e., c T s.t. F (i)() = 1 F (i) F (i) n F (i) n +G (i) 0, i = 1,,K G h, A = b as an SDP, as it is readily transformed to an SDP by forg a large block diagonal LMI c T s.t. tr(g h,f (1) (),,F (K) ()) 0 A = b SJTU Ying Cui 48 / 58
49 Semidefinite prograg (SDP) standard form SDP with C,A i S n inequality form SDP with B,A i S k tr(cx) X s.t. tr(a i X) = b i, i = 1,,p X 0 c T s.t. 1 A A n A n B A = b SJTU Ying Cui 49 / 58
50 LP and SOCP as SDP LP and equivalent SDP LP: c T s.t. A b SDP : c T s.t. diag(a b) 0 (note different interpretation of generalized inequality ) SOCP and equivalent SDP SOCP : f T s.t. A i +b i 2 ci T +d i, i = 1,...,m SDP : s.t. f T [ ] (c T i +d i )I A i +b i (A i +b i ) T ci T 0, i = 1,...,m +d i SJTU Ying Cui 50 / 58
51 SDP-eamples eigenvalue imization R n λ ma (A()) where A() = A A n A n (with given A i S k ) equivalent SDP: t R n,t R s.t. A() ti constraint follows from λ ma (A) t A ti SJTU Ying Cui 51 / 58
52 SDP-eamples matri norm imization A() 2 = (λ ma (A() T A())) 1/2 R n where A() = A A n A n (with given A i R p q ) equivalent SDP: variables R n,t R constraint follows from t R n,t R [ ] ti A() s.t. A() T 0 ti A 2 t A T A t 2 I, t 0 [ ] ti A A T 0 ti SJTU Ying Cui 52 / 58
53 Vector optimization general vector optimization problem (w.r.t.k) f 0 () s.t. f i () 0, i = 1,...,m h i () = 0, i = 1,...,p with vector objective f 0 : R n R q imized w.r.t. proper cone K R q, and constraint functions f i : R n R, h i : R n R conve vector optimization problem (w.r.t.k) f 0 () s.t. f i () 0, i = 1,...,m A = b with f 0 K-conve, f i conve and h i affine SJTU Ying Cui 53 / 58
54 Optimal and Pareto optimal points set of achievable objective values O = {f 0 () feasible} R q f 0 () is the imum value of O (f 0 () K f 0 (y), y O): is optimal and f0 () is the optimal value most vector optimization problems do not have an optimal point (value), but this does occur in some special cases f 0 () is a imal value of O (if there eists y O such that f 0 (y) K f 0 (), then f 0 (y) = f 0 ()) is Pareto optimal and f0 () is a Pareto optimal value a vector optimization problem can have many Pareto optimal points (values) O O f0( po ) f0( ) Figure 4.7 The set O of achievable values for a vector optimization with objective values in R 2, with cone K = R 2 +, is shown shaded. In this case, the point labeled f0( ) is the optimal value of the problem, and is an optimal point. The objective value f0( ) can be compared to every other achievable value f0(y), and is better than or equal to f0(y). (Here, better than or equal to means is below and to the left of.) The lightly shaded region is f0( )+K, which is the set of all z R 2 corresponding to objective values worse than (or equal to) f0( ). Figure 4.8 The set O of achievable values for a vector optimization problem with objective values in R 2, with cone K = R 2 +, is shown shaded. This problem does not have an optimal point or value, but it does have a set of Pareto optimal points, whose corresponding values are shown as the darkened curve on the lower left boundary of O. The point labeled f0( po ) is a Pareto optimal value, and po is a Pareto optimal point. The lightly shaded region is f0( po ) K, which is the set of all z R 2 corresponding to objective values better than (or equal to) f0( po ). SJTU Ying Cui 54 / 58
55 Multicriterion optimization vector optimization problem with K = R q + f 0 () = (F 1 (),...,F q ()) q different scalar objectives F i ; roughly speaking we want all F i s to be small feasible is optimal if f 0 ( ) f 0 (y) for all feasible y if there eists an optimal point, the objectives are noncompeting feasible po is Pareto optimal if that there eists feasible y such that f 0 (y) f 0 ( po ) implies f 0 ( po ) = f 0 (y) if there are multiple Pareto optimal values, there is a trade-off between the objectives SJTU Ying Cui 55 / 58
56 Multicriterion optimization-eamples regularized least-squares (w.r.t.r 2 +) ( A b 2 2, 2 2) 15 F2() = F1() = A b 2 2 Figure 4.11 Optimal trade-off curve for a regularized least-squares problem. The shaded set is the set of achievable values ( A b 2 2, 2 2). The optimal trade-off curve, shown darker, is the lower left part of the boundary. eample for A R ; heavy line is formed by Pareto optimal points SJTU Ying Cui 56 / 58
57 Multicriterion optimization-eamples risk return trade-off in portfolio optimization (w.r.t.r 2 +) ( p T, T ) s.t. 1 T = 1, 0 R n is investment portfolio; i is fraction invested in asset i p R n is vector of relative asset price changes; modeled as a random variable with mean p and covariance p T = Er is epected return; T = var r is return var. 15% mean return 10% 5% 0% 1 0% 10% 20% (4) (3) (2) allocation 0.5 (1) 0 0% 10% 20% standard deviation of return Figure 4.12 Top. Optimal risk-return trade-off curve for a simple portfolio optimization problem. The lefthand endpoint corresponds to putting all resources in the risk-free asset, and so has zero standard deviation. The righthand endpoint corresponds to putting all resources in asset 1, which has highest mean return. Bottom. Corresponding optimal allocations. SJTU Ying Cui 57 / 58
58 Scalarization general vector optimization problems: find Pareto optimal points by choosing λ K 0 and solving scalar problem λ T f 0 () s.t. f i () 0, i = 1,...,m h i () = 0, i = 1,...,p optimal point for the scalar problem is Pareto optimal for vector optimization problem conve vector optimization problems: can find (almost) all Pareto optimal points by varying λ K 0 O f0(1) λ1 f0(3) f0(2) λ2 Figure 4.9 Scalarization. The set O of achievable values for a vector optimization problem with cone K = R 2 +. Three Pareto optimal values f0(1), f0(2), f0(3) are shown. The first two values can be obtained by scalarization: f0(1) imizes λ T 1 u over all u O and f0(2) imizes λ T 2 u, where λ1, λ2 0. The value f0(3) is Pareto optimal, but cannot be found by scalarization. SJTU Ying Cui 58 / 58
Convex optimization problems. Optimization problem in standard form
Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Convex Optimization Fourth lecture, 05.05.2010 Jun.-Prof. Matthias Hein Reminder from last time Convex functions: first-order condition: f(y) f(x) + f x,y x, second-order
More informationConvex Optimization Problems. Prof. Daniel P. Palomar
Conve Optimization Problems Prof. Daniel P. Palomar The Hong Kong University of Science and Technology (HKUST) MAFS6010R- Portfolio Optimization with R MSc in Financial Mathematics Fall 2018-19, HKUST,
More information4. Convex optimization problems (part 1: general)
EE/AA 578, Univ of Washington, Fall 2016 4. Convex optimization problems (part 1: general) optimization problem in standard form convex optimization problems quasiconvex optimization 4 1 Optimization problem
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More informationLecture 10. ( x domf. Remark 1 The domain of the conjugate function is given by
10-1 Multi-User Information Theory Jan 17, 2012 Lecture 10 Lecturer:Dr. Haim Permuter Scribe: Wasim Huleihel I. CONJUGATE FUNCTION In the previous lectures, we discussed about conve set, conve functions
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More informationLecture 7: Weak Duality
EE 227A: Conve Optimization and Applications February 7, 2012 Lecture 7: Weak Duality Lecturer: Laurent El Ghaoui 7.1 Lagrange Dual problem 7.1.1 Primal problem In this section, we consider a possibly
More informationConvexity II: Optimization Basics
Conveity II: Optimization Basics Lecturer: Ryan Tibshirani Conve Optimization 10-725/36-725 See supplements for reviews of basic multivariate calculus basic linear algebra Last time: conve sets and functions
More informationGeometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as
Chapter 8 Geometric problems 8.1 Projection on a set The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as dist(x 0,C) = inf{ x 0 x x C}. The infimum here is always achieved.
More informationLecture 4: September 12
10-725/36-725: Conve Optimization Fall 2016 Lecture 4: September 12 Lecturer: Ryan Tibshirani Scribes: Jay Hennig, Yifeng Tao, Sriram Vasudevan Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationConvex Optimization and l 1 -minimization
Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationCS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems Instructor: Shaddin Dughmi Outline 1 Convex Optimization Basics 2 Common Classes 3 Interlude: Positive Semi-Definite
More informationTutorial on Convex Optimization for Engineers Part II
Tutorial on Convex Optimization for Engineers Part II M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationOperations Research Letters
Operations Research Letters 37 (2009) 1 6 Contents lists available at ScienceDirect Operations Research Letters journal homepage: www.elsevier.com/locate/orl Duality in robust optimization: Primal worst
More informationConve functions 2{2 Conve functions f : R n! R is conve if dom f is conve and y 2 dom f 2 [0 1] + f ( + (1 ; )y) f () + (1 ; )f (y) (1) f is concave i
ELEC-E5420 Conve functions 2{1 Lecture 2 Conve functions conve functions, epigraph simple eamples, elementary properties more eamples, more properties Jensen's inequality quasiconve, quasiconcave functions
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationOptimisation convexe: performance, complexité et applications.
Optimisation convexe: performance, complexité et applications. Introduction, convexité, dualité. A. d Aspremont. M2 OJME: Optimisation convexe. 1/128 Today Convex optimization: introduction Course organization
More informationExercises. Exercises. Basic terminology and optimality conditions. 4.2 Consider the optimization problem
Exercises Basic terminology and optimality conditions 4.1 Consider the optimization problem f 0(x 1, x 2) 2x 1 + x 2 1 x 1 + 3x 2 1 x 1 0, x 2 0. Make a sketch of the feasible set. For each of the following
More informationMidterm 1 Solutions. 1. (2 points) Show that the Frobenius norm of a matrix A depends only on its singular values. Precisely, show that
EE127A L. El Ghaoui YOUR NAME HERE: SOLUTIONS YOUR SID HERE: 42 3/27/9 Midterm 1 Solutions The eam is open notes, but access to the Internet is not allowed. The maimum grade is 2. When asked to prove something,
More informationLECTURE 7. Least Squares and Variants. Optimization Models EE 127 / EE 227AT. Outline. Least Squares. Notes. Notes. Notes. Notes.
Optimization Models EE 127 / EE 227AT Laurent El Ghaoui EECS department UC Berkeley Spring 2015 Sp 15 1 / 23 LECTURE 7 Least Squares and Variants If others would but reflect on mathematical truths as deeply
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history of convex optimization 1 1 Mathematical
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More informationIntroduction to Alternating Direction Method of Multipliers
Introduction to Alternating Direction Method of Multipliers Yale Chang Machine Learning Group Meeting September 29, 2016 Yale Chang (Machine Learning Group Meeting) Introduction to Alternating Direction
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationSemidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization
Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Instructor: Farid Alizadeh Author: Ai Kagawa 12/12/2012
More informationSparse Optimization Lecture: Basic Sparse Optimization Models
Sparse Optimization Lecture: Basic Sparse Optimization Models Instructor: Wotao Yin July 2013 online discussions on piazza.com Those who complete this lecture will know basic l 1, l 2,1, and nuclear-norm
More informationLecture 4: January 26
10-725/36-725: Conve Optimization Spring 2015 Lecturer: Javier Pena Lecture 4: January 26 Scribes: Vipul Singh, Shinjini Kundu, Chia-Yin Tsai Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More information8. Geometric problems
8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set Löwner-John ellipsoid
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationRobust-to-Dynamics Linear Programming
Robust-to-Dynamics Linear Programg Amir Ali Ahmad and Oktay Günlük Abstract We consider a class of robust optimization problems that we call robust-to-dynamics optimization (RDO) The input to an RDO problem
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More informationChapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1)
Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33 Example of a Linear Program Remarks. minimize 2x 1 x 2 + 4x 3 subject to x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3
More information9. Geometric problems
9. Geometric problems EE/AA 578, Univ of Washington, Fall 2016 projection on a set extremal volume ellipsoids centering classification 9 1 Projection on convex set projection of point x on set C defined
More informationEE5138R: Problem Set 5 Assigned: 16/02/15 Due: 06/03/15
EE538R: Problem Set 5 Assigned: 6/0/5 Due: 06/03/5. BV Problem 4. The feasible set is the convex hull of (0, ), (0, ), (/5, /5), (, 0) and (, 0). (a) x = (/5, /5) (b) Unbounded below (c) X opt = {(0, x
More informationApplications of Robust Optimization in Signal Processing: Beamforming and Power Control Fall 2012
Applications of Robust Optimization in Signal Processing: Beamforg and Power Control Fall 2012 Instructor: Farid Alizadeh Scribe: Shunqiao Sun 12/09/2012 1 Overview In this presentation, we study the applications
More informationConvex Optimization in Communications and Signal Processing
Convex Optimization in Communications and Signal Processing Prof. Dr.-Ing. Wolfgang Gerstacker 1 University of Erlangen-Nürnberg Institute for Digital Communications National Technical University of Ukraine,
More information1 Robust optimization
ORF 523 Lecture 16 Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Any typos should be emailed to a a a@princeton.edu. In this lecture, we give a brief introduction to robust optimization
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 18
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 18 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 31, 2012 Andre Tkacenko
More informationChapter 2: Preliminaries and elements of convex analysis
Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-14-15.shtml Academic year 2014-15
More informationDuality Uses and Correspondences. Ryan Tibshirani Convex Optimization
Duality Uses and Correspondences Ryan Tibshirani Conve Optimization 10-725 Recall that for the problem Last time: KKT conditions subject to f() h i () 0, i = 1,... m l j () = 0, j = 1,... r the KKT conditions
More informationCanonical Problem Forms. Ryan Tibshirani Convex Optimization
Canonical Problem Forms Ryan Tibshirani Convex Optimization 10-725 Last time: optimization basics Optimization terology (e.g., criterion, constraints, feasible points, solutions) Properties and first-order
More informationEE364 Review Session 4
EE364 Review Session 4 EE364 Review Outline: convex optimization examples solving quasiconvex problems by bisection exercise 4.47 1 Convex optimization problems we have seen linear programming (LP) quadratic
More informationNonlinear Optimization: The art of modeling
Nonlinear Optimization: The art of modeling INSEAD, Spring 2006 Jean-Philippe Vert Ecole des Mines de Paris Jean-Philippe.Vert@mines.org Nonlinear optimization c 2003-2006 Jean-Philippe Vert, (Jean-Philippe.Vert@mines.org)
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationConvex Optimization & Lagrange Duality
Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT
More informationLecture 4: Linear and quadratic problems
Lecture 4: Linear and quadratic problems linear programming examples and applications linear fractional programming quadratic optimization problems (quadratically constrained) quadratic programming second-order
More informationPreliminaries Overview OPF and Extensions. Convex Optimization. Lecture 8 - Applications in Smart Grids. Instructor: Yuanzhang Xiao
Convex Optimization Lecture 8 - Applications in Smart Grids Instructor: Yuanzhang Xiao University of Hawaii at Manoa Fall 2017 1 / 32 Today s Lecture 1 Generalized Inequalities and Semidefinite Programming
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationEE Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko, JPL Third Term
EE 150 - Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko JPL Third Term 2011-2012 Due on Thursday May 3 in class. Homework Set #4 1. (10 points) (Adapted
More informationA : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution:
1-5: Least-squares I A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: f (x) =(Ax b) T (Ax b) =x T A T Ax 2b T Ax + b T b f (x) = 2A T Ax 2A T b = 0 Chih-Jen Lin (National
More informationminimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,
4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x
More informationMOSEK Modeling Cookbook Release 3.0. MOSEK ApS
MOSEK Modeling Cookbook Release 3.0 MOSEK ApS 13 August 2018 Contents 1 Preface 1 2 Linear optimization 3 2.1 Introduction................................... 3 2.2 Linear modeling.................................
More informationA : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution:
1-5: Least-squares I A : k n. Usually k > n otherwise easily the minimum is zero. Analytical solution: f (x) =(Ax b) T (Ax b) =x T A T Ax 2b T Ax + b T b f (x) = 2A T Ax 2A T b = 0 Chih-Jen Lin (National
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationAgenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)
Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More information9. Interpretations, Lifting, SOS and Moments
9-1 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC 2003 2003.12.07.04 9. Interpretations, Lifting, SOS and Moments Polynomial nonnegativity Sum of squares (SOS) decomposition Eample
More informationSubgradients. subgradients. strong and weak subgradient calculus. optimality conditions via subgradients. directional derivatives
Subgradients subgradients strong and weak subgradient calculus optimality conditions via subgradients directional derivatives Prof. S. Boyd, EE364b, Stanford University Basic inequality recall basic inequality
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationUNCONSTRAINED OPTIMIZATION PAUL SCHRIMPF OCTOBER 24, 2013
PAUL SCHRIMPF OCTOBER 24, 213 UNIVERSITY OF BRITISH COLUMBIA ECONOMICS 26 Today s lecture is about unconstrained optimization. If you re following along in the syllabus, you ll notice that we ve skipped
More informationConvex Optimization in Classification Problems
New Trends in Optimization and Computational Algorithms December 9 13, 2001 Convex Optimization in Classification Problems Laurent El Ghaoui Department of EECS, UC Berkeley elghaoui@eecs.berkeley.edu 1
More information3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions
3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The
More informationConvex Optimization. 9. Unconstrained minimization. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University
Convex Optimization 9. Unconstrained minimization Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2017 Autumn Semester SJTU Ying Cui 1 / 40 Outline Unconstrained minimization
More informationA Tutorial on Convex Optimization
A Tutorial on Conve Optimization Haitham Hindi Palo Alto Research Center (PARC), Palo Alto, California email: hhindi@parc.com Abstract In recent years, conve optimization has become a computational tool
More informationEconomics 205 Exercises
Economics 05 Eercises Prof. Watson, Fall 006 (Includes eaminations through Fall 003) Part 1: Basic Analysis 1. Using ε and δ, write in formal terms the meaning of lim a f() = c, where f : R R.. Write the
More informationLecture 1: January 12
10-725/36-725: Convex Optimization Fall 2015 Lecturer: Ryan Tibshirani Lecture 1: January 12 Scribes: Seo-Jin Bang, Prabhat KC, Josue Orellana 1.1 Review We begin by going through some examples and key
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More informationShiqian Ma, MAT-258A: Numerical Optimization 1. Chapter 4. Subgradient
Shiqian Ma, MAT-258A: Numerical Optimization 1 Chapter 4 Subgradient Shiqian Ma, MAT-258A: Numerical Optimization 2 4.1. Subgradients definition subgradient calculus duality and optimality conditions Shiqian
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationConvex Optimization. Newton s method. ENSAE: Optimisation 1/44
Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)
More informationLMI Methods in Optimal and Robust Control
LMI Methods in Optimal and Robust Control Matthew M. Peet Arizona State University Lecture 02: Optimization (Convex and Otherwise) What is Optimization? An Optimization Problem has 3 parts. x F f(x) :
More informationConvex Optimization for Signal Processing and Communications: From Fundamentals to Applications
Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications Chong-Yung Chi Institute of Communications Engineering & Department of Electrical Engineering National Tsing
More informationCSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization
CSCI 1951-G Optimization Methods in Finance Part 10: Conic Optimization April 6, 2018 1 / 34 This material is covered in the textbook, Chapters 9 and 10. Some of the materials are taken from it. Some of
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More informationLinear and non-linear programming
Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)
More informationIntroduction and Math Preliminaries
Introduction and Math Preliminaries Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Appendices A, B, and C, Chapter
More informationOptimization and Optimal Control in Banach Spaces
Optimization and Optimal Control in Banach Spaces Bernhard Schmitzer October 19, 2017 1 Convex non-smooth optimization with proximal operators Remark 1.1 (Motivation). Convex optimization: easier to solve,
More informationConvex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Definition convex function Examples
More informationStatistical Geometry Processing Winter Semester 2011/2012
Statistical Geometry Processing Winter Semester 2011/2012 Linear Algebra, Function Spaces & Inverse Problems Vector and Function Spaces 3 Vectors vectors are arrows in space classically: 2 or 3 dim. Euclidian
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationLecture Semidefinite Programming and Graph Partitioning
Approximation Algorithms and Hardness of Approximation April 16, 013 Lecture 14 Lecturer: Alantha Newman Scribes: Marwa El Halabi 1 Semidefinite Programming and Graph Partitioning In previous lectures,
More informationLikelihood Bounds for Constrained Estimation with Uncertainty
Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 5 Seville, Spain, December -5, 5 WeC4. Likelihood Bounds for Constrained Estimation with Uncertainty
More informationApproximation Algorithms
Approximation Algorithms Chapter 26 Semidefinite Programming Zacharias Pitouras 1 Introduction LP place a good lower bound on OPT for NP-hard problems Are there other ways of doing this? Vector programs
More informationLecture: Introduction to LP, SDP and SOCP
Lecture: Introduction to LP, SDP and SOCP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2015.html wenzw@pku.edu.cn Acknowledgement:
More information