Convex Optimization Problems. Prof. Daniel P. Palomar
|
|
- Cora Payne
- 5 years ago
- Views:
Transcription
1 Conve Optimization Problems Prof. Daniel P. Palomar The Hong Kong University of Science and Technology (HKUST) MAFS6010R- Portfolio Optimization with R MSc in Financial Mathematics Fall , HKUST, Hong Kong
2 Outline 1 Optimization Problems 2 Conve Optimization 3 Quasi-Conve Optimization 4 Classes of Conve Problems: LP, QP, SOCP, SDP 5 Multicriterion Optimization (Pareto Optimality) 6 Solvers
3 Outline 1 Optimization Problems 2 Conve Optimization 3 Quasi-Conve Optimization 4 Classes of Conve Problems: LP, QP, SOCP, SDP 5 Multicriterion Optimization (Pareto Optimality) 6 Solvers
4 Optimization Problem in Standard Form General optimization problem in standard form: f 0 () subject to f i () 0 i = 1,..., m h i () = 0 i = 1,..., p where = ( 1,..., n ) is the optimization variable f 0 : R n R is the objective function f i : R n R, i = 1,..., m are inequality constraint functions h i : R n R, i = 1,..., p are equality constraint functions. Goal: find an optimal solution that s f 0 while satisfying all the constraints. D. Palomar Conve Problems 4 / 48
5 Optimization Problem in Standard Form Feasibility: a point dom f 0 is feasible if it satisfies all the constraints and infeasible otherwise a problem is feasible if it has at least one feasible point and infeasible otherwise. Optimal value: p = inf {f 0 () f i () 0, i = 1,..., m, h i () = 0, i = 1,..., p} p = if problem infeasible (no satisfies the constraints) p = if problem unbounded below. Optimal solution: such that f ( ) = p (and feasible). D. Palomar Conve Problems 5 / 48
6 Global and Local Optimality A feasible is optimal if f 0 () = p ; X opt is the set of optimal points. A feasible is locally optimal if is optimal within a ball Eamples: f 0 () = 1/, dom f 0 = R ++ : p = 0, no optimal point f 0 () = 3 3: p =, local optimum at = 1. D. Palomar Conve Problems 6 / 48
7 Implicit Constraints The standard form optimization problem has an eplicit constraint: m D = dom f i i=0 p dom f i i=1 D is the domain of the problem The constraints f i () 0, h i () = 0 are the eplicit constraints A problem is unconstrained if it has no eplicit constraints Eample: log ( b a T ) is an unconstrained problem with implicit constraint b > a T. D. Palomar Conve Problems 7 / 48
8 Feasibility Problem Sometimes, we don t really want to any objective, just to find a feasible point: find subject to f i () 0 i = 1,..., m h i () = 0 i = 1,..., p This feasibility problem can be considered as a special case of a general problem: 0 subject to f i () 0 i = 1,..., m h i () = 0 i = 1,..., p where p = 0 if constraints are feasible and p = otherwise. D. Palomar Conve Problems 8 / 48
9 Outline 1 Optimization Problems 2 Conve Optimization 3 Quasi-Conve Optimization 4 Classes of Conve Problems: LP, QP, SOCP, SDP 5 Multicriterion Optimization (Pareto Optimality) 6 Solvers
10 Conve Optimization Problem Conve optimization problem in standard form: f 0 () subject to f i () 0 i = 1,..., m A = b where f 0, f 1,..., f m are conve and equality constraints are affine. Local and global optima: any locally optimal point of a conve problem is globally optimal. Most problems are not conve when formulated. Reformulating a problem in conve form is an art, there is no systematic way. D. Palomar Conve Problems 10 / 48
11 Eample The following problem is nonconve (why not?): subject to 1 / ( ) 0 ( ) 2 = 0 The objective is conve. The equality constraint function is not affine; however, we can rewrite it as 1 = 2 which is then a linear equality constraint. The inequality constraint function is not conve; however, we can rewrite it as 1 0 which again is linear. We can rewrite it as subject to = 2 D. Palomar Conve Problems 11 / 48
12 Equivalent Reformulations Eliminating/introducing equality constraints: is equivalent to f 0 () subject to f i () 0 i = 1,..., m A = b f 0 (Fz + 0 ) z subject to f i (Fz + 0 ) 0 i = 1,..., m where F and 0 are such that A = b = Fz + 0 for some z. D. Palomar Conve Problems 12 / 48
13 Equivalent Reformulations Introducing slack variables for linear inequalities: is equivalent to f 0 () subject to ai T b i i = 1,..., m,s f 0 () subject to ai T + s i = b i i = 1,..., m s i 0 D. Palomar Conve Problems 13 / 48
14 Equivalent Reformulations Epigraph form: a standard form conve problem is equivalent to t,t subject to f 0 () t 0 f i () 0 A = b Minimizing over some variables: is equivalent to where f 0 () = inf y f 0 (, y). i = 1,..., m,y f 0 (, y) subject to f i () 0 i = 1,..., m f 0 () subject to f i () 0 i = 1,..., m D. Palomar Conve Problems 14 / 48
15 Outline 1 Optimization Problems 2 Conve Optimization 3 Quasi-Conve Optimization 4 Classes of Conve Problems: LP, QP, SOCP, SDP 5 Multicriterion Optimization (Pareto Optimality) 6 Solvers
16 Quasiconve Optimization Quasi-conve optimization problem in standard form: f 0 () subject to f i () 0 i = 1,..., m A = b where f 0 : R n R is quasiconve and f 1,..., f m are conve. Observe that it can have locally optimal points that are not (globally) optimal: D. Palomar Conve Problems 16 / 48
17 Quasiconve Optimization Conve representation of sublevel sets of a quasiconve function f 0 : there eists a family of conve functions φ t () for fied t such that Eample: f 0 () t φ t () 0. f 0 () = p () q () with p conve, q concave, and p () 0, q () > 0 on dom f 0. We can choose: φ t () = p () tq () for t 0, φ t () is conve in p () /q () t if and only if φ t () 0. D. Palomar Conve Problems 17 / 48
18 Quasiconve Optimization Solving a quasiconve problem via conve feasibility problems: the idea is to solve the epigraph form of the problem with a sandwich technique in t: for fied t the epigraph form of the original problem reduces to a feasibility conve problem φ t () 0, f i () 0 i, A b if t is too small, the feasibility problem will be infeasible if t is too large, the feasibility problem will be feasible start with upper and lower bounds on t (termed u and l, resp.) and use a sandwitch technique (bisection method): at each iteration use t = (l + u) /2 and update the bounds according to the feasibility/infeasibility of the problem. D. Palomar Conve Problems 18 / 48
19 Outline 1 Optimization Problems 2 Conve Optimization 3 Quasi-Conve Optimization 4 Classes of Conve Problems: LP, QP, SOCP, SDP 5 Multicriterion Optimization (Pareto Optimality) 6 Solvers
20 Linear Programming (LP) LP: subject to c T + d G h A = b Conve problem: affine objective and constraint functions. Feasible set is a polyhedron: D. Palomar Conve Problems 20 / 48
21 l 1 - and l - Norm Problems as LPs l -norm minimization: is equivalent to the LP subject to G h A = b t, subject to t t1 t1 G h A = b. D. Palomar Conve Problems 21 / 48
22 l 1 - and l - Norm Problems as LPs l 1 -norm minimization: is equivalent to the LP subject to t, subject to 1 G h A = b i t i t t G h A = b. D. Palomar Conve Problems 22 / 48
23 Linear-Fractional Programming Linear-fractional programming: ( c T + d ) / ( e T + f ) subject to G h A = b with dom f 0 = { e T + f > 0 }. It is a quasiconve optimization problem (solved by bisection). Interestingly, the following LP is equivalent: y,z subject to c T y + dz Gy hz Ay = bz e T y + fz = 1 z 0 D. Palomar Conve Problems 23 / 48
24 Quadratic Programming (QP) Quadratic programming: subject to (1/2) T P + q T + r G h A = b Conve problem (assuming P S n 0): conve quadratic objective and affine constraint functions. Minimization of a conve quadratic function over a polyhedron: D. Palomar Conve Problems 24 / 48
25 Quadratically Constrained QP (QCQP) Quadratically constrained QP: (1/2) T P 0 + q T 0 + r 0 subject to (1/2) T P i + qi T + r i 0 i = 1,..., m A = b Conve problem (assuming P i S n 0): conve quadratic objective and constraint functions. D. Palomar Conve Problems 25 / 48
26 Second-Order Cone Programming (SOCP) Second-order cone programming: f T subject to A i + b i ci T + d i i = 1,..., m F = g Conve problem: linear objective and second-order cone constraints For A i row vector, it reduces to an LP. For c i = 0, it reduces to a QCQP. More general than QCQP and LP. D. Palomar Conve Problems 26 / 48
27 Semidefinite Programming (SDP) SDP: subject to c T 1 F F n F n G A = b Inequality constraint is called linear matri inequality (LMI). Conve problem: linear objective and linear matri inequality (LMI) constraints. Observe that multiple LMI constraints can always be written as a single one. D. Palomar Conve Problems 27 / 48
28 Semidefinite Programming (SDP) subject to LP and equivalent SDP: c T A b SOCP and equivalent SDP: c T subject to diag (A b) 0 f T subject to A i + b i ci T + d i, i = 1,..., m subject to f T [ ( ) c T i + d i I Ai + b i (A i + b i ) T c T i + d i ] 0, i = 1,..., m D. Palomar Conve Problems 28 / 48
29 Semidefinite Programming (SDP) Eigenvalue minimization: λ ma (A ()) where A () = A A n A n, is equivalent to SDP subject to t A () ti It follows from λ ma (A ()) t A () ti D. Palomar Conve Problems 29 / 48
30 Outline 1 Optimization Problems 2 Conve Optimization 3 Quasi-Conve Optimization 4 Classes of Conve Problems: LP, QP, SOCP, SDP 5 Multicriterion Optimization (Pareto Optimality) 6 Solvers
31 Scalarization for Multicriterion Problems Consider a multicriterion optimization with q different objectives: f 0 () = (F 1 (),..., F q ()). To find Pareto optimal points, the positive weighted sum: λ T f 0 () = λ 1 F 1 () + + λ q F q (). Eample: regularized least-squares: A b γ 2 2. D. Palomar Conve Problems 31 / 48
32 References on Conve Problems Chapter 4 of Stephen Boyd and Lieven Vandenberghe, Conve Optimization. Cambridge, U.K.: Cambridge University Press, D. Palomar Conve Problems 32 / 48
33 Outline 1 Optimization Problems 2 Conve Optimization 3 Quasi-Conve Optimization 4 Classes of Conve Problems: LP, QP, SOCP, SDP 5 Multicriterion Optimization (Pareto Optimality) 6 Solvers
34 Solvers A solver is an engine for solving a particular type of mathematical problem, such as a conve program. Every programming language (e.g., Matlab, Octave, R, Python, C, C++) has a long list of available solvers to choose from. Solvers typically handle only a certain class of problems, such as LPs, QPs, SOCPs, SDPs, or GPs. They also require that problems be epressed in a standard form. Most problems do not immediately present themselves in a standard form, so they must be transformed into standard form. D. Palomar Conve Problems 34 / 48
35 Solver Eample: Matlab s linprog A program for solving LPs: = linprog( c, A, b, A_eq, B_eq, l, u ) Problems must be epressed in the following standard form: subject to c T A b A eq = b eq l u D. Palomar Conve Problems 35 / 48
36 Conversion to Standard Form: Common Tricks Representing free variables as the difference of nonnegative variables: free = +, + 0, 0 Elimininating inequality constraints using slack variables: a T b = a T + s = b, s 0 Splitting equality constraints into inequalities: a T = b = a T b, a T b D. Palomar Conve Problems 36 / 48
37 Solver Eample: SeDuMi A program for solving LPs, SOCPs, SDPs, and related problems: = sedumi( A, b, c, K ) Solves problems of the form: subject to c T A = b K K 1 K 2 K L where each set K i R n i, i = 1, 2,..., L is chosen from a very short list of cones. The Matlab variable K gives the number, types, and dimensions of the cones K i. D. Palomar Conve Problems 37 / 48
38 The cones must be arranged in this order, i.e., the free variables first, then the nonnegative orthants, then the second-order cones, then the semidefinite cones. D. Palomar Conve Problems 38 / 48 Solver Eample: SeDuMi Cones supported by SeDuMi: free variables: R n i a nonnegative orthant: R n i + (for linear inequalities) a real or comple second-order cone: Q n {(, y) R n R 2 y} Q n c {(, y) C n R 2 y} a real or comple semidefinite cone: { } S n + X R n n X = X T, X 0 { } H n + X C n n X = X H, X 0
39 Eample: Norm Approimation Consider the norm approimation problem: A b An optimal value s the residuals r k = a T k b k, k = 1, 2,..., m according to the measure defined by the norm Obviously, the value of depends significantly upon the choice of that norm and so does the process of conversion to standard form D. Palomar Conve Problems 39 / 48
40 Eample: Euclidean or l 2 -Norm Norm approimation problem: ( m ( ) ) A b 2 = k=1 a T 2 1/2 k b k No need to use any solver here: this is a least squares (LS) problem, with an analytic solution: = (A T A) 1 A T b In Matlab or Octave, a single command computes the solution: > > = A \ b Similarly, in R: > > = solve(a, b) D. Palomar Conve Problems 40 / 48
41 Eample: Chebyshev or l -Norm Norm approimation problem: A b = ma 1 k m a T k b k This can be epressed as a linear program:,t subject to t t1 A b t1 or, equivalently,,t subject to [ 0 T 1 ] [ ] [ ] t [ A 1 A 1 t ] [ b b ] D. Palomar Conve Problems 41 / 48
42 Eample: Chebyshev or l -Norm Recall the final formulation: [ 0 T 1 ] [ ],t [ ] t [ A 1 subject to A 1 t ] [ b b ] Matlab s linprog call: > > t = linprog( [zeros(n,1); 1],... [A,-ones(m,1); -A,-ones(m,1)],... [b; -b] ) > > = t(1:n) D. Palomar Conve Problems 42 / 48
43 Eample: Manhattan or l 1 -Norm Norm approimation problem: A b 1 = m k=1 a T k b k This can be epressed as a linear program:,t subject to 1 T t t A b t or, equivalently,,t subject to [ 0 T 1 T ] [ ] [ ] [ t ] A I A I t [ b b ] D. Palomar Conve Problems 43 / 48
44 Eample: Manhattan or l 1 -Norm Recall the final formulation: [ 0 T 1 T ] [ ],t [ ] [ t ] A I subject to A I t [ b b ] Matlab s linprog call: > > t = linprog( [zeros(n,1); ones(n,1)],... [A,-eye(m,1); -A,-eye(m,1)],... [b; -b] ) > > = t(1:n) D. Palomar Conve Problems 44 / 48
45 Eample: Constrained Euclidean or l 2 -Norm Constrained norm approimation problem: subject to A b 2 C = d l u This is not a least squares problem, but it is QP and an SOCP. This can be epressed as,y,t,s l,s u subject to t A b = y C = d s l = l + s u = u s l, s u 0 y 2 t D. Palomar Conve Problems 45 / 48
46 Eample: Constrained Euclidean or l 2 -Norm Equivalently: SeDuMi call: [ 0 T 0 T 0 T 0 T 1 ],y,t,s l,s u A I subject to C I I I I R n R n + R n + Q m > > AA = [ A, zeros(m,n), zeros(m,n), -eye(m), 0; C, zeros(p,n), zeros(p,n) zeros(p,n), 0; eye(n), -eye(n), zeros(n,n), zeros(n,n), 0; eye(n), zeros(n,n), eye(n), zeros(n,n), 0 ] > > bb = [ b; d; l; u ] > > cc = [ zeros(3*n+m,1); 1 ] > > K.f = n; K.l = 2*n; K.q = m + 1; > > syz = sedumi( AA, bb, cc, K ) > > = syz(1:n) D. Palomar Conve Problems 46 / 48 b d l u
47 Modeling Frameworks: cv A modeling framework simplifies the use of a numerical technology by shielding the user from the underlying mathematical details. Eamples: SDPSOL, YALMIP, cv, etc. cv is designed to support conve optimization or, more specifically, disciplined conve programming (available in Matlab, Python, R, Julia, Octave, etc.) People don t simply write optimization problems and hope that they are conve; instead they draw from a "mental library"of functions and sets with known conveity properties and combine them in ways that conve analysis guarantees will produce conve results. Disciplined conve programming formalizes this methodology. Links: cv: cv user guide: cv for R (cvr): D. Palomar Conve Problems 47 / 48
48 Thanks For more information visit:
4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More information4. Convex optimization problems
Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization
More informationConvex optimization problems. Optimization problem in standard form
Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality
More informationConvex Optimization. 4. Convex Optimization Problems. Prof. Ying Cui. Department of Electrical Engineering Shanghai Jiao Tong University
Conve Optimization 4. Conve Optimization Problems Prof. Ying Cui Department of Electrical Engineering Shanghai Jiao Tong University 2017 Autumn Semester SJTU Ying Cui 1 / 58 Outline Optimization problems
More informationLecture: Convex Optimization Problems
1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization
More informationLagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual
More informationTutorial on Convex Optimization for Engineers Part II
Tutorial on Convex Optimization for Engineers Part II M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de
More informationConvex Functions. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)
Convex Functions Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Definition convex function Examples
More informationConvex Optimization Overview (cnt d)
Conve Optimization Overview (cnt d) Chuong B. Do November 29, 2009 During last week s section, we began our study of conve optimization, the study of mathematical optimization problems of the form, minimize
More informationSparse Optimization Lecture: Basic Sparse Optimization Models
Sparse Optimization Lecture: Basic Sparse Optimization Models Instructor: Wotao Yin July 2013 online discussions on piazza.com Those who complete this lecture will know basic l 1, l 2,1, and nuclear-norm
More information4. Convex optimization problems (part 1: general)
EE/AA 578, Univ of Washington, Fall 2016 4. Convex optimization problems (part 1: general) optimization problem in standard form convex optimization problems quasiconvex optimization 4 1 Optimization problem
More informationLecture 7: Weak Duality
EE 227A: Conve Optimization and Applications February 7, 2012 Lecture 7: Weak Duality Lecturer: Laurent El Ghaoui 7.1 Lagrange Dual problem 7.1.1 Primal problem In this section, we consider a possibly
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationConvex Optimization and Modeling
Convex Optimization and Modeling Convex Optimization Fourth lecture, 05.05.2010 Jun.-Prof. Matthias Hein Reminder from last time Convex functions: first-order condition: f(y) f(x) + f x,y x, second-order
More informationAgenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)
Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in
More informationPrimal/Dual Decomposition Methods
Primal/Dual Decomposition Methods Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2018-19, HKUST, Hong Kong Outline of Lecture Subgradients
More informationLecture 10. ( x domf. Remark 1 The domain of the conjugate function is given by
10-1 Multi-User Information Theory Jan 17, 2012 Lecture 10 Lecturer:Dr. Haim Permuter Scribe: Wasim Huleihel I. CONJUGATE FUNCTION In the previous lectures, we discussed about conve set, conve functions
More informationDuality Uses and Correspondences. Ryan Tibshirani Convex Optimization
Duality Uses and Correspondences Ryan Tibshirani Conve Optimization 10-725 Recall that for the problem Last time: KKT conditions subject to f() h i () 0, i = 1,... m l j () = 0, j = 1,... r the KKT conditions
More informationLecture 4: September 12
10-725/36-725: Conve Optimization Fall 2016 Lecture 4: September 12 Lecturer: Ryan Tibshirani Scribes: Jay Hennig, Yifeng Tao, Sriram Vasudevan Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications
ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March
More informationEE5138R: Problem Set 5 Assigned: 16/02/15 Due: 06/03/15
EE538R: Problem Set 5 Assigned: 6/0/5 Due: 06/03/5. BV Problem 4. The feasible set is the convex hull of (0, ), (0, ), (/5, /5), (, 0) and (, 0). (a) x = (/5, /5) (b) Unbounded below (c) X opt = {(0, x
More information9. Interpretations, Lifting, SOS and Moments
9-1 Interpretations, Lifting, SOS and Moments P. Parrilo and S. Lall, CDC 2003 2003.12.07.04 9. Interpretations, Lifting, SOS and Moments Polynomial nonnegativity Sum of squares (SOS) decomposition Eample
More informationLecture: Duality.
Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong
More informationConvex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013
Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research
More informationConstrained Optimization and Lagrangian Duality
CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may
More informationLecture: Duality of LP, SOCP and SDP
1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:
More informationConvex Optimization M2
Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization
More informationConvex Optimization and l 1 -minimization
Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l
More information5. Duality. Lagrangian
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationCourse Outline. FRTN10 Multivariable Control, Lecture 13. General idea for Lectures Lecture 13 Outline. Example 1 (Doyle Stein, 1979)
Course Outline FRTN Multivariable Control, Lecture Automatic Control LTH, 6 L-L Specifications, models and loop-shaping by hand L6-L8 Limitations on achievable performance L9-L Controller optimization:
More informationSemidefinite Programming Basics and Applications
Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent
More informationCSCI : Optimization and Control of Networks. Review on Convex Optimization
CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one
More informationReview of Optimization Basics
Review of Optimization Basics. Introduction Electricity markets throughout the US are said to have a two-settlement structure. The reason for this is that the structure includes two different markets:
More informationEE Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko, JPL Third Term
EE 150 - Applications of Convex Optimization in Signal Processing and Communications Dr. Andre Tkacenko JPL Third Term 2011-2012 Due on Thursday May 3 in class. Homework Set #4 1. (10 points) (Adapted
More informationCHAPTER 2: QUADRATIC PROGRAMMING
CHAPTER 2: QUADRATIC PROGRAMMING Overview Quadratic programming (QP) problems are characterized by objective functions that are quadratic in the design variables, and linear constraints. In this sense,
More informationTutorial on Convex Optimization: Part II
Tutorial on Convex Optimization: Part II Dr. Khaled Ardah Communications Research Laboratory TU Ilmenau Dec. 18, 2018 Outline Convex Optimization Review Lagrangian Duality Applications Optimal Power Allocation
More informationA Brief Review on Convex Optimization
A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review
More information12. Interior-point methods
12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity
More informationConvex Optimization Boyd & Vandenberghe. 5. Duality
5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized
More informationConvex Optimization in Communications and Signal Processing
Convex Optimization in Communications and Signal Processing Prof. Dr.-Ing. Wolfgang Gerstacker 1 University of Erlangen-Nürnberg Institute for Digital Communications National Technical University of Ukraine,
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More information1. Introduction. mathematical optimization. least-squares and linear programming. convex optimization. example. course goals and topics
1. Introduction Convex Optimization Boyd & Vandenberghe mathematical optimization least-squares and linear programming convex optimization example course goals and topics nonlinear optimization brief history
More informationIntro to Nonlinear Optimization
Intro to Nonlinear Optimization We now rela the proportionality and additivity assumptions of LP What are the challenges of nonlinear programs NLP s? Objectives and constraints can use any function: ma
More informationDisciplined Convex Programming
Disciplined Convex Programming Stephen Boyd Michael Grant Electrical Engineering Department, Stanford University University of Pennsylvania, 3/30/07 Outline convex optimization checking convexity via convex
More informationLecture 7: Convex Optimizations
Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1
More informationConvex Optimization for Signal Processing and Communications: From Fundamentals to Applications
Convex Optimization for Signal Processing and Communications: From Fundamentals to Applications Chong-Yung Chi Institute of Communications Engineering & Department of Electrical Engineering National Tsing
More informationLecture 4: January 26
10-725/36-725: Conve Optimization Spring 2015 Lecturer: Javier Pena Lecture 4: January 26 Scribes: Vipul Singh, Shinjini Kundu, Chia-Yin Tsai Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationConvexity II: Optimization Basics
Conveity II: Optimization Basics Lecturer: Ryan Tibshirani Conve Optimization 10-725/36-725 See supplements for reviews of basic multivariate calculus basic linear algebra Last time: conve sets and functions
More informationNonlinear Optimization for Optimal Control
Nonlinear Optimization for Optimal Control Pieter Abbeel UC Berkeley EECS Many slides and figures adapted from Stephen Boyd [optional] Boyd and Vandenberghe, Convex Optimization, Chapters 9 11 [optional]
More informationLecture Note 5: Semidefinite Programming for Stability Analysis
ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State
More information1 Kernel methods & optimization
Machine Learning Class Notes 9-26-13 Prof. David Sontag 1 Kernel methods & optimization One eample of a kernel that is frequently used in practice and which allows for highly non-linear discriminant functions
More informationIntroduction to Convex Optimization
Introduction to Convex Optimization Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2018-19, HKUST, Hong Kong Outline of Lecture Optimization
More informationIntroduction to Alternating Direction Method of Multipliers
Introduction to Alternating Direction Method of Multipliers Yale Chang Machine Learning Group Meeting September 29, 2016 Yale Chang (Machine Learning Group Meeting) Introduction to Alternating Direction
More informationLecture 16: October 22
0-725/36-725: Conve Optimization Fall 208 Lecturer: Ryan Tibshirani Lecture 6: October 22 Scribes: Nic Dalmasso, Alan Mishler, Benja LeRoy Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer:
More informationI.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010
I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0
More informationOptimisation convexe: performance, complexité et applications.
Optimisation convexe: performance, complexité et applications. Introduction, convexité, dualité. A. d Aspremont. M2 OJME: Optimisation convexe. 1/128 Today Convex optimization: introduction Course organization
More informationHW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.
HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard
More informationConvexification of Mixed-Integer Quadratically Constrained Quadratic Programs
Convexification of Mixed-Integer Quadratically Constrained Quadratic Programs Laura Galli 1 Adam N. Letchford 2 Lancaster, April 2011 1 DEIS, University of Bologna, Italy 2 Department of Management Science,
More informationOperations Research Letters
Operations Research Letters 37 (2009) 1 6 Contents lists available at ScienceDirect Operations Research Letters journal homepage: www.elsevier.com/locate/orl Duality in robust optimization: Primal worst
More informationLECTURE 7. Least Squares and Variants. Optimization Models EE 127 / EE 227AT. Outline. Least Squares. Notes. Notes. Notes. Notes.
Optimization Models EE 127 / EE 227AT Laurent El Ghaoui EECS department UC Berkeley Spring 2015 Sp 15 1 / 23 LECTURE 7 Least Squares and Variants If others would but reflect on mathematical truths as deeply
More informationFrom Convex Optimization to Linear Matrix Inequalities
Dep. of Information Engineering University of Pisa (Italy) From Convex Optimization to Linear Matrix Inequalities eng. Sergio Grammatico grammatico.sergio@gmail.com Class of Identification of Uncertain
More information1. Sets A set is any collection of elements. Examples: - the set of even numbers between zero and the set of colors on the national flag.
San Francisco State University Math Review Notes Michael Bar Sets A set is any collection of elements Eamples: a A {,,4,6,8,} - the set of even numbers between zero and b B { red, white, bule} - the set
More informationThe proximal mapping
The proximal mapping http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghes lecture notes Outline 2/37 1 closed function 2 Conjugate function
More informationCS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems. Instructor: Shaddin Dughmi
CS675: Convex and Combinatorial Optimization Fall 2016 Convex Optimization Problems Instructor: Shaddin Dughmi Outline 1 Convex Optimization Basics 2 Common Classes 3 Interlude: Positive Semi-Definite
More informationChapter 2: Linear Programming Basics. (Bertsimas & Tsitsiklis, Chapter 1)
Chapter 2: Linear Programming Basics (Bertsimas & Tsitsiklis, Chapter 1) 33 Example of a Linear Program Remarks. minimize 2x 1 x 2 + 4x 3 subject to x 1 + x 2 + x 4 2 3x 2 x 3 = 5 x 3 + x 4 3 x 1 0 x 3
More informationCONSTRAINED OPTIMIZATION LARS IMSLAND
Introduction to course TTK16 Mixed integer optimization in energy and oil and gas systems CONSTRAINED OPTIMIZATION LARS IMSLAND TTK16 overview This introduction lecture One week intensive course Starting
More informationLecture: Cone programming. Approximating the Lorentz cone.
Strong relaxations for discrete optimization problems 10/05/16 Lecture: Cone programming. Approximating the Lorentz cone. Lecturer: Yuri Faenza Scribes: Igor Malinović 1 Introduction Cone programming is
More informationDuality revisited. Javier Peña Convex Optimization /36-725
Duality revisited Javier Peña Conve Optimization 10-725/36-725 1 Last time: barrier method Main idea: approimate the problem f() + I C () with the barrier problem f() + 1 t φ() tf() + φ() where t > 0 and
More information13. Convex programming
CS/ISyE/ECE 524 Introduction to Optimization Spring 2015 16 13. Convex programming Convex sets and functions Convex programs Hierarchy of complexity Example: geometric programming Laurent Lessard (www.laurentlessard.com)
More informationLecture: Examples of LP, SOCP and SDP
1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:
More informationA Tutorial on Convex Optimization
A Tutorial on Conve Optimization Haitham Hindi Palo Alto Research Center (PARC), Palo Alto, California email: hhindi@parc.com Abstract In recent years, conve optimization has become a computational tool
More information15. Conic optimization
L. Vandenberghe EE236C (Spring 216) 15. Conic optimization conic linear program examples modeling duality 15-1 Generalized (conic) inequalities Conic inequality: a constraint x K where K is a convex cone
More informationFast Algorithms for SDPs derived from the Kalman-Yakubovich-Popov Lemma
Fast Algorithms for SDPs derived from the Kalman-Yakubovich-Popov Lemma Venkataramanan (Ragu) Balakrishnan School of ECE, Purdue University 8 September 2003 European Union RTN Summer School on Multi-Agent
More informationExercises. Exercises. Basic terminology and optimality conditions. 4.2 Consider the optimization problem
Exercises Basic terminology and optimality conditions 4.1 Consider the optimization problem f 0(x 1, x 2) 2x 1 + x 2 1 x 1 + 3x 2 1 x 1 0, x 2 0. Make a sketch of the feasible set. For each of the following
More informationUsing quadratic convex reformulation to tighten the convex relaxation of a quadratic program with complementarity constraints
Noname manuscript No. (will be inserted by the editor) Using quadratic conve reformulation to tighten the conve relaation of a quadratic program with complementarity constraints Lijie Bai John E. Mitchell
More informationLecture 14: Optimality Conditions for Conic Problems
EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems
More informationCanonical Problem Forms. Ryan Tibshirani Convex Optimization
Canonical Problem Forms Ryan Tibshirani Convex Optimization 10-725 Last time: optimization basics Optimization terology (e.g., criterion, constraints, feasible points, solutions) Properties and first-order
More informationAlgorithm-Hardware Co-Optimization of Memristor-Based Framework for Solving SOCP and Homogeneous QCQP Problems
L.C.Smith College of Engineering and Computer Science Algorithm-Hardware Co-Optimization of Memristor-Based Framework for Solving SOCP and Homogeneous QCQP Problems Ao Ren Sijia Liu Ruizhe Cai Wujie Wen
More informationEconomics 205 Exercises
Economics 05 Eercises Prof. Watson, Fall 006 (Includes eaminations through Fall 003) Part 1: Basic Analysis 1. Using ε and δ, write in formal terms the meaning of lim a f() = c, where f : R R.. Write the
More informationLikelihood Bounds for Constrained Estimation with Uncertainty
Proceedings of the 44th IEEE Conference on Decision and Control, and the European Control Conference 5 Seville, Spain, December -5, 5 WeC4. Likelihood Bounds for Constrained Estimation with Uncertainty
More informationConvex Optimization. Prof. Nati Srebro. Lecture 12: Infeasible-Start Newton s Method Interior Point Methods
Convex Optimization Prof. Nati Srebro Lecture 12: Infeasible-Start Newton s Method Interior Point Methods Equality Constrained Optimization f 0 (x) s. t. A R p n, b R p Using access to: 2 nd order oracle
More informationLecture 6: Conic Optimization September 8
IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions
More informationDual Methods. Lecturer: Ryan Tibshirani Convex Optimization /36-725
Dual Methods Lecturer: Ryan Tibshirani Conve Optimization 10-725/36-725 1 Last time: proimal Newton method Consider the problem min g() + h() where g, h are conve, g is twice differentiable, and h is simple.
More informationConvex Optimization Overview
Convex Optimization Overview Zico Kolter October 19, 2007 1 Introduction Many situations arise in machine learning where we would like to optimize the value of some function. That is, given a function
More informationConvex Optimization M2
Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial
More informationPrinciples of Optimization Math 364
Principles of Optimization Math 364 Tom Asaki February 26, 2019 Topics List In this course we will cover the following list of topics and other topics as time allows. 1. Optimization Concepts and Notation
More informationCITY UNIVERSITY OF HONG KONG
CITY UNIVERSITY OF HONG KONG Topics in Optimization: Solving Second-Order Conic Systems with Finite Precision; Calculus of Generalized Subdifferentials for Nonsmooth Functions Submitted to Department of
More informationModule 04 Optimization Problems KKT Conditions & Solvers
Module 04 Optimization Problems KKT Conditions & Solvers Ahmad F. Taha EE 5243: Introduction to Cyber-Physical Systems Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ taha/index.html September
More informationGETTING STARTED INITIALIZATION
GETTING STARTED INITIALIZATION 1. Introduction Linear programs come in many different forms. Traditionally, one develops the theory for a few special formats. These formats are equivalent to one another
More informationEE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17
EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko
More informationConve functions 2{2 Conve functions f : R n! R is conve if dom f is conve and y 2 dom f 2 [0 1] + f ( + (1 ; )y) f () + (1 ; )f (y) (1) f is concave i
ELEC-E5420 Conve functions 2{1 Lecture 2 Conve functions conve functions, epigraph simple eamples, elementary properties more eamples, more properties Jensen's inequality quasiconve, quasiconcave functions
More informationModern Optimal Control
Modern Optimal Control Matthew M. Peet Arizona State University Lecture 19: Stabilization via LMIs Optimization Optimization can be posed in functional form: min x F objective function : inequality constraints
More informationc 2005 Society for Industrial and Applied Mathematics
SIAM J. MATRIX ANAL. APPL. Vol. 27, No. 2, pp. 532 546 c 2005 Society for Industrial and Applied Mathematics LEAST-SQUARES COVARIANCE MATRIX ADJUSTMENT STEPHEN BOYD AND LIN XIAO Abstract. We consider the
More informationInterior Point Methods: Second-Order Cone Programming and Semidefinite Programming
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationEE364 Review Session 4
EE364 Review Session 4 EE364 Review Outline: convex optimization examples solving quasiconvex problems by bisection exercise 4.47 1 Convex optimization problems we have seen linear programming (LP) quadratic
More informationsubject to (x 2)(x 4) u,
Exercises Basic definitions 5.1 A simple example. Consider the optimization problem with variable x R. minimize x 2 + 1 subject to (x 2)(x 4) 0, (a) Analysis of primal problem. Give the feasible set, the
More informationThe Q-parametrization (Youla) Lecture 13: Synthesis by Convex Optimization. Lecture 13: Synthesis by Convex Optimization. Example: Spring-mass System
The Q-parametrization (Youla) Lecture 3: Synthesis by Convex Optimization controlled variables z Plant distubances w Example: Spring-mass system measurements y Controller control inputs u Idea for lecture
More informationConic Linear Programming. Yinyu Ye
Conic Linear Programming Yinyu Ye December 2004, revised October 2017 i ii Preface This monograph is developed for MS&E 314, Conic Linear Programming, which I am teaching at Stanford. Information, lecture
More informationCourse Notes for EE227C (Spring 2018): Convex Optimization and Approximation
Course otes for EE7C (Spring 018): Conve Optimization and Approimation Instructor: Moritz Hardt Email: hardt+ee7c@berkeley.edu Graduate Instructor: Ma Simchowitz Email: msimchow+ee7c@berkeley.edu October
More informationInterior Point Methods. We ll discuss linear programming first, followed by three nonlinear problems. Algorithms for Linear Programming Problems
AMSC 607 / CMSC 764 Advanced Numerical Optimization Fall 2008 UNIT 3: Constrained Optimization PART 4: Introduction to Interior Point Methods Dianne P. O Leary c 2008 Interior Point Methods We ll discuss
More information