Lecture Note 5: Semidefinite Programming for Stability Analysis

Save this PDF as:
 WORD  PNG  TXT  JPG

Size: px
Start display at page:

Download "Lecture Note 5: Semidefinite Programming for Stability Analysis"

Transcription

1 ECE7850: Hybrid Systems:Theory and Applications Lecture Note 5: Semidefinite Programming for Stability Analysis Wei Zhang Assistant Professor Department of Electrical and Computer Engineering Ohio State University, Columbu, Ohio, USA Spring 2017 Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 1 / 39

2 Outline Motivation Some Linear Algebra A Tiny Bit of Optimization and Duality Linear Matrix Inequalities Semidefinite Programming Problems Some Examples Conclusion Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 2 / 39

3 Motivation Converse Lyapunov function theorems are not constructive Basic idea for Lyapunov function synthesis - Select Lyapunov function structure (e.g. quadratic, polynomial, piecewise quadratic,...) - Parameterize Lyapunov function candidates - Find values of parameters to satisfy Lyapunov conditions Many Lyapunov synthesis problems can be formulated as Semidefinite Programming (SDP) problems. SDP has broad applications in many other fields as well Motivation Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 3 / 39

4 Real Symmetric Matrices S n : set of real symmetric matrices All eigenvalues are real There exists a full set of orthogonal eigenvectors Spectral decomposition: If A S n, then A = QΛQ T, where Λ diagonal and Q is unitary. Some Linear Algebra Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 4 / 39

5 Positive Semidefinite Matrices I A S n is called positive semidefinite (p.s.d.), denoted by A 0, if x T Ax 0, x R n A S n is called positive definite (p.d.), denoted by A 0, if x T Ax > 0 for all nonzero x R n S n +: set of all p.s.d. (symmetric) matrices S n ++: set of all p.d. (symmetric) matrices p.s.d. [ or p.d. matrices ] can also be defined for non-symmetric matrices. 1 1 e.g.: 1 1 We assume p.s.d. and p.d. are symmetric (unless otherwise noted) Notation: A B (resp. A B ) means A B S n + (resp. A B S n ++) Some Linear Algebra Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 5 / 39

6 Positive Semidefinite Matrices II Other equivalent definitions for symmetric p.s.d. matrices: - All 2 n 1 principal minors of A are nonnegative - All eigs of A are nonnegative - There exists a factorization A = B T B Other equivalent definitions for p.d. matrices: - All n leading principal minors of A are positive - All eigs of A are strictly positive - There exists a factorization A = B T B with B square and nonsingular. Some Linear Algebra Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 6 / 39

7 Positive Semidefinite Matrices III Useful facts: - If T nonsingular, A 0 T T AT 0; and A 0 T T AT 0 - Inner product on R m n : < A, B > tr(a T B) A B. Some Linear Algebra Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 7 / 39

8 Positive Semidefinite Matrices IV - For A, B S n +, tr(ab) 0 - For any symmetric A S n, λ min(a) µ A µi and λ max(a) β A βi Some Linear Algebra Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 8 / 39

9 Convex Set Recall from Lecture 3. Convex Set: A set S is convex if x 1, x 2 S αx 1 + (1 α)x 2 S, α [0, 1] Convex combination of x 1,..., x k : { α 1 x 1 + α 2 x α k x k : α i 0, and i α i = 1 } Convex hull: co {S} set of all convex combinations of points in S Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 9 / 39

10 Convex Cone A set S is called a cone if λ > 0, x S λx S. Conic combination of x 1 and x 2 : x = α 1 x 1 + α 2 x 2 with α 1, α 2 0 Convex cone: 1. a cone that is convex 2. equivalently, a set that contains all the conic combinations of points in the set Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 10 / 39

11 Positive Semidefinite Cone The set of positive semidefinite matrices (i.e. S n +) is a convex cone and is referred to as the positive semidefinite (PSD) cone Recall that if A, B S n +, then tr(ab) 0. This indicates that the cone S n + is acute. Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 11 / 39

12 Operations that preserve convexity I Intersection of possibly infinite number of convex sets: - e.g.: polyhedron: - e.g.: PSD cone: Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 12 / 39

13 Operations that preserve convexity II Affine mapping f : R n R m (i.e. f(x) = Ax + b) - f(x) = {f(x) : x X} is convex whenever X R n is convex e.g.: Ellipsoid: E 1 = {x R n : (x x c) T P (x x c) 1} or equivalently E 2 = {x c + Au : u 2 1} - f 1 (Y ) = {x R n : f(x) Y } is convex whenever Y R m is convex e.g.: {Ax b} = f 1 (R n +), where R n + is nonnegative orthant Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 13 / 39

14 Convex Function I Consider a finite dimensional vector space X. Let D X be convex. Definition 1 (Convex Function). A function f : D R is called convex if f(αx 1 + (1 α)x 2 ) αf(x 1 ) + (1 α)f(x 2 ), x 1, x 2 D, α [0, 1] f : D R is called strictly convex if f(αx 1 + (1 α)x 2 ) < αf(x 1 ) + (1 α)f(x 2 ), x 1 x 2 D, α [0, 1] f : D R is called concave if f is convex Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 14 / 39

15 Convex Function II How to check a function is convex? Directly use definition First-order condition: If f is differentiable over an open set that contains D, then f is convex over D iff f(z) f(x) + f(x) T (z x), x, z D Second-order condition: Suppose f is twicely differentiable over an open set that contains D, then f is convex over D iff 2 f(x) 0, x D Many other conditions, tricks,... see [BV04]. Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 15 / 39

16 Convex Function III Examples of convex functions: In general, affine functions are both convex and concave - e.g.: f(x) = a T x + b, for x R n - e.g.: f(x) = tr(a T X) + c = m n i=1 j=1 AijXij + c, for X Rm n All norms are convex - e.g. in R n : f(x) = x p = ( n i=1 xi p) 1/p ; f(x) = x = max k x k - e.g. in R m n : f(x) = X 2 = σ max(x) Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 16 / 39

17 Convex Function IV Lemma 1 (Jensen s Inequality). If f : D R is a convex function, then for any possible choices of x 1,..., x m D and α 1,..., α m 0 with m l=1 α l = 1 we have f(α 1x α mx m) α 1f(x 1) + + α mf(x m) α k s can be viewed as the probability that x k occurs Probabilistic version of Jensen s inequality reads: f(e(x)) E(f(X)) for any random variable X taking value in D Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 17 / 39

18 Nonlinear Optimization and Duality I Nonlinear Optimization: minimize: subject to: f 0 (x) f i (x) 0, i = 1,... m h i (x) = 0, i = 1,..., q decision variable x R n, domain D, referred to as primal problem optimal value p is called a convex optimization problem if f 0,..., f m are convex and h 1,..., h q are affine typically convex optimization can be solved efficiently Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 18 / 39

19 Nonlinear Optimization and Duality II Associated Lagrangian: L : D R m R q R m q L(x, λ, ν) = f 0 (x) + λ i f i (x) + ν i h i (x) i=1 i=1 weighted sum of objective and constraints functions λ i : Lagrangian multiplier associated with f i (x) 0 ν i : Lagrangian multiplier associated with h i (x) = 0 Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 19 / 39

20 Nonlinear Optimization and Duality III Lagrange dual function: g : R m R q : R g(λ, ν) = inf L(x, λ, ν) x D { m = inf f 0 (x) + λ i f i (x) + x D i=1 g is concave, can be for some λ, ν } q ν i h i (x) Lower bound property: If λ 0 (elementwise), then g(λ, ν) p i=1 Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 20 / 39

21 Nonlinear Optimization and Duality IV Lagrange Dual Problem: { maximize : g(λ, ν) subject to: λ 0 Find the best lower bound on p using the Lagrange dual function a convex optimization problem even when the primal is nonconvex optimal value denoted d (λ, ν) is called dual feasible if λ 0 and (λ, ν) dom(g) Often simplifed by making the implicit constraint (λ, ν) dom(g) explicit Example: Linear Program and its dual minimize: c T x subject to: Ax = b x 0 { maximize: b T ν subject to: A T ν + c 0 Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 21 / 39

22 Nonlinear Optimization and Duality V Duality Theorems Weak Duality: d p - always hold (for convex and nonconvex problems) - can be used to find nontrivial lower bounds for difficult problems Strong Duality: d = p - not true in general, but typically holds for convex problems - conditions that guarantee strong duality in convex problems are called constraint qualifications - Slater s constraint qualification: Primal is strictly feasible Basic Optimization Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 22 / 39

23 Linear Matrix Inequalities I Standard form: Given symmetric matrices F 0,..., F m S n, F (x) = F 0 + x 1 F x m F m 0 is called a Linear Matrix Inequality in x = (x 1,..., x m ) T R m The function F (x) is affine in x The constraint set {x R n : F (x) 0} is nonlinear but convex Linear Matrix Inequalities Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 23 / 39

24 Linear Matrix Inequalities II Example 1 (LMI in Standard Form). Characterize the constraint set: F (x) = [ ] x1 + x 2 x x x 3 Linear Matrix Inequalities Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 24 / 39

25 Linear Matrix Inequalities III General Linear Matrix Inequalities (LMI) - Let X be a finite-dimensional real vector space. - F : X S n is an affine mapping from X to n n symmetric matrices - Then F (X) 0 is called also an LMI in variable X X - Translation to standard form: Choose a basis X 1,..., X m of X and represent X = x 1X x mx m for any X X. For a given affine mapping F : X S n, we can define ˆF : R m S n as ˆF (x) F (X) = F (0) + m x i[f (X i) F (0)] i=1 where x is the coordinate of X w.r.t. the basis X 1,..., X m. Linear Matrix Inequalities Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 25 / 39

26 Linear Matrix Inequalities IV Example 2. Find conditions on matrix P to ensure that V (x) = x T P x is a Lyapunov function for a linear system ẋ = Ax Linear Matrix Inequalities Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 26 / 39

27 Schur Complement Lemma I Lemma 2 (Schur Complement Lemma). [ ] A B Define M = B T. The following three sets of inequalities are equivalent. C M 0 { A 0 C B T A 1 B 0 { C 0 A BC 1 B T 0 Proof: The lemma follows immediately from the following identities: [ ] [ ] [ ] [ ] I 0 A B I A 1 B A 0 B T A 1 I B T = C 0 I 0 C B T A 1 B [ I BC 1 ] [ A B ] [ 0 I B T C I 0 C 1 B T I ] = [ A BC 1 B T 0 0 C ] Linear Matrix Inequalities Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 27 / 39

28 Schur Complement Lemma II The proof of Schur complement lemma also reveals more general relations between the numbers of negative, zero, positive eigenvalues of - M vs. A and C B T A 1 B - M vs. C and A BC 1 B T Schur complement lemma is a very useful result to transform nonlinear (quadratic or bilinear) matrix inequalities to linear ones. Linear Matrix Inequalities Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 28 / 39

29 Semidefinite Programming I Semidefinite Programming (SDP) Problem: Optimization problem with linear objective, and Linear Matrix Inequality and linear equality constraints: minimize: c T x subject to: F 0 + x 1 F x m F m 0 (1) Ax = b Linear equality constraint in (1) can be eliminated. So essentially SDP can be viewed as optimizing linear function subject to only LMI constraints. SDP is a particular class of convex optimization problem. Global optimal solution can be found efficiently. Optimizing nonlinear but convex cost function subject to LMI constraints is also a convex optimization that can often be solved efficiently. Semidefinite Programming Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 29 / 39

30 Semidefinite Programming II Standard forms of SDP in matrix variable: SDP Standard Prime Form: min subject to: X S n : f p(x) = C X A i X = b i, i = 1,..., m X 0 (2) SDP Dual form: max : y R m subject to: f d(y) = b T y m i=1 y ia i C (3) One can derive the dual from the prime using either standard Lagrange duality method or more specialized Fenchel duality results The dual form (3) is equivalent to (1) (after eliminating the equality constraint Ax = b in (1)) Semidefinite Programming Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 30 / 39

31 Semidefinite Programming III SDP Weak Duality: f p (X) f d (y) for any primal and dual feasible X and y SDP Strong Duality: f p (X ) = f d (y ) holds under Slater s condition: Many control and optimization problem can be formulated or translated into SDP problems Various computationally difficult optimization problems can be effectively approximated by SDP problems (SDP relaxation...) We will see some examples after introducing an important technique: S-procedure Semidefinite Programming Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 31 / 39

32 S-Procedure I Many stability/engineering problems require to certify that a given function is sign-definite over certain subset of the space Mathematically, this condition can be stated as follows: g 0 (x) 0 on {x R n g 1 (x) 0,..., g m (x) 0} (4) Given functions g 0,..., g m, we want to know whether the condition holds. Sometimes we may also want to find a g 0 satisfying this condition for given g 1,..., g m. Conservative but useful condition: PSD functions s i (x) s.t. g 0 (x) i s i (x)g i (x) 0, x R n This is the so-called Generalized S-Procedure Semidefinite Programming Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 32 / 39

33 S-Procedure II Now consider an important special case: g i (x) = x T G i x, i = 0, 1,... are quadratic functions Requirement (4) becomes: x R n, x T G 1 x 0,..., x T G k x 0 x T G 0 x 0 Sufficient condition (S-procedure): α 1,..., α m 0 with G 0 α 1 G α m G m S-Procedure is lossless if m = 1 and ˆx s.t. ˆx T G 1ˆx > 0 (constraint qualification) Semidefinite Programming Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 33 / 39

34 Some Examples I Example 3 (Eigenvalue Optimization). Given symmetric matrices A 0, A 1,..., A m. Let S(w) = A 0 + i w ia i. Find weights {w i } m i=1 to minimize λ max(s(w)) SDP Examples Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 34 / 39

35 Some Examples II Example 4 (Ellipsoid inequality). Given R S n ++, the set E = {x R n : (x x c ) T R(x x c ) < 1} is an ellipsoid with center x c. Find the point in E that is the closet to the origin. SDP Examples Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 35 / 39

36 Some Examples III Example 5 (Linear Feedback Control Gain Design). Given a linear control system ẋ = Ax + Bu with linear state feedback u = Kx. Find K to stabilize the system SDP Examples Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 36 / 39

37 Some Examples IV Example 6 (Robust Stabiltiy). Given system ẋ = Ax + u with uncertain feedback u = g(x). Suppose all we know is that the feedback law satisfies: g(x) 2 β x 2. Find Lyapunov function V (x) = x T P x to ensure exponential stability. SDP Examples Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 37 / 39

38 Concluding Remarks Basic knowledge on optimization and Lagrange duality Linear matrix inequalities impose convex constraints Semidefinite programming problem: optimize linear cost subject to LMI constraints SDP has broad applications in various engineering fields: signal processing, networking, communication, control, machine learning, big data... Further reading [BV04; Boy+94; VB00] Next Lecture: Stability analysis for hybrid systems Concluding Remarks Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 38 / 39

39 References [Boy+94] [VB00] [BV04] Stephen Boyd et al. Linear matrix inequalities in system and control theory. SIAM, Jeremy G VanAntwerp and Richard D Braatz. A tutorial on linear and bilinear matrix inequalities. In: Journal of process control 10.4 (2000). Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, Concluding Remarks Lecture 5 (ECE7850 Sp17) Wei Zhang(OSU) 39 / 39

I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010

I.3. LMI DUALITY. Didier HENRION EECI Graduate School on Control Supélec - Spring 2010 I.3. LMI DUALITY Didier HENRION henrion@laas.fr EECI Graduate School on Control Supélec - Spring 2010 Primal and dual For primal problem p = inf x g 0 (x) s.t. g i (x) 0 define Lagrangian L(x, z) = g 0

More information

Introduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research

Introduction to Machine Learning Lecture 7. Mehryar Mohri Courant Institute and Google Research Introduction to Machine Learning Lecture 7 Mehryar Mohri Courant Institute and Google Research mohri@cims.nyu.edu Convex Optimization Differentiation Definition: let f : X R N R be a differentiable function,

More information

Lagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)

Lagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST) Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual

More information

Lecture 6: Conic Optimization September 8

Lecture 6: Conic Optimization September 8 IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions

More information

14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness.

14. Duality. ˆ Upper and lower bounds. ˆ General duality. ˆ Constraint qualifications. ˆ Counterexample. ˆ Complementary slackness. CS/ECE/ISyE 524 Introduction to Optimization Spring 2016 17 14. Duality ˆ Upper and lower bounds ˆ General duality ˆ Constraint qualifications ˆ Counterexample ˆ Complementary slackness ˆ Examples ˆ Sensitivity

More information

EE 227A: Convex Optimization and Applications October 14, 2008

EE 227A: Convex Optimization and Applications October 14, 2008 EE 227A: Convex Optimization and Applications October 14, 2008 Lecture 13: SDP Duality Lecturer: Laurent El Ghaoui Reading assignment: Chapter 5 of BV. 13.1 Direct approach 13.1.1 Primal problem Consider

More information

Lecture Note 1: Background

Lecture Note 1: Background ECE5463: Introduction to Robotics Lecture Note 1: Background Prof. Wei Zhang Department of Electrical and Computer Engineering Ohio State University Columbus, Ohio, USA Spring 2018 Lecture 1 (ECE5463 Sp18)

More information

Semidefinite Programming Basics and Applications

Semidefinite Programming Basics and Applications Semidefinite Programming Basics and Applications Ray Pörn, principal lecturer Åbo Akademi University Novia University of Applied Sciences Content What is semidefinite programming (SDP)? How to represent

More information

Tutorial on Convex Optimization for Engineers Part II

Tutorial on Convex Optimization for Engineers Part II Tutorial on Convex Optimization for Engineers Part II M.Sc. Jens Steinwandt Communications Research Laboratory Ilmenau University of Technology PO Box 100565 D-98684 Ilmenau, Germany jens.steinwandt@tu-ilmenau.de

More information

4. Algebra and Duality

4. Algebra and Duality 4-1 Algebra and Duality P. Parrilo and S. Lall, CDC 2003 2003.12.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone

More information

Convex Optimization M2

Convex Optimization M2 Convex Optimization M2 Lecture 3 A. d Aspremont. Convex Optimization M2. 1/49 Duality A. d Aspremont. Convex Optimization M2. 2/49 DMs DM par email: dm.daspremont@gmail.com A. d Aspremont. Convex Optimization

More information

Lecture: Duality of LP, SOCP and SDP

Lecture: Duality of LP, SOCP and SDP 1/33 Lecture: Duality of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2017.html wenzw@pku.edu.cn Acknowledgement:

More information

5. Duality. Lagrangian

5. Duality. Lagrangian 5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

More information

Constrained Optimization and Lagrangian Duality

Constrained Optimization and Lagrangian Duality CIS 520: Machine Learning Oct 02, 2017 Constrained Optimization and Lagrangian Duality Lecturer: Shivani Agarwal Disclaimer: These notes are designed to be a supplement to the lecture. They may or may

More information

Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming

Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio

More information

Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization

Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The

More information

6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC

6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 6-1 The Positivstellensatz P. Parrilo and S. Lall, ECC 2003 2003.09.02.10 6. The Positivstellensatz Basic semialgebraic sets Semialgebraic sets Tarski-Seidenberg and quantifier elimination Feasibility

More information

Module 04 Optimization Problems KKT Conditions & Solvers

Module 04 Optimization Problems KKT Conditions & Solvers Module 04 Optimization Problems KKT Conditions & Solvers Ahmad F. Taha EE 5243: Introduction to Cyber-Physical Systems Email: ahmad.taha@utsa.edu Webpage: http://engineering.utsa.edu/ taha/index.html September

More information

Lecture: Duality.

Lecture: Duality. Lecture: Duality http://bicmr.pku.edu.cn/~wenzw/opt-2016-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/35 Lagrange dual problem weak and strong

More information

A Brief Review on Convex Optimization

A Brief Review on Convex Optimization A Brief Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one convex, two nonconvex sets): A Brief Review

More information

Convex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014

Convex Optimization. Dani Yogatama. School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA. February 12, 2014 Convex Optimization Dani Yogatama School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, USA February 12, 2014 Dani Yogatama (Carnegie Mellon University) Convex Optimization February 12,

More information

SEMIDEFINITE PROGRAM BASICS. Contents

SEMIDEFINITE PROGRAM BASICS. Contents SEMIDEFINITE PROGRAM BASICS BRIAN AXELROD Abstract. A introduction to the basics of Semidefinite programs. Contents 1. Definitions and Preliminaries 1 1.1. Linear Algebra 1 1.2. Convex Analysis (on R n

More information

ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications

ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March

More information

Semidefinite Programming Duality and Linear Time-invariant Systems

Semidefinite Programming Duality and Linear Time-invariant Systems Semidefinite Programming Duality and Linear Time-invariant Systems Venkataramanan (Ragu) Balakrishnan School of ECE, Purdue University 2 July 2004 Workshop on Linear Matrix Inequalities in Control LAAS-CNRS,

More information

Convex Optimization Boyd & Vandenberghe. 5. Duality

Convex Optimization Boyd & Vandenberghe. 5. Duality 5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

More information

Example: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma

Example: feasibility. Interpretation as formal proof. Example: linear inequalities and Farkas lemma 4-1 Algebra and Duality P. Parrilo and S. Lall 2006.06.07.01 4. Algebra and Duality Example: non-convex polynomial optimization Weak duality and duality gap The dual is not intrinsic The cone of valid

More information

Optimization for Machine Learning

Optimization for Machine Learning Optimization for Machine Learning (Problems; Algorithms - A) SUVRIT SRA Massachusetts Institute of Technology PKU Summer School on Data Science (July 2017) Course materials http://suvrit.de/teaching.html

More information

Convex Optimization. (EE227A: UC Berkeley) Lecture 28. Suvrit Sra. (Algebra + Optimization) 02 May, 2013

Convex Optimization. (EE227A: UC Berkeley) Lecture 28. Suvrit Sra. (Algebra + Optimization) 02 May, 2013 Convex Optimization (EE227A: UC Berkeley) Lecture 28 (Algebra + Optimization) 02 May, 2013 Suvrit Sra Admin Poster presentation on 10th May mandatory HW, Midterm, Quiz to be reweighted Project final report

More information

CSCI : Optimization and Control of Networks. Review on Convex Optimization

CSCI : Optimization and Control of Networks. Review on Convex Optimization CSCI7000-016: Optimization and Control of Networks Review on Convex Optimization 1 Convex set S R n is convex if x,y S, λ,µ 0, λ+µ = 1 λx+µy S geometrically: x,y S line segment through x,y S examples (one

More information

Lecture 2: Convex Sets and Functions

Lecture 2: Convex Sets and Functions Lecture 2: Convex Sets and Functions Hyang-Won Lee Dept. of Internet & Multimedia Eng. Konkuk University Lecture 2 Network Optimization, Fall 2015 1 / 22 Optimization Problems Optimization problems are

More information

Duality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities

Duality. Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Duality Lagrange dual problem weak and strong duality optimality conditions perturbation and sensitivity analysis generalized inequalities Lagrangian Consider the optimization problem in standard form

More information

Linear and non-linear programming

Linear and non-linear programming Linear and non-linear programming Benjamin Recht March 11, 2005 The Gameplan Constrained Optimization Convexity Duality Applications/Taxonomy 1 Constrained Optimization minimize f(x) subject to g j (x)

More information

Lecture: Convex Optimization Problems

Lecture: Convex Optimization Problems 1/36 Lecture: Convex Optimization Problems http://bicmr.pku.edu.cn/~wenzw/opt-2015-fall.html Acknowledgement: this slides is based on Prof. Lieven Vandenberghe s lecture notes Introduction 2/36 optimization

More information

Semidefinite Programming

Semidefinite Programming Semidefinite Programming Notes by Bernd Sturmfels for the lecture on June 26, 208, in the IMPRS Ringvorlesung Introduction to Nonlinear Algebra The transition from linear algebra to nonlinear algebra has

More information

4. Convex optimization problems

4. Convex optimization problems Convex Optimization Boyd & Vandenberghe 4. Convex optimization problems optimization problem in standard form convex optimization problems quasiconvex optimization linear optimization quadratic optimization

More information

LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009

LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009 LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix

More information

Lecture: Examples of LP, SOCP and SDP

Lecture: Examples of LP, SOCP and SDP 1/34 Lecture: Examples of LP, SOCP and SDP Zaiwen Wen Beijing International Center For Mathematical Research Peking University http://bicmr.pku.edu.cn/~wenzw/bigdata2018.html wenzw@pku.edu.cn Acknowledgement:

More information

From Convex Optimization to Linear Matrix Inequalities

From Convex Optimization to Linear Matrix Inequalities Dep. of Information Engineering University of Pisa (Italy) From Convex Optimization to Linear Matrix Inequalities eng. Sergio Grammatico grammatico.sergio@gmail.com Class of Identification of Uncertain

More information

N. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form:

N. L. P. NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP. Optimization. Models of following form: 0.1 N. L. P. Katta G. Murty, IOE 611 Lecture slides Introductory Lecture NONLINEAR PROGRAMMING (NLP) deals with optimization models with at least one nonlinear function. NLP does not include everything

More information

Real Symmetric Matrices and Semidefinite Programming

Real Symmetric Matrices and Semidefinite Programming Real Symmetric Matrices and Semidefinite Programming Tatsiana Maskalevich Abstract Symmetric real matrices attain an important property stating that all their eigenvalues are real. This gives rise to many

More information

HW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given.

HW1 solutions. 1. α Ef(x) β, where Ef(x) is the expected value of f(x), i.e., Ef(x) = n. i=1 p if(a i ). (The function f : R R is given. HW1 solutions Exercise 1 (Some sets of probability distributions.) Let x be a real-valued random variable with Prob(x = a i ) = p i, i = 1,..., n, where a 1 < a 2 < < a n. Of course p R n lies in the standard

More information

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17

EE/ACM Applications of Convex Optimization in Signal Processing and Communications Lecture 17 EE/ACM 150 - Applications of Convex Optimization in Signal Processing and Communications Lecture 17 Andre Tkacenko Signal Processing Research Group Jet Propulsion Laboratory May 29, 2012 Andre Tkacenko

More information

ICS-E4030 Kernel Methods in Machine Learning

ICS-E4030 Kernel Methods in Machine Learning ICS-E4030 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 28. September, 2016 Juho Rousu 28. September, 2016 1 / 38 Convex optimization Convex optimisation This

More information

Relaxations and Randomized Methods for Nonconvex QCQPs

Relaxations and Randomized Methods for Nonconvex QCQPs Relaxations and Randomized Methods for Nonconvex QCQPs Alexandre d Aspremont, Stephen Boyd EE392o, Stanford University Autumn, 2003 Introduction While some special classes of nonconvex problems can be

More information

Robust and Optimal Control, Spring 2015

Robust and Optimal Control, Spring 2015 Robust and Optimal Control, Spring 2015 Instructor: Prof. Masayuki Fujita (S5-303B) G. Sum of Squares (SOS) G.1 SOS Program: SOS/PSD and SDP G.2 Duality, valid ineqalities and Cone G.3 Feasibility/Optimization

More information

E5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming

E5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program

More information

The Lagrangian L : R d R m R r R is an (easier to optimize) lower bound on the original problem:

The Lagrangian L : R d R m R r R is an (easier to optimize) lower bound on the original problem: HT05: SC4 Statistical Data Mining and Machine Learning Dino Sejdinovic Department of Statistics Oxford Convex Optimization and slides based on Arthur Gretton s Advanced Topics in Machine Learning course

More information

Convex Optimization & Lagrange Duality

Convex Optimization & Lagrange Duality Convex Optimization & Lagrange Duality Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Convex optimization Optimality condition Lagrange duality KKT

More information

Lagrange duality. The Lagrangian. We consider an optimization program of the form

Lagrange duality. The Lagrangian. We consider an optimization program of the form Lagrange duality Another way to arrive at the KKT conditions, and one which gives us some insight on solving constrained optimization problems, is through the Lagrange dual. The dual is a maximization

More information

MIT Algebraic techniques and semidefinite optimization February 14, Lecture 3

MIT Algebraic techniques and semidefinite optimization February 14, Lecture 3 MI 6.97 Algebraic techniques and semidefinite optimization February 4, 6 Lecture 3 Lecturer: Pablo A. Parrilo Scribe: Pablo A. Parrilo In this lecture, we will discuss one of the most important applications

More information

MATH 829: Introduction to Data Mining and Analysis Support vector machines and kernels

MATH 829: Introduction to Data Mining and Analysis Support vector machines and kernels 1/12 MATH 829: Introduction to Data Mining and Analysis Support vector machines and kernels Dominique Guillot Departments of Mathematical Sciences University of Delaware March 14, 2016 Separating sets:

More information

Support Vector Machines

Support Vector Machines Support Vector Machines Support vector machines (SVMs) are one of the central concepts in all of machine learning. They are simply a combination of two ideas: linear classification via maximum (or optimal

More information

Introduction and Math Preliminaries

Introduction and Math Preliminaries Introduction and Math Preliminaries Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Appendices A, B, and C, Chapter

More information

Sparse Optimization Lecture: Dual Certificate in l 1 Minimization

Sparse Optimization Lecture: Dual Certificate in l 1 Minimization Sparse Optimization Lecture: Dual Certificate in l 1 Minimization Instructor: Wotao Yin July 2013 Note scriber: Zheng Sun Those who complete this lecture will know what is a dual certificate for l 1 minimization

More information

Optimality, Duality, Complementarity for Constrained Optimization

Optimality, Duality, Complementarity for Constrained Optimization Optimality, Duality, Complementarity for Constrained Optimization Stephen Wright University of Wisconsin-Madison May 2014 Wright (UW-Madison) Optimality, Duality, Complementarity May 2014 1 / 41 Linear

More information

CSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming

CSC Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming CSC2411 - Linear Programming and Combinatorial Optimization Lecture 10: Semidefinite Programming Notes taken by Mike Jamieson March 28, 2005 Summary: In this lecture, we introduce semidefinite programming

More information

Lagrangian Duality and Convex Optimization

Lagrangian Duality and Convex Optimization Lagrangian Duality and Convex Optimization David Rosenberg New York University February 11, 2015 David Rosenberg (New York University) DS-GA 1003 February 11, 2015 1 / 24 Introduction Why Convex Optimization?

More information

Lecture 14: Optimality Conditions for Conic Problems

Lecture 14: Optimality Conditions for Conic Problems EE 227A: Conve Optimization and Applications March 6, 2012 Lecture 14: Optimality Conditions for Conic Problems Lecturer: Laurent El Ghaoui Reading assignment: 5.5 of BV. 14.1 Optimality for Conic Problems

More information

CS-E4830 Kernel Methods in Machine Learning

CS-E4830 Kernel Methods in Machine Learning CS-E4830 Kernel Methods in Machine Learning Lecture 3: Convex optimization and duality Juho Rousu 27. September, 2017 Juho Rousu 27. September, 2017 1 / 45 Convex optimization Convex optimisation This

More information

c 2000 Society for Industrial and Applied Mathematics

c 2000 Society for Industrial and Applied Mathematics SIAM J. OPIM. Vol. 10, No. 3, pp. 750 778 c 2000 Society for Industrial and Applied Mathematics CONES OF MARICES AND SUCCESSIVE CONVEX RELAXAIONS OF NONCONVEX SES MASAKAZU KOJIMA AND LEVEN UNÇEL Abstract.

More information

Geometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as

Geometric problems. Chapter Projection on a set. The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as Chapter 8 Geometric problems 8.1 Projection on a set The distance of a point x 0 R n to a closed set C R n, in the norm, is defined as dist(x 0,C) = inf{ x 0 x x C}. The infimum here is always achieved.

More information

Lecture 7: Convex Optimizations

Lecture 7: Convex Optimizations Lecture 7: Convex Optimizations Radu Balan, David Levermore March 29, 2018 Convex Sets. Convex Functions A set S R n is called a convex set if for any points x, y S the line segment [x, y] := {tx + (1

More information

Convex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013

Convex Optimization. (EE227A: UC Berkeley) Lecture 6. Suvrit Sra. (Conic optimization) 07 Feb, 2013 Convex Optimization (EE227A: UC Berkeley) Lecture 6 (Conic optimization) 07 Feb, 2013 Suvrit Sra Organizational Info Quiz coming up on 19th Feb. Project teams by 19th Feb Good if you can mix your research

More information

Convex optimization problems. Optimization problem in standard form

Convex optimization problems. Optimization problem in standard form Convex optimization problems optimization problem in standard form convex optimization problems linear optimization quadratic optimization geometric programming quasiconvex optimization generalized inequality

More information

Homework Set #6 - Solutions

Homework Set #6 - Solutions EE 15 - Applications of Convex Optimization in Signal Processing and Communications Dr Andre Tkacenko JPL Third Term 11-1 Homework Set #6 - Solutions 1 a The feasible set is the interval [ 4] The unique

More information

Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization

Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Semidefinite and Second Order Cone Programming Seminar Fall 2012 Project: Robust Optimization and its Application of Robust Portfolio Optimization Instructor: Farid Alizadeh Author: Ai Kagawa 12/12/2012

More information

A Note on Nonconvex Minimax Theorem with Separable Homogeneous Polynomials

A Note on Nonconvex Minimax Theorem with Separable Homogeneous Polynomials A Note on Nonconvex Minimax Theorem with Separable Homogeneous Polynomials G. Y. Li Communicated by Harold P. Benson Abstract The minimax theorem for a convex-concave bifunction is a fundamental theorem

More information

A ten page introduction to conic optimization

A ten page introduction to conic optimization CHAPTER 1 A ten page introduction to conic optimization This background chapter gives an introduction to conic optimization. We do not give proofs, but focus on important (for this thesis) tools and concepts.

More information

COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion

COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION   henrion COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F

More information

Introduction to Semidefinite Programming I: Basic properties a

Introduction to Semidefinite Programming I: Basic properties a Introduction to Semidefinite Programming I: Basic properties and variations on the Goemans-Williamson approximation algorithm for max-cut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite

More information

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE

LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE LECTURE 25: REVIEW/EPILOGUE LECTURE OUTLINE CONVEX ANALYSIS AND DUALITY Basic concepts of convex analysis Basic concepts of convex optimization Geometric duality framework - MC/MC Constrained optimization

More information

Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.

Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min. MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.

More information

A sensitivity result for quadratic semidefinite programs with an application to a sequential quadratic semidefinite programming algorithm

A sensitivity result for quadratic semidefinite programs with an application to a sequential quadratic semidefinite programming algorithm Volume 31, N. 1, pp. 205 218, 2012 Copyright 2012 SBMAC ISSN 0101-8205 / ISSN 1807-0302 (Online) www.scielo.br/cam A sensitivity result for quadratic semidefinite programs with an application to a sequential

More information

Summer School: Semidefinite Optimization

Summer School: Semidefinite Optimization Summer School: Semidefinite Optimization Christine Bachoc Université Bordeaux I, IMB Research Training Group Experimental and Constructive Algebra Haus Karrenberg, Sept. 3 - Sept. 7, 2012 Duality Theory

More information

subject to (x 2)(x 4) u,

subject to (x 2)(x 4) u, Exercises Basic definitions 5.1 A simple example. Consider the optimization problem with variable x R. minimize x 2 + 1 subject to (x 2)(x 4) 0, (a) Analysis of primal problem. Give the feasible set, the

More information

arzelier

arzelier COURSE ON LMI OPTIMIZATION WITH APPLICATIONS IN CONTROL PART II.1 LMIs IN SYSTEMS CONTROL STATE-SPACE METHODS STABILITY ANALYSIS Didier HENRION www.laas.fr/ henrion henrion@laas.fr Denis ARZELIER www.laas.fr/

More information

Chapter 2: Preliminaries and elements of convex analysis

Chapter 2: Preliminaries and elements of convex analysis Chapter 2: Preliminaries and elements of convex analysis Edoardo Amaldi DEIB Politecnico di Milano edoardo.amaldi@polimi.it Website: http://home.deib.polimi.it/amaldi/opt-14-15.shtml Academic year 2014-15

More information

Duality. Geoff Gordon & Ryan Tibshirani Optimization /

Duality. Geoff Gordon & Ryan Tibshirani Optimization / Duality Geoff Gordon & Ryan Tibshirani Optimization 10-725 / 36-725 1 Duality in linear programs Suppose we want to find lower bound on the optimal value in our convex problem, B min x C f(x) E.g., consider

More information

ORF 523 Lecture 9 Spring 2016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, March 10, 2016

ORF 523 Lecture 9 Spring 2016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, March 10, 2016 ORF 523 Lecture 9 Spring 2016, Princeton University Instructor: A.A. Ahmadi Scribe: G. Hall Thursday, March 10, 2016 When in doubt on the accuracy of these notes, please cross check with the instructor

More information

3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions

3. Convex functions. basic properties and examples. operations that preserve convexity. the conjugate function. quasiconvex functions 3. Convex functions Convex Optimization Boyd & Vandenberghe basic properties and examples operations that preserve convexity the conjugate function quasiconvex functions log-concave and log-convex functions

More information

Convex Optimization in Communications and Signal Processing

Convex Optimization in Communications and Signal Processing Convex Optimization in Communications and Signal Processing Prof. Dr.-Ing. Wolfgang Gerstacker 1 University of Erlangen-Nürnberg Institute for Digital Communications National Technical University of Ukraine,

More information

Characterizing Robust Solution Sets of Convex Programs under Data Uncertainty

Characterizing Robust Solution Sets of Convex Programs under Data Uncertainty Characterizing Robust Solution Sets of Convex Programs under Data Uncertainty V. Jeyakumar, G. M. Lee and G. Li Communicated by Sándor Zoltán Németh Abstract This paper deals with convex optimization problems

More information

Semidefinite Programming

Semidefinite Programming Semidefinite Programming Basics and SOS Fernando Mário de Oliveira Filho Campos do Jordão, 2 November 23 Available at: www.ime.usp.br/~fmario under talks Conic programming V is a real vector space h, i

More information

Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS

Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution

More information

Trust Region Problems with Linear Inequality Constraints: Exact SDP Relaxation, Global Optimality and Robust Optimization

Trust Region Problems with Linear Inequality Constraints: Exact SDP Relaxation, Global Optimality and Robust Optimization Trust Region Problems with Linear Inequality Constraints: Exact SDP Relaxation, Global Optimality and Robust Optimization V. Jeyakumar and G. Y. Li Revised Version: September 11, 2013 Abstract The trust-region

More information

Two hours. To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER. xx xxxx 2017 xx:xx xx.

Two hours. To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER. xx xxxx 2017 xx:xx xx. Two hours To be provided by Examinations Office: Mathematical Formula Tables. THE UNIVERSITY OF MANCHESTER CONVEX OPTIMIZATION - SOLUTIONS xx xxxx 27 xx:xx xx.xx Answer THREE of the FOUR questions. If

More information

Motivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem:

Motivation. Lecture 2 Topics from Optimization and Duality. network utility maximization (NUM) problem: CDS270 Maryam Fazel Lecture 2 Topics from Optimization and Duality Motivation network utility maximization (NUM) problem: consider a network with S sources (users), each sending one flow at rate x s, through

More information

Solving large Semidefinite Programs - Part 1 and 2

Solving large Semidefinite Programs - Part 1 and 2 Solving large Semidefinite Programs - Part 1 and 2 Franz Rendl http://www.math.uni-klu.ac.at Alpen-Adria-Universität Klagenfurt Austria F. Rendl, Singapore workshop 2006 p.1/34 Overview Limits of Interior

More information

Convex Optimization M2

Convex Optimization M2 Convex Optimization M2 Lecture 8 A. d Aspremont. Convex Optimization M2. 1/57 Applications A. d Aspremont. Convex Optimization M2. 2/57 Outline Geometrical problems Approximation problems Combinatorial

More information

Lecture Notes on Support Vector Machine

Lecture Notes on Support Vector Machine Lecture Notes on Support Vector Machine Feng Li fli@sdu.edu.cn Shandong University, China 1 Hyperplane and Margin In a n-dimensional space, a hyper plane is defined by ω T x + b = 0 (1) where ω R n is

More information

Convex Optimization. Lecture 12 - Equality Constrained Optimization. Instructor: Yuanzhang Xiao. Fall University of Hawaii at Manoa

Convex Optimization. Lecture 12 - Equality Constrained Optimization. Instructor: Yuanzhang Xiao. Fall University of Hawaii at Manoa Convex Optimization Lecture 12 - Equality Constrained Optimization Instructor: Yuanzhang Xiao University of Hawaii at Manoa Fall 2017 1 / 19 Today s Lecture 1 Basic Concepts 2 for Equality Constrained

More information

FRTN10 Multivariable Control, Lecture 13. Course outline. The Q-parametrization (Youla) Example: Spring-mass System

FRTN10 Multivariable Control, Lecture 13. Course outline. The Q-parametrization (Youla) Example: Spring-mass System FRTN Multivariable Control, Lecture 3 Anders Robertsson Automatic Control LTH, Lund University Course outline The Q-parametrization (Youla) L-L5 Purpose, models and loop-shaping by hand L6-L8 Limitations

More information

Introduction to Nonlinear Stochastic Programming

Introduction to Nonlinear Stochastic Programming School of Mathematics T H E U N I V E R S I T Y O H F R G E D I N B U Introduction to Nonlinear Stochastic Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio SPS

More information

Largest dual ellipsoids inscribed in dual cones

Largest dual ellipsoids inscribed in dual cones Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that

More information