Optimization in. Stephen Boyd. 3rd SIAM Conf. Control & Applications. and Control Theory. System. Convex

Size: px
Start display at page:

Download "Optimization in. Stephen Boyd. 3rd SIAM Conf. Control & Applications. and Control Theory. System. Convex"

Transcription

1 Optimization in Convex and Control Theory System Stephen Boyd Engineering Department Electrical University Stanford 3rd SIAM Conf. Control & Applications 1

2 Basic idea Many problems arising in system and control theory can be cast convex optimization problems as Hence, are fundamentally tractable Recent interior-point methods can exploit problem structure to such problems very eciently solve 2

3 Convex optimization Some examples Interior-point methods Outline 3

4 minimize f0(x) Convex optimization subject to f1(x) 0;:::;f L (x)0 f i :R n!r convex can have linear equality constraints dierentiability not needed other formulations possible, e.g.: multicriterion, monotone variational inequality feasibility, 4

5 speaking,) (Roughly optimization problems are fundamentally tractable convex in theory and practice Algorithms: Classical optimization algorithms do not work Ellipsoid algorithm (Shor, Nemirovsky, Yudin 1970s) very simple, universally applicable { ecient in terms of worst-case complexity theory { { slow but robust in practice (General) interior-point methods (Nesterov, Nemirovsky 1980s) { ecient in theory and practice 5

6 Convex optimization Some examples Interior-point methods Outline 6

7 h i z,i design variables: x =[h0h1 ::: h n ] T hi 1:1H(e j0 ) (max. 10% step response overshoot) example: lter Well-known FIR design n X = H(z) function: transfer sample convex constraints: i=0 H(e j0 ) = 1 (unity DC gain) H(e j! 0) = 0 (notch at!0) j! )j0:01 for! s! jh(e 40dB atten. in stop band) (min. j! )j1:12 for 0!! b jh(e 1dB upper ripple in pass band) (max. h = n,i (linear phase constraint) i h t X = s(t) i=0 7

8 4 3 sample rate 2=n sec, FIR lter design example (M. Grant) linear phase max 1dB ripple to 0:4Hz up min 40dB atten 0:8Hz above minimize max i jh i j some solution times: = 255: 5 sec n n = 2047: 4 min t f 8H(e2jf) h(t)

9 exp j(k() T p,!t); k() =,[cos sin ] T p1 w i y i Beamforming antenna elements at positions p1;:::;p n 2R 2 omnidirectional wave incident from angle : plane to y = exp(jk() T p i ) demodulate i get X n y() = sum weighted form i=1 k() variables: x =[Re w T Im w T ] T design array weights or shading coecients) (antenna G() = jy()j antenna gain pattern 9

10 Sample convex constraints: y( t ) = 1 (target direction normalization) G(0) = 0(null in direction 0) w is real (amplitude only shading) jw i j1 (attenuation only shading) Sample convex objectives: max fg() jj, t j5 g level with 10 beamwidth) (sidelobe 2 X i jw i j 2 (noise power in y) 10

11 Discrete-time linear system, input u(t) 2 R p, output y(t) 2 R q max t;i jy i (t), y des i (t)j (peak tracking error) Input design (trajectory planning) Sample convex constraints: ju i (t)ju(limit on input amplitude) ju i (t+1),u i (t)js(limit on input slew rate) l i (t) y i (t) u i (t) (envelope bounds for output) Sample convex objective: extends to multi-plant (robust) case, Immediately control, etc. predictive 11

12 Input design example (M. Grant) rapid thermal processor 3 inputs, 8 outputs, states 8 amplitude limits on inputs slew limits on 3 outputs minimize peak tracking on 5 outputs error 12 t y i (t) u i (t)

13 H = P zw + P zu K(I, P yu K),1 P yw (static case for simplicity) exogenous inputs Linear controller design w z u P actuator inputs sensed outputs K y critical outputs linear plant P given; design linear feedback controller K closed-loop I/O relation: z = Hw, most specications, objectives convex in H, not K 13

14 Q = K(I, PyuK),1 set K = Q(I + P yu Q),1 Transform to convex problem Linear-fractional transformation: H = P zw + P zu QP yw : constraints, objectives are convex in Q! design Q via convex programming Extends to dynamic case... time and frequency domain limits on actuator eort, regulation, error tracking some robustness specications 14

15 Quadratic Lyapunov function search there a quadratic Lyapunov function V (z) = z T Pz that proves Is of dierential inclusion stability _x(t) =A(t)x(t); A(t)2Co fa1;:::;a L g? Equivalent to: is there P s.t. P > 0; A T i P + PA i < 0; i =1;:::;L? convex feasibility problem in P ; no analytic solution but a solved readily looks simple, but more powerful than many well known (multivariable circle criteria,...) methods extends to a huge variety of other problems 15

16 Other examples synthesis of Lyapunov functions, state feedback lter/controller realization system identication problems truss design VLSI transistor sizing design centering computational geometry 16

17 Convex optimization Some examples Interior-point methods Outline 17

18 Interior-point convex programming methods History: Dikin; Fiacco & McCormick's SUMT (1960s) Karmarkar's LP algorithm (1984); many more since then Nesterov & Nemirovsky's general formulation (1989) General: # iterations small, grows slowly with problem size number: 5 { 50) (typical each iteration is basically least-squares problem 18

19 c x T m minimize X + F0 to subject i = F T i 2 R nn, c are given. F Semidenite programming (SDP) Semidenite program: i=1 x i F i 0 special but wide class of nonlinear convex problems can pose most system and control theory convex problems as SDPs powerful interior-point methods for SDPs recently developed & Nemirovsky, Alizadeh, others) (Nesterov 19

20 A(x) = A0 + x1a1 + +x k A k linear program: SDP examples c T x minimize to Ax b subject as SDP: take F (x) =diag(b, Ax) matrix norm minimization: as SDP: minimize t subject to minimize ka(x)k ; 2 ti A(x) 64 T ti A(x) 20

21 primal SDP: dual SDP: Duality min T x p m = c X + F0 to subject i=1 x i F i 0 = max,trf0z d to TrF i Z = c i ; i =1;:::;m subject Z 0 Z dual feasible =) p,trf0z(easy) p = d (usually) duality gap = c T x + TrF0Z gap 0, gap = 0 at optimum 21

22 '(x; Z) = q log(gap) + log det F (x),1 + log det Z,1 third term keeps Z>0 hence, can solve SDP by minimizing smooth function ' for x, Z strictly feasible q>nis a parameter Primal-dual potential function rst term rewards decrease in gap second term keeps F (x) > 0 main properties: gap exp 1 q,n '(x; Z) ' unbounded below 22

23 theorem: '(x; Z), '(x + ;Z + ) >0 in practice, '(x; Z), '(x + ;Z + ) Primal-dual potential reduction algorithm x = x (0), Z = Z (0) strictly feasible initialization: repeat 1. nd search directions x, Z by solving least-squares problemy 2. plane search: minimize ' (x + x; Z + Z)over, 2 R 3. update: x := x + x;z:= Z + Z until duality gap. corollary: (polynomial) convergence yleast-squares problem, >0 depend on particular method 23

24 ,1 10,2 10,3 10,4 Typical example: matrix norm minimization minimize ka0 + x1a1 + +x k A k k two specic problems: 5 matrices, 55; 50 matrices, 5050 duality gap vs. iteration problem 1 problem iteration 24

25 per iteration: Cost Newton direction, a least squares problem with same computing structure as original problem (Toeplitz, etc.) Hence: of solving convex problem cost 5{50 cost of solving similar least-squares problem Hence: solve least-squares problem eciently can =) can solve convex problem eciently 25

26 Exploiting problem structure via CG Gradients: solve min x kax, bk, x 2 R m via m Conjugate of x! Ax and y! A T y evaluations roughly: can evaluate response and adjoint fast can solve least-squares problem fast =) (=) can solve convex problem fast) don't need exact solution for interior-point methods early termination) (allows preconditioning (problem specic) Examples: FIR lter: fast (N log N) convolution Input design: system state, co-state simulation Lyapunov function search: matrix (Kronecker) structure 26

27 10 5 LSSOL INTPT FIR lter design example (M. Grant) forward, adjoint operator: FFT #taps 2 #variables #constraints 10 #variables > 1000 variables, > solved in 4 constraints min, 4Mb 27 #variables Memory Time

28 10 5 LSSOL INTPT Input design example (M. Grant) forward, adjoint operator: co-state simulation state, #vbles = 3 #time steps #constr 7 #vbles > 1500 variables, > solved in 12 constraints min, 5Mb 28 #variables Memory Time

29 feasibility problem in P = P T 2 R kk 10 4 k =10;:::; # vbles 50;:::; ,1 Lyapunov function search (Vandenberghe) P > 0; A T i P + PA i < 0; i =1;:::;L two matrices, size k k LS: O(k 5:7 ); CG: O(k 3:9 ) k = 30 (# vbles 450) 12 min; CG: 30 sec LS: CG 100 k 10 cpu-time (sec)

30 can evaluate response, adjoint fast (exploiting structure) can solve least-squares problem fast (using conj grad) can solve convex problem fast (using int-pt methods) Exploiting structure in convex problems

31 Main point Many problems arising in engineering analysis and design can be as convex optimization problems cast Hence, can be eciently solved by interior-point methods exploit problem structure that 31

32 Nesterov and Nemirovsky, Interior-point polynomial algorithms convex programming, SIAM, in Boyd, El Ghaoui, Feron, Balakrishnan, Linear matrix in system and control theory, SIAM, inequalities (A few) references Vandenberghe and Boyd, Semidenite programming, isl.stanford.edu ftp 32

33 Software LMI-lab (Gahinet, Nemirovsky, Laub, Chilali) toolbox for control analysis/design Matlab LMI-tool (El Ghaoui, Delebecque, Nikoukhah) (ftp ftp.ensta.fr) Matlab SP (Vandenberghe, Boyd) with Matlab interface (ftp isl.stanford.edu) C SDPSOL (Boyd, Wu) for SDPs (ftp isl.stanford.edu) parser/solver 33

34 .. the great watershed in optimization isn't between. and nonlinearity, but convexity and linearity nonconvexity. R. Rockafellar, SIAM Review

Example 1 linear elastic structure; forces f 1 ;:::;f 100 induce deections d 1 ;:::;d f i F max i, several hundred other constraints: max load p

Example 1 linear elastic structure; forces f 1 ;:::;f 100 induce deections d 1 ;:::;d f i F max i, several hundred other constraints: max load p Convex Optimization in Electrical Engineering Stephen Boyd and Lieven Vandenberghe EE Dept., Stanford ISL EE370 Seminar, 2/9/96 1 Main idea many electrical engineering design problems can be cast as convex

More information

June Engineering Department, Stanford University. System Analysis and Synthesis. Linear Matrix Inequalities. Stephen Boyd (E.

June Engineering Department, Stanford University. System Analysis and Synthesis. Linear Matrix Inequalities. Stephen Boyd (E. Stephen Boyd (E. Feron :::) System Analysis and Synthesis Control Linear Matrix Inequalities via Engineering Department, Stanford University Electrical June 1993 ACC, 1 linear matrix inequalities (LMIs)

More information

Determinant maximization with linear. S. Boyd, L. Vandenberghe, S.-P. Wu. Information Systems Laboratory. Stanford University

Determinant maximization with linear. S. Boyd, L. Vandenberghe, S.-P. Wu. Information Systems Laboratory. Stanford University Determinant maximization with linear matrix inequality constraints S. Boyd, L. Vandenberghe, S.-P. Wu Information Systems Laboratory Stanford University SCCM Seminar 5 February 1996 1 MAXDET problem denition

More information

Course Outline. FRTN10 Multivariable Control, Lecture 13. General idea for Lectures Lecture 13 Outline. Example 1 (Doyle Stein, 1979)

Course Outline. FRTN10 Multivariable Control, Lecture 13. General idea for Lectures Lecture 13 Outline. Example 1 (Doyle Stein, 1979) Course Outline FRTN Multivariable Control, Lecture Automatic Control LTH, 6 L-L Specifications, models and loop-shaping by hand L6-L8 Limitations on achievable performance L9-L Controller optimization:

More information

The Q-parametrization (Youla) Lecture 13: Synthesis by Convex Optimization. Lecture 13: Synthesis by Convex Optimization. Example: Spring-mass System

The Q-parametrization (Youla) Lecture 13: Synthesis by Convex Optimization. Lecture 13: Synthesis by Convex Optimization. Example: Spring-mass System The Q-parametrization (Youla) Lecture 3: Synthesis by Convex Optimization controlled variables z Plant distubances w Example: Spring-mass system measurements y Controller control inputs u Idea for lecture

More information

FRTN10 Multivariable Control, Lecture 13. Course outline. The Q-parametrization (Youla) Example: Spring-mass System

FRTN10 Multivariable Control, Lecture 13. Course outline. The Q-parametrization (Youla) Example: Spring-mass System FRTN Multivariable Control, Lecture 3 Anders Robertsson Automatic Control LTH, Lund University Course outline The Q-parametrization (Youla) L-L5 Purpose, models and loop-shaping by hand L6-L8 Limitations

More information

Fast Algorithms for SDPs derived from the Kalman-Yakubovich-Popov Lemma

Fast Algorithms for SDPs derived from the Kalman-Yakubovich-Popov Lemma Fast Algorithms for SDPs derived from the Kalman-Yakubovich-Popov Lemma Venkataramanan (Ragu) Balakrishnan School of ECE, Purdue University 8 September 2003 European Union RTN Summer School on Multi-Agent

More information

Advances in Convex Optimization: Theory, Algorithms, and Applications

Advances in Convex Optimization: Theory, Algorithms, and Applications Advances in Convex Optimization: Theory, Algorithms, and Applications Stephen Boyd Electrical Engineering Department Stanford University (joint work with Lieven Vandenberghe, UCLA) ISIT 02 ISIT 02 Lausanne

More information

12. Interior-point methods

12. Interior-point methods 12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity

More information

Convex Optimization and l 1 -minimization

Convex Optimization and l 1 -minimization Convex Optimization and l 1 -minimization Sangwoon Yun Computational Sciences Korea Institute for Advanced Study December 11, 2009 2009 NIMS Thematic Winter School Outline I. Convex Optimization II. l

More information

Lecture 9 Sequential unconstrained minimization

Lecture 9 Sequential unconstrained minimization S. Boyd EE364 Lecture 9 Sequential unconstrained minimization brief history of SUMT & IP methods logarithmic barrier function central path UMT & SUMT complexity analysis feasibility phase generalized inequalities

More information

An LMI Approach to the Control of a Compact Disc Player. Marco Dettori SC Solutions Inc. Santa Clara, California

An LMI Approach to the Control of a Compact Disc Player. Marco Dettori SC Solutions Inc. Santa Clara, California An LMI Approach to the Control of a Compact Disc Player Marco Dettori SC Solutions Inc. Santa Clara, California IEEE SCV Control Systems Society Santa Clara University March 15, 2001 Overview of my Ph.D.

More information

Robust Least Squares and Applications Laurent El Ghaoui and Herve Lebret Ecole Nationale Superieure de Techniques Avancees 3, Bd. Victor, 7739 Paris, France (elghaoui, lebret)@ensta.fr Abstract We consider

More information

FIR Filter Design via Semidenite Programming and Spectral Factorization Shao-Po Wu, Stephen Boyd, Lieven Vandenberghe Information Systems Laboratory Stanford University, Stanford, CA 9435 clive@isl.stanford.edu,

More information

CONVEX OPTIMIZATION OVER POSITIVE POLYNOMIALS AND FILTER DESIGN. Y. Genin, Y. Hachez, Yu. Nesterov, P. Van Dooren

CONVEX OPTIMIZATION OVER POSITIVE POLYNOMIALS AND FILTER DESIGN. Y. Genin, Y. Hachez, Yu. Nesterov, P. Van Dooren CONVEX OPTIMIZATION OVER POSITIVE POLYNOMIALS AND FILTER DESIGN Y. Genin, Y. Hachez, Yu. Nesterov, P. Van Dooren CESAME, Université catholique de Louvain Bâtiment Euler, Avenue G. Lemaître 4-6 B-1348 Louvain-la-Neuve,

More information

ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications

ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications ELE539A: Optimization of Communication Systems Lecture 15: Semidefinite Programming, Detection and Estimation Applications Professor M. Chiang Electrical Engineering Department, Princeton University March

More information

12. Interior-point methods

12. Interior-point methods 12. Interior-point methods Convex Optimization Boyd & Vandenberghe inequality constrained minimization logarithmic barrier function and central path barrier method feasibility and phase I methods complexity

More information

Linear Matrix Inequalities in Robust Control. Venkataramanan (Ragu) Balakrishnan School of ECE, Purdue University MTNS 2002

Linear Matrix Inequalities in Robust Control. Venkataramanan (Ragu) Balakrishnan School of ECE, Purdue University MTNS 2002 Linear Matrix Inequalities in Robust Control Venkataramanan (Ragu) Balakrishnan School of ECE, Purdue University MTNS 2002 Objective A brief introduction to LMI techniques for Robust Control Emphasis on

More information

Though the form of the SDP (1) appears very specialized, it turns out that it is widely encountered in systems and control theory. Examples include: m

Though the form of the SDP (1) appears very specialized, it turns out that it is widely encountered in systems and control theory. Examples include: m Algorithms and Software for LMI Problems in Control Lieven Vandenberghe y and Venkataramanan Balakrishnan z A number of important problems from system and control theory can be numerically solved by reformulating

More information

From Convex Optimization to Linear Matrix Inequalities

From Convex Optimization to Linear Matrix Inequalities Dep. of Information Engineering University of Pisa (Italy) From Convex Optimization to Linear Matrix Inequalities eng. Sergio Grammatico grammatico.sergio@gmail.com Class of Identification of Uncertain

More information

Introduction to Convex Optimization

Introduction to Convex Optimization Introduction to Convex Optimization Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2018-19, HKUST, Hong Kong Outline of Lecture Optimization

More information

Lecture 6: Conic Optimization September 8

Lecture 6: Conic Optimization September 8 IE 598: Big Data Optimization Fall 2016 Lecture 6: Conic Optimization September 8 Lecturer: Niao He Scriber: Juan Xu Overview In this lecture, we finish up our previous discussion on optimality conditions

More information

Interior Point Algorithms for Constrained Convex Optimization

Interior Point Algorithms for Constrained Convex Optimization Interior Point Algorithms for Constrained Convex Optimization Chee Wei Tan CS 8292 : Advanced Topics in Convex Optimization and its Applications Fall 2010 Outline Inequality constrained minimization problems

More information

Barrier Method. Javier Peña Convex Optimization /36-725

Barrier Method. Javier Peña Convex Optimization /36-725 Barrier Method Javier Peña Convex Optimization 10-725/36-725 1 Last time: Newton s method For root-finding F (x) = 0 x + = x F (x) 1 F (x) For optimization x f(x) x + = x 2 f(x) 1 f(x) Assume f strongly

More information

Rank-one LMIs and Lyapunov's Inequality. Gjerrit Meinsma 4. Abstract. We describe a new proof of the well-known Lyapunov's matrix inequality about

Rank-one LMIs and Lyapunov's Inequality. Gjerrit Meinsma 4. Abstract. We describe a new proof of the well-known Lyapunov's matrix inequality about Rank-one LMIs and Lyapunov's Inequality Didier Henrion 1;; Gjerrit Meinsma Abstract We describe a new proof of the well-known Lyapunov's matrix inequality about the location of the eigenvalues of a matrix

More information

LMI solvers. Some notes on standard. Laboratoire d Analyse et d Architecture des Systèmes. Denis Arzelier - Dimitri Peaucelle - Didier Henrion

LMI solvers. Some notes on standard. Laboratoire d Analyse et d Architecture des Systèmes. Denis Arzelier - Dimitri Peaucelle - Didier Henrion Some notes on standard LMI solvers Denis Arzelier - Dimitri Peaucelle - Didier Henrion Laboratoire d Analyse et d Architecture des Systèmes LAAS-CNRS 5 L = h l 1 l i Motivation State-feedback stabilization:

More information

POLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS

POLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS POLYNOMIAL OPTIMIZATION WITH SUMS-OF-SQUARES INTERPOLANTS Sercan Yıldız syildiz@samsi.info in collaboration with Dávid Papp (NCSU) OPT Transition Workshop May 02, 2017 OUTLINE Polynomial optimization and

More information

Interior Point Methods for Mathematical Programming

Interior Point Methods for Mathematical Programming Interior Point Methods for Mathematical Programming Clóvis C. Gonzaga Federal University of Santa Catarina, Florianópolis, Brazil EURO - 2013 Roma Our heroes Cauchy Newton Lagrange Early results Unconstrained

More information

CS711008Z Algorithm Design and Analysis

CS711008Z Algorithm Design and Analysis CS711008Z Algorithm Design and Analysis Lecture 8 Linear programming: interior point method Dongbo Bu Institute of Computing Technology Chinese Academy of Sciences, Beijing, China 1 / 31 Outline Brief

More information

Convex optimization examples

Convex optimization examples Convex optimization examples multi-period processor speed scheduling minimum time optimal control grasp force optimization optimal broadcast transmitter power allocation phased-array antenna beamforming

More information

w 1... w L z 1... w e z r

w 1... w L z 1... w e z r Multiobjective H H 1 -Optimal Control via Finite Dimensional Q-Parametrization and Linear Matrix Inequalities 1 Haitham A. Hindi Babak Hassibi Stephen P. Boyd Department of Electrical Engineering, Durand

More information

5. Duality. Lagrangian

5. Duality. Lagrangian 5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

More information

Robust Anti-Windup Compensation for PID Controllers

Robust Anti-Windup Compensation for PID Controllers Robust Anti-Windup Compensation for PID Controllers ADDISON RIOS-BOLIVAR Universidad de Los Andes Av. Tulio Febres, Mérida 511 VENEZUELA FRANCKLIN RIVAS-ECHEVERRIA Universidad de Los Andes Av. Tulio Febres,

More information

Chapter 7 Interconnected Systems and Feedback: Well-Posedness, Stability, and Performance 7. Introduction Feedback control is a powerful approach to o

Chapter 7 Interconnected Systems and Feedback: Well-Posedness, Stability, and Performance 7. Introduction Feedback control is a powerful approach to o Lectures on Dynamic Systems and Control Mohammed Dahleh Munther A. Dahleh George Verghese Department of Electrical Engineering and Computer Science Massachuasetts Institute of Technology c Chapter 7 Interconnected

More information

Robust linear optimization under general norms

Robust linear optimization under general norms Operations Research Letters 3 (004) 50 56 Operations Research Letters www.elsevier.com/locate/dsw Robust linear optimization under general norms Dimitris Bertsimas a; ;, Dessislava Pachamanova b, Melvyn

More information

Agenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP)

Agenda. 1 Cone programming. 2 Convex cones. 3 Generalized inequalities. 4 Linear programming (LP) 5 Second-order cone programming (SOCP) Agenda 1 Cone programming 2 Convex cones 3 Generalized inequalities 4 Linear programming (LP) 5 Second-order cone programming (SOCP) 6 Semidefinite programming (SDP) 7 Examples Optimization problem in

More information

Robust Anti-Windup Controller Synthesis: A Mixed H 2 /H Setting

Robust Anti-Windup Controller Synthesis: A Mixed H 2 /H Setting Robust Anti-Windup Controller Synthesis: A Mixed H /H Setting ADDISON RIOS-BOLIVAR Departamento de Sistemas de Control Universidad de Los Andes Av. ulio Febres, Mérida 511 VENEZUELA SOLBEN GODOY Postgrado

More information

Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method

Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach

More information

Primal-Dual Interior-Point Methods. Ryan Tibshirani Convex Optimization /36-725

Primal-Dual Interior-Point Methods. Ryan Tibshirani Convex Optimization /36-725 Primal-Dual Interior-Point Methods Ryan Tibshirani Convex Optimization 10-725/36-725 Given the problem Last time: barrier method min x subject to f(x) h i (x) 0, i = 1,... m Ax = b where f, h i, i = 1,...

More information

Lecture 4: Linear and quadratic problems

Lecture 4: Linear and quadratic problems Lecture 4: Linear and quadratic problems linear programming examples and applications linear fractional programming quadratic optimization problems (quadratically constrained) quadratic programming second-order

More information

State feedback gain scheduling for linear systems with time-varying parameters

State feedback gain scheduling for linear systems with time-varying parameters State feedback gain scheduling for linear systems with time-varying parameters Vinícius F. Montagner and Pedro L. D. Peres Abstract This paper addresses the problem of parameter dependent state feedback

More information

minimize x subject to (x 2)(x 4) u,

minimize x subject to (x 2)(x 4) u, Math 6366/6367: Optimization and Variational Methods Sample Preliminary Exam Questions 1. Suppose that f : [, L] R is a C 2 -function with f () on (, L) and that you have explicit formulae for

More information

Lecture 15 Newton Method and Self-Concordance. October 23, 2008

Lecture 15 Newton Method and Self-Concordance. October 23, 2008 Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications

More information

Linear Systems with Saturating Controls: An LMI Approach. subject to control saturation. No assumption is made concerning open-loop stability and no

Linear Systems with Saturating Controls: An LMI Approach. subject to control saturation. No assumption is made concerning open-loop stability and no Output Feedback Robust Stabilization of Uncertain Linear Systems with Saturating Controls: An LMI Approach Didier Henrion 1 Sophie Tarbouriech 1; Germain Garcia 1; Abstract : The problem of robust controller

More information

1 Introduction Semidenite programming (SDP) has been an active research area following the seminal work of Nesterov and Nemirovski [9] see also Alizad

1 Introduction Semidenite programming (SDP) has been an active research area following the seminal work of Nesterov and Nemirovski [9] see also Alizad Quadratic Maximization and Semidenite Relaxation Shuzhong Zhang Econometric Institute Erasmus University P.O. Box 1738 3000 DR Rotterdam The Netherlands email: zhang@few.eur.nl fax: +31-10-408916 August,

More information

Primal-Dual Interior-Point Methods. Javier Peña Convex Optimization /36-725

Primal-Dual Interior-Point Methods. Javier Peña Convex Optimization /36-725 Primal-Dual Interior-Point Methods Javier Peña Convex Optimization 10-725/36-725 Last time: duality revisited Consider the problem min x subject to f(x) Ax = b h(x) 0 Lagrangian L(x, u, v) = f(x) + u T

More information

Marcus Pantoja da Silva 1 and Celso Pascoli Bottura 2. Abstract: Nonlinear systems with time-varying uncertainties

Marcus Pantoja da Silva 1 and Celso Pascoli Bottura 2. Abstract: Nonlinear systems with time-varying uncertainties A NEW PROPOSAL FOR H NORM CHARACTERIZATION AND THE OPTIMAL H CONTROL OF NONLINEAR SSTEMS WITH TIME-VARING UNCERTAINTIES WITH KNOWN NORM BOUND AND EXOGENOUS DISTURBANCES Marcus Pantoja da Silva 1 and Celso

More information

Optimization based robust control

Optimization based robust control Optimization based robust control Didier Henrion 1,2 Draft of March 27, 2014 Prepared for possible inclusion into The Encyclopedia of Systems and Control edited by John Baillieul and Tariq Samad and published

More information

1 New advances in nonlinear convex optimization A linear program (LP) is an optimization problem with linear objective and linear equality and inequal

1 New advances in nonlinear convex optimization A linear program (LP) is an optimization problem with linear objective and linear equality and inequal Control Applications of Nonlinear Convex Programming Stephen Boyd 1, Cesar Crusius 2, Anders Hansson 3 Information Systems Laboratory, Electrical Engineering Department, Stanford University Abstract Since

More information

IE 521 Convex Optimization

IE 521 Convex Optimization Lecture 14: and Applications 11th March 2019 Outline LP SOCP SDP LP SOCP SDP 1 / 21 Conic LP SOCP SDP Primal Conic Program: min c T x s.t. Ax K b (CP) : b T y s.t. A T y = c (CD) y K 0 Theorem. (Strong

More information

Semidefinite Programming

Semidefinite Programming Chapter 2 Semidefinite Programming 2.0.1 Semi-definite programming (SDP) Given C M n, A i M n, i = 1, 2,..., m, and b R m, the semi-definite programming problem is to find a matrix X M n for the optimization

More information

Lecture 18: Optimization Programming

Lecture 18: Optimization Programming Fall, 2016 Outline Unconstrained Optimization 1 Unconstrained Optimization 2 Equality-constrained Optimization Inequality-constrained Optimization Mixture-constrained Optimization 3 Quadratic Programming

More information

Static Output Feedback Stabilisation with H Performance for a Class of Plants

Static Output Feedback Stabilisation with H Performance for a Class of Plants Static Output Feedback Stabilisation with H Performance for a Class of Plants E. Prempain and I. Postlethwaite Control and Instrumentation Research, Department of Engineering, University of Leicester,

More information

Semidefinite Programming Duality and Linear Time-invariant Systems

Semidefinite Programming Duality and Linear Time-invariant Systems Semidefinite Programming Duality and Linear Time-invariant Systems Venkataramanan (Ragu) Balakrishnan School of ECE, Purdue University 2 July 2004 Workshop on Linear Matrix Inequalities in Control LAAS-CNRS,

More information

CS295: Convex Optimization. Xiaohui Xie Department of Computer Science University of California, Irvine

CS295: Convex Optimization. Xiaohui Xie Department of Computer Science University of California, Irvine CS295: Convex Optimization Xiaohui Xie Department of Computer Science University of California, Irvine Course information Prerequisites: multivariate calculus and linear algebra Textbook: Convex Optimization

More information

On Computing the Worst-case Performance of Lur'e Systems with Uncertain Time-invariant Delays

On Computing the Worst-case Performance of Lur'e Systems with Uncertain Time-invariant Delays Article On Computing the Worst-case Performance of Lur'e Systems with Uncertain Time-invariant Delays Thapana Nampradit and David Banjerdpongchai* Department of Electrical Engineering, Faculty of Engineering,

More information

Convex Optimization Boyd & Vandenberghe. 5. Duality

Convex Optimization Boyd & Vandenberghe. 5. Duality 5. Duality Convex Optimization Boyd & Vandenberghe Lagrange dual problem weak and strong duality geometric interpretation optimality conditions perturbation and sensitivity analysis examples generalized

More information

Sparse Covariance Selection using Semidefinite Programming

Sparse Covariance Selection using Semidefinite Programming Sparse Covariance Selection using Semidefinite Programming A. d Aspremont ORFE, Princeton University Joint work with O. Banerjee, L. El Ghaoui & G. Natsoulis, U.C. Berkeley & Iconix Pharmaceuticals Support

More information

EE363 homework 8 solutions

EE363 homework 8 solutions EE363 Prof. S. Boyd EE363 homework 8 solutions 1. Lyapunov condition for passivity. The system described by ẋ = f(x, u), y = g(x), x() =, with u(t), y(t) R m, is said to be passive if t u(τ) T y(τ) dτ

More information

Connections Between Duality in Control Theory and

Connections Between Duality in Control Theory and Connections Between Duality in Control heory an Convex Optimization V. Balakrishnan 1 an L. Vanenberghe 2 Abstract Several important problems in control theory can be reformulate as convex optimization

More information

1 Outline Part I: Linear Programming (LP) Interior-Point Approach 1. Simplex Approach Comparison Part II: Semidenite Programming (SDP) Concludin

1 Outline Part I: Linear Programming (LP) Interior-Point Approach 1. Simplex Approach Comparison Part II: Semidenite Programming (SDP) Concludin Sensitivity Analysis in LP and SDP Using Interior-Point Methods E. Alper Yldrm School of Operations Research and Industrial Engineering Cornell University Ithaca, NY joint with Michael J. Todd INFORMS

More information

Lecture 1: Introduction

Lecture 1: Introduction EE 227A: Convex Optimization and Applications January 17 Lecture 1: Introduction Lecturer: Anh Pham Reading assignment: Chapter 1 of BV 1. Course outline and organization Course web page: http://www.eecs.berkeley.edu/~elghaoui/teaching/ee227a/

More information

Research Article An Equivalent LMI Representation of Bounded Real Lemma for Continuous-Time Systems

Research Article An Equivalent LMI Representation of Bounded Real Lemma for Continuous-Time Systems Hindawi Publishing Corporation Journal of Inequalities and Applications Volume 28, Article ID 67295, 8 pages doi:1.1155/28/67295 Research Article An Equivalent LMI Representation of Bounded Real Lemma

More information

Lagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST)

Lagrange Duality. Daniel P. Palomar. Hong Kong University of Science and Technology (HKUST) Lagrange Duality Daniel P. Palomar Hong Kong University of Science and Technology (HKUST) ELEC5470 - Convex Optimization Fall 2017-18, HKUST, Hong Kong Outline of Lecture Lagrangian Dual function Dual

More information

Lieven Vandenberghe. Katholieke Universiteit Leuven. Departement Elektrotechniek, Afdeling ESAT. K. Mercierlaan 94, 3001 Leuven, Belgium

Lieven Vandenberghe. Katholieke Universiteit Leuven. Departement Elektrotechniek, Afdeling ESAT. K. Mercierlaan 94, 3001 Leuven, Belgium Semidenite Programming Lieven Vandenberghe Katholieke Universiteit Leuven Departement Elektrotechniek, Afdeling ESAT K. Mercierlaan 94, 3001 Leuven, Belgium Internet: Lieven.Vandenberghe@esat.kuleuven.ac.be

More information

NONLINEAR PID CONTROL OF LINEAR PLANTS FOR IMPROVED DISTURBANCE REJECTION

NONLINEAR PID CONTROL OF LINEAR PLANTS FOR IMPROVED DISTURBANCE REJECTION NONLINEAR PID CONTROL OF LINEAR PLANTS FOR IMPROVED DISTURBANCE REJECTION Jinchuan Zheng, Guoxiao Guo Youyi Wang Data Storage Institute, Singapore 768, e-mail: Zheng Jinchuan@dsi.a-star.edu.sg Guo Guoxiao@dsi.a-star.edu.sg

More information

Parameterized Linear Matrix Inequality Techniques in Fuzzy Control System Design

Parameterized Linear Matrix Inequality Techniques in Fuzzy Control System Design 324 IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL. 9, NO. 2, APRIL 2001 Parameterized Linear Matrix Inequality Techniques in Fuzzy Control System Design H. D. Tuan, P. Apkarian, T. Narikiyo, and Y. Yamamoto

More information

Agenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primal-dual path following algorithms

Agenda. Interior Point Methods. 1 Barrier functions. 2 Analytic center. 3 Central path. 4 Barrier method. 5 Primal-dual path following algorithms Agenda Interior Point Methods 1 Barrier functions 2 Analytic center 3 Central path 4 Barrier method 5 Primal-dual path following algorithms 6 Nesterov Todd scaling 7 Complexity analysis Interior point

More information

minimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1,

minimize x x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x 2 u 2, 5x 1 +76x 2 1, 4 Duality 4.1 Numerical perturbation analysis example. Consider the quadratic program with variables x 1, x 2, and parameters u 1, u 2. minimize x 2 1 +2x2 2 x 1x 2 x 1 subject to x 1 +2x 2 u 1 x 1 4x

More information

Interior-Point Methods for Linear Optimization

Interior-Point Methods for Linear Optimization Interior-Point Methods for Linear Optimization Robert M. Freund and Jorge Vera March, 204 c 204 Robert M. Freund and Jorge Vera. All rights reserved. Linear Optimization with a Logarithmic Barrier Function

More information

A CONIC DANTZIG-WOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING

A CONIC DANTZIG-WOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING A CONIC DANTZIG-WOLFE DECOMPOSITION APPROACH FOR LARGE SCALE SEMIDEFINITE PROGRAMMING Kartik Krishnan Advanced Optimization Laboratory McMaster University Joint work with Gema Plaza Martinez and Tamás

More information

Lecture 15: October 15

Lecture 15: October 15 10-725: Optimization Fall 2012 Lecturer: Barnabas Poczos Lecture 15: October 15 Scribes: Christian Kroer, Fanyi Xiao Note: LaTeX template courtesy of UC Berkeley EECS dept. Disclaimer: These notes have

More information

Interior Point Methods. We ll discuss linear programming first, followed by three nonlinear problems. Algorithms for Linear Programming Problems

Interior Point Methods. We ll discuss linear programming first, followed by three nonlinear problems. Algorithms for Linear Programming Problems AMSC 607 / CMSC 764 Advanced Numerical Optimization Fall 2008 UNIT 3: Constrained Optimization PART 4: Introduction to Interior Point Methods Dianne P. O Leary c 2008 Interior Point Methods We ll discuss

More information

8. Geometric problems

8. Geometric problems 8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 Minimum volume ellipsoid around a set Löwner-John ellipsoid

More information

Problem structure in semidefinite programs arising in control and signal processing

Problem structure in semidefinite programs arising in control and signal processing Problem structure in semidefinite programs arising in control and signal processing Lieven Vandenberghe Electrical Engineering Department, UCLA Joint work with: Mehrdad Nouralishahi, Tae Roh Semidefinite

More information

2 Abstract We show that a variety of antenna array pattern synthesis problems can be expressed as convex optimization problems, which can be (numerica

2 Abstract We show that a variety of antenna array pattern synthesis problems can be expressed as convex optimization problems, which can be (numerica 1 Antenna Array Pattern Synthesis via Convex Optimization Herve Lebret ENSTA, DFR/A 32, bd Victor 75739 PARIS Cedex 15 France Internet: lebret@ensta.fr Stephen Boyd Information Systems Laboratory, Electrical

More information

A Survey Of State-Feedback Simultaneous Stabilization Techniques

A Survey Of State-Feedback Simultaneous Stabilization Techniques University of New Mexico UNM Digital Repository Electrical & Computer Engineering Faculty Publications Engineering Publications 4-13-2012 A Survey Of State-Feedback Simultaneous Stabilization Techniques

More information

Linear Matrix Inequality (LMI)

Linear Matrix Inequality (LMI) Linear Matrix Inequality (LMI) A linear matrix inequality is an expression of the form where F (x) F 0 + x 1 F 1 + + x m F m > 0 (1) x = (x 1,, x m ) R m, F 0,, F m are real symmetric matrices, and the

More information

Primal-Dual Interior-Point Methods. Ryan Tibshirani Convex Optimization

Primal-Dual Interior-Point Methods. Ryan Tibshirani Convex Optimization Primal-Dual Interior-Point Methods Ryan Tibshirani Convex Optimization 10-725 Given the problem Last time: barrier method min x subject to f(x) h i (x) 0, i = 1,... m Ax = b where f, h i, i = 1,... m are

More information

H State-Feedback Controller Design for Discrete-Time Fuzzy Systems Using Fuzzy Weighting-Dependent Lyapunov Functions

H State-Feedback Controller Design for Discrete-Time Fuzzy Systems Using Fuzzy Weighting-Dependent Lyapunov Functions IEEE TRANSACTIONS ON FUZZY SYSTEMS, VOL 11, NO 2, APRIL 2003 271 H State-Feedback Controller Design for Discrete-Time Fuzzy Systems Using Fuzzy Weighting-Dependent Lyapunov Functions Doo Jin Choi and PooGyeon

More information

IOE 611/Math 663: Nonlinear Programming

IOE 611/Math 663: Nonlinear Programming 1. Introduction I course logistics I mathematical optimization I least-squares and linear programming I convex optimization I example I course goals and topics I nonlinear optimization I brief history

More information

1. Gradient method. gradient method, first-order methods. quadratic bounds on convex functions. analysis of gradient method

1. Gradient method. gradient method, first-order methods. quadratic bounds on convex functions. analysis of gradient method L. Vandenberghe EE236C (Spring 2016) 1. Gradient method gradient method, first-order methods quadratic bounds on convex functions analysis of gradient method 1-1 Approximate course outline First-order

More information

Convex Optimization Lecture 13

Convex Optimization Lecture 13 Convex Optimization Lecture 13 Today: Interior-Point (continued) Central Path method for SDP Feasibility and Phase I Methods From Central Path to Primal/Dual Central'Path'Log'Barrier'Method Init: Feasible&#

More information

Fast algorithms for solving H -norm minimization. problem

Fast algorithms for solving H -norm minimization. problem Fast algorithms for solving H -norm imization problems Andras Varga German Aerospace Center DLR - Oberpfaffenhofen Institute of Robotics and System Dynamics D-82234 Wessling, Germany Andras.Varga@dlr.de

More information

ROBUST ANALYSIS WITH LINEAR MATRIX INEQUALITIES AND POLYNOMIAL MATRICES. Didier HENRION henrion

ROBUST ANALYSIS WITH LINEAR MATRIX INEQUALITIES AND POLYNOMIAL MATRICES. Didier HENRION  henrion GRADUATE COURSE ON POLYNOMIAL METHODS FOR ROBUST CONTROL PART IV.1 ROBUST ANALYSIS WITH LINEAR MATRIX INEQUALITIES AND POLYNOMIAL MATRICES Didier HENRION www.laas.fr/ henrion henrion@laas.fr Airbus assembly

More information

Modern Optimal Control

Modern Optimal Control Modern Optimal Control Matthew M. Peet Arizona State University Lecture 19: Stabilization via LMIs Optimization Optimization can be posed in functional form: min x F objective function : inequality constraints

More information

E5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming

E5295/5B5749 Convex optimization with engineering applications. Lecture 5. Convex programming and semidefinite programming E5295/5B5749 Convex optimization with engineering applications Lecture 5 Convex programming and semidefinite programming A. Forsgren, KTH 1 Lecture 5 Convex optimization 2006/2007 Convex quadratic program

More information

8. Geometric problems

8. Geometric problems 8. Geometric problems Convex Optimization Boyd & Vandenberghe extremal volume ellipsoids centering classification placement and facility location 8 1 Minimum volume ellipsoid around a set Löwner-John ellipsoid

More information

Supplement: Universal Self-Concordant Barrier Functions

Supplement: Universal Self-Concordant Barrier Functions IE 8534 1 Supplement: Universal Self-Concordant Barrier Functions IE 8534 2 Recall that a self-concordant barrier function for K is a barrier function satisfying 3 F (x)[h, h, h] 2( 2 F (x)[h, h]) 3/2,

More information

Convex Optimization. Lieven Vandenberghe Electrical Engineering Department, UCLA. Joint work with Stephen Boyd, Stanford University

Convex Optimization. Lieven Vandenberghe Electrical Engineering Department, UCLA. Joint work with Stephen Boyd, Stanford University Convex Optimization Lieven Vandenberghe Electrical Engineering Department, UCLA Joint work with Stephen Boyd, Stanford University Ph.D. School in Optimization in Computer Vision DTU, May 19, 2008 Introduction

More information

Convex Optimization. Newton s method. ENSAE: Optimisation 1/44

Convex Optimization. Newton s method. ENSAE: Optimisation 1/44 Convex Optimization Newton s method ENSAE: Optimisation 1/44 Unconstrained minimization minimize f(x) f convex, twice continuously differentiable (hence dom f open) we assume optimal value p = inf x f(x)

More information

A Generalized Homogeneous and Self-Dual Algorithm. for Linear Programming. February 1994 (revised December 1994)

A Generalized Homogeneous and Self-Dual Algorithm. for Linear Programming. February 1994 (revised December 1994) A Generalized Homogeneous and Self-Dual Algorithm for Linear Programming Xiaojie Xu Yinyu Ye y February 994 (revised December 994) Abstract: A generalized homogeneous and self-dual (HSD) infeasible-interior-point

More information

Primal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization

Primal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization Primal-Dual Geometry of Level Sets and their Explanatory Value of the Practical Performance of Interior-Point Methods for Conic Optimization Robert M. Freund M.I.T. June, 2010 from papers in SIOPT, Mathematics

More information

Agenda. Applications of semidefinite programming. 1 Control and system theory. 2 Combinatorial and nonconvex optimization

Agenda. Applications of semidefinite programming. 1 Control and system theory. 2 Combinatorial and nonconvex optimization Agenda Applications of semidefinite programming 1 Control and system theory 2 Combinatorial and nonconvex optimization 3 Spectral estimation & super-resolution Control and system theory SDP in wide use

More information

Gradient Descent. Ryan Tibshirani Convex Optimization /36-725

Gradient Descent. Ryan Tibshirani Convex Optimization /36-725 Gradient Descent Ryan Tibshirani Convex Optimization 10-725/36-725 Last time: canonical convex programs Linear program (LP): takes the form min x subject to c T x Gx h Ax = b Quadratic program (QP): like

More information

Optimization: Interior-Point Methods and. January,1995 USA. and Cooperative Research Centre for Robust and Adaptive Systems.

Optimization: Interior-Point Methods and. January,1995 USA. and Cooperative Research Centre for Robust and Adaptive Systems. Innite Dimensional Quadratic Optimization: Interior-Point Methods and Control Applications January,995 Leonid Faybusovich John B. Moore y Department of Mathematics University of Notre Dame Mail Distribution

More information

Identifying Redundant Linear Constraints in Systems of Linear Matrix. Inequality Constraints. Shafiu Jibrin

Identifying Redundant Linear Constraints in Systems of Linear Matrix. Inequality Constraints. Shafiu Jibrin Identifying Redundant Linear Constraints in Systems of Linear Matrix Inequality Constraints Shafiu Jibrin (shafiu.jibrin@nau.edu) Department of Mathematics and Statistics Northern Arizona University, Flagstaff

More information

Multiobjective Optimization Applied to Robust H 2 /H State-feedback Control Synthesis

Multiobjective Optimization Applied to Robust H 2 /H State-feedback Control Synthesis Multiobjective Optimization Applied to Robust H 2 /H State-feedback Control Synthesis Eduardo N. Gonçalves, Reinaldo M. Palhares, and Ricardo H. C. Takahashi Abstract This paper presents an algorithm for

More information

LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009

LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009 LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix

More information

REGLERTEKNIK AUTOMATIC CONTROL LINKÖPING

REGLERTEKNIK AUTOMATIC CONTROL LINKÖPING Invariant Sets for a Class of Hybrid Systems Mats Jirstrand Department of Electrical Engineering Linkoping University, S-581 83 Linkoping, Sweden WWW: http://www.control.isy.liu.se Email: matsj@isy.liu.se

More information