LMI solvers. Some notes on standard. Laboratoire d Analyse et d Architecture des Systèmes. Denis Arzelier - Dimitri Peaucelle - Didier Henrion

Transcription

1 Some notes on standard LMI solvers Denis Arzelier - Dimitri Peaucelle - Didier Henrion Laboratoire d Analyse et d Architecture des Systèmes LAAS-CNRS

2 5 L = h l 1 l i Motivation State-feedback stabilization: ẋ(t)=ax(t)+bu(t) and u(t)=kx(t) - Quadratic Lyapunov function: v(x(t)=x 0 P ;1 x(t) - Closed-loop Lyapunov equation: (A+BK)P+P(A+BK) 0 < 0 - Linearizing change of variable: L = KP - LMI s : AP+PA 0 + BL+L 0 B 0 < 0 P > 0 A = ; B = P = p 1 p 4 p p 3 3 Example : ;p 1 l 4 1 l 1 p 3 l < 0 p 1 p 4 p p 3 5 > 0 3

3 {z } F(x) Standard LMI problem 3 - Feasibility problem: x m R j F 9 0 m x + i F i 0 < i=1, min xr m λ max(f(x)) - Optimization problem: min c 0 x under F(x) < 0, min c 0 x under λ max (F(x)) < 0 8 i F i S n, linearly independant.

4 Standard LMI problem 4 Result 1 : The LM I problem is a convex and non differentiable optimization problem. Example : [G. Pataki 00] 9 ; 4 ;x 1 1 ;x ;x ;1+x λ max F(x)= λ max (F(x)) = 1+ q (x 1 + x ) x 5 5 x 1 Nota : [J. Cullum 75], [F. Alizadeh 95] and [Y. Nesterov 94]

5 Semidefinite programming (SDP) 5 Primal problem: Dual problem: p = min c 0 x under F 0 m x + i F i 0 < i=1 where x R m, Z S n, 8 i F i S m, c R m. Z) = = d min ;Trace(F 0 under ;Trace(F i Z)=c i i 1 m Z 0 Applications: automatic control, (robust control), geometric problems with quadratic forms, statistics, structural design, combinatorial problems, [Nesterov 94], [Vandenberghe 94], [El Ghaoui 00], [Helmberg 00], [Wolcowicz 00]

6 Some associated problems 6 Second Order Cone Programming : SOCP min c x 0 under jjc i x+d i jj e 0 i x+ f i i = 1 L where R x m, C n i m i R, di n R i, e m i, R f i R. Lorentz cone - ice cream cone - quadratic cone: Q ni e 0 i " Ci # " di x+ fi 0 # Q ni = t (" u # u n R i ;1 ) t R jjujj j t Embedding of Q ni in S ni : jjujj t, t1 u u t and i f i)1 C i x+d i (C i x+d i (ei f Applications: [M.S. Lobo 98] grasping force optimization, FIR filter design, truss design.

7 Some associated problems 7 Determinant maximization : MAXDET min c 0 x+log det G(x) ;1 under G(x) > 0 F(x) 0 where x R m and G : R m! S l, F : R m! S n are affine mappings. - log det G(x) ;1 is related to volume in geometric problems. - c 0 x+log det G(x) ;1 is a convex function on fx j G(x) > 0g. - G(x)=1, SDP problem. - c = 0 et F(x)=1, find the analytical center x of G(x) > 0. Applications: minimum volume ellipsoids, matrix completion problems, linear estimation, Gaussian Channel capacity, [Vandenberghe 96].

8 Some associated problems 8 Generalized eigenvalue problems : GEVP min λ max (G(x) F(x)) under G(x) > 0 H(x) > 0 where x R m and G : R m! S l, F : R m! S l, H : R m! S n are affine mappings. - λ max (G(x) F(x)) is quasi-convex. - Generalized linear fractional problem: 0 x+ 0 > x+ min c x+d e f under Ax+b 0 e f 0 Applications: automatic control, linear algebra, [Nesterov 91], [Boyd 94].

9 Summary of key points 9 - Large number of applications: Automatic control, (robust control). Truss design. Mathematic, (geometry, graphs, statistics) Signal processing, (FIR, antennas). - Duality theory: Theoritical analysis. Algorithms design. - Algorithmic: Non differentiable optimization methods. Interior-point methods.

10 A slice of duality 10 Primal problem: Dual problem: p = min c 0 x under F 0 m x + i F i 0 i=1 Z) = = d min ;Trace(F 0 under ;Trace(F i Z)=c i i 1 m Z 0 Weak duality: p d and Strong duality: p = d Theorem 1 : - If primal is strict. feasible with finite p then = p d, reached by dual. - If dual is strict. feasible with finite d then = p d, reached by primal. - If primal and dual are strict. feasible = then p d.

11 A slice of duality (continued) 11 Nota: Duality gap, conic duality, 6= duality in LP, [Vandenberghe 96], [Ben-Tal 00]. Example : p = d = 0 and no solution for the primal min x 1 max ;λ under 6 6 x 1 ; x 1 3 Q under λ 1 λ = ;λ 1 λ = 4 x 1 + x λ Q région primal réalisable 1 x x 1

12 A slice of duality (continued) 1 If primal and dual are strictly feasible: KKT optimality conditions: Y 0 + Y F 0 m x + i F i 0 = i=1 Z 0 Trace(F i Z)+c i = 0 i = 1 m F(x)Z = 0 Complementarity

13 Algorithms, solvers and codes 13 Non differentiable optimization methods: min xr m λ max (F(x))+c 0 x - Convex analysis tools, (subdifferential, cutting planes...). - Subgradient! methods bundle methods. - [Polyak 77], [Shor 85], [Hiriart-Urruty 93], [El Ghaoui 00]. Interior-point methods: min xr m c 0 x F(x) < 0 ;! min xr m c 0 x ; µlog det F(x) - Reduction of the initial problem into a sequence of unconstraint differentiable problems: self-concordant barrier, Newton methods. - Centers methods, primal-dual methods and projective method. - [Fiacco 68], [Karmakar 84], [Nesterov 88], [Nesterov 94], [Alizadeh 95].

14 Primal-dual interior-point methods 14 - Primal-dual central path: (x(µ) Z(µ) µ> 0) min ;trace(f 0 Z) ; µlog det Z Z > 0 ;trace(f i Z)=c i i = 1 m > Y + 0 Y F 0 m x + i F i 0 = i=1 Z 0 Trace(F > i Z)+c i 0 = F(x)Z = µ1 - Newton method - Simultaneous primal-dual solution ;! (x Z ) (x(µ) Z(µ)) µ k 0! - 3 directions: XZ (HRVW,KSH,M), NT, XZ+ZX (AHO).

15 A first table 15 ALGO VERSION METHOD MATLAB INPUTS LMI LANG DIR Pb. SDPA PIPD YES SDPA YALMIP C++ 3 SDP SeDuMi 1.05β PIPD YES DUAL SDPA SDPpack YALMIP SeDuMiInt LMILab MATLAB C - SDP SOCP CSDP DSDP PIPD NO SDPA YALMIP C HRVW SDP PIDS YES SDPA NO C - SDP SDPT PIPD YES DUAL SDPA YALMIP LMILab MATLAB C HKM NT SDP SOCP MOSEK PI YES QPS NO C NT SOCP LOQO NLP YES DUAL SDPA AMPL NO C - SOCP

16 A second table 16 ALGO VERSION METHOD MATLAB INPUTS LMI LANG DIR Pb. SBMethod BUNDLE NO SBM NO C++ - SDP SDPHA (SDPT3) 3.0 PIPD YES DUAL LMITOOL LMILab MATLAB C 3 (+4) SDP SDPpack 0.9β PIPD YES DUAL LMITOOL LMILab C AHO SDP SQLP MAXDET α PIPD YES MATLAB SDPSOL MATLAB C NN MAXDET SDPSOL SP SOCP LMISOL β α PIPD YES MATLAB YALMIP MATLAB NT SDP LMITOOL C LMILab PIPD YES MATLAB YALMIP MATLAB NN SOCP C PROJECTIVE NO LMISOL LMISOL LMISOL C SDP LMILab PROJECTIVE YES MATLAB LMILab MATLAB C - SDP GEVP

17 Some comparisons 17 Large scale, (nber of variables m and nber of rows n), reliability, (feasibility and 6= accuracy, measure of errors)), speed, (cpu execution time and memory size), sparsity. - [H.D. Mittelmann 01] : An independent benchmarking of SDP and SOCP solvers Comparison of 10 solvers, 8 are available, on 50 problems SDP and SOCP : Every code is interesting for a sub-class of problems. Good general codes: SDPA et SeDuMi. - SDPA, SeDuMi, CSDP, DSDP3, SDPT3, MOSEK, LOQO, - BMPR, BMZ, BUNDLE - [Fujisawa 98] : Numerical evaluation of SDPA Evaluation of SDPA and some comparison between SDPT3, CSDP and SDPSOL

18 References [1] CSDP : borchers/csdp.html [] MAXDET : boyd/maxdet/ [3] SDPA : ftp://ftp.is.titech.ac.jp/pup/opres/software/sdpa/ [4] SDPT3 : mattohkc/ [5] SeDuMi : [6] SOCP : boyd/socp/ [7] SP : vandenbe/sp/ [8] SDPpack : [9] SBMethod : [10] LOQO : rvdb/ [11] MOSEK : [1] DSPD3 : benson/

19 [13] LMISOL : mauricio/lmisol10.html [14] SDPHA : rsheng/sdpha/sdpha.html [15] YALMIP : johanl/yalmip.html [16] SDPSOL : boyd/sdpsol.html [17] LMITOOL : [18] LMITRANS : guiness/lmitrans.html [19] SeDuMiInterface : peaucelle [0] F. Alizadeh, Interior point methods in semidefinite programming with applications to combinatorial optimization, SIAM J. Optimization, Vol, 5, pp13-51, [1] C. Beck, Computational issues in solving LMIs, Proceedings of the CDC, Brighton, [] R. Bellman, K. Fan, On systems of linear inequalities in Hermitian matrix variables, Proc. Sympos. Pure Math., V.L. Klee, eds. pp. 1-11, [3] A. Ben-Tal, A. Nemirovskii, Convex optimization in engineering, Technical Report, Technion University, 000. [4] S. Boyd, L. El Ghaoui, E. Feron, V. Balakrishnan, Linear matrix inequalities in systems and control theory, Vol. 15 of SIAM studies in Applied Mathematics, 1994.

20 [5] S. Boyd, Convex optimization, course reader EE36B, University of California, Los Angeles, [6] J. Cullum, W.E. Donath, P. Wolfe, The minimization of certain nondifferentiable sums of eigenvalues of symmetric matrices, Mathematical Programming Study, 3, pp , [7] L. El Ghaoui, S. Niculescu, Advances in linear matrix inequality methods in control, SIAM advances in design and control, 000. [8] A.V. Fiacco, G.P. McCormick, Nonlinear programming: sequential unconstrained minimization techniques, John Wiley and Sons, NY, [9] P. Gahinet, A. Nemirovskii, The projective method for solving linear matrix inequalities, Mathematical programming, Vol. 77, pp , [30] C. Helmberg, Semidefinite programming for combinatorial optimization, ZIB-Report 00-34, 000. [31] J.B. Iriart-Urruty, C. Lemarechal, Convex analysis and minimization algorithms I et II, Vol. 306, Springer-verlag, [3] N. Karmakar, A new polynomial-time algorithm for linear programming, Combinatorica, Vol. 4, pp , [33] M.S. Lobo, L. Vandenberghe, S. Boyd, H. Lebret, Applications of second-order cone programming, Linear algebra and its applications, Vol. 84, pp , [34] H.D. Mittelmann, An independant benchmarking of SDP and SOCP solvers, Septi ème DIMACS Implementation Challenge on Semidefinite and Related Optimization Problems, Technical Report, Dept. of Mathematics, Arizona State University, 001. [35] Y. Nesterov, A. Nemirovskii, A general approach to polynomial-time algorithms design for convex programming, Technical Report, Centr. Econ. and Math. Inst., USSR Acad. Sci., Moscow, USSR, [36] Y. Nesterov, A. Nemirovskii, Interior-point polynomial algorithms in convex programming, Vol. 13 of SIAM studies in Applied Mathematics, 1994.

21 [37] B.T. Polyak, Subgradient mehods: a survey of soviet research, in Nonsmooth optimization, C. Lemar échal and R. Miffin, eds., Pergamon Press, NY, [38] N.Z. Shor, Minimization methods for non-differentiable functions, Springer verlag, [39] M.V. Ramana, An exact duality theory for semidefinite programming and its complexity implications, Mathematical programming, Vol. 77, pp ,1997. [40] L. Vandenberghe, V. Balakrishnan, Algorithms and software tools for LMI problems in control: an overview, Proceedings of the symposium on computer-aided control system design, Dearborn, [41] L. Vandenberghe, S. Boyd, Semidefinite programming, SIAM Review, Vol. 38, pp , [4] L. Vandenberghe, S. Boyd, S. P. Wu, Determinant maximization with linear matrix inequality constraints, SIAM J. on Matrix Analysis and Applications, Vol. 19, No., pp , [43] H. Wolkowicz, R. Saigal, L. Vandenberghe, Handbook of semidefinite programming, Kluwer s international series, 000. [44] S. Zhang, Conic optimization and the duality theory, HPOPT, Rotterdam, 1998.