w Kluwer Academic Publishers Boston/Dordrecht/London HANDBOOK OF SEMIDEFINITE PROGRAMMING Theory, Algorithms, and Applications

Transcription

1 HANDBOOK OF SEMIDEFINITE PROGRAMMING Theory, Algorithms, and Applications Edited by Henry Wolkowicz Department of Combinatorics and Optimization Faculty of Mathematics University of Waterloo Waterloo, Ontario, Canada I\I2L 3G1 Canada Romesh Saigal Department of Industrial and Operations Engineering University of Michigan Ann Arbor, Michigan, USA Lieven Vandenberghe Electrical Engineering Department UCLA Los Angeles, CA USA KM w Kluwer Academic Publishers Boston/Dordrecht/London

2 Contents Contents Contributing Authors List of Figures List of Tables Preface 1 Introduction Henry Wolkowicz, Romesh Saigal, Lieven Vandenherghe 1.1 Semidefinite programming 1.2 Overview of the handbook 1.3 Notation General comments Overview Part I THEORY 2 Convex Analysis on Symmetrie Matrices Florian Jarre 2.1 Introduction 2.2 Symmetrie matrices Operations on Symmetrie matrices 2.3 Analysis with Symmetrie matrices Continuity of eigenvalues Smoothness of eigenvalues The Courant-Fischer-Theorem and its consequ Positive definite matrices Monotonicity of the Löwner partial order Majorization Convex matrix funetions Convex real-valued funetions of matrices

3 vin HANDBOOK OF SEMIDEFINITE PROGRAMMING 3 The Geometry of Semidefinite Programming 29 Gabor Pataki 3.1 Introduction Preliminaries The geometry of cone-lp's: main results Facial structure, nondegeneracy and strict complementarity Tangent spaces The boundary structure inequalities The geometry of the feasible sets expressed with different variables A detailed example Semidefinite Combinatorics The Multiplicity of Optimal Eigenvalues The geometry of a max-cut relaxation The embeddability of graphs Two algorithmic aspects Finding an extreme point Solution Sensitivity Analysis Literature Appendices A: The faces of the semidefinite cone B: Proof of Lemma Duality and Optimality Conditions 67 Alexander Shapiro and Katya Scheinberg 4.1 Duality, optimality conditions, and perturbation analysis 67 Alexander Shapiro Introduction Duality Optimality conditions Stability and sensitivity analysis Notes Parametric Linear Semidefinite Programming 92 Katya Scheinberg Optimality conditions Parametric Objective Function Optimal Partition Sensitivity Analysis Conclusions Self-Dual Embeddings 111 Btienne de Klerk, Tamds Terlaky, Kees Roos 5.1 Introduction Preliminaries The embedding strategy Solving the embedding problem Existence of the central path - a constructive proof Obtaining maximally complementary solutions Separating small and large variables Remaining duality and feasibility issues 131

4 Contents IX 5.9 Embedding extended Lagrange-Slater duals Summary Robustness 139 Aharon Ben-Tal, Laurent El Ghaoui, Arkadi Nemirovski 6.1 Introduction SDPs with uncertain data Problem definition Affine perturbations Quality of approximation Rational Dependence Linear-fractional representations Robustness analysis via Lagrange relaxations Comparison with earlier results Special cases Linear programming with affine uncertainty Robust quadratic programming with affine uncertainty Robust conic quadratic programming Operator-norm bounds Examples A link with combinatorial optimization A link with Lyapunov theory in control Interval computations Worst-case Simulation for uncertain dynamical Systems Robust structural design Concluding Remarks Error Analysis 163 Zhiquan Luo and Jos Sturm 7.1 Introduction / Preliminaries Forward and backward error Faces of the cone 167 Second order cone 167 Positive semidefinite cone 169 General case The regularized backward error Regularization steps Infeasible Systems Systems of quadratic inequalities Convex quadratic Systems Generalized convex quadratic Systems 188 Part II ALGORITHMS 8 Symmetrie Cones, Potential Reduction Methods 195 Fand Alizadeh, Stefan Schtnieta 8.1 Introduction Semidefinite programming: Cone-LP over Symmetrie cones 198

5 X HANDBOOK OF SEMIDEFINITE PROGRAMMING 8.3 Euclidean Jordan algebras Definitions and basic properties Eigenvalues, degree, rank and norms Simple Jordan algebras and decomposition theorem 208 Group one: Symmetrie and Hermitian matrices 211 Group 2: The algebra of quadratic forms 212 Group 3: The Exceptional Albert algebra Complementarity in semidefinite programming Potential reduetion algorithms for semidefinite programming The logarithmic barrier funetion for Symmetrie cones Potential funetions Potential reduetion and polynomial time solvabiiity Feasibility and boundedness Properties of Potential funetions Properties of Linear scalings A potential reduetion algorithm using linear scaling Properties of projeetive scaling Potential reduetion with projeetive scaling The Recipe Potential Reduetion and Primal-Dual Methods 235 Levent Tuncel 9.1 Introduction Fundamental ingredients What are the uses of a potential funetion? Kojima-Shindoh-Hara Approach Nesterov-Todd Approach Self-scaled Barriers and Long Steps Scaling, notions of primal-dual symmetry and scale invariance An Abstraction of the v space Approach A potential reduetion framework Path-Following Methods 267 Renata Monteiro, Michael Todd 10.1 Introduction The central path Search directions Primal-dual path-following methods The MZ primal-dual framework and a scaling procedure Short-step and predictor-corrector algorithms Long-step method Convergence results for other families of directions 300 Monteiro and Tsuchiya family 301 KSH family Bündle Methods and Eigenvalue Functions 307 Christoph Helmberg, Francois Oustry 11.1 Introduction The maximum eigenvalue funetion General scheme 310

6 Contents XI 11.4 The proximal bündle method The spectral bündle method The mixed polyhedral-semidefinite method A second-order proximal bündle method Second-order development of / Quadratic step The dual metric The second-order proximal bündle method Implementations Computing the eigenvalues Structure of the mapping Solving the quadratic semidefinite program The rieh oracle Numerical results The spectral bündle method The mixed polyhedral-semidefinite bündle method The second-order proximal bündle method 336 Part IM APPLICATIONS and EXTENSIONS 12 Combinatorial Optimization 343 Michel Goemans, Franz Rendl 12.1 From combinatorial optimization to SDP Quadratic problems in binary variables as SDP Modeling linear inequalities Specific combinatorial optimization problems Equipartition Stable sets and the 9 funetion 349 Perfect graphs Traveling salesman problem Quadratic assignment problem Computational aspects SDPs reducing to eigenvalue bounds Approximation results through SDP Nonconvex Quadratic Optimization 361 Yuri Nesterov, Henry Wolkowicz, Yinyu Ye 13.1 Introduction Lagrange Multipliers for Q 2 P Global Quadratic Optimization via Conic Relaxation 363 Yuri Nesterov Convex conic constraints on squared variables Using additional information General constraints on squared variables Why the linear constraints are difficult? Maximization with a smooth constraint Some applications Discussion Quadratic Constraints 387 Yinyu Ye

7 xil HANDBOOK OF SEMIDEFINITE PROGRAMMING Positive Semi-Definite Relaxation Approximation Analysis Results for Other Quadratic Problems Relaxations of Q 2 P 395 Henry Wolkowicz Relaxations for the Max-cut Problem 396 Several Different Relaxations 396 A Strengthened Bound for MC 399 Alternative Strengthened Relaxation General Q 2 P 402 The Lagrangian Relaxation of a General Q 2 P 403 Valid Inequalities 404 Specific Instances of SDP Relaxation Strong Duality 411 Convex Quadratic Programs 412 Nonconvex Quadratic Programs 413 Rayleigh Quotient 413 Trust Region Subproblem 413 Two Trust Region Subproblem 415 General Q 2 P 415 Orthogonally Constrained Programs with Zero Duality Gaps SDP in Systems and Control Theory 421 Venkataramanan Balakrishnan, Fan Wang 14.1 Introduction Control System analysis and design: An introduction Linear fractional representation of uncertain Systems Poiytopic Systems Robust stability analysis and design problems Robustness analysis and design for linear poiytopic Systems using quadratic Lyapunov functions Robust stability analysis Stabilizing state-feedback Controller synthesis Gain-scheduled Output feedback Controller synthesis Robust stability analysis of LFR Systems in the IQC framework Diagonal nonlinearities Parametric uncertainties Structured dynamic uncertainties Stabilizing Controller design for LFR Systems Quadratic stability analysis of LFR Systems State feedback Controller design for LFR Systems Gain-scheduled output feedback Controller design Conclusion Structural Design 443 Aharon Ben-Tal, Arkadi Nernirovski 15.1 Structural design: general setting Semidefinite reformulation of (Pini) From primal to dual From dual to primal 458

8 Contents xiü 15.5 Explicit forms of the Standard truss and shape problems Concluding remarks Moment Problems and Semidefinite Optimization 469 Dimitris Bertsimas, Jay Sethuraman 16.1 Introduction Semidefinite Relaxations for Stochastic Optimization Problems Model description The Performance optimization problem Linear constraints Positive semidefinite constraints On the power of the semidefinite relaxation Optimal Bounds in Probability Optimal bounds for the univariate case using semidifinite optimization Explicit bounds for the (n, 1, fi), (n, 2, Ä")-bound proble ms The complexity of the (n, 2, i?" ), (n, k, i?")-bound proble ms Moment Problems in Finance Bounds in one dimension Bounds in multiple dimensions Moment Problems in Discrete Optimization Concluding Remarks Design of Experiments in Statistics 511 Valerii Fedorov and Jon Lee 17.1 Design of Regression Experiments 511 Valerii Fedorov Main Optimization Problem 511 Models and Information Matrix 511 Characterization of Optimal Designs Constraints Imposed on Designs 519 Linear Constraints 519 Linearization of Nonlinear Convex Constraints 520 Directly Constrained Design Measures 521 Marginal Design Measures Numerical Construction of Optimal Designs 523 Direct Approaches 523 The First Order Algorithms 523 Second Order Algorithms 525 Linear Constraints. Direct First Order Algorithm. 526 Nonlinear Constraints ( Semidefinite programming in experimental design 528 Jon Lee Covariance Matrices 528 Reliability of Test Scores 529 Maximum-Entropy Sampling Linear Models 531 E-Optimal Design 531 A-Optimal Design 532 D-Optimal Design 532

9 xiv HANDBOOK OF SEMIDEFINITE PROGRAMMING 18 Matrix Completion Problems 533 Abdo Alfakih, Henry Wolkowicz 18.1 Introduction Weighted Closest Euclidean Distance Matrix Distance Geometry Program Formulations Duality and Optimality Primal-Dual Interior-Point Algorithm Weighted Closest Positive Semidefinite Matrix Primal-Dual Interior-Point Algorithms Other Completion Problems Eigenvalue Problems and Nonconvex Minimization 547 Florian Jarre 19.1 Introduction Selected Eigenvalue Problems Generalization of Newtons method An algorithm for unconstrained minimization 552 Discussion A method for constrained problems The constrained problem Outline of the method Solving the barrier subproblem ("centering step") The predictor step The overall algorithm Conclusion General Nonlinear Programming 563 Serge Kruk and Henry Wolkowicz 20.1 Introduction The Simplest Case Multiple Trust-Regions Approximations of Nonlinear Programs Quadratically Constrained Quadratic Programming Conclusion 574 References 577 Appendix 643 A-.l Conclusion and Further Historical Notes 643 A-.l.l Combinatorial Problems 644 A-.l.2 Complementarity Problems 644 A-.l.3 Complexity, Distance to Ill-Posedness, and Condition Numbers 644 A-.l.4 Cone Programming 645 A-.l.5 Eigenvalue Functions 645 A-.l.6 Engineering Applications 645 A-.1.7 Financial Applications 645 A-.l.8 Generalized Convexity 645 A-.l.9 Geometry 645 A-.l.10 Implementation 646

10 Contents XV A-.l.ll Matrix Completion Problems 646 A Nonlinear and Nonconvex SDPs 646 A Nonlinear Programming 647 A-.1.14Quadratic Constrained Quadratic Programs 647 A Sensitivity Analysis 647 A-.1.16Statistics 647 A-.1.17Books and Related Material 647 A Review Articles 648 A Computer Packages and Test Problems 648 A-.2 Index 649