CITY UNIVERSITY OF HONG KONG Topics in Optimization: Solving Second-Order Conic Systems with Finite Precision; Calculus of Generalized Subdifferentials for Nonsmooth Functions Submitted to Department of Mathematics in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy by Vera Roshchina June 2009
Abstract This work consists of three parts. The first one is devoted to the work done jointly with my supervisor Prof. Felipe Cucker from the City University of Hong Kong and Prof. Javier Peña from the Carnegie-Mellon University on the finite-precision analysis of an interior-point method for solving second-order conic systems. The second part concerns results related to calculus of exhausters, which were obtained jointly or with advice of Prof. Vladimir Fedorovich Demyanov from St.-Petersburg State University. The third chapter is devoted to the calculus of generalized differentials and contains authors s original results on this subject. In Chapter 1, an interior-point method to decide feasibility problems of secondorder conic systems is described and analyzed. A main feature of this algorithm is that arithmetic operations are performed with finite precision. Bounds for both the number of arithmetic operations and the finest precision required are exhibited. This work has given rise to publication [21]. Chapter 2 is devoted to the study of exhausters and some related problems. In Section 2.2 we introduce the notions of upper and lower exhausters and give some historic background, then in Sections 2.3 and 2.4 we discuss optimality conditions in terms of exhausters. The optimality conditions in terms of proper exhausters were stated by Demyanov [23], and the optimality conditions in terms of adjoint exhausters were obtained by Roshchina in [105]. In Section 2.5 we address the problem of constructing exhausters, and show how to construct an exhauster of an arbitrary locally Lipschitz function. This was originally published by Roshchina in [100]. Section 2.6 is based on the work [103] and devoted to the problem of minimality of exhausters. In Sections 2.7 and i
2.8 the problems of reducing exhausters and converting lower exhausters to upper ones and vice versa are discussed. These two sections are based on the work [104]. In Chapter 3 relationships between exhausters and generalized subdifferentials are discussed. In Section 3.2 we introduce the relationship between exhausters, Fréchet and Gâteaux subdifferentials and provide some calculus rules based on this relationship. In Section 3.3 we study the relationships between exhausters and the Mordukhovich subdifferential, following the lines of [101]. Section 3.4 is devoted to a study of Mordukhovich subdifferential of a minimum of approximate convex functions. This study was motivated by the results obtained in calculus of exhausters, and is based on [102]. The thesis has two appendices. Appendix A contains some technical details for the finite-precision analysis of Chapter 1, and Appendix B contains some classical facts from Nonsmooth Analysis. ii
Contents Abstract i Certification of approval by the Panel of Examiners iii Acknowledgements iv Contents v List of figures x Abbreviations xii Notations xiii 1 Solving SOC systems with finite precision 1 1.1 Introduction.............................. 1 1.1.1 Second-order conic programming (SOCP)......... 1 1.1.2 Examples and applications of SOCP............ 2 1.1.3 Euclidean Jordan algebras on second-order cones..... 3 1.1.4 Interior-point methods.................... 4 1.1.5 Software for solving SOCP.................. 6 1.1.6 Finite-precision analysis and other computational issues.. 6 1.1.7 The Problem and The Main Result............. 7 1.2 Basic objects and notations..................... 11 1.3 Main Ideas............................... 12 v
1.3.1 Reformulation......................... 12 1.3.2 The algorithm......................... 15 1.3.3 On the update of ( x; y; s)................... 17 1.3.4 The stepping stones...................... 18 1.3.5 Proof of Theorem 1.1.1.................... 21 1.4 Preliminaries............................. 21 1.4.1 Second-order cones...................... 22 1.4.2 Second-order cones and Jordan algebras.......... 23 1.4.3 Barrier functions....................... 25 1.4.4 Local norms.......................... 25 1.5 Some useful lemmas......................... 27 1.6 Floating-point arithmetic....................... 28 1.7 Proofs of Propositions 1.3.1 and 1.3.2................ 32 1.7.1 Proof of Proposition 1.3.1................... 32 1.7.2 Proof of Proposition 1.3.2.................. 34 1.7.3 Proof of Lemma 1.7.1..................... 34 1.8 Proofs of Propositions 1.3.3 and 1.3.4................ 37 1.8.1 Proof of Proposition 1.3.3.................. 37 1.8.2 Proof of Proposition 1.3.4.................. 39 1.8.3 Proof of Lemma 1.8.1..................... 42 1.9 Proof of Proposition 1.3.6...................... 45 1.9.1 Proof of Step 1........................ 45 1.9.2 Proof of Step 2........................ 59 1.9.3 Proof of Step 3........................ 61 1.9.4 Proof of Step 4........................ 61 2 Exhausters and related problems 62 2.1 Introduction.............................. 62 2.2 Families of upper and lower approximations. Exhausters..... 65 vi
2.3 Unconstrained Optimality Conditions in Terms of Exhausters.. 68 2.3.1 Unconstrained Optimality Conditions in Terms of Proper Exhausters.......................... 68 2.3.2 Unconstrained Optimality Conditions in Terms of Adjoint Exhausters.......................... 71 2.3.3 Examples........................... 74 2.4 Constrained Optimality Conditions in Terms of Exhausters.... 76 2.4.1 Constrained Optimality Conditions in Terms of Proper Exhausters............................ 76 2.4.2 Constrained Optimality Conditions in Terms of Adjoint Exhausters............................ 80 2.4.3 Examples........................... 81 2.5 Constructing Exhausters....................... 82 2.6 Minimality of Exhausters....................... 86 2.6.1 Introduction.......................... 86 2.6.2 Definitions of minimality................... 86 2.6.3 Necessary conditions for the minimality of exhausters... 88 2.6.4 Sufficient conditions for the set minimality of exhausters. 94 2.7 Reducing Exhausters......................... 97 2.7.1 Reduction of exhauster by alteration of shape of the sets. 97 2.7.2 Reduction of exhauster by removing abundant sets.... 102 2.8 Convertors............................... 104 2.8.1 Simple convertors....................... 105 2.8.2 Advanced convertors..................... 105 3 Calculus of generalized subdifferentials 109 3.1 Introduction.............................. 109 3.2 Exact Calculus Rules for the Fréchet Subdifferential........ 113 vii
3.2.1 Relationship between the Fréchet and Gâteaux subdifferential and exhausters...................... 114 3.2.2 Exact calculus rules for the Fréchet subdifferential of Hadamard directionally differentiable functions............. 121 3.2.3 Examples........................... 131 3.2.4 General conclusion concerning Section 3.2......... 136 3.3 Relationship between the Mordukhovich subdifferential and exhausters................................ 137 3.3.1 Set convergence and the definition of the Mordukhovich subdifferential........................... 137 3.3.2 Mordukhovich subdifferential and upper exhausters.... 143 3.3.3 Example............................ 149 3.4 Mordukhovich subdifferential of pointwise minimum of approximate convex functions........................ 151 3.4.1 Introduction.......................... 151 3.4.2 Proof of Theorem 3.4.1.................... 153 3.4.3 Examples........................... 161 Index 167 Bibliography 170 A Technical details for Chapter 1 183 A.1 Proofs of useful lemmas...................... 183 A.1.1 Proof of Lemma 1.5.1..................... 185 A.1.2 Proof of Lemma 1.5.2..................... 186 A.1.3 Proof of Lemma 1.5.3..................... 186 A.1.4 Proof of Lemma 1.5.4..................... 194 A.1.5 Proof of Lemma 1.5.5..................... 194 A.1.6 Proof of Lemma 1.5.6..................... 195 viii
A.2 Proof of Lemma 1.6.1......................... 196 B Some basic facts from Nonsmooth Analysis 205 B.1 Convex Analysis............................ 205 B.1.1 Convex Sets.......................... 205 B.1.2 Convex Functions and the subdifferential.......... 207 B.1.3 Sublinearity and support functions............. 208 B.2 Lipschitz continuity.......................... 209 B.3 Directional derivatives and optimality conditions.......... 209 B.4 Quasidifferentiability......................... 213 ix
List of Figures 2.1 An upper exhauster that is not minimal, but satisfies a necessary condition for the minimality (Example 5).............. 93 2.2 Difference between M (C) and M (C)............... 94 2.3 An upper exhauster that satisfies the set-minimality, but is not minimal (Example 7)......................... 97 2.4 Geometrical construction of Q (C, g 0 ) and B (C, g 0 ) (Examples 8-9) 98 2.5 Reducing an upper exhauster (Example 12)............ 102 2.6 Using convertors and modified convertors to obtain a lower exhauster from an upper one (Example 13).............. 108 3.1 An upper exhauster and the Fréchet subdifferential (Example 14) 131 3.2 An upper exhauster and the Fréchet subdifferential (Example 15) 132 3.3 Constructing an exhauster of f (Example 17).......... 134 3.4 Evaluating the Fréchet subdifferential via an upper exhauster (Example 18)............................... 136 3.5 Obtaining the Fréchet and the Mordukhovich subdifferentials via exhausters (Example 19)....................... 149 3.6 Mordukhovich subdifferential of the minimum of strictly differentiable functions (Example 20).................... 162 3.7 Mordukhovich subdifferential of the minimum of two convex functions (Example 21).......................... 163 x
3.8 Failure of Theorem 3.4.1 to hold without the approximate convexity assumption (Example 23)...................... 165 xi
Abbreviations D.-d.d. Dini directionally differentiable H.-d.d. Hadamard directionally differentiable IPM interior point method l.c.a. lower concave approximation L.E.F. exhaustive family of lower concave approximations (l.c.a s) LP linear programming NT Nesterov-Todd (algorithm or direction) p.h. positively homogeneous QCQP convex quadratically constrained quadratic programming QP quadratic programming SDP semidefinite programming SOCP second-order cone programming u.c.a. upper convex approximation U.E.F. exhaustive family of upper convex approximations (u.c.a s) xii
Notations IN The set of natural numbers IR The set of real numbers IR n Real vector space (n-dimensional) IB The unit ball in the space under consideration S The unit sphere in the space under consideration S n The unit sphere in IR n cl Closure of a set in IR n co Convex hull of a set in IR n int Interior of a set in IR n Depending on the context, denotes either the boundary of a set or a subdifferential C, C Subdifferential in the sense of Convex Analysis + C Superdifferential (of a concave function) in the sense of Convex Analysis Cl Clarke subdifferential F, F Fréchet (lower) subdifferential + F Fréchet upper subdifferential G, G Gâteaux (lower) subdifferential xiii
xiv + G Gâteaux upper subdifferential M Mordukhovich subdifferential, Demyanov-Rubinov sub- and superdifferentials, The scalar product given by a, b = a T b = n i=1 a ib i for every a, b in IR n Euclidean norm (2-norm) given by x = x, x For vectors x, y and z the following are synonymous x y z = (xt, y T, z T ) T = (x; y; z). For any a, b IR, a b by (a, b) and [a, b] we denote the open and the closed intervals respectively with endpoints in a and b. By {x k } or {x k } 1 we denote a sequence of points in the space under consideration indexed by the set of natural numbers. By t k t we mean that a sequence of real numbers {t k } converges to t, and t k > t for every k in IN.