ON THE ARITHMETICGEOMETRIC MEAN INEQUALITY AND ITS RELATIONSHIP TO LINEAR PROGRAMMING, BAHMAN KALANTARI


 Kristin Skinner
 1 years ago
 Views:
Transcription
1 ON THE ARITHMETICGEOMETRIC MEAN INEQUALITY AND ITS RELATIONSHIP TO LINEAR PROGRAMMING, MATRIX SCALING, AND GORDAN'S THEOREM BAHMAN KALANTARI Abstract. It is a classical inequality that the minimum of the ratio of the (weighted) arithmetic mean to the geometric mean of a set of positive variables is equal to one, and is attained at the center of the positivity cone. While there are numerous proofs of this fundamental homogeneous inequality, in the presence of an arbitrary subspace, and/or the replacement of the arithmetic mean with an arbitrary linear form, the new minimization is a nontrivial problem. We prove a generalization of this inequality, also relating it to linear programming, to the diagonal matrix scaling problem, as well as to Gordan's theorem. Linear programming is equivalent to the search for a nontrivial zero of a linear or positive semidenite quadratic form over the nonnegative points of a given subspace. The goal of this paper is to present these intricate, surprising, and signicant relationships, called scaling dualities, and via an elementary proof. Also, to introduce two conceptually simple polynomialtime algorithms that are based on the scaling dualities, signicant bounds, as well as Nesterov and Nemirovskii's machinery of selfconcordance. The algorithms are simultaneously applicable to linear programming, to the computation of a separating hyperplane, to the diagonal matrix scaling problem, and to the minimization of the arithmeticgeometric mean ratio over the positive points of an arbitrary subspace. The scaling dualities, the bounds, and the algorithms are special cases of a much more general theory on convex programming, developed by the author. For instance, via the scaling dualities semidenite programming is a problem dual to a generalization of the classical tracedeterminant ratio minimization over the positive denite points of a given subspace of the Hilbert space of symmetric matrices. Key words. Convexity. Arithmeticgeometric mean inequality, Linear programming, Gordan's theorem, Diagonal matrix scaling, Department of Computer Science, Rutgers University, New Brunswick, NJ
2 2 BAHMAN KALANTARI. Introduction. The classical (weighted) arithmeticgeometric mean inequality can be viewed as a statement on the minimization of the ratio of the two (weighted) means over the positive orthant: the minimum is attained at the center of this cone, any positive scalar multiple of the vector of ones. While there are numerous proofs of this fundamental homogeneous inequality, see e.g. Hardy, Littlewood, and Polya [5], Bullen, Mitrinovic, and Vasic [3], Mitrinovic, Pecaric, and Fink [8], and Alzer []; in the presence of a proper subspace not containing the center, the new minimization is a nontrivial problem. A more general version of the above problem is the minimization of the ratio of an arbitrary linear function to the geometric mean, over the positive points of an arbitrary subspace. Still an even more general version is when the ratio is replaced with that of an arbitrary positive semidenite quadratic form, to the square of the geometric mean. On the other hand linear programming can be formulated as the problem of computing a nontrivial zero of a linear function over the intersection of the nonnegative orthant and a given subspace of the Euclidean space, or proving that such zero does not exist. This problem is in fact Karmarkar's canonical linear programming [5]. A more general, more interesting, and more important problem than Karmarkar's canonical LP problem is when the linear form is replaced with a positive semidenite quadratic form, considered in Kalantari [7]. Indeed, as proved in [7], by removing the positive semideniteness assumption, the zeronding problem becomes NPcomplete. Linear programming is also equivalent to the problem of testing if the convexhull of a given set of points in the Euclidean space contains the origin. A more extended version of this problem is the following: determine if the convexhull of a given set of points contains the origin; otherwise, compute a separating hyperplane. In fact the latter problem calls for an algorithmic proof of Gordan's theorem. Another problem of interest is the diagonal matrix scaling problem. Given a positive semidenite symmetric matrix, this is the problem of computing a positive denite diagonal matrix such that pre and post multiplication of the given matrix by the computed diagonal matrix results in a matrix whose row sums (hence column sums) equal prescribed positive numbers; or proving that such diagonal matrix does not exist. The case of diagonal scaling of matrices with nonnegative entries, not necessarily positive semidenite, has been a problem of interest for a very long time. The matrix scaling problem can also be stated in the presence of a subspace. In this more general case of the problem, the diagonal matrix is replaced with the product of a diagonally scaled orthogonal projection matrix, and a positive denite diagonal matrix induced by a positive point of the underlying subspace. A diagonal scaling problem can also be dened for the case of linear forms over a given subspace. This turns out to be a problem dual to Karmarkar's canonical LP. The goal of this paper is to present the intricate, surprising, and signicant relationships between the above stated problems, in the form dualities, called scaling dualities, and via an elementary proof. Also, to introduce two conceptually simple polynomialtime algorithms that are based on the scaling dualities, several signicant bounds, as well as Nesterov and Nemirovskii's machinery of selfconcordance. The algorithms are simultaneously applicable to linear programming, to the computation of a separating hyperplane, to the diagonal matrix scaling problem, and to the minimization of the arithmeticgeometric mean ratio over the positive points of a given subspace. The scaling dualities, the bounds, and the algorithms are special cases of a much more general theory on convex programming, developed by the author (see Kalantari [3]). For instance, via the scaling dualities semidenite programming is a problem dual to a generalization of the classical tracedeterminant ratio minimization over the positive denite points of a given subspace of the Hilbert space of symmetric matrices. Indeed, all the theorems to be stated in this paper, as well as the two algorithms can be shown to hold for analogous problems dened with respect to the cone of positive semidenite symmetric matrices. Although the proof of general scaling dualities is nontrivial, the proof of those presented in this paper are established via elementary means. The simplicity of the proofs is due
3 AGMean Inequality, LP Matrix Scaling, Gordan's Theorem 3 to the very special nature of the underlying problems, i.e., linearity or quadraticity of the homogeneous objective function, as well as symmetric properties of the underlying cone, the nonnegative orthant. In x 2, we preset the main theorems. In x 3 and x 4, we present their proofs. In x 5, we describe the two algorithms, state their complexities, the main ingredients that result in the algorithms, and their application in the derivation of the claimed complexity results. One of the major ingredient is the scaling dualities. Other ingredients are signicant bounding theorems that will be stated without proof. The proof of these bounds is given in [3] and in much more generality. Another ingredient is a theorem that combines two fundamental but most basic properties from Nesterov and Nemirovskii's theory on selfconcordance, [9]. In our concluding remarks, x 6, we briey describe generalization of the theorems and the algorithms, in particular with respect to corresponding problems dened over the cone of positive semidenite matrices. 2. The arithmeticgeometric mean inequality, linear programming, Gordan's separation theorem, and matrix scaling. Let K = fx 2 < n : x 0g, the nonnegative orthant, and P K = fx 2 < n : x > 0g, the positive orthant. Let 2 K n be an arbitrary vector of weights, and = i= i. The classical (weighted) arithmeticgeometric mean inequality is the following (2.) T x Qn i= x i i = ; 8x 2 K : Moreover, the minimum is attained at any positive scalar multiple of e = (; : : :; ) T. Let W be a subspace of < n. If W is a proper subspace, we will assume that W = fx 2 < n : Ax = 0g, where A is a given m n matrix of rank m. Let S = fx 2 < n : kxk = g. We will assume that d 0 2 W \ K \ S is available. Let (x) = 2 xt Qx, where Q is an n n symmetric matrix, also assumed to be positive semidenite, i.e., x T Qx 0, for all x 2 < n. Let (2.2) F (x) =? nx i= i ln x i : Linear programming is equivalent to the problem of testing if has a nontrivial zero over W \ K, see Kalantari [7]. Indeed, the latter problem is more general and more important than Karmarkar's canonical linear programming problem [5], which is the following problem: given c 2 < n, determine if c T x = 0, for some x 2 W \ K, x 6= 0. It is easy to see that has a desired zero if and only if = 0, where (2.3) = minf(x) : x 2 W \ K \ Sg: We shall refer to the problem of testing if = 0 as homogeneous programming (HP). Let the logarithmic potential, and the homogeneous potential be dened as (2.4) respectively. Let (2.5) (x) = (x) + F (x); X (x) = (x) exp(? 2 F (x)) = (x) n i= x2 i= i = inff (x) : x 2 W \ K g; X = inffx (x) : x 2 W \ K g: ; The scaling problem (SP) is to determine if =?, and if is nite, to compute its value, together with a corresponding minimizer d. The Homogeneous scaling problem (HSP) is to determine if X =?, and if X is nite, to compute its value, together with a corresponding minimizer d. Since (x) is convex and F (x) is strictly convex, (x) is strictly convex.
4 4 BAHMAN KALANTARI For a given d 2 K, let (2.6) D = r 2 F (d)?=2 = diag( d d p ; ; p n ); n (2.7) e = D? d = (p ; ; p n ) T ; (2.8) ;d (x) = (D x) = 2 xt D QD x; (2.9) ;d(x) = (D x) = ;d (x) + F (x) + F (d): Let P ;d be the orthogonal projection operator onto the subspace W ;d = D? W = fx 2 <n : AD x = 0g. Thus, if W = < n, then P ;d = I, the identity matrix. Otherwise, (2.0) P ;d = I? D A T (AD 2 AT )? AD : The algebraic scaling problem (ASP) is to test the solvability of the scaling equation (SE) : (2.) P ;d r ;d (e ) = P ;d D QD e = e ; d 2 W \ K : Theorem 2.. The following statements are equivalent: () : > 0. (2) : 9 d 2 W \ K such that (d) > 0, and X (x) X (d), 8 x 2 W \ K. (3) : 9 d 2 W \ K such that (x) (d), 8 x 2 W \ K. (4) : 9 d 2 W \ K such that P e;d D e QD e e = P e;d. (5) : 9 d 2 W \ K such that P e;d D e QD e e > 0. (6) : 9 d 2 W \ K such that P ;d D QD e = e. (7) : 9 d 2 W \ K such that kp ;d D QD e? e k < min = minf ; : : :; n g. (8) : 9 d 2 W \ K such that P ;d D QD e > 0. (9) : 9 d 2 W \ K such that P 2 ;dd 2QD 2 =, where 2 = ( 2 ; ; 2 n) T. Moreover, (3), (4), and (6) have a unique and common solution d, and d also satises (2). The following is an obvious but important corollary of Theorem 2.. Corollary 2.2. (Scaling dualities for positive semidenite quadratic forms) Either = 0, or condition (i) is true, i = 2; : : :; 9; but not both. The equivalence of the statements of Theorem 2. justies the naming of the corresponding dualities of Corollary 2.2 as scaling dualities. From the algorithmic point of view it turns out that the dualities that concern diagonal scaling are the more important dualities. Indeed, Theorem 2. can be stated in more generality, where (x) can be an arbitrary convex homogeneous function. The corresponding proof however becomes more involved than the simple proofs presented in this paper. For the proof of some of the corresponding parts of Theorem 2., as derived for general
5 AGMean Inequality, LP Matrix Scaling, Gordan's Theorem 5 convex homogeneous functions, see Kalantari []. We mention that linearly constrained convex quadratic programming can be formulated as the problem of computing a nontrivial root of the homogeneous function 2 xt Qx=e T x + c T x over W \ K, for some positive semidenite Q, and c 2 < n, see Kalantari [3]. In fact, most of the scaling dualities implied by Theorem 2. can be stated in much more generality, where the underlying cone is an arbitrary closed convex pointed cone, and an arbitrary convex form dened over this cone (see x 5). Our study of what we now call scaling dualities, and their algorithmic signicance began in Kalantari [6], and was continued in [7][]. We emphasize that the goal of the present paper is to present these signicant dualities for the very special problems considered, and via an elementary proof. Also, to introduce two conceptually simple, but very capable polynomialtime algorithms that are motivated by these dualities. Indeed, the linear programming/matrix scaling algorithm of Khachiyan and Kalantari [6] is also based on a scaling duality, implied by Gordan's theorem (see [3]). Remark. When W = < n, Corollary 2.2 implies that either there exists x 0; x 6= 0, such that Qx = 0, or there exists d > 0 such that Qd > 0; but not both. It is easy to show that this duality is equivalent to Gordan's theorem, see Kalantari []. Gordan's theorem (see Dantzig [2], Schrijver [20]) is the following: given a real m n matrix B, either there exists x 0; x 6= 0, such that Bx = 0, or there exists y such that B T y > 0; but not both. If the latter condition occurs, the vector y induces a hyperplane separating the column vectors of B from the origin. If we let Q = B T B, and Qd > 0, then Bd can be taken to be y. We will discuss the complexity of computing a desired zero, or a separating hyperplane in x 5. Remark 2. When W = < n, Corollary 2.2 implies that either there exists x 0; x 6= 0, such that Qx = 0, or given any > 0, there exists d > 0 such that D e QD e e = ; but not both. When = e, this is the quasi doublystochastic scaling problem, a problem which has been of considerable interest for matrices of nonnegative entries, see e.g. Marshal and Olkin [7], Kalantari []. Another statement implied by this corollary is that > 0 if and only if given any > 0, there exists d > 0, such that D QD e = e. In other words, there exists a diagonal scaling of Q such that e is a xedpoint. Moreover, the latter result is true if and only if there exits a diagonal scaling of Q such that D 2QD 2 =, i.e., itself is a xed point. The case of D e QD e e = e scaling was shown to be polynomially solvable in [6] via a simple analysis, see also [3]. But in fact all other cases considered in this paper, including matrix scaling over a given subspace, are polynomially solvable. However, the polynomialtime solvability of the subspace constrained case is considerably demanding, requiring several important and nontrivial auxiliary results (see x 5). The following theorem will be shown to be a consequence of Theorem 2.. It in particular implies a generalization of the familiar (weighted) arithmeticgeometric mean inequality. Let c 2 < n be arbitrary, and dene (2.2) f (x) = c T x Q n i= xi= i ; g (x) = c T x? nx i= i ln x i Theorem 2.3. The following statements are equivalent: ( 0 ) : c T x > 0, 8 x 2 W \ K, x 6= 0. (2 0 ) : 9 d 2 W \ K such that c T d > 0, and f (x) f (d), 8x 2 W \ K. (3 0 ) : 9 d 2 W \ K such that g (x) g (d), 8x 2 W \ K. (4 0 ) : 9 d 2 W \ K such that P e;d D e c = P e;d. (5 0 ) : 9 d 2 W \ K such that P e;d D e c > 0. (6 0 ) : 9 d 2 W \ K such that P ;d D c = e.
6 6 BAHMAN KALANTARI (7 0 ) : 9 d 2 W \ K such that kp ;d D c? e k < min = minf ; : : :; n g. (8 0 ) : 9 d 2 W \ K such that P ;d D c > 0. (9 0 ) : 9 d 2 W \ K such that P 2 ;dd 2c =, where 2 = ( 2 ; ; 2 n) T. Moreover, (3 0 ), (4 0 ), and (6 0 ) have a unique and common solution d, and d also satises (2 0 ). The following is an obvious but important corollary of Theorem 2.3. Corollary 2.4. (Scaling dualities for linear forms) Either c T x = 0, for some x 2 W \ K, x 6= 0, or condition (i) is true, i = 2 0 ; : : :; 9 0 ; but not both. Remark 3. For i = 2 0, Corollary 2.4 implies that either Karmarkar's canonical LP is solvable, or the potential function f (x) is unbounded from below; but not both. In fact Karmarkar's algorithm, [5], can be viewed as an algorithmic proof of this fact, see also Kalantari [6], [9]. Another duality implied by this corollary is that either Karmarkar's canonical LP is solvable, or there there exists d 2 W \ K such that P e;d D e c > 0, but not both. This duality which implies Gordan's theorem gives rise to a simple variation of Karmarkar's algorithm (see Kalantari [0]), a polynomialtime algorithm that actually establishes this duality. and 3. Proof of Theorem 2.. Proof. We will prove the theorem by proving the following circle of implications () =) (2) =) (3) =) (4) =) (6) =) (7) =) (8) =) (); () () (5); (6) () (9): () =) (2) : Suppose > 0. Consider the set W \K \S, S = fx 2 E : kxk = g. Since X (x) approaches innity as x approaches a boundary point of W \ K \ S, its inmum is attained at some point, say, x. We claim that x must be the minimum of X (x) over W \ K. Otherwise, there exists x 2 W \ K such that X (x) < X (x ). But as X is homogeneous of degree zero, we get X (x=kxk) = X (x) < X (x ), a contradiction. (2) =) (3) : Suppose there exists d 2 W \ K such that (d) > 0, and X (x) X (d), for all x 2 W \ K. From the rst order optimality condition, d must be a stationary point of X, i.e., (3.) P rx (d) = 0; where P = P e;e = I? A T (AA T )? A. Dierentiating X (x), we get (3.2) rx (x) = r(x) + 2 (x)rf (x) exp( 2 F (x)): For any positive real, we have (3.3) r(x) = r(x); rf (x) = rf (x): From (3.), (3.2), and (3.3) we conclude that for any positive real, we have (3.4) P r(d) + 2 (2 (d))( rf (d)) = 0:
7 AGMean Inequality, LP Matrix Scaling, Gordan's Theorem 7 In particular, since (d) > 0, we can let = p =2(d). Then, setting d = d, we have (3.5) P r (d ) = 0: Equivalently, this implies (3.6) r (d ) = A T v; for some vector v of Lagrange multipliers. Since is convex, for all x 2 W \ K we have (3.7) (x) (d ) + r (d ) T (x? d ) = (d ) + v T A(x? d ) = (d ); hence (3) is satised. (3) =) (4) : Suppose d satises (3). Then P r (d) = 0. Equivalently, (3.8) r (d) = QD e e? D? e = A T v; for some v. Multiplying the latter equation by D e, we get (3.9) D e QD e e? = D e A T v: Multiply the above by AD e, we can solve for v to get (3.0) v = (A T D 2 e A)? AD e (D e QD e? ): Substituting the above in the previous equation, we get (3.) P e;d (D e QD e? ) = 0: Thus, (4) is valid. (4) =) (6) : If d satises (4), then it follows that r (d) = A T v, for some v. Equivalently, (3.2) r (d) = QD e? D? e = A T v: Multiplying the latter equation by D, we get (3.3) D QD e? e = D A T v: Multiply the above by AD, we can solve for v to get (3.4) v = (A T D 2 A)? AD (D QD e? e ): Substituting the above in the previous equation, we get (3.5) P ;d (D QD e? e ) = 0: Since AD e = Ad = 0, P ;d e = e. But this implies (6) is valid. (6) =) (7) : This is immediate. (7) =) (8) : Suppose that d satises (7). Assume that a given y 2 < n satises the inequality ky? k < min. Then, jy i? i j < min, for all i = ; : : :; n. Thus, it follows that y > 0. Hence (8) must hold.
8 8 BAHMAN KALANTARI (8) =) () : Suppose that d satises (8), and = 0. Since Q is positive semidenite, there exist x 2 W \ K, x 6= 0, such that (3.6) Qx = 0; Ax = 0: Let w = D? x. Note that w 0, w 6= 0. Since P ;dw = w, and D e = d, we have (3.7) w T P ;d D QD e = w T D QD e = x T Qd: On the one hand, since Qx = 0, we must have x T Qd = 0. On the other hand, since P ;d D QD e > 0, and w 0, w 6= 0, we get x T Qd > 0, a contradiction. Thus, if (8) holds, then so does (). () =) (5) : Since () implies (4), given any 2 K, we can select = e. Then, there exists d 2 W \K such that P e;d D e QD e e = P e;d e = e. Thus, (5) holds. (5) =) () : Suppose that d satises (5). Then, d satises (8) for = e. Hence, > 0. (6) =) (9) : This implication and and its converse are trivial. To prove the remaining claims of the theorem, we need to observe that since is strictly convex, the minimizer d of, if it exists, must be unique. We have already seen that if d satises (4) or (6), then it must satisfy P r (d) = 0. Thus, d is the minimizer of. Hence it satises (3) and is unique. Next we show that d also satises (2). Suppose that d 0 satises (2). Then, P rx (d 0 ) = 0. Since (d 0 ) > 0, some positive scalar multiple of d 0, say d 0 satises P r (d 0 ) = 0. But then we must have d 0 = d. Since X (x) = X(x), for all x 2 W \ K, d also satises (2). 4. Proof of Theorem 2.3. Proof. Assume that ( 0 ) holds. Let Q = cc T. Then, the corresponding is positive. From Theorem 2. we conclude that for each i = 2; : : :; 9, condition (i) is satised. (2) =) (2 0 ) : Since X (x) = f 2 (x), and ct x > 0, for x 2 W \ K, x 6= 0, (2 0 ) follows. (3) =) (3 0 ) : Suppose that d satises (3). Then cc T d? D e? = AT v, for some vector of lagrange multipliers, v. Since = c T d > 0, from the above equation we get c? D? e = A T v, where d = d=, v = v=. But this implies that d is the minimizer of g (x) = c T x + F (x). (4) =) (4 0 ) : Suppose d satises (4), i.e., P e;d D e cc T D e e = P e;d. Since = c T d > 0, ^d = d lies in W \ K. Since P e;d = P e; ^d, we get P e; ^d ^D e c = P e; ^d. Analogously to the above, it follows that for i = 5? 8, if d satises (i), then ^d = (c T d)d satises (i 0 ). (8 0 ) =) ( 0 ) : Follows from similar argument given for proof of the corresponding part of Theorem 2.. (6 0 ) =) (9 0 ) : This and its converse are trivial. The proof of the remaining parts follow in a similar fashion as in Theorem Two polynomialtime algorithms for HP, SP, HSP, and ASP. Here we will describe two polynomialtime algorithms for solving approximate version of these problems. These algorithms are conceptually easy. However, their analysis requires the use of scaling dualities, signicant bounds, as well as some results from Nesterov and Nemirovskii's machinery of selfconcordance. First, we dene  approximate version of the four problems. Next, we describe the two algorithms. We will then state without proof the main ingredients, other than the scaling dualities already proved. Finally, we will justify how the main ingredients result in the stated complexity results. For the proof of the stated theorems and in much more generality, see Kalantari [3].
9 AGMean Inequality, LP Matrix Scaling, Gordan's Theorem 9 Given 2 (0; ], HP, SP, HSP, and ASP are dened as follows. HP is to compute d 2 W \K \S such that (d), or proving that such point does not exist. Given 2 K, ASP is to test the solvability of kp ;d D QD e? e k <. SP is to compute, if it exists, a point d 2 W \ K such that (d)?. HSP is to compute, if it exists, a point d 2 W \ K such that X (d)=x exp(). In particular, if = min, then from Theorem 2. the solution of ASP will result in a point d 2 W \ K such that P ;d D QD e > 0. For the convexhull problem (see Remarks and 3) such a point gives rise to a separating hyperplane. Let u =?P r (d 0 ), where d 0 is a given point in W \ K \ S. For each t 2 (0; ), dene Given d 2 W \ K, let f (t) ;d f (t) (x) = t(x) + tut x + F (x): (t) (x) = f (D x) = t ;d (x) + tu T Dx + F (x) + F (d): Also, given d 2 W \ K, let y be the Newton direction with respect to, i.e., the solution to the equation P r 2 (d)y =?P r (d), where P is the orthogonal projection operator onto W. The Newton decrement is dened as (d) = [y T r 2 (d)y] =2. Let 2 [ ; ), where = 2? p 3. The Newton iterate is d 0 = NEW ( ; d) = d + (d)y; (d) = Analogously, corresponding to f (t) 8 >< >: ; if (d) > +(d) ;?(d) (d)(3?(d)) if (d) 2 [ ; ]; ; if (d) <., one denes y t, t (d), and d 0 t = NEW (f (t) ; d). Consider the following PotentialReduction algorithm: PotentialReduction: Initialization. Let d = d 0. Iterative Step. Replace d with d 0 = NEW ( ; d) and repeat. Theorem 5.. (PotentialReduction Complexity Theorem, Kalantari [3]) Assume that e. Let be in (0; ). Consider the PotentialReduction algorithm, and let = exp[ 2 (d 0 )? + ln ], and 2 R = supfexp(? 2 F (x)) : x 2 W \ K ; kxk = g. If = 0, the number of iterations to solve HP is O( ln R ). If > 0, the number of iterations to solve SP, HSP, or ASP is O( ln R kqk + ln ln ). Consider the following PathFollowing algorithm, where t 2 (0; ) is an appropriately selected number: PathFollowing: Initialization. Let t =, d = d 0. Phase I. While t > t, replace (d; t) with (d 0 t ; t0 ), where d 0 t absolute positive constant (can be taken equal to =9). Phase II. Replace d with d 0 = NEW (f (t) ; d) and repeat. (t) = NEW (f ; d), t0 = t exp(?c = p ), c an Theorem 5.2. (PathFollowing Complexity Theorem, Kalantari [3]) Assume e. Let be in (0; ). Consider the PathFollowing algorithm.
10 0 BAHMAN KALANTARI If = 0, the number of iterations to solve HP is O( p ln kuk iterations to solve ASP is O( p ln kuk is O( p ln kuk + ln ln kqk ). + ln ln kqk). If > 0, the number of + ln ln kqk), and the number of iterations to solve SP or HSP Remark 4. The above algorithms also apply to the case of an arbitrary > 0, not necessarily satisfying e. Essentially all is needed is to replace with 0 = = min. These algorithm can be used to solve linear programs, to compute separating hyperplanes, to diagonally scale positive semidenite symmetric matrices over an arbitrary subspace, or to compute the minimum of the ratio of an arbitrary linear (positive semidenite form) to the geometric mean (square of the geometric mean) over the positive points of an arbitrary subspace. The implementation of the above two algorithms for SP, HSP, and ASP requires the computation of a lower bounds on, assuming that it is positive. For the computation of such a lower bound for the case where Q has rational or algebraic entries, see [2]. These algorithms can also be applied to solve Karmarkar's canonical LP. All is needed is to replace the linear form c T x with (c T x) 2. In particular, taking = e, the PathFollowing algorithm gives an O( p n ln nkuk ) iteration complexity for solving approximate version of Karmarkar's problem. As usual, over the rational inputs can be taken to be O(2?L ), where L is the size of LP (see [2] for dierent notions of size, over the rationals or the algebraic numbers). The PathFollowing algorithm not only results in the bestknown iteration complexity for solving LP (see e.g. [4]), while being one of the simplest of such algorithms, but also is an algorithm that is capable of solving more general diagonal matrix scaling problems than that considered in [6], as well as having the capability of solving SP and HSP. We now explain the main ingredients, other than the scaling dualities already proved, that result in the above two complexity theorems. The following theorem characterizes basic properties of Newton's method (see [9], Theorem 2.2.2), and main properties of the parametric family f (t) (see [9], Theorem 3..): Theorem 5.3. (Nesterov and Nemirovskii [9]) Given d 2 W \ K, d 0 = NEW ( ; d) 2 W \ K, and 8 >< >: (d 0 )? (d)?((d)? ln( + (d))); if (d) > ; (d 0 ) (6(d)? 4 2 (d)? ); if (d) 2 [ ; ]; (d 0 ) 2 (d) ; if (d) < (?(d)) 2. If (d) < 3, then = inff (x) : x 2 W \ K g >? and (d)?!2 ((d))( +!((d)) ;!((d)) =? (? 3(d)) =3 : 2(?!((d)) Given t 2 (0; ), all the above apply to f (t). Moreover, suppose that t(d) =4, then t 0(d) =4, where t 0 = t exp(?c p ). Now we state some other signicant results. Theorem 5.4. (Kalantari [3], x 8) Assume that e. Suppose that d 2 W \K satises (d) < =2. Then, kqk (d) kp ;d r ;d (e )k = O (d) : Suppose that > 0. Given any t 2 (0; ], suppose that d 2 W \ K satises t (d) <. Then, kqk kuk kp ;d rf (t) ;d (e )k = O t (d) :
11 AGMean Inequality, LP Matrix Scaling, Gordan's Theorem In either case ( > 0, or 0), given any t 2 (0; ], suppose that d 2 W \ K is a point obtained via Phase I of the PathFollowing algorithm, and t (d) < =2. Then, kp ;d rf (t) ;d (e )k = O kqk t (d)( t p ) c : Theorem 5.5. (Kalantari [3], x 9) Let the kth iterate of the PotentialReduction algorithm be denoted by d k. If (d k ), then d k kd k R exp? 2k ; =? ln( + ): k Theorem 5.6. (Kalantari [3], x 9) Suppose that > 0. Then, for all x 2 W \ K we have X (x) ln X 2 (x)? : The following theorem is the nal ingredient, a signicant consequence of the scaling dualities: Theorem 5.7. (Kalantari [3], x 0) Let be a number in (0; ]. Given t 2 (0; ], suppose that d 2 W \ K satises kp ;d rf (t) ;d (e )k. Let ^d = p td. If kp 2 ; ^d r ; ^d (e )k, then ( kdk d ) C()t, where C() = [2 + (p + )]kuk 2 : 2 Now we describe how Theorem 2., and Theorems imply the desired complexity theorems, Theorem 5., and Theorem 5.2. We will rst describe this for Theorem 5.. From Theorem 2., HP is solvable (i.e. = 0) if and only if the other three problems are not. If = 0, Theorem 5.5 gives the desired complexity bound to solve HP. If > 0, since (d k =kd k k), Theorem 5.5 implies that in O( ln R ) iterations we obtain a point x 2 W \ K such that (x) <. Let x k be the subsequent iterates of the PotentialReduction algorithm, where x 0 = x. From Theorem 5.3 it follows that (5.) (x k ) r 2k ; r = (? ) 2 : The above together with the rst bound in Theorem 5.4 implies that the number of iterations, k, to get a point x k satisfying ASP, is O(ln ln kqk ). Hence the claimed complexity for solving ASP. From Theorem 5.3 it also follows that (see [3]) once we have obtained a point x k such that (x k ) < minf ; (? 3!( ))g, we have (5.2) (x k )? : This together with the bound in Theorem 5.6 gives the desired complexity for solving SP, or HSP. We will next derive the complexity result of Theorem 5.2. From Theorem 5.3 it follows that given any t 2 (0; ], since (d 0 ) = 0, the number of iterations, k t, to obtain d 2 W \K such that t (d) <, satises (5.3) O p ln ( t ) :
12 2 BAHMAN KALANTARI Suppose that = 0, then from Theorem 2. we must have kp ; ^d r ; ^d (e )k min. Theorem 5.7 it suces to compute d 2 W \ K such that From this, and (5.4) kp ;d rf (t) d (e d )k 2 min; where t satises (5.5) = C( min) : t Thus, the number of iterations of Phase I will equal O( p ln kuk ), and at the termination of this phase we have a point x 2 W \ K such that t (x) <. Let x 0 = x; x ; : : :; x k be the sequence of iterates of Phase II. From Theorem 5.3, it follows that t (x k ) r 2k. From this and the third upper bound given in Theorem 5.4, it follows that the number of iterations of Phase II to solve HP is O(ln ln kqk?p =c ). Hence the claimed combined complexity of the two phases. Suppose that > 0. To solve ASP, from Theorem 5.7 it suces to compute d 2 W \ K such that (5.6) kp ;d rf (t) d (e d )k 2 ; where t satises (5.7) t = 2C() : Thus, the number of iterations of Phase I is O( p ln kuk ). At the termination of this phase we have a point x 2 W \ K such that t (x) <. Let x 0 = x; x ; : : :; x k be the sequence of iterates of Phase II. From the second bound of Theorem 5.4, and since t (x k ) r 2k, the number of iterations of Phase II is O(ln ln kqk kuk ). Hence the claimed combined complexity. To solve SP, or HSP, we rst solve ASP, i.e., compute a point d > 0 such that kp ;d D QD e? e k <, in O( p ln kuk ) iterations. We then replace d with NEW ( ; d), and repeat this step. The number of latter iterations can be estimated as it was done for the PotentialReduction algorithm. 6. Concluding remarks. In this paper we have proved several dualities relating a generalization of the classical arithmetic geometric mean inequality to linear programming, to Gordan's theorem, and to several diagonal matrix scaling problems. These dualities, called scaling dualities, although were derived via elementary means are remarkable and signicant from the theoretical and algorithmic point of view. Many of the scaling dualities can be stated with respect to much more general convex programming problems, relating their equivalent HP formulation to problems that can be viewed as analogues of SP, HSP, and ASP, considered in this paper. The latter three problems are genuinely dual to the HP formulation of convex programming problems, i.e., HP is solvable if and only if the other three are not. Indeed, as shown in [3], given an arbitrary closed convex pointed cone K in a nite dimensional space, a logarithmically homogeneous barrier F (x) for the interior of K, a subspace W, and a homogeneous function with homogeneous degree p > 0, the scaling equation can be dened as P d r d (e d ) = e d, or P d r 2 d (e d ) = (p? )? e d, if p 6= ; where d 2 W \ K, D = r 2 F (d)?=2, d (x) = (Dx), e d = D? d, and P d the orthogonal projection onto W d = D? W. For instance, semidenite programming can be formulated as the problem of testing the existence of a nontrivial zero of tr(cx), over an arbitrary subspace of the space of symmetric matrices, and its intersection with the cone of positive semidenite symmetric matrices, K = S + n, where c is a given symmetric matrix, and tr() is the trace function. Now suppose that given 2 K, we take F (x) =? ln det(x ), where x is dened as follows. Firstly, given a matrix x 2 K, the matrix
13 AGMean Inequality, LP Matrix Scaling, Gordan's Theorem 3 exp(x) = P j= xj =j! is welldened and lies in K. If x is a diagonal positive denite matrix, dene ln x as the matrix diag(ln x ; : : :; lnx n ). If x is an arbitrary positive denite matrix, dene ln x = U x ln x U T x, where x = U x x U T x, with U x unitary, and x the diagonal matrix of eigenvalues of x. Dene x as the matrix exp( =2 ln x =2 ). Now if for a given d 2 K we dene D = r 2 F (d)?=2, and e = I the identity matrix, and D e the operator that maps x 2 S + n to d =2 xd =2, then all the results proved or presented in this paper apply, verbatim. These include all the complexity theorems, and the auxiliary theorems that imply them. The quantity min will stand for the minimum eigenvalue of. These results are the subject of a forthcoming paper, [4]. In particular, in [4] we prove the following inequality which is a generalization of the arithmeticgeometric mean inequality as well as the wellknown tracedeterminant inequality: let be an arbitrary matrix in K, the cone of positive denite symmetric matrices, and = tr(). Then, tr(x) (det(x )) = ; 8x 2 K : REFERENCES [] H. Alzer, A proof of arithmeticgeometric mean inequality, Amer. Math. Monthly, 03 (996) 585. [2] G.B. Dantzig, Linear Programming and Extensions (Princeton University Press, Princeton, New Jersey, 963). [3] P.S. Bullen, D.S. Mitrinovic, and P.M. Vasic, Means and Their Inequalities (Reidel, Dordrecht, 988). [4] C.C. Gonzaga, Pathfollowing methods for linear programming, SIAM Review, 34 (992) [5] G.H. Hardy, J.E. Littlewood, and G. Polya, Inequalities (Cambridge University Press, Cambridge, 952). [6] B. Kalantari, Karmarkar's algorithm with improved steps, Math. Programming, 46 (990) [7] B. Kalantari, Canonical problems for quadratic programming and projective methods for their solution, Contemporary Mathematics, 4 (990) [8] B. Kalantari, Derivation of a generalized and strengthened Gordan theorem from generalized Karmarkar potential and logarithmic barrier functions, Technical Report LCSRTR2, Department of Computer Science, Rutgers University, New Brunswick, NJ, 989. [9] B. Kalantari, Generalization of Karmarkar's algorithm to convex homogeneous functions, Oper. Res. Lett., (992) [0] B. Kalantari, A simple polynomial time algorithm for a convex hull problem equivalent to linear programming, Combinatorics Advances, Kluwer Academic Publishers (995) [] B. Kalantari, A Theorem of the alternative for multihomogeneous functions and its relationship to diagonal scaling of matrices, Linear Algebra and its Applications, 236 (996) 24. [2] B. Kalantari and M.R. EmamyK, On linear programming and matrix scaling over the algebraic numbers, Linear Algebra and its Applications, 262 (997) [3] B. Kalantari, Scaling dualities and selfconcordant homogeneous programming in nite dimensional spaces, Technical Report DCSTR 359, Department of Computer Science, Rutgers University, New Brunswick, NJ, 998. [4] B. Kalantari, On the tracedeterminant inequality, semidenite programming, and matrix scaling over the cone matrices, forthcoming. [5] N. Karmarkar, A new polynomial time algorithm for linear programming, Combinatorica, 4 (984) [6] L. Khachiyan and B. Kalantari, Diagonal matrix scaling and linear programming, SIAM J. Optim., 4 (992) [7] A.W. Marshall and I. Olkin, Scaling of matrices to achieve specied row and column sums, Numer. Math., 2 (968) [8] D.S. Mitrinovic, J.E. Pecaric, and A.M. Fink, Classical and New Inequalities in Analysis (Kluwer Academic Publishers, Dordrecht, 993). [9] Y. Nesterov and A.S. Nemirovskii, InteriorPoint Polynomial Algorithms in Convex Programming (SIAM, Philadelphia, PA, 994). [20] A. Schrijver, Theory of Linear and Integer Programming (John Wiley and Sons, New York, 986).
SCALING DUALITIES AND SELFCONCORDANT HOMOGENEOUS PROGRAMMING IN FINITE DIMENSIONAL SPACES
SCALING DUALITIES AND SELFCONCORDANT HOMOGENEOUS PROGRAMMING IN FINITE DIMENSIONAL SPACES BAHMAN KALANTARI Abstract. In this paper first we prove four fundamental theorems of the alternative, called scaling
More informationLinear Algebra, 4th day, Thursday 7/1/04 REU Info:
Linear Algebra, 4th day, Thursday 7/1/04 REU 004. Info http//people.cs.uchicago.edu/laci/reu04. Instructor Laszlo Babai Scribe Nick Gurski 1 Linear maps We shall study the notion of maps between vector
More informationLinear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) 1.1 The Formal Denition of a Vector Space
Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) Contents 1 Vector Spaces 1 1.1 The Formal Denition of a Vector Space.................................. 1 1.2 Subspaces...................................................
More information1 Introduction Semidenite programming (SDP) has been an active research area following the seminal work of Nesterov and Nemirovski [9] see also Alizad
Quadratic Maximization and Semidenite Relaxation Shuzhong Zhang Econometric Institute Erasmus University P.O. Box 1738 3000 DR Rotterdam The Netherlands email: zhang@few.eur.nl fax: +3110408916 August,
More informationChapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.
Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space
More informationLMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAASCNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009
LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAASCNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix
More information4.1 Eigenvalues, Eigenvectors, and The Characteristic Polynomial
Linear Algebra (part 4): Eigenvalues, Diagonalization, and the Jordan Form (by Evan Dummit, 27, v ) Contents 4 Eigenvalues, Diagonalization, and the Jordan Canonical Form 4 Eigenvalues, Eigenvectors, and
More informationCHAPTER III THE PROOF OF INEQUALITIES
CHAPTER III THE PROOF OF INEQUALITIES In this Chapter, the main purpose is to prove four theorems about HardyLittlewood Pólya Inequality and then gives some examples of their application. We will begin
More informationAssignment 1: From the Definition of Convexity to Helley Theorem
Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x
More informationChapter 1. Preliminaries
Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between
More informationLecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.
MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.
More informationA Finite Element Method for an IllPosed Problem. MartinLutherUniversitat, Fachbereich Mathematik/Informatik,Postfach 8, D Halle, Abstract
A Finite Element Method for an IllPosed Problem W. Lucht MartinLutherUniversitat, Fachbereich Mathematik/Informatik,Postfach 8, D699 Halle, Germany Abstract For an illposed problem which has its origin
More informationKey words. Complementarity set, Lyapunov rank, BishopPhelps cone, Irreducible cone
ON THE IRREDUCIBILITY LYAPUNOV RANK AND AUTOMORPHISMS OF SPECIAL BISHOPPHELPS CONES M. SEETHARAMA GOWDA AND D. TROTT Abstract. Motivated by optimization considerations we consider cones in R n to be called
More information58 Appendix 1 fundamental inconsistent equation (1) can be obtained as a linear combination of the two equations in (2). This clearly implies that the
Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution
More informationAPPROXIMATING THE COMPLEXITY MEASURE OF. Levent Tuncel. November 10, C&O Research Report: 98{51. Abstract
APPROXIMATING THE COMPLEXITY MEASURE OF VAVASISYE ALGORITHM IS NPHARD Levent Tuncel November 0, 998 C&O Research Report: 98{5 Abstract Given an m n integer matrix A of full row rank, we consider the
More informationThe Nearest Doubly Stochastic Matrix to a Real Matrix with the same First Moment
he Nearest Doubly Stochastic Matrix to a Real Matrix with the same First Moment William Glunt 1, homas L. Hayden 2 and Robert Reams 2 1 Department of Mathematics and Computer Science, Austin Peay State
More informationA Generalized Homogeneous and SelfDual Algorithm. for Linear Programming. February 1994 (revised December 1994)
A Generalized Homogeneous and SelfDual Algorithm for Linear Programming Xiaojie Xu Yinyu Ye y February 994 (revised December 994) Abstract: A generalized homogeneous and selfdual (HSD) infeasibleinteriorpoint
More informationPermutation invariant proper polyhedral cones and their Lyapunov rank
Permutation invariant proper polyhedral cones and their Lyapunov rank Juyoung Jeong Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland 21250, USA juyoung1@umbc.edu
More informationA BASIC FAMILY OF ITERATION FUNCTIONS FOR POLYNOMIAL ROOT FINDING AND ITS CHARACTERIZATIONS. Bahman Kalantari 3. Department of Computer Science
A BASIC FAMILY OF ITERATION FUNCTIONS FOR POLYNOMIAL ROOT FINDING AND ITS CHARACTERIZATIONS Bahman Kalantari 3 Department of Computer Science Rutgers University, New Brunswick, NJ 08903 Iraj Kalantari
More informationAbsolute value equations
Linear Algebra and its Applications 419 (2006) 359 367 www.elsevier.com/locate/laa Absolute value equations O.L. Mangasarian, R.R. Meyer Computer Sciences Department, University of Wisconsin, 1210 West
More informationContents. 2.1 Vectors in R n. Linear Algebra (part 2) : Vector Spaces (by Evan Dummit, 2017, v. 2.50) 2 Vector Spaces
Linear Algebra (part 2) : Vector Spaces (by Evan Dummit, 2017, v 250) Contents 2 Vector Spaces 1 21 Vectors in R n 1 22 The Formal Denition of a Vector Space 4 23 Subspaces 6 24 Linear Combinations and
More informationDSGA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.
DSGA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1
More informationRobust linear optimization under general norms
Operations Research Letters 3 (004) 50 56 Operations Research Letters www.elsevier.com/locate/dsw Robust linear optimization under general norms Dimitris Bertsimas a; ;, Dessislava Pachamanova b, Melvyn
More informationInequality Constraints
Chapter 2 Inequality Constraints 2.1 Optimality Conditions Early in multivariate calculus we learn the significance of differentiability in finding minimizers. In this section we begin our study of the
More informationLinear Algebra 1 Exam 2 Solutions 7/14/3
Linear Algebra 1 Exam Solutions 7/14/3 Question 1 The line L has the symmetric equation: x 1 = y + 3 The line M has the parametric equation: = z 4. [x, y, z] = [ 4, 10, 5] + s[10, 7, ]. The line N is perpendicular
More informationELEMENTARY LINEAR ALGEBRA
ELEMENTARY LINEAR ALGEBRA K R MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND First Printing, 99 Chapter LINEAR EQUATIONS Introduction to linear equations A linear equation in n unknowns x,
More informationCHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.
1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function
More informationIntroduction to Nonlinear Stochastic Programming
School of Mathematics T H E U N I V E R S I T Y O H F R G E D I N B U Introduction to Nonlinear Stochastic Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio SPS
More informationELEMENTARY LINEAR ALGEBRA
ELEMENTARY LINEAR ALGEBRA K. R. MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND Second Online Version, December 1998 Comments to the author at krm@maths.uq.edu.au Contents 1 LINEAR EQUATIONS
More informationApproximation Algorithms for Maximum. Coverage and Max Cut with Given Sizes of. Parts? A. A. Ageev and M. I. Sviridenko
Approximation Algorithms for Maximum Coverage and Max Cut with Given Sizes of Parts? A. A. Ageev and M. I. Sviridenko Sobolev Institute of Mathematics pr. Koptyuga 4, 630090, Novosibirsk, Russia fageev,svirg@math.nsc.ru
More informationThe Simplest Semidefinite Programs are Trivial
The Simplest Semidefinite Programs are Trivial Robert J. Vanderbei Bing Yang Program in Statistics & Operations Research Princeton University Princeton, NJ 08544 January 10, 1994 Technical Report SOR9312
More informationy Ray of Halfline or ray through in the direction of y
Chapter LINEAR COMPLEMENTARITY PROBLEM, ITS GEOMETRY, AND APPLICATIONS. THE LINEAR COMPLEMENTARITY PROBLEM AND ITS GEOMETRY The Linear Complementarity Problem (abbreviated as LCP) is a general problem
More informationOn well definedness of the Central Path
On well definedness of the Central Path L.M.Graña Drummond B. F. Svaiter IMPAInstituto de Matemática Pura e Aplicada Estrada Dona Castorina 110, Jardim Botânico, Rio de JaneiroRJ CEP 22460320 Brasil
More informationLinear Algebra: Characteristic Value Problem
Linear Algebra: Characteristic Value Problem . The Characteristic Value Problem Let < be the set of real numbers and { be the set of complex numbers. Given an n n real matrix A; does there exist a number
More informationON SUM OF SQUARES DECOMPOSITION FOR A BIQUADRATIC MATRIX FUNCTION
Annales Univ. Sci. Budapest., Sect. Comp. 33 (2010) 273284 ON SUM OF SQUARES DECOMPOSITION FOR A BIQUADRATIC MATRIX FUNCTION L. László (Budapest, Hungary) Dedicated to Professor Ferenc Schipp on his 70th
More informationLecture 5: Hyperbolic polynomials
Lecture 5: Hyperbolic polynomials Robin Pemantle University of Pennsylvania pemantle@math.upenn.edu Minerva Lectures at Columbia University 16 November, 2016 Ubiquity of zeros of polynomials The one contribution
More informationMAT Linear Algebra Collection of sample exams
MAT 342  Linear Algebra Collection of sample exams Ax. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system
More informationSupplement: Universal SelfConcordant Barrier Functions
IE 8534 1 Supplement: Universal SelfConcordant Barrier Functions IE 8534 2 Recall that a selfconcordant barrier function for K is a barrier function satisfying 3 F (x)[h, h, h] 2( 2 F (x)[h, h]) 3/2,
More informationPrimalDual InteriorPoint Methods for Linear Programming based on Newton s Method
PrimalDual InteriorPoint Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach
More informationRankone LMIs and Lyapunov's Inequality. Gjerrit Meinsma 4. Abstract. We describe a new proof of the wellknown Lyapunov's matrix inequality about
Rankone LMIs and Lyapunov's Inequality Didier Henrion 1;; Gjerrit Meinsma Abstract We describe a new proof of the wellknown Lyapunov's matrix inequality about the location of the eigenvalues of a matrix
More informationScaling of Symmetric Matrices by Positive Diagonal Congruence
Scaling of Symmetric Matrices by Positive Diagonal Congruence Abstract Charles R. Johnson a and Robert Reams b a Department of Mathematics, College of William and Mary, P.O. Box 8795, Williamsburg, VA
More informationOn selfconcordant barriers for generalized power cones
On selfconcordant barriers for generalized power cones Scott Roy Lin Xiao January 30, 2018 Abstract In the study of interiorpoint methods for nonsymmetric conic optimization and their applications, Nesterov
More informationMotakuri Ramana y Levent Tuncel and Henry Wolkowicz z. University of Waterloo. Faculty of Mathematics. Waterloo, Ontario N2L 3G1, Canada.
STRONG DUALITY FOR SEMIDEFINITE PROGRAMMING Motakuri Ramana y Levent Tuncel and Henry Wolkowicz z University of Waterloo Department of Combinatorics and Optimization Faculty of Mathematics Waterloo, Ontario
More information16 Chapter 3. Separation Properties, Principal Pivot Transforms, Classes... for all j 2 J is said to be a subcomplementary vector of variables for (3.
Chapter 3 SEPARATION PROPERTIES, PRINCIPAL PIVOT TRANSFORMS, CLASSES OF MATRICES In this chapter we present the basic mathematical results on the LCP. Many of these results are used in later chapters to
More informationLecture 5. Theorems of Alternatives and SelfDual Embedding
IE 8534 1 Lecture 5. Theorems of Alternatives and SelfDual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c
More informationA matrix over a field F is a rectangular array of elements from F. The symbol
Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F ) denotes the collection of all m n matrices over F Matrices will usually be denoted
More informationCopositive matrices and periodic dynamical systems
Extreme copositive matrices and periodic dynamical systems Weierstrass Institute (WIAS), Berlin Optimization without borders Dedicated to Yuri Nesterovs 60th birthday February 11, 2016 and periodic dynamical
More informationMay 9, 2014 MATH 408 MIDTERM EXAM OUTLINE. Sample Questions
May 9, 24 MATH 48 MIDTERM EXAM OUTLINE This exam will consist of two parts and each part will have multipart questions. Each of the 6 questions is worth 5 points for a total of points. The two part of
More information2. Dual space is essential for the concept of gradient which, in turn, leads to the variational analysis of Lagrange multipliers.
Chapter 3 Duality in Banach Space Modern optimization theory largely centers around the interplay of a normed vector space and its corresponding dual. The notion of duality is important for the following
More informationSemidefinite Programming
Chapter 2 Semidefinite Programming 2.0.1 Semidefinite programming (SDP) Given C M n, A i M n, i = 1, 2,..., m, and b R m, the semidefinite programming problem is to find a matrix X M n for the optimization
More informationWHEN DOES THE POSITIVE SEMIDEFINITENESS CONSTRAINT HELP IN LIFTING PROCEDURES?
MATHEMATICS OF OPERATIONS RESEARCH Vol. 6, No. 4, November 00, pp. 796 85 Printed in U.S.A. WHEN DOES THE POSITIVE SEMIDEFINITENESS CONSTRAINT HELP IN LIFTING PROCEDURES? MICHEL X. GOEMANS and LEVENT TUNÇEL
More informationAn Alternative Proof of Primitivity of Indecomposable Nonnegative Matrices with a Positive Trace
An Alternative Proof of Primitivity of Indecomposable Nonnegative Matrices with a Positive Trace Takao Fujimoto Abstract. This research memorandum is aimed at presenting an alternative proof to a well
More informationLargest dual ellipsoids inscribed in dual cones
Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that
More informationIn English, this means that if we travel on a straight line between any two points in C, then we never leave C.
Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from
More informationLecture 15 Newton Method and SelfConcordance. October 23, 2008
Newton Method and SelfConcordance October 23, 2008 Outline Lecture 15 Selfconcordance Notion Selfconcordant Functions Operations Preserving Selfconcordance Properties of Selfconcordant Functions Implications
More information5 Eigenvalues and Diagonalization
Linear Algebra (part 5): Eigenvalues and Diagonalization (by Evan Dummit, 27, v 5) Contents 5 Eigenvalues and Diagonalization 5 Eigenvalues, Eigenvectors, and The Characteristic Polynomial 5 Eigenvalues
More informationMATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products.
MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products. Orthogonal projection Theorem 1 Let V be a subspace of R n. Then any vector x R n is uniquely represented
More informationThe Pacic Institute for the Mathematical Sciences http://www.pims.math.ca pims@pims.math.ca Surprise Maximization D. Borwein Department of Mathematics University of Western Ontario London, Ontario, Canada
More informationOptimization: InteriorPoint Methods and. January,1995 USA. and Cooperative Research Centre for Robust and Adaptive Systems.
Innite Dimensional Quadratic Optimization: InteriorPoint Methods and Control Applications January,995 Leonid Faybusovich John B. Moore y Department of Mathematics University of Notre Dame Mail Distribution
More informationCOURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion
COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F
More informationThe maximal stable set problem : Copositive programming and Semidefinite Relaxations
The maximal stable set problem : Copositive programming and Semidefinite Relaxations Kartik Krishnan Department of Mathematical Sciences Rensselaer Polytechnic Institute Troy, NY 12180 USA kartis@rpi.edu
More informationVector Space Basics. 1 Abstract Vector Spaces. 1. (commutativity of vector addition) u + v = v + u. 2. (associativity of vector addition)
Vector Space Basics (Remark: these notes are highly formal and may be a useful reference to some students however I am also posting Ray Heitmann's notes to Canvas for students interested in a direct computational
More informationINDEFINITE TRUST REGION SUBPROBLEMS AND NONSYMMETRIC EIGENVALUE PERTURBATIONS. Ronald J. Stern. Concordia University
INDEFINITE TRUST REGION SUBPROBLEMS AND NONSYMMETRIC EIGENVALUE PERTURBATIONS Ronald J. Stern Concordia University Department of Mathematics and Statistics Montreal, Quebec H4B 1R6, Canada and Henry Wolkowicz
More informationMath 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.
Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,
More informationCSCI 1951G Optimization Methods in Finance Part 01: Linear Programming
CSCI 1951G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cashflow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150
More informationSpectral inequalities and equalities involving products of matrices
Spectral inequalities and equalities involving products of matrices ChiKwong Li 1 Department of Mathematics, College of William & Mary, Williamsburg, Virginia 23187 (ckli@math.wm.edu) YiuTung Poon Department
More information1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations
The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear
More informationIntroduction to Semidefinite Programming I: Basic properties a
Introduction to Semidefinite Programming I: Basic properties and variations on the GoemansWilliamson approximation algorithm for maxcut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite
More informationOn the ChvatálComplexity of Binary Knapsack Problems. Gergely Kovács 1 Béla Vizvári College for Modern Business Studies, Hungary
On the ChvatálComplexity of Binary Knapsack Problems Gergely Kovács 1 Béla Vizvári 2 1 College for Modern Business Studies, Hungary 2 Eastern Mediterranean University, TRNC 2009. 1 Chvátal Cut and Complexity
More informationMore FirstOrder Optimization Algorithms
More FirstOrder Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM
More informationSQUARE ROOTS OF 2x2 MATRICES 1. Sam Northshield SUNYPlattsburgh
SQUARE ROOTS OF x MATRICES Sam Northshield SUNYPlattsburgh INTRODUCTION A B What is the square root of a matrix such as? It is not, in general, A B C D C D This is easy to see since the upper left entry
More informationL. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones. definition. spectral decomposition. quadratic representation. logdet barrier 181
L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation logdet barrier 181 Introduction This lecture: theoretical properties of the following
More informationConvex analysis on Cartan subspaces
Nonlinear Analysis 42 (2000) 813 820 www.elsevier.nl/locate/na Convex analysis on Cartan subspaces A.S. Lewis ;1 Department of Combinatorics and Optimization, University of Waterloo, Waterloo, Ontario,
More information1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)
Notes for 20170130 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty
More informationCONCENTRATION OF THE MIXED DISCRIMINANT OF WELLCONDITIONED MATRICES. Alexander Barvinok
CONCENTRATION OF THE MIXED DISCRIMINANT OF WELLCONDITIONED MATRICES Alexander Barvinok Abstract. We call an ntuple Q 1,...,Q n of positive definite n n real matrices αconditioned for some α 1 if for
More informationOn John type ellipsoids
On John type ellipsoids B. Klartag Tel Aviv University Abstract Given an arbitrary convex symmetric body K R n, we construct a natural and nontrivial continuous map u K which associates ellipsoids to
More informationInterior Point Methods: SecondOrder Cone Programming and Semidefinite Programming
School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: SecondOrder Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio
More informationAn E cient A nescaling Algorithm for Hyperbolic Programming
An E cient A nescaling Algorithm for Hyperbolic Programming Jim Renegar joint work with Mutiara Sondjaja 1 Euclidean space A homogeneous polynomial p : E!R is hyperbolic if there is a vector e 2E such
More informationDO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO
QUESTION BOOKLET EECS 227A Fall 2009 Midterm Tuesday, Ocotober 20, 11:1012:30pm DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO You have 80 minutes to complete the midterm. The midterm consists
More information[3] (b) Find a reduced rowechelon matrix rowequivalent to ,1 2 2
MATH Key for sample nal exam, August 998 []. (a) Dene the term \reduced rowechelon matrix". A matrix is reduced rowechelon if the following conditions are satised. every zero row lies below every nonzero
More informationSpecial Classes of Fuzzy Integer Programming Models with AllDierent Constraints
Transaction E: Industrial Engineering Vol. 16, No. 1, pp. 1{10 c Sharif University of Technology, June 2009 Special Classes of Fuzzy Integer Programming Models with AllDierent Constraints Abstract. K.
More informationNonlinear Optimization: What s important?
Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global
More informationChapter 1: Systems of Linear Equations
Chapter : Systems of Linear Equations February, 9 Systems of linear equations Linear systems Lecture A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where
More informationA Criterion for the Stochasticity of Matrices with Specified Order Relations
Rend. Istit. Mat. Univ. Trieste Vol. XL, 55 64 (2009) A Criterion for the Stochasticity of Matrices with Specified Order Relations Luca Bortolussi and Andrea Sgarro Abstract. We tackle the following problem:
More informationA priori bounds on the condition numbers in interiorpoint methods
A priori bounds on the condition numbers in interiorpoint methods Florian Jarre, Mathematisches Institut, HeinrichHeine Universität Düsseldorf, Germany. Abstract Interiorpoint methods are known to be
More informationContents. 4 Arithmetic and Unique Factorization in Integral Domains. 4.1 Euclidean Domains and Principal Ideal Domains
Ring Theory (part 4): Arithmetic and Unique Factorization in Integral Domains (by Evan Dummit, 018, v. 1.00) Contents 4 Arithmetic and Unique Factorization in Integral Domains 1 4.1 Euclidean Domains and
More informationIMC 2015, Blagoevgrad, Bulgaria
IMC 05, Blagoevgrad, Bulgaria Day, July 9, 05 Problem. For any integer n and two n n matrices with real entries, B that satisfy the equation + B ( + B prove that det( det(b. Does the same conclusion follow
More informationLecture 9 Monotone VIs/CPs Properties of cones and some existence results. October 6, 2008
Lecture 9 Monotone VIs/CPs Properties of cones and some existence results October 6, 2008 Outline Properties of cones Existence results for monotone CPs/VIs Polyhedrality of solution sets Game theory:
More informationLinear Algebra. Min Yan
Linear Algebra Min Yan January 2, 2018 2 Contents 1 Vector Space 7 1.1 Definition................................. 7 1.1.1 Axioms of Vector Space..................... 7 1.1.2 Consequence of Axiom......................
More informationPrimaldual relationship between LevenbergMarquardt and central trajectories for linearly constrained convex optimization
Primaldual relationship between LevenbergMarquardt and central trajectories for linearly constrained convex optimization Roger Behling a, Clovis Gonzaga b and Gabriel Haeser c March 21, 2013 a Department
More informationa 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2.
Chapter 1 LINEAR EQUATIONS 11 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,, a n, b are given real
More informationNonsymmetric potentialreduction methods for general cones
CORE DISCUSSION PAPER 2006/34 Nonsymmetric potentialreduction methods for general cones Yu. Nesterov March 28, 2006 Abstract In this paper we propose two new nonsymmetric primaldual potentialreduction
More informationChapter 3 Least Squares Solution of y = A x 3.1 Introduction We turn to a problem that is dual to the overconstrained estimation problems considered s
Lectures on Dynamic Systems and Control Mohammed Dahleh Munther A. Dahleh George Verghese Department of Electrical Engineering and Computer Science Massachuasetts Institute of Technology 1 1 c Chapter
More informationMathematical Olympiad Training Polynomials
Mathematical Olympiad Training Polynomials Definition A polynomial over a ring R(Z, Q, R, C) in x is an expression of the form p(x) = a n x n + a n 1 x n 1 + + a 1 x + a 0, a i R, for 0 i n. If a n 0,
More informationA PREDICTORCORRECTOR PATHFOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE
Yugoslav Journal of Operations Research 24 (2014) Number 1, 3551 DOI: 10.2298/YJOR120904016K A PREDICTORCORRECTOR PATHFOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE BEHROUZ
More informationMath 341: Convex Geometry. Xi Chen
Math 341: Convex Geometry Xi Chen 479 Central Academic Building, University of Alberta, Edmonton, Alberta T6G 2G1, CANADA Email address: xichen@math.ualberta.ca CHAPTER 1 Basics 1. Euclidean Geometry
More informationUsing Schur Complement Theorem to prove convexity of some SOCfunctions
Journal of Nonlinear and Convex Analysis, vol. 13, no. 3, pp. 41431, 01 Using Schur Complement Theorem to prove convexity of some SOCfunctions JeinShan Chen 1 Department of Mathematics National Taiwan
More informationExtreme Abridgment of Boyd and Vandenberghe s Convex Optimization
Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very wellwritten and a pleasure to read. The
More informationExercise Solutions to Functional Analysis
Exercise Solutions to Functional Analysis Note: References refer to M. Schechter, Principles of Functional Analysis Exersize that. Let φ,..., φ n be an orthonormal set in a Hilbert space H. Show n f n
More informationMath Introduction to Numerical Analysis  Class Notes. Fernando Guevara Vasquez. Version Date: January 17, 2012.
Math 5620  Introduction to Numerical Analysis  Class Notes Fernando Guevara Vasquez Version 1990. Date: January 17, 2012. 3 Contents 1. Disclaimer 4 Chapter 1. Iterative methods for solving linear systems
More information