# ON THE ARITHMETIC-GEOMETRIC MEAN INEQUALITY AND ITS RELATIONSHIP TO LINEAR PROGRAMMING, BAHMAN KALANTARI

Size: px
Start display at page:

Download "ON THE ARITHMETIC-GEOMETRIC MEAN INEQUALITY AND ITS RELATIONSHIP TO LINEAR PROGRAMMING, BAHMAN KALANTARI"

## Transcription

1 ON THE ARITHMETIC-GEOMETRIC MEAN INEQUALITY AND ITS RELATIONSHIP TO LINEAR PROGRAMMING, MATRIX SCALING, AND GORDAN'S THEOREM BAHMAN KALANTARI Abstract. It is a classical inequality that the minimum of the ratio of the (weighted) arithmetic mean to the geometric mean of a set of positive variables is equal to one, and is attained at the center of the positivity cone. While there are numerous proofs of this fundamental homogeneous inequality, in the presence of an arbitrary subspace, and/or the replacement of the arithmetic mean with an arbitrary linear form, the new minimization is a nontrivial problem. We prove a generalization of this inequality, also relating it to linear programming, to the diagonal matrix scaling problem, as well as to Gordan's theorem. Linear programming is equivalent to the search for a nontrivial zero of a linear or positive semidenite quadratic form over the nonnegative points of a given subspace. The goal of this paper is to present these intricate, surprising, and signicant relationships, called scaling dualities, and via an elementary proof. Also, to introduce two conceptually simple polynomialtime algorithms that are based on the scaling dualities, signicant bounds, as well as Nesterov and Nemirovskii's machinery of self-concordance. The algorithms are simultaneously applicable to linear programming, to the computation of a separating hyperplane, to the diagonal matrix scaling problem, and to the minimization of the arithmetic-geometric mean ratio over the positive points of an arbitrary subspace. The scaling dualities, the bounds, and the algorithms are special cases of a much more general theory on convex programming, developed by the author. For instance, via the scaling dualities semidenite programming is a problem dual to a generalization of the classical trace-determinant ratio minimization over the positive denite points of a given subspace of the Hilbert space of symmetric matrices. Key words. Convexity. Arithmetic-geometric mean inequality, Linear programming, Gordan's theorem, Diagonal matrix scaling, Department of Computer Science, Rutgers University, New Brunswick, NJ

2 2 BAHMAN KALANTARI. Introduction. The classical (weighted) arithmetic-geometric mean inequality can be viewed as a statement on the minimization of the ratio of the two (weighted) means over the positive orthant: the minimum is attained at the center of this cone, any positive scalar multiple of the vector of ones. While there are numerous proofs of this fundamental homogeneous inequality, see e.g. Hardy, Littlewood, and Polya [5], Bullen, Mitrinovic, and Vasic [3], Mitrinovic, Pecaric, and Fink [8], and Alzer []; in the presence of a proper subspace not containing the center, the new minimization is a nontrivial problem. A more general version of the above problem is the minimization of the ratio of an arbitrary linear function to the geometric mean, over the positive points of an arbitrary subspace. Still an even more general version is when the ratio is replaced with that of an arbitrary positive semidenite quadratic form, to the square of the geometric mean. On the other hand linear programming can be formulated as the problem of computing a nontrivial zero of a linear function over the intersection of the nonnegative orthant and a given subspace of the Euclidean space, or proving that such zero does not exist. This problem is in fact Karmarkar's canonical linear programming [5]. A more general, more interesting, and more important problem than Karmarkar's canonical LP problem is when the linear form is replaced with a positive semidenite quadratic form, considered in Kalantari [7]. Indeed, as proved in [7], by removing the positive semideniteness assumption, the zero-nding problem becomes NP-complete. Linear programming is also equivalent to the problem of testing if the convex-hull of a given set of points in the Euclidean space contains the origin. A more extended version of this problem is the following: determine if the convex-hull of a given set of points contains the origin; otherwise, compute a separating hyperplane. In fact the latter problem calls for an algorithmic proof of Gordan's theorem. Another problem of interest is the diagonal matrix scaling problem. Given a positive semidenite symmetric matrix, this is the problem of computing a positive denite diagonal matrix such that pre and post multiplication of the given matrix by the computed diagonal matrix results in a matrix whose row sums (hence column sums) equal prescribed positive numbers; or proving that such diagonal matrix does not exist. The case of diagonal scaling of matrices with nonnegative entries, not necessarily positive semidenite, has been a problem of interest for a very long time. The matrix scaling problem can also be stated in the presence of a subspace. In this more general case of the problem, the diagonal matrix is replaced with the product of a diagonally scaled orthogonal projection matrix, and a positive denite diagonal matrix induced by a positive point of the underlying subspace. A diagonal scaling problem can also be dened for the case of linear forms over a given subspace. This turns out to be a problem dual to Karmarkar's canonical LP. The goal of this paper is to present the intricate, surprising, and signicant relationships between the above stated problems, in the form dualities, called scaling dualities, and via an elementary proof. Also, to introduce two conceptually simple polynomial-time algorithms that are based on the scaling dualities, several signicant bounds, as well as Nesterov and Nemirovskii's machinery of self-concordance. The algorithms are simultaneously applicable to linear programming, to the computation of a separating hyperplane, to the diagonal matrix scaling problem, and to the minimization of the arithmetic-geometric mean ratio over the positive points of a given subspace. The scaling dualities, the bounds, and the algorithms are special cases of a much more general theory on convex programming, developed by the author (see Kalantari [3]). For instance, via the scaling dualities semidenite programming is a problem dual to a generalization of the classical trace-determinant ratio minimization over the positive denite points of a given subspace of the Hilbert space of symmetric matrices. Indeed, all the theorems to be stated in this paper, as well as the two algorithms can be shown to hold for analogous problems dened with respect to the cone of positive semidenite symmetric matrices. Although the proof of general scaling dualities is nontrivial, the proof of those presented in this paper are established via elementary means. The simplicity of the proofs is due

3 AG-Mean Inequality, LP Matrix Scaling, Gordan's Theorem 3 to the very special nature of the underlying problems, i.e., linearity or quadraticity of the homogeneous objective function, as well as symmetric properties of the underlying cone, the nonnegative orthant. In x 2, we preset the main theorems. In x 3 and x 4, we present their proofs. In x 5, we describe the two algorithms, state their complexities, the main ingredients that result in the algorithms, and their application in the derivation of the claimed complexity results. One of the major ingredient is the scaling dualities. Other ingredients are signicant bounding theorems that will be stated without proof. The proof of these bounds is given in [3] and in much more generality. Another ingredient is a theorem that combines two fundamental but most basic properties from Nesterov and Nemirovskii's theory on self-concordance, [9]. In our concluding remarks, x 6, we briey describe generalization of the theorems and the algorithms, in particular with respect to corresponding problems dened over the cone of positive semidenite matrices. 2. The arithmetic-geometric mean inequality, linear programming, Gordan's separation theorem, and matrix scaling. Let K = fx 2 < n : x 0g, the nonnegative orthant, and P K = fx 2 < n : x > 0g, the positive orthant. Let 2 K n be an arbitrary vector of weights, and = i= i. The classical (weighted) arithmetic-geometric mean inequality is the following (2.) T x Qn i= x i i = ; 8x 2 K : Moreover, the minimum is attained at any positive scalar multiple of e = (; : : :; ) T. Let W be a subspace of < n. If W is a proper subspace, we will assume that W = fx 2 < n : Ax = 0g, where A is a given m n matrix of rank m. Let S = fx 2 < n : kxk = g. We will assume that d 0 2 W \ K \ S is available. Let (x) = 2 xt Qx, where Q is an n n symmetric matrix, also assumed to be positive semidenite, i.e., x T Qx 0, for all x 2 < n. Let (2.2) F (x) =? nx i= i ln x i : Linear programming is equivalent to the problem of testing if has a nontrivial zero over W \ K, see Kalantari [7]. Indeed, the latter problem is more general and more important than Karmarkar's canonical linear programming problem [5], which is the following problem: given c 2 < n, determine if c T x = 0, for some x 2 W \ K, x 6= 0. It is easy to see that has a desired zero if and only if = 0, where (2.3) = minf(x) : x 2 W \ K \ Sg: We shall refer to the problem of testing if = 0 as homogeneous programming (HP). Let the logarithmic potential, and the homogeneous potential be dened as (2.4) respectively. Let (2.5) (x) = (x) + F (x); X (x) = (x) exp(? 2 F (x)) = (x) n i= x2 i= i = inff (x) : x 2 W \ K g; X = inffx (x) : x 2 W \ K g: ; The scaling problem (SP) is to determine if =?, and if is nite, to compute its value, together with a corresponding minimizer d. The Homogeneous scaling problem (HSP) is to determine if X =?, and if X is nite, to compute its value, together with a corresponding minimizer d. Since (x) is convex and F (x) is strictly convex, (x) is strictly convex.

4 4 BAHMAN KALANTARI For a given d 2 K, let (2.6) D = r 2 F (d)?=2 = diag( d d p ; ; p n ); n (2.7) e = D? d = (p ; ; p n ) T ; (2.8) ;d (x) = (D x) = 2 xt D QD x; (2.9) ;d(x) = (D x) = ;d (x) + F (x) + F (d): Let P ;d be the orthogonal projection operator onto the subspace W ;d = D? W = fx 2 <n : AD x = 0g. Thus, if W = < n, then P ;d = I, the identity matrix. Otherwise, (2.0) P ;d = I? D A T (AD 2 AT )? AD : The algebraic scaling problem (ASP) is to test the solvability of the scaling equation (SE) : (2.) P ;d r ;d (e ) = P ;d D QD e = e ; d 2 W \ K : Theorem 2.. The following statements are equivalent: () : > 0. (2) : 9 d 2 W \ K such that (d) > 0, and X (x) X (d), 8 x 2 W \ K. (3) : 9 d 2 W \ K such that (x) (d), 8 x 2 W \ K. (4) : 9 d 2 W \ K such that P e;d D e QD e e = P e;d. (5) : 9 d 2 W \ K such that P e;d D e QD e e > 0. (6) : 9 d 2 W \ K such that P ;d D QD e = e. (7) : 9 d 2 W \ K such that kp ;d D QD e? e k < min = minf ; : : :; n g. (8) : 9 d 2 W \ K such that P ;d D QD e > 0. (9) : 9 d 2 W \ K such that P 2 ;dd 2QD 2 =, where 2 = ( 2 ; ; 2 n) T. Moreover, (3), (4), and (6) have a unique and common solution d, and d also satises (2). The following is an obvious but important corollary of Theorem 2.. Corollary 2.2. (Scaling dualities for positive semidenite quadratic forms) Either = 0, or condition (i) is true, i = 2; : : :; 9; but not both. The equivalence of the statements of Theorem 2. justies the naming of the corresponding dualities of Corollary 2.2 as scaling dualities. From the algorithmic point of view it turns out that the dualities that concern diagonal scaling are the more important dualities. Indeed, Theorem 2. can be stated in more generality, where (x) can be an arbitrary convex homogeneous function. The corresponding proof however becomes more involved than the simple proofs presented in this paper. For the proof of some of the corresponding parts of Theorem 2., as derived for general

5 AG-Mean Inequality, LP Matrix Scaling, Gordan's Theorem 5 convex homogeneous functions, see Kalantari []. We mention that linearly constrained convex quadratic programming can be formulated as the problem of computing a nontrivial root of the homogeneous function 2 xt Qx=e T x + c T x over W \ K, for some positive semidenite Q, and c 2 < n, see Kalantari [3]. In fact, most of the scaling dualities implied by Theorem 2. can be stated in much more generality, where the underlying cone is an arbitrary closed convex pointed cone, and an arbitrary convex form dened over this cone (see x 5). Our study of what we now call scaling dualities, and their algorithmic signicance began in Kalantari [6], and was continued in [7]-[]. We emphasize that the goal of the present paper is to present these signicant dualities for the very special problems considered, and via an elementary proof. Also, to introduce two conceptually simple, but very capable polynomial-time algorithms that are motivated by these dualities. Indeed, the linear programming/matrix scaling algorithm of Khachiyan and Kalantari [6] is also based on a scaling duality, implied by Gordan's theorem (see [3]). Remark. When W = < n, Corollary 2.2 implies that either there exists x 0; x 6= 0, such that Qx = 0, or there exists d > 0 such that Qd > 0; but not both. It is easy to show that this duality is equivalent to Gordan's theorem, see Kalantari []. Gordan's theorem (see Dantzig [2], Schrijver [20]) is the following: given a real m n matrix B, either there exists x 0; x 6= 0, such that Bx = 0, or there exists y such that B T y > 0; but not both. If the latter condition occurs, the vector y induces a hyperplane separating the column vectors of B from the origin. If we let Q = B T B, and Qd > 0, then Bd can be taken to be y. We will discuss the complexity of computing a desired zero, or a separating hyperplane in x 5. Remark 2. When W = < n, Corollary 2.2 implies that either there exists x 0; x 6= 0, such that Qx = 0, or given any > 0, there exists d > 0 such that D e QD e e = ; but not both. When = e, this is the quasi doubly-stochastic scaling problem, a problem which has been of considerable interest for matrices of nonnegative entries, see e.g. Marshal and Olkin [7], Kalantari []. Another statement implied by this corollary is that > 0 if and only if given any > 0, there exists d > 0, such that D QD e = e. In other words, there exists a diagonal scaling of Q such that e is a xed-point. Moreover, the latter result is true if and only if there exits a diagonal scaling of Q such that D 2QD 2 =, i.e., itself is a xed point. The case of D e QD e e = e scaling was shown to be polynomially solvable in [6] via a simple analysis, see also [3]. But in fact all other cases considered in this paper, including matrix scaling over a given subspace, are polynomially solvable. However, the polynomial-time solvability of the subspace constrained case is considerably demanding, requiring several important and nontrivial auxiliary results (see x 5). The following theorem will be shown to be a consequence of Theorem 2.. It in particular implies a generalization of the familiar (weighted) arithmetic-geometric mean inequality. Let c 2 < n be arbitrary, and dene (2.2) f (x) = c T x Q n i= xi= i ; g (x) = c T x? nx i= i ln x i Theorem 2.3. The following statements are equivalent: ( 0 ) : c T x > 0, 8 x 2 W \ K, x 6= 0. (2 0 ) : 9 d 2 W \ K such that c T d > 0, and f (x) f (d), 8x 2 W \ K. (3 0 ) : 9 d 2 W \ K such that g (x) g (d), 8x 2 W \ K. (4 0 ) : 9 d 2 W \ K such that P e;d D e c = P e;d. (5 0 ) : 9 d 2 W \ K such that P e;d D e c > 0. (6 0 ) : 9 d 2 W \ K such that P ;d D c = e.

6 6 BAHMAN KALANTARI (7 0 ) : 9 d 2 W \ K such that kp ;d D c? e k < min = minf ; : : :; n g. (8 0 ) : 9 d 2 W \ K such that P ;d D c > 0. (9 0 ) : 9 d 2 W \ K such that P 2 ;dd 2c =, where 2 = ( 2 ; ; 2 n) T. Moreover, (3 0 ), (4 0 ), and (6 0 ) have a unique and common solution d, and d also satises (2 0 ). The following is an obvious but important corollary of Theorem 2.3. Corollary 2.4. (Scaling dualities for linear forms) Either c T x = 0, for some x 2 W \ K, x 6= 0, or condition (i) is true, i = 2 0 ; : : :; 9 0 ; but not both. Remark 3. For i = 2 0, Corollary 2.4 implies that either Karmarkar's canonical LP is solvable, or the potential function f (x) is unbounded from below; but not both. In fact Karmarkar's algorithm, [5], can be viewed as an algorithmic proof of this fact, see also Kalantari [6], [9]. Another duality implied by this corollary is that either Karmarkar's canonical LP is solvable, or there there exists d 2 W \ K such that P e;d D e c > 0, but not both. This duality which implies Gordan's theorem gives rise to a simple variation of Karmarkar's algorithm (see Kalantari [0]), a polynomial-time algorithm that actually establishes this duality. and 3. Proof of Theorem 2.. Proof. We will prove the theorem by proving the following circle of implications () =) (2) =) (3) =) (4) =) (6) =) (7) =) (8) =) (); () () (5); (6) () (9): () =) (2) : Suppose > 0. Consider the set W \K \S, S = fx 2 E : kxk = g. Since X (x) approaches innity as x approaches a boundary point of W \ K \ S, its inmum is attained at some point, say, x. We claim that x must be the minimum of X (x) over W \ K. Otherwise, there exists x 2 W \ K such that X (x) < X (x ). But as X is homogeneous of degree zero, we get X (x=kxk) = X (x) < X (x ), a contradiction. (2) =) (3) : Suppose there exists d 2 W \ K such that (d) > 0, and X (x) X (d), for all x 2 W \ K. From the rst order optimality condition, d must be a stationary point of X, i.e., (3.) P rx (d) = 0; where P = P e;e = I? A T (AA T )? A. Dierentiating X (x), we get (3.2) rx (x) = r(x) + 2 (x)rf (x) exp( 2 F (x)): For any positive real, we have (3.3) r(x) = r(x); rf (x) = rf (x): From (3.), (3.2), and (3.3) we conclude that for any positive real, we have (3.4) P r(d) + 2 (2 (d))( rf (d)) = 0:

7 AG-Mean Inequality, LP Matrix Scaling, Gordan's Theorem 7 In particular, since (d) > 0, we can let = p =2(d). Then, setting d = d, we have (3.5) P r (d ) = 0: Equivalently, this implies (3.6) r (d ) = A T v; for some vector v of Lagrange multipliers. Since is convex, for all x 2 W \ K we have (3.7) (x) (d ) + r (d ) T (x? d ) = (d ) + v T A(x? d ) = (d ); hence (3) is satised. (3) =) (4) : Suppose d satises (3). Then P r (d) = 0. Equivalently, (3.8) r (d) = QD e e? D? e = A T v; for some v. Multiplying the latter equation by D e, we get (3.9) D e QD e e? = D e A T v: Multiply the above by AD e, we can solve for v to get (3.0) v = (A T D 2 e A)? AD e (D e QD e? ): Substituting the above in the previous equation, we get (3.) P e;d (D e QD e? ) = 0: Thus, (4) is valid. (4) =) (6) : If d satises (4), then it follows that r (d) = A T v, for some v. Equivalently, (3.2) r (d) = QD e? D? e = A T v: Multiplying the latter equation by D, we get (3.3) D QD e? e = D A T v: Multiply the above by AD, we can solve for v to get (3.4) v = (A T D 2 A)? AD (D QD e? e ): Substituting the above in the previous equation, we get (3.5) P ;d (D QD e? e ) = 0: Since AD e = Ad = 0, P ;d e = e. But this implies (6) is valid. (6) =) (7) : This is immediate. (7) =) (8) : Suppose that d satises (7). Assume that a given y 2 < n satises the inequality ky? k < min. Then, jy i? i j < min, for all i = ; : : :; n. Thus, it follows that y > 0. Hence (8) must hold.

8 8 BAHMAN KALANTARI (8) =) () : Suppose that d satises (8), and = 0. Since Q is positive semidenite, there exist x 2 W \ K, x 6= 0, such that (3.6) Qx = 0; Ax = 0: Let w = D? x. Note that w 0, w 6= 0. Since P ;dw = w, and D e = d, we have (3.7) w T P ;d D QD e = w T D QD e = x T Qd: On the one hand, since Qx = 0, we must have x T Qd = 0. On the other hand, since P ;d D QD e > 0, and w 0, w 6= 0, we get x T Qd > 0, a contradiction. Thus, if (8) holds, then so does (). () =) (5) : Since () implies (4), given any 2 K, we can select = e. Then, there exists d 2 W \K such that P e;d D e QD e e = P e;d e = e. Thus, (5) holds. (5) =) () : Suppose that d satises (5). Then, d satises (8) for = e. Hence, > 0. (6) =) (9) : This implication and and its converse are trivial. To prove the remaining claims of the theorem, we need to observe that since is strictly convex, the minimizer d of, if it exists, must be unique. We have already seen that if d satises (4) or (6), then it must satisfy P r (d) = 0. Thus, d is the minimizer of. Hence it satises (3) and is unique. Next we show that d also satises (2). Suppose that d 0 satises (2). Then, P rx (d 0 ) = 0. Since (d 0 ) > 0, some positive scalar multiple of d 0, say d 0 satises P r (d 0 ) = 0. But then we must have d 0 = d. Since X (x) = X(x), for all x 2 W \ K, d also satises (2). 4. Proof of Theorem 2.3. Proof. Assume that ( 0 ) holds. Let Q = cc T. Then, the corresponding is positive. From Theorem 2. we conclude that for each i = 2; : : :; 9, condition (i) is satised. (2) =) (2 0 ) : Since X (x) = f 2 (x), and ct x > 0, for x 2 W \ K, x 6= 0, (2 0 ) follows. (3) =) (3 0 ) : Suppose that d satises (3). Then cc T d? D e? = AT v, for some vector of lagrange multipliers, v. Since = c T d > 0, from the above equation we get c? D? e = A T v, where d = d=, v = v=. But this implies that d is the minimizer of g (x) = c T x + F (x). (4) =) (4 0 ) : Suppose d satises (4), i.e., P e;d D e cc T D e e = P e;d. Since = c T d > 0, ^d = d lies in W \ K. Since P e;d = P e; ^d, we get P e; ^d ^D e c = P e; ^d. Analogously to the above, it follows that for i = 5? 8, if d satises (i), then ^d = (c T d)d satises (i 0 ). (8 0 ) =) ( 0 ) : Follows from similar argument given for proof of the corresponding part of Theorem 2.. (6 0 ) =) (9 0 ) : This and its converse are trivial. The proof of the remaining parts follow in a similar fashion as in Theorem Two polynomial-time algorithms for HP, SP, HSP, and ASP. Here we will describe two polynomial-time algorithms for solving -approximate version of these problems. These algorithms are conceptually easy. However, their analysis requires the use of scaling dualities, signicant bounds, as well as some results from Nesterov and Nemirovskii's machinery of self-concordance. First, we dene - approximate version of the four problems. Next, we describe the two algorithms. We will then state without proof the main ingredients, other than the scaling dualities already proved. Finally, we will justify how the main ingredients result in the stated complexity results. For the proof of the stated theorems and in much more generality, see Kalantari [3].

9 AG-Mean Inequality, LP Matrix Scaling, Gordan's Theorem 9 Given 2 (0; ], -HP, -SP, -HSP, and -ASP are dened as follows. -HP is to compute d 2 W \K \S such that (d), or proving that such point does not exist. Given 2 K, -ASP is to test the solvability of kp ;d D QD e? e k <. -SP is to compute, if it exists, a point d 2 W \ K such that (d)?. -HSP is to compute, if it exists, a point d 2 W \ K such that X (d)=x exp(). In particular, if = min, then from Theorem 2. the solution of -ASP will result in a point d 2 W \ K such that P ;d D QD e > 0. For the convex-hull problem (see Remarks and 3) such a point gives rise to a separating hyperplane. Let u =?P r (d 0 ), where d 0 is a given point in W \ K \ S. For each t 2 (0; ), dene Given d 2 W \ K, let f (t) ;d f (t) (x) = t(x) + tut x + F (x): (t) (x) = f (D x) = t ;d (x) + tu T Dx + F (x) + F (d): Also, given d 2 W \ K, let y be the Newton direction with respect to, i.e., the solution to the equation P r 2 (d)y =?P r (d), where P is the orthogonal projection operator onto W. The Newton decrement is dened as (d) = [y T r 2 (d)y] =2. Let 2 [ ; ), where = 2? p 3. The Newton iterate is d 0 = NEW ( ; d) = d + (d)y; (d) = Analogously, corresponding to f (t) 8 >< >: ; if (d) > +(d) ;?(d) (d)(3?(d)) if (d) 2 [ ; ]; ; if (d) <., one denes y t, t (d), and d 0 t = NEW (f (t) ; d). Consider the following Potential-Reduction algorithm: Potential-Reduction: Initialization. Let d = d 0. Iterative Step. Replace d with d 0 = NEW ( ; d) and repeat. Theorem 5.. (Potential-Reduction Complexity Theorem, Kalantari [3]) Assume that e. Let be in (0; ). Consider the Potential-Reduction algorithm, and let = exp[ 2 (d 0 )? + ln ], and 2 R = supfexp(? 2 F (x)) : x 2 W \ K ; kxk = g. If = 0, the number of iterations to solve -HP is O( ln R ). If > 0, the number of iterations to solve -SP, -HSP, or -ASP is O( ln R kqk + ln ln ). Consider the following Path-Following algorithm, where t 2 (0; ) is an appropriately selected number: Path-Following: Initialization. Let t =, d = d 0. Phase I. While t > t, replace (d; t) with (d 0 t ; t0 ), where d 0 t absolute positive constant (can be taken equal to =9). Phase II. Replace d with d 0 = NEW (f (t) ; d) and repeat. (t) = NEW (f ; d), t0 = t exp(?c = p ), c an Theorem 5.2. (Path-Following Complexity Theorem, Kalantari [3]) Assume e. Let be in (0; ). Consider the Path-Following algorithm.

10 0 BAHMAN KALANTARI If = 0, the number of iterations to solve -HP is O( p ln kuk iterations to solve -ASP is O( p ln kuk is O( p ln kuk + ln ln kqk ). + ln ln kqk). If > 0, the number of + ln ln kqk), and the number of iterations to solve -SP or -HSP Remark 4. The above algorithms also apply to the case of an arbitrary > 0, not necessarily satisfying e. Essentially all is needed is to replace with 0 = = min. These algorithm can be used to solve linear programs, to compute separating hyperplanes, to diagonally scale positive semidenite symmetric matrices over an arbitrary subspace, or to compute the minimum of the ratio of an arbitrary linear (positive semidenite form) to the geometric mean (square of the geometric mean) over the positive points of an arbitrary subspace. The implementation of the above two algorithms for -SP, -HSP, and -ASP requires the computation of a lower bounds on, assuming that it is positive. For the computation of such a lower bound for the case where Q has rational or algebraic entries, see [2]. These algorithms can also be applied to solve Karmarkar's canonical LP. All is needed is to replace the linear form c T x with (c T x) 2. In particular, taking = e, the Path-Following algorithm gives an O( p n ln nkuk ) iteration complexity for solving -approximate version of Karmarkar's problem. As usual, over the rational inputs can be taken to be O(2?L ), where L is the size of LP (see [2] for dierent notions of size, over the rationals or the algebraic numbers). The Path-Following algorithm not only results in the best-known iteration complexity for solving LP (see e.g. [4]), while being one of the simplest of such algorithms, but also is an algorithm that is capable of solving more general diagonal matrix scaling problems than that considered in [6], as well as having the capability of solving SP and HSP. We now explain the main ingredients, other than the scaling dualities already proved, that result in the above two complexity theorems. The following theorem characterizes basic properties of Newton's method (see [9], Theorem 2.2.2), and main properties of the parametric family f (t) (see [9], Theorem 3..): Theorem 5.3. (Nesterov and Nemirovskii [9]) Given d 2 W \ K, d 0 = NEW ( ; d) 2 W \ K, and 8 >< >: (d 0 )? (d)?((d)? ln( + (d))); if (d) > ; (d 0 ) (6(d)? 4 2 (d)? ); if (d) 2 [ ; ]; (d 0 ) 2 (d) ; if (d) < (?(d)) 2. If (d) < 3, then = inff (x) : x 2 W \ K g >? and (d)?!2 ((d))( +!((d)) ;!((d)) =? (? 3(d)) =3 : 2(?!((d)) Given t 2 (0; ), all the above apply to f (t). Moreover, suppose that t(d) =4, then t 0(d) =4, where t 0 = t exp(?c p ). Now we state some other signicant results. Theorem 5.4. (Kalantari [3], x 8) Assume that e. Suppose that d 2 W \K satises (d) < =2. Then, kqk (d) kp ;d r ;d (e )k = O (d) : Suppose that > 0. Given any t 2 (0; ], suppose that d 2 W \ K satises t (d) <. Then, kqk kuk kp ;d rf (t) ;d (e )k = O t (d) :

11 AG-Mean Inequality, LP Matrix Scaling, Gordan's Theorem In either case ( > 0, or 0), given any t 2 (0; ], suppose that d 2 W \ K is a point obtained via Phase I of the Path-Following algorithm, and t (d) < =2. Then, kp ;d rf (t) ;d (e )k = O kqk t (d)( t p ) c : Theorem 5.5. (Kalantari [3], x 9) Let the k-th iterate of the Potential-Reduction algorithm be denoted by d k. If (d k ), then d k kd k R exp? 2k ; =? ln( + ): k Theorem 5.6. (Kalantari [3], x 9) Suppose that > 0. Then, for all x 2 W \ K we have X (x) ln X 2 (x)? : The following theorem is the nal ingredient, a signicant consequence of the scaling dualities: Theorem 5.7. (Kalantari [3], x 0) Let be a number in (0; ]. Given t 2 (0; ], suppose that d 2 W \ K satises kp ;d rf (t) ;d (e )k. Let ^d = p td. If kp 2 ; ^d r ; ^d (e )k, then ( kdk d ) C()t, where C() = [2 + (p + )]kuk 2 : 2 Now we describe how Theorem 2., and Theorems imply the desired complexity theorems, Theorem 5., and Theorem 5.2. We will rst describe this for Theorem 5.. From Theorem 2., HP is solvable (i.e. = 0) if and only if the other three problems are not. If = 0, Theorem 5.5 gives the desired complexity bound to solve -HP. If > 0, since (d k =kd k k), Theorem 5.5 implies that in O( ln R ) iterations we obtain a point x 2 W \ K such that (x) <. Let x k be the subsequent iterates of the Potential-Reduction algorithm, where x 0 = x. From Theorem 5.3 it follows that (5.) (x k ) r 2k ; r = (? ) 2 : The above together with the rst bound in Theorem 5.4 implies that the number of iterations, k, to get a point x k satisfying -ASP, is O(ln ln kqk ). Hence the claimed complexity for solving -ASP. From Theorem 5.3 it also follows that (see [3]) once we have obtained a point x k such that (x k ) < minf ; (? 3!( ))g, we have (5.2) (x k )? : This together with the bound in Theorem 5.6 gives the desired complexity for solving -SP, or -HSP. We will next derive the complexity result of Theorem 5.2. From Theorem 5.3 it follows that given any t 2 (0; ], since (d 0 ) = 0, the number of iterations, k t, to obtain d 2 W \K such that t (d) <, satises (5.3) O p ln ( t ) :

12 2 BAHMAN KALANTARI Suppose that = 0, then from Theorem 2. we must have kp ; ^d r ; ^d (e )k min. Theorem 5.7 it suces to compute d 2 W \ K such that From this, and (5.4) kp ;d rf (t) d (e d )k 2 min; where t satises (5.5) = C( min) : t Thus, the number of iterations of Phase I will equal O( p ln kuk ), and at the termination of this phase we have a point x 2 W \ K such that t (x) <. Let x 0 = x; x ; : : :; x k be the sequence of iterates of Phase II. From Theorem 5.3, it follows that t (x k ) r 2k. From this and the third upper bound given in Theorem 5.4, it follows that the number of iterations of Phase II to solve -HP is O(ln ln kqk?p =c ). Hence the claimed combined complexity of the two phases. Suppose that > 0. To solve -ASP, from Theorem 5.7 it suces to compute d 2 W \ K such that (5.6) kp ;d rf (t) d (e d )k 2 ; where t satises (5.7) t = 2C() : Thus, the number of iterations of Phase I is O( p ln kuk ). At the termination of this phase we have a point x 2 W \ K such that t (x) <. Let x 0 = x; x ; : : :; x k be the sequence of iterates of Phase II. From the second bound of Theorem 5.4, and since t (x k ) r 2k, the number of iterations of Phase II is O(ln ln kqk kuk ). Hence the claimed combined complexity. To solve -SP, or -HSP, we rst solve -ASP, i.e., compute a point d > 0 such that kp ;d D QD e? e k <, in O( p ln kuk ) iterations. We then replace d with NEW ( ; d), and repeat this step. The number of latter iterations can be estimated as it was done for the Potential-Reduction algorithm. 6. Concluding remarks. In this paper we have proved several dualities relating a generalization of the classical arithmetic geometric mean inequality to linear programming, to Gordan's theorem, and to several diagonal matrix scaling problems. These dualities, called scaling dualities, although were derived via elementary means are remarkable and signicant from the theoretical and algorithmic point of view. Many of the scaling dualities can be stated with respect to much more general convex programming problems, relating their equivalent HP formulation to problems that can be viewed as analogues of SP, HSP, and ASP, considered in this paper. The latter three problems are genuinely dual to the HP formulation of convex programming problems, i.e., HP is solvable if and only if the other three are not. Indeed, as shown in [3], given an arbitrary closed convex pointed cone K in a nite dimensional space, a logarithmically homogeneous barrier F (x) for the interior of K, a subspace W, and a homogeneous function with homogeneous degree p > 0, the scaling equation can be dened as P d r d (e d ) = e d, or P d r 2 d (e d ) = (p? )? e d, if p 6= ; where d 2 W \ K, D = r 2 F (d)?=2, d (x) = (Dx), e d = D? d, and P d the orthogonal projection onto W d = D? W. For instance, semidenite programming can be formulated as the problem of testing the existence of a nontrivial zero of tr(cx), over an arbitrary subspace of the space of symmetric matrices, and its intersection with the cone of positive semidenite symmetric matrices, K = S + n, where c is a given symmetric matrix, and tr() is the trace function. Now suppose that given 2 K, we take F (x) =? ln det(x ), where x is dened as follows. Firstly, given a matrix x 2 K, the matrix

13 AG-Mean Inequality, LP Matrix Scaling, Gordan's Theorem 3 exp(x) = P j= xj =j! is well-dened and lies in K. If x is a diagonal positive denite matrix, dene ln x as the matrix diag(ln x ; : : :; lnx n ). If x is an arbitrary positive denite matrix, dene ln x = U x ln x U T x, where x = U x x U T x, with U x unitary, and x the diagonal matrix of eigenvalues of x. Dene x as the matrix exp( =2 ln x =2 ). Now if for a given d 2 K we dene D = r 2 F (d)?=2, and e = I the identity matrix, and D e the operator that maps x 2 S + n to d =2 xd =2, then all the results proved or presented in this paper apply, verbatim. These include all the complexity theorems, and the auxiliary theorems that imply them. The quantity min will stand for the minimum eigenvalue of. These results are the subject of a forthcoming paper, [4]. In particular, in [4] we prove the following inequality which is a generalization of the arithmetic-geometric mean inequality as well as the well-known trace-determinant inequality: let be an arbitrary matrix in K, the cone of positive denite symmetric matrices, and = tr(). Then, tr(x) (det(x )) = ; 8x 2 K : REFERENCES [] H. Alzer, A proof of arithmetic-geometric mean inequality, Amer. Math. Monthly, 03 (996) 585. [2] G.B. Dantzig, Linear Programming and Extensions (Princeton University Press, Princeton, New Jersey, 963). [3] P.S. Bullen, D.S. Mitrinovic, and P.M. Vasic, Means and Their Inequalities (Reidel, Dordrecht, 988). [4] C.C. Gonzaga, Path-following methods for linear programming, SIAM Review, 34 (992) [5] G.H. Hardy, J.E. Littlewood, and G. Polya, Inequalities (Cambridge University Press, Cambridge, 952). [6] B. Kalantari, Karmarkar's algorithm with improved steps, Math. Programming, 46 (990) [7] B. Kalantari, Canonical problems for quadratic programming and projective methods for their solution, Contemporary Mathematics, 4 (990) [8] B. Kalantari, Derivation of a generalized and strengthened Gordan theorem from generalized Karmarkar potential and logarithmic barrier functions, Technical Report LCSR-TR-2, Department of Computer Science, Rutgers University, New Brunswick, NJ, 989. [9] B. Kalantari, Generalization of Karmarkar's algorithm to convex homogeneous functions, Oper. Res. Lett., (992) [0] B. Kalantari, A simple polynomial time algorithm for a convex hull problem equivalent to linear programming, Combinatorics Advances, Kluwer Academic Publishers (995) [] B. Kalantari, A Theorem of the alternative for multihomogeneous functions and its relationship to diagonal scaling of matrices, Linear Algebra and its Applications, 236 (996) -24. [2] B. Kalantari and M.R. Emamy-K, On linear programming and matrix scaling over the algebraic numbers, Linear Algebra and its Applications, 262 (997) [3] B. Kalantari, Scaling dualities and self-concordant homogeneous programming in nite dimensional spaces, Technical Report DCS-TR 359, Department of Computer Science, Rutgers University, New Brunswick, NJ, 998. [4] B. Kalantari, On the trace-determinant inequality, semidenite programming, and matrix scaling over the cone matrices, forthcoming. [5] N. Karmarkar, A new polynomial time algorithm for linear programming, Combinatorica, 4 (984) [6] L. Khachiyan and B. Kalantari, Diagonal matrix scaling and linear programming, SIAM J. Optim., 4 (992) [7] A.W. Marshall and I. Olkin, Scaling of matrices to achieve specied row and column sums, Numer. Math., 2 (968) [8] D.S. Mitrinovic, J.E. Pecaric, and A.M. Fink, Classical and New Inequalities in Analysis (Kluwer Academic Publishers, Dordrecht, 993). [9] Y. Nesterov and A.S. Nemirovskii, Interior-Point Polynomial Algorithms in Convex Programming (SIAM, Philadelphia, PA, 994). [20] A. Schrijver, Theory of Linear and Integer Programming (John Wiley and Sons, New York, 986).

### SCALING DUALITIES AND SELF-CONCORDANT HOMOGENEOUS PROGRAMMING IN FINITE DIMENSIONAL SPACES

SCALING DUALITIES AND SELF-CONCORDANT HOMOGENEOUS PROGRAMMING IN FINITE DIMENSIONAL SPACES BAHMAN KALANTARI Abstract. In this paper first we prove four fundamental theorems of the alternative, called scaling

### Linear Algebra, 4th day, Thursday 7/1/04 REU Info:

Linear Algebra, 4th day, Thursday 7/1/04 REU 004. Info http//people.cs.uchicago.edu/laci/reu04. Instructor Laszlo Babai Scribe Nick Gurski 1 Linear maps We shall study the notion of maps between vector

### Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) 1.1 The Formal Denition of a Vector Space

Linear Algebra (part 1) : Vector Spaces (by Evan Dummit, 2017, v. 1.07) Contents 1 Vector Spaces 1 1.1 The Formal Denition of a Vector Space.................................. 1 1.2 Subspaces...................................................

### 1 Introduction Semidenite programming (SDP) has been an active research area following the seminal work of Nesterov and Nemirovski [9] see also Alizad

Quadratic Maximization and Semidenite Relaxation Shuzhong Zhang Econometric Institute Erasmus University P.O. Box 1738 3000 DR Rotterdam The Netherlands email: zhang@few.eur.nl fax: +31-10-408916 August,

### Chapter 1. Preliminaries. The purpose of this chapter is to provide some basic background information. Linear Space. Hilbert Space.

Chapter 1 Preliminaries The purpose of this chapter is to provide some basic background information. Linear Space Hilbert Space Basic Principles 1 2 Preliminaries Linear Space The notion of linear space

### LMI MODELLING 4. CONVEX LMI MODELLING. Didier HENRION. LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ. Universidad de Valladolid, SP March 2009

LMI MODELLING 4. CONVEX LMI MODELLING Didier HENRION LAAS-CNRS Toulouse, FR Czech Tech Univ Prague, CZ Universidad de Valladolid, SP March 2009 Minors A minor of a matrix F is the determinant of a submatrix

### 4.1 Eigenvalues, Eigenvectors, and The Characteristic Polynomial

Linear Algebra (part 4): Eigenvalues, Diagonalization, and the Jordan Form (by Evan Dummit, 27, v ) Contents 4 Eigenvalues, Diagonalization, and the Jordan Canonical Form 4 Eigenvalues, Eigenvectors, and

### CHAPTER III THE PROOF OF INEQUALITIES

CHAPTER III THE PROOF OF INEQUALITIES In this Chapter, the main purpose is to prove four theorems about Hardy-Littlewood- Pólya Inequality and then gives some examples of their application. We will begin

### Assignment 1: From the Definition of Convexity to Helley Theorem

Assignment 1: From the Definition of Convexity to Helley Theorem Exercise 1 Mark in the following list the sets which are convex: 1. {x R 2 : x 1 + i 2 x 2 1, i = 1,..., 10} 2. {x R 2 : x 2 1 + 2ix 1x

### Chapter 1. Preliminaries

Introduction This dissertation is a reading of chapter 4 in part I of the book : Integer and Combinatorial Optimization by George L. Nemhauser & Laurence A. Wolsey. The chapter elaborates links between

### Lecture 1. 1 Conic programming. MA 796S: Convex Optimization and Interior Point Methods October 8, Consider the conic program. min.

MA 796S: Convex Optimization and Interior Point Methods October 8, 2007 Lecture 1 Lecturer: Kartik Sivaramakrishnan Scribe: Kartik Sivaramakrishnan 1 Conic programming Consider the conic program min s.t.

### A Finite Element Method for an Ill-Posed Problem. Martin-Luther-Universitat, Fachbereich Mathematik/Informatik,Postfach 8, D Halle, Abstract

A Finite Element Method for an Ill-Posed Problem W. Lucht Martin-Luther-Universitat, Fachbereich Mathematik/Informatik,Postfach 8, D-699 Halle, Germany Abstract For an ill-posed problem which has its origin

### Key words. Complementarity set, Lyapunov rank, Bishop-Phelps cone, Irreducible cone

ON THE IRREDUCIBILITY LYAPUNOV RANK AND AUTOMORPHISMS OF SPECIAL BISHOP-PHELPS CONES M. SEETHARAMA GOWDA AND D. TROTT Abstract. Motivated by optimization considerations we consider cones in R n to be called

### 58 Appendix 1 fundamental inconsistent equation (1) can be obtained as a linear combination of the two equations in (2). This clearly implies that the

Appendix PRELIMINARIES 1. THEOREMS OF ALTERNATIVES FOR SYSTEMS OF LINEAR CONSTRAINTS Here we consider systems of linear constraints, consisting of equations or inequalities or both. A feasible solution

### APPROXIMATING THE COMPLEXITY MEASURE OF. Levent Tuncel. November 10, C&O Research Report: 98{51. Abstract

APPROXIMATING THE COMPLEXITY MEASURE OF VAVASIS-YE ALGORITHM IS NP-HARD Levent Tuncel November 0, 998 C&O Research Report: 98{5 Abstract Given an m n integer matrix A of full row rank, we consider the

### The Nearest Doubly Stochastic Matrix to a Real Matrix with the same First Moment

he Nearest Doubly Stochastic Matrix to a Real Matrix with the same First Moment William Glunt 1, homas L. Hayden 2 and Robert Reams 2 1 Department of Mathematics and Computer Science, Austin Peay State

### A Generalized Homogeneous and Self-Dual Algorithm. for Linear Programming. February 1994 (revised December 1994)

A Generalized Homogeneous and Self-Dual Algorithm for Linear Programming Xiaojie Xu Yinyu Ye y February 994 (revised December 994) Abstract: A generalized homogeneous and self-dual (HSD) infeasible-interior-point

### Permutation invariant proper polyhedral cones and their Lyapunov rank

Permutation invariant proper polyhedral cones and their Lyapunov rank Juyoung Jeong Department of Mathematics and Statistics University of Maryland, Baltimore County Baltimore, Maryland 21250, USA juyoung1@umbc.edu

### A BASIC FAMILY OF ITERATION FUNCTIONS FOR POLYNOMIAL ROOT FINDING AND ITS CHARACTERIZATIONS. Bahman Kalantari 3. Department of Computer Science

A BASIC FAMILY OF ITERATION FUNCTIONS FOR POLYNOMIAL ROOT FINDING AND ITS CHARACTERIZATIONS Bahman Kalantari 3 Department of Computer Science Rutgers University, New Brunswick, NJ 08903 Iraj Kalantari

### Absolute value equations

Linear Algebra and its Applications 419 (2006) 359 367 www.elsevier.com/locate/laa Absolute value equations O.L. Mangasarian, R.R. Meyer Computer Sciences Department, University of Wisconsin, 1210 West

### Contents. 2.1 Vectors in R n. Linear Algebra (part 2) : Vector Spaces (by Evan Dummit, 2017, v. 2.50) 2 Vector Spaces

Linear Algebra (part 2) : Vector Spaces (by Evan Dummit, 2017, v 250) Contents 2 Vector Spaces 1 21 Vectors in R n 1 22 The Formal Denition of a Vector Space 4 23 Subspaces 6 24 Linear Combinations and

### DS-GA 1002 Lecture notes 0 Fall Linear Algebra. These notes provide a review of basic concepts in linear algebra.

DS-GA 1002 Lecture notes 0 Fall 2016 Linear Algebra These notes provide a review of basic concepts in linear algebra. 1 Vector spaces You are no doubt familiar with vectors in R 2 or R 3, i.e. [ ] 1.1

### Robust linear optimization under general norms

Operations Research Letters 3 (004) 50 56 Operations Research Letters www.elsevier.com/locate/dsw Robust linear optimization under general norms Dimitris Bertsimas a; ;, Dessislava Pachamanova b, Melvyn

### Inequality Constraints

Chapter 2 Inequality Constraints 2.1 Optimality Conditions Early in multivariate calculus we learn the significance of differentiability in finding minimizers. In this section we begin our study of the

### Linear Algebra 1 Exam 2 Solutions 7/14/3

Linear Algebra 1 Exam Solutions 7/14/3 Question 1 The line L has the symmetric equation: x 1 = y + 3 The line M has the parametric equation: = z 4. [x, y, z] = [ 4, 10, 5] + s[10, 7, ]. The line N is perpendicular

### ELEMENTARY LINEAR ALGEBRA

ELEMENTARY LINEAR ALGEBRA K R MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND First Printing, 99 Chapter LINEAR EQUATIONS Introduction to linear equations A linear equation in n unknowns x,

### CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS. W. Erwin Diewert January 31, 2008.

1 ECONOMICS 594: LECTURE NOTES CHAPTER 2: CONVEX SETS AND CONCAVE FUNCTIONS W. Erwin Diewert January 31, 2008. 1. Introduction Many economic problems have the following structure: (i) a linear function

### Introduction to Nonlinear Stochastic Programming

School of Mathematics T H E U N I V E R S I T Y O H F R G E D I N B U Introduction to Nonlinear Stochastic Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio SPS

### ELEMENTARY LINEAR ALGEBRA

ELEMENTARY LINEAR ALGEBRA K. R. MATTHEWS DEPARTMENT OF MATHEMATICS UNIVERSITY OF QUEENSLAND Second Online Version, December 1998 Comments to the author at krm@maths.uq.edu.au Contents 1 LINEAR EQUATIONS

### Approximation Algorithms for Maximum. Coverage and Max Cut with Given Sizes of. Parts? A. A. Ageev and M. I. Sviridenko

Approximation Algorithms for Maximum Coverage and Max Cut with Given Sizes of Parts? A. A. Ageev and M. I. Sviridenko Sobolev Institute of Mathematics pr. Koptyuga 4, 630090, Novosibirsk, Russia fageev,svirg@math.nsc.ru

### The Simplest Semidefinite Programs are Trivial

The Simplest Semidefinite Programs are Trivial Robert J. Vanderbei Bing Yang Program in Statistics & Operations Research Princeton University Princeton, NJ 08544 January 10, 1994 Technical Report SOR-93-12

### y Ray of Half-line or ray through in the direction of y

Chapter LINEAR COMPLEMENTARITY PROBLEM, ITS GEOMETRY, AND APPLICATIONS. THE LINEAR COMPLEMENTARITY PROBLEM AND ITS GEOMETRY The Linear Complementarity Problem (abbreviated as LCP) is a general problem

### On well definedness of the Central Path

On well definedness of the Central Path L.M.Graña Drummond B. F. Svaiter IMPA-Instituto de Matemática Pura e Aplicada Estrada Dona Castorina 110, Jardim Botânico, Rio de Janeiro-RJ CEP 22460-320 Brasil

### Linear Algebra: Characteristic Value Problem

Linear Algebra: Characteristic Value Problem . The Characteristic Value Problem Let < be the set of real numbers and { be the set of complex numbers. Given an n n real matrix A; does there exist a number

### ON SUM OF SQUARES DECOMPOSITION FOR A BIQUADRATIC MATRIX FUNCTION

Annales Univ. Sci. Budapest., Sect. Comp. 33 (2010) 273-284 ON SUM OF SQUARES DECOMPOSITION FOR A BIQUADRATIC MATRIX FUNCTION L. László (Budapest, Hungary) Dedicated to Professor Ferenc Schipp on his 70th

### Lecture 5: Hyperbolic polynomials

Lecture 5: Hyperbolic polynomials Robin Pemantle University of Pennsylvania pemantle@math.upenn.edu Minerva Lectures at Columbia University 16 November, 2016 Ubiquity of zeros of polynomials The one contribution

### MAT Linear Algebra Collection of sample exams

MAT 342 - Linear Algebra Collection of sample exams A-x. (0 pts Give the precise definition of the row echelon form. 2. ( 0 pts After performing row reductions on the augmented matrix for a certain system

### Supplement: Universal Self-Concordant Barrier Functions

IE 8534 1 Supplement: Universal Self-Concordant Barrier Functions IE 8534 2 Recall that a self-concordant barrier function for K is a barrier function satisfying 3 F (x)[h, h, h] 2( 2 F (x)[h, h]) 3/2,

### Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method

Primal-Dual Interior-Point Methods for Linear Programming based on Newton s Method Robert M. Freund March, 2004 2004 Massachusetts Institute of Technology. The Problem The logarithmic barrier approach

### Rank-one LMIs and Lyapunov's Inequality. Gjerrit Meinsma 4. Abstract. We describe a new proof of the well-known Lyapunov's matrix inequality about

Rank-one LMIs and Lyapunov's Inequality Didier Henrion 1;; Gjerrit Meinsma Abstract We describe a new proof of the well-known Lyapunov's matrix inequality about the location of the eigenvalues of a matrix

### Scaling of Symmetric Matrices by Positive Diagonal Congruence

Scaling of Symmetric Matrices by Positive Diagonal Congruence Abstract Charles R. Johnson a and Robert Reams b a Department of Mathematics, College of William and Mary, P.O. Box 8795, Williamsburg, VA

### On self-concordant barriers for generalized power cones

On self-concordant barriers for generalized power cones Scott Roy Lin Xiao January 30, 2018 Abstract In the study of interior-point methods for nonsymmetric conic optimization and their applications, Nesterov

### Motakuri Ramana y Levent Tuncel and Henry Wolkowicz z. University of Waterloo. Faculty of Mathematics. Waterloo, Ontario N2L 3G1, Canada.

STRONG DUALITY FOR SEMIDEFINITE PROGRAMMING Motakuri Ramana y Levent Tuncel and Henry Wolkowicz z University of Waterloo Department of Combinatorics and Optimization Faculty of Mathematics Waterloo, Ontario

### 16 Chapter 3. Separation Properties, Principal Pivot Transforms, Classes... for all j 2 J is said to be a subcomplementary vector of variables for (3.

Chapter 3 SEPARATION PROPERTIES, PRINCIPAL PIVOT TRANSFORMS, CLASSES OF MATRICES In this chapter we present the basic mathematical results on the LCP. Many of these results are used in later chapters to

### Lecture 5. Theorems of Alternatives and Self-Dual Embedding

IE 8534 1 Lecture 5. Theorems of Alternatives and Self-Dual Embedding IE 8534 2 A system of linear equations may not have a solution. It is well known that either Ax = c has a solution, or A T y = 0, c

### A matrix over a field F is a rectangular array of elements from F. The symbol

Chapter MATRICES Matrix arithmetic A matrix over a field F is a rectangular array of elements from F The symbol M m n (F ) denotes the collection of all m n matrices over F Matrices will usually be denoted

### Copositive matrices and periodic dynamical systems

Extreme copositive matrices and periodic dynamical systems Weierstrass Institute (WIAS), Berlin Optimization without borders Dedicated to Yuri Nesterovs 60th birthday February 11, 2016 and periodic dynamical

### May 9, 2014 MATH 408 MIDTERM EXAM OUTLINE. Sample Questions

May 9, 24 MATH 48 MIDTERM EXAM OUTLINE This exam will consist of two parts and each part will have multipart questions. Each of the 6 questions is worth 5 points for a total of points. The two part of

### 2. Dual space is essential for the concept of gradient which, in turn, leads to the variational analysis of Lagrange multipliers.

Chapter 3 Duality in Banach Space Modern optimization theory largely centers around the interplay of a normed vector space and its corresponding dual. The notion of duality is important for the following

### Semidefinite Programming

Chapter 2 Semidefinite Programming 2.0.1 Semi-definite programming (SDP) Given C M n, A i M n, i = 1, 2,..., m, and b R m, the semi-definite programming problem is to find a matrix X M n for the optimization

### WHEN DOES THE POSITIVE SEMIDEFINITENESS CONSTRAINT HELP IN LIFTING PROCEDURES?

MATHEMATICS OF OPERATIONS RESEARCH Vol. 6, No. 4, November 00, pp. 796 85 Printed in U.S.A. WHEN DOES THE POSITIVE SEMIDEFINITENESS CONSTRAINT HELP IN LIFTING PROCEDURES? MICHEL X. GOEMANS and LEVENT TUNÇEL

### An Alternative Proof of Primitivity of Indecomposable Nonnegative Matrices with a Positive Trace

An Alternative Proof of Primitivity of Indecomposable Nonnegative Matrices with a Positive Trace Takao Fujimoto Abstract. This research memorandum is aimed at presenting an alternative proof to a well

### Largest dual ellipsoids inscribed in dual cones

Largest dual ellipsoids inscribed in dual cones M. J. Todd June 23, 2005 Abstract Suppose x and s lie in the interiors of a cone K and its dual K respectively. We seek dual ellipsoidal norms such that

### In English, this means that if we travel on a straight line between any two points in C, then we never leave C.

Convex sets In this section, we will be introduced to some of the mathematical fundamentals of convex sets. In order to motivate some of the definitions, we will look at the closest point problem from

### Lecture 15 Newton Method and Self-Concordance. October 23, 2008

Newton Method and Self-Concordance October 23, 2008 Outline Lecture 15 Self-concordance Notion Self-concordant Functions Operations Preserving Self-concordance Properties of Self-concordant Functions Implications

### 5 Eigenvalues and Diagonalization

Linear Algebra (part 5): Eigenvalues and Diagonalization (by Evan Dummit, 27, v 5) Contents 5 Eigenvalues and Diagonalization 5 Eigenvalues, Eigenvectors, and The Characteristic Polynomial 5 Eigenvalues

### MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products.

MATH 304 Linear Algebra Lecture 19: Least squares problems (continued). Norms and inner products. Orthogonal projection Theorem 1 Let V be a subspace of R n. Then any vector x R n is uniquely represented

The Pacic Institute for the Mathematical Sciences http://www.pims.math.ca pims@pims.math.ca Surprise Maximization D. Borwein Department of Mathematics University of Western Ontario London, Ontario, Canada

### Optimization: Interior-Point Methods and. January,1995 USA. and Cooperative Research Centre for Robust and Adaptive Systems.

Innite Dimensional Quadratic Optimization: Interior-Point Methods and Control Applications January,995 Leonid Faybusovich John B. Moore y Department of Mathematics University of Notre Dame Mail Distribution

### COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS. Didier HENRION henrion

COURSE ON LMI PART I.2 GEOMETRY OF LMI SETS Didier HENRION www.laas.fr/ henrion October 2006 Geometry of LMI sets Given symmetric matrices F i we want to characterize the shape in R n of the LMI set F

### The maximal stable set problem : Copositive programming and Semidefinite Relaxations

The maximal stable set problem : Copositive programming and Semidefinite Relaxations Kartik Krishnan Department of Mathematical Sciences Rensselaer Polytechnic Institute Troy, NY 12180 USA kartis@rpi.edu

### Vector Space Basics. 1 Abstract Vector Spaces. 1. (commutativity of vector addition) u + v = v + u. 2. (associativity of vector addition)

Vector Space Basics (Remark: these notes are highly formal and may be a useful reference to some students however I am also posting Ray Heitmann's notes to Canvas for students interested in a direct computational

### INDEFINITE TRUST REGION SUBPROBLEMS AND NONSYMMETRIC EIGENVALUE PERTURBATIONS. Ronald J. Stern. Concordia University

INDEFINITE TRUST REGION SUBPROBLEMS AND NONSYMMETRIC EIGENVALUE PERTURBATIONS Ronald J. Stern Concordia University Department of Mathematics and Statistics Montreal, Quebec H4B 1R6, Canada and Henry Wolkowicz

### Math 350 Fall 2011 Notes about inner product spaces. In this notes we state and prove some important properties of inner product spaces.

Math 350 Fall 2011 Notes about inner product spaces In this notes we state and prove some important properties of inner product spaces. First, recall the dot product on R n : if x, y R n, say x = (x 1,...,

### CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming

CSCI 1951-G Optimization Methods in Finance Part 01: Linear Programming January 26, 2018 1 / 38 Liability/asset cash-flow matching problem Recall the formulation of the problem: max w c 1 + p 1 e 1 = 150

### Spectral inequalities and equalities involving products of matrices

Spectral inequalities and equalities involving products of matrices Chi-Kwong Li 1 Department of Mathematics, College of William & Mary, Williamsburg, Virginia 23187 (ckli@math.wm.edu) Yiu-Tung Poon Department

### 1. Algebraic and geometric treatments Consider an LP problem in the standard form. x 0. Solutions to the system of linear equations

The Simplex Method Most textbooks in mathematical optimization, especially linear programming, deal with the simplex method. In this note we study the simplex method. It requires basically elementary linear

### Introduction to Semidefinite Programming I: Basic properties a

Introduction to Semidefinite Programming I: Basic properties and variations on the Goemans-Williamson approximation algorithm for max-cut MFO seminar on Semidefinite Programming May 30, 2010 Semidefinite

### On the Chvatál-Complexity of Binary Knapsack Problems. Gergely Kovács 1 Béla Vizvári College for Modern Business Studies, Hungary

On the Chvatál-Complexity of Binary Knapsack Problems Gergely Kovács 1 Béla Vizvári 2 1 College for Modern Business Studies, Hungary 2 Eastern Mediterranean University, TRNC 2009. 1 Chvátal Cut and Complexity

### More First-Order Optimization Algorithms

More First-Order Optimization Algorithms Yinyu Ye Department of Management Science and Engineering Stanford University Stanford, CA 94305, U.S.A. http://www.stanford.edu/ yyye Chapters 3, 8, 3 The SDM

### SQUARE ROOTS OF 2x2 MATRICES 1. Sam Northshield SUNY-Plattsburgh

SQUARE ROOTS OF x MATRICES Sam Northshield SUNY-Plattsburgh INTRODUCTION A B What is the square root of a matrix such as? It is not, in general, A B C D C D This is easy to see since the upper left entry

### L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones. definition. spectral decomposition. quadratic representation. log-det barrier 18-1

L. Vandenberghe EE236C (Spring 2016) 18. Symmetric cones definition spectral decomposition quadratic representation log-det barrier 18-1 Introduction This lecture: theoretical properties of the following

### Convex analysis on Cartan subspaces

Nonlinear Analysis 42 (2000) 813 820 www.elsevier.nl/locate/na Convex analysis on Cartan subspaces A.S. Lewis ;1 Department of Combinatorics and Optimization, University of Waterloo, Waterloo, Ontario,

### 1 Vectors. Notes for Bindel, Spring 2017 Numerical Analysis (CS 4220)

Notes for 2017-01-30 Most of mathematics is best learned by doing. Linear algebra is no exception. You have had a previous class in which you learned the basics of linear algebra, and you will have plenty

### CONCENTRATION OF THE MIXED DISCRIMINANT OF WELL-CONDITIONED MATRICES. Alexander Barvinok

CONCENTRATION OF THE MIXED DISCRIMINANT OF WELL-CONDITIONED MATRICES Alexander Barvinok Abstract. We call an n-tuple Q 1,...,Q n of positive definite n n real matrices α-conditioned for some α 1 if for

### On John type ellipsoids

On John type ellipsoids B. Klartag Tel Aviv University Abstract Given an arbitrary convex symmetric body K R n, we construct a natural and non-trivial continuous map u K which associates ellipsoids to

### Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming

School of Mathematics T H E U N I V E R S I T Y O H F E D I N B U R G Interior Point Methods: Second-Order Cone Programming and Semidefinite Programming Jacek Gondzio Email: J.Gondzio@ed.ac.uk URL: http://www.maths.ed.ac.uk/~gondzio

### An E cient A ne-scaling Algorithm for Hyperbolic Programming

An E cient A ne-scaling Algorithm for Hyperbolic Programming Jim Renegar joint work with Mutiara Sondjaja 1 Euclidean space A homogeneous polynomial p : E!R is hyperbolic if there is a vector e 2E such

### DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO

QUESTION BOOKLET EECS 227A Fall 2009 Midterm Tuesday, Ocotober 20, 11:10-12:30pm DO NOT OPEN THIS QUESTION BOOKLET UNTIL YOU ARE TOLD TO DO SO You have 80 minutes to complete the midterm. The midterm consists

### [3] (b) Find a reduced row-echelon matrix row-equivalent to ,1 2 2

MATH Key for sample nal exam, August 998 []. (a) Dene the term \reduced row-echelon matrix". A matrix is reduced row-echelon if the following conditions are satised. every zero row lies below every nonzero

### Special Classes of Fuzzy Integer Programming Models with All-Dierent Constraints

Transaction E: Industrial Engineering Vol. 16, No. 1, pp. 1{10 c Sharif University of Technology, June 2009 Special Classes of Fuzzy Integer Programming Models with All-Dierent Constraints Abstract. K.

### Nonlinear Optimization: What s important?

Nonlinear Optimization: What s important? Julian Hall 10th May 2012 Convexity: convex problems A local minimizer is a global minimizer A solution of f (x) = 0 (stationary point) is a minimizer A global

### Chapter 1: Systems of Linear Equations

Chapter : Systems of Linear Equations February, 9 Systems of linear equations Linear systems Lecture A linear equation in variables x, x,, x n is an equation of the form a x + a x + + a n x n = b, where

### A Criterion for the Stochasticity of Matrices with Specified Order Relations

Rend. Istit. Mat. Univ. Trieste Vol. XL, 55 64 (2009) A Criterion for the Stochasticity of Matrices with Specified Order Relations Luca Bortolussi and Andrea Sgarro Abstract. We tackle the following problem:

### A priori bounds on the condition numbers in interior-point methods

A priori bounds on the condition numbers in interior-point methods Florian Jarre, Mathematisches Institut, Heinrich-Heine Universität Düsseldorf, Germany. Abstract Interior-point methods are known to be

### Contents. 4 Arithmetic and Unique Factorization in Integral Domains. 4.1 Euclidean Domains and Principal Ideal Domains

Ring Theory (part 4): Arithmetic and Unique Factorization in Integral Domains (by Evan Dummit, 018, v. 1.00) Contents 4 Arithmetic and Unique Factorization in Integral Domains 1 4.1 Euclidean Domains and

IMC 05, Blagoevgrad, Bulgaria Day, July 9, 05 Problem. For any integer n and two n n matrices with real entries, B that satisfy the equation + B ( + B prove that det( det(b. Does the same conclusion follow

### Lecture 9 Monotone VIs/CPs Properties of cones and some existence results. October 6, 2008

Lecture 9 Monotone VIs/CPs Properties of cones and some existence results October 6, 2008 Outline Properties of cones Existence results for monotone CPs/VIs Polyhedrality of solution sets Game theory:

### Linear Algebra. Min Yan

Linear Algebra Min Yan January 2, 2018 2 Contents 1 Vector Space 7 1.1 Definition................................. 7 1.1.1 Axioms of Vector Space..................... 7 1.1.2 Consequence of Axiom......................

### Primal-dual relationship between Levenberg-Marquardt and central trajectories for linearly constrained convex optimization

Primal-dual relationship between Levenberg-Marquardt and central trajectories for linearly constrained convex optimization Roger Behling a, Clovis Gonzaga b and Gabriel Haeser c March 21, 2013 a Department

### a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2.

Chapter 1 LINEAR EQUATIONS 11 Introduction to linear equations A linear equation in n unknowns x 1, x,, x n is an equation of the form a 1 x 1 + a x + + a n x n = b, where a 1, a,, a n, b are given real

### Nonsymmetric potential-reduction methods for general cones

CORE DISCUSSION PAPER 2006/34 Nonsymmetric potential-reduction methods for general cones Yu. Nesterov March 28, 2006 Abstract In this paper we propose two new nonsymmetric primal-dual potential-reduction

### Chapter 3 Least Squares Solution of y = A x 3.1 Introduction We turn to a problem that is dual to the overconstrained estimation problems considered s

Lectures on Dynamic Systems and Control Mohammed Dahleh Munther A. Dahleh George Verghese Department of Electrical Engineering and Computer Science Massachuasetts Institute of Technology 1 1 c Chapter

Mathematical Olympiad Training Polynomials Definition A polynomial over a ring R(Z, Q, R, C) in x is an expression of the form p(x) = a n x n + a n 1 x n 1 + + a 1 x + a 0, a i R, for 0 i n. If a n 0,

### A PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE

Yugoslav Journal of Operations Research 24 (2014) Number 1, 35-51 DOI: 10.2298/YJOR120904016K A PREDICTOR-CORRECTOR PATH-FOLLOWING ALGORITHM FOR SYMMETRIC OPTIMIZATION BASED ON DARVAY'S TECHNIQUE BEHROUZ

### Math 341: Convex Geometry. Xi Chen

Math 341: Convex Geometry Xi Chen 479 Central Academic Building, University of Alberta, Edmonton, Alberta T6G 2G1, CANADA E-mail address: xichen@math.ualberta.ca CHAPTER 1 Basics 1. Euclidean Geometry

### Using Schur Complement Theorem to prove convexity of some SOC-functions

Journal of Nonlinear and Convex Analysis, vol. 13, no. 3, pp. 41-431, 01 Using Schur Complement Theorem to prove convexity of some SOC-functions Jein-Shan Chen 1 Department of Mathematics National Taiwan

### Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization

Extreme Abridgment of Boyd and Vandenberghe s Convex Optimization Compiled by David Rosenberg Abstract Boyd and Vandenberghe s Convex Optimization book is very well-written and a pleasure to read. The