TARGET FOLLOWING ALGORITHMS FOR SEMIDEFINITE PROGRAMMING

Transcription

1 TARGET FOLLOWING ALGORITHMS FOR SEMIDEFINITE PROGRAMMING CHEK BENG CHUA C & O Research Report: CORR May 10, 006 Abstract. We present a target-foowing framework for semidefinite programming, which generaizes the target-foowing framework for inear programming. We use this framework to buid weighted path-foowing interior-point agorithms of three distinct favors: shortstep, predictor-corrector, and arge-update. These agorithms have worse-case iteration bounds that parae their counterparts in inear programming. We further consider the probem of finding anaytic centers given a pair of prima-dua stricty feasibe soutions. An agorithm that moves towards the anaytic center prior to reducing the duaity gap has a better iteration bound than the weighted path-foowing agorithms. In the case of inear programming, this bound is aso an improvement over existing simiar agorithms. 000 Mathematics Subject Cassification. 90C; 90C5. Key words and phrases. Semidefinite programming; Weighted anaytic centers; Weighted centra path; Target-foowing agorithm; Weighted path-foowing agorithm; Target space; Choesky search directions; Efficient centering; Loca Lipschitz constant of Choesky factorization. This research was supported in part by a grant from the Facuty of Mathematics, University of Wateroo and by a Discovery Grant from NSERC. 1

2 Contents 1. Introduction 1.1. Organization of materia Notations and conventions 3. Target-Foowing Framework 5.1. Expanded semidefinite programming probems 6.. Choice of targets An Exampe Based on the Monteiro-Zhang Famiy Anaysis of agorithm Weighted path-foowing agorithms 4. Target-Foowing Framework Based on Choesky Search Directions Anaysis of agorithm Large-update agorithm 3 5. Finding Anaytic Centers Approximation of anaytic centers 36 References Introduction The target-foowing framework was first introduced by Mizuno [8] for inear compementarity probems and Jansen, Roos, Teraky and Via [6] for inear programming as a unifying framework for various prima-dua path-foowing agorithms and agorithms that find anaytic centers. The essentia ingredient of this framework is the target map x, s) [x 1 s 1,..., x n s n ] T, defined for each pair of positive n-vectors x, s). An important feature of the target map is its bijection between the prima-dua stricty feasibe region and the cone of positive n-vectors R n ++ [6, 7], whence identifying the prima-dua stricty feasibe region with the reativey simpe cone R n ++ known as the target space or v-space). Interior-point agorithms based on the target map are known as target-foowing agorithms, which are conceptuay simpe when viewed as foowing a sequence of targets in the target space. Various attempts were made to generaize the concept of target maps to semidefinite programming SDP) [10, 11, 16], symmetric cone programming [5, 18] and genera convex conic programming [19]. We present a target map and a target-foowing framework for SDP, from which we derive weighted path-foowing agorithms and target-foowing agorithms with provabe poynomia worse-case iteration bounds. Our target map is based on the notion of Choesky weighted anaytic centers first introduced by the author in [3]. In recent reports [, 3], the author anayzed the convergence behavior of the weighted centra paths corresponding to the Choesky weighted centers. In these reports, the study of Choesky weighted centers were mainy motivated by homogeneous cone programming: the centra path for a homogeneous cone programming probem coincide with certain weighted centra path of a particuar SDP-representation of the probem. In this paper, we expore a different aspect of Choesky weighted centers: the target map derived from these weighted centers. We present a generic target-foowing framework based on this target map, and anayze the iteration compexity of target-foowing agorithms based

3 TARGET FOLLOWING ALGORITHMS 3 on two distinct choices of search directions, and weighted path-foowing agorithms of three distinct favors Organization of materia. This paper is organized as foows. We begin section with a generic target-foowing framework based on the target map derived from Choesky weighted centers. These weighted centers were first introduced by the author in [3], and are reated to a notion of weighted centers studied by Monteiro and Zanjácomo [1] in a genera framework. We present a different perspective on these weighted centers that reates them with anaytic centers of arger SDP probems which we caed expanded SDP probems. We define a measure of proximity to the Choesky weighted centers based on the -proximity measure for the expanded SDP probems. We aso show that search directions for the expanded SDP probems, which transate naturay to search directions for the origina SDP probems, can be efficienty computed. In section 3, we use the search directions from the Monteiro-Zhang famiy [9, 13] in our target-foowing framework to produce a target-foowing agorithm. We further consider two weighted path-foowing agorithms: a short-step agorithm and a predictor-corrector agorithm. Our anayses on these agorithms show that both take O nρ) iterations to improve the duaity gap by a fixed fraction, where ρ denotes the ratio of the average weight to the smaest weight. These bounds parae their counterparts in inear programming. However the computation of search directions in each iteration may require the soving for Θn 3 ) rea variabes, in contrast with On ) variabes in a reguar path-foowing agorithm. This issue is addressed in section 4, where we reduce the size of the Newton system to match that of a typica Newton system in a reguar path-foowing agorithm. This is achieved with a specific choice of search directions, which we caed the Choesky search directions. These search directions were discussed in a genera framework by Burer and Monteiro [1], with which they buit a ong-step path-foowing agorithm. Their anaysis was based on the derivatives of the map X, S) L T SXL S, where L S denotes the Choesky factor of S. In contrast, we use the oca Lipschitz property of the Choesky factorization X L X. In section 5, we investigate the appication of our target-foowing framework in the approximation of anaytic centers. We work in a subset of the target space containing ony diagona matrices, hence our investigation is very cosey reated, and directy appicabe, to the work of Mizuno [8] on inear compementarity probems and the work of Jansen et. a. [6] on inear programming. From a given pair of prima-dua stricty feasibe soutions, we generate a finite sequence of targets towards the pair of soution on the centra path with the same duaity gap as the given pair. Using a technique first deveoped by Todd [17] for inear programming, and subsequenty used by Nesterov and Todd [15], and Nemirovski and Nesterov [14] for genera convex conic programming, we derive an upper bound on the number of targets in the sequence. For SDP probems, we obtained the improved worse-case iteration bound O n og ρ). For inear programming probems, this bound is an improvement over the existing best bound O nog ρ + og ρ)), where ρ [1, n] denotes the ratio of the argest weight to the average weight see [6, 8]). 1.. Notations and conventions. Throughout this paper, we use the foowing notations and conventions.

4 4 C. B. CHUA We use uppercase bod etters e.g., X, L, etc.) to denote matrices, and use owercase bod etters e.g., y, b, etc.) to denote vectors. The space of rea n-vectors is denoted by R n, and the cone of vectors in R n with nonnegative resp., positive) entries is denoted by R n + resp., R n ++). The cone of vectors in R n ++ with entries in nonincreasing order is denoted by R n,++. The space of rea n-by-n matrices is denoted by M n. We equip M n with the inner product : A, B) M n M n tra T B). The induced norm F is the Frobenius norm. The direct sum of matrix spaces M n 1 M n k is equipped with the inner product A, B) k A B. =1 The transposes, inverses, products and Choesky factors of tupes in the direct sum are defined componentwise. The subspace of ower trianguar resp., upper trianguar) matrices in M n is denoted by L n resp., U n ). For any matrix M M n, the unique ower trianguar matrix L satisfying M L U n and L ii = M ii / for i = 1,..., n, is denoted by M. For any matrix M M n, we denote by M H the symmetric matrix M + M T. Consequenty M H denotes the unique symmetric matrix whose entries in the ower trianguar part coincide with those of M. For any symmetric, positive definite matrix X S n ++, its unique Choesky factor i.e., the unique ower trianguar matrix L L n with positive diagona entries satisfying LL T = X) is denoted by L X. The group of orthogona matrices in M n is denoted by O n. The space of symmetric matrices of order n is denoted by S n, and the cone of symmetric, positive semidefinite resp., positive definite) matrices of order n is denoted by S n + resp., S n ++). The subspace of diagona matrices in S n is denoted by D n, and its intersection with S n + and S n ++ are, respectivey, denoted by D n + and D n ++. The cone of matrices in D n ++ with diagona entries in nonincreasing order is denoted by D n,++. For each diagonaizabe matrix M M n, we denote by λm) the vector of eigenvaues of M in nonincreasing order. For any m-by-n matrix M and any subsets of indices I {1,..., m} and J {1,..., n}, the sub-matrix of M with row indices in I and coumn indices in J is denoted by M IJ. If I = {i} resp., J = {j}) is a singeton, we may aso write i resp., j) in pace of {i} resp., {j}). For any matrix M, we denote by [M] i the square sub-matrix M {1,...,i},{1,...,i}. The zero matrix and the identity matrix of appropriate size in the context used) are denoted, respectivey, by 0 and I. The vector of ones of appropriate size in the context used) is denoted by 1. For each inear map A : E F between two Eucidean spaces, A H : F E denotes its adjoint map. For each sequence x 1,..., x n of rea numbers, Diagx 1,..., x n ) denotes the diagona matrix in D n with x 1,..., x n on its diagona. For each matrix M M n, we denote by diagm) the vector [M 11,..., M nn ] T R n.

5 and TARGET FOLLOWING ALGORITHMS 5 For each pair of rea numbers x, y), we denote by x y the greater of the two.. Target-Foowing Framework We consider the foowing pair of prima-dua SDP probems: inf X C X subject to A k X = b k 1 k m), X S n +, sup S,y subject to b T y m y k A k + S = C, S S n +, k=1 SDP ) SDD) where A 1,..., A m, C S n and b R m are given. We assume there exists prima-dua stricty feasibe soutions X, Ŝ); i.e., a pair of primadua feasibe soutions in S n ++ S n ++. Consider the target map T : S n ++ S n ++ S n ++ defined by X, S) QDQ T, where Q T XSQ = D + U U n is a Schur-decomposition of XS with diagu) = 0, and D D n,++. Theorem 1. The map T is we defined. Moreover, it is a bijection between the cone S n ++ and the set of prima-dua stricty feasibe soutions of the prima-dua pair SDP, SDD). Proof. See [3, Theorem 10]. Using the target map T, we propose the foowing genera framework for target-foowing agorithms: Agorithm 1. Target-foowing framework for SDP) Given a pair of prima-dua stricty feasibe soutions X in, S in ). 1) Find a target W + S n ++ cose to T X in, S in ). Set X +, S + ) = X in, S in ). ) Repeat the foowing: a) Pick W ++ S n ++ cose to W +. b) Compute a pair of prima-dua stricty feasibe soutions X ++, S ++ ) that approximates T 1 W ++ ). c) Update X +, S + ) X ++, S ++ ) and W + W ++. The two main steps in this framework are the choosing of W ++ and the computation of approximate soutions X ++, S ++ ). The sequence of W ++ chosen is caed the sequence of targets, and the sequence approximate soutions X ++, S ++ ) computed is caed the sequence of iterates. In the next section, we consider the probem of computing the next pair of iterates. Foowing that, we address the issue of choosing the next target W ++.

6 6 C. B. CHUA.1. Expanded semidefinite programming probems. For the sake of carity, we assume that W ++ is the diagona matrix D ++ D n,++. This is without any oss of generaity as we can transform our prima-dua SDP probems via the orthonorma simiarity transformation X, S) Q T XQ, Q T SQ), where Q O n is such that Q T W ++ Q = D ++ is a diagonaization of W ++. Consider the pair of Choesky weighted centers T 1 D ++ ): the unique pair of matrices X, S) satisfying A k X = b k 1 k m), X S n ++, m y k A k + S = C, S S n ++, CP D++ ) k=1 L T SXL S = D ++. Suppose further that a entries in D ++ are rationa numbers. Then there exists a positive rea number κ and positive integers w 1,..., w n such that D ++ = κ Diagw 1,..., w n ). Reca that w 1 w n. For each {1,..., n 1}, et π denote the difference w w +1, and et π n = w n. Let L denote the index set { : π > 0}. Note that L {n} is nonempty. Let S denote the direct sum } S 1 {{ S } 1 S } {{ S } S } n {{ S n }, π 1 copies π copies π n copies { and et P denote the index set j=1 π j,..., } j=1 π j for each {1,..., n} so that the p-th component X p of any eement X S is a matrix in S whenever p P. Note that P is empty when / L. Let S + and S ++ denote cones of S containing eements with positive semidefinite and positive definite components, respectivey. Define the injective inear map E : S n S by Its adjoint map E H satisfies X [X] 1,..., [X] }{{} 1, [X],..., [X],..., [X] }{{} n,..., [X] n ). }{{} π 1 copies π copies π n copies E H X)) ij = n =i j p P X p ) ij 1 i, j n). Let A 1,..., A m and C denote, respectivey, EA 1 ),..., EA m ) and EC). Consider the expanded prima-dua SDP probems inf X subject to w 1 p=1 w 1 p=1 C p X p A k ) p X p = b k 1 k m), X S +, SDP )

7 and sup S,y subject to TARGET FOLLOWING ALGORITHMS 7 b T y m y k A k ) p + S p = C p 1 p w 1 ), S S +. k=1 SDD) Let us ook at the pair of anaytic centers XκI), SκI)) of the pair of probems SDP, SDD): the unique pair of tupes X, S) satisfying the centra path equations w 1 p=1 A k ) p X p = b k 1 k m), X S ++, m y k A k ) p + S p = C p 1 p w 1 ), S S ++, k=1 X p S p H = κi 1 p w 1 ). Let X and Ŝ denote, respectivey, EH XκI)) and E 1 SκI)). It is straightforward to check that v R n v T Xv, whence X, is positive definite, and that Ak X = b k for each k {1,..., m}. Thus X is stricty feasibe for SDP ). Aso, Ŝ = SκI) w 1 S n ++ and m k=1 y ka k ) w1 + SκI) w1 = C w1 shows that Ŝ is stricty feasibe for SDD). Moreover, the biinear equations in the centra path equations impy that XŜ H = D ++, or equivaenty, L Tb S XL b S = D ++. Thus E H XκI)), E 1 SκI))) is the pair of Choesky weighted centers T 1 D ++ ). This observation aows us to view Choesky weighted centers as unweighted) anaytic centers of a pair of arger prima-dua SDP probems. Moreover, a existing path-foowing agorithms and their anayses appy directy to Choesky weighted centers via this observation. It is immediatey cear that without further expoitation of the specia structures of the expanded probems, this approach is i-advised as dims) = n =1 w, the size of the ex-, the size of the origina pair. This much arger size affects computationa compexity of the resuting agorithm in two ways: 1) the step size at each iteration, hence the worse-case iteration bound, and ) the compexity of the computation of search directions. panded pair, is generay) much arger than dims n ) = n = Proximity measure. We sha use the foowing measure of proximity to anaytic centers of the expanded SDP probems: w1 d : X, S; µ) S ++ S ++ R ++ µ 1 λx p S p ) µ1 p=1 p=1 w1 = µ 1 L T S p X p L Sp µi F.

8 8 C. B. CHUA The definition of d ony requires S S ++. Moreover, we can extend its definition continuousy to incude a S S + \ S ++. Thus d is we defined over S S + R ++. This eads to the foowing measure of proximity to T 1 D ++ ): X, S) inf X { d X, ES); κ) : E H X) = X}. We compute the infimum in Lemma using the foowing emma. Lemma 1. Suppose that u 1 u n > u n+1 = 0, X, S) S n S n ++, and µ > 0. Then for every sequence of symmetric matrices {X S } n =1 satisfying it hods X ij = n u u +1 )X ) ij 1 i, j n),.1) =i j n u u +1 ) [L S ] T X [L S ] µi F =1 n u 1 i j LT SXL S ) ij µu i I ij ) = D 1 L T SXL S µd ) H F, where D denotes the diagona matrix Diagu 1,..., u n ). Moreover, equaity hods if and ony if where L denotes the set { : u > u +1 }. X = [L S ] T [D 1 L T SXL S ) H ] [L S ] 1 L,.) Proof. For each L, et Z = [L S ] T X [L S ]. In terms of Z, n u u +1 ) [L S ] T X [L S ] µi F =1 = u u +1 ) = n, i j Z ) ij µi ij ) u u +1 )Z ) ij µi ij ). Using Cauchy s inequaity, we bound, for each i, j {1,..., n}, u u +1 )Z ) ij µi ij ), i j, i j u u +1 ) ) 1, i j u u +1 )Z ) ij µi ij ))..3)

9 The first part of the emma then foow from n u u +1 )Z ) ij = u u +1 ), i j for a i, j {1,..., n}. Equaity in.3) hods if and ony if i.e., there exists Z S n such that TARGET FOLLOWING ALGORITHMS 9 = = =i j n eı=i n eı=i n ej=j =i j eı=i L S ) eıi X ) eıej L S ) ejj ej=j n u u +1 )L S ) eıi X ) eıej L S ) ejj n L S ) eıi X eıej L S ) ejj = L T SXL S ) ij, ej=j Z ) ij = Z e ) ij, L {i j,..., n}; Z = [Z] L..4) By.1), any Z S n satisfying the above set of equations must aso satisfy X ij = u u +1 )[L S ] T [Z] [L S ] 1 ) ij 1 i, j n)..5), i j Let F : S n S be restriction of the map X [X] 1,..., [X] n ) to the subspace S. Consider the foowing inner product on S : A, B) u u +1 )A B. Under this inner product, the adjoint F H of F satisfies n F H X)) ij = u u +1 )X ) ij 1 i, j n), =i j hence.5) is equivaent to X = F H FL S ) T FZ)FL S ) 1 ). For a W S n, F H FL S ) T FZ)FL S )) W = u u +1 )[Z] [L S ] [W] [L S ] T = u u +1 )[Z] [L S WL T S] = L T SF H FZ))L S ) W, hence.5) is equivaent to X = L T S F H FZ))L 1 S. The map F is ceary injective, hence F H F) 1 is bijective. Subsequenty the ony Z S n satisfying.5) is F H F) 1 L T SXL S ) = D 1 L T SXL S ) H, where the equaity foows from F H F : V S n D V ) H. Consequenty equaity in.3) hods if and ony if.) hods.

10 10 C. B. CHUA Lemma. Suppose X, S) S n S n ++ and µ > 0. Then { inf d X, ES); µ) : E H X) = X} X = d ELS ) T EκD 1 ++ L T SXL S ) H )EL S ) 1, ES); µ ) = µ 1 n w 1 i j LT SXL S ) ij µw i I ij ) = µ 1 κd 1 ++ L T SXL S µκ 1 D ++ ) H F. Proof. Let S = ES) and et X E H ) 1 X) be arbitrary. For each L, it foows from Cauchy s inequaity and the triange inequaity on the Frobenius norm that ) L T S p X p L Sp µi π 1 L T S p X p L Sp µi F p P where X denotes the average π 1 inf X F π 1 p P p P ) L T S p X p L Sp µi = π L T [S] X L [S] µi F, p P X p. Since X is arbitrary, it foows { d X, ES); µ) : E H X) = X} = inf X { d X, ES); µ) : E H X) = X, X p = X q p, q P, L}. Thus we may assume without oss of generaity that for each L and a p P, X p = X. The proposition then foows from Lemma 1. F The foowing emma shows that under the proximity measure { X, S) S n S n + inf d X, ES); κ) : E H X) = X} X = κ 1 n w 1 i j LT SXL S ) ij κw i I ij ),.6) soutions on the boundary of the prima-dua feasibe regions are at a distance at east κwn = D ++ ) nn from T 1 D ++ ). This suggests scaing the measure.6) by D ++ ) 1 nn. Lemma 3. If u 1 u n > 0, µ > 0, and Z S n, then n n u 1 i j Z ij µu i I ij ) u 1 i λz) i µu i ). i=1

11 Consequenty inf { µ 1 n TARGET FOLLOWING ALGORITHMS 11 u 1 i j Z ij µu i I ij ) : Z S n + \ S n ++ } = µu n. Proof. By expanding both sides of the desired inequaity, it is cear that we ony need to bound the sum n u 1 i j Z ij from beow by n i=1 u 1 i λz) i. Since Z is symmetric, there exists an orthogona matrix Q O n such that Z = QDiagλZ)) Q T, which eads to Z 1 Q 11 Q 1n λz) 1. =......, Z ) nn Q n1 Q nn λz) n where the matrix on the right side of the above equation is douby-stochastic. By the Hardy, Littewood and Póya theorem [4], we have Z ) ii i=1 for a {1,..., n}. Consequenty by writing n and using the upper bounds u 1 i j Z ij = u 1 n Z ij λz) i i=1 n n 1 Z ij u 1 +1 u 1 Z ) ii i=1 we concude the desired inequaity n hence proving the theorem. u 1 i j Z ij =1 λz) i i=1 n i=1 u 1 i λz) i, ) 1 n), Z ij, We sha use the scaed proximity measure d : S n S n + D n,++ R by X, S; D) D 1 nn = D 1 nn n ) D 1 i j,i j L T SXL S ) ij D ij D 1 L T SXL S D ) H F. Note that we do not restrict D to have rationa entries ony. When we restrict D to be a positive mutipe of the identity matrix I, the proximity measure naturay reduces to the standard -proximity measure.

12 1 C. B. CHUA When D has rationa entries with D = κ Diagw 1,..., w n ), κ positive and w 1,..., w n integers, this proximity measure can be written { d X, S; D) = w 1 n inf d X, ES); κ) : E H X) = X} X.7) = w 1 d n EL S ) T EκD 1 L T SXL S ) H )EL S ) 1, ES); κ). In the context of the expanded SDP probems, the scaing factor wn 1/ corresponds to the use of a arger neighborhood size, hence aowing for arger step sizes, or fu steps with farther targets. This scaing factor is further justified in the foowing theorem. Theorem. Suppose that α [0, 1] X α, S α, µ α ) S n S n R ++ is continuous with X 0, S 0 ) S n ++ S n ++. If there exist D D n,++ and β < 1 such that µ 1 α D 1 L T S α X α L Sα µ α D ) H F β D nn.8) whenever S α S n ++, then X α, S α ) S n ++ S n ++ for a α [0, 1]. Proof. Let α = inf{α [0, 1] : X α, S α ) / S n ++ S n ++}. Suppose on the contrary α 1. By the continuity assumption, α > 0. Under the hypothesis.8), n ) D 1 i j,i j L T S α X α L Sα ) ij µ α D ij β D nn µ α for a 0 α < α, and subsequenty n ) im inf µ 1 α D 1 i j,i j L T S α bα α X α L Sα ) ij µ α D ij β D nn µ bα < D nn µ bα. On the other hand, Lemma 3 impies n ) im inf µ 1 α D 1 i j,i j L T S α bα α X α L Sα ) ij µ α D ij Dnn µ bα, a contradiction..1.. Search directions. In this section, we discuss the computation of search directions. Once again, we use the pair of expanded SDP probems SDP, SDD) for our purpose. As discussed in the preceding section, we may and shoud) use X +, S + ) = EL S ) T EκD 1 ++ L T SXL S ) H )EL S ) 1, ES)) { = argmin d X, S; κ) : X +, S + ) = E H X), E S))} 1 X,S as the pair of current iterates for the expanded SDP probems. The pair of search directions X, S ) for the expanded SDP probems is obtained by inearizing the constraints H X p, S p ) = κi L, p P ) at X +, S + ), for some maps {H : S ++ S ++ S : L} satisfying H X, S) = µi XS = µi

13 for a µ > 0. In another words, X, S ) soves w 1 p=1 TARGET FOLLOWING ALGORITHMS 13 A k ) p X ) p = 0 1 k m),.9a) m y ) k A k ) p + S ) p = 0 1 p w 1 ),.9b) k=1 DH X + ) p, S + ) p ) X ) p, S ) p ) = κi H X + ) p, S + ) p ) L, p P ),.9c) where DH X + ) p, S + ) p ) denotes the gradient of H at X + ) p, S + ) p ); i.e., the inear map U, V) S S X H X + ) p, S + ) p )[U] + S H X + ) p, S + ) p )[V]. This inear system has n =1 w +m = Θ π ) variabes. The pair of search directions for the origina SDP probems is then given by X, S ) = E H X ), E 1 S )). By the choice of X +, there exists, for each L, X + ) S such that X + ) = X + ) p p P ). By symmetry, it foows that there is a search direction X such that for each L, X ) p = X) p P ) for some X) S. Thus we may compute the search directions by soving the smaer system n π [A k ] X) = 0 1 k m),.10a) =1 m y ) k A k + S = 0,.10b) k=1 DH X + ), S + ) ) X), [ S ] ) = κi H X + ), S + ) ) L), where S + ) denotes [S + ], and set X ) ij =, i j.10c) π X) ) ij.11) for each i, j {1,..., n}. This system has Θ ) = On 3 ) variabes. The number of variabes is actuay Ωn 3 ) in certain cases; e.g., when L = {1,..., n}. It is necessary for the above system to have unique soution so that the search directions are we defined. Since distinct soutions of the above system give distinct soutions to.9), this requirement is satisfied by.9) having unique soution. Typicay, sufficient conditions for this is given by X +, S + ) S ++ S ++, and at times together with the existence of some µ > 0 such that λx + ) p S + ) p ) µ1 γµ p {1,..., w 1 },

14 14 C. B. CHUA where γ 0, 1) is given. As S + S n ++ impies S + S ++, the above condition is sufficient for X + S ++. The foowing emma shows that this condition is satisfied when X +, S + ) is sufficienty cose to T 1 D ++ ). Lemma 4. It hods for a p {1,..., w 1 }. Proof. By definition, λx + ) p S + ) p ) κ1 κd X +, S + ; D ++ ) κd X +, S + ; D ++ ) = w 1 n w1 L T S + ) p X + ) p L S+ ) p κi F p=1 w 1 n L T S + ) p X + ) p L S+ ) p κi F p P n = V κi F, where V denotes the matrix κd 1 ++ L T SXL S ) H. Consequenty for any p {1,..., w 1 }, κd X +, S + ; D ++ ) V κi F [V] κi F where L is such that p P. = L T S + ) p X + ) p L S+ ) p κi F = λx + ) p S + ) p ) κ1,.. Choice of targets. Suppose that the pair of input matrices X in, S in ) satisfies L T S in X in L Sin D n,++. This is without oss of generaity if we appy the orthonorma simiarity transformation defined by the orthogona matrix that upper-trianguarizes the product X in S in to both prima and dua SDP probems. We first consider the task of picking the initia target W +. Using the proximity measure d, the proximity of T X in, S in ) to W + can be quantified by d Q T +X in Q +, Q T +S in Q + ; D + ), where Q + O n and D + D n,++ are such that QT +W + Q + = D + is a diagonaization of W +. By Lemma 3, for each fixed D + D n,++, the above measure is minimized at Q + = I. Thus it makes sense to pick W + D n,++. Henceforth, we sha assume that W + is the diagona matrix D + D n,++. We now consider the task of picking the next target W ++. Once again, the next target W ++ shoud thus be chosen so that d Q T ++X + Q ++, Q T ++S + Q ++ ; D ++ ) can be readiy bounded, where Q ++ O n and D ++ D n,++ are such that QT ++W ++ Q ++ = D ++ is a

15 TARGET FOLLOWING ALGORITHMS 15 diagonaization of W ++. An natura criterion woud be the size of d W ++, I; D + ). Once again, since Lemma 3 impies that inf{d Q T D ++ Q, I; D + ) : Q O n } = d D ++, I; D + ) = D + ) 1 nn D ++ D 1 + D 1 + F, it makes sense to choose W ++ D n,++. With these choices of targets, we can use the foowing emma to get an upper bound on d X +, S + ; D ++ ) in terms of d X +, S + ; D + ) and d D ++, I; D + ). Lemma 5. If d X +, S + ; D + ) β and D + ) 1 nn D ++ D 1 + D 1 + F δ for some β, δ 0, 1), then d X +, S + ; D ++ ) β + δ 1 δ. Proof. For simpicity of notation, et Z denote the product L T S + X + L S+. From definition, with d X +, S + ; D ++ ) n = D ++ ) 1 nn D ++ ) 1 i j,i j Z ij D ++ ) ij ) D ++ ) 1 nn n D + ) 1 i j,i j Z ij D ++ ) ij ) n D + ) 1 i j,i j Z ij D ++ ) ij ) n D + ) 1 i j,i j Z ij D + ) ij ) max i=1,...,n D+ ) ii D++ ) ii + D ++ D 1 + D 1 + F = D + ) nn d X +, S + ; D + ) + D + ) 1 nn D ++ D 1 + D 1 + F ). If D + ) 1 nn D ++ D 1 + D 1 + F δ, then δ D + ) 1 nn n i=1 for a i {1,..., n}, and hence D ++ ) ii ) D + ) ii D+ ) ii 1 D ++ ) ii 1 D + ) ii min i=1,...,n D++ ) ii D+ ) ii 1 δ.

16 16 C. B. CHUA Consequenty, d X +, S + ; D ++ ) D ++ ) 1 nn D+ ) nn β + δ) max i=1,...,n under the hypotheses of the emma. D+ ) ii D++ ) ii β + δ 1 δ 3. An Exampe Based on the Monteiro-Zhang Famiy As an iustration of the discussion in section, we appy search directions from the Monteiro-Zhang famiy to the target-foowing framework. We reca that the Monteiro-Zhang famiy of search directions is the set of search directions derived from the maps parameterized by P S n ++. The agorithm is given as foows: H : X, S) 1 PXSP 1 ) H Agorithm. Target-foowing agorithm based on Monteiro-Zhang search directions) Given a pair of prima-dua stricty feasibe soutions X in, S in ) with T X in, S in ) D n,++. 1) Find a target D + D n,++ satisfying d X in, S in ; D + ) β for some β 0, 1). Set X +, S + ) = X in, S in ). ) Repeat the foowing: a) Pick target D ++ D n,++ D + ) 1 nn for some δ 0, 1). Write with rationa entries and D ++ D 1 + D 1 F + δ D ++ = κ Diagw 1,..., w n ), where κ R ++ and w 1,..., w n are positive integers. Set π = w w +1 = 1,..., n 1), π n = w n and L = { : π > 0}. b) Set X = [L S+ ] T [κd 1 ++ L T S + X + L S+ ) H ] [L S+ ] 1 and S = [S + ] for each L. Pick nonsinguar matrices {P M } and sove.10) with X + ), S + ) ) = X, S ) and H : X, S) P XSP 1 ) H. Set X ++, S ++ ) = X + + X, S + + S ), where X denotes the matrix in S n satisfying X ) ij = π X) ) ij 1 i, j n)., i j c) Update X +, S + ) X ++, S ++ ) and D + D ++.

17 TARGET FOLLOWING ALGORITHMS Anaysis of agorithm. For the anaysis, we consider one iteration of the agorithm. We begin with various technica resuts of Monteiro [9]. Lemma 6. Suppose that X, S) S n S n + and P M n is a nonsinguar matrix. Then for any µ R, a) λ X S) µ1 = λxs) µ1, where X = PXP T and S = P T SP 1 ; b) λxs) µ1 1PXS µi)p 1 ) H F with equaity hoding when PXSP 1 S n. Proof. See proof of [9, Lemma.1]. Proposition 1. Suppose P M n is a nonsinguar matrix and A 1,..., A m S n are ineary independent. If X, S) S n ++ S n ++ is such that for some µ > 0, then the system λxs) µi µ A k dx) = 0 k = 1,..., m), m dy) k A k + ds = 0, k=1 P[dX)S + XdS)]P 1 ) H = σµi PXSP 1 ) H has unique soution for every σ R. Proof. As the system 3.1) is square, this emma is equivaent to Lemma 3. of [9]. 3.1a) 3.1b) 3.1c) Lemma 7. If X, S) S n ++ S n ++ and dx, ds) satisfies 3.1c) for some nonsinguar P M n and some σ, µ R, then for every θ R, it hods X 1 dx) S + X ds) + X S σµi X 1 1 δ x X S θµ X 1 1 S 3.) F and X 1 X + α dx)) S + α ds)) 1 α + ασ)µi X F 1 α) λ X S) µ1 + α δ x δ s + α 1 1 δ x X S θµ X 1 1 S 3.3) for a α [0, 1], where X, S, dx and ds denote, respectivey, PXP T, P T SP 1, PdXP T and P T dsp 1, and δ x := X 1 dx) S 1, δ s := S 1 ds) X ) F F Proof. See proof of [9, Lemma 3.6].

18 18 C. B. CHUA Lemma 8. If X, S) S n ++ S n ++ satisfies λxs) µ1 γµ for some µ > 0 and some γ 0, 1), then X 1 S γ)µ. 3.5) Proof. See proof of [9, Lemma 3.7] The foowing is an adaptation of a main resut of Monteiro [9] to the expanded SDP probems. Lemma 9. If d X +, S + ; D ++ ) γ for some γ 0, 1) satisfying γ 1 γ 1, 3.6) then the inear system.10), with X + ), S + ) ) = X, S ), H : S S S defined by X, S) P XSP 1 ) H, and κ repaced by σκ for some σ [0, 1], has a unique soution X), S, y ). Moreover for every α [0, 1], P π X P T ) 1 P [X,α S,α κ α I]P 1 P X P T { 1 α)γ + α χγ } 1 γ + χ wn 4α κ, 3.7) 1 γ) where X,α, S,α and κ α denote, respectivey, X + α X), S + α[ S ] and 1 α + ασ)κ, and { σd X +, S + ; σd ++ ) if σ > 0, χ = κ wn 1 π λx S ) if σ = 0. Proof. It is straightforward to deduce from 3.6) that γ < 1/. It thus foows from Lemma 4 and Proposition 1 that the Newton system.9), whence.10), has unique soution. For the second part of the emma, we sha adapt the proofs of Lemmas 3.8 and 3.9 of [9]. For each L, et X, S, X) and S) denote, respectivey, P X P T, P T P T [ S ] P 1, F S P 1, P X) P T and et δ x, and δ s, be defined by 3.4) with X, S) and dx, ds) repaced by X, S ) and X), S) ), respectivey, and et V,θ and W denote, respectivey, the matrices X 1 1 S θκ X 1 S 1 and X 1 [ X) S + X S) + X S σκi] X 1.

19 Using these notations, we have ) π δ x, + δs, = π X 1 TARGET FOLLOWING ALGORITHMS 19 = π W X 1 π W X 1 X) S 1 + X 1 S) S 1 S 1 V,σ S 1 F F ) + V,σ F, F where we have used.10a) and.10b) to deduce π tr X) S) = π tr X) [ S ] = 0 in the first equation. The triange inequaity on the -norm of R n further bounds ) π δ x, + δs, W π X 1 S 1 + π V,σ F. By Lemmas 4 and 6, d X + S + ; D ++ ) γ = L, λ X S ) κ1 = λx S ) κ1 γκ. Thus we may appy 3.) and 3.5) with for each L to bound, for a θ R, and F X, S, dx, ds, P, µ) = X, S, X), [ S ], P, κ) V,θ F X 1 S γ)κ π V F 1 1 γ)κ π W X 1 S 1 F S 1 1 X S θκi F λ X S ) θκ1, 3.8) π λx S ) σκ1 = χ w n κ 1 γ), π W F X 1 S 1 π δx, V,1 1 F 1 γ)κ 1 π 1 γ) κ δx, λ X S ) κ1 γ π 1 γ) δx,.

20 0 C. B. CHUA Thus, under the hypothesis 3.7), { max π δx,, π δs, } 1 ) π δ x, + δs, whence { max π δx,, } π δs, We can actuay bound each δ s,. First note that γ π δx, + χ w n κ 1 γ 1 γ { 1 max π δx,, } 1 π δs, + χ w n κ, 1 γ δ s,n = w n π n δ s,n 4 χ 1 γ κ. For the remaining δ s, s, observe that for each L, δs, = S 1 S) X 1 1 = tr S and thus We may then use to bound for each L. F 4 χ 1 γ w nκ. 3.9) S) X S) = tr S 1 [ S ] X [ S ] = tr ) ) ) L 1 S [ S ] L T S L T S X L S L 1 S [ S ] L T S, δ s, κ L 1 S [ S ] L T S F tr ) L 1 S [ S ] L T S L T S X L S κi ) L 1 S [ S ] L T L 1 S [ S ] L T S F γκ L 1 S [ S ] L T S F. L 1 S [ S ] L T S L T S X L S κi = L 1 [S + ] [ S ] L T [S + ] = [L 1 S + S L T S + ] δs, 1 + γ)κ [L 1 S + S L T S + ] 1 + γ)κ χ F L 1 S ) S + S L T S γ 1 γ δ s,n 4 1 γ) κ 3.10) 3 F

21 TARGET FOLLOWING ALGORITHMS 1 Appying 3.3) with X, S, dx, ds, µ, θ) = X, S, X), S), κ, 1) for each L gives X 1 X + α X) ) S + α 1 S) ) 1 α + ασ)κi ) X F 1 α) λ X S ) κ1 + α δ x, δ s, + α δ x, V,1, and thus the triange inequaity on the -norm of R n impies π X 1 nx + α ) X) S + α ) S) κ α I 1 α) π λ X S ) κ1 + α π δx, V,1 Using 3.8), 3.9) and 3.10), we further bound and Finay π δ x,δ s, max δ s,. + α 1 ) X π δ x,δ s, π δx, χ γ) w nκ 4 π δ x, V,1 1 1 γ)κ π λ X S ) κ1 competes the proof. γ κ 1 γ F π δx, λx S ) κ1 F π δ x, 4 χ γ 1 γ) w nκ. = w n κ d X +, S + ; D ++ ) γ w n κ The foowing theorem shows that with suitabe choices of the parameters β and δ, Agorithm is we defined. Theorem 3. If β, δ 0, 1) satisfies γ 1 γ + 4 γ β, 3.11) 1 γ) where γ = β +δ)/1 δ), then in each iteration of Agorithm, the search directions are we defined. Moreover, in each iteration, the iterates are prima-dua stricty feasibe soutions satisfying d X +, S + ; D + ) β.

22 C. B. CHUA Proof. We sha prove the theorem by induction on the iterations. Suppose that at the beginning of an iteration, the iterates X +, S + ) are stricty feasibe and d X +, S + ; D + ) is at most β. This is certainy true for the first iteration. By the choice of D ++ and Lemma 5, we have d X +, S + ; D ++ ) γ, where γ = β + δ)/1 δ). If 3.11) hods with β < 1, then it is straightforward to check that γ satisfies 3.6). Consequenty, by Lemma 9, the search directions X and S are we defined. Let X,α and S,α denote, respectivey, the sums X + α X) and S + α[ S ]. By.7), we have ) 1/ d X + + α X, S + + α S ; D ++ ) κ 1 w 1 n π λ X,α S,α ) κ1 whenever S + + α S S n ++. Appying Lemma 6 with X, S) = X,α, S,α ), µ = κ and P = P := P X P T P gives ) 1/ κ 1 w 1 n π λ X,α S,α ) κ1 which is no more than κ 1 w 1 n κ 1 w 1 n π 1 ) P X,α S,α κi) P 1 ) P 1 π P X,α S,α κi F H F / /, by the triange inequaity on the Frobenius norm. Thus the inequaity 3.7) with σ = 1 shows that d X + + α X, S + + α S ; D ++ ) 1 α)γ + α γ 1 γ + γ 4α 1 γ) whenever S + + α S S n ++. Under the hypothesis 3.11), the above upper bound is at most 1 α)γ + αβ < 1 for a α [0, 1]. We then concude from Theorem that the next pair of iterates X ++, S ++ ) = X + + X, S + + S ) are positive definite, whence stricty feasibe as they ceary satisfy the inear equations in their respective SDP probems. Finay, the induction is competed by observing that the upper bound 1 α)γ + αβ) is precisey β when α = Weighted path-foowing agorithms. In this section, we describe two weighted path-foowing agorithms using the above target-foowing framework based on the Monteiro- Zhang famiy of search directions. The first is a short-step agorithm that is actuay a specia case of the above target-foowing framework. The next is a weighted path-foowing version of the Mizuno-Todd-Ye MTY) predictor-corrector agorithm. The anayses of both agorithms demonstrate the same worse-case iteration bound of O nρ ogε 1 )) to obtain

23 TARGET FOLLOWING ALGORITHMS 3 an pair of prima-dua feasibe soutions X out, S out ) satisfying X out S out εx in S in, where ρ denotes the ratio X in S in nλx in S in ) n. We begin with the foowing generic weighted path-foowing agorithm: Agorithm 3. MZ weighted path-foowing agorithm) Given a pair of prima-dua stricty feasibe soutions X in, S in ) with T X in, S in ) D n,++, and the required accuracy ε > 0. 1) Find a target D + D n,++ with rationa entries and d X in, S in ; D + ) β for some β 0, 1). Set X +, S + ) = X in, S in ). Write D + = κ + Diagw 1,..., w n ) where κ + R ++ and w 1,..., w n are positive integers. Set π = w w +1 = 1,..., n 1), π n = w n and L = { : π > 0}. Set D = Diagw 1,..., w n ). ) Whie X + S + > εx in S in ), a) Pick σ [0, 1]. b) Set X = [L S+ ] T [ D 1 L T S + X + L S+ ) H ] [L S+ ] 1 and S = [S + ] for each L. Pick nonsinguar matrices {P M } and sove.10) with X + ), S + ) ) = X, S ), H : X, S) P XSP 1 ) H, and κ = σκ +. For each α [0, 1], et X α, S α ) = X + +α X, S + +α S ), where X denotes the matrix in S n satisfying X ) ij = π X) ) ij 1 i, j n),, i j and et κ α = 1 α + ασ)κ +. Pick β 0, 1). Pick α [0, 1] such that S bα S n ++ and d X bα, S bα ; κ bα D) β. c) Update X +, S + ) X bα, S bα ) and κ + κ bα. 3) Output X out, S out ) = X +, S + ) Short-step agorithm. Let ρ denote the ratio w i /nw n ). The short-step agorithm uses the choices σ = 1 δnρ) 1 for a fixed constant δ 0, 1), and β = β throughout a iterations. Theorem 4. If β, δ 0, 1) satisfies the hypothesis of Theorem 3, then in each iteration of Agorithm 3, with σ = 1 δnρ) 1, the search directions are we defined and we may use α = 1. Moreover, with this choice of α, the agorithm terminates after at most O nρ ogε 1 )) iterations. Proof. If we define, in each iteration, D + = κ + D and D++ = σκ + D, then it is straightforward to check that d D ++, I; D + ) = δ, whence the proof of Theorem 3 shows that we may use α = 1 in each iteration. Therefore Agorithm 3, with σ = 1 δnρ) 1 in each iteration, is precisey Agorithm with D ++ = σκ + D. Consequenty the first part of the theorem

24 4 C. B. CHUA hods. Moreover, the duaity gap of the iterates decreases by a factor of 1 δnρ) 1 iteration, whence the iteration bound hods. in each 3... Predictor-corrector agorithm. The MTY predictor-corrector agorithm aternates between σ, β) = 0, β) and σ, β) = 1, β). The iterations in the former case are caed the predictor steps, and those in the atter the corrected steps. As before, et ρ denote the ratio wi /nw n ). Theorem 5. If β, δ 0, 1) satisfies 4β 1 β + 4 4β β, 3.1) 1 β) then in each iteration of Agorithm 3, with σ, β) aternating between 0, β) and 1, β), the search directions are we defined and we may take α to be the positive rea root of α 1 α)β + α β + nρ)β 1 β + 4α β + nρ) 1 β) 1 α)β 3.13) in the predictor steps, and α = 1 in the corrector steps. Moreover, with these choices of α, the agorithm terminates after at most O nρ ogε 1 )) iterations. Proof. We sha prove by induction on the iterations that under the hypothesis of the theorem, a search directions are we defined and a iterates are stricty feasibe, d X +, S + ; κ + D) β in a predictor steps, and d X +, S + ; κ + D) β in a corrector steps. Suppose that at the beginning of a predictor step, we have stricty feasibe X +, S + ) satisfying d X +, S + ; κ + D) β. This is certainy true for the first predictor step, which happens to be the very first iteration. If 3.1) hods, then it is straightforward to check that γ := β satisfies 3.6). Consequenty, by Lemma 9, the search directions X and S are we defined. Simiar to the second part of the proof of Theorem 3, we use.7), Lemma 6 with X, S) = X,α, S,α ) := X + α X), S + α[ S ] ), µ = 1 α)κ + and P = P := P X P T P, and 3.7) with D ++ = κ + D and σ = 0 to deduce that whenever S+ + α S S n +, 1 α)d X + + α X, S + + α S ; κ α D) κ 1 + w 1 ) P 1 n π P X,α S,α 1 α)κ + I 1 α)β + α χβ 1 β + 4α χ 1 β) F /

25 TARGET FOLLOWING ALGORITHMS 5 π λx S ). Using the triange inequaity on the -norm of where χ = κ + wn 1 R n, we bound κ + wn χ π λx S ) κ + I This eads to the bound κ + wn β + nρ). 1 α)d X + + α X, S + + α S ; κ α D) 1 α)β + α β + nρ)β 1 β + π κ + I + 4α β + nρ) 1 β). This upper bound is not ony quadratic in α, it is in fact increasing in α whenever α is nonnegative. Thus if we take α to be the positive rea root of 3.13), then we have d X + + α X, S + + α S ; κ α D) β for a α [0, α]. Thus we concude from Theorem that the next iterates X bα, S bα ) are positive definite, whence stricty feasibe as they ceary satisfy the inear equations in their respective SDP probems. Furthermore, d X bα, S bα ; κ bα D) β, whence in the next iteration we have d X +, S + ; κ + D) β, which is a corrector step. Now consider a corrector step. Suppose that at the beginning of the corrector step, we have with stricty feasibe X +, S + ) satisfying d X +, S + ; κ + D) β. This is shown above to be true for the first corrector step. As before, we concude from Lemma 9 that the search directions X and S are we defined. Taking D + := κ + D and D++ := κ + D in the proof of Theorem 3 shows that we may use α = 1 in this iteration. Moreover the same proof shows that d X 1, S 1 ; κ 1 D) 4β 1 β + 4 4β 1 β) β under the hypothesis 3.1), whence d X +, S + ; κ + D) β in the next iteration, which is a predictor step. This competes the induction. Finay, since α = Ωnρ) 1 ) for each predictor step, the duaity gap decreases by a factor of 1 Ωnρ) 1 ) every two iterations. Thus the iteration bound hods. 4. Target-Foowing Framework Based on Choesky Search Directions In this section, we highight a choice of search directions whose Newton system can be further reduced in size. Consider the maps H : S ++ S ++ S defined by H : X, S) L T SXL S. Note that H n X, S) = D is precisey the defining equation for the weighted centers T 1 D) for each D D n,++. The gradient DH X, S) of H at X, S) is given by U, V) L T SUL S + L T SXL S L 1 S VL T S ) H.

26 6 C. B. CHUA We sha use this choice of H in the inear system.9) with This gives X +, S + ) = EL S ) T EκD 1 ++ L T SXL S ) H )EL S ) 1, ES)) { = argmin d X, S; κ) : X +, S + ) = E H X), E S))} 1. X,S w 1 p=1 A k ) p X ) p = 0 1 k m), 4.1a) m y ) k A k ) p + S ) p = 0 1 p w 1 ), 4.1b) k=1 κ[z] + L T S + ) p X ) p L S+ ) p ) + κ[z] L 1 S + ) p S ) p L T S + ) p = κi L, p P ), 4.1c) where Z denotes the matrix D 1 ++ L T S + X + L S+ ) H, so that H κ[z] = L T S + ) p X + ) p L S+ ) p for each L and each p P. The corresponding pair of search directions for the origina SDP probems is caed the pair of Choesky search directions, and is given by X, S ) = E H X ), E 1 S )). Adding up 4.1c) over a L and a p P gives V + L T S + X L S+ + A k X = 0 1 k m), 4.a) m y ) k A k + S = 0, 4.b) k=1 ) V L 1 S + S L T S + H = D ++, 4.c) where V denotes L T S + X + L S+. Thus we may compute the pair of search directions X, S ) by soving a inear system with ony On ) variabes. Moreover, we do not require that D ++ has rationa entries for this system to be we defined. Not surprisingy, this system is the inearization of CP D++ ). The agorithm based on the Choesky search directions is the foowing: Agorithm 4. Target-foowing agorithm based on Choesky search directions) Given a pair of prima-dua stricty feasibe soutions X in, S in ) with T X in, S in ) D n,++, and the required accuracy ε > 0. 1) Find a target D + D n,++ satisfying d X in, S in ; D + ) β for some β 0, 1). Set X +, S + ) = X in, S in ). ) Whie X + S + > εx in S in ),

27 a) Pick target D ++ D n,++ satisfying TARGET FOLLOWING ALGORITHMS 7 D + ) 1 nn D ++ D 1 + D 1 + F δ for some δ 0, 1). b) Sove 4.) and set X ++, S ++ ) = X + + X, S + + S ). c) Update X +, S + ) X ++, S ++ ) and D + D ++. 3) Output X out, S out ) = X +, S + ) Anaysis of agorithm. For the anaysis of this agorithm, we focus on each iteration of the agorithm. We write D ++ = Diagw 1,..., w n ), where w 1,..., w n R ++. Note that we no onger require the w i s to be integers. Let π denote w w +1 for {1,..., n 1}, et π n denote w n, and et L denote { : π > 0}. For each L, et X and S denote, respectivey, [L S+ ] T [D 1 ++ L T S + X + L S+ ) H ] [L S+ ] 1 and [S + ]. We further simpify 4.) to à k X = 0 1 k m), 4.3a) m y ) k à k + S = 0, 4.3b) k=1 V + X + V ) S = D ++, 4.3c) H where X and S denote, respectivey, L T S + X L S+ and L 1 S + S L T S +, and Ãk denotes L 1 S + A k L T S + for each k {1,..., m}. For each α R, et X α and S α denote, respectivey, the sums V + α X and I + α S. It is easy to check that for each α satisfying S α S n ++, it hods d X α, S α ; D) = d X α, S α ; D). Consider the foowing inear system: n π [Ãk] X) = 0 k = 1,..., m), 4.4a) =1 m y ) k à k + S = 0, 4.4b) k=1 [Z] + X) + [Z] [ ) S ] H = I L), 4.4c) which is actuay.10) with A k = Ãk and X + ), S + ) ) = X, S ). Thus the soution of this system is reated to the soution of 4.3) via.11). We sha now derive an upper bound on the error in the inearization 4.1). The foowing bound on the Newton step is usefu. Lemma 10. If d X +, S + ; D ++ ) γ for some γ 0, 1/ ), and X) S L) and S S n satisfy π tr X) [ S ] 0

28 8 C. B. CHUA and X) + [Z] [ ) S ] = M L) H for Z = D 1 ++ L T S + X + L S+ ) H, and some M S L), then { max π X) F, } π [ S ] 1 F 1 π M γ) F. Proof. Since w 1 n n w i j w 1 i j V ij I ij ) n w 1 i j V ij I ij ) = Z I F it foows that Z I F γ. 4.5) By summing the foowing inequaities max{ X) F, [ S ] F } X) + [ S ] F tr X) [ S ] L), we deduce, using π tr X) [ S ] 0, that { max π X) F, } π [ S ] F π X) + [ S ] F. It then foows from X) + [Z] [ ) S ] = M L) and the triange inequaity on H the -norm of R n that { } max π [ S ] F π M F π X) F, Using 4.5) we estimate [Z] I) [ ) S ] Consequenty proves the emma. + H F π [Z] I) [ ) S ] [Z] I) [ S ] F γ [ S ] γ F { max π X) F, } π [ S ] F π M F + [ γ π S ] H F. [ S ] F. F

29 TARGET FOLLOWING ALGORITHMS 9 In addition, we require the foowing oca Lipschitz constant of Choesky factorization. Lemma 11. If S n satisfies F 1/, then L I+ I F F. Proof. Let L t) denote the ower trianguar matrix L I+t I. Note that L t) H + L t) L t) T = t. For each t R, et λt) denote λ L t) L t) T ). For t [0, 1], we have Soving this quadratic in L t) F gives L t) F 1 t F = L t) H + L t) L t) T F L t) H F L t) L t) T F L t) F L t) F. 1 t F or L t) F 1 + Since L t), whence L t) F, is continuous in t, it foows that L t) F 1 1 t F 1 t F. 4.6) whenever t F 1/. Under the hypothesis F 1/, this indeed hod for t = 1, thus L I+ I F 1 1 F 1. Finay, appying this upper bound in 4.6) with t = 1 gives F L I+ I F 1 L I+ I F = 1 L I+ I F as required. Lemma 1. If d X +, S + ; D ++ ) γ for some γ 0, 1/ ), then the inear system 4.4), with the matrix I in 4.4c) repaced by σi for some σ [0, 1], has a unique soution X), S, y ). Moreover for every α [0, 1], w 1 n L T π S e X,α L,α S e µ α I,α F 1 α)γ + α χ 6 + 4γ) 1 γ) + α3 χ 3 1 γ) 3, 4.7) where, for each L and each α R, X,α, S,α and κ α denote, respectivey, the sums [Z] + α X), I + α[ S ] and 1 α + ασ), and { σd X +, S + ; σd ++ ) if σ > 0, χ = w 1 n π [Z] F if σ = 0.

30 30 C. B. CHUA Proof. Since the system 4.4) is square, Lemma 10 shows that it has unique soution whenever d X +, S + ; D ++ ) < 1/. For M = σi [Z] L), we have π M F = π w 1 i j V ij σi ij ) = = n n =i j n It thus foows from Lemma 10 that { n wn 1 max π X) F, =1 A usefu consequence of this bound is S F w 1 n π w 1 i j V ij σi ij ) w i j w 1 i j V ij σi ij ) wn χ. } n π [ S ] F =1 n π [ S ] F =1 χ 1 γ). 4.8) χ 1 γ). 4.9) For each L, et L,α denote the ower trianguar matrix LS e,α I = L I+α[ e S ] I. In ) terms of L,α, the difference L Te X,α L S,α S e µ α I is,α [Z] 1 α)i ασi + α X) + [Z] L,α ) H + α X) L,α ) H + L T,α[Z] L,α + αl T,α X) L,α. Using 4.4c) with σi repacing I, and α[ S ] = L,α ) H + L,α L T,α, this reduces to 1 α)[z] I) + [Z] L,α α [ ) S ] ) H + α X) L,α ) H + L T,αL,α + L T,α[Z] I)L,α + αl T,α X) L,α, = 1 α)[z] I) L,α L T,α [Z] I) L,α L T,α ) H + α X) L,α ) H + L T,αL,α + L T,α[Z] I)L,α + αl T,α X) L,α. Using Lemma 11, we bound L,α L T,α F = L T,αL,α F L,α F α [ S ] F, [Z] I) L,α L T,α ) H F [Z] I F L,α L T,α F [Z] I F L,α L T,α F α [Z] I F [ S ] F, α X) L,α ) H α X) F L,α F α X) F [ S ] F,

31 and TARGET FOLLOWING ALGORITHMS 31 L T,α[Z] I)L,α F [Z] I F L,α F α [Z] I F [ S ] F, αl T,α X) L,α F α X) F L,α F α 3 X) F [ S ] F. Thus if d X +, S + ; D ++ ) γ for some γ 0, 1/ ), then for a α [0, 1] with S α S n +, π L T X,α L es,α S e I,α F is bounded above by the sum of and 1 α) π [Z] I F 4α 4α α α 3 π [ S ] 4 F π [Z] I F [ S ] 4 F π X) F [ S ] F π X) F [ S ] 4 F = 1 α) w n d X +, S + ; D ++ ) w n 1 α)γ, 4α S F π [ S ] F 4α S F 4 w n α χ γ 1 γ), α S F w n α χ 1 γ), α 3 S F w n α 3 χ 3 1 γ) 3, where we have used 4.8) and 4.9) to bound the ast four terms. We are ready to give the main theorem of this section. Theorem 6. If β, δ 0, 1) satisfies 4 w n α χ 1 γ), π [Z] I F π X) F π X) F γ 6 + 4γ) 1 γ) + γ 3 1 < β, 4.10) γ) 3 where γ = β +δ)/1 δ), then in each iteration of Agorithm 4, the search directions are we defined. Moreover, in each iteration, the iterates are prima-dua stricty feasibe soutions satisfying d X +, S + ; D + ) β.

32 3 C. B. CHUA Proof. We sha prove the theorem by induction on the iterations. Suppose that at the beginning of an iteration, the iterates X +, S + ) are stricty feasibe and d X +, S + ; D + ) is at most β. This is certainy true for the first iteration. By the choice of D ++ and Lemma 5, we have d X +, S + ; D ++ ) γ, where γ = β +δ)/1 δ). If 4.10) hods with β < 1, then it is straightforward to check that γ < 1/. Thus we may appy Lemma 1 with σ = 1 to deduce that the search directions X and S are we defined, and that for a α [0, 1], ) 1/ w 1 n L T π S e X,α L,α S e I,α 1 α)γ + α γ 6 + 4γ) 1 γ) + α3 γ 3 1 γ) 3, where X,α and S,α denote, respectivey, the sums X +α X) and S +α[ S ]. By Lemma 1, we have ) 1/ d X + + α X, S + + α S ; D ++ ) w 1 L T n π S e X,α L,α S e I,α 1 α)γ + α γ 6 + 4γ) 1 γ) + α3 γ 3 1 γ) 3 whenever S + + α S S n ++. Under the hypothesis 4.10), the above upper bound is at most 1 α)γ + αβ < 1 for a α [0, 1]. We then concude from Theorem that the next pair of iterates X ++, S ++ ) = X + + X, S + + S ) are positive definite, whence stricty feasibe as they ceary satisfy the inear equations in their respective SDP probems. Finay, the induction is competed by observing that the upper bound 1 α)γ + αβ) is precisey β when α = Large-update agorithm. Just as we did for the Monteiro-Zhang famiy of search directions, we can design short-step and predictor-corrector weighted path-foowing agorithms based on Choesky search directions. A simiar anaysis based on Lemma 1 instead of Lemma 9 wi produce simiar compexity resuts. In this section, we present another weighted path-foowing agorithm based on Choesky search directions: the arge-update agorithm. Rather aiming for conservativey cose targets, the arge-update agorithm aims at weighted anaytic centers with duaity gap that is a constant fraction σ of the current duaity gap. As Newton s method is not guaranteed to perform we when not in a neighborhood of the target, we sha use damped Newton steps instead. Agorithm 5. Large-update weighted path-foowing agorithm) Given a pair of prima-dua stricty feasibe soutions X in, S in ) with T X in, S in ) D n,++, and the required accuracy ε > 0.