real and symmetric positive denite. The examples will only display the upper triangular part of matrices, with the understanding that the lower triang

Transcription

1 NEESSRY ND SUFFIIENT SYMOLI ONDITION FOR THE EXISTENE OF INOMPLETE HOLESKY FTORIZTION XIOGE WNG ND RNDLL RMLEY DEPRTMENT OF OMPUTER SIENE INDIN UNIVERSITY - LOOMINGTON KYLE. GLLIVN DEPRTMENT OF ELETRIL ND OMPUTER ENGINEERING UNIVERSITY OF ILLINOIS T URN-HMPIGN ugust, 99 bstract. This paper presents a sucient condition on sparsity patterns for the existence of the incomplete holesky factorization. Given the sparsity pattern P () of a matrix, and a target sparsity pattern P satisfying the condition, incomplete holesky factorization successfully completes for all symmetric positive denite matrices with the same pattern P (). It is also shown that this condition is necessary in the sense that for a given P () and target pattern P,ifP does not satisfy the condition then there is at least one symmetric positive denite matrix whose holesky factor has the same sparsity pattern as the holesky factor of, for which incomplete holesky factorization fails because of a nonpositive pivot.. Introduction. Incomplete holesky factorization (I) is a widely known and eective method of accelerating the convergence of conjugate gradient (G) iterative methods for solving symmetric positive denite linear systems. major weakness of I is that it may break down due to nonpositive pivots. Methods of overcoming this problem can be divided into two classes: numerical and structural strategies. numerical strategy uses numerical values generated during the factorization process to modify the factorization, as in the work by [Jennings and Malik, 9, Manteuel, 99, Munksgaard, 98, Wittum and Liebau, 989]. structural strategy, asinthework by [oleman, 988], selects the sparsity pattern to insure the completion of the I process. In this paper, we do not give any specic algorithm for modifying the sparsity pattern to assure the existence of I. Instead, we prove a sucient and necessary condition on the sparsity pattern of the incomplete holesky factor for the existence of I on the symmetric positive denite matrices with a given sparsity. The condition is more general than that in [oleman, 988], and includes oleman's sparsity pattern condition as a special case. Structural strategies are especially important for applications where a sequence of linear systems must be solved, each coecient matrix with the same non-zero pattern. This occurs when solving linear programming problems using interior point methods, or when solving discretized nonlinear partial dierential equations with a xed mesh. lthough the values can change from one step to another, the sparsity pattern is xed. Using a target I sparsity pattern that satises our sucient condition, the preconditioner can be set up on each step using a static data structure and with assurance that the preconditioner will exist. Some specic algorithms that modify a sparsity pattern to satisfy the sucient condition have been proposed and tested in [Wang et al., 99b, Wang, 99].. Denitions and Notation. efore the main results are presented, some denitions and notation that are used are given. Unless otherwise specied, we assume that matrices are THIS WORK WS SUPPORTED Y THE NTIONL SIENE FOUNDTION UNDER GRNT NOS. D-99 ND R-9

2 real and symmetric positive denite. The examples will only display the upper triangular part of matrices, with the understanding that the lower triangular parts are specied by symmetry. apital letters denote matrices, and the elements of a matrix will be denoted as the corresponding lower case letter with two subscripts indicating the row and column indices. For example, a ij denotes the element of matrix in the ith row andjth column. Let P n = f(i; j)j i j ng be the set of all possible non-zero positions in an n n upper triangular matrix. The sparsity pattern P () of a symmetric matrix is dened as P () =f(i; j)ja ij = a ji =; i j ng. Note that P () only considers the positions of the non-zero elements of in upper triangular part and P () P n. Let U be the upper triangular holesky factor of formed assuming no cancellation, so that = U T U. Since it is clear from context when we are discussing a triangular or symmetric matrix, we exploit a convenient abuse of notation and let P (U) denote the sparsity pattern of U and P (U) P n. The incomplete holesky factorization of a matrix using sparsity pattern P refers to a holesky factorization where all entries occurring outside of the specied sparsity pattern are immediately discarded, and no other numerical modication is made to the matrix. Given a matrix and a sparsity pattern P P n of the target factor U, it is assumed that P P (U) without loss of generality because a position (i; j) P (U) will not aect the nal factor. In order for the I factorization to complete all diagonal positions (i; i) must be in P, a requirement we now impose on all the sparsity patterns P dealt with in this paper. The I algorithm can be described as follows: lgorithm [R]=I[,P] begin for k = ; ; :::; n; if a kk > then () r kk = p a kk for j = ( k +; k +; :::; n (k; j) P () r kj = a kj =r kk (k; j) P endfor for i = k +; k +; :::; n for j = i; i +; :::; n () a ij = a ij, a ki a kj (i; j) P; (k; j) P and (k; i) P endfor endfor else () Stop (incomplete factorization fails) endif endfor end lthough the algorithm is most commonly stated (and implemented) by initially copying to R and then referring to R alone, the form shown above is valid for target sparsity patterns P P (), and simplies the proof of Theorem. In practice, the updates to would not be performed since generally needs to be retained for the iterative method that is preconditioned by R. Next, we dene a new property on sparsity patterns. This property uses an auxiliary This assumption is made throughout the paper when referring to the holesky factorization of a matrix

3 sparsity pattern Q, and when Q is the sparsity pattern, P (U), of the complete holesky factorization the property gives our sucient condition for existence of the I factorization. Definition.. Let Q P n and P P n be given sparsity patterns. P is said to have property + on Q, ifthe following condition is satised: for any position (j; k) P \ Q, if (i; j) and (i; k), i<j, are both in Q, then they both are in P or neither is in P. If the condition is not satised then P is said to violate property + on Q. Definition.. Let Q P n and P P n be given sparsity patterns and let + (Q) denote the set of all patterns in P n that have property + on Q. P + (Q) and P + (Q) will denote P having property + on Q and violating it, respectively. From the matrix sparsity point of view, P + (Q) means that for any (j; k) P \ Q, columns j and k of any matrix with sparsity pattern P have the same structure in rows i<j<kwhere (i; j) and(i; k) areinq. ecause property + requires either both elements to be present or both elements to be absent, it can also be viewed as an \not exclusive or" condition, modulo the sparsity pattern Q. The structure of rows i<jwhere at least one of (i; j) and (i; k) isnotinq, is unrestricted. Example. Let be a full matrix. ll sparsity patterns P P are in + (P ) except the following two patterns: P = f(; ); (; ); (; ); (; ); (; )g and P = f(; ); (; ); (; ); (; ); (; )g. The example follows by considering the following cases. If (; ) P then among the possible combinations of (; ), (; ) in or not in P, only the listed patterns violate property + (P ). If (; ) P, then any combination of (; ) and (; ) in or not in P will satisfy property + (P ). Example. If is the matrix arising from a ve-point dierence operator on a regular two-dimensional mesh using the red-black ordering, then P () + (P (U)). simpleway of generating a pattern for I is to start with U and consider subsets of P (U). The next example shows that a simple selection strategy is consistent with property +. Example. Let be a matrix with holesky factor U. If V is a submatrix of U that consists of all the diagonal elements and any subset of the set of rows of U, then P (V ) + (P (U)). To see this fact, note rst that (j; k) P (V ) implies (j; k) P (U). Now consider any pair of positions (i; j) and (i; k) in P (U). If row i of U is a row included in V, then both (i; j) and (i; k) are in P (V ) by construction of V. Otherwise, by construction, the complete row i of U is not in V and both (i; j) and (i; k) are not in P (V ). No other possibilities exists so by the denition, we have P (V ) + (P (U)). For a given matrix, there is a combinatorially large number of submatrices V satisfying the construction conditions in Example, but for each P (V ) + (P (U)). In addition, the density of target sparsity patterns can vary from as sparse as a diagonal matrix to the pattern of the complete holesky factor U. Example. Suppose the graph of is chordal and the factorization of is performed using the perfect elimination ordering that guarantees no ll-in, then P () = P (U) and P () + (P (U)), i.e., this is a special case of Example. Example indicates the relationship of oleman's preconditioner and property +. oleman identies submatrices of which are chordal and permutes so that these chordal blocks form a block diagonal submatrix of. The block diagonal matrix is used as a preconditioner by performing holesky factorization on each diagonal block using the appropriate perfect elimination ordering. Example shows that such a preconditioner satises proper +.

4 Example. Let = ;so that U = is the symbolic holesky factor of, andp (U) =P () [f(; ); (; )g. Then P () + (P (U)). Theorem below shows that P () + (P (U)) implies that I(), incomplete holesky factorization with zero levels of ll-in, exists regardless of the numerical values in the positive denite matrix of Example. When I() fails to accelerate convergence suciently, a standard technique for generating a pattern with better convergence accleration is to augment P () and move towards P (U). Suppose P = P () [f(; )g in Example. Then P + (P (U)) and I will also exist for this pattern. However, if we letq = P () [f(; )g then Q + (P (U)) because (; ) is in Q, both (; ) and (; ) are in P (U), but only one of (; ) and (; ) is in Q. In this case the I factorization may ormay not exist, but in Section we will show there exists some matrix whose full holesky factor has the same sparsity pattern as that of, and for which I with the augmented pattern Q will fail to exist. These examples indicate how property + can be satised and violated in some common circumstances. The next Section shows that property + is sucient for completion of I factorization.. Suciency of Property +. The rst theorem shows that property + guarantees the existence of the I factorization. Theorem. Let matrix < nn be symmetric positive denite and have holesky factor U. Suppose that position set P + (P (U)). The incomplete holesky factorization of using position set P completes successfully. Proof: Since any(i; j) P (U) will not aect the computation of the incomplete holesky factorization, we assume that P P (U) in the proof without loss of generality. Let i = fjj(j; i) P g be the set of row indices of elements of P in column i for each i n. onstruct a set S i in the following way: Initialize S i = i, and then repeat the augmentation of S i S i SjS i j until S i does not grow (which occurs in n, or fewer steps). This construction assures that for any j S i, j S i. The principal submatrix of dened by S i consists of the components of with row and column indices both in S i. We prove the theorem by showing that if P + (P (U)), then the computation of the incomplete holesky factorization of on each principal submatrix dened by S i, i n is equivalent to the computation of complete holesky factorization on the submatrix. Therefore the diagonal elements of the matrix at each step of the incomplete holesky factor will always be positive. This implies that the I factorization will not break down due to nonpositive pivots. We rst show that for any i and any fj; kg S i,if(j; k) P (U) then(j; k) P. ssume that (j; k) P. Then there is (j; l) P with l S i, or otherwise j would not be in S i. Now consider the position (minfl; kg; maxfl; kg). Since both (j; l) and (j; k) are in P (U), it follows that (minfl; kg; maxfl; kg), the position that will be aected (updated or lled) by the element (j; k) and(j; l), must be in P (U).If (minfl; kg; maxfl; kg) inp, this contradicts P + (P (U)) because (j; l) isinp but (j; k) is not in P. If (minfl; kg; maxfl; kg) P then we have a position (minfl; kg; maxfl; kg) which has the same status as (j; k): fl; kg S i, (minfl; kg; maxfl; kg) P (U), but (minfl; kg; maxfl; kg) P. Note that minfl; kg >jand maxfl; kg k. y replacing (j; k) with (minfl; kg; maxfl; kg) and applying this argument recursively, eventually we reach a row number m S i that has only one o-diagonal index

5 in S i. This last element must be in P, or m would not be in S i, proving the assertion by contradiction and (j; k) P follows. On the other hand, any position (j; k) with fj; kg S i will not aect the values of the elements on the submatrix dened by S i in the incomplete factorization. To see this, let's examine how a element in such a principal submatrix changes its value during the I factorization. Let fs; tg S i. Now a st will be updated by a st = a st, a ks a kt =a kk only when there exists a k such that both (k; s) and (k; t) are in P. If k S i then it can be shown that either (k; s) P (U) or (k; s) P : consider (k; s) P (U), otherwise it does not aect the computation. If (k; s) P then k will be in S i by construction of S i,contradicting the assumption that k S i. Hence (k; s) P. Therefore, (k; s) is either not in P or not in P (U). The same argument can also be applied to (k; t). Therefore, the value of elements in the principal submatrix dened by S i are not aected by elements outside this submatrix. Therefore, for any i n, the computation of the diagonal element a ii of the incomplete holesky factorization of is equivalent to computing the last diagonal element of a complete holesky factorization of the principal submatrix dened by S i. ecause any principal submatrix of a symmetric positive denite matrix is symmetric positive denite, holesky factorization of the principal submatrix of dened by S i can be completed, and hence all the diagonal elements will remain positive in the incomplete holesky factorization of with pattern P. This establishes a sucient condition on the target factor for the existence of an incomplete holesky factorization. In order to illustrate Theorem and its proof, consider a particular matrix that has the sparsity patterns given in Example, for which P () + (P (U)). The steps of the incomplete holesky factorization algorithm with sparsity pattern P = P () shown earlier, with i dened as the updated matrix after step i of I factorization, are: = = =, 8 9, ; = ; = ; =, ;, 9= The I factor R can then be obtained by setting r ii = p a ii,andr ij = a ij =r ii if (i; j) P, giving R =, p = 9= From the denition of S i given in the proof of the theorem, S = fg, S = f; g, S = f; g, S = f; ; g, S = f; ; ; g, :

6 Let i be the principal submatrix of dened by S i. If we consider the computation of I of above and extract the principal submatrix dened by S i after each step, we see exactly the computation of complete holesky factorization on the submatrix. For example, complete holesky factorization for i = gives, = =, 8! ; =,! ; and for i =, = = = 9 = = 9= The proof of Theorem relied on noting that these computations are exactly the same as those of I of on the principal submatrix dened by S and S, respectively. On the other hand, from Example, P = P () [f(; )g + (P (U)). Using this target sparsity pattern causes I to break down because of a zero pivot in step : = = =, 8 9, ; = ; =,,. Necessity ofproperty +. P + (P (U)) is not a necessary condition for I completion on a particular symmetric positive denite matrix; examples are readily constructed to show this. However, property + is a necessary condition in the sense that incomplete holesky factorization with a given target sparsity P will exist for all symmetric positive denite matrices with the same pattern P (U) onlyifp + (P (U)). Lemma.. symmetric positive denite matrix R nn can be created with any given sparsity pattern P and any given symmetric positive denite principal submatrix S such that the sparsity pattern of S is consistent with P. For example, given the sparsity pattern P = ; :

7 and given the positive denite principal submatrix ( : ; :)= says that the unspecied entries in () = ; Lemma. can be dened so that is positive denite. Proof: The proof is by construction. ssume that S R kk. Let P be the symmetrically permuted pattern of P so that the principal submatrix S of becomes the leading submatrix. Generate as the symmetric positive denite matrix with sparsity pattern P in the following way: Let the leading principle k k submatrix of be S. The expansion starts from row and column k +. t each step i, k + i n, compute the holesky factorization of the leading submatrix of size i, of, and denote the lower triangular factor as L i,. Let v be a vector of length i, with sparsity pattern consistent with that of the o{diagonal part of column i of. Set the ith diagonal element to be a ii = v T, i, v + = vt L,T i, L, i, v +, where > is arbitrary, and dene the o{diagonal part of column i to equal v (of course, the o{diagonal part of row i must also be set to v T to retain symmetry.) ontinue this process until an n n symmetric positive denite matrix is generated. Then apply the symmetric inverse permutation to, completing the creation of matrix. To illustrate the process in Lemma, we construct the unspecied entries in Equation (). First, permute the rst row and column of sparsity pattern P to the be the last so that the principal submatrix dened by indices f; ; g in the matrix will be the leading principal submatrix. The permuted pattern P becomes P = Denote i as the leading submatrix of of size i. The expansion starts with i =. = S =. Let v =( ) T, and =. Then a = v T L,T i, L, i, v + =,giving =. Next, let v = ( )T and =,giving a = and so = Lemma.. Let R. Finally, apply the inverse permutation to get = be symmetric positive denite. Let U be theupper triangular holesky factor of. If P P (U) does not have property + on P (U), then there is a.

8 symmetric positive denite matrix R such that the holesky factor of has the same structure as U and incomplete holesky factorization of using P fails because of a nonpositive pivot. Proof: The only possible pattern P (U) with subsets P P (U) not in + (P (U)) is P (U) =P. Furthermore, if (; ) P then P + (P (U)). So assume that (; ) P. Then P + (P (U)) implies P = f(; ); (; ); (; ); (; ); (; )g or P = f(; ); (; ); (; ); (; ); (; )g; as in Example. Note that in both cases the incompleteness aects the pattern in the rst row. The trailing principal submatrix of order is dense in both. Therefore, the strategy of the proof is to make the trailing principal submatrix of order positive denite after one step of complete holesky is performed and not positive denite after one step of incomplete holesky. ase : P = f(; ); (; ); (; ); (; ); (; )g and P (U) =P. Dene as follows: Let the leading full matrix of be any symmetric positive denite matrix and b be any non-zero number. Two elements in the third column are yet to be determined. If we apply two steps of I with the pattern P the value of the third pivot can be determined and set to. This yields the value of b = b b =(b b, b ) >. Satisfying this condition means that the trailing principal submatrix of order is not positive denite after one step of I. Note that b does not appear in the value for b due to P. Therefore, we must set b to guarantee that if complete holesky is performed it will succeed. This can be done by taking it to be a solution of the inequality b (b b, b b ) >. It follows that ( <b < b b =b if b b > b b =b <b < if b b < pplying complete holesky and incomplete holesky factorization on shows that is positive denite and I breaks down with the third pivot nonpositive. ase : P = f(; ); (; ); (; ); (; ); (; )g and P (U) = P. Let the trailing submatrix of be any symmetric positive denite matrix. We can set the values in the rst row of the matrix to achieve the same two conditions as in ase. Let b be any non-zero number. Determining the nal pivot of I and setting it to yields b = b b =(b b,b ) >. The completion of the holesky factorization can be guaranteed by setting the remaining element b. Let b be a solution of the inequalities: Solving the inequalities gives ( b <b b b, b =b, (b, b b b ) =(b, b b ) > : ( <b < minfb b =b ; p b b g if b b > maxfb b =b ;, p b b g <b < if b b < : pplying holesky and incomplete holesky factorization on again shows that is positive denite but I breaks down when using pattern P. 8

9 Examples of the two cases of Lemma. are easily generated. Let be the leading principal submatrix of a symmetric positive denite matrix. Dene to have the same leading principle submatrix, b =,andb = b b =(b b, b ) = =(, )=. Since b b >, b isrequired to satisfy the condition < b < b b =b =. Dene = b ==, giving =. is symmetric positive but I with the pattern of ase breaks down.! Similarly, let be the trailing submatrix of a symmetric positive denite. Dene to have the same trailing submatrix, b =, and b = b b =(b b, b ) = =(, ) =. Since b b >, b needs to be a solution tothe inequalities p = <b < minfb b =b ; b b g =. Let b ==. We thenhave =. is symmetric positive but I with the pattern in ase breaks down. Theorem. Let R nn be symmetric positive denite, U be the upper triangular holesky factor of, and P (U) be the sparsity pattern of U.! If P P n does not have the property + on P (U), then there is a symmetric positive denite matrix R nn such that the complete holesky factor of has the same non-zero structure as U, and incomplete holesky factorization of using P will fail due to a nonpositive pivot. Proof: The theorem is proven by induction on the size of the problem, n. Note that all sparsity patterns for matrices necessarily have property + on P (U). Lemma. shows that the theorem is true for problems of size less or equal to, establishing the induction basis. We assume that for all the problems of size less or equal to n, the theorem is true and show below that it is true for matrices of order n. Let P l be the \leading" subset of P dened by P l = f(i; j)j(i; j) P; i<j<ng and let P t be the \trailing" subset of P dened by P t = f(i; j)j(i; j) P; <i<jng. Similarly, denote P (U) l = f(i; j)j(i; j) P (U);i < j <ng, P (U) t = f(i; j)j(i; j) P (U); <i<jg. P is in at least one of the following three cases: ase : P l does not have property + on P (U) l. ase : P t does not have property + on P (U) t. ase : P l has property + on P (U) l and P t has property + on P (U) t but P does not have property + on P (U). In ase the induction hypothesis implies that there is a symmetric positive denite matrix S of size n, whose complete holesky factor has the same sparsity as P (U) l, and for which incomplete holesky factorization breaks down when applied to S using P l. Let the matrix be constructed so that its leading principal submatrix of order n, is equal to S and then use the result of Lemma. to construct its last row and column. Then incomplete holesky factorization on the constructed symmetric positive denite matrix breaks down during the rst n, steps of computation. In ase, create a matrix in the following way: Since P t does not have property + on P (U) t,by the induction hypothesis there is a symmetric positive denite matrix S of order n, such thatits complete holesky factor has the same structure of P (U) t and the incomplete holesky factorization of S using P t break down. From Lemma. we can create a symmetric positive denite matrix R nn such that it has the same non-zero structure 9

10 as on the rst row and column and its trailing principal submatrix is equal to S. Dene the vector v of length n as: 8 >< v i = >: ; i = ; (;i) P; c i = p c ; (;j) P; and let = + vv T. is symmetric and positive denite because is symmetric positive denite and vv T is nonnegative denite. Furthermore, the non-zero structure of the complete holesky factor of is the same as P (U). fter applying one step of incomplete holesky factorization to, the reduced matrix of order n, is equal to S. y the denition of, incomplete holesky factorization will break down. In ase, P l and P t have property + on P (U) l and P (U) t, respectively but, P violates property + on P (U). Therefore, there must be a (i; n) P such that both (;i) and (;n) are in P (U) but only one of them is in P. fter applying one step of incomplete holesky factorization on, consider incomplete holesky factorization of the reduced matrix of order n, with sparsity pattern P t. Since P t has property + on P (U) t, we cannot use ase above to show that I can fail for some set of values assigned to the non-zero positions. However, we nowshow that the non-zero values can be set so that the reduced matrix of order n, has a principal submatrix which is not positive denite. Dene the symmetric positive denite matrix as follows: let 's principal submatrix given by indices f;i;ng be equal to the matrix shown in Lemma. corresponding to the one of the two possible sparsity patterns that applies. Use Lemma. to expand the matrix to have the same sparsity pattern as. s shown in the proof of Theorem the computation of the I factorization of the reduced matrix on the elements corresponding to index set S t n, of P t is equivalent to the computation of the complete holesky factorization on the principal submatrix dened by the index set Sn,. t ecause was constructed using the techniques in the proof of Lemma., the principal submatrix dened by fi; ng S t n, is not positive denite after one step of I. Therefore, the complete holesky factorization of the principal submatrix dened by S t n, breaks down and the incomplete holesky factorization on the reduced matrix must also fail. Together, this shows that for a problem of order n, if P does not have property + on (P (U)), then there is a symmetric positive denite matrix such that its holesky factor has the same sparsity pattern as the holesky factor of, but incomplete holesky factorization of will fail. y induction the theorem is true. To illustrate the proof of ase above, let P () = so that P (U) =P. Let P = P (),f(; )g. Then P l + (P (U) l ) and P t + (P (U) t ), but P + (P (U)). Now for i =, as indicated in the proof (i; n) P, both (;i) and (;n) are in P (U), but only one of them is in P. onstruct the matrix as follows: let 's = principal submatrix given by indices f; ; g be ; which is equal to the ;

11 matrix shown in the example corresponding to ase in Lemma.. Using Lemma. we can expand the matrix to have the same sparsity pattern as, for example, = = pplying one step of incomplete holesky factorization using P on gives = = Since (; ) P, no updates are performed on column in computing. Now consider incomplete holesky factorization of the reduced matrix of order n, with sparsity pattern P t, and recall that P t has property + on P (U) t. s shown in the proof of Theorem performing incomplete holesky factorization on the reduced matrix with pattern P t has the same eect on the elements corresponding to index set S t n, = f; ; g as the computation : : of complete holesky factorization on the principal submatrix : However, the principal submatrix dened by f; g S t n, is not positive denite. Therefore, the complete holesky factorization of the principal submatrix dened by S t n, breaks down and the incomplete holesky factorization on the reduced matrix also fails on the next step: = = = =. Summary and Future Work. In this paper we dene property + for a target sparsity pattern for incomplete holesky factorization. Property + is a sucient condition on the sparsity pattern that ensures the existence of incomplete holesky factorization of a symmetric positive denite matrix. If property + is not satised, then a symmetric positive denite matrix with the same non-zero structure of its holesky factor can be found so that when applying incomplete holesky factorization with the specied pattern, the factorization will break down due to a nonpositive pivot. These results show that property + is fundamental for incomplete holesky factorization. Every other sucient condition on the sparsity pattern that we know of is a special case of property +, including the reordered chordal graph strategy in [oleman, 988] and the trivial cases of P containing only diagonal entries and P = P (U). Theorem implies that property + is necessary and sucient, if only the sparsity pattern of the matrix is considered. The proof of these results show that property + relates portions of the incomplete factorization of a matrix to complete factorizations of particular principal submatrices. This work was originally motivated by research on the incomplete Gram-Schmidt factorization

12 (IGS) of a least squares problem, for which is the coecient matrix of the normal equations. The existence of that IGS factorization can be guaranteed [Wang et al., 99a], and property + is the condition under which I applied to is essentially equivalent to IGS and therefore also guaranteed to exist. lgorithms for modifying a given sparsity pattern to assure that property + holds have been proposed in [Wang, 99]. However, the algorithms proceed simply by either always dropping positions that cause violations, or by always adding positions into P to prevent violations. more eective strategy is likely to be based on a combination of those two approaches, allowing the amount of storage used by the preconditioner to be specied in advance. This work is of great importance in applications where a sequence of problems are solved, each with the same sparsity pattern. This occurs in the solution of nonlinear partial dierential equations, and in the iterative solution of the linear systems occurring in interior point methods for linear programming. This suggests one direction of future work: what conditions on a discretization will a priori assure that I(s), incomplete holesky factorization with s levels of ll, will exist? Extensive research has also been carried out by other researchers to assure the existence of incomplete holesky factorization based on modifying the numerical values encountered, most often by augmenting pivot elements. ombining the two approaches, structural and numerical, is another promising research direction. REFERENES [oleman, 988] oleman, T. (988). chordal preconditioner for large-scale optimization. Mathematical Programming, :{8. [Jennings and Malik, 9] Jennings,. and Malik, G. M. (9). Partial elimination. Journal of the Institute of Mathematics and Its pplications, :{. [Manteuel, 99] Manteuel, T.. (99). Shifted incomplete holeski factorization. In Du, I. S. and Stewart, G., editors, Sparse Matrix Proceedings 98, Philadelphia,P. SIM Publications. [Munksgaard, 98] Munksgaard, N. (98). Solving sparse symmetric sets of linear equations by preconditioned conjugate gradients. M Transactions on Mathematical Software, :{9. [Wang, 99] Wang, X. (99). Incomplete Factorization Preconditioning for Linear Least Squares Problems. PhD thesis, University of Illinois Urbana-hampaign. lso availableastech. Rep. UIUDS-R-9-8, omputer Science Department, University of Illinois { Urbana. [Wang et al., 99a] Wang, X., Gallivan, K.., and ramley, R. (99a). IMGS: incomplete orthogonalization preconditioner. Technical Report 9, Indiana University{loomington, loomington, IN. Submitted to SIM J. Sci. omp. [Wang et al., 99b] Wang, X., Gallivan, K.., and ramley, R. (99b). Incomplete holesky factorization with sparsity pattern modication. Technical Report 9, Indiana University{ loomington, loomington, IN. [Wittum and Liebau, 989] Wittum, G. and Liebau, F. (989). On truncated incomplete decompositions. IT, 9:9{.