Perturbation Analyses for the Cholesky Factorization with Backward Rounding Errors

Transcription

1 Perturbation Analyses for the holesky Fatorization with Bakward Rounding Errors Xiao-Wen hang Shool of omputer Siene, MGill University, Montreal, Quebe, anada, H3A A7 Abstrat. This paper gives perturbation analyses of the holesky fatorization with the form of perturbations we ould expet from the equivalent bakward error in A resulting from numerially stable omputations. The analyses more aurately reflet the sensitivity of the problem than previous suh results. Both numerial results and an analysis show the standard method of symmetri pivoting usually improves the ondition of the problem. It follows that the omputed R will usually have more auray when we use the standard symmetri pivoting strategy. Introdution Let A R n n be a symmetri positive definite matrix. Then A has a unique holesky fatorization of the form A = R T R, where R is upper triangular with positive diagonal elements and is alled the holesky fator of A. Suppose there is a perturbation A in A and A + A is also symmetri positive definite, then A + A has a unique holesky fatorization A + A = (R + R) T (R + R). The goal of the perturbation analysis is to give a bound on R (or R ) in terms of A (or A ). There have been several papers dealing with the perturbation analysis for the holesky fator for a general perturbation A, see [], [8], and [0] for normwise analyses, and [, ] for omponentwise analyses. Reently perturbation results of a different flavor were presented in [4]. For A orresponding to the bakward rounding error in A resulting from standard numerial stable omputations, a nie norm-based perturbation result was presented in [4]. Note [, ] also inluded omponent perturbation bounds for a different and somewhat ompliated form of the bakward rounding error in A. The purpose of this paper is to establish new first-order perturbation bounds for A orresponding to the bakward rounding error, whih are sharper than the orresponding first-order result in [4]. Now we introdue notation. To simplify the presentation, for any n n matrix X, we define the upper triangular matrix x x x n 0 up(x) x x n......, () 0 0 x nn

2 and the diagonal matrix diag(x) diag(x, x,..., x nn ) diag(x ii ). For any matrix ( ij ) [,..., n ] R n n, denote by (i) j the vetor of the leading i elements of j. Using u to denote upper, we define uve() [ () T, () T,, n (n) T ] T, () the vetor formed by staking the olumns of the upper triangular part of into one long vetor. Note that for any upper triangular X, uve(x) = X F. To help our norm analysis, for any matrix R n n we define duve() diag (,,,...,,,...,, )uve(). (3) }{{}}{{} n Thus for any symmetri matrix X we have duve(x) = X F. Finally let D n be the set of all n n real positive-definite diagonal matries. At the end of this setion we summarize the strongest norm-based results for a general perturbation A presented in [], as those results will be ompared with the new results for A having the form of bakward rounding error. Theorem. (see []) Let A R n n be symmetri positive definite with the holesky fatorization A = R T R. Let A be symmetri. If ǫ A F / A satisfies κ (A)ǫ <, where κ (A) A A, then A+ A has the holesky fatorization A + A = (R + R) T (R + R), suh that where R F R κ (A)ǫ + O(ǫ ), (4) κ/ (A) κ (A) κ (A) κ (A), (5) κ (A) W R A /, (6) κ (A) inf D D n κ (A, D), κ (A, D) κ (R)κ (D R), (7) with W R R n(n+) n(n+) defined by r r r r r W R = rn r. (8) rn rn r r r n r n r nn

3 If standard symmetri pivoting is used in omputing the holesky fatorization PAP T = R T R, we have κ/ (A) κ (PAP T ) κ (PAP T ) κ / (A) n(n + )(4 n + 6n )/6. In [] we showed the bound (4) is attainable to first order in ǫ. So κ (A) is the ondition number for the holesky fator R. It is expensive to estimate or evaluate κ (A) diretly by using the usual methods. Fortunately we an use κ (A) as an approximation of κ (A). Aording to the well-known result of van de Sluis [3], if D is hosen to equilibrate the rows of R, i.e., all rows of D R have unit length, then κ (A, D) with suh a D is at most a fator of n off the infimum κ (A). Notie that κ (A, D) an be estimated by a standard ondition estimator in O(n ) flops. Main results In this setion we first derive a tight perturbation bound, leading to the ondition number χ (A) for perturbation having the form of equivalent bakward rounding error. Also we derive a pratial estimate χ (A) of χ (A). Then we ompare χ (A) with κ (A), and χ (A) with κ (A). Finally we show how standard pivoting improves the ondition number χ (A). Before proeeding we introdue the following result presented in [3], see also [6, Theorems 0.5 and 0.7]. Lemma. (see [3]) Let A D HD R n n be a symmetri positive definite floating point matrix, where D diag(a) /. If nǫ H <, (9) where ǫ (n+)u/( (n+)u) with u being the unit round-off, then holesky fatorization applied to A sueeds (barring underflow and overflow) and produes a nonsingular R, whih satisfies where d i = a / ii. A + A = R T R, A ǫ dd T, (0) This lemma is appliable to any standard algorithms for the holesky fatorization (see, for example, [5] for the standard algorithms). Based on this, we establish the following result. Theorem 3. Suppose all of the assumptions in Lemma hold. Let A = R T R be the holesky fatorization of A. Set ˆR RD. Then for the perturbation A and result R in (0) we have with R R R that R F R χ (A)ǫ + O(ǫ ), () χ (A) χ (A) n H, () χ (A) H, (3)

4 where χ (A) n D W ˆR / R, χ (A) inf D D n χ (A, D), χ (A, D) n ˆR ˆR D D R / R, with D diag (a /, a/, a/ } {{ },..., a / nn, a / nn,..., a / nn ) R n(n+) }{{} n n(n+), (4) and W ˆR being just W R in (8) with eah entry r ij replaed by ˆr ij. The bound in () is approximately attainable to first-order in ǫ. Proof. Let G A/ǫ. By (0) and (9) it is easy to show A + tg is symmetri positive definite for all t [0, ǫ], and so it has the holesky fatorization A + tg = R T (t)r(t), t ǫ, (5) with R(0) = R and R(ǫ) = R R + R. Differentiating (5) and setting t = 0 in the result gives G = R T Ṙ(0) + Ṙ(0)T R, (6) whih, ombined with R = ˆRD, gives ˆR T Ṙ(0)D + D Ṙ T (0) ˆR = D GD, (7) The upper and lower triangular parts of (7) ontain idential information, and it is easy to show that by using () and (3) the upper triangular part of (7) an be rewritten as the following matrix-vetor equation form W ˆR uve(ṙ(0)d ) = duve(d GD ). (8) Notie from () that uve(ṙ(0)d ) = D uve(ṙ(0)) with D as in (4), then from (8) we have uve(ṙ(0)) = D W ˆR whih with G = A/ǫ and (0) gives duve(d GD ), (9) Ṙ(0) F D W ˆR D dd T D F = n D W ˆR. (0) The Taylor expansion of R(t) about t = 0 gives at t = ǫ R + R R = R(ǫ) = R(0) + ǫṙ(0) + O(ǫ ), () whih, ombined with (0), gives (). Obviously there exists a symmetri matrix F R n n suh that D W ˆR duve(d FD ) = D W ˆR D FD F.

5 Then by taking G = (min fij 0 d i d j / f ij )F, we have A ǫ dd T and from (9) that Ṙ(0) F = ( min d id j / f ij ) D W f ij 0 ˆR D FD F D W ˆR, whih shows the first-order bound in () is approximately attained for suh G. It remains to prove () and (3). From (6) with R = ˆRD, it follows that Ṙ(0)R + R T Ṙ T (0) = ˆR T D GD ˆR. From this we see with notation up (see ()) that Ṙ(0) = up( ˆR T D GD ˆR )R, () so with (9) and the fat that up(xd) = up(x)d and up(x) F X F for any X R n n and D D n we have D W ˆR duve(d GD ) = up( ˆR T D GD ˆR )R F ˆR D GD F ˆR D D R. Atually this holds for any symmetri G R n n sine it was essentially obtained from the matrix equation R T X + X T R = G with X triangular. Notie duve(d GD ) = D GD F, thus we must have D W ˆR ˆR ˆR D D R. Sine this is true for any D D n, we see that the first inequality in () holds. The seond inequality in () follows from χ (A, I) = n ˆR = n H. Similarly we an prove (3) by using the fat that up(x) F X F for any symmetri X R n n. Sine the first-order bound () is approximately attainable, χ (A) an be regarded as the ondition number for the holesky fatorization with the form of perturbation error satisfying (0). Reently Drma et al. [4] gave a perturbation analysis with the same form of perturbation error. Their asymptoti perturbation bound an be presented as follows: R R ( + log n )n H ǫ + O(ǫ ). (3) This suggests that H an be regarded as a onditioning measure. From () we see our bound () is sharper [ than (3). ] In fat [ it ] an be muh sharper. δ δ As an illustration suppose A = δ δ + δ 4, R = 0 δ. Then it is easy to show for small δ > 0, χ (A) χ (A) = O(/δ), H = O(/δ ). Thus (3) may overestimate the true relative error of the omputed holesky fator, and the approximation χ (A) to the ondition number χ (A) is a signifiant improvement to H. Furthermore it is easy to see H is invariant if pivoting is used in omputing the holesky fatorization of A, whereas χ (A)

6 and χ (A) depend on any pivoting. Thus χ (A) and χ (A) more losely reflets the true sensitivity of the holesky fatorization than H. As far as we an see, it is expensive to ompute or approximate χ (A) diretly by the usual approah. Fortunately χ (A) is quite easy to estimate. By the result of van der Sluis [3], ˆR D D R will be nearly minimum when the rows of D R are equilibrated. Then a proedure for obtaining an O(n ) ondition estimator for the holesky fatorization with bakward rounding errors is to hoose D = D r diag( R(i, :) ), and then use a standard norm estimator to estimate all fators in χ (A, D). Numerial experiments suggest usually χ (A) is a reasonable approximation of χ (A). But χ (A) an still be very larger than χ (A), even though it an be muh smaller than n H. For the example above we have χ (A) = O(). Numerial tests suggest that usually χ (A) is smaller or muh smaller than κ (A), defined by (6). This is not surprising sine (0) provides more information about the perturbation in the data. In [] we proved the following results. Theorem 4. n χ maxi aii (A) κ (A) min i a ii χ (A), (4) n χ (A) κ maxi aii (A) min i a ii χ (A). The first inequality in (4) is attainable, sine equality will hold by taking A = I with > 0. The seond inequality is at least nearly attainable. In fat taking A = R T R, where R = diag(δ n, δ n,...,δ, ) + e e T n with small δ > 0, we easily obtain max i a ii κ (A) = O( ), χ δn (A) = min i a ii δ n O() = O( ). δn This example also suggests that possibly κ (A) is muh larger than χ (A) if the maximum element is muh larger than the minimum one on the diagonal of A. The standard symmetri pivoting strategy an usually improve χ (A), just as it an usually improve κ (A). In [] we showed the following theorem. Theorem5. Let A R n n be symmetri positive definite with the holesky fatorization PAP T = R T R when the standard pivoting strategy is used. Then χ (PAP T ) χ (PAP T ) H / n n(n + )(4 n + 6n )/6, (5) where PAP T D HD with D diag(pap T ) /. One may not be impressed by the 4 n fator in the upper bound, and may wonder if it an signifiantly be improved. In fat we an prove for any n the upper bound an nearly be approximated by a parametrized family of matries (f. [, pp.5 6]). But suh examples are rare in pratie. Our numerial experiments onfirms that χ (PAP T ) is usually (muh) smaller than χ (A). Thus holesky fatorization with standard symmetri pivoting will usually give more aurate R. By following the approah of [7], it is straightforward, but detailed and lengthy, to extend the first-order results to provide rigorous perturbation bounds. For suh results, see [].

7 Table. Results for Pasal matries without pivoting. n χ (A) χ (A, D r) κ (A) κ (A, D r) 5.0e 0.0e e 0.0e+00 7.e 0.e e+00.e e e e e+0 9.7e e+0 3.6e e+0.3e+0 5.5e+0 4.8e+0 3.0e+0 5.7e+0.4e e+0 4.9e+03.5e e+0 9.e+03.5e+03 5.e+04.e e e+04.9e e+05.0e e e+05.5e e+06.7e e+04.3e+06.3e e+07.5e e+05.5e+07.e e+08.4e+08.7e+06 9.e e e+09.e e e e+08.0e+.e e e e+09.e+ 9.8e+0 4.0e+08.e+0 7.e+0.4e+3 8.8e e+08.3e+ 6.5e+.6e+4 7.9e+ Table. Results for Pasal matries with pivoting, Ã PAP T. n χ (Ã) χ(ã, Dr) κ(ã) κ(ã, Dr) 5.0e 0.0e e 0.0e+00 7.e 0.6e e+00.5e+00 4.e e e+00.8e+0 5.e+00.6e+0 3.6e+0 4.3e+0 6.e+0.e+0 8.0e+0 3.0e e+0.e+0 8.3e+0 3.3e+0.5e e+0 6.7e+0.5e+0.3e+03.e e+0.e e+0 5.e+03.0e e+0 7.8e e+03.4e+04.7e+06 9.e+03.7e+04.6e+04.0e+05.5e e e e e+05.4e e+03.7e+05.4e+05.8e+06.e+09.8e+04 9.e e+05 8.e+06.e e+04.9e+06 3.e+06 3.e e e e+06.3e+07.e e+ 5.e+05.8e e e e+ 3 Numerial experiments In setion we presented a new first-order perturbation bound for the holesky fator with the hange aused by bakward rounding errors, defined χ (A) n D W ˆR / A / as the ondition number, and suggested χ (A) ould be estimated in pratie by χ (A, D r ) n ˆR ˆR D r Dr R / R with D r = diag( R(i, :) ), whih an be estimated by standard norm estimators

8 in O(n ) flops. Our new first-order result is (muh) better than the previous orresponding result. Also we ompare χ (A) with κ (A), the ondition number for general perturbation A, and ompare the orresponding estimates χ (A) with κ (A) as well. Now we give a set of examples to show our findings. The matries are n n Pasal matries, (with elements a j = a i =, a ij = a i,j + a i,j ), n =,,..., n. The results are shown in Table without pivoting and in Table with pivoting. Note in Tables and how n H an be worse than χ (A). In Table pivoting is seen to give a signifiant improvement to χ (A). Also we observe from both the tables that χ (A) is a reasonable approximation of χ (A). We see χ (A) is smaller than κ (A) for n >. Aknowledgement. I would like to thank hris Paige for his valuable omments and suggestions. Referenes. hang, X.-W.: Perturbation analysis of some matrix fatorizations. Ph.D thesis, MGill University, February 997. hang, X.-W., Paige,.., Stewart, G. W.: New perturbation analyses for the holesky fatorization. IMA J. Numer. Anal., 6 (996) Demmel, J. W.: On floating point errors in holesky. Tehnial Report S-89-87, Department of omputer Siene, University of Tennessee, 989. LAPAK Working Note 4 4. Drma, Z., Omladi, M., Veselić, K.: On the perturbation of the holesky fatorization. SIAM J. Matrix Anal. Appl., 5 (994) Golub, G. H., Van Loan,. F.: Matrix omputations. Third Edition. The Johns Hopkins University Press, Baltimore, Maryland, Higham, N. J.: Auray and Stability of Numerial Algorithms. Soiety for Industrial and Applied Mathematis, Philadelphia, PA, Stewart, G. W.: Error and perturbation bounds for subspaes assoiated with ertain eigenvalue problems. SIAM Rev., 5 (973) Stewart, G. W.: Perturbation bounds for the QR fatorization of a matrix. SIAM J. Numer. Anal., 4 (977) Stewart, G. W.: On the perturbation of LU, holesky, and QR fatorizations. SIAM J. Matrix Anal. Appl., 4 (993) Sun, J.-G.: Perturbation bounds for the holesky and QR fatorization. BIT, 3 (99) Sun, J.-G.: Rounding-error and perturbation bounds for the holesky and LDL T fatorizations. Linear Algebra and Appl., 73 (99) Sun, J.-G.: omponentwise perturbation bounds for some matrix deompositions. BIT, 3 (99) van der Sluis, A.: ondition numbers and equilibration of matries. Numerishe Mathematik, 4 (966) 4 3 This artile was proessed using the L A TEX maro pakage with LLNS style