MATH 387 ASSIGNMENT 2

Transcription

1 MATH 387 ASSIGMET 2 SAMPLE SOLUTIOS BY IBRAHIM AL BALUSHI Problem 4 A matrix A ra ik s P R nˆn is called symmetric if a ik a ki for all i, k, and is called positive definite if x T Ax ě 0 for all x P R n, with x T Ax 0 only when x 0. Suppose that A P R nˆn is symmetric and positive definite. (a) Show that a ą 0 for all i. (b) Show that max i a max i,k a ik. (c) Let A k ra pkq ij s be the matrix that enters in the k-th step of the Gaussian elimination process (with A A). Show that for each k,..., n, the submatrix ra pkq ij s kďi,jďn is symmetric and positive definite. Conclude that Gaussian elimination does not break down (hence in particular, that A is invertible). (d) Show that a pkq ď a pkq for k ď i ď n and for all k 2, 3,..., n. Conclude that for Gaussian elimination in exact arithmetics, the growth factor is. ote that in exact arithmetics, the growth factor would be defined by gpaq max i,j,k a pkq ij. max i,j a ij Solution (a) Let e j P R n be jth canonical basis vector for R n. (b) Let x e i αe j for some α P R. e T j Ae j a jj ą n. x T Ax e T i Ape i αe j q αe T j Ape i αe j q a 2αa ij ` α 2 a jj. Suppose that some i j the quantity a ij is maximal. otherwise it will contradict the assumption. If a ij is positive then pick α and obtain x T Ax a αa ij ` αpαa jj a ji q x T Ax pa a ij q ` pa jj a ji q ă 0 The entry a ij cannot be zero; Date: Winter 206.

2 2 SAMPLE SOLUTIOS BY IBRAHIM AL BALUSHI whereas if a ji is negative pick α and obtain x T Ax a ` a ij ` pa jj ` a ji q ă 0 (because diagonal entries are always positive). (c) We may write L j I l j e T j where l j 0, x j`,j x jj,..., x nj x jj T P R n. L j L j` pi l j e T j qpi l j` e T j`q I l j e T j l j` e T j` because e T j l j` 0. Therefore at any k th step, L L k I l j e T j l k e T k and rl L k s kďi,jďn O P R kˆk zero matrix, so the symmetry of kth step submatrix ra pkq ij s kďi,jďn remains unchanged. As for positive definiteness, it follows from the fact x T Ax ě P tx P R n : x j ď j ă ku. It follows that at each step a pkq ą 0 for every k ď i ď n and therefore the resulting matrix A n LA, with L L L n, is upper triangular with non-zero diagonal entries; detpa n q is nonzero and A A n L. (d) By direct computation: Let k ď i ď n. rl k e T k s a pkq {apkq i,k k,k It follows that a pkq rl k A k s apkq i,k a pkq k,k a pkq 2 i,k a pkq a pkq ď a pkq. gpaq max i,j,k a pkq ij max i,j a ij a pkq k,k Problem 6 a pkq i,k ď maxi,j a pq ij. max i,j a ij (a) Let U be an upper triangular matrix with no zeroes on its diagonal. Let x P R n be the result of back-substitution applied to the system U x b in floating point arithmetic (with the machine epsilon ε ą 0). Show that there exists an upper triangular matrix Ũ, such that Ũ x b in exact arithmetics and that the entries of Ũ U can be bounded in absolute value by an expression depending only on ε, n, and U. Argue that backsubstitution is backward stable.

3 MATH 387 ASSIGMET 2 3 (b) Recall that Gaussian elimination in floating point arithmetics produces matrices L and Ũ, where L is lower triangular with unit diagonal and Ũ is upper triangular, satisfying } LŨ A} 8 ď Turn this into the following bound 3ngε p εq 2 }A} 8. } LŨ A} ď C ngε}a}, for all small ε, where } } is the matrix norm induced by the Euclidean norm in R n. In particular, try get a near-optimal value for the constant C n. (c) By combining the preceding two results, perform a backward error analysis of the Gaussian elimination process for solving the equation Ax b. That is, complete the analysis we did in class by taking into account the round-off errors of the forward elimination (solution of Ly b) and back substitution (solution of Ũx y). Solution (a) Entries of matrix U P R nˆn are given by u ij. Backward substitution algorithm is given by x j b j x k u jk u jj, j n, n,...,. k j` Let y j b j řn k j` x ku jk. Per iteration we do x j y j c u jj. Axiom on floating point operations: for some ε j ď ε mac we have x j y j u jj p ` ε j q. We want to quantify a perturbation matrix δu of U. ote that then ε j ε j y x pε j q ` ε j ` ε j ðñ ε j ε j ` ε j ε j ` Opε 2 j q which implies ε j ď ε mac ` Opε 2 macq. So if p ` εq then y xp`ε q for ε ε ` Opε 2 q. The computation goes as follows: x n b n c u nn b n u nn p ` ε q where by the previous remark may be written x n as x n rb n a p x n b u n,n qs c u n,n b n u nnp`ε q for some ε ď ε mac`opε 2 macq. We may write b n a Σ n pb n Σ n qp`ɛ q and Σ n x n bu n,n x n u n,n p` η q so x n b n x n u n,n p ` η q u n,n p ` ε 2 qp ` ɛ q.

4 4 SAMPLE SOLUTIOS BY IBRAHIM AL BALUSHI The algorithm terminates for j. ««nà nà x b a x k b u k ff c u b a x k b u k ff u p ` ε q It is important to recognize the nesting nature of carrying a sequence of floating point operations when we deal with À n. Observe that a b c pa bq c rpa ` bqp ` ɛ qs c prpa ` bqp ` ɛ qs ` cqp ` ɛ 2 q. Rewrite into nà j j ná b a x k b u k b a p x 2 b u 2 q x k b u k k 3 and b a p x 2 b u 2 q pb x 2 b u 2 qp ` ɛ 2 q so we rewrite into j n á b x 2 b u 2 x k b u k p ` ɛ 2 q p ` ɛ 2q for ɛ 2 ď ε mac ` Opε 2 macq. We arrive at Again, x x k 3 " b x 2 b u 2 j n á k 3 j* x k b u k p ` ɛ 2 q u p ` ε qp ` ɛ 2q. " j ná j* b x 2 b u 2 x 3 b u 3 p ` ɛ 2 q x k b u k p ` ɛ 2 qp ` ɛ 3 q k 4 u p ` ε qp ` ɛ 2qp ` ɛ 3q. We arrive at (we have included the contribution from b): # + k ź px b u k x k p ` η k q p ` ɛ j q u p ` ε q which we can rewrite to u p ` ε q which we can rewrite to u p ` ε q nź p ` ɛ k q x ` nź p ` ɛ k q x ` j 2 nź p ` ɛ k q. k ź u k x k p ` η k q p ` ɛ j q b. j k ź u k x k p ` η k q p ` ɛ j q b. j 2

5 MATH 387 ASSIGMET 2 5 p ` ε qp ` ɛ 2q p ` ɛ nqu x ` p ` η 2 qu 2 x 2 ` p ` η 3 qp ` ɛ 2 qu 3 x 3 ` ` p ` η n qp ` ɛ 2 q p ` ɛ n qu n x n b We note that all ε i, ɛ i, η i ď ε mac and ε i ď ε mac`opε 2 macq and for sufficiently small ε we always have ś m i p ` εq ` mε ` Opε2 q. In light of the expression Ũ x pδu ` Uq x b, we have n 2 n 2 n n n 3 n 2 δu... ď.. ε U 2 mac ` Opε 2 macq. 2 looooooooooooooooooooooomooooooooooooooooooooooon αpnq where by and { we mean term-wise absolute value and division of entries. Ũ U δu so Ũ U αpnq U ε mac ` Opε 2 macq U where the is also taken as term-wise multiplication. (b) We first show that }M} 8 ď? n}m} ď n}m} 8 for n ˆ n matrices. Recall that for x P R n we have }x} 8 ď }x} ď? n}x} 8 }M} 8 max M ij Then and ε pεq 2 ďiďn j? } LŨ A} ď } LŨ A} 8 ď 3ngε n p εq 2 }A} 8 ď εp 2ε ` Opε 2 qq so (c) The exact solution x satisfies } LŨ A} ď 3n2 gε}a}. LUx b. 3ngε p εq 2? n}a} When we solve this by Gaussian elimination, we perform the following steps: Perform the LU decomposition in inexact arithmetics: LŨ A ` E. By (b), the size of E can be estimated as }E} Opεq. Forward elimination: Solve Ly b inexactly, as p L ` δlqỹ b. By (a), the size of δl can be estimated as }δl} Opεq. Backward substotution: Solve Ũz ỹ inexactly, as pũ ` δuq x ỹ. This solution x is the final result. By (a), the size of δu can be estimated as }δu} Opεq.

6 6 SAMPLE SOLUTIOS BY IBRAHIM AL BALUSHI If we combine the aforementioned steps, we get p L ` δlqpũ ` δuq x b, or p LŨ ` δlũ ` LδU ` δlδuq x LUx. This can be rearranged to yield LUpx xq pe ` δlũ ` LδU ` δlδuq x. Since each of E, δl, δu is of size Opεq, we conclude that }x x} Opεq. Problem 7 In class, we have shown that if K is a square matrix with }K} ă, then I K is invertible, and I ` K ` K 2 `... ` K m Ñ pi Kq as m Ñ 8. We can use this fact to design an iterative method to solve Ax b. The starting point should be to somehow write A in terms of I K, where K has small norm. We can write A I pi Aq and set K I A, but we would need }I A} ă to ensure convergence. As a simple way to introduce some flexibility, let us multiply Ax b by some number ω P Rzt0u, to get and then introduce K I ωa, yielding If }K} }I ωa} ă, then ωax ωb, pi Kqx ωb ðñ Ax b. x m : pi ` K ` K 2 `... ` K m qωb Ñ x. The iterates x m satisfy the recurrent relation x m` ωb ` KpI ` K `... ` K m qωb ωb ` Kx m ωb ` pi ωaqx m x m ` ωpb Ax m q, which is convenient for implementation. (a) Assuming that }IωA} ă, derive an estimate on }x mx} that goes to 0 geometrically as m Ñ 8. (b) Assuming that A is diagonalizable, and that all its eigenvalues are positive, estimate }I ωa} in terms of λ, λ n, and ω. Here λ and λ n are the smallest and the largest eigenvalues of A, respectively. (c) In the estimate derived in (b), optimize the choice of the parameter ω.

7 (a) We have Then for 0 ă α ă MATH 387 ASSIGMET 2 7 Solution x m x ωb ` pi ωaqx m x ωax ` pi ωaqx m x pi ωaqx m pi ωaqx pi ωaqpx m xq. }x m x} ď }I ωa}}x m x} ď α}x m x}, pm ě q. Then }x m x} ď α}x m x} ď α 2 }x m2 x} ď ď α m }x 0 x}. (b) For invertible matrix Q with unit norm we write A QDQ for some diagonal matrix D. Then I ωa QQ ωqλq QpI ωλqq. If is any diagonal matrix with entries i then } } max i i. Therefore (c) Look at the function }I ωa} ď }Q}}I ωd}}q } maxt ωλ, ωλ n u. fpωq maxt ωλ, ωλ n u ˆ ωλ ` ωλ n ` ˇˇ ωλ ωλ n ˇˇ 2 The minimum occurs when ωλ ` ωλ n, which corresponds to ω 2 λ `λ 2.