Topics. Review of lecture 2/11 Error, Residual and Condition Number. Review of lecture 2/16 Backward Error Analysis The General Case 1 / 22

Size: px

Start display at page:

Download "Topics. Review of lecture 2/11 Error, Residual and Condition Number. Review of lecture 2/16 Backward Error Analysis The General Case 1 / 22"

Jocelyn Francis
5 years ago
Views:

1 Topics Review of lecture 2/ Error, Residual and Condition Number Review of lecture 2/6 Backward Error Analysis The General Case / 22

2 Theorem (Calculation of 2 norm of a symmetric matrix) If A = A t is symmetric then A 2 is given by A 2 = max{ λ : λ is an eigenvalue of A}. Theorem (Matrix -norm and -norm) We have A = max i N A = max j N N a ij, j= N a ij. i= 2 / 22

3 Topics Review of lecture 2/ Error, Residual and Condition Number Review of lecture 2/6 Backward Error Analysis The General Case 3 / 22

4 Approximate solution x of Ax = b error : = e = x x, residual : = r = b A x. Small residual need not imply small error the point P = (0.5, 0.7) and the 2 2 linear system x y = 0 0.8x + y = /2. Plot is two almost-parallel lines with x between them, far from the solution. 4 / 22

5 Justification of picture Normalize each equation in the system so that a 2 + a2 2 = Normal distance from x to the line is r = b a x a 2 x 2 Hence r 2 =square root of sum of squares of normal distance from x to lines. Same for 3D and higher dimensions 5 / 22

6 Recall qualitative facts If A is invertible then r = 0 if and only if e = 0. If A is well conditioned then r and e are comparable in size. If A is ill conditioned then r can be small while e is large. det(a) is not the right way to quantify this connection. 6 / 22

7 Condition number Definition Let be a matrix norm. Then the condition number of the matrix A induced by the vector norm is cond (A) := A A. Usually use κ κ(a) = cond(a) = condition number of A. 7 / 22

8 Matlab cond(a) or cond(a,p) condest(a) rcond(a) = reciprocal condition number 8 / 22

9 Theorem (Relative Error cond(a) Relative Residual) Let Ax = b and let ˆx be an approximation to the solution x. With r = b Aˆx x ˆx x cond(a) r b. 9 / 22

10 Proof. Ae = r so e = A r Ax = b so b = Ax A x. e = A r A r. () Dividing the smaller side of () by the larger quantity and the larger side of () by the smaller gives e A x A r. b Rearrangement proves the theorem. 0 / 22

11 I = so cond(i) = in any induced matrix norm. Similarly for an orthogonal matrix cond 2 (O) =. Example: error = (cond(a))*residual / 22

12 Topics Review of lecture 2/ Error, Residual and Condition Number Review of lecture 2/6 Backward Error Analysis The General Case 2 / 22

13 Example (The Hilbert Matrix) The N N Hilbert matrix H N N is the matrix with entries H ij = i + j, i, j n. This matrix is extremely ill conditioned even for quite moderate values on n / 22

14 Topics Review of lecture 2/ Error, Residual and Condition Number Review of lecture 2/6 Backward Error Analysis The General Case 4 / 22

15 [Basic Result of Backward Error Analysis] The result ˆx of solving Ax = b by Gaussian elimination in finite precision arithmetic subject to rounding errors is precisely the same as the exact solution of a perturbed problem (A + E)ˆx = b + f (2) where E A, f = O(machine precision). b 5 / 22

16 Theorem (Effect of Storage errors in A) Let A be an N N matrix. Suppose Ax = b and (A + E)ˆx = b. Then, x ˆx ˆx cond(a) E A. Theorem (Effect of Storage Errors in b) Let Ax = b and Aˆx = b + f. Then x ˆx x cond(a) f b. 6 / 22

17 Sketch proof Theorem 45 Proof. Step : By subtraction get an equation for the error driven by the perturbation: Step 2: Take norms Ax = b (A + E) x = b subtract A(x x) = E x x x = A E x x x = A E x A E x A A Step 3: rearrange using def of cond no x ˆx ˆx cond(a) E A. 7 / 22

18 Sketch proof Theorem 46 Proof. Since Aˆx = Ax + f, x ˆx = A f x ˆx A f = cond(a) f A x ˆx cond(a) f b x because b A x. 8 / 22

19 Facts about condition number When E is due to roundoff errors E / A = O(machine precision). cond(a) tells you how many significant digits are lost (worst case) when solving Ax = b cond(a) and cond(i) =. Scaling A does not influence cond(a): cond(a) depends on the norm chosen but usually it is of the same order of magnitude for different norms. If A is symmetric then cond 2 (A) = λ max / λ min. If A is symmetric, positive definite and = 2, then cond(a) equals the spectral condition number, λ max /λ min cond(a) = cond(a ). cond 2 (A) = λ max /λ min. 9 / 22

20 Topics Review of lecture 2/ Error, Residual and Condition Number Review of lecture 2/6 Backward Error Analysis The General Case 20 / 22

21 Working toward the general case Lemma (Spectral localization) For any N N matrix B and any matrix norm: λ(b) B. Proof. For an eigenpair (λ, φ), Bφ = λφ. Thus λ φ = Bφ B φ. 2 / 22

22 Theorem 49 Theorem lim n Bn = 0 if and only if there exists a norm with B <. Proof. We prove B < B n B n 0 Start the induction with n = 2, B 2 = B B B B B 2. Assume B n B n, then B n+ = B n B B n B B n+ B n B n 0 as n. 22 / 22

Lecture 9. Errors in solving Linear Systems. J. Chaudhry (Zeb) Department of Mathematics and Statistics University of New Mexico

Lecture 9. Errors in solving Linear Systems. J. Chaudhry (Zeb) Department of Mathematics and Statistics University of New Mexico Lecture 9 Errors in solving Linear Systems J. Chaudhry (Zeb) Department of Mathematics and Statistics University of New Mexico J. Chaudhry (Zeb) (UNM) Math/CS 375 1 / 23 What we ll do: Norms and condition