Let x be an approximate solution for Ax = b, e.g., obtained by Gaussian elimination. Let x denote the exact solution. Call. r := b A x.

Transcription

1 ESTIMATION OF ERROR Let x be an approximate solution for Ax = b, e.g., obtained by Gaussian elimination. Let x denote the exact solution. Call the residual for x. Then r := b A x r = b A x = Ax A x = A (x x) x x = A r or the error e = x x is the exact solution of Ae = r Thus we can solve this numerically to obtain an estimate ê of our error e.

2 EXAMPLE Recall the linear system.729x +.8x 2 +.9x 3 =.6867 x + x 2 + x 3 = x +.2x 2 +.x 3 =.000 The true solution, rounded to four significant digits, is x = [.2245,.284,.3279] T Using Gaussian elimination without pivoting and four digit decimal floating point arithmetic with rounding, the resulting solution and error are Then x = [.225,.2790,.3295] T, e = [.0006,.0024,.006] T r = b A x = [ , , ] T Solving Ae = r by Gaussian elimination, we obtain e ê = [ ,.00250,.00504] T

3 THE RESIDUAL CORRECTION METHOD If in the above we had taken ê and added it to x, then we would have obtained an improved answer: Recall x x + ê = [.2247,.28,.3280] T x = [.2245,.284,.3279] T With the new approximation, we can repeat the earlier process of estimating the error and then using it to improve the answer. This iterative process is called the residual correction method.

4 ERROR ANALYSIS Begin with a simple example. The system 7x + 0y = 5x + 7y =.7 has the solution The perturbed system x = 0, y =. has the solution 7 x + 0ŷ =.0 5 x + 7ŷ =.69 x =.7, ŷ =.22 Why is there such a big difference in the two solutions?

5 Consider the following Hilbert matrix example. H 3 = H 3 = H 3 =, H3 = We have changed H 3 in the fifth decimal place (by rounding the fractions to four decimal digits). But we have ended with a change in H 3 in the third decimal place.

6 VECTOR NORMS A norm is a generalization of the absolute value function, and we use it to measure the size of a vector. There are a variety of ways of defining the norm of a vector, with each definition tied to certain applications. Euclidean norm : Let x be a column vector with n components. Define 2 n x 2 = x j 2 This is the standard definition of the length of a vector, giving us the straight line distance between head and tail of the vector. j=

7 -norm : Let x be a column vector with n components. Define x = n x j j= For planar applications (n = 2), this is sometimes called the taxi cab norm, as it corresponds to distance as measured when driving in a city laid out with a rectangular grid of streets. -norm : Let x be a column vector with n components. Define x = max j n x j This is also called the maximum norm and the Chebyshev norm. It is often used in numerical analysis where we want to measure the maximum error component in some vector quantity.

8 EXAMPLE Let x = [ 2 3 ] T Then x = 6 x 2 = 4. = 3.74 x = 3

9 PROPERTIES Let denote a generic norm. Then: (a) x = 0 if and only if x = 0. (b) cx = c x for any vector x and constant c. (c) x + y x + y, for all vectors x and y.

10 MATRIX NORMS We also need to measure the sizes of general matrices, and we need to have some way of relating the sizes of A and x to the size of Ax. In doing this, we will consider only square matrices A. We say a matrix norm is a way of defining the size of a matrix, again satisfying the properties seen with vector norms. Thus: (a) A = 0 if and only if A = 0. (b) ca = c A for any matrix A and constant c. (c) A + B A + B, for all matrices A and B of equal order.

11 In addition, we can multiply matrices, forming AB from A and B. With absolute values, we have ab = a b for all complex numbers a and b. There is no way of generalizing exactly this relation to matrices A and B. But we can obtain definitions for which (d) AB A B Finally, if we are given some vector norm v, we can obtain an associated matrix norm definition for which (e) Ax v A x v for all n n matrices A and n vectors x. Often we use as our definition of A the smallest number for which this last inequality is satisfied for all vectors x. In that case, we also obtain the useful property I =

12 Vector norm and matrix norm used in pairs For a given vector norm, we will always use the associated matrix norm in the sense that the following compatibility relations hold: AB A B, Ax A x and I =. We will consider three such pairs of norms next.

13 Let the vector norm be for n vectors x. Then the associated matrix norm definition is n A = max a i,j i n j= This is sometimes called the row norm of a matrix A. EXAMPLE. Let [ 2 A = 5 7 ] [, z = ] [, Az = 2 ] Then A = 2, z =, Az = 2 and clearly Az A z. Also let z = [, ] T. Then [ ] 3 Az =, z 2 =, Az = 2 and Az = A z.

14 Let the vector norm be. Then the associated matrix norm definition is n A = max a i,j j n This is sometimes called the column norm of the matrix A. Let the vector norm be 2. Then the associated matrix norm definition is A 2 = r σ (A T A) To understand this, let B denote an arbitrary square matrix of order n n. Then introduce i= σ(b) = {λ an eigenvalue of B} r σ (B) = max λ σ(b) λ The set σ(b) is called the spectrum of B, and it contains all the eigenvalues of B. The number r σ (B) is called the spectral radius of B. There are easily computable bounds for A 2, but the norm itself is difficult to compute.

15 ERROR BOUNDS Let Ax = b and A x = b, and we are interested in cases with b b. Then x x A A b b x b where the matrix norm is associated with the vector norm. Proof : Ax A x = b b A (x x) = b b ( x x = A b b ) ( b) x x = A b A b b x x A b b x x

16 Rewrite this as x x x A A b b A x Since Ax = b, we have Using this, b = Ax A x x x x A A b b b This completes the proof of the earlier assertion. The quantity cond(a) = A A is called a condition number for the matrix A.

17 EXAMPLE Recall the earlier example: Then 7x + 0x 2 = 5x + 7x 2 =.7 7 x + 0 x 2 =.0 5 x + 7 x 2 =.69 [ ] [ ] x 0 = x 2. [ ] [ ] x.7 = x 2.22 b =, b b =.0, x =., x x =.7 [ ] [ ] 7 0 A =, A 7 0 = A = 7, A = 7, cond(a) = 289 x x x b b b =.7.0 = 70 cond(a) x x x cond(a) b b b

18 The result x x b b v cond(a) v x v b v has another aspect which we do not prove here. Given any matrix A, then there is a vector b and a nearby perturbation b for which the above inequality can be replaced by equality. Moreover, there is no simple way to know of these b and b in advance. For such b and b, we have cond(a) = x x b b v v x v b v Thus if cond(a) is very large, say 0 8, then there are b and b for which x x b b v = 0 8 v x v b v We call such systems ill-conditioned.

19 Recall an earlier discussion of error in Gaussian elimination. Let x denote an approximate solution for Ax = b; perhaps x is obtained by Gaussian elimination. Let x denote the exact solution. Then introduce the residual r = b A x We then obtained x x = A r. But we could also have discussed this as a special case of our present results. Write Ax = b and A x = b r b Then becomes x x v x v x x v x v b b cond(a) b v cond(a) r v b v v

20 ILL-CONDITIONED EXAMPLE Define the 4 4 Hilbert matrix: Its inverse is given by H 4 = For the matrix row norm, H 4 = cond(h 4 ) = = So rounding error in b may lead to errors in solving H 4 x = b that are larger than the rounding errors by a factor of 0 4 or more.