ESTIMATION OF ERROR. r = b Abx a quantity called the residual for bx. Then

Transcription

1 ESTIMATION OF ERROR Let bx denote an approximate solution for Ax = b; perhaps bx is obtained by Gaussian elimination. Let x denote the exact solution. Then introduce r = b Abx a quantity called the residual for bx. Then r = b Abx = Ax Abx = A (x bx) x bx = A r or the error e = x bx is the exact solution of Ae = r Thus we can solve this to obtain an estimate be of our error e.

2 EXAMPLE. Recall the linear system.729x +.8x 2 +.9x 3 =.6867 x + x 2 + x 3 = x +.2x 2 +.x 3 =.000 The true solution, rounded to four significant digits, is x =[.2245,.284,.3279] T Using Gaussian elimination without pivoting and four digit decimal floating point arithmetic with rounding, the resulting solution and error are Then bx =[.225,.2790,.3295] T e =[.0006,.0024,.006] T r = b Abx =[ , , ] T Solving Ae = r by Gaussian elimination, we obtain e be =[ ,.00250,.00504] T

3 THERESIDUALCORRECTIONMETHOD If in the above we had taken be and added it to bx, then we would have obtained an improved answer: x bx + be =[.2247,.28,.3280] T Recall x =[.2245,.284,.3279] T With the new approximation, we can repeat the earlier process of estimating the error and then using it to improve the answer. This iterative process is called the residual correction method. It is illustrated with another example on page 286 in the text.

4 ERROR ANALYSIS Begin with a simple example. The system has the solution 7x + 0y = 5x + 7y =.7 The perturbed system has the solution x =0, y =. 7bx + 0by =.0 5bx + 7by =.69 bx =.7, by =.22 Whyistheresuchadifference?

5 Consider the following Hilbert matrix example. H 3 = H 3 = fh 3 =, f H 3 = We have changed H 3 in the fifth decimal place (by rounding the fractions to four decimal digits). But we have ended with a change in H3 in the third decimal place.

6 VECTOR NORMS A norm is a generalization of the absolute value function, and we use it to measure the size of a vector. There are a variety of ways of defining the norm of a vector, with each definition tied to certain applications. Euclidean norm : Let x be a column vector with n components. Define kxk 2 = nx x j 2 j= This is the standard definition of the length of a vector, giving us the straight line distance between head and tail of the vector. 2

7 -norm : Let x be a column vector with n components. Define kxk = nx j= x j For planar applications (n = 2), this is sometimes called the taxi cab norm, as it corresponds to distance as measured when driving in a city laid out with a rectangular grid of streets. -norm : Let x be a column vector with n components. Define kxk = max j n x j This is also called the maximum norm and the Chebyshev norm. It is often used in numerical analysis where we want to measure the maximum error component in some vector quantity.

8 EXAMPLES Let Then x = h 2 3 i T kxk = 6 kxk 2 = sqrt(4). =3.74 kxk = 3 PROPERTIES Let k k denote a generic norm. Then: (a) kxk =0ifandonlyifx =0. (b) kcxk = c kxk for any vector x and constant c. (c) kx + yk kxk + kyk, forallvectorsx and y.

9 MATRIX NORMS We also need to measure the sizes of general matrices, and we need to have some way of relating the sizes of A and x to the size of Ax. In doing this, we will consider only square matrices A. We say a matrix norm is a way of defining the size of a matrix, again satisfying the properties seen with vector norms. Thus:. kak =0ifandonlyifA =0. 2. kcak = c kak for any matrix A and constant c. 3. ka + Bk kak + kbk, for all matrices A and B of equal order.

10 In addition, we can multiply matrices, forming AB from A and B. With absolute values, we have ab = a b for all complex numbers a and b. There is no way of generalizing exactly this relation to matrices A and B. But we can obtain definitions for which (d) kabk kak kbk Finally,ifwearegivensomevectornormk k v,wecan obtain an associated matrix norm definition for which (e) kaxk v kak for all n n matrices A and n vectorsx. Often we use as our definition of kak the smallest number for which this last inequality is satisfied for all vectors x. In that case, we also obtain the useful property kik =

11 Let the vector norm be k k for n vectors x. Then the associated matrix norm definition is kak = max i n nx j= a i,j This is sometimes called the row norm of a matrix A. EXAMPLE. Let " 2 A = 5 7 #, z = " #, Az = " 2 # Then kak =2, kzk =, kazk =2 and clearly kazk kakkzk.alsoletz = h, i T. Then " # 3 Az =, 2 kzk =, kazk =2 and kazk = kakkzk.

12 Let the vector norm be k k. matrix norm definition is kak = max j n nx i= Then the associated a i,j This is sometimes called the column norm of the matrix A. Let the vector norm be k k 2. Then the associated matrix norm definition is kak =sqrt h r σ (A T A) i To understand this, let B denote an arbitrary square matrix of order n n. Then introduce σ(b) ={λ an eigenvalue of B} r σ (B) = max λ λ σ(b) The set σ(b) is called the spectrum of B, and it contains all the eigenvalues of B. The number r σ (B) is called the spectral radius of B. There are easily computable bounds for kak, but the norm itself is difficult to compute.

13 ERROR BOUNDS Let Ax = b and Abx = b b, and we are interested in cases with b b b.then kx bxk v kak A b b b kbk v where k k v is some vector norm and k k is an associated matrix norm. v Proof : Ax Abx = b b b A (x bx) = b b b x bx = A ³ b b b kx bxk v = kx bxk v A ³ b b b v A b b b A b b b v v

14 Rewrite this as kx bxk v Since Ax = b, wehave Using this, kak A b b b v kak kbk v = kaxk v kak A kx bxk v kak kbk v This completes the proof of the earlier assertion. b b b v The quantity cond(a) =kak A is called a condition number for the matrix A.

15 EXAMPLE. Recall the earlier example: 7x + 0x 2 = 5x + 7x 2 =.7 7bx + 0bx 2 =.0 5bx + 7bx 2 =.69 Then kbk =, " x x 2 " b x bx 2 # # b b =.0 kxk =., kx bxk =.7 " # 7 0 A =, A 5 7 = kak =7, kx bxk kxk = = " " 0. # ".7.22 A =7, cond(a) = 289 b b b kx bxk kxk kbk =.7.0 b b cond(a) kbk # = 70 cond(a) #

16 The result kx bxk b b v cond(a) kbk v has another aspect which we do not prove here. Given any matrix A, then there is a vector b and a nearby perturbation b for which the above inequality can be replaced by equality. Moreover, there is no simple way to know of these b and b in advance. For such b and b b,wehave cond(a) = kx bxk v v b b b kbk v Thus if cond(a) is very large, say 0 8,thenthereare b and b for which kx bxk v =0 8 b b b kbk v We call such systems ill-conditioned. v v

17 Recall an earlier discussion of error in Gaussian elimination. Let bx denote an approximate solution for Ax = b; perhaps bx is obtained by Gaussian elimination. Let x denote the exact solution. Then introduce the residual r = b Abx We then obtained x bx = A r. But we could also have discussed this as a special case of our present results. Write Ax = b and Abx = b r b b Then becomes kx bxk v b b cond(a) kbk v v kx bxk v cond(a) krk v kbk v

18 ILL-CONDITIONED EXAMPLE Define the 4 4 Hilbert matrix: H 4 = Its inverse is given by H 4 = For the matrix row norm, cond(h 4 )= = Thus rounding error in defining b should lead to errors in solving H 4 x = b that are larger than the rounding errors by a factor of 0 4 or more.