Error Analysis for Solving a Set of Linear Equations

Transcription

1 Error Analysis for Solving a Set of Linear Equations We noted earlier that the solutions to an equation Ax = b depends significantly on the matrix A. In particular, a unique solution to Ax = b exists if and only if A is non-singular. In addition, in some case, depending on the elements of A, the solution may be very sensitive to numerical errors. Specifically, the accuracy of the solution depends on the condition number of matrix A. Matrix A is non-singular if one of the following conditions is satisfied: 1. A has an inverse 2. deta 0 3. A is full-rank for an n n matrix A, the number of linearly independent rows or columns is n 4. For any non-zero vector z, the product Az 0. If A is singular, there are two possibilities: 1. No solution exists 2. An infinite number of non-unique solutions exist. In this case, if x is a solution to Ax = b, then x + γz, where Az = 0 and γ is any scalar, is also a solution. To show this, assume with x being a solution and Ax = b 1 Az =0 2 for z 0z is in the null-space of A. Multiplying 2 by a scalar γ and adding to 1, we have Ax + γz =0 3 which means that x + γz is also a solution. Then we can an infinite number of solutions for different values of γ. Solve x1 x 2 = Since deta =8 15 = 7 0, the matrix is non-singular and the equation has a unique solution x =1 2. 1

2 Solve x1 x 2 = In this case, the determinant is zero. In addition, there is no x that will satisfy Ax = b. Then there is no solution for this equation. Solve x1 x 2 = This equation has a solution at x =2 0. In addition, Az =0forz =1 equation has an infinite number of solutions of the form x + γz where γ x + γz = + γ 2 = 2γ Then, the 7 for any non-zero value of γ. If the matrix A is non-singular, how can we quantify is quality? This is done through the condition number of a matrix, which is based on vector and matrix norms. Vector Norms For a scalar value x, its magnitude or size is given by its absolute value x. How about vector magnitudes? Magnitude of an n 1 vector x is given by its vector norm, called the p-norm and denoted by x p where x p = n 1/p x i p 8 where p is an integer. Typically, p-norm is evaluated for the following values of p. n 1 norm = x 1 = x i n 2 norm = x 2 = 9 1/2 x i 2 10 norm = x = max 1 i n x i 11 That is, 1-norm is the sum of the absolute values of the vector; 2-norm is the Euclidean distance of the point x from the origin in an n dimensional space; -norm is the maximum 12 2

3 absolute value among the elements of x. Let In this case Note that which is true for any vector x. Matrix Norms x = x 1 = = = x 2 = = = 2 15 x = max{ 1.6, 1.2 } = x 1 x 2 x 17 For a matrix A, itsp-norm is defined as A p = Ax p max x 0 x p 18 where the maximization is carried out over the non-zero vector x. Thus, the matrix norm measures the maximum relative stretching a matrix does to any vector. Commonly used matrix norms are A 1 = n max a ij j 19 A = max i n j=1 a ij where i and j are the row and column indices, respectively. That is, matrix 1-norm A 1 is the maximum absolute column sum and the -norm A is the maximum absolute row sum, which are evaluated easily for any matrix A. Example Consider A = In this case, A 1 = max{6, 2, 6} = 6 23 A = max{4, 2, 8} = Some properties of matrix norms: 3

4 1. A > 0ifA is a non-zero matrix. 2. γa = γ A for any scalar γ. 3. Ax A x for any vector x. Matrix condition number The condition number of a matrix A is given by conda = A A 1 26 that is, the norm of A multiplier by the norm of the inverse of A. If A is singular, the inverse does not exist and the conda =. Typically, the condition number is evaluated based on 1-norm or -norm. Equivalently, the condition number of a matrix can be written as conda = max x 0 Ax x / min x 0 Ax x Thus, the condition number number gives the ratio of the maximum stretching caused by a vector to the minimum stretching. In 2D that is, x =x 1 x 2 the condition number measures the distortion cause by the matrix to a unit circle see Figure 1 and 2. In 3D that is, x = x 1 x 2 x 3 it measures the distortion cause to a unit sphere. 27 Figure 1: Distortion of a circle into an ellipse by multiplying a matrix Figure 2: Distortion of a circle into an ellipse by multiplying a matrix Note that the higher the condition number the more elongated ill-conditioned the resulting ellipse becomes. 4

5 Consider A = For this matrix, Then A = A 1 1 = max{1.5, 4.5, 1.5} = A 1 = max{3.5, 3.5, 1.5} = Thus, cond 1 A =6 4.5 = cond A =8 3.5 = Error bounds and sensitivity in solving Ax = b based on the condition number Assume that x is the solution to the equation Ax = b. By sensitivity we mean the effect on the solution x if the right-hand side vector b is perturbed a little this may happen, for example, when trying to measure the output of a sensor, which introduces small random errors. Ideally, we would like to have the effect on x as small as possible. That is, the equation should not be very sensitive to small variations in b. Assume that the perturbation in b is b and the resulting solution is x + x. That is, where x is the solution to Ax + x =b + b 34 and x is the change in x in response to the perturbation b in b. Comparing 35 and 34, we have Then Also, Ax = b 35 A x = b 36 x = A 1 b 37 Ax = b A x 38 5

6 where the inequality is the result of the properties of matrix norms. Similarly Combining 38 and 39, we have x = A 1 b A 1 b 39 x x A 1 b A b = A A 1 b b conda b b That is, the perturbation in b is amplified by the condition number of matrix A when finding the solution to Ax = b. If the condition number is high, then the effect of a small perturbation disturbance in b on the solution x will be high as well the solution is very sensitive. See Figure 3 for an illustration. Figure 3: Sensitivity of the solution of Ax = b based on the conditioning of A In addition, the condition number gives the error bound on x for a given error in b. For example, if the input data b is accurate to the machine precision ɛ mach, then the solution x is accurate to conda ɛ mach. For example, input b is accurate to 4 decimal places and the condition number of matrix A is 10 4, then the solution x is accurate only to the nearest integer. Resources: 1. Steven Chapra and Raymond Canale, Numerical Methods for Engineers, Fourth Edition, McGraw-Hill, 2002 see Section