Outline. Math Numerical Analysis. Errors. Lecture Notes Linear Algebra: Part B. Joseph M. Mahaffy,

Transcription

1 Math 54 - Numerical Analysis Lecture Notes Linear Algebra: Part B Outline Joseph M. Mahaffy, jmahaffy@mail.sdsu.edu Department of Mathematics and Statistics Dynamical Systems Group Computational Sciences Research Center San Diego State University San Diego, CA Vector 3 and Gaussian Elimination Spring 208 (/2 (2/2 Errors Errors: Consider the system Consider the system Ax b Ax b with x A b The coefficients in A and values in b are rarely known exactly Experimental (observational and round-off errors enter almost every system How much effect is there from perturbations to the system? Problems arise when A is singular or nearly singular Singular matrices result in either no solution to the system or the solution is not unique If A is near the identity, then small changes in b result in small changes in x Almost always some computed error Denote this by x The error is given by e x x The residual is given by r b Ax If either error is zero, theory gives the other being zero What if one of the errors is small? (3/2 (4/2

2 Roundoff Error Example Example: Perform 3-digit arithmetic on the system: ( ( ( x x Perform Gaussian Elimination with partial pivoting, beginning with R R 2 The multiplier is m and the operation is R 2 R 2 m R, resulting in ( ( ( x x This gives x and x (5/2 Roundoff Error Example Example from before is: ( ( ( x 0.27 x and 3-digit arithmetic gave the solution x and x The residual is easily calculated ( ( (.000 r b Ax ( (.000 which has each component < 0 3 It is easy to see that the actual solution is ( ( x.000 x which is far from the computed solution, ( (6/2, Roundoff Error Example Vector In this Example, the matrix is close to singular, so not typical. With 6 or more digits, the Gaussian Elimination with partial pivoting gives ( and the correct solution arises ( ( x x ( ( x x Because of the singularity almost any answer could arise for x 2. The computations produce a very small residual, and the two equations are very close to each other. If singular, then a single equation satisfies the problem. Thus, the first equation of the original problem is nearly parallel and close to the first equation of the problem with a little error. (7/2 Definition (l p Norm Consider an n-dimensional vector x [x,..., x n ] T. The l p norm for the vector x is defined by the following: ( n /p x p x i p. i Almost always the norms use p, p 2 (Euclidean or distance, or p (max ( x For x, we have x 2 ( x 2 + x 2 2 /2 x 2 (8/2

3 Unit Circles Vector Vector Consider x in different norms l Norm x2 x l2 Norm x2 x l Norm x2 x x x 2 x or or or ( x + x 2 x 2 + x 2 2 /2 max{ x, x 2 } Let x [x,..., x n ] T R n, then the norms for p, p 2, or p satisfy: n x x i Property (Norm x 2 i ( n i x i 2 2 x max{ x i } i Given an n-dimensional vector x [x,..., x n ] T, then: x > 0, if x i 0 for some i, x 0, if x i 0 for all i. (9/2 (0/2 Norm Example Vector Norm of a Matrix Vector Example: Consider x [0.2, 0.4, 0.6, 0.8]. For p, x 4 x i i MatLab command is norm(x, For p 2, ( 4 /2 x 2 x i i MatLab command is norm(x or norm(x,2 For p, x max x i 0.8 i MatLab command is norm(x,inf (/2 Definition (Matrix Norm A matrix norm on the set of all n n matrices is a real-valued function,, defined on this set, satisfying for all n n matrices A and B and all real numbers α: A 0; 2 A 0, if and only if A is 0, the matrix with all entries 0; 3 αa α A ; 4 A + B A + B ; 5 AB A B ; (2/2

4 Norm of a Matrix Vector Range of Changes for a Matrix and Gaussian Elimination Norm of a Matrix: There are a number of norms on a matrix. The most common norm for a matrix is defined by the vector norms for R n Theorem (Matrix Norm If is a vector norm on R n, then is a matrix norm. It follows that for any x Ax A max Ax max x x 0 x As x is varied, Ax varies. The range satisfies: M Ax max x 0 x m Ax min x 0 x Note: If A is singular, then m 0 Definition ( Let M and m be defined as above for a matrix A. The Condition Number for A is given by: A Ax x or Ax A x (3/2 κ(a M m By the definition it follows that κ (4/2 Rounding Error and Gaussian Elimination and Gaussian Elimination Consider a system of equations Ax b Assume that A is known, and there is an error in x with x + δx A(x + δx b + δb This assumes a roundoff error in x and the result of A acting on x produces a roundoff error δb With norms we see for A(δx δb If A is invertible, then δx A (δb δx A (δb A δb Definition (Alternate: The for a matrix A satisfies: κ(a A A. or A(δx δb or δb A δx δx A δb The, κ(a, measures how much impact roundoff has (5/2 (6/2

5 Error in Solution and Gaussian Elimination Properties and Gaussian Elimination For the perturbation, δx A (δb A δb κ(a δb A From Ax b, we have b A x or A x b It follows that Rearranging δx κ(a δb A κ(a δb x b δx x κ(a δb b The relative error in b gives the relative error provided κ(a Roughly, κ(a A A measures the relative error magnification factor (7/2 Some properties of the Property (Basic Properties The satisfies κ(a 2 If P is a Permutation Matrix, then κ(p 3 If D is a Diagonal Matrix, then κ(d max d ii min d ii (8/2 Example with l -norm and Gaussian Elimination Example with l -norm and Gaussian Elimination Example with l -norm: Let ( A Clearly, Ax b with Make a small perturbation, so The solution becomes, b ( b 3.8 and x b ( x Let δb b b and δx x x, then ( (, x 0 δb 0.0 and δx.63 (9/2 Example with l -norm: From before δb 0.0 and δx.63 Thus, a small perturbation in b completely changes x The relative changes are δb b and δx x.63 Because κ(a is the maximum magnification factor κ(a The b and δb were chosen to give the maximum, so κ(a (20/2

6 and Gaussian Elimination and Gaussian Elimination The plays a fundamental role in the analysis of roundoff errors during the solution of Gaussian Elimination Assume that A and b are exact and x is obtained from the floating point arithmetic of the machine with ɛ precision Assuming no singularities, the following inequalities can be established: b Ax A x x x x ρɛ, ρκ(aɛ, where ρ 0 The first inequality implies that the relative residual is approximately the same as the machine roundoff error The second inequality implies that the relative error requires A is nonsingular, and the roundoff error scales with the condition number, κ(a (2/2