AMS 147 Computational Methods and Applications Lecture 17 Copyright by Hongyun Wang, UCSC

Transcription

1 Lecture 17 Copyright by Hongyun Wang, UCSC Recap: Solving linear system A x = b Suppose we are given the decomposition, A = L U. We solve (LU) x = b in 2 steps: *) Solve L y = b using the forward substitution Cost = N 2 *) Solve U x = y using the backward substitution Cost = N 2 Cost of the 2 steps = 2 N 2. LU decomposition: Cost = 2 3 N 3 The main cost of solving A x = b is in the LU decomposition. Numerical solution ˆ x can be viewed as the exact solution of a perturbed system A ˆx = b + b If the numerical method is bad, the perturbation may be very large An example to demonstrate the importance of pivoting (continued) Consider linear system a 1 x 1 + x 2 =1 x 1 + a 2 x 2 a 1 = 2 64, a 2 = 1, b 2 =1 Method #1: Gauss elimination without pivoting - 1 -

2 x ˆ 1 = 0 x ˆ 2 =1 Substituting the numerical solution into the original linear system, we have a 1 ˆx 1 + ˆx 2 = 1 ˆx 1 + a 2 ˆx 2 + ( 2) Large perturbation T can be viewed as the exact solution of a perturbed The numerical solution x ˆ = x ˆ 1, x ˆ 2 system. In this example, the numerical method (Gauss elimination without pivoting) is bad. As a result, the perturbation is very large relative to the machine precision. As we will see below, for a good numerical method, the perturbation is of the magnitude of machine precision. Method #2: Gauss elimination with pivoting "Pivoting" means we use the largest element to do the elimination. Elimination: Interchange row 1 and row 2, we have a a 2 b 2 1 a 2 b 2 a Add a 1 ( row 1) to ( row 2) 1 a 2 b 2 0 fl1 a 1 a 2 fl( 1 a 1 b 2 ) Let us calculate these floating point numbers. Notice that a 1 a 2 = 2 64 and a 1 b 2 = We have fl( 1 a 1 a 2 )= fl( )= 1 fl( 1 a 1 b)= fl( )= 1 Substitution: Row 2: fl( 1 a 1 a 2 )ˆx 2 = fl( 1 a 1 b 2 ) = 1 1 = 1 ==> ˆx 2 = fl 1 a b 1 fl 1 a 1 a 2-2 -

3 Row 1: ˆx 1 + a 2 ˆx 2 ==> x ˆ 1 a 2 x ˆ 2 =1 ( 1) = 2 ==> x ˆ 1 = 2 x ˆ 2 =1 Substituting the numerical solution into the original linear system, we obtain a 1 ˆx 1 + ˆx 2 = ˆx 1 + a 2 ˆx 2 Small perturbation The numerical solution ˆ x can be viewed as the exact solution of a perturbed system. For Gauss elimination with pivoting, the perturbed system is very close to the original system. Small perturbation is the character of good numerical methods. Error analysis in solving Ax = b Goal: to study the effect of round-off error. To prepare for the error analysis, we review vector norm and define matrix norm. Vector norm: x = ( x 1, x 2,, x N ) T N p p-norm: x p = x j j =1 1-norm: x 1 = x j N j =1 N 2 2-norm: x 2 = x j j =1 1 p 1 2 -norm: x = max j x j Matrix norm: A p = max x0 Ax p x p - 3 -

4 It follows directly from the definition that Ax p x p A p ==> Ax p A p x p We will use this result in the error analysis. Note: The definition does not give us a direct way of calculating the matrix norm. Fortunately we have explicit formulas for a few cases: a 11 a 1N A = a N1 a NN 1-norm: A 1 = max j N i =1 a ij 2-norm: A 2 = max j A T A j -norm: j ( A T A) denotes the j-th eigenvalue of matrix A T A. A = max i N j =1 a ij Notation for error analysis: x : the exact solution of Ax = b ˆ x : the numerical solution of Ax = b (the solution obtained using IEEE double precision) Numerical solution x ˆ can be viewed as the exact solution of a perturbed system A ˆx = b + b We write the numerical solution x ˆ as x ˆ = x + x Specific goal of the error analysis: to relate the relative perturbation to the relative error

5 To relate b b Relative perturbation to x. x Relative error We start with A ˆx = b + b. A ˆx = b + b ==> A( x+ x)= b + b ==> Ax+ A x = b + b (Using Ax= b) ==> A x = b (Multiplying by A 1 ) ==> x = A 1 b ==> x = A 1 b A 1 b (R1) On the other hand, starting with Ax= b, we have b = Ax ==> b = Ax A x (Dividing by b x ) ==> 1 x A 1 b (R2) Combining (R1) and (R2), we obtain x x A 1 A b b Definition: The condition number of matrix A is defined as cond( A) = A 1 A In terms of the condition number, we have - 5 -

6 x x Relative error in solution b cond( A) b Relative perturbation Note: When cond(a) is large (for example, cond( A) ), we say matrix A is ill-conditioned. When cond(a) is small (for example, cond( A) 10 6 ), we say matrix A is well-conditioned. Two causes for large error x x : b b ~ O( 1) If the numerical method is bad, then the relative perturbation is large. Consequently, the relative error in the solution is large. Remedy: use a good numerical method. cond ( A ) ~ O 1 If the system is ill conditioned, then the relative error in the solution is large even when we use a good numerical method. Two remedies: 1) use a higher precision 2) go back to the modeling process to formulate a well-conditioned system A geometric view an ill-conditioned system: [Draw two nearly perpendicular lines to show a well-conditioned system] [Draw two nearly parallel lines to show an ill-conditioned system] Solving sparse linear systems A u = b A is an N N matrix satisfying - 6 -

7 N is large (for example, N =10 6 ); Each row of A has only a few non-zero elements (for example, at most, 5 non-zero elements); A w can be calculated efficiently without using the matrix form of A. Example: The boundary value problem of 1-D Poisson equation: u xx = b x u( 0)= 0, u 1 The numerical grid We divide 0,1 ()= 0 [ ] into n subintervals. h = 1 n, x i = ih, i = 0, 1,, n Recall the numerical differentiation method for approximating the second derivative f ( x h ) + f x + h f x 2 f x Applying this to u xx = b( x), we obtain The numerical discretization: u i1 2u i + u i+1 = b i, i = 1, 2,, n 1 b i = b( x i ) u i = numerical approximation of u( x i ). The boundary condition: u 0 = 0, u n = 0 The numerical discretization has the form of a linear system Au = b u = u j,1 j n 1 { } { } b = b j,1 j n 1-7 -

8 A is an N N matrix satisfying N = (n 1), number of elements in u Each row of A has at most 3 non-zero elements For any vector w = { w j,1jn1}, A w is calculated efficiently without using the matrix form of A ( Aw ) i = w 2w + w i1 i i+1, 1 i n 1 w 0 = w n = 0 Note: For the 1-D problem, it is actually easy to write out the matrix form of A. 2 1 A = But working with the matrix form of A is not viable in 2-D or 3-D problems the matrix form of A is very complicated. The boundary value problem of 2-D Poisson equation: u xx + u yy = bx,y in [ 0,1] [ 0,1] ux,y = 0 on the boundary of [ 0,1] [ 0,1] [Draw the square to show the computational domain and the boundary]. The numerical grid We divide 0,1 [ ] into n subintervals. h = 1 n, x i = ih, i = 0, 1,, n y j = jh, j = 0, 1,, n The numerical discretization: u i1 j 2u ij + u i+1 j b ij = bx i, y j + u ij1 2u ij + u ij+1 = b ij, 1 i, j n 1-8 -

9 u ij = numerical approximation of ux ( i, y j ). The boundary condition: u 0 j = 0, u nj = 0, u i 0 = 0, u in = 0, 1 i, j n 1 The numerical discretization has the form of a linear system Au = b { } { } u = u ij,1i, j n 1 b = b ij,1i, j n 1 A is an N N matrix satisfying N = (n 1) 2, number of elements in u. Each row of A has at most 5 non-zero elements. For any vector w = { w ij,1i, j ( n 1) }, Aw is calculated efficiently without using the matrix form of A ( Aw ) ij = w i1 j + w i+1 j + w ij1 + w ij+1 4 w ij, 1 i, j ( n 1) w 0 j = 0, w nj = 0, w i 1 = 0, w in = 0, 1 i, j n 1-9 -