Chapter 8 Gauss Elimination. Gab-Byung Chae

Transcription

1 Chapter 8 Gauss Elimination Gab-Byung Chae

2 2 Chapter Objectives How to solve small sets of linear equations with the graphical method and Cramer s rule Gauss Elimination Understanding how to count flops to evaluate the efficiency of an algorithm Understanding the concepts of singularity and ill-condition Partial pivoting and complete pivoting Recognizing how the banded structure of a tridiagonal system can be exploited to obtain extremely efficient solutions Solving small numbers of equations The Graphical Method 3x 1 + 2x 2 = 18 x 1 + 2x 2 = 2 or x 2 = 3 2 x x 2 = 1 2 x The solution is x 1 = 4and x 2 = 3 no solution system, infinite solutions system and ill-conditioned system(only visually) Determinants and Cramer s rule Determinants [A]{x} = {b} where [A] is the coefficient matrix [A] = a 11 a 12 a 13 a 21 a 22 a 23 a 31 a 32 a 33

3 3

4 4 The determinant of this system is determinant of [A] and is represented as a 11 a 12 a 13 D = a 21 a 22 a 23 a 31 a 32 a 33 For 2 2 matrix, the determinant is a D = 11 a 12 a 21 a 22 = a 11a 22 a 12 a 21 For 3 3 matrix, the determinant is a D = a a 23 a 32 a 33 a a 21 a a 31 a 33 + a 13 a 21 a 22 a 31 a 32 where the 2 2 determinants are called minors Example 01 Find the determinant of the systems represented in Figs 1 and Figs 2 Solution For Figs 81: D = = 3(2) 2( 1) = 8 For Figs 82a: For Figs 82b: For Figs 82c: D = = 1 2 (1) 1( 1 2 ) = 0 D = = 1 (2) 1( 1) = 0 2 D = = 1 2 (1) 1( 23 5 ) = 004 The singular system have zero determinant The system that is almost singular(figs 82c) has a determinant that is close to zero Cramer s rule

5 5 Example 02 Solve 03x x 2 + x 3 = x 1 + x x 3 = x x x 3 = 044 Solution D = = The solution using Cramer s rule is b 1 a 12 a b 2 a 22 a b 3 a 32 a x 1 = = D = = 149 x 2 = a 11 b 1 a 13 a 21 b 2 a 23 a 31 b 3 a 33 D = = = 295 x 3 = a 11 a 12 b 1 a 21 a 22 b 2 a 31 a 32 b 3 D = = = 198 Elimination of unknown The usual way that you use when you solve a system of equation by hand(middle school mathematics) Naive Gauss Elimination Forward Elimination a 11 a 12 a 13 b 1 a 21 a 22 a 23 b 2 a 31 a 32 a 33 b 3

6 6 a 11 a 12 a 13 b 1 0 a 22 a 23 b 2 0 a 32 a 33 b 3 a 11 a 12 a 13 b 1 0 a 22 a 23 b a 33 b 3 Back substitution x 3 = x 2 = b 3 a 33 (b 2 a 23 x3) a 22 x 1 = (b 1 a 13 x 3 a 12 x 2 ) a 11 This method is called naive Gauss elimination because it does not avoid division by zero Section 83 will deal with the additional features required for an effective program to this problem Consider a general set of n equations: a 11 x 1 + a 12 x a 1n x n = b 1 a 21 x 1 + a 22 x a 2n x n = b 2 a n1 x 1 + a n2 x a nn x n = b n To eliminate x 1 multiply the second row of the system by a 21 /a 11 : a 21 x 1 + a 21 a 11 a 12 x a 21 a 11 a 1n x n = a 21 a 11 b 1 and subtract from the second row of the system : or denote it ( a 21 a ) ( 21 a 12 x a 2n a ) 21 a 1n x n = b 2 a 21 b 1 a 11 a 11 a 11 a 22x a 2nx n = b 2

7 7 This process is then repeated for the remaining equations ( for example, multipling the third row of the system by a 31 /a 11 ) so that we have: a 11 x 1 + a 12 x a 1n x n = b 1 + a 22x a 2nx n = b 2 + a 32x a 3nx n = b 3 + a n2x a nnx n = b n Here the first equation is called the pivot equation and a 11 in a 21 /a 11 or a 31 /a 11 is called pivot element Sometimes the division operation is referred to as normalization A zro pivot element can interfere with normalization by causing a division by zero The next step is to eliminate x 2 we repeat the process above to get Finally, we have a 11 x 1 + a 12 x 2 + a 13 x a 1n x n = b 1 + a 22x 2 + a 23x a 2nx n = b 2 a 33x a 3nx n = b 3 a n3x a nnx n = b n Back substitution a 11 x 1 + a 12 x 2 + a 13 x a 1n x n = b 1 + a 22x 2 + a 23x a 2nx n = b 2 a 33x a 3nx n = b 3 a (n 1) nn x n = b (n 1) n x n = b(n 1) n a (n 1) nn This result can be back-substituted into the (n 1)th equation to solve for x n 1 This process is repeated to evaluate the remaining x s and we have x i = b(i 1) i n j=i+1 a(i 1) ij x j a (i 1) ii

8 8 Example 03 < Homework > Solve by Gauss elimination with exactly this order of equations(different order to the one in the book) 03x 1 02x x 3 = 714 3x 1 01x 2 02x 3 = x 1 + 7x 2 03x 3 = 193 MATLAB M-file : GaussNaive function x = GaussNaive(A,b) % GaussNaive(A,b); % Gauss elimination without pivoting % input : % A = coefficient matrix % b = right hand side vector % output : % x = solution vector [m,n] = size(a); if m ~= n, error( Matrix A must be square ); nb = n+1; Aug = [A, b]; % forward elimination for k = 1 :n-1 for i = k+1:n factor = Aug(i,k) / Aug(k,k) ; Aug(i,k+1:nb) = Aug(i, k+1:nb) - factor*aug(k,k+1:nb); % back substitution x = zeros(n,1); x(n) = Aug(n,nb)/Aug(n,n);

9 9 for i = n-1:-1:1 x(i) = ( Aug(i,nb) - Aug(i, i+1:n)*x(i+1:n))/aug(i,i); Operation counting The execution time depson the amount of floating-point operations(or flops) involved in the algorithm The time consumed to perform addition/subtraction and multiplication/division is about the same In forward elimination: The number of iteration of the inner loop will be n 1 1 In the loop there is one division to calculate the factor 2 The next line performs a multiplication and a subtraction for each column element from 2 to nb multiplications and n subtractions Because nb = n + 1, going from 2 to nb results in n 3 Therefore n + 1 multiplications/divisions and n addition/subtractions for every iteration of the inner loop 4 The total for the first pass through the outer loop is (n 1)(n + 1) multiplications/divisions and (n 1)n addition/subtractions 5 Similar reasoning can be estimate the flops for the subsequent iterations of the outer loop These can be summarized as Outer Inner Addition/ Multiplication/ Loop k Loop i Subtraction flops Division flops 1 2, n (n-1)n (n-1)(n+1) 2 3, n (n-2)(n-1) (n-2)n k k+1, n (n-k)(n+1-k) (n-k)(n+2-k) n-1 n, n (1)(2) (1)(3) Total Addition/Subtraction flops n 1 n 1 (n k)(n + 1 k) = [n(n + 1) k(2n + 1) + k 2 ] k=1 k=1

10 10 or n 1 n(n + 1) k=1 [n 3 + O(n)] [n 3 + O(n 2 )] + n 1 1 (2n + 1) k=1 n 1 k + k=1 k 2 [ ] 1 3 n3 + O(n 2 ) = n3 3 + O(n) Similarly for Multiplication/division flops [ ] 1 [n 3 + O(n 2 )] [n 3 + O(n)] + 3 n3 + O(n 2 ) = n3 3 + O(n2 ) Summing these results give 2 3 n3 + O(n 2 ) So for large n, in forward elimination the time consumed converges on 2n3 3 In back substitution: consumed (WHY? Homework!! ) n 2 + O(n) The total effort in naive Gauss Elimination is 2 3 n3 + O(n 2 ) + n 2 + O(n) = 2 3 n3 + O(n 2 ) n Elimination Back Total Percent Due substitution Flops to Elimination % % % PIVOTING Division by zero(or division by very very small number) 2x 2 + 3x 3 = 8 4x 1 + 6x 2 + 7x 3 = 3 2x 1 3x 2 + 6x 3 = 5 Division a number by relatively very small number also causes a round-off errors, then give us a wrong solution Partial Pivoting

11 11 Before each row is normalized, choose the largest absolute value in the column below the pivot element the rows can then be switched so that the largest element is the pivot element complete Pivoting If columns as well as rows are searched for the largest element and then switched, the procedure is called complete pivoting It is rarely used because switching columns changes the order of the x s and adds significant complexity to the computer program Aside from avoiding division by zero, pivoting also minimizes round-off error Example x x 2 = x x 2 = Solution Multiplying 1/00003 yields x x 2 = 6667 Eliminating x 1 from the second equation 9999x 2 = 6666 Then x 2 = 2/3 and x 1 = (2/3) Significant Absolute value of Percent Figures x 2 x 1 Relative Error for x On the other hand, if the equation are solved in reverse order, 10000x x 2 = x x 2 = 20001

12 12 Then x 2 = 2/3 and x 1 = 1 (2/3) 1 This case is much less sensitive to the number of significant figures in the computation: Significant Absolute value of Percent Figures x 2 x 1 Relative Error for x MATLAB M-file : GaussPivot function x = GaussPivot(A,b) % GaussPivot(A,b); % Gauss elimination with pivoting % input : % A = coefficient matrix % b = right hand side vector % output : % x = solution vector [m,n] = size(a); if m ~= n, error( Matrix A must be square ); nb = n+1; Aug = [A, b]; % forward elimination for k = 1 :n-1 % partial pivoting [big, i] = max(abs(aug(k:n,k))); ipr = i+k-1; if ipr ~=k %pivot the rows Aug([k,ipr],:)=Aug([ipr,k],:);

13 13 for i = k+1:n factor = Aug(i,k) / Aug(k,k) ; Aug(i,k+1:nb) = Aug(i, k+1:nb) - factor*aug(k,k+1:nb); % back substitution x = zeros(n,1); x(n) = Aug(n,nb)/Aug(n,n); for i = n-1:-1:1 x(i) = ( Aug(i,nb) - Aug(i, i+1:n)*x(i+1:n))/aug(i,i); [y,i] = max(x) return y, the largest element in the vector x, and i, the index corresponding to that element TRIDIAGONAL SYSTEMS as A tridiagonal system has a bandwidth of 3 and can be represented generally f 1 g 1 e 2 f 2 g 2 e 3 f 3 g 3 e n 1 f n 1 g n 1 e n f n x 1 x 2 x 3 x n 1 x n = r 1 r 2 r 3 r n 1 r n

14 14 Example x 1 x 2 x 3 x 4 = Solution f 2 = ( 1) = r 2 = 08 1 (408) = After performing similar calculation, we have By back substitution, the solution is: x 1 x 2 x 3 x 4 = x 4 = r 4 f 4 = = x 3 = r 3 g 3 x 4 f 3 x 2 = r 2 g 2 x 3 f 2 x 1 = r 1 g 1 x 2 f ( 1) = = ( 1) = = ( 1)93778 = = function x = Tridiag(e,f,g,r) % Tridiag(e,f,g,r); % Tridiagonal system solver % input : % e = subdiagonal vector % f = diagonal vector % g = superdiagonal vector % r = right hand side vector % output :

15 15 % x = solution vector n = length(f); % forward elimination for k = 2 :n factor = e(k)/f(k-1); f(k) = f(k) - factor*g(k-1); r(k) = r(k) - factor*r(k-1); % back substitution x = zeros(n,1); x(n) = r(n)/f(n); for k = n-1:-1:1 x(k) = ( r(k) - g(k)*x(k+1))/f(k);