Linear Algebraic Equations

Transcription

1 Linear Algebraic Equations

2 Linear Equations: a + a + a + a a = c n n 1 a + a + a + a a = c n n 2 a + a + a + a a = c n n a + a + a + a a = c m1 1 m2 2 m3 3 m4 4 mn n m m equations in n unknowns Where a ij are constant coefficients, c j are constants.

3 Brief Review of Matrices [ A] a11 a12... a1 n a a... a am1 am2... amn 22 2n = [m*n] matri İf m=n square matri If m=1, [1*n] row vector and if n=1 [m*1] column vector [ ] C c1 c2... cn b b.. b = 1 b 2 = m

4 Matri is symmetric if: a ij =a ji Matri is diagonal if a i j=0 for i j for all i Matri is identity Matri, I, if a i j=0 for i j for all i and a ii =1 Upper and Lower Triangular Matrices Banded matri Transpose of a matri

5 Matri Operations [A]=[B] if both matrices have the same m rows, n columns and a ij =b ij for all i,j Addition and Subtraction: [C]=[A]+[B], c ij =a ij +b ij [C]=[B]+[A] = [A]+[B] Commutative ([A]+[B])+[C]= [A]+([B]+[C]) Associative

6 Multiplication by a constant: k[a]=g g ij =ka ij Multiplication of Matrices: [C]= [A] [B] m*l m*n n*l c ij = Σa ik b kj (AB)C=A(BC) (Associative) A(B+C)=AB + AC (Distributive) AB BA (is not commutative)

7 Trace of a Matri: Augmentation: [ ] tr A n = a i= 1 ii Matri A augmented by an identity Matri: [ ] a a a = AI, a a a a a a AX Matri A augmented by B: = B a a a. b A, B a a a. b [ ] = a a a. b

8 Inverse of a Matri A 1 A 1 If (Inverse of Matri A)İs such that 1 1 A A= AA = I AX = B Multiply both sides by A 1 Since 1 A A= I 1 1 A AX = A B X = 1 A B

9 DETERMINANT (Defined for square matrices only) 22 a a a A a a a [ ] = a a a a a D = = a11a22 a12a21 a21 a22 Determinant of Matri A a a a D= a a a a a a or a a a a a a a a a D = a a a = a a + a a32 a33 a31 a33 a31 a a a a a a a a a a a a a D= a a a = a a + a a32 a33 a32 a33 a22 a a a a

10 Gauss Elimination a + a + a + a a = c n n 1 a + a + a + a a = c n n 2 a + a + a + a a = c n n a + a + a + a a = c n1 1 n2 2 n3 3 n4 4 nn n n n equations in n unknowns

11 When n is small: (Graphical Method) a + a = b a + a = b Solve for 2 a = + b a12 a = + b 21 2 a22 = slope* + intercept = = 2 3 = =

12 No Solution Infinite Solutions Ill Conditioned

13 When n is small Cramer s Rule a11 a12 a13 1 b1 a a a = b a31 a32 a 33 3 b 3 1 = b a a b a a 2 23 b a a D D is the determinant of the coefficient Matri A

14 When n is small: Elimination of the unknowns a + a = b a + a = b a a + a a = a b a a + a a = a b a22a112 a21a122 = a11b2 a21b1 2 = a b a a a b a a

15 Naive Gauss Elimination a + a + a + a a = c n n 1 a + a + a + a a = c n n 2 a + a + a + a a = c n n a + a + a + a a = c n1 1 n2 2 n3 3 n4 4 nn n n Step 1: eliminate 1 from the second upto nth equation a21 Multiply the first equation by: a21 a11 a11 a a a a a a211 + a122 + a133 + a a1 nn = c1 a a a a a Subtract it from the second equation: the result is: a a a a a a + a a a a = c c or n 1n n 2 1 a11 a11 a11 a11 a' + a' a' = b' n n

16 Repeat the same procedure to eliminate 1 from the other equations a + a + a + a a = c n n 1... a' + a' + a' a' = c' n n 2... a' + a' + a' a' = c' n n 3... a' + a' + a' a' = c' n2 2 n3 3 n4 nn n n First equation is the pivot equation and a 11 is the pivot coefficient Repeat the above procedure to eliminate 2 from the third upto nth equation a + a + a + a a = c n n 1... a' + a' + a' a' = c' n n a" + a" a" = c" a" n n 3 + a" a" = c" + n3 3 n4 4 nn n n

17 Repeat the procedure (n-1) times till n-1 is eliminated from the nth equation. a + a + a + a a = c n n 1... a' + a' + a' a' = c' n n a" + a" a" = c" Upper triangular system is formed n n 3 n... + a = c ( 1) ( n 1) nn n n Determine i by back substitution n = c a ( n 1) n ( n 1) nn i = n ( i 1) ( i 1) i ij j j=+ i 1 c a a ( i 1) ii

18 Forward elimination Back Sustitution

19 Eample (Pr.9.10): 2 + = = = 5 3 Back Substitution = = = = = = [ ] = = [ 1 ( 1) 3 ] = 14 2 Forward Elimination = = = 10 Check your result!!! 3

20 Eample = = = 2 3 Augmented Matri We shall use 3 digits Row2-1/3*Row1... Row3-1/3*Row Row3-(-1.334/2.333)*Row X 3 =1.993, 2 =( *1.993)/2.333=1.008, X 1 =(12-2* *1.008)/3=3.007 (actual result is 2,1,3)

21 Operation Count Elimination: Outer Loop k k Middle Loop i 2,n 3,n.. k+1,n FLOPS [n-1][1+n] [n-2][n] [n-k][1n+2- k] n-1 n,n [1][3] Number of Mult&Div= n On 2 ( ) Back Substitution: 2 On ( ) Therefore, the number of operations is proportional to n 3 /3

22 Pitfalls: 1. Division by zero: If the pivot element is zero i.e. a 11 =0 2. Round off errors (especially when n is large) 3. Ill conditioned systems (when the determinant D 0 but, D~0 Improvements: 1. Use more significant figures (time?) 2. Pivoting Partial pivoting: Determine the largest coefficient in the column below the pivot element and switch rows (equations). Complete pivoting: Determine the largest element (in rows and columns and switch the rows (equations) and columns ( s) Too costly!!! 3. Scaling (normalization- make the highest coefficient equal to unity)

23 Eample (Partial pivoting): No Pivoting = = We shall use 4 digit accuracy. With Pivoting = = Multiply the first eq. By /0.0003= Multiply the first eq. By /0.3454= = = = = X 2 = X 1 =( *1.001)/.0003= X 2 = X 1 =( *0.9999)/0.3454=

24 Eample (Scaling): We shall use 3 significant digits = = = = X 2 =1 and 1 =0 Pivot, but do not scale = = = = If we scale the equations Pivot = =... + = = 1 + = = 1 X 2 =1 and 1 =1 2

25 Eample (Partial Pivoting) = = = = Eliminate Eliminate Pivot change Use back substitution to obtain: X 4 = ; 3 = ; 2 = ; 1 = Pivot change Eliminate

26 Pseudo Code

27 Comments on pseudocode Equations are not scaled but scaled values of the elements are used to determine whether pivoting is required. Diagonal term is monitored to determine near-zero occcurrence s i : is the value of the maimum element on ith row. a: Coefficient matri, b: column vector of a set of equations a=b n: number of equations (or unknowns) to be solved. tol, a small value defined by the user. If the diagonal element <tol the matri is considered as singular and er is set to -1. er is to be returned by the program.

28 Gauss Jordan Augmented Matri Interchange row 1&4 and normalize = = = = Eliminate Normalize 3rd row and eliminate Interchange Row 2&3, normalize & eliminate all elements in the second column. Normalize 4th row and eliminate Requires more work (n 3 /2, compared to n 3 /3 İn Gauss elimination)

29 Gauss-Jordan Method Eliminate the unknown from all but one of the equations. Pivot and Normalize 2 + = = = 5 3 Eliminate /3 1/3 5/ /5 2/5 4/ /2 1/2 1/2 1 1/3 1/3 5/3 1 1/3 1/3 5/ /15 1/15 37/ /6 5/6 7/ Right Column is the result. No back substitution

30 Inverse of a matri Using Gauss-Jordan: AA -1 =I Augment A with I 1 A = Change Row 1&2 and normalize Normalize and eliminate Eliminate Normalize and eliminate A = One can also use LU decomposition eplained in the book (pg 273).

31 Iterative Methods Guess a solution and improve your solution The Equations must be diagonally dominant Consider: a + a + a = b a + a + a = b a + a + a = b Solve the first eq. For 1, second for 2 and the third for 3 : = = = b a a a b a a a b a a a

32 Iterative Method 1: Gauss-Seidel 1. Assume i =0 (i=0, 1,2, n) 2. Solve for 1 from the first eq. 3. Solve for 2 from the second eq. (use 1 obtained in step 2) 4. Solve for 3 from the 3rd eq. (use 1 and 2 values) 5. Solve for 1 from the 1st eq. (use the latest values available) 6. Repeat until: j j 1 i i ε ai, = *100% < ε j s i b a a = a11 b a a = a22 b a a = a

33 8 + = = = 12 3 = = = ε s <10-6

34 PseudoCode for GaussSeidel

35 Start Iteration Iter=1 Do While Iter<MaIterations Sentinel=1 For i= 0 to n-1 Old=(i) (i)= BC(i) For j= 0 to n-1 If i<>j Then (i)=(j)-ac(i,j)*(j) Net j (i)=lambda*(i)+(1-lambda)*old = λ + (1 λ) new new old i i i 0< λ < 2 Relaation: λ=1 Original Gauss-Seidel λ<1 underrelaation λ>1 overrelaation

36 Iterative Method 2: Jacobi Compute a set of new values before using them in the equations. Gauss-Seidel Jacobi

37 Eample: 8 + = = = 12 3 = = = Jacobi Gauss-Seidel ε s <

38 Matri Norms and Matri Condition Number Norm is the largest sum of the columns A = ma n 1 1 j n i = 1 a ij Norm is largest sum of the rows A = ma 1 i n n j = 1 a ij Uniform Matri Norm or Row-Sum Norm Condition Number of Matri A (>1) 1 Cond [ A] = A A

39 System of Nonlinear equations N nonlinear equations in n unknowns f (,,,,,... ) = f (,,,,,... ) = f (,,,,,... ) = 0 n 3 4 n n n Sometimes you can use a fied point iteration or an iteration similar to Gauss Seidel Iteration for linear equations. Newton Raphson method for n variables Newton Raphson Method for one variable: f( ) = f( ) + '( )( f ) ( 0 f f f = f + ) + ( ) ( ) + HOTerms 0 i+ 1 1, i 1, i 1, i i+ 1 i i i+ 1 i 1, i+ 1 1, i 1, i+ 1 1, i 2, i+, i n, i+ 1 n, i = i f2, i f2, i f2, i f( i ) f2, i+ 1 = f2, i + ( 1, i+ 1 1, i) + ( 2, i+ 1 2, i) ( n, i+ 1 n, i) + HOTerms f '( i )... f f f f = f + ( ) + ( ) ( ) + HOTerms 0 ni, ni, ni, ni, + 1 ni, 1, i+ 1 1, i 2, i+, i ni, + 1 ni, f f f f f f = f , i 1, i 1, i 1, i 1, i 1, i 1, i+, i+ 1 n, i+ 1 1, i 1, i 2, i n, i n n f f f f f f = f , i 2, i 2, i 2, i 2, i 2, i 1, i+, i+ 1 n, i+, i 1, i 2, i n n f f f f f f = f ni, ni, ni, ni, ni, ni, 1, i+, i+ 1 ni, + 1 ni, 1, i 2, i ni, n n ni,

40 J = i i+ 1 Bi J i f f f... d d d f f f i 2 i n i = d1 d i 2 d i n i fn fn fn.. d d d 1 i 2 i n i B i f1 f1 f1 f1, i + 1, i + 2, i... + n, i 1 i 2 i n i f2 f2 f 2 f2, i + 1, i + 2, i... + n, i = 1 i 2 i n i... fn fn fn fni, + 1, i+ 2, i... + ni, 1 i 2 i n i Solve for i+1 using any one of the methods discussed and repeat till there is convergence Or, if we let u i+1 = i+1 - i then J iu = i+ 1 Ci Where C i f1, i f 2, i =... fni, Solve for u i+1, i+1 =u i+1 + i

41 or Let: δ ki, = ki, + 1 ki, Aδ = B Then: i i i A i f f f... d d d f f f i 2 i n i = Ji = d1 d i 2 d i n i fn fn fn.. d d d 1 i 2 i n i B i f1, i f 2, i =... f2, i δ i δ1, i δ 2, i =... δ2, i = + +δ ki, 1 ki, ki,