Iterative Techiques for Solvig Ax b -(8) Cosider solvig liear systems of them form: Ax b where A a ij, x x i, b b i Assume that the system has a uique solutio Let x be the solutio The x A b Jacobi ad Gauss-Seidel Methods: Let A D L U where D diag a ii, L l ij, a lower triagular matrix where l ij a ij for i j, adu u ij a upper triagular matrix where u ij a ij for i j 4 5 6 The 7 8 D 5, L 4, U 6 7 8 Assume a ii fori,, Ax b D L U x b a Jacobi s Method: Dx L U x b x D L U x D b x k D L U x k D b b Gauss-Seidel Method: D L x Ux b D L Ux D L b x k D L Ux k D L b c Successive Over-Relaxatio (SOR) Methods: Cosider Ax b Ax b D L U b D D D L U b D L x D U x b x D L D U x D L b Choose so that D L is ivertible ad T is as small as possible Whe, the method is called SOR method; whe, the method is called successive uder-relaxatio method Example Solve the system of liear equatios Ax b where A 4 5 ad b 7 by Jacobi, Gauss-Seidel ad SOR method usig MatLab file: ex_jacobi_gs_sorm
Covergece of Jacobi, Gauss-Seidel ad SOR Methods: Cosider the iterative methods of the form: x k Tx k c where T is a matrix ad c is vector For Jacobi Method: T Jac D L U, c Jac D b; for Gauss-Seidel Method: T GS D L U, c GS D L b ad for SOR methods: T SOR D L D U, c SOR D L b Questios: a Uder what coditio(s), x k x; ad b uder what coditio(s), x k x A b? Lemma If T, the I T exists ad I T I T T j Proof: Sice T, all eigevalues of T are less tha So, zero is ot a eigevalue of I T ad therefore, I T is ivertible ad I T exists Let S m m j T j The S m I T T T m, TS m T T T m S m TS m I T m, I T S m I T m, S m I T I T m Sice T, lim k T k lim S m m m lim I T I T m I T T j Theorem For ay x i R, x k coverges if ad oly if T Proof: Observe that x k Tx k c T Tx k c c T x k Tc c T k x T k T k T I c limx k lim T k x T k T k T I c I T c k k So, if T, x k I T c Is I T c A b? Observe the followig: Jacobi Method: I T Jac c Jac I D L U D b D I D L U b D L U b A b x Gauss-Seidel Method:
I T GS c GS I D L U D L b D L I D L U b D L U b A b x SOR Methods: I T SOR c SOR I D L D U D L b D L D U D L D L b D L U b A b A b Hece, if x k coverges the it coverges to x Uder what coditio(s), T Jac, T GS ad T SOR? Defiitio A matrixa a ij is said to be strictly diagoal domiat if a ii i j a ij for all i,, 4 Example A 4, A 4 A is strictly diagoal domiat but A is ot Theorem If A is strictly diagoal domiat, the for ay choice of x, both the Jacobi ad Gauss-Seidel methods coverge We will show that if A is strictly diagoal domiat, the () T Jac ad () T GS () Show T Jac byshowigthat T Jac Sice A is strictly diagoal domiat, for i, a ii j i a ij a ii j i T Jac D L U, T Jac max i a ij a ii j i a ij T Jac T Jac () Show T GS by showig that all eigevalue i s of T GS are less tha ad the clearly, T GS max i i Recall that T GS D L ULet, x be a eigepair of T GS where x i R, x The T GS x x D L Ux x Ux D L x Suppose that x p for some p First let us assume that p, that is, x ad x i for i,, Observe that the first row of the equatio: Ux D L x: a x a x a x The a x x a a x a x a x a x a x a x a a
ad a a a Now let us assume that p, that is, x ad x i fori,,, Observe that the d row of the equatio: Ux D L x: a x a x a x a x The a x a x a x a x ad a x x a a x a x a x a x a x a x a x a x a x a a a that implies, a a a a From this iequality, we have a a a a For more geeral, assume x p ad x i fori,,p,p,, Observe that the pth row of the equatio: Ux D L x: a p,p x p a p,p x p a p, x a pp x p a p x a p x a p x a pp x p a p,p x p a p,p x p a p, x a p x a p x a p x a pp x p a pp a p,p x p a p,p x p a p, x a p x a p x a p,p x p a p,i a pi i p i p a pp x p i p a pi a pp x p i i p a p,i a pp x p p i a pi a pi i p a p,i 4 4 ad b ad x 4 Eigevalues of T Jac are:, ad ; ad eigevalues of T 4 4 GS are, 59496569 ad 66569 Hece, T Jac ad T GS 66569 Both methods coverge Sice T GS T Jac we expect Gauss-Seidel Method coverges faster Check with the MatLab programs Jacobim ad Gauss_Seidelm [xsol,x,flag,k] Jacobi(A,[;;],,zeros(,),^(-8),); 4
Jacobi Method coverges k 6 [xsol,x,flag,k] Gauss_Seidel(A,[;;],,zeros(,),^(-8),); Gauss-Seidel Method coverges k 5 ad b ad x Note that the coditio i Theorem is a sufficiet coditio, that is, Jacobi or Gauss-Seidel Method could still coverge eve if A is ot positive defiite For this example, clearly A is ot strictly positive defiite However, eigevalues of T Jac are:, ad ; ad eigevalues of T GS are,, ad Hece, T Jac ad T GS Both methods coverge Sice T GS T Jac we expect Gauss-Seidel Method coverges faster Check with the MatLab programs Jacobim ad Gauss_Seidelm [xsol,x,flag,k] Jacobi(A,[;;],,zeros(,),^(-8),); Jacobi Method coverges k 54 [xsol,x,flag,k] Gauss_Seidel(A,[;;],,zeros(,),^(-8),); Gauss-Seidel Method coverges k 9 Theorem If a ij, for each i j, ad a ii for each i,,,, the oe ad oly oe of the followig statemets holds: a T GS T J ; b T J T GS ; c T J T GS ; d T J T GS Therefore, if both methods coverge, the Gauss-Seidel Method is better The previous example has illustrated the results give i Theorem Defiitio A symmetric matrix A is positive defiite if x T Ax for all ozero vector i R Note: A symmetric matrix is positive defiite if ad oly if all its eigevalues are positive 4 4 4 is symmetric Its eigevalues are 5,5, so it is positive defiite Defiitio A matrixa a ij is a tridiagoal matrix if a ij wheever i j or j i 5
4 4 4 4 is a tridiagoal matrix Theorem 4 Let T T SOR for a give The () If a ii, for each i,,,, the T () If A is a positive defiite matrix ad, the the SOR method coverges for ay choice of iitial approximatio vector x () If A is a positive defiite ad tridiagoal matrix, the T GS T Jac, ad the optimal choice of for the SOR method is opt T Jac With this choice of opt, T opt opt () Let i be eigevalue of T det T det D L D U det D L det D U a a a a det T k k k at least for oe k T max k k This implies that the SOR method ca coverge oly if ad b ad x Previously, we have T Jac ad T GS T Jac A is a symmetric positive defiite tridiagoal matrix The opt 757875 ad T opt 757875 757875 Check with the MatLab programs: 6
[xsol,x,flag,k] SOR(A,[;;],,zeros(,),^(-8),, ); SOR Method coverges k k 5 7 7 57 875 4 4 5 5 Rate of Covergece of Jacobi ad Gauss-Seidel Methods: Theorem 5: If T for all atural matrix orm ad c is a give vector, the x k defied by x k Tx k c coverges for ay x to a vector x where x ad x are i R, ad a x x k T k x x ; ad b x x k T k T x x where x A b Proof: a Observe that x Tx c, ad x x k Tx c Tx k c T x x k T x x k T k x x Sice x x k T x x x x k x x k T, by the defiitio of rate of covergece, we kow x k x liearly, ad the asymptotic error costat is less tha T So, the smaller T is, the faster x k x b Because we do t kow x at advace, x x Tx c x x x Tx c x Tx c x T x x x x I T x x x x I T x x x x k T k x x T k I T x x T k I T T x x T k T T x x T k T x x 7