Iterative methods: introduction and background. Vrije Universiteit Amsterdam

Transcription

1 Iterative methods: introduction and background André Ran Vrije Universiteit Amsterdam 1

2 Consider the set of linear equations (I + αt)u = v, where v is given, α > 0 and T is the symmetric tridiagonal Toeplitz matrix T =

3 Direct methods Last time we discussed direct methods: 3

4 Direct methods Last time we discussed direct methods: direct inversion of I + αt, 3

5 Direct methods Last time we discussed direct methods: direct inversion of I + αt, LU-method, based on the fact that I + αt is positive definite, 3

6 Direct methods Last time we discussed direct methods: direct inversion of I + αt, LU-method, based on the fact that I + αt is positive definite, dedicated methods based on the fact that I + αt is Toeplitz, fast Toeplitz inversion method based on Gohberg-Semencul formula. 3

7 Direct methods Last time we discussed direct methods: direct inversion of I + αt, LU-method, based on the fact that I + αt is positive definite, dedicated methods based on the fact that I + αt is Toeplitz, fast Toeplitz inversion method based on Gohberg-Semencul formula. This time we will discuss an iterative method. 3

8 Revised problem Rewrite the problem as follows: where T 1 = T 2I. ((1 + 2α)I + αt 1 )u = v, 4

9 Revised problem Rewrite the problem as follows: where T 1 = T 2I. ((1 + 2α)I + αt 1 )u = v, After dividing through by 1 + 2α: (I + M)u = α v = y, where M = α 1+2α T 1. 4

10 Now rewrite the problem in the following way: (I + M)u = y u = y Mu. 5

11 Now rewrite the problem in the following way: (I + M)u = y u = y Mu. Defining f : R n R n by f(u) = y Mu, we see that a solution of the original matrix-vector equation can be viewed as a fixed point of f, that is, a point for which f(u) = u. 5

12 Banach s fixed point theorem A map f : R n R n is called a contraction if there is a constant 0 c < 1 such that f(x 1 ) f(x 2 ) < c x 1 x 2 for all vectors x 1 and x 2. 6

13 Banach s fixed point theorem A map f : R n R n is called a contraction if there is a constant 0 c < 1 such that f(x 1 ) f(x 2 ) < c x 1 x 2 for all vectors x 1 and x 2. A very important theorem is Banach s fixed point theorem (also called the contraction mapping principle): 6

14 Banach s fixed point theorem A map f : R n R n is called a contraction if there is a constant 0 c < 1 such that f(x 1 ) f(x 2 ) < c x 1 x 2 for all vectors x 1 and x 2. A very important theorem is Banach s fixed point theorem (also called the contraction mapping principle): Theorem If f is a contraction on R n, then f has a unique fixed point ˆx. Moreover, for every vector x 0 in R n we have that the sequence of iterations of f on x 0 converges to the unique fixed point, that is if x n = f (n) (x 0 ), then limx n = ˆx. 6

15 Back to the original problem Recall the problem: where M = α 1 + 2α T 1 = α 1 + 2α u = y Mu;

16 Now the corresponding f(u) = y Mu has the property that f(u 1 ) f(u 2 ) M u 1 u 2. 8

17 Now the corresponding f(u) = y Mu has the property that f(u 1 ) f(u 2 ) M u 1 u 2. For every α > 0 we have that M < 1, since for every n the norm of T 1 is below 2, and M = α 1+2α T 1. 8

18 Now the corresponding f(u) = y Mu has the property that f(u 1 ) f(u 2 ) M u 1 u 2. For every α > 0 we have that M < 1, since for every n the norm of T 1 is below 2, and M = α 1+2α T The norm of T

19 Hence, for every x 0 R n the iteration x k+1 = y Mx k will converge to the unique solution of u = y Mu. Recall also that y = α v. 9

20 Hence, for every x 0 R n the iteration x k+1 = y Mx k will converge to the unique solution of u = y Mu. Recall also that y = α v. So to find the solution, iterate untill x k+1 x k < ε, where ε > 0 is a given tolerance. 9

21 Hence, for every x 0 R n the iteration x k+1 = y Mx k will converge to the unique solution of u = y Mu. Recall also that y = α v. So to find the solution, iterate untill x k+1 x k < ε, where ε > 0 is a given tolerance. This is what amounts to for solving the original equation (I + αt)u = v. 9

22 for the heat equation Denote by u m,k n Recall α = δτ δx 2. the value of u m n = u(nδx, mδτ) at the k-th iteration. 10

23 for the heat equation Denote by u m,k n Recall α = δτ δx 2. the value of u m n = u(nδx, mδτ) at the k-th iteration. in coordinates can be written as u m+1,k+1 n = α (bm n + α(u m+1,k n 1 + u m+1,k n+1 ) (in the notation of the book). 10

24 for the heat equation Denote by u m,k n Recall α = δτ δx 2. the value of u m n = u(nδx, mδτ) at the k-th iteration. in coordinates can be written as u m+1,k+1 n = α (bm n + α(u m+1,k n 1 + u m+1,k n+1 ) (in the notation of the book). For the initial vector x 0 = u m+1,0 at time level m + 1, we take the vector u m that results from the iterative computation at the previous time level. 10

25 for the heat equation Denote by u m,k n Recall α = δτ δx 2. the value of u m n = u(nδx, mδτ) at the k-th iteration. in coordinates can be written as u m+1,k+1 n = α (bm n + α(u m+1,k n 1 + u m+1,k n+1 ) (in the notation of the book). For the initial vector x 0 = u m+1,0 at time level m + 1, we take the vector u m that results from the iterative computation at the previous time level. Then iterate untill u m+1,k+1 u m,k < ε, where ε > 0 is a given tolerance. 10

26 rate The rate of convergence depends on M 2α 1+2α. We have for every k: u k+1 u k M u k u k 1 M k u 1 u 0. 11

27 rate The rate of convergence depends on M 2α 1+2α. We have for every k: u k+1 u k M u k u k 1 M k u 1 u 0. For large α this gets close to one, so then convergence is very slow. 11

28 rate The rate of convergence depends on M 2α 1+2α. We have for every k: u k+1 u k M u k u k 1 M k u 1 u 0. For large α this gets close to one, so then convergence is very slow. For moderate values of α, convergence can still be fairly good: α = 12 gives 2α 1+2α = = alpha/(1+2 alpha) for alpha in [1,12]

29 rate as function of α α=12, 0.96 k in blue, α=4, (8/9) k in black, α=1, (2/3) k in red, for k=1,..., Conclusion: when applying it pays to keep α moderate (near 1, say), otherwise the iteration will convergence very slowly. 12