Technische Universität München WT 1/13 Fakultät für Mathematik Prof. Dr. M. Mehl B. Gatzhammer January 1, 13 Numerical Programming I (for CSE) Tutorial 1: Iterative Methods 1) Relaxation Methods a) Let C R n n be a diagonalizable matrix. Show: ρ(c) < 1 lim i C i =. Hint: Write down the powers of C by using its diagonalized representation. b) Consider the SOR method, i.e., the Gauss-Seidel method with relaxation, x (i+1) = x (i) + αy (i), c) where y (i) is the update of the Gauss-Seidel iteration. Prove that α (, ) is a necessary (but not sufficient) condition for convergence. Hint: Use the determinant of the iteration matrix det(c) of the SOR method and consider the relation between the eigenvalues λ i of C and det(c). An intermediate result is det(c) = (1 α) n. For which matrices do the Jacobi method and the Gauss-Seidel method converge? 3 1 1.5 1 (i) 1 3 1 (ii) 1 1 (iii) 1 1 1 1 3.5 1 Solution: a) If C is diagonalizable, we can represent it as C = S 1 DS, where D is a diagonal matrix with the eigenvalues of C and S is a matrix made up by the eigenvectors of C. Computing C i, we get C i = (S 1 DS) i = S 1 DS S 1 DS... S 1 DS = S 1 DD...DS = S 1 D i S Since ρ(c) < 1, we have λ i < 1 and D i = diag(λ i 1,..., λ i n) i. Thus, lim i Ci = lim S 1 D i S = S 1 S =. i Remark: This can be also shown for an arbitrary matrix C. In the general case, you have to prove for the Jordan matrix J that J i i.
Numerical Programming I (for CSE) Tutorial 1: Iterative Methods, page b) With M = 1 α D A + L A, we get det(c) = det( M 1 (A M)) = triangular matrixes = = det(a M) = det (( 1 1 ) ) α DA + U A det( M) det ( 1 α D ) A L A det (( 1 1 ) ) ( α DA 1 det ( 1 α D ) = α) det(da ) ( A α) det(da ) ( ) 1 α 1 = (1 α) n α Thus, we have α / (, ) det(c) 1 i : λ i 1 a) C i i x x (i) i c) Matrix A is strictly diagonally dominant, iff a ii > j i a ij, i Jacobi Gauss-Seidel (i) + + A strictly diagonally dominant! (ii) + + Consider spectral radius of the iteration matrix! (iii) + Consider spectral radius of the iteration matrix! (ii) Jacobi: 3 M = D A = 1 1 C = M 1 3 4 3 (A M) = 1 1 = 1 eigenvalues of C: λ 1,,3 = ρ(c) = < 1 convergence (ii) Gauss-Seidel: equivalent to Jacobi iteration convergence M = D A + L A = D A Attention: The positive eigenvalues of A are not sufficient for convergence of Gauss-Seidel (or Jacobi) because it holds: positive eigenvalues But, for symmetric matrices, it follows in general positive definite. symmetric + positive eigenvalues positive definite.
Numerical Programming I (for CSE) Tutorial 1: Iterative Methods, page 3 (iii) Jacobi: M = D A = I, C = M 1 (A M) = I A = 1 1 From characterisitc polynomial: det(c λi) = λ 3 = it follows, that eigenvalues of C: λ 1,,3 = ρ(c) = < 1 convergence (iii) Gauss-Seidel: 1 M = D A + L A = 1 1, 1 1 C = M 1 (A M) = 1 1 1 = 3 1 eigenvalues of C: λ 1 =, λ,3 = ρ(c) = 1 no convergence ) Multilevel Methods Given the one-dimensional stationary heat equation T (x) x = s(x), (1) with x the spatial variable, T the temperature value, and s a source term. We look for a solution T (x) of equation (1) in the spatial domain Ω = [, π], with boundary conditions T () = T (π) =, and a source term s(x) = sin(x). To solve this problem numerically, a central finite difference scheme is applied, transforming equation (1) into the discrete algebraic form T i+1 + T i T i 1 h = s i. () Subindex i =,..., n + 1 denotes the discrete position x i = ih, where h is the discretization interval h = π/(n + 1), and n is the number of internal grid points, as denoted in Figure 1. The discrete unknowns T i are located at the internal nodes, boundary values are prescribed for T = T n+1 =. In the following, you will perform one coarse-grid correction of the fine-grid level solution. T 1 T i =T(x i ) T n 3.14 x h Figure 1: Spatial grid of discretized heat equation. a) For n f = 7, setup the fine-level system of equations A f x f = b f of the discrete problem given by equation (), the boundary conditions and source term. How can the resulting system matrix be classified? b) We do not aim at using the matrix in an explicit form, but to evaluate the equations only individually per unknown T i, i.e., node-wise, in an iterative manner. What advantages does this have? Using a Gauss-Seidel iterative scheme, write down the equation for updating one unknown, using index k and k+1 to denote the old and new iteration state of involved variables.
Numerical Programming I (for CSE) Tutorial 1: Iterative Methods, page 4 P c) P d) P e) P f) P g) Implement the Gauss-Seidel method for the given problem. Perform iterations on the fine grid (so called pre-smoothing iterations) starting from an initial guess of Ti =, i. For comparison, compute the numerical solution of the fine-grid system using Matlab s builtin direct equation solver and also derive the analytical solution of equation (1). Plot and explain the differences between the three solutions. Hint: The analytical solution is a simple trigonometric function. The direct solver solution is very close to it, while the smoothed solution (after two iterations) is about a quarter in magnitude of the others. To obtain a coarse-grid correction for the fine-grid approximation computed by the Gauss- Seidel method, we need to use the residual r f of the fine grid approximation. Derive and implement a node-wise evaluation of the residual. After computing the residual, setup a coarser mesh with h c = h f and restrict the computed residual to the coarse mesh by using injection, i.e., coarse-grid residual values are set by copying the values from coinciding fine-grid nodes. Hint: The residuals should be approximately as large as the expected errors for this example. On the coarse level, we now want to solve for the error A c e c = r c, to later use the coarse-grid error as correction on the fine-grid level. Setup the coarse-grid equation system, assuming zero error at the boundaries. Solve the system by using Matlab s builtin direct solver. Prolongate the coarse grid error e c up to the fine grid e f by linear interpolation, i.e., coinciding nodes have same value and intermediate fine grid nodes get an average of left and right coarse grid node values (remember the zero boundaries on the coarse grid). Correct the smoothed solution by x f = x f e f. Plot and compare the corrected solution to the other solutions obtained in c). Hint: After the correction, the solution should approximately at 8% of the direct solution. Perform post-smoothing iterations with the Gauss-Seidel method implemented in c). Why do you think is a post-smoothing useful? Plot and compare the post-smoothed solution to the other solutions obtained in c) and f). Solution: a) A f x f = 64 π h = π 8, 1 h = 64 π 1 1 1 1 1 1 1 1 1 1 1 1 T 1 T T 3 T 4 T 5 T 6 T 7 sin(1π/8) sin(π/8) sin(3π/8) = b f = sin(4π/8) sin(5π/8) sin(6π/8) sin(7π/8) The system matrix A f is a tri-diagonal matrix, i.e., a band-matrix with one main, upper, and lower diagonal. It is also sparse, i.e., most entries are zero, and symmetric. b) The advantage of not setting up an explicit matrix lies in not needing the storage for it. This allows for efficient dynamic grid adaptivity (e.g., grid changes and refinements) and possibly enables to solve for a larger amount of unknowns.
Numerical Programming I (for CSE) Tutorial 1: Iterative Methods, page 5 Gauss-Seidel update rule: Ti+1 k k+1 + Ti Ti 1 k+1 h = s i Ti k+1 = 1 (h s i Ti 1 k+1 T i+1) k Ti k+1 = 1 (h sin(x i ) Ti 1 k+1 T i+1) k P c) P d) Show and execute Matlab codes. Analytical solution: Exact solution of the original continuous problem. Direct solver solution: Numerical solution of the discretized problem and, thus, including a discretization and rounding error. Smoothed solution: Only the inital guess with smoothed error. The smoothening allows to represent the residual on a coarser grid, necessary for the coarse grid correction. Residual in matrix notation: r f = b f A f x f Component wise (omitting subindex f): n r i = b i a ij x j = sin(x i ) + 1 h (T i 1 T i + T i+1 ) j=1 Number of nodes on coarse level: n c = π h c 1 = π h f 1 = Injection of residuals: (draw sketch of injection) π π/(n f + 1) 1 = n f + 1 1 = 7 + 1 1 = 3. r f, r c = r f,4 r f,6 P e) P f) P g) Coarse-grid correction system: A c e c = 16 1 π Execute Matlab codes. 1 1 1 e c,1 e c, e c,3 Prolongation: (draw sketch of linear interpolation).5e c,1 e c,1.5(e c,1 + e c, ) e f = e c,.5(e c, + e c,3 ) e c,3.5(e c,3 ) Execute matlab codes. Execute Matlab codes. = r c = Post-smoothing removes high-frequency errors potentially introduced by the prolongation. r f, r f,4 r f,6