Notes on Multigrid Metods Qingai Zang April, 17 Motivation of multigrids. Te convergence rates of classical iterative metod depend on te grid spacing, or problem size. In contrast, convergence rates of multigrid metods does not. Te complexity is O(n). Boo: Brigg, Henson, and McCormic A multigrid tutorial, SIAM, nd ed. Encyclopedic website: www.mgnet.org 1 Te model problem: 1D Possion equation. On te unit 1D domain x [, 1], we numerically solve Poisson equation wit omogeneous boundary condition u = f, x() = x(1) =. (1) Discretize te domain wit n cells and locate te nowns f j and unnowns u j at nodes x j = j/n = j, j =, 1,..., n. We would lie to approximate te second derivative of u using te discrete values at te nodes. Using Taylor expansion, we ave u x = u j+1 + u j 1 u j j + O( ). () Definition 1. Te one-dimensional second-order discrete Laplacian is a Toeplitz matrix A R (n 1) (n 1) as, i = a ij = 1, i = ±1 (3), oterwise Ten we are going to solve te linear system were f j = f(x j ). Au = f, (4) Proposition. 1 (Au) j ( u) xj = O( ), j = 1,..., n 1. (5) Proposition 3. Te eigenvalues λ and eigenvectors w of A are were j, = 1,,..., n 1. λ (A) = 4 sin π n, (6) w,j = sin jπ n, (7) 1
Proof. use te trigonmetric identity sin α + sin β = sin α + β cos α β. Remar 1. Te D counterpart of A is A I + I A. Note tat tis is not true for variable coefficient Poisson equation. Te residual equation Definition 4. For an approximate solution ũ j u j, te error is e = u ũ, te residual is r = f Aũ. Ten Ae = r (8) olds and it is called te residual equation. As one advantage, te residual equation lets us focus on omogenuous Diriclet condition WLOG. Question 1. For inexact aritmetic, does a small residual imply a small error? Definition 5. Te condition number of a matrix A is cond(a) = A A 1. It indicates ow well te residual measures te error. Ax (Ax, Ax) (x, A A = sup = sup = sup T Ax) = λ max (A x x x (x, x) x (x, x) T A) Since A is symmetric, A = λ max (A). A 1 = λ max (A 1 ) = λ 1 min (A). To give you an idea about te magnitude of cond(a), for n = 8, cond(a)=3, for n = 14, cond(a)=4.3e+5. Teorem 6. 1 r e cond(a) r (9) cond(a) f u f Proof. Use Ax A x and te equations Ae = r, A 1 f = u. 3 Fourier modes and aliasing Hereafter Ω denote bot te uniform grid wit n intervals and te corresponding vector space. Wavelengt refers to te distance of one sinusoidal period. Proposition 7. Te t Fourier mode w,j = sin(x j π) as wavelengt L w =. Proof. sin(x j π) = sin(x j + L )π implies x jπ = (x j + L )π π. Hence = L w. Te wavenumber is te number of crests and trougs in te unit domain. Question. Wat is te range of representable wavenumbers on Ω? For n = 8, consider = 1,, 8. rep [1, n). Wat appens to modes of > n? E.g. te mode wit = 3n/ is represented by = n/. Plot te case of n = 4. Proposition 8. On Ω, a Fourier mode w = sin(x j π) wit n < < n is actually represented as te mode w were = n. Proof. sin(x j π) = sin(jπ x j π) = sin(x j (n )π) = sin(x j π) = w. Definition 9. On Ω, te Fourier modes wit wavenumbers [1, n/) are called low-frenquency (LF) or smoot modes, tose wit [n/, n 1) ig-frenquency (HF) or oscillatory modes.
4 Te spectral property of weigted Jacobi Te scalar fixed-point iteration converts te problem of finding a root of f(x) = to te problem of finding a fixed point of g(x) = x were f(x) = c(g(x) x) c. Classical iterative metods split A as A = M N and convert (4) to a fixed point (FP) problem u = M 1 Nu + M 1 f. Let T = M 1 N, c = M 1 f. Ten fixed point iteration yields After l iterations u (l+1) = T u (l) + c. (1) e (l) = T l e (). (11) Obviously, te FP iteration will converge iff ρ(t ) = λ(t ) max < 1. Decompose A as A = D + L + U. Jacobi iteration as M = D, N = (L + U), T = D 1 (L + U), i.e. { 1 t ij =, i j = ±1 (1), oterwise Here ρ(t ) = 1 O( ). As, ρ(t ) 1, and Jacobi converges slowly. Consider a generalization of te Jacobi iteration. Definition 1. Te weigted Jacobi metod splits A as A = D + L + U were D, L, U are diagonal, lower triangular, and upper triangular, respectively, and ten performs fixed point iterations, u = D 1 (L + U)u (l) + D 1 f, u (l+1) = (1 ω)u (l) + ωu. (13a) (13b) Setting ω = 1 yields Jacobi. Proposition 11. Te weigted Jacobi as te iteration matrix T ω = (1 ω)i ωd 1 (L + U) = I ω A, (14) wose eigenvectors are te same as tose of A, wit te corresponding eigenvalues as were = 1,,..., n 1. λ (T ω ) = 1 ω sin π n, (15) See Fig..7. For n = 64, ω [, 1], ρ(t ω ).9986. Not a great iteration metod eiter. Wy? Loo more under te ood to consider ow weigted Jacobi damps different modes. Write e () = c w, ten e (l) = Tωe l () = c λ l (T ω )w. (16) No value of ω will reduce te smoot components of te error effectively. λ 1 (T ω ) = 1 ω sin π n 1 ωπ. (17) Having accepted tat no value of ω damps te smoot components satisfactorily, we as wat value of ω provides te best damping of te oscillatory modes. Definition 1. Te smooting factor µ is te smallest factor by wic te HF modes are damped per iteration. An iterative metod is said to ave te smooting property if µ is small and independent of te grid size. 3
For weigted Jacobi, tis optimization problem is µ = min [n/,n) λ (T ω ), for ω (, 1]. (18) λ (T ω ) is a monotonically decreasing function, and te minimum is terefore obtained by setting λ n/ (T ω ) = λ n (T ω ) ω = 3. (19) Exercise: ω = 3 λ µ = 1 3 () See Figure.8 and.9. Regular Jacobi is only good for modes 16 48. For ω = 3, te modes 16 < 64 are all damped out quicly. 5 Two-grid correction Proposition 13. Te t mode on Ω becomes te t mode on Ω : However, te LF modes [ n 4, n ) of Ω will become HF modes on Ω. Proof. w,j = sin jπ n w,j = w,j. (1) jπ = sin n/ = w,j, () were [1, n/). Because of te smaller range of on Ω, te mode wit [ n 4, n ) are HF by definition since te igest wavenumber is n on Ω. Definition 14. Te restriction operator I : Rn 1 R n/ 1 maps a vector on te fine grid Ω to its counterpart on te coarse grid Ω : I v = v. (3) A common restriction operator is te full-weigting operator were j = 1,,..., n 1. v j = 1 4 (v j 1 + v j + v j+1), (4) Definition 15. Te prolongation or interpolation operator I : Rn/ 1 R n 1 maps a vector on te coarse grid Ω to its counterpart on te fine grid Ω : A common prolongation is te linear interpolation operator I v = v. (5) vj = vj, vj+1 = 1 (v j + vj+1 ). (6) Te ey idea is tat te weigted Jacobi wit ω = 3 damps HF modes effectively, we can exploit tis on a series of successively coarsened grides to eliminite HF modes. Definition 16. For Au = f, te two grid correction sceme consists of te following steps: v MG(v, f, ν 1, ν ) (7) 4
1) Relax A u = f ν 1 times on Ω wit initial guess v : v T ν1 ω v + c (f), ) compute te fine-grid residual r = f A v and restrict it to te coarse grid by r = I r I (f A v ), 3) solve A e = r on Ω : e (A ) 1 r, r : 4) interpolate te coarse-grid error to te fine grid by e = I e and correct te fine-grid approximation: v v + I e, 5) Relax A u = f ν times on Ω wit initial guess v : v T ν1 ω v + c (f). Proposition 17. Let T G denote te iteration matrix of te two-grid correction sceme. Ten [ T G = Tω ν I I (A ) 1 I A ] Tω ν1. (8) Proof. By definition, te two-grid correction sceme replaces te initial guess wit v Tω ν1 v + c (f) + I(A ) 1 I [ f A (T ν1 ω v + c (f)) ], (9) wic also olds for te exact solution u. Subtracting te two equations yields (8). 5.1 Te spectral picture Our objective is to sow tat T G.1 for ν 1 =, ν =. For tis purpose, we need to examine te intergrid transfer operators. Definition 18. w ( [1, n/)) and w ( = n ) are called complementary modes on Ω. Proposition 19. For a pair of complementary modes on Ω, we ave Proof. w,j = sin (n )jπ n ( = sin jπ jπ n w,j = ( 1) j+1 w,j (3) ) = ( 1) j+1 w,j. Lemma. Te action of te full-weigting operator on a pair of complementary modes is I w = cos π n w = c w, I w π = sin n w = s w, were [1, n/), = n. In addition, I w n/ =. Proof. For te smoot mode, I w = 1 (j 1)π sin + 1 jπ sin 4 n n + 1 (j + 1)π sin = 1 4 n (1 + cos π jπ ) sin n n (31a) (31b) π = cos n w, were te last step uses Proposition 13. As for te HF mode, follow te same procedure, but replace wit n, use Proposition 8 for aliasing, and notice tat j is even. Te full-weigting operator tus maps a pair of complementary modes to a multiple of te smoot mode, wic migt be a HF mode on te coarse grid. Lemma 1. Te action of te interpolation operator on Ω is were = n. Iw = c w s w, (3) 5
Proof. Proposition 19 and trignometric identities yield ( c w s w = cos π n + ( 1)j sin π ) { w = n On te oter and, by Definition 15, we ave { ( I w ) j == 1 π(j 1)/ sin n/ + 1 w, cos π n w, w, π(j+1)/ sin n/ = cos π n w j is even, j is odd. j is even,, j is odd. Hence, te range of te interpolation operator contains bot smoot and oscillatory modes. In oter words, it excites oscillatory modes on te fine grid. However, if n, te amplitudes of tese HF modes s O( n ). Teorem. Te two-grid correction operator is invariant on te subspaces W = span{w, w }. were λ is te eigenvalue of T ω. T Gw = λ ν1+ν s w + λ ν1 λν s w T Gw Proof. Consider first te case of ν 1 = ν =. (33a) = λ ν1 λ ν c w + λ ν1+ν c w, (33b) A w = 4s w I A w = 16c s w (A ) 1 I A w = 16c s 4 sin π w = w n/ (34a) (34b) (34c) I (A ) 1 I A w = c w + s w (34d) [ I I(A ) 1 I A ] w = s w + s w, (34e) were te additional factor of 4 in (34b) comes from te fact tat te residual is scaled by and te trigonmetric identity sin(θ) = sin θ cos θ is applied in (34c). Similarly, A w = 4s w = 4c w (35a) I A w = 16c s w (A ) 1 I A w = 16c s 4 sin π w = w n/ (35b) (35c) I (A ) 1 I A w = c w s w (35d) ( I I(A ) 1 I A ) w = c w + c w. (35e) Note tat in te first equation we ave used c = s. Adding pre-smooting incurs a scaling of λ ν1 for (34e) and λν1 for (35e). In contrast, adding postsmooting incurs a scaling of λ ν for w and a scaling of λν for w in bot (34e) and (35e). Hence (33) olds. (33) can be rewritten as T G [ w w ] = [ λ ν 1+ν s λ ν1 λν s λ ν1 λ ν c λ ν1+ν c ] [ ] [ w c = w c 3 c 4 ] [ w w ]. (36) For n, altoug λ ν1+ν Figure 1 for four examples. 1, s n, ence c 1 1. Also, λ ν1 1, ence c, c 3, c 4 1. See 6
1.9 c 1.9 c.8.8.7.7.6.6.5.5.4.4.3.3...1.1 1 3 4 5 6 7 (a) ν 1 =, ν = 1 3 4 5 6 7 (b) ν 1 =, ν =.35.3.5 c.1.1 c..8.15.1.6.5.4.5..1.15 1 3 4 5 6 7 (c) ν 1 = 1, ν = 1 1 3 4 5 6 7 (d) ν 1 =, ν =.1.1 c.7.6 c.5.8.4.6.3.4...1 1 3 4 5 6 7 (e) ν 1 =, ν = 1 3 4 5 6 7 (f) ν 1 = 4, ν = Figure 1: Te damping coefficients of two-grid correction wit weigted Jacobi for n = 64. Te x-axis is. 7
5. Te algebraic picture Lemma 3. Te full-weigting operator and te linear-interpolation operator satisfy te variational properties I = c(i ) T, c R. (37a) (37b) is also called te Galerin condition. I A I = A. (37b) Proposition 4. A basis for te range of te interpolation operator R(I ) is given by its columns, ence dim R(I ) = n 1. N (I ) = {}. Proof. R(I ) = {I v : v Ω }. Te maximum dimension of R(I ) is tus n 1. Any v can be expressed as v = vj e j. It is obvious tat te columns of I are linearly-independent. Corollary 5. For te full-weigting operator, dim R(I ) = n 1, dim N (I ) = n. (38) Proof. Te fundamental teorem of linear algebra states tat for a matrix A : R m R n, R m = R(A) N (A T ), (39) R n = R(A T ) N (A), (4) were R(A) N (A T ) and R(A T ) N (A). If A as ran r, from te singular value decomposition A = UΣV T, we ave R(A) = span{u 1, U,..., U r }, (41) N (A) = span{v n r+1, V n r+,..., V n }, (4) R(A T ) = span{v 1, V,..., V r }, (43) N (A T ) = span{u m r+1, U m r+,..., U m }. (44) See Figure 5.7 on page 85. Te rest of te proof follows from (37). Proposition 6. A basis for te null space of te full-weigting operator is given by were e j is te jt unit vector on Ω. N (I ) = span{a e j : j is odd}, (45) Proof. Consider I A. Te jt row of I as (j 1) leading zeros and te next tree nonzero entries are 1/4, 1/, 1/4. Since A as bandwidt of 3, it suffices to only consider five columns of A for potentially non-zero dot-product i (I ) ji(a ) i. For j ± 1, tese dot products are zero; for j, te dot product is 1/; for j ±, te dot product is 1/4; Hence for any odd j, we ave I A e j =. Te above proposition states tat te basis vector of N (I ) are of te form (,,..., 1,, 1,...,, ) T. Hence N (I ) consists of bot smoot and oscillatory modes. Teorem 7. Te null space of te two-grid correction operator is te range of interpolation: N (T G) = R(I ). (46) 8
Proof. If s R(I ), ten s = I q. T Gs = [ I I (A ) 1 I A ] I q =, were te last step comes from (37b). By Proposition 6, t N (I A ) implies implies tat t = j is odd t je j. Consequently, T Gt = [ I I (A ) 1 I A ] t = t, i.e., TG is te identity operator wen acting on N (I ). Tis implies tat dimn (T G) dimr(i ), wic completes te proof. Now tat we ave bot te spectral decomposition Ω = L H and te subspace decomposition Ω = R(I ) N (I ), te combination of relaxation wit TG correction is equivalent to projecting te initial error vector to te L axis and ten to te N axis. Repeating tis process reduces te error vector to te origin; see Figure 5.8-Figure 5.11 for an illustration. 6 Multigrid cycles Definition 8. Te V-cycle sceme is an algoritm wit te following steps. 1) relax ν 1 times on A u = f wit a given initial guess v, ) if Ω is te coarsest grid, go to step 4), oterwise v V (v, f, ν 1, ν ) (47) f I (f Av ), v, v V (v, f ). 3) interpolate error bac and correct te solution: v v + I v. 4) relax ν times on A u = f wit te initial guess v. Definition 9. Te Full Multigrid V-cycle is an algoritm wit te following steps. 1) If Ω is te coarsest grid, set v and go to step 3), oterwise ) correct v I v, v F MG (f, ν 1, ν ) (48) f I f, v F MG (f, ν 1, ν ). 3) perform a V-cycle wit initial guess v : v V (v, f, ν 1, ν ). See Figure 3.6 for te above two metods. Note tat in Figure 3.6(c) te initial descending to te coarsest grid is missing. 9