On solving linear systems arising from Shishkin mesh discretizations

On solving linear systems arising from Shishkin mesh discretizations Petr Tichý Faculty of Mathematics and Physics, Charles University joint work with Carlos Echeverría, Jörg Liesen, and Daniel Szyld October 20, 2016, Prague Seminar of Numerical Mathematics, KNM MFF UK 1

Problem formulation Convection-diffusion boundary value problem ɛ u + α u + β u = f in Ω = (0, 1), u(0) = u 0, u(1) = u 1, α > 0, β 0 constants, the problem is convection dominated 0 < ɛ α. [Stynes, 2005] (Acta Numerica) [Roos, Stynes, and Tobiska, 1996, 2008] (book) [Miller, O Riordan, and Shishkin, 1996] (book) 2

Solution and boundary layers ɛ = 0.01, α = 1, β = 0, u(0) = u(1) = 0. There are small subregions where the solution has a large gradient. 3

Numerical solution, equidistant mesh Standard techniques: u (ih) u i+1 u i 1 2h, u (ih) u i u i 1. h Unnatural oscillations or cannot resolve the layers. Remedy: stabilization or non-equidistant mesh. We study discretizations for a Shishkin mesh. 4

Outline 1 Shishkin mesh and discretization 2 How to solve the linear system? 3 Multiplicative Schwarz method 4 Convergence analysis 5 Schwarz method as a preconditioner 6 Numerical examples 5

Shishkin mesh on [0, 1] Piecewise equidistant N even, define the transition point 1 τ and n by { 1 τ min 2, ɛ } α 2 ln N, n N 2. If ɛ α, then 1 τ is close to 1. Next define H and h by H 1 τ n, h τ n and consider the Shishkin mesh, x 0 = 0, x i ih, x n+i x n + ih, i = 1,..., n. 6

Discretization on the Shishkin mesh - details For simplicity u(0) = u(1) = 0 The upwind difference scheme is given by ɛ δ 2 xu i + α D x u i + βu i = f i, u 0 = u N = 0, and the central difference scheme by where ɛ δ 2 xu i + αd 0 x + βu i = f i, u 0 = u N = 0, δ 2 xu i = 2u i 1 (H + h)h 2u i Hh + 2u i+1 (H + h)h, i = n, and Dx u i = u i u i 1, 1 i n, D 0 H xu i = u i+1 u i 1 h + H, i = n. surveys [Linss, Stynes, 2001], [Stynes, 2005], [Kopteva, O Riordan, 2010] 7

Shishkin mesh and ɛ-uniform convergence The upwind difference scheme ɛ δ 2 xu i + α D x u i + βu i = f i, u 0 = u N = 0. There exists a constant C such that ( ) ln N u(x i ) u i C, i = 0,..., N, N see, e.g., [Stynes, 2005]. The central difference scheme ɛ δ 2 xu i + α D 0 xu i + βu i = f i, u 0 = u N = 0. There exists a constant C such that ( ) ln N 2 u(x i ) u i C, i = 0,..., N, N [Andreev and Kopteva, 1996], a difficult proof, the scheme does not satisfy a discrete maximum principle. [Kopteva and Linss, 2001]. 8

Outline 1 Shishkin mesh and discretization 2 How to solve the linear system? 3 Multiplicative Schwarz method 4 Convergence analysis 5 Schwarz method as a preconditioner 6 Numerical examples 9

Structure of the matrix A = a H b H c H............ bh c H a H b H c a b c h a h b h c h............ bh c h a h 10

Entries The upwind scheme c H = ɛ H 2 α H, 2ɛ c = H(H + h) α H, c h = ɛ h 2 α h, a H = 2ɛ H 2 + α H + β, b H = ɛ H 2, a = 2ɛ hh + α H + β, b = 2ɛ h(h + h), a h = 2ɛ h 2 + α h + β, b h = ɛ h 2. The central difference scheme c H = ɛ H 2 α 2H, 2ɛ c = H(H + h) α h + H, c h = ɛ h 2 α 2h, b H = ɛ H 2 + α 2H, a = 2ɛ hh + β, b = 2ɛ h(h + h) + a H = 2ɛ H 2 + β, a h = 2ɛ h 2 + β, b h = ɛ h 2 + α 2h. α h + H, 11

Matrix properties nonsymmetric A is M-matrix for the upwind scheme A is not an M-matrix for the central difference scheme A is highly nonnormal. Consider ɛu + u = 1, u(0) = 0, u(1) = 0, ɛ = 10 8 and N = 46, and the spectral decomposition A = Y DY 1. upwind upwind sc. central central sc. κ(a) 4.05 10 10 2.96 10 3 6.23 10 10 2.95 10 3 κ(y ) 1.51 10 17 1.23 10 19 4.11 10 3 1.87 10 2 12

Solving linear system using GMRES 10 0 Convergence of GMRES for the upwind scheme 10 1 10 2 relative residuial norm 10 3 10 4 10 5 10 6 10 7 10 8 GMRES, A original GMRES, A scaled 5 10 15 20 25 30 35 40 45 number of iterations 13

Outline 1 Shishkin mesh and discretization 2 How to solve the linear system? 3 Multiplicative Schwarz method 4 Convergence analysis 5 Schwarz method as a preconditioner 6 Numerical examples 14

Multiplicative Schwarz method Idea of solving Ax = b Given an approximation x (k), then x = x (k) + y and y satisfies Ay = b Ax (k). 15

Multiplicative Schwarz method Idea of solving Ax = b Given an approximation x (k), then x = x (k) + y and y satisfies Ay = b Ax (k). [ Restriction operators R 1 = I n 0 ] [, R 2 = 0 I n ]. 15

Multiplicative Schwarz method Idea of solving Ax = b Given an approximation x (k), then x = x (k) + y and y satisfies Ay = b Ax (k). [ Restriction operators R 1 = Solve on the first domain I n 0 ] [, R 2 = 0 I n ]. (R 1 AR T 1 ) ỹ = R 1 (b Ax (k) ) and approximate y by prolongation of ỹ, i.e., by R1 T ỹ. Define x (k+ 1 2 ) = x (k) + R T 1 (R 1 AR T 1 ) 1 R 1 (b Ax (k) ). 15

Multiplicative Schwarz method Formalism Define P i = R T i A 1 i R i A, A i R i AR T i, i = 1, 2. The multiplicative Schwarz is the iterative scheme x (k+1) = T x (k) + v, T = (I P 2 )(I P 1 ), where v is defined such that the scheme is consistent. 16

Multiplicative Schwarz method Experiment 10 0 upwind difference scheme 10 2 Relative Residual 10 4 10 6 10 8 GMRES Multiplicative Schwarz 0 5 10 15 20 25 30 35 40 45 Iterations 17

Outline 1 Shishkin mesh and discretization 2 How to solve the linear system? 3 Multiplicative Schwarz method 4 Convergence analysis 5 Schwarz method as a preconditioner 6 Numerical examples 18

Convergence analysis Exploiting the structure x x (k+1) x x 0 T k+1. Using T = (I P 2 )(I P 1 ) we are able to show that t 1. T = 0... 0 t n+1 0... 0 = t e T n+1.. t N 1 19

Convergence analysis Exploiting the structure x x (k+1) x x 0 T k+1. Using T = (I P 2 )(I P 1 ) we are able to show that t 1. T = 0... 0 t n+1 0... 0 = t e T n+1.. t N 1 Therefore, T 2 = t (e T n+1 t) et n+1 = t n+1 T, and T k+1 = t n+1 k T. How to bound t n+1, and T in a convenient norm ( )? 19

Convergence analysis Details A = A H b H c a b c h A h. Let m n 1, ρ t n+1... the convergence factor. Then, ( ) bb H A 1 H ρ = ( ) a cb H A 1 H m,m m,m ( ) cc h A 1 h a bc h ( 1,1 ) A 1 h 1,1. 20

Convergence analysis Bounding ( A 1 H )m,m and ( ) A 1 h 1,1 A matrix B = [b i,j ] is called a nonsingular M-matrix when B is nonsingular, b i,i > 0 for all i, b i,j 0 for all i j, and B 1 0 (elementwise). If A H and A h are nonsingular M-matrices, then using [Nabben 1999], ( { } 1 A 1 H )m,m min b H, 1, c H ( A 1 h )1,1 min { 1 } b h, 1 c h. A sufficient condition: The sign conditions & irreducibly diagonal dominant nonsingular M-matrix. [Meurant, 1996], [Hackbusch, 2010] 21

Convergence analysis The upwind scheme The matrices A H and A h are M-matrices, and we know that e (k+1) e (0) ρ k T. 22

Convergence analysis The upwind scheme The matrices A H and A h are M-matrices, and we know that e (k+1) e (0) ρ k T. Theorem (the upwind scheme) [Echeverría, Liesen, T., Szyld, 2016] For the upwind scheme we have ρ ɛ ɛ + αh ɛ ɛ + α, N and ɛ T ɛ + αh. 22

Convergence analysis The central difference scheme A h is still an M-matrix. If αh > 2ɛ, i.e. b H > 0, then A H is not an M-matrix... the most common situation from a practical point of view. 23

Convergence analysis The central difference scheme A h is still an M-matrix. If αh > 2ɛ, i.e. b H > 0, then A H is not an M-matrix... the most common situation from a practical point of view. Recall ( ) bb H A 1 H ρ = ( ) a cb H A 1 H m,m m,m ( ) cc h A 1 h a bc h ( 1,1 ) A 1 h 1,1 How to bound ( A 1 ) H m,m?... results by [Usmani 1994]. 23

Convergence analysis The central difference scheme A h is M-matrix, if αh > 2ɛ, A H is not an M-matrix. Theorem (the central diff. scheme) [Echeverría, Liesen, T., Szyld, 2016] Let m = N/2 1 be even, and let αh > 2ɛ. For the central differences we have ρ < 2mɛ ɛ + αh 2 < N ɛ ɛ + α, N T < 2. Thus, the error of the multiplicative Schwarz method satisfies e (k+1) ( 2mɛ e (0) < 2 ɛ + αh 2 ) k. 24

Remarks on diagonally scaled linear algebraic systems DAx = Db The ill-conditioning can be avoided by diagonal scaling [Roos 1996]: Ax = b is multiplied from the left by d H I m D = d. d h I m 25

Remarks on diagonally scaled linear algebraic systems DAx = Db The ill-conditioning can be avoided by diagonal scaling [Roos 1996]: Ax = b is multiplied from the left by d H I m D = d. d h I m Such a scaling preserves the Toeplitz structure and the M-matrix property of the submatrices. Analysis depends on these properties and on ratios between elements in the same row such as b/a and b H /c H. These ratios are invariant under diagonal scaling. 25

Outline 1 Shishkin mesh and discretization 2 How to solve the linear system? 3 Multiplicative Schwarz method 4 Convergence analysis 5 Schwarz method as a preconditioner 6 Numerical examples 26

Schwarz method as a preconditioner We have consistent scheme x (k+1) = T x (k) + v. Hence, x solves Ax = b and also the preconditioned system (I T )x = v. 27

Schwarz method as a preconditioner We have consistent scheme x (k+1) = T x (k) + v. Hence, x solves Ax = b and also the preconditioned system (I T )x = v. We can formally define a preconditioner M such Ax = b M 1 Ax = M 1 b (I T )x = v. Clearly M = A(I T ) 1. 27

Schwarz method as a preconditioner for GMRES The multiplicative Schwarz method as well as GMRES applied to the preconditioned system obtain their approximations from the same Krylov subspace. In terms of the residual norm, the preconditioned GMRES will always converge faster than the multiplicative Schwarz. Moreover, in this case, the iteration matrix T has rank-one structure, and dim (K k (I T, r 0 )) 2. Therefore, GMRES converges in at most 2 steps,... a motivation for more dimensional cases. 28

How to multiply by T Schwarz or preconditioned GMRES only multiply by T, T = (I P 2 )(I P 1 ), P i = R T i (R i AR T i ) 1 R i A, i.e., to solve systems of the form (m = n 1 = N 2 1) A H c b H a [ y1:m y m+1 ] = [ z1:m z m+1 ]. 29

How to multiply by T Schwarz or preconditioned GMRES only multiply by T, T = (I P 2 )(I P 1 ), P i = R T i (R i AR T i ) 1 R i A, i.e., to solve systems of the form (m = n 1 = N 2 1) A H c b H a Using the Schur complement, [ y1:m y m+1 ] = [ z1:m z m+1 ( A H b ) Hc a e me T b H m y 1:m = z 1:m z m+1 e m. a H Then apply Sherman-Morrison formula. We need only to solve systems with A H (Toeplitz)! ]. 29

Outline 1 Shishkin mesh and discretization 2 How to solve the linear system? 3 Multiplicative Schwarz method 4 Convergence analysis 5 Schwarz method as a preconditioner 6 Numerical examples 30

Numerical examples Consider i.e. ɛu + u = 1, u(0) = 0, u(1) = 0, α = 1, β = 0, f(x) 1. Choose N = 198, various values of ɛ. upwind central differences ɛ ρ up our bound ρ cd our bound 10 8 9.4 10 7 9.9 10 7 1.8 10 4 3.9 10 4 10 6 9.4 10 5 9.9 10 5 1.8 10 2 3.9 10 2 10 4 9.3 10 3 9.8 10 3 8.3 10 1 3.8 10 0 ρ up < ɛ ɛ + αh, ρ cd < 2mɛ ɛ + αh 2. 31

Numerical examples Upwind, ɛ = 10 8 10 0 Upwind T=(I P 2 )(I P 1 ) error bound T=(I P 1 )(I P 2 ) error bound Error norms and bounds 10 5 10 10 10 15 10 20 0 1 2 3 4 5 6 7 8 9 10 k 32

Numerical examples Upwind, ɛ = 10 4 10 0 Upwind T=(I P 2 )(I P 1 ) error bound T=(I P 1 )(I P 2 ) error bound Error norms and bounds 10 5 10 10 10 15 10 20 0 1 2 3 4 5 6 7 8 9 10 k 33

Numerical examples Central differences, ɛ = 10 8 10 0 Central differences T=(I P 2 )(I P 1 ) T=(I P 1 )(I P 2 ) error bound Error norms and bounds 10 5 10 10 10 15 10 20 0 1 2 3 4 5 6 7 8 9 10 11 12 k 34

Numerical examples Central differences, ɛ = 10 4 10 0 Central differences T=(I P 2 )(I P 1 ) T=(I P 1 )(I P 2 ) error bound Error norms and bounds 10 5 10 10 10 15 10 20 0 1 2 3 4 5 6 7 8 9 10 11 12 k 35

Conclusions and further work We considered finite difference discretizations of the 1D singularly-perturbed convection-diffusion equation posed on a Shishkin mesh. 36

Conclusions and further work We considered finite difference discretizations of the 1D singularly-perturbed convection-diffusion equation posed on a Shishkin mesh. The matrices that arise from the upwind and the central difference schemes are nonsymmetric and highly nonnormal. For the upwind scheme, we proved rapid convergence of the multiplicative Schwarz method in the most relevant case Nɛ < α. The convergence for the central difference scheme is slower, but still rapid, when N 2 ɛ < α and if N/2 1 is even. 36

Related papers C. Echeverría, J. Liesen, P. Tichý, and D. Szyld, [Convergence of the multiplicative Schwarz method for singularly perturbed convection-diffusion problems discretized on a Shishkin mesh, (2016), in preparation] J. Miller, E. O Riordan, and G. Shishkin, [Fitted Numerical Methods for Singular Perturbation Problems: Error Estimates in the Maximum Norm for Linear Problems in One and Two Dimensions, World Scientific, 1996.] H-G. Roos, M. Stynes, L. Tobiska, [Robust Numerical Methods for Singularly Perturbed Differential Equations, second edition, Springer Series in Computational Mathematics, Springer-Verlag, Berlin, 2008, 604 pp.] M. Stynes, [Steady-state convection-diffusion problems, Acta Numerica, 14 (2005), pp. 445 508.] Thank you for your attention! 37