Iterative methods for positive definite linear systems with a complex shift

Transcription

1 Iterative methods for positive definite linear systems with a complex shift William McLean, University of New South Wales Vidar Thomée, Chalmers University November 4, 2011

2 Outline 1. Numerical solution of the heat equation 2. Richardson iteration 3. Conjugate gradients 4. Preconditioning

3 Part I Numerical solution of the heat equation

4 Initial-boundary value problem Spatial domain Ω R d. Seek u = u(x, t) such that t u (a u) = f (x, t) for x Ω and t > 0, u(x, 0) = u 0 (x) for x Ω, u(x, t) = 0 for x Ω and t > 0. L 2 inner product (u, v) = Ω u v dx. Weak formulation: seek u : [0, T ] H0 1 (Ω) such that (u t, v) + (a u, v) = ( f (t), v ) for all v H 1 0 (Ω), with u(0) = u 0.

5 Laplace transformation Write Since w(z) û(z) = L{u(t)} = 0 e zt u(t) dt, Rz > 0. L{( t u, v)} = ( zû(z) u(0), v ) = z ( w(z), v ) (u 0, v), define g(z) = u 0 + ˆf (z) then w(z) H0 1 (Ω) satisfies z ( w(z), v) + ( a w(z), v ) = ( g(z), v ) for all v H0 1 (Ω). Spectrum σ(a) on positive real axis, so if 0 < δ < π then w(z) C δ g(z) 1 + z for arg z > π δ.

6 The contour Γ

7 Laplace inversion formula For simplicity, choose Γ = { x + iy : (x 1) 2 y 2 = 1 and x 0 }. Assume ˆf (z) bounded and analytic, on and to the right of Γ, then u(t) = 1 e zt w(z) dz for t > 0, w = û. 2πi Parametric representation gives u(t) = Γ z(ξ) = 1 cosh ξ + i sinh ξ for < ξ <, v(ξ, t) dξ where v(ξ, t) = ez(ξ)t 2πi w ( z(ξ) ) z (ξ).

8 Quadrature Step size k = log q/q > 0. For j q, ξ j = jk, z j = z(ξ j ), z j = z (ξ j ). Equal-weight quadrature approximation u(t) = v(ξ, t) dξ U q (t) = k q v(ξ j, t), j= q that is, U q (t) = k 2πi q e z j t w(z j )z j. j= q Points furthest from 0: z ±q = (q + q 1 ) ± 1 2 i(q q 1 ) 1 2 ( 1 ± i)q.

9 Spatial discretization Weak solution u(t) H0 1 (Ω) satisfies (u t, v) + (a u, v) = ( f (t), v ) for all v H 1 0 (Ω), with u(0) = u 0. Conforming, piecewise-linear finite element space V h H 1 0 (Ω). Spatially discrete solution u h (t) V h satisfies (u h,t, χ) + (a u h, χ) = ( f (t), χ ) for all χ V h, subject to u h (0) = L 2 -projection of u 0 onto V h, so ( uh (0), χ ) = (u 0, χ) for all χ V h.

10 Fully-discrete solution Recall z ( w(z), v) + ( a w(z), v ) = ( g(z), v ) for all v H 1 0 (Ω). Similarly, w h (z) = û h (z) V h satisfies z ( w h (z), χ) + ( a w h (z), χ ) = ( g(z), χ ) for all χ V h. Put U q,h (t) = k 2πi q e z j t w h (z j )z j. j= q

11 Linear system Interior (free) nodes: P 1, P 2,..., P N. Nodal basis functions: Φ 1, Φ 2,..., Φ N satisfying Φ n (P m ) = δ mn. Then w h (x, z j ) = N w n (z j )Φ n (x) with w n (z j ) = w h (P n, z j ). n=1 Mass matrix, stiffness matrix, load vector: M mn = (Φ n, Φ m ), S mn = (a Φ n, Φ m ), g m = ( g(z j ), Φ m ). Need to solve (z j M + S)w = g or (z j I + M 1 S)w = M 1 g, where w = [w n ] C N and g = [g n ] C N.

12 Accuracy required for an iterative solver Iterative solver computes w h (z j ) w h (z j ), leading to The error is Ũ q,h (t) = k 2πi q e z j t w h (z j )z j. j= q Ũ q,h (t) u(t) = [ Ũ q,h (t) U q,h (t) ] + [ U q,h (t) u h (t) ] + [ u h (t) u(t) ] = k q e z j t [ w h (z j ) w h (z j ) ] z j 2πi j= q + O(e cq/ log q ) + O(h 2 ), so we desire w h (z j ) w h (z j ) ɛ j with k 2π q e x j t ɛ j z j e cq/ log q h 2. j= q

13 Part II Richardson iteration

14 The setting Finite dimensional complex vector space V with inner product (, ). Hermitian positive-definite operator A : V V, Equation with complex shift, (Av, w) = (v, Aw). A z w = b where A z = zi + A and arg z < π δ. In our example, V = C N with A = M 1 S, b = M 1 g, and so N (v, w) = Mv, w, v, w = v l w l, l=1 (Av, w) = Sv, w = v, Sw = (v, Aw).

15 The basic iteration Sheen, Sloan and Thomée (2003). For any α C, exact solution satisfies Initial guess w 0. Richardson iteration w = (I αa z )w + αb, w n+1 = (I αa z )w n + αb for n = 0, 1, 2,.... Optimal choice of the acceleration parameter α? Error e n = w n w satisfies e n+1 = (I αa z )e n, so choose α = α to minimise the error reduction ratio I αa z V V = max 1 α(z + λ). λ σ(a)

16 Slow convergence Order the eigenvalues of A as 0 < λ 1 < λ 2 < < λ N. Classical case z = 0: ɛ 0 = I α A 0 = λ N λ 1 λ N + λ 1 if α = For general z with arg z < π, can compute and find α = α (z, λ 1, λ N ) 2 λ 1 + λ N. ɛ z = I α A z 1 cλ 1 N, c = c(z, λ 1). In our case, λ N (M 1 S) = O(h 2 ) so ɛ z = I α (zi + M 1 S) 1 ch 2.

17 Preconditioning Hermitian positive-definite operator B z : V V ; mimics A 1 z. Preconditioned equation B z A z w = B z b. Preconditioned Richardson iteration w n+1 = (I αb z A z )w n + αb z b. Again seek optimal acceleration parameter α to minimize ɛ z I αb z A z.

18 Implementation without preconditioning Recall or equivalently, (zm + S)w = g, or equivalently, (zi + A)w = M 1 g, A = M 1 S, A z w = g, A z = MA z = M(zI + A) = zm + S. Residual r n = g z A z w n. Richardson iteration: w n+1 = (I αm 1 A z )w n + αm 1 g z = w n + αm 1 r n,

19 Implementation with preconditioning Let B z = B z M, with B z positive-definite and Hermitian with respect to the standard inner product, on C N, so B z is Hermitian with respect to (, ): Residual r n = g A z w n. (B z v, w) = B z Mv, Mw = (v, B z w), Preconditioned Richardson iteration w n+1 = (I αb z A z )w n + αb z M 1 g = (I αb z MA z )w n + αb z MM 1 g = (I αb z A z )w n + αb z g = w n + αb z r n.

20 Model problem finite element mesh Triangulation of Ω has N = 2663 free nodes, with maximum element diameter h =

21 Model problem other details Constant diffusivity a = 1/15 chosen to normalize the minimum eigenvalue of A = M 1 S: λ 1 (A) = and λ N (A) = Errors U h,q (t) u(t) h using a direct solver: t q = 10 q = 20 q = 30 u(t) h e e e e e e e e e e e e

22 Effect of special preconditioner B z = (µ z I + A) 1 Using optimal acceleration parameter α = ρe iϕ. Without B z With B z j x j y j ρ ϕ ɛ z ρ ϕ µ z ɛ z e e e e e e e e e e e e

23 Related literature Freund (1990): Krylov methods for (zi + A)w = b with A Hermitian and possibly indefinite. Motivated by the Helmholtz equation ω 2 u + iσu 2 u = f. Benzi and Bertaccini (2003), Bellavia et al. (2011): preconditioners in the special case of real z = x > 0, with efficient updates for different x. Motivated by implicit finite difference timestepping for parabolic PDEs. Simoncini (2003): full orthogonalization method for z = x R and xi + A nonsingular. Meerbergen (2003), Meerbergen and Bai (2010): Krylov methods for S αm, α [α min, α max ].

24 Part III Conjugate gradients

25 The setting Finite dimensional complex Hilbert space V with inner product (v, w). Positive-definite Hermitian linear operator A : V V. Seek w V satisfying where A z w = b, A z = zi + A, z = x + iy, arg z < π.

26 Krylov subspaces Initial guess w 0. Initial residual r 0 = b A z w 0 0. Krylov subspace V n = span{r 0, A z r 0,..., A n 1 z r 0 } for n 1. Notice but r 0 still depends on z. V n = span{r 0, Ar 0,..., A n 1 r 0 },

27 Galerkin method Exact solution satisfies (A z w, ϕ) = (b, ϕ) for all ϕ V. Conjugate gradient (CG) solution w n = w 0 + v n, v n V n, determined by (A z w n, ϕ) = (b, ϕ) for all ϕ V n. Thus, or ( Az (w n w 0 ), ϕ ) = (b A z w 0, ϕ), v n V n and (A z v n, ϕ) = (r 0, ϕ) for all ϕ V n.

28 Uniqueness and existence If (A z v n, ϕ) = 0 for all ϕ V n, then taking ϕ = v n gives (A z v n, v n ) = z v n 2 + (Av n, v n ) = 0, or in other words, x v n 2 + (Av n, v n ) = 0 and y v n 2 = 0, so v n = 0, by our assumption arg z < π. Hence, v n is unique. Hence, v n exists (because V is finite dimensional).

29 Key tools Put v 2 = z v 2 + (Av, v) for v V. Sesquilinear form (A z v, w) not an inner product if Rz 0, but... Lemma If θ z = arg z ( π, π) then, for all v, ϕ V, (A z v, ϕ) v ϕ and (A z v, v) cos( 1 2 θ z) v 2 Lemma (Galerkin orthogonality) The CG error e n = w n w satisfies (A z e n, ϕ) = 0 for all ϕ V n.

30 CG solution is quasi-optimal Theorem The CG solution w n satisfies w n w sec( 1 2 θ z) inf v w. v w 0 +V n Proof. Error e n = w n w satisfies cos( 1 2 θ z) e n 2 ( A z e n, w n w ) = ( A z e n, (w n v) + (v w) ) = (A z e n, w n v) + (A }{{} z e n, v w) V n = 0 + (A z e n, v w) e n v w

31 Error reduction ratio Theorem The CG error satisfies where e n 2 sec( 1 2 θ z) η n z + η n z e 0 λn + z λ 1 + z η z = λn + z + λ 1 + z and η z 1 c(z, λ 1 )λ 1/2 N. Thus, for A = M 1 S, e n 2 sec( 1 2 θ z) η z n e 0 with η z 1 ch.

32 Model problem (z = z 15 )

33 Orthogonal basis of residuals Residual r n = b A z w n. There exists N N = dim V such that r n 0 for 0 n < N but r n = 0 for n N. Furthermore r n V n+1 and (r n, ϕ) = 0 for all ϕ V n and n 0. so r 0, r 1,..., r n 1 form an orthogonal basis for V n if n N. Thus, V 0 V 1 V N = V N +1 =.

34 Conjugate search directions Put p 0 = r 0 and, for 0 n < N, p n+1 = r n+1 + In this way, n k=0 β nk p k where β nk = (A zr n+1, p k ) (A z p k, p k ). p n V n+1 and (A z p n, p j ) = 0 for 0 j < n. If n N then p 0, p 1,..., p n 1 form a basis for V n. Find β nk = 0 for 0 k < n so If Iz 0 and 0 j < n, then p n+1 = r n+1 + β n p n, β n = β nn. (A z p j, p n ) = (A z p j, p n ) (p j, A z p n ) = (z z)(p j, p n ) 0.

35 Abstract CG algorithm p 0 = r 0 = b A z w 0 for n = 0, 1, 2,... do if converged then break end if α n = r n 2 /(A z p n, p n ) w n+1 = w n + α n p n r n+1 = r n α n A z p n β n = (r n+1, A z p n )/(A z p n, p n ) p n+1 = r n+1 + β n p n end for In the classical CG algorithm, α n is real and α n (r n+1, A z p n ) = (r n+1, r n α n A z p n ) = r n+1 2 so β n = r n+1 2 / r n 2.

36 Concrete CG algorithm Recall A z = MA z = zm + S, A z w = g, (v, w) = Mv, w. p 0 = r 0 = M 1 (g A z w 0 ) for n = 0, 1, 2,... do if converged then break end if α n = Mr n, r n / A z p n, p n w n+1 = w n + α n p n r n+1 = r n α n M 1 A z p n β n = r n+1, A z p n / A z p n, p n p n+1 = r n+1 + β n p n end for Can use lumped mass approximation M D, with D diagonal.

37 Part IV Preconditioning

38 Special preconditioner Consider B z = (µ z I + A) 1, µ z > λ 1 (A). Since A z = zi + A = (z µ z )I + (µ z I + A) the equation (zi + A)w = b is equivalent to one of the same form: ( zi + B z )w = zb z b, z = (z µ z ) 1.

39 Error reduction ratio Norm Error e n = w n w satisfies v 2 = z v 2 + (B z v, v). e n 2 sec( 1 2 θ z) η n z + η n z e 0 2 sec( 1 2 θ z) η z n e 0, with η z c(z, λ 1, µ z ) < 1 uniformly as λ N. Optimal choice for preconditioner parameter µ z = λ 1 + q z 1 q z (λ N λ 1 ) > λ 1, q z = z + λ 1 λ 1 as λ N. z + λ 1 z + λ N,

40 Results for model problem j x j y j η z η z µ z η z µ z

41 General preconditioning Arbitrary Hermitian positive-definite B z. Preconditioned equation B z A z w = B z b. Both B z and B z A are Hermitian with respect to [v, w] = (Bz 1 v, w), but B z A z = zb z + B z A is not of the form zi + Hermitian. Preconditioned residual Krylov subspaces r n = B z b B z A z w n = B z (b A z w n ) = B z r n. Ṽ n = span{ r 0, B z A z r 0,..., (B z A z ) n 1 r 0 }.

42 Quasi-optimality Preconditioned CG iterates w n = w 0 + v n, with v n Ṽ n, defined by (A z w n, ϕ) = (b, ϕ) for all ϕ Ṽ n. Equivalently [B z A z w n, ϕ] = [B z b, ϕ] for all ϕ Ṽ n. Galerkin orthogonality: (A z e n, ϕ) = [B z A z e n, ϕ] = 0 for all ϕ Ṽ n. Theorem w n w sec( 1 2 θ z) inf v w. v w 0 +Ṽn But no error bound.

43 Recursion involves all previous iterates Preconditioned residual r n = B z (b A z w n ) = B z r n satisfies r n Ṽ n+1 and [ r n, ϕ] = (r n, ϕ) = 0 for all ϕ Ṽ n. Set n p 0 = r 0 and p n+1 = r n+1 + β nk p k for n 0, k=0 where the β nk satisfy the (n + 1) (n + 1) lower-triangular system j (A z p k, p j )β nk = (A z r n, p j ) for 0 j n, k=0 so that 0 p n Ṽ n+1 and (A z p n, p j ) = 0 for 0 k < n < N.

44 Abstract preconditioned CG algorithm r 0 = b A z w 0 p 0 = r 0 = B z r 0 for n = 0, 1, 2,... do if converged then break end if α n = (r n, r n )/(A z p n, p n ) w n+1 = w n + α n p n r n+1 = r n α n A z p n r n+1 = B z r n Solve j k=0 (A zp k, p j )β nk = (A z r n, p j ), 0 j n p n+1 = r n+1 + n k=0 β nkp k end for In practice, restart the iteration after every m steps.

45 Concrete preconditioned CG algorithm Mr 0 = g A z w 0 p 0 = r 0 = B z Mr 0 for n = 0, 1, 2,... do if converged then break end if α n = Mr n, r n / A z p n, p n w n+1 = w n + α n p n Mr n+1 = Mr n α n A z p n r n+1 = B z Mr n Solve j k=0 A zp k, p j β nk = A z r n, p j, 0 j n p n+1 = r n+1 + n k=0 β nkp k end for No need to compute the action of M 1.

46 Iterations for model problem Stopping criterion w h (z j ) w h (z j ) ɛ j. j INV IC AMG(1) w j ɛ j e e e e e e e e e e e e e e e e e e e e e e-03