IDR(s) A family of simple and fast algorithms for solving large nonsymmetric systems of linear equations

Transcription

1 IDR(s) A family of simple and fast algorithms for solving large nonsymmetric systems of linear equations Harrachov 2007 Peter Sonneveld en Martin van Gijzen August 24, Delft University of Technology

2 Outline Introduction The Induced Dimension Reduction Theorem The IDR(s) algorithm Numerical experiments Conclusions August 24,

3 Product Bi-CG methods Bi-CG solves nonsymmetric linear systems using (short) CG recursions but needs extra matvec with A H. Idea of Sonneveld: use wasted matvec in a more useful way. Result: transpose-free methods: CGS (Sonneveld, 1989) Bi-CGSTAB (Van der Vorst, 1992) BiCGSTAB2 (Gutknecht, 1993) TFQMR (Freund, 1993) BiCGstab(l) (Sleijpen and Fokkema, 1994) Many other variants August 24,

4 Historical remarks Sonneveld first developed IDR (1980). Analysis showed that IDR was Bi-CG combined with linear minimal residual steps. The fact that IDR is transpose free, combined with the relation with Bi-CG led to the development of a now famous algorithm: CGS. Later Van der Vorst proposed another famous method: Bi-CGSTAB, which is mathematicaly equivalent with IDR. As a result of these developments, the basic IDR idea was abandoned for the Bi-CG approach. IDR is now forgotten. August 24,

5 The IDR idea The IDR-idea is to generate a sequence of subspaces G 0 G j with the following operations: - Intersect G j 1 with fixed subspace S, - Compute G j = (I ω j A)(G j 1 S). The subspaces G 0 G j are nested and of shrinking dimension. August 24,

6 The IDR theorem Theorem 1 (IDR) Let A be any matrix in C N N, let v 0 be any nonzero vector in C N, and let G 0 be the complete Krylov space K N (A,v 0 ). Let S denote any (proper) subspace of C N such that S and G 0 do not share a nontrivial invariant subspace of A, and define the sequence G j, j = 1,2,... as G j = (I ω j A)(G j 1 S) where the ω j s are nonzero scalars. Then (i) G j G j 1 for all j > 0. (ii) G j = {0} for some j N. August 24,

7 Making an IDR algorithm The IDR theorem can be used to construct solution algorithms. This is done by constructing residuals r n G j. According to the IDR theorem ultimately r n G M = {0}. In order to generate residuals and corresponding solution approximations we first look at the basic recursions. August 24,

8 Krylov methods: basic recursion (1) A Krylov-type solver produces iterates x n, for which the residuals r n = b Ax n are in the Krylov spaces K n (A,r 0 ) = r 0 Ar 0 A 2 r 0 A n r 0, The next residual r n+1 can be generated by r n+1 = αar n + bj j=0 β j r n j. The parameters α, β j determine the specific method and must be such that x n+1 can be computed. August 24,

9 Krylov methods: basic recursion (2) By using the difference vector r k = r k+1 r k = A(x n+1 x n ), an explicit way to satisfy this requirement is r n+1 = r n αar n bj j=1 γ j r n j, which leads to the following update for the x estimate: x n+1 = x n + αr n bj j=1 γ j x n j, August 24,

10 Computation of a new residual (1) Residuals are computed that are forced to be in the subspaces G j, by application of the IDR-theorem. The residual r n+1 is in G j+1 if r n+1 = (I ω j+1 A)v with v G j S. The main problem is to find v. August 24,

11 Computation of a new residual (2) August 24,

12 Computation of a new residual (3) The vector v is a combination of the residuals r l in G j. v = r n bj j=1 γ j r n j. Let the space S be the left null space of some N s matrix P: P = (p 1 p 2... p s ), S = N(P H ). Since v is also in S = N(P H ), it must satisfy P H v = 0. Combining these two yields an s ĵ linear system for the coefficients γ j that (normally) is uniquely solvable if ĵ = s. August 24,

13 Computation of a new residual (4) Hence with the residual r n, and a matrix R consisting of the last s residual differences: R = ( r n 1 r n 2... r n s ) a suitable v can be found by Solve s s system Calculate (P H R)c = P H r n v = r n Rc August 24,

14 Building G j+1 (1) Assume r n and all columns of R are in G j, and let r n+1 be calculated as r n+1 = v ω j+1 Av Then r n+1 G j+1. Since G j+1 G j (theorem) we automatically have r n+1 G j Now the next R is made by repeating the calculations. In this way we find s + 1 residuals in G j+1 August 24,

15 Building G j+1 (2) August 24,

16 Building G j+1 (3) August 24,

17 Building G j+1 (4) August 24,

18 A few details 1. The first s + 1 residuals, starting with r 0 can be constructed by any Krylov-based iteration, such as a local minimum residual method. 2. In our actual implementation, all steps are identical. However, in calculating the first residual in G j+1, a new value ω j+1 may be chosen. We choose ω j+1 such that v ω j+1 Av is minimal. August 24,

19 Basic IDR(s) algorithm. while r n > TOL or n < MAXIT do for k = 0 to s do Solve c from P H dr n c = P H r n v = r n dr n c; t = Av; if k = 0 then end if end for end while ω = (t H v)/(t H t); dr n = dr n c ωt; dx n = dx n c + ωv; r n+1 = r n + dr n ; x n+1 = x n + dx n ; n = n + 1; dr n = (dr n 1 dr n s ); dx n = (dx n 1 dx n s ); August 24,

20 Vector operations per MATVEC Method DOT AXPY Memory Requirements IDR(1) IDR(2) IDR(4) IDR(6) Full GMRES n+1 2 n+1 2 n + 2 BiCGSTAB August 24,

21 Relation with other methods Although the approach is different, IDR(s) is closely related to some Bi-CGSTAB methods: IDR(1) and Bi-CGSTAB yield the same residuals at the even steps. ML(k)BiCGSTAB (Yeung and Chen, 1999) seems closely related to IDR(s), BUT IDR(s) is MUCH simpler (both conceptually and its implementation) Other, more natural extensions are possible, e.g. to avoid breakdown. August 24,

22 Performance of IDR(s) The IDR theorem states that - it is possible to generate a sequence of nested subspace G j of shrinking dimension, - but does not say how fast the dimension shrinks It can be proven that the dimension reduction is (normally) s, So dim(g j+1 ) = dim(g j ) s. IDR(s) requires at at most N + N s matrix-vector multiplications to compute the exact solution. August 24,

23 Numerical experiments We will present two typical numerical examples A 2D Ocean Circulation Problem A 3D Helmholtz Problem August 24,

24 A 2D Ocean Circulation Problem We compare IDR(s) with Full GMRES, restarted GMRES and Bi-CGSTAB. This ocean example is representative for a wide class of CFD problems. We will compare: Rate of convergence Stagnation level (of the true residual norm) August 24,

25 Stommel s model for ocean circulation Balance between bottom friction, wind stress and Coriolis force. r ψ β ψ x = ( F) z plus circulation condition around islands k r ψ Γ k n ds = F s ds. Γ k ψ: streamfunction r: bottom friction parameter β: Coriolis parameter F: Wind stress August 24,

26 Discretization of the ocean problem Discretization with linear finite elements Results in nonsymmetric system of Eigenvalues are (almost) real Solution parameters: ILU(0) preconditioning P: s 1 random vectors plus r 0 (for comparison with Bi-CGSTAB) August 24,

27 Solution of the ocean problem August 24,

28 Convergence for the ocean problem Convergence 2D ocean problem IDR1 IDR2 IDR4 IDR6 BICGSTAB GMRES Scaled residual norm Number of MATVECS August 24,

29 Some observations Required number of MATVECS decreases if s is increased. IDR(4) and IDR(6) are close to the optimal convergence curve of full GMRES. Convergence curves of IDR(1) and Bi-CGSTAB coincide. Stagnation levels of IDR(s) comparable with Bi-CGSTAB. August 24,

30 Required number of MATVECS Method Number of MATVECS Vectors Full GMRES GMRES(20) > GMRES(50) Bi-CGSTAB IDR(1) IDR(2) IDR(4) IDR(6) Tolerance: b Ax n < 10 8 b August 24,

31 A 3D Helmholtz Problem Example models sound propagation in a room of 4 4 4m 3. A harmonic sound source gives acoustic pressure field p(x, t) = p(x)e 2πift. The pressure function p can be determined from in which (2πf) 2 c 2 p p = δ(x x s ) in Ω. - c: the sound speed (340 m/s) - δ(x x s ): the harmonic point source, in the center of the room. August 24,

32 Boundary conditions Five of the walls are reflecting, modeled by p n = 0, and the remaining wall is sound absorbing, p n = 2πif c p on Γ 3. August 24,

33 Discretization Discretization with FEM yields linear system [ (2πf) 2 M + 2πifC + K]p = b Frequency f = 100Hz. System matrix complex, symmetric (but not Hermitian) and indefinite: difficult for iterative methods gridsize h = 8 cm: equations Solution parameters: ILU(0) preconditioning P: Initial residual plus s 1 real random vectors Only comparison with BiCGstab(l) August 24,

34 Results Helmholtz Problem Method Number of Elapsed time MATVECS [s] IDR(1) IDR(2) IDR(4) IDR(6) BiCGstab(1) BiCGstab(2) BiCGstab(4) BiCGstab(8) Tolerance: b Ax n < 10 8 b August 24,

35 Conclusions The IDR-theorem offers a new approach for the development of iterative solution algorithms The IDR(s) algorithm presented here is quite promising and seems to outperform state-of-the-art Bi-CG-type methods for important classes of problems. More information: - Report: IDR(s): a family of simple and fast algorithms for solving large nonsymmetric linear systems, submitted - Matlab code, (includes preconditioning and deflation) August 24,