Combination Preconditioning of saddle-point systems for positive definiteness

Transcription

1 Combination Preconditioning of saddle-point systems for positive definiteness Andy Wathen Oxford University, UK joint work with Jen Pestana Eindhoven, 2012 p.1/30

2 compute iterates with residuals Krylov subspace methods for Ax = b x k x 0 + K k (A,r 0 ) b Ax k = r k r 0 + AK k (A,r 0 ) since ie. r k = p(a)r 0, p Π k,p(0) = 1 K k (A,r 0 ) = span{r 0,Ar 0,...,A k 1 r 0 } with some optimality or orthogonality condition. Eindhoven, 2012 p.2/30

3 Given an inner product, : R n R n R Conjugate Gradients (Hestenes & Stiefel (1952)) computes iterates which minimize A(x x k ),x x k when A is self-adjoint and positive definite in, MINRES (Paige & Saunders (1975)) computes iterates for which r k,r k is minimal when A is self-adjoint in, GMRES (Saad & Schultz (1986)) computes iterates for which r k,r k is minimal for general A Eindhoven, 2012 p.3/30

4 Conjugate Gradient Method (CG) Choose x 0, compute r 0 = b Ax 0, set p 0 = r 0 for k = 0 until convergence do α k = r k,r k / Ap k,p k x k+1 = x k + α k p k r k+1 = r k α k Ap k <Test for convergence> β k = r k+1,r k+1 / r k,r k p k+1 = r k+1 + β k p k enddo computes iterates {x k } such that A(x x k ),x x k is minimal when Au,v = u,av and Au,u > 0 Eindhoven, 2012 p.4/30

5 Thus given any symmetric and positive definite matrix H if u,v = u,v H = u T Hv Choose x 0, compute r 0 = b Ax 0, set p 0 = r 0 for k = 0 until convergence do α k = r k,r k H / Ap k,p k H x k+1 = x k + α k p k r k+1 = r k α k Ap k <Test for convergence> β k = r k+1,r k+1 H / r k,r k H p k+1 = r k+1 + β k p k enddo computes iterates {x k } such that A(x x k ),x x k H = (x x k ) T A T H(x x k ) is minimal when Au,v H = u,av H and Au,u H > 0 Eindhoven, 2012 p.5/30

6 Similarly for the MINRES method: v 0 = 0,w 0 = 0,w 1 = 0, choose x 0 Compute r 0 = b Ax 0, set v 1 = r 0,γ 1 = v 1,v 1 H Set η = γ 1,s 0 = s 1 = 0,c 0 = c 1 = 1 for j = 1 until convergence do v j = v j /γ j δ j = v j,av j H v j+1 = Av j δ j v j γ j v j 1 γ j+1 = v j+1,v j+1 H α 0 = c j δ j c j 1 s j γ j,α 1 = α γ2 j+1 α 2 = s j δ j + c j 1 c j γ j,α 3 = s j 1 γ j c j+1 = α 0 /α 1 ; s j+1 = γ j+1 /α 1 w j+1 = (v j+1 α 3 w j 1 α 2 w j )/α 1 x j = x j 1 + c j+1 ηw j+1 η = s j+1 η, then <Test for convergence> enddo minimises r k,r k H when Ax,y H = x,ay H Eindhoven, 2012 p.6/30

7 Self-adjointness: assume, : R n R n R is a symmetric bilinear form or an inner product A R n n is self-adjoint in, iff Ax,y = x,ay Self-adjointness of A in, H thus means for all x,y ( x,y H = x T Hy) x T A T Hy = Ax,y H = x,ay H = x T HAy for all x,y A T H = HA is the relation for self-adjointness of A in, H Eindhoven, 2012 p.7/30

8 Preconditioning For example left preconditioning: Âx = P 1 Ax = P 1 b = b induces a self-adjoint matrix in, H iff A T P T H = HP 1 A Eindhoven, 2012 p.8/30

9 An important example The Bramble-Pasciak CG for saddle point problems: A B T A0 0 A = with preconditioner P = B C B I The (left) preconditioned matrix  = P 1 A = A 1 0 A A 1 0 BT BA 1 0 A B BA 1 0 BT + C is not symmetric but is self-adjoint and positive definite when A A0 0 H = 0 I defines an inner product x,y H := x T Hy CG can be used in this inner product Eindhoven, 2012 p.9/30

10 Basic properties: If A 1 and A 2 are self-adjoint in, H then for LEMMA any α,β R, αa 1 + βa 2 is self-adjoint in, H Eindhoven, 2012 p.10/30

11 Basic properties: If A 1 and A 2 are self-adjoint in, H then for LEMMA any α,β R, αa 1 + βa 2 is self-adjoint in, H LEMMA If A is self-adjoint in, H1 and in, H2 then A is self-adjoint in, αh1 +βh 2 for every α,β R Eindhoven, 2012 p.10/30

12 Basic properties: If A 1 and A 2 are self-adjoint in, H then for LEMMA any α,β R, αa 1 + βa 2 is self-adjoint in, H LEMMA If A is self-adjoint in, H1 and in, H2 then A is self-adjoint in, αh1 +βh 2 for every α,β R and of relevance when preconditioning: For symmetric A, Â = P 1 A is self-adjoint in LEMMA, H if and only if P T H is self-adjoint in, A Eindhoven, 2012 p.10/30

13 Basic properties: If A 1 and A 2 are self-adjoint in, H then for LEMMA any α,β R, αa 1 + βa 2 is self-adjoint in, H LEMMA If A is self-adjoint in, H1 and in, H2 then A is self-adjoint in, αh1 +βh 2 for every α,β R and of relevance when preconditioning: For symmetric A, Â = P 1 A is self-adjoint in LEMMA, H if and only if P T H is self-adjoint in, A PROOF (P T H) T A = HP 1 A = (P 1 A) T H = A(P T H) Eindhoven, 2012 p.10/30

14 also combining the above: LEMMA If P 1 and P 2 are left preconditioners for the symmetric matrix A for which symmetric matrices H 1 and H 2 exist with P 1 1 A self-adjoint in, H 1 and P 1 2 A self-adjoint in, H2 and if for any α,β αp T 1 H 1 + βp T 2 H 2 = P T 3 H 3 for some matrix P 3 and some symmetric matrix H 3 then P 1 3 A is self-adjoint in, H 3. shows: if we have two instances of this structure and can find such a splitting we have found a new preconditioner and a bilinear form in which the matrix is self-adjoint. Eindhoven, 2012 p.11/30

15 also combining the above: LEMMA If P 1 and P 2 are left preconditioners for the symmetric matrix A for which symmetric matrices H 1 and H 2 exist with P 1 1 A self-adjoint in, H 1 and P 1 2 A self-adjoint in, H2 and if for any α,β αp T 1 H 1 + βp T 2 H 2 = P T 3 H 3 for some matrix P 3 and some symmetric matrix H 3 then P 1 3 A is self-adjoint in, H 3. shows: if we have two instances of this structure and can find such a splitting we have found a new preconditioner and a bilinear form in which the matrix is self-adjoint. Combination Preconditioners Eindhoven, 2012 p.11/30

16 Intriguing possibilities: if H 1,H 2,H 3 are all positive definite (so define inner products) but P 1 1 A,P 1 2 A are indefinite, can P 1 3 A be positive definite? Eindhoven, 2012 p.12/30

17 Intriguing possibilities: if H 1,H 2,H 3 are all positive definite (so define inner products) but P 1 1 A,P 1 2 A are indefinite, can P 1 3 A be positive definite? can H 1 and H 2 be indefinite but H 3 be positive definite? Eindhoven, 2012 p.12/30

18 Intriguing possibilities: if H 1,H 2,H 3 are all positive definite (so define inner products) but P 1 1 A,P 1 2 A are indefinite, can P 1 3 A be positive definite? can H 1 and H 2 be indefinite but H 3 be positive definite? Answers: YES and YES Eindhoven, 2012 p.12/30

19 Saddle Point examples Bramble-Pasciak CG (Bramble & Pasciak (1988)) widely used CG technique with preconditioner P 1 A 1 = 0 0 BA 1 0 I and inner product matrix H = A A0 0 0 I Eindhoven, 2012 p.13/30

20 Saddle Point examples Bramble-Pasciak CG (Bramble & Pasciak (1988)) widely used CG technique with preconditioner P 1 A 1 = 0 0 BA 1 0 I and inner product matrix H = A A0 0 0 I main drawback: requires A 0 < A but P 1 A is always positive definite when this is true Eindhoven, 2012 p.13/30

21 Examples: BP with Schur complement preconditioner (Klawonn (1998), Meyer et al. (2001), Simoncini (2001)) Inner product: P 1 = H = A S 1 0 BA 1 0 S 1 0 A A0 0 0 S 0 similar conditions as BP for positive definiteness Eindhoven, 2012 p.14/30

22 Examples: Zulehner (Zulehner (2001), Schöberl & Zulehner (2007)) P = A0 B T B BA 1 0 BT S 0 = I 0 BA 1 0 I A0 B T 0 S 0 gives P 1 A self-adjoint in, H, A0 A 0 H = 0 BA 1 0 BT S 0 So H defines an inner product if A 0 > A and S 0 < BA 1 0 BT Whenever H is positive definite, then P 1 A is positive definite in, H. Eindhoven, 2012 p.15/30

23 Examples: Benzi-Simoncini (Benzi and Simoncini (2006)) extension of CG method of Fischer, Ramage, Silvester & W (1998) P 1 I 0 = 0 I inner product: H = A γi B T B γi Extension for C 0 (Liesen (2006), Liesen & Parlett (2007)): A γi B T H = B γi C Eindhoven, 2012 p.16/30

24 Example: Bramble-Pasciak + method (BP + )(Stoll & W(2008)) P 1 A 1 = 0 0 BA 1 0 I and inner product H = A + A0 0 0 I Note: H defines an inner product for any symmetric and positive definite preconditioner A 0 can always apply MINRES in this inner product Eindhoven, 2012 p.17/30

25 Example: Bramble-Pasciak + method (BP + )(Stoll & W(2008)) P 1 A 1 = 0 0 BA 1 0 I and inner product H = A + A0 0 0 I Note: H defines an inner product for any symmetric and positive definite preconditioner A 0 can always apply MINRES in this inner product But preconditioned matrix always indefinite in this inner product Eindhoven, 2012 p.17/30

26 Similarly there exists a Schöberl-Zulehner + method (SZ + ) (Pestana & W(2012)) A0 B T P = B BA 1 0 BT + S 0 = I 0 BA 1 0 I A0 0 0 S 0 I A 1 0 BT 0 I gives P 1 A self-adjoint in, H, A0 + A 0 H = 0 BA 1 0 BT + S 0 So H always defines an inner product, but P 1 A is always indefinite in this inner product. Eindhoven, 2012 p.18/30

27 final example: Block Diagonal Preconditioner (BD) (Silvester& W (1993), Murphy, Golub & W (2000), Korzak (1999), Kuznetsov (1995)) A0 0 P = H = 0 S 0 for which is clearly symmetric. HP 1 A = A = A B T B C But A indefinite P 1 A is always indefinite in, H Eindhoven, 2012 p.19/30

28 Many of the above are special cases of the Krzyzanowski preconditioner (Krzyzanowski (2011)) P = I 0 cba 1 0 I A0 0 I da 1 0 BT 0 S 0 0 I for which P 1 A is self-adjoint in, H with A0 ca 0 H = ǫ 0 S 0 + cdba 1 0 BT + dc,ǫ = ±1 General (but not simple in general) formulae for eigenvalues of P 1 A are available (Pestana & W (2012)) Eindhoven, 2012 p.20/30

29 Combination preconditioning Final lemma above shows that if can find P 3 and H 3 with αp T 1 H 1 + βp T 2 H 2 = P T 3 H 3 this gives a new preconditioner P 3 and the symmetric bilinear form, H3 in which P 1 3 A is self-adjoint Eindhoven, 2012 p.21/30

30 Combine Bramble-Pasciak and Benzi-Simoncini: αp 1 1 H 1 + βp 1 2 H 2 = (αa βi)a (α + βγ)i (αa βi)b T βb (α + βγ)i One possibility for αp T 1 H 1 + βp T 2 H 2 = P T P T 3 = αa βi 0 0 βi 3 H 3 is and H 3 = A (α + βγ)(αa βi) 1 B T B α+βγ β I Eindhoven, 2012 p.22/30

31 Combine BP and BP + αp T 1 H 1 + (1 α)p T 2 H 2 = A 1 0 A + (1 2α)I A 1 0 BT 0 (1 2α)I Eindhoven, 2012 p.23/30

32 Combine BP and BP + αp T 1 H 1 + (1 α)p T 2 H 2 = A 1 0 A + (1 2α)I A 1 0 BT 0 (1 2α)I can be split as P T A 1 3 = 0 A 1 0 BT 0 (1 2α)I,H 3 = A + (1 2α)A0 0 0 I Eindhoven, 2012 p.23/30

33 Combine BP and BP + αp T 1 H 1 + (1 α)p T 2 H 2 = A 1 0 A + (1 2α)I A 1 0 BT 0 (1 2α)I can be split as P T A 1 3 = 0 A 1 0 BT 0 (1 2α)I,H 3 = A + (1 2α)A0 0 0 I Recall P 1 3 A is self-adjoint in, H 3 Eindhoven, 2012 p.23/30

34 Combine BP and BP + αp T 1 H 1 + (1 α)p T 2 H 2 = A 1 0 A + (1 2α)I A 1 0 BT 0 (1 2α)I can be split as P T A 1 3 = 0 A 1 0 BT 0 (1 2α)I,H 3 = A + (1 2α)A0 0 0 I Recall P 1 3 A is self-adjoint in, H 3 (α = 1 BP, α = 0 BP + ) Eindhoven, 2012 p.23/30

35 Combine BP + and SZ + P 3 = H 3 = I 0 BA 1 0 I 1 α+β A 0 β α+β BT 0 S 0 A + A0 0 0 (α + β)s 0 + βba 1 0 BT Clearly H 3 positive definite at least for some α,β but P 1 3 A always indefinite when, H 3 defines an inner product. Eindhoven, 2012 p.24/30

36 Combine BP + and BD P 3 = H 3 = A 0 0 α α+β B 1 α+β S 0 α(a + A0 ) + βa S 0, Theorem: if α > 0 and α + β < 0 then H 3 is positive definite and so, H3 defines an inner product with respect to which P 1 3 A is positive definite if and only if A 0 > α α + β A Eindhoven, 2012 p.25/30

37 CG iteration counts for the 4 standard ifiss Stokes test problems: Taylor-Hood elements (C = 0), A 0 : no-fill ichol, S 0 : mass matrix Problem h BP + BD Comb (α,β) % reduction Channel flow Backward step Regularized cavity Colliding flow (1.3,-2) (1.7,-2) (0.7,-0.6) (1.4,-2) (1.4,-1.6) (1.2,-1) (1.1,-1.8) (1.2,-1.5) (1.9,-2) (1.1,-1.8) (0.8,-1) (1.4,-1.5) 21 Eindhoven, 2012 p.26/30

38 Relative preconditioned residual BDI BP+ BDW Comb MR Comb CG Iteration Eindhoven, 2012 p.27/30

39 CG iteration counts for the 4 standard ifiss Stokes test problems: Taylor-Hood elements (C = 0), A 0 : 1 AMG V-cycle, S 0 : mass matrix Problem h BP + BD Comb (α,β) % reduction Channel flow Backward step Cavity flow Colliding flow (1.1,-2) (1.1,-2) (1.1,-2) (1.1,-2) (1.1,-2) (1.1,-2) (1.1,-2) (1.1,-2) (1.1,-2) (1.1,-2) (1.1,-2) (1.1,-2) 36 Eindhoven, 2012 p.28/30

40 Summary iterative methods more reliable/descriptive convergence theory/know what preconditioning is trying to achieve when matrix is symmetric or self-adjoint Eindhoven, 2012 p.29/30

41 Summary iterative methods more reliable/descriptive convergence theory/know what preconditioning is trying to achieve when matrix is symmetric or self-adjoint for saddle-point problems, examples of self-adjointness in non-standard inner products exist and can be combined to give further examples by interpolation between such examples Eindhoven, 2012 p.29/30

42 Summary iterative methods more reliable/descriptive convergence theory/know what preconditioning is trying to achieve when matrix is symmetric or self-adjoint for saddle-point problems, examples of self-adjointness in non-standard inner products exist and can be combined to give further examples by interpolation between such examples two indefinite examples can be combined to give a positive definite preconditioned matrix (and so allow CG in the associated inner product) Eindhoven, 2012 p.29/30

43 Summary iterative methods more reliable/descriptive convergence theory/know what preconditioning is trying to achieve when matrix is symmetric or self-adjoint for saddle-point problems, examples of self-adjointness in non-standard inner products exist and can be combined to give further examples by interpolation between such examples two indefinite examples can be combined to give a positive definite preconditioned matrix (and so allow CG in the associated inner product) application here to saddle-point matrices, but theory is more general Eindhoven, 2012 p.29/30

44 Acknowledgement This work is partially supported by Award No. KUK-C made by King Abdullah University of Science and Technology (KAUST) Eindhoven, 2012 p.30/30