Structured Preconditioners for Saddle Point Problems

Transcription

1 Structured Preconditioners for Saddle Point Problems V. Simoncini Dipartimento di Matematica Università di Bologna p. 1

2 Collaborators on this project Mario Arioli, RAL, UK Michele Benzi, Emory University (GA) Ilaria Perugia, Università di Pavia Miro Rozloznik, Institute of Computer Science, Academy of Science, Prague p. 2

3 Application problems Computational Fluid Dynamics Elasticity problems Mixed (FE) formulations of II and IV order elliptic PDEs Linearly Constrained Programs Linear Regression in Statistics Weighted Least Squares (Image restoration)... p. 3

4 Application problems Computational Fluid Dynamics Elasticity problems Mixed (FE) formulations of II and IV order elliptic PDEs Linearly Constrained Programs Linear Regression in Statistics Weighted Least Squares (Image restoration)... A B BT C u v = f g p. 3

5 Motivational Application Constrained Quadratic minimization problem minimize J(u) = 1 Au, u f, u 2 subject to Bu = g A n n symmetric, B m n, m n full rank Lagrange multipliers approach Karush-Kuhn-Tucker (KKT) system p. 4

6 The algebraic Saddle Point Problem A B 0 BT u v = f g A sym. pos.semidef., B full rank p. 5

7 The algebraic Saddle Point Problem A B BT C u v = f g A sym. pos.semidef., B full rank A sym. pos.semidef., B rank defic., C sym. [semi]def. p. 5

8 The algebraic Saddle Point Problem A B BT C u v = f g A sym. pos.semidef., B full rank A sym. pos.semidef., B rank defic., C sym. [semi]def. A nonsym., B and C as above p. 5

9 The algebraic Saddle Point Problem A B BT C u v = f g A sym. pos.semidef., B full rank A sym. pos.semidef., B rank defic., C sym. [semi]def. A nonsym., B and C as above A sym, Mx = b M sym. indef. With n positive and m negative real eigenvalues p. 5

10 Typical Sparsity pattern (3D problem) nz = 768 p. 6

11 Spectral properties M = A B 0 BT 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of B p. 7

12 Spectral properties M = A B 0 BT 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of B (Rusten & Winther 1992) Λ(M) subset of» q 1 2 (λ n λ 2 n + 4σ2 1 ),, 1 2 (λ 1 qλ σ2 m )» λ n, 1 2 (λ 1 + qλ σ2 m ) p. 7

13 Spectral properties M = A B BT C 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of B (Rusten & Winther 1992) Λ(M) subset of» q 1 2 (λ n λ 2 n + 4σ2 1 ), 1 2 (λ 1 qλ σ2 m )» λ n, 1 2 (λ 1 + qλ σ2 m ) (Silvester & Wathen 1994), 0 σ m σ 1 λ n λ max (C) q (λ n + λ max (C)) 2 + 4σ 2 1 p. 7

14 Spectral properties M = A B BT C 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of B (Rusten & Winther 1992) Λ(M) subset of» q 1 2 (λ n λ 2 n + 4σ2 1 ),, 1 2 (λ 1 qλ σ2 m )» λ n, 1 2 (λ 1 + qλ σ2 m ) (Silvester & Wathen 1994), 0 σ m σ 1 λ n λ max (C) q (λ n + λ max (C)) 2 + 4σ 2 1 More results for special cases (e.g. Perugia & S. 2000) p. 7

15 General preconditioning strategy Find P such that MP 1 û = b û = Pu is easier (faster) to solve than Mu = b p. 8

16 General preconditioning strategy Find P such that MP 1 û = b û = Pu is easier (faster) to solve than Mu = b A look at efficiency: - Dealing with P should be cheap - Storage for P should be low - Properties (algebraic/functional) exploited p. 8

17 Structure preserving preconditioning Idealized case: p. 9

18 Structure preserving preconditioning Idealized case: A nonsing., C = 0: P = A 0 0 B T A 1 B MP 1 eigs 1, 1 2 ± 5 2 MINRES converges in at most 3 iterations (Murphy, Golub & Wathen, 2002) p. 9

19 Structure preserving preconditioning Idealized case: A nonsing., C = 0: P = A 0 0 B T A 1 B MP 1 eigs 1, 1 2 ± 5 2 MINRES converges in at most 3 iterations (Murphy, Golub & Wathen, 2002) A nonsing., C 0: P = A B 0 B T A 1 B + C MP 1 = I 0 B T A 1 I GMRES converges in at most 2 iterations p. 9

20 Block diagonal Preconditioner P = Ã 0 0 C sym. pos. def. Ã A C BA 1 B T + C Rusten Winther (1992), Silvester Wathen ( ), Klawonn (1998) Fischer Ramage Silvester Wathen ( ),... p. 10

21 Block diagonal Preconditioner P = Ã 0 0 C sym. pos. def. Ã A C BA 1 B T + C Rusten Winther (1992), Silvester Wathen ( ), Klawonn (1998) Fischer Ramage Silvester Wathen ( ),... λ 0 eigs of P 1 2 MP 1 2, λ [ a, b] [c, d] p. 10

22 Q = [ Ã B C B T Constraint Preconditioner Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), Braess Sarazin (1997) Golub Wathen (1998) Vassilevski Lazarov (1996), Lukšan Vlček ( ), Perugia S. Arioli (1999), Keller Gould Wathen (2000) Perugia S. (2000), Gould Hribar Nocedal (2001), Rozloznik S. (2002), Durazzi Ruggiero (2003), Axelsson Neytcheva (2003),... ] p. 11

23 Q = [ Ã B C B T Constraint Preconditioner Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), Braess Sarazin (1997) Golub Wathen (1998) Vassilevski Lazarov (1996), Lukšan Vlček ( ), Perugia S. Arioli (1999), Keller Gould Wathen (2000) Perugia S. (2000), Gould Hribar Nocedal (2001), Rozloznik S. (2002), Durazzi Ruggiero (2003), Axelsson Neytcheva (2003),... ] λ 0 eigs di MQ 1, λ R +, λ {1} [α 0, α 1 ] p. 11

24 Q = [ Ã B C B T Constraint Preconditioner Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), Braess Sarazin (1997) Golub Wathen (1998) Vassilevski Lazarov (1996), Lukšan Vlček ( ), Perugia S. Arioli (1999), Keller Gould Wathen (2000) Perugia S. (2000), Gould Hribar Nocedal (2001), Rozloznik S. (2002), Durazzi Ruggiero (2003), Axelsson Neytcheva (2003),... ] λ 0 eigs di MQ 1, λ R +, λ {1} [α 0, α 1 ] Remark: B T x k = 0 for all iterates x k p. 11

25 Q = [ Ã B C B T Constraint Preconditioner Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), Braess Sarazin (1997) Golub Wathen (1998) Vassilevski Lazarov (1996), Lukšan Vlček ( ), Perugia S. Arioli (1999), Keller Gould Wathen (2000) Perugia S. (2000), Gould Hribar Nocedal (2001), Rozloznik S. (2002), Durazzi Ruggiero (2003), Axelsson Neytcheva (2003),... ] λ 0 eigs di MQ 1, λ R +, λ {1} [α 0, α 1 ] Remark: B T x k = 0 for all iterates x k Q 1 = [ I B T 0 I ] [ I 0 0 (BB T + C) 1 ] [ I 0 B I ] p. 11

26 Computational Considerations. I 3D Magnetostatic problem Size QMR QMR(Q) > > p. 12

27 Computational Considerations. II 3D Magnetostatic problem H BB T + C with H: Incomplete Cholesky fact. (ICT package, Saad & Chow) p. 13

28 Computational Considerations. II 3D Magnetostatic problem H BB T + C with H: Incomplete Cholesky fact. (ICT package, Saad & Chow) Elapsed Time Size MA27 QMR QMR Q Q(2)(it) ILDLT(10) (18) (18) (20) (29) (45) p. 13

29 Computational Considerations. II 3D Magnetostatic problem H BB T + C with H: Incomplete Cholesky fact. (ICT package, Saad & Chow) Elapsed Time Size MA27 QMR QMR QMR Q Q(2)(it) ILDLT(10) P P(2)(it) (18) (18) (20) (29) (45) p. 13

30 Spectrum of perturbed problem 3D Magnetostatic problem 0.5 (a) p. 14

31 Spectrum of perturbed problem 3D Magnetostatic problem 0.5 (a) 0.5 (d) (BB T + C) H BB T + C p. 14

32 Triangular preconditioner A spd, P = Ã BT 0 C Bramble & Pasciak, Elman, Klawonn, Axelsson & Neytcheva, S p. 15

33 Triangular preconditioner A spd, P = Ã BT 0 C Bramble & Pasciak, Elman, Klawonn, Axelsson & Neytcheva, S Spectrum of MP 1 : small complex cluster around 1 + real interval p. 15

34 Triangular preconditioner A spd, P = Ã BT 0 C Bramble & Pasciak, Elman, Klawonn, Axelsson & Neytcheva, S Spectrum of MP 1 : small complex cluster around 1 + real interval More precisely: θ Λ(MP 1 ) I(θ) 0 θ 1 1 λ min (AÃ 1 ) (if 1 λ min (AA e 1 ) 0) I(θ) = 0 θ [χ 1, χ 2 ] 1 p. 15

35 A BT B C u v = A different perspective f M x = d g Polyak 1970,..., Fischer & Ramage & Silvester & Wathen 1997, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004,... M positive real Λ(M ) in C + p. 16

36 A BT B C u v = A different perspective f M x = d g Polyak 1970,..., Fischer & Ramage & Silvester & Wathen 1997, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004,... M positive real Λ(M ) in C + More refined spectral information possible p. 16

37 A BT B C u v = A different perspective f M x = d g Polyak 1970,..., Fischer & Ramage & Silvester & Wathen 1997, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004,... M positive real Λ(M ) in C + More refined spectral information possible New classes of preconditioners p. 16

38 A BT B C u v = A different perspective f M x = d g Polyak 1970,..., Fischer & Ramage & Silvester & Wathen 1997, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004,... M positive real Λ(M ) in C + More refined spectral information possible New classes of preconditioners General framework for spectral analysis of some indefinite preconditioners p. 16

39 Spectral properties of M M = A BT B C A n n sym. semidef. matrix, B m n, m n M has at least n m real eigenvalues p. 17

40 Reality condition Let C = βi. If λ min (A + βi) 4 λ max (B T A 1 B + βi), then all eigenvalues of M are real. e.g. Stokes problem(c = 0) Benzi & S. (in prep.) p. 18

41 Reality condition Let C = βi. If λ min (A + βi) 4 λ max (B T A 1 B + βi), then all eigenvalues of M are real. e.g. Stokes problem(c = 0) Benzi & S. (in prep.) Moreover, if X is s.t. M = XΛX 1, then κ(x) X 2 1 χ, χ = 4 λ max(b T A 1 B + βi) λ min (A + βi) (columns of X have unit norm) p. 18

42 Location of spectrum Let λ Λ(M ), A spd (cf. Sidi 2003 for C = 0) If I(λ) 0 then 1 2 (λ min(a) + λ min (C)) R(λ) 1 2 (λ max(a) + λ max (C)) I(λ) σ max (B). p. 19

43 Location of spectrum Let λ Λ(M ), A spd (cf. Sidi 2003 for C = 0) If I(λ) 0 then 1 2 (λ min(a) + λ min (C)) R(λ) 1 2 (λ max(a) + λ max (C)) If I(λ) = 0 then I(λ) σ max (B). min{λ min (A), λ min (C)} λ (λ max (A) + λ max (C)). p. 19

44 Location of spectrum Let λ Λ(M ), A spd (cf. Sidi 2003 for C = 0) If I(λ) 0 then 1 2 (λ min(a) + λ min (C)) R(λ) 1 2 (λ max(a) + λ max (C)) If I(λ) = 0 then I(λ) σ max (B). min{λ min (A), λ min (C)} λ (λ max (A) + λ max (C)). M = λ 1 = 1, λ 2,3 = 3 2 ± i 3 2. p. 19

45 Hermitian Skew-Hermitian Preconditioning M = A BT B C = A 0 0 C + 0 BT B 0 = H + S p. 20

46 Hermitian Skew-Hermitian Preconditioning M = A BT B C = A 0 0 C + 0 BT B 0 = H + S Use the preconditioner R α = 1 (H + αi)(s + αi) α R, α > 0 2α Bai & Golub & Ng 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004, Benzi & Ng, 2004 p. 20

47 Reality condition Assume A is sym. positive definite, C = 0. If α 1 2 λ min(a) then all eigenvalues η s of R 1 α M are real p. 21

48 Reality condition Assume A is sym. positive definite, C = 0. If α 1 2 λ min(a) then all eigenvalues η s of R 1 α M are real We provide bounds for real and imaginary part of eigenvalues Stokes Problem α Lower bound η min η max Upper bound p. 21

49 Spectral bounds Spectrum of R 1 α M (case C = 0) 0 2 (S. & Benzi 2004) p. 22

50 A General framework. I p. 23

51 A General framework. I Eigenvalue problem A BT B C A BT B C x y x y = λ x y = λ I 0T 0 I x y p. 23

52 A General framework. I Eigenvalue problem A BT B C A BT B C x y x y = λ x y = λ I 0T 0 I x y Generalization A BT x B C y = λ K 0T 0 H x y K, H spd p. 23

53 A B BT C z = λpz  B A General Framework. II B T w = λ I 0T w Ĉ 0 I  0, Ĉ 0 p. 24

54 A B BT C z = λpz  B A General Framework. II B T w = λ I 0T w Ĉ 0 I  0, Ĉ 0 Indefinite Block diagonal Preconditioner P = 2 4 A b 0 0 C b 3 5 cf. Fischer et al p. 24

55 A B BT C z = λpz  B A General Framework. II B T w = λ I 0T w Ĉ 0 I  0, Ĉ 0 Indefinite Block diagonal Preconditioner P = 2 4 A b 0 0 C b 3 5 cf. Fischer et al Hermitian Skew-Hermitian Preconditioner (C = βi) p. 24

56 A B BT C z = λpz  B A General Framework. II B T w = λ I 0T w Ĉ 0 I  0, Ĉ 0 Indefinite Block diagonal Preconditioner P = 2 4 A b 0 0 C b 3 5 cf. Fischer et al Hermitian Skew-Hermitian Preconditioner (C = βi) Inexact Constraint Preconditioner Q = 2 4 I 0 B I I 0 0 H I BT 0 I 3 5 H BB T + C p. 24

57 Final Considerations Performance of preconditioners is problem dependent p. 25

58 Final Considerations Performance of preconditioners is problem dependent Ad-Hoc preconditioners usually designed e.g. Choices of Ã, C (information from application problem) p. 25

59 Final Considerations Performance of preconditioners is problem dependent Ad-Hoc preconditioners usually designed e.g. Choices of Ã, C (information from application problem) In new framework, eigenvector analysis may be challenging p. 25

60 Final Considerations Performance of preconditioners is problem dependent Ad-Hoc preconditioners usually designed e.g. Choices of Ã, C (information from application problem) In new framework, eigenvector analysis may be challenging Non-Hermitian problem not fully understood p. 25

61 Final Considerations Performance of preconditioners is problem dependent Ad-Hoc preconditioners usually designed e.g. Choices of Ã, C (information from application problem) In new framework, eigenvector analysis may be challenging Non-Hermitian problem not fully understood Visit simoncin p. 25