Structured Preconditioners for Saddle Point Problems V. Simoncini Dipartimento di Matematica Università di Bologna valeria@dm.unibo.it. p.1/16
Collaborators on this project Mario Arioli, RAL, UK Michele Benzi, Emory University (GA) Ilaria Perugia, Università di Pavia Miro Rozloznik, Institute of Computer Science, Academy of Science, Prague. p.2/16
Motivational Application Constrained minimization problem subject to Bu = g minimize J(u) = 1 Au, u f, u 2 A n n symmetric, B m n, m n Lagrange multipliers approach Karush-Kuhn-Tucker (KKT) system. p.3/16
Application problems Computational Fluid Dynamics Elasticity problems Mixed (FE) formulations of II and IV order elliptic PDEs Linearly Constrained Programs Linear Regression in Statistics Weighted Least Squares (Image restoration).... p.4/16
The algebraic Saddle Point Problem A B 0 BT u v = f g A sym. pos.semidef., B full rank. p.5/16
The algebraic Saddle Point Problem A B BT C u v = f g A sym. pos.semidef., B full rank A sym. pos.semidef., B rank defic., C sym. [semi]def.. p.5/16
The algebraic Saddle Point Problem A B BT C u v = f g A sym. pos.semidef., B full rank A sym. pos.semidef., B rank defic., C sym. [semi]def. A nonsym., B and C as above. p.5/16
The algebraic Saddle Point Problem A B BT C u v = f g A sym. pos.semidef., B full rank A sym. pos.semidef., B rank defic., C sym. [semi]def. A nonsym., B and C as above A sym, Mx = b M sym. indef. With n positive and m negative real eigenvalues. p.5/16
Typical Sparsity pattern (3D problem) 0 100 200 300 400 500 600 700 800 0 100 200 300 400 500 600 700 800 nz = 768. p.6/16
Spectral properties M = A B 0 BT 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of B. p.7/16
Spectral properties M = A B 0 BT 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of B (Rusten & Winther 1992) Λ(M) subset of» q 1 2 (λ n λ 2 n + 4σ2 1 ),, 1 2 (λ 1 qλ 21 + 4σ2 m )» λ n, 1 2 (λ 1 + qλ 21 + 4σ2 m ). p.7/16
Spectral properties M = A B BT C 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of B (Rusten & Winther 1992) Λ(M) subset of» q 1 2 (λ n λ 2 n + 4σ2 1 ), 1 2 (λ 1 qλ 21 + 4σ2 m )» λ n, 1 2 (λ 1 + qλ 21 + 4σ2 m ) (Silvester & Wathen 1994), 0 σ m σ 1 λ n λ max (C) q (λ n + λ max (C)) 2 + 4σ 2 1. p.7/16
Spectral properties M = A B BT C 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of B (Rusten & Winther 1992) Λ(M) subset of» q 1 2 (λ n λ 2 n + 4σ2 1 ),, 1 2 (λ 1 qλ 21 + 4σ2 m )» λ n, 1 2 (λ 1 + qλ 21 + 4σ2 m ) (Silvester & Wathen 1994), 0 σ m σ 1 λ n λ max (C) q (λ n + λ max (C)) 2 + 4σ 2 1 More results for special cases (e.g. Perugia & S. 2000). p.7/16
General preconditioning strategy Find P such that MP 1 û = b û = Pu is easier (faster) to solve than Mu = b. p.8/16
General preconditioning strategy Find P such that MP 1 û = b û = Pu is easier (faster) to solve than Mu = b A look at efficiency: - Dealing with P should be cheap - Storage for P should be low - Properties (algebraic/functional) exploited. p.8/16
Structure preserving preconditioning Idealized case:. p.9/16
Structure preserving preconditioning Idealized case: A nonsing., C = 0: P = A 0 0 B T A 1 B MP 1 eigs 1, 1 2 ± 5 2 MINRES converges in at most 3 iterations (Murphy, Golub & Wathen, 2002). p.9/16
Structure preserving preconditioning Idealized case: A nonsing., C = 0: P = A 0 0 B T A 1 B MP 1 eigs 1, 1 2 ± 5 2 MINRES converges in at most 3 iterations (Murphy, Golub & Wathen, 2002) A nonsing., C 0: P = A B 0 B T A 1 B + C MP 1 = I 0 B T A 1 I GMRES converges in at most 2 iterations. p.9/16
Block diagonal Preconditioner P = Ã 0 0 C sym. pos. def. Rusten Winther (1992), Silvester Wathen (1993-1994), Klawonn (1998) Fischer Ramage Silvester Wathen (1998...),.... p.10/16
Block diagonal Preconditioner P = Ã 0 0 C sym. pos. def. Rusten Winther (1992), Silvester Wathen (1993-1994), Klawonn (1998) Fischer Ramage Silvester Wathen (1998...),... λ 0 eigs of P 1 2 MP 1 2, λ [ a, b] [c, d]. p.10/16
Q = [ Ã B C B T Constraint Preconditioner Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), Braess Sarazin (1997) Golub Wathen (1998) Vassilevski Lazarov (1996), Lukšan Vlček (1998-1999), Perugia S. Arioli (1999), Keller Gould Wathen (2000) Perugia S. (2000), Gould Hribar Nocedal (2001), Rozloznik S. (2002), Durazzi Ruggiero (2003), Axelsson Neytcheva (2003),... ]. p.11/16
Q = [ Ã B C B T Constraint Preconditioner Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), Braess Sarazin (1997) Golub Wathen (1998) Vassilevski Lazarov (1996), Lukšan Vlček (1998-1999), Perugia S. Arioli (1999), Keller Gould Wathen (2000) Perugia S. (2000), Gould Hribar Nocedal (2001), Rozloznik S. (2002), Durazzi Ruggiero (2003), Axelsson Neytcheva (2003),... ] λ 0 eigs di MQ 1, λ R +, λ {1} [α 0, α 1 ]. p.11/16
Q = [ Ã B C B T Constraint Preconditioner Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), Braess Sarazin (1997) Golub Wathen (1998) Vassilevski Lazarov (1996), Lukšan Vlček (1998-1999), Perugia S. Arioli (1999), Keller Gould Wathen (2000) Perugia S. (2000), Gould Hribar Nocedal (2001), Rozloznik S. (2002), Durazzi Ruggiero (2003), Axelsson Neytcheva (2003),... ] λ 0 eigs di MQ 1, λ R +, λ {1} [α 0, α 1 ] Remark: B T x k = 0 for all iterates x k. p.11/16
Q = [ Ã B C B T Constraint Preconditioner Axelsson (1979), Ewing Lazarov Lu Vassilevski (1990), Braess Sarazin (1997) Golub Wathen (1998) Vassilevski Lazarov (1996), Lukšan Vlček (1998-1999), Perugia S. Arioli (1999), Keller Gould Wathen (2000) Perugia S. (2000), Gould Hribar Nocedal (2001), Rozloznik S. (2002), Durazzi Ruggiero (2003), Axelsson Neytcheva (2003),... ] λ 0 eigs di MQ 1, λ R +, λ {1} [α 0, α 1 ] Remark: B T x k = 0 for all iterates x k Q 1 = [ I B T 0 I ] [ I 0 0 (BB T + C) 1 ] [ I 0 B I ]. p.11/16
A BT B C u v = A different perspective f M x = d g Polyak 1970,..., Fischer & Ramage & Silvester & Wathen 1997, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004,... Λ(M ) in C +. p.12/16
A BT B C u v = A different perspective f M x = d g Polyak 1970,..., Fischer & Ramage & Silvester & Wathen 1997, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004,... Λ(M ) in C + More refined spectral information possible. p.12/16
A BT B C u v = A different perspective f M x = d g Polyak 1970,..., Fischer & Ramage & Silvester & Wathen 1997, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004,... Λ(M ) in C + More refined spectral information possible New classes of preconditioners. p.12/16
A BT B C u v = A different perspective f M x = d g Polyak 1970,..., Fischer & Ramage & Silvester & Wathen 1997, Bai & Golub & Ng 2003, Sidi 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004,... Λ(M ) in C + More refined spectral information possible New classes of preconditioners General framework for spectral analysis of some inexact preconditioners. p.12/16
Spectral properties of M A n n sym. semidef. matrix, B m n, m n M has at least n m real eigenvalues. p.13/16
Spectral properties of M A n n sym. semidef. matrix, B m n, m n M has at least n m real eigenvalues Let C = βi. If λ min (A + βi) 4 λ max (B T A 1 B + βi), then all eigenvalues of M are real.. p.13/16
Spectral properties of M A n n sym. semidef. matrix, B m n, m n M has at least n m real eigenvalues Let C = βi. If λ min (A + βi) 4 λ max (B T A 1 B + βi), then all eigenvalues of M are real. Let C be sym. Let λ Λ(M ). If I(λ) 0, then 1 2 (λ min(a) + λ min (C)) R(λ) 1 2 (λ max(a) + λ max (C)) If I(λ) = 0 then I(λ) σ max (B). 2 min{λ min (A), λ min (C)} λ (λ max (A) + λ max (C)). cf. Sidi 2003 for C = 0. p.13/16
Hermitian Skew-Hermitian Preconditioning M = A BT B C = A 0 0 C + 0 BT B 0 = H + S. p.14/16
Hermitian Skew-Hermitian Preconditioning M = A BT B C = A 0 0 C + 0 BT B 0 = H + S Use the preconditioner R = 1 (H + αi)(s + αi) α R, α > 0 2α Bai & Golub & Ng 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004. p.14/16
Hermitian Skew-Hermitian Preconditioning M = A BT B C = A 0 0 C + 0 BT B 0 = H + S Use the preconditioner R = 1 (H + αi)(s + αi) α R, α > 0 2α Bai & Golub & Ng 2003, Benzi & Gander & Golub 2003, Benzi & Golub 2004, S. & Benzi 2004 Sharp spectral bounds for C = 0 (S. & Benzi 2004). p.14/16
General framework P 1 A B BT C Ã B T B C Ã 0, C 0 We can provide a spectral analysis when P is:. p.15/16
General framework P 1 A B BT C Ã B T B C Ã 0, C 0 We can provide a spectral analysis when P is: Inexact Constraint Preconditioner. p.15/16
General framework P 1 A B BT C Ã B T B C Ã 0, C 0 We can provide a spectral analysis when P is: Inexact Constraint Preconditioner Hermitian Skew-Hermitian Preconditioner (C = βi). p.15/16
General framework P 1 A B BT C à B T B C à 0, C 0 We can provide a spectral analysis when P is: Inexact Constraint Preconditioner Hermitian Skew-Hermitian Preconditioner (C = βi) Indefinite Block diagonal Preconditioner  0 0 Ĉ cf. Fischer et al. 1997. p.15/16
Final Considerations Performance of preconditioners is problem dependent. p.16/16
Final Considerations Performance of preconditioners is problem dependent Ad-Hoc preconditioners usually designed (information from application problem). p.16/16
Final Considerations Performance of preconditioners is problem dependent Ad-Hoc preconditioners usually designed (information from application problem) Non-Hermitian problem not fully understood. p.16/16
Final Considerations Performance of preconditioners is problem dependent Ad-Hoc preconditioners usually designed (information from application problem) Non-Hermitian problem not fully understood Visit http://www.dm.unibo.it/ simoncin. p.16/16