Spectral Properties of Saddle Point Linear Systems and Relations to Iterative Solvers Part I: Spectral Properties. V. Simoncini

Transcription

1 Spectral Properties of Saddle Point Linear Systems and Relations to Iterative Solvers Part I: Spectral Properties V. Simoncini Dipartimento di Matematica, Università di ologna 1

2 Outline of the -hour Presentation Schematic presentation of certain algebraic preconditioners (Today) Iterative solvers. Some (hopefully) helpful considerations... (Tomorrow) Spectral analysis of nonsymmetric preconditioners (Last Talk)

3 A The problem T u = f C v g Computational Fluid Dynamics (Elman, Silvester, Wathen 00) Elasticity problems Mixed (FE) formulations of II and IV order elliptic PDEs Linearly Constrained Programs Linear Regression in Statistics Image restoration... Survey: enzi, Golub and Liesen, Acta Num 00

4 A The problem. Simplifications T C u = f v g To make things simple: A symmetric positive (semi)definite T tall, possibly rank deficient C symmetric positive (semi)definite Warning: we shall use g = 0 in some cases 4

5 Spectral properties M = A T O 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of σ(m) subset of (Rusten & Winther 199)» q 1 (λ n λ n + 4σ1 ), 1 q (λ 1 λ 1 + 4σ m)» λ n, 1 (λ 1 + qλ 1 + 4σ1 ) xxxx

6 Spectral properties M = A T O 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of σ(m) subset of (Rusten & Winther 199)» q 1 (λ n λ n + 4σ 1 ), 1 (λ 1 qλ 1 + 4σ m ) A nonsingular» λ n, 1 (λ 1 + qλ 1 + 4σ1 ) 6

7 Spectral properties M = A T O 0 = λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of σ(m) subset of» q 1 (λ n λ n + 4σ1 ), 1 q (λ 1 λ 1 + 4σ m)» α 0, 1 (λ 1 + qλ 1 + 4σ1 ) A singular but ut Au u T u > α 0 > 0, u Ker() 7

8 Spectral properties M = A T O 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of σ(m) subset of (Rusten & Winther 199)» q 1 (λ n λ n + 4σ1 ), 1 q (λ 1 λ 1 + 4σ m) full rank» λ n, 1 (λ 1 + qλ 1 + 4σ1 ) 8

9 Spectral properties M = A T C 0 < λ n λ 1 eigs of A 0 = σ m σ 1 sing. vals of σ(m) subset of» q» 1 ( γ 1 + λ n (γ 1 + λ n ) + 4σ1 ),1 (λ 1 qλ 1 + 4θ) λ n, 1 (λ 1 + qλ 1 + 4σ1 ) rank deficient, but θ = λ min ( T + C) full rank γ 1 = λ max (C) 9

10 Spectral properties M = A T O 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of σ(m) subset of (Rusten & Winther 199)» q 1 (λ n λ n + 4σ1 ), 1 q (λ 1 λ 1 + 4σ m) Good (= slim) spectrum: λ 1 λ n, σ 1 σ m» λ n, 1 (λ 1 + qλ 1 + 4σ1 ) e.g. M = I U UT O, UU T = I 10

11 Spectral properties M = A T O 0 < λ n λ 1 eigs of A 0 < σ m σ 1 sing. vals of σ(m) subset of (Rusten & Winther 199)» q 1 (λ n λ n + 4σ1 ), 1 q (λ 1 λ 1 + 4σ m) Good (= slim) spectrum: λ 1 λ n, σ 1 σ m» λ n, 1 (λ 1 + qλ 1 + 4σ1 ) e.g. M = I U UT O, UU T = I 11

12 Find P such that General preconditioning strategy MP 1 û = b û = Pu is easier (faster) to solve than Mu = b A look at efficiency: - Dealing with P should be cheap - Storage requirements for P should be low Possibly zero storage - Properties (algebraic/functional) should be exploited Mesh/parameter independence Structure preserving preconditioners 1

13 lock diagonal Preconditioner A nonsing., C = 0: P 1 0 MP 1 0 = 4 P 0 = A 0 0 A 1 T I A 1 T (A 1 T ) 1 (A 1 T ) 1 A 1 0 MINRES converges in at most iterations. σ(p 1 0 MP 1 0 ) = {1, 1/ ± /} A more practical choice: P = Ã 0 spd. Ã A S A 1 T 0 S eigs in [ a, b] [c, d], a, b, c, d > 0 Still an Indefinite Problem 1

14 lock diagonal Preconditioner A nonsing., C = 0: P 1 0 MP 1 0 = 4 P 0 = A 0 0 A 1 T I A 1 T (A 1 T ) 1 (A 1 T ) 1 A 1 0 MINRES converges in at most iterations. σ(p 1 0 MP 1 0 ) = {1, 1/ ± /} A more practical choice: P = Ã 0 spd. Ã A S A 1 T 0 S eigs in [ a, b] [c, d], a, b, c, d > 0 Still an Indefinite Problem 14

15 Giving up symmetry... Change the preconditioner: Mimic the LU factors M = 4 I O A 1 I 4 A T O A 1 T + C P Change the preconditioner: Mimic the Structure M = A T P M C 4 A T O A 1 T + C Change the matrix: Eliminate indef. M = A T C Change the matrix: Regularize (C = 0) M M γ = 4 A T γw or M γ = 4 A + 1 γ T W 1 T O 1

16 Giving up symmetry... Change the preconditioner: Mimic the LU factors M = 4 I O A 1 I 4 A T O A 1 T + C P 4 A T O A 1 T + C Change the preconditioner: Mimic the Structure M = A T M P C Change the matrix: Eliminate indef. M = A T C Change the matrix: Regularize (C = 0) M M γ = 4 A T γw or M γ = 4 A + 1 γ T W 1 T O 16

17 Giving up symmetry... Change the preconditioner: Mimic the LU factors M = 4 I O A 1 I 4 A T O A 1 T + C P 4 A T O A 1 T + C Change the preconditioner: Mimic the Structure M = A T P M C Change the matrix: Eliminate indef. M = A T C Change the matrix: Regularize (C = 0) M M γ = 4 A T γw or M γ = 4 A + 1 γ T W 1 T O 17

18 Giving up symmetry... Change the preconditioner: Mimic the LU factors M = 4 I O A 1 I 4 A T O A 1 T + C P 4 A T O A 1 T + C Change the preconditioner: Mimic the Structure M = A T P M C Change the matrix: Eliminate indef. M = A T C Change the matrix: Regularize (C = 0) M M γ = 4 A T γw or M γ = 4 A + 1 γ T W 1 T O 18

19 Nonstandard inner product:... ut recovering symmetry in disguise Let W be any of MP 1, M For σ(w) in R +, find sym matrix H such that (that is, W is H-symmetric) WH = HW T If H is spd then W is diagonalizable Use PCG on W with H-inner product 19

20 Nonstandard inner product:... ut recovering symmetry in disguise Let W be any of MP 1, M For σ(w) in R +, find sym matrix H such that (that is, W is H-symmetric) WH = HW T If H is spd then W is diagonalizable Use PCG on W with H-inner product 0

21 Triangular preconditioner A spd, P = à T 0 C à A, C A 1 T + C Ideal case: A e = A, C e = A 1 T + C MP 1 = 4 I 0 A 1 I Recovering symmetry? If C = C nonsing., then σ(mp 1 ) in R + If à < A then σ(mp 1 ) in R + with λ [χ 1, χ ] 1, χ j = χ j (( T à 1 + C) C 1, à 1 A) 1

22 Triangular preconditioner A spd, P = à T 0 C à A, C A 1 T + C Ideal case: A e = A, C e = A 1 T + C MP 1 = 4 I 0 A 1 I Recovering symmetry? If C = C nonsing., then σ(mp 1 ) in R + If à < A then σ(mp 1 ) in R + with λ [χ 1, χ ] 1, χ j = χ j (( T à 1 + C) C 1, à 1 A)

23 Constraint (Indefinite) Preconditioner P = Ã T C MP 1 = AÃ 1 (I Π) + Π O I with Π = ( e A 1 T + C) 1 e A 1 If C nonsing all eigs real and positive If T C = 0 and T + C > 0 all eigs real and positive Special case: C = 0 at most m unit eigs with Jordan blocks

24 Constraint (Indefinite) Preconditioner. Generalizations P = à T C Primal-based: C C nonsing, à A + T C 1 If A + T C 1 > à and C > C all eigs real and positive Dual-based: (C = O) à A, C = S à 1 T for some S If à > A and C < 0 all eigs real and positive MP 1 is H-symmetric with H = blkdiag(ã A, à 1 T S) 4

25 Constraint (Indefinite) Preconditioner. Generalizations P = à T C Primal-based: C C nonsing, à A + T C 1 If A + T C 1 > à and C > C all eigs real and positive Dual-based: (C = O) à A, C = S à 1 T for some S If à > A and C < 0 all eigs real and positive MP 1 is H-symmetric with H = blkdiag(ã A, à 1 T S)

26 The minus-signed Problem M = A T C full rank M positive stable eigs in C + Important facts M has always at least n m real eigs If < λ min (A) λ max (C) all eigs real and positive for C = 0, condition simplifies: λ min (A) > 4λ max ( T A 1 ) If full rank and λ min (A) > λ max (C) apon scaling all eigs real and positive Hermitian-Skew-Hermitian preconditioners lock diagonal preconditioner + nonsym solver 6

27 The minus-signed Problem M = A T C full rank M positive stable eigs in C + Important facts M has always at least n m real eigs If < λ min (A) λ max (C) all eigs real and positive for C = 0, condition simplifies: λ min (A) > 4λ max ( T A 1 ) If full rank and λ min (A) > λ max (C) apon scaling all eigs real and positive Hermitian-Skew-Hermitian preconditioners lock diagonal preconditioner + nonsym solver 7

28 The minus-signed Problem M = A T C full rank M positive stable eigs in C + Important facts M has always at least n m real eigs If < λ min (A) λ max (C) all eigs real and positive for C = 0, condition simplifies: λ min (A) > 4λ max ( T A 1 ) If full rank and λ min (A) > λ max (C) apon scaling all eigs real and positive Hermitian-Skew-Hermitian preconditioners lock diagonal preconditioner + nonsym solver 8

29 Regularized Problem Augmented Lagrangian approach: M γ = A + 1 γ T W 1 T O Particularly interesting for A indefinite or singular Any of the above preconditioners may be used. Somehow related preconditioner for M = A O P = A + T W 1 k T O W T : 9

30 Regularized Problem Augmented Lagrangian approach: M γ = A + 1 γ T W 1 T O Particularly interesting for A indefinite or singular Any of the above preconditioners may be used. Somehow related preconditioner for M = P = A + T W 1 O A T W T O : 0

31 Questions (to be answered) Which formulation/preconditioner for which iterative solver? Is the theoretical spectral information useful in practice? Are the imposed constraints needed? 1

32 Questions (to be answered) Which formulation/preconditioner for which iterative solver? Is the theoretical spectral information useful in practice? Are the imposed constraints needed?

33 Questions (to be answered) Which formulation/preconditioner for which iterative solver? Is the theoretical spectral information useful in practice? Are the imposed constraints needed?