Targeting (for MSPAI)

Size: px

Start display at page:

Download "Targeting (for MSPAI)"

Patience Gaines
5 years ago
Views:

1 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning Targeting for MSPAI min M P AM T F Assume, T is a goo sparse preonitioner for A. Improve T y omputing M an solving the aove Froenius norm minimization. Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 7 of 79

2 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning Targeting for MSPAI min M P AM T F Assume, T is a goo sparse preonitioner for A. Improve T y omputing M an solving the aove Froenius norm minimization. Appliation: Assume that A is given y two parts, e.g. an avetion part an a iffusion part. We an hoose T as Laplaian relative to the iffusion part easy to solve an then we a M for improving T relative to the avetion part. Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 7 of 79

3 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning Proing for MSPAI Fin preonitioner of speial form triiag, an for preonitioning a matri that is not given epliitly, ut only y its ation on ertain proing-vetors, e.g. e T S = f T. Eample: S is Shur omplement or general matri. Choose, e.g. e =,,..., T, e =,,,,... T,... Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 7 of 79

4 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning Proing for MSPAI Fin preonitioner of speial form triiag, an for preonitioning a matri that is not given epliitly, ut only y its ation on ertain proing-vetors, e.g. e T S = f T. Eample: S is Shur omplement or general matri. Choose, e.g. e =,,..., T, e =,,,,... T,... Disavantage: Can use only very speial pattern for M an speial proing vetors e. Eample: triiagonal proing Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 7 of 79

5 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning. Moifie SPAI: SPAI with Targeting an Proing Generalize Froenius norm minimization to min CM B F = min C M P M P ρu T M B ρv T For eample: original SPAI etene y an aitional norm minimization to eliver espeially goo results on vetor e: min CM B F = min A I M P M P ρe T M A ρe T F F Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 75 of 79

6 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning SPAI for Triangular Matries Solve L = with L triangular. Goal: Improve Jaoi iteration! SPAI: min LΛ I F with pattern of Λ lie L. Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 76 of 79

7 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning SPAI for Triangular Matries Solve L = with L triangular. Goal: Improve Jaoi iteration! SPAI: min LΛ I F with pattern of Λ lie L. Moifiation: Sparse approimate lo inverse. Replae retangular Least Squares prolem in SPAI LI, JΛ j = e j y square linear system LJ, JΛ j = e j. Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 76 of 79

8 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning SPAI for Triangular Matries Solve L = with L triangular. Goal: Improve Jaoi iteration! SPAI: min LΛ I F with pattern of Λ lie L. Moifiation: Sparse approimate lo inverse. Replae retangular Least Squares prolem in SPAI LI, JΛ j = e j y square linear system LJ, JΛ j = e j. Stationary iterative metho ase on preonitioner Λ: = LΛΛ = LΛy Solution = Λy iaglλy + = LΛ y Convergene again guarantee, faster than Jaoi, fully parallel! Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 76 of 79

9 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning Hyri SPAI - Gauss-Seiel First ompute SPAI-lie preonitioner via min AM I F. Replae = A y = AMM = AM = Ã. Apply Gauss-Seiel splitting on Ã = D + L + U = L + U. Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 77 of 79

10 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning Hyri SPAI - Gauss-Seiel First ompute SPAI-lie preonitioner via min AM I F. Replae = A y = AMM = AM = Ã. Apply Gauss-Seiel splitting on Ã = D + L + U = L + U. In every iteration step we have to solve triangular system in L. Apply sparse approimate lo inverse preonitioner for L. Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 77 of 79

11 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning Hyri SPAI - Gauss-Seiel First ompute SPAI-lie preonitioner via min AM I F. Replae = A y = AMM = AM = Ã. Apply Gauss-Seiel splitting on Ã = D + L + U = L + U. In every iteration step we have to solve triangular system in L. Apply sparse approimate lo inverse preonitioner for L. Everything fully parallel. Improve onvergene ompare to plain SPAI or Gauss-Seiel Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 77 of 79

12 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning Outloo: Further Researh. Apply MSPAI with the proing feature to multigri or regularization prolems smoother, preonitioner. In onnetion with omain eomposition: In the small omains use MSPAI as smoother Use MSPAI as preonitioner for Shur omplement. Fin goo lo pattern that reflets the physial onnetions omines olumns with very similar pattern to one LS prolem. Fatorize SPAI for inefinite matries Use a priori permutations perfet mathing an loing to avoi reaowns. Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 78 of 79

13 Stationary Methos Nonstationary Methos Nonstationary Methos Preonitioning Preonitioning Preonitioning Availale Coe SPAI, MSPAI, FSPAI availale in parallel version MPI, C/C++ SPAI University of Basel: SPAI, MSPAI TUM: FSPAI TUM: Puliations on SPAI, MSPAI, FSPAI availale on these wesites! Parallel Numeris, WT 7/8 5 Iterative Methos for Sparse Linear Systems of Equations page 79 of 79

14 Overview. ILU an LU fatorization for sparse triiagonal matries via fie point iteration an Newton s metho: A=LU S on pattern S.. Solving iiagonal an sparse triangular linear systems L via aelerate Jaoi metho as iret solver

15 Iterative Parallel Asynhronous ILU Chow Set initial values for L an U e.g. via Gauss-Seiel L an U For sweep,,. until onvergene parallel for i,j ϵ S if i > j then else enif enfor enfor,,,,, j j i j i j j j i u l a u l,,,, i j i j i j i u l a u p inepenent threas, istriute S on p threas. Always using new information if availale.

16 Convergene of Iterative ILU: The onvergene of iterative ILU is guarantee. Compare Jaoie for triangular matri. Using ILU for preonitioning, only a few asynhronous iteration steps are suffiient to gain a preonitioner lose to ILU. Convergene epens on the onition numer of A, resp. LU fators! Matri A ill-onitione LU shoul e ill-onitione slow onvergene! This hols espeially for iterative MILU an for eat sparse LU, e.g. for A triiagonal lie [-,,-]! Aelerating the onvergene y using Newton s metho:

17 Fie point iteration for Computing the Cholesy fatorization of a triiagonal matri: a a a a!

18 Orering of funtion g: 5,,,,,, a a a a g D J Derivative J of f, neessary for omputing net Newton step:!

19 Looing for the zero of the funtion f:,,,,,, a a a a f J Derivative J of f, neessary for omputing net Newton step:!

20 Newton Iteration for Computing the Inomplete Cholesy Fatorization of a General Sparse Matri: T! LL A S on sparse pattern S Let i,j ε S: f J f L l i, l j, a! i, j S,,..., nn i, j i, j L, i, S, j, S Looing for the zero of vetor funtion f with omponents f i,j = fj =. Orering the unnowns in L olumnwise: f=f,, f=f,, Unnowns fj = f i,j, J=,, nnztrila=nn for J nnz

21 Newton Step: g : J \ f, v new v ol g Upate as long as g is not small enough. J is - iiagonal for A triiagonal - sparse lower triangular for general sparse A

22 Biiagonal or sparse triangular Solves In general many prolems lea to - iiagonal linear systems e.g. triiagonal A in eigenvalue omputations - sparse triangular systems ILU an MILU, Gauss-Seiel, Newton for ILU,

23 Diret Solvers for iiagonal/sparse triangular matries: - Analyzing the graph for eteting parallel operations - Fatorization in sparse triangular fators - Sherman-Morrison-Wooury Restrite parallel effiieny! Looing for parallel iterative solver.

24 Iterative Solvers for iiagonal/sparse triangular matries: Jaoi iteration for L=: assume iagl=i I L L with iteration matri L Iteration matri L is lower triangular with iagl =, therefore guarantee onvergene. But again onvergene will e very slow in pratial ill-onitione eamples.

25 Aelerating onvergene y reasonale preonitioning: L P PL I P L P P PL L, y L y LP I y y LPy y Py LPy L Left preonitioning Right preonitioning Again Gauss-Seiel, ILU. o not mae sense as preonitioner! Blo Jaoi or approimate inverses are useful.

26 Sparse Approimate Inverse vs. Inomplete SAI F L S M S LM I min SAI: min L S M S e LM, min I e J M J I L LI,J: ISAI: LJ,J: LM-I S =

27 ISAI-Preonitioner P for L: LP = I S ISAI with pattern of L. P invl Better in parallel. Blo Jaoi Preonitioner L = P S iagonal los. P L has to e solve in every step. Less memory or epliit invert.

28 Aelerating onvergene y asynhronous upates: for j : n in parallel P L j j j, : enfor Eah upate always uses the newest information on availale. Preonitioning an asynhronous iterations reue the numer of sequential iterations replaing them y more effiient parallel omputations. But still the onvergene is too slow in most eamples!

29 Solving Biiagonal Systems Reursively: Oservation: The iteration matri L := I L is very sparse one iagonal only. Therefore, powers L, L, L 8, - an e ompute easily y repeate squaring, - are sparse one iagonal only, an - after logn steps they get zero., L, L, L 8 L

30 Solving Biiagonal Systems Reursively: L L I L L L L L L L L L L L L L L L L First approimation: Neumann series,,... Seon approimation: Stationary iteration with L ^

31 Neumann Series vs Euler Epansion Neumann series: L... L... Euler epansion:... I L... I L I L alreay use y van er Vorst,

32 Solving Biiagonal Systems Reursively: Allows fast omputation of the Neumann series L I L L L... via upating L an : enfor L L L L I n j for log :

33 Coe for Biiagonal Solver: while <logn p^+:n=p^+:n+:n-^.p:n-^; :n-^=:n-^.^+:n; Upate solution vetor Upate iteration matri en =+;

34 Reursive Coe: Upate an L reursively for =,,,8, with ISAI Preonitioner P an matri L: L=I-PL =P; while z < logn =L+I; L=LL; % ee=norml-; z=z+; en = ; L are the powers of the iteration matri an s are the onstant part in the iteration or the partial Neumann series.

35 Reursive Coe for general sparse L: Upate an L reursively for =,,,8, with ISAI Preonitioner P an matri L. If iteration matri gets to thi, swith to stanar stationary sheme: L=I-PL; =P; while ee>er && z<n if L is not too ense =L+I; L=LL; en =+L; ee=norml-; z=z+; en L are the powers of the iteration matri an are the onstant part in the iteration. Avoi too thi matries y stopping the reursion an swithing to stanar Jaoi.

36 Numerial Eample L = pentaiag-,-,,,, n=, ill onitione Level Density n n 5n 9n 7n n 65n 9n 57n 5n #Iter L = pentaiag-,-,,,, n=, well onitione Level 5 6 Density n n 5n 9n 7n n #Iter

37 Avantages - more ense operations matri-matri y preonitioning an L - faster onvergene - guarantee solution after logn steps - in the iiagonal ase only heap vetor sum - in the reusive stage first entry eat, then in the seon step - entries eat, then - entries eat, then -8 eat,. For general sparse traingular matri: Comine - preonitioner PL=I S, - reursive aeleration of Jaoi iteration upto thi iteration matri - asynhronous iteration with thi iteration matri

38 6. Domain Deomposition Consier ellipti PDE on region Ω with ounary Γ, e.g. with Dirihlet ounary onitions. Eample: in y g y u in y f y u u u u u yy,,, Ω Γ Parallel solution?

39 Overlapping DD Partition region Ω in two regions Ω an Ω, with new ounaries Γ an Γ whih are partially given y ol ounary Γ an some unnown parts Γ an Γ : Ω Γ Γ Ω Γ Γ Ω Γ

40 Overlapping DD II We an isretize an solve the given PDE on Ω with ounary Γ, ut we nee the values of u,y on the new artifiial ounary Γ. In a first step we an assume any values for Γ, e.g. u,y=. Then we an solve the linear system relative to region Ω. The same an e one in parallel for Ω. The values of the resulting solution on Γ an e use, to ompute in the net step the solution for region Ω with ounary Γ. In the same way we an use the resulting solution on Γ, to ompute the solution for region Ω with ounary Γ. So we an generate solutions on the partial regions whih provie us with approimate values for the unnown ounary of the other partial solution. The sequene of solutions onverges in eah region against the solution on Ω.

41 Overlapping DD III First step: Solve in parallel the PDE on all small suomains with ertain ounary on. Seon step: Ehange ounary values Repeat until onvergene. For getting etter interior ounary values: Solve whole prolem on oarse mesh an interpolate

42 Overlapping DD IV Matri representation of overlapping DD: First step: Seon step: Ω A A A Ω A A A Repeat until onvergene. Green parts are relate to the other omain an we asssume to now the relate omponents in the vetor of unnowns. They are move to the right-han-sie. 5

43 Nonoverlapping DD Partition region Ω Γ in the form Ω Γ Ω new Ω Γ Disretization of the original prolem with numering of the unnowns relative to the partitioning given y Ω an Ω leas to a linear system with a matri in issetion form: 6

44 7 Nonoverlapping DD II f f f u u u A G G F A F A Au f A is the so alle interfae matri. We an solve Au=f iteratively with pg an preonitioner: Here, for M we an use the ientity or an approimate inverse for the Shur omplement. Hene, we reue the original prolem to two partial suprolems an one interfae Shur omplement system. M A A

45 Nonoverlapping DD III Leas to 6 lo matries on the iagonal A,, A 6 an Shur omplement S. A A G A G 6 6 F F A 6 7, S A7 G A F G6A6F6 Overlapping easy parallel But slow onvergene Solve small prolems e.g. with Multigri in parallel. Preonition Shur omplement e.g. with MSPAI Nonoverlapping harer parallel, ut more influene on onvergene in S. 8

46 Solving the Shur Complement S A G A F G A F S S ense an large. So one shoul avoi omputing S epliitly! For iterative solver you have only to ompute S in every step goo. But also a preonitioner is neessary - a. MSPAI for S: Use SPAI for sparse approimation of A - Choose proing vetors e. an S y S. Solve: min S' M T e S I e T for preonitioner M 9

47 Multigri Starting point: Solve Partial Differential Equation, e.g. u u yy f, y with ounary onitions, e.g. Dirihlet BC. Disretization leas to system of linear equations, resp. matri A. A is sparse, struture, an ill-onitione. Looing for On solver. Diret solver On or On logn PCG also > On eause matri is ill-onitione

48 Iea Vetor of unnowns: Projet the fine isretization on a oarser smaller prolem Solve the oarse matri smaller matri Projet the oarse solution a on fine gri Prolems: - only possile for smooth vetor without high osillatory omponents - aprojetion introues high-osillatory errors that have to e remove, too.

49 Oservation: For typial PDE matries high-osillatory vetors are relate to the suspae to large eigenvalues an are remove e.g. y the stationary Gauss-Seiel iteration: I M - A small for eigenvetors to large eigenvalues! To solve A=f: Apply a few steps Gauss-Seiel smoothing steps a Resiual equation A a +=f A = f - A a = r Projet restrit r, resp. A on oarse gri y mean value Solve A =r Projet prolongate a on fine gri y interpolation new approimate solution a + Improve approimate solution y Gauss-Seiel steps Repeat until onvergene Apply also reursively for solving oarse equations

50 V-Cyle Fine gri, smoothing restrition smoothing restrition smoothing restrition smoothing restrition Fine gri, smoothing prolongation smoothing prolongation smoothing prolongation smoothing prolongation Diret solution of oarse resiual equation Repeat until onvergene

51 Parallel Aspets Ω Ω Ω Ω Eah proessor p r has ata for omain Ω r an oes projetions an smoothing for its omponents. Nees ata from neighoring proessor for projetion an GS. Loa alaning!

52 Ghost layers Jaoi smoothing: Eah proess performs Jaoi iteration inepenently Sen messages to upate ghost layers Use ommuniation/omputation overlap for interior omputations an ehange of ounary ata 5

53 Restrition/Prolongation Prolems with wor loa: On oarse gris less omputations, more ommuniation! Effiieny goes lie /logp with the numer of proessors. 6

54 Moifiations Agglomeration: Coarse gri partitions are no longer aligne with the finer gri partitions in orer to avoi ineffiieny on oarsest gris More ommuniation in gri transfer, less in oarse gri solve. Larger ghost layers an reue the ommuniation Reue numer of V-yle steps y using more powerful projetions an smoothers or aggressive oarsening less ata transfer 7

: L l A l P l f A Collet all matries from ifferent levels in one ig linear system an

55 Aitive Multigri Consier the ifferent levels at the same time, Compute the relate orretions in parallel an sum up the orretions. : L l A l P l f A Collet all matries from ifferent levels in one ig linear system an apply smoothing on eah level in parallel. A P, A P, A P, A.. P L, A L... L Hyri onjugate graient iteration with MG as preonitioner Less V-yles 8

Chapter 2: One-dimensional Steady State Conduction

Chapter 2: One-dimensional Steady State Conduction 1 Chapter : One-imensional Steay State Conution.1 Eamples of One-imensional Conution Eample.1: Plate with Energy Generation an Variable Conutivity Sine k is variable it must remain insie the ifferentiation