Solving the Convection Diffusion Equation on Distributed Systems

Transcription

1 Solving the Convection Diffusion Equation on Distributed Systems N. Missirlis, F. Tzaferis, G. Karagiorgos, A. Theodorakos, A. Kontarinis, A. Konsta Department of Informatics and Telecommunications University of Athens

2 This presentation Considers the local MSOR Determines the optimum parameters. Parallel Implementation. Considers the Load Balancing Problem Results

3 The problem Assume a N N array of processors and assign a processor to each grid point. Jacobi is highly parallel but (local communication) SOR is difficult to parallelize T Needs multicoloring Adaptive use of needs global communication and results in same as Jacobi! = O( N) T ω = O( N)

4 The solution Use local iterative methods (Ehrlich, Botta, Russel) Use of for each grid point. Local communication ω T = O( N) Related research work Local SOR (Kuo et. Al. 1987) Local MSOR (Boukas and Missirlis 1997). Local SI-MEGS (Missirlis and Tzaferis 2002).

5 Introduction

6 Introduction ( Convection Diffusion Equation ) 2 2 u + u f( x, y) u g( x, y) u = x y x y (1) Discretization u = u + r u + t u + b u i 1, j i+ 1, j i, j+ 1 i, j 1 i = 1,2,..., N j = 1, 2,..., N. (2)

7 Introduction Jacobi Ax = b u = u + r u + t u + b u ( n+ 1) ( n) ( n) ( n) ( n) i 1, j i+ 1, j i, j+ 1 i, j 1 i, j point stencil i 1, j i, j i+ 1, j i, j 1

8 Ordering x x 1 ih u k 1 ih u k 0 jk 2 y 0 jk 2 y natural red-black

9 R/B ordering 5,0 5,1 5,2 5,3 5,4 5, ,0 4,1 4,2 4,3 4,4 4,5 Boundary points ,0 3,1 3,2 3,3 3,4 3, ,0 2,1 2,2 2,3 2,4 2, ,0 1,1 1,2 1,3 1,4 1,5 0,0 0,1 0,2 0,3 0,4 0,5

10 Local Modified ESOR u = (1 τ ) u + τ J u ( n+ 1) ( n) ( n) (i+j even) u = (1 τ ) u + ω Ju + ( τ ω ) Ju ( n 1) ( n) ( n 1) ( n) (i+j odd) where J u = u + ru + t u + b u ( n) ( n) ( n) ( n) ( n) i 1, j i+ 1, j i, j+ 1 i, j 1

11 Local Modified SOR u = (1 ω ) u + ω J u ( n+ 1) ( n) ( n) u = (1 ω ) u + ω J u ( n 1) ( n) ( n 1) (i+j even) (i+j odd)

12 Optimum Parameters J The Jacobi operator has real eigenvalues ω = ω1, i, j : = 2 2 µµ + µ µ 2 1 (1 )(1 ) ω = ω2, i, j : = µµ + µ µ 2 1 (1 )(1 )

13 Optimum Parameters J The Jacobi operator has imaginary eigenvalues ω = ω1, i, j : = 2 2 µµ + + µ + µ 2 1 (1 )(1 ) ω = ω2, i, j : = µµ + + µ + µ Optimum values for complex eigenvalues? Open Problem 2 1 (1 )(1 )

14 Optimum Parameters µ = If then 0 Local MSOR = Local SOR ω ω 2 = µ 2 2 = µ 2 real complex

15 Local Modified SOR Real I = ( 1)( 1) = ( ) S ω, ω ω1, i, j ω2, i, j 1, i, j 2, i, j Imaginary I = (1 )(1 ) = ( ) S ω, ω ω1, i, j ω2, i, j 1, i, j 2, i, j 1 µ 1 µ µ + 1 µ µ 1+ µ µ + 1+ µ 2 2

16 Local Modified SOR Conclusion Local MSOR will attain at least the convergence rate of local SOR, whereas its convergence rate will increase as µ increases.

17 Numerical Results The coefficients used in each problem are: f( x, y) = Re(2x 10), gxy (, ) = Re(2y 10) 3 2. f( x, y) = Re(2x 10), gxy (, ) = Re(2y 10) (i) (ii) µ max < 0.1 ( ) min max µ, µ [0.1,1.2] µ (iii) LMSOR outperforms all methods min 1

18 Numerical Results Comparison of local iterative methods for h=1/81, * indicates no convergence after iterations. # Method Re = 1 Re = 10 Re = 10 2 Re = 10 3 Re = 10 4 ( R, I ) (0,6400) (0,6400) (0,6400) (0,6400) (0,6400) 1 µ min µ max LSOR Nat * LSOR R/B * LMSOR

19 Numerical Results # Method Re = 1 Re = 10 Re = 10 2 Re = 10 3 Re = 10 4 ( R, I ) (6400,0) (6400,0) (0,6400) (0,6400) (0,6400) 2 µ min µ max LSOR Nat LSOR R/B LMSOR

20 Local Modified EGS The scheme for the Local Modified EGS is: u = (1 τ ) u + τ ( u + ru + t u + b u ) ( n+ 1) ( n) ( n) ( n) ( n) ( n) i 1, j i+ 1, j i, j+ 1 i, j 1 red points u = (1 τ ) u + ( u + ru + t u + b u ) + ( n+ 1) ( n) ( n+ 1) ( n+ 1) ( n+ 1) ( n+ 1) i 1, j i+ 1, j i, j+ 1 i, j 1 ( ) ( ) ( ) ( ) ( τ 1)( ui 1, j + ru i+ 1, j + tui, j+ 1 + bui, j 1) black points

21 LMEGS Optimum values Case Optimum τ Optimum ( ) τ S I τ, τ R1 µ < µ < µ µ 2 2(1 µ ) µ µ 2 2 µ µ µ µ R2 µ > µ > µ µ 2 2(1 µ ) µ µ 2 2 µ µ 2 2 µ + µ 2

22 LMEGS Optimum values Case Optimum τ Optimum ( ) τ S I τ, τ I1 µ µ µ + µ 2 2( µ + 1) µ + µ 2 2 µ µ µ + µ

23 Semi-Iterative LMEGS Let ( ), σ = S Iτ τ and the sequence (2) δ = 1 ( 2 δ : 1 0.5σ ) 1 (1) = ( ) ( k+ 1) 2 ( k) δ : = σ δ 1 Improvement: u : = δ u + (1 δ ) u ( k 1) ( k) ( k 1) ( k) ( k 1)

24 Numerical Results Comparison of LMSOR and SI-LMEGS: 2 f ( xy, ) = gxy (, ) = Re x N = 40 Re = 1 Re = 10 Re = 10 2 Re = 10 3 Re = 10 4 (R, I) R R I I I min max SI-LMEGS LMSOR

25 Numerical Results N = 120 Re = 1 Re = 10 Re = 10 2 Re = 10 3 Re = 10 4 (R, I) R R R I I min max SI-LMEGS LMSOR

26 Numerical Results Conclusions: If µ max < 3, SI-LMEGS is better than LMSOR

27 Parallel Implementation We divide the 2 points mesh into p equal rectangles of size N x N y. p p y N T p y p N T + 1 T + 2 x p x p T 2xp 2 T1 T2 T x p x p x

28 Parallel Implementation Two phases in each step: Phase 1: Communication of black boundary points. Phase 2: Communication of red boundary points. Computation and communication time for every process: 2 N tcomp ( Ti ) = O( N p) tcomm( Ti ) = 2 + xp N y p

29 Parallel Implementation

30 Communication Halo points Boundary points Interior poimts

31 Parallel Implementation Speed up: Efficiency: S E p p T p O T 1 ( ) p + y p + xp N 1 = = S 1 p = = O p 1+ ( y ) p + xp N S p p O( p) and E p 1 while 0. N

32 Parallel Implementation 0,006 0,005 Time Tp 0,004 0,003 0,002 0,001 N=60 N=84 N= Processors

33 Parallel Implementation 3,5 3 Speedup Sp 2,5 2 1,5 1 0, Processors

34 Parallel Implementation 1,2 1 Efficiency Ep 0,8 0,6 0,4 0,2 N=60 N=84 N= Processors

35 The Load Balancing Problem

36 The Diffusion Method ( ) ( n+ 1) ( n) ( n) ( n) u i = ui + u j ui [ECMWF Conf 00] j A () i Initial Load Load Transfer Load Balanced

37 Domain Decomposition Initial Decomposition Adjusted Nesting Nesting

38 Load Transfer Policy

39 The Load f(x) M f ( x) = M e x x d d 0 x 0 x

40 Numerical Results 6 processors Without LB With LB size Comm. Comp. Time Comm. Comp. Time Gain % Scale Factor(M) = 0, x20 1,321 14,710 16,031 2,371 14,186 16,557-3,28% 40x40 1,620 14,729 16,349 15,250 6,541 21,791-33,29% 80x80 8,819 38,036 46, ,029 9, , ,37% 100x100 13,097 65,018 78, ,644 13, , ,9 % Scale Factor(M) = 0,001 20x20 1,332 17,256 18,588 2,371 15,277 17,648 5,06% 40x40 1,623 17,896 19,519 15,267 5,832 21,099-8,09% 80x80 8,745 38,927 47, ,098 10, , ,13% 100x100 12,586 66,280 78, ,027 13, , ,84% Scale Factor(M) = 0, 1 20x20 1,298 21,451 22,749 2,369 14,331 16,7 26,59% 40x40 1,601 48,174 49,775 15,380 27,548 42,928 13,76% 80x80 8, , , ,143 54, ,566-19,46% 100x100 12, , ,895 70, ,469-43,03%

41 Numerical Results 6 processors Without LB With LB Size Comm. Comp. Time Comm Comp. Time Gain% Scale Factor(M) = 1 20x20 1, , ,205 2, , ,974 36,44% 40x40 1, , ,153 15, , ,674 38,43% 80x80 8, , ,28 107, , ,871 39,62% 100x100 12, , ,52 197, , ,252 32,6 % Scale Factor(M) = x20 1, , ,452 2, , ,739 37,72% 40x40 1, , ,764 15, , ,261 41,83% 80x80 8, , , , , ,923 49,52% 100x100 13, , , , , ,667 46,89% Scale Factor(M) = x20 1, , ,188 2, , ,174 37,73% 40x40 1, , ,698 15, , ,917 41,86% 80x80 8, , , , , ,228 49,63% 100x100 12, , , , , ,636 47,04%

42 Numerical Results Gain 2 processors 100,00% 50,00% 0,00% -50,00% -100,00% 20x20 40x40 80x80 100x100 0, ,001 0,

43 Numerical Results Gain 100,00% 4 processors 50,00% 0,00% -50,00% -100,00% 20x20 40x40 80x80 100x100 0, ,001 0,

44 Numerical Results Gain 6 processors 100,00% 50,00% 0,00% -50,00% -100,00% 20x20 40x40 80x80 100x100 0, ,001 0,

45 Numerical Results Gain 100,00% 50,00% 0,00% -50,00% -100,00% 20x20 40x40 80x80 100x100 0, ,001 0,

46 CONCLUSIONS Local MSOR is easy to parallelize Efficient S p O( p) and E p 1 p while 0. N It does worth to LB!

47 Future Work Determination of optimum values for the Local MESOR Dynamic LB for Solving the Adjusted Nesting Problem Heterogeneous, Asynchronous Load Balancing

48 Thank you for your attention Any Questions?