ECE 669 Parallel Computer Architecture

Transcription

1 ECE 669 Parallel Computer Architecture Lecture 4 Parallel Applicatios

2 Outlie Motivatig Problems (applicatio case studies) Classifyig problems Parallelizig applicatios Examiig tradeoffs Uderstadig commuicatio costs Remember: software ad commuicatio!

3 Simulatig Ocea Currets (a) Cross sectios (b) Spatial discretizatioof a cross sectio Model as two-dimesioal grids Discretize i space ad time fier spatial ad temporal resolutio => greater accuracy May differet computatios per time step - set up ad solve equatios Cocurrecy across ad withi grid computatios Static ad regular

4 Creatig a Parallel Program Pieces of the job: Idetify work that ca be doe i parallel - work icludes computatio, data access ad I/O Partitio work ad perhaps data amog processes Maage data access, commuicatio ad sychroizatio Simplificatio: How to represet big problem usig simple computatio ad commuicatio Idetifyig the limitig factor Later: balacig resources

5 4 Steps i Creatig a Parallel Program Partitioig D e c o m p o s i t i o A s s i g m e t p 0 p p 2 p 3 O r c h e s t r a t i o p 0 p p 2 p 3 M a p p i g P 0 P P 2 P 3 Sequetial computatio Tasks Processes Parallel Processors program Decompositio of computatio i tasks Assigmet of tasks to processes Orchestratio of data access, comm, sych Mappig processes to processors

6 Decompositio Idetify cocurrecy ad decide level at which to exploit it Break up computatio ito tasks to be divided amog processors Tasks may become available dyamically No of available tasks may vary with time Goal: Eough tasks to keep processors busy, but ot too may Number of tasks available at a time is upper boud o achievable speedup

7 Limited Cocurrecy: Amdahl s Law Most fudametal limitatio o parallel speedup If fractio s of seq executio is iheretly serial, speedup <= /s Example: 2-phase calculatio sweep over -by- grid ad do some idepedet computatio sweep agai ad add each value to global sum Time for first phase = 2 /p Secod phase serialized at global variable, so time = 2 Speedup <= 2 2 or at most p 2 Trick: divide secod phase ito two accumulate ito private sum durig sweep add per-process private sum ito global sum Parallel time is 2 /p + 2/p + p, ad speedup at best p 2

8 Uderstadig Amdahl s Law (a) 2 2 work doe cocurretly (b) p 2 /p 2 p (c) 2 /p 2 /p p Time

9 Cocurrecy Profiles,400,200,000 Cocurrecy Clock cycle umber Area uder curve is total work doe, or time with processor Horizotal extet is lower boud o time (ifiite processors) Speedup is the ratio: k= f k k=, base case: Amdahl s law applies to ay overhead, ot just limited cocurrecy f k k k p s + -s p

10 Applicatios Classes of problems Cotiuum Particle Graph, Combiatorial Goal: Demystifyig Differetial equatios ---> Parallel Program

11 Particle Problems Simulate the iteractios of may particles evolvig over time Computig forces is expesive Locality Methods take advatage of force law: G m m 2 r 2 Star o which for ces ar e beig computed Large gr oup far eough away to appr oximate Star too close to appr oximate Small gr oup far eough away to appr oximate to ceter of mass May time-steps, plety of cocurrecy across stars withi oe

12 Graph problems Travelig salesma Network flow Dyamic programmig Searchig, sortig, lists, Geerally ustructured

13 Cotiuous systems Hyperbolic Parabolic 2 A C 2 T 2 = 2 A + B A C T = 2 A + B Elliptic 0 = 2 A +B Laplace: B is zero Poisso: B is o-zero Examples: Heat diffusio Electrostatic potetial Electromagetic waves

14 Numerical solutios fiite differece methods fiite elemet methods Result i system of equatios Let s do fiite differece first Solve Discretize Form system of equatios Solve ---> Eg A T = 2 A x 2 Direct methods Idirect methods Iterative

15 Discretize Time Where Space st 2d A x Where x = A x A T = A + A t t = T steps A i + A i x x = = X grid poits Forward differece Space A ( A i + A i ) A i A i ( ) x 2 A Time Boudary coditios 2 A = A i + 2 A i + A i x 2 x 2 Ca use other discretizatios - Backward - Leap frog

16 D Case A T = 2 A x 2 + B A i + A i t = [ 2 A i + A i- ] + Bi x 2 A i + Or A i + = t [ A x 2 i + 2 A i + A i ]+ B i t + A i A x + A i + A 2 + A i + = 0 t x 2 2 t x 2 + t x 2 0 A x A i A 2 A i + B

17 Poisso s For 2 A x 2 + B = 0 i 0 = x 2 [ A i + 2A i + A i ] + B i Or 0 2 x 2 x 2 x 2 0 A 0 A Ai = B 0 B Bi A x = b

18 2-D case + i, j A A i, j t + Α i,j = t s 2 = A T = 2 A + x 2 s 2 2 A y 2 + B A i +, j + A i, j + A i, j + + [ A i, j 4 A i, j ]+ B i, j [ Α i +, j + Α i, j + Α i,j + + Α i,j 4Α i,j]+ B i,j t + Α i, j s s A2 A22 A A2 A3 [ A i, + j ] =? [ ][ Ai, j] + [ B i, j] What is the form of this matrix?

19 Curret status We saw how to set up a system of equatios How to solve them Poisso: Basic idea 0 = Or I iterative methods s 2 Iterate till o differece The ultimate parallel method Iterative Direct Jacobi, Multigrid [ A i +, j + A i, j + A i, j + + A i k, j 4 A i, j ] + B i, j A i, j = A i +,j + A i, j + A i, j + + Ai, j 4 k + A i, j = k i +, j k + C i, j A + A i, j + A i,j + + A i, j 4 k k = C i, j 0 for Laplace

20 I Matrix otatio Ax = b Set up a system of equatios Now, solve Direct: Iterative: Direct methods Semi-direct - CG Iterative Solve Ax=b directly LU Gaussia elim Recursive dbl Jacobi MG Ax = b = -Ax+b Mx = Mx - Ax + b Mx = (M - A) x + b Mx k+ = (M - A) x k + b Solve iteratively

21 Machie model Itercoectio etwork M M M P P P Data is distributed amog memories (igore iitial I/O costs) Commuicatio over etwork-explicit Processor ca compute oly o data i local memory To effect commuicatio, processor seds data to other ode (writes ito other memory)

22 Summary May types of parallel applicatios Attempt to specify as classes (graph, particle, cotiuum) We examie cotiuum problems as a series of fiite differeces Partitio i space ad time Distribute computatio to processors Uderstad processig ad commuicatio tradeoffs