Parallel Programming: Speedups and Amdahl s law. Definition of Speedup

Transcription

1 Programmg: Seedus ad Amdahl s law Mke Baley mjb@cs.oregostate.edu Orego State Uversty Orego State Uversty Comuter Grahcs seedus.ad.amdahls.law.tx Defto of Seedu 2 If you are usg rocessors, your Seedu s: Seedu T T where T s the executo tme o oe core ad T s the executo tme o cores. ote that Seedu should be >. Ad your Seedu Effcecy s: Effcecy Seedu Orego State Uversty Comuter Grahcs whch could be as hgh as., but robably ever wll be.

2 However, Multcore s ot a ree Luch: Amdahl s Law 3 If you ut rocessors, you should get tmes Seedu (ad 00% Seedu Effcecy), rght? Wrog! There are always some fracto of the total oerato that s heretly sequetal ad caot be arallelzed o matter what you do. Ths cludes readg data, settg u calculatos, cotrol logc, storg results, etc. If you thk of all the oeratos that a rogram eeds to do as beg dvded betwee a fracto that s arallelzable ad a fracto that s t (.e., s stuck at beg sequetal), the Amdahl s Law says: Seedu T T arallel arallel sequetal ( arallel ) Ths fracto ca be reduced by deloyg multle rocessors. Orego State Uversty Comuter Grahcs Ths fracto ca t. A Vsual Exlaato of Amdahl s Law 4 Executo Tme Sequetal Porto Porto Sequetal Porto Porto 2 4 # of Cores The Sequetal Porto does t go away, ad t also does t get ay smaller. It just gets more ad more domat. Sequetal Porto Porto Sequetal Porto ~ Orego State Uversty Comuter Grahcs 2

3 Amdahl s Law as a ucto of umber of Processors ad arallel arallel : 90% 6.00 x Seedu % 60% 40% # Processors 20% Orego State Uversty Comuter Grahcs 6 # rocessors X Seedu racto Orego State Uversty Comuter Grahcs 3

4 Amdahl s Law 7 ote that these fractos ut a uer boud o how much beeft you wll get from addg more rocessors: max Seedu lmseedu sequetal arallel Orego State Uversty Comuter Grahcs arallel maxseedu You ca also solve for arallel usg Amdahl s Law f you kow your seedu ad the umber of rocessors 8 Amdahl s law says: T S T ( ) ( ) S S T T T TT TT ( ) ( ) ( ) If you ve got several (,S) values, you ca take the average (whch s actually a least squares ft): T T, 2.. ( ) T 2 ( ) S Use ths f you Use ths f you kow Solvg for : kow the tmg the seedu T T T T T ( ) T T Seedu ote that whe =, T T 4

5 A More Otmstc Take o Amdahl s Law: Gustafso s Observato 9 Gustafso observed that as you crease the umber of rocessors, you have a tedecy to attack larger ad larger versos of the roblem. He also observed that whe you use the same arallel rogram o larger datasets, the arallel fracto,, creases. Let P be the amout of tme set o the arallel orto of a orgal task ad S set o the seral orto. The P P S Tme Seral Tme or P P S Wthout loss of geeralty, we ca set P= so that, really, S s ow a fracto of P. We ow have: S A More Otmstc Take o Amdahl s Law: Gustafso s Observato 0 We kow that f we multly the amout of data to rocess by, the the amout of arallel work becomes P. Surely the seral work must crease too, but we do t kow how much. Let s say t does t crease at all, so that we kow we are gettg a uer boud aswer. I that case, the ew arallel fracto s: Ad substtutg for P (=) ad for S, we have: ' P ' P P ' S P S ' S 5

6 A More Otmstc Take o Amdahl s Law: Gustafso s Observato If we tabulate ths, we get a table of values: Or, grahg t: A More Otmstc Take o Amdahl s Law: Gustafso s Observato 2 6

7 A More Otmstc Take o Amdahl s Law: Gustafso s Observato We ca also tur to a Maxmum Seedu: 3 7