Programmg: Seedus ad Amdahl s law Mke Baley mjb@cs.oregostate.edu Orego State Uversty Orego State Uversty Comuter Grahcs seedus.ad.amdahls.law.tx Defto of Seedu 2 If you are usg rocessors, your Seedu s: Seedu T T where T s the executo tme o oe core ad T s the executo tme o cores. ote that Seedu should be >. Ad your Seedu Effcecy s: Effcecy Seedu Orego State Uversty Comuter Grahcs whch could be as hgh as., but robably ever wll be.
However, Multcore s ot a ree Luch: Amdahl s Law 3 If you ut rocessors, you should get tmes Seedu (ad 00% Seedu Effcecy), rght? Wrog! There are always some fracto of the total oerato that s heretly sequetal ad caot be arallelzed o matter what you do. Ths cludes readg data, settg u calculatos, cotrol logc, storg results, etc. If you thk of all the oeratos that a rogram eeds to do as beg dvded betwee a fracto that s arallelzable ad a fracto that s t (.e., s stuck at beg sequetal), the Amdahl s Law says: Seedu T T arallel arallel sequetal ( arallel ) Ths fracto ca be reduced by deloyg multle rocessors. Orego State Uversty Comuter Grahcs Ths fracto ca t. A Vsual Exlaato of Amdahl s Law 4 Executo Tme Sequetal Porto Porto Sequetal Porto Porto 2 4 # of Cores The Sequetal Porto does t go away, ad t also does t get ay smaller. It just gets more ad more domat. Sequetal Porto Porto Sequetal Porto ~ Orego State Uversty Comuter Grahcs 2
Amdahl s Law as a ucto of umber of Processors ad arallel 5 8.00 7.00 arallel : 90% 6.00 x Seedu 5.00 4.00 3.00 2.00 80% 60% 40%.00 0.00 3 5 7 9 3 5 7 9 2 23 25 27 29 # Processors 20% Orego State Uversty Comuter Grahcs 6 # rocessors X Seedu racto Orego State Uversty Comuter Grahcs 3
Amdahl s Law 7 ote that these fractos ut a uer boud o how much beeft you wll get from addg more rocessors: max Seedu lmseedu sequetal arallel Orego State Uversty Comuter Grahcs arallel maxseedu 0.00.00 0.0. 0.20.25 0.30.43 0.40.67 0.50 2.00 0.60 2.50 0.70 3.33 0.80 5.00 0.90 0.00 0.95 20.00 0.99 00.00 You ca also solve for arallel usg Amdahl s Law f you kow your seedu ad the umber of rocessors 8 Amdahl s law says: T S T ( ) ( ) S S T T T TT TT ( ) ( ) ( ) If you ve got several (,S) values, you ca take the average (whch s actually a least squares ft): T T, 2.. ( ) T 2 ( ) S Use ths f you Use ths f you kow Solvg for : kow the tmg the seedu T T T T T ( ) T T Seedu ote that whe =, T T 4
A More Otmstc Take o Amdahl s Law: Gustafso s Observato 9 Gustafso observed that as you crease the umber of rocessors, you have a tedecy to attack larger ad larger versos of the roblem. He also observed that whe you use the same arallel rogram o larger datasets, the arallel fracto,, creases. Let P be the amout of tme set o the arallel orto of a orgal task ad S set o the seral orto. The P P S Tme Seral Tme or P P S Wthout loss of geeralty, we ca set P= so that, really, S s ow a fracto of P. We ow have: S A More Otmstc Take o Amdahl s Law: Gustafso s Observato 0 We kow that f we multly the amout of data to rocess by, the the amout of arallel work becomes P. Surely the seral work must crease too, but we do t kow how much. Let s say t does t crease at all, so that we kow we are gettg a uer boud aswer. I that case, the ew arallel fracto s: Ad substtutg for P (=) ad for S, we have: ' P ' P P ' S P S ' S 5
A More Otmstc Take o Amdahl s Law: Gustafso s Observato If we tabulate ths, we get a table of values: Or, grahg t: A More Otmstc Take o Amdahl s Law: Gustafso s Observato 2 6
A More Otmstc Take o Amdahl s Law: Gustafso s Observato We ca also tur to a Maxmum Seedu: 3 7