Assignment 1 - Solutions. ECSE 420 Parallel Computing Fall November 2, 2014

Transcription

1 Aigmet - Solutio ECSE 420 Parallel Computig Fall 204 ovember 2, 204. (%) Decribe briefly the followig term, expoe their caue, ad work-aroud the idutry ha udertake to overcome their coequece: (i) Memory wall, (ii) Frequecy wall, ad (iii) Power wall. (i) Memory wall: CPU performace i the pat ha icreaed too fat for the matchig ytem memory, which yield memory which i below the ytem performace. Therefore thi create a bottleeck while a CPU i waitig for memory to be traferred to/from memory. To overcome thi iue, idutry ha itroduced cache (L, L2, L) betwee the CPU ad ytem memory which are much fater, however mall. Other method created are thig uch a out-of-order executio ad itructio-level parallelim. (ii) Frequecy wall: The frequecy wall i the limit at which CPU clock cycle ca be horteed. The reao for thi i the traitor iide the CPU have a defied maximum witchig time before heat diipatio a well a frequecy iterferece become a problem. Thee limit alo iclude uch thig a power coumptio ad the phyical ize of the traitor ad i relatio the chip) To overcome the frequecy wall, ytem have added ehaced coolig techique a well a delegatig multiple core of lower frequecy allowig for greater parallelim rather tha imply higher clock peed. (iii) Power wall: Thi term decribe CPU heat diipatio problem. A traitor icreae their frequecy, heat diipatio icreae. Approache to overcome the power wall i to ue ehaced coolig method. However thee coolig ytem ca ed up beig quite expeive. B: The above awer may ot cover all poibillitie, thu may differ lightly compared to your. A log a the mai idea for each problem i properly addreed ad decribed, your repoe i accepted a valid awer.

2 2. (%) A uiproceor applicatio i parallelized for 4 proceor, yieldig a.8 peedup. Give the time breakdow of the variou fuctio ee i the graph, what i the miimum total time that the uiproceor applicatio pet while Buy ad i performig Data Acce? Figure : Time breakdow of the variou fuctio Let T i be the total time ped per proceor P i. The peedup ca be expreed a: Speedup Sequetial Time Max(T 0, T, T 2, T ).8 Sequetial Time 60m Sequetial Time.8 60m = 608m. (20%) For a machie with the commuicatio overhead, ait occupacy ad etwork delay (meage tart-up time) of 200 ad the aymptotic peak badwidth of GB/, calculate the meage legth for reachig the /rd ad 2/rd of the peak badwidth. Aumig that the cotetio at thee two poit add 00 ad 00 to the tart-up time, repectively, what meage legth would the be obtaied? From lecture lide, the total Trafer time i: T () = T 0 + B 2

3 where i the meage legth, B i the aymptotic peak badwidth, ad T 0 the meage tart-up time. Let B be the actual badwidth of our ytem: B = T () = T 0 + B The actual badwidth i a fractio of the aymptotic peak badwidth B = λb. For the meage legth to be computed, we have: B = T 0 + B λb = T 0 + B λbt 0 + λ = ( λ) = λbt 0 = λ λ BT 0 Therefore, for the two aked ceario we have: /rd of the peak badwidth ad = 00 tart-up time: = BT 0 = 2 ( ) ( 0 9 byte ) = 20byte 2/rd of the peak badwidth ad = 700 tart-up time: = 2 2 BT 0 = 2 ( ) ( 0 9 byte ) = 7000byte B : Thee awer may differ lightly if you do a coverio with 024 for byte/bit tuff. Both are accepted a valid awer here. 4. (20%) Suppoe we have a program that ca be divided ito cocurret tak. Oe of thee tak require twice the executio time tha the other 4. (a) Baed o the Amdhal law compute the peed up (S) of uig proceor (oe proceor per tak) (b) Exted Amdahl law by takig ito coideratio the fixed overhead O i the commuicatio ad the etup of parallel procee. Derive a expreio for Amdahl boud that properly take the overhead ito accout. (c) Give thi ew expreio, which would be a more realitic value for the peed-up S computed above? (a) From the lecture lide, the peed-up S i: S = SP + ( SP )

4 where i the umber of proceor ad SP the equetial part of our program. Oe of our tak require twice the executio time (t = 2 T ) tha the other 4 (t = t 2 = t = t 4 = T ). The total time for thee tak to be executed equetially i t proceor = 6 T Uig the proceor, we ca execute the 4 tak ad the firt half of the th tak i parallel. I other word, the ecod half of the th tak i the part of our program that caot be executed i parallel: Thu, we have the aked peed-up: S = SP = T 6 T = ( 6 ) = (b) Takig the fixed overhead O ito coideratio, we have the ruig time of a program o proceor: T = SP T + ( SP ) T where i the umber of proceor,sp the equetial part of our program ad T the ruig time of the program o proceor. Thu, the peed-up i S = T T = SP + ( SP ) + O T = + O T SP T + ( SP ) T + O = SP + ( SP ) + O T ( ) lim = SP + O T We mut therefore keep O ad SP a low a poible i order to achieve maximal peedup a icreae. (c) Becaue of the fixed overhead O, a more realitic reult would be a maller value compared to the peed-up S we computed above.. (0%) Suppoe we have a machie with 00 core that acce a grid of ( = 0 0 ) elemet. Each oe of thee elemet i of double floatig poit preciio. We ca partitio the grid i two differet way: D - block of cotiguou colum 2D - Two-dimeioal domai decompoitio (blockig) 4

5 ote that i both cae we have equally ized partitio but the meage ize betwee adjacet partitio chage. The meage tart-up time i 4 u (We aume that it iclude all SW ad HW overhead, acceig the etwork iterface ad cro the etwork it ca be thought of a the time to ed the zero-legth meage). I additio, the aymptotic peak badwidth i 400 MB/. Compute the commuicatio overhead for each cae. Which oe of the two partitio would you elect, for thi total overhead to be reduced? We are aked to compute the total commuicatio overhead per cae: T () = T 0 + B where i the total meage legth, B i the aymptotic peak badwidth, ad T 0 the meage tart-up time. We therefore have to evaluate the total amout of data ( byte) that hould be commuicated betwee adjacet partitio. D - block of cotiguou colum: = 98 (2 8byte) + ( 8byte) + ( 8byte) = 98 (8 0 byte) So, we have the total trafer time: T () = 98 T 0 + B = (8 0 byte) byte T () = D - Two-dimeioal domai decompoitio (blockig): = 64 ( byte)+2 ( 0 4 8byte)+4 ( byte) = 60 (8 0 4 byte) So, we have the total trafer time: T () = 60 T 0 + B = (8 04 byte) byte T () = We ca eaily oberve that the ecod 2D partitio i the mot efficiet choice for the total commuicatio overhead to be reduced. B: Thee awer may differ lightly if you do ot multiply the umber of meage to be et by the meage tart-up time T 0. A log a the commuicatio cheme betwee adjacet partitio i properly decribed, both are accepted a valid awer here.