2.7.2 Limits to Parallelism

Transcription

1 Chapter 2 Exercie 53 The 1990 will find a broader ue of multiproceor a the peed of individual proceor reache the limit of metal interconnection. The highet utainable clock rate for metal interconnection i roughly 200 to 250 MHz for a typical conductor geometry, although the clock rate can be booted even higher at great expene by reducing the dimenion of all component and conductor. Computer in all clae from microproceor to high-end machine tarted the 1990 within one or two generation of thi clock limit. To utain increae in performance through the decade, the indutry mut embrace multiproceing in virtually all computer, or mut abandon metal interconnection technology for another technology uch a optical fiber or optical wave guide technology. 22 Today (1992), the high-range for microproceor clock peed i 150 MHz or a clock cycle of 6.7 nanoecond. For example, DEC new Alpha RISC microproceor run at a clock peed of 150 MHz. The Cray-3 upercomputer ha a clock cycle of 2 nanoecond or a 500 MHz clock. Uing gallium arenide (GaA) integrated circuit chip which are three time fater than ilicon, Seymour Cray wa able to puh the limit beyond Stone 250 MHz limit. However, we wonder how far the limit can be puhed Limit to Parallelim Since phyical contraint for calar proceor have been or will oon be reached, computer deigner mut reort to parallelim. One wonder what limit the degree of poible parallelim. The degree of parallelim i limited by operation whoe argument are dependent on the reult of other operation (data dependencie). What doe thi mean? We will explore thi iue with a erie of example on how fat one can add. We will aume we are adding integer repreented in b bit, e. g., 32 bit. For our deign, we will ue a full adder which i a circuit that perform the addition of two correponding bit, one from each argument. c c in Fig A 1-bit or Block Diagram The truth table for a full adder i hown in Figure 2.2. c in c Stone, Harold S., High-Performance Computer Architecture, 2nd edition, Addion-Weley, 1990, pp. x.

2 54 CHAPTER 2 MEASURING PERFORMANCE Fig Truth Table For a A full adder i decribed by the following two combinational logic equation: = c in c = (( ) c in ) + ( ) where +,, and are or, and and excluive or, repectively. We need not worry ab the detail of the inide of the full adder other than note that a good deign ha two level of gating which caue a time delay. If t d i the time delay through a logic gate, then the full adder take two t d to perform the operation. A erial adder will ue one full adder and will compute one bit of the reult every few clock cycle. Let aume we are adding the bit in an MD regiter to the Accumulator (AC regiter) and uing clocked circuit. ov MD Regiter AC Regiter c c in FF Carry Fig A Serial For a poible implementation of a erial adder, below i the peudo code for the hardware. 1. Set Carry flip flop to 0. Do b time 2. c and are reult of Add(,, c in ). Set right mot bit in AC regiter to ; Set Carry to c. 3. Circular hift MD and AC regiter one bit to right. End Do 4. Set overflow bit (ov) to Carry Since each numbered tep take a clock period, one clock period i needed to perform tep 1, and one clock period for tep 4. Step 2 and 3 are done b time and take 2*b clock period. t erial adder = 2*(b + 1)*t clock If we aume t clock, the clock period, i αt d, then

3 Chapter 2 Exercie 55 t erial adder = 2*(b + 1)*αt d To peed up the addition, we utilize parallelim by replicating the hardware to ue a full adder for each correponding bit of the argument. Below i hown the hardware for a 4-bit parallel adder. MD Regiter AC Regiter c c in c c in c c in c c in ov AC Regiter Fig A 4-bit Carry-ripple Parallel In Figure 2.14, we have hown the AC regiter twice to facilitate drawing the wire. Thi parallel adder i called a carry-ripple adder ince the carry ripple through the full adder. The peed of thi adder i dependent on how long it take the carry to propagate through all the full adder. The time i two gate delay per full adder for the carry to propagate, plu one additional clock period to update the accumulator. t carry-ripple = 2*b*t d + αt d What i the peedup of the parallel adder over the erial adder? Firt, we mut aign a value to α, the number of gate delay in a clock period. We will be conervative and allow a maximum of ix gate delay in a clock period. With α = 6 and b = 32 bit, we have the following: t erial adder = 2*(b + 1)*αt d = 2*(32 + 1)*6*t d = 396t d t carry-ripple = 2*b*t d + αt d = 2*32*t d + 6*t d = 70t d peedup = t erial adder t carry-ripple = 396t d 70t d = 5.66 Therefore, even though we had 32 time more hardware, we have only a peedup of A tated before, the degree of parallelim i limited by operation whoe argument are dependent on the reult of other operator. In the carry-ripple adder, each full adder i dependent on the reult of all the full adder to it right. The carry i propagated in a linear chain like a linear linked lit.

4 56 CHAPTER 2 MEASURING PERFORMANCE If we can horten the carry path, a fater adder can be contructed. From our experience in data tructure, we know we can earch a binary tree fater than a linear lit. Therefore, we will retructure the linear chain of carrie to anticipate each carry with a binary tree of logic gate. Oberve that each full adder c in i dependent on the and argument of all the full adder to the right. The carry look-ahead adder i baed on thee idea. For more detail on the carry lookahead adder conult a digital deign text uch a Hill and Peteron. 23 The time for a carry lookahead adder i given by the time to obtain the mot ignificant um bit, which i the height of the bigget binary tree or log 2 b multiplied by t cl, the gate delay of the carry lookahead gate; plu the time to travel two full adder; and plu a clock period to et the detination regiter. t carry look-ahead = 2*t fa + log 2 (b )*t cl + αt d A tated before t fa i 2*t d, and from analyzing the look ahead logic circuit, we would find t cl to be cloe to 2*t d. Therefore, the time for b = 32 bit i the following: t carry look-ahead = 4*t d + 2*log 2 (b )*t d + 6t d = 20t d and the peedup i : peedup = t erial adder t carry look-ahead = 396t d 20t d = 19.8 The behavior of the carry-ripple adder i ϑ(b) where b i the number of bit while the carry look-ahead adder i ϑ(log 2 b). Can we add even fater? Shmuel Winograd ha proven that the lower bound to add two b-bit binary number i the following: t lower bound add = log f 2*b * t where the bae of the logarithm i f the fan-in of the logic gate and t the delay time [Winograd, 1968]. If we aume only two input on our logic gate, i.e., f = 2, Winograd lower bound i the following: t lower bound add = log 2 2*b * t Since the theory of complexity lower bound proven by Winograd i ϑ(log 2 b), we are confident that the carry look-ahead i a fat adder. However, if we relax ome of Winograd aumption in hi proof, we can add even fater than hi lower bound! In 1961, Algirda Avizieni howed how to add in ϑ(1), independent of the number of bit [Avizieni, 1961]. He wa able to add fater btroducing redundanc the bit by uing a number repreentation called igned digit number. Unfortunately, igned digit number have everal undeirable characteritic, e. g., non-unique repreentation for a number, which make them impractical for ue in computer. By tudying how fat a computer can add, we have learned everal important leon. Firt, replicating the hardware n time did not necearily give the device a peedup of n. We aw that 23 Hill, Frederick J., and Gerald R. Peteron, Digital Sytem: Hardware Organization and Deign, 3rd ed., John Wiley and Son, 1987.

5 Chapter 2 Exercie 57 the degree of parallelim i limited by operation whoe argument are dependent on the reult of other operation (data dependencie). Therefore, our 32-bit carry-ripple adder wa only five time fater than the erial adder, even though we ued 32 time the hardware. Second, if we can retructure the olution to eliminate or reduce the data dependencie, we can exploit more parallelim and increae the peedup. By replacing the linear chain of carrie in the carry-ripple adder with anticipatory carry logic in the form of binary tree, we were able to increae the peedup from 5.66 to 19.8 for a 32-bit add. Therefore, for more parallelim, we look for a different way or algorithm to olve the ame problem. Third, beware of theory of complexity reult! Since they are general reult, they may milead you into believing you can t olve a problem fater. With your particular problem to be olved, the aumption may be relaxed and more parallelim and, therefore, better performance may be poible. If you can bound your problem, you may be able to compute fater than the theory of complexity reult indicate. Latly, if we can retructure the olution to have k independent computation, we can exploit the maximum parallelim. Since there i no inherent limit on how large k can be, we can obtain a peedup of k dependent only on our available reource. Reearcher have dubbed thee ituation embarraingly parallel.

6 58 CHAPTER 2 MEASURING PERFORMANCE Chapter 2 Exercie 1. What i MIPS? I it a good meaure of performance? Explain. 2. The Fantatic Computer ha 16 proceor. A proceor can perform two add and a multiply in each clock cycle. If the clock i 33 MHz, what i the theoretical peak performance of the Fantatic Computer in MFLOPS? 3. Prove that Hockney parameter r i 1/lope of the line of Figure 2.6 and that n 1/2 i the negative of the n-axi intercept. 4. Retrieve the Standard LINPACK benchmark off the Internet and execute it on everal computer to which you have acce. The benchmark i available by ending electronic mail to netlib@ornl.gov with the one line meage end linpack from benchmark. You will need to upply a FORTRAN function called SECOND which return the elaped CPU-time in econd. If a computer ha everal FORTRAN compiler, try them all. Alo, compare the MFLOPS rating for the different compiler option which optimize the code or ue a floating-point co-proceor. Write up your reult in a report. 5. Explore what it i available at netlib@ornl.gov by ending electronic mail to netlib@ornl.gov with the one line meage end index. After you have retrieved the index, requet everal file and explore their performance behavior on a computer available to you. 6. On a computer to which you have acce, run the Hockney benchmark program of Figure 2.7. If a computer ha everal FORTRAN compiler, try them all. Alo, compare the MFLOPS rating for the different compiler option which optimize the code, or ue a floating-point co-proceor. Write up your reult in a report which include a liting of your program, a liting of the timing data, a plot of the data, the determination of Hockney parameter r and n 1/2, and dicuion. 7. Aume that the fraction of equential code in a program i 1/k, how that the maximum peedup i k, i. e., prove Amdahl' law. 8. Draw the curve of peedup veru the number of proceor for Amdahl law when the equential code i the following: a.) 10% b.) 1%

7 Chapter 2 Exercie 59 Page left blank intentionally.