Pipeline Datapath. With some slides from: John Lazzaro and Dan Garcia

Transcription

1 Pipeline Datapath With some slides from: John Lazzaro and Dan Garcia

2 The single cycle CPU Instrction [25 ] Shift Jmp address [3 ] left PC+ [3 28] Instrction [3 26] Control RegDst Jmp Branch em emtoreg Op em Src Reg Shift left 2 reslt PC address Instrction [3 ] Instrction Instrction [25 2] Instrction [2 6] Instrction [5 ] register register 2 Registers 2 register Zero reslt ress Data Instrction [5 ] 6 Sign etend control Instrction [5 ]

3 lticycle implementation with Control PC ress emory emdata Instrction [25 2] Instrction [2 6] Instrction [5 ] Instrction register Instrction [5 ] emory register PCCond PC IorD Otpts em em emtoreg IR Control Op [5 ] Instrction [5 ] PCSorce Op SrcB 6 SrcA Reg RegDst Instrction [25 ] Shift left 2 Instrction [3-26] PC [3-28] register register 2 Registers register Sign etend 2 Shift left 2 A B 2 3 control Zero reslt Jmp address [3-] Ot 2 Instrction [5 ]

4 Advantages of mlticycle Save time: architectre instrctions take different nmber of clock cycles Save components (ths cost, power, heat): Can rese components for different parts of the cycle

5 Today: Pipelined architectre Better throghpt Improves overall delay/latency Better se of the hardware (ths power, heat) 5

6 Performance Eqation Seconds Program = Instrctions Program Cycles Instrction Seconds Cycle Goal is to optimize eection time, not individal eqation terms. achines are optimized with respect to program workloads. The CPI of the program. Reflects the program s instrction mi. Clock period. Optimize jointly with machine CPI. 6

7 Performance Eqation Seconds Program = Instrctions Program Cycles Instrction Seconds Cycle Goal is to optimize eection time, not individal eqation terms. achines are optimized with respect to program workloads. The CPI of the program. Reflects the program s instrction mi. Clock period. Optimize jointly with machine CPI. How to save in overall rntime? 7

8 Performance Eqation Seconds Program = Instrctions Program Cycles Instrction Seconds Cycle Goal is to optimize eection time, not individal eqation terms. achines are optimized with respect to program workloads. The CPI of the program. Reflects the program s instrction mi. Clock period. Optimize jointly with machine CPI. How to save in overall rntime? New idea: Pipelining 8

9 PC instrction registers Data מבנה ה - path rd rs rt + imm. Instrction Fetch 2. Decode/ Register 3. Eecte. emory 5. Back 9

10 Gotta Do Landry Ann, Brian, Cathy, Dave each have one load of clothes to wash, dry, fold, and pt away Washer takes 3 mintes A B C D Dryer takes 3 mintes Folder takes 3 mintes Stasher takes 3 mintes to pt clothes into drawers

11 Seqential Landry 6 P A T a s k O r d e r A B C D Time Seqential landry takes 8 hors for loads

12 Pipelined Landry 2 2 A 6 P T a s k O r d e r A B C D Pipelined landry takes 3.5 hors for loads! Time 2

13 General Definitions Latency: time to completely eecte a certain task for eample, time to read a sector from disk is disk access time or disk latency Throghpt: amont of work that can be done over a period of time 3

14 Pipelining Lessons (/2) T a s k O r d e r 6 P A B C D Time Pipelining doesn t help latency of single task, it helps throghpt of entire workload ltiple tasks operating simltaneosly sing different resorces Potential speedp = Nmber pipe stages Time to fill pipeline and time to drain it redces speedp: 2.3X v. X in this eample

15 Pipelining Lessons (2/2) T a s k O r d e r 6 P A B C D Time Sppose new Washer takes 2 mintes, new Stasher takes 2 mintes. How mch faster is pipeline? Pipeline rate limited by slowest pipeline stage Unbalanced lengths of pipe stages also redces speedp 5

16 Inspiration: Atomobile assembly line Assembly line moves on a steady clock. Each station does the same task on each car. The clock Car body shell erge station Bolting station Car chassis 6

17 Inspiration: Atomobile assembly line Simpler station tasks more cars per hor. Simple tasks take less time, clock is faster. 7

18 Inspiration: Atomobile assembly line Line speed limited by slowest task. ost efficient if all tasks take same time to do 8

19 Inspiration: Atomobile assembly line Simpler tasks, comple car long line! These lines go 2 7, and rarely sht down. Why? 9

20 Lessons from car assembly lines Faster line movement yields more cars per hor off the line. Faster line movement reqires more stages, each doing simpler tasks. To maimize efficiency, all stages shold take same amont of time (if not, workers in fast stages are idle) Filling, flshing, and stalling assembly line are all bad news. 2

21 PC instrction registers Data מבנה ה - path rd rs rt + imm. Instrction Fetch 2. Decode/ Register 3. Eecte. emory 5. Back 22

22 Pipelined Eection Representation Time IFtch Dcd Eec em IFtch Dcd Eec em IFtch Dcd Eec em IFtch Dcd Eec em IFtch Dcd Eec em IFtch Dcd Eec em Every instrction mst take same nmber of 23 steps, also called pipeline stages, so some will go idle sometimes

23 Key Analogy: The instrction is the car Pipeline Stage # Stage #2 Stage #3 Stage # Stage #5 Instrction Fetch IR IR IR IR Controls hardware in stage 2 Controls hardware in stage 3 Controls hardware in stage Controls hardware in stage 5 Data-stationary control 2

24 Representation #: Timeline IF (Fetch) ID (Decode) EX () E IR IR IR IR Good for visalizing pipeline fills. Sample Program I: I2: I3: I: I5: ADD R,R3,R2 AND R6,R5,R SUB R,R9,R8 XOR R3,R2,R OR R7,R6,R5 Time: t t2 t3 t t5 t6 t7 t8 Inst I: I2: I3: I: I5: I6: IF ID IF EX ID IF Pipeline is fll E EX ID IF E EX ID IF E EX ID IF E EX ID 25 E EX

25 Representation #2: Resorce Usage IF (Fetch) ID (Decode) EX () E IR IR IR IR Good for visalizing pipeline stalls. Sample Program I: I2: I3: I: I5: ADD R,R3,R2 AND R6,R5,R SUB R,R9,R8 XOR R3,R2,R OR R7,R6,R5 Time: t t2 t3 t t5 t6 t7 t8 Stage IF: ID: EX: E: : I I2 I I3 I2 I Pipeline is fll I I3 I2 I I5 I I3 I2 I I6 I5 I I3 I2 I7 I6 I5 I I3 26 I8 I7 I6 I5 I

26 I n s t r. O r d e r Graphical Pipeline Representation (In Reg, right half highlight read, left half write) Time (clock cycles) Load Store Sb Or I$ Reg I$ Reg I$ D$ Reg I$ Reg D$ Reg I$ Reg D$ Reg Reg D$ Reg D$ Reg 28

27 Eample Sppose 2 ns for access, 2 ns for operation, and ns for register file read or write Nonpipelined Eection: lw : IF + Reg + + emory + Reg = = 8 ns add: IF + Reg + + Reg = = 6 ns Pipelined Eection: a(if, Reg,,emory, Reg) = 2 ns 29

28 חלוקה לשלבים IF: Instrction fetch ID: Instrction decode/ register file read EX: Eecte/ address calclation E: emory access : back Shift left 2 reslt PC ress Instrction Instrction register register 2 Registers 2 register Zero reslt ress Data 6 Sign etend 3

29 Instrction הוספת הרגיסטרים IF/ID ID/EX EX/E E/ Shift left 2 reslt PC ress Instrction register register 2 Registers 2 register Zero reslt ress Data 6 Sign etend

30 I n s t r c t i o n l w I n s t r c t i o n f e t c h IF/ID I F / I D I D / E X E X / E E / W B A d d A d d A d d r e s l t S h i f t l e f t 2 P C A d d r e s s I n s t r c t i o n m e m o r y R e a d r e g i s t e r R e a d r e g i s t e r 2 R e g i s t e r s W r i t e r e g i s t e r W r i t e d a t a R e a d d a t a R e a d d a t a 2 A L U Z e r o A L U r e s l t A W r i t e d a t a R e a d d a t a 6 S i g n e t e n d

31 I n s t r c t i o n l w I F / I D I n s t r c t i o n d e c o d e ID/EX I D / E X E X / E E / W B A d d A d d A d d r e s l t S h i f t l e f t 2 P C A d d r e s s I n s t r c t i o n m e m o r y R e a d r e g i s t e r R e a d r e g i s t e r 2 R e g i s t e r s W r i t e r e g i s t e r W r i t e d a t a R e a d d a t a R e a d d a t a 2 A L U Z e r o A L U r e s l t A d d r e s s W r i t e d a t a D a t a m e m o r y R e a d d a t a 6 S i g n e t e n d 3 2 3

32 35 Instrction ress reslt Shift left 2 Instrction IF/ID EX/E PC Registers 2 register register 2 6 Sign etend register reslt Zero ID/EX E/ Eection lw ress Data EX/E

33 l w e m o r y E/ I F / I D I D / E X E X / E E / W B A d d A d d A d d r e s l t S h i f t l e f t 2 P C A d d r e s s I n s t r c t i o n m e m o r y I n s t r c t i o n R e a d r e g i s t e r R e a d r e g i s t e r 2 R e g i s t e r s W r i t e r e g i s t e r W r i t e d a t a R e a d d a t a R e a d d a t a 2 A L U Z e r o A L U r e s l t A d d r e s s W r i t e d a t a D a t a m e m o r y R e a d d a t a 6 S i g n e t e n d

34 I n s t r c t i o n l w W r i t e b a c k I F / I D I D / E X E X / E E / W B A d d A d d A d d r e s l t S h i f t l e f t 2 P C A d d r e s s I n s t r c t i o n m e m o r y R e a d r e g i s t e r R e a d r e g i s t e r 2 R e g i s t e r s W r i t e r e g i s t e r W r i t e d a t a R e a d d a t a R e a d d a t a 2 A L U Z e r o A L U r e s l t A d d r e s s D a t a W r i t e d a t a m e m o r y R e a d d a t a 6 S i g n e t e n d

35 Instrction תיקון!!! correction A IF/ID ID/EX EX/E E/ Shift left 2 reslt PC ress Instrction register register 2 Registers 2 register Zero reslt ress Data 6 Sign etend 38 Keep the right Rd all the way!

36 Instrction So here is the pdated CPU; IF/ID ID/EX EX/E E/ Shift left 2 reslt PC ress Instrction register register 2 Registers 2 register Zero reslt ress Data 6 Sign etend 39

37 Instrction What abot the control wires? PCSrc IF/ID ID/EX EX/E E/ Reg Shift left 2 reslt Branch PC ress Instrction register register 2 Registers 2 register Instrction [5 ] 6 Sign etend Src 6 control Zero reslt ress em Data em emtoreg Instrction [2 6] Instrction [5 ] RegDst Op

38 קווי הבקרה Eection/ress Calclation stage control lines emory access stage control lines -back stage control lines Instrction Reg Dst Op Op Src Branch em em Reg write em to Reg R-format lw sw X X beq X X Instrction Control EX 2 IF/ID ID/EX EX/E E/

39 Instrction emtoreg em Reg Datapath with Control PCSrc Control ID/EX EX/E E/ IF/ID EX PC ress Instrction register register 2 Registers register 2 Shift left 2 reslt Src Zero reslt Branch ress Data Instrction 6 [5 ] Sign etend 6 control em Instrction [2 6] Instrction [5 ] RegDst Op 3

40 דוגמא A demonstration of a seqence of instrctions: Lw $,2($) Sb $,$2,$3 And $2,$,$5 Or $3,$6,$7 $,$8,$9

41 Instrction emtoreg em Reg Instrction emtoreg em Reg IF: lw $, 2($) ID: before<> EX: before<2> E: before<3> : before<> IF/ID Control ID/EX EX EX/E E/ PC ress Instrction register register 2 Registers 2 register Shift left 2 reslt Src Zero reslt Branch ress Data Instrction [5 ] Sign etend control em Clock Instrction [2 6] Instrction [5 ] RegDst Op IF: sb $, $2, $3 ID: lw $, 2($) EX: before<> E: before<2> : before<3> IF/ID lw Control ID/EX EX EX/E E/ PC ress Instrction X 2 register register 2 Registers $X 2 register Instrction [5 ] Sign etend $ 2 Shift left 2 reslt control Src Zero reslt Branch ress Data em 5 Clock 2 X Instrction [2 6] Instrction [5 ] X RegDst Op

42 Instrction emtoreg em Reg Instrction emtoreg em Reg IF: and $2, $, $5 ID: sb $, $2, $3 EX: lw $,... E: before<> : before<2> IF/ID sb Control ID/EX EX EX/E E/ PC ress Instrction 2 3 register $2 register 2 Registers $3 2 register Shift left 2 $ reslt Src Zero reslt Branch ress Data X Instrction [5 ] Sign etend X 2 control em Clock 3 X Instrction [2 6] Instrction [5 ] X RegDst Op IF: or $3, $6, $7 ID: ID: and and $2, $2, $2, $, $3 $5 EX: sb $,... E: lw $,... : before<> IF/ID and Control ID/EX EX EX/E E/ PC ress Instrction 5 X Shift left 2 register $ $2 register 2 Registers $5 $3 2 register Instrction [5 ] Sign etend X reslt control Src Zero reslt Branch ress Data em 6 Clock X 2 Instrction [2 6] Instrction [5 ] X 2 RegDst Op

43 Instrction emtoreg em Reg Instrction emtoreg em Reg IF: add $, $8, $9 ID: or $3, $6, $7 EX: and $2,... E: sb $,... : lw $,... IF/ID or Control ID/EX EX EX/E E/ PC ress Instrction 6 7 register $6 register 2 Registers $7 2 register Shift left 2 $ $5 reslt Src Zero reslt Branch ress Data X Instrction [5 ] Sign etend X control em Clock 5 X 3 Instrction [2 6] Instrction [5 ] X 3 2 RegDst Op IF: after<> ID: add $, $8, $9 EX: or $3,... E: and $2,... : sb $,... IF/ID add Control ID/EX EX EX/E E/ PC ress Instrction 8 9 X register register 2 Registers $9 2 register Instrction [5 ] Sign etend $8 X Shift left 2 $6 $7 reslt control Src Zero reslt Branch ress Data em 7 Clock 6 X Instrction [2 6] Instrction [5 ] X 3 RegDst Op 2

44 The internal strctre of the Register File Rd reg (= Rs) 5 write Rd reg 2 (= Rt) 5 2 Wr reg (= Rd) 5 E Reg שתי היציאות קוראות בו זמנית ערכים של שני רגיסטרים שונים כותבים לאחד הרגיסטרים האחרים )בעליית השעון הבאה( 8

45 Instrction emtoreg em Reg Instrction emtoreg em Reg IF: after<2> ID: after<> EX: add $,... E: or $3,... : and $2,... IF/ID Control ID/EX EX EX/E E/ PC ress Instrction 2 register register 2 Registers 2 register Shift left 2 $8 $9 reslt Src Zero reslt Branch ress Data Instrction [5 ] Sign etend control em Clock 7 Instrction [2 6] Instrction [5 ] RegDst Op 3 2 IF: after<3> ID: after<2> EX: after<> E: add $,... : or $3,... IF/ID Control ID/EX EX EX/E E/ PC ress Instrction 3 register register 2 Registers 2 register Instrction [5 ] Sign etend Shift left 2 reslt control Src Zero reslt Branch ress Data em 9 Clock 8 Instrction [2 6] Instrction [5 ] RegDst Op 3

46 e m t o R e g e m W r i t e R e g W r i t e I F : a f t e r < > I D : a f t e r < 3 > E X : a f t e r < 2 > E : a f t e r < > W B : a d d $,... I F / I D C o n t r o l I D / E X W B E X E X / E W B E / W B W B A d d P C A d d r e s s I n s t r c t i o n m e m o r y I n s t r c t i o n R e a d r e g i s t e r R e a d R e a d d a t a r e g i s t e r 2 R e g i s t e r s R e a d d a t a 2 W r i t e r e g i s t e r W r i t e d a t a S h i f t l e f t 2 A d d A d d r e s l t A L U S r c Z e r o A L U A L U r e s l t B r a n c h A d d r e s s D a t a m e m o r y W r i t e d a t a R e a d d a t a I n s t r c t i o n [ 5 ] S i g n e t e n d A L U c o n t r o l e m R e a d C l o c k 9 I n s t r c t i o n [ 2 6 ] I n s t r c t i o n [ 5 ] R e g D s t A L U O p 5

47 Pipeline Hazard: atching socks in later load 2 2 A 6 P T a s k O r d e r A B C D E F bbble Time A depends on D; stall since folder tied p 5

48 Problems Limits to pipelining: Hazards prevent net instrction from eecting dring its designated clock cycle Strctral hazards: HW cannot spport this combination of instrctions (single person to fold and pt clothes away) Data hazards: Instrction depends on reslt of prior instrction still in the pipeline Control hazards: Pipelining of branches & other instrctions stall the pipeline ntil the hazard bbbles in the pipeline 52

49 An eample for hazards: sb $2, $, $3 and $2, $2, $5 or $3, $6, $2 add $, $2, $2 sw $5, ($2) 53

50 An eample for hazards: sb $2, $, $3 and $2, $2, $5 or $3, $6, $2 add $, $2, $2 sw $5, ($2) An eample for hazards: Register $2 is pdated only at the phase, i.e., the 5th clock cycle (actally at the end of the 5th clock cycle). However, we try to se it at the 3rd clock cycle when we read $2 at the decode phase of the and instrction 5

51 Graphic representation of hazards: Time (in clock cycles) Vale of register $2: Program eection order (in instrctions) sb $2, $, $3 CC CC 2 CC 3 CC CC 5 CC 6 I Reg CC 7 CC 8 CC 9 / D Reg and $2, $2, $5 I Reg D Reg or $3, $6, $2 I Reg D Reg add $, $2, $2 I Reg D Reg sw $5, ($2) I Reg D Reg 55

52 Solving hazards by adding nops sb $2, $, $3 nop nop nop and $2, $2, $5 or $3, $6, $2 add $, $2, $2 sw $5, ($2) 56

53 Solving hazards by adding nops P r o g r a m e e c t i o n o r d e r ( i n i n s t r c t i o n s ) s b $ 2, $, $ 3 T i m e ( i n c l o c k c y c l e s ) V a l e o f r e g i s t e r $ 2 : C C C C 2 C C 3 C C C C 5 C C 6 I R e g D R e g C C 7 C C 8 C C 9 / C C C C C C nop I R e g D R e g nop I R e g D R e g nop I R e g D R e g a n d $ 2, $ 2, $ 5 I R e g D R e g o r $ 3, $ 6, $ 2 I R e g D R e g a d d $, $ 2, $ 2 I R e g D R e g 57 s w $ 5, ( $ 2 ) I R e g D R e g

54 We cold earn ck cycle if GPR is transparent P r o g r a m e e c t i o n o r d e r ( i n i n s t r c t i o n s ) s b $ 2, $, $ 3 T i m e ( i n c l o c k c y c l e s ) V a l e o f r e g i s t e r $ 2 : C C C C 2 C C 3 C C C C 5 C C 6 I R e g D R e g C C 7 C C 8 C C 9 / C C C C C C nop I R e g D R e g nop I R e g D R e g a n d $ 2, $ 2, $ 5 I R e g D R e g o r $ 3, $ 6, $ 2 I R e g D R e g a d d $, $ 2, $ 2 We cold earn ck cycle if GPR is transparent, i.e, we cold see the write to the GPR at the GPR otpts (if the write address eqals the read address), i.e., dring Ck #5. I R e g D R e g s w $ 5, ( $ 2 ) I R e g D R g e 58

55 The internal strctre of the Register File Rd reg (= Rs) 5 write Rd reg 2 (= Rt) 5 2 Wr reg (= Rd) 5 E Reg שתי היציאות קוראות בו זמנית ערכים של שני רגיסטרים שונים כותבים לאחד הרגיסטרים האחרים )בעליית השעון הבאה( 59

56 The internal strctre of the modified Register File. We bypass the inpt (the write ) to the read otpt whenever Rs=Rd/Rt (i.e., whenever read reg=write reg bt not zero). We bypass the inpt (the write ) to the read 2 otpt whenever Rt=Rd/Rt (i.e., whenever read reg2=write reg, bt not zero). Rd reg (= Rs) 5 Wr reg 5 write write Rd reg 2 (= Rt) Wr reg 5 5 write 2 Wr reg (= Rd) 5 E Reg 6

57 After doing that change we only need 2 nops sb $2, $, $3 nop nop and $2, $2, $5 or $3, $6, $2 add $, $2, $2 sw $5, ($2) After the change the of an early instrction can happen at the same time with the read reg (decode) phase of a newer instrction (3 with two other instrctions in between). In case we have a hazard, we need to add only two nop instrctions. Unfortnately, this happens too often. We need a better soltion! 6

58 Time (in clock cycles) Vale of register $2: Program eection order (in instrctions) sb $2, $, $3 CC CC 2 CC 3 CC CC 5 CC 6 I Reg CC 7 CC 8 CC 9 / D Reg and $2, $2, $5 I Reg D Reg or $3, $6, $2 I Reg D Reg add $, $2, $2 I Reg D Reg sw $5, ($2) 62 I Reg D Reg

59 גניבת הערכים Forwarding Time (in clock cycles) CC CC 2 CC 3 CC CC 5 CC 6 CC 7 CC 8 CC 9 Vale of register $2 : / Vale of EX/E : X X X 2 X X X X X Vale of E/ : X X X X 2 X X X X Program eection order (in instrctions) sb $2, $, $3 I Reg D Reg and $2, $2, $5 I Reg D Reg or $3, $6, $2 I Reg D Reg add $, $2, $2 I Reg D Reg sw $5, ($2) I Reg D Reg 63

60 Forwarding (done at the eecte phase) Instrction ID/EX EX/E Control E/ IF/ID EX PC Instrction Registers Data IF/ID.RegisterRs Rs IF/ID.RegisterRt Rt IF/ID.RegisterRt IF/ID.RegisterRd Rt Rd EX/E.RegisterRd Forwarding nit E/.RegisterRd If ID/EX.Rs=EX/E.Rd, i.e., the Rd of the previos instrction eqals the Rs of the crrent instrction (which is in the decode phase), then we se the ot of the previos instrction instead of the otpt of the GPR. If ID/EX.Rs=E/.Rd, i.e., the Rd of the previos instrction eqals the Rs of the crrent instrction (which is in the decode phase), then we se the ot of the previos instrction instead of the otpt of the GPR. [ similarly, compare also ID/EX.Rt to E/.Rd ] 6 Similarly, compare also ID/EX.Rt to EX/E.Rd and to E/.Rd

61 Data hazard from previos instrction: Src A: If (ID/EX.Rs = = EX/E.Rd) se the Ot instead of Rs I.e., if Rs of the crrent eecting instrction = = Rd of the previos instrction The actal eqations are: if ((EX/E.Reg = = )&& (EX/E.Rd <> )&& (ID/EX.Rs = = EX/E.Rd)) => ForwardA=, Src B: If (ID/EX.Rt = = EX/E.Rd) se the Ot instead of Rt I.e., if Rt of the crrent eecting instrction = = Rd of the previos instrction The actal eqations are: if ((EX/E.Reg = = )&& (EX/E.Rd <> )&& (ID/EX.Rt = = EX/E.Rd)) => ForwardB=, 65

62 Data hazard from 2 instrctions back: Src A: If (ID/EX.Rs = = E/.Rd) se the GPR write instead of Rs I.e., if Rs of the crrent eecting instrction = = Rd of 2 instrctions ago The actal eqations are: if ((E/.Reg = = )&& (E/.Rd <> )&& (ID/EX.Rs = = E/.Rd)) => ForwardA=, Src B: If (ID/EX.Rt = = E/.Rd) se the GPR write instead of Rt I.e., if Rt of the crrent eecting instrction = = Rd of 2 instrctions ago The actal eqations are: if ((E/.Reg = = )&& (E/.Rd <> )&& (ID/EX.Rt = = E/.Rd)) => ForwardB=, Doble hazard: If there is a hazard from previos inst and the instrction before that?we shold chhose the from the previos instrction, it is p to date ( newer )! 66

63 דוגמא An eample for forwarding Sb $2, $, $3 And $, $2, $5 needs forwarding from the previos instrction Or $, $, $2 needs forwarding from two instrctions back $9, $, $2 needs forwarding from 3 instrctions back (thr the transparent GPR) Here we discss the $2 register only (The first two cases are handled in the eecte phase, the last one, in the decode phase). 67

64 דוגמא An eample for forwarding Sb $2, $, $3 And $, $2, $5 Or $, $, $2 needs forwarding from the previos instrction $9, $, $2 needs forwarding from the previos instrction Here we discss the $ register and there are two case (the 2nd one in prple) 68

65 Instrction Instrction or $, $, $2 and $, $2, $5 sb $2, $, $3 before<> before<2> ID/EX EX/E Control E/ IF/ID EX Sb $2, $, $3 And $, $2, $5 PC Clock 3 Or $, $, $2 Instrction add $9, $, $2 2 5 Registers $2 $5 2 5 $ $3 3 2 or $, $, $2 and $, $2, $5 ID/EX Forwarding nit sb $2,... Data before<> $9, $, $2 Control EX/E E/ IF/ID EX PC Instrction 6 Registers $ $2 $2 $5 Data Forwarding nit 2 69 Clock Since Rs=2 and Rd of previos inst. was 2, we se ot instead of Rs

66 Instrction Instrction after<> add $9, $, $2 or $, $, $2 and $,... sb $2,... ID/EX EX/E Control E/ IF/ID EX PC Instrction 2 2 Registers $ $2 $ $2 Data Forwarding nit Clock 5 after<2> after<> add $9, $, $2 or $,... and $,... ID/EX EX/E Control E/ IF/ID EX $ PC Instrction Registers $2 Data 2 9 Clock 6 Forwarding nit 7 In ble we see forwarding from two instrctions back (em->eec.), in red, from previos instrction (->Eec.), in prple, from 3 instrctions back (->Decode).

67 לא תמיד הפתרון עובד - lw The soltion does not work for (in lw we do not have the in the pipe!, it comes from the!) Program eection order (in instrctions) lw $2, 2($) Time (in clock cycles) CC CC 2 CC 3 CC CC 5 CC 6 I Reg D Reg CC 7 CC 8 CC 9 and $, $2, $5 I Reg D Reg or $8, $2, $6 I Reg D Reg add $9, $, $2 I Reg D Reg slt $, $6, $7 I Reg D Reg If the previos instrction was lw to a register and we try to se the register in the crrent instrction, we have a problem, since we cannot go back in time! One soltion is to avoid sch cases by adding a nop (by the Assembler) whenever Rt of the lw is eqal to Rs or Rt of the following instrction. 7

68 Another h/w soltion is to add Bbbles, i.e., add nop by hardware Program eection order (in instrctions) Time (in clock cycles) CC CC 2 CC 3 CC CC 5 CC 6 CC 7 CC 8 CC 9 CC lw $2, 2($) I Reg D Reg and $, $2, $5 I Reg Reg D Reg or $8, $2, $6 add $9, $, $2 slt $, $6, $7 I nop I Reg D Reg bbble I Reg D Reg I Reg D Reg 72 We need to hold IF/ID for one ck cycle and insert a nop: into ID/EX. This has the same effect as adding a nop instrction by the Assembler.

69 Instrction PC IF/ID Rs, Rt of crrent inst. IF/ID Hazard detection nit Hazard detection nit Control ID/EX.em ID/EX EX identifies lw EX/E E/ PC Instrction Registers Data IF/ID.RegisterRs Rt from prev. inst. IF/ID.RegisterRt IF/ID.RegisterRt IF/ID.RegisterRd ID/EX.RegisterRt Rt Rd Rs Rt Forwarding nit EX/E.RegisterRd E/.RegisterRd We need to hold the IF/ID and PC for one ck cycle and insert a nop: into ID/EX. This has the same effect as adding a nop instrction by the Assembler. If (ID/EX.emRd)&& ( (ID/EX.Rt= =IF/ID.Rs) (ID/EX.Rt= =IF/ID.Rt) ) we mst stall the pipeline. This means that prev. inst was lw and it was to the crrent Rs or Rt. (of corse if one of them is not sed, don t stall) 73 Holding means freeze the IF/ID and the PC for clock cycle Hold the IF/ID by not giving a IF/IDWrire signal and do not increment the PC (which already points at the ne instrction) by not giving the PC signal. Inserting a nop is by clearing all control signals.

70 דוגמא An eample for lw hazard detection lw $2, 2($) And $, $2, $5 Or $, $, $2 $9, $, $2 7

71 Instrction PC IF/ID Instrction PC IF/ID and $, $2, $5 lw $2, 2($) before<> before<2> IF/ID X Hazard detection nit Control ID/EX.em ID/EX EX EX/E E/ before<3> PC Instrction X Registers $ $X Data ID/EX.RegisterRt X 2 Forwarding nit Clock 2 or $, $, $2 and $, $2, $5 2 5 Hazard detection nit ID/EX.em ID/EX lw $2, 2($) EX/E before<> before<2> IF/ID Control EX E/ PC Instrction 2 5 Registers $2 $5 $ $X Data ID/EX.RegisterRt X Forwarding nit 75 Clock 3

72 Instrction PC IF/ID Instrction PC IF/ID or $, $, $2 IF/ID and $, $2, $5 2 5 Hazard detection nit Control ID/EX.em ID/EX EX bbble EX/E lw $2,... E/ before<> PC Instrction 2 5 Registers $2 $5 $2 $5 Data ID/EX.RegisterRt Forwarding nit 2 Clock add $9, $, $2 IF/ID or $, $, $2 2 Hazard detection nit Control ID/EX.em ID/EX EX and $, $2, $5 bbble lw $2,... EX/E E/ PC Instrction 2 2 Registers $ $2 $2 $5 Data ID/EX.RegisterRt Forwarding nit 2 76 Clock 5 The lw instrction is in the phase. $2 is being written. We can se $2 in the Eecte phase of the and instrction, with the help of forwarding.

73 Instrction PC IF/ID Instrction PC IF/ID after<> add $9, $, $2 or $, $, $2 and $,... bbble 2 Hazard detection nit ID/EX.em ID/EX EX/E IF/ID Control EX E/ PC Instrction 2 Registers $ $2 $ $2 Data 2 2 ID/EX.RegisterRt 9 Forwarding nit Clock 6 after<2> IF/ID after<> Hazard detection nit Control ID/EX.em ID/EX EX add $9, $, $2 or $,... and $,... EX/E E/ $ PC Instrction Registers $2 Data 2 ID/EX.RegisterRt 9 Forwarding nit 77 Clock 7

74 Branch Hazards 78

75 Instrction emtoreg em Reg Jst to remind s how branch is handled we show again the Datapath with Control PCSrc Control ID/EX EX/E E/ IF/ID EX PC ress Instrction register register 2 Registers 2 register Shift left 2 reslt Src Zero reslt Branch ress Data Instrction 6 [5 ] Sign etend 6 control em Instrction [2 6] Instrction [5 ] RegDst Op 79

76 Branch Hazards Program eection order (in instrctions) Time (in clock cycles) CC CC 2 CC 3 CC CC 5 CC 6 CC 7 CC 8 CC 9 beq $, $3, 7 I Reg D Reg and $2, $2, $5 I Reg D Reg 8 or $3, $6, $2 I Reg D Reg 52 add $, $2, $2 I Reg D Reg 72 lw $, 5($7) Here we calc.rs-rt I Reg D Reg These 3 instrctions shold be killed before they do harm, I.e., change any register. Here we decide to branch (switching the address to the PC and issing PC Cond) 8 In cc5 we already se the new PC calclated by the branch. (PC=72)

77 ontrol Hazard: Branching (/7) I n s t r. O r d e r beq Instr Instr 2 Instr 3 Instr Time (clock cycles) I$ Reg D$ Reg I$ Reg D$ Reg I$ Where do we do the compare for the branch? I$ Reg D$ Reg Reg D$ Reg I$ Reg D$ Reg 8

78 Control Hazard: Branching (2/7) We pt branch decision-making hardware in stage therefore two more instrctions after the branch will always be fetched, whether or not the branch is taken Desired fnctionality of a branch if we do not take the branch, don t waste any time and contine eecting normally if we take the branch, don t eecte any instrctions after the branch, jst go to the desired label 82

79 Control Hazard: Branching (3/7) Initial Soltion: Stall ntil decision is made insert no-op instrctions: those that accomplish nothing, jst take time Drawback: branches take 3 clock cycles each (assming comparator is pt in stage) 83

80 Control Hazard: Branching (/7) Optimization #: move asynchronos comparator p to Stage 2 as soon as instrction is decoded (Opcode identifies is as a branch), immediately make a decision and set the vale of the PC (if necessary) Benefit: since branch is complete in Stage 2, only one nnecessary instrction is fetched, so only one no-op is needed Side Note: This means that branches are idle in Stages 3, and 5. 8

81 85 PC Instrction Registers EX ID/EX EX/E E/ Data Hazard detection nit Forwarding nit IF.Flsh IF/ID Sign etend Control = Shift left 2

82 Control Hazard: Branching (5/7) I n s t r. O r d e r Insert a single no-op (bbble) add beq lw Time (clock cycles) I$ Reg D$ Reg I$ Reg D$ Reg bb ble I$ Reg D$ Reg Impact: 2 clock cycles per branch instrction slow 86

83 Control Hazard: Branching (6/7) Optimization #2: Redefine branches Old definition: if we take the branch, none of the instrctions after the branch get eected by accident New definition: whether or not we take the branch, the single instrction immediately following the branch gets eected (called the branch-delay slot) 87

84 Control Hazard: Branching (7/7) Notes on Branch-Delay Slot Worst-Case Scenario: can always pt a noop in the branch-delay slot Better Case: can find an instrction preceding the branch which can be placed in the branch-delay slot withot affecting flow of the program re-ordering instrctions is a common method of speeding p programs compiler mst be very smart in order to find instrctions to do this sally can find sch an instrction at least 5% 88 of the time Jmps also have a delay slot

85 Eample: Nondelayed vs. Delayed Branch Nondelayed Branch Delayed Branch or $8, $9,$ add $,$2,$3 add $,$2,$3 sb $, $5,$6 beq $, $, Eit or $, $,$ sb $, $5,$6 beq $, $, Eit or $8, $9,$ or $, $,$ Eit: Eit: 89

86 Qestion (/2) Assme instr/clock, delayed branch, 5 stage pipeline, forwarding, interlock on nresolved load hazards (after 3 loops, so pipeline fll) Loop: lw $t, ($s) add $t, $t, $s2 sw $t, ($s) addi $s, $s, - bne $s, $zero, Loop nop How many pipeline stages (clock cycles) per loop iteration to eecte this code?

87 Answer (/2) Assme instr/clock, delayed branch, 5 stage pipeline, forwarding, interlock on nresolved load hazards. 3 iterations, so pipeline fll. 2. ( hazard so stall) Loop:. lw $t, ($s) 3. add $t, $t, $s2. sw $t, ($s) 5. addi $s, $s, - 6. bne $s, $zero, Loop 7. nop (delayed branch so eec. nop) How many pipeline stages (clock cycles) per loop iteration to eecte this code?

88 Qestion (2/2) Assme instr/clock, delayed branch, 5 stage pipeline, forwarding, interlock on nresolved load hazards (after 3 loops, so pipeline fll). Rewrite this code to redce pipeline stages (clock cycles) per loop to as few as possible. Loop: lw $t, ($s) add $t, $t, $s2 sw $t, ($s) addi $s, $s, - bne $s, $zero, Loop nop How many pipeline stages (clock cycles) per loop iteration to eecte this code?

89 A (2/2) How long to eecte? Rewrite this code to redce clock cycles per loop to as few as possible: (no hazard since etra cycle) Loop:. lw $t, ($s) 2. addi $s, $s, - 3. add $t, $t, $s2 bne $s, $zero, Loop. (modified sw to pt past addi) 5. sw $t, +($s) How many pipeline stages (clock cycles) per loop iteration to eecte yor revised code? (assme pipeline is fll)

90 Peer Instrction A. Thanks to pipelining, I have redced the time it took me to wash my shirt. B. Longer pipelines are always a win (since less work per stage & a faster clock). C. We can rely on compilers to help s avoid hazards by reordering instrs. ABC : FFF 2: FFT 3: FTF : FTT 5: TFF 6: 9 TFT 7: TTF 8: TTT

91 Peer Instrction Answer A. Throghpt better, not eection time B. longer pipelines do sally mean faster clock, bt branches case problems! C. they happen too often & delay too long. Forwarding! (e.g, em ) F A L S E A. Thanks to pipelining, I have redced the time it took me to wash my shirt. B. Longer pipelines are always a win (since less work per stage F & a A faster L clock). S E C. We can rely on compilers to help s avoid hazards by reordering instrs. F A L S E ABC : FFF 2: FFT 3: FTF : FTT 5: TFF 6: 95 TFT 7: TTF 8: TTT

92 The sitation was better if we some how moved the branch address calclation one ck earlier. This is easy to do since sign etension and shift are only wires. We jst need to move the branch address register to the left. Rverything happens ck earlier and so we ll have to kill only two instrctions. Instrction Reg em emtoreg Net, we ll add a fast comparator which will compare Rs and Rt at the same ck cycle of the decode phase. (Instead of sing the to calc. Rs-Rt, we ll bilt a simple and fast or circit). This means etra h/w bt now we earned one more ck cycle. So, we have to kill only a single instrction. Killing an instrction also called flshing the pipeline, is easily done by clreaing the IF/ID register of the instrction following the branch (if the branch is sccessfl) PCSrc Control ID/EX EX/E E/ IF/ID EX PC ress Instrction register register 2 Registers 2 register Shift left 2 reslt Src Zero reslt Branch ress Data Instrction 6 [5 ] Sign etend Instrction [2 6] Instrction [5 ] 6 control RegDst Op em 96

93 97 Flshing PC Instrction Registers EX ID/EX EX/E E/ Data Hazard detection nit Forwarding nit IF.Flsh IF/ID Sign etend Control = Shift left 2

94 דוגמא An eample for flshing sb $, $, $8 beq $, $3, 7 and $2, $2, $5 lw $, 5($7) 98

95 Data hazards: * Forward from previos instrction * Forward from two instrctions ago Smmary of hazards * (Forward thr transparent GPR = from 3 instrctions ago) * If we cannot forward, (after lw) we stall the pipe by inserting a nop and freezing IF/ID and PC for ck cycle Control hazards: * If branch is sccessfl we flsh the instrction following the branch (which is at the IF/ID register. We jst clear the register) Notes: In the real IPS CPU, no flsh was employed. This give the compiler the opportnity to pt sefl instrctions following the branch. This eplains why the simlator always performs the instrction following the branch.this is called a delayed branch. Also, in the real IPS CPU no lw stall was sed. Again this give some freedom to the compiler to choose whether to pt a nop following lw or some sefl instrction. This is called a delayed load. 99

96 עוד פתרונות Sperpipelining means more than 5 stages of pipelining יותר שלבים Dynamic pipeline schedling change the order of eecting instrctions to fill gaps if possible (= instead of bbbles) מיקבול שלבים Sperscalar- Performing two instrctions simltaneosly. This means fetch two instrctions together, decode them at the same time(have more inpts and otpts in the GPR), eecte, i.e., almost doble the hardware ltithreading lticores