6 PM Midnight A B C D. Time. T a s k. O r d e r. Computer Architecture CTKing/TTHwang. Pipelining-1. Pipelining-3 CTKing/TTHwang

Transcription

1 CS: PP ii pp ee ll ii nn ii nn gg Otline d a t a t h P c D a t a h a z a s a D a t a h a z a s a s t a h h a z a s E c S c a a d y m An overview of pipelining A pipelined pa ipelined ont rol rd nd forwa rd ing rd nd lls Bra nc rd ept ions pers la r nd na ic pipelining Adapted from Prof. D. Patterson s c l ass notes C opy ri g h t 9 9 8, U C B Pipelining- Compter Architectre P ip elining I s N a tr a l! S eq entia l L a nd r y L a nd ry e a m ple: 6 P idnight Ann, Bria n, C a t h y, D a ve ea c h h a ve one loa d of c lot h es t o wa s h, d ry, a nd fold W a s h er t a k es 3 m in t es D ry er t a k es m in t es F old er t a k es m in t es Pipelining- A B C D Compter Architectre T a s k O r d e r A B C D Time S eq ent ia l la nd ry t a k es 6 h o rs for loa d s I f t h ey lea rned pipelining, h ow long wo ld it t a k e? Pipelining-3 Compter Architectre

2 P ip elined L a nd r y : S ta r t A S A P P ip elining L es s o ns T a s k O r d e r 6 P idnight Time 3 A B C D Pi pel i ned l a ndry tak es 3.5 h o rs for l oads T a s k O r d e r A B C D 6 P Time 3 D oes n t h elp la t enc y of s ingle t a s k, b t t h ro gh p t of ent ire P ipeline ra t e lim it ed b y s lowes t s t a ge lt iple t a s k s work ing a t s a m e t im e s ing d ifferent res o rc es P ot ent ia l s peed p = N m b er pipe s t a ges U nb a la nc ed s t a ge lengt h ; t im e t o fill & d ra in t h e pipeline red c e s peed p S t a ll for d epend enc es Pipelining- Compter Architectre Pipelining-5 Compter Architectre Clk Clk Cycle S ing le-, Single Cycle Implementation: ltiple Cycle Implementation: Load Ifetch Eec em Wr Pipeline Implementation: lti-c y c le, v s. P ip eline Cycle Cycle 3 Cycle Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle Load Ifetch Eec em Wr Store Store Ifetch Eec em Wr R-type Cycle Cycle Load Store Waste Ifetch Eec em Wr Pipelining-6 Ifetch Eec em R-type Ifetch Compter Architectre Program eection order Time (in instrctions) lw $, ($) lw $, ($) lw $3, 3($) Program eection Time order (in instrctions) lw $, ($) lw $, ($) lw $3, 3($) P ip elining Instrction fetch Instrction fetch ns 8 ns access I P S E ec tio n Pipelining-7 Instrction fetch 8 ns access 6 8 Instrction fetch ns Instrction fetch access access access ns ns ns ns ns Instrction fetch F ig ns... Compter Architectre

3 W I n s t r. O r d e r Inst Inst Inst Inst 3 Inst h y P ip eline? B ec a s e th e R es o r c es A r e T h er e! Time (clock cycles) Im Dm Im Dm Im Dm Im Dm Singlec y c le D a t a p a t h Im Dm Otline d a t a t h P c D a t a h a z a s a D a t a h a z a s a s t a h h a z a s E c S c a a d y m An overview of pipelining A pipelined pa ipelined ont rol rd nd forwa rd ing rd nd lls Bra nc rd ept ions pers la r nd na ic pipelining Pipelining-8 Compter Architectre Pipelining-9 Compter Architectre D es ig ning a P ip elined P r o c es s o r U s e ltic y c le E ec tio n S tep s E a m ine t h e d a t a pa t h a nd c ont rol d ia gra m St a r t ing w it h s ingle- o r m lt i-c y c le d a t a p a t h? Single- o r m lt i-c y c le c o nt r o l? P a rt it ion d a t a pa t h int o s t a ges : I F (ins t r c t io n f et c h ), I D (ins t r c t io n d ec o d e a nd r egis t er f ile r ea d ), E (e ec t io n o r a d d r es s c a lc la t io n), E (d a t a m em o r y a c c es s ), W B (w r it e b a c k ) As s oc ia t e res o rc es wit h s t a t es E ns re t h a t flows d o not c onflic t, or fig re o t h ow t o res olve As s ert c ont rol in a ppropria t e s t a ge Step name Instrction fetch Instrction decode/ fetch Action for R-type instrctions Action for -reference Action for instrctions branches IR = emory[] = + A = [IR[5-]] B = [IR[-6]] Ot = + (sign-etend (IR[5-]) << ) Action for jmps Eection, address Ot = A op B Ot = A + sign-etend if (A ==B) then = [3-8] II comptation, branch/ (IR[5-]) = Ot (IR[5-]<<) jmp completion emory access or R-type [IR[5-]] = Load: DR = emory[ot] completion Ot or Store: emory [Ot] = B emory read completion Load: [IR[-6]] = DR Bt, se single-c y c le d a ta p a th... Pipelining- Compter Architectre Pipelining- Compter Architectre

4 S p lit S ing le-c y c le D a ta p a th A d d P ip eline R eg is ter s IF: Instrction fetch ID: Instrction decode/ file read : Eecte/ address calclation E: emory access : back F eed b a c k P a t h P ipeline regis t ers ( la t c h es ) /E E/ reslt reslt left ress Instrction Instrction 6 etend 3 left reslt ress F ig ress Instrction F ig. 6. Instrction 6 etend 3 reslt ress What to add to split the path in to stag es? U s e regis t ers b et ween s t a ges t o c a rry d a t a a nd c ont rol Pipelining- Compter Architectre Pipelining-3 Compter Architectre C o ns id er load P ip elining load Cycle Cycle Cycle 3 Cycle Cycle 5 Clock Cycle Cycle Cycle 3 Cycle Cycle 5 Cycle 6 Cycle 7 Load Ifetch /Dec Eec em Wr I F I t c t F c h F c h t h t r c t f r o m t h I t r c t o r y : ns r ion et et e ins io n e ns io n em I D : I ns t r c t ion D ec od e is t er s f et c h a nd ins t r c t io n d ec o d e E : C a lc la t e t h e m em ory a d d res s E : Rea d t h e d a t a from t h e D a t a em ory W B: W rit e t h e d a t a b a c k t o t h e regis t er file st lw Ifetch /Dec Eec em Wr nd lw Ifetch /Dec Eec em Wr 3rd lw Ifetch /Dec Eec em Wr 5 f nc t iona l nit s in t h e pipeline d a t a pa t h a re: I ns t r c t io n em o r y f o r t h e I f et c h s t a ge is t er F ile s Rea d p o r t s (b s A a nd b s B ) f o r t h e / D ec s t a ge A L U f o r t h e E ec s t a ge D a t a em o r y f o r t h e E s t a ge is t er F ile s W r it e p o r t (b s W ) f o r t h e W B s t a ge Pipelining- Compter Architectre Pipelining-5 Compter Architectre

5 I F S ta g e o f load I D S ta g e o f load I R = m em [ P C ]; P C = P C + lw Instrction fetch I R, P C + F ig. 6. A = [ I R[ 5 - ]]; B = [ I R[ - 6 ]]; AL U o t = P C + ( s ign-e t ( I R[ 5 - ]) < < ) lw Instrction decode F ig. 6. /E E/ /E E/ reslt reslt left left ress Instrction Instrction 6 etend 3 reslt ress ress Instrction Instrction 6 etend 3 reslt ress Pipelining-6 Compter Architectre Pipelining-7 Compter Architectre E S ta g e o f load E S ta te o f load AL U o t = A + s ign-e t ( I R[ 5 - ]) lw Eection F ig D R = m em [ AL U o t ] lw emory /E E/ /E E/ reslt reslt left left ress Instrction Instrction reslt ress ress Instrction Instrction reslt ress 6 etend 3 F ig etend 3 Pipelining-8 Compter Architectre Pipelining-9 Compter Architectre

6 W W B S ta g e o f load T h e F o r S ta g es o f R -ty p e [ I R[ - 6 ]] = D R Cycle Cycle Cycle 3 Cycle ress Instrction Instrction h o will s pply t h is a d d res s? left reslt reslt /E ress E/ lw back R-type Ifetch /Dec Eec Wr I F c h t h t c t t h I t c t I D t c h a t c t d E U t t h t t s W U t t b a c k t t h t : fet e ins r ion from e ns r ion em ory : regis ers fet nd ins r ion ec od e : AL opera es on e wo regis er opera nd B: writ e AL o p o e regis er file F ig etend 3 Pipelining- Compter Architectre Pipelining- Compter Architectre P ip elining R -ty p e a nd load I m p o r ta nt Ob s er v a tio n Clock R-type Cycle Cycle Cycle 3 Cycle Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Ifetch /Dec Eec Wr R-type Ifetch /Dec Eec Wr Load Ifetch /Dec Eec em Wr R-type Ifetch /Dec Eec Wr R-type Ops! We have a problem! Ifetch /Dec Eec Wr W e h a ve a strctral hazard: T w o ins t r c t io ns t r y t o w r it e t o t h e r egis t er f ile a t t h e s a m e t im e! O nly o ne w r it e p o r t E a c h t c a b s t c t E a c h t m s t b s a t t h s a m s t a a t c t L o a d s t F w r p o r t d r s 5 t h s t a f nc iona l nit n only e ed onc e per ins r ion f nc iona l nit e ed e e ge for ll ins r ions : es is er ile s it e ing it ge Load 3 5 Ifetch /Dec Eec em Wr R-t y p e s es is t er F ile s w r it e p o r t d r ing it s t h s t a ge R-type 3 Ifetch /Dec Eec Wr S e v e ral w ay s to so lv e : f o rw ardi n g, addi n g p i p e li n e b b b le, m ak i n g i n strcti o n s sam e le n g th Pipelining- Compter Architectre Pipelining-3 Compter Architectre

7 S o ltio n: D ela y R -ty p e s W r ite T h e F o r S ta g es o f store Clock R-type D ela y R-t y pe s regis t er writ e b y one c y c le: R-t y p e a ls o s e F ile s w r it e p o r t a t St a ge 5 E is a N O P s t a ge: no t h ing is b eing d o ne. Cycle Cycle Cycle 3 Cycle Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Ifetch /Dec Eec em Wr R-type R-type 3 5 Ifetch /Dec Eec em Wr Ifetch /Dec Eec em Wr Store Cycle Cycle Cycle 3 Cycle Ifetch /Dec Eec em I F : fet c h t h e ins t r c t ion from t h e I ns t r c t ion em ory I D : regis t ers fet c h a nd ins t r c t ion d ec od e E : c a lc la t e t h e m em ory a d d res s E : writ e t h e d a t a int o t h e D a t a em ory Wr R-t y pe a ls o h a s 5 s t a ges Load Ifetch /Dec Eec em Wr R-type Ifetch /Dec Eec em Wr R-type Ifetch /Dec Eec em Wr Ad d a n e t ra s t a ge: W B: N O P Pipelining- Compter Architectre Pipelining-5 Compter Architectre T h e T h r ee S ta g es o f b eq P ip elined D a ta p a th Cycle Cycle Cycle 3 Cycle F ig Beq Ifetch /Dec Eec em Wr /E E/ I F : fet c h t h e ins t r c t ion from t h e I ns t r c t ion em ory I D : regis t ers fet c h a nd ins t r c t ion d ec od e E : c o m p a r es t h e t w o r egis t er o p er a nd s elec t c o r r ec t b r a nc h t a r get a d d r es s la t c h int o P C Ad d t wo e t ra s t a ges : E : N O P W B: N O P ress Instrction Instrction 6 etend 3 left reslt reslt ress Pipelining-6 Compter Architectre Pipelining-7 Compter Architectre

8 G r a p h ic a lly R ep r es enting P ip elines E a m p le : C y c le Program eection order (in instrctions) lw $, ($) Time (in clock cycles) CC CC CC 3 CC CC 5 CC 6 I D lw $, ($) Instrction fetch /E E/ reslt sb $, $, $3 I D left C a n h elp wit h a ns wering q es t ions lik e: H o w m a ny c y c les t o e ec t e t h is c o d e? W h a t is t h e A L U d o ing d r ing c y c le? H elp nd er s t a nd d a t a p a t h s ress Instrction F ig Clock Instrction 6 etend 3 reslt ress Pipelining-8 Compter Architectre Pipelining-9 Compter Architectre E a m p le : C y c le E a m p le : C y c le 3 sb $, $, $3 Instrction fetch lw $, ($) Instrction decode sb $, $, $3 Instrction decode lw $, ($) Eection /E E/ /E E/ left reslt left reslt ress Instrction F ig Clock Instrction 6 etend 3 reslt ress ress Instrction F ig Clock 3 Instrction 6 etend 3 reslt ress Pipelining-3 Compter Architectre Pipelining-3 Compter Architectre

9 E a m p le : C y c le E a m p le : C y c le 5 sb $, $, $3 Eection lw $, ($) emory sb $, $, $3 emory lw $, ($) back /E E/ /E E/ reslt reslt left left ress Instrction Instrction reslt ress ress Instrction Instrction reslt ress F ig etend 3 F ig etend 3 Clock Clock 5 Pipelining-3 Compter Architectre Pipelining-33 Compter Architectre E a m p le : C y c le 6 Otline ress Instrction Instrction left reslt reslt /E ress E/ sb $, $, $3 back d a t a t h P c D a t a h a z a s a D a t a h a z a s a s t a h h a z a s E c S c a a d y m An overview of pipelining A pipelined pa ipelined ont rol rd nd forwa rd ing rd nd lls Bra nc rd ept ions pers la r nd na ic pipelining F ig etend 3 Clock 6 Pipelining-3 Compter Architectre Pipelining-35 Compter Architectre

10 P ip eline C o ntr o l: C o ntr o l S ig na ls G r o p S ig na ls A c c o r d ing to S ta g es P CSrc C a n s e c ont rol s igna ls of s ingle-c y c le C P U ( F ig. 6. 3, 6. < = = > 5., 5. 6 ) A dd ress Instrction m em ory F ig. 6. ID / / E E /W B Instrction R ead R ead R egiste rs W rite re gister W rite da ta Instrction [5 ] Instrction [ 6] Instrction [5 ] W rite Rea d d ata Rea d d ata 6 3 etend S hift left A LU Src 6 c ontrol A LUO p reslt A LU reslt Branch ress em W rite m em ory em em tor eg Eection/ress Calclation stage control lines emory access stage control lines -back stage control lines Dst Op Op Src Branch em em write em to F ig Ds t Pipelining-36 Compter Architectre Pipelining-37 Compter Architectre Stationary P a s s c ont rol s igna ls a long j s t lik e t h e d a t a ain control generates control signals dring ID Stationary ( c ont. ) als for (E t O p, A L U Src,... ) are se d c y c le lat e r als for E ( e m W r, B ranc h ) are se d c y c le s lat e r als for W B ( e m t or e g, e m W r) are se d 3 c y c le s lat e r Instrction ID E /E E/ Pipelining-38 Fig. 6.6 Pipelining-39 Compter Architectre Compter Architectre ister ain EtOp Src Op Dst emwr Branch emto Wr ID/E ister EtOp Src Op Dst emwr Branch emto Wr E/E ister emw Branch emto Wr E/ ister emto Wr

11 p ath w ith L e t s T ry it O t Src C o ntrol ID/E W B / E W B E /W B lw $, ($) P C Ad dress Fig. 6.7 In str ctio n m em ory Instrction R ead R ead R eg iste rs W rite reg ister W rite d ata Instrction [5 ] Instrction [ 6] Instrction [5 ] R ea d d ata R ea d d ata 6 3 Sig n eten d left 6 A dd A LU con trol Re gd st reslt AL U Src AL U Op A LU re s lt B ran ch W rite em A ddre ss D a ta m em ory Rea d e m R ead W B emto sb $, $, $3 and $, $, $5 or $3, $6, $7 add $, $8, $9 Pipelining- Compter Architectre Pipelining- Compter Architectre E am p le : Cyc le E am p le : Cyc le IF: lw $, ($) ID: before<> : before<> E : before<3> W B: before<> IF: sb $, $, $3 ID: lw $, ($) : before<> E: before<> : before<3> /E E/W B lw /E E/W B A dd A dd ress Instrction Clock Instrction Instrction [5 ] In strction [ 6] In strction [ 5 ] d ata d ata eten d left control Dst reslt A LU Src Op reslt Bra nch ress da ta em em emto ress Instrction Clock Instrction Instrction [5 ] In strction [ 6] In strction [ 5 ] d ata d ata eten d $ $ left control Dst reslt A LU Src Op reslt Bra nch ress da ta em em emto Pipelining- Compter Architectre Pipelining-3 Compter Architectre

12 E am p le : Cyc le 3 E am p le : Cyc le IF: and $, $, $ 5 ID: sb $, $, $3 E : lw $,... E : before<> W B : before<> IF: or $3, $6, $7 ID: and $, $, $3 : sb $,... E: lw $,... W B: before<> IF /ID sb W B / E W B E/W B IF /ID and /E E/ re ss Instrction m em ory Instrction 3 W rite W rite $ $3 $ left Ad d reslt Src Z ero reslt Branch W rite em re ss mem ory emto re ss Instrction Instrction 5 left $ $ $5 $3 W rite W rite Ad d reslt Src reslt Branch re ss em memo ry emto Clock 3 Instrction [5 ] Instrction [ 6] Instrction [5 ] e tend c ontrol R egds t Op em Clock Instrction [5 ] Instrction [ 6] Instrction [5 ] etend control Dst Op em Pipelining- Compter Architectre Pipelining-5 Compter Architectre E am p le : Cyc le 5 E am p le : Cyc le 6 IF: add $, $8, $9 ID: or $3, $6, $7 : and $,... E: sb $,... : lw $,... IF: after<> ID: add $, $8, $9 : or $3,... E: and $,... : sb $,... or /E E/ add E /E E/ ress Instrction Instrction 6 7 R egisters $6 $7 $ $5 left reslt A LUSrc reslt Branch ress em emto ress Instrction Instrction 8 9 $8 $9 $6 $7 S hift left reslt A LUSrc reslt Branch em ress emto Instrction [5 ] etend control em Instrction [5 ] etend control em Clock 5 3 Instrction [ 6] Instrction [5 ] 3 Dst Op Clock 6 Instrction [ 6] Instrction [5 ] 3 Dst Op Pipelining-6 Compter Architectre Pipelining-7 Compter Architectre

13 E am p le : Cyc le 7 E am p le : Cyc le 8 IF: after<> ID: after<> : add $,... E: or $3,... W B: and $,... IF: after<3> ID: after<> : after<> E : add $,... W B: or $3,... /E E/ /E E/ ress Instrction Fig. 6.3 Clock 7 Instrction Instrction [5 ] Instrction [ 6] Instrction [5 ] etend $8 $9 left control Dst reslt Src Op reslt Branch em ress em 3 emto ress Instrction Fig. 6.3 Clock 8 Instrction 3 Instrction [5 ] Instrction [ 6] Instrction [5 ] etend left control Dst reslt Src Op reslt Branch ress em em 3 emto Pipelining-8 Compter Architectre Pipelining-9 Compter Architectre E am p le : Cyc le 9 S m m ary of P ip e line B as ic s IF: after<> ress Instrction Clock 9 ID: after<3> : after<> E: after<> : add $,... Instrction Instrction [5 ] Instrction [ 6] Instrction [5 ] etend left control Dst reslt Src Op reslt E /E Branch em ress em E/ emto P ip e l in in g is a f n d a m e n t a l c o n c e p t ltiple steps sing distinct resorces U tiliz e ca pa b ilities of da ta pa th b y pipelined instrction processing Start net i ns tr nc ti o n w h i l e w o rk i ng o n th e c rrent o ne L i m i ted b y l eng th o f l o ng es t s tag e ( p l s f i l l / f l s h ) N eed to d etec t and res o l v e h az ard s W h a t m a k e s it e a s y in I P S? A ll instrctions a re of th e sa m e length J st a f ew instrction f orm a ts em ory opera nds only in loa ds a nd stores W h a t m a k e s p ip e l in in g h a r d? h a z a r d s Pipelining-5 Compter Architectre Pipelining-5 Compter Architectre

14 O tline P ip e line H az ard s A n o v e r v w o f p e l A p e l e d d a t a p a t h P e l e d c o n t r o l D a t a h a z a r d s a n d f o r w a r d D a t a h a z a r d s a n d s t a l l s B r a n c h h a z a r d s E c e p t n s S p e r s c a l a r a n d d y n a m p e l ie ip in in g ip in ip in in g io ic ip in in g P ip e l in e H a z a r d s : S trctra l h a z a rds: a ttem pt to se th e sa m e resorce in tw o dif f erent w a y s a t th e sa m e tim e E. : c o m b i ned w as h er/ d ry er o r f o l d er b s y d o i ng s o m eth i ng el s e ( w atc h i ng T V ) D a ta h a z a rds: a ttem pt to se item b ef ore rea dy I ns tr c ti o n d ep end s o n res l t o f p ri o r i ns tr c ti o n s ti l l i n th e p i p el i ne C ontrol h a z a rds: a ttem pt to m a k e decision b ef ore condition is ev a la ted E. : w as h f o o tb al l ni f o rm s and need to s ee res l t o f p rev i o s l o ad to g et p ro p er d eterg ent l ev el B ranc h i ns tr c ti o ns C a n a l w a y s r e s o l v e h a z a r d s b y w a it in g pipeline control m st detect th e h a z a rd ta k e a ction ( or dela y a ction) to resolv e h a z a rds Pipelining-5 Compter Architectre Pipelining-53 Compter Architectre I n s t r. O r d e r Time Str c t ral H az ard : Sing le e m ory Load Instr Instr Instr 3 Instr em em em em em em em em em em Time (in clock cycles) Vale of $: Program eection order (in instrctions) sb $, $, $3 and $, $, $5 or $3, $6, $ add $, $, $ I I H az ard s CC CC CC 3 CC CC 5 CC 6 I D CC 7 CC 8 CC 9 / D D Fig I D U s e m e m o r y : d a t a m e m o r y a n d in s t r c t io n m e m o r y sw $5, ($) I D Pipelining-5 Compter Architectre Pipelining-55 Compter Architectre

15 T yp e s of H az ard s P ip e line H az ard s I ll s trate d T h r e e t y p e s : ( in s t. i f o l l o w e d b y in s t. i ) R A W ( r e a d a f t e r w r it e ) : i t r ie s t o r e a d o p e r a n d b e f o r e i w r it e s it W A R ( w r it e a f t e r r e a d ) : i t r ie s t o w r it e o p e r a n d b e f o r e i r e a d s it G ets w rong opera nd, e.g., a toincrem ent a ddr. C a n t h a ppen in I P S 5 -sta ge pipeline b eca se: A l l i ns tr c ti o ns tak e 5 s tag es, and read s are al w Pipelining-56 ay s i n s tag e, and w ri tes are al w ay s i n s tag e 5 W A W ( w r it e a f t e r w r it e ) : i t r ie s t o w r it e o p e r a n d b e f o r e i w r it e s it L ea v es w rong reslt ( i s not i s); occr only in pipelines th a t w rite in m ore th a n one sta ge C a n t h a ppen in I P S 5 -sta ge pipeline b eca se: A l l i ns tr c ti o ns tak e 5 s tag es, and w ri tes are al w ay s i n s tag e 5 Compter Architectre IF ID E Strctral Hazard IF ID E IF ID E Hazard IF ID. IF ID Pipelining-57 RAW (read after write) Hazard IF ID E IF ID E IF ID. IF ID E WAW Hazard (write after write) em WAR Hazard (write after read) Compter Architectre H and ling H az ard s U s e s p l e, f e d d e s s E W A R b y a a y f ( I D E W A W b y a w b a ( T h f h a v a w I S A I n t e r n a l f o r w a r d r e t e r f e : W f h a a h a R w h a w h a z a b a a D e t e c t a n d r e s o l v e r e m a o n e s C N O P S F a im i ign lim ina te lw s etch ing opera nds ea rly ) in pipeline lim ina te doing ll rite ck s in order la st sta ge, sta tic) ese ea tres e lot to do ith design in g in gis il rite in irst lf of clock nd rea d in second lf ea d deliv ers t is ritten, resolv e rd etw een sb nd dd in in g om piler inserts ta ll orw rd Sof tw are Sol tion H a v e c o m p il e r g a r a n t e e n o h a z a r d s W h e r e d o w e in s e r t t h e N O P s? sb $, $, $3 and $, $, $5 or $3, $6, $ add $, $, $ sw $5, ($) P r o b l e m : t h is r e a l l y s l o w s s d o w n! Pipelining-58 Compter Architectre Pipelining-59 Compter Architectre

16 Vale of $: Program eection order (in instrctions) Time (in clock cycles) sb $, $, $3 and $, $, $5 or $3, $6, $ I I H az ard s CC CC CC 3 CC CC 5 CC 6 D Fig I D CC 7 CC 8 CC 9 / D De te c ting H az ard s H a z a r d c o n d it io n s : a. E / E.R egisterr d = I D / E.R egisterr s b. E / E.R egisterr d = I D / E.R egisterr t a. E / W B.R egisterr d = I D / E.R egisterr s b. E / W B.R egisterr d = I D / E.R egisterr t T w o o p t im iz a t io n s : D on t f orw a rd if instrction does not w rite = > ch eck if R egw rite is a sserted D on t f orw a rd if destina tion is $ = > ch eck if R egisterr d = add $, $, $ I D sw $5, ($) I D Pipelining-6 Compter Architectre Pipelining-6 Compter Architectre De te c ting H az ard s ( c ont. ) R e s olv ing H az ard s : F orw ard ing H a z a r d c o n d it io n s s in g c o n t r o l s ign a l s : A t E sta ge: E / E.R egw rite a nd ( E / E.R egr d ) a nd ( E / E.R egr d= I D / E.R egr s) A t E sta ge: E / W B.R egw rite a nd ( E / W B.R egr d ) a nd ( E / W B.R egr d= I D / E.R egr s) ( repla ce I D / E.R egr t f or I D / E.R egr s f or th e oth er tw conditions) o U s e t e m p o r a r y r e s l t s, e.g., t h o s e in p ip e l in e r e gis t e r s, d o n t w a it f o r t h e m t o b e w r it t e n P r og ra m e e c tio n o rd e r (in instrctions) s b $, $, $ 3 a n d $, $, $ 5 Time (in clock cycles ) C C C C C C 3 C C C C 5 C C 6 I R e g I R e g C C 7 C C 8 C C 9 V a le o f re g ister $ : / Vale of / E : V a l e o f E /W B : R e g D R e g D Fig o r $ 3, $ 6, $ I R e g D R e g a d d $, $, $ I R e g D R e g s w $ 5, ($ ) I R e g D R e g Pipelining-6 Compter Architectre Pipelining-63 Compter Architectre

17 F orw ard ing L og ic P ip e line w ith F orw ard ing Fo r w a r d in g: in p t t o A L U f r o m a n y p ip e r e g. A dd m ltiple ors to A L U inpt C ontrol f orw a rding in E = > ca rry R s in I D / E C o n t r o l s ign a l s f o r f o r w a r d in g: I f b oth W B a nd E f orw a rd, e.g., add $,$,$; add $,$,$3; add $,$,$; = > let E f orw a rd E h a z a rd: if (/E. and (/E.Rd ) and (/E.Rd=.Rs)) ForwardA= E h a z a rd: if (E/. and (E/.Rd ) and (/E.Rd..Rs) and (E/.Rd=.Rs)) ForwardA= (I D / E. R e g R t <->I D / E. R e g R s, F o r w a r d B <-> F o r w a r d A ) Instrction Instrction.isterRs.isterRt.isterRt.isterRd Rs Rt Rt Rd nit /E ForwardA ForwardB Fig. 6.3 /E.isterRd E/.isterRd E/ Pipelining-6 Compter Architectre Pipelining-65 Compter Architectre E am p le 3 : Cyc le 3 E am p le 3 : Cyc le or $, $, $ and $, $, $5 sb $, $, $3 before<> before<> add $9, $, $ or $, $, $ and $, $, $5 sb $,... before<> /E /E E/ E/ Instrction Instrction 5 $ $5 $ $3 Instrction Instrction 6 $ $ $ $ nit Fig. 6. nit Clock 3 Clock Pipelining-66 Compter Architectre Pipelining-67 Compter Architectre

18 E am p le 3 : Cyc le 5 E am p le 3 : Cyc le 6 after<> add $9, $, $ or $, $, $ and $,... sb $,... after<> after<> add $9, $, $ or $,... and $,... /E /E E/ E/ $ $ $ Instrction Instrction $ $ Instrction Instrction $ Fig nit Fig nit Clock 5 Clock 6 Pipelining-68 Compter Architectre Pipelining-69 Compter Architectre Can' t A lw ays F orw ard Stalling lw c a n s t il l c a s e a h a z a r d : if is f ollow ed b y a n instrction to rea d th e loa ded reg. (in instrctions) lw $, ($) I D S t a l l p ip e l in e b y k e e p in g in s t r c t io n s in s a m e s t a ge a n d in s e r t in g a n N O P in s t e a d and $, $, $5 I D Fig. 6.3 Fig or $8, $, $6 I D add $9, $, $ slt $, $6, $7 U s e s t a l l in g o r c o m p il e r t o r e s o l v e I D I D Pipelining-7 Compter Architectre Pipelining-7 Compter Architectre

19 H and ling Stalls P ip e line w ith Stalling U nit H a z a r d d e t e c t n n I D t o s e r t s t a l l b e t w e e n a l o a d s t r c t n a n d s s e : H o w t o s t a l l S I F a I D : a P C a I F / I D = > W h a m E : a N O P b y a E, W B f I D / E W B d m w io it in in in io it if (.em and ((.isterrt =.isterrs) or (.isterrt =.Rt)) stall the pipeline for one cycle (.em= indicates a load instrction)? ta ll instrction in nd not ch nge nd th e sta ges re-e ecte th e instrctions t to ov e into insert n ch nging E, control ields of pipeline to as control signals propagate, all control signals to, E, are easserted and no s or em ories are ritten Fo r w a r d in g c o n t r o l s A L U in p t s, h a z a r d d e t e c t io n c o n t r o l s P C, I F/ I D, c o n t r o l s ign a l s Instrction Instrction Hazard detection nit.isterrs.isterrt.isterrt.isterrd ID /.isterrt.em W B Rt Rd Rs Rt nit /E /E.isterRd E/.isterRd Fig E / Pipelining-7 Compter Architectre Pipelining-73 Compter Architectre E am p le : Cyc le E am p le : Cyc le 3 and $, $, $5 lw $, ($) before<> before<> before<3> or $, $, $ and $, $, $5 lw $, ($) before<> before<> Hazard detection nit.em /E E/ 5 Hazard detection nit.em /E E/ Instrction Instrction $ $ Instrction Instrction 5 $ $5 $ $ 5.isterRt nit.isterrt nit Clock Clock 3 Pipelining-7 Compter Architectre Pipelining-75 Compter Architectre

20 E am p le : Cyc le Eample : Cycle 5 or $, $, $ and $, $, $5 Hazard detection nit 5.em bbble /E lw $,... E/ before<> add $9, $, $ or $, $, $ Hazard detection nit.em and $, $, $5 bbble lw $,... /E E/ Instrction Instrction 5 $ $5 $ $5 Instrction Instrction $ $ $ $ isterRt nit.isterrt nit Clock Clock 5 Pipelining-76 Compter Architectre Pipelining-77 Compter Architectre Eample : Cycle 6 Eample : Cycle 7 after<> add $9, $, $ or $, $, $ and $,... bbble after<> after<> add $9, $, $ or $,... and $,... Hazard detection nit.em /E E/ Hazard detection nit.em /E E/ Instrction Instrction $ $ $ $ Instrction Instrction $ $ Fig. 6.9.isterRt 9 nit Fig. 6.9.isterRt 9 nit Clock 6 Clock 7 Pipelining-78 Compter Architectre Pipelining-79 Compter Architectre

21 O t li n e Pipeline D a t a pa t h w it h C o nt r o l S ig na ls A n o v e r v w o f p e l A p e l e d d a t a p a t h P e l e d c o n t r o l D a t a h a z a r d s a n d f o r w a r d D a t a h a z a r d s a n d s t a l l s B r a n c h h a z a r d s E c e p t n s S p e r s c a l a r a n d d y n a m p e l ie ip in in g ip in ip in in g io ic ip in in g A dd ress Instrction m em ory Fig. 6. ID / / E E /W B Instrction R ead R ead R egiste rs W rite re gister W rite da ta Instrction [5 ] Instrction [ 6] Instrction [5 ] W rite Rea d d ata Rea d d ata 6 3 etend S hift left A LU Src 6 c ontrol A LUO p reslt A LU reslt Branch ress P CSrc em W rite m em ory em em tor eg Ds t Pipelining-8 Compter Architectre Pipelining-8 Compter Architectre B r an ch H az ar d s H an d li n g B r an ch H az ar d W h e n d e c id e t o b r a n c h, o t h e r in s t. a r e in p ip e l in e! order (in instrctions) beq $, $3, 7 and $, $, $5 8 or $3, $6, $ 5 add $, $, $ 7 lw $, 5($7) I I I D I D I D D Fig D P r e d t b r a n c h a l w a y s n o t t a k e n B m E = > I F, I D, E b y c c v R e d c e d e l a y o f t a k e n b r a n c h b y m o v b r a n c h e e c t n e a r l r t h e p e l e b c I D C k b I D ( O R ) b y c p I D B m E = > A c I F z I F / I D = > m P D y n a m b r a n c h p r e d t n C o m p e r r e s c h e d l d e l a y b r a n c h ic Need to add hardware for flshing inst. if wrong ranc h dec ision ade at need to flsh instrc tion in hanging ontrol ales to in g io ie in ip in ov e p ranc h address alc lation to hec ranc h eq ality at sing om aring the two s read dring ranc h dec ision ade at one instrc tion to flsh dd a ontrol signal,.f lsh, to ero instrc tion field of ak ing the instrc tion an NO ic ic io il in g, Pipelining-8 Compter Architectre Pipelining-83 Compter Architectre

22 P i peli n e w i t h F l s h i n g Eample 5: Cycle 3 IF.Flsh and $, $, $5 b eq $, $3, 7 s b $, $, $8 before<> b efore<> Hazard detection nit /E Fig. 6. E/ IF.F l sh 7 8 Hazard detection n it Con trol 8 ID / W B /E E/W B Instrction left = 8 In strc tion 7 mem ory S hift left 7 7 Re gisters = $ $3 $ $8 mem o ry etend eten d nit Fig F orwa rd ing nit Clock 3 Pipelining-8 Compter Architectre Pipelining-85 Compter Architectre Eample 5: Cycle D elayed B r an ch lw $, 5 ($7) bbble (nop) IF.F lsh Hazard detection nit left In strction 76 7 m em ory etend Fig Clock Re gisters = ID / W B beq $, $3, 7 sb $,... befo re<> $ $3 F orwa rding nit / E mem ory E /W B P r e d ic t -n o t -t a k e n + b r a n c h d e c is io n a t I D = > t h e f o l l o w in g in s t r c t io n is a l w a y s e e c t e d = > b r a n c h e s t a k e e f f e c t c y c l e l a t e r I n s t r. O r d e r add beq misc lw Time (clock cycles) em em c loc k c y c le p enalty p er b ranc h instrc tion if c an find instrc tion to p t in slot ( 5 % of tim e) em em em em em em Pipelining-86 Compter Architectre Pipelining-87 Compter Architectre

23 H O t li n e A n o v e r v w o f p e l A p e l e d d a t a p a t h P e l e d c o n t r o l D a t a h a z a r d s a n d f o r w a r d D a t a h a z a r d s a n d s t a l l s B r a n c h h a z a r d s E c e p t n s S p e r s c a l a r a n d d y n a m p e l ie ip in in g ip in ip in in g io ic ip in in g W h at ab o t Ecept i o n s? 5 i n s t r c t i o n s e e c t i n g i n 5 s t a g e p i p e l i n e How to stop the pipeline? restart? W ho c a sed the interr pt? W ho to serv e f irst, if m ltiple interr pts at the sam e tim e? N e e d t o k n o w i n w h i c h s t a g e a n e c e p t i o n c a n o c c r = > h e l p d e t e r m i n e c a s e Stage P r o b l em i n ter r p ts o c c r r i n g I F P a ge f a l t ; m is a l ign e d m e m o r y a c c e s s ; m e m o r y -p r o t e c t io n v io l a t io n I D U n d e f in e d o r il l e ga l o p c o d e E A r it h m e t ic e c e p t io n E P a ge f a l t ; m is a l ign e d m e m o r y a c c e s s ; m e m o r y e r r o r ; m e m -p r o t e c t io n v io l a t io n ; Pipelining-88 Compter Architectre Pipelining-89 Compter Architectre an d li n g Ecept i o n s P i peli n e w i t h Ecept i o n S p p o s e o v e r f l o w o c c r a t D W B, I F I D E c m z c ( c E = > F I F, b y P C U S E P C l t l e t e r r p t s : s e p r r y h a r d w a r e t o c h o o s e t h e e a r l s t s t r c t n t o t e r r p t E t e r n a l t e r r p t s : f l e l e w h e n t o t e r r p t add $,$,$ isab le writes of instrc tions till trap hits e.g., flsh following instrc tions sing.f lsh,.f lsh,.f lsh to ase ltip le ers to ero ontrol signals ov erflow e ep tion detec ted at flsh offending instrc tion) orc e trap instrc tion into e.g., fetc h from he adding he to inp t av e address of offending instrc tion in ip in io it ie in io in in ib in in IF.Flsh Instrction Hazard detection nit left etend ID.Flsh = Case Ecept.Flsh /E Fig E/ nit Pipelining-9 Compter Architectre Pipelining-9 Compter Architectre

24 O t li n e D i f f er en t P i peli n ed D es i g n s A n o v e r v w o f p e l A p e l e d d a t a p a t h P e l e d c o n t r o l D a t a h a z a r d s a n d f o r w a r d D a t a h a z a r d s a n d s t a l l s B r a n c h h a z a r d s E c e p t n s S p e r s c a l a r a n d d y n a m p e l ie ip in in g ip in ip in in g io ic ip in in g Pipelining Sper-pipeline - Isse one instrction per (fast) cycle - takes mltiple cycles Sper-scalar - Isse mltiple scalar instrctions per cycle VLIW (EPIC) - Each instrction specifies mltiple scalar operations - Compiler determines parallelism IF D E W IF D E W IF D E W IF D E IF D E W IF D E W IF D E W IF D E W IF D E W IF D E W IF D E W IF D E W IF D E W E W E W E W W Limitation Isse rate, FU stalls, FU depth Clock skew, FU stalls, FU depth Hazard resoltion Packing Pipelining-9 Compter Architectre Vector operations - Each instrction specifies series of identical operations IF D E W E W E W E W Pipelining-93 Applicability Compter Architectre A lt er n at i v e S i mple S per s calar lt i ple P i pes ( H ar d er S per s calar ) I n d e p e n d e n t I N T a n d FP is s e t o s e p a r a t e p ip e l in e s IR IR Isses: Operand / Reslt Bsses INT INT Unit I-Cache Inst Isse and Bypass Load / Store Unit D-Cache FP FP FP l D$ A R T B ister File B R T A D$ R e gis t e r f il e p o r t s D e t e c t in g d a t a d e p e n d e n c ie s B y p a s s in g R A W H a z a r d W A R H a z a r d l t ip l e l o a d / s t o r e o p s? B r a n c h e s Pipelining-9 Compter Architectre Pipelining-95 Compter Architectre

25 S i mple I P S S per s calar D yn ami c S ch ed li n g Instrction Fig. 6.5 T h e h a r d w a r e p e r f o r m s t h e s c h e d l in g? hardware tries to find instrc tions to e ec te ot of order e ec tion is p ossib le sp ec lativ e e ec tion and dy nam ic b ranc h p redic tion A l l m o d e r n p r o c e s s o r s a r e v e r y c o m p l ic a t e d D E C A lp ha 6 : 9 stage p ip eline, 6 instrc tion isse P owerp C and P entim : b ranc h history tab le C om p iler tec hnology im p ortant etend etend ress Pipelining-96 Compter Architectre Pipelining-97 Compter Architectre T h r ee P r i mar y U n i t s S mmar y Reservation station Reser vation station Instrction fetch and decode nit Reservation station Reser vation station In-order isse P e l e s p a s s c o n t r o l f o r m a t n d o w n t h e p e j s t a s d a t a m o v e s d o w n p e r w a r d s t a l l s h a n d l e d b y l o c a l c o n t r o l E c e p t n s s t o p t h e p e l e I P S s t r c t n s e t a r c h e c t r e m a d e p e l e v l e ( d e l a y e d b r a n c h, d e l a y e d l o a d ) o r e p e r f o r m a n c e f r o m d e e p e r p e l e s, p a r a l l e l m ip in in io ip ip Fo in g/ io ip in in io it ip in is ib ip in is Fnctional nits Integer Integer Floating point Load/ Store Ot-of-order eecte In-order commit Commit nit Fig Pipelining-98 Compter Architectre Pipelining-99 Compter Architectre