Queuing Theory: Memory Buffer Limits on Superscalar Processing

Transcription

1 Cle/ Model of I/O Queug Theory: Memory Buffer Lms o Superscalar Processg Cle reques respose Devce Fas CPU s cle for slower I/O servces Buffer sores cle requess ad s a slower server respose rae Laecy Tme bewee cle reques ad buffer server respose Throughpu Number of servces provded per u me Badwdh Maxmum daa rasfer rae of I/O chael (cludg buffer) Capacy Maxmum hroughpu of server hrough buffer Depeds o badwdh ad servce rae (server speed) Ulzao Reques rae as proporo of capacy Buffer Operao Cle reques respose Cle requess Ierme hgh speed requess (bursy) Peak cle reques rae >> Average cle reques rae Servce resposes Peak cle rae > rae > Average cle rae FIFO buffers requess order of arrval Sores requess arrvg a hgher cle reques rae Forwards requess o server a lower server respose rae Reques g rae = server respose rae Devce Buffer Laecy ad Overflow Cle reques respose Devce Mmum laecy Deermed by maxmum servce respose rae Buffer overflow Buffer fll rae = Cle Reques Rae Servce Respose Rae Buffer flls couously for oo log buffer overflow Example Peak CPU dsk read rae = read/cycle = 9 read requess/secod Dsk ca provde = 7 resposes per secod = CPU cycles/read CPU sees mmum laecy of abou CPU cycles Buffer ca hold requess Couous requess overflow /( 9 7 ) ~ - secods 3 4

2 5 Ulzao Laecy Trade Off Ulzao Hgher cle reques rae More servces per secod Hgher ulzao cao work faser (servce rae s fxed) More requess are buffered loger queue legh (hgher buffer level) Toal laecy for oe reques = server laecy queug me More requess buffer queue loger queug me Hgher ulzao Hgher oal laecy Buffer overflow Average reques rae > average respose rae More requess eer buffer ha leave Buffer level rses Afer log me, buffer overflows laecy buffer level ulzao.8.9 Queug Theory Cle Assumpos Cle Requess Arrve depedely (Posso sascs) Have radom legh (byes o rasfer) Buffer Sores requess ad s order of arrval (FIFO) a servce rae Reaches a average "seady sae" level of sored requess Provdes servces o each reques depedely (Posso sascs) Resuls Average laecy = / (average servce rae average arrval rae) Average buffer level = average arrval rae average buffer laecy (Lle's Law) reques respose Devce Posso Sascs Radom eves Occur depedely Average oal umber of eves grows learly wh me probably ( eves me ) ( α) P k = k = e k! k α () probably ( me erval bewee eves s ) p = = αe α α = Average me bewee eves = α e d = α α = average rae of eves k = k = Average umber of eves = kp k = α α = average eve rae me erval Reques ad Respose Sascs = average reques rae ( ) k Probably of k requess me = e k! Dsrbuo of mes bewee requess = e = average volume (byes) per respose μ C = server capacy (maxmum respose volume per secod) server volume / secod = = average respose rae average volume / respose ( ) Probably of k resposes me = e k! Dsrbuo of server respose mes = e k 7 8

3 9 Buffer Sae B Δ = requess buffer a me Δ Possble cases for B( Δ ) =. B() = Durg, ew reques ad fshed resposes () [ Δ] B = Durg, ew requess ad fshed respose [ Δ] B() = Durg [, Δ] B () = [ Δ] - ew reques ad fshed resposes Durg, ew reques ad fshed respose Buffer Sae Traso probably P Δ = B Δ = () B () B () B () B = AND reques AND fsh OR = AND reques AND fsh = probably OR = AND reques AND fsh OR = AND reques AND fsh = P () PΔ ( reques ) PΔ ( fsh) P () PΔ( requess ) PΔ( fsh) P() PΔ ( requess ) PΔ ( fsh) P () P ( reques ) P ( fsh) Δ Δ Buffer Sae Traso Seady Sae Assumpo ( reques ) ( reques ) ( fsh ) ( fsh ) ( Δ) ( Δ) ( Δ) ( Δ) ( ) Δ P Δ = e = Δ o Δ! ( ) Δ P Δ = e = Δ o Δ! ( ) Δ P Δ = e = Δ o Δ! ( ) Δ P Δ = e = Δ o Δ! ( Δ ) = () ( Δ )( μ Δ ) ()( Δ )( μ Δ ) () P P P C P C P Δ Δ Δ Δ o Δ () ( μ ) () () μ () { } = P C P P CP Δ o Δ For large, buffer level reaches "seady sae" (almos cosa) B () ad P() P New requess balace fshed resposes Almos Empy Buffer () AND P B() ( reques ) ( fsh) P B = reques = = AND fsh P PΔ = P PΔ P Δ = P Δ Δ P = P = P = P Δ = = average reques rae average servce rae = ulzao P( reques) B = B = P( fsh)

4 3 Geeral Seady Sae () P P ( ) ( Δ ) d P P = P () = lm d Δ Δ ( ) P () P () P () Δ o ( Δ) = lm Δ Δ = P C CP P ()( μ ) μ () () P = ( ) P P P P = = P P = (cosa) Average Buffer Level From recurso relao, P P P P P P P P P P P P P P P = ( ) ( ) ( ) ( ) ( ) = = = 3 3 = = = P = P sum of probables = = P = P = P P = = Probably of Buffer Lev el Average Buffer Level = = P = ( ) = = ( ) = = = = 4 Toal Laecy ad Buffer Level Laecy ad Buffer Level versus Ulzao T = servce me for oe reques buffer laecy = servce me for oe reques me o servce FIFO queue μ C = = μ C = = = = = average servce rae average reques rae Lle's Law = = = T = ( average reques rae) ( average laecy) Normalzed Laecy Buffer Level T ( ) = = = Laecy (secods) Normalzed Laecy = Laecy wh zero ulzao T 5

5 7 Memory Queug Memory Orderg Buffer (MOB) Bewee Load/Sore ad daa cache Load Sore Buffers read/wre operaos Eables "smulaeous" reads ad wres MOB Issues loads before sores Ehaces hroughpu of speculave reads Forwards queued sores o read requess prory L Daa cache Hard drve SATA 3. susaed hroughpu = Gbps < MB/s PCI Express 4. susaed hroughpu < GB/sec Typcal dsk wre buffer = 4 MB CPU Overflow me = 4 MB / [(.) GB/s] = 4. ms Safe removal of exeral drve Dump buffer o fle Mark fle closed fle sysem Usafe removal possble daa loss Buffer Dsk g Isruco Decoder (Cle) Cle srucos (requess) Isruco Pool (Reorder Buffer) execue () rereme (resposes) sruco wdow Execuo Us () Decoder seds srucos per cycle o ROB s ca operae o srucos (oe sruco wdow) per cycle Goal Maxmze ulzao of s = whou ROB overflow For fxed sruco wdow maxmze No overflow average decoder rae < sruco wdow 8 Geeral Superscalar Model Execuo us (s) operae parallel sages Ideal case Every sage of every workg o every clock cycle Mulple srucos ppeled hrough sages Example ALUs cycle per sruco ALU Load Sore cycles per sruco MEM FPU 3 cycles per sruco FPU Fech Decode LOAD R, a ADD R3, R, R SUB R4, R, R ADDF F, F, F MULTF F4, F5, F DIVF F8, F9, F STORE b, R8 Isruco Pool ADD Sore SUB DIVF Load MULTF ADDF MEM FPU 7 srucos varous sages of execuo FPU 3 Rere Dealed Aalyss of ILP Ppele srucure u = execuo us () of ype u = u = oal execuo us () CPU u s = ppele sages of ype IC = u s = oal ppele sages CPU s = ppele sages of ype s = srucos execug all s = sze of sruco wdow = srucos execug parallel (ILP) u = s = = = u u u sruco wdow = IC = u s u s IC average ppele sages s 9

6 Queug Theory ad Decoder Rae Isrucos Per Clock Cycle scalar u u IPC = = u IPC = = CPI CPI CPI scalar sall u IPC = IPC = = = average respose rae sall CPI Average buffer (ROB) level = = = IPC Maxmze ulzao ROB level = sruco wdow IC IPC s u IPC = = IC = = IPC IC s u sall deal CPI = IPC = u = u deal s u = = srucos aempg o execue per cycle s u Example of Isruco Wdow Execuo us eger ALUs CC per execuo FPUs 3 CC per execuo load/sore us CC per execuo Isruco wdow u = u = = IPC deal u s = s = 3 = = u IC = u s = srucos execug parallel Decoder rae (deal) deal Maxmze = = whou overflow (ROB level = IC ) IPC deal deal s u = = > = = = = 5.54 s u Implcaos for ILP Scalably Scalg sruco wdow ad decoder rae execuo us u u' = αuu ( βα s u ) s u deal deal ppele sages s s' = βss ' = ( βα s u) s u sruco wdow IC IC ' = αβic u s Scalg 5 s wh 8 superppeled sages 5 8 αu = βs = αu βs = IC = srucos execug parallel deal 5 > 4.9 srucos decoded per CC Dffcules Decode 5 srucos per CC wh cache msses, mspredcos, Maa wdow of depede srucos 5 3 brach srucos wdow large mspredco probably 3