Performance Modeling of Hierarchical Memories

Transcription

1 Performane Modelng of Herarhal Memores Marwan Sleman, Lester Lpsky, Kshor Konwar Department of omputer Sene and Engneerng Unversty of onnetut Storrs, T Emal: {marwan, lester, kshor}@engr.uonn.edu Abstrat As memory herarhy expands n modern omputng envronment from PU regsters and loal memory to mddle-ter, network storage, and nternet storage, the optmal goal of a omputer arhtet beomes to desgn a memory herarhy that maxmzes the overall desgn of hs mahne wth a mnmal ost. Ths requres dedng on the number, speed and sze of the herarhal layers. As the gap between proessor and memory speed s growng exponentally, t beomes more mportant to develop an analytal model to apture all these herarhal levels and optmze the memory aess tme to make a good utlzaton of both PU and memory. In ths paper we study the performane of systems wth mult-level herarhal memores by modelng ther aess tme whh helps the desgner optmze the ost and aess tme. We use a lnearalgebra queung theory approah to aheve our goal and we explan why prevous attempts faled to provde aurate models. Our model dffers from all the prevous related work by beng global and general and by usng some probablst equatons that show the nterdependene between the dfferent levels and by usng the P-K formula to dstngush between the memory aess tme and queung tme. Our approah s ndependent of the applaton usng the memory whle lassal approahes were program dependent. Moreover, our model aheves hgher levels of auray whle beng expandable to multple levels. Keywords: Memory Herarhy, Performane Analyss, Markov model, Queung, Aess Tme. INTRODUTION Beng able to make aurate estmates of how long a memory aess wll take to fnsh n a mult-level herarhal memory system s of prmary nterest n the performane ommunty. In suh a herarhal envronment, we have multple levels of memory startng from PU regsters and extendng to ahes, man RAM memory, loal dsks, and network storage [6]. As the memory herarhy extends to network and nternet storage passng by mddle-ter arhteture and ahng, the problem of optmzng the memory aess tme beomes more hallengng. Data s stored and held n eah level untl t s used or replaed. Eah memory level dffers from the others by ts sze and speed. As the memory beomes loser to the PU, t beomes faster but smaller and more expensve. Thus, when desgnng a memory herarhy, our am s to get the fastest desgn whle mantanng a ompromse between sze, speed and ost. Havng found that the prevous approahes n studyng memory aess tme were lmted wth the number of memory levels they an represent, depended on the applaton, and dd not provde wth hgh auray, we represented the herarhal memory by a M/G/ queue [] and we alulated the aess tme by usng lnear algebra queung theory. Then we onsdered the queung tme of the onseutve memory requests that an our n a database applaton for example and we showed the dfferene between the aess tme and the queung tme. Ths dfferene explans the ause of the nauray of prevous approahes n predtng the memory aess tme. We also study the behavor of the varane aess tme whh s hghly rtal beause t an dramatally affet the mss rato of the memory system and ts performane. The model we bult s more general than the lassal models beause t an take as nput dfferent nput parameters lke, aess tme, ht rato, ost, and memory request dstrbuton. The remanng of ths paper s organzed as follows: In seton, we present a lterature survey about prevous efforts related to the top and we explan our motvaton. We dsuss the PK formula method to alulate the memory request queung tme n seton 3. In seton 4, we present two ases for evaluatng our methods. Then, n seton 5, we show the alulaton results by plottng the values resultng from eah alternatve and we show the dfferene between the two. Fnally, n seton 6, we propose some tops for further nvestgaton, and we onlude n seton 7.. BAKGROUND AND MOTIVATION The herarhal memory aess tme was studed by several researhers but all the prevous approahes to model and optmze the aess tme were lmted. The lmtatons are the result ether from the dependene of the models on the applaton or from the lmtatons of the analytal model that an not represent the deep herarhes. For example, Balasubramonan et al. explan that the reent memory herarhy organzatons do not math the applatons requrements whh results n a degradaton n the performane []. Jn et al. develop a lmted analytal model that aptures only a two-level ahe [3], but we see n ther work a bg dsrepany between the predted and measured memory performane. Most researhers foused on two-level memory as shown n artles [8] and [9] and we don t see any work that foused enough on deep memory herarhes. None of the prevous researhers talked about the varane of the aess tme of the memory herarhy. Ths varane s of prmary mportane beause a hgh varane n the aess tme orresponds to a hgher mss rato and unexpeted delay -whh s undesrable. In a prevous work [4], we have shown that the aess tme for a herarhal memory wth an nfnte depth s power taled []. In ths paper, we expand our prevous work and work on optmzng the memory desgn by tryng to buld a memory wth a mnmal response tme and mantanng a mnmal ost. We also show analytally the ause of dsrepany between the analytal and measured values for prevous researhers. Our Model s based on Markov han analyss whh s ndependent of the dstrbuton of memory request that depends on the applaton. We onsder a herarhal memory that onssts of L levels and a lowest memory level m as shown n fgure. We model ths herarhy n a state dagram as shown n fgure. Eah physal memory level,, n fgure orresponds to two states n fgure : The upper state orresponds to the lookup tme whle the lower state orresponds to the memory aess tme. The frst state orresponds to the memory request from the proessor whle the last state orresponds to the lowest memory n the herarhy.

2 PU Level Level Level l m Aess Tme Fgure. Herarhal Memory Model: PU wth L ntermedate levels of memory and man memory m. T T T l Fgure. State dagram of a herarhal memory system wth L ntermedate levels: Intermedate memory levels are represented by two states. The fgure shows the ht rato h n every level and the aess tme T at every state. Ths herarhal memory system s a M/G/ queue. In order to buld the lnear algebra model for ths system [], we defne the followng terms: X s the random varable representng the system tme that orresponds to the total memory aess tme through all memory stages. P s the sub-stohast matrx that orresponds to the transton from one state to another one. p s the entrane vetor that orresponds to the state of the system when at the frst memory request. p s a row vetor of sze L +, where L s the number of ntermedate levels. ε s the unt olumn vetor of sze L +. M s the transton rate matrx; t orresponds to the rates of leavng eah state. M s a dagonal matrx of the same sze as P. and M. h I s the dentty matrx of the same dmenson as P B M(I P) V B - s the nverse of B. T T T l T m - h T -h (-) T - -h -h (l-) T l- h h l l l -h l 0 h h P hl hl T M 0 0 ε T 0 0 Tm We have shown n [4] that the memory aess tme s gven by the frst moment of V and t s ndependent of the dstrbuton of both the memory aess request (whh s dependent on the ompler) and the serve tme of the nodes (whh depend on the hardware spefatons of the memory levels). The mean memory aess tme s gven by: EX ( ) x p Vε () However the varane of the aess tme s dependent on the dstrbuton of the serve tme of the memory nodes. It s dependent on the frst and seond moments of V. For exponental dstrbutons, t s gven by: σ ex p V ε - (pvε ) () For non-exponental dstrbutons, the varane s gven by: σ X σ ex + p V T Γ ε Where Γ dag( v-, v-,, vl-) EX ( Where ) x v s the oeffent of varaton of state x n Fg.. In an applaton that has multple onseutve memory requests lke a database applaton for example, the memory requests wll be queued and must wat to get serve from a busy memory, so nether the prevous models nor the above model wll be suffent to predt the exat tme. Thus we use the P-K formula n the next paragraph to fnd the exat queung tme. 3. THE P-K FORMULA The Pollazek-Khnthne formula (alled P-K formula) [7] gves the expeted average number of ustomers n queue and n proess n M/G/ queues. The P-K formula was ombned wth Lttle s theorem [] to show that the mean tme spent by a ustomer n an M/G/ queue s gven by: x xρ T + ρ ρ (3)

3 Where, s oeffent of varaton, ρ s the utlzaton fator, ρ λ x, and λ s the arrval rate. σ ex, x In ths paper we use equaton (3) to predt the queung tme for our herarhal memory system shown n fgure whh beomes as shown n fgure 3. The queung ours when we have a system wth multple onseutve memory requests that an not be aessed by the sequental memory at the same tme, so the memory requests are buffered n a queue; ths an be the ase of a shared memory on a parallel mahne, a smple database aess applaton, or a ppelned PU. In the last two ases the queung auses a bottlenek and affets the performane of the system beause t nreases the PI n a ppelned proessor [9] and nreases the query exeuton tme n a database applaton [0]. - h T -h (-) T - -h -h (l-) T l- l -h l dfferent mnma. We suppose that we have a herarhal memory system we are buldng and we assume that the system has a ost. Przybylsk [] used Agarwal s ahe mss model [] to show that the ahe mss rato s nversely proportonal to ts sze; so the ht rato of eah memory level, whh s the probablst omplement of the mss rato, s proportonal too to the nverse of ts sze and thus ts ost. But extendng ths observaton to multlevel memory s a lttle omplated and requres more alulatons; so we defne the followng parameters for the system n Fg.: a s the probablty of fndng data n the ntermedate memory level. S s the sze of eah ntermedate memory level. s the ost per unt of sze of eah ntermedate memory level. β a, where β s a onstant. S The total ost of the L-levels herarhal system beomes: L S (4) 4.. TWO-LEVEL ASH MEMORY SYSTEM We frst onsder the -level ash memory system shown n fgure 4. λ h h h l m T - h T 3 3 -h T T T l Fgure 3. Queung dagram of a herarhal memory system wth L ntermedate levels: Now the arrvng memory requests arrve wth rate λ and are queued before the PU. The varane of the queung tme of the model shown n fgure 3 s the same as that of the model shown n the prevous paragraph whh s shown n equaton (). It s obvous from both equatons () and (3) that the mean memory aess tmes depend on several parameters nludng ht rato h and aess tme T at eah memory level. 4. SAMPLE ASES FOR TEST AND EVALUATION In order to show the dfferene between the mean memory aess tme n equaton () and the mean queung tme for memory nput/output requests n equaton (3), we apply equatons () and (3) to several ases then ompare the results for eah ase. Our am s to show that the mean aess tme s not the same for both methods and has l Fgure 4. State dagram of a two-level ash memory: In ths fgure we dstngush between the memory ht rato and the probablty of fndng data n eah level. We name the frst level M and the seond level M. We defne the followng terms: S s the sze of memory level, M h h T T4 4 M M h s the ost per unt of sze of memory level, M S s the sze of memory level, M s the ost per unt of sze of memory level, M Y s the random varable representng the probablty of fndng data n a memory level a h M/ Y M) a h M) a h M M) h M) hh h( h) h( h) h M/ Y M) hh h h m

4 Where, h and h are respetvely the ht ratos of memory level and memory level. The ost of ths system s gven by: S S + S (5) 4.. THREE-LEVEL ASH MEMORY SYSTEM Then we onsder the 3-level ash memory system shown n Fg.5. T - h T 3 -h T 5 - l- h h h 3 -h 3 m h H H H a b HbH( Ha) h H ahh b H( Hb) h3 HH b The ost of the memory system s gven by: S S + S + S (6) a b 3 It s lear from equatons () and (3) and from the above dervatons n ths seton that the mean tme s a funton of the ht rato and sze of eah ntermedate level, so to optmze the mean aess tme, we wll have to optmze () and (3) versus these parameters. For the two systems we show here, we suppose that we have a onstant ost and we try to optmze the ntermedate memory ost and sze to get the fastest possble desgn as shown n the next seton. l T T 4 T 6 M M M 3 Fgure 5. State dagram of a three-level ash memory: In ths fgure we dstngush between the memory ht rato and the probablty of fndng data n eah level. We name the frst level M, the seond level M, and the thrd one M 3. Agan we defne the followng terms: S s the sze of memory level, M h s the ht rato at memory level, M, h Y M Y M a Pr( / ) s the ost per unt of sze of memory level, M s the ost per unt of sze of memory level, M b s the ost per unt of sze of memory level 3, M 3 Y s the random varable representng the probablty of fndng data n a memory level 5. ALULATIONS AND RESULTS Now that we have the analytal model to alulate the memory aess tme and queung tme, we wrote a Matlab ode to mplement our equatons and to verfy that what we mentoned s aurate. We have arred out an exhaustve set of program runs over several parameters. Sne the results are onsstent wth eah other we present only a few here. We frst onsder the -Level memory system n fgure 4 and we assume that t has an arbtrary fxed ost. To study the mean response tme versus S, the sze of memory level, we assume that S s n an arbtrary nterval (4.4<S<7.8). So S, the sze of memory level, wll be gven by dret dervaton from Equaton (5): S S We plot both the mean memory tme, E(x), and the queung tme, E(T), versus the sze of the level memory n fgure 6. We remark that they have dfferent mnma -whh onfrms our assumpton about the dfferene between them. We also defne the followng probabltes: H H a b 3 3 H M ) a 3 3 We an proof by a smple alulaton that the ht ratos are gven by: Fgure 6. Mean memory aess tme E(X) and mean queung tme E(T) versus the sze S of the Level memory, M, for a -Level herarhal memory system. E(x) has ts mnmum for S 6.4, whle E(T) has ts mnmum for S 6.8 Then we plot the varane of the memory aess tme obtaned from equaton () for the two-level memory system n fgure 7. The behavor of the varane s as mportant as that of the mean

5 memory tme and queung tme beause t auses devaton from the mean tme. Devaton from the eah sde of the mean tme s undesrable: fnshng too late s obvously undesrable beause t an dramatally affet the performane (lke nreasng the PI n a ppelned proessor for example), but fnshng too early s also undesrable beause we waste our memory resoures. We remark here that, whle the memory tme has a onvex behavor, the varane dereases as we nrease the memory sze and ths s normal beause t depends on the seond moment of the memory aess tme [4] whh nreases faster as the memory sze nreases. shown n fgure 5. Now the system s more omplated beause we have more varables. Here too, we assume that the system has an arbtrary ost and we assume that S and S 3, the szes of the seond and thrd memory levels, are n arbtrary ntervals (<S <6 and 6<S 3 <8). So, S, the sze of memory level, wll be gven by dret dervaton from Equaton (6): bs S3 S a We plot both the mean memory tme, E(x), and the queung tme, E(T), versus the szes of the memory levels and 3 n fgure 9. We remark here too that both surfaes are smlar and they have dfferent mnma. Fgure 7. Varane of the memory aess tme versus the sze S of the Level memory, M, for a -Level herarhal memory system. The varane dereases as the memory sze nreases. To emphasze more on the dfferene between E(X) and E(T), we alulate the dfferene between the value of the mnmum of E(T) and the value of X at the same value of S. We all ths dfferene DffT. We plot DffT versus the nput rate λ n fgure 8. We selet the values of λ to keep the system utlzaton ρ fator between zero and one []. We remark n fgure 8 that the dfferene s more sgnfant as the traff nput rate and system utlzaton nreases. Ths value goes up to.5 % of the mnmum value of the mean tme for a utlzaton fator lose to. Fgure 9. Mean memory aess tme E(X) and mean queung tme E(T) versus the szes of memory levels and 3 for a 3-Level herarhal memory system. Both surfaes look onave. E(x) has ts mnmum of.59 for S 4.5 and S 3 6, whle E(T) has ts mnmum of 3.68 for S 4.75 and S 3 6. The plot of the dfferene between the value of the mnmum of E (T) and the value of X versus λ n fgure 0 shows here a more sgnfant dfferene equal to % of the mnmal value of the memory aess tme. Ths dfferene explans the dfferene between the predted and measured performane that Jn et al. get n ther paper about performane predton on shared memory programs (3) n effet the authors there alulate the mean aess tme whle the values they measure are those of the queung tme, ths s why they get a 0% dfferene! Fgure 8. Dfferene between Mn(T) and the value of X for the same value of S for dfferent values of nput rate λ, for a -Level herarhal memory system. To show that our results are ndependent of the number of level n a herarhal memory, we repeat what we dd for the -level ash memory to the 3-level ash memory Fgure 0. Dfferene between Mn(T) and the value of X for the same value of S for dfferent values of nput rate λ, for a 3-Level herarhal memory system. Fnally to ompare the performane of the two memory systems we plot the varane of the memory aess tme for the three-level memory system n fgure. We remark n fgure that the

6 standard devaton s more senstve to the upper level memory and t dereases faster as we nrease Sb, the sze of the upper level. If we ompare fgure to fgure 7, we remark the standard devaton s hgher n fgure whh means that addng one more level to the herarhy nreases the varane and ths observaton s very mportant beause the omputer arhtet must take t nto onsderaton when desgnng systems senstve to the varane. the levels and add more levels to the herarhal memory system: Inreasng the sze of a memory redues the varane; however addng more levels nreases t. Our observaton helps the desgner dede whether to use bgger levels of memores or use more levels n hs desgn. Our analytal model shown n equaton (3), s a unversal model and an represent deep memory herarhes that extend beyond the onept of loal mahne storage to network storage. Ths model uses Markov han analyss and an take dfferent types of memory request dstrbutons. Our model s also deal for optmzaton beause t an take dfferent nputs lke the ht rato, sze, ost and speed parameters of the ntermedate memory level. Ths flexblty of takng dfferent parameters makes t easy to expand and apture any level of herarhy. Fgure. Varane of the memory aess tme versus the szes of the ntermedate Levels for a 3-Level herarhal memory system. The varane dereases as the memory szes nreases. 6. FUTURE WORK Ths paper s part of a larger work examnng performane of herarhal memory systems wth both analytal and smulaton tehnques. There are many tops we are ether already nvestgatng or hope to nvestgate soon. We ntend to valdate our performane model by omparng the predted aess tmes aganst exeuton tmes measured on real mahnes by usng benhmarks. We ntend to nlude models that aount for loaltes and workng sets. In addton to the mean and varane, we are plannng to study the more performane metrs and measures of the memory aess tme for both the exponental and non exponental ases. These measures nlude, but are not lmted to, the onfdene nterval and the relablty of the aess tme n addton to the rato of the lag between the target tme and atual tme to the varane. We wll work on proofng the onvexty of the mean tmes obtaned n equatons () and (3) and applyng several optmzaton tehnques on equaton (3) to optmze the desgn of mult-level herarhal memores to ome out wth the fastest possble ost-effetve system. 7. ONLUSION We have developed an analytal model to evaluate the mean and varane of the aess tme for memory requests n a herarhal mult-level memory envronment. We have shown and explaned the dfferene between the mean memory aess tme and the memory requests queung tme. Ths dfferene explans the dsrepany between the analytal values and the pratal values obtaned by prevous researhers. We have also shown the behavor of the varane of the aess tme as we nrease REFERENES [] Lester Lpsky, Queueng Theory - A Lnear Algebra Approah, Maxwell Mamllan Internatonal publshng group, 99. [] Rajeev Balasubramonan, Davd Albonesz, Alper Buyuktosunoglu, and Sandhya Dwarkadas, Dynam Memory Herarhy Performane Optmzaton, 7 th nternatonal symposum on omputer arhteture, June 000. [3] Ruomng Jn, Gagan Agrawal, Performane Predton for Random Wrte Redutons: A ase Study n Modelng Shared Memory Programs, Proeedngs of the 00 AM SIGMETRIS nternatonal onferene on Measurement and modelng of omputer systems, Marna Del Rey, alforna, pages: 7 8, 00. [4] Kshor M. Konwar Lester Lpsky Marwan Sleman, Moments of Memory Aess Tme for Systems Wth Herarhal Memores, st Internatonal onferene on omputers and Ther Applatons (ATA- 006), Seattle WA, Marh 006. [5] rstna Hrstea, Danel Lenosk, and John Keen. Measurng Memory Herarhy Performane of ahe-oherent Multproessors Usng Mro Benhmarks. Proeedngs of S 97, 997. [6] Jon Wllam Togo, The Holy Gral of Network Storage Management, Prente Hall, 004 [7] Danel P Heyman and Matthew J Sobel, Stohast Models n Operatons Researh: Stohast Proesses and Operatng haratersts, ourer Dover Publatons, 004 [8] A. Smth, ahe Memores, omputng Surveys, 4(3): p , 98. [9] A. Smth, Dsk ahe-mss rato analyss and desgn onsderatons. AM Transaton on omputer Systems, 3(3), p 6-03, 985. [0] I. MaIntyre and B. Press, The Effet of ahe on the Performane of a Mult-Threaded Ppelned RIS Proessor, the Engneerng Insttute of anada, 99. [] S. Manegold, P. Bonz, and M. Kersten, Gener Database ost Models for Herarhal Memory Systems, Proeedngs of the 8th VLDB onferene, Hong Kong, hna, 00. [] S. Przybylsk, ahe and Memory Desgn: A Performane-Dreted Approah, Morgan Kaufmann Publshers, 990. [] A. Agarwal, Analyss of ahe Performane for Operatng Systems and Multprogrammng, Ph.D. thess, Stanford unversty, May 987. [3] R. Jn and G. Agrawal, Performane Predton for Random Wrte Redutons: A ase Study n Modelng Shared Memory Programs, Proeedngs of the 00 AM SIGMETRIS nternatonal onferene on Measurement and modelng of omputer systems, Marna Del Rey, alforna, p 7-8