Dynamc Power Managemen Based on Connuous-Tme Markov Decson Processes* Qnru Qu and Massoud Pedram Deparmen of Elecrcal Engneerng-Sysems Unversy of Souhern Calforna Los Angeles Calforna USA {qnru pedram}@usc.edu Absrac Ths paper nroduces a connuous-me conrollable Markov process model of a power-managed sysem. The sysem model s composed of he correspondng sochasc models of he servce queue and he servce provder. The sysem envronmen s modeled by a sochasc servce reques process. The problem of dynamc power managemen n such a sysem s formulaed as a polcy opmzaon problem and solved usng an effcen polcy eraon algorhm. Compared o prevous work on dynamc power managemen our formulaon allows beer modelng of he varous sysem componens he power-managed sysem as a whole and s envronmen. In addon capures dependences beween he servce queue and servce provder saus. Fnally he resulng power managemen polcy s asynchronous hence s more power-effcen and more useful n pracce. Expermenal resuls demonsrae he effecveness of our polcy opmzaon algorhm compared o a number of heursc (me-ou and N- polcy) algorhms. I. INTRODUCTION Wh he rapd progress n he semconducor echnology he chp densy and operaon frequency have grealy ncreased makng power consumpon n baery-operaed porable devces a maor concern. The goal of low-power desgn for baery-powered devces s o exend he baery lfeme whle meeng he performance requremen. Reducng power dsspaon s a desgn goal even for non-porable devces snce excessve power dsspaon resuls n ncreased cos of packagng and coolng as well as poenal relably problems. Many compuer aded desgn mehodologes and echnques for low power have been proposed []. The acvy of many componens n a compung sysem s evendrven; for example he acvy of dsplay servers communcaon nerfaces and user nerface funcons s rggered by exernal evens and s ofen nerleaved wh long perods of quescence. An nuve way of reducng he average power dsspaed by he whole sysem consss of shung down he resources durng her perods of nacvy. In oher words one can adop a sysem-level power managemen polcy ha dcaes how and when he varous componens should be shu down. The problem of fndng a power managemen scheme (or polcy) ha mnmzes power dsspaon under performance consrans s of grea neres o sysem desgners. Several heursc power managemen polces have been repored n he pas. A smple *Ths work was suppored n par by SRC under conrac No. 98-DJ-66 NSF under conrac No. MIP-968999 and a gran from Toshba Corp. heursc polcy s he me-ou polcy. In hs polcy a devce s pu n s power-down mode afer has been dle for a ceran amoun of me. Obvously hs smple polcy s no effcen. To overcome he lmaons of he sac shu-down polcy Srvasava e al. [6] proposed a predcve power managemen sraegy whch uses a regresson equaon based on he componen s prevous on and off me o esmae he nex urn-on me. In [7] Hwang and Wu have nroduced a more complex predcve shu-down sraegy ha has a beer performance. However hese mehods are only applcable o cases n whch he requess are hghly correlaed. The choce of he polcy ha mnmzes power under performance consrans (or maxmzes performance under power consran) s a new knd of consraned opmzaon problem whch s of grea relevance for low-power elecronc sysems. Ths problem s ofen referred o as he polcy opmzaon (PO) problem. In [] Paleologo e al. proposed a sochasc model for a rgorous mahemacal formulaon of he problem and gve a procedure for s exac soluon. The soluon s compued n polynomal me by solvng a lnear opmzaon problem. Ther approach s based on a sochasc model of power-managed devces and workloads and leverages sochasc opmzaon echnques based on he heory of dscree-me Markov decson chans. In he model of [] me s dvded no small nervals of lengh L. I s assumed ha he sysem can only change s sae a he begnnng of a me nerval. Durng nerval (L (+)L) he ranson probably of he sysem depends only on he sae of he sysem a me L (hence he Markovan propery) and he command ssued by he power manager. The sysem model consss of four componens: a power manager (PM) a servce provder (SP) a servce requesor (SR) and a servce reques queue (). Once he model and s parameers have been deermned an opmal power managemen polcy s obaned o acheve bes power-delay rade-off. Ths approach offers sgnfcan mprovemen over prevous power managemen echnques n erms of s heorecal foundaon and a robus sysem model. Ths approach however has some shorcomngs. Frsly he power-managed sysem s modeled n he dscree-me doman whch lms s n real applcaons. Secondly he model does no dsngush beween he busy sae and he dle sae of he SP (hey are lumped no he power-up sae) herefore he sae ranson probably of he sysem model canno be calculaed accuraely. Thrdly he assumpon ha he ransons of he and he SP are ndependen s naccurae and hus affecs he overall accuracy of hs model. Fnally he power managemen program needs o send conrol sgnals o he componens n every me-slce whch resuls n heavy sgnal raffc and heavy load on he sysem resources (herefore more power dsspaon). In hs work we overcome he shorcomngs of [] by nroducng a new sysem model based on connuous-me Markov decson processes. Ths new model has he followng characerscs:
. The new model s based on he connuous-me Markov decson processes whch s closer o he scenaros encounered n pracce.. The resulng power managemen polcy s asynchronous whch s more suable for mplemenaon as par of he operang sysem. 3. The new model nroduces a ransfer sae n he model of he ; n hs way can dsngush beween he busy and dle saes of he SP. 4. The new model consders he correlaon beween he sae of he and he sae of he SP. 5. A polcy eraon algorhm s used o solve he polcy opmzaon problem. The new algorhm ends o be more effcen han he lnear programmng mehod. We also explore he class of N-polces and show ha under ceran condons hs class of algorhms whch are very easy o mplemen produces opmal soluons. Ths paper s organzed as follows Secon II provdes he background for connuous-me Markov processes and connuous-me Markov decson processes. Secon III descrbes our sysem model for he dynamc power managemen he defnon of cos funcon and he polcy eraon algorhm. Secon V gves he expermenal resuls concludng remarks. II. BACKGROUND Defnon. A sochasc process s a famly of random varables {X() } where s he me parameer. The values assumed by he process are called he saes and he se of possble values s called he sae space. Defnon. A sochasc process X() s called a Markov process f for any se of me < < < n < s condonal dsrbuon has he propery: P [ X ( ) x X ( n ) = xn K X ( ) = x ] = P[ X ( ) x X ( n ) = xn ] where n T and x x x n S. T and S are called he parameer space and sae space of he Markov process respecvely. When T s a connuous space and S s a dscree space he Markov process s called he connuous-me Markov process. Gven a connuous-me Markov process wh n saes s generaor marx G s defned as an n n marx as shown n Eqn. (.). An enry σ n G s called he ranson rae from sae o sae. All enres are defned n Eqn. (.) and Eqn. (.3). Eqn. (.4) gves he relaonshp beween σ and σ. Marx G (also known as he ranson rae marx) s called a dfferenal marx f s enres sasfy propery (.4). σ σ σ L σ σ σ L G = (.) σ σ σ L M M M O p ( ) σ = lm = p () = n (.) p ( ) σ = lm = p () = n; (.3) σ = σ = n; (.4) where p () s he ranson probably from sae (drecly or ndrecly) o sae durng me o and p () s s dervave. The generaor marx n he connuous-me Markov process s he analogue of he ranson probably marx n he dscree-me Markov process. We can calculae he lmng dsrbuon (seady) sae probables of he connuous-me Markov process from s generaor marx. Theorem. shows he relaon beween hs marx and he lmng dsrbuon probables [7]. Before sang he heorem we gve some defnons. Defnon.3 A sae s sad o be recurren f and only f sarng from evenual reurn o hs sae s ceran. A recurren sae s sad o be posve recurren f and only f he mean me o reurn o hs sae s fne. A sae s sad o be ransen f and only f sarng from here s a posve probably ha he process may no evenually reurn o hs sae. Defnon.4 Sae s sad o be accessble from sae f can be reached from whn fne me whch s denoed as. If and hey are sad o be communcae whch s denoed as. The se of all saes of a Markov process ha communcae wh each oher forms a communcang class. Defnon.5 If he se of all saes of a sochasc process X form a sngle communcang class hen X s rreducble. Theorem. () If he Markov process s rreducble hen he lmng dsrbuon lm p () = p S exss and s ndependen of he nal condons of he process The lms {p n n S) are such ha hey eher vansh dencally (.e. p = for all S) or are all posve and form a probably dsrbuon (.e. p > for all S Σ S p = ). () The lmng dsrbuon {p S } of an rreducble posve recurren Markov process s gven by he unque soluon of he equaon: pg = and Σ S p = where p = (p p ). Defnon.6 If we map he saes of a Markov process as verces of a graph and he saes ransons as dreced edges beween he verces. The Markov process s called a conneced Markov process f hs graph s a conneced graph. For he dscussons n he res of hs paper we wll om he erm connuous-me for more concse descrpon. Unless oherwse saed all processes are assumed connuous-me. Frs we descrbe a Markov process wh reward. Assume he sysem earns a reward a rae r (per un me) durng all he me ha occupes sae. When makes a ranson from sae o sae ( ) receves a reward of r. Noe ha r and r have dfferen dmensons. I s no necessary ha he sysem earns accordng o boh reward raes and ranson rewards bu hese defnons gve us generaly. We defne he earnng rae of sae as: r = r + σ r. Le v () be he expeced oal reward ha he sysem wll earn durng a me perod of f sars n sae. The oal expeced reward durng a me perod of +d ha s v (+d) can be wren as: v + d) = ( σ d)[ r d + v ( )] + d[ r + v ( )] ( σ I can be nerpreed as follows. Durng he me nerval d he sysem may reman n sae or make a ranson o some oher sae. If remans n sae for a me d wll earn a rae r d plus he expeced reward ha wll earn n he remanng uns of me v (). The probably ha remans n sae for a me d s ( σ d ). On he oher hand he sysem may make a ranson o some sae durng he me nerval d wh probably σ d. In hs case he sysem would receve he reward
r plus he expeced reward o be made f sars n sae wh me remanng v (). The produc of probably and reward mus hen be summed over all saes o oban he oal conrbuon o he expeced values. Wh d and usng he defnon of earnng rae r we have: d d n v ( ) = r + σ v ( = n (.5) = ) where n s he oal number of saes of he process. Eqn. (.5) gves a se of lnear consan coeffcen dfferenal equaons ha relae he oal reward n me from a sarng sae o he quanes of r and σ. Secondly a conrollable Markov process s a Markov process whose sae ranson raes can be conrolled by conrollng commands (defned as acons). When he sysem s n sae an acon a s chosen from a fne se A whch ncludes all possble acons for sae. We denoe hs sae acon relaon as <a >. If he chosen acon changes as he me changes we denoe he acon as a me-dependen varable a (). Hence he sae-acon par s wren as <a ()>. The sae ranson raes σ have a ( ) dfferen values when dfferen acons are aken. We use σ o denoe he ranson rae from sae o sae when acon a () s aken for sae a me. As a resul he generaor marx of a conrollable Markov process can be represened by a parameerzed (acon s he parameer) marx. Defnon.7 A polcy π s he se of sae-acon pars for all he saes of a conrollable Markov process ha s π={ <a ()> a () A n}. A Markov decson process s a conrollable Markov process wh ( ) rewards. In a Markov decson process snce σ s acondependen he reward rae r becomes also acon-dependen a () whch s denoed as r a () A. The expeced oal reward v () depends on he chosen acon of each sae.e. becomes polcy-dependen and s denoed as v (). The generaor marx a G s hen also polcy-dependen and s denoed as G π. Le () denoe he probably of beng n sae a me when he nal sae s and he sae ranson raes are deermned by polcy π. The oal expeced reward ha he process can earn for a me perod of can be wren as [9]: p = n = a ( τ ) v ( ) p ( τ ) r Gven wo polces π and π f we can fnd a me ξ such ha v ( ) v ( ) for all >ξ = n we denoe as π π. A polcy π s called he opmal polcy f π {any possble polcy for he Markov decson process}. Le v = lm v ( ). The goal of a Markov decson process s o fnd he opmal polcy ha maxmzes pracce we canno use v for all. However n v drecly for calculang he opmal polcy snce v () approaches nfny when approaches nfny. Two alernave quanes are commonly used. () lmng average reward: n a ( τ ) avg = lm p ( τ ) r = v Obvously maxmzng lmng average reward s he same as maxmzng he oal expeced reward. () dscouned reward: n aτ a ( τ ) ds ) = v ( α) = lm e p ( τ r Boh reward models are meanngful for dfferen conex. The decson based on he average lmng reward assumes ha he sysem wll run forever herefore consders reurn n near fuure and far fuure o have he same sgnfcance. Whle he decson based on he dscouned reward consders he fuure as unpredcable.e. he sysem may be ermnaed any me n he fuure. Therefore emphaszes he reurn of he near fuure. The dscoun facor α decdes how greedy hs reward model s. The larger α s he less consders he fuure. When α approaches he dscouned reward approaches he oal expeced reward. Defnon.8 A polcy π s saonary f he decson-makng (acon) s only a funcon of he sae and ndependen of me ha s: π={ <a > a A n}. Theorem. [9] For any α here exss a saonary polcy whch maxmzes v ( ) for all = n. Such a polcy s called ds α α-opmal. Defnon.9 A polcy π s pecewse-saonary f for any τ nerval [ τ) can be dvded no a fne number of nervals [ ) [ ) [ m- τ) such ha nsde each nerval he polcy s saonary. Theorem.3 [9] There exss a saonary polcy ha s α-opmal for π a se of α havng as a lm pon. Ths polcy maxmzes v avg over he class of pecewse-saonary polces. We herefore do no lose generaly f our search for he opmum polcy s resrced on he se of saonary polces. Acons and polces ha we wll dscuss laer are hus me-ndependen. The goal of a Markov decson process s o fnd a polcy ha maxmzes he expeced reward. In our case we wan o fnd a polcy ha mnmzes our cos funcon (delay and power). These wo problems are equvalen f we use he negave of cos as he reward. In he remander of he paper we wll use he erm cos nsead of reward and use c and c nsead of r and r. For he res of he dscusson our goal s o mnmze he cos under eher he dscouned model or lmng average model. III. SYSTEM MODELING We assume he sysem s embedded n an envronmen where here s only a sngle source of requess whch s defned as he servce requesor (SR). Requess ssued by he SR are servced by he sysem. The sysem self consss of hree componens: a resource ha processes requess (he SP) a queue whch sores he requess ha canno be servced mmedaely upon arrval () and a power manager (PM). Boh he reques arrval even and reques servce even are sochasc processes. We assume ha hey follow he Posson process (.e. durng me ( ] he number of he evens has he Posson dsrbuon wh mean λ). Consequenly he reques ner-arrval me (from he SR) and he servce me (he me needed by he SP o servce a reques) follow he exponenal dsrbuon wh mean /λ. In order o be more general n our model we assume ha he SP has more han one workng mode herefore can servce he
requess wh more han one servce speed. We also assume ha all requess have he same servce prory. The servce of he requess are based on a FIFO order. The SP can operae n a number of dfferen power modes. We assume ha he me needed for he SP o swch from one sae o anoher follows he exponenal dsrbuon. The PM s a conroller ha reads he sysem sae (he on saes of he SP and he ) and ssues mode-swchng commands o he SP. In he remander of hs paper we wll use upper case bold leers (e.g. M) o denoe marces lowercase bold leers (e.g. v) o denoe vecors alczed Aral leers (e.g. S) o denoe ses uppercase alczed leers (e.g. S) o denoe scalar consans and lower case alczed leers (e.g. x) o denoe scalar varables. The Servce Requesor (SR) has only one reques generang mode. The average nerval me of requess generaed by SR follows he exponenal dsrbuon wh mean value /λ. An SR model wh one reques generang mode s a smplfed model of a real SR whose reques generang speed vares from me o me dependng on he workload. From our observaons however we fnd ha he average ner-arrval me of a gven Posson process can be esmaed whn 5% error afer observng 5 evens. Therefore f he npu says sable long enough he power manager can observe and esmae he npu rae dynamcally and adapvely change s polcy. The Servce Provder (SP) s modeled as a saonary connuousme conrollable Markov process wh sae (operaon mode) se S={s s.. = S} acon se A and generaor marx G SP ( a A. I can be descrbed by a quadruple (χ µ(s) pow(s) ene(s s )) where: () χ s an S S marx; () µ(s) s a funcon µ: S ; () pow(s) s a funcon pow: S ; (v) ene(s s ) s a funcon eng: S S. We call χ he swchng speed marx. The ()h enry of χ s denoed as χ and represens he swchng speed from sae s s s o s. The average swchng me from sae s o sae s s hen / χ. We se χ o be because he swch from sae s o s s s s self s nsananeous. The enres of he parameerzed generaor marx G SP ( can be calculaed as: σ s s ( a ) = δ ( s χ s s s s ; σ ( = σ s s a ) s s s s δ(s equals o f s s he desnaon sae of a oherwse δ(s equals o. A servce rae µ(s) represens he servce speed of SP n sae s. Therefore /µ(s) gves he average me ha s needed by SP o complee he servce for one reques when SP s n sae s. A power consumpon rae pow(s) s assocaed wh each sae s S. I represens he power consumpon of SP durng he me occupes sae s. The cos rae c ss of sae s s equal o pow(s). A swchng energy ene(s s ) s assocaed wh each sae par (s s ) s s S s s. I represens he energy needed for SP o swch from sae s o sae s. The cos c s s s equal o ene(s s ). From Eqn. (3.5) we know ha he expeced power consumpon of SP when s n sae s and acon a s s chosen can be calculaed by: c = pow s) + σ ( a ) ene( s s ). s ( s s s s s We can dvde he sae se S no wo groups: () The se of acve saes Sacve where µ(s a ) s larger han for each s a Sacve. () The se of nacve saes Snacve where µ(s na ) s for each s na Snacve. Furhermore we can dvde he marx G SP ( no wo pars: AA G ( G ( = SP SP IA GSP ( AI G SP( II GSP( AA SP where marx G ( conans he ranson rae for ransons beween nacve saes. Marx G SP ( conans he ranson raes for ransons from any nacve sae o any acve sae. IA SP II SP G ( and G ( are defned smlarly. Example 4. Consder a SP wh hree saes S={acve wang sleepng}. Laer we wll also denoe acve as A. wang as W. sleepng as S.. Le he acon se be defned as A={wakeup wa sleep}. Assume ha all hree commands are vald n any sae. The swchng speed marx χ s a 3 3 marx. The power consumpon rae pow(s) can be represened by a vecor. The swchng energy ene(s s ) can be represened by a wo-dmensonal able. Assume ha he chosen polcy s: {<A. wa> <W. sleep> <S. wakeup>} Fgure gves an llusraon of hs Markov process. Noe ha he self-loops are no shown n hs fgure. sleepng wang Fgure Markov process model of he SP The Servce Queue () s modeled as a saonary connuousme conrollable Markov process wh sae se Q=Qsable Qransfer where Qsable={q s.. = Q} Qransfer={q - s.. = Q} and he generaor marx G (s a (qs) ) where s s he SP sae a (qs) s he acon when SP s n sae s and s n sae q. The model of s consruced based on ha of an M/M/ queue. The queue lengh s Q. We assume ha he reques wll be los f he s full a he me he reques arrves. The sae se Qransfer s he se of ransfer saes whch represen he saes of he when he servce of a reques has been fnshed and he servce of he nex reques has no sared. Noe ha here s a concurrency consran beween he and he SP as follows: Whenever he s n a ransfer sae he SP s ransonng from one sae o nex. Furhermore he leaves he ransfer sae exacly when he SP ranson s complee. The sae se Qsable s he se of sable saes.e. saes of he oher han he ransfer saes. We denoe a sable sae as q whch also mples ha here are requess n he. Gven he sae of SP he ranson raes beween he saes of are fxed. There are four ypes of possble ransons. They are: () The ranson from sable sae q o sable sae q +. The wll make hs ranson when s n sae q and a reques s generaed by he SR. The ranson rae s: σ q = λ = AI q + Q- where λ s he reques generaon rae of SR. () The ranson from sable sae q o ransfer sae q -. The wll make hs ranson when s n sae q and he servce for he curren reques s compleed by he SP. The ranson rae s: σ = ( ) = Q q s q µ.9 acve
(3) The ranson from ransfer sae q - o sable sae q -. The wll make hs ranson when s sae q - and he SP complees swchng and sars o provde servce for he nex reques. The ranson rae s: σ = χ = Q- q q s s where s s he desnaon sae of acon a (qs). (4) The ranson from ransfer sae q - o ransfer sae q +. The wll make hs ranson when s n sae q - and a reques s generaed by he. The ranson rae s: σ q = λ q = Q-. + For he sake of brevy we do no descrbe he boundary case when he s n sae q Q Q- and a reques s generaed. Transon beween saes oher han hese four classes has a rae of. The self-ranson rae can hen be calculaed as: σ = q q Q. q q σ q q q q Based on he above groupng we can dvde he marx G (s no four pars: SS SS G ( s G ( s = TS G ( s ST G ( s where marx TT G ( s G ( s conans he ranson rae for ransons beween ST sable saes. Marx G ( s conans he ranson raes for TS ransons from any sable sae o any ransfer sae. G ( s TT and G ( s are defned smlarly. Example 4.3 Consder he SP model gven n prevous examples and assume a maxmum queue lengh of. Assume ha whenever he s n ransfer sae he PM wll ssue he sleep command. Fgure gves he llusraon of he Markov process model of hs. Noe ha he self-loops are no shown n he fgure. λ q q χ A.S. q µ(s) χ A.S. λ Fgure Markov process model of he The Power-Managed Sysem (SYS) can be modeled as a saonary connuous-me conrollable Markov process whch s he composon of he Markov processes of he SP and he. The sae se s gven X=S Qsable Sacve Q ransfer. The acon se s he same as ha n he SP model. A parameerzed generaor marx G SYS (x gves he sae ranson raes under acon a. A cos funcon Cos(x gves he sysem cos under acon a when he SYS s n sae x. There s an acon se A x assocaed wh each sae x. When he sysem s n sae x he PM chooses a command from he A x. The acon gves he mode of SP ha should swch o. No all acons are vald n all saes. Consrans on a vald acon are as follows: () When he s n sable sae he SP canno swch from acve sae o nacve sae. () When he s n sable sae q Q ( s full) he SP canno swch from an nacve sae o anoher nacve sae wh longer wakeup me. λ q µ(s) q (3) When he s n ransfer sae q Q Q- he SP canno swch from an acve sae o anoher acve sae wh longer servce me. The frs consran ensures ha he work of SP wll no be nerruped by he command ssued by he PM and all commands ssued by he PM wll be acceped by he SP wh probably.the las wo consrans ensure ha he resulng SYS model s a conneced Markov process. Consequenly he lmng dsrbuon of he sae probably exss and s ndependen of he nal sae [7]. These wo consrans are also reasonable because when SP and he are n hese forbdden saes hen he servce speed canno follow he generaon speed of he requess. Therefore we need o ncrease he servce speed. There s some dependence beween he Markov process model of and he Markov process model of SP because he ransfer saes of are assocaed wh he acve saes of SP and her ransons are synchronzed. When he s n sable sae however he s ndependen from he SP. Defnon 4.4 Consder wo marces A and B: a a b b A = and B = a a. The ensor produc b b ab ab C=AB s gven by C =. The ensor sum ab ab C=A B s gven by: C = A I + I B where n s he n n order of A n s he order of B I s he deny marx of order n. We can wre he generaor marx G SYS ( as (please refer o [8] for proof) : SS G SP ( G ( M( G SYS ( = A TT GSP ( N I S G ( acve ST I G ( ) M = a A AA AI I Q O G SP = GSP GSP O O s a S nacve (Q+) (S acve Q) marx of all zeros O s a column vecor of all zeros. The dagonal enres of G SYS ( are calculaed as: σ ( a ) = σ (. S acve [ ] n N = [ ] The cos of he sysem s relaed o he sae x of he SYS and he acon a aken by he SYS n sae x. We use he average power consumpon C pow (x and he average number of wang requess C sq o capure he sysem cos. Le x be denoed by (s q) where s S q Q. The power cos can be calculaed as: C ( x = pow( s) + σ ( ene( s s'. pow s s ) s' S s' s The delay cos s: C sq = when s n sable sae q or ransfer sae q +. We defne a oal cos as a weghed summaon of he power and delay coss Cos(x= C pow (x+w C lsq (x) (3.) IV. POLICY OPTIMIZATION The problem of power managemen s o fnd he opmal se of sae-acon pars for he PM such ha he expeced power consumpon s mnmzed subec o he performance consrans. Ths problem can be formally wren as: mn lm p x X x x ( τ ) C pow ( x a π )
s.. lm p x X π x x ( τ ) C sq ( x) D M x x X Where p x x (τ ) s he sae ranson (drec or ndrec) probably from sae x o x n a me perod of τ under polcy π. a π n C pow (xa π ) denoes he acon n sae x. Anoher problem formulaon s: mn lm px x ( τ ) Cos( x a ) x x X ' x X By adusng he weghs n Eqn. (3.) we can acheve mnmum power under dfferen delay consrans. Fgure 3 gves he workflow of our polcy opmzaon algorhm. The polcy eraon algorhm s he same as ha n [9]. Deals are omed here o save space. Sysem Model Polcy Ieraon Algorhm Does he polcy mee he performance consran? YES Oupu opmal polcy Fgure 3 Polcy opmzaon workflow Increase he weghs of delay n (3.) NO V. EXPERIMENTAL RESULTS Frs we descrbe a class of heursc polces ha can gve rade off beween power and performance. An N-polcy s a polcy ha acvaes he server when here are N cusomers wang for servce and deacvaes he server when here are no cusomer n he sysem []. When he server has only wo saes: acve and sleepng can easly be shown ha he N-polcy gves he mnmum power compared o oher saonary polces wh he same performance consran. Our expermens show ha however for a sysem wh more han wo server saes he N- polcy does no gve he opmal power-delay radeoff. Our expermenal seup s as follows. We have wren an evendrven smulaor for smulang he real-me operaon of a porable sysem ogeher wh he power managemen polcy. The smulaor smulaes he operaons of he server he queue and he power manager under real-me npu requess. The server has hree saes: acve wang and sleepng. We se he lengh of he queue o 5. Tasks are represened by a sequence of evens. The nerval me beween wo consecuve requess s generaed randomly o follow an exponenal dsrbuon wh mean value of 6sec. Therefore λ=.67 n he sochasc model of he sysem. The oal number of requess s 5. The servce me of each ask s also generaed accordng o an exponenal dsrbuon wh mean value.5sec. Therefore µ(acve)=.67 n he sochasc model of he sysem. When he sysem sae changes he power manager s rggered and a new command s ssued accordng o he curren sysem sae. The swchng me of he server s also generaed randomly. Eqn. (4.) ( gves he expermenal value of he average swchng me. The me s gven n seconds. Noe ha hese refer o he values of /χ n he sochasc model. We se he server power dsspaon when he server s acve wang and sleepng o 4w 5w and.w respecvely. These values are assgned o he correspondng cos raes c n he sochasc model. The energy needed for each ranson (gven n J) s gven n Eqn. (4.) (b). These values are assgned o he correspondng ranson coss c n he model. The performance and he cos mercs are measured by he average number of wang requess and he average power dsspaon of he sysem durng he smulaon.....5 ( r _ me =.5. (b) r _ energy =. (4.)..5 5 In he frs expermen we changed he value of he performance wegh of our algorhm and obaned a se of opmum polces. We also generaed a se of N-polces usng N= 5. Fgure 4 shows he comparson of he smulaed values of perfomance and power of he wo ses. Noe ha he lefmos (rghmos) N-polcy soluon n Fgure 4 corresponds o N= (N=5). We fnd our polcy gves beer power-delay radoff han he N-polcy. In he expermens we also calculaed he funconal value of he queue lengh and energy cos (by usng he sae probably and he sae cos) and found ha he funconal value and he smulaed value are almos he same. Ths shows ha our sochasc model of he power-managed sysem maches he real suaon very well. Power (w) 4.5 4 3.5 3.5.5 Power Performance Trade Off Pars N-polcy our polcy.5.5.5.5 3 3.5 Average Number of Wang Requess Fgure 4 Comparson of our polcy and The N-polcy In he second expermen we assume ha he performance consran of he sysem s o keep he average oupu rae (hroughpu) he same as he npu rae. Tha s he average me ha each ask says n he queue (.e. average wang me) should be equal o or less han he average ner-arrval me of asks. In he algorhm he performance consran s defned n erms of he average number of wang requess. Therefore we mus conver he average wang me o he average number of wang requess. We used he approxmaaon ha he average number of wang requess equals he npu rae mes he average wang me of each reques. Table gves he smulaed values of he average wang me and he correspondng queue lengh. I shows ha he approxmaon s whn 5% error of he acual value. In he las expermen we used a se of npu asks where he npu rae vared from /8 o /3. The correspondng average nerval me of he asks vared from 8sec o 3sec. We compared he power-delay curves for our polcy wh four heursc algorhms. Among heursc approaches one s a greedy algorhm whch
deacvaes (acvaes) he server as soon as he queue s empy (he queue s no empy). The oher hree are me-ou polces whch deacvae he server n seconds afer becomes dle. In me-ou polcy () n s fxed o sec. In polcy () n s equal o he average ner-arrval me of he npu asks. In polcy (3) n s equal o half of he ner-arrval me of npu asks. Fgure 5 shows he smulaed value of power and he average wang me. We can see ha our algorhm gves bes power dsspaon whle sasfyng he performance consran. Table Comparson of real average queue lengh and he approxmaed average queue lengh Inpu Rae (/sec) /8 /7 /6 /5 /4 /3 Avg. Wang Tme (sec) 6.493 6.8 5.658 5.8 3.5 3.3 Aprox. # of Wang Requess.8.868.943..875. Acual # of Wang Requess.86.869.94.53.86.5 Error of Apporxmaon(%) -.6 -..3-4.9.6 4.7 VI. Power (w) 3.5 Average Wang Tme (sec) 3.5.5.5 9 8 7 6 5 4 3 Power Dsspaon a Dfferen Inpu Raes our polcy greedy polcy me-ou polcy () me-ou polcy () me-ou polcy (3) 3 4 5 6 7 8 9 Iner-arrval Tme for Task (sec) ( Performance of Polcy a Dfferen Inpu Rae our polcy greedy polcy me-ou polcy () me-ou polcy () me-ou polcy (3) 3 4 5 6 7 8 9 Iner-arrval Tme for Task (sec) (b) Fgure 5 Comparson of our polcy and heursc polces CONCLUSION We have proposed a new sysem model and mehod for dynamc power managemen n sysem-level. The problem of sysem-level power managemen was formulaed as he connuous-me Markov decson process based on he heores of connuous-me Markov decson process and sochasc nework. Compared o prevous work our model can represen he sysem behavor more nuvely and more accuraely by consderng he close relaonshp beween he server saus and he queue saus. By modelng he sysem as a queue n he doman of connuous-me he parameers n he model become more realsc such ha hey can be colleced easly and precsely. Expermenal resuls were presened o show ha our approach s more flexble and more effecve han heursc approaches o acheve he bes powerperformance radeoff. REFERENCES [] A. Chandrakasan R. Brodersen Low Power Dgal CMOS Desgn Kluwer Academc Publshers July 995. [] M. Horowz T. Indermaur and R. Gonzalez Low-Power Dgal Desgn IEEE Symposum on Low Power Elecroncs pp.8-994. [3] A. Chandrakasan V. Gunk and T. Xanhopoulos Daa Drven Sgnal Processng: An Approach for Energy Effcen Compung 996 Inernaonal Symposum on Low Power Elecroncs and Desgn pp. 347-35 Aug. 996. [4] J. Rabaey and M. Pedram Low Power Desgn Mehodologes Kluwer Academc Publshers 996 [5] L. Benn and G. De Mchel Dynamc Power Managemen: Desgn Technques and CAD Tools Kluwer Academc Publshers 997. [6] Inel Mcrosof and Toshba Advanced Confguraon and Power Inerface specfcaon URL: hp://www.nel.com/al/powermgm/specs.hml 996 [7] U. Narayan Bha Elemens Of Appled Sochasc Processes John Wley & Sons Inc. 984 [8] B. Mller Fne Sae Connuous Tme Markov Decson Processes Wh an Fne Plannng Horzon. SIAM J. Conrol Vol. 5 No. pp. 66-8 968. [9] B. Mller Fne Sae Connuous Tme Markov Decson Processes Wh an Infne Plannng Horzon. J. Of Mahemacal Analyss and Applcaons No. pp. 55-569 968. [] R.A.Howard Dynamc Programmng and Markov Processes Wley New York 96 [] G. A. Paleologo L. Benn e.al Polcy Opmzaon for Dynamc Power Managemen Proceedngs of Desgn Auomaon Conference pp.8-87 Jun. 998. [] D. P. Heyman M. J. Sobel Sochasc Models n Operaons Research McGraw-Hll Book Company 98 [3] L. Benn A. Boglolo S. Cavallucc B. Rcco Monorng Sysem Acvy For OS-Dreced Dynamc Power Managemen Proceedngs of Inernaonal Symposum of Low Power Elecroncs and Desgn Conference pp. 85-9 Aug. 998. [4] L. Benn R. Hodgson P. Segel Sysem-level Esmaon And Opmzaon Proceedngs of Inernaonal Symposum of Low Power Elecroncs and Desgn Conference pp. 73-78 Aug. 998. [5] G. Bolch S. Grener H. D. Meer and K. S. Trved Queueng Neworks and Markov Chans John Wley & Sons Inc. 998 [6] M. Srvasava A. Chandrakasan. R. Brodersen Predcve sysem shudown and oher archecural echnques for energy effcen programmable compuaon" IEEE Transacons on VLSI Sysems Vol. 4 No. (996) pages 4-55. [7] C.-H. Hwang and A. Wu A Predcve Sysem Shudown Mehod for Energy Savng of Even-Drven Compuaon Proc. of he Inl. Conference on Compuer Aded Desgn pages 8-3 November 997. [8] Q. Qu Q. Wu and M. Pedram Dynamc Power managemen: A Connuous-Tme Sochasc Approach USC EE-Sysems Dep. CENG 99-.