IEEE JOURAL O SELECTED AREAS I COMMUICATIOS, VOL. 33, O. 3, MARCH 205 467 Optmum Polces for an Energy Harvestng Transmtter Under Energy Storage Losses Kaya Tutuncuoglu, Student Member, IEEE, Ayln Yener, Fellow, IEEE, and Sennur Ulukus, Member, IEEE Abstract We consder an energy harvestng network where the transmtter harvests energy from nature, and the harvested energy can be saved n an mperfect battery whch suffers from chargng/ dschargng neffcency. In partcular, when E unts of energy s to be stored n the battery, only ηe unts s saved and ( η)e s lost due to chargng/dschargng neffcency, where 0 η represents the storng effcency. We determne the optmum offlne transmt power schedule for such a system for sngle-user and broadcast channel models, for statc and fadng channels, wth and wthout a fnte battery sze. We show that the optmum polcy s a double-threshold polcy: specfcally, we store energy n the battery only when the harvested energy s above an upper threshold, and retreve energy from the battery only when the harvested energy s below a lower threshold; when the harvested energy s n between these two thresholds, we use t n ts entrety n the current slot. We show that the two thresholds reman constant unless the battery s depleted or full. We provde an algorthm to determne the sequence of optmum thresholds. For the case wth fadng, we develop a drectonal water-fllng algorthm whch has a double-threshold structure. Fnally, we formulate the onlne problem usng dynamc programmng, and numercally observe that the onlne polcy exhbts a double-threshold structure as well. Index Terms Energy harvestng communcatons, optmal packet schedulng, nodes wth rechargeable batteres, neffcent energy storage. I. ITRODUCTIO WE COSIDER an energy harvestng network where the transmtter harvests energy from nature to sustan ts operaton. In partcular, the transmtter uses the energy harvested from nature to transmt ts data packets. Such energy harvestng capabltes brng new constrants nto the communcaton problem n the physcal layer n the form of energy causalty and no-energy-overflow condtons. The frst constrant mposes that the energy that has not yet been harvested cannot be used for communcaton, and the second constrant mposes that no harvested energy should be allowed to overflow due to a fnte battery sze. Therefore, n such energy harvestng systems, packet Manuscrpt receved Aprl, 204; revsed September 5, 204; accepted December 6, 204. Date of publcaton January 3, 205; date of current verson March 9, 205. Ths work was supported by SF Grants CS 09-64364/ CS 09-64632 and CCF 4-22347/CCF 4-22. Ths paper was presented n part at the 46th Conference on Informaton Scences and Systems, Prnceton, J, USA, March 202, and the Aslomar Conference on Sgnals, Systems, and Computers, Pacfc Grove, CA, USA, ovember 202. K. Tutuncuoglu and A. Yener are wth the Department of Electrcal Engneerng, Pennsylvana State Unversty, Unversty Park, PA 6802 USA (e-mal: kaya@psu.edu; yener@ee.psu.edu). S. Ulukus s wth the Department of Electrcal and Computer Engneerng, Unversty of Maryland, College Park, MD 20742 USA (e-mal: ulukus@ umd.edu). Color versons of one or more of the fgures n ths paper are avalable onlne at http://eeexplore.eee.org. Dgtal Object Identfer 0.09/JSAC.205.2395 schedulng and the correspondng energy management scheme must be carefully optmzed n the physcal layer n order to guarantee a certan optmum performance. Energy constraned communcaton was studed wdely n the lterature [] [6], ncludng fnte horzon scenaros wth delay constrants [7] [0]. The offlne energy management problem n an energy harvestng settng was frst formulated n [], whch consdered the problem of mnmzng the transmsson completon tme for a gven number of data packets n an offlne settng. Ths reference ntroduced the energy causalty constrant, and showed that the transmtter should use as constant power as possble, subject to energy causalty mposed by the energy harvestng profle. For a fnte-szed battery, [2] formulated the throughput maxmzaton problem n a smlar communcaton settng. Reference [2] ntroduced the noenergy-overflow constrant due to the fnte-szed battery, and showed that the transmtter should use as constant power as possble subject to energy causalty and no-energy-overflow constrants, partcularly, usng harvested energy slow enough not to volate energy causalty but fast enough to open up space n the fnte-szed battery and cause no energy overflows. Reference [3] consdered a fadng channel and developed a drectonal water-fllng algorthm where energy (water) s flled over the fadng profle, wth a drectonal flow of water to the rght only, due to energy causalty constrants: energy can be saved and used n the future, but the energy that wll be harvested n the future cannot be used earler; see also [4] for a treatment of the fadng case. Ths lne of work has been extended for broadcast channels n [5] [7], multple access channels n [8], nterference channels n [9], two-hop relay channels n [20] [26]. The effects of crcut power have been consdered n [27] [30], where the transmtter ncurs energy loss by beng on,.e., when the transmt power s non-zero. Ths, then, dsfavors long and constant stretches of transmt powers as ths ncreases crcut energy consumpton. Reference [27] shows the optmalty of a drectonal glue-pourng algorthm n ths case. Ths lne of offlne energy management has also been extended to energy cooperaton n [3] where users transfer energy to one another, leadng to two-dmensonal drecton water-fllng n two-way and multple access channels; see also bdrectonal cooperaton n [32], [33]. Recever sde energy harvestng has been consdered n [34], [35]. Common n all of these works s the assumpton that the energy harvestng nodes have perfect energy storage unts (batteres) nto whch energy can be stored wthout any losses and n whch energy can be saved wthout any leakages untl t s eventually used. In ths paper, we consder energy storage mperfectons n the form of chargng/dschargng neffcency and ther effects on 0733-876 205 IEEE. Personal use s permtted, but republcaton/redstrbuton requres IEEE permsson. See http://www.eee.org/publcatons_standards/publcatons/rghts/ndex.html for more nformaton.
468 IEEE JOURAL O SELECTED AREAS I COMMUICATIOS, VOL. 33, O. 3, MARCH 205 the offlne throughput maxmzaton problem. Energy storage/ retreval mperfectons can manfest themselves n many dfferent ways [36], [37], for nstance: mperfectons n energy converson from one technology to another, chargng/dschargng mperfectons (only a porton of the avalable energy can be saved n the battery at the tme of chargng), energy leakage over tme (saved energy s leaked and lost over tme), battery sze degradaton (battery capacty sze gets smaller at every recharge), etc. The frst work to formulate a form of practcal energy storage neffcency n the context of offlne throughput maxmzaton s [38]. In [38], two forms of storage mperfectons are consdered: leakage of saved energy over tme and battery degradaton. The major effect of such mperfectons on the throughput maxmzaton problem s that they modfy the energy feasblty tunnel, whch s the tunnel that s formed by the energy causalty upper starcase and the no-energy-overflow lower starcase [2]. The dstance between these two starcases s exactly the sze of the battery, E max, n the case of perfect storage [2]. Reference [38] demonstrated that, n the cases of energy leakage and battery degradaton, the energy feasblty tunnel gets narrower by upper starcase decreasng and lower starcase ncreasng over tme, and developed the optmum offlne power allocaton polcy that maxmzes the throughput. The mperfectons studed n [38] are long-term effects on energy storage, that affect communcatons n duratons much larger than typcal symbol duratons. In ths paper, we study another class of energy storage neffcences, whch occur at the tme of energy storage, almost nstantaneously, at much shorter tme duratons. In partcular, we consder the neffcency (loss) that occurs at the tme of chargng/dschargng : when E unts of energy s to be stored n the battery, only ηe unts s saved and ( η)e s lost nstantaneously due to chargng/dschargng neffcency, where 0 η represents the storage effcency. Dependng on the technology used n energy storage, η can be as low as 66% [36], [37], [39], [40]. Such losses have been consdered n communcatons n [4], [42] for duty-cyclng wth constant transmsson rate under energy neutralty condtons, but not n the context of offlne throughput maxmzaton. In ths paper, we consder the offlne throughput maxmzaton under such losses and determne the correspondng optmum energy management polces. We start wth a sngle-user Gaussan channel wth nfnte-szed battery (Secton III), then consder a fnte-szed battery (Secton IV), extend to fadng channels (Secton V), and extend to a mult-user broadcast settng (Secton VI). The effects of mperfectons at chargng/dschargng consdered n ths paper are sgnfcantly dfferent than leakage/ degradaton mperfectons studed n [38]. In partcular, whle leakage/degradaton mperfectons affect the shape of the energy feasblty tunnel, n our case, the energy feasblty tunnel s unaffected. Instead, n our case, we need to determne, what porton of the ncomng energy to store despte storage losses, and how to use the stored energy. We show that the optmal power polcy has a double-threshold structure: whenever the harvested energy s below a lower threshold, we use a constant Whle we consder both neffcences at the tme of chargng/dschargng, we show that, from a mathematcal pont of vew, these two mperfectons can be clubbed together nto a sngle effectve neffcency only at the tme of chargng. transmt power equal to that threshold by retrevng energy from the battery; whenever the harvested energy s above an upper threshold, we use a constant transmt power equal to that threshold by storng some of the harvested energy; and whenever the harvested energy s between these two thresholds, we use the harvested energy for transmsson n ts entrety wthout storng any of t n the battery. It then only remans to determne these thresholds. These thresholds change throughout the communcaton sesson and depend on the harvested energy profle and the storage effcency η. We dentfy the propertes these threshold should satsfy, and then provde an algorthm to determne these thresholds. In partcular, we show that the optmal thresholds are constant between battery events,.e., they change only when the battery s depleted or the battery s full, and may only ncrease f the battery s empty and may only decrease f the battery s full. In the case of a fadng channel, we develop a modfed verson of drectonal water-fllng whch takes ths double-threshold polcy nto account. For the broadcast channel, we determne the largest throughput regon by employng double-threshold polces on weghted sum rate maxmzaton problems. Overall, we observe that contrary to the results of prevous work wth deal batteres [] [7], where the optmal polces were shown to be pecewse constant, here, the optmal polcy may favor transmttng mmedately wth the harvested energy,.e., wthout storng or retrevng energy to/from battery. In essence, the thresholds n the double-threshold polcy defne an nterval wthn whch storng energy s not worth ncurrng the storage losses. Hence, our work demonstrates how optmal polces need to adapt to the trade-off between schedulng and storage neffcency. Fnally, we formulate the onlne verson of the problem as a dynamc program (Secton VII). We observe numercally that the soluton of the onlne dynamc problem formulaton also has a double-threshold structure. Due to the complexty of dynamc programmng solutons, we propose smpler threshold-based polces and evaluate ther performance va smulatons (Secton VIII) and observe that they perform near-optmal. II. SYSTEM MODEL Tme s slotted wth unt slot length τ = over a fnte sesson of tme slots. 2 The system model s shown n Fg.. At the begnnng of the th tme slot, the transmtter harvests an energy n the amount of E 0 unts. It retreves an addtonal r unts of energy from the battery, and allocates s for storage n the battery. Ths leaves the energy p = E s + r () for transmsson n the th tme slot. The battery has a storng effcency of 0 η : when s unts of energy s allocated for storng, only ηs unts can be stored for future use and ( η)s unts of energy s lost due to storage neffcency. 3 2 We consder unt length tme slots solely for ease of presentaton. The results extend trvally to any postve slot length τ > 0. 3 Smlarly, a loss may occur when energy s retreved from the battery. In ths work, these two losses are combned n the model n η, whch effectvely represents the fracton of energy that can be drawn from the battery per unt energy stored.
TUTUCUOGLU et al.: OPTIMUM POLICIES FOR A EERGY HARVESTIG TRASMITTER UDER EERGY STORAGE LOSSES 469 Fg.. Energy harvestng transmtter wth neffcent storage and fnte-szed battery over a Gaussan channel. The power polcy of the node conssts of energy values chosen for storage and retreval, namely s for storage and r for retreval, respectvely, at tme slots =,...,. From these two varables, transmt power p s calculated usng () for each tme slot of unt duraton. ote that by defnton, s 0, r 0 and p 0for =,...,. From (), the last condton mposes that E + r s 0onr and s. Furthermore, the energy drawn from the battery cannot exceed the energy stored n the battery up to any tme slot. Followng [], we refer to ths constrant as energy causalty. Let the ntal charge of the battery be E 0. Denotng the amount of energy n the battery at tme slot as B, the energy causalty constrants are gven by B = E 0 + j= (ηs j r j ) 0, =,...,. (2) In addton, the battery has a maxmum storage capacty (sze) of E max, and the energy n excess of ths capacty s lost f attempted to be stored. Clearly, t s sub-optmal to allow such energy overflows, and t s shown n [2] that restrctng power polces to those avodng overflows yelds an optmal polcy. Hence, we enforce a set of no-energy-overflow [2] constrants B = E 0 + j= (ηs j r j ) E max, =,..., (3) whch ensure that the energy allocated for storage does not exceed the capacty of the battery at any tme slot. We consder an addtve whte Gaussan nose communcaton channel wth a fadng coeffcent of h at tme slot. Wth allocated transmt power p and fadng h, the communcaton rate n a slot s gven by g(p)= log( + hp). (4) 2 Under ths channel model, we consder the problem of maxmzng the average throughput of ths system,.e., maxmzng the average of g(p ) over a duraton of tme slots, by choosng the optmal power polcy {(s,r )} =. Ths requres adaptng the power polcy to the harvestng process, neffcent storage, and channel coeffcents. III. OPTIMAL TRASMISSIO POLICY FOR A IFIITE-SIZED BATTERY, E max = We frst consder a non-fadng channel,.e., h = h for all, and an nfnte-szed battery, E max =. The throughput maxmzaton problem for the model n Fg. over an -slot communcaton sesson s expressed as max {s,r } = s.t. E 0 + g(e s + r ), j= (5a) (ηs j r j ) 0, =,...,, (5b) E s + r 0, =,...,, (5c) s 0, r 0, =,...,, (5d) where g(p) s gven n (4). We frst present the followng lemma whch states that t s sub-optmal to store and retreve energy smultaneously n the same tme slot for η <. The η = case s omtted snce effcent storng and restorng n the same tme slot s equvalent to not storng for ths case. Lemma : For η <, the soluton to (5) satsfes s r = 0for all,.e., the optmal polcy never stores and retreves energy smultaneously. Proof: Let {(s,r )} = be a feasble power polcy whch satsfes s j r j > 0forsome j.let s j =[s j r j /η] +, r j =[r j ηs j ] + (6) where [x] + = max(0,x). For all j, let s = s and r = r. ote that the battery dynamcs n (2) are unaffected by ths change, snce ηs r = η s r, for all. Therefore, the polcy {( s, r )} = s feasble. On the other hand, the resultng transmt power p j at tme slot j becomes { E p j = E j s j + r j = j s j + r j /η, f ηs j r j, (7) E j ηs j + r j, otherwse, and consequently p j > p j due to η <, s j > 0 and r j > 0. Snce the rate g(p) s ncreasng n p, wehaveg( p j ) > g(p j ), whle g( p )=g(p ) for j. Hence, the power polcy {( s, r )} = acheves a larger throughput than {(s,r )} =, and the latter polcy cannot be optmal. Lemma shows that we can restrct our search for the optmal polcy to those that do not store and retreve energy smultaneously at any tme. We remark that smultaneously chargng and dschargng a battery may or may not be physcally possble, but through Lemma, we show that t s mathematcally sub-optmal for our problem. A. Double-Threshold Polcy We next observe a property of the optmal power polcy. Snce all constrants are lnear and g(p) s concave n p, the problem n () s a convex optmzaton problem. Hence, Karush-Kuhn-Tucker (KKT) condtons are necessary and suffcent for optmalty. The Lagrangan of (5) s ( ) L = (g(e s + r )+λ E 0 + (ηs j r j ) = j= + µ (E s + r )+σ s + ν r ), (8) where λ, µ, σ, and ν, =,..., are non-negatve Lagrange multplers correspondng to the energy causalty,
470 IEEE JOURAL O SELECTED AREAS I COMMUICATIOS, VOL. 33, O. 3, MARCH 205 non-negatvty of power, and non-negatvty of stored and retreved energy, respectvely. The KKT optmalty condtons are found by takng the dervatves wth respect to s and r for =,..., as h + η + hp λ j µ + σ =0, j= =,...,, (9) h λ j + µ + ν =0, + hp =,...,, (0) j= wth the complementary slackness condtons ) λ (E 0 + (ηs j r j ) =0, =,...,, (a) j= µ (E s + r )=0, =,...,, (b) σ s = 0, ν r =0, =,...,. (c) From (9) and (0), we fnd the optmal transmt powers p as p = η j= λ j µ + σ h = j= λ, j µ ν h =,...,. (2) We defne two sets of thresholds, p s and p r,as p s = η j= λ, j h =,...,, (3a) p r = j= λ, j h =,...,. (3b) ote that these varables satsfy p s p r, =,..., (4) and are related as + hp r = η, + hp s =,...,. (5) We note that whenever p > 0, we have µ = 0 from (b). Then, from the frst equalty n (2), snce σ 0, we have p p s. Smlarly, from the second equalty n (2), snce ν 0, we have p p r. Therefore, for p > 0, we have p s p p r. (6) We refer to p s and p r as thresholds: when transmt power p > 0, t must be larger than the lower threshold p r, and smaller than the upper threshold p s. In the followng lemma, we show that chargng and dschargng are also related to these thresholds n the optmal polcy. Lemma 2: Whenever the battery s beng charged,.e., s > 0, a non-zero transmt power must satsfy p = p s. Conversely, whenever the battery s beng dscharged,.e., r > 0, a non-zero transmt power must satsfy p = p r. Proof: For a non-zero transmt power p > 0, due to (b) we have µ = 0. When the battery s beng charged,.e., s > 0, from (c), we get σ = 0. Substtutng ths n the frst equalty n (2) yelds p = p s. When the battery s beng dscharged,.e., r > 0, from (c), we get ν = 0. Substtutng ths n the second equalty n (2) yelds p = p r. Due to Lemma 2, we call p s the storng threshold and p r the retrevng threshold. We observe from Lemma that we have ether s > 0 and r =0, or s =0 and r >0, or s =0 and r =0. When s =r =0, from (), we have p =E, whch must satsfy (6). These condtons show that there s a double-threshold polcy on p. Specfcally, when the battery s beng charged, the transmt power equals the storng threshold p s ; and when the battery s beng dscharged, the transmt power equals the retrevng threshold p r. If the battery s nether beng charged or dscharged,.e., the battery s passve, then p = E,.e., the transmtter uses all the harvested energy n the current slot. Theorem : The power polcy solvng (5) has the followng double-threshold structure: ) If E > p s, then p =[p s ] +. Consequently, s = E [p s ] + > 0 and r = 0 (storng). 2) If E < p r, then p = p r. Consequently, s = 0 and r = p r E > 0 (retrevng). 3) If p s E p r, then s = r = 0 and p = E (passve). Proof: We prove each case separately: a) Consder the case E > p s. From () and Lemma, we have s E. We consder the three dstnct cases s = 0, 0 < s < E and s = E as follows. When s = 0, from (), we get p E. Together wth E > p s, ths contradcts (6). Therefore, ths case cannot be optmal. When E > s > 0, from () we get p > 0 and from Lemma 2 we have p = p s. Fnally, when s = E and therefore r = 0, () yelds p = 0. Snce σ = 0 from (c), substtutng n (2) gves p s p = 0. Hence, for all possble sub-cases n ths case, p =[p s ] +. b) Consder the case E < p r. We consder the three dstnct cases, r = 0 and p > 0, r = 0 and p = 0, and r > 0as follows. When r = 0 and p > 0, from (), we get p E. Together wth E < p r, ths contradcts (6), and therefore ths case cannot be optmal. When r = 0 and p = 0, from () we have s = E, mplyng that σ = 0 due to (c). From (2) and (4), we get p r p = 0, whch contradcts E < p r. Therefore, ths case cannot be optmal. Fnally, when r > 0, ths mples p > 0 due to () and Lemma. From Lemma 2 we have p = p r. Hence, for the E < p r case, the only possble transmt power s p = p r. c) Consder the case p s E p r. When s > 0, then from () and Lemma, we get p < E.Duetop s E,ths contradcts Lemma 2. Therefore, ths case cannot be optmal. On the other hand, when r > 0, then () and Lemma yeld p > E.DuetoE p r, ths contradcts Lemma 2, and therefore ths case cannot be optmal. Hence, s = r = 0 n ths case, yeldng p = E from (). In summary, Theorem shows that the optmal polcy s a double-threshold polcy that can be expressed as p = mn ( max(e, p r ),[p s ] +), (7a) s =[E p ] +, r =[p E ] +. (7b) To fnd the entre polcy, t remans to fnd the thresholds p s and p r for =,...,, whch we descrbe next.
TUTUCUOGLU et al.: OPTIMUM POLICIES FOR A EERGY HARVESTIG TRASMITTER UDER EERGY STORAGE LOSSES 47 B. Fndng the Thresholds To determne the thresholds defned n (3), we make the observatons stated n the followng two lemmas: Lemma 3 states that these thresholds are non-decreasng n general, and reman constant n stretches of tme slots when there s energy n the battery. Therefore, they only potentally ncrease when the battery s depleted. Lemma 4 states that the battery must be depleted by the end of the communcaton sesson, otherwse the throughput can be ncreased by retrevng and usng the remanng energy n the battery n the last slot. Lemma 3: The thresholds p s and p r n (3) are nondecreasng, and reman constant unless the battery s depleted,.e., p s(+) = p s and p r(+) = p r for all when B > 0. Proof: The non-decreasng property follows from λ 0 n (3a), (3b). The second property n the lemma s a consequence of the complementary slackness condton n (a), whch mples that when B > 0wehaveλ = 0 and p s and p r reman constant from (3a), (3b). Lemma 4: In the optmal polcy, the battery s depleted at the end of the sesson,.e., B = 0. Proof: The proof s by contradcton. Let {(s,r )} = be a feasble polcy wth B > 0. Let s = s for =,...,, r = r for =,...,, and r = r + B. ote that {( s, r )} = s a feasble polcy. For ths new polcy, we have g( p )=g(p ) for =,..., and g( p ) > g(p ), yeldng a larger throughput. Hence, {(s,r )} = cannot be optmal. In lght of Lemmas 3 and 4, we seek a set of thresholds that are non-decreasng for all, only ncreasng when B = 0, and depletng the battery at the end of the transmsson. ote that t suffces to fnd p s values only, and p r can be calculated from the fxed relatonshp n (5). For ths purpose, we propose the algorthm below. Algorthm : Start from tme slot j =. Usng lnear search, fnd the largest threshold p s 0, and the correspondng p r from (5), for whch the transmt power polcy gven by (7) s feasble n = j,...,. Fnd the smallest l> j such that B l =0, and assgn optmal thresholds p s and p r to tme slots = j,...,l. If l<, repeat the above procedure startng from j =l+. The procedure n Algorthm ensures that the resultng thresholds are non-decreasng and reman constant whle the battery s not empty, as requred by Lemma 3. The nondecreasng property can be seen as follows: At a step startng from tme slot j, the prevous threshold p s( j ) s feasble n = j,..., by constructon. Hence, the new threshold p sj p s( j ). ext, we prove the optmalty of these thresholds. Theorem 2: The polcy n (7) wth thresholds {(p s, p r )} = found usng Algorthm s the soluton to (5). Proof: We show that usng p s and p r, =,...,,asetof Lagrange multplers satsfyng all KKT condtons n (9) () can be found. ote that p s and p r are non-decreasng. Let λ = η ( p s + /h) η ( p (8) s(+) + /h) for =,...,, wth p s(+) = by defnton. Ths satsfes λ 0 snce p s s non-decreasng, and satsfes (a) snce p s only changes when B = 0. Fg. 2. Example optmal polcy wth transmsson power thresholds p s and p r. ext, let µ = 0, =,...,, whch satsfy (b). Snce p s 0 and p r 0 by constructon, from (7a) we get p s p p r for all. Calculate σ and ν from (2) as h h h σ = + hp + hp, ν = s + hp h r + hp. (9) ote that these values are non-negatve snce p s p p r. Furthermore, they satsfy (c) as follows: when s > 0, (7b) mples E > p. Thus, from (7a), we have p =[p s ]+ = p s, and therefore (9) yelds σ = 0. Smlarly, when r > 0, (7b) mples E < p, and therefore (7a) mples p = p r. Hence, (9) yelds ν = 0. An example run of Algorthm and the resultng optmal transmsson polcy s shown n Fg. 2. The example s over = 5 tme slots wth storage effcency η = 0.5, energy harvests {E } = {9,4,2,3,4}, ntal charge E 0 = 0, and h =. Startng from j =, the largest feasble thresholds satsfyng (5) are found as p s = 7 and p r = 3, depletng the battery at the end of tme slot = 3. Settng these thresholds for =,2,3, the second set of thresholds startng from j = 4 are found as p s = and p r = 5, depletng the battery at the end of tme slot = 5 =. Wth these thresholds, the optmal transmt powers p are shown n red. Energy stored at the frst tme slot, marked as I, s retreved at the thrd tme slot, marked as II. Smlarly, energy stored n the fourth tme slot marked as III s retreved and consumed entrely n the ffth tme slot marked as IV. ote that snce p s E p r at = 2, no chargng or dschargng takes place. In ths slot, transmtter s n the passve state n Theorem, and uses only the ncomng energy for transmsson,.e., p = E. The optmal polcy derved n ths secton can be shown to converge to the results of [] when energy storage s deal. Ths s when η =, and (5) yelds p s = p r for all. Due to (7a), ths mples p =[p s ] + at all tmes. Consequently, the optmal polcy conssts of pecewse constant power transmssons, wth the transmt power ncreasng only at nstances of empty battery due to Lemma 3. Ths concdes wth the result wth deal battery n [] whch fnds the longest constant power stretches, and changes the power only when the battery s depleted. As battery effcency η decreases, the two thresholds p s and p r separate, yeldng a larger regon for the passve state. Our optmal polcy descrbes the transton from the constant power polcy n one extreme, η =, to the spend what you get polcy wthout storage n the other, η = 0.
472 IEEE JOURAL O SELECTED AREAS I COMMUICATIOS, VOL. 33, O. 3, MARCH 205 IV. OPTIMAL TRASMISSIO POLICY FOR A FIITE-SIZED BATTERY, E max < In practce, energy storage devces are of fnte sze. In ths secton, we extend the nfnte-szed battery problem n (5) to the case of a fnte-szed battery by ncludng the addtonal noenergy-overflow constrant n (3). For a battery of sze E max,the throughput maxmzaton problem becomes max {s,r } = s.t. 0 E 0 + g(e s + r ), j= (20a) (ηs j r j ) E max, =,...,, (20b) E s + r 0, =,...,, (20c) s 0, r 0, =,...,, (20d) where g(p) s defned n (4). The Lagrangan of (20) s ( ) L = (g(e s + r )+λ E 0 + (ηs j r j ) = β (E 0 + j= j= (ηs j r j ) E max ) + µ (E s + r )+σ s + ν r ), (2) where β, =,..., are the non-negatve Lagrange multplers for the no-energy-overflow constrants. The KKT optmalty condtons are h +η +hp h + hp j= j= (λ j β j ) µ +σ =0, =,...,, (22) (λ j β j )+µ + ν =0, =,...,. (23) The complementary slackness condtons correspondng to β are β (E 0 + j= (ηs j r j ) E max )= 0, =,...,, (24) whch, together wth those lsted n (), consttute the complementary slackness condtons for the problem n (20). From (22) and (23), we fnd the optmal transmt powers p as p = η j= (λ j β j ) µ + σ h = j= (λ, =,...,. (25) j β j ) µ ν h In vew of the new multplers β, we update the defnton of thresholds, p s and p r,as p s = η j= (λ j β j ), =,...,, (26a) h p r = j= (λ j β j ), =,...,, (26b) h whch satsfy (4) and (5). Observng that µ = 0 when p > 0, (6) must also hold for the optmal polcy. A. Fndng the Thresholds for a Fnte-Szed Battery In ths fnte-szed battery case, Lemmas, 2 and 4 contnue to hold,.e., n ths fnte-szed battery case also s r = 0(smultaneous storng and retreval s sub-optmal), f s > 0 then p = p s (when storng, the power must be equal to the storng threshold), f r > 0 then p = p r (when retrevng, the power must be equal to the retrevng threshold), and B = 0 (the battery must be depleted at the end of the communcaton sesson). However, due to β, the new thresholds n (26) no longer satsfy Lemma 3,.e., the new thresholds are no longer monotone. Instead, they satsfy the property stated n the followng lemma. Lemma 5: The thresholds p s and p r n (26) are nondecreasng whle B < E max, and non-ncreasng whle B > 0. Consequently, they reman constant f the battery s not depleted or full,.e., p s(+) = p s and p r(+) = p r for all whle 0 < B < E max. Proof: For B < E max, (24) gves β = 0. Substtutng n (26), ths mples that p s and p r are non-decreasng. Smlarly, for B > 0, (a) gves λ = 0. Substtutng n (26), ths mples that p s and p r are non-ncreasng. Fnally, for 0 < B < E max, (a) and (24) gve λ = 0 and β = 0, whch yelds p s(+) = p s and p r(+) = p r from (26), whch mples that p s and p r reman constant. Hence, we are lookng for a feasble set of thresholds satsfyng Lemmas, 2, 4, and 5. We propose the followng algorthm to fnd the optmal polcy. Algorthm 2: Start from tme slot j =. Fnd the largest threshold p s 0, and the correspondng p r from (5), such that the transmt power polcy gven by (7) does not volate (2) frst,.e., ether the polcy s feasble for = j,...,, or(3)s volated before (2). Fnd the smallest l> j such that B l = 0or B l = E max, and assgn optmal thresholds p s and p r to the tme slots = j,...,l. Ifl<, repeat the above procedure startng from j = l +. The next lemma shows that the thresholds found by ths algorthm satsfy Lemma 5. Lemma 6: The thresholds found by Algorthm 2 satsfy the condtons n Lemma 5. Proof: Startng from some j, let the algorthm output p sj, p rj, and l. Consder the case B l =0. Then, the constant threshold p sj must yeld a full battery at some k>l, or be feasble untl =, snce otherwse a smaller p sj would have been chosen by the algorthm. Hence, startng from tme slot l, the next threshold cannot be less than p sj. ow, consder the case B l = E max. Then, the constant threshold p sj volates (2) or depletes the battery at some k >l by constructon. Hence, startng from tme slot l, the next threshold cannot be greater than p sj. As a result of Lemma 6, we have that the thresholds found by Algorthm 2 are non-decreasng f B = 0, non-ncreasng f B = E max, and by constructon constant n between. ext, we prove the optmalty of the resultng polcy.
TUTUCUOGLU et al.: OPTIMUM POLICIES FOR A EERGY HARVESTIG TRASMITTER UDER EERGY STORAGE LOSSES 473 Theorem 3: The polcy n (7) wth thresholds {(p s, p r )} = found usng Algorthm 2 s the soluton to (20). Proof: We show that usng p s and p r, =,...,,asetof Lagrange multplers satsfyng all KKT condtons n () and (22) (24) can be found. Frst, note that p s and p r are constant unless the battery s depleted or full, non-decreasng f B = 0 and non-ncreasng f B = E max, as shown n Lemma 6. Let [ ] + λ = η ( p s + /h) η ( p s(+) + /h), (27a) [ ] + β = η ( p s(+) + /h) η ( p s + /h), (27b) for =,...,, wth p s(+) = by defnton. These satsfy (a) and (24) due to Lemma 6. The rest of the Lagrangan multplers are found as n the proof of Theorem 2, by replacng (2) wth (25). The polcy for the fnte battery case n ths secton converges to the prevous results for the deal battery case studed n [2], when η =. In ths case, the thresholds are equal and thus the optmal polcy s a constant power polcy as n [2]. For equal thresholds, the condtons n Lemma 5 concde wth those n [2, Theorem ]. V. O PTIMAL TRASMISSIO POLICY FOR A FADIG CHAEL We now consder a fadng channel, where the fadng channel coeffcent h, =,...,, s constant throughout tme slot, but changes from one tme slot to another. The coeffcents are known non-causally at the transmtter. Ths s an extenson of [3] to the neffcent energy storage case; see also [4]. The nstantaneous rate n slot s gven n (4), whch we wll denote as g(p,h) n ths secton, to emphasze ts dependence on the channel gan h. The throughput maxmzaton problem n a fadng channel for a transmtter wth a fnte-szed battery becomes max {s,r } = s.t. 0 E 0 + g(e s + r,h ), j= (28a) (ηs j r j ) E max, =,...,, (28b) E s + r 0, =,...,, (28c) s 0, r 0, =,...,, (28d) yeldng the KKT optmalty condtons h + η + h p h + h p j= j= (λ j β j ) µ + σ =0, =,...,, (29) (λ j β j )+µ + ν =0, =,...,, (30) and the complementary slackness condtons n () and (24). We note that Lemma holds for the fadng case as well, snce t only depends on the rate functon g(p,h) beng nondecreasng n p. For the fadng case, we defne the followng water-level thresholds, v s = whch satsfy η j= (λ j β j ), v r = j= (λ j β j ), (3) v r = ηv s, =,...,. (32) Wth these defntons, we observe that for a postve transmt power, p > 0, (b) gves µ = 0. Therefore, f the battery s beng charged,.e., s > 0, from (c) and (29) we have p = v s /h. Smlarly, f the battery s beng dscharged,.e., r > 0, (c) and (30) yelds p = v r /h.ifp = 0, then s = E s beng stored, and hence σ = 0. Snce µ 0 by defnton, (29) gves v s /h < 0. Hence, for the storng and retrevng cases, the optmal transmt powers are p = [ v s h ] + (storng), p = [ v r h ] + (retrevng). (33) ote that the thresholds v s and v r no longer equal transmt powers drectly n these cases, as n Sectons III and IV, but set the water-levels over whch water-fllng s to be performed. In partcular, f we can fnd water-levels v s and v r satsfyng (32) such that the power polcy ( ) ] ) + p = mn max (E,v r h, [v s h (34) s feasble, and all KKT condtons are satsfed, then ths polcy s optmal. Fndng these water-level thresholds s possble exactly as n Algorthm 2 after replacng the power polcy n (7) wth (34) and the thresholds p s and p r wth v s and v r, respectvely. An example of drectonal water-fllng wth thresholds s gven n Fg. 3 for a storage effcency of η = 0.5 and = 5. Fadng levels and the harvested energy for =,...,5are shown n Fg. 3(a) n gray and blue, respectvely. In partcular, the heght of the grey area represents h, and the heght of the blue area represents p n each tme slot. The battery capacty s suffcently large to store all harvested energy n ths example. Two pars of thresholds satsfyng (32) are found such that the battery s empty at the end of tme slots 3 and 5. Consequently, the thresholds only change at the end of the thrd tme slot. Energy n the areas marked as I and III are stored, and later retreved and consumed n the areas marked as II and IV, respectvely. VI. OPTIMAL TRASMISSIO POLICY FOR A BROADCAST CHAEL We next consder a Gaussan broadcast channel, whch conssts of an energy harvestng transmtter wth an neffcent battery, and two recevers, as shown n Fg. 4. At tme slot,the transmtter allocates the power p for transmsson, achevng a
474 IEEE JOURAL O SELECTED AREAS I COMMUICATIOS, VOL. 33, O. 3, MARCH 205 Fg. 4. Energy harvestng transmtter wth neffcent storage n a Gaussan broadcast channel. Furthermore, the polcy {( s, r )} s feasble snce (2) and (3) are lnear n s and r. Hence, {(ḡ j )} s achevable, and therefore G EH s convex. As a result of Lemma 8, the boundary of G EH can be traced by solvng a weghted sum throughput maxmzaton problem. In partcular, we solve for weghts w 0, Fg. 3. Drectonal water-fllng wth energy storage and retreval thresholds: (a) The ntal water levels wth p = E, and resultng thresholds, and (b) resultng water levels and optmal transmt powers. rate par (g,g 2 ) G(p ) where G(p) s the capacty regon for transmt power p, gven by G(p)= {(g,g 2 ) g 2 ( log + αp ) σ 2, g 2 ( ) } 2 log ( α)p + αp + σ 2, 0 α, (35) 2 wth nose varances σ 2 and σ2 2 σ2 for recevers and 2, respectvely [43]. The followng lemma presents a property that s common to all capacty regons, n partcular for the regon n (35), whch s an mmedate result of tme-sharng. Lemma 7: Let (g,g 2 ) G(p) and (ḡ,ḡ 2 ) G( p). Then, (λg +( λ)ḡ,λg 2 +( λ)ḡ 2 ) G (λp +( λ) p). (36) For ths channel, we characterze the maxmum throughput regon G EH as the set of achevable throughput pars under the energy harvestng constrants n (2) and (3). Ths s the extenson of the maxmum departure regon n [7, Defn. ] to the case of neffcent storage. Specfcally, we wrte {( ) G EH = g, g 2 (g,g 2 ) G(E s + r ), = = } s,r 0, E s + r 0, (2),(3). (37) We frst present the followng result, whch s an extenson of [5, Lemma 2] to the case of neffcent energy storage. Lemma 8: The throughput regon G EH s convex. Proof: Let {(s,r )} and {(s,r )}, be two feasble polces yeldng transmt powers {p } and {p }, and achevng rate pars {(g,g 2 )} and {(g,g 2 )}, respectvely. Let s = λs +( λ)s and r = λr +( λ)r, =,...,, whch yelds p = λp +( λ)p, =,...,. Then, due to Lemma 7, {( s, r )} can acheve the rates ḡ j λg j +( λ)g j, j =, 2. max {g,g 2 } s.t. w = ( = g + g, = = g 2, (38a) g 2 ) G EH. (38b) By substtutng (37) n (38), and separatng the maxmzaton over {s,r } and {g,g 2 }, the weghted sum throughput maxmzaton problem becomes max {s,r } f w (E s + r ), = (39a) s.t. E s + r 0, =,...,, (39b) s 0, r 0, (2), (3), =,...,, (39c) where f w (p) s the maxmum weghted sum rate functon defned as f w (p)= max wg + g 2, s.t. (g,g 2 ) G(p). (40) {g,g 2 } We next show the concavty of f w (p) n the followng lemma. 4 Lemma 9: The maxmum weghted sum rate functon f w (p) n (40) s concave n p. Proof: Let f w (p)=wg +g 2 and f w (p )=wg +g 2, wth (g,g 2 ) G(p) and (g,g 2 ) G(p ). By Lemma 7, we have (λg +( λ)g,λg 2 +( λ)g 2 ) G(λp +( λ)p ).From the defnton n (40), we can wrte f w ( λp +( λ)p ) w ( λg +( λ)g ) + λg2 +( λ)g 2 (4) =λ f w (p)+( λ) f w (p ), (42) whch mples the concavty of f w (p) n p. Wth a concave objectve (39a) and lnear constrants (39b), (39c), (39) s a convex program. Ths problem dffers from that n (20) only n the objectve. Fndng the respectve KKT 4 We remark that the concavty property n Lemma 9 s also shown n [7, Lemma 2] specfcally for a Gaussan broadcast channel wth M 2 recevers. In fact, the weghted sum rate f w (p) for any settng s concave due to the possblty of tme-sharng between dfferent transmt powers. Hence, these results can be generalzed to a larger class of channels.
TUTUCUOGLU et al.: OPTIMUM POLICIES FOR A EERGY HARVESTIG TRASMITTER UDER EERGY STORAGE LOSSES 475 condtons, the relaton between the thresholds p s and p r,gven for the sngle lnk n (5), becomes f w(p s ) f w(p = η, =,..., (43) r ) where p s and p r are defned as the solutons to f w(p s )=η f w(p r )= j= j= (λ j β j ), =,...,, (44a) (λ j β j ), =,...,, (44b) and f w(p) denotes the dervatve of f w (p) wth respect to transmt power p. By constructon, the propertes n Lemmas 2, 4, and 5 extend to ths case. The optmal power allocaton s therefore found as n Algorthm 2 by substtutng (5) wth (43). The resultng thresholds satsfy (6) and Lemma 6 by constructon, and therefore yeld vald Lagrange multplers through (9) and (27). We remark that the optmal polcy conforms to the doublethreshold structure defned n Theorem, regardless of what the weght w s. However, unlke the effcent storage case n [7, Lemma 3], the power polces are no longer dentcal for all weghts w. In partcular, the relatonshp n (43) depends on the weght w, and hence the thresholds depletng or fllng the battery n Algorthm 2 are affected by the weght. Ths nsght apples to other channel models, and the sngle lnk, as well: n [], [2], the optmal power polcy s found to be the same for all concave power-rate functons g(p). However, n the neffcent storage case, the dervatve g (p) affects how the thresholds are related, and thus the optmal polcy. As a concluson, unlke prevous work wth deal batteres, the rate functon plays a drect role n determnng the optmal polcy n the neffcent storage case. VII. TRASMISSIO POLICIES WITH CAUSAL EERGY HARVESTIG IFORMATIO The prevous sectons derve optmal polces when the harvestng process over the duraton of the sesson,.e., E, =,...,, s known before the sesson starts. Ths approach provdes a benchmark soluton as well as nsghts for effcent power allocaton, and s applcable n scenaros where the harvested energy s controlled or predctable [44]. For other applcatons where such nformaton may not be avalable noncausally, n ths secton, we develop polces that only requre causal knowledge of the harvested energy. A. Optmal Onlne Polcy We refer to those transmsson polces where the transmtter chooses ts power value based on the energy harvested up to that pont n tme,.e., wth causal nformaton, as onlne polces. The optmal such polcy can be found by solvng a dynamc program [45], whch we formulate next. Let the harvested energy values E and fadng coeffcents h be..d. or frst order Markov processes. Such harvestng processes are consdered prevously n [4], [46], [47], and recent work wth emprcal solar and wnd harvestng data confrms that a Markov process s a good model for harvested energy [48]. Fnte state Markov channels are also known to be good models for Raylegh fadng channels [49] [5]. For an energy harvestng transmtter, the states of the system at the begnnng of tme slot nclude the energy stored n the battery, B, hstory of energy harvests, E = E,...,E, fadng coeffcents, h = h,...,h, and the slot ndex,. The node decdes on ts transmt power based on these varables, and hence ts decson can be expressed as the acton p = φ(b,e,h,). We remark that smultaneous storage and retreval of energy s sub-optmal also n the onlne case,.e., Lemma extends to onlne polces. Thus, gven the power polcy {p }, the stored and retreved energy values are found from (7b). Takng the acton p = φ(b,e,h,), the system acheves a throughput g(p,h ) n tme slot, and leaves energy B = B + ηs r n the battery for future tme slots. The value functon, whch s the acheved throughput n tme slot and the expected future throughput of the system after tme slot, s gven by the Bellman equaton, max g ( [ φ(b,e,h ),),h +E g ( φ(b j,e j,h j ) ], j),h j φ = max φ g ( φ(b,e,h,),h ) j=+ (45a) + E [ V ( B,E +,h +, + )], (45b) where the expectatons are taken over the dstrbuton of the harvestng process E and fadng process h.theoptmal onlne power polcy φ (.) s the maxmzer of the Bellman equaton n (45) [45]. ote that the dmenson of the optmzaton varable n (45) ncreases wth. Thus, solvng ths problem through value teraton has exponental complexty, and s ntractable for large. However, t s possble to smplfy the problem by observng the Markovan property of the harvestng and channel fadng processes. In partcular, only B and E mpose a constrant on the transmt power, and only h affects the rate, n tme slot. Snce E and h are..d or Markovan, future realzatons of these varables are ndependent of the past values gven ther current values. Hence, havng dfferent actons for dfferent values of {E } and {h } does not affect the value functon, and therefore we can smplfy the actons as φ(b,e,h,) =φ(b,e,h,). Ths yelds the smplfed Bellman equaton for..d. arrvals and fadng, V (B,E,h,)=maxg(φ(B,E,h,),h ) φ + E[V (B,E +,h +, + )], (46) whch can be easly computed usng value teraton. amely, startng from = and choosng V (B,E,h, + ) = 0, optmal actons φ(b,e,h,) and value functons V (B,E,h,) are calculated. These values are then used to calculate the optmal actons and value functons at = from (46), and the process s repeated for all. Fnally, we consder an nfnte-horzon problem,.e.,, and fnd the optmal onlne polcy usng a dscounted problem.
476 IEEE JOURAL O SELECTED AREAS I COMMUICATIOS, VOL. 33, O. 3, MARCH 205 Fg. 5. Optmal onlne transmsson power for..d. energy arrvals. Ths s also the polcy that the value teraton algorthm on (46) converges to for very large. To fnd ths polcy, denoted by φ (B,E,h ), we ntroduce a dscount factor β and wrte the Bellman equaton as V (B,E,h )=maxg(φ(b,e,h ),h ) φ + βe[v (B,E +,h + )]. (47) Fg. 6. model. Optmal onlne transmsson power for the bursty energy harvestng Startng wth an arbtrary set of ntal actons, teratng (47) converges to the optmal polcy φ (B,E,h ). As the dscount factor β, the resultng polcy approaches the optmal nfnte horzon polcy. As an example, Fg. 5 shows the optmal nfnte horzon polcy for a non-fadng Gaussan channel wth h =, a fnteszed battery E max = 00, and..d. unform energy harvests n [0,20]. ote that for a fxed stored energy, the optmal onlne polcy exhbts a double-threshold structure smlar to that n Theorem, e.g., the bold lne for B = 60 n the fgure. The optmal transmt power s equal to the harvested energy for a range of B and E values, marked as regon I. Regons II and III are separated from regon I by a set of thresholds, ndcated wth dashed lnes. Wthn these regons, the transmt powers vary only slghtly wth harvested energy rate E. The two thresholds separatng regon I from regons II and III are observed to satsfy the relatonshp n (5) for each B. The thresholds, however, change wth B : For small B, the thresholds are lower, wth p r = 0atB = 0 snce retrevng energy s not feasble. For large B, the thresholds are hgher, reachng to p s > 20 mw at B = E max, snce storng energy s not feasble. For harvestng processes wth memory, we consder two scenaros wth Markovan energy harvests n Fg. 6 and Fg. 7. In Fg. 6, harvestng s a bursty process where the next energy harvest remans the same,.e., E + = E, wth probablty 0.5, and a new value that s unform n [0, 20] s generated wth probablty 0.5. Hence, the process conssts of bursts of constant rate harvests. As seen n Fg. 6, ths harvestng model also yelds a double-threshold polcy that resembles the..d. case n Fg. 5. In Fg. 7, harvested energy E performs a random walk on [0, 20], where t ncreases or decreases unt wth probablty 0.4 each, and remans the same wth probablty 0.2. In ths case, the Fg. 7. Optmal onlne transmsson power for the random walk energy harvestng model. optmal polcy s to consume all harvested energy, whch does not show a threshold characterstc. Ths s ntutve, because a hgh or low harvest rate s sustaned for extended perods n ths model, and consstently storng or retrevng would lkely overflow or deplete the battery. B. Proposed Onlne Polcy Sectons III VI show that the optmal polcy has a doublethreshold structure where the thresholds are related for all =,...,. The nfnte-horzon optmal onlne polcy found n Secton VII-A for..d. arrvals also exhbts a smlar doublethreshold structure. Wth these n mnd, n ths secton, we propose a smpler onlne double-threshold polcy by assgnng constant thresholds throughout the communcaton sesson. We frst consder a non-fadng channel and Markovan harvested energy values E wth statonary probablty dstrbuton f E (E). We propose fndng fxed thresholds p s = p s and p r = p r, =,...,, that satsfy (5) and η p s (e p s ) f E (e)de pr 0 (p r e) f E (e)de = 0. (48)
TUTUCUOGLU et al.: OPTIMUM POLICIES FOR A EERGY HARVESTIG TRASMITTER UDER EERGY STORAGE LOSSES 477 Fg. 8. Throughput for a statc channel wth..d. energy arrvals and an average harvestng rate of 0 mw. Fg. 0. Throughput for a statc channel wth Markov (random walk) energy rrvals and an average harvestng rate of 0 mw. Fg. 9. Throughput for a statc channel wth Markov (bursty) energy arrvals and an average harvestng rate of 0 mw. Fg.. Throughput for a Raylegh fadng channel wth..d. energy arrvals and an average harvestng rate of 0 mw. Ths equaton can be nterpreted as an energy stablty condton, snce t ensures that the expected energy stored n and retreved from the battery are equal. Thus, nether the energy storage s underutlzed, nor an excessve amount of energy s stored wthout utlty. ote that at η =, ths reduces to a constant power polcy that preserves energy-neutralty, and resembles the best-effort transmsson scheme of [52] whch s optmal for nfnte length transmsson. On the other hand, at η = 0, (48) s only satsfed wth p r = 0 and p s. Ths means that no energy s stored,.e., p = E, whch s optmal snce η = 0. Above polcy satsfyng (48) can be readly extended to a fadng channel wth Markovan channel coeffcents h and jont statonary dstrbuton f E,H (e,h) by fndng water level thresholds v s = v s and v r = v r, =,...,, that satsfy (32) and [ [ ] η e v s + + [ v r 0 h] ] + h e f E,H (e,h)dedh= 0, (49) where (49) s the fadng equvalent of the energy stablty condton n (48). VIII. UMERICAL RESULTS In ths secton, we provde numercal results on the performances of the optmal offlne polcy and the onlne polces. We smulate communcaton sessons consstng of = 0 4 tme slots, wth a slot length of τ = 0 ms. Snce the model n Secton II assumed unt slot length, the optmal polces n ths case are found by scalng transmt powers and consumed energy values accordngly. We consder an energy harvestng transmtter node equpped wth a battery of sze mj and ntal charge E0 bat = 0. We have the Gaussan nose spectral densty of 0 = 0 9 W/Hz at the recever, and a bandwdth of MHz. The path loss between the transmtter and recever s h = 00 db. For the purpose of comparson, we ntroduce two algorthms. The frst s the drectonal water-fllng (DWF) algorthm of [3], whch s ndfferent to the storage effcency η, and a feasble polcy s obtaned as n [3]. The second s the effcencyadaptve drectonal water-fllng algorthm, whch s obtaned by forcng the two thresholds of the optmal offlne algorthm n Secton III to be equal, thus resemblng DWF n [3]. However, t accounts for the storage effcency η when choosng ts constant water levels, and therefore s a near-optmal heurstc.
478 IEEE JOURAL O SELECTED AREAS I COMMUICATIOS, VOL. 33, O. 3, MARCH 205 Fg. 2. Throughput for a statc channel wth..d. energy arrvals and an average harvestng rate of 80 µw. Fg. 4. Throughput for a statc channel wth Markov (random walk) energy arrvals and an average harvestng rate of 80 µw. Fg. 3. Throughput for a statc channel wth Markov (bursty) energy arrvals and an average harvestng rate of 80 µw. We consder the sngle-user settng wth a fnte-szed battery n Secton IV. Fg. 8 shows the throughput for the offlne and onlne polces versus storage effcency η when the harvested energy E at each tme slot of length 0 ms s generated n an..d. fashon, dstrbuted unformly n [0, 200] µj. Ths corresponds to an average energy harvestng rate of 0 mw. Smulatons are repeated for bursty and random walk arrval models of Secton VII-A n Fg. 9 and Fg. 0, respectvely, for a harvested energy range of [0, 200] µj. We observe that the performance of DWF degrades rapdly wth decreasng η, snce t does not adapt to storage effcency. Effcency-adaptve DWF performs reasonably well for hgh storage effcency, but worse at low storage effcency snce t also reles on frequently storng and retrevng energy. Moreover, n all cases, the proposed onlne polcy performs very close to the optmal onlne polcy, both provdng a notable mprovement over DWF and effcencyadaptve DWF. In Fg., we compare the throughput of offlne and onlne polces for a fadng channel. We consder Raylegh fadng wth E[h ]= 00 db, and the remanng parameters are unchanged from those n Fg. 8. Here, the optmal offlne and onlne polces compare smlar to the non-fadng case. We observe that Fg. 5. Throughput for a Raylegh fadng channel wth..d. energy arrvals and an average harvestng rate of 80 µw. effcency-adaptve DWF performs close to optmal for hgh storage effcency, whle DWF rapdly departs from the optmal as η decreases. The proposed onlne polcy s notably close to the optmal for all storage effcency values. We next consder an average harvestng rate of 80 µw, whch s more realstc for small-szed energy harvestng sensor nodes wth lmted access to ambent energy. We generate energy arrvals accordngly, whle the remanng parameters are unchanged. Fgs. 2 5 present the throughput for the offlne and onlne polces versus storage effcency η for..d., bursty, and random walk energy harvests n a statc channel, and..d. energy harvests n a Raylegh fadng channel, respectvely. The sgnfcance of the double-threshold polcy s more pronounced n ths low-power scenaro, as the performance of both DWF and effcency-adaptve DWF quckly depart from that of the optmal as η decreases. We plot the throughput of the above polces relatve to the optmal offlne polcy,.e., scaled by the optmal throughput, as a functon of average energy harvestng rate n Fg. 6, for..d. energy harvests n a statc channel wth the same parameters. We observe that whle the performance of the optmal onlne and proposed onlne algorthms are vrtually
TUTUCUOGLU et al.: OPTIMUM POLICIES FOR A EERGY HARVESTIG TRASMITTER UDER EERGY STORAGE LOSSES 479 Fg. 6. Throughput of effcency-adaptve DWF, DWF, and onlne polces relatve to optmal throughput. dentcal to that of the optmal offlne polcy, the performance of effcency-adaptve DWF and DWF falls to as low as 90% and 67% of the optmal throughput, respectvely, at low harvestng rates. Fnally, to examne the dynamcs of the polces further, we present a smaller numercal example wth = 5 tme slots of duraton 0 ms, storage effcency η = 0.66, battery capacty E max = 20 µj, and energy harvests [8 20 2 9 4] µj. In ths scenaro, the optmal power polcy sets thresholds p s =.43 mw and p r = 0.6 mw, and yelds the transmt powers p = [.43.43 0.6 0.90 0.6] mw wth an average throughput of 0.486 bts/sec/hz. In comparson, effcency-adaptve DWF yelds p =[0.93 0.93 0.93 0.93 0.93] mw wth average throughput 0.4733 bts/sec/hz, and DWF yelds p = [0.70 0.70 0.70 0.70 0.70] mw wth average throughput 0.3825 bts/sec/hz. ote that the optmal offlne polcy consumes more energy n the frst two tme slots, and does not nsst on equalzng the powers p 2 and p 3. Ths s to ts beneft, because for these transmt powers, energy storage loss overcomes the advantage of constant power transmsson. Meanwhle, beng aware of the storage neffcency, the effcency-adaptve DWF algorthm chooses a constant transmt power of 0.93 mw, whle the DWF algorthm chooses a constant transmt power of 0.70 mw. As a result, the average throughput of the optmal offlne algorthm s sgnfcantly better than that of DWF, and s only approached by effcency-adaptve DWF. IX. COCLUSIO We dentfy the throughput optmal transmt power polcy for an energy harvestng transmtter wth a battery that has storage neffcency. We show that the optmal polcy has a doublethreshold structure, where the thresholds are determned by the energy harvestng process and the storage effcency. We show that the thresholds are constant f battery s not completely empty or full, and the thresholds only ncrease when the battery s empty, and only decrease when the battery s full. We develop an algorthm to fnd the optmum thresholds. We extend the soluton to a fadng channel, and develop a drectonal waterfllng algorthm wth a double-threshold polcy actng on water levels. In the broadcast channel, we obtan throughput regons by maxmzng weghted sum throughput for all weghts, where the optmal polcy n each case s a double-threshold polcy. Fnally, we use a dynamc programmng formulaton to develop an optmum onlne algorthm, and numercally observe that t follows a smlar double-threshold structure. We further propose a smpler onlne double-threshold polcy wth low complexty, and observe expermentally that t performs close to ts optmal counterpart. An nsght drawn from these results s that when battery neffcency s taken nto consderaton, the optmal power polcy s no longer pecewse constant as was the case n prevous work wth deal batteres. Instead, two thresholds emerge n both the offlne and onlne optmal polces, between whch harvested energy s consumed mmedately,.e., wthout energy storage or retreval. When battery s set to be lossless, these two thresholds are equal, and the polces converge to prevous results. In essence, double-threshold polces result from the neffcency of the battery, and ntroduce an nterval wthn whch the losses due to neffcency outwegh the benefts of storage. In addton, we observe that the conventonal drectonal water-fllng algorthm, whch does not adapt to storage neffcency, ncurs a sgnfcant performance loss as storage effcency decreases. Important future drectons are ncorporatng other practcal aspects of physcal systems such as processng power and communcaton overheads to our model, leadng to practcal mplementaton of these communcaton systems. Fnally, the results of ths paper where we have nvestgated chargng/dschargng mperfectons that occur at the tme of energy savng/retreval should further be combned wth other types of battery mperfectons such as battery degradaton and energy leakage that occur over tme. REFERECES [] A. J. Goldsmth and S. B. Wcker, Desgn challenges for energyconstraned ad hoc wreless networks, IEEE Wreless Commun., vol. 9, no. 4, pp. 8 27, Aug. 2002. [2] A. Ephremdes, Energy concerns n wreless networks, IEEE Wreless Commun., vol. 9, no. 4, pp. 48 59, Aug. 2002. [3] E. Uysal-Bykoglu, B. Prabhakar, and A. El Gamal, Energy-effcent packet transmsson over a wreless lnk, IEEE/ACM Trans. etw., vol. 0, no. 4, pp. 487 499, Aug. 2002. [4] E. Uysal-Bykoglu and A. El Gamal, On adaptve transmsson for energy effcency n wreless data networks, IEEE Trans. Inf. Theory, vol. 50, no. 2, pp. 308 3094, Dec. 2004. [5] M. Zorz and R. R. Rao, Energy-constraned error control for wreless channels, IEEE Pers. Commun., vol. 4, no. 6, pp. 27 33, Dec. 997. [6] S. Cu, A. J. Goldsmth, and A. Baha, Energy-constraned modulaton optmzaton, IEEE Trans. Wreless Commun., vol. 4, no. 5, pp. 2349 2360, Sep. 2005. [7] M. A. Zafer and E. Modano, A calculus approach to energy-effcent data transmsson wth qualty-of-servce constrants, IEEE/ACM Trans. etw., vol. 7, no. 3, pp. 898 9, Jun. 2009. [8] M. A. Zafer and E. Modano, Optmal rate control for delay-constraned data transmsson over a wreless channel, IEEE Trans. Inf. Theory, vol. 54, no. 9, pp. 4020 4039, Sep. 2008. [9] M. J. eely, Energy optmal control for tme-varyng wreless networks, IEEE Trans. Inf. Theory, vol. 52, no. 7, pp. 295 2934, Jul. 2006. [0] M. J. eely, Optmal energy and delay tradeoffs for multuser wreless downlnks, IEEE Trans. Inf. Theory, vol. 53, no. 9, pp. 3095 33, Sep. 2007. [] J. Yang and S. Ulukus, Optmal packet schedulng n an energy harvestng communcaton system, IEEE Trans. Commun., vol. 60, no., pp. 220 230, Jan. 202. [2] K. Tutuncuoglu and A. Yener, Optmum transmsson polces for battery lmted energy harvestng nodes, IEEE Trans. Wreless Commun.,vol., no. 3, pp. 80 89, Mar. 202.
480 IEEE JOURAL O SELECTED AREAS I COMMUICATIOS, VOL. 33, O. 3, MARCH 205 [3] O. Ozel, K. Tutuncuoglu, J. Yang, S. Ulukus, and A. Yener, Transmsson wth energy harvestng nodes n fadng wreless channels: Optmal polces, IEEE J. Sel. Areas Commun., vol. 29, no. 8, pp. 732 743, Sep. 20. [4] C. K. Ho and R. Zhang, Optmal energy allocaton for wreless communcatons wth energy harvestng constrants, IEEE Trans. Sgnal Process., vol. 60, no. 9, pp. 4808 488, Sep. 202. [5] J. Yang, O. Ozel, and S. Ulukus, Broadcastng wth an energy harvestng rechargeable transmtter, IEEE Trans. Wreless Commun., vol., no. 2, pp. 57 583, Feb. 202. [6] M. A. Antepl, E. Uysal-Bykoglu, and H. Erkal, Optmal packet schedulng on an energy harvestng broadcast lnk, IEEE J. Sel. Areas Commun., vol. 29, no. 8, pp. 72 73, Sep. 20. [7] O. Ozel, J. Yang, and S. Ulukus, Optmal broadcast schedulng for an energy harvestng rechargeable transmtter wth a fnte capacty battery, IEEE Trans. Wreless Commun., vol., no. 6, pp. 293 2203, Jun. 202. [8] J. Yang and S. Ulukus, Optmal packet schedulng n a multple access channel wth energy harvestng transmtters, J. Commun. etw., vol. 4, no. 2, pp. 40 50, Apr. 202. [9] K. Tutuncuoglu and A. Yener, Sum-rate optmal power polces for energy harvestng transmtters n an nterference channel, J. Commun. etw., vol. 4, no. 2, pp. 5 6, Apr. 202. [20] D. Gunduz and B. Devllers, Two-hop communcaton wth energy harvestng, n Proc. 4th IEEE CAMSAP, Dec. 20, pp. 20 204. [2] C. Huang, R. Zhang, and S. Cu, Throughput maxmzaton for the Gaussan relay channel wth energy harvestng constrants, IEEE J. Sel. Areas Commun., vol. 3, no. 8, pp. 469 479, Aug. 203. [22] O. Orhan and E. Erkp, Optmal transmsson polces for energy harvestng two-hop networks, n Proc. 46th Annu. CISS, Mar. 202, pp. 6. [23] O. Orhan and E. Erkp, Throughput maxmzaton for energy harvestng two-hop networks, n Proc. IEEE ISIT, Jul. 203, pp. 596 600. [24] I. Ahmed, A. Ikhlef, R. Schober, and R. K. Mallk, Power allocaton for conventonal and buffer-aded lnk adaptve relayng systems wth energy harvestng nodes, IEEE Trans. Wreless Commun., vol. 3, no. 3, pp. 82 95, Mar. 204. [25] I. Ahmed, A. Ikhlef, R. Schober, and R. K. Mallk, Jont power allocaton and relay selecton n energy harvestng AF relay systems, IEEE Wreless Commun. Lett., vol. 2, no. 2, pp. 239 242, Apr. 203. [26] Y. Luo, J. Zhang, and K. B. Letaef, Optmal schedulng and power allocaton for two-hop energy harvestng communcaton systems, IEEE Trans. Wreless Commun., vol. 2, no. 9, pp. 4729 474, Sep. 203. [27] O. Orhan, D. Gunduz, and E. Erkp, Throughput maxmzaton for an energy harvestng communcaton system wth processng cost, n Proc. IEEE ITW, Sep. 202, pp. 84 88. [28] J. Xu and R. Zhang, Throughput optmal polces for energy harvestng wreless transmtters wth non-deal crcut power, IEEE J. Sel. Areas Commun., vol. 32, no. 2, pp. 322 332, Feb. 204. [29] M. Gregor and M. Payaro, Throughput maxmzaton for a wreless energy harvestng node consderng the crcutry power consumpton, n Proc. IEEE VTC, Sep. 202, pp. 5. [30] Q. Ba and J. A. ossek, Throughput maxmzaton for energy harvestng nodes wth generalzed crcut power modelng, n Proc. IEEE 3th Int. Workshop SPAWC, Jun. 202, pp. 485 489. [3] B. Gurakan, O. Ozel, J. Yang, and S. Ulukus, Energy cooperaton n energy harvestng communcatons, IEEE Trans. Commun., vol. 6,no. 2, pp. 4884 4898, Dec. 203. [32] K. Tutuncuoglu and A. Yener, Multple access and two-way channels wth energy harvestng and bdrectonal energy cooperaton, n Proc. ITA, Feb. 203, pp. 8. [33] K. Tutuncuoglu and A. Yener, Cooperatve energy harvestng communcatons wth relayng and energy sharng, n Proc. IEEE ITW,Sep.203, pp. 5. [34] H. Mahdav-Doost and R. D. Yates, Energy harvestng recevers: Fnte battery capacty, n Proc. IEEE ISIT, Jul. 203, pp. 799 803. [35] K. Tutuncuoglu and A. Yener, Communcatng wth energy harvestng transmtters and recevers, n Proc. ITA, Feb. 202, pp. 240 245. [36] S. Sudevalayam and P. Kulkarn, Energy harvestng sensor nodes: Survey and mplcatons, IEEE Commun. Surveys Tuts., vol. 3, no. 3, pp. 443 46, 20. [37] J. Taneja, J. Jeong, and D. Culler, Desgn, modelng, capacty plannng for mcro-solar power sensor networks, n Proc. IPS, Apr. 2008, pp. 407 48. [38] B. Devllers and D. Gunduz, A general framework for the optmzaton of energy harvestng communcaton systems wth battery mperfectons, J. Commun. etw., vol. 4, no. 2, pp. 30 39, Apr. 202. [39] H. Ibrahm, A. Ilnca, and J. Perron, Energy storage systems Characterstcs and comparsons, Renew. Sustan. Energy Rev., vol. 2, no. 5, pp. 22 250, Jun. 2008. [40] I. Hadjpaschals, A. Poullkkas, and V. Efthmou, Overvew of current and future energy storage technologes for electrc power applcatons, Renew. Sustan. Energy Rev., vol. 3, no. 6, pp. 53 522, Aug./Sep. 2009. [4] S. Reddy and C. R. Murthy, Duty cyclng and power management wth a network of energy harvestng sensors, n Proc. 4th IEEE Int. Workshop CAMSAP, Dec. 20, pp. 205 208. [42] S. Reddy and C. R. Murthy, Dual-stage power management algorthms for energy harvestng sensors, IEEE Trans. Wreless Commun., vol., no. 4, pp. 434 445, Apr. 202. [43] T. M. Cover and J. A. Thomas, Elements of Informaton Theory. Hoboken, J, USA: Wley, 2006. [44] M. Gorlatova, A. Wallwater, and G. Zussman, etworkng low-power energy harvestng devces: Measurements and algorthms, n Proc. IEEE IFOCOM, Apr. 20, pp. 602 60. [45] D. P. Bertsekas, Dynamc Programmng and Optmal Control. Belmont, MA, USA: Athena Scentfc, 995. [46] B. Medepally,. B. Mehta, and C. R. Murthy, Implcatons of energy profle and storage on energy harvestng sensor lnk performance, n Proc. IEEE GLOBECOM, ov. 2009, pp. 6. [47] J. Le, R. D. Yates, and L. Greensten, A generc model for optmzng sngle-hop transmsson polcy of replenshable sensors, IEEE Trans. Wreless Commun., vol. 8, no. 2, pp. 547 55, Feb. 2009. [48] C. K. Ho, P. D. Khoa, and P. C. Mng, Markovan models for harvested energy n wreless communcatons, n Proc. IEEE ICCS, ov. 200, pp. 3 35. [49] H. S. Wang and. Moayer, Fnte-state Markov channel A useful model for rado communcaton channels, IEEE Trans. Veh. Technol., vol. 44, no., pp. 63 7, Feb. 995. [50] H. S. Wang and P. Chang, On verfyng the frst-order Markovan assumpton for a Raylegh fadng channel model, IEEE Trans. Veh. Technol., vol. 45, no. 2, pp. 353 357, May 996. [5] Q. Zhang and S. A. Kassam, Fnte-state Markov model for Raylegh fadng channels, IEEE Trans. Commun., vol. 47, no., pp. 688 692, ov. 999. [52] O. Ozel and S. Ulukus, Achevng AWG capacty under stochastc energy harvestng, IEEE Trans. Inf. Theory, vol. 58, no. 0, pp. 647 6483, Oct. 202. Kaya Tutuncuoglu (S 08) receved the two B.S. degrees n electrcal and electroncs engneerng and n physcs from the Mddle East Techncal Unversty, Ankara, Turkey, n 2008 and 2009, respectvely. He s currently pursung the Ph.D. degree and has been a Graduate Research Assstant wth the Department of Electrcal Engneerng, Pennsylvana State Unversty, Unversty Park, PA, USA, snce 2009. Hs research nterests nclude green communcatons, energy harvestng, and resource allocaton for wreless networks. He receved the AT&T Graduate Fellowshp at Penn State n 202, the IEEE Marcon Prze Paper Award n Wreless Communcatons n 204, and the Leonard A. Doggett Award for Outstandng Wrtng n Electrcal Engneerng at Penn State n 204.
TUTUCUOGLU et al.: OPTIMUM POLICIES FOR A EERGY HARVESTIG TRASMITTER UDER EERGY STORAGE LOSSES 48 Ayln Yener (S 9 M 00 SM 3 F 4) receved the B.Sc. degree n electrcal and electroncs engneerng, and the B.Sc. degree n physcs, from Bogazc Unversty, Istanbul, Turkey; and the M.S. and Ph.D. degrees n electrcal and computer engneerng from Wreless Informaton etwork Laboratory (WILAB), Rutgers Unversty, ew Brunswck, J. She s a professor of Electrcal Engneerng at The Pennsylvana State Unversty, Unversty Park, PA snce 200, where she joned the faculty as an assstant professor n 2002. Durng the academc year 2008 2009, she was a Vstng Assocate Professor wth the Department of Electrcal Engneerng, Stanford Unversty, CA. Her research nterests are n nformaton theory, communcaton theory and network scence wth recent emphass on green communcatons and nformaton securty. She receved the SF CAREER award n 2003, the best paper award n Communcaton Theory n the IEEE Internatonal Conference on Communcatons n 200, the Penn State Engneerng Alumn Socety (PSEAS) Outstandng Research Award n 200, the IEEE Marcon Prze paper award n 204, the PSEAS Premer Research Award n 204, and the Leonard A. Doggett Award for Outstandng Wrtng n Electrcal Engneerng at Penn State n 204. Dr. Yener s currently a member of the board of governors of the IEEE Informaton Theory Socety where she was prevously the treasurer (202 204). She served as the student commttee char for the IEEE Informaton Theory Socety 2007 20, and was the co-founder of the Annual School of Informaton Theory n orth Amerca co-organzng the school n 2008, 2009 and 200. She was a techncal (co)-char for varous symposa/tracks at IEEE ICC, PIMRC, VTC, WCC and Aslomar (2005 204), and served as an edtor for IEEE TRASACTIOS O COMMUICATIOS (2009 202), an edtor and an edtoral advsory board member for IEEE Transactons on Wreless Communcatons (200 202), a guest edtor for IEEE Transactons on Informaton Forenscs and Securty (20) and a guest edtor for IEEE Journal on Selected Areas n Communcatons (205). Sennur Ulukus (S 90 M 98) receved the B.S. and M.S. degrees n electrcal and electroncs engneerng from Blkent Unversty and the Ph.D. degree n electrcal and computer engneerng from the Wreless Informaton etwork Laboratory (WILAB), Rutgers Unversty. She was a Senor Techncal Staff Member at AT&T Labs-Research. She s a Professor of Electrcal and Computer Engneerng at the Unversty of Maryland at College Park, where she also holds a jont appontment wth the Insttute for Systems Research (ISR). Her research nterests are n wreless communcaton theory and networkng, network nformaton theory for wreless communcatons, sgnal processng for wreless communcatons, nformaton theoretc physcal layer securty, and energy harvestng communcatons. She receved the 2003 IEEE Marcon Prze Paper Award n Wreless Communcatons, a 2005 SF CAREER Award, the 200 20 ISR Outstandng Systems Engneerng Faculty Award, and the 202 George Corcoran Educaton Award. She served as an Assocate Edtor for the IEEE TRASACTIOS O IFORMATIO THEORY (2007 200) and IEEE TRASACTIOS O COMMUICATIOS (2003 2007). She served as a Guest Edtor for the IEEE JOURAL O SELECTED AREAS I COMMUICATIOS specal ssue on wreless communcatons powered by energy harvestng and wreless energy transfer (205), the Journal of Communcatons and etworks specal ssue on energy harvestng n wreless networks (202), IEEE TRASACTIOS O IFORMATIO THEORY specal ssue on nterference networks (20), and IEEE JOURAL O SELECTED AREAS I COMMUICATIOS specal ssue on multuser detecton for advanced communcaton systems and networks (2008). She served as the TPC co-char of the 204 IEEE PIMRC, the Communcaton Theory Symposum at 204 IEEE Globecom, the Communcaton Theory Symposum at 203 IEEE ICC, the Physcal-Layer Securty Workshop at 20 IEEE Globecom, the Physcal-Layer Securty Workshop at 20 IEEE ICC, the 20 Communcaton Theory Workshop (IEEE CTW), the Wreless Communcatons Symposum at 200 IEEE ICC, the Medum Access Control Track at 2008 IEEE WCC, and the Communcaton Theory Symposum at 2007 IEEE Globecom. She was the Secretary of the IEEE Communcaton Theory Techncal Commttee (CTTC) n 2007 2009.