Finite Horizon Energy-Efficient Scheduling with Energy Harvesting Transmitters over Fading Channels

Transcription

1 Finite Horizon Energy-Efficient Scheduing with Energy Harvesting Transmitters over Fading Channes arxiv: v [cs.it] 2 Feb 207 Baran Tan Bacinogu, Eif Uysa-Biyikogu, Can Emre Koksa METU, Ankara, Turkey, The Ohio State University E-mai: barantan@metu.edu.tr, ueif@metu.edu.tr, koksa.2@osu.edu, Abstract In this paper, energy-efficient transmission schemes achieving maxima throughput over a finite time interva are studied in a probem setting incuding energy harvests, data arrivas and channe variation. The goa is to express the offine optima poicy in a way that faciitates a good onine soution. We express any throughput maximizing energy efficient offine schedue (EE-TM-OFF) expicity in terms of water eves. This aows per-sot rea-time evauation of transmit power and rate decisions, using estimates of the associated offine water eves. To compute the onine power eve, we construct a stochastic dynamic program that incorporates the offine optima soution as a stochastic process. We introduce the Immediate Fi metric which provides a ower bound on the efficiency of any onine poicy with respect to the corresponding optima offine soution. The onine agorithms obtained this way exhibit performance cose to the offine optima, not ony in the ong run but aso in short probem horizons, deeming them suitabe for practica impementations. I. INTRODUCTION Energy efficient packet scheduing with data arriva and deadine constraints has been the topic of numerous studies (e.g., [] [4]). Energy harvesting constraints have been incorporated in the recent years within these offine and onine formuations (e.g., [5] [6].) A criticism that offine formuations often received is that the resuting offine poicies did itte to suggest good onine poicies. On the other hand, direct onine formuations have been disconnected from This paper is an extension of the study reported in [26].

2 2 offine formuations and the resuting poicies (optima poicies or heuristics) have euded expicit cosed form expression as opposed to offine poicies. The probem of throughput maximization in energy harvesting communication systems and networks has aso been widey studied and structura properties of throughput maximizing soutions have been investigated. For the throughput maximization probem in [] and [7], it has been proved that the offine optima soution can be expressed in terms of mutipe distinct water eves (to be made precise ater in this paper) that are non-decreasing. In [2], this resut is generaized for a continuous time system by introducing directiona water-fiing interpretation of the offine soution. Simiar resuts are aso shown in [3], [4] and [5] for the throughput maximization probem over fading channes with energy harvesting transmitters. Structura resuts on optima adjustment of transmission rate/power according to energy harvesting processes naturay have duaity reations with adapting to data arriva processes. Yet, few studies in the iterature have addressed energy harvest and data arriva constraints simutaneousy. To fi this gap, [5] studied the offine soution that minimizes the transmission competion time where both packet arrivas and energy harvests occur during transmissions under static channe conditions. In [8], the offine probem in [5] was extended to fading channes, and in [9] the broadcast channe with energy and data arrivas was considered, though the structure of the proposed soutions are not expicit and furthermore, do not provide much insight into in the derivation of onine soutions. Asymptoticay throughput optima and deay optima transmission poicies were studied in [20] under stochastic packet and energy arrivas. Onine formuations of the energy harvesting scheduing probem based on dynamic programming were considered in [7], and in [2], which suggests onine heuristics with reduced compexity. In [2], the onine soution maximizing overa throughput was formuated using a Markov Decision Process approach. The MDP approach was aso used in [22] to obtain the performance imits of energy harvesting nodes with data and energy buffers. In [23], a earning theoretic approach was empoyed to maximize ong term (infinite horizon) throughput. The competitive ratio anaysis was used in [24] for a throughput maximization probem on an energy harvisting channe with arbitrary channe variation and a simpe onine poicy was shown to have a competitive ratio equa to the number of remaining time sots. Recenty, for an energy harvesting system with genera i.i.d. energy arrivas and finite size battery, an onine power contro poicy [25] was shown to maintain a constant-gap

3 3 approximation to the optima ong term throughput average. The offine probem considering energy arrivas over fading channes has been studied in [], [7], [2] and the offine probem considering energy and data arrivas over a static channe has been investigated in [5]. To the best of our knowedge, the generic soution of the offine probem that considers energy and data arrivas together over time-varying channes has been covered excusivey in [26] and this study is an extension of [26]. Most significanty, this study presents an aternative approach to characterize the offine optima soutions as opposed to an agorithm that iterates throughout the entire schedue rather than focusing on the optima decision at a particuar time sot. The offine soution introduced in this study, differs from existing soutions mainy in the construction of transmission schedues. We characterize the offine probem as that of finding optima decisions successivey in each time sot rather than searching for a compete transmission schedue. In our offine soution, we expicity formuate the offine optima decision at a given time sot and accordingy we can construct offine schedues sot by sot as if they are onine schedues with known arriva patterns. The offine optima decisions, which are water eves individuay set for each time sot, are expressed as expicit functions of present energy and data buffer states, channe variations and future energy-data arrivas. In particuar, the effect of channe variations are separated for each individua sot with the use of channe correction terms determining the optima offine water eve. This formuation of the offine soution aows us to characterize offine optima decisions as random variabes in a stochastic probem setting as in our onine probem. The onine probem we consider in this study is formuated through stochastic dynamic programming which is aso the typica approach taken by prior studies to express the onine soution. On the other hand, different than existing dynamic programming formuations, our formuation of the onine throughput maximization probem incorporates offine optima schedues as stochastic processes that onine optima poicies shoud foow cosey and minimize expected regret due the variation of offine optima decision in each successive onine decision. Even tough the estimation of the offine optima soution is not the primary goa of the onine probem, good estimates of the offine optima soution coud capture the most of onine optima poicies. In genera, the dynamic programming soution suffers from the exponentia compexity of the optima soution as onine decisions determine the future states of the system and highy depend on the time evoution of the system state in the optima sense. In order to overcome this drawback, our onine soution

4 4 reies on offine optima decisions which aready consider future benefits in terms of energyefficiency, however the cost-to-go function in our onine formuation sti carries an importance as the system might go to a state where offine optima decisions can be better estimated. In addition to this formuation of the onine optima soution, we introduce the immediate fi approach that ower bounds the ratio of expected performance of an onine poicy reative to the expected performance of offine optima schedues and aso suggests the maximization of the immediate fi metric in every sot to maximize this ower bound. Moreover, based on the offine optima soution, we propose an onine heuristic and through numerica anaysis we show that this heuristic can achieve average throughput rates cose to the offine optima performance even in the finite probem horizons for an arbirary numerica scenario. II. SYSTEM MODEL We consider a system (see Figure ) of a point-to-point communication channe where an energy harvesting transmitter S sends data to a destination D through a time-varying channe by judiciousy adapting its transmission rate and power. The actions of S are governed by three distinct exogenous processes, namey, energy harvesting, packet arriva and channe fading. We consider the system in discrete time and over a finite horizon divided by equa time sots. Let {H n }, {B n } and {γ n } be discrete time sequences over the finite horizon n =,2,...N, representing energy arrivas, packet arrivas and channe gain, respectivey, over a transmission window of N < sots, where n is the time sot index. Particuary, H n is the amount of energy that becomes avaiabe in sot n (harvested during sot n ), B n is the amount of data that becomes avaiabe at the beginning of sot n and γ n is the channe gain observed at sot n. Let e n and b n be energy and data buffer eves at sot n, where transmit power ρ n is used in sot n and the received power is ρ n γ n. The transmit power and rate decisions ρ n and r n are assumed to obey a one-to-one reation r n = f(+ρ n γ n ) 2, where the function f( ) has the foowing properties: f(x) is concave, increasing and differentiabe. f() = 0, f (+x) < and im x f (+x) = 0. The update equations for energy and data buffers can be expressed as in beow: 2 The function f( ) is a genera performance function as in [27].

5 5 Update Equation for the Energy Buffer: e n+ = e n +H n ρ n,ρ n e n, for a n. () Update Equation for the Data Buffer: b n+ = b n +B n f(+ρ n γ n ),f(+ρ n γ n ) b n, for a n. (2) Harvested Energy {H n } Data Buffer Energy Buffer Received Data {B n } S Fading Channe {γ n } D Fig. : An iustration of the system mode. III. OFFLINE PROBLEM We consider the foowing offine probem over a finite horizon of N sots: N Maximize f(+ρ γ ) = subject to constraints in () and (2) As the probem is offine, we assume {H n }, {B n } and {γ n } time sequences are a priory known. Accordingy, energy and data constraints can be competey determined as in the foowing inequaities: ρ e n + =n =n+ n+v f(+ρ γ ) b n + =n H,u =,2,...,(N n), (3) ρ n e n, for a n. n+v =n+ f(+ρ n γ n ) b n, for a n. B,v =,2,...,(N n) (4) We make the foowing definitions to characterize offine poicies and depict a cear distinction between the concepts of energy efficiency and throughput maximization.

6 6 Tota Throughput EE-TM-OFF Schedue EE-OFF schedues FEASIBLE SCHEDULES Consumed Energy Fig. 2: An iustration of feasibe offine schedues in terms of achieved tota throughput versus consumed energy. Definition : Any coection of power eve decisions ρ = (ρ,ρ 2,...,ρ N ), satisfying energy and data constraints in (3) and (4), is a feasibe offine schedue. Definition 2: An energy efficient offine transmission (EE-OFF) schedue is a feasibe offine schedue such that there is no other feasibe offine schedue that can achieve higher throughput by consuming the same tota amount of energy or achieve the same throughput by consuming ess energy for a given reaization of {H n }, {B n } and {γ n } time series. Definition 3: Among a EE-OFF schedues, those that achieve the maximum throughput 3 are caed energy efficient thoughput maximizing offine transmission (EE-TM-OFF) schedues. To identify the schedues in an aternative way, we define water eves which wi be usefu in Theorem. Definition 4: A water eve w n is the unique soution of the foowing: ρ n = [(f ) ( ) γ n w n γ n Proposition : The water eve w n is non-decreasing in ρ n and f(+ρ γ ). Proof: As f( ) is increasing and concave, (f ) ( w nγ n ) is non-decreasing in w n. Remark : For ρ n > 0, the partia derivative of f(+ρ n γ n ) with respect to ρ n is equa to w n. Ceary, any power eve ρ n can be obtained from a propery chosen water eve w n. Hence, any offine transmission schedue can be aso defined by corresponding water eves (w,w 2,...,w n ). ] + 3 Note that not a feasibe offine schedues that maximize the tota throughput are EE-TM-OFF schedues. A schedue can be throughput optima by deivering the data received during transmission but this can be done by consuming more energy than the corresponding EE-TM-OFF schedue.

7 7 For the soution of offine throughput maximization probem, it wi be shown in Theorem that offine optima water eve for an EE-TM-OFF schedue is the maximum water eve that barey empties data or energy buffer if it is appied continuosy. Theorem : In an EE-OFF scheme, the water eve w n is bounded as: where K (e) w (e) n (w n) = w (b) n (w n) = w n min{w (e) n (w n),w (b) n (w n)} min u=0,...,(n n) min v=0,...,(n n) e n + b n + =n+ n+v =n+ H + u+ =n n+v B + =n K (e) (w n ) K (b) (w n ) v + (w n ) = w n ] ( + [ ] ) + [(f ) ( ),K (b) (w n ) = w n f + (f ) ( ) γ w n γ w n γ Particuary, water eves in an EE-TM-OFF schedue shoud satisfy the inequaity above with equaity, i.e. w n = min{w (e) n (w n),w (b) n (w n)} for a n in {,2,...,N}. Proof: See the Appendix. IV. OFFLINE PROBLEM WITH LOGARITHMIC RATE FUNCTION In the offine probem, the throughput functionf( ) coud be chosen as og 2 2( ) that represents the AWGN capacity[ of the channe. Then, the water eve w n in this case determines the power eve ρ n as ρ n = n(2) +. w γ n 2 nγ n ] For this case, an EE-TM-OFF schedue can be obtained by setting the water eve w 4 n to min{wn e,wb n } for each time sot n where we n and wb n are defined as foows: w e n = og 2 (w b n) = min u=0,...,(n n) e n + min v=0,...,(n n) n+u =n+ n+u H + M (e) (w n) u+ =n n+v b n + B + 2 =n+ n+v =n M (b) (w n) (5) (v +) (6) 2 4 Since n(2) is a constant, in the rest, we wi reset w 2 n to n(2) wn in order to simpify the notation. 2

8 8 where { } { }) M (e) (w n ) = min,w n,m (b) (w n ) = og γ 2 (min,w n γ The characterization of the offine optima water eve can be expicity expressed as in the above. Due to the correction terms M (e) (w n ) and M (b) (w n ), the offine optima water eve wn corresponds to the unique fixed point of min{we n,wb n } and shoud be computed iterativey. To find the water eve satisfying min{w e n,w b n}, any fixed point iteration method can be used. For exampe, the throughput maximizing water eve w n min{wn e,wb n } as foows: w where w () n = w max n (k+) n can be found by iterativey evauating = wn=w (k) n min{we n,w b n} (7) which is guaranteed to be higher than wn. The proposotion in the beow states that the iteration in Eq. 7 converges. Proposition 2: The sequence of water eve iterations, w () n,w (2) n,... converges to w n. w (k+) n Proof: From (5),(6) and (7), w (k+) n < w (k) n for some k, then w (k+2) n is decreasing with decreasing w (k) n. Accordingy, if < w (k+) n shoud be true and setting w () n to a arge enough vaue can guarantee that w (2) n < w () n. As w (k) n s are bounded beow by zero, the iterations converge. Unesswn is reached, the iterations have not stopped, hence the iterations wi converge to w n if w () n is above w n. The offine optima power eve ρ n computing the sequence, w () n,w (2) n,..., which converges w n that maximizes tota throughput can be approached by ρ n = im k by Proposition 2. [ w n (k) ] + (8) γ n V. ONLINE PROBLEM The onine probem formuation is an onine counterpart of the offine probem with ogarithmic 5 rate function. We formuate the probem as a dynamic programming to maximize the expected tota throughput. Let x n = (e n,b n,γ n ) be the state vector, θ n = (H n,bn,γn ) be the history and X n = (H n+,b n+,γ n+ γ n ) exogeneous processes at the sot n. 5 For the sake of simpicity, the ogarithmic rate function wi be used in onine formuations. However, simiar formuations and resuts can be obtained aso for the genera concave function f( ).

9 9 w b n w e n γ n n N Fig. 3: An iustration of an EE-TM-OFF poicy Water Leves EE-TM-OFF Expected EE-TM-OFF Sot Index Fig. 4: Sampe water eves of an EE-TM-OFF schedue and the expected EE-TM-OFF given knowedge for a sampe reaization where energy harvesting, data arriva and channe fading processes are generated by 4 state DTMCs. DefineA(x n ) as the set of admissibe decisions such that [w n γ n ] + e n and [og 2 (w n γ n )] + b n, w n A(x n ). For w n A(x n ), the dynamic program for throughput maximization can be written as beow: ˆV n θ n (x n ) = max ˆV n θn (w n,x n ) (9) w n A(x n) ˆV n θn w n,x n ) = [og 2 (w n γ n )] + +E Xn [ˆV n+ θ n+ (x n +X n φ(w n ;γ n )) x n,θ n ] (0)

10 0 where φ(w n ;γ n ) = ([w n γ n ] +,[og 2 (w n γ n )] +,0) and ψ n = (H N n+,b N n+,γ N n+) represents the exogeneous processes for sots between n and N. The soution of this dynamic programming formuation constitutes the onine optima poicy maximizing expected tota throughput to be achieved within the finite probem horizon. The drawback of this soution is that it suffers from the exponentia time/memory computationa compexity of the dynamic programming. On the other hand, when the vector ψ n = (H N n+,bn n+,γn n+ ) is deterministic, the onine probem is no different than the offine probem. The soution to the offine probem for the reaization of ψ n can be a reference for the onine probem. We observed that a poicy, which simpy appies the statistica average of EE-TM-OFF water eves as its onine water eve at each and every time sot, typicay cosey foows the origina EE- TM-OFF schedue (Fig. 4). Motivated by this observation, we consider EE-TM-OFF decisions as stochastic processes in the onine probem domain. The next subsection wi introduce an aternative dynamic programming formuation of minimizing the expected throughput oss of the onine decisions with respect to the corresponding offine optima decisions. A. Onine Soution Based On Offine Soution Let w n = w n (x n) be the offine optima water eve which is a random variabe generated over the reaizations of ψ n given the state vector x n. Then, the tota throughput achieved by appying offine optima water eves unti the end of transmission time window can be expressed as: Ṽ n θ n (x n ) = [og 2 ( w n γ n)] + +Ṽ n+ θ n+ (x n +X n φ( w n ;γ n)) () The onine throughput maximization probem can be reformuated by the foowing cost minimization probem: where J n θ n (x n ) = min w n A(x n) J n θ n (w n,x n ) (2) J n θn (w n,x n ) = E ψn [Ṽ n θ n (x n ) x n,θ n ] ˆV n θn (w n,x n ) (3) The cost function J n θn (w n,x n ) can be separated into two parts: The expected throughput achieved by appying offine water eves for sots [n, N] minus the expected throughput achieved by appying the decision w n at the sot n, then appying

11 offine optima water eves for the rest, i.e. in [n +,N]. Let E ψn [ F n ( w n,w n ) x n,θ n ] represent this term. The expected throughput achieved by appying offine water eves for sots[n+, N] minus the expected tota throughput achieved by onine optima decision for sots [n +, N] after the decision w n is appied at the sot n. Let Dn+ θ n+ (x n +X n φ(w n ;γ n )) represent this term. J n θn (w n,x n )) = E ψn [ F n ( w n,w n) x n,θ n ]+Dn+ θ n+ (x n +X n φ(w n ;γ n )) (4) Ceary, both of the terms are nonnegative for any w n since, by definition, EE-TM-OFF schedues are superior to onine throughput maximizing schedues for any given reaization. The first term E ψn [ F n ( w n(x n ),w n ) x n,θ n ] is the conditiona expectation of the variabe F n ( w n,w n) as foows: F n ( w n,w n ) = (og 2 ( w nγ n )) + (og 2 (w n γ n )) + +Ṽ n+ θ n+ (x n +X n φ( w n ;γ n)) Ṽ n+ θ n+ (x n +X n φ(w n ;γ n )) The equation in (4) can be rewritten as in beow: J n θn (w n,x n ) = E ψn [F n ( w n,w n) x n,θ n ]+D n+ θ n+ (x n +X n φ(w n ;γ n )) (5) where F n ( w n,w n) = E ψn [ F n ( w n,w n) w n,x n,θ n ]. Accordingy, the function F n ( w n,w n) can be seen as a oss function for the decision w n since it corrresponds to the throughput oss that cannot be recovered even with offine optima poicies. The expectation of this oss term wi be caed as the immediate oss of the decision w n as we define in beow. Definition 5: Define E ψn [F n ( w n,w n ) x n,θ n ] as the immediate oss of the decision w n. On the other hand, the second term D n+ θ n+ ( ) can be expressed as : where x n+ = x n +X n φ(w n ;γ n ). D n+ θ n+ (x n+ ) = E Xn [J n+ θ n+ (x n+ ) x n,θ n ] (6) Therefore, the probem has the foowing dynamic programming formuation: J n θ n (x n ) = min w n A(x n) E ψn [F n ( w n,w n) x n,θ n ]+E Xn [J n+ θ n+ (x n +X n φ(w n ;γ n )) x n,θ n ] (7)

12 2 As this formuation is equivaent to the initia formuation in (9), its soution gives the onine optima poicy. Whie the exact computation of this soution may aso have exponentia compexity, the formuation wi ead us to define the immediate fi metric which wi be a vehice toward the derivation of onine soutions with performance guarantees. B. Immediate Fi The performance of any onine poicy w can be aso evauated by the ratio of its expected tota throughput to the expected tota throughput of the offine optima poicies. Definition 6: Define the onine-offine efficiency, or simpy the efficiency of an onine poicy w as foows: η w (x n,θ n ) = ˆV n θ w n (x n ) E ψn [Ṽ n θ n (x n ) x n,θ n ] (8) where ˆV w n θ n (x n ) is the expected tota throughput achieved by the onine poicy w given the present state x n and the history θ n. Any decision in the onine schedue wi incur an immediate throughput gain. However, this decision aso may cause a oss of potentia future throughput that woud be accessibe to an offine agorithm. We ca this the immediate oss as we define in the previous section. Definition 7: Define the ratio of immediate gain to its sum with immediate oss as immediate fi. For the sot n, et µ w n (x n,θ n ) be the immediate fi of poicy w when the system is in x n with the history θ n. µ w n(x n,θ n ) = (og 2 (w n γ n )) + (og 2 (w n γ n )) + +E ψn [F n ( w n,w n ) x n,θ n ] (9) According to the above definition of the immediate fi µ w n(x n,θ n ), we wi show that the minima immediate fi of the poicy w ower bounds its onine-offine efficiency. Theorem 2: The efficiency of an onine poicy w with w N = w N is ower bounded by the minimum immediate fi observed by that poicy: Proof: See the Appendix. η w (x n,θ n ) min m n min (x m,θ m) µw m (x m,θ m ) (20) Next section considers stochastic offine optima decisions in a simper case, namey static channe case, in order to demonstrate how simpe bounds on immediate fi can be found and the distribution of offine optima decisions can be characterized.

13 3 Immediate Fi L O S S Throughput E ψn+ [Ṽ n+ θn+ (x n +X n φ(w n ;γ n ))] E ψn [Ṽ n θn (x n)] Fig. 5: An iustration of the immediate fi approach. The expectation of the achievabe tota throughput by offine optima decisions decreases as the state of the system changes due to an onine decision. Hence each onine decision opens a gap between expected throughput potentias of offine optima poicy and this gap is partiay fied by the throughput gain achieved within the corresponding sot LB 0.9 N n = 25 LB 0.9 N n = p n 5 n 55 e n = p n 5 n 55 e n = (a) (b) Fig. 6: The ower bounds of the immediate fi metric at (a) N n = 25 (b) N n = 5 for ρ n = E[ ρ n ] poicy where {H n} is a Bernoui process with Pr(H n = 0) = p and Pr(H n = 24 units) = p.

14 4 C. Resuts on the Static Channe Case In this section, we focus on the case where the channe is static, i.e. γ n = for a n, and the data buffer is aways fu, i.e. b n = for a n. Accordingy, the onine power eve and the offine optima power eve can be represented by ρ n = w n and ρ n = w n. Then, the offine optima power eve at sot n can be expressed as: ρ n = min u=0,...,(n n) e n + =n+ u+ H (2) Proposition 3: Assuming that the the channe is static, i.e. γ n = for a n, and the data buffer is aways fu, i.e. b n = for a n, the immediate fi is ower bounded as foows: µ w n (x n,θ n ) n(+ρ n ) E[n(+ ρ n)]+e [ (ρn ρ n )+ + ρ n+ where ρ n+ is the offine optima decision at sot n+ after the decision ρ n is made. Proof: See the Appendix. Proposition 4: Let µˇw n (x n,θ n ) represent the maximum (achievabe) immediate fi at sot n, i.e, µˇw n(x n,θ n ) = max w n A(x n) µw n(x n,θ n ). Then, the inequaity beow shoud hod: µˇw m (x m,θ m ) and it can be simpified as in the foowing: which impies: Proof: See the Appendix. µˇw m (x m,θ m ) µˇw m (x m,θ m ) [ ] + E (E[ ρ n ] ρ n )+ + ρ n+ n(+e[ ρ n ]) + E[(E[ ρ n ] ρ n )+ ] n(+e[ ρ n ]), Var( ρ + n) n(+e[ ρ n ]),(LB) In Fig. 6, the ower bound LB in Proposition 4 is potted against varying arriva probabiities of a Bernoui energy harvesting process at different system states of energy eve e n and remaining number of sots N n. ]

15 5 Next, we consider the CDF of ρ n under Bernoui energy harvesting assumption and characterize it for arge N, i.e. as N n goes to infinity. Theorem 3: Let {H n } be a Bernoui process with Pr(H n = 0) = p and Pr(H n = h) = p. Then, and (i) (ii) for m N + and h m < e n, im x + Pr( ρ n < r e n = x) = 0 im +Pr( ρ n < r e n = x) = x r im N + Pr( ρ n < h m e n = x) = Φ(m) mx h where Φ(m) function is the minimum vaue in (0, ] satisfying the foowing equation: Proof: See the Appendix. pφ(m) m Φ(m)+ p = 0 Pr( ρ n < r en = x) r Simuated CDF Computed CDF Fig. 7: A comparison of Monte Caro simuated CDF of ρ n at N n = 99 versus the CDF of ρ n computed for N n + using the resut in Theorem 3 where e n = 88 and {H n } is a Bernoui process with Pr(H n = 0) = 0.55 and Pr(H n = 80 units) = 0.45.

16 6 D. Onine Heuristic The onine probem formuation in the previous sections assumes statistica information on exogeneous processes energy harvesting, packet arriva and channe fading. Then, the offine optima decisions take these processes as their inputs in Eq. (5) and Eq. (6). Aternativey, a heuristic poicy coud use Eq. (5) and Eq. (6) with estimated vaues of H, B, M (e) (w n ) and M (b) (w n ). We propose such a poicy where H, =n+ n+u =n+ =n+ n+u B, =n =n M (e) (w n ) and =n =n the estimated vaues of wn e and wb n as foows: e n H n + H ŵn e (N n) n + = (e) e n + M n (w n ) where og 2 (ŵn b ) = 2b n + M (e) n (w n ) = n =n+ M (b) (w n ) are estimated through observed time averages giving 2(b n B n) (N n) + B n + H n = n n = M (b) n (w n ) n H, B n = n = M (e) (w n ), M (e) n (w n ) ; e n H n ; o.w. M (b) n (w n ) ; b n B n n = B M (b) n (w n ) = n ; o.w. n = M (b) (w n ) The estimate of the throughput maximizing water eve can be computed iterativey: ŵ (k+) n = min{ } wn=ŵ ŵ e n (k) n,ŵb n where ŵ (k) n is the kth iteration of the estimated vaue of throughput maximizing water eve and ŵ () n = min { e n,2 2bn }. (22) (23) VI. NUMERICAL STUDY OF THE ONLINE VS OFFLINE POLICIES The purpose of the numerica study is to compare the onine heuristic with the offine optima poicy, under Markovian arriva processes. For the packet arriva process, a Markov mode having two states as no packet arriva state and a packet arriva of constant size 0 KB per sot state with transition probabiities q 00 = 0.9, q 0 = 0., q 0 = 0.58, q = 0.42 where sot duration is ms and the transmission window is N = 00 sots. Gibert-Eiot channe is assumed where

17 7 good (γ good = 30) and bad (γ bad = 2) states appear with equa probabiities, i.e. P(γ n = γ good ) = P(γ n = γ bad ) = 0.5. Simiary, in energy harvesting process, energy harvests of 50nJs are assumed to occur with a probabiity of 0.5 at each sot. For a typica sampe reaization of packet arriva, energy harvesting and channe fading processes, water eve profies of throughput maximizing offine optima poicy and onine heuristic poicy are shown in Fig. 8 (a) and (b). Fig. 8 (a) shows water eve profies when transmission window size N is set to 00 sots and Fig. 8 (b) shows water eve profies when transmission window size is extended to 200 sots. In the first 00 sot, water eve profies are simiar to each other though, due to the reaxation of the deadine constraint, both optima and heuristic water eves sigthy decrease when transmission window size is doubed. To iustrate the effect of transmission window size, average throughput performances and energy consumption of throughput maximizing offine optima poicy and onine heuristic are compared against varying transmission window size in Fig. 9 (a) and (b), respectivey. The average performances of both offine optima poicy and onine heuristic tend to saturate as transmission window size increases beyond 00 sots. The experiment is repeated in Fig. 0, for the case where energy harvesting process has a memory remaning in the same state with 0.9 probabiity and switching to other state with probabiity 0.. In Fig., our onine heuristic is compared with the Power-Having poicy proposed in [7].The power-having poicy basicay operates as foows: in each sot except the ast one, it keeps haf the stored energy in the battery, and uses the other haf. It has been shown in [7], the average throughput performance of the Power-Having poicy can reach %80 %90 of average throughput of offine optima poicy. On the other hand, our onine heuristic proposed in this paper uses casua information on energy-data arrivas and channes states to achieve average throughput rate much coser to offine optima average throughput rates. Note that the parameters of Markov processes have been arbitrariy chosen as it is hard to cover a wide range of possibe settings. VII. CONCLUSION In this paper, we investigated finite horizon energy efficient transmission schemes in both offine and onine probem settings. Whie the offine probem is a direct extention to existing offine probem formuations, our characterization of the offine optima soution and the onine approach that we introduce differ from previous studies as we intend to estabish onine optimaity

18 Power (in mw) Power (in mw) Offine Optima Onine Heuristic Window Size (in number of sots) (a) 40 Offine Optima Onine Heuristic Window Size (in number of sots) (b) Fig. 8: Water eve profies of throughput maximizing offine optima poicy and onine heuristic poicy for a sampe reaization of packet arriva, energy harvesting and channe fading processes when N = 00 (a) and N = 200 (b). Avg. Throughput (in Mbit/s) Offine Optima Onine Heuristic Window Size (in number of sots) (a) Energy Consumption per sot (in nj) Offine Optima Onine Heuristic Window Size (in number of sots) (b) Fig. 9: Average throughput (a) and energy consumption per sot (b) comparison of throughput maximizing offine optima poicy and onine heuristic poicy against varying transmission window size for stationary energy harvesting.

19 9 Avg. Throughput (in Mbit/s) Offine Optima Onine Heuristic Window Size (in number of sots) (a) Energy Consumption per sot (in nj) Offine Optima Onine Heuristic Window Size (in number of sots) (b) Fig. 0: Average throughput (a) and energy consumption per sot (b) comparison of throughput maximizing offine optima poicy and onine heuristic poicy against varying transmission window size for energy harvesting with memory. 6 Avg. Throughput (in Mbit/s) Offine Optima Onine Heuristic Power-Having Window Size (in number of sots) Fig. : Average throughput comparison of throughput maximizing offine optima poicy and onine heuristic poicy and Power-Having poicy against varying transmission window size for stationary energy harvesting of 90nJs occuring with 0. probabiity in each time sot.

20 20 in reation with the statistica behavior of offine optima transmission decisions. We beieve these formuations and resuts coud be usefu aso for simiar probems where onine performance over finite durations is crucia. In particuar, the key contributions of this paper are the foowing: In an offine setting, energy efficient transmission with a generic concave rate function is studied over a finite horizon considering energy and data arrivas as we as channe variations. The soution to the offine probem is formuated by offine optima transmission decisions that depend ony on future vaues of energy harvests, data arrivas and channe variations. Based on the stochastic dynamic programming, the onine optima poicy is characterized as the poicy that successivey minimizes expected throughput osses with respect to the offine optima transmission decisions. To measure the efficiency of any onine poicy reative to the performance of the offine optima transmission poicy, the immediate fi metric is introduced. This metric can be aso used to derive new onine poicies with performance guarantees as it can be ower bounded anayticay. Considering the simper static channe case, the immediate fi of the poicy that appies the expectation of offine optima power eve as the onine power eve is ower bounded and the distribution of offine optima power eve is characterized as the probem horizon approaches to infinity. ACKNOWLEDGMENTS This work was funded in part by TUBITAK and in part by the Science Academy of Turkey under a BAGEP award. APPENDIX A. The proof of Theorem Proof: We divide the proof of Theorem into two parts: (i) We show that if the water eve of any sot n is higher than the water eve of the next sot n+ (w n > w n+ ), then, there is an offine transmission schedue which achieves at east the same throughput or consumes at the most the same amount of energy with the initia schedue, i.e. the initia schedue with w n > w n+ for some sot n is not an EE-OFF schedue.

21 2 (ii) We show that in the offine optima (EE-TM-OFF) poicy, the water eve w n is not ower than the maximum feasibe eve incurred by the inequaities resuting from the argument of part (i), i.e. w n = min{w (e) n (w n ),w (b) n (w n )} shoud be satisfied for any sot n in an EE-TM-OFF poicy. Part (i): Suppose that in a given transmission scheme π, w n > w n+ for some n. One can show that π can be improved by reducing w n and increasing w n+ through one of the foowing: (Case a) move some data form sot n to sot n+ whie keeping the tota throughput achieved during (n,n + ) fixed, (Case b) move some energy from sot n to sot n + whie keeping the tota energy consumed during (n,n+) fixed. Let ρ π n and ρπ n+ eves for sots (n,n+) beonging to the scheme π. be the transmission power (Case a) Consider the foowing convex optimization probem for sots (n, n + ): min ρ n,ρ n+ ρ n +ρ n+ f(+ρ n γ n )+f(+ρ n+ γ n+ ) = D n,n+ ρ n 0,ρ n+ 0 where D n,n+ corresponds to the tota throughput obtained by the scheme π during (n,n+), i.e. f( + ρ π nγ n ) + f( + ρ π n+γ n+ ). The Lagrangian of the above probem can be written as foows: L(ρ n,ρ n+,λ,µ n,µ n+ ) = (ρ n +ρ n+ )+λ(f(+ρ n γ n )+f(+ρ n+ γ n+ ) D n,n+ ) By setting L ρ n = 0, we get the foowing: µ n ρ n µ n+ ρ n+ γ n f (+ρ n γ n ) = µ n + λ Aso, considering the compementary sackness forµ n,µ n shoud be set to zero wheneverρ n 0. Therefore, the optima soution ρ n can be expressed as in the foowing: ρ n = [ (f ) ( ] + ) γ n λγ n

22 22 Simiary, the optima ρ n+ is as in beow: ρ n+ = [ (f ) ( ) γ n+ λγ n+ Accordingy, (ρ n + ρ n+ ) is minimized when both water eves w n and w n+ are set to λ that satisfies the tota throughput constraint. When w n > w n+, the optima water eve shoud be inside (w n,w n+ ) as the tota throughput stricty decreasing with decreasing w n as ong as ρ n > 0. Therefore, the water eves w n and w n+ can aways be equaized by transferring some data from sot n to n+. This does not vioate data causaity as the throughput at sot n is reduced whie the tota throughput achieved during (n,n+) is preserved by increasing the throughput at sot n+ to compensate. (Case b) Simiary, we consider the foowing optimization probem: ] + max f(+ρ n γ n )+f(+ρ n+ γ n+ ) ρ n,ρ n+ ρ n +ρ n+ = E n,n+ ρ n 0,ρ n+ 0 where E n,n+ corresponds to the tota energy consumption by the scheme π during (n,n+), i.e. E n,n+ = ρ π n +ρπ n+. The Lagrangian of the above probem can be written as foows: L(ρ n,ρ n+,λ,µ n,µ n+ ) = f(+ρ n γ n )+f(+ρ n+ γ n+ )+λ((ρ n +ρ n+ ) E n,n+ ) By setting L ρ n = 0, we get the foowing: µ n ρ n µ n+ ρ n+ γ n f (+ρ n γ n ) = µ n λ After setting the KKT mutipier µ n to zero where ρ n 0, we get the foowing expression for the optima ρ n: ρ n = [(f ) ( λ ] + ) γ n γ n

23 Simiary, for optimizing ρ n+ setting ρ n+ : L ρ n+ = 0 gives the foowing expression for the optima ρ n+ = [ (f ) ( λ ] + ) γ n+ γ n+ When both water eves w n and w n+ are equaized to λ 23 that satisfies the tota energy constraint, the tota throughput achieved during the sots (n,n + ) is maximized and this can be done whenever w n > w n+ by transferring energy from n and n+ without vioating energy causaity or tota energy constraints. Therefore in an EE-OFF schedue, water evesw n s are non-decreasing with increasing n. Part (ii): By the energy causaity, tota energy consumption is bounded as foows: ρ e n + =n Expressing ρ using water eves: =n γ =n+ H,u =,2,...,(N n), [ (f ) ( ] + ) e n + w γ =n+ In an optima scheme, w n w m for any sot m > n as it is proven in Part (i), thus: [ ] + (f ) [ ( ) (f ) ( ] + ) w n γ w γ And accordingy: =n γ =n γ =n γ [ ] + (f ) ( ) e n + w n γ =n+ The above inequaity shoud be satisfied for any u =,2,...,(N n) and it can be seen that w n is bounded by its owest vaue for which the inequaity hods with equaity for some u =,2,...,(N n). To find the energy bound vaue for w n, the inequaity can be transformed into the foowing form. w n e n + =n+ H + u+ =n K (e) (w n ) The maximum vaue of w n that satisfies the energy causaity is given by the foowing: w (e) n (w n) = min u=0,...,(n n) e n + =n+ H + u+ =n H H K (e) (w n )

24 24 Simiary, the data causaity bounds the water eve w n as foows: w (b) n (w n) = min v=0,...,(n n) b n + n+v =n+ n+v B + v + =n K (b) (w n ) Any EE-TM-OFF schedue is EE-OFF by definition, hence w n min{w (e) n (w n ),w (b) n (w n )} for any EE-TM-OFF schedue. We wi show that in EE-TM-OFF schedue water eve w n aso shoud not be smaer than min{w (e) n (w n ),w (b) n (w n )}, i.e. w n min{w (e) n (w n ),w (b) n (w n )}. Consider an EE-OFF schedue where w n = min{w (e) n (w n ),w (b) n (w n )} for sot n and w m min{w (e) m (w m ),w (b) m (w m )} for sots m > n since the schedue is EE-OFF. The seection of w n ony affects the throughput achieved during the sots n to N, hence if the reseection of w n as w n < min{w (e) n (w n ),w (b) n (w n )} coud improve the throughput achieved by the schedue within [n, N] whie keeping EE-OFF property, then the modified schedue coud be EE-TM-OFF. This is not possibe due to the observation in Remark. When w n = min{w (e) n (w n ),w (b) n (w n )}, to improve the tota throughput achieved in ater sots n+,n+2,...,n, some energy/data can be moved from n to ater sots, however the throughput decrease in sot n woud be arger than the possibe increase in some ater sot m > n as the derivative of the throughput with respect to power eve (Remark ) decreases with increasing water eve and w m w n in an EE-OFF poicy. Hence, seecting the water eve as w n = min{w (e) n (w n ),w (b) n (w n )} aways maximizes the tota throughput as ong as w m w n for m > n which means a of the water eves w m s after n shoud be aso seected as w m = min{w (e) m (w m ),w (b) m (w m )}. B. The proof of Theorem 2 Proof: Consider the inequaity for n = N: which means, η w (x N,θ N ) min m N min (x m,θ m) µw m (x m,θ m ) (24) η w (x N,θ N ) min (x N,θ N ) µw N (x N,θ N ) The above inequaity aways hods as the offine optima water eve of the ast sot w N is deterministic givenx N impying thatη w (x N,θ N ) andµ w N (x N,θ N ) are both equa toifw N = w N for any x N and θ N.

25 25 Now, consider the foowing inequaity: η w (x n+,θ n+ ) min min µ w m (x m,θ m ) m n+(x m,θ m) We wi show that the above inequaity impies the inequaity (20). The efficiency of the onine poicy w can be expressed as foows: η w (x n,θ n ) = (og 2 (w n γ n )) + +E Xn [ˆV n+ θ n+ (x n+ ) x n,θ n ] (og 2 (w n γ n )) + +E ψn [F n ( w n,w n )+Ṽ n+ θ n+ (x n+ ) x n,θ n ] { min µ w n (x n,θ n ), E } X n [ˆV n+ θ n+ (x n+ ) x n,θ n ] E ψn [Ṽ n+ θ n+ (x n+ ) x n,θ n ] { } min µ w n(x n,θ n ), min (x n+,θ n+ ) ηw (x n+,θ n+ ) = min min µ w m (x m,θ m ) m n (x m,θ m) Simiary, by the backward induction, the inequaity (24) impies the inequaity (20). C. The proof of Proposition 3 Proof: To obtain the ower bound in Proposotion 3 for the immediate fi of the decision ρ n = w n, we first consider the immediate oss term E ψm [F m ( w m,w m) x m,θ m ] which is basicay the expected throughput difference between the schedues(ρ n, ρ n+,..., ρ N ) and ( ρ n, ρ n+,..., ρ N ) where ρ n+,..., ρ N are offine optima power eves foowing the decision ρ n. The immediate oss is the expectation of the throughput difference in beow: where ξ(ρ n ) = N k=n+ og 2 (+ ρ k) og 2 (+ ρ n) og 2 (+ρ n )+ξ(ρ n ) k=n+ N k=n+ og 2 (+ ρ k) Then, we can bound the difference as foows: N ρ ξ(ρ n ) = og 2 (+ k ) ρ k + ρ k ( ρ k ρ k ) max N S(ρ n) k=n+ ) k og 2 (+ + ρ k k

26 26 where is the vector [ n+, n+2,..., N ] and S(ρ n ) is the set of a vectors for which is a possibe instance of the vector [ ρ n+ ρ n+, ρ n+2 ρ n+2,..., ρ N ρ N ]. We know the foowing facts for any vector in the sets(ρ n ): If S(ρ n ),0 [ ρ n+, ρ n+,..., ρ N ] and = (ρ n ρ n) since the energy consumption of both (ρ n, ρ n+2,..., ρ N ) and ( ρ n, ρ n+,..., ρ N ) schedues shoud be equa. Now, consider the case ρ n < ρ n. Ceary, ξ(ρ n) < 0 for this case since the offine optima decisions ρ ks have more energy to spend than the offine optima decisions ρ k s. Therefore, we can upper bound ξ(ρ n) considering the instances of ρ n where ρ n ρ n: N ( ξ(ρ n ) max og 2 + ) N ( k max og + ρ 2 + ) k k + ρ n+ max =(ρ n ρ n ) ρ n ρ n S(ρ n) ρ n ρn N k=n+ k=n+ sup N N + ρ n ρ n (N n)og 2 S(ρ n) ρ n ρn k=n+ ( og 2 + ) ( k = (N n)og + ρ 2 + n+ ( + ) (ρ n ρ n ) N n + ρ n+ Therefore, ξ(ρ n ) can be upper bounded as: = im N + ρ n ρn (N n)og 2 = n(2) ρ n ρ n, ρ + ρ n ρ n n+ ξ(ρ n ) n(2) (ρ n ρ n )+ + ρ n+ ) (ρ n ρ n ) N n, ρ + ρ n ρ n n+ ( + ) (ρ n ρ n ) N n + ρ n+ since ξ(ρ n ) < 0 for ρ n < ρ n. Acccordingy, [ ] E ψm [F m ( w m,w m ) x m,θ m ] E[og 2 (+ ρ (ρn ρ n n)] og 2 (+ρ n )+n(2)e )+ + ρ n+ Hence, µ w n(x n,θ n ) = og 2 (+ρ n ) og 2 (+ρ n )+E ψm [F m ( w m,w m) x m,θ m ] n(+ρ n ) E[n(+ ρ n)]+e [ (ρn ρ n )+ + ρ n+ ]

27 27 D. The proof of Proposition 4 Proof: The bound in Proposition 3 can be simpified as foows, µ w n (x n,θ n ) n(+ρ n ) E[n(+ ρ n)]+e[(ρ n ρ n) + ] When ρ n = E[ ρ n], E[n(+ ρ n)] n(+e[ ρ n]) due to Jensen s inequaity. Therefore, µˇw m (x m,θ m ) + E + + E[(E[ ρ n] ρ n) + ] n(+e[ ρ n]) [ (E[ ρ n ] ρ n )2 ] n(+e[ ρ n]) Var( ρ n ) n(+e[ ρ n]) where the ast step used Jensen s inequaity on. function. E. The proof of Theorem 3 Proof: (i) Ceary, for any given N and e n = x, ρ n is ower bounded by x N n+ which goes to infinity as x goes to infinity hence the probabiity that ρ n is smaer than some r shoud go to 0. Simiary, ρ n is upper bounded by x hence the probabiity that ρ n is smaer than some r shoud go to as x gets arbitrariy cose to r. (ii) The probabiity function Pr( ρ n < h m e n = x) can be interpreted as the probabiity that an energy outage occurs unti the end of probem horizon when the power eve is continuousy appied after the sot n where the energy eve is given as e n = x. By definition, for x h m, Pr( ρ n < h m e n = x) = h m energy/sot For x > h, the energy outage does not occur at sot n. Therefore, if it occurs, the energy outage m shoud occur after the sot n: which means: Pr( ρ n < h m e n = x) = Pr( ρ n+ < h m e n+ = x h m +H n) Pr( ρ n < h m e n= x) = ( p)pr( ρ n+ < h m e n+= x h m ) +ppr( ρ n+ < h e m n+= x h +h) (25) m

28 28 For N = n, Pr( ρ n < h m e n = x) = rect( mx h 2 ) Simiary, for N = n +, Eq. (25) gives the foowing as Pr( ρ n+ < h m e n+ = x) = rect( mx h 2 ): Now, suppose that: Pr( ρ n < h m e n = x) = rect( mx h 2 )+( p)rect(mx h 3 2 ) Pr( ρ n+ < h m e n+ = x) = rect( mx h K 2 )+ or assumming a 0,n+ =, or By Eq. (25), Pr( ρ n+ < h m e n+ = x) = K m+ + j= + K j=0 j= a j,n+ rect( mx h 2 j) a j,n+ rect( mx h 2 j) Pr( ρ n < h m e n = x) = rect( mx h 2 ) (( p)a j,n+ +pa j+m,n+ )rect( mx h 2 j) K j=k m+ Pr( ρ n < h m e n = x) = (( p)a j,n+ rect( mx h 2 j) K+ j=0 a j,n rect( mx h 2 j) where a 0,n =, a j,n = (( p)a j,n+ + pa j+m,n+ ) for j =,2,...,K m + and a j,n = ( p)a j,n+ for j = K m+2,...,k. By induction, it can be seen that K = N n: Pr( ρ n < h m e n = x) = N n+ j=0 a j,n rect( mx h 2 j) As N +, a j,n+ a j,n, the reccurence reation a j,n = ( p)a j,n +pa j+m,n shoud hod and it can be satisfied when a j,n = Φ(m) j where: pφ(m) m Φ(m)+ p = 0

29 29 Accordingy, or im N + Pr( ρ n < h m e n = x) = j=0 Φ(m) j rect( mx h 2 j) im N + Pr( ρ n < h m e n = x) = Φ(m) mx h Note that the equation pφ(m) m Φ(m)+ p = 0 aways has a root at Φ(m) =. However, if the equation has a root in (0,), Φ(m) shoud be equa that vaue since 0 < a j,n < uness a j,n = and a j+m,n =. Hence, Φ(m) is the minimum root of pφ(m) m Φ(m)+ p = 0 in (0,]. REFERENCES [] E. Uysa-Biyikogu, B. Prabhakar, and A. E. Gama, Energy-efficient packet transmission over a wireess ink, IEEE Trans. on Networking, vo. 0, pp , Aug [2] R. A. Berry and R. G. Gaager, Communication over fading channes with deay constraints, IEEE Trans. on Information Theory, vo. 48, pp , May [3] P. Nuggehai, V. Srinivashan, and R. R. Rao, Deay constrained energy efficient transmission strategies for wireess devices, in Proc. IEEE INFOCOM, vo. 3, June 2002, pp [4] M. A. Zafer and E. Modiano, A cacuus approach to energy-efficient data transmission with quaity of service constraints, IEEE Trans. on Networking, vo. 7, pp , June [5] J. Yang and S. Uukus, Optima packet scheduing in an energy harvesting communication system, IEEE Trans. on Communications, vo. 60, pp , January 202. [6] K. Tutuncuogu and A. Yener, Optimum transmission poicies for battery imited energy harvesting nodes, IEEE Trans. on Wireess Communications, vo., pp , March 202. [7] M. A. Antepi, E. Uysa-Biyikogu, and H. Erka, Optima packet scheduing on an energy harvesting broadcast ink, IEEE Journa on Seected Areas in Communications, vo. 29, pp , September 20. [8] S. Chen, P. Sinha, N. B. Shroff, and C. Joo, Finite-horizon energy aocation and routing scheme in rechargeabe sensor networks, in Proc. IEEE INFOCOM, Apri 20, pp [9] O. Oze, K. Tutuncuogu, J. Yang, S. Uukus, and A. Yener, Adaptive transmission poicies for energy harvesting wireess nodes in fading channes, in Information Sciences and Systems (CISS), 20 45th Annua Conference on, March 20, pp. 6. [0] M. Gatzianas, L. Georgiadis, and L. Tassiuas, Contro of wireess networks with rechargeabe batteries, IEEE Trans. on Communications, vo. 9, pp , Feb [] C. K. Ho and R. Zhang, Optima energy aocation for wireess communications powered by energy harvesters, in Proc. IEEE Int. Symposium on Information Theory, June 200, p

30 30 [2] O. Oze, K. Tutuncuogu, J. Yang, S. Uukus, and A. Yener, Transmission with energy harvesting nodes in fading wireess channes: Optima poicies, Seected Areas in Communications, IEEE Journa on, vo. 29, no. 8, pp , September 20. [3] O. Oze, J. Yang, and S. Uukus, Optima scheduing over fading broadcast channes with an energy harvesting transmitter, in Computationa Advances in Muti-Sensor Adaptive Processing (CAMSAP), 20 4th IEEE Internationa Workshop on, Dec 20, pp [4] O. Orhan, D. Gunduz, and E. Erkip, Throughput maximization for an energy harvesting communication system with processing cost, in Information Theory Workshop (ITW), 202 IEEE, Sept 202, pp [5] N. Roseveare and B. Natarajan, A structured approach to optimization of energy harvesting wireess sensor networks, in Consumer Communications and Networking Conference (CCNC), 203 IEEE, Jan 203, pp [6] D. Gunduz, K. Stamatiou, N. Micheusi, and M. Zorzi, Designing inteigent energy harvesting communication systems, Communications Magazine, IEEE, vo. 52, no., pp , January 204. [7] C. K. Ho and R. Zhang, Optima energy aocation for wireess communications with energy harvesting constraints, Signa Processing, IEEE Trans. on, vo. 60, no. 9, pp , Sept 202. [8] F. Ozceik, G. Uctu, and E. Uysa-Biyikogu, Minimization of transmission duration of data packets over an energy harvesting fading channe, Communications Letters, IEEE, vo. 6, no. 2, pp , December 202. [9] H. Erka, F. Ozceik, and E. Uysa-Biyikogu, Optima offine broadcast scheduing with an energy harvesting transmitter, EURASIP Journa on Wireess Communications and Networking, vo. 203, no., p. 97, 203. [Onine]. Avaiabe: [20] V. Sharma, U. Mukherji, V. Joseph, and S. Gupta, Optima energy management poicies for energy harvesting sensor nodes, Wireess Communications, IEEE Trans. on, vo. 9, no. 4, pp , 200. [2] A. Sinha and P. Chaporkar, Optima power aocation for a renewabe energy source, in Communications (NCC), 202 Nationa Conference on, Feb 202, pp. 5. [22] R. Srivastava and C. Koksa, Basic performance imits and tradeoffs in energy-harvesting sensor nodes with finite data and energy storage, Networking, IEEE/ACM Trans. on, vo. 2, no. 4, pp , Aug 203. [23] P. Basco, D. Gunduz, and M. Doher, A earning theoretic approach to energy harvesting communication system optimization, Wireess Communications, IEEE Trans. on, vo. 2, no. 4, pp , Apri 203. [24] R. Vaze, R. Garg, and N. Pathak, Dynamic power aocation for maximizing throughput in energy-harvesting communication system, IEEE/ACM Trans. on Networking, vo. 22, no. 5, pp , Oct 204. [25] D. Shaviv and A. Ozgur, Universay near-optima onine power contro for energy harvesting nodes, in 206 IEEE Internationa Conference on Communications (ICC), May 206, pp. 6. [26] B. T. Bacinogu and E. Uysa-Biyikogu, Finite horizon onine azy scheduing with energy harvesting transmitters over fading channes, in 204 IEEE Internationa Symposium on Information Theory (ISIT), June 204, pp [27] V. Rodriguez and R. Mathar, Generaised water-fiing: costy power optimay aocated to sub-carriers under a genera concave performance function, in Information Sciences and Systems (CISS), th Annua Conference on, March 200, pp. 3.