Cooperative Energy-Harvesting Communication Systems. Arin Minasian

Transcription

1 Cooperative Energy-Harvesting Communication Systems by Arin Minasian A thesis submitted in conformity with the requirements for the degree of Master of Applied Sciences Graduate Department of Electrical and Computer Engingeering University of Toronto c Copyright 2014 by Arin Minasian

2 Abstract Cooperative Energy-Harvesting Communication Systems Arin Minasian Master of Applied Sciences Graduate Department of Electrical and Computer Engingeering University of Toronto 2014 In this thesis, we study the problem of throughput maximization in an energy-harvesting, two-hop, amplify-and-forward relay network. We obtain optimal transmission policies for offline and online settings. In the offline setting, we assume that non-causal knowledge of the harvested energy and the fading channel states is available. We propose an effective algorithm to solve the power allocation problem in this case. For the online setting, we assume that the information for the energy harvesting process and the fading channels is known causally. In this case, we cast the problem as a Markov decision process (MDP). We also consider the case where power control at the transmitting nodes is limited to on-off switching. To address the inherent complexity of the MDP formulation, we also propose a computationally simple power allocation scheme. The performance of the proposed schemes are evaluated using computer simulations and are compared to existing methods in the literature. ii

3 Dedication To my family iii

4 Acknowledgements I would like to express my sincere gratitude to many people who supported me during the course of my studies. First and foremost, I would like to thank my supervisors, Professor Raviraj Adve and Professor Shahram ShahbazPanahi for their helpful support and guidance. I greatly appreciate their thoughtful advices and words of encouragement. I couldn t have asked for better supervisors. I would like to thank my friends for their friendship and support in every aspect of my graduate life. Most importantly, I would like to thank my parents, Serozh and Hasmik Minasian and my sisters Siuneh and Arineh for their never ending love and support through different stages of my life. iv

5 Contents 1 Introduction Energy Harvesting Technologies Relevant Literature Thesis Contributions Organization of this Thesis Throughput Maximization in the Offline Setting System and Signal Model Problem Formulation High SNR Approximation Biconcavity and Alternate Convex Search Numerical Results Summary Throughput Maximization in the Online Setting Preliminaries Problem Definition Decision Epochs and the Decision Making Horizon States and Actions Rewards and Transition Probabilities Decision Rules and Policies Finite Horizon MDP Formulation The Expected Total Reward Optimality Criteria On-Off Power Control Case Infinite Horizon MDP Formulation The Expected Total Discounted Reward Optimality Criteria Suboptimal Solutions Harvesting Rate Assisted Scheme A Threshold Based Approach Numerical Results Summary Conclusions Future Work v

6 Bibliography 53 vi

7 Chapter 1 Introduction The proliferation of wireless devices and the spread of wireless communication networks render the task of energy supply an ever-growing challenge. Reliable and sustainable energy sources should be deployed to guarantee effective performance of wireless networks. Although the state of the art energy storage technologies and batteries provide a convenient and cost-effective solution to this problem in short-term, these units can not serve as the sole power source in the long run. For example, in industrial wireless networks, the expected continuous operation lifetime exceeds 10 years while no battery can provide such a lifetime for a wireless node [1]. Furthermore, battery replacements can be challenging and expensive or in some cases impossible. For instance, biomedical devices such as pacemakers require surgical operations for battery replacements. In recent years, energy harvesting (EH) technologies are emerging as a promising method to power the nodes of a wireless communication network. Energy harvesting communication devices are able to harvest ambient energy from their surrounding environment, and are therefore less dependent to nonrenewable energy sources and can enjoy a prolonged lifetime. The rechargeable battery in an energy harvesting transmitter or receiver can serve as the primary or a backup power source. The idea of using renewable ambient energy sources rises the possibility of having green communication networks. Using energy harvesting devices in communication systems changes the nature of the challenges faced when designing optimal performance schemes for such networks. The main challenge in the design of energy harvesting communications systems is that the available energy at any time instance is essentially a random variable. In other words, the ambient energy becomes available at the nodes in random quantities and at random times throughout the communication horizon. Hence, optimal transmission policies, that account for the time-varying available energy, must be designed so that the network can exploit this random ambient energy arrivals in the most efficient way. Wireless communication can suffer heavily from multipath fading. Employing multiple-input multipleoutput (MIMO) systems in wireless networks is a widely acknowledged approach to overcome detrimental effects of fading. In these systems, several antennas are used in order to transmit or receive signals. Transmitting multiple copies of a signal from the transmitter to the receiver over statistically independent paths creates diversity. By appropriately combining these signals, a large gain in the performance of the system can be achieved. To achieve this diversity gain, the antennas at a wireless device should be appropriately spaced from each other to provide statistically independent channels from the transmitter to the receiver. However, in certain wireless communication devices such as cell phones or wireless sen- 1

8 Chapter 1. Introduction 2 sors, due to space, power and hardware implementation limitations, it is not feasible to install several antennas. One way to overcome this issues is for users to share their communication resources in a cooperative manner to create a virtual multiple-antenna system, thereby producing spatial diversity and increasing the performance of the system. In such a system, each user can relay another users data along with its own traffic, providing the other user with an additional path to the receiver, thus generating diversity. Cooperative behavior allows users to benefit from increased performance and reliability in terms of bit error rate, block error rate, outage probability, etc. On the other hand, user cooperation gives rise to several challenges in the design of communication systems. Issues such as increased interference in networks, specific transmit and receive requirements and the need for the design of new techniques for resource allocation in the network must be addressed in this context. Motivated by the advantages offered by cooperation in communication systems, we aim to study the use of energy harvesting technologies in the context of cooperative wireless communication systems. 1.1 Energy Harvesting Technologies In this section we will briefly review and compare some of the existing energy harvesting technologies. We refer the reader to [2], [3] and references therein, for a more detailed survey of energy harvesting devices. Energy harvesting is the process of converting the energy present in the environment to usable electrical energy and storing it in a storage unit [2]. The ambient energy can be harvested from various sources such as light, radio waves, acoustics, mechanical vibrations, airflow, heat and temperature variations and ocean waves. Solar energy harvesting devices provide the highest output power levels compared to the other energy harvesting technologies. In solar energy harvesters, also known as solar cells, the light from the sun or other illuminated sources is converted to electricity by means of photovoltaic cells. The performance of the solar energy harvesters is highly dependent on the environmental conditions such as the intensity and the consistency of the incident light, therefore, the application of solar energy harvesting technologies can be limited. For example, standard crystalline silicon solar cells with less than 20% conversion efficiency produce 100 mw/cm 2 in a bright sunny day and approximately 100 µw/cm 2 in an illuminated office environment [3]. Ongoing research in this field aims to increase the efficiency of the photovoltaic cells. The best research devices have achieved a conversion efficiency of 34% [3]. While photovoltaic cells can serve as convenient energy sources for outdoor use, generating useful electrical power in indoor conditions requires large surface areas of solar cells. In this case, vibrational energy harvesting mechanisms are more favorable to use, especially, in industrial environments, where the constant mechanical vibrations from different equipment can be used to power wireless communication devices [1]. There are several methods for converting mechanical vibrations to electrical energy as listed below Piezoelectric (PZ) materials : Piezoelectricity is the ability to create electrical potential from mechanical stress. Applying mechanical strains across PZ materials, such as crystals or certain types of ceramics, results in charge separation. This separation generates a surface charge on the material, producing an electrical field. The voltage drop across this field is proportional to the

9 Chapter 1. Introduction 3 Table 1.1: Comparison of vibrational energy harvesting technologies [2]. Mechanism Energy density Current size Drawbacks Piezoelectric 35.4 mj/cm 3 Macro Low output voltages. Electromagnetic 24.8 mj/cm 3 Macro Very low output voltages. Electrostatic 4 mj/cm 3 Integrated Need for initial charge source. mechanical strains that are being applied to the material. With the oscillations of the mechanical load, the PZ device can serve as an alternating current (AC) power source [2]. Inductive systems : In these systems, a coil attached to a vibrating object passes through an static magnetic field provided by a stationary magnet. With the movement of the coil through a changing magnetic flux, according to Farady s law, an electric potential is induced across the coil. This voltage drop can in turn be amplified using transformers. Electrostatic (capacitive) systems : This method makes use of varactors which are capacitors with variable capacity. As the plates of an initially charged varactor are separated due to vibrations, mechanical energy is converted to electrical energy. For example, if the voltage drop across the plates of the capacitor is kept constant, separation of the plates will decrease the capacity, resulting in a flow of electrical charges out of the device. One of the main advantages of capacitive energy harvesting systems is that they can be easily implemented on ICs. The amount of the energy harvested from mechanical vibrations depends on the frequency of the vibrating source; most of the vibrational energy harvesters are tuned for specific frequency bands and thus their usability can be limited to certain environments. Table 1.1 summarizes the output energy densities, the implementation size and the drawbacks of the vibrational energy harvesting mechanisms mentioned above. Human power is another potential source of ambient energy. Everyday, an average human body uses roughly 10.5 MJ of energy or an equivalent 121 W of power [1]. This makes human activities an attractive energy source. The most promising method to harvest energy from human body is by tapping the gait. On average, humans exert 130% of their body weight across their shoes during a heel strike. This is equivalent to 7 W of generated power for a 70 Kg person at a 1 Hz walk [3]. This amount of power is potentially available from heel strike alone while no extra effort is required on humans side to generate this power. Different methods can be used to harvest this energy, examples include piezoelectric insoles or hydraulic piezoelectric actuator shoes. A summary of the performance of different energy harvesting technologies is presented in Table 1.2. It is worth mentioning that although the scope of harvesting energy from environment is not limited to the aforementioned methods, these technologies are best suited for use in communication systems. Other ambient energy sources such as heat, temperature differences in the environment or the radio frequency waves provide output power levels in orders of µw and are not appropriate for use in communication devices.

10 Chapter 1. Introduction 4 Table 1.2: Demonstrated performance of various energy harvesting technologies [3]. Energy Source Performance Notes Ambient light Heel strike Vibrational micro-generators 100 mw/cm 2 (direct sunlight) 100 µw/cm 2 (illuminated office) 7 W potentially available (1 cm deflection at 70 Kg per 1 Hz walk) 4 µw/cm 3 (human motion - Hz) 800 µw/cm 3 (machines - KHz) Common polycrystalline solar cells are 16% 17% efficient, while standard monocrystalline cells approach 20%. The numbers at left could vary widely with the intensity of the light in given environment s. Demonstrated systems: 800 mw with dielectric elastomer heel, mw with hydraulic piezoelectric actuator shoes, 10 mw with piezoelectric insole. Highly dependent on excitation. Large structure can achieve higher power densities. 1.2 Relevant Literature Energy harvesting communication systems have attracted a great deal of attention in the research community in recent years. Investigations in this field can be categorized into two cases: Offline case: In this case, it is assumed that the energy harvesting profiles of the nodes of the network are known non-causally, prior to the beginning of the transmission. More specifically, in this scenario, the arrival times and the amounts of the harvested energy packets are available deterministically. Furthermore, the channel state information is also assumed to be available. Although these assumptions may not be realistic, this approach provides an upper-bound for the performance of the energy harvesting communication systems in different scenarios. Online case: In this case, the energy harvesting and the channel fading profiles are known causally. In other words, at each time instance, only the history and the current information of energy arrivals and the fading channel states are available. The statistics of the random energy packets and the fading channel coefficients may also be available. Clearly, these assumptions are more in accordance with the real-life situations. One of the pioneering works in the field of energy harvesting communications was the study of the information theoretic capacity of a point-to-point energy harvesting communication system in [4]. The authors derived the capacity of a point-to-point wireless communication channel with additive white Gaussian noise (AWGN) and an energy harvesting transmitter. It is shown that the capacity of an AWGN channel with random energy arrivals is equal to the capacity of a conventional (non-energy harvesting) AWGN channel with an average power constraint equal to the average recharge rate of the EH transmitter. Two different capacity achieving schemes, namely save-and-transmit and best-efforttransmit were introduced. The basic intuition behind these capacity achieving schemes is the fact that if the transmitter remains idle for a long enough period, using a battery to store the energy in this period will help eliminate the randomness of the energy arrivals. In this publication, the authors showed that the prior knowledge of the stochastic energy arrivals will not increase the capacity of the AWGN channel as long as the battery capacity of the EH transmitter is unlimited.

11 Chapter 1. Introduction 5 The optimal packet scheduling problem for a point-to-point communication system with an energy harvesting transmitter is studied in [5]. The transmitter receives data packets and energy packets randomly during the transmission. The goal is to minimize the delivery time of the received data packets to the destination. This is achieved by adaptive control of the transmission rate (which is a concave function of the transmission power), according to the available data packets and the amount of energy present in the battery of the EH transmitter. This work considers an offline setting, i.e., it uses non-causal knowledge of the energy and data packet arrivals. It is shown that the transmission power must remain constant between data packet or energy packet arrivals. It is also proven that in the optimal power allocation scheme, transmit power levels form a non-decreasing sequence over time. Using these structural properties, the optimal policy is obtained using a geometric framework similar to that introduced in [6]. In [7], the authors investigate the problem of short-term throughput maximization for a wireless link with a rechargeable transmitter. In this work, the problem is studied under both offline and online settings. In the offline setting, by applying the Karush-Kuhn-Tucker (KKT) conditions to the throughput maximization problem, the authors introduce a novel directional waterfilling algorithm to solve for the optimal transmission power levels. In this algorithm, as opposed to the conventional waterfilling algorithm and due to energy causality constraints, the excessive energy at each transmission interval can only be distributed to the next intervals and not to the earlier blocks. Furthermore, in this scheme, the water level is not constant across the entire transmission interval and it changes according to the amount of available energy at different blocks. In the online setting, stochastic dynamic programming is used to derive the optimal online power allocation policy under random fading and energy arrival processes. Two suboptimal power allocation approaches with low computational complexity are also introduced in this work. Moreover, the authors show that the problem of throughput maximization is equivalent to the problem of transmission time minimization in [5]. Assuming a point-to-point wireless communication system, the authors of [8] have also examined the problem of power allocation for throughput maximization over a finite horizon. Structural properties of the optimal power allocation scheme are obtained using non-causal and causal information of the harvested energy and channel conditions. A general framework for the optimization of point-to-point communication systems with an energy harvesting transmitter is introduced in [9]. In this work, the limitations caused by battery imperfections and leakages is taken into consideration. The authors identify the structure of the throughput-optimal transmission policy for the EH communication system. In [10], the problem of maximizing the rate of information transfer is examined for a single-link communication system under a fading channel. This work examines the problem in the online setting. To find the optimal transmission policy, the authors of [10] have used discrete dynamic programming by formulating the problem as a Markov decision process (MDP). It is also shown that the optimal transmission policy is a non-decreasing function of the battery level and the fading channel state. Another online investigation for throughput maximization in a point to point EH wireless communication system is carried out in [11]. In this publication, the goal is to find the optimal transmission power as a function of the available energy in the battery at each time instance. In this work, it is assumed that the energy harvesting process is a compound Poisson process and the channel between the source and the destination is static. These assumptions lead to a compound Poisson dam model for the energy storage unit. Using a calculus-of-variations approach, Mitran obtains a necessary condition for optimality of the online power consumption policies. This condition is used to derive the transmission power as a function

12 Chapter 1. Introduction 6 of the battery content. This work is extended to the case with a time variant fading channel in [12], where the fading channel is modeled as a continuous-time finite-state Markov chain. An upper-bound to the ergodic channel throughput is also derived in [12]. In [13], a similar approach to [11] is used to find the optimal transmission policy for sum-rate maximization in a multiple access EH communication system. The studies mentioned assume that the power consumption in an EH transmitter is only due to transmission power and the power consumption for other purposes is neglected. The authors of [14 16], have adapted an alternative approach. In these publications, the processing energy costs and the nonideality of the RF circuits are also considered in the power consumption model. In [14], assuming that the consumed power is a convex function of the transmission power, the authors have investigated the problem of throughput maximization for a point-to-point EH communication system in the offline setting. A similar study is carried out in [15] where the authors havemodeled the consumed power in the transmitter as a linear function of the transmission power. It is shown that the resulting optimal policy is different from the results of the directional waterfilling algorithm of [7] where the authors have assumed that there is no processing energy cost. In [16] a directional glue pouring algorithm is introduced to solve for optimal power levels that maximize the throughput of an EH point-to-point communication system with processing energy costs. The use of energy harvesting has also been studied in the context of relay networks. One of the earliest studies in the field of EH cooperative communication systems was carried out in [17]. In this work, the source communicates with the destination through a set of EH relays. These relays volunteer to amplify and forward the message from the source when they have enough energy for transmission. The authors show that the design considerations for an EH cooperative communication system are significantly different from that for a conventional relay network. In [18 21], the problem of throughput maximization in a single-relay, two-hop communication system with energy harvesting nodes is studied in the offline setting, assuming non-causal knowledge of the energy harvesting profiles of transmitting nodes. In [18], the optimal energy-use scheme for an energy harvesting source and relay is obtained when the relay operates in a full-duplex mode. Also, for the case of half-duplex relaying, the problem is solved for the case where one energy packet arrives at the source and multiple energy packets arrive at the relay. The problem of throughput maximization is solved for an energy harvesting source and a non-energy harvesting half-duplex relay in [19], where the directional waterfilling algorithm introduced in [7] is used to derive the optimal energy-use solution. In [20], it is assumed that both the source and the relay can harvest energy during transmission. The properties of the optimal energy-use policy, which maximizes the number of delivered bits up to a given deadline, are introduced. However, the study in [20] considers the case of two energy packet arrivals at the source and at the relay. The case of multiple energy arrivals at the relay is studied in [21]. The approach common to [18 21], is that in these investigations, the problem is studied based on deriving the properties of the optimal transmission policy rather than applying standard optimization methods. In other words, rather than searching for the optimal solution in a feasible set, these investigations aim to characterize the optimal policies and construct a solution which satisfies these characteristics. Moreover, they all consider continuous-time transmission, and mainly focus on full-duplex relaying scenarios. The problem of throughput maximization for the Gaussian relay channel with energy harvesting constraints is considered in [22]. The authors of [22] assume that the relay operates in half-duplex mode and

13 Chapter 1. Introduction 7 uses the decode-and-forward (DF) relaying protocol. The study conducted in [22] is extended in [23] by investigating the problem of throughput-maximization for buffer-aided link adaptive energy harvesting relaying systems. In [24], the authors examine joint relay selection and power allocation for throughput maximization in an amplify-and-forward (AF) relay network. A high signal to noise ratio (SNR) approximation (HSA) is used by the authors of [24] to obtain the energy-use policy, assuming a priori knowledge of the energy harvesting profiles of the nodes of the networks. In [24], two sub-optimal but computationally simple online transmission schemes, namely the harvesting rate (HR) assisted scheme and the naive scheme, are also introduced. In the naive scheme, also known as the greedy scheme in the literature, the harvested energy is used immediately and no energy is saved for future use. The harvesting rate assisted scheme is explained in Section Thesis Contributions We consider a two-hop amplify-and-forward (AF) relaying system with an energy harvesting source and relay. We aim to maximize the total throughput of the system over a finite time horizon under energy harvesting constraints. The problem of throughput maximization for this system in the offline setting has been examined in the [24], however, there is still room for a more efficient solution. Furthermore, the authors of [24] provide no insight in how the available energy is used in the optimal power allocation scheme. On the other hand, no mathematically tractable solution has been introduced to tackle this problem in the online setting which is a more realistic scenario compared to the offline case. In addition, there is need for an efficient but computationally simple power allocation scheme which can be implemented in case the nodes of the network have limited processing capability. This thesis addresses these issues by making the following contributions: We present a novel solution to tackle the problem of throughput maximization for an AF halfduplex relay network in the offline setting. By applying KKT conditions, we derive several properties of the optimal power allocation scheme in the high-snr regime. For the online setting, we propose an MDP-based approach to find the optimal transmission policies and derive certain properties of the optimal solution for the special case of on-off power control at the nodes. We introduce a low-complexity heuristic approach for the cases where the computational cost associated with implementation of the MDP solution may not be affordable. The results of the throughput maximization problem in the offline settings have been published in [25] and [26]. The investigation for the online setting is presented in part in [26] and [27]. Throughout this thesis, we assume that the power used is for transmission only and we ignore circuit power in the nodes.

14 Chapter 1. Introduction Organization of this Thesis In Chapter 2, the problem of throughput maximization for a two-hop amplify and forward relay network is considered in the offline setting, i.e., assuming non-causal knowledge of the energy harvesting process and channel fading information. Using KKT conditions for the high-snr settings, we prove certain interesting structural properties of the optimal power allocation policies. Furthermore, the alternative convex search (ACS) algorithm is introduced to find the optimal power levels. Chapter 3 includes the MDP formulation of the throughput maximization problem in the online setting and explores certain properties of the optimal transmission policy in the special case of on-off power control. The MDP formulation is carried out for two cases of finite and infinite transmission horizons. A simple heuristic power allocation method is also introduced in Chapter 3. Chapter 4 presents final remarks and discusses possible future extensions to our work.

15 Chapter 2 Throughput Maximization in the Offline Setting In this chapter, we propose an algorithm to maximize the total throughput in an energy harvesting two-hop amplify and forward (AF) relay network in finite signal-to-noise ratio (SNR) regimes. This algorithm is for the offline setting, assuming non-causal knowledge of the energy harvesting profiles of the source and the relay in the network. Furthermore, using the method of Lagrangian multipliers, we present some properties of the optimal power levels in the high-snr regime. 2.1 System and Signal Model The system under consideration is depicted in Figure 2.1. The source (S) and the relay (R) can harvest energy from the environment. The transmission takes place over a time horizon comprising N transmission blocks 1, each of length 2T. Each transmission block includes two time slots. In the first time slot, the source transmits its message to the relay. In the second time slot, the relay retransmits an amplified version of the signal from the source to the destination. We assume that the source and the relay can adaptively change the transmission rate by changing the transmission power. Furthermore, we will assume that the number of channel uses in each slot is large enough so that the AWGN channel capacity expression in [28] is a valid measure for the rate of the transmission. The channel coefficients, in the i-th transmission block, corresponding to the source-relay (S-R) and the relay-destination (R-D) links are denoted by h sr (i) and h rd (i), respectively. We assume that there is no direct link between the source and the destination. The source and the relay harvest E s (i) and E r (i) units of energy at the beginning of the i-th transmission block. The energy consumed for purposes other than transmission is neglected, specifically we ignore energy for receiving and processing the received signals at the relay and destination. At the beginning of the i-th block, the energy stored in the battery of the source and that of the relay are denoted as B s (i) and B r (i), respectively. The source and the relay batteries have limited capacities, denoted as Bs max and Br max, respectively. The data buffer size of the relay is assumed to be infinite so there will be no data loss due to limited data storage capacity in the relay. 1 Throughout this thesis, the phrases transmission block and transmission interval are used interchangeably. 9

16 Chapter 2. Throughput Maximization in the Offline Setting 10 E s (i) E r (i) h sr (i) h rd (i) S R D Figure 2.1: A single-relay two-hop network with energy harvesting nodes. During the i-th transmission block, the signal x s (i) transmitted by the source using power p s (i) is given by x s (i) = p s (i)m(i) (2.1) wherem(i) isthei-thtransmittedmessagewith E{ m(i) 2 } = 1, ande{ }denotesstatisticalexpectation. The signal y r (i) received at the relay and the corresponding SNR can respectively be written as y r (i) = h sr (i) p s (i)m(i)+n sr (i) (2.2) SNR r (i) = h sr(i) 2 p s (i) σ 2 sr where n sr (i) is the additive white Gaussian noise with zero mean and variance σsr 2. In the second time slot of the i-th block, the relay transmits signal x r (i) which is given by (2.3) x r (i) = α(i)y r (i) (2.4) where α(i) denotes the signal amplification at the relay during the i-th transmission block. Denoting the relay transmit power in the i-th transmission interval as p r (i), α(i) is given by p r (i) α(i) = p s (i) h sr (i) 2 +σsr 2. (2.5) At the destination, the received signal y d (i) and the corresponding SNR are, respectively, expressed as y d (i) = α(i) p s (i)h rd (i)h sr (i)m(i)+α(i)h rd (i)n sr (i)+n rd (i) (2.6) SNR d (i) = γ sr (i)p s (i)γ rd (i)p r (i) γ sr (i)p s (i)+γ rd (i)p r (i)+1 where γ sr (i) h sr (i) 2 /σsr 2 and γ rd (i) h rd (i) 2 /σrd 2. In what follows, without loss of generality, we will assume that σsr 2 = σ2 rd = 1. We also assume that perfect channel state information at the receiver (CSIR) is available for decoding. (2.7)

17 Chapter 2. Throughput Maximization in the Offline Setting Problem Formulation In this section we try to tackle the throughput maximization problem in the offline setting, that is assuming that the energy harvesting profile of the source and that of the relay are known non-causally. Furthermore, it is assumed that the channel coefficients are known non-causally before the beginning of the transmission. Although these assumptions may not be realistic, devising transmission policies under these assumptions provides the performance benchmark for energy harvesting relay networks. The throughput of the system over N transmission blocks is given by R(p s,p r ) = 1 2 N log(1+snr d (i)) = 1 2 N ( ) γ sr (i)p s (i)γ rd (i)p r (i) log 1+ γ sr (i)p s (i)+γ rd (i)p r (i)+1 (2.8) where p s [p s (1),,p s (N)] T, p r [p r (1),,p r (N)] T and the factor 1/2 is used to signify that the relay operates in a half-duplex mode. In (2.8), the log( ) is calculated in base 2 and we have assumed that the number of channel uses in each transmission block is big enough so that the Shannon s channel capacity formula is an appropriate measure for the throughput of the system. One of the main considerations in designing optimal transmission policies for energy harvesting communication systems is that a transmitter cannot use the energy packets which are yet to arrive. These constraints, often referred to as energy consumption causality constraints, can be expressed as k p s (i) 1 T k p r (i) 1 T k E s (i), for k = 1,...,N (2.9) k E r (i), for k = 1,...,N. (2.10) Throughout the transmission, i.e., for i = 1,,N, the energy stored in the source battery and that stored in the relay battery are changing as B s (i) =f s (B s (i 1),p s (i 1),E s (i)) min{b s (i 1) Tp s (i 1)+E s (i),b max s } (2.11) B r (i) =f r (B r (i 1),p r (i 1),E r (i)) min{b r (i 1) Tp r (i 1)+E r (i),b max r } (2.12) where B s (0) and B r (0) correspond to the initial energy stored in the source and the relay batteries, respectively, and p s (0) = p r (0) = 0. Note that, (2.11) and (2.12) imply that the harvested energy at the beginning of the i-th transmission block can be used for transmission in that interval. At the beginning of a transmission interval, if there is not enough battery capacity to store the newly arrived energy packet, the energy will be wasted. As battery overflow is an undesired situation, we avoid this situation by imposing the following constraints on our problem: k+1 E s (i) T k+1 E r (i) T k k p s (i) B max s, for k = 1,...,N 1 (2.13) p r (i) B max r, for k = 1,...,N 1. (2.14) Note that the set of constraints in (2.13) and (2.14) can be applied only for the offline setting since in

18 Chapter 2. Throughput Maximization in the Offline Setting 12 this case the amount of harvested energy at each time interval is assumed to be known a-priori. Using (2.8)-(2.14), the throughput maximization problem can be written as 1 P1: max p s,p r 2 subject to: N ( ) γ sr (i)p s (i)γ rd (i)p r (i) log 1+ γ sr (i)p s (i)+γ rd (i)p r (i)+1 k p s (i) 1 T k p r (i) 1 T k+1 E s (i) T k+1 E r (i) T k E s (i) for k = 1,...,N k E r (i) for k = 1,...,N k k (2.15a) (2.15b) (2.15c) p s (i) B max s for k = 1,...,N 1 (2.15d) p r (i) B max r for k = 1,...,N 1 (2.15e) p s (i) 0, p r (i) 0, for i = 1,...,N. (2.15f) The objective function in (2.15a) is not concave in p s and p r. To show this, consider the following example. For N = 10, let p 1 s = p1 r = [0.1,,0.1], and p2 s = p2 r = [0.2,,0.2], be two feasible points2. Then for γ sr (i) = γ rd (i) = 1 for i = 1,,N we have 1 2 ( R(p 1 (p s,p 1 1 r )+R(p2 s,p2 r )) = R( s,p 1 r )+(p2 s,p2 r ) ) = Despite the fact that P1 is not a convex optimization problem, different approaches can be used to solve this problem as will be described in following sections. 2.3 High SNR Approximation In high-snr conditions, SNR d in (2.7) can be approximated as SNR d γ sr(i)p s (i)γ rd (i)p r (i) γ sr (i)p s (i)+γ rd (i)p r (i). (2.16) 2 Obviously it is easy to see that for these two points there exits a set of E s s, E r s, Bs max, and Br max points are feasible. such that these

19 Chapter 2. Throughput Maximization in the Offline Setting 13 Using this approximation, the optimization problem is the relaxed to 1 P2 : max p s,p r 2 subject to: N ( log 1+ γ ) sr(i)p s (i)γ rd (i)p r (i) γ sr (i)p s (i)+γ rd (i)p r (i) k p s (i) 1 T k p r (i) 1 T k+1 E s (i) T k+1 E r (i) T k E s (i) for k = 1,...,N k E r (i) for k = 1,...,N k k (2.17a) (2.17b) (2.17c) p s (i) B max s for k = 1,...,N 1 (2.17d) p r (i) B max r for k = 1,...,N 1 (2.17e) p s (i) > 0, p r (i) > 0, for i = 1,...,N. Proposition 2.1. The objective function in (2.17) is a concave function of p s and p r. Proof. Consider the function g(x, y) as follows g(x,y) = xy x+y. (2.17f) The Hessian matrix of g is then given as For positive x and y, we then have 2 g(x,y) = [ 2y 2 (x+y) 3 2yx (x+y) 3 2yx (x+y) 3 2x2 (x+y) 3 ]. (2.18) tr ( 2 g(x,y) ) = 2 x2 +y 2 (x+y) 3 < 0 det ( 2 g(x,y) ) = 0. Therefore, this matrix has one negative and one zero eigenvalue, and is negative semi-definite, which in turn means that g(x,y) is a concave function of x and y. In addition, logarithm is a concave and increasing function, hence, log(1 + g(x, y)) is a concave function of x, y as well. The objective function of P2 can be written as the summation of a set of concave functions of p s and p r as N log(1+g(γ sr (i)p s (i),γ rd (i)p r (i)) (2.19) and therefore, it is also a concave function. The constraints in (2.17) are obviously affine. As a result, P2 is a convex optimization problem [29] that can be efficiently solved by using standard convex optimization solvers such as the CVX package[30], [31]. However, investigating the Karush-Kuhn-Tucker (KKT) conditions for this problem provides an interesting insight into the HSA solution.

20 Chapter 2. Throughput Maximization in the Offline Setting 14 To form the Lagrangian of the problem in (2.17), let λ k, δ k, η k and µ k denote the multipliers associated with the kth constraints of (2.17b), (2.17c), (2.17d) and (2.17e), respectively. The Lagrangian is then given by L(p s,p r,λ,δ,η,µ) = 1 N ( log 1+ γ ) sr(i)p s (i)γ rd (i)p r (i) 2 γ sr (i)p s (i)+γ rd (i)p r (i) N k ) k( p s (i) k=1λ 1 k E s (i) T N k ) k( p r (i) k=1δ 1 k E r (i) T ( N 1 k+1 ) k η k E s (i) T p s (i) Bs max k=1 N 1 ( k+1 µ k E r (i) T k=1 k p r (i) B max r ) (2.20) where λ [λ 1,...,λ N ] T, δ [δ 1,...,δ N ] T, η [η 1,...,η N 1,0] T, µ [µ 1,...,µ N 1,0] T. The corresponding slackness conditions are λ k ( k (p s (i) 1 T E s(i)) δ k ( k (p r (i) 1 T E r(i)) ( k+1 η k E s (i) T ( k+1 µ k E r (i) T k k ) ) = 0, for k = 1,...,N (2.21) = 0, for k = 1,...,N (2.22) p s (i) B max s p r (i) B max r ) ) = 0, for k = 1,...,N 1 (2.23) = 0, for k = 1,...,N 1. (2.24) By applying the KKT conditions, the following conditions are obtained for the optimal primal and dual variables L p s (i) = γ sr (i)γrd 2 (i)p2 r (i) γ sr (i)p s (i)+γ rd (i)p r (i) 1 L p r (i) = γ rd (i)γsr 2(i)p2 s (i) γ sr (i)p s (i)+γ rd (i)p r (i) 1 γ sr (i)p s (i)+γ rd (i)p r (i)+γ sr (i)γ rd (i)p s (i)p r (i) N k=i γ sr (i)p s (i)+γ rd (i)p r (i)+γ sr (i)γ rd (i)p s (i)p r (i) N k=i N 1 λ k + η k = 0. k=i (2.25) N 1 δ k + µ k = 0. k=i (2.26) After some calculations, we can write the optimal source and relay powers in terms of the optimal

21 Chapter 2. Throughput Maximization in the Offline Setting 15 values of the dual variables as 1 N λ k N η k k=i k=i γ sr(i) + N δ k N µ k k=i k=i γ rd (i) 2 p s (i) = ( N ) γ sr (i) λ k N η k k=i k=i 1 N λ k N η k k=i k=i γ sr(i) + N λ k N η k k=i k=i γ sr(i) + N δ k N µ k k=i k=i γ rd (i) N δ k N µ k k=i k=i γ rd (i) 2 (2.27) p r (i) = ( N ) γ rd (i) δ k N µ k k=i k=i N λ k N η k k=i k=i γ sr(i) + N δ k N µ k k=i k=i γ rd (i). (2.28) Based on these equations, we are able to prove the following theorem that states that as long as the batteries are not depleted or saturated, the power used at the source and at the relay is constant. Theorem 2.1. In the optimal power allocation scheme, comparing the allocations in the m-th and the (m+1)-th block, if the channel coefficients of the S-R and R-D link remain constant, the transmission power of the source and that of the relay remain constant unless the battery of the source or that of the relay is saturated or exhausted prior to the beginning of (m + 1)-th transmission block. Proof. To prove this theorem, we have to show that if none of the source or relay batteries are depleted or saturated before the (m+1)-th block, we have p s (m) = p s (m+1) p r (m) = p r (m+1). (2.29) If the battery of the source and that of the relay are not exhausted prior to the (m + 1)-th block then we have m (p s (i) 1 T E s(i)) 0 (2.30) m (p r (i) 1 T E r(i)) 0. (2.31) therefore, using the slackness conditions in (2.21) and (2.22), respectively, we can conclude that λ m = 0 and δ m = 0. This in turn implies that N N λ k = λ k (2.32) k=m N δ k = k=m+1 N k=m k=m+1 δ k. (2.33) On the other hand, if the batteries of the source and the relay are not saturated with the arrival of the

22 Chapter 2. Throughput Maximization in the Offline Setting 16 (m+1)-th energy packet (before the beginning of the (m+1)-th block), we have m+1 m+1 E s (i) T E r (i) T m m p s (i) B max s 0 (2.34) p r (i) B max r 0. (2.35) According to the slackness conditions in (2.23) and (2.24), respectively, we have η m = 0 and µ m = 0. Using this we can write N η k = k=m N µ k = k=m N k=m+1 N k=m+1 η k (2.36) µ k. (2.37) Using the expressions derived in (2.32), (2.33), (2.36) and (2.37), we can conclude that if the battery of the source and that of the relay are neither saturated nor exhausted prior to the beginning of (m+1)-th transmission block, the following holds 1 N λ k N η k k=m k=m γ sr(m) + N δ k N µ k k=m k=m γ rd (m) 2 p s (m) = = = ( N ) γ sr (m) λ k N η k k=m k=m ( γsr (m) 1 N k=m+1 ( γsr (m+1) λ k N 1 N k=m+1 N λ k N η k k=m k=m γ sr(m) + N λ k N η k k=m+1 k=m+1 γ sr(m) + k=m+1 ) η k N λ k N η k k=m+1 k=m+1 γ sr(m+1) + λ k N k=m+1 ) η k N δ k N µ k k=m k=m γ rd (m) N δ k N µ k k=m+1 k=m+1 γ rd (m) N λ k N η k k=m+1 k=m+1 γ sr(m) + N k=m+1 δ k N k=m+1 µ k γ rd (m+1) N λ k N η k k=m+1 k=m+1 γ sr(m+1) + 2 N δ k N µ k k=m+1 k=m+1 γ rd (m) 2 N k=m+1 δ k N k=m+1 µ k γ rd (m+1) (2.38) (2.39) (2.40) = p s (m+1) (2.41) where (2.40) is due to the assumption that γ sr (m) = γ sr (m+1) and γ rd (m) = γ rd (m+1). Using a similar argument, in this case, we have p r (m) = p r (m+1). This concludes the proof. Our second theorem states that the optimal power used is a non-decresing function of time index.

23 Chapter 2. Throughput Maximization in the Offline Setting 17 Theorem 2.2. In the special case of static channels (i.e., constant channel fading coefficients throughout the transmission period) and large enough battery capacity, the optimal power levels form a nondecreasing sequence. In other words p s (1) p s (2) p s (N) and p r (1) p r (2) p r (N). Proof. In the following proof, without loss of generality, we will assume that γ sr (i) = γ rd (i) = 1 for i = 1,,N. Using this assumption, the optimal power levelsgiven in (2.27)and (2.28)can be rewritten as ( ( N ( 1 λ k N N ) η k )+ ) δ k N 2 µ k k=i k=i k=i k=i p s (i) = ( N λ k N N ( k)( ( λ k k=i k=iη N N ) η k )+ ) (2.42) δ k N µ k k=i k=i k=i k=i ( ( N ( 1 λ k N N ) η k )+ ) δ k N 2 µ k k=i k=i k=i k=i p r (i) = ( N ) ( ( δ k N N µ k λ k N η k )+ k=i k=i k=i k=i ( N ) δ k N µ k k=i k=i ). (2.43) First, note that if the battery capacity is large enough, we can assume that the inequalities (2.17d), and (2.17e) will never be met with equalities. Therefore, their associated Lagrangian multipliers, η k, µ k are equal to zero, for k = 1,,N. The resulting optimal power levels are then given by p s (i) = p r (i) = ( ) 2 N N 1 λ k + δ k k=i k=i ( ) (2.44) N N N λ k λ k + δ k k=i k=i k=i ( ) 2 N N 1 λ k + δ k k=i k=i ( ). (2.45) N N N δ k λ k + δ k k=i k=i k=i N In addition, since the Lagrangian multipliers λ k and δ k are non-negative, the summations λ k and N δ k are non-increasing functions of i. This is due to the fact that as i increases the summation is k=i calculated over less number of non-negative numbers. Using this fact, it is seen that the denominators of right hand sides of (2.44) and (2.45) are non-increasing in i while the numerators are non-decreasing in i, therefore, p s (i) and p r (i) are non-decreasing over i. Theorem 2.2 is thus proven. The properties proven in Theorem 2.1 and Theorem 2.2 can be used to design suboptimal and computationally simple power allocations schemes in the online setting. Investigations in the design of such power allocation methods using the properties shown above is left as an extension to the work presented in this thesis. k=i

24 Chapter 2. Throughput Maximization in the Offline Setting Biconcavity and Alternate Convex Search In this section, we introduce an alternative approach to solve the optimization problem in (2.15). This solution is based on the biconcavity of the objective function in (2.15), in terms of p s and p r. The definition of a biconcave function is given in [32] as below Definition 2.1. A function f : X Y R is said to be biconcave if f x ( ) f(x, ) : Y R (2.46) is a concave function on Y, for every fixed x X, and f y ( ) f(,y) : X R (2.47) is a concave function on X, for every fixed y Y. Essentially, f(x,y) is biconcave if it is concave in each variable while keeping the other constant. Given a fixed p r, the objective function of P1 is a concave function in p s, and vice versa, and thus it is a biconcave function of p s and p r on the feasible set. This biconcavity can be used to solve the original problem in an iterative algorithm known as the Alternate Convex Search (ACS) [32]. In this algorithm, given initial feasible power levels, the optimization is carried out over p s and p r separately and in an iterative manner. The steps of this algorithm are presented in Algorithm 1. Note that, the sequence of throughputs, R(p (k) s,p (k) r ), generated by iterations of the ACS algorithm is a non-decreasing sequence. To see this, consider the (k + 1)-th iteration of the ACS algorithm as follows R(p (k) s,p (k) r ) max R(p s,p k p s r) = R(p (k+1) s,p (k) r ) max R(p (k+1) s p r,p r ) = R(p (k+1) s,p (k+1) r ). (2.50) Furthermore, the objective function of the optimization problem is bounded from above in the feasible set due to the finiteness of the total available energy. Hence, using Theorem 4.5 of [32], the ACS algorithm will converge after a finite number of iterations. However, since the original problem is not a convex optimization problem, convergence to the globally optimal solution cannot be claimed. Also note that the optimization problems in (2.48) and (2.49) are convex optimization problems with linear inequalities and can be easily solved using standard solvers such as [30]. 2.5 Numerical Results This section presents our simulation results demonstrating the performance of our proposed solution to the throughput maximization problem in the offline setting. We have performed the simulations for two different energy harvesting settings: uniform energy arrivals and exponential energy arrivals. In the uniform energy arrivals case, it is assumed that E s (i) and E r (i) can independently take values from the ternary set {0,H T,2H T} with equal probability where H is the average energy harvesting rate or the average recharge rate. For the exponential energy arrivals case, the energy packets are generated according to an exponential distribution with an average of H T. Throughout this simulations, we have set T = 1.

25 Chapter 2. Throughput Maximization in the Offline Setting 19 Algorithm 1 The Alternating Convex Search Algorithm 1. Choose an arbitrarily small stopping threshold ǫ. 2. Set k = 0 and choose a feasible p (k) s and p (k) r, e.g., p (0) s (i) = E s (i)/t and p (0) i = 1,,N. 3. Starting from a fixed p (k) r, solve the following convex optimization problem for p (k+1) s : ( ) p (k+1) s 1 = argmax p s 2 subject to: N γ sr (i)p s (i)γ rd (i)p (k) r (i) log 1+ γ sr (i)p s (i)+γ rd (i)p (k) r (i)+1 k p s (i) 1 k E s (i), for k = 1,...,N T k+1 E s (i) T k 4. Solve the following optimization problem for p (k+1) 4. If R(p (k+1) s p (k+1) r 1 = argmax p r 2 subject to: p s (i) B max s, for k = 1,...,N 1, r (i) = E r (i)/t for p s (i) 0, i = 1,...,N. (2.48) ( N log 1+ k p r (i) 1 T k+1 E r (i) T r : γ sr (i)p (k+1) s (i)γ rd (i)p r (i) γ sr (i)p (k+1) s k k (i)+γ rd (i)p r (i)+1 E r (i), for k = 1,...,N ) p r (i) Br max, for k = 1,...,N 1 p r (i) 0, i = 1,...,N. (2.49),p (k+1) r ) R(p (k) s,p (k) r ) ǫ stop, otherwise set k = k +1 and go to Step 3. Figure 2.2 plots the throughput curves of the system for the ACS method, the high-snr approximation (HSA) based scheme introduced in [24] and a greedy method. In the greedy scheme, the source and the relay use all the available energy in each transmission block, more specifically, p s (i) = B s (i)/t and p r (i) = B r (i)/t, for i = 1,,N. The source and relay battery capacities are set to 10 units of energy and the average recharge rate H is 0.5. This example uses a static channel, i.e., γ sr (i) = γ rd (i) = 1 for i = 1,,N. Clearly, the proposed alternating convex search approach outperforms the other two methods for both uniform and exponential energy arrival cases. This is due to the fact that the setting used in this simulation (i.e. H = 0.5 unit of energy / second) can be considered as a low SNR regime, therefore the HSA based scheme suffers from an approximation error. Figure 2.3 presents a plot similar to Figure 2.2 for a Rayleigh block fading channel where the channel amplitudes are exponentially distributed with average values of γ sr = γ rd = 1 and are assumed to be statistically independent from one transmission interval to another. As can be seen from this figure, compared to the case with a static channel, the advantage of using ACS method over the HSA-based scheme is smaller. Figure 2.4 depicts the effect of the battery capacity of the source and that of the relayin the resulting average total throughput. The results in this figure are obtained using the ACS algorithm, for N = 100 and H = 1. As is seen in this figure, increasing the battery capacity from 5 to 20 will result in a