Opportunistic Spectrum Access for Energy-Constrained Cognitive Radios

Transcription

1 1206 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 8, NO. 3, MARCH 2009 Opportunistic Spectrum Access for Energy-Constrained Cognitive Radios Anh Tuan Hoang, Ying-Chang Liang, David Tung Chong Wong, Yonghong Zeng, and Rui Zhang Abstract This paper considers a scenario in which a secondary user (SU) opportunistically accesses a channel allocated to some primary network (PN) that switches between idle and active states in a time-slotted manner. At the beginning of each time slot, SU can choose to stay idle or to carry out spectrum sensing to detect the state of PN. If PN is detected to be idle, SU can carry out data transmission. Spectrum sensing consumes time and energy and introduces false alarms and mis-detections. The objective is to dynamically decide, for each time slot, whether SU should stay idle or carry out sensing, and if so, for how long, to maximize the expected reward. We formulate this as a partially observable Markov decision process and prove important properties of the optimal control policies. Heuristic control policies with low complexity and good performance are also proposed. Numerical results show the significant performance gain of our dynamic control approach for opportunistic spectrum access. Index Terms Cognitive radio, spectrum sensing, false alarms, mis-detections, energy constraint, POMDP. I. INTRODUCTION THE traditional approach of fixed radio spectrum allocation leads to under-utilization. It has been reported in recent studies by the US FCC that there are vast temporal and spatial variations in the usage of allocated spectrum [1]. This motivates the concepts of opportunistic spectrum access (OSA), which allows secondary cognitive radio (CR) systems to opportunistically exploit the under-utilized spectrum. We consider a secondary user (SU) making opportunistic use of a channel allocated to some primary network (PN). PN operates in a time-slotted manner and switches between idle and active states according to a stationary Markovian process. At the beginning of each time slot, SU needs to decide whether to stay idle or to carry out spectrum sensing to detect the state of PN. If SU chooses to carry out sensing, it needs to decide the duration of the sensing period and to meet a minimum detection probability. Subsequently, if spectrum sensing indicates that PN is idle, SU transmits data during the rest of the slot. There are important trade-offs when SU makes the above control decisions. By staying idle in a particular time slot, the user conserves energy, but suffers increase in delay and reduction in throughput. By carrying out spectrum sensing, SU consumes time and energy to acquire knowledge of the state of the primary network, and stands a chance to transmit data if PN is idle. Furthermore, when the required probability Manuscript received June ; revised as a transactions letter October ; accepted October The associate editor coordinating the review of this paper and approving it for publication was T. Hou. The authors are with the Institute for Infocomm Research, A STAR, 1 Fusionopolis Way, #21-01 Connexis, Singapore ( athoang, ycliang, wongtc, yhzeng, rzhang}@i2r.a-star.edu.sg). Digital Object Identifier /TWC /09$25.00 c 2009 IEEE of detection is fixed, increasing the sensing time reduces the probability of false alarm and therefore increases the probability of transmission for SU. However, increasing the sensing time also reduces the time available for transmission. For SU, given the delay cost associated with staying idle in a time slot, the energy costs associated with spectrum sensing and data transmission, and the throughput gain associated with a successful transmission, we consider the problem of finding an optimal policy which decides the idle and sensing modes, together with spectrum sensing time, to maximize the expected net reward. Here, the reward is defined as a function of delay and energy costs and throughput gain. We analyze this problem using the framework of partially observable Markov decision processes (POMDP) and prove important properties of the optimal policies. Heuristic control policies with low complexity and good performance are also proposed. Recent works on controlling spectrum-sensing in OSA systems can roughly be classified into two groups, i.e., those that focus on the control within each time slot, when the status of a primary network is more or less static [2] [4] and those that focus on the time dynamics of the control problem [5], [6]. In [2] [4], the control objective is to trade off between sensing accuracy and time available for communications, i.e., the longer the sensing duration, the lower the false alarm probability and the longer the sensing duration, the less time is available for communications. However, the dynamics of primary networks is not taken into account in [2] [4]. In [5], Zhao et al. consider a spectrum access scenario similar to ours and prove the important separation principle which decouples the sensing policy from spectrum access policy. However, in [5], energy consumption and sensing duration are not part of the control problem. The work in [6] does take into account the energy consumption but assume that spectrum sensing is perfect. This paper bridges the gap between the problems considered in [2] [4] and in [5], [6]. The rest of this paper is organized as follows. In Section II, we describe the system model and the control problem. Important properties of the optimal control policies are discussed in Section III. In Section IV, heuristic policies are proposed. Numerical results are presented in Section V. Finally, we conclude the paper in Section VI. II. SYSTEM MODEL A. Primary Network We assume that PN is either active or idle in each time slot of duration T and, from one time slot to the next, randomly switches states according to a stationary Markovian process. The process is specified by two parameters b and g, whereb is the probability that PN becomes active in the next time slot, given that it is idle in the current slot and g is the probability

2 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 8, NO. 3, MARCH Slots PU idle idle active idle active SU s trans. ack s quiet s quiet idle s trans. ack time out Fig. 1. Operations of a primary network and secondary user. PN switches between active and idle according to a Markovian process. SU must carry out spectrum sensing before transmitting in each time slot. Sensing can introduces false alarms (e.g. in slot 2) and mis-detections (e.g. in slot 5). that PN becomes idle in the next time slot, given that it is active in the now. The stationary probabilities of being idle and active for PN are π i = g/(b + g) and π a = b/(b + g), respectively. We further assume that μ =1 b g>0, i.e., the switching process contains positive memory [7]. B. Opportunistic Spectrum Access SU opportunistically accesses the channel by first synchronizing with the slot structure of PN and then carrying out the following mechanism (illustrated in Fig. 1). 1) Spectrum Sensing: If SU wishes to transmit in a particular slot, it first spends a duration τ to carry out spectrum sensing, i.e., a binary hypothesis test: H 0 : the primary network is idle, (1) versus H 1 : the primary network is active. Let θ denote the outcome of the above binary hypothesis test, where θ =0means H 0 is detected and θ =1otherwise. Associated with the spectrum sensing activity are probability of false alarm, i.e., mistaking H 0 for H 1, and probability of mis-detection, i.e., mistaking H 1 for H 0.ItisassumedthatSU must meet a fixed probability of detection P d. Then, the probability of false alarm is a function of the sensing time τ and is denoted by P fa (τ). The sensing duration τ must be within the interval [τ min,τ max ],where0 <τ min τ max <T.Itis assumed that for the given range of τ, 0 <P fa (τ) <P d < 1 and P fa (τ) is continuous, differentialbe, and decreasing in τ. 2) Data transmission: If the sensing outcome is θ =0,SU proceeds to transmit data in the rest of the time slot. Otherwise, it must stay quiet and wait until the next time slot to try again. 3) Acknowledgment: Even though θ = 0, this can be due to a mis-detection. Mis-detections result in collisions between primary and secondary transmissions. We assume that if SU transmission is carried out when the primary network is actually idle, a positive acknowledgment (ACK) is returned, otherwise, collisions can be detected based on ACK time out. C. POMDP Formulation Our control problem can be classified as a discrete-time POMDP with the following components. 1) Belief State: It is well known ( [8]) that for each POMDP, all information that is useful for making decisions can be encapsulated in the posterior distribution vector of the system states. In our control problem, at the beginning of each time slot, based on previous actions and observations, SU can calculate the probability that the primary network is idle in the time slot. We denote this probability by p and name it the belief state. After each time slot, depending on the action taken by SU and the corresponding outcome, p can be updated according to one of the following four cases. Case 1: SU stays idle and does not carry out spectrum sensing. Then, the next belief state, i.e., the probability that PN is idle in the next time slot is: L 1 (p) =p(1 b)+(1 p)g = p(1 b g)+g. (2) Case 2: SU senses the channel for the duration τ, obtains θ =1, i.e., PN is active, and therefore keeps quiet in the rest of the time slot. Using Bayes rule, the belief state in the next time slot is: L 2 (p, τ) = pp fa(τ)(1 b)+(1 p)p d g. (3) pp fa (τ)+(1 p)p d Case 3: SU senses the channel for the duration τ, obtains θ = 0, carries out transmission, and subsequently receives an ACK at the end of the slot. The ACK implies that PN is actually idle during the current time slot and therefore, the belief state in the next time slot is L 3 =1 b. Case 4: All the same as Case 3, except that an ACK is not received at the end of the slot. This implies that PN is active in the current time slot but is mis-detected by SU. Therefore, the belief state in the next time slot is L 4 = g. As can be noted, L 3 and L 4 do not depend on p and τ. However, to simplify the notation, we use L 1 (p, τ), L 3 (p, τ), L 4 (p, τ) interchangeable for L 1 (p), L 3, L 4. Also, letting Q i (p, τ), i =2, 3, 4 denote the probabilities that Case i above happens, we have: Q 2 (p, τ) =pp fa (τ)+(1 p)p d, Q 3 (p, τ) =p(1 P fa (τ)) Q 4 (p, τ) =(1 p)(1 P d ). 2) Properties of L i (p, τ): From (2) and the assumption that the state-transition matrix M has positive memory, i.e., 1 b g>0, it follows that L 1 (p) is increasing in p. From (3), L 2 (p, τ) p (4) = (1 b g)p dp fa (τ) [P d p(p d P fa (τ))] 2. (5) Furthermore, as 1 b g>0 and P d >P fa (τ) > 0, L2(p,τ ) p is positive and increasing in p. Therefore, L 2 (p, τ) is convex and increasing in p. It can also be easily verified that L 4 (p) L 2 (p, τ) L 3 (p) for all τ.

3 1208 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 8, NO. 3, MARCH ) Costs, Reward, and Control Objective: We assume that the energy cost of carrying out spectrum sensing for τ units of time is a continuous, non-negative and increasing function in τ and is denoted by c s (τ). Letr t and c t respectively be the gain in throughput and the energy cost, both are measured per unit of transmission time. It is reasonable to assume that r t >c t 0, otherwise, there is no justification for SU to carry out transmission. Note that SU can also choose to stay idle during a time slot to conserve energy. However, doing so negatively affect throughput and latency. We assume that, for SU, the cost of staying idle during each time slot is c i, c i 0. We assume that the slot duration T is much longer than ACK time/time-out so that this effect can be ignored. In a particular time slot, given that the probability of PN being idle is p and SU carries out sensing for τ units of time, the expected net gain for SU can be calculated as: G(p, τ) =p(1 P fa (τ))(t τ)(r t c t ) (6) (1 p)(1 P d )(T τ)c t c s (τ). It can be verified that G(p, τ) is continuous and increasing in p and G(p, τ) is also continuous in τ. To simplify the notation, we use τ =0to represent that SU chooses to stay idle. We then have the following expected reward when the sensing decision is set to τ. ci τ =0 R(p, τ) = (7) G(p, τ), τ min τ τ max. We are interested in the following problem. Definition 1: Let p n denote the probability that the primary network is idle during time slot n, select the sensing time τ n, τ n 0, [τ min,τ max ]}, to maximize the following discounted reward function E N 1 n=0 α n R(p n,τ n ) } p0 = p, (8) where 0 <α<1 is a discounting factor, and 1 N is the control horizon. In general, discounted reward is reasonable when SU values the reward obtained now more than what it can get in the future. Also, there is a close relationship between averagereward and discounted-reward problems, i.e., when α is closed enough to 1, then an optimal solution to the α-discounted problem is also optimal for an average-reward problem [8]. For the control problem in Definition 1, we can set p 0 = π i, where π i is definedinsectionii.a. III. STRUCTURE OF OPTIMAL POLICIES A. Monotonicity and Convexity of Value Functions When N<, letv N (p) denote the maximum achievable discounted reward function in Definition 1, V N (p) satisfies the following Bellman equation ( [8]): V N (p) = max c i + αv N 1 (L 1 (p)), G(p, τ)+α } (9) Q i (p, τ)v N 1 (L i (p, τ)), N > 1, where V 1 (p) = max ci, G(p, τ) }. (10) When N =, letv (p) denote the maximum achievable discounted reward function in Definition 1, V (p) satisfies the following Bellman equation ( [8]): V (p) = max c i + αv (L 1 (p)), G(p, τ)+α } Q i (p, τ)v (L i (p, τ)). (11) It can be shown ( [8]) that V N (p) and V (p) are continuous in p and lim n V n (p) =V (p). Proposition 1: V N (p) and V (p) are nondecreasing and convex in p. Due to limited space, we omit the proof. We note that the convexity and nondecreasing property of the value functions in the belief state is quite well-known in the POMDP literature [9], [10]. Remark 1: Proposition 1 states intuitively that, the higher the probability p that PN is idle at the beginning of the control process, the higher the maximum achievable expected reward V N (p) and V (p). Furthermore,asV N (p) and V (p) are convex, V N (p) and V (p) increase at least linearly in p. B. Properties of Optimal Policies Let us explore some useful structural properties of the optimal control policies. Letting G (p) = max G(p, τ), (12) as G(p, τ) is increasing in p, so is G (p). We state the following property of the optimal control policies. Proposition 2: Let p be the minimum value of p such that G (p ) > c i. If, in a particular time slot, the probability of PN being idle is p such that p p, then an optimal policy must carry out spectrum sensing in that time slot. Proof: Let us prove for the case when N < }. We have V 1 (p) = max τ [τmin,τ max] c i, G(p, τ) = max c i,g (p)}, whereg (p) G (p ) > c i, therefore, spectrum sensing should be carried out when N =1. For N > 1, equation (9) applied. Next, notice that Q i(p, τ) = 1 and Q i(p, τ)l i (p, τ) = L 1 (p). Then, as V N 1 (p) is convex, V N 1 (L 1 (p)) Q i (p, τ)v N 1 (L i (p, τ)), τ. (13) From (9) and (13), and the fact that G (p) G (p ) > c i, it is clear that the system should carry out spectrum sensing in the current time slot. The proof for N = is similar. Remark 2: Proposition 2 gives a sufficient condition on the value of p, i.e., p p, for carrying out spectrum sensing. However, this may not be a necessary condition. In general, the optimal policies for POMDP may not possess the thresholdbased structure in the belief state (see exceptions in [9], [10]).

4 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 8, NO. 3, MARCH A natural question is, when sensing is carried out, how the optimal time τ would vary with p. Let F N (p, τ) =α Q i (p, τ)v N 1 (L i (p, τ)) (14) be the expected discounted future reward given the belief state in a particular time slot is p and sensing is carried out for a duration τ. Similarly, for infinite control horizon, define F (p, τ) =α Q i (p, τ)v (L i (p, τ)), τ min τ τ max. (15) The following statement highlights the effect of increasing the spectrum sensing time τ. Proposition 3: F N (p, τ) and F (p, τ) are nondecreasing in τ. Due to limited space, we omit the proof. Essentially, Proposition 3 makes concrete the intuition that the more sensing being carried out in the current time slot, the better the expected reward in the future time slots. This is because increasing the sensing time gives the secondary user more accurate knowledge of the state of the primary network, which in turn improves future control. To further study the effect of varying the sensing time, we need to make the following assumptions. A1: The probability of false alarm, i.e., P fa (τ), isconvex and decreasing in τ, τ min τ τ max. A2: The energy cost of sensing, i.e., c s (τ), is convex and increasing in τ. For assumption A1, the justification can be found in [2] while for A2, we give an example of energy detection. Energy detection comprises of three main steps, i.e., i) to take K samples of the signal; ii) to calculate the average power of the K samples; and iii) to compare the average power to a certain threshold. The complexity of the first and second steps is linear in K. Furthermore,K is linear in τ, soc s (τ) can be assumed linear in τ when energy detection is employed. It can be easily verified that, when P fa (τ) is convex and decreasing while c s (τ) is convex and increasing in τ, G(p, τ) is concave in τ. Now, as G(p, τ) is concave, continuous, and has a strictly decreasing first order derivative for all τ in [τ min,τ max ], there exists an unique maximum point for G(p, τ). Letting τ (p) = arg max G(p, τ), (16) τ min τ τ max the following proposition relates the optimal sensing time to τ (p). Proposition 4: Given assumptions A1 and A2, if at the beginning of a particular time slot, the probability of the primary network being idle is p and sensing is carried out, then the optimal sensing time τ opt is greater than or equal to τ (p). Proof: We prove for N =, the case when N < is similar. The proof is by contradiction. Suppose the optimal sensing time τ opt < τ (p). Due to the concavity of G(p, τ opt ) <G(p, τ (p)). Furthermore, from Proposition 3, we have F (p, τ opt ) F (p, τ (p)), therefore G(p, τ opt )+F (p, τ opt ) <G(p, τ (p)) + F (p, τ (p)), (17) which contradicts to the fact that τ opt is the optimal sensing time given p. This completes the proof. Remark 3: Proposition 4 states that when the probability of the primary network being idle is p and sensing is carried out, then the lower bound of the optimal sensing time is τ (p). It can also be further proved that the lower bound τ (p) is always monotonic in p. IV. HEURISTIC POLICIES Directly solving the POMDP described in Section III can be computational challenging. In this section, suboptimal control policies that can be obtained at lower complexity are discussed. A. Grid-based Approximation Grid-based approximation is a widely-used approach for approximating solutions to POMDPs. In this approach, the value function is approximated at a finite number of belief points on a grid, using standard value iteration algorithm [8]. The value function at belief points not belonging to the grid is subsequently evaluated using interpolation. In this paper, we employ the fixed-resolution, regular-grid approach proposed by Lovejoy [11]. B. Myopic Policy ζ m (.) Proposition 2 identifies the sufficient condition on the probability that PN is idle for sensing to be carried out. Furthermore, Proposition 4 gives the lower bound on the optimal sensing time. Based on these, we consider the myopic policy that set the sensing time in slot n as τ n = ζ m (p n ) where ζ m (p) =τ (p) if p p and ζ m (p) =0otherwise. Setting the sensing time τ = ζ m (p) myopically maximizes the instantaneous gain when the belief state is p. Note that while the grid-based policy mentioned above requires the use of value iteration, this myopic policy can be directly calculated. C. Static Policy ζ s (.) Consider a static spectrum access policy that always carries out sensing with a fixed sensing duration. Note that the stationary probability of the primary network being idle is π i = b b+g. We can calculate τ (π i ) = arg max G (π i,τ). τ min τ τ max Then, for every time slot, sensing is carried out for the duration of τ (π i ). We term this policy ζ s (.). D. Genie s Policy We are going to normalize the performance of different control policies to that of the following so called Genie s policy. Suppose that at the beginning of each time slot, a genie tells the secondary user exactly what the state of the primary network is. Then, the best action for the secondary user is to carry out transmission if the primary network is idle, and to stay idle if the primary network is active.

5 1210 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 8, NO. 3, MARCH μ=0.5 Normalized V(p) p (prob. of primary network being idle) Fig. 2. Normalized optimal reward function. Optimal reward function V (p) is convex and increasing in p. P d =10%, c i =0, s =1, c t =2, r t =3, and µ =0.9. The optimal reward function is normalized by the reward of Genie s Policy. V. NUMERICAL RESULTS AND DISCUSSION In this section, we present numerical results that illustrate our theoretical analysis. We focus on the infinite control horizon scenario, i.e., when N =. A. Model for Numerical Studies We assume that PN operates on a channel with bandwidth of B =6MHz. Time slot duration is T =10msecs. SU employs energy detection for spectrum sensing, with channel being oversampled at rate f s = 7/8B. Given the required probability of detection P d = 10%, the sensing SNR of γ = 15dB, and the sensing duraration τ, the probability of false alarm can be calculated by ( [2]): P fa (τ) =Q( 2γ +1Q 1 (P d )+ τf s γ), (18) where Q(.) is the complementary distribution function of a standard Gaussian variable. We assume that the sensing cost is linear in sensing time, i.e., c s (τ) =sτ, s>0. The memory of PN switching process, i.e., μ =1 b g, is varied from 0.1 to While doing so, we always set b = g =(1 μ)/2. B. Characteristics of Optimal Policies To obtain optimal solutions to our POMDP, we use the solver package provided by A. Cassandra [12]. 1) Optimal reward function is convex and nondecreasing: In Fig. 2, we plot the optimal reward function V (p) versus the initial belief state p. As stated in Proposition 1, V (p) is convex and nondecreasing in p. As can be seen, for high value of p, V (p) is close to a linear function. This can be explained by the fact that the optimal control action does not vary much for high value of p (refer to Figs. 3 and 4). 2) Optimal sensing time τ opt (p): In Figs. 3 and 4, we plot the sensing time versus the probability of PN being idle for different control policies, i.e., optimal, grid-based approximation, myopic, and static. It can be observed that there is a threshold value for the belief state p above which it τ/t Optimal Grid 0.02 ζ m (.) ζ s (.) p (prob. of primary network being idle) Fig. 3. Sensing times for different policies when the memory of primary user switching process is set to µ =0.5. P d =10%, c i =0, s =1, c t =2, r t =3. τ/t μ=0.9 Optimal Grid 0.02 ζ m (.) ζ s (.) p (prob. of primary network being idle) Fig. 4. Sensing times for different policies when the memory of primary user switching process is set to µ =0.9, P d =10%, c i = 1000, s =1, c t =2, r t =3. is optimal for the secondary user to carry out spectrum sensing and below which it is optimal to stay idle. In other words, the optimal sensing strategies tend to exhibit the threshold-based characteristic. From Fig. 3, it can also be observed that for a relatively low value of memory of PN switching process (μ =0.5), both the grid-based and myopic policies approximate the optimal sensing durations well. However, in Fig. 4, when the memory increases (μ = 0.9), while the grid-based policy is close to optimal, myopic policy is significantly different from the optimal policy for low value of p. This difference can be explained by the fact that the myopic policy only focuses on instantaneous gain and ignores future effects. As a result, it is not able to exploit the memory in PN states. Comparing Fig. 3 and 4, it is evident that when the memory of of the PN switching process increases, the optimal sensing time also increases. This is because the higher the memory,

6 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL. 8, NO. 3, MARCH Normalized reward Optimal Grid (20 points) Grid (10 points) ζ m (.) ζ s (.) 10 point grid approximation 20 point grid approximation μ (memory of primary network switching process) Fig. 5. Normalized rewards achieved by different control policies. The rewards achieved by different policies are normalized by that of the Genie s Policy. c i =1000, c s =1, c t =2, r t =3. the slower the change in status of the primary network and therefore, the more useful the sensing activity for predicting the future state of the primary network. Also, in Fig. 4, when the cost of staying idle is increased to c i = 1000, sensingis carried out given smaller value of probability that PN is idle. C. Performance Comparison In Fig. 5, we plot the expected reward versus the memory μ of PN switching process for optimal, grid-based, myopic, and static policies, normalized to that of Genie s policy. As can be seen, the performance of the optimal, grid-based and myopic polices all improves with increase in memory μ. This is because the higher the memory, the more slowly PN switches state, which implies the more useful sensing activities for future control. On the other hand, the performance of the static policy (ζ s (.)) does not change with μ. This is because when μ is varied, we always keeps the stationary probability of PN being active or idle unchanged (π i =0.5) sothatin every time slot, ζ s (.) always carries out sensing for a fixed duration. It is also important to note that, at relatively high value of μ, both grid-based and myopic policies, which employ our approach of dynamically controlling the sensing/channelaccess operation of SU, achieve significant performance gain compared to the static policy. It can be observed that for the low range of memory, the performance of both grid-based and myopic policies are close to that of the optimal policy. On the other hand, for higher value of memory, i.e., when PN switches state more slowly, only grid-based policy can approximate the performance of the optimal policy. The explanation for this is the same as for the difference in the sensing time in Figs. 3 and 4. In particular, for low memory, both myopic and grid-based policies can approximate the optimal decisions well. However, for high value of memory, only grid-based policy has decisions close to optimal. It is also evident that the performance of a 10- point grid-based approximation is very close to optimal, i.e., grid-based approximation can be employed without loss of performance in our POMDP problem. VI. CONCLUSION In this paper, we study spectrum-sensing policies that take into account the dynamics of the primary networks and determine the spectrum sensing duration in order to optimize secondary users performance. As such, this paper bridges the gap between the two groups of existing spectrum-sensing/control literature that either focus on adapting to the dynamics of the primary networks or on optimizing the spectrum-sensing duration, but not on both. We present theoretical analysis and numerical results to highlight the effectiveness of our control approach. REFERENCES [1] FCC, Spectrum policy task force report, FCC , Nov [2] Y. C. Liang, Y. H. Zeng, E. Peh, and A. T. Hoang, Sensing-throughput tradeoff for cognitive radio networks, IEEE Trans. Wireless Commun., vol. 7, no. 4, pp , Apr [3] A. Ghasemi and E. Sousa, Optimization of spectrum sensing for opportunistic spectrum access in cognitive radio networks, in Proc. 4th IEEE Consumer Communications and Networking Conference (CCNC), Las Vegas, USA, Jan [4] Y. Pei, A. T. Hoang, and Y.-C. Liang, Sensing-throughput tradeoff in cognitive radio networks: How frequently should spectrum sensing be carried out? in Proc. 18th IEEE PIMRC, Athens, Greece, Sept [5] Q. Zhao, L. Tong, A. Swami, and Y. Chen, Decentralized cognitive MAC for opportunistic spectrum access in ad hoc networks: a pomdp framework, IEEE J. Select. Areas Commun., vol. 25, no. 3, pp , Apr [6] A. T. Hoang and Y.-C. Liang, Adaptive scheduling of spectrum sensing periods in cognitive radio networks, in Proc. 50th IEEE Global Telecommunications Conference (Globecom), Washington DC, USA, Nov [7] M. Mushkin and I. Bar-David, Capacity and coding for the Gilbert- Elliott channels, IEEE Trans. Inform. Theory, vol. 35, no. 6, pp , Nov [8] D. P. Bertsekas, Dynamic Programming and Optimal Control, 2nd ed., vols. 1 and 2. Athena Scientific, [9] S. C. Albright, Structural results for partially observable markov decision processes, Operations Research, vol. 27, no. 5, pp , Sept./Oct [10] W. S. Lovejoy, Some monotonicity results for partially observable markov decision processes, Operations Research, vol. 35, no. 5, pp , Sept./Oct [11] W. Lovejoy, Computationally feasible bounds for partially observed markov decision processes, Operations Research, vol. 39, no. 1, pp , Jan./Feb [12] A. R. Cassandra, Tony s pomdp page, website edu/research/ai/pomdp/.