Delay Performance of Threshold Policies for Dynamic Spectrum Access

IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO. 7, JULY 83 Delay Performance of Threshold Policies for Dynamic Sectrum Access Rong-Rong Chen and Xin Liu Abstract In this aer, we analyze the delay erformance of a secondary user SU under dynamic sectrum access. We design simle time-threshold olicies for the SU to minimize the average delay while satisfying the collision robability constraint of the rimary user PU. Such olicies erform closely to an otimized olicy found by a Markov Decision Process MDP formulation, while facilitating analytical analysis of the delay and collision robability. For general PU busy and idle eriod distributions, we analyze the erformance of threshold olicies through a onedimensional Markov chain, and develo analytical exressions to aroximate the delay and collision robability. The accuracy of the Markov chain analysis and the analytical aroximations is examined under various busy and idle distributions. We investigate the imact of busy and idle distributions on system erformance. We find that while the idle distribution determines the time caacity of SU access, the busy distribution significantly affects the delay erformance of the threshold olicies. The effect of imerfect sensing is also studied. Index Terms Cognitive radio, dynamic sectrum access, delay, collision robability, Markov Decision Process. I. INTRODUCTION COGNITIVE Radio CR technology has great otential to alleviate sectrum scarcity in wireless communication networks. It allows secondary users SUs to oortunistically access sectrum licensed by rimary users PUs while rotecting the PUs activity. This new aradigm is tyically referred to as dynamic sectrum access DSA []. Most existing work focuses on either sectrum sensing or dynamic sectrum allocation. The delay erformance of cognitive radio networks, however, remains an under-exlored area, desite its imortance in characterizing the quality of service rovided by such networks. There are few aers on the delay analysis of SUs in cogntive radio networks [] [5], artially due to the technical difficulties of such an analysis. Hence, the delay erformance of cognitive radio networks in general is not wellunderstood. The goal of this work is to rovide an analytical study of the delay erformance of cognitive radio networks based on a class of threshold olicies. A distinguishing feature of this work is that we analyze the delay erformance of the SU while exlicitly taking into account the interference to the Manuscrit received July 8, ; revised October 3, and Aril 3, ; acceted Aril 5,. The associate editor coordinating the review of this aer and aroving it for ublication was S. Liew. R.-R. Chen is with the Deartment of Electrical and Comuter Engineering, University of Utah e-mail: rchen@ece.utah.edu. The work of R.-R. Chen was suorted in art by NSF under grant ECS-547433. X. Liu is with the Comuter Science Deartment, University of California, Davis e-mail: liu@cs.ucdavis.edu. The work of X. Liu was artially suorted by NSF through awards CNS-975 and CNS-44863, and Fujitsu Inc. through a gift grant. Digital Object Identifier.9/TWC..55.9 PU. Secifically, we adot an imortant constraint on the SU transmission such that the collision robability of a PU acket is less than a threshold secified arioriby the PU. While this constraint is commonly adoted for the rotection of PU in the design of DSA [6] [8], to the best of our knowledge, it has not been considered in rior work on the delay analysis of DSA [] [5]. Furthermore, we consider the DSA under general PU busy and idle distributions, which go beyond the simle exonential distribution considered in other existing work. We study the time-threshold olicies for the following reasons: The threshold olicies erform closely to the otimal olicy found by the Markov Decision Process MDP formulation, but are simler in structure, and rovide better insights and being comutationally more efficient than the MDP olicy. The threshold olicies facilitate analytical characterizations of the delay and collision robability under general PU busy and idle distributions. The threshold olicies were first studied in [9] to maximize the time caacity of the SU access for the case of backlogged traffic. In comarison, the threshold olicies develoed here are for minimizing delay under dynamic SU acket arrivals. Hence, the delay analysis develoed in this aer involves new techniques that are significantly different from and more challenging than those of [9]. The main contributions of this work are summarized as follows. We establish a class of time-threshold olicies as near otimal oortunistic transmission olicies for minimizing the delay of the SU under an exlicit PU collision robability constraint. Such olicies achieve close to otimal erformance comared with an otimal olicy we develoed based on an MDP formulation. For general PU busy and idle distributions, we rovide exact analytical characterizations of the erformance of the roosed threshold olicies through a Markovian analysis. Novel techniques are develoed to overcome the non-markovian nature of the original roblem and to reduce the system dimension. These techniques yield accurate analytical exressions for both the delay erformance and the PU collision robability. We conduct steady-state analysis and use techniques from renewal theory to derive closed-form aroximations of the delay and collision robability for the threshold olicies under various PU busy and idle distributions. These rovide valuable insights on the imact of system arameters on the delay and collision robability. The accuracy of these aroximations is confirmed through simulations. In related work [], a fluid queue aroximation aroach 536-76/$5. c IEEE

84 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO. 7, JULY PU transmit SU sense t= SU transmit t= t=3 time... collision t=k PU return Fig.. Illustration of the system model. Each SU slot consists of a sensing eriod and a transmission eriod. A collision occurs in the k-th SU slot, when the SU starts transmission after sensing the PU channel to be idle, and the PU returns before the end of the SU transmission eriod. was develoed to analyze the delay erformance of SUs that contend for the available rimary sectrum using random access. In [3], caacity and delay bounds for SUs were derived using a G/G/ queueing model. Our work differs from [], [3] in that we exlicitly consider the PU collision robability constraint in the delay analysis due to the imortance of PU rotection in DSA networks. In [5], under a PU collision robability constraint, the authors designed oortunistic scheduling olicies for SUs to maximize system throughut using Lyaunov functions. While the authors stated that there exists a tradeoff between throughut and delay, and a coarse delay bound can be derived following the framework in [], delay was not exlicitly analyzed in [5]. In [4], the delay analysis was develoed assuming exonentially distributed busy and idle eriods. In comarison, our work considers general busy and idle distributions which are more difficult to analyze. Due to the technical challenges of theoretical analysis, in this aer we consider only the case of a single SU accessing a PU channel, ossibly shared by multile PUs. In the analysis we also make the idealized assumtion of erfect sensing and rovide only numerical results for the imerfect sensing case. We show that the erformance analysis we develoed for the single SU case rovides a good aroximation of the erformance of the multile SUs scenario under a simle round-robin rotocol. Extensions of the analysis to the more realistic scenarios of multile SUs and multile PU bands under more sohisticated rotocols are imortant directions for future research, but are out of the scoe of this aer. The remainder of the aer is organized as follows. In Section II, we introduce the system model that characterizes the PU and SU activities. In Section III, we resent the time-threshold olicies. We analyze the erformance of the threshold olicies in Section IV through Markovian analysis, and derive closed-form analytical exressions to aroximate the delay and collision robability of such olicies in Section V. In Section VII, we study the multile SU scenario. Numerical results are resented in Section VI. Finally, we conclude in Section VIII. II. SYSTEM MODEL We consider one sectrum band that is assigned to the PU. One SU oortunistically exloits sectrum oortunities vacated by the PU under the rotection requirement of the PU. While it is ossible that there are multile PUs sharing the sectrum band, we assume that the SU does not distinguish among different PUs, and can only access the channel when no PU is active. Thus, the SU treats all PUs collectively as one aggregated PU in designing the sectrum access schemes. A. PU Model We assume that PU activities follow an alternating busy-idle attern. Multile PU ackets, ossibly with various lengths, are transmitted within a busy eriod. When all PU ackets in the queue have been transmitted, the PU channel becomes idle. The PU channel remains idle until the arrival of the next PU acket, which is the start of the next busy-idle cycle. We denote the sojourn time of the PU idle state as I, its robability density function PDF as f I, its cumulative distribution function CDF as F I, its mean as μ I = vf I vdv, and its second moment as ν I = v f I vdv. Similarly, B, f B, F B, μ B, ν B reresent the sojourn time of the PU busy state, the df, the cdf, the mean, and the second moment, resectively. B. SU Model We consider a acketized and time-slotted system for the SU. The SU has a fixed acket length that is no greater than that of the PU. Smaller values of the SU acket length allow more freedom for designing the SU access strategy. The arrival rocess of the SU is modeled as a Bernoulli rocess such that with robability an SU acket arrives in a time slot and with robability there is no acket arrival. Each SU time slot consists of a short sensing eriod, followed by a transmission eriod, as shown in Fig.. The SU senses the channel during the sensing eriod. If the channel is sensed busy, the SU does not transmit in the transmission eriod, and will sense the channel again in the sensing eriod of the next SU time slot. If the channel is sensed idle, the SU has the otions to either transmit, or not transmit, according to some transmission olicies described in Section III. For examle, as shown in Fig., the SU does not transmit in time slot after sensing, either because its queue is emty or because its transmission olicy decides so. If the SU transmits and the PU channel remains idle for the entire duration of the SU transmission eriod, then the transmission is successful. Otherwise, if the PU returns in the middle of the SU transmission eriod, a acket collision occurs. This is illustrated in Fig., where a collision occurs in time-slot k. For ease of resentation, in this aer we do not consider PU/SU acket re-transmission in the event of a collision, even though such modifications should be straightforward. We also assume an infinite buffer size at the SU and thus the acket droing robability is not considered. C. PU Collision Probability Requirement We denote c as the average acket collision robability erceived by the PU in the long-run, given by K k= c = lim su N ck K K k= N k, where N c k and N k are random variables reresenting the total number of collided and transmitted PU ackets in the k-th busy-idle cycle, resectively. The PU acket collision

CHEN and LIU: DELAY PERFORMANCE OF THRESHOLD POLICIES FOR DYNAMIC SPECTRUM ACCESS 85 robability constraint, c η, is imosed by either the PU or the sectrum regulator and is known to the SU ariori. Under the assumtions that the SU acket length is no greater than the PU acket length, and that the sensing outcome of thesuiserfect,wenotethatthere is at most one PU acket collision within a busy-idle cycle, which may only occur at the beginning of a busy eriod when the PU returns after the SU has already sensed the channel to be idle and started a transmission. During the next SU time slot, the SU will sense the channel to be busy and refrain from transmission, thus avoiding additional collisions with PU ackets. The analysis develoed in this aer is based on these assumtions. When the sensing outcome is not error-free, the SU can ossibly miss detect PU activities, causing multile PU acket collisions within a busy eriod. We will study the latter scenario through simulation in Section VI. III. TRANSMISSION POLICIES FOR MINIMIZING DELAY In this section, we study transmission olicies to minimize the average delay of the SU subject to the constraint c η. The SU is assumed to have erfect knowledge of η, f I, and f B. An otimal olicy can be found using the owerful tool of MDP. In articular, the state sace is two-dimensional: time and queue length; and the action is either to transmit or not to transmit. Through an MDP formulation, we can numerically comute the otimal olicy that minimizes the average cost in an infinite horizon. The cost considered here has two comonents: the delay cost and the collision cost. The collision cost can be adjusted numerically to meet the collision robability constraint. Due to sace limitation, we refer the readers to our technical reort [] for a detailed descrition of the MDP formulation for this roblem. The calculation of the otimal MDP olicy, however, is cumbersome and rone to numerical inaccuracy. Furthermore, the MDP olicy rovides little insight on the relationshi between delay and other system arameters. Therefore, we are motivated to look for olicies that are more structured. It is shown in [9] that for the case of backlogged traffic, under certain conditions, the otimal SU transmission olicy that achieves the time caacity of a PU channel is a timethreshold olicy. The SU should transmit only when the elased time since the channel has been idle, denoted by t, is below a threshold. The intuition behind this is that the SU should transmit only when the robability of a collision with the PU is small. For most idle distributions considered, [9] shows that conditioned uon t, the robability of a collision due to an SU transmission at time t is an increasing function of t. This naturally yields a time-threshold olicy so that the SU will transmit only when t is below a threshold. We can easily adat the time-threshold olicy of [9] here to the case when the arrival of SU ackets is dynamic. The time-threshold olicy with threshold is defined such that the SU will transmit only when the following three conditions are met: i the channel is sensed idle, ii t < note that is in general different from the that maximizes caacity, and iii the SU queue length M is greater than zero. The time-threshold should be adjusted to satisfy the PU collision robability constraint. As shown in Sections IV and V, the simlicity of time-threshold olicies facilitates theoretical analysis of the delay and collision robability. Furthermore, we find that the time-threshold olicy erforms closely to the otimal MDP olicy desite its simlicity. This reveals that the elased time t is a major factor that affects the delay and collision robability of a transmission olicy, hence justifies the usage of the time-threshold olicies considered here. IV. MARKOV CHAIN ANALYSIS OF THRESHOLD POLICIES In this section, we develo a Markovian model to analyze the erformance of the threshold olicies. Since the SU is not synchronized with the PU, the SU can only estimate I and B through eriodic sensing. Hence, even though in general I and B are continuous random variables, their estimates obtained by the SU, denoted by I d and B d, resectively, are discrete random variables whose values are integer multiles of the SU slot length. For instance, if the SU detects the PU channel to be idle for m consecutive SU slots sensing is done only once within each SU slot, then the length of the idle eriod observed by the SU is I d = m, assuming that each SU slot has unit length. The length of the busy eriod B d is defined similarly. To simlify analysis, we assume erfect sensing, i.e., the length of the sensing eriod is zero, and the sensing outcome is error-free. A. Probability Mass Function of I d and B d Given a continuous idle distribution with df f I v and cdf F I v, the robability mass function of I d can be comuted as follows. Let Z denote the time between the start of an idle eriod and the start of the next SU slot. We can model Z as a uniformly distributed random variable in [, ]. Assuming erfect sensing, the SU will sense the PU channel to be idle for m consecutive SU slots if and only if z+m I z+m. It follows that for every m =,,, P I d z+m = m = f I vdv dz = z+m FI z + m F I z + m dz. The robability mass function of B d can be comuted in a similar fashion. In the remainder of this section, for notational simlicity we suress the suerscrit in I d and B d and let I and B denote the estimated integer lengths of the idle and busy eriods, resectively. B. Construction of a one-dimensional Markov Chain Assume that the k-th PU busy-idle cycle observed by the SU consists of B k busy slots and I k idle slots. To model the dynamics of the number of SU ackets in the system, we define X k = {Number of SU ackets at the beginning of the k-thbusyeriod} Y k = {Number of SU ackets at the beginning of the k-th idle eriod}

86 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO. 7, JULY X k i Y k j = {Number of SU ackets at the end of the i-th slot of the k-th busy eriod, i B k }, = {Number of SU ackets at the end of the j-th slot of the k-th idle eriod, j I k }. Clearly, we have Y k I k = X k+ because the number of SU ackets at the end of the k-th idle eriod equals the number of SU ackets at the beginning of the k +-th busy eriod. Similarly, Y k = X k B k also holds. The dynamics of the number of SU ackets is characterized by the sequence of random variables X,X,,X B = Y Y I =X,X,Y,Y,,Y,,X B = Y I,,Y,Y, The above sequence of random variables does not form a Markov chain, because the number of SU ackets at time n deends on how long the PU channel has been in a busy/idle state. One can overcome this roblem by tracking not only the number of SU ackets, but also the elased time from the last busy-idle change. This aroach, however, results in a Markov chain with a large two-dimensional state sace and thus has rohibitive comlexity. Our aroach is based on the imortant observation that,k =,, } at the beginning of each busy-idle cycle forms a Markov chain. We treat each busy-idle cycle as a single ste in a discrete-time Markov chain and average over the lengths of busy and idle eriods to comute the one-ste transition robability matrix P.Let the sequence of random variables {X k P k+ i,j =P X =j X k =i=p Y k I k =j X k =i. 3 Next, we consider a tyical busy-idle cycle and dro the index k to write P i,j =P Y I =j X = i = P Y v = j X = ip I = v v= [ ] = P Y v = j X = i, B =up B =u P I =v. v= u= Note that the term P Y v = j X = i, B = u in 4 deends on of the threshold olicy. Details for comuting P can be found in []. Assume that the steady-state distribution of the Markov chain is π =π,π,,π n,, whereπ n denotes the robability that there are n SU ackets at the beginning of a busy eriod. We can find π by solving the equation πp = π. Thus, the average number of SU ackets at the beginning of a busy-idle cycle is EX = iπ i. C. Calculation of Average Delay The average number of SU ackets in the system is N = EX + + X B + Y + + Y I. 5 μ B + μ I Following Little s formula [], the average delay is W =N/. Here, W excludes the time slot that an SU acket i= 4 is transmitted. In [], we show that W can be comuted using 6, where P Y n == π i P Y n = X = i,n= i=,,, is comuted in 4. D. Calculation of Collision Probability Given, an SU transmits during the t-th slot of an idle eriod if t and the number of ackets at the end of the t -th idle slot is greater than zero, i.e., Y t >. This transmission will result in a collision if the PU returns during the t-th slot, i.e., I = t. Hence, we obtain c = P I = tp Y t > EN t= = P I = t[ P Y t =], 7 EN t= where EN is the average number of PU ackets er busy eriod. Here, we have used the fact that at most one acket collision occurs within each busy-idle cycle. V. ANALYTICAL APPROXIMATIONS OF THRESHOLD POLICIES While in Section IV we rovided exact Markovian analysis of the threshold olices, the resulting analytical exressions for the delay and collision robability 6 and 7 take comlex forms. In this section, we will derive simler analytical aroximations to rovide closed-form exressions for the delay and collision robability. We first resent a useful lemma that is imortant for the analysis resented in this section. Lemma : Assume that there are n SU ackets at the beginning of an idle eriod. Then on average it takes n time slots for the SU queue length to first reach zero, assuming that the SU can transmit whenever the channel is idle and the idle eriod is sufficiently long to facilitate all SU transmissions. The roof of Lemma is straightforward. Hence, the roof is omitted for brevity. An imortant consequence of Lemma is that to ensure system stability, we must choose such that > n. Furthermore, the relation between, n,and the length of the idle eriod v, lays an imortant role in determining the SU queue length by the end of the idle eriod, as we will see from subsequent analysis. A. Aroximation of n = EY Following similar techniques of Section IV, here we conduct the erformance analysis over a tyical busy-idle cycle. We find that a useful quantity is n = EY, i.e., the average number of SU ackets at the beginning of an idle eriod. Clearly, the delay of a newly arrived SU acket in a busy-idle cycle is closely related to n. Lemma below resents a key equation for n that follows from the steady-state analysis of the dynamic system. Lemma : The solution to the following equation gives a good aroximation to n = EY. n μ B = + n n + v f I v dv + n f I v dv v f I v dv. 8

CHEN and LIU: DELAY PERFORMANCE OF THRESHOLD POLICIES FOR DYNAMIC SPECTRUM ACCESS 87 W = EX { νb μ B + + μ B μ I + P I = vv v μ B + μ I v= + v v + P I = v +v + v=+ v } + P I = v P Y n =v n+ P I = v P Y n =v n, 6 v= n= v=+ n= Proof of Lemma : When the system is in steady-state, we have n = EY =EX B =EX +μ B. Hence, we obtain EY I =EX =n μ B.Letmv, n = EY v Y = n. We can aroximate EY I by EY I + mv, n f I vdv = n f I vmv, n dv+ n f I vmv, n dv f I vmv, n dv. 9 Next, we comute mv, n for each integral term in 9. First, when v< n,wehavemv, n n + v because during an idle eriod of length v, on average there are a total of v new acket arrivals and v acket deartures. Second, when n <v<, wehavemv, n. This is because it follows from Lemma that on average the SU queue length reaches zero after n time slots. Each new acket that arrives afterwards will be transmitted during the next time slot, excet for the acket that arrives in the last slot of the idle eriod. This imlies that with robability there is one acket at the end of the idle eriod and with robability there is no acket left in the queue. Third, when <v, the SU ackets that are still in the queue by the end of the idle eriod are new arrivals from [,v]. Hence, we have mv, n v. We can solve 8 to obtain n, sometimes in closedform, as shown in the examles in Section V-D. The delay aroximation resented section V-B uses knowledge of n. average delay of an SU acket, denoted by W,as W = μ B + μ I W I + W B, where μ B +μ I is the average length of a busy-idle cycle. Next, we aim to find analytical aroximations of W I and W B.The main idea is to classify SU ackets according to their arrival time and then comute the average delay for each class of ackets resectively. Aroximation of W I : We find that the following three classes of ackets that arrive during an idle eriod contribute most significantly to W I. A acket is defined as a class acket if v > and x,v. Such a acket arrives after the SU has stoed transmission within the idle eriod. We aroximate d I x, v by d I x, v v x + x + μ B,wherev x is the residual idle time, x is the queueing delay due to acket arrivals in [,x], andμ B is the average delay incurred by the next busy eriod. Hence, the total average delay due to class SU ackets is given by L = v v d I x, v dx f I vdv μ B + + v f I vdv. B. Delay Aroximation To comute the delay of a tyical SU acket, we again carry out the comutation over a tyical busy-idle cycle. Since the SU arrival follows a Bernoulli rocess, a new SU acket may arrive equally likely in any of the SU slots within the busy-idle cycle. However, the average delay of this SU acket clearly deends on whether it arrives during the busy eriod, or the idle eriod, and also the secific time slot that it arrives. These lead to the following definitions. Let d I x, v and d B x, u denote the average delay of a acket that arrives x time-units after the start of an idle eriod of length v, or after the start of a busy eriod of length u, resectively. The total average delay due to acket arrivals in different slots of an idle eriod, or in that of a busy eriod, are given by W I = v d Ix, vdxf I vdv and W B = u d Bx, udxf B udu, resectively. We then aly the renewal theory [] to obtain an exression for the A acket is defined as a class acket if v> n and x n,. It follows from Lemma that this acket is likely to be transmitted within the idle eriod, and thus d I x, v n + x x = n + x. This yields L = = n n n n n n d I x, v dx f I vdv n + x dx f I vdv f I vdv. A acket is defined as a class 3 acket if v,. Because the length of the idle eriod is short, this acket might need to wait for one more busy-idle cycle for transmission. This n

88 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO. 7, JULY leads to d I x, v n + x + μ B. Thus, L 3 = = n n n v v d I x, v dx f I vdv [ n + μ B + x dx] f I vdv [n + μ B v + v ] f I vdv. 3 We then combine,, and 3 to obtain W I L + L + L 3. Aroximation of W B : We follow similar techniques to those used for estimating W I. For any acket that arrives x time-units after the start of a busy eriod of length u, d B x, u must include the average queueing delay n +x, wheren = EX =n μ B, and the residual busy time u x. Welet R B denote the associated delay R B = u n + x + u x dxf B udu = n μ B + + u f B udu. 4 When a acket is not transmitted in the current busy-idle cycle, for instance, if n + x > minv,, wherev is the length of the idle eriod following the current busy eriod, then d B x, u has to include additional delay due to the residual idle time beyond and the next busy eriod. The associated delay is denoted by R N. We consider three classes of ackets that contribute to R N and let R Ni denote the contribution of class i ackets to R N,wherei =,, 3. The classification of ackets and the corresonding delay are given by class : v<n <. R N n u class : v n, and x R N n v n u μ Bdx f Buduf Ivdv = μ B v n v n,u. n f Ivdv. 5 μ Bdx f Buduf Ivdv. 6 class 3: v, and x n,u. u R N3 μ B + v dxf Buduf Ivdv. 7 n n We combine 4 7 to obtain W B R B +R N +R N + R N3. Substituting this into, we have W L + L + L 3 + R B + R N + R N + R N3. μ B + μ I 8 C. Aroximation of Collision Probability It follows from Lemma that if the length of the idle eriod v is less than n, then most likely that the SU queue has not reached zero when the PU returns and a acket collision will occur. If v [ n, ], then a collision occurs with robability, because this is the robability that there is one SU acket in transmission when the PU returns. This yields n c f I vdv + f I vdv. 9 EN n D. Examles The aroximations of W and c given in 8 and 9 are alicable to general busy and idle distributions. Next, we resent several examles in which we obtain closed-form exressions of such aroximations. Examle : Fixed Busy Distribution and Uniform Idle Distribution. Assume that f I t follows a uniform distribution in [, μ I ], and the busy eriod is fixed to be μ B.Wesolve8 to obtain [ μi n 4 μ I ] +4μB μ I + μ I. For the delay aroximation 8, we aly 3 to obtain L μ I 3 + + μ B μ I, μ I 4μ I n L μ I n, 4 μ I L 3 n n + μ B 4μ I n 3 μ I, and aly 4 7 to obtain R B n μ B ++ μ B, R N n μ B, μ I R N μ3 B, R N3 =. 4μ I The collision robability can be comuted from 9 as c μ I EN n +. 3 Examle : Exonential Busy Distribution and Uniform Idle Distribution. This examle differs from Examle only in the busy distribution. Hence,, and 3 remain the same, and is relaced by R B μ B + +n μ B, R N μ3 B e n μ B, μ I R N3 μ B μ I R N n μ B μ I, 4μ I + μ B μ I e n μ B. 4 Examle 3: Fixed Busy Distribution and Weibull Idle Distribution. We consider a Weibull idle distribution with f I t = t λ e t λ. We solve 8 to obtain n λ erfinv erfc λ + μ B πλ. 5 Here, erfx is the error function, erfinvx is the inverse error function, and erfcx is the comlementary error function. For

CHEN and LIU: DELAY PERFORMANCE OF THRESHOLD POLICIES FOR DYNAMIC SPECTRUM ACCESS 89 the delay aroximation 8, we aly 3 to obtain λe λ L μ πerfc λ I + μ Bμ I π.5.5 erfc λ, n n λ, L e L 3 μ I n + μ B λ π erf n λ n n e n λ μ I λ [ n + e λ λ ], 6 and aly 4 7 to obtain R B μ B + +n μ B,R N μ B n e λ,r N3 =, R N μ B μ B n + μ B + n e μ B +n λ + μ B n n + μ B e n λ π + μ B λ erf n λ erfμ B + n. 7 λ The collision robability is comuted from 9 c e n λ + e λ EN e λ. 8 VI. NUMERICAL RESULTS In this section, we resent numerical results to examine the erformance of the threshold olicies and the accuracy of the analysis. A. Simulation Setu We generate the PU traffic in continuous time, consisting of 7 consecutive busy-idle cycles. The length of each idle eriod is continuous and follows the df f I v. Assuming that each PU acket has unit length, the busy eriod takes an integer value equals the number of PU ackets in the busy eriod, obtained by rounding the continuous random variable B generated following f B u. Both the SU slot length and the SU acket length are set to be one. The SU acket arrival in each SU slot is Bernoulli with robability. The SU senses the channel only at the beginning of each SU slot to obtain I d and B d.given of the threshold olicy, the SU transmits when its queue is nonemty and I d. The SU acket delay is comuted by averaging over the delay exerienced by each SU acket transmitted over the entire simulation eriod. The simulated PU acket collision robability is evaluated following. From Section VI-B to VI-E, we assume erfect sensing with zero sensing time and error-free sensing outcome. Detailed simulation setu for the case of imerfect sensing can be found in Section VI-F. B. Comarison of Threshold Policy and MDP Policy In Fig., we comare the threshold olicy with the MDP olicy comuted from the discounted MDP formulation described in []. The busy eriod is fixed to be μ B =. The length of the idle eriod is uniformly distributed in [, 3] with mean μ I = 5. The caacity of this system is Delay W SU time slot 85 8 75 7 65 6 55 5 45 4 35 3 5 Threshold olicy MDP olicy.8.85.9.95..5. SU arrival rate Fig.. Comarisons of threshold olicy and MDP olicy for a fixed busy distribution and a uniform idle distribution. For each, wefind a threshold olicy and an MDP olicy such that c = η =.. queue length M 9 8 7 6 5 4 3 Transmits under both olicies = 94 for threshold olicy Transmits under MDP olicy, does not transmit under threshold olicy line line does not transmit under either olicy 5 5 75 5 5 75 5 5 75 3 idle time t SU time slot Fig. 3. Transmission regions of the MDP olicy and the threshold olicy for a fixed busy time distribution and a uniform idle distribution. Assume =.. The two olicies are found such that c = η =.. C =.4 for η =. [9]. For each SU acket arrival rate, we first determine for the threshold olicy such that c = η and then find delay W under this olicy. Using the MDP formulation, for each, we adjust the cost C m to find an MDP olicy so that c = η and evaluate its delay. In Fig., we lot the delay of threshold olicy and MDP olicy as a function of. It shows that the threshold olicy erforms very closely to MDP olicy for the entire range of considered. In Fig. 3, we comare the two-dimensional transmission regions of MDP olicy and threshold olicy for =.. The threshold olicy has =94. The MDP olicy is a function of both the elased idle time t and the queue length M. The SU transmits when t, M falls into the region to the left of line shown in Fig. 3. As oosed to the MDP olicy, the threshold olicy is indeendent of M and its transmission region is to the left of line corresonding to t =94. It is interesting to note that the corner oint of the MDP curve t, M =9, is very close to of the threshold olicy. While the transmission region of MDP olicy is larger than that of the threshold olicy, the two olicies yield similar delay erformance. This is because the robability that t, M

9 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO. 7, JULY Delay W SU time slot PU Collision Prob. c 35 5 5 5 95 85 75 65 55 Markov chain analysis ex/unif Simulation ex/unif Delay arox. ex/unif Markov chain analysis fix/wbl Simulation fix/wbl Delay arox. fix/wbl 45 6 7 8 9 3 4 5 6 Threshold SU time slot a Delay.5 x 3.45.4.35.3.5..5..5.95.9.85 66 76 86 96 6 6 6 36 46 56 Threshold SU time slot b Collision robability Fig. 4. Performance of threshold olicies as a function of. Two grous of curves are resented. Consider an exonential busy distribution and a uniform idle distribution for =.. Comarisons of Markov chain analysis 6 and 7, analytical aroximations 8 and 9 based on, 3, 4, and simulation results. Consider a fixed busy distribution and a Weibull idle distribution for =.. Comarisons of Markov chain analysis 6 and 7, analytical aroximations 8 and 9 based on 6 8, and simulation results. The legend of sub-figure b is the same as that of subfigure a. belongs to the middle region between line and line is small. Its effect on the delay erformance is thus negligible. C. Accuracy of Markov Chain Analysis and Analytical Aroximations In Fig. 4 we examine the accuracy of the Markov chain analysis in Section IV and the analytical aroximations in Section V. Here, we fix and lot W and c as functions of. The curves for the Markov chain analysis are numerically evaluated from 6 for W and 7 for c. The aroximations for W and c are from 8 and 9, evaluated using closedform exressions resented in Section V-D. Two grous of curves are considered in Fig. 4. First, we consider an exonential busy distribution and a uniform idle distribution for =.. It shows that the Markov chain analysis matches the simulation results erfectly. A total of states is found to be sufficient to obtain accurate numerical results for Markov chain analysis. Following Examle in Section V-D, we Delay W SU time slot PU Collision Prob. c 3 Fixed busy dist. Uniform busy dist. Exonential busy dist. Weibull busy dist. 9 8 7 6 5 4 6 7 8 9 3 4 5 6 Threshold SU time slot.6 x 3.4...8.6 a Delay Fixed busy dist. Uniform busy dist. Exonential busy dist. Weibull busy dist..4..98.96.94.9.9 6 7 8 9 3 4 5 6 Threshold SU time slot b Collision robability Fig. 5. Performance of threshold olicies as a function of. Delay and collision robability comarisons of various busy distributions. Assume a uniform idle distribution and =.. evaluate the closed-form analytical aroximations for W and c. Both aroximations are very tight, yielding errors less than 3% 4%. Similar results hold for Examle, but are not resented here due to sace limitation. Second, we consider the fixed busy distribution and Weibull idle distribution. Since Weibull distribution yields a higher caacity of C =.3, we choose a higher =. to oerate near caacity. While the Markov chain analysis remains accurate, the analytical aroximations from Examle 3 become looser comared to that of the exonential busy distribution and uniform idle distribution. The error of the delay aroximation, as shown in Fig. 4 a, is roughly 5% excet for =66. Note that the delay increases raidly around this value of, andsome SU ackets might need to wait for more than two busy-idle cycles for transmission. This is not taken into account in the analysis of Section V, which may contribute to the inaccuracy of the aroximation at this oint. The error of the collision robability aroximation is about 8% in the case. D. The Imact of Busy Distribution In Fig. 5, we examine the effect of busy distribution on W and c. We consider four different busy distributions with the same mean: exonential, uniform, Weibull, and the fixed busy

CHEN and LIU: DELAY PERFORMANCE OF THRESHOLD POLICIES FOR DYNAMIC SPECTRUM ACCESS 9 Delay W SU time slot 6 4 8 6 4 Exonential idle distribution Uniform idle distribution Weibull idle distribution * =.75 for uniform dist. * =.6 for exonential dist. * =.95 for Weibull dist...4.6.8...4.6.8. SU arrival rate Fig. 6. Delay erformance of the threshold olicies as a function of. For each, we choose of the threshold olicy such that c = η =.. Assume a uniform busy distribution and various idle distributions. time. Fig. 5 a shows that the busy distribution affects W significantly. The exonential busy distribution and the fixed busy distribution induce the largest delay and the smallest delay, resectively. On the other hand, for the same, Fig.5 b shows that c changes only slightly with busy distributions. E. The Imact of Idle Distribution Next, we examine the effect of idle distribution on W. We assume a uniform busy distribution and consider three idle distributions with the same mean μ I = 5: uniform, Weibull, and exonential. For the first two idle distributions and for each, we determine such that c = η =.. For the exonential distribution, due to its memoryless roerty, it is otimal to use a greedy olicy, under which the SU transmits whenever the channel is sensed idle and the SU queue is nonemty. Given η =., the time caacity C =.6,.4,.3, for the exonential, uniform, and Weibull idle distribution, resectively. In Fig. 6, we lot the delay of the threshold olicy as a function of for η =.. For each idle distribution, there exists some such that when <, the threshold olicy becomes the greedy olicy. We find that =.6,.75,.95 for the exonential, uniform, and Weibull distribution, resectively. Fig. 6 shows that for these idle distributions, the delay of the threshold olicy is similar in the region < where the greedy olicy is otimal. For the exonential distribution, =.6 is the highest arrival rate such that c does not exceed η. In comarison, the caacity of the uniform distribution is higher. Therefore, when [.75,.4, a threshold olicy can be found to ensure that c = η at the cost of increased delay. We note that the delay of the threshold olicy increases raidly as aroaches C =.4. The Weibull distribution has the highest caacity. When [.95,.], the delay of the threshold olicy increases with, but at a slower rate than that of the uniform distribution. F. Imerfect Sensing The analysis develoed in this aer assumes erfect sensing, i.e., the sensing time is zero and the sensing outcome is error-free. In Fig. 7, we examine more realistic scenarios of imerfect sensing. The busy and idle distributions are uniform with μ B = and μ I = 5, resectively. The PU acket length is fixedtobems.eachsuslotisms long, in which 5% is for sensing, and 95% is for transmission. We first consider three imerfect sensing scenarios in which the miss detection robability = 3, and the false alarm robability γ f is.,., and., resectively. We set low in order to limit PU acket collisions. Due to the ossibility of false alarm, the SU will determine the transition from an idle eriod to a busy eriod only when, within an idle eriod, the PU channel is detected to be busy for several consecutive sensing eriods. Similarly, due to the ossibility of miss detection, the transition from a busy eriod to an idle eriod is determined only when, within a busy eriod, the PU channel is detected to be idle for a few consecutive sensing eriods. As shown in Fig. 7, both W and c decrease as γ f decreases. A fourth imerfect sensing scenario, in which = γ f =and the other system arameters are the same as the revious three scenarios, is also lotted as a erformance benchmark. This is the best erformance that one can achieve for the given system setu with nonzero sensing time. Fig. 7 shows that indeed the curves for = γ f =are very close to that of the analytical curves obtained from 8 and 9, which assume error-free sensing outcome and zero sensing time. We also observe from Fig. 7 b that for c, the analytical curve gives close aroximation to all four curves of imerfect sensing scenarios, and the largest ga is less than 6%. In comarison, imerfect sensing has a stronger effect on W.As shown in Fig. 7 a, the ga between the imerfect sensing curves and the analytical curve becomes more ronounced as γ f increases. VII. DISCUSSION ON MULTIPLE SUS In this section, we discuss the scenario where multile SUs share one PU channel. There has been a significant amount of work on various multile access mechanisms, such as slotted ALOHA, CSMA, ooling, and TDM, e.g., in [3], [4]. Here, in order to focus on the imact of delay analysis, we consider a modified TDM version. We assume that all SUs are synchronized. The SUs have a re-assigned rank, so that if all SUs have ackets to send, their multile access is in a TDM round-robin manner. Note that it requires only minor signaling to establish ranking and synchronization through a control channel. Existing literature in cognitive radio advocates dedicated control channels either virtual or hysical [5]. Assume that there are N SUs, numbered as {,,,N}. SU i is allocated time slots i + Nk,wherek =,,..., for all k such that i + Nk. To imrove channel utilization, we allow other SUs to use slots i + Nk when SU i has an emty queue. The rank of the SUs for slot i + Nk is i, i +,,N,,,,i, wheresui has the highest rank. To facilitate channel access, we divide each SU time slot into multile mini-slots, each with length α. Assume that there

9 IEEE TRANSACTIONS ON WIRELESS COMMUNICATIONS, VOL., NO. 7, JULY Delay W SU time slot PU Collision Prob. c 3 9 8 7 6 Analytical =, γ f = = 3, γ f =. = 3, γ f =. = 3, γ f =. 5 6 7 8 9 3 4 5 6 Threshold SU time slot.6 x 3.4...8.6.4..98.96.94.9 a Delay Analytical =, γ f = = 3, γ f =. = 3, γ f =. = 3, γ f =..9 6 7 8 9 3 4 5 6 Threshold SU time slot b Collision robability Delay W SU time slot PU Collision Prob. c 3 9 8 7 6 Analytical N= α=.5, N=3 α=., N=3 α=.5, N=3 5 6 7 8 9 3 4 5 6 Threshold SU time slot.9.7.5.3..99.97 x 3 a Delay Analytical N= α=.5, N=3 α=., N=3 α=.5, N=3.95 6 7 8 9 3 4 5 6 Threshold SU time slot b Collision robability Fig. 7. Performance of threshold olicies as a function of. Comarisons of delay and collision robability for various imerfect sensing scenarios and the analytical aroximations from 8 and 9 that assume erfect sensing. Assume a uniform idle distribution and a uniform busy distribution, =.. Fig. 8. Consider a uniform busy distribution with μ B = and a uniform idle distribution with μ I = 5. Assume =.. Performance comarisons of the single user case N =with multile SU case N =3under a roundrobin rotocol. The analytical aroximations 8 and 9 for N =are rovided as erformance benchmarks. are m SUs whose ranks are higher than SU j. ThenSUj is allowed to transmit starting from the beginning of the m+- th mini-slot, if none of the higher ranked SUs transmitted in the first m mini-slots. We also assume that each SU attemts to transmit at most once within an SU time slot, and thus if the first N mini-slots are idle, then the remainder of the SU slot will be idle. This round-robin rotocol requires that each SU erforms channel sensing not only at the beginning of each SU slot, but also at the beginning of each mini-slot, until the channel is sensed to be busy due to either an SU transmission or a PU transmission. For examle, consider N =3.Letthe length of an SU time slot be and the length of a mini-slot be α =.. Thefirst SU slot in an idle eriod is assigned to SU. The rank of the SUs is {,, 3}. If SU s queue is non-emty, then it transmits and uses the entire SU time slot. If SU s queue is emty, then after α, SU will sense the channel, and will transmit if the channel is idle and its queue is non-emty. If SU also has an emty queue, then after another α, SU 3 will sense the channel and will transmit in the remainder of the SU slot if the channel is idle. For the next SU slot within the same idle eriod, the rank of the SUs becomes {, 3, }. The rank of the SUs is reset to {,, 3} for the first SU slot in the next idle eriod. In Fig. 8, we comare the erformance of the round-robin multile SU rotocol N = 3 with that of the single SU case N =. We consider a uniform busy distribution with μ B = and a uniform idle distribution with μ I = 5. For both delay W and collision robability c,wevary to resent the analytical curve 8 and 9 derived for N =, simulation curves for N =,andforn =3with α =.5,.,.5. It shows that the analytical curves for N =remain good aroximations within % ofw and c under the multiuser round-robin rotocol. We observe that as shown in Fig. 8 a, W increases only slightly with increasing N and α. asshowninfig.8b, c decreases slightly with increasing N and α. These can be exlained as follows. For the case of N =, channel sensing is erformed only at the beginning of each SU slot. A collision occurs if the SU starts transmission after sensing the channel to be idle at the beginning of an SU slot, and the PU returns later during this time slot. For the multiuser case, it is ossible to avoid such collisions. For instance, during an SU time slot [, ], the PU returns at time.5. For the single user case, a collision occurs when the SU has a non-emty queue. For

CHEN and LIU: DELAY PERFORMANCE OF THRESHOLD POLICIES FOR DYNAMIC SPECTRUM ACCESS 93 the multiuser case, if α =. and the first ranked SU has an emty queue, then the second ranked SU will refrain from transmission since it senses the channel at time. and detects the PU transmission. Therefore, a collision is avoided at the exense of the deferred SU transmission. Since the robability of such events increases with α, we observe in Fig. 8 b that c decreases with increasing α. Another ossible aroach for multile SUs to share a single PU channel is to develo back-ressure-tye of algorithms [6]. Intuitively, users with longer queues will have higher riority of access. Back-ressure algorithms are likely to be throughut-otimal in our setting assuming aroriate multile access schemes. Along this line, one can rove queue stability of such algorithms e.g., through Lyaunov functions [], and there are various heuristics that can balance the tradeoff between delay and throughut. However, the delay of such an algorithm cannot be analyzed recisely as in our current aroach, to the best of our knowledge. VIII. CONCLUSIONS In this work, we roosed and analyzed threshold olicies to minimize the delay of the SU subject to a PU collision robability constraint. Such threshold olicies were shown to erform closely to the otimal olicy found through a discounted MDP formulation. A novel Markovian aroach was develoed to analyze the erformance of the threshold olicies. This aroach treats each PU busy-idle cycle as a one-ste transition in the Markov chain, which effectively reduces the dimension of the state sace to facilitate numerical comutations. We develod analytical exressions to aroximate the delay and collision robability of the threshold olicies under general busy and idle distributions. The accuracy of the roosed aroximations was confirmed numerically for several commonly used busy and idle distributions. Furthermore, we showed that the busy time distribution significantly imacts the delay of the SU, while the idle distribution largely determines the transmission threshold and collision robability. This is a dual observation of the results of [9], which showed that the PU idle time distribution determines the time caacity of the SU access. Future work includes extension of the erformance analysis to more general scenarios such as arbitrary SU arrival rocesses, multile SUs and multile PU channels. REFERENCES [] Q. Zhao and B. Sadler, A survey of dynamic sectrum access: signal rocessing, networking, and regulatory olicy, IEEE Signal Process. Mag., vol. 55, no. 5,. 94 39, 7. [] S. Wang, J. Zhang, and L. Tong, Delay analysis for cognitive radio networks with random access: a fluid queue view,. [3] F. Borgonovo, M. Cesana, and L. Fratta, Throughut and delay bounds for cognitive transmissions, in Advances in Ad Hoc Networking, 8. [4] E. W. M. Wong and C. H. Foh, Analysis of cognitive radio sectrum access with finite user oulation, IEEE Commun. Lett., 9. [5] R. Urgaonkar and M. J. Neely, Oortunistic scheduling with reliability guarantees in cognitive radio networks, IEEE INFOCOM, Ar. 8. [6] S. Huang, X. Liu, and Z. Ding, Oortunistic sectrum access in cognitive radio networks, IEEE INFOCOM, Ar. 8. [7] Q. Zhao, S. Geirhofer, L. Tong, and B. M. Sadler, Otimal dynamic sectrum access via eriodic channel sensing, in Proc. Wireless Communications and Networking Conference, 7. [8] T. Shu, S. Cui, and M. Krunz, Medium access control for multichannel arallel transmission in cognitive radio networks, in Proc. IEEE GLOBECOM, Dec. 6. [9] S. Huang, X. Liu, and Z. Ding, Otimization of transmission strategies for oortunistic access in cognitive radio networks, IEEE Trans. Mobile Comuting, 9. [] L. Georgiadis, M. J. Neely, and L. Tassiulas, Resource Allocation and Cross-Layer Control in Wireless Networks, Foundations and Trends in Networking. [] R.-R. Chen and X. Liu, Delay erformance of threshold olicy for dynamic sectrum access. Available: www.cs.ucdavis.edu/ liu/aer/rongrongmdp.df,. [] S. M. Ross, Introduction to Probability Models, 8th edition. Academic Press, 3. [3] J. W. Lee and J. Walrand, Zero collision random backoff algorithm, EECS Deartment, University of California, Berkeley, tech. re. UCB/EECS-7-63, May 7. [4] D. Bertsekas and R. Gallager, Data Networks. Prentice Hall, 987. [5] R. W. Brodersen, A. Wolisz, D. Cabric, S. M. Mishra, and D. Willkomm, CORVUS: a cognitive radio aroach for usage of virtual unlicensed sectrum, 4. [6] L. Tassiulas and A. Ehremides, Stability roerties of constrained queueing systems and scheduling olicies for maximum throughut in multiho radio networks, IEEE Trans. Automatic Control, vol. 37, no.,. 936 948, Dec. 99. Rong-Rong Chen received her Ph.D. degree in electrical and comuter engineering ECE from the University of Illinois at Urbana-Chamaign UIUC in 3. She is currently an assistant rofessor in the Deartment of ECE. at the University of Utah. Her main research interest is in communication systems and networks, with an emhasis on MIMO communication, statistical soft detection, underwater acoustic communication, and cognitive radio networks. She was the reciient of the M. E. Van Valkenburg Graduate Research Award for excellence in doctoral research in the Deartment of ECE at UIUC. She is a reciient of the restigious National Science Foundation NSF Faculty Early Career Develoment CAREER award in 6. Xin Liu received her Ph.D. degree in electrical engineering from Purdue University in. She is currently an associate rofessor in the Comuter Science Deartment at the University of California, Davis. Before joining UC Davis, she was a ostdoctoral research associate in the Coordinated Science Laboratory at UIUC. Her research is on wireless communication networks, with a focus on resource allocation and dynamic sectrum management. She received the Best Paer of year award of the Comuter Networks Journal in 3, NSF CAREER award in 5, the Outstanding Engineering Junior Faculty Award from the College of Engineering, University of California, Davis in 5.