Towards Achieving Full Secrecy Rate in Wireless Networks: A Control Theoretic Approach

Transcription

1 owards Achieving Full Secrecy Rate in Wireless Networks: A Control heoretic Approach Zhoujia Mao Department of ECE he Ohio State University maoz@ece.osu.edu Can Emre Koksal Department of ECE he Ohio State University koksal@ece.osu.edu Ness B. Shroff Departments of ECE and CSE he Ohio State University shroff@ece.osu.edu Abstract In this paper, we consider a single-user secure data communication system. Data packets arriving at the transmitter are enqueued at a data queue to be transmitted to the receiver over a block fading channel, securely from an eavesdropper that listens to the transmitter over another independent block fading channel. We address two separate problems, both of which involve the maximization of a long-term average utility, defined as a function of the number of secure packets transmitted in each time slot. We propose a transmission controller and an admission controller based on simple index policies that do not rely on any prior statistical information on the data arrival process. he former chooses a random key generation and transmission) rate as well as the secure data transmission rate in each time slot. Part of the data is secured by the available secrecy rate while the other part is encrypted by the key bits, enqueued at both the transmitter and the receiver. he latter chooses the amount of data admitted by the transmitter to be enqueued in the data queue. We show that our controller pair has a provably efficient performance. Also, we illustrate via simulations that the use of a key queue reduces the queuing delay for the data packets, while serving packets that are admitted at the maximum admissible rate. o our best knowledge, this is the first work that addresses the queuing delay in the context of secrecy. I. INRODUCION Motivated by the seminal paper by, there has been a large number of investigations e.g., 8) on wireless information theoretic secrecy. hese studies have significantly enhanced our understanding of the basic limits and principles of the design and the analysis of secure wireless communication systems. Despite the significant progress in information theoretic secrecy, most of the work has focused on physical layer techniques. he application of wireless information theoretic secrecy remains mainly unresolved as it relates to the design of wireless networks and its impact on network control and protocol development. Indeed, our understanding of the interplay between the secrecy requirements and the critical functionalities of wireless networks, such as scheduling, routing, and congestion control remains very limited. o that end, there have been some recent efforts to utilize the insights drawn from the aforementioned investigations on information theoretic secrecy to build secure wireless networks. In 9 3 the fundamental capacity and connectivity scaling laws of wireless networks with secrecy have been addressed. In 4, 5, single hop uplink scenario has been considered in which nodes enqueue arriving private and open data packets to be transmitted to a base station over block fading channels. A node is scheduled to transmit information privately from the other nodes and rate is controlled carefully to maximize an overall utility. he solution provided follows up on the stochastic network optimization framework e.g., as treated in 6 9) and generalizes the uplink scenario to incorporate secrecy as a quality of service requirement. In a separate direction 0 proposed the idea of the use of a key queue in a single user system. here, a key queue is kept at the transmitter and the receiver, separately from the data queues. Instead of using the entire instantaneous secrecy rate for information transmission at all times, some of it is utilized to transmit key bits, generated randomly at the transmitter. hese stored key bits are used later to secure information bits in such a way that, even when the instantaneous secrecy rate is 0, information bits can still be transmitted to the destination securely from the eavesdropper. Hence, the idea of key sharing allows one to bank secrecy rates at certain times to be utilized at other times. It is shown in that, using this idea, a long-term constant secrecy rate, identical to the secrecy capacity expected instantaneous secrecy rate) of the channel is achievable. In this paper, we address the single user setting in the presence of arrival of data packets being enqueued at a data queue to be transmitted to the receiver over a block fading channel, securely from an eavesdropper that listens to the transmitter over another independent block fading channel. We consider two separate problems, which involve the maximization of a long-term average utility, defined as a function of the number of secure packets transmitted in each time slot. While, one would be inclined to exploit the entire secrecy rate for data transmission in each time slot in a greedy fashion, we show that this approach leads to a performance loss. Instead, the use of a key queue leads to a smoother secrecy rate, which in turn maximizes a concave utility, since it is negatively affected by the second order factors caused by the variability of the service. We propose a controller, which chooses the key generation and transmission) rate along with the secure data transmission rate in each time slot using a transmission control component. he admission control component chooses the amount of data admitted by the transmitter to be enqueued in the data queue. hese two components are based on simple index policies that do not rely on any prior statistical information on the data arrival process. We show that our

2 q k t) data queue µt) q d t) R k t) q k t) µ k t) µ k t) + R m t) xmit µ t) µ k t) Fig.. shared key queues System model t) R e recv R k t) µ k t) eavesdrop controller achieves a utility, arbitrarily close to the optimal utility. Also, we illustrate via simulations that the use of key queue reduces the queuing delay for the data packets, while serving packets that are admitted at the maximum admissible rate. o our best knowledge, this is the first work that addresses the queuing delay in the context of secrecy. II. SYSEM MODEL We consider the single-user system illustrated in Fig., in which the transmitter enqueues data packets to be transmitted to the receiver over the main channel at a fixed power, securely from an eavesdropper, overhearing the transmission over a separate channel. ime is slotted, and the time-varying rates of the main and the eavesdropper channel follow general processes R m = {R m 0), R m ),..., R m ),...} and R e = {R e 0), R e ),..., R e ),...}, respectively. In this paper, we assume perfect knowledge of these rates at the transmitter. We also assume the time slots are long enough and as shown in, the achievable instantaneous secrecy rate at a given slot t is identical to R s t) = R m t) R e t)) +, t 0, where ) + = max, 0. In a given time slot, this rate is fully utilized: part of it is used to secure data from the data queue and the remaining part is used to transmit randomly generated key bits to be stored at the both key queues at the transmitter and the receiver. he size of the data and the key buffers are infinite. As shown in Fig., the amount of secure data transmitted at a time t is µt). A part µ k t) bits) of this data is secured using µ k t) key bits by a simple bit-by-bit XOR operation. he remaining µt) µ k t) bits is secured using the available secrecy rate R s t). Since the secrecy rate is fully utilized, the portion of the secrecy rate, not used to secure data is used to transmit R k t) key bits. he data arrivals to the system is represented by the arrival process {At)}. he data queue state is denoted by q d t). he Lindley equation that models the state evolution of the key queue is: q k t + ) = q k t) + R k t) µ k t). We provide an equivalent model in the following lemma along with the constraints that specify the relationships between the parameters. Lemma : he key queue q k can be modeled with the state evolution equation q k t + ) = q k t) + R s t) µt) with the constraints 0 µt) minq k t) + R s t), R m t), 0 µ k t) minµt), R e t), and µt) µ k t) ) + R k t) = R s t). Proof: We have the following relationships between the system parameters: ) µ k t) µt): the amount of key bits used to secure data does not exceed the amount of transmitted data. ) µt) µ k t) + R k t) = R s t): the instantaneous secrecy rate is fully utilized: µt) µ k t) is the amount of transmitted data secured over the wiretap channel in slot t and the rest of it is used to generate key bits. 3) µ k t) min{r m t), R e t), q k t)+r k t)}: the amount of used key bits cannot exceed the main channel rate, since we cannot send data at a higher rate even if all of it is secured using key bits, i.e., µ k t) R m t). Also, we cannot use more key bits than the amount available in the key queue, i.e., µ k t) q k t) + R k t). Finally, for full secrecy, the amount of shared randomness we need is no more than the eavesdropper channel rate. Using more key bits will be wasteful, i.e., µ k t) R e t). Claim : Constraints ), ), and 3) imply µt) R m t). If R m t) R e t), then R s t) = 0 and by Constraint ), µt) = µ k t) R k t) µ k t). With Constraint ), we have µt) = µ k t), i.e., when R s t) = 0, we use key bits to secure all the data. Likewise, from Constraint 3), we obtain µt) min{r m t), q k t) + R k t)} R m t); If R m t) > R e t), then R s t) = R m t) R e t) and by Constraint ), µt) = R m t) R e t) + µ k t) R k t) R m t) + µ k t) R e t). Since µ k t) R e t) from Constraint 3), µt) R m t). Claim : By Constraints ) and 3), µt) = R s t) R k t) + µ k t) q k t) + R k t) + R s t) R k t) = q k t) + R s t). Claim 3: he key state evolution is equivalent to q k t + ) = q k t) + R k t) µ k t) = q k t) + R s t) µt) by Constraint ). Claim 4: Combining µt) R m t) and µ k t) µt) leads to µ k t) R m t). Claims,,3 and 4 complete the proof. III. PROBLEM FORMULAION We consider two problems. In our first problem, we assume an infinitely backlogged data queue, i.e., q d 0) =. he objective is to maximize the long-term average utility, which is a function of the transmission rate. Our control parameters are the number, µ k t), of used key bits, the number, µt), of served data bits, and the number, R k t), of generated key bits. In particular, we have:

3 3 A) max Uµt)) µ, µ k, R k s.t. q k t + ) = q k t) + R s t) µt), ) 0 µt) minq k t) + R s t), R m t), ) 0 µ k t) minµt), R e t), 3) µt) µk t) ) + R k t) = R s t), 4) where the utility function U ) is assumed to be monotonically increasing, reversible and differentiable on the half real line R + {0}. Note that if there were no key queue, then we would have q k t) = 0, µt) = R s t), µ k t) = 0, R k t) = 0, t 0. Also note that, the maximum achievable average secrecy rate is upper bounded by the average secrecy capacity R s = R st). In our second problem, we assume a general data arrival process, {At)} at the input of the data queue. At time t, only a portion Rt) of all arrivals are admitted into the data queue in order to keep the data queue stable. All the admitted packets are required to be served by the system eventually. In the second problem, our objective is maximize the long-term average admitted data rate. B) max Rt) R, µ, µ k, R k s.t. q d t + ) = q d t) µt)) + + Rt), 5) q k t + ) = q k t) + R s t) µt), 6) 0 Rt) At), 7) 0 µt) minq k t) + R s t), R m t), 8) q d t) <, 9) 0 µ k t) minµt), R e t), 0) µt) µk t) ) + R k t) = R s t). ) Note that, the maximum achievable average secrecy rate, which happens to be the objective function here, is upper bounded by the average secrecy capacity R s. As we shall show, R s can be achieved even without a key queue. However, we will also illustrate that our solutions that involve the use of the key queue lead to smaller queueing delays, compared to the one without the key queue. In these two problems, constraint 5) describes the data queue evolution, and constraints ) and 6) describe the key queue evolution. Constraint 7) bounds the actual amount of sensed data Rt) by the available amount of data At) at time t. Constraints ) and 8) state that the amount of transmitted data is bounded by both the main channel rate and the amount of keys available. Constraint 9) guarantees data queue stability. Constraints 3) and 0) state that the amount of key bits used to secure data is bounded by the eavesdropper channel rate and does not exceed the amount of transmitted data. Constraints 4) and ) mean that the secure capacity is fully utilized by the transmission of secure data and key bits. irtual Queue: In order to have a fair rate allocation, we do not want the key queue to be drained frequently, which would lead to outages whenever R s t) = 0. We define q k as the virtual key queue and try to avoid key outage by making the virtual key queue stable similar ideas of utilizing virtual queue are used in 9,. he virtual queue evolves according to the following equation: q k t + ) = q k t) ɛ) + + µt) R s t) + o t) ) +, ) where ɛ > 0 can be chosen arbitrarily, and o t) = key queue hits zero state from higher states in slot t { 0 if µt) = 0 or µt) < qk t) + R = s t) otherwise 3) is the indicator that the key queue is drained in slot t. Without loss of generality, the initial state q k 0) can be set to be zero. I. CONROL ALGORIHM AND PERFORMANCE ANALYSIS In this section, we provide a simple control algorithm, analyze its performance, and show that its provably optimal for both problems described in the previous section. A. Algorithm Our algorithm for Problem A) involves only a transmission rate controller. he transmission controller attempts to provide a smooth service by the help of the key bits. ransmission Control C): We define R + to be the control parameter of our algorithm. In slot t, the controller solves the following optimization problem and transmits with the calculated rate: max µt) Πt) Uµt)) q kt)µt), 4) where Πt) = {µt) : 0 µt) minq k t)+r s t), R m t)} is a compact and nonempty set. Furthermore, key generation and usage rates R k t), µ k t)) are chosen as follows: If µt) > R s t), then R k t) = 0 and µ k t) = µt) R s t); if µt) R s t), then µ k t) = 0 and R k t) = R s t) µt). his ensures that constraint µt) µ k t) ) + R k t) = R s t) is satisfied. It is not surprising that µ k t)r k t) = 0, since any solution with µ k t) > 0 and R k t) > 0, can be equivalently replicated by using the secrecy rate to transmit data rather than generating and using key bits at the same time. Note that R s t) = R m t) R e t) ) + Rm t) R e t), then for µt) > R s t), we have µ k t) = µt) R s t) R m t) R s t) R e t). his leads to constraint 0 µ k t) minµt), R e t) being satisfied. he set, Πt) of possible data transmission guarantees constraint ) on µt) in Problem A). If U ) is concave, the objective function is a concave function of µt). Consequently, C solves a simple convex optimization problem in each time slot. he positive term Uµt)) can be viewed as a utility obtained from the transmission rate µt) and the term q k t)µt) can be viewed as its associated cost. When the virtual key queue q k t) is small, C tries to allocate a high amount of transmitted data to increase the utility; and when

4 4 q k t) is large, C allocates a small amount of transmitted data to reduce cost. his pushes the served data rate to be smoother over time. It is also notable that 4) involves only µt). he key generation and usage rates are not part of this optimization, and are chosen subsequently. In Problem B), we need to control both the admission and transmission rate such that the admitted rate is maximized while keeping the data queue stable. In our algorithm, there are two components: a admission control component and a transmission control component. he transmission control component is the identical to the one described above for Problem A), and the admission control component is as follows: Admission Control AC): In slot t, the controller solves the following optimization problem and admit the calculated amount of data arrivals: max 0 Rt) At) URt)) q dt)rt), 5) Both C and AC are index policies, i.e., the solutions are memoryless and they depend only on the instantaneous values of the system variables. B. Performance Analysis Recall that At) is the original data arrival and Rt) is the amount of data admitted to the data queue. he natural question one would ask here is, whether our admission controller rejects too many packets in the first place to synthetically keep the data queue stable. In the following theorem, we show that this is not the case. Indeed, the admission rate associated with AC and C can be made closer to the optimum by increasing the control parameter. We use the notation y = Ox) to represent y going to 0 as x goes to 0. heorem : If ) U ) is strictly concave on R + {0}, and its slope at 0 satisfies 0 β = U 0) <, ) 0 R max <, t 0, then C achieves: and AC achieves: A t) < and 0 R m t) Uµt)) Uµ t)) O ), 6) q d t) β, t 0 7) URt)) UR t)) O ), Rt) For instance, U + R) = log + R). 8) R t) as, 9) where µ = {µ 0), µ ),..., µ ),...} and R = {R 0), R ),..., R ),...} are the optimal solutions to Problem A) and Problem B), respectively. he proof of heorem can be found in Appendix A. Equation 7) shows that the data queue q d is stable. In Equation 6), the gap between the average transmission rate with our algorithm and the optimal average transmission rate can be made arbitrarily small by choosing parameter large. Similarly, by Equation 9), the admission rate can be close to optimum with large. As a tradeoff, the data queue length increases as increases. From Equation 8), we observe that, if we plug the rates allocated by our algorithm in the utility function, it still remains close to the utility achieved by the optimal solution of Problem B). his implies that, AC and C allocate rates smoothly over time, as opposed to the case without a key queue. Based on this observation, combined with Equation 6), we expect the queueing delay to be smaller with a key queue. We will verify this in the following numerical example.. NUMERICAL EXAMPLE In this section we simulate our algorithms and numerically compare them with the optimal performance. In the simulation, the number of time slots is = 0 6. We use the utility function Ux) = log + x) x 0. he main channel rate is uniformly distributed over 0, 00 and the eavesdropper channel rate is uniformly distributed over 0, 50. As a result of the simulation, the average secure capacity R s = 9.. We also set the virtual key queue parameter ɛ = 0.0. We first use an arrival process At), t 0, that is composed of independent Poisson random variables with mean 30 each slot. In this example, Ā > R s. We run the simulation for different values of the control coefficient and compare the results with the optimal value. In Figure a), one can observe that, as increases, the average utility of the transmission rate with a key queue approaches the optimal value, which is consistent with Equation 6). For the case without a key queue, the average utility remains a constant and is quite smaller. Figure b) shows that, as increases, the average admission rate both with and without a key queue) also increases to the optimum, which is consistent with Equation 9). As a tradeoff, the average queue length increases but remains bounded as shown in Figure c). Although the optimal admission rate can be approached both with and without a key queue, the delay performance with a key queue is better as we can see in Figure d). Figure 3 illustrates the same scenario, with a more bursty arrival process. his time, At) = 0 w.p. and At) = 50 w.p. independently for each time slot. Consequently, Ā = 5 < R s. Similar observations to the previous case can be made with this arrival process. Note that the optimal value for Problem A) is U R s) and for Problem B) is minā, R s.

5 5 Average Utility of ransmission Rate Average Queue Length Optimal x a) x 0 4 c) Average Admission Rate Average Queue Length Optimal x b) Average Admission Rate Fig.. Performance Evaluation of C and AC with respect to the solutions of Problem A) and B) under Poisson Arrivals: a) Control Parameter vs. Average Utility of ransmission Rate; b) Control Parameter vs. Average Admission Rate; c) Control Parameter vs. Queueing Delay; d) hroughput vs. Delay Curve Average Utility of ransmission Rate Average Queue Length Optimal x a) x 0 4 c) Average Admission Rate Average Queue Length d) Optimal x b) Average Admission Rate Fig. 3. Performance Evaluation of C and AC for Problem A) and B) under ariable Arrivals I. CONCLUSION In this paper, we considered a single-user secure data communication system and addressed two separate problems, d) both of which involve the maximization of a long-term average utility, defined as a function of the number of secure packets transmitted in each time slot. We proposed a transmission controller and an admission controller based on simple index policies that do not rely on any prior statistical information on the data arrival process. We showed that our controller pair has a provably efficient performance. Also, we illustrated via simulations that the use of a key queue reduces the queuing delay for the data packets, while serving packets that are admitted at the maximum admissible rate. his is due to the fact that, the transmission controller is designed to choose the rate of served packets as uniformly over time as possible. REFERENCES A. Wyner, he Wire-ap Channel, he Bell System echnical Hournal, vol. 54, no. 8, pp , October 975. P. K. Gopala, L. Lai, and H. El Gamal, On the secrecy capacity of fading channels, vol. 54, no. 0, pp , Oct J. Barros and M. R. D. Rodrigues, Secrecy capacity of wireless channels, in Proc. IEEE Int. Symposium Inform. heory, Seattle, WA, July 006, pp D. Gunduz, R. Brown, and H.. Poor, Secret communication with feedback, in Proc. IEEE Intl. Symposium on Information heory and its Applications, Auckland, New Zealand, Dec A. Khisti and G. W. Wornell, Secure transmission with multiple antennas: he MISOME wiretap channel, IEEE rans. Inform. heory, 009, to appear. 6 Y. Liang, H.. Poor, and S. Shamai Shitz), Secure communication over fading channels, IEEE rans. Inform. heory, vol. 54, no. 6, pp , June E. Ekrem and S. Ulukus, he secrecy capacity region of the Gaussian MIMO multi-receiver wiretap channel, Mar. 009, submitted. 8 M. Yuksel, X. Liu, and E. Erkip, A secure communication game with a relay helping the eavesdropper, aormina, Italy, Oct. 009, to appear. 9 O. O. Koyluoglu, C. E. Koksal, and H. E. Gamal, On secrecy capacity scaling in wireless networks, 00, submitted. 0, On the effect of colluding eavesdroppers on secrecy scaling, in Proceedings of European Wireless, EW, Lucca, Italy, 00. S. Goel,. Aggarwal, A. Yener, and A. R. Calderbank, Modeling location uncertainty for eavesdroppers: A secrecy graph approach, Austin, X, June 00. S. asudevan, D. Goeckel, and D. F. owsley, Security-capacity tradeoff in large wireless networks using keyless secrecy, Chicago, IL, September A. Sarkar and M. Haenggi, Secrecy coverage, Pacific Grove, CA, Nov C. E. Koksal and O. Ercetin, Control of wireless networks with secrecy, in Asilomar Conference on Signals, Systems, and Computers, Pacific Grove, CA, Nov C. E. Koksal, O. Ercetin, and Y. Sarikaya, Control of wireless networks with secrecy, CoRR, vol. abs/0.3444, 0. 6 L. assiulas and A. Ephremides, Jointly optimal routing and scheduling in packet ratio networks, IEEE ransactions on Information heory, vol. 38, no., pp , Jan X. Liu, E. K. P. Chong, and N. B. Shroff, A framework for opportunistic scheduling in wireless networks, Computer Networks, vol. 4, no. 4, pp , A. Stolyar, Greedy primal-dual algorithm for dynamic resource allocation in complex networks, Queueing Systems, vol. 54, pp. 03 0, 006, 0.007/s Online. Available: M. Neely, Energy Optimal Control for ime arying Wireless Networks, IEEE ransactions on Information heory, vol. 5, no. 7, pp , July K. Khalil, M. Youssef, O. O. Koyluoglu, and H. El Gamal, On the delay limited secrecy capacity of fading channels, Seoul, Korea, June - July 009. O. Gungor, J. an, C. E. Koksal, H. El Gamal, and N. B. Shroff, Joint power and secret key queue management for delay limited secure communication, San Diego, CA, March 00.

6 6 Z. Mao, C. Koksal, and N. Shroff, Resource Allocation in Sensor Networks with Renewable Energy, in Proc. of the 9th International Conference on Computer Communication Networks, 00. A. Proof of heorem APPENDIX Proof of Equation 7): Equation 7) directly follows from the following lemma: Lemma : Under algorithm AC and C, we have q d t) β, q kt) β. Proof: Since U ) is concave on R + {0}, we have Uµt)) U0)+βµt), t 0, where 0 β = U 0) <. hen, Uµt)) q kt)µt) U0) + β µt) q kt)µt) where µt) is the solution of C. β If µt) q kt)µt) < 0, then we get Uµt)) q k t)µt) < U0). However, C chooses µt) that maximizes Uµt)) q kt)µt) which means Uµt)) q k t)µt) U0) since 0 Πt). hen, we must have β µt) q kt)µt) 0, i.e., q k t)µt) β µt). 0) We now prove the result by induction. Without loss of generality, let q k 0) β. Suppose for all t, q kt ) β holds. In slot t, if µt) = 0, then q k t) q k t ) β by Equation ). Otherwise, µt) 0, and by Equation 0), we have q k t) β. q d t) β can be obtained using the same argument. Proof of Equation 6): We define the Lyapunov function L q k t)) = q k t)), and q k t)) = L q k t+)) L q k t)). From Equation ), we have then qk t + ) ) qk t) ɛ ) + µt) Rs t) + o t) ) + qt) ɛ ) + µt) Rs t) + o t) ) = q k t)) q k t) ) + ɛ + + R max ) + ɛrmax + q k t) o t) + q k t)µt) q k t)r s t), Uµt)) Uµt)) + ɛ + + R max ) + ɛrmax + q k t) o t) + q k t)µt) q k t)r s t) Uµt)) + ɛ + ) + R max + ɛrmax + β o t) Uµt)) q kt)µt) q k t)r s t). It is apparent that C is trying to maximize the value of the term Uµt)) q kt)µt). Since the optimal solution for Problem A) may not be unique, we let U be the optimal solution set and µ U be any optimal solution, for Problem A) given any sample path. Since the constraint set Πt) is queue dynamic related, it is possible that µ t) / Πt). Lemma 3: In slot t, if by solving C, we get Uµt)) q kt)µt) < Uµ t)) q k t)µ t), then µt) < µ t) and o t ) = for some t t and t t <. Proof: In time slot t, let µ m t) be the value that maximize the unconstrained objective function Uµt)) q kt)µt). Claim : q k t) + R s t) R m t). Otherwise, Πt) = 0, R m t) which is not queue dynamic related, then Uµt)) q kt)µt) Uµ t)) q k t)µ t). Claim : µ m t), µ t) > q k t) + R s t). If µ m t), µ t) Πt), we must have Uµt)) q kt)µt) Uµ t)) q k t)µ t), then µ t) > q k t) + R s t). If µ m t) < 0, we will have U0) = Uµt)) q kt)µt) Uµ t)) q k t)µ t), then µ m t) > q k t) + R s t). By the above claims, µt) < µ t) and µt) < µ m t). Suppose µt) < q k t) + R s t), since the objective function of C is concave in µt), we can increase µt) to increase the objective without violating the constraint. hus, µt) = q k t)+r s t) and o t) =. It is also possible that o t ) = for some t < t and R s τ) = 0, τ t, t. Note that t t <, otherwise, R st) = 0. Let N = max{n : for any t 0, R s τ) = 0, τ t, t + n}. By using Lemma 3 and µt), µ t) R m t) R max, t 0, we have N < and Uµt)) Uµ t)) + ɛ + ) + R max + ɛrmax + q k t) µ t) R s t) + β + NUR max )) o t). Lemma 4: q k t)µ t) R s t) O ). Proof: Note that µ t) q k 0) + R s t) 0, then ) µ t) R s t) δ < q k 0), ) where δ can be arbitrarily small. By divided by and taking of both sides, we have ) µ t) R s t) δ < 0. ) Construct an auxiliary queue with the following evolution: + q kt + ) = q kt) R s t) δ) + µ t), then with Equation ), q k t) is strongly stable. By multiplying q k t) for both sides of the inequality q k t + ) q k t) R st) δ + µ t) and rearranging terms, we obtain

7 7 q k t) µ t) R s t) q k t) q k t + ) q k t) + δ. By summing from 0 to, dividing by and taking, we have + + R max + ɛ =O ) + δβ, q k t)µ t) R s t) q kt) q k t ) q k t) ) t= q k ) q k ) q k0) q k 0) q k t) δ q kt) + δ β t= since the average queue length of the auxiliary queue remains stable and is not related to. By letting δ 0, we finish the proof. Lemma 5: If q k t) β, then q kt) <. Proof: First, we provide a rough idea of the proof: by exploring the relations between q k t) and q k t), we notice that as q k t) increases from 0 to at most β, q kt) will hit zero at some slot. Once q k t) becomes zero, C results in µt) = minq k t) + R s t), R m t). Since R s t) R m t), q k t) will either be zero or decrease. We now give the proof details. Without loss of generality, let q k 0) = 0. We have the following cases: i) if µt) R s t), o t) = 0 and q k t) > 0, then q k t + ) q k t) and q k t + ) q k t) q k t) q k t + ), i.e., even if q k t) increases, the increment is no larger than the decrement of q k t); ii) if µt) < R s t), o t) = 0, then if q k t + ) > 0, q k t) q k t+) q k t+) q k t), i.e., the decrement of q k t) is no less than the increment of q k t), else if q k t + ) = 0, it goes to case iv); iii) if o t) =, then q k t + ) = 0 by Equation ) and Equation 3); iv) if q k t) = 0, by Equation 4), C chooses µt) = minq k t) + R s t), R m t), then either q b t + ) = 0, or q k t + ) = q k t) R m t) + R s t) q k t). From the above discussion, we have q k t) β <. Lemma 6: If both the key queue q k t) and virtual key queue q k t) are strongly stable, i.e., qk t) + q k t) ) <, then ot) ɛ. Proof: Using the idea similar to 9, we have the fact that if any queue represented with Qt) is strongly stable, then Q ) = 0. Hence, if q k t) and q k t) are strongly q stable, k ) q = k ) = 0. From Equation ), we have q k t + ) q k t) ɛ + o t) + µt) R s t). Note that q k t + ) = q k t) µt) + R s t). By adding from 0 to, dividing by and taking on both sides, we have q k ) q k 0) lim ɛ + o t) + lim q k 0) q k ). q Since k ) q = lim k 0) = q lim k 0) q = lim k ) = 0, so we get ot) ɛ. By summing from 0 to, dividing by and, taking over Equation ), combined with Lemma, Lemma 5, Lemma 6, and Lemma 4, we get Uµt)) ɛnur max ) + β). Uµ t)) O ) By letting ɛ 0, we obtain Equation 6). Proof of Equation 8) and Equation 9): We define Lq d t)) = q d t)), and q d t)) = Lq d t+)) Lq d t)). By Equation 5), we have = q d t)) URt)) URt)) q dt)rt) + A t) + Rmax q d t)µt) 3) P ) max Rt) R P ) max R s.t. URt)) q d t + ) = q d t) µt)) + + Rt), 0 Rt) At), Rs t) µt) = 0, q d t) <, Lemma 7: P) and P) have different objective functions under the same set of constraints. Let R be the maximizer of P), then it is also the maximizer of P). Proof: Suppose there exists R such R t) < R t),

8 8 then there exists R such that R t) R t) At), t 0 and R t) < R t) Rt). i.e., there are infinitely many slots in which R t) Rt) < 0. Since U ) is strictly concave, UR t)) URt)) < β m R t) Rt)) < 0 if R t) Rt) < 0, where 0 < β m = min{ UR t)) UR t)) R t) R t) : R t) Rt) < 0}. hus, UR t)) < UR t)), which contradicts the fact that R is the maximizer of P). Note that q k t + ) = q k t) + R s t) µt). By Lemma 5, we have the fact that C results into Rs t) µt) = 0, 4) then Problem B) is reduced to P). By Lemma 7, R is also the maximizer of Problem B). Substituting R t) into Equation 3), we obtain URt)) UR t)) + A t) + Rmax + q d t) R t) µt). Note that R t) R s t)) 0 and A t) <. combining with Equation 4) and using the same idea as in Lemma 4, we have q d t)r t) µt) O ). hen, we further obtain URt)) UR t)) O ), where R is the maximizer of P), P) and Problem B). his equation also means that as, R becomes the maximizer of P). By applying Lemma 7 again, we have Rt) R t) as.