Seung Jun Baek 1 and Joon-Sang Park Introduction. example, autonomous vehicles, remote surgery, and automated

Transcription

1 Hindawi Mathematica Probems in Engineering Voume 2017, Artice ID , 15 pages Research Artice Deay-Optima Scheduing for Two-Hop Reay Networks with Randomy Varying Connectivity: Join the Shortest Queue-Longest Connected Queue Poicy Seung Jun Baek 1 and Joon-Sang Park 2 1 Coege of Information and Communications, Korea University, Anam-dong 5-ga, Seongbuk-gu, Seou , Repubic of Korea 2 Department of Computer Engineering, Hongik University, 72-1 Sangsoo-dong, Mapo-gu, Seou , Repubic of Korea Correspondence shoud be addressed to Seung Jun Baek; sbaek@korea.ac.kr Received 1 June 2017; Accepted 30 October 2017; Pubished 21 November 2017 Academic Editor: Haipeng Peng Copyright 2017 Seung Jun Baek and Joon-Sang Park. This is an open access artice distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the origina work is propery cited. We consider a scheduing probem for a two-hop queueing network where the queues have randomy varying connectivity. Customers arrive at the source queue and are ater routed to mutipe reay queues. A reay queue can be served ony if it is in connected state, and the state changes randomy over time. The source queue and reay queues are served in a time-sharing manner; that is, ony one customer can be served at any instant. We propose Join the Shortest Queue-Longest Connected Queue (JSQ-LCQ) poicy as foows: (1) if there exist nonempty reay queues in connected state, serve the ongest queue among them; (2) if there are no reay queues to serve, route a customer from the source queue to the shortest reay queue. For symmetric systems in which the connectivity has symmetric statistics across the reay queues, we show that JSQ-LCQ is strongy optima, that is, minimizes the deay in the stochastic ordering sense. We use stochastic couping and show that the systems under couping exist in two distinct phases, due to dynamic interactions among source and reay queues. By carefu construction of couping in both phases, we estabish the stochastic dominance in deay between JSQ-LCQ and any arbitrary poicy. 1. Introduction We consider a scheduing probem in queueing systems with random connectivity of servers. For exampe, in wireess communication systems, the communication channe may randomy become unavaiabe for data transmissions due to fuctuation of channe quaity over time. To cover areas with poor channe quaity, reay networks have been widey adopted [1 3] in which there exist reay nodes responsibe for reaying data packets in a hop-by-hop manner to destination. In this paper, we investigate the minimum deay scheduing in two-hop reay networks with random connectivity. Deayoptima scheduing in mutihop networks with random connectivity is an open probem and has euded researchers even for very simpe modes. Low-atency communications have recenty attracted much attention in upcoming 5G (5thgeneration) communication networks [4]. There exist many 5G appications which require extremey ow atency, for exampe, autonomous vehices, remote surgery, and automated factories. We consider a queueing mode depicted in Figure 1. The system consists of one source queue (SQ) and n reay queues (RQs). We consider a time-sotted system, where a customer arrives at the SQ with probabiity λ at each time sot. The customers at the SQ are routed to one of the RQs seected bythescheduer.eachrqisassociatedwithaservicestate caed connectivity: the scheduer can serve a RQ ony if the RQ is in connected state. The connectivity of the RQs changes randomy over time. Customers are routed or served in a time-sharing manner; at a given time sot, either a customer is routed from the SQ to a RQ, or a customer is served from a RQ in connected state and exits the system. This is a common mode for reay networks in haf-dupex operation; that is, the queues in adacent hops cannot be served simutaneousy. Potentia engineering appications of our queueing mode incude wireess reay networks. In 5G communication

2 2 Mathematica Probems in Engineering Source queue. Reay queues Haf dupex C 1 =1 C 2 =1 C 3 =0 C n =1 Destination Figure 1: Two-hop queueing network mode. The system consists of one source queue (SQ) and n reay queues (RQs). Ony one customer can be served at any instant (haf dupex). Binary variabe C i represents the connectivity of the ith RQ,,...,n. systems, wireess reays are expected to be widey used to enhance capacity and coverage of the network [5]. In such networks, data packets can be deivered to a mobie user through mutipe reay devices, for exampe, as in device-todevice (D2D) communications [6, 7]. Our mode is appicabe to downink packet transmissions in such next-generation ceuar networks as foows. In the downink scenario, we regard the SQ as the queue ocated at the base station (BS). Aso a RQ is a queue ocated at a reay node (RN), and the customers correspond to data packets. Suppose the BS intends to send a stream of packets to a mobie user, say user i, which is ocated at ce edge with poor channe quaity. Instead of transmitting packets directy to user i, which is ikey to fai most of the time, the BS chooses to transmit packets to one ofthemutiperns.thernstypicayhavebetterchannes to user i and ater transmit the temporariy stored packets to user i. EachchannefromaRNtouseri may undergo fading due to, for exampe, the mobiity of user i.asaresut, the channe between a RQ and user i may randomy switch between on and off state; in our mode, this can be viewed as the RQs being randomy connected to user i over time. A reated technique of utiizing mutipe reays over timevarying channes, caed cooperative reaying or opportunistic reaying, has been studied extensivey [8 13]. Meanwhie, our mode is appicabe to the upink transmission for ceuar networks as we. In the upink scenario, a user node becomes the source queue and utiizes mutipe reay nodes equipped withqueues.thepacketsareeventuayreayedtothebase station, which is the destination node, by the reay nodes. The time-sharing service of customers between SQ and RQs is anaogous to haf-dupex transmissions of packets in wireess reay networks, that is, ony either BS or RN can transmit at a time sot. Whie fu-dupex reays are recenty under investigation [14, 15], haf-dupex reays are sti widey used [14] since they are simpe to design and cost-effective. In addition, the connectivity from the SQ to RQs is assumed to be aways in connected state in our mode. A common architectureforceuarnetworkswithreaysproposesusing nodes dedicated for reaying caed Reay Stations (RSs). The RSs are typicay instaed in fixed ocations which are in ineof-sight (LOS) to the BS [16]. Therefore, the channe quaity between the BS and a RS is typicay very high. Aso, in D2D networks in which mobie nodes are seected to act as RNs, it makes sense to seect mobies which have good channes to thebsinthefirstpace.suchahighquaitychannebetween the BS and RSs can be modeed as being aways in on state, which is assumed in our mode. We introduce a poicy caed Join the Shortest Queue- Longest Connected Queue (JSQ-LCQ). Definition 1. The JSQ-LCQ poicy is defined as foows: (1) If there exist RQs which are nonempty and connected, serve a customer from the ongest queue among the connected RQs. (2) If there is no RQ to serve, route a customer from the SQ to the shortest queueamongtherqs. It is observed that the JSQ-LCQ is a simpe and greedy poicy with tehe foowing properties: (i) JSQ-LCQ prioritizes serving the RQs over serving the SQ.IfthereisanychancetoserveaRQ,itwidoso. (ii) Among the connected RQs, it wi serve the ongest one. This is an attempt to make the RQs as baanced as possibe. (iii) When JSQ-LCQ routes a customer to RQ, it chooses theshortestrq.againthepoicyattemptstobaance the RQs. JSQ-LCQ focuses on baancing queues during the service and routing so as to maximize the opportunism of time-varying connectivity, that is, so that as many nonempty queues as possibe can observe the connected state. LCQ is inspired bythepoicyin[17]withthesamename.inthesingehop case where the queues are served in parae, the LCQ poicy is deay-optima for the symmetric system; that is, connectivity and arriva processes have identica statistics across the queues [17]. Meanwhie, in a two-hop network, if a the queues are in connected state and are served in parae without time-sharing constraints, it is we-known that the JSQ poicy is deay-optima [18]. However, two-hop networks with the time-sharing constraint and random connectivity are considered in our probem. Two-hop networks are very different from one-hop networks, since the customers served in the first hop do not eave but stay in the system, waiting to be served; moreover, the service avaiabiity randomy changes over time. The question is as foows: does there exist a deay-optima poicy for such two-hop networks? If so, can we find the optima poicy? In this paper, we answer these questions in the affirmative as foows. Assume a symmetric system; that is, the connectivity has a symmetric distribution across the RQs. Our caim is that JSQ-LCQ is optima in a strong sense, that is, in the stochastic ordering sense. In this paper, we estabish the deay optimaity of a twohop network mode with time-varying connectivity. We show

3 Mathematica Probems in Engineering 3 that JSQ-LCQ poicy is strongy optima, such that it minimizes the number of customers in the system in the stochastic ordering sense. We use the couping argument to show that the queue ength processes under JSQ-LCQ is stochasticay dominated by any other feasibe poicy. However, unike typica couping, we show that the couped systems can exist in two distinct phases. Such a system behaviour is attributabe to dynamic interactions among random connectivity, queue states, and the time-sharing constraint of service. The system phases are characterized in terms of certain reations between the queue states of JSQ-LCQ and another arbitrary poicy in comparison. Specificay, the phases are defined based on (i) the difference in the source queue engths and (ii) the weak maorization reations between the vectors of reay queue engths, of the compared poicies. We carefuy deveop the couping argument for these phases, which eads to the stochastic dominance of the tota number of customers in the system. To our knowedge, deay optimaity for two-hop reay networks, even for simpe channe modes, has not been we-known yet. Considering that there exist few works on deay-optima scheduing, we beieve that our work opens a new possibiity for deeper understanding of the probem. In summary, the key contributions of our work are isted as foows: this paper (i) proposes JSQ-LCQ and proves the deay optimaity of the agorithm for two-hop reay networks with timevarying connectivity, which is of theoretica significance; (ii) introduces a nove couping technique associated with the transition of system phases defined in terms of the maorization reations among source and reay queues. This paper is organized as foows. We present reated worksinsection2.insection3,wedescribethesystem mode. The optimaity of JSQ-LCQ is proved in Section 4. Simuation resuts are reported in Section 5. Section 6 concudes the paper. 2. Reated Work Deay-optima scheduing is not ony important from an engineering perspective but aso of theoretica and mathematica interest. Deay optimaity is notoriousy hard to achieve with time-varying service capacity, and there exist ony a few resuts which we review beow. In their semina work [17], Tassiuas and Ephremides considered a singe-hop scheduing of parae queues with time-varying connectivity, that is, with on-off channes. They proposed LCQ and showed that, under the symmetry assumptions, LCQ is deayoptima in the stochastic ordering sense. Yeh and Cohen [19] considered singe-hop scheduing over mutiaccess fading channes when the capacity region of the users service has a poymatroid structure [20]. They proposed Longest Queue Highest Possibe Rate (LQHPR) poicy which successivey aocates higher rates to onger queues. LQHPR is shown to be deay-optima when the arriva statistics and capacity region are symmetric across users. For mutihop case, authors of [21] consider scheduing tandem queues with interference constraints; that is, the adacent queues cannot be served simutaneousy. In their mode, queues are aways in connected state. The deay-optima poicy is obtained by serving the nonempty queue cosest to the destination and then serving the next nonempty and noninterfering queue cosest to the destination, which is iterativey done over the entire network. Interestingy, we observe that JSQ-LCQ poicy has a simiar principe to the poicy in [21]: JSQ-LCQ prioritizes serving the RQs ( coser to the destination) whenever possibe, and the SQ has the ower priority. However, it is highy nontrivia to extend the optimaity resut to networks with time-varying connectivity ike our mode. Recenty, Cui et a.[22]studiedatwo-hopnetworkwith on-off channes. However, there exists ony one reay queue in their mode, and the authors propose a poicy which is asymptoticay optima using a dynamic programming (DP) approach. We observe that deay-optima scheduing has been found ony for simpe modes, for exampe, singe-hop network with symmetric service capacity. To our knowedge, even with symmetric connectivity, there is no known deay-optima scheduing for two-hop reay networks invoving mutipe reay queues under the time-sharing (haf-dupex) constraint. Our scheme can be regarded as a reay seection and scheduing scheme for two-hop cooperative reay networks, that is, a cooperative transmission utiizing mutipe RNs, for exampe, [8 10, 23, 24]. Betsas et a. [8] consider a reay seection agorithm for two-hop cooperative reay networks where there exist mutipe RNs for a source-destination pair. The best reay is chosen based on either the minimum or harmonic mean of the instantaneous channe gains of sourcereay (S-R) and reay-destination (R-D) inks. A time sot is divided by haf, and S-R transmission occurs at the first haf of the time sot, and R-D transmission occurs at the second haf. Cui et a. [9] considered two-hop reay networks with mutipe source, reay, and destination nodes. Their scheme seects RNs in an opportunistic manner, that is, based on favorabe Signa-to-Noise-Ratio (SNR) among mutipe S-R and R-D pairs. In [23], the authors considered a mode in which the RNs have buffers where the transmission occurs over two phases (time sots) as foows. In the first time sot, a S-R pair with the best channe is schedued, and the transmitted packet is stored in the seected reay for ater transmission. In the second time sot, the best R-D pair is schedued in which the RN transmits previousy stored data. Note these works focus on maximizing transmission rates or throughput; however they do not consider the deay issue. For exampe, the queues are assumed to be infinitey backogged in the aforementioned works. Aso the inks are schedued in a fixed manner; for exampe, the seected S-R and R-D pairs are schedued to transmit over two consecutive time sots. In contrast, our work not ony considers reay seection but aso ink scheduing. For exampe, when our scheme seects the shortest RQ to route (JSQ), it can be regarded as seecting best reay. In other words, by baancing the RQs, the poicy wi enhance opportunism by making as many nonempty RQs as possibe observe the connected state. In a genera approach to ensure deay optimaity for mutihop cooperative networks, one needs a probem formuation via Markov

4 4 Mathematica Probems in Engineering Decision Process (MDP), for exampe, [25, 26]. To achieve deay optimaity, one requires soving infinite time horizon MDP. However, it is difficut to appy MDP-based poicies to arge systems due to curse of dimensionaity. Wang et a. [11] considered queue-based cooperative reaying by approximatey soving MDP using a stochastic earning approach. Theauthorsproposedadistributedonineagorithmwhich isshowntobeasymptoticayoptimaundertheheavy-traffic imit. In contrast to deay optimaity, throughput optima poicies are reativey we-known; a poicy is said to be throughput optima if the poicy makes a queueing system stabe whenever stabiity is feasibe. Tassiuas and Ephremides [27] showed that the routing and scheduing under backpressure agorithm based on backog differentias are throughput optima for mutihop networks with ink constraints. However, backpressure agorithm ony ensures throughput optimaity but does not provide any guarantee on achievabe deay. A number of enhancements to backpressure agorithm have been proposed to address the issue of deay performance [28 34]. Backpressure agorithm often suffers from ong deays in arge mutihop networks. This is because the agorithm expores a possibe paths from source to destination. The routing based soey on backpressure may create a ong path resuting in arge deays. To aeviate this probem, in [31, 32] an agorithm is proposed which adaptivey expoits short paths, whie maintaining the stabiizing property of backpressure agorithm. Specificay, the agorithm uses shortest paths under the ight traffic but utiizes onger paths with increasing traffic to ensure stabiity. Practica impementations of backpressure agorithm are proposed in [35 37]. Note that the aforementioned works considered throughput optima schemes; however, throughput optimaity is a reativey weak form of performance as compared to deay optimaity. 3. System Mode Consider a time-sotted system consisting of one SQ and n RQs. Ony one customer can be served at a time sot: either a customer is routed from the SQ to one of the RQs, or a customer is served at one of the RQs. A customer can beservedfromarqonyiftherqisinconnectedstate. The service at the RQs has randomy varying connectivity. ARQisconnectedwithprobabiityp, and the connectivity is independent over time sots and across the RQs (simiar to [17], we do not need the independence of connectivity across the RQs. We ony need the assumption that the oint distribution of connectivity is symmetric across the RQs, under which our optimaity resut wi hod as we. However, the independence assumption wi simpify the arguments on, e.g., stabiity and stochastic couping, which we discuss ater). ThearrivaofacustomerattheSQisi.i.d.overtimesotswith probabiity λ. The number of customers at SQ (resp., RQs) at time t is denoted by X(t) Z + (resp., Y(t) = (Y 1 (t),...,y n (t)) Z n + ). Let C(t) = (C 1(t),...,C n (t)) {0, 1} n be a random process indicating the connectivity at the RQs. Specificay, C 1 (t),...,c n (t) are i.i.d. Bernoui random variabes with parameter p. Denote the arriva process by A(t) {0, 1}. The state of the system at time t is denoted by S(t) f (A(t), C(t), X(t), Y(t)). We define the set of actions which a poicy can take at a given timesot. An action takes a vaue from the set of symbos defined by A f {I, R 1,R 2,...,R n,s 1,S 2,...,S n }.SymboI stands for the poicy being ide, R i standsforroutingacustomerfromthesq to the ith RQ, and S i represents serving a customer from the ith RQ. Poicy π(t) is defined as a scheduing decision at time t which takes a vaue from A. Notethatπ(t) is based on the entire history of scheduing actions ({π(t), s < t}) and system states ({S(t), s t}). The SQ and RQs evove as foows: X (t+1) =[X(t) 1 (π (t) {R 1,...,R n })] + +A(t), Y i (t+1) =[Y i (t) 1 (π (t) =S i ) C i (t)] (π (t) =R i ) for i = 1,...,n,where1( ) is the indicator function and [x] + f max(x, 0). Next we consider the condition for stabiity. The arriva rate to the system is λ. The service rate from the SQ to RQs is onepertimesot,whereastheoveraservicerateattherqs is 1 (1 p) n on average. Hence the utiization at the first and second hops is given by λ and λ/{1 (1 p) n },respectivey. Due to the time-sharing constraint, the combined utiization must be ess than 1; that is, λ 1>λ+ 1 (1 p) n λ< 1 (1 p)n 2 (1 p) n. It is known that throughput optima poicies such as backpressure agorithm [27] can stabiize the system under condition (2). Later, we wi show that JSQ-LCQ is deay optima; that is, the average number of customers in the system under JSQ-LCQ is no more than that under any other poicy incuding backpressure agorithm. Thus, JSQ-LCQ is a stabe poicy under condition (2); note that deay optimaity impies throughput optimaity. Due to the memoryess property of connectivity, it suffices to consider π(t) which is a static statefeedback poicy; that is, there exists optima π(t) which bases its decision ony on the present state of the system. Under such π(t), queue state process (X(t), Y(t)), t = 0, 1, 2,..., forms a Markov chain; if π(t) is throughput optima, the Markov chain is stationary and ergodic under stabiity condition (2). 4. Deay Optimaity of JSQ-LCQ In this section, we wi prove the deay optimaity of JSQ- LCQ. We wi show that JSQ-LCQ is optima in the stochastic ordering sense which we define as foows. For two random variabes A and B in R, eta Bdenotethattheyhavethe identica distribution. Definition 2 (see [38]). Let U and V be random variabes in R. U is said to be stochasticay smaer than V, denoted by U st V, if there exist random variabes U and V such that (1) (2)

5 Mathematica Probems in Engineering 5 (1) U U ; (2) V V ; (3) U V a.s. Note that U st V is equivaent to stating that, for any increasing function f:r R, P (f (U) >z) P (f (V) >z), t 0. (3) For vector x R n,et x f n x i. We state the main theorem as foows. Theorem 3. Let X(t) and Y(t) denote the queue ength processes of the SQ and RQs under JSQ-LCQ. Aso et X(t) and Ỹ(t) denote the ength of the SQ and RQs under an arbitrary poicy. Suppose (X(t), Y(t)) and ( X(t), Ỹ(t)) areinanarbitrary initia state (x 0, y 0 ) at time t=0.then,fort 0, X (t) + Y (t) st X (t) + Ỹ (t). (4) Theorem 3 states that the number of customers in the system under JSQ-LCQ is stochasticay smaer than that under any other poicy. By Litte s aw, the theorem impies that JSQ-LCQ wi minimize the deay in the stochastic ordering sense. Theorem 3 is a much stronger statement than, for exampe, achieving the minimum average deay, as we can see from (3). To prove Theorem 3, we wi use stochastic couping arguments everaging the forward induction technique [18]. Specificay, we show that one can construct the sampe paths of queue ength processes under JSQ-LCQ and another arbitrary poicy by propery couping the arriva and connectivity processes so that (4) hods Couping. Let π(t) denote the JSQ-LCQ poicy. Let π(t) denote another arbitrary poicy. (X(t), Y(t)) (resp., ( X(t), Ỹ(t)) in Theorem 3 denote the queue ength processes under π(t) (resp., π(t)). In the foowing we use forward induction [18, Section 8.3] by couping the connectivity of the RQs under π(t) and π(t) as foows. Suppose the process of RQs under π(t),or Y(t), has the connectivity given by C(t) = (c 1,...,c n ) at time t.wewicoupec(t) with the connectivity variabes for Ỹ(t) as foows: if the ith ongest queue of Y(t) has the connectivity c {0,1},thenettheith ongest queue of Ỹ(t) havethesameconnectivityc,for,...,n.inother words, Y [i] (t) and Y [i] (t) see the same connectivity variabe by the couping for i = 1,...,n. This couping wi not change the margina distributions of the connectivity seen by Y(t) and Ỹ(t) as is required in (1) and (2) of Definition 2 (see aso [18, Proposition 8.3.2]), because this couping invoves simpy permuting the connectivity variabes across the RQs. Specificay, the connectivity variabes are i.i.d. across the RQs and thus their oint distribution is symmetric or invariant to permutation; that is, P (C 1 (t) =c 1,...,C n (t) =c n ) = P (C 1 (t) =c P(1),...,C n (t) =c P(n) ), (5) where P is an arbitrary permutation of the index set {1,2,...,n}. Next, the arrivas to the system are couped as foows: if an arriva occurs at the SQ under π(t), then et there be an arriva at the SQ under π(t). In the rest of the proof, we wi assume that the queue ength processes under π and π are couped in the above fashion. Unike previous works on singe-hop scheduing, the couping argument in our probem must consider dynamic interactions among queues, connectivity, and the haf-dupex constraint, as foows. JSQ-LCQ prioritizes serving the RQs; that is, it wi serve the RQs whenever possibe, in a baanced manner. Thus, the RQs wi tend to be short under JSQ-LCQ. This means that, due to haf-dupex operation, the SQ wi get reativey ong. However, if many RQs become empty due to prioritized service, the number of nonempty and connected RQswibecomesma.HenceJSQ-LCQmaybeforcedto frequenty route customers to the RQs, in a baanced manner, in which case the RQs wi buid up. However, as the number of nonempty RQs grows, there wi be many nonempty and connected RQs, and JSQ-LCQ wi again begin to activey serve the RQs. Thus, we observe that the system exhibits some cycic patterns in the services and evoution of the queue states. Based on this observation, we identify that there exist two distinct phases in our couping process. In the first phase, the RQs tend to be short and the SQ tends to be ong under JSQ- LCQ. In the second phase, the SQ tends to be short and RQs tend to be ong. The first phase is caed weak maorization (WM) phase, and the second phase is caed water-fiing maorization (WFM)phase.Wewiexpainthephasesin more detai in the subsequent sections. We show that, by introducing the concept of phases, we are abe to hande the aforementioned patterns in system behaviour. Specificay, under proper couping, we show that the system remains in either of the two phases or makes the transition to the other phase.laterwewishowthatthisconstructionimpiesthe desired stochastic maorization given by (4) Weak Maorization (WM) Phase. For vector x R n,et x [k] denote the kth ongest entry of x;thatis,x [1] x [n]. Definition 4. For two vectors x, y R n,if k=1 x [k] k=1 y [k], =1,...,n, (6) x is said to be weaky maorized by y, which is denoted as x w y. Reca that (X(t), Y(t)) (resp., ( X(t), Ỹ(t))) isthequeue ength processes under JSQ-LCQ or π (resp., an arbitrary poicy or π). Definition 5. We say the system is in weak maorization (WM) phase if (1) the foowing reation hods: 0 X(t) X (t) 1 2 { Ỹ (t) Y (t) }; (7)

6 6 Mathematica Probems in Engineering equivaenty, there exists integer δ 0such that X (t) X (t) =δ, Ỹ (t) Y (t) 2δ; (8) (JSQ-LCQ) X(t) Y(t) (2) Y(t) is weaky maorized by Ỹ(t);that is, Source queue Y (t) w Ỹ (t). (9) Reay queues We discuss the impication of the WM phase. Firsty, JSQ- LCQ wi greediy serve the RQs whenever possibe, making the overa ength of RQs sma. By contrast, the customers that arrived at the SQ wi have to wait reativey ong due to the haf-dupex constraint. Condition (8) represents these properties; that is, the SQ under JSQ-LCQ is reativey ong, and the sum-ength of the RQs is reativey sma. In addition, if we rearrange the inequaity on the right of condition (8), Source queue X(t) Y(t) Y (t) +δ Ỹ (t) δ, (10) which is interpreted as foows. There are an excess of δ customers backogged at the SQ under JSQ-LCQ. Thus, if we compare ony the SQs, JSQ-LCQ appears to be δ customers behind π. Now suppose JSQ-LCQ adds these δ customers to Y(t) by serving the SQ for δ times to catch up π,which takes δ time sots. During this time interva, π can serve δ customers from the RQs to push the customers out of the system as much as possibe, in which case Ỹ(t) is reduced by δ.however,(10)showsthat,evenaftersuchactionsby π,the number of customers under JSQ-LCQ sti is no more than that under π. Thus, condition (8) indicates that JSQ-LCQ is in fact sufficienty ahead of π in WM phase. Secondy, not ony wi the RQs be short, but aso they are we baanced due to JSQ-routing and LCQ-scheduing principes. Condition (9) represents this property. An exampe of the system in WM phase is depicted in Figure 2. However,thesystemmaygetoutofWMphaseifmostof the RQs are emptied out, after which JSQ-LCQ wi mainy serve the SQ. Consequenty, the RQs wi become reativey ong but the SQ wi become reativey short, in contrast to WM phase. In that case, the system makes the transition to WFM phase, which we wi define and discuss in detai in Section 4.3. To use forward induction we wi show that if the system is in WM phase at time t, under a proper couping of connectivity and arrivas, the system wi either remain in WM phase or make the transition to WFM phase at time t+1.later,wewimakeasimiarcoupingargumentfor the system in WFM phase: a system in WFM phase at time t either stays in WFM phase or makes the transition to WM phase at time t+1. This impies that the same argument hods for a time s such that s tdue to forward induction; that is, the aforementioned reation between the queue states wi propagate over time through couping [18, 21]. Lemma 6. Consider the queue ength processes defined in Theorem 3. There exists couping between (X(t), Y(t)) and ( X(t), Ỹ(t)) such that if the system is in WM phase at time Y(t) w Y(t) Reay queues Figure 2: Exampe of the system in weak maorization (WM) phase. The figure in the above (resp., beow) shows the queues under poicy JSQ-LCQ or π (resp., π). The SQ under π is onger than that under π; that is, X(t) = 2 and X(t) = 1. In addition, the number of customers in the RQs satisfies the foowing: we have Ỹ(t) Y(t) = 5 3 = 2 2{X(t) X(t)}, which satisfies (8). Next, we have Y(t) w Ỹ(t) satisfying (9); that is, the RQs under JSQ-LCQ are better baanced than those under π. t 0, either the system remains in WM phase or it makes the transition to WFM phase at time t+1. Proof. Initiay, at t=0, the queue states are identica to initia condition (x 0, y 0 ). By definition, the system is in WM phase at time 0 because (8)-(9) are satisfied when the queue states are identica. Now consider the system at time t wherewemakethe induction hypothesis; that is, the system is in WM phase at time t. Once the connectivity and queue states are couped, π(t) and π(t) may take different actions from A.Forthesake of simpicity, we wi define new symbos for actions denoted by S, R,andI with some abuse of notation: (i) π(t) = S denotes that a service has occurred at one of therqs(aprecisenotationwibeπ(t) {S 1,...,S n }) under π at time t. (ii) π(t) = R denotes that a routing from the SQ to one of the RQs (a precise notation wi be π(t) {R 1,...,R n }) has occurred under π at time t. (iii) π(t) = I denotes that the poicy ides at time t. For instance, (π(t), π(t)) = (S, R) denotes the event that π served a RQ and π routed a customer from the SQ to a RQ. We wi consider a tota of 9 cases, since (π(t), π(t)) can possiby take 9 action pairs. Reca that, in a cases, we wi use the couping introduced in Section 4.1; that is, Y [i] (t) and Y [i] (t)

7 Mathematica Probems in Engineering 7 see the same connectivity variabe for i = 1,...,n.AsoX(t) and X(t) see the same arriva variabe. Case 1 ((π(t), π(t)) = (S, S)). In this case, a service has occurred under both poicies. Suppose the service has occurred at th ongest RQ under π. Aso suppose that the kth ongest RQ was served under π.sinceπ uses LCQ, the served RQ was the ongest among the connected queues. This impies that the connectivity variabes for Y [i] (t),,..., 1,must be 0. From the construction of couping, this impies that the connectivity of Y [i] (t) is aso 0 for,..., 1.Thus,k musthod.wehavethat,forany=1,...,n, [ Y [i] (t+1)] [ =[ 1 ( k) + [ 1 ( )+ Y [i] (t+1)] Y [i] (t)] Y [i] (t)] = 1 ( ) 1 ( k) +[ [ Y [i] (t)] 0 Y [i] (t)] (11) due to k and induction hypothesis Y(t) w Ỹ(t). Thus, we have Y(t + 1) w Ỹ(t + 1); thatis,(9)hodsattimet+1. In addition, (8) is satisfied at time t+1,becausex(t) and X(t) did not change, and Ỹ(t) Y(t) remains unchanged since Y(t + 1) = Y(t) 1 and Ỹ(t + 1) = Ỹ(t) 1. Consequenty, the system is in WM phase at time t+1. Case 2 ((π(t), π(t)) = (R, R)). Both poicies routed a customer to a RQ; thus (8) ceary hods at time t+1.next,suppose π has routed a customer to the kth ongest RQ, whereas π has routed a customer to the shortest RQ. We have that, for any =1,...,n, [ Y [i] (t+1)] [ =[1 ( k) + Y [i] (t+1)] Y [i] (t)] which hods due to k nand the induction hypothesis. Thus, we have that Y(t+1) w Ỹ(t+1);thatis,(9)hodsattimet+1. Thus the system is in WM phase at time t+1. Case 3 ((π(t), π(t)) = (I, I)). Since there is no change in the queue states under both poicies, WM phase is maintained at time t+1. Case 4 ((π(t), π(t)) = (S, R)). In this case, π serves a RQ, and π routes a customer to a RQ. Since π serves a RQ, we have that Y(t + 1) w Y(t). Since π routes a customer to a RQ, Ỹ(t) w Ỹ(t + 1) hods. By induction hypothesis (9), this impies that Y(t+1) w Ỹ(t+1);thatis,(9)hodsattimet+1. Suppose the service occurred at queue under π,andthe routing occurred at queue k under π. Asoitisimpiedthat X(t) > 0 inthiscase.wehavethat X (t+1) =X(t), X (t+1) = X (t) 1, Y (t+1) =Y (t) 1, Y k (t+1) = Y k (t) +1. Let δ=x(t) X(t) 0.From(13),wehavethat (13) (14) X (t+1) X (t+1) =X(t) X (t) +1=δ+1. (15) Aso Ỹ(t) Y(t) 2δ impies that, from (14), Ỹ (t+1) Y (t+1) = Ỹ (t) Y (t) +2 2(δ+1). (16) Thus, (8) is satisfied at time t+1. Consequenty, the system is in WM phase at time t+1. Case 5 ((π(t), π(t)) = (R, I)).Thisisthecasewhereπ serves the SQ and π ides. We consider two cases. Case 5.1 (X(t) > X(t)). In this case, at time t+1we have that X (t+1) =X(t) 1, X (t+1) = X (t), Y (t+1) = Y (t) +1, Ỹ (t+1) = Ỹ (t). (17) [1 ( =n) + Y [i] (t)] = 1 ( k) 1 ( =n) +[ [ Y [i] (t)] 0, Y [i] (t)] (12) Let δ f X(t) X(t) > 0.Then Ỹ (t+1) Y (t+1) =2δ 1 2(δ 1) (18) hods, and X(t + 1) X(t + 1) = δ 1. Thus (8) is satisfied at time t+1. Next we check if weak maorization (9) hods at time t+1. Let us define y f Y [n] (t). Suppose that there were k more queues which have had ength y at time t.

8 8 Mathematica Probems in Engineering Firsty consider the case where k=0. This impies that the RQ to which the customer is routed is sti the shortest queue at time t+1under π. Sinceπ performs JSQ, Y [n] (t) is incremented by 1. Thus we have that However, since Y [i] (t) = y for y>k,wehavethat n { Y [i] (t)} 2δ+(n k )y>(n k )y. (25) i=k +1 Y [n] (t+1) =Y [n] (t) +1, Y [i] (t+1) =Y [i] (t),,...,n 1. (19) However since Y [k ](t) = y,wemusthavethat Y [i] (t) y,for i>k, and thus (25) cannot hod, yieding a contradiction. Thus, we have that k <n k, which impies that We wi show that Y [i] (t+1) Y [i] (t+1) (20) hods for =1,...,n.Ceary,(20)hodsfor1 n 1.For =n,considerthefoowing.sincex(t) and X(t) differ by at east one customer, Y(t) and Ỹ(t) differ by at east two customers by induction hypothesis (8). This impies that n Y [i] (t+1) =1+ n Y [i] (t) 1+ Ỹ (t). (21) Since Ỹ(t + 1) = Ỹ(t), weconcudethat(20)hodsfor =n.consequenty,y(t + 1) w Ỹ(t + 1); thatis,theweak maorization is maintained at time t+1. Secondy, consider the case where k>0. At time t+1, the RQ which received a customer under π becomes (n k)th ongest queue, or, equivaenty, we are incrementing (n k)th ongest queue at time t (which was of ength y)byone.thus, we have that Y [n k] (t + 1) = y + 1 and Y [i] (t + 1) = Y [i] (t) = y for i n k+1.wewishowthat Y [i] (t+1) Y [i] (t+1), (22) for =n k, which impies Y(t+1) w Ỹ(t+1);thisisbecause (22) aready hods for <n kdue to induction hypothesis of weak maorization at time t; and if (22) hods for =n k, it wi do so for >n kby construction. Define k = max { Y [i] (t) = Y [i] (t)}. (23) Here we impicity assume the set in the RHS of (23) is nonempty; otherwise Y(t + 1) w Ỹ(t + 1) hods and we are done. We have that k =n,becausek>0. We wi show that k < n k hods by contradiction. Suppose k n k.thenwemusthavethaty [k ](t) = Y [k ](t) = y.sincethereareateast2δ > 0 more customers in Ỹ(t),wehavethat n { Y [i] (t) Y [i] (t)} 2δ. (24) i=k +1 n k Y [i] (t) < n k Y [i] (t). (26) Since π routes a customer to the (n k)th ongest RQ, (26) impies that (22) hods for = n k.thus,theweak maorization hypothesis continues to hod for time t+1. Case 5.2. (X(t) = X(t)). In this case, we have X(t + 1) < X(t + 1). Thus, condition (8) ceases to hod, and the system makes the transition to WFM phase. We wi discuss WFM phase in detai and prove the transition of phases in the next section. Case 6 ((π(t), π(t)) = (I, S)). This is the case where π ides and π serves a RQ. Since π is ide, we must have X(t) = 0,which in turn impies X(t) = 0 due to induction hypothesis (8). Suppose the service has occurred on th ongest RQ under π. WemusthaveY [] (t) = 0; otherwiseπ woud have served the th ongest RQ instead of iding, because the th ongest RQ is in connected state due to couping. Given Y [] (t) = 0, we cannot have Y [i](t) = Y [i] (t),because 1 Y [i] (t) = > 1 1 Y [i] (t) = Y [] (t) + Y [i] (t), Y [i] (t) (27) which wi vioate induction hypothesis (9). Thus, we have that This impies that Y [i] (t) < Y [i] (t+1) Y [i] (t). (28) Y [i] (t+1). (29) Combining this with the induction hypothesis (9), we concude that Y(t + 1) w Ỹ(t + 1). Case 7 ((π(t), π(t)) = (S, I)). This is the case where π serves a RQ and π ides. Ceary (8) hods at time t+1,becausethesq remains unchanged. Since one of the RQs was served under π,wecearyhavethat Y (t+1) w Y (t) w Ỹ (t) = Ỹ (t+1) ; (30) that is, (9) hods at time t+1.

9 Mathematica Probems in Engineering 9 Case 8 ((π(t), π(t)) = (I, R)). This case cannot happen, becauseifitweretohappen,wemusthavex(t) = 0 because π is work-conserving. This in turn woud impy X(t) = 0 by the induction hypothesis. Case 9 ((π(t), π(t)) = (R, S)). In this case, π serves the SQ and π serves a RQ. Suppose the service has occurred at the th ongest queue of Ỹ(t). π wiroutethecustomertotheshortest queue in Y(t),orY [n] (t). Since a departure occurred at the th ongest queue of Ỹ(t) but did not occur at Y(t),wemusthave Y [k] (t) = 0 for k.thus,thecustomerisroutedtooneof the empty RQs under π, which eads to Y [] (t + 1) 1. We wi consider two cases: X(t) = X(t) and X(t) > X(t). Case 9.1 (X(t) > X(t)). This case impies that X(t) and X(t) differ by at east one customer. From (8), this impies that there are at east two more customers in Ỹ(t).Theth ongest RQ under π has been incremented by one where π has been decremented. Firsty, consider the case Y [] (t) 2.Sincetheth ongest RQ was served under π, this impies that Y [] (t + 1) 1. However, since we have Y [] (t + 1) 1, weak maorization Y(t+1) w Ỹ(t+1) wi ceary hod due to induction hypothesis (9) at time t. Secondy, consider the case Y [] (t) = 1. Supposethere were more than one queue with ength 1 under π at time t. Thenwehavethat Y [] (t+1) = 1 Y [] (t+1).sincey [k] (t) = 0 for k, this impies Y(t + 1) w Ỹ(t + 1),andwearedone. Now suppose that the th ongest RQ under π was the ony RQ with ength one. This impies that Y (t) = Y [i] (t), Ỹ (t) = Y [i] (t). After the poicies took the actions, we have that Y [i] (t+1) = 1+ Y [i] (t+1) =1+ Y [i] (t), Y [i] (t). (31) (32) By induction hypothesis, there have been at east 2δ > 0 more customers for Ỹ(t) than those for Y(t). Thus if we combine (31)-(32) we have that { Y [i] (t+1) Y [i] (t+1)} = 2+ { Y [i] (t) Y [i] (t)} 2δ 2 0. (33) Accordingy, we caim that the foowing hods for a k : k Y [i] (t+1) k Y [i] (t+1) (34) since Y [k] (t + 1) = Y [k] (t) and Y [k] (t + 1) = Y [k] (t) for a k due to (33) and the induction hypothesis. If =n,itimpies that Y(t + 1) w Ỹ(t + 1).If<n,sinceY [k] (t) = Y [k] (t + 1) = 0 for k>,weseethaty(t + 1) w Ỹ(t + 1) hodsaswe. Next we examine if (8) hods at time t+1. By assumption we have X(t) X(t)=δ>0,andthusX(t + 1) X(t + 1) = δ 1 0.Aso Ỹ (t+1) Y (t+1) = Ỹ (t) Y (t) 2 2δ 2=2(δ 1). (35) Thus, (8) hods at time t+1.inconcusion,thesystemremains in WM phase at time t+1. Case 9.2 (X(t) = X(t)). In this case, the system makes the transition to WFM phase which we introduce in the next section. The phase transition wi be proved ater as we Water-Fiing Maorization (WFM) Phase. We have the foowing definition. Definition 7. Consider x and x in Z + and n-dimensiona vectors y and ỹ in Z n +.Wesay(x, y) is water-fiing maorized by ( x, ỹ) denoted by if the foowing hods: (x, y) wf w ( x, ỹ) (36) (1) x< x. (2) Let ξ denote the difference between x and x; thatis, ξ = x x.letỹ Z n + be a vector formed by adding ξ to the entries of ỹ ina water-fiing manner. Specificay, et (ξ) be a number that satisfies ξ= n =1 [ (ξ) y [] ] +. (37) Let us add [(ξ) y [] ] + to y [] for =1,...,n,andet ỹ denotetheresutingvector.thenwehavethat y w ỹ. (38) We say that the system is in WFM phase if, at time t, In other words, we have (X (t), Y (t)) wf w ( X (t), Ỹ (t)). (39) X (t) < X (t), (40) Y (t) w Ỹ (t), (41)

10 10 Mathematica Probems in Engineering (JSQ-LCQ) SQ SQ RQ RQ Y(t) Water-fiing routing (JSQ-LCQ) X(t) Y(t) X(t) Y(t) X(t) Y(t) w Y(t) SQ SQ X(t) Y(t) w Y (t) RQ RQ Y (t) Figure 3: Exampe of the system in water-fiing maorization (WFM) phase. The SQ under π has an excess of ξ f X(t) X(t) = 2 customers compared to that under π, which satisfies (40). The RQs at time t under JSQ-LCQ are not necessariy better baanced than those under π;weobservethaty(t) wỹ(t),ifwecomparethesystem in the upper eft and the bottom eft. However, if we construct Ỹ (t) by performing a water-fiing routing of ξ = 2 customers from X(t) to Ỹ(t),weobservethatY(t) is better baanced than Ỹ (t),ifwe compare the queues in the upper right and the bottom right. Thus, we have Y(t) w Ỹ (t), which satisfies (41). where Ỹ (t) is constructed by routing ξ = X(t) X(t) customers to Ỹ(t) in the water-fiing manner. We discuss the impication of the WFM phase. As mentioned earier, JSQ-LCQ prioritizes serving the RQs and hence wi generate many empty RQs. As a resut, JSQ-LCQ may be forced to route customers, resuting in a short SQ, forexampe,asin(40),andwicausetherqstobuidup. During the WFM phase, it is possibe that Y(t) w Ỹ(t);even the number of customers in the RQs under JSQ-LCQ can be arger than that under π; for exampe, see the exampe in Figure 3. However, from a broader perspective, the queues are sti we-baanced under JSQ-LCQ in WFM phase as foows. There are ξ more customers in the SQ under π, and if we distribute those ξ customers to the RQ in the waterfiing manner (equivaenty, route ξ customers one after another using JSQ) and denote the resuting vector of RQs by Ỹ (t),thenwehavey(t) w Ỹ (t).putdifferenty,ashorter SQ means that JSQ-LCQ is sti ahead of π in pushing customers coser to the destination. Suppose π attempts to catch up the difference in a baanced manner, that is, by routing ξ head-of-ine customers in the SQ to the RQs in a water-fiing manner. The RQs under JSQ-LCQ are sti better baanced; that is, Y(t) w Ỹ(t). Insummary,ifwecompare ony the RQs in WFM phase, JSQ-LCQ may appear worse than other poicies; however, if we consider the SQ and RQs inacombinedway,wefindthatjsq-lcqisbetteroffinterms of baancing queues, which eads to enhanced opportunism. In the proof of Lemma 6, we argued that in cases (5.2) and(9.2)thesystemmakesthetransitiontowfmphase,and hence we wi check if the transition actuay occurred, that is, whether (X(t + 1), Y(t + 1)) wf w ( X(t + 1), Ỹ(t + 1)) hods in those cases. Beow we wi continue and concude the proof. Proof of Lemma 6, Continued. Case 5.2 ((π(t), π(t)) = (R, I)). Note that X(t) = X(t);thuswe have X(t + 1) < X(t + 1). Hence property (40) hods at time t+1. Next we wi show (41) hods at time t+1.notethat Y(t + 1) is obtained by performing the JSQ routing to Y(t). Since ξ f X(t+1) X(t+1) = 1, Ỹ (t+1) can be constructed by performing water-fiing routing of one customer to Ỹ(t + 1). Note that water-fiing routing of one customer is equivaent to the JSQ routing of one customer. In other words, Y(t + 1) and Ỹ (t + 1) are obtained by performing JSQ each on Y(t) and Ỹ(t + 1), respectivey.notethaty(t) w Ỹ(t) = Ỹ(t + 1) hodsduetoinductionhypothesis(9).thus,giventhereation Y(t) w Ỹ(t + 1), ifweperformthejsqroutingtobothy(t) and Ỹ(t+1), the weak maorization reation wi be preserved after JSQ. Thus, we have Y(t+1) w Ỹ (t+1), satisfying (41) at ( X(t+ 1), Ỹ(t + 1)) hods, and the system is in WFM phase at time t+1. time t+1.thus,weconcudethat(x(t+1), Y(t+1)) wf w Case 9.2 ((π(t), π(t)) = (R, S)). We have that X(t + 1) + 1 = X(t) = X(t) = X(t + 1); thus (40) is satisfied at time t+1. Suppose there was a service from the th ongest queue from Ỹ(t). The th ongest queue of Y(t) must be empty; otherwise the queue must have been served under π. Thus, we have that Y [] =0, Y [] (t) 1. (42) In order to construct Ỹ (t + 1), we first need to take action (π(t), π(t)) = (R, S) and then perform water-fiing routing of ξ customers to Ỹ(t + 1).Supposethesestepsaretakeninthe foowing sequence: (1) π(t) takes action S. (2) π(t) takes action R. (3) Perform water-fiing routing to Ỹ(t+1) to yied Ỹ (t+ 1). Let us denote the RQs after step (1) under π by Z(t). Due to Y(t) w Ỹ(t) and (42), a departure from th ongest queue does not affect the weak maorization among RQs; that is, Y(t) w Z(t) hods. Next, consider steps (2) and (3). We have X(t+1) X(t+1) = ξ = 1; ceary, (2) is the JSQ operation on Y(t) with a singe customer under π, and (3) is aso the JSQ operation on Z(t) with ξ=1customer under π. Therefore, Y(t) w Z(t) impies that Y(t+1) w Ỹ (t+1); that is, (41) hods at time t+1. In concusion, the system makes the transition to WFM phase at time t+1. Next we consider the couping of queues under π and π in WFM phase. Lemma 8. There exists a couping between queue ength processes (X(t), Y(t)) and ( X(t), Ỹ(t)) such that if the system is in WFM phase at time t 0, either the system remains in

11 Mathematica Probems in Engineering 11 WFM phase or it makes the transition to WM phase at time t+1. Proof. As previousy, the queue connectivity is couped such that Y [i] (t) and Y [i] (t) have the same connectivity variabe for,...,n. The arrivas at X(t) and X(t) are couped; that is, they see the same arriva variabe. Simiar to WM phase, we consider a tota of 9 action pairs of (π(t), π(t)). Case 1 ((π(t), π(t)) = (S, S)). Firsty, there is no change in the SQs; thus we have X(t + 1) = X(t) and X(t + 1) = X(t), and hence (40) hods at time t+1. Secondy, suppose that the thongestrqhasbeenserved under π and the kth ongest RQ has been served under π.in the proof of Case 1 oflemma6,wehaveshownthatk hods due to the LCQ poicy in π. We further showed that the foowing hods: Y (t+1) w Ỹ (t+1). (43) Since Ỹ (t + 1) is obtained by routing ξ f X(t) X(t) > 0 customers to Y(t + 1), thuscearywehavethatỹ(t + 1) w Ỹ (t+1) hods. Consequenty, Y(t+1) w Ỹ(t+1) hods, which shows that (41) hods at time t+1. Therefore, the system remains in WFM phase at time t+1. Cases 2 and 3 ((π(t), π(t)) = (I, I) or (R, R)). Simiar to the case for WM phase, it is straightforward to show that if π and π perform identica actions of I and R, property(39)is preserved at time t+1. Case 4 ((π(t), π(t)) = (R, S)). Let ξ f X(t) X(t). Sinceπ performed routing and π performed a service, X(t+1) X(t+ 1)=ξ+1>0, satisfying (40) at time t+1. Next we wi show that Y(t + 1) w Ỹ (t + 1). Suppose the service has occurred at th ongest queue of Ỹ(t). Let us denote the index of this th ongest queue by.dueto couping of connectivity, we must have that the th ongest queue of Y(t) is zero. In order to construct Ỹ (t + 1), we need to take two steps on Ỹ(t): (i)serverq and (ii) route ξ+1customers to the RQs in a water-fiing manner. We wi rearrange these steps to compare Y(t) and Ỹ (t) as foows: (1) Serve a customer from RQ under π. (2) Route ξ customers to the RQ in the water-fiing manner under π. (3) Perform JSQ of a customer from X(t) and X(t) so as to yied Y(t + 1) and Ỹ (t + 1),respectivey. Let Z(t) denote the vector of RQs under π after steps (1) and (2) are competed. We wi consider two cases. Case 4.1. This is the case where the foowing is assumed: 1 Y [i] (t) 1 Z [i] (t). (44) Since Y [] (t) = 0, wehavethaty [k] =0, k. This impies that Y [k] (t) Z [k] (t), k.thus,wehavethaty(t) w Z(t). Since both Y(t + 1) and Ỹ (t + 1) are formed by performing JSQ routing to Y(t) and Z(t), Y(t) w Z(t) impies that Y(t + 1) w Ỹ (t + 1). Case 4.2. This is the case where the foowing is assumed: 1 Y [i] (t) > 1 In order for (45) to hod, we must have Z [i] (t). (45) Y [ 1] (t) =Z [ 1] (t) +1 (46) from the induction hypothesis Y(t) w Ỹ (t). That is, because ony one customer is served from the th ongest RQ in Ỹ(t), Y [ 1] (t) and Z [ 1] (t) cannot differ by more than one customer; otherwise it woud vioate the induction hypothesis. Next, we wi show that the foowing hods: Z [ 1] (t) =Z [] (t) = =Z [n] (t), (47) using contradiction. Suppose there exists 1 k n 1such that Z [k] (t) > Z [k+1] (t). The ony difference between Ỹ (t) and Z(t) is that Z(t) is formed by serving the th ongest queue in step (1), before performing water-fiing routing in step (2). Therefore, we have that Y [i] (t) =Z [i] (t),,...,k. (48) However, (48) contradicts (46), and hence (47) hods. Next, we consider step (3). Reca that the ength of the th ongest RQ in Y(t) is zero. After performing JSQ at step (3), we have that Y [] (t+1) =1, Y [i] (t+1) =0, i=+1,...,n. (49) Next, we consider Z(t). From (46) and (47), ( 1)th ongest queue is the shortest queue in Z(t); otherwisey(t) and Z(t) wi differ by more than two customers, vioating induction hypothesis(41).thus,instep(3),onecustomerfrom X(t) wi be routed to the ( 1)th shortest queue of Z(t).Thuswehave that, from (46), Y [ 1] (t+1) =Z [ 1] (t) +1=Y [ 1] (t) =Y [ 1] (t+1). (50) Aso, in (47), we must have Z [ 1] (t) > 0 because it is formed by water-fiing of ξ>0customers.then we have that Y [ 1] (t+1) = Y [ 1] (t+1), Y [i] (t+1) Y [i] (t+1), i. (51)

12 12 Mathematica Probems in Engineering Thus, we have that Y(t + 1) w Ỹ (t + 1); that is, (41) hods at time t+1.weconcudethatthesystemremainsinwfm phase at time t+1. Case 5 ((π(t), π(t)) = (I, S)). In this case, we can show that the system remains in WFM phase at time t+1in a simiar manner to that used for (R, S), becauseactionpair(i, S) can be regarded as a specia case of (R, S) with X(t) = 0 (i.e., π routes zero customers to the RQ). Case 6 ((π(t), π(t)) = (R, I)). Since a routing has occurred at the SQ under π,cearywehavex(t + 1) < X(t + 1).Letξ= X(t) X(t).ToconstructỸ (t+1),weneedtoperformwaterfiing routing to Ỹ(t+1) withξ+1 customers; however, we can aternativey construct Y(t + 1) and Ỹ (t + 1) for comparison purposes as foows: (1) Perform water-fiing routing of ξ customers from X(t) to Ỹ(t). (2) Perform JSQ from both X(t) and X(t) so as to yied Y(t + 1) and Ỹ (t + 1),respectivey. By induction hypothesis, Y(t) is weaky maorized by the resuting vector in step (1) which is Ỹ (t).instep(2),jsqhas been performed equay on Y(t) and Ỹ (t), and thus the weak maorization reation is preserved between the RQs. Thus, we have Y(t + 1) w Ỹ (t + 1). Case 7 ((π(t), π(t)) = (S, I)). Since there is no change in the SQs, X(t + 1) < X(t + 1) hods. Aso, since a RQ has been served under π,we have Y (t+1) w Y (t) w Ỹ (t) = Ỹ (t+1) ; (52) that is, the system is in WFM phase at time t+1. Case 8 ((π(t), π(t)) = (I, R)). Irrespective of which RQ π routes a customer to, we have Y (t+1) = Y (t) w Ỹ (t) w Ỹ (t+1) (53) by induction hypothesis and the definition of Ỹ (t). Thus, (41) hods at time t+1.thesystemmaymakethetransitionto either WFM phase or WM phase depending on the ength of the SQs. If ξ=1,thatis, X(t) = X(t) + 1, wehavethat X(t + 1) = X(t + 1), and (53) impies that Y(t + 1) w Ỹ(t + 1). Thus, the system makes the transition to WM phase at time t+1.otherwise(ξ > 1), from(53),thesystemremainsat WFM phase. Case 9 ((π(t), π(t)) = (S, R)). Using a simiar argument to Case 8, we have Y (t+1) w Y (t) w Ỹ (t) w Ỹ (t+1). (54) Aso, the system makes the transition to either WFM phase or WM phase. If X(t) X(t) = 1,(54)impiesY(t+1) w Ỹ(t+1), and thus the system is in WM phase at time t+1;otherwise, from (54), the system remains in WFM phase Proof and Remarks. We are now ready to prove Theorem 3. ProofofTheorem3. Lemmas 6 and 8 impy that we can coupe the queue ength processes such that the system is either in WM phase or in WFM phase for a s t,byusingforward induction [18]. If the system is in WM phase at time t, itis impied that X (t) + Y (t) X (t) + Ỹ (t) (55) because X(t) X(t) and Y(t) w Ỹ(t).IfthesystemisinWFM phase at time t,wehavethatsincey(t) w Ỹ (t), Y (t) Ỹ (t) = Ỹ (t) +ξ = Ỹ (t) + X (t) X(t), (56) which impies (55) as we. In concusion, using the proposed couping, we can construct sampe paths under which (55) is satisfied for a t = 0, 1, 2,.... This competes the proof of (3). Remark 9. One coud ask, can we construct direct couping between the processes of sum-queues which eads to deay optimaity? That is, if we define Z(t) f X(t) + Y(t) and Z(t) f X(t) + Ỹ(t), can we directy coupe Z(t) and Z(t) to show optimaity, instead of couping the vectors of SQ and RQs as in our proof? We beieve it is quite unikey, because the information on individua queue engths, which is ost if we take the sum of queue engths, is vita in proving the optimaity of JSQ-LCQ. For exampe, if we consider action pair (π, π) = (R, S), the number of customers in the system reduces by one under π; however it remains fixed under JSQ- LCQ. Thus π appearstogetaheadofjsq-lcqintermsof reducing the sum-queue ength. However, this is not true in the ong term, as we have anayzed through couping in WM and WFM phases which were defined based on the (weak) maorization of queue vectors. Importanty, in a haf-dupex two-hop network, we route a customer at the expense of the opportunitytoservearqinthattimesotandviceversa. Thus, routing and service are inherenty in a tradeoff reation, and thus it is crucia that the scheduing and routing decisions propery baance queues so as to maximize the time-varying opportunity in channe connectivity. We have rigorousy showed that JSQ-LCQ is abe to capture those aspects and as a resut achieves deay optimaity in the stochastic ordering sense. 5. Simuation In this section, we evauate the performance of JSQ-LCQ via simuations. We used MATLAB as the discrete event simuator. A time-sotted system is simuated with the foowing parameters: the probabiity of a customer arriving atthesourcequeuegivenbyλ; theprobabiitythatareay queue is in ON state given by p; the number of reay nodes given by n. The performance metric in our simuation is the average number of customers in the system. We compared

13 Mathematica Probems in Engineering Average number of customers Average number of customers Number of reays Probabiity of ON JSQ-LCQ Backpressure Figure 4: Comparison of the average number of customers in the system versus the number of reay nodes. JSQ-LCQ Backpressure Figure 5: Comparison of the average number of customers in the system versus the connectivity probabiity of reays. JSQ-LCQ with the we-known backpressure (BP) scheduing agorithm. Note that the BP agorithm is known to be throughput optima; however it does not guarantee deay optimaity. Figure 4 shows the performance of JSQ-LCQ and BP against n orthenumberofreaynodes.wevaryn from 2 to 9, where the parameters p and λ are given by 0.14 and 0.1, respectivey. We observe that JSQ-LCQ achieves the ower number of customers over a the vaues of n. Weaso observe that the average number of customers decreases with increasing n in both cases. This is because the overa service rate at n reays is given by 1 (1 p) n which increases in n.the rate of decrease in the number of customers is higher at ower vaues of n, which aso can be expained from the dependency of service rate 1 (1 p) n on n; that is, the service rate increases faster for smaer n. Figure 5 shows the performance of JSQ-LCQ and BP against p which is the probabiity that a reay queue is in ON state. The parameters n and λ are given by 4 and 0.2, respectivey. We observe that the number of customers is smaer with JSQ-LCQ over a the vaues of p. Weaso observethattheaveragenumberofcustomersdecreasesas aconvexfunctionofp simiar to Figure 4; however, the rate of decrease is not as steep as that with respect to n, because the service rate 1 (1 p) n is a poynomia function of p in contrast to its exponentia dependence in n. Figure 6 shows the performance of JSQ-LCQ and BP against the arriva rate λ. The parameters p and n are givenby0.8and4,respectivey.weobservethatjsq-lcq outperforms BP over a vaues of λ. Weasoobservethat the number of customers increases with increasing λ. The number of customers wi bow up as λ tighty approaches condition (2). In concusion, the simuation resuts verify that JSQ-LCQ indeed achieves the smaer number of customers in the system over a simuated parameters, as is expected from our theoretica resut on optimaity. Average number of customers Probabiity of arrivas JSQ-LCQ Backpressure Figure 6: Comparison of the average number of customers in the system versus the arriva rates. 6. Concusion In this paper, we studied a deay-optima poicy for hafdupex two-hop reay networks with symmetric connectivity on the service. We showed that JSQ-LCQ poicy is strongy optima; that is, the tota queue ength under JSQ-LCQ is stochasticay dominated by any other poicy. Our future work incudes devising simpe poicies for two-hop reay networks with asymmetric connectivity and studying deay optimaity for cooperative reay networks with more than two hops. Under certain reaistic conditions for time-varying channes, for exampe, fading due to the user mobiity having power-aw distributions [39], the i.i.d. assumption

14 14 Mathematica Probems in Engineering of channe connectivity over time may not hod, and JSQ- LCQ is not guaranteed to be deay-optima. In fact, for many reaistic channe modes, it is difficut to guarantee deay optimaity. The significance of our work is that, however, we have estabished the deay optimaity of a two-hop network mode with time-varying connectivity, abeit its simpicity, considering that there exists a imited amount of works on deay-optima scheduing. Conficts of Interest The authors decare that there are no conficts of interest regarding the pubication of this paper. Acknowedgments ThisworkissupportedinpartbyNationaResearchFoundation of Korea (NRF) grant funded by the Korea government (MSIP) (NRF-2016R1A2B ) and by Institute for Information & Communications Technoogy Promotion (IITP) grant funded by the Korea government (MSIP) (no. B ) and in part by Basic Science Research Program through the Nationa Research Foundation of Korea (NRF) funded by the Ministry of Education (NRF-2016R1D1A1B ). References [1]Y.Yang,H.Hu,J.Xu,andG.Mao, Reaytechnoogiesfor WiMAX and LTE-advanced mobie systems, IEEE Communications Magazine,vo.47,no.10,pp ,2009. [2] K. T. Phan, T. Le-Ngoc, and L. B. Le, Optima Resource Aocation for Buffer-Aided Reaying with Statistica QoS Constraint, IEEE Transactions on Communications, vo.64,no.3,pp , [3] S. W. Peters, A. Y. Panah, K. T. Truong, and R. W. Heath, Reay architectures for 3GPP LTE-advanced, EURASIP Journa on Wireess Communications and Networking,vo.2009,ArticeID , [4] R. Ford, M. Zhang, M. Mezzavia, S. Dutta, S. Rangan, and M. Zorzi, Achieving utra-ow atency in 5G miimeter wave ceuar networks, IEEE Communications Magazine,vo.55,no. 3, pp , [5] C.-X. Wang, F. Haider, X. Gao et a., Ceuar architecture and key technoogies for 5G wireess communication networks, IEEE Communications Magazine, vo.52,no.2,pp , [6] M. Tehrani, M. Uysa, and H. Yanikomerogu, Device-todevice communication in 5G ceuar networks: chaenges, soutions, and future directions, IEEE Communications Magazine,vo.52,no.5,pp.86 92,2014. [7] D.Feng,L.Lu,Y.Y.Wu,G.Li,S.Li,andG.Feng, Device-todevice communications in ceuar networks, IEEE Communications Magazine,vo.52,no.4,pp.49 55,2014. [8]A.Betsas,A.Khisti,D.P.Reed,andA.Lippman, Asimpe cooperative diversity method based on network path seection, IEEE Journa on Seected Areas in Communications,vo.24,no. 3, pp , [9] S.Cui,A.M.Haimovich,O.Somekh,andH.V.Poor, Opportunistic reaying in wireess networks, Institute of Eectrica and Eectronics Engineers Transactions on Information Theory, vo. 55,no.11,pp ,2009. [10] Y. Jing and H. Jafarkhani, Singe and mutipe reay seection schemes and their achievabe diversity orders, IEEE Transactions on Wireess Communications, vo. 8, no. 3, pp , [11] R. Wang, V. K. Lau, and Y. Cui, Queue-aware distributive resource contro for deay-sensitive two-hop MIMO cooperative systems, IEEE Transactions on Signa Processing,vo.59,no. 1, pp , [12] N. Zatanov, A. Ikhef, T. Isam, and R. Schober, Buffer-aided cooperative communications: opportunities and chaenges, IEEE Communications Magazine, vo.52,no.4,pp , [13] H. Xu, L. Huang, L. Chen, and S. Lin, Joint reay assignment and rate power aocation for mutipe paths in cooperative networks, Wireess Networks, vo. 22,no. 3, pp , [14] A. Sabharwa, P. Schniter, D. Guo, D. W. Biss, S. Rangaraan, and R. Wichman, In-band fu-dupex wireess: chaenges and opportunities, IEEE Journa on Seected Areas in Communications,vo.32,no.9,pp ,2014. [15] Z.Zhang,X.Chai,K.Long,A.V.Vasiakos,andL.Hanzo, Fu dupex techniques for 5G networks: sef-interference canceation, protoco design, and reay seection, IEEE Communications Magazine,vo.53,no.5,pp ,2015. [16] M. Iwamura, H. Takahashi, and N. Satoshi, Reay technoogy in te-advanced, NTT DoCoMo Technica Journa,vo.12,no.2, pp.29 36,2010. [17] L. Tassiuas and A. Ephremides, Dynamic server aocation to parae queues with randomy varying connectivity, Institute of Eectrica and Eectronics Engineers Transactions on Information Theory,vo.39,no.2,pp ,1993. [18] J. Warand, An Introduction to Queueing Networks, Prentice Ha, [19] E. M. Yeh and A. S. Cohen, Throughput and deay optima resource aocation in mutiaccess fading channes, in Proceedings of the IEEE Internationa Symposium on Information Theory (ISIT),2003. [20] D. N. Tse and S. V. Hany, Mutiaccess fading channes. I. Poymatroid structure, optima resource aocation and throughput capacities, Institute of Eectrica and Eectronics Engineers Transactions on Information Theory, vo.44,no.7,pp , [21] L. Tassiuas and A. Ephremides, Dynamic scheduing for minimum deay in tandem and parae constrained queueing modes, Annas of Operations Research,vo.48,no.1-4,pp , [22] Y. Cui, V. K. Lau, and E. Yeh, Deay optima buffered decodeand-forwardfortwo-hopnetworkswithrandominkconnectivity, Institute of Eectrica and Eectronics Engineers Transactions on Information Theory,vo.61,no.1,pp ,2015. [23] A. Ikhef, D. S. Michaopouos, and R. Schober, Max-max reay seection for reays with buffers, IEEE Transactions on Wireess Communications, vo. 11, no. 3, pp , [24] Y. Cui, E. M. Yeh, and R. Liu, Enhancing the deay performance of dynamic backpressure agorithms, IEEE/ACM Transactions on Networking,vo.24,no.2,pp ,2016. [25] D. P. Bertsekas, Dynamic Programming and Optima Contro, vo. 1, Athena Scientific Bemont, Mass, USA, [26] X. R. Cao, Stochastic Learning and Optimization: A Sensitivity- Based Approach, Springer-Verag, New York, NY, USA, 1st edition, 2007.

15 Mathematica Probems in Engineering 15 [27] L. Tassiuas and A. Ephremides, Stabiity properties of constrained queueing systems and scheduing poicies for maximum throughput in mutihop radio networks, Institute of Eectrica and Eectronics Engineers Transactions on Automatic Contro,vo.37,no.12,pp ,1992. [28] L. Georgiadis, M. J. Neey, and L. Tassiuas, Resource aocation and cross-ayer contro in wireess networks, Foundations and Trends in Networking,vo.1,no.1,2006. [29] M. J. Neey, E. Modiano, and C. E. Rohrs, Dynamic power aocation and routing for time-varying wireess networks, IEEE Journa on Seected Areas in Communications, vo.23,no. 1, pp , [30] A. Eryimaz and R. Srikant, Joint congestion contro, routing, and MAC for stabiity and fairness in wireess networks, IEEE Journa on Seected Areas in Communications,vo.24,no.8,pp , [31] L. Bui, R. Srikant, and A. Stoyar, Nove architectures and agorithms for deay reduction in back-pressure scheduing and routing, in Proceedings of the 28th Conference on Computer Communications, IEEE INFOCOM 2009, pp , Brazi, Apri [32] L. Ying, S. Shakkottai, A. Reddy, and S. Liu, On combining shortest-path and back-pressure routing over mutihop wireess networks, IEEE/ACM Transactions on Networking,vo.19,no.3, pp ,2011. [33] M. Aresaini, M. Sathiamoorthy, B. Krishnamachari, and M. J. Neey, Backpressure with adaptive redundancy (BWAR), in Proceedings of the IEEE Conference on Computer Communications(INFOCOM 12),pp ,March2012. [34]E.Athanasopouou,L.X.Bui,T.Ji,R.Srikant,andA.Stoyar, Back-pressure-based packet-by-packet adaptive routing in communication networks, IEEE/ACM Transactions on Networking,vo.21,no.1,pp ,2013. [35] S. Moeer, A. Sridharan, B. Krishnamachari, and O. Gnawai, Routing without routes: the backpressure coection protoco, in Proceedings of the 9th ACM/IEEE Internationa Conference on Information Processing in Sensor Networks (IPSN 10), pp , IEEE, Apri [36] A.Warrier,S.Janakiraman,S.Ha,andI.Rhee, DiffQ:practica differentia backog congestion contro for wireess networks, in Proceedings of the 28th IEEE Internationa Conference on Computer Communications (INFOCOM 09), pp , IEEE, Rio de Janeiro, Brazi, Apri [37] J.Ryu,V.Bhargava,N.Paine,andS.Shakkottai, Back-pressure routing and rate contro for ICNs, in Proceedings of the 16th Annua Conference on Mobie Computing and Networking, MobiCom 2010, pp , USA, September [38] D. Stoyan and D. J. Daey, Comparison Methods for Queues and Other Stochastic Modes, John Wiey & Sons, New York, NY, USA, [39] M. C. Gonzáez, C. A. Hidago, and A.-L. Barabási, Understanding individua human mobiity patterns, Nature,vo.453, no.7196,pp ,2008.

16 Advances in Operations Research Advances in Decision Sciences Journa of Appied Mathematics Agebra Journa of Probabiity and Statistics The Scientific Word Journa Internationa Journa of Differentia Equations Submit your manuscripts at Internationa Journa of Advances in Combinatorics Mathematica Physics Journa of Compex Anaysis Internationa Journa of Mathematics and Mathematica Sciences Mathematica Probems in Engineering Journa of Mathematics Journa of Voume 201 Discrete Dynamics in Nature and Society Journa of Function Spaces Abstract and Appied Anaysis Internationa Journa of Journa of Stochastic Anaysis Optimization