Repeated Game Analysis for Cooperative MAC with Incentive Design for Wireless Networks

Transcription

1 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 1 Repeated Game Analysis for Cooperative MAC with Incentive Design for Wireless Networks Peijian Ju and Wei Song, Senior Member, IEEE Abstract Cooperative communications offer appealing potentials to improve quality of service for wireless networks. Many existing works on cooperative communications assume participation in cooperative relaying is unconditional. In practice, however, due to resource consumption, it is vital to provide incentives for selfish cooperating peer nodes. In this paper, we analyze a cooperative medium access control (MAC) protocol with incentive design using a game theoretical approach. Specifically, our analysis addresses two questions: 1) Can cooperating agreement be reached between the peer nodes? and 2) can cooperating lead to higher utility than not-cooperating? We first formulate a onestage game for the slotted Aloha-based cooperative MAC protocol where not-cooperating is a Nash equilibrium (NE) strategy while cooperating is not. To exploit cooperation gain, the one-stage game is extended to a two-stage game by incorporating an incentive mechanism that adapts channel access probabilities with tuning factors. Based on the Markov chain analysis of the system states with the repeated two-stage games, we determine valid tuning factors guaranteeing that cooperating attains an NE and provides expected utility not less than that of the notcooperating NE. Moreover, a special case with saturated and symmetric assumptions is investigated and closed-form criteria for the tuning factors are derived. Last, we compare the derived tuning factors for individual nodes with an optimal choice that is selected from the system perspective to maximize the overall system utility. The numerical results confirm our analytical conclusions and demonstrate that the tuning factors selected according to our derived criteria can achieve high utility that is slightly lower than that of the optimal choice. Index Terms Cooperative MAC, cooperative communications, incentive design, repeated game, Nash equilibrium, Markov chain. I. INTRODUCTION Cooperative communications, which offer promising benefits in extending coverage, increasing capacity, and improving energy efficiency, have been widely studied in the literature. Taking advantage of the broadcast nature of wireless medium, a relay node can forward data packets overheard from a source node to the destination. A diversity gain can thus be achieved at the destination by combining multiple copies received over independent links. To fully harvest the cooperation gain at the physical layer, it is essential to adapt the upper-layer protocols to accommodate cooperative communications as Copyright c 2015 IEEE. Personal use of this material is permitted. However, permission to use this material for any other purposes must be obtained from the IEEE by sending a request to pubs-permissions@ieee.org. Manuscript received December 12, 2014, revised April 12, 2015, and accepted June 3, This work was supported in part by Natural Sciences and Engineering Research Council of Canada (NSERC). The review of this paper was coordinated by J. Deng. P. Ju and W. Song ({i1689, wsong}@unb.ca) are with Faculty of Computer Science, University of New Brunswick, Fredericton, Canada. additional protocol overhead is also involved with cooperation. In particular, a cooperative medium access control (MAC) protocol needs to appropriately address fundamental issues such as when to cooperate, with whom to cooperate, and how to cooperate [1]. Many existing studies assume that all nodes unconditionally participate in cooperative transmission. That is, if a node is able to relay a packet, the node chooses to help regardless of its own benefits or preferences. Obviously, this is a strong assumption because even a capable node may decline a request due to lack of incentives for cooperation. Thus, an effective cooperative MAC should take into account incentives [2,3], so that relaying nodes are stimulated to make informed and intelligent decisions for cooperation based on foreseeable benefits. Moreover, many previous works consider a cooperative communication scenario where a pair of source and destination nodes are served by a dedicated helper node. The helper forwards overheard data from the source to the destination, but the source does not relay data for the helper. Such an imbalanced situation without mutual interests makes it challenging to incentivize cooperation. On the other hand, if the source and the helper are peer nodes that can relay traffic for each other, there will be a stronger foundation to reach a cooperation agreement. Focusing on the latter bidirectional cooperative communication scenario, we aim to address how to incorporate incentives in a cooperative MAC protocol. Specifically, we use a game theoretic approach [4] to model and analyze the mutual decision-making process for cooperation. The key contributions of this work are multi-fold. Firstly, a one-stage game is formulated for the single-round cooperative transmission. Here, we consider that the cooperation participants are half-duplex wireless nodes using slotted Aloha for channel access. It is proved that not-cooperating is the only Nash equilibrium (NE) strategy and cooperation is never attained at both player sides. To achieve higher benefits with cooperation and stimulate engagement in cooperation, an incentive mechanism is further incorporated into the MAC protocol to adapt channel access probabilities by tuning factors. If successful cooperation is reached, the access probability of the helper is increased as reward, while that of the requester is tuned down as cost. Accordingly, the one-stage game is extended to a two-stage game, which can be played in a repeated fashion between the peer nodes. The system state transitions in the repeated game with various cooperation strategies are modeled by a Markov chain. Based on rigorous analysis of the Markov chain and the achievable utilities with underlying strategies, we can obtain valid tuning factors to adapt channel access probabilities so that a cooperating

2 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 2 strategy is ensured to attain an NE and offer higher expected utility than a not-cooperating NE strategy. Moreover, we investigate a special case, where nodes have saturated traffic and symmetric settings. Closed-form criteria are derived for valid tuning factors to guarantee that the repeated two-stage game converges to an NE with the cooperating strategy and provides higher utility. In addition, we compare the derived tuning factors from the perspective of individual nodes with an optimal choice that aims to maximize the overall system utility. The numerical results confirm our analytical conclusions and demonstrate that the tuning factors selected according to the derived criteria can achieve high utility that is only slightly lower than that of the optimal choice. The remainder of this paper is organized as follows. Section II briefly reviews the related work. Section III introduces the system model. Section IV gives the formulation and analysis of a one-stage game for single-round cooperation. In Section V, a repeated two-stage game is proposed and analyzed by a Markov chain. Numerical results are presented in Section VI, followed by conclusions in Section VII. II. RELATED WORK Cooperative communications have been extensively studied in the past decade. At the physical layer, fundamental relaying protocols such as amplify-and-forward (AF) and decode-andforward (DF) were proposed in [5]. In [5], signal-to-noise ratio (SNR) of relaying channel using AF and DF was analyzed. Based on SNR, various quality of service (QoS) metrics can be further considered. For example, [6] studied outage capacity of a cooperative relaying system with Rayleigh fading. At the MAC layer, [1] and [7] extensively surveyed existing cooperative MAC protocols, including centralized schemes [8] [10] as well as distributed schemes [11,12]. However, many existing works assume unconditional participation in cooperation and do not address cooperation incentives. As a powerful mathematical tool, game theory [4] has been widely applied to study wireless networks [13]. The application of game theory in cooperative communications was reviewed in [14], which classifies cooperation incentive mechanisms into three categories, namely, resource exchange-based, pricing-based and reputation-based. In resource exchangebased and pricing-based schemes, the focus is to decide how to allocate resources assuming that cooperation is beneficial. Specifically, the resource exchange-based scheme in [15] aims to determine how much power each node should contribute to transmitting others data so that optimal benefit is reached. On the other hand, pricing-based schemes often use auction models to decide which relay a source should choose given the relays price list [16,17], or which source a relay should serve given the sources bid list [18,19]. The work in [20] uses double auction to match sources and relays to satisfy certain desirable economic properties. Different from above, the reputation-based mechanism in [21] aims to find out whether cooperation is beneficial, which shares a similar objective as this paper. It is worth noting two major differences between [21] and our work. Firstly, [21] focuses on the physical layer and leverages power allocation C Direct links Fig. 1. Paired cooperating S-D nodes. B A Indirect links as reward/cost assuming full duplex among cooperative nodes. We take into account the more realistic half-duplex constraint, and exploit channel access probability at the MAC layer as an incentive lever. Secondly, in [21], a static one-shot cooperative game is extended to a repeated case with a discount factor. In contrast, this study involves different game types depending on the system state, and the stochastic transitions among feasible states is modeled by a Markov chain. The works on cooperation incentives can also be categorized according to the form of incentives. Basically, there are two types of incentives: moneyless and monetary. Moneyless incentives may vary with the specific radio access technology. They can be higher transmit power, such as in [15,21], or higher priority to access the channel as in our work. Monetary incentives are money or virtual currency. That is, the helper node is rewarded money which can also be spent to exchange for resources or services. For monetary incentives, the main challenges are involved with the pricing and allocation of resources. Some works based on auction theory often consider monetary incentives, such as [14,20]. III. SYSTEM MODEL In this paper, we consider a cooperative communication scenario in Fig. 1, where there are two source nodes in set N = {A, B} and two destination nodes C and D. Without loss of generality, we assume A transmits to C and B transmits to D. Also, A can relay data from B to D, while B can relay data fromato C. In other words,aand B are peer nodes that can relay data for each other, and none is a dedicated helper to the other. This bidirectional cooperative communication scenario involves two pairs of source-destination (S-D) nodes. Effective relay selection algorithms [22,23] can be used to match two S-D pairs that can mutually benefit from the cooperative relaying of each other. A variety of metrics can be taken into account in the relay selection algorithms, such as nodes spatial locations, link conditions, traffic loads, and battery levels. Here, we consider two cooperation pairs mainly based on the concern that employing more relays will incur larger coordination overhead although there can be a higher diversity gain [24]. It can lead to tractable game formulations that clearly demonstrate the cooperation negotiation between two player sides, i.e., the benefactor and beneficiary. Besides, D

3 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 3 Arrival slot Access Arrival Access Fig. 2. Sequence of packet arrival and channel access in each timeslot. this model is widely accepted in game theoretic analysis on cooperative communications [21,25] and cooperative Aloha [26] [28]. Notice that Fig. 1 is a generalized version of the system model in these works, since we do not restrict that all the wireless devices share a common destination. More details of the system setup are given as below. Slotted Aloha and channel access probability. Time is slotted and all nodes are subject to half-duplex constraint, which means that a node cannot transmit and receive simultaneously during one timeslot. Assuming channel access with slotted Aloha, when the packet transmit buffer of node n (n N) is not empty, node n accesses channel and sends the head packet in the buffer with probability p n, where 0 < p min p n p max < 1. Here, the lower and upper bounds of access probability, p min and p max, can be adapted with traffic demands. The channel is successfully accessed in a timeslot if only one node transmits in that slot, while a collision happens if more than one node transmits. If none transmits, the slot is skipped. Packet arrivals. For analysis simplicity, we assume that node n has a new packet arrival with probability λ n (0 < λ n 1) at the beginning of a timeslot only if the transmit buffer is empty. If the buffer is not empty, node n sends the head packet of the buffer following the channel access procedure. The time sequence of packet arrival and channel access in each timeslot is shown in Fig. 2. It is worth clarifying that the packets from the requester can be also placed in the transmit buffer of the helper node. Depending on the action of the helper, the helper s own packets and the packets from the requester can be treated in different manners (to be discussed in Section IV-A). Transmission success probability. Let s n denote the probability that a packet from source node n is successfully decoded at the destination over the direct link, and s n denote the success probability over the indirect link with cooperative relaying. Here, we assume that s n > s n for any n N. Important notations used in this paper are summarized in Table III in Appendix A. slot IV. ONE-STAGE GAME FORMULATION AND ITS NE ANALYSIS In this section, we first formulate a one-stage game with different game states for the cooperative communication scenario specified in Section III. Then we prove that the cooperating strategy cannot attain an NE for any game state, while the not-cooperating strategy can reach an NE if the access cost is small enough. T A. One-Stage Game Formulation The basic elements of the one-stage game are defined as follows: Player set. Naturally, the set of source nodes, N, is the player set. We use r to represent the requester of cooperation and h for the helper node, where r,h N,r h. Note that r and h are not fixed to node A or B. If r is node A, then h is node B, and vice versa. Action set. Each node in set N can choose cooperating (C) or not-cooperating (N C) as its action. Thus, the action set is {C,NC}. The actions of nodes A and B are respectively denoted by a A and a B, where a A,a B {C,NC}. Utility and access cost. Utility is a mathematical abstraction of a node s overall payoff. We consider a nonnegative monotonically increasing function v(s) to quantify a node s gain with a transmission success probability s, where v(s) 0 and v(0) = 0. Further taking into account an access cost c > 0 for transmitting a packet [29], we evaluate the overall utility by v(s) c. Due to the half-duplex constraint, cooperation decisions cannot be made at r and h simultaneously within one timeslot. Hence, we consider two timeslots t 1 and t 2, where in each slot there can be a packet arrival event followed by a channel access event as shown in Fig. 2. In order to start a game for the cooperative relaying procedure, there must be one and only one node that has already successfully captured the channel. If none or both nodes attempt to access the channel, there will be no transmission or a failed transmission due to collision. Then, no game will be started. In addition, h must correctly overhear the packet from r so that the cooperation game is meaningful. Fig. 3 depicts the one-stage game process when the condition to start a game is met. Take the scenario that both r and h choose to cooperate as an example. In t 1, r must have a packet in its transmit buffer and has captured the channel in that slot. Since r chooses cooperating action C, it piggybacks cooperation information in the packet header and deliberately gives up access in t 2. Because h also chooses cooperating action C, if h correctly overhears the packet of r in t 1, h transmits that packet without contention in t 2. As seen in the above game formulation, the utility of a node with a chosen action is related to two aspects, i.e., the identity of the requester in t 1 and the buffer state in t 2 after the packet arrival event. Specifically, we can use a two-element tuple b A,b B to represent the buffer state of A and B, where b A,b B = 1 or 0 indicate that there is or is not a local packet to transmit, respectively. Depending on whether A or B is the cooperation requester and their buffer state, we define eight game states in Table I, where a game state g G = {G i : i = 1,...,8}. Then, letting U n (a A,a B ;g;p A,p B ) be node n s utility in game state g, with strategy (a A,a B ) and channel access probability p A and p B for A and B, respectively, we express the utilities of A and B with different strategies in (1)-(8).

4 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 4 U A (C,C;g;p A,p B ) = (1) { v( sa ) c, g = G 1,G 2,G 3,G 4 r Start h c, g = G 5,G 6,G 7,G 8 U A (C,NC;g;p A,p B ) = (2) { v(sa ) c, g = G 1,G 2,G 3,G 4 cp A, g = G 5,G 6,G 7,G 8 U A (NC,C;g;p A,p B ) = (3) [v( s A ) c]p B +[v(s A ) c](1 p B ), g = G 1,G 3 [v(s A ) 2c]p A p B +[v( s A ) c](1 p A )p B +[2v(s A ) 2c]p A (1 p B ) +[v(s A ) c](1 p A )(1 p B ), g = G 2,G 4 0, g = G 5,G 6 [v(s A ) c]p A, g = G 7,G 8 U A (NC,NC;g;p A,p B ) = (4) v(s A ) c, g = G 1,G 3 v(s A ) c+[v(s A ) c]p A, g = G 2 v(s A ) c cp A p B +[v(s A ) c]p A (1 p B ), g = G 4 0, g = G 5,G 6 [v(s A ) c]p A, g = G 7 cp A p B +[v(s A ) c]p A (1 p B ), g = G 8 U B (C,C;g;p A,p B ) = (5) { c, g = G1,G 2,G 3,G 4 v( s B ) c, g = G 5,G 6,G 7,G 8 U B (C,NC;g;p A,p B ) = (6) 0, g = G 1,G 2 [v(s B ) c]p B, g = G 3,G 4 [v( s B ) c]p A +[v(s B ) c](1 p A ), g = G 5,G 7 [v(s B ) 2c]p A p B +[v( s B ) c](1 p B )p A +[2v(s B ) 2c]p B (1 p A ) +[v(s B ) c](1 p A )(1 p B ), g = G 6,G 8 U B (NC,C;g;p A,p B ) = (7) { cpb, g = G 1,G 2,G 3,G 4 v(s B ) c, g = G 5,G 6,G 7,G 8 U B (NC,NC;g;p A,p B ) = (8) 0, g = G 1,G 2 [v(s B ) c]p B, g = G 3 cp A p B +[v(s B ) c]p B (1 p A ), g = G 4 v(s B ) c, g = G 5,G 7 v(s B ) c+[v(s B ) c]p B, g = G 6 v(s B ) c cp A p B +[v(s B ) c]p B (1 p A ), g = G 8. Next, we use some examples to demonstrate how a node s utility varies with game states. Firstly, we assume that both nodes choose C and the buffer state is 0,0. In this case, the utilities of A and B are given by (1) and (5), respectively. If A requests cooperation, i.e., r = A, the game state will be G 1. According to Fig. 3, A gives up t 2 so that B can relay for A if the packet from A is correctly overheard. Thus, A s utility in state G 1 with strategy (C,C) is v( s A ) c. On the other hand, NC Transmit t1 t2 Pkt in buffer? Y Contend channel Action? C Piggyback cooperation info and transmit; Give up t2 End Fig. 3. One-stage game process. N Y N N Overheard pkt? Action? NC Pkt in buffer? Pkt in buffer? TABLE I STATES OF ONE-STAGE GAME. Y N C Transmit overheard pkt Game state g Requester r Buffer state G 1 A 0,0 G 2 A 1,0 G 3 A 0,1 G 4 A 1,1 G 5 B 0,0 G 6 B 0,1 G 7 B 1,0 G 8 B 1,1 Y Defer own pkt if B requests cooperation from A, the game state will be G 5. Since A relays B s packet, A achieves utility c. Secondly, let us see another example that both nodes choose NC and r = A. In this case, the utilities of A and B are given by (4) and (8), respectively. For instance, if the buffer state is 0, 0, the game state will be G 1, which means that A has transmitted its packet in t 1 and has no packet to send in t 2. Thus, A s utility is v(s A ) c. If the buffer state is 1,0, the game state will be G 2. Then, A s utility is [v(s A ) c]+[(v(s A ) c)p A ], which consists of the utilities gained in both t 1 and t 2. B. Nash Equilibrium of One-Stage Game Proposition 1. g G, (C,C) is not an NE strategy for the one-stage game. Proof. According to the definition of NE, g G, (C,C) must satisfy the following constraints to be an NE strategy for the one-stage game: U A (C,C;g;p A,p B ) U A (NC,C;g;p A,p B ) (9) U B (C,C;g;p A,p B ) U B (C,NC;g;p A,p B ). (10) From (1) and (3), it is obvious that U A (C,C;g;p A,p B ) < U A (NC,C;g;p A,p B ), when g = G 5,G 6, since c < 0. When g = G 7,G 8, U A (NC,C;g;p A,p B ) > cp A > c, since 0 < p A < 1. As a result, U A (C,C;g;p A,p B ) < U A (NC,C;g;p A,p B ), when g = G 7,G 8. Therefore, when

5 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 5 g = G 5,G 6,G 7,G 8, (9) cannot be satisfied. Similarly, when g = G 1,G 2,G 3,G 4, (10) does not hold according to (5) and (6). Thus, (9) and (10) cannot be satisfied simultaneously for any game state and Proposition 1 is proved. Proposition 2. Given an upper boundp max for channel access probability, (NC,NC) is an NE strategy for the one-stage game in any game state g G if c min n N [ (1 pmax )v(s n ) ]. (11) Proof. For (N C, N C) to be an NE strategy, the following constraints must be satisfied for any g G: U A (NC,NC;g;p A,p B ) U A (C,NC;g;p A,p B ) (12) U B (NC,NC;g;p A,p B ) U B (NC,C;g;p A,p B ). (13) According to (2) and (4), it is easily found that (12) holds when g = G 1,G 3,G 5,G 6. For other game states, satisfaction of (12) requires [ v(sa ) c ] p A 0, when g = G 2,G 7 (14) [ (1 pb )v(s A ) c ] p A 0, when g = G 4,G 8. (15) If (11) holds, since 0 p B p max, we have c (1 p max )v(s A ) (1 p B )v(s A ) v(s A ). Thus, both (14) and (15) are satisfied. Thus, (12) is true when (11) holds. Likewise, (13) can be proved based on (7), (8), and (11). Thus, (NC,NC) is an NE strategy for the one-stage game under the condition in (11). V. TWO-STAGE GAME FORMULATION AND MARKOV CHAIN ANALYSIS FOR REPEATED GAME Since (C,C) is not an NE strategy for the one-stage game, we further form a two-stage game with two adjacent one-stage games and explore the possibility of reaching cooperation agreement in an NE. An incentive mechanism is designed to charge the requester and reward the helper. Specifically, we tune up the access probability of the helper as reward, while the cooperation requester is charged by reducing its access probability. The main reason for considering two stages is to provide incentive to the helper as soon as possible. Otherwise, an impatient helper is more likely to abandon cooperation if it has to contribute for a long period before collecting reward. The two-stage game can be played repeatedly, while the system state evolves according to a Markov chain. With the extended game formulation, we need to determine the criteria for the tuning factors in the incentive mechanism, so that the cooperating strategy can attain an NE that offers higher utility than the not-cooperating strategy. A. Two-Stage Game Formulation with Incentive The results of Propositions 1 and 2 seem disappointing in that cooperation agreement cannot be reached in an NE. We observe in (1)-(8) that the helper always obtains negative utility when it chooses to cooperate, since the helper is never rewarded even after it offered help. The requester becomes a free rider who enjoys cooperation benefit without any charge. Thus, an incentive mechanism should be in place to prevent the requester from abusing free resources and to stimulate the helper to cooperate. To fulfill incentive for the helper, we consider two adjacent one-stage games and model that as a two-stage game. The action for the two-stage game is denoted by (a A1 a A2,a B1 a B2 ), where (a A1,a B1 ) are the actions of A and B in the first stage, while (a A2,a B2 ) are actions in the second stage. We are particularly interested in the cooperating strategy (C NC,C NC). In this strategy,(c,c) are taken in the first stage, which can be viewed as investment of the helper by offering cooperative relaying. In the subsequent stage, the NE strategy (NC,NC) of the one-stage game is chosen by both nodes, so that the helper can collect its reward by not-cooperating. In the two-stage game, we define two tuning factors, µ and ε, which adapt the channel access probabilities of the requester(r) and the helper(h) to incorporate cost and reward, respectively. With µ and ε, the basic slotted Aloha protocol in Section III is extended as follows. If both r and h choose action C in the first stage, the channel access probability p r of the requester is reduced to µp r in the second stage, and the access probability p h of the helper is increased to εp h, where 0 µ 1 and 1 ε pmax p h. If r and h take other actions, p r and p h keep unchanged. In this manner, r reduces its contention privilege as cost to exchange for cooperative relaying of h, while h is rewarded with a higher channel access probability for its cooperative relaying in the first stage. The two-stage game can be played repeatedly so that the nodes are more likely to utilize the cooperating strategies that cannot attain an NE in the one-stage game. As the repeated two-stage game provides cooperation incentives, it is possible that strategy (C NC,C NC) reaches an NE. We still need to address two important questions for the repeated game: 1) Under what conditions is (C NC,C NC) an NE strategy? and 2) can (C NC,C NC) offer higher utility than another obvious NE strategy(nc NC,NC NC)? For (C NC,C NC) to be an NE strategy better than (NC NC,NC NC), we should determine the tuning factors(µ and ε) appropriately. First, we need to characterize the system state evolution with the repeating of the two-stage game. B. System States of Repeated Two-Stage Game To analyze the repeated two-stage game, we redefine the system states according to three aspects: 1) Packet arrival or channel access event, 2) buffer state after an event, and 3) whether a game is started. Accordingly, the system states are categorized into three groups for each stage j (j = 1,2). Temporary states: The buffer state after a channel access event is marked as a temporary state. Similar to the one-stage formulation in Section IV-A, we use 1 or 0 to indicate that there is or is not a packet in the transmit buffer, respectively. Considering the buffer state of A and B, the set of temporary states, denoted by T j = {T j l : l = 1,...,4}, contains 4 possible states. Game states: When a game is started after a channel access event in timeslot t 1 and the system reaches a temporary state, this temporary state further goes through a packet arrival event and reaches a game state in timeslot

6 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 6 TABLE II RELATIONSHIPS BETWEEN TEMPORARY / INTERMEDIATE STATES AND BUFFER STATES. Temporary state Buffer state Intermediate state Buffer state T j 1 0,0 I j 1 0,0 T j 2 1,0 I j 2 1,0 T j 3 0,1 I j 3 0,1 T j 4 1,1 I j 4 1,1 t 2. Table I has shown all definitions of game states in set G j = {G j i : i = 1,...,8}. Intermediate states: When a game is not started after a channel access event in t 1, the system reaches a temporary state, which further experiences a packet arrival event in t 2 but reaches an intermediate state. The set of intermediate states is denoted by I j = {I j k : k = 1,...,4}. The temporary states and intermediate states are related to the buffer states as given in Table II. C. Markov Chain Analysis To design an effective incentive mechanism with the tuning factors for channel access probabilities, we need to answer the two questions at the end of Section V-A, i.e., deriving the conditions for (C NC,C NC) to be an NE strategy that offers higher utility than (NC NC,NC NC). Thus, we need to compare the utilities with eight related strategies, which are divided into two classes, namely, cooperation set S c and non-cooperation set S nc. Here, S c = {(C NC,C NC),(C C,C NC),(C NC,C C)}, where both nodes take C in the first stage in each strategy of S c. The rest of related strategies are grouped into S nc = {(NC NC,NC NC),(NC C,C NC),(NC NC,C NC),(C NC,NC C),(C NC,NC NC)}. The state transitions betweeni j, G j, andt j, with the strategies in non-cooperation set S nc, are analyzed in Appendix B. It is worth mentioning that each transition from I j to G j or from I j to itself needs to pass through T j. Since we are interested in the states that the system converges to a game, we can hide the transitory T j and derive an aggregate transition probability matrix with strategy (a A1 a A2,a B1 a B2 ), denoted by P Snc (a A1 a A2,a B1 a B2 ), as shown in Table IV of Appendix B. Then, the limiting probability matrix, denoted by Π nc = {π nc (θ) : θ I 1 I 2 G 1 G 2 }, can be obtained by solving the following balance equations: Π nc = Π nc P Snc (a A1 a A2,a B1 a B2 ) θ I 1 I 2 G 1 G 2 π nc(θ) = 1. Focusing on feasible games in G j, we have the normalized limiting probability for a game state G j i as follows: ρ nc (G j i ) = π nc (G j i ) θ G 1 G 2 π nc (θ), Gj i Gj. On the other hand, when a cooperating strategy in S c is used, the system evolves in a different manner. Because both nodes choose C in the first stage, the incentive mechanism is activated and the access probabilities of both nodes are updated in the second stage depending on their roles in the first stage. Therefore, we need to distinguish the state notations for the second stage based on the requester s identity in the first stage. Here, we use I 2A = {Ik 2A : k = 1,...,4} and G 2A = {G 2A i : i = 1,...,8} to represent the states when A started a game, but use I 2B = {Ik 2B : k = 1,...,4} and G 2B = {G 2B i : i = 1,...,8} for the states when B started a game. Appendix C presents the analysis of state transitions with the cooperating strategies in S c. The corresponding aggregate transition probability matrix P Sc (a A1 a A2,a B1 a B2 ) is given in Table V of Appendix C. Similarly, we can obtain the limiting probability matrix of the states, denoted by Π c = {π c (θ) : θ I 1 I 2A I 2B G 1 G 2A G 2B }, by solving the following balance equations: Π c = Π c P Sc (a A1 a A2,a B1 a B2 ) θ I 1 I 2A I 2B G 1 G 2A G 2B π c (θ) = 1. The normalized limiting probability of a game state in G 1, G 2A, and G 2B is then given by ρ c (G 1 i ) = π c (G 1 i ) θ G 1 G 2A G 2B π c(θ), G1 i G1 ρ c (G 2n i ) = where n {A,B}. π c (G 2n i ) θ G 1 G 2A G 2B π c(θ), G2n i G 2n D. Tuning Factors for General Case Based on the normalized limiting probability, we can evaluate the expected utility for node n. Specifically, the expected utility of node n with a strategy (a A1 a A2,a B1 a B2 ) in S nc and S c is given by U n (a A1 a A2,a B1 a B2 ) (16) 8 [ i=1 ρnc (G 1 i )U n(a A1,a B1 ;G 1 i ;p A,p B ) +ρ nc (G 2 i )U n(a A2,a B2 ;G 2 i ;p A,p B ) ], if (a A1 a A2,a B1 a B2 ) S nc = 8 [ i=1 ρc (G 1 i )U n(a A1,a B1 ;G 1 i ;p A,p B ) +ρ c (G 2A i )U n (a A2,a B2 ;G 2A i ;µp A,εp B ) +ρ c (G 2B i )U n (a A2,a B2 ;G 2B i ;εp A,µp B ) ], if (a A1 a A2,a B1 a B2 ) S c. For (C NC,C NC) to be an NE, (C NC) should maximize the expected utility of A given that B takes the same strategy, and vice versa. This requirement can be translated into the following constraints: U A (C NC,C NC) U A (C C,C NC) (17) U A (C NC,C NC) U A (NC C,C NC) (18) U A (C NC,C NC) U A (NC NC,C NC) (19) U B (C NC,C NC) U B (C NC,C C) (20) U B (C NC,C NC) U B (C NC,NC C) (21) U B (C NC,C NC) U B (C NC,NC NC). (22) In addition, for (C NC,C NC) to achieve higher expected utility than (NC NC,NC NC), it is required that U A (C NC,C NC) U A (NC NC,NC NC) (23) U B (C NC,C NC) U B (NC NC,NC NC). (24)

7 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 7 Thus, we can determine the tuning factors µ and ε to satisfy the constraints in (17)-(24), so that (C NC,C NC) is an NE strategy better than the not-cooperating NE strategy(n C NC,NC NC). In a general case, we can numerically evaluate the related utilities with µ and ε in the feasible range (0 µ 1,1 ε pmax p h ) to find valid values that meet the above constraints. E. Tuning Factors for A Special Case In Section V-D, (17)-(24) give the constraints that the tuning factors need to meet to ensure that (C NC,C NC) is an NE better than not-cooperating. Furthermore, we can derive closed-form criteria for µ and ε for a special case with: Saturated traffic, where A and B are always busy with a packet arrival probability λ A = λ B = 1; and Symmetric parameters for A and B, including channel access probability p A = p B = p, transmission gain over direct link v(s A ) = v(s B ) = v 0, and transmission gain over indirect link v( s A ) = v( s B ) = δv, δ > 1. The relationship between v and c can be expressed as v c = ζv, 0 < ζ < 1. For this special case, we have the following conclusion regarding the tuning factors, which is proved in Appendix D. Theorem 1. Given the saturated and symmetric assumptions, (C NC,C NC) is an NE strategy of the repeated twostage game and achieves higher expected utility than (NC NC,NC NC) for both nodes, if and only if the tuning factors µ and ε satisfy µ = 0 (25) ε ε min = max(x 1,X 2,X 3,1) (26) where X 1, X 2, X 3 are given by 2 δ ζ +(3ζ +δ 2)p δp2 X 1 = ζp 2 δ ζ +(3ζ 1)p 2p2 X 2 = ζp 2 δ ζ +4ζp 4p2 X 3 =. ζp VI. NUMERICAL RESULTS AND DISCUSSIONS In this section, we provide numerical results to validate our analysis in Section V for the general case and the special case with saturated traffic and symmetric settings. We show that a set of valid tuning factors can be found for the incentive mechanism in the repeated two-stage game. With such valid tuning factors, the cooperating strategy (C NC,C NC) can attain an NE and achieve higher utility than the notcooperating NE strategy (NC NC,NC NC). Further, we define an overall system utility and determine an optimal choice of tuning factors that maximize the system utility. It is shown that the tuning factors selected according to the criteria in Theorem 1 can closely approach the optimal choice in the achieved utility. Fig. 4. Set of valid tuning factors (ε,µ) in the general case, where λ A = 0.1, λ B = 0.2, p A = p B = 0.2, s A = 0.1, s A = 0.9, s B = 0.3, s B = 0.7, p min = 0, p max = 1, and c = A. General Case In the general case, the system parameters for the paired cooperative nodes can be any reasonable values and they are not necessarily being symmetric. Fig. 4 shows a set of valid tuning factors (in the shaded area) based on the constraints in (17)-(24) to guarantee that the cooperating strategy (C NC,C NC) reaches an NE and achieves higher utility than the not-cooperating strategy (NC NC,NC NC). Here, the upper boundary of the shaded area gives the maximum valid µ for each possible ε. Any point in the shaded area can be considered as a valid pair of µ and ε. To demonstrate the utility performance of the cooperating NE strategy, Fig. 5 shows the expected utilities of A and B with varying packet arrival probability λ A = λ B = λ. The other system parameters are the same as those for Fig. 4. Here, we assume that the requester chooses the highest µ that satisfies the constraints in (17)-(24) for the minimum decrease of access probability (least charge), while the helper chooses the highest ε for the maximum increase of access probability (largest reward). As seen in Fig. 5(a), the cooperating strategy (C NC,C NC) attains the highest utility for A, while the utility also increases with λ. A similar trend is observed in Fig. 5(b). Therefore, the cooperating strategy (C NC,C NC) achieves an NE, since neither node will deviate to play other strategies assuming that each node knows that its partner is playing (C N C). Another interesting observation is that, when the packet arrival probability is low (e.g., less than 0.3), the strategy (C C,C NC) also results in high utility for A, while the strategy (C NC,C C) achieves high utility for B. However, A and B choose different actions in these two strategies. Thus, an agreement cannot be reached simultaneously at both nodes, and they cannot be viewed as NE strategies. As derived in the analysis in Section V, the tuning factors should satisfy certain constraints to ensure that the cooperating strategy (C NC,C NC) attains an NE. Fig. 6 presents the percentage of such valid tuning factors over the entire set of available values in the feasible range, i.e., 0 µ 1 and 1 ε pmax p h. Here, we first choose an interval to obtain a large number of feasible values in the feasible range, e.g., N µ values for µ and N ε values for ε. Then there are totally N µ N ε

8 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 8 Expected utility of A (C NC,C NC) (C C,C NC) (NC C,C NC) (NC NC,C NC) (NC NC,NC NC) Percentage of valid tuning pairs λ (a) Expected utility of A λ (a) Expected utility of B (C NC,C NC) (C NC,C C) (C NC,NC C) (C NC,NC NC) (NC NC,NC NC) Percentage of valid tuning pairs λ (b) Expected utility of B. Fig. 5. Expected utilities of A and B v.s. packet arrival probability λ A = λ B = λ with different strategies, where p A = p B = 0.4, s A = 0.5, s A = 0.9, s B = 0.3, s B = 0.7, and c = candidate pairs for the tuning factors. Each candidate pair is tested to see if it satisfies (17)-(24). A counter N ne is used to record the number of valid tuning pairs that satisfy these constraints. The percentage, defined by Nne N µn ε, can indicate how hard it is to find a valid tuning pair. A small percentage implies it is difficult to find a valid tuning pair that ensures an NE is reached under the current system setup. On the other hand, a large percentage means there exist many valid tuning pairs that guarantee (C NC,C NC) to be an NE. Fig. 6(a) shows how the percentage varies with packet arrival rate λ A = λ B = λ. For all numerical results in Section VI, we take p min = 0, p max = 1, and N µ = N ε = As seen in Fig. 6(a), when λ goes from 0.1 to 0.35, the percentage increases, because it is easier to find valid tuning factors when there is busier incoming traffic but still at a low level. When λ exceeds 0.35, the percentage drops, which means the set of valid tuning factors shrinks gradually. Finally, when the traffic is almost saturated with the arrival probability approaching 1, the percentage goes to the lowest. In addition, Fig. 6(b) shows how the percentage varies with channel access probability p A = p B = p. As seen, when p goes up, both nodes try to access channel more aggressively, which leads to more collisions. Consequently, it becomes harder to find valid tuning factors for a stable cooperating strategy p (b) Fig. 6. (a) Percentage of valid tuning factors v.s. λ, with λ A = λ B = λ and p A = p B = 0.4. (b) Percentage of valid tuning factors v.s. p, with p A = p B = p and λ A = λ B = 0.2. For both cases, s A = 0.1, s A = 0.9, s B = 0.3, s B = 0.7, and c = B. Saturated and Symmetric Case In this section, we show numerical results to validate Theorem 1 for the selection of tuning factors in the saturated and symmetric case. Theorem 1 proves that the requester must be forbidden from accessing the channel in the second stage, but it is still possible to reach cooperation agreement as long as the reward to the helper is high enough. In other words, Theorem 1 gives a lower bound ε min for the tuning factor ε to adapt the access probability of the helper, while the upper bound is naturally 1 pmax p since we limit 1 ε p to ensure p εp p max. Then, any ε between the lower bound and the upper bound together with µ = 0 form a valid pair of tuning factors to attain an NE with the cooperating strategy and provide higher utility than not-cooperating. Fig. 7 shows the dependence of the lower bound ε min on channel access probability p. The solid line shows the theoretical results from (26) of Theorem 1. The triangle markers indicate the results that are obtained numerically by searching N ε = 1000 samples of ε in the feasible range. As seen, the theoretical lower bound determined by Theorem 1 accurately matches the real lower bound that is numerically located. Closer examination shows that ε min > 1 when p < 0.4, which means the helper requires to tune up its access probability as incentive to help. In contrast, when p > 0.4, we see that

9 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 9 ε min Numerical Theorem p Fig. 7. Lower bound of valid ε v.s. channel access probability p, with δ = 1.5 and ζ = 0.35, where v(s A ) = v(s B ) = v, v( s A ) = v( s B ) = δv, and v c = ζv. System utility Selfish choice Optimal choice p Fig. 8. System utility with optimal choice and selfish choice of tuning factors, with λ A = 0.1, λ B = 0.3, s A = 0.1, s A = 0.9, s B = 0.3, and s B = 0.7. ε min = 1, which implies the helper may choose to help even without a reward. C. Optimal Choice v.s. Selfish Choice As shown in the theoretical analysis in Section V and the above numerical results, multiple pairs of valid tuning factors can be used with the cooperating strategy to attain an NE with higher utility than not-cooperating. From the perspective of individual nodes, any pair of such tuning factors are good since they guarantee to reach an NE with the highest utility for each node, as shown in Fig. 5. Nonetheless, such choices of individual nodes may not be the best concerning the entire system. To evaluate the overall system benefit, we define a new system-level utility as ( ) UA (µ,ε) U sys (µ,ε) = min maxu A (µ,ε), U B (µ,ε) (27) maxu B (µ,ε) where U n (µ,ε) is used to simplify the notation U n (a A1 a A2,a B1 a B2 ) in (16) for the expected utility of node n with strategy (a A1 a A2,a B1 a B2 ). As seen, this overall system utility is actually the minimum normalized utility achieved by the cooperating nodes. Since multiple pairs of valid tuning factors are available, a best pair can be selected by solving the multi-object optimization problem in (27). Using MATLAB Optimization ToolBox [30], we can find the optimal choice that maximizes the system utility. Fig. 8 compares the system utility with the optimal choice to that of a selfish choice. The selfish choice means the highest µ and the highest ε that satisfy constraints (17)-(24). That is, among all valid tuning factors, the selfish choice tends to reward the helper the most while charging the requester the least. It is observed in Fig. 8 that the optimal choice achieves slightly higher system utility but shows a more stable trend. This is because the optimal choice selects the tuning factor that maximizes the overall system utility. In contrast, as the selfish choice sets the tuning factors from the perspective of individual nodes, this leads to minor utility fluctuations but the utility loss is surprisingly small (less than 5%). In other words, by properly setting the tuning factors, the cooperating strategy is only subject to slight system efficiency degradation due to the selfish behavior of the nodes. In view of the high efficiency and distributed nature of the proposed strategy, it is not always necessary or cost-effective to employ the optimal choice since a central controller needs to collect system information, which is less favourable especially when there is no significant utility improvement. VII. CONCLUSIONS In this work, we used a game theoretical approach to study cooperative MAC protocols for wireless networks. We aim to find out: 1) Whether cooperation agreement can be reached on both sides of the paired cooperative nodes given that each node only seeks to maximize its own utility; 2) whether cooperation can attain higher utility than non-cooperation; and 3) how to incorporate an incentive mechanism into the MAC protocol so that higher utility is provided by a cooperating NE strategy. Specifically, we consider a bidirectional cooperative communication scenario with two half-duplex peer nodes that are matched to relay data for each other using slotted Aloha for channel access. A one-stage game is formulated and shown that cooperating is not an NE strategy, while not-cooperating is an NE strategy if the access cost is low enough. To further explore cooperation gain, the one-stage game is extended to a two-stage game with an incentive mechanism, in which the channel access probabilities of cooperating nodes are adapted by tuning factors to charge the cooperation requester and reward the helper. When the two-stage game is repeated, we formulated a Markov chain to characterize the system state evolution with different strategies. Valid tuning factors are determined to ensure that the cooperating strategy (C NC,C NC) reaches an NE and achieves higher utility than the not-cooperating NE strategy (NC NC,NC NC). In addition to the general case, we further examined a saturated and symmetric case, for which closed-form lower bound is derived for the tuning factor to reward the helper. The theoretical analysis is verified by numerical results, which demonstrate that with valid tuning factors the cooperating strategy (C NC,C NC) indeed reaches an NE and provides the highest utility for each individual node. The valid tuning factors selected according to the derived criteria are also compared with an optimal choice that aims to maximize

10 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 10 the overall system utility. It is shown that the optimal choice results in less fluctuations in system utility, while the tuning factors satisfying our derived criteria can achieve reasonably high utility though slightly lower than that of the optimal choice. APPENDIX A IMPORTANT NOTATIONS. Table III summaries the important notations used in the one-stage game formulation, two-stage game formulation and Markov analysis. APPENDIX B ANALYSIS OF STATE TRANSITIONS FOR NON-COOPERATION SET As defined in Section V-B, the system states of the two-stage game are classified into three groups for stage j (j = 1,2), namely, temporary states T j, game states G j, and intermediate states I j. Also, since we are particularly interested in the cooperating strategy (C NC,C NC), the eight related strategies are classified into two groups in Section V-C, namely, noncooperation set S nc and cooperation set S c. According to the system model in Section III, we can obtain the transitions among I j, G j and T j in Fig. 9(a). Here, we denote the transition probability matrix between I j and T j by P IT = {p kl : k = 1,...,4,l = 1,...,4}, where p kl is the transition probability from state I j k to Tj l Note that the superscript j for the stage is skipped because. P IT is actually the same for the two stages. It is also the case for the other transition probability matrices among I j, G j and T j with the strategies in S nc. Then, according to Fig. 9(a), we can write P IT as P IT = p A 1 p A 0 0 p B 0 1 p B 0 p A p B p B (1 p A ) p A (1 p B ) (1 p A )(1 p B ). (28) Denoting the transition probability matrix between T j and G j by P TG = {p li : l = 1,...,4,i = 1,...,8}, where p li is the transition probability from state T j l to G j i, we have (29) on top of the next page. Similarly, the transition probability matrix between T j and I j is denoted by P TI = {p lk : l = 1,...,4,k = 1,...,4}, where p lk is the transition probability from state T j l to I j k given by, (1 λ A )(1 λ B ) (1 λ B )λ A λ B (1 λ A ) λ A λ B 0 1 λ P TI = B 0 λ B λ A λ. A (30) The transition probability matrix between G j and T j with action (a A,a B ) is denoted by P GT (a A,a B ) = {p il (a A,a B ) : i = 1,...,8,l = 1,...,4}, where p il (a A,a B ) is the transition probability from state G j i to Tj l with action (a A,a B ). As mentioned earlier, the transition probability matrices are the same in the two stages. Hence, (a A,a B ) can be action (a A1,a B1 ) in the first stage, or (a A2,a B2 ) in the second stage. We can easily obtain P GT (a A,a B ) for three feasible actions in the TABLE III IMPORTANT NOTATIONS. Symbol Definition n Any peer node in N N Set of cooperative peer source nodes, N = {A,B} h / r Cooperation helper / requester node in N C / N C Cooperating action / not-cooperating action a n Action of node n, a n {C,NC} (a A,a B ) Strategy of peer nodes A and B in one-stage game (a A1 a A2, a B1 a B2 ) Strategy of peer nodes A and B in two-stage game p n Channel access probability of node n p min / p max Lower / upper bound of channel access probability s n / s n Transmission success probability of node n over direct / indirect link c Access cost for transmission v(s) Function of transmission gain with success probability s λ n Packet arrival probability of node n U n(a A,a B ;g; p A,p B ) Utility of node n in one-stage game with strategy (a A,a B ) U n(a A1 a A2, a B1 a B2 ) Expected utility of node n with strategy (a A1 a A2,a B1 a B2 ) µ Tuning factor to reduce access probability of requester to µp r, 0 µ 1 ε Tuning factor to increase access probability of helper to εp h, 1 ε pmax p h ε min Lower bound for tuning factor ε λ Symmetric packet arrival probability of peer nodes, λ A = λ B = λ p Symmetric channel access probability of peer n- odes, p A = p B = p v Symmetric transmission gain over direct links, v(s A ) = v(s B ) = v δ Ratio of symmetric gain over indirect and direct links, v( s A ) = v( s B ) = δv ζ Factor related with v and c,v c = ζv,0 < ζ < 1 S nc / S c Set of non-cooperation / cooperation strategies G Set of game states in one-stage game, G = {G i : i = 1,...,8} G j Set of game states in stage j of two-stage game, G j = {G j i : i = 1,...,8} T j Set of temporary states in stage j of two-stage game, T j = {T j l : l = 1,...,4} I j Set of intermediate states in stage j of two-stage game, I j = {T j k : k = 1,...,4} P IT Transition probability matrix between I j and T j P TG Transition probability matrix between T j and G j P TI Transition probability matrix between T j and I j P GT (a A,a B ) Transition probability matrix between G j and T j with action (a A,a B ) P r IT Transition probability matrix between I j and T j with requester r P r GT (a A2,a B2 ) Transition probability matrix between G j and T j with requester r and (a A2,a B2 ) P Snc (a A1 a A2, a B1 a B2 ) Aggregate state transition probability matrix with strategies in S nc P Sc (a A1 a A2, a B1 a B2 ) Aggregate state transition probability matrix with strategies in S c ρ nc(g j i ) Normalized limiting probability of game state G j i with S nc, G j i Gj ρ c(g 1 i ) Normalized limiting probability of game state G 1 i with S c, G 1 i G1 ρ c(g 2n i ) Normalized limiting probability of game state G 2n i with S c, G 2n i G 2n, where node n {A,B}

11 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 11 (1 λ A )(1 λ B ) λ A (1 λ B ) (1 λ A )λ B λ A λ B (1 λ A )(1 λ B ) (1 λ A )λ B λ A (1 λ B ) λ A λ B λ P TG = B λ B λ A λ A (29) strategies of S nc, i.e., (NC,NC), (C,NC), and (NC,C), as follows: P GT (NC,NC) = p A 1 p A 0 0 p B 0 1 p B 0 p A p B (1 p A )p B p A (1 p B ) (1 p A )(1 p B ) (31) p B 0 1 p B 0 p A 1 p A 0 0 p A p B (1 p A )p B p A (1 p B ) (1 p A )(1 p B ) p B 0 1 p B 0 P GT (C,NC) = 0 p B 0 1 p B (32) p B 0 1 p B p B 0 1 p B p A 1 p A P GT (NC,C) = 0 0 p A 1 p A. (33) p A 1 p A p A 1 p A It is noted in Fig. 9(a) that each transition from I j to G j or from I j to itself passes through T j. Since we are interested in the system states that eventually converge to a game, we can reduce Fig. 9(a) to Fig. 9(b) by hiding states in T j. Then, we obtain the aggregate transition probability matrix P Snc (a A1 a A2,a B1 a B2 ) in Table IV. Agame is started No game is started No game is started Agame is started Channel access event Packet arrival event (a) State transitions. (b) Reduced state transitions for S nc. r=b r=a APPENDIX C ANALYSIS OF STATE TRANSITIONS FOR COOPERATION SET For all the strategies in S c, both nodes take action C in the first stage. To respond to their cooperation actions, the channel access probabilities of the requester and the helper are adapted in the second stage by the tuning factors, µ and ε. Depending on whetheraor B requests cooperation in the first stage, P IT in (28) is modified to P A IT and PB IT, respectively: P A IT = µp A 1 µp A 0 0 εp B 0 1 εp B 0 µp A εp B εp B (1 µp A ) µp A (1 εp B ) (1 µp A )(1 εp B ) P B IT = εp A 1 εp A 0 0 µp B 0 1 µp B 0 µp A εp B µp B (1 εp A ) εp A (1 µp B ) (1 εp A )(1 µp B ) Comparing all the strategies in S nc, we can see that the cooperative nodes can take three feasible actions, i.e., (NC,NC), (C,NC), and (NC,C), in the first or second stage. Accordingly, we have obtained in (31) and (32) P GT (a A,a B ) for S nc with (a A,a B ) among these three actions. On the other hand, any strategy in S c has action (C,C) in the first stage and an action among {(NC,NC),(C,NC),(NC,C)} in the. (c) Reduced state transitions for S c. Fig. 9. State transitions among I j, G j and T j with strategies in S nc and S c. second stage. Thus, when deriving P GT for the strategies in S c, we need to distinguish action (a A1,a B1 ) in the first stage and action (a A2,a B2 ) in the second stage. In particular, P GT (a A1,a B1 ) with only (a A1,a B1 ) = (C,C) is given by T P GT (C,C) = Further, P GT (a A2,a B2 ) for action (a A2,a B2 ) in the second stage depends on whether node A or B requests cooperation in the first stage. For (a A2,a B2 ) = (NC,NC), (C,NC), and (NC,C),P A GT (a A2,a B2 ) andp B GT (a A2,a B2 ) are obtained as P A GT (NC,NC) = µp A 1 µp A 0 0 εp B 0 1 εp B 0 µp A εp B (1 µp A )εp B µp A (1 εp B ) (1 µp A )(1 εp B )

12 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 12 TABLE IV AGGREGATE TRANSITION PROBABILITY MATRIX FOR STRATEGIES IN S nc. I1 1 I1 4 G 1 1 G1 8 I1 2 I2 4 G 2 1 G2 8 I1 1 I1 4 P IT P TI P IT P TG 0 0 G 1 1 G P GT (a A1,A B1 )P TI 0 I1 2 I P IT P TI P IT P TG G 2 1 G2 8 P GT (a A2,A B2 )P TI P B GT (NC,NC) = µp B 0 1 µp B 0 εp A 1 εp A 0 0 µp A εp B (1 εp A )µp B εp A (1 µp B ) (1 εp A )(1 µp B ) P A GT (C,NC) = εp B 0 1 εp B 0 0 εp B 0 1 εp B P B GT (C,NC) = µp B 0 1 µp B µp B 0 1 µp B P A GT (NC,C) = µp A 1 µp A p A 1 p A P B GT (NC,C) = εp A 1 εp A εp A 1 εp A. As seen above, because P IT and P GT (a A,a B ) vary with different strategies in S nc and S c, we have updated P IT and P GT (a A,a B ) according to the identity of the cooperation requester in the first stage. On the contrary, P TG and P TI are insensitive to the strategies in S nc and S c and remain the same as given in (29) and (30). Then, similar to Fig. 9(b), we can reduce the state transition diagram in Fig. 9(a) to Fig. 9(c) with the strategies in S c. The aggregate transition probability matrix for S c can be obtained as in Table V. APPENDIX D PROOF OF THEOREM 1 From the Markov chain analysis in Section V-C, we can numerically evaluate the limiting probability of the system states in the saturated and symmetric case, e.g., by using MAT- LAB Symbolic Math Toolbox [30]. The normalized limiting probabilities with the strategies in S nc and S c are respectively derived by { 1 ρ nc (g) = 4, g = G1 4,G1 8,G2 4,G2 8 (34) 0, otherwise 1 4, g = G1 4,G 1 8 µ(εp 1) ϕ, g = G2A 4,G 2A 8 ρ c (g) = 4(2µεp ε µ) (35) ε(µp 1) ψ, g = G2B 4 4(2µεp ε µ),g2b 8 0, otherwise. Then, based on the expected utility defined in (16) in Section V-D, we obtain the following proposition. Proposition 3. Given the saturated and symmetric assumptions, the expected utility with the strategies in S nc and S c for nodes A and B satisfy U B (C NC,C C) = U A (C C,C NC) (36) U B (C NC,NC C) = U A (NC C,C NC) (37) U B (C NC,NC NC) = U A (NC NC,C NC) (38) U B (C NC,C NC) = U A (C NC,C NC) (39) U B (NC NC,NC NC) = U A (NC NC,NC NC). (40) Proof. TakeU B (C NC,C C) andu A (C C,C NC) in (36) as an example. Applying the normalized limiting probability in (35) to the definition of expected utility in (16), we have U B (C NC,C C) = (41) 1 4 U B(C,C;G 1 4;p A,p B )+ 1 4 U B(C,C;G 1 8;p A,p B ) +ϕu B (NC,C;G 2A 4 ;µp A,εp B ) +ϕu B (NC,C;G 2A 8 ;µp A,εp B ) +ψu B (NC,C;G 2B 4 ;εp A,µp B ) +ψu B (NC,C;G 2B 8 ;εp A,µp B ) U A (C C,C NC) = (42) 1 4 U A(C,C;G 1 4;p A,p B )+ 1 4 U A(C,C;G 1 8;p A,p B ) +ϕu A (C,NC;G 2A 4 ;µp A,εp B ) +ϕu A (C,NC;G 2A 8 ;µp A,εp B ) +ψu A (C,NC;G 2B 4 ;εp A,µp B ) +ψu A (C,NC;G 2B 8 ;εp A,µp B ). As seen, (41) and (42) involve the utility functions defined in (1), (2), (5), and (7). Given the saturated and symmetric assumptions, we can easily infer that U B (C,C;G 1 4 ;p A,p B ) = U A (C,C;G 1 8 ;p A,p B ) U B (C,C;G 1 8 ;p A,p B ) = U A (C,C;G 1 4 ;p A,p B ) U B (NC,C;G 2A 4 ;µp A,εp B ) = U A (C,NC;G 2B U B (NC,C;G 2A 8 ;µp A,εp B ) = U A (C,NC;G 2B U B (NC,C;G 2B 4 ;εp A,µp B ) = U A (C,NC;G 2A U B (NC,C;G 2B 8 ;εp A,µp B ) = U A (C,NC;G 2A 8 ;εp A,µp B ) 4 ;εp A,µp B ) 8 ;µp A,εp B ) 4 ;µp A,εp B ) (43) Applying the equalities in (43) to (41) and (42), we conclude that (36) holds. Likewise, (37)-(39) can be proved. According to Proposition 3, the tuning factors µ and ε only need to satisfy (17)-(19) and (23) to guarantee that (C NC,C NC) is an NE strategy that offers utility not less than the not-cooperating NE strategy (NC NC,NC NC).

13 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 13 TABLE V AGGREGATE TRANSITION PROBABILITY MATRIX FOR STRATEGIES IN S c. I1 1 I1 4 G 1 1 G1 4 G 1 5 G1 8 I1 2A I4 2A I1 2B I4 2B G 2A 1 G 2A 8 G 2B 1 G 2B 8 I1 1 I1 4 P IT P TI P IT P TG P IT P TG G 1 1 G P GT (a A1,a B1 )P TI G 1 5 G P GT (a A1,a B1 )P TI 0 0 I1 2A I4 2A P A IT P TI 0 P A IT P TG 0 I1 2B I4 2B P B IT P TI 0 P B IT P TG G 2A 1 G 2A 8 P A GT (a A2,a B2 )P TI G 2B 1 G 2B 8 P B GT (a A2,a B2 )P TI The constraint in (17) requires that U A (C NC,C NC) U A (C C,C NC) = εµpv[2(1 δ)+(2δ 3)(ε+µ)p+2(2 δ)εµp2 ] 4(2εµp µ ε) 0. (44) Because 0 < εp < 1 and 0 < µp < 1, we see that the denominator in (44) satisfy 4(2εµp µ ε) < 0. (45) Moreover, according to Second Derivatives Test Lemma in [31], it can be proved that, when 1 ε 1 p and 0 µ 1, the term inside the square brackets in the numerator of (44) satisfy 2(1 δ)+(2δ 3)(ε+µ)p+2(2 δ)εµp 2 < 0. (46) Therefore, based on (45) and (46), we also have U A (C NC,C NC) U A (C C,C NC) 0. (47) Then, we conclude from (44) and (47) that U A (C NC,C NC) U A (C C,C NC) = 0 µ = 0. (48) To satisfy (18), it is required that U A (C NC,C NC) U A (NC C,C NC) = v[δ 1 (δ +1)p+(µ+ε)pζ +(3p 1)(1 ζ)+(δ 2εµ)p 2 ] 0 ε 2 δ ζ +(3ζ +δ 2)p δp2 ζp X 1. (49) Similarly, the constraints in (19) and (23) are equivalent to U A (C NC,C NC) U A (NC NC,C NC) = v[δ 1 2p+(µ+ε)pζ +(3p 1)(1 ζ)+2(1 εµ)p 2 ] 0 ε X 2 (50) U A (C NC,C NC) U A (NC NC,NC NC) = v[δ 2+ζ 4ζp+(µ+ε)pζ +2(2 εµ)p 2 ] 0 ε X 3. (51) Therefore, we can prove Theorem 1 from (48)-(51) and the constraint 1 ε 1/p. REFERENCES [1] W. Zhuang and Y. Zhou, A survey of cooperative MAC protocols for mobile communication networks, Journal of Internet Technology, vol. 14, no. 4, pp , [2] W. Zhuang and M. Ismail, Cooperation in wireless communication networks, IEEE Wireless Commun. Mag., vol. 19, no. 2, pp , [3] Y. Li, P. Wang, D. Niyato, and W. Zhuang, A hierarchical framework of dynamic relay selection for mobile users and profit maximization for service providers in wireless relay networks, Wireless Communications and Mobile Computing, vol. 14, no. 12, pp , [4] M. J. Osborne, An Introducttion to Game Theory. Oxford University Press, [5] J. N. Laneman, D. N. C. Tse, and G. W. Wornell, Cooperative diversity in wireless networks: Efficient protocols and outage behavior, IEEE Trans. Inform. Theory, vol. 50, no. 12, pp , [6] T. Nechiporenko, K. T. Phan, C. Tellambura, and H. H. Nguyen, On the capacity of Rayleigh fading cooperative systems under adaptive transmission, IEEE Trans. Wireless Commun., vol. 8, no. 4, pp , [7] P. Ju, W. Song, and D. Zhou, Survey on cooperative medium access control protocols, IET Communications, vol. 11, no. 9, pp , [8] P. Liu, Z. Tao, S. Narayanan, T. Korakis, and S. S. Panwar, Coop- MAC: A cooperative MAC for wireless LANs, IEEE J. Select. Areas Commun., vol. 25, no. 2, pp , [9] P. Ju, W. Song, and D. Zhou, An enhanced cooperative MAC protocol based on perceptron training, in Proc. IEEE WCNC, [10], Extending IEEE based cooperative MAC for mobile relays via link availability prediction, in Proc. IEEE ICC, [11] H. Shan, H. Cheng, and W. Zhuang, Cross-layer cooperative MAC protocol in distributed wireless networks, IEEE Trans. Wireless Commun., vol. 10, no. 8, pp , [12] W. Song, P. Ju, A. Jin, and Y. Cheng, Distributed opportunistic two-hop relaying with backoff-based contention among spatially random relays, IEEE Trans. Veh. Technol., vol. 64, no. 5, pp , [13] V. Srivastava, J. Neel, A. MacKenzie, R. Menon, L. Dasilva, J. Hicks, J. Reed, and R. Gilles, Using game theory to analyze wireless ad hoc networks, IEEE Communications Surveys & Tutorials, vol. 7, no. 4, pp , [14] D. Yang, X. Fang, and G. Xue, Game theory in cooperative communications, IEEE Wireless Commun. Mag., vol. 19, no. 2, pp , [15] M. Janzamin, M. Pakravan, and H. Sedghi, A game-theoretic approach for power allocation in bidirectional cooperative communication, in Proc. IEEE WCNC, [16] B. Wang, Z. Han, and K. J. R. Liu, Distributed relay selection and power control for multiuser cooperative communication networks using Stackelberg game, IEEE Trans. Mobile Comput., vol. 8, no. 7, pp , [17], Distributed relay selection and power control for multiuser cooperative communication networks using buyer/seller game, in Proc. IEEE INFOCOM, 2007, pp [18] J. Huang, Z. Han, M. Chiang, and H. V. Poor, Auction-based resource allocation for cooperative communications, IEEE J. Select. Areas Commun., vol. 26, no. 7, pp , [19] G. Zhang, L. Cong, L. Zhao, K. Yang, and H. Zhang, Competitive resource sharing based on game theory in cooperative relay networks, ETRI Journal, vol. 31, no. 1, pp , [20] D. Yang, X. Fang, and G. Xue, Truthful auction for cooperative communications, in Proc. IEEE MOBIHOC, 2011, pp

14 IEEE TRANSACTIONS ON VEHICULAR TECHNOLOGY (ACCEPTED) 14 [21] Y. Chen and S. Kishore, A game-theoretic analysis of decode-andforward user cooperation, IEEE Trans. Wireless Commun., vol. 7, no. 5, pp , [22] Y. Zhou, J. Liu, L. Zheng, C. Zhai, and H. Chen, Link-utility-based cooperative MAC protocol for wireless multi-hop networks, IEEE Trans. Wireless Commun., vol. 10, no. 3, pp , [23] V. Mahinthan, L. Cai, J. W. Mark, and X. Shen, Maximizing cooperative diversity energy gain for wireless networks, IEEE Trans. Wireless Commun., vol. 6, no. 7, pp , [24] H. Shan, W. Zhuang, and Z. Wang, Distributed cooperative MAC for multihop wireless networks, IEEE Commun. Mag., vol. 47, no. 2, pp , [25] M. Janzamin, M. Pakravan, and H. Sedghi, Theoretic approach for power allocation in bidirectional cooperative communication, in Proc. IEEE WCNC, [26] Y.-W. Hong, C.-K. Lin, and S.-H. Wang, Exploiting cooperative advantages in slotted ALOHA random access networks, IEEE Trans. Inform. Theory, vol. 56, no. 8, pp , [27] C.-K. Lin and Y.-W. Hong, On the finite-user stability region of slotted ALOHA with cooperative users, in Proc. IEEE ICC, [28] Y.-W. Hong, C.-K. Lin, and S.-H. Wang, On the stability region of two-user slotted ALOHA with cooperative relays, in Proc. IEEE International Symposium on Information Theory, [29] A. B. Mackenzie and S. B. Wicker, Stablity of multipacket slotted Aloha with selfish user and perfect information, in Proc. IEEE INFOCOM, [30] MathWorks, MATLAB R2013b, [31] J. Stewart, Calculus, 5th ed. Thomosn Learning, 2003, ch. 14, p Peijian Ju received B.Eng. and M.Eng. degrees from Huazhong University of Science and Technology, Wuhan, China, in 2009 and 2011, respectively. He is working toward the Ph.D. degree at the University of New Brunswick, Fredericton, NB, Canada. His research interests include cross-layer design and game theory for cooperative wireless networks. Wei Song (M 09-SM 14) received the Ph.D. degree in electrical and computer engineering from University of Waterloo, Waterloo, ON, Canada, in In 2009, she joined the Faculty of Computer Science, University of New Brunswick, Fredericton, NB, Canada, where she is now an Associate Professor. Her current research interests include mobile cloud computing, cooperative wireless networking, energy-efficient wireless networks, and device-todevice communications. She received a UNB Merit Award in 2014, a Best Student Paper Award from IEEE CCNC 2013, a Top 10% Award from IEEE MMSP 2009, and a Best Paper Award from IEEE WCNC She is the Communications/Computer Chapter Chair of IEEE New Brunswick Section. She is also an editor for IEEE Transactions on Vehicular Technology and Wireless Communications and Mobile Computing (Wiley).