Repeated Inter-Session Network Coding Games: Efficiency and Min-Max Bargaining Solution

Transcription

1 Repeated Inter-Session Networ Coding Games: Efficiency and Min-Max Bargaining Solution Hamed Mohsenian-Rad, Member, IEEE, Jianwei Huang, Senior Member, IEEE, Vincent W.S. Wong, Senior Member, IEEE, and Robert Schober, Fellow, IEEE Abstract Recent results have shown that selfish and strategic users do not have an incentive to participate in inter-session networ coding in a static non-cooperative game setting. Because of this, the worst-case networ efficiency i.e., the price-ofanarchy can be as low as %. In this paper, we show that if the same game is played repeatedly, then the price-of-anarchy can be significantly improved to 36%. In this regard, we design a grim-trigger strategy that encourages users to cooperate and participate in the inter-session networ coding. A ey challenge here is to determine a common cooperative coding rate that the users should mutually agree on. We resolve the conflict of interest among the users through a bargaining process, and obtain tight upper bounds for the price-of-anarchy which are valid for any possible bargaining scheme. Moreover, we propose a simple and efficient min-max bargaining solution that can achieve these upper bounds, as confirm through simulation studies. The coexistence of multiple selfish networ coding sessions as well as the co-existence of selfish networ coding and routing sessions are also investigated. Our results represent a first step towards designing practical inter-session networ coding schemes which achieve reasonable performance for selfish users. I. INTRODUCTION Since the seminal paper by Ahlswede et al. [1, a rich body of wor has been reported on how networ coding can improve performance in both wired and wireless networs [ [4. In general, networ coding is performed by jointly encoding multiple pacets either from the same user i.e., intra-session networ coding, e.g., as in [1, [ or from different users i.e., inter-session networ coding, e.g., as in [3 [5. A common assumption in most existing networ coding schemes is that the users are cooperative and do not pursue their own interests. However, this assumption can be violated in practice. In non-cooperative networ coding, each user individually decides on whether to use and how to use networ coding to maximize its payoff. However, in inter-session networ coding, users rely on each other as they need to receive some remedy pacets to decode the coded data that they receive at their destinations. This leads to a game among users. Recent results in [6, [7 show that if the inter-session networ coding game is played once i.e., as a static game, then users do not have the incentive to provide each other with the remedy pacets. H. Mohsenian-Rad is with the Electrical Engineering Department, University of California, Riverside, CA, USA, hamed@ee.ucr.edu. J. Huang is with the Information Engineering Department, the Chinese University of Hong Kong, China, jwhuang@ie.cuh.edu.h. V. Wong and R. Schober are with the ECE Department, University of British Columbia, Vancouver, BC, Canada, s: {vincentw, rschober}@ece.ubc.ca. Manuscript received on January 17, 11; revised on August 3, 1 and April 7, 13; and accepted on June 1, 13. This wor was supported by the Natural Sciences and Engineering Research Council NSERC of Canada. The review of this paper was coordinated by Prof. Suhas Diggavi. This paper has been presented in part in the IEEE International Conference on Computer Communications INFOCOM 1, San Diego, CA, March 1. Hence, no networ coding is performed at Nash equilibrium. This significantly affects the networ performance; the priceof-anarchy PoA, i.e., the worst-case efficiency compared with the optimal networ performance, can be only % [6. In this paper, we study the more realistic scenario where the inter-session networ coding game in [6 is liely to be played repeatedly. This reflects the case where users have many pacets to transmit. As users continue sending more pacets, they can tae into account the history of the game e.g., whether the other users have provided the needed remedy pacets in the past and plan their future actions accordingly. It is well nown that repeated interactions can encourage cooperation [8 [11. However, the ey challenge in our model is that it is not immediately clear for the inter-session networ coding users how they can cooperate. This introduces a bargaining problem among users to search for a mutually acceptable coding rate. We show that a good bargaining solution together with a grim-trigger strategy can encourage cooperation in inter-session networ coding. We also analyze the general properties of all possible bargaining schemes, and provide universal upper bounds on the PoA for any bargaining method. Finally, we show that the PoA in the repeated game can be improved by using a min-max bargaining scheme. The contributions of this paper are as follows: New Formulation: To the best of our nowledge, we are the first to formulate non-cooperative inter-session networ coding as a repeated resource allocation game. Equilibrium Strategy Design: We show that a grim-trigger strategy can form a subgame perfect equilibrium for the repeated inter-session networ coding game, as long as the networ coding users can agree on the inter-session networ coding rate. Reaching such an agreement is nontrivial in general. It involves solving a bargaining problem that resolves the conflict of interest among users. Performance Bounds for All Bargaining Schemes: We show that, for any bargaining method, the PoA of the repeated inter-session networ coding game in the studied networ is upper-bounded by 36%, 44%, and 48% when the networ has one networ coding pair and several routing sessions, two networ coding pairs, and one networ coding pair, respectively. Our results show that the improvement in the PoA of repeated inter-session networ coding games is not as drastic as most typical repeated games. We explain the reasons behind this observation. Efficient Bargaining: We propose a novel min-max bargaining method, which can reach the obtained performance upper bounds for the α-fair utility functions. The results in this paper are different from the existing

2 TABLE I SUMMARY OF THE RESULTS ON THE ACHIEVABLE POA REPEATED INTER-SESSION NETWORK CODING GAMES. FOR STATIC AND Players Static Game Repeated Game One Networ Coding Pair % 48% Two Networ Coding Pairs - 44% Networ Coding and Routing Sessions % 36% Reference [6 This Paper results on networ coding games, e.g., [6, [7, [1 [1. The studies in [1 [16, [, [1 focused on intra-session networ coding, while here we address inter-session networ coding. Similar to [7, [17, we study inter-session networ coding in a butterfly networ topology. However, we further investigate the impacts of users utility functions, lin costs, and the PoA. Moreover, unlie the system models in [7, [17, [18, [, we address the case where the networ includes both networ coding and pure routing users. Finally, we study repeated games, while the results in [6, [7, [1 [19 are for static networ coding games. A ey motivation of this study is our prior wor on static networ coding games in [6. A comparison of the main results in [6 and this paper is given in Table I. Compared to [6, the extension in this paper is non-trivial and reveals several interesting properties of repeated inter-session networ coding games. For example, we show that even if a grim-trigger strategy and an arbitrarily large discount factor are used, no coding can still remain a dominant strategy under certain repeated game scenarios. Moreover, unlie the system model in [6 that includes only a single networ coding pair, here we also consider the case with more than one networ coding pair. The new model allows us to study the co-existence of multiple selfish inter-session networ coding pairs, in addition to studying the co-existence of selfish networ coding pairs and selfish routing sessions. The rest of this paper is organized as follows. In Section II, we introduce the system model and review the static game results in [6. The repeated game is formulated in Section III. Our results on subgame perfect equilibrium, bargaining, and the PoA bounds are given in Section IV. The min-max bargaining solution and its PoA are discussed in Section V. Numerical results are presented in Section VI. Conclusions and directions for future wor are provided in Section VII. II. SYSTEM MODEL AND BACKGROUND Consider the networ topology in Fig. 1, which is usually referred to as a butterfly networ in the networ coding literature 1. It consists of N end-to-end users and M + 1 wired lins, where M 1. The bottlenec lin i, j is shared by all users N = {1,..., N}. For each user n N, the source and the destination nodes are denoted by s n and t n, respectively. We first distinguish two different types of users: Networ Coding Users in set M = {1,..., M, N M + 1,..., N}, who can perform inter-session networ coding. 1 Although the networ coding scenario in Fig. 1 is simple, it can be used as a building bloc for more general scenarios. For example, [, [3 showed that a networ can be modeled as a superposition of several butterfly networs. Thus, understanding Fig. 1 is a ey to study more general networs. s 1 s M s M+1 s N-M s N-M+1 s N v 1 y, z 1 1 y M+1 y N-M y,z N N v N C i N n1 C 1 v 1 C M v M M y n max z m, z N m 1 C N v 1 M1 NM C N v N m 1 j t N t N-M+1 t N-M t M+1 Fig. 1. A butterfly networ with N unicast sessions, called users. Users 1,..., M and N M + 1,..., N are networ coding users. Users M + 1,..., N M are routing users. As an example, users 1 and N can perform inter-session networ coding over lins i, j, s 1, t N, and s N, t 1. Pacet X 1 X N is obtained by joint encoding of pacets X 1 and X N. Routing Users in set N \M = {M + 1,..., N M}, who cannot perform inter-session networ coding. Networ coding users can mar their pacets e.g., by setting a single-bit flag in the pacet header for either routing or networ coding. However, routing users can setup all their pacets only for routing. At the intermediate node i, all pacets that are mared for routing are simply forwarded to node j through lin i, j. However, the pacets that are mared for networ coding are treated differently. Consider users 1 and N that form a networ coding pair. Let X 1 and X N denote two pacets which are mared for networ coding and are sent from nodes s 1 and s N to node i, respectively. Node i can encode pacets X 1 and X N e.g., using XOR encoding [3, and send the resulting pacet, denoted by X 1 X N, to node j and from there to t 1 and t N. Given the remedy data X 1 from side lin s 1, t N and the remedy data X N from side lin s N, t 1, nodes t N and t 1 can decode their own pacets X N and X 1 based on the encoded data they receive from the intermediate node j. Clearly, the benefit of networ coding is to reduce the traffic on bottlenec lin i, j, by sending only one pacet X 1 X N rather than two pacets X 1 and X N. We define the following notations for data rates in Fig. 1: y n : Transmission rate of routing pacets sent from node s n to node i, for each user n N. z n : Transmission rate of networ coding pacets sent from node s n to node i, for each user n M. v n : Transmission rate of remedy pacets sent from node s n over its side lin, for each user n M. In this paper, we mae the following ey assumption: Assumption 1: Users are autonomous and have full control over their transmission rates. The networ coding users individually indicate, via maring, whether their pacets should be encoded or simply be forwarded over lin i, j. Node i encodes pacets at rate min{z m, z N m+1 }, for each m M, and forwards the rest of the pacets, without encoding their contents at rate N n=1 y n + M m=1 z m z N m+1. Thus, the total rate on bottlenec lin i, j becomes N n=1 y n + M m=1 max{z m, z N m+1 }. 1 Destination nodes t n, for n = M + 1,..., N M, receive information at rate y n while nodes t m and t N m+1, for m = t M t1

3 1,..., M, receive information at rates y m +min{z m, v N m+1 } and y N m+1 +min{z N m+1, v m }, respectively. A. Utility, Cost, and Price Functions Each user n N has a utility function U n, representing its evaluation of its achieved data rate. Lin i, j has a cost function C, which depends on its total traffic load N n=1 y n + M m=1 max{z m, z N m+1 }. Similarly, each set of side lins s m, t N m+1 and s N m+1, t m have cost functions C m and C N m+1, which depend on their loads v m and v N m+1. Assumption : The utility functions U 1,..., U N are concave, non-negative, increasing, and differentiable [4. Assumption 3: The lin cost functions are given as Cq = a q and C m q = bm q for all m M, where a>, b m >, and q. These convex quadratic cost functions are related to linear price functions pq = aq and p m q = b m q. In fact, Cq = q pθdθ and C mq = q p mθdθ for all m M. Quadratic cost and linear price functions are the only cost and price functions that satisfy the four axioms of rescaling, consistency, positivity, and additivity in cost-sharing systems [5. They are often used in networ resource management cf. [6 [3 to model either actual transmission cost e.g., in dollars or simply the queueing delay on each lin. B. Optimization-based Resource Allocation Let y = y n, n N, v = v m, m M, and z = z m, m M. The networ aggregate surplus is defined as the total utility of the users minus the total cost of the lins: Sy, z, v = M m=1 [U m y m +min{z m, v N m+1 } + U N m+1 y N m+1 + min{z N m+1, v m } + N M n=m+1 U n y n M m=1 [C mv m + C N m+1 v N m+1 C N n=1 y n + M m=1 max{z m, z N m+1 }. Given complete nowledge and centralized control of the networ in Fig. 1, we can compute the efficient rate allocation by solving the following optimization problem: Problem 1 Networ Surplus Maximization Problem: maximize Sy, z, v y,z,v subject to y n, n N, z m, v m, m M. Let y S = y S 1, n N, v S = v S m, m M, and z S = z S m, m M be an optimal solution for Problem 1. We can verify that v S m = z S m for all m M, i.e., the networ coding users send the coded and remedy pacets at the same rate for the optimal rate allocation. C. Pricing and Resource Allocation Game If the networ has no centralized controller and Assumption 1 holds, pricing can be used to encourage efficient resource allocation in a distributed fashion [4. Given the rate vectors y and z from the users, the shared lin i, j can set a price N µy, z = n=1 y n + M m=1 max{z m, z N m+1 } for any uncoded data rate it carries, where price function pq is described in Assumption 3. For coded pacets, however, the bottlenec lin can set a separate reduced price N n=1 y n + M m=1 max{z m, z N m+1 } σy, z = β p. 3 Here, β, 1 is the price discrimination parameter, and the intuition is to charge less for coded pacets to encourage networ coding. Note that only the choice of β = 1 can avoid over- or under-charging of networ coding users [6. Assumption 4: Throughout this paper, we set β = 1. Given data rates v for the remedy pacets, for each m = 1,..., M, side lins s m, t N m+1 and s N m+1, t m set their prices as p m v m and p N m+1 v N m+1 for the data they carry. Networ coding users are charged as follows: User m pays the following payment to lin i, j: σy, z minz m, z N m+1 + µy, z y m + µy, z z m minz m, z N m+1 = µy, z y m +z m 1 β min{z m, z N m+1 }, and pays v m p m v m to lin s m, t N m+1. User N m + 1 maes similar payments to lins i, j and s N m+1, t m. Each routing user n N \M pays µy, z y n to lin i, j. Users set their rates to maximize their surplus, i.e., utility minus charges [6, [9. Each user s surplus also depends on the rates set by other users, leading to a game among users: Game 1 Non-cooperative Resource Allocation Game: Players: Users in set N. Strategies: Transmission rates y, z, and v. Payoffs: P 1,..., P N, where for any m = 1,..., M: P m y m, z m, v m ; y 1, z 1, v 1 = U m y m + min{z m, v N m+1 } v m p m v m y m + z m 1 β min{z m, z N m+1 } p N r=1 y r + M r=1 max{z r, z N r+1 }, P N m+1 y N m+1, z N m+1, v N m+1 ; y N m+1, z N m+1, v N m+1 = U N m+1 y N m+1 + min{z N m+1, v m } v N m+1 p N m+1 v N m+1 y N m+1 +z N m+1 1 β min{z m, z N m+1 } p N r=1 y r + M r=1 max{z r, z N r+1 }, and for the routing user n N \M: P n y n ; y n, z, v = U n y n y n p N r=1 y r + M m=1 max{z m, z N m+1 }. Here, y n =y 1,..., y n 1, y n+1,..., y N for any n = 1,..., N; z m = z 1,..., z m 1, z m+1,..., z M, z N M+1,..., z N and v m = v 1,..., v m 1, v m+1,..., v M, v N M+1,..., v N for any m = 1,..., M; and z m = z 1,..., z M, z N M+1,..., z N m, z N m+,..., z N and v m = v 1,..., v M, v N M+1,..., v N m, v N m+,..., v N for any m=n M+ 1,..., N. Game 1 is a static game and is played once. The repeated version of Game 1 will be formulated in Section III.

4 D. Efficiency and Price-of-Anarchy of Game 1 The selfish nature of the players in Game 1 leads to undesirable and inefficient networ performance. To see this, we first introduce the following definitions for future reference. Definition 1 Nash equilibrium: The non-negative rates y = y n, n N, v =v m, m M, and z =z m, m M form a Nash equilibrium of Game 1 if no user n N can increase its payoff by unilaterally changing its data rates. The Nash equilibrium predicts how Game 1 will be played. Definition Efficiency: For a certain choice of system parameters, the efficiency at Nash equilibrium y, z, v is the ratio of the achieved networ aggregate surplus Sy, z, v to the optimal networ aggregate surplus Sy S, z S, v S. Definition 3 Price-of-anarchy: The price-of-anarchy, denoted by PoAGame 1, Problem 1, is the worst-case efficiency of a Nash equilibrium among all possible choices of parameters number of users and the utility, cost, and price functions. Next, we notice that payoff P m for any m M is decreasing in v m. Thus, a selfish networ coding user m will always choose to send no remedy pacets to avoid payments over its side lin. Being aware of this issue, all networ coding users will not participate in networ coding, as they cannot decode any encoded pacets without the remedy pacets. The following results are from [6, Theorem 11. Theorem 1: a Game 1 has a unique Nash equilibrium. b At Nash equilibrium of Game 1, we have v m = z m = m M. 4 c If N = and M = 1 then PoA Game 1, Problem 1 = %. 5 9 d If N > and M = 1 then PoA Game 1, Problem 1 = 1 = %. 6 5 The PoA results in Theorem 1 are significantly less than the 67% PoA for a similar game with routing users only that is showed in [6. The results in Theorem 1 imply that although inter-session networ coding can potentially improve networ performance, it is more sensitive to selfish behavior than routing. Next, we show that we can design better strategies with better PoAs when Game 1 is played repeatedly. III. REPEATED INTER-SESSION NETWORK CODING GAME Consider the case where Game 1 is played repeatedly. That is, every time users play Game 1 called one stage, they will play Game 1 again with a probability δ. Parameter δ is the discount factor [8. A repeated game formulation is natural if users have many pacets to transmit. If Game 1 is played multiple times, then the strategy space of the users will expand to include their data rates at each stage of the game. Let y = y n, n N, z = z m, m M, and v = v m, m M denote the actions chosen by users at stage 1. At the beginning of stage, the rates that have been already played in stages 1,..., 1 form the history of the game, while the rates to be played in stages, +1,... are strategies of users. For notational simplicity, for each 1 l, we define R t=l = { y t, z t, v t} t=l. 7 In this regard, R 1 t=1 denotes the history and R t= denotes the strategies of the users, at each stage 1. Game Repeated Game 1: Players: Users in set N. Histories: Data rates R 1 t=1, at each stage 1. Strategies: Contingency plans for selection of rates R t= at each stage 1 for any given history profile R 1 t=1. Payoffs: Q n for each user n N, where at each 1, Q n R t= R 1 t=1 = t= δt P n y t, z t, v t. In Game, the single-stage payoffs P 1,..., P N are the same as in Game 1. Payoffs Q 1,..., Q N are the discounted summations of the users payoffs in the future. The term δ t denotes the probability that Game is played at stage t >, given that it is currently played at stage 1. Definition 4 Subgame: Given a history profile R 1 t=1 at stage 1 of Game, the rest of the repeated game at stages, + 1,... is defined as a subgame at stage. The solution concept for a repeated game is the subgame perfect equilibrium which is defined as follows [8: Definition 5 Subgame Perfect Equilibrium: A strategy profile R =1 is a subgame perfect equilibrium of Game, if at any stage, the restricted strategy profile R t= is a Nash equilibrium for any subgame at stage formed by every given history R 1 t=1. That is, at any stage and for any history profile, no user n N can increase its payoff Q n by unilaterally changing its own data rates in future stages. Definition 6 Efficiency: The efficiency at subgame perfect equilibrium R =1 is defined as the average efficiency among all stages of Game, where the efficiency for rates y, z, v at stage is defined according to Definition. Definition 7 Price-of-anarchy: The price-of-anarchy, denoted by PoAGame, Problem 1, is the worst-case i.e., the smallest efficiency at a subgame perfect equilibrium of Game among all possible choices of system parameters. Before we end this section, we note that the parameters of Game are assumed to be fixed at all stages of the game. IV. PUNISHMENT AND BARGAINING IN INTER- SESSION NETWORK CODING In this section, we analyze repeated Game and show the following. First, a grim-trigger strategy encourages users to cooperate. Second, if the networ coding users cooperate, they will select the same networ coding rates. Third, the common networ coding rate can be determined via bargaining. Finally, the PoA of Game is better than that of Game 1. A. Punishment and Grim-trigger Strategy At the end of each stage of Game, user m, for each m = 1,..., M, nows whether user N M +1 has cooperated i.e., sent enough remedy pacets such that user m can decode all

5 received encoded pacets during the current stage. Thus, user m can punish user N m+1 in the next stage, if user N m+1 has cheated. This is also true for user N m + 1. Networ coding users m and N m + 1 may consider various punishment strategies against a cheating user. For example, if user N m + 1 cheats at stage 1 of Game, then user m may select its data rates y m, z m, v m to minimize user N m + 1 s payoff in the next stage. Another option for user m is not to participate in networ coding by setting v m = z m =. Punishment strategies can be either limited scope, lasting for only a few stages, or unlimited scope, lasting until the game ends. In this paper, we assume that the punishment is not to participate in networ coding for the rest of the game. We will show that this punishment strategy can prevent cheating. To start with, we show that if users decide to cooperate, they must choose the same networ coding rates. Theorem : Assume that users select data rates y, z, and v at a stage of repeated Game with v m = z m > v N m+1 = z N m+1, 8 for any m = 1,..., M. In this case, neither user m nor user N m + 1 cheat, but user m wants to participate in networ coding with a higher rate than user N m + 1. Then, user m can switch to new rates ȳ m, v m, z m such that ȳ m = y m+z m z N m+1, v m = z 1 = v N m+1 = z N m+1, 9 to strictly increase its own payoff at stage, while eeping the payoff of all the other users unchanged at stage. The proof of Theorem is given in Appendix A. From Theorem, if users m and N m+1 do not plan to cheat and want to cooperate, they must choose the same coding rates: z m = v m = z N m+1 = v N m+1, 1. 1 These results can help us predict how networ coding users behave if they choose to cooperate. However, we still need to answer two questions: 1 Which common networ coding rate z m = v m = z N m+1 = v N m+1 = z m,n m+1 11 should users m and N m + 1 choose in stage? How do networ coding users m and N m+1 enforce cooperation such that they both have the incentive to send remedy pacets at the desired rate z m,n m+1? We will answer the second question first. The first question will be answered in Section IV-C when we discuss bargaining. Next, we explain how users behave at each stage 1 of repeated Game if 11 holds for a given z m,n m+1. For the ease of exposition, we define a new static game which is derived from Game 1 and is parameterized with z m,n m+1. Game 3 Reduced Game 1 for a Given z m,n m+1 : Players: Users in set N. Strategies: Transmission rates y, when for each m = 1,..., M, the networ coding rates v and z are fixed at z m = v m = z N m+1 = v N m+1 = z m,n m+1. 1 Payoffs: P n for each user n N as in Game 1. Games 1 and 3 differ only due to 1. Since the networ coding rates are pre-determined, the strategy of users in Game 3 is reduced to routing rates y only. From Theorem 1a, Game 1 has a unique Nash equilibrium. Clearly, the Nash equilibrium of Game 3 depends on the choice of parameter z m,n m+1. Given z m,n m+1, we denote the Nash equilibrium of Game 3 by y z. Therefore, the payoff for each user n N at Nash equilibrium of Game 3 is denoted by P n y z, z 1,N,..., z M,N M For example, for networ coding user 1, we have P 1 y z, z = U 1 y 1z + z 1,N z 1,N p 1 z 1,N y 1z + βz 1,N p N r=1 y rz + M m=1 z m,n m+1. We now return to repeated Game. Clearly, if the networ coding users agree on selecting their networ coding rates according to 11, then at each stage 1, the users simply select their routing data rates to be y = y z. This helps us to introduce a strategy profile that can enforce cooperation, answering our second question posed earlier in this section. Definition 8 Grim-trigger Strategy: Given a set of predetermined coding rates z m,n m+1, for all m = 1,..., M, a grim-trigger strategy for Game is defined as Step 1: For networ coding pairs m and N m + 1, where m = 1,..., M, always participate in networ coding adopting the coding rate z m,n m+1. All users choose their routing rates according to the Nash equilibrium of Game 3 for given z 1,N,..., z M,N M+1. Go to Step if users m or N m + 1, for any m = 1,..., M, deviate from coding rate z m,n m+1. Step : Refuse networ coding forever. That is, at any future stage, set networ coding rates v m = z m = for all m M and routing rates y n = y n for all n N. The above is an unlimited scope punishment. Both networ coding and routing users participate in the punishment as they all set their rates according to the new Nash equilibrium of Game 3 with no networ coding. We can show the following: Theorem 3: Given fixed common coding rates z = z m,n m+1, m = 1,..., M, there exists a δ min, 1 such that the grim-trigger strategy in Definition 8 forms a subgame perfect equilibrium for Game if and only if the discount factor is δ min δ 1 and we have P m y z, z P m y,, m M. 14 The proof is given in Appendix B. If 14 holds, then all networ coding users are better off to perform networ coding at rate z as in Step 1 instead of no coding as in Step. B. Examples and Comparison with Prisoner s Dilemma As an example, consider Fig. 1, with one networ coding pair N = and M = 1. The system parameters are set as U 1 x = log1 + x, U x =.75 log1 + x, 15 a = 1, b 1 =.5, b =.5, β =.5. 16

6 TABLE II PAYOFFS AT EACH STAGE OF GAME WHEN 15 AND 16 HOLD. User 1 User Strategy Cooperate Cheat Cooperate.19,.1 -.8,.14 Cheat.4, -.1.1,.8 TABLE III PAYOFFS AT EACH STAGE OF GAME WHEN 17 AND 18 HOLD. User 1 User Strategy Cooperate Cheat Cooperate.7, , -.6 Cheat.8, -.3.5, We can verify that if we select z 1, =.3, then y1z 1, =.18, yz 1, =, and at each stage of repeated Game, the users play a game according to Table II, where the numbers in each box indicate the payoffs for user 1 and user, respectively. In this example, the grim-trigger strategy is a subgame perfect equilibrium and the users always pay Cooperate, Cooperate if discount factor δ max{ , }.38. We note that the payoffs in Table II resemble the payoffs in the prisoner s dilemma game [8, p. 11. However, in general, Game has two ey differences with the prisoner s dilemma game. First, cooperation is not well-defined in Game, as it is not immediately clear for the networ coding users which common and cooperative coding rate z 1, they should choose. We will address this issue when we discuss bargaining in Section IV-C. Second, even if no user cheats and both users transmit the remedy pacets at the same rate as the coding pacets, cooperation may still remain non-beneficial for one or both users. This second property can be seen in our next example where the system parameters are set as U 1 x = x, U x =.x, 17 a = 1, b 1 =.1, b =., β = If we select z 1, =.1, then y 1z 1, =.45, y z 1, =, and at each stage of repeated Game, the users play a game according to Table III. In this case, while cooperation is beneficial for user 1, it is damaging for user, something that does not occur in a prisoner s dilemma game. Therefore, regardless of the value of the discount factor, users always pay Cheat, Cheat as the sub-game perfect equilibrium. C. Bargaining So far, we have assumed that the common networ coding rate z m,n m+1, for any m = 1,..., M, is given. In this section, we will discuss how the networ coding users m and N m + 1 can agree on the choice of z m,n m+1. Clearly, networ coding user m prefers to choose z m,n m+1 to maximize P my z m,n m+1, z m,n m+1 z m,n m+1, z m,n m+1, 19 where z m,n m+1 = z r,n r+1, r =1,..., m 1, m+1,..., M. Similarly, user N m + 1 would prefer the solution of maximize P N m+1y z m,n m+1, z m,n m+1 z m,n m+1, z m,n m+1. However, in either case, the solution may not be fair and mutually acceptable to both users m and N m+1. A natural way to resolve this is bargaining in cooperative game theory [33, where two players negotiate on the details of cooperation. One option is the well-nown Nash bargaining solution [34. However, this solution is usually computationally complex. Thus, we see to find an alternative bargaining approach that is simple yet efficient as we will see in Section V and based on modifying problems 19 and. But first, we analyze some of the general properties that hold in a repeated inter-session networ coding game for any bargaining solution. D. Upper Bounds on Price-of-Anarchy of Game For a pair of networ coding users m and N m+1, where m = 1,..., M, assume that given z m,n m+1, either or P m y z m,n m+1, z m,n m+1, z m,n m+1 < P m y, z m,n m+1,, z m,n m+1 >, P N m+1 y z m,n m+1, z m,n m+1, z m,n m+1 1 < P N m+1 y, z m,n m+1,, z m,n m+1 >. Then the grim-trigger strategy in Definition 8 is a subgame perfect equilibrium if and only if the common coding rate is z m,n m+1 =, 3 for any value of discount factor δ, 1. That is, no networ coding is performed at the subgame perfect equilibrium between users m and N m + 1 because at least one of them is worse off when it participates in networ coding. In an extreme case, if 1 or holds for any m = 1,..., M given z m,n m+1 =, then regardless of the bargaining approach being used, at subgame perfect equilibrium we have: z =, y = y, 1. 4 That is, users simply play the Nash equilibrium of Game 1 at every stage of Game. Thus, in such special cases, efficiency at subgame perfect equilibrium of the repeated game is equal to efficiency at the Nash equilibrium of the static game. It is shown in [6, Theorem 11 that the worst-case efficiency of static Game 1 occurs under the following conditions: The utility functions of the users are linear. That is, U n x = γ n x, n N. 5 The cost parameters for side lins are negligible. That is, b m, m M. 6 The intuition behind 6 is clear: if the side lins have low cost, then performing inter-session networ coding can bring significant throughput gains to the users without significantly increasing the cost. Thus, not performing inter-session networ coding in this case will hurt the system performance the most. The above discussions imply that we expect to find tight upper bounds for PoA Game, Problem 1 if we can obtain the worst-case efficiency among all choices of system parameters which satisfy 5, 6, and either 1 or. In this regard, we study three cases separately as we see next.

7 1 One Networ Coding Pair N = and M = 1: Assume that the networ in Fig. 1 has only two networ coding users. Proposition 1: Given z 1,, if conditions 5 and 6 hold, then a at Nash equilibrium of the reduced Game 3: a If z 1, < γ γ 1 /a1 + β then [ y 1 z 1, y z 1, = [ γ1 γ a1+βz 1, γ γ 1 a1+βz 1,, 7 b If γ γ 1 /a1 + β z 1, < γ 1 /a1 + β then [ [ y 1 z 1, γ1 a1+βz 1, yz = a, 8 1, c If γ 1 /a1 + β z 1, then [ [ y 1 z 1, yz = 1,. 9 Without loss of generality, here we assumed that γ 1 γ. The proof of Proposition 1 is given in Appendix C. It helps us obtain closed-form expressions for P 1 y z 1,, z 1, and P y z 1,, z 1, for any z 1, and chec conditions 1 and to determine whether the networ coding users 1 and can reach a non-zero bargaining solution. Theorem 4: Assume that parameters N = and M = 1. a Among all choices of utility and cost parameters such that Condition 1: Both 5 and 6 hold, and Condition : Either 1 or holds for m = 1, the worst-case efficiency at the subgame perfect equilibrium of Game is 1 5 and occurs when a = 1 and γ = γ b For any bargaining scheme, we have PoA Game, Problem 1 1 = 48% The proof of Theorem 4 is given in Appendix D. From Condition, our focus is on scenarios where any bargaining scheme would lead to z 1, = and repeated Game is played just lie static Game 1. From Condition 1, we further focus on those scenarios where the static Game 1 has poor performance. Networ Coding and Routing Sessions N > and M = 1: Next, we consider the case where there is at least one routing user in the networ together with two networ coding users that form one networ coding pair. Proposition : Given z 1,N, if conditions 5 and 6 hold, then at Nash equilibrium of the reduced Game 3: a If z 1,N < γ γ 1 q z 1,N /a1 + β, then [ y 1 z 1,N y z 1,N = [ γ1 γ a1+βz 1,N aq z 1,N γ γ 1 a1+βz 1,N aq z 1,N, 3 where q z 1,N = N 1 r= y rz 1,N. b If γ γ 1 q z 1,N /a1 + β z 1,N < γ 1 q z 1,N /a1 + β, then [ y 1 z 1,N y z 1,N = [ γ1 a1+βz 1,N aq z 1,N a. 33 c If γ 1 q z 1,N a1 + β z 1,N, then [ [ y 1 z 1,N yz =. 34 1,N The proof of Proposition is similar to that of Proposition 1. We notice that if N = and q z =, then the expressions in Proposition reduce to the expressions in Proposition 1. Theorem 5: Assume that parameters N > and M = 1. a Among all choices of utility and cost parameters such that Condition 1: Both 5 and 6 hold, and Condition : Either 1 or holds for m = 1, the worst-case efficiency at the subgame perfect equilibrium of Game is 4 11 and occurs at N, a = 1, γ =... = γ N 1 = 3 4 γ 1, γ N = 3 8 γ 1. b For any bargaining scheme, we have PoA Game, Problem % The proof of Theorem 5 is given in Appendix E. We can see that the PoA upper bound significantly drops when the networ includes both networ coding and routing users. 3 Two Networ Coding Pairs N =4 and M =: Assume that the networ has two parallel networ coding sessions. For notational simplicity, we first define Φ 1,4 = a1 + 3βz 1,4 + a1 βz,3, Φ,3 = a1+3βz,3 +a1 βz 1,4, Ψ 1,4 = a1+βz 1,4 +a1 βz,3, Ψ,3 = a1+βz,3 +a1 βz 1,4, Λ 1,4 = a1+βz 1,4 +az,3, and Λ,3 = a1+βz,3 +az 1,4. Proposition 3: Given z 1,4, z,3, if 5 and 6 hold, then at Nash equilibrium of the reduced Game 3, we have a If 4γ 4 γ 1 γ γ 3 > Φ 1,4 and 4γ 3 γ 1 γ γ 4 > Φ,3, then y1z 1,4, z,3 = 4γ 1 γ γ 3 γ 4 Φ 1,4 /5a, yz 1,4, z,3 = 4γ γ 1 γ 3 γ 4 Φ 1,4 /5a, y3z 1,4, z,3 = 4γ 3 γ 1 γ γ 4 Φ,3 /5a, and y4z 1,4, z,3 = 4γ 4 γ 1 γ γ 3 Φ,3 /5a. b If 4γ 3 γ 1 γ γ 4 Φ,3 < 3γ γ 1 γ 4 and 3γ 4 γ 1 γ > Ψ 1,4, then y 1z 1,4, z,3 = 3γ 1 γ 4 γ Ψ 1,4 /4a, y z 1,4, z,3 = 3γ γ 1 γ 4 Φ,3 /4a, y 3z 1,4, z,3 =, and y 4z 1,4, z,3 = 3γ 4 γ 1 γ Ψ 1,4 /4a. c If 4γ 4 γ 1 γ γ 3 Φ 1,4 < 3γ 1 γ γ 3 and 3γ 3 γ 1 γ > Ψ,3, then y 1z 1,4, z,3 = 3γ 1 γ γ 3 Φ 1,4 /4a, y z 1,4, z,3 = 3γ γ 1 γ 3 Ψ,3 /4a, y 3z 1,4, z,3 = 3γ 3 γ 1 γ Ψ,3 /4a, and y 4z 1,4, z,3 =. d If 3γ γ 1 γ 3 Ψ 1,4 <γ 1 γ and 3γ 3 γ 1 γ Ψ,3 <γ γ 1, then y 1z 1,4, z,3 =γ 1 γ Ψ 1,4 /, y z 1,4, z,3 = γ γ 1 Ψ,3 /, y 3z 1,4, z,3 =y 4z 1,4, z,3 =. e If γ γ 1 Λ 1,4 and Φ,3 3γ γ 1 γ 4, then y 1z 1,4, z,3 = γ 1 γ 4 Λ 1,4 /, y z 1,4, z,3 =, y 3z 1,4, z,3 =, and y 4z 1,4, z,3 = γ 4 γ 1 Λ 1,4 /. f If γ 3 γ 1 Λ 1,4 < γ 1 and Φ,3 γ γ 1, then y 1z 1,4, z,3 = γ 1 Λ 1,4 /a, y z 1,4, z,3 =, y 3z 1,4, z,3 =, and y 4z 1,4, z,3 =. g If Φ 1,4 3γ 1 γ γ 3 and Λ,3 < γ 3 γ, then y 1z 1,4, z,3 =, y z 1,4, z,3 = γ γ 3 Λ,3 /, y 3z 1,4, z,3 = γ 3 γ Λ,3 /, and y 4z 1,4, z,3 =.

8 h If Ψ 1,4 γ 1 γ and Λ,3 γ, then y 1z 1,4, z,3 =, y z 1,4, z,3 =γ Λ,3 /a, y 3z 1,4, z,3 =y 4z 1,4, z,3 =. i If Λ 1,4 γ 1 and γ 3 γ Λ,3 <γ, then y 1z 1,4, z,3 =, y z 1,4, z,3 =, y 3z 1,4, z,3 =, and y 4z 1,4, z,3 =. The proof of Proposition 3 is similar to Proposition 1. Theorem 6: Assume that parameters N = 4 and M =. a Among all choices of utility and cost parameters such that Condition 1: Both 5 and 6 hold, and Condition : Either 1 or holds for m = 1 or, the worst-case efficiency at the subgame perfect equilibrium of repeated Game is 4 9 and it occurs at a = 1, γ 7 γ 1, γ 3 = 5 7 γ 1, γ 4 = 7 γ 1. b For any bargaining scheme, we have PoA Game, Problem % The proof of Theorem 6 is similar to that of Theorems 4 and 5. Theorems 4-6 suggest that adding more networ coding or routing users reduces the PoA upper bounds in Game. The PoA upper bounds for other scenarios e.g., with arbitrary number of networ coding and routing users can be analyzed in a similar way. However, as it is evident from Proposition 3, the computational complexity of the analysis grows drastically as we consider more networ coding pairs. V. MIN-MAX BARGAINING SOLUTION The idea in the min-max bargaining scheme is to let each networ coding user m and N m+1, for any m = 1,..., M, individually mae a choice for the coding rate z m,n m+1, and select the bargaining solution such that 14 holds and both users benefit from networ coding. Given z m,n m+1, consider the following set for networ coding user m: Z m = { z m,n m+1 ẑ m,n m+1 [, z m,n m+1, P m y z m,n m+1, z m,n m+1, z m,n m+1 P m y ẑ m,n m+1, z m,n m+1, ẑ m,n m+1 P m y, z m,n m+1, }. User m s payoff is monotonically increasing over set Z m. Any z m,n m+1 Z m satisfies 14 for user m and is acceptable for user m. Similarly, for user N m + 1, we define Z N m+1 = { z m,n m+1 ẑ m,n m+1 [, z m,n m+1, P N m+1 y z m,n m+1, z m,n m+1, z m,n m+1 P N m+1 y ẑ m,n m+1, z m,n m+1, ẑ m,n m+1 P N m+1 y, z m,n m+1, }. It is clear that the payoffs for both users m and N m + 1 are monotonically increasing over the intersection set Z m Z N m+1. Therefore, any choice of z m,n m+1 Z m Z N m+1 satisfies 14 for both users m and N m + 1 and is a potential bargaining solution. From Theorem 3, we can conclude that the grim-trigger strategy in Definition 8 is a subgame perfect equilibrium for Game if we choose any z m,n m+1 Z m Z N m+1 37 User Payoff / Networ Aggregate Surplus User Payoff / Networ Aggregate Surplus z * Common Coding Rate z 1, z * z * 1, = z* = a Networ Aggregate Surplus P 1 y * z 1, ;z 1, P y * z 1, ;z 1, z * 1, = z* 1 > z * P 1 y * ; P y * ; Common Coding Rate z 1, b Networ Aggregate Surplus P 1 y * z 1, ; z 1, P y * z 1, ; z 1, z * 1 P 1 y * ; P y * ; Fig.. Calculating the min-max bargaining solution z1, based on the model in 38, where z1 and z are as in 39 and 4, respectively. Dashed lines indicate the payoffs when no networ coding is performed. a An example with non-zero coding rate. b An example with zero coding rate. and a discount factor δ δ min for some δ min, 1. Formally, the min-max bargaining solution is calculated as where and z m,n m+1 = min{z m, z N m+1}, 38 z m = z N m+1 = max z m,n m+1, 39 z m,n m+1 Z m max z m,n m+1. 4 z m,n m+1 Z N m+1 Interestingly, zm and zn m+1 are the solutions of selfish problems 19 and as long as these problems are convex. Otherwise, zm and zn m+1 are simply the smallest local maximizers of problems 19 and, respectively. If zm < zn m+1, e.g., as in the example in Fig. a when N = and M = 1, then user N m + 1 prefers a lower coding rate than user m; however, due to Theorem, user N m+1 is worse off by selecting vn m+1 =z N m+1 > z 1 at any stage 1. A similar statement is true for user m. Thus, users m and N m+1 can agree on rate z m,n m+1 = zm,n m+1, after they individually announce z m and zn m+1,

9 1 1.9 z 1, = z 1, >.9 z 1,4 as a function of z,3 z,3 as a function of z 1, Efficiency z 1, Min max Bargaining Solution: z * 1,4 =.43, z*,3 = Subgame Perfect Equilibrium of Game Nash Equilibrium of Game Linear Utility Parameter γ z,3 Fig. 3. An example of the impact that changing the utility parameters has on the efficiency of networ coding games. Here, N = users form M = 1 networ coding pair. The linear utility parameters are γ = 1 and γ 1, 1. respectively. Given z m,n m+1 = zm,n m+1, for all m = 1,..., M, the users can then play the grim-trigger strategy. The min-max bargaining does not always lead to a nonzero coding rate. If the min-max bargaining solution is zero, then it is only because at least one networ coding user is better off if no networ coding is performed. This is shown in Fig. b. In this example, we have N =, M = 1, a = 1, b 1 =.1, b =., β =.5, and utility functions are linear with parameters γ 1 = 1 and γ =.. We can see that while z1 =.84 and user 1 is interested in performing networ coding with user, user has z = as it is better off not to participate in networ coding with user 1. Any other bargaining method similarly results in a zero coding rate. This reveals the main difference between a repeated inter-session networ coding and a typical repeated game e.g., a Prisoner s dilemma: here, the coding rate may still remain zero i.e., still no cooperation even if the game is played repeatedly. Fig. 3 illustrates the improvements in efficiency when the min-max bargaining solution is used for a networ with N = users that form M = 1 networ coding pair. Here, we assume that the utility functions are linear with γ = 1 and γ 1 varying from to 1. We can see that if < γ 1 1 4, i.e., the utility functions of user 1 and user are very different, then user 1 has no interest in participating in networ coding and the minmax bargaining solution is z1, =. The worst-case efficiency occurs when γ 1 = 1 4 as expected based on Theorem 4. As γ 1 increases, the two users have more motivation to agree on a non-zero common coding rate leading to a significant improvement in efficiency in the subgame perfect equilibrium of Game compared to the Nash equilibrium of Game 1. From the results in Fig. 3, for those cases where repeated game cannot do any better than static game, the efficiency of the static game is much better than its worst-case efficiency. Next, consider the case where N = 4 and M =. We are interested in calculating two min-max bargaining solutions z1,4 and z,3. Since all four users share the bottlenec lin, the choice of z1,4 to be decided by users 1 and 4 also depends on the choice of z,3 which is decided by users and 3. Fig. 4. An example for calculating the min-max bargaining solutions in the presence of two networ coding pairs, i.e., when N = 4 and M =. Similarly, the choice of z,3 to be decided by users and 3 also depends on the choice of z 1,4 which is decided by users 1 and 4. Therefore, min-max bargaining solutions at equilibrium are obtained at the cross section of z 1,4 plotted as a function of z,3 and z,3 plotted as a function of z 1,4. An example is shown in the Fig. 4. In this example, we have γ 1 = 1, γ =.8, γ 3 =.7, γ 4 =.75, a = 1, b 1 =.5, b =.4, b 3 =., b 4 =.7, and β =.5. In this case, we have z 1,4 =.43 and z,3 =.41. Efficiency at the sub-game perfect equilibrium of the repeated inter-session networ coding game is The efficiency at the Nash equilibrium of the static inter-session networ coding game in this scenario is.44. VI. NUMERICAL RESULTS In this section, we evaluate the min-max bargaining scheme for various choices of parameters N, M, a, b 1,..., b N, and U 1,..., U N. We assume that δ =.99. Numerical results for 1 random scenarios are shown in Fig. 5. For the results in Fig. 5a, we have N = and M = 1 and the networ includes only one networ coding pair. For the results in Fig. 5b, we have N = 4 and M = and the networ includes two parallel networ coding pairs. Finally, for the results in Fig. 5c, N is selected randomly between 5 and 5 and there are always some routing users in the networ in addition to M = 1 networ coding pair. In each scenario, the lin cost parameters a, 1, b 1, 5, and b, 5 are selected randomly. The utility functions are α-fair [35: U n x = κ n 1 α n 1 x 1 αn, n N, 41 where α n [, 1 and κ n, 1 are selected randomly. We can verify that the utility functions in 41 satisfy Assumption. They include the linear case in 5 when α n =. From Fig. 5a, the subgame perfect equilibrium of Game that is formed when users play the proposed grim-trigger strategy based on the proposed min-max bargaining solution has a higher efficiency than the Nash equilibrium of Game 1 in every scenario. Furthermore, the efficiency of Game is always greater than or equal to 48%, suggesting that PoA Game, Problem 1 48% for min-max bargaining

10 1.9 a 1 9 Repeated Game Static Game.8 8 Efficiency Subgame Perfect Equilibrium of Repeated Game Nash Equilibrium of Static Game.1 Efficiency Level 48% Theorem 4 Efficiency Level % Theorem Scenario Number Cumulative Probability % Efficiency 1 b Fig. 6. The cumulative probability of efficiency in static and repeated games as a mean to measure relative performance when N = and M = 1. Efficiency Efficiency Subgame Perfect Equilibrium of Repeated Game.1 Nash Equilibrium of Static Game Efficiency Level at 44% Theorem Scenario Number c. Subgame Perfect Equilibrium of Repeated Game Nash Equilibrium of Static Game.1 Efficiency Level at 36% Theorem 5 Efficiency Level at % Theorem Scenario Number Fig. 5. Efficiency at Nash equilibrium of static Game 1 and subgame perfect equilibrium of repeated Game for 1 random scenarios where the networ topology is as in Fig. 1 and the min-max bargaining solution is being used. The number of users N and the number of networ coding pairs M are chosen as follows: a N = and M = 1. b N = 4 and M =. c N > and M = 1, where we randomly choose N 5, 5. We can see that the PoA upper bounds in Theorems 4-6 are reached when the min-max bargaining solution is used. We also note that in addition to the improvement in the PoA, i.e., the worst-case efficiency, the average efficiency also increases by 64.% when the networ coding game is played repeatedly rather than only once. solution. From this, together with the upper bound in Theorem 4, we can conclude that: the worst-case efficiency of repeated Game when the proposed grim-trigger strategy and minmax bargaining solution are adopted occurs when even the bargaining process cannot help to encourage users to perform networ coding. When the number of networ coding pairs increases to M =, as in Fig. 5b, the results are similar. Again, efficiency improves when the game is played repeatedly and the users play the proposed grim-trigger strategy based on the proposed min-max bargaining solution. Furthermore, the efficiency of Game is always greater than or equal to 44%, suggesting that PoA Game, Problem 1 44%. From Fig. 5c, Game usually has a better efficiency than Game 1. We can see that the efficiency of Game is always greater than or equal to 36%. Thus, the PoA of Game achieves the upper bound in Theorem 5. In a few scenarios, e.g., the 1st and the 6th scenarios, the efficiency of static Game 1 is better than that of the repeated Game. In these scenarios, the bargaining between networ coding users leads to a coding rate higher than the optimal coding rate of Problem 1, which hurts the utilities of routing users. This interesting observation remains to be further investigated in the future. The results in Fig. 5 can be evaluated also in terms of average performance. When N = and M = 1, the average efficiency in a Nash equilibrium of static game is.45 while the average efficiency in a subgame perfect equilibrium of repeated game when users adopt min-max bargaining solution is.75. When N = 4 and M =, the average efficiency in a Nash equilibrium of static game is.34 and the average efficiency in a subgame perfect equilibrium of repeated game when users adopt min-max bargaining solution is.7. When N 5, 5 and M = 1, the average efficiency in a Nash equilibrium of static game is.48 and the average efficiency in a subgame perfect equilibrium of repeated game when users adopt min-max bargaining solution is.66. The relative performance of static and repeated networ coding games can illustrated also in a probabilistic fashion as shown in Fig. 6. Here, we assume that N = and M = 1. At each efficiency level Ω, the cumulative probability is calculated

11 Efficiency Subgame Perfect Equilibrium of Repeated Game Side Lin Cost Parameters b 1 = b Fig. 7. The impact of the side lin cost parameters on efficiency of the repeated networ coding game when N = and M = 1. Here, the values of the side lin cost parameters b 1 and b vary from to 1, while a = 1. as P {Efficiency < Ω}. For example, based on the red dashed vertical line, if the game is played statically, then in 9% of scenarios the efficiency at Nash equilibrium is less than.7. In contrast, if the game is played repeatedly based using our proposed grim-trigger strategy based on our proposed minmax bargaining solution, then in only 31% of scenarios the efficiency at the sub-game perfect equilibrium is less than.7. Similar comparisons can be made at any other efficiency level. Last but not least, we note that although utility parameters have a significant impact on the PoA, as suggested by Theorems 4-6 and Fig. 3, other parameters also have a major impact on efficiency. For example, Fig. 7 shows the efficiency when the values of the side lin cost parameters b 1 and b vary from to 1. Other parameters are fixed to a = 1, γ 1 = 1, γ =.5, and we have N = and M = 1. As b 1, and b, efficiency approaches the results in Theorem 4. Therefore, not only the utility parameters but also the cost parameters have an impact on efficiency in the repeated game. VII. CONCLUSIONS AND FUTURE WORK This wor represents a first step towards understanding noncooperative inter-session networ coding in a repeated game. Our focus was on a butterfly networ with one or two pairs of networ coding users and possibly several routing users. We showed that if the networ coding game is played repeatedly, then it is possible for networ coding users to achieve a mutually desirable positive coding rate via bargaining. This is in sharp contrast to static inter-session networ coding games, where no networ coding is performed at the Nash equilibrium. We investigated the price-of-anarchy PoA, i.e., the worst-case efficiency compared to an optimal and cooperative networ design. We showed that for all possible bargaining schemes, the PoA of the repeated networ coding game is upper-bounded by 36% with one networ coding pair and several routing users, 44% with two networ coding pairs, and 48% with one networ coding pair. These bounds can be reached by a simple min-max bargaining. This indicates a major improvement compared to the % and % PoA results 1 5 for static inter-session networ coding for the same settings. Numerical results showed that, in addition to the worst-case efficiency, the average efficiency also improves significantly when the networ coding game is played repeatedly and the users adopt our proposed min-max bargaining solution. The results in this paper can be extended in several directions. First, our analysis can be applied to more general networ topologies such as those which are superposition of several butterfly networs. Second, efficiency may be improved by using user-specific pricing functions. Third, while we only considered a simple coding approach such as XOR, more general coding schemes may lead to different cooperation behaviors. Finally, non-cooperative networ coding models may be studied as games with incomplete information. A. Proof of Theorem APPENDIX Let m = zm zn m+1 >. In this case, we have ȳ P m m, z m, v m, y m, z m, v m = U m y m + m+vn m+1 v m m pm v m m ym+ m+zm m 1 βzn m+1 N p n=1 y n+ m + M r=1,r m max{z r,z N r+1 }+zm m = U m y m + m+vn m+1 + m p m v m m v m p m v m m ym + zm 1 βzn m+1 N p n=1 y n + M r=1,r m max{z r, z N r+1 }+zm > U m y m +vn m+1 v m p m v N m+1 m ym + zm 1 βzn m+1 N p n=1 y n + M r=1,r m max{z r, z N r+1 } + zm = P m y, z, v, where the inequality is due to m p m v N m+1 m > and ym + m + vn m+1 > y m + vn m+1, and since U m is increasing. Moreover, we can also show that ȳ P N m+1 m, z m, v m, y m, z m, v m = U N m+1 y N m+1 + zn m+1 vn m+1p N m+1 vn m+1 yn m+1 + βzn m+1 p N n=1 y n + M r=1,r m max{z r, z N r+1 }+ m+zm m = P N m+1 y, z, v. Finally, it is easy to verify that for each routing user n = M + 1,..., N M, we have P n ȳ m, z m, v m, y m, z m, v m = Pn y, z, v. Therefore, user m is better off by switching to new rates ȳ m, v m, z m, without changing other users payoffs.

12 B. Proof of Theorem 3 We prove that grim-trigger is a subgame perfect equilibrium of Game for user m. Assume that all users cooperate and play Step 1 in Definition 8. In that case, at each stage 1 of Game, if user m follows the grim-trigger strategy and sets its rates according to Step 1, it expects a long-term payoff δ t P m y z, z = P my z, z. 4 1 δ t= Next, assume that user m can reach the best payoff Γ m P m y z, z at the current stage of Game if it deviates from Step 1. Then, user m expects long-term payoff P m y z, z + δ t P m y, t=+1 43 = Γ m + δ 1 δ P my,. Comparing 4 and 43, it is best for user m to cooperate if and only if there exist discount factors δ, 1 such that P m y z, z 1 δ Γ m + After reordering the terms, it is required that δ 1 δ P my,. 44 Γ m P m y z, z Γ m P m y δ 1. 45, Clearly, the inequality 45 holds for some δ, 1 if and only if 14 holds for m. A similar argument is true for any other networ coding user, including user N m + 1, for any m = 1,..., M. We also note that the proof for routing users n = M + 1,..., N M is evident as the routing users simply play Nash equilibrium for the coding rates given by networ coding users. In summary, for the grim-trigger strategy to form a subgame perfect equilibrium, it is required that δ min = max m M This concludes the proof. C. Proof of Proposition 1 Γ m P m y z, z Γ m P m y,. Case I If y 1z 1, > and y z 1, >, then γ 1 = a y 1z 1, +y z 1, +1+βz 1, + ay 1z 1,, 46 γ = a y 1z 1, +y z 1, +1+βz 1, + ay z 1,. 47 From 47, we have y1z 1, = γ ayz 1, βz 1,. 48 a Replacing 48 in 46, we have y1z 1, = γ 1 γ a1 + βz 1,, 49 yz 1, = γ γ 1 a1 + βz 1,. 5 From 49 and nowing that y1z 1, >, we have γ γ 1 a1 + βz 1, > z 1, < γ γ 1 a1 + β. 51 Similarly, from 5 and nowing that y z 1, >, we have γ 1 γ a1 + βz 1, > z 1, < γ 1 γ a1 + β. 5 Since γ 1 γ, inequalities 51 and 5 reduce to z 1, < γ γ 1 a1 + β. 53 Thus, the data rates in 49 and 5 hold only if 53 holds. Case II If y 1z 1, > and y z 1, =, then γ 1 = a y 1z 1, +1+βz 1, + ay 1z 1,, 54 γ a y 1z 1, +1+βz 1,. 55 From 54 and after reordering the terms, we have y1z 1, = γ 1 a1 + βz 1,. 56 a Replacing 56 in 55, we have γ γ 1 + a1 + βz 1, z 1, γ γ 1 a1 + β. 57 Moreover, from 56 and nowing that y1z 1, >, we have γ 1 γ 1 > a1 + βz 1, z 1, < a1 + β. 58 Case III If y 1z 1, = and y z 1, =, then γ 1 a1 + βz 1,, γ a1 + βz 1,. 59 Since γ 1 γ, the above leads to z 1, γ 1 /a1 + β. D. Proof of Theorem 4 Without loss of generality, we assume that γ 1 γ. Given z 1,, the data rates y1z 1, and yz 1, are obtained from Proposition 1. We consider three cases separately. Case I If γ γ 1 < γ and 5 and 6 hold, then P y, = γ γ γ 1 a γ γ 1 γ1 + γ. On the other hand, if z 1, < γ γ1 a1+β and β = 1, then P y z 1,, z 1, = γ γ γ 1 + z 1, From 6 and 61, we have a γ γ 1 6 γ1 + γ. 61 P y z 1,, z 1, P y, = γ z 1, >, for any z 1, such that z 1,, γ γ 1. 6 a1 + β Therefore, does not hold. A similar statement is true for 1. Thus, Condition 1 does not hold if γ γ 1 < γ. Case II If γ γ 1 4γ and 5 and 6 hold, then P y, =. 63

13 On the other hand, if z 1, < γ1 a1+β and β = 1, then γ1 a + z 1, 4 P y z 1,, z 1, = γ z 1, az 1, Furthermore, we have. 64 d P y z 1,, z 1, lim = lim z 1, d z γ γ 1 1, z 1, 4 az 1, 4 = γ γ 1 4 >. Therefore, does not hold. A similar statement is true for 1. Thus, Condition 1 does not hold if γ γ 1 < 4γ. Case III If 4γ γ 1 and 5 and 6 hold, then If z 1, < P y, =. 65 γ1 a1+β and β = 1, then 64 holds and we have P y z 1,, z 1, P y, = z 1, γ γ1 az 4 1, 4 <, z 1, γ, 1 a1+β, 66 where the inequality is due to 4γ γ 1. On the other hand, γ if 1 a1+β z 1, and β = 1, then Therefore, P y z 1,, z 1, = γ z 1, az 1,. 67 P y z 1,, z 1, P y, z = z 1, γ az1, <, 1, γ1 a1+β, 68 where the inequality is due to γ az 1, < γ a γ1 a1 + 1 = γ γ 1 < From 66 and 69, inequality holds if and only if < γ γ In that case, z 1, =, y1 = γ1 a, and y =. Thus, S y γ1 z 1,, z 1, = γ 1 a γ1 3γ 1 = a a 8a. 71 On the other hand, we can show that Sy S, z S, v S = γ 1 + γ. 7 a Therefore, the worst-case efficiency of Game is obtained by solving the following optimization problem minimize γ 1,γ,a 3γ 1 8a γ 1+γ a 73 subject to < γ γ 1 4. We can see that the worst-case efficiency does not depend on the value of shared-lin cost parameter a. It only depends on the relative value of utility parameters γ 1 and γ, c.f. 73. The objective function in 73 is decreasing in γ. Thus, the minimum occurs when γ = γ1 4. Thus, the efficiency becomes 3γ 1 8a γ 1+ γ 1 4 a This concludes the proof. = = E. Proof of Theorem 5 Let γ max = max n N γ n and assume that γ 1 γ N. From [6, Theorem 1, no networ coding is desired at optimal resource allocation if γ 1 + γ N < γ max. Therefore, we focus only on the case when γ 1 + γ N γ max. We have Sy S, z S, v S = γ 1 + γ N. 75 a Moreover, we can verify that P N y z 1,N, z 1,N < P N y,, z 1,N >, 76 if and only if < γ N γ aq We notice that if q = then condition 77 reduces to 7 and the results will be as in Theorem 4. Therefore, we only focus on the case when q z 1,N >. Next, we can verify that the worst-case efficiency occurs when N, γ =... = γ N The proof is similar to that of [6, Theorem 11b and [6, Theorem 3. On the other hand, we can show that for each routing user n =,..., N 1, we have γ n = aq z 1,N + y 1z 1,N + y Nz 1,N + z 1,N + ay nz 1,N. Replacing 78 in 79, we have 79 γ = aq z 1,N + y 1z 1,N + y Nz 1,N + z 1,N. 8 We notice that if 77 holds, then z 1,N = and 8 becomes γ = aq + y 1 + y N. 81 Next, we consider three cases separately: and Case I If y1 > and yn >, then γ 1 = a q + y 1 + y N + ay 1, 8 γ N = a q + y 1 + y N + ay N. 83 From Proposition, we have yn = γ γ 1 aq. 84 However, replacing 77 in 84, we have yn. This contradicts the initial assumption that yn >. Thus, 76 does not occur. Thus, Condition does not hold in this case. Case II If y1 > and yn =, then γ 1 = a q + y 1 + ay 1, γ N a q + y 1. From Proposition, we have 85 y1 = γ 1 aq. 86 a On the other hand, from 81 and the fact that yn =, aq = γ ay 1. 87

14 By replacing 87 in 86 and after reordering the terms, y1 = γ 1 γ. 88 a From 87 and 88, we can further show that aq = γ γ 1. Replacing this in 77, inequality in 76 holds if and only if < γ N γ γ γ 1 = γ Therefore, in this case, we have Sy γ1 γ z 1,N =, z 1,N = = γ 1 + γ N a γ γ 1 + γ a γ1 γ + γ γ 1 9 a a a = γ 1 γ 1 γ + 3 γ. a From 75 and 9, the worst-case efficiency of Game is obtained by solving the following optimization problem minimize γ 1,γ,γ N γ 1 γ 1 γ + 3 γ.5 γ 1 + γ N subject to γ N γ, γ 1 + γ N γ max, γ N γ 1, < γ 1, γ, γ N γ max. From the 1st, 3rd, and 4th constraints in 91, we have 91 γ γ max γ 1 + γ N γ 1 γ 1 γ. 9 Thus, we can remove constraints γ N γ 1 and γ N γ max. The optimization problem 91 reduces to minimize γ 1,γ,γ N γ 1 γ 1 γ + 3 γ.5 γ 1 + γ N subject to γ N γ, γ 1 + γ N γ max, < γ 1, γ γ max. 93 The objective function in 93 is decreasing in γ N. Thus, the worst-case efficiency occurs at upper bound of γ N, i.e., when we have γ N = γ. Therefore, problem 93 further reduces to γ1 γ 1 γ + 3 minimize γ γ 1,γ.5 γ 1 + γ subject to γ 1 + γ γ max, < γ 1, γ γ max. 94 Problem 94 is not a convex minimization problem with respect to variables γ 1 and γ. However, we can still solve problem 94 as follows. We first assume that γ is fixed. By solving the Karush-Kuhn-Tucer KKT conditions of problem 94 with respect to γ 1, we can identify three KKT points: γ 1 = γ max, 95 γ 1 = γ max γ, 96 γ 1 = 4 3 γ. 97 The global minimizer choice of γ 1 is among the above three KKT conditions. We start by replacing 95 in problem 94. After reordering the terms, problem 94 becomes γmax γ max γ + 3 minimize γ γ.5 γ max + γ subject to < γ γ max. 98 Problem 98 is convex with respect to γ. By taing derivatives, we can show that the minimum occurs when γ = 3 4 γ max. Replacing this in the objective function in 98, the worst-case efficiency in this case becomes γmax γ max = Next, we replace 96 in problem 94. The minimum occurs when γ = 6 11 γ max and the worst-case efficiency is Finally, we replace 97 in 94. The worst-case efficiency is obtained as in 99 which occurs when γ = 3 4 γ max. Case III If y1 = and yn =, then γ N γ 1 aq. 1 Furthermore, from 81, we have γ = aq. Thus, in this case, γ N γ 1 γ. By following similar steps as in Case II, we can verify that the worst-case efficiency in this case is 4 9. Combining the results in Cases I, II, and III, the worst-case efficiency when Condition 1 and Condition hold becomes min { 4 11, 4 9 } = 4 11, 11 which occurs when 36 holds. Before concluding this proof, We note that although the above 4 11 bound also appears in [6, Theorem 8, Theorem 5 in this paper is very different from [6, Theorem 8. Here, the ey concern is to find scenarios where no non-zero networ coding rate and thus no solution of any bargaining approach can benefit one of the two users involved in networ coding. However, this concept was not studied in [6, Theorem 8 or any other part of Section IV in [6. REFERENCES [1 R. Ahlswede, N. Cai, S. Li, and R. Yeung, Networ information flow, IEEE Trans. on Info. Theory, vol. 46, no. 4, pp , Apr.. [ L. Chen, T. Ho, S. Low, M. Chiang, and J. Doyle, Optimization based rate control for multicast with networ coding, in Proc. of IEEE INFOCOM, Anchorage, AK, May 7. [3 D. Trasov, N. Ratnaar, D. S. Lun, R. Koetter, and M. Medard, Networ coding for multiple unicasts: An approach based on linear optimization, in Proc. of IEEE ISIT, Seattle, WA, July 6. [4 C. C. Wang and N. B. Shroff, Beyond the butterfly: Graph-theoretic characterization of the feasibility of networ coding with two simple unicast sessions, in Proc. of IEEE ISIT, Nice, France, June 7. [5 A. Khreishah, C. C. Wang, and N. B. Shroff, Optimization based rate control for communication networs with inter-session networ coding, in Proc. of IEEE INFOCOM, Phoenix, AZ, Apr. 8. [6 H. Mohsenian-Rad, J. Huang, V. W. S. Wong, S. Jaggi, and R. Schober, Inter-session networ coding with strategic users: A game-theoretic analysis of networ coding, IEEE Trans. on Communications, vol. 61, no. 4, pp , Apr. 13. [7 J. Price and T. Javidi, Networ coding games with unicast flows, IEEE JSAC, vol. 6, no. 9, pp , Sept. 8. [8 D. Fudenberg and J. Tirole, Game Theory. MIT Press, 1991.

15 [9 C. Long, Q. Zhang, B. Li, H. Yang, and X. Guan, Non-cooperative power control for wireless ad hoc networs with repeated games, IEEE J. on Sel. Areas in Commun., vol. 5, no. 6, pp , Aug. 7. [1 Z. Kong and Y. K. Kwo, Game theoretic pacet scheduling in a noncooperative wireless environment, Communications in Computer and Information Science, vol. 56, no. 1, pp. 9 16, Aug. 9. [11 M. L. Treust and S. Lasaulce, A repeated game formulation of energyefficient decentralized power control, IEEE Trans. on Wireless Communications, vol. 9, no. 9, pp , Sept. 1. [1 Z. Li, Cross-monotonic multicast, in Proc. of IEEE INFOCOM, Phoenix, AZ, Apr. 8. [13 X. Zhang and B. Li, Dice: a game theoretic framewor for wireless multipath networ coding, in Proc. of ACM MobiHoc, Hong Kong, China, May 8. [14 S. Bhadra, S. Shaottai, and P. Gupta, Min-cost selfish multicast with networ coding, IEEE Trans. on Information Theory, vol. 5, no. 11, pp , Nov. 6. [15 C. Cheuri, J. Chuzhoy, L. Lewin-Eytan, J. Naor, and A. Orda, Noncooperative multicast and facility location games, IEEE JSAC, vol. 5, no. 6, pp , Aug. 7. [16 X. B. Liang, Matrix games in the multicast networs: Maximum information flows with networ switching, IEEE Trans. on Information Theory, vol. 5, no. 6, pp , June 6. [17 J. Price and T. Javidi, A game-theoretic approach to coding for information networs, in Proc. of Allerton Conference on Communication, Control, and Computing, Urbana Champaign, IL, 8. [18 J. R. Marden and M. Effros, A game theoretic approach to networ coding, in Proc. of IEEE Information Theory Worshop on Networing and Information Theory, Volos, Greece, June 9. [19 V. Reddy, S. Shaottai, A. Sprintson, and N. Gautam, Multipath wireless networ coding: A population game perspective, in Proc. of IEEE INFOCOM 1, San Diego, CA, Mar. 1. [ E. Ahmed, A. Eryilmaz, A. Ozdaglar, and M. Medard, Economic aspects of networ coding, in Proc. of Allerton Conference on Communication, Control, and Computing, Urbana Champaign, IL, 6. [1 T. Chen and S. Zhong, Inpac: An enforceable incentive scheme for wireless networs using networ coding, in Proc. of IEEE INFOCOM 1, San Diego, CA, Mar. 1. [ H. Mohsenian-Rad, J. Huang, V. W. S. Wong, and R. Schober, Bargaining and price-of-anarchy in repeated inter-session networ coding games, in Proc. of IEEE INFOCOM 1, San Diego, CA, Mar. 1. [3 T. Ho, Y. Chang, and K. Han, On constructive networ coding for multiple unicasts, in Proc. of Annual Allerton Conference on Communications, Control, and Computing, Urbana, IL, Sept. 6. [4 F. P. Kelly, A. Maulloo, and D. Tan, Rate control for communication networs: Shadow prices, proportional fairness and stability, Journal of Operations Research Society, vol. 49, pp. 37 5, Mar [5 D. Samet and Y. Tauman, The determination of marginal cost prices under a set of axioms, Econometrica, vol. 59, pp , 198. [6 R. Johari and J. N. Tsitsilis, A scalable networ resource allocation mechanism with bounded efficiency loss, IEEE J. on Selected Areas in Commun., vol. 4, no. 5, pp , May 6. [7 T. Hars and K. Miller, Efficiency and stability of Nash equilibria in resource allocation games, Technical Report, Department of Mathematics, Technische Universitt Berlin, 8. [8 R. Johari, S. Mannor, and J. N. Tsitsilis, Efficiency loss in a networ resource allocation game: The case of elastic supply, IEEE Trans. on Automatic Control, vol. 5, no. 11, pp , Nov. 5. [9 S. Yang and B. Haje, VCG-Kelly mechanisms for allocation of divisible goods: Adapting VCG mechanisms to one-dimensional signals, in Proc. of CISS, Princeton, NJ, Mar. 6. [3 Y. Chen and J. Zhang, Design of price mechanisms for networ resource allocation via price of anarchy, Technical Report, IEOR Department, University of California at Bereley, 8. [31 N. Shetty, G. Schwartz, and J. Walrand, Networ neutrality: Avoiding the extremes, in Proc. of Allerton Conference, Urbana, IL, Sept. 8. [3 A. Mas-Colell, M. D. Whinston, and J. R. Green, Microeconomic Theory. Oxford University Press, [33 T. Driessen, Cooperative Games, Solutions and Applications. Kluwer Academic Publishers, [34 A. E. Roth, Axiomatic Models of Bargaining: Lecture Notes in Economics and Mathematical Systems. Springer-Verlag, 199. [35 J. Mo and J. Walrand, Fair end-to-end window-based congestion control, IEEE/ACM Trans. on Net., vol. 8, pp , Oct.. Hamed Mohsenian-Rad M 9 received a masters degree in Elec- trical Engineering from Sharif University of Technology in 4 and a Ph.D. degree in ECE from The University of British Columbia in 8. Currently, he is an Assistant Professor at the Department of EE at the University of California at Riverside. Dr. Mohsenian-Rad is the recipient of the CAREER Award from the National Science Foundation NSF in 1 as well as the Best Paper Award in the IEEE International Conference on Smart Grid Communications 1. Jianwei Huang SM 11 is an Associate Professor in the Department of Information Engineering at the Chinese University of Hong Kong. He received Ph.D. from Northwestern University in 5. Dr. Huang currently leads the Networ Communications and Economics Lab, with the main research focus on nonlinear optimization and game theoretical analysis of networs, especially on networ economics, cognitive radio networs, and smart grid. He is the co-recipient of IEEE Marconi Prize Paper Award in Wireless Communications 11, and Best Paper Awards from IEEE WiOPT 13, IEEE SmartGridComm 1, WiCON 11, IEEE GLOBECOM 1, and APCC 9. He received the IEEE ComSoc Asia-Pacific Outstanding Young Researcher Award in 9. He has been an editor of IEEE JSAC and IEEE Trans. on Wireless Communications. He is the Chair of IEEE Multimedia Communications Technical Committee. Vincent W.S. Wong SM 7 received the B.Sc. degree from the University of Manitoba, Winnipeg, MB, Canada, in 1994, the M.A.Sc. degree from the University of Waterloo, Waterloo, ON, Canada, in 1996, and the Ph.D. degree from the University of British Columbia UBC, Vancouver, BC, Canada, in. He joined the ECE Department at UBC in and is currently a Professor. His research areas include protocol design, optimization, and resource management of communication networs, with applications to the Internet, wireless networs, smart grid, RFID systems, and intelligent transportation systems. Dr. Wong is an Associate Editor of the IEEE Trans. on Communications. He has served on the editorial boards of IEEE Trans. on Vehicular Technology and Journal of Communications & Networs. Dr. Wong is the Symposium Co-chair of IEEE SmartGridComm 13 Communications Networs for Smart Grid and Smart Metering Symposium, and IEEE Globecom 13 Communication Software, Services, and Multimedia Application Symposium. Robert Schober F 1 received the Diplom Univ. and the Ph.D. degrees in electrical engineering from the University of Erlangen-Nuermberg in 1997 and, respectively. Since May he has been with the University of British Columbia UBC, Vancouver, Canada, where he is now a Full Professor. Since January 1 he is an Alexander von Humboldt Professor and the Chair for Digital Communication at the Friedrich Alexander University FAU, Erlangen, Germany. His research interests include Communication Theory, Wireless Communications, and Statistical Signal Processing. Dr. Schober s awards include the Heinz MaierLeibnitz Award of the German Science Foundation DFG, the 4 Innovations Award of the Vodafone Foundation for Research in Mobile Communications, the 6 UBC Killam Research Prize, the 7 Wilhelm Friedrich Bessel Research Award of the Alexander von Humboldt Foundation, the 8 Charles McDowell Award for Excellence in Research from UBC, and a 1 NSERC E.W.R. Steacie Fellowship. In addition, he received best paper awards from the German Information Technology Society ITG, the EURASIP, IEEE WCNC 1, IEEE Globecom 11, IEEE ICUWB 6, the International Zurich Seminar on Broadband Communications, and European Wireless. Dr. Schober is a Fellow of the Canadian Academy of Engineering and a Fellow of the Engineering Institute of Canada. He is currently the Editor-in-Chief of the IEEE Trans. on Communications.