Minimizing Price of Anarchy in Resource Allocation Games

Transcription

1 University of Colorado, Boulder CU Scholar Electrical, Computer & Energy Engineering Graduate Theses & Dissertations Electrical, Computer & Energy Engineering Spring Minimizing Price of Anarchy in Resource Allocation Games Yassmin Shalaby University of Colorado Boulder, Follow this and additional works at: Part of the Electrical and Computer Engineering Commons, and the Operational Research Commons Recommended Citation Shalaby, Yassmin, "Minimizing Price of Anarchy in Resource Allocation Games" (2014). Electrical, Computer & Energy Engineering Graduate Theses & Dissertations This Thesis is brought to you for free and open access by Electrical, Computer & Energy Engineering at CU Scholar. It has been accepted for inclusion in Electrical, Computer & Energy Engineering Graduate Theses & Dissertations by an authorized administrator of CU Scholar. For more information, please contact

2 Minimizing Price of Anarchy in Resource Allocation Games by Yassmin Shalaby B.S., Cairo University, 2006 M.S., University of Toronto, 2010 A thesis submitted to the Faculty of the Graduate School of the University of Colorado in partial fulfillment of the requirements for the degree of Master of Science Department of Electrical, Computer, and Energy Engineering 2014

3 This thesis entitled: Minimizing Price of Anarchy in Resource Allocation Games written by Yassmin Shalaby has been approved for the Department of Electrical, Computer, and Energy Engineering Jason Marden Eric Frew Date The final copy of this thesis has been examined by the signatories, and we find that both the content and the form meet acceptable presentation standards of scholarly work in the above mentioned discipline.

4 iii Shalaby, Yassmin (M.S., Electrical, Computer, and Energy Engineering) Minimizing Price of Anarchy in Resource Allocation Games Thesis directed by Prof. Jason Marden Resource allocation refers to problems where there is a set of resources to be allocated efficiently among a group of agents. The distributed nature of resource allocation motivates modeling it as a distributed control problem. One of the strong modeling frameworks for distributed control problems is the game theoretic framework. Game theory provides mathematical models that aid in studying the aggregate behavior of a group of decision makers. The main challenge in modeling a distributed optimization problem as a game is the design of agents utility functions. A utility function is designed as a distribution rule of some welfare; and the goal is to distribute the welfare in a way that incentivizes players to land in a good equilibrium point. The ratio between the performance of the worst possible equilibrium point and the optimal outcome of a game is called the price of anarchy. A distribution rule that distributes the welfare exactly is called budget-balanced, and one that distributes the welfare with excess is said to satisfy a relaxed budget-balance condition. On the other hand, if it causes a deficit we say that it violates the budget-balance condition. In this thesis, we study the design of utility functions in resource allocation games that minimize the price of anarchy. We compare two families of utility functions that guarantee equilibrium existence, namely the Shapley value and the marginal contribution. The Shapley value is a budgetbalanced distribution rule, while the marginal contribution satisfies the relaxed budget-balance condition given that the welfare being distributed is submodular. We derive price of anarchy bounds for the marginal contribution utility in resource allocation games and compare them to those for the Shapley value, derived in the literature. We also perform a small-scale study for a wider range of utility functions. Lastly, we examine the connection between the price of anarchy and the satisfiability of the budget-balance conditions of the utility designs. We show that violating the budget-balance condition worsens the price of anarchy.

5 Dedication To my beloved husband, Zyad.

6 v Acknowledgements I would like to express deep gratitude towards my advisor, Jason Marden, for the guidance and support he offered me throughout our research journey. I would like to greatly thank Raga Gopalakrishnan for his insights on this research; and my thesis committee for reviewing my work. I was blessed to have wonderful lab-mates, Holly, Yilan, Philip, Matt and many others who have joined our lab occasionally. I am thankful for the fruitful discussions we have had over research and the enjoyable lunch and coffee breaks we have shared together. Warm thanks to my friends Aya, Samah, Heba, Hoda, Paria, Nancy, Khawla, Rehab, Khadija, Kelly, Marie, Neveen, Reham and many others who have been by my side in the ups and downs. I greatly acknowledge Sara Fall, at the graduate writing support center, for helping me in editing my thesis. I would also like to acknowledge all the employees at the University of Colorado at Boulder, from graduate student advisors, to chefs and staff at the C4C cafeteria and Buff-bus and night-ride drivers, for making my life as a graduate student much easier. Special thanks to Adam, my counselors Dorothy and Glenda and my thesis support group; Sara, Richard, and Charlie. Words will never be enough to praise my family and friends back home who, though geographically far away, have always been very close to me, supporting and encouraging me. My parents have always been my backbone and have given me the chance to fulfill my dreams and excel in every way in life. My sister, brother and my in-laws have always been there for me. I passionately thank my husband, Zyad, for all the sacrifices he did for me and for the time he spent babysitting our son so that I can get work done. I am so grateful to have you in my life. Thanks to my 3-years-old son for bringing much joy into my life and inspiring me in so many ways.

7 All my success is due to Allah (God); in Him I trust and to Him I turn. vi

8 Contents Chapter 1 Introduction Overview Motivation Contributions Organization of Thesis Background Normal-Form Games Equilibrium Types of Equilibria Existence of Equilibrium Performance Metrics Welfare Distribution Rules Equal Share Marginal Contribution Shapley Value The weighted Shapley Value Summary Distribution Rules Performance Metrics

9 viii 3 Related work Valid Utility Systems Smooth Games Potential Games Necessity Efficiency of Shapley Value in Resource Allocation Games Summary Marginal Contribution Utility Function in Resource Allocation Games Game Model Price of Anarchy Bound Symmetric Action Sets Asymmetric Action Sets Comparative Analysis Illustrative Examples Analytical Proof Worst Case Format Definitions Shapley value worst case format Marginal Contribution worst case format Weighted Shapley value Summary Small-Scale Study on Utility functions that minimize the Price of Anarchy for Resource Allocation Games player Resource Allocation Games Game Definition Game Analysis Price of Anarchy Analysis for 2-player games

10 ix Price of Stability Analysis for 2-player games player Resource Allocation Games Game Analysis Price of Anarchy Analysis for 3-player games Numerical Example Resource Allocation Games with a finite number of players Summary Effect of Violating the Budget-Balance Condition on the Robust Price of Anarchy Reverse Carpooling Game Game Notation Cost Distribution Rules Discussion Non-Budget-Balanced Smooth Games Smoothness of Reverse Carpooling Games Summary Conclusions and Future Recommendations Contributions Related Research Questions Bibliography 147 Appendix A Efficiency bounds 150 B Proof for some lemmas 152

11 C All possible 3-player Resource Allocation Games 155 x

12 Tables Table possible cases for Optimal (OPT) and Nash Equilibrium (NE) allocations in G 2. The numbers in the table correspond to the naming convention of the allocations given by Figure 5.3c Reduction to 7 cases. The numbers in the table correspond to the naming convention of the allocations given by Figure 5.3c Reduction to 5 cases. The numbers in the table correspond to the naming convention of the allocations given by Figure 5.3c Final Reduction to 4 cases. The numbers in the table correspond to the naming convention of the allocations given by Figure 5.3c Summary of 4 main cases for 2-player games Summary of the results of Figures 5.8 to Comparison between the different Cost Sharing Rules for Reverse Carpooling Games The Price of Anarchy associated with the reverse carpooling game with f z distribution rule for different values of z

13 Figures Figure 2.1 Prisoner s Dilemma Game Traffic Intersection Game A Joint Probability Distribution for the Traffic Intersection Game Different sets of Equilibria Matching Pennies Stag Hunt The figure shows the payoff matrix of a potential game and the associated potential values. A unilateral deviation of a player causes a change in the utility equal to the change in the potential value Stag Hunt: Total Welfare Prisoner s Dilemma Game The price of anarchy of the submodular functions W r (x) = x d in the symmetric setting for the Shapley value and the marginal contribution utility design

14 xiii 4.2 The price of anarchy of the submodular functions W r (x) = x d in the asymmetric setting for the Shapley value and the marginal contribution utility design. We prove, in Section 4.3.2, that the marginal contribution bound is greater than or equal the Shapley value bounds derived by [20] for any submodular function; which is shown in the figure for the submodular function x d, for all values of 0 d 1. Also, as we see in the figure, the marginal contribution price of anarchy of Example 3 lies below the marginal contribution bound, but is higher than the Shapley value bound for all values of d A player is denoted by o in the optimal allocation and by x in the Nash equilibrium allocation The price of anarchy of the ɛ-family of submodular functions in the asymmetric setting for the Shapley value and the marginal contribution utility design. We prove, in Section 4.3.2, that the marginal contribution bound is greater than or equal the Shapley value bounds derived by [20] for any submodular function; which is shown in the figure for the piecewise linear family with slope ɛ, for all values of 0 ɛ 1. Also, as we see in the figure, the marginal contribution price of anarchy of Example 4 lies below the marginal contribution bound, but is higher than the Shapley value bound for all values of ɛ A subset of the halfplanes that define γ(w) for x 0.5 example with 2 players Game G is grouped into several games each of which is in the class G c Game Ḡ is grouped into several games each of which is in the class G s Action Set of Players: A 1 = {R 1, R 2 } and A 2 = {R 2, R 3 } The figure shows the 3 resources and the 2 players in each case. A player in optimal allocation is denoted by o and in the NE by x Price of Anarchy as U(2) changes

15 xiv 5.5 Comparing the Shapley value utility function to the utility that minimizes the price of anarchy for 2-player games Price of Anarchy and Price of Stability as U(2) changes Comparing P oa A, P oa B and P oa C for 3 players. The figure shows that P oa A dominates the others for U(3) > 1, P oa B is dominant for d 1 U(3) 1 and P oa C is dominant for d 2 U(3) d Price of Anarchy as U(3) changes where d 1 = 0.4 and d 2 = 0.2. Minimum PoA determined by P oa F, somewhere between the marginal contribution value and the Shapley value of 3 players; d 2 = U MC (3) U(3) U SV (3) = 1+d 1+d 2 3, more specifically at the value of the utility where P oa C intersects P oa D Price of Anarchy as U(3) changes where d 1 = 0.7 and d 2 = 0.1. Minimum PoA determined by the maximum of P oa F and P oa C P oa D Price of Anarchy as U(3) changes where d 1 = 0.9, d 2 = 0.5. PoA minimized at P oa C = P oa D, somewhere between the marginal contribution value and the Shapley value of 3 players; d 2 = U MC (3) U(3) U SV (3) = 1+d 1+d 2 3, more specifically at the value of the utility where P oa C intersects P oa D Price of Anarchy as U(3) changes where d 1 = 0.9, d 2 = 0.8. Minimum PoA determined by P oa F, somewhere between the marginal contribution value and the Shapley value of 3 players; d 2 = U MC (3) U(3) U SV (3) = 1+d 1+d 2 3, more specifically at the value of the utility where P oa C intersects P oa D D plot showing The Price of Anarchy as (U(2), U(3)) change where d 1 = 0.7, d 2 = 0.1. The figure shows that the minimum price of anarchy at U(2) = U MC (2) = d 1 is determined by P oa F and the minimum price of anarchy at U(2) = U SV (2) = 1+d 1 2 is determined by the intersection of P oa D and P oa C. It also shows that the minimum price of anarchy as a function of U(2) and U(3) occurs somewhere in the middle of the graph and it is not corresponding to U(2) = u

16 6.1 Reverse Carpooling Price of Anarchy increases as the cost distribution violates the budget-balance condition141 xv C.1 3 players on 1 resource in the Nash equilibrium C.2 2 players on 1 resource in the Nash equilibrium C.3 3 players on 3 different resources in the Nash equilibrium C.4 3-player Games that exploit worst case price of anarchy

17 Chapter 1 Introduction 1.1 Overview Resource allocation refers to a set of problems where there are multiple resources to be allocated efficiently between a group of agents. Many real life optimization problems can be classified as resource allocation problems. Examples include, but are not limited to, channel allocation in wireless communications, transportation network planning, routing in sensor networks, power management in distributed systems, robot coordination and wind farm control [4, 11, 12, 16, 21, 34]. All these applications share a general theme: a group of agents, a set of resources and a global objective function to be optimized. The agents could either be people or programmable entities. The set of resources, on the other hand, could range from physical elements of a transportation network, like roads, streets, and highways, to non-tangible elements of a computer network like bandwidth in wireless media. The objective is application dependent; for example, in a transportation network planning problem, one would have a transportation system with a finite number of resources to be shared between people going from home to work everyday and the objective could be to minimize the total time traveled or to minimize average congestion on the roads. In a computer network, with wireless media to be shared between different entities, the objective could be to maximize the throughput or minimize the delay. Another example of a resource allocation problem is wind farm control, with the wind turbines being the agents and the wind being the resource to be shared between the turbines. The objective in this setting would be to maximize the overall energy produced from the farm. In summary, all resource allocation problems have an objective function to

18 2 be optimized; maximized or minimized depending on the domain. Both centralized and distributed schemes have been utilized in the literature to solve resource allocation problems [4, 5, 11, 39]. Centralized control, as the name suggests, requires a central controller that solves the optimization problem and manages the resource allocation between the group of agents. Some optimization problems possess desirable properties, such as convexity, and thus could be solved optimally using a central controller. However, in many cases finding an optimal solution is not possible; even for convex problems centralized control could become infeasible as the number of agents/resources gets large. In centralized control, communication is crucial between the controller and the agents which creates a lot of overhead on the system. Also, the reliability of the system is fully dependent on that of the central controller; in other words, if the controller fails, the whole system fails. On the other hand, distributed control is not dependent on a single controller, but rather each agent is self controlled. On the negative side, in a distributed control approach, the problem would be divided into several smaller parts, the solution of which can be suboptimal, rather than optimal, to the original problem. However, on the positive side, the system is more reliable and sustainable as minimal or no communication is required between agents and the failure of one agent does not necessarily mean the failure of the entire system. In summary, due to the distributed nature of the resource allocation problem, distributed control is more appealing than centralized control for solving large-scale instances of the problem. Among the various modeling frameworks for distributed control, game theoretic framework is a popular one. Recently, several studies have modeled the resource allocation problem as a game and used game theoretic tools to solve the underlying optimization problem [1, 12, 18, 21, 24, 26, 39, 42]. Game theory is widely utilized in economics, political science, biology, computer science and many other disciplines [6, 14, 29]. The theory incorporates a group of mathematical tools that facilitate the study of the behavior of rational decision makers. The decision makers are called the players in the game and each one of them has an internal preference, a set of actions to choose from, and a possible outcome for each action. The internal preference of players could either be inherited or designed depending on the domain. The outcome of the game is dependent on the collective

19 3 behavior of the players. Therefore, the expected outcome for the player is not only dependent on its own action, but rather on everyone s action in the game. As a simple example, in a card game, the selection of which card to play would include a prediction of what other players would play in response. Although game theory originated in the social sciences where players are people involved in a decision process, it found its way to computer science where players are programmable entities. A game designer in the social sciences, also known as a mechanism designer, is responsible for designing the rules of the game, whereas in the computer science field, a game designer is mainly responsible for designing the preferences of the players which are in this case programmable agents [25]. We focus on modelling an engineering optimization problem as a game, and thus the player is a programmable entity and the preference of the player is a design parameter. To clarify, we list the following examples: Transportation network planning problem Players: Source-Destination pairs in a transportation network Resource: Transportation network Action Sets: Different routes in the network between the source and destination Objective Function: Minimize the total delay or congestion Preference Design: A tolling scheme that directs the choice of routes by players Wind farm optimization Players: Wind Turbines Resource: Wind Action Sets: Varying speed and pitch of the blades Objective Function: Maximize the overall energy produced Preference Design: A utility function for each individual turbine such as:

20 4 Maximize power generated by the turbine, or Maximize power generated by a turbine and its neighboring turbines 1 Vehicle target assignment problem Players: Vehicles Resources: Targets with different values Action Sets: A subset of the targets Objective Function: Eliminate all targets Preference Design: A utility function for each vehicle that assigns a payoff according to the target being eliminated In this thesis, we study a class of resource allocation games and analyze the behavior of these games. In a resource allocation game, the agents are the different players in the game. The set of resources would be available for players to choose from according to their action sets. The global objective would be to maximize welfare or minimize cost depending on the domain. The global objective function would be given to the designer of the game, along with the number of players and the available resources for each player. The game designer is tasked with assigning for each player a utility function that determines the player s preferences among different action sets; setting this local utility function equal to the global objective function does not necessarily lead to optimal behavior. The utility function is usually designed in the form of a distribution rule of the total welfare. A distribution rule that distributes the welfare exactly is called budget-balanced, and one that distributes the welfare with excess is said to satisfy a relaxed budget-balance condition. On the other hand, if it causes a deficit we say that it violates the budget-balance condition. Each programmed player seeks to optimize its own objective function and the outcome of the game would be the result of these interactions. Since each of the players in the game setting is 1 Note that the amount of power generated by a turbine is affected by its neighbors.

21 5 selfishly optimizing a personal utility function, the outcome of the game could either converge or not. In other words, the optimization of these personal utility functions could either be conflicting and lead to divergence of the system, or agreeing and lead to convergence of the system. If the outcome converges, we say that the game has a stable outcome. There are different notions of stability or equilibrium that can arise in a game. We aim to study the the efficiency of the different forms of equilibria in a resource allocation game for different utility functions. Furthermore, there are many ways to measure the efficiency of a stable outcome; we focus on the worst case measure, the price of anarchy [13]. The Price of Anarchy (P oa) is the ratio of the worst possible equilibrium point to the optimal outcome of the system in a centralized control setting. Since the utility function of the agents is a design parameter in this domain, we aim to design utilities that ensure that the system has an equilibrium point with desirable efficiency guarantees. Equilibrium existence is guaranteed by having an instance of a potential game [27], where the players utilities are aligned with a global potential function and thus the optimizer of this function is an equilibrium point. We compare two families of utility functions, namely the Shapley value and the marginal contribution, that ensure that the game is a potential game [8] and thus guarantee equilibrium existence. The Shapley value utility function distributes the welfare fairly [3] among the players, while the marginal contribution utility gives each player a portion of the welfare equal to its additive contribution to the system; in other words, the difference between the welfare when this player is in the game and when it is not in the game. The Shapley value is a budget-balanced distribution rule, while the marginal contribution satisfies the relaxed budget-balance condition given that the welfare being distributed is submodular. We derive price of anarchy bounds for the marginal contribution utility in resource allocation games and compare them to those derived for the Shapley value in the literature. We also perform a small-scale study for a wider range of utility functions. We show that the utility design that minimizes the price of anarchy for 2-player and 3-player games lies in between the Shapley value and the marginal contribution value. Lastly, we examine the connection between the price of anarchy and the satisfiability of the budget-balance conditions of the utility designs. We show that violating the budget-balance condition worsens the

22 6 price of anarchy. 1.2 Motivation In this section, we outline our motivation for pursuing this research. We start with a general overview of game theory and its uses in this literature, and then we explain the research gap that justifies our contributions to this field. In social sciences, game theory is used in a descriptive manner to predict the emergent behavior of interacting agents/players in social systems, such as auctions, voting mechanisms, markets, social networks, etc. and thus aid in the efficient design of these mechanisms. The preferences of players in social systems are usually private and the objective of the designer is to create a mechanism that has a desirable outcome, possibly by incentivizing the players to reveal their truthful value. This type of design is known as mechanism design [10, 33] and is beyond the scope of this thesis. On the other hand, in computer science and control engineering, game theory is used in a prescriptive manner. In the latter disciplines, agents are programmable entities whose preferences could be designed in order to reach a desired objective. The preference of a player in the game is presented in the form of a utility function that maps the player s action into a value that represents the outcome for this player. This utility function is a design parameter and needs to be designed so as to meet some objective. From now on we will call this type of design utility design. Utility Design has been the topic of significant recent research work such as [1,15,20,24]. The utility function designed is sometimes referred to as a cost/welfare distribution rule. The reason for that naming convention is that in distributed control systems, there is a global welfare function (cost function) to be maximized (minimized) and in many cases the local utility function is designed as a distribution rule of the global function. The authors in [24] compare different well-known distribution rules, such as Shapley value utility, wonderful life utility, 2 and equal-share utility and they study their effect on the performance of the overall system. Their performance metrics include, 2 Wonderful life utility is the same as marginal contribution utility.

23 7 existence of equilibrium, budget-balance of the system, 3 tractability, and information requirement. The authors in [20], on the other hand, focus on one distribution rule, which is the Shapley value rule, and derive efficiency bounds for a class of resource allocation games. Both of these research papers study the game from a stationary point of view, where no learning algorithms are utilized to reach an equilibrium. In [1, 15] dynamic games are studied, where communication between agents is assumed and desired and thus the goals of the utility design in these two papers include locality 4 and convergence of negotiation mechanisms. In [15], the authors construct a methodology for designing utility functions for state-based potential games, 5 that ensures efficiency and locality. Additionally, in [1], the authors study utility functions and negotiation mechanism for vehicle-target assignment problem. In our work, we study a class of resource allocation games similar to that introduced in [20]. In their work they study the Shapley value utility which ensures that the resulting game is a potential game and thus guarantees that an equilibrium exists, whereas, we study the marginal contribution utility and the weighted Shapley value utility which also ensure that the game is a potential game. The importance of potential games is highlighted by [30]; they state and prove in their paper that Potential Games are necessary to ensure Nash equilibria in cost sharing games. To this end, we would like to mention that the families of distribution rules that ensure a potential game are the weighted Shapley value family, which contains the Shapley value, weighted Shapley value and the weighted marginal contribution family [7]. Thus by studying the marginal contribution function and the weighted Shapley value function, and comparing them to the Shapley value function studied in [20] we complete the spectrum of functions that guarantee equilibrium existence. Moreover, if equilibrium existence is not required, we do a small-scale study on two-player and three-player resource allocation games and show that there is no need to consider distribution rules that give a share of the welfare that is lower than the marginal contribution share or higher than the Shapley 3 Budget-balance means that the local utility functions sum up to the global welfare function, for each action profile. 4 Locality means that only local information is required, as opposed to global information. 5 State-based potential games are introduced in [17].

24 8 value share, and we study the continuum of possible utility functions in the range in between. One of the important design criteria that has been mentioned in the literature [20,24,25] is budget-balance of the utility function. To clarify, budget-balance refers to a distribution rule that will distribute the welfare/cost with no excess or deficit. We study the effect of having a budget-balanced utility function on the efficiency of distributed cost/welfare games. 1.3 Contributions In this thesis we use game theory to model a distributed resource allocation problem. Modeling the optimization problem as a game requires designing a utility function or a cost sharing rule for agents in the game. It is very important to note that the global objective function to be optimized is different from the function optimized by each agent separately. If there were a central controller to the system, the controller would aim to optimize the global welfare function. We would like to design a utility function for each agent such that when each agent selfishly optimizes its own utility function, the overall system performance would be close to the optimal performance. To elaborate more on this concept, we need to highlight the underlying research questions. The first question we would like to ask is, Does the utility function designed guarantee the existence of an equilibrium?. According to [30], potential games are necessary to ensure an equilibrium in resource allocation games. Therefore, if equilibrium existence is a design parameter, we are to choose a utility function from the Shapley value and marginal contribution families. Second, we would like to ask, How bad can an equilibrium get?. To quantify the efficiency we use a worst case measure, known as the price of anarchy [13]. We also examine, at a small-scale, 6 other utility functions that could optimize the price of anarchy, but do not necessarily ensure equilibrium; i.e. do not ensure that the game is a potential game. Another interesting question [25] that we tackle as well is, Does having a budget-balanced utility function affect the price of anarchy in a positive way? The main contributions of this thesis are summarized below 6 By small scale we mean 2 and 3 players in a game.

25 (1) We conduct a comparative study of two important utility functions, namely Shapley value 9 and marginal contribution as applied to a class of resource allocation games with submodular welfare functions. 7 These two functions have special properties that ensure that the game is a potential game and thus guarantee the existence of an equilibrium [30]. We derive bounds on the price of anarchy of marginal contribution function and compare it to the Shapley value bounds derived in [20]. After that we get more specific by studying worst case example-formats in both cases, in addition to the weighted Shapley value case, for the same class of resource allocation problems. (2) We do a small-scale study and show that for resource allocation games with 2 or 3 players, there is no need to consider utilities that provide a share higher than Shapley value or lower than marginal contribution. 8 After that, we explore other utility functions that fall in between Shapley value and marginal contribution and study the effect of varying the utility on the price of anarchy of these games. (3) We also study the effect of having a budget-balanced utility function on the price of anarchy. By budget-balanced we mean that the sum of the utilities add up to the value of the welfare. As a working example, we study a network coding example, named reverse car-pooling game in [19]. 1.4 Organization of Thesis This thesis is organized as follows. Chapter 2 lists the basic game theory notation used in this thesis. Then, Chapter 3 surveys briefly the relevant literature on the thesis topic. After that, a detailed study of marginal contribution utility is presented in Chapter 4. We start by deriving bounds on marginal contribution utility function and compare them to Shapley value 7 The assumptions about the class of resource allocation problems will be introduced in Chapter 2 and further explained in Chapter 4. 8 For the games we are studying, the Shapley value distribution rule always provides a greater share of the welfare than the marginal contribution rule.

26 10 bounds derived in [20]. Then we study worst case examples in each of the two cases and quantify the efficiency when using each of these utility functions. We study briefly the weighted Shapley value utility function as well. Chapter 5 studies the range of utility functions that can be produced in 2 and 3 player games quantifying the equilibrium points in this case and comparing the results to those of Shapley value and marginal contribution utility. Our last contribution, presented in Chapter 6, studies the effect of having budget-balanced utility functions on the price of anarchy. The working example in this latter chapter is a network coding problem. Lastly, we conclude the thesis in Chapter 7 and give some recommendations for future work.

27 Chapter 2 Background In this chapter, we will outline some of the basic definitions of game theory that are used in the rest of this thesis. There is a broad division of game theory into non-cooperative and cooperative game theory. In this thesis, we are concerned with non-cooperative game theory; however, we do borrow some definitions and terms from the cooperative game theory. In Section 2.1, we start by defining normal-form games, also known as strategic games, which are part of the non-cooperative game theory. By non-cooperative we do not necessarily mean that the players aim to hurt each other, but rather that the players are selfish each player is aiming to maximize its own utility. Similarly, cooperative game theory is not concerned with players who are trying to benefit each other, but rather is concerned with games where the unit is a team of players and not just one player. Sections 2.2 and 2.3 explain concepts like equilibrium, price of anarchy and price of stability which are used to analyze games. In Section 2.4, we outline welfare distribution rules in resource allocation games. Some terms from cooperative game theory, that will be used extensively in this work, will be defined in this section. Lastly, in Section 2.5 we summarize the chapter and explain its relationship to our work. This chapter is by no means comprehensive and thus we refer interested readers to [6,14,29, 38, 41] for more details on game theory and multi-agent systems.

28 Normal-Form Games Informally, a normal-form game has three main components: a set of players, an action set for each player and a utility function that defines the player s preferences over different actions. To explain these concepts in more depth, let us consider the well known game prisoner s dilemma. In this game there are 2 players that are prisoners suspected of a crime and are interrogated separately. Each player can either confess to (C) or deny (D) the crime. If both players confess, they both spend 1 year in jail. If only one of them confesses and the other denies, the one who confesses spends 4 years in jail and the one who denies is set free. However, if they both deny they both spend 3 years in jail. Figure 2.1 represents this game in matrix form which is the most common way to represent strategic or normal-form games [14]. C D C 1, 1 4, 0 D 0, 4 3, 3 Figure 2.1: Prisoner s Dilemma Game The rows in the figure represent the actions of the row player while the columns represent the actions of the column player. The first number in every box is the utility of the row player and the second number is the utility of the column player for this associated box. For example, the top right box represents the case when the row player chooses to confess while the column player chooses to deny; the utility of the row player in this case is 4 and that of the column player is 0; these numbers reflect the number of years spent in jail. The prisoner s dilemma example given above is a 2-player game, where each player has only 2 actions to choose from. In general, in a strategic-form game, there could be up to n players, each having a finite number of actions to choose from. The possible outcome of the game would be represented as the cross combination of the different actions. More formally, we define normal-form games as follows: Definition 1 (Normal-form game) A finite normal-form game G consists of

29 13 A set of players N = {1,, n}, indexed by i A set of actions A i for each player i, which forms the action profile A = A 1 A n A utility function U i : A R that defines the player s preferences among different actions. The utility function is sometimes written as U i (a i, a i ) to denote that it is a function of a i, the action of player i, as well as, a i the action of the other players; a i = {a 1,, a i 1, a i+1,, a n } 2.2 Equilibrium In any dynamic system, an equilibrium point is defined to be a point where the system is stable. Although in general it makes sense to talk about the dynamics before explaining what we mean by an equilibrium point, in all what follows we will consider a stationary game with no dynamics. We will examine each of the possible outcomes of the game and check whether they satisfy the equilibrium conditions or not. Looking back at the prisoner s dilemma example, that is summarized in Figure 2.1, we have 4 possible outcomes for this game represented by the 4 boxes in the figure. When we want to discuss whether a certain outcome is an equilibrium point or not, we look at each player separately and see if this player has an incentive to deviate or not. If no player has an incentive to unilaterally deviate, then the corresponding outcome is an equilibrium. For instance, in the prisoner s dilemma example, both players choosing to deny (D) is the only equilibrium in this game. This is because, if any player unilaterally deviates 1 ; i.e. chooses to confess (C), this deviating player would spend 4 years in jail instead of 3. Note that we do not consider that both players deviate at the same time and that s what we mean by the word unilateral. In Section 2.2.1, we formally define the different types of equilibrium that can arise in a game. After that, in Section we question the existence of equilibria and describe a class of games called potential games that guarantee the existence of an equilibrium. 1 A unilateral deviation of the row player means moving from the bottom right box to the top right, while a unilateral deviation of the column player means moving to the bottom left box

30 Types of Equilibria There are different notions of stability or equilibrium that can arise in a game, such as pure Nash equilibria, mixed Nash equilibria, correlated and coarse correlated equilibria. Before defining the different types of equilibrium, we would like to differentiate between two types of strategies for playing a game; namely pure strategy and mixed strategy. On one hand, pure strategy is the case when each player selects a specific action with certainty. On the other hand, a mixed strategy is the case when each player determines a probability distribution for his possible actions. For example, choosing D (deny) is a pure strategy for the row player in the prisoner s dilemma game, while choosing D with probability 0.7 and C with probability 0.3 is a mixed strategy. For the set of actions available for player i, (A i ) denotes the probability distribution over the actions of that player. In the following sections, we will formally define these 4 different types of equilibrium Nash Equilibrium Pure Nash equilibrium is related to pure strategies. Informally, a pure Nash equilibrium is an action profile where no player has an incentive to unilaterally deviate to another pure action. Definition 2 (Pure Nash Equilibrium) An action profile a ne A is called a pure Nash equilibrium, if for each player i N, U i (a ne i, a ne i) = max U i (a i, a ne i) (2.1) a i A i Pure Nash equilibrium could be further divided into strong Nash and weak Nash equilibrium. Strong Nash equilibrium would satisfy the stronger condition U i (a ne i, a ne i ) > U i(a i, a ne i ) for every a i A i, whereas weak Nash equilibrium satisfies the weaker condition of the definition, that can be rewritten as U i (a ne i, a ne i ) U i(a i, a ne i ) for every a i A i. Mixed Nash equilibria are related to mixed strategies. A mixed Nash equilibrium is a strategy profile where no player has an incentive to unilaterally deviate to another mixed strategy. By strategy profile we mean a probability distribution over the different actions available for each player.

31 Definition 3 (Mixed Nash Equilibrium) A strategy profile s ne = {s ne 1,, sne n }, where s ne i (A i ), is called a mixed strategy Nash equilibrium, if for each player i N, 15 where U i (s i, s i ) is the expected utility for player i, defined by U i (s ne i, s ne i) = max U i(s i, s ne i) (2.2) s i (A i ) U i (s i, s i ) = U i (a)s a 1 1 san n a A and s a i i is the probability that player i will play action a i. We would like to note that any pure Nash equilibrium can be written as a mixed Nash equilibrium; for each player one of the actions is played with probability 1 and all other actions are played with probability Correlated Equilibrium Definition 4 (Correlated Equilibrium) A probability distribution z is a correlated equilibrium if for each player i N and for all actions a i, a i A i, U i (a i, a i )z (ai,a i) a i A i a i A i U i (a i, a i )z (ai,a i) (2.3) where z a is the probability of the joint action a and a A za = 1 In order to explain correlated equilibrium we will consider the traffic intersection game, shown in Figure 2.2. There are 2 players in this game at a traffic intersection. Each player has 2 possible actions: stop and go. If both players choose to go, there will be an accident and thus the utility for each one of them is 0. However, if one player always stops and the other always goes, the player who stops will always be delayed and thus receives a utility of 1 (better than the accident case) and the player who goes benefits and receives a utility of 6. Lastly, if both players choose to stop their corresponding utility is 3 which is better than the accident and better than stopping for another person, but worse than going alone. There are two pure Nash equilibria for this game which correspond to the action profiles where one player stops and the other goes. Each one of

32 these equilibria involves that only one player always has the right to go and the other always has to stop. An external observer or referee of the game would coordinate the actions of the 2 players, by allowing the players to stop and go alternatively. This involves giving the outcome {go,stop} a probability of 0.5 and that of {stop,go} a probability of 0.5 as shown in Figure 2.3. Each player now is given the choice either to obey the external observer s probability distribution, or play any other action. A correlated equilibrium is one where no player has an incentive to unilaterally disobey the referee given that all other players will obey him. Note that the probability distribution given in Figure 2.3 cannot be written in a mixed strategy format. However, the opposite is always possible; any mixed strategy can be written as a joint probability distribution. Thus, any mixed Nash equilibrium is a correlated equilibrium. A mixed strategy can be written as a joint strategy by defining z a = Go Stop Go 0, 0 6, 1 Stop 1, 6 3, 3 Figure 2.2: Traffic Intersection Game i N s a i i. 16 Go Stop Go Stop Figure 2.3: A Joint Probability Distribution for the Traffic Intersection Game Coarse Correlated Equilibrium Definition 5 (Coarse Correlated Equilibrium) A probability distribution z is a coarse correlated equilibrium if for each player i N and for all actions a i A i, U i (a)z a a A a i A i U i (a i, a i )z a i i (2.4)

33 17 where z a is the probability of the joint action a and z a i i is the marginal distribution of all players other than player i and can be written as z a i i = a i A i z (a i,a i) Coarse correlated equilibrium is a more general solution concept than the correlated one. 2 It is also known as no-regret equilibrium point. Consider a similar scenario to the traffic intersection one, this time each player is given a choice either to opt in the game and obey the referee or to opt out of the game given that every other player opts in. A coarse correlated equilibrium is one where no player has a unilateral incentive to opt out of the game. It can be shown that any correlated equilibrium is also a coarse correlated equilibrium. The relationship between the different sets of equilibria is shown in Figure 2.4. As noted earlier, each set is contained in the subsequent one. Pure Nash Eq Correlated Eq Mixed Nash Eq Coarse Correlated Eq Figure 2.4: Different sets of Equilibria Existence of Equilibrium After explaining the different notions of equilibria that can arise in a normal-form game, a natural question would be does an equilibrium always exists?. The answer to this question depends on what type of equilibrium we are looking for. As we have seen in the prisoners dilemma example, a unique pure Nash equilibrium exits in this game. However, in general, in some games a pure Nash equilibrium does not exist, while in other games multiple pure Nash equilibria exist. Consider the following two examples for illustration. 2 In game theory, a solution concept is another name for an equilibrium point.

34 18 Example 1: Matching Pennies Figure 2.5 represents a (2-player) matching pennies game, where each player gets to choose either Heads or Tails. The row player wins if the pennies match and the column player wins if they dismatch. As we can see the objectives of the players are contradicting and therefore there is no stable outcome of the game; i.e. there does not exist an outcome where none of the players has an incentive to deviate. Nevertheless, if both players play heads with probability 50% and tails with probability 50%, then this mixed strategy results in a mixed Nash equilibrium. heads tails heads 1, 1 1, 1 tails 1, 1 1, 1 Figure 2.5: Matching Pennies Example 2: Stag Hunt Figure 2.6 shows a stag hunt game, where there are 2 players going for a hunt together and each player can either choose to hunt for a stag or a hare. The players are only successful in hunting the stag if they cooperate; i.e. both choose stag. As for the hare, any player can hunt it on his own. The stag is worth more than a hare. As we can see, in this game there are 2 equilibria, one if both players choose stag and the other when both choose hare. Therefore, multiple pure Nash equilibria could exist in the same game. Also, a more careful observer would notice that one of the equilibria is better than the other, specifically the case when they both choose stag. We will get back to this concept later when we discuss the performance metrics in Section 2.3. Stag Hare Stag 2, 2 0, 1 Hare 1, 0 1, 1 Figure 2.6: Stag Hunt It is worth noting that John Nash has a popular theorem that states that every game with finite number of players and finite number of actions has at least one (mixed) Nash equilibrium [28].

35 Potential Games Potential games are a special class of games that ensure the existence of a pure Nash equilibrium. In a potential game, a change in a player s utility is aligned with a change in a global function, as shown in Figure 2.7. In other words, there exists a function, called the potential function, φ : A R such that for every player i N, every a i A i and for every 2 actions a i, a i A i, U i (a i, a i ) U i (a i, a i ) = φ i (a i, a i ) φ i (a i, a i ) (2.5) The maximizer of the potential function will correspond to a pure Nash equilibrium and therefore, every potential game has at least one pure Nash equilibrium. A B C 3, 4 2, 1 D 1, 6 0, 3 (a) Payoff Matrix A B C 5 2 D 3 0 (b) Potential Figure 2.7: The figure shows the payoff matrix of a potential game and the associated potential values. A unilateral deviation of a player causes a change in the utility equal to the change in the potential value. 2.3 Performance Metrics In this section, we define two quantities, namely the price of anarchy and the price of stability, that are used to compare different equilibria in a game to the optimal performance of the game. Definition 6 (Price of Anarchy (PoA)) For a class of games G, the price of anarchy is an upper bound on the welfare of the worst equilibrium that could arise in this class of games. ( P oa(g) = sup max G G a ne G W (a opt ; G) ) W (a ne ; G) (2.6) where W (a; G) is the welfare associated with the outcome (a) in a specific game G G, a ne denotes a Nash equilibrium action profile and a opt represents the optimal outcome of the game; i.e. a opt arg max a A W (a)

36 Definition 7 (Price of Stability (PoS)) For a class of games G, the price of stability is a lower bound on the welfare of the worst equilibrium that could arise in this class of games. 20 ( P os(g) = sup G G min a ne G W (a opt ; G) ) W (a ne ; G) (2.7) where W (a; G) is the welfare associated with the outcome (a) in a specific game G G, a ne denotes a Nash equilibrium action profile and a opt represents the optimal outcome of the game; i.e. a opt arg max a A W (a) In order to explain the concept of the price of anarchy and the price of stability, we will consider the following simple examples. Consider the stag hunt example depicted in Figure 2.6. Let us assume that there is an observer who cares about both players and constructs a global welfare function equal to the sum of their utilities as shown in Figure From the observer s point of view, the Nash equilibrium produced when both players select stag is better than that produced when they both select hare. Thus, an optimal outcome would be the case when both players select stag. However, if both players choose hare, they would be 2 4 = 50% worse off than the optimal outcome. In this example, the optimal outcome is also an equilibrium, but that might not always be the case. Stag Hare Stag 4 1 Hare 1 2 Figure 2.8: Stag Hunt: Total Welfare Consider for illustration again that there is an external observer for the prisoner s dilemma game shown in Figure 2.1. Figure 2.9 shows for the same game the sum of utilities as the global welfare. Note that now the optimal 4 outcome is when both players select confess, however this is not an equilibrium in the game. The only equilibrium in that game is 6 2 = 3 times worse than 3 In general, the global welfare function is not necessarily defined as the sum of the players utilities. 4 Also known as pareto optimal in the game theory literature. We are using the word optimal here for simplicity.

37 21 the optimal. 5 C D C 2 4 D 4 6 Figure 2.9: Prisoner s Dilemma Game The price of anarchy is the ratio between the worst equilibrium and the optimal output in a game, while the price of stability is the ratio between the best equilibrium and the optimal output in a game [10, 31]. For demonstration, consider the 2 simple examples mentioned above, namely the stag hunt and prisoners dilemma. In the stag hunt example, the welfare associated with the optimal outcome is 4, the one associated with the best equilibrium outcome is also 4, and that associated with the worst equilibrium outcome is 2. Therefore, the price of anarchy in this case would be equal to 4 2 = 2, and the price of stability in the stag hunt case would be equal to 4 4 = 1. In the prisoners dilemma example, the welfare associated with the optimal outcome is 2 and that associated with the (only) equilibrium outcome is 6. Therefore, both the price of anarchy and the price of stability in this case would be equal to 6 2 = Welfare Distribution Rules So far we have explained the basic terms in a normal-form game, with 2-player examples. In the rest of this thesis, we will be considering a more advanced game setting. In particular, we will 5 Note that, a utility of ( 6) represents 6 years in jail and therefore, is worse than a utility of ( 2) which represents 2 years in jail.

38 22 have n players, each with a finite number of actions to choose from. As n gets larger than 3, we usually do not use the matrix form anymore to represent the game. The resource allocation game we are considering will consist of: A finite set of agents N = {1,..., n} A finite set of resources R = {r 1,..., r m } A set of actions for every player A i 2 R 6 An action profile is given by a = (a 1, a 2,..., a n ) A, where a i A i The set of action profiles is denoted by A = A 1 A n A class of welfare functions W A welfare function for each resource r, denoted by W r W A global welfare function W : A R that is separable; i.e. equal to the summation of the welfare at each resource W (a) = r R W r(a r ) A local utility function for each player U i (a) = U i (a i, a i ) : A R, where local means that the utility function is solely dependent on the set of players selecting the same resource as player i. We define the utility function as a distribution rule of the welfare on resource r as follows 7 U i (a) = f r (i, {a} r ) r a i where {a} r = {j N : r a j } In the rest of this section, we will outline different distribution rules that allow us to define the utility function of a player U i (a) as a distribution rule of the welfare function W r (a). 6 In most of this thesis, we will be dealing with single-selection resource allocation games where each player can only select one resource at a time: A i R. 7 In most of this thesis, we will be dealing with anonymous resource allocation games where the distribution rule is dependent on the number of players on the resource and not which set of players: f r(i, S) = f r(j, T ) i, j N : r A i A j, if S = T.

39 Equal Share The most natural welfare distribution rule, is the equal share rule. In other words, divide the welfare among the players equally. More formally, the utility of a player, selecting resource r is given by Ui ES (a i = r, a i ) = fr ES (i, {a} r ) ai =r = W r({a} r ) (2.8) a r where W r denotes the welfare function associated with resource r, {a} r = {i N : r = a i } denotes the set of players selecting resource r in allocation (a), and a r = {i N : r = a i } denotes the number of players who selected resource r in allocation (a) Marginal Contribution The marginal contribution utility function, also known as wonderful life utility, assigns for each player its marginal contribution to the game. In other words, the difference between the welfare function when this player is in the game and when it is not in the game. More formally, it is defined as follows Ui MC (a i = r, a i ) = fr MC (i, {a} r ) ai =r = W r ({a} r ) W r ({a} r \ {i}) (2.9) where W r denotes the welfare function associated with resource r, and {a} r = {i N : r = a i } denotes the set of players selecting resource r in allocation (a) Shapley Value The Shapley value distribution rule was originally introduced by Lloyd Shapley [37], in cooperative game theory, to internally distribute welfare between players that are considered a team. Lately, the Shapley value has been used in non-cooperative game theory as a distribution rule of the welfare function among competing players as in [20, 24].

40 24 The Shapley value is formally defined as follows: Ui SV (a i = r, a i ) = fr SV (i, {a} r ) ai =r T!( a r T 1)! = (W r (T i) W r (T )) (2.10) a r! T {a} r\{i} where W r denotes the welfare function associated with resource r, and {a} r = {i N : r = a i } denotes the set of players selecting resource r in allocation (a) The weighted Shapley Value The weighted Shapley value is a more general form of the Shapley value, where each player has a weight w i > 0. The weighted Shapley value reduces to the Shapley value when all weights are equal to 1; w i = w j = 1 i, j N. The weighted Shapley value is given by Ui WSV (a i = r, a i ) = fr WSV (i, {a} r ) ai =r = w ( i j T w j T {a} r:i T T 1 T ) ( 1) T T1 W r (T 1 ) (2.11) where w i > 0 is the weight of player i N, W r denotes the welfare function associated with resource r, and {a} r = {i N : r = a i } denotes the set of players selecting resource r in allocation (a). 2.5 Summary Distribution Rules To sum up, we would like to highlight the differences between the distribution rules. Marginal Contribution, Shapley value and weighted Shapley value rules all ensure that the game is a potential game and thus guarantee the existence of an equilibrium [23]. For the marginal contribution case, the potential function is the same as the welfare function.

41 For the Shapley value family of functions, the potential function may be different from the welfare function. 25 The Shapley value is more computationally intensive than the marginal contribution. The Shapley value function is budget-balanced, which means that the utilities sum up to the total welfare for every allocation (a). i N U SV i (a) = W (a) (2.12) The Shapley value function reduces to the equal share function when the game is anonymous Performance Metrics Revisiting the price of anarchy and the price of stability definitions outlined in Section 2.3 we would like to note that The optimal outcome in the resource allocation game would be the maximizer of the global welfare function. In other words, consider there is a central controller that is allowed to allocate each agent to a resource from its action set. The optimal outcome of the game would be governed by this central controller which may or may not coincide with an equilibrium of the game. If the optimal outcome is an equilibrium of the game, this means that the price of stability is equal to 1. As a side note, since the marginal contribution ensures that the game is a potential game and the potential function is the welfare function, it is always the case that the 8 In anonymous resource allocation games the distribution rule is dependent on the number of players on the resource and not which set of players: f r(i, S) = f r(j, T ) i, j N : r A i A j, if S = T.

42 26 price of stability is 1 for marginal contribution. This means that the best equilibrium is an optimal point. However, in general there could be several Nash equilibria that are suboptimal and that is why it is important to have guarantees for all Nash equilibria. More specifically, the price of anarchy quantifies how bad can the worst equilibrium get. 9 If an instance of a game has its price of anarchy equal to 1, this means that for this instance every equilibrium is an optimal point. In the rest of this thesis, we would like to draw conclusions about the different distribution rules mentioned in Section 2.4 and their effect on the price of anarchy and price of stability defined in Section In the literature, there are distributed learning algorithms that provide probabilistic convergence to the best Nash equilibria, e.g., log linear learning [2, 22, 35]. However, these algorithms have convergence rates that are either uncharacterized or grow exponentially in the size of the game [35].

43 Chapter 3 Related work In this chapter we will briefly survey the game theory literature in the area of designing utility functions and bounding their efficiency. We focus on the main theorems that are the foundation of our work. First, in Section 3.1 we define a valid utility system and revise the main theorem of [40]. Then, in Section 3.2, we define smooth games and robust price of anarchy, which were first defined in [32], and explain their importance in the domain of our work. We also summarize the main conclusion of [30] regarding potential games, in Section 3.3. After that, in Section 3.4 we demonstrate the results of the work in [20], which studies the efficiency bounds of the Shapley value utility design in distributed resource allocation games. Lastly, we conclude the chapter in Section 3.5 and draw relations between the literature summarized and our work. 3.1 Valid Utility Systems In this section, we restate the definition of valid utility systems, originally defined by Vetta in [40] and state one of the main theorems in their work, that bounds the price of anarchy of valid utility systems. 1 We will start with some basic definitions: A set function is a function that takes a set as its argument and outputs a real number; f : 2 S R, where S is a set. A submodular function is a set function that has decreasing marginal increments; in other words, as the set gets larger, the marginal contribution of an element gets smaller. For 1 We change the notation used in the original paper to match the notation used in this thesis.

44 28 example, consider a group of people working on a project, if there are only 2 in the team, then each one of them contributes more than if there are 10 individuals in the team; so the contribution of 1 person decreases as the team size gets larger. Formally, a submodular function f : S R satisfies the following condition f(x) + f(y ) f(x Y ) + f(x Y ); X, Y S (3.1) The discrete derivative of X in the direction of D S X is defined as f D = f(x D) f(x) (3.2) A well-known result states that if a function f is submodular, then A B f D(A) f D(B); D S B and A B S, i S (A B) f(a {i}) f(a) f(b {i}) f(b) Definition 8 (Valid Utility System [40]) Let G be a system/game with A set of players N = {1,, n}, indexed by i A set of actions A i for each player i, which forms the action profile A = A 1 A n i is a null strategy for player i; captures the case when player i opts out of the game A set of utility functions {U i } i N, where U i : A R is a utility function for player i that defines the player s preferences among different actions. A global objective W : A R +. Then, G is a valid utility system if it satisfies the following conditions (i) The global objective W is submodular (ii) U i (a) W (a) W ( i, a i ) for every agent i N and action profile a A

45 29 (iii) i N U i(a) W (a) for every action profile a A Condition (i) ensures that the objective function has decreasing marginal increments, condition (ii) states that a player s utility is always greater than or equal to the player s marginal contribution to the global objective W, and condition (iii) states that the utility functions satisfy a relaxed budgetbalance condition; i.e. guarantee that the welfare is divided among the players with a possible excess remaining, but never a deficit. We would like to note that a marginal contribution utility satisfies condition (ii) with equality. On the other hand, Shapley value utility and equal share utility satisfy condition (iii) with equality. This means that in a valid utility system/game, the minimum share that can be given to players is their marginal contribution share, while the maximum share that can be given to them is a share that balances the budget, which ensures that the total welfare is distributed between the players, with no excess. Theorem 1 (Vetta, 2002 [40]) Let W be a non-decreasing, submodular set function. If G = (N, {A i } i N, W, {U i } i N ) is a valid utility system, then for any Nash equilibrium W (a opt ) 2W (a ne ), where W (a opt ) denotes the welfare at the optimal allocation and W (a ne ) denotes the welfare at the Nash equilibrium. 2 This theorem informally states that for a valid utility system if a pure Nash equilibrium exists, then the price of anarchy is Smooth Games In this section, we will define smooth games and robust price of anarchy that were introduced by Roughgarden in [32]. A welfare-maximization game is a game where the objective is a welfare function W, to be maximized, while a cost-minimization game is a game where the objective is a cost function C, to 2 Note that any Nash includes mixed Nash equilibrium.

46 30 be minimized. 3 The price of anarchy, in this thesis, is defined such that it is always a number greater than 1; i.e. games. W (a opt ) W (a ne ) for welfare maximization games and C(ane ) C(a opt ) for cost minimization Players in welfare maximization games have utility functions, U i (a), to be maximized, while in cost-minimization games, they have cost functions, J i (a), to be minimized, where (a) is an action profile. Definition 9 [Smooth Games [32]] A welfare-maximization game is (λ, µ)-smooth with λ 0 and µ > 0, if for every two outcomes a, a A, n U i (a i, a i ) λ W (a ) µ W (a) (3.3) i=1 A cost-minimization game is (λ, µ)-smooth with λ 0 and µ < 1, if for every two outcomes a, a A, n J i (a i, a i ) λ C(a ) + µ C(a) (3.4) i=1 Definition 10 (Robust POA [32]) The robust price of anarchy of a welfare-maximization game that is (λ, µ) smooth, is { 1 + µ inf λ } : (λ, µ) s.t. the game is (λ, µ)-smooth (3.5) The robust price of anarchy of a cost-minimization game that is (λ, µ)-smooth, is inf { λ } : (λ, µ) s.t. the game is (λ, µ)-smooth 1 µ (3.6) It is stated, in [32], that Definition 10 is for smooth games that satisfy a sum objective condition given by W (a) i C(a) i U i (a) (3.7) J i (a) (3.8) 3 In Chapters 4 and 5 we deal with welfare maximization games, while in Chapter 6 a cost minimization game is introduced.

47 31 Equations (3.7) and (3.8) satisfy a relaxed budget-balance condition; i.e. allow for excess in the budget, but not for deficit. Proving that a welfare-maximization game is (λ, µ) smooth, implies that a robust price of anarchy bound of (1+µ) λ holds for all mixed Nash equilibria, correlated equilibria, and coarse correlated equilibria [32]. Similarly, a cost-minimization game that is (λ, µ) smooth, has a robust price of anarchy bound of λ (1 µ) that holds for all sets of equilibria [32]. Informally, a game is smooth if there exist a family of parameters (λ, µ) that satisfies certain conditions that relate the system objective function to the players personal utilities. If a game is smooth, according to the formal definitions above, then the price of anarchy bounds are the same for all equilibrium sets shown earlier in Figure 2.4. This is a great result, because usually as the set gets larger, the worst case could get worse which means that the price of anarchy gets larger. However, for smooth games the equilibria sets get larger, but the price of anarchy stays the same. 3.3 Potential Games Necessity In this section, we will relate some of the basic concepts explained in Chapter 2 to the work done by Gopalakrishnan et al. in [30]. In Section 2.2.2, we explained the existence of equilibrium in non-cooperative games and we defined potential games as a class of games where pure Nash equilibrium always exists. After that, in Section 2.4, we outlined the model for our resource allocation problem, and we listed some of the possible distribution rules for this model. Moreover, in Section 2.5, we highlighted the main differences between those distribution rules, such as: Marginal Contribution, Shapley value and weighted Shapley value rules all ensure that the game is a potential game and thus guarantee the existence of an equilibrium. The Shapley value is more computationally intensive than the marginal contribution. The Shapley value function is budget-balanced, which means that the utilities sum up to the total welfare i N U i(a) = W (a)

48 32 In [30], the authors stated and proved that Potential Games are Necessary to ensure Pure Nash Equilibrium in Cost Sharing Games. The authors use a model that is very similar to ours; in fact the only difference is that in their model players are allowed to select multiple resources, while in our model players can only select a single resource at a time i.e. A i 2 R rather than A i R. First, they represent the welfare function W in terms of its basis representation. 4 Then, they show that any distribution rule that guarantees equilibrium existence must have a basis representation. It is worth noting that applying the Shapley value distribution rule to a welfare function W, is equivalent to applying the marginal contribution rule to some other welfare function W. W can be interpreted as the potential function of the game, since a difference in the potential function is aligned with any player s unilateral deviation, which corresponds to a marginal contribution to the potential function. To this end, we will restate the main theorems in [30] in an informal 5 manner as follows: Theorem 2 (Gopalakrishnan et al., 2013 [30]) Given a set of fixed local welfare functions, all resource allocation games possess an equilibrium if and only if the distribution rules are equivalent to generalized weighted Shapley value 6 distribution rules on some ground welfare functions. Theorem 3 (Gopalakrishnan et al., 2013 [30]) Given a set of fixed local welfare functions, all resource allocation games possess an equilibrium if and only if the distribution rules are equivalent to generalized weighted marginal contribution 7 distribution rules on some ground welfare functions. Note that the two theorems are closely related according to the natural relation between Shapley value and marginal contribution distribution rules explained above. 4 The basis representation was first introduced in [36]. It is a method of breaking down a set function into its basic components. 5 The notation of the original paper is omitted for simplicity and brevity 6 Refer to [30] for the formal definition of generalized weighted Shapley value 7 Refer to [30] for the formal definition of generalized weighted marginal contribution

49 Efficiency of Shapley Value in Resource Allocation Games The Shapley value utility function, shown in Equation (2.10), is a distribution rule that divides the welfare function, W r among the players on resource r. The authors in [20] study the Shapley value utility function as applied to a single-selection, anonymous resource allocation game with submodular welfare functions. 8 Single-selection means that each player is allowed to select only one resource at a time; A i R, whereas, anonymous means that sets of players of the same size are equivalent; i.e. the welfare produced at a resource is dependent on the number of players on that resource and not which set of players are on the resource. The resource allocation game with the Shapley value utility forms a valid utility system; i.e satisfies all the conditions of Definition 8 stated above. Therefore, according to Theorem 1 the system will have a price of anarchy of 2. As noted earlier, the price of anarchy is an upper bound for the welfare associated with the worst case equilibrium. The authors in [20] aim to tighten the price of anarchy, derive a lower upper bound, for the resource allocation games aforementioned. Towards this goal, they derive the following theorems which tighten the upper bound of the worst case equilibrium, beyond 2, for resource allocation games with submodular functions and the Shapley value utility design. 9 They study two distinct cases: players with symmetric action sets and players with asymmetric action sets. Symmetric action sets refers to the case where all players have the same action set; A i = A j i, j N. In the symmetric setting, they bound the efficiency of the pure Nash equilibrium, by deriving the price of anarchy. Asymmetric action sets refers to the case where players action sets could be different; A i A j for some i, j N. In the asymmetric setting, they bound the efficiency of all sets of equilibria, by deriving the robust price of anarchy. 8 Refer to Section 2.4 for a formal definition of the resource allocation game. 9 The authors in [20] refer to resource allocation games as distributed welfare games.

50 34 Theorem 4 (Marden, Roughgarden, 2012 [20]) Consider any anonymous, single-selection distributed welfare game with n agents, symmetric action sets, the Shapley value utility design, and submodular welfare functions. The price of anarchy associated with pure Nash equilibria is equal to { Wr (k) 1 + max r R, k m n W r (m) k }. (3.9) m Theorem 5 (Marden, Roughgarden, 2012 [20]) Consider any single-selection distributed welfare game with n agents, asymmetric action sets, the Shapley value utility design, and submodular welfare functions. The robust price of anarchy is bounded above by 1 + max r R max k m n { Wr (k) W r (m) β(n) k } m (3.10) where β(n) = min r R min 1 x n W r (x + 1)/(x + 1). W r (x)/x The following theorem tightens the robust price of anarchy a little more than Theorem 5. Theorem 6 ((Marden, Roughgarden, 2012 [20])) Consider any single-selection distributed welfare game with n agents, asymmetric action sets, the Shapley value utility design, and submodular welfare functions. The robust price of anarchy is bounded above by 1 + η(n) where ( ( )) max Wr(k) r R,k m n W η(n) = max max{m+k n,0}+min{n m,k} βr(m) r(m) m, ( )) (3.11) max r R,k m n (1 max{k+m n,0}+min{n m,k} βr(k) where k 3.5 Summary β r (m) = m m + 1 W r (m + 1). W r (m) In summary, there are a few key studies that are the foundation of our work:

51 (1) Valid Utility Systems: These are systems/games that impose some conditions on the utility function and the welfare function and guarantee a price of anarchy of (2) Smooth Games: These are games that satisfy what is called a smoothness argument. A smoothness argument refers to a family of parameters that relate the local utility functions of players to the system s global objective function and guarantee what is called a robust price of anarchy. A robust price of anarchy is a price of anarchy that works for all sets of equilibria. (3) Potential Games necessity: Potential Games are necessary to ensure pure Nash equilibrium in cost sharing (welfare distribution) games. (4) Tightening the Price of Anarchy beyond 2: By restricting the valid utility systems/games and imposing some rules on the game, one can tighten the price of anarchy beyond 2, which means increasing the efficiency of the system above 50%.

52 Chapter 4 Marginal Contribution Utility Function in Resource Allocation Games In this chapter, we model the distributed resource allocation problem as a strategic game and study the efficiency of the marginal contribution utility design as compared to Shapley value utility design. The resource allocation game is defined in Section 2.4. Recall that the game is anonymous and single-selection, therefore the marginal contribution utility given in Equation (2.9) can be reduced to U MC i (a i = r, a i ) = W r ( a r ) W r ( a r 1) (4.1) and the Shapley value utility given in Equation (2.10) can be reduced to where a r = {i N : a i = r}. Summary of Contributions in this chapter: U SV i (a i = r, a i ) = W r( a r ) a r (4.2) In Section 4.2, we derive price of anarchy bounds for resource allocation games that utilize the marginal contribution design shown in Equation (4.1). In Theorem 7 we show that if the players action sets are symmetric, the price of anarchy is 1. This result implies that every equilibrium in the symmetric case is also an optimal allocation. In Theorem 8, we derive a robust price of anarchy bound for the more general case of

53 37 asymmetric action sets given by, { 2 min min {Wr (x + 1) W r (x) } r R x n W r (x)/x Then, in Section 4.3 we perform a comparative analysis between our marginal contribution bounds and the Shapley value bounds derived in [20]. We start by illustrating some motivating examples that show that the Shapley bounds in the asymmetric setting are less than or equal the marginal contribution bounds, for a family of submodular functions given by x d for x R + and for all values of 0 d 1. Then we formally prove in Theorem 9 that for any family of submodular functions, the robust price of anarchy bounds derived are always greater than or equal the Shapley value bounds derived in the literature. After that, in Section 4.4, we derive worst case example templates for resource allocation games that utilize the Shapley value and the marginal contribution utilities, respectively. We derive these formats by starting from a general game and going through a group of reductions that reserve the Nash equilibrium conditions, but make the welfare produced in the optimal allocation increase. These reductions imply that the price of anarchy is getting higher. Proposition 13 states that the worst possible form of equilibrium in resource allocation games that utilize the Shapley value function given in Equation (4.2), will be in the form of having all players collocated on one resource in the Nash equilibrium and not in the optimal allocation. Proposition 14 states that the worst possible form of equilibrium in resource allocation games that utilize the marginal contribution function given in Equation (4.1), will be in the form of having all players spread on different resources in the Nash equilibrium and shifted by one resource in the optimal allocation.

54 We also show, in Proposition 16, that for anonymous games weighted Shapley value utility function results in a worse price of anarchy than the equal-weight Shapley value. 38 Lastly, we summarize the chapter in Section Game Model Recall from Section 2.4 that a resource allocation game G G can be represented by the tuple G = {N, R, {W r W} r R, {A i } i N, {U i } i N }. The components of the game G G can be summarized as follows: A finite set of agents N = {1,..., n} A finite set of resources R = {r 1,..., r m } A set of actions for every player A i R (single-selection) A class of anonymous submodular welfare functions W: An anonymous welfare function W r W for each resource r. A welfare function is anonymous if it is a function of the number of players on the resource a r = {i N : a i = r} rather than the specific set of players on the resource {i N : a i = r} The global welfare function W : A R is separable, where separable means that it is equal to the summation of the welfare at each resource W (a) := r R W r ( a r ) A marginal contribution utility function for each player U i (a) = U i (a i, a i ) : A R that defines the share of player i for the action profile A U MC i (a i = r, a i ) = W r ( a r ) W r ( a r 1)

55 Price of Anarchy Bound In this section, we derive price of anarchy bounds for the resource allocation game using the marginal contribution utility design. First we show that every single-selection, anonymous resource allocation game with submodular welfare functions, and the marginal contribution utility design satisfies conditions (i) (iii) in Definition 8. Condition (i) follows naturally and Condition (ii) follows tightly by definition of the marginal contribution utility function given by Equation (4.1). Condition (iii) can be proved as follows Ui (a i, a i ) = W (a) W (( i, a i ) (4.3) i N W (a 1:i, i+1:n ) W (a 1:i 1, i:n ) i N (4.4) = W (a) (4.5) where i:n represents the null strategy of players i to n; i.e. the case when those players opt out of the game. Hence, Theorem 1 ensures that the price of anarchy is at most 2, irrespective of the number of resources or the number of players. In Sections and we derive a tighter bound for the price of anarchy by focusing on the specific class of games given above in Section 4.1. More specifically, in Section we study the symmetric case where players have the same action set; i.e. A i = R i N, while in Section we study the more general case of asymmetric action sets where players action sets are not necessarily the same; A i R and A i A j for some i, j N Symmetric Action Sets In the case of symmetric action sets the agents action sets are identical i.e. A i = R; i N. The following theorem implies that, using the marginal contribution utility design with symmetric action sets, where every player is allowed to select any resource, every pure Nash equilibrium is an optimal allocation. Theorem 7 Consider any anonymous, single-selection resource allocation game with n agents, symmetric action sets, the marginal contribution utility design given by Equation (4.1), and sub-

56 modular welfare functions. The price of anarchy associated with pure Nash equilibria is equal to one. 40 Proof: Assume for contradiction that there exists a pure Nash equilibrium that is not an optimal allocation. As a first step of the proof, we will assume that there are only 2 resources, r 1 and r 2. Then we will generalize this result for m resources. Let x r, y r be the number of players on resource r in the Nash equilibrium allocation and the optimal allocation respectively. Without loss of generality, suppose that x 1 > y 1 (4.6) x 2 < y 2 (4.7) And note that the number of players n is constant in both allocations and therefore, x 1 + x 2 = y 1 + y 2 = n (4.8) = x 1 y 1 = y 2 x 2 (4.9) As explained earlier, the global objective is to find an allocation that optimizes the system welfare a opt arg max W (a), and the Nash equilibrium condition states that no player has a unilateral a A incentive to deviate. W r1 (y 1 ) + W r2 (y 2 ) > W r1 (x 1 ) + W r2 (x 2 ) (4.10) W r1 (x 1 ) W r1 (x 1 1) W r2 (x 2 + 1) W r2 (x 2 ) (4.11) Equation (4.10) follows from the optimal condition and Equation (4.11) follows from the Nash equilibrium condition. Note that from Equation (4.6) x 1 > y 1 0 = x 1 > 0, therefore Equation (4.11) is valid. From the submodularity of W ri, we have W r1 (x 1 ) W r1 (y 1 ) (x 1 y 1 )(W r1 (x 1 ) W r1 (x 1 1)) (4.12) and W r2 (y 2 ) W r2 (x 2 ) (y 2 x 2 )(W r1 (x 2 + 1) W r1 (x 2 )) (4.13)

57 41 Equations (4.9), (4.11), (4.12) and (4.13) imply that W r1 (x 1 ) W r1 (y 1 ) W r2 (y 2 ) W r2 (x 2 ) (4.14) Rearranging Equation (4.14), we get equation (4.15) which contradicts equation (4.10). W r1 (y 1 ) + W r2 (y 2 ) W r1 (x 1 ) + W r2 (x 2 ) (4.15) This shows that for the 2 resource case, the Nash Equilibrium allocation is an optimal allocation. Now, we consider the more general case of having multiple resources. First, we divide the resources into 2 sets, the first set of resources, R 1 is such that x i > y i for every r i R 1 and the other set, R 2 is such that x j y j for every r j R 2. We use the subscript i to denote a resource on R 1 and the subscript j to denote a resource on R 2. Again we note that the number of players is constant in both allocations and therefore, x r = y r = n r R r R = (x i y i ) = (y j x j ) (4.16) r i R 1 r j R 2 W ri (y i ) + W rj (y j ) r i R 1 r j R 2 W ri (x i ) + W rj (x j ) (4.17) r i R 1 r j R 2 W ri (x i ) W ri (x i 1) W rj (x j + 1) W rj (x j ) (4.18) Equation (4.17) follows from the optimal condition and Equation (4.18) follows from the Nash equilibrium condition for any 2 pairs of resources, where r i R 1 and r j R 2 (since players action sets are symmetric, any player can unilaterally deviate from a resource r i R 1 to a resource r j R 2 ). From the submodularity of W ri, we have W ri (x i ) W ri (y i ) (x i y i )(W ri (x i ) W ri (x i 1)) (4.19) r i R 1 r i R 1 W rj (y j ) W rj (x j ) (y j x j )(W rj (x j + 1) W rj (x j )) (4.20) r j R 2 r j R 2

58 Multiplying both sides of Equation (4.18) by (x i y i )(y j x j ) where (x i, y i ) R 1 and (x j, y j ) R 2, we can write it as 42 ( ) ( ) (x i y i )(y j x j ) W ri (x i ) W ri (x i 1) (x i y i )(y j x j ) W rj (x j + 1) W rj (x j ) (4.21) Summing Equation (4.21) over all resources in R 1 and in R 2, we have ( ) (x i y i )(y j x j ) W ri (x i ) W ri (x i 1) r i R 1 r j R 2 ( ) (x i y i )(y j x j ) W rj (x j + 1) W rj (x j ) r j R 2 r i R 1 which can be written as, (y j x j ) (x i y i )(W ri (x i ) W ri (x i 1)) r j R 2 r i R 1 (x i y i ) (y j x j )(W rj (x j + 1) W rj (x j )) (4.22) r i R 1 r j R 2 Using Equation (4.16), we can write (4.22) as, (x i y i )(W ri (x i ) W ri (x i 1)) (y j x j )(W rj (x j + 1) W rj (x j )) (4.23) r i R 1 r j R 2 Equations, (4.19), (4.20) and (4.23) imply that W ri (x i ) W ri (y i ) W rj (y j ) W rj (x j ) (4.24) r i R 1 r j R 2 Rearranging Equation (4.24), we get Equation (4.25) which contradicts Equation (4.17). W ri (y i ) + W rj (y j ) r i R 1 r j R 2 W ri (x i ) + W rj (x j ) (4.25) r i R 1 r j R 2 Therefore, there can t be a pure Nash equilibrium that is not an optimal allocation, when using the marginal contribution utility design and allowing for symmetric action sets Asymmetric Action Sets Consider the case of asymmetric action sets where each agent s action set is not necessarily identical A i A j for some i, j N.

59 Theorem 8 Consider any single-selection resource allocation game with n agents, asymmetric action sets, the marginal contribution utility design given by Equation (4.1), and submodular 43 welfare functions. The robust price of anarchy is bounded above by { 2 min min {Wr (x + 1) W r (x) } r R x n W r (x)/x (4.26) Proof: Let x r, y r be the number of players on resource r in the Nash Equilibrium allocation (a) and the optimal allocation (a ), respectively. 1 We can write the total welfare function in the optimal allocation and the Nash equilibrium allocation as follows W (a ) = r R W r (y r ) (4.27) W (a) = W r (x r ) (4.28) r R Let z r be the number of players who are on resource r in both allocations. z r = {i N : a i = r} {i N : a i = r} (4.29) Then we can sum the unilateral deviation of each player from its Nash equilibrium allocation to its optimal allocation and write it as follows, U i (a i, a i ) = z r (W r (x r ) W r (x r 1)) + (y r z r )(W r (x r + 1) W r (x r ))) (4.30) i N r R The first term on the right-hand side of Equation (4.30) is the utility received by players who are on r in both the Nash equilibrium and the optimal allocations, and therefore when any of these players unilaterally deviates it stays put on the same resource and receives the same utility. However, the second term refers to players who deviate from r to r and thus receive a marginal contribution utility corresponding to one added player to the players on r in the Nash equilibrium allocation. Note that the word unilateral refers to a player deviating to its optimal allocation, while all other players are still selecting their Nash equilibrium allocation. Equation (4.30) is increasing in z r, and 0 z r min{x r, y r }; thus, substituting z r = 0 gives a lower bound: U i (a i, a i ) y r (W r (x r + 1) W r (x r ))) (4.31) i N r R 1 In this proof we will refer to the Nash equilibrium allocation by a instead of a ne and to the optimal allocation by a instead of a opt.

60 Define R opt to be the set of resources on which y r x r and R ne to be the set of resources on which x r > y r and thus R = R opt R ne ; Rewrite the right hand side of equation (4.31) as y r (W r (x r + 1) W r (x r ))) = r R r R opt R ne y r (W r (x r + 1) W r (x r ))) ± W (a ) ± W (a) ± Using (4.27) and (4.28), Equation (4.32) can be further expanded as y r (W r (x r + 1) W r (x r ))) = r R 44 r R opt x r (W r (x r + 1) W r (x r ) (4.32) y r (W r (x r + 1) W r (x r ))) r R opt + y r (W r (x r + 1) W r (x r ))) r R ne + W (a ) W r (y r ) W r (y r ) r R opt r R ne W (a) + W r (x r ) + W r (x r ) r R opt r R ne ± x r (W r (x r + 1) W r (x r ) r R opt which can be written as y r (W r (x r + 1) W r (x r ))) = W (a ) W (a) r R + y r (W r (x r + 1) W r (x r )) r R ne + W r (x r ) W r (y r ) r R ne + x r (W r (x r + 1) W r (x r ) r R opt (W r (y r ) W r (x r )) r R opt + By submodularity of W r, we have that r R opt, where y r x r r R opt (y r x r )(W r (x r + 1) W r (x r )) (4.33) (y r x r )(W r (x r + 1) W r (x r )) W r (y r ) W r (x r ) (4.34)

61 45 Using Inequality (4.34) in (4.33), we get, U i (a i, a i ) W (a ) W (a) i N + r R ne y r (W r (x r + 1) W r (x r )) + r R ne W r (x r ) W r (y r ) + Now, we notice that since y r < x r r R ne, the term r R opt x r (W r (x r + 1) W r (x r )) (4.35) y r (W r (x r + 1) W r (x r )) + W r (x r ) W r (y r ) is decreasing in y r for 0 y r x r 1. This can be seen by differentiating with respect to y r and noticing that the derivative is always negative, because W r(y) W r (x r + 1) W r (x r ) 0 y r x r 1 due to the submodularity (decreasing marginal increments) of W r. Therefore, substituting y r = x r 1 in (4.35) gives a lower bound on the sum of unilateral deviations and we get U i (a i, a i ) W (a ) W (a) i N + x r (W r (x r + 1) W r (x r )) r R + W r (x r ) W r (x r 1) r R ne (W r (x r + 1) W r (x r )) r R ne (4.36) By submodularity of W r, the summation over r R ne, is a positive quantity and thus we can further reduce equation (4.36) into U i (a i, a i ) W (a ) W (a) + x r (W r (x r + 1) W r (x r )) (4.37) i N r R

62 Using the smoothness argument explained earlier in Section 3.2, we will derive the robust price of anarchy. In order to write (4.37) in the form of i N U i(a i, a i) λw (a ) µw (a), we will bound the last summation as follows x r (W r (x r + 1) W r (x r )) = ( ( Wr (x r + 1) )) W r (x r ) x r 1 W r (x r ) r R r R { ( Wr (x r + 1) )} min x r 1 W r (x r ) x r W r (x r ) { = min x r r R ( Wr (x r + 1) W r (x r ) x r W r (x r ) 46 )} W (a) (4.38) Substituting (4.38) back into (4.37), we get ( { U i (a i, a i ) W (a Wr (x r + 1) W r (x r ) } ) ) W (a) 1 min (4.39) x r W r(x r) i N From Equation (4.39), the λ, µ parameters are given by, x r λ = 1 µ = { Wr (x r + 1) W r (x r ) } 1 min x r W r(x r) and therefore the robust price of anarchy is given by 1 + µ λ { = 2 min min {Wr (x + 1) W r (x) r R x n W r (x)/x x r } 4.3 Comparative Analysis As a first step to compare the Shapley value (SV) utility function to the marginal contribution (MC) utility function, we compare the bounds for SV derived in [20] to our bounds for MC derived in Section 4.2. We would like to note that comparing the bounds is not conclusive, because the bounds derived are not tight. In order to completely prove that one utility (A) performs better than the other utility (B), we would further have to show that for every class of submodular welfare functions there exists a game where utility (B) performs worse than the bound of utility (A). 2 2 We use (A) and (B) here to avoid claiming that SV is better than MC or the opposite.

63 47 We start in Section by showing that the price of anarchy bounds for SV are lower than the price of anarchy bounds for MC for a class of submodular functions, namely the polynomial functions class, x d. Furthermore, we show that there exists a game where the MC utility function is used and the price of anarchy of that game is always worse than the price of anarchy of the SV utility function, which shows that for the x d class of submodular functions, SV always performs better than MC. After that, in Section we analytically show that the robust price of anarchy bound for MC is always higher than that of SV for any class of submodular functions. However, again we note that this result does not conclude that MC utility function performs worse than SV utility functions for all submodular functions Illustrative Examples In this section, we will demonstrate some examples that compare between the price of anarchy bounds for the two utility functions, namely, the Shapley value and the marginal contribution. Consider the family of submodular functions W r = x d, where 0 d 1. Example 1 x d : Symmetric Action Sets First, we will study the symmetric case where all players have the same action set. Figure 4.1 compares the price of anarchy bound for the Shapley value utility function to the corresponding bound for the marginal contribution utility function, in the symmetric setting, derived in Theorem 4 and Theorem 7 respectively. The price of anarchy for the MC utility function is always 1 and therefore for the symmetric action sets, the MC guarantees the lowest possible price of anarchy for any class of submodular functions. Example 2 x d : Asymmetric Action Sets Secondly, we will consider the more general asymmetric case where players action sets are not necessarily identical. The welfare function of each resource belongs to the same family of submodular functions W r = x d where 0 d 1. Figure 4.2 compares between the robust price of anarchy

64 2 1.9 Symmetric case W r (x) = x d SV Thm4 MC Thm POA d Figure 4.1: The price of anarchy of the submodular functions W r (x) = x d in the symmetric setting for the Shapley value and the marginal contribution utility design. bounds for the Shapley value and the marginal contribution utility design in the asymmetric setting, derived in Theorem 5, Theorem 6 and Theorem 8 respectively. The expressions, compared in the figure, are shown in Equations (3.10), (3.11), (4.26), and (4.40) respectively Asymmetric case W r (x) = x d SV Thm5 MC Thm8 MC example SV Thm POA d Figure 4.2: The price of anarchy of the submodular functions W r (x) = x d in the asymmetric setting for the Shapley value and the marginal contribution utility design. We prove, in Section 4.3.2, that the marginal contribution bound is greater than or equal the Shapley value bounds derived by [20] for any submodular function; which is shown in the figure for the submodular function x d, for all values of 0 d 1. Also, as we see in the figure, the marginal contribution price of anarchy of Example 3 lies below the marginal contribution bound, but is higher than the Shapley value bound for all values of d.

65 49 Example 3 x d : Specific example In addition to the comparison of the bounds, Figure 4.2 shows a specific game example where the MC utility function always performs worse than SV bound for the class of submodular functions x d. In this game, there are 3 resources, {A, B, C}, and 2 players, {P 1, P 2 }. The action set of P 1 is A 1 = {A, B}, while the action set of P 2 is A 2 = {B, C}. The welfare of each resource as function of the number of players allocated to that resource is defined as follows: W A (1) = 1 d = 1, W B (1) = 1 d = 1 W B (2) = 2 d W C (1) = 2 d 1 where 0 d 1 The worst Nash equilibrium allocation and the optimal allocation can be defined as Nash Eqm P 1 B P 2 C Optimal P 1 A P 2 B which leads to a price of anarchy of P oa MC = 2 2 d = 21 d (4.40) allocation. Figure 4.3 shows the players in the optimal allocation and in the worst Nash equilibrium A B C o o x x Figure 4.3: A player is denoted by o in the optimal allocation and by x in the Nash equilibrium allocation

66 50 We would like to emphasize why in the symmetric setting this type of example is not valid. Note that, in the symmetric setting the action set of both players will include the 3 resources; i.e. A 1 = A 2 = {A, B, C}. Thus, having player P 1 on resource B and player P 2 on resource C is no longer a Nash equilibrium, since player P 2 will have a unilateral incentive to deviate from resource C to resource A and increase its utility. Therefore, leading to a price of anarchy of 1 for this example, and actually for any other example with the marginal contribution utility function. Example 4 Piecewise linear function: Asymmetric Case This example is similar to Example 3 with a different welfare function. Consider the same asymmetric setting of Example 3: 3 resources, {A, B, C}, and 2 players, {P 1, P 2 }. The action set of P 1 is A 1 = {A, B}, while the action set of P 2 is A 2 = {B, C}. The welfare of each resource as function of the number of players allocated to that resource is defined as follows: W A (1) = 1, W B (1) = 1 W B (2) = 1 + ɛ W C (1) = ɛ where 0 ɛ 1 The worst Nash equilibrium allocation and the optimal allocation can be defined as Nash Eqm P 1 B P 2 C Optimal P 1 A P 2 B which leads to a price of anarchy of P oa MC = 2 (1 + ɛ)

67 51 Figure 4.3 shows the players in the optimal allocation and in the worst Nash equilibrium allocation. For this family of submodular functions, Figure 4.4 compares between the bounds of the price of anarchy for the Shapley value and the marginal contribution utility design in the asymmetric setting. The figure also includes the price of anarchy of the marginal contribution utility function for this 2-player 3-resource game. The figure shows that for each value of ɛ, the price of anarchy in the marginal contribution lies above the PoA bound of the Shapley value utility design Asymmetric case: Piecewise Linear Function W r (x) = [1, 1 + ε, 1+2 ε] SV Thm5 SV Thm6 MC example MC Th POA ε Figure 4.4: The price of anarchy of the ɛ-family of submodular functions in the asymmetric setting for the Shapley value and the marginal contribution utility design. We prove, in Section 4.3.2, that the marginal contribution bound is greater than or equal the Shapley value bounds derived by [20] for any submodular function; which is shown in the figure for the piecewise linear family with slope ɛ, for all values of 0 ɛ 1. Also, as we see in the figure, the marginal contribution price of anarchy of Example 4 lies below the marginal contribution bound, but is higher than the Shapley value bound for all values of ɛ Analytical Proof In this section, we will compare between the price of anarchy bounds of resource allocation games when using the Shapley value utility function and when using the marginal contribution utility function. The Shapley value bounds were derived in [20] and were reviewed in Section 3.4, while the marginal contribution bounds were derived in Section 4.2. We will start with the trivial comparison of the two bounds for the symmetric action sets. After that, we will compare the robust price of anarchy in the two cases and show that for this more general setting, the Shapley value

68 52 bound is always lower than the marginal contribution bound Symmetric action sets Theorem 7 proves that for the symmetric action sets the price of anarchy of resource allocation games when using the marginal contribution (MC) utility function is always 1. The interpretation of this result is that the worst equilibrium resulting in such a game is an optimal allocation. Comparing the bound of 1 to the bound in Theorem 4 for the Shapley value utility function shows that for the symmetric action sets case the marginal contribution utility function outperforms the Shapley value utility function. Yet, the symmetric action sets is not a general case and in many cases players actions are asymmetric Asymmetric action sets In this section, we will re-derive the robust price of anarchy bounds for the resource allocation games using a more general proof technique. 4 The general proof technique works for any utility function that satisfies the conditions of valid utility games in Definition 8. The aim of this proof is to show that the robust price of anarchy bound derived for the marginal contribution utility function is always worse than that derived for the Shapley value utility function. This result suggests that it might be the case that the Shapley value utility is better in single-selection, anonymous resource allocation games. We have already demonstrated, in Section 4.3.1, that for the x d family this result is true. Theorem 9 For single-selection, anonymous resource allocation games with submodular welfare functions the robust price of anarchy (RPoA) for the Shapley value utility function, given by Equation (3.10), is lower than the robust price of anarchy for the marginal contribution utility function, given by Equation (4.26). 3 Since the robust price of anarchy bounds derived are not tight, comparing the two bounds does not conclude that the Shapley value utility function is better than marginal contribution utility function. 4 This proof technique is similar to the one introduced in Roughgarden s paper [32].

69 Proof: For a fixed set of submodular welfare functions, W, let G(W) be the set of single-selection, anonymous resource allocation games with W r W and welfare distribution rule f r (i, {a} r ). 53 Let f r (i, {a} r ) be the welfare distribution rule for player i on resource r, where {a} r is the set of players on resource r in the allocation a. Since players are anonymous, we can drop the dependence on player i and the set {a} r and define a function that is only dependent on the number of players on r. Thus, we define u r (x) as the share of a player on resource r with x players on it. G G satisfies the conditions of Definition 8 The number of players on a resource in the Nash equilibrium allocation is denoted by x and in the optimal allocation is denoted by x. Note that we omit the subscript r, and write x instead of x r, whenever it is clear from the context. Define γ(w) = inf { 1 + µ λ : (λ, µ) s.t. u r (x + 1)x λw r (x ) µw r (x))} (4.41) where λ > 0, µ 0, x 0 and x 1. Define γ(w) = + if there are no (λ, µ) pair that satisfies the constraints in (4.41) First we give the following proposition Proposition 10 γ(w) upper bounds the RPoA of games G(W) Proof: The share of player i N on resource r R is denoted by u i ; i N u i = r R x ru r (x). Let x r, x r be the number of players on resource r in a Nash equilibrium and an optimal allocation, respectively. We have, u i (x i, x i ) u r (x r + 1)x r (4.42) i N r R λw r (x r) µw r (x r ) (4.43) r R = λw (a ) µw (a) (4.44)

70 54 where Inequality (4.42) represents the sum of the unilateral deviations of players from their Nash equilibrium allocation to their optimal allocation. At most x r players deviate to resource r and the number of players on the resource after each unilateral deviation is x r + 1. Inequality (4.43) follows from (4.41) and Inequality (4.44) follows from the separable property of the system. From Inequality (4.44) it is clear that the pair (λ, µ) that satisfies (4.41) upper bounds the RPoA. Next we will solve the optimization problem in (4.41) as a linear optimization problem. We will define the halfplanes that define the problem in a 2 D space and analyze those halfplanes and compare them in the case of Shapley value and of marginal contribution. A (W, n) denotes the family of parameters (λ, µ) that satisfy the constraints in (4.41) for finite games G(W, u) with n players. Given a welfare function W W, a welfare distribution rule u and a pair (x, x ), we have, H {W,u,x,x } = {(λ, µ) : u r (x + 1)x λw r (x ) µw r (x)} (4.45) which defines the corresponding halfplane and H {W,u,x,x } denotes the corresponding line segment that satisfies the inequality with equality. For each line segment H {W,u,x,x } we can write λ in terms of µ as follows: λ = u r(x + 1)x + µw r (x) W r (x ) = u r(x + 1)x [ W r (x 1 + µw ] r(x) ) u r (x + 1)x (4.46) (4.47) Define β {W,u,x,x } is defined as β {W,u,x,x } = and is well-defined since x 1 and u r (x + 1) > 0 W r (x) u r (x + 1)x (4.48) We will refer to β {W,u,x,x } as β afterwards for simplicity. From (4.47) and (4.48), we can express 1+µ λ as follows 1 + µ λ = W r(x ) [ 1 + µ ] u r (x + 1)x 1 + βµ (4.49)

71 55 Since we are trying to solve the linear optimization problem in Equation (4.41), we are interested in minimizing the expression 1+µ λ as µ in terms of λ: µ = over the (λ, µ)-space. Equation (4.47) can be rewritten ( Wr (x ) ) λ 1 W r (x) β (4.50) Geometrically, we have lines with positive slope of Wr(x ) W r(x), intersecting the vertical (µ)-axis at ( 1 β ). Figure 4.5 shows a subset of the halfplanes that define γ(w) for x0.5 example with 2 players, and will be further explained later. 1.5 Minimizing 1+µ λ over the (λ, µ)-space µ β > 1 β < 1 µ =0 µ =1 λ = λ Figure 4.5: A subset of the halfplanes that define γ(w) for x 0.5 example with 2 players Using Equation (4.49), we make the following assertions: (i) If β = 1 then 1+µ λ is constant for every point in A (W, n) H W,u,x,x (ii) If β = 0, then x = 0 x β = 0 = x = 0 = λ u(1) W r (x ) (4.51) Without loss of generality, let u(1) = 1. 5 From the submodularity of W r (x), x W r(x ) is 5 If u(1) 1, we normalize by dividing u(x) by u(1) x n.

72 increasing in x. Therefore, all the halfplanes in the form of H W,u,0,x are redundant except 56 the one at x = 1. The unique minimizer of 1+µ λ in A (W, n) H W,u,x,x, is the one with λ = 1 and minimum value of µ. (iii) If β < 1 and x 1, the unique minimizer of 1+µ λ corresponds to the minimum value of µ and hence the minimum value of λ (since the lines are with positive slope). (iv) If β > 1, the unique minimizer of 1+µ λ corresponds to the maximum value of µ and hence the maximum value of λ (since the lines are with positive slope). Figure 4.5 shows a solid line with β > 1 and a dotted line with β < 1 (it can be deduced from the intersection with the mu axis at 1/β). The figure also shows the λ = 1 boundary that corresponds to the case of β = 0. Moreover, since (W r, u r ) satisfy the valid utility games conditions, in definition 8, we know that the game is at least (1, 1) smooth and therefore, we could further add a horizontal line at µ = 1 that defines the space of allowable values. The last part of the proof shows that for submodular welfare functions and utility functions that satisfy the conditions in definition 8, the β < 1 lines are always lower than the β > 1 lines, and therefore, γ(w) will be defined by the intersection of the uppermost β > 1 line and the line λ = 1, from assertion (iv) above. In addition to that, we will prove that the uppermost line for the marginal contribution utility function is always higher than that of the Shapley value utility function, and thus the price of anarchy of former is always greater than the latter. Lemma 11 For every line segment l 1 with x > x > 0, there exists another line l 2 with x = x that is pointwise above l 1 for λ 1, which further implies that for every line with β < 1 there exists another line with β 1 that is pointwise above it. 6 Proof: Let l 1 be a line with x 1 > x 1, therefore µ 1 = W r(x 1 ) W r (x 1 ) λ u r(x 1 + 1)x 1 W r (x 1 ) (4.52) 6 Note that this lemma is only valid for the class of games that satisfy the conditions in Definition 8.

73 57 Let l 2 be a line with x 2 = x 2 and x 2 = x 1, µ 2 = W r(x 2 ) W r (x 2 ) λ u r(x 2 + 1)x 2 W r (x 2 ) = W r(x 1 ) W r (x 1 ) λ u r(x 1 + 1)x 1 W r (x 1 ) = λ u r(x 1 + 1)x 1 W r (x 1 ) (4.53) (4.54) (4.55) Required to show that µ 1 (λ) µ 2 (λ) for all values of λ 1. W r (x 1 ) W r (x 1 ) λ u r(x 1 + 1)x 1 W r (x 1 ) ( W r(x 1 ) λ u r(x 1 + 1)x 1 W r (x 1 ) (4.56) W r (x 1 ) 1)λ u r(x 1 + 1) (x 1 x 1 ) (4.57) W r (x 1 ) λ u r(x 1 + 1)(x 1 x 1) (W r (x 1 ) W (4.58) r(x 1 ) Note that canceling W r (x 1 ) is allowed, since we are not considering x = 0. In other words, x 1 and therefore W r (x) > 0. The right hand side of Inequality (4.58) can be bounded above by W r (x 1 + 1) ( (x 1 x 1) ) x 1 (W r (x 1 ) W r(x 1 ) which is greater than 1 (by submodularity of W r (x) and since x 1 > x 1). Therefore, Inequality (4.58) holds true for all values of λ 1. Moreover, β < 1 = W r(x) x < u r (x + 1) W r(x + 1) x + 1 W r(x) = x > x (4.59) x x = x = β = = β 1 W r (x) u r (x + 1)x 1 (4.60)

74 58 Therefore, every line with β < 1 corresponds to a line with x > x > 0 which is dominated by another line where x = x and thus has a corresponding β 1. From Lemma 11 we deduce that cases where we have two intersecting lines one with β < 1 and another with β > 1 will not occur between λ = 0 and λ = 1 for submodular welfare functions and utility functions satisfying Definition 8. We thus make the following assertion: (v) The minimizer of (4.41) will occur at the intersection of a β > 1 line, which has x x, and λ = 1 line. 7 From assertions (iv) and (v), we have { 1 + µ } { 1 + µ(λ) } inf = inf λ=1 = 1 + max λ λ (µ(λ = 1, x, {x,x } x )) (4.61) From Equations (4.50) and (4.61) we get, { 1 + µ } inf λ { Wr (x ) = 1 + max x x W r (x) u r(x + 1)x } W r (x) (4.62) Substituting u r for the Shapley value utility function, we get the same expression of Theorem 5, where k = x and m = x. { 1 + µ } inf λ = 1 + max x x { Wr (x ) W r (x) W r(x + 1)x } (x + 1)(W r (x)) (4.63) Next we will show that for the marginal contribution function, the lines of x < x are dominated by those of x = x and thus we get the same expression as in Theorem 8. Lemma 12 For the marginal contribution distribution rule, every line segment l 3 with x 3 < x 3, there exists another line l 2 with x 4 = x 4 which intersects the line λ = 1 at a higher value. Proof: 7 Note that β > 1 does not imply that x x, but the result of Lemma 11 shows that we only need to consider lines where x x.

75 59 Let l 3 be a line with x 3 < x 3 µ 3 = W r(x 3 ) W r (x 3 ) λ u r(x 3 + 1)x 3 W r (x 3 ) (4.64) Let l 2 be a line with x 4 = x 4 and x 4 = x 3 µ 4 = W r(x 4 ) W r (x 4 ) λ u r(x 4 + 1)x 4 W r (x 4 ) = W r(x 3 ) W r (x 3 ) λ u r(x 3 + 1)x 3 W r (x 3 ) = λ u r(x 3 + 1)x 3 W r (x 3 ) (4.65) (4.66) (4.67) Required to show that µ 3 (λ) µ 4 (λ) at λ = 1. W r (x 3 ) W r (x 3 ) λ u r(x 3 + 1)x 3 W r (x 3 ) u r (x 3 + 1) W r (x 3 ) λ u r(x 3 + 3)x 3 W r (x 3 ) (4.68) (x 3 x 3) (1 W r(x 3 ) )λ W r (x 3 ) (4.69) = λ u r(x 3 + 1)(x 3 x 3 ) (W r (x 3 ) W r (x 3 ) (4.70) Note that canceling W r (x 3 ) is allowed, since we are not considering β = 0. In other words, x 1 and therefore W r (x) > 0. Substituting u r for the marginal contribution utility function in (4.70), we get λ (W r(x 3 + 1) W r (x 3 ))(x 3 x 3 ) (W r (x 3 ) W r (x 3 ) (4.71) At λ = 1 we have (W r (x 3 ) W r (x 3 ) (x 3 x 3 ) (W r (x 3 + 1) W r (x 3 )) (4.72) which is true by the submodularity of W r From Lemma 12 we deduce that the following assertion: (vi) The minimizer of (4.41) for a marginal contribution utility function will occur at the intersection of a β > 1 line and λ = 1 line and at x = x.

76 60 From assertions (iv) and (vi) and from (4.62), we have { 1 + µ } inf λ { Wr (x ) = 1 + max x,x W r (x) umc r (x + 1)x } x=x (4.73) W r (x) { Wr (x) = 1 + max x W r (x) (W r(x + 1) W r (x))x } (4.74) W r (x) { = 1 + max 1 (W r(x + 1) W r (x))x } (4.75) x W r (x) { (Wr (x + 1) W r (x))x } = 2 min (4.76) x W r (x) Equation (4.76) is the same expression as that of Theorem 8. Therefore, we have shown that robust price of anarchy is bounded by 1+max x x {µ(λ, x, x ) λ=1 } for submodular welfare functions and utility functions that satisfy valid utility games conditions. It remains to show that the marginal contribution bound is always greater than the Shapley value bound; it is sufficient to show that 1+µ SV (λ = 1, x, x ) 1+µ MC (λ = 1, x, x ). Substituting from Equation (4.50) 1 + µ SV (1, x, x ) 1 + µ MC (1, x, x ) (4.77) 1 + W r(x ) W r (x) usv r (x + 1)x W r (x) 1 + W r(x ) W r (x) umc r (x + 1)x W r (x) The Shapley value and the marginal contribution utilities are given by, u SV r (x + 1) = W r(x + 1) x + 1 (4.78) (4.79) u MC r (x + 1) = W r (x + 1) W r (x) (4.80) For any submodular function W r (x), the Shapley value utility function is always greater than the marginal contribution utility function. Therefore, for a fixed submodular function W r (x) and for every pair (x, x ) Inequality (4.78) holds true which implies that maximizing over values of x x will yield a lower bound for the Shapley value utility as compared to the marginal contribution utility function.

77 Worst Case Format In this section, we will derive a worst case game-format for single-selection, anonymous resource allocation games. 8 By worst case format, we mean a standard game-format that will exploit the worst possible price of anarchy for the class of games under consideration. We start by introducing some definitions, in Section Then, in Section 4.4.2, we study the worst case format when utilizing the Shapley value utility function, and in Section we study the worst case format for the marginal contribution utility function. Lastly, in Section we show that for anonymous players there is no need to consider the weighted Shapley value utility function Definitions Types of Allocations As stated earlier, an allocation is an action profile where each player selects one resource from its action set. Since we are concerned with studying game forms that exploit the worst price of anarchy, we will focus on only 2 allocations per game, namely the optimal allocation and the worst possible Nash equilibrium allocation. 9 Therefore, without loss of generality, we will consider that each player has only 2 resources in its action set, one that is selected in the optimal allocation and another that is selected in the worst Nash equilibrium allocation. Consider a game with n players and m resources, where m n. Definition 11 (Collocated Allocation) A collocated allocation is an allocation where all players are on the same resource. Resource R 1 R 2 R n R m Number of players n The work in this section was done in collaboration with Ragavendran Gopalakrishnan. 9 In general there could be more than one optimal allocation and also there could be several allocations that are pure Nash equilibrium. All optimal allocations are considered equivalent for analysis purposes and we will focus on the worst Nash equilibrium.

78 Definition 12 (Scattered Allocation) A scattered allocation is an allocation where each player is on a different resource. 62 Resource R 1 R 2 R n R m Number of players Definition 13 (Combined Allocation) A combined allocation is an allocation where some players are collocated on a single resource and the rest of the players are spread evenly on different resources. Resource R 1 R 2 R n p+1 R m Number of players p Definition 14 (Collocated-Allocation Game) A collocated-allocation game is a game in which the Nash equilibrium allocation has all n players collocated on one resource (R 1 ) and the optimal allocation has p < n players collocated on R 1 and the remaining n p players scattered on n p other resources. In other words, the Nash equilibrium is a collocated allocation and the optimal is a combined allocation. The class of all collocated-allocation games is denoted by G c. Resource R 1 R 2 R n p+1 Number of players in Nash equilibrium allocation n 0 0 Number of players in the optimal allocation p 1 1 Definition 15 (Shifted-Allocation Game) A shifted-allocation game is a game in which the optimal allocation has n players scattered on the first n resources (player i on resource r i ), and the Nash equilibrium allocation is a shifted version of the optimal allocation, where player i is on resource r i+1 instead of resource r i. The action set of player i is A i = {r i, r i+1 }. Both allocations in this case are scattered allocations. The class of all shifted-allocation games is denoted by G s. Resource R 1 R 2 R n R n+1 Number of players in Nash equilibrium allocation Number of players in the optimal allocation

79 Tails of a submodular function Recall that we are considering submodular welfare functions W r W with decreasing marginal increments. To elaborate, we have Wr(x) x decreasing in x and similarly W r (x + 1) W r (x) decreasing in x for all values of x. In order to study the worst case format for games with marginal contribution utility function, in Section 4.4.3, we will consider a larger set of submodular functions that includes a function and its tails. The tail of a submodular function W r is defined as a shifted scaled version of the original function. Let W r (x) be a discrete submodular function (i.e. x Z + ). W r (0) = 0 W r (i + 1) W r (i) = d i, from submodularity d i d i 1 ; i 1, where d 0 = W r (1) WLOG, let W r (1) = 1 (normalize the function) For a given resource k, W k is the submodular welfare function associated with that resource. W k (i) represents the welfare associated with i players on the k th resource and can be expressed as follows, i 1 W k (i) = 1 + Therefore, W k can be written as a row vector of dimension n, where n is the maximum number of players in the game. j=1 d j W k (1 : n) = [1, 1 + d 1, 1 + d 1 + d 2, 1 + d d n 1 ] (4.81) Define W j k as the jth -shifted version of W k ; start the function W j k by the difference d j, W j k (1) = W k(j + 1) W k (j) = d j then keep adding the differences in their original order such that W j k (2) = d j + d j+1

80 64 up till the n th and last difference d n 1 is reached, after that extend linearly by the last difference, d n 1 such that W j k (n) = W j k (n 1) + d n 1 The following equations define the shifted version more formally W j k (i) = W k(i + j) W k (j); 1 i n j (4.82) W j k (i) = W j k (n j) + (i (n j)) d n 1; n j + 1 i n (4.83) After shifting the function, we normalize it by dividing by the value W j k (1) = d j. Starting from Equation (4.81), shifting by j and then scaling by d j we get W k (1 : n) = [1 1 + d d 1 + d d d n 1 ] (4.84) W j k (1 : n) = [d j d j + d j+1 d j + + d n 1 d j + + jd n 1 ] (4.85) W k j (1 : n) = [1 1 + d j+1 d j d n 1 d j j d n 1 d j ] (4.86) Shapley value worst case format Let G c be a class of collocated-allocation games, according to Definition 14. Proposition 13 Given any single-selection, anonymous resource allocation game G G with Shapley value utility function given by Equation (4.2), there exists another collocated-allocation game G c G c that has a worse price of anarchy. Proof: Let x r be the number of players on resource r in the Nash Equilibrium allocation (a) and y r be the number of players on resource r in the optimal allocation (a ). We can write the total welfare function in the optimal allocation and the Nash equilibrium allocation as follows W (a ) = r R W r (y r ) (4.87) W (a) = r R W r (x r ) (4.88) = P oa = W (a ) W (a) = r R W r(y r ) r R W r(x r ) (4.89)

81 Starting from the given game G, we construct a new game G that has a worse price of anarchy. Divide the set of resources into three groups, namely R 0, R 1 and R 2, such that 65 x r = 0 and y r 1 for r R 0 x r < y r and x r 0 for r R 1 x r y r for r R 2 A player i on a resource r in the Nash equilibrium has utility function u SV i (a) = Wr(xr) x r and does not have a unilateral incentive to deviate. A player j who has r in its action set, but is on r in the Nash equilibrium allocation receives a utility u j (a) = W r(x r) x r Wr(xr+1) x r+1. Step I For each resource r 1 R 1, we do the following Define a set of players P 1 that contains y 1 x 1 players who are on r 1 in the optimal allocation and not on r 1 in the Nash equilibrium allocation. Add y 1 x 1 new resources to R 0 with welfare function W r (1) = { Wr (x r ) } min r R (r 1 ) x r W r defined as (4.90) where R (r 1 ) is defined to be the set of resources in the action sets of players in P 1. Add one of the new resources r 0 R 0 to the action set of each player in P 1. Note that this addition will not change the Nash equilibrium allocation, since the welfare function of the added resources, Wr (1), is set equal to the lowest possible Shapley value utility received by any player in P 1 in the Nash equilibrium allocation; therefore no player has a unilateral incentive to deviate to any of the new resources in the Nash equilibrium. W r (1) = { Wr (x r ) } min W r 1 (x 1 + 1) r R (r 1 ) x r x (4.91) Also by submodularity of W r and since y 1 > x 1, we have W r1 (x 1 + 1) x W r 1 (y 1 ) y 1 W r 1 (y 1 ) W r1 (x 1 ) y 1 x 1 (4.92)

82 66 Move the y 1 x 1 players in P 1 from r 1 to one of the new resources r 0 R 0 in the optimal allocation. From Equation (4.91) and (4.92) we have W r1 (y 1 ) W r1 (x 1 ) + (y 1 x 1 ) W r (1) (4.93) Equation (4.93) shows that these changes in players actions increase the optimal welfare. After repeating the set of steps above for every resource r 1 R 1, we end up with two sets of resources, R 0 and R 2 x r = 0 and y r 1 for all r R 0 x r y r for r R 2 Step II For each resource r R 0, we do the following Add y r 1 resources with welfare W r (1) to R 0 and move y r 1 players to one of the new added resources in the optimal allocation. Since x r = 0, adding extra resources with the same welfare function will not change the Nash equilibrium. By submodularity of W r, we have W r (y r ) y r W r (1) (4.94) Equation (4.94) shows that the welfare at the optimal allocation will increase. Now every resource r R 0 has x r = 0 and y r = 1 and every resource in r R 2 has x r y r. Next, we will divide the game into a group of games, each of which is in the set G c described above. Step III Since we are studying single-selection games, the number of players on all resources in an allocation sums up to the total number of players; n = r x r = r y r Thus, for every r R 2, 10 there exits x r y r resources in R 0, where there is only one 10 On r R 2 there x r players in Nash equilibrium and y r players in the optimal allocation and x r y r.

83 67 player in the optimal allocation. For every r R 2, we group x r y r resources from R 0 and consider this one subsetgame of G. Since we are dealing with anonymous players, each subset is considered an independent game on its own. Figure 4.6 shows a tabular form of the resulting game, G. The resulting game consists of a group of smaller games, { G i } 1 i l, each of which is in the class of games G c ; i.e. each one of the smaller games has x i players collocated on one resource in the Nash equilibrium and y i players on the same resource in the optimal allocation and x i y i players spread on the rest of the resources in the optimal allocation. 11 G 1 Gi Gl x i 0 y i 1 x y x l 0 0 y l 1 1 Figure 4.6: Game G is grouped into several games each of which is in the class G c As mentioned earlier, Equations (4.93) and (4.94) show that the optimal welfare of the resulting game G is greater than that of the original game G. Moreover, since the Nash equilibrium allocation has not changed then the increase in the optimal allocation welfare implies that the price of anarchy of the new resulting game has increased. That is to say, the price of anarchy of the resulting game is worse than that of the original game. The last step in the proof is to show that the price of anarchy of the G is dominated by one of its subgames G i for some i, 1 i l. Using Lemma 40 in Appendix B, we prove that the price 11 Note that x j could be equal to y j for some games G j.

84 68 of anarchy of one of the subsets is greater than the price of anarchy of the overall game G. P oa(g) P oa( G) (4.95) = r R W r(y r ) r R W (4.96) r(x r ) G = i G r R i W r (y r ) (4.97) G i G r R i W r (x r ) { r R max i W r (y r ) } (4.98) G i G r R i W r (x r ) = P oa( G m ) (4.99) where R i is the set of resources in the game G i G; R i = R and G m is the subset-game at which the max price of anarchy is attained. Since G m G c, the proof is complete. We note that the price of anarchy for a game G c G c matches the price of anarchy of the bound in Theorem 4 derived by the authors in [20]. To see this, consider a game G c G c with m players collocated on R 1 in the Nash equilibrium allocation and a subset of the players (k) collocated on R 1 in the optimal allocation. The rest m k players are spread on m k resources (each with welfare W 2 ) in the optimal allocation. From the Nash equilibrium condition we have The price of anarchy for this game is given by W 1 (m) m W 2(1) (4.100) P oa(g c ) = W (aopt ) W (a ne ) = W 1(k) + (m k)w 2 (1) W 1 (m) W 1(k) + (m k) W 1(m) m W 1 (m) = 1 + W 1(k) W 1 (m) k m (4.101) (4.102) (4.103) (4.104) where Inequality (4.103) follows from Equation (4.100). Equation (4.104) matches the bound in Equation (3.9).

85 69 On a side note, we would like to define the relationship between the pair (k, m) that are in Equation (4.104). Given a submodular function W r (m), for every 1 m n the value of k that maximizes expression (4.104) can be given by: W r (x) x = W r(m) m at x = k (4.105) Equation (4.105) states that the slope of the function at k is parallel to the slope of the line joining the origin and W r (m). For every submodular function, there exists a pair (k, m) where expression (4.104) is maximized Marginal Contribution worst case format Let G s be a class of shifted-allocation games, according to definition 15. Proposition 14 Given any single-selection, anonymous resource allocation game G G with the marginal contribution utility function, given by Equation (4.1), there exists another shiftedallocation game G s G s that has a worse price of anarchy. Proof: Let x r be the number of players on resource r in the Nash Equilibrium allocation (a) and y r be the number of players on resource r in the optimal allocation (a ). We can write the total welfare function in the optimal allocation and the Nash equilibrium allocation as follows W (a ) = r R W r (y r ) (4.106) W (a) = r R W r (x r ) (4.107) = P oa = W (a ) W (a) = r R W r(y r ) r R W r(x r ) (4.108) Starting from the given game G, we construct a new game Ḡ that has a worse price of anarchy. Divide the set of resources of G into 3 sets namely R 1, R 2 and R 3 such that x r < y r for r R 1 x r > y r for r R 2

86 70 x r = y r for r R 3 A player i on a resource r in the Nash equilibrium has utility function u MC i (a) = W r (x r ) W r (x r 1) and does not have a unilateral incentive to deviate. A player j who has r in its action set, but is on r in the Nash equilibrium allocation receives a utility u j (a) = W r(x r) W r (x r 1) W r (x r + 1) W r (x r ). Step I Create three empty sets R 10, R 11 and R 01 such that in the new game Ḡ R 10 will contain resources with one player in the optimal allocation and no players in the Nash equilibrium. R 11 will contain resources with one player in the optimal allocation and one player in the Nash equilibrium. R 01 will contain resources with no players in the optimal allocation and one player in the Nash equilibrium. Step II For each resource r 1 R 1, we do the following Add x 1 new resources to R 11 with a welfare function, Wr that starts at the x th r tail 12 of W r. W r (1) = W x 1 1 r (1) = W r (x 1 ) W r (x 1 1) (4.109) W r (2) = W x 1 1 r (2) = W r (x 1 + 1) W r (x 1 1) (4.110) Equation (4.109) expresses the welfare produced if there is 1 player on a resource in R 11, and (4.110) expresses the welfare produced if there are 2 players on it. Add y 1 x 1 resources to R 10 with a welfare function W r that starts at the the x r + 1 st tail of W r. 12 Tails of the submodular function are explained in section W r (1) = W x 1 r (1) = W r (x 1 + 1) W r (x 1 ) (4.111) W r (2) = W x 1 r (2) = W r (x 1 + 2) W r (x 1 ) (4.112)

87 71 Equation (4.111) expresses the welfare produced if there is 1 player on a resource in R 10, and (4.112) expresses the welfare produced if there are 2 players on it. Move each of the x 1 players, who were previously on r 1 R 1 in the Nash equilibrium, to one of the new resources in R 11 in the Nash equilibrium allocation and the optimal allocation of the new game Ḡ. Move the remaining y 1 x 1 players, who were on r 1 R 1 in the optimal allocation in the game G, to one of the new resources in R 10 in the optimal allocation of the new game Ḡ. To sum up, for each r 1 R 1 in the game G we construct y 1 new resources in the game Ḡ. Each one of the first x 1 resources has one player in Nash equilibrium allocation and one player in optimal allocation, while the remaining y 1 x 1 resources will have one player in optimal allocation and no players in Nash equilibrium allocation of the new game Ḡ. This conversion does not change the Nash equilibrium of the game, since players who were previously on r 1 received W r (x 1 ) W r (x 1 1) which is the same as the welfare they receive on r 11 R 11, given by Equation (4.109). Step III For each resource r 2 R 2, we do the following Add y 2 new resources to R 11 with the welfare function W r shown in Equations (4.109) and (4.110). Add x 2 y 2 resources to R 01 with the welfare function W r shown in Equations (4.109) and (4.110). Move each of the y 2 players, who were previously on r 2 R 2 in the optimal allocation, to one of the new resources in R 11 in the Nash equilibrium allocation and the optimal allocation of the new game Ḡ. Move the remaining x 2 y 2 players, who were on r 2 R 2 in the Nash equilibrium in the game G, to one of the new resources in R 01 in the Nash equilibrium allocation of

88 72 the new game Ḡ. Similarly, for each resource r R 3, add x 3 = y 3 new resources to R 11 with the welfare function W r shown in Equations (4.109) and (4.110). Step IV Now we will show that the new game Ḡ has a worse price of anarchy than the original game G. Starting from Equation (4.106) P oa(g) = r R W r(y r ) r R W r(x r ) r R = 1 + W r(y r ) W r (x r )) r R W r(x r ) = 1 + ( r R 1 W r (y r ) W r (x r ))) ( r R 2 W r (x r ) W r (y r ))) r R W r(x r ) (4.113) Note that r R 3 (W r (y r ) W r (x r )) = 0 since y r = x r r R 3. P oa(ḡ) = r R 1 x r (W r (x r ) W r (x r 1)) + (y r x r )(W r (x r + 1) W r (x r )) r R x r(w r (x r ) W r (x r 1)) r R + 2 y r (W r (x r ) W r (x r 1)) r R x r(w r (x r ) W r (x r 1)) r R + 3 x r (W r (x r ) W r (x r 1)) r R x (4.114) r(w r (x r ) W r (x r 1)) We can write Equation (4.114) in the form of 1 + and compare it to Equation (4.113) P oa(ḡ) = 1 + r R 1 (y r x r )(W r (x r + 1) W r (x r )) r R x r(w r (x r ) W r (x r 1)) r R 2 (x r y r )(W r (x r ) W r (x r 1)) r R x r(w r (x r ) W r (x r 1)) Compare Equation (4.113) to Equation (4.115) (4.115) For r R 1, we have y r > x r and by the submodularity of W r we get (y r x r )(W r (x r + 1) W r (x r )) (W r (y r ) W r (x r )) (4.116) For r R 2, we have x r > y r and by the submodularity of W r we get (x r y r )(W r (x r ) W r (x r 1)) (W r (x r ) W r (y r )) (4.117)

89 73 For all r R by the submodularity of W r we have W r (x r ) x r (W r (x r ) W r (x r 1)) (4.118) From Equations (4.116), (4.117) and (4.118) it is clear that P oa(g) P oa(ḡ). Next, we will divide the game into a group of games each of which is in the set G s described above. Step V After performing steps I, II and III to every resource in the game G, we end up with a new game Ḡ that has m + n resources where m = r R 1 y r x r = r R 2 x r y r and n is the number of players. Each player i originally had two actions in its action set and each one of the resources in G has multiple resources corresponding to it in the new game Ḡ. After the conversion to Ḡ, still every player will have two actions in its action set depending on the resources in its original actions. It is simple to see that after the conversion player i s new action set, A i = {a 1, a 2 }, will fall under one of the following 3 cases: 13 (a) a 1 R 10 and a 2 R 11 (b) a 1 R 01 and a 2 R 11 (c) a 1 R 11 and a 2 R 11 Similarly, every new resource, in any of the new sets R 10 or R 01, is in the action set of at most 1 player and any resource in the set R 11 is in the action set of at most 2 players. Since we are studying single-selection games, the number of players on all resources in an allocation sums up to the total number of players; n = r x r = r y r Thus, for every r R 01 there exits a resource in r R Note that the case a 1 R 10 and a 2 R 01 can be trivially omitted as it represents one player who is alone on a resource r 10 in the optimal and alone on a resource r 01 in the Nash and since we are dealing with marginal contribution utility function, this case will not occur unless W r10 = W r01.

90 74 Arrange the players in order, {1, 2,, n} such that: Player 1 has A 1 = {r a, r b } where r a R 10 and r b R 11 Player 2 has A 2 = {r b, r c } where r b is common with player 1 and r c R 11 Player 3 has A 3 = {r c, r d } where r c is common with player 2 and r d R 11 or r d R 01 Continue like that till you reach a player who has r R 01 in its action set. After that, look for the next player who has r R 10 in its action set. Figure 4.7 shows a tabular form of the resulting game, Ḡ. The resulting game consists R 10 R 11 R 01 Optimal allocation Ḡ 1 Nash equilibrium allocation Optimal allocation Ḡ 2 Nash equilibrium allocation G L Optimal allocation Nash equilibrium allocation Figure 4.7: Game Ḡ is grouped into several games each of which is in the class G s of a group of smaller games, {Ḡi} 1 i l, each of which is in the class of shifted-allocation games G s ; according to Definition 15. As mentioned earlier, comparing Equation (4.113) to Equation (4.115) shows that the price of anarchy of the new game Ḡ is worse than that of the original game G. The last step in the proof is to show that the price of anarchy of the Ḡ is dominated by one of its subgames Ḡi for some i, 1 i l. Using Lemma 40 in Appendix B, we prove that the price

91 75 of anarchy of one of the subsets is greater than the price of anarchy of the overall game Ḡ. P oa(g) P oa(ḡ) (4.119) = W r R r (y r ) W (4.120) r R r (x r ) Ḡ = i Ḡ r R Wr i (y r ) (4.121) Ḡ i Ḡ r R Wr i (x r ) { r R Wr max i (y r ) } (4.122) Ḡ i Ḡ r R Wr i (x r ) = P oa(ḡm) (4.123) where R i is the set of resources in the game Ḡi Ḡ; R i = R and Ḡm is the subset-game at which the max price of anarchy is attained. Since Ḡm G s, the proof is complete. Proposition 15 For a fixed submodular welfare function W r, the worst case single-selection, anonymous resource allocation game with the marginal contribution utility function, given by Equation (4.1), has a price of anarchy of where d i = W r (i + 1) W r (i) 2 min i 1 d i d i 1 Proof: Consider the set W that contains the submodular function W r and its tails. Define From Equation (4.86) argmin i 1 d = min i 1 d i d i 1 (4.124) d i d i 1 = j + 1 (4.125) W k j (1 : n) = [1, 1 + d j+1 d j, d n 1 d j, j d n 1 d j ] (4.126) = [1, 1 + d, ] (4.127) Using the result of Proposition 14, we construct a shifted-allocation game G s G s with n players and n + 1 resources.

92 76 Player i has an action set A i = {R i, R i+1 } Resources R 1 and R 2 have a welfare function W 1 = [1, 1 + d] Resource R i i 2 has a welfare function d i 2 W 1 = [d i 2, d i 1 ] Since every player is receiving the marginal contribution utility function, we have Every player i selecting resource R i+1 is a Nash equilibrium allocation. Every player i selecting resource R i is the optimal allocation. This game has a price of anarchy equal to W (a opt ) W (a ne ) = 2 + d + d2 + + d n d + d d n 1 (4.128) = d n d + d d n 1 (4.129) = dn 1 n 1 i=0 di (4.130) = 1 + (1 dn 1 )(1 d) 1 d n (4.131) Taking the limit as the number of players goes to infinity lim 1 + (1 dn 1 )(1 d) n 1 d n = 2 d (4.132) d = min i 1 We note that Equation (4.132) approaches the bound of 2 derived by [40] as d 0. Since d i d i 1, then if d i = 0 for some i N, then this example would have a price of anarchy of 2 which is an upper bound on the price of anarchy of valid utility games Weighted Shapley value In this section, we prove that for single-selection, anonymous resource allocation games the weighted Shapley value produces a worse price of anarchy than the Shapley value. Therefore, it

93 77 is never beneficial to give players different weights if the welfare function is anonymous. For an anonymous, single-selection resource allocation game the weighted Shapley value utility given in Equation (2.11) can be reduced to U WSV i (a i = r, a i ) = w i ({a} r ) j {a} r w j ({a} r ) (W r( a r )) (4.133) where {a} r = {i N : a i = r}, w i ({a} r ) > 0 is the weight of player i when the set {a} r is on r, and a r = {i N : a i = r}. Proposition 16 Given a single-selection, anonymous resource allocation game with submodular welfare function, the weighted Shapley value utility function, given by Equation (4.133), has a worse price of anarchy than the Shapley value with equal weights, given by Equation (4.2). Proof: Using the result of Proposition 13, we start with a collocated-allocation game G c G c with x players and x y + 1 resources, where x > y are deduced according to Equations (4.104) and (4.105). There are x players on R 0 in the Nash equilibrium allocation There are y players on R 0 in the optimal allocation and x y players on R 1 R x y in the optimal allocation Resource R 0 has a welfare of W 0 All other resources have a scaled version of W 0 equal to W 0(x) x The price of anarchy in this case is equal to P oa SV = W 0(y) + (x y) W 0(x) x W 0 (x) (4.134) From the optimal allocation condition we have W 0 (y) + (x y) W 0(x) x W 0 (y + 1) + (x y 1) W 0(x) x (4.135)

94 78 and which implies W 0 (y) + (x y) W 0(x) x W 0 (y 1) + (x y + 1) W 0(x) x (4.136) W 0 (y + 1) W 0 (y) W 0(x) x W 0 (y) W 0 (y 1) W 0(y) y (4.137) Now we will construct another game with the weighted Shapley value utility function that has the same Nash equilibrium allocation and has a better optimal allocation; and thus a worse price of anarchy than that of Shapley value utility function. Arrange players in increasing order of their utilities such that u 1 < u 2 < < u x Since weighted Shapley value is budget-balanced i u i(x) = W 0 (x) Thus, the first y players have u i (y) W 0(y) y ; 1 i y and the remaining x y players have u i (x y) W 0(x) x y + 1 i x (4.138) Resource R 0 has a welfare of W 0 Resource R i for i 1 has a scaled welfare function equal to u y+i W 0 The action set of the first y players is {R 0 } The action set of player i, for i y + 1, is {R 0, R i } The Nash equilibrium remains the same; all x players on R 0 From Equation (4.137), the optimal allocation also stays the same

95 79 The new price of anarchy is now equal to P oa WSV = W 0(y) + x i=y+1 u i W 0 (x) W 0(y) + (x y) W 0(x) x W 0 (x) = P oa SV (4.139) where the last inequality follows from (4.138). 4.5 Summary In this chapter, we compared the performance of the marginal contribution utility function to that of the Shapley value in anonymous, single-selection resource allocation games. We made the following contributions: We derived price of anarchy (PoA) bounds using the marginal contribution utility function; we consider 2 cases: the symmetric action sets and the asymmetric action sets of players. We analytically proved that the marginal contribution PoA bounds in the asymmetric setting is always above (worse) than the Shapley value PoA bounds, while in the symmetric setting the marginal contribution equilibrium is always guaranteed to be optimal and thus outperforms the Shapley value bounds. We supported our analytical proof by an example, where the welfare function is a polynomial in the form of x d. It is worth noting that proving one bound is always worse than another is not conclusive that the performance is likewise always worse, unless both bounds are tight, which is not the case in the asymmetric setting. Therefore, it remains an open question whether the marginal contribution utility bound always performs worse than Shapley value utility or not. We derived worst case format for all anonymous, single-selection resource allocation games that use the Shapley value distribution rule. We also showed that the price of anarchy of this class of examples matches the bound for the symmetric setting derived in [20].

96 We also derived another worst case format for all anonymous, single-selection resource allocation games that use the marginal contribution distribution rule. 80 We formally showed that there is no need to consider weights for Shapley value utility function in anonymous, single-selection resource allocation games.

97 Chapter 5 Small-Scale Study on Utility functions that minimize the Price of Anarchy for Resource Allocation Games In this chapter, we perform a small-scale study on utility functions that guarantee the lowest price of anarchy for resource allocation games. We focus on n-player games for n = 2 and n = 3. We are studying the same anonymous, single-selection resource allocation games that were introduced in Section 2.4. We consider discrete submodular welfare functions with a finite number of players. Let W 0 (x) be a discrete submodular function (i.e. x Z + ) such that: W 0 (0) = 0 W 0 (1) = 1 (normalized function) W 0 (i + 1) W 0 (i) = d i, from submodularity d i d i 1 ; i 1, where d 0 = 1 For a given resource k, W k is a scaled version of W 0, associated with the k th resource. W k (x) = v k W 0 (x) represents the welfare associated with x players on the k th resource, where v k is a scaling factor. Therefore, W k can be written as a row vector of dimension n, where n is the maximum number of players in the game. W k (1 : n) = v k [1, 1 + d 1, 1 + d 1 + d 2, 1 + d d n 1 ] (5.1)

98 82 As explained earlier in Section 4.4.1, to study the price of anarchy of an example, we need to focus on only 2 allocations per game, specifically the optimal allocation and the worst possible Nash equilibrium allocation. Therefore, without loss of generality, we consider that each player has only 2 resources in its action set: one that is selected in the optimal allocation and another that is selected in the worst Nash equilibrium allocation. Recall from Section 2.4 that the general welfare distribution rule f r (i, {a} r ) is the share of player i on resource r and is dependent on the set {a} r. For anonymous games, we remove the dependance on i and {a} r and thus the distribution rule is only a function of the number of players on a resource, written as U r (x) where x is the number of players on resource r; f r (i, {a} r ) = U r ( a r ). Furthermore, we define U r (x) = v r U(x), where U(x) is the base utility function and is designed as a distribution rule of the base welfare function W 0 (x). In this chapter, we study the effect of varying U(x) for x {2, 3} on the price of anarchy and price of stability of 2-player and 3-player games. We set U(1) = 1 and vary U(2) over all possible values in R + and study the effect on the family of 2-player anonymous, single-selection resource allocation games, with submodular welfare functions. Similarly, for 3-player games, we set U(1) = 1 and we set U(2) to the Shapley value or the marginal contribution and study the effect of varying U(3) over all values in R + on the price of anarchy of the corresponding 3-player games. We also study briefly, the effect of varying U(2) and U(3) simultaneously on 3-player games. Summary of Contributions in this chapter: We start, in Definition 16, by defining a specific class of anonymous, single-selection resource allocation games with 2 players. This specific class has 3 resources and 2 players where each player can select one of two resources and the 2 players have one resource in common in their action sets. In Claim 17 we show that this specific class of resource allocation games is the only class that exploits the worst price of anarchy for anonymous, single-selection 2-player games. Thus, we can represent the game in a 2-player 2-action matrix form.

99 Through a series of reductions, Claim 18 reduces all possible cases into 4 cases, shown in Figure Analyzing the 4 cases thoroughly in Proposition 19, we derive an expression for the price of anarchy of all 2-player anonymous, single-selection games as a function of U(2) and W 0 (2) { 1 + U(2) P oa(u(2), W 0 (2)) = max W 0 (2), U(2), 2U(2) } W 0 (2) (5.2) where W 0 (2) = (1 + d 1 ) is the base submodular welfare function value with 2 players on a resource. The variable U(2) represents the share of one player when there are 2 players on a resource and is a design parameter in this setting, and U(1) = 1 (normalized). Furthermore, in Proposition 20, we derive an expression for u that minimizes the expression in (5.2) and thus minimizes the price of anarchy for 2-player games. We show that u falls between the Shapley value share ( 1+d 1 2 ) and the marginal contribution share (d 1 ) in 2-player games. We also show in Lemma 21 that the price of anarchy increases for shares less than marginal contribution share or greater than Shapley value share; i.e. U(2) > 1+d 1 2 or U(2) < d 1. In Lemma 22, we show that for all 2-player games the Shapley value utility function guarantees a lower price of anarchy than the marginal contribution utility function. Similarly, Proposition 23 analyzes the price of stability for the 4 cases of 2-player games. Proposition 26 shows that the marginal contribution utility uniquely minimizes the price of stability in 2-player games. Figure 5.6 shows the relationship between the price of anarchy and price of stability in 2-player anonymous, single-selection resource allocation games for a given W 0. A similar analysis is done for 3-player games and we reduce the cases to 6 limiting cases. The details of the game reduction are in Appendix C.

100 Analyzing the 6 cases thoroughly in Claim 27, we derive an expression for the price of anarchy of all 3-player anonymous, single-selection games as a function of U(2), U(3), W 0 (2) and W 0 (3): P oa(u(2), U(3), W 0 ) = { 3U(3) max W 0 (3), 1 + 2U(3) W 0 (3), W 0(2) + U(3) W 0 (3), W 0(2) + U(2) 1 + U(2) + U(3), W 0 (2) + U(3) W 0 (2) + U(3), U(2) } U(2) 2 + U(2) + 1 where W 0 (2) = 1 + d 1 and W 0 (3) = 1 + d 1 + d 2 are the values of the base submodular welfare function with 2 and 3 players on a resource. U(2) and U(3) are design parameters; U(2) is the share of one player when there are 2 players on a resource, U(3) is the share of one player when there are 3 players on a resource. Figures 5.8 to 5.11 show the price of anarchy for some 3-player games. Figure 5.12 shows a 3-D plot for the price of anarchy as a function of U(2) and U(3) for a given W player Resource Allocation Games Game Definition Definition 16 Given a fixed submodular function W 0, define a class of 2-player, anonymous, single-selection resource allocation games as G 2 = {N, R, {A i } {i N }, W 0, {v r } {r R}, U} where N = 2, R = 3 and A 1 = {R 1, R 2 } and A 2 = {R 2, R 3 }. W 0 = [1, 1 + d 1 ] is the given base submodular welfare function, v r is the scaling factor for the welfare of resource r R, and U is a distribution rule and is a function of d 1. G 2 can be further detailed as follows:

101 Player P 1 R 1 R 2 R 3 Player P 2 85 Figure 5.1: Action Set of Players: A 1 = {R 1, R 2 } and A 2 = {R 2, R 3 } Set of players N = {P 1, P 2 } Set of Resources R = {R 1, R 2, R 3 } Action Sets of players: A 1 = {R 1, R 2 } (5.3) A 2 = {R 2, R 3 } (5.4) Submodular welfare function W 0 : W 0 (1) = 1 (5.5) W 0 (2) = 1 + d 1 (5.6) where 0 d 1 1 A welfare function per resource that is a scaled version of W 0 (x). For x = {1, 2} and i = {1, 2, 3} we have W i (x) = v i W 0 (x). A utility function U(x) that is defined as a distribution rule of the base welfare function. U k (x) defines the share of a player on resource k when there are x players on the resource. U k (1) = v k U(1) = v k (5.7) U k (2) = v k U(2) (5.8) Ideally, we would like to define U(2) as a distribution rule of the base welfare function W 0 and thus, for 2 players, as a function of the first difference d 1 ; U(2) = f(d 1 )

102 86 Claim 17 We claim that the class of games specified in Definition 16 is the most general setting for 2-player, anonymous, single-selection resource allocation games, required for a worst case analysis. In other words, having 3 resources and specifying the action sets of the players to have only 1 resource in common covers all 2-player games that exploit a price of anarchy greater than 1. Proof: The maximum number of resources required for 2 players with 2 actions per player is 4 resources. 1 Having 4 resources that are fully utilized in Nash equilibrium allocation and optimal allocation implies that each player is alone on a resource in the Nash equilibrium and in the optimal allocation, which further implies that the price of anarchy is equal to 1. Assume, without loss of generality, that player P 1 selects resource R 1 in the optimal allocation and resource R 3 in the Nash equilibrium allocation and player P 2 selects resource R 2 in the optimal allocation and resource R 4 in the Nash equilibrium allocation. Then we have, P oa = W 1(1) + W 2 (1) W 3 (1) + W 4 (1) (5.9) From the Nash equilibrium conditions we have W 3 (1) W 1 (1) and W 4 (1) W 2 (1) and from the optimal condition we have W 1 (1) + W 2 (1) W 3 (1) + W 4 (1). Therefore, W 1 (1) = W 3 (1) and W 2 (1) = W 4 (1). Substituting back in Equation (5.9) we get that the P oa = 1 in this case. Having 2 resources implies one of the following cases: 2 players together on R 1 in optimal and together on R 2 in Nash equilibrium. This game can be expressed in the form of G 2, by setting v 3 = v 1 and moving one of the players in the optimal allocation to R 3 instead of R 1. After the conversion the new game has a worse price of anarchy, since W 0 is submodular. 1 Note that in single-selection resource allocation games every player can select only 1 resource at a time.

103 87 2 players together on a resource in the optimal allocation and separated on the 2 different resources in the Nash equilibrium allocation or vice versa. This game can be expressed in the form of G 2, by setting v 3 = 0 2 players are on R 1 and R 2 in the optimal allocation and swap in the Nash equilibrium allocation. This game will have a price of anarchy of 1, since players are anonymous. Having 3 resources, asymmetric action sets are more general than symmetric action sets. Therefore, the case of having 3 resources with only one common resource between the action sets of the players is the most general setting in the 2-player case Game Analysis Claim 18 We claim that for the class of games specified in Definition 16, only 4 different game forms, illustrated in Figure 5.2, are required for a worst case analysis. By game forms we mean 4 different cases of Nash equilibrium allocation and optimal allocation combination and by worst case analysis we mean price of anarchy and price of stability analysis. Case A Case B Case C Case D R 1 R 2 R 3 R 1 R 2 R 3 R 1 R 2 R 3 R 1 R 2 R 3 o o o o o o oo xx x x xx x x Figure 5.2: The figure shows the 3 resources and the 2 players in each case. A player in optimal allocation is denoted by o and in the NE by x Proof: Since we are dealing with 2-player, 2-action games, we could represent the game in matrix form as outlined in Chapter 2. Figure 5.3a shows the matrix representation of the game and Figure 5.3b shows the corresponding welfare received. As shown in the figures, there are 4 different allocations, numbered from 1 to 4 as shown by Figure 5.3c.

104 R 2 R 3 R 1 v 1, v 2 v 1, v 3 R 2 v 2 U(2), v 2, v 3 v 2 U(2) (a) A General 2-player Resource Allocation Game R 2 R 3 R 1 v 1 + v 2 v 1 + v 3 R 2 v 2 (1 + d 1 ) v 2 + v 3 (b) Welfare Function for each allocation R 2 R 3 R R (c) Naming convention of allocations 88 For example, the first allocation, on one hand, represents the case when player P 1 selects resource R 1 and player P 2 selects resource R 2. The utility received by player P 1 when alone on resource R 1 is v 1 and similarly for P 2 is v 2. The welfare in this case is v 1 W 0 (1)+v 2 W 0 (1) = v 1 +v 2. The third allocation, on the other hand, represents the case when both players are on resource R 2 and thus each of them takes a share of v 2 U(2) and the welfare is v 2 W 0 (2) = v 2 (1 + d 1 ). In order to have a price of anarchy (PoA) that is strictly greater than 1, the optimal and the Nash equilibrium allocation should be different. There are 12 different ways to choose 2 allocations from 4 allocations where the order is important. The 12 cases are listed in Table 5.1. ( ) 4 4 P 2 = 2! = 2 4! (4 2)! = 4! 2! = 12 OPT NE OPT NE OPT NE OPT NE Table 5.1: 12 possible cases for Optimal (OPT) and Nash Equilibrium (NE) allocations in G 2. The numbers in the table correspond to the naming convention of the allocations given by Figure 5.3c. Allocation 4 is equivalent to allocation 1 if v 1 and v 3 are swapped; for example if allocation 1 is the optimal allocation and 3 is the Nash equilibrium allocation, this case is equivalent to the case where allocation 4 is the optimal allocation and 3 is the Nash equilibrium allocation. Therefore we can eliminate 5 cases as follows and reduce the 12 cases to 7 cases as shown in Table 5.2.

105 OPT NE OPT NE OPT NE OPT NE Table 5.2: Reduction to 7 cases. The numbers in the table correspond to the naming convention of the allocations given by Figure 5.3c. Moreover, cases where there is one player alone on a resource in the optimal allocation and alone on a different resource in the Nash equilibrium allocation (and the other player is fixed on the third resource) can be eliminated. We claim that for these cases the price of anarchy is 1. To explain this elimination, consider the case where allocation 1 is the optimal allocation and allocation 2 is the Nash equilibrium allocation. Player P 1 is fixed on R 1 in both allocations and player P 2 unilaterally deviates from R 2 to R 3. From the optimal condition, we have v 1 + v 2 v 1 + v 3 = v 2 v 3 (5.10) and from the Nash equilibrium condition we have, v 3 v 2 (5.11) Equations (5.10) and (5.11) imply that v 2 = v 3 (5.12) = P oa = v 1 + v 2 v 1 + v 3 = 1 (5.13) Therefore we can eliminate 2 more cases and reduce the 7 cases to 5 cases as shown in Table 5.3. OPT NE OPT NE OPT NE Table 5.3: Reduction to 5 cases. The numbers in the table correspond to the naming convention of the allocations given by Figure 5.3c.

106 90 Lastly, we can omit one more case that contradicts the submodularity of W 0. Consider the case where allocation 3 is the optimal allocation and allocation 2 is the Nash equilibrium allocation. From the optimal condition and since we are looking for P oa > 1, we have v 2 (1 + d 1 ) > v 1 + v 3 (5.14) and from the Nash equilibrium condition we have, v 1 v 2 (5.15) v 3 v 2 (5.16) = v 1 + v 3 2v 2 (5.17) Equations (5.14) and (5.17) represent a contradiction, since d 1 1. Therefore we can eliminate 1 more case and reduce the 5 cases to 4 cases as shown in Table 5.4. OPT NE OPT NE OPT NE Table 5.4: Final Reduction to 4 cases. The numbers in the table correspond to the naming convention of the allocations given by Figure 5.3c. Figure 5.2 shows an illustration of the 4 cases and Table 5.5 summarizes them. Case Optimal Allocation NE allocation A 1 : {R 1, R 2 } 3 : {R 2, R 2 } B 1 : {R 1, R 2 } 4 : {R 2, R 3 } C 2 : {R 1, R 3 } 3 : {R 2, R 2 } D 3 : {R 2, R 2 } 1 : {R 1, R 2 } Table 5.5: Summary of 4 main cases for 2-player games Next, we will analyze the 4 cases given by Table 5.5 and derive the price of anarchy and price of stability of the class of games G 2 as a function of the distribution rule and the base welfare function.

107 Price of Anarchy Analysis for 2-player games Proposition 19 For a fixed submodular function W 0, let G 2 = {N, R, {A i } {i N }, W 0, {v r } {r R}, U} be the set of all 2-player, anonymous, single-selection resource allocation games with base welfare function W 0, as outlined in Definition 16. The price of anarchy of G 2 as a function of the distribution rule U and the base welfare function W 0 can be expressed as { 1 + U(2) P oa(u(2), W 0 ) = max W 0 (2), U(2), 2U(2) } W 0 (2) (5.18) where W 0 (2) = 1 + d 1 is the base submodular welfare function value with 2 players on a resource. U(2) represents the share of one player when there are 2 players on a resource, and it is desired to design U(2) such that the price of anarchy is minimized. Proof: For each case in Table 5.5, we will use the Nash equilibrium condition to derive the worst possible price of anarchy. For illustration, we represent the 3 resources and the 2 players in each case; a player in the Nash equilibrium is denoted by an x and in the optimal allocation by an o. R 1 R 2 R 3 Case - A- o o xx The price of anarchy for the game instance in case A can be given by P oa A = W 1(1) + W 2 (1) W 2 (2) = v 1 + v 2 v 2 W 0 (2) = v 1 + v 2 v 2 (1 + d 1 ) (5.19) From the Nash equilibrium condition, player P 1 does not have a unilateral incentive to deviate from R 2 to R 1 U 2 (2) U 1 (1) (5.20) v 2 U(2) v 1 U(1) (5.21) v 2 U(2) v 1 (5.22)

108 92 Substituting from (5.22) in (5.19) we get P oa A = v 1 + v 2 v 2 (1 + d 1 ) v 2U(2) + v 2 v 2 (1 + d 1 ) (1 + U(2)) = (1 + d 1 ) (5.23) (5.24) (5.25) Equation (5.25) is valid for U(2) > d 1 (to have P oa > 1) R 1 R 2 R 3 Case -B- o o x x The price of anarchy for the game instance in case B can be given by P oa B = W 1(1) + W 2 (1) W 2 (1) + W 3 (1) = v 1 + v 2 v 2 + v 3 (5.26) From the Nash equilibrium condition, player P 1 does not have a unilateral incentive to deviate from R 2 to R 1 and player P 2 does not have a unilateral incentive to deviate from R 3 to R 2. U 2 (1) U 1 (1) = v 2 v 1 (5.27) U 3 (1) U 2 (2) = v 3 v 2 U(2) (5.28) Substituting from (5.27) and (5.28) in (5.26) we get Equation (5.31) is valid for U(2) < 1 (to have P oa > 1) P oa B = v 1 + v 2 v 2 + v 3 (5.29) v 2 + v 2 v 2 + v 2 U(2) (5.30) = 2 (1 + U(2)) (5.31) R 1 R 2 R 3 Case -C- o o xx

109 93 The price of anarchy for the game instance in case C can be given by P oa C = W 1(1) + W 3 (1) W 2 (2) = v 1 + v 3 v 2 (1 + d 1 ) (5.32) From the Nash equilibrium condition, player P 1 does not have a unilateral incentive to deviate from R 2 to R 1 and player P 2 does not have a unilateral incentive to deviate from R 2 to R 3. U 2 (2) U 1 (1) = v 2 U(2) v 1 (5.33) U 2 (2) U 3 (1) = v 2 U(2) v 3 (5.34) = 2v 2 U(2) v 1 + v 3 (5.35) Substituting from (5.35) in (5.32) we get P oa C = v 1 + v 3 v 2 (1 + d 1 ) 2v 2U(2) v 2 (1 + d 1 ) = 2U(2) (1 + d 1 ) (5.36) (5.37) (5.38) Note that this example is not valid if U(2) < 1, because the value of the welfare at the optimal allocation is greater than the welfare in any other allocation. 2 and thus, v 1 + v 3 > v 2 + v 3 = v 1 > v 2 (5.39) v 1 + v 3 > v 1 + v 2 = v 3 > v 2 (5.40) If U(2) < 1, Equation (5.39) and (5.40) contradict Equations (5.33) and (5.34) respectively. R 1 R 2 R 3 Case - D- oo x x 2 The value of the welfare for each allocation is shown in Figure 5.3b.

110 94 The price of anarchy for the game instance in case D can be given by P oa D = W 2 (2) W 1 (1) + W 2 (1) = v 2W 0 (2) v 1 + v 2 = v 2(1 + d 1 ) v 1 + v 2 (5.41) From the Nash equilibrium condition, player P 1 does not have a unilateral incentive to deviate from R 1 to R 2 U 1 (1) U 2 (2) (5.42) v 1 U(1) v 2 U(2) (5.43) v 1 v 2 U(2) (5.44) Substituting from (5.44) in (5.41) we get Equation (5.47) is valid for U(2) < d 1 (to have P oa > 1) P oa D = v 2(1 + d 1 ) v 1 + v 2 (5.45) v 2(1 + d 1 ) (5.46) v 2 U(2) + v 2 (1 + d 1 ) = (5.47) (1 + U(2)) Moreover, from submodularity of W 0, we have 1 + d 1 2 (5.48) = P oa D P oa B (5.49) which implies that case D can be omitted, since it is upper bounded by case B. Therefore, the price of anarchy of G 2 is given by max {P oa A, P oa B, P oa C } (5.50) = P oa(u(2), W 0 ) = max { 1 + U(2) W 0 (2), U(2), 2U(2) } W 0 (2) (5.51) (5.52)

111 95 Proposition 20 For a fixed submodular function W 0, let G 2 = {N, R, {A i } {i N }, W 0, {v r } {r R}, U} be the set of all 2-player, anonymous, single-selection resource allocation games with base welfare function W 0. The utility function that minimizes the price of anarchy of G 2 is given by u = (2(W 0 (2)) 1 where W 0 (2) = 1 + d 1 is the base submodular welfare function value with 2 players on a resource. Proof: Analyzing the price of anarchy expression in (5.51), we will prove the following: (1) P oa A P oa C for U(2) 1. Since for U(2) 1 we have, P oa A = 1 + U(2) W 0 (2) 2U(2) W 0 (2) (5.53) (5.54) = P oa C (5.55) Recall that P oa C is only valid when U(2) > 1. (2) P oa A = P oa B at U(2) = u = (2(W 0 (2)) 1 P oa A = P oa B (5.56) = 1 + U(2) W 0 (2) = U(2) (5.57) = (1 + U(2)) 2 = 2W 0 (2) (5.58) = (1 + U(2)) = 2W 0 (2) (5.59) = U(2) = 2W 0 (2) 1 = u (5.60) (3) P oa A P oa B for 0 U(2) u

112 96 P oa A is only valid for U(2) > d 1, P oa B is only valid for U(2) < 1, and P oa A P oa B (5.61) = 1 + U(2) W 0 (2) U(2) (5.62) = (1 + U(2)) 2 2W 0 (2) (5.63) = U(2) 2W 0 (2) 1 = u (5.64) (4) P oa A P oa B for u U(2) 1 Similarly, P oa A P oa B (5.65) = 1 + U(2) W 0 (2) U(2) (5.66) = U(2) 2W 0 (2) 1 = u (5.67) (5) It remains to show that P oa B is decreasing in U(2) and P oa A and P oa C are both increasing in U(2): P oa B = 2 1+U(2) is a decreasing function in U(2) P oa A = 1+U(2) W 0 (2) is an increasing linear function in U(2) P oa C = 2U(2) W 0 (2) is an increasing linear function in U(2), with a higher slope than P oa A. Therefore, given a fixed submodular function W 0, any 2-player, anonymous, single-selection resource allocation game will have a worst case price of anarchy that starts at U(2) = 0 with value of P oa B U(2)=0 = 2 and decreases until U(2) = u = 2W 0 (2) 1 and has a value of 2W0 (2) W 0 (2) = 2 W 0 (2). After that, the price of anarchy starts to increase again and keeps increasing for all values of U(2) > u. Section represents a numerical example that illustrates the results of Proposition 20.

113 Lemma 21 Given a submodular welfare function W 0, let G 2 be the set of all 2-player, anonymous, single-selection resource allocation games with base welfare function W 0. For all G G 2, setting the utility function less than marginal contribution (U(2) < d 1 ) or greater than Shapley value (U(2) > 1+d 1 2 ) worsens the price of anarchy. Proof: The proof of this lemma follows directly from the proof of Proposition 20. Since we proved that the price of anarchy decreases as U(2) increases till it reaches u and then increases again, we just need to show that d 1 < u and 1+d 1 2 > u for all d 1 < 1 d 1 < u (5.68) = d 1 < 2(1 + d 1 ) 1 (5.69) = 1 + d 1 < 2(1 + d 1 ) (5.70) = (1 + d 1 ) 2 < 2(1 + d 1 ) (5.71) = (1 + d 1 ) < 2 (5.72) = d 1 < 1 (5.73) 97 and 1 + d 1 2 = 1 + d 1 2 > u (5.74) > 2(1 + d 1 ) 1 (5.75) = 1 + d > 2(1 + d 1 ) (5.76) 2 ( ) d1 = > 2(1 + d 1 ) (5.77) 2 = 9 + 6d 1 + d > 2(1 + d 1 ) (5.78) = 9 + 6d 1 + d 2 1 > 8(1 + d 1 ) (5.79) = 1 2d 1 + d 2 1 > 0 (5.80) = (1 d 1 ) 2 > 0 (5.81) Inequality (5.81) is always true. For d 1 = 1, we have U MC (2) = u = U SV (2) = 1.

114 Lemma 22 Given a submodular welfare function W 0, let G 2 be the set of all 2-player, anonymous, single-selection resource allocation games with base submodular welfare function W 0. For all G G 2, setting the utility function equal to the Shapley value U(2) = U SV (2) = 1+d 1 2 results in a better price of anarchy than setting the utility function equal to the marginal contribution (U(2) = U MC (2) = d 1 ). 98 Proof: From Lemma 21, we proved that d 1 u 1+d 1 2 for all 0 d 1 1 and from Proposition 20, we showed that P oa B expression dominates for U(2) u and P oa A expression dominates for u U(2) 1. Therefore, the price of anarchy for Shapley value utility function will correspond to P oa A and that of marginal contribution will correspond to P oa B. We will show that P oa(d 1, W 0 ) P oa( 1+d 1 2, W 0 ) where P oa(u(2), W 0 ) is given by Equation (5.18). 1 + U(2) 1 + d U(2)=U SV (2)= 1+d (1 + d 1 )/(2) 1 + d d U(2) U(2)=U MC (2)=d 1 (5.82) d 1 (5.83) 1 (5.84) = d 1 1 (5.85) which is true by submodularity of W 0. Next, we will give a numerical example to illustrate the results of this section Numerical Example The price of anarchy, in Equation (5.51), can be expressed as a function of U(2) and d 1 as follows, P oa(u(2), d 1 ) = max { 1 + U(2) (1 + d 1 ), 2 (1 + U(2)), 2U(2) } (1 + d 1 ) (5.86) Figure 5.4 shows the price of anarchy as a function of U(2), for different values of d 1. As explained earlier, the maximum price of anarchy corresponds to P oa B from 0 U(2) u and then it corresponds to P oa A from u U(2) 1 and after that it corresponds to P oa C from

115 Change in POA as U(2) changes d 1 = 0.1 Change in POA as U(2) changes d 1 = Price of Anarchy POA U MC (2) U * (2) U SV (2) Price of Anarchy POA U MC (2) U * (2) U SV (2) U(2) Utility of a player when sharing the resource with another player (a) d 1 = U(2) Utility of a player when sharing the resource with another player (b) d 1 = 0.5 Figure 5.4: Price of Anarchy as U(2) changes U(2) 1. As shown in Figure 5.4, the price of anarchy decreases from U(2) = 0 till it reaches a minimum value of ( 2/W 0 (2)) at U(2) = u. After that, it increases with a slope of 1 and at U(2) > 1, the price of anarchy increases with a higher slope of 2. In Figure 5.4, it is clear that the price of anarchy associated with the Shapley value utility is less than that associated with the marginal contribution utility. It is also clear that the value of U is close to the value of the Shapley value U SV in all cases. Figure 5.5 shows the difference and the ratio between U SV and U for all possible values of d 1. The figure shows that the maximum difference between the 2 utilities occurs at d 1 = 0 and is bounded by The maximum ratio is 1.2 which means that the Shapley value is at most 20% higher than u. The two utilities are almost equivalent for d Price of Stability Analysis for 2-player games Proposition 23 For a fixed submodular function W 0, let G 2 = {N, R, {A i } {i N }, W 0, {v r } {r R}, U} be the set of all 2-player, anonymous, single-selection resource allocation games with base submodular welfare function W 0. The price of stability of G 2 as a function of the distribution rule U(2)

116 0.1 Difference between the Shapley value utility and the utility that minimizes the PoA for 2 players 100 U SV U * U SV U d 1 Ratio between the Shapley value utility and the utility that minimizes the PoA for 2 players d 1 Figure 5.5: Comparing the Shapley value utility function to the utility that minimizes the price of anarchy for 2-player games. and the base welfare function W 0 is given by { 1 + U(2) P os(u(2), W 0 ) = max W 0 (2), W 0 (2) } 1 + U(2) (5.87) where W 0 (2) = 1 + d 1 is the base submodular welfare function value when there are 2 players on the resource. U(2) is the share of one player when there are 2 players on a resource and it is a design parameter. Proof: For each case in Table 5.5, we will use the Nash equilibrium condition to derive W 0(a opt ) W 0 (a ne ). Since we are studying specific game instances, we need to further prove that the given Nash equilibrium allocation is the best equilibrium for each specific instance, in order to consider the ratio W 0 (a opt ) W 0 (a ne ) for the price of stability analysis. Recall that the price of stability, given by Equation (2.7), is the worst possible ratio of the optimal to the best equilibrium. R 1 R 2 R 3 Case - A- o o xx

117 101 The W 0(a opt ) W 0 (a ne ) ratio for the game instance in case A can be given by W 0 (a opt ) W 0 (a ne ) = W 1(1) + W 2 (1) = v 1 + v 2 W 2 (2) v 2 W 0 (2) = v 1 + v 2 v 2 (1 + d 1 ) (5.88) From the Nash equilibrium condition, player P 1 does not have a unilateral incentive to deviate from R 2 to R 1 U 2 (2) U 1 (1) (5.89) v 2 U(2) v 1 U(1) (5.90) v 2 U(2) v 1 (5.91) Setting v 2 U(2) > v 1, v 2 U(2) > v 3 and v 1 > v 2 d 1 ensures that this equilibrium allocation (both players on R 2 ) is the unique equilibrium and thus the best equilibrium for this instance. Let v 1 = v 2 (U(2) ɛ) and substitute in (5.88) P os A = v 1 + v 2 v 2 (1 + d 1 ) = v 2(U(2) ɛ) + v 2 v 2 (1 + d 1 ) = U(2) ɛ + 1 (1 + d 1 ) (5.92) (5.93) (5.94) Taking limit when ɛ 0 U(2) ɛ + 1 (1 + U(2)) P os A = lim = ɛ 0 (1 + d 1 ) (1 + d 1 ) (5.95) Equation (5.95) is valid for U(2) > d 1 R 1 R 2 R 3 Case - B- o o x x Claim 24 We claim that for this case the price of stability is always 1.

118 Proof: We will show that, either the optimal allocation is a Nash equilibrium allocation and thus the best equilibrium for the game has a P os = 1, or the two allocations are different but in the limit the price of stability is equal to 1. The W 0(a opt ) W 0 (a ne ) ratio for the game instance in case B can be given by 102 W 0 (a opt ) W 0 (a ne ) = W 1(1) + W 2 (1) W 2 (1) + W 3 (1) = v 1 + v 2 (5.96) v 2 + v 3 First recall that if U(2) > 1, this example is not valid, therefore U(2) 1. Assume that having players on R 1 and R 2 is an optimal allocation but not an equilibrium, and that having them on R 2 and R 3 is an equilibrium. {R 1, R 2 } not an equilibrium implies that at least one of the players has a unilateral incentive to deviate; either player P 1 has an incentive to deviate from R 1 to R 2 or player P 2 has an incentive to deviate from R 2 to R 3. v 2 U(2) > v 1 (5.97) OR v 3 > v 2 (5.98) {R 1, R 2 } not an optimal implies that the value of the welfare for this allocation is greater than in any other allocation. 3 v 1 > v 3 (5.99) v 2 > v 3 (5.100) v 1 < v 2 d 1 (5.101) {R 2, R 3 } is an equilibrium implies that player P 1 does not have a unilateral incentive to deviate from R 2 to R 1 and player P 2 does not have a unilateral incentive to deviate 3 The value of the welfare for each allocation is shown in Figure 5.3b.

119 103 from R 3 to R 2. v 1 v 2 (5.102) v 3 v 2 U(2) (5.103) Note that the condition in (5.98) contradicts the optimal condition in (5.100) and thus is never satisfied. Therefore, we need only check the cases for the condition in (5.97). If Equation (5.97) holds and using Equation (5.103), then W 0 (a opt ) W 0 (a ne ) = v 1 + v 2 v 2 + v 3 (5.104) v 2U(2) + v 2 v 2 + v 2 U(2) (5.105) = 1 (5.106) = P os = 1 (5.107) If Equation (5.97) does not hold, then the optimal allocation is a Nash equilibrium allocation and thus P os = 1. R 1 R 2 R 3 Case - C- o o xx Claim 25 We claim that for this case the price of stability is always 1. Proof: We show this by showing that the optimal allocation is always a Nash equilibrium for this case. Since the allocation {R 1, R 3 } is an optimal allocation, then the value of the welfare for this allocation is greater than in any other allocation. 4 v 1 + v 3 > v 2 + v 3 = v 1 > v 2 (5.108) v 1 + v 3 > v 1 + v 2 = v 3 > v 2 (5.109) 4 The value of the welfare for each allocation is shown in Figure 5.3b.

120 104 Equation (5.108) and (5.109) imply that the allocation {R 1, R 3 } is also a Nash equilibrium allocation, since no player will have a unilateral incentive to deviate from this allocation. Recall, from the P oa analysis that this example is not valid if U(2) < 1. The intuition behind the fact that the price of stability is one while the price of anarchy is greater than one for this example, is that the players only benefit from being on R 2 if they are together on it, since U(2) > 1. R 1 R 2 R 3 Case - D- oo x x The W 0(a opt ) W 0 (a ne ) ratio for the game instance in case D can be given by W 0 (a opt ) W 0 (a ne ) = W 2 (2) W 1 (1) + W 2 (1) = v 2W 0 (2) = v 2(1 + d 1 ) (5.110) v 1 + v 2 v 1 + v 2 From the Nash equilibrium condition, player P 1 does not have a unilateral incentive to deviate from R 1 to R 2 U 2 (2) U 1 (1) (5.111) v 2 U(2) v 1 U(1) (5.112) v 2 U(2) v 1 (5.113) Setting v 2 U(2) < v 1, v 2 < v 3 and v 1 < v 2 d 1 ensures that this equilibrium allocation (player P 1 on R 1 and player P 2 on R 2 ) is the unique equilibrium and thus the best equilibrium for this instance. Let v 1 = v 2 (U(2) + ɛ) and substitute in (5.110) P os D = v 2(1 + d 1 ) (5.114) v 1 + v 2 v 2 (1 + d 1 ) = (5.115) v 2 (U(2) + ɛ) + v 2 = (1 + d 1 ) U(2) + ɛ + 1 (5.116)

121 105 Taking limit when ɛ 0 P os D = lim ɛ 0 (1 + d 1 ) U(2) + ɛ + 1 = (1 + d 1) (1 + U(2)) (5.117) Equation (5.117) is valid for U(2) < d 1 Therefore, the price of stability of G 2 can be given as a function of the distribution rule U and the welfare function W 0 : P os(u(2), W 0 ) = max U(2) {P OS A, P OS D, 1} (5.118) { 1 + U(2) = max U(2) W 0 (2), W 0 (2) } (5.119) 1 + U(2) Proposition 26 For a fixed submodular function W 0, let G 2 = {N, R, {A i } {i N }, W 0, {v r } {r R}, U} be the set of all 2-player, anonymous, single-selection resource allocation games with base welfare function W 0. The marginal contribution utility function minimizes the price of stability of G 2. Proof: The proof follows directly from Equation (5.119). Note that 1+d 1 1+U(2) for 0 U(2) d 1,while 1+U(2) 1+d 1 is increasing in U(2) for U(2) > d 1. Moreover, 1 + d U(2) is decreasing in U(2) = 1 + U(2) 1 + d 1 (5.120) = U(2) = d 1 (5.121) Numerical Example Figure 5.6 shows the price of anarchy and the price of stability as a function of U(2), for different values of d 1. We note the following for all games in G 2 :

122 Change in POA and POS as U(2) changes d 1 = 0.1 Change in POA and POS as U(2) changes d 1 = POA POS U MC (2) U * (2) U SV (2) POA POS U MC (2) U * (2) U SV (2) U(2) Utility of a player when sharing the resource with another player (a) d 1 = U(2) Utility of a player when sharing the resource with another player (b) d 1 = 0.5 Figure 5.6: Price of Anarchy and Price of Stability as U(2) changes The price of stability is minimized at the marginal contribution utility function U(2) = d 1 as stated by Proposition 26 The price of anarchy at the Shapley value utility function is always less than at the marginal contribution utility function for 2-player games. The vertical line corresponding to the Shapley value utility function cuts the PoA curves in a lower point than the vertical line corresponding to the marginal contribution utility. The minimum price of anarchy occurs at u = 2(1 + d 1 ) 1, which falls between the Shapley value utility function and the marginal contribution utility function. There is a trade-off between the price of stability and the price of anarchy for d 1 U(2) u, where the price of stability is increasing, meaning that the best guaranteed equilibrium is getting worse, while the price of anarchy is decreasing, meaning that the worst possible equilibrium is getting better. Setting a utility function less than marginal contribution, U(2) < d 1, or more than Shapley value U(2) > 1+d 1 2 makes both the price of anarchy and the price of stability worse.

123 player Resource Allocation Games Game Analysis Refer to Appendix C for 3 players game analysis Price of Anarchy Analysis for 3-player games In this section, we will derive the price of anarchy bounds for the 6 games in Claim 43 in Appendix C. 5 Claim 27 For a fixed submodular function W 0, let G 3 = {N, R, {A i } {i N }, W 0, {v r } {r R}, U} be the set of all 3-player, anonymous, single-selection resource allocation games with base welfare function W 0. The price of anarchy of G 3 as a function of U(2), U(3) and W 0 can be given by, P oa(u(2), U(3), W 0 ) (5.122) { 3U(3) = max W 0 (3), 1 + 2U(3), W 0(2) + U(3), W 0(2) + U(2) 1 + U(2) + U(3), W 0 (3) W 0 (3) W 0 (2) + U(3) W 0 (2) + U(3), 2 + U(2) } U(2) 2 + U(2) + 1 where W 0 (2) = 1 + d 1 and W 0 (3) = 1 + d 1 + d 2 are the values of the base submodular welfare function with 2 and 3 players on a resource. U(2) and U(3) are design parameters; U(2) is the share of one player when there are 2 players on a resource, U(3) is the share of one player when there are 2 players on a resource. Proof: The price of anarchy bounds for the 6 games in Claim 43 can be derived as follows: R 1 R 2 R 3 R 4 Case -A- xxx o o o 5 A summary of all the configurations of the games considered is shown in Figure C.4.

124 108 Action sets of players: A 1 = {R 1, R 2 } (5.123) A 2 = {R 1, R 3 } (5.124) A 3 = {R 1, R 4 } (5.125) Without loss of generality let v 2 = v 3 = v 4. From the Nash equilibrium condition we have, U 1 (3) v 2 (5.126) = v 1 U(3) v 2 (5.127) The price of anarchy can be given by P oa A = 3W 2(1) W 1 (3) 3v 2 = v 1 W 0 (3) 3U(3)v 1 v 1 W 0 (3) = 3U(3) W 0 (3) (5.128) (5.129) (5.130) (5.131) This example is valid for U(3) > W 0(3) 3 = U SV (3) R 1 R 2 R 3 Case -B- xxx o o o Action sets of players: A 1 = {R 1, R 2 } (5.132) A 2 = {R 1, R 2 } (5.133) A 3 = {R 1, R 3 } (5.134)

125 109 Without loss of generality let v 2 = v 3. From the Nash equilibrium condition we have U 1 (3) v 2 (5.135) = v 1 U(3) v 2 (5.136) The price of anarchy can be given by P oa B = W 1(1) + 2W 2 (1) W 1 (3) = v 1 + 2v 2 v 1 W 0 (3) v 1 + 2U(3)v 1 v 1 W 0 (3) = 1 + 2U(3) W 0 (3) (5.137) (5.138) (5.139) (5.140) This example is valid for U(3) > W 0(3) W 0 (1) 3 = d 1+d 2 2 R 1 R 2 Case -C- xxx oo o Action sets of players: A 1 = {R 1, R 2 } (5.141) A 2 = {R 1, R 2 } (5.142) A 3 = {R 1, R 2 } (5.143) From the Nash equilibrium condition we have U 1 (3) v 2 (5.144) = v 1 U(3) v 2 (5.145)

126 110 The price of anarchy can be given by P oa C = W 1(2) + W 2 (1) W 1 (3) = v 1W 0 (2) + v 2 v 1 W 0 (3) v 1W 0 (2) + U(3)v 1 v 1 W 0 (3) = W 0(2) + U(3) W 0 (3) (5.146) (5.147) (5.148) (5.149) This example is valid for U(3) > W 0 (3) W 0 (2) = d 2 = U MC (3) R 1 R 2 R 3 Case -D- xx x oo o Action sets of players: A 1 = {R 1, R 3 } (5.150) A 2 = {R 1, R 3 } (5.151) A 3 = {R 1, R 2 } (5.152) From the Nash equilibrium condition we have v 1 U(2) v 3 (5.153) v 2 v 1 U(3) (5.154) The price of anarchy can be given by This example is valid for U(3) < U(2) P oa D = v 1W 0 (2) + v 3 v 1 W 0 (2) + v 2 (5.155) v 1W 0 (2) + v 1 U(2) v 1 W 0 (2) + v 1 U(3) = W 0(2) + U(2) W 0 (2) + U(3) (5.156) (5.157)

127 111 R 1 R 2 R 3 Case -E- xx x o o o Action sets of players: A 1 = {R 1, R 2 } (5.158) A 2 = {R 1, R 3 } (5.159) A 3 = {R 1, R 2 } (5.160) From the Nash equilibrium condition we have v 1 U(2) v 2 U(2) = v 1 v 2 (5.161) v 2 v 1 U(3) (5.162) The price of anarchy can be given by P oa E = v 1 + v 2 + v 3 v 1 W 0 (2) + v 2 (5.163) v 1 + v 2 + v 1 U(2) v 1 W 0 (2) + v 2 (5.164) v 1 + v 2 min + v 1 U(2) v 1 W 0 (2) + v 2 min (5.165) = v 1 + v 1 U(3) + v 1 U(2) v 1 W 0 (2) + v 1 U(3) 1 + U(2) + U(3) = W 0 (2) + U(3) (5.166) (5.167) where Inequality (5.165) holds from the result of Lemma 41. This example is valid for U(2) > d 1 R 1 R 2 R 3 R 4 Case -F- x x x o o o

128 112 Action sets of players: A 1 = {R 1, R 2 } (5.168) A 2 = {R 2, R 3 } (5.169) A 3 = {R 3, R 4 } (5.170) From the Nash equilibrium condition we have v 1 v 2 U(2) (5.171) v 2 v 3 U(2) (5.172) v 3 v 4 (5.173) The price of anarchy can be given by P oa F = v 2 + v 3 + v 4 (5.174) v 1 + v 2 + v 3 v 2 + 2v 3 (5.175) v 2 U(2) + v 2 + v 3 = = v 2 min + 2v 3 v 2 min (U(2) + 1) + v 3 (5.176) v 3 U(2) + 2v 3 v 3 (U(2) 2 + U(2) + 1) (5.177) 2 + U(2) (U(2) 2 + U(2) + 1) (5.178) where Inequality holds from the result of Lemma 41. This example is valid for U(2) < 1 Therefore, The price of anarchy of G 3 as a function of U(2), U(3) and W 0 can be given by, P oa(u(2), U(3), W 0 ) { 3U(3) = max W 0 (3), 1 + 2U(3) W 0 (3), W 0(2) + U(3) W 0 (3), W 0(2) + U(2) 1 + U(2) + U(3), W 0 (2) + U(3) W 0 (2) + U(3), 2 + U(2) } U(2) 2 + U(2) + 1

129 Numerical Example In this section, we numerically analyze the price of anarchy of anonymous, single-selection resource allocation games with 3 players and submodular welfare function. First, we compare the value of the maximum of the first 3 terms in Equation (5.122), which increase linearly in U(3). { 3U(3) max{p oa A, P oa B, P oa C } = max It is quite direct to see from Equation (5.180) that: = W 0 (3), 1 + 2U(3) W 0 (3), W 0(2) + U(3) } W 0 (3) (5.179) 1 W 0 (3) max{3u(3), 1 + 2U(3), 1 + d 1 + U(3)} (5.180) P oa A > P oa B > P oa C for U(3) > 1, P oa B > P oa C and P oa B > P oa A for d 1 < U(3) < 1 P oa C > P oa B > P oa A for d 2 < U(3) < d 1 We note that the slope of the price of anarchy with respect to U(3) starts at 1 for d 2 < U(3) < d 1, then increases to 2 for d 1 < U(3) < 1. After that, the slope increases to 3 for U(3) > 1. Price of Anarchy (PoA) d 1 = 0.7, d 2 = 0.1 A B C 1 d1 d U(3) Figure 5.7: Comparing P oa A, P oa B and P oa C for 3 players. The figure shows that P oa A dominates the others for U(3) > 1, P oa B is dominant for d 1 U(3) 1 and P oa C is dominant for d 2 U(3) d 1

130 114 Figure 5.7 shows an example of this comparison for d 1 = 0.7 and d 2 = 0.1. Next, we study the expression in Equation (5.122) numerically, by plotting the price of anarchy of the 6 games in the expression for different values of d 1, d 2 and U(2), as U(3) changes from 0. We change d 1 from 0 to 1, d 2 from 0 to d 1 and, using the result of Lemma 21, we change U(2) from d 1 to 1+d 1 2. We display a subset of our plots in Figures 5.8 to 5.11 that represent the family of plots for all possible values of d 1, d 2 and U(2), where d 1 U(2) 1+d 1 2 ; i.e. any other plot will fall under one of the plots in this subset. For each value of (d 1, d 2 ) we consider two different cases for U(2): U(2) = U MC (2) = d 1 and U(2) = U SV (2) = 1+d 1 2. We originally considered 3 cases for U(2), where the third case is U(2) = u according to Proposition 20. However, as demonstrated earlier in Figure 5.5, u is very close to U SV, therefore we omitted the plots for U(2) = u as they are almost equivalent to those of U(2) = U SV (2) Change in POA as U(3) changes d 1 = 0.4, d 2 = 0.2, U(2) = Change in POA as U(3) changes d 1 = 0.4, d 2 = 0.2, U(2) = 0.70 Price of Anarchy (PoA) A 1.4 B C 1.3 D E 1.2 F U MC (3) 1.1 U SV (3) U(2) U(3) Price of Anarchy (PoA) U SV (3) U(2) U(3) A B C D E F U MC (3) (a) d 1 = 0.4, d 2 = 0.2, U(2) = U MC = 0.4. (b) d 1 = 0.4, d 2 = 0.2, U(2) = U SV = 0.7 Figure 5.8: Price of Anarchy as U(3) changes where d 1 = 0.4 and d 2 = 0.2. Minimum PoA determined by P oa F, somewhere between the marginal contribution value and the Shapley value of 3 players; d 2 = U MC (3) U(3) U SV (3) = 1+d 1+d 2 3, more specifically at the value of the utility where P oa C intersects P oa D Table 5.6 summarizes the results of the plots.

131 Change in POA as U(3) changes d 1 = 0.7, d 2 = 0.1, U(2) = 0.70 Change in POA as U(3) changes d 1 = 0.7, d 2 = 0.1, U(2) = Price of Anarchy (PoA) A B C D E F U MC (3) U SV (3) U(2) Price of Anarchy (PoA) A B C D E F U MC (3) U SV (3) U(2) U(3) U(3) (a) d 1 = 0.7, d 2 = 0.1, U(2) = U MC = 0.7 (b) d 1 = 0.7, d 2 = 0.1, U(2) = U SV = Figure 5.9: Price of Anarchy as U(3) changes where d 1 = 0.7 and d 2 = 0.1. Minimum PoA determined by the maximum of P oa F and P oa C P oa D. Change in POA as U(3) changes d 1 = 0.9, d 2 = 0.5, U(2) = 0.90 Change in POA as U(3) changes d 1 = 0.9, d 2 = 0.5, U(2) = Price of Anarchy (PoA) A B C D E F U MC (3) U SV (3) U(2) Price of Anarchy (PoA) A B C D E F U MC (3) U SV (3) U(2) U(3) U(3) (a) d 1 = 0.9, d 2 = 0.5, U(2) = U MC = 0.9 (b) d 1 = 0.9, d 2 = 0.5, U(2) = U SV = 0.95 Figure 5.10: Price of Anarchy as U(3) changes where d 1 = 0.9, d 2 = 0.5. PoA minimized at P oa C = P oa D, somewhere between the marginal contribution value and the Shapley value of 3 players; d 2 = U MC (3) U(3) U SV (3) = 1+d 1+d 2 3, more specifically at the value of the utility where P oa C intersects P oa D From our plots we infer the following: P oa E is always dominated by P oa F or P oa D. 6 6 Note that always here refers to cases of U(2) between marginal contribution and Shapley value. For instance, if U(2) = 1, then P OA E dominates P oa D and P oa F for low values of d 1, d 2 and U(3). Therefore, case E cannot be

132 Change in POA as U(3) changes d 1 = 0.9, d 2 = 0.8, U(2) = 0.90 Change in POA as U(3) changes d 1 = 0.9, d 2 = 0.8, U(2) = Price of Anarchy (PoA) A B C D E F U MC (3) U SV (3) U(2) Price of Anarchy (PoA) A B C D E F U MC (3) U SV (3) U(2) U(3) U(3) (a) d 1 = 0.9, d 2 = 0.8, U(2) = U MC = 0.9 (b) d 1 = 0.9, d 2 = 0.8, U(2) = U SV = 0.95 Figure 5.11: Price of Anarchy as U(3) changes where d 1 = 0.9, d 2 = 0.8. Minimum PoA determined by P oa F, somewhere between the marginal contribution value and the Shapley value of 3 players; d 2 = U MC (3) U(3) U SV (3) = 1+d 1+d 2 3, more specifically at the value of the utility where P oa C intersects P oa D The price of anarchy increases for values of U(3) > U SV (3) = W 0(3) 3 and for values of U(3) < U MC (3) = d 2. (Similar to the result of Lemma 21). Therefore we do not need to consider P oa A or P oa E For low values of U(2), P oa F = d 2 < U(3) < 1+d 1+d U(2) 1+U(2)+U(2) 2 maximizes the expression in (5.122) for For low values of d 1 and/or d 2, P oa B = 1+2U(3) W 0 (3) maximizes the expression in (5.122) for d 1 < U(3) < 1. For high values of U(2), P oa D = W 0(2)+U(2) W 0 (2)+U(3) maximizes the expression in (5.122) for U(3) < d 1 +d 2 2 The minimum possible value for the expression in (5.122) is either equal to the expression of P oa F or the intersection of P oa D and P oa C. omitted from the original analysis.

133 Figure d 1 d 2 U(2) minimum P oa min P oa Figure 5.8a U MC (2) = 0.4 P oa F = Figure 5.8b U SV (2) = 0.7 P oa F = Figure 5.9a U MC (2) = 0.7 P oa F = Figure 5.9b U SV (2) = 0.85 P oa C = P oa D = 1.19 Figure 5.10a U MC (2) = 0.9 P oa C = P oa D = 1.08 Figure 5.10b U SV (2) = 0.95 P oa C = P oa D = 1.09 Figure 5.11a U MC (2) = 0.9 P oa F = Figure 5.11b U SV (2) = 0.95 P oa F = Table 5.6: Summary of the results of Figures 5.8 to 5.11 However, when U(2) has a high value, the expression of P oa F is lower than when U(2) has a low value. Now we mathematically calculate the value of U(3) for which P oa C = P oa D P oa C = P OA D (5.181) = W 0(2) + U(3) W 0 (3) = W 0(2) + U(2) W 0 (2) + U(3) (5.182) Cross multiplying, we get (W 0 (2) + U(3)) 2 = W 0 (3)(W 0 (2) + U(2)) (5.183) = (W 0 (2) + U(3)) = (W 0 (3)(W 0 (2) + U(2))) (5.184) = U (3) = (W 0 (3)(W 0 (2) + U(2))) W 0 (2) (5.185) The plots (even those omitted for brevity) show that the minimum price of anarchy occurs between the Shapley value and the marginal contribution value. It also can be easily shown that U MC (3) U (3) U SV (3). Figure 5.12 shows a 3D plot for the change in price of anarchy as a function of both U(2) and U(3). The boundaries of the figure are determined by the marginal contribution and Shapley value utilities; i.e. d 1 U(2) 1+d 1 2 and d 2 U(3) 1+d 1+d 2 3, where d 1 = 0.7 and d 2 = 0.1. As inferred

134 from Figure 5.9, the minimum price of anarchy at U(2) = U MC (2) = d 1 is determined by P oa F and the minimum price of anarchy at U(2) = U SV (2) = 1+d 1 2 is determined by the intersection of P oa D and P oa C. Moreover, the 3D plot shows that the minimum price of anarchy as a function of U(2) and U(3) occurs somewhere in the middle of the graph and it is not corresponding to U(2) = u POA_D U(2) = U_SV(2) 1.4 PoA_C PoA 1.3 min{poa} PoA_F U(2) = U_MC(2) 0.75 UH2L UH3L Figure 5.12: 3D plot showing The Price of Anarchy as (U(2), U(3)) change where d 1 = 0.7, d 2 = 0.1. The figure shows that the minimum price of anarchy at U(2) = U MC (2) = d 1 is determined by P oa F and the minimum price of anarchy at U(2) = U SV (2) = 1+d 1 2 is determined by the intersection of P oa D and P oa C. It also shows that the minimum price of anarchy as a function of U(2) and U(3) occurs somewhere in the middle of the graph and it is not corresponding to U(2) = u The point at which the price of anarchy is minimized in Figure 5.12 can be calculated by equating the expressions P oa C = P oa D = P oa F. From P oa C = P oa D, we get an expression for U(3) in terms of U(2) given by Equation (5.185). Substituting back into P oa D and equating it to P oa F we get another expression for U(2) = u 2. It is clear from the figure that u 2 u derived 7 Middle: refers to a value between the Shapley value and the marginal contribution for both of U(2) and U(3).

135 119 in Proposition 20. u can be given as the solution of Equation (5.186) (1 + d 1 + u 2 ) (1 + d1 + u 2 )(1 + d 1 + d 2 ) = 2 + u u 2 + (u 2 )2 (5.186) We end this section by noting that for 3-player games the price of anarchy was in many cases determined by the worst case example in case F, which is an example of a shifted-allocation game in Definition 15. In shifted-allocation games, any resource has at most 2 players at a time, and thus the price of anarchy is determined by the value of U(2). In the limit, shifted-allocation games, with U(2), has a price of anarchy that tends to 2 U(2). 8 This type of example has a worst-case flavor for any number of players and is always dependent only on the share of a player when there are 2 players on a resource. 5.3 Resource Allocation Games with a finite number of players In this section, we would like to draw some conclusions from our analysis of 2 and 3-player games and see if we can generalize to n-player games. Optimal Utility function Comparing the expression of U (3) in Equation (5.185) to the expression of U (2) in Proposition 20, we see a similarity: U (3) = (W 0 (3)(W 0 (2) + U(2))) W 0 (2) (5.187) U (2) = (2W 0 (2)) 1 (5.188) where U (2) can be written as U (2) = (W 0 (2)(W 0 (1) + U(1))) W 0 (1) (5.189) since W 0 (1) = U(1) = 1. 8 Similar to the proof of Proposition 15.

136 120 Is it true that in general, U (n)? = (W 0 (n)(w 0 (n 1) + U(n 1))) W 0 (n 1) (5.190) minimizes the price of anarchy for n-player games? Utility Higher than Shapley Value We claim that for anonymous, single-selection, n-player resource allocation games it is never beneficial to set the utility of players to a value higher than that of the Shapley value. The intuition for this claim follows from the worst-case example format presented in Proposition 13, which is also a worst case format for games where the utility of players is set to a value higher than Shapley value. Moreover, the price of anarchy will worsen as the utility increases above Shapley value. We prove this later more formally in Chapter 6, Corollary 34. Utility Lower than Marginal Contribution Similarly, using the worst case format derived for resource allocation games with the marginal contribution utility, in Proposition 14, it is clear that setting the utility to a value lower than the marginal contribution will lead to the same worst case example with a worse price of anarchy. Therefore, it is never helpful to the price of anarchy to consider utility values outside the range of marginal contribution and Shapley value. We claim that the minimizer of the price of anarchy as a function of {U(i)} i N is neither equal to the Shapley value nor the marginal contribution, but lies somewhere in between, as suggested by the 3D plot in Figure Summary In this chapter we performed an exhaustive search on all possible 2-player and 3-player, anonymous, single-selection resource allocation games. We made the following contributions:

137 121 Derived a worst case game format for anonymous, single-selection, 2-player games. Analyzed the price of anarchy for 2-player games, deriving a value of U(2) = u that minimizes the price of anarchy for all anonymous, single-selection, 2-player games. We showed that, for all values of d 1, u is very close to the Shapley value (U SV (2) = 1+d 1 2 ). Moreover, for d u U SV (2). Analyzed the price of stability for 2-player games, and we showed that there is a trade-off between the price of anarchy and the price of stability for d 1 = U MC (2) U(2) u 2 U SV (2) = 1 + d 1 2 Analyzed 3-player games and derived an expression for the price of anarchy. Explained the importance of selecting U(2), by giving a worst case example that is solely dependent on U(2), even for games where the number of players is strictly greater than 2. Inferred the effect of changing d 1, d 2, U(2) on the price of anarchy as U(3) changes. Analyzed, briefly, price of anarchy as a function of the pair (U(2), U(3)) and showed that in a 3D plot for some welfare function W 0. Listed some directions for generalizing to n-player games.

138 Chapter 6 Effect of Violating the Budget-Balance Condition on the Robust Price of Anarchy In Section 3.2 we defined smooth games and robust price of anarchy, which were introduced by Roughgarden in [32]. Recall that the robust price of anarchy, in Definition 10, holds for smooth games that are budget-balanced or at least satisfy the relaxed budget-balance condition, given by C(a) n J i (a) in cost-minimization games and W (a) n U i (a) in payoff-maximization games. i=1 In a budget-balanced system the total welfare is exactly distributed among players with no excess or deficit. On the other hand, a relaxed budget-balance condition allows for excess in the total welfare (or cost) after each player has taken (or paid) its share, but does not allow for deficit. In this chapter, we will explore the effect of allowing for deficit in the budget on the robust price of anarchy of a smooth game. Allowing for a deficit implies that C(a) > n J i (a) in cost-minimization games or W (a) < n U i (a) in payoff-maximization games. i=1 We start, in Section 6.1, by reviewing the reverse carpooling game, presented in [19], which is a motivation for this chapter. In their work, the authors study different cost distribution rules i=1 i=1 for a network coding problem that is formulated as a game. 1 Moreover, they study the effect of each of these cost distribution rules on the price of anarchy of the game, and they conclude with an optimal cost distribution rule that guarantees the lowest price of anarchy and it happens to satisfy the relaxed budget-balance condition. In addition to that, they study a cost distribution rule that does not satisfy the relaxed condition and show that the further the cost functions are from 1 Network Coding is a node-level coding, in a computer network, that allows nodes to combine packets together such that they are transmitted efficiently through the network.

139 123 balancing the budget, the worse the price of anarchy of the associated game is. This work motivated us to study whether this phenomena was coincidental or fundamental in resource allocation games, and thus derive the general results presented in Section 6.2. In Section 6.2, we re-derive the robust price of anarchy for games that do not necessarily satisfy any budget-balance condition. We show that non-budget-balanced, smooth games have a worse robust price of anarchy, which worsens the more we go off budget in the wrong direction. After that, in Section 6.3, we formalize the cost distribution rules of the reverse carpooling game, given in [19], and show that the game is smooth and derive the (λ, µ) parameters. Then we apply the results of Section 6.2 to the reverse carpooling game and show that they match the results, in [19], of the original paper. Lastly, we summarize the chapter in Section Reverse Carpooling Game Network Coding is a mapping technique implemented on a node-level in computer networks [9]. It allows nodes to combine packets together such that they are transmitted efficiently through the network. An efficient transmission is one that satisfies a certain objective function, such as minimizing the power consumption, increasing the network throughput, improving the robustness of communication or increasing the system-level security. Reverse Carpooling is a type of network coding where a relay node 2 codes two packets received from opposite directions and broadcasts their bit-wise sum (XOR) and thus combine two transmissions into one transmission that can be decoded easily at the destinations. Figure 6.1 shows a transmission from node X to node Y through a relay node. Node X sends packet P x to node Y through node R, and node Y sends packet P y to node X through the same relay node. The relay node broadcasts one packet P x P y instead of sending two separate packets to X and to Y. Node X is able to decode the the received packet P x P y easily by knowledge of the original packet sent P x, similarly node Y is able to decode the received packet P x P y easily by knowledge of the original packet sent P y. In short, reverse carpooling allows two flows to share a single transmission, 2 A relay node is a node between two communicating nodes that relays the messages between them

140 Figure 6.1: Reverse Carpooling 124 if they are in opposite direction, and thus reduce the power consumption of the transmission. The authors in [19], present the reverse carpooling optimization problem as a game and they design several cost distribution rules for this game and quantify each of the rules using worst case efficiency measures. In Section and Section 6.1.2, we review the notation of the reverse carpooling game and the cost distribution rules presented in [19] Game Notation A network is defined by a set of nodes (or vertices), V = {v 1,, v m }, and a set of links (or edges) between those nodes, E = {e 1,, e l }. The neighbors of v i are defined as the nodes which are connected to v i by a link; N (v i ) V for each v i V. The reverse carpooling game is defined as follows: A finite set of players N = {1,, n} Player i is a source-destination pair of nodes; source s i to destination t i, where (s i, t i ) V 2. An action set A i for player i A i is the set of all possible paths from s i to t i ; i.e. the set of possible actions available to player i. A path a i from source s i to destination t i is a set of (relay) nodes a i = {v 1, v 2,..., v ai }, where a i denotes the number of nodes in a i, v 1 = s i, v ai = t i and v k+1 N (v k ) for all k in between.