Stable Utility Design for Distributed Resource Allocation*

Transcription

1 Stable Utility Design for Distributed Resource Allocation* Ragavendran Gopalakrishnan 1 and Sean D. Nixon 2 and Jason R. Marden 3 Abstract The framework of resource allocation games is becoming an increasingly popular modeling choice for the distributed control and optimization. In recent years, this approach has evolved into the paradigm of game-theoretic control, which consists of first modeling the interaction between the distributed agents as a noncooperative game, and then designing local utility functions for these agents such that the resulting game possesses a stable outcome (e.g., a pure Nash equilibrium) that is efficient (e.g., good price of anarchy properties). One then appeals to the large, existing literature on learning in games for distributed algorithms for agents that guarantee convergence to such an equilibrium. An important first problem is to obtain a characterization of stable utility designs, that is, those that guarantee equilibrium existence for a large class of games. Recent work has explored this question in the general, multi-selection context, that is, when agents are allowed to choose more than one resource at a time, showing that the only stable utility designs are the so-called weighted Shapley values. It remains an open problem to obtain a similar characterization in the single-selection context, which several practical problems such as vehicle target assignment, sensor coverage, etc. fall into. We survey recent work in the multiselection scenario, and show that, perhaps surprisingly, even though other utility designs become stable for specific singleselection applications, in a broader context, the limitation to weighted Shapley value utility design continues to prevail despite the restriction to single-selection games. I. INTRODUCTION Resource allocation is a fundamental problem that is at the core of several application domains ranging from socioeconomic systems to systems engineering [1], [2], [3], [4], [5], [6]. A persistent example in the computer science literature is that of routing data through a shared-link network, where the global objective is to minimize the average delay [7], [8]. An alternative example in multiagent systems is the problem of deploying sensors in a given mission space where the global objective is to maximize the area covered and/or the quality of coverage [3]. The central objective in all these problems is to allocate resources to optimize some global objective. Increasingly, these problems need to be solved in a distributed, decentralized manner, especially in large scale engineering systems where a centralized approach is too costly and perhaps even impossible. Game-theoretic control, in recent years, has emerged as a promising approach for distributed resource allocation [9], *This research was supported by AFOSR grant #FA and ONR grant #N Department of Electrical, Computer, and Energy Engineering, University of Colorado, Boulder, CO 80309, USA raga@colorado.edu 2 Department of Mathematics and Statistics, University of Vermont, Burlington, VT 05405, USA Sean.Nixon@uvm.edu 1 Department of Electrical, Computer, and Energy Engineering, University of Colorado, Boulder, CO 80309, USA jason.marden@colorado.edu [10], [5], [6]. This approach is motivated by the fact that the underlying decision making architecture in economic systems and distributed engineering systems is identical, that is, local decisions based on local information resulting in emergent global behavior. Game theoretic control consists of two distinct steps. First, the interactions of autonomous agents are modeled within the framework of a noncooperative game where the agents are self-interested. This involves specifying decision makers ( players ), their respective choices ( action sets ), and a local utility function for each agent, so that the resulting game has an equilibrium (in this paper, we consider pure Nash equilibria), which takes on the role of a stable operating point. The second step is to specify a local learning rule for the agents according to which they process available information to make individual decisions that collectively steer the system towards an equilibrium of the game. The goal is to complete these two design steps, referred to as utility design and learning design respectively, in order to ensure that the emergent global behavior is desirable [11], [12], [13]. Our focus in this paper is on utility design for separable resource allocation problems, where separable means that the global objective function can be decomposed into local objective functions, one for each resource, which can be thought of as the welfare (cost/revenue) garnered at that resource. A key constraint of utility design is that it must be local, meaning that an agent s utility function can only depend on the resources selected, the welfare at each resource, and the other agents that selected the same resources. Therefore, designing local utility functions reduces to the problem of defining a distribution rule that specifies how the welfare garnered from each resource is distributed to the players who have chosen that particular resource. A fundamental research problem in utility design is to characterize the space of distribution rules that guarantee equilibrium existence in resource allocation games. Such a characterization would provide a structured search space while optimizing for distribution rule(s) whose equilibria have the best efficiency properties (how well the equilibrium outcome performs in relation to the globally optimal outcome) and/or are easy to converge to. Chen et al. [14], and later, Marden and Wierman [15], were the first to address this problem and showed the existence of a special, worst-case, welfare function for which, any budget-balanced distribution rule (one that completely distributes the welfare garnered among the agents without surplus or deficit) that guarantees the existence of a pure Nash equilibrium in any resource allocation game must be equivalent to a weighted

2 Shapley value. 1 Most recently, Gopalakrishnan et al. [22] generalized the result of [14], [15], i.e., the weighted Shapley value characterization holds for any welfare function, and even if the budget-balance constraint is dropped. The above work does not, however, investigate characterizations of stable distribution rules when agents are restricted to choosing only a single resource. It is entirely possible that in such single-selection settings, distribution rules other than weighted Shapley values may also guarantee equilibrium existence. In fact, recent work suggests that the landscape of single-selection resource allocation games could be vastly different. In particular, [23] shows that for coverage games such as vehicle target assignment, a much simpler distribution rule called proportional share guarantees equilibrium existence. It simply involves assigning certain weights to agents and distributing the welfare at a resource among the agents at that resource, in proportion to their weights. This compels the question: Is the emergence of such alternate stable distribution rules simply for a few isolated welfare functions related to coverage-type problems, or is it a broader phenomenon to be expected for all welfare functions? Our goal in this paper is to explore this question by investigating for characterizations of the space of distribution rules that guarantee equilibrium existence in single-selection resource allocation games for all welfare functions. Of course, since single-selection games are just a special subclass of multi-selection games, weighted Shapley values continue to guarantee equilibrium existence; so, to begin with, we do inherit for free, a partial characterization from previous work on multi-selection games. However, it is important to search for other feasible distribution rules for the following two reasons: (i) weighted Shapley values cannot be computed in polynomial time; therefore, simpler distribution rules such as proportional share are appealing from a computational perspective, and (ii) recent work [8] has shown that there are settings where there are distribution rules other than weighted Shapley values that result in more efficient equilibria. In particular, [8] shows that the proportional share distribution rule has the best equilibrium efficiency across all budgetbalanced distribution rules, and even beats the non-budgetbalanced, but popular, marginal contribution rule. Our Contributions: First, in Section III, we survey recent work that has (i) completely characterized distribution rules that guarantee equilibrium existence in multi-selection resource allocation games as weighted Shapley values, and (ii) shown that other distribution rules guarantee equilibrium existence in special settings of single-selection resource allocation games. Not only does this survey help consolidate important recent efforts in stable utility design, thereby solidifying the context of our work in this paper, but it also exposes the stark contrast between the general results limit- 1 The (weighted) Shapley value is a game-theoretic solution concept of enormous importance in the economics literature [16], [17], [18], due to its fairness properties. Moreover, a design through weighted Shapley values guarantees that the resulting game is a so called (weighted) potential game, for which there are many well understood learning dynamics [19], [20], [21] that guarantee equilibrium convergence. ing utility design to weighted Shapley values in the multiselection scenario and the special results that identify other stable utility designs in specific single-selection scenarios. Motivated by this observation, and the lack of any general characterization results for single-selection games, we investigate whether the extra flexibility in utility design obtained by restricting the structure of the action sets to single-selection, for the specific setting of coverage games, is a phenomenon that holds more generally. Our results, presented in Section IV, provide the first complete characterizations for budget-balanced 2 distribution rules that guarantee equilibrium existence for single-selection resource allocation games. In particular, we show that: 1) For any linear welfare function with no dummy players, 3 the only budget-balanced distribution rules that guarantee equilibrium existence in all single-selection games are weighted Shapley values (Theorem 1). 2) For any welfare function, the only budget-balanced distribution rules that guarantee equilibrium existence in all two-player single-selection games are weighted Shapley values (Theorem 2). Thus, perhaps surprisingly, we show that the limitation to weighted Shapley value utility design (and the accompanying limitation to (weighted) potential games) persists in a broader sense, even when restricted to single-selection action sets. II. MODEL We consider a simple, but general, model of a resource allocation game, where there is a set of self-interested agents/players N = {1,..., n} (n > 1) that each select a collection of resources from a set R = {r 1,..., r m } (m > 1). That is, each agent i N is capable of selecting potentially multiple resources in R (multi-selection scenario); therefore, we say that agent i has an action set A i 2 R. If, on the other hand, agents are restricted to selecting only a single resource, resulting in their action sets consisting of only singleton actions, then we are in the single-selection scenario. The resulting action profile, or (joint) allocation, is a tuple a = (a 1,..., a n ) A where the set of all possible allocations is denoted by A = A 1... A n. We occasionally denote an action profile a by (a i, a i ) where a i A i denotes the actions of all agents except agent i. Each allocation generates a welfare, W(a), which corresponds to the global objective function of the optimization problem that a system manager would seek to maximize. We assume W(a) is (linearly) separable across resources, i.e., W(a) = r R W r ({a} r ), where {a} r = {i N : r a i } is the set of agents that are allocated to resource r in a, and W r : 2 N R is the local welfare function at resource 2 Unlike economic systems where monetary incentives are involved, there is no obvious motivation for requiring budget-balance in designing utilities for distributed agents in engineering systems. However, a bevy of recent work [24], [25], [26], [27], [28], [8] has shown that imposing budget constrains on utility design leads to good equilibrium efficiency guarantees. Despite these efforts, it remains an open problem to characterize precisely, the relationship between budget-balance and efficiency. 3 Note that such welfare functions, which we define precisely in Section IV, constitute a large class.

3 r. This is a standard assumption [29], [30], [14], [23], and is quite general. Without loss of generality, we assume W r ( ) = 0 for all r R. Given W, the goal is to design the utility functions U i : A R that the agents seek to maximize. Because the welfare is assumed to be separable, it is natural that the utility functions should follow suit, i.e., U i(a) = r a i f r (i, {a} r), (1) where f r : N 2 N R is the local distribution rule at resource r, i.e., f r (i, S) is the portion of the local welfare W r that is allocated to agent i S when sharing with S. Finally, we assume that resources with identical local welfare functions have identical distribution rules, i.e., for any two resources r, r R, W r = W r = f r = f r. (2) In other words, the distribution rule f r only depends on the resource r through the corresponding local welfare function W r. In light of this assumption, in the rest of this paper, we write f Wr instead of f r. For completeness, we define f Wr (i, S) := 0 whenever i / S. The requirement (1) on the functional form of an agent s utility function embodies the feature of locality that is often a critical constraint in distributed systems, i.e., each agent s utility function only depends on its selected resources and the agents that select the same resources. In addition, defining a resource-specific distribution rule {f r } r R of the form (2) enables the utility design to be independent of the identity of the resource as well as the specific structure of the action sets {A i } i N, allowing for a design that is scalable. A distribution rule f Wr is said to be budget-balanced 4 if, for any player set S N, i S f Wr (i, S) = W r (S). In order to simplify the statement and discussion of our results, we impose a stricter form of scalability, by assuming that there is just one base welfare function W, that is scaled at each resource r by a strictly positive coefficient, v r R ++. That is, W r = v r W, for all r R. And therefore, f r = v r f W, where f W is the base distribution rule. However, our results hold more generally. We represent a resource allocation game as: ( ) G = N, R, {A i} i N, {v r} r R, W, f W, (3) and the design of f W is the focus of this paper. The primary goals when designing the distribution rules are to guarantee (i) equilibrium existence, and (ii) equilibrium efficiency. Our focus in this work is entirely on (i) and we consider pure Nash equilibria; however, other equilibrium concepts are also of interest [31], [32], [33]. Recall that a (pure Nash) equilibrium is an outcome a A such that ( i N) U i(a i, a i) = max a i A i U i(a i, a i). 4 Unlike economic systems where monetary incentives are involved, there is no obvious motivation for requiring budget-balance in designing utilities for distributed agents in engineering systems. However, a bevy of recent work [24], [25], [26], [27], [28], [8] has shown that imposing budget constrains on utility design leads to good equilibrium efficiency guarantees. Despite these efforts, it remains an open problem to characterize precisely, the relationship between budget-balance and efficiency. Examples of Distribution Rules: In this subsection, we introduce and discuss three distribution rules that commonly occur in the cost sharing literature, namely proportional share, weighted Shapley value, and weighted marginal contribution distribution rules. Table I summarizes their properties. All distribution rules are (exogenously) parameterized by ω R N ++, a vector of strictly positive player weights. The proportional share distribution rule, denoted by [ω], is defined as: f W P R fp W ω R [ω](i, S) = i j S ω j W (S). (4) In other words, the welfare is simply divided among the players in proportion to their weights. Note that this is very easy to compute, requiring just a single call to the welfare function. A special case emerges when all the weights are equal, which is known as the equal share distribution rule, and has been the subject of much prior work [29], [34], [35], [30], [36] in cost sharing. The proportional share distribution rule does not always guarantee equilibrium existence, but recent work [8] has shown that in some settings, when it does, it minimizes the price of anarchy among all budget-balanced distribution rules. The weighted Shapley value distribution rule, denoted by fw W SV [ω], is defined as: fw W SV [ω](i, S) = T S:i T ω i j T ω j R T( 1) T R W (R). (5) One way to interpret this distribution rule is as follows. For any given subset of players S, imagine these players arriving one at a time to the resource, according to some order π. Each player i can be thought of as contributing W (Pi π {i}) W (Pi π) to the welfare W (S), where P i π denotes the set of players in S that arrived before i in π. This is the marginal contribution of player i to the welfare, according to the order π. Then, the weighted Shapley value is the expected marginal contribution, where the expectation is over all S! orders, according to a probability distribution determined by the weights ω. A special case emerges when all the weights are equal, which is known as the (unweighted) Shapley value. These distribution rules are named after their creator, Lloyd Shapley [16], [17]. The importance of weighted Shapley value distribution rules is that they guarantee equilibrium existence in any game, and also guarantee that the resulting games are so-called (weighted) potential games [37], [38], for which there are many well understood learning dynamics [19], [20], [21] that guarantee equilibrium convergence. However, the key drawback is that computing them is often intractable [39], [40], since it requires computing exponentially many marginal contributions. The weighted marginal contribution distribution rule, denoted by fw W MC [ω], is defined as: f W W MC [ω](i, S) = ω i (W (S) W (S {i})). (6) That is, each player s share is simply their marginal contribution to the welfare, scaled by their own weight. A special

4 TABLE I: Properties of Common Distribution Rules NAME BUDGET- BALANCED? GUARANTEED EQUILIBRIUM EXISTENCE? POTENTIAL GAME? COMPUTABILITY Proportional share Yes No No Easy Weighted Shapley value Weighted marginal contribution Yes Yes Yes Hard No Yes Yes Easy case emerges when players all have unit weight, resulting in the (unweighted) marginal contribution distribution rule, which is also known as the wonderful life utility [41]. Clearly, fw W MC [ω] is not budget-balanced. However, like the weighted Shapley value, it guarantees equilibrium existence, and the resulting game is a (weighted) potential game. In addition, the potential function is the same as the welfare function; therefore, the welfare maximizing allocation is always an equilibrium, i.e., the price of stability is one. Unfortunately, there are generally additional suboptimal equilibria. Note that computing the marginal contribution is easy, requiring just two calls to W. Example: The Vehicle Target Assignment Problem: The vehicle target assignment problem [42] consists of a finite set of targets (resources) denoted by R and each target r R has a relative worth v r > 0. There are a finite number of vehicles (agents) denoted by N = {1, 2,..., n}. The set of possible assignments for vehicle i is A i 2 R, and A = i N A i represents the set of joint assignments. In general, the structure of A is not available to a system designer a priori. Lastly, each vehicle i N is parameterized with an invariant success probability 0 p i 1 that indicates the probability vehicle i will successfully eliminate a target r given that r a i. The benefit of a subset of agents S N, S, being assigned to a target r is W r (S) = v r W (S), where ( W (S) = 1 i S(1 ) p i) (7) represents the joint probability of successfully eliminating target r. Accordingly, the target assignment problem is to find a joint assignment a A that maximizes the global objective W(a) = W r({a} r), (8) r R:{a} r where {a} r = {i N : r a i }. Note that this is a scalable and separable resource allocation problem that fits our model. Thinking of W r as the local welfare garnered at resource r, designing local agent utilities of the form (1) for this problem reduces to designing distribution rules f W for the base welfare function W in (7). III. EXISTING RESULTS ON STABLE DISTRIBUTION RULES Before presenting our new results on stable utility design for single-selection resource allocation games, we briefly survey existing results that provide complete and/or partial characterizations of distribution rules that guarantee equilibrium existence, as this would solidify the motivation and context for our new results. Table II summarizes these results. In this section and the next, we will be concerned with the following two classes of games: G(N, f W, W ) denotes the class of all resource allocation games G (see (3)) with player set N, base welfare function W, and base distribution rule f W. G s (N, f W, W ) G(N, f W, W ) denotes the subclass of all single-selection games in G(N, f W, W ). A. Complete Characterizations As mentioned earlier, Chen et al. [14] and Marden and Wierman [15] took the first steps toward characterizing distribution rules that guarantee equilibrium existence. Proposition 1 combines the main contributions of these two papers: Proposition 1: (Chen et al. [14], Marden and Wierman [15]) There exists a base welfare function W for which all games in G(N, f W, W ) possess a pure Nash equilibrium for a budget-balanced f W if and only if there exists a weight-vector ω such that f W is the weighted Shapley value distribution rule, fw W SV [ω]. Less formally, Proposition 1 states that if one wants to use a distribution rule that is budget-balanced and guarantees equilibrium existence for all possible welfare functions and action sets, then one is limited to the class of weighted Shapley value distribution rules. This result is shown by exhibiting a specific worst-case base welfare function W for which this limitation holds. When designing a distribution rule, one often knows the specific welfare function W for the application, and Proposition 1 claims nothing in cases where W W, where, in particular, there may be other budget-balanced distribution rules that guarantee equilibrium existence for all games. In addition, weighted marginal contribution distribution rules are non-budget-balanced distribution rules that guarantee equilibrium existence, and there could potentially be others as well. Following on the above broad characterization, Gopalakrishnan et al. [22] searched for characterizations of distribution rules (not necessarily budget-balanced) that guarantee equilibrium existence for an arbitrary base welfare function. Proposition 2 states their main result. Proposition 2: (Gopalakrishnan et al. [22]) Given any base welfare function W, all games in G(N, f W, W ) possess a pure Nash equilibrium if and only if there exists a weightvector ω and a ground welfare function W such that f W is the weighted Shapley value distribution rule, f W W SV [ω], where W is the actual welfare that is distributed by f W, defined as: ( S N) W (S) = f W (i, S). (9) i S Note that Proposition 2 is a much stronger statement than Proposition 1, because it states that for any base welfare function, the distribution rule must be equivalent to a weighted Shapley value on some ground welfare function to guarantee equilibrium existence. This holds true even when the distribution rule is not budget-balanced. In other words, any distribution rule that guarantees equilibrium existence for all games must be a weighted Shapley value on the actual welfare that it distributes.

5 Proposition 2, at first glance, may seem to contradict the fact that weighted marginal contribution distribution rules, which are non-budget-balanced, also guarantee equilibrium existence in all games. However, [22] exposes a surprising connection between the weighted Shapley value and weighted marginal contribution distribution rules: Lemma 1: (Gopalakrishnan et al. [22]) Let W be the space of all welfare functions. Then, for every weight-vector ω, there exists a bijection h ω : W W such that, ( W W) fw W SV [ω] = f hω(w ) W MC [ω]. Informally, Lemma 1 says that weighted Shapley values and weighted marginal contributions are equivalent, except with respect to different ground welfare functions, whose relationship is given by h ω. 5 This observation immediately leads to the following equivalent characterization: Proposition 3: (Gopalakrishnan et al. [22]) Given any base welfare function W, all games in G(N, f W, W ) possess a pure Nash equilibrium if and only if there exists a weightvector ω and a ground welfare function W such that f W is the weighted marginal contribution distribution rule, f W W MC [ω], where W = h ω (W ), W is defined in (9), and h ω is the bijection of Lemma 1. Importantly, Proposition 3 provides an alternate way of designing distribution rules that guarantee equilibrium existence. The advantage of this alternate design is the easier computability of marginal contributions; however, it becomes much harder to control budget-balance, because W is not the actual welfare that is distributed. If one desires to control the budget, it is necessary to start with the desired welfare to be distributed, W, and then perform a preprocessing step of transforming it into W using the bijection h ω. Event though this transformation takes exponential time [22], it needs to be performed only once. A fundamental, and perhaps surprising, message conveyed by the above characterizations (Propositions 1-3) is that the only way to guarantee equilibrium existence in all resource allocation games is to ensure that a (weighted) potential game emerges, because Shapley values and marginal contributions are known to result in potential games [37]. B. Partial Characterization for Single-Selection Games Single-selection resource allocation games are, of course, a subclass of multi-selection resource allocation games; therefore, any distribution rule that guarantees equilibrium existence in the latter class also does so in the former, e.g., weighted Shapley value and weighted marginal contribution distribution rules. However, because the class of games for which we desire equilibrium existence is smaller, there could be other distribution rules that now guarantee equilibrium existence. Marden and Wierman [23] outline three sufficient conditions for a distribution rule f W to guarantee equilibrium existence for all single-selection games: 1) Let i, j N be any two players. If f W (i, S {i, j}) > f W (j, S {i, j}) for some player set S N {i, j}, 5 For the precise definition of the bijection h ω, refer to [22]. TABLE II: Distribution Rules that Guarantee Equilibrium Existence CLASS OF GAMES Multi-selection Single-selection FOR ALL WELFARE FUNCTIONS, BUDGET-BALANCED ONLY weighted Shapley values (Proposition 1) Weighted Shapley values (follows from Proposition 1) FOR A GIVEN WELFARE FUNCTION ONLY weighted Shapley values (Proposition 2) ONLY weighted marginal contributions (Proposition 3) Weighted shapley values and weighted marginal contributions (follows from Propositions 2-3) Those satisfying three sufficient conditions (Proposition 4) Proportional shares for coverage games (see [23]) then, f W (i, S {i, j}) f W (j, S {i, j}) for any player set S N {i, j}. For this situation, we say that player i is stronger than player j. 2) If player i is stronger than player j, then for any player set S N {i, j}, we have, f W (i, S {i}) f W (i, S {i, j}). 3) If player i is stronger than player j, then for any player set S N {i, j}, we have, f W (j, S {j}) f W (i, S {i}) f W (j, S {i, j}) f W (i, S {i, j}). We now state their main result, which provides a partial characterization of stable distribution rules for singleselection games. Proposition 4: (Marden and Wierman [23]) Given any base welfare function W, all games in G s (N, f W, W ) possess a pure Nash equilibrium if f W satisfies the above three sufficient conditions. Note that Proposition 4 does not require budget-balance. While it is easy to see that the space of distribution rules that satisfy the three sufficient conditions is nonempty, the question remains open as to whether they guarantee additional properties pertaining to the structure of the game besides equilibrium existence. In [23], it is also shown that for the specific application of coverage problems, a weightvector ω can be chosen such that the proportional share distribution rule, fp W R [ω] satisfies these three conditions and thus guarantees equilibrium existence for all single-selection coverage games. As an illustrative example, consider the vehicle target assignment problem introduced towards the end of Section II. If the agent probabilities p i are interpreted as their weights, that is, ω i = p i, then, it can be verified that the proportional share distribution rule is given by fp W R [ω] satisfies all three sufficient conditions. The player strengths that these conditions bring out are simply the agent probabilities/weights. IV. NEW RESULTS AND DISCUSSION It can be seen that, unlike for multi-selection games, none of the characterizations for single-selection games in Table II are complete. To combat this deficiency, in this section, we

6 investigate for complete characterizations of budget-balanced distribution rules that guarantee equilibrium existence for all single-selection games. Informally, our main results are as follows: 1) For any linear base welfare function with no dummy players, the only budget-balanced distribution rules that guarantee equilibrium existence in all single-selection games are weighted Shapley values (Theorem 1). 2) For any base welfare function, when there are only two players, the only budget-balanced distribution rules that guarantee equilibrium existence in all single-selection games are weighted Shapley values (Theorem 2). Before proceeding, we quickly define W l, the class of all linear welfare functions with no dummy players. This is the set of all welfare functions W which satisfy the following two properties: 1) Linearity: ( S N) W (S) = i S W ({i}). 2) No Dummy Player: ( i N) W ({i}) 0. We now state our first result formally. We defer the proof to the Appendix. Theorem 1: Given any base welfare function W W l, all games in G s (N, f W, W ) possess a pure Nash equilibrium for a budget-balanced f W if and only if there exists a weight-vector ω such that f W is the weighted Shapley value distribution rule, fw W SV [ω]. The effect of Theorem 1 is twofold: It provides a complete characterization (the first of its kind) for a large class of welfare functions, for single-selection games. In spirit, it resembles the multiselection result of Proposition 2, except that we only characterize budget-balanced distribution rules, and only for a specific class of welfare functions. A straightforward corollary of this result is that, if one wants to use a distribution rule that is budgetbalanced and guarantees equilibrium existence for all possible welfare functions and single-selection action sets, then one is limited to the class of weighted Shapley value distribution rules. This exactly parallels the multiselection result of Proposition 1, effectively adding the prefix ONLY to the bottom left box of Table II. Our second result provides a complete characterization of budget-balanced distribution rules for any base welfare function, but just for the class of two-player single-selection games (we defer the proof to the Appendix): Theorem 2: For any base welfare function W, all games in G s ({i, j}, f W, W ) possess a pure Nash equilibrium for a budget-balanced f W if and only if there exists a weightvector ω = (ω i, ω j ) such that f W is the weighted Shapley value distribution rule, fw W SV [ω]. The limitation to weighted Shapley values for stable, budget-balanced utility design was first exposed by Chen et al. [14]. Theorems 1 and 2 shed new light on how robust this limitation is to restrictions on the structure of action sets (single/multi-selection). In particular, this limitation continues to hold even if the action sets are restricted to be singleselection, for a large class of welfare functions (Theorem 1). When further restricted to two-player games, the limitation holds for any welfare function (Theorem 2). Future Work:: It might be of interest to explore deeper to find complete characterizations of equilibrium guaranteeing distribution rules for an arbitrary, nonlinear welfare function with single-selection action sets and three or more players (these are classes of games not covered by our two characterization theorems). However, we expect this to be a severely challenging ordeal, because the parallel characterization in the multi-selection scenario is complex enough to begin with [22], starting from which one has to add more distribution rules depending on the specific welfare function at hand. APPENDIX: PROOFS In this appendix, we present the proofs of our main results, Theorems 1 and 2. Both proofs use the same proof technique, one that mirrors those used in the previous multi-selection characterizations by Chen et al. [14], Marden and Wierman [15], and Gopalakrishnan et al. [22]. Before presenting the individual proofs, we present the common details: Note that, for both theorems, we only need to prove one direction, since it is known that for any weight-vector ω, all games in G s (N, fw W SV [ω], W ) have a pure Nash equilibrium [37]. Thus, we present the bulk of the proofs the other direction proving that for distribution rules f W that are not weighted Shapley values, there exists a counterexample game G G s (N, f W, W ) for which no pure Nash equilibrium exists. Recall that specifying a resource allocation game consists of specifying its many parameters; see (3). In searching for a counterexample game, we are restricted to the class G s (N, f W, W ), so the player set N, distribution rule f W, and the welfare function W are fixed. We choose the free parameters (resource set R, player action sets A i, and resource coefficients v r ) cleverly, depending on the welfare function W, and the given distribution rule f W. Proof of Theorem 1: Let the player set N, base welfare function W W l, and distribution rule f W fw W SV [ω] for any weightvector ω be given. Per the proof technique described above, all that we need to do is exhibit a counterexample game G G s (N, f W, W ) for which no equilibrium exists. Before proceeding to present the counterexample, we state a quick and useful lemma: Lemma 2: For any W W l, and any weight-vector ω, ( S N) ( i S) f W W SV [ω](i, S) = W ({i}). The proof of the lemma is a straightforward evaluation the formula (5), using the linearity property of the welfare function W ; we omit the details here. Perhaps remarkably, this lemma states that, for any linear welfare function, the space of all weighted Shapley values collapses to a single, trivial distribution rule every player i gets W ({i}), no matter which other players they are sharing a resource with. We now proceed with the proof of the theorem. Let S N be a smallest (in terms of cardinality) set for which there exists a player i S such that f W (i, S) W ({i}). Such a set must exist, from Lemma 2, and the fact that

7 the given distribution rule f W is not a weighted Shapley value. Without loss of generality, let f W (i, S) < W ({i}). Because f W is budget-balanced, it follows that there is some j S {i} with f W (j, S) > W ({j}). (a) The game fixed players: (b) The payoff matrix,,,,,, Fig. 1: Counterexample 1 Counterexample 1: Consider the game in Figure 1a, with resource set R = {r i, r j, r ij } and local resource coefficients v ri = v i, v rj = v j, and v rij = 1. Player i can choose between the resources r i and r ij, and player j can choose between the resources r j and r ij. All other players in S have a fixed action they choose resource r ij. Formally, { {{r k }, {r ij }}, k {i, j} A k = {{r ij }}, k S {i, j}. This is essentially a game between i and j with payoff matrix in Figure 1b. Note that by assumption, f W distributes W ({i}) and W ({j}) to players i and j respectively, in all strict subsets of S. To complete the specification of Counterexample 1, we need to specify the values of v i > 0 and v j > 0. We now show that these values can be picked carefully in such a way that Counterexample 1 does not possess an equilibrium. Consider each of the four outcomes: 1) In action profile {{r i }, {r j }}, player i will have an incentive to deviate to r ij if v i W ({i}) < W ({i}). 2) In action profile {{r ij }, {r j }}, player j will have an incentive to deviate to r ij if v j W ({j}) < f W (j, S). 3) In action profile {{r ij }, {r ij }}, player i will have an incentive to deviate to r i if v i W ({i}) > f W (i, S). 4) In action profile {{r i }, {r ij }}, player j will have an incentive to deviate to r j if v j W ({j}) > W ({j}). Combining (1) and (3), we get: f W (i, S) < v i W ({i}) < W ({i}). Similarly, combining (2) and (4), we get: W ({j}) < v j W ({j}) < f W (j, S). Since f W (i, S) < W ({i}) and f W (j, S) > W ({j}), it is clear that we can pick v i > 0 and v j > 0 satisfying the above inequalities. This completes the proof. Proof of Theorem 2: Here, the player set N = {i, j}. Let the base welfare function W, and distribution rule f W fw W SV [ω] for any weight-vector ω = (ω i, ω j ) be given. Per the proof technique described above, all that we need to do is exhibit a counterexample game G G s (N, f W, W ) for which no equilibrium exists. We first present the proof by assuming that W is submodular, that is, W ({i, j}) W ({i}) + W ({j}). Later, we explain how the supermodular case is proved. Before proceeding to present the counterexample, we state a quick and useful lemma: Lemma 3: For any submodular W, if a budget-balanced distribution rule f W satisfies: 1) W ({i, j}) W ({j}) f W (i, {i, j}) W ({i}), and 2) W ({i, j}) W ({i}) f W (j, {i, j}) W ({j}), then, there exists a weight-vector ω = (ω i, ω j ) such that f W = fw W SV [ω]. The proof of the lemma is fairly straightforward. The formula (5) for S = {i, j} can be used to obtain an equation in terms of the weights ω i and ω j, positive solutions to which exist if conditions (1) and (2) of the lemma are satisfied. (In fact, for any budget-balanced f W, note that these two conditions imply each other.) We omit the details here due to space constraints. We now proceed with the proof of the theorem. From Lemma 3, budget-balance of f W, and the fact that f W is not a weighted Shapley value, it follows that either of the two conditions must hold: or, f W (i, {i, j}) > W ({i}), and f W (j, {i, j}) < W ({i, j}) W ({i}) W ({j}) f W (j, {i, j}) > W ({j}), and f W (i, {i, j}) < W ({i, j}) W ({j}) W ({i}) (10) (11) Without loss of generality, we assume that (10) holds. (Otherwise, interchange the labels of i and j.) Counterexample 2: Consider the game in Figure 2a, with resource set R = {r 1, r 2 } and local resource coefficients v r1 = v r2 = 1. Players i and j both have the same action set they can choose between the resources r 1 and r 2. Formally, A i = A j = {{r 1 }, {r 2 }}. This symmetric game between i and j has the payoff matrix in Figure 2b. Note that due to budget-balance, f W distributes W ({i}) and W ({j}) to players i and j respectively, when they are by themselves at a resource. In order to show that Counterexample 2 does not possess an equilibrium, we show that none of the four action profiles can be an equilibrium: In action profiles {{r 1 }, {r 1 }} and {{r 2 }, {r 2 }}, player j will deviate, because f(j, {i, j}) < W ({j}), from (10). In action profiles {{r 1 }, {r 2 }} and {{r 2 }, {r 1 }}, player i will deviate, because f(i, {i, j}) > W ({i}), from (10).

8 (a) The game (b) The payoff matrix,,,,,,,, Fig. 2: Counterexample 2 If W is supermodular, the proof is exactly the same, except that all relevant inequalities, including those in Lemma 3 are reversed, and thus the labels of players i and j must be interchanged in Counterexample 2. This completes the proof. REFERENCES [1] A. Garcia, D. Reaume, and R. L. Smith, Fictitious play for finding system optimal routings in dynamic traffic networks, Transport. Res. B-Meth., vol. 34, no. 2, pp , [2] A. Akella, S. Seshan, R. Karp, S. Shenker, and C. Papadimitriou, Selfish behavior and stability of the Internet: A game-theoretic analysis of TCP, in ACM SIGCOMM, 2002, pp [3] C. G. Cassandras and W. Li, Sensor networks and cooperative control, Eur. J. Control, vol. 11, no. 4-5, pp , [4] B. Kauffmann, F. Baccelli, A. Chaintreau, V. Mhatre, K. Papagiannaki, and C. Diot, Measurement-based self organization of interfering wireless access networks, in IEEE INFOCOM, 2007, pp [5] E. Campos-Náñez, E. Garcia, and C. Li, A game-theoretic approach to efficient power management in sensor networks, Oper. Res., vol. 56, no. 3, pp , [6] T. Alpcan, L. Pavel, and N. Stefanovic, A control theoretic approach to noncooperative game design, in CDC, 2009, pp [7] E. Anshelevich, A. Dasgupta, E. Tardos, and T. Wexler, Near-optimal network design with selfish agents, in STOC, 2003, pp [8] V. Gkatzelis, K. Kollias, and T. Roughgarden, Optimal cost-sharing in weighted congestion games, Available at kkollias/optimal.pdf, [9] Y. Zou and K. Chakrabarty, Uncertainty-aware and coverage-oriented deployment for sensor networks, J. Parallel Distr. Com., vol. 64, no. 7, pp , [10] V. Srivastava, J. Neel, A. B. MacKenzie, R. Menon, L. A. Dasilva, J. E. Hicks, J. H. Reed, and R. P. Gilles, Using game theory to analyze wireless ad hoc networks, IEEE Communications Surveys & Tutorials, vol. 7, no. 4, pp , [11] G. Arslan, J. R. Marden, and J. S. Shamma, Autonomous vehicle target assignment: A game theoretical formulation, J. Dyn. Syst.-T. ASME, [12] J. R. Marden and A. Wierman, Overcoming limitations of gametheoretic distributed control, in CDC, 2009, pp [13] R. Gopalakrishnan, J. R. Marden, and A. Wierman, An architectural view of game theoretic control, SIGMETRICS Perf. Eval. Rev., vol. 38, no. 3, pp , [14] H.-L. Chen, T. Roughgarden, and G. Valiant, Designing network protocols for good equilibria, SIAM J. Comput., vol. 39, no. 5, pp , [15] J. R. Marden and A. Wierman, Overcoming the limitations of utility design for multiagent systems, IEEE Trans. Autom. Control, vol. 58, no. 6, pp , [16] L. S. Shapley, Additive and non-additive set functions, Ph.D. dissertation, Department of Mathematics, Princeton University, [17], A value for n-person games, in Contributions to the theory of games. Princeton University Press, 1953, vol. 2, pp [18] D. Monderer and L. S. Shapley, Potential games, Game. Econ. Behav., vol. 14, no. 1, pp , [19] L. E. Blume, The statistical mechanics of strategic interaction, Game. and Econ. Behav., vol. 5, no. 3, pp , [20] J. R. Marden, G. Arslan, and J. S. Shamma, Joint strategy fictitious play with inertia for potential games, Trans. on Autom. Control, vol. 54, no. 2, pp , [21] J. R. Marden and J. S. Shamma, Revisiting log-linear learning: Asynchrony, completeness and payoff-based implementation, Game. Econ. Behav., vol. 75, no. 2, pp , [22] R. Gopalakrishnan, J. R. Marden, and A. Wierman, Potential games are necessary to ensure pure Nash equilibria in cost sharing games, Math. Oper. Res., 2014, to appear. [23] J. R. Marden and A. Wierman, Distributed welfare games, Oper. Res., vol. 61, no. 1, pp , [24] H. Moulin and S. Shenker, Strategyproof sharing of submodular costs: Budget balance versus efficiency, Econ. Theory, vol. 18, no. 3, pp , [25] A. Vetta, Nash equilibria in competitive societies, with applications to facility location, traffic routing and auctions, in FOCS, 2002, pp [26] T. Roughgarden, Intrinsic robustness of the price of anarchy, in STOC, 2009, pp [27] M. Gairing, Covering games: Approximation through noncooperation, in WINE, 2009, pp [28] H. Moulin, An efficient and almost budget balanced cost sharing method, Games and Economic Behavior, vol. 70, no. 1, pp , [29] E. Anshelevich, A. Dasgupta, J. Kleinberg, É. Tardos, T. Wexler, and T. Roughgarden, The price of stability for network design with fair cost allocation, in FOCS, 2004, pp [30] C. Chekuri, J. Chuzhoy, L. Lewin-Eytan, J. Naor, and A. Orda, Noncooperative multicast and facility location games, IEEE J. Sel. Areas Commun., vol. 25, no. 6, pp , [31] S. Adlakha, R. Johari, G. Weintraub, and A. Goldsmith, Oblivious equilibrium: an approximation to large population dynamic games with concave utility, in GameNets, 2009, pp [32] Y. Su and M. van der Schaar, Conjectural equilibrium in multiuser power control games, IEEE Trans. Signal Process., vol. 57, no. 9, pp , [33] J. R. Marden, State based potential games, Automatica, vol. 48, no. 12, pp , [34] J. Corbo and D. C. Parkes, The price of selfish behavior in bilateral network formation, in PODC, 2005, pp [35] A. Fiat, H. Kaplan, M. Levy, S. Olonetsky, and R. Shabo, On the price of stability for designing undirected networks with fair cost allocations, in ICALP, 2006, pp [36] G. Christodoulou, C. Chung, K. Ligett, E. Pyrga, and R. Stee, On the price of stability for undirected network design, in WAOA, 2010, pp [37] S. Hart and A. Mas-Colell, Potential, value, and consistency, Econometrica, vol. 57, no. 3, pp , [38] T. Ui, A shapley value representation of potential games, Game. Econ. Behav., vol. 31, no. 1, pp , [39] T. Matsui and Y. Matsui, A survey of algorithms for calculating power indices of weighted majority games, J. Oper. Res. Soc. Japan, vol. 43, pp , [40] V. Conitzer and T. Sandholm, Computing Shapley values, manipulating value division schemes, and checking core membership in multiissue domains, in AAAI, 2004, pp [41] D. H. Wolpert and K. Tumer, An introduction to collective intelligence, Handbook of Agent technology. AAAI, Tech. Rep., [42] R. A. Murphey, Target-based weapon target assignment problems, in Nonlinear Assignment Problems, ser. Combinatorial Optimization, P. M. Pardalos and L. S. Pitsoulis, Eds. Springer, 2000, vol. 7, pp