Strong Equilibrium in Congestion Games

Transcription

1 Strong Equilibrium in Congestion Games Research Thesis In Partial Fulfillment of The Requirements for the Degree of Master of Science in Computer Science Ola Rozenfeld Submitted to the Senate of the Technion - Israel Institute of Technology Adar, 5767 Haifa February 2007

2

3 The Research Thesis Was Done Under The Supervision of Professor Moshe Tennenholtz in the Faculty of Computer Science. The Generous Financial Help Of Technion Is Gratefully Acknowledged.

4

5 Contents 1 Preliminaries Abstract Symbols and Acronyms Introduction Introduction to Game Theory The Concept of a Mediator Congestion Games Road Map Our Results The Power of Mediators in General Games Games in strategic form Strong equilibrium concepts Information sets Routing mediators The existence of strong mediated equilibrium Strong Equilibrium in Monotone Congestion Games SE: the case of singleton strategies Congestion game forms, bad configurations and tree representations Structural characterization of existence of SE Structural characterization of existence of CSE The Use Of Mediators In Congestion Games Conclusions 48 i

6 Chapter 1 Preliminaries 1.1 Abstract In many multi-agent systems it makes little sense to assume that agents will stick to their parts when provided with suggested protocol, if a deviation can increase their payoffs. While for any multi-agent encounter, represented as a strategic form game, there always exists a strategy profile for which unilateral deviations are not beneficial, requirements regarding joint deviations of multiple players are rarely satisfied. In order to tackle this issue we consider the use of routing mediators, extending upon previous work in game theory and AI. A mediator is a reliable entity that can interact with the players, and potentially act on behalf of the players that give him a permission to do so. However, a mediator can not enforce behavior any agent is free to participate in the game without the help of the mediator. This notion is highly natural; in many systems there is some form of reliable party that can be used as a mediator. In particular, brokers in financial markets, bidding club organizers in auctions, as well as routers in communication networks, can be viewed as mediators. Notice that we assume that the multi-agent interaction (formalized as a game) is given, and all the mediator can do is to perform actions on behalf of the agents that explicitly allow it to do so. The mediator s behavior on behalf of the agents that give it the right of play is pre-specified, and is conditioned on the information available to the mediator on all agents actions. Our study concentrates on the use of routing mediators in order to reach correlated strong equilibrium a multi-agent behavior which is stable against deviations by 1

7 coalitions. We study the relationships between the power of different routing mediators in establishing correlated strong equilibrium. Our main result shows a natural class of routing mediators that allow implementing fair and efficient outcomes as a correlated super-strong equilibrium in a very wide class of games. We then apply this work to the widely studied setting of congestion games. We study the existence of strong and correlated strong equilibria in congestion games without a mediator, and then explore the extent to which routing mediators can aid in attaining these solutions. 2

8 1.2 Symbols and Acronyms R: The set of real numbers P (S): The power set (the set of all subsets) of a set S NE: Nash equilibrium SE: Strong equilibrium 2-SE: 2-Strong equilibrium CSE: Correlated strong equilibrium 2-CSE: Correlated 2-strong equilibrium CSSE: Correlated super strong equilibrium 2-CSSE: Correlated 2-super strong equilibrium i: Players other than i (X): The set of probability distributions with finite support over X 3

9 Chapter 2 Introduction 2.1 Introduction to Game Theory A finite game in strategic form consists of: A finite set of players N. For each player i N a finite non-empty set X i, which is the set of actions (or strategies, we use the terms interchangeably) available to player i. We denote by X = i N X i the set of all possible action profiles. For each player i N there is a utility function u i : X R, which represents the contentment of the player with each specific strategy profile. It is assumed to be common knowledge that all players want to maximize their expected utility. Game theory strives to find solutions for games. Informally, a solution is a reasonable prediction of how the game might be played. In particular, one is interested in a solution (a strategy profile x X) that possesses a certain kind of stability: for every player i, if he knows that all the other players play according to the prescribed solution, then i has no incentive to deviate from it. Formally, we say that a profile of actions x X is a Nash equilibrium (NE) if i N, x i X i : u i (x i, x i) u i (x i, x i) Here by (x i, x i ) we mean the profile in which i plays x i, and players other than i play according to x. One possible motivation for this stability property is: in 4

10 settings where a stable centralized solution can be proposed to all the players, all the players would agree to it, i.e. no one will have a profitable deviation. Other solution concepts have been presented and investigated, some of which have stronger stability properties. In particular, (Aumann, 1959) introduced the notion of strong equilibrium: we say that a profile of actions x X is a strong equilibrium (SE) if for no non-empty coalition S N there is a choice of actions ( x i X i, i S such that i S u i xs, x S) > ui (x ). Here by (x S, x S ) we mean the profile in which the each player in i S playa x i, and other players play according to x. Hence, strong equilibrium extends upon Nash equilibrium by requiring stability against deviations by coalitions. In our research, we focus on strong equilibrium, and even stronger forms of solutions, such as the superstrong equilibrium (where no coalition can deviate so that at least one member strictly increases his profit, and the others don t decrease their profit) as the desired solution concept. Furthermore, we strengthen both concepts even more by allowing the deviating players to use correlated mixed strategies: that is, we assume that a coalition will deviate from the suggested profile even if in any pure deviation some member strictly loses, as long as each member s expected utility increases. We call these concepts correlated strong (super-strong) equilibrium (CSE or CSSE accordingly). Needless to say, strong equilibrium (even in its pure form) is a most desirable solution to search for; however, unlike Nash equilibrium, strong equilibrium exists in very few settings. Super strong equilibrium is even rarer; as to their correlated versions, these are not known to exist except for degenerate cases. 2.2 The Concept of a Mediator One of the major objections to the use of equilibrium concepts as solution concepts in games is that they are based on somewhat problematic assumptions: namely, that the other players reasoning is similar to ours, in the sense that they also assume that the profile being played is the proposed solution. This assumption can be especially problematic if multiple profiles matching the desired criteria exist. As we already mentioned, a popular way to work around this objection is to introduce a commonly known reliable authority into the game; such authority, at the very least, can suggest to the players to use a particular stable solution, thereby making 5

11 it a reasonable prediction of the players chosen strategy. Obviously, if our reliable authority has the power to limit agents strategies in a game, or to influence their utility values (for example, by issuing fines), any action profile can be achieved as a solution. However, in real world systems it is often infeasible (or undesirable) to endow this authority with such power. Instead, we choose to explore situations where the power of this reliable party is much more limited. A mediator is a reliable entity that can interact with the players, and potentially act on behalf of the players that give him a permission to do so. However, a mediator can not enforce behavior. Indeed, an agent is free to participate in the game without the help of the mediator. This notion is highly natural; in many systems there is some form of reliable party that can be used as a mediator. In particular, brokers in financial markets, bidding club organizers in auctions, as well as routers in communication networks, can be viewed as mediators. The mediator s behavior on behalf of the agents that give it the right of play is pre-specified, and is conditioned on the information available to the mediator on all agents actions. Notice that we assume that the multi-agent interaction (formalized as a game) is given, and all the mediator can do is to perform actions on behalf of the agents that explicitly allow it to do so. This natural setting is different from the one discussed in the theory of mechanism design (see (Mas-Colell et al., 1995; Jackson, 2001) for an introduction) where a designer designs a new game from scratch in order to yield some desired behavior. The simplest form of a mediator discussed in the game theory literature is captured by the notion of correlated equilibrium (Aumann, 1974). In this setting, a mediator can draw a strategy profile from a commonly known distribution and then advise each agent (in private) to play by this profile. Equilibria that arise in this setting where all agents play what the mediator advises are known as correlated equilibria; in many settings much better outcomes (to all players) can be achieved with correlated equilibria than with any NE. This notion was generalized to communication equilibrium by (Forges, 1986; Myerson, 1986). In another setting, presented in (Monderer and Tennenholtz, 2004), the mediator can commit to non-negative monetary transfers to the players, based on the different strategy profiles that they choose. Obviously, in this setting the mediator can implement any outcome he wants, by investing enough money, assuming 6

12 minimal rationality of the players, i.e. the desired strategy profile will be the only non-dominated one; therefore, the interesting question is: when can the mediator pay very little, or nothing at all, while still the desired strategy profile will be the only reasonable one. However, in all these settings the mediator cannot act on behalf of the agents. A more powerful type of mediators, which can perform actions on behalf of the agents who give the mediator the right of play is considered in (Monderer and Tennenholtz, 2006). This type of mediators turns out to be a specific case of the mediators discussed in our work. Namely, in this restricted setting the agents need to decide whether to give the mediator the right of play, while the mediator does not possess any further information on the agents actions. In order to illustrate the power of a reliable mediator consider the famous prisoners dilemma game: C D C 4,4 0,6 D 6,0 1,1 In this classical example we get that in the only equilibrium both agents will play D, yielding both of them a payoff of 1. However, this equilibrium, which is also a dominant strategy equilibrium, is inefficient; indeed, if both agents deviate from D to C then both of them will improve their payoffs. Formally, (D, D) is not a strong equilibrium; it is not stable against deviations by coalitions. Indeed, there is no strong equilibrium in the prisoners dilemma game. Consider now a reliable mediator who offers the agents the following protocol: if both agents agree to use the mediator services then he will play C on behalf of both agents. However, if only one agent agrees to use his services then he will play D on behalf of that agent. Notice that when accepting the mediator s offer the agent is committed to actual behavior as determined by the above protocol. However, there is no way to enforce the agents to accept the suggested protocol, and each agent is free to play C or D without using the mediator s services. Hence, the mediator s protocol generates a new game, which we call a mediated game, where a new strategy M is available for using the mediator services: 7

13 C D M C 4,4 0,6 0,6 D 6,0 1,1 1,1 M 6,0 1,1 4,4 The mediated game has a most desirable property: in this game there is a strong equilibrium; that is, an equilibrium which is stable against deviations by coalitions. In this equilibrium both agents will use the mediator services, which will lead them to a payoff of 4 each! Although the above example illustrates the potential power of a mediator who can act on behalf of agents who give him the right of play, the above mediator was very restricted. In general, a routing mediator possesses information about the actions taken by the agents. Consider a router in a typical communication network. Assume that messages submitted to the system must pass through this router. In addition, the router can suggest a protocol to the agents who may wish to use its services also in selecting appropriate routes. Hence, in this situation it is most natural to assume that the router can observe selected actions of all agents, and not only of those who give it the right of play. In order to illustrate the power of such routing mediator consider the following example: A B A 6,6 0,7 B 0,7 6,6 Given the above game, assume we are interested in obtaining a socially optimal outcome, in which each agent gets a payoff of 6. It is easy to see that the column player can guarantee himself an expected payoff of 6.5 by playing A or B with probability 0.5 each. As a result, a routing mediator who is not informed about the pure realizations of the column player s strategies cannot implement any outcome in which that player gets a payoff less than 6.5. However, a routing mediator who can see the pure realizations of the deviating player and choose the punishment appropriately can enforce the outcome (6, 6) as a strong equilibrium! This can be achieved, for example, by the following protocol: If only agent i selects the mediator s services, the mediator will copy the action of the other player, j, on behalf of i; if both agents select to use the mediator s services then the mediator will perform B on behalf of both agents. 8

14 This example illustrates the important role of information in the definition of a routing mediator. Indeed, routing mediators can be classified based on the information they can access about the agents actions. We introduce routing mediators, a general tool for coordinating and influencing agents behavior in games. A routing mediator is always assumed to be reliable, and he is endowed with a single ability: to play for the players who explicitly give him the right to play for them. The mediator cannot enforce the players to use his services, neither can he affect the players utility values. However, routing mediators differ in the information they have on the strategy profile which is being chosen by the players. We distinguish between two types of available information: 1. Information about the actual action instantiations. 2. Information about the agents programs (i.e. correlated strategies). As we will show, having access to the agents actions may sometimes be more powerful than having access to their programs, and vice versa. Given the general setting of routing mediators, we prove that they can indeed significantly increase the set of desired outcomes that can be obtained by multiagent behaviors which are robust against deviations by coalitions. Namely, we show that any minimally fair game possesses a fair and efficient correlated superstrong mediated equilibrium. In a correlated super-strong equilibrium, correlated deviations by any subset of agents can not benefit one of them without hurting another. A minimally fair game is a game in which all actions are available to all agents, and agents who select identical actions get identical payoffs. A particular example of minimally fair games are the symmetric games, but the family of minimally fair games is much wider. In particular, our results are applicable to congestion games and job scheduling, two major topics of study in the interface between computer science and game theory. 2.3 Congestion Games A congestion game (Rosenthal, 1973) is defined as follows: A finite set of players, N = {1,..., n}. A finite non-empty set of facilities, M. 9

15 For each player i N a non-empty set A i P (M), which is the set of actions available to player i (an action now is a subset of facilities). With every facility m M and integer number 1 k n a real number v m (k) is associated, having the following interpretation: v m (k) is the utility to each user of m when the total number of users of m is k. Let a A; the ( M dimensional) congestion vector corresponding to a is σ (a) = (σ m (a)) m M where σ m (a) = {i m a i }. The utility function of player i, u i : A R, is then defined as follows: u i (a) = v m (σ m (a)) m a i Congestion games are a well-investigated class of games with many practical applications. They have been used to model situations such as rush hour traffic (the players are drivers, the facilities are road segments), demand for factors of production (the players are producers, the facilities are factors of production), and more (see e.g. (Anshelevich et al., 2004; Roughgarden and Tardos, 2002; Roughgarden, 2005) for work in applying a game-theoretic approach to congestion settings). The goal of this thesis was to apply mediators to congestion games and see how they can aid in achieving stable solutions (namely, strong and correlated strong equilibrium). Therefore, our first concern was to investigate these solutions in congestion games without a mediator, to provide us with a baseline for improvement. The remainder of this section is dedicated to this subgoal below, we list our methods and results in characterizing the existence of strong and correlated strong equilibria in monotone congestion games. We call a congestion game monotone-increasing (or simply increasing) if m M, 1 k < n v m (k) v m (k + 1) These games model settings where congestion has a positive effect on the players, e.g. settings in which the cost of using a facility is shared between its users. We call a congestion game monotone-decreasing (or simply decreasing) if m M, 1 k < n v m (k) v m (k + 1) 10

16 These games model settings where congestion has a negative effect on the players, e.g. routing games, where cost represents latency. (Holzman and Law-Yone, 1997b; 1997a) explored the conditions for existence of strong equilibria in monotone decreasing congestion games. They start by observing that a strong equilibrium always exists in the case where all strategies are singletons. Following that, they explore the structural properties of the strategy sets that are necessary and sufficient to guarantee the existence of strong equilibria. These structural properties may, for example, refer to the underlying graph structure in route selection games. In this work we first explore the conditions for existence of strong equilibria in monotone increasing congestion games. Then, we extend the study of both the decreasing and increasing settings to the solution concept of correlated strong equilibrium. Our contributions can therefore be described by the following table: Pure deviations Correlated deviations Increasing This work This work Decreasing Holzman & Law-Yone This work Throughout this work, when we refer to strong equilibrium, we present the results of (Holzman and Law-Yone, 1997b) for the decreasing setting alongside our results for the increasing setting. This is done for the sake of viewing the complete picture and ease of comparing between the two settings. In section we explore the case of singleton strategies, i.e. resource selection games where each player should choose a single resource from a set of resources available to him. In the decreasing setting Holzman and Law-Yone observe that every Nash equilibrium of the game is, in fact, a strong equilibrium. For the increasing setting, we present an efficient algorithm for constructing a strong equilibrium; however, unlike in the decreasing setting, we show that not every Nash equilibrium of the game is strong. In section we develop a notation, congestion game forms, that allows us to speak about the underlying structure of congestion games; using this notation we will be able to formalize statements such as a certain structural property is necessary and sufficient for the existence of SE in all games with that underlying structure. We define two substructures, which we call d-bad configuration and i-bad configuration and prove some simple properties of strategy spaces that avoid 11

17 bad configurations. These properties will serve as a technical tool in some of our proofs. Section explores the conditions for existence of strong equilibrium. In the decreasing setting, (Holzman and Law-Yone, 1997b) shows that a SE always exists if and only if d-bad configurations are avoided. In the increasing setting we show that a SE always exists if and only if i-bad configurations are avoided, in which case the equilibrium can be efficiently computed. As we will show, our results imply that avoiding i-bad configurations makes the games essentially isomorphic to the case of singleton strategies. Section deals with the concept of correlated strong equilibrium. We show that a CSE might not be achievable even in simple (two players, two strategies) examples of the decreasing setting. In the increasing setting, though, we show that all our results regarding SE still hold with CSE, namely that a CSE always exists if and only if i-bad configurations are avoided, in which case it can be efficiently computed. Moreover, we show that in this case every SE of the game is also a CSE (a claim which doesn t hold if i-bad configurations are not avoided). Together, our results provide full characterization for the connection between the underlying game structure and the existence of SE and CSE for both the decreasing and the increasing cases. 2.4 Road Map The remainder of this work is organized as follows: In section 3.1 we present our results concerning routing mediators in general games. Section 3.2 deals with congestion games (without mediators) and presents our results on characterization of existence of SE and CSE. Section 3.3 explores the extent to which routing mediators can aid in achieving SE and CSE in congestion games, beyond the results of section 3.1. Finally, we conclude with a general discussion of the results and the future work. 12

18 Chapter 3 Our Results 3.1 The Power of Mediators in General Games Games in strategic form Recall that a game in strategic form is a tuple Γ = N, (X i ) i N, (u i ) i N, where N is a finite set of players. X i is the strategy set of player i. u i : X R is the payoff function for player i, where X = i N X i. If N = n, whenever convenient we assume N = {1,..., n}. Γ is finite if the strategy sets are finite. For every S N we denote X S = i S X i. When the set N is clear, X N\S will be also denoted by X S, and moreover, X {i} will be also denoted by X i. When we discuss subsets of a given set we implicitly assume non-emptiness unless we specify otherwise. Let Y be a set. The set of probability distributions with finite support over Y is denoted by (Y ). That is, every c (Y ) is a function c : Y [0, 1] such that {y Y c(y) > 0} is finite, and y Y c(y) = 1. For every y Y we denote by δ y the probability distribution that assigns probability 1 to y. Let I be a finite set, and let c i (Y i ), i I, where I is a finite set of indexes, and Y i is a set for every i I. We denote by i I c i the product probability distribution on Y = i I Y i that assigns to every y Y the probability i I c i(y i ). 13

19 Let Γ = N, (X i ) i N, (u i ) i N be a game in strategic form, and let S N. Every c (X S ) is called a correlated strategy for S. A correlated strategy for the set of all players N is also called a correlated strategy, and for every i, a correlated strategy for {i} is also called a mixed strategy for i. Let c (X) and let S N, the marginal probability induced by c on X S is denoted by c [S]. That is, c [s] (x S ) = x S X S c(x S, x S ) The expected payoff of i with respect to a correlated strategy c is denoted by U i (c). That is, U i (c) = u i (x)c(x) x X For every i, and for every x i X i, the mixed strategy δ xi is called a pure strategy Strong equilibrium concepts Let Γ = N, (X i ) i N, (u i ) i N be a game in strategic form, and let x X. Definition 1. We say that x is a pure strong equilibrium (Aumann, 1959) in Γ if for every subset, S of players and for every y S X S there exists i S such that u i (y S, x s ) u i (x). Definition 2. Let c (X). We say that c is a correlated strong equilibrium, if for every subset of players S, and for every correlated strategy for S, ξ S (X S ) there exists i S such that U i (ξ S c [ S] ) U i (c). The notion of pure strong equilibrium is the one typically studied in the game theory literature when considering deviations by coalitions. The concept of correlated strong equilibrium considers also correlated deviations. Of course, a pure strong equilibrium is not necessarily a correlated strong equilibrium. Correlated strong equilibrium is a very natural requirement. In an environment in which the players fear/hope that correlated deviations are possible, we expect a strategy profile to be a correlated strong equilibrium in order to be believed/played by the players. In the scope of this work, we will consider such correlation possible, and therefore when we use the term strong equilibrium, we refer to correlated strong equilibrium. 14

20 The above definitions assumed that a player would agree to join a deviating coalition only if the deviation would result in a strict improvement of his payoff. A much stronger stability concept is needed if we fear that players may be persuaded to join a deviating coalition also in cases where such deviation would leave their payoff unchanged, but would increase the payoff of at least one of the other coalition members. Definition 3. We say that c (X) is a correlated super-strong equilibrium (or super-strong equilibrium) if there is no S N and ξ S (X S ), such that: for all ( ) i S, U i ξs, c [ S] Ui (c), and for at least one i S the inequality is strict. A super-strong equilibrium is, in particular, a strong equilibrium, but not vice versa Information sets Definition 4. Let Y be a set. We say that Ω is an information set on Y if Ω is a partition of Y. Let y Y. By ω(y) we refer to the equivalence class of y in Ω, i.e. to P Ω s.t. y P. We use these definitions in order to model situations in which an element y Y is chosen, and an agent has only partial information about the chosen element. It is assumed that, if Ω is the information set that is available to such agent, the agent is informed of ω(y). Definition 5. Let Ω 1, Ω 2 be information sets on Y. We say that Ω 1 refines Ω 2 if for all y 1, y 2 Y if ω 1 (y 1 ) = ω 1 (y 2 ) then ω 2 (y 1 ) = ω 2 (y 2 ). That is, an agent with available information Ω 1 is informed at least as well as an agent with Ω 2. The refinement relation is a partial ordering of the information sets, which forms a lattice: the maximal possible information on Y is Ω full = {{y} y Y }; having such information means that for any choice of y Y, the agent knows exactly which element was chosen. The minimal possible information on Y is Ω = {Y }; that means that the agent has no information at all about any chosen element y Y. 15

21 3.1.4 Routing mediators We now introduce routing mediators, a general tool for coordinating and influencing agents behavior in games. A routing mediator is always assumed to be reliable, and he is endowed with a single ability: to play for the players who explicitly give him the right to play for them. The mediator cannot enforce the players to use his services, neither can he affect the players utility values. However, routing mediators differ in the information they have on the strategy profile which is being chosen by the players. Let Γ be a game in strategic form. A routing mediator for Γ is a tuple m, Ω, Ψ, (c ω,ψ ) ω Ω,ψ Ψ, where the following holds: m / X i for all i N. m denotes the new strategy that is now available to each player: to send a message to the mediator, indicating that the player agrees to give the mediator the right of play for him. 1 Ω is an information set on Z, where Z = i N Z i, and Z i = X i {m}. Ω represents the information available to the mediator about each pure realization of the players correlated strategy. Ω refines Ω basic, which is defined as follows: Definition 6. Given z Z, let T (z) denote {j N z j = m}. That is, T (z) denotes the players who agree to give the mediator the right of play for them in z. Then, Ω basic is the partition induced by the following equivalence relation on Z: z 1 z 2 iff T (z 1 ) = T (z 2 ) (For ω Ω we will use T (ω) to denote T (z) for some z ω). That is, we demand that in each pure realization of the agents strategy the mediator knows exactly which players give him the right of play for them. Ψ is an information set on (Z), which represents the information available to the mediator about the correlated strategy of the players. We place no restrictions on Ψ, in particular Ψ may equal Ψ basic = { (Z)}, meaning that the mediator has no such information. 1 (Monderer and Tennenholtz, 2006) show that no power is added if the communication between the mediator and the players includes more than one message type. 16

22 For every ω Ω, ψ Ψ, c ω,ψ ( ) X T (ω). That is, c is the conditional contract that is offered by the mediator: it specifies exactly which actions the mediator will perform on behalf of the players who agree to use his services, as a function of the information available to it about the strategy profile chosen by the agents. Every mediator M for Γ induces a new game Γ(M) in strategic form in which the strategy set of player i is Z i. The payoff function of i is defined for every c (Z) as follows: U M i (c) = c(z)u i (c ω(z),ψ(c), z [ T (z)] ) z Z The above definitions express the following interaction between the mediator and the players: 1. The players choose a profile c (Z) 2. A joint coin flip is issued, determining a pure realization z Z of c 3. The mediator is informed of ω(z) and ψ(c) 4. The mediator issues a coin flip to determine a pure realization δ T (z) of c ω(z),ψ(c) ( ) 5. The payoff of each player i is given according to u i δt (z), z [ T (z)] The payoffs of the players from the profile c are assumed to be their expected payoffs from both coin flips in steps 2,4. The mediator is guaranteed to behave in a pre-specified way, however this behavior is conditional on the actions of the other agents. For this reason, although U i (c) is well defined for any profile c (Z), in particular for pure profiles z Z, U i (c) does not necessarily equal z Z U i(z)c(z), unless Ψ = Ψ basic. Nevertheless, the concepts of strong equilibrium can still be applied to Γ(M). Let Γ be a game in strategic form, m / i N X i, Ω an information set on Z, Ψ an information set on (Z). Let m N Z denote the pure strategy profile i N m. A correlated strategy c (X) is a strong mediated equilibrium relatively to Ω, Ψ if there exists a mediator for Γ, M = m, Ω, Ψ, (c ω,ψ ) ω Ω,ψ Ψ, with c ω(m N ),ψ(m N ) = c, for which m N is a correlated strong equilibrium in Γ(M). The notion of super-strong mediated equilibrium can be defined similarly. 17

23 Note that the above definition coincides with the one of (Monderer and Tennenholtz, 2006) in the case where Ω = Ω basic, Ψ = Ψ basic ; i.e. they considered the case in which the mediator is only informed of the players that chose to delegate him the right to play for them in step 2; he has no information about the actions of the other players, neither about the correlated strategy that was chosen in step 1. Situations which can be modeled by Ψ = Ψ full = {{c} c (Z)} are situations in which the mediator either is informed about the correlated strategy of the players, or doesn t decide/reveal its action before the players choose theirs. For example, in a computerized setting, where the correlated deviation is implemented using a program which is submitted and executed on a server, and the mediator is located on this server, the mediator is fully informed about the correlated strategy of the players. In another example, even if the mediator is not informed about correlated deviations, the players might fear that such information leak is possible. That way, the players may consider a worst case behavior of the mediator, and therefore would agree to deviate only if the mediator can t possibly harm any one of them. Ω Ω basic can be used to model all the situations in which the mediator does indeed, as the name implies, stand between the players and the actual game, e.g. situations in which the play is only available through the mediator. For example, if the players strategies are messages in a network and the mediator sits on the router, it might be reasonable to assume that Ω = Ω full. Lastly, we can imagine many situations in which the mediator is only partially informed, both in terms of Ω and of Ψ. For example, if the mediator is a router in a communication network, he may be informed only of the messages that are passed through him and not through other routers; another example is when a mediator is only informed about strategies of particular players. Observations: Suppose c (X) is a strong (super-strong) mediated equilibrium relatively to Ω, Ψ. 1. Let w(c) = (w 1,..., w n ) be the payoff vector induced by c. Then, for any c (X) such that w(c ) = w(c), c is a strong (resp. super-strong) mediated equilibrium relatively to Ω, Ψ. In light of this, we will sometimes use the following notation: a payoff vector w can be implemented as a strong (super-strong) mediated equilibrium relatively to Ω, Ψ if there exists c 18

24 (X) such that w(c) = w and c is a strong (resp. super-strong) mediated equilibrium relatively to Ω, Ψ. 2. If Ψ = Ψ full, c can be implemented as a strong (resp. super-strong) mediated equilibrium relatively to Ω, Ψ even if the mediator is restricted to use only pure actions (i.e. satisfies c ω,ψ X T (ω) ). 3. If Ω refines Ω, then c is also a strong mediated equilibrium relatively to Ω, Ψ 4. If Ψ refines Ψ, then c is also a strong mediated equilibrium relatively to Ω, Ψ Observations 3, 4 show that the possible values of Ω, Ψ induce a four-end lattice of strong mediated equilibrium concepts, where the strongest (most stable) concept is obtained from Ω basic, Ψ basic (since it assumes nothing on the information available to the mediator), and the weakest concept is obtained from Ω full, Ψ full. Part of this lattice is depicted below (arrows mean increase in stability): Ω basic, Ψ basic Ω basic, Ψ full Ω full, Ψ basic Ω full, Ψ full A natural question to ask is whether there exists a general relationship between the strong mediated equilibria concepts induced by Ω basic, Ψ full and Ω full, Ψ basic. The question is, what empowers the mediator more increasing the information available to him on each pure outcome, or increasing the information about the correlated strategies? As it turns out, this question does not have a general answer: 19

25 Proposition 1. There exists a game Γ and a strategy profile c which is a strong mediated equilibrium relatively to Ω full, Ψ basic, but not a strong mediated equilibrium relatively to Ω basic, Ψ full. An example of such game and such profile appears in the introduction. A less intuitive result is the following: Proposition 2. There exists a game Γ and a strategy profile c which is a strong mediated equilibrium relatively to Ω basic, Ψ full, but not a strong mediated equilibrium relatively to Ω full, Ψ basic. A B C A B C A 12,0,0 0,0,0 0,0,0 P layer3 A 24,-12,0 0,0,0 0,0,0 B 0,0,0 0,12,0 0,0,0 B 0,0,0 0,12,0 0,0,0 C 0,0,0 0,0,0 5,5,5 C 0,0,0 0,0,0 0,0,0 Proof. Let Γ be the 3-person game depicted above, and let c = (C, C, A). In order to see that c is not a strong mediated equilibrium in Γ relatively to Ω full, Ψ basic, consider the possible actions of the mediator on behalf of player 3 given the profile (A, A, m) Z. The mediator has to choose a strategy on behalf of player 3 without knowing the correlated strategy of players {1, 2} that caused the realization (A, A). No matter what distribution between A and B will the mediator choose, the total payoff of the deviators will be 12, with player 1 getting more than player 2; note that the pure profile (B, B) guarantees the deviators the payoffs (0, 12); therefore, no matter what strategy the mediator will choose, players {1, 2} will be able to randomize between (A, A) and (B, B) so that each of them will get a payoff of 6, making the deviation beneficial for them. However, note that the players 1, 2 were able to guarantee themselves the payoffs (6, 6) as a response to any contract offered by the mediator. In a sense, the information set Ψ basic means that the players are more informed than the mediator the strategy of the mediator in this setting is fixed, and cannot depend on the correlated strategy chosen by the players; while their correlated strategy can be a best response to the mediator s strategy. When the information set Ψ full is considered, the mediator becomes more informed than the players: he can devise a response based on their correlated strategy. In this example, a mediator who is aware of the exact probabilities that the players 1, 2 assign to the profiles (A, A) and (B, B) can 20

26 always respond by a pure strategy A or B on behalf of the third player in a way that will cause one of the players to get a payoff less than 5, making the deviation not beneficial for them The existence of strong mediated equilibrium In this section we present our main result. We show a natural class of routing mediators that allow to implement fair and efficient outcomes as a correlated superstrong equilibrium in a very wide class of games. Some definitions are needed: Definition 7. A permutation of the set of players is a one-to-one function from N to N. For every permutation π and for every action profile x X we denote by πx the permutation of x by π. That is, (πx) πi = x i for every player i N. Definition 8. Let Γ be a game in strategic form. Γ is a symmetric game if X i = X j for all i, j N and u i (x) = u π(i) (πx) for every player i, for every action profile x X and for every permutation π. Needless to say that symmetric games are most popular in computerized settings. For example, the extremely rich literature on congestion games in computer science deals with particular form of symmetric games (Papadimitriou, 2001). However, our main result requires a weaker notion of symmetry: Definition 9. Let Γ be a game in strategic form. Γ is a minimally fair game if for all i, j N, X i = X j and for every action profile x X, x i = x j implies that u i (x) = u j (x). That is, a game is minimally fair if players who play the same strategy get the same payoff. The exact value of the received payoff may depend on the identities of the players who chose the strategy, as well as on the rest of the profile. In particular, every symmetric game is a minimally fair game; however, minimally fair games capture a much wider class of settings. For example, typical job scheduling games are minimally fair games. Note that in a minimally fair game, unlike in a symmetric game, the symmetric socially optimal payoff is not necessarily feasible. In order to capture our notion 21

27 of fair and efficient outcomes in such games, we need to define the notion of lexicographic ordering on vectors of payoffs: Definition 10. Let sort : R n R n be a function that sorts the elements of real vectors in increasing order. Given two vectors w, w R n we say that w l w (w is lexicographically smaller than w ) if there exists 1 i n such that sort(w) i < sort(w ) i and for all 1 j < i sort(w) j = sort(w ) j. Definition 11. Let w, w R n be two feasible payoff vectors in Γ. We say that w Pareto dominates w if w w and i N w i w i. We say that w is Pareto optimal if there is no feasible w in Γ such that w Pareto dominates w. Definition 12. Let Γ be a minimally fair game. We say that a vector w R n is the max-min fair payoff of Γ if w is the lexicographically maximal feasible payoff vector in Γ. It is easy to see that the max-min fair payoff vector always exists, and is unique. Also, such vector is, by definition, Pareto optimal. The fairness criterion we use is in fact the well-known max-min fairness criterion, as discussed in the communication networks literature (see e.g. (Kumar and Kleinberg, 2000)). We are now ready to state our main result: Theorem 1. Let Γ be a minimally fair game. Then, the max-min fair payoff of Γ can be implemented as a super-strong mediated equilibrium relatively to Ω full, Ψ basic. Proof. Let w be the max-min fair payoff vector of Γ. Suppose w.l.o.g. that the vector w is sorted in increasing order and the players in N are renamed appropriately. Let m / i N X i. We define a routing mediator M = m, Ω full, Ψ basic, (c ω,ψ ) ω Ωfull,ψ Ψ basic as follows: Ψ basic contains a single element, therefore it suffices to define c ω(z) for all z Z. Let z Z. If T (z) = N, we define c ω(z) to be the strategy profile that implements w. Otherwise, let T = T (z), let i = max {N \ T } (i.e. i is the highest indexed player among those who don t cooperate with the mediator in z), and let a = z i. We then define c ω(z) = i T δ a. 22

28 That is, the action of the mediator for each pure realization of the correlated profile is to have all the cooperating players copy the action of the deviator with the highest payoff in w. Let S N, ξ S (Z S ). Let ξ = ξ S ( i N\S δ m ) be the profile in Γ (M) that is obtained from the deviation of S to ξ S. Suppose, for contradiction, that S weakly benefits from deviating to ξ S in Γ (M) relatively to m N ; that is, i S U i (ξ) w i and i S U i (ξ) > w i. Let k be the highest indexed player in S. In every pure realization z of ξ, all the players in N \ S play the same action as k: either because k T (z), or because k is, in particular, the highest indexed player in N \ T (z). Therefore, U j (ξ) = U k (ξ) for every player j N \S; therefore, for every j {1,..., k}\s, U j (ξ) w k w j, which means that all the players in S = {1,..., k} can deviate to the correlated profile ξ, in which the players in S play by ξ and the players in S \ S copy the strategy of player k, and S would also weakly benefit from this deviation. Therefore, we can assume w.l.o.g. that S = {1,..., k} for some 1 k < n. Let w be the payoff vector that is induced by ξ. We know that for every k j n, w j = w k w k. Since w is Pareto optimal, we know that k < n (otherwise, w would Pareto dominate w); w.l.o.g. we can assume that w k < w k+1. where We define a payoff vector w as follows: { w w if max i S {w i = } w k εw + (1 ε) w otherwise ε arg min i:w i >w k { w k+1 w i w i w i + w k+1 w k+1 First, we note that (w 1,..., w k ) Pareto dominates (w 1,..., w k ): if w = w this follows from the fact that S weakly benefits from the deviation that results in w ; in the other case, the same holds since ε > 0. Next, we note that the choice of ε ensures that w i w j for j k, i > k; combined with our previous observation, this implies that w l w, contradicting our choice of w. } Below we present two additional results, that are proven using similar logic as behind the proof of Theorem 1. The proofs of these results may therefore be 23

29 skipped if convenient. Both results attempt to relax the requirements on the information set Ω, at the price of restricting attention to the class of symmetric games (Theorem 2) or considering strong instead of super-strong equilibrium (Theorem 3). The relaxation of the requirements on Ω is based on the idea that in order to punish a set of deviating agents, we don t have to see the actions of all of them; it suffices, under some conditions, to observe only the action selected by a single deviator. Formally, Ω must satisfy the following: there exists an ordering < P on N such that for all z, z Z if z max<p {N\T (z)} z max <P {N\T (z)} then ω (z) ω (z ). That is, Ω allows to distinguish between different actions of the highest ranked player (by < P ) among the deviators. Theorem 2. Let Γ be a symmetric game, and suppose than an information set Ω on Z satisfies the above property. Then, the symmetric optimal payoff vector w = (V,..., V ) in Γ can be implemented as a super-strong mediated equilibrium relatively to Ω, Ψ basic. Proof. Let w be the symmetric optimal payoff vector of Γ. Let m / i N X i. We define a routing mediator M = m, Ω, Ψ basic, (c ω,ψ ) ω Ω,ψ Ψbasic as follows: Ψ basic contains a single element, therefore it suffices to define c ω for all ω Ω. If T (ω) = N, we define c ω to be the strategy profile that implements w. Otherwise, let T = T (ω), let i = max <P {N \ T } (i.e. i is the highest ranked player by < P among those who don t cooperate with the mediator), let z ω and let a = z i. By our assumption on < P, a is well defined. We then define c ω = i T δ a. That is, the action of the mediator for each pure realization of the correlated profile is to have all the cooperating players copy the (known) action of the highest ranked deviator according to < P. Let S N, ξ S (Z S ). Let ξ = ξ S ( ) i N\S δ m be the profile in Γ (M) that is obtained from the deviation of S to ξ S. Suppose, for contradiction, that S weakly benefits from deviating to ξ S in Γ (M) relatively to m N ; that is, i S U i (ξ) w i and i S U i (ξ) > w i. Let i be the highest indexed player by < P in S. In every pure realization z of ξ, all the players in N \ S play the same action as i: either because i T (z), or 24

30 because i is, in particular, the highest indexed player by < P in N\T (z). Therefore, U j (ξ) = U i (ξ) for every player j N \ S. But then U j (ξ) V for every j N \ S; since S weakly benefits from the deviation, we also have U j (ξ) V for every j S, with at least one strict inequality; therefore we have that ξ Pareto dominates c N, contradicting our choice of w. Theorem 3. Let Γ be a minimally fair game, and suppose than an information set Ω on Z satisfies the above property. Then, the max-min fair payoff of Γ can be implemented as a strong mediated equilibrium relatively to Ω, Ψ basic. Proof. Let w be the max-min fair payoff vector of Γ. Let m / i N X i, and let M = m, Ω, Ψ basic, (c ω,ψ ) ω Ω,ψ Ψbasic be a routing mediator defined in the proof of Thm. 2. Note, that this mediator offers a contract by which the cooperating players copy the action of the highest ranked deviator (by < P ); this deviator is not necessarily the one with the lowest payoff according to w (unlike in Thm. 1). However, recall that unlike in Thm. 1 we only require strong (and not super-strong) equilibrium. Therefore, if we suppose for contradiction that a subset S N can benefit from a deviation to ξ S and follow the logic of the proof of Thm. 1, we still obtain that the lowest payoff in the profile ξ that results from the deviation is strictly higher than in w, which contradicts the choice of w. 3.2 Strong Equilibrium in Monotone Congestion Games In this section we consider SE and CSE in congestion games. In the following section (3.3) we will revisit routing mediators and their application to implementing SE and CSE in congestion games SE: the case of singleton strategies Here we investigate the case in which only singleton strategies are allowed, i.e. resource selection games where each player should choose a facility from among a set of facilities available to him. 2 2 In particular, this classical setting can model simple route selection games. In a simple route selection game each player has to select a link for reaching from source to target in a graph consisting of several parallel links. In general, each player may have a different subset of the links that he may 25

31 First, recall the result for the decreasing case: Theorem 4. (Holzman and Law-Yone, 1997b) Let G be a monotone decreasing congestion game in which all strategies are singletons. Then G possesses a strong equilibrium; moreover, every Nash equilibrium of G is SE. We now address the existence of SE in monotone-increasing congestion games: Theorem 5. Let G be a monotone increasing congestion game in which all strategies are singletons. Then G possesses a strong equilibrium; moreover, a SE can be efficiently computed. Proof. Consider the following algorithm for computing a strong equilibrium: at each step, we assign a facility to a non empty subset of the remaining players, in the following way: for each facility m M, we compute v m (k), where k is the maximal number of the remaining players that can choose {m} as their strategy. We choose m for which such v m (k) is maximal, and assign {m} to all the players that can choose it. We continue in the same fashion until all players are assigned a facility. We claim that the resulting strategy profile is a SE. We prove by induction on the steps of the algorithm, that no player can belong to a deviating coalition in which his payoff strictly increases: in the first step of the algorithm, this is obvious because each assigned player gets the highest possible payoff in the game (due to monotonicity); at subsequent steps, we use the induction hypothesis and assume that all players from the previous steps don t belong to the deviating coalition, i.e. all of them use the facilities they were assigned; but this means that the game is effectively reduced to the remaining players and the remaining facilities, so the same reasoning applies: due to monotonicity, each assigned player gets the highest possible payoff in the (new) game. Regarding the complexity of the algorithm, it is trivial to verify that the most straightforward implementation runs in O(m 2 n 2 ); it is also a simple exercise to construct an implementation that runs in O(mn). Unlike in the decreasing setting, not every Nash equilibrium (NE) of the decreasing game is a SE. Consider, for example, an instance with two facilities {m 1, m 2 } and two players, where the cost of a facility is shared equally between use. 26