The Distribution of Optimal Strategies in Symmetric Zero-sum Games

Transcription

1 The Distribution of Optimal Strategies in Symmetric Zero-sum Games Florian Brandl Technische Universität München Given a skew-symmetric matrix, the corresponding two-player symmetric zero-sum game is defined as follows: one player, the row player, picks a row and the other player, the column player, picks a column. The payoff of the column player is given by the corresponding matrix entry, the row player receives the negative of the column player. A randomized strategy is optimal for the column player if it guarantees an expected payoff of at least 0 independently of the strategy of the row player. We determine the probability that an optimal strategy for the column player randomizes over exactly k columns when the payoffs of the game are i.i.d. random variables that are symmetric around 0 and the game almost surely admits a unique optimal strategy. Our result has consequences for subclasses of symmetric zero-sum games and implies a result by Fisher and Reeves (1995) on tournament games. 1 Introduction A (two-player) zero-sum game is played on a matrix where the row player chooses a row and the column player chooses a column. The payoff of the column player is given by the corresponding matrix entry, the row player receives the negative of the column player. Both players may randomize over their actions. Von Neumann s minimax theorem shows that every zero-sum game admits a value, i.e., the column player can guarantee an expected payoff for himself that is equal to the negative of the expected payoff that the row player can guarantee for himself. A strategy that maximizes the minimal guaranteed expected payoff of a player is a maximin strategy for this player. Pairs of maximin strategies correspond to Nash equilibria of the game. We call a maximin strategy for the column player an optimal strategy. For various classes of games the question about the number of Nash equilibria has been studied. Wilson (1971) showed that the number of equilibria is finite and odd for almost 1

2 all n-person normal form games. A different proof for the same statement was given by Harsanyi (1973a). In zero-sum games every convex combination of Nash equilibria is again a Nash equilibrium. Hence there is either a unique Nash equilibrium or infinitely many. However, Wilson s theorem does not imply that Nash equilibria are almost always unique in subclasses of games, e.g., zero-sum games, symmetric zero-sum games, or tournament games. 1 Fisher and Ryan (1992) showed that every tournament game admits a unique Nash equilibrium. This result was generalized by Laffond et al. (1997) and Le Breton (2005) to symmetric zero-sum games where all payoffs are odd integers or satisfy a more general congruency condition, respectively. Nash equilibria are not generally unique in zero-sum games, however, Jonasson (2004) showed that they are almost surely unique if the payoffs are given by continuous i.i.d. random variables that are symmetric around 0. Moreover, he proves that the expected fraction of actions in the support of an optimal strategy is close to 1 /2 when the number of actions goes to infinity. In this paper we give a unified treatment of these questions for symmetric zero-sum games and a large family of subclasses thereof. For these games every maximin strategy of the column player is also a maximin strategy of the row player. Hence there is a unique Nash equilibrium if and only if there is a unique optimal strategy. In this paper we determine the probability that a randomly chosen game admits an optimal strategy that uses a certain set of actions. This also implies a result by Fisher and Reeves (1995). On the way we prove that optimal stratagies are almost surely unique if the payoffs are given by absolutely continuous random variables. 2 Preliminaries For n N, let [n] = {1,..., n}. A zero-sum game G is a matrix in R m n, where m, n N. Both players have a set of actions that consists of choosing a row or column, respectively. The entry in the ith row and jth column of G represents the payoff of the column player if the row player choses row i and the column player choses column j. The set of all probability distributions over a finite set S is denoted by (S), i.e., (S) = {x R S 0 : i S x i = 1}. A (randomized) strategy for the row player or the column player is a probability distribution over the rows or columns of G, respectively. The support p + of a strategy p ([n]) is the set of actions to which p assigns positive probability, i.e., p + = {i [n]: p i > 0}. Analogously, we define p = {i [n]: p i < 0}. This notation extends to all vectors in R n. A strategy p is a maximin strategy for the column player in G if it maximizes his minimum expected payoff, i.e., min q ([m]) qt Gp max min p ([n]) q ([m]) qt Gp. Maximin strategies for the row player are defined analogously. By von Neumann s minimax theorem (von Neumann, 1928) the minimum expected payoff of the column player when 1 Tournament games are symmetric zero-sum games in which all off-diagonal payoffs are either 1 or 1. 2

3 he plays a maximin strategy is equal to the negative of the minimum expected payoff of the row player when he plays a maximin strategy. This payoff is called the value of the game. We say that p is an optimal strategy of G if it is a maximin strategy for the column player. Note that the set of optimal strategies of G is convex. A zero-sum game is symmetric if G is skew-symmetric, i.e., G = G T. For the remainder of this paper, we will simply use game to denote a symmetric zero-sum game. The set of all games is denoted by G. For a set of actions S, a game G, and a vector v, we denote by G S = (G ij ) i,j S and v S = (v i ) i S the sub-matrix and sub-vector induced by S, respectively. We define extra notation for particular classes of games. The set of games with multiple optimal strategies is denoted by G >1, i.e., G >1 = {G G: G has two distinct optimal strategies}. The set G (S) contains all games that have an optimal strategy with support S, i.e., G (S) = {G G: Gp 0 for some strategy p with p + = S}. We will introduce a probability distribution over the set of games by means of probability distributions over the payoffs. Let X ij, i, j [n], i < j be real-valued random variables and X ij if i < j, X = X ji if i > j, and 0 otherwise. Realizations of X are games and hence, X induces a probability distribution over games. For a set of games G, we denote by P X (G ) the probability that the realization of X is in G. We say that X ij is symmetric around 0 if P Xij (I) = P Xij ( I) for every set I R, where I = { v : v I}. Finally X is symmetric around 0 if all X ij are symmetric around 0. 3 Size of the Optimal Strategy For every set of alternatives, we determine the exact probability that this set is the support of an optimal strategy. The proof requires that the payoffs are given by i.i.d. random variables that are symmetric around 0 and for which the optimal strategy is almost surely unique. This is for example the case if the payoffs follow independent normal distributions. We first state three lemmas to prepare the proof of our main theorem. The proof of Lemma 1 makes use of the equalizer theorem, which states that every action that yields payoff 0 against every optimal strategy is played with positive probability in some optimal strategy. Theorem 1 (Raghavan, 1994). Let G be a game and i N. If (Gp) i = 0 for all optimal strategies p of G, then there is an optimal strategy p of G with p i > 0. 3

4 Following Harsanyi (1973b), an optimal strategy is quasi-strict 2 if it yields strictly positive payoff against every action which is not in its support. It is a well-known fact that if a game only admits quasi-strict optimal strategies, then it in fact has only one optimal strategy. Lemma 1 shows that the converse is also true, i.e., if a game has an optimal strategy that is not quasi-strict, then this cannot be the unique optimal strategy of the game. Lemma 1. Let G be a game and p an optimal strategy of G. If (Gp) i = 0 for some i p +, then G has multiple optimal strategies. Proof. Assume for contradiction that p is the unique optimal strategy of G and (Gp) i = 0 for some i p +. Then it follows from Theorem 1 that G has an optimal strategy p with p i > 0 and, in particular, p p. This contradicts uniqueness of p. The next lemma shows that every strategy that is the unique optimal of some game puts positive probability on an odd number of actions. This does not hold for non-symmetric zero-sum games, e.g., the game known as matching pennies has a unique optimal strategy of size 2. Lemma 2. Let G be a game and p the unique optimal strategy of G. Then p puts positive probability on an odd number of actions. Proof. Assume for contradiction that p puts positive probability on an even number of actions. Without loss of generality, let p + = [k] for some even k N. Since p is the unique optimal strategy of G, it follows from Lemma 1 that (Gp) i > 0 for all i [k]. As k 1 is odd and G [k 1] is skew-symmetric it follows that there is v R n \{0} with v v + [k 1] and G [k 1] v [k 1] = 0. Assume without loss of generality that (Gv) k 0 (otherwise we may take v). Then, for ɛ > 0 small enough, we have that p ɛ = (1 ɛ)p + ɛv ([n]) and Gp ɛ 0, i.e., p ɛ is an optimal strategy of G. This contradicts uniqueness of p. Lastly, we show that if the probability that a randomly chosen game admits two distinct optimal strategies is zero, then also the probability that some sub-game admits two distinct optimal strategies is zero. The proof relies on the fact that the X ij are independent and symmetric around 0. Lemma 3. Let X ij, i, j [n], i < j be independent random variables that are symmetric around 0 and P X (G >1 ) = 0. Then P X ({G G: G S has two distinct optimal strategies}) = 0 for all S [n]. Proof. Let X ij, i, j [n], i < j be i.i.d. random variables that are symmetric around 0 and P X (G >1 ) = 0. Furthermore, let S [n] and G >1 (S) = {G G: G S has two distinct optimal strategies}. 2 Harsanyi introduced the concept of quasi-strong equilibria, which however was referred to as quasi-strict equilibria in subsequent papers to avoid confusion with Aumann s notion of strong equilibria (Aumann, 1959). 4

5 For G G >1 (S), let p, p ([n]) such that p S and p S are distinct optimal strategies of G S. We define p λ = (1 λ)p + λp. Note that p λ S is an optimal strategy of G S for every λ [0, 1], since the set of optimal strategies is convex. For λ > 0 small enough, we have that (Gp λ ) i < 0 if and only if (Gp) i < 0 or (Gp) i = 0 and (Gp ) i < 0 for all i [n]. Now let G p,p G such that, for all i, j [n], { G G p,p ij if (Gp) i < 0 or (Gp) i = 0 and (Gp ) i < 0, and ij = otherwise. G ij Then p and p λ are optimal strategies in G p,p for small λ > 0. In particular, G p,p has two distinct optimal strategies. Finally, let G >1 = {G p,p G: p, p are optimal strategies of G S }. Because P X (G >1 (S)) > 0 it follows from the fact that the X ij s are independent and symmetric around 0 that P X (G >1 ) > 0. As G >1 G >1, this contradicts the assumption that P X (G >1 ) = 0. We now assume that the probability that a randomly chosen game admits two distinct optimal strategies is 0. Then, it follows quickly from Lemma 2 that the probability that a game has an optimal strategy with even support size is 0. Interestingly, it turns out that the probability that a game has an optimal strategy with a given support of odd size is independent of the chosen support. This is again specific to symmetric zero-sum games and does not hold in general for zero-sum games. Theorem 2. Let X ij, i, j [n], i < j be i.i.d. random variables that are symmetric around 0 and P X (G >1 ) = 0. Then, for every S [n], { P X (G 2 (n 1) if S is odd, and (S)) = 0 otherwise. Note that the number of subsets of odd size of [n] is 2 n 1. P X (G (S)) sum up to 1. Hence, the probabilities Proof. For simplicity, we introduce additional notation. The set of all games that have an optimal strategy on G S with support S is denoted by G(S). Formally, G(S) = {G G: G S p S 0 for some strategy p with p + = S}. Moreover, the set of games with an optimal strategy on G S with support S such that actions in T yield non-negative payoff versus this strategy is denoted by G T (S), i.e., G T (S) = {G G: G S p S 0 and (Gp) i 0 iff i T for some strategy p with p + = S}. 5

6 Lastly, let G = (S) be the set of games with a vector in the null space where only entries in S are positive, i.e., G = (S) = {G G: Gv = 0 for some v R n with v + = S}. Note that the union over all non-empty S of G = (S) is G if n is odd, since no game of odd size has full rank. It follows directly from Lemma 2 that P X (G (S)) = 0 if S is even, since, by assumption, P X (G >1 ) = 0. Next, we determine the probability that X has an optimal strategy with full support if n is odd. For every set of actions S, let S be its complement with respect to [n], i.e., S = [n] \ S. We define a function φ S on the set of games where { G ij if i, j S or i, j (φ S (G)) ij = S, and otherwise. G ij Intuitively, φ S changes the sign of the payoffs that correspond to pairs of actions where one action is in S and the other one is not. Thus, φ S is self-inverse, i.e., φ S = φ 1 S. If Xv = 0 and v i = 0 for some v R n and i [n], then φ v (X)v = 0, where v [n]\v = v [n]\v and v v = v v. Then, v / v is an optimal strategy of φ v (X) with v i = 0 and φ v (X) v / v = 0. It follows from Lemma 1 that φ v (X) has multiple optimal strategies. This is, by assumption, as probability-zero event. As X is symmetric around 0, v has almost surely no zero entry. This implies that P X (G = (S) \ G = ( S)) = 0 for all S [n]. If X G = (S) G = (T ) with S T, T, there are v, w R n with v + = S and w + = T and Xv = Xw = 0. Then, there is c R such that (v + cw) i = 0 for some i [n] and, clearly, X(v + cw) = 0. As before, this is a probability-zero event by Lemma 1. So we get that { P X (G = (S) G = P X (G = (S)) if S = T or (T )) = S = T, and 0 otherwise. The next step is to show that P X (G = (S)) = P X (G = (T )) for all sets of actions S, T of odd size: the cases S = T and S = T are already covered by the above statement. Now let T S, S. If φ S (X) G = (S), then φ T (X) G = (T ) or, equivalently, if X G = (S), then φ S φ T (X) = φ S T (X) G = (T ), where S T denotes the symmetric difference of S and T. Thus, G = (T ) can be obtained from G = (S) by negating all entries that correspond to pairs of actions where one action is in S T and the other one is not for all games in G = (S). Since X is symmetric around 0, we get that P X (G = (S)) = P X (G = (T )) as desired. In summary, we have G = (S)) = G, P X (G = (S) \ G = ( S)) = 0 for all S [n], and S [n] P X (G = (S)) = P X (G = (T )) if S and T are odd. 6

7 Thus, P X (G ([n])) = P X (G = ([n])) = 2 (n 1). Now, let n N, S [n], and k = S. We want to show that P X (G (S)) = 2 (n 1) if k is odd and 0 otherwise. Note that, if X has an optimal strategy with support S, then X S has an optimal strategy with support S. It follows from what we have shown above and Lemma 3 that the probability that X S has an optimal strategy with support S is 2 (k 1) if k is odd and 0 otherwise. So P X (G (S)) = 0 if k is even. Hence we consider the case where k is odd. Note that G T (S) can only be non-empty if S T and thus, for at most 2 n k distinct T. Suppose that X G T (S) G T (S), where T and T are distinct. Then, there are strategies p and p such that X S p S = X S p S = 0 and T = {i [n]: (Xp) i 0} and T = {i [n]: (Xp ) i 0}. Hence, p and p are distinct and thus X S has two distinct optimal strategies, which is a probability zero event by Lemma 3. From this it follows that P (G T (S) G T (S)) = 0 for all distinct T and T. Moreover, the union over all T of G T (S) is equal to G(S). Lastly, we show that P (G T (S)) = P (G T (S)) for all T and T. It suffices to consider the case where T = T {i } for some i T. The rest follows from repeated application of this case. Assume X G T (S) with witness p, i.e., X S p S = 0 and (Xp) i 0 if and only if i T. Furthermore, let X be equal to X except that the sign of all payoffs that correspond to pairs of actions {i, i } where i S is reversed. Then (X p) i 0 if and only if i T. Hence, G T (S) can be obtained from G T (S) by reversing the sign of all payoffs that correspond to pairs of actions {i, i } where i S for all games. Since X is symmetric around 0, it follows that P (G T (S)) = P (G T (S)). In summary we have that P (G T (S)) = 2 (n k) P (G(S)) = 2 (n 1), as there are 2 n k distinct sets of actions T that contain S. From the fact that G N (S) = G (S) we get the desired statement. As we have noted before, Theorem 2 does not hold for non-symmetric zero-sum games. Experiments by Faris and Maier (1987) suggest that the size of the optimal strategy of a uniformly at random chosen zero-sum game approximately follows a binomial distribution with parameters n and 1 /2. Finding the exact distribution of optimal strategies for the general case is still an open problem. A precondition for Theorem 2 is that P X (G >1 ) = 0. We show that this condition is satisfied for a large class of distributions, i.e., all absolutely continuous distributions. This for example covers the case that all X ij are random normal variables. Lemma 4. Let X ij, i, j [n], i < j be absolutely continuous (w.r.t. the Lebesgue measure) random variables. Then P X (G >1 ) = 0. Proof. We will show that there is an even-sized square sub-matrix of X that is singular if X G >1. If there is an optimal strategy of X with even-sized support, say p, then X supp(p) is an even-sized square matrix of X that is singular. Second, we consider the case when there are two optimal strategies of X, say p and q, with distinct support of odd size. Without loss of generality there is i supp(q) \ supp(p) such that (Xp) i = 0. Then, the matrix X supp(p) {i} is singular and even sized. The remaining case is that all optimal strategies of G have the same support and this support contains an odd number of actions. 7

8 Let p and q be two such optimal strategies and p λ = (1 + λ)p λq. Then, there are λ 0 and i [n] \ supp(p) such that p λ 0 and (Xp λ ) i = 0. Hence, X supp(p) {i} is even-sized and singular. Thus, in any case, X has an even-sized square sub-matrix that is singular. Since the X ij are absolutely continuous, every even-sized square sub-matrix of X is almost surely regular. Since the number of sub-matrices of X is finite, the claim follows. Theorem 2 applies to a wide range of distributions over payoffs. By Lemma 4 it holds for all absolutely continuous distributions that are symmetric around 0, e.g., a normal distribution or a uniform distribution on [ 1, 1]. Moreover, if the X ij s are uniformly distributed on { 1, 1}, we obtain a uniform distribution over tournament games. Fisher and Ryan (1992) showed that optimal strategies are unique in tournament games. Thus, Theorem 2 implies a result by Fisher and Reeves (1995). Acknowledgements This material is based upon work supported by Deutsche Forschungsgemeinschaft under grant BR 2312/10-1 and TUM Institute for Advanced Study through a Hans Fischer Senior Fellowship. The author thanks Felix Brandt for helpful discussions. References R. J. Aumann. Acceptable points in general cooperative n-person games. In A. W. Tucker and R. D. Luce, editors, Contributions to the Theory of Games IV, volume 40 of Annals of Mathematics Studies, pages Princeton University Press, W. G. Faris and R. S. Maier. The value of a random game: The advantage of rationality. Complex Systems, 1: , D. C. Fisher and R. B. Reeves. Optimal strategies for random tournament games. Linear Algebra and its Applications, 217:83 85, D. C. Fisher and J. Ryan. Optimal strategies for a generalized scissors, paper, and stone game. American Mathematical Monthly, 99(10): , J. C. Harsanyi. Oddness of the number of equilibrium points: A new proof. International Journal of Game Theory, 2(1): , 1973a. J. C. Harsanyi. Games with randomly disturbed payoffs: A new rationale for mixedstrategy equilibrium points. International Journal of Game Theory, 2(1):1 23, 1973b. J. Jonasson. On the optimal strategy in a random game. Electronic Communications in Probability, 9: ,

9 G. Laffond, J.-F. Laslier, and M. Le Breton. A theorem on symmetric two-player zero-sum games. Journal of Economic Theory, 72(2): , M. Le Breton. On the uniqueness of equilibrium in symmetric two-player zero-sum games with integer payoffs. Économie publique, 17(2): , T. E. S. Raghavan. Zero-sum two person games. In R. J. Aumann and S. Hart, editors, Handbook of Game Theory with Economic Applications, chapter 20. North-Holland, J. von Neumann. Zur Theorie der Gesellschaftspiele. Mathematische Annalen, 100(1): , R. Wilson. Computing equilibria in n-person games. SIAM Journal on Applied Mathematics, 21(1):80 87,