Efficient Nash Equilibrium Computation in Two Player Rank-1 G

Transcription

1 Efficient Nash Equilibrium Computation in Two Player Rank-1 Games Dept. of CSE, IIT-Bombay China Theory Week 2012 August 17, 2012 Joint work with Bharat Adsul, Jugal Garg and Milind Sohoni

2 Outline Games, Nash equilibrium and history Two player finite games Known results Rank and tractability Rank-1 games

3 Games Set of players - rational, intelligent, selfish. Each with a set of strategies - finite or infinite. Payoffs - preference over outcome is described through payoffs. Equilibrium - State from where no player gains by unilateral deviation.

4 Example: Rock-Paper-Scissor Two players, say Row and Column. Each with three strategies - R, P and S. R P S R 0,0-1,1 1,-1 P 1,-1 0,0-1,1 S -1,1 1,-1 0,0 Finite games have finitely many players each with finitely many strategies.

5 Example: Rock-Paper-Scissor Two players, say Row and Column. Each with three strategies - R, P and S. R P S R 0,0-1,1 1,-1 P 1,-1 0,0-1,1 S -1,1 1,-1 0,0 Finite games have finitely many players each with finitely many strategies. Equilibrium?

6 Mixed Strategies and Nash Equilibrium Row plays i and Column plays j j j i A i B Payoffs: A ij B ij

7 Mixed Strategies and Nash Equilibrium Mixed strategy: Probability density function over strategy set Row plays x and Column plays y x 1. x n y 1... y n A x 1. x n y 1... y n B

8 Mixed Strategies and Nash Equilibrium Mixed strategy: Probability density function over strategy set Row plays x and Column plays y x 1. x n y 1... y n Expected Payoffs: i,j (x iy j )A ij = x T Ay A x 1. y 1... y n B x n i,j (x iy j )B ij = x T By

9 Mixed Strategies and Nash Equilibrium Mixed strategy: Probability density function over strategy set Row plays x and Column plays y x 1. x n y 1... y n Expected Payoffs: i,j (x iy j )A ij = x T Ay A x 1. y 1... y n B x n i,j (x iy j )B ij = x T By Nash Equilibrium (NE): No player gains by unilateral deviation. x T Ay x T Ay, x and x T By x T By, y

10 von Neumann and Nash von Neumann (1928) - In a (finite) 2-player zero-sum game, equilibrium, mini-max strategies, exists. John Nash (1951) - Existence of equilibrium in finite games. Proof is through Brouwer s fixed point theorem, hence highly non-constructive.

11 von Neumann and Nash von Neumann (1928) - In a (finite) 2-player zero-sum game, equilibrium, mini-max strategies, exists. John Nash (1951) - Existence of equilibrium in finite games. Proof is through Brouwer s fixed point theorem, hence highly non-constructive. Irrational NE in a 3-player game (3-Nash) (Nash, 1951).

12 Computational Results Lemke-Howson algorithm (1964) for exact 2-Nash - Exponential. Path following. Establishes rationality for 2-Nash.

13 Computational Results Lemke-Howson algorithm (1964) for exact 2-Nash - Exponential. Path following. Establishes rationality for 2-Nash. Papadimitriou (1992) defined PPAD. Guaranteed existence but difficult to compute. Contains Sperner s lemma, approximate fixed point, exact 2-Nash. Approximate fixed point is PPAD-hard.

14 Contd. PPAD-hardness Daskalakis, Goldberg and Papadimitriou (2005) - approximate 4-Nash. Chen and Deng (2006) - exact 2-Nash. Chen, Deng and Teng (2006) - approximate 2-Nash.

15 Contd. PPAD-hardness Daskalakis, Goldberg and Papadimitriou (2005) - approximate 4-Nash. Chen and Deng (2006) - exact 2-Nash. Chen, Deng and Teng (2006) - approximate 2-Nash. NP-hardness for 2-Nash: existence of two NE, NE with x payoff, NE with t non-zero strategies, etc.

16 Contd. PPAD-hardness Daskalakis, Goldberg and Papadimitriou (2005) - approximate 4-Nash. Chen and Deng (2006) - exact 2-Nash. Chen, Deng and Teng (2006) - approximate 2-Nash. NP-hardness for 2-Nash: existence of two NE, NE with x payoff, NE with t non-zero strategies, etc. Dantzig (1963): mini-max strategies equivalent to linear primal-dual solution. Zero-sum games can be solved efficiently.

17 2-player (Bimatrix) Games: Rank and Tractability Lipton et al. (2003) - when ranks of A and B are constant.

18 2-player (Bimatrix) Games: Rank and Tractability Lipton et al. (2003) - when ranks of A and B are constant. Kannan and Theobald (2007) Rank of game (A,B) is rank(a + B). FPTAS for constant rank games.

19 2-player (Bimatrix) Games: Rank and Tractability Lipton et al. (2003) - when ranks of A and B are constant. Kannan and Theobald (2007) Rank of game (A,B) is rank(a + B). FPTAS for constant rank games. Zero-sum rank-0 - an LP captures all the NE.

20 2-player (Bimatrix) Games: Rank and Tractability Lipton et al. (2003) - when ranks of A and B are constant. Kannan and Theobald (2007) Rank of game (A,B) is rank(a + B). FPTAS for constant rank games. Zero-sum rank-0 - an LP captures all the NE. Rank-1: No polynomial time algorithm was known. Difficulty: Disconnected NE set. Reduces to solving rank-1 QP (known to be NP-hard in general).

21 This Talk Efficient algorithm for an exact NE in rank-1 games. A simplex-like enumeration algorithm for rank-1; finds a NE in general. Based on [Adsul, Garg, M, Sohoni STOC 11].

22 Nash Equilibrium in Bimatrix Games Game (A,B) Strategy sets: S 1 = {1,...,m} and S 2 = {1,...,n}. i = Set of probability distributions over S i.

23 Nash Equilibrium in Bimatrix Games Game (A,B) Strategy sets: S 1 = {1,...,m} and S 2 = {1,...,n}. i = Set of probability distributions over S i. x 1 and y 2 i,j (x iy j )A ij = x T Ay and i,j (x iy j )B ij = x T By

24 Nash Equilibrium in Bimatrix Games Game (A,B) Strategy sets: S 1 = {1,...,m} and S 2 = {1,...,n}. i = Set of probability distributions over S i. x 1 and y 2 i,j (x iy j )A ij = x T Ay and i,j (x iy j )B ij = x T By Nash Equilibrium (NE): No player gains by unilateral deviation. x T Ay x T Ay, x and x T By x T By, y

25 NE Characterization in Bimatrix Games Notations: A i is i-th row, A j is j-th column.

26 NE Characterization in Bimatrix Games Notations: A i is i-th row, A j is j-th column. Given a y 2, i S 1 fetches (A i y) to the player 1.

27 NE Characterization in Bimatrix Games Notations: A i is i-th row, A j is j-th column. Given a y 2, i S 1 fetches (A i y) to the player 1. Example: y = (0.5, 0.3, 0.2) T A y = R P S =

28 NE Characterization in Bimatrix Games Notations: A i is i-th row, A j is j-th column. Given a y 2, i S 1 fetches (A i y) to the player 1. Example: y = (0.5, 0.3, 0.2) T A y = R P S = Best Response (BR): P BR: i S 1 fetching maximum payoff (A i y = max k S1 A k y)

29 Contd. Payoff from x 1 is a conv. comb. of A i y s. x maximizes payoff iff i S1, x i > 0 i is a BR to y.

30 Contd. Payoff from x 1 is a conv. comb. of A i y s. x maximizes payoff iff i S1, x i > 0 i is a BR to y. Similarly, for player 2: Given a x 1 payoff from j S 2 is x T B j.

31 Contd. Payoff from x 1 is a conv. comb. of A i y s. x maximizes payoff iff i S1, x i > 0 i is a BR to y. Similarly, for player 2: Given a x 1 payoff from j S 2 is x T B j. Nash Equilibrium: i S 1, x i > 0 j S 2, y j > 0 A i y = max k A k y x T B j = max k x T B k

32 Best Response Polyhedra, Fully-labeled and NE NE: i S 1, x i > 0 A i y = max k A k y j S 2, y j > 0 x T B j = max k x T B k

33 Best Response Polyhedra, Fully-labeled and NE NE: i S 1, x i > 0 A i y = max k A k y j S 2, y j > 0 x T B j = max k x T B k Best Response Polyhedra (BRPs) (Assume non-degeneracy) P = {(y, π 1) A i y π 1 0, i S 1; y j 0, j S 2; Σ j S2 y j = 1} Q = {(x, π 2) x i 0, i S 1; x T B j π 2 0, j S 2; Σ i S1 x i = 1} Where π i captures the best payoff to player i.

34 Best Response Polyhedra, Fully-labeled and NE NE: i S 1, x i > 0 A i y = max k A k y j S 2, y j > 0 x T B j = max k x T B k Best Response Polyhedra (BRPs) (Assume non-degeneracy) P = {(y, π 1) A i y π 1 0, i S 1; y j 0, j S 2; Σ j S2 y j = 1} Q = {(x, π 2) x i 0, i S 1; x T B j π 2 0, j S 2; Σ i S1 x i = 1} Where π i captures the best payoff to player i. Fully-labeled: x i (A i y π 1 ) = 0, i; y j (x T B j π 2 ) = 0, j

35 Best Response Polyhedra, Fully-labeled and NE NE: i S 1, x i > 0 A i y = max k A k y j S 2, y j > 0 x T B j = max k x T B k Best Response Polyhedra (BRPs) (Assume non-degeneracy) P = {(y, π 1) A i y π 1 0, i S 1; y j 0, j S 2; Σ j S2 y j = 1} Q = {(x, π 2) x i 0, i S 1; x T B j π 2 0, j S 2; Σ i S1 x i = 1} Where π i captures the best payoff to player i. Fully-labeled: x i (A i y π 1 ) = 0, i; y j (x T B j π 2 ) = 0, j NE iff fully-labeled. Only vertex pairs.

36 Best Response Polyhedra, Fully-labeled and NE NE: i S 1, x i > 0 A i y = max k A k y j S 2, y j > 0 x T B j = max k x T B k Best Response Polyhedra (BRPs) (Assume non-degeneracy) P = {(y, π 1) A i y π 1 0, i S 1; y j 0, j S 2; Σ j S2 y j = 1} Q = {(x, π 2) x i 0, i S 1; x T B j π 2 0, j S 2; Σ i S1 x i = 1} Where π i captures the best payoff to player i. Fully-labeled: x i (A i y π 1 ) = 0, i; y j (x T B j π 2 ) = 0, j NE iff fully-labeled. Only vertex pairs. Note: x i (A i y π 1 ) 0, i; y j (x T B j π 2 ) 0, j

37 QP for NE Note: In P Q x T Ay π 1 0; x T By π 2 0 NE iff x T Ay π 1 = 0; x T By π 2 = 0 (fully-labeled).

38 QP for NE Note: In P Q x T Ay π 1 0; x T By π 2 0 NE iff x T Ay π 1 = 0; x T By π 2 = 0 (fully-labeled). QP: max : x T (A + B)y π 1 π 2 (y,π 1 ) P, (x,π 2 ) Q Optimal value is always zero.

39 Solving a Rank-1 Game (A, B) rank(a + B) = 1 A + B = α β T, α R m, β R n.

40 Solving a Rank-1 Game (A, B) rank(a + B) = 1 A + B = α β T, α R m, β R n. max : (x T α)(β T y) π 1 π 2 (y, π 1 ) P, (x, π 2 ) Q

41 Solving a Rank-1 Game (A, B) rank(a + B) = 1 A + B = α β T, α R m, β R n. max : (x T α)(β T y) π 1 π 2 (y, π 1 ) P, (x, π 2 ) Q Consider B as A + α β T and replace x T α by λ every where.

42 B = A + αβ T ; Replace x T α by λ P = {(y, π 1) A i y π 1 0, i S 1; y j 0, j S 2; Σ j S2 y j = 1} Q = {(x, π 2) x i 0, i S 1; x T B j π 2 0, j S 2; Σ i S1 x i = 1} Q = {(x,λ,π 2 ) x i 0, i; x T A j + (x T α)β j π 2 0, j; Σ i x i = 1}

43 B = A + αβ T ; Replace x T α by λ P = {(y, π 1) A i y π 1 0, i S 1; y j 0, j S 2; Σ j S2 y j = 1} Q = {(x, π 2) x i 0, i S 1; x T B j π 2 0, j S 2; Σ i S1 x i = 1} Q = {(x,λ,π 2 ) x i 0, i; x T A j + λβ j π 2 0, j; Σ i S1 x i = 1}

44 B = A + αβ T ; Replace x T α by λ P = {(y, π 1) A i y π 1 0, i S 1; y j 0, j S 2; Σ j S2 y j = 1} Q = {(x, π 2) x i 0, i S 1; x T B j π 2 0, j S 2; Σ i S1 x i = 1} Q = {(x,λ,π 2 ) x i 0, i; x T A j + λβ j π 2 0, j; Σ i S1 x i = 1} H α : λ x T α = 0. Q = Q H α.

45 B = A + αβ T ; Replace x T α by λ P = {(y, π 1) A i y π 1 0, i S 1; y j 0, j S 2; Σ j S2 y j = 1} Q = {(x, π 2) x i 0, i S 1; x T B j π 2 0, j S 2; Σ i S1 x i = 1} Q = {(x,λ,π 2 ) x i 0, i; x T A j + λβ j π 2 0, j; Σ i S1 x i = 1} H α : λ x T α = 0. Q = Q H α. QP : max : λ(β T y) π 1 π 2 (y, π 1 ) P, (x, λ, π 2 ) Q

46 B = A + αβ T ; Replace x T α by λ P = {(y, π 1) A i y π 1 0, i S 1; y j 0, j S 2; Σ j S2 y j = 1} Q = {(x, π 2) x i 0, i S 1; x T B j π 2 0, j S 2; Σ i S1 x i = 1} Q = {(x,λ,π 2 ) x i 0, i; x T A j + λβ j π 2 0, j; Σ i S1 x i = 1} H α : λ x T α = 0. Q = Q H α. QP : max : λ(β T y) π 1 π 2 (y, π 1 ) P, (x, λ, π 2 ) Q In P Q, λ(β T y) π 1 π 2 0. Equality iff fully-labeled.

47 Solutions of QP λ(β T y) π 1 π 2 0. Equality iff fully-labeled. N : Solutions of QP. Set of fully-labeled points of P Q.

48 Solutions of QP λ(β T y) π 1 π 2 0. Equality iff fully-labeled. N : Solutions of QP. Set of fully-labeled points of P Q. NE of (A, B) N H α. Recall: Q = Q H α. NE iff fully-labeled in P Q. Goal: Find a point in N H α.

49 Structure of N Recall: Fully-labeled points of P Q are vertex pairs. In P Q, λ gives an extra degree of freedom for fully-labeled. N: one infinite path, and may be a set of cycles on 1-skeleton.

50 Structure of N Recall: Fully-labeled points of P Q are vertex pairs. In P Q, λ gives an extra degree of freedom for fully-labeled. N: one infinite path, and may be a set of cycles on 1-skeleton. N(a) : Points of N with λ = a.

51 Structure of N Recall: Fully-labeled points of P Q are vertex pairs. In P Q, λ gives an extra degree of freedom for fully-labeled. N: one infinite path, and may be a set of cycles on 1-skeleton. N(a) : Points of N with λ = a. OPT(δ) - max : δ(β T y) π 1 π 2 (x, π 1 ) P, (y, λ, π 2 ) Q, λ = δ N(a), and OPT(a) = N(a), a R. N forms a single path with λ being monotonic.

52 Enumeration Algorithm Recall: NE of (A, B) N H α. H α Follow the path N and output whenever hit H α (complementary pivot).

53 The Efficient Algorithm Recall: NE of (A, B) N H α. H α : λ x T α = 0 a 1 = min i α i, a 2 = max α i ; a 1 i i α i x i a 2, x 1. H α H α H + α N (a 1 ) N (a 2 )

54 The Efficient Algorithm Recall: NE of (A, B) N H α. H α : λ x T α = 0 a 1 = min i α i, a 2 = max α i ; a 1 i i α i x i a 2, x 1. H α H α H + α N (a 1 ) N (a 2 ) Since N(a) = OPT(a), obtain it by solving an LP.

55 The Efficient Algorithm: BinSearch Recall: NE of (A, B) N H α. a 1 = min i BinSearch: α i, a 2 = max α i ; a 1 i i α i x i a 2, x 1. 1 Let a = a1+a2 2. If N(a) H α then output NE; Exit. H α H α H + α N (a 1 ) N (a) N (a 2 )

56 The Efficient Algorithm: BinSearch Recall: NE of (A, B) N H α. a 1 = minα i, a 2 = max α i ; a 1 α i x i a 2, x 1. i i i BinSearch: 1 Let a = a1+a2 2. If N(a) H α then output NE; Exit. 2 Else if N(a) Hα then a 1 = a else a 2 = a. Repeat from 1. H α H α H + α N (a 1 ) N (a 2 )

57 The Efficient Algorithm: BinSearch Recall: NE of (A, B) N H α. a 1 = minα i, a 2 = max α i ; a 1 α i x i a 2, x 1. i i i BinSearch: 1 Let a = a1+a2 2. If N(a) H α then output NE; Exit. 2 Else if N(a) Hα then a 1 = a else a 2 = a. Repeat from 1. Time Complexity (polynomial time): Solve an LP to obtain N(a) in Step 1. #iterations is O(L), where L is the input bit length.

58 Extension to Rank-k A + B = k l=1 αl β lk.

59 Extension to Rank-k A + B = k l=1 αl β lk. QP: max k l=1 (xt α l )(β lt y) π 1 π 2 s.t. (y, π 1 ) P, (x, π 2 ) Q

60 Extension to Rank-k A + B = k l=1 αl β lk. QP: max k l=1 (xt α l )(β lt y) π 1 π 2 s.t. (y, π 1 ) P, (x, π 2 ) Q Consider B as A + k l=1 αl β lk and replace x T α l by λ l.

61 Extension to Rank-k A + B = k l=1 αl β lk. QP: max k l=1 (xt α l )(β lt y) π 1 π 2 s.t. (y, π 1 ) P, (x, π 2 ) Q Consider B as A + k l=1 αl β lk and replace x T α l by λ l. N k-skeleton of P Q

62 Extension to Rank-k A + B = k l=1 αl β lk. QP: max k l=1 (xt α l )(β lt y) π 1 π 2 s.t. (y, π 1 ) P, (x, π 2 ) Q Consider B as A + k l=1 αl β lk and replace x T α l by λ l. N k-skeleton of P Q H α l : λ l = x T α l. NE of (A, B) N l H α l

63 Extension to Rank-k A + B = k l=1 αl β lk. QP: max k l=1 (xt α l )(β lt y) π 1 π 2 s.t. (y, π 1 ) P, (x, π 2 ) Q Consider B as A + k l=1 αl β lk and replace x T α l by λ l. N k-skeleton of P Q H α l : λ l = x T α l. NE of (A, B) N l H α l N(ā), N(ā) = OPT(ā), ā R k Helps to prove some monotonicity properties.

64 Discussion What about rank-k? Are they hard? The structural analysis for rank-k may help to answer. We resolved a special class of rank-1 QP; NP-hard in general. Extend the technique to generalize this class. Structural: When does N contain only the path?

65 λ 2 λ 1 Thanks