Computational aspects of two-player zero-sum games Course notes for Computational Game Theory Section 4 Fall 2010

Transcription

1 Computational aspects of two-player zero-sum games Course notes for Computational Game Theory Section 4 Fall 010 Peter Bro Miltersen January 16, 011 Version 1.

2 4 Finite Perfect Information Games Definition 1 (Finite Perfect information games) A finite perfect information game is an extensive form game where all information sets are singletons ( A plain game tree ). An example of such a game is seen in Figure 1. Note that for such games we can cut off any subtree thereby yielding a subgame. The reason we can do this, is because we will never cut in such a way that we split two informations sets. In general if we cut off a subtree without splitting information sets we get a subgame. d L I D II R 7 u 3 I U 1 Figure 1: A perfect information game Proposition Two-player zero finite perfect information games have pure maximin/minimax strategies Proof (Backward induction) We do induction in the height of the tree. We have three cases, which are shown in Figures, 3, 4. I v 1 v v l Figure : Case 1

3 II v 1 v v l Figure 3: Case Chance α l v 1 α 1 v α v l Figure 4: Case 3 In the first case the root node belongs to player 1. Each of the l subgames have pr. induction a pure maximin strategy and a value. Player 1 s strategy will simply be to choose the subgame with the largest value, so we get v = max(v 1,..., v l ). The second case is very similar, here instead the value will be the minimum: v = min(v 1,..., v l ). And in the last case we have a chance node at the root. Here the value will be the weighted value of the subgames: v = n i=1 v iα i. In this case we have to specify behavior in all subgames because we can end up in all of these (where α i 0). We can apply the technique from the proof on the example of Figure 1 This is done in Figure 5. We see that the value of the game and all of the subgames in this case is 3. This computation can be done by a tree traversal in linear time. Fact 3 Given a two-player, zero-sum perfect information game, value and pure maximin strategy can be computed in linear time. It is also easy to see that sublinear time for computing value and pure maximin strategy is not possible in general, by looking at the case of a move with a single node belonging to max and N leaves. If the algorithm outputs the answer without having inspected all N leaves, the output can not be guaranteed to be correct. However, in the next subsection we shall see that in important special cases, sublinear evaluation time is possible. 3

4 I max=3 D II min =3 L R I 7 d u max = 3 3 U 1 Figure 5: Computing value for game and subgames 4.1 Sublinear time evaluation of perfect game trees Definition 4 (Perfect Game Tree) A perfect game tree is a special case of a two-player zero-sum finite perfect information game where we furthermore have: In each position there are exactly two options/ each node has exactly two children There is perfect alternation between the two players The tree is perfectly balanced Payoffs are 1 or 1 Theorem 5 (Saks and Wigderson 1986) There is a randomized algorithm that computes the value and a maximin plan for a perfect game tree and runs in expected time O(N log ( ) ), i.e., O(N 0.76 ). In the rest of the section we do the proof for Theorem 5. It is interesting to know that the main result of Saks and Wigderson s paper actually is that no randomized algorithm (which always gives the right answer) has a better running time than the one in Theorem 5. We will not give a proof of this as the result is rather technical. It is a curious fact that the proof uses von Neuman the minmax theorem in an interesting way (and not applied to the game defined by the game tree!). 4

5 As far as Theorem 5, for simplicity we only describe how to compute the value of the game - a plan may be derived in a fairly straightforward way using slightly messy programming code. Recall that when we compute the value in the tree traversal it corresponds to taking the maximum when the node belongs to player I and the minimum for player II. So what we want is an algorithm that is able to evaluate a balanced alternating Max/Min formula with inputs from { 1, 1}. To simplify this, we note that min(a, b) = max( a, b). This converts the formula into one with only max nodes, but with - on all inputs to all nodes (except leaves, but we can add them there too by negating the payoffs at leaves). If we now define Negamax(a, b) = max( a, b), we have a perfectly balanced formula containing only binary Negamax operations. If we evaluate Negamax an all possible inputs we get: Negamax(1,1) = -1, Negamax(-1,1) = 1, Negamax(1,- 1) = 1, Negamax(-1,-1) = 1 and observe if one of the inputs are -1, then Negamax is 1. We use this information in the recusive algorithm, Algorithm 1 for evaluating the expression (we omit the trivial base case of the recursion). Algorithm 1 Eval(Negamax(f 1, f )) 1: Flip a coin and choose i {1, } uniformly at random. : v = Eval(f i ) 3: if ( v = -1 ) then 4: return 1 5: else 6: v = Eval(f 3 i ) 7: if (v =-1) then 8: return 1 9: else 10: return -1 11: end if 1: end if We now analyze the algorithm. Let T v (h) := Expected running time on a tree of height h, when output is v. The output 1 can only occur in one case, that is for N max (1, 1). We get the recurrence: T 1 (h) T 1 (h 1) + c The output 1 can be obtained in three cases, that is for N max ( 1, 1), N max ( 1, 1) and N max (1, 1). The last two are symmetric. In either of the two last cases, with probability 1 both subformula are evaluated and with 5

6 probabiliyt 1 only the one evaluating to 1. This leads to the recurrence: T 1 (h) max{t 1 (h 1) + c, T 1 (h 1) + 1 T 1 (h 1) + c } = 1 T 1 (h 1) + T 1 (h 1) + c By fixing unit of time appropriately we can assume c = 1 and define This gives the following inequalities. g v (h) := T v (h) + 1 g 1 (h) g 1 (h 1) From the above inequalities we get g 1 (h) 1 g1 (h 1) + 1 g 1 (h 1) g 1 (h) 1 g1 (h 1) + g 1 (h ) For small value of c we set g 1 (c) = c, where c is a constant. The theorem of Saks and Wigderson now follows from the following theorem on recurrences whose proof we include as it is short and sweet. Theorem 6 Suppose g : N N satisfies g(n) a 1 g(n 1) + a g(n ) a k g(n k) for a i 0, n k Then g(n) = O(α n ), where α is the unique positive root of the function f : t 1 a 1 t a... a t k t k. Proof (by induction that g(n) cα n for some constant c) Base case: n = 1,..., k, pick c so that it is true. Step: g(n) a 1 g(n 1) + a g(n ) a k g(n k) a 1 cα n 1 + a cα n a k cα n k = cα n (a 1 α 1 + a α a k α k ) = cα n 1 (a 1 α + a 1 α a 1 k α ) k = cα n (1 f(α)) = cα n 6

7 The second inequality holds because of the induction hypothesis and the last equality holds because of α is the unique root of the function f and therefore f(α) = 0. As far as I know it is an open problem to show that the case of perfect game trees is a best case for determining the value. In other words, can it be shown that no family of max-min formulas of arbitrary size N can be evaluated in expected time N γ, where γ < log ( )? Algorithm Eval(max(f 1, f,..., f l, α, β) 1: (Contract: The value is returned if it is in the interval [α, β], otherwise the nearest endpoint is) : Randomly permute f 1, f,..., f l 3: for j = 1 to l do 4: α := max(α, Eval(f j, α, β)) 5: if α β then 6: return β 7: end if 8: end for 9: return α Algorithm 3 Eval(min(f 1, f,..., f l, α, β) 1: (Contract: The value is computed if it is in the interval [α, β], otherwise the nearest endpoint is) : Randomly permute f 1, f,..., f l 3: for j = 1 to l do 4: β := min(β, Eval(f j, α, β)) 5: if α β then 6: return α 7: end if 8: end for 9: return β There are clearly cases that are much worse that perfect trees, like all-or or the all-and formulae. Still, the algorithm itself generalizes and will be sublinear on inputs similar to perfect trees. This is done in the mutually recursive code given as Algorithm and 3. Again, we omit the trivial base case of the recursion. This algorithm is called the α β algorithm and it is very useful in practice, e.g., for constructing chess programs. When called from the outside to evaluate a game tree, it should be called with α = 7

8 and β = to compute the correct value of the given formula. The code can be viewed as an instance of the branch-and-bound method. As usual with branch-and-bound, it matters a lot which subproblem is investigated first. Therefore Randomly in the algorithm is often replaced by Cleverly, in order to minimize execution time. 8