A subexponential lower bound for the Random Facet algorithm for Parity Games

Size: px
Start display at page:

Download "A subexponential lower bound for the Random Facet algorithm for Parity Games"

Transcription

1 A subexponential lower bound for the Random Facet algorithm for Parity Games Oliver Friedmann 1 Thomas Dueholm Hansen 2 Uri Zwick 3 1 Department of Computer Science, University of Munich, Germany. 2 Center for Algorithmic Game Theory, Department of Computer Science, Aarhus University, Denmark. 3 School of Computer Science, Tel Aviv University, Israel. January 23, 2011 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 1/19

2 The RandomFacet algorithm RandomFacet: Randomized pivoting rule for the simplex algorithm for linear programming (LP-type problems). Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 2/19

3 The RandomFacet algorithm RandomFacet: Randomized pivoting rule for the simplex algorithm for linear programming (LP-type problems). Introduced by Matoušek, Sharir and Welzl (1992). A different variant was introduced by Kalai (1992). Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 2/19

4 The RandomFacet algorithm RandomFacet: Randomized pivoting rule for the simplex algorithm for linear programming (LP-type problems). Introduced by Matoušek, Sharir and Welzl (1992). A different variant was introduced by Kalai (1992). Terminates in a subexponential number of steps: 2 O( n log m), (n variables, m constraints). Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 2/19

5 The RandomFacet algorithm RandomFacet: Randomized pivoting rule for the simplex algorithm for linear programming (LP-type problems). Introduced by Matoušek, Sharir and Welzl (1992). A different variant was introduced by Kalai (1992). Terminates in a subexponential number of steps: 2 O( n log m), (n variables, m constraints). This is the best known upper bound in n and m for any algorithm for the problem. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 2/19

6 The RandomFacet algorithm RandomFacet: Randomized pivoting rule for the simplex algorithm for linear programming (LP-type problems). Introduced by Matoušek, Sharir and Welzl (1992). A different variant was introduced by Kalai (1992). Terminates in a subexponential number of steps: 2 O( n log m), (n variables, m constraints). This is the best known upper bound in n and m for any algorithm for the problem. Matoušek (1994): 2 Ω( n) lower bound in abstract setting. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 2/19

7 The RandomFacet algorithm RandomFacet: Randomized pivoting rule for the simplex algorithm for linear programming (LP-type problems). Introduced by Matoušek, Sharir and Welzl (1992). A different variant was introduced by Kalai (1992). Terminates in a subexponential number of steps: 2 O( n log m), (n variables, m constraints). This is the best known upper bound in n and m for any algorithm for the problem. Matoušek (1994): 2 Ω( n) lower bound in abstract setting. Until recently a candidate for solving linear programs in strongly polynomial time: Disproved by the continuation of the work presented here. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 2/19

8 Games and LP-type problems Stochastic games were introduced by Shapley (1953). Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 3/19

9 Games and LP-type problems Stochastic games were introduced by Shapley (1953). Turn-based stochastic games are played on directed graphs with vertices controlled by maximizer, minimizer and nature. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 3/19

10 Games and LP-type problems Stochastic games were introduced by Shapley (1953). Turn-based stochastic games are played on directed graphs with vertices controlled by maximizer, minimizer and nature. Major open problem, first stated by Condon (1992): The problem of solving turn-based stochastic games is in NP conp, but no polynomial time algorithm is known. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 3/19

11 Games and LP-type problems Stochastic games were introduced by Shapley (1953). Turn-based stochastic games are played on directed graphs with vertices controlled by maximizer, minimizer and nature. Major open problem, first stated by Condon (1992): The problem of solving turn-based stochastic games is in NP conp, but no polynomial time algorithm is known. Halman (2007): Turn-based stochastic games are of LP-type. Vertices correspond to variables and edges correspond to constraints. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 3/19

12 Games and LP-type problems Stochastic games were introduced by Shapley (1953). Turn-based stochastic games are played on directed graphs with vertices controlled by maximizer, minimizer and nature. Major open problem, first stated by Condon (1992): The problem of solving turn-based stochastic games is in NP conp, but no polynomial time algorithm is known. Halman (2007): Turn-based stochastic games are of LP-type. Vertices correspond to variables and edges correspond to constraints. The RandomFacet algorithm is the fastest known algorithm for solving turn-based stochastic games. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 3/19

13 Notable special cases 1 player: Markov decision processes (MDPs): maximizer and nature. Introduced by Bellman (1957). Manne (1960): MDPs can be solved by linear programming. No known strongly polynomial time algorithm for MDPs. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 4/19

14 Notable special cases 1 player: Markov decision processes (MDPs): maximizer and nature. Introduced by Bellman (1957). Manne (1960): MDPs can be solved by linear programming. No known strongly polynomial time algorithm for MDPs. Deterministic MDPs: maximizer. Solves the minimum mean cost cycle problem. Karp (1978): O(nm) time algorithm, n vertices and m edges. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 4/19

15 Notable special cases 1 player: Markov decision processes (MDPs): maximizer and nature. Introduced by Bellman (1957). Manne (1960): MDPs can be solved by linear programming. No known strongly polynomial time algorithm for MDPs. Deterministic MDPs: maximizer. Solves the minimum mean cost cycle problem. Karp (1978): O(nm) time algorithm, n vertices and m edges. No vertices controlled by nature: Mean payoff games: maximizer and minimizer. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 4/19

16 Notable special cases 1 player: Markov decision processes (MDPs): maximizer and nature. Introduced by Bellman (1957). Manne (1960): MDPs can be solved by linear programming. No known strongly polynomial time algorithm for MDPs. Deterministic MDPs: maximizer. Solves the minimum mean cost cycle problem. Karp (1978): O(nm) time algorithm, n vertices and m edges. No vertices controlled by nature: Mean payoff games: maximizer and minimizer. Parity games: maximizer and minimizer, special structure. Equivalent to the problem of µ-calculus model checking. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 4/19

17 Overview Abstract LP-type problems Concrete Turn-based stochastic games 2 1 /2 players Linear programming Mean payoff games 2 players Markov decision problems 1 1 /2 players Parity games 2 players Deterministic MDPs 1 player Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 5/19

18 Overview Abstract LP-type problems Concrete Turn-based stochastic games 2 1 /2 players Linear programming Mean payoff games 2 players Markov decision problems 1 1 /2 players Parity games 2 players NP conp Deterministic MDPs 1 player P Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 5/19

19 Main result Theorem The RandomFacet algorithm may require 2 Ω( n/ log n) expected steps to solve n-state parity games, mean payoff games and turn-based stochastic games. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 6/19

20 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

21 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

22 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8, 7 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

23 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8, 7, 5 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

24 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8, 7, 5, 4 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

25 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8, 7, 5, 4, 5 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

26 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8, 7, 5, 4, 5 A (positional) strategy, σ or τ, is an outgoing edge from each vertex controlled by the corresponding player. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

27 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8, 7, 5, 4, 5, 4 A (positional) strategy, σ or τ, is an outgoing edge from each vertex controlled by the corresponding player. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

28 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8, 7, 5, 4, 5, 4, 5 A (positional) strategy, σ or τ, is an outgoing edge from each vertex controlled by the corresponding player. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

29 Parity games EVEN (circle) wins if the largest priority seen infinitely often is even, ODD (square) wins otherwise: Observed priorities: 8, 7, 5, 4, 5, 4, 5, 4,... A (positional) strategy, σ or τ, is an outgoing edge from each vertex controlled by the corresponding player. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 7/19

30 Mean payoff games Priorities are replaced by rewards on edges. Players take the roles of maximizer and minimizer. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 8/19

31 Mean payoff games Priorities are replaced by rewards on edges. Players take the roles of maximizer and minimizer. The value val σ,τ (v) of a vertex v is the average reward of the cycle reached from v when moving according to σ and τ. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 8/19

32 Mean payoff games Priorities are replaced by rewards on edges. Players take the roles of maximizer and minimizer. The value val σ,τ (v) of a vertex v is the average reward of the cycle reached from v when moving according to σ and τ. The path leading to the cycle is of secondary importance. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 8/19

33 Mean payoff games Priorities are replaced by rewards on edges. Players take the roles of maximizer and minimizer. The value val σ,τ (v) of a vertex v is the average reward of the cycle reached from v when moving according to σ and τ. The path leading to the cycle is of secondary importance. Reduction from parity games: ( n) 5 5 ( n) 5 ( n) 5 ( n) 5 A cycle has positive value iff its largest priority is even. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 8/19

34 A simpler view In our lower bound examples we always reach a cycle with value zero (essentially). Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 9/19

35 A simpler view In our lower bound examples we always reach a cycle with value zero (essentially). The path leading to the cycle is then the primary focus. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 9/19

36 A simpler view In our lower bound examples we always reach a cycle with value zero (essentially). The path leading to the cycle is then the primary focus. For simplicity I will use val σ,τ (v) to denote the total sum of rewards on the path to the cycle. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 9/19

37 Optimal strategies and improving switches We generally assume that the minimizer plays an optimal counter-strategy: val σ (v) = min τ val σ,τ (v) Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 10/19

38 Optimal strategies and improving switches We generally assume that the minimizer plays an optimal counter-strategy: val σ (v) = min τ val σ,τ (v) σ is optimal from v if for all σ, val σ (v) val σ (v). Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 10/19

39 Optimal strategies and improving switches We generally assume that the minimizer plays an optimal counter-strategy: val σ (v) = min τ val σ,τ (v) σ is optimal from v if for all σ, val σ (v) val σ (v). σ is optimal if it is optimal from all v. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 10/19

40 Optimal strategies and improving switches We generally assume that the minimizer plays an optimal counter-strategy: val σ (v) = min τ val σ,τ (v) σ is optimal from v if for all σ, val σ (v) val σ (v). σ is optimal if it is optimal from all v. Shapley (1957): Optimal positional strategies are guaranteed to exist. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 10/19

41 Optimal strategies and improving switches We generally assume that the minimizer plays an optimal counter-strategy: val σ (v) = min τ val σ,τ (v) σ is optimal from v if for all σ, val σ (v) val σ (v). σ is optimal if it is optimal from all v. Shapley (1957): Optimal positional strategies are guaranteed to exist. An edge (u, v) is an improving switch w.r.t. σ if the value of u is improved by switching to (u, v): val σ[(u,v)] (u) val σ (u) Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 10/19

42 Optimal strategies and improving switches We generally assume that the minimizer plays an optimal counter-strategy: val σ (v) = min τ val σ,τ (v) σ is optimal from v if for all σ, val σ (v) val σ (v). σ is optimal if it is optimal from all v. Shapley (1957): Optimal positional strategies are guaranteed to exist. An edge (u, v) is an improving switch w.r.t. σ if the value of u is improved by switching to (u, v): val σ[(u,v)] (u) val σ (u) σ is optimal iff there are no improving switches. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 10/19

43 The RandomFacet algorithm Function RandomFacet(G, σ) if E 0 = σ then return σ else Choose e E 0 \ σ uniformly at random σ RandomFacet(G \ {e}, σ) if e is improving switch w.r.t. σ then σ σ [e] return RandomFacet(G, σ ) else return σ Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 11/19

44 Example: Binary choices e σ e σ e 1 σ σ e 1 σ Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 12/19

45 Example: Binary choices e σ e σ e 1 σ σ e 1 σ σ Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 12/19

46 Example: Binary choices e σ e σ e 1 σ σ e 1 σ σ σ Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 12/19

47 Example: Binary choices e σ e σ e 1 σ σ σ e 1 σ σ σ Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 12/19

48 Example: Binary choices e 2 σ e 2 σ e 1 σ σ σ e 1 σ σ σ val(g \ {e 1 }) val(g \ {e 2 })... val(g \ {e n }) Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 12/19

49 Example: Binary choices The number of steps is upper bounded by: f (0) = 1 f (n) = f (n 1) + 1 n 1 f (i) for n > 0 n i=0 The recurrence is bounded by: f (n) = 2 Θ( n) Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 13/19

50 The construction T : 1 0 c 3 : 8 B 3 : 2 b 3 : 2 D 3 : 3 A 3 : 2 a 3 : 2 0 c 2 : 6 B 2 : 2 b 2 : 2 D 2 : 3 A 2 : 2 a 2 : 2 0 c 1 : 4 B 1 : 2 b 1 : 2 D 1 : 3 A 1 : 2 a 1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 14/19

51 The construction T : 1 0 c 3 : 8 B 3 : 2 b 3 : 2 D 3 : 3 A 3 : 2 a 3 : 2 0 c 2 : 6 B 2 : 2 b 2 : 2 D 2 : 3 A 2 : 2 a 2 : 2 0 c 1 : 4 B 1 : 2 b 1 : 2 D 1 : 3 A 1 : 2 a 1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 14/19

52 The construction T : 1 0 c 3 : 8 B 3 : 2 b 3 : 2 D 3 : 3 A 3 : 2 a 3 : 2 1 c 2 : 6 B 2 : 2 b 2 : 2 D 2 : 3 A 2 : 2 a 2 : 2 1 c 1 : 4 B 1 : 2 b 1 : 2 D 1 : 3 A 1 : 2 a 1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 14/19

53 The construction T : 1 1 c 3 : 8 B 3 : 2 b 3 : 2 D 3 : 3 A 3 : 2 a 3 : 2 1 c 2 : 6 B 2 : 2 b 2 : 2 D 2 : 3 A 2 : 2 a 2 : 2 1 c 1 : 4 B 1 : 2 b 1 : 2 D 1 : 3 A 1 : 2 a 1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 14/19

54 The construction T : 1 1 c 3 : 8 B 3 : 2 b 3 : 2 D 3 : 3 A 3 : 2 a 3 : 2 1 c 2 : 6 B 2 : 2 b 2 : 2 D 2 : 3 A 2 : 2 a 2 : 2 1 c 1 : 4 B 1 : 2 b 1 : 2 D 1 : 3 A 1 : 2 a 1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 14/19

55 The construction T : 1 1 c 3 : 8 B 3 : 2 b 3 : 2 D 3 : 3 A 3 : 2 a 3 : 2 0 c 2 : 6 B 2 : 2 b 2 : 2 D 2 : 3 A 2 : 2 a 2 : 2 0 c 1 : 4 B 1 : 2 b 1 : 2 D 1 : 3 A 1 : 2 a 1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 14/19

56 The construction T : 1 1 c 3 : 8 B 3 : 2 b 3 : 2 D 3 : 3 A 3 : 2 a 3 : 2 0 c 2 : 6 B 2 : 2 b 2 : 2 D 2 : 3 A 2 : 2 a 2 : 2 0 c 1 : 4 B 1 : 2 b 1 : 2 D 1 : 3 A 1 : 2 a 1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 14/19

57 Randomized bitcounter Start with n bits with value 0: Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 15/19

58 Randomized bitcounter Start with n bits with value 0: Pick a random bit i and fix it: Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 15/19

59 Randomized bitcounter Start with n bits with value 0: Pick a random bit i and fix it: Count recursively with the remaining n 1 bits: Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 15/19

60 Randomized bitcounter Start with n bits with value 0: Pick a random bit i and fix it: Count recursively with the remaining n 1 bits: Increment the i th bit: Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 15/19

61 Randomized bitcounter Start with n bits with value 0: Pick a random bit i and fix it: Count recursively with the remaining n 1 bits: Increment the i th bit: Reset the i 1 lower bits: Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 15/19

62 Randomized bitcounter Start with n bits with value 0: Pick a random bit i and fix it: Count recursively with the remaining n 1 bits: Increment the i th bit: Reset the i 1 lower bits: Count recursively with the i 1 lower bits: Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 15/19

63 Randomized bitcounter Start with n bits with value 0: Pick a random bit i and fix it: Count recursively with the remaining n 1 bits: Increment the i th bit: Reset the i 1 lower bits: Count recursively with the i 1 lower bits: Expected number of steps: f (0) = 1 f (n) = f (n 1) + 1 n 1 f (i) for n > 0 n i=0 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 15/19

64 Randomized bitcounter Start with n bits with value 0: Pick a random bit i and fix it: Count recursively with the remaining n 1 bits: Increment the i th bit: Reset the i 1 lower bits: Count recursively with the i 1 lower bits: Expected number of steps: f (0) = 1 f (n) = f (n 1) + 1 n 1 f (i) for n > 0 n i=0 Same recurrence as for upper bound: f (n) = 2 Θ( n) Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 15/19

65 The modified RandomFacet algorithm Function RandomFacet (G, σ, ϕ) if E 0 = σ then return σ else e argmin e E 0 \σ ϕ(e ) σ RandomFacet (G \ {e}, σ, ϕ) if e is improving switch w.r.t. σ then σ σ [e] return RandomFacet (G, σ, ϕ) else return σ Same expected number of steps when the permutation ϕ is picked uniformly at random. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 16/19

66 Ensuring worst-case behavior Good permutation: 1 i : ϕ(b i, B i ) < ϕ(a i, A i ) T : 1 c3 : 8 B3 : 2 b3 : 2 D3 : 3 A3 : 2 a3 : 2 c2 : 6 B2 : 2 b2 : 2 D2 : 3 A2 : 2 a2 : 2 c1 : 4 B1 : 2 b1 : 2 D1 : 3 A1 : 2 a1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 17/19

67 Ensuring worst-case behavior Good permutation: 1 i : ϕ(b i, B i ) < ϕ(a i, A i ) c3 : 8 B3 : 2 l T : 1 b3 : 2 Increase probability of picking a good permutation at random by duplication: Pr[(1) not satisfied] n (l!)2 (2l)! n 2 l l D3 : 3 A3 : 2 a3 : 2 l c2 : 6 B2 : 2 b2 : 2 l D2 : 3 A2 : 2 a2 : 2 l c1 : 4 B1 : 2 b1 : 2 l D1 : 3 A1 : 2 a1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 17/19

68 Ensuring worst-case behavior Good permutation: 1 i : ϕ(b i, B i ) < ϕ(a i, A i ) c3 : 8 B3 : 2 l T : 1 b3 : 2 Increase probability of picking a good permutation at random by duplication: Pr[(1) not satisfied] n (l!)2 (2l)! n 2 l Here we lose a logarithmic factor in the exponent. l D3 : 3 A3 : 2 a3 : 2 l c2 : 6 B2 : 2 b2 : 2 l D2 : 3 A2 : 2 a2 : 2 l c1 : 4 B1 : 2 b1 : 2 l D1 : 3 A1 : 2 a1 : 2 Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 17/19

69 Concluding remarks We constructed parity games where the expected running time of the RandomFacet algorithm is 2 Ω( n), almost matching the upper bound by Matoušek, Sharir and Welzl (1992). Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 18/19

70 Concluding remarks We constructed parity games where the expected running time of the RandomFacet algorithm is 2 Ω( n), almost matching the upper bound by Matoušek, Sharir and Welzl (1992). By replacing the minimizer with vertices controlled by nature, we have later managed to prove the same lower bound for MDPs and linear programming. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 18/19

71 Concluding remarks We constructed parity games where the expected running time of the RandomFacet algorithm is 2 Ω( n), almost matching the upper bound by Matoušek, Sharir and Welzl (1992). By replacing the minimizer with vertices controlled by nature, we have later managed to prove the same lower bound for MDPs and linear programming. Using similar techniques we have also managed to show that RandomEdge (repeatedly switch a random improving edge), when applied to the same settings, may require 2 Ω(n1/4 ) expected steps. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 18/19

72 Concluding remarks We constructed parity games where the expected running time of the RandomFacet algorithm is 2 Ω( n), almost matching the upper bound by Matoušek, Sharir and Welzl (1992). By replacing the minimizer with vertices controlled by nature, we have later managed to prove the same lower bound for MDPs and linear programming. Using similar techniques we have also managed to show that RandomEdge (repeatedly switch a random improving edge), when applied to the same settings, may require 2 Ω(n1/4 ) expected steps. Major open problem: Polynomial time algorithm for Parity Games. Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 18/19

73 The end Thank you for listening! Friedmann, Hansen, Zwick Lower bound for Random Facet algorithm Page 19/19

Dantzig s pivoting rule for shortest paths, deterministic MDPs, and minimum cost to time ratio cycles

Dantzig s pivoting rule for shortest paths, deterministic MDPs, and minimum cost to time ratio cycles Dantzig s pivoting rule for shortest paths, deterministic MDPs, and minimum cost to time ratio cycles Thomas Dueholm Hansen 1 Haim Kaplan Uri Zwick 1 Department of Management Science and Engineering, Stanford

More information

Lecture notes for Analysis of Algorithms : Markov decision processes

Lecture notes for Analysis of Algorithms : Markov decision processes Lecture notes for Analysis of Algorithms : Markov decision processes Lecturer: Thomas Dueholm Hansen June 6, 013 Abstract We give an introduction to infinite-horizon Markov decision processes (MDPs) with

More information

Algorithmic Game Theory and Applications. Lecture 15: a brief taster of Markov Decision Processes and Stochastic Games

Algorithmic Game Theory and Applications. Lecture 15: a brief taster of Markov Decision Processes and Stochastic Games Algorithmic Game Theory and Applications Lecture 15: a brief taster of Markov Decision Processes and Stochastic Games Kousha Etessami warning 1 The subjects we will touch on today are so interesting and

More information

The Simplex and Policy Iteration Methods are Strongly Polynomial for the Markov Decision Problem with Fixed Discount

The Simplex and Policy Iteration Methods are Strongly Polynomial for the Markov Decision Problem with Fixed Discount The Simplex and Policy Iteration Methods are Strongly Polynomial for the Markov Decision Problem with Fixed Discount Yinyu Ye Department of Management Science and Engineering and Institute of Computational

More information

The Complexity of Solving Stochastic Games on Graphs

The Complexity of Solving Stochastic Games on Graphs The Complexity of Solving Stochastic Games on Graphs Daniel Andersson and Peter Bro Miltersen Department of Computer Science, Aarhus University, Denmark Abstract. We consider some well-known families of

More information

A Note on the Approximation of Mean-Payoff Games

A Note on the Approximation of Mean-Payoff Games A Note on the Approximation of Mean-Payoff Games Raffaella Gentilini 1 1 University of Perugia, Italy Abstract. We consider the problem of designing approximation schemes for the values of mean-payoff

More information

The Complexity of the Simplex Method

The Complexity of the Simplex Method The Complexity of the Simplex Method John Fearnley University of Liverpool Liverpool United Kingdom john.fearnley@liverpool.ac.uk Rahul Savani University of Liverpool Liverpool United Kingdom rahul.savani@liverpool.ac.uk

More information

Section Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018

Section Notes 9. Midterm 2 Review. Applied Math / Engineering Sciences 121. Week of December 3, 2018 Section Notes 9 Midterm 2 Review Applied Math / Engineering Sciences 121 Week of December 3, 2018 The following list of topics is an overview of the material that was covered in the lectures and sections

More information

CS 573: Algorithmic Game Theory Lecture date: January 23rd, 2008

CS 573: Algorithmic Game Theory Lecture date: January 23rd, 2008 CS 573: Algorithmic Game Theory Lecture date: January 23rd, 2008 Instructor: Chandra Chekuri Scribe: Bolin Ding Contents 1 2-Player Zero-Sum Game 1 1.1 Min-Max Theorem for 2-Player Zero-Sum Game....................

More information

Motivating examples Introduction to algorithms Simplex algorithm. On a particular example General algorithm. Duality An application to game theory

Motivating examples Introduction to algorithms Simplex algorithm. On a particular example General algorithm. Duality An application to game theory Instructor: Shengyu Zhang 1 LP Motivating examples Introduction to algorithms Simplex algorithm On a particular example General algorithm Duality An application to game theory 2 Example 1: profit maximization

More information

Perfect-information Stochastic Parity Games

Perfect-information Stochastic Parity Games Perfect-information Stochastic Parity Games Wies law Zielonka LIAFA, case 7014 Université Paris 7 2, Place Jussieu 75251 Paris Cedex 05, France zielonka@liafa.jussieu.fr Abstract. We show that in perfect-information

More information

arxiv: v2 [cs.ds] 29 Apr 2016

arxiv: v2 [cs.ds] 29 Apr 2016 Polynomial-Time Algorithms for Energy Games with Special Weight Structures Krishnendu Chatterjee Monika Henzinger Sebastian Krinninger Danupon Nanongkai arxiv:1604.08234v2 [cs.ds] 29 Apr 2016 Abstract

More information

A reduction from parity games to simple stochastic games

A reduction from parity games to simple stochastic games A reduction from parity games to simple stochastic games Krishnendu Chatterjee IST Austria (Institute of Science and Technology, Austria) krishnendu.chatterjee@ist.ac.at Nathanaël Fijalkow IST Austria

More information

Theoretical Computer Science

Theoretical Computer Science Theoretical Computer Science 458 (2012) 49 60 Contents lists available at SciVerse ScienceDirect Theoretical Computer Science journal homepage: www.elsevier.com/locate/tcs Energy parity games Krishnendu

More information

The Complexity of Ergodic Mean-payoff Games,

The Complexity of Ergodic Mean-payoff Games, The Complexity of Ergodic Mean-payoff Games, Krishnendu Chatterjee Rasmus Ibsen-Jensen Abstract We study two-player (zero-sum) concurrent mean-payoff games played on a finite-state graph. We focus on the

More information

The priority promotion approach to parity games

The priority promotion approach to parity games The priority promotion approach to parity games Massimo Benerecetti 1, Daniele Dell Erba 1, and Fabio Mogavero 2 1 Università degli Studi di Napoli Federico II 2 Università degli Studi di Verona Abstract.

More information

Randomness for Free. 1 Introduction. Krishnendu Chatterjee 1, Laurent Doyen 2, Hugo Gimbert 3, and Thomas A. Henzinger 1

Randomness for Free. 1 Introduction. Krishnendu Chatterjee 1, Laurent Doyen 2, Hugo Gimbert 3, and Thomas A. Henzinger 1 Randomness for Free Krishnendu Chatterjee 1, Laurent Doyen 2, Hugo Gimbert 3, and Thomas A. Henzinger 1 1 IST Austria (Institute of Science and Technology Austria) 2 LSV, ENS Cachan & CNRS, France 3 LaBri

More information

Faster Pseudopolynomial Algorithms for Mean-Payoff Games

Faster Pseudopolynomial Algorithms for Mean-Payoff Games Faster Pseudopolynomial Algorithms for Mean-Payoff Games 1 Faster Pseudopolynomial Algorithms for Mean-Payoff Games L. Doyen, R. Gentilini, and J.-F. Raskin Univ. Libre de Bruxelles Faster Pseudopolynomial

More information

A Survey of Partial-Observation Stochastic Parity Games

A Survey of Partial-Observation Stochastic Parity Games Noname manuscript No. (will be inserted by the editor) A Survey of Partial-Observation Stochastic Parity Games Krishnendu Chatterjee Laurent Doyen Thomas A. Henzinger the date of receipt and acceptance

More information

A Survey of Stochastic ω-regular Games

A Survey of Stochastic ω-regular Games A Survey of Stochastic ω-regular Games Krishnendu Chatterjee Thomas A. Henzinger EECS, University of California, Berkeley, USA Computer and Communication Sciences, EPFL, Switzerland {c krish,tah}@eecs.berkeley.edu

More information

On positional strategies over finite arenas

On positional strategies over finite arenas On positional strategies over finite arenas Damian Niwiński University of Warsaw joint work with Thomas Colcombet Berlin 2018 Disclaimer. Credits to many authors. All errors are mine own. 1 Perfect information

More information

THE STEVENS-STIRLING-ALGORITHM FOR SOLVING PARITY GAMES LOCALLY REQUIRES EXPONENTIAL TIME

THE STEVENS-STIRLING-ALGORITHM FOR SOLVING PARITY GAMES LOCALLY REQUIRES EXPONENTIAL TIME International Journal of Foundations of Computer Science c World Scientific Publishing Company THE STEVENS-STIRLING-ALGORITHM FOR SOLVING PARITY GAMES LOCALLY REQUIRES EXPONENTIAL TIME OLIVER FRIEDMANN

More information

Mean-Payoff Games and the Max-Atom Problem

Mean-Payoff Games and the Max-Atom Problem Mean-Payoff Games and the Max-Atom Problem Albert Atserias Universitat Politècnica de Catalunya Barcelona, Spain Elitza Maneva Universitat Politècnica de Catalunya Barcelona, Spain February 3, 200 Abstract

More information

CS 580: Algorithm Design and Analysis. Jeremiah Blocki Purdue University Spring 2018

CS 580: Algorithm Design and Analysis. Jeremiah Blocki Purdue University Spring 2018 CS 580: Algorithm Design and Analysis Jeremiah Blocki Purdue University Spring 2018 Chapter 9 PSPACE: A Class of Problems Beyond NP Slides by Kevin Wayne. Copyright @ 2005 Pearson-Addison Wesley. All rights

More information

The Stevens-Stirling-Algorithm For Solving Parity Games Locally Requires Exponential Time

The Stevens-Stirling-Algorithm For Solving Parity Games Locally Requires Exponential Time The Stevens-Stirling-Algorithm For Solving Parity Games Locally Requires Exponential Time Oliver Friedmann Institut für Informatik, LMU München Oliver.Friedmann@googlemail.com Abstract. This paper presents

More information

Lecture 5: The Principle of Deferred Decisions. Chernoff Bounds

Lecture 5: The Principle of Deferred Decisions. Chernoff Bounds Randomized Algorithms Lecture 5: The Principle of Deferred Decisions. Chernoff Bounds Sotiris Nikoletseas Associate Professor CEID - ETY Course 2013-2014 Sotiris Nikoletseas, Associate Professor Randomized

More information

The Complexity of Nash Equilibria in Simple Stochastic Multiplayer Games *

The Complexity of Nash Equilibria in Simple Stochastic Multiplayer Games * The Complexity of Nash Equilibria in Simple Stochastic Multiplayer Games * Michael Ummels 1 and Dominik Wojtczak 2,3 1 RWTH Aachen University, Germany E-Mail: ummels@logic.rwth-aachen.de 2 CWI, Amsterdam,

More information

Lecture notes on OPP algorithms [Preliminary Draft]

Lecture notes on OPP algorithms [Preliminary Draft] Lecture notes on OPP algorithms [Preliminary Draft] Jesper Nederlof June 13, 2016 These lecture notes were quickly assembled and probably contain many errors. Use at your own risk! Moreover, especially

More information

Discrete and Lexicographic Helly Theorems and their Relations to LP-Type Problems

Discrete and Lexicographic Helly Theorems and their Relations to LP-Type Problems TEL AVIV UNIVERSITY The Raymond and Beverly Sackler Faculty of Exact Sciences School of Mathematical Sciences Discrete and Lexicographic Helly Theorems and their Relations to LP-Type Problems Thesis submitted

More information

ON THE ALGORITHMIC ASPECTS OF DISCRETE AND LEXICOGRAPHIC HELLY-TYPE THEOREMS AND THE DISCRETE LP-TYPE MODEL

ON THE ALGORITHMIC ASPECTS OF DISCRETE AND LEXICOGRAPHIC HELLY-TYPE THEOREMS AND THE DISCRETE LP-TYPE MODEL SIAM J. COMPUT. Vol. 0, No. 0, pp. 000 000 c 200X Society for Industrial and Applied Mathematics ON THE ALGORITHMIC ASPECTS OF DISCRETE AND LEXICOGRAPHIC HELLY-TYPE THEOREMS AND THE DISCRETE LP-TYPE MODEL

More information

MDPs with Energy-Parity Objectives

MDPs with Energy-Parity Objectives MDPs with Energy-Parity Objectives Richard Mayr, Sven Schewe, Patrick Totzke, Dominik Wojtczak University of Edinburgh, UK University of Liverpool, UK Abstract Energy-parity objectives combine ω-regular

More information

Chapter 9. PSPACE: A Class of Problems Beyond NP. Slides by Kevin Wayne Pearson-Addison Wesley. All rights reserved.

Chapter 9. PSPACE: A Class of Problems Beyond NP. Slides by Kevin Wayne Pearson-Addison Wesley. All rights reserved. Chapter 9 PSPACE: A Class of Problems Beyond NP Slides by Kevin Wayne. Copyright @ 2005 Pearson-Addison Wesley. All rights reserved. 1 Geography Game Geography. Alice names capital city c of country she

More information

A lower bound for discounting algorithms solving two-person zero-sum limit average payoff stochastic games

A lower bound for discounting algorithms solving two-person zero-sum limit average payoff stochastic games R u t c o r Research R e p o r t A lower bound for discounting algorithms solving two-person zero-sum limit average payoff stochastic games Endre Boros a Vladimir Gurvich c Khaled Elbassioni b Kazuhisa

More information

MS&E338 Reinforcement Learning Lecture 1 - April 2, Introduction

MS&E338 Reinforcement Learning Lecture 1 - April 2, Introduction MS&E338 Reinforcement Learning Lecture 1 - April 2, 2018 Introduction Lecturer: Ben Van Roy Scribe: Gabriel Maher 1 Reinforcement Learning Introduction In reinforcement learning (RL) we consider an agent

More information

Umans Complexity Theory Lectures

Umans Complexity Theory Lectures Umans Complexity Theory Lectures Lecture 12: The Polynomial-Time Hierarchy Oracle Turing Machines Oracle Turing Machine (OTM): Deterministic multitape TM M with special query tape special states q?, q

More information

9. PSPACE 9. PSPACE. PSPACE complexity class quantified satisfiability planning problem PSPACE-complete

9. PSPACE 9. PSPACE. PSPACE complexity class quantified satisfiability planning problem PSPACE-complete Geography game Geography. Alice names capital city c of country she is in. Bob names a capital city c' that starts with the letter on which c ends. Alice and Bob repeat this game until one player is unable

More information

9. PSPACE. PSPACE complexity class quantified satisfiability planning problem PSPACE-complete

9. PSPACE. PSPACE complexity class quantified satisfiability planning problem PSPACE-complete 9. PSPACE PSPACE complexity class quantified satisfiability planning problem PSPACE-complete Lecture slides by Kevin Wayne Copyright 2005 Pearson-Addison Wesley Copyright 2013 Kevin Wayne http://www.cs.princeton.edu/~wayne/kleinberg-tardos

More information

THE BIG MATCH WITH A CLOCK AND A BIT OF MEMORY

THE BIG MATCH WITH A CLOCK AND A BIT OF MEMORY THE BIG MATCH WITH A CLOCK AND A BIT OF MEMORY KRISTOFFER ARNSFELT HANSEN, RASMUS IBSEN-JENSEN, AND ABRAHAM NEYMAN Abstract. The Big Match is a multi-stage two-player game. In each stage Player hides one

More information

The Small Progress Measures algorithm for Parity games

The Small Progress Measures algorithm for Parity games The Small Progress Measures algorithm for Parity games Material: Small Progress Measures for Solving Parity Games, Marcin Jurdziński Jeroen J.A. Keiren jkeiren@win.tue.nl http://www.win.tue.nl/ jkeiren

More information

An Introduction to Randomized algorithms

An Introduction to Randomized algorithms An Introduction to Randomized algorithms C.R. Subramanian The Institute of Mathematical Sciences, Chennai. Expository talk presented at the Research Promotion Workshop on Introduction to Geometric and

More information

Chapter 0 Introduction Suppose this was the abstract of a journal paper rather than the introduction to a dissertation. Then it would probably end wit

Chapter 0 Introduction Suppose this was the abstract of a journal paper rather than the introduction to a dissertation. Then it would probably end wit Chapter 0 Introduction Suppose this was the abstract of a journal paper rather than the introduction to a dissertation. Then it would probably end with some cryptic AMS subject classications and a few

More information

Complexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler

Complexity Theory VU , SS The Polynomial Hierarchy. Reinhard Pichler Complexity Theory Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität Wien 15 May, 2018 Reinhard

More information

SOLUTION: SOLUTION: SOLUTION:

SOLUTION: SOLUTION: SOLUTION: Convert R and S into nondeterministic finite automata N1 and N2. Given a string s, if we know the states N1 and N2 may reach when s[1...i] has been read, we are able to derive the states N1 and N2 may

More information

Outline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181.

Outline. Complexity Theory EXACT TSP. The Class DP. Definition. Problem EXACT TSP. Complexity of EXACT TSP. Proposition VU 181. Complexity Theory Complexity Theory Outline Complexity Theory VU 181.142, SS 2018 6. The Polynomial Hierarchy Reinhard Pichler Institut für Informationssysteme Arbeitsbereich DBAI Technische Universität

More information

Zero-Sum Stochastic Games An algorithmic review

Zero-Sum Stochastic Games An algorithmic review Zero-Sum Stochastic Games An algorithmic review Emmanuel Hyon LIP6/Paris Nanterre with N Yemele and L Perrotin Rosario November 2017 Final Meeting Dygame Dygame Project Amstic Outline 1 Introduction Static

More information

A Polynomial-time Nash Equilibrium Algorithm for Repeated Games

A Polynomial-time Nash Equilibrium Algorithm for Repeated Games A Polynomial-time Nash Equilibrium Algorithm for Repeated Games Michael L. Littman mlittman@cs.rutgers.edu Rutgers University Peter Stone pstone@cs.utexas.edu The University of Texas at Austin Main Result

More information

Perfect-Information Stochastic Mean-Payo Parity Games

Perfect-Information Stochastic Mean-Payo Parity Games Perfect-Information Stochastic Mean-Payo Parity Games Krishnendu Chatterjee and Laurent Doyen and Hugo Gimbert and Youssouf Oualhadj Technical Report No. IST-2013-128-v1+1 Deposited at UNSPECIFIED http://repository.ist.ac.at/128/1/full

More information

Deciding the Winner in Parity Games Is in UP co-up

Deciding the Winner in Parity Games Is in UP co-up c 1998 Elsevier Science B.V. Marcin Jurdziński, Deciding the Winner in Parity Games Is in UP co-up. Information Processing Letters 68(3): 119 124, November 1998. Personal use of this material for research/educational

More information

Multi-Armed Bandit: Learning in Dynamic Systems with Unknown Models

Multi-Armed Bandit: Learning in Dynamic Systems with Unknown Models c Qing Zhao, UC Davis. Talk at Xidian Univ., September, 2011. 1 Multi-Armed Bandit: Learning in Dynamic Systems with Unknown Models Qing Zhao Department of Electrical and Computer Engineering University

More information

Distributed Optimization. Song Chong EE, KAIST

Distributed Optimization. Song Chong EE, KAIST Distributed Optimization Song Chong EE, KAIST songchong@kaist.edu Dynamic Programming for Path Planning A path-planning problem consists of a weighted directed graph with a set of n nodes N, directed links

More information

CPSC 320 Sample Final Examination December 2013

CPSC 320 Sample Final Examination December 2013 CPSC 320 Sample Final Examination December 2013 [10] 1. Answer each of the following questions with true or false. Give a short justification for each of your answers. [5] a. 6 n O(5 n ) lim n + This is

More information

Experts in a Markov Decision Process

Experts in a Markov Decision Process University of Pennsylvania ScholarlyCommons Statistics Papers Wharton Faculty Research 2004 Experts in a Markov Decision Process Eyal Even-Dar Sham Kakade University of Pennsylvania Yishay Mansour Follow

More information

Boundedness Games. Séminaire du LIGM, April 16th, Nathanaël Fijalkow. Institute of Informatics, Warsaw University Poland

Boundedness Games. Séminaire du LIGM, April 16th, Nathanaël Fijalkow. Institute of Informatics, Warsaw University Poland Boundedness Games Séminaire du LIGM, April 16th, 2013 Nathanaël Fijalkow Institute of Informatics, Warsaw University Poland LIAFA, Université Paris 7 Denis Diderot France (based on joint works with Krishnendu

More information

Optimization 4. GAME THEORY

Optimization 4. GAME THEORY Optimization GAME THEORY DPK Easter Term Saddle points of two-person zero-sum games We consider a game with two players Player I can choose one of m strategies, indexed by i =,, m and Player II can choose

More information

arxiv: v3 [cs.gt] 10 Apr 2009

arxiv: v3 [cs.gt] 10 Apr 2009 The Complexity of Nash Equilibria in Simple Stochastic Multiplayer Games Michael Ummels and Dominik Wojtczak,3 RWTH Aachen University, Germany E-Mail: ummels@logicrwth-aachende CWI, Amsterdam, The Netherlands

More information

Stochastic Games with Time The value Min strategies Max strategies Determinacy Finite-state games Cont.-time Markov chains

Stochastic Games with Time The value Min strategies Max strategies Determinacy Finite-state games Cont.-time Markov chains Games with Time Finite-state Masaryk University Brno GASICS 00 /39 Outline Finite-state stochastic processes. Games over event-driven stochastic processes. Strategies,, determinacy. Existing results for

More information

Lecture 6: April 25, 2006

Lecture 6: April 25, 2006 Computational Game Theory Spring Semester, 2005/06 Lecture 6: April 25, 2006 Lecturer: Yishay Mansour Scribe: Lior Gavish, Andrey Stolyarenko, Asaph Arnon Partially based on scribe by Nataly Sharkov and

More information

Week 2: Defining Computation

Week 2: Defining Computation Computational Complexity Theory Summer HSSP 2018 Week 2: Defining Computation Dylan Hendrickson MIT Educational Studies Program 2.1 Turing Machines Turing machines provide a simple, clearly defined way

More information

CS261: Problem Set #3

CS261: Problem Set #3 CS261: Problem Set #3 Due by 11:59 PM on Tuesday, February 23, 2016 Instructions: (1) Form a group of 1-3 students. You should turn in only one write-up for your entire group. (2) Submission instructions:

More information

Lecture 23: More PSPACE-Complete, Randomized Complexity

Lecture 23: More PSPACE-Complete, Randomized Complexity 6.045 Lecture 23: More PSPACE-Complete, Randomized Complexity 1 Final Exam Information Who: You On What: Everything through PSPACE (today) With What: One sheet (double-sided) of notes are allowed When:

More information

Recitation 6. Randomization. 6.1 Announcements. RandomLab has been released, and is due Monday, October 2. It s worth 100 points.

Recitation 6. Randomization. 6.1 Announcements. RandomLab has been released, and is due Monday, October 2. It s worth 100 points. Recitation 6 Randomization 6.1 Announcements RandomLab has been released, and is due Monday, October 2. It s worth 100 points. FingerLab will be released after Exam I, which is going to be on Wednesday,

More information

Long-run Average Reward for Markov Decision Processes

Long-run Average Reward for Markov Decision Processes Long-run Average Reward for Markov Decision Processes Based on a paper at CAV 2017 Pranav Ashok 1, Krishnendu Chatterjee 2, Przemysław Daca 2, Jan Křetínský 1 and Tobias Meggendorfer 1 August 9, 2017 1

More information

Computational complexity estimates for value and policy iteration algorithms for total-cost and average-cost Markov decision processes

Computational complexity estimates for value and policy iteration algorithms for total-cost and average-cost Markov decision processes Computational complexity estimates for value and policy iteration algorithms for total-cost and average-cost Markov decision processes Jefferson Huang Dept. Applied Mathematics and Statistics Stony Brook

More information

Infinite Games. Sumit Nain. 28 January Slides Credit: Barbara Jobstmann (CNRS/Verimag) Department of Computer Science Rice University

Infinite Games. Sumit Nain. 28 January Slides Credit: Barbara Jobstmann (CNRS/Verimag) Department of Computer Science Rice University Infinite Games Sumit Nain Department of Computer Science Rice University 28 January 2013 Slides Credit: Barbara Jobstmann (CNRS/Verimag) Motivation Abstract games are of fundamental importance in mathematics

More information

Recursive Stochastic Games with Positive Rewards

Recursive Stochastic Games with Positive Rewards Recursive Stochastic Games with Positive Rewards K. Etessami 1, D. Wojtczak 1, and M. Yannakakis 2 1 LFCS, School of Informatics, University of Edinburgh 2 Dept. of Computer Science, Columbia University

More information

Łukasz Kaiser Joint work with Diana Fischer and Erich Grädel

Łukasz Kaiser Joint work with Diana Fischer and Erich Grädel Quantitative Systems, Modal Logics and Games Łukasz Kaiser Joint work with Diana Fischer and Erich Grädel Mathematische Grundlagen der Informatik RWTH Aachen AlgoSyn, February 28 Quantitative Systems (Łukasz

More information

Notes from Week 9: Multi-Armed Bandit Problems II. 1 Information-theoretic lower bounds for multiarmed

Notes from Week 9: Multi-Armed Bandit Problems II. 1 Information-theoretic lower bounds for multiarmed CS 683 Learning, Games, and Electronic Markets Spring 007 Notes from Week 9: Multi-Armed Bandit Problems II Instructor: Robert Kleinberg 6-30 Mar 007 1 Information-theoretic lower bounds for multiarmed

More information

Value Iteration. 1 Introduction. Krishnendu Chatterjee 1 and Thomas A. Henzinger 1,2

Value Iteration. 1 Introduction. Krishnendu Chatterjee 1 and Thomas A. Henzinger 1,2 Value Iteration Krishnendu Chatterjee 1 and Thomas A. Henzinger 1,2 1 University of California, Berkeley 2 EPFL, Switzerland Abstract. We survey value iteration algorithms on graphs. Such algorithms can

More information

, and rewards and transition matrices as shown below:

, and rewards and transition matrices as shown below: CSE 50a. Assignment 7 Out: Tue Nov Due: Thu Dec Reading: Sutton & Barto, Chapters -. 7. Policy improvement Consider the Markov decision process (MDP) with two states s {0, }, two actions a {0, }, discount

More information

Computing Minmax; Dominance

Computing Minmax; Dominance Computing Minmax; Dominance CPSC 532A Lecture 5 Computing Minmax; Dominance CPSC 532A Lecture 5, Slide 1 Lecture Overview 1 Recap 2 Linear Programming 3 Computational Problems Involving Maxmin 4 Domination

More information

Introduction to Linear Programming

Introduction to Linear Programming Nanjing University October 27, 2011 What is LP The Linear Programming Problem Definition Decision variables Objective Function x j, j = 1, 2,..., n ζ = n c i x i i=1 We will primarily discuss maxizming

More information

CSC 8301 Design & Analysis of Algorithms: Lower Bounds

CSC 8301 Design & Analysis of Algorithms: Lower Bounds CSC 8301 Design & Analysis of Algorithms: Lower Bounds Professor Henry Carter Fall 2016 Recap Iterative improvement algorithms take a feasible solution and iteratively improve it until optimized Simplex

More information

Selecting Efficient Correlated Equilibria Through Distributed Learning. Jason R. Marden

Selecting Efficient Correlated Equilibria Through Distributed Learning. Jason R. Marden 1 Selecting Efficient Correlated Equilibria Through Distributed Learning Jason R. Marden Abstract A learning rule is completely uncoupled if each player s behavior is conditioned only on his own realized

More information

RECURSIVE CONCURRENT STOCHASTIC GAMES

RECURSIVE CONCURRENT STOCHASTIC GAMES RECURSIVE CONCURRENT STOCHASTIC GAMES KOUSHA ETESSAMI a AND MIHALIS YANNAKAKIS b a LFCS, School of Informatics, University of Edinburgh, UK e-mail address: kousha@inf.ed.ac.uk b Department of Computer

More information

Infinite-Duration Bidding Games

Infinite-Duration Bidding Games Infinite-Duration Bidding Games Guy Avni 1, Thomas A. Henzinger 2, and Ventsislav Chonev 3 1 IST Austria, Klosterneuburg, Austria 2 IST Austria, Klosterneuburg, Austria 3 Max Planck Institute for Software

More information

Totally Corrective Boosting Algorithms that Maximize the Margin

Totally Corrective Boosting Algorithms that Maximize the Margin Totally Corrective Boosting Algorithms that Maximize the Margin Manfred K. Warmuth 1 Jun Liao 1 Gunnar Rätsch 2 1 University of California, Santa Cruz 2 Friedrich Miescher Laboratory, Tübingen, Germany

More information

Nash-solvable bidirected cyclic two-person game forms

Nash-solvable bidirected cyclic two-person game forms DIMACS Technical Report 2008-13 November 2008 Nash-solvable bidirected cyclic two-person game forms by Endre Boros 1 RUTCOR, Rutgers University 640 Bartholomew Road, Piscataway NJ 08854-8003 boros@rutcor.rutgers.edu

More information

arxiv: v1 [cs.gt] 4 Apr 2017

arxiv: v1 [cs.gt] 4 Apr 2017 Communication Complexity of Correlated Equilibrium in Two-Player Games Anat Ganor Karthik C. S. Abstract arxiv:1704.01104v1 [cs.gt] 4 Apr 2017 We show a communication complexity lower bound for finding

More information

Linear Programming Randomization and Abstract Frameworks

Linear Programming Randomization and Abstract Frameworks appeared in Proc. 13th Ann. Symp. on Theoret. Aspects of Comput. Sci. Lecture Notes in Computer Science 1046 (1996) 669-687. Linear Programming Randomization and Abstract Frameworks Bernd Gärtner and Emo

More information

Notes for Lecture Notes 2

Notes for Lecture Notes 2 Stanford University CS254: Computational Complexity Notes 2 Luca Trevisan January 11, 2012 Notes for Lecture Notes 2 In this lecture we define NP, we state the P versus NP problem, we prove that its formulation

More information

POLYNOMIAL SPACE QSAT. Games. Polynomial space cont d

POLYNOMIAL SPACE QSAT. Games. Polynomial space cont d T-79.5103 / Autumn 2008 Polynomial Space 1 T-79.5103 / Autumn 2008 Polynomial Space 3 POLYNOMIAL SPACE Polynomial space cont d Polynomial space-bounded computation has a variety of alternative characterizations

More information

Entropy Games and Matrix Multiplication Games. EQINOCS seminar

Entropy Games and Matrix Multiplication Games. EQINOCS seminar Entropy Games and Matrix Multiplication Games Eugene Asarin Julien Cervelle Aldric Degorre Cătălin Dima Florian Horn Victor Kozyakin IRIF, LACL, IITP EQINOCS seminar 2016-05-11 A game of freedom The story

More information

Decision making, Markov decision processes

Decision making, Markov decision processes Decision making, Markov decision processes Solved tasks Collected by: Jiří Kléma, klema@fel.cvut.cz Spring 2017 The main goal: The text presents solved tasks to support labs in the A4B33ZUI course. 1 Simple

More information

c 2017 Spencer Gordon

c 2017 Spencer Gordon c 2017 Spencer Gordon THE COMPLEXITY OF CONTINUOUS LOCAL SEARCH BY SPENCER GORDON THESIS Submitted in partial fulfillment of the requirements for the degree of Master of Science in Computer Science in

More information

Clarkson s Algorithm for Violator Spaces. Yves Brise, ETH Zürich, CCCG Joint work with Bernd Gärtner

Clarkson s Algorithm for Violator Spaces. Yves Brise, ETH Zürich, CCCG Joint work with Bernd Gärtner Clarkson s Algorithm for Violator Spaces Yves Brise, ETH Zürich, CCCG 20090817 Joint work with Bernd Gärtner A Hierarchy of Optimization Frameworks Linear Programming Halfspaces (constraints), Optimization

More information

On the reduction of total cost and average cost MDPs to discounted MDPs

On the reduction of total cost and average cost MDPs to discounted MDPs On the reduction of total cost and average cost MDPs to discounted MDPs Jefferson Huang School of Operations Research and Information Engineering Cornell University July 12, 2017 INFORMS Applied Probability

More information

Solving Parity Games via Priority Promotion

Solving Parity Games via Priority Promotion Noname manuscript No. (will be inserted by the editor) Solving Parity Games via Priority Promotion Massimo Benerecetti Daniele Dell Erba Fabio Mogavero Received:? / Accepted:? Abstract We consider parity

More information

1 Primals and Duals: Zero Sum Games

1 Primals and Duals: Zero Sum Games CS 124 Section #11 Zero Sum Games; NP Completeness 4/15/17 1 Primals and Duals: Zero Sum Games We can represent various situations of conflict in life in terms of matrix games. For example, the game shown

More information

1 Review of Vertex Cover

1 Review of Vertex Cover CS266: Parameterized Algorithms and Complexity Stanford University Lecture 3 Tuesday, April 9 Scribe: Huacheng Yu Spring 2013 1 Review of Vertex Cover In the last lecture, we discussed FPT algorithms for

More information

Computational Game Theory Spring Semester, 2005/6. Lecturer: Yishay Mansour Scribe: Ilan Cohen, Natan Rubin, Ophir Bleiberg*

Computational Game Theory Spring Semester, 2005/6. Lecturer: Yishay Mansour Scribe: Ilan Cohen, Natan Rubin, Ophir Bleiberg* Computational Game Theory Spring Semester, 2005/6 Lecture 5: 2-Player Zero Sum Games Lecturer: Yishay Mansour Scribe: Ilan Cohen, Natan Rubin, Ophir Bleiberg* 1 5.1 2-Player Zero Sum Games In this lecture

More information

Quantum algorithms (CO 781, Winter 2008) Prof. Andrew Childs, University of Waterloo LECTURE 11: From random walk to quantum walk

Quantum algorithms (CO 781, Winter 2008) Prof. Andrew Childs, University of Waterloo LECTURE 11: From random walk to quantum walk Quantum algorithms (CO 781, Winter 2008) Prof. Andrew Childs, University of Waterloo LECTURE 11: From random walk to quantum walk We now turn to a second major topic in quantum algorithms, the concept

More information

Lecture 10. Sublinear Time Algorithms (contd) CSC2420 Allan Borodin & Nisarg Shah 1

Lecture 10. Sublinear Time Algorithms (contd) CSC2420 Allan Borodin & Nisarg Shah 1 Lecture 10 Sublinear Time Algorithms (contd) CSC2420 Allan Borodin & Nisarg Shah 1 Recap Sublinear time algorithms Deterministic + exact: binary search Deterministic + inexact: estimating diameter in a

More information

Final Examination. Adrian Georgi Josh Karen Lee Min Nikos Tina. There are 12 problems totaling 150 points. Total time is 170 minutes.

Final Examination. Adrian Georgi Josh Karen Lee Min Nikos Tina. There are 12 problems totaling 150 points. Total time is 170 minutes. Massachusetts Institute of Technology 6.042J/18.062J, Fall 02: Mathematics for Computer Science Prof. Albert Meyer and Dr. Radhika Nagpal Final Examination Your name: Circle the name of your Tutorial Instructor:

More information

THREE PUZZLES ON MATHEMATICS, COMPUTATION, AND GAMES

THREE PUZZLES ON MATHEMATICS, COMPUTATION, AND GAMES THREE PUZZLES ON MATHEMATICS, COMPUTATION, AND GAMES GIL KALAI HEBREW UNIVERSITY OF JERUSALEM AND YALE UNIVERSITY ABSTRACT. In this lecture I will talk about three mathematical puzzles involving mathematics

More information

COMP Analysis of Algorithms & Data Structures

COMP Analysis of Algorithms & Data Structures COMP 3170 - Analysis of Algorithms & Data Structures Shahin Kamali Computational Complexity CLRS 34.1-34.4 University of Manitoba COMP 3170 - Analysis of Algorithms & Data Structures 1 / 50 Polynomial

More information

Introduction to Reinforcement Learning. CMPT 882 Mar. 18

Introduction to Reinforcement Learning. CMPT 882 Mar. 18 Introduction to Reinforcement Learning CMPT 882 Mar. 18 Outline for the week Basic ideas in RL Value functions and value iteration Policy evaluation and policy improvement Model-free RL Monte-Carlo and

More information

Linear Programming. Linear Programming I. Lecture 1. Linear Programming. Linear Programming

Linear Programming. Linear Programming I. Lecture 1. Linear Programming. Linear Programming Linear Programming Linear Programming Lecture Linear programming. Optimize a linear function subject to linear inequalities. (P) max " c j x j n j= n s. t. " a ij x j = b i # i # m j= x j 0 # j # n (P)

More information

Chapter 6 Randomization Algorithm Theory WS 2012/13 Fabian Kuhn

Chapter 6 Randomization Algorithm Theory WS 2012/13 Fabian Kuhn Chapter 6 Randomization Algorithm Theory WS 2012/13 Fabian Kuhn Randomization Randomized Algorithm: An algorithm that uses (or can use) random coin flips in order to make decisions We will see: randomization

More information

Artificial Intelligence. Non-deterministic state model. Model for non-deterministic problems. Solutions. Blai Bonet

Artificial Intelligence. Non-deterministic state model. Model for non-deterministic problems. Solutions. Blai Bonet Artificial Intelligence Blai Bonet Non-deterministic state model Universidad Simón Boĺıvar, Caracas, Venezuela Model for non-deterministic problems Solutions State models with non-deterministic actions

More information

ON A THEOREM OF KALAI AND SAMET

ON A THEOREM OF KALAI AND SAMET ON A THEOREM OF KALAI AND SAMET When Do Pure Equilibria Imply a Potential Function? Tim Roughgarden (Stanford) THE WEIGHTED SHAPLEY VALUE The Shapley Value Coalitional game: set N, set function Shapley

More information