Introduction to Game Theory Project Group DynaSearch November 5th, 2013 Maximilian Drees Source: Fotolia, Jürgen Priewe Introduction to Game Theory Maximilian Drees 1
Game Theory In many situations, the outcome does not only depend on the actions of a single entity, but on those of multiple entities, each with its own personal agenda. For example, in many boardgames (e.g. chess), the winner is determined by his own moves as well as by the moves of the other players. This can be applied to other, more serious fields, e.g. the stock market. Introduction to Game Theory Maximilian Drees 2
Game Theory Interactions of rational decision-makers decision-makers: agents/players interactions: multiple agents act simultaneously or consequently rational: each agent has preferences over outcomes, choose action which most likely leads to best feasible outcome Goals understand/predict behavior of players predict outcomes know how to gain advantages design systems Introduction to Game Theory Maximilian Drees 3
Agenda Part 1: Normal Form Games & Nash Equilibria general introduction into game theory basic notions first examples complexity issues Part 2: Selfish Routing & Price of Anarchy special case of games more examples quality of outcomes Introduction to Game Theory Maximilian Drees 4
Part 1: Normal Form Games & Nash Equilibria Introduction to Game Theory Maximilian Drees 5
Normal Form Games Definition A normal form game is a triple (N, (S i ) i N, (c i ) i N ), where N is the set of players, N = n S i is the set of strategies of player i c i : S 1 S 2... S n R is the cost function of player i Example: Prisoner s Dilemma Silent Confess Silent 2/2 5/1 Confess 1/5 4/4 Introduction to Game Theory Maximilian Drees 6
Pure Strategy Profiles Definition For player i with strategy set S i = {Si 1,..., Si k }, a pure strategy is a S j i S i. each player chooses exactly one strategy Definition A pure strategy profile S is a vector of pure strategies, i.e. S S 1... S n. the costs of each player depends on the whole (pure) strategy profile Introduction to Game Theory Maximilian Drees 7
Pure Nash Equilibrium Definition A pure Nash equilibrium (pure NE) is a pure strategy profile in which no player can improve its costs by unilaterally changing its pure strategy. a pure NE is optimal for each player, provided the strategies of the other players are fixed Example: Prisoner s Dilemma Silent Confess Silent 2/2 5/1 Confess 1/5 4/4 Introduction to Game Theory Maximilian Drees 8
Pure Nash Equilibrium Example: Rock-Paper-Scissors Rock Paper Scissors Rock 0/0 0/1 1/0 Paper 1/0 0/0 0/1 Scissors 0/1 1/0 0/0 Introduction to Game Theory Maximilian Drees 9
Pure Nash Equilibrium Example: Rock-Paper-Scissors Rock Paper Scissors Rock 0/0 0/1 1/0 Paper 1/0 0/0 0/1 Scissors 0/1 1/0 0/0 there is no pure Nash equilibrium! Introduction to Game Theory Maximilian Drees 9
Mixed Strategy Profiles remember: in a pure strategy profile, each player chooses exactly one of its strategies Definition For player i with strategy set S i = {S 1 i,..., S k i }, a mixed strategy is a vector x i = (x 1 i,..., x k i ) R k 0 with k i=1 xk i = 1 x j i denotes probability of player i playing strategy S j i Definition A mixed strategy profile is a vector of mixed strategies, i.e. (x 1,..., x k ), s.t. x i is a mixed strategy for player i. Introduction to Game Theory Maximilian Drees 10
Mixed Nash equilibrium Definition A mixed Nash equilibrium (mixed NE) is a mixed strategy profile in which no player can improve its expected costs by unilaterally changing its mixed strategy. Example: Rock-Paper-Scissors player A plays rock with probability 1 2 Rock Paper Scissors Rock 0/0 0/1 1/0 Paper 1/0 0/0 0/1 Scissors 0/1 1/0 0/0 plays paper with probability 1 3 mixed strategy ( 1 2, 1 3, 1 6 plays scissors with probability 1 6 player B plays rock with probability 1 3 plays paper with probability 1 3 mixed strategy ( 1 3, 1 3, 1 3 plays scissors with probability 1 3 Introduction to Game Theory Maximilian Drees 11 ) )
Mixed Nash equilibrium Example: Rock-Paper-Scissors player A: mixed strategy ( 1 2, 1 3, 1 6 ) A/B Rock Paper Scissors Rock 0/0 0/1 1/0 Paper 1/0 0/0 0/1 Scissors 0/1 1/0 0/0 player B: mixed strategy ( 1, 1, 1 ) 3 3 3 mixed strategy profile (( 1, 1, ) ( 1 2 3 6, 1, 1, )) 1 3 3 3 Result for player A: ( 1 1 2 3 0 + 1 3 0 + 1 ) 3 1 + 1 ( 1 3 3 1 + 1 3 0 + 1 ) 3 0 + 1 ( 1 6 3 0 + 1 3 1 + 1 ) 3 0 = 1 6 + 1 9 + 1 18 = 1 3 Introduction to Game Theory Maximilian Drees 12
Mixed Nash equilibrium Example: Rock-Paper-Scissors A/B Rock Paper Scissors Rock 0/0 0/1 1/0 Paper 1/0 0/0 0/1 Scissors 0/1 1/0 0/0 there is a mixed Nash equilibrium based on the mixed strategy profile (( 1 3, 1 3, 1 ) ( 1, 3 3, 1 3, 1 )) 3 expected payoff for both players is 1 3 Introduction to Game Theory Maximilian Drees 13
Nash s Theorem Theorem (Nash) Every finite normal form game has a mixed Nash equilibrium. John Forbes Nash, Jr (13.06.1928) movie: A Beautiful Mind 28-page dissertation on non-cooperative games Introduction to Game Theory Maximilian Drees 14
Complexity of Nash Equilibria How hard is it to compute Nash equilibria? not a decision problem, because we already know mixed NE exist not an optimization problem, because a mixed NE does not have to be good Definition The binary relation P(x, y) is contained in TFNP if and only if for every x, there exists a y such that P(x, y) holds there is a poly. time algo. to determine whether P(x, y) holds TFNP contains search problems Definition PPAD is the class of search problems from TFNP which can be reduced to End-of-a-Line. PPAD stands for Polynomial Parity Argument, Directed Case Introduction to Game Theory Maximilian Drees 15
End-of-a-Line Definition Let V be a set of solutions in which each v has at most one successor and one predecessor. Given an initial solution v 0 and two poly. time computable functions S : V V which computes the successor of a solution P : V V which computes the predecessor of a solution the task for the problem End-of-a-Line is to find a final solution with no successor or one with no predecessor and different from v 0. Intuition behind the problem on the board. Introduction to Game Theory Maximilian Drees 16
Theorem (Cheng and Deng, 2005) The problem of computing a mixed Nash equilibrium (NASH) is PPAD-complete. Concept of proof: 1 reduce NASH to End-of-a-Line NASH is part of PPAD Introduction to Game Theory Maximilian Drees 17
Theorem (Cheng and Deng, 2005) The problem of computing a mixed Nash equilibrium (NASH) is PPAD-complete. Concept of proof: 1 reduce NASH to End-of-a-Line NASH is part of PPAD 2 reduce End-of-a-Line to NASH NASH is at least as hard as End-of-a-Line NASH is PPAD-hard Introduction to Game Theory Maximilian Drees 17
Part 2: Selfish Routing & Price of Anarchy Introduction to Game Theory Maximilian Drees 18
Wardrop s Traffic Model graph G = (V, E) with latency functions l e(x) on the edges k commodities with demand r i flow of volume r i is sent from source s i to sink t i latency function maps the amount of flow on an edge to a cost value Notations P i set of paths from s i to t i P = k i=1 P i set of all relevant paths flow f i defines a value f i (P) [0, r i ] for every P P i satisfying P P i f i (P) = r i flow f contains all values defined by all f i f P = k i=1 f i(p) is the total flow along path P P (P / P i f i (P) = 0) flow along edge e E is f e = P e f P latency on edge e E at flow f is l e(f) = l e(f e) latency on path P P at flow f is l P (f) = l P (f P ) = e P le(fe) Introduction to Game Theory Maximilian Drees 19
Equilibrium Flow Definition (Equilibrium flow) A flow f is at an equilibrium if for every i [k] and for every pair of paths P 1, P 2 P i with f P1 > 0, it holds that l P1 (f) l P2 (f). imagine a flow to consist of infinitely many particles of infinitesimal size a flow is at a equilibrium if no particle can choose a path with smaller latency for every i [k] and every pair of paths P 1, P 2 P i which are actually used by f, l P1 (f) = l P2 (f) Example: Braess Paradox (on the board) Introduction to Game Theory Maximilian Drees 20
Definition The average latency of a flow f is C(f) := f P l P (f) = f e l e(f) P P e E A flow minimizing the average latency is called optimal flow. the average latency is the sum of the individual cost of each particle the average latency of a flow equilibrium can be greater than the average latency of an optimal flow Another example on the board. Introduction to Game Theory Maximilian Drees 21
Price of Anarchy Definition Let flow f be at an equilibrium and let every other flow g at an equilibrium have a smaller average latency, i.e. C(f) C(g). Let flow opt be an optimal flow. Then the price of anarchy is defined as PoA := C(f) C(opt) PoA compares to worst-case situation with the optimal one it measures the need to introduce a centralized control instance or enforce a set of rules on the players Theorem (Roughgarden and Tardos, 2001) The price of anarchy for routing games with linear latency functions is at most 4 3. Introduction to Game Theory Maximilian Drees 22
Price of Anarchy Definition Let flow f be at an equilibrium and let every other flow g at an equilibrium have a smaller average latency, i.e. C(f) C(g). Let flow opt be an optimal flow. Then the price of anarchy is defined as PoA := C(f) C(opt) PoA compares to worst-case situation with the optimal one it measures the need to introduce a centralized control instance or enforce a set of rules on the players Theorem (Roughgarden and Tardos, 2001) The price of anarchy for routing games with linear latency functions is at most 4 3. Now: Proof Introduction to Game Theory Maximilian Drees 22
Price of Anarchy Theorem (Roughgarden and Tardos, 2001) The price of anarchy for routing games with linear latency functions is at most 4. 3 assume all latency functions to be monotone increasing Lemma A flow f is at an equilibrium if and only if for every flow g the following holds: (f P g P ) l P (f P ) 0 P P Introduction to Game Theory Maximilian Drees 23
Price of Anarchy Theorem (Roughgarden and Tardos, 2001) The price of anarchy for routing games with linear latency functions is at most 4. 3 assume all latency functions to be monotone increasing Lemma A flow f is at an equilibrium if and only if for every flow g the following holds: (f P g P ) l P (f P ) 0 P P Lemma For every edge e E g(e) [l e(f e) l e(g e)] 1 fe le(fe) 4 Introduction to Game Theory Maximilian Drees 23
Summary Part 1: Normal Form Games & Nash Equilibria general introduction into game theory - Normal Form games basic notions - pure/mixed strategy profiles, Nash equilibrium first examples - Prisoner s Dilemma, Rock-Paper-Scissors complexity issues - complexity class PPAD (search problems), NASH PPAD-complete Part 2: Selfish Routing & Price of Anarchy special case of games - routing games based on Wardrop s traffic model more examples - Braess Paradox quality of outcomes - Price of Anarchy compares optimal outcome with worst equilibrium Introduction to Game Theory Maximilian Drees 24
Thank you for your attention! Maximilian Drees Heinz Nixdorf Institute & Department of Computer Science Address: Fürstenallee 11 33102 Paderborn Germany Source: Fotolia, Jürgen Priewe Phone: +49 5251 60-6434 Fax: +49 5251 60-6482 E-mail: maxdrees@mail.upb.de Web: http://wwwhni.upb.de/en/alg/staff/max Introduction to Game Theory Maximilian Drees 25