Game Theory, Evolutionary Dynamics, and Multi-Agent Learning. Prof. Nicola Gatti

Size: px
Start display at page:

Download "Game Theory, Evolutionary Dynamics, and Multi-Agent Learning. Prof. Nicola Gatti"

Transcription

1 Game Theory, Evolutionary Dynamics, and Multi-Agent Learning Prof. Nicola Gatti

2 Game theory

3 Game theory: basics Normal form Players Actions Outcomes Utilities Strategies Solutions

4 Game theory: basics Normal form Players Actions Outcomes Utilities Strategies Solutions

5 Game theory: basics Rock Paper Scissors Normal form Players Actions Outcomes Utilities Strategies Solutions Rock Paper Scissors

6 Game theory: basics Rock Paper Scissors Normal form Players Rock Tie wins wins Actions Outcomes Utilities Strategies Paper wins Tie wins Solutions Scissors wins wins Tie

7 Game theory: basics Rock Paper Scissors Normal form Players Actions Outcomes Utilities Strategies Solutions Rock 0,0-1,1 1,-1 Paper 1,-1 0,0-1,1 Scissors -1,1 1,-1 0,0

8 Game theory: basics Rock Paper Scissors Normal form Players Rock 0,0-1,1 1,-1 1(Rock) Actions Outcomes Utilities Strategies Paper 1,-1 0,0-1,1 1(Paper) Solutions Scissors -1,1 1,-1 0,0 1(Scissors) 2(Rock) 2(Paper) 2(Scissors)

9 Game theory: basics Rock Paper Scissors Normal form Players Rock 0,0-1,1 1,-1 1/3 Actions Outcomes Utilities Strategies Paper 1,-1 0,0-1,1 1/3 Solutions Scissors -1,1 1,-1 0,0 1/3 1/3 1/3 1/3

10 Nash Equilibrium A strategy profile 1 2 arg max arg max 2 n n 1U 1 1U 2 ( 1, 2 ) 2 o 2 o is a Nash equilibrium if and if:

11 Evolutionary dynamics

12 An evolutionary interpretation 33% 33% 33% Rock Paper Scissors

13 An evolutionary interpretation Infinite population of individuals 33% 33% 33% Rock Paper Scissors

14 Bacterials

15 An evolutionary interpretation 33% 33% 30% 50% 33% 20% Rock Paper Scissors Rock Paper Scissors

16 An evolutionary interpretation Rock Paper Scissors 33% 33% Rock 0,0-1,1 1,-1 Paper 1,-1 0,0-1,1 30% 50% 33% Scissors -1,1 1,-1 0,0 20% Rock Paper Scissors Rock Paper Scissors

17 An evolutionary interpretation Rock Paper Scissors 33% 33% Rock 0,0-1,1 1,-1 30% 50% Paper 1,-1 0,0-1,1 33% Scissors -1,1 1,-1 0,0 20% Rock Paper Scissors Rock Paper Scissors At each round, each individual of a population plays against each individual of the opponent s population

18 An evolutionary interpretation Utility = Utility = 0.1 Rock Paper Scissors 33% 33% Rock 0,0-1,1 1,-1 Paper 1,-1 0,0-1,1 30% 50% 33% Scissors -1,1 1,-1 0,0 20% Rock Paper Scissors Rock Paper Scissors Utility = 0.2

19 An evolutionary interpretation Utility = % 33% Utility = % Rock Paper Scissors Utility = 0.2

20 An evolutionary interpretation Utility = % 33% Utility = % Rock Paper Scissors Utility = 0.2 The average utility is 0

21 An evolutionary interpretation Utility = % 33% Utility = % Rock Paper Scissors Utility = 0.2 The average utility is 0 A positive (utility - average utility) leads to an increase of the population A negative (utility - average utility) leads to a decrease of the population

22 An evolutionary interpretation 17% 37% 46% Rock Paper Scissors New population after the replication

23 Revision protocol Question: how the populations change? 1(a, t) = 1 (a, t) Replicator dynamics e a U 1 2 (t) 1(t)U 1 2 (t) Utility given by playing a with a probability of 1 Average population utility

24 Evolutionary Stable Strategies A strategy is an ESS if it is immune to invasion by mutant strategies, given that the mutants initially occupy a small fraction of population Every ESS is an asymptotically stable fixed point of the replicator dynamics

25 Evolutionary Stable Strategies A strategy is an ESS if it is immune to invasion by mutant strategies, given that the mutants initially occupy a small fraction of population Every ESS is an asymptotically stable fixed point of the replicator dynamics Nash equilibria ESS While a NE always exists, an ESS may not exist

26 Prisoner s dilemma Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

27 Prisoner s dilemma Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

28 Prisoner s dilemma 1 Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1 s cooperate probability 0.75 y x 1 s cooperate probability

29 Prisoner s dilemma 1 Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1 s cooperate probability 0.75 y asymptotically stable x 1 s cooperate probability

30 Stag hunt Stag Hare Stag 4,4 1,3 Hare 3,1 3,3

31 Stag hunt Stag Hare Stag 4,4 1,3 Hare 3,1 3,3

32 Stag hunt Stag Hare Stag 4,4 1,3 Hare 3,1 3,3 s stag probability x 1 s stag probability

33 Stag hunt 1 asymptotically stable Stag Hare 0.75 Stag 4,4 1,3 Hare 3,1 3,3 s stag probability asymptotically stable x 1 s stag probability

34 Matching pennies Head Tail Head 0,1 1,0 Tail 1,0 0,1

35 Matching pennies Head Tail Head 0,1 1,0 Tail 1,0 0,1

36 Matching pennies 1 Head Tail 0.75 Head 0,1 1,0 Tail 1,0 0,1 s head probability y x 1 s head probability

37 Matching pennies 1 Head Tail 0.75 Head 0,1 1,0 Tail 1,0 0,1 s head probability y x 1 s head probability

38 Multi-agent learning

39 Markov decision problem

40 Reinforcement learning

41 Q-learning (1) For every pair state/action: Q(s, a) Q(s, a)+ h i r + max a 0 Q(s, a 0 ) Q(s, a) learning rate reward discount factor

42 Example: normal-form games 1 state 2 actions (Cooperate, Defeat) Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

43 Example: normal-form games Q(a) Q(a)+ r Q(a) =0.2 1(a) = 1(a) = ( 1.0 a = Cooperate 0.0 ( a = Defeat 0.2 a = Cooperate 0.8 a = Defeat Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

44 Example: normal-form games Q(a) Q(a)+ r Q(a) =0.2 round s action s Q function t =0 Q(Cooperate) = 0 t = 1 a = Cooperate Q(Cooperate) = 0.6 t = 2 a = Defeat Q(Cooperate) = 0.48 t = 3 a = Defeat Q(Cooperate) = t = 4 a = Defeat Q(Cooperate) = t = 5 a = Defeat Q(Cooperate) = t = 6 a = Cooperate Q(Cooperate) = ( 1.0 a = Cooperate 1(a) = 0.0 a = Defeat ( 0.2 a = Cooperate 1(a) = 0.8 a = Defeat Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

45 Example: normal-form games Q(a) Q(a)+ r Q(a) =0.2 round s action s Q function t =0 Q(Cooperate) = 0 t = 1 a = Cooperate Q(Cooperate) = 0.6 t = 2 a = Defeat Q(Cooperate) = 0.48 t = 3 a = Defeat Q(Cooperate) = t = 4 a = Defeat Q(Cooperate) = t = 5 a = Defeat Q(Cooperate) = t = 6 a = Cooperate Q(Cooperate) = ( 1.0 a = Cooperate 1(a) = 0.0 a = Defeat ( 0.2 a = Cooperate 1(a) = 0.8 a = Defeat Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

46 Example: normal-form games Q(a) Q(a)+ r Q(a) =0.2 round s action s Q function t =0 Q(Cooperate) = 0 t = 1 a = Cooperate Q(Cooperate) = 0.6 t = 2 a = Defeat Q(Cooperate) = 0.48 t = 3 a = Defeat Q(Cooperate) = t = 4 a = Defeat Q(Cooperate) = t = 5 a = Defeat Q(Cooperate) = t = 6 a = Cooperate Q(Cooperate) = ( 1.0 a = Cooperate 1(a) = 0.0 a = Defeat ( 0.2 a = Cooperate 1(a) = 0.8 a = Defeat Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

47 Example: normal-form games Q(a) Q(a)+ r Q(a) =0.2 round s action s Q function t =0 Q(Cooperate) = 0 t = 1 a = Cooperate Q(Cooperate) = 0.6 t = 2 a = Defeat Q(Cooperate) = 0.48 t = 3 a = Defeat Q(Cooperate) = t = 4 a = Defeat Q(Cooperate) = t = 5 a = Defeat Q(Cooperate) = t = 6 a = Cooperate Q(Cooperate) = ( 1.0 a = Cooperate 1(a) = 0.0 a = Defeat ( 0.2 a = Cooperate 1(a) = 0.8 a = Defeat Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

48 Example: normal-form games Q(a) Q(a)+ r Q(a) =0.2 round s action s Q function t =0 Q(Cooperate) = 0 t = 1 a = Cooperate Q(Cooperate) = 0.6 t = 2 a = Defeat Q(Cooperate) = 0.48 t = 3 a = Defeat Q(Cooperate) = t = 4 a = Defeat Q(Cooperate) = t = 5 a = Defeat Q(Cooperate) = t = 6 a = Cooperate Q(Cooperate) = ( 1.0 a = Cooperate 1(a) = 0.0 a = Defeat ( 0.2 a = Cooperate 1(a) = 0.8 a = Defeat Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

49 Example: normal-form games Q(a) Q(a)+ r Q(a) =0.2 round s action s Q function t =0 Q(Cooperate) = 0 t = 1 a = Cooperate Q(Cooperate) = 0.6 t = 2 a = Defeat Q(Cooperate) = 0.48 t = 3 a = Defeat Q(Cooperate) = t = 4 a = Defeat Q(Cooperate) = t = 5 a = Defeat Q(Cooperate) = t = 6 a = Cooperate Q(Cooperate) = ( 1.0 a = Cooperate 1(a) = 0.0 a = Defeat ( 0.2 a = Cooperate 1(a) = 0.8 a = Defeat Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

50 Example: normal-form games Q(a) Q(a)+ r Q(a) =0.2 round s action s Q function t =0 Q(Cooperate) = 0 t = 1 a = Cooperate Q(Cooperate) = 0.6 t = 2 a = Defeat Q(Cooperate) = 0.48 t = 3 a = Defeat Q(Cooperate) = t = 4 a = Defeat Q(Cooperate) = t = 5 a = Defeat Q(Cooperate) = t = 6 a = Cooperate Q(Cooperate) = ( 1.0 a = Cooperate 1(a) = 0.0 a = Defeat ( 0.2 a = Cooperate 1(a) = 0.8 a = Defeat Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

51 Q-learning (2) Softmax (a.k.a. Boltzam exploration) i(a) = exp(q(s, a)/ ) P a 0 exp(q(s, a 0 )/ )

52 Q-learning (2) Softmax (a.k.a. Boltzam exploration) i(a) = exp(q(s, a)/ ) P a exp(q(s, a 0 )/ ) 0 temperature

53 Q-learning (2) Softmax (a.k.a. Boltzam exploration) i(a) = exp(q(s, a)/ ) P a exp(q(s, a 0 )/ ) 0 temperature Every action is played with strictly positive probability The larger the temperature, the smoother the function If the temperature is 0, we would have a best response

54 Example: normal-form games Q(Cooperate) Q(Defeat) 1(Cooperate) 1(Cooperate)

55 Example: normal-form games Q(Cooperate) Q(Defeat) 1(Cooperate) 1(Cooperate)

56 Example: normal-form games Q(Cooperate) Q(Defeat) 1(Cooperate) 1(Cooperate)

57 Example: normal-form games Q(Cooperate) Q(Defeat) 1(Cooperate) 1(Cooperate)

58 Self-play Q-learning dynamics

59 Self-play learning Q-learning algorithm Q-learning algorithm Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1

60 Learning dynamics Assumptions: Time is continuous All the actions can be selected simultaneously

61 Learning dynamics Assumptions: Time is continuous All the actions can be selected simultaneously 1(a, t) = 1(a, t) e a U 1 2 (t) 1(t)U 1 2 (t) 1 (a, t) log( 1 (a)) X 1(a 0 ) log( 1 (a 0 )) a 0

62 Learning dynamics Assumptions: Time is continuous All the actions can be selected simultaneously 1(a, t) = 1(a, t) e a U 1 2 (t) 1(t)U 1 2 (t) 1 (a, t) log( 1 (a)) X 1(a 0 ) log( 1 (a 0 )) a 0 exploitation term exploration term

63 Learning dynamics Assumptions: Time is continuous All the actions can be selected simultaneously 1(a, t) = 1(a, t) e a U 1 2 (t) 1(t)U 1 2 (t) 1 (a, t) log( 1 (a)) X 1(a 0 ) log( 1 (a 0 )) a 0 exploitation term exploration term When the temperature is 0, the Q-learning behaves as the replicator dynamics

64 0 Prisoner s dilemma 1 Cooperate Defeat Cooperate 3,3 0,5 Defeat 5,0 1,1 s cooperate probability y asymptotically stable x 1 s cooperate probability

65 Stag hunt 1 asymptotically stable Stag Hare 0.75 Stag 4,4 1,3 Hare 3,1 3,3 s stag probability y asymptotically stable x 1 s stag probability

66 Matching pennies 1 Head Tail Head 0,1 1,0 Tail 1,0 0,1 s head probability y Player x 1 s head probability

Belief-based Learning

Belief-based Learning Belief-based Learning Algorithmic Game Theory Marcello Restelli Lecture Outline Introdutcion to multi-agent learning Belief-based learning Cournot adjustment Fictitious play Bayesian learning Equilibrium

More information

A (Brief) Introduction to Game Theory

A (Brief) Introduction to Game Theory A (Brief) Introduction to Game Theory Johanne Cohen PRiSM/CNRS, Versailles, France. Goal Goal is a Nash equilibrium. Today The game of Chicken Definitions Nash Equilibrium Rock-paper-scissors Game Mixed

More information

Evolution & Learning in Games

Evolution & Learning in Games 1 / 27 Evolution & Learning in Games Econ 243B Jean-Paul Carvalho Lecture 2. Foundations of Evolution & Learning in Games II 2 / 27 Outline In this lecture, we shall: Take a first look at local stability.

More information

Game Theory for Linguists

Game Theory for Linguists Fritz Hamm, Roland Mühlenbernd 4. Mai 2016 Overview Overview 1. Exercises 2. Contribution to a Public Good 3. Dominated Actions Exercises Exercise I Exercise Find the player s best response functions in

More information

First Prev Next Last Go Back Full Screen Close Quit. Game Theory. Giorgio Fagiolo

First Prev Next Last Go Back Full Screen Close Quit. Game Theory. Giorgio Fagiolo Game Theory Giorgio Fagiolo giorgio.fagiolo@univr.it https://mail.sssup.it/ fagiolo/welcome.html Academic Year 2005-2006 University of Verona Summary 1. Why Game Theory? 2. Cooperative vs. Noncooperative

More information

DECENTRALIZED LEARNING IN GENERAL-SUM MATRIX GAMES: AN L R I LAGGING ANCHOR ALGORITHM. Xiaosong Lu and Howard M. Schwartz

DECENTRALIZED LEARNING IN GENERAL-SUM MATRIX GAMES: AN L R I LAGGING ANCHOR ALGORITHM. Xiaosong Lu and Howard M. Schwartz International Journal of Innovative Computing, Information and Control ICIC International c 20 ISSN 349-498 Volume 7, Number, January 20 pp. 0 DECENTRALIZED LEARNING IN GENERAL-SUM MATRIX GAMES: AN L R

More information

Iterated Strict Dominance in Pure Strategies

Iterated Strict Dominance in Pure Strategies Iterated Strict Dominance in Pure Strategies We know that no rational player ever plays strictly dominated strategies. As each player knows that each player is rational, each player knows that his opponents

More information

Exponential Moving Average Based Multiagent Reinforcement Learning Algorithms

Exponential Moving Average Based Multiagent Reinforcement Learning Algorithms Exponential Moving Average Based Multiagent Reinforcement Learning Algorithms Mostafa D. Awheda Department of Systems and Computer Engineering Carleton University Ottawa, Canada KS 5B6 Email: mawheda@sce.carleton.ca

More information

Computation of Efficient Nash Equilibria for experimental economic games

Computation of Efficient Nash Equilibria for experimental economic games International Journal of Mathematics and Soft Computing Vol.5, No.2 (2015), 197-212. ISSN Print : 2249-3328 ISSN Online: 2319-5215 Computation of Efficient Nash Equilibria for experimental economic games

More information

Exponential Moving Average Based Multiagent Reinforcement Learning Algorithms

Exponential Moving Average Based Multiagent Reinforcement Learning Algorithms Artificial Intelligence Review manuscript No. (will be inserted by the editor) Exponential Moving Average Based Multiagent Reinforcement Learning Algorithms Mostafa D. Awheda Howard M. Schwartz Received:

More information

Computational Evolutionary Game Theory and why I m never using PowerPoint for another presentation involving maths ever again

Computational Evolutionary Game Theory and why I m never using PowerPoint for another presentation involving maths ever again Computational Evolutionary Game Theory and why I m never using PowerPoint for another presentation involving maths ever again Enoch Lau 5 September 2007 Outline What is evolutionary game theory? Why evolutionary

More information

Game Theory and Algorithms Lecture 2: Nash Equilibria and Examples

Game Theory and Algorithms Lecture 2: Nash Equilibria and Examples Game Theory and Algorithms Lecture 2: Nash Equilibria and Examples February 24, 2011 Summary: We introduce the Nash Equilibrium: an outcome (action profile) which is stable in the sense that no player

More information

Prisoner s Dilemma. Veronica Ciocanel. February 25, 2013

Prisoner s Dilemma. Veronica Ciocanel. February 25, 2013 n-person February 25, 2013 n-person Table of contents 1 Equations 5.4, 5.6 2 3 Types of dilemmas 4 n-person n-person GRIM, GRIM, ALLD Useful to think of equations 5.4 and 5.6 in terms of cooperation and

More information

Basic Game Theory. Kate Larson. January 7, University of Waterloo. Kate Larson. What is Game Theory? Normal Form Games. Computing Equilibria

Basic Game Theory. Kate Larson. January 7, University of Waterloo. Kate Larson. What is Game Theory? Normal Form Games. Computing Equilibria Basic Game Theory University of Waterloo January 7, 2013 Outline 1 2 3 What is game theory? The study of games! Bluffing in poker What move to make in chess How to play Rock-Scissors-Paper Also study of

More information

Introduction to Game Theory. Outline. Topics. Recall how we model rationality. Notes. Notes. Notes. Notes. Tyler Moore.

Introduction to Game Theory. Outline. Topics. Recall how we model rationality. Notes. Notes. Notes. Notes. Tyler Moore. Introduction to Game Theory Tyler Moore Tandy School of Computer Science, University of Tulsa Slides are modified from version written by Benjamin Johnson, UC Berkeley Lecture 15 16 Outline 1 Preferences

More information

The Dynamics of Generalized Reinforcement Learning

The Dynamics of Generalized Reinforcement Learning The Dynamics of Generalized Reinforcement Learning Ratul Lahkar and Robert M. Seymour August 21, 2012 Abstract We consider reinforcement learning in games with both positive and negative payoffs. The Cross

More information

Game-Theoretic Learning:

Game-Theoretic Learning: Game-Theoretic Learning: Regret Minimization vs. Utility Maximization Amy Greenwald with David Gondek, Amir Jafari, and Casey Marks Brown University University of Pennsylvania November 17, 2004 Background

More information

Game Theory and Evolution

Game Theory and Evolution Game Theory and Evolution Toban Wiebe University of Manitoba July 2012 Toban Wiebe (University of Manitoba) Game Theory and Evolution July 2012 1 / 24 Why does evolution need game theory? In what sense

More information

Computing Minmax; Dominance

Computing Minmax; Dominance Computing Minmax; Dominance CPSC 532A Lecture 5 Computing Minmax; Dominance CPSC 532A Lecture 5, Slide 1 Lecture Overview 1 Recap 2 Linear Programming 3 Computational Problems Involving Maxmin 4 Domination

More information

Quantum Games. Quantum Strategies in Classical Games. Presented by Yaniv Carmeli

Quantum Games. Quantum Strategies in Classical Games. Presented by Yaniv Carmeli Quantum Games Quantum Strategies in Classical Games Presented by Yaniv Carmeli 1 Talk Outline Introduction Game Theory Why quantum games? PQ Games PQ penny flip 2x2 Games Quantum strategies 2 Game Theory

More information

C31: Game Theory, Lecture 1

C31: Game Theory, Lecture 1 C31: Game Theory, Lecture 1 V. Bhaskar University College London 5 October 2006 C31 Lecture 1: Games in strategic form & Pure strategy equilibrium Osborne: ch 2,3, 12.2, 12.3 A game is a situation where:

More information

CSC304 Lecture 5. Game Theory : Zero-Sum Games, The Minimax Theorem. CSC304 - Nisarg Shah 1

CSC304 Lecture 5. Game Theory : Zero-Sum Games, The Minimax Theorem. CSC304 - Nisarg Shah 1 CSC304 Lecture 5 Game Theory : Zero-Sum Games, The Minimax Theorem CSC304 - Nisarg Shah 1 Recap Last lecture Cost-sharing games o Price of anarchy (PoA) can be n o Price of stability (PoS) is O(log n)

More information

Graph topology and the evolution of cooperation

Graph topology and the evolution of cooperation Provided by the author(s) and NUI Galway in accordance with publisher policies. Please cite the published version when available. Title Graph topology and the evolution of cooperation Author(s) Li, Menglin

More information

BELIEFS & EVOLUTIONARY GAME THEORY

BELIEFS & EVOLUTIONARY GAME THEORY 1 / 32 BELIEFS & EVOLUTIONARY GAME THEORY Heinrich H. Nax hnax@ethz.ch & Bary S. R. Pradelski bpradelski@ethz.ch May 15, 217: Lecture 1 2 / 32 Plan Normal form games Equilibrium invariance Equilibrium

More information

Static (or Simultaneous- Move) Games of Complete Information

Static (or Simultaneous- Move) Games of Complete Information Static (or Simultaneous- Move) Games of Complete Information Introduction to Games Normal (or Strategic) Form Representation Teoria dos Jogos - Filomena Garcia 1 Outline of Static Games of Complete Information

More information

Evolutionary Game Theory

Evolutionary Game Theory Evolutionary Game Theory ISI 330 Lecture 18 1 ISI 330 Lecture 18 Outline A bit about historical origins of Evolutionary Game Theory Main (competing) theories about how cooperation evolves P and other social

More information

Introduction to Game Theory

Introduction to Game Theory COMP323 Introduction to Computational Game Theory Introduction to Game Theory Paul G. Spirakis Department of Computer Science University of Liverpool Paul G. Spirakis (U. Liverpool) Introduction to Game

More information

Game Theory. Professor Peter Cramton Economics 300

Game Theory. Professor Peter Cramton Economics 300 Game Theory Professor Peter Cramton Economics 300 Definition Game theory is the study of mathematical models of conflict and cooperation between intelligent and rational decision makers. Rational: each

More information

Industrial Organization Lecture 3: Game Theory

Industrial Organization Lecture 3: Game Theory Industrial Organization Lecture 3: Game Theory Nicolas Schutz Nicolas Schutz Game Theory 1 / 43 Introduction Why game theory? In the introductory lecture, we defined Industrial Organization as the economics

More information

CS 798: Multiagent Systems

CS 798: Multiagent Systems CS 798: Multiagent Systems and Utility Kate Larson Cheriton School of Computer Science University of Waterloo January 6, 2010 Outline 1 Self-Interested Agents 2 3 4 5 Self-Interested Agents We are interested

More information

Introduction to Game Theory

Introduction to Game Theory Introduction to Game Theory Project Group DynaSearch November 5th, 2013 Maximilian Drees Source: Fotolia, Jürgen Priewe Introduction to Game Theory Maximilian Drees 1 Game Theory In many situations, the

More information

Population Games and Evolutionary Dynamics

Population Games and Evolutionary Dynamics Population Games and Evolutionary Dynamics William H. Sandholm The MIT Press Cambridge, Massachusetts London, England in Brief Series Foreword Preface xvii xix 1 Introduction 1 1 Population Games 2 Population

More information

Level K Thinking. Mark Dean. Columbia University - Spring 2017

Level K Thinking. Mark Dean. Columbia University - Spring 2017 Level K Thinking Mark Dean Columbia University - Spring 2017 Introduction Game theory: The study of strategic decision making Your outcome depends on your own actions and the actions of others Standard

More information

Multiagent Learning Using a Variable Learning Rate

Multiagent Learning Using a Variable Learning Rate Multiagent Learning Using a Variable Learning Rate Michael Bowling, Manuela Veloso Computer Science Department, Carnegie Mellon University, Pittsburgh, PA 15213-3890 Abstract Learning to act in a multiagent

More information

An Introduction to Evolutionary Game Theory

An Introduction to Evolutionary Game Theory An Introduction to Evolutionary Game Theory Lectures delivered at the Graduate School on Nonlinear and Stochastic Systems in Biology held in the Department of Applied Mathematics, School of Mathematics

More information

Decision Theory: Q-Learning

Decision Theory: Q-Learning Decision Theory: Q-Learning CPSC 322 Decision Theory 5 Textbook 12.5 Decision Theory: Q-Learning CPSC 322 Decision Theory 5, Slide 1 Lecture Overview 1 Recap 2 Asynchronous Value Iteration 3 Q-Learning

More information

Microeconomics. 2. Game Theory

Microeconomics. 2. Game Theory Microeconomics 2. Game Theory Alex Gershkov http://www.econ2.uni-bonn.de/gershkov/gershkov.htm 18. November 2008 1 / 36 Dynamic games Time permitting we will cover 2.a Describing a game in extensive form

More information

Game Theory: Spring 2017

Game Theory: Spring 2017 Game Theory: Spring 207 Ulle Endriss Institute for Logic, Language and Computation University of Amsterdam Ulle Endriss Plan for Today We have seen that every normal-form game has a Nash equilibrium, although

More information

Algorithmic Game Theory. Alexander Skopalik

Algorithmic Game Theory. Alexander Skopalik Algorithmic Game Theory Alexander Skopalik Today Course Mechanics & Overview Introduction into game theory and some examples Chapter 1: Selfish routing Alexander Skopalik Skopalik@mail.uni-paderborn.de

More information

Writing Game Theory in L A TEX

Writing Game Theory in L A TEX Writing Game Theory in L A TEX Thiago Silva First Version: November 22, 2015 This Version: November 13, 2017 List of Figures and Tables 1 2x2 Matrix: Prisoner s ilemma Normal-Form Game............. 3 2

More information

Empathy in Bimatrix Games

Empathy in Bimatrix Games Empathy in Bimatrix Games Brian Powers, Michalis Smyrnakis and Hamidou Tembine arxiv:1708.01910v1 [cs.gt] 6 Aug 017 August 8, 017 Abstract Although the definition of what empathetic preferences exactly

More information

EVOLUTIONARY STABILITY FOR TWO-STAGE HAWK-DOVE GAMES

EVOLUTIONARY STABILITY FOR TWO-STAGE HAWK-DOVE GAMES ROCKY MOUNTAIN JOURNAL OF MATHEMATICS olume 25, Number 1, Winter 1995 EOLUTIONARY STABILITY FOR TWO-STAGE HAWK-DOE GAMES R. CRESSMAN ABSTRACT. Although two individuals in a biological species often interact

More information

Introduction to Quantum Game Theory

Introduction to Quantum Game Theory Introduction to Quantum Game Theory Multiparticipant decision problems strategic interdependence Classical game theory began in 1944 with The Theory of Games and Economic Behavior, by John von Neumann

More information

Mohammad Hossein Manshaei 1394

Mohammad Hossein Manshaei 1394 Mohammad Hossein Manshaei manshaei@gmail.com 1394 2 Concept related to a specific branch of Biology Relates to the evolution of the species in nature Powerful modeling tool that has received a lot of attention

More information

Introduction to Game Theory. Outline. Proposal feedback. Proposal feedback: written feedback. Notes. Notes. Notes. Notes. Tyler Moore.

Introduction to Game Theory. Outline. Proposal feedback. Proposal feedback: written feedback. Notes. Notes. Notes. Notes. Tyler Moore. Introduction to Game Theory Tyler Moore CSE 7338 Computer Science & Engineering Department, SMU, Dallas, TX Lectures 7 8 Outline Proposal feedback 2 3 4 5 2 / 6 Proposal feedback Proposal feedback Each

More information

Games A game is a tuple = (I; (S i ;r i ) i2i) where ffl I is a set of players (i 2 I) ffl S i is a set of (pure) strategies (s i 2 S i ) Q ffl r i :

Games A game is a tuple = (I; (S i ;r i ) i2i) where ffl I is a set of players (i 2 I) ffl S i is a set of (pure) strategies (s i 2 S i ) Q ffl r i : On the Connection between No-Regret Learning, Fictitious Play, & Nash Equilibrium Amy Greenwald Brown University Gunes Ercal, David Gondek, Amir Jafari July, Games A game is a tuple = (I; (S i ;r i ) i2i)

More information

Outline for today. Stat155 Game Theory Lecture 16: Evolutionary game theory. Evolutionarily stable strategies. Nash equilibrium.

Outline for today. Stat155 Game Theory Lecture 16: Evolutionary game theory. Evolutionarily stable strategies. Nash equilibrium. Outline for today Stat155 Game Theory Lecture 16: Evolutionary game theory Peter Bartlett October 20, 2016 Nash equilibrium 1 / 21 2 / 21 A strategy profile x = (x1,..., x k ) S 1 Sk is a Nash equilibrium

More information

EVOLUTIONARY GAMES WITH GROUP SELECTION

EVOLUTIONARY GAMES WITH GROUP SELECTION EVOLUTIONARY GAMES WITH GROUP SELECTION Martin Kaae Jensen Alexandros Rigos Department of Economics University of Leicester Controversies in Game Theory: Homo Oeconomicus vs. Homo Socialis ETH Zurich 12/09/2014

More information

Other Equilibrium Notions

Other Equilibrium Notions Other Equilibrium Notions Ichiro Obara UCLA January 21, 2012 Obara (UCLA) Other Equilibrium Notions January 21, 2012 1 / 28 Trembling Hand Perfect Equilibrium Trembling Hand Perfect Equilibrium We may

More information

Normal-form games. Vincent Conitzer

Normal-form games. Vincent Conitzer Normal-form games Vincent Conitzer conitzer@cs.duke.edu 2/3 of the average game Everyone writes down a number between 0 and 100 Person closest to 2/3 of the average wins Example: A says 50 B says 10 C

More information

The Stability of Strategic Plasticity

The Stability of Strategic Plasticity The Stability of Strategic Plasticity Rory Smead University of California, Irvine Kevin J.S. Zollman Carnegie Mellon University April 30, 2009 Abstract Recent research into the evolution of higher cognition

More information

Costly Signals and Cooperation

Costly Signals and Cooperation Costly Signals and Cooperation Károly Takács and András Németh MTA TK Lendület Research Center for Educational and Network Studies (RECENS) and Corvinus University of Budapest New Developments in Signaling

More information

Computing Minmax; Dominance

Computing Minmax; Dominance Computing Minmax; Dominance CPSC 532A Lecture 5 Computing Minmax; Dominance CPSC 532A Lecture 5, Slide 1 Lecture Overview 1 Recap 2 Linear Programming 3 Computational Problems Involving Maxmin 4 Domination

More information

A Generic Bound on Cycles in Two-Player Games

A Generic Bound on Cycles in Two-Player Games A Generic Bound on Cycles in Two-Player Games David S. Ahn February 006 Abstract We provide a bound on the size of simultaneous best response cycles for generic finite two-player games. The bound shows

More information

Learning to Compete, Compromise, and Cooperate in Repeated General-Sum Games

Learning to Compete, Compromise, and Cooperate in Repeated General-Sum Games Learning to Compete, Compromise, and Cooperate in Repeated General-Sum Games Jacob W. Crandall Michael A. Goodrich Computer Science Department, Brigham Young University, Provo, UT 84602 USA crandall@cs.byu.edu

More information

Complexity in social dynamics : from the. micro to the macro. Lecture 4. Franco Bagnoli. Lecture 4. Namur 7-18/4/2008

Complexity in social dynamics : from the. micro to the macro. Lecture 4. Franco Bagnoli. Lecture 4. Namur 7-18/4/2008 Complexity in Namur 7-18/4/2008 Outline 1 Evolutionary models. 2 Fitness landscapes. 3 Game theory. 4 Iterated games. Prisoner dilemma. 5 Finite populations. Evolutionary dynamics The word evolution is

More information

EC3224 Autumn Lecture #04 Mixed-Strategy Equilibrium

EC3224 Autumn Lecture #04 Mixed-Strategy Equilibrium Reading EC3224 Autumn Lecture #04 Mixed-Strategy Equilibrium Osborne Chapter 4.1 to 4.10 By the end of this week you should be able to: find a mixed strategy Nash Equilibrium of a game explain why mixed

More information

Weak Dominance and Never Best Responses

Weak Dominance and Never Best Responses Chapter 4 Weak Dominance and Never Best Responses Let us return now to our analysis of an arbitrary strategic game G := (S 1,...,S n, p 1,...,p n ). Let s i, s i be strategies of player i. We say that

More information

Reinforcement Learning. Introduction

Reinforcement Learning. Introduction Reinforcement Learning Introduction Reinforcement Learning Agent interacts and learns from a stochastic environment Science of sequential decision making Many faces of reinforcement learning Optimal control

More information

Political Economy of Institutions and Development: Problem Set 1. Due Date: Thursday, February 23, in class.

Political Economy of Institutions and Development: Problem Set 1. Due Date: Thursday, February 23, in class. Political Economy of Institutions and Development: 14.773 Problem Set 1 Due Date: Thursday, February 23, in class. Answer Questions 1-3. handed in. The other two questions are for practice and are not

More information

Games with Perfect Information

Games with Perfect Information Games with Perfect Information Yiling Chen September 7, 2011 Non-Cooperative Game Theory What is it? Mathematical study of interactions between rational and self-interested agents. Non-Cooperative Focus

More information

Meaning, Evolution and the Structure of Society

Meaning, Evolution and the Structure of Society Meaning, Evolution and the Structure of Society Roland Mühlenbernd November 7, 2014 OVERVIEW Game Theory and Linguistics Pragm. Reasoning Language Evolution GT in Lang. Use Signaling Games Replicator Dyn.

More information

Christopher Watkins and Peter Dayan. Noga Zaslavsky. The Hebrew University of Jerusalem Advanced Seminar in Deep Learning (67679) November 1, 2015

Christopher Watkins and Peter Dayan. Noga Zaslavsky. The Hebrew University of Jerusalem Advanced Seminar in Deep Learning (67679) November 1, 2015 Q-Learning Christopher Watkins and Peter Dayan Noga Zaslavsky The Hebrew University of Jerusalem Advanced Seminar in Deep Learning (67679) November 1, 2015 Noga Zaslavsky Q-Learning (Watkins & Dayan, 1992)

More information

UC Berkeley Haas School of Business Game Theory (EMBA 296 & EWMBA 211) Summer 2016

UC Berkeley Haas School of Business Game Theory (EMBA 296 & EWMBA 211) Summer 2016 UC Berkeley Haas School of Business Game Theory (EMBA 296 & EWMBA 211) Summer 2016 More on strategic games and extensive games with perfect information Block 2 Jun 12, 2016 Food for thought LUPI Many players

More information

Evolutionary Dynamics and Extensive Form Games by Ross Cressman. Reviewed by William H. Sandholm *

Evolutionary Dynamics and Extensive Form Games by Ross Cressman. Reviewed by William H. Sandholm * Evolutionary Dynamics and Extensive Form Games by Ross Cressman Reviewed by William H. Sandholm * Noncooperative game theory is one of a handful of fundamental frameworks used for economic modeling. It

More information

The Evolution of Cooperation under Cheap Pseudonyms

The Evolution of Cooperation under Cheap Pseudonyms The Evolution of Cooperation under Cheap Pseudonyms Michal Feldman John Chuang School of Information Management and Systems University of California, Berkeley Abstract A wide variety of interactions on

More information

arxiv:math/ v1 [math.oc] 29 Jun 2004

arxiv:math/ v1 [math.oc] 29 Jun 2004 Putting the Prisoner s Dilemma in Context L. A. Khodarinova and J. N. Webb Magnetic Resonance Centre, School of Physics and Astronomy, University of Nottingham, Nottingham, England NG7 RD, e-mail: LarisaKhodarinova@hotmail.com

More information

Bounded Rationality, Strategy Simplification, and Equilibrium

Bounded Rationality, Strategy Simplification, and Equilibrium Bounded Rationality, Strategy Simplification, and Equilibrium UPV/EHU & Ikerbasque Donostia, Spain BCAM Workshop on Interactions, September 2014 Bounded Rationality Frequently raised criticism of game

More information

Multiagent (Deep) Reinforcement Learning

Multiagent (Deep) Reinforcement Learning Multiagent (Deep) Reinforcement Learning MARTIN PILÁT (MARTIN.PILAT@MFF.CUNI.CZ) Reinforcement learning The agent needs to learn to perform tasks in environment No prior knowledge about the effects of

More information

6 The Principle of Optimality

6 The Principle of Optimality 6 The Principle of Optimality De nition A T shot deviation from a strategy s i is a strategy bs i such that there exists T such that bs i (h t ) = s i (h t ) for all h t 2 H with t T De nition 2 A one-shot

More information

Game Theory -- Lecture 4. Patrick Loiseau EURECOM Fall 2016

Game Theory -- Lecture 4. Patrick Loiseau EURECOM Fall 2016 Game Theory -- Lecture 4 Patrick Loiseau EURECOM Fall 2016 1 Lecture 2-3 recap Proved existence of pure strategy Nash equilibrium in games with compact convex action sets and continuous concave utilities

More information

MAS and Games. Department of Informatics Clausthal University of Technology SS 18

MAS and Games. Department of Informatics Clausthal University of Technology SS 18 . Dix/J. Müller Department of Informatics, TUC MAS and Games (MSc), SS 18 1 MAS and Games Prof. Dr. Jürgen Dix Prof. Dr. Jörg Müller Department of Informatics Clausthal University of Technology SS 18 J.

More information

Game theory lecture 4. September 24, 2012

Game theory lecture 4. September 24, 2012 September 24, 2012 Finding Nash equilibrium Best-response or best-reply functions. We introduced Nash-equilibrium as a profile of actions (an action for each player) such that no player has an incentive

More information

arxiv: v1 [physics.soc-ph] 14 Mar 2019

arxiv: v1 [physics.soc-ph] 14 Mar 2019 arxiv:1903.05991v1 [physics.soc-ph] 14 Mar 2019 The Rock Paper Scissors Game Hai-Jun Zhou Institute of Theoretical Physics, Chinese Academy of Sciences, Zhong-Guan-Cun East Road 55, Beijing 100190, China

More information

Mechanism Design: Implementation. Game Theory Course: Jackson, Leyton-Brown & Shoham

Mechanism Design: Implementation. Game Theory Course: Jackson, Leyton-Brown & Shoham Game Theory Course: Jackson, Leyton-Brown & Shoham Bayesian Game Setting Extend the social choice setting to a new setting where agents can t be relied upon to disclose their preferences honestly Start

More information

4. Opponent Forecasting in Repeated Games

4. Opponent Forecasting in Repeated Games 4. Opponent Forecasting in Repeated Games Julian and Mohamed / 2 Learning in Games Literature examines limiting behavior of interacting players. One approach is to have players compute forecasts for opponents

More information

University of Warwick, Department of Economics Spring Final Exam. Answer TWO questions. All questions carry equal weight. Time allowed 2 hours.

University of Warwick, Department of Economics Spring Final Exam. Answer TWO questions. All questions carry equal weight. Time allowed 2 hours. University of Warwick, Department of Economics Spring 2012 EC941: Game Theory Prof. Francesco Squintani Final Exam Answer TWO questions. All questions carry equal weight. Time allowed 2 hours. 1. Consider

More information

Multiagent Value Iteration in Markov Games

Multiagent Value Iteration in Markov Games Multiagent Value Iteration in Markov Games Amy Greenwald Brown University with Michael Littman and Martin Zinkevich Stony Brook Game Theory Festival July 21, 2005 Agenda Theorem Value iteration converges

More information

Perforce there perished many a stock, unable: By propagation to forge a progeny.

Perforce there perished many a stock, unable: By propagation to forge a progeny. 4 Evolution such things survived, being organized spontaneously in a fitting way; whereas those which grew otherwise perished and continue to perish.... Aristotle, Physics1 Evolution The ideas of natural

More information

Advanced Microeconomics

Advanced Microeconomics Advanced Microeconomics Games in strategic form Harald Wiese University of Leipzig Harald Wiese (University of Leipzig) Advanced Microeconomics 1 / 80 Part C. Games and industrial organization 1 Games

More information

Zero-Sum Games Public Strategies Minimax Theorem and Nash Equilibria Appendix. Zero-Sum Games. Algorithmic Game Theory.

Zero-Sum Games Public Strategies Minimax Theorem and Nash Equilibria Appendix. Zero-Sum Games. Algorithmic Game Theory. Public Strategies Minimax Theorem and Nash Equilibria Appendix 2013 Public Strategies Minimax Theorem and Nash Equilibria Appendix Definition Definition A zero-sum game is a strategic game, in which for

More information

Decision Theory: Markov Decision Processes

Decision Theory: Markov Decision Processes Decision Theory: Markov Decision Processes CPSC 322 Lecture 33 March 31, 2006 Textbook 12.5 Decision Theory: Markov Decision Processes CPSC 322 Lecture 33, Slide 1 Lecture Overview Recap Rewards and Policies

More information

Biology as Information Dynamics

Biology as Information Dynamics Biology as Information Dynamics John Baez Stanford Complexity Group April 20, 2017 What is life? Self-replicating information! Information about what? How to self-replicate! It is clear that biology has

More information

Game interactions and dynamics on networked populations

Game interactions and dynamics on networked populations Game interactions and dynamics on networked populations Chiara Mocenni & Dario Madeo Department of Information Engineering and Mathematics University of Siena (Italy) ({mocenni, madeo}@dii.unisi.it) Siena,

More information

Evolutionary Game Theory: Overview and Recent Results

Evolutionary Game Theory: Overview and Recent Results Overviews: Evolutionary Game Theory: Overview and Recent Results William H. Sandholm University of Wisconsin nontechnical survey: Evolutionary Game Theory (in Encyclopedia of Complexity and System Science,

More information

Convergence and No-Regret in Multiagent Learning

Convergence and No-Regret in Multiagent Learning Convergence and No-Regret in Multiagent Learning Michael Bowling Department of Computing Science University of Alberta Edmonton, Alberta Canada T6G 2E8 bowling@cs.ualberta.ca Abstract Learning in a multiagent

More information

Lecture Notes on Game Theory

Lecture Notes on Game Theory Lecture Notes on Game Theory Levent Koçkesen Strategic Form Games In this part we will analyze games in which the players choose their actions simultaneously (or without the knowledge of other players

More information

Computing Equilibria of Repeated And Dynamic Games

Computing Equilibria of Repeated And Dynamic Games Computing Equilibria of Repeated And Dynamic Games Şevin Yeltekin Carnegie Mellon University ICE 2012 July 2012 1 / 44 Introduction Repeated and dynamic games have been used to model dynamic interactions

More information

15-780: Graduate Artificial Intelligence. Reinforcement learning (RL)

15-780: Graduate Artificial Intelligence. Reinforcement learning (RL) 15-780: Graduate Artificial Intelligence Reinforcement learning (RL) From MDPs to RL We still use the same Markov model with rewards and actions But there are a few differences: 1. We do not assume we

More information

Evolutionary Game Theory

Evolutionary Game Theory UTRECHT UNIVERSITY BACHELOR THESIS MATHEMATICS AND APPLICATIONS Evolutionary Game Theory Author Daphne VAN HESTEREN Supervisor Dr. Karma DAJANI November 28, 2017 2 Abstract This thesis provides an introduction

More information

Bounded Rationality Lecture 2. Full (Substantive, Economic) Rationality

Bounded Rationality Lecture 2. Full (Substantive, Economic) Rationality Bounded Rationality Lecture 2 Full (Substantive, Economic) Rationality Mikhail Anufriev EDG, Faculty of Business, University of Technology Sydney (UTS) European University at St.Petersburg Faculty of Economics

More information

REPEATED GAMES. Jörgen Weibull. April 13, 2010

REPEATED GAMES. Jörgen Weibull. April 13, 2010 REPEATED GAMES Jörgen Weibull April 13, 2010 Q1: Can repetition induce cooperation? Peace and war Oligopolistic collusion Cooperation in the tragedy of the commons Q2: Can a game be repeated? Game protocols

More information

Computational Problems Related to Graph Structures in Evolution

Computational Problems Related to Graph Structures in Evolution BACHELOR THESIS Štěpán Šimsa Computational Problems Related to Graph Structures in Evolution Department of Applied Mathematics Supervisor of the bachelor thesis: Study programme: Study branch: Prof. Krishnendu

More information

Learning Equilibrium as a Generalization of Learning to Optimize

Learning Equilibrium as a Generalization of Learning to Optimize Learning Equilibrium as a Generalization of Learning to Optimize Dov Monderer and Moshe Tennenholtz Faculty of Industrial Engineering and Management Technion Israel Institute of Technology Haifa 32000,

More information

Chapter 7. Evolutionary Game Theory

Chapter 7. Evolutionary Game Theory Chapter 7 Evolutionary Game Theory In Chapter 6, we developed the basic ideas of game theory, in which individual players make decisions, and the payoff to each player depends on the decisions made by

More information

Population Games and Evolutionary Dynamics

Population Games and Evolutionary Dynamics Population Games and Evolutionary Dynamics (MIT Press, 200x; draft posted on my website) 1. Population games 2. Revision protocols and evolutionary dynamics 3. Potential games and their applications 4.

More information

Some Thought-ettes on Artificial Agent Modeling and Its Uses

Some Thought-ettes on Artificial Agent Modeling and Its Uses Some Thought-ettes on Artificial Agent Modeling and Its Uses Steven O. Kimbrough University of Pennsylvania http://opim-sun.wharton.upenn.edu/ sok/ kimbrough@wharton.upenn.edu ECEG, Washington, D.C., 1/26/04

More information

Game Theory: Lecture 3

Game Theory: Lecture 3 Game Theory: Lecture 3 Lecturer: Pingzhong Tang Topic: Mixed strategy Scribe: Yuan Deng March 16, 2015 Definition 1 (Mixed strategy). A mixed strategy S i : A i [0, 1] assigns a probability S i ( ) 0 to

More information

Opponent Modelling by Sequence Prediction and Lookahead in Two-Player Games

Opponent Modelling by Sequence Prediction and Lookahead in Two-Player Games Opponent Modelling by Sequence Prediction and Lookahead in Two-Player Games Richard Mealing and Jonathan L. Shapiro {mealingr,jls}@cs.man.ac.uk Machine Learning and Optimisation Group School of omputer

More information

Machine Learning. Reinforcement learning. Hamid Beigy. Sharif University of Technology. Fall 1396

Machine Learning. Reinforcement learning. Hamid Beigy. Sharif University of Technology. Fall 1396 Machine Learning Reinforcement learning Hamid Beigy Sharif University of Technology Fall 1396 Hamid Beigy (Sharif University of Technology) Machine Learning Fall 1396 1 / 32 Table of contents 1 Introduction

More information