Evolutionary Bargaining Strategies

Transcription

1 Evolutionary Bargaining Strategies Nanlin Jin

2 Evolutionary Bargaining Two players alternative offering game x A =?? Player A Rubinstein 1982, 1985: Subgame perfect equilibrium found Slight game modification Laborious work on new solutions Player B Co-evolution Approximated Rubinstein 1982, 1985 Technical details non-trivial Incentive methods helped Proposal: EC for approximating solutions on new games

3 Evolutionary Bargaining Strategies Prisoners Dilemma Co-evolution GA vs PBIL Bubinstein s Model Offer at time t = f (r A, r B, t) x A*, x B* emerged as best results Other solutions emerged occasionally Current work Asymmetric information Outside options

4 Evolutionary Rubinstein Bargaining, Overview Game theorists solved Rubinstein bargaining problem Subgame Perfect Equilibrium (SPE) Slight problem alterations lead to different solutions Outside option, Asymmetric info, Different time intervals Evolutionary computation Succeeded in solving a wide range of problems EC found SPE in Rubinstein 82 model Can EC find solutions close to unknown SPE? Co-evolution is an alternative approximation method to find game theoretical solutions Less time for approximate SPEs Less modifications for new problems

5 Rubinstein Solution vs Experimental Results (δ A, δ B ) Rubinstein Experimental x A Solution x A (0.4, 0.4) (0.4, 0.6) (0.4, 0.9) (0.9, 0.4) (0.9, 0.6) (0.9, 0.9) (0.9, 0.99) x A : agreement made by the best strategies in the final (300 th ) generation Population size 100; Crossover rates 0 to 0.1; Mutation rates 0.01 to 0.5; Tournament size 3

6 Evolutionary Computation for Bargaining Technical Details

7 Issues Addressed, EC for Bargaining Representation Should t be in the language? One or two population? How to evaluate fitness Fixed or relative fitness? How to contain search space? Discourage irrational strategies: Ask for x A >1? Ask for more over time? Ask for more when δ A is low? 1 δ B / 1 δ A δ B

8 Two populations, co-evolution We want to deal with asymmetric games E.g. two players may have different information One population for training each player s strategies Co-evolution, using relative fitness Alternative: use absolute fitness Player 1 Player 2 Evolve over time

9 Representation of Strategies A tree represents a mathematical function g Terminal set: {1, δ A, δ B } Functional set: {+,,, } Given g, player with discount rate r plays at time t g (1 r) t Language can be enriched: Could have included e or time t to terminal set Could have included power ^ to function set Richer language larger search space harder search problem

10 Incentive Method: Constrained Fitness Function No magic in evolutionary computation Larger search space less chance to succeed Constraints are heuristics to focus a search Focus on space where promising solutions may lie Incentives for the following properties in the function returned: The function returns a value in (0, 1) Everything else being equal, lower δ A smaller share Everything else being equal, lower δ B larger share Note: this is the key to our search effectiveness

11 Models with known equilibriums Complete Information Rubinstein 82 model: Alternative offering, both A and B know δ A & δ B Evolved solutions approximates theoretical Working on a model with outside option Incomplete Information Rubinstein 85 model: B knows δ A & δ B A knows δ A and δ B weak & δ B strong with probability Ω weak Evolved solutions approximates theoretical

12 Models with unknown equilibriums Modified Rubinstein 85 models Incomplete knowledge B knows δ B but not δ A ; A knows δ A but not δ B Asymmetric knowledge B knows δ A & δ B ; A knows δ A but not δ B Asymmetric, limited knowledge B knows δ A & δ B A knows δ A and a normal distribution of δ B Outside option

13 Evolutionary Bargaining, Conclusions Demonstrated GP s flexibility Models with known and unknown solutions Outside option Incomplete, asymmetric and limited information Co-evolution is an alternative approximation method to find game theoretical solutions Relatively quick for approximate solutions Relatively easy to modify for new models Genetic Programming with incentive / constraints Constraints used to focus the search in promising spaces

14 Bargaining in Game Theory Rubinstein 1982 Model: π = Cake to share between A and B (= 1) In reality: A and B make alternate offers Offer at time t = f (r x A = A s share (x B = π A, r x B, t) A ) Is r A = it A s necessary? discount rate Is t = it # rational? of rounds, at (What time isper rational?) round A s payoff x A drops as time goes by A s Payoff = x A exp( r A t ) Important Assumptions: Both players rational Both players know everything Equilibrium solution for A: µ A = (1 δ B ) / (1 δ A δ B ) where δ i = exp( r i ) 0? π Decay of Payoff over time A Payoff Time t Optimal offer: x A = µ A at t=0 x A x B B Notice: No time t here

15 Discussions on Bargaining

16 What is Rationality? Are we all logical? What if Computation is involved? Does Consequential Closure hold? If we know P is true and P Q, then we know Q is true We know all the rules in Chess, but not the optimal moves due to combinatorial explosion Rationality depends on computation power! Think faster more rational

17 Combinatorial Explosion A B C D E F G H Put 1 penny in square 1 2 pennies in square 2 4 pennies in square 3, etc. Even the world s richest man can t afford it p = 100,000,000 Billion Pennies Squares

18 Stochastic Search, Motivation Schedule 30 jobs to 10 machines: Search space: leaf nodes Generously allow: Explore one in every leaf nodes! Examine nodes per second! Problem may take 300 years to solve!!! May be lucky to find first solution But finding optimality takes time Complete methods limited by combinatorial explosion

19 Game Theory Hall of Frame 1994 Nobel Prize John Harsanyi John Nash Reinhard Selten 2005 Nobel Prize Robert Aumann Thomas Schelling

20 1994 Nobel Economic Prize Winners John Harsanyi (Berkeley) Incomplete information John Forbes Nash (Princeton) Non-cooperative games Reinhard Selten (Bonn) Bounded rationality (after Herbert Simon) Experimental economics

21 1978 Nobel Economic Prize Winner Artificial intelligence For his pioneering research into the decisionmaking process within economic organizations" The social sciences, I thought, needed the same kind of rigor and the same mathematical underpinnings that had made the "hard" sciences so brilliantly successful. Bounded Rationality A Behavioral model of Rational Choice 1957 Herbert Simon (CMU) Artificial intelligence Sources:

22 2005 Nobel Economic Prizes Winners Robert J. Aumann, and Thomas C. Schelling won 2005 s Noel memorial prize in economic sciences For having enhanced our understanding of conflict and cooperation through game-theory analysis Robert J. Aumann 75 Thomas C. Schelling 84 Source: Updated: 2:49 p.m. ET Oct. 10, 2005

23 Robert J. Aumann Winner of 2005 Nobel Economic Prize Born 1930 Hebrew Univ of Jerusalem & US National Academy of Sciences Producer of Game Theory (Schelling) Repeated games Defined Correlated Equilibrium Uncertainty not random But depend on info on opponent Common knowledge

24 Thomas C. Schelling Winner of 2005 Nobel Economic Prize Born 1921 University of Maryland User of Game Theory (Schelling) Book The Strategy of Conflict 1960 Bargaining theory and strategic behavior Book Arms and Influence 1966 foreign affairs, national security, nuclear strategy,... Paper Dynamic models of segregation 1971 Small preference to one s neighbour segregation