Game Theory with Information: Introducing the Witsenhausen Intrinsic Model

Transcription

1 Game Theory with Information: Introducing the Witsenhausen Intrinsic Model Michel De Lara and Benjamin Heymann Cermics, École des Ponts ParisTech France École des Ponts ParisTech March 15, 2017

2 Information plays a crucial role in competition Information who knows what and when plays a crucial role in competitive contexts Concealing, cheating, lying, deceiving are effective strategies Our goals are to 1. introduce the notion of game in intrinsic form 2. contribute to the analysis of decentralized, non-cooperative decision settings 3. provide a (very) general mathematical language for game theory and mechanism design

3 What is a game in intrinsic form? Nature, the source of all randomness, or states of Nature Agents, who hold information make decisions, by means of admissible strategies, those fueled by information Players, who hold beliefs about states of Nature hold a subset of agents under their exclusive control (team of executives) hold objectives, that they achieve by selecting proper admissible strategies for the agents under their control In Witsenhausen s intrinsic form of a game, there is no tree structure (whereas Kuhn s extensive form of a game relies on a tree)

4 Outline of the presentation Ingredients of Witsenhausen intrinsic model Players and Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

5 Outline of the presentation Ingredients of Witsenhausen intrinsic model Players and Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

6 Outline of the presentation Ingredients of Witsenhausen intrinsic model Agents and decisions, Nature, history Information fields and stochastic systems Strategies and admissible strategies Solvability and solution map Players and Nash equilibrium in Witsenhausen intrinsic model Players in Witsenhausen intrinsic model Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

7 We will distinguish an individual from an agent An individual who makes a first, followed by a second decision, is represented by two agents (two decision makers) An individual who makes a sequence of decisions one for each period t = 0, 1, 2,..., T 1 is represented by T agents, labelled t = 0, 1, 2,..., T 1 N individuals each i of whom makes a sequence of decisions, one for each period t = 0, 1, 2,..., T i 1 is represented by N i=1 T i agents, labelled by (i, t) N {j} {0, 1, 2,..., T j 1} j=1

8 Agents, decisions and decision space Let A be a finite set, whose elements are called agents (or decision-makers) Each agent a A is supposed to make one decision u a U a where the set Ua is the set of decisions for agent a and is equipped with a σ-field Ua We define the decision space as the product set U A = b A U b equipped with the product decision field U A = b A U b Examples A = {0, 1, 2,..., T 1} (T sequential decisions) A = {Pr, Ag} (principal-agent models)

9 States of Nature and history space A state of Nature (or uncertainty, or scenario) is ω Ω where the set Ω is a measurable set, the sample space, equipped with a σ-field F (at this stage of the presentation, we do not need probability distribution, as we focus only on information) The history space is the product space H = U A Ω = b A U b Ω equipped with the product history field H = U A F = b A U b F Examples States of Nature Ω can include types of players, randomness, stochastic processes

10 One agent, two possible decisions, two states of Nature Agents Decision set and field A = {a} U a = {ua, 1 ua} 2, U a = {, {ua, 1 ua}, 2 {ua}, 1 {ua}} 2 Sample space and field Ω = {ω 1, ω 2 }, F = {, {ω 1, ω 2 }, {ω 1 }, {ω 2 }} History space and field H = U a Ω = {ua, 1 ua} 2 {ω 1, ω 2 }, H = 2 H

11 Two agents, two possible decisions, two states of Nature Agents Decision sets and fields A = {a, b} U a = {ua, 1 ua} 2, U a = {, {ua, 1 ua}, 2 {ua}, 1 {ua}} 2 and U b = {ub, 1 ub} 2, U b = {, {ub, 1 ub}, 2 {ub}, 1 {ub}} 2 Sample space and field Ω = {ω 1, ω 2 }, F = {, {ω 1, ω 2 }, {ω 1 }, {ω 2 }} History space and field H = U a U b Ω = {ua, 1 ua} 2 {ub, 1 ub} 2 {ω 1, ω 2 }, H = 2 H

12 Two players, T stages Agents Decision sets and fields and A = {p, q} {0, 1,..., T 1} U (p,t) = R np, U (p,t) = B o np R, t = 0, 1,..., T 1 U (q,t) = R nq, U (q,t) = B o nq R, t = 0, 1,..., T 1 Sample space and field (Ω, F) History space and field H = T 1 t=0 T 1 U (p,t) t=0 T 1 U (q,t) Ω, H = t=0 T 1 U (p,t) t=0 U (q,t) F

13 Outline of the presentation Ingredients of Witsenhausen intrinsic model Agents and decisions, Nature, history Information fields and stochastic systems Strategies and admissible strategies Solvability and solution map Players and Nash equilibrium in Witsenhausen intrinsic model Players in Witsenhausen intrinsic model Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

14 Information fields The information field of agent a A is a σ-field I a H In this representation, I a is a subfield of the history field H which represents the information available to agent a when he makes a decision Therefore, the information of agent a may depend on the states of Nature and on other agents decisions

15 One agent, two possible decisions, two states of Nature History space and field H = U a Ω = {u 1 a, u 2 a} {ω 1, ω 2 }, F = 2 Ω, H = 2 H Case where agent a knows nothing I a = {, U a } {, Ω} = {, {u 1 a, u 2 a}} {, {ω 1, ω 2 }} Case where agent a knows the state of Nature I a ={, U a } F ={, U a } {, {ω 1, ω 2 }, {ω 1 }, {ω 2 }} = {, {ua, 1 ua}} 2 }{{} {, {ω 1, ω 2 }, {ω 1 }, {ω 2 }} }{{} undistinguishable distinguishable

16 Two agents, two possible decisions, two states of Nature Nested information fields History space and field H = U a U b Ω = {ua, 1 ua} 2 {ub, 1 ub} 2 {ω 1, ω 2 }, H = 2 H Agent a knows the state of Nature I a = {, U a } {, U b } {, {ω 1, ω 2 }, {ω 1 }, {ω 2 }} and agent b knows the state of Nature and what agent a does I b = {, {ua, 1 ua}, 2 {ua}, 1 {ua}} 2 {, U b } {, {ω 1, ω 2 }, {ω 1 }, {ω 2 }} In this example, information fields are nested I a I b meaning that agent b knows what agent a knows

17 Two agents, two decisions, two states of Nature Non nested information fields History space and field H = U a U b Ω = {u 1 a, u 2 a} {u 1 b, u 2 b} {ω 1, ω 2 }, H = 2 H Agent a only knows the state of Nature I a = {, U a } {, U b } {, {ω 1, ω 2 }, {ω 1 }, {ω 2 }} and agent b only knows what agent a does I b = {, {u 1 a, u 2 a}, {u 1 a}, {u 2 a}} {, U b } {, {ω 1, ω 2 }} Information fields are not nested, I a I b, as they cannot be compared by inclusion

18 Classical information patterns in game theory Two agents: the principal Pr (leader) and the agent Ag (follower) Moral hazard (the insurance company cannot observe if the insured plays with matches at home) Stackelberg leadership model I Pr {, U Ag } {, U Pr } F I Ag {, U Ag } U Pr F, I Pr {, U Ag } {, U Pr } F Adverse selection (the insurance company cannot observe if the insured has good health) {, U Ag } {, U Pr } F I Ag, I Pr U Ag {, U Pr } {, Ω} Signaling (the peacock s tail signals his good genes) {, U Ag } {, U Pr } F I Ag, I Pr = U Ag {, U Pr } {, Ω}

19 Two players, T stages Agents A = {p, q} {0, 1,..., T } Information fields (at most, past decisions and state of Nature) t 1 T t 1 T I (p,t) U (p,s) {, U (p,s) } U (q,s) {, U (q,s) } F s=0 s=t s=0 s=t t 1 T t 1 T I (q,t) U (p,s) {, U (p,s) } U (q,s) {, U (q,s) } F s=0 s=t s=0 s=t

20 Stochastic system Stochastic system A stochastic system is a collection consisting of a finite set A of agents states of Nature (Ω, F) decision sets, fields and information fields {U a, U a, I a } a A

21 We will consider stochastic systems that display absence of self-information Absence of self-information A stochastic system displays absence of self-information when for any agent a A I a U A\{a} F Absence of self-information means that the information of agent a may depend on the states of Nature and on all the other agents decisions but not on his own decision Absence of self-information makes sense once we have distinguished an individual from an agent (else, it would lead to paradoxes)

22 What land have we covered? What comes next? The stage is in place; so are the actors Nature agents information How can actors play? admissible strategies solvability

23 Outline of the presentation Ingredients of Witsenhausen intrinsic model Agents and decisions, Nature, history Information fields and stochastic systems Strategies and admissible strategies Solvability and solution map Players and Nash equilibrium in Witsenhausen intrinsic model Players in Witsenhausen intrinsic model Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

24 Information fuels admissible strategies A strategy (or policy, control law, control design) for agent a is a measurable mapping λ a : (H, H) (U a, U a ) Admissible strategy An admissible strategy for agent a is a mapping λ a : (H, H) (U a, U a ) which is measurable w.r.t. the information field I a of agent a, that is, λ 1 a (U a ) I a This condition expresses the property that an admissible strategy for agent a may only depend upon the information I a available to him

25 Set of admissible strategies We denote the set of admissible strategies of agent a by Λ ad a = { λ a : (H, H) (U a, U a ) λ 1 a (U a ) I a } and the set of admissible strategies of all agents is Λ ad A = a A Λ ad a

26 Examples of admissible strategies Consider a stochastic system with two agents a and b, and suppose that σ-fields U a, U b and F contain singletons Absence of self-information I a {, U a } U b F, I b U a {, U b } F Then, admissible strategies λ a and λ b have the form λ a ( u a, u b, ω) = λ a (u b, ω), λ b (u a, u b, ω) = λ b (u a, ω) Sequential I a = {, U a } {, U b } F, I b = U a {, U b } F Then, admissible strategies λ a and λ b have the form λ a ( u a, u b, ω) = λ a (ω), λ b (u a, u b, ω) = λ b (u a, ω)

27 Outline of the presentation Ingredients of Witsenhausen intrinsic model Agents and decisions, Nature, history Information fields and stochastic systems Strategies and admissible strategies Solvability and solution map Players and Nash equilibrium in Witsenhausen intrinsic model Players in Witsenhausen intrinsic model Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

28 Solvability (I) In the Witsenhausen s intrinsic model, agents make decisions in an order which is not fixed in advance Briefly speaking, solvability is the property that, for each state of Nature, the agents decisions are uniquely determined by their admissible strategies The solvability property is crucial to develop Witsenhausen s theory: without the solvability property, we would not be able to determine the agents decisions

29 Solvability (II) The solvability problem consists in finding for any collection λ = {λ a } a A Λ ad A for any state of Nature ω Ω of admissible policies decisions u U A satisfying the implicit ( closed loop ) equation or, equivalently, u = λ(u, ω) u a = λ a ({u b } b A, ω), a A Solvability property A stochastic system displays the solvability property when λ Λ ad A, ω Ω,!u U A, u = λ(u, ω)

30 Solvability and information patterns Sequential in which case I a = {, U a } {, U b } F, I b = U a {, U b } F u a = λ a ( u a, u b, ω) = λ a (ω), u b = λ b (u a, u b, ω) = λ b (u a, ω) always displays a unique solution (u a, u b ), whatever ω Ω and λ a and λ b Deadlock I a = {, U a } U b {, Ω}, I b = U a {, U b } {, Ω} in which case u a = λ a (u b ), u b = λ b (u a ) may display zero solutions, one solution or multiple solutions, depending on the functional properties of λ a and λ b

31 Solvability makes it possible to define a solution map Solution map Suppose that the solvability property holds true. We define the solution map S λ : Ω H, that maps states of Nature towards histories, by (u, ω) = S λ (ω) u = λ(u, ω), (u, ω) U A Ω We include the state of Nature ω in the image of S λ (ω), so that we map the set Ω towards the history space H, making it possible to interpret S λ (ω) as a history driven by the admissible strategy λ (in classical control theory, a state trajectory is produced by a policy) For example, in the sequential case, S λ (ω) = ( λ a (ω), λ b ( λ a (ω), ω), ω)

32 What land have we covered? What comes next? The stage is in place; so are the actors Nature agents information Actors know how they can play admissible strategies solvability In a non-cooperative context, we need objectives beliefs a notion of equilibrium

33 Outline of the presentation Ingredients of Witsenhausen intrinsic model Players and Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

34 Outline of the presentation Ingredients of Witsenhausen intrinsic model Agents and decisions, Nature, history Information fields and stochastic systems Strategies and admissible strategies Solvability and solution map Players and Nash equilibrium in Witsenhausen intrinsic model Players in Witsenhausen intrinsic model Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

35 Players hold teams of executive agents, objective functions and beliefs The set of players is denoted by P Every player p P has a team of executive agents A p A, where (A p) p P forms a partition of the set A of agents a criterion (objective function) j p : H R, a measurable function over the history space H a belief P p : F [0, 1], a probability distribution over the states of Nature (Ω, F)

36 Example: two players, one agent per player Agents A = {a, b} Players p = {a}, A p = {a}, q = {b}, A q = {b} Criteria j {a} (u a, u b, ω), j {b} (u a, u b, ω) Beliefs P {a} and P {b} over (Ω, F)

37 Example: two players, T stages Agents A = {p, q} {0, 1,..., T 1} Players P = {p, q} A p = {p} {0, 1,..., T 1}, A q = {q} {0, 1,..., T 1} Criteria T 1 j p (u (p,0),..., u (p,t 1), u (q,0),..., u (q,t 1), ω) = L p,t (u (p,t), u (q,t), ω) T 1 j q (u (p,0),..., u (p,t 1), u (q,0),..., u (q,t 1), ω) = L q,t (u (p,t), u (q,t), ω) Beliefs P p and P q over (Ω, F) t=0 t=0

38 How player p evaluates an admissible strategies profile λ Measurable solution map attached to λ Λ ad A is S λ : Ω H Measurable criterion (costs or payoffs) is j p : H R The composition of criteria with the solution map provides a random variable j p S λ : Ω R The random variable can be integrated w.r.t. the belief P p, yielding E Pp [ jp S λ ] R where E Pp denotes the mathematical expectation w.r.t. the probability P p on (Ω, F)

39 Outline of the presentation Ingredients of Witsenhausen intrinsic model Agents and decisions, Nature, history Information fields and stochastic systems Strategies and admissible strategies Solvability and solution map Players and Nash equilibrium in Witsenhausen intrinsic model Players in Witsenhausen intrinsic model Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

40 Pure (admissible) strategies profiles A pure (admissible) strategy for player p is an element of Λ ad A p = Λ ad a a A p The set of pure (admissible) strategies for all players is Λ ad A p = Λ ad a = Λ ad a = Λ ad A p P p P a A p a A An (admissible) strategies profile is λ = (λ p ) p P p P Λ ad A p When we focus on player p, we write λ = (λ p, λ p ) Λ ad A p p p Λ ad A p } {{ } Λ ad A p

41 Pure Bayesian Nash equilibrium We say that the pure (admissible) strategies profile λ = (λ p ) p P p P Λ ad A p is a Bayesian Nash equilibrium if (in case of payoffs), for all p P, ] [ ] E Pp [j p S (λp,λ p) E Pp j p S (λp,λ p), λ p Λ ad A p

42 Mixed (admissible) strategies profiles A mixed (admissible) strategy (or randomized strategy) for player p is an element of ( Λ ad ) ( A p = Λ ad ) a a A p the set of probability distributions over the set of (admissible) strategies of his executives in A p The definition of mixed strategies for player p reflects his ability to coordinate his team of executives in A p By contrast, behavioral (admissible) strategies for player p are a A p ( ) ( Λ ad a Λ ad ) a a A p and they do not require any correlating procedure

43 Mixed (admissible) strategies for players The set of mixed (admissible) strategies profiles is p P ( Λ ad A p ) = p P A mixed (admissible) strategies profile is µ = (µ p ) p P p P ( Λ ad ) a a A p ( Λ ad A p ) When we focus on player p, we write µ = (µ p, µ p ) ( Λ ad A p ) p p ( Λ ad ) A p

44 Mixed Bayesian Nash equilibrium We say that the mixed (admissible) strategies profile µ = (µ p ) p P p P ( Λ ad A p ) is a Bayesian Nash equilibrium if (in case of payoffs), for all p P, ] µ p (dλ p ) µ p (dλ p ) E Pp [j p S (λp,λ p) Λ ad p Λad p Λ ad p Λad p µ p (dλ p ) µ p (dλ p ) E Pp [j p S (λp,λ p) ], µ p ( Λ ad ) p

45 Technical difficulties With which σ-algebra M can we equip the set of admissible strategies Λ ad A? So that we can consider ( ) Λ ad A, the set of probability distributions over Λ ad A Is the solution map Λ ad A measurable w.r.t. M F? Ω H, (λ, ω) S λ (ω) Do we have to restrict to a subset of Λ ad A?

46 What land have we covered? What comes next? Witsenhausen intrinsic games cover deterministic games (with finite or measurable decision sets) deterministic dynamic games (finite span time) stochastic games stochastic dynamic games (finite span time) games in Kuhn extensive form (finite span time) For games with enumerable or continuous span time, the Witsenhausen intrinsic model has to be adapted

47 Outline of the presentation Ingredients of Witsenhausen intrinsic model Players and Nash equilibrium in Witsenhausen intrinsic model Open questions (and research agenda)

48 Research questions How should we talk about games using WIM? Can we extend the Bayesian Nash Equilibrium concept to general risk measures? Can we re-organize the games bestiary using WIM? How does the notion of subgame perfect Nash equilibrium translate within this framework? WIM: game theoretical results What would a Nash theorem be in the WIM setting? When do we have a generalized backward induction mechanism? Under proper sufficient conditions on the information structure (extension of perfect recall), can we restrict the search among behavioral strategies instead of mixed strategies? Applications of WIM What kind of applications do we target? Can we use the WIM framework for mechanism design?

49 Nash Equilibrium with general risk measures We denote real-valued random variables on (Ω, F) by L(Ω, F) = {X : (Ω, F) (R, B R ), X 1 (B R ) F} A risk measure G p for the player p is a mapping G p : L(Ω, F) R {+ } We say that the players mixed (admissible) strategies profile µ = (µ p ) p P p P ( Λ ad A p ) is a Nash equilibrium if (in case of payoffs), for all p P, ] µ p (dλ p ) µ p (dλ p ) G p [j p S (λp,λ p) Λ ad p Λad p Λ ad p Λad p µ p (dλ p ) µ p (dλ p ) G p [j p S (λp,λ p) ], µ p ( Λ ad ) p

50 Can we re-organize the games bestiary using WIM? With four relations between agents, introduced by Witsenhausen, Precedence relation P Subsystem relation S Information-memory relation M Decision-memory relation D we can provide a typology of systems Static team Station Sequential systems Partially nested systems Quasiclassical systems Classical systems Hierarchical systems Parallel coordinated systems

51 Subgames and subgame perfect Nash equilibrium A subgame can be defined thanks to the notion of subsystem of agents in the WIM setting What are the conditions on a subsystem w.r.t. players and their criteria that make it possible to define a subgame?

52 We obtain a Nash theorem in the WIM setting Theorem Any finite, solvable, Witsenhausen game has a mixed Nash equilibrium Proof The set of strategies is finite, as strategies map the finite history set towards finite decision sets To each strategy profile, we associate a payoff vector We thus obtain a matrix game and we can apply Nash theorem

53 Generalized existence result of Nash equilibria By discretization Discretize decisions sets and sample space, and equip them with trace σ-fields Introduce discretized history set and σ-field Current difficulties: How can we discretize information fields so that they are subfields of the trace history σ-field? With which topology can we equip the sample space Ω and the set Λ ad A of admissible strategies? Can we prove continuity for the solution map Λ ad A Ω H, (λ, ω) S λ (ω)? Do we have to restrict to a subset of Λ ad A (like continuous admissible strategies)?

54 Generalized existence result of Nash equilibria By best-reply set-valued mapping Define the best-reply set-valued mapping ( Λ ad ) A p ( Λ ad ) A p Current difficulties: p P p P With which topology can we equip the sample space Ω and the set Λ ad A of admissible strategies? Can we prove continuity for the solution map Λ ad A Ω H, (λ, ω) S λ (ω)? Do we have to restrict to a subset of Λ ad A (like continuous admissible strategies)? What are the properties of the best-reply set-valued mapping? (measurability, convexity, continuity)? What are the proper fixed point theorems for set-valued mappings?

55 When do we have a generalized backward induction mechanism? Witsenhausen introduced the notion of strategy independence of conditional expectation (SICE) He showed that SICE was a key assumption for a generalized backward induction mechanism in stochastic optimal control Under assumption SICE, we provide sufficient conditions for a two players Bayesian Nash equilibrium to be obtained by bi-level optimization

56 Behavioral vs mixed strategies Mixed strategies profiles are p P ( Λ ad ) a a A p and reflect the synchronization of his agents by the player Behavioral strategies profiles are ( Λ ad p P a A p a ) and they do not require any correlating procedure Under proper sufficient conditions on the information structure generalizing perfect recall we expect to prove that some games can be solved over the smaller set of behavioral strategies profiles instead of the large set of mixed strategies profiles ( ) Λ ad a ( Λ ad ) a p P a A p p P a A p }{{}}{{} behavioral mixed

57 What kind of applications do we target? The WIM is of particular interest for non sequential games In particular we envision applications for networks, auctions and decentralized energy systems

58 Mechanism design presented in the intrinsic framework The designer (= principal) can extend the natural history set, by offering new decisions to every agent (messages) He is free to extend the information fields of the agents as he wishes He can partly shape the objective functions of the players

59 Conclusion a rich language a lot of open questions, and a lot of things not yet properly defined we are looking for feedback Thank you :-)