Smooth Calibration, Leaky Forecasts, Finite Recall, and Nash Dynamics

Transcription

1 Smooth Calibration, Leaky Forecasts, Finite Recall, and Nash Dynamics Sergiu Hart August 2016

2 Smooth Calibration, Leaky Forecasts, Finite Recall, and Nash Dynamics Sergiu Hart Center for the Study of Rationality Dept of Mathematics Dept of Economics The Hebrew University of Jerusalem

3 Joint work with Dean P. Foster University of Pennsylvania & Amazon Research

4 Papers

5 Papers Dean P. Foster and Sergiu Hart Smooth Calibration, Leaky Forecasts, Finite Recall, and Nash Dynamics 2012 revised:

6 Papers Dean P. Foster and Sergiu Hart Smooth Calibration, Leaky Forecasts, Finite Recall, and Nash Dynamics 2012 revised: Dean P. Foster and Sergiu Hart An Integral Approach to Calibration 2016 (in preparation)

7 Calibration

8 Calibration Forecaster says: "The chance of rain tomorrow is p"

9 Calibration Forecaster says: "The chance of rain tomorrow is p" Forecaster is CALIBRATED if for every p: the proportion of rainy days among those days when the forecast was p equals p (or is close to p in the long run)

10 Calibration Forecaster says: "The chance of rain tomorrow is p" Forecaster is CALIBRATED if for every p: the proportion of rainy days among those days when the forecast was p equals p (or is close to p in the long run) Dawid 1982

11 Calibration

12 Calibration CALIBRATION can be guaranteed (no matter what the weather is)

13 Calibration CALIBRATION can be guaranteed (no matter what the weather is) Foster and Vohra 1994 [publ 1998]

14 Calibration CALIBRATION can be guaranteed (no matter what the weather is) Forecaster uses mixed forecasting (e.g.: with probability 1/2, forecast = 25% with probability 1/2, forecast = 60%) Foster and Vohra 1994 [publ 1998]

15 Calibration CALIBRATION can be guaranteed (no matter what the weather is) Forecaster uses mixed forecasting (e.g.: with probability 1/2, forecast = 25% with probability 1/2, forecast = 60%) Foster and Vohra 1994 [publ 1998] Hart 1995: proof using Minimax Theorem

16 The MINIMAX Theorem

17 The MINIMAX Theorem THEOREM (von Neumann 1928) IF X R n, Y R m are compact convex sets, and f : X Y R is a continuous function that is convex-concave, i.e., f(, y) : X R is convex for fixed y, and f(x, ) : Y R is concave for fixed x, min max x X y Y THEN f(x, y) = max min y Y x X f(x, y).

18 The MINIMAX Theorem

19 The MINIMAX Theorem THEOREM (von Neumann 1928)

20 The MINIMAX Theorem THEOREM (von Neumann 1928) IF For every strategy of the opponent I have a strategy such that my payoff is at least v

21 The MINIMAX Theorem THEOREM (von Neumann 1928) IF For every strategy of the opponent I have a strategy such that my payoff is at least v THEN I have a strategy that guarantees that my payoff is at least v (for every strategy of the opponent)

22 The MINIMAX Theorem THEOREM (von Neumann 1928) IF For every strategy of the opponent I have a strategy such that my payoff is at least v THEN I have a strategy that guarantees that my payoff is at least v (for every strategy of the opponent) finite game;

23 The MINIMAX Theorem THEOREM (von Neumann 1928) IF For every strategy of the opponent I have a strategy such that my payoff is at least v THEN I have a strategy that guarantees that my payoff is at least v (for every strategy of the opponent) finite game; probabilistic (mixed) strategies

24 Calibration Proof: Minimax

25 Calibration Proof: Minimax FINITE δ-grid, FINITE HORIZON FINITE 2-person 0-sum game

26 Calibration Proof: Minimax FINITE δ-grid, FINITE HORIZON FINITE 2-person 0-sum game IF the strategy of the rainmaker IS KNOWN THEN the forecaster can get δ-calibrated forecasts

27 Calibration Proof: Minimax FINITE δ-grid, FINITE HORIZON FINITE 2-person 0-sum game IF the strategy of the rainmaker IS KNOWN THEN the forecaster can get δ-calibrated forecasts MINIMAX THEOREM the forecaster can GUARANTEE δ-calibrated forecasts (without knowing the rainmaker s strategy)

28 Calibration Proof: Minimax FINITE δ-grid, FINITE HORIZON FINITE 2-person 0-sum game IF the strategy of the rainmaker IS KNOWN THEN the forecaster can get δ-calibrated forecasts MINIMAX THEOREM the forecaster can GUARANTEE δ-calibrated forecasts (without knowing the rainmaker s strategy) Hart 1995

29 Calibration

30 Calibration CALIBRATION can be guaranteed (no matter what the weather is) Forecaster uses mixed forecasting (e.g.: with probability 1/2, forecast = 25% with probability 1/2, forecast = 60%) Foster and Vohra 1994 [publ 1998] Hart 1995: proof using Minimax Theorem

31 Calibration CALIBRATION can be guaranteed (no matter what the weather is) Forecaster uses mixed forecasting (e.g.: with probability 1/2, forecast = 25% with probability 1/2, forecast = 60%) Foster and Vohra 1994 [publ 1998] Hart 1995: proof using Minimax Theorem Hart and Mas-Colell 1996 [publ 2000]: proof using Blackwell s Approachability

32 Calibration CALIBRATION can be guaranteed (no matter what the weather is) Forecaster uses mixed forecasting (e.g.: with probability 1/2, forecast = 25% with probability 1/2, forecast = 60%) Foster and Vohra 1994 [publ 1998] Hart 1995: proof using Minimax Theorem Hart and Mas-Colell 1996 [publ 2000]: proof using Blackwell s Approachability Foster 1999: a simple way to calibrate

33 Calibration CALIBRATION can be guaranteed (no matter what the weather is) Forecaster uses mixed forecasting (e.g.: with probability 1/2, forecast = 25% with probability 1/2, forecast = 60%) BACK-casting (not fore-casting!) ("Politicians Lemma") Foster and Vohra 1994 [publ 1998] Hart 1995: proof using Minimax Theorem Hart and Mas-Colell 1996 [publ 2000]: proof using Blackwell s Approachability Foster 1999: a simple way to calibrate

34 No Calibration

35 No Calibration CALIBRATION cannot be guaranteed when:

36 No Calibration CALIBRATION cannot be guaranteed when: Forecast is known before the rain/no-rain decision is made ("LEAKY FORECASTS")

37 No Calibration CALIBRATION cannot be guaranteed when: Forecast is known before the rain/no-rain decision is made ("LEAKY FORECASTS") Forecaster uses a deterministic forecasting procedure

38 No Calibration CALIBRATION cannot be guaranteed when: Forecast is known before the rain/no-rain decision is made ("LEAKY FORECASTS") Forecaster uses a deterministic forecasting procedure Oakes 1985

39 Smooth Calibration

40 Smooth Calibration SMOOTH CALIBRATION: combine together the days when the forecast was close to p

41 Smooth Calibration SMOOTH CALIBRATION: combine together the days when the forecast was close to p (smooth out the calibration score)

42 Smooth Calibration SMOOTH CALIBRATION: combine together the days when the forecast was close to p (smooth out the calibration score) Main Result: There exists a deterministic procedure that is SMOOTHLY CALIBRATED.

43 Smooth Calibration SMOOTH CALIBRATION: combine together the days when the forecast was close to p (smooth out the calibration score) Main Result: There exists a deterministic procedure that is SMOOTHLY CALIBRATED. Deterministic result holds also when the forecasts are leaked

44 Calibration

45 Calibration Set of ACTIONS: A R m (finite set) Set of FORECASTS: C = (A) Example: A = {0, 1}, C = [0, 1]

46 Calibration Set of ACTIONS: A R m (finite set) Set of FORECASTS: C = (A) Example: A = {0, 1}, C = [0, 1] CALIBRATION SCORE at time T for a sequence (a t, c t ) t=1,2,... in A C:

47 Calibration Set of ACTIONS: A R m (finite set) Set of FORECASTS: C = (A) Example: A = {0, 1}, C = [0, 1] CALIBRATION SCORE at time T for a sequence (a t, c t ) t=1,2,... in A C: K T = 1 T ā t c t T where ā t = t=1 T s=1 1 c s =c t a s T s=1 1 c s =c t

48 Smooth Calibration

49 Smooth Calibration A "smoothing function" is a Lipschitz function Λ : C C [0, 1] with Λ(c, c) = 1 for every c.

50 Smooth Calibration A "smoothing function" is a Lipschitz function Λ : C C [0, 1] with Λ(c, c) = 1 for every c. Λ(x, c) = "weight" of x relative to c

51 Smooth Calibration A "smoothing function" is a Lipschitz function Λ : C C [0, 1] with Λ(c, c) = 1 for every c. Λ(x, c) = "weight" of x relative to c Example: Λ(x, c) = [δ x c ] + /δ

52 Indicator Function

53 Indicator Function 1 x=c 1 c x

54 Indicator and Λ Functions Λ(x, c) 1 x=c 1 c δ c c + δ x

55 Smooth Calibration A "smoothing function" is a Lipschitz function Λ : C C [0, 1] with Λ(c, c) = 1 for every c. Λ(x, c) = "weight" of x relative to c

56 Smooth Calibration A "smoothing function" is a Lipschitz function Λ : C C [0, 1] with Λ(c, c) = 1 for every c. Λ(x, c) = "weight" of x relative to c Λ-CALIBRATION SCORE at time T :

57 Smooth Calibration A "smoothing function" is a Lipschitz function Λ : C C [0, 1] with Λ(c, c) = 1 for every c. Λ(x, c) = "weight" of x relative to c Λ-CALIBRATION SCORE at time T : K Λ T = 1 T T ā Λ t cλ t t=1

58 Smooth Calibration A "smoothing function" is a Lipschitz function Λ : C C [0, 1] with Λ(c, c) = 1 for every c. Λ(x, c) = "weight" of x relative to c Λ-CALIBRATION SCORE at time T : K Λ T = 1 T T t=1 ā Λ t cλ t ā Λ t = T s=1 Λ(c s, c t ) a s T s=1 Λ(c s, c t ), c Λ t = T s=1 Λ(c s, c t ) c s T s=1 Λ(c s, c t )

59 The Calibration Game

60 The Calibration Game In each period t = 1, 2,... :

61 The Calibration Game In each period t = 1, 2,... : Player C ("forecaster") chooses c t C Player A ("action") chooses a t A

62 The Calibration Game In each period t = 1, 2,... : Player C ("forecaster") chooses c t C Player A ("action") chooses a t A a t and c t chosen simultaneously: REGULAR setup

63 The Calibration Game In each period t = 1, 2,... : Player C ("forecaster") chooses c t C Player A ("action") chooses a t A a t and c t chosen simultaneously: REGULAR setup a t chosen after c t is disclosed: LEAKY setup

64 The Calibration Game In each period t = 1, 2,... : Player C ("forecaster") chooses c t C Player A ("action") chooses a t A a t and c t chosen simultaneously: REGULAR setup a t chosen after c t is disclosed: LEAKY setup Full monitoring, perfect recall

65 Smooth Calibration

66 Smooth Calibration A strategy of Player C is

67 Smooth Calibration A strategy of Player C is (ε, L)-SMOOTHLY CALIBRATED

68 Smooth Calibration A strategy of Player C is (ε, L)-SMOOTHLY CALIBRATED if there is T 0 such that K Λ T ε holds for: every T T 0, every strategy of Player A, and every smoothing function Λ with Lipschitz constant L

69 Smooth Calibration: Result

70 Smooth Calibration: Result For every ε > 0 and L < there exists a procedure that is (ε, L)-SMOOTHLY CALIBRATED.

71 Smooth Calibration: Result For every ε > 0 and L < there exists a procedure that is (ε, L)-SMOOTHLY CALIBRATED. Moreover: it is deterministic,

72 Smooth Calibration: Result For every ε > 0 and L < there exists a procedure that is (ε, L)-SMOOTHLY CALIBRATED. Moreover: it is deterministic, it has finite recall (= finite window, stationary),

73 Smooth Calibration: Result For every ε > 0 and L < there exists a procedure that is (ε, L)-SMOOTHLY CALIBRATED. Moreover: it is deterministic, it has finite recall (= finite window, stationary), it uses a fixed finite grid, and

74 Smooth Calibration: Result For every ε > 0 and L < there exists a procedure that is (ε, L)-SMOOTHLY CALIBRATED. Moreover: it is deterministic, it has finite recall (= finite window, stationary), it uses a fixed finite grid, and forecasts may be leaked

75 Smooth Calibration: Implications

76 Smooth Calibration: Implications For forecasting:

77 Smooth Calibration: Implications For forecasting: nothing good... (easier to pass the test)

78 Smooth Calibration: Implications For forecasting: nothing good... (easier to pass the test) For game dynamics:

79 Smooth Calibration: Implications For forecasting: nothing good... (easier to pass the test) For game dynamics: Nash dynamics

80 Calibrated Learning

81 Calibrated Learning CALIBRATED LEARNING:

82 Calibrated Learning CALIBRATED LEARNING: Foster and Vohra 1997

83 Calibrated Learning CALIBRATED LEARNING: every player uses a calibrated forecast on the play of the other players Foster and Vohra 1997

84 Calibrated Learning CALIBRATED LEARNING: every player uses a calibrated forecast on the play of the other players every player best replies to his forecast Foster and Vohra 1997

85 Calibrated Learning CALIBRATED LEARNING: every player uses a calibrated forecast on the play of the other players every player best replies to his forecast time average of play (= empirical distribution of play) is an approximate CORRELATED EQUILIBRIUM Foster and Vohra 1997

86 Smooth Calibrated Learning

87 Smooth Calibrated Learning SMOOTH CALIBRATED LEARNING:

88 Smooth Calibrated Learning SMOOTH CALIBRATED LEARNING: (F) A smoothly calibrated deterministic procedure, which gives in each period t a "forecasted" joint distribution of play c t in (A) (Π i N A i )

89 Smooth Calibrated Learning SMOOTH CALIBRATED LEARNING: (F) A smoothly calibrated deterministic procedure, which gives in each period t a "forecasted" joint distribution of play c t in (A) (Π i N A i ) (P) A Lipschitz approximate best-reply mapping g i : (A) (A i ) for each player i

90 Smooth Calibrated Learning SMOOTH CALIBRATED LEARNING: (F) A smoothly calibrated deterministic procedure, which gives in each period t a "forecasted" joint distribution of play c t in (A) (Π i N A i ) (P) A Lipschitz approximate best-reply mapping g i : (A) (A i ) for each player i In each period t, each player i: 1. runs the procedure (F) to get c t 2. plays g i (c t ) given by (P)

91 Smooth Calibrated Learning

92 Smooth Calibrated Learning SMOOTH CALIBRATED LEARNING (with appropriate parameters):

93 Smooth Calibrated Learning SMOOTH CALIBRATED LEARNING (with appropriate parameters): is a stochastic uncoupled dynamic

94 Smooth Calibrated Learning SMOOTH CALIBRATED LEARNING (with appropriate parameters): is a stochastic uncoupled dynamic has finite MEMORY and is stationary

95 Smooth Calibrated Learning SMOOTH CALIBRATED LEARNING (with appropriate parameters): is a stochastic uncoupled dynamic has finite MEMORY and is stationary Nash ε-equilibria are played at least 1 ε of the time in the long run (a.s.)

96 Smooth Calibrated Learning: Proof

97 Smooth Calibrated Learning: Proof play t = g(c t )

98 Smooth Calibrated Learning: Proof smooth calibration play t = g(c t ) c t

99 Smooth Calibrated Learning: Proof smooth calibration play t = g(c t ) c t use: g is Lipschitz

100 Smooth Calibrated Learning: Proof smooth calibration play t = g(c t ) c t use: g is Lipschitz g approximate best reply play t is an approximate Nash equilibrium

101 Smooth Calibrated Learning: Proof smooth calibration play t = g(c t ) c t use: g is Lipschitz g approximate best reply play t is an approximate Nash equilibrium g(play t ) = g(g(c t )) g(c t ) = play t

102 Why Smooth?

103 Why Smooth? SMOOTH CALIBRATION

104 Why Smooth? SMOOTH CALIBRATION deterministic

105 Why Smooth? SMOOTH CALIBRATION deterministic same forecast for all players

106 Why Smooth? SMOOTH CALIBRATION deterministic same forecast for all players leaky

107 Why Smooth? SMOOTH CALIBRATION deterministic same forecast for all players leaky actions depend on forecast

108 Why Smooth? SMOOTH CALIBRATION deterministic same forecast for all players leaky actions depend on forecast calibrated

109 Why Smooth? SMOOTH CALIBRATION deterministic same forecast for all players leaky actions depend on forecast calibrated forecast equals actions

110 Why Smooth? SMOOTH CALIBRATION deterministic same forecast for all players leaky actions depend on forecast calibrated forecast equals actions FIXED POINT

111 Why Smooth? SMOOTH CALIBRATION deterministic same forecast for all players leaky actions depend on forecast calibrated forecast equals actions FIXED POINT SMOOTH BEST REPLY

112 Why Smooth? SMOOTH CALIBRATION deterministic same forecast for all players leaky actions depend on forecast calibrated forecast equals actions FIXED POINT SMOOTH BEST REPLY fixed point = NASH EQUILIBRIUM

113 Dynamics and Equilibrium

114 Dynamics and Equilibrium "LAW OF CONSERVATION OF COORDINATION":

115 Dynamics and Equilibrium "LAW OF CONSERVATION OF COORDINATION": There must be some COORDINATION

116 Dynamics and Equilibrium "LAW OF CONSERVATION OF COORDINATION": There must be some COORDINATION either in the EQUILIBRIUM notion,

117 Dynamics and Equilibrium "LAW OF CONSERVATION OF COORDINATION": There must be some COORDINATION either in the EQUILIBRIUM notion, or in the DYNAMIC

118 Dynamics and Equilibrium "LAW OF CONSERVATION OF COORDINATION": There must be some COORDINATION either in the EQUILIBRIUM notion, (CORRELATED EQUILIBRIUM) or in the DYNAMIC

119 Dynamics and Equilibrium "LAW OF CONSERVATION OF COORDINATION": There must be some COORDINATION either in the EQUILIBRIUM notion, (CORRELATED EQUILIBRIUM) or in the DYNAMIC (NASH EQUILIBRIUM)

120 Dynamics and Equilibrium "LAW OF CONSERVATION OF COORDINATION": There must be some COORDINATION either in the EQUILIBRIUM notion, (CORRELATED EQUILIBRIUM) or in the DYNAMIC (NASH EQUILIBRIUM) (Hart and Mas-Colell 2003)

121 Integral Approach to Calibration

122 Integral Approach to Calibration INTEGRAL CALIBRATION score:

123 Integral Approach to Calibration INTEGRAL CALIBRATION score: G Λ t (z) = 1 t t Λ(c s, z)(a s c s ) s=1

124 Integral Approach to Calibration INTEGRAL CALIBRATION score: G Λ t (z) = 1 t ( G Λ t 2 = t s=1 C Λ(c s, z)(a s c s ) G Λ t (z) 2 dζ(z) ) 1/2

125 Integral Approach to Calibration INTEGRAL CALIBRATION score: G Λ t (z) = 1 t ( G Λ t 2 = t s=1 C Λ(c s, z)(a s c s ) G Λ t (z) 2 dζ(z) ) 1/2 INTEGRAL CALIBRATION: Guarantee that G Λ t 2 ε for all t large enough, uniformly

126 Integral Calibration

127 Integral Calibration Choose the forecast c t such that Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A

128 Integral Calibration Choose the forecast c t such that Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A Existence of such c t is guaranteed by a Fixed Point Theorem

129 Integral Calibration Choose the forecast c t such that Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A Existence of such c t is guaranteed by a Fixed Point Theorem Deterministic Integral Calibration

130 Integral Calibration Choose the forecast c t such that Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A Existence of such c t is guaranteed by a Fixed Point Theorem Deterministic Integral Calibration Deterministic Smooth Calibration

131 Integral Calibration Choose the forecast c t such that Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A Existence of such c t is guaranteed by a Fixed Point Theorem Deterministic Integral Calibration Deterministic Smooth Calibration Deterministic Weak Calibration

132 Integral Calibration Choose the forecast c t such that Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A Existence of such c t is guaranteed by a Fixed Point Theorem Deterministic Integral Calibration Deterministic Smooth Calibration Deterministic Weak Calibration Almost Deterministic Calibration

133 Deterministic Nonlinear Calibration Choose the forecast c t such that Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A Existence of such c t is guaranteed by a Fixed Point Theorem Deterministic Integral Calibration Deterministic Smooth Calibration Deterministic Weak Calibration Almost Deterministic Calibration

134 Integral Calibration

135 Integral Calibration Choose the distribution of the forecast c t s.t. [ ] E Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A

136 Integral Calibration Choose the distribution of the forecast c t s.t. [ ] E Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A Existence of such distribution is guaranteed by a Separation / Minimax Theorem

137 Integral Calibration Choose the distribution of the forecast c t s.t. [ ] E Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A Existence of such distribution is guaranteed by a Separation / Minimax Theorem Probabilistic Calibration

138 Stochastic Linear Calibration Choose the distribution of the forecast c t s.t. [ ] E Λ(c t, z) G Λ t 1 (z) dζ(z) (a c t) 0 for every a A Existence of such distribution is guaranteed by a Separation / Minimax Theorem Probabilistic Calibration

139 Integral Approach to Calibration

140 Integral Approach to Calibration fixed point deterministic calibration

141 Integral Approach to Calibration fixed point deterministic calibration separation / minimax stochastic calibration

142 Previous Work

143 Previous Work Calibration and Correlated Equilibria:

144 Previous Work Calibration and Correlated Equilibria: Foster and Vohra 1998 Foster and Vohra 1997 Hart and Mas-Colell 2000,...

145 Previous Work Calibration and Correlated Equilibria: Foster and Vohra 1998 Foster and Vohra 1997 Hart and Mas-Colell 2000,... Weak Calibration (deterministic):

146 Previous Work Calibration and Correlated Equilibria: Foster and Vohra 1998 Foster and Vohra 1997 Hart and Mas-Colell 2000,... Weak Calibration (deterministic): Kakade and Foster 2004 / 2008 Foster and Kakade 2006

147 Previous Work

148 Previous Work Nash Dynamics:

149 Previous Work Nash Dynamics: Foster and Young 2003 Kakade and Foster 2004 / 2008 Foster and Young 2006 Hart and Mas-Colell 2006 Germano and Lugosi 2007 Young 2009 Babichenko 2012

150 Previous Work

151 Previous Work Online Regression Problem:

152 Previous Work Online Regression Problem: Foster 1991 J. Foster 1999 Vovk 2001 Azoury and Warmuth 2001 Cesa-Bianchi and Lugosi 2006

153 Successful Economic Forecasting

154 Successful Economic Forecasting correctly forecasting

155 Successful Economic Forecasting... correctly forecasting 8 of the last 5 recessions!