Non-Zero-Sum Stochastic Differential Games of Controls and St

Transcription

1 Non-Zero-Sum Stochastic Differential Games of Controls and Stoppings October 1, 2009

2 Based on two preprints: to a Non-Zero-Sum Stochastic Differential Game of Controls and Stoppings I. Karatzas, Q. Li, 2009 A to Non-Zero-Sum Stochastic Differential Games of Controls and Stoppings I. Karatzas, Q. Li, S. Peng, 2009

3 Bibliography

4 Non-Zero-Sum Game and Nash Equilibrium John F. Nash (1949): One may define a concept of AN n-person GAME in which each player has a finite set of pure strategies and in which a definite set of payments to the n players corresponds to each n-tuple of pure strategies, one strategy being taken by each player. One such n-tuple counters another if the strategy of each player in the countering n-tuple yields the highest obtainable expectation for its player against the n 1 strategies of the other players in the countered n-tuple. A self-countering n-tuple is called AN EQUILIBRIUM POINT.

5 Non-Zero-Sum Game and Nash Equilibrium Aside: In a non-zero-sum game, each player chooses a strategy as his best response to other players strategies. In a Nash equilibrium, no player will profit from unilaterally changing his strategy.

6 Non-Zero-Sum Game and Nash Equilibrium Generalization of zero-sum games: 0-sum 0-sum non-0-sum Player I Player II optimal (s 1, s 2 ) max R(s 1, s 2 ) min R(s 1, s 2 ) saddle s 1 s 2 R(s 1, s 2 ) R(s 1, s 2 ), R(s 1, s 2 ) R(s 1, s 2) max s 1 R(s 1, s 2 ) max s 2 R(s 1, s 2 ) R(s 1, s 2 ) R(s 1, s 2 ), R(s 1, s 2 ) R(s 1, s 2) max s 1 R 1 (s 1, s 2 ) max s 2 R 2 (s 1, s 2 ) equilibrium R 1 (s 1, s 2 ) R1 (s 1, s 2 ), R 2 (s 1, s 2 ) R2 (s 1, s 2)

7 Non-Zero-Sum Game and Nash Equilibrium Simple and understandable example, if there has to be: go watching A Beautiful Mind, Universal Pictures, 2001 (11th Mar. 2009, Columbia University) Kuhn : Don t learn game theory from the movie. The blonde thing is not a Nash equilibrium! Odifreddi : How you invented the theory, I mean, the story about the blonde, was it real? Nash : No!!! Odifreddi : Did you apply game theory to win Alicia? Nash :...Yes... (followed by 10 min s discussion on personal life and game theory)

8 Stochastic Differential Games Martingale Method: Rewards can be functionals of state process. Beneš, 1970, 1971 M H A Davis, 1979 Karatzas and Zamfirescu, 2006, 2008

9 Stochastic Differential Games BSDE Method: Identify value of a game to solution to a BSDE, then seek uniqueness and especially existence of solution. Bismut, 1970 s Pardoux and Peng, 1990 El Karoui, Kapoudjian, Pardoux, Peng, and Quenez, 1997 Cvitanić and Karatzas, 1996 Hamadène, Lepeltier, and Peng, 1997

10 Stochastic Differential Games PDE Method: Rewards are functions of state process. Regularity theory by Bensoussan, Frehse, and Friedman. Facilitates numerical computation. Bensoussan and Friedman, 1977 Bensoussan and Frehse, 2000 H.J. Kushner and P. Dupuis

11 Our Results Main results: (non-) existence of equilibrium stopping rules necessity and sufficiency of Isaacs condition Martingale part: equilibrium stopping rules, L U, L> U equivalent martingale characterization of Nash equilibrium BSDE part: multi-dim reflective BSDE equilibrium stopping rules, L U

12

13 B is a d-dimensional Brownian motion w.r.t. its generated filtration{f t } t 0 on the probability space (Ω, F,P). Change of measure dp u,v dp F t = exp{ t σ 1 (s, X)f(s, X, u s, v s )db s t 0 σ 1 (s, X)f(s, X, u s, v s ) 2 ds}, (1) standardp u,v -Brownian motion B u,v t := B t t 0 σ 1 (s, X)f(s, X, u s, v s )ds, 0 t T. (2)

14 State process X t =X 0 + =X 0 + Hamiltonian t 0 t 0 σ(s, X)dB s, f(s, X, u s, v s )ds + t 0 σ(s, X)dB u,v s, 0 t T. (3) H 1 (t, x, z 1, u, v) := z 1 σ 1 (t, x)f(t, x, u, v) + h 1 (t, x, u, v); H 2 (t, x, z 2, u, v) := z 2 σ 1 (t, x)f(t, x, u, v) + h 2 (t, x, u, v). (4)

15 Admissible controls u U and v V. u, v : [0, T] R R R Rrandom fields. τ,ρ S t = set of stopping rules defined on the pathsω, which generate{f t } t 0 -stopping times on Ω. Strategy: Player I - (u,τ(u, v)); Player II - (v,ρ(u, v)). Reward processes R 1 (τ,ρ, u, v) and R 2 (τ,ρ, u, v). Players expected reward processes J i t (τ,ρ, u, v) =Eu,v [R i t (τ,ρ, u, v) F t], i = 1, 2. (5)

16 Nash equilibrium strategies (u, v,τ,ρ ) Find admissible control strategies u U and v V, and stopping rulesτ andρ in S t,t, that maximize expected rewards. V 1 (t) := J 1 t (τ,ρ, u, v ) J 1 t (τ,ρ, u, v ),τ S t,t, u U; V 2 (t) := J 2 t (τ,ρ, u, v ) J 2 t (τ,ρ, u, v), ρ S t,t, v V. (6) no profit from unilaterally changing strategy

17 Analysis, in the spirit of Nash (1949) For u 0 U, v 0 V, andτ 0,ρ 0 S t,t, find (u 1, v 1,τ 1,ρ 1 ) that counters (u 0, v 0,τ 0,ρ 0 ), i.e. (u 1,τ 1 ) = arg max τ S t,t max u U J1 t (τ,ρ0, u, v 0 ); (v 1,ρ 1 ) = arg max ρ S t,t max v V J2 t (τ0,ρ, u 0, v). (7) The equilibrium (τ,ρ, u, v ) is fixed point of the mapping Γ : (τ 0,ρ 0, u 0, v 0 ) (τ 1,ρ 1, u 1, v 1 ) (8)

18 More about Control Sets (t) = instantaneous volatility process of player i s reward process Jt i (τ,ρ, u, v), i.e. Z u,v i djt i u,v (τ,ρ, u, v) = d finite variation part + Z (t)db u,v (t). (9) i Partial observation u t = u(t), and v t = v(t). Full observation u t = u(t, x), and v t = v(t, x). Observing volatility u t = u(t, x, Z u,v u,v (t), Z 1 2 (t)), and v t = v(t, x, Z u,v u,v (t), Z 1 2 (t)). (10)

19 More about Control Sets - Why caring about Z u,v? - Risk sensitive control. Sensitive to not only expectation but also variance of the reward. Bensoussan, Frehse, and Nagai, 1998 El Karoui and Hamadène, 2003

20

21 Rewards and Assumptions τ ρ R 1 t (τ,ρ, u, v) := h 1 (s, X, u s, v s )ds + L 1 (τ)½ {τ<ρ} + U 1 (ρ)½ {ρ τ<t} t +ξ 1 ½ {τ ρ=t} ; τ ρ R 2 t (τ,ρ, u, v) := h 2 (s, X, u s, v s )ds + L 2 (ρ)½ {ρ<τ} + U 2 (τ)½ {τ ρ<t} t +ξ 2 ½ {τ ρ=t}. boundedness: h, L, U,ξ measurabilities: h, L, U,ξ continuity: L, U (11)

22 Equivalent Martingale Characterization Notations. Y 1 (t;ρ, v) := sup Y 2 (t;τ, u) := sup sup τ S t,ρ u U sup ρ S t,τ v V J 1 t (τ,ρ, u, v); J 2 t (τ,ρ, u, v). (12) V 1 (t;ρ, u, v) :=Y 1 (t;ρ, v) + V 2 (t;τ, u, v) :=Y 2 (t;τ, u) + t 0 t 0 h 1 (s, X, u s, v s )ds; h 2 (s, X, u s, v s )ds. (13)

23 Equivalent Martingale Characterization Thm. (τ,ρ, u, v ) is an equilibrium point, if and only if the following three conditions hold. (1) Y 1 (τ ;ρ, v ) = L 1 (τ )½ {τ <ρ } + U 1 (ρ )½ {ρ τ <T} +ξ 1 ½ {τ ρ =T}, and Y 2 (ρ ;τ, u ) = L 2 (ρ )½ {ρ <τ } + U 2 (τ )½ {τ ρ <T} +ξ 2 ½ {τ ρ =T}; (2) V 1 ( τ ;ρ, u, v ) and V 2 ( ρ ;τ, u, v ) are P u,v -martingales; (3) For every u U,V 1 ( τ ;ρ, u, v ) is ap u,v -supermartingale. For every v V, V 2 ( ρ ;τ, u, v) is ap u,v -supermartingale.

24 Equilibrium Stopping Rules Def. Generic controls (u, v) U V. The equilibrium stopping rules are a pair (τ,ρ ) S 2, such that t,t J 1 t (τ,ρ, u, v) J 1 t (τ,ρ, u, v), τ S t,t ; J 2 t (τ,ρ, u, v) J 2 t (τ,ρ, u, v), ρ S t,t, (14)

25 Equilibrium Stopping Rules Notations. Y 1 (t, u;ρ, v) := sup J 1 t (τ,ρ, u, v); τ S t,t Y 2 (t, v;τ, u) := sup ρ S t,t J 2 t (τ,ρ, u, v). (15) Q 1 (t, u;ρ, v) :=Y 1 (t, u;ρ, v) + Q 2 (t, v;τ, u) :=Y 2 (t, v;τ, u) + t 0 t 0 h 1 (s, X, u s, v s )ds h 2 (s, X, u s, v s )ds (16)

26 Equilibrium Stopping Rules Lem. (τ,ρ ) is a a pair of equilibrium stopping rules, iff (1) Y 1 (τ, u;ρ, v) = L 1 (τ )½ {τ <ρ } + U 1 (ρ )½ {ρ τ <T} +ξ 1 ½ {τ ρ =T}, Y 2 (ρ, v;τ, u) = L 2 (ρ )½ {ρ <τ } + U 2 (τ )½ {τ ρ <T} +ξ 2 ½ {τ ρ =T}; (17) (2) The stopped supermartingales Q 1 ( τ, u;ρ, v) and Q 2 ( ρ, v;τ, u) arep u,v -martingales.

27 Equilibrium Stopping Rules L(t) U(t)½ {t<t} +ξ½ {t=t}, for all 0 t T. If (τ,ρ ) solve the equations τ = inf{t s<ρ Y 1 (s, u;ρ, v) = L 1 (s)} ρ ; ρ = inf{t s<ρ Y 2 (s, v;τ, u) = L 2 (s)} τ, (18) on first hitting times, then (τ,ρ ) are equilibrium.

28 Equilibrium Stopping Rules L(t) U(t)½ {t<t} +ξ½ {t=t} +ǫ, for all 0 t T, someǫ> 0. If L is uniformly continuous inω Ω, then equilibrium stopping rules do not exist.

29 Martingale Structures Suppose (τ,ρ, u, v ) is an equilibrium point. Doob-Meyer V 1 (t;ρ, u, v) =Y 1 (0;ρ, v) A 1 (t;ρ, u, v) + M 1 (t;ρ, u, v), 0 t τ ; V 2 (t;τ, u, v) =Y 2 (0;τ, v) A 2 (t;τ, u, v) + M 2 (t;τ, u, v), 0 t ρ. (19) Martingale representation M 1 (t;ρ, u, v) = M 2 (t;τ, u, v) = t 0 t 0 Z v 1 (s)dbu,v s ; Z u 2 (s)dbu,v s, (20)

30 Martingale Structures Finite variation part A 1 (t;τ, u 1, v) A 1 (t;τ, u 2, v) = t 0 0 t τ ; (H 1 (s, X, Z 1 (s), u 1 s, v s) H 1 (s, X, Z 1 (s), u 2 s, v s))ds, A 2 (t;ρ, u, v 1 ) A 2 (t;ρ, u, v 2 ) = t 0 (H 2 (s, X, Z 2 (s), u s, v 1 s ) H 2(s, X, Z 2 (s), u s, v 2 s ))ds, 0 t ρ. (21)

31 Isaacs Condition Necessity, stochastic maximum principle Prop. If (τ,ρ, u, v ) is an equilibrium point, then H 1 (t, X, Z 1 (t), u t, v t ) H 1(t, X, Z 1 (t), u t, v t ), for all 0 t τ, u U; H 2 (t, X, Z 2 (t), u t, v t ) H 2(t, X, Z 2 (t), u t, v t), for all 0 t ρ, v V. (22)

32 Isaacs Condition Sufficiency Thm. Letτ,ρ S t,t be equilibrium stopping rules. If a pair of controls (u, v ) U V satisfies Isaacs condition H 1 (t, x, z 1, u t, v t ) H 1(t, x, z 1, u t, v t ); H 2 (t, x, z 2, u t, v t ) H 2(t, x, z 2, u t, v t), (23) for all 0 t T, u U,and v V, then u, v are equilibrium controls.

33

34 Each Player s Reward Terminated by Himself Game 2.1 τ R 1 t (τ,ρ, u, v) := h 1 (s, X, u s, v s )ds + L 1 (τ)½ {τ<t} +η 1 ½ {τ=t} ; t ρ R 2 t (τ,ρ, u, v) := h 2 (s, X, u s, v s )ds + L 2 (ρ)½ {ρ<t} +η 2 ½ {ρ=t}. L 1 η 1, L 2 η 2, a.s. t Assumption A 2.1 (Isaac s condition) There exist admissible controls (u, v ) U V, such that t [0, T], u U, v V, (24) H 1 (t, x, z 1, (u, v )(t, x, z 1, z 2 )) H 1 (t, x, z 1, u(t, x,, ), v (t, x, z 1, z 2 )); H 2 (t, x, z 2, (u, v )(t, x, z 1, z 2 )) H 2 (t, x, z 2, u (t, x, z 1, z 2 ), v(t, x,, )). (25)

35 Each Player s Reward Terminated by Himself Thm 2.1 Let (Y, Z, K) be solution to reflective BSDE T T Y(t) =η + H(s, X, Z(s), u, v )ds Z(s)dB s + K(T) K(t); Y(t) L(t), 0 t T; Optimal stopping rules t T 0 (Y(t) L(t))dK i (t) = 0. t (26) τ := inf{s [t, T] : Y 1 (s) L 1 (s)} T; ρ := inf{s [t, T] : Y 2 (s) L 2 (s)} T. (27) (τ,ρ, u, v ) is optimal for Game 2.1. Further more, V i (t) = Y i (t), i = 1, 2.

36 Game with Interactive Stoppings Game 2.2 τ ρ R 1 t (τ,ρ, u, v) := h 1 (s, X, u s, v s )ds + L 1 (τ)½ {τ<ρ} + U 1 (ρ)½ {ρ τ<t} t +ξ 1 ½ {τ ρ=t} ; τ ρ R 2 t (τ,ρ, u, v) := h 2 (s, X, u s, v s )ds + L 2 (ρ)½ {ρ<τ} + U 2 (τ)½ {τ ρ<t} t +ξ 2 ½ {τ ρ=t}. Assumption A 2.2 (Isaac s condition) There exist admissible controls (u, v ) U V, such that (28) H 1 (t, x, z 1, (u, v )(t, x)) H 1 (t, x, z 1, u(t, x), v (t, x)), t [0, T], u U H 2 (t, x, z 2, (u, v )(t, x)) H 2 (t, x, z 2, u (t, x), v(t, x)), t [0, T], v V (29)

37 Game with Interactive Stoppings Assumption A 2.3 (1) For i = 1, 2, the reward processes U i ( ) redefined as U i (t), 0 t< T; U i (t) = ξ i, t = T, (30) are increasing processes. L(t) i U i (t) ξ, a.s. ( patience pays ) (2) Both reward processes{u(t)} 0 t T and{l(t)} 0 t T are right continuous in time t. [0, T] Ω. (3) h i c, i = 1, 2.

38 Game with Interactive Stoppings Thm 2.2 Under assumptions A 2.2 and A 2.3, then there exists an equilibrium point (τ,ρ, u, v ) of Game 2.2.

39 Game with Interactive Stoppings Thm 2.3 (Associated BSDE) T Y i (t) =ξ i + H i (s, X, Z i (s), (u, v)(t, X, Z 1 (s), Z 2 (s))ds t T Z i (s)db s + K i (T) K i (t) + N i (t, T), t T Y i (t) L i (t), 0 t T; (Y i (t) L i (t))dk i (t) = 0; i = 1, 2, 0 (31) where N i (t, T) := (U i (s) Y i (s))½ {Yj (s)=l j (s)}, i, j = 1, 2. (32) t s T (being kicked up to U, when the other player drops down to L)

40 Game with Interactive Stoppings Optimal stopping times τ :=τ u,v t (u, v) := inf{s [t, T] : Y1 (s) L 1(s)} T; ρ :=ρ u,v t (u, v) := inf{s [t, T] : Y2 (s) L 2(s)} T. (33)

41

42 Lipschitz Growth m-dim reflective BSDE T Y 1 (t) =ξ 1 + g 1 (s, Y(s), Z(s))ds Y 1 (t) L 1 (t), 0 t T, (Y 1 (t) L 1 (t)) dk 1 (t) = 0, 0 T T Y m (t) =ξ m + g m (s, Y(s), Z(s))ds Z m (s) db s + K m (T) K m Y m (t) L m (t), 0 t T, t t T T 0 T t t Z 1 (s) db s + K 1 (T) K 1 (t); (Y m (t) L m (t)) dk m (t) = 0. (34)

43 Lipschitz Growth Seek solution (Y, Z, K) in the spaces Y = (Y 1,, Y m ) M 2 (m; 0, T) :={m-dimensional predictable process φ s.t. E[supφ 2 t ] }; [0,T] Z = (Z 1,, Z m ) L 2 (m d; 0, T) :={m d-dimensional predictable process φ s.t. E[ T 0 φ 2 t dt] }; K = (K 1,, K m ) = continuous, increasing process inm 2 (m; 0, T). (35)

44 Lipschitz Growth Assumption A 3.1 (1) The random field g = (g 1,, g m ) : [0, T] R m R m d R m (36) is uniformly Lipschitz in y and z, i.e. there exists a constant b> 0, such that g(t, y, z) g(t, ȳ, z) b( y ȳ + z z ), t [0, T]. (37) Further more, E[ T 0 g(t, 0, 0) 2 dt]<. (38) (2) The random variableξis F T -measurable and square-integrable. The lower reflective boundary L is progressively measurable, and satisfy E[sup L + (t) 2 ]<. L ξ,p-a.s. [0,T]

45 Lipschitz Growth Results: existence and uniqueness of solution, via contraction method 1-dim Comparison Theorem (EKPPQ, 1997) continuous dependency property

46 Linear Growth, Markovian System l(elle)-dim forward equation X t,x (s) = x, 0 s t; dx t,x (s) =σ(s, X t,x (s)) db s, t< s T. (39) m-dim backward equation T Y t,x (s) =ξ(x t,x (T)) + g i (r, X t,x (r), Y t,x (r), Z t,x (r))dr s T Z t,x (r) db r + K t,x (T) K t,x (s); s Y t,x (s) L(s, X t,x (s)), t s T, T t (Y t,x (s) L(s, X t,x (s))) dk t,x (s) (40)

47 Linear Growth, Markovian System Assumption A 4.1 (1) g : [0, T] R l R m R m d R m is measurable, and for all (t, x, y, z) [0, T] R l R m R m d, g(t, x, y, z) b(1 + x p + y + z ), for some positive constant b; (2) for every fixed (t, x) [0, T] R, g(t, x,, ) is continuous. (3)E[ξ(X(T)) 2 ]< ;E[sup L + (t, X(t)) 2 ]<. L ξ,p-a.s. [0,T]

48 Linear Growth, Markovian System Results existence of solution, via Lipschitz approximation 1-dim Comparison Theorem continuous dependency property

49 THAT S ALL THANK YOU