LQG Mean-Field Games with ergodic cost in R d

Transcription

1 LQG Mean-Field Games with ergodic cost in R d Fabio S. Priuli University of Roma Tor Vergata joint work with: M. Bardi (Univ. Padova) Padova, September 4th, 2013

2 Main problems Problems we are interested in: I. Nash equilibria for N players LQG games with ergodic cost (via HJB + KFP equations)

3 Main problems Problems we are interested in: I. Nash equilibria for N players LQG games with ergodic cost (via HJB + KFP equations) II. Solutions to MFE obtained formally from the above (as in Lasry & Lions)

4 Main problems Problems we are interested in: I. Nash equilibria for N players LQG games with ergodic cost (via HJB + KFP equations) II. Solutions to MFE obtained formally from the above (as in Lasry & Lions) III. Rigorous relation between the two as N +

5 Main problems Problems we are interested in: I. Nash equilibria for N players LQG games with ergodic cost (via HJB + KFP equations) II. Solutions to MFE obtained formally from the above (as in Lasry & Lions) III. Rigorous relation between the two as N + Goal: Generalize results by Bardi (2011) in 1d case

6 Main problems Problems we are interested in: I. Nash equilibria for N players LQG games with ergodic cost (via HJB + KFP equations) II. Solutions to MFE obtained formally from the above (as in Lasry & Lions) III. Rigorous relation between the two as N + Goal: Generalize results by Bardi (2011) in 1d case [see also Lasry & Lions (2006, 2007), Guéant, Lasry & Lions (2011), Huang, Caines & Malhamé (2004, 2007), Bensoussan, Sung, Yam & Yung (2011)]

7 N players games Formulation of the problem N players LQG games in R d We consider games with linear stochastic dynamics w.r.t. state & control quadratic ergodic cost w.r.t. state & control

8 N players games Formulation of the problem N players LQG games in R d We consider games with linear stochastic dynamics w.r.t. state & control quadratic ergodic cost w.r.t. state & control For simplicity we focus on nearly identical players, i.e., we assume all players have the same dynamics all players have the same cost for the control players are indistinguishable (symmetry assumption) but analogous results hold also for general games!

9 N players games Formulation of the problem Consider for i = 1,...,N dx i t = (AXi t αi t )dt+σdwi t X i 0 = xi R d

10 N players games Formulation of the problem Consider for i = 1,...,N dx i t = (AXi t αi t )dt+σdwi t X i 0 = xi R d where A R d d, α i t controls, σ invertible, Wi t Brownian,

11 N players games Formulation of the problem Consider for i = 1,...,N dx i t = (AXi t αi t )dt+σdwi t X i 0 = xi R d J i (X 0,α 1,...,α N ). = liminf T [ 1 T T E 0 (α i t )T Rα i t 2 +(X t X i ) T Q i (X t X i ) dt }{{} F i (X 1,...,X N ) ] where A R d d, α i t controls, σ invertible, Wi t Brownian,

12 N players games Formulation of the problem Consider for i = 1,...,N dx i t = (AXi t αi t )dt+σdwi t X i 0 = xi R d J i (X 0,α 1,...,α N ). = liminf T [ 1 T T E 0 (α i t )T Rα i t 2 +(X t X i ) T Q i (X t X i ) dt }{{} F i (X 1,...,X N ) ] where A R d d, αt i controls, σ invertible, Wi t Brownian, R R d d symm. pos. def., X t = (Xt,...,X 1 t N ) R Nd state var., X i = (Xi 1,...,XN i ) R Nd vector of favorite positions, Q i R Nd Nd block matrix

13 N players games Formulation of the problem X i = (X 1 i,...,xn i ) R Nd s.t.

14 N players games Formulation of the problem X i = (X 1 i,...,xn i ) R Nd s.t. X i i = h i (preferred position) X j i = r j i (reference position)

15 N players games Formulation of the problem X i = (X 1 i,...,xn i ) R Nd s.t. X i i = h i (preferred position) X j i = r j i (reference position) Q i R Nd Nd block matrix (Q i jk ) j,k R d d s.t. satisfies N F i (X 1,...,X N ) = (X j t X j i )T Q i jk (Xk t Xk i ) j,k=1

16 N players games Formulation of the problem X i = (X 1 i,...,xn i ) R Nd s.t. X i i = h i (preferred position) X j i = r j i (reference position) Q i R Nd Nd block matrix (Q i jk ) j,k R d d s.t. F i (X 1,...,X N ) = satisfies N (X j t X j i )T Q i jk (Xk t Xk i ) j,k=1 Q i ii = Q symm. pos. def. i Q i ij = Qi ji = B j i

17 N players games Formulation of the problem X i = (X 1 i,...,xn i ) R Nd s.t. X i i = h i (preferred position) X j i = r j i (reference position) Q i R Nd Nd block matrix (Q i jk ) j,k R d d s.t. F i (X 1,...,X N ) = satisfies N (X j t X j i )T Q i jk (Xk t Xk i ) j,k=1 Q i ii = Q symm. pos. def. i Q i ij = Qi ji = B j i Q i jj = C i j i Q i jk = Qi kj = D i j k i j

18 N players games Formulation of the problem Admissible strategies A control α i t adapted to W i t is an admissible strategy if E[X i t],e[x i t(x i t) T ] C for all t > 0 probability measure m α i s.t. lim T + [ 1 T ] T E g(xt i )dt = g(ξ)dm α i(ξ) 0 R d for any polynomial g, with deg(g) 2, loc. unif. in X i 0.

19 N players games Formulation of the problem Admissible strategies A control α i t adapted to W i t is an admissible strategy if E[X i t],e[x i t(x i t) T ] C for all t > 0 probability measure m α i s.t. lim T + [ 1 T ] T E g(xt i )dt = g(ξ)dm α i(ξ) 0 R d for any polynomial g, with deg(g) 2, loc. unif. in X i 0. Example. Any affine α i (x) = Kx+c with K A > 0 is admissible and the corresponding diffusion process dx i t = ( (A K)X i t c) dt+σdw i t is ergodic with m α i = multivariate Gaussian

20 N players games Formulation of the problem Admissible strategies A control αt i adapted to Wi t is an admissible strategy if E[Xt],E[X i t(x i t) i T ] C for all t > 0 probability measure m α i s.t. lim T + [ 1 T ] T E g(xt)dt i = g(ξ)dm α i(ξ) 0 R d for any polynomial g, with deg(g) 2, loc. unif. in X i 0. Nash equilibria Any set of admissible strategies α 1,...,α N such that J i (X,α 1,...,α N ) = min ω J i (X,α 1,...,α i 1,ω,α i+1,...,α N ) for any i = 1,...,N

21 N players games HJB+KFP For this N players game HJB+KFP are given by where tr(νd 2 v i )+H(x, v i )+λ i = f i (x;m 1,...,m N ) ( tr(νd 2 m i )+div x R d m i > 0 m i H ) p (x, vi ) = 0 m i (x)dx = 1 R d ν = σt σ H(x,p) = p T R p pt Ax f i (x;m 1,...,m N ) =. F i (ξ 1,...,ξ i 1,x,ξ i+1,...ξ N ) dm j (ξ j ) R (N 1)d j i (1)

22 N players games HJB+KFP For this N players game HJB+KFP are given by where tr(νd 2 v i )+H(x, v i )+λ i = f i (x;m 1,...,m N ) ( tr(νd 2 m i )+div x R d m i > 0 m i H ) p (x, vi ) = 0 m i (x)dx = 1 R d ν = σt σ H(x,p) = p T R p pt Ax f i (x;m 1,...,m N ) =. F i (ξ 1,...,ξ i 1,x,ξ i+1,...ξ N ) dm j (ξ j ) R (N 1)d j i (1) 2N equations with unknowns λ i,v i,m i, but always x R d!

23 N players games HJB+KFP For this N players game HJB+KFP are given by where tr(νd 2 v i )+H(x, v i )+λ i = f i (x;m 1,...,m N ) ( tr(νd 2 m i )+div x R d m i > 0 m i H ) p (x, vi ) = 0 m i (x)dx = 1 R d ν = σt σ H(x,p) = p T R p pt Ax f i (x;m 1,...,m N ) =. F i (ξ 1,...,ξ i 1,x,ξ i+1,...ξ N ) dm j (ξ j ) R (N 1)d j i Search for solutions of Quadratic Gaussian (QG) type + identically distr. λ i R v i (x) = x T Λ { 2 x+ρx mi (x) = γexp 1 } 2 (x µ)t Σ(x µ) (1)

24 N players games Main result Theorem 1. For N players LQG game Existence & uniquess λ i,v i,m i sol. to (1) with v i,m i QG Algebraic conditions

25 N players games Main result Theorem 1. For N players LQG game Existence & uniquess λ i,v i,m i sol. to (1) with v i,m i QG Algebraic conditions α i = R 1 v i (x) provides Nash equilibria strategies and λ i = J i (X 0,α 1,...,α N ) for i = 1,...,N

26 N players games Main result Theorem 1. For N players LQG game Existence & uniquess λ i,v i,m i sol. to (1) with v i,m i QG Algebraic conditions α i = R 1 v i (x) provides Nash equilibria strategies and λ i = J i (X 0,α 1,...,α N ) for i = 1,...,N Proof. (i) By plugging into (1) v i (x) = x T Λ 2 x+ρx { mi (x) = γexp 1 } 2 (x µ)t Σ(x µ) algebraic conditions on ρ,µ R d, Λ,Σ R d d (ii) Verification theorem, using Dynkin s formula and ergodicity

27 Indeed, v i (x) = Λx+ρ N players games Algebraic conditions m i (x) = m i (x)σ(x µ)

28 Indeed, v i (x) = Λx+ρ N players games Algebraic conditions m i (x) = m i (x)σ(x µ) KFP

29 Indeed, v i (x) = Λx+ρ N players games Algebraic conditions m i (x) = m i (x)σ(x µ) KFP Λ = R ( νσ+a ) ρ = RνΣµ

30 Indeed, v i (x) = Λx+ρ N players games Algebraic conditions m i (x) = m i (x)σ(x µ) KFP HJB Λ = R ( νσ+a ) ρ = RνΣµ

31 Indeed, v i (x) = Λx+ρ N players games Algebraic conditions m i (x) = m i (x)σ(x µ) KFP Λ = R ( νσ+a ) ρ = RνΣµ HJB ( A T RA 2 Σ νrν 2 Σ AT RA 2 = Q ) +Q+(N 1)B µ = Qh (N 1)Br (µ) T ΣνRνΣ 2 µ tr(νrνσ+νra)+λ i = f i (Σ,µ)

32 Indeed, v i (x) = Λx+ρ N players games Algebraic conditions m i (x) = m i (x)σ(x µ) KFP Λ = R ( νσ+a ) ρ = RνΣµ HJB Σ solves ARE ( A T RA 2 X νrν ( A T 2 X RA 2 ) +Q = 0 ) +Q+(N 1)B µ = Qh (N 1)Br (µ) T ΣνRνΣ 2 µ tr(νrνσ+νra)+λ i = f i (Σ,µ)

33 Indeed, v i (x) = Λx+ρ N players games Algebraic conditions m i (x) = m i (x)σ(x µ) KFP Λ = R ( νσ+a ) ρ = RνΣµ HJB Σ solves ARE X νrν ( A T 2 X RA 2 ) +Q = 0 µ solves linear system By = C for B. = AT RA 2 +Q+(N 1)B (µ) T ΣνRνΣ 2 µ tr(νrνσ+νra)+λ i = f i (Σ,µ)

34 Indeed, v i (x) = Λx+ρ N players games Algebraic conditions m i (x) = m i (x)σ(x µ) KFP Λ = R ( νσ+a ) ρ = RνΣµ HJB Σ solves ARE X νrν ( A T 2 X RA 2 ) +Q = 0 µ solves linear system By = C for B. = AT RA 2 +Q+(N 1)B λ i = explicit function of Σ and µ

35 N players games Algebraic conditions Algebraic conditions Existence Uniqueness

36 N players games Algebraic conditions Algebraic conditions Existence there holds rank B = rank [B,C] [iff the system By = C has solutions] Uniqueness

37 N players games Algebraic conditions Algebraic conditions Existence there holds rank B = rank [B,C] [iff the system By = C has solutions] the unique Σ > 0 that solves ARE also solves Sylvester s eq. XνR RνX = RA A T R [iff R ( νσ+a ) symm. matrix] Uniqueness

38 N players games Algebraic conditions Algebraic conditions Existence there holds rank B = rank [B,C] [iff the system By = C has solutions] the unique Σ > 0 that solves ARE also solves Sylvester s eq. XνR RνX = RA A T R [iff R ( νσ+a ) symm. matrix] Uniqueness B is invertible [iff the system By = C has solutions]

39 Mean field equations Formulation of the problem Mean field equations Nearly identical players implies that the costs F i (X 1,...,X N ) can be written as function of the empirical density of other players [ ] F i (X 1,...,X N ) = V i 1 N δ X j (X i ) N 1 j i

40 Mean field equations Formulation of the problem Mean field equations Nearly identical players implies that the costs F i (X 1,...,X N ) can be written as function of the empirical density of other players [ ] F i (X 1,...,X N ) = V i 1 N δ X j (X i ) N 1 j i where V i N : {prob. meas. on Rd } {quadratic polynomials on R d }

41 Mean field equations Formulation of the problem Mean field equations Nearly identical players implies that the costs F i (X 1,...,X N ) can be written as function of the empirical density of other players [ ] F i (X 1,...,X N ) = V i 1 N δ X j (X i ) N 1 j i where V i N : {prob. meas. on Rd } {quadratic polynomials on R d } V i N [m](x) =. (X h) T Q N (X h) +(N 1) ((X h) T BN BN (ξ r)+(ξ r)t R 2 2 d +(N 1) (ξ r) T (Ci N Di N )(ξ r)dm(ξ) R ( d ) T + (N 1) (ξ r)dm(ξ) Di N R d ) (X h) dm(ξ) ( ) (N 1) (ξ r)dm(ξ) R d

42 Mean field equations Formulation of the problem Assuming that the coefficients scale as follows as N Q N ˆQ > 0 Ci N (N 1) Ĉ B N (N 1) ˆB DN i (N 1) 2 ˆD then for any prob. measure m on R d and all i = 1,...,N V i N[m](X) ˆV[m](X) loc. unif. in X

43 Mean field equations Formulation of the problem Assuming that the coefficients scale as follows as N Q N ˆQ > 0 Ci N (N 1) Ĉ B N (N 1) ˆB DN i (N 1) 2 ˆD then for any prob. measure m on R d and all i = 1,...,N where V i N[m](X) ˆV[m](X) loc. unif. in X ˆV[m](X) =. (X h) T ˆQ(X h) ( + (X h) ˆB ) ˆB T (ξ r)+(ξ r)t (X h) R 2 2 d + (ξ r) T Ĉ(ξ r)dm(ξ) R ( d ) T ) + (ξ r)dm(ξ) ˆD( (ξ r)dm(ξ) R d R d dm(ξ)

44 Mean field equations Formulation of the problem Thus passing formally to the limit as N in HJB+KFP tr(νd 2 u)+h(x,du)+λ = ˆV[m](x) ( ) tr(νd 2 m) div m H (x,du) = 0 p x R d m > 0 m(x)dx = 1 R d (MFE) We look for solutions λ,u,m such that u,m is QG u(x) = x T Λ { 2 x+ρx m(x) = γexp 1 } 2 (x µ)t Σ(x µ)

45 Mean field equations Main result Thus passing formally to the limit as N in HJB+KFP tr(νd 2 u)+h(x,du)+λ = ˆV[m](x) ( ) tr(νd 2 m) div m H (x,du) = 0 p x R d m > 0 m(x)dx = 1 R d (MFE) Theorem 2. Existence & uniquess λ,u,m sol. to MFE with u,m QG Algebraic conditions

46 Mean field equations Main result Thus passing formally to the limit as N in HJB+KFP tr(νd 2 u)+h(x,du)+λ = ˆV[m](x) ( ) tr(νd 2 m) div m H (x,du) = 0 p x R d m > 0 m(x)dx = 1 R d (MFE) Theorem 2. Existence & uniquess λ,u,m sol. to MFE with u,m QG Algebraic conditions ˆV is a monotone operator iff ˆB 0 and if so QG sol. is the unique solution to MFE

47 Limit as N Main result Limit as N Theorem 3. Assume (i) QN ˆQ B N (N 1) ˆB Ci N (N 1) Ĉ DN i (N 1)2 ˆD (ii) HJB+KFP for N players admit QG sol. (v N,m N,λ 1 N,...λN N ) (iii) MFE admits unique QG solution (u,m,λ)

48 Limit as N Main result Theorem 3. Assume (i) QN ˆQ Ci N (N 1) Ĉ Limit as N B N (N 1) ˆB DN i (N 1)2 ˆD (ii) HJB+KFP for N players admit QG sol. (v N,m N,λ 1 N,...λN N ) (iii) MFE admits unique QG solution (u,m,λ) Then v N u in C 2 loc (Rd ) m N m in C k (R d ) for all k λ i N λ for all i

49 Conclusions Conclusions

50 Conclusions Conclusions Characterization of existence & uniqueness for QG sols to N players LQG games, which give Nash equilibrium strategies

51 Conclusions Conclusions Characterization of existence & uniqueness for QG sols to N players LQG games, which give Nash equilibrium strategies Characterization of existence & uniqueness for QG sols to MFE + Characterization of monotonicity ˆV

52 Conclusions Conclusions Characterization of existence & uniqueness for QG sols to N players LQG games, which give Nash equilibrium strategies Characterization of existence & uniqueness for QG sols to MFE + Characterization of monotonicity ˆV Convergence QG sols of HJB+KFP to sols of MFE as N +

53 Conclusions Conclusions Characterization of existence & uniqueness for QG sols to N players LQG games, which give Nash equilibrium strategies Characterization of existence & uniqueness for QG sols to MFE + Characterization of monotonicity ˆV Convergence QG sols of HJB+KFP to sols of MFE as N + Algebraic conditions can be directly verified for some games

54 Conclusions Example N players game with R = ri d, ν = ni d, A symmetric, B 0

55 Conclusions Example N players game with R = ri d, ν = ni d, A symmetric, B 0 Algebraic conditions X νrν 2 X = AT RA +Q = XνR RνX = RA A T R 2 becomes nr(x X) = r(a A T ), true for all matrices X B = Q+r A2 2 + B 2 > 0

56 Conclusions Example N players game with R = ri d, ν = ni d, A symmetric, B 0 Algebraic conditions X νrν 2 X = AT RA +Q = XνR RνX = RA A T R 2 becomes nr(x X) = r(a A T ), true for all matrices X B = Q+r A2 2 + B 2 > 0! QG solution, with Σ,Λ,µ,ρ satisfying Σ 2 = 2 rn 2 (r A2 2 +Q Λ = r ( nσ+a ) ) Bµ = C ρ = rnσµ

57 Conclusions Example N players game with R = ri d, ν = ni d, A symmetric, B 0 Algebraic conditions X νrν 2 X = AT RA +Q = XνR RνX = RA A T R 2 becomes nr(x X) = r(a A T ), true for all matrices X B = Q+r A2 2 + B 2 > 0! QG solution, with Σ,Λ,µ,ρ satisfying Σ = 2 n r Λ = r ( nσ+a ) r A2 2 +Q µ = B 1 C ρ = rnσµ

58 And Finally... Thanks for Your Attention!