Explicit solutions of some Linear-Quadratic Mean Field Games

Transcription

1 Explicit solutions of some Linear-Quadratic Mean Field Games Martino Bardi Department of Pure and Applied Mathematics University of Padua, Italy Men Field Games and Related Topics Rome, May 12-13, 2011 Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

2 Plan L-Q games with N players and ergodic cost Symmetric and almost identical players The limit as N + Other limiting cases Vanishing viscosity Cheap control Discounted cost and small discount Models of population distribution Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

3 References We look for L-Q analogues of the results for stationary periodic problems in J.-M. Lasry, P.-L. Lions: Jeux à champ moyen. I. Le cas stationnaire. C. R. A. S J.-M. Lasry, P.-L. Lions: Jpn. J. Math Related discounted L-Q problems were studied by M. Huang, P.E. Caines, R.P. Malhamé: Proc. IEEE Conf M. Huang, P.E. Caines, R.P. Malhamé: IEEE Trans. Automat. Control 2007 Related models of population distribution are in O. Guéant: J. Math. Pures Appl O. Guéant, J.-M. Lasry, P.-L. Lions: Mean field games and applications, to appear. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

4 L-Q games with N players and ergodic payoff Control systems W i dx i t = (A i X i t α i t)dt + σ i dtw i t, X i 0 = x i R, i = 1,..., N t independent Brownian motions, αt i = control of i-th player, long-time-average cost functional: J i (X, α 1,..., α N ) := [ 1 T lim T + T E 0 R i 2 (αi t) 2 + F i (Xt 1,..., Xt N )dt ], with R i > 0, quadratic running cost, for some reference position X i F i (x 1,..., x N ) := ( ) T X X i Q i ( ) X X i N = qjk i (x j x j i )(x k x k i ), qii i > 0, j,k=1 Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

5 An admissible strategy α is a Nash equilibrium if J i (X, α) = min α i J i (X, α 1,..., α i 1, α i, α i+1,..., α N ) i = 1,..., N. Admissible strategies are adapted controls such that E[Xt i ], E[(X i t )2 ] C, t > 0 and prob. measure m α i : [ ] 1 T ( ) lim T + T E g Xt i dt = g(x)dm α i (x). 0 for any polynomial g, deg(g) 2, locally uniformly in x i = X i 0. Example: affine feedbacks α i (x) = K i x + c i, K i > A i, whose corresponding diffusion process is ergodic dx i t = [(A i K i )X i t c i ]dt + σ i dw i t. R Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

6 The HJB-K PDEs As in Lasry-Lions we set H i (x, p) = p2 A i xp, ν i := (σi ) 2 2R i 2, f i (x; m 1,..., m N ) := F i (x 1,..., x i 1, x, x i+1,..., x N ) dm j (x j ), R N 1 j i ν i v i xx + (v i x) 2 2R i A i xv i x + λ i = f i (x; m 1,..., m N ), i = 1,..., N ( ν i (m i ) xx v i x R i m i A i xm i = 0, i = 1,..., N )x R m i(x)dx = 1, m i > 0 in R, Cannot normalize v i by R v i (x)dx = 0 as in the periodic case! Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

7 Quadratic-Gaussian solutions Look for solutions with each v i quadratic and measures m i Gaussian v i (x) = (x µ i) 2 + R ia i x 2, 2s i 2 ( m i (x) = c i exp v i (x) ν i + R ia i x 2 ) = R i 2 1 2πsi ν i R i exp Now the unknowns are the 3N constants µ i, s i, λ i. A useful auxiliary matrix B is B ii := 2q i ii + R i (A i ) 2, B ij := 2q i ij i j. ( (x µ i) 2 ) 2s i ν i R i Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

8 Theorem: Q-G solution of the 2N HJB-K system If det B 0 then i) there exists unique µ i, s i, λ i such that the Q-G solution v i, m i solves the 2N HJB-K system and s i = ( 2q i ii R i + (R i A i ) 2) 1/2, µ = B 1 p, p i := 2q i ii x i i + 2 j i q i ij x j i, ii) the affine feedback α i (x) = x µ i s i R i + A i x, i = 1,..., N, is a Nash equilibrium point for all initial positions X R N among the admissible strategies and J i (X, α) = λ i for all X and i. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

9 Proof i) Plug the candidate solutions in the equation, find 2nd degree polynomial and equate the coefficients, solve 3N algebraic equations for the 3N unknown parameters. ii) It s a verification theorem, the idea of proof is classical, here must be careful with the unbounded terms... Remark: the Nash equilibrium feedback does NOT depend on the noise intensities σ i. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

10 Proof i) Plug the candidate solutions in the equation, find 2nd degree polynomial and equate the coefficients, solve 3N algebraic equations for the 3N unknown parameters. ii) It s a verification theorem, the idea of proof is classical, here must be careful with the unbounded terms... Remark: the Nash equilibrium feedback does NOT depend on the noise intensities σ i. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

11 Proof i) Plug the candidate solutions in the equation, find 2nd degree polynomial and equate the coefficients, solve 3N algebraic equations for the 3N unknown parameters. ii) It s a verification theorem, the idea of proof is classical, here must be careful with the unbounded terms... Remark: the Nash equilibrium feedback does NOT depend on the noise intensities σ i. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

12 A symmetry condition on the running cost F i symmetric with respect to the position of any two other players: F i (x 1,..., x j,..., x k,..., x N ) = F i (x 1,..., x k,..., x j,..., x N ) j, k i. This is equivalent, j, k, l, m i, l j, k l, k m, q i ij = q i ik =: β i 2 = primary cost of cross-displacement x j i = x k i =: r i = reference position of the other players q i jj = q i kk =: η i q i lj = q i kl = q i km =: γ i We also set q i := q i ii h i := x i i = secondary cost of self-displacement = secondary cost of cross-displacement. = primary cost of self-displacement = preferred own position (happy state) Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

13 Now each F i involves only 6 parameters and can be written as (V i ) F i (x 1,..., x N ) = V i 1 δ xj (x i ) N 1 j i for V i : {prob. measures} {quadratic polynomials} V i [m](x) := q i (x h i ) 2 + β i (x h i )(N 1) (y r i ) dm(y) R ) 2 + γ i ((N 1) (y r i ) dm(y) R + (η i γ i )(N 1) (y r i ) 2 dm(y). Viceversa, F i of the form (V i ) implies F i symmetric. R Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

14 Almost Identical players We say the players are A. I. if F i are symmetric and the players have the same - control system, i.e., A i = A and σ i = σ (so ν i = ν > 0) for all i, - cost of the control, i.e., R i = R > 0 for all i, - reference positions, i.e., h i = h and r i = r for all i, - primary costs of displacement, i.e., q i = q > 0 and β i = β for all i. Secondary costs of displacement γ i, η i may still depend on i. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

15 Theorem: identically distributed Q-G solutions Almost Identical players and 2q + RA 2 β(1 N) = the unique µ, s > 0, λ i, i = 1,..., N such that v i (x) = v(x) := m i (x) = m(x) := (x µ)2 2s 1 2πsνR exp + RAx 2 2 ( (x µ)2 2sνR solve the 2N HJB-K equations are ( s = 2qR + R 2 A 2) 1/2 2qh + rβ(n 1), µ = 2q + β(n 1) + RA 2, ) λ i = ν s + νra µ2 2Rs 2 + qh2 hβ(n 1)(µ r) + γ i (N 1)(N 2)(µ r) 2 + η i (N 1)(sνR + (µ r) 2 ). Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

16 The limit as N + Assume the parameters A, ν, R, h, r independent of N and the coefficients of F i,n scale, as N + q N q, β N β N, ηn i η N, γn i γ N 2 Then for any prob. measure m on R, i, V N i [m](x) V [m](x) locally uniformly in x, V [m](x) := q(x h) 2 + β(x h) R (y r) dm(y) ( 2 + γ (y r) dm(y)) + η R R (y r) 2 dm(y). Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

17 The Mean Field Equations (MFE) νv xx + (vx )2 2R Axv x + λ = V [m](x) in R, νm xx ( v x R m Axm) = 0 in R, x ] [ min v(x) RAx 2 2 = 0, R m(x)dx = 1, m > 0 in R. Remark: the normalization of v is different from v dx = 0 of the periodic case. Uniqueness V is monotone increasing in L 2 β 0 so in this case there is at most one solution of MFEs. Meaning: β > 0 if imitation is costly, β < 0 if imitation is rewarding. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

18 Q-G solution of the MFEs 2q + RA 2 β = the MFEs has exactly one solution v, m, λ of the Q-G form v(x) := given by (x µ)2 2s s = + RAx 2 ) 2, m(x) := 1 (x µ)2 exp (, 2πsνR 2sνR ( 2qR + R 2 A 2) 1/2, µ = 2qh + rβ β + 2q + RA 2, λ = ν s + νra µ2 2Rs 2 + qh2 hβ(µ r) + (γ + η)(µ r) 2 + ηsνr Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

19 Convergence Theorem Call v N, m N, λ N i the identically distributed Q-G solutions of the 2N HJB-K equations for N players. Then 2q + RA 2 β = as N +, v N v in C 2 loc (R) m N m in C k (R) for all k, λ N i λ for all i, where v, m, λ is the Q-G solution of the MFEs. Remark: in the critical case no Q-G solution if 2qh βr, 2q + RA 2 = β a continuum of Q-G solutions if 2qh = βr. A similar phenomenon of infinitely many Q-G solutions occurs in the MF log-model of population distribution studied by Guéant. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

20 Convergence Theorem Call v N, m N, λ N i the identically distributed Q-G solutions of the 2N HJB-K equations for N players. Then 2q + RA 2 β = as N +, v N v in C 2 loc (R) m N m in C k (R) for all k, λ N i λ for all i, where v, m, λ is the Q-G solution of the MFEs. Remark: in the critical case no Q-G solution if 2qh βr, 2q + RA 2 = β a continuum of Q-G solutions if 2qh = βr. A similar phenomenon of infinitely many Q-G solutions occurs in the MF log-model of population distribution studied by Guéant. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

21 The vanishing viscosity limit for N players In the system of 2N HJB-K equations let ν j = (σ j ) 2 /2 0 for the j-th player. From the explicit formulas we see v j, α j, µ j, s j independent of ν j, m j δ µj and the j-th Kolmogorov equation ( v j x R j m j A j xm j )x = 0 is satisfied in the sense of distributions. If all ν i = (σ i ) 2 /2 0, i = 1,..., N, the limit 2N HJB-K equations are of first order and the Quadratic-Dirac solutions give the value of a deterministic game for the same feedback Nash equilibrium α i (x) = x µ i s i R i + A i x. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

22 The vanishing viscosity limit in the MFEs Letting ν 0 in the Mean Field equations we have v, s, µ independent of ν, m δ µ and get a Quadratic-Dirac solution of the first order MFEs (v x ) 2 2R Axv x + λ = V [m](x) ( v x R m Axm) x = 0 [ ] min v(x) RAx 2 2 = 0, m(x)dx = 1, m > 0. Rmk: the limits ν 0 and N commute. R Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

23 Other singular limits Cheap control : can take the limit as R i 0 of solutions to 2N HJB-K equations for N players. Again m i Dirac mass, limits R 0 and N commute. Large cost of cross-displacement : if lim N N βn = + instead of β, the term β N (N 1)(x h) (y r) dm(y) is singular but the solutions have a Q-G limit with µ = r. R Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

24 Other singular limits Cheap control : can take the limit as R i 0 of solutions to 2N HJB-K equations for N players. Again m i Dirac mass, limits R 0 and N commute. Large cost of cross-displacement : if lim N N βn = + instead of β, the term β N (N 1)(x h) (y r) dm(y) is singular but the solutions have a Q-G limit with µ = r. R Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

25 Discounted cost functional and vanishing discount For ρ > 0, infinite horizon and discounted running cost: [ + ( ) ] J i Ri = E 2 (αi t) 2 + F i (Xt 1,..., Xt N ) e ρt dt. 0 For quadratic F i find again Q-G solutions of 2N HJB-K equations where λ i is replaced by ρv i. Can show that as ρ 0 ρv i ρ λ i, m ρ i m i, v i ρ(x) v i ρ(0) + (µ ρ i )2 /2s ρ i v i (x) and the limit solves the 2N HJB-K equations of the ergodic games, as N + and ρ > 0 fixed the Q-G solutions converge to the MFEs with 1st equation νv xx + (v x ) 2 /2R Axv x + ρv = V [m](x) limits ρ 0 and N commute. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

26 Models of population distribution Guéant studied a model with discounted cost functional [ + ( α i E t 2 ) ] 2 + q X t i h 2 log m(xt i ) e ρt dt, q 0, 0 and therefore the first MF equation is (A = 0, R = 1) νv xx + (v x ) 2 /2 + ρv = q x h 2 log m(x) V is NOT integral operator, not deriving from limit of N players; V decreasing in m models a population whose agents wish to resemble their peers, imitation is rewarding; MFEs have one Q-G solution of mean h q > 0, a continuum of Q-G solutions with any mean µ if q = 0. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

27 L-Q model with A = 0, R = 1, r = h has first MF equation νv xx + (v x ) 2 /2 + λ = q(x h) 2 + β(x h) (y r) dm(y) R + γ( (y r) dm(y)) 2 + η (y r) 2 dm(y) R R if β > 0 imitation is costly, uniqueness of solution, if β < 0 imitation is rewarding as in the LOG model, MFEs have one Q-G solution of mean h q > 0 and q β/2, a continuum of Q-G solutions with any mean µ if q = β/2: same bifurcation phenomenon at different critical value, Var(m) + as q 0, m tends to uniform distribution. Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24

28 Developments d dimensional control system for each player, d 1: need o solve matrix Riccati equations; the MFEs are PDEs instead of ODEs. Interacting populations with L-Q costs, to be compared with the log-model of Guéant. Thanks for your attention! Thanks Fabio, Italo and Maurizio for the very nice meeting! Martino Bardi (Università di Padova) L-Q MFG Roma, May / 24