Mean Field Games on networks - PDF Free Download

Mean Field Games on networks Claudio Marchi Università di Padova joint works with: S. Cacace (Rome) and F. Camilli (Rome) C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 1 / 35

Outline Brief introduction Definition of networks Formal derivation of the MFG system on networks Study of the MFG system on networks A numerical scheme C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 2 / 35

Model example on the torus T n [Lasry-Lions, 06] Consider a game with N rational and indistinguishable players. The i-th player s dynamics is dx i t = α i tdt + 2νdW i t, X i 0 = x i T n where ν > 0, W i are independent Brownian motions and α i is the control chosen so to minimize the cost functional 1 T lim inf T + T E x L(X i s, αs) i + V 1 δ j N 1 X ds s. 0 The Nash equilibria are characterized by a system of 2N equations. As N +, this system reduces to the following one: j i C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 3 / 35

(MFG-T n ) ν u + H(x, Du) + ρ = V ([m]) in T n ( ) ν m + div m H p (x, Du) = 0 in T n T m dx = 1, m > 0 n T u dx = 0 n H(x, p) := sup q R n{ p q L(x, q)}; [m] means that V depends on m in a local or in a nonlocal way. Theorem [Lasry-Lions 06] There exists a smooth solution (u, m, ρ) to the above problem; Assume either V is strictly monotone in m (i.e. (V ([m T n 1 ]) V ([m 2 ])) (m 1 m 2 )dx 0 implies m 1 = m 2 ) or V is monotone in m (i.e. (V ([m T n 1 ]) V ([m 2 ]))(m 1 m 2 )dx 0) and H is strictly convex in p. Then the solution is unique. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 4 / 35

Basic References for MFG theory: Lasry-Lions, C.R. Math. Acad. Sci. Paris 343 (2006), 619-625. Lasry-Lions, C.R. Math. Acad. Sci. Paris 343 (2006), 679-684. Lasry-Lions, Jpn. J. Math. 2 (2007), 229-260. Huang-Malhamé-Caines, Commun. Inf. Syst. 6 (2006), 221-251. Lions course at College de France 06-12 and 16-17 www.college-de-france.fr Cardaliaguet, Notes on MFG (from Lions lectures at College de France), www.ceremade.dauphine.fr/ cardalia/ Achdou-Capuzzo Dolcetta, SIAM J. Num. Anal. 48 (2010), 1136-1162. Achdou-Camilli-Capuzzo Dolcetta, SIAM J. Num. Anal. 51 (2013),2585-2612. MFG on graphs (i.e., agents have a finite number of states) Discrete time, finite state space: Gomes-Mohr-Souza, J. Math. Pures Appl. 93 (2010), 308-328. Continuous time, finite state space: Gomes-Mohr-Souza, Appl. Math. Optim., 68 (2013), 99-143. Guéant, Appl. Math. Optim. 72 (2015), 291-303. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 5 / 35

Network A network is a connected set Γ consisting of vertices V := {v i } i I and edges E := {e j } j J connecting the vertices. We assume that the network is embedded in the Euclidian space R n and that any two edges can only have intersection at a vertex. (a) An example of network C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 6 / 35

Some Notations Inc i := {j J : e j incident to v i } is the set of edges incident to the vertex v i. A vertex v i is a transition vertex if it has more than one incident edge. We denote by Γ T = {v i, i I T } the set of transition vertices. A vertex v i is a boundary vertex if it has only one incident edge. For simplicity, we assume that the set of boundary vertices is empty. Any edge e j is parametrized by a smooth function π j : [0, l j ] R n. For a function u : Γ R we denote by u j : [0, l j ] R its restriction to e j, i.e. u(x) = u j (y) for x e j, y = π 1 j (x). The derivative are considered w.r.t. the parametrization. The oriented derivative of a function u at a transition vertex v i is { limh 0 +(u j u(v i ) := j (h) u j (0))/h, if v i = π j (0) lim h 0 +(u j (l j h) u j (l j ))/h, if v i = π j (l j ). C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 7 / 35

Some functional spaces u C q,α (Γ), for q N and α (0, 1], when u C 0 (Γ) and u j C q,α ([0, l j ]) for each j J. We set u (q+α) Γ = max u j (q+α) j J [0,l j ]. u L p (Γ), p 1 if u j L p (0, l j ) for each j J. We set u L p = ( j J u j p L p (e j ) )1/p. u W k,p (Γ), for k N, k 1 and p 1 if u C 0 (Γ) and u j W k,p (0, l j ) for each j J. We set u W k,p = ( j J u j p W k,p (e j ) )1/p. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 8 / 35

Formal derivation of MFG systems on networks Dynamics of a generic player. Inside each edge e j, the dynamics of a generic player is dx t = α t dt + 2ν j dw t where α is the control, ν j > 0 and W is an independent Brownian motions. At any internal vertex v i, the player spends zero time a.s. at v i and it enters in one of the incident edges, say e j, with probability β ij with β ij > 0, j Inc i β ij = 1. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 9 / 35

Discussion on the transition condition: probabilistic approach We consider the uncontrolled case. Fix a vertex v i. Rigorous definition of enters in one of the incident edges... For δ > 0, consider θ δ := inf{t > 0 dist(x t, v i ) = δ}. Then, lim P{X δ 0 + θ δ e j } = β ij. Fattening interpretation. Let M ε be the set in R n obtained enlarging each edge e j by a ball of radius εβ ij. One can obtain these dynamics as the limit as ε 0 + of a Brownian motion in M ε with normal reflection at the boundary. Itô s formula still holds true. See: [Freidlin-Wentzell, Ann. Prob. 93], [Freidlin-Sheu, PTRF 00]. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 10 / 35

Discussion on the transition condition: analytical approach Consider the operator A defined on C 0 (Γ), defined for x e j by with domain d 2 u A j u := ν j (y) + ᾱdu dy 2 dy (y), D(A) := { u C 2 (Γ) y = π 1 (x) j Inc i β ij j u(v i ) = 0 }{{} (weighted) Kirchhoff condition A generates on Γ the Markov process X t described before. A fulfills the Maximum Principle. Fattening interpretation. The solution of Au = 0 is the lim ε 0 + of u ε, solution of a similar problem in M ε with uε n = 0 on M ε. See: [Freidlin-Wentzell, Ann. Prob. 93], [Below-Nicaise, CPDE 96].In [Lions course, 17]: fattening interpretation for some controlled cases. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 11 / 35 }.

Formal derivation of the MFG system on the network We formally derive the MFG system on the network: the HJB equation is obtained through the dynamic programming principle while the FP equation is obtained as adjoint of the linearized HJB one. Hence, the HJB equation is ν j 2 u + H j (x, u) + ρ = V [m] x e j, j J j Inc i β ij j u(v i ) = 0 i I T u j (v i ) = u k (v i ) j, k Inc i. The linearized equation is ν j 2 w + p H j (x, u) w = 0 x e j, j J j Inc i β ij j w(v i ) = 0 i I T w j (v i ) = w k (v i ) j, k Inc i. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 12 / 35

Writing the weak formulation for a test function m, we get 0 = ( νj 2 w + p H j (x, u) w ) m dx j J e j = [ νj 2 m (m p H j (x, u)) ] w dx j J e j + ( νj j m(v i ) + p H(v i, u)m j (v i ) ) w(v i ) i I T i I T j Inc i j Inc i ν j m j (v i ) j w(v i ) }{{} =0 if m j (v i )ν j β ij By the integral term, we obtain = m k (v i )ν k β ik ν j 2 m + (m p H j (x, u)) = 0 x e j, j J.. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 13 / 35

The MFG system on a network Assume ν j β ij (MFG Γ ) = ν k β ik j, k Inc i, i I T. The MFG systems is ν 2 u + H(x, u) + ρ = V (m) ν 2 m + (m p H(x, u)) = 0 ν j j u(v i ) = 0 j Inc i [ν j j m(v i ) + p H j (v i, j u)m j (v i )]=0 j Inc i u j (v i ) = u k (v i ) m j (v i ) = m k (v i ) u(x)dx = 0 Γ m(x)dx = 1, m 0. Γ x Γ x Γ i I T i I T j, k Inc i, i I T j, k Inc i, i I T C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 14 / 35

The MFG systems on networks Theorem (Camilli-M., SJCO 16) We assume H j C 2 (e j R), convex, with δ p 2 C H j (x, p) δ p 2 + C, ν j > 0, V C 1 ([0, + )). Then, there exists a solution (u, m, ρ) C 2 (Γ) C 2 (Γ) R to (MFG Γ ). Moreover, assume either V is strictly monotone in m or V is monotone in m and H is strictly convex in p. Then the solution is unique. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 15 / 35

Step 1: On the HJB equation. Sketch of the proof For f C 0,α (Γ),!(u, ρ) C 2 (Γ) R solution to (HJB) ν 2 u + H(x, u) + ρ = f (x), x Γ j Inc i ν j j u(v i ) = 0 i I T u j (v i ) = u k (v i ) j, k Inc i, i I T Γ u(x) = 0. Moreover u C 2,α (Γ) and: u C 2,α (Γ) C, ρ max Γ H(, 0) f ( ). The proof is based on u λ W 1,2 (Γ), weak solution to the discounted approximation ν 2 u λ + H(x, u λ ) + λu λ = f (x) x Γ as in [Boccardo-Murat-Puel, 83]; the Comparison Principle applies; u λ C 2,α (Γ) by the 1-d of the problem and Sobolev theorem; as λ 0 +, λu λ ρ and (u λ min u λ ) u. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 16 / 35

Step 2: On the FP equation. For b C 1 (Γ), there exists a unique weak solution m W 1,2 (Γ) to (FP) ν 2 m + (b(x) m) = 0 x Γ j Inc i [b(v i )m j (v i ) + ν j j m(v i )] = 0 i I T m j (v i ) = m k (v i ) j, k Inc i, i I T m 0, m(x)dx = 1. Γ Moreover, m is a classical solution with m H 1 C, 0 < m(x) C (for some C > 0 depending only on b and ν). The proof is based on the existence of a weak solution is based on the theory of bilinear forms; the adjoint problem (both equation and transition condition) fulfills the Maximum Principle; m C 2 (Γ) by the 1-d of the problem and Sobolev theorem. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 17 / 35

Step 3: Fixed point argument. We set K := {µ C 0,α (Γ) : µ 0, Γ µdx = 1} and we define an operator T : K K according to the scheme as follows: µ u m given µ K, solve (HJB) with f (x) = V (µ(x)) for the unknowns u = u µ and ρ, which are uniquely defined by Step 1; given u µ, solve (FP) with b(x) = p H(x, u µ ) for the unknown m which is uniquely defined by Step 2; set T (µ) := m. Since T is continuous with compact image, Schauder s fixed point theorem ensures the existence of a solution. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 18 / 35

Step 4: Uniqueness. Cross-testing the equations in (MFG Γ ), by the transition conditions, we get (m 1 m 2 )(V (m 1 ) V (m 2 ))dx + j J e j }{{} 0 by monotonicity [ m 1 Hj (x, j u 2 ) H j (x, j u 1 ) p H j (x, j u 1 ) j (u 2 u 1 ) ] dx+ e j }{{} j J j J 0 by convexity [ m 2 (Hj (x, j u 1 ) H j (x, j u 2 ) p H j (x, j u 2 ) j (u 1 u 2 ) ] dx = 0. e j }{{} 0 by convexity Therefore, each one of these three lines must vanish and we conclude as in [Lasry-Lions 06]. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 19 / 35

A finite difference scheme for MFG on network We introduce a grid on Γ. For the parametrization π j : [0, l j ] R n of e j, let y j,k = kh j (k = 0,..., N h j ) be an uniform partition of [0, l j ]: G h = {x j,k = π j (y j,k ), j J, k = 0,..., N h j }. Inc + i := {j Inc i : v i = π j (0)}, Inc i := {j Inc i : v i = π j (N h j h j )}. We introduce the (1-dimensional) finite difference operators (D + U) j,k = U j,k+1 U j,k h j, [D h U] j,k = ( (D + U) j,k, (D + U) j,k 1 ) T, (Dh 2 U) j,k = U j,k+1 + U j,k 1 2U j,k hj 2. We introduce the inner product. For U, W : G h R, set (U, W ) 2 = j J N h j 1 k=1 h j U j,k W j,k + i I ( j Inc + i h j 2 U j,0w j,0 + j Inc i h j 2 U ) j,n W j h j,n h. j C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 20 / 35

We introduce the numerical Hamiltonian g j : [0, l j ] R 2 R, s.t.: (G 1 ) monotonicity: g j (x, q 1, q 2 ) is nonincreasing with respect to q 1 and nondecreasing with respect to q 2. (G 2 ) consistency: g j (x, q, q) = H j (x, q), x [0, l j ], q R. (G 3 ) differentiability: g j is of class C 1. (G 4 ) superlinear growth : g j (x, q 1, q 2 ) α((q 1 )2 + (q + 2 )2 ) γ/2 C for some α > 0, C R and γ > 1. (G 5 ) convexity : (q 1, q 2 ) g j (x, q 1, q 2 ) is convex. We introduce a continuous numerical potential V h such that C independent of h such that max (V h [M]) j,k C, (V h [M]) j,k (V h [M]) j,l C y j,k y j,l. j,k for all M K h := {M : M is continuous, M j,k 0, (M, 1) 2 = 1}. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 21 / 35

We get the following system in the unknown (U, M, R) ν j (DhU) 2 j,k + g(x j,k, [D h U] j,k ) + R = V h (M j,k ) ν j (D 2 hm) j,k + B h (U, M) j,k = 0, j Inc + i j Inc + i j Inc i [ νj (D + U) j,0 + h j 2 (V j,0 R) ] U, M continuous at v i, i I, (M, 1) 2 = 1, (U, 1) 2 = 0, j Inc i [ νj (D + M) j,0 + M j,1 g q 2 (x j,1, [D h U] j,1 ) ] [ νj (D + U) j,n h j 1 h j 2 (V j,n h j [ νj (D + M) j,n h 1 + M g j,n j j h 1 (x q j,n h 1 j 1, [D hu] j,n h j 1 )] = 0 R) ] = 0 C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 22 / 35

Theorem (Cacace-Camilli-M., M2AN 17) For any h = {h j } j J, the discrete problem has at least a solution (U h, M h, ρ h ). Moreover ρ h C 1, U h + D h U h C 2 for some constants C 1, C 2 independent of h. Moreover, if V h is strictly monotone, then the solution is unique. If (u, m, ρ) is the solution of the MFG system (MFG Γ ), then [ ] U h u + M h m + ρ h ρ = 0. lim h 0 C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 23 / 35

Solution of the discrete MFG system via Nonlinear-Least-Squares The solution of the MFG discrete system is usually performed by means of some regularizations, e.g. large time approximation or ergodic approximation. We propose a different method: We collect the unknowns in a vector X = (U, M, R) of length 2N h + 1; we consider the nonlinear map F : R 2Nh +1 R 2Nh +2 defined by ν j (DhU) 2 j,k + g(x j,k, [D h U] j,k ) + R V h (M j,k ), F(X) = ν j (DhM) 2 j,k + B h (U, M) j,k, [ νj (D + U) j,0 + h j 2 (V j,0 R) ] j Inc + i j Inc + i j Inc i j Inc i [ νj (D + M) j,0 + M j,1 g q 2 (x j,1, [D h U] j,1 ) ] (M, 1) 2 1 (U, 1) 2. [ νj (D + M) j,n h j 1 + M j,n h j 1 [ νj (D + U) j,n h 1 h j j 2 (V j,n j h g (x q j,n h 1 j 1, [D hu] j,n h j 1 )] The solution of the discrete MFG is the unique X s.t. F(X ) = 0. R) ] C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 24 / 35

The system F(X ) = 0 is formally overdetermined (2N h + 2 equations in 2N h + 1 unknowns), hence the solution is meant in the following nonlinear-least-squares sense: X 1 = arg min X 2 F(X) 2 2. The previous optimization problem is solved by means of the Gauss-Newton method J F (X k )δ X = F(X k ), X k+1 = X k + δ X. via the QR factorization of the Jacobian J F (X k ). C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 25 / 35

Numerical experiments We consider a network with 2 vertices and 3 edges (boundary vertices are identified!). Each edge has unit length and connects (0, 0) to (cos(2πj/3), sin(2πj/3)) with j = 0, 1, 2. Data: Uniform diffusion ν j ν H j (x, p) = 1 2 p 2 + f (x) f (x) = s j (1 + cos(2π ( 0 x + 1 ) ) 2 ) -0.5 s j {0, 1} V [m] = m 2-1 1 0.5-0.5 0 0.5 1 Computational time for N h 5000 is of the order of seconds! C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 26 / 35

Cost active on all edges ν = 0.1, s 0 = 1, s 1 = 1, s 2 = 1, V [m] = m 2 1 0.5 0 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1-0.1 0-0.2-0.5-1 -0.5 0 0.5 1 M -0.6-0.4-0.2 0 0.2 0.4 0.6 0 0.8 1-1 -0.8-0.6-0.4-0.2 M (blu) and U (red) 0.2 0.4 0.6 0.8 1 Computed ρ = 1.066667 C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 27 / 35

Cost active on two edges ν = 0.1, s 0 = 1, s 1 = 1, s 2 = 0, V [m] = m 2 1 0.5 0-0.5 1.2 1 0.8 0.6 0.4 0.2 0-0.2-0.4 0.6 0.81-1 -0.5 0 0.5 1 M -0.6-0.4-0.2 0 0.2-0.6-0.4-0.2 00.20.4 0.4 0.6 0.8 1-1-0.8 M (blu) and U (red) Computed ρ = 0.741639 C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 28 / 35

Small viscosity, cost active on all edges ν = 10 4, s 0 = 1, s 1 = 1, s 2 = 1, V [m] = m 2 1 0.5 0 1 0.8 0.6 0.4 0.2 0-0.2-0.5-1 -0.5 0 0.5 1 M -0.6-0.4-0.2 1 0.8 0.6 0 0.4 0.2 0 0.2 0.4 0.6 0.8 1-1 -0.8-0.6-0.4-0.2 M (blu) and U (red) Computed ρ = 1.116603 C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 29 / 35

Small viscosity, cost active on two edges ν = 10 4, s 0 = 1, s 1 = 1, s 2 = 0, V [m] = m 2 1 0.5 0 1.2 1 0.8 0.6 0.4 0.2 0-0.2-0.5-1 -0.5 0 0.5 1 M -0.6-0.4-0.2 1 0.8 0.6 0 0.4 0.2 0.2 0.4 0 0.6 0.8-0.8-0.6-0.4-0.2 1-1 M (blu) and U (red) Computed ρ = 0.725463 C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 30 / 35

Numerical experiments ν = 0.1, s 0 = 1, s 1 = 1, s 2 = 1, V [m] = 1 4 π arctan(m) 1 0.5 0 1.2 1 0.8 0.6 0.4 0.2 0-0.2-0.4-0.5-1 -0.5 0 0.5 1 M -0.6-0.4-0.2 0 0.2 0.4 0.6 0.8 1-1 -0.8-0.6-0.4-0.2 0 M and U 0.2 0.4 0.6 0.8 1 Computed Λ = 1.219979 C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 31 / 35

Numerical experiments ν = 10 3, s 0 = 1, s 1 = 1, s 2 = 1, V [m] = 1 4 π arctan(m) 1 0.5 0 1 0.5 0-0.5-1 -0.5-1 -0.5 0 0.5 1 M -0.6-0.4-0.2 1 0 0.8 0.6 0.2 0.4 0.2 0.4 0 0.6 0.8-0.6-0.4-0.2-0.8 1-1 M and U Computed Λ = 2.832411 C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 32 / 35

More complex networks 0.5 0-0.5-1 -1.5-2 -2.5-3 -3.5-2 0 2 4 6 8 10 12 0.5 0.4 0.3 0.2 0.1 0 0.8 1 0.6 0.4 0.2-0.2 0-0.4 0 2 0 4 2 6 0 4 0 M -0.5 6 U -0.5 8-1 -1-1.5 8-1.5 10-2 -2-2.5 10-2.5 C. Marchi (Univ. of12 Padova) -3 Mean Field Games on networks 12Roma, -3 June 14 th, 2017 33 / 35

Comments and Remarks: rigorous derivation of the system starting from the game with N players. more general transitions conditions (arbitrary weights for the edges, controlled weights...) and/or lack of continuity condition. first order MFG systems on networks. C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 34 / 35

Thank You! C. Marchi (Univ. of Padova) Mean Field Games on networks Roma, June 14 th, 2017 35 / 35