Convex Analysis for LQG Systems with Applications to Major Minor LQG Mean Field Game Systems

Transcription

1 arxiv: v1 [cs.sy 16 Oc 218 Convex Analysis for LQG Sysems wih Applicaions o Major Minor LQG Mean Field Game Sysems 1 Absrac Dena Firoozi, Sebasian Jaimungal, Peer E. Caines LQG mean field game sysems consising of a major agen and a large populaion of minor agens have been addressed in he lieraure. In his paper, a novel convex analysis approach is uilized o rerieve he bes response sraegies for he major agen and each individual minor agen which collecively yield an ǫ-nash equilibrium for he enire sysem. 2 Inroducion In a convex analysis approach o opimizaion for saic sysems, he Gâeaux derivaive of he funcional o be opimized is used o solve he problem (see e.g., [1, [2). In [3, he relaionship beween he Gâeaux derivaive of he cos funcional of a dynamic sysem and is Hamilonian is esablished. A sochasic racking problem in finance is sudied in [4 using he convex analysis approach, while an algorihmic rading problem is invesigaed in [5 and he bes response rading sraegies are obained for a large number of heerogeneous raders using he convex analysis approach. In his work, a convex analysis mehod is used o rerieve he bes response sraegies for he major minor LQG mean field game (MM LQG MFG) sysems addressed in [6. 1

2 3 Major Minor LQG Mean Field Game Sysems A large populaionn of minor agens wih a major agen, where agens are subjec o sochasic linear dynamics and quadraic cos funcionals are considered. Each agen is coupled wih oher agens hrough heir dynamics and cos funcional wih he average sae of minor agens, i.e. he empirical mean field. 3.1 Dynamics he dynamics of he major and minor agens are assumed o be given, respecively, by dx = [A x +F x (N) +B u +b ()d+σ dw, (1) dx i = [A kx i +F kx (N) +B k u i +b k()d+σ k dw i, (2) where, 1 i N <, and he subscrip k, 1 k K N, denoes he ype of a minor agen. Here x i R n, i N, are he saes, u i R m, i N are he conrol inpus, w i, i N} denoes (N +1) independen sandard Wiener processes in R r, wherew i is progressively measurable wih respec o he filraion F w := (F w ) [,. All marices in (1) and (2) are consan and of appropriae dimension; vecors b (), and b k () are deerminisic funcions of ime Agens ypes Minor agens are given in K disinc ypes wih 1 K <. he noaion Ψ k Ψ(θ i ), θ i = k is inroduced where θ i Θ, wih Θ being he parameer se, and Ψ may be any dynamical parameer in (2) or wigh marix in he cos funcional (5). he symbol I k denoes I k = i : θ i = k, 1 i N}, 1 k K where he cardinaliy of I k is denoed by N k = I k. hen, π N = (π1 N,...,πK N), πn k = N k, 1 k K, denoes he empirical disribuion of he N parameers(θ 1,...,θ N ) sampled independenly of he iniial condiions and Wiener processes of he agensa i,1 i N. he firs assumpion is as follows. Assumpion 1. here exissπ such ha lim N π N = π a.s. 2

3 3.1.2 Conrol σ-fields We denoe by F i := (F) i [,, 1 i N, he naural filraion generaed by he i-h minor agen s sae (x i ) [,, by F := (F ) [, he naural filraion generaed by he major agen s sae (x ) [,, and F := (F ) [, he naural filraion generaed by he saes of all agens((x i ) 1 i N,x ) [,. Nex, we inroduce wo admissible conrol ses. Le U denoe he se of feedback conrol laws wih second momen lying in L 1 [,, for any finie, which are adaped o he local informaion se of he major agena, i.e. F. he se of conrol inpus U i, 1 i N, based upon he local informaion se of he minor agen A i, 1 i N, consiss of he feedback conrol laws adaped o he filraion F i,r := (F i,r ) [,, where F i,r := F i F, 1 i N, while Ug N is adaped o he general filraion F := (F ) [,, and he L 1 [, consrain on second momens applies in each case. 3.2 Cos funcionals he individual (finie) large populaion finie horizon cos funcional for he major agen is specified by J N (u,u ) = 1 [ 2 E x Φ(x (N) ) 2 G + e ρ x Φ(x (N) ) 2 Q where +2 ( x Φ(x (N) ) ) N u + u 2 R }d, (3) Φ(.) := H x (N) +η. (4) Assumpion 2. For he cos funcional (3) o be convex, we assume ha G, R >, and Q N R 1 N >. he individual (finie) large populaion finie horizon cos funcional for a minor agena i,1 i N, is specified as Ji N (u i,u i ) = 1 [ 2 E x i Ψ(x(N) ) 2 G k + e ρ x i Ψ(x(N) ) 2 Q k +2 ( x i Ψ(x(N) ) ) N k u i + ui 2 R k }d, (5) where Ψ(.) := H k x +Ĥπ k x(n) +η k. (6) 3

4 Assumpion 3. For he cos funcional (5) o be convex, we assume ha G k, R k >, andq k N k R 1 k N k > for1 k K. We noe ha he major agena and minor agensa i, 1 i N are coupled wih each oher hrough he average ermx (N) = 1 N N i=1 xi in heir dynamics and cos funcionals given by, respecively, (1)-(2) and (3)-(5). 4 Inroducion o Convex Analysis Approach Le V be a reflexive Banach space wih he dual space V and U be a non-empy closed convex subse ofv. (We noe ha in his paperv = R m.) Definiion 1 (Gâeaux Derivaive). he funcion J defined on a neighbourhood of u V wih values in R is differeniable in he sense of Gâeaux a u in he direcion ofω, if here exissj (u) V such ha J J(u+ǫω) J(u) (u),ω = lim. (7) ǫ ǫ he funcionj (u) is called he Gâeaux derivaive ofj a u. heorem 1 (Euler Inequaliy). Assume ha he funcionj is convex, coninuous, proper, and Gˆaeaux differeniable wih coninuous derivaivej (u). hen J(u) = inf J(v), (8) v U if and only if J (u),v u, v U. (9) Remark 1 (Euler Equaliy). In he case where U = V, ω = v u produces he whole space ofv, and herefore (9) reduces o Euler equaliy which implies ha J (u),ω =, ω V, (1) J (u) =. (11) Proof of heorem 1 and more deails on convex analysis approach may be found in [1 and [2. 4

5 5 Rereival of Major Minor LQG MFG Sraegies: Convex Analysis Approach Following he mean field game mehodology wih a major agen [7, [6, he problem is firs solved in he infinie populaion case where he average erms in he finie populaion dynamics and cos funcional of each agen are replaced wih heir infinie populaion limi, i.e. he mean field. hen specializing o LQG MFG sysems, he major agen s sae is exended wih he mean field, while he minor agen s sae is exended wih he major agen s sae, and mean field; his yields sochasic opimal conrol problems for each agen linked only hrough he major agen s sae and mean field. Finally he infinie populaion bes response sraegies are applied o he finie populaion sysem which yields an ǫ-nash equilibrium [6. he following heorem specifies he conrol laws which yield he infinie populaion Nash equilibrium and heir relaion wih he finie populaion ǫ-nash equilibrium. heorem 2 (ǫ-nash Equilibrium for LQG MFG Sysems). [6 Assume ha he condiions of [6 for he exisence and uniqueness of Nash equilibrium hold, hen he sysem equaions (1)-(5) ogeher wih he mean field equaions (78)-(79) generae a se of conrol lawsumf ui, ;i } whereu i, is given by such ha u, = R 1 u i, = R 1 k [ (N +B Π () )[ (x ),( x ) +B s (), (12) [ (N k +B k Π k() )[ (x i ),(x ),( x ) +B k s k(), (13) (i) he se of infinie populaion conrol laws U MF ui, ;i } yields he infinie populaion Nash equilibrium. J i (u i,,u i, ) = inf u i U,L i J i (u i,u i, ); (ii) All agen sysemsa i, i N, are second order sable. (iii) he se of conrol laws U N MF ui, ; i N}, 1 N <, yields an ǫ-nash equilibrium for all ǫ, i.e. for all ǫ >, here exiss N(ǫ) such ha for alln N(ǫ); Ji N (u i,,u i, ) ǫ inf J u i U N,L i N (u i,u i, ) Ji N (u i,,u i, ). i 5

6 he proof of heorem 2 consiss of wo pars: (i) he se of conrol laws UMF yields he Nash equilibrium for he infinie populaion sysem, (ii) when a finie subse of he conrol laws UMF N is applied o he finie populaion sysem, all agen sysems are second order sable and i yields an ǫ-nash equilibrium. In his secion, a novel convex analysis approach is presened o rerieve he se of bes response sraegiesumf which yields he Nash equilibrium. 5.1 Mean Field Evoluion We inroduce he empirical sae average as x (N k) k = 1 N k N k j=1 x k j, 1 k K, and wrie x (N) = [x (N 1) 1,x (N 2) 2,...,x (N K) K, where he poinwise in ime L2 limi of x (N), if i exiss, is called he mean field of he sysem and is denoed by x = [ x 1,..., x K. We consider for each minor agen A i of ype k, 1 k K, a uniform (wih respec o i) feedback conrol u k i U i,l U i, where U i,l consiss of linear ime invarian conrols, as u i,k = L k 1 xi,k +Σ K l=1 ΣN l j=1 Lk,l 2 xj,l +L k 3 x +mk (), where, L k 1, L k,l 2 andl k 3 are consan marices, andm k is a coninuous bounded funcion of ime. If we subsiueu i,k in (2) for1 i N, and ake he average of he saes of closed loop sysems of ype k, 1 k K, and hence calculae x (N), i can be shown ha he L 2 limi x of x (N), i.e. he mean field saisfies d x = Ā x d+ḡx d+ m()d, (14) where Ā, Ḡ, and m are o be solved for in he racking soluion. By abuse of language, he mean value of he sysem s Gaussian mean field given by he sae process x = [ x 1,..., xk shall also be ermed he sysem s mean field. 5.2 Major Agen: Infinie Populaion o solve he infinie populaion racking problem for he major agen A, firs, is sae is exended wih he mean field process x, where his is assumed o exis. 6

7 hen he dynamics of major agen s exended sae X given as (see [6) where [ A F A = Ḡ Ā [ (x ), ( x ) is dx = A X d+b u d+m ()d+σ dw, (15) [ B, B = [ b (), M () = m() [ σ, Σ = [ w, W = (16) he infinie populaion individual cos funcional for he major agen is given by J (u ) = 1 [ 2 E (X ) G X + (Xs ) Q Xs +2(X s ) N u s +(u s ) R u s 2(Xs) η 2(u s) n }ds, (17). where he corresponding weigh marices are specified by G = [I n, H π G [I n, H π, Q = [I n, H π Q [I n, H π, N = [I n, H π N, η = [I n, H π Q η, n = N η. (18) he dynamics (15) ogeher wih he cos funcional (17) consiue a sochasic LQ opimal conrol problem for he major agen A s exended sysem in he infinie populaion limi. o deermine he opimal conrol u,, Corollary 1 is uilized. o his purpose, he Gâeaux derivaive J (u ) of (17) is compued as follows. he soluionx,u o he sae represenaion of he major agena s exended sysem (15) subjec o he conrol acionu is given by X,u = e A X + e A ( s) ( B u s +M (s) ) ds+ e A ( s) Σ (s)dw s, (19) wherex Rn+nK denoes he iniial sae a =, andφ (,s) = e A( s), s, denoes he major agen s sae ransiion marix. Le X,u+ǫω denoe he soluion o (15) subjec o a perurbed conrol acion u +ǫω in he direcion ofω given by X,u+ǫω = e A X + e A ( s) ( B u s +M (s) ) ds+ 7 +ǫ e A ( s) Σ (s)dw s e A( s) B ωs ds. (2)

8 o find he relaion beween X,u yields X,u+ǫω = X,u +ǫ and X,u+ǫω, (19) is subsiued in (2) which e A( s) B ωs ds. (21) hen, by differeniaing boh sides of (21), he evoluion of X,u+ǫω is given by X,u in erms of dx,u+ǫω = dx,u +ǫb ω d+ǫa e A( s) B ωs ds. (22) he major agen A s cos induced by he perurbed conrol acion u +ǫω and, subsequenly, he perurbed saex,u+ǫω is given by J (u +ǫω ) = 1 [ 2 E e ρ (X,u+ǫω ) G X,u+ǫω + e ρs (Xs,u+ǫω ) Q X,u+ǫω s +2(Xs,u+ǫω ) N (u s+ǫωs)+(u s+ǫωs) R (u s+ǫωs) 2(X s,u+ǫω ) η 2(u s+ǫωs) n }ds where he erminal cos, by uilizing he inegraion by pars echnique, can be presened in inegral form as e ρ (X,u+ǫω ) G X,u+ǫω = (X ) G X + = (X ) G X +2 e ρs (X,u+ǫω s ) G dxs,u+ǫω ρ d [ e ρs (Xs,u+ǫω ) G Xs,u+ǫω ) e ρs (X,u+ǫω s ) G Xs,u+ǫω ds. o wrie J (u + ǫω ) in erms of J (u ), u and X,u, firs (24), and hen (21)-(22) are subsiued in (23) which gives rise o J (u +ǫω ) = J (u )+ǫ e ρs ( s e A (s ) B ω d ) ( G dx,u s +(Q Xs,u +N u s +A G Xs,u ρg Xs,u η )ds ) + ( (Xs,u ) N ωs +(X,u s ) G B ωs ) ( s s +(u s ) R ωs n ω s ds }+ǫ 2 e ρs e A(s ) B ω d) ( G A e A(s ) B ω d 8, (24) (23)

9 s s ρg e A(s ) B ωd+g B ωs+q e A(s ) B ωd+n ωs) +(ω s ) R ωs) } ds. hen he Gaeaux derivaive of J (u ) in he direcion of ω is obained by firs akingj (u ) o he lef hand side of (25), hen dividing boh sides of he equaion byǫ, and finally aking he limi as ǫ, which yields J (u ),ω = e ρs ( s (25) e A(s ) B ωd ) ( G dxs,u +(Q Xs,u +N u s+a G Xs,u ρg X,u s η )ds ) + ( (X,u s ) N ω s+(x,u s ) G B ω s+(u s) R ω s n ω s ) ds }. An applicaion of Fubini s heorem o change he order of inegraion in (26) resuls in [ J (u ),ω = E (ω ) e ρ B G X,u +e ρ N X,u +e ρ R u e ρ n +B (26) e ρs e A (s ) ( G dx,u s + ( A G X,u s ρg X,u s +Q X,u s +N u s η ) ds ) } d By using he inegraion by pars echnique, we have e ρs e (s )( ) A A G Xsds+G dxs ρg Xs,u = d(e ρs e A (s ) G Xs) = e ρ e A ( ) G X e ρ G X, (28) whose subsiuion in (27) yeilds [ J (u ),ω = E (ω ) e ρ B ea ( ) G X,u +e ρ N X,u +e ρ R u e ρ n ) +B e ρs e A (s ) (Q Xs,u +N u s η ds }d. (29) Using he ieraed expecaion propery, he Gâeaux derivaive (29) may be rewrien as. (27) 9

10 [ J (u ),ω = E (ω ) e ρ N X,u +e ρ R u e ρ n [ +B E e ρ e A ( ) G X,u + ) } e ρs e A (s ) (Q Xs,u +N u F s η ds d. (3) hen he following maringale is defined [ M = E e ρ e A G X,u + and is subsiued in (3) o give e ρs e A s (Q Xs,u +N u s η )ds [ J (u ),ω = E (ω) e ρ N X,u +e ρ R u e ρ n +B ( e A M F e ρs e A (s ) (Q X,u s +N u s η )ds) } d, (31) As per Corollary 1, he necessary and sufficien condiion foru, o be he opimal conrol is given by J (u, ),ω =, a.s. for all possible pahs of ω, (32) which implies ha [ ( u, = R 1 N X, n +B eρ Le define p as p = eρ( e A M hen he ansaz forp e A M. e ρs e A (s ) (Q X, s +N u, s η )ds). (33) e ρs e A (s ) (Q X, s +N u, s η )ds ). (34) is proposed o be p = Π ()X +s (), (35) and is subsiued in (33) o give [ u, = R 1 N X n ( +B Π ()X +s () ). (36) 1

11 o find Π () and s (), boh sides of (35) are firs differeniaed, and hen (15) and (36) are subsiued o yield dp = [ ( Π +Π A Π B R 1 N Π B R 1 B Π +Π M ()+Π B R 1 n +ṡ () Nex, boh sides of (34) are differeniaed o give ) X Π B R 1 B s () d+π Σ ()dw, (37) dp = (ρp A p Q X N u + η )d+e ρ e A dm, (38) where according o he maringale represenaion heorem, he maringalem may be wrien as and hence M = M + Z sdw s, (39) dm = Z dw. (4) hen, equaions (35), (36) and (4) are subsiued in (38) o ge [ dp = (ρπ Q +N R 1 N +N R 1 B Π A Π )X, +ρs ()+(N R 1 B A )s ()+ η N R 1 n d+q dw, (41) whereq = eρ e A Z. Finally, for (37) and (41) o be equal, he corresponding drifs and diffusions mus be equal. Hence he following equaions mus hold q = Π Σ (), (42) ρπ = Π +Π A +A Π (B Π +N ) R 1 (B Π +N )+Q, Π () = G, (43) ρs () = ṡ ()+[(A B R 1 N ) Π B R 1 B s () +Π (M ()+B R 1 n )+N R 1 n η, s () =. (44) As can be seen from (43)-(44), he Riccai marixπ () and he offse vecors () saisfy deerminisic differenial equaions. 11

12 5.3 Minor Agen: Infinie Populaion o solve he infinie populaion racking problem for a minor agen A i, 1 i N, firs, is sae is exended wih he major agen s sae and he mean field process x, where his is assumed o exis. hen he dynamics of minor agen A i s exended saex i [ (x i ), (x ), ( x ) is given as (see [6) dx i = A k X i d+b k u i d+m k ()d+σ k dw i, (45) where [ Ak [H A k = k,fk π A B R 1 N B R 1 B, B k = Π [ [ b M k () = k () σk M () B R 1 B, Σ s () k = Σ [ Bk,, W i = [ w i W. (46) he infinie populaion individual cos funcional for minor agena i, 1 i N, is given by J i (u i ) = 1 2 E [ e ρ (X i ) G k X i + where he corresponding weigh marices are specified by e (X ρs s i ) Q k Xs i +2(Xi s ) N k u i s +(ui s ) R k u i s 2(Xs) i η k 2(u i s) n k }ds, (47) G k = [I n, H k, Ĥπ k G k [I n, H k, Ĥπ k, Q k = [I n, H k, Ĥπ k Q k [I n, H k, Ĥπ k, N k = [I n, H k, Ĥπ k N, η k = [I n, H k,ĥπ k Q k η k, n k = N k η k. (48) he dynamics (45) ogeher wih he cos funcional (47) consiue a sochasic LQ opimal conrol problem for he minor agen A i s exended sysem in he infinie populaion limi. o deermine he opimal conrol u i,, Corollary 1 is uilized. For his purpose, he Gâeaux derivaiveji (u i ) of (47) is compued as follows. he soluionx i,u o he sae represenaion of he minor agen A i s exended sysem (45) subjec o he conrol acionu i is given by X i,u = e Ak X i + e ( A k( s) B k u i s +M k(s) ) ds+ e A k( s) Σ k (s)dw i s, (49) 12

13 where X i R 2n+nK denoes he iniial sae a =, and φ k (,s) = e Ak( s), s, denoes he sae ransiion marix for minor agen A i, 1 i N. Le X i,u+ǫω denoe he soluion o (45) subjec o a perurbed conrol acion u i +ǫωi in he direcion ofωi given by X i,u+ǫω = e A k X i + e A k( s) ( B k u i s +M k (s) ) ds + o find he relaion beween X i,u yields X i,u+ǫω e A k( s) Σ k (s)dw i s +ǫ e A k( s) B k ω i sds. (5) and X i,u+ǫω, (49) is subsiued in (5) which = X i,u +ǫ e A k( s) B k ω i sds. (51) hen, by differeniaing boh sides of (51), he evoluion of X i,u+ǫω X i,u is given by dx i,u+ǫω in erms of = dx i,u +ǫb k ωd+ǫa i k e Ak( s) B k ωsds. i (52) he minor agen A i s cos induced by he perurbed conrol acion u i + ǫωi and, subsequenly, he perurbed saex i,u+ǫω is given by Ji (u i +ǫω i ) = 1 [ 2 E e ρ (X i,u+ǫω ) G k X i,u+ǫω + e ρs (Xs i,u+ǫω ) Q k Xs i,u+ǫω +2(Xs i,u+ǫω ) N k (u i s +ǫωi s )+(ui s +ǫωi s ) R k (u i s +ǫωi s ) 2(Xi,u+ǫω s ) η k 2(u i s +ǫωi s ) n k }ds where he erminal cos, by uilizing he inegraion by pars echnique, can be presened in inegral form as e ρ (X i,u+ǫω ) G k X i,u+ǫω = (X) i G k X+ i = (X i ) G k X i +2 e ρs (X i,u+ǫω s ) G k dxs i,u+ǫω ρ d [ e ρs (Xs i,u+ǫω ) G k X i,u+ǫω e ρs (X i,u+ǫω s s ) ) G k Xs i,u+ǫω ds. (54), (53) 13

14 o wrie Ji (u i +ǫω i ) in erms of Ji (u i ), u i and Xi,u, firs (54), and hen (51)- (52) are subsiued in (53) which gives rise o Ji (u i +ǫω i ) = Ji (u i )+ǫ e ρs ( s e A k(s ) B k ω i d) ( Gk dx i,u s +(Q k Xs i,u +N k u i s+a kg k Xs i,u ρg Xs i,u η k )ds ) + ( (Xs i,u ) N k ωs+(x i s i,u ) G k B k ωs i ) ( s +(u i s) R k ωs n i kωs i ds }+ǫ 2 e ρs e Ak(s ) B k ωd ) s ( i Gk A k e Ak(s ) B k ωd i s ρg k e Ak(s ) B k ω i d+g kb k ωs i +Q k s e A k(s ) B k ω i d+n kω i s) +(ω i s ) R k ω i s) } ds. o obain he Gâeaux derivaivej i (u i ) of minor agena i s cos funcional, firs J i (u i ) is aken o he lef hand side of (55), hen boh sides of he equaion are divided by ǫ, and finally is limi as ǫ is aken, which yields ( s Ji (u i ),ω i = e ρs e Ak(s ) B k ωd ) ( i Gk dxs i,u +(Q k Xs i,u +N k u i s+a kg k Xs i,u ρg k Xs i,u η k )ds ) + ( } ) (Xs i,u ) N k ωs+(x i s i,u ) G k B k ωs+(u i i s) R k ωs n i kωs i ds. An applicaion of Fubini s heorem o change he order of inegraion in (56) gives J +B k (56) [ i (u i ),ω i = E (ω i ) e ρ B k G kx i,u +e ρ N k Xi,u +e ρ R k u i e ρ n k e ρs e A k (s ) ( G k dx i,u s + ( A kg k X i,u s ρg k X i,u s +Q k X i,u s +N k u i s η k ) ds ) } d Using he inegraion by pars echnique, we have e ρs e A k (s ) (A k G kx i,u s ds+g kdx i,u s ρg k X i,u s ) = d(e ρs e A k (s ) G k X i,u s ), (55). (57) = e ρ e A k ( ) G k X i,u e ρ G k X i,u, (58) 14

15 whose subsiuion in (57) yields [ Jk (ui ),ω i = E (ω i ) e ρ B k ea k ( ) G k X i,u +e ρ N k Xi,u +e ρ R k u i e ρ n k ) e ρs e A k (s ) (Q k Xs i,u +N k u i s η k ds }d. (59) +B k hen he ieraed expecaion propery is used o rewrie (57) as [ Jk (ui ),ω i = E (ω i ) e ρ N k Xi,u +e ρ R k u i e ρ n k [ +B ke e ρ e A k ( ) G k X i,u + ) } e ρs e A k (s ) (Q k Xs i,u +N k u i F s η k ds i d. (6) hen he following maringale is defined as M i = E [ e ρ e A k G k X i,u + and is subsiued in (6) o give e ρs e A k s (Q k Xs i,u +N k u i s η k)ds [ Ji (u i ),ω i = E (ω i ) e ρ N k Xi,u +e ρ R k u i e ρ n k +B k (e A k M i F i e ρs e A k (s ) (Q k X i,u s +N k u i s η k)ds) } d, (61). (62) According o Corollary 1, he necessary and sufficien condiion foru i, o be he opimal conrol acion for minor agena i, 1 i N, is given by J i (u i, ),ω i =, a.s. for all possible pahs of ω i, (63) which implies [ ( u i, = R 1 k N k Xi, n k +B k eρ e A k M i e ρs e A k (s ) (Q k Xs i, +N k u i, s η k )ds). (64) 15

16 Le define p i as p i = e ρ( e A k M i hen i is proposed ha he ansaz for (65) is given by e ρs e A (s ) (Q k X i, s +N k u i, s η k )ds ). (65) p i = Π k ()X i +s k (), (66) whose subsiuion in (64) yields a sae feedback form foru i, as u i, [ = R 1 k N k X i n ( k +B k Πk ()X i +s k() ). (67) o find Π k () and s k (), firs boh sides of (66) are differeniaed, and hen (45) and (67) are subsiued o give [ ) dp i = ( Πk +Π k A k Π k B k R 1 k N k Π k B k R 1 k B kπ k X i Π k B k R 1 k B ks k () ( +Π k Mk ()+B k R 1 k n k) +ṡk () d+π k Σ k ()dw i, (68) Nex, boh sides of (65) are differeniaed, which resuls in dp i = (ρpi A k pi Q kx i N ku i + η k)d+e ρ e A k dm i. (69) As per he maringale represenaion heorem, he maringale M i can be wrien as M i = Mi + Zs k dwi s, (7) and subsequenly dm i = Zk dwi. (71) hen, equaions (66), (67) and (71) are subsiued in (69), which gives rise o [ dp i = (ρπk Q k +N k R 1 k N k +N kr 1 k B k Π k A k Π ) k X i, +ρs k ()+(N k R 1 k B k A k)s k () N k R 1 k n k + η k d+qdw k, i (72) whereq k = e A k Z k. Finally, for (68) and (72) o be equal, he corresponding drifs and diffusions mus be equal. herefore, he following equaions mus hold q k = Π kσ k (), (73) 16

17 ρπ k = Π k +Π k A k +A k Π k (B k Π k +N k ) R 1 k (B k Π k +N k )+Q k, Π k () = G k, (74) ρs k () = ṡ k ()+[(A k B k R 1 k N k ) Π k B k R 1 k B k s k() +Π k (M k ()+B k R 1 k n k)+n k R 1 k n k η k, s k () =. (75) As can be seen from (74)-(75), he Riccai marixπ k () and he offse vecors k () saisfy deerminisic differenial equaions. 5.4 Mean Field Consisency Condiions o obain he consisency condiions, we subsiue (67) ino (2) which resuls in ( [ dx i = A k x i B ( ) kr 1 N k [(xi ),(x ), x ) n k +B k Πk [(x i ),(x ), x +s k k +H k x +F π k x +b k )d+σ k dw i. (76) Le define Π k = Π k,11 Π k,12 Π k,13 Π k,21 Π k,22 Π k,23 Π k,31 Π k,32 Π k,33, 1 k K, ande k = [ n n,..., n n,i n, n n,..., n n, where hen n ideniy marixi n is a hekh block. If we ake he average of (76) over subpopulaiona k, 1 k K, and hen ake he L 2 limi as he numbern k of agens wihin he subpopulaion goes o infiniy ( i.e. N k ), we ge d x k = ( F π k +[A k B k R 1 k (N k,1 +B kπ k,11 )e k B k R 1 k B kπ k,13 ) x d +(H k B k R 1 k B k Π k,12)x d+(b k +B k R 1 k n k B k R 1 k B k s k)d. (77) If we equae (77) wih (14), hen by consisency requiremen a compac descripion of he major minor mean field equaions deerminingā,ḡ, m is given 17

18 by ρπ = Π +Π A +A Π (B Π +N ) R 1 (B Π +N )+Q, Π () = G, ρπ k = Π k +Π k A k +A k Π k (B k Π k +N k ) R 1 k (B k Π k +N k )+Q k, Π k () = G k, k, Ā k = Fk π +[A k B k R 1 k (N k,1 +B k Π k,11)e k B k R 1 k B k Π k,13, k, Ḡ k = H k B k R 1 k B k Π k,12, k, (78) ρs () = ṡ ()+[(A B R 1 N ) Π B R 1 B s () +Π (M ()+B R 1 n )+N R 1 n η, s () =, ρs k () = ṡ k ()+[(A k B k R 1 k N k ) Π k B k R 1 k B k s k() +Π k (M k ()+B k R 1 k n k)+n k R 1 k n k η k, s k () =, k, m k = b k +B k R 1 k n k B k R 1 k B k s k, k. (79) Remark 2 (Finie Horizon LQG MFG Sysems). ypically, he cos funcional for finie horizon LQG MFG sysems is no discouned, i.e. ρ =, and hence he Riccai and offse equaions (43)-(44) for he major agen reduce o Π = Π A +A Π (B Π +N ) R 1 (B Π +N )+Q, ṡ () = [(A B R 1 N ) Π B R 1 B s ()+Π (M ()+B R 1 n )+N R 1 n η, (8) subjec o he erminal condiionsπ () = G, s () =. Similarly, he Riccai and offse equaions (74)-(75) for minor agen A i, 1 i N, reduce o Π k = Π k A k +A k Π k (B k Π k +N k ) R 1 k (B k Π k +N k )+Q k, ṡ k = [(A k B k R 1 k N k ) Π k B k R 1 k B k s k +Π k (M k +B k R 1 k n k)+n k R 1 k n k η k, (81) subjec o he erminal condiionsπ k () = G k, s k () =. Remark 3 (Infinie Horizon LQG MFG Sysems). For Infinie horizon LQG MFG sysems he erminal cos is se o zero and he erminal ime is se o infiniy in he major agen s cos funcionals (17) o ge J N (u,u ) = 1 2 E [ e ρ x Φ(x(N) ) 2 Q +2 ( x Φ(x(N) ) ) N u + u 2 R }d, (82) 18

19 Similarly, he discouned infinie horizon cos funcional for minor agen A i, 1 i N is given by Ji N (u i,u i ) = 1 [ 2 E e ρ x i Ψ(x (N) ) 2 Q k +2 ( x i Ψ(x (N) ) ) N k u i + u i 2 R k }d. he dynamics (1)-(2) for he major agen and minor agens remain he same in he infinie horizon LQG MFG sysems. Assumpion 4. he pair(l a,a (ρ/2)i) is deecable, and for eachk = 1,...,K, he pair (L b,a k (ρ/2)i) is deecable, where L a = Q 1/2 [I, H π and L b = Q 1/2 k [I, Hk, Ĥπ k. Assumpion 5. he pair (A (ρ/2)i,b ) is sabilizable and (A k (ρ/2)i,b k ) is sabilizable for eachk = 1,...,K. Given ha Assumpions 4-5 hold, for he major agen s sysem (1), (82), he bes response sraegy is given by (36), where he seady sae Riccai marix Π saisfies an algebraic Riccai equaion given by ρπ = Π A +A Π (B Π +N ) R 1 (B Π +N )+Q, (84) and he seady sae offse vecors saisfies he differenial equaion ρs () = ṡ ()+[(A B R 1 N ) Π B R 1 B s ()+Π (M ()+B R 1 n )+N R 1 n η. (85) Similarly, for minor agen A i s sysem (2), (83), 1 i N, he bes response sraegy is given by (67), where he seady sae Riccai marix Π k and offse marix s k saisfy he following algebraic Riccai equaion and differenial offse equaion. ρπ k = Π k A k +A k Π k (B k Π k +N k ) R 1 k (B k Π k +N k )+Q k, k, ρs k () = ṡ k ()+[(A k B k R 1 k N k ) Π k B k R 1 k B k s k()+π k (M k ()+B k R 1 k n k) +N k R 1 k n k η k, s k () =, k. (86) 6 Conclusions We used a convex analysis mehod o rederive he bes response sraegies ha yield an ǫ-nash equilibrium for he classical major minor LQG mean field game sysems addressed in [6. 19 (83)

20 References [1 I. Ekeland and R. émam, Convex Analysis and Variaional Problems. Sociey for Indusrial and Applied Mahemaics, [2 G. Allaire, Numerical analysis and opimizaion: An inroducion o mahemaical modeling and numerical simulaion. Oxford Universiy Press, 27. [3 R. Carmona, Lecures on BSDEs, Sochasic Conrol, and Sochasic Differenial Games wih Financial Applicaions. Philadelphia, PA: Sociey for Indusrial and Applied Mahemaics, 216. [4 P. Bank, H. M. Soner, and M. Voß, Hedging wih emporary price impac, Mahemaics and Financial Economics, vol. 11, no. 2, pp , 217. [5 P. Casgrain and S. Jaimungal, Algorihmic rading wih parial informaion: A mean field game approach, arxiv, 218. [6 M. Huang, Large-populaion LQG games involving a major player: he Nash cerainy equivalence principle, SIAM Journal on Conrol and Opimizaion, vol. 48, no. 5, pp , 21. [7 M. Nourian and P. E. Caines, ǫ-nash mean field game heory for nonlinear sochasic dynamical sysems wih major and minor agens, SIAM Journal on Conrol and Opimizaion, vol. 51, no. 4, pp ,