Lmtng Bhavor of LQ Dtrmnstc Infnt Horzon Nash Gams wth Symmtrc Playrs as th Numbr of Playrs gos to Infnty G.P.Papavasslopoulos Dpt. of Elctrcal and Computr Engnrng Natonal chncal Unvrsty of Athns 9 Iroon Polytchnou Str., 57 73 Athns, Grc (yorgos@ntmod.ntua.gr) ABSRAC A Lnar Quadratc Dtrmnstc Contnuous m Gam wth many symmtrc playrs s consdrd and th Lnar Fdback Nash stratgs ar studd as th numbr of playrs gos to nfnty. W show that undr som conons th lmt of th solutons xsts and can b usd to approxmat th cas wth a fnt but larg numbr of playrs. It s shown that n th lmt ach playr acts as f h wr facd wth on playr only, who rprsnts th avrag bhavor of th othrs.. INRODUCION In most paprs n gam thory thr may b fw or many playrs and solutons and proprts ar usually studd for a fxd such numbr. hr ar also paprs whr th multtud of playrs s nfnt s Rfs.8, 9 as wll as many rcnt paprs on gams playd by an nfnt numbr of automata ach on of whch ntracts wth a fnt numbr of nghbors. In th rcnt ltratur of man fld gams (Rfs. -6), stochastc gams ar consdrd wth an nfnt numbr of playrs who ar modld n a statstcal sns. hr ar also paprs whr a vrson of th problm s solvd for a fnt numbr of playrs and th valy or voluton of som proprty s studd as th numbr of playrs gos to nfnty (Rfs. 5-7).h prsnt papr blongs to ths last catgory. W consdr a fnt numbr of playrs nvolvd n a Lnar Quadratc, rmnstc, contnuous tm, nfnt horzon Nash Gam. o ach on corrsponds hs control and a part of th ovrall stat calld hs stat. h gam s symmtrc n th sns that th voluton of th part of th stat that prtans to ach playr s nfluncd n a symmtrc mannr by th controls and stats of th othrs. Fnally th cost of ach playr dpnds only on hs stat and control and t s also symmtrc for all th playrs. W mploy th Lnar Fdback Nash qulbrum concpt and study how th solutons chang as th numbr of playrs gos to nfnty. It should b notcd that th prsnt work s rlatd to rcnt work on man fld gams, Rfs. -4, and n partcular 5 and 6, but thr ar ssntal dffrncs. Frst of all our problm s rmnstc whras thos n Rfs. 5 and 6 ar stochastc. In Rf.5, Nash Opn Loop fnt tm horzon solutons ar studd for stochastc LQ Gams, whras w consdr Lnar Fdback Nash Solutons for th nfnt tm horzon that satsfy th Prncpl of Dynamc Programmng. Also Rf.6 consdrs for a stochastc LQ gam th Nash Lnar Fdback soluton lk w do, but thr th scalar cas s consdrd only whras w consdr th rmnstc st up for th matrx cas. For th partcular structur of gam chosn, w show that th lmt xsts and can b usd to approxmat th cas whr th numbr of playrs s vry larg. h conons for ths to happn ar rlatd to th xstnc and proprts of solutons of a gnralzd Rcatt quaton, whch ylds stablzng solutons for th ovrall gam. h study of an assocatd not standard Hamltonan problm s shown to b crucal for th whol analyss.h rsultng nfnt numbr of playrs cas shows that undr som conons, ach playr ssntally acts as f h wr facd wth on fcttous playr who rprsnts th avrag bhavor of all th othrs or th markt as a whol.
. PROBLE SAEEN n Lt us consdr a dynamcal systm wth stat x ( x, x,... x ). Each x R volvs as dx A B A x + ( x+ x+... + x ) + Bu + ( u+ u+... + u ) () m whr u R s th control of th -th playr. W hav playrs and as th stat quaton shows w can thnk of x as th part of th stat, or subsystm, whch prtans to th playr. h cost of th -th playr s gvn by: + J ( x Qx u u ) () h matrcs A ( n n), A ( n n), B ( n m), B( n m), Q( m m), Q Q ar ral and constant. All th x s hav th sam dmnson. Smlarly all th u s hav th sam dmnson. Notc that all th subsystms hav th sam A B A, A, B, B, Q Q and thus w hav symmtry among thm. h constants, rprsnt th xplct couplng btwn th subsystm of playr and th subsystms and controls of th othr playrs. W ar thus facd wth a many playr gam problm for whch th Nash qulbrum wll b sought. h multtud of th playrs wll b consdrd as a gvn constant. Our man objctv s to drv th solutons and study th rsultng bhavor n trms of both stat and costs as grows towards nfnty. h soluton concpt that w wll mploy s th Nash qulbrum whr th playrs us lnar stratgs n th currnt stat x ( x, x,... x ) and Dynamc Programmng holds (.. Lnar Fdback Stratgs n th parlanc of th ltratur Rfs -4). * Dfnton A st of lnar stratgs * * * * u L x,,...,, (whr th L, L,..., L ar constant m n matrcs), ar sad to b n Nash Equlbrum f for any ntal conon x() ( x(), x(),... x ()) all * * * * * * th J ( u, u,..., u, u, u+,..., u ),,,..., ar fnt and * * * * * * * * * * * J ( u, u,..., u, u, u+,..., u ) J ( u, u,..., u, u, u+..., u ), for any othr u,,,..., (quvalntly: for any othr L ). h fntnss of th costs can b warrantd by assumng that th closd loop systm s asymptotcally stabl.. th closd loop matrx has all ts gnvalus n th opn lft-hand plan. * Dfnton A st of lnar stratgs * * * * u L x,,...,, (whr th L, L,..., L ar constant m n matrcs), ar sad to b n an ε (, x()) -Nash Equlbrum f for any ntal conon x() ( x(), x(),... x ()) all * * * * * * th J ( u, u,..., u, u, u+,..., u ),,,..., ar fnt and * * * * * * * * * * * J ( u, u,..., u, u, u+,..., u ) J ( u, u,..., u, u, u+..., u ) + ε(, x()), for any othr u,,,..., (quvalntly: for any othr L ). Clarly th noton of ε (, x()) -Nash Equlbrum s of ntrst f th ε (, x()) s small and ts magntud can b qualfd n trms of ts argumnts, s Commnt 6.
h rasons for th choc of ths typ of Nash qulbrum ar svral. W know that n th corrspondng rmnstc dscrt tm framwork, many and prhaps nonlnar stratgs may xst; but f w ntroduc som nondgnrat nos n th stat quaton, thn only th lnar ons survv (arkov or Prfct qulbra, Rfs,). In th contnuous tm cas, a smlar phnomnon appars as rgularzaton du to nos, of th systm of th Hamlton-Jacob-Bllman quatons that charactrz (as suffcnt conons) th Nash closd loop no mmory solutons n th rmnstc cas. 3. NASH SOLUION h Nash soluton for ach playr s sought n th form (3),.. rstrct ourslvs to Symmtrc Lnar Stratgs. (It s known that for th nfnt tm problm consdrd hr,thr mght xst nonsymmtrc stratgs. h rason w consdr symmtrc ons s bcaus our gam s symmtrc n th way th stat quatons and th costs ar dscrbd. In adon, f on wr to consdr Lnar Fdback Stratgs for th fnt horzon cas.. th ntgrals n () wr from to a fnt, thn th Lnar Fdback Stratgs f thy xstd-thy would hav to b symmtrc, as an nspcton of th assocatd Rcatt typ quatons would show). u L x + L z (3) whr w st: z ( x + x +... + x ) (4) h L ( m n), L ( m n) ar constant matrcs. hs form s du to th symmtry of th stat quaton and th costs. h constants L, L can b rmnd from th systm of th coupld Rccat-typ quatons that charactrz th Nash soluton at hand (Rfs,3,4). Anothr but quvalnt way of rmnng thm s th followng: Lt us consdr th problm facd by playr.h ss th stat quatons () for,,...,..: dx A B A x+ ( x + x +... + x ) + Bu + ( u + u +... + u ) (5.) dx A B A x+ ( x+ x+... + x ) + Bu + ( u+ u+... + u ) (5.) dx3 A B A x + ( x + x +... + x ) + B u + ( u + u +... + u ) (5.3) 3 3...... dx A B A x + ( x+ x+... + x ) + Bu + ( u + u+... + u ) (5.) whr h consdrs n (5.)-(5.): u L x + L z, for,,3,... (6) Addng up (5.)-(5.) ylds: d( x+ x+... + x ) A ( x+ x+... + x ) + + A ( x + x +... + x ) + B ( u + u +... + u ) + + B ( u + u +... + u ) ( A + A )( x+ x+... + x ) + ( B + B )( u + u+... + u ) or 3
dz ( A + A ) z+ ( B + B ) ( u + u+... + u ) (7) Addng up th quatons (6) for,3,..., ylds: u + u +... + u L ( x + x +... + x ) + ( ) L z 3 3 L ( z x ) + ( ) L z L x + [ L + ( ) L ] z (8) Substtutng u+ u3 +... + u from (8) nto (5.) ylds: dx A B A x+ ( x+ x+... + x ) + Bu + ( u+ u+... + u ) B B A x+ A z+ ( B + ) u + ( u+... + u ) B B A x+ A z+ ( B + ) u + { L x+ [ L + ( ) L ] z} B B ( A L ) x+ ( A + BL + B L ) z+ ( B + ) u Substtutng u+ u3 +... + u from (8) nto (7) ylds dz ( A + A ) z+ ( B + B ) ( u + u+... + u ) ( A + A ) z+ ( B + B) u+ ( B + B) { L x+ [ L + ( ) L ] z} ( B + B ) L x + [ A + A + ( B + B )( L + L )] z+ ( B + B ) u or d x x A Bu + z z Whr (9) 4
A BwL A + B ( L + ( w) L ) A ( B B ) wl A A ( B B )( L ( w) L ) + + + + + A A B ( L + L ) B L B L w A A + ( B B )( L L ) + ( B B ) L ( B B) L + + + + + () A A B B + [ L + L] w A + A B + B [ L L] B B + A A B I B + [ L L] w [ L L] A + A B + B I B + B B + Bw B ( B + B) w w () It s clar now that th problm of playr s to mnmz hs cost subjct to th quatons (9).h rason s that th stat x and th cost Jof th playr ar nfluncd by x, u and z on whch z th total nflunc of u s through (7).h soluton s gvn by th formula: x u ( B Bw) ( B B ) w K + + z () Whr K K K K K (3) s th soluton of th matrx Rcatt quaton: B + B w Q KA+ A K+ Q K ( B + Bw) ( B + B) w K, Q ( B + B) w (4) h L, L that ar prsnt n th Rcatt quaton (4), through th matrx A of () ar to b dntfd wth [ ] ( ) L L B + B w ( B + B ) w K (5) Substtutng n () th L, L, by thr quals usng (5) w obtan for A th quvalnt form (6) that w dnot by A( K, w ) n ordr to mphasz th dpndnc on K and w : 5
A A B I B A( K, w) [ L L] w [ L L] A A + B B I B B + + + A A B I B ( B Bw) ( B B ) w K w ( B Bw) ( B B) w K A A B B + + I + B B + + + + + A A B I B I A + A ( ) B + B B K w B B B K I B B + I + + B B + w B K w B ( B B ) K B B + + B B + + (6) It s ths A( K, w ) of (6) that s usd n (4). Notc that (4) has svral quadratc trms n K bsds th last on apparng n (4), snc A( K, w ) tslf s a lnar functon of K.h soluton of (4) f t xsts, s a functon K( w ) whch has th valu K () for w. Snc w ar gong to lt, w wll study th bhavor of th Rcatt quaton (4) by allowng w to b a contnuous varabl clos to. Lt us st: B + Bw R( K, w) KA( K, w) + A ( K, w) K + Q K ( B + Bw) ( B + B ) w K ( B + B ) w (7) h functon R( K, w ) s analytc n ts argumnts. Lt us st K w K + K w+ K w + (8) ( )... W plug K( w) K+ Kw+ Kw +... n (7) and group togthr th trms corrspondng to th sam powrs of w to obtan B + Bw R( K( w), w) K( w) Α ( K( w), w) +Α ( K( w), w) K( w) + Q K( w) ( B + Bw) ( B + B ) w K( w) ( B + B ) w R ( K ) + wr ( K, K ) + w R ( K, K, K ) +... + w R ( K, K,..., K ) +... (9) n n n Whr A A BB I BB R( K) K{ K } + {...} K + Q K K A A ( B B ) B I () + + 6
A A BB I R ( K, K) K{ K } + {...} K + A + A ( B + B ) B ) I B I K { B K } {...} K + B B I BB BB K K K K+ Quadratc trms of( K) () Smlarly w can drv th formula for th othr Rn ( K, K,..., K n ) s. Lt us consdr th matrx: A A BB I BB Ac () K K A A ( B B ) B ) I + + A A BB K K I BB K K A A ( B B) B ) K K I K K () + + A BB K A BB ( K+ K) BB K Ac Ac A A ( B B ) B ( K K) A + + + c It holds: A A B B K c Ac A + A ( B + B ) B ( K+ K) (3) Ac A BB ( K+ K) BB K Ac Ac Ac hn A B B K A BB ( K+ K) B B K BB R ( K) K + [..] K + Q + K K A + A ( B + B ) B ( K+ K) K K A BB K A BB ( K+ K) BB K B B + [.. ] K + Q K K K K + A + A ( B + B ) B ( K+ K) K( A BB K) K( A BB ( K+ K) BB K) + K( A + A ( B + B ) B ( K+ K)) [...] + + K ( A BB K) K ( A BB ( K+ K) BB K) + K( A + A ( B + B ) B ( K+ K)) Q+ KB B K KB B K + K B B K K BB K (4) Lt us ntroduc th oprator L( K, X ) whch s actually th drvatv of R( K, w ) wth rspct to K calculatd at w. For 7
χ χ Χ χ χ w dfn: A A BB I BB L( K, X ) X{ K K} {...} X A A ( B B ) B ) I + + + + B I B I K B X ( K B X ) B + B I B + B I XA () + A () X + c ( KBB + K( B + B ) B )( χ+ χ) ( KBB + K( B + B ) B )( χ+ χ) ( K BB + K( B + B ) B )( χ+ χ) ( K BB + K( B + B ) B )( χ+ χ) χ χ Ac Ac ( KBB + K( B + B ) B )( χ+ χ) ( KBB + K( B + B) B )( x+ x) + A () X χ χ A c ( K BB + K( B + B ) B )( χ+ χ) ( K BB + K( B + B ) B )( x+ x) χ A χ A + χ A ( K B B + K( B + B ) B )( χ + χ) L ( K, X ) L ( K, X ) + K, X ) L ( K, X ) c c c [...] χ A ( ( ) )( ) L( c χ Ac + χ Ac K B B + K B + B B χ + χ Equvalntly, th oprator L( K, X ) can b wrttn as a matrx oprator multplyng a vctor : L ( K, X ) χac + A cχ L ( K, X ) L ( χ) χ Ac + ( A Y ( B + B ) B ) χ L ( K, X ) L3 ( χ) L3 ( χ) χ Ac + A cχ whch maks th nvrtblty study of L( K, X ) qut transparnt. W can now stat th followng proposton: Proposton If at w th quaton R( K,) has a soluton K,.. R( K,) and f th oprator L( K, X ) as a lnar oprator on X s nvrtbl,thn n som nghborhood of w th quaton R( K, w ) has a unqu soluton K( w) whch s an analytc functon of w and has an xpanson (8), whr th K, K, K,... ar th unqu solutons of R( K), R ( K, K), R ( K, K, K),... Proof h proof of ths proposton s a straghtforward applcaton of th mplct functon thorm for analytc functons (s horm 8.6 nrf.) and uss th formula alrady dvlopd n (7)-(7). It s mportant to notc that f w want to fnd th n w n th quaton ( ( ), ) (7) (5) (6) K n s for n w hav to fnd th coffcnt of th powr K of th form: R K w w and st t qual to,whch ylds an quaton lnar n n 8
L( K, Kn) F( K, K,..., Kn ),whr th lnar oprator L( K, K n) s th on dfnd n (6) and F( K, K,..., K ) s a nonlnar functon contanng multplcatv trms of ts argumnts. n Lt us now ntroduc th closd loop matrx Ac ( w ) by th formula: A A B w Ac ( w) A( K, w) ( B Bw) ( B B ) w K( w) A A B B + + w w + + + (8) B + Bw ( B + Bw) ( B + B ) w K( w) ( B + B ) w hs s th closd loop matrx of (9) that rsults whn all th playrs us thr optmal stratgs and w wll hav: d x x Ac ( w) z z (9) W can now stat th scond proposton that prtans to th xstnc of a Nash qulbrum. Proposton. If th Rcatt quaton R( K, w ) has a soluton K( w) whch ylds an asymptotcally stabl closd loop * * * matrx Ac ( w ), thn th rsultng u of (5) and th smlar u,..., u ar n Nash qulbrum.. If th Rcatt quaton R( K ), whr R( K) s gvn n () has a soluton K and th matrx Ac () of () (or from (8) wth w ) s asymptotcally stabl, and th oprator L( K, X ) s nvrtbl, thn for suffcntly larg, all th gams consdrd hav a Nash qulbrum wth asymptotcally stabl closd loop matrcs Ac ( w ) (whch ar as n(8)) and tnd to A c() as.h corrspondng K( w) whch rmns * th u can b approxmatd up to ordr on n w by K+ Kw,whr K, K solv th two quatons R ( K ), R ( K, K )..3 h L, L that ar calculatd by usng (5) wth K and w consttut an ε (, x()) -Nash qulbrum. Proof h proof of ths Proposton s an mmdat consqunc of th prvously prsntd analyss. Commnt h quaton R( K ) rsults n th systm: K A + A K + Q K B B K K ( A B B ( K + K) B B K) + K( A + A ( B + B ) B ( K + K)) + ( A B B K ) K+ K B B K K ( A B B ( K + K) B B K) + ( A B B ( K + K) B B K) K + K ( A + A ( B + B ) B ( K + K)) + ( A + A ( B + B ) B ( K+ K)) K 9
Substtutng th scond on wth th sum of th frst two w hav th followng quvalnt systm that K, K + K, K hav to satsfy: K A + A K + Q K B B K ( K + K)( A + A ) + A ( K+ K ) ( K+ K )( B + B ) B ( K+ K ) + Q K ( A B B ( K + K) B B K) + ( A B B ( K + K) B B K) K + K ( A + A ( B + B ) B ( K + K)) + ( A + A ( B + B ) B ( K + K)) K or K A + A K + Q K B B K (3) Y ( A + A ) + A Y Y ( B + B ) B Y + Q K Y K c c c c K A + A K + K A + A K (3) (3) Whr A A B B K c A A + A ( B + B ) B Y c A A A c c c (33) It s clar that th gan K that multpls th stat x of playr dpnds only on hs part of th systm and cost. h Rcatt quaton (3) for K s th classcal on and th K xsts and s postv dfnt undr th usual assumptons. h gan K that coupls th controllr of th playr wth th stats of th othrs may fal to xst snc ts xstnc dpnds on th gnralzd Rcatt (3).Notc that th study of (3) s rducd quvalntly to th study of th Prturbd Hamltonan A + A ( B + B ) B A BB A BB Η + (34) Q A Q A If Y satsfs (3), thn: I A + A ( B + B ) B I A + A ( B + B ) B Y ( B + B ) B Y I Q A Y I Y ( A + A ) A Y + Y ( B + B ) B Y Q A + Y ( B + B ) B A + A ( B + B ) B Y ( B + B ) B A + Y ( B + B ) B (35) and thus th matrcs A ( ) c A + A B + B BY, ( A Y ( B + B ) B ) ) hav th gnvalus of H and can b calculatd by usng th corrspondng gnvctors of H, s Rf.9. Notc that n ordr for th Nash gam to hav a soluton w nd both Ac, A c to b asymptotcally stabl, a fact that wll hold for Ac f A, B s a controllabl par. h conons for th asymptotc stablty of A c ar lss
obvous and dpnd on th gnstructur ofη. Actually thr ar cass whr thr s no soluton rsultng n A c asymptotcally stabl, or thr can b mor than on soluton that yld A c asymptotcally stabl whch would man corrspondngly that w hav no or many Nash qulbra. Fnally notc that solvng for K s a lnar problm that always has a soluton f Ac s asymptotcally stabl. hus th xstnc of a Nash soluton amounts to studyng (3) and (3) and dmandng that both Ac, Ac ar asymptotcally stabl. Commnt h nvrtblty of L( K, X ) s quvalnt to havng nvrtblty of th oprators: L ( χ ) χ A + A χ c c L ( χ) χ A + ( A B B K B ( B + B ) K ) χ c c L ( χ ) χ A + A χ 33 c c h frst and thrd on ar nvrtbl f Ac, Ac ar asymptotcally stabl. It s th nvrtblty of th scond on that s th mor ntrstng. Lt us look at t mor carfully: χ Ac + ( A B ( B + B ) ( K+ K )) χ (37) χ( A + A ( B + B ) B ( K + K)) + ( A ( K + K)( B + B ) B ) χ hs quaton nvolvs th matrcs A ( ) c A + A B + B BY and ( A Y ( B + B ) B ) that appar n (3) (or (35)), and thus w hav nvrtblty of L( K, X ) f and only f : (36) gnvalua + gnvalu( A ( K + K)( B + B ) B )) (38) c Notc also that quaton (37) s actually th prturbaton of th quaton that (3) that gvs Y K+ K. o s that, lt us prturb th soluton Y of (3) to Y + to gt: ( Y + )( A + A ) + A ( Y + ) ( Y + )( B + B ) B ( Y + ) + Q Y ( A + A ) + A Y Y ( B + B ) B Y + Q+ ( A + A ( B + B ) B ( K + K)) + ( A ( K + K)( B + B ) B ) + ( B + B ) B ( A + A ( B + B ) B ( K + K)) + ( A ( K + K)( B + B ) B ) ( B + B ) B (39) hrfor th conon for nvrtblty of L( K, X ) s quvalnt to askng that th prturbaton of th quaton that rmns Y K+ K has no zro gnvalu. If w dmand mor than that, namly that all th prturbaton gnvalus of th prturbd quaton fory K+ K ar ngatv,..: gnvalu( A + A ( B + B ) B ( K+ K)) + gnvalu( A ( K+ K)( B + B ) B )) < (4) or
gnvalua + gnvalu( A ( K + K)( B + B ) B )) < c ths would mply a knd of stablty of th solutons of (3) that s bnfcal for any algorthm that solvs (3). Actually ths s quvalnt to somthng mor ntrstng, namly t s quvalnt to askng that th nfnt tm solutons,.. th L gans, can b approachd as stabl lmts of th gans of th fnt tm horzon Nash gam. * Such a Nash qulbrum w wll call Stabl Nash qulbrum. (W rmnd th radr that a soluton of th nfnt horzon problm s not ncssarly a lmt of th fnt horzon soluton. In rlaton to that and th ntrplay btwn fnt tm horzon and Infnt m horzon solutons, srfs.7,8, 3, 4). h justfcaton of ths clam follows. Consdr th cost: t f f f f + + J x ( t ) Q x ( t ) ( x ( t) Qx ( t) u ( t) u ( t)) (4) for som Q f. h stat quatons rman th sam and all th transformatons n ()-() hold. h fnt tm Rcatt quaton ylds a tm varyng soluton whch satsfs: dk( w, t) Q f R( K( w, t), w), K( w, t f ) Lt K( w, t) K( w, t) K( w, t) K( w, t) K( w, t) Assumng w w s that th followng dffrntal quatons ar obtand : dk( t) K( t) A + A K( t) + Q K( t) B B K( t), K( t f ) Q f (4a) dy ( t) Y ( t)( A + A ) + A Y ( t) Y ( t)( B + B ) B Y ( t) + Q, Y ( t f ) (4b) K( t) Y ( t) K ( t) dk( t) K( t) Ac + Ac K( t) + K ( t) Ac + Ac K( t) + K ( t) BB K( t), K( t f ) (4c) Ac ( t) A + A ( B + B ) B Y ( t)) h soluton of (4a) gos to th postv dfnt stablzng soluton of (3) undr th usual controllablty assumptons on A, B.h dffrntal quaton (4b) has as qulbrum pont th soluton Y of (3).Lnarzng (4b) around ths Y w gt th lnarzd quaton (39), and thus w conclud that th Y s an asymptotcally stabl qulbrum of (4b) f and only f: gnvaluac + gnvalu( A ( K+ K)( B + B ) B )) <.hrfor, f gnvalua c< and gnvalua c < and gnvaluac + gnvalu( A ( K+ K)( B + B ) B )) < w wll hav a Nash qulbrum of th nfnt horzon LQ gam that can b also consdrd Stabl n th sns that s th lmt of a fnt tm horzon qulbrum. Lt us now summarz th matral of Commnts and 3 n th form of a Proposton. Proposton 3
3. h Rcatt quaton R( K ) or quvalntly th systm (3)-(3) has a soluton that rsults n asymptotcally stabl Ac, A c f th par A, B s compltly controllabl and f th quaton (3) has a soluton Y that maks th A c asymptotcally stabl. hs last rqurmnt s quvalnt to askng that th H matrx has n ngatv gnvalus λ, λ,..., λn whos gnvctors hav thr uppr n-dmnsonal parts lnarly ndpndnt, and th othr n gnvalus λn+, λn+,..., λn satsfy: λ λ j,,,..., n, j n+, n+,...,n.h gnvctors corrspondng to λ, λ,..., λ n ar usd to construct Y (s Rf.9). 3. If th assumpton of 3. hold and n adon th gnvalus satsfy: λ < λ j,,,..., n, j n+, n+,...,n, thn th Nash Equlbrum s a Stabl on n th sns dlnatd abov. Proof h proof s actually gvn n Commnts and 3. W us th classcal constructon of Rf.9. As vdncd from (37), H has th gnvalus of Ac and th ngatv gnvalus of A ( K+ K)( B + B ) B A Y ( B + B ) B. Commnt 3 h study of H s qut cntral to th xstnc and charactr of th Nash soluton and as such t mrts ndpndnt nvstgaton. Frst of all t s clar from (34) that H s qual to th classcal Hamltonan whch corrsponds to th classcal Rcatt (3), and snc th gnvalus ar contnuous functons of th matrx ntrs, f w thnk of A, B as prturbatons, thn for suffcntly small valus of thm th assumptons and thus th conclusons of Proposton 3 hold. hus t s asy to produc suffcncy nonmpty conons for Proposton 3 to hold. Notc also that H s not Hamltonan n th sns ncountrd n th Lnar Quadratc Control hory and th study of th classcal Rcatt quaton, s Rfs.3, 4, and 9 and t can b any arbtrary n n matrx as th chocs of A, A, B, B can produc any valu for th trms thy rmn n H. h Q trm of H sms to ntroduc a rstrcton snc t s symmtrc and postv smdfnt. Actually any quadratc matrx quaton of th form HY + YH + YH Y + H can b transformd to havng H symmtrc and postv smdfnt by usng H UDV th sngular valu dcomposton of H,wth D dagonal and postv smdfnt. Pr- and post- multplyng th quaton wth U and V rspctvly and consdrng as nw unknown thy U YV, w hav a nw quadratc matrx quaton wth symmtrc postv smdfnt constant trm. hus th Q trm dos not rally provd any structur and thus th H matrx w dal wth hr dos not hav any partcular structur. Commnt 4 h undrlyng thm of ths work s th prsnc of vry many symmtrc playrs, and thrfor th lmtng bhavor as th numbr of playrs grows to nfnty s consdrd. It should b notcd that nowhr dd w rfr to a gam wth an nfnt numbr of playrs, or to K as dfnng th Nash qulbrum stratgy of a playr n th prsnc of an nfnt numbr of playrs. Nonthlss f on wr to mak sns of such a lmt pr s-as f th playrs ar nfnt n multtud, th quatons (9) ar of mportanc. Notc by th way, that w drvd (9) whr th varabl z concatnats th nflunc of all th othr playrs on playr on, and drvd thn th Nash qulbrum, although w could hav drvd th Nash qulbrum workng drctly on quatons (5.)-(5. ).Lt us look mor carfully at quatons (9), rstatd blow for convnnc. If w consdr that th L, L hav valus that convrg to dfnt lmts as, a fact that holds undr th assumptons of Proposton, thn th systm (9) n th lmt bhavs lk: 3
dx A x + ( A + L + L ) z+ u, u BB Kx (43) dz ( A A ( B B ) B ( K K)) z + + + (44) Equvalntly w can say that playr on facs a control problm of mnmzng J (as n ()), whr hs control s u, and hs stat x s nfluncd by a stat varabl z whch z s avalabl to hm but s not at all nfluncd by hm: h stat z obys th voluton quaton (44) whch s not at all nfluncd by x or u,as should b xpctd, snc on playr on hs own should not hav any nflunc on th collctv bhavor of a nfnt numbr of fllow playrs. h quston s whthr w can thnk of th quaton (44) as rsultng from a control problm wth stat quaton: dz ( A + A ) z + ( B + B ) u (45) and cost to b mnmzd + + mn J ( ( z ( t) Q z ( t) u ( t) S z ( t) u ( t) u ( t)) (46) whch for approprat chocs of Q S, has as soluton: u B ( K+ K)) z (47) (Notc that s a partcular cas of an nvrs problm and such problms hav bn studd n th past n mor gnralty,s for xampl:rfs, ). h stat z and th control u, can b thought of as th collctv stat and control of th othr nfnt n multtud playrs that rmn th bhavor of th markt facd by playr on. Of cours t s assumd that: Q S Q S S S I h soluton of ths problm s: u [( ) B + B Ρ+ S ] z whr Ρ solvs th Rcatt quaton: Ρ ( A + A ( B + B ) S ) + ( A + A ( B + B ) S ) Ρ+ Q S S Ρ ( B + B )( B + B ) Ρ Q, S hav to b chosn so that th optmal control soluton (47) has th valu u B ( K+ K)) z,.. ( B + B ) Ρ+ S B ( K+ K) or S B Y ( B + B ) Ρ Substtutng S wth t s qual from abov w gt: Ρ ( A + A ) + ( A + A ) Ρ+ Q ( B Y ) B Y hrfor n ordr that th problm (45)-(46) has th soluton (47), w should choos: Q Q, S, Q S S, ΡΡ (48) so that th followng hold: S B Y ( B + B ) Ρ (49) 4
Q Ρ ( A + A ) ( A + A ) Ρ+ ( B Y ) B Y (5) whr Y solvs Y ( A + A ) + A Y Y ( B + B ) B Y + Q (5) hs conons gv Q, S, Ρ as functons of th paramtrs A, A, B, B, Q and thy can always b satsfd as th followng choc shows: P, S B Y, Q ( B Y ) B Y Wth ths choc th cost (46) s: Y B B Y Y B B Y I z( t) z( t) mn J u ( ) ( ) t u t whch rsults n optmal valu snc u B ( K+ K)) z Sz A spcal possbl choc s to hav S and thn fndng Ρ, Q may b or not b possbl. For xampl, lt B so that w hav couplng of th stat quatons of th playrs only through th A trm. W can tak, Y Ρ, f Y Y, whch ylds: S Q Y ( A + A ) ( A + A ) Y + Y B B Y Q Y ( A + A ) A Y + YB B Y or Q Q+ A Y h cas Y Y can occur for xampl f A Y YA (a fact that cannot b guarantd a pror) snc thn th quaton for Y can b wrttn as Y ( A + A ) + ( A + A ) Y + Q YBB Y and obvously has a symmtrc soluton. For xampl f th couplng matrx A ai wth a a ral scalar, ths s th cas and Q Q+ AY Q+ ay s accptabl f t s postv smdfnt. Commnt 5 It would b mportant to consdr whthr K, K, or K ndfnt.for xampl, f t s postv dfnt, ths mans that as th numbr of playrs gos to nfnty th cost of ach playr at th Nash qulbrum s dcrasng. Lt us consdr th quaton: R ( K, K ) whch can b wrttn as: L ( K, K) χac + A cχ r L ( K, K) L ( χ) χ Ac ( A YB ( B B ) ) χ r + + (5) L ( K, K) L3 ( χ) L3 ( χ) χ Ac + A cχ r Whr K K χ χ K K K χ χ 5
r r B I I K { B ( B B ) K B K } {...} B ( B B ) K r r + + + + B B I I + B B + BB B ( B + B ) + K K ( B + B ) B (53) It holds: B I I r [ I ]{ K { B ( B B ) K B K } {...} B ( B B ) K B B + I + I + + + B B + BB B ( B + B ) I + K K} ( B + B ) B hs s asy to prov by just carryng out th multplcaton usng th rght hand sd of (53). hrfor to fnd K, on has to solv: K A + A K c c and snc w want an asymptotcally stabl closd loop systm t wll b that all th gnvalus of A c ar ngatv, and thus K.hrfor th K matrx wll ncssarly hav th form K K K K whch s ndfnt, xcpt f also K whch s not ncssary.w s that as th numbr of playrs gos to nfnty, th cost of ach playr dos not bhav monotoncally,although th part of th cost that dpnds on hs ntal conon x () rmans constant up a frst ordr of w,..: K x() J( w) [ x() z() ] ( K+ w O( w )) + K z() K (54) hs s not tru for th cost of th concatnatd playr who ss as hs stat z. h cas whr K s worthy xamnng snc t would mply that th cost J ( ) w bhavs monotoncally n w up to ordr two. hs holds only for vry spcal valus of th paramtrs of th gam, and f t wr to hold,t would man that th playrs hav th cost: x() () x J( w) [ x() z() ] K w z () Kz() O( w ) z() + + z() whr th frst ordr wth rspct to w chang of th cost dpnds only on th avrag stat z Commnt 6 h ε (, x()) of th ε (, x()) -Nash Equlbrum dfnd arlr nds to b furthr qualfd.from th analyss for K, K w s that: x() x() ε (, x()) w K w x () w x () 6
hs maks manngful th noton of ε (, x()) qulbrum, snc for vry larg, or quvalntly for vry small, th approxmat Nash qulbrum approachs th xact on. Notc that f nstad of usng n (5) K and w for calculatng L, L w us th K + wk,w ar gong to hav agan an ε (, x()) -Nash qulbrum whr w wll hav a bttr approxmaton of ordr w. w Commnt 7 h analyss of th prsnt papr can b asly xtndd to th cas whr nstad of th costs () w hav costs of th form: x ( Q Q x j j j j 3 k ) z Q z Q + + + + j j j< k, j, k J u u u S u u S u u S u whr th approprat postv (sm)dfntnss assumptons ar mad. Notc that ths form prsrvs th symmtry and thus th solutons ar agan of th form (3). 4. A SCALAR EXAPLE In th modl ()-() w us A a, A a, B, B b, Q q, Q f q f,all scalars. Each x volvs as dx a x + a ( x + x +... + x ) + u b + ( u + u +... + u ) h cost of th -th playr s gvn by: ( ) + J qx u h Nash soluton for ach playr wll b of th form: k k x k k u lx + lz [ + bw,( + bw], K k k z k k Whr K s th soluton of th matrx Rcatt quaton R( K, w ), a a b a k a b( k+ k) k λ λ Ac () K _ K a a b a a ( b)( k k) λ + + + + + A λ, A λ, A λ c c c h quaton R( K ) rsults n th systm: k + a k + q ( b) y ( a a) y q + + + + k[ a by] k + k[ a + a ( + b) y] y k+ k h oprator L( K, X ) can b wrttn as a matrx multplyng a vctor: (55) 7
χ λχ λ χ Λ χ λχ + λ χ ( k+ b( k+ k))( χ + χ) λ ( k+ b( k+ k)) a ( + b) y χ χ λχ + λχ ( k + ( k + k))( χ+ χ) ( k + ( k + k)) λ ( k + ( k + k)) λ χ h mportant quantts ar: λ a k λ a + a ( + b) y λ a ( + b) y Solvng (55) w hav obvously only on accptabl root forλ < and two possbl roots forλ : k a + a + q λ a + q ε y k+ k ± ( + b) a + a+ ( a+ a) + 4 q( + b), ε a ε a + a + q + b λ ( ) 4 ( ) a ε a + a + q + b λ For havng λ ral w nd: ( a + a) + 4 q( + b) ( ) 4 ( ) For th Proposton to hold w nd: λ, λ, λ + λ. For Proposton to hold w nd: λ <, λ <, λ + λ.lt us focus on th conons for Proposton Cas I. For th closd loop systm to b asymptotcally stabl and L( K, X ) nvrtbl w nd: λ <, λ <, λ + λ h conon λ + λ amounts to ( a a) 4 q( b) + + + >.W can hav two accptabl λ <, f a ( a + a) + 4 q( + b) < < a.w can hav only on accptabl λ f a( a + a) + q( + b) > (56a) (56b) Cas II. For th closd loop systm to b asymptotcally stabl, L( K, X ) nvrtbl and th Nash soluton to b Stabl w nd : λ <, λ <, λ + λ < h conon λ + λ < amounts to ε and ( ) 4 ( ) a a + a + q + b λ ( a a) 4 q( b) + + + >.hrfor f th soluton 8
s ngatv t corrsponds to an asymptotcally stabl closd loop matrx and th corrspondng Nash soluton s ( ) 4 ( ) a+ a + a + q + b Stabl. If th othr soluton for λ s also ngatv, t corrsponds to anothr Nash soluton that rsults to an asymptotcally stabl closd loop matrx, but ths Nash soluton s not Stabl. Accordng to Commnt 4, w xamn th problm: mn ( ( ) ( ) ( ) ( ) ) + + J q z t s z t u t u t subjct to: dz ( a + a) z + ( + b) u whr t must b q s and+ b.it has as soluton: u [( + b) ρ+ s ] z whr ρ s th postv soluton of th Rcatt quaton: ρ( a + a ( + b) s ) + q s ρ ( + b) W want q, s chosn so that : ( + b) ρ+ s k+ k or + + ( + ( + ) ) + ( + ) ( ) a a a a b s b q s y k+ k + b Rcall that y k+ k satsfs (56a). Equatng th two xprssons for ( k+ k) w fnd: ( a + a ( + b) s ) + ( + b) ( q s ) a+ ε ( a + a) + 4 q( + b) λ a + a Notc that th lft hand sd has mnmum valu zro (achvd for s, q s ) and can achv any + b postv valu for appropratly chosn q s q s.h rght hand sd s achvd f th root λ < whch wll,, happn for th roots that yld an asymptotcally stabl closd loop matrx.. always whn w hav a Nash qulbrum. 5. CONCLUSIONS In th prsnt papr th lmtng bhavor of a dynamc Nash gam was studd wth rspct to th numbr of playrs gong to nfnty. Smlar qustons for th dscrt tm LQ Nash gam and for th Stacklbrg qulbrum can b consdrd. h stochastc vrson of th problm studd hr can also b consdrd for th cas whr th stratgy u s lnar n th stmat of th avrag (markt) stat z and th stmat of th stat x. Of ntrst would b hr to xamn n th sprt of Rf.5 undr what conons bttr masurmnts (for xampl lss masurmnt nos) ar bnfcal for th fnt tm horzon cass as th horzon ncrass and or as wll as whn th numbr of playrs dos. 6.ACKNOWLEDGEEN 9
hs rsarch has bn cofnancd by th Europan Unon (Europan Socal Fund ESF) and Grk natonal funds through th Opratonal Program Educaton and Lflong Larnng of th Natonal Stratgc Rfrnc Framwork (NSRF) - Rsarch Fundng Program: HALES. Invstng n knowldg socty through th Europan Socal Fund and th program ARISEIA, projct nam HEPHAISOS. 7. REFERENCES..Basar and G.J.Olsdr Dynamc Noncoopratv Gam hory, SIA nd on 999.. D.Fudnbrg and J.rol Gam hory, I Prss 99. 3. H.Abou-Kandl,G.Frlng,V.Ionscu and G.Jank. atrx Rccat Equatons n Control and Systms hory,brkhausr 3. 4. J.Engwrda. LQ Dynamc Optmzaton and Dynamc Gams, John Wly&Sons 5.. 5.. u and G.P.Papavasslopoulos On th Informatonal Proprts of th Nash Soluton of LQG Dynamc Gams," IEEE rans. on Automatc Control, Vol. AC-3, No. 4, Aprl 985, pp. 377-385. 6..Drshr, Probablty of a Pur Equlbrum Pont n n-prson Gams, Journal of Combnatoral hory, Vol. 8, pp. 34-45, 97. 7. G.P.Papavasslopoulos On th Probablty of Exstnc of Pur Stratgy Equlbra n atrx Gams," JOA, Vol. 85, No., Novmbr 995, pp. 49-439, and JOA, Vol. 9, No. 3, 996. 8. R. J. Aumann arkts wth a Contnuum of radrs, Economtrca Vol. 3, No. /, Jan. - Apr., 964 9. J. W. lnor and L. S. Shaply Valus of Larg Gams II: Ocanc Gams athmatcs of Opratons Rsarch Vol. 3, No. 4, Nov. 978.. Huang, R. P. alhamé and P. E. Cans Nash Equlbra for Larg-Populaton Lnar Stochastc Systms of Wakly Coupld Agnts Analyss, Control and Optmzaton of Complx Dynamcal Systms 5, II, 5-5.. Huang, R.P. alhamé, and P.E. Cans "Larg populaton stochastc dynamc gams: closd-loop ckan- Vlasov systms and th Nash crtanty quvalnc prncpl". Communcatons n Informaton and Systms, vol. 6, no. 3, pp. -5, 6. [pdf].. J.-.Lasry and P.-L. Lons, (6). Jux a champ moyn I - L cas statonnar, Compts Rndus d l'acadm ds Scncs, Srs I, 343, 69-65. 3.. J.-.Lasry and P.-L. Lons, (6). Jux a champ moyn II. Horzon fn t contrôl Optmal, Compts Rndus d l'acad_m ds Scncs, Srs I, 343, 679-684. 4.. J.-.Lasry and P.-L. Lons, (7). an fld gams. Japans Journal of athmatcs, (), 9-6. 5. A. Bnsoussan, K. C. J. Sung,S. C. P. Yam, and S. P. Yung, Lnar-Quadratc an Fld Gams,
6..Bard, Explct solutons of som Lnar-Quadratc an Fld Gams, 7. E. agrou, Valus and Stratgs for Infnt m Lnar Quadratc Gams, IEEE-AC, August 976, Vol., No. pp547-55 8. D. Jacobson, On valus and stratgs for nfnt-tm lnar quadratc gams, IEEE-AC, Jun 977, Vol., pp. 49-49 9. J.E.Pottr, atrx Quadratc Solutons, J.SIA Appl. ath.,vol 4,No.3, ay 966,pp. 496-5. L.Kaup and B.Kaup, Holomorphc Functons of Svral Varabls, d Gruytr Studs n athmatcs 3, Waltr d Gruytr, Brln-Nw York,983. R.Kalman, Whn s a Lnar Control Systm Optmal?,ASE ransactons, Journal of Basc Engnrng,Vol.86,pp.56-57,964. A.Jamson and E.Krndlr, Invrs Problm of Lnar Optmal Control, SIA Journal on Control, Vol., pp. -9, 973.