t,k E t u k +f t,k , Xt t = x, k T t, t T,

Transcription

1 MIXED EQUILIBRIUM SOLUION OF IME-INONSISEN SOHASI LQ PROBLEM YUAN-HUA NI XUN LI JI-FENG ZHANG AND MIROSLAV KRSI Abstract. In ths paper we propose a nove equbrum souton noton for the tme-nconsstent stochastc near-quadratc probem. hs noton s caed the mxed equbrum souton whch conssts of two parts: a pure-feedbac-strategy part and an open-oop-contro part. When the purefeedbac-strategy part s zero or the open-oop-contro part does not depend on the nta state the mxed equbrum souton reduces to the open-oop equbrum contro and the feedbac equbrum strategy respectvey. Usng a maxmum-prncpe-e methodoogy wth forward-bacward stochastc dfference equatons a necessary and suffcent condton s estabshed to characterze the exstence of a mxed equbrum souton. hen by decoupng the forward-bacward stochastc dfference equatons three sets of dfference equatons whch together portray the exstence of a mxed equbrum souton are obtaned. Moreover the case wth a fxed tme-state nta par and the case wth a the nta pars are separatey nvestgated. Furthermore an exampe s constructed to show that the mxed equbrum souton exsts for a the nta pars athough nether the open-oop equbrum contro nor the feedbac equbrum strategy exsts for some nta pars. Key words. tme nconsstency stochastc near-quadratc optma contro mean-fed optma contro forward-bacward stochastc dfference equaton equbrum souton. AMS subect cassfcatons. 93E Introducton. In ths paper we consder a cass of mean-fed stochastc near-quadratclq for short contro probems. he system dynamcs are descrbed by the foowng dscrete-tme stochastc dfference equaton S E for short X1 t = A t X t ĀtE t X t B tu B t E t u f t =1 t X t t E tx t D t u D t E tu dt w 1.1 Xt t = x t t where = {0...N 1} t = {t N 1} and A t Āt t t Rn n B t B t D t D t Rn m f t d t Rn are determnstc matrces and {X t t } X t and {u t } u wth t = {t1...n} are the state process and contro process respectvey. he nose {w } s assumed to be a vector-vaued martngae dfference sequence defned on a probabty space Ω F P wth 1.2 E w = 0 E w 2 = where = δ p p are assumed to be determnstc. E t n 1.1 denotes the condtona mathematca expectaton E F t where F t s defned as σ{w = hs wor s supported n part by the Natona Natura Scence Foundaton of hna Hong Kong RG grants and oege of Artfca Integence Nana Unversty ann P.R. hna. Ema: yhn@nana.edu.cn. Department of Apped Mathematcs he Hong Kong Poytechnc Unversty Kowoon Hong Kong P.R. hna. Ema: maxun@poyu.edu.h. Key Laboratory of Systems and ontro Insttute of Systems Scence Academy of Mathematcs and Systems Scence hnese Academy of Scences Beng ; Schoo of Mathematca Scences Unversty of hnese Academy of Scences Beng P.R. hna. Pease address a the correspondences to J-Feng Zhang Ema: f@ss.ac.cn Phone: Department of Mechanca and Aerospace Engneerng Unversty of aforna San Dego USA. Ema: rstc@ucsd.edu. 1

2 2 NI LI ZHANG KRSI 01 t 1} and F 0 s understood as { Ω} and E n 1.2 s smary defned. x of 1.1 s n 2 F t;rn = { ζ R n ζ s Ft -measurabe E ζ 2 < }. Repacng tr n of 2 F t;rn wth R n or R m we then have 2 F ;Rn 2 F ;Rm. We ntroduce the cost functon Jtx;u = =t E t {X t Q t X t E t X t Qt E t X t u R tu E t u Rt E t u 2q t Xt 2ρ t u } 1.3 E t X t N G t X t N Et X t N Ḡ t E t X t N 2F t xg t E t X t N where Q t Q t R t R t t G t Ḡt are determnstc symmetrc matrces of approprate dmensons and q t ρ t t g t are determnstc vectors. Let 2 F t ;R m = { ν = {ν t } ν s F -measurabee ν 2 < t }. hen we pose the foowng optma contro probem. Probem LQ. For the tme-state nta par tx fnd u 2 F t;r m such that Jtx;u = nf Jtx;u. u 2 F t;rm ompared wth the standard stochastc LQ probems Probem LQ has three unconventona features. Frst the cost weghtng and system matrces depend expcty on the nta tme t. Second the term 2F t xg t E t XN t maes Jtx;u a state-dependentor ran-dependent utty. hrd Jt x; u contans nonnear terms of the condtona expectaton of state and contro. hese three features are deepy rooted n the feds of economcs and fnance. he frst feature s an abstracton of the genera dscountng functons; see 4 14 for exampes of hyperboc dscountng and quas-geometrc dscountng. he second feature s of ran-dependent utty 5 and a notabe exampe of the thrd feature s the mean-varance utty It s nown that any of the three features w run the tme consstency of the optma contro namey Beman s prncpe of optmaty w no onger wor for Probem LQ. Probems wth the nonnear term of the condtona expectaton n the cost functona are cassfed as mean-fed stochastc optma contro probems 31. Reazng the tme nconsstency caed nonseparabty there L and Ng 16 used an embeddng scheme to derve the optma pocy for the mut-perod mean-varance portfoo seecton. Note that the optma pocy of 16 s wth respect to the nta par that s t s an optma pocy ony when vewed at the nta tme. hs dervaton s now caed the pre-commtted optma souton. However we fnd that a pre-commtted optma contro wth respect to an nta par w no onger serve as an optma contro for an ntertempora nta par. Athough the pre-commtted optma souton s of some practca and theoretca vaue t negects and does not fuy address the tme nconsstency. Another approach s to hande the tme nconsstency n a dynamc manner by seeng tme-consstent equbrum soutons nstead of a precommtted optma contro; ths has many been motvated by practca appcatons n economcs and fnance and has recenty attracted consderabe research nterest. he quatatve anayss of tme nconsstency can be traced bac to the deas of the father of free maret economcs and mora phosopher Adam Smth 23. In 1955

3 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 3 Strotz 24 gave the frst quanttatve formuaton of tme nconsstency and studed the genera dscountng probem. Hs approach successfuy taced tme nconsstency usng a ead-foower game wth a herarchca structure. Inspred by Strotz hundreds of wors have sought to tace practca probems n economcs and fnance by focusng on the tme nconsstency of dynamc systems descrbed by ordnary dfference or dfferenta equatons; see for exampe and the references theren. Unfortunatey as Eeand 9 10 ponted out t s hard to prove the exstence of Strotz s equbrum pocy. herefore t s necessary and of great mportance to deveop a genera theory of tme-nconsstent contro. In recent years ths topc has attracted consderabe attenton from the theoretca contro communty; see for exampe and the references theren. Wth respect to the tme-nconsstent LQ probems two nds of tme-consstent equbrum soutons have been nvestgated namey the open-oop equbrum contro and the cosed-oop equbrum strategy he two formuatons are nvestgated separatey because n dynamc game theory open-oop contro dffers sgnfcanty from the cosed-oop strategy o compare the am of open-oop formuaton s to fnd an open-oop equbrum contro whe the strategy s the obect of cosed-oop formuaton. Yong further deveoped Strotz s equbrum souton 24 whch s essentay a cosed-oop equbrum strategy nto the LQ optma contro and the nonnear optma contro he open-oop equbrum contro has been extensvey studed by Hu-Jn-Zhou Yong 31 N-Zhang-Krstc 18 and Q-Zhang 22. In partcuary the cosed-oop formuaton can be vewed as an extenson of Beman s dynamc programmng and the correspondng equbrum strategy f t exsts s constructed by a bacward procedure Dfferenty the open-oop equbrum contro s characterzed va a maxmum-prncpe-e methodoogy It s we nown that the am of portfoo seecton s to see the best aocaton of weath among a baset of securtes. he snge-perod mean-varance formuaton ntated by Marowtz 17 s the cornerstone of modern portfoo theory and s wdey used n both academc studes and the fnanca ndustry. he mut-perod mean-varance portfoo seecton s the natura extenson of 17 whch has been extensvey studed. L-Ng 16 and Zhou-L 32 were the frst to report the anaytca pre-commtment optma poces for the dscrete-tme case and the contnuoustme case respectvey. In fact the mut-perod mean-varance portfoo seecton probem whch s a partcuar exampe of tme-nconsstent probem stmuated the recent deveopments n tme-nconsstent probems and the revsts to mut-perod mean-varance portfoo seecton In ths paper we examne the aforementoned Probem LQ. In Secton 2 we ntroduce the mxed equbrum souton to ProbemLQ. he souton contans two d- fferent parts: a pure-feedbac-strategy part and an open-oop-contro part. By ettng the open-oop-contro part be ndependent of the nta state or the pure-feedbacstrategy part be zero the correspondng mxed equbrum souton s reduced to a near feedbac equbrum strategy and open-oop equbrum contro respectvey. Secton 3 characterzes the mxed equbrum souton usng a maxmum-prncpe-e methodoogy wth convexty statonarty and forward-bacward stochastc dfference equatons FBS Es. It s shown that the convexty and statonarty condtons can be equvaenty characterzed va soutons to three sets of dfference equatons. Based on the resuts for the mxed equbrum souton we then obtan the resuts for the open-oop equbrum contro and near feedbac equbrum strategy wth respect

4 4 NI LI ZHANG KRSI to a fxed nta par. For the case wth a the nta pars condtons n terms of sovabty of three sets of dfference equatons are gven to ensure the exstence of mxed equbrum souton. hese condtons are necessary and suffcent to determne the open-oop equbrum contro and near feedbac equbrum strategy. Interestngy for a of the nta pars the exstence of genera feedbac equbrum strategy s shown to be equvaent to the exstence of near feedbac equbrum s- trategy whch can be obtaned by a bacward procedure. Furthermore the bacward procedure wors ony when the feedbac equbrum strategy exsts for a of the nta pars and cannot be apped to the case where we now ony of the exstence of a feedbac equbrum strategy for a fxed nta par. Secton 4 gves an exampe to ustrate the deveoped theory. Fnay n Secton 5 we dscuss future topcs that are worth nvestgatng. hs paper maes the foowng novetes. Most of the exstng resuts for tme-nconsstent LQ probems are for the contnuous-tme case he dscrete-tme mut-perod mean-varance portfoo seecton probem s a notabe exampe of Probem LQ and ts nvestgaton cas for the deveopment of genera theory of dscrete-tme tme-nconsstent LQ optma contro. Furthermore the mode and methodoogy deveoped n ths paper are more genera than those n 18. he noton of mxed equbrum souton s ntroduced and t seems that no smar noton has been reported for tme-nconsstent optma contro. Necessary and suffcent condtons are estabshed to characterze a par of pure-feedbac strategy and open-oop contro as a mxed equbrum souton for a tme-state nta par. Usng the noton of mxed equbrum souton the condtons to equvaenty ensure the exstence of an open-oop equbrum contro and a near feedbac equbrum strategy can be smutaneousy obtaned. In other words we can nvestgate the two equbrum soutons n a unfed way. Importanty the mxed equbrum souton s not a hoow concept. In Secton 4 t s shown that nether the open-oop equbrum contro nor the feedbac equbrum strategy exsts for the nta par tx wth t = 01 and x 2 F t;r2 athough we are abe to construct 10 mxed equbrum soutons. herefore t s necessary to study the mxed equbrum souton whch gves us more fexbty to dea wth the tme-nconsstent optma contro. he wor of 19 serves as a companon to ths paper n terms of testng our deveoped theory and pursung the sovabty of the mut-perod mean-varance portfoo seecton probem. he non-degenerate assumpton was removed n 19 whch s popuar n the terature on mut-perod mean-varance portfoo seecton. Neat condtons have been obtaned n 19 to characterze the exstence of the equbrum soutons. o emphasze the dependence on the nta par Probem LQ for the nta par tx s denoted as Probem LQ tx throughout ths paper. Furthermore for notatona smpcty we denote n ths paper A t = A t Āt B t = B t B t t = t t D t = D t D t Q t = Q t Q t R t = R t R t G t = G t Ḡt t t. 2. Mxed equbrum souton. Before ntroducng the mxed equbrum souton we gve the defntons of open-oop equbrum contro and feedbac equbrum strategy. By a strategy we mean a decson rue that a controer uses to

5 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 5 seect a contro acton based on the avaabe nformaton set. Mathematcay a s- trategy s a mappng or an operator defned on the nformaton set. Substtutng the avaabe nformaton nto a strategy we obtan the open-oop vaue or reazaton of ths strategy. Defnton At stage t a functon f s caed an admssbe feedbac strategy or smpy a feedbac strategy at f for ζ F 2 ;Rn f ζ F 2 ;Rm. he set of ths type of feedbac strateges s denoted by F and F t F s denoted by F t.. Let f = f t...f F t. For t and ζ F 2 ;Rn f ζ can be dvded nto two parts namey f ζ = f c fp ζ where fc = f 0 s the nhomogeneous part and the remander f p s the pure-feedbac-strategy part of f. Furthermore f p t...f p s caed a pure-feedbac strategy. Defnton A strategy ψ F t s caed a feedbac equbrum strategy of Probem LQ tx f the foowng two ponts hod: a ψ does not depend on x; b For any t and any u F 2 ;Rm t hods that 2.1 J X tx ;ψ X ψ J X tx ;u ψ X uψ 1. In 2.1 ψ X ψ and ψ X u ψ 1 wth = {...} 1 = { 1...N 1} are gven by ψ X ψ = ψ X ψ...ψ X ψ ψ X u ψ 1 = ψ 1 X u ψ 1...ψ X u ψ where X ψ X u ψ are as foows X ψ 1 = A X ψ f =1 X ψ B ψ X ψ ĀE X ψ B E ψ X ψ Xψ D ψ X ψ E X ψ w D E ψ X ψ d = X tx X u ψ 1 = A X u ψ B E ψ X u ψ p =1 Xu ψ B ψ X u ψ ĀE X u ψ f D ψ X u ψ w D E ψ X u ψ d 1 = A X u ψ B u f X u ψ X u ψ =1 = X tx 1. X u ψ D u d w Furthermore n and 2.3 X tx X tx E X u ψ s computed va 1 = A X tx B ψ X tx f =1 X tx D ψ X tx d w X tx t = x t.

6 6 NI LI ZHANG KRSI. Let Ψγ 2 t ;R m n 2 t ;R m. Here for H = R m n R m { 2 t ;H = ν = {ν t } ν H s determnstc ν 2 < t }. If Ψ andγ do not depend on x andψ of s equa toψγ namey ψ ξ = Ψ ξγ t ξ F 2 ;Rn then Ψγ s caed a near feedbac equbrum strategy of Probem LQ tx. Defnton 2.3. A contro u tx F 2 t;r m s caed an open-oop equbrum contro of Probem LQ tx f J X tx ;u tx J X tx 2.4 ;u u tx 1 hods for any t and any u F 2 ;Rm. Here u tx and u tx 1 are the restrctons of u tx on and 1 respectvey; and X tx s computed va X tx 1 = A X tx B u tx f =1 X tx D utx d w X tx t = x t. Defnton A par Φv tx 2 t ;R m n 2 F t;r m s caed a mxed equbrum souton of Probem LQ tx f the foowng two ponts hod: a Φ does not depend on x and v tx depends on x; b For any t and any u 2 F ;Rm t hods that 2.5 J X tx ;Φ X Φ v tx J X tx ;u Φ X u Φ v tx 1. In 2.5 Φ X Φ v tx and Φ X uφ v tx 1 are gven respectvey by Φ X Φ v tx Φ X Φ vtx Φ1 X u Φ 1 v tx 1 Φ X u Φ v tx where X Φ X u Φ are defned by X Φ A 1 = B Φ X Φ B E v tx f X Φ D Φ = X tx X u Φ 1 = E X Φ A B Φ X u Φ B v tx B E v tx f D Φ E X u Φ 1 = A X u Φ B u f X u Φ X u Φ =1 = X tx 1. Ā B Φ E X Φ B v tx =1 D Φ X Φ D vtx D E v tx d Ā B Φ E X u Φ =1 D Φ X u Φ D u d w w D vtx D E v tx d X u Φ w

7 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 7 he state X tx n and 2.7 s computed va 2.8 X tx 1 = A B Φ X tx B v tx f =1 D Φ X tx X tx t = x t. D vtx d w. Φ and v tx n are respectvey the pure-feedbac-strategy part and the open-oop-contro part of the mxed equbrum souton Φv tx.. Lettng Φ = 0 n the correspondng v tx satsfyng 2.5 denoted as v tx s then an open-oop equbrum contro of Probem LQ tx. v. If v tx of happens not to depend on x and denote such v tx as v then the correspondng Φv s a near feedbac equbrum strategy of Probem LQ tx. Remar 2.5. By the defnton a mxed equbrum souton Φv tx s tme consstent aong X tx namey for any t Φv tx s a mxed equbrum souton for the nta par X tx. Notng Φ X Φ v tx = Φ X Φ v tx Φ XΦ v tx 1 u Φ X utφ v tx 1 s obtaned by repacng Φ X Φ v tx and X Φ of Φ X Φ v tx wth u and X uφ respectvey. Furthermore note that the v tx s on both sdes of 2.5 are the same. hs s why we ca Φ the pure-feedbac-strategy part and v tx the open-oop-contro part. 3. haracterzaton of the mxed equbrum souton he case wth the fxed tme-state nta par t x. he foowng emma descrbes the cost dfference formua under contro perturbaton. Lemma 3.1. Let ū 2 F ;Rm and λ R. hen we have 3.1 J X tx ;Φ Xū λ v tx λū Φ X ūλ v tx 1 J X tx ;Φ X Φ v tx = λ 2 J0;ū 2λ =1 R Φ X Φ D E Y Φ 1 w ρ ū v tx B E Y Φ 1 where 3.2 J0;ū = = E α ū Q Φ R Φ α ū E α ū Q Φ R Φ E α ū ū E R ū E α ū N G α ū N E α ū N Ḡ E α ū N

8 8 NI LI ZHANG KRSI and X ū λ α ū Y Φ are gven respectvey by the S Es X ū λ 1 = A B Φ Xū λ Ā B Φ E Xū λ B v tx B E v tx f =1 D Φ Xū λ D vtx D E v tx d w D Φ E Xū λ X ū λ 1 = A B Φ Xū λ B v tx λb ū f =1 D Φ Xū λ D vtx λdū d w X ū λ α ū = X tx 1 1 = A B Φ α ū Ā B Φ E α ū =1 D Φ α ū D Φ α ū 1 = B ū =1 D ūw α ū = 0 1 and the bacward stochastc dfference equaton BS E for short E α ū w 3.5 Y Φ Y Φ N = Q Φ R Φ X Φ A B Φ E Y Φ =1 Q Φ R Φ E X Φ 1 Ā B Φ E Y Φ 1 D Φ E Y Φ 1 w D Φ E Y Φ 1 w Φ R v tx Φ R E v tx Φ ρ q = G X Φ N ḠE X Φ N F X tx g. Proof. See Appendx A. heorem 3.2. he foowng statements are equvaent: Probem LQ tx admts a mxed equbrum souton. here exsts a par Φv tx 2 t ;R m n F 2 t;r m such that the statonary condton = R Φ X Φ =1 v tx B E Y Φ 1 D E Y Φ 1 w ρ t and the convexty condton 3.7 nf J0;ū 0 t ū 2 F ;Rm hod. Here Y Φ 1 s computed va the foowng FBS E

9 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem X Φ 1 = A B Φ X Φ Ā B Φ E X Φ B v tx B E v tx f Y Φ X Φ Y Φ N =1 D Φ X Φ D vtx D E v tx = Q X Φ Q E X Φ d w D Φ E X Φ Φ R Φ X Φ Φ R Φ E X Φ Φ R v tx Φ R E v tx E A B Φ Y Φ 1 Ā B E Φ Y Φ 1 =1 D Φ E Y Φ D Φ = X tx 1 w E Y Φ 1 w Φ ρ q = G X Φ N ḠE X Φ N F X tx g and J0;ū s gven n 3.2. In 3.8 X tx s computed va X tx 1 = A B Φ X tx B v tx f =1 X tx t = x t. D Φ X tx D vtx d w Furthermore under any of the above condtons Φv tx gven n s a mxed equbrum souton of Probem LQ tx. Proof. hs foows from the defnton and Lemma 3.1. o proceed we frst study the expresson of Y Φ of 3.8 under some addtona condton. Lemma 3.3. If for t v tx = Γ X tx v tx wth Γ v tx beng determnstc then the bacward state Y Φ of 3.8 has the foowng expresson: where Y Φ = S X Φ S E X Φ X tx U X tx π t E X tx S = Q Φ R Φ S1 A B Φ A B Φ =1 δ D Φ S1 D Φ S = Q Φ R Φ A B Φ S1 Ā B Φ S 1 A B Φ Ā B S1 Φ S 1 A B Φ =1 δ D Φ S1 D Φ D Φ S1 D Φ S N = G SN = Ḡ

10 10 NI LI ZHANG KRSI { = Φ R S1 A B Φ B =1 δ D Φ S1 D} Γ 1 A B Φ A B Φ B Γ =1 δ D Φ 1 D Φ D Γ = {Φ R A B Φ S1 B S 1 B Ā B Φ S1 S 1 B =1 δ D Φ S1 D D Φ S D } Γ A B Φ 1 A B Φ B Γ Ā B Φ 1 1 A B Φ B Γ =1 δ D Φ 1 D Φ D Γ N = 0 N = 0 and { U = A B Φ U 1 U N = F π = β v tx A B Φ S1 S 1 f π 1 A B Φ 1 1 f =1 δ D Φ S1 d D Φ 1 d Φ ρ q π N = g wth β = Φ R A B Φ S1 S 1 B 1 1 B =1 δ D Φ S1 D 1D. Proof. See Appendx B. For a matrx M R n m et M be ts Moore-Penrose nverse. hen we have the foowng emma 1. Lemma 3.4. Let matrces L M and N be gven wth approprate sze. hen LXM = N has a souton X f and ony f LL NMM = N. Moreover the souton of LXM = N can be expressed as X = L NM V L LVMM where V s a matrx wth approprate sze. If M = I n Lemma 3.4 then LL N = N s equvaent to RanN RanL. Here RanN s the range of N. he foowng theorem s concerned wth the necessary and suffcent condton for the exstence of a mxed equbrum souton. heorem 3.5. he foowng statements are equvaent: Probem LQ tx admts a mxed equbrum souton. here exsts Φ 2 t ;R m n such that the foowng assertons hod. a he couped equatons

11 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem S = Q Φ R Φ S1 A B Φ A B Φ p =1 δ D Φ S 1 D Φ S = Q Φ R Φ S1 A B Φ A B Φ =1 δ D Φ S 1 D Φ S N = G S N = G Ḡ O = R B S 1B =1 δ D S 1 D 0 t are sovabe n the sense of O 0 t namey O t are a nonnegatve defnte. b he condton L X tx θ Ran O t s satsfed. Here X tx s computed va X tx 1 = A B O L X tx B O θ f =1 X tx t = x t D O L X tx D O θ d w and O L θ t are gven by O = R B S1 1 B =1 δ D S 1 1 D L = p =1 δ D S 1 1 B S1 1 A B U 1 θ = p =1 δ D S 1 1 d B S1 1 f B π 1 ρ t where { = Φ R S1 A B Φ B =1 δ D Φ S1 D} Γ 1 A B Φ A B Φ B Γ =1 δ D Φ 1 D Φ D Γ { = Φ R S1 A B Φ B =1 δ D Φ S1 D} Γ 1 A B Φ A B Φ B Γ =1 δ D Φ 1 D Φ D Γ N = 0 N = 0 t

12 12 NI LI ZHANG KRSI and { U = A B Φ U 1 U N = F t π = β O θ A B Φ S1 f π 1 =1 δ D Φ S1 d D Φ 1 d A B Φ 1 f Φ ρ q π N = g t wth 3.11 Γ = O L Φ t 3.12 and β = Φ R A B Φ S1 B 1 B =1 δ D Φ S1 D 1D. Furthermore under condton et v tx = Γ X tx O θ t and Φ Γ t are gven n ; then Φv tx s a mxed equbrum souton of Probem LQ tx. Proof. See Appendx. Remar 3.6. In heorem 3.5 the sovabty of 3.9 s to characterze the convexty 3.7 whe 3.10 s to characterze the statonary condton 3.6. If Φ Γ 1 have been determned then O L can be further constructed. Notng 3.11 t s mpossbe to determne the vaue of Φ by usng the property Φ Γ = O L and any Φv tx that satsfes condton of heorem 3.5 s a mxed equbrum souton. Nevertheess the freedom of seectng Φ coud enabe us to dea wth the open-oop equbrum contro and near feedbac equbrum strategy n a unfed way. From heorem 3.5 the foowng two coroares are straghtforward. he frst concerns the open-oop equbrum contro whch s obtaned by ettng Φ = 0 n heorem 3.5. oroary 3.7. he foowng statements are equvaent: Probem LQ tx admts an open-oop equbrum contro. he foowng assertons hod. a he couped equatons 3.13 Ŝ = Q A Ŝ1A =1 δ Ŝ 1 Ŝ = Q A Ŝ1A =1 δ Ŝ 1 Ŝ N = G Ŝ N = G Ḡ Ô = R B Ŝ1B =1 δ D Ŝ 1 D 0 t

13 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 13 are sovabe n the sense of Ô 0 t. b he condton 3.14 L Xtx θ Ran Ô t s satsfed. Here X tx s computed va X tx 1 = A B Ô L Xtx =1 X tx t = x t and Ô L θ t are gven by B Ô θ f D Ô L Xtx D Ô θ d w 3.15 Ô = R BŜ1 1 B =1 δ D Ŝ 1 1 D L = p =1 δ D Ŝ 1 1 BŜ1 1 A BÛ1 θ = p =1 δ D Ŝ 1 1 d BŜ1 1 f B π 1 ρ t where = A 1 A =1 δ 1 { A Ŝ1B =1 δ Ŝ 1 D A 1 B =1 δ 1 D }Ô L = A 1 A =1 δ 1 { A Ŝ1B =1 δ Ŝ 1 D A 1 B =1 δ 1 D }Ô L N = 0 N = 0 t and wth { Û = A Û 1 Û N = F t π = β Ô θ A Ŝ1 f π 1 A 1 f =1 δ Ŝ 1 d 1 d q π N = g t β = A Ŝ1 B 1 B =1 δ Ŝ 1 D 1 D.

14 14 NI LI ZHANG KRSI Furthermore under condton the contro v tx = Ô L Xtx Ô θ t s an open-oop equbrum contro of Probem LQ tx. Note that the near feedbac equbrum strategy has nothng to do wth the nta state x. he second coroary s concerned wth the exstence of a near feedbac equbrum strategy whch s obtaned by ettng Γ = 0 t n heorem 3.5. oroary 3.8. he foowng statements are equvaent: Probem LQ tx admts a near feedbac equbrum strategy. he foowng assertons hod. a he couped equatons S = Q Φ R Φ A B Φ S1 p A B Φ =1 δ D Φ S 1 D Φ S = Q Φ R Φ A B Φ S1 A B Φ =1 δ D Φ S 1 D Φ S N = G SN = G Ḡ 3.16 Õ = R B S 1 B =1 δ D S1 D 0 t are sovabe n the sense of Õ 0 t. b he condton 3.17 L Xtx θ Ran Õ t s satsfed. Here X tx s computed va X tx 1 = A B Õ L Xtx B Õ θ f =1 X tx t = x t D Õ L Xtx D Õ θ d w and L θ t are gven by L = B S 1 A =1 δ D S1 B Ũ1 θ = B S 1 f =1 δ D S1 d B π 1 ρ t where { Ũ = A B Φ Ũ1 Ũ N = F t

15 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 15 π = β Õ θ A B Φ S1 f π 1 =1 δ D Φ S1 d Φ ρ q π N = g t wth β = Φ R A B Φ S1 B =1 δ D Φ S1 D. c Φ above s gven by { Φ = Õ L t }. Furthermore under condton { Õ L Õ θ t } s a near feedbac equbrum strategy of Probem LQ tx. We now consder the unque exstence of open-oop equbrum contro and near feedbac equbrum strategy. heorem 3.9. Let t and x 2 F t;rn. he foowng s true. he foowng statements are equvaent: a Probem LQ tx admts a unque open-oop equbrum contro. b 3.13 s sovabe and Ô t are nvertbe whch are gven n c For any t and any ξ 2 F ;Rn Probem LQ ξ admts a unque open-oop equbrum contro. he foowng statements are equvaent: d Probem LQ tx admts a unque near feedbac equbrum strategy. e Õ 0 t namey Õ t are a postve defnte whch are gven n f For any t and any ξ 2 F ;Rn Probem LQ ξ admts a unque near feedbac equbrum strategy. Proof.. a b. Let v tx beanopen-oopequbrumcontroofprobemlq tx. In ths case 3.6 becomes = R v tx B E Y 1 =1 D E Y 1 w ρ t. Mmcng the proof of heorem 3.5 and based on Lemma 3.4 a contro of the foowng form 3.19 v tx = Ô L Xtx Ô θ I Ô ÔV t aso satsfes 3.18 where V R m s determnstc and X tx θ are gven by X tx 1 = A B Ô L Ô Xtx B θ I Ô ÔV f =1 D L Xtx Ô D Ô θ I Ô ÔV d w X tx t = x t and θ = B Ŝ1 1 f =1 δ D Ŝ 1 1 d B π 1 ρ

16 16 NI LI ZHANG KRSI wth π 1 computed va Ô π = β θ I Ô ÔV A Ŝ1 f π 1 A 1 f =1 δ Ŝ 1 d 1 d q π N = g t. ombnng the sovabty of 3.13 we now that any contro of the form 3.19 s an open-oop equbrum contro of Probem LQ tx. herefore Probem LQ tx admts a unque open-oop equbrum contro f and ony f Ô t are nvertbe and 3.13 s sovabe. b c. As 3.13 s sovabe and Ô t are nvertbe we now from oroary 3.7 that Probem LQ ξ admts an open-oop equbrum contro. Based on the proofofa b t foowsthat ProbemLQ ξ admts aunque open-oopequbrum contro. c a. It s obvous.. d e. Let Φv be a near feedbac equbrum strategy of Probem LQ tx. Õ 0 foows from the sovabty of We now prove that Õ are a nvertbe. Note that the near feedbac equbrum strategy s ndependent of x. If some of Õ are snguar then smar to those of a b v can be seected as any one of the foowng forms: 3.20 v = Õ θ I Õ ÕV t. In 3.20 V R m t are determnstc and θ s gven by θ = B S 1 f =1 δ D S1 d B π 1 ρ wth π 1 computed va Õ π = β θ I Õ ÕV A B Φ S1 f π 1 =1 δ D Φ S1 d Φ ρ q π N = g t. herefore Probem LQ tx admts a unque near feedbac equbrum strategy f and ony f Õ are a nvertbe and thus are postve defnte. e f and f d are obvous. hs competes the proof. Remar Probem LQ tx admttng a unque open-oop equbrum contro s a oca property whch s ony of the unque exstence for the fxed nta par t x. Interestngy ths oca property coud ensure a sem-goba property namey for any t after t and any ξ F 2 ;Rn Probem LQ ξ aso admts a unque open-oop equbrum contro. A smar property aso hods for the near feedbac equbrum strategy he case wth a of the nta pars. Smpy nowng that Probem LQ tx admts an open-oop equbrum contro or a near feedbac equbrum s- trategy t s hard or generay mpossbe to derve sharp resuts e those of heorem 3.9. Aternatvey n ths secton we consder the case that the nta par s aowed to vary. o begn we frst state the resuts for the open-oop equbrum contro. heorem he foowng statements are equvaent:

17 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 17 For any tx wth t and x 2 F t;rn Probem LQ tx admts an openoop equbrum contro. he couped equatons Ŝ = Q A Ŝ1A =1 δ Ŝ 1 Ŝ = Q A Ŝ1A =1 δ Ŝ 1 Ŝ N = G Ŝ N = G Ḡ Ô 0 = A 1 A =1 δ 1 { A Ŝ1B A 1 B =1 δ Ŝ 1 D 1 D } Ô L = A 1 A =1 δ 1 { A Ŝ1B A 1 B =1 δ Ŝ 1 D 1 D } Ô L N = 0 N = 0 Ô Ô L = L and 3.23 π = β Ô θ A Ŝ1 f π 1 A 1 f =1 δ Ŝ 1 d 1 d q π N = g Ô Ô θ = θ are sovabe n the sense of where wth Ô 0 Ô Ô L L = 0 Ô Ô θ θ = 0 Ô = R B Ŝ1B =1 δ D Ŝ 1 D Ô = R B Ŝ1 1 B =1 δ D Ŝ 1 1 D L = BŜ1 1 A BÛ1 =1 δ D Ŝ 1 1 θ = B Ŝ1 1 f B π 1 ρ =1 δ D Ŝ 1 1 d { Û = A Û 1 Û N = F

18 18 NI LI ZHANG KRSI and β = A Ŝ1 B 1 B =1 δ Ŝ 1 D 1 D. Proof.. From the sovabty of 3.22 and 3.23 we now that 3.14 hods for any tx. herefore hods. L Xtx. Note that 3.14 s equvaent to ÔÔ θ = L Xtx θ t. Lettng = t and tang dfferent x s we have ÔtÔ L t t = L t Ô t Ô t θ t = θ t. As for any tx wth t and x F 2 t;rn Probem LQ tx admts an open-oop equbrum contro we must have the sovabty of he foowng resut s for the feedbac equbrum strategy. heorem he foowng statements are equvaent: For any tx wth t and x 2 F t;rn Probem LQ tx admts a near feedbac equbrum strategy. he couped equatons 3.24 S = Q A S 1 A =1 δ S1 A S 1 B =1 δ S1 D Õ L L Õ B S1 A =1 δ D S1 L Õ R B S 1 B =1 δ D S1 D Õ L S = Q A S 1 A =1 δ A S 1 B L Õ S1 =1 δ S1 D Õ L B S1 A =1 δ D S1 L Õ R B S 1 B =1 δ D S1 D Õ L S N = G SN = G Ḡ Õ 0 Õ Õ L = L and 3.25 π = β Õ θ A B Õ L S1 f π 1 =1 δ D Õ L S1 d L Õ ρ q π N = g Õ Õ θ = θ are sovabe n the sense of Õ 0 Õ Õ L = L Õ Õ θ = θ

19 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 19 where Õ = R B S 1 B =1 δ D S1 D L = B S 1 A =1 δ D S1 B Ũ1 θ = B S 1 f =1 δ D S1 d B π 1 ρ wth { Ũ = A B Õ L Ũ1 Ũ N = F and β = L Õ R A B Õ L S1 B =1 δ D Õ L S1 D. here exsts a par Φv 2 ;R m n 2 F ;Rm such that for any tx wth t and x 2 F t;rn Φv t s a near feedbac equbrum strategy of Probem LQ tx. Here Φv t s the restrcton of Φv on t. v here exsts a ψ F such that for any tx wth t and x 2 F t;rn ψ t s a feedbac equbrum strategy of Probem LQ tx. Here ψ t s the restrcton of ψ on t. Furthermore under any of the above condtons the par Φ t v t wth Φ t = { Õ L t } v t = { Õ θ t } s a feedbac equbrum strategy of Probem LQ tx. Proof. See Appendx D. We now consder the mxed equbrum souton. If t exsts we have some freedom to seect the pure-feedbac-strategy part of the mxed equbrum souton as ponted out n Remar 3.6. In heorem 3.13 we have the necessary and suffcent condton to ensure the exstence of a mxed equbrum souton for a of the nta pars. Because dfferent nta pars may correspond to dfferent pure-feedbacstrategy parts of the mxed equbrum souton the condton of heorem 3.13 s for the case that specfes the pure-feedbac-strategy part Φ. heorem he foowng statements are equvaent: For any tx wth t and x 2 F t;rn Probem LQ tx admts a mxed equbrum souton and the pure-feedbac-strategy part s Φ t. Here Φ 2 ;R m n and Φ t s the restrcton of Φ on t. here exsts Φ 2 ;R m n such that the foowng dfference equatons

20 20 NI LI ZHANG KRSI S = Q Φ R Φ S1 A B Φ p A B Φ =1 δ D Φ S 1 D Φ S = Q Φ R Φ S1 A B Φ A B Φ =1 δ D Φ S 1 D Φ S N = G S N = G Ḡ O = R B S 1B =1 δ D S 1 D 0 { = Φ R S1 A B Φ B =1 δ D Φ S1 D} Γ 1 A B Φ A B Φ B Γ =1 δ D Φ 1 D Φ D Γ { = Φ R S1 A B Φ B =1 δ D Φ S1 D} Γ 1 A B Φ A B Φ B Γ =1 δ D Φ 1 D Φ D Γ N = 0 N = 0 O O L = L and π = β O θ A B Φ S1 f π 1 =1 δ D Φ S1 d D Φ 1 d 1 A B Φ f Φ ρ q π N = g O O θ = θ are sovabe n the sense of O 0 O O L = L O O θ = θ where O = R B S1 1 B =1 δ D S 1 1 D L = B S1 1 A B U 1 =1 δ D S 1 1 θ = B S1 1 f B π 1 ρ =1 δ D S 1 1 d

21 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 21 and Γ = O L Φ wth { U = A B Φ U 1 U N = F and β = Φ R A B Φ S1 B 1 B =1 δ D Φ S1 D 1D. For any t and x F 2 t;rn et v tx t where X tx 1 = A B O L X tx B O θ f =1 X tx t = x t. = O L Φ X tx O θ D O L X tx D O θ d w Under condton and for any tx wth t and x F 2 t;rn Φ t v tx s a mxed equbrum souton of Probem LQ tx. Proof.. hs foows from heorem In ths case for anytx wth t and x F 2 t;rn the pure-feedbacstrategy part of the mxed equbrum souton s Φ t. Note that 3.10 s equvaent to O O L X tx θ = L X tx θ t. Lettng = t and tang dfferent x s we have O t O t L t = L t O t O t θ t = θ t. herefore and 3.28 are sovabe. o end ths secton we pose the foowng assumpton. H Q t Q t Q t G t G t Ḡ 0R t R t R t 0t t. he foowng resut s straghtforward. heorem Lettng H hod then Õ are a postve defnte. Furthermore for any t and any x 2 F t;rn Probem LQ tx admts a unque feedbac equbrum strategy Φ v wth 3.29 Φ = { Õ 1 L t } v = { Õ 1 θ t }. Proof. A smpe cacuaton shows that 3.24 s equa to S = Q L Õ RÕ L A B Õ S1 L D Õ L A B Õ L p =1 δ S 1 D Õ L S = Q L Õ RÕ L A B Õ L S1 A B Õ L =1 δ S 1 D Õ L S N = G SN = G Ḡ Õ 0 Õ Õ L = L t. D Õ L Due to H we have that S S 0Õ 0. herefore 3.24 and 3.25 are sovabe. hs competes the proof.

22 22 NI LI ZHANG KRSI 4. An exampe. onsder a dscrete-tme stochastc LQ probem whose system dynamcs and cost functona are gven respectvey by { X1 = A X B u D u w X t = x t {0123} {t...3} and Jtx;u = 3 E t X Q X t u R u Et X 4 GX 4 Et X 4 ḠE t X 4 =t where A 0 = A = A = A 3 = B = B = B 1 2 = B 3 = D = D = D = D 3 = Q 0 0 = Q = Q 2 = Q = R = 0 R 1 = R 2 = 1 R 3 = 0.5 G = Ḡ = and {w = 0123} s a martngae dfference wth constant second-order condtona moment E w 2 = 1 = Open-oop equbrum contro For ths LQ probem by performng the teraton 3.21 we have Ô0 = Ô 1 = Ô2 = Ô3 = Because Ô1 = <0 for tx wth t = 01 and x 2 F 0;R2 or x 2 F 1;R2 and based on oroary 3.7 the open-oop equbrum contro of ths LQ probem must not exst. Feedbac equbrum strategy By performng the teraton 3.24 we have Õ0 = Õ 1 = Õ 2 = Õ3 = Because Õ0 < 0Õ2 < 0 for tx wth t = 012 and x 2 F 0;R2 or x 2 F 1;R2 and based on oroary 3.8 and heorem 3.12 the feedbac equbrum strategy of ths LQ probem must not exst. Mxed equbrum souton We use the command randn of MALAB to randomy generate Φ = {Φ = 0123}. Note that Φ R 1 2 O O R 1 = 0123 and et ψ = Φ 0 Φ 1 Φ 2 Φ 3 O = O 0 O 1 O 2 O 3 O = O 0 O 1 O 2 O 3. By performng the teratons we seect 10 ψs and get the correspondng Os and Os ψ = O = O =

23 ψ = ψ = ψ = ψ = ψ = ψ = ψ = ψ = ψ = Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem O = O = O = O = O = O = O = O = O = O = O = O = O = O = O = O = O = O = For a 10 cases O = 0123 are a postve and O = 0123 are a nvertbe. hen due to heorem 3.13 for any tx wth t {0123} and x 2 F t;r2 the above 10 cases correspond to 10 mxed equbrum soutons of the consdered LQ probem whch can be easy constructed from heorem For exampe wth the ast ψ gven above the mxed equbrum souton s as foows. Let and Φ 0 = Φ 1 = Φ 2 = Φ 3 = v 0x = O L Φ X 0x {0123}

24 24 NI LI ZHANG KRSI wth and { X 0x 1 = A B O L X 0x D O L X 0x w X 0x 0 = x {0123} O 0 L 0 = O 1 L 1 = O 2 L 2 = O 3 L 3 = hen Φv 0x s a mxed equbrum souton of ths LQ probem for the nta par 0x where Φ = {Φ = 0123}. 5. Summary. In ths paper the noton of mxed equbrum souton s ntroduced for the tme-nconsstent dscrete-tme mean-fed stochastc LQ optma contro. For a par of pure-feedbac strategy and open-oop contro necessary and suffcent condtons are gven to ensure that such a par s a mxed equbrum souton. On ths bass the open-oop equbrum contro and feedbac equvaent strategy can be deat wth n a unfed way. Athough we provde some reevant resuts the theory for mxed equbrum soutonsfarfrommature. FromtheexampenSecton4 wenowthataremarabe property of mxed equbrum souton s ts non-unqueness. hus we propose that the foowng topcs warrant further study: haracterze the set of a of the mxed equbrum soutons of ProbemLQ. Fnd the best mxed equbrum souton whch shoud be the one under whch the equbrum vaue functon attans ts extreme. As a test the mut-perod mean-varance portfoo seecton must be thoroughy nvestgated. v Fnay the anayss shoud be extended beyond the ream of the LQ contros and to a contnuous-tme settng. REFERENES 1 M. At Ram X. hen and X.Y. Zhou Dscrete-tme ndefnte LQ contro wth state and contro dependent noses Journa of Goba Optmzaton 2002 vo. 23 pp S. Basa and G. habaaur Dynamc mean-varance asset aocaton Revew of Fnanca Studes 2010 vo. 23 pp Başar and G.J. Osder Dynamc Noncooperatve Game heory 2nd Edton SIAM asscs n Apped Mathematcs Bor and A. Murgoc A genera theory of Marovan tme nconsstent stochastc contro probems avaabe at DOI: /ssrn or Bor A. Murgoc and X.Y. Zhou Mean-varance portfoo optmzaton wth state dependent rs averson Mathematca Fnance 2014 vo. 24 pp X. u D. L and X. L Mean-varance pocy tme consstency n effcency and mnmumvarance sgned supermartngae measure for dscrete-tme cone constraned marets Mathematca Fnance 2017 vo. 27 no. 2 pp X. u D. L S. Wang and S. Zhu Better than dynamc meanvarance: tme nconsstency and free cash fow stream Mathematca Fnance 2012 vo. 22 pp X. u X. L and D. L Unfed framewor for optma mut-perod mean-varance portfoo seecton under mean-fed formuaton IEEE ransactons on Automatc ontro 2014 vo. 59 pp I. Eeand and A. Lazra Beng serous about non-commtment: subgame perfect equbrum n contnuous tme arxv: math/

25 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem I. Eeand and.a. Prvu Investment and consumpton wthout commtment Mathematcs and Fnanca Economcs 2008 vo. 2 no. 1 pp S.M. Godman onsstent pan Revew of Economc Studes 1980 vo. 47 pp Y. Hu H. Jn and X.Y. Zhou me-nconsstent stochastc near-quadratc contro SIAM Journa on ontro and Optmzaton 2012 vo. 50 pp Y. Hu H. Jn and X.Y. Zhou me-nconsstent stochastc near-quadratc contro: characterzaton and unqueness of equbrum SIAM Journa on ontro and Optmzaton 2017 vo. 50 no. 3 pp P. Kruse and A.A. Smth onsumpton and savngs decsons wth quas-geometrc dscountng Econometrca 2003 vo. 71 no. 1 pp D. Labson Goden eggs and hyperboc dscountng he Quartery Journa of Economcs 1997 vo. 112 pp D. L and W.L. Ng Optma dynamc portfoo seecton: mut-perod mean-varance formuaton Mathematca Fnance 2000 vo. 10 pp H. Marowtz Portfoo seecton he Journa of Fnance 1952 vo. 7 pp Y.H. N J.F. Zhang and M. Krstc me-nconsstent mean-fed stochastc LQ probem: openoop tme-consstent contro accepted by IEEE ransactons on Automatc ontro 2018 vo.63 no.9 pp Y.H. N X. L J.F. Zhang and M. Krstc Equbrum soutons of mut-perod mean-varance portfoo secton arxv: Y.H. N J.F. Zhang and X. L Indefnte mean-fed stochastc near-quadratc optma contro IEEE ransactons on Automatc ontro 2015 vo. 60 no. 7 pp I. Paacos-Huerta me-nconsstent preferences n Adam Smth and Davs Hume Hstory of Potca Economy 2003 vo. 35 pp Q. Q and H.S. Zhang me-nconsstent stochastc near quadratc contro for dscrete-tme systems Scence hna Informaton Scences 2017 vo. 60: A. Smth he heory of Mora Sentments Frst Edton 1759; Reprnt Oxford Unversty Press R.H. Strotz Myopa and nconsstency n dynamc utty maxmzaton Revew of Economc Studes 1955 vo. 23 pp H.Y. Wang and Z. Wu me-nconsstent optma contro probem wth random coeffcents and stochastc equbrum HJB equaton Mathematca ontro and Reated Reds 2015 vo. 5 no. 3 pp Q. We Z. Yu and J.M. Yong me-nconsstent recursve stochastc optma contro probems SIAM Journa on ontro Optmzaton 2017 vo. 55 no. 6 pp J.M. Yong A determnstc near quadratc tme-nconstent optma contro probem Mathematca ontro and Reated Reds 2011 vo. 1 no. 1 pp J.M. Yong me-nconsstent optma contro probems and the equbrum HJB equaton Mathematca ontro and Reated Reds vo. 2 no. 3 pp J.M. Yong Determnstc tme-nconsstent optma contro probems an essentay cooperatve approach Acta Mathematcae Appcatae Snca 2012 vo. 28 pp J.M. Yong Dfferenta games a concse ntroducton Word Scentfc Pubsher Sngapore J.M. Yong Lnear-quadratc optma contro probems for mean-fed stochastc dfferenta equatons tme-consstent soutons ransactons of the Amercan Mathematca Socety 2017 vo. 369 pp X.Y. Zhou and D. L ontnuous-tme mean-varance portfoo seecton: a stochastc LQ framewor Apped Mathematcs and Optmzaton 2000 vo. 42 no. 1 pp Appendx A. Proof of Lemma 3.1. From 2.6 and 3.3 we have X ū λ 1 X Φ 1 λ X ū λ 1 X Φ 1 λ X ū λ X Φ = A B Φ Xūλ =1 D Φ D Φ E X λ = B ū =1 D ūw λ = 0 1. X Φ λ Xū λ X Φ λ E X u λ Ā B E X Φ λ w E X u λ

26 26 NI LI ZHANG KRSI X Φ. hen α ū = {α ū } satsfes 3.4. Obvous-. hen we obtan Denote X ū λ λ y we have X ū λ A.1 by α ū = X Φ λα ū J X tx ;Φ X ū λ v tx λū Φ X ūλ v tx 1 J X tx ;Φ X Φ v tx { = 2λE = Φ X Φ ρ Φ α ū X Φ v tx Q α ū E X Φ Q E α ū qα ū R Φ α ū E Φ X Φ R Φ X Φ v tx ρ ū v tx R Φ E α ū G X Φ N F X tx α ū g N E X Φ N Ḡ E α ū N λ 2{ = E α ū Q α ū α ū Φ R Φ α ū E α ū ū E R ū E α ū N G α ū N { = 2λE = Q E X Φ R Φ X Φ E α ū Q E α ū Φ R Φ E α ū } E α ū N Ḡ E α ū N α Q X Φ Φ R Φ X Φ v tx q Φ ρ ū Φ R Φ E X Φ v tx ρ E X Φ N Ḡ E α ū N From 3.4 and 3.5 t foows that { E = = E E v tx α ū ū G X Φ N F X tx α ū g N } λ 2 J0;ū. α Q X Φ Φ R Φ X Φ v tx q Φ ū ρ Q E X Φ R Φ X Φ Φ R Φ E X Φ v tx ρ E X Φ N Ḡ E α ū N = } E { Q Φ R Φ X Φ E E v tx α ū ū G X Φ N F X tx α ū g N A B Φ E Y Φ 1 E Y Φ 1 D Φ E Y Φ 1 w E Y Φ 1 w =1 Y Φ E X Φ Φ R v tx E v tx α E Y Φ ū E α ū Q Φ R Φ E X Φ }

27 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 27 Φ R E v tx q Φ ρ =1 E } A B Φ E Y Φ 1 E Y Φ α ū R Φ X Φ = R Φ X Φ v tx B E Y Φ 1 v tx B E Y Φ 1 From A.1 we can compete the proof. Appendx B. Proof of Lemma 3.3. By smpe cacuaton we have E Y Φ N E Y Φ N = E D Φ E Y Φ 1 w =1 =1 G X Φ N ḠE X Φ N F X tx g D E Y Φ 1 w ρ ū D E Y Φ 1 w ρ ū. = G A B Φ X Φ G Ā B Φ ḠA B Φ E X Φ G B Γ X tx G B ḠB Γ E X tx G B v tx G f F X tx g = G A B Φ E X Φ G B Γ E X tx vtx G f F X tx g E Y Φ N w = E G X Φ N ḠE X Φ N F X tx g w = G =1 δ D Φ X Φ D Φ E X Φ D Γ X tx D Γ E X tx D vtx d E Y Φ N w = G From 3.8 we have =1 D δ D Φ E X Φ Γ E X tx vtx d. Y Φ = Q X Φ Q E X Φ Φ R Φ X Φ Φ R Φ E X Φ Φ R Γ X tx Φ R Γ E X tx vtx A B Φ E Y Φ N Ā B Φ E Y Φ N vtx

28 28 NI LI ZHANG KRSI =1 D Φ E Y Φ N w D Φ E Y Φ N w Φ ρ q { = Q Φ R Φ A B Φ G A B Φ =1 δ DΦ G D Φ } X Φ { Q Φ R Φ A B Φ G Ā B Φ ḠA B Φ Ā B Φ G A B Φ δ DΦ G D Φ =1 D Φ G D Φ } E X Φ { Φ R G A B Φ B δ DΦ } G D Γ X tx =1 vtx {Φ R A B Φ G B ḠB Ā B G Φ B δ D Φ G D =1 D Φ G D } Γ E X tx A B Φ G f F X tx g Ā B Φ G f F X tx g δ DΦ G d =1 vtx D Φ G d Φ ρ q = S X Φ S E X Φ X tx E X txx U X tx π. In the above we appy the property v tx 2 t ;R m. By deducton we can get the

29 desred resut. Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 29 Appendx. Proof of heorem Let Φv tx be a mxed equbrum souton of Probem LQ tx whch satsfes 3.6 and 3.7. By smpe cacuaton we have J0;ū = = E α ū 1 E α ū 1 S 1 α ū 1 E α ū 1 α ū α ū E α ū S α ū E α ū E α ū Q Φ R Φ α ū E α ū.1 E α ū 1 S 1 E α ū 1 E α ū S E α ū E α ū Q Φ R Φ E α ū ū E R ū = E ū O ū = ū O ū. From 3.7 and.1 t hods that nf J0;ū = nf ū O ū 0 ū L 2 F ;Rm ū L 2 F ;Rm whch mpes O 0. hen 3.9 s sovabe. We now prove b and c. Lettng = N 1 n 3.6 and notng E Y Φ N.2 = G A B Φ X Φ E Y Φ N w = G we have G f F X tx =1 g G B v tx δ D Φ X Φ D vtx d 0 = R Φ X tx vtx B E Y Φ N D E Y Φ N w ρ =1 = R B G B δ D G D Φ X tx =1 BG A vtx δ D G =1 B F X tx B G f δ D G d B g ρ =1

30 30 NI LI ZHANG KRSI.3 = O Φ X tx vtx L X tx θ. Here X Φ = X tx and O = R B G B =1 δ D G D L = B G A B F =1 δ D G θ = B G f =1 δ D G d B g ρ. Note that Φv tx s a mxed equbrum souton and X tx s gven n 2.8. As Φ X tx vtx satsfes.3 t hods from Lemma 3.4 that.3 s equvaent to.4 L X tx θ Ran O and for some η R m.5 Φ X tx vtx = O L X tx O θ I O O η. eary.5 s equvaent to.6 v tx = O L Φ X tx O θ I O O η for some η R m. If we repace v tx of.6 by.7 v tx = O L Φ X tx O θ then the new par Φv tx wth v tx gven n.7 can aso serve as a mxed equ- wth vtx gven n.7 n.3 s seected as brum souton. By submttng the par Φ v tx the equatons n.3 are aso satsfed. herefore the v tx.8 v tx = O L Φ X tx O θ. Substtutng ths v tx nto Lemma 3.3 we have Y N 2Φ In ths case t hods that = S N 2 X N 2Φ S N 2 E N 2 X N 2Φ N 2 X tx N 2 E N 2 X tx U N 2X tx N 2 π N 2. E N 2 Y N 2Φ = S N 2 N 2 A N 2N 2 X tx N 2 B N 2N 2 ΦN 2 X tx N 2 vtx N 2 fn 2N 2

31 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 31 U N 2 X tx N 2 π N 2 and E N 2 Y N 2Φ wn 2 p = SN 2 N 2 =1 δ N 2 N 2N 2 Xtx N 2 D N 2N 2 ΦN 2 X tx N 2 vtx N 2 d N 2N 2. herefore we have 0 = R N 2N 2 Φ N 2 X tx N 2 vtx N 2 B N 2N 2 E N 2Y N 2Φ DN 2N 2 E N 2 Y N 2Φ wn 2 ρn 2N 2 = =1 R N 2N 2 B N 2N 2 SN 2 N 2 BN 2N 2 =1 δ N 2 D N 2N 2 S N 2 N 2 D N 2N 2 Φ N 2 X tx N 2 vtx N 2 B N 2N 2 A N 2N 2 B N 2N 2 U N 2 =1 SN 2 N 2 δ N 2 D N 2N 2 S N 2 N 2 N 2N 2 X tx N 2 B N 2N 2 SN 2 N 2 fn 2N 2 =1 δ N 2 D N 2N 2 S N 2 N 2 d N 2N 2 B N 2N 2 π N 2 ρ N 2N 2.9 = O N 2 ΦN 2 X tx N 2 vtx N 2 LN 2 X tx N 2 θ N 2 where X N 2Φ N 2 = X tx N 2 and O N 2 = R N 2N 2 B N 2N 2 SN 2 N 2 BN 2N 2 =1 δ N 2 D N 2N 2 S N 2 N 2 D N 2N 2 L N 2 = BN 2N 2 SN 2 N 2 AN 2N 2 BN 2N 2 U N 2 =1 δ N 2 D N 2N 2 S N 2 N 2 N 2N 2 θ N 2 = BN 2N 2 SN 2 N 2 fn 2N 2 =1 δ N 2 D N 2N 2 S N 2 N 2 d N 2N 2 B N 2N 2 π N 2 ρ N 2N 2. he foowngarguments smar to that between.4 and.8. Note that Φv tx s a mxed equbrum souton and X tx s gven n 2.8. Because Φ N 2 X tx N 2

32 32 NI LI ZHANG KRSI v tx N 2 satsfes.9 we have from Lemma 3.4 that.9 s equvaent to.10 and for some η N 2 R m L N 2 X tx N 2 θ N 2 Ran O N 2 Φ N 2 X tx N 2 vtx N 2 = O N 2 L N 2X tx N 2 O N 2 θ N 2 I O N 2 O N 2 ηn 2 or equvaenty v tx N 2 = O N 2 L N 2 Φ N 2 X tx N 2 O N 2 θ N 2 I O N 2 O N 2 ηn 2. If we repace v tx vtx N 2 of Φvtx by.8 and.11 v tx N 2 = O N 2 L N 2 Φ N 2 X tx N 2 O N 2 θ N 2 then the new par Φv tx s aso a mxed equbrum souton. By repeatng the procedure between.2 and.11 we have the propertes b and c.. For.1 and O 0 we have nf J0;ū = nf ū O ū 0 ū L 2 F ;Rm ū L 2 F ;Rm whch mpes3.7. Furthermore based on Lemma 3.4 and by reversng the procedure of we can assert that Φv tx wth v tx gven n 3.12 s a mxed equbrum souton of Probem LQ tx. Appendx D. Proof of heorem Notethat3.17sequvaenttoÕÕ L X tx θ = L X tx θ t. Lettng = tandtangdfferentxs wehaveõtõ t L t = L t ÕtÕ t θ t = θ t. Because for any tx wth t and x 2 F t;rn Probem LQ tx admts a near feedbac equbrum strategy we must have the sovabty of Furthermore from the sovabty of t s not hard to confrm the exstence of a near feedbac equbrum strategy.. Let Φ = { Õ L } v = { Õ θ }. hen for any tx wth t and x 2 t;r n Φv t s a near feedbac equbrum strategy. v. Let ψ = Φv. hen ths ψ satsfes the property of v. v. We adopt a bacward procedure to prove. Frst ettng t = N 1 then 2.1 reads as D.1 Notng X x JN 1x;u J N 1X x ;ψ X ψ J N 1X x ;u u 2 FN 1;R m. = X ψ = x t foows that = x Q A G A 2A F δ G x =1

33 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 33 2 x A G B F B δ G D f G B =1 δ d G D ρ g B u =1 u R BG B δ D G D u =1 2x q A G f A g δ G d f G f 2g f =1 δ d G d =1 J N 1x;ψ x >. he ast nequaty occurs because ψ F. Based on a emma n 20 we have Õ 0 Õ Õ L = L Õ Õ θ = θ and for any u L 2 F N 1;Rm JN 1x;u = Õu L x θ Õ Õu L x θ x P x2x L Õ θ q A G f δ G d =1 D.2 Here A g f G f δ d G d 2g f θ Õ θ =1 JN 1x;ũ x. Õ = R B G B =1 δ D G D L = B G A B F =1 δ D G θ = B G f ρ B g =1 δ D G d

34 34 NI LI ZHANG KRSI and P = Q A G A δ G L Õ L =1 and ũ x = Õ L x Õ θ. From ths and D.1 D.2 one can seect ψ as Õ L Õ θ.e. ψ x = Õ L x Õ θ. Assume we have obtaned ψ = Φ ṽ 1 namey ψz = Φ z ṽ wth Φ ṽ R m n R m. Let us derve the expresson of ψ. Now consder Probem LQ for the nta par x. By addng to and subtractng three terms =1 =1 2 =1 X u E ψ 1 E X u ψ P1 1 X u ψ 1 E X u ψ 1 X u ψ E X u ψ P X u ψ E X u ψ E X u ψ P1 1 E X u ψ 1 E X u ψ P E X u ψ E σ 1 X u ψ 1 σ X u ψ from J x;u ψ X u ψ 1 we have J x;u ψ X u ψ 1 = E x Q xu R u 2q x2ρ u =1 E X u ψ E X u ψ Q Φ R Φ A B Φ P 1 A B Φ δ D Φ P 1 D Φ P =1 X u ψ E X u ψ E X u ψ Q Φ R Φ A B Φ P 1 A B Φ δ D Φ P 1 D Φ P E X u ψ =1 2 q Φ R ṽ Φ ρ =1 δ D Φ P 1 D ṽ d A B Φ P 1 B ṽ f A B Φ σ 1 σ X u ψ

35 Mxed Equbrum Souton of me-inconsstent Stochastc LQ Probem 35 2ρ ṽ ṽ R t ṽ =1 δ D tṽ d P D ṽ d B t ṽ f P 1 B ṽ f σ1b ṽ f 2x F E X u ψ N E X u ψ 1 E X u ψ 1 P 1 X u ψ 1 E X u ψ 1 E X u ψ 1 P 1 E X u ψ 1 2σ 1E X u ψ 1. Let P = Q Φ R Φ A B Φ P 1 A B Φ =1 δ D Φ P 1 D Φ P = Q Φ R Φ A B Φ P 1 A B Φ =1 δ D Φ P 1 D Φ σ = q Φ R ṽ Φ ρ =1 δ D Φ P 1 Dṽ d A B Φ P 1 B Φ f A B ṽ σ 1 P N = G P N = G σ N = g 1 and γ 1 = =1 2ρ ṽ ṽ R ṽ =1 δ D ṽ d P D ṽ d B ṽ f P 1 B ṽ f σ1b ṽ f F A B Φ B N 2 ṽ N 2 f N 2 F A B Φ A 1 B 1 Φ 1 B ṽ f N 3 = hen t hods that F A B Φ A 1 B 1 Φ 1 f F B ṽ f. J x;u ψ X u ψ 1 = x Q A P 1 A u R B P 1 B 2x A P 1B =1 2u ρ B P 1f 2x q A P 1 f =1 =1 δ P 1 2Ũ 1A x δ D P 1 D u δ P 1 D Ũ 1 B u =1 =1 δ D P 1 d B σ 1 δ P 1 d A σ 1

36 36 NI LI ZHANG KRSI f P 1f =1 J x;ψ X ψ > δ d P 1 d 2σ 1 f γ 1 where Ũ1 = A 1 B 1 Φ 1 A B Φ F. hen we have Õ 0 ÕÕ L = L ÕÕ θ = θ and for any u 2 F ;Rm where J x;u ψ X u ψ 1 J x;ũ xψ X ũ xψ 1 Õ = R B P 1B =1 δ D P 1 D L = B P 1A =1 δ D P 1 B Ũ1 θ = B P 1f =1 δ D P 1 d ρ B σ 1 and ũ x = Õ L x Õ θ. Hence ψ can be seected as Õ L Õ θ.e. ψ x = Õ L x Õ θ. Furthermore S = P S = P σ = π 1 where S S π s gven n 3.24 and By the method of nducton we have the sovabty of and {ψ = Õ L Õ θ } s a feedbac equbrum strategy that s ψ x = Õ L x Õ θ. hs competes the proof.