Multiple-objective risk-sensitive control and its small noise limit

Transcription

1 Avalable onlne at Automatca 39 (2003) Bref Paper Multple-objectve rk-entve control and t mall noe lmt Andrew E.B. Lm a, Xun Yu Zhou b; ;1, John B. Moore c a Department of Indutral Engneerng and Operaton Reearch, Unverty of Calforna, Berkeley, CA 94720, USA b Department of Sytem Engneerng and Engneerng Management, The Chnee Unverty of Hong Kong, Shatn, NT, Hong Kong c Department of Sytem Engneerng, Autralan Natonal Unverty, Canberra, ACT 0200, Autrala Receved 3 July 2001; receved n reved form 6 Augut 2002; accepted 4 November 2002 Abtract Th paper concerned wth a (mnmzng) multple-objectve rk-entve control problem. Aymptotc analy lead to the ntroducton of a new cla of two-player, zero-um, determntc derental game. The dtnguhng feature of th cla of game that the cot functonal multple-objectve n nature, beng compoed of the rk-neutral ntegral cot aocated wth the orgnal rk-entve problem. More precely, the oppong player n uch a game eek to maxmze the mot vulnerable member of a gven et of cot functonal whle the orgnal controller eek to mnmze the wort damage that the opponent can do over th et. It then hown that the problem of ndng an ecent rk-entve controller equvalent, aymptotcally, to olvng th derental game. Surprngly, th derental game proved to be ndependent of the weght on the derent objectve n the orgnal multple-objectve rk-entve problem. A a by-product, our reult generalze the extng reult for the ngle-objectve rk-entve control problem to a ubtantally larger cla of nonlnear ytem, ncludng thoe wth control-dependent duon term.? 2002 Elever Scence Ltd. All rght reerved. Keyword: Rk-entve control; Multple-objectve optmzaton; Derental game; Hamlton Jacob Bellman equaton; Upper/lower Iaac equaton; Vcoty oluton 1. Introducton The dtnguhng feature of rk-entve control problem that cot functonal nvolve the expectaton of an exponental where the exponent of th exponental the cot functonal of a tandard (rk-neutral) tochatc control problem. One conequence of the exponental term that larger value of the exponent are weghted more heavly. For th reaon, robut (or rk-avere) controller can be obtaned by mnmzng the rk-entve cot. Another Th paper wa not preented at any IFAC meetng. Th paper wa recommended for publcaton n reved form by Aocate Edtor R. Srkant under the drecton of Edtor Tamer Baar. Correpondng author. Tel.: ; fax: E-mal addree: lm@eor.berkeley.edu (A.E.B. Lm), xyzhou@e.cuhk.edu.hk (X.Y. Zhou), john.moore@anu.edu.au (J.B. Moore). 1 The reearch of th author wa upported by the RGC Earmarked Grant CUHK 4054/98E and CUHK 4234/01E. mportant property of the rk-entve control problem t relatonhp wth the cla of two-player, zero-um, determntc derental game aocated wth the o-called H control problem. In th ettng, the controller (for the rk-entve problem) take the part of the mnmzng player n the derental game whle the opponent may be nterpreted a a wort cae dturbance. A a conequence, robutne ue for lnear, nonlnear and tochatc ytem can be tuded n the framework of rk-entve control a well a derental game. For further detal of uch nterpretaton of the H control problem, we refer the reader to Baar and Bernhard (1995). In th paper, we tudy (mnmzng) rk-entve control problem wth multple objectve. Due to the exponental form of the ndvdual cot functonal, however, t dcult to nd an elegant oluton to th problem, and we wll focu ntead on t aymptotc properte (mall noe lmt). In th regard, we explore the connecton between multple-objectve rk-entve control problem and a new cla of determntc derental /03/$ - ee front matter? 2002 Elever Scence Ltd. All rght reerved. do: /s (02)

2 534 A.E.B. Lm et al. / Automatca 39 (2003) game, to be ntroduced n th paper. To make t prece, aymptotc analy lead to the ntroducton of a cla of two-player, zero-um, determntc derental game. The dtnguhng feature of th new cla of problem that t cot multple-objectve n nature, beng compred of the rk-neutral ntegral cot aocated wth the orgnal rk-entve problem. Specfcally, n th derental game a et of ntely many cot functonal gven. Rather than eekng to maxmze a ngle cot functonal (a n the uual derental game), the opponent eek to maxmze the larget member of th et of cot functonal that, the opponent attack the mot vulnerable member of the et, and dregard the other. On the other hand, the orgnal controller tre to mnmze the wort damage that the opponent can do. Naturally, th general tuaton are n many applcaton. We how that the problem of ndng ecent rk-entve controller equvalent, aymptotcally, to olvng the aforementoned determntc derental game. Interetngly, the ame determntc game problem aocated wth derent performance tradeo for the orgnal multple-objectve rk-entve problem. We alo pont out that our reult apply to a large cla of nonlnear tochatc ytem under reaonably mld aumpton. In partcular, our work ncorporate ome farly recent advance on tochatc control n whch the duon term may depend on the tate and/or the control. For a complete account of the latet reult on th topc, the reader drected to the book Yong and Zhou (1999). We hghlght that control dependence of the duon term gve re to nteretng phenomena whch reveal ome eental derence between determntc and tochatc control problem; for ntance, n the tochatc lnear quadratc problem, the control weghtng matrx may be negatve dente, but the problem reman well poed (Chen, L, & Zhou, 1998). Alo, unlke the analy n Jame (1992) for the ngle-objectve problem, we need not aume that the control enter lnearly n the drft (or duon) and certan moothne aumpton on the value functon are not requred n the analy. In fact, one conequence of our analy that many of the reult n Jame (1992) for the ngle objectve problem can be generalzed to a ubtantally larger cla of nonlnear ytem under mlder aumpton. The paper organzed a follow. In Secton 2, we formulate multple-objectve rk-entve control problem, and n Secton 3, we ntroduce a new cla of determntc derental game that are cloely related to the multple-objectve rk-entve problem. In Secton 4, a tranformaton that requred n the ubequent aymptotc analy dcued. In Secton 5, we tudy the aymptotc properte of the value functon aocated wth the multple-objectve rk-entve problem and how that t converge to the value functon of the determntc derental game that wa ntroduced n Secton 3. Fnally, Secton 6 gve ome concludng remark. 2. Multple-objectve rk-entve control 2.1. Statement of mult-objectve rk-entve problem A fundamental problem n many applcaton ndng a control nput that meet the requrement of everal competng objectve, multaneouly. Suppoe that each of l1 objectve repreented by an objectve functon J ( ), =0;:::;l. Suppoe that by lowerng the value of a partcular objectve functonal, the correpondng objectve more uccefully met. A control nput u( ) ad to be ecent f none of the objectve functonal J 0 (u( ));:::;J l (u( )) can be decreaed any further wthout ncreang one of the other. The vector of objectve functonal aocated wth an ecent control ad to belong to the ecent fronter. In th ubecton we formulate the multple-objectve rk-entve control problem under conderaton n th paper. Snce we hall be ung dynamc programmng, we hall work wthn the framework of the weak formulaton for tochatc control (ee Yong & Zhou, 1999) whch we hall ummarze later for reader convenence. Let (; x) [0;T) R n be gven and xed. Conder the followng controlled SDE: dx(t)=b(t; x(t);u(t)) dt (t; x(t);u(t)) db(t); 22 (1) x()=x; where 0 and 0 are parameter and u( ) the U-valued control nput. Note n partcular that unlke the model tuded n Jame (1992), the drft may be nonlnear n u, and the duon may depend on u. For every =0;:::;l, we have an aocated rk-entve cot functonal, whch dened a follow: J (; x; u( )) =E exp 1 [ ] f (t; x(t);u(t)) dt g (x(t)) : (2) We ntroduce the followng aumpton: Aumpton. (A1) U R m compact, and T 0. (A2) The map b :[0;T] R n U R n, :[0;T] R n U R n, f :[0;T] R n U R and g : R n R; ;:::;l, are unformly contnuou and bounded. Alo, there ext a contant L 0 uch that for (t; x; u) =b(t; x; u), (t; x; u), f (t; x; u), g (x), (t; x; u) (t; y; u) 6 L x y ; t [0;T]; x;y R n ; u U: (3) The cla of admble control (n the weak formulaton) a follow (ee Yong & Zhou, 1999, for more detal). For every xed [0;T), let U[; T] denote the et of 5-tuple (; F;P;B( );u( )) atfyng: (1) (; F;P) a complete

3 A.E.B. Lm et al. / Automatca 39 (2003) probablty pace; (2) B(t) t a 1-dmenonal tandard Brownan moton dened on (; F;P) over [; T ], and F t = B(r) 6 r 6 t, augmented by all the P-null et n F; (3) u :[; T] U an F t t -progrevely meaurable proce on (; F;P); and (4) Under u( ), for any ntal condton x R n, the SDE (1) admt a unque oluton x( )on(; F; F t t ;P), and the ntegral n the cot functonal (2) are well-dened wth th oluton. If t clear from the context, we wll wrte u( ) U[; T] a horthand for 5-tuple (; F;P;B( );u( )) U[; T ]. Let R l1, 0 be gven and xed. For every uch, conder the cot functonal L (; x; ; u( )) = J (; x; u( )) (4) and the followng tochatc control problem: Problem. For a gven (; x) [0;T) R n and R l1, 0, nd ( ; F; P; B( ); u( )) U[; T ] uch that: L (; x; ; u( )) = mn u( ) U[;T ] L (; x; ; u( )): (5) The orgnal multple-objectve optmal control problem aocated wth (1) and (2) to nd, for every gven (; x) [0;T) R n, a 5-tuple ( ; F; P; B( ); u( )) U[; T] whch ecent for J0 (; x; u( ));:::;J l (; x; u( )). In the cae when the cot functonal J (; x; u( )) are convex n u( ) and the SDE (1) lnear n x( ) and u( ) (a typcal uch cae the o-called lnear quadratc control), the entre ecent fronter aocated wth (1) and (2) can be traced out by olvng the optmal control problem (5) wth derent R l1, 0, 0. On the other hand, f the problem not convex, the optmal control to (5) wll generally only gve re to a ubet of the ecent fronter (ee Chankong & Hame, 1983; Yu, 1971, for detal). Neverthele, (5) tll play a key role n the multple-objectve problem, and we hall focu our attenton on problem of th form. Later, t wll be hown that the oluton of th problem ndependent, aymptotcally, of. In the paper Lm and Moore (1997), Lm and Zhou (1999), the multple-objectve lnear quadratc (LQ) control problem tuded. Snce a lnear combnaton of convex quadratc functonal alo convex quadratc, the Lagrangan L (; x; ; u( )) reultng from ntegral quadratc functonal J (; x; u( )) alo a convex ntegral quadratc functonal. Therefore, n the LQ cae, (5) a tandard LQ problem, and ecent control are completely obtaned by olvng LQ problem. On the other hand, a lnear combnaton of rk-entve cot functonal J (; x; u( )) cannot be reduced to omethng mpler. It eem that elegant oluton cannot be obtaned, even n the mplet cae. For th reaon we hall focu on the aymptotc properte of th problem Hamlton Jacob Bellman equaton In th ubecton, we preent the Hamlton Jacob Bellman (HJB) equaton aocated wth (1) and (5). Due to contrant on paper length, we have removed the ummary of denton and reult from the theory of vcoty oluton of PDE from th veron of the paper. Th ummary can be found n the appendx of Lm and Zhou (2001). For a detaled dcuon of vcoty oluton the reader drected to the paper Crandall, Evan, and Lon (1984) and Crandall and Lon (1983), and book Flemng and Soner (1993) and Yong and Zhou (1999). For techncal reaon (the requrement that termnal condton are bounded n the theory of vcoty oluton), t convenent to replace the cot functonal (2) wth an equvalent expreon whch we now ntroduce. By (A2), the functonal f are unformly bounded. Hence, there a contant K uch that f (t; x(t);u(t)) dt 6 K; (; x) [0;T) R n ; u( ) U[; T]: (6) Let : R R be any mooth, unformly Lpchtz functonal uch that: 2K ; y (2K; ); (y)= y; y [ 2K; 2K]; (7) 2K ; y ( ; 2K): Due to the unform bound (6), we may replace (2) by J (; x; u( )) = E exp 1 [ ( ) ] f (t; x(t);u(t)) dt g (x(t )) : (8) In order to tudy (5) (wth (2) replaced by (8)) ung dynamc programmng, we make the followng problem tranformaton. For convenence, we hall denote y(t) = [y 0 (t);:::;y l (t)] wth f(t; x; u) dened mlarly. For every (; x; y) [0;T] R n R l1, conder the optmal control problem wth dynamc: dx(t)=b(t; x(t);u(t)) dt (t; x(t);u(t)) db(t); (9) 22 dy(t) =f(t; x(t); u(t)) dt; x()=x; y()=y and cot: L (; x; y; ; u( )) =E exp 1 [g (x(t)) (y (T))] : (10)

4 536 A.E.B. Lm et al. / Automatca 39 (2003) The HJB equaton aocated wth (9) and (10) vt (t; x; y) nf v u x(t; x; y) b(t; x; u) vy (t; x; y) f (t; x; u) 4 2 tr[(t; x; u)(t; x; u) v xx(t; x; y)] =0; (t; x; y) [0;T) R n R l1 v (T; x; y)= exp 1 [g (x) (y )]; (x; y) R n R l1 : (11) We have the followng reult. Theorem 2.1. Suppoe that (A1) and (A2) hold. Then v (; x; y) = nf u( ) U[;T ] L (; x; y; ; u( )); (; x; y) [0;T] R n R l1 (12) the unque bounded vcoty oluton of (11), where L (; x; y; ; u( )) gven by (10) and (x( );y( )) the unque oluton of (9) correpondng to u( ) U[; T ]. Proof. Snce (A1) and (A2) hold and the termnal cot (10) unformly bounded, from Yong and Zhou (1999, Chapter 4, Theorem 5.2 and 6.1) the reult follow mmedately. Throughout th paper, for any gven and xed R l1 uch that 0, we hall refer to v (; x; y) a dened by (12) a the value functon aocated wth (9) and (10). 3. A derental game In th ecton, we ntroduce a two-player, zero-um, determntc derental game whch characterzed by a nontandard cot functonal (14) below. We how n Secton 5 that th derental game cloely related to the multple-objectve rk-entve problem ntroduced n Secton 2.1. Once agan, we aume throughout that the functon b, and f atfy (A2). We hall adopt the derental game formulaton of Ellott Kalton; ee Ellott and Kalton (1972) and Evan and Sougand (1984), for example. Suppoe that the ytem dynamc are governed by the ordnary derental equaton (ODE): ẋ(t)=b(t; x(t);u(t)) (t; x(t);u(t))w(t); (13) x()=x; where u( ) theu-valued nput of player 1 (the control player), and w( ) thew-valued nput of player 2 (the dturbance player, or opponent). Let R l1, 0be gven and xed. The cot aocated wth nput u( ) and w( ) gven by J (; x; u( );w( )) =max f (t; x(t);u(t)) dt g (x(t)) 2 w(t) 2 dt : (14) The et of admble control/dturbance for player 1 and 2 are U d [; T]=u( ): [; T] U u( ) B[; T]-meaurable; (15) W d [; T]=w( ): [; T] W w( ) B[; T]-meaurable; (16) repectvely, where B[; T] denote the Borel -algebra on [; T]. We make the followng aumpton: Aumpton. (A1) : U R m and W R are compact, and T 0. In addton, for any t (; T], two admble nput u 1 ( ), u 2 ( ) U d [; T], are ad to be equvalent on [; T]fu 1 ( )= u 2 ( )on[; t]. We hall denote th by u 1 ( ) u 2 ( )on[; t]. (Smlarly for dturbance). We note that under (A1) and (A2), the ODE (13) ha a unque oluton x( ) for every u( ) U d [; T] and w( ) W d [; T]. We are ntereted n the upper derental game aocated wth (13) and (14), whch call for the denton of a trategy for player 2. The cla of admble tratege for player 2 d [; T]=: U d [; T] W d [; T] for every t [; T ] u 1 ( ) u 2 ( ) on[; t] [u 1 ( )] [u 2 ( )] on [; t]: (17) The upper derental game aocated wth (13) and (14) can be tated a follow: Fnd ( u( ); w( )) U d [; T ] W d [; T] uch that V (; x):=j (; x; u( ); w( )) = nf up u( ) U d [;T ] [ ] d [;T ] J (; x; u( );[u( )]): (18) V known a the upper value of the derental game. A n the rk-entve cae, the boundedne of f n (A2) allow u to replace (14) wth an equvalent expreon nvolvng the functon ( ) a dened by (7). To employ

5 A.E.B. Lm et al. / Automatca 39 (2003) the dynamc programmng approach to the derental game jut ntroduced, let y R l1 be gven and conder the ODE: ẋ(t)=b(t; x(t);u(t)) (t; x(t);u(t))w(t); ẏ(t)=f(t; x(t);u(t)); x()=x; y()=y and cot: J (; x; y; u( );w( )) =max (y (T)) g (x(t )) 2 w(t) 2 dt : (19) (20) Clearly, (13) and (14) correpond to the pecal cae y =0 n (19) and (20). The upper Iaac equaton aocated wth (19) and (20) V t nf up (b w) V x f V y 2 w 2 =0; u U w W (t; x; y) [0;T) R n R l1 ; V (T; x; y) = max g (x) (y ); (x; y) R n R l1 : (21) We have the followng characterzaton of the oluton of (21). Theorem 3.1. Under (A1) and (A2), V (; x; y) = nf up u( ) U d [;T ] [ ] d [;T ] J (; x; y; u( );[u( )]); (; x; y) [0;T] R n R l1 (22) the unque vcoty oluton of (21), where J (; x; y; u( ); w( )) gven by (20) and (x( );y( )) the oluton of (19) correpondng to (u( );w( )) U d [; T ] W d [; T ]. Proof. Th reult can be hown by followng the argument n Flemng and Sougand (1989). We note that (13) and (14) (mlar comment apply to (19) and (20)) a derental game problem n whch player 2 chooe an nput w( ) W d [; T ] to maxmze the cot J (; x; u( );w( )) by makng the larget of the term J (; x; u( );w( )) = f (t; x(t);u(t)) dt g (x(t )) 2 w(t) 2 dt; a large a poble; that, ntutvely, w( ) nct damage by attackng the cot functonal J (; x; u( );w( )) that t can do mot harm to (the mot vulnerable one), and gnore the other. On the other hand, player 1 chooe the nput u( ) U d [; T] whch mnmze the wort damage that w( ) can do over all thee J (; x; u( );w( )). 4. Tranformaton To how the relatonhp between the multple-objectve rk-entve problem (9) and (10) and the derental game (19) and (20), we ntroduce the followng tranformaton: V (t; x; y)= ln v (t; x; y): (23) In th ecton, we prove everal reult on the properte of V. Thee are requred n our tudy of the aymptotc behavor of V and the correpondence between the multple-objectve rk-entve problem and the derental game. Frt we recall that D 1;2; t;(x;y) V (; x; y) (repectvely D 1;2; t;(x;y) V (; x; y)) denote the (econd-order parabolc) uper-(repectvely, ub-) derental of V ; ee Yong and Zhou (1999, p. 191) for denton. We ntroduce the followng aumpton. Aumpton. (A3): (t; x; u)=(t; u), and b(t; x; u), f (t; x; u) =0;:::;l are derentable n x. Propoton 4.1. Suppoe that Aumpton (A1) (A3) hold. Let V (; x; y) be dened by (23). Then there ext K, ndependent of (; x; y) [0;T] R n R l1 and 0, uch that p 6 K, for all (q; p; P) D 1;2; t;(x;y) V (; x; y) D 1;2; t;(x;y) V (; x; y). Proof. The proof follow along the ame lne a the proof of Propoton IV.1 n Lm and Zhou (2001). Propoton 4.1 a generalzaton of Lemma 2.1 n Jame (whch proved ung dea from Flemng (1971)). However, certan key derence hould be noted. Frtly, our reult doe not rely on the extence of an optmal control u( ) and an optmal dturbance trategy [ ]. Secondly, unlke Lemma 2.1, the duon term n Propoton 4.1 may depend on u( ). Fnally, we have not aumed that V mooth. The next reult characterze V a the vcoty oluton of a certan PDE: Propoton 4.2. Suppoe that (A1) (A3) hold. Let 0 and R l1, 0 be gven and xed. Let v be the value functon of the rk-entve problem (9) and (10) and V = ln v. Then there ext a compact ubet W R whch ndependent of 0 uch that V the unque

6 538 A.E.B. Lm et al. / Automatca 39 (2003) vcoty oluton of the PDE: V t (t; x; y) nf u U up w W Vx (t; x; y) (b(t; x; u)(t; u)w) Vy (t; x; y) f (t; x; u) 2 w tr[(t; u)(t; u) Vxx(t; x; y)] =0; (t; x; y) [0;T) R n R l1 ; V (T; x; y)=ln exp 1 [g (x) (y )] ; (x; y) R n R l1 : (24) Proof. It follow mmedately from Theorem 3.1 that V the unque bounded vcoty oluton of the PDE (24) but wth W = R. To ee that W may be replaced by an -ndependent compact ubet of R, conder the followng. For every (q; p; P) D 1;2; t;(x;y) V (; x; y) where p =(p (1) ;p (2) ) R n R l1, we have q nf u U up w W p (1) (b(t; x; u)(t; u)w) p (2) f (t; x; u) 2 w (t; u) P(t; u) 0; (25) where the maxmzng w n (25) gven by w = p(1) (t; u): (26) Snce (t; u) unformly bounded on [0; T] U (Aumpton (A2)), and p bounded (unformly n (t; x; y) and 0) for every (q; p; P) D 1;2; t;(x;y) V (t; x; y) D 1;2; t;(x;y) V (t; x; y) (Propoton 4.1), t follow that w = p(1) (t; u) W; (t; x; y) [0;T] R n R l1 ;u U for ome compact W R, whch ndependent of 0. Therefore, we may replace W = R wth any compact ubet W R (wth W W ), and the maxmzng w n (25) wll tll be gven by (26). It eay to ee that the ame argument apple for the cae of ub-oluton. 5. Aymptotc reult: 0 Clearly, t follow from (A2) that the termnal value V (T; x; y) of(21) and V (T; x; y) of(24) are contnuou. The followng unform convergence reult requred n the proof of Theorem 5.1. Lemma 5.1. Suppoe that (A2) hold. Let R l1 be gven uch that 0. Then ln exp 1 [g (x) (y )] maxg (x) (y ) a 0; (27) unformly on compact ubet of R n R l1. Proof. In fact, we how that convergence unform on the whole pace R n R l1. Frt, note that there a contant K uch that ln K for every 0. Alo, let u denote (x) := arg maxg (x) (y ): Snce ( ) ln maxg (x) (y ) ln exp 1 [g (x) (y )] ln (x) maxg (x) (y ); t follow that ( ) ln ln exp 1 [g (x) (y )] maxg (x) (y ) K whch mple the reult. It nteretng to note that the lmt n (27) doe not depend on the value of the parameter, whch account for the eemngly urprng fact that the mall noe lmt of the multple-objectve rk-entve problem ndependent of the weght on the derent objectve (ee Theorem 5.1 below). The followng aymptotc reult how the relatonhp between the rk-entve problem (9) and (10) and the dfferental game (19) and (20). The convergence proof follow the general method of Barle and Perthame (1988). In partcular, the noton of oluton that ued n th approach (and n the proof of Theorem 5.1) the generalzed denton of a dcontnuou vcoty oluton. Th requred nce the functon (29) and (33) below are only em-contnuou n general. In addton, the proof ue a

7 A.E.B. Lm et al. / Automatca 39 (2003) comparon theorem for em-contnuou vcoty ub- and uper-oluton. The denton of a dcontnuou vcoty oluton qute mlar to that of a contnuou oluton. The reader hould refer to Flemng and Sougand (1989, Chapter VII) for a detaled decrpton of the Barle and Perthame method. The denton of a dcontnuou vcoty oluton a well a the comparon theorem for em-contnuou uband uper-oluton can alo be found there. Theorem 5.1. Aume that (A1) (A3) hold. For every 0, let v be the value functon of the multple-objectve rk-entve problem (9) and (10), a dened by (12), and V = ln v. Let W R be the compact et a determned n Propoton 4.2, and V be the upper value of the aocated determntc derental game (19) and (20). Then V and V are the unque vcoty oluton of (21) and (24), repectvely. Moreover, lm V (t; x; y)=v (t; x; y); (28) 0 unformly on compact ubet. Proof. Let W R be a tated n the Theorem. By Theorem 3.1, V the unque vcoty oluton of (21) and the upper value of (19) and (20). Alo, by Propoton 4.2, V the unque vcoty oluton of (24). To prove (28), for every (t; x; y) [0;T] R n R l1, dene V (t; x; y) = lm up V (; x; y): (29) 0; t; x x; y y Snce V (; x; y) unformly bounded on compact ubet, t follow that V well dened and upper-em-contnuou. We now prove that V a vcoty ub-oluton of (21). Frt, nce V (T; x; y) V (t; x; y) unformly on compact ubet, ee Lemma 5.1, t follow from Flemng and Sougand (1989, Propoton VII.5.1) that V (T; x; y) = max g (x) (y ): (30) Let C ((0;T) R n R l1 ). Suppoe that V ha a local maxmum at (t 0 ;x 0 ;y 0 ) (0;T) R n R l1. It follow from (29) that there ext a equence (t ;x ;y ) uch that V ha a local maxmum at (t ;x ;y ), (t ;x ;y ) (t 0 ;x 0 ;y 0 ) and V (t ;x ;y ) V (t 0 ;x 0 ;y 0 ). Snce V a vcoty ub-oluton of (24), t follow that at every (t ;x ;y ) (denotng = (t ;x ;y ) etc.), we have t nf u U up w W f 4 2 tr( xx)(b w) x y 2 w 2 0 (31) for every 0. Hence, by the contnuty properte of, the convergence properte of (t ;x ;y ) and Aumpton (A1) (A2), we can let 0 whch gve tnf up (bw) x f y 2 w 2 0 u U w W (32) at (t 0 ;x 0 ;y 0 ). Therefore, t follow from (29) and (32) that V a vcoty ub-oluton of (21). In the ame way, the functon V (t; x; y) = lm nf V (; x; y) (33) 0; t; x x; y y well dened, lower em-contnuou, ate the termnal condton V (T; x; y) = max g (x) (y ); (ung Lemma 5.1 and (Flemng and Sougand, 1989, Propoton VII.5.1) a above) and a vcoty uperoluton of (21). By the denton of V and V, we have V V. On the other hand, the comparon theorem for em-contnuou vcoty ub- and uper-oluton, ee Flemng and Sougand (1989, Theorem VII.8.1), gve V 6 V. Therefore, Ṽ := V = V contnuou on [0;T] R n R l1, and Ṽ (t; x; y) = lm 0 V (t; x; y) unformly on compact ubet. Fnally, Ṽ V by the unquene of vcoty oluton to (21). Th complete the proof. It nteretng that V ndependent of the actual value of whch can be vewed, n the rk-entve ettng, a a weght for the th crtera: the hgher the value of the more mportant the th crtera become wth derent value of repreentng derent performance trade-o between the l 1 crtera. Theorem 5.1 ugget that varou derent performance tradeo for the multple-objectve rk-entve problem are equvalent, aymptotcally, to the ame lmtng determntc derental game. Th eem to be qute urprng and counter-ntutve. However, the multple-objectve nature of the orgnal problem ndeed captured by the cot functonal (20) aocated wth the lmtng derental game. The proof of Theorem 5.1 analogou to t ngleobjectve counterpart, namely, Theorem 4.1 n Jame (1992). One derence, however, hould be noted. In Jame (1992), t aumed that V a clacal oluton of the aocated PDE. Th fact then ued n the convergence proof to etablh the parallel nequalty to (31). However, t can be een from the proof of Theorem 5.1 that moothne of V, whch a well known not a reaonable aumpton, not eental for the reult to hold. In partcular, we have only made aumpton whch guarantee that V a vcoty oluton of (24).

8 540 A.E.B. Lm et al. / Automatca 39 (2003) The (upper) derental game (13) and (14) play an mportant role n robut control. (For a detaled account on the role of derental game n robut control, the reader referred to the book Baar and Bernhard (1995)). The relatonhp between the (ngle-objectve) rk-entve control problem and the upper value of the derental game (13) and (14) wa rt etablhed n Jame (1992). The ngle-objectve problem a pecal cae of the problem we are tudyng. One contrbuton of th paper to how that the reult n Jame (1992) hold under mlder condton for a gncantly larger cla of ytem. In partcular, the dffuon term may depend on the control. Alo, we need not aume that the drft term n (1) lnear n the control. 6. Concluon In th paper, we have tuded the relatonhp between multple-objectve rk-entve control problem and a cla of new determntc derental game ntroduced n th paper. A urprng and nteretng ndng that derent tradeo n the multple-objectve problem become unmportant aymptotcally. In other word, a the noe approache to zero, the whole ef- cent fronter hrnk to a ngle pont, whch can be determned by olvng the lmtng derental game problem. Although we have focued on multple-objectve problem, we have alo been able to generalze ome known reult for the ngle-objectve cae. On the other hand, unlke the tandard (rk-neutral) tochatc lnear quadratc cae (Chen et al., 1998), when the duon depend on the control, the lnear quadratc rk-entve problem and the lnear quadratc tochatc and determntc derental game have not been olved, even n the ngle-objectve cae. They are nteretng open problem. Reference Barle, G., & Perthame, B. (1988). Ext tme problem n optmal control and vanhng vcoty oluton of Hamlton Jacob equaton. SIAM Journal on Control and Optmzaton, 26, Baar, T., & Bernhard, P. (1995). H optmal control and related mnmax degn problem: A dynamc game approach. Boton: Brkhauer. Chankong, V., & Hame, Y. Y. (1983). Multobjectve decon makng: Theory and methodology. New York: North-Holland. Chen, S. P., L, X. J., & Zhou, X. Y. (1998). Stochatc lnear quadratc regulator wth ndente control weght cot. SIAM Journal on Control and Optmzaton, 36, Crandall, M. G., Evan, L. C., & Lon, P. L. (1984). Some properte of vcoty oluton of Hamlton Jacob equaton. Tranacton of the Amercan Mathematcal Socety, 282, Crandall, M. G., & Lon, P. L. (1983). Vcoty oluton of Hamlton Jacob equaton. Tranacton of the Amercan Mathematcal Socety, 277, Ellott, R. J., & Kalton, N. J. (1972). The extence of value n derental game, Memor of the Amercan Mathematcal Socety, No Evan, L. C., & Sougand, P. E. (1984). Derental game and repreentaton formula for oluton of Hamlton Jacob Iaac equaton. Indana Unverty Mathematc Journal, 33, Flemng, W. H. (1971). Stochatc control for mall noe ntente. SIAM Journal on Control and Optmzaton, 9, Flemng, W. H., & Soner, H. M. (1993). Controlled Markov procee and vcoty oluton. New York: Sprnger. Flemng, W. H., & Sougand, P. E. (1989). On the extence of value functon of two player, zero-um tochatc derental game. Indana Unverty Mathematc Journal, 38(2), Jame, M. R. (1992). Aymptotc analy of nonlnear tochatc rk-entve control and derental game. Mathematc of Control, Sgnal and Sytem, 5, Lm, A. E. B., & Moore, J. B. (1997). A qua-eparaton theorem for LQG optmal control wth IQ contrant. Sytem & Control Letter, 32, Lm, A. E. B., & Zhou, X. Y. (1999). Stochatc optmal LQR control wth ntegral quadratc contrant and ndente control weght. IEEE Tranacton on Automatc Control, 44(7), Lm, A. E. B., & Zhou, X. Y. (2001). Rk-entve control wth HARA utlty. IEEE Tranacton on Automatc Control, 46(4), Yong, J., & Zhou, X. Y. (1999). Stochatc control: Hamltonan ytem and HJB equaton. New York: Sprnger. Yu, P. L. (1971). Cone convexty, cone extreme pont and nondomnated oluton n decon problem wth multple objectve. Journal of Optmaton Theory and Applcaton, 7, Andrew Lm obtaned h Ph.D. n Sytem Engneerng from the Autralan Natonal Unverty n He ha conducted reearch at a number of nttuton ncludng the Chnee Unverty of Hong Kong, the Unverty of Maryland (College Park), and Columba Unverty (New York). Durng the academc year, he wa an atant profeor n the Indutral Engneerng and Operaton Reearch Department at Columba Unverty, and joned the Indutral Engneerng and Operaton Reearch Department at the Unverty of Calforna (Berkeley) n H reearch and teachng nteret nclude tochatc control, backward tochatc derental equaton, Markov decon procee, and appled probablty. In recent year, he ha focued on applcaton n Fnancal Engneerng. Xun Yu Zhou got h B.Sc. n pure mathematc n 1984 and h Ph.D. n operaton reearch and control theory n 1989, both from Fudan Unverty. Durng and , he wa dong potdoctoral reearche at Kobe Unverty and Unverty of Toronto, repectvely. He joned the Chnee Unverty of Hong Kong n 1993 and now a full Profeor. H reearch nteret are n optmal tochatc control, mathematcal nance/nurance, and dcrete-event manufacturng ytem. A Prncpal Invetgator of numerou reearch grant, he ha publhed 60 journal paper, 1 monograph, and 2 edted book. He wa awarded Alexander von Humboldt Fellowhp of Germany n 1991 and DAAD Reearch Fellowhp of Germany n He a enor member of IEEE, a member of SIAM, and on the edtoral board of Operaton

9 A.E.B. Lm et al. / Automatca 39 (2003) Reearch (1999 preent), IEEE Tranacton on Automatc Control ( ), and Mathematcal Fnance (2001 preent). John B. Moore wa born n Chna n He receved h bachelor and mater degree n Electrcal Engneerng n 1963 and 1964, repectvely, and h doctorate n Electrcal Engneerng from the Unverty of Santa Clara, CA, n He wa apponted Senor Lecturer at the Electrcal Engneerng Department, Unverty of Newcatle, UK n 1967, and promoted to Aocate Profeor n 1968, and full Profeor (peronal char) n He wa Department Head for the perod In 1982, he wa apponted a a Profeoral Fellow n the Department of Sytem Engneerng, Reearch School of Phycal Scence, Autralan Natonal Unverty, Canberra and promoted to Profeor n He ha been head of the department nce , 2002 preent. The department now located n the Reearch School of Informaton Scence and Engneerng. Profeor Moore a Fellow of the Autralan Academy of Technologcal Scence and the Autralan Academy of Scence.