Dynamic Team Decision Theory. EECS 558 Project Shrutivandana Sharma and David Shuman December 10, 2005

Transcription

1 Dynamc Team Decson Theory EECS 558 Proec Shruvandana Sharma and Davd Shuman December 0, 005

2 Oulne Inroducon o Team Decson Theory Decomposon of he Dynamc Team Decson Problem Equvalence of Sac and Dynamc Teams Concludng Remarks

3 Inroducon Many conrolled sysems have mulple decson makers In general, he ndvdual decson makers dffer n hree ways: They conrol dfferen decson varables They base her decsons on dfferen nformaon They have dfferen goals or obecves A eam s an organzaon n whch here s a sngle goal or payoff common o all decson makers.e., he decson makers dffer n only he frs wo ways above 3

4 Formulaon of he Dynamc Team Decson Problem Fg.. A Pcoral Represenaon y u y u y u y T u T y u y u y u y T u T M K y u M K y u M K y u M K y T u T x 0 x x x - x x T- x T Fne me horzon, T M observaon poss a each me sep K conrol saons a each me sep Informaon avalable o each conrol saon s some subse of all pror observaons and conrol npus a all poss and saons Decson makers have a common obecve u u K u u u K u u u K u u T u T K u T Re-label o Generalze u u uk u K + u u = T* K 4

5 Formulaon of he Dynamc Team Decson Problem (con.) The Team The eam consss of decson makers, DM, =,,, The decson makers ac n a fxed sequence (w.o.l.g. DM, DM,, DM ), Basc Random Varables The basc random varables are represened by he vecor ξ ξ has a known probably dsrbuon, F(ξ), and s ndep. of he conrol law Informaon Srucure Informaon z avalable o each DM s gven by z = η (ξ,u), =,,, η:=(η, η,, η ) s called he nformaon srucure of he eam Admssble decson rules are of he form u = (z ) Common Obecve The obecve s o fnd a decson rule :=(,,, ) o mnmze J() =E {w(ξ,u)}, where w( ) s he oal cos funcon w( ), η( ), and F(ξ) are assumed o be common nformaon, known a pror o all decson makers Sac vs. Dynamc Teams A eam s sad o be sac f η, =,,,, s ndependen of u,.e., z = η (ξ) A eam ha s no sac s sad o be dynamc 5

6 Oulne Inroducon o Team Decson Theory Decomposon of he Dynamc Team Decson Problem Equvalence of Sac and Dynamc Teams Concludng Remarks 6

7 Prelmnary: Precedence Dagrams o Represen Informaon Srucures Defnon u affecs z, wren R, f here exs ξ, u a :=[u,u,,u -,u a,u +,,u ] T, and u b :=[u,u,,u -,u b,u +,,u ] T such ha η (ξ,u a ) η (ξ,u b ). Defnon DM s a preceden of DM, wren DM DM, f ) R, or ) here exs r, s,, such ha Rr, rrs,, R. Defnon 3 Informaon z of DM s nesed n nformaon z of DM, wren Dm DM, f here exss a measurable funcon f g such ha for any ξ, z = f g (z ). Example Team of four DMs ξ = (ξ,ξ ) Informaon Srucure: Z = (ξ ) Z 3 = [ξ,u ] T Precedence Dagram 3 4 Z = ξ + u Z 4 = [ξ,ξ + u,(u 3 ) ] T 7

8 Decomposon Scenaro : Independen Paron Defnon 4 Defnon 5 (H,H,,H K ) s a paron of H f U K = H = H and H H = 0, /, =,,..., K, I An -paron s a paron for whch here s no precedence relaon beween any par of groups and he oal cos funcon s equal o he sum of he cos funcons for each group Formally, a paron (H,H,,H K ) s an -paron of H f (), {,,..., K}, : DM ' H and DM ' H DM ' DM ' () w( ξ, u) w( ξ, ) = K = u H Example H = { DM : =,,3,4} H H 3 4 w( ξ, u) = w ( ξ, u, u) + w ( ξ, u3, u4) Le a eam H have an -paron (H,H,,H K ). Theorem (Yoshkawa) H If Subproblem : {mnmze E { w ( ξ, u H )} wh respec o }, =,,,K has an opmal soluon H, hen = { H, =,,..., K} s an opmal soluon o he orgnal problem H 8

9 Decomposon Scenaro : Sequenal Paron Defnon 6 An s-paron s a paron for whch he nformaon of any group s nesed n ha of he group mmedaely afer, and whch does no conradc he precedence relaon of he groups Formally, a paron (H,H,,H K ) s an s-paron of H f () () {,,..., K } { +,..., K} DM ' H DM ' H : DM ' DM ' {,,..., K } DM ' H DM ' H+ : DM ' DM ' Example 3 H H H 3 Dscusson s-paron can be deermned from precedence dagram alone Dynamc programmng echnque can be appled o a eam wh an s-paron The las group s acons do no affec any of he prevous ones, and can herefore be opmzed ndependenly for any fxed se of prevous acons The cos funcon can hen be updaed accordngly, and he second-o-las group can perform opmzaon The opmal polcy for he enre sysem can be found by proceedng backwards n hs manner See also Theorem n Yoshkawa for furher deals 9

10 z = Hξ + Du DM DM Decomposon Scenaro 3: Paral esedness Assumpon Basc random varables are ndependen, zero mean, and only Gaussan Lnear nformaon srucure z Hz ξ Hξ D, ud, uwhere H and D are known o all DMs T T T Quadrac payoff funcon J(,,... ) = E u Qu+ u Sξ + u c = = + + (0, X) Defnon 7 An nformaon srucure s parally nesed f DM DM mples DM DM The follower can always deduce he acons of s precedens for a gven Theorem (Radner) For sac eams wh above assumpon, he opmal conrol law s gven by: u z Az b b b b c Q Q A H XH S XH T T T T T T = ( ) = +, (,,..., ) =, ( ) = Theorem 3 (Ho & Chu) Consder a dynamc eam wh parally nesed nformaon srucure. zˆ = [{ H ξ DM DM or = }] Ths nformaon srucure s equvalen o nformaon srucure n sac form for any fxed se of conrol laws. z = Hξ + D u DM DM, an Theorem 4 (Ho & Chu) In a dynamc eam wh parally nesed nformaon srucure, he opmal conrol for each member exss, s unque, and s lnear n z 0

11 Oulne Inroducon o Team Decson Theory Decomposon of he Dynamc Team Decson Problem Equvalence of Sac and Dynamc Teams Concludng Remarks

12 Equvalence of Sac and Dynamc Teams I was hypoheszed for a long me ha dynamc eams are more dffcul han sac eams Equvalence Wsenhausen showed n 988 ha all sequenal dscree varable dynamc eam problems and many connuous varable dynamc eam problems are no harder han sac eam problems Hs basc mehodology was o shf he dependence amongs he decson makers nformaon varables no he cos funcon oe ha hs does no make he problem any less complcaed, bu he resul s a sac eam problem Fg.. Mappng from a Dynamc Team o an Equvalen Sac Team L w ( ξ, ( z), ( z), K, ( z )) dpξ dpz, z,..., z = L w( ξ, ( z), ( z), K, ( z )) dpξ f ( ξ, z, z, K, z ) + + Capures Dependence Indep. = dpˆ z = L [ w( ξ, ( z), ( z), K, ( z )) f ( ξ, z, z, K, z ) dpξ ] = L wˆ ( z, z, K, z ) ew Cos Funcon Capures Dependence = dpˆ z = dpˆ z

13 Oulne Inroducon o Team Decson Theory Decomposon of he Dynamc Team Decson Problem Equvalence of Sac and Dynamc Teams Concludng Remarks 3

14 References H.S. Wsenhausen, Separaon of esmaon and conrol for dscree me sysems, Proc. IEEE, vol. 59 (97), Y.C. Ho and K.C. Chu, Team decson heory and nformaon srucures n opmal conrol problems Par I, IEEE Trans. Auoma. Conrol, vol.7 (97), 5- P. Varaya and J. Walrand, On delayed sharng paerns, IEEE Trans. Auoma. Conrol, vol. 3 (978), T. Yoshkawa, Decomposon of dynamc eam decson problems, IEEE Trans. Auoma. Conrol, vol. 3 (978), H.S. Wsenhausen, Equvalen sochasc conrol problems, Mahemacs of Conrol, Sgnals, and Sysems, vol. (988), 3-4

15 Applcaon: Decomposon of he One Sep Communcaon Delay Problem Problem Descrpon Dscree me sochasc sysem wh wo conrol saons: X + = f ( X, U, U, ),,,..., T Y = = h ( X, M ), =, where X,,,, T, M, M,, M T are muually ndep. H H H 3 Precedence Dagram u u u 3 u u u 3 Informaon srucure s one sep delay sharng.e. Z = ( Y, Y,..., Y, Y, U, U,..., U ), =, H T- u T u T Cos funcon descrbed by w = T = c ( X +, X, U ) H T u T u T Analyss of he Problem H = {H, H, H T } shown n he dagram above represens an s-paron Ths nformaon srucure s also parally nesed Eher of hese facs (and her assocaed heorems) can be used o show ha n he case he sysem s lnear, he nose s Gaussan, and he cos funcon s quadrac (LQG), he opmal soluon s affne n he avalable nformaon More generally, Varaya and Walrand showed ha even n he one sep delay sharng whou he LQG assumpon, here s no loss of opmaly n resrcng he search for he opmal * o separaed conrol laws of he form φ (Y,F ), where F s he condonal dsrbuon of X - gven (Y,,Y -,U,,U - ). As n he sochasc conrol problem wh mperfec nformaon, F, he resul of a flerng problem, s ndependen of he conrol law and he cos funcon However, Varaya and Walrand also showed hrough a counerexample ha neher of hese resuls holds n he case of wo or more sep delay sharng (even wh LQG assumpon) 5