Novembe 23, 2006 edited Line Contolles e Unifomly Optiml fo the Witsenhusen Counteexmple Michel Rotkowitz 1,2 IEEE Confeence on Decision nd Contol, 2006 Abstct In 1968, Witsenhusen intoduced his celebted counteexmple, which illustted tht when n omtion ptten is nonclssicl, the contolles which optimize n expected qudtic cost my be nonline. It is shown hee tht fo the Witsenhusen Counteexmple, when one insted consides the induced nom, then line contolles e in fct optiml. 1 Intoduction A clssicl omtion ptten ssumes tht, t evey time step, the contolle cn ccess not only omtion fom tht time, but fom ll peceding times s well. When this holds, nd when the dynmics e line, the cost qudtic, nd the noise Gussin, optiml contolles e line. The Witsenhusen Counteexmple [6] showed tht when nonclssicl omtion ptten exists, then nonline contolles my be optiml fo the LQG nom. In this ppe, we will conside the sme poblem setup, but fo the optimum of the nom induced by the 2-nom, sometimes clled unifomly optiml contol. In the cse of the L 1 nom, which is induced by diffeent nom, it ws shown tht [1] line contolles e optiml ove those which e diffeentible in the oigin, by simple gument which would extend to most induced noms including ous. Howeve, it ws lso shown tht when one dops this diffeentibility ssumption, nd cn optimize the contolle fo ech possible diection of the input noise, then such nonline contolle my outpefom ll line contolles [5], even fo centlized omtion ptten. In the cse of unifomly optiml contol, it hs been shown tht line contolles e optiml fo the cen- 1 Resech School of Infomtion Sciences nd Engineeing, The Austlin Ntionl Univesity, Cnbe, Austli, e-mil: otkowitz@stnfodlumni.og) 2 This wok ws pimily completed t the Royl Institute of Technology, Stockholm, Sweden, whee it ws poted by the Euopen Commission though the Integted Poject RUNES nd by the Swedish Sttegic Resech Foundtion though n INGVAR gnt. tlized cse [2, 3], just s in LQG contol. This ppe shows tht line contolles e unifomly optiml s well fo poblem with nonclssicl omtion ptten, nd in pticul, fo the sme poblem which elucidted the possibility of nonline optimlity fo the LQG poblem. 2 Peliminies We intoduce the poblem setup, sevel bsic definitions which will be used thoughout the ppe, nd thei immedite consequences. 2.1 Poblem Fomultion Given noise nd contol lws w = [ w1 w 2 u 1 = γ 1 ) u 2 = γ 2 y 2 ) the system then evolves s follows, s indicted in Figue 1 x 0 = σw 1 v = w 2 = x 0 y 2 = x 1 v u 1 = γ 1 ) u 2 = γ 2 y 2 ) x 1 = x 0 u 1 x 2 = x 1 u 2 nd we wish to keep the following vibles smll ] z 1 = ku 1 z 2 = x 2 We cn then put ll of this togethe to obtin z = [ k γ1 σw 1 ) ] σw 1 γ 1 σw 1 ) γ 2 σw 1 γ 1 σw 1 ) w 2 ) In the oiginl poblem [6], the noise ws nomlly distibuted w N 0, I) nd we seeked γ 1, γ 2 to minimize E z 2 1
Novembe 23, 2006 edited v = w 2 í 1 u 1 Î y 2 í u 2 2 w 1 û x 0 x Î 1 Î à x 2 Figue 1: The Witsenhusen Counteexmple In this ppe, we insted conside the induced nom, nd seek γ 1, γ 2 to minimize z 2 w 0 w 2 nd the optiml cost s J = γ 1 γ 2 Jγ 1, γ 2 ) In ode fo the 2-nom to be consistent with Witsenhusen s objective function, the weighting on u 1 needs to be k. Note when consideing the efeences, howeve, tht some, such s [4], use weighting of k. 2.2 Pol Coodintes Unless we stte othewise, ove is ssumed to be ove π. We stte fily obvious lemm nd omit the poof. Lemm 1. Jγ 1, γ 2 ) is finite only if γ 1 0) = γ 2 0) = 0. We will conside pol coodintes w 1 = cos w 2 = sin such tht the signls pssed to the contolles become = σ cos y 2 = σ cos γ 1 σ cos ) sin The gin fo given contol lws nd given input cn be defined s Q 2 γ 1, γ 2,, ) = z 2 2 w 2 2 k γ1 σ cos ) = σ cos γ 1σ cos ) γ 2σ cos γ 1 σ cos ) sin ) We then define the cost fo given contol lws s Jγ 1, γ 2 ) = >0 Qγ 1, γ 2,, ) 2.3 Definition of Piecewise Affine Contol Lws Fo ny,, b, b R, we define piecewise ffine contol lws If we let ) = such tht if 0 γ 1 ) = if 0 b y 2 if y 2 0 γ 2 y 2 ) = b y 2 if y 2 0 if < 0 if 0 b) = γ 1, )) = ), ) γ 2 y 2, )) = b)y 2, ) b if y 2 < 0 b if y 2 0 2
the gin fo given contol lws of this fom nd given input cn then be defined s Q 2 p,, b, b,, ) k )σ cos ) = )σ cos ) σ cos b)σ cos )σ cos ) sin ) = k )σ cos σ cos 1 ))1 b)) sin 1) Noting tht this is independent of, we heefte efe to the gin s Q p,, b, b, ), nd then define the cost fo given contol lws s J p,, b, b ) = nd the optiml cost s J p =, Q p,, b, b, ) J p,, b, b ) 2.4 Definition of Line Contol Lws We similly define the subclss of line contol lws nd thei costs. Fo ny, b R, we define γ 1 ) = γ 2 y 2 ) = by 2 The gin fo given contol lws of this fom nd given input cn then be defined s Q 2 l, b, ) = the cost fo given contol lws s k σ cos 2) σ cos 1 )1 b) sin J l, b) = nd the optiml cost s J l = Q l, b, ) b J l, b) 3 Line Domintes Piecewise Affine In this section, we show tht the best piecewise ffine contol lw is ctully line, nd lso find the optiml gins. 3.1 Optiml Vlues fo Second Novembe Contolle 23, 2006 edited Let be defined s the point t which > 0 nd y 2 = 0. Thus cos ) > 0 nd σ1 ) cos ) sin ) = 0 o = ctn σ1 )) π/2, π/2) We now focus on the hlf-cicle π/2, fom hee until Section 3.4. The vlue of Q p on this hlf-cicle is unffected by, nd we thus efe to the gin s Q p, b, b, ). We then define the wost-cse cost on the hlf-cicle fo given contol lws s J hc, b, b ) = π/2 nd the optiml such cost s J hc = Q p, b, b, ) J hc, b, b ) At, the gin Q p is unffected by ou choice of b, b, nd so it epesents lowe bound on ou wost cse cost J hc, n ide which is fomlized t the end of this subsection. Theefoe, if we cn choose b, b such tht Q p ) is the mximum ove ll, then it would be n uppe bound s well, nd we will hve found the optiml vlues of b, b. This is only possible if the left nd ight deivtives with espect to e both non-positive. We cn chieve Q p, b, b, ) = = 0 if b = b = b ) whee b ) = σ2 k 2 1 ) 1 σ 2 1 3) Fo given vlue of, it emins to be checked whethe this vlue of b leds to pek o tough t. Due to the peiodicity of Q p we only need to show tht the second deivtive t this point is negtive, nd we will hve the pek ove ll ngles. Since the vlue t is unffected by ou choice of b, b, we hve Q p, b, b, )) which implies = Q p, b ), b ), )) b, b J hc, b, b ) Q p, b ), b ), )) b, b which then implies J hc, b, b ) Q p, b ), b ), )) 4) which futhe implies J hc Q p, b ), b ), )) 5) 3
3.2 Optiml Vlue fo Fist Contolle In this subsection, fo ech vlue of R we ssume tht = ) nd tht b = b = b ) s defined bove. We then seek the vlue of which minimizes Q p, b ), b ), )), the lowe bound fom 4), We define this lowe bound s Q lb ), nd plugging 3) into 1) we find tht Q 2 lb) = σ2 k 2 1 ) 1 σ 2 1 This function is plotted in Figue 2, long with its symptotes. 3.5 3 Q lb ), with k = 0.1, σ = 10 Q lb * = 1 nd Novembe 23, 2006 edited kσ 2 2 2 ) α 2 1 = 2 β 2 α ) α 2 β 2 > 0 which leds to nd β 2 kσ 2 2 1 1 1 = α 2 β 2 > 0 kσ 2 2 2 1 2 = α 2 β 2 < 0 Hence the second deivtive is positive only fo 1, nd Q lb ) must chieve its emum eithe t 1 o t ±. Letting γ = kσ 2 k 1, Q lb 2.5 2 1.5 2 k1) 1/2 Q 2 lb 1 ) = 2k 1)α 2 k 1)β 2 γ 2 2γ kα) α 2 β 2 α 2 k 1)β 2 γ 2 2α α 2 β 2 As ±, Q 2 lb ) k 1. 1 0.5 5 0 5 Figue 2: Q lb ) = Q p, b ), b ), )) Letting α = kσ 2 k 1 nd β = 2σ k, we find tht Q2 lb ) = 0 iff 1, 2 } whee 1 = 2 α α 2 β 2 )/β 2 2 = 2 α α 2 β 2 )/β 2 These points e lso shown in Figue 2, nd we see tht the minimum occus t 1. We now pove tht this is lwys the cse. Noting tht α 2 β 2 = kσ 2 1 2σ 2 1)k 2 2k > 0 nd thus, tht 1, 2 R nd 1, 2 < 0, we cn expess the second deivtive t these points s 2 Q 2 lb ) 2 = 2σ2 kσ 2 2 1) 1 σ 2 1 1, 2} Then, kσ 2 2 2 ) α 1 1 = 2 β 2 α ) α 2 β 2 < 0 β 2 6) k 1) Q 2 lb 1 ) = nd thus kβ 2 α 2 β 2 α 2 k 1)β 2 γ 2 2α α 2 β 2 > 0 Q lb ) = Q lb 1 ) We heefte efe to = 1. We hve now shown Q p, b ), b ), )) = Q p, b ), b ), )) 7) nd combining 5), 7), we hve J hc Q p, b ), b ), )) 8) 3.3 Optiml Cost on Hlf-Cicle We showed tht fo ny, choosing b = b = b ) would yield lowe bound on the induced nom t ), which would be equl to the induced nom if the second deivtive ws negtive. Now tht we hve the vlue of fo which this bound is minimized, we need only check the second deivtive fo tht vlue. 2 Q 2 p, b, b, ) 2 = = 2σ2 k 2 1 ) 1 σ 2 1 kσ 2 2 1) 4
We ve ledy seen in 6) tht nd thus, kσ 2 1 < 0 2 Q 2 p, b, b, ) 2 = < 0 The fist deivtive ws ledy foced to zeo by ou choice of b, nd thus J hc, b ), b )) = Q p, b ), b ), )) This tells us tht which implies Novembe 23, 2006 edited J p Q p,, b ), b ), )) 13) nd thus, combining 12), 13) J p = Q p,, b ), b ), )) In othe wods, the emum is chieved whee = nd b = b, nd the optiml piecewise ffine contolle is line. The optiml line contolle must then esult in the sme optiml cost J hc, b, b ) Q p, b ), b ), )) nd combining 4), 9), we hve 9) J hc, b, b ) = Q p, b ), b ), )) which then implies J hc Q p, b ), b ), )) 10) Lstly, combining 8), 10), we get, y 2, Q p 1 0.5 J l = Q l, b ), )) 14) = *, = *, b = b * * ), b = b * * ), k =0.1, σ =10 J hc = Q p, b ), b ), )) 11) 3.4 Optiml Cost on Full Cicle In this subsection, we show tht this optimum ove the hlf-cicle is lso the optimum ove the whole cicle, nd thus, tht piecewise ffine offes no dvntge ove line. J p J hc 12) = Q p,, b ), b ), )) whee the inequlity holds becuse the emum is tken ove subset in the ltte expession, nd the equlity follows fom 11). Since cos π) = cos nd sin π) = sin it follows fom 1) tht Q p,, b, b, nπ) = Q p,, b, b, ) n Z nd thus, when the contol lws e line the cost my be detemined on the hlf-cicle, nd so J p,, b ), b )) = Q p,, b ), b ), ) = Q p,, b ), b ), ) π 2 = J hc,, b ), b )) = Q p,, b ), b ), )) 0 0.5 1 0.8 0.6 0.4 0.2 0 0.2 0.4 0.6 0.8 1 / π Q p,,b,b,) / 2σ ) y 2 / 2σ ) = 0 y 2 = 0 Figue 3:, y 2, Q p,, b ), b ), ) The vlue of the gin with these optiml vlues is shown long the cicle in Figue 3, nd the signls pssed to the contolles t ech ngle e shown s well. Note tht the wost cse cost indeed occus whee y 2 = 0, tht is, t. 4 Line Domintes All Nonline If we hd ssumed tht γ 1, γ 2 wee meely left- nd ight-diffeentible in the oigin, then we would be done, since the induced nom would lwys be lowe bounded by bitily smll inputs, nd thus only piecewise ffine contolles of the fom consideed in the pevious section would hve to be consideed t ll. But we need not mke ny such ssumption, s shown in this section. The following theoem sttes tht the optiml line contolle chieves the sme pefomnce s the optiml ovell contolle. 5
Theoem 2. J = J l Poof. Since the best contolle is t lest s good s the best line contolle, we clely hve Given ny γ 1, γ 2 : R R, let nd then let nd then choose such tht nd J J l 15) = γ 1 1) 0 = ctn σ1 )) = ) 0 = sec 0 ) / σ 0, 0 ) = σ 0 cos 0 ) = 1 y 2 0, 0 ) = 0 nd thus, using this, Lemm 1, 2), nd 14), espectively, Q 2 γ 1, γ 2, 0, 0 )) ) 2 = kσ cos0 ) σ cos kσ cos 0 ) γ 20) = kσ cos0 ) σ cos kσ cos 0 ) = Q 2 l, b ), )) = J 2 l, b )) which yields Jγ 1, γ 2 ) J l, b )) In othe wods, fo bity γ 1, γ 2 we cn find line contolle which pefoms t lest s well, nd thus 0 J J l 16) 5 Discussion nd Conclusions Novembe 23, 2006 edited We hve shown tht given the sme setup s the Witsenhusen Counteexmple, if ou objective is to optimize the induced nom, then line contolles e optiml. This ws shown to hold tue fo bity vlues of the constnts. We showed in Section 3 tht the best piecewise ffine contolle is line, nd specificlly clculted those contol lws. Then in Section 4, we showed tht fo ny contol lws, one cn find line contolles which pefom just s well, nd thus, tht line contolles e optiml. This ppoch ws ptilly edundnt. It ws not necessy to fist show tht the optiml contolle is optiml mongst ll piecewise ffine contolles, since it is lte shown tht line contolle domintes ny given contol lw. Howeve, it is not yet cle which ppoch will be most useful to genelize these esults to othe omtion stuctues, nd so Section 3 is pesented in its entiety. This esult ises the question of which, if not ll, othe omtion stuctues dmit unifomly optiml contolles which e line. Acknowledgments The utho would like to thnk C. Lngbot fo his excitement upon heing this poblem suggested, nd thus fo the motivtion to evisit it nd solve it. Refeences [1] M. A. Dhleh nd J. S. Shmm. Rejection of pesistent bounded distubnces: Nonline contolles. Systems nd Contol Lettes, 18:245 252, 1992. [2] A. Feintuch nd B. A. Fncis. Unifomly optiml contol of line feedbck systems. Automtic, 215):563 574, 1985. [3] P. P. Khgonek nd K. R. Pool. Unifomly optiml contol of line time-invint plnts: Nonline timevying contolles. Systems nd Contol Lettes, 6:303 308, 1986. [4] S. Mitte nd A. Shi. Infomtion nd contol: Witsenhusen evisited. Lectue Notes in Contol nd Infomtion Sciences, 241:281 293, 1999. [5] A. A. Stoovogel. Nonline l 1 optiml contolles fo line systems. IEEE Tnsctions on Automtic Contol, 404):694 696, 1995. [6] H. S. Witsenhusen. A counteexmple in stochstic optimum contol. SIAM Jounl of Contol, 61):131 147, 1968. nd the desied esult follows fom 15), 16). 6