Optimal State-Feedback Control Under Sparsity and Delay Constraints

Transcription

1 Opimal Sae-Feedback Conrol Under Spariy and Delay Conrain Andrew Lamperki Lauren Leard 2 3 rd IFAC Workhop on Diribued Eimaion and Conrol in Neworked Syem NecSy pp , 22 Abrac Thi paper preen he oluion o a general decenralized ae-feedback problem, in which he plan and conroller mu aify he ame combinaion of delay conrain and pariy conrain. The conrol problem i decompoed ino independen ubproblem, which are olved by dynamic programming. In pecial cae wih only pariy or only delay conrain, he conroller reduce o exiing oluion. Noaion y : Time hiory of y: {y, y,..., y }. a b Direced edge from node a o node b. d ij Shore delay from node j o node i. M r Block ubmarix M ij i,j r, e.g. M M 2 M 3 M 2 M 22 M 23 M 3 M 32 M 33 Inroducion {2,3},{} = [ ] M2. M 3 Thi paper udie a cla of decenralized linear quadraic conrol problem. In decenralized conrol, inpu o a dynamic yem are choen by muliple conroller wih acce o differen informaion. In hi work, he dynamic yem i decompoed ino ubyem, each wih a correponding ae and conroller. A given conroller ha immediae acce o ome ae, delayed acce o ome oher, and no acce o he re. Since ome conroller canno acce cerain ae componen, he ranfer marix for he overall conroller mu aify pariy conrain. While decenralized ynhei i difficul in general, ome clae of problem are known o be racable and can be reduced o convex, albei infinie dimenional, opimizaion problem. See [3] and [5]. Thee paper alo A. Lamperki i wih he Deparmen of Conrol and Dynamical Syem a he California Iniue of Technology, Paadena, CA 925 USA. andyl@cd.calech.edu 2 L. Leard i wih he Deparmen of Auomaic Conrol a Lund Univeriy, Lund, Sweden. lauren.leard@conrol.lh.e The econd auhor would like o acknowledge he uppor of he Swedih Reearch Council hrough he LCCC Linnaeu Cener ugge mehod for olving he opimizaion problem. The former give a equence of approximae problem whoe oluion converge o he global opimum, and he laer ue vecorizaion o conver he problem o a much larger one ha i unconrained. An efficien LMI mehod wa alo propoed by [4]. The cla of problem udied in hi paper have parially need informaion conrain, which guaranee ha linear opimal oluion exi, []. Explici aepace oluion have been repored for cerain pecial cae. For ae-feedback problem in which he conroller ha pariy conrain bu i delay-free, oluion were given in [6], and [7]. For ae-feedback wih delay bu no pariy, a pecial cae of he dynamic programming argumen of hi paper wa given by [2]. In Secion 2 and 3, we ae he problem and explain he ae and inpu decompoiion we will ue. Main reul appear in Secion 4, followed by dicuion and proof. 2 Problem Saemen In hi paper, we conider a nework of dicree-ime linear ime-invarian dynamical yem. We will illurae our noaion and problem eup hrough a imple example, and hen decribe he problem eup in i full generaliy. Example. x + x 2 + x 3 + Conider he ae-pace equaion A A 2 x = A 2 A 22 x 2 A 3 A 32 A 33 x 3 B B 2 + B 2 B 22 B 3 B 32 B 33 u u 2 u 3 + w w 2 w 3 For quaniie wih a ime dependence, we ue ubcrip o pecify he ime index, while upercrip denoe ubyem or e of ubyem. Thu, x i i he ae of ubyem i a ime. The u i are he inpu, and he w i are independen Gauian diurbance. The goal i o chooe a ae-feedback conrol policy ha minimize

2 2 3 Figure : Nework graph for Example. Each node repreen a ubyem, and he arrow indicae he pariy of boh he dynamical ineracion a well a he informaion conrain 3. Addiionally, he label indicae he propagaion delay from one conroller o anoher. he andard finie-horizon quadraic co T E = [ x u ] T [ ] [ ] Q S x S T + x T R T Q f x T 2 wih he uual requiremen ha R i poiive definie, while Q SR S T and Q f are poiive definie. Alo, differen conroller may have acce o differen informaion. For he purpoe of hi example, uppoe he dependencie are a follow u u = γ x :, x 2 : u 2 = γ 2 x :, x 2 : u 3 = γ 3 x :, x 2 :, x 3 : 3 Noe ha here i a combinaion of pariy and delay conrain; ome ae informaion may never be available o a paricular conroller, while oher ae informaion migh be available bu delayed. I i convenien o viualize hi example uing a direced graph wih labeled edge. We call hi he nework graph. See Fig.. Noe ha in Example, boh he plan and he conroller 3 hare he ame pariy conrain. Namely, x 3 doe no influence x or x 2 via he dynamic, nor can i affec u or u 2 via he conroller. Thi condiion will be aumed for he mo general cae conidered herein. A explained in Appendix A, hi condiion i ufficien o guaranee ha he opimal conrol policy γ i i linear, a very powerful fac. A quaniy of inere in hi paper i he delay marix, where each enry d ij i defined a he um of he delay along he fae direced pah from j o i. If no pah exi, d ij =. In Example, he delay marix i: d = 4 Thi paricular delay marix only conain,, and, bu for more inricae graph, he delay marix can conain any nonnegaive ineger, a long a i i he oal delay of he faed direced pah beween wo node. General Cae. In general, we conider any direced graph GV, E wih verice V = {,..., N}. Every uch edge i labeled wih d ij {, }. We refer o hi graph a he nework graph. We hen define d ij for all oher pair of verice according o he fae-pah definiion above. We conider yem defined by he ae-pace equaion: x i + = j V d ij Aij x j + B ij u j + w i for all i V 5 The iniial ae x can be fixed and known, uch a x =, or i may be a Gauian random variable. Noe ha he A ij denoe marice, which can be viewed a ubmarice of a larger A marix. If we ack he variou vecor and marice, we obain a more compac repreenaion of he ae-pace equaion: x + = Ax + Bu + w 6 where A and B have pariy paern deermined by he d ij. Namely, we can have A ij whenever d ij {, } and imilarly for B ij. Thi fac can be verified in Example by comparing and 4. We aume he noie erm are Gauian, IID for all ime, and independen beween ubyem. In oher word, we have Ewτ i w k T = whenever τ or i k, and Eww i i T = W i for all. Sae informaion propagae among ubyem a every imeep according o he graph opology and he delay along he link. Each conroller may ue any ae informaion ha ha had ufficien ime o reach i. More formally, u i i a funcion of he form u i = γ i x : d i,..., x N : d in. 7 Here if < d ij e.g. when d ij =, hen u i ha no acce o x j a all. A a echnical poin, we aume ha G conain no direced cycle wih zero lengh. If i did, we could collape he node of ha cycle ino a ingle node. The objecive i o find a conrol policy ha aifie he informaion conrain 7 and minimize he quadraic co funcion 2. According o our formulaion, he conroller may be any funcion of he pa informaion hiory, which grow wih he ize of he ime horizon T. However, we how in hi paper how o conruc an opimal conroller ha i linear and ha a finie memory which doe no depend on T. 3 Spaio-Temporal Decompoiion The oluion preened herein depend on a pecial decompoiion of he ae and inpu ino independen componen. In Subecion 3., we define he informaion graph, which decribe how diurbance injeced a each 2

3 {, 2, 3} {} {2, 3} {3} w w 2 w 3 Figure 2: The informaion graph for Example. The node of hi graph are he ube of node in he nework graph ee Fig. affeced by differen noie. For example, w 2 injeced a node 2 affec node {2, 3} immediaely, and affec {, 2, 3} afer one imeep. {, 2, 3} {} Node w 3 w 2 w w {2, 3} Node 2 w 2 3 w 2 2 w 2 w 2 {3} Node 3 w 3 3 w 3 2 w 3 w 3 Time Figure 3: Noie pariion diagram for Example refer o Fig. and 2. Each row repreen a differen node of he nework graph, and each column repreen a differen ime index, flowing from lef o righ. For example, w 2 2 will reach all hree node a ime, o i belong o he label e {, 2, 3}. node propagae hroughou he nework. Then, in Subecion 3.2, we how how he informaion graph can be ued o define a ueful pariion of he noie hiory. Finally, Subecion 3.3 explain he decompoiion of ae and inpu, which will prove crucial for our approach. 3. Informaion Graph The informaion graph a ueful alernaive way of repreening he informaion flow in he nework. Raher han hinking abou node and heir connecion, we rack he propagaion of he noie ignal w i injeced a he variou node of he nework graph. The informaion graph for Example i hown in Fig. 2. Formally, we define he informaion graph a follow. Le j k be he e of node reachable from node j wihin k ep: j k = {i V : d ij k}. The informaion graph, ĜU, F, i given by U = { j k : k, j V } F = { j k, j k+ : k, j V }. We will ofen wrie w i o indicae ha = i, hough we do no coun he w i among he node of he informaion graph, a a maer of convenion. The informaion graph can be conruced by racking each of he w i a hey propagae hrough he nework graph. For example, conider w 2, he noie injeced ino ubyem 2. In Fig., we ee ha he node {2, 3} are affeced immediaely. Afer one imeep, he noie ha reached all hree node. Thi lead o he pah w 2 {2, 3} {, 2, 3} in Fig. 2. When wo pah reach he ame ube, we merge hem ino a ingle node. For example, he pah aring from w alo reache {, 2, 3} afer one ep. Propoiion. Given an informaion graph ĜU, F, he following properie hold. i Every node in he informaion graph ha exacly one decendan. In oher word, for every r U, here i a unique U uch ha r. ii Every pah evenually hi a node wih a elf-loop. iii If d = max ij {d ij : d ij < }, hen an upper bound for he number of node in he informaion graph i given by U Nd. Example 2. Fig. 4 how he nework and informaion graph for a more complex four-node nework. We will refer o hi example hroughou he re of he paper. Noe ha he informaion graph may have everal conneced componen. Thi happen whenever he nework graph i no rongly conneced. For example, Fig. 2 ha wo conneced componen becaue here i no pah from node 3 o node Noie Pariion The noie hiory a ime i he e random variable coniing of all pa noie: H = {w i τ : i V, τ } where we have ued he convenion x = w. We now define a pecial pariion of H. Thi pariion i relaed o he informaion graph; here i one ube correponding o each U. We call hee ube label e, and hey are defined in he following lemma. Lemma 2. For every U, and for all, define he label e recurively uing L = {x i : w i } 8 L + = {w i : w i } L r. 9 r The label e {L } U pariion he noie hiory H : H = L and L r L = whenever r U 3

4 {, 2, 3, 4} {, 2, 3} 2 3 a Nework graph for Example 2 {2, 3, 4} {} {2} {3} {3, 4} w w 2 w 3 w 4 b Informaion graph for Example 2 {, 2, 3, 4} {, 2, 3} Node w 3 w 2 w w {2, 3, 4} Node 2 w 2 3 w 2 2 w 2 w 2 {3, 4} Node 3 w 3 3 w 3 2 w 3 w 3 Node 4 w 4 3 w 4 2 w 4 w 4 4 {} {2} {3} c Noie pariion diagram for Example 2 Time Figure 4: Nework graph a, informaion graph b, and noie pariion diagram c for Example 2. Proof. We proceed by inducion. A =, we have H = {x,..., x N }. For each i V, = i i he unique e uch ha x i L. Thu {L } U pariion H. Now uppoe ha {L } U pariion H for ome. By Propoiion, for all r U here exi a unique U uch ha r. Therefore each elemen w i k H i conained in exacly one e L +. For each i V, we have ha L i + i he unique label e conaining wi. Therefore {L +} U mu pariion H +. The pariion defined in Lemma 2 can be viualized uing a noie pariion diagram. Example i hown in Fig. 3 and Example 2 i hown in Fig. 4c. Thee diagram how he pariion explicily a ime by indicaing which par of he noie hiory belong o which label e. Each label e i agged wih i correponding node in he informaion graph. 3.3 Sae and Inpu Decompoiion We how in Appendix A ha our problem eup i parially need, which implie ha here exi an opimal conrol policy ha i linear. We alo how here ha x and u are linear funcion of he noie hiory H. Individual componen u i will no depend on he full noie hiory H, becaue cerain noie ignal will no have had ufficien ime o ravel o node i. Thi fac can be read direcly off he noie pariion diagram. To ee wheher a noie ymbol w k i L affec u i, we imply check wheher i. We ae hi reul a a lemma. Lemma 3. The inpu u i depend on he elemen of L if and only if i. The ae x i depend on he elemen of L if and only if i. The noie pariion decribed in Subecion 3.2 induce a decompoiion of he inpu and ae ino componen. We may wrie: u = U I V, ϕ and x = U I V, ζ where ϕ and ζ each depend on he elemen of L. Here I i a large ideniy marix wih block row and column conforming o he dimenion of x i or u i depending on he conex. The noaion I V, indicae he ubmarix in which he block-row correponding o i V and he block-column correponding o j have been eleced. For example, he ae in Example can be wrien a: x = I ζ {} + I I ζ {2,3} + I ζ {3} + I I I ζ {,2,3} The vecor ζ each have differen ize, which i due o Lemma 3. For example, x 2 i only a funcion of ζ {2,3} and ζ {,2,3}, ince hee are he only label e ha conain 2. We alo ue he upercrip noaion for oher marice. Taking a an example, if = {3} and r = {2, 3}, hen A r = [ ] A 32 A 33. The ae equaion ha define x 5 have a counerpar in he ζ coordinae. We ae he following lemma wihou proof. Lemma 4. The componen {ζ } U and {ϕ } U aify he recurive equaion: ζ = I,{i} x i w i ζ+ = r A r ζ r + B r ϕ r + I,{i} w i 2 w i Two imporan properie of he inpu and ae decompoiion follow from reul in hi ecion. Fir, 4

5 each inpu u i i a funcion of a paricular ube of he informaion hiory H, which follow direcly from Lemma 3. Corollary 5. The inpu u i depend on he elemen of i L. In paricular, u i ha he abiliy o compue any ae ζ for which i. Secondly, Lemma 2 implie ha he label e for a given ime index coni of muually independen noie. So our decompoiion provide independen coordinae: Corollary 6. Suppoe r, U, and r. Then: 4 Main Reul [ ] [ ] ζ r E ζ T ϕ r ϕ = Conider he general problem eup decribed in Secion 2, and generae he correponding informaion graph ĜU, F a explained in Secion 3.. For every node r U, define he marice X:T r recurively a follow: X r T = Q rr f X r = Q rr + A rt X +A r S rr + A rt X +B r R rr + B rt X +B r S rr + A rt X +B r T 3 where U i he unique node uch ha r. Finally, define he gain marice K:T r for every r U: K r = R rr + B rt X +B r S rr + A rt X +B r T Theorem 7. The opimal conrol policy i given by u = U 4 I V, ϕ where ϕ = K ζ 5 where he ae ζ evolve according o 2. The correponding opimal co i V x = race X {i},{i} E x i x i T i V w i T + race X {i},{i} W i 6 = Theorem 7 decribe an opimal conroller a a funcion of he diurbance w. The conroller can be ranformed ino an explici ae-feedback form a follow. For every U, define he ube w = {i : w i }. Thi e will be a ingleon {i} when he node i a roo node of he informaion graph, and will be empy oherwie. For example, conider Example 2 Fig. 4b. When = {3, 4} we have w = {4}, and when = {2, 3, 4} we have w =. Each U can be pariioned a = w w, where w = \ w. The ae of he conroller are given by η = I w, ζ for all U aifying w Theorem 8. Le Ār = A r +B r K r. The opimal aefeedback conroller i given by he ae-pace equaion η = η + = I w, Ā r I r,rw η r r + r u = U + U I w, Ā r I r,rw x rw I V, K I,w η I V, K I,w x w I rw,vw η v v r v r I w,vw η v v v 7 8 See Secion 6 for complee proof of Theorem 7 and 8. 5 Dicuion Noe ha for a elf-loop, r r, 3 define a claical Riccai difference equaion. Since he informaion graph can have a mo N elf-loop, a mo N Riccai equaion mu be olved. For oher edge, 3 propagae he oluion of he elf-loop Riccai equaion down he informaion graph oward he roo node. The oluion exend naurally o infinie horizon, provided ha a he elf-loop, r r, he marice aify claical condiion for abilizing oluion o he correponding algebraic Riccai equaion. Indeed, a T, X rr approach eady ae value. Since all gain are compued from he elf-loop Riccai equaion, he gain mu alo approach eady ae limi. The reul can alo be exended o ime-varying yem imply by replacing A, B, Q, R, S, and W wih A, B, Q, R, S, and W, repecively. The reul of [7] and [6] on conrol wih pariy conrain correpond o he pecial cae of hi work in which all edge have zero delay. In hi cae, he informaion graph coni of N elf-loop. Thu, N Riccai equaion mu be olved, bu hey are no propagaed. The reul on delayed yem of [2] correpond o he pecial cae of rongly conneced graph in which all edge have a delay of one imeep. Here, all pah in he informaion graph evenually lead o he elf-loop V V. Thu, a ingle Riccai equaion i olved and propagaed down he informaion graph. 5

6 6 Proof of Main Reul 6. Proof of Theorem 7 The proof ue dynamic programming and ake advanage of he decompoiion decribed in Secion 3. Begin by defining he co-o-go from ime a a funcion of he curren ae and fuure inpu: T J x, u :T = τ= [ xτ u τ ] T [ ] [ ] Q S xτ S T + x T R T Q f x T Noe ha J i a random variable becaue i doe no depend explicily on x +:T. Thee fuure ae are defined recurively uing 6, and hu depend on he noie erm w :T. Uing he decompoiion, he ae i divided according o x = U IV, ζ. We ue he abridged noaion ζ o denoe {ζ : U}, and we ue a imilar noaion o denoe he decompoiion of u ino ϕ. In hee new coordinae, he co-o-go become: J ζ, ϕ :T = T [ ] ζ T [ τ Q S ] [ ζ τ U τ= ϕ τ S T R ϕ τ u τ ] + ζt T Q f ζt where he fuure ae ζ +:T are defined recurively uing 2. Now define he value funcion, which i he minimum expeced co-o-go: V ζ = min ϕ :T E J ζ, ϕ :T Here, i i implied ha he minimizaion i aken over admiible policie, namely ϕ i a linear funcion of L a explained in Secion 3.3. Unlike he co-o-go funcion, he value funcion i deerminiic. The original minimum co conidered in he Inroducion 2 i imply V x. The value funcion aifie he Bellman equaion V ζ = min ϕ E U [ ζ ϕ ] T [ Q S ] [ ζ S T R ϕ ] + V + ζ + 9 Lemma 9. The Bellman equaion 9 i aified by a quadraic value funcion of he form: V ζ = E ζ T X ζ + c 2 U Proof. The proof ue inducion proceeding backward in ime. A = T, he co-o-go i imply he erminal co x T T Q f x T, and he value funcion i he expeced value of hi quaniy. Decompoe x T ino i ζ T coordinae. The ζ T coordinae are muually independen by Corollary 6, o we may wrie V T ζ T = E ζt T Q f ζt U Thu, 2 hold for = T, by eing X T = Q f for every U and c T =. Now uppoe 2 hold for +. Equaion 9 become: V ζ = min E [ ] ζ T [ Q S ] [ ] ζ ϕ ϕ S T R ϕ U ζ+ T X+ζ + + c + + E U 2 Subiue he recurion for ζ + defined in 2, and ake advanage of he muual independence of he ζ and ϕ erm proved in Corollary 6. Finally, we obain: V ζ = min E [ ] ζ r T [ ] ϕ ϕ r Γ r ζ r + ϕ r + c + r U + race X + {i},{i} W i i V w i where Γ r + i given by: S rrt R rr 22 [ Q Γ r rr S rr ] + = + [ A r B r] T [ X + A r B r] and i he unique node in he informaion graph uch ha r, ee Propoiion. Equaion 22 can be decompoed ino independen quadraic opimizaion problem, one for each ϕ r : [ ] ϕ r ζ r T [ ] = arg min E ϕ r ϕ r Γ r ζ r + ϕ r for all r U The opimal co i again a quadraic funcion, which verifie our inducive hypohei. The opimal inpu found by olving he quadraic opimizaion problem in Lemma 9 are given by ϕ = K ζ. Thi policy i admiible by Corollary 5. Subiuing he opimal policy, and comparing boh ide of he equaion, we obain he deired recurion relaion a well a he opimal co Proof of Theorem 8 Conider a node U wih w i. Thu he e w i nonempy, and w = {i}. From, x i = v i I w,v ζ v = v I w,v ζ v 23 The econd equaliy follow becaue v i implie ha v. Indeed, for any oher j, we mu have d ji =. Thu, if v i, hen v j a well. Rearranging 23 give I w, ζ = x i I w,v ζ v v v 6

7 Since w and w pariion, we may wrie ζ a A Parial Needne ζ = I,w I w, ζ + I,w I w, ζ. = I,w I w, ζ + I,w = I,w η + I,w x i I w,v ζ v v v x i I w,vw η v v v 24 where we ubiued he definiion η = I w, ζ in he final ep and ued he fac ha i v w, becaue v and we can have boh w i v and w i. We can now find ae-pace equaion in erm of he new η coordinae. The inpu u i found by ubiuing 24 ino 5, and he η dynamic are found by ubiuing 24 ino 2 and muliplying on he lef by I w,. Noe ha he conroller now depend on he ae x raher han he diurbance w. Reference [] Y-C. Ho and K-C. Chu. Team deciion heory and informaion rucure in opimal conrol problem Par I. IEEE Tranacion on Auomaic Conrol, 7:5 22, 972. [2] Andrew Lamperki and John C. Doyle. Dynamic programming oluion for decenralized ae-feedback LQG problem wih communicaion delay. In American Conrol Conference, 22. [3] Xin Qi, M.V. Salapaka, P.G. Voulgari, and M. Khammah. Srucured opimal and robu conrol wih muliple crieria: a convex oluion. IEEE Tranacion on Auomaic Conrol, 49:623 64, 24. [4] Ander Ranzer. Linear quadraic eam heory reviied. In American Conrol Conference, page , 26. [5] M. Rokowiz and S. Lall. A characerizaion of convex problem in decenralized conrol. IEEE Tranacion on Auomaic Conrol, 52: , 26. [6] Parikhi Shah and Pablo A. Parrilo. H 2-opimal decenralized conrol over poe: A ae pace oluion for ae-feedback. In IEEE Conference on Deciion and Conrol, page , 2. [7] J. Swigar. Opimal Conroller Synhei for Decenralized Syem. PhD hei, Sanford Univeriy, 2. In hi ecion, we dicu parial needne, a concep fir inroduced by []. Define he e: I i = {x j τ : j V, τ d ij } Comparing wih 7, I i i preciely he informaion ha u i ha acce o when making i deciion. Definiion. A dynamical yem 6 wih informaion rucure 7 i parially need if for every admiible e of policie γ, i whenever u j τ affec u i, hen Iτ j I. i The main reul regarding parial needne i given in he following lemma. Lemma []. Given a parially need rucure, he opimal conrol for each member exi, i unique, and i linear. In order o apply hi reul, we mu fir how ha he problem conidered in hi paper are of he correc ype. Lemma 2. The informaion rucure decribed in 7 i parially need. Proof. Suppoe u j τ affec u i in he mo direc manner poible. Namely, u j τ direcly affec x µ τ+, which affec a equence of oher ae unil i reache x k σ, and x k σ I. i Furher uppoe ha x l ρ Iτ j. u j τ affec x k σ direcly = d kj σ τ 25 x k σ I i = d ik σ 26 x l ρ I j τ = d jl τ ρ 27 Adding ogeher and uing he riangle inequaliy, we obain d il ρ. Thu, x l ρ I i. I follow ha I j τ I i, a required. If u j τ affec u i via a more complicaed pah, apply he above argumen o each conecuive pair of inpu along he pah o obain he chain of incluion I j τ I i. Wih parial needne eablihed, Lemma implie ha we have a unique linear opimal conroller. In paricular, he opimal γ i are linear funcion of I i. Looking back a 5, we conclude ha x and u mu be linear funcion of he noie hiory w :, ince w canno affec x or u inananeouly. In general, individual componen uch a u i will no be funcion of he full noie hiory w :. Thi opic i dicued in deail in Secion 3. 7